Commit Graph

3 Commits

Author SHA1 Message Date
emrgnt-cmplxty bc53dc9c4f Dev minor (#1436)
* fix-actions (#1426)

* up

* modify

* add to github path

* Contextual Chunk Enrichment (#1433)

* add semantic chunking

* working

* precommit

* pre-commits

* Entity Deduplication (#1431)

* Modify graphrag prompt (#1421)

* Fix async JSON parsing (#1408)

* Fix async JSON parsing

* Remove score completion from js

* clean up js

* lockfile

* Feature/build custom logger (#1409)

* building a custom logger for r2r

* fix log

* maintain bkwd compat

* Feature/add kg description prompt (#1411)

* add kg desc prompt

* add kg desc prompt

* add kg desc prompt

* fix prompt name

* separate test run freq (#1412)

* separate test run freq

* task_id check fix

* add ingestion docs

* updatet

* add

* rm old prompts

* rm old prompots

* rm old prompts

* rm old prompts

* Prod fixes + enhancements (#1407)

* change default settings back to fp32

* add logging and cache triples

* up

* up

* pre-commit and cleanups

* making community summary prompt async

* up

* up

* revert prompt changes

* up

* up

* modify default

* bump test timeout due to stricter concurrency limits

* bump sleep

* rm ubuntu from windows/mac workflows

* up

* add tests

* Feature/include vectors option document chunks (#1419)

* Fix async JSON parsing (#1408)

* Fix async JSON parsing

* Remove score completion from js

* clean up js

* lockfile

* Feature/build custom logger (#1409)

* building a custom logger for r2r

* fix log

* maintain bkwd compat

* Feature/add kg description prompt (#1411)

* add kg desc prompt

* add kg desc prompt

* add kg desc prompt

* fix prompt name

* separate test run freq

* task_id check fix

* add ingestion docs

* updatet

* add

* rm old prompts

* rm old prompots

* rm old prompts

* rm old prompts

* add option to include vectors in document chunks

* checkin

* update vector

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>

* Allow env var to set the default R2R deployment for the dashboard (#1417)

* modify community_summary_prompt function and corresponding prompt

* add tests

* up

* Feature/various documentation tweaks (#1422)

* Fix async JSON parsing (#1408)

* Fix async JSON parsing

* Remove score completion from js

* clean up js

* lockfile

* Feature/build custom logger (#1409)

* building a custom logger for r2r

* fix log

* maintain bkwd compat

* Feature/add kg description prompt (#1411)

* add kg desc prompt

* add kg desc prompt

* add kg desc prompt

* fix prompt name

* separate test run freq

* task_id check fix

* add ingestion docs

* updatet

* add

* rm old prompts

* rm old prompots

* rm old prompts

* rm old prompts

* add option to include vectors in document chunks

* checkin

* update vector

* some various documentation tweaks

* some various documentation tweaks

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>

* Graphrag tests (#1418)

* Fix async JSON parsing (#1408)

* Fix async JSON parsing

* Remove score completion from js

* clean up js

* lockfile

* Feature/build custom logger (#1409)

* building a custom logger for r2r

* fix log

* maintain bkwd compat

* Feature/add kg description prompt (#1411)

* add kg desc prompt

* add kg desc prompt

* add kg desc prompt

* fix prompt name

* separate test run freq (#1412)

* separate test run freq

* task_id check fix

* add ingestion docs

* updatet

* add

* rm old prompts

* rm old prompots

* rm old prompts

* rm old prompts

* Prod fixes + enhancements (#1407)

* change default settings back to fp32

* add logging and cache triples

* up

* up

* pre-commit and cleanups

* making community summary prompt async

* up

* up

* revert prompt changes

* up

* up

* modify default

* bump test timeout due to stricter concurrency limits

* bump sleep

* rm ubuntu from windows/mac workflows

* up

* add tests

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* Modify graphrag tests timeouts (#1416)

* Fix async JSON parsing (#1408)

* Fix async JSON parsing

* Remove score completion from js

* clean up js

* lockfile

* Feature/build custom logger (#1409)

* building a custom logger for r2r

* fix log

* maintain bkwd compat

* Feature/add kg description prompt (#1411)

* add kg desc prompt

* add kg desc prompt

* add kg desc prompt

* fix prompt name

* separate test run freq (#1412)

* separate test run freq

* task_id check fix

* add ingestion docs

* updatet

* add

* rm old prompts

* rm old prompots

* rm old prompts

* rm old prompts

* Prod fixes + enhancements (#1407)

* change default settings back to fp32

* add logging and cache triples

* up

* up

* pre-commit and cleanups

* making community summary prompt async

* up

* up

* revert prompt changes

* up

* up

* modify default

* bump test timeout due to stricter concurrency limits

* bump sleep

* rm ubuntu from windows/mac workflows

* modify timeouts

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* feat: Make prompt provider methods asynchronous (comments below) (#1415)

* Fix async JSON parsing (#1408)

* Fix async JSON parsing

* Remove score completion from js

* clean up js

* lockfile

* Feature/build custom logger (#1409)

* building a custom logger for r2r

* fix log

* maintain bkwd compat

* Feature/add kg description prompt (#1411)

* add kg desc prompt

* add kg desc prompt

* add kg desc prompt

* fix prompt name

* separate test run freq (#1412)

* separate test run freq

* task_id check fix

* add ingestion docs

* updatet

* add

* rm old prompts

* rm old prompots

* rm old prompts

* rm old prompts

* Prod fixes + enhancements (#1407)

* change default settings back to fp32

* add logging and cache triples

* up

* up

* pre-commit and cleanups

* making community summary prompt async

* up

* up

* revert prompt changes

* up

* up

* modify default

* bump test timeout due to stricter concurrency limits

* bump sleep

* rm ubuntu from windows/mac workflows

* feat: Make prompt provider methods asynchronous

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* up

* up

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* Add routes, service level methods around conversations (#1420)

* Add routes, service level methods around conversations

* Slight refactor to match project conventions, add JS methods

* Updated JS methods

* JS docs

* Add python

* Update JS user tests

* add deduplication pipe, workflow, api, sdk, cli

* add summary workflow

* bug fixes

* pre-commit

* working

* search working

* adding dedup test files

* modify the update query

* precommit

* more testing

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* Refactor Python SDK for Intellisense, Thread Safety (#1430)

* Refactor Python SDK

* Fix CLI after SDK changes

* Add convo to agent

* Update conversation error handling, JS

* Remove unused, bad import

* Feature/cleanup refactor vector collection (#1432)

* Feature/include vectors option document chunks (#1419)

* Fix async JSON parsing (#1408)

* Fix async JSON parsing

* Remove score completion from js

* clean up js

* lockfile

* Feature/build custom logger (#1409)

* building a custom logger for r2r

* fix log

* maintain bkwd compat

* Feature/add kg description prompt (#1411)

* add kg desc prompt

* add kg desc prompt

* add kg desc prompt

* fix prompt name

* separate test run freq

* task_id check fix

* add ingestion docs

* updatet

* add

* rm old prompts

* rm old prompots

* rm old prompts

* rm old prompts

* add option to include vectors in document chunks

* checkin

* update vector

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>

* Allow env var to set the default R2R deployment for the dashboard (#1417)

* Feature/various documentation tweaks (#1422)

* Fix async JSON parsing (#1408)

* Fix async JSON parsing

* Remove score completion from js

* clean up js

* lockfile

* Feature/build custom logger (#1409)

* building a custom logger for r2r

* fix log

* maintain bkwd compat

* Feature/add kg description prompt (#1411)

* add kg desc prompt

* add kg desc prompt

* add kg desc prompt

* fix prompt name

* separate test run freq

* task_id check fix

* add ingestion docs

* updatet

* add

* rm old prompts

* rm old prompots

* rm old prompts

* rm old prompts

* add option to include vectors in document chunks

* checkin

* update vector

* some various documentation tweaks

* some various documentation tweaks

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>

* Graphrag tests (#1418)

* Fix async JSON parsing (#1408)

* Fix async JSON parsing

* Remove score completion from js

* clean up js

* lockfile

* Feature/build custom logger (#1409)

* building a custom logger for r2r

* fix log

* maintain bkwd compat

* Feature/add kg description prompt (#1411)

* add kg desc prompt

* add kg desc prompt

* add kg desc prompt

* fix prompt name

* separate test run freq (#1412)

* separate test run freq

* task_id check fix

* add ingestion docs

* updatet

* add

* rm old prompts

* rm old prompots

* rm old prompts

* rm old prompts

* Prod fixes + enhancements (#1407)

* change default settings back to fp32

* add logging and cache triples

* up

* up

* pre-commit and cleanups

* making community summary prompt async

* up

* up

* revert prompt changes

* up

* up

* modify default

* bump test timeout due to stricter concurrency limits

* bump sleep

* rm ubuntu from windows/mac workflows

* up

* add tests

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* Modify graphrag tests timeouts (#1416)

* Fix async JSON parsing (#1408)

* Fix async JSON parsing

* Remove score completion from js

* clean up js

* lockfile

* Feature/build custom logger (#1409)

* building a custom logger for r2r

* fix log

* maintain bkwd compat

* Feature/add kg description prompt (#1411)

* add kg desc prompt

* add kg desc prompt

* add kg desc prompt

* fix prompt name

* separate test run freq (#1412)

* separate test run freq

* task_id check fix

* add ingestion docs

* updatet

* add

* rm old prompts

* rm old prompots

* rm old prompts

* rm old prompts

* Prod fixes + enhancements (#1407)

* change default settings back to fp32

* add logging and cache triples

* up

* up

* pre-commit and cleanups

* making community summary prompt async

* up

* up

* revert prompt changes

* up

* up

* modify default

* bump test timeout due to stricter concurrency limits

* bump sleep

* rm ubuntu from windows/mac workflows

* modify timeouts

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* feat: Make prompt provider methods asynchronous (comments below) (#1415)

* Fix async JSON parsing (#1408)

* Fix async JSON parsing

* Remove score completion from js

* clean up js

* lockfile

* Feature/build custom logger (#1409)

* building a custom logger for r2r

* fix log

* maintain bkwd compat

* Feature/add kg description prompt (#1411)

* add kg desc prompt

* add kg desc prompt

* add kg desc prompt

* fix prompt name

* separate test run freq (#1412)

* separate test run freq

* task_id check fix

* add ingestion docs

* updatet

* add

* rm old prompts

* rm old prompots

* rm old prompts

* rm old prompts

* Prod fixes + enhancements (#1407)

* change default settings back to fp32

* add logging and cache triples

* up

* up

* pre-commit and cleanups

* making community summary prompt async

* up

* up

* revert prompt changes

* up

* up

* modify default

* bump test timeout due to stricter concurrency limits

* bump sleep

* rm ubuntu from windows/mac workflows

* feat: Make prompt provider methods asynchronous

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* bump pyproject version

* first commit

* towards slimmer vector implementation logic

* up

* iterate

* up

* checkin

* up

* work doc chunks

* working vector search

* working full text search

* remove asyncpg

* passing vector tests

* up

* merge

* rm pytest

* up

* up

* fix delete

* up

* up

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>

* Add tests + Cleanup (#1437)

* up

* add tests

* test rename to sdk

* up

* fix tests

* typo

* modify chunk enrichment prompt (#1438)

* modify prompt

* up

* Fix type error on port argument of CLI (#1439)

* finish (#1440)

* finish

* up

* fix

* fix

* up

* fix

* final cleanups

* fix naming convention

* fix schema error

* increase timeout

* split graphrag actions

* fix collection exists error

* up (#1442)

* Add error message (#1443)

* up

* sdk fix

* locally testing build

* up docs (#1445)

* checkin work (#1444)

* checkin work

* finish index functionality extension

* fix concurrency

* add alembic (#1446)

* Prompt Tuning (#1447)

* Check in

* Fix after merging dev-minor in

* Ensure to not cause int overflow with hatchet (#1454)

* Bump JS (#1456)

* Ensure to not cause int overflow with hatchet

* bump js

* improve migration implementation (#1452)

* improve migration implementation

* refine migrations to include kg

* add alembic cli

* extend documentation

* extend docs and all that

* Revert change of default behaviour of entities endpoint, docs, tests (#1455)

* change def behavior of entities + delete endpoint

* pre-commit

* add deduplication tests

* Delete graph (#1450)

* up docs

* up

* up

* rename to raw_chunks

* up

* add tests

* up

* up

* change default

* change cli

* separate out deduplication tests

* change run type in the test

* up

* up

* add test concurrency

* up

* rm concurrency groups

* rm dedup tests

* remove json

* tests

* up

* fix lock

* Update postgres.py

* Feature/merge dev minor main (#1457)

* add run without orchestration (#1448)

* add run without orchestration

* bump versions

* bump versions

* bump versions

* fix

* up

* add end points

* add run without orchestration (#1448) (#1458)

* add run without orchestration

* bump versions

* bump versions

* bump versions

* fix

* up

* sync migration changes

* Ensure that we await ingest files in ingest_files method (#1460)

* Nolan/await update files (#1461)

* Ensure that we await ingest files in ingest_files method

* Await update files as well

* Docs changes (#1462)

* up

* up

* up

* up

* fix failed find and replace (#1463)

* fix failed find and replace

* fix

* Fix JS Client for Ingest Chunks (#1464)

* Ensure that we await ingest files in ingest_files method

* Await update files as well

* Fix js client

---------

Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>
Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
2024-10-22 21:59:10 -07:00
emrgnt-cmplxty a640fb20b2 Release 3.2.11 (#1365)
• Added Knowledge Graph (KG) tests (#1351)
  - CLI tests
  - SDK tests
  - Workflow ordering changes

• Added collection integration tests (#1352)
  - Fixed SDK test port
  - Fixed delete collection return check

• Fixed document info serialization (#1353)

• Added GraphRAG documentation (#1362)

• Implemented concurrent index creation (#1363)
  - Allow -1 for paginated entries

• Updated webdev-template for current Next.js and r2r-js SDK (#1218)

• Extended integration tests (#1361)
  - Cleaned up and simplified CLI
  - Added new workflows and actions
  - Implemented matrix test flow
  - Docker setup and configuration

• Rebased to r2r vars (#1364)
  - Updated Windows workflow
  - Revived full workflow and tests

• Fixed typos (#1366)

• Added ingestion concurrency limit (#1367)

• Fixed Ollama Tool Calling (#1372)

• Cleaned up Docker Compose (#1368)

• Fixed bug in deletion, improved validation error handling (#1374)

• Added vector index creation endpoint (#1373)

• Modified KG Endpoints and updated API spec (#1369)

• Implemented new Docker setup (#1371)
  - Updated actions and scripts

• Cleaned up ingest chunks, added to JS SDK (#1375)

• Various updates and fixes (#1376, #1378, #1379)
  - Increased entity limit
  - Changed Aristotle back to v2
  - Added test_ingest_sample_file_2_sdk

• Added GraphRAG documentation and refined code (#1382)
  - Added Python SDK documentation

• Added R2R_ prefix to env vars (#1383)
  - Cleaned up docker compose
  - Bumped Python and JS package versions
  - Updated form-data

---------

Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>
Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: FutureProofTechOps <operations@theobald.nz>
Co-authored-by: Simeon <simeon@theobald.nz>
Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com>
2024-10-11 15:51:49 -07:00
emrgnt-cmplxty 4beb82616a Add KG tests (#1351) (#1360)
* Add KG tests (#1351)

* cli tests

* add sdk tests

* typo fix

* change workflow ordering

* add collection integration tests (#1352)

* bump pkg

* remove workflows

* fix sdk test port

* fix delete collection return check

* Fix document info serialization (#1353)

* Update integration-test-workflow-debian.yml

* pre-commit

* slightly modify

* up

* up

* smaller file

* up

* typo, change order

* up

* up

* change order

---------

Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <owen@algofi.org>
Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>

* add graphrag docs (#1362)

* add documentation

* up

* Update js/sdk/src/models.tsx

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* pre-commit

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Concurrent index creation, allow -1 for paginated entries (#1363)

* update webdev-template for current next.js and r2r-js sdk (#1218)

Co-authored-by: Simeon <simeon@theobald.nz>

* Feature/extend integration tests rebased (#1361)

* cleanups

* add back overzealous edits

* extend workflows

* fix full setup

* simplify cli

* add ymls

* rename to light

* try again

* start light

* add cli tests

* fix

* fix

* testing..

* trying complete matrix testflow

* cleanup matrix logic

* cleanup matrix logic

* cleanup matrix logic

* cleanup matrix logic

* cleanup matrix logic

* cleanup matrix logic

* cleanup matrix logic

* up

* up

* up

* All actions

* rename to runner

* rename to runner

* rename to runner

* rename to runner

* rename to runner

* rename to runner

* rename to runner

* rename to runner

* rename to runner

* try offic pgvec formula

* sudo make

* sudo make

* push and pray

* push and pray

* add new actions

* add new actions

* docker push & pray

* inspect manifests during launch

* inspect manifests during launch

* inspect manifests during launch

* inspect manifests during launch

* setup docker

* setup docker

* fix default

* fix default

* Feature/rebase to r2r vars (#1364)

* cleanups

* add back overzealous edits

* extend workflows

* fix full setup

* simplify cli

* add ymls

* rename to light

* try again

* start light

* add cli tests

* fix

* fix

* testing..

* trying complete matrix testflow

* cleanup matrix logic

* cleanup matrix logic

* cleanup matrix logic

* cleanup matrix logic

* cleanup matrix logic

* cleanup matrix logic

* cleanup matrix logic

* up

* up

* up

* All actions

* rename to runner

* rename to runner

* rename to runner

* rename to runner

* rename to runner

* rename to runner

* rename to runner

* rename to runner

* rename to runner

* try offic pgvec formula

* sudo make

* sudo make

* push and pray

* push and pray

* add new actions

* add new actions

* docker push & pray

* inspect manifests during launch

* inspect manifests during launch

* inspect manifests during launch

* inspect manifests during launch

* setup docker

* setup docker

* fix default

* fix default

* make changes

* update the windows workflow

* update the windows workflow

* remove extra workflows for now

* bump pkg

---------

Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>
Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: FutureProofTechOps <operations@theobald.nz>
Co-authored-by: Simeon <simeon@theobald.nz>
2024-10-08 18:45:07 -07:00