940b81fc3fe82eb8f2f3064fd3a7f26f394a83d5
23 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nolan Tremelling
|
940b81fc3f | Add statement_cache_size parameter to allow connections to supabase (#1676) | ||
|
emrgnt-cmplxty
|
d693f690a6 |
Feature/v3 api (#1656)
* improve ci/cd runtime * update prompt tests * improve ci/cd runtime (#1535) * improve ci/cd runtime * update prompt tests * Support Python ^3.10 (#1534) * add azure * up * up * spec out v3 api * checkin document router * adding chunk abstr * add list chunks * add chunk search * up * add users routes * up * checkin progress * add collections annotations * add indices * add user * checkin work * up * complete conversations CRUD * fix type errors * add graph router * add graphs * Update JS (#1563) * Feature/add graph to v3 (#1565) * complete simple tests, cleanup routers * up * Harmonize Pagination across endpoints (#1564) * Pagination * Add fixmes * Fix nested deletion filter bug (#1567) * Remove Mintlify docs (#1569) * Nolan/list collection (#1568) * Check in * More * Fix nested transactions issue in sqlite logger * Fix update collection return type * JS V3 (#1571) * Sync collections JS * More documents * Clean up messy code * list not List * Users first pass * User tests and fixmes * More * typo * More prompts * Pre-commit improvements * Remove prints * Cleanups on conversations * Branches response * Chunks * More work on the return types * Jest config * Fix branch creation time * Fix lock * Nolan/v3 tests (#1578) * Add deprecated command back * Add warning * Fix GraphRAG tests (#1579) * More cleanup (#1580) * More cleanup * More * Fix test * More cleanups * More cleanups * More * Merge main * Python SDK V3 (#1585) * Python SDK V3 * Fix * First pass (#1586) * More V3 (#1587) * Validation errors * Update js test * more * Fix sync methods on v2 sdk, add check for download files (#1588) * More CLI (#1589) * Print logs on failing tests (#1590) * Print logs on failing tests * MOre * cleanup * Again * Again * More JS testing (#1591) * More JS testing * Cleanup * More refactors for tests (#1592) * System Routes (#1594) * Fix type errors, pass collection id (#1595) * Hotfix: dict * V3 graph implmentations (#1593) * complete simple tests, cleanup routers * up * up * checkin * up * up * response models * checkin * up * checkin * up * up * up * up * up * up * v2 * up * up * up * up --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Allow passing of collection id at document ingestion (#1596) * KG Response sync (#1597) * fix * Fix Prompt Override (#1599) * Fix Prompt Override * print * Caching * Fix * Updated Graph Models, Drop SID (#1598) * New Graph Models * Fix * minor tweaks * fix summary model (#1604) * incr progress * Add /users/me (#1605) * Add /users/me * oops * Resolve Merge Conflicts (#1607) * Fix conflicts * Clean up * Nolan/conflicts (#1608) * expose reset data to admin (#1602) * up (#1603) * up * up * wtf github is a piece of garbage --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> * wrapup walkthrough * Add delete user method, sync JS to camel case (#1609) * V3 graph testing (#1606) * up * up * up * graph crud * up * community endpts * up * up * up * up * up * up * up * up * add back routers * up * pre-commit * Fix Broken V2 Graphs, Better Response Models (#1612) * Increase test coverage * Fix v2 graphs, better response models * Remaining types * Add types to Python SDK * Typo * update tests * revert test change * up * Add types to package export (#1613) * Graph refactor (#1611) * up * up * add back routers * up * pre-commit * update tests * revert test change * up * simplify * up * add the add/remove endpoints * up * include routers back * Create branch update (#1617) * Graph refactor (#1616) * up * up * add back routers * up * pre-commit * update tests * revert test change * up * simplify * up * add the add/remove endpoints * up * include routers back * List collections (#1619) * up * up * up * up * Graph refactor (#1620) * up * up * add back routers * up * pre-commit * update tests * revert test change * up * simplify * up * add the add/remove endpoints * up * include routers back * up * up * up * up * Nolan/update graph (#1621) * List collections * Update Graph JS SDK * up * up * cleanup * Graph refactor (#1622) * up * up * add back routers * up * pre-commit * update tests * revert test change * up * simplify * up * add the add/remove endpoints * up * include routers back * up * up * up * up * up * up * cleanup * up * up * up * remove unnecessary functions * up * up * complete document embedding workflow * working get command on graph * checkin progress * up * add entity and relationship deletions * no verif * up * up * up * up (#1636) * up * sync graph * up * up * fix relationship distance calc. * fix issue with faulty collection filter (#1637) * Patch/alternative fix logics 2 (#1638) * fix issue with faulty collection filter * further refinements, like fixing limits * up * fix logic around include metadata and scores * fix double collection assignment * up * fix communities * working clusters * up * add collection extraction * add collection extraction * up * prep for merge * Patch/alternative up with nolan (#1643) * SDK First pass * Add feature tracking * Typo * Check in * Rebase * Add Graph tests * Fix Agent empty message bug * Check in JS routes * More tests, examples * Sync python * Expose Entity/Relationship Params in Routes (#1640) * Expose Entity/Relationship Params * Descriptions * Modify create entities * Create relationships * set parent_id * Update entitiy * Update Relationships * Check in * Ellipsis fixes * More cleanup * Start CRUD on communities * Communities DB * Explicit working path * Once again * Fail fast false * Testing around community creation * Delete community test * Update community tests * Clean up type errors, cleaner code * More cleanup * More * remove chunk_entity * Delete bad, unused methods * More * fixup crud * rm pull --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> * Feature/fix graph permissions (#1645) * update docs / collections * up * Feature/fix auth checks (#1647) * update docs / collections * up * fix super user and more * up * up (#1648) * Feature/rm v2 api (#1649) * SDK First pass * Add feature tracking * Typo * Check in * Rebase * Add Graph tests * Fix Agent empty message bug * Check in JS routes * More tests, examples * Sync python * Expose Entity/Relationship Params in Routes (#1640) * Expose Entity/Relationship Params * Descriptions * Modify create entities * Create relationships * set parent_id * Update entitiy * Update Relationships * Check in * Ellipsis fixes * More cleanup * Start CRUD on communities * Communities DB * Explicit working path * Once again * Fail fast false * Testing around community creation * Delete community test * Update community tests * Clean up type errors, cleaner code * More cleanup * More * remove chunk_entity * Delete bad, unused methods * More * remove v2 api * rm kg router * cleanups * fixup delete by filter * fixup delete by filter * fixes * up * up --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> * Improved Data Structures (#1650) * Check in * Most tests fixed * fix tables * Once more * Move to a single community table * Don't modify existing migration script--keep them atomic * Migration * Migration, more clean up * All but deletion working * Up * Feature/tweaks for prod (#1651) * tweaks for prod * up * final tweaks * Nolan/deletion (#1652) * Check in * Most tests fixed * fix tables * Once more * Move to a single community table * Don't modify existing migration script--keep them atomic * Migration * Migration, more clean up * All but deletion working * Up * Fix deletion * Working migration (#1654) * Feature/production tweaks (#1653) * tweaks for prod * up * final tweaks * prod tweaks * fixed --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> * sort --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> |
||
|
emrgnt-cmplxty
|
2a3f06a541 | expose reset data to admin (#1602) | ||
|
emrgnt-cmplxty
|
2378f58242 |
Dev minor (#1574)
* add add-hoc rerank implementation to embedding, add async rerank (#1572) * add HF defaults * Feature/add document summary to ingestion (#1573) * adds document summary to ingestion pipeline * cleanup impl * new hybrid document search * implement hybrid document search * Feature/add document summary to ingestion (#1575) * adds document summary to ingestion pipeline * cleanup impl * new hybrid document search * implement hybrid document search * add migration script * make the summary change non-breaking (#1576) * make the summary change non-breaking * rollbk * up * Feature/tweak downgrade logic (#1577) * tweak downgrade * fix js sdk * fix js sdk * fix upgrade logic * up |
||
|
Shreyas Pimpalgaonkar
|
3f73329adb |
Entity deduplication by description (#1551)
* checkin * checkin * up |
||
|
Shreyas Pimpalgaonkar
|
00aec7716c |
Enrichment caching (#1550)
* up * up * checkin * checkin * up * up * up * up |
||
|
emrgnt-cmplxty
|
9b67ddea2a |
Feature/add documents search (#1549)
* up * rm extra printout * bump release * add ot js * return metadata |
||
|
Shreyas Pimpalgaonkar
|
6fe18bd48c |
Updating Enrichment status (#1544)
* minor fixes * up * adding bin support and make it default (#1508) * Feature/tweak actions (#1507) * up * tweak actions * adding bin sup and making it default * tested and vetted * up (#1510) * up * set verification to default false * up * cleanup (#1512) * cleanup * cleanup prompt mgmt * up * cleanup printout * cleanup new parser logic, set vlm as default for all providers * allow user to re-override * cleanup; pre-commit * rm launch json * checkin work * up * modify endpoint * up --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> |
||
|
emrgnt-cmplxty
|
23372cd1f3 |
Feature/add verification code rebased (#1547)
* improve ci/cd runtime (#1535) * improve ci/cd runtime * update prompt tests * Support Python ^3.10 (#1534) * Patch/add azure to workflows (#1537) * improve ci/cd runtime * update prompt tests * add azure * up * up * revert latest ingestion changes (#1539) * revert latest ingestion changes * revert latest ingestion changes * Update pyproject.toml * Update local_llm.toml (#1542) * Update local_llm.toml * Update pyproject.toml * Update full_azure.toml * expose user verification code --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> |
||
|
emrgnt-cmplxty
|
ddfe870f42 |
dev-minor (#1509)
* adding bin support and make it default (#1508) * Feature/tweak actions (#1507) * up * tweak actions * adding bin sup and making it default * tested and vetted * up (#1510) * up * set verification to default false * cleanup (#1512) * cleanup * cleanup prompt mgmt * up * cleanup printout * cleanup new parser logic, set vlm as default for all providers * allow user to re-override * modify exp backoff implementation (#1513) * Feature/tweak actions (#1507) * up * tweak actions * modify exp backoff impl --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> * Patch/touchups (#1515) * cleanup * cleanup prompt mgmt * up * cleanup printout * cleanup new parser logic, set vlm as default for all providers * allow user to re-override * add touchups * add extra parsers (#1516) * add extra parsers (#1518) * minor fixes (#1514) * Feature/add back ollama provider (#1522) * add extra parsers * add back ollama * rvert auth workflow * Feature/add prompt tests and cleanup (#1523) * add extra parsers * add prompt tests, cleanup * add prompt tests, cleanup * merge * set mock console as default * set mock console as default * fix config * Update community model (#1524) * Feature/tweak actions (#1507) * up * tweak actions * Sync JS SDK, Harmonize Python SDK KG Methods (#1511) * Feature/move logging (#1492) * move logging provider out * move logging provider to own directory, remove singleton * cleanup * fix refactoring tweak (#1496) * Fix JSON serialization and Prompt ID Bugs for Prompts (#1491) * Bug in get prompts * Add tests * Prevent verbose logging on standup * Remove kg as required key in config, await get_all_prompts * Remove reference to fragment id * comment out ingestion * complete logging port (#1499) * Feature/dev rebased (#1500) * Feature/move logging (#1493) * move logging provider out * move logging provider to own directory, remove singleton * cleanup * Update js package (#1498) * fix refactoring tweak (#1496) * Fix JSON serialization and Prompt ID Bugs for Prompts (#1491) * Bug in get prompts * Add tests * Prevent verbose logging on standup * Remove kg as required key in config, await get_all_prompts * Remove reference to fragment id * comment out ingestion * complete logging port (#1499) --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * Fix handling for R2R exceptions (#1501) * fix doc test (#1502) * Harmonize python SDK KG methods for optional params, add missing JS methods --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Clean up pagination and offset around KG (#1519) * Move to R2R light for integration testing (#1521) * Update community model --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Patch/fix import bleed (#1526) * Feature/tweak actions (#1507) * up * tweak actions * Sync JS SDK, Harmonize Python SDK KG Methods (#1511) * Feature/move logging (#1492) * move logging provider out * move logging provider to own directory, remove singleton * cleanup * fix refactoring tweak (#1496) * Fix JSON serialization and Prompt ID Bugs for Prompts (#1491) * Bug in get prompts * Add tests * Prevent verbose logging on standup * Remove kg as required key in config, await get_all_prompts * Remove reference to fragment id * comment out ingestion * complete logging port (#1499) * Feature/dev rebased (#1500) * Feature/move logging (#1493) * move logging provider out * move logging provider to own directory, remove singleton * cleanup * Update js package (#1498) * fix refactoring tweak (#1496) * Fix JSON serialization and Prompt ID Bugs for Prompts (#1491) * Bug in get prompts * Add tests * Prevent verbose logging on standup * Remove kg as required key in config, await get_all_prompts * Remove reference to fragment id * comment out ingestion * complete logging port (#1499) --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * Fix handling for R2R exceptions (#1501) * fix doc test (#1502) * Harmonize python SDK KG methods for optional params, add missing JS methods --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Clean up pagination and offset around KG (#1519) * Move to R2R light for integration testing (#1521) --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * Patch/fix import bleed (#1527) * Feature/tweak actions (#1507) * up * tweak actions * Sync JS SDK, Harmonize Python SDK KG Methods (#1511) * Feature/move logging (#1492) * move logging provider out * move logging provider to own directory, remove singleton * cleanup * fix refactoring tweak (#1496) * Fix JSON serialization and Prompt ID Bugs for Prompts (#1491) * Bug in get prompts * Add tests * Prevent verbose logging on standup * Remove kg as required key in config, await get_all_prompts * Remove reference to fragment id * comment out ingestion * complete logging port (#1499) * Feature/dev rebased (#1500) * Feature/move logging (#1493) * move logging provider out * move logging provider to own directory, remove singleton * cleanup * Update js package (#1498) * fix refactoring tweak (#1496) * Fix JSON serialization and Prompt ID Bugs for Prompts (#1491) * Bug in get prompts * Add tests * Prevent verbose logging on standup * Remove kg as required key in config, await get_all_prompts * Remove reference to fragment id * comment out ingestion * complete logging port (#1499) --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * Fix handling for R2R exceptions (#1501) * fix doc test (#1502) * Harmonize python SDK KG methods for optional params, add missing JS methods --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Clean up pagination and offset around KG (#1519) * Move to R2R light for integration testing (#1521) * fix ollama pdf parser --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * email auth false and js bump * fix actions (#1528) * Feature/add poppler check and fallback (#1529) * fix actions * fallback * Patch/import shutil (#1530) * fix actions * fallback * import shutil * Feature/include basic pdf parsing everywhere (#1531) * fix actions * fallback * import shutil * add basic pdf as extra parser in all configs * Remove non existent user login? * attempt login * Change password back * add explicit setting, trigger rebuild --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> |
||
|
Nolan Tremelling
|
35f047c7ae | Clean up pagination and offset around KG (#1519) | ||
|
emrgnt-cmplxty
|
52b7b1afaf |
Dev rebased (#1503)
* fix refactoring tweak (#1496) * Fix JSON serialization and Prompt ID Bugs for Prompts (#1491) * Bug in get prompts * Add tests * Prevent verbose logging on standup * Remove kg as required key in config, await get_all_prompts * Remove reference to fragment id * comment out ingestion * complete logging port (#1499) * Fix handling for R2R exceptions (#1501) * fix doc test (#1502) * up --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> |
||
|
emrgnt-cmplxty
|
ff38e1c33b | up (#1488) | ||
|
emrgnt-cmplxty
|
1ab36930fc |
Feature/dev minor merge main again (#1487)
* Feature/port kg to handler rebased (#1478) * up * up * add back database * merge * merge * removing prompt provider (#1479) * removing prompt provider * migrate prompt provider * update lock file * rm file provider * refactor imports (#1481) * making types more explicit across codebase (#1482) * cleanup prompt handler * cleanup prompt handler * add kg create tables * Feature/dev minor rebased (#1485) * Update pyproject.toml (#1471) * add alembic migrations to pypi release (#1472) * add alembic migrations to pypi release * modify path * run on main (#1474) * minor kg-search bug (#1476) * minor kg-search bug * fix * Update pyproject.toml * Move to psycopg binary (#1477) --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * Chunk CRUD (#1483) * check in * Finish ingest chunks, delete, and testing * Docs and JS tests * Ensure that we retrieve KG Status for documents in collection (#1484) * Update pyproject.toml (#1471) * add alembic migrations to pypi release (#1472) * add alembic migrations to pypi release * modify path * run on main (#1474) * minor kg-search bug (#1476) * minor kg-search bug * fix * Update pyproject.toml * Move to psycopg binary (#1477) * Ensure that we retrieve KG Status for documents in collection * github sucks --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> |
||
|
emrgnt-cmplxty
|
bc53dc9c4f |
Dev minor (#1436)
* fix-actions (#1426) * up * modify * add to github path * Contextual Chunk Enrichment (#1433) * add semantic chunking * working * precommit * pre-commits * Entity Deduplication (#1431) * Modify graphrag prompt (#1421) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests * Feature/include vectors option document chunks (#1419) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * Allow env var to set the default R2R deployment for the dashboard (#1417) * modify community_summary_prompt function and corresponding prompt * add tests * up * Feature/various documentation tweaks (#1422) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector * some various documentation tweaks * some various documentation tweaks --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * Graphrag tests (#1418) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Modify graphrag tests timeouts (#1416) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * modify timeouts --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * feat: Make prompt provider methods asynchronous (comments below) (#1415) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * feat: Make prompt provider methods asynchronous --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * up * up --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Add routes, service level methods around conversations (#1420) * Add routes, service level methods around conversations * Slight refactor to match project conventions, add JS methods * Updated JS methods * JS docs * Add python * Update JS user tests * add deduplication pipe, workflow, api, sdk, cli * add summary workflow * bug fixes * pre-commit * working * search working * adding dedup test files * modify the update query * precommit * more testing --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Refactor Python SDK for Intellisense, Thread Safety (#1430) * Refactor Python SDK * Fix CLI after SDK changes * Add convo to agent * Update conversation error handling, JS * Remove unused, bad import * Feature/cleanup refactor vector collection (#1432) * Feature/include vectors option document chunks (#1419) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * Allow env var to set the default R2R deployment for the dashboard (#1417) * Feature/various documentation tweaks (#1422) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * add option to include vectors in document chunks * checkin * update vector * some various documentation tweaks * some various documentation tweaks --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * Graphrag tests (#1418) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * up * add tests --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Modify graphrag tests timeouts (#1416) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * modify timeouts --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * feat: Make prompt provider methods asynchronous (comments below) (#1415) * Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows * feat: Make prompt provider methods asynchronous --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * bump pyproject version * first commit * towards slimmer vector implementation logic * up * iterate * up * checkin * up * work doc chunks * working vector search * working full text search * remove asyncpg * passing vector tests * up * merge * rm pytest * up * up * fix delete * up * up --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Add tests + Cleanup (#1437) * up * add tests * test rename to sdk * up * fix tests * typo * modify chunk enrichment prompt (#1438) * modify prompt * up * Fix type error on port argument of CLI (#1439) * finish (#1440) * finish * up * fix * fix * up * fix * final cleanups * fix naming convention * fix schema error * increase timeout * split graphrag actions * fix collection exists error * up (#1442) * Add error message (#1443) * up * sdk fix * locally testing build * up docs (#1445) * checkin work (#1444) * checkin work * finish index functionality extension * fix concurrency * add alembic (#1446) * Prompt Tuning (#1447) * Check in * Fix after merging dev-minor in * Ensure to not cause int overflow with hatchet (#1454) * Bump JS (#1456) * Ensure to not cause int overflow with hatchet * bump js * improve migration implementation (#1452) * improve migration implementation * refine migrations to include kg * add alembic cli * extend documentation * extend docs and all that * Revert change of default behaviour of entities endpoint, docs, tests (#1455) * change def behavior of entities + delete endpoint * pre-commit * add deduplication tests * Delete graph (#1450) * up docs * up * up * rename to raw_chunks * up * add tests * up * up * change default * change cli * separate out deduplication tests * change run type in the test * up * up * add test concurrency * up * rm concurrency groups * rm dedup tests * remove json * tests * up * fix lock * Update postgres.py * Feature/merge dev minor main (#1457) * add run without orchestration (#1448) * add run without orchestration * bump versions * bump versions * bump versions * fix * up * add end points * add run without orchestration (#1448) (#1458) * add run without orchestration * bump versions * bump versions * bump versions * fix * up * sync migration changes * Ensure that we await ingest files in ingest_files method (#1460) * Nolan/await update files (#1461) * Ensure that we await ingest files in ingest_files method * Await update files as well * Docs changes (#1462) * up * up * up * up * fix failed find and replace (#1463) * fix failed find and replace * fix * Fix JS Client for Ingest Chunks (#1464) * Ensure that we await ingest files in ingest_files method * Await update files as well * Fix js client --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> |
||
|
emrgnt-cmplxty
|
21686b35a4 |
Dev minor (#1410)
* Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> |
||
|
emrgnt-cmplxty
|
eed926235d |
D3v (#1398) (#1399)
* D3v (#1398) * Feature/add zerox parser (#1396) * Add KG tests (#1351) * cli tests * add sdk tests * typo fix * change workflow ordering * add collection integration tests (#1352) * bump pkg * remove workflows * fix sdk test port * fix delete collection return check * Fix document info serialization (#1353) * Update integration-test-workflow-debian.yml * pre-commit * slightly modify * up * up * smaller file * up * typo, change order * up * up * change order --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * add graphrag docs (#1362) * add documentation * up * Update js/sdk/src/models.tsx Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * pre-commit --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Concurrent index creation, allow -1 for paginated entries (#1363) * update webdev-template for current next.js and r2r-js sdk (#1218) Co-authored-by: Simeon <simeon@theobald.nz> * Feature/extend integration tests rebased (#1361) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * Feature/rebase to r2r vars (#1364) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * make changes * update the windows workflow * update the windows workflow * remove extra workflows for now * bump pkg * push and pray * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive tests * revive tests * revive tests * revive tests * update tests * fix typos (#1366) * update tests * up * up * up * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * Add ingestion concurrency limit (#1367) * up * up * up --------- Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * tweaks and fixes * Fix Ollama Tool Calling (#1372) * Update graphrag.mdx * Fix Ollama tool calling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Clean up Docker Compose (#1368) * Fix hatchet, dockerfile * Update compose * point to correct docker image * Fix bug in deletion, better validation error handling (#1374) * Update graphrag.mdx * Fix bug in deletion, better validation error handling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * vec index creation endpoint (#1373) * Update graphrag.mdx * upload files * create vector index endpoint * add to fastapi background task * pre-commit * move logging * add api spec, support for all vecs * pre-commit * add workflow * Modify KG Endpoints and update API spec (#1369) * Update graphrag.mdx * modify API endpoints and update documentation * Update ingestion_router.py * try different docker setup (#1371) * try different docker setup * action * add login * add full * update action * cleanup upload script * cleanup upload script * tweak action * tweak action * tweak action * tweak action * tweak action * tweak action * Nolan/ingest chunks js (#1375) * Update graphrag.mdx * Clean up ingest chunks, add to JS SDK * Update JS docs --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * up (#1376) * Bump JS package (#1378) * add conversation * checkin progress * checkin progress * Fix Create Graph (#1379) * up * up * modify assertion * up * up * increase entity limit * changing aristotle back to v2 * pre-commit * typos * add test_ingest_sample_file_2_sdk * Update server.py * checkin progress * up * update * Graphrag docs (#1382) * add docs and refine code * add python SDK documentation * up * update * checkin * up * cleanup * working sync logging * test conversation history * fix runner tests, rename `CHUNKS` to `chunks` * adding zerox parser --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * Nolan/update hatchet (#1397) * Move Hatchet to latest * Update js package-lock --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> * Add hatchet logging (#1391) * Add KG tests (#1351) * cli tests * add sdk tests * typo fix * change workflow ordering * add collection integration tests (#1352) * bump pkg * remove workflows * fix sdk test port * fix delete collection return check * Fix document info serialization (#1353) * Update integration-test-workflow-debian.yml * pre-commit * slightly modify * up * up * smaller file * up * typo, change order * up * up * change order --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * add graphrag docs (#1362) * add documentation * up * Update js/sdk/src/models.tsx Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * pre-commit --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Concurrent index creation, allow -1 for paginated entries (#1363) * update webdev-template for current next.js and r2r-js sdk (#1218) Co-authored-by: Simeon <simeon@theobald.nz> * Feature/extend integration tests rebased (#1361) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * Feature/rebase to r2r vars (#1364) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * make changes * update the windows workflow * update the windows workflow * remove extra workflows for now * bump pkg * push and pray * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive tests * revive tests * revive tests * revive tests * update tests * fix typos (#1366) * update tests * up * up * up * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * Add ingestion concurrency limit (#1367) * up * up * up --------- Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * tweaks and fixes * Fix Ollama Tool Calling (#1372) * Update graphrag.mdx * Fix Ollama tool calling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Clean up Docker Compose (#1368) * Fix hatchet, dockerfile * Update compose * point to correct docker image * Fix bug in deletion, better validation error handling (#1374) * Update graphrag.mdx * Fix bug in deletion, better validation error handling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * vec index creation endpoint (#1373) * Update graphrag.mdx * upload files * create vector index endpoint * add to fastapi background task * pre-commit * move logging * add api spec, support for all vecs * pre-commit * add workflow * Modify KG Endpoints and update API spec (#1369) * Update graphrag.mdx * modify API endpoints and update documentation * Update ingestion_router.py * try different docker setup (#1371) * try different docker setup * action * add login * add full * update action * cleanup upload script * cleanup upload script * tweak action * tweak action * tweak action * tweak action * tweak action * tweak action * Nolan/ingest chunks js (#1375) * Update graphrag.mdx * Clean up ingest chunks, add to JS SDK * Update JS docs --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * up (#1376) * Bump JS package (#1378) * Fix Create Graph (#1379) * up * up * modify assertion * up * up * increase entity limit * changing aristotle back to v2 * pre-commit * typos * add test_ingest_sample_file_2_sdk * Update server.py * add docs and refine code * add python SDK documentation * up * add logs * clean * rm vq * rm conflicts * pre-commit * up * add logging * update logs * up * up * Update kg_service.py --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * feat: Add delete_node_via_document_id method to KGProvider (#1387) Co-authored-by: shou.hsu <shou.hsu@utonia.com> --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> Co-authored-by: Shou-Hsu <148736054+Shou-Hsu@users.noreply.github.com> Co-authored-by: shou.hsu <shou.hsu@utonia.com> * Vector + Index quantization (#1400) * Add KG tests (#1351) * cli tests * add sdk tests * typo fix * change workflow ordering * add collection integration tests (#1352) * bump pkg * remove workflows * fix sdk test port * fix delete collection return check * Fix document info serialization (#1353) * Update integration-test-workflow-debian.yml * pre-commit * slightly modify * up * up * smaller file * up * typo, change order * up * up * change order --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * add graphrag docs (#1362) * add documentation * up * Update js/sdk/src/models.tsx Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * pre-commit --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Concurrent index creation, allow -1 for paginated entries (#1363) * update webdev-template for current next.js and r2r-js sdk (#1218) Co-authored-by: Simeon <simeon@theobald.nz> * Feature/extend integration tests rebased (#1361) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * Feature/rebase to r2r vars (#1364) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * make changes * update the windows workflow * update the windows workflow * remove extra workflows for now * bump pkg * push and pray * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive tests * revive tests * revive tests * revive tests * update tests * fix typos (#1366) * update tests * up * up * up * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * Add ingestion concurrency limit (#1367) * up * up * up --------- Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * tweaks and fixes * Fix Ollama Tool Calling (#1372) * Update graphrag.mdx * Fix Ollama tool calling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Clean up Docker Compose (#1368) * Fix hatchet, dockerfile * Update compose * point to correct docker image * Fix bug in deletion, better validation error handling (#1374) * Update graphrag.mdx * Fix bug in deletion, better validation error handling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * vec index creation endpoint (#1373) * Update graphrag.mdx * upload files * create vector index endpoint * add to fastapi background task * pre-commit * move logging * add api spec, support for all vecs * pre-commit * add workflow * Modify KG Endpoints and update API spec (#1369) * Update graphrag.mdx * modify API endpoints and update documentation * Update ingestion_router.py * try different docker setup (#1371) * try different docker setup * action * add login * add full * update action * cleanup upload script * cleanup upload script * tweak action * tweak action * tweak action * tweak action * tweak action * tweak action * Nolan/ingest chunks js (#1375) * Update graphrag.mdx * Clean up ingest chunks, add to JS SDK * Update JS docs --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * up (#1376) * Bump JS package (#1378) * Fix Create Graph (#1379) * up * up * modify assertion * up * up * increase entity limit * changing aristotle back to v2 * pre-commit * typos * add test_ingest_sample_file_2_sdk * Update server.py * add docs and refine code * add python SDK documentation * up * add logs * merge changes * mc * more * add index + vector quantization * pre-commits * chnage default back to FP32 * kg vector and index test * rm duplicate import * Update r2r.toml --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * Feature/add zerox to unstructured rebased (#1406) * Feature/add zerox parser (#1396) * Add KG tests (#1351) * cli tests * add sdk tests * typo fix * change workflow ordering * add collection integration tests (#1352) * bump pkg * remove workflows * fix sdk test port * fix delete collection return check * Fix document info serialization (#1353) * Update integration-test-workflow-debian.yml * pre-commit * slightly modify * up * up * smaller file * up * typo, change order * up * up * change order --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * add graphrag docs (#1362) * add documentation * up * Update js/sdk/src/models.tsx Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * pre-commit --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Concurrent index creation, allow -1 for paginated entries (#1363) * update webdev-template for current next.js and r2r-js sdk (#1218) Co-authored-by: Simeon <simeon@theobald.nz> * Feature/extend integration tests rebased (#1361) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * Feature/rebase to r2r vars (#1364) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * make changes * update the windows workflow * update the windows workflow * remove extra workflows for now * bump pkg * push and pray * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive tests * revive tests * revive tests * revive tests * update tests * fix typos (#1366) * update tests * up * up * up * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * Add ingestion concurrency limit (#1367) * up * up * up --------- Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * tweaks and fixes * Fix Ollama Tool Calling (#1372) * Update graphrag.mdx * Fix Ollama tool calling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Clean up Docker Compose (#1368) * Fix hatchet, dockerfile * Update compose * point to correct docker image * Fix bug in deletion, better validation error handling (#1374) * Update graphrag.mdx * Fix bug in deletion, better validation error handling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * vec index creation endpoint (#1373) * Update graphrag.mdx * upload files * create vector index endpoint * add to fastapi background task * pre-commit * move logging * add api spec, support for all vecs * pre-commit * add workflow * Modify KG Endpoints and update API spec (#1369) * Update graphrag.mdx * modify API endpoints and update documentation * Update ingestion_router.py * try different docker setup (#1371) * try different docker setup * action * add login * add full * update action * cleanup upload script * cleanup upload script * tweak action * tweak action * tweak action * tweak action * tweak action * tweak action * Nolan/ingest chunks js (#1375) * Update graphrag.mdx * Clean up ingest chunks, add to JS SDK * Update JS docs --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * up (#1376) * Bump JS package (#1378) * add conversation * checkin progress * checkin progress * Fix Create Graph (#1379) * up * up * modify assertion * up * up * increase entity limit * changing aristotle back to v2 * pre-commit * typos * add test_ingest_sample_file_2_sdk * Update server.py * checkin progress * up * update * Graphrag docs (#1382) * add docs and refine code * add python SDK documentation * up * update * checkin * up * cleanup * working sync logging * test conversation history * fix runner tests, rename `CHUNKS` to `chunks` * adding zerox parser --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * Nolan/update hatchet (#1397) * Move Hatchet to latest * Update js package-lock --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> * Add hatchet logging (#1391) * Add KG tests (#1351) * cli tests * add sdk tests * typo fix * change workflow ordering * add collection integration tests (#1352) * bump pkg * remove workflows * fix sdk test port * fix delete collection return check * Fix document info serialization (#1353) * Update integration-test-workflow-debian.yml * pre-commit * slightly modify * up * up * smaller file * up * typo, change order * up * up * change order --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * add graphrag docs (#1362) * add documentation * up * Update js/sdk/src/models.tsx Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * pre-commit --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Concurrent index creation, allow -1 for paginated entries (#1363) * update webdev-template for current next.js and r2r-js sdk (#1218) Co-authored-by: Simeon <simeon@theobald.nz> * Feature/extend integration tests rebased (#1361) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * Feature/rebase to r2r vars (#1364) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * make changes * update the windows workflow * update the windows workflow * remove extra workflows for now * bump pkg * push and pray * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive tests * revive tests * revive tests * revive tests * update tests * fix typos (#1366) * update tests * up * up * up * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * Add ingestion concurrency limit (#1367) * up * up * up --------- Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * tweaks and fixes * Fix Ollama Tool Calling (#1372) * Update graphrag.mdx * Fix Ollama tool calling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Clean up Docker Compose (#1368) * Fix hatchet, dockerfile * Update compose * point to correct docker image * Fix bug in deletion, better validation error handling (#1374) * Update graphrag.mdx * Fix bug in deletion, better validation error handling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * vec index creation endpoint (#1373) * Update graphrag.mdx * upload files * create vector index endpoint * add to fastapi background task * pre-commit * move logging * add api spec, support for all vecs * pre-commit * add workflow * Modify KG Endpoints and update API spec (#1369) * Update graphrag.mdx * modify API endpoints and update documentation * Update ingestion_router.py * try different docker setup (#1371) * try different docker setup * action * add login * add full * update action * cleanup upload script * cleanup upload script * tweak action * tweak action * tweak action * tweak action * tweak action * tweak action * Nolan/ingest chunks js (#1375) * Update graphrag.mdx * Clean up ingest chunks, add to JS SDK * Update JS docs --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * up (#1376) * Bump JS package (#1378) * Fix Create Graph (#1379) * up * up * modify assertion * up * up * increase entity limit * changing aristotle back to v2 * pre-commit * typos * add test_ingest_sample_file_2_sdk * Update server.py * add docs and refine code * add python SDK documentation * up * add logs * clean * rm vq * rm conflicts * pre-commit * up * add logging * update logs * up * up * Update kg_service.py --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * feat: Add delete_node_via_document_id method to KGProvider (#1387) Co-authored-by: shou.hsu <shou.hsu@utonia.com> * Finishing `zerox` integration * Finishing `zerox` integration * fix zerox, add cli docs * mention runtime config --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> Co-authored-by: Shou-Hsu <148736054+Shou-Hsu@users.noreply.github.com> Co-authored-by: shou.hsu <shou.hsu@utonia.com> * Improve ingestion errors, status bugs (#1405) * Move Hatchet to latest * Update js package-lock * Better handling for errors on ingestion * Filter out already processed chunks in create graph (#1404) * add simple filter * up * modify query * add zerox test * fix conversation history * attempt fix tests * adding sleep before retrieving the list of entities * bump version --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> Co-authored-by: Shou-Hsu <148736054+Shou-Hsu@users.noreply.github.com> Co-authored-by: shou.hsu <shou.hsu@utonia.com> |
||
|
emrgnt-cmplxty
|
a640fb20b2 |
Release 3.2.11 (#1365)
• Added Knowledge Graph (KG) tests (#1351) - CLI tests - SDK tests - Workflow ordering changes • Added collection integration tests (#1352) - Fixed SDK test port - Fixed delete collection return check • Fixed document info serialization (#1353) • Added GraphRAG documentation (#1362) • Implemented concurrent index creation (#1363) - Allow -1 for paginated entries • Updated webdev-template for current Next.js and r2r-js SDK (#1218) • Extended integration tests (#1361) - Cleaned up and simplified CLI - Added new workflows and actions - Implemented matrix test flow - Docker setup and configuration • Rebased to r2r vars (#1364) - Updated Windows workflow - Revived full workflow and tests • Fixed typos (#1366) • Added ingestion concurrency limit (#1367) • Fixed Ollama Tool Calling (#1372) • Cleaned up Docker Compose (#1368) • Fixed bug in deletion, improved validation error handling (#1374) • Added vector index creation endpoint (#1373) • Modified KG Endpoints and updated API spec (#1369) • Implemented new Docker setup (#1371) - Updated actions and scripts • Cleaned up ingest chunks, added to JS SDK (#1375) • Various updates and fixes (#1376, #1378, #1379) - Increased entity limit - Changed Aristotle back to v2 - Added test_ingest_sample_file_2_sdk • Added GraphRAG documentation and refined code (#1382) - Added Python SDK documentation • Added R2R_ prefix to env vars (#1383) - Cleaned up docker compose - Bumped Python and JS package versions - Updated form-data --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> |
||
|
emrgnt-cmplxty
|
55d7d873b0 |
Dev minor (#1321)
* no-verify (#1314) * no-verify * update readme * Kg testing (#1300) * Feature/encapsulate orchestration (#1265) * fully encapsulate orchestration * fully encapsulate orchestration * complete encapsulation * revert import cmt * making default r2r lighter (#1268) * making default r2r lighter * fix bug in ingest files * checkin * workingupdate * complete simple orch * update docs * up (#1273) * up * up * merge (#1276) * Postgres configuration settings (#1277) * Improvements on Auth in JS, CLI (#1267) * CLI Telemetry (#1266) * check in * working * redundant * JS auth improvements (#1263) * Check in JS auth improvements * Update login with toke * Fix to allow disabling telemetry * fix lock * Try to avoid merge conflicts * Clean up collection bugs * remove comments * Add Postgres configuration settings * Image * bad github conflict * merge (#1278) * port KG to postgres (#1272) * create + cluster * local search * up * clean * format * basics * add collection_id and paginate * rename * change api * up * kg_creation_status * up * up * up * Feature/cleanup docker (#1279) * merge * up * rm neo4j refs and cleanup docker cmds * fixup * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * Patch/cleanup kg migration (#1281) * cleanup kg migration * up * Kg testing (#1280) * up * up * up * up * slay neo4j --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * add back poetry lock * Default Collections (#1282) * Default collections * Naughty naughty need to follow the SRP * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * Testing (#1284) * CICD * actions * poetry * poetry * Add env vars * name * increase timeout * add user to collection * change postgres project name * Kg testing (#1283) * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * change postgres project name --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Feature/fix logic bugs (#1285) * fixing minor logic bugs in dev branch * fixing minor logic bugs in dev branch * merge * up * Application docs * add image (#1287) * Add version to CLI telemetry (#1288) * add image * Add version to cli telemetry * up * KG hatchet orchestration (#1286) * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * change postgres project name * up * up --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Feature/update documentation rebased (#1289) * up * merge * rebase * fix ingestion issues (#1291) * fix ingestion issues * fix lock file * fix embedding * Fix SDK KG Serialization (#1292) * add image * serialization * cleanup cli (#1294) * CLI serialization (#1295) * add image * Fix more serialization around kg * Nolan/schemacreation (#1296) * add image * Fix more serialization around kg * add quotes to prevent reserved keywords from failing * Prevent errors if config name is reserved name in postgres (#1297) * Prevent reserved words (#1298) * default collection ID * up * Move default collection id method to utils (#1299) * up * Allow json fallback (#1301) * hotfix: import * Fix description error (#1302) * up * push * up (#1303) * up * up * up * up * minor tweaks * up * mypy * add back missing file * up * up * up * fix id * up --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * add way to access entities and relationships (#1313) * Feature/encapsulate orchestration (#1265) * fully encapsulate orchestration * fully encapsulate orchestration * complete encapsulation * revert import cmt * making default r2r lighter (#1268) * making default r2r lighter * fix bug in ingest files * checkin * workingupdate * complete simple orch * update docs * up (#1273) * up * up * merge (#1276) * Postgres configuration settings (#1277) * Improvements on Auth in JS, CLI (#1267) * CLI Telemetry (#1266) * check in * working * redundant * JS auth improvements (#1263) * Check in JS auth improvements * Update login with toke * Fix to allow disabling telemetry * fix lock * Try to avoid merge conflicts * Clean up collection bugs * remove comments * Add Postgres configuration settings * Image * bad github conflict * merge (#1278) * port KG to postgres (#1272) * create + cluster * local search * up * clean * format * basics * add collection_id and paginate * rename * change api * up * kg_creation_status * up * up * up * Feature/cleanup docker (#1279) * merge * up * rm neo4j refs and cleanup docker cmds * fixup * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * Patch/cleanup kg migration (#1281) * cleanup kg migration * up * Kg testing (#1280) * up * up * up * up * slay neo4j --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * add back poetry lock * Default Collections (#1282) * Default collections * Naughty naughty need to follow the SRP * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * Testing (#1284) * CICD * actions * poetry * poetry * Add env vars * name * increase timeout * add user to collection * change postgres project name * Kg testing (#1283) * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * change postgres project name --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Feature/fix logic bugs (#1285) * fixing minor logic bugs in dev branch * fixing minor logic bugs in dev branch * merge * up * Application docs * add image (#1287) * Add version to CLI telemetry (#1288) * add image * Add version to cli telemetry * up * KG hatchet orchestration (#1286) * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * change postgres project name * up * up --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Feature/update documentation rebased (#1289) * up * merge * rebase * fix ingestion issues (#1291) * fix ingestion issues * fix lock file * fix embedding * Fix SDK KG Serialization (#1292) * add image * serialization * cleanup cli (#1294) * CLI serialization (#1295) * add image * Fix more serialization around kg * Nolan/schemacreation (#1296) * add image * Fix more serialization around kg * add quotes to prevent reserved keywords from failing * Prevent errors if config name is reserved name in postgres (#1297) * Prevent reserved words (#1298) * default collection ID * up * Move default collection id method to utils (#1299) * up * Allow json fallback (#1301) * hotfix: import * Fix description error (#1302) * up * push * up (#1303) * up * up * up * up * minor tweaks * up * mypy * add back missing file * up * add way to access entities and relationships * up * up * fix id * up * refine end pts --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Add collection (#1317) * add filters (in progress) * todo comment * minor addition * modify command * Permission Fixes around Collection Management (#1316) * Allow super users to update others to super user status * Fix auth on collections endpoints * Better error message * filters (#1318) * Cost estimate (#1319) * up * slightly modify * up * minor fix * docs --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> |
||
|
emrgnt-cmplxty
|
3721fcb7ad |
dev (#1269)
* Feature/encapsulate orchestration (#1265) * fully encapsulate orchestration * fully encapsulate orchestration * complete encapsulation * revert import cmt * making default r2r lighter (#1268) * making default r2r lighter * fix bug in ingest files * checkin * workingupdate * complete simple orch * update docs * up (#1273) * up * up * merge (#1276) * Postgres configuration settings (#1277) * Improvements on Auth in JS, CLI (#1267) * CLI Telemetry (#1266) * check in * working * redundant * JS auth improvements (#1263) * Check in JS auth improvements * Update login with toke * Fix to allow disabling telemetry * fix lock * Try to avoid merge conflicts * Clean up collection bugs * remove comments * Add Postgres configuration settings * Image * bad github conflict * merge (#1278) * port KG to postgres (#1272) * create + cluster * local search * up * clean * format * basics * add collection_id and paginate * rename * change api * up * kg_creation_status * up * up * up * Feature/cleanup docker (#1279) * merge * up * rm neo4j refs and cleanup docker cmds * fixup * Patch/cleanup kg migration (#1281) * cleanup kg migration * up * Kg testing (#1280) * up * up * up * up * slay neo4j --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * add back poetry lock * Default Collections (#1282) * Default collections * Naughty naughty need to follow the SRP * Testing (#1284) * CICD * actions * poetry * poetry * Add env vars * name * increase timeout * add user to collection * Kg testing (#1283) * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * change postgres project name --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Feature/fix logic bugs (#1285) * fixing minor logic bugs in dev branch * fixing minor logic bugs in dev branch * merge * Application docs * add image (#1287) * Add version to CLI telemetry (#1288) * add image * Add version to cli telemetry * KG hatchet orchestration (#1286) * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * change postgres project name * up * up --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Feature/update documentation rebased (#1289) * up * merge * rebase * fix ingestion issues (#1291) * fix ingestion issues * fix lock file * fix embedding * Fix SDK KG Serialization (#1292) * add image * serialization * cleanup cli (#1294) * CLI serialization (#1295) * add image * Fix more serialization around kg * Nolan/schemacreation (#1296) * add image * Fix more serialization around kg * add quotes to prevent reserved keywords from failing * Prevent errors if config name is reserved name in postgres (#1297) * Prevent reserved words (#1298) * Move default collection id method to utils (#1299) * Allow json fallback (#1301) * hotfix: import * Fix description error (#1302) * up (#1303) * rename to `full` (#1304) * rename to `full` * add html parser * Remove postgres vecs variables (#1306) * Feature/rename ingest files (#1307) * rename to `full` * add html parser * Vec Removal (#1308) * Remove postgres vecs variables * up * change kg settings parsing (#1309) * offset + limit (#1305) * offset + limit * fix order * update query * change entity offset * leiden seed --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> |
||
|
emrgnt-cmplxty
|
3e0938b5d5 |
Feature/typing with pagination (#1219) (#1230)
* Feature/typing with pagination (#1219) * fix imports * bump pkg * Improve pagination in responses * Add js * Fix collections * fixing typing issues * checkin work * more * fix all typing issues * fix type errors * flake8 and type * flake8 and type * up (#1221) * up * add mypy * add mypy to pre-commit * Feature/share abstractions (#1223) * no verify * fix shared * finish share * Fix: Docker compose bind mount issue (#1222) * bind mount updated * Update compose.yaml * Update compose.yaml * yaml formatted * Update compose.yaml * missing files (#1225) * Yc (#1224) * up * frontend * update search types * modify default queries * up * fix formatting * fix docs * up * up * modify dockerfile * revert middleware change * Improve UI (#1227) * up * minor fix * Improve messaging (#1228) * up * up * rm print statements --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * fixes breaks (#1226) * fixes breaks * fixes breaks * rm prints * add back regression tests * fix llm content (#1229) --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: Rajan Goswami <148398808+rajangoswamiTWS@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * bump version --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: Rajan Goswami <148398808+rajangoswamiTWS@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> |
||
|
emrgnt-cmplxty
|
51d258262e |
Release 3.1 (#1011)
* Feature/orchestration v0 (#1006)
* Feature/remove extra r2r abstraction (#996)
* moving kg construction to enrich-graph (#984)
* checkin
* up
* done
* formatting
* Feature/update ingestion issues (#985)
* udpate ingestion issues
* keep unbounded limit support, but default to bounded
* fix
* fmt
* removes an unnecessary abstraction
* sync changes
---------
Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>
* first commit
* move towards orchestration
* tweaks
* check in working ingestion
* move
* kg enrichment
* update future, postgres compose
* hatchetize ingestion pipeline
* ready for prime time
* finish
---------
Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>
* Feature/add update files workflow (#1010)
* add update files workflow
* rm ingestion pipeline
* Feature/add enrichment flow (#1013)
* add update files workflow
* rm ingestion pipeline
* v0 restructure orch
* Feature/merged enrichment flow (#1016)
* add update files workflow
* rm ingestion pipeline
* v0 restructure orch
* kg orchestration
* finish kg orchestration
* update service
* merge
* cleanups
* Rm graspologic (#1034)
* moving kg construction to enrich-graph (#984)
* checkin
* up
* done
* formatting
* Feature/update ingestion issues (#985)
* udpate ingestion issues
* keep unbounded limit support, but default to bounded
* fix
* fmt
* Add support for CharacterTextSplitter (#986)
* Add support for CharacterTextSplitter
Allows R2R client to override the text splitter. Example:
```python
ingestion_response = client.ingest_files(
file_paths=[file_path],
metadatas=metadata,
# optionally override chunking settings at runtime
chunking_settings={
"provider": "r2r",
"method": "character",
"extra_fields": {
"separator": "---"
},
}
)
```
* fixup! Add support for CharacterTextSplitter
* fixup! fixup! Add support for CharacterTextSplitter
* Patch/ollama base cli (#992)
* Dev (#990)
* moving kg construction to enrich-graph (#984)
* checkin
* up
* done
* formatting
* Feature/update ingestion issues (#985)
* udpate ingestion issues
* keep unbounded limit support, but default to bounded
* fix
* fmt
* Add support for CharacterTextSplitter (#986)
* Add support for CharacterTextSplitter
Allows R2R client to override the text splitter. Example:
```python
ingestion_response = client.ingest_files(
file_paths=[file_path],
metadatas=metadata,
# optionally override chunking settings at runtime
chunking_settings={
"provider": "r2r",
"method": "character",
"extra_fields": {
"separator": "---"
},
}
)
```
* fixup! Add support for CharacterTextSplitter
* fixup! fixup! Add support for CharacterTextSplitter
---------
Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>
Co-authored-by: Manuel R. Ciosici <manuelrciosici@gmail.com>
* fix ollama cli
---------
Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>
Co-authored-by: Manuel R. Ciosici <manuelrciosici@gmail.com>
* Ingestion refactor (#991)
* fix test (#993)
* Increase Neo4j memory limits, add GDS plugin, and update LLM concurrency limit to 256.
* Update ingestion sample file, disable KG node extraction pipe, add community processing in clustering, and enhance graph clustering queries.
* Update runners (#1007)
* Refactor KG clustering process to simplify community processing and enhance entity-triple retrieval from Neo4j.
* Refactor Neo4j configuration for memory settings and update graph clustering logic in the KG provider.
* Fix pipeline by enabling node extraction and refactor community processing logic in KGClusteringPipe.
* hatchet works
* throw error if you run global search before enrichment
* Fix communities in local search
* turn off node desc embedding
* fix rag endpoint
* Increase hatchet msg size
* Update ingestion.py
* Refactor and clean up code formatting
* modified workflow
* Add graph creation functionality
* Refactor KG parameters and logging.
* review
* up
---------
Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <owen@algofi.org>
Co-authored-by: Manuel R. Ciosici <manuelrciosici@gmail.com>
Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
* Feature/add hatchet api key setup rebased (#1040)
* add update files workflow
* rm ingestion pipeline
* v0 restructure orch
* kg orchestration
* finish kg orchestration
* update service
* merge
* cleanups
* add hatchet api key setup
* cleanup
* add hatchet api key setup (#1037)
* add hatchet api key setup
* cleanup
* fix merge
* cleanups
* Feature/nolan logs refactored (#1041)
* Update runners (#1007)
* Check in logs
---------
Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
* Pull open PRs into dev (#1042)
* Pull in subnet and graph PR
* Add in templates
* Add python files for templates in cli (#1043)
* working hatchet integration (#1046)
* Update local_llm_neo4j_kg.toml
* Unstructured fixes (#1048)
* dockerfile
* Update ingestion file with new sample URL and enhance unstructured chunking configuration and error handling.
* clean up
* clean up dockerfile
* up
* Update sample file and clean code
* Add hatchet-sdk dependency in project.
* Update providers to include local option.
* Introduce File Provider (#1044)
* Draft of file provider
* Some cleanup
* Regenearte lock
* Stream it
* Use document_id as primary key
* Pydantic v2
* File provider finished
* Make 7272 the default port (#1045)
* Fix poetry.lock
* Precommit
* Enhance Dockerfile and add telemetry events (#1049)
* Fix File Provider (#1050)
* Fix
* Fix parsing pipeline
* working
* Feature/improve docs (#1051)
* improve documentation
* fix unstr
* add ingestion
* fix compose
* Add unstructured chunking configuration updates
* Revert "Add unstructured chunking configuration updates"
This reverts commit
|
||
|
emrgnt-cmplxty
|
7cd3743630 |
Feature/merge dev and main (#968)
* Feature/merge graphrag group mgmt (#876) * add group ids to document abstraction, first steps * extend group permissions * up * add tests for new group features * up * fixup auth * onboard extensive regression tests * adding regression tests * finish tests * rm selenium * test observability * uncomment tests * checkin first set of group tests * modify search, passing vector tests * checkin work * full delete logic * update search to use new filters * check in * Clean up * Check in * add search * tests/test_end_to_end.py::test_ingest_txt_document passing * cleanup logging * make schemas explicit * move to run logger abstraction * cleanup some test workflows * revive tests * tweak to pass tests * tweak rrf * finish hybrid search cleanup * fixup on regr tests, regen payloads * refresh payloads * refactor api model * Feature/refactor api model (#868) * cleanup imports * flake and cleanup * coherent global import / export structure * add ingestion response models * add management response models * cleanups * checkin work on routes * remove request models * last fixes * merge * add user / group gating * working test groups * updating client --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> * Clean up API (#878) * Get running * fixes in sdk * Add in more fixes * Feature/merge dev owen changes (#880) * add group ids to document abstraction, first steps * extend group permissions * up * add tests for new group features * up * fixup auth * onboard extensive regression tests * adding regression tests * finish tests * rm selenium * test observability * uncomment tests * checkin first set of group tests * modify search, passing vector tests * checkin work * full delete logic * update search to use new filters * check in * Clean up * Check in * add search * tests/test_end_to_end.py::test_ingest_txt_document passing * cleanup logging * make schemas explicit * move to run logger abstraction * cleanup some test workflows * revive tests * tweak to pass tests * tweak rrf * finish hybrid search cleanup * fixup on regr tests, regen payloads * refresh payloads * refactor api model * Feature/refactor api model (#868) * cleanup imports * flake and cleanup * coherent global import / export structure * add ingestion response models * add management response models * cleanups * checkin work on routes * remove request models * last fixes * merge * add user / group gating * working test groups * updating client * rename service to restructure * add get documents for group endpoint * fix client bugs * return delete format * merge cleanups * merge * finalize --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> * Shreyas/graphrag test (#881) * add group ids to document abstraction, first steps * extend group permissions * up * add tests for new group features * up * fixup auth * onboard extensive regression tests * adding regression tests * finish tests * rm selenium * test observability * uncomment tests * checkin first set of group tests * modify search, passing vector tests * checkin work * full delete logic * update search to use new filters * check in * Clean up * Check in * add search * tests/test_end_to_end.py::test_ingest_txt_document passing * cleanup logging * make schemas explicit * move to run logger abstraction * cleanup some test workflows * revive tests * tweak to pass tests * tweak rrf * finish hybrid search cleanup * fixup on regr tests, regen payloads * refresh payloads * refactor api model * Feature/refactor api model (#868) * cleanup imports * flake and cleanup * coherent global import / export structure * add ingestion response models * add management response models * cleanups * checkin work on routes * remove request models * last fixes * merge * add user / group gating * sync * enrich * up * fix global search * rag * remove client.py * rm configs * rm configs --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> * Feature/fix embedding pipe (#882) * up * fixup concurrency * fix ollama embeddings * fix batching with ollama * checkin all cleanups * rm kg cruft (#884) * rm kg cruft * tweaks * tweak 2 (#885) * Feature/fix retrieval endpoint cruft (#887) * tweak 2 * fix retrieval endpoint descriptions * Python SDK (#886) Clean up Python SDK and routes * Separate out SDK, add js and go sdk to monorepo (#888) * Add r2r-js sdk * Add go sdk * Pull out python sdk * remove venv * Update packages * Check in fixes * Remove alembic dependencies * Feature/merge w nolan (#894) * cleanup hybrid search * cleanups in * Fix structure * Make graspologic optional * fix rag stream (#895) * add py r2r (#896) * Clean up (#897) * fix agent (#898) * define `RAGAgentResponse` (#899) * Shreyas/unstructured (#900) * api + oss lib * rm pdb * rm poetry lock * update version * fixes * Feature/cleanup client obj logic (#901) * define `RAGAgentResponse` * cleanup client logic * Shreyas/tests (#889) * init * tests * rename service * api model * add * merge * rm restructure router * print descriptions * Refactor CLI (#903) * Rm files readded by git (#904) * Remove Execution Wrapper (#905) * Rm files readded by git * Fix merge botch * Feature/fix auth revive tests rebased (#906) * adding the client touch ups * fix auth, revive tests * add back tests * uncomment run auth workflow * decruft * refresh test kg * fixup toml (#908) * Feature/fix ingestion update (#909) * fixup toml * fix update * Fix CLI Tests (#912) Fix CLI tests * Shreyas/kg runtime cfg (#913) add kg runtime config * rename kgenrichmentresponse (#914) * Feature/add nltk hybrid expansion rebased (#917) * expand hybrid search with nltk * cleanups * cleanup hybrid search * format * add setup.py * update * add script (#918) * Fix bug in document chunks (#921) * Fix bug in update files (#923) * Shreyas/unstructured (#922) * fix dockerfiles * adding config * fix paths * mv unstructured dep to docker * clean * Update docker_utils.py * Update unstructured_parsing.py * Update r2r_chunking.py * Update app_entry.py * Feature/repair logging (#925) * fixing logs * fix * rm double logging (#929) * Configs (#926) * Fix config logic * Update config * Clean up cli entry point * Disable SSL when installing nltk wordnet (#930) * Fix analytics endpoint * Update OpenAI sdk calls (#933) * Feature/revive advanced rag (#932) * rm double logging * revive advanced rag examples * merge (#934) * sync model (#935) * Feature/remove version from ingestion end pt (#936) * sync model * remove ability to set version * tweak versions impl * fix version bug * Move docker (#938) * Move docker * remove from root * Clean up sdk/restructure.py * Fix js tests, completion scoring (#939) * Shreyas/unstructured docker image (#940) unstructured docker image * Update JS (#941) * Update models (#942) * Feature/complete group logic (#945) * fix group logic * up * Fix Dockerbuild, Symlink Readme (#944) * Add back tast prompt override and include title if availible * Fix docker, sym link readme * Fix compose file path * Shreyas/KG Search Result model (#937) * return type to kg_search_result * add model * local and global results * modify config * refresh should not be gated by auth (#946) * Linting sync (#947) * Remove email from refresh (#948) * Fix link to image * Feature/rm print cruft rebase (#953) * refresh should not be gated by auth * rm print cruft * black and sort * merge * rm * update api return type * Update Actions (#954) * Update Github Actions (#956) * Update Actions * Update actions * Shreyas/kgsearchresult model (#957) * return type to kg_search_result * add model * local and global results * modify config * add models * up * fix config path * fix models * Login and refresh token bug (#959) * Update Actions * Fix bug in login with refresh token * Point pytest to linux (#960) * collection docs (#955) * Feature/merge dev to main (#962) * merge dev and main * git rm * add back collection fix * fix docker builds (#963) * Running unstructured docker + code cleanups (#964) * Small bugfixes on prompts, return types (#965) * Fix failing CLI tests * NPM publish action * remove tarball * Feature/fix dev tests (#966) * update auth tests * fix tests * back and sort * decruft * revert back to gpt-4o --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> |