0aeb00cbb1cfaed125fbb0f6bb123af695a5092c
19 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nolan Tremelling
|
d91ea8213a | Lint and fix type errors on ingestion servcie (#1935) | ||
|
Cristian Cezar Moisés
|
0a8abeae2d |
Update main.py
Add rate limiting for API endpoints Implement caching for frequent similar requests Add prometheus metrics endpoint Include pagination for large result sets Add distributed tracing support Implement background task processing for large graphs Add comprehensive unit tests with hypothesis for property-based testing |
||
|
Nolan Tremelling
|
06138e7f6c |
Clean up os contrib (#1900)
* Clean up os contrib * Clean up * More * Fix |
||
|
Cristian Cezar Moisés
|
47ff83f135 |
Some improvements (#1882)
* Update main.py
* Update main.py
Type Annotations and Descriptions: Added Field descriptions for better API documentation and validation.
Improved Logging: Enhanced logging to include more context and error tracebacks for easier debugging.
Input Validation: Used List from typing to enforce type checking and improve clarity.
Response Model: Ensured the response model is clearly defined with descriptions.
Exception Handling: Enhanced error handling to log the stack trace.
Code Formatting and Comments: Improved overall code formatting and added comments for clarity.
This revised version provides better maintainability and clarity for users of the API.
* Update collections.py
Utility Function: Introduced execute_collection_command to handle API calls, reducing code repetition and improving readability.
Type Hints: Added type hints for function arguments and return types, enhancing code clarity and allowing for better static analysis.
Error Messages: Improved error messages to specify the type of exception encountered, which can be useful for debugging.
Documentation: Updated docstrings for more clarity on what each command does.
Optional Parameters: Set default values for optional parameters in type hints.
This structure makes the code easier to maintain and understand while providing a clearer interface for handling API interactions.
* Update config.py
Type Hints: Added type hints to functions for better clarity and type checking.
Improved Docstrings: Enhanced the docstrings for clarity on what each method does.
Code Structure: Improved the organization of the code, particularly in the view function, to enhance readability.
Use of Constants: Used constants for repeated strings to avoid hardcoding, allowing easier adjustments in the future.
Console Output: Kept feedback messages concise and clear for better user experience.
* Update conversations.py
Error Messages: Improved the clarity and specificity of error messages to indicate which operation failed and with which ID.
Default Values: Clarified the purpose of the offset and limit options in the list and list_users commands.
Response Handling: Used response.get("results", []) to safely handle cases where the response may not contain the expected "results" key.
Consistent Formatting: Ensured consistent formatting in echo messages across different commands.
Improved Documentation: Enhanced the command descriptions for better understanding of their functionality.
These enhancements should improve the usability and maintainability of the CLI.
* Update database.py
Function check_connection_and_run: This function encapsulates the logic for checking the database connection and running the Alembic command, reducing duplication across your command functions.
Improved Exit Handling: All commands now return their results to handle exit codes more consistently.
Type Annotations: Each async command function is annotated with types for clarity.
User Prompts: The confirmation prompt for downgrades is improved to inform the user of the consequences of their action.
This refactored code should be cleaner, easier to maintain, and more user-friendly.
* Update documents.py
Modularization: Created helper functions like create_document_table, create_metadata_table, create_chunk_table, and create_collection_table to reduce code duplication and improve readability.
Error Handling: Enhanced error messages for better visibility and understanding of errors.
Use of Context Managers: Use of with statements for file handling, ensuring that files are properly closed.
Type Annotations: Ensured type annotations are consistent throughout the code, improving code clarity.
General Cleanup: Improved formatting, added comments for clarity, and ensured consistent usage of string formatting.
* Update graphs.py
Helper Functions: Added handle_response and handle_error functions to reduce code duplication and improve readability.
Consistent Success Messages: Each command now has a consistent success message upon successful execution.
Improved Error Handling: Centralized error handling simplifies the main command logic and enhances readability.
Comments: Added comments to provide context for the structure and purpose of functions.
* Update indices.py
Improved Error Messages: Specific prefixes for R2R errors help in distinguishing them from unexpected errors.
Check for Empty Results: Added a check in the list command to inform the user if no indices are found.
Type Annotations: Added type annotations for method parameters to improve readability and help with type checking.
Functionality in Delete Command: It seems your delete command was incorrectly retrieving instead of deleting. This has been corrected assuming a delete method exists in the client.indices object.
User Feedback: Added a success message for the delete operation to inform the user that the operation was completed successfully.
With these enhancements, the code is not only cleaner but also user-friendly, providing better feedback and handling various scenarios more gracefully.
* Update prompts.py
Logging: Added logging to track the flow of commands and errors. This can help in debugging and maintaining the code.
Type Hints: Added type hints for function parameters and return types for better clarity and tooling support.
Default Values and Help Text: Added help text for options in the retrieve command to provide more context to the user.
Use of get Method: Used .get() to handle potential KeyError when accessing the results key in the response dictionary in the list command.
Error Messages: Simplified the error messages for consistency and clarity.
These improvements should make the code more maintainable, user-friendly, and easier to debug.
* Update retrieval.py
Readability: Improved formatting and added comments for clarity.
Consistency: Maintained consistent naming conventions and structure across both command functions.
Error Handling: Improved error messages for better clarity.
Documentation: Enhanced docstrings for commands and parameters to provide better context.
Code Duplication: Reduced redundancy in the handling of search settings between the search and rag commands.
* Update system.py
Type Hints: Added type hints to function signatures for clarity.
Improved Error Messages: Enhanced error messages to be more descriptive, indicating the operation that failed.
Code Organization: Organized imports and structured the code for readability.
Consolidated Logic: Simplified condition checks where possible.
Consistent Logging: Ensured consistent logging of messages to assist with debugging and user feedback.
Function Documentation: Kept docstrings consistent and informative for each function.
* Update users.py
Improved Error Messages: Each command now provides more context in error messages, indicating which operation failed and why.
Use of get Method: When accessing dictionary keys (like response["results"]), switched to using .get() to avoid potential KeyError.
Documentation: Enhanced docstrings for each command to provide clearer explanations.
Type Hinting: Ensured all parameters have clear type hints for better readability and maintainability.
Code Consistency: Maintained consistent formatting and structure throughout the code for readability.
UUID Handling: The UUID conversion is done in a more concise way.
Default Values: Clearly stated default values in the help message for options.
* Bump nanoid in /js/sdk in the npm_and_yarn group across 1 directory
Bumps the npm_and_yarn group with 1 update in the /js/sdk directory: [nanoid](https://github.com/ai/nanoid).
Updates `nanoid` from 3.3.7 to 3.3.8
- [Release notes](https://github.com/ai/nanoid/releases)
- [Changelog](https://github.com/ai/nanoid/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ai/nanoid/compare/3.3.7...3.3.8)
---
updated-dependencies:
- dependency-name: nanoid
dependency-type: indirect
dependency-group: npm_and_yarn
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump python-multipart in /py in the pip group across 1 directory
Bumps the pip group with 1 update in the /py directory: [python-multipart](https://github.com/Kludex/python-multipart).
Updates `python-multipart` from 0.0.9 to 0.0.18
- [Release notes](https://github.com/Kludex/python-multipart/releases)
- [Changelog](https://github.com/Kludex/python-multipart/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Kludex/python-multipart/compare/0.0.9...0.0.18)
---
updated-dependencies:
- dependency-name: python-multipart
dependency-type: direct:production
dependency-group: pip
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump gunicorn in /py in the pip group across 1 directory
Bumps the pip group with 1 update in the /py directory: [gunicorn](https://github.com/benoitc/gunicorn).
Updates `gunicorn` from 21.2.0 to 22.0.0
- [Release notes](https://github.com/benoitc/gunicorn/releases)
- [Commits](https://github.com/benoitc/gunicorn/compare/21.2.0...22.0.0)
---
updated-dependencies:
- dependency-name: gunicorn
dependency-type: direct:production
dependency-group: pip
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update .pre-commit-config.yaml
* Pre-commit
* Remove venv directory from tracking
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
|
||
|
Nolan Tremelling
|
197e8b8410 | Update Unstructured with NLTK changes (#1852) | ||
|
Nolan Tremelling
|
0cee9e488c |
Implementing Missing R2R Parsers, Image Ingestion (#1702)
* Implement missing parsers * Working with Unstructured * Check in * Good to go |
||
|
emrgnt-cmplxty
|
4fc603a7e0 | merge in nolan + main | ||
|
emrgnt-cmplxty
|
ed336167fc | fix remote url | ||
|
emrgnt-cmplxty
|
196355b36a | add clustering logic | ||
|
emrgnt-cmplxty
|
32a6fdfa25 | add clustering as a service | ||
|
emrgnt-cmplxty
|
d693f690a6 |
Feature/v3 api (#1656)
* improve ci/cd runtime * update prompt tests * improve ci/cd runtime (#1535) * improve ci/cd runtime * update prompt tests * Support Python ^3.10 (#1534) * add azure * up * up * spec out v3 api * checkin document router * adding chunk abstr * add list chunks * add chunk search * up * add users routes * up * checkin progress * add collections annotations * add indices * add user * checkin work * up * complete conversations CRUD * fix type errors * add graph router * add graphs * Update JS (#1563) * Feature/add graph to v3 (#1565) * complete simple tests, cleanup routers * up * Harmonize Pagination across endpoints (#1564) * Pagination * Add fixmes * Fix nested deletion filter bug (#1567) * Remove Mintlify docs (#1569) * Nolan/list collection (#1568) * Check in * More * Fix nested transactions issue in sqlite logger * Fix update collection return type * JS V3 (#1571) * Sync collections JS * More documents * Clean up messy code * list not List * Users first pass * User tests and fixmes * More * typo * More prompts * Pre-commit improvements * Remove prints * Cleanups on conversations * Branches response * Chunks * More work on the return types * Jest config * Fix branch creation time * Fix lock * Nolan/v3 tests (#1578) * Add deprecated command back * Add warning * Fix GraphRAG tests (#1579) * More cleanup (#1580) * More cleanup * More * Fix test * More cleanups * More cleanups * More * Merge main * Python SDK V3 (#1585) * Python SDK V3 * Fix * First pass (#1586) * More V3 (#1587) * Validation errors * Update js test * more * Fix sync methods on v2 sdk, add check for download files (#1588) * More CLI (#1589) * Print logs on failing tests (#1590) * Print logs on failing tests * MOre * cleanup * Again * Again * More JS testing (#1591) * More JS testing * Cleanup * More refactors for tests (#1592) * System Routes (#1594) * Fix type errors, pass collection id (#1595) * Hotfix: dict * V3 graph implmentations (#1593) * complete simple tests, cleanup routers * up * up * checkin * up * up * response models * checkin * up * checkin * up * up * up * up * up * up * v2 * up * up * up * up --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Allow passing of collection id at document ingestion (#1596) * KG Response sync (#1597) * fix * Fix Prompt Override (#1599) * Fix Prompt Override * print * Caching * Fix * Updated Graph Models, Drop SID (#1598) * New Graph Models * Fix * minor tweaks * fix summary model (#1604) * incr progress * Add /users/me (#1605) * Add /users/me * oops * Resolve Merge Conflicts (#1607) * Fix conflicts * Clean up * Nolan/conflicts (#1608) * expose reset data to admin (#1602) * up (#1603) * up * up * wtf github is a piece of garbage --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> * wrapup walkthrough * Add delete user method, sync JS to camel case (#1609) * V3 graph testing (#1606) * up * up * up * graph crud * up * community endpts * up * up * up * up * up * up * up * up * add back routers * up * pre-commit * Fix Broken V2 Graphs, Better Response Models (#1612) * Increase test coverage * Fix v2 graphs, better response models * Remaining types * Add types to Python SDK * Typo * update tests * revert test change * up * Add types to package export (#1613) * Graph refactor (#1611) * up * up * add back routers * up * pre-commit * update tests * revert test change * up * simplify * up * add the add/remove endpoints * up * include routers back * Create branch update (#1617) * Graph refactor (#1616) * up * up * add back routers * up * pre-commit * update tests * revert test change * up * simplify * up * add the add/remove endpoints * up * include routers back * List collections (#1619) * up * up * up * up * Graph refactor (#1620) * up * up * add back routers * up * pre-commit * update tests * revert test change * up * simplify * up * add the add/remove endpoints * up * include routers back * up * up * up * up * Nolan/update graph (#1621) * List collections * Update Graph JS SDK * up * up * cleanup * Graph refactor (#1622) * up * up * add back routers * up * pre-commit * update tests * revert test change * up * simplify * up * add the add/remove endpoints * up * include routers back * up * up * up * up * up * up * cleanup * up * up * up * remove unnecessary functions * up * up * complete document embedding workflow * working get command on graph * checkin progress * up * add entity and relationship deletions * no verif * up * up * up * up (#1636) * up * sync graph * up * up * fix relationship distance calc. * fix issue with faulty collection filter (#1637) * Patch/alternative fix logics 2 (#1638) * fix issue with faulty collection filter * further refinements, like fixing limits * up * fix logic around include metadata and scores * fix double collection assignment * up * fix communities * working clusters * up * add collection extraction * add collection extraction * up * prep for merge * Patch/alternative up with nolan (#1643) * SDK First pass * Add feature tracking * Typo * Check in * Rebase * Add Graph tests * Fix Agent empty message bug * Check in JS routes * More tests, examples * Sync python * Expose Entity/Relationship Params in Routes (#1640) * Expose Entity/Relationship Params * Descriptions * Modify create entities * Create relationships * set parent_id * Update entitiy * Update Relationships * Check in * Ellipsis fixes * More cleanup * Start CRUD on communities * Communities DB * Explicit working path * Once again * Fail fast false * Testing around community creation * Delete community test * Update community tests * Clean up type errors, cleaner code * More cleanup * More * remove chunk_entity * Delete bad, unused methods * More * fixup crud * rm pull --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> * Feature/fix graph permissions (#1645) * update docs / collections * up * Feature/fix auth checks (#1647) * update docs / collections * up * fix super user and more * up * up (#1648) * Feature/rm v2 api (#1649) * SDK First pass * Add feature tracking * Typo * Check in * Rebase * Add Graph tests * Fix Agent empty message bug * Check in JS routes * More tests, examples * Sync python * Expose Entity/Relationship Params in Routes (#1640) * Expose Entity/Relationship Params * Descriptions * Modify create entities * Create relationships * set parent_id * Update entitiy * Update Relationships * Check in * Ellipsis fixes * More cleanup * Start CRUD on communities * Communities DB * Explicit working path * Once again * Fail fast false * Testing around community creation * Delete community test * Update community tests * Clean up type errors, cleaner code * More cleanup * More * remove chunk_entity * Delete bad, unused methods * More * remove v2 api * rm kg router * cleanups * fixup delete by filter * fixup delete by filter * fixes * up * up --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> * Improved Data Structures (#1650) * Check in * Most tests fixed * fix tables * Once more * Move to a single community table * Don't modify existing migration script--keep them atomic * Migration * Migration, more clean up * All but deletion working * Up * Feature/tweaks for prod (#1651) * tweaks for prod * up * final tweaks * Nolan/deletion (#1652) * Check in * Most tests fixed * fix tables * Once more * Move to a single community table * Don't modify existing migration script--keep them atomic * Migration * Migration, more clean up * All but deletion working * Up * Fix deletion * Working migration (#1654) * Feature/production tweaks (#1653) * tweaks for prod * up * final tweaks * prod tweaks * fixed --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> * sort --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> |
||
|
Shreyas Pimpalgaonkar
|
946743e662 | unstructured docker typo fix (#1533) | ||
|
emrgnt-cmplxty
|
21686b35a4 |
Dev minor (#1410)
* Fix async JSON parsing (#1408) * Fix async JSON parsing * Remove score completion from js * clean up js * lockfile * Feature/build custom logger (#1409) * building a custom logger for r2r * fix log * maintain bkwd compat * Feature/add kg description prompt (#1411) * add kg desc prompt * add kg desc prompt * add kg desc prompt * fix prompt name * separate test run freq (#1412) * separate test run freq * task_id check fix * add ingestion docs * updatet * add * rm old prompts * rm old prompots * rm old prompts * rm old prompts * Prod fixes + enhancements (#1407) * change default settings back to fp32 * add logging and cache triples * up * up * pre-commit and cleanups * making community summary prompt async * up * up * revert prompt changes * up * up * modify default * bump test timeout due to stricter concurrency limits * bump sleep * rm ubuntu from windows/mac workflows --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> |
||
|
emrgnt-cmplxty
|
eed926235d |
D3v (#1398) (#1399)
* D3v (#1398) * Feature/add zerox parser (#1396) * Add KG tests (#1351) * cli tests * add sdk tests * typo fix * change workflow ordering * add collection integration tests (#1352) * bump pkg * remove workflows * fix sdk test port * fix delete collection return check * Fix document info serialization (#1353) * Update integration-test-workflow-debian.yml * pre-commit * slightly modify * up * up * smaller file * up * typo, change order * up * up * change order --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * add graphrag docs (#1362) * add documentation * up * Update js/sdk/src/models.tsx Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * pre-commit --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Concurrent index creation, allow -1 for paginated entries (#1363) * update webdev-template for current next.js and r2r-js sdk (#1218) Co-authored-by: Simeon <simeon@theobald.nz> * Feature/extend integration tests rebased (#1361) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * Feature/rebase to r2r vars (#1364) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * make changes * update the windows workflow * update the windows workflow * remove extra workflows for now * bump pkg * push and pray * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive tests * revive tests * revive tests * revive tests * update tests * fix typos (#1366) * update tests * up * up * up * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * Add ingestion concurrency limit (#1367) * up * up * up --------- Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * tweaks and fixes * Fix Ollama Tool Calling (#1372) * Update graphrag.mdx * Fix Ollama tool calling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Clean up Docker Compose (#1368) * Fix hatchet, dockerfile * Update compose * point to correct docker image * Fix bug in deletion, better validation error handling (#1374) * Update graphrag.mdx * Fix bug in deletion, better validation error handling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * vec index creation endpoint (#1373) * Update graphrag.mdx * upload files * create vector index endpoint * add to fastapi background task * pre-commit * move logging * add api spec, support for all vecs * pre-commit * add workflow * Modify KG Endpoints and update API spec (#1369) * Update graphrag.mdx * modify API endpoints and update documentation * Update ingestion_router.py * try different docker setup (#1371) * try different docker setup * action * add login * add full * update action * cleanup upload script * cleanup upload script * tweak action * tweak action * tweak action * tweak action * tweak action * tweak action * Nolan/ingest chunks js (#1375) * Update graphrag.mdx * Clean up ingest chunks, add to JS SDK * Update JS docs --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * up (#1376) * Bump JS package (#1378) * add conversation * checkin progress * checkin progress * Fix Create Graph (#1379) * up * up * modify assertion * up * up * increase entity limit * changing aristotle back to v2 * pre-commit * typos * add test_ingest_sample_file_2_sdk * Update server.py * checkin progress * up * update * Graphrag docs (#1382) * add docs and refine code * add python SDK documentation * up * update * checkin * up * cleanup * working sync logging * test conversation history * fix runner tests, rename `CHUNKS` to `chunks` * adding zerox parser --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * Nolan/update hatchet (#1397) * Move Hatchet to latest * Update js package-lock --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> * Add hatchet logging (#1391) * Add KG tests (#1351) * cli tests * add sdk tests * typo fix * change workflow ordering * add collection integration tests (#1352) * bump pkg * remove workflows * fix sdk test port * fix delete collection return check * Fix document info serialization (#1353) * Update integration-test-workflow-debian.yml * pre-commit * slightly modify * up * up * smaller file * up * typo, change order * up * up * change order --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * add graphrag docs (#1362) * add documentation * up * Update js/sdk/src/models.tsx Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * pre-commit --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Concurrent index creation, allow -1 for paginated entries (#1363) * update webdev-template for current next.js and r2r-js sdk (#1218) Co-authored-by: Simeon <simeon@theobald.nz> * Feature/extend integration tests rebased (#1361) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * Feature/rebase to r2r vars (#1364) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * make changes * update the windows workflow * update the windows workflow * remove extra workflows for now * bump pkg * push and pray * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive tests * revive tests * revive tests * revive tests * update tests * fix typos (#1366) * update tests * up * up * up * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * Add ingestion concurrency limit (#1367) * up * up * up --------- Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * tweaks and fixes * Fix Ollama Tool Calling (#1372) * Update graphrag.mdx * Fix Ollama tool calling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Clean up Docker Compose (#1368) * Fix hatchet, dockerfile * Update compose * point to correct docker image * Fix bug in deletion, better validation error handling (#1374) * Update graphrag.mdx * Fix bug in deletion, better validation error handling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * vec index creation endpoint (#1373) * Update graphrag.mdx * upload files * create vector index endpoint * add to fastapi background task * pre-commit * move logging * add api spec, support for all vecs * pre-commit * add workflow * Modify KG Endpoints and update API spec (#1369) * Update graphrag.mdx * modify API endpoints and update documentation * Update ingestion_router.py * try different docker setup (#1371) * try different docker setup * action * add login * add full * update action * cleanup upload script * cleanup upload script * tweak action * tweak action * tweak action * tweak action * tweak action * tweak action * Nolan/ingest chunks js (#1375) * Update graphrag.mdx * Clean up ingest chunks, add to JS SDK * Update JS docs --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * up (#1376) * Bump JS package (#1378) * Fix Create Graph (#1379) * up * up * modify assertion * up * up * increase entity limit * changing aristotle back to v2 * pre-commit * typos * add test_ingest_sample_file_2_sdk * Update server.py * add docs and refine code * add python SDK documentation * up * add logs * clean * rm vq * rm conflicts * pre-commit * up * add logging * update logs * up * up * Update kg_service.py --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * feat: Add delete_node_via_document_id method to KGProvider (#1387) Co-authored-by: shou.hsu <shou.hsu@utonia.com> --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> Co-authored-by: Shou-Hsu <148736054+Shou-Hsu@users.noreply.github.com> Co-authored-by: shou.hsu <shou.hsu@utonia.com> * Vector + Index quantization (#1400) * Add KG tests (#1351) * cli tests * add sdk tests * typo fix * change workflow ordering * add collection integration tests (#1352) * bump pkg * remove workflows * fix sdk test port * fix delete collection return check * Fix document info serialization (#1353) * Update integration-test-workflow-debian.yml * pre-commit * slightly modify * up * up * smaller file * up * typo, change order * up * up * change order --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * add graphrag docs (#1362) * add documentation * up * Update js/sdk/src/models.tsx Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * pre-commit --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Concurrent index creation, allow -1 for paginated entries (#1363) * update webdev-template for current next.js and r2r-js sdk (#1218) Co-authored-by: Simeon <simeon@theobald.nz> * Feature/extend integration tests rebased (#1361) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * Feature/rebase to r2r vars (#1364) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * make changes * update the windows workflow * update the windows workflow * remove extra workflows for now * bump pkg * push and pray * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive tests * revive tests * revive tests * revive tests * update tests * fix typos (#1366) * update tests * up * up * up * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * Add ingestion concurrency limit (#1367) * up * up * up --------- Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * tweaks and fixes * Fix Ollama Tool Calling (#1372) * Update graphrag.mdx * Fix Ollama tool calling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Clean up Docker Compose (#1368) * Fix hatchet, dockerfile * Update compose * point to correct docker image * Fix bug in deletion, better validation error handling (#1374) * Update graphrag.mdx * Fix bug in deletion, better validation error handling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * vec index creation endpoint (#1373) * Update graphrag.mdx * upload files * create vector index endpoint * add to fastapi background task * pre-commit * move logging * add api spec, support for all vecs * pre-commit * add workflow * Modify KG Endpoints and update API spec (#1369) * Update graphrag.mdx * modify API endpoints and update documentation * Update ingestion_router.py * try different docker setup (#1371) * try different docker setup * action * add login * add full * update action * cleanup upload script * cleanup upload script * tweak action * tweak action * tweak action * tweak action * tweak action * tweak action * Nolan/ingest chunks js (#1375) * Update graphrag.mdx * Clean up ingest chunks, add to JS SDK * Update JS docs --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * up (#1376) * Bump JS package (#1378) * Fix Create Graph (#1379) * up * up * modify assertion * up * up * increase entity limit * changing aristotle back to v2 * pre-commit * typos * add test_ingest_sample_file_2_sdk * Update server.py * add docs and refine code * add python SDK documentation * up * add logs * merge changes * mc * more * add index + vector quantization * pre-commits * chnage default back to FP32 * kg vector and index test * rm duplicate import * Update r2r.toml --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * Feature/add zerox to unstructured rebased (#1406) * Feature/add zerox parser (#1396) * Add KG tests (#1351) * cli tests * add sdk tests * typo fix * change workflow ordering * add collection integration tests (#1352) * bump pkg * remove workflows * fix sdk test port * fix delete collection return check * Fix document info serialization (#1353) * Update integration-test-workflow-debian.yml * pre-commit * slightly modify * up * up * smaller file * up * typo, change order * up * up * change order --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * add graphrag docs (#1362) * add documentation * up * Update js/sdk/src/models.tsx Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * pre-commit --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Concurrent index creation, allow -1 for paginated entries (#1363) * update webdev-template for current next.js and r2r-js sdk (#1218) Co-authored-by: Simeon <simeon@theobald.nz> * Feature/extend integration tests rebased (#1361) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * Feature/rebase to r2r vars (#1364) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * make changes * update the windows workflow * update the windows workflow * remove extra workflows for now * bump pkg * push and pray * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive tests * revive tests * revive tests * revive tests * update tests * fix typos (#1366) * update tests * up * up * up * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * Add ingestion concurrency limit (#1367) * up * up * up --------- Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * tweaks and fixes * Fix Ollama Tool Calling (#1372) * Update graphrag.mdx * Fix Ollama tool calling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Clean up Docker Compose (#1368) * Fix hatchet, dockerfile * Update compose * point to correct docker image * Fix bug in deletion, better validation error handling (#1374) * Update graphrag.mdx * Fix bug in deletion, better validation error handling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * vec index creation endpoint (#1373) * Update graphrag.mdx * upload files * create vector index endpoint * add to fastapi background task * pre-commit * move logging * add api spec, support for all vecs * pre-commit * add workflow * Modify KG Endpoints and update API spec (#1369) * Update graphrag.mdx * modify API endpoints and update documentation * Update ingestion_router.py * try different docker setup (#1371) * try different docker setup * action * add login * add full * update action * cleanup upload script * cleanup upload script * tweak action * tweak action * tweak action * tweak action * tweak action * tweak action * Nolan/ingest chunks js (#1375) * Update graphrag.mdx * Clean up ingest chunks, add to JS SDK * Update JS docs --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * up (#1376) * Bump JS package (#1378) * add conversation * checkin progress * checkin progress * Fix Create Graph (#1379) * up * up * modify assertion * up * up * increase entity limit * changing aristotle back to v2 * pre-commit * typos * add test_ingest_sample_file_2_sdk * Update server.py * checkin progress * up * update * Graphrag docs (#1382) * add docs and refine code * add python SDK documentation * up * update * checkin * up * cleanup * working sync logging * test conversation history * fix runner tests, rename `CHUNKS` to `chunks` * adding zerox parser --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * Nolan/update hatchet (#1397) * Move Hatchet to latest * Update js package-lock --------- Co-authored-by: NolanTrem <34580718+NolanTrem@users.noreply.github.com> * Add hatchet logging (#1391) * Add KG tests (#1351) * cli tests * add sdk tests * typo fix * change workflow ordering * add collection integration tests (#1352) * bump pkg * remove workflows * fix sdk test port * fix delete collection return check * Fix document info serialization (#1353) * Update integration-test-workflow-debian.yml * pre-commit * slightly modify * up * up * smaller file * up * typo, change order * up * up * change order --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * add graphrag docs (#1362) * add documentation * up * Update js/sdk/src/models.tsx Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * pre-commit --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Concurrent index creation, allow -1 for paginated entries (#1363) * update webdev-template for current next.js and r2r-js sdk (#1218) Co-authored-by: Simeon <simeon@theobald.nz> * Feature/extend integration tests rebased (#1361) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * Feature/rebase to r2r vars (#1364) * cleanups * add back overzealous edits * extend workflows * fix full setup * simplify cli * add ymls * rename to light * try again * start light * add cli tests * fix * fix * testing.. * trying complete matrix testflow * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * cleanup matrix logic * up * up * up * All actions * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * rename to runner * try offic pgvec formula * sudo make * sudo make * push and pray * push and pray * add new actions * add new actions * docker push & pray * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * inspect manifests during launch * setup docker * setup docker * fix default * fix default * make changes * update the windows workflow * update the windows workflow * remove extra workflows for now * bump pkg * push and pray * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive full workflow * revive tests * revive tests * revive tests * revive tests * update tests * fix typos (#1366) * update tests * up * up * up * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * bump max connections * Add ingestion concurrency limit (#1367) * up * up * up --------- Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * tweaks and fixes * Fix Ollama Tool Calling (#1372) * Update graphrag.mdx * Fix Ollama tool calling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Clean up Docker Compose (#1368) * Fix hatchet, dockerfile * Update compose * point to correct docker image * Fix bug in deletion, better validation error handling (#1374) * Update graphrag.mdx * Fix bug in deletion, better validation error handling --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * vec index creation endpoint (#1373) * Update graphrag.mdx * upload files * create vector index endpoint * add to fastapi background task * pre-commit * move logging * add api spec, support for all vecs * pre-commit * add workflow * Modify KG Endpoints and update API spec (#1369) * Update graphrag.mdx * modify API endpoints and update documentation * Update ingestion_router.py * try different docker setup (#1371) * try different docker setup * action * add login * add full * update action * cleanup upload script * cleanup upload script * tweak action * tweak action * tweak action * tweak action * tweak action * tweak action * Nolan/ingest chunks js (#1375) * Update graphrag.mdx * Clean up ingest chunks, add to JS SDK * Update JS docs --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * up (#1376) * Bump JS package (#1378) * Fix Create Graph (#1379) * up * up * modify assertion * up * up * increase entity limit * changing aristotle back to v2 * pre-commit * typos * add test_ingest_sample_file_2_sdk * Update server.py * add docs and refine code * add python SDK documentation * up * add logs * clean * rm vq * rm conflicts * pre-commit * up * add logging * update logs * up * up * Update kg_service.py --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> * feat: Add delete_node_via_document_id method to KGProvider (#1387) Co-authored-by: shou.hsu <shou.hsu@utonia.com> * Finishing `zerox` integration * Finishing `zerox` integration * fix zerox, add cli docs * mention runtime config --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> Co-authored-by: Shou-Hsu <148736054+Shou-Hsu@users.noreply.github.com> Co-authored-by: shou.hsu <shou.hsu@utonia.com> * Improve ingestion errors, status bugs (#1405) * Move Hatchet to latest * Update js package-lock * Better handling for errors on ingestion * Filter out already processed chunks in create graph (#1404) * add simple filter * up * modify query * add zerox test * fix conversation history * attempt fix tests * adding sleep before retrieving the list of entities * bump version --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: FutureProofTechOps <operations@theobald.nz> Co-authored-by: Simeon <simeon@theobald.nz> Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com> Co-authored-by: Shou-Hsu <148736054+Shou-Hsu@users.noreply.github.com> Co-authored-by: shou.hsu <shou.hsu@utonia.com> |
||
|
emrgnt-cmplxty
|
55d7d873b0 |
Dev minor (#1321)
* no-verify (#1314) * no-verify * update readme * Kg testing (#1300) * Feature/encapsulate orchestration (#1265) * fully encapsulate orchestration * fully encapsulate orchestration * complete encapsulation * revert import cmt * making default r2r lighter (#1268) * making default r2r lighter * fix bug in ingest files * checkin * workingupdate * complete simple orch * update docs * up (#1273) * up * up * merge (#1276) * Postgres configuration settings (#1277) * Improvements on Auth in JS, CLI (#1267) * CLI Telemetry (#1266) * check in * working * redundant * JS auth improvements (#1263) * Check in JS auth improvements * Update login with toke * Fix to allow disabling telemetry * fix lock * Try to avoid merge conflicts * Clean up collection bugs * remove comments * Add Postgres configuration settings * Image * bad github conflict * merge (#1278) * port KG to postgres (#1272) * create + cluster * local search * up * clean * format * basics * add collection_id and paginate * rename * change api * up * kg_creation_status * up * up * up * Feature/cleanup docker (#1279) * merge * up * rm neo4j refs and cleanup docker cmds * fixup * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * Patch/cleanup kg migration (#1281) * cleanup kg migration * up * Kg testing (#1280) * up * up * up * up * slay neo4j --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * add back poetry lock * Default Collections (#1282) * Default collections * Naughty naughty need to follow the SRP * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * Testing (#1284) * CICD * actions * poetry * poetry * Add env vars * name * increase timeout * add user to collection * change postgres project name * Kg testing (#1283) * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * change postgres project name --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Feature/fix logic bugs (#1285) * fixing minor logic bugs in dev branch * fixing minor logic bugs in dev branch * merge * up * Application docs * add image (#1287) * Add version to CLI telemetry (#1288) * add image * Add version to cli telemetry * up * KG hatchet orchestration (#1286) * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * change postgres project name * up * up --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Feature/update documentation rebased (#1289) * up * merge * rebase * fix ingestion issues (#1291) * fix ingestion issues * fix lock file * fix embedding * Fix SDK KG Serialization (#1292) * add image * serialization * cleanup cli (#1294) * CLI serialization (#1295) * add image * Fix more serialization around kg * Nolan/schemacreation (#1296) * add image * Fix more serialization around kg * add quotes to prevent reserved keywords from failing * Prevent errors if config name is reserved name in postgres (#1297) * Prevent reserved words (#1298) * default collection ID * up * Move default collection id method to utils (#1299) * up * Allow json fallback (#1301) * hotfix: import * Fix description error (#1302) * up * push * up (#1303) * up * up * up * up * minor tweaks * up * mypy * add back missing file * up * up * up * fix id * up --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> * add way to access entities and relationships (#1313) * Feature/encapsulate orchestration (#1265) * fully encapsulate orchestration * fully encapsulate orchestration * complete encapsulation * revert import cmt * making default r2r lighter (#1268) * making default r2r lighter * fix bug in ingest files * checkin * workingupdate * complete simple orch * update docs * up (#1273) * up * up * merge (#1276) * Postgres configuration settings (#1277) * Improvements on Auth in JS, CLI (#1267) * CLI Telemetry (#1266) * check in * working * redundant * JS auth improvements (#1263) * Check in JS auth improvements * Update login with toke * Fix to allow disabling telemetry * fix lock * Try to avoid merge conflicts * Clean up collection bugs * remove comments * Add Postgres configuration settings * Image * bad github conflict * merge (#1278) * port KG to postgres (#1272) * create + cluster * local search * up * clean * format * basics * add collection_id and paginate * rename * change api * up * kg_creation_status * up * up * up * Feature/cleanup docker (#1279) * merge * up * rm neo4j refs and cleanup docker cmds * fixup * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * Patch/cleanup kg migration (#1281) * cleanup kg migration * up * Kg testing (#1280) * up * up * up * up * slay neo4j --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * add back poetry lock * Default Collections (#1282) * Default collections * Naughty naughty need to follow the SRP * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * Testing (#1284) * CICD * actions * poetry * poetry * Add env vars * name * increase timeout * add user to collection * change postgres project name * Kg testing (#1283) * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * change postgres project name --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Feature/fix logic bugs (#1285) * fixing minor logic bugs in dev branch * fixing minor logic bugs in dev branch * merge * up * Application docs * add image (#1287) * Add version to CLI telemetry (#1288) * add image * Add version to cli telemetry * up * KG hatchet orchestration (#1286) * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * change postgres project name * up * up --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Feature/update documentation rebased (#1289) * up * merge * rebase * fix ingestion issues (#1291) * fix ingestion issues * fix lock file * fix embedding * Fix SDK KG Serialization (#1292) * add image * serialization * cleanup cli (#1294) * CLI serialization (#1295) * add image * Fix more serialization around kg * Nolan/schemacreation (#1296) * add image * Fix more serialization around kg * add quotes to prevent reserved keywords from failing * Prevent errors if config name is reserved name in postgres (#1297) * Prevent reserved words (#1298) * default collection ID * up * Move default collection id method to utils (#1299) * up * Allow json fallback (#1301) * hotfix: import * Fix description error (#1302) * up * push * up (#1303) * up * up * up * up * minor tweaks * up * mypy * add back missing file * up * add way to access entities and relationships * up * up * fix id * up * refine end pts --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * Add collection (#1317) * add filters (in progress) * todo comment * minor addition * modify command * Permission Fixes around Collection Management (#1316) * Allow super users to update others to super user status * Fix auth on collections endpoints * Better error message * filters (#1318) * Cost estimate (#1319) * up * slightly modify * up * minor fix * docs --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> |
||
|
emrgnt-cmplxty
|
3721fcb7ad |
dev (#1269)
* Feature/encapsulate orchestration (#1265) * fully encapsulate orchestration * fully encapsulate orchestration * complete encapsulation * revert import cmt * making default r2r lighter (#1268) * making default r2r lighter * fix bug in ingest files * checkin * workingupdate * complete simple orch * update docs * up (#1273) * up * up * merge (#1276) * Postgres configuration settings (#1277) * Improvements on Auth in JS, CLI (#1267) * CLI Telemetry (#1266) * check in * working * redundant * JS auth improvements (#1263) * Check in JS auth improvements * Update login with toke * Fix to allow disabling telemetry * fix lock * Try to avoid merge conflicts * Clean up collection bugs * remove comments * Add Postgres configuration settings * Image * bad github conflict * merge (#1278) * port KG to postgres (#1272) * create + cluster * local search * up * clean * format * basics * add collection_id and paginate * rename * change api * up * kg_creation_status * up * up * up * Feature/cleanup docker (#1279) * merge * up * rm neo4j refs and cleanup docker cmds * fixup * Patch/cleanup kg migration (#1281) * cleanup kg migration * up * Kg testing (#1280) * up * up * up * up * slay neo4j --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * add back poetry lock * Default Collections (#1282) * Default collections * Naughty naughty need to follow the SRP * Testing (#1284) * CICD * actions * poetry * poetry * Add env vars * name * increase timeout * add user to collection * Kg testing (#1283) * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * change postgres project name --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Feature/fix logic bugs (#1285) * fixing minor logic bugs in dev branch * fixing minor logic bugs in dev branch * merge * Application docs * add image (#1287) * Add version to CLI telemetry (#1288) * add image * Add version to cli telemetry * KG hatchet orchestration (#1286) * up * up * cleanup kg migration * up * up * up * Kg testing (#1280) * up * up * up * up * rename * project name * up * add chunk order * fragments => extractions * bug squash * up * up * up * change postgres project name * up * up --------- Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Feature/update documentation rebased (#1289) * up * merge * rebase * fix ingestion issues (#1291) * fix ingestion issues * fix lock file * fix embedding * Fix SDK KG Serialization (#1292) * add image * serialization * cleanup cli (#1294) * CLI serialization (#1295) * add image * Fix more serialization around kg * Nolan/schemacreation (#1296) * add image * Fix more serialization around kg * add quotes to prevent reserved keywords from failing * Prevent errors if config name is reserved name in postgres (#1297) * Prevent reserved words (#1298) * Move default collection id method to utils (#1299) * Allow json fallback (#1301) * hotfix: import * Fix description error (#1302) * up (#1303) * rename to `full` (#1304) * rename to `full` * add html parser * Remove postgres vecs variables (#1306) * Feature/rename ingest files (#1307) * rename to `full` * add html parser * Vec Removal (#1308) * Remove postgres vecs variables * up * change kg settings parsing (#1309) * offset + limit (#1305) * offset + limit * fix order * update query * change entity offset * leiden seed --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> |
||
|
Nolan Tremelling
|
8644a086c3 |
Improvements on Auth in JS, CLI (#1267)
* CLI Telemetry (#1266) * check in * working * redundant * JS auth improvements (#1263) * Check in JS auth improvements * Update login with toke * Fix to allow disabling telemetry * fix lock * Try to avoid merge conflicts |
||
|
emrgnt-cmplxty
|
4cab317585 |
Dev minor (#1207)
* Feature/revamp agent streaming (#1204) * revamp agent streaming * revamp agent streaming * Modify return spec (#1205) * Feature/improve r2r telemetry (#1122) * improve telemetry * finish telemetry tweaks * Feature/improve cli infra (#1123) * improve telemetry * finish telemetry tweaks * up * Feature/add serve fallback to main (#1125) * improve telemetry * finish telemetry tweaks * up * fallback to main * Merge fragments (#1127) * troubleshooting docs (#1128) * troubleshooting docs (#1129) * add system diagram (#1130) * add system diagram * rm multi * fix overview * cleanup and fix * fix syntax * change to fast strategy by default (#1133) * Update parameter passing in js sdk (#1132) * Docs changes + add entity and relationship types (#1134) * up * up * up * up * reduce verbosity * Feature/dev minor cleanups (#1135) * cleanups * bump pkg * Update Tesseract OCR version in Dockerfile and change chunking strategy to "auto" * Update chunking strategy to "auto" in r2r.toml * up * set response model * add communities --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Modify return spec (#1206) * Feature/improve r2r telemetry (#1122) * improve telemetry * finish telemetry tweaks * Feature/improve cli infra (#1123) * improve telemetry * finish telemetry tweaks * up * Feature/add serve fallback to main (#1125) * improve telemetry * finish telemetry tweaks * up * fallback to main * Merge fragments (#1127) * troubleshooting docs (#1128) * troubleshooting docs (#1129) * add system diagram (#1130) * add system diagram * rm multi * fix overview * cleanup and fix * fix syntax * change to fast strategy by default (#1133) * Update parameter passing in js sdk (#1132) * Docs changes + add entity and relationship types (#1134) * up * up * up * up * reduce verbosity * Feature/dev minor cleanups (#1135) * cleanups * bump pkg * Update Tesseract OCR version in Dockerfile and change chunking strategy to "auto" * Update chunking strategy to "auto" in r2r.toml * up * set response model * add communities * update docs * revert * chore: Remove unused 'reload' parameter in run_local_serve function --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> * Patch/fix kg issue (#1208) * revamp agent streaming * revamp agent streaming * fix kg bug * Feature/run pre commit (#1209) * revamp agent streaming * revamp agent streaming * fix kg bug * rm print * add pre-commit --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com> |
||
|
Shreyas Pimpalgaonkar
|
0c9ea9d119 | move unstructured to services |