emrgnt-cmplxty/R2R
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,957 Commits 24 Branches 67 Tags
4e18d5d02e48d341c57a41fdb5f525cb81fcc925
Commit Graph

23 Commits

Author SHA1 Message Date
Artur Tanona e2c6484967 Add XAI API KEY (#1865) 2025-01-23 09:49:32 -08:00
emrgnt-cmplxty a640fb20b2 Release 3.2.11 (#1365) 
• Added Knowledge Graph (KG) tests (#1351)
  - CLI tests
  - SDK tests
  - Workflow ordering changes

• Added collection integration tests (#1352)
  - Fixed SDK test port
  - Fixed delete collection return check

• Fixed document info serialization (#1353)

• Added GraphRAG documentation (#1362)

• Implemented concurrent index creation (#1363)
  - Allow -1 for paginated entries

• Updated webdev-template for current Next.js and r2r-js SDK (#1218)

• Extended integration tests (#1361)
  - Cleaned up and simplified CLI
  - Added new workflows and actions
  - Implemented matrix test flow
  - Docker setup and configuration

• Rebased to r2r vars (#1364)
  - Updated Windows workflow
  - Revived full workflow and tests

• Fixed typos (#1366)

• Added ingestion concurrency limit (#1367)

• Fixed Ollama Tool Calling (#1372)

• Cleaned up Docker Compose (#1368)

• Fixed bug in deletion, improved validation error handling (#1374)

• Added vector index creation endpoint (#1373)

• Modified KG Endpoints and updated API spec (#1369)

• Implemented new Docker setup (#1371)
  - Updated actions and scripts

• Cleaned up ingest chunks, added to JS SDK (#1375)

• Various updates and fixes (#1376, #1378, #1379)
  - Increased entity limit
  - Changed Aristotle back to v2
  - Added test_ingest_sample_file_2_sdk

• Added GraphRAG documentation and refined code (#1382)
  - Added Python SDK documentation

• Added R2R_ prefix to env vars (#1383)
  - Cleaned up docker compose
  - Bumped Python and JS package versions
  - Updated form-data

---------

Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>
Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: FutureProofTechOps <operations@theobald.nz>
Co-authored-by: Simeon <simeon@theobald.nz>
Co-authored-by: --global=Shreyas Pimpalgaonkar <--global=shreyas.gp.7@gmail.com>
 2024-10-11 15:51:49 -07:00
emrgnt-cmplxty 55d7d873b0 Dev minor (#1321) 
* no-verify (#1314)

* no-verify

* update readme

* Kg testing (#1300)

* Feature/encapsulate orchestration (#1265)

* fully encapsulate orchestration

* fully encapsulate orchestration

* complete encapsulation

* revert import cmt

* making default r2r lighter (#1268)

* making default r2r lighter

* fix bug in ingest files

* checkin

* workingupdate

* complete simple orch

* update docs

* up (#1273)

* up

* up

* merge (#1276)

* Postgres configuration settings (#1277)

* Improvements on Auth in JS, CLI (#1267)

* CLI Telemetry (#1266)

* check in

* working

* redundant

* JS auth improvements (#1263)

* Check in JS auth improvements

* Update login with toke

* Fix to allow disabling telemetry

* fix lock

* Try to avoid merge conflicts

* Clean up collection bugs

* remove comments

* Add Postgres configuration settings

* Image

* bad github conflict

* merge (#1278)

* port KG to postgres (#1272)

* create + cluster

* local search

* up

* clean

* format

* basics

* add collection_id and paginate

* rename

* change api

* up

* kg_creation_status

* up

* up

* up

* Feature/cleanup docker (#1279)

* merge

* up

* rm neo4j refs and cleanup docker cmds

* fixup

* up

* up

* cleanup kg migration

* up

* up

* up

* Kg testing (#1280)

* up

* up

* up

* up

* rename

* Patch/cleanup kg migration (#1281)

* cleanup kg migration

* up

* Kg testing (#1280)

* up

* up

* up

* up

* slay neo4j

---------

Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>

* add back poetry lock

* Default Collections (#1282)

* Default collections

* Naughty naughty need to follow the SRP

* project name

* up

* add chunk order

* fragments => extractions

* bug squash

* up

* up

* up

* Testing (#1284)

* CICD

* actions

* poetry

* poetry

* Add env vars

* name

* increase timeout

* add user to collection

* change postgres project name

* Kg testing (#1283)

* up

* up

* cleanup kg migration

* up

* up

* up

* Kg testing (#1280)

* up

* up

* up

* up

* rename

* project name

* up

* add chunk order

* fragments => extractions

* bug squash

* up

* up

* up

* change postgres project name

---------

Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* Feature/fix logic bugs (#1285)

* fixing minor logic bugs in dev branch

* fixing minor logic bugs in dev branch

* merge

* up

* Application docs

* add image (#1287)

* Add version to CLI telemetry (#1288)

* add image

* Add version to cli telemetry

* up

* KG hatchet orchestration (#1286)

* up

* up

* cleanup kg migration

* up

* up

* up

* Kg testing (#1280)

* up

* up

* up

* up

* rename

* project name

* up

* add chunk order

* fragments => extractions

* bug squash

* up

* up

* up

* change postgres project name

* up

* up

---------

Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* Feature/update documentation rebased (#1289)

* up

* merge

* rebase

* fix ingestion issues (#1291)

* fix ingestion issues

* fix lock file

* fix embedding

* Fix SDK KG Serialization (#1292)

* add image

* serialization

* cleanup cli (#1294)

* CLI serialization (#1295)

* add image

* Fix more serialization around kg

* Nolan/schemacreation (#1296)

* add image

* Fix more serialization around kg

* add quotes to prevent reserved keywords from failing

* Prevent errors if config name is reserved name in postgres (#1297)

* Prevent reserved words (#1298)

* default collection ID

* up

* Move default collection id method to utils (#1299)

* up

* Allow json fallback (#1301)

* hotfix: import

* Fix description error (#1302)

* up

* push

* up (#1303)

* up

* up

* up

* up

* minor tweaks

* up

* mypy

* add back missing file

* up

* up

* up

* fix id

* up

---------

Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com>
Co-authored-by: emrgnt-cmplxty <owen@algofi.org>
Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>

* add way to access entities and relationships (#1313)

* Feature/encapsulate orchestration (#1265)

* fully encapsulate orchestration

* fully encapsulate orchestration

* complete encapsulation

* revert import cmt

* making default r2r lighter (#1268)

* making default r2r lighter

* fix bug in ingest files

* checkin

* workingupdate

* complete simple orch

* update docs

* up (#1273)

* up

* up

* merge (#1276)

* Postgres configuration settings (#1277)

* Improvements on Auth in JS, CLI (#1267)

* CLI Telemetry (#1266)

* check in

* working

* redundant

* JS auth improvements (#1263)

* Check in JS auth improvements

* Update login with toke

* Fix to allow disabling telemetry

* fix lock

* Try to avoid merge conflicts

* Clean up collection bugs

* remove comments

* Add Postgres configuration settings

* Image

* bad github conflict

* merge (#1278)

* port KG to postgres (#1272)

* create + cluster

* local search

* up

* clean

* format

* basics

* add collection_id and paginate

* rename

* change api

* up

* kg_creation_status

* up

* up

* up

* Feature/cleanup docker (#1279)

* merge

* up

* rm neo4j refs and cleanup docker cmds

* fixup

* up

* up

* cleanup kg migration

* up

* up

* up

* Kg testing (#1280)

* up

* up

* up

* up

* rename

* Patch/cleanup kg migration (#1281)

* cleanup kg migration

* up

* Kg testing (#1280)

* up

* up

* up

* up

* slay neo4j

---------

Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>

* add back poetry lock

* Default Collections (#1282)

* Default collections

* Naughty naughty need to follow the SRP

* project name

* up

* add chunk order

* fragments => extractions

* bug squash

* up

* up

* up

* Testing (#1284)

* CICD

* actions

* poetry

* poetry

* Add env vars

* name

* increase timeout

* add user to collection

* change postgres project name

* Kg testing (#1283)

* up

* up

* cleanup kg migration

* up

* up

* up

* Kg testing (#1280)

* up

* up

* up

* up

* rename

* project name

* up

* add chunk order

* fragments => extractions

* bug squash

* up

* up

* up

* change postgres project name

---------

Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* Feature/fix logic bugs (#1285)

* fixing minor logic bugs in dev branch

* fixing minor logic bugs in dev branch

* merge

* up

* Application docs

* add image (#1287)

* Add version to CLI telemetry (#1288)

* add image

* Add version to cli telemetry

* up

* KG hatchet orchestration (#1286)

* up

* up

* cleanup kg migration

* up

* up

* up

* Kg testing (#1280)

* up

* up

* up

* up

* rename

* project name

* up

* add chunk order

* fragments => extractions

* bug squash

* up

* up

* up

* change postgres project name

* up

* up

---------

Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* Feature/update documentation rebased (#1289)

* up

* merge

* rebase

* fix ingestion issues (#1291)

* fix ingestion issues

* fix lock file

* fix embedding

* Fix SDK KG Serialization (#1292)

* add image

* serialization

* cleanup cli (#1294)

* CLI serialization (#1295)

* add image

* Fix more serialization around kg

* Nolan/schemacreation (#1296)

* add image

* Fix more serialization around kg

* add quotes to prevent reserved keywords from failing

* Prevent errors if config name is reserved name in postgres (#1297)

* Prevent reserved words (#1298)

* default collection ID

* up

* Move default collection id method to utils (#1299)

* up

* Allow json fallback (#1301)

* hotfix: import

* Fix description error (#1302)

* up

* push

* up (#1303)

* up

* up

* up

* up

* minor tweaks

* up

* mypy

* add back missing file

* up

* add way to access entities and relationships

* up

* up

* fix id

* up

* refine end pts

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>

* Add collection (#1317)

* add filters (in progress)

* todo comment

* minor addition

* modify command

* Permission Fixes around Collection Management (#1316)

* Allow super users to update others to super user status

* Fix auth on collections endpoints

* Better error message

* filters (#1318)

* Cost estimate (#1319)

* up

* slightly modify

* up

* minor fix

* docs

---------

Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>
Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
 2024-10-02 18:12:00 -07:00
emrgnt-cmplxty 3721fcb7ad dev (#1269) 
* Feature/encapsulate orchestration (#1265)

* fully encapsulate orchestration

* fully encapsulate orchestration

* complete encapsulation

* revert import cmt

* making default r2r lighter (#1268)

* making default r2r lighter

* fix bug in ingest files

* checkin

* workingupdate

* complete simple orch

* update docs

* up (#1273)

* up

* up

* merge (#1276)

* Postgres configuration settings (#1277)

* Improvements on Auth in JS, CLI (#1267)

* CLI Telemetry (#1266)

* check in

* working

* redundant

* JS auth improvements (#1263)

* Check in JS auth improvements

* Update login with toke

* Fix to allow disabling telemetry

* fix lock

* Try to avoid merge conflicts

* Clean up collection bugs

* remove comments

* Add Postgres configuration settings

* Image

* bad github conflict

* merge (#1278)

* port KG to postgres (#1272)

* create + cluster

* local search

* up

* clean

* format

* basics

* add collection_id and paginate

* rename

* change api

* up

* kg_creation_status

* up

* up

* up

* Feature/cleanup docker (#1279)

* merge

* up

* rm neo4j refs and cleanup docker cmds

* fixup

* Patch/cleanup kg migration (#1281)

* cleanup kg migration

* up

* Kg testing (#1280)

* up

* up

* up

* up

* slay neo4j

---------

Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>

* add back poetry lock

* Default Collections (#1282)

* Default collections

* Naughty naughty need to follow the SRP

* Testing (#1284)

* CICD

* actions

* poetry

* poetry

* Add env vars

* name

* increase timeout

* add user to collection

* Kg testing (#1283)

* up

* up

* cleanup kg migration

* up

* up

* up

* Kg testing (#1280)

* up

* up

* up

* up

* rename

* project name

* up

* add chunk order

* fragments => extractions

* bug squash

* up

* up

* up

* change postgres project name

---------

Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* Feature/fix logic bugs (#1285)

* fixing minor logic bugs in dev branch

* fixing minor logic bugs in dev branch

* merge

* Application docs

* add image (#1287)

* Add version to CLI telemetry (#1288)

* add image

* Add version to cli telemetry

* KG hatchet orchestration (#1286)

* up

* up

* cleanup kg migration

* up

* up

* up

* Kg testing (#1280)

* up

* up

* up

* up

* rename

* project name

* up

* add chunk order

* fragments => extractions

* bug squash

* up

* up

* up

* change postgres project name

* up

* up

---------

Co-authored-by: emrgnt-cmplxty <owen@algofi.org>

* Feature/update documentation rebased (#1289)

* up

* merge

* rebase

* fix ingestion issues (#1291)

* fix ingestion issues

* fix lock file

* fix embedding

* Fix SDK KG Serialization (#1292)

* add image

* serialization

* cleanup cli (#1294)

* CLI serialization (#1295)

* add image

* Fix more serialization around kg

* Nolan/schemacreation (#1296)

* add image

* Fix more serialization around kg

* add quotes to prevent reserved keywords from failing

* Prevent errors if config name is reserved name in postgres (#1297)

* Prevent reserved words (#1298)

* Move default collection id method to utils (#1299)

* Allow json fallback (#1301)

* hotfix: import

* Fix description error (#1302)

* up (#1303)

* rename to `full` (#1304)

* rename to `full`

* add html parser

* Remove postgres vecs variables (#1306)

* Feature/rename ingest files (#1307)

* rename to `full`

* add html parser

* Vec Removal (#1308)

* Remove postgres vecs variables

* up

* change kg settings parsing (#1309)

* offset + limit (#1305)

* offset + limit

* fix order

* update query

* change entity offset

* leiden seed

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>
 2024-10-01 18:43:52 -07:00
emrgnt-cmplxty d84109868a Feature/patch to main (#1194) 
* checkin

* collection to groups

* Fix bug in deletion (#1190)

* fix pr

* fix pr (#1192)

* collect and raise kg extraction errors

* cleanup

* finish mods

* bump pkg

* make logout async

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
 2024-09-17 21:47:57 -07:00
emrgnt-cmplxty ff214da0b4 groups to collections (#1191) 
* checkin

* collection to groups

* Fix bug in deletion (#1190)

* fix pr (#1192)

* Feature/fix collection to groups (#1193)

* fix pr

* collect and raise kg extraction errors

* cleanup

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
 2024-09-17 21:45:32 -07:00
emrgnt-cmplxty 99b7657056 Dev (#614) 
* Optimize Docker build (#609)

* Move to a multi-stage docker build

* 3 more MB

* default to generation config from config.json (#608)

* default to generation config from config.json

* Update r2r/main/api/routes/retrieval.py

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* gen cfg changes

* remove gen_cfg err

* remove reload

* rm collection

* rm cfg class var

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Feature/patch cli r2r bug (#611)

* patch cli missing `config=`

* up

* allow `POSTGRES_URI` to be passed, also allow unix socket connections in old worklfow (#613)

* Feature/patch cli r2r bug (#612)

* patch cli missing `config=`

* up

* Fix version bug in update documents (#615)

* Feature/correctly handle embedding failures rebase (#617)

* add correct embedding failure handling

* fix error logic

* checkin progress

* cleanups

* fix async and bugs

* Feature/modularize ingestion service rebased (#626)

* add correct embedding failure handling

* fix error logic

* checkin progress

* cleanups

* fix async and bugs

* modular ingestion

* extend tests (#627)

* fix ingestion bugs (#628)

* fix ingestion bugs

* cleanup

* Patch/fix circular import (#629)

* fix ingestion bugs

* cleanup

* fix circl imprt

* Patch/fix import bugs (#630)

* fix ingestion bugs

* cleanup

* fix circl imprt

* Patch/fix import bugs (#631)

* fix ingestion bugs

* cleanup

* fix circl imprt

* up (#633)

* Patch/fix exec (#632)

* fix ingestion bugs

* cleanup

* fix circl imprt

* up

* Feature/tweak router logic and logging (#636)

* tweak router logic and add logging

* up

* leave test e2e untouched

* Fix docker in dev branch

* Move to app notation

* fomatting

* Fix client servr logic

* Improve execution logic

* Cleanup

* Fix client-server mode in cli

* Fix upload path

* Clean up tuple

* Update wrapper name

* Fix folder error in execution.py

* Remove superfluous file

* Fix precommit

* Fix test errors

* Remove test artifacts

* check in work

* Add better asynchio support

* don't save artifact

* Change CI python version

* Update version

* Improve async tests

* Patch/fix client side logic for default config (#649)

* make client side play nice w/ default config

* make client side play nice w/ default config

* make client side play nice w/ default config

* make client side play nice w/ default config

* make client side play nice w/ default config

* Patch/minor dev tweak (#651)

* final dev cleanups

* flake

* up

* upd lock (#652)

* Feature/rm cli flag (#653)

* upd lock

* rm flag from cli

* Patch/final dev cleanups (#650)

* final dev cleanups

* flake

---------

Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
 2024-07-09 12:57:10 -07:00
emrgnt-cmplxty d46df23c87 add env example (#516) 2024-06-23 11:21:14 -07:00
emrgnt-cmplxty 78ae0e021c V2 Overhaul (#351) 
* renovate the pipelines

* add storage and do other tweaks

* rn to pipe

* checkin

* add loggable

* implement vector search

* include text

* add fan in / fan out logic

* checkin work

* merge

* add prompt

* checkin

* up

* checkin working rag workflow

* add rag pipe

* tweak

* merge

* working, clean rag pipeline

* up

* tweaks

* up

* add pipeline separation

* rm need for async add to pipeline

* up

* up

* fix logging

* getting ready to blast fan in / fan out

* up

* up

* up

* add tests

* checkin up

* add vecs, extend tests

* up

* add redis logging

* add embedding test

* add vector db tests

* add basic llm response tests

* fix imports

* add first crack streaming

* make stream logic more robust

* add streaming rag test

* fix

* up

* add other

* test server / client workflow

* clean ups

* up

* tweaks

* up

* cleanup

* add movie processing

* fixed tests

* clean

* cleanups

* clean and good

* add eval

* tweak stuff

* add run pipe util

* updates

* add

* refactor

* fix tests

* checkin

* checkin

* commit

* add custom rag pipeline

* tweaks

* update docs

* up

* update readme

* checkin readme

* checkin readme

* checkin readme

* checkin readme

* checkin readme

* checkin readme
 2024-05-22 09:20:48 -07:00
gravitronics 445505c55d Update .env.example (#274) 
reorder examples
 2024-04-04 12:54:01 -07:00
emrgnt-cmplxty 10ec419ff1 ... (#261) 
* fix metadata

* add exa integration
 2024-04-01 18:35:57 -07:00
emrgnt-cmplxty 6e5ca3ea3c Feature/add reducto pipeline rebased (#217) 
* Move to reducto pipeline

* revert run client changes

* format

* tweak

* custom error
 2024-03-26 22:40:08 -07:00
emrgnt-cmplxty ec86378992 Feature/update docs and setup (#93) 
* update docs and setup

* cleanup docs
 2024-02-27 13:08:43 -08:00
emrgnt-cmplxty 52527ac488 Update .env.example 2024-02-27 05:38:07 -08:00
emrgnt-cmplxty eaf843ef6a cleanup docs (#78) 2024-02-26 05:55:58 -08:00
Jan Carbonell 7a31bb523d glossary reorder 2024-02-26 03:08:24 -08:00
emrgnt-cmplxty 58e20e7ca0 Feature/add local db (#65) 
* step towards dependency cleanup

* hammer out kinks

* update README

* cleanup

* add isort cfg

* flake8
 2024-02-25 15:14:48 -08:00
Jan Carbonell d9f5b54dff basic bash config 2024-02-24 13:40:50 -08:00
emrgnt-cmplxty bcf7c8bc18 Feature/add code search (#48) 
* add in error checks

* add logging

* add simple code search ex.
 2024-02-21 21:19:53 -08:00
emrgnt-cmplxty 933bca2afd Feature/tweaks rebased (#25) 
* minor tweaks

* cleanup
 2024-02-17 17:51:28 -08:00
emrgnt-cmplxty 4c28e1f9a2 Feature/add qdrant (#23) 
* add qdrant

* Onboard Qdrant

* flake
 2024-02-17 10:57:34 -08:00
emrgnt-cmplxty 6c261475f0 Feature/rebased (#22) 
* fix formatting

* Add serper

* tweak

* tweak
 2024-02-16 08:41:28 -08:00
emrgnt-cmplxty e665e309e8 add env example (#20) 
* add env example

* fix printout

* tweaks and flow
 2024-02-14 22:34:12 -08:00