* Update OpenAI model info and remove duplicate modelsinfo.py (#4700)
* Update OpenAI model info and remove duplicate modelsinfo.py
* Fix max_tokens for gpt-4-0613
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
* Update count_message_tokens to support new OpenAI models
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
* Fix error message in count_message_tokens
---------
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Erik Peterson <e@eriklp.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Further changes:
* remove `init` param from `get_memory()`, replace usages by `memory.clear()`
* make token length calculation optional in `MemoryItem.dump()`
* Extract open ai api calls and retry at lowest level
* Forgot a test
* Gotta fix my local docker config so I can let pre-commit hooks run, ugh
* fix: merge artiface
* Fix linting
* Update memory.vector.utils
* feat: make sure resp exists
* fix: raise error message if created
* feat: rename file
* fix: partial test fix
* fix: update comments
* fix: linting
* fix: remove broken test
* fix: require a model to exist
* fix: BaseError issue
* fix: runtime error
* Fix mock response in test_make_agent
* add 429 as errors to retry
---------
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
Co-authored-by: Luke K (pr-0f3t) <2609441+lc0rp@users.noreply.github.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
* Correct and clean up JSON handling
* Use ast for message history too
* Lint
* Add comments explaining why we use literal_eval
* Add descriptions to llm_response_format schema
* Parse responses in code blocks
* Be more careful when parsing in code blocks
* Lint
* Implement Batch Running Summarization to avoid max token error (#4652)
* Fix extra space in prompt
---------
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
* Add config as attribute to Agent, rename old config to ai_config
* Code review: Pass ai_config
---------
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
* Add settings for custom base url and embedding dimension
Making the openai base url and embedding dimension configurable, these are useful to integrate AutoGPT with other models, like LLaMA
* Update to milvus.py to load the configuration also in the init_collection function
* Update radismem.py to get rid of Config() loading
* Update local.py to get rid of Config() loading
* Correct code format (python black)
* Revert DEFAULT_EMBED_DIM name to EMBED_DIM to keep tests valid
* Better description for EMBED_DIM setting
* Set MockConfig to the type Config in Milvus test
* Fix formatting
* Update Milvus test, using Config() instead of building a mock config
* using the last milvus test code from main
* Remove embed_dim , no more needed after #4208
* Add example for OPENAI_BASE_URL
---------
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
* feat: set max token limits for better user experience
* fix: use OPEN_AI_CHAT_MODELS max limits
* fix: use the old default of 8000
* fix: formatting so isort/black checks pass
* fix: avoid circular dependencies
* fix: use better to avoid circular imports
* feat: introduce soft limits and use them
* fix: circular import issue and missing field
* fix: move import to avoid overriding doc comment
* feat: DRY things up and set token limit for fast llm models too
* tests: make linter tests happy
* test: use the max token limits in config.py test
* fix: remove fast token limit from config
* feat: remove smart token limit from config
* fix: remove unused soft_token_limit var
* fix: remove unneeded tests, settings aren't in config anymore
---------
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
* Fix issues with file reading and writing with Python code
* Change error message, use Workspace.get_path
---------
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
* Rearrange tests into unit/integration/challenge categories
* Fix linting + `tests.challenges` imports
* Fix obscured duplicate test in test_url_validation.py
* Move VCR conftest to tests.vcr
* Specify tests to run & their order (unit -> integration -> challenges) in CI
* Fail Docker CI when tests fail
* Fix import & linting errors in tests
* Fix `get_text_summary`
* Fix linting errors
* Clean up pytest args in CI
* Remove bogus tests from GoCodeo
