Commit Graph

514 Commits

Author SHA1 Message Date
merwanehamadi
0abfa3a68f Remove config from command decorator (#4736) 2023-06-18 20:00:23 -07:00
merwanehamadi
a7f805604c Pass config everywhere in order to get rid of singleton (#4666)
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-06-18 19:05:41 -07:00
Erik Peterson
096d27f342 Fix execute_command coming from plugins (#4729)
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-06-17 14:53:58 -07:00
merwanehamadi
10d7747ae2 Use JSON format for commands signature (#4714)
* Use JSON for command signature

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Improve plugin backward compatibility (#4716)

* Fixed plugin test

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Fix Docker-CI

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Put back commands, clean typing and signatures

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

---------

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Erik Peterson <e@eriklp.com>
Co-authored-by: Luke K (pr-0f3t) <2609441+lc0rp@users.noreply.github.com>
2023-06-17 08:39:17 -07:00
merwanehamadi
0b6fec4a28 Fix summarization happening in first cycle (#4719) 2023-06-16 18:17:47 -07:00
merwanehamadi
d923004e20 Remove app commands, audio text and playwright (#4711)
* Remove App Commands and Audio Text
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

* Remove self feedback

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

---------

Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Erik Peterson <e@eriklp.com>
2023-06-15 15:04:51 -07:00
merwanehamadi
a30e5a85b2 Remove write_tests command (#4707)
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Erik Peterson <e@eriklp.com>
2023-06-15 13:32:20 -07:00
merwanehamadi
512d7ba208 Remove analyze_code (#4705)
Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>
Co-authored-by: Erik Peterson <e@eriklp.com>
2023-06-15 12:55:50 -07:00
Erik Peterson
195a7fcad8 Refactor module layout of command classes (#4706) 2023-06-15 11:34:41 -07:00
Reinier van der Leer
f0a5250da5 Implement loading MemoryItems from file in JSONFileMemory (#4703)
Further changes:
* remove `init` param from `get_memory()`, replace usages by `memory.clear()`
* make token length calculation optional in `MemoryItem.dump()`
2023-06-15 17:45:14 +02:00
James Collins
6e6e7fcc9a Extract openai API calls and retry at lowest level (#3696)
* Extract open ai api calls and retry at lowest level

* Forgot a test

* Gotta fix my local docker config so I can let pre-commit hooks run, ugh

* fix: merge artiface

* Fix linting

* Update memory.vector.utils

* feat: make sure resp exists

* fix: raise error message if created

* feat: rename file

* fix: partial test fix

* fix: update comments

* fix: linting

* fix: remove broken test

* fix: require a model to exist

* fix: BaseError issue

* fix: runtime error

* Fix mock response in test_make_agent

* add 429 as errors to retry

---------

Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
Co-authored-by: Luke K (pr-0f3t) <2609441+lc0rp@users.noreply.github.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-06-14 07:59:26 -07:00
Erik Peterson
49d1a5a17b Rework plugin config to be file-based (#4673) 2023-06-13 20:54:55 -07:00
Erik Peterson
07d9b584f7 Correct and clean up JSON handling (#4655)
* Correct and clean up JSON handling

* Use ast for message history too

* Lint

* Add comments explaining why we use literal_eval

* Add descriptions to llm_response_format schema

* Parse responses in code blocks

* Be more careful when parsing in code blocks

* Lint
2023-06-13 09:54:50 -07:00
Kinance
7bf39cbb72 Include the token length of the current summary (#4670)
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-12 16:29:11 -07:00
Reinier van der Leer
f277e6aa2d Strip spaces from the end of truncated ai_name in log_cycle.py (#4672) 2023-06-12 15:14:15 -07:00
merwanehamadi
d5afbbee26 Add challenge name and level to pytest logs (#4661) 2023-06-12 08:03:14 -07:00
Reinier van der Leer
a9d177eeeb Remove unused function split_file from file_operations.py (#4658) 2023-06-12 02:20:39 +02:00
Kinance
ff46c16805 Remove extra spaces in summarization prompt (#4660)
* Implement Batch Running Summarization to avoid max token error (#4652)

* Fix extra space in prompt

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-12 02:13:47 +02:00
Erik Peterson
fd04db12fa Use prompt_toolkit to enable keyboard navigation in CLI (#4649)
* Use prompt_toolkit to enable keyboard navigation in CLI

* Also update other tests

---------

Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-11 14:19:42 -07:00
Kinance
bc5dbb6692 Implement Batch Summarization in MessageHistory Class to manage context length under model's token limit (#4652)
* Implement Batch Running Summarization to avoid max token error

* Rename test func
2023-06-11 13:04:41 -07:00
Erik Peterson
0594ba33a2 Pass agent to commands instead of config (#4645)
* Add config as attribute to Agent, rename old config to ai_config

* Code review: Pass ai_config

* Pass agent to commands instead of config

* Lint

* Fix merge error

* Fix memory challenge a

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-10 15:48:50 -07:00
Erik Peterson
6b9e3b21d3 Add config as attribute to Agent, rename old config to ai_config (#4638)
* Add config as attribute to Agent, rename old config to ai_config

* Code review: Pass ai_config

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-10 14:47:26 -07:00
Erik Peterson
15c6b0c1c3 Implement directory-based plugin system (#4548)
* Implement directory-based plugin system

* Fix Selenium test

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com>
2023-06-10 13:16:00 -07:00
DGdev91
6ff8478118 Add settings for custom base url (#2594)
* Add settings for custom base url and embedding dimension

Making the openai base url and embedding dimension configurable, these are useful to integrate AutoGPT with other models, like LLaMA

* Update to milvus.py to load the configuration also in the init_collection function

* Update radismem.py to get rid of Config() loading

* Update local.py to get rid of Config() loading

* Correct code format (python black)

* Revert DEFAULT_EMBED_DIM name to EMBED_DIM to keep tests  valid

* Better description for EMBED_DIM setting

* Set MockConfig to the type Config in Milvus test

* Fix formatting

* Update Milvus test, using Config() instead of building a mock config

* using the last milvus test code from main

* Remove embed_dim , no more needed after #4208

* Add example for OPENAI_BASE_URL

---------

Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-10 13:57:42 +02:00
Erik Peterson
fdca233fe3 Fix prompt issue causing 'No Command' issues and challenge to fail (#4623)
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-09 17:25:03 -07:00
Erik Peterson
5fe600af9d Clean up and fix issues with env configuration and .env.template (#4630)
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-09 15:28:30 -07:00
Erik Peterson
ff4e53d0e6 Streamline / clarify shell command control configuration (#4628)
* Streamline / clarify shell command control configuration

* Fix lint
2023-06-09 11:48:20 -07:00
Erik Peterson
94280b2d14 Add command for directly executing python code (#4581)
* Add command for directly executing python code

* Fix docstring

* Clarify / update filename references

---------

Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
2023-06-09 10:40:32 -07:00
Stefan Ayala
1e851ba3ea Feat set token limits based on model (#4498)
* feat: set max token limits for better user experience

* fix: use OPEN_AI_CHAT_MODELS max limits

* fix: use the old default of 8000

* fix: formatting so isort/black checks pass

* fix: avoid circular dependencies

* fix: use better to avoid circular imports

* feat: introduce soft limits and use them

* fix: circular import issue and missing field

* fix: move import to avoid overriding doc comment

* feat: DRY things up and set token limit for fast llm models too

* tests: make linter tests happy

* test: use the max token limits in config.py test

* fix: remove fast token limit from config

* feat: remove smart token limit from config

* fix: remove unused soft_token_limit var

* fix: remove unneeded tests, settings aren't in config anymore

---------

Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-07 10:16:53 +02:00
Marc Bornträger
c9f1f050af feat: log new-line so user can differentiate sections better in console (#1569)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Nicholas Tindle <nicktindle@outlook.com>
2023-06-07 00:32:43 -05:00
Erik Peterson
fdc6e12945 Improve logic and error messages for file reading and writing with Python code (#4567)
* Fix issues with file reading and writing with Python code

* Change error message, use Workspace.get_path

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-07 02:54:02 +02:00
merwanehamadi
1b04e5cafc Fix duckduckgo rate limiting (#4592)
* Fix duckduckgo rate limiting

* use list instead of loop

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-07 01:20:52 +02:00
Erik Peterson
055806e124 Fix inverted logic for deny_command (#4563) 2023-06-06 22:56:17 +02:00
Reinier van der Leer
dafbd11686 Rearrange tests & fix CI (#4596)
* Rearrange tests into unit/integration/challenge categories

* Fix linting + `tests.challenges` imports

* Fix obscured duplicate test in test_url_validation.py

* Move VCR conftest to tests.vcr

* Specify tests to run & their order (unit -> integration -> challenges) in CI

* Fail Docker CI when tests fail

* Fix import & linting errors in tests

* Fix `get_text_summary`

* Fix linting errors

* Clean up pytest args in CI

* Remove bogus tests from GoCodeo
2023-06-06 10:48:49 -07:00
Benny van der Lans
74e8a886e6 Add replace_in_file command (#4565)
Resubmission of #3643

---------

Co-authored-by: Reinier van der Leer <github@pwuts.nl>
2023-06-04 19:37:35 +02:00
kaneda2004
63b79a88c6 Adding support for openai_organization env variable (#289) 2023-05-30 19:02:16 -07:00
merwanehamadi
87776b2886 Make the information retrieval challenge a harder while still passing (#4468) 2023-05-30 15:56:58 -07:00
Luke K (pr-0f3t)
1446ffddb0 Fix #942: Respect --gpt4only, --gpt3only cli args (#3144)
Co-authored-by: Luke K <KayLuke@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
2023-05-29 20:10:51 +03:00
merwanehamadi
ee9f10a8d8 remove unused imports automatically (#4449)
* remove unused imports automatically

* add linters to pr template

* remove useless try statement
2023-05-28 05:50:50 -07:00
Ryan Johns
03036c1bd6 Added three more tests to check for edge cases in URL validation (#4441)
Co-authored-by: Ryan Johns <rkjohns@verisk.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
2023-05-27 18:58:38 +03:00
k-boikov
064e95b46b Allow spinning to be disabled (#4329) 2023-05-27 00:01:34 +03:00
merwanehamadi
e7c0d3330c Create cassette submodule (#4420)
* gfeat: specify directory of cassettes and automatically load them depending on module

fix: formatting for linter

test: commit newly generated cassettes to their respective folder

tests: update latest fixtures with master

fix: update .gitattributes with updated path to cassettes

fix: use cassettes from master instead of generating them myself

fix: update path in .gitattributes

fix: make sure to match default functionality by using test name for cassette directory

fix: actually add git submodule

ci: checkout git submodules in CI

ci: update git submodules separately to ensure it gets called

feat: add a hooks directory so we can update git submodules on post-checkout

feat: make sure we push the tests/cassettes submodule on merge into master

ci: remove unused code now that we are using git submodules to keep cassettes in sync

fix: simplify how we load the submodule and fix updating cassettes on merge to master

chore: remove echo of checkout hook, it's unneeded

ci: remove unneccesary step

* cassettes submodule

* cassettes submodule

* cassettes submodule

* cassettes submodule

* cassettes submodule

---------

Co-authored-by: Stefan Ayala <stefanayala3266@gmail.com>
2023-05-26 12:33:49 -07:00
Nicholas Tindle
acfd966aa4 Pass Configs to Commands and remove CFG = Config() in the commands/ folder (#4328)
* feat: pass config to call_ai_functions in coimmands

* feat: config for read_audio_from_file

* feat: file operations cfg

NOTE: we replaced the CFG in the command enable with TRUE b/c not sure how to handle this yet

* feat: git command conversion

* feat: google search

* feat: image generation

* feat: extract cfg from browser commands

* feat: remove cfg from execute code commands

* fix: file operation related tests

* fix: linting

* fix: tests for read_audio

* fix: test error

* feat: update cassettes

* fix: linting

* fix: test typechecking

* fix: google_search errors if unexpected kw arg is passed

* fix: pass config param to google search test

* fix: agent commands were broken + cassettes

* fix: agent test

* feat: cassettes

* feat: enable/disable logic for commands

* fix: some commands threw errors

* feat: fix tests

* Add new cassettes

* Add new cassettes

* ci: trigger ci

* Update autogpt/commands/execute_code.py

Co-authored-by: Reinier van der Leer <github@pwuts.nl>

* fix prompt

* fix prompt + rebase

* add config remove useless imports

* put back CFG just for download file

* lint

* The signature should be mandatory in the decorator

* black isort

* fix: remove the CFG

* fix: non typed arg

* lint: type some args

* lint: add types for libraries

* Add new cassettes

* fix: windows compatibility

* fix: add config access to decorator

* fix: remove twitter mention

* DDGS search works at 3.0.2 version

* ci: linting

---------

Co-authored-by: Auto-GPT-Bot <github-bot@agpt.co>
Co-authored-by: merwanehamadi <merwanehamadi@gmail.com>
Co-authored-by: Reinier van der Leer <github@pwuts.nl>
Co-authored-by: kinance <kinance@gmail.com>
2023-05-26 08:39:25 -07:00
k-boikov
f07fcdf0a7 Pull auto-gpt-plugin-template from GitHub instead of PyPI (#4402)
* switch from pypi to github for auto-gpt-plugin-template

* adapt tests to new plugin interface
2023-05-26 14:48:36 +02:00
Reinier van der Leer
bfbe613960 Vector memory revamp (part 1: refactoring) (#4208)
Additional changes:

* Improve typing

* Modularize message history memory & fix/refactor lots of things

* Fix summarization

* Move memory relevance calculation to MemoryItem & improve test

* Fix import warnings in web_selenium.py

* Remove `memory_add` ghost command

* Implement overlap in `split_text`

* Move memory tests into subdirectory

* Remove deprecated `get_ada_embedding()` and helpers

* Fix used token calculation in `chat_with_ai`

* Replace Message TypedDict by dataclass

* Fix AgentManager singleton issues in tests

---------

Co-authored-by: Auto-GPT-Bot <github-bot@agpt.co>
2023-05-25 20:31:11 +02:00
Nicholas Tindle
43565b280f Remove Twitter Command (#4381) 2023-05-24 22:40:30 -05:00
Kinance
5168cb5219 Migrate google search to use DDGS.text function (#4383) 2023-05-24 19:36:23 +03:00
Sicong Zhao
a8af3313c1 Update get_hyperlinks description (#4122)
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: Auto-GPT-Bot <github-bot@agpt.co>
2023-05-23 16:47:58 +03:00
Robin Richtsfeld
77ee9f8119 Pass command line args as list (#1486)
Co-authored-by: Nicholas Tindle <nick@ntindle.com>
Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com>
Co-authored-by: Richard Beales <rich@richbeales.net>
2023-05-22 22:53:31 +03:00
k-boikov
360d5cd577 Prevent docker compose to break config by creating folders (#4125) 2023-05-22 13:36:20 +03:00