Commit Graph

5367 Commits

Author SHA1 Message Date
Reinier van der Leer
0df2199c42 feat(autogpt_server): Add GET /graphs/{graph_id}/executions endpoint (#7330)
* feat(autogpt_builder): Add `AutoGPTServerAPI` client

* migrate API calls in Flow.tsx to new API client

* feat(autogpt_server): Add `/graphs/{graph_id}/executions` endpoint

In `data/execution.py`:
- Add `list_executions` function
- Rename `get_executions` to `get_execution_results`

In `server/server.py`:
- Add route
- Add `AgentServer.list_graph_runs`
- Rename `AgentServer.get_executions` to `get_run_execution_results`

* feat(autogpt_builder): Add `listFlowRunIDs` endpoint to `AutoGPTServerAPI` client

* Move `Schema` to `types.ts` and rename to `ObjectSchema`
2024-07-05 10:51:27 +01:00
Reinier van der Leer
200800312a feat(autogpt_builder): Add AutoGPTServerAPI client (#7328)
- Add fully typed `AutoGPTServerAPI` client in lib/autogpt_server_api.ts
- Migrate API calls in `Flow.tsx` to new API client
2024-07-05 11:03:59 +02:00
Reinier van der Leer
b7a90ce768 fix(autogpt_builder): Unignore lib/ and add lib/utils.ts 2024-07-04 17:40:59 -06:00
Reinier van der Leer
f359ed0983 feat(agent_builder): Add shad/cn UI library (#7316)
- Add shad/cn + Radix Icons + Tailwind
- move favicon.ico to static folder
- delete empty custominput.css

---------

Co-authored-by: Bentlybro <tomnoon9@gmail.com>
2024-07-05 08:14:45 +02:00
Aarushi
6456285753 feat(autogpt_builder) Update custom node to handle deeply nested structures (#7319)
* feat(rnd): Add type hint and strong pydantic type validation for block input/output + add reddit agent-blocks.

* feat(rnd): Add type hint and strong pydantic type validation for block input/output + add reddit agent-blocks.

* Fix reddit block

* Fix serialization

* Eliminate deprecated class property

* Remove RedditCredentialsBlock

* Cache jsonschema computation, add dictionary construction

* Add dict_split and list_split to output, add more blocks

* Add objc_split for completeness, int both input and output

* Update reddit block

* Add reddit test (untested)

* Resolved json issue on pydantic

* Add creds check on client

* Add dict <--> pydantic object flexibility

* Fix error retry

* Skip reddit test

* Code cleanup

* Chang prompt

* Make this work

* Fix linting

* Hide input_links and output_links from Node

* Add docs

* updating UI to handle deeply nested data structures for reddit usecase

* changing expected key in reddit post to comment

---------

Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2024-07-04 17:54:41 +01:00
Zamil Majdy
833944e228 feat(rnd): Add strong pydantic type & composite data extraction for Block input/output schema + add reddit agent-blocks (#7288)
* feat(rnd): Add type hint and strong pydantic type validation for block input/output + add reddit agent-blocks.

* feat(rnd): Add type hint and strong pydantic type validation for block input/output + add reddit agent-blocks.

* Fix reddit block

* Fix serialization

* Eliminate deprecated class property

* Remove RedditCredentialsBlock

* Cache jsonschema computation, add dictionary construction

* Add dict_split and list_split to output, add more blocks

* Add objc_split for completeness, int both input and output

* Update reddit block

* Add reddit test (untested)

* Resolved json issue on pydantic

* Add creds check on client

* Add dict <--> pydantic object flexibility

* Fix error retry

* Skip reddit test

* Code cleanup

* Chang prompt

* Make this work

* Fix linting

* Hide input_links and output_links from Node

* Add docs

---------

Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
2024-07-04 11:37:28 +01:00
Reinier van der Leer
db0e726954 fix(agent, benchmark): Specify path_type=Path for CLI path options/arguments
Without `path_type=Path`, an option/argument with `type=click.Path()` will return a `str`.
2024-07-03 15:21:04 -06:00
Krzysztof Czerwinski
08612cc3bf refactor(agent, forge): Move tests from autogpt to forge (#7247)
- Move `autogpt/tests/vcr_cassettes` submodule to `forge/tests/vcr_cassettes`
- Remove not needed markers from `pyproject.toml`: `"requires_openai_api_key", "requires_huggingface_api_key"`
- Update relevant GitHub workflows

Moved relevant tests from `autogpt/tests` to appropiate directories:
- Component tests to their respective component dirs
- `autogpt/tests/unit/test_web_search.py` → `forge/components/web/test_search.py`
- `autogpt/tests/unit/test_git_commands.py` → `forge/components/git_operations/test_git_operations.py`
- `autogpt/tests/unit/test_file_operations.py` → `forge/components/file_manager/test_file_manager.py`
- `autogpt/tests/integration/test_image_gen.py` → `forge/components/image_gen/test_image_gen.py`
- `autogpt/tests/integration/test_web_selenium.py` → `forge/components/web/test_selenium.py`
- `autogpt/tests/integration/test_execute_code.py` → `forge/components/code_executor/test_code_executor.py`
- `autogpt/tests/unit/test_s3_file_storage.py` → `forge/file_storage/test_s3_file_storage.py`
- `autogpt/tests/unit/test_gcs_file_storage.py` → `forge/file_storage/test_gcs_file_storage.py`
- `autogpt/tests/unit/test_local_file_storage.py` → `forge/file_storage/test_local_file_storage.py`
- `autogpt/tests/unit/test_json.py` → `forge/json/test_parsing.py`
- `autogpt/tests/unit/test_logs.py` → `forge/logging/test_utils.py`
- `autogpt/tests/unit/test_url_validation.py` → `forge/utils/test_url_validator.py`
- `autogpt/tests/unit/test_text_file_parsers.py` → `forge/utils/test_file_operations.py`

- (Re)moved dependencies from `autogpt/pyproject.toml` that were only used in these test files.

Also:
- Added `load_env_vars` fixture to `forge/conftest.py`
- Fixed a type error in `forge/components/web/test_search.py`
- Merged `autogpt/.gitattributes` into root `.gitattributes`

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2024-07-04 02:09:01 +02:00
Reinier van der Leer
7415e24fc3 fix(forge): Add google-api-python-client-stubs and fix type error (#7303)
- Add `google-api-python-client-stubs` dev dependency
- Add version specification to `google-api-python-client` dependency
- Fix type error (by ignoring it) in forge/components/web/search.py
2024-07-03 23:26:37 +02:00
Dimitri Brooks
ecb054af56 feat: Add support for Claude 3.5 Sonnet (#7301)
add support for Claude 3.5 Sonnet
2024-07-03 16:02:14 +02:00
Krzysztof Czerwinski
39f70b0c83 docs(agent, forge): Update component configuration docs (#7232) 2024-07-03 09:50:38 +01:00
Krzysztof Czerwinski
7cb4d4a903 feat(forge, agent, benchmark): Upgrade to Pydantic v2 (#7280)
Update Pydantic dependency of `autogpt`, `forge` and `benchmark` to `^2.7`
[Pydantic Migration Guide](https://docs.pydantic.dev/2.7/migration/)

- Migrate usages of now-deprecated functions to their replacements
- Update `Field` definitions
  - Ellipsis `...` for required fields is deprecated
  - `Field` no longer supports extra `kwargs`, replace use of this feature with field metadata
- Replace `Config` class for specifying model configuration with `model_config = ConfigDict(..)`
- Removed `ModelContainer` in `BaseAgent`, component configuration dict is now directly serialized using Pydantic v2 helper functions
- Forked `agent-protocol` and updated `packages/client/python` for Pydantic v2 support: https://github.com/Significant-Gravitas/agent-protocol

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2024-07-02 20:45:32 +02:00
Aarushi
8feaced92e (rnd) Add support for multiple, dynamic inputs (#7296)
multi dynamic inputs
2024-07-02 09:56:33 +01:00
Reinier van der Leer
97e4cceb94 feat(agent, forge): Markdown-formatted history -> message history (#7228)
- Implement message based history in `ActionHistoryComponent`
- Make non-summarized message count configurable (`ActionHistoryComponent.full_message_count`)
- Run `ActionHistoryComponent` after `SystemComponent` so that history messages are last in the prompt
- Omit final instruction message if prompt already contains assistant messages
- Filter `raw_message` from `ActionProposal.schema()`

---------

Co-authored-by: Krzysztof Czerwinski <kpczerwinski@gmail.com>
2024-07-02 03:47:55 +02:00
Reinier van der Leer
2fa4fd23af fix(agent): Make build dependencies optional to unbreak install (#7298)
* Create optional `build` dependency group
* Move `cx-freeze` dependency to `build` dependency group

To include the `build` group when installing dependencies, run `poetry install --with=build`.

Fixes #7297 (`cx-freeze` dependency install fails after #7271)
2024-07-02 02:08:02 +02:00
angeousta
976ea7cd3c fix readme (broken link) (#7293)
Update README.md
2024-07-01 09:44:46 +02:00
Bently
d5ab83aa34 Updating api calls in AutoGPT builder (#7275)
* update api endpoints

* get multi-node working + fix node output

* updated multi-node running + re-add "wire" on rebuild

* Fix node data mapping

* removed getStatusValue
2024-06-28 10:39:26 +01:00
Reinier van der Leer
cbae8b5c14 chore(agent, forge, benchmark): Clean up dependencies (#7286)
* Remove unused dependencies
* Move dependencies for moved code from `autogpt` to `forge`
* Loosen dependency for `uvicorn` to improve compatibility
2024-06-28 02:21:36 +02:00
Aarushi
854080f7af ci(builder): Add CI for AutoGPT Builder (#7267)
Add a simple Node/NPM linting workflow for `rnd/autogpt_builder/`

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2024-06-28 01:30:34 +02:00
Reinier van der Leer
fbb3891e79 chore(forge, agent, benchmark): Update pytest-asyncio to v0.23.x
Resolves #7283
2024-06-27 14:09:36 -06:00
Reinier van der Leer
4d8ee65ca7 fix(pre-commit): Add benchmark/poetry.lock to "Typecheck - Benchmark" trigger file pattern 2024-06-27 14:05:12 -06:00
Aarushi
6093acc813 Add more functionality to Nodes (#7278)
updating node behaviour
2024-06-27 17:03:10 +01:00
Nicholas Tindle
785a40ff9d feat(server, autogpt): Add Example files and update build option (#7271) 2024-06-27 09:56:21 -05:00
Zamil Majdy
2bc22c5450 feat(rnd): Add support for dynamic input as list for AgentServer Block (#7268)
On AgentServer, To create a Block like StringFormatterBlock or LllmCallBlock, we need some way to dynamically link input pins and aggregate them into a single list input. This will give a better experience for the user to construct an input and link it from the output of the other nodes. The scope of this change is adding support for that in the least intrusive way.

Proposal
To differentiate the input list name and its singular entry we are using the $_<index> prefix. For example:
For the input items: list[int], you can set a pin items with values like [1,2,3,4]. But you can also add input pins like items_$_0 or items_$_4 with values 1 or 2, which will be appended to the items input in alphabetical order.
The execution engine will guarantee to wait for the execution until all the input pin value is produced, so input pin with list input will produce fix-sized list.
2024-06-27 21:51:34 +07:00
Aarushi
cdc658695f Add reactflow component in AutoGPT builder (#7270)
* Getting started with nextjs

* fix linting

* remove gitignore for package.json

* pulling in reactflow components

* updating css

* use environment variables

* clean up css / ui a lil

* Fixed nodes/run button animation

so they are always visible

---------

Co-authored-by: Bentlybro <tomnoon9@gmail.com>
2024-06-27 10:14:25 +01:00
Aarushi
dd960f9306 Add support for nextjs based app (#7266)
* Getting started with nextjs

* fix linting

* remove gitignore for package.json
2024-06-27 10:02:54 +01:00
Krzysztof Czerwinski
6e1c9d44a4 Update Agent Server README.md 2024-06-26 19:41:12 +02:00
Zamil Majdy
26bcb26bb7 feat(rnd): Refactor AgentServer Node Input/Output Relation & Block output interface (#7231)
### Background

The current implementation of AgentServer doesn't allow for a single pin to be connected to multiple nodes, this will be problematic when you have a single output node that needs to be propagated into many nodes. Or multiple nodes that possibly feed the data into a single pin (first come first serve).

This infra change is also part of the preparation for changing the `block` interface to return a stream of output instead of a single output.  Treating blocks as streams requires this capability.

### Changes 🏗️

* Update block run interface from returning `(output_name, output_data)` to `Generator[(output_name, output_data)]`
* Removed `agent` term in the API, replace it with `graph` for consistency.
* Reintroduced `AgentNodeExecutionInputOutput`. `AgentNodeExecution` input & output will be a list of `AgentNodeExecutionInputOutput` which describes the input & output data of its execution. Making an execution has 1-many relation to its input output data.
* Propagating the relation and block interface change into the execution engine.
2024-06-26 12:41:55 +02:00
Nicholas Tindle
f04ddceacf docs(forge): Update and rename QUICKSTART.md to FORGE-QUICKSTART.md (#7215) 2024-06-25 19:03:05 -05:00
Bently
3e01b19d6f chore(forge): Update duckduckgo-search to v6.1.7 (#7254)
This should (for now) mitigate the RateLimitErrors that people have been experiencing.
---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2024-06-25 03:24:10 +02:00
Zamil Majdy
9f1e521857 feat(rnd): Add AutoGPT server scheduling service (#7226)
### Background

Agent execution should be able to be triggered in a recurring manner.
This PR introduced an ExecutionScheduling service, a process responsible for managing the execution schedule and triggering its execution based on a predefined cron expression.

### Changes 🏗️

* Added `scheduler.py` / `ExecutionScheduler` implementation.
* Added scheduler test.
* Added `AgentExecutionSchedule` table and its logical model & prisma queries.
* Moved `add_execution` from API server to `execution_manager`
2024-06-24 09:41:02 +07:00
Zamil Majdy
d9226888b2 feat(rnd): Add node metadata on Agent Server Node object (#7237) 2024-06-21 17:50:50 +01:00
Zamil Majdy
210d7738b9 feat(rnd): Add IPC support on autogpt_server (#7212)
### Background

This PR adds support on IPC on autogpt_server.
To make this happen, there are a couple of refactoring efforts being made (will be described in the `Changes` section).
Currently, there are three independent processes:

```
AgentServer ----> ExecutionManager
    | 
     --> ExecutionScheduler
```


### Changes 🏗️

* Added Pyro5 for IPC support.
* Introduced `AppService`: a class to construct an independent process that can expose a method to other running processes (this is analogous to a microservice).
* Introduced `AppProcess`: used by `AppService` a class for creating a child process that can be executed in the background.
* Adapting existing codebase to user `AppService`.
2024-06-19 22:49:47 +07:00
Krzysztof Czerwinski
c19ab2b24f feat(forge): Component-specific configuration (#7170)
Remove many env vars and use component-level configuration that could be loaded from file instead.

### Changed

- `BaseAgent` provides `serialize_configs` and `deserialize_configs` that can save and load all component configuration as json `str`. Deserialized components/values overwrite existing values, so not all values need to be present in the serialized config.
- Decoupled `forge/content_processing/text.py` from `Config`
- Kept `execute_local_commands` in `Config` because it's needed to know if OS info should be included in the prompt
- Updated docs to reflect changes
- Renamed `Config` to `AppConfig`

### Added

- Added `ConfigurableComponent` class for components and following configs:
  - `ActionHistoryConfiguration`
  - `CodeExecutorConfiguration`
  - `FileManagerConfiguration` - now file manager allows to have multiple agents using the same workspace
  - `GitOperationsConfiguration`
  - `ImageGeneratorConfiguration`
  - `WebSearchConfiguration`
  - `WebSeleniumConfiguration`
- `BaseConfig` in `forge` and moved `Config` (now inherits from `BaseConfig`) back to `autogpt`
- Required `config_class` attribute for the `ConfigurableComponent` class that should be set to configuration class for a component
`--component-config-file` CLI option and `COMPONENT_CONFIG_FILE` env var and field in `Config`. This option allows to load configuration from a specific file, CLI option takes precedence over env var.
- Added comments to config models

### Removed

- Unused `change_agent_id` method from `FileManagerComponent`
- Unused `allow_downloads` from `Config` and CLI options (it should be in web component config if needed)
- CLI option `--browser-name` (the option is inside `WebSeleniumConfiguration`)
- Unused `workspace_directory` from CLI options
- No longer needed variables from `Config` and docs
- Unused fields from `Config`: `image_size`, `audio_to_text_provider`, `huggingface_audio_to_text_model`
- Removed `files` and `workspace` class attributes from `FileManagerComponent`
2024-06-19 09:14:01 +01:00
Reinier van der Leer
02dc198a9f fix(agent): Fix resumption from mid-cycle state in CLI mode (#7224)
When an agent is resumed from a mid-cycle state (having made a proposal but not executed it yet), we need to use the previously determined `current_episode.action` proposal instead of calling `agent.propose_action()` again.
2024-06-18 19:30:24 +02:00
Reinier van der Leer
227cf41612 fix(agent, forge): Make LLM API key check provider-agnostic (#7220)
* Rename `assert_config_has_openai_api_key` to `assert_config_has_required_llm_api_keys`
* Make OpenAI credential check conditional (only if an OpenAI model is selected in the config)
* Implement checks for Groq and Anthropic credentials
* Use API calls for Groq and OpenAI credential checks to make sure the keys are valid
2024-06-14 20:56:03 +02:00
Reinier van der Leer
66f373fb57 docs: Update CONTRIBUTING.md
- Add reference to Code Guidelines
- Reorder list of directives to more logical order
2024-06-14 07:45:14 -07:00
Nicholas Tindle
8f3ed733b9 chore(rnd): Fix warning during server tests (#7213) 2024-06-14 06:56:53 -05:00
Krzysztof Czerwinski
9f71cd2437 feat(forge): Unbreak forge agent (#7196)
Revert some changes to fix forge agent and enable components support.
- Rename forge `Agent` to `ProtocolAgent`
- Bring back and update `forge/app.py` and `forge/agent/forge_agent.py`
- `ForgeAgent` inherits from `BaseAgent`, supports component execution and runs the same pipelines as autogpt Agent
- Update forge version from 0.1.0 to 0.2.0
- Update code comments
2024-06-12 13:45:00 +01:00
Nicholas Tindle
6ec708c771 ci(server): Agent Server CI (#7193) 2024-06-12 00:29:23 +07:00
Zamil Majdy
af3febd79f feat(rnd): Add missing code on execution logic for AutoGPT Server (#7209)
Add missing changes from previous commit
2024-06-11 16:15:52 +02:00
SwiftyOS
60e0d4c530 fix(rnd): fixed cli repeated star cmd quality of life 2024-06-11 13:10:53 +02:00
SwiftyOS
4aeae53a61 fixed typo 2024-06-11 12:29:13 +02:00
SwiftyOS
d5c4eca739 Check if pid exists 2024-06-11 12:27:19 +02:00
SwiftyOS
1e4ef7b313 chore(benchmark): delete notebooks 2024-06-11 11:30:46 +02:00
Swifty
fd18877dae feat(rnd): CLI to Run and Stop the Server (#7195) 2024-06-11 11:21:34 +02:00
Nicholas Tindle
c754ecd5d1 fix(docs): bad link from Options to built in components config (#7207) 2024-06-10 17:57:29 +01:00
Zamil Majdy
e688cc31f0 feat(rnd): Implement agent block execution logic for AutoGPT Server (#7194)
### Background

This PR implements the main logic of the block execution engine for AutoGPT-Server.
An integration test is added to test the behavior.

*What you can do now with this PR*:
You can manually create a graph, by using the existing blocks as nodes (or write your own). Then execute the graph with an input.

*What you can't do yet*:
Listen to the graph execution result/update (you can follow the `AgentNodeExecution` table result, though).

### Changes 🏗️
* Split `data.py` (model file) into three modules:
    * `execution`: a model for node execution.
    * `graph`: a model for graph structure.
    * `block`: a model for agent block/component.
* Implemented executor main logic
* Simplify db structure:
    * Remove `AgentBlockInputOutput` in favor of `inputSchema` & `outputSchema` using serialized json/dict structure.
    * Remove `id` on `AgentBlock` in favor of using name (class name of the block) as its identifier.
    * Added `constantInput` column for `AgentNode` for hard-coded input/block configuration. Hence, removing`executionStateData` on `AgentNodeExecution`.
    * Rename AgentNodeLink input/output to source/sink to avoid confusion
* Change multithreading to multiprocessing, to allow the use of multiple `prisma` asynchronous client.
2024-06-10 19:30:34 +07:00
Nicholas Tindle
b803e42189 feat(docs): add example uses and description of autogpt to the readme (#7205) 2024-06-10 13:08:55 +01:00
Reinier van der Leer
04769c54e4 docs: Update CONTRIBUTING.md
- Add links to roadmap
- Disable links to kanban board (which currently is not in active use)
- Clarify instruction to not submit broken code
2024-06-09 23:43:25 +02:00