- Fix type signatures of `sendWebSocketMessage(..)`, `onWebSocketMessage(..)`, `runGraph(..)` in `autogpt-server-api/client`
- Add `WebsocketMessageTypeMap`
- Fix type signature of `updateNodesWithExecutionData` in `FlowEditor`
### Background
When multiple executors are executing the same node within the same graph execution, two node executions can read the same queue of input and read the same value—making the data that is supposed to be consumed once, consumed by two executions. The lack of lock & concurrency support for parallel execution within a single graph causes this issue.
Node concurrency also introduces poor UX in the current frontend implementation, when two nodes are executed in parallel, the current UI will not display its parallel execution update, but instead, it shows the updates that override each other. Until the execution observability is improved on the builder UI, this capability will be limited.
### Changes 🏗️
The scope of this change is to solve this issue by:
* Decouple Graph execution & Node execution, each has its own configured process pool.
* Make sure there is only 1 execution per node (we still allow parallel executions on different nodes) in a graph.
* Fixed concurrency issue by adding distributed lock API on agent_server.
* Few cleanups:
- Add more logging with geid & neid prefix on graph/node executions
- Moved execution status update to agent-server for a single source of status update (required by conn-manager/web-socket)
- Configured node parallelism to 10 & graph parallelism to 10 by default, so in the very rare worst-case, there can be 100 node executions.
- Re-use server resource for each integration test run
- Set `node.data.connections` based on `graph.links` in `loadGraph(..)`
- Use reactflow's built-in `useNodesState`, `useEdgesState` to replace some boilerplate functions that do exactly the same
Resolves#7392
* refactor(MathsBlock): Simplify output to return numeric result directly
- Remove MathsResult class and explanation field
- Update Output schema to use float type
- Simplify run method to yield numeric result only
- Adjust error handling to return inf or nan for errors
- Update test cases to reflect new output structure
* run format
* refactor(CounterBlock): Simplify output to return count as integer
- Remove CounterResult class
- Update Output schema to use int type directly
- Simplify run method to yield count without explanation
- Modify error handling to return -1 for any errors
- Update test case to reflect new output structure
* ci(server): add sqlite processing
* ci(server): try setting DATABASE_URL based on db platform
* fix(server): swap default back to sqlite
* ci(server): go back to database url
---------
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
* replace SQLite with Postgres
* dockerfiles and optional docker compose set up
* Update rnd/autogpt_builder/Dockerfile
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
* address feedback
* Update .dockerignore
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
* Remove example files folder
* remove backend and frontend from docker compose
---------
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
* feat: Add YouTubeTranscriber block for extracting transcripts from YouTube videos
* feat: Add youtube-transcript-api dependency to pyproject.toml
* feat: Add SchemaField and test_mock to YoutTube Transcriber.
* feat: update lock
* fix(server): the agbenchmark was out of date
* fix(server): linting
* fix(server): drop mock
* fix(server): poetry locked in
* fix(server): test had ... at the end?
---------
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Fixes#7508
- Amend `app/configurator.py:check_model(..)` to check multiple models at once and save duplicate API calls
- Amend `MultiProvider.get_available_providers()` to verify availability by fetching models and handle failure
feat: Add support for new Groq models
The commit adds support for new Groq models, including LLAMA3_1_405B, LLAMA3_1_70B, and LLAMA3_1_8B. These models are part of the preview release and offer enhanced reasoning and versatility capabilities.
* feat(blocks): Add MathsBlock for performing mathematical operations
The commit adds a new block called MathsBlock to perform various mathematical operations such as addition, subtraction, multiplication, division, and exponentiation. The block takes input parameters for the operation type, two numbers, and an option to round the result. It returns the result of the calculation along with an explanation of the performed operation.
---------
Co-authored-by: Swifty <craigswift13@gmail.com>
* feat: Add RSSReaderBlock for reading RSS feeds
The commit adds a new `RSSReaderBlock` class in the `rss-reader-block.py` file. This block allows users to read RSS feeds by providing the URL of the feed, start datetime, polling rate, and a flag to run the block continuously. The block fetches the feed using the `feedparser` library and returns the title, link, description, publication date, author, and categories of each RSS item.
This commit also includes the addition of the `feedparser` dependency in the `pyproject.toml` file.
* fix(server): update lock file
* updated poetry lock
* fixed rss reader testing
* Updated error message in test to include check info
* Set starttime as 1 day ago
* Changed start time to time period
---------
Co-authored-by: Swifty <craigswift13@gmail.com>
- Handles:
- Add `NodeHandle` to draw input and output handles
- Position handles relatively
- Make entire handle label clickable/connectable
- Add input/output types below labels
- Change color on hover and when connected
- "Connected" no longer shows up when connected
- Edges:
- Draw edge above node when connecting to the same node
- Add custom `ConnectionLine`; drawn when making a connection
- Add `CustomEdge`; drawn for existing connections
- Add arrow to the edge end
- Colorize depending on type
- Input field modal:
- Select all text when opened
- Disable node dragging
- CSS:
- Remove not needed styling
- Use tailwind classes instead of css for some components
- Minor style changes
- Add shadcn switch
- Change bottom node buttons (for properties and advanced) to switches
- Format code
- fix(builder/monitor): Export `Graph` rather than `GraphMeta`
- Fixes#7557
- refactor(builder): Split up `lib/autogpt_server_api` into multi-file module
- Resolves#7555
- Rename `lib/autogpt_server_api` to `lib/autogpt-server-api`
- Split up `lib/autogpt-server-api` into `/client`, `/types`
- Move `ObjectSchema` from `lib/types` to `lib/autogpt-server-api/types`
- Make definition of `Node['metadata']['position']` independent of `reactflow.XYPosition`
- fix(builder/monitor): Strip secrets from graph on export
- Resolves#7492
- Add `safeCopyGraph` function in `lib/autogpt-server-api/utils`
- Use `safeCopyGraph` to strip secrets from graph on export in `/monitor` > `FlowInfo`
### Background
Input from the input pin is consumed only once, and the default input can always be used. So when you have an input pin overriding the default input, the value will be used only once and the following run will always fall back to the default input. This can mislead the user.
Expected behaviour: the node should NOT RUN, making connected pins only use their connection(s) for sources of data.
### Changes 🏗️
* Make pin connection the mandatory source of input and not falling back to default value.
* Fix the type flakiness on block input & output. Unify the typing for BlockInput & BlockOutput using the right alias to avoid wrong typing.
* Add comment on alias
* automated test on the new behaviour.
