- Add `ajv` dependency to check values against json schema
- Add `errors` and `setErrors` to `CustomNodeData`
- Add `validateNodes` run before executing agent
- Add `*` on labels for required fields
- Add `setNestedProperty` and `removeEmptyStringsAndNulls` utility function
- Fix type signatures of `sendWebSocketMessage(..)`, `onWebSocketMessage(..)`, `runGraph(..)` in `autogpt-server-api/client`
- Add `WebsocketMessageTypeMap`
- Fix type signature of `updateNodesWithExecutionData` in `FlowEditor`
### Background
When multiple executors are executing the same node within the same graph execution, two node executions can read the same queue of input and read the same value—making the data that is supposed to be consumed once, consumed by two executions. The lack of lock & concurrency support for parallel execution within a single graph causes this issue.
Node concurrency also introduces poor UX in the current frontend implementation, when two nodes are executed in parallel, the current UI will not display its parallel execution update, but instead, it shows the updates that override each other. Until the execution observability is improved on the builder UI, this capability will be limited.
### Changes 🏗️
The scope of this change is to solve this issue by:
* Decouple Graph execution & Node execution, each has its own configured process pool.
* Make sure there is only 1 execution per node (we still allow parallel executions on different nodes) in a graph.
* Fixed concurrency issue by adding distributed lock API on agent_server.
* Few cleanups:
- Add more logging with geid & neid prefix on graph/node executions
- Moved execution status update to agent-server for a single source of status update (required by conn-manager/web-socket)
- Configured node parallelism to 10 & graph parallelism to 10 by default, so in the very rare worst-case, there can be 100 node executions.
- Re-use server resource for each integration test run
- Set `node.data.connections` based on `graph.links` in `loadGraph(..)`
- Use reactflow's built-in `useNodesState`, `useEdgesState` to replace some boilerplate functions that do exactly the same
Resolves#7392
* refactor(MathsBlock): Simplify output to return numeric result directly
- Remove MathsResult class and explanation field
- Update Output schema to use float type
- Simplify run method to yield numeric result only
- Adjust error handling to return inf or nan for errors
- Update test cases to reflect new output structure
* run format
* refactor(CounterBlock): Simplify output to return count as integer
- Remove CounterResult class
- Update Output schema to use int type directly
- Simplify run method to yield count without explanation
- Modify error handling to return -1 for any errors
- Update test case to reflect new output structure
* ci(server): add sqlite processing
* ci(server): try setting DATABASE_URL based on db platform
* fix(server): swap default back to sqlite
* ci(server): go back to database url
---------
Co-authored-by: Aarushi <50577581+aarushik93@users.noreply.github.com>
* replace SQLite with Postgres
* dockerfiles and optional docker compose set up
* Update rnd/autogpt_builder/Dockerfile
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
* address feedback
* Update .dockerignore
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
* Remove example files folder
* remove backend and frontend from docker compose
---------
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
* feat: Add YouTubeTranscriber block for extracting transcripts from YouTube videos
* feat: Add youtube-transcript-api dependency to pyproject.toml
* feat: Add SchemaField and test_mock to YoutTube Transcriber.
* feat: update lock
* fix(server): the agbenchmark was out of date
* fix(server): linting
* fix(server): drop mock
* fix(server): poetry locked in
* fix(server): test had ... at the end?
---------
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
