### Changes 🏗️
Avoid other threads accessing the channel within the same process.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Manual agent runs
<!-- Clearly explain the need for these changes: -->
### Changes 🏗️
This PR simply changes the logging type from info to debug of node
outputs in the agent.py file.
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [x] `.env.example` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
---------
Co-authored-by: Bentlybro <Github@bentlybro.com>
<!-- Clearly explain the need for these changes: -->
### Changes 🏗️
This change simply changes the logging level of node inputs and outputs
to debug level. This change is needed because currently logging all node
data causes logs that are too large for the logger to prevent nodes from
running.
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [x] `.env.example` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
We have seen instances where the executor gets stuck in a failing
message-consuming loop due to the upstream RabbitMQ being down. The
current message-consuming pattern is not optimal for handling this.
### Changes 🏗️
* Add a retry limit to the execution loop limit.
* Use `basic_consume` instead of `basic_get` for handling message
consumption.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Run agents cancel them
### Changes 🏗️
The recent change to the execution cancelation fix turns out to only
work on the first request.
This PR change fixes it by reworking how the thread_cached work on async
functions.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Cancel agent executions multiple times
<!-- Clearly explain the need for these changes: -->
Swap to pooling supabase connections rather than depending on x number
of max open connections
### Changes 🏗️
Adds direct connect URL to be used throughout the system
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Test thoroughly all of the endpoints in the dev env with switched
infra matching pr
- [x] Follow the new release plan tests
- [x] Follow the old release plan tests
#### For configuration changes:
- [x] `.env.example` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>configuration changes</summary>
- Change how we connect to the database to use direct when configured
and database URL when not
- update prisma for this
- have default matching database and default
</details>
### Changes 🏗️
- Update onboarding to give user rewards for completing steps
- Remove `canvas-confetti` lib and add `party-js` instead; the former
didn't allow to play confetti from a component
- Add onboarding videos in `frontend/public/onboarding/`
- Remove Balance (`CreditsCard.tsx`) and add openable `Wallet.tsx` (and
accompanying `WalletTaskGroup.tsx`) instead that displays grouped
onboarding tasks with descriptions and short instructional videos
- Further relevant updates to `useOnboarding`, `types.ts`
- Implement onboarding rewards
- Add `onboarding_reward` function in `credit.py` that is used to reward
user for finished onboarding tasks safely - transaction key is
deterministic, so the same user won't be rewarded twice for the same
step.
- Add `reward_user` in `onboarding.py`
- Update `UserOnboarding` model and add a migration
<img width="464" alt="Screenshot 2025-04-05 at 6 06 29 PM"
src="https://github.com/user-attachments/assets/fca8d09e-0139-466b-b679-d24117ad01f0"
/>
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Onboarding works
- [x] Tasks can be completed
- [x] Rewards are added correctly for all completed tasks
Currently the execution task is not properly distributed between
executors because we need to send the execution request to the execution
server.
The execution manager now accepts the execution request from the message
queue. Thus, we can remove the synchronous RPC system from this service,
let the system focus on executing the agent, and not spare any process
for the HTTP API interface.
This will also reduce the risk of the execution service being too busy
and not able to accept any add execution requests.
### Changes 🏗️
* Remove the RPC system in Agent Executor
* Allow the cancellation of the execution that is still waiting in the
queue (by avoiding it from being executed).
* Make a unified helper for adding an execution request to the system
and move other execution-related helper functions into
`executor/utils.py`.
* Remove non-db connections (redis / rabbitmq) in Database Manager and
let the client manage this by themselves.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Existing CI, some agent runs
The graph execution queue is not disk-persisted; when the executor dies,
the executions are lost.
The scope of this issue is migrating the execution queue from an
inter-process queue to a RabbitMQ message queue. A sync client should be
used for this.
- Resolves#9746
- Resolves#9714
### Changes 🏗️
Move the execution manager from multiprocess.Queue into persisted
Rabbit-MQ.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Execute agents.
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
<!-- Clearly explain the need for these changes: -->
We want to be able to filter errors according to where they occur in
sentry so we need to track and include that data. We also are not
logging everything from app services correctly so fix that up
### Changes 🏗️
<!-- Concisely describe all of the changes made in this pull request:
-->
- Adds env tracking for frontend
- adds sentry init in app service spawn
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Tested by running and making sure all events + logs are inserted
into sentry correctly
Distilled from #9541 to reduce the scope of that PR.
- Part of #9307
- ❗ Blocks #9786
- ❗ Blocks #9541
### Changes 🏗️
- Fix `LibraryAgent` schema (for #9786)
- Fix relationships between `LibraryAgent`, `AgentGraph`, and
`AgentPreset`
- Impose uniqueness constraint on `LibraryAgent`
- Rename things that are called `agent` that actually refer to a
`graph`/`agentGraph`
- Fix singular/plural forms in DB schema
- Simplify reference names of closely related objects (e.g.
`AgentGraph.AgentGraphExecutions` -> `AgentGraph.Executions`)
- Eliminate use of `# type: ignore` in DB statements
- Add `typed` and `typed_cast` utilities to `backend.util.type`
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] CI static type checking (with all risky `# type: ignore` removed)
- [x] Check that column references in views are updated
<!-- Clearly explain the need for these changes: -->
We were duplicating placeholder values across all agents 😨
### Changes 🏗️
<!-- Concisely describe all of the changes made in this pull request:
-->
Deep copies the schema instead
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Test the broken agent in dev
- Resolves#9792
### Changes 🏗️
- Replace all `default=[]` -> `default_factory=list`
- Replace all `default={}` -> `default_factory=dict`
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] CI
---------
Co-authored-by: Krzysztof Czerwinski <kpczerwinski@gmail.com>
The linter currently exits with exit code 0 even if linting fails. This
makes the CI linter permissive which isn't good.
Changes:
- Make linter exit with an error code if a linting step fails
- Fix existing formatting issues
This PR is to add the new [Meta: Llama 4
Maverick](https://openrouter.ai/meta-llama/llama-4-maverick) and [Meta:
Llama 4 Scout](https://openrouter.ai/meta-llama/llama-4-scout) models
via [OpenRouter](https://openrouter.ai/)
### Changes 🏗️
Added the model names to ``llm.py``
```
META_LLAMA_4_SCOUT = "meta-llama/llama-4-scout"
META_LLAMA_4_MAVERICK = "meta-llama/llama-4-maverick"
```
and the modela metadata
```
LlmModel.META_LLAMA_4_SCOUT: ModelMetadata("open_router", 131072, 131072),
LlmModel.META_LLAMA_4_MAVERICK: ModelMetadata("open_router", 1048576, 1000000),
```
and i have added the model price to ``block_cost_config.py``
```
LlmModel.META_LLAMA_4_SCOUT: 1,
LlmModel.META_LLAMA_4_MAVERICK: 1,
```
### Checklist 📋
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Open the build page and place a ai text block, open the model
select and scroll to the bottom and select either of the 2 models
- [x] test them with a prompt and wait for a reply!
This is a prerequisite infra change for
https://github.com/Significant-Gravitas/AutoGPT/issues/9714.
We will need a service where we can maintain our own client (db, redis,
rabbitmq, be it async/sync) and configure our own cadence of
initialization and cleanup.
While refactoring the service.py, an option to use Pyro as an RPC
protocol is also removed.
### Changes 🏗️
* Decouple resource initialization and cleanup from the parent
AppService logic.
* Removed Pyro.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] CI
<!-- Clearly explain the need for these changes: -->
Now that we are trying to use Sentry more, cleaning up some errors ->
warnings is a good idea
### Changes 🏗️
- reduces log level of retry to warning
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] check it comes through in sentry
<!-- Clearly explain the need for these changes: -->
We got this error in
sentry:[AUTOGPT-SERVER-33P](https://significant-gravitas.sentry.io/issues/6462614597/events/bb4871d796b04e759ade55197498cff9/)
```
Level: Error
'Secrets' object has no attribute 'ProviderName.GOOGLE_client_id'
```
### Changes 🏗️
- Follows pattern used when accessing these in
`_get_provider_oauth_handler` in the router
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Test to make sure getting works
<!-- Clearly explain the need for these changes: -->
Sentry just released logs so lets enrich our details there too
### Changes 🏗️
- Adds sentry logging
- Adds dependencies tracking all of our sentry integrations
- Adds environment tracking to sentry
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Tested to make sure events show up in sentry with the correct
environment logging
- Resolves#9731
### Changes 🏗️
- feat: Add "Steps" showing `node_execution_count` to agent run view
- Add `GraphExecutionMeta.stats.node_exec_count` attribute
- feat(backend/executor): Send graph execution update after *every* node
execution (instead of only I/O node executions)
- Update graph execution stats after every node execution
- refactor: Move `GraphExecutionMeta` stats into sub-object
(`cost`, `duration`, `total_run_time` -> `stats.cost`, `stats.duration`,
`stats.node_exec_time`)
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- View an agent run with 1+ steps on `/library/agents/[id]`
- [x] "Info" section layout doesn't break
- [x] Number of steps is shown
- Initiate a new agent run
- [x] "Steps" increments in real time during execution
- Resolves#9679
### Changes 🏗️
Frontend:
- Fix crash on `payload` graph input
- Fix crash on object type agent I/O values
- Hide "+ New run" if `graph.webhook_id` is set
Backend:
- Add computed field `webhook_id` to `GraphModel`
- Add computed property `webhook_input_node` to `GraphModel`
- Refactor:
- Move `Node.webhook_id` -> `NodeModel.webhook_id`
- Move `NodeModel.block` -> `Node.block` (computed property)
- Replace `get_block(node.block_id)` with `node.block` where sensible
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Create and run a simple graph
- [x] Create a graph with a webhook trigger and ensure it works
- [x] Check out the runs of a webhook-triggered graph and ensure the
page works
<!-- Clearly explain the need for these changes: -->
Uvicorn and our logs were ending up in different places, this pr enures
uvicorn using our logging config, not their own.
### Changes 🏗️
- Clears uvicorn's loggers for rest, ws
- always log to stdout,stderr and additionally log to gcp is appropriate
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Test all possible variants of the log cloud vs not and ensure that
uvicorn logs show up in the same place that rest of the system logs do
for all
---------
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
<!-- Clearly explain the need for these changes: -->
One of the pull request review notes from when these were first made is
that they don't belong in the v2 folder. This pr fixes where they are.
### Changes 🏗️
- Moves from v2 to routers for the postmark tooling
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Check that linting and tests pass
---------
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
<!-- Clearly explain the need for these changes: -->
We have logged 272k timeout errors in the past week from the event loop.
Don't raise those as errors.
Also along the way for diagnosing this we found that some items were
inserted into batches with incomplete datasets so handle that too.
### Changes 🏗️
- Handle timeout errors explicitly
- Add better messaging for other error types
- Add filtering for queueing bad mezsaging
- add filtering for reading bad batches
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Pull dev db
- [x] Test new code to check stability + error reduction
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
The cleanup command was only called on SIGTERM, making it possible for
the service to close without being cleaned. Risking the connection not
being proactively closed when the service is unused.
### Changes 🏗️
Call the cleanup command on the service finally block.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Run the service, stop it, see the log is printed (locally)
<!-- Clearly explain the need for these changes: -->
I want to be able to insert data into the graph as a webhook from
various services without making a provider specific webhook for things
like discord, slack, uptime bots, etc.
### Changes 🏗️
- Adds a generic webhook block that others can use
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Test the endpoint that is generated with a graph, making sure to
pass data and consts to it
Bumps [psutil](https://github.com/giampaolo/psutil) from 6.1.1 to 7.0.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/giampaolo/psutil/blob/master/HISTORY.rst">psutil's
changelog</a>.</em></p>
<blockquote>
<h1>7.0.0</h1>
<p>2025-02-13</p>
<p><strong>Enhancements</strong></p>
<ul>
<li>669_, [Windows]: <code>net_if_addrs()</code>_ also returns the
<code>broadcast</code> address
instead of <code>None</code>.</li>
<li>2480_: Python 2.7 is no longer supported. Latest version supporting
Python
2.7 is psutil 6.1.X. Install it with: <code>pip2 install
psutil==6.1.*</code>.</li>
<li>2490_: removed long deprecated <code>Process.memory_info_ex()</code>
method. It was
deprecated in psutil 4.0.0, released 8 years ago. Substitute is
<code>Process.memory_full_info()</code>.</li>
</ul>
<p><strong>Bug fixes</strong></p>
<ul>
<li>2496_, [Linux]: Avoid segfault (a cPython bug) on
<code>Process.memory_maps()</code>
for processes that use hundreds of GBs of memory.</li>
<li>2502_, [macOS]: <code>virtual_memory()</code>_ now relies on
<code>host_statistics64</code>
instead of <code>host_statistics</code>. This is the same approach used
by <code>vm_stat</code>
CLI tool, and should grant more accurate results.</li>
</ul>
<p><strong>Compatibility notes</strong></p>
<ul>
<li>2480_: Python 2.7 is no longer supported.</li>
<li>2490_: removed long deprecated <code>Process.memory_info_ex()</code>
method.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ea5b55605f"><code>ea5b556</code></a>
pre-release</li>
<li><a
href="d6e28b7a83"><code>d6e28b7</code></a>
try to fix tests</li>
<li><a
href="104bb3228b"><code>104bb32</code></a>
test cpu_times() for process children</li>
<li><a
href="16c091b380"><code>16c091b</code></a>
test cpu_times() for process children</li>
<li><a
href="eee09da72a"><code>eee09da</code></a>
[OSX] proc.c: Fix goo.gl link in comment for source reference (<a
href="https://redirect.github.com/giampaolo/psutil/issues/2505">#2505</a>)</li>
<li><a
href="17e27801e6"><code>17e2780</code></a>
ci: build aarch64 wheel on GHA aarch64 runner (<a
href="https://redirect.github.com/giampaolo/psutil/issues/2503">#2503</a>)</li>
<li><a
href="1ba8667c89"><code>1ba8667</code></a>
pin black version to 24.X, because new 25.X breaks style</li>
<li><a
href="9c114a5137"><code>9c114a5</code></a>
[OSX] use <code>host_statistics64</code> to get memory metrics (<a
href="https://redirect.github.com/giampaolo/psutil/issues/2502">#2502</a>)</li>
<li><a
href="08d7d43894"><code>08d7d43</code></a>
pin black version to 24.X, because new 25.X breaks style</li>
<li><a
href="a509e5aa18"><code>a509e5a</code></a>
669 windows broadcast addr (<a
href="https://redirect.github.com/giampaolo/psutil/issues/2501">#2501</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/giampaolo/psutil/compare/release-6.1.1...release-7.0.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
- Follow-up to #9657
<img width="280" alt="image"
src="https://github.com/user-attachments/assets/2f3cd683-db63-485f-8914-5654c34f1a4c"
/>
<img width="520" alt="image"
src="https://github.com/user-attachments/assets/de7e7cb9-61d4-4071-aea8-393ff5200c54"
/>
### Changes 🏗️
* Implement the input UI for Agent Input subtypes.
* Refactor node-input-component, extra out data type decision logic,
share it with runner/library input.
* Add `format` field for short-text, long-text, and mediafile type.
* Unify UI data type enum.
Out of scope:
- Styling for these inputs.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Use all the available agent input subtypes in an agent and run it
---------
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
- Resolves#8782
### Changes 🏗️
- feat(frontend/library): Use WS subscription to get real-time execution
updates
- feat(backend/ws_api): Send `GraphExecutionUpdate` on all new agent I/O
- Include agent I/O in `GraphExecutionUpdate` (by subclassing
`GraphExecution`)
- Add `IO_BLOCK_IDs` to `.blocks.io`
- feat(backend/ws_api): Add `subscribe_graph_executions` method to
WebSocket API
- feat(backend): Withhold `GraphExecution.node_executions` from requests
by non-graph-owners
- Split `GraphExecutionWithNodes` off of `GraphExecution`
- Use `GraphExecution` as much as possible, as it's a much cheaper query
than `GraphExecutionWithNodes`
- refactor(frontend): Make `GraphExecution.node_executions` optional
- fix(frontend): Parse dates in responses of `/executions` and
`/graphs/{graph_id}/executions`
- refactor(frontend/library): Move sorting logic for agent runs list
from `AgentRunsPage` to `AgentRunsSelectorList`
- refactor(backend/ws_api): Clean up message handler implementations
- refactor(backend/tests): Use `.data.execution.get_graph_execution(..)`
directly instead of `AgentServer.test_get_graph_run_results(..)`
Out-of-scope changes:
- refactor(backend): Remove unnecessary query include from
`.data.graph.get_graph_metadata(..)`
Demo:
https://github.com/user-attachments/assets/8ea6225d-7334-49cb-a522-83f153d840da
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- Go to `/library/agents/[id]` for an agent with inputs and outputs
- Draft and run a new run
- [x] -> should appear in the list of runs at the top
- [x] -> should be selected as soon as the request finishes
- [x] -> new I/O should appear as it is generated
- [x] -> status should be updated in real-time (both in list and in
adjacent details view)
- Click "Run again"
- [x] -> should appear in the list of runs at the top
- [x] -> should be selected as soon as the request finishes
- [x] -> new I/O should appear as it is generated
- [x] -> status should be updated in real-time (both in list and in
adjacent details view)
- Click "Open in builder" under "Agent actions"; run the agent from the
builder
- [x] -> should work the same as before
- [x] -> node I/O should appear in real-time
- [x] -> node execution statuses should update in real-time
This pull request includes several changes to improve the backend
functionality and configuration of the `autogpt_platform`. The most
important changes involve adding a RabbitMQ service for testing,
enhancing logging configuration, updating the linter script to handle
errors gracefully, and modifying test configurations.
Backend configuration improvements:
*
[`autogpt_platform/backend/docker-compose.test.yaml`](diffhunk://#diff-f6a211ff1c6d96d19adb5641ee287258a6af8d72a99e33dafb4a334094205a43R29-R43):
Added RabbitMQ service configuration for testing, including health
checks and environment variables.
*
[`autogpt_platform/backend/.env.example`](diffhunk://#diff-62020caf1b9a15e0e3b9b3b1b69d5f6464bf7643f62354cbbaabf755d57b6064R191-R192):
Added a section delimiter for optional API keys for use in finding the
optional keys end when auto generating integrations.
Error handling and logging enhancements:
*
[`autogpt_platform/backend/linter.py`](diffhunk://#diff-0787e3ef718ac9963df64d9ab1d8e7a3b35dc4ab0cb874c65da6c2901e1e4991R3):
Updated the `run` function to handle `subprocess.CalledProcessError`
exceptions and print error output to `stderr` and prevent raising a
stack trace when it should not.
[[1]](diffhunk://#diff-0787e3ef718ac9963df64d9ab1d8e7a3b35dc4ab0cb874c65da6c2901e1e4991R3)
[[2]](diffhunk://#diff-0787e3ef718ac9963df64d9ab1d8e7a3b35dc4ab0cb874c65da6c2901e1e4991L13-R23)
Testing configuration updates:
*
[`autogpt_platform/backend/pyproject.toml`](diffhunk://#diff-26ebebd91da791c6484f07d9d91484a66f52836708f5294b24365603438b880cR111):
Added `asyncio_default_fixture_loop_scope` to pytest configuration for
better control over asyncio fixtures.
*
[`autogpt_platform/backend/run_tests.py`](diffhunk://#diff-f09930577243a4ef5213bf6191a3c500a4b8d3dcfee2d4b452cf7ce66b3c494fL55):
Removed the `postgres-test` service from the test setup script as we
need all of docker services up for the tests to run.
This PR publicly exposes all the agents listed in the store to the
internet.
### Changes 🏗️
Remove the auth requirement for an agent to download the agent.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Accessing
`http://localhost:8006/api/store/download/agents/{agentId}` without
authorization key.
When we are cancelling a running graph execution, it's possible that the
graph is already terminated.
We need to allow this process to proceed and update the rest of its node
execution to terminate.
### Changes 🏗️
Instead of erroring out the graph execution status update, we proceed on
updating the node execution status.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Stop an already terminated graph
- Resolves#9693
### Changes 🏗️
- Catch the DB error and log a descriptive error message
- Add `NotFoundError` to `backend.util.exceptions`
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [ ] ~~I have tested my changes according to the test plan:~~
- Low-stakes change, high effort to test: we'll see if it works from the
production logs
- Prep work for #8782
- Prep work for #8779
### Changes 🏗️
- refactor(platform): Differentiate graph/node execution events
- fix(platform): Subscribe to execution updates by `graph_exec_id`
instead of `graph_id`+`graph_version`
- refactor(backend): Move all execution related models and functions
from `.data.graph` to `.data.execution`
- refactor(backend): Reorganize & refactor `.data.execution`
- fix(libs): Remove `load_dotenv` in `.auth.config` to fix test config
issues
- dx: Bump version of `black` in pre-commit config to v24.10.0 to match
poetry.lock
- Other minor refactoring in both frontend and backend
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- Run an agent in the builder
- [x] -> works normally, node I/O is updated in real time
- Run an agent in the library
- [x] -> works normally
---------
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
This is a follow up to
https://github.com/Significant-Gravitas/AutoGPT/pull/9511 fixing some
issues and updating onboarding.
### Changes 🏗️
- Update `UserOnboarding` data
- Update schema and add migration
- Change `step` in `UserOnboarding` to `completedSteps` array with
`OnboardingStep` enum
- Remove `isCompleted`: this is now inferred from `completedSteps`
values
- Don't onboard if <2 marketplace agents; that prevents self-host
onboarding
- Add endpoints:
- `is_onboarding_enabled`: to check if users should be onboarded (not if
they finished onboarding); now check if there are at least 2 marketplace
agents
- `get_store_agent`: returns `StoreAgentDetails` for given
`store_listing_version_id`
- `get_graph_meta_by_store_listing_version_id`: returns `GraphMeta`
- Add agent to Library just before running it and not when chosen and
remove code that was responsible for removing agent that wasn't run
- Move onboarding to `OnboardingProvider` (it'll be needed globally for
Phase 2)
- Multiple fixes, renames for clarity
### Checklist 📋
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Don't onboard if less than 2 marketplace agents
- [x] Avoid non-input and credentials agents
- [x] Onboarding works and can be finished
- [x] Onboarding resumes
- [x] Onboarding agent runs correctly
---------
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Some existing nodes use models that no longer exist as values on
`LlmModel` enum.
### Changes 🏗️
- Update models for all blocks with `LlmModel` fields that do not exist
in `LlmModel` enum to `gpt-4o`, directly in `AgentNode->constantInput`
db column, on server startup
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Updates wrong models to `gpt-4o` for all affected `AgentNode`s
- [x] Doesn't update correct models
- [x] Doesn't insert model when unnecessary
- [x] Doesn't break other values in jsonb
The current block web requests utility has a logic to avoid the system
firing into blocklisted IPs.
However, the current logic is still prone to a few security issues:
* DNS rebinding attack: due to the lack of guarantee on the used IP not
being changed during the IP checking and firing step.
* Open redirect: due to the request sensitive request headers are still
being propagated throughout the web redirect.
### Changes 🏗️
* Uses IP pinning to request the web.
* Strip `Authorization`, `Proxy-Authorization`, `Cookie` upon web
redirects.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Test the web request block, add more tests with different
validation scenarios.
(cherry picked from commit f0df4c9174)
The current block web requests utility has a logic to avoid the system
firing into blocklisted IPs.
However, the current logic is still prone to a few security issues:
* DNS rebinding attack: due to the lack of guarantee on the used IP not
being changed during the IP checking and firing step.
* Open redirect: due to the request sensitive request headers are still
being propagated throughout the web redirect.
### Changes 🏗️
* Uses IP pinning to request the web.
* Strip `Authorization`, `Proxy-Authorization`, `Cookie` upon web
redirects.
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Test the web request block, add more tests with different
validation scenarios.
Currently, an import statement like `from backend.blocks.basic import
AgentInputBlock` will initialize `backend.blocks` and thereby load all
other blocks. This has quite high potential to cause circular import
issues, and it's bad for performance in cases where we don't want to
load all blocks (yet).
The same goes for `backend.integrations.webhooks`.
### Changes 🏗️
- Change `__init__.py` of `backend.blocks` and
`backend.integrations.webhooks` to cached loader functions rather than
init-time code
- Change type of `BlockWebhookConfig.provider` to `ProviderName`
<!-- test edit to check that this doesn't break anything -->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Set up and use an agent with a webhook-triggered block
---------
Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
Agent Input Block subtypes do not have a proper input UI yet on the
library & run input page.
### Changes 🏗️
Provide a toggle to enable these blocks and set it to False by default.
### Checklist 📋
#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [ ] ...
<details>
<summary>Example test plan</summary>
- [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
- [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
- [ ] Edit an agent from monitor, and confirm it executes correctly
</details>
#### For configuration changes:
- [ ] `.env.example` is updated or already compatible with my changes
- [ ] `docker-compose.yml` is updated or already compatible with my
changes
- [ ] I have included a list of my configuration changes in the PR
description (under **Changes**)
<details>
<summary>Examples of configuration changes</summary>
- Changing ports
- Adding new services that need to communicate with each other
- Secrets or environment variable changes
- New or infrastructure changes such as databases
</details>
Zamil Majdy