Commit Graph

491 Commits

Author SHA1 Message Date
Nicholas Tindle
9bef383df2 fix(backend): Make URL pinning work with extra_url_validator (#9940)
Github Blocks use an URL transformer passed to `Requests` to convert web
URLs to the API URLs. This doesn't always work with the anti-SSRF URL
pinning mechanism that was implemented in #8531.

### Changes 🏗️
In `Requests.request(..)`:
- Apply `validate_url` *after* `extra_url_validator`, to prevent
mismatch between `pinned_url` and `original_hostname`
- Simplify logic & add clarifying comments

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Tested the github blocks that had the issue

---------

Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2025-05-15 13:12:39 +00:00
Chirag Modi
2dc038b6c0 Add Llama API Support (#9899)
The changes in this PR are to add Llama API support.

### Changes 🏗️
We add both backend and frontend support.

**Backend**:
- Add llama_api provider
- Include models supported by Llama API along with configs
- llm_call
- credential store and llama_api_key field in Settings

**Frontend**:
- Llama API as a type
- Credentials input and provider for Llama API
 

### Checklist 📋

#### For code changes:
- [X] I have clearly listed my changes in the PR description
- [X] I have tested my changes according to the test plan:

**Test Plan**:

<details>
  <summary>AI Text Generator</summary>
  
  - [X] Start-up backend and frontend:
- Start backend with Docker services: `docker compose up -d --build`
     - Start frontend: `npm install && npm run dev`
- By visiting http://localhost:3000/, test inference and structured
outputs
  - [X] Create from scratch 
  - [X] Request for Llama API Credentials
  
<img width="2015" alt="image"
src="https://github.com/user-attachments/assets/3dede402-3718-4441-9327-ecab25c63ebf"
/>

  - [X] Execute an agent with at least 3 blocks
 
<img width="2026" alt="image"
src="https://github.com/user-attachments/assets/59d6d56b-2ccc-4af5-b511-4af312c3f7f8"
/>

  - [X] Confirm it executes correctly
</details>

<details>
  <summary>Structured Response Generator</summary>
  
  - [X] Start-up backend and frontend:
- Start backend with Docker services: `docker compose up -d --build`
     - Start frontend: `npm install && npm run dev`
- By visiting http://localhost:3000/, test inference and structured
outputs
  - [X] Create from scratch 
  - [X] Execute an agent 
<img width="2023" alt="image"
src="https://github.com/user-attachments/assets/d1107638-bf1b-45b1-a296-1e0fac29525b"
/>

  - [X] Confirm it executes correctly
</details>

---------

Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2025-05-14 19:45:40 +00:00
Zamil Majdy
1999ba38d9 feat(backend): Truncate logging on NotificationManager and LLMBlocks (#9939)
> Log entry with size 865.8K exceeds maximum size of 256.0K

Some logs are just too large.

### Changes 🏗️

Truncate logging on NotificationManager and LLMBlocks

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] Existing CI
2025-05-14 19:18:59 +00:00
Toran Bruce Richards
08639bb1f0 fix(backend): Change library agents fetch sort from version to updatedAt (#9932)
<!-- Clearly explain the need for these changes: -->

### Changes 🏗️
This pull request includes a small change to the `get_my_agents`
function in `autogpt_platform/backend/backend/server/v2/store/db.py`.
The change updates the sorting order for retrieving library agents to
use the `updatedAt` field instead of `agentGraphVersion`.

This aligns with the later sorting that is applied to the fetched page
in the frontend.

<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [ ] ...

<details>
  <summary>Example test plan</summary>
  
  - [ ] Create from scratch and execute an agent with at least 3 blocks
- [ ] Import an agent from file upload, and confirm it executes
correctly
  - [ ] Upload agent to marketplace
- [ ] Import an agent from marketplace and confirm it executes correctly
  - [ ] Edit an agent from monitor, and confirm it executes correctly
</details>

#### For configuration changes:
- [x] `.env.example` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

<details>
  <summary>Examples of configuration changes</summary>

  - Changing ports
  - Adding new services that need to communicate with each other
  - Secrets or environment variable changes
  - New or infrastructure changes such as databases
</details>
2025-05-13 07:49:55 +00:00
Zamil Majdy
64ff161323 fix(backend): Avoid executor stuck on cleanup waiting execution completion 2025-05-12 22:29:41 +01:00
Nicholas Tindle
2b5b93a0f7 feat(backend): send emails daily instead of hourly (#9927)
<!-- Clearly explain the need for these changes: -->

John thinks an hourly email is a bit much; I'm inclined to agree.

### Changes 🏗️

<!-- Concisely describe all of the changes made in this pull request:
-->
swaps the time for dealing the batch for the agent runs from one hour to
one day

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] No testing done
2025-05-12 18:24:14 +00:00
Nicholas Tindle
79cc08787b feat(blocks): initial google calendar (#9920)
We want to add support for reading + writing events for google calendar
so agents can do cool stuff with that!

### Changes 🏗️

* Add support for google calendar blocks

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Run manual tests of each block
  - [x] Build and run automatic tests
2025-05-12 18:21:05 +00:00
Reinier van der Leer
b740a6edc0 fix(backend): Unbreak existing Agent Executor nodes (#9928)
- Follow-up fix for #9862

### Changes 🏗️

- Rename the stored `data` input field to `inputs` on all Agent Executor
nodes in the DB

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Dry-run the query in Supabase to verify that the data operation
works
  - [x] CI
2025-05-12 11:41:07 +00:00
Zamil Majdy
30086357bc Merge branch 'dev' of github.com:Significant-Gravitas/AutoGPT into dev 2025-05-10 05:19:38 +01:00
Zamil Majdy
e090195e57 feat(backend): User notification service to send a system alert 2025-05-10 05:19:22 +01:00
Zamil Majdy
4413366ea7 fix(blocks): Fix load_all_blocks in concurrent calls (#9926)
```
2025-05-09 11:54:15,171 ERROR  Error executing graph 954e6fc8-9c90-46fa-be5b-4063eb519ec7: Block ID AIMusicGeneratorBlock error: 44f6c8ad-d75c-4ae1-8209-aad1c0326928 is already in use
```

### Changes 🏗️

This PR avoids the use of global variables messing up with the way
`load_all_blocks` is cached.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] CI
2025-05-09 13:48:15 +00:00
Zamil Majdy
c736d401a6 fix(backend): Fix walltime & cputime stats on interrupted execution 2025-05-09 14:08:37 +01:00
Zamil Majdy
e8bc83445a feat(backend): Immediate alert on late execution check job failure (#9925)
Currently, there is no guarantee that an error will be reported right
away. And late execution is a serious issue that needs to be addressed
quickly

### Changes 🏗️

Provided a direct alert when late execution occurs.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Manual run on the late executions job on artificially created late
executions.
2025-05-09 07:31:23 +00:00
Zamil Majdy
8de88395f1 fix(backend): Continue stats accounting on aborted or broken executions (#9921)
This is a follow-up to
https://github.com/Significant-Gravitas/AutoGPT/pull/9903

The continued graph execution restarted all the execution stats from
zero, making the execution stats misleading.

### Changes 🏗️

Continue the execution stats when continuing the graph execution.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
- [x] Existing tests, manual graph run with the graph execution aborted
midway.
2025-05-09 06:46:30 +00:00
Zamil Majdy
82cf0bcde7 refactor(backend): Clear out Notification Service code blockage (#9915)
Some of the code paths in the notification & scheduler service were
synchronous HTTP calls that execute a long-running job that blocks. This
makes the service threads busy waiting.

### Changes 🏗️

* Remove queue_notification API
* Remove DTO
* Move heavy tasks intothe  executor

<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
- [x] Manually executing notification service jobs through the scheduler
API
2025-05-09 06:16:43 +00:00
Reinier van der Leer
433b76b539 fix(backend/scheduler): Unbreak Scheduler.get_execution_schedules (#9919)
- Resolves #9918
- Follow-up fix for #9914

### Changes 🏗️

- In `get_graph_execution_schedules`, skip jobs when their kwargs can't
be parsed as `GraphExecutionJobArgs`
- Rename methods of `Scheduler` to clarify their scope (scheduled
*graph* executions)

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - Go to `/library/agents/[id]` (which calls `GET /api/schedules`)
    - [x] -> `GET /api/schedules` request returns HTTP 200
2025-05-08 11:26:51 +00:00
Reinier van der Leer
1ad6c76f9c feat(backend): Require discriminator value on graph save (#9858)
If a node has a multi-credentials input (e.g. AI Text Generator block)
but the discriminator value (e.g. model choice) is missing, the input
can't be discriminated into a single-provider input. Discrimination into
a single-provider input is necessary to make a graph-level credentials
input for use in the Library.

### Changes 🏗️

- feat(backend): Require discriminator fields to always have a value

- dx(frontend): Improve typing of discriminator stuff
- dx(frontend): Fix typing in `NodeOneOfDiscriminatorField` component

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Saving & running graphs with and without credentials works
normally
- Note: We don't have any blocks with a discriminator that doesn't have
a default value, so currently I don't think it's possible to produce a
case where this mechanism would be triggered.
2025-05-08 09:45:22 +00:00
Bently
104928c614 feat(platform): Add captcha to login, signup and password reset pages (#9847)
This PR adds Cloudflare's Turnstile CAPTCHA to the login, signup, and
password reset pages. it is setup to only show and work when behave as
is set to CLOUD so it will not show for local hosted users.

### Changes 🏗️

#### Backend Changes
-
**[backend/server/v2/turnstile/routes.py](https://github.com/Significant-Gravitas/AutoGPT/compare/dev...bently/secrt-1169-implement-captcha-on-sign-up?expand=1#diff-2c5c2cb13346370fc48bdde8691a0d3bbfc030f7718288101b67b641c7948c10)**:
Created API endpoint at `/api/turnstile/verify` to proxy verification
requests to Cloudflare
-
**[backend/server/v2/turnstile/service.py](https://github.com/Significant-Gravitas/AutoGPT/compare/dev...bently/secrt-1169-implement-captcha-on-sign-up?expand=1#diff-296991fdc3ea821ae5a568ca96bb89789f2fc7dda7b62f59ef6bcadfaea16e56)**:
Implements service to verify CAPTCHA tokens with Cloudflare using
server-side secret key

#### Frontend Changes
-
**[frontend/src/lib/turnstile.ts](https://github.com/Significant-Gravitas/AutoGPT/compare/dev...bently/secrt-1169-implement-captcha-on-sign-up?expand=1#diff-a698e2718e0f6b0afe1d0c7fda571a7bfcbec6aeacc963c2b3620cc683dc4448)**:
Client-side function to call the backend verification endpoint
-
**[frontend/src/components/auth/Turnstile.tsx](https://github.com/Significant-Gravitas/AutoGPT/compare/dev...bently/secrt-1169-implement-captcha-on-sign-up?expand=1#diff-71a73d58d0ba5e46e5702f2f2599284e72a8fcf6c5d0b5c72e7358570d631aa7)**:
Reusable Turnstile component that renders and manages the CAPTCHA widget
-
**[frontend/src/hooks/useTurnstile.ts](https://github.com/Significant-Gravitas/AutoGPT/compare/dev...bently/secrt-1169-implement-captcha-on-sign-up?expand=1#diff-4a6a9363243ab2a88dbfb498917f464896ada059617bd8b0fb51df532c73827d)**:
Custom hook that manages Turnstile state and conditionally activates
based on environment

#### Auth Flow Integration
- Modified server actions in `login`, `signup`, and `reset_password` to
accept and verify Turnstile tokens
- Updated auth page components to integrate the CAPTCHA widget with form
submissions

### Configuration Changes
- Added two new environment variables:
- `NEXT_PUBLIC_CLOUDFLARE_TURNSTILE_SITE_KEY`: Public site key for
frontend
- `CLOUDFLARE_TURNSTILE_SECRET_KEY`: Secret key for backend verification

### Test Plan 📋
- Ask Bently for the keys to test locally!
- [x] Test login, signup and password reset with Turnstile enabled
(BEHAVE_AS=CLOUD)
- [x] Verify CAPTCHA appears and must be completed before form
submission
  - [x] Verify error message appears if CAPTCHA is not completed
  - [x] Verify form submission works after completing CAPTCHA
- [x] Test login, signup and password reset with Turnstile disabled
(BEHAVE_AS=LOCAL)
  - [x] Verify CAPTCHA does not appear
  - [x] Verify form submission works without CAPTCHA
- [x] Test with invalid site key to ensure proper error handling

---------

Co-authored-by: Krzysztof Czerwinski <34861343+kcze@users.noreply.github.com>
2025-05-07 21:08:12 +00:00
Reinier van der Leer
0726a00fb7 fix(backend): Include sub-graphs in graph-level credentials support (#9862)
The Library Agent credentials UX (#9789) currently doesn't work for
sub-graphs.

### Changes 🏗️

- Include sub-graphs in generating `Graph.credentials_input_schema`
- Propagate `node_credentials_input_map` into `AgentExecutionBlock`
executions
- Fix: also apply `node_credentials_input_map` in `_enqueue_next_nodes`

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - Import a graph with sub-graphs that need credentials
  - Run this agent from the Library
  - [x] -> Should work
2025-05-07 17:28:39 +00:00
Zamil Majdy
ac8ef9bdb2 feat(backend): Introduce late execution check scheduled job (#9914)
Introduce a late execution check scheduled job. The late threshold
duration is configurable.
This initial version only reports the error to Sentry.

### Changes 🏗️

* Added late execution check scheduled job
* Move the registration weekly notification processing job out of API
call and calling it directly from the scheduler service.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] Manual firing of scheduled job through an exposed API
2025-05-07 05:00:37 +00:00
Nicholas Tindle
505320fcd3 feat(backend): Move Scheduler (#9904)
<!-- Clearly explain the need for these changes: -->
We want the scheduler shouldn't scale with the rest API lol

### Changes 🏗️
pulls out the scheduler into its own service
<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] test it

---------

Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2025-05-05 18:59:28 +00:00
Krzysztof Czerwinski
6f1578239a feat(platform): Update Marketplace Agent listing buttons (#9843)
Currently agent listing on Marketplace have bad UX.

### Changes 🏗️

- Add function and endpoint to check if user has `LibraryAgent` by given
`storeListingVersionId`
- Redesign listing buttons
- `Add to library` shown when user is logged in and doesn't have an
agent in library
  - `See runs` shown when user logged in as has the agent in the library
  - `Download agent` always shown
  - Disabled buttons during processing (adding/downloading)
- Stop raising when owner is trying to add own agent. Now it'll simply
redirect to Library.
- Remove button appearing/flickering after a delay on listing page -
logged in status is now checked in server component.
- Show error toast on adding/redirecting to library and downloading
error
- Update breadcrumbs and page title to say `Marketplace` instead of
`Store`
- `font-geist` -> `font-sans` (`font-geist` var doesn't exist)

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Button on a listing is `Add to library` (no library agent)
  - [x] Agent can be added and user is redirected
- [x] Button on the listing is `See runs` and clicking it redirects to
the library agent
  - [x] Remove agent from library
  - [x] Buttons shows `Add to library` again
  - [x] Agent can be re-added
  - [x] Agent can be downloaded
  - [x] `Add to library` Button is hidden when user is logged out

---------

Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2025-05-05 16:47:58 +00:00
Zamil Majdy
79319ad1a7 fix(backend): Avoid broken process pool by not failing process initializer (#9907)
Process initializer on the process pool should never fail, but we do
network-related stuff there.
This cause the pool to be in a broken state.

### Changes 🏗️

Remove the health check step on process initializer.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] Existing CI test
2025-05-05 13:27:04 +00:00
Nicholas Tindle
afb66f75ec fix: disable google sheets in prod based on oauth review (#9906)
<!-- Clearly explain the need for these changes: -->

Our oauth review wants us to drop this in favor of a diff scope that
will require additional work

### Changes 🏗️
Disables the oauth sheets scopes in prod

<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [ ] set env locally
2025-05-02 19:40:51 +00:00
Krzysztof Czerwinski
59ec61ef98 feat(platform): Onboarding design&UX update (#9905)
A collection of updates regarding onboarding and wallet.

### Changes 🏗️

- `try-except` instead of `if` when rewarding (skip unnecessary db call)
- Make external services question onboarding step optional
- Add `SmartImage` component to lazy load images with pulse animation
and use it throughout onboarding
- Use store agent name instead of graph graph name (run page)
- Fix some images breaking layout on the agent card (run page)
- Center agent card vertically and horizontally (center on the left half
of page) (run page)
- Delay and tweak confetti when opening wallet and when task finished
(wallet)
- Flash wallet when credits change value
- Make tutorial video grayscale on completed steps (wallet)
- Fix confetti triggering on page refresh (wallet)
- Redirect to agent run page instead of Library after onboarding
- Expand task groups by default (wallet) - this means tutorial videos
are visible by default

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Services step is optional and skipping it doesn't break onboarding
  - [x] `SmartImage` works properly
  - [x] Agent card is aligned properly, including on page scroll
  - [x] Wallet flash when credits value change
  - [x] User is redirected to the agent runs page after onboarding
2025-05-02 14:42:01 +00:00
Zamil Majdy
d7077b5161 feat(backend): Continue instead of retrying aborted/broken agent execution (#9903)
Currently, the agent/graph execution engine is consuming the execution
queue and acknowledges the message after fully completing its execution
or failing it.

However, in the case of the agent executor failing due to a
hardware/resource issue, or the executor did not manage to acknowledge
the execution message. Another agent executor will pick it up and start
the execution again from the beginning.

The scope of this PR is to make the next executor pick up the next work
to continue the pre-existing execution instead of starting it all over
from the beginning.

### Changes 🏗️

* Removed `start_node_execs` from `GraphExecutionEntry`
* Populate the starting graph node from the DB query instead (fetching
Running & Queued node executions).
* Removed `get_incomplete_node_executions` from DB manager.
* Use get_node_executions with a status filter instead.
* Allow graph execution to end in non-FAILED/COMPLETED status, e.g, when
the executor is interrupted, it should be stuck in the running status,
and let other executors continue the task.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
- [x] Run an agent, stop the executor midway, re-reun the executor, the
execution should be continued instead of restarted.
2025-05-01 16:02:03 +00:00
Zamil Majdy
475c5a5cc3 fix(backend): Avoid executing any agent with zero balance (#9901)
### Changes 🏗️

* Avoid executing any agent with a zero balance.
* Make node execution count global across agents for a single user.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
- [x] Run agents by tweaking the `execution_cost_count_threshold` &
`execution_cost_per_threshold` values.
2025-05-01 15:11:38 +00:00
Zamil Majdy
86d5cfe60b feat(backend): Support flexible RPC client (#9842)
Using sync code in the async route often introduces a blocking
event-loop code that impacts stability.

The current RPC system only provides a synchronous client to call the
service endpoints.
The scope of this PR is to provide an entirely decoupled signature
between client and server, allowing the client can mix & match async &
sync options on the client code while not changing the async/sync nature
of the server.

### Changes 🏗️

* Add support for flexible async/sync RPC client.
* Migrate scheduler client to all-async client.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Scheduler route test.
  - [x] Modified service_test.py
  - [x] Run normal agent executions
2025-05-01 04:38:06 +00:00
Zamil Majdy
3526986f98 fix(backend): Failing test on a new Pydantic version (#9897)
```
FAILED test/model_test.py::test_agent_preset_from_db - pydantic_core._pydantic_core.ValidationError: 1 validation error for AgentNodeExecutionInputOutput

E       pydantic_core._pydantic_core.ValidationError: 1 validation error for AgentNodeExecutionInputOutput
E       data
E         JSON input should be string, bytes or bytearray [type=json_type, input_value=Json, input_type=Json]
E           For further information visit https://errors.pydantic.dev/2.11/v/json_type
```

### Changes 🏗️

Manually creating a Prisma model often breaks, and we have such an
instance in the test.
This PR fixes the test to make the new Pydantic happy.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] CI
2025-04-30 13:59:17 +00:00
Nicholas Tindle
04c4340ee3 feat(frontend,backend): user spending admin dashboard (#9751)
<!-- Clearly explain the need for these changes: -->
We need a way to refund people who spend money on agents wihout making
manual db actions

### Changes 🏗️
- Adds a bunch for refunding users
- Adds reasons and admin id for actions
- Add admin to db manager
- Add UI for this for the admin panel
- Clean up pagination controls
<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] Test by importing dev db as baseline
- [x] Add transactions on top for "refund", and make sure all existing
transactions work

---------

Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2025-04-29 17:39:25 +00:00
Zamil Majdy
9fa62c03f6 feat(backend): Improve cancel execution reliability (#9889)
When an executor dies, an ongoing execution will not be retried and will
just stuck in the running status.
This change avoids such a scenario by allowing an execution of an entry
that is not in QUEUED status with the low-probability risk of double
execution.

### Changes 🏗️

* Allow non-QUEUED status to be re-executed.
* Improve cleanup of node & graph executor.
* Make a cancellation request consumption a separate thread to avoid
being blocked by other messages.
* Remove unused retry loop on the execution manager.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] Run agent, kill the server, re-run it, agent restarted.
2025-04-29 17:06:03 +00:00
Mareddy Lohith Reddy
d5dc687484 fix: handle empty 204 responses in SendWebRequestBlock (#9887)
<!-- Clearly explain the need for these changes: -->
This PR fixes [Issue
#9883](https://github.com/Significant-Gravitas/AutoGPT/issues/9883),
where the SendWebRequestBlock crashes when receiving a 204 No Content
response, such as when posting to a Discord webhook. The fix ensures
that empty responses are handled gracefully, and the block does not
crash.

### Changes 🏗️
- Added a check to handle empty HTTP responses (like 204 status) in
SendWebRequestBlock
- Fallback to empty string or None if there is no response content
- Prevents server errors when parsing non-existent response bodies

<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] Send a POST request to an endpoint that returns 204 No Content
  - [x]  Confirm that SendWebRequestBlock handles it without crashing
  - [x] Confirm that regular 200 OK JSON responses still work

---------

Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
Co-authored-by: Lohith-11 <lohithr011@gamil.com>
Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
2025-04-28 19:16:04 +00:00
Nicholas Tindle
8fdfd75cc4 feat: allow admins to download agents for review (#9881)
<!-- Clearly explain the need for these changes: -->
for admins to approve agents for the marketplace, we need to be able to
run them. this is a quick workaround for downloading them so you can put
them in your marketplace to check

### Changes 🏗️
- clones various endpoints related to downloading into an admin side
with logging, and admin checks
- adds download button and removes open in builder action
<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [ ] Test downloading agents from local marketplace
2025-04-28 17:58:23 +00:00
Nicholas Tindle
7d83f1db05 feat(block): bring back PrintConsoleBlock (#9850)
### Changes 🏗️

Bring back PrintConsoleBlock

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Print console block

---------

Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2025-04-28 09:55:57 +00:00
Zamil Majdy
f07696e3c1 fix(backend): Fix top-up with zero transaction flow (#9886)
The transaction with zero payment amount will not generate a payment ID,
so the checkout failed for this scenario.

### Changes 🏗️

Don't use payment id as transaction key on top-up with zero payment
amount.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Top-up with stripe coupon
2025-04-27 14:22:08 +00:00
sentry-autofix[bot]
9715ea5313 fix: handle token limits and estimate token count for llm calls (#9880)
👋 Hi there! This PR was automatically generated by Autofix 🤖

This fix was triggered by Toran Bruce Richards.

Fixes
[AUTOGPT-SERVER-1ZY](https://sentry.io/organizations/significant-gravitas/issues/6386687527/).
The issue was that: `llm_call` calculates `max_tokens` without
considering `input_tokens`, causing OpenRouter API errors when the
context window is exceeded.

- Implements a function `estimate_token_count` to estimate the number of
tokens in a list of messages.
- Calculates available tokens based on the context window, estimated
input tokens, and user-defined max tokens.
- Adjusts `max_tokens` for LLM calls to prevent exceeding context window
limits.
- Reduces `max_tokens` by 15% and retries if a token limit error is
encountered during LLM calls.

If you have any questions or feedback for the Sentry team about this
fix, please email [autofix@sentry.io](mailto:autofix@sentry.io) with the
Run ID: 32838.

---------

Co-authored-by: sentry-autofix[bot] <157164994+sentry-autofix[bot]@users.noreply.github.com>
Co-authored-by: Krzysztof Czerwinski <kpczerwinski@gmail.com>
2025-04-25 13:45:47 +00:00
Zamil Majdy
ef022720d5 fix(backend): Avoid releasing lock that is no longer owned by the current thread (#9878)
There are instances of node executions that were failed and end up stuck
in the RUNNING status due to the execution failed to release the lock:
```
2025-04-24 20:53:31,573 INFO  [ExecutionManager|uid:25eba2d1-e9c1-44bc-88c7-43e0f4fbad5a|gid:01f8c315-c163-4dd1-a8a0-d396477c5a9f|nid:f8bf84ae-b1f0-4434-8f04-80f43852bc30]|geid:2e1b35c6-0d2f-4e97-adea-f6fe0d9965d0|neid:590b29ea-63ee-4e24-a429-de5a3e191e72|-] Failed node execution 590b29ea-63ee-4e24-a429-de5a3e191e72: Cannot release a lock that's no longer owned
```

### Changes 🏗️

Check the ownership of the lock before releasing.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] Existing CI tests.
2025-04-25 07:39:10 +00:00
Zamil Majdy
91f34966c8 fix(block): Fix Smart Decision Block missing input beads & incompability with input in special characters (#9875)
Smart Decision Block was not able to work with sub agent with custom
name input & the bead were not properly propagated in the execution UI.
The scope of this PR is fixing it.

### Changes 🏗️

* Introduce an easy to parse format of tool edge:
`{tool}_^_{func}_~_{arg}`. Graph using SmartDecisionBlock needs to be
re-saved before execution to work.
* Reduce cluttering on a smart decision block logic.
* Fix beads not being shown for a smart decision block tool calling.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Execute an SDM with some special character input as a tool

<img width="672" alt="image"
src="https://github.com/user-attachments/assets/873556b3-c16a-4dd1-ad84-bc86c636c406"
/>
2025-04-24 19:24:41 +00:00
Krzysztof Czerwinski
0675a41e42 fix(backend): Strip secrets, credentials when forking agent (#9874)
Strip secrets, credentials when forking agent

### Changes 🏗️

<!-- Concisely describe all of the changes made in this pull request:
-->

### Checklist 📋

#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
  - [ ] ...
2025-04-24 15:09:41 +00:00
Zamil Majdy
7fbe135ec8 feat(backend): Expose execution prometheus metrics (#9866)
Currently, we have no visibility on the state of the execution manager,
the scope of this PR is to open up the observability of it by exposing
Prometheus metrics.

### Changes 🏗️

Re-use the execution manager port to expose the Prometheus metrics.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Hit /metrics on 8002 port
2025-04-24 07:48:38 +00:00
Zamil Majdy
eb6a0b34e1 feat(backend): Use forkserver on process creation if possible (#9864)
### Changes 🏗️

Set process starting mode to forkserver instead of spawn, if possible,
for performance benefits.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Existing tests
2025-04-24 07:36:36 +00:00
Zamil Majdy
1e3236a041 feat(backend): Add retry on executor process initialization (#9865)
Executor process initialization can fail and cause this error:
```
concurrent.futures.process.BrokenProcessPool: A child process terminated abruptly, the process pool is not usable anymore
```

### Changes 🏗️

Add retry to reduce the chance of the initialization error to happen.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Existing tests
2025-04-23 21:24:17 +00:00
Krzysztof Czerwinski
160a622ba4 feat(platform): Forking agent in Library (#9870)
This PR introduces copying agents feature in the Library. Users can copy
and download their library agents but they can edit only the ones they
own (included copied ones).

### Changes 🏗️

- DB migration: add relation in `AgentGraph`: `forked_from_id` and
`forked_from_version`
- Add `fork_graph` function that makes a hardcopy of agent graph and its
nodes (all with new ids)
- Add `fork_library_agent` that copies library agent and its graph for a
user
- Add endpoint `/library/agents/{libraryAgentId}/fork`
- Add UI to `library/agents/[id]/page.tsx`: `Edit a copy` button with
dialog confirmation

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Agent can be copied, edited and runs
2025-04-23 16:28:42 +00:00
Zamil Majdy
20d39f6d44 fix(platform): Fix Google Maps API Key setting through env (#9848)
Setting the Google Maps API through the API has never worked on the
platform.

### Changes 🏗️

Set the default api key from the environment variable.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] Test GoogleMapsBlock
2025-04-22 03:00:47 +07:00
Bently
d5b82c01e0 feat(backend): Adds latest llm models (#9856)
This PR adds the following models:
OpenAI's O3: https://platform.openai.com/docs/models/o3
OpenAI's GPT 4.1: https://platform.openai.com/docs/models/gpt-4.1
Anthropics Claude 3.7: https://www.anthropic.com/news/claude-3-7-sonnet
Googles gemini 2.5 pro:
https://openrouter.ai/google/gemini-2.5-pro-preview-03-25
2025-04-21 19:26:21 +00:00
Krzysztof Czerwinski
67af77e179 fix((backend): Fix migrate llm models in existing agents (#9810)
https://github.com/Significant-Gravitas/AutoGPT/pull/9452 was throwing
`operator does not exist: text ? unknown` on deployed dev and so the
function call was commented as a hotfix.
This PR fixes and re-enables the llm model migration function.

### Changes 🏗️

- Uncomment and fix `migrate_llm_models` function

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Migrate nodes with non-existing models
  - [x] Don't migrate nodes without any model or with correct models

---------

Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
2025-04-19 12:52:36 +00:00
Zamil Majdy
9052ee7b95 fix(backend): Clear RabbitMQ connection cache on execution-manager retry 2025-04-19 07:50:04 +02:00
Zamil Majdy
c783f64b33 fix(backend): Handle add execution API request failure (#9838)
There are cases where the publishing agent execution is failing, making
the agent execution appear to be stuck in a queue, but the execution has
never been in a queue in the first place.

### Changes 🏗️

On publishing failure, we set the graph & starting node execution status
to FAILED and let the UI bubble up the error so the user can try again.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Normal add execution flow
2025-04-18 18:35:43 +00:00
Zamil Majdy
055a231aed feat(backend): Add retry mechanism for pika publish_message (#9839)
For unknown reason publishing message can fail sometimes due to the
connection being broken:
MessageQueue suddenly unavailable, connection simply broke, connection
being reset, etc.

### Changes 🏗️

Adding a tenacity retry on AMQP or ConnectionError, which hopefully can
alleviate the issue.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Simple add execution
2025-04-18 17:56:27 +00:00
Reinier van der Leer
417d7732af feat(platform/library): Add credentials UX on /library/agents/[id] (#9789)
- Resolves #9771
- ... in a non-persistent way, so it won't work for webhook-triggered
agents
    For webhooks: #9541

### Changes 🏗️

Frontend:
- Add credentials inputs in Library "New run" screen (based on
`graph.credentials_input_schema`)
- Refactor `CredentialsInput` and `useCredentials` to not rely on XYFlow
context

- Unsplit lists of saved credentials in `CredentialsProvider` state

- Move logic that was being executed at component render to `useEffect`
hooks in `CredentialsInput`

Backend:
- Implement logic to aggregate credentials input requirements to one per
provider per graph
- Add `BaseGraph.credentials_input_schema` (JSON schema) computed field
    Underlying added logic:
- `BaseGraph._credentials_input_schema` - makes a `BlockSchema` from a
graph's aggregated credentials inputs
- `BaseGraph.aggregate_credentials_inputs()` - aggregates a graph's
nodes' credentials inputs using `CredentialsFieldInfo.combine(..)`
- `BlockSchema.get_credentials_fields_info() -> dict[str,
CredentialsFieldInfo]`
- `CredentialsFieldInfo` model (created from
`_CredentialsFieldSchemaExtra`)

- Implement logic to inject explicitly passed credentials into graph
execution
  - Add `credentials_inputs` parameter to `execute_graph` endpoint
- Add `graph_credentials_input` parameter to
`.executor.utils.add_graph_execution(..)`
  - Implement `.executor.utils.make_node_credentials_input_map(..)`
  - Amend `.executor.utils.construct_node_execution_input`
  - Add `GraphExecutionEntry.node_credentials_input_map` attribute
  - Amend validation to allow injecting credentials
    - Amend `GraphModel._validate_graph(..)`
    - Amend `.executor.utils._validate_node_input_credentials`
- Add `node_credentials_map` parameter to
`ExecutionManager.add_execution(..)`
    - Amend execution validation to handle side-loaded credentials
    - Add `GraphExecutionEntry.node_execution_map` attribute
- Add mechanism to inject passed credentials into node execution data
- Add credentials injection mechanism to node execution queueing logic
in `Executor._on_graph_execution(..)`

- Replace boilerplate logic in `v1.execute_graph` endpoint with call to
existing `.executor.utils.add_graph_execution(..)`
- Replace calls to `.server.routers.v1.execute_graph` with
`add_graph_execution`

Also:
- Address tech debt in `GraphModel._validate_gaph(..)`
- Fix type checking in `BaseGraph._generate_schema(..)`

#### TODO
- [ ] ~~Make "Run again" work with credentials in
`AgentRunDetailsView`~~
- [ ] Prohibit saving a graph if it has nodes with missing discriminator
value for discriminated credentials inputs

### Checklist 📋

#### For code changes:
- [ ] I have clearly listed my changes in the PR description
- [ ] I have made a test plan
- [ ] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [ ] ...
2025-04-18 14:27:13 +00:00