mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-02-10 14:55:16 -05:00
e596ea87cbae55eedba3bc58f144702e9e310504
106 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Bently
|
de0ec3d388 |
chore(llm): remove deprecated Claude 3.7 Sonnet model with migration and defensive handling (#11841)
## Summary Remove `claude-3-7-sonnet-20250219` from LLM model definitions ahead of Anthropic's API retirement, with comprehensive migration and defensive error handling. ## Background Anthropic is retiring Claude 3.7 Sonnet (`claude-3-7-sonnet-20250219`) on **February 19, 2026 at 9:00 AM PT**. This PR removes the model from the platform and migrates existing users to prevent service interruptions. ## Changes ### Code Changes - Remove `CLAUDE_3_7_SONNET` enum member from `LlmModel` in `llm.py` - Remove corresponding `ModelMetadata` entry - Remove `CLAUDE_3_7_SONNET` from `StagehandRecommendedLlmModel` enum - Remove `CLAUDE_3_7_SONNET` from block cost config - Add `CLAUDE_4_5_SONNET` to `StagehandRecommendedLlmModel` enum - Update Stagehand block defaults from `CLAUDE_3_7_SONNET` to `CLAUDE_4_5_SONNET` (staying in Claude family) - Add defensive error handling in `CredentialsFieldInfo.discriminate()` for deprecated model values ### Database Migration - Adds migration `20260126120000_migrate_claude_3_7_to_4_5_sonnet` - Migrates `AgentNode.constantInput` model references - Migrates `AgentNodeExecutionInputOutput.data` preset overrides ### Documentation - Updated `docs/integrations/block-integrations/llm.md` to remove deprecated model - Updated `docs/integrations/block-integrations/stagehand/blocks.md` to remove deprecated model and add Claude 4.5 Sonnet ## Notes - Agent JSON files in `autogpt_platform/backend/agents/` still reference this model in their provider mappings. These are auto-generated and should be regenerated separately. ## Testing - [ ] Verify LLM block still functions with remaining models - [ ] Confirm no import errors in affected files - [ ] Verify migration runs successfully - [ ] Verify deprecated model gives helpful error message instead of KeyError |
||
|
Nicholas Tindle
|
7668c17d9c |
feat(platform): add User Workspace for persistent CoPilot file storage (#11867)
Implements persistent User Workspace storage for CoPilot, enabling
blocks to save and retrieve files across sessions. Files are stored in
session-scoped virtual paths (`/sessions/{session_id}/`).
Fixes SECRT-1833
### Changes 🏗️
**Database & Storage:**
- Add `UserWorkspace` and `UserWorkspaceFile` Prisma models
- Implement `WorkspaceStorageBackend` abstraction (GCS for cloud, local
filesystem for self-hosted)
- Add `workspace_id` and `session_id` fields to `ExecutionContext`
**Backend API:**
- Add REST endpoints: `GET/POST /api/workspace/files`, `GET/DELETE
/api/workspace/files/{id}`, `GET /api/workspace/files/{id}/download`
- Add CoPilot tools: `list_workspace_files`, `read_workspace_file`,
`write_workspace_file`
- Integrate workspace storage into `store_media_file()` - returns
`workspace://file-id` references
**Block Updates:**
- Refactor all file-handling blocks to use unified `ExecutionContext`
parameter
- Update media-generating blocks to persist outputs to workspace
(AIImageGenerator, AIImageCustomizer, FluxKontext, TalkingHead, FAL
video, Bannerbear, etc.)
**Frontend:**
- Render `workspace://` image references in chat via proxy endpoint
- Add "AI cannot see this image" overlay indicator
**CoPilot Context Mapping:**
- Session = Agent (graph_id) = Run (graph_exec_id)
- Files scoped to `/sessions/{session_id}/`
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [ ] I have tested my changes according to the test plan:
- [ ] Create CoPilot session, generate image with AIImageGeneratorBlock
- [ ] Verify image returns `workspace://file-id` (not base64)
- [ ] Verify image renders in chat with visibility indicator
- [ ] Verify workspace files persist across sessions
- [ ] Test list/read/write workspace files via CoPilot tools
- [ ] Test local storage backend for self-hosted deployments
#### For configuration changes:
- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)
🤖 Generated with [Claude Code](https://claude.ai/code)
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Medium Risk**
> Introduces a new persistent file-storage surface area (DB tables,
storage backends, download API, and chat tools) and rewires
`store_media_file()`/block execution context across many blocks, so
regressions could impact file handling, access control, or storage
costs.
>
> **Overview**
> Adds a **persistent per-user Workspace** (new
`UserWorkspace`/`UserWorkspaceFile` models plus `WorkspaceManager` +
`WorkspaceStorageBackend` with GCS/local implementations) and wires it
into the API via a new `/api/workspace/files/{file_id}/download` route
(including header-sanitized `Content-Disposition`) and shutdown
lifecycle hooks.
>
> Extends `ExecutionContext` to carry execution identity +
`workspace_id`/`session_id`, updates executor tooling to clone
node-specific contexts, and updates `run_block` (CoPilot) to create a
session-scoped workspace and synthetic graph/run/node IDs.
>
> Refactors `store_media_file()` to require `execution_context` +
`return_format` and to support `workspace://` references; migrates many
media/file-handling blocks and related tests to the new API and to
persist generated media as `workspace://...` (or fall back to data URIs
outside CoPilot), and adds CoPilot chat tools for
listing/reading/writing/deleting workspace files with safeguards against
context bloat.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
|
||
|
Nicholas Tindle
|
0953983944 |
feat(platform): disable onboarding redirects and add $5 signup bonus (#11862)
Disable automatic onboarding redirects on signup/login while keeping the
checklist/wallet functional. Users now receive $5 (500 credits) on their
first visit to /copilot.
### Changes 🏗️
- **Frontend**: `shouldShowOnboarding()` now returns `false`, disabling
auto-redirects to `/onboarding`
- **Backend**: Added `VISIT_COPILOT` onboarding step with 500 credit
($5) reward
- **Frontend**: Copilot page automatically completes `VISIT_COPILOT`
step on mount
- **Database**: Migration to add `VISIT_COPILOT` to `OnboardingStep`
enum
NOTE: /onboarding/1-welcome -> /library now as shouldShowOnboardin is
always false
Users land directly on `/copilot` after signup/login and receive $5
invisibly (not shown in checklist UI).
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] New user signup (email/password) → lands on `/copilot`, wallet
shows 500 credits
- [x] Verified credits are only granted once (idempotent via onboarding
reward mechanism)
- [x] Existing user login (already granted flag set) → lands on
`/copilot`, no duplicate credits
- [x] Checklist/wallet remains functional
#### For configuration changes:
- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)
No configuration changes required.
---
OPEN-2967
🤖 Generated with [Claude Code](https://claude.ai/code)
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Introduces a new onboarding step and adjusts onboarding flow.
>
> - Adds `VISIT_COPILOT` onboarding step (+500 credits) with DB enum
migration and API/type updates
> - Copilot page auto-completes `VISIT_COPILOT` on mount to grant the
welcome bonus
> - Changes `/onboarding/enabled` to require user context and return
`false` when `CHAT` feature is enabled (skips legacy onboarding)
> - Wallet now refreshes credits on any onboarding `step_completed`
notification; confetti limited to visible tasks
> - Test flows updated to accept redirects to `copilot`/`library` and
verify authenticated state
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
|
||
|
Zamil Majdy
|
fb58827c61 |
feat(backend;frontend): Implement node-specific auto-approval, safety popup, and race condition fixes (#11810)
## Summary
This PR implements comprehensive improvements to the human-in-the-loop
(HITL) review system, including safety features, architectural changes,
and bug fixes:
### Key Features
- **SECRT-1798: One-time safety popup** - Shows informational popup
before first run of AI-generated agents with sensitive actions/HITL
blocks
- **SECRT-1795: Auto-approval toggle UX** - Toggle in pending reviews
panel to auto-approve future actions from the same node
- **Node-specific auto-approval** - Changed from execution-specific to
node-specific using special key pattern
`auto_approve_{graph_exec_id}_{node_id}`
- **Consolidated approval checking** - Merged `check_auto_approval` into
`check_approval` using single OR query for better performance
- **Race condition prevention** - Added execution status check before
resuming to prevent duplicate execution when approving while graph is
running
- **Parallel auto-approval creation** - Uses `asyncio.gather` for better
performance when creating multiple auto-approval records
## Changes
### Backend Architecture
- **`human_review.py`**:
- Added `check_approval()` function that checks both normal and
auto-approval in single query
- Added `create_auto_approval_record()` for node-specific auto-approval
using special key pattern
- Added `get_auto_approve_key()` helper to generate consistent
auto-approval keys
- **`review/routes.py`**:
- Added execution status check before resuming to prevent race
conditions
- Refactored auto-approval record creation to use parallel execution
with `asyncio.gather`
- Removed obvious comments for cleaner code
- **`review/model.py`**: Added `auto_approve_future_actions` field to
`ReviewRequest`
- **`blocks/helpers/review.py`**: Updated to use consolidated
`check_approval` via database manager client
- **`executor/database.py`**: Exposed `check_approval` through
DatabaseManager RPC for block execution context
- **`data/block.py`**: Fixed safe mode checks for sensitive action
blocks
### Frontend
- **New `AIAgentSafetyPopup`** component with localStorage-based
one-time display
- **`PendingReviewsList`**:
- Replaced "Approve all future actions" button with toggle
- Toggle resets data to original values and disables editing when
enabled
- Shows warning message explaining auto-approval behavior
- **`RunAgentModal`**: Integrated safety popup before first run
- **`usePendingReviews`**: Added polling for real-time badge updates
- **`FloatingSafeModeToggle` & `SafeModeToggle`**: Simplified visibility
logic
- **`local-storage.ts`**: Added localStorage key for popup state
tracking
### Bug Fixes
- Fixed "Client is not connected to query engine" error by using
database manager client pattern
- Fixed race condition where approving reviews while graph is RUNNING
could queue execution twice
- Fixed migration to only drop FK constraint, not non-existent column
- Fixed card data reset when auto-approve toggle changes
### Code Quality
- Removed duplicate/obvious comments
- Moved imports to top-level instead of local scope in tests
- Used walrus operator for cleaner conditional assignments
- Parallel execution for auto-approval record creation
## Test plan
- [ ] Create an AI-generated agent with sensitive actions (e.g., email
sending)
- [ ] First run should show the safety popup before starting
- [ ] Subsequent runs should not show the popup
- [ ] Clear localStorage (`AI_AGENT_SAFETY_POPUP_SHOWN`) to verify popup
shows again
- [ ] Create an agent with human-in-the-loop blocks
- [ ] Run it and verify the pending reviews panel appears
- [ ] Enable the "Auto-approve all future actions" toggle
- [ ] Verify editing is disabled and shows warning message
- [ ] Click "Approve" and verify subsequent blocks from same node
auto-approve
- [ ] Verify auto-approval persists across multiple executions of same
graph
- [ ] Disable toggle and verify editing works normally
- [ ] Verify "Reject" button still works regardless of toggle state
- [ ] Test race condition: Approve reviews while graph is RUNNING
(should skip resume)
- [ ] Test race condition: Approve reviews while graph is REVIEW (should
resume)
- [ ] Verify pending reviews badge updates in real-time when new reviews
are created
|
||
|
Zamil Majdy
|
f9f984a8f4 |
fix(db): Remove redundant migration and fix pgvector schema handling (#11822)
### Changes 🏗️ This PR includes two database migration fixes: #### 1. Remove redundant Supabase extensions migration Removes the `20260112173500_add_supabase_extensions_to_platform_schema` migration which was attempting to manage Supabase-provided extensions and schemas. **What was removed:** - Migration that created extensions (pgcrypto, uuid-ossp, pg_stat_statements, pg_net, pgjwt, pg_graphql, pgsodium, supabase_vault) - Schema creation for these extensions **Why it was removed:** - These extensions and schemas are pre-installed and managed by Supabase automatically - The migration was redundant and could cause schema drift warnings - Attempting to manage Supabase-owned resources in our migrations is an anti-pattern #### 2. Fix pgvector extension schema handling Improves the `20260109181714_add_docs_embedding` migration to handle cases where pgvector exists in the wrong schema. **Problem:** - If pgvector was previously installed in `public` schema, `CREATE EXTENSION IF NOT EXISTS` would succeed but not actually install it in the `platform` schema - This causes `type "vector" does not exist` errors because the type isn't in the search_path **Solution:** - Detect if vector extension exists in a different schema than the current one - Drop it with CASCADE and reinstall in the correct schema (platform) - Use dynamic SQL with `EXECUTE format()` to explicitly specify the target schema - Split exception handling: catch errors during removal, but let installation fail naturally with clear PostgreSQL errors **Impact:** - No functional changes - Supabase continues to provide extensions as before - pgvector now correctly installs in the platform schema - Cleaner migration history - Prevents schema-related errors ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Verified migrations run successfully without the redundant file - [x] Confirmed Supabase extensions are still available - [x] Tested pgvector migration handles wrong-schema scenario - [x] No schema drift warnings #### For configuration changes: - [x] .env.default is updated or already compatible with my changes - [x] docker-compose.yml is updated or already compatible with my changes - [x] I have included a list of my configuration changes in the PR description (under **Changes**) - N/A - No configuration changes required |
||
|
Zamil Majdy
|
5d0cd88d98 |
fix(backend): Use unqualified vector type for pgvector queries (#11818)
## Summary
- Remove explicit schema qualification (`{schema}.vector` and
`OPERATOR({schema}.<=>)`) from pgvector queries in `embeddings.py` and
`hybrid_search.py`
- Use unqualified `::vector` type cast and `<=>` operator which work
because pgvector is in the search_path on all environments
## Problem
The previous approach tried to explicitly qualify the vector type with
schema names, but this failed because:
- **CI environment**: pgvector is in `public` schema → `platform.vector`
doesn't exist
- **Dev (Supabase)**: pgvector is in `platform` schema → `public.vector`
doesn't exist
## Solution
Use unqualified `::vector` and `<=>` operator. PostgreSQL resolves these
via `search_path`, which includes the schema where pgvector is installed
on all environments.
Tested on both local and dev environments with a test script that
verified:
- ✅ Unqualified `::vector` type cast
- ✅ Unqualified `<=>` operator in ORDER BY
- ✅ Unqualified `<=>` in SELECT (similarity calculation)
- ✅ Combined query patterns matching actual usage
## Test plan
- [ ] CI tests pass
- [ ] Marketplace approval works on dev after deployment
Fixes: AUTOGPT-SERVER-763, AUTOGPT-SERVER-764, AUTOGPT-SERVER-76B
|
||
|
Zamil Majdy
|
f31c160043 |
feat(platform): add endedAt field and fix execution analytics timestamps (#11759)
## Summary
This PR adds proper execution end time tracking and fixes timestamp
handling throughout the execution analytics system.
### Key Changes
1. **Added `endedAt` field to database schema** - Executions now have a
dedicated field for tracking when they finish
2. **Fixed timestamp nullable handling** - `started_at` and `ended_at`
are now properly nullable in types
3. **Fixed chart aggregation** - Reduced threshold from ≥3 to ≥1
executions per day
4. **Improved timestamp display** - Moved timestamps to expandable
details section in analytics table
5. **Fixed nullable timestamp bugs** - Updated all frontend code to
handle null timestamps correctly
## Problem Statement
### Issue 1: Missing Execution End Times
Previously, executions used `updatedAt` (last DB update) as a proxy for
"end time". This broke when adding correctness scores retroactively -
the end time would change to whenever the score was added, not when the
execution actually finished.
### Issue 2: Chart Shows Only One Data Point
The accuracy trends chart showed only one data point despite having
executions across multiple days. Root cause: aggregation required ≥3
executions per day.
### Issue 3: Incorrect Type Definitions
Manually maintained types defined `started_at` and `ended_at` as
non-nullable `Date`, contradicting reality where QUEUED executions
haven't started yet.
## Solution
### Database Schema (`schema.prisma`)
```prisma
model AgentGraphExecution {
// ...
startedAt DateTime?
endedAt DateTime? // NEW FIELD
// ...
}
```
### Execution Lifecycle
- **QUEUED**: `startedAt = null`, `endedAt = null` (not started)
- **RUNNING**: `startedAt = set`, `endedAt = null` (in progress)
- **COMPLETED/FAILED/TERMINATED**: `startedAt = set`, `endedAt = set`
(finished)
### Migration Strategy
```sql
-- Add endedAt column
ALTER TABLE "AgentGraphExecution" ADD COLUMN "endedAt" TIMESTAMP(3);
-- Backfill ONLY terminal executions (prevents marking RUNNING executions as ended)
UPDATE "AgentGraphExecution"
SET "endedAt" = "updatedAt"
WHERE "endedAt" IS NULL
AND "executionStatus" IN ('COMPLETED', 'FAILED', 'TERMINATED');
```
## Changes by Component
### Backend
**`schema.prisma`**
- Added `endedAt` field to `AgentGraphExecution`
**`execution.py`**
- Made `started_at` and `ended_at` optional with Field descriptions
- Updated `from_db()` to use `endedAt` instead of `updatedAt`
- `update_graph_execution_stats()` sets `endedAt` when status becomes
terminal
**`execution_analytics_routes.py`**
- Removed `created_at`/`updated_at` from `ExecutionAnalyticsResult` (DB
metadata, not execution data)
- Kept only `started_at`/`ended_at` (actual execution runtime)
- Made settings global (avoid recreation)
- Moved OpenAI key validation to `_process_batch` (only check when LLM
actually runs)
**`analytics.py`**
- Fixed aggregation: `COUNT(*) >= 1` (was 3) - include all days with ≥1
execution
- Uses `createdAt` for chart grouping (when execution was queued)
**`late_execution_monitor.py`**
- Handle optional `started_at` with fallback to `datetime.min` for
sorting
- Display "Not started" when `started_at` is null
### Frontend
**Type Definitions**
- Fixed manually maintained `types.ts`: `started_at: Date | null` (was
non-nullable)
- Generated types were already correct
**Analytics Components**
- `AnalyticsResultsTable.tsx`: Show only `started_at`/`ended_at` in
2-column expandable grid
- `ExecutionAnalyticsForm.tsx`: Added filter explanation UI
**Monitoring Components** - Fixed null handling bugs:
- `OldAgentLibraryView.tsx`: Handle null in reduce function
- `agent-runs-selector-list.tsx`: Safe sorting with `?.getTime() ?? 0`
- `AgentFlowList.tsx`: Filter/sort with null checks
- `FlowRunsStatus.tsx`: Filter null timestamps
- `FlowRunsTimeline.tsx`: Filter executions with null timestamps before
rendering
- `monitoring/page.tsx`: Safe sorting
- `ActivityItem.tsx`: Fallback to "recently" for null timestamps
## Benefits
✅ **Accurate End Times**: `endedAt` is frozen when execution finishes,
not updated later
✅ **Type Safety**: Nullable types match reality, exposing real bugs
✅ **Better UX**: Chart shows all days with data (not just days with ≥3
executions)
✅ **Bug Fixes**: 7+ frontend components now handle null timestamps
correctly
✅ **Documentation**: Field descriptions explain when timestamps are null
## Testing
### Backend
```bash
cd autogpt_platform/backend
poetry run format # ✅ All checks passed
poetry run lint # ✅ All checks passed
```
### Frontend
```bash
cd autogpt_platform/frontend
pnpm format # ✅ All checks passed
pnpm lint # ✅ All checks passed
pnpm types # ✅ All type errors fixed
```
### Test Data Generation
Created script to generate 35 test executions across 7 days with
correctness scores:
```bash
poetry run python scripts/generate_test_analytics_data.py
```
## Migration Notes
⚠️ **Important**: The migration only backfills `endedAt` for executions
with terminal status (COMPLETED, FAILED, TERMINATED). Active executions
(QUEUED, RUNNING) correctly keep `endedAt = null`.
## Breaking Changes
None - this is backward compatible:
- `endedAt` is nullable, existing code that doesn't use it is unaffected
- Frontend already used generated types which were correct
- Migration safely backfills historical data
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Introduces explicit execution end-time tracking and normalizes
timestamp handling across backend and frontend.
>
> - Adds `endedAt` to `AgentGraphExecution` (schema + migration);
backfills terminal executions; sets `endedAt` on terminal status updates
> - Makes `GraphExecutionMeta.started_at/ended_at` optional; updates
`from_db()` to use DB `endedAt`; exposes timestamps in
`ExecutionAnalyticsResult`
> - Moves OpenAI key validation into batch processing; instantiates
`Settings` once
> - Accuracy trends: reduce daily aggregation threshold to `>= 1`;
optional historical series
> - Monitoring/analytics UI: results table shows/export
`started_at`/`ended_at`; adds chart filter explainer
> - Frontend null-safety: update types (`Date | null`) and fix
sorting/filtering/rendering for nullable timestamps across monitoring
and library views
> - Late execution monitor: safe sorting/display when `started_at` is
null
> - OpenAPI specs updated for new/nullable fields
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
|
||
|
Zamil Majdy
|
8b83bb8647 |
feat(backend): unified hybrid search with embedding backfill for all content types (#11767)
## Summary This PR extends the embedding system to support **blocks** and **documentation** content types in addition to store agents, and introduces **unified hybrid search** across all content types using a single `UnifiedContentEmbedding` table. ### Key Changes 1. **Unified Hybrid Search Architecture** - Added `search` tsvector column to `UnifiedContentEmbedding` table - New `unified_hybrid_search()` function searches across all content types (agents, blocks, docs) - Updated `hybrid_search()` for store agents to use `UnifiedContentEmbedding.search` - Removed deprecated `search` column from `StoreListingVersion` table 2. **Pluggable Content Handler Architecture** - Created abstract `ContentHandler` base class for extensibility - Implemented handlers: `StoreAgentHandler`, `BlockHandler`, `DocumentationHandler` - Registry pattern for easy addition of new content types 3. **Block Embeddings** - Discovers all blocks using `get_blocks()` - Extracts searchable text from: name, description, categories, input/output schemas 4. **Documentation Embeddings** - Scans `/docs/` directory for `.md` and `.mdx` files - Extracts title from first `#` heading or uses filename as fallback 5. **Hybrid Search Graceful Degradation** - Falls back to lexical-only search if query embedding generation fails - Redistributes semantic weight proportionally to other components - Logs warning instead of throwing error 6. **Database Migrations** - `20260115200000_add_unified_search_tsvector`: Adds search column to UnifiedContentEmbedding with auto-update trigger - `20260115210000_remove_storelistingversion_search`: Removes deprecated search column and updates StoreAgent view 7. **Orphan Cleanup** - `cleanup_orphaned_embeddings()` removes embeddings for deleted content - Always runs after backfill, even at 100% coverage ### Review Comments Addressed - ✅ SQL parameter index bug when user_id provided (embeddings.py) - ✅ Early return skipping cleanup at 100% coverage (scheduler.py) - ✅ Inconsistent return structure across code paths (scheduler.py) - ✅ SQL UNION syntax error - added parentheses for ORDER BY/LIMIT (hybrid_search.py) - ✅ Version numeric ordering in aggregations (migration) - ✅ Embedding dimension uses EMBEDDING_DIM constant ### Files Changed - `backend/api/features/store/content_handlers.py` (NEW): Handler architecture - `backend/api/features/store/embeddings.py`: Refactored to use handlers - `backend/api/features/store/hybrid_search.py`: Unified search + graceful degradation - `backend/executor/scheduler.py`: Process all content types, consistent returns - `migrations/20260115200000_add_unified_search_tsvector/`: Add tsvector to unified table - `migrations/20260115210000_remove_storelistingversion_search/`: Remove old search column - `schema.prisma`: Updated UnifiedContentEmbedding and StoreListingVersion models - `*_test.py`: Added tests for unified_hybrid_search ## Test Plan 1. ✅ All tests passing on Python 3.11, 3.12, 3.13 2. ✅ Types check passing 3. ✅ CodeRabbit and Sentry reviews addressed 4. Deploy to staging and verify: - Backfill job processes all content types - Search results include blocks and docs - Search works without OpenAI API (graceful degradation) 🤖 Generated with [Claude Code](https://claude.ai/code) --------- Co-authored-by: Swifty <craigswift13@gmail.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Swifty
|
3b1b2fe30c |
feat(backend): Extract backend copilot/chat enhancements from hackathon (#11719)
This PR extracts backend changes from the hackathon/copilot branch, adding enhanced chat capabilities, agent management tools, store embeddings, and hybrid search functionality. ### Changes 🏗️ **Chat Features:** - Added chat database layer (`db.py`) for conversation and message persistence - Extended chat models with new types and response structures - New onboarding system prompt for guided user experiences - Enhanced chat routes with additional endpoints - Expanded chat service with more capabilities **Chat Agent Tools:** - `agent_output.py` - Handle agent execution outputs - `create_agent.py` - Tool for creating new agents via chat - `edit_agent.py` - Tool for modifying existing agents - `find_library_agent.py` - Search and discover library agents - Enhanced `run_agent.py` with additional functionality - New `models.py` for shared tool types **Store Enhancements:** - `embeddings.py` - Vector embeddings support for semantic search - `hybrid_search.py` - Combined keyword and semantic search - `backfill_embeddings.py` - Utility for backfilling existing data - Updated store database operations **Admin:** - Enhanced store admin routes **Data Layer:** - New `understanding.py` module for agent understanding/context **Database Migrations:** - `add_chat_tables` - Chat conversation and message tables - `add_store_embeddings` - Embeddings storage for store items - `enhance_search` - Search index improvements ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Chat endpoints respond correctly - [x] Agent tools (create/edit/find/run) function properly - [x] Store embeddings and hybrid search work - [x] Database migrations apply cleanly #### For configuration changes: - [x] `.env.default` is updated or already compatible with my changes - [x] `docker-compose.yml` is updated or already compatible with my changes - [x] I have included a list of my configuration changes in the PR description (under **Changes**) --------- Co-authored-by: Torantulino <40276179@live.napier.ac.uk> |
||
|
Swifty
|
5ac941fe2f |
feat(backend): add hybrid search for store listings, docs and blocks (#11721)
This PR adds hybrid search functionality combining semantic embeddings with traditional text search for improved store listing discovery. ### Changes 🏗️ - Add `embeddings.py` - OpenAI-based embedding generation and similarity search - Add `hybrid_search.py` - Combines vector similarity with text matching for better search results - Add `backfill_embeddings.py` - Script to generate embeddings for existing store listings - Update `db.py` - Integrate hybrid search into store database queries - Update `schema.prisma` - Add embedding storage fields and indexes - Add migrations for embedding columns and HNSW index for vector search ### Architecture Decisions 🏛️ **Fail-Fast Approach (No Silent Fallbacks)** We explicitly chose NOT to implement graceful degradation when hybrid search fails. Here's why: ✅ **Benefits:** - Errors surface immediately → faster fixes - Tests verify hybrid search actually works (not just fallback) - Consistent search quality for all users - Forces proper infrastructure setup (API keys, database) ❌ **Why Not Fallback:** - Silent degradation hides production issues - Users get inconsistent results without knowing why - Tests can pass even when hybrid search is broken - Reduces operational visibility **How We Prevent Failures:** 1. Embedding generation in approval flow (db.py:1545) 2. Error logging with `logger.error` (not warning) 3. Clear error messages (ValueError explains what's wrong) 4. Comprehensive test coverage (9/9 tests passing) If embeddings fail, it indicates a real infrastructure issue (missing API key, OpenAI down, database issues) that needs immediate attention, not silent degradation. ### Test Coverage ✅ **All tests passing (1625 total):** - 9/9 hybrid_search tests (including fail-fast validation) - 3/3 db search integration tests - Full schema compatibility (public/platform schemas) - Error handling verification ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Test hybrid search returns relevant results - [x] Test embedding generation for new listings - [x] Test backfill script on existing data - [x] Verify search performance with embeddings - [x] Test fail-fast behavior when embeddings unavailable #### For configuration changes: - [x] `.env.default` is updated or already compatible with my changes - [x] `docker-compose.yml` is updated or already compatible with my changes - [x] Configuration: Requires `openai_internal_api_key` in secrets --------- Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Zamil Majdy
|
88731b1f76 |
feat(platform): marketplace update notifications with enhanced publishing workflow (#11630)
## Summary This PR implements a comprehensive marketplace update notification system that allows users to discover and update to newer agent versions, along with enhanced publishing workflows and UI improvements. <img width="1500" height="533" alt="image" src="https://github.com/user-attachments/assets/ee331838-d712-4718-b231-1f9ec21bcd8e" /> <img width="600" height="610" alt="image" src="https://github.com/user-attachments/assets/b881a7b8-91a5-460d-a159-f64765b339f1" /> <img width="1500" height="416" alt="image" src="https://github.com/user-attachments/assets/a2d61904-2673-4e44-bcc5-c47d36af7a38" /> <img width="1500" height="1015" alt="image" src="https://github.com/user-attachments/assets/2dd978c7-20cc-4230-977e-9c62157b9f23" /> ## Core Features ### 🔔 Marketplace Update Notifications - **Update detection**: Automatically detects when marketplace has newer agent versions than user's local copy - **Creator notifications**: Shows banners for creators with unpublished changes ready to publish - **Non-creator support**: Enables regular users to discover and update to newer marketplace versions - **Version comparison**: Intelligent logic comparing `graph_version` vs marketplace listing versions ### 📋 Enhanced Publishing Workflow - **Builder integration**: Added "Publish to Marketplace" button directly in the builder actions - **Unified banner system**: Consistent `MarketplaceBanners` component across library and marketplace pages - **Streamlined UX**: Fixed layout issues, improved button placement and styling - **Modal improvements**: Fixed thumbnail loading race conditions and infinite loop bugs ### 📚 Version History & Changelog - **Inline version history**: Added version changelog directly to marketplace agent pages - **Version comparison**: Clear display of available versions with current version highlighting - **Update mechanism**: Direct updates using `graph_version` parameter for accuracy ## Technical Implementation ### Backend Changes - **Database schema**: Added `agentGraphVersions` and `agentGraphId` fields to `StoreAgent` model - **API enhancement**: Updated store endpoints to expose graph version data for version comparison - **Data migration**: Fixed agent version field naming from `version` to `agentGraphVersions` - **Model updates**: Enhanced `LibraryAgentUpdateRequest` with `graph_version` field ### Frontend Architecture - **`useMarketplaceUpdate` hook**: Centralized marketplace update detection and creator identification - **`MarketplaceBanners` component**: Unified banner system with proper vertical layout and styling - **`AgentVersionChangelog` component**: Version history display for marketplace pages - **`PublishToMarketplace` component**: Builder integration with modal workflow ### Key Bug Fixes - **Thumbnail loading**: Fixed race condition where images wouldn't load on first modal open - **Infinite loops**: Used refs to prevent circular dependencies in `useThumbnailImages` hook - **Layout issues**: Fixed banner placement, removed duplicate breadcrumbs, corrected vertical layout - **Field naming**: Fixed `agent_version` vs `version` field inconsistencies across APIs ## Files Changed ### Backend - `autogpt_platform/backend/backend/server/v2/store/` - Enhanced store API with graph version data - `autogpt_platform/backend/backend/server/v2/library/` - Updated library API models - `autogpt_platform/backend/migrations/` - Database migrations for version fields - `autogpt_platform/backend/schema.prisma` - Schema updates for graph versions ### Frontend - `src/app/(platform)/components/MarketplaceBanners/` - New unified banner component - `src/app/(platform)/library/agents/[id]/components/` - Enhanced library views with banners - `src/app/(platform)/build/components/BuilderActions/` - Added marketplace publish button - `src/app/(platform)/marketplace/components/AgentInfo/` - Added inline version history - `src/components/contextual/PublishAgentModal/` - Fixed thumbnail loading and modal workflow ## User Experience Impact - **Better discovery**: Users automatically notified of newer agent versions - **Streamlined publishing**: Direct publish access from builder interface - **Reduced friction**: Fixed UI bugs, improved loading states, consistent design - **Enhanced transparency**: Inline version history on marketplace pages - **Creator workflow**: Better notifications for creators with unpublished changes ## Testing - ✅ Update banners appear correctly when marketplace has newer versions - ✅ Creator banners show for users with unpublished changes - ✅ Version comparison logic works with graph_version vs marketplace versions - ✅ Publish button in builder opens modal correctly with pre-populated data - ✅ Thumbnail images load properly on first modal open without infinite loops - ✅ Database migrations completed successfully with version field fixes - ✅ All existing tests updated and passing with new schema changes 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Lluis Agusti <hi@llu.lu> Co-authored-by: Ubbe <hi@ubbe.dev> Co-authored-by: Reinier van der Leer <pwuts@agpt.co> |
||
|
Reinier van der Leer
|
3dbc03e488 |
feat(platform): OAuth API & Single Sign-On (#11617)
We want to provide Single Sign-On for multiple AutoGPT apps that use the Platform as their backend. ### Changes 🏗️ Backend: - DB + logic + API for OAuth flow (w/ tests) - DB schema additions for OAuth apps, codes, and tokens - Token creation/validation/management logic - OAuth flow endpoints (app info, authorize, token exchange, introspect, revoke) - E2E OAuth API integration tests - Other OAuth-related endpoints (upload app logo, list owned apps, external `/me` endpoint) - App logo asset management - Adjust external API middleware to support auth with access token - Expired token clean-up job - Add `OAUTH_TOKEN_CLEANUP_INTERVAL_HOURS` setting (optional) - `poetry run oauth-tool`: dev tool to test the OAuth flows and register new OAuth apps - `poetry run export-api-schema`: dev tool to quickly export the OpenAPI schema (much quicker than spinning up the backend) Frontend: - Frontend UI for app authorization (`/auth/authorize`) - Re-redirect after login/signup - Frontend flow to batch-auth integrations on request of the client app (`/auth/integrations/setup-wizard`) - Debug `CredentialInputs` component - Add `/profile/oauth-apps` management page - Add `isOurProblem` flag to `ErrorCard` to hide action buttons when the error isn't our fault - Add `showTitle` flag to `CredentialsInput` to hide built-in title for layout reasons DX: - Add [API guide](https://github.com/Significant-Gravitas/AutoGPT/blob/pwuts/sso/docs/content/platform/integrating/api-guide.md) and [OAuth guide](https://github.com/Significant-Gravitas/AutoGPT/blob/pwuts/sso/docs/content/platform/integrating/oauth-guide.md) ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Manually verify test coverage of OAuth API tests - Test `/auth/authorize` using `poetry run oauth-tool test-server` - [x] Works - [x] Looks okay - Test `/auth/integrations/setup-wizard` using `poetry run oauth-tool test-server` - [x] Works - [x] Looks okay - Test `/profile/oauth-apps` page - [x] All owned OAuth apps show up - [x] Enabling/disabling apps works - [ ] ~~Uploading logos works~~ can only test this once deployed to dev #### For configuration changes: - [x] `.env.default` is updated or already compatible with my changes - [x] `docker-compose.yml` is updated or already compatible with my changes - [x] I have included a list of my configuration changes in the PR description (under **Changes**) |
||
|
Krzysztof Czerwinski
|
bd37fe946d |
feat(platform): Builder search history (#11457)
Preserve user searches in the new builder and cache search results for more efficiency. Search is saved, so the user can see their previous searches. ### Changes 🏗️ - Add `BuilderSearch` column&migration to save user search (with all filters) - Builder `db.py` now caches all search results using `@cached` and returns paginated results, so following pages are returned much quicker - Score and sort results - Update models&routes - Update frontend, so it works properly with modified endpoints - Frontend: store `serachId` and use it for subsequent searches, so we don't save partial searches (e.g. "b", "bl", ..., "block"). Search id is reset when user clears the search field. - Add clickable chips to the Suggestions builder tab - Add `HorizontalScroll` component (chips use it) ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Search works and is cached - [x] Search sorts results - [x] Searches are preserved properly --------- Co-authored-by: Reinier van der Leer <pwuts@agpt.co> |
||
|
Zamil Majdy
|
e4d0dbc283 |
feat(platform): add Agent Output Demo field to marketplace submission form (#11538)
## Summary - Add Agent Output Demo field to marketplace agent submission form, positioned below the Description field - Store agent output demo URLs in database for future CoPilot integration - Implement proper video/image ordering on marketplace pages - Add shared YouTube URL validation utility to eliminate code duplication ## Changes Made ### Frontend - **Agent submission form**: Added Agent Output Demo field with YouTube URL validation - **Edit agent form**: Added Agent Output Demo field for existing submissions - **Marketplace display**: Implemented proper video/image ordering: 1. YouTube/Overview video (if exists) 2. First image (hero) 3. Agent Output Demo (if exists) 4. Additional images - **Shared utilities**: Created `validateYouTubeUrl` function in `src/lib/utils.ts` ### Backend - **Database schema**: Added `agentOutputDemoUrl` field to `StoreListingVersion` model - **Database views**: Updated `StoreAgent` view to include `agent_output_demo` field - **API models**: Added `agent_output_demo_url` to submission requests and `agent_output_demo` to responses - **Database migration**: Added migration to create new column and update view - **Test files**: Updated all test files to include the new required field ## Test Plan - [x] Frontend form validation works correctly for YouTube URLs - [x] Database migration applies successfully - [x] Backend API accepts and returns the new field - [x] Marketplace displays videos in correct order - [x] Both frontend and backend formatting/linting pass - [x] All test files include required field to prevent failures 🤖 Generated with [Claude Code](https://claude.ai/code) --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Swifty
|
2cb6fd581c |
feat(platform): Integration management from external api (#11472)
Allow the external api to manage credentials
### Changes 🏗️
- add ability to external api to manage credentials
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] tested it works
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Introduces external API endpoints to manage integrations (OAuth
initiation/completion and credential CRUD), adds external OAuth state
fields, and new API key permissions/config.
>
> - **External API – Integrations**:
> - Add router `backend/server/external/routes/integrations.py` with
endpoints to:
> - `GET /v1/integrations/providers` list providers (incl. default
scopes)
> - `POST /v1/integrations/{provider}/oauth/initiate` and `POST
/oauth/complete` for external OAuth (custom callback, state)
> - `GET /v1/integrations/credentials` and `GET /{provider}/credentials`
to list credentials
> - `POST /{provider}/credentials` to create `api_key`, `user_password`,
`host_scoped` creds; `DELETE /{provider}/credentials/{cred_id}` to
delete
> - Wire router in `backend/server/external/api.py`.
> - **Auth/Permissions**:
> - Add `APIKeyPermission` values: `MANAGE_INTEGRATIONS`,
`READ_INTEGRATIONS`, `DELETE_INTEGRATIONS` (schema + migration +
OpenAPI).
> - **Data model / Store**:
> - Extend `OAuthState` with external-flow fields: `callback_url`,
`state_metadata`, `api_key_id`, `is_external`.
> - Update `IntegrationCredentialsStore.store_state_token(...)` to
accept/store external OAuth metadata.
> - **OAuth providers**:
> - Set GitHub handler `DEFAULT_SCOPES = ["repo"]` in
`integrations/oauth/github.py`.
> - **Config**:
> - Add `config.external_oauth_callback_origins` in
`backend/util/settings.py` to validate allowed OAuth callback origins.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
|
||
|
Zamil Majdy
|
7b951c977e |
feat(platform): implement graph-level Safe Mode toggle for HITL blocks (#11455)
## Summary This PR implements a graph-level Safe Mode toggle system for Human-in-the-Loop (HITL) blocks. When Safe Mode is ON (default), HITL blocks require manual review before proceeding. When OFF, they execute automatically. ## 🔧 Backend Changes - **Database**: Added `metadata` JSON column to `AgentGraph` table with migration - **API**: Updated `execute_graph` endpoint to accept `safe_mode` parameter - **Execution**: Enhanced execution context to use graph metadata as default with API override capability - **Auto-detection**: Automatically populate `has_human_in_the_loop` for graphs containing HITL blocks - **Block Detection**: HITL block ID: `8b2a7b3c-6e9d-4a5f-8c1b-2e3f4a5b6c7d` ## 🎨 Frontend Changes - **Component**: New `FloatingSafeModeToggle` with dual variants: - **White variant**: For library pages, integrates with action buttons - **Black variant**: For builders, floating positioned - **Integration**: Added toggles to both new/legacy builders and library pages - **API Integration**: Direct graph metadata updates via `usePutV1UpdateGraphVersion` - **Query Management**: React Query cache invalidation for consistent UI updates - **Conditional Display**: Toggle only appears when graph contains HITL blocks ## 🛠 Technical Implementation - **Safe Mode ON** (default): HITL blocks require manual review before proceeding - **Safe Mode OFF**: HITL blocks execute automatically without intervention - **Priority**: Backend API `safe_mode` parameter takes precedence over graph metadata - **Detection**: Auto-populates `has_human_in_the_loop` metadata field - **Positioning**: Proper z-index and responsive positioning for floating elements ## 🚧 Known Issues (Work in Progress) ### High Priority - [ ] **Toggle state persistence**: Always shows "ON" regardless of actual state - query invalidation issue - [ ] **LibraryAgent metadata**: Missing metadata field causing TypeScript errors - [ ] **Tooltip z-index**: Still covered by some UI elements despite high z-index ### Medium Priority - [ ] **HITL detection**: Logic needs improvement for reliable block detection - [ ] **Error handling**: Removing HITL blocks from graph causes save errors - [ ] **TypeScript**: Fix type mismatches between GraphModel and LibraryAgent ### Low Priority - [ ] **Frontend API**: Add `safe_mode` parameter to execution calls once OpenAPI is regenerated - [ ] **Performance**: Consider debouncing rapid toggle clicks ## 🧪 Test Plan - [ ] Verify toggle appears only when graph has HITL blocks - [ ] Test toggle persistence across page refreshes - [ ] Confirm API calls update graph metadata correctly - [ ] Validate execution behavior respects safe mode setting - [ ] Check styling consistency across builder and library contexts ## 🔗 Related - Addresses requirements for graph-level HITL configuration - Builds on existing FloatingReviewsPanel infrastructure - Integrates with existing graph metadata system 🤖 Generated with [Claude Code](https://claude.ai/code) |
||
|
Swifty
|
00148f4e3d |
feat(platform): add external api routes for store search and tool usage (#11463)
We want to allow external tools to explore the marketplace and use the chat agent tools ### Changes 🏗️ - add store api routes - add tool api routes ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: <!-- Put your test plan here: --> - [x] tested all endpoints work --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> |
||
|
Zamil Majdy
|
3d08c22dd5 |
feat(platform): add Human In The Loop block with review workflow (#11380)
## Summary This PR implements a comprehensive Human In The Loop (HITL) block that allows agents to pause execution and wait for human approval/modification of data before continuing. https://github.com/user-attachments/assets/c027d731-17d3-494c-85ca-97c3bf33329c ## Key Features - Added WAITING_FOR_REVIEW status to AgentExecutionStatus enum - Created PendingHumanReview database table for storing review requests - Implemented HumanInTheLoopBlock that extracts input data and creates review entries - Added API endpoints at /api/executions/review for fetching and reviewing pending data - Updated execution manager to properly handle waiting status and resume after approval ## Frontend Components - PendingReviewCard for individual review handling - PendingReviewsList for multiple reviews - FloatingReviewsPanel for graph builder integration - Integrated review UI into 3 locations: legacy library, new library, and graph builder ## Technical Implementation - Added proper type safety throughout with SafeJson handling - Optimized database queries using count functions instead of full data fetching - Fixed imports to be top-level instead of local - All formatters and linters pass ## Test plan - [ ] Test Human In The Loop block creation in graph builder - [ ] Test block execution pauses and creates pending review - [ ] Test review UI appears in all 3 locations - [ ] Test data modification and approval workflow - [ ] Test rejection workflow - [ ] Test execution resumes after approval 🤖 Generated with [Claude Code](https://claude.ai/code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Human-In-The-Loop review workflows to pause executions for human validation. * Users can approve or reject pending tasks, optionally editing submitted data and adding a message. * New "Waiting for Review" execution status with UI indicators across run lists, badges, and activity views. * Review management UI: pending review cards, list view, and a floating reviews panel for quick access. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Bently
|
dcecb17bd1 |
feat(backend): Remove deprecated LLM models and add migration script (#11331)
These models have become deprecated - deepseek-r1-distill-llama-70b - gemma2-9b-it - llama3-70b-8192 - llama3-8b-8192 - google/gemini-flash-1.5 I have removed them and setup a migration, the migration is to convert all the old versions of the model to new versions, the model changes will happen like so - llama3-70b-8192 → llama-3.3-70b-versatile - llama3-8b-8192 → llama-3.1-8b-instant - google/gemini-flash-1.5 → google/gemini-2.5-flash - deepseek-r1-distill-llama-70b → gpt-5-chat-latest - gemma2-9b-it → gpt-5-chat-latest ### Changes 🏗️ <!-- Concisely describe all of the changes made in this pull request: --> ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: <!-- Put your test plan here: --> - [x] Check to see if old models where removed - [x] Check to see if migration worked and converted old models to new one in graph |
||
|
Zamil Majdy
|
4922f88851 |
feat(backend/executor): Implement cascading stop for nested graph executions (#11277)
## Summary Fixes critical issue where child executions spawned by `AgentExecutorBlock` continue running after parent execution is stopped. Implements parent-child execution tracking and recursive cascading stop logic to ensure entire execution trees are terminated together. ## Background When a parent graph execution containing `AgentExecutorBlock` nodes is stopped, only the parent was terminated. Child executions continued running, leading to: - ❌ Orphaned child executions consuming credits - ❌ No user control over execution trees - ❌ Race conditions where children start after parent stops - ❌ Resource leaks from abandoned executions ## Core Changes ### 1. Database Schema (`schema.prisma` + migration) ```sql -- Add nullable parent tracking field ALTER TABLE "AgentGraphExecution" ADD COLUMN "parentGraphExecutionId" TEXT; -- Add self-referential foreign key with graceful deletion ALTER TABLE "AgentGraphExecution" ADD CONSTRAINT "AgentGraphExecution_parentGraphExecutionId_fkey" FOREIGN KEY ("parentGraphExecutionId") REFERENCES "AgentGraphExecution"("id") ON DELETE SET NULL ON UPDATE CASCADE; -- Add index for efficient child queries CREATE INDEX "AgentGraphExecution_parentGraphExecutionId_idx" ON "AgentGraphExecution"("parentGraphExecutionId"); ``` ### 2. Parent ID Propagation (`backend/blocks/agent.py`) ```python # Extract current graph execution ID and pass as parent to child execution = add_graph_execution( # ... other params parent_graph_exec_id=graph_exec_id, # NEW: Track parent relationship ) ``` ### 3. Data Layer (`backend/data/execution.py`) ```python async def get_child_graph_executions(parent_exec_id: str) -> list[GraphExecution]: """Get all child executions of a parent execution.""" children = await AgentGraphExecution.prisma().find_many( where={"parentGraphExecutionId": parent_exec_id, "isDeleted": False} ) return [GraphExecution.from_db(child) for child in children] ``` ### 4. Cascading Stop Logic (`backend/executor/utils.py`) ```python async def stop_graph_execution( user_id: str, graph_exec_id: str, wait_timeout: float = 15.0, cascade: bool = True, # NEW parameter ): # 1. Find all child executions if cascade: children = await _get_child_executions(graph_exec_id) # 2. Stop all children recursively in parallel if children: await asyncio.gather( *[stop_graph_execution(user_id, child.id, wait_timeout, True) for child in children], return_exceptions=True, # Don't fail parent if child fails ) # 3. Stop the parent execution # ... existing stop logic ``` ### 5. Race Condition Prevention (`backend/executor/manager.py`) ```python # Before executing queued child, check if parent was terminated if parent_graph_exec_id: parent_exec = get_db_client().get_graph_execution_meta(parent_graph_exec_id, user_id) if parent_exec and parent_exec.status == ExecutionStatus.TERMINATED: # Skip execution, mark child as terminated get_db_client().update_graph_execution_stats( graph_exec_id=graph_exec_id, status=ExecutionStatus.TERMINATED, ) return # Don't start orphaned child ``` ## How It Works ### Before (Broken) ``` User stops parent execution ↓ Parent terminates ✓ ↓ Child executions keep running ✗ ↓ User cannot stop children ✗ ``` ### After (Fixed) ``` User stops parent execution ↓ Query database for all children ↓ Recursively stop all children in parallel ↓ Wait for children to terminate ↓ Stop parent execution ↓ All executions in tree stopped ✓ ``` ### Race Prevention ``` Child in QUEUED status ↓ Parent stopped ↓ Child picked up by executor ↓ Pre-flight check: parent TERMINATED? ↓ Yes → Skip execution, mark child TERMINATED ↓ Child never runs ✓ ``` ## Edge Cases Handled ✅ **Deep nesting** - Recursive cascading handles multi-level trees ✅ **Queued children** - Pre-flight check prevents execution ✅ **Race conditions** - Child spawned during stop operation ✅ **Partial failures** - `return_exceptions=True` continues on error ✅ **Multiple children** - Parallel stop via `asyncio.gather()` ✅ **No parent** - Backward compatible (nullable field) ✅ **Already completed** - Existing status check handles it ## Performance Impact - **Stop operation**: O(depth) with parallel execution vs O(1) before - **Memory**: +36 bytes per execution (one UUID reference) - **Database**: +1 query per tree level, indexed for efficiency ## API Changes (Backward Compatible) ### `stop_graph_execution()` - New Optional Parameter ```python # Before async def stop_graph_execution(user_id: str, graph_exec_id: str, wait_timeout: float = 15.0) # After async def stop_graph_execution(user_id: str, graph_exec_id: str, wait_timeout: float = 15.0, cascade: bool = True) ``` **Default `cascade=True`** means existing callers get the new behavior automatically. ### `add_graph_execution()` - New Optional Parameter ```python async def add_graph_execution(..., parent_graph_exec_id: Optional[str] = None) ``` ## Security & Safety - ✅ **User verification** - Users can only stop their own executions (parent + children) - ✅ **No cycles** - Self-referential FK prevents infinite loops - ✅ **Graceful degradation** - Errors in child stops don't block parent stop - ✅ **Rate limits** - Existing execution rate limits still apply ## Testing Checklist ### Database Migration - [x] Migration runs successfully - [x] Prisma client regenerates without errors - [x] Existing tests pass ### Core Functionality - [ ] Manual test: Stop parent with running child → child stops - [ ] Manual test: Stop parent with queued child → child never starts - [ ] Unit test: Cascading stop with multiple children - [ ] Unit test: Deep nesting (3+ levels) - [ ] Integration test: Race condition prevention ## Breaking Changes **None** - All changes are backward compatible with existing code. ## Rollback Plan If issues arise: 1. **Code rollback**: Revert PR, redeploy 2. **Database rollback**: Drop column and constraints (non-destructive) --- **Note**: This branch contains additional unrelated changes from merging with `dev`. The core cascading stop feature involves only: - `schema.prisma` + migration - `backend/data/execution.py` - `backend/executor/utils.py` - `backend/blocks/agent.py` - `backend/executor/manager.py` All other file changes are from dev branch updates and not part of this feature. 🤖 Generated with [Claude Code](https://claude.ai/code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Nested graph executions: parent-child tracking and retrieval of child executions * **Improvements** * Cascading stop: stopping a parent optionally terminates child executions * Parent execution IDs propagated through runs and surfaced in logs * Per-user/graph concurrent execution limits enforced * **Bug Fixes** * Skip enqueuing children if parent is terminated; robust handling when parent-status checks fail * **Tests** * Updated tests to cover parent linkage in graph creation <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Bently
|
9db443960a |
feat(blocks/claude): Remove Claude 3.5 Sonnet and Haiku model (#11260)
Removes CLAUDE_3_5_SONNET and CLAUDE_3_5_HAIKU from LlmModel enum, model
metadata, and cost configuration since they are deprecated
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Verify the models are gone from the llm blocks
|
||
|
Swifty
|
d25997b4f2 |
Revert "Merge branch 'swiftyos/secrt-1709-store-provider-names-and-en… (#11225)
Changes to providers blocks to store in db ### Changes 🏗️ - revet change ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: <!-- Put your test plan here: --> - [x] I have reverted the merge |
||
|
Swifty
|
e72ce2f9e7 | Merge branch 'swiftyos/secrt-1709-store-provider-names-and-env-vars-in-db' into dev | ||
|
Swifty
|
bd7a79a920 | Merge branch 'swiftyos/secrt-1706-improve-store-search' into dev | ||
|
Swifty
|
fa2731bb8b | Merge branch 'dev' into swiftyos/secrt-1709-store-provider-names-and-env-vars-in-db | ||
|
Swifty
|
2dc0c97a52 | Add block registry and updated | ||
|
Zamil Majdy
|
0bb2b87c32 |
fix(backend): resolve UserBalance migration issues and credit spending bug (#11192)
## Summary
Fix critical UserBalance migration and spending issues affecting users
with credits from transaction history but no UserBalance records.
## Root Issues Fixed
### Issue 1: UserBalance Migration Complexity
- **Problem**: Complex data migration with timestamp logic issues and
potential race conditions
- **Solution**: Simplified to idempotent table creation only,
application handles auto-population
### Issue 2: Credit Spending Bug
- **Problem**: Users with $10.0 from transaction history couldn't spend
$0.16
- **Root Cause**: `_add_transaction` and `_enable_transaction` only
checked UserBalance table, returning 0 balance for users without records
- **Solution**: Enhanced both methods with transaction history fallback
logic
### Issue 3: Exception Handling Inconsistency
- **Problem**: Raw SQL unique violations raised different exception
types than Prisma ORM
- **Solution**: Convert raw SQL unique violations to
`UniqueViolationError` at source
## Changes Made
### Migration Cleanup
- **Idempotent operations**: Use `CREATE TABLE IF NOT EXISTS`, `CREATE
INDEX IF NOT EXISTS`
- **Inline foreign key**: Define constraint within `CREATE TABLE`
instead of separate `ALTER TABLE`
- **Removed data migration**: Application creates UserBalance records
on-demand
- **Safe to re-run**: No errors if table/index/constraint already exists
### Credit Logic Fixes
- **Enhanced `_add_transaction`**: Added transaction history fallback in
`user_balance_lock` CTE
- **Enhanced `_enable_transaction`**: Added same fallback logic for
payment fulfillment
- **Exception normalization**: Convert raw SQL unique violations to
`UniqueViolationError`
- **Simplified `onboarding_reward`**: Use standardized
`UniqueViolationError` catching
### SQL Fallback Pattern
```sql
COALESCE(
(SELECT balance FROM UserBalance WHERE userId = ? FOR UPDATE),
-- Fallback: compute from transaction history if UserBalance doesn't exist
(SELECT COALESCE(ct.runningBalance, 0)
FROM CreditTransaction ct
WHERE ct.userId = ? AND ct.isActive = true AND ct.runningBalance IS NOT NULL
ORDER BY ct.createdAt DESC LIMIT 1),
0
) as balance
```
## Impact
### Before
- ❌ Users with transaction history but no UserBalance couldn't spend
credits
- ❌ Migration had complex timestamp logic with potential bugs
- ❌ Raw SQL and Prisma exceptions handled differently
- ❌ Error: "Insufficient balance of $10.0, where this will cost $0.16"
### After
- ✅ Seamless spending for all users regardless of UserBalance record
existence
- ✅ Simple, idempotent migration that's safe to re-run
- ✅ Consistent exception handling across all credit operations
- ✅ Automatic UserBalance record creation during first transaction
- ✅ Backward compatible - existing users unaffected
## Business Value
- **Eliminates user frustration**: Users can spend their credits
immediately
- **Smooth migration path**: From old User.balance to new UserBalance
table
- **Better reliability**: Atomic operations with proper error handling
- **Maintainable code**: Consistent patterns across credit operations
## Test Plan
- [ ] Manual testing with users who have transaction history but no
UserBalance records
- [ ] Verify migration can be run multiple times safely
- [ ] Test spending credits works for all user scenarios
- [ ] Verify payment fulfillment (`_enable_transaction`) works correctly
- [ ] Add comprehensive test coverage for this scenario
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
Swifty
|
29895c290f | store providers in db | ||
|
Zamil Majdy
|
73c0b6899a |
fix(backend): Remove advisory locks for atomic credit operations (#11143)
## Problem High QPS failures on `spend_credits` operations due to lock contention from `pg_advisory_xact_lock` causing serialization and seconds of wait time. ## Solution Replace PostgreSQL advisory locks with atomic database operations using CTEs (Common Table Expressions). ### Key Changes - **Add persistent balance column** to User table for O(1) balance lookups - **Atomic CTE-based operations** for all credit transactions using UPDATE...RETURNING pattern - **Comprehensive concurrency tests** with 7 test scenarios including stress testing - **Remove all advisory lock usage** from the credit system ### Implementation Details 1. **Migration**: Adds balance column with backfill from transaction history 2. **Atomic Operations**: All credit operations now use single atomic CTEs that update balance and create transaction in one query 3. **Race Condition Prevention**: WHERE clauses in UPDATE statements ensure balance never goes negative 4. **BetaUserCredit Compatibility**: Preserved monthly refill logic with updated `_add_transaction` signature ### Performance Impact - ✅ Eliminated lock contention bottlenecks - ✅ O(1) balance lookups instead of O(n) transaction aggregation - ✅ Atomic operations prevent race conditions without locks - ✅ Supports high QPS without serialization delays ### Testing - All existing tests pass - New concurrency test suite (`credit_concurrency_test.py`) with: - Concurrent spends from same user - Insufficient balance handling - Mixed operations (spends, top-ups, balance checks) - Race condition prevention - Integer overflow protection - Stress testing with 100 concurrent operations ### Breaking Changes None - all existing APIs maintain compatibility 🤖 Generated with [Claude Code](https://claude.ai/code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Enhanced top‑up flows with top‑up types, clearer credit→dollar formatting, and idempotent onboarding rewards. * **Bug Fixes** * Fixed race conditions for concurrent spends/top‑ups, added integer‑overflow and underflow protection, stronger input validation, and improved refund/dispute handling. * **Refactor** * Persisted per‑user balance with atomic updates for reliable balances; admin history now prefetches balances. * **Tests** * Added extensive concurrency, refund, ceiling/underflow and migration test suites. * **Chores** * Database migration to add persisted user balance; APIKey status extended (SUSPENDED). <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Swifty <craigswift13@gmail.com> |
||
|
Swifty
|
773f545cfd | update existing rows when migration is ran | ||
|
Swifty
|
84ad4a9f95 | updated migration and query | ||
|
Swifty
|
794aee25ab | add full text search | ||
|
Krzysztof Czerwinski
|
9c8652b273 |
feat(backend): Whitelist Onboarding Agents (#11149)
Some agents aren't suitable for onboarding. This adds per-store agent setting to allow them for onboarding. In case no agent is allowed fallback to the former search. ### Changes 🏗️ - Add `useForOnboarding` to `StoreListing` model and `StoreAgent` view (with migration) - Remove filtering of agents with empty input schema or credentials ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Only allowed agents are displayed - [x] Fallback to the old system in case there aren't enough allowed agents |
||
|
Swifty
|
36f634c417 |
fix(backend): Update store agent view to return only latest version (#11065)
This PR fixes duplicate agent listings on the marketplace home page by updating the StoreAgent view to return only the latest approved version of each agent. ### Changes 🏗️ - Updated `StoreAgent` database view to filter for only the latest approved version per listing - Added CTE (Common Table Expression) `latest_versions` to efficiently determine the maximum version for each store listing - Modified the join logic to only include the latest version instead of all approved versions - Updated `versions` array field to contain only the single latest version **Technical details:** - The view now uses a `latest_versions` CTE that groups by `storeListingId` and finds `MAX(version)` for approved submissions - Join condition ensures only the latest version is included: `slv.version = lv.latest_version` - This prevents duplicate entries for agents with multiple approved versions ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Verified marketplace home page shows no duplicate agents - [x] Confirmed only latest version is displayed for agents with multiple approved versions - [x] Checked that agent details page still functions correctly - [x] Validated that run counts and ratings are still accurate #### For configuration changes: - [x] `.env.default` is updated or already compatible with my changes - [x] `docker-compose.yml` is updated or already compatible with my changes - [x] I have included a list of my configuration changes in the PR description (under **Changes**) |
||
|
Swifty
|
18e169aa51 |
feat(platform): Log Marketplace Search Terms (#11092)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Reinier van der Leer <Pwuts@users.noreply.github.com> |
||
|
Krzysztof Czerwinski
|
f33ec1f2ec |
feat(platform): New retention-focused tasks and wallet update (#10977)
### Changes 🏗️ - Rename wallet and update design - Update tasks and add Hidden Tasks section - Update onboarding backend code and related db migration - Add progress bar for some tasks ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] All tasks can be finished - [x] Finished tasks add correct amount of credits --------- Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co> Co-authored-by: Reinier van der Leer <pwuts@agpt.co> |
||
|
Zamil Majdy
|
7a331651ba |
feat(backend): enhance database indexes for AgentGraph and AgentGraphExecution performance (#10985)
## Summary Enhances database performance by improving indexes on `AgentGraph` and `AgentGraphExecution` tables for better query efficiency. ### Changes 🏗️ - **Database Schema**: Updated Prisma schema to enhance database indexes - Modified `AgentGraph` index from `[userId, isActive]` to `[userId, isActive, id, version]` for better compound query performance - Enhanced `AgentGraphExecution` index from `[userId]` to `[userId, isDeleted, createdAt]` to support filtered queries with sorting - **Migration**: Auto-generated Prisma migration to implement the index changes - Drops existing indexes: `AgentGraph_userId_isActive_idx` and `AgentGraphExecution_userId_idx` - Creates new compound indexes: `AgentGraph_userId_isActive_id_version_idx` and `AgentGraphExecution_userId_isDeleted_createdAt_idx` ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Verified migration runs successfully - [x] Confirmed database queries continue to work with new indexes - [x] Tested that existing functionality remains unaffected #### For configuration changes: - [x] `.env.default` is updated or already compatible with my changes - [x] `docker-compose.yml` is updated or already compatible with my changes - [x] I have included a list of my configuration changes in the PR description (under **Changes**) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Nicholas Tindle
|
dd84fb5c66 |
feat(platform): Add public share links for agent run results (#10938)
<!-- Clearly explain the need for these changes: --> This PR adds the ability for users to share their agent run results publicly via shareable links. Users can generate a public link that allows anyone to view the outputs of a specific agent execution without requiring authentication. This feature enables users to share their agent results with clients, colleagues, or the community. https://github.com/user-attachments/assets/5508f430-07d0-4cd3-87bc-301b0b005cce ### Changes 🏗️ #### Backend Changes - **Database Schema**: Added share tracking fields to `AgentGraphExecution` model in Prisma schema: - `isShared`: Boolean flag to track if execution is shared - `shareToken`: Unique token for the share URL - `sharedAt`: Timestamp when sharing was enabled - **API Endpoints**: Added three new REST endpoints in `/backend/backend/server/routers/v1.py`: - `POST /graphs/{graph_id}/executions/{graph_exec_id}/share`: Enable sharing for an execution - `DELETE /graphs/{graph_id}/executions/{graph_exec_id}/share`: Disable sharing - `GET /share/{share_token}`: Retrieve shared execution data (public endpoint) - **Data Models**: - Created `SharedExecutionResponse` model for public-safe execution data - Added `ShareRequest` and `ShareResponse` Pydantic models for type-safe API responses - Updated `GraphExecutionMeta` to include share status fields - **Security**: - All share management endpoints verify user ownership before allowing changes - Public endpoint only exposes OUTPUT block data, no intermediate execution details - Share tokens are UUIDs for security #### Frontend Changes - **ShareButton Component** (`/frontend/src/components/ShareButton.tsx`): - Modal dialog for managing share settings - Copy-to-clipboard functionality for share links - Clear warnings about public accessibility - Uses Orval-generated API hooks for enable/disable operations - **Share Page** (`/frontend/src/app/(no-navbar)/share/[token]/page.tsx`): - Clean, navigation-free page for viewing shared executions - Reuses existing `RunOutputs` component for consistent output rendering - Proper error handling for invalid/disabled share links - Loading states during data fetch - **API Integration**: - Fixed custom mutator to properly set Content-Type headers for POST requests with empty bodies - Generated TypeScript types via Orval for type-safe API calls ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: <!-- Test plan: --> - [x] Enable sharing for an agent execution and verify share link is generated - [x] Copy share link and verify it copies to clipboard - [x] Open share link in incognito/private browser and verify outputs are displayed - [x] Disable sharing and verify share link returns 404 - [x] Try to enable/disable sharing for another user's execution (should fail with 404) - [x] Verify share page shows proper loading and error states - [x] Test that only OUTPUT blocks are shown in shared view, no intermediate data = |
||
|
Zamil Majdy
|
33679f3ffe |
feat(platform): Add instructions field to agent submissions (#10931)
## Summary Added an optional "Instructions" field for agent submissions to help users understand how to run agents and what to expect. <img width="1000" alt="image" src="https://github.com/user-attachments/assets/015c4f0b-4bdd-48df-af30-9e52ad283e8b" /> <img width="1000" alt="image" src="https://github.com/user-attachments/assets/3242cee8-a4ad-4536-bc12-64b491a8ef68" /> <img width="1000" alt="image" src="https://github.com/user-attachments/assets/a9b63e1c-94c0-41a4-a44f-b9f98e446793" /> ### Changes Made **Backend:** - Added `instructions` field to `AgentGraph` and `StoreListingVersion` database models - Updated `StoreSubmission`, `LibraryAgent`, and related Pydantic models - Modified store submission API routes to handle instructions parameter - Updated all database functions to properly save/retrieve instructions field - Added graceful handling for cases where database doesn't yet have the field **Frontend:** - Added instructions field to agent submission flow (PublishAgentModal) - Positioned below "Recommended Schedule" section as specified - Added instructions display in library/run flow (RunAgentModal) - Positioned above credentials section with informative blue styling - Added proper form validation with 2000 character limit - Updated all TypeScript types and API client interfaces ### Key Features - ✅ Optional field - fully backward compatible - ✅ Proper positioning in both submission and run flows - ✅ Character limit validation (2000 chars) - ✅ User-friendly display with "How to use this agent" styling - ✅ Only shows when instructions are provided ### Testing - Verified Pydantic model validation works correctly - Confirmed schema validation enforces character limits - Tested graceful handling of missing database fields - Code formatting and linting completed ## Test plan - [ ] Test agent submission with instructions field - [ ] Test agent submission without instructions (backward compatibility) - [ ] Verify instructions display correctly in run modal - [ ] Test character limit validation - [ ] Verify database migrations work properly 🤖 Generated with [Claude Code](https://claude.ai/code) --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Reinier van der Leer
|
23eafa178c |
fix(backend/db): Unbreak store materialized views refresh job (#10906)
- Resolves #10898 ### Changes 🏗️ - Fix and re-create `refresh_store_materialized_views` DB function and its pg_cron job ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Migration applies without issues (locally) - [x] Refresh function can be run without issues (locally) |
||
|
Reinier van der Leer
|
6a86e70fd6 |
fix(backend/db): Keep CreditTransaction entries on User delete (#10917)
This is a non-critical improvement for bookkeeping purposes. ### Changes 🏗️ - Change `CreditTransaction` <- `User` relation to `ON DELETE NO ACTION` so that `CreditTransactions` are not automatically deleted when we delete a user's data. ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Migration applies without problems |
||
|
Reinier van der Leer
|
2ffd249aac |
fix(backend/external-api): Improve security & reliability of API key storage (#10796)
Our API key generation, storage, and verification system has a couple of issues that need to be ironed out before full-scale deployment. ### Changes 🏗️ - Move from unsalted SHA256 to salted Scrypt hashing for API keys - Avoid false-negative API key validation due to prefix collision - Refactor API key management code for clarity - [refactor(backend): Clean up API key DB & API code (#10797)](https://github.com/Significant-Gravitas/AutoGPT/pull/10797) - Rename models and properties in `backend.data.api_key` for clarity - Eliminate redundant/custom/boilerplate error handling/wrapping in API key endpoint call stack - Remove redundant/inaccurate `response_model` declarations from API key endpoints Dependencies for `autogpt_libs`: - Add `cryptography` as a dependency - Add `pyright` as a dev dependency ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - Performing these actions through the UI (still) works: - [x] Creating an API key - [x] Listing owned API keys - [x] Deleting an owned API key - [x] Newly created API key can be used in Swagger UI - [x] Existing API key can be used in Swagger UI - [x] Existing API key is re-encrypted with salt on use |
||
|
Zamil Majdy
|
46e0f6cc45 |
feat(platform): Add recommended run schedule for agent execution (#10827)
## Summary <img width="1000" alt="Screenshot 2025-09-02 at 9 46 49 PM" src="https://github.com/user-attachments/assets/d78100c7-7974-4d37-a788-757764d8b6b7" /> <img width="1000" alt="Screenshot 2025-09-02 at 9 20 24 PM" src="https://github.com/user-attachments/assets/cd092963-8e26-4198-b65a-4416b2307a50" /> <img width="1000" alt="Screenshot 2025-09-02 at 9 22 30 PM" src="https://github.com/user-attachments/assets/e16b3bdb-c48c-4dec-9281-b2a35b3e21d0" /> <img width="1000" alt="Screenshot 2025-09-02 at 9 20 38 PM" src="https://github.com/user-attachments/assets/11d74a39-f4b4-4fce-8d30-0e6a925f3a9b" /> • Added recommended schedule cron expression as an optional input throughout the platform • Implemented complete data flow from builder → store submission → agent library → run page • Fixed UI layout issues including button text overflow and ensured proper component reusability ## Changes ### Backend - Added `recommended_schedule_cron` field to `AgentGraph` schema and database migration - Updated API models (`LibraryAgent`, `MyAgent`, `StoreSubmissionRequest`) to include the new field - Enhanced store submission approval flow to persist recommended schedule to database ### Frontend - Added recommended schedule input to builder page (SaveControl component) with overflow-safe styling - Updated store submission modal (PublishAgentModal) with schedule configuration - Enhanced agent run page with schedule tip display and pre-filled schedule dialog - Refactored `CronSchedulerDialog` with discriminated union types for better reusability - Fixed layout issues including button text truncation and popover width constraints - Implemented robust cron expression parsing with 100% reversibility between UI and cron format 🤖 Generated with [Claude Code](https://claude.ai/code) --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Zamil Majdy
|
2e38f132e7 |
feat(backend/store): implement sub-agent approval support (#10842)
## Summary Implement comprehensive sub-agent approval flow following the business requirements from the flow diagram. <img width="1956" height="1448" alt="image" src="https://github.com/user-attachments/assets/8de35e5b-9d3e-4dc2-bff0-47b49dbebc83" /> ### Key Features - ✅ **Auto-approve sub-agents** when main agent is approved - ✅ **Handle all scenarios**: new listings, existing versions, missing versions - ✅ **Transaction safety** with atomic operations via database transactions - ✅ **Parallel processing** using asyncio.gather for performance optimization - ✅ **Hidden from store** with isAvailable=false for all sub-agents ### Implementation Details - **Replaced** `_get_missing_sub_store_listing` with comprehensive `_handle_sub_agent_approvals` - **Added** `_approve_sub_agent` function with early returns for clean, readable code flow - **Used** `transaction()` context manager to ensure data consistency across operations - **Process sub-agents in parallel** while maintaining transaction integrity ### Business Logic Flow 1. **Check if sub-agent is already listed** in store 2. **If not listed**: create new store listing with `isAvailable=false` 3. **If listed but not approved**: approve the correct version 4. **If correct version not listed**: create store listing version and approve it 5. **If already approved**: no action needed (early return) All sub-agents remain **hidden from public store** while being internally approved for system use. ## Files Changed - `backend/server/v2/store/db.py` - Core implementation of sub-agent approval logic ## Test Plan - [ ] Verify main agent approval triggers sub-agent approvals - [ ] Test all sub-agent scenarios: new, existing unapproved, existing approved - [ ] Confirm sub-agents remain hidden (`isAvailable=false`) - [ ] Validate transaction rollback on failures - [ ] Check parallel processing works correctly 🤖 Generated with [Claude Code](https://claude.ai/code) |
||
|
Reinier van der Leer
|
e16e69ca55 |
feat(library, executor): Make "Run Again" work with credentials (#10821)
- Resolves [OPEN-2549: Make "Run again" work with credentials in `AgentRunDetailsView`](https://linear.app/autogpt/issue/OPEN-2549/make-run-again-work-with-credentials-in-agentrundetailsview) - Resolves #10237 ### Changes 🏗️ - feat(frontend/library): Make "Run Again" button work for runs with credentials - feat(backend/executor): Store passed-in credentials on `GraphExecution` ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - Go to `/library/agents/[id]` for an agent with credentials inputs - Run the agent manually - [x] -> runs successfully - [x] -> "Run again" shows among the action buttons on the newly created run - Click "Run again" - [x] -> runs successfully |
||
|
Nicholas Tindle
|
2bb8e91040 |
feat(backend): Add user timezone support to backend (#10707)
Co-authored-by: Swifty <craigswift13@gmail.com> resolve issue #10692 where scheduled time and actual run |
||
|
Nicholas Tindle
|
76090f0ba2 |
feat(backend,frontend): Send applicant email on review response (#10718)
### Changes 🏗️ This PR implements email notifications for agent creators when their agent submissions are approved or rejected by an admin in the marketplace. Specifically, the changes include: - Added `AGENT_APPROVED` and `AGENT_REJECTED` notification types to `schema.prisma`. - Created `AgentApprovalData` and `AgentRejectionData` Pydantic models for notification data. - Configured the notification system to use immediate queues and new Jinja2 templates for these types. - Designed two new email templates: `agent_approved.html.jinja2` and `agent_rejected.html.jinja2`, with dynamic content for agent details, reviewer feedback, and relevant action links. - Modified the `review_store_submission` function to: - Include `User` and `Reviewer` data in the database query. - Construct and queue the appropriate email notification based on the approval/rejection status. - Ensure email sending failures do not block the agent review process. ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Approve an agent via the admin dashboard. - [x] Verify the agent creator receives an "Agent Approved" email with correct details and a link to the store. - [x] Reject an agent via the admin dashboard (providing a reason). - [x] Verify the agent creator receives an "Agent Rejected" email with correct details, the rejection reason, and a link to resubmit. - [x] Verify that if email sending fails (e.g., misconfigured SMTP), the agent approval/rejection process still completes successfully without error. <img width="664" height="975" alt="image" src="https://github.com/user-attachments/assets/d397f2dc-56eb-45ab-877e-b17f1fc234d1" /> <img width="664" height="975" alt="image" src="https://github.com/user-attachments/assets/25597752-f68c-46fe-8888-6c32f5dada01" /> --- Linear Issue: [SECRT-1168](https://linear.app/autogpt/issue/SECRT-1168) <a href="https://cursor.com/background-agent?bcId=bc-7394906c-0341-4bd0-8842-6d9d6f83c56c"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/open-in-cursor-dark.svg"> <source media="(prefers-color-scheme: light)" srcset="https://cursor.com/open-in-cursor-light.svg"> <img alt="Open in Cursor" src="https://cursor.com/open-in-cursor.svg"> </picture> </a> <a href="https://cursor.com/agents?id=bc-7394906c-0341-4bd0-8842-6d9d6f83c56c"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://cursor.com/open-in-web-dark.svg"> <source media="(prefers-color-scheme: light)" srcset="https://cursor.com/open-in-web-light.svg"> <img alt="Open in Web" src="https://cursor.com/open-in-web.svg"> </picture> </a> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> |
||
|
Abhimanyu Yadav
|
2610c4579f |
feat(platform/dashboard): Enable editing for agent submissions (#10545)
- resolves - https://github.com/Significant-Gravitas/AutoGPT/issues/10511 In this PR, I’ve added backend endpoints and a frontend UI for edit functionality on the Agent Dashboard. Now, users can update their store submission, if status is `PENDING` or `APPROVED`, but not for `REJECTED` and `DRAFT`. When users make changes to a pending status submission, the changes are made to the same version. However, when users make changes to an approved status submission, a new store listing version is created. Backend works something like this: <img width="866" height="832" alt="Screenshot 2025-08-15 at 9 39 02 AM" src="https://github.com/user-attachments/assets/209c60ac-8350-43c1-ba4c-7378d95ecba7" /> ### Changes - I’ve updated the `StoreSubmission` view to include `video_url` and `categories`. - I’ve added a new frontend UI for editing submissions. - I’ve created an endpoint for editing submissions. - I’ve added more end-to-end tests to ensure the edit submission functionality works as expected. ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] I have checked manually, everything is working perfectly. - [x] All e2e tests are also passing. --------- Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co> Co-authored-by: neo <neo.dowithless@gmail.com> Co-authored-by: Reinier van der Leer <pwuts@agpt.co> Co-authored-by: Swifty <craigswift13@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Ubbe <hi@ubbe.dev> Co-authored-by: Lluis Agusti <hi@llu.lu> |
||
|
Zamil Majdy
|
f81d7e6a56 |
feat(backend): Add Missing FK indexes and remove unused & redundant indexes (#10412)
### Changes 🏗️ This PR optimizes database performance by adding missing foreign key indexes, removing unused/redundant indexes, cleaning up all legacy untracked indexes, and adding performance indexes for materialized views to achieve 100% optimized database indexing. **Foreign Key Indexes Added:** - `AgentGraph`: `[forkedFromId, forkedFromVersion]` - For fork relationship queries - `AgentGraphExecution`: `[agentPresetId]` - For preset-based execution filtering - `AgentNodeExecution`: `[agentNodeId]` - For node execution lookups - `AgentNodeExecutionInputOutput`: `[agentPresetId]` - For preset input/output queries - `AgentPreset`: `[agentGraphId, agentGraphVersion]` & `[webhookId]` - For graph and webhook lookups - `LibraryAgent`: `[agentGraphId, agentGraphVersion]` & `[creatorId]` - For agent and creator queries - `StoreListing`: `[agentGraphId, agentGraphVersion]` - For marketplace agent lookups - `StoreListingReview`: `[reviewByUserId]` - For user review queries **Unused/Redundant Indexes Removed:** - `User.email` - Unused index identified by linter - `AnalyticsMetrics.userId` - Unused index causing write overhead - `AnalyticsDetails.type` - Redundant (covered by composite `[userId, type]`) - `APIKey.key`, `APIKey.status` - Unused indexes - Named index `"analyticsDetails"` - Converted to standard composite index **All Legacy Untracked Indexes Removed:** - `idx_store_listing_version_status` - Redundant with Prisma composite index - `idx_slv_agent` - Redundant with Prisma-managed `[agentGraphId, agentGraphVersion]` - `idx_store_listing_version_approved_listing` - Redundant with unique constraint - `StoreListing_agentId_owningUserId_idx` - Legacy index superseded by current strategy - `StoreListing_isDeleted_isApproved_idx` - Replaced by optimized composite index - `StoreListing_isDeleted_idx` - Redundant with composite index - `StoreListingVersion_agentId_agentVersion_isDeleted_idx` - Legacy index replaced - `idx_store_listing_approved` - Redundant with existing `[owningUserId, slug]` unique constraint and `[isDeleted, hasApprovedVersion]` index - `idx_slv_categories_gin` - Specialized array search index removed (can be re-added if category filtering is implemented) - `idx_profile_user` - Duplicate of Prisma-managed `Profile_userId_idx` **Materialized View Performance Indexes Added:** - `idx_mv_review_stats_rating` on `mv_review_stats(avg_rating DESC)` - Optimizes sorting agents by rating - `idx_mv_review_stats_count` on `mv_review_stats(review_count DESC)` - Optimizes sorting agents by review count **Result: 100% Optimized Database Indexing** - All database indexes are now defined and managed through Prisma schema - No more untracked indexes requiring manual SQL maintenance - Added performance indexes for materialized views used by marketplace views - Improved query performance for agent sorting and filtering - Enhanced maintainability and consistency across environments **Schema Comments Updated:** - Removed all references to dropped untracked indexes - Simplified documentation to reflect Prisma-only approach for regular tables - Added comprehensive documentation for materialized view indexes and their purposes - Maintained documentation for materialized view refresh strategy ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Schema changes compile successfully with Prisma - [x] Migration adds required FK indexes and materialized view performance indexes - [x] Migration drops all legacy indexes and redundant untracked indexes - [x] All pre-commit hooks pass (linting, formatting, type checking) - [x] No breaking changes to existing foreign key relationships - [x] Verified existing Prisma indexes cover all query patterns - [x] Schema comments comprehensively document all indexing strategy - [x] Materialized view performance indexes optimize marketplace sorting 🤖 Generated with [Claude Code](https://claude.ai/code) --------- Co-authored-by: Swifty <craigswift13@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Swifty
|
6dba6ec3e6 |
feat(backend): Add database index for improved query performance (#10411)
This PR adds a database index to improve query performance based on Supabase query performance insights and index recommendations. These indexes target frequently queried columns and relationships to reduce query execution time. ### Changes 🏗️ - Added index on `AgentNodeExecutionInputOutput.agentPresetId` ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Verified schema changes are valid Prisma syntax - [x] Confirmed indexes target frequently queried columns based on Supabase recommendations - [x] Ensured no duplicate indexes are created #### For configuration changes: - [x] `.env.example` is updated or already compatible with my changes - [x] `docker-compose.yml` is updated or already compatible with my changes - [x] I have included a list of my configuration changes in the PR description (under **Changes**) |
