## Summary
This PR introduces a complete cloud storage infrastructure and file
upload system that agents can use instead of passing base64 data
directly in inputs, while maintaining backward compatibility for the
builder's node inputs.
### Problem Statement
Currently, when agents need to process files, they pass base64-encoded
data directly in the input, which has several limitations:
1. **Size limitations**: Base64 encoding increases file size by ~33%,
making large files impractical
2. **Memory usage**: Large base64 strings consume significant memory
during processing
3. **Network overhead**: Base64 data is sent repeatedly in API requests
4. **Performance impact**: Encoding/decoding base64 adds processing
overhead
### Solution
This PR introduces a complete cloud storage infrastructure and new file
upload workflow:
1. **New cloud storage system**: Complete `CloudStorageHandler` with
async GCS operations
2. **New upload endpoint**: Agents upload files via `/files/upload` and
receive a `file_uri`
3. **GCS storage**: Files are stored in Google Cloud Storage with
user-scoped paths
4. **URI references**: Agents pass the `file_uri` instead of base64 data
5. **Block processing**: File blocks can retrieve actual file content
using the URI
### Changes Made
#### New Files Introduced:
- **`backend/util/cloud_storage.py`** - Complete cloud storage
infrastructure (545 lines)
- **`backend/util/cloud_storage_test.py`** - Comprehensive test suite
(471 lines)
#### Backend Changes:
- **New cloud storage infrastructure** in
`backend/util/cloud_storage.py`:
- Complete `CloudStorageHandler` class with async GCS operations
- Support for multiple cloud providers (GCS implemented, S3/Azure
prepared)
- User-scoped and execution-scoped file storage with proper
authorization
- Automatic file expiration with metadata-based cleanup
- Path traversal protection and comprehensive security validation
- Async file operations with proper error handling and logging
- **New `UploadFileResponse` model** in `backend/server/model.py`:
- Returns `file_uri` (GCS path like
`gcs://bucket/users/{user_id}/file.txt`)
- Includes `file_name`, `size`, `content_type`, `expires_in_hours`
- Proper Pydantic schema instead of dictionary response
- **New `upload_file` endpoint** in `backend/server/routers/v1.py`:
- Complete new endpoint for file upload with cloud storage integration
- Returns GCS path URI directly as `file_uri`
- Supports user-scoped file storage for proper isolation
- Maintains fallback to base64 data URI when GCS not configured
- File size validation, virus scanning, and comprehensive error handling
#### Frontend Changes:
- **Updated API client** in
`frontend/src/lib/autogpt-server-api/client.ts`:
- Modified return type to expect `file_uri` instead of `signed_url`
- Supports the new upload workflow
- **Enhanced file input component** in
`frontend/src/components/type-based-input.tsx`:
- **Builder nodes**: Still use base64 for immediate data retention
without expiration
- **Agent inputs**: Use the new upload endpoint and pass `file_uri`
references
- Maintains backward compatibility for existing workflows
#### Test Updates:
- **New comprehensive test suite** in
`backend/util/cloud_storage_test.py`:
- 27 test cases covering all cloud storage functionality
- Tests for file storage, retrieval, authorization, and cleanup
- Tests for path validation, security, and error handling
- Coverage for user-scoped, execution-scoped, and system storage
- **New upload endpoint tests** in `backend/server/routers/v1_test.py`:
- Tests for GCS path URI format (`gcs://bucket/path`)
- Tests for base64 fallback when GCS not configured
- Validates file upload, virus scanning, and size limits
- Tests user-scoped file storage and access control
### Benefits
1. **New Infrastructure**: Complete cloud storage system with
enterprise-grade features
2. **Scalability**: Supports larger files without base64 size penalties
3. **Performance**: Reduces memory usage and network overhead with async
operations
4. **Security**: User-scoped file storage with comprehensive access
control and path validation
5. **Flexibility**: Maintains base64 support for builder nodes while
providing URI-based approach for agents
6. **Extensibility**: Designed for multiple cloud providers (GCS, S3,
Azure)
7. **Reliability**: Automatic file expiration, cleanup, and robust error
handling
8. **Backward compatibility**: Existing builder workflows continue to
work unchanged
### Usage
**For Agent Inputs:**
```typescript
// 1. Upload file
const response = await api.uploadFile(file);
// 2. Pass file_uri to agent
const agentInput = { file_input: response.file_uri };
```
**For Builder Nodes (unchanged):**
```typescript
// Still uses base64 for immediate data retention
const nodeInput = { file_input: "data:image/jpeg;base64,..." };
```
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] All new cloud storage tests pass (27/27)
- [x] All upload file tests pass (7/7)
- [x] Full v1 router test suite passes (21/21)
- [x] All server tests pass (126/126)
- [x] Backend formatting and linting pass
- [x] Frontend TypeScript compilation succeeds
- [x] Verified GCS path URI format (`gcs://bucket/path`)
- [x] Tested fallback to base64 data URI when GCS not configured
- [x] Confirmed file upload functionality works in UI
- [x] Validated response schema matches Pydantic model
- [x] Tested agent workflow with file_uri references
- [x] Verified builder nodes still work with base64 data
- [x] Tested user-scoped file access control
- [x] Verified file expiration and cleanup functionality
- [x] Tested security validation and path traversal protection
#### For configuration changes:
- [x] No new configuration changes required
- [x] `.env.example` remains compatible
- [x] `docker-compose.yml` remains compatible
- [x] Uses existing GCS configuration from media storage
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude AI <claude@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
This is the frontend for AutoGPT's next generation
🧢 Getting Started
This project uses pnpm as the package manager via corepack. Corepack is a Node.js tool that automatically manages package managers without requiring global installations.
Prerequisites
Make sure you have Node.js 16.10+ installed. Corepack is included with Node.js by default.
⚠️ Migrating from yarn
This project was previously using yarn1, make sure to clean up the old files if you set it up previously with yarn:
rm -f yarn.lock && rm -rf node_modulesThen follow the setup steps below.
Setup
-
Enable corepack (run this once on your system):
corepack enableThis enables corepack to automatically manage pnpm based on the
packageManagerfield inpackage.json. -
Install dependencies:
pnpm i -
Start the development server:
pnpm dev
Open http://localhost:3000 with your browser to see the result.
You can start editing the page by modifying app/page.tsx. The page auto-updates as you edit the file.
Subsequent Runs
For subsequent development sessions, you only need to run:
pnpm dev
Every time a new Front-end dependency is added by you or others, you will need to run pnpm i to install the new dependencies.
Available Scripts
pnpm dev- Start development serverpnpm build- Build for productionpnpm start- Start production serverpnpm lint- Run ESLint and Prettier checkspnpm format- Format code with Prettierpnpm type-check- Run TypeScript type checkingpnpm test- Run Playwright testspnpm test-ui- Run Playwright tests with UIpnpm fetch:openapi- Fetch OpenAPI spec from backendpnpm generate:api-client- Generate API client from OpenAPI specpnpm generate:api-all- Fetch OpenAPI spec and generate API client
This project uses next/font to automatically optimize and load Inter, a custom Google Font.
🔄 Data Fetching Strategy
Note
You don't need to run the OpenAPI commands below to run the Front-end. You will only need to run them when adding or modifying endpoints on the Backend API and wanting to use those on the Frontend.
This project uses an auto-generated API client powered by Orval, which creates type-safe API clients from OpenAPI specifications.
How It Works
- Backend Requirements: Each API endpoint needs a summary and tag in the OpenAPI spec
- Operation ID Generation: FastAPI generates operation IDs using the pattern
{method}{tag}{summary} - Spec Fetching: The OpenAPI spec is fetched from
http://localhost:8006/openapi.jsonand saved to the frontend - Spec Transformation: The OpenAPI spec is cleaned up using a custom transformer (see
autogpt_platform/frontend/src/app/api/transformers) - Client Generation: Auto-generated client includes TypeScript types, API endpoints, and Zod schemas, organized by tags
API Client Commands
# Fetch OpenAPI spec from backend and generate client
pnpm generate:api-all
# Only fetch the OpenAPI spec
pnpm fetch:openapi
# Only generate the client (after spec is fetched)
pnpm generate:api-client
Using the Generated Client
The generated client provides React Query hooks for both queries and mutations:
Queries (GET requests)
import { useGetV1GetNotificationPreferences } from "@/app/api/__generated__/endpoints/auth/auth";
const { data, isLoading, isError } = useGetV1GetNotificationPreferences({
query: {
select: (res) => res.data,
// Other React Query options
},
});
Mutations (POST, PUT, DELETE requests)
import { useDeleteV2DeleteStoreSubmission } from "@/app/api/__generated__/endpoints/store/store";
import { getGetV2ListMySubmissionsQueryKey } from "@/app/api/__generated__/endpoints/store/store";
import { useQueryClient } from "@tanstack/react-query";
const queryClient = useQueryClient();
const { mutateAsync: deleteSubmission } = useDeleteV2DeleteStoreSubmission({
mutation: {
onSuccess: () => {
// Invalidate related queries to refresh data
queryClient.invalidateQueries({
queryKey: getGetV2ListMySubmissionsQueryKey(),
});
},
},
});
// Usage
await deleteSubmission({
submissionId: submission_id,
});
Server Actions
For server-side operations, you can also use the generated client functions directly:
import { postV1UpdateNotificationPreferences } from "@/app/api/__generated__/endpoints/auth/auth";
// In a server action
const preferences = {
email: "user@example.com",
preferences: {
AGENT_RUN: true,
ZERO_BALANCE: false,
// ... other preferences
},
daily_limit: 0,
};
await postV1UpdateNotificationPreferences(preferences);
Server-Side Prefetching
For server-side components, you can prefetch data on the server and hydrate it in the client cache. This allows immediate access to cached data when queries are called:
import { getQueryClient } from "@/lib/tanstack-query/getQueryClient";
import {
prefetchGetV2ListStoreAgentsQuery,
prefetchGetV2ListStoreCreatorsQuery
} from "@/app/api/__generated__/endpoints/store/store";
import { HydrationBoundary, dehydrate } from "@tanstack/react-query";
// In your server component
const queryClient = getQueryClient();
await Promise.all([
prefetchGetV2ListStoreAgentsQuery(queryClient, {
featured: true,
}),
prefetchGetV2ListStoreAgentsQuery(queryClient, {
sorted_by: "runs",
}),
prefetchGetV2ListStoreCreatorsQuery(queryClient, {
featured: true,
sorted_by: "num_agents",
}),
]);
return (
<HydrationBoundary state={dehydrate(queryClient)}>
<MainMarkeplacePage />
</HydrationBoundary>
);
This pattern improves performance by serving pre-fetched data from the server while maintaining the benefits of client-side React Query features.
Configuration
The Orval configuration is located in autogpt_platform/frontend/orval.config.ts. It generates two separate clients:
- autogpt_api_client: React Query hooks for client-side data fetching
- autogpt_zod_schema: Zod schemas for validation
For more details, see the Orval documentation or check the configuration file.
🚚 Deploy
TODO
📙 Storybook
Storybook is a powerful development environment for UI components. It allows you to build UI components in isolation, making it easier to develop, test, and document your components independently from your main application.
Purpose in the Development Process
- Component Development: Develop and test UI components in isolation.
- Visual Testing: Easily spot visual regressions.
- Documentation: Automatically document components and their props.
- Collaboration: Share components with your team or stakeholders for feedback.
How to Use Storybook
-
Start Storybook: Run the following command to start the Storybook development server:
pnpm storybookThis will start Storybook on port 6006. Open http://localhost:6006 in your browser to view your component library.
-
Build Storybook: To build a static version of Storybook for deployment, use:
pnpm build-storybook -
Running Storybook Tests: Storybook tests can be run using:
pnpm test-storybook -
Writing Stories: Create
.stories.tsxfiles alongside your components to define different states and variations of your components.
By integrating Storybook into our development workflow, we can streamline UI development, improve component reusability, and maintain a consistent design system across the project.
🔭 Tech Stack
Core Framework & Language
- Next.js - React framework with App Router
- React - UI library for building user interfaces
- TypeScript - Typed JavaScript for better developer experience
Styling & UI Components
- Tailwind CSS - Utility-first CSS framework
- shadcn/ui - Re-usable components built with Radix UI and Tailwind CSS
- Radix UI - Headless UI components for accessibility
- Lucide React - Beautiful & consistent icons
- Framer Motion - Animation library for React
Development & Testing
- Storybook - Component development environment
- Playwright - End-to-end testing framework
- ESLint - JavaScript/TypeScript linting
- Prettier - Code formatting
Backend & Services
- Supabase - Backend-as-a-Service (database, auth, storage)
- Sentry - Error monitoring and performance tracking
Package Management
Additional Libraries
- React Hook Form - Forms with easy validation
- Zod - TypeScript-first schema validation
- React Table - Headless table library
- React Flow - Interactive node-based diagrams
- React Query - Data fetching and caching
- React Query DevTools - Debugging tool for React Query
Development Tools
NEXT_PUBLIC_REACT_QUERY_DEVTOOL- Enable React Query DevTools. Set totrueto enable.