Files
AutoGPT/autogpt_platform/CLAUDE.md
Nicholas Tindle 568f5a449e feat(backend): cache control headers (#10160)
### Why? 🤔
  <!-- Clearly explain the need for these changes: -->
  We need to prevent sensitive data (authentication tokens, API
  keys, user credentials, personal information) from being cached by
   browsers and proxies. Following the principle of "secure by
  default", we're switching from a deny list to an allow list
  approach for cache control.

  ### Changes 🛠️
  <!-- Concisely describe all of the changes made in this pull
  request: -->
  - **Refactored cache control middleware from deny list to allow 
  list approach**
    - By default, ALL endpoints now have `Cache-Control: no-store,
  no-cache, must-revalidate, private` headers
    - Only explicitly allowed paths (static assets, health checks,
  public store pages) can be cached
    - This ensures new endpoints are automatically protected without
   developers having to remember to add them to a list

  - **Updated `SecurityHeadersMiddleware` in 
  `/backend/backend/server/middleware/security.py`**
    - Renamed `SENSITIVE_PATHS` to `CACHEABLE_PATHS`
    - Inverted the logic in `is_cacheable_path()` method
    - Cache control headers are now applied to all paths NOT in the
  allow list

  - **Updated test suite to match new behavior**
    - Tests now verify that most endpoints have cache control
  headers by default
    - Tests verify that only allowed paths (static assets, health
  endpoints, etc.) can be cached

  - **Updated documentation in `CLAUDE.md`**
    - Documented the new allow list approach
    - Added instructions for developers on how to allow caching for
  new endpoints


### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] Test modified endpoints work still
  - [x] Test modified endpoints correctly have no cache rules

---------

Co-authored-by: Swifty <craigswift13@gmail.com>
2025-06-20 14:19:52 +00:00

4.7 KiB

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Repository Overview

AutoGPT Platform is a monorepo containing:

  • Backend (/backend): Python FastAPI server with async support
  • Frontend (/frontend): Next.js React application
  • Shared Libraries (/autogpt_libs): Common Python utilities

Essential Commands

Backend Development

# Install dependencies
cd backend && poetry install

# Run database migrations
poetry run prisma migrate dev

# Start all services (database, redis, rabbitmq)
docker compose up -d

# Run the backend server
poetry run serve

# Run tests
poetry run test

# Run specific test
poetry run pytest path/to/test_file.py::test_function_name

# Lint and format
# prefer format if you want to just "fix" it and only get the errors that can't be autofixed
poetry run format  # Black + isort
poetry run lint    # ruff

More details can be found in TESTING.md

Creating/Updating Snapshots

When you first write a test or when the expected output changes:

poetry run pytest path/to/test.py --snapshot-update

⚠️ Important: Always review snapshot changes before committing! Use git diff to verify the changes are expected.

Frontend Development

# Install dependencies
cd frontend && npm install

# Start development server
npm run dev

# Run E2E tests
npm run test

# Run Storybook for component development
npm run storybook

# Build production
npm run build

# Type checking
npm run type-check

Architecture Overview

Backend Architecture

  • API Layer: FastAPI with REST and WebSocket endpoints
  • Database: PostgreSQL with Prisma ORM, includes pgvector for embeddings
  • Queue System: RabbitMQ for async task processing
  • Execution Engine: Separate executor service processes agent workflows
  • Authentication: JWT-based with Supabase integration
  • Security: Cache protection middleware prevents sensitive data caching in browsers/proxies

Frontend Architecture

  • Framework: Next.js App Router with React Server Components
  • State Management: React hooks + Supabase client for real-time updates
  • Workflow Builder: Visual graph editor using @xyflow/react
  • UI Components: Radix UI primitives with Tailwind CSS styling
  • Feature Flags: LaunchDarkly integration

Key Concepts

  1. Agent Graphs: Workflow definitions stored as JSON, executed by the backend
  2. Blocks: Reusable components in /backend/blocks/ that perform specific tasks
  3. Integrations: OAuth and API connections stored per user
  4. Store: Marketplace for sharing agent templates

Testing Approach

  • Backend uses pytest with snapshot testing for API responses
  • Test files are colocated with source files (*_test.py)
  • Frontend uses Playwright for E2E tests
  • Component testing via Storybook

Database Schema

Key models (defined in /backend/schema.prisma):

  • User: Authentication and profile data
  • AgentGraph: Workflow definitions with version control
  • AgentGraphExecution: Execution history and results
  • AgentNode: Individual nodes in a workflow
  • StoreListing: Marketplace listings for sharing agents

Environment Configuration

  • Backend: .env file in /backend
  • Frontend: .env.local file in /frontend
  • Both require Supabase credentials and API keys for various services

Common Development Tasks

Adding a new block:

  1. Create new file in /backend/backend/blocks/
  2. Inherit from Block base class
  3. Define input/output schemas
  4. Implement run method
  5. Register in block registry

Modifying the API:

  1. Update route in /backend/backend/server/routers/
  2. Add/update Pydantic models in same directory
  3. Write tests alongside the route file
  4. Run poetry run test to verify

Frontend feature development:

  1. Components go in /frontend/src/components/
  2. Use existing UI components from /frontend/src/components/ui/
  3. Add Storybook stories for new components
  4. Test with Playwright if user-facing

Security Implementation

Cache Protection Middleware:

  • Located in /backend/backend/server/middleware/security.py
  • Default behavior: Disables caching for ALL endpoints with Cache-Control: no-store, no-cache, must-revalidate, private
  • Uses an allow list approach - only explicitly permitted paths can be cached
  • Cacheable paths include: static assets (/static/*, /_next/static/*), health checks, public store pages, documentation
  • Prevents sensitive data (auth tokens, API keys, user data) from being cached by browsers/proxies
  • To allow caching for a new endpoint, add it to CACHEABLE_PATHS in the middleware
  • Applied to both main API server and external API applications