### Changes 🏗️ This PR optimizes database performance by adding missing foreign key indexes, removing unused/redundant indexes, cleaning up all legacy untracked indexes, and adding performance indexes for materialized views to achieve 100% optimized database indexing. **Foreign Key Indexes Added:** - `AgentGraph`: `[forkedFromId, forkedFromVersion]` - For fork relationship queries - `AgentGraphExecution`: `[agentPresetId]` - For preset-based execution filtering - `AgentNodeExecution`: `[agentNodeId]` - For node execution lookups - `AgentNodeExecutionInputOutput`: `[agentPresetId]` - For preset input/output queries - `AgentPreset`: `[agentGraphId, agentGraphVersion]` & `[webhookId]` - For graph and webhook lookups - `LibraryAgent`: `[agentGraphId, agentGraphVersion]` & `[creatorId]` - For agent and creator queries - `StoreListing`: `[agentGraphId, agentGraphVersion]` - For marketplace agent lookups - `StoreListingReview`: `[reviewByUserId]` - For user review queries **Unused/Redundant Indexes Removed:** - `User.email` - Unused index identified by linter - `AnalyticsMetrics.userId` - Unused index causing write overhead - `AnalyticsDetails.type` - Redundant (covered by composite `[userId, type]`) - `APIKey.key`, `APIKey.status` - Unused indexes - Named index `"analyticsDetails"` - Converted to standard composite index **All Legacy Untracked Indexes Removed:** - `idx_store_listing_version_status` - Redundant with Prisma composite index - `idx_slv_agent` - Redundant with Prisma-managed `[agentGraphId, agentGraphVersion]` - `idx_store_listing_version_approved_listing` - Redundant with unique constraint - `StoreListing_agentId_owningUserId_idx` - Legacy index superseded by current strategy - `StoreListing_isDeleted_isApproved_idx` - Replaced by optimized composite index - `StoreListing_isDeleted_idx` - Redundant with composite index - `StoreListingVersion_agentId_agentVersion_isDeleted_idx` - Legacy index replaced - `idx_store_listing_approved` - Redundant with existing `[owningUserId, slug]` unique constraint and `[isDeleted, hasApprovedVersion]` index - `idx_slv_categories_gin` - Specialized array search index removed (can be re-added if category filtering is implemented) - `idx_profile_user` - Duplicate of Prisma-managed `Profile_userId_idx` **Materialized View Performance Indexes Added:** - `idx_mv_review_stats_rating` on `mv_review_stats(avg_rating DESC)` - Optimizes sorting agents by rating - `idx_mv_review_stats_count` on `mv_review_stats(review_count DESC)` - Optimizes sorting agents by review count **Result: 100% Optimized Database Indexing** - All database indexes are now defined and managed through Prisma schema - No more untracked indexes requiring manual SQL maintenance - Added performance indexes for materialized views used by marketplace views - Improved query performance for agent sorting and filtering - Enhanced maintainability and consistency across environments **Schema Comments Updated:** - Removed all references to dropped untracked indexes - Simplified documentation to reflect Prisma-only approach for regular tables - Added comprehensive documentation for materialized view indexes and their purposes - Maintained documentation for materialized view refresh strategy ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Schema changes compile successfully with Prisma - [x] Migration adds required FK indexes and materialized view performance indexes - [x] Migration drops all legacy indexes and redundant untracked indexes - [x] All pre-commit hooks pass (linting, formatting, type checking) - [x] No breaking changes to existing foreign key relationships - [x] Verified existing Prisma indexes cover all query patterns - [x] Schema comments comprehensively document all indexing strategy - [x] Materialized view performance indexes optimize marketplace sorting 🤖 Generated with [Claude Code](https://claude.ai/code) --------- Co-authored-by: Swifty <craigswift13@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>
AutoGPT Platform
Welcome to the AutoGPT Platform - a powerful system for creating and running AI agents to solve business problems. This platform enables you to harness the power of artificial intelligence to automate tasks, analyze data, and generate insights for your organization.
Getting Started
Prerequisites
- Docker
- Docker Compose V2 (comes with Docker Desktop, or can be installed separately)
- Node.js & NPM (for running the frontend application)
Running the System
To run the AutoGPT Platform, follow these steps:
-
Clone this repository to your local machine and navigate to the
autogpt_platformdirectory within the repository:git clone <https://github.com/Significant-Gravitas/AutoGPT.git | git@github.com:Significant-Gravitas/AutoGPT.git> cd AutoGPT/autogpt_platform -
Run the following command:
cp .env.example .envThis command will copy the
.env.examplefile to.env. You can modify the.envfile to add your own environment variables. -
Run the following command:
docker compose up -dThis command will start all the necessary backend services defined in the
docker-compose.ymlfile in detached mode. -
Navigate to
frontendwithin theautogpt_platformdirectory:cd frontendYou will need to run your frontend application separately on your local machine.
-
Run the following command:
cp .env.example .env.localThis command will copy the
.env.examplefile to.env.localin thefrontenddirectory. You can modify the.env.localwithin this folder to add your own environment variables for the frontend application. -
Run the following command:
Enable corepack and install dependencies by running:
corepack enable pnpm iGenerate the API client (this step is required before running the frontend):
pnpm generate:api-clientThen start the frontend application in development mode:
pnpm dev -
Open your browser and navigate to
http://localhost:3000to access the AutoGPT Platform frontend.
Docker Compose Commands
Here are some useful Docker Compose commands for managing your AutoGPT Platform:
docker compose up -d: Start the services in detached mode.docker compose stop: Stop the running services without removing them.docker compose rm: Remove stopped service containers.docker compose build: Build or rebuild services.docker compose down: Stop and remove containers, networks, and volumes.docker compose watch: Watch for changes in your services and automatically update them.
Sample Scenarios
Here are some common scenarios where you might use multiple Docker Compose commands:
-
Updating and restarting a specific service:
docker compose build api_srv docker compose up -d --no-deps api_srvThis rebuilds the
api_srvservice and restarts it without affecting other services. -
Viewing logs for troubleshooting:
docker compose logs -f api_srv ws_srvThis shows and follows the logs for both
api_srvandws_srvservices. -
Scaling a service for increased load:
docker compose up -d --scale executor=3This scales the
executorservice to 3 instances to handle increased load. -
Stopping the entire system for maintenance:
docker compose stop docker compose rm -f docker compose pull docker compose up -dThis stops all services, removes containers, pulls the latest images, and restarts the system.
-
Developing with live updates:
docker compose watchThis watches for changes in your code and automatically updates the relevant services.
-
Checking the status of services:
docker compose psThis shows the current status of all services defined in your docker-compose.yml file.
These scenarios demonstrate how to use Docker Compose commands in combination to manage your AutoGPT Platform effectively.
Persisting Data
To persist data for PostgreSQL and Redis, you can modify the docker-compose.yml file to add volumes. Here's how:
-
Open the
docker-compose.ymlfile in a text editor. -
Add volume configurations for PostgreSQL and Redis services:
services: postgres: # ... other configurations ... volumes: - postgres_data:/var/lib/postgresql/data redis: # ... other configurations ... volumes: - redis_data:/data volumes: postgres_data: redis_data: -
Save the file and run
docker compose up -dto apply the changes.
This configuration will create named volumes for PostgreSQL and Redis, ensuring that your data persists across container restarts.
API Client Generation
The platform includes scripts for generating and managing the API client:
pnpm fetch:openapi: Fetches the OpenAPI specification from the backend service (requires backend to be running on port 8006)pnpm generate:api-client: Generates the TypeScript API client from the OpenAPI specification using Orvalpnpm generate:api-all: Runs both fetch and generate commands in sequence
Manual API Client Updates
If you need to update the API client after making changes to the backend API:
-
Ensure the backend services are running:
docker compose up -d -
Generate the updated API client:
pnpm generate:api-all
This will fetch the latest OpenAPI specification and regenerate the TypeScript client code.