Files
AutoGPT/autogpt_platform
Zamil Majdy 57a06f7088 fix(blocks, security): Fixes for various DoS vulnerabilities (#10798)
This PR addresses multiple critical and medium security vulnerabilities
that could lead to Denial of Service (DoS) attacks. All fixes implement
defense-in-depth strategies with comprehensive testing.

### Changes 🏗️

#### **Critical Security Fixes:**

1. **GHSA-m2wr-7m3r-p52c - ReDoS in CodeExtractionBlock** 
- Fixed catastrophic backtracking in regex patterns `\s+[\s\S]*?` and
`\s+(.*?)`
   - Replaced with safer patterns: `[ \t]*\n([^\s\S]*?)`
   - Files: `backend/blocks/code_extraction_block.py`

2. **GHSA-955p-gpfx-r66j - AITextSummarizerBlock Memory Amplification**
   - Added 1MB text size limit and 100 chunk maximum
   - Prevents 10K input → 50G memory amplification attacks
   - Files: `backend/blocks/llm.py`

3. **GHSA-5cqw-g779-9f9x - RSS Feed XML Bomb DoS**
   - Added 10MB feed size limit and 30s timeout
   - Prevents deep XML parsing memory exhaustion
   - Files: `backend/blocks/rss.py`

4. **GHSA-7g34-7fvq-xxq6 - File Storage Disk Exhaustion**
   - Added 100MB per file and 1GB per execution directory limits
   - Prevents disk space exhaustion from file uploads
   - Files: `backend/util/file.py`

5. **GHSA-pppq-xx2w-7jpq - ExtractTextInformationBlock ReDoS**
   - Added 1MB text limit, 1000 match limit, and 5s timeout protection
   - Prevents lookahead pattern memory exhaustion
   - Files: `backend/blocks/text.py`

6. **GHSA-vw3v-whvp-33v5 - Docker Logging Disk Exhaustion**
- Added log rotation limits at Docker (10MB × 3 files) and application
levels
   - Prevents unbounded log growth causing disk exhaustion
- Files: `docker-compose.platform.yml`,
`autogpt_libs/autogpt_libs/logging/config.py`

#### **Additional Security Improvements:**

7. **StepThroughItemsBlock DoS Prevention**
   - Added 10,000 item limit and 1MB input size limit
   - Prevents large iteration DoS attacks
   - Files: `backend/blocks/iteration.py`

8. **XMLParserBlock XML Bomb Prevention**
   - Added 10MB XML input size limit
   - Files: `backend/blocks/xml_parser.py`

#### **Code Quality:**
- Fixed Python 3.10 typing compatibility issues
- Added comprehensive security test suite
- All code formatted and linted

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Created comprehensive security test suite covering all
vulnerabilities
  - [x] Verified ReDoS patterns are fixed and don't cause timeouts
  - [x] Confirmed memory limits prevent amplification attacks
  - [x] Tested file size limits prevent disk exhaustion
  - [x] Validated log rotation prevents unbounded growth
  - [x] Ensured backward compatibility for normal usage

#### For configuration changes:
- [x] `docker-compose.yml` is updated with logging limits
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

### Test Plan 🧪

**Security Tests:**
1. **ReDoS Protection**: Tested with malicious regex inputs (large
spaces) - completes without hanging
2. **Memory Limits**: Verified 2MB text input gets truncated to 1MB,
chunk limits enforced
3. **File Size Limits**: Confirmed 200MB files rejected, directory size
limits enforced
4. **Iteration Limits**: Tested 20K item arrays rejected, large JSON
strings rejected
5. **Timeout Protection**: Dangerous regex patterns timeout after 5s
instead of hanging

**Compatibility Tests:**
- Normal functionality preserved for all blocks
- Existing tests pass with new security limits
- Performance impact minimal for typical usage

### Security Impact 🛡️

**Before:** Multiple attack vectors could cause:
- CPU exhaustion (ReDoS attacks)
- Memory exhaustion (amplification attacks)  
- Disk exhaustion (file/log bombs)
- Service unavailability

**After:** All attack vectors mitigated with:
- Input validation and size limits
- Timeout protections
- Resource quotas
- Defense-in-depth approach

All fixes maintain backward compatibility while preventing DoS attacks.

🤖 Generated with [Claude Code](https://claude.ai/code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Adds robust DoS protections across blocks (regex, memory, iteration,
XML/RSS, file I/O) and enables app/Docker log rotation with
comprehensive tests.
> 
> - **Security hardening**:
> - Replace unsafe regex in `backend/blocks/code_extraction_block.py` to
prevent ReDoS; add safer extraction/removal patterns.
> - Constrain LLM summarizer chunking in `backend/blocks/llm.py` (1MB
cap, chunk/overlap validation, chunk count limit).
> - Limit RSS fetching in `backend/blocks/rss.py` (scheme validation,
10MB cap, timeout, bounded read) and return empty on failure.
>   - Impose XML size limit (10MB) in `backend/blocks/xml_parser.py`.
> - Add file upload/download limits in `backend/util/file.py`
(100MB/file, 1GB dir quota) and enforce scanning before write.
> - Enable rotating file logs in `autogpt_libs/logging/config.py` (size
+ backups) and Docker json-file log rotation in
`docker-compose.platform.yml`.
> - **Iteration block**:
> - Add item count/string size limits; fix yielded key for dicts; cap
iterations in `backend/blocks/iteration.py`.
> - **Tests**:
> - New `backend/blocks/test/test_security_fixes.py` covering ReDoS,
timeouts, memory/size and iteration limits, XML/file constraints.
> - **Misc**:
> - Typing fallback for `NotRequired` in `activity_status_generator.py`.
>   - Dependency updates in `backend/poetry.lock`.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
500e1578b1. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Tindle <ntindle@users.noreply.github.com>
Co-authored-by: Zamil Majdy <majdyz@users.noreply.github.com>
Co-authored-by: Reinier van der Leer <Pwuts@users.noreply.github.com>
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2025-10-02 12:55:55 +00:00
..

AutoGPT Platform

Welcome to the AutoGPT Platform - a powerful system for creating and running AI agents to solve business problems. This platform enables you to harness the power of artificial intelligence to automate tasks, analyze data, and generate insights for your organization.

Getting Started

Prerequisites

  • Docker
  • Docker Compose V2 (comes with Docker Desktop, or can be installed separately)

Running the System

To run the AutoGPT Platform, follow these steps:

  1. Clone this repository to your local machine and navigate to the autogpt_platform directory within the repository:

    git clone <https://github.com/Significant-Gravitas/AutoGPT.git | git@github.com:Significant-Gravitas/AutoGPT.git>
    cd AutoGPT/autogpt_platform
    
  2. Run the following command:

    cp .env.default .env
    

    This command will copy the .env.default file to .env. You can modify the .env file to add your own environment variables.

  3. Run the following command:

    docker compose up -d
    

    This command will start all the necessary backend services defined in the docker-compose.yml file in detached mode.

  4. After all the services are in ready state, open your browser and navigate to http://localhost:3000 to access the AutoGPT Platform frontend.

Docker Compose Commands

Here are some useful Docker Compose commands for managing your AutoGPT Platform:

  • docker compose up -d: Start the services in detached mode.
  • docker compose stop: Stop the running services without removing them.
  • docker compose rm: Remove stopped service containers.
  • docker compose build: Build or rebuild services.
  • docker compose down: Stop and remove containers, networks, and volumes.
  • docker compose watch: Watch for changes in your services and automatically update them.

Sample Scenarios

Here are some common scenarios where you might use multiple Docker Compose commands:

  1. Updating and restarting a specific service:

    docker compose build api_srv
    docker compose up -d --no-deps api_srv
    

    This rebuilds the api_srv service and restarts it without affecting other services.

  2. Viewing logs for troubleshooting:

    docker compose logs -f api_srv ws_srv
    

    This shows and follows the logs for both api_srv and ws_srv services.

  3. Scaling a service for increased load:

    docker compose up -d --scale executor=3
    

    This scales the executor service to 3 instances to handle increased load.

  4. Stopping the entire system for maintenance:

    docker compose stop
    docker compose rm -f
    docker compose pull
    docker compose up -d
    

    This stops all services, removes containers, pulls the latest images, and restarts the system.

  5. Developing with live updates:

    docker compose watch
    

    This watches for changes in your code and automatically updates the relevant services.

  6. Checking the status of services:

    docker compose ps
    

    This shows the current status of all services defined in your docker-compose.yml file.

These scenarios demonstrate how to use Docker Compose commands in combination to manage your AutoGPT Platform effectively.

Persisting Data

To persist data for PostgreSQL and Redis, you can modify the docker-compose.yml file to add volumes. Here's how:

  1. Open the docker-compose.yml file in a text editor.

  2. Add volume configurations for PostgreSQL and Redis services:

    services:
      postgres:
        # ... other configurations ...
        volumes:
          - postgres_data:/var/lib/postgresql/data
    
      redis:
        # ... other configurations ...
        volumes:
          - redis_data:/data
    
    volumes:
      postgres_data:
      redis_data:
    
  3. Save the file and run docker compose up -d to apply the changes.

This configuration will create named volumes for PostgreSQL and Redis, ensuring that your data persists across container restarts.

API Client Generation

The platform includes scripts for generating and managing the API client:

  • pnpm fetch:openapi: Fetches the OpenAPI specification from the backend service (requires backend to be running on port 8006)
  • pnpm generate:api-client: Generates the TypeScript API client from the OpenAPI specification using Orval
  • pnpm generate:api: Runs both fetch and generate commands in sequence

Manual API Client Updates

If you need to update the API client after making changes to the backend API:

  1. Ensure the backend services are running:

    docker compose up -d
    
  2. Generate the updated API client:

    pnpm generate:api
    

This will fetch the latest OpenAPI specification and regenerate the TypeScript client code.