Files
AutoGPT/autogpt_platform
Eve 647c8ed8d4 feat(backend/blocks): enhance list concatenation with advanced operations (#12105)
## Summary

Enhances the existing `ConcatenateListsBlock` and adds five new
companion blocks for comprehensive list manipulation, addressing issue
#11139 ("Implement block to concatenate lists").

### Changes

- **Enhanced `ConcatenateListsBlock`** with optional deduplication
(`deduplicate`) and None-value filtering (`remove_none`), plus an output
`length` field
- **New `FlattenListBlock`**: Recursively flattens nested list
structures with configurable `max_depth`
- **New `InterleaveListsBlock`**: Round-robin interleaving of elements
from multiple lists
- **New `ZipListsBlock`**: Zips corresponding elements from multiple
lists with support for padding to longest or truncating to shortest
- **New `ListDifferenceBlock`**: Computes set difference between two
lists (regular or symmetric)
- **New `ListIntersectionBlock`**: Finds common elements between two
lists, preserving order

### Helper Utilities

Extracted reusable helper functions for validation, flattening,
deduplication, interleaving, chunking, and statistics computation to
support the blocks and enable future reuse.

### Test Coverage

Comprehensive test suite with 188 test functions across 29 test classes
covering:
- Built-in block test harness validation for all 6 blocks
- Manual edge-case tests for each block (empty inputs, large lists,
mixed types, nested structures)
- Internal method tests for all block classes
- Unit tests for all helper utility functions

Closes #11139

## Test plan

- [x] All files pass Python syntax validation (`ast.parse`)
- [x] Built-in `test_input`/`test_output` tests defined for all blocks
- [x] Manual tests cover edge cases: empty lists, large lists, mixed
types, nested structures, deduplication, None removal
- [x] Helper function tests validate all utility functions independently
- [x] All block IDs are valid UUID4
- [x] Block categories set to `BlockCategory.BASIC` for consistency with
existing list blocks


<!-- greptile_comment -->

<h2>Greptile Overview</h2>

<details><summary><h3>Greptile Summary</h3></summary>

Enhanced `ConcatenateListsBlock` with deduplication and None-filtering
options, and added five new list manipulation blocks
(`FlattenListBlock`, `InterleaveListsBlock`, `ZipListsBlock`,
`ListDifferenceBlock`, `ListIntersectionBlock`) with comprehensive
helper functions and test coverage.

**Key Changes:**
- Enhanced `ConcatenateListsBlock` with `deduplicate` and `remove_none`
options, plus `length` output field
- Added `FlattenListBlock` for recursively flattening nested lists with
configurable `max_depth`
- Added `InterleaveListsBlock` for round-robin element interleaving
- Added `ZipListsBlock` with support for padding/truncation
- Added `ListDifferenceBlock` and `ListIntersectionBlock` for set
operations
- Extracted 12 reusable helper functions for validation, flattening,
deduplication, etc.
- Comprehensive test suite with 188 test functions covering edge cases

**Minor Issues:**
- Helper function `_deduplicate_list` has redundant logic in the `else`
branch that duplicates the `if` branch
- Three helper functions (`_filter_empty_collections`,
`_compute_list_statistics`, `_chunk_list`) are defined but unused -
consider removing unless planned for future use
- The `_make_hashable` function uses `hash(repr(item))` for unhashable
types, which correctly treats structurally identical dicts/lists as
duplicates
</details>


<details><summary><h3>Confidence Score: 4/5</h3></summary>

- Safe to merge with minor style improvements recommended
- The implementation is well-structured with comprehensive test coverage
(188 tests), proper error handling, and follows existing block patterns.
All blocks use valid UUID4 IDs and correct categories. The helper
functions provide good code reuse. The minor issues are purely stylistic
(redundant code, unused helpers) and don't affect functionality or
safety.
- No files require special attention - both files are well-tested and
follow project conventions
</details>


<details><summary><h3>Sequence Diagram</h3></summary>

```mermaid
sequenceDiagram
    participant User
    participant Block as List Block
    participant Helper as Helper Functions
    participant Output
    
    User->>Block: Input (lists/parameters)
    Block->>Helper: _validate_all_lists()
    Helper-->>Block: validation result
    
    alt validation fails
        Block->>Output: error message
    else validation succeeds
        Block->>Helper: _concatenate_lists_simple() / _flatten_nested_list() / etc.
        Helper-->>Block: processed result
        
        opt deduplicate enabled
            Block->>Helper: _deduplicate_list()
            Helper-->>Block: deduplicated result
        end
        
        opt remove_none enabled
            Block->>Helper: _filter_none_values()
            Helper-->>Block: filtered result
        end
        
        Block->>Output: result + length
    end
    
    Output-->>User: Block outputs
```
</details>


<sub>Last reviewed commit: a6d5445</sub>

<!-- greptile_other_comments_section -->

<sub>(2/5) Greptile learns from your feedback when you react with thumbs
up/down!</sub>

<!-- /greptile_comment -->

---------

Co-authored-by: Otto <otto@agpt.co>
2026-02-16 05:39:53 +00:00
..

AutoGPT Platform

Welcome to the AutoGPT Platform - a powerful system for creating and running AI agents to solve business problems. This platform enables you to harness the power of artificial intelligence to automate tasks, analyze data, and generate insights for your organization.

Getting Started

Prerequisites

  • Docker
  • Docker Compose V2 (comes with Docker Desktop, or can be installed separately)

Running the System

To run the AutoGPT Platform, follow these steps:

  1. Clone this repository to your local machine and navigate to the autogpt_platform directory within the repository:

    git clone <https://github.com/Significant-Gravitas/AutoGPT.git | git@github.com:Significant-Gravitas/AutoGPT.git>
    cd AutoGPT/autogpt_platform
    
  2. Run the following command:

    cp .env.default .env
    

    This command will copy the .env.default file to .env. You can modify the .env file to add your own environment variables.

  3. Run the following command:

    docker compose up -d
    

    This command will start all the necessary backend services defined in the docker-compose.yml file in detached mode.

  4. After all the services are in ready state, open your browser and navigate to http://localhost:3000 to access the AutoGPT Platform frontend.

Running Just Core services

You can now run the following to enable just the core services.

# For help
make help

# Run just Supabase + Redis + RabbitMQ
make start-core

# Stop core services
make stop-core

# View logs from core services 
make logs-core

# Run formatting and linting for backend and frontend
make format

# Run migrations for backend database
make migrate

# Run backend server
make run-backend

# Run frontend development server
make run-frontend

Docker Compose Commands

Here are some useful Docker Compose commands for managing your AutoGPT Platform:

  • docker compose up -d: Start the services in detached mode.
  • docker compose stop: Stop the running services without removing them.
  • docker compose rm: Remove stopped service containers.
  • docker compose build: Build or rebuild services.
  • docker compose down: Stop and remove containers, networks, and volumes.
  • docker compose watch: Watch for changes in your services and automatically update them.

Sample Scenarios

Here are some common scenarios where you might use multiple Docker Compose commands:

  1. Updating and restarting a specific service:

    docker compose build api_srv
    docker compose up -d --no-deps api_srv
    

    This rebuilds the api_srv service and restarts it without affecting other services.

  2. Viewing logs for troubleshooting:

    docker compose logs -f api_srv ws_srv
    

    This shows and follows the logs for both api_srv and ws_srv services.

  3. Scaling a service for increased load:

    docker compose up -d --scale executor=3
    

    This scales the executor service to 3 instances to handle increased load.

  4. Stopping the entire system for maintenance:

    docker compose stop
    docker compose rm -f
    docker compose pull
    docker compose up -d
    

    This stops all services, removes containers, pulls the latest images, and restarts the system.

  5. Developing with live updates:

    docker compose watch
    

    This watches for changes in your code and automatically updates the relevant services.

  6. Checking the status of services:

    docker compose ps
    

    This shows the current status of all services defined in your docker-compose.yml file.

These scenarios demonstrate how to use Docker Compose commands in combination to manage your AutoGPT Platform effectively.

Persisting Data

To persist data for PostgreSQL and Redis, you can modify the docker-compose.yml file to add volumes. Here's how:

  1. Open the docker-compose.yml file in a text editor.

  2. Add volume configurations for PostgreSQL and Redis services:

    services:
      postgres:
        # ... other configurations ...
        volumes:
          - postgres_data:/var/lib/postgresql/data
    
      redis:
        # ... other configurations ...
        volumes:
          - redis_data:/data
    
    volumes:
      postgres_data:
      redis_data:
    
  3. Save the file and run docker compose up -d to apply the changes.

This configuration will create named volumes for PostgreSQL and Redis, ensuring that your data persists across container restarts.

API Client Generation

The platform includes scripts for generating and managing the API client:

  • pnpm fetch:openapi: Fetches the OpenAPI specification from the backend service (requires backend to be running on port 8006)
  • pnpm generate:api-client: Generates the TypeScript API client from the OpenAPI specification using Orval
  • pnpm generate:api: Runs both fetch and generate commands in sequence

Manual API Client Updates

If you need to update the API client after making changes to the backend API:

  1. Ensure the backend services are running:

    docker compose up -d
    
  2. Generate the updated API client:

    pnpm generate:api
    

This will fetch the latest OpenAPI specification and regenerate the TypeScript client code.