fix(backend): Validate 'name' attribute when constructing AgentInput/OutputBlock

Updated block costs in `backend/backend/data/block_cost_config.py`:
  - **AIShortformVideoCreatorBlock**: Updated from 50 credits to 307
  - **AIAdMakerVideoCreatorBlock**: Added cost of 714 credits
  - **AIScreenshotToVideoAdBlock**: Added cost of 612 credits

  ### Checklist 📋

  #### For code changes:
  - [x] I have clearly listed my changes in the PR description
  - [x] I have made a test plan
  - [x] I have tested my changes according to the test plan:
- [x] Verify AIShortformVideoCreatorBlock costs 307 credits when
executed
- [x] Verify AIAdMakerVideoCreatorBlock costs 714 credits when executed
- [x] Verify AIScreenshotToVideoAdBlock costs 612 credits when executed

.github/copilot-instructions.md

@@ -12,6 +12,7 @@ This file provides comprehensive onboarding information for GitHub Copilot codin
 - **Infrastructure** - Docker configurations, CI/CD, and development tools
 
 **Primary Languages & Frameworks:**
+
 - **Backend**: Python 3.10-3.13, FastAPI, Prisma ORM, PostgreSQL, RabbitMQ
 - **Frontend**: TypeScript, Next.js 15, React, Tailwind CSS, Radix UI
 - **Development**: Docker, Poetry, pnpm, Playwright, Storybook
@@ -23,15 +24,17 @@ This file provides comprehensive onboarding information for GitHub Copilot codin
 **Always run these commands in the correct directory and in this order:**
 
 1. **Initial Setup** (required once):
+
    ```bash
    # Clone and enter repository
    git clone <repo> && cd AutoGPT
-   
+
    # Start all services (database, redis, rabbitmq, clamav)
    cd autogpt_platform && docker compose --profile local up deps --build --detach
    ```
 
 2. **Backend Setup** (always run before backend development):
+
    ```bash
    cd autogpt_platform/backend
    poetry install                    # Install dependencies
@@ -48,6 +51,7 @@ This file provides comprehensive onboarding information for GitHub Copilot codin
 ### Runtime Requirements
 
 **Critical:** Always ensure Docker services are running before starting development:
+
 ```bash
 cd autogpt_platform && docker compose --profile local up deps --build --detach
 ```
@@ -58,6 +62,7 @@ cd autogpt_platform && docker compose --profile local up deps --build --detach
 ### Development Commands
 
 **Backend Development:**
+
 ```bash
 cd autogpt_platform/backend
 poetry run serve                     # Start development server (port 8000)
@@ -68,6 +73,7 @@ poetry run lint                      # Lint code (ruff) - run after format
 ```
 
 **Frontend Development:**
+
 ```bash
 cd autogpt_platform/frontend
 pnpm dev                            # Start development server (port 3000) - use for active development
@@ -81,23 +87,27 @@ pnpm storybook                      # Start component development server
 ### Testing Strategy
 
 **Backend Tests:**
+
 - **Block Tests**: `poetry run pytest backend/blocks/test/test_block.py -xvs` (validates all blocks)
 - **Specific Block**: `poetry run pytest 'backend/blocks/test/test_block.py::test_available_blocks[BlockName]' -xvs`
 - **Snapshot Tests**: Use `--snapshot-update` when output changes, always review with `git diff`
 
 **Frontend Tests:**
+
 - **E2E Tests**: Always run `pnpm dev` before `pnpm test` (Playwright requires running instance)
 - **Component Tests**: Use Storybook for isolated component development
 
 ### Critical Validation Steps
 
 **Before committing changes:**
+
 1. Run `poetry run format` (backend) and `pnpm format` (frontend)
 2. Ensure all tests pass in modified areas
 3. Verify Docker services are still running
 4. Check that database migrations apply cleanly
 
 **Common Issues & Workarounds:**
+
 - **Prisma issues**: Run `poetry run prisma generate` after schema changes
 - **Permission errors**: Ensure Docker has proper permissions
 - **Port conflicts**: Check the `docker-compose.yml` file for the current list of exposed ports. You can list all mapped ports with:
@@ -108,6 +118,7 @@ pnpm storybook                      # Start component development server
 ### Core Architecture
 
 **AutoGPT Platform** (`autogpt_platform/`):
+
 - `backend/` - FastAPI server with async support
   - `backend/backend/` - Core API logic
   - `backend/blocks/` - Agent execution blocks
@@ -121,6 +132,7 @@ pnpm storybook                      # Start component development server
 - `docker-compose.yml` - Development stack orchestration
 
 **Key Configuration Files:**
+
 - `pyproject.toml` - Python dependencies and tooling
 - `package.json` - Node.js dependencies and scripts
 - `schema.prisma` - Database schema and migrations
@@ -136,6 +148,7 @@ pnpm storybook                      # Start component development server
 ### Development Workflow
 
 **GitHub Actions**: Multiple CI/CD workflows in `.github/workflows/`
+
 - `platform-backend-ci.yml` - Backend testing and validation
 - `platform-frontend-ci.yml` - Frontend testing and validation
 - `platform-fullstack-ci.yml` - End-to-end integration tests
@@ -146,11 +159,13 @@ pnpm storybook                      # Start component development server
 ### Key Source Files
 
 **Backend Entry Points:**
+
 - `backend/backend/server/server.py` - FastAPI application setup
 - `backend/backend/data/` - Database models and user management
 - `backend/blocks/` - Agent execution blocks and logic
 
 **Frontend Entry Points:**
+
 - `frontend/src/app/layout.tsx` - Root application layout
 - `frontend/src/app/page.tsx` - Home page
 - `frontend/src/lib/supabase/` - Authentication and database client
@@ -160,6 +175,7 @@ pnpm storybook                      # Start component development server
 ### Agent Block System
 
 Agents are built using a visual block-based system where each block performs a single action. Blocks are defined in `backend/blocks/` and must include:
+
 - Block definition with input/output schemas
 - Execution logic with proper error handling
 - Tests validating functionality
@@ -167,6 +183,7 @@ Agents are built using a visual block-based system where each block performs a s
 ### Database & ORM
 
 **Prisma ORM** with PostgreSQL backend including pgvector for embeddings:
+
 - Schema in `schema.prisma`
 - Migrations in `backend/migrations/`
 - Always run `prisma migrate dev` and `prisma generate` after schema changes
@@ -174,13 +191,15 @@ Agents are built using a visual block-based system where each block performs a s
 ## Environment Configuration
 
 ### Configuration Files Priority Order
+
 1. **Backend**: `/backend/.env.default` → `/backend/.env` (user overrides)
-2. **Frontend**: `/frontend/.env.default` → `/frontend/.env` (user overrides)  
+2. **Frontend**: `/frontend/.env.default` → `/frontend/.env` (user overrides)
 3. **Platform**: `/.env.default` (Supabase/shared) → `/.env` (user overrides)
 4. Docker Compose `environment:` sections override file-based config
 5. Shell environment variables have highest precedence
 
 ### Docker Environment Setup
+
 - All services use hardcoded defaults (no `${VARIABLE}` substitutions)
 - The `env_file` directive loads variables INTO containers at runtime
 - Backend/Frontend services use YAML anchors for consistent configuration
@@ -189,6 +208,7 @@ Agents are built using a visual block-based system where each block performs a s
 ## Advanced Development Patterns
 
 ### Adding New Blocks
+
 1. Create file in `/backend/backend/blocks/`
 2. Inherit from `Block` base class with input/output schemas
 3. Implement `run` method with proper error handling
@@ -198,6 +218,7 @@ Agents are built using a visual block-based system where each block performs a s
 7. Consider how inputs/outputs connect with other blocks in graph editor
 
 ### API Development
+
 1. Update routes in `/backend/backend/server/routers/`
 2. Add/update Pydantic models in same directory
 3. Write tests alongside route files
@@ -205,21 +226,76 @@ Agents are built using a visual block-based system where each block performs a s
 5. Run `poetry run test` to verify changes
 
 ### Frontend Development
-1. Components in `/frontend/src/components/`
-2. Use existing UI components from `/frontend/src/components/ui/`
-3. Add Storybook stories for component development
-4. Test user-facing features with Playwright E2E tests
-5. Update protected routes in middleware when needed
+
+**📖 Complete Frontend Guide**: See `autogpt_platform/frontend/CONTRIBUTING.md` and `autogpt_platform/frontend/.cursorrules` for comprehensive patterns and conventions.
+
+**Quick Reference:**
+
+**Component Structure:**
+
+- Separate render logic from data/behavior
+- Structure: `ComponentName/ComponentName.tsx` + `useComponentName.ts` + `helpers.ts`
+- Exception: Small components (3-4 lines of logic) can be inline
+- Render-only components can be direct files without folders
+
+**Data Fetching:**
+
+- Use generated API hooks from `@/app/api/__generated__/endpoints/`
+- Generated via Orval from backend OpenAPI spec
+- Pattern: `use{Method}{Version}{OperationName}`
+- Example: `useGetV2ListLibraryAgents`
+- Regenerate with: `pnpm generate:api`
+- **Never** use deprecated `BackendAPI` or `src/lib/autogpt-server-api/*`
+
+**Code Conventions:**
+
+- Use function declarations for components and handlers (not arrow functions)
+- Only arrow functions for small inline lambdas (map, filter, etc.)
+- Components: `PascalCase`, Hooks: `camelCase` with `use` prefix
+- No barrel files or `index.ts` re-exports
+- Minimal comments (code should be self-documenting)
+
+**Styling:**
+
+- Use Tailwind CSS utilities only
+- Use design system components from `src/components/` (atoms, molecules, organisms)
+- Never use `src/components/__legacy__/*`
+- Only use Phosphor Icons (`@phosphor-icons/react`)
+- Prefer design tokens over hardcoded values
+
+**Error Handling:**
+
+- Render errors: Use `<ErrorCard />` component
+- Mutation errors: Display with toast notifications
+- Manual exceptions: Use `Sentry.captureException()`
+- Global error boundaries already configured
+
+**Testing:**
+
+- Add/update Storybook stories for UI components (`pnpm storybook`)
+- Run Playwright E2E tests with `pnpm test`
+- Verify in Chromatic after PR
+
+**Architecture:**
+
+- Default to client components ("use client")
+- Server components only for SEO or extreme TTFB needs
+- Use React Query for server state (via generated hooks)
+- Co-locate UI state in components/hooks
 
 ### Security Guidelines
+
 **Cache Protection Middleware** (`/backend/backend/server/middleware/security.py`):
+
 - Default: Disables caching for ALL endpoints with `Cache-Control: no-store, no-cache, must-revalidate, private`
 - Uses allow list approach for cacheable paths (static assets, health checks, public pages)
 - Prevents sensitive data caching in browsers/proxies
 - Add new cacheable endpoints to `CACHEABLE_PATHS`
 
 ### CI/CD Alignment
+
 The repository has comprehensive CI workflows that test:
+
 - **Backend**: Python 3.11-3.13, services (Redis/RabbitMQ/ClamAV), Prisma migrations, Poetry lock validation
 - **Frontend**: Node.js 21, pnpm, Playwright with Docker Compose stack, API schema validation
 - **Integration**: Full-stack type checking and E2E testing
@@ -229,6 +305,7 @@ Match these patterns when developing locally - the copilot setup environment mir
 ## Collaboration with Other AI Assistants
 
 This repository is actively developed with assistance from Claude (via CLAUDE.md files). When working on this codebase:
+
 - Check for existing CLAUDE.md files that provide additional context
 - Follow established patterns and conventions already in the codebase
 - Maintain consistency with existing code style and architecture
@@ -237,8 +314,9 @@ This repository is actively developed with assistance from Claude (via CLAUDE.md
 ## Trust These Instructions
 
 These instructions are comprehensive and tested. Only perform additional searches if:
+
 1. Information here is incomplete for your specific task
 2. You encounter errors not covered by the workarounds
 3. You need to understand implementation details not covered above
 
-For detailed platform development patterns, refer to `autogpt_platform/CLAUDE.md` and `AGENTS.md` in the repository root.
+For detailed platform development patterns, refer to `autogpt_platform/CLAUDE.md` and `AGENTS.md` in the repository root.

.github/workflows/claude-ci-failure-auto-fix.yml

@@ -0,0 +1,97 @@
+name: Auto Fix CI Failures
+
+on:
+  workflow_run:
+    workflows: ["CI"]
+    types:
+      - completed
+
+permissions:
+  contents: write
+  pull-requests: write
+  actions: read
+  issues: write
+  id-token: write # Required for OIDC token exchange
+
+jobs:
+  auto-fix:
+    if: |
+      github.event.workflow_run.conclusion == 'failure' &&
+      github.event.workflow_run.pull_requests[0] &&
+      !startsWith(github.event.workflow_run.head_branch, 'claude-auto-fix-ci-')
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.workflow_run.head_branch }}
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Setup git identity
+        run: |
+          git config --global user.email "claude[bot]@users.noreply.github.com"
+          git config --global user.name "claude[bot]"
+
+      - name: Create fix branch
+        id: branch
+        run: |
+          BRANCH_NAME="claude-auto-fix-ci-${{ github.event.workflow_run.head_branch }}-${{ github.run_id }}"
+          git checkout -b "$BRANCH_NAME"
+          echo "branch_name=$BRANCH_NAME" >> $GITHUB_OUTPUT
+
+      - name: Get CI failure details
+        id: failure_details
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const run = await github.rest.actions.getWorkflowRun({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              run_id: ${{ github.event.workflow_run.id }}
+            });
+
+            const jobs = await github.rest.actions.listJobsForWorkflowRun({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              run_id: ${{ github.event.workflow_run.id }}
+            });
+
+            const failedJobs = jobs.data.jobs.filter(job => job.conclusion === 'failure');
+
+            let errorLogs = [];
+            for (const job of failedJobs) {
+              const logs = await github.rest.actions.downloadJobLogsForWorkflowRun({
+                owner: context.repo.owner,
+                repo: context.repo.repo,
+                job_id: job.id
+              });
+              errorLogs.push({
+                jobName: job.name,
+                logs: logs.data
+              });
+            }
+
+            return {
+              runUrl: run.data.html_url,
+              failedJobs: failedJobs.map(j => j.name),
+              errorLogs: errorLogs
+            };
+
+      - name: Fix CI failures with Claude
+        id: claude
+        uses: anthropics/claude-code-action@v1
+        with:
+          prompt: |
+            /fix-ci 
+            Failed CI Run: ${{ fromJSON(steps.failure_details.outputs.result).runUrl }}
+            Failed Jobs: ${{ join(fromJSON(steps.failure_details.outputs.result).failedJobs, ', ') }}
+            PR Number: ${{ github.event.workflow_run.pull_requests[0].number }}
+            Branch Name: ${{ steps.branch.outputs.branch_name }}
+            Base Branch: ${{ github.event.workflow_run.head_branch }}
+            Repository: ${{ github.repository }}
+
+            Error logs:
+            ${{ toJSON(fromJSON(steps.failure_details.outputs.result).errorLogs) }}
+          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
+          claude_args: "--allowedTools 'Edit,MultiEdit,Write,Read,Glob,Grep,LS,Bash(git:*),Bash(bun:*),Bash(npm:*),Bash(npx:*),Bash(gh:*)'"

.github/workflows/claude-dependabot.yml

@@ -0,0 +1,379 @@
+# Claude Dependabot PR Review Workflow
+# 
+# This workflow automatically runs Claude analysis on Dependabot PRs to:
+# - Identify dependency changes and their versions
+# - Look up changelogs for updated packages  
+# - Assess breaking changes and security impacts
+# - Provide actionable recommendations for the development team
+#
+# Triggered on: Dependabot PRs (opened, synchronize)
+# Requirements: ANTHROPIC_API_KEY secret must be configured
+
+name: Claude Dependabot PR Review
+
+on:
+  pull_request:
+    types: [opened, synchronize]
+
+jobs:
+  dependabot-review:
+    # Only run on Dependabot PRs
+    if: github.actor == 'dependabot[bot]'
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+
+    permissions:
+      contents: write
+      pull-requests: read
+      issues: read
+      id-token: write
+      actions: read # Required for CI access
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 1
+
+      # Backend Python/Poetry setup (mirrors platform-backend-ci.yml)
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"  # Use standard version matching CI
+
+      - name: Set up Python dependency cache
+        uses: actions/cache@v4
+        with:
+          path: ~/.cache/pypoetry
+          key: poetry-${{ runner.os }}-${{ hashFiles('autogpt_platform/backend/poetry.lock') }}
+
+      - name: Install Poetry
+        run: |
+          # Extract Poetry version from backend/poetry.lock (matches CI)
+          cd autogpt_platform/backend
+          HEAD_POETRY_VERSION=$(python3 ../../.github/workflows/scripts/get_package_version_from_lockfile.py poetry)
+          echo "Found Poetry version ${HEAD_POETRY_VERSION} in backend/poetry.lock"
+          
+          # Install Poetry
+          curl -sSL https://install.python-poetry.org | POETRY_VERSION=$HEAD_POETRY_VERSION python3 -
+          
+          # Add Poetry to PATH
+          echo "$HOME/.local/bin" >> $GITHUB_PATH
+
+      - name: Check poetry.lock
+        working-directory: autogpt_platform/backend
+        run: |
+          poetry lock
+          if ! git diff --quiet --ignore-matching-lines="^# " poetry.lock; then
+            echo "Warning: poetry.lock not up to date, but continuing for setup"
+            git checkout poetry.lock  # Reset for clean setup
+          fi
+
+      - name: Install Python dependencies
+        working-directory: autogpt_platform/backend
+        run: poetry install
+
+      - name: Generate Prisma Client
+        working-directory: autogpt_platform/backend
+        run: poetry run prisma generate
+
+      # Frontend Node.js/pnpm setup (mirrors platform-frontend-ci.yml)
+      - name: Set up Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: "21"
+
+      - name: Enable corepack
+        run: corepack enable
+
+      - name: Set pnpm store directory
+        run: |
+          pnpm config set store-dir ~/.pnpm-store
+          echo "PNPM_HOME=$HOME/.pnpm-store" >> $GITHUB_ENV
+
+      - name: Cache frontend dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.pnpm-store
+          key: ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml', 'autogpt_platform/frontend/package.json') }}
+          restore-keys: |
+            ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml') }}
+            ${{ runner.os }}-pnpm-
+
+      - name: Install JavaScript dependencies
+        working-directory: autogpt_platform/frontend
+        run: pnpm install --frozen-lockfile
+
+      # Install Playwright browsers for frontend testing
+      # NOTE: Disabled to save ~1 minute of setup time. Re-enable if Copilot needs browser automation (e.g., for MCP)
+      # - name: Install Playwright browsers
+      #   working-directory: autogpt_platform/frontend
+      #   run: pnpm playwright install --with-deps chromium
+
+      # Docker setup for development environment
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Copy default environment files
+        working-directory: autogpt_platform
+        run: |
+          # Copy default environment files for development
+          cp .env.default .env
+          cp backend/.env.default backend/.env
+          cp frontend/.env.default frontend/.env
+
+      # Phase 1: Cache and load Docker images for faster setup
+      - name: Set up Docker image cache
+        id: docker-cache
+        uses: actions/cache@v4
+        with:
+          path: ~/docker-cache
+          # Use a versioned key for cache invalidation when image list changes
+          key: docker-images-v2-${{ runner.os }}-${{ hashFiles('.github/workflows/copilot-setup-steps.yml') }}
+          restore-keys: |
+            docker-images-v2-${{ runner.os }}-
+            docker-images-v1-${{ runner.os }}-
+
+      - name: Load or pull Docker images
+        working-directory: autogpt_platform
+        run: |
+          mkdir -p ~/docker-cache
+          
+          # Define image list for easy maintenance
+          IMAGES=(
+            "redis:latest"
+            "rabbitmq:management"
+            "clamav/clamav-debian:latest"
+            "busybox:latest"
+            "kong:2.8.1"
+            "supabase/gotrue:v2.170.0"
+            "supabase/postgres:15.8.1.049"
+            "supabase/postgres-meta:v0.86.1"
+            "supabase/studio:20250224-d10db0f"
+          )
+          
+          # Check if any cached tar files exist (more reliable than cache-hit)
+          if ls ~/docker-cache/*.tar 1> /dev/null 2>&1; then
+            echo "Docker cache found, loading images in parallel..."
+            for image in "${IMAGES[@]}"; do
+              # Convert image name to filename (replace : and / with -)
+              filename=$(echo "$image" | tr ':/' '--')
+              if [ -f ~/docker-cache/${filename}.tar ]; then
+                echo "Loading $image..."
+                docker load -i ~/docker-cache/${filename}.tar || echo "Warning: Failed to load $image from cache" &
+              fi
+            done
+            wait
+            echo "All cached images loaded"
+          else
+            echo "No Docker cache found, pulling images in parallel..."
+            # Pull all images in parallel
+            for image in "${IMAGES[@]}"; do
+              docker pull "$image" &
+            done
+            wait
+            
+            # Only save cache on main branches (not PRs) to avoid cache pollution
+            if [[ "${{ github.ref }}" == "refs/heads/master" ]] || [[ "${{ github.ref }}" == "refs/heads/dev" ]]; then
+              echo "Saving Docker images to cache in parallel..."
+              for image in "${IMAGES[@]}"; do
+                # Convert image name to filename (replace : and / with -)
+                filename=$(echo "$image" | tr ':/' '--')
+                echo "Saving $image..."
+                docker save -o ~/docker-cache/${filename}.tar "$image" || echo "Warning: Failed to save $image" &
+              done
+              wait
+              echo "Docker image cache saved"
+            else
+              echo "Skipping cache save for PR/feature branch"
+            fi
+          fi
+          
+          echo "Docker images ready for use"
+
+      # Phase 2: Build migrate service with GitHub Actions cache
+      - name: Build migrate Docker image with cache
+        working-directory: autogpt_platform
+        run: |
+          # Build the migrate image with buildx for GHA caching
+          docker buildx build \
+            --cache-from type=gha \
+            --cache-to type=gha,mode=max \
+            --target migrate \
+            --tag autogpt_platform-migrate:latest \
+            --load \
+            -f backend/Dockerfile \
+            ..
+
+      # Start services using pre-built images
+      - name: Start Docker services for development
+        working-directory: autogpt_platform
+        run: |
+          # Start essential services (migrate image already built with correct tag)
+          docker compose --profile local up deps --no-build --detach
+          echo "Waiting for services to be ready..."
+          
+          # Wait for database to be ready
+          echo "Checking database readiness..."
+          timeout 30 sh -c 'until docker compose exec -T db pg_isready -U postgres 2>/dev/null; do 
+            echo "  Waiting for database..."
+            sleep 2
+          done' && echo "✅ Database is ready" || echo "⚠️ Database ready check timeout after 30s, continuing..."
+          
+          # Check migrate service status
+          echo "Checking migration status..."
+          docker compose ps migrate || echo "  Migrate service not visible in ps output"
+          
+          # Wait for migrate service to complete
+          echo "Waiting for migrations to complete..."
+          timeout 30 bash -c '
+            ATTEMPTS=0
+            while [ $ATTEMPTS -lt 15 ]; do
+              ATTEMPTS=$((ATTEMPTS + 1))
+              
+              # Check using docker directly (more reliable than docker compose ps)
+              CONTAINER_STATUS=$(docker ps -a --filter "label=com.docker.compose.service=migrate" --format "{{.Status}}" | head -1)
+              
+              if [ -z "$CONTAINER_STATUS" ]; then
+                echo "  Attempt $ATTEMPTS: Migrate container not found yet..."
+              elif echo "$CONTAINER_STATUS" | grep -q "Exited (0)"; then
+                echo "✅ Migrations completed successfully"
+                docker compose logs migrate --tail=5 2>/dev/null || true
+                exit 0
+              elif echo "$CONTAINER_STATUS" | grep -q "Exited ([1-9]"; then
+                EXIT_CODE=$(echo "$CONTAINER_STATUS" | grep -oE "Exited \([0-9]+\)" | grep -oE "[0-9]+")
+                echo "❌ Migrations failed with exit code: $EXIT_CODE"
+                echo "Migration logs:"
+                docker compose logs migrate --tail=20 2>/dev/null || true
+                exit 1
+              elif echo "$CONTAINER_STATUS" | grep -q "Up"; then
+                echo "  Attempt $ATTEMPTS: Migrate container is running... ($CONTAINER_STATUS)"
+              else
+                echo "  Attempt $ATTEMPTS: Migrate container status: $CONTAINER_STATUS"
+              fi
+              
+              sleep 2
+            done
+            
+            echo "⚠️ Timeout: Could not determine migration status after 30 seconds"
+            echo "Final container check:"
+            docker ps -a --filter "label=com.docker.compose.service=migrate" || true
+            echo "Migration logs (if available):"
+            docker compose logs migrate --tail=10 2>/dev/null || echo "  No logs available"
+          ' || echo "⚠️ Migration check completed with warnings, continuing..."
+          
+          # Brief wait for other services to stabilize
+          echo "Waiting 5 seconds for other services to stabilize..."
+          sleep 5
+
+      # Verify installations and provide environment info
+      - name: Verify setup and show environment info
+        run: |
+          echo "=== Python Setup ==="
+          python --version
+          poetry --version
+          
+          echo "=== Node.js Setup ==="
+          node --version
+          pnpm --version
+          
+          echo "=== Additional Tools ==="
+          docker --version
+          docker compose version
+          gh --version || true
+          
+          echo "=== Services Status ==="
+          cd autogpt_platform
+          docker compose ps || true
+          
+          echo "=== Backend Dependencies ==="
+          cd backend
+          poetry show | head -10 || true
+          
+          echo "=== Frontend Dependencies ==="
+          cd ../frontend
+          pnpm list --depth=0 | head -10 || true
+          
+          echo "=== Environment Files ==="
+          ls -la ../.env* || true
+          ls -la .env* || true
+          ls -la ../backend/.env* || true
+          
+          echo "✅ AutoGPT Platform development environment setup complete!"
+          echo "🚀 Ready for development with Docker services running"
+          echo "📝 Backend server: poetry run serve (port 8000)"
+          echo "🌐 Frontend server: pnpm dev (port 3000)"
+
+
+      - name: Run Claude Dependabot Analysis
+        id: claude_review
+        uses: anthropics/claude-code-action@v1
+        with:
+          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
+          claude_args: |
+            --allowedTools "Bash(npm:*),Bash(pnpm:*),Bash(poetry:*),Bash(git:*),Edit,Replace,NotebookEditCell,mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*), Bash(gh pr diff:*), Bash(gh pr view:*)"
+          prompt: |
+            You are Claude, an AI assistant specialized in reviewing Dependabot dependency update PRs. 
+            
+            Your primary tasks are:
+            1. **Analyze the dependency changes** in this Dependabot PR
+            2. **Look up changelogs** for all updated dependencies to understand what changed
+            3. **Identify breaking changes** and assess potential impact on the AutoGPT codebase
+            4. **Provide actionable recommendations** for the development team
+            
+            ## Analysis Process:
+            
+            1. **Identify Changed Dependencies**: 
+               - Use git diff to see what dependencies were updated
+               - Parse package.json, poetry.lock, requirements files, etc.
+               - List all package versions: old → new
+            
+            2. **Changelog Research**:
+               - For each updated dependency, look up its changelog/release notes
+               - Use WebFetch to access GitHub releases, NPM package pages, PyPI project pages. The pr should also have some details
+               - Focus on versions between the old and new versions
+               - Identify: breaking changes, deprecations, security fixes, new features
+            
+            3. **Breaking Change Assessment**:
+               - Categorize changes: BREAKING, MAJOR, MINOR, PATCH, SECURITY
+               - Assess impact on AutoGPT's usage patterns
+               - Check if AutoGPT uses affected APIs/features
+               - Look for migration guides or upgrade instructions
+            
+            4. **Codebase Impact Analysis**:
+               - Search the AutoGPT codebase for usage of changed APIs
+               - Identify files that might be affected by breaking changes
+               - Check test files for deprecated usage patterns
+               - Look for configuration changes needed
+            
+            ## Output Format:
+            
+            Provide a comprehensive review comment with:
+            
+            ### 🔍 Dependency Analysis Summary
+            - List of updated packages with version changes
+            - Overall risk assessment (LOW/MEDIUM/HIGH)
+            
+            ### 📋 Detailed Changelog Review
+            For each updated dependency:
+            - **Package**: name (old_version → new_version)
+            - **Changes**: Summary of key changes
+            - **Breaking Changes**: List any breaking changes
+            - **Security Fixes**: Note security improvements
+            - **Migration Notes**: Any upgrade steps needed
+            
+            ### ⚠️ Impact Assessment
+            - **Breaking Changes Found**: Yes/No with details
+            - **Affected Files**: List AutoGPT files that may need updates
+            - **Test Impact**: Any tests that may need updating
+            - **Configuration Changes**: Required config updates
+            
+            ### 🛠️ Recommendations
+            - **Action Required**: What the team should do
+            - **Testing Focus**: Areas to test thoroughly
+            - **Follow-up Tasks**: Any additional work needed
+            - **Merge Recommendation**: APPROVE/REVIEW_NEEDED/HOLD
+            
+            ### 📚 Useful Links
+            - Links to relevant changelogs, migration guides, documentation
+            
+            Be thorough but concise. Focus on actionable insights that help the development team make informed decisions about the dependency updates.

.github/workflows/claude.yml

@@ -30,18 +30,296 @@ jobs:
         github.event.issue.author_association == 'COLLABORATOR'
       )
     runs-on: ubuntu-latest
+    timeout-minutes: 45
+
     permissions:
-      contents: read
+      contents: write
       pull-requests: read
       issues: read
       id-token: write
+      actions: read # Required for CI access
     steps:
-      - name: Checkout repository
+      - name: Checkout code
         uses: actions/checkout@v4
         with:
           fetch-depth: 1
+
+      # Backend Python/Poetry setup (mirrors platform-backend-ci.yml)
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"  # Use standard version matching CI
+
+      - name: Set up Python dependency cache
+        uses: actions/cache@v4
+        with:
+          path: ~/.cache/pypoetry
+          key: poetry-${{ runner.os }}-${{ hashFiles('autogpt_platform/backend/poetry.lock') }}
+
+      - name: Install Poetry
+        run: |
+          # Extract Poetry version from backend/poetry.lock (matches CI)
+          cd autogpt_platform/backend
+          HEAD_POETRY_VERSION=$(python3 ../../.github/workflows/scripts/get_package_version_from_lockfile.py poetry)
+          echo "Found Poetry version ${HEAD_POETRY_VERSION} in backend/poetry.lock"
+          
+          # Install Poetry
+          curl -sSL https://install.python-poetry.org | POETRY_VERSION=$HEAD_POETRY_VERSION python3 -
+          
+          # Add Poetry to PATH
+          echo "$HOME/.local/bin" >> $GITHUB_PATH
+
+      - name: Check poetry.lock
+        working-directory: autogpt_platform/backend
+        run: |
+          poetry lock
+          if ! git diff --quiet --ignore-matching-lines="^# " poetry.lock; then
+            echo "Warning: poetry.lock not up to date, but continuing for setup"
+            git checkout poetry.lock  # Reset for clean setup
+          fi
+
+      - name: Install Python dependencies
+        working-directory: autogpt_platform/backend
+        run: poetry install
+
+      - name: Generate Prisma Client
+        working-directory: autogpt_platform/backend
+        run: poetry run prisma generate
+
+      # Frontend Node.js/pnpm setup (mirrors platform-frontend-ci.yml)
+      - name: Set up Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: "21"
+
+      - name: Enable corepack
+        run: corepack enable
+
+      - name: Set pnpm store directory
+        run: |
+          pnpm config set store-dir ~/.pnpm-store
+          echo "PNPM_HOME=$HOME/.pnpm-store" >> $GITHUB_ENV
+
+      - name: Cache frontend dependencies
+        uses: actions/cache@v4
+        with:
+          path: ~/.pnpm-store
+          key: ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml', 'autogpt_platform/frontend/package.json') }}
+          restore-keys: |
+            ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml') }}
+            ${{ runner.os }}-pnpm-
+
+      - name: Install JavaScript dependencies
+        working-directory: autogpt_platform/frontend
+        run: pnpm install --frozen-lockfile
+
+      # Install Playwright browsers for frontend testing
+      # NOTE: Disabled to save ~1 minute of setup time. Re-enable if Copilot needs browser automation (e.g., for MCP)
+      # - name: Install Playwright browsers
+      #   working-directory: autogpt_platform/frontend
+      #   run: pnpm playwright install --with-deps chromium
+
+      # Docker setup for development environment
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Copy default environment files
+        working-directory: autogpt_platform
+        run: |
+          # Copy default environment files for development
+          cp .env.default .env
+          cp backend/.env.default backend/.env
+          cp frontend/.env.default frontend/.env
+
+      # Phase 1: Cache and load Docker images for faster setup
+      - name: Set up Docker image cache
+        id: docker-cache
+        uses: actions/cache@v4
+        with:
+          path: ~/docker-cache
+          # Use a versioned key for cache invalidation when image list changes
+          key: docker-images-v2-${{ runner.os }}-${{ hashFiles('.github/workflows/copilot-setup-steps.yml') }}
+          restore-keys: |
+            docker-images-v2-${{ runner.os }}-
+            docker-images-v1-${{ runner.os }}-
+
+      - name: Load or pull Docker images
+        working-directory: autogpt_platform
+        run: |
+          mkdir -p ~/docker-cache
+          
+          # Define image list for easy maintenance
+          IMAGES=(
+            "redis:latest"
+            "rabbitmq:management"
+            "clamav/clamav-debian:latest"
+            "busybox:latest"
+            "kong:2.8.1"
+            "supabase/gotrue:v2.170.0"
+            "supabase/postgres:15.8.1.049"
+            "supabase/postgres-meta:v0.86.1"
+            "supabase/studio:20250224-d10db0f"
+          )
+          
+          # Check if any cached tar files exist (more reliable than cache-hit)
+          if ls ~/docker-cache/*.tar 1> /dev/null 2>&1; then
+            echo "Docker cache found, loading images in parallel..."
+            for image in "${IMAGES[@]}"; do
+              # Convert image name to filename (replace : and / with -)
+              filename=$(echo "$image" | tr ':/' '--')
+              if [ -f ~/docker-cache/${filename}.tar ]; then
+                echo "Loading $image..."
+                docker load -i ~/docker-cache/${filename}.tar || echo "Warning: Failed to load $image from cache" &
+              fi
+            done
+            wait
+            echo "All cached images loaded"
+          else
+            echo "No Docker cache found, pulling images in parallel..."
+            # Pull all images in parallel
+            for image in "${IMAGES[@]}"; do
+              docker pull "$image" &
+            done
+            wait
+            
+            # Only save cache on main branches (not PRs) to avoid cache pollution
+            if [[ "${{ github.ref }}" == "refs/heads/master" ]] || [[ "${{ github.ref }}" == "refs/heads/dev" ]]; then
+              echo "Saving Docker images to cache in parallel..."
+              for image in "${IMAGES[@]}"; do
+                # Convert image name to filename (replace : and / with -)
+                filename=$(echo "$image" | tr ':/' '--')
+                echo "Saving $image..."
+                docker save -o ~/docker-cache/${filename}.tar "$image" || echo "Warning: Failed to save $image" &
+              done
+              wait
+              echo "Docker image cache saved"
+            else
+              echo "Skipping cache save for PR/feature branch"
+            fi
+          fi
+          
+          echo "Docker images ready for use"
+
+      # Phase 2: Build migrate service with GitHub Actions cache
+      - name: Build migrate Docker image with cache
+        working-directory: autogpt_platform
+        run: |
+          # Build the migrate image with buildx for GHA caching
+          docker buildx build \
+            --cache-from type=gha \
+            --cache-to type=gha,mode=max \
+            --target migrate \
+            --tag autogpt_platform-migrate:latest \
+            --load \
+            -f backend/Dockerfile \
+            ..
+
+      # Start services using pre-built images
+      - name: Start Docker services for development
+        working-directory: autogpt_platform
+        run: |
+          # Start essential services (migrate image already built with correct tag)
+          docker compose --profile local up deps --no-build --detach
+          echo "Waiting for services to be ready..."
+          
+          # Wait for database to be ready
+          echo "Checking database readiness..."
+          timeout 30 sh -c 'until docker compose exec -T db pg_isready -U postgres 2>/dev/null; do 
+            echo "  Waiting for database..."
+            sleep 2
+          done' && echo "✅ Database is ready" || echo "⚠️ Database ready check timeout after 30s, continuing..."
+          
+          # Check migrate service status
+          echo "Checking migration status..."
+          docker compose ps migrate || echo "  Migrate service not visible in ps output"
+          
+          # Wait for migrate service to complete
+          echo "Waiting for migrations to complete..."
+          timeout 30 bash -c '
+            ATTEMPTS=0
+            while [ $ATTEMPTS -lt 15 ]; do
+              ATTEMPTS=$((ATTEMPTS + 1))
+              
+              # Check using docker directly (more reliable than docker compose ps)
+              CONTAINER_STATUS=$(docker ps -a --filter "label=com.docker.compose.service=migrate" --format "{{.Status}}" | head -1)
+              
+              if [ -z "$CONTAINER_STATUS" ]; then
+                echo "  Attempt $ATTEMPTS: Migrate container not found yet..."
+              elif echo "$CONTAINER_STATUS" | grep -q "Exited (0)"; then
+                echo "✅ Migrations completed successfully"
+                docker compose logs migrate --tail=5 2>/dev/null || true
+                exit 0
+              elif echo "$CONTAINER_STATUS" | grep -q "Exited ([1-9]"; then
+                EXIT_CODE=$(echo "$CONTAINER_STATUS" | grep -oE "Exited \([0-9]+\)" | grep -oE "[0-9]+")
+                echo "❌ Migrations failed with exit code: $EXIT_CODE"
+                echo "Migration logs:"
+                docker compose logs migrate --tail=20 2>/dev/null || true
+                exit 1
+              elif echo "$CONTAINER_STATUS" | grep -q "Up"; then
+                echo "  Attempt $ATTEMPTS: Migrate container is running... ($CONTAINER_STATUS)"
+              else
+                echo "  Attempt $ATTEMPTS: Migrate container status: $CONTAINER_STATUS"
+              fi
+              
+              sleep 2
+            done
+            
+            echo "⚠️ Timeout: Could not determine migration status after 30 seconds"
+            echo "Final container check:"
+            docker ps -a --filter "label=com.docker.compose.service=migrate" || true
+            echo "Migration logs (if available):"
+            docker compose logs migrate --tail=10 2>/dev/null || echo "  No logs available"
+          ' || echo "⚠️ Migration check completed with warnings, continuing..."
+          
+          # Brief wait for other services to stabilize
+          echo "Waiting 5 seconds for other services to stabilize..."
+          sleep 5
+
+      # Verify installations and provide environment info
+      - name: Verify setup and show environment info
+        run: |
+          echo "=== Python Setup ==="
+          python --version
+          poetry --version
+          
+          echo "=== Node.js Setup ==="
+          node --version
+          pnpm --version
+          
+          echo "=== Additional Tools ==="
+          docker --version
+          docker compose version
+          gh --version || true
+          
+          echo "=== Services Status ==="
+          cd autogpt_platform
+          docker compose ps || true
+          
+          echo "=== Backend Dependencies ==="
+          cd backend
+          poetry show | head -10 || true
+          
+          echo "=== Frontend Dependencies ==="
+          cd ../frontend
+          pnpm list --depth=0 | head -10 || true
+          
+          echo "=== Environment Files ==="
+          ls -la ../.env* || true
+          ls -la .env* || true
+          ls -la ../backend/.env* || true
+          
+          echo "✅ AutoGPT Platform development environment setup complete!"
+          echo "🚀 Ready for development with Docker services running"
+          echo "📝 Backend server: poetry run serve (port 8000)"
+          echo "🌐 Frontend server: pnpm dev (port 3000)"
+
       - name: Run Claude Code
         id: claude
-        uses: anthropics/claude-code-action@beta
+        uses: anthropics/claude-code-action@v1
         with:
           anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
+          claude_args: |
+            --allowedTools "Bash(npm:*),Bash(pnpm:*),Bash(poetry:*),Bash(git:*),Edit,Replace,NotebookEditCell,mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*), Bash(gh pr diff:*), Bash(gh pr view:*), Bash(gh pr edit:*)"
+            --model opus
+          additional_permissions: |
+            actions: read

.github/workflows/platform-autogpt-deploy-dev.yaml

@@ -5,6 +5,13 @@ on:
     branches: [ dev ]
     paths:
       - 'autogpt_platform/**'
+  workflow_dispatch:
+    inputs:
+      git_ref:
+        description: 'Git ref (branch/tag) of AutoGPT to deploy'
+        required: true
+        default: 'master'
+        type: string
 
 permissions:
   contents: 'read'
@@ -19,6 +26,8 @@ jobs:
     steps:
       - name: Checkout code
         uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.inputs.git_ref || github.ref_name }}
 
       - name: Set up Python
         uses: actions/setup-python@v5
@@ -48,4 +57,4 @@ jobs:
           token: ${{ secrets.DEPLOY_TOKEN }}
           repository: Significant-Gravitas/AutoGPT_cloud_infrastructure
           event-type: build_deploy_dev
-          client-payload: '{"ref": "${{ github.ref }}", "sha": "${{ github.sha }}", "repository": "${{ github.repository }}"}'
+          client-payload: '{"ref": "${{ github.event.inputs.git_ref || github.ref }}", "repository": "${{ github.repository }}"}'

.github/workflows/platform-autogpt-deploy-prod.yml

@@ -3,6 +3,7 @@ name: AutoGPT Platform - Deploy Prod Environment
 on:
   release:
     types: [published]
+  workflow_dispatch:
 
 permissions:
   contents: 'read'
@@ -17,6 +18,8 @@ jobs:
     steps:
       - name: Checkout code
         uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref_name || 'master' }}
 
       - name: Set up Python
         uses: actions/setup-python@v5
@@ -36,7 +39,7 @@ jobs:
           DATABASE_URL: ${{ secrets.BACKEND_DATABASE_URL }}
           DIRECT_URL: ${{ secrets.BACKEND_DATABASE_URL }}
 
-          
+
   trigger:
     needs: migrate
     runs-on: ubuntu-latest
@@ -47,4 +50,5 @@ jobs:
           token: ${{ secrets.DEPLOY_TOKEN }}
           repository: Significant-Gravitas/AutoGPT_cloud_infrastructure
           event-type: build_deploy_prod
-          client-payload: '{"ref": "${{ github.ref }}", "sha": "${{ github.sha }}", "repository": "${{ github.repository }}"}'
+          client-payload: |
+            {"ref": "${{ github.ref_name || 'master' }}", "repository": "${{ github.repository }}"}

.github/workflows/platform-backend-ci.yml

@@ -37,9 +37,7 @@ jobs:
 
     services:
       redis:
-        image: bitnami/redis:6.2
-        env:
-          REDIS_PASSWORD: testpassword
+        image: redis:latest
         ports:
           - 6379:6379
       rabbitmq:
@@ -201,10 +199,9 @@ jobs:
           DIRECT_URL: ${{ steps.supabase.outputs.DB_URL }}
           SUPABASE_URL: ${{ steps.supabase.outputs.API_URL }}
           SUPABASE_SERVICE_ROLE_KEY: ${{ steps.supabase.outputs.SERVICE_ROLE_KEY }}
-          SUPABASE_JWT_SECRET: ${{ steps.supabase.outputs.JWT_SECRET }}
+          JWT_VERIFY_KEY: ${{ steps.supabase.outputs.JWT_SECRET }}
           REDIS_HOST: "localhost"
           REDIS_PORT: "6379"
-          REDIS_PASSWORD: "testpassword"
           ENCRYPTION_KEY: "dvziYgz0KSK8FENhju0ZYi8-fRTfAdlz6YLhdB_jhNw=" # DO NOT USE IN PRODUCTION!!
 
     env:

.github/workflows/platform-frontend-ci.yml

@@ -160,7 +160,7 @@ jobs:
 
       - name: Run docker compose
         run: |
-          docker compose -f ../docker-compose.yml up -d
+          NEXT_PUBLIC_PW_TEST=true docker compose -f ../docker-compose.yml up -d
         env:
           DOCKER_BUILDKIT: 1
           BUILDX_CACHE_FROM: type=local,src=/tmp/.buildx-cache

.pr_agent.toml

@@ -1,6 +1,3 @@
-[pr_reviewer]
-num_code_suggestions=0
-
 [pr_code_suggestions]
 commitable_code_suggestions=false
 num_code_suggestions=0

autogpt_platform/CLAUDE.md

@@ -61,24 +61,41 @@ poetry run pytest path/to/test.py --snapshot-update
 
 ```bash
 # Install dependencies
-cd frontend && npm install
+cd frontend && pnpm i
+
+# Generate API client from OpenAPI spec
+pnpm generate:api
 
 # Start development server
-npm run dev
+pnpm dev
 
 # Run E2E tests
-npm run test
+pnpm test
 
 # Run Storybook for component development
-npm run storybook
+pnpm storybook
 
 # Build production
-npm run build
+pnpm build
+
+# Format and lint
+pnpm format
 
 # Type checking
-npm run types
+pnpm types
 ```
 
+**📖 Complete Guide**: See `/frontend/CONTRIBUTING.md` and `/frontend/.cursorrules` for comprehensive frontend patterns.
+
+**Key Frontend Conventions:**
+
+- Separate render logic from data/behavior in components
+- Use generated API hooks from `@/app/api/__generated__/endpoints/`
+- Use function declarations (not arrow functions) for components/handlers
+- Use design system components from `src/components/` (atoms, molecules, organisms)
+- Only use Phosphor Icons
+- Never use `src/components/__legacy__/*` or deprecated `BackendAPI`
+
 ## Architecture Overview
 
 ### Backend Architecture
@@ -92,11 +109,16 @@ npm run types
 
 ### Frontend Architecture
 
-- **Framework**: Next.js App Router with React Server Components
-- **State Management**: React hooks + Supabase client for real-time updates
+- **Framework**: Next.js 15 App Router (client-first approach)
+- **Data Fetching**: Type-safe generated API hooks via Orval + React Query
+- **State Management**: React Query for server state, co-located UI state in components/hooks
+- **Component Structure**: Separate render logic (`.tsx`) from business logic (`use*.ts` hooks)
 - **Workflow Builder**: Visual graph editor using @xyflow/react
-- **UI Components**: Radix UI primitives with Tailwind CSS styling
+- **UI Components**: shadcn/ui (Radix UI primitives) with Tailwind CSS styling
+- **Icons**: Phosphor Icons only
 - **Feature Flags**: LaunchDarkly integration
+- **Error Handling**: ErrorCard for render errors, toast for mutations, Sentry for exceptions
+- **Testing**: Playwright for E2E, Storybook for component development
 
 ### Key Concepts
 
@@ -149,12 +171,23 @@ Key models (defined in `/backend/schema.prisma`):
 
 **Adding a new block:**
 
+Follow the comprehensive [Block SDK Guide](../../../docs/content/platform/block-sdk-guide.md) which covers:
+
+- Provider configuration with `ProviderBuilder`
+- Block schema definition
+- Authentication (API keys, OAuth, webhooks)
+- Testing and validation
+- File organization
+
+Quick steps:
+
 1. Create new file in `/backend/backend/blocks/`
-2. Inherit from `Block` base class
-3. Define input/output schemas
-4. Implement `run` method
-5. Register in block registry
-6. Generate the block uuid using `uuid.uuid4()`
+2. Configure provider using `ProviderBuilder` in `_config.py`
+3. Inherit from `Block` base class
+4. Define input/output schemas using `BlockSchema`
+5. Implement async `run` method
+6. Generate unique block ID using `uuid.uuid4()`
+7. Test with `poetry run pytest backend/blocks/test/test_block.py`
 
 Note: when making many new blocks analyze the interfaces for each of these blocks and picture if they would go well together in a graph based editor or would they struggle to connect productively?
 ex: do the inputs and outputs tie well together?
@@ -168,10 +201,20 @@ ex: do the inputs and outputs tie well together?
 
 **Frontend feature development:**
 
-1. Components go in `/frontend/src/components/`
-2. Use existing UI components from `/frontend/src/components/ui/`
-3. Add Storybook stories for new components
-4. Test with Playwright if user-facing
+See `/frontend/CONTRIBUTING.md` for complete patterns. Quick reference:
+
+1. **Pages**: Create in `src/app/(platform)/feature-name/page.tsx`
+   - Add `usePageName.ts` hook for logic
+   - Put sub-components in local `components/` folder
+2. **Components**: Structure as `ComponentName/ComponentName.tsx` + `useComponentName.ts` + `helpers.ts`
+   - Use design system components from `src/components/` (atoms, molecules, organisms)
+   - Never use `src/components/__legacy__/*`
+3. **Data fetching**: Use generated API hooks from `@/app/api/__generated__/endpoints/`
+   - Regenerate with `pnpm generate:api`
+   - Pattern: `use{Method}{Version}{OperationName}`
+4. **Styling**: Tailwind CSS only, use design tokens, Phosphor Icons only
+5. **Testing**: Add Storybook stories for new components, Playwright for E2E
+6. **Code conventions**: Function declarations (not arrow functions) for components/handlers
 
 ### Security Implementation
 

autogpt_platform/Makefile

@@ -0,0 +1,57 @@
+.PHONY: start-core stop-core logs-core format lint migrate run-backend run-frontend
+
+# Run just Supabase + Redis + RabbitMQ
+start-core:
+	docker compose up -d deps
+
+# Stop core services
+stop-core:
+	docker compose stop deps
+
+reset-db:
+	rm -rf db/docker/volumes/db/data
+	cd backend && poetry run prisma migrate deploy
+	cd backend && poetry run prisma generate
+	
+# View logs for core services
+logs-core:
+	docker compose logs -f deps
+
+# Run formatting and linting for backend and frontend
+format:
+	cd backend && poetry run format
+	cd frontend && pnpm format
+	cd frontend && pnpm lint
+
+init-env:
+	cp -n .env.default .env || true
+	cd backend && cp -n .env.default .env || true
+	cd frontend && cp -n .env.default .env || true
+
+
+# Run migrations for backend
+migrate:
+	cd backend && poetry run prisma migrate deploy
+	cd backend && poetry run prisma generate
+
+run-backend:
+	cd backend && poetry run app
+
+run-frontend:
+	cd frontend && pnpm dev
+
+test-data:
+	cd backend && poetry run python test/test_data_creator.py
+	
+help:
+	@echo "Usage: make <target>"
+	@echo "Targets:"
+	@echo "  start-core - Start just the core services (Supabase, Redis, RabbitMQ) in background"
+	@echo "  stop-core - Stop the core services"
+	@echo "  reset-db - Reset the database by deleting the volume"
+	@echo "  logs-core - Tail the logs for core services"
+	@echo "  format - Format & lint backend (Python) and frontend (TypeScript) code"
+	@echo "  migrate - Run backend database migrations"
+	@echo "  run-backend - Run the backend FastAPI server"
+	@echo "  run-frontend - Run the frontend Next.js development server"
+	@echo "  test-data - Run the test data creator"

autogpt_platform/README.md

@@ -38,6 +38,37 @@ To run the AutoGPT Platform, follow these steps:
 
 4. After all the services are in ready state, open your browser and navigate to `http://localhost:3000` to access the AutoGPT Platform frontend.
 
+### Running Just Core services
+
+You can now run the following to enable just the core services.
+
+```
+# For help
+make help
+
+# Run just Supabase + Redis + RabbitMQ
+make start-core
+
+# Stop core services
+make stop-core
+
+# View logs from core services 
+make logs-core
+
+# Run formatting and linting for backend and frontend
+make format
+
+# Run migrations for backend database
+make migrate
+
+# Run backend server
+make run-backend
+
+# Run frontend development server
+make run-frontend
+
+```
+
 ### Docker Compose Commands
 
 Here are some useful Docker Compose commands for managing your AutoGPT Platform:

autogpt_platform/autogpt_libs/autogpt_libs/api_key/key_manager.py

@@ -1,35 +0,0 @@
-import hashlib
-import secrets
-from typing import NamedTuple
-
-
-class APIKeyContainer(NamedTuple):
-    """Container for API key parts."""
-
-    raw: str
-    prefix: str
-    postfix: str
-    hash: str
-
-
-class APIKeyManager:
-    PREFIX: str = "agpt_"
-    PREFIX_LENGTH: int = 8
-    POSTFIX_LENGTH: int = 8
-
-    def generate_api_key(self) -> APIKeyContainer:
-        """Generate a new API key with all its parts."""
-        raw_key = f"{self.PREFIX}{secrets.token_urlsafe(32)}"
-        return APIKeyContainer(
-            raw=raw_key,
-            prefix=raw_key[: self.PREFIX_LENGTH],
-            postfix=raw_key[-self.POSTFIX_LENGTH :],
-            hash=hashlib.sha256(raw_key.encode()).hexdigest(),
-        )
-
-    def verify_api_key(self, provided_key: str, stored_hash: str) -> bool:
-        """Verify if a provided API key matches the stored hash."""
-        if not provided_key.startswith(self.PREFIX):
-            return False
-        provided_hash = hashlib.sha256(provided_key.encode()).hexdigest()
-        return secrets.compare_digest(provided_hash, stored_hash)

autogpt_platform/autogpt_libs/autogpt_libs/api_key/keysmith.py

@@ -0,0 +1,78 @@
+import hashlib
+import secrets
+from typing import NamedTuple
+
+from cryptography.hazmat.primitives.kdf.scrypt import Scrypt
+
+
+class APIKeyContainer(NamedTuple):
+    """Container for API key parts."""
+
+    key: str
+    head: str
+    tail: str
+    hash: str
+    salt: str
+
+
+class APIKeySmith:
+    PREFIX: str = "agpt_"
+    HEAD_LENGTH: int = 8
+    TAIL_LENGTH: int = 8
+
+    def generate_key(self) -> APIKeyContainer:
+        """Generate a new API key with secure hashing."""
+        raw_key = f"{self.PREFIX}{secrets.token_urlsafe(32)}"
+        hash, salt = self.hash_key(raw_key)
+
+        return APIKeyContainer(
+            key=raw_key,
+            head=raw_key[: self.HEAD_LENGTH],
+            tail=raw_key[-self.TAIL_LENGTH :],
+            hash=hash,
+            salt=salt,
+        )
+
+    def verify_key(
+        self, provided_key: str, known_hash: str, known_salt: str | None = None
+    ) -> bool:
+        """
+        Verify an API key against a known hash (+ salt).
+        Supports verifying both legacy SHA256 and secure Scrypt hashes.
+        """
+        if not provided_key.startswith(self.PREFIX):
+            return False
+
+        # Handle legacy SHA256 hashes (migration support)
+        if known_salt is None:
+            legacy_hash = hashlib.sha256(provided_key.encode()).hexdigest()
+            return secrets.compare_digest(legacy_hash, known_hash)
+
+        try:
+            salt_bytes = bytes.fromhex(known_salt)
+            provided_hash = self._hash_key_with_salt(provided_key, salt_bytes)
+            return secrets.compare_digest(provided_hash, known_hash)
+        except (ValueError, TypeError):
+            return False
+
+    def hash_key(self, raw_key: str) -> tuple[str, str]:
+        """Migrate a legacy hash to secure hash format."""
+        salt = self._generate_salt()
+        hash = self._hash_key_with_salt(raw_key, salt)
+        return hash, salt.hex()
+
+    def _generate_salt(self) -> bytes:
+        """Generate a random salt for hashing."""
+        return secrets.token_bytes(32)
+
+    def _hash_key_with_salt(self, raw_key: str, salt: bytes) -> str:
+        """Hash API key using Scrypt with salt."""
+        kdf = Scrypt(
+            length=32,
+            salt=salt,
+            n=2**14,  # CPU/memory cost parameter
+            r=8,  # Block size parameter
+            p=1,  # Parallelization parameter
+        )
+        key_hash = kdf.derive(raw_key.encode())
+        return key_hash.hex()

autogpt_platform/autogpt_libs/autogpt_libs/api_key/test_keysmith.py

@@ -0,0 +1,79 @@
+import hashlib
+
+from autogpt_libs.api_key.keysmith import APIKeySmith
+
+
+def test_generate_api_key():
+    keysmith = APIKeySmith()
+    key = keysmith.generate_key()
+
+    assert key.key.startswith(keysmith.PREFIX)
+    assert key.head == key.key[: keysmith.HEAD_LENGTH]
+    assert key.tail == key.key[-keysmith.TAIL_LENGTH :]
+    assert len(key.hash) == 64  # 32 bytes hex encoded
+    assert len(key.salt) == 64  # 32 bytes hex encoded
+
+
+def test_verify_new_secure_key():
+    keysmith = APIKeySmith()
+    key = keysmith.generate_key()
+
+    # Test correct key validates
+    assert keysmith.verify_key(key.key, key.hash, key.salt) is True
+
+    # Test wrong key fails
+    wrong_key = f"{keysmith.PREFIX}wrongkey123"
+    assert keysmith.verify_key(wrong_key, key.hash, key.salt) is False
+
+
+def test_verify_legacy_key():
+    keysmith = APIKeySmith()
+    legacy_key = f"{keysmith.PREFIX}legacykey123"
+    legacy_hash = hashlib.sha256(legacy_key.encode()).hexdigest()
+
+    # Test legacy key validates without salt
+    assert keysmith.verify_key(legacy_key, legacy_hash) is True
+
+    # Test wrong legacy key fails
+    wrong_key = f"{keysmith.PREFIX}wronglegacy"
+    assert keysmith.verify_key(wrong_key, legacy_hash) is False
+
+
+def test_rehash_existing_key():
+    keysmith = APIKeySmith()
+    legacy_key = f"{keysmith.PREFIX}migratekey123"
+
+    # Migrate the legacy key
+    new_hash, new_salt = keysmith.hash_key(legacy_key)
+
+    # Verify migrated key works
+    assert keysmith.verify_key(legacy_key, new_hash, new_salt) is True
+
+    # Verify different key fails with migrated hash
+    wrong_key = f"{keysmith.PREFIX}wrongkey"
+    assert keysmith.verify_key(wrong_key, new_hash, new_salt) is False
+
+
+def test_invalid_key_prefix():
+    keysmith = APIKeySmith()
+    key = keysmith.generate_key()
+
+    # Test key without proper prefix fails
+    invalid_key = "invalid_prefix_key"
+    assert keysmith.verify_key(invalid_key, key.hash, key.salt) is False
+
+
+def test_secure_hash_requires_salt():
+    keysmith = APIKeySmith()
+    key = keysmith.generate_key()
+
+    # Secure hash without salt should fail
+    assert keysmith.verify_key(key.key, key.hash) is False
+
+
+def test_invalid_salt_format():
+    keysmith = APIKeySmith()
+    key = keysmith.generate_key()
+
+    # Invalid salt format should fail gracefully
+    assert keysmith.verify_key(key.key, key.hash, "invalid_hex") is False

autogpt_platform/autogpt_libs/autogpt_libs/auth/__init__.py

@@ -1,13 +1,13 @@
-from .depends import requires_admin_user, requires_user
-from .jwt_utils import parse_jwt_token
-from .middleware import APIKeyValidator, auth_middleware
+from .config import verify_settings
+from .dependencies import get_user_id, requires_admin_user, requires_user
+from .helpers import add_auth_responses_to_openapi
 from .models import User
 
 __all__ = [
-    "parse_jwt_token",
-    "requires_user",
+    "verify_settings",
+    "get_user_id",
     "requires_admin_user",
-    "APIKeyValidator",
-    "auth_middleware",
+    "requires_user",
+    "add_auth_responses_to_openapi",
     "User",
 ]

autogpt_platform/autogpt_libs/autogpt_libs/auth/config.py

@@ -1,11 +1,90 @@
+import logging
 import os
 
+from jwt.algorithms import get_default_algorithms, has_crypto
+
+logger = logging.getLogger(__name__)
+
+
+class AuthConfigError(ValueError):
+    """Raised when authentication configuration is invalid."""
+
+    pass
+
+
+ALGO_RECOMMENDATION = (
+    "We highly recommend using an asymmetric algorithm such as ES256, "
+    "because when leaked, a shared secret would allow anyone to "
+    "forge valid tokens and impersonate users. "
+    "More info: https://supabase.com/docs/guides/auth/signing-keys#choosing-the-right-signing-algorithm"  # noqa
+)
+
 
 class Settings:
     def __init__(self):
-        self.JWT_SECRET_KEY: str = os.getenv("SUPABASE_JWT_SECRET", "")
-        self.ENABLE_AUTH: bool = os.getenv("ENABLE_AUTH", "false").lower() == "true"
-        self.JWT_ALGORITHM: str = "HS256"
+        self.JWT_VERIFY_KEY: str = os.getenv(
+            "JWT_VERIFY_KEY", os.getenv("SUPABASE_JWT_SECRET", "")
+        ).strip()
+        self.JWT_ALGORITHM: str = os.getenv("JWT_SIGN_ALGORITHM", "HS256").strip()
+
+        self.validate()
+
+    def validate(self):
+        if not self.JWT_VERIFY_KEY:
+            raise AuthConfigError(
+                "JWT_VERIFY_KEY must be set. "
+                "An empty JWT secret would allow anyone to forge valid tokens."
+            )
+
+        if len(self.JWT_VERIFY_KEY) < 32:
+            logger.warning(
+                "⚠️ JWT_VERIFY_KEY appears weak (less than 32 characters). "
+                "Consider using a longer, cryptographically secure secret."
+            )
+
+        supported_algorithms = get_default_algorithms().keys()
+
+        if not has_crypto:
+            logger.warning(
+                "⚠️ Asymmetric JWT verification is not available "
+                "because the 'cryptography' package is not installed. "
+                + ALGO_RECOMMENDATION
+            )
+
+        if (
+            self.JWT_ALGORITHM not in supported_algorithms
+            or self.JWT_ALGORITHM == "none"
+        ):
+            raise AuthConfigError(
+                f"Invalid JWT_SIGN_ALGORITHM: '{self.JWT_ALGORITHM}'. "
+                "Supported algorithms are listed on "
+                "https://pyjwt.readthedocs.io/en/stable/algorithms.html"
+            )
+
+        if self.JWT_ALGORITHM.startswith("HS"):
+            logger.warning(
+                f"⚠️ JWT_SIGN_ALGORITHM is set to '{self.JWT_ALGORITHM}', "
+                "a symmetric shared-key signature algorithm. " + ALGO_RECOMMENDATION
+            )
 
 
-settings = Settings()
+_settings: Settings = None  # type: ignore
+
+
+def get_settings() -> Settings:
+    global _settings
+
+    if not _settings:
+        _settings = Settings()
+
+    return _settings
+
+
+def verify_settings() -> None:
+    global _settings
+
+    if not _settings:
+        _settings = Settings()  # calls validation indirectly
+        return
+
+    _settings.validate()

autogpt_platform/autogpt_libs/autogpt_libs/auth/config_test.py

@@ -0,0 +1,306 @@
+"""
+Comprehensive tests for auth configuration to ensure 100% line and branch coverage.
+These tests verify critical security checks preventing JWT token forgery.
+"""
+
+import logging
+import os
+
+import pytest
+from pytest_mock import MockerFixture
+
+from autogpt_libs.auth.config import AuthConfigError, Settings
+
+
+def test_environment_variable_precedence(mocker: MockerFixture):
+    """Test that environment variables take precedence over defaults."""
+    secret = "environment-secret-key-with-proper-length-123456"
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": secret}, clear=True)
+
+    settings = Settings()
+    assert settings.JWT_VERIFY_KEY == secret
+
+
+def test_environment_variable_backwards_compatible(mocker: MockerFixture):
+    """Test that SUPABASE_JWT_SECRET is read if JWT_VERIFY_KEY is not set."""
+    secret = "environment-secret-key-with-proper-length-123456"
+    mocker.patch.dict(os.environ, {"SUPABASE_JWT_SECRET": secret}, clear=True)
+
+    settings = Settings()
+    assert settings.JWT_VERIFY_KEY == secret
+
+
+def test_auth_config_error_inheritance():
+    """Test that AuthConfigError is properly defined as an Exception."""
+    assert issubclass(AuthConfigError, Exception)
+    error = AuthConfigError("test message")
+    assert str(error) == "test message"
+
+
+def test_settings_static_after_creation(mocker: MockerFixture):
+    """Test that settings maintain their values after creation."""
+    secret = "immutable-secret-key-with-proper-length-12345"
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": secret}, clear=True)
+
+    settings = Settings()
+    original_secret = settings.JWT_VERIFY_KEY
+
+    # Changing environment after creation shouldn't affect settings
+    os.environ["JWT_VERIFY_KEY"] = "different-secret"
+
+    assert settings.JWT_VERIFY_KEY == original_secret
+
+
+def test_settings_load_with_valid_secret(mocker: MockerFixture):
+    """Test auth enabled with a valid JWT secret."""
+    valid_secret = "a" * 32  # 32 character secret
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": valid_secret}, clear=True)
+
+    settings = Settings()
+    assert settings.JWT_VERIFY_KEY == valid_secret
+
+
+def test_settings_load_with_strong_secret(mocker: MockerFixture):
+    """Test auth enabled with a cryptographically strong secret."""
+    strong_secret = "super-secret-jwt-token-with-at-least-32-characters-long"
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": strong_secret}, clear=True)
+
+    settings = Settings()
+    assert settings.JWT_VERIFY_KEY == strong_secret
+    assert len(settings.JWT_VERIFY_KEY) >= 32
+
+
+def test_secret_empty_raises_error(mocker: MockerFixture):
+    """Test that auth enabled with empty secret raises AuthConfigError."""
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": ""}, clear=True)
+
+    with pytest.raises(Exception) as exc_info:
+        Settings()
+    assert "JWT_VERIFY_KEY" in str(exc_info.value)
+
+
+def test_secret_missing_raises_error(mocker: MockerFixture):
+    """Test that auth enabled without secret env var raises AuthConfigError."""
+    mocker.patch.dict(os.environ, {}, clear=True)
+
+    with pytest.raises(Exception) as exc_info:
+        Settings()
+    assert "JWT_VERIFY_KEY" in str(exc_info.value)
+
+
+@pytest.mark.parametrize("secret", [" ", "  ", "\t", "\n", " \t\n "])
+def test_secret_only_whitespace_raises_error(mocker: MockerFixture, secret: str):
+    """Test that auth enabled with whitespace-only secret raises error."""
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": secret}, clear=True)
+
+    with pytest.raises(ValueError):
+        Settings()
+
+
+def test_secret_weak_logs_warning(
+    mocker: MockerFixture, caplog: pytest.LogCaptureFixture
+):
+    """Test that weak JWT secret triggers warning log."""
+    weak_secret = "short"  # Less than 32 characters
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": weak_secret}, clear=True)
+
+    with caplog.at_level(logging.WARNING):
+        settings = Settings()
+        assert settings.JWT_VERIFY_KEY == weak_secret
+        assert "key appears weak" in caplog.text.lower()
+        assert "less than 32 characters" in caplog.text
+
+
+def test_secret_31_char_logs_warning(
+    mocker: MockerFixture, caplog: pytest.LogCaptureFixture
+):
+    """Test that 31-character secret triggers warning (boundary test)."""
+    secret_31 = "a" * 31  # Exactly 31 characters
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": secret_31}, clear=True)
+
+    with caplog.at_level(logging.WARNING):
+        settings = Settings()
+        assert len(settings.JWT_VERIFY_KEY) == 31
+        assert "key appears weak" in caplog.text.lower()
+
+
+def test_secret_32_char_no_warning(
+    mocker: MockerFixture, caplog: pytest.LogCaptureFixture
+):
+    """Test that 32-character secret does not trigger warning (boundary test)."""
+    secret_32 = "a" * 32  # Exactly 32 characters
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": secret_32}, clear=True)
+
+    with caplog.at_level(logging.WARNING):
+        settings = Settings()
+        assert len(settings.JWT_VERIFY_KEY) == 32
+        assert "JWT secret appears weak" not in caplog.text
+
+
+def test_secret_whitespace_stripped(mocker: MockerFixture):
+    """Test that JWT secret whitespace is stripped."""
+    secret = "a" * 32
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": f"  {secret}  "}, clear=True)
+
+    settings = Settings()
+    assert settings.JWT_VERIFY_KEY == secret
+
+
+def test_secret_with_special_characters(mocker: MockerFixture):
+    """Test JWT secret with special characters."""
+    special_secret = "!@#$%^&*()_+-=[]{}|;:,.<>?`~" + "a" * 10  # 40 chars total
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": special_secret}, clear=True)
+
+    settings = Settings()
+    assert settings.JWT_VERIFY_KEY == special_secret
+
+
+def test_secret_with_unicode(mocker: MockerFixture):
+    """Test JWT secret with unicode characters."""
+    unicode_secret = "秘密🔐キー" + "a" * 25  # Ensure >32 bytes
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": unicode_secret}, clear=True)
+
+    settings = Settings()
+    assert settings.JWT_VERIFY_KEY == unicode_secret
+
+
+def test_secret_very_long(mocker: MockerFixture):
+    """Test JWT secret with excessive length."""
+    long_secret = "a" * 1000  # 1000 character secret
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": long_secret}, clear=True)
+
+    settings = Settings()
+    assert settings.JWT_VERIFY_KEY == long_secret
+    assert len(settings.JWT_VERIFY_KEY) == 1000
+
+
+def test_secret_with_newline(mocker: MockerFixture):
+    """Test JWT secret containing newlines."""
+    multiline_secret = "secret\nwith\nnewlines" + "a" * 20
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": multiline_secret}, clear=True)
+
+    settings = Settings()
+    assert settings.JWT_VERIFY_KEY == multiline_secret
+
+
+def test_secret_base64_encoded(mocker: MockerFixture):
+    """Test JWT secret that looks like base64."""
+    base64_secret = "dGhpc19pc19hX3NlY3JldF9rZXlfd2l0aF9wcm9wZXJfbGVuZ3Ro"
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": base64_secret}, clear=True)
+
+    settings = Settings()
+    assert settings.JWT_VERIFY_KEY == base64_secret
+
+
+def test_secret_numeric_only(mocker: MockerFixture):
+    """Test JWT secret with only numbers."""
+    numeric_secret = "1234567890" * 4  # 40 character numeric secret
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": numeric_secret}, clear=True)
+
+    settings = Settings()
+    assert settings.JWT_VERIFY_KEY == numeric_secret
+
+
+def test_algorithm_default_hs256(mocker: MockerFixture):
+    """Test that JWT algorithm defaults to HS256."""
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": "a" * 32}, clear=True)
+
+    settings = Settings()
+    assert settings.JWT_ALGORITHM == "HS256"
+
+
+def test_algorithm_whitespace_stripped(mocker: MockerFixture):
+    """Test that JWT algorithm whitespace is stripped."""
+    secret = "a" * 32
+    mocker.patch.dict(
+        os.environ,
+        {"JWT_VERIFY_KEY": secret, "JWT_SIGN_ALGORITHM": "  HS256  "},
+        clear=True,
+    )
+
+    settings = Settings()
+    assert settings.JWT_ALGORITHM == "HS256"
+
+
+def test_no_crypto_warning(mocker: MockerFixture, caplog: pytest.LogCaptureFixture):
+    """Test warning when crypto package is not available."""
+    secret = "a" * 32
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": secret}, clear=True)
+
+    # Mock has_crypto to return False
+    mocker.patch("autogpt_libs.auth.config.has_crypto", False)
+
+    with caplog.at_level(logging.WARNING):
+        Settings()
+        assert "Asymmetric JWT verification is not available" in caplog.text
+        assert "cryptography" in caplog.text
+
+
+def test_algorithm_invalid_raises_error(mocker: MockerFixture):
+    """Test that invalid JWT algorithm raises AuthConfigError."""
+    secret = "a" * 32
+    mocker.patch.dict(
+        os.environ,
+        {"JWT_VERIFY_KEY": secret, "JWT_SIGN_ALGORITHM": "INVALID_ALG"},
+        clear=True,
+    )
+
+    with pytest.raises(AuthConfigError) as exc_info:
+        Settings()
+    assert "Invalid JWT_SIGN_ALGORITHM" in str(exc_info.value)
+    assert "INVALID_ALG" in str(exc_info.value)
+
+
+def test_algorithm_none_raises_error(mocker: MockerFixture):
+    """Test that 'none' algorithm raises AuthConfigError."""
+    secret = "a" * 32
+    mocker.patch.dict(
+        os.environ,
+        {"JWT_VERIFY_KEY": secret, "JWT_SIGN_ALGORITHM": "none"},
+        clear=True,
+    )
+
+    with pytest.raises(AuthConfigError) as exc_info:
+        Settings()
+    assert "Invalid JWT_SIGN_ALGORITHM" in str(exc_info.value)
+
+
+@pytest.mark.parametrize("algorithm", ["HS256", "HS384", "HS512"])
+def test_algorithm_symmetric_warning(
+    mocker: MockerFixture, caplog: pytest.LogCaptureFixture, algorithm: str
+):
+    """Test warning for symmetric algorithms (HS256, HS384, HS512)."""
+    secret = "a" * 32
+    mocker.patch.dict(
+        os.environ,
+        {"JWT_VERIFY_KEY": secret, "JWT_SIGN_ALGORITHM": algorithm},
+        clear=True,
+    )
+
+    with caplog.at_level(logging.WARNING):
+        settings = Settings()
+        assert algorithm in caplog.text
+        assert "symmetric shared-key signature algorithm" in caplog.text
+        assert settings.JWT_ALGORITHM == algorithm
+
+
+@pytest.mark.parametrize(
+    "algorithm",
+    ["ES256", "ES384", "ES512", "RS256", "RS384", "RS512", "PS256", "PS384", "PS512"],
+)
+def test_algorithm_asymmetric_no_warning(
+    mocker: MockerFixture, caplog: pytest.LogCaptureFixture, algorithm: str
+):
+    """Test that asymmetric algorithms do not trigger warning."""
+    secret = "a" * 32
+    mocker.patch.dict(
+        os.environ,
+        {"JWT_VERIFY_KEY": secret, "JWT_SIGN_ALGORITHM": algorithm},
+        clear=True,
+    )
+
+    with caplog.at_level(logging.WARNING):
+        settings = Settings()
+        # Should not contain the symmetric algorithm warning
+        assert "symmetric shared-key signature algorithm" not in caplog.text
+        assert settings.JWT_ALGORITHM == algorithm

autogpt_platform/autogpt_libs/autogpt_libs/auth/dependencies.py

@@ -0,0 +1,47 @@
+"""
+FastAPI dependency functions for JWT-based authentication and authorization.
+
+These are the high-level dependency functions used in route definitions.
+"""
+
+import fastapi
+
+from .jwt_utils import get_jwt_payload, verify_user
+from .models import User
+
+
+async def requires_user(jwt_payload: dict = fastapi.Security(get_jwt_payload)) -> User:
+    """
+    FastAPI dependency that requires a valid authenticated user.
+
+    Raises:
+        HTTPException: 401 for authentication failures
+    """
+    return verify_user(jwt_payload, admin_only=False)
+
+
+async def requires_admin_user(
+    jwt_payload: dict = fastapi.Security(get_jwt_payload),
+) -> User:
+    """
+    FastAPI dependency that requires a valid admin user.
+
+    Raises:
+        HTTPException: 401 for authentication failures, 403 for insufficient permissions
+    """
+    return verify_user(jwt_payload, admin_only=True)
+
+
+async def get_user_id(jwt_payload: dict = fastapi.Security(get_jwt_payload)) -> str:
+    """
+    FastAPI dependency that returns the ID of the authenticated user.
+
+    Raises:
+        HTTPException: 401 for authentication failures or missing user ID
+    """
+    user_id = jwt_payload.get("sub")
+    if not user_id:
+        raise fastapi.HTTPException(
+            status_code=401, detail="User ID not found in token"
+        )
+    return user_id

autogpt_platform/autogpt_libs/autogpt_libs/auth/dependencies_test.py

@@ -0,0 +1,335 @@
+"""
+Comprehensive integration tests for authentication dependencies.
+Tests the full authentication flow from HTTP requests to user validation.
+"""
+
+import os
+
+import pytest
+from fastapi import FastAPI, HTTPException, Security
+from fastapi.testclient import TestClient
+from pytest_mock import MockerFixture
+
+from autogpt_libs.auth.dependencies import (
+    get_user_id,
+    requires_admin_user,
+    requires_user,
+)
+from autogpt_libs.auth.models import User
+
+
+class TestAuthDependencies:
+    """Test suite for authentication dependency functions."""
+
+    @pytest.fixture
+    def app(self):
+        """Create a test FastAPI application."""
+        app = FastAPI()
+
+        @app.get("/user")
+        def get_user_endpoint(user: User = Security(requires_user)):
+            return {"user_id": user.user_id, "role": user.role}
+
+        @app.get("/admin")
+        def get_admin_endpoint(user: User = Security(requires_admin_user)):
+            return {"user_id": user.user_id, "role": user.role}
+
+        @app.get("/user-id")
+        def get_user_id_endpoint(user_id: str = Security(get_user_id)):
+            return {"user_id": user_id}
+
+        return app
+
+    @pytest.fixture
+    def client(self, app):
+        """Create a test client."""
+        return TestClient(app)
+
+    async def test_requires_user_with_valid_jwt_payload(self, mocker: MockerFixture):
+        """Test requires_user with valid JWT payload."""
+        jwt_payload = {"sub": "user-123", "role": "user", "email": "user@example.com"}
+
+        # Mock get_jwt_payload to return our test payload
+        mocker.patch(
+            "autogpt_libs.auth.dependencies.get_jwt_payload", return_value=jwt_payload
+        )
+        user = await requires_user(jwt_payload)
+        assert isinstance(user, User)
+        assert user.user_id == "user-123"
+        assert user.role == "user"
+
+    async def test_requires_user_with_admin_jwt_payload(self, mocker: MockerFixture):
+        """Test requires_user accepts admin users."""
+        jwt_payload = {
+            "sub": "admin-456",
+            "role": "admin",
+            "email": "admin@example.com",
+        }
+
+        mocker.patch(
+            "autogpt_libs.auth.dependencies.get_jwt_payload", return_value=jwt_payload
+        )
+        user = await requires_user(jwt_payload)
+        assert user.user_id == "admin-456"
+        assert user.role == "admin"
+
+    async def test_requires_user_missing_sub(self):
+        """Test requires_user with missing user ID."""
+        jwt_payload = {"role": "user", "email": "user@example.com"}
+
+        with pytest.raises(HTTPException) as exc_info:
+            await requires_user(jwt_payload)
+        assert exc_info.value.status_code == 401
+        assert "User ID not found" in exc_info.value.detail
+
+    async def test_requires_user_empty_sub(self):
+        """Test requires_user with empty user ID."""
+        jwt_payload = {"sub": "", "role": "user"}
+
+        with pytest.raises(HTTPException) as exc_info:
+            await requires_user(jwt_payload)
+        assert exc_info.value.status_code == 401
+
+    async def test_requires_admin_user_with_admin(self, mocker: MockerFixture):
+        """Test requires_admin_user with admin role."""
+        jwt_payload = {
+            "sub": "admin-789",
+            "role": "admin",
+            "email": "admin@example.com",
+        }
+
+        mocker.patch(
+            "autogpt_libs.auth.dependencies.get_jwt_payload", return_value=jwt_payload
+        )
+        user = await requires_admin_user(jwt_payload)
+        assert user.user_id == "admin-789"
+        assert user.role == "admin"
+
+    async def test_requires_admin_user_with_regular_user(self):
+        """Test requires_admin_user rejects regular users."""
+        jwt_payload = {"sub": "user-123", "role": "user", "email": "user@example.com"}
+
+        with pytest.raises(HTTPException) as exc_info:
+            await requires_admin_user(jwt_payload)
+        assert exc_info.value.status_code == 403
+        assert "Admin access required" in exc_info.value.detail
+
+    async def test_requires_admin_user_missing_role(self):
+        """Test requires_admin_user with missing role."""
+        jwt_payload = {"sub": "user-123", "email": "user@example.com"}
+
+        with pytest.raises(KeyError):
+            await requires_admin_user(jwt_payload)
+
+    async def test_get_user_id_with_valid_payload(self, mocker: MockerFixture):
+        """Test get_user_id extracts user ID correctly."""
+        jwt_payload = {"sub": "user-id-xyz", "role": "user"}
+
+        mocker.patch(
+            "autogpt_libs.auth.dependencies.get_jwt_payload", return_value=jwt_payload
+        )
+        user_id = await get_user_id(jwt_payload)
+        assert user_id == "user-id-xyz"
+
+    async def test_get_user_id_missing_sub(self):
+        """Test get_user_id with missing user ID."""
+        jwt_payload = {"role": "user"}
+
+        with pytest.raises(HTTPException) as exc_info:
+            await get_user_id(jwt_payload)
+        assert exc_info.value.status_code == 401
+        assert "User ID not found" in exc_info.value.detail
+
+    async def test_get_user_id_none_sub(self):
+        """Test get_user_id with None user ID."""
+        jwt_payload = {"sub": None, "role": "user"}
+
+        with pytest.raises(HTTPException) as exc_info:
+            await get_user_id(jwt_payload)
+        assert exc_info.value.status_code == 401
+
+
+class TestAuthDependenciesIntegration:
+    """Integration tests for auth dependencies with FastAPI."""
+
+    acceptable_jwt_secret = "test-secret-with-proper-length-123456"
+
+    @pytest.fixture
+    def create_token(self, mocker: MockerFixture):
+        """Helper to create JWT tokens."""
+        import jwt
+
+        mocker.patch.dict(
+            os.environ,
+            {"JWT_VERIFY_KEY": self.acceptable_jwt_secret},
+            clear=True,
+        )
+
+        def _create_token(payload, secret=self.acceptable_jwt_secret):
+            return jwt.encode(payload, secret, algorithm="HS256")
+
+        return _create_token
+
+    async def test_endpoint_auth_enabled_no_token(self):
+        """Test endpoints require token when auth is enabled."""
+        app = FastAPI()
+
+        @app.get("/test")
+        def test_endpoint(user: User = Security(requires_user)):
+            return {"user_id": user.user_id}
+
+        client = TestClient(app)
+
+        # Should fail without auth header
+        response = client.get("/test")
+        assert response.status_code == 401
+
+    async def test_endpoint_with_valid_token(self, create_token):
+        """Test endpoint with valid JWT token."""
+        app = FastAPI()
+
+        @app.get("/test")
+        def test_endpoint(user: User = Security(requires_user)):
+            return {"user_id": user.user_id, "role": user.role}
+
+        client = TestClient(app)
+
+        token = create_token(
+            {"sub": "test-user", "role": "user", "aud": "authenticated"},
+            secret=self.acceptable_jwt_secret,
+        )
+
+        response = client.get("/test", headers={"Authorization": f"Bearer {token}"})
+        assert response.status_code == 200
+        assert response.json()["user_id"] == "test-user"
+
+    async def test_admin_endpoint_requires_admin_role(self, create_token):
+        """Test admin endpoint rejects non-admin users."""
+        app = FastAPI()
+
+        @app.get("/admin")
+        def admin_endpoint(user: User = Security(requires_admin_user)):
+            return {"user_id": user.user_id}
+
+        client = TestClient(app)
+
+        # Regular user token
+        user_token = create_token(
+            {"sub": "regular-user", "role": "user", "aud": "authenticated"},
+            secret=self.acceptable_jwt_secret,
+        )
+
+        response = client.get(
+            "/admin", headers={"Authorization": f"Bearer {user_token}"}
+        )
+        assert response.status_code == 403
+
+        # Admin token
+        admin_token = create_token(
+            {"sub": "admin-user", "role": "admin", "aud": "authenticated"},
+            secret=self.acceptable_jwt_secret,
+        )
+
+        response = client.get(
+            "/admin", headers={"Authorization": f"Bearer {admin_token}"}
+        )
+        assert response.status_code == 200
+        assert response.json()["user_id"] == "admin-user"
+
+
+class TestAuthDependenciesEdgeCases:
+    """Edge case tests for authentication dependencies."""
+
+    async def test_dependency_with_complex_payload(self):
+        """Test dependencies handle complex JWT payloads."""
+        complex_payload = {
+            "sub": "user-123",
+            "role": "admin",
+            "email": "test@example.com",
+            "app_metadata": {"provider": "email", "providers": ["email"]},
+            "user_metadata": {
+                "full_name": "Test User",
+                "avatar_url": "https://example.com/avatar.jpg",
+            },
+            "aud": "authenticated",
+            "iat": 1234567890,
+            "exp": 9999999999,
+        }
+
+        user = await requires_user(complex_payload)
+        assert user.user_id == "user-123"
+        assert user.email == "test@example.com"
+
+        admin = await requires_admin_user(complex_payload)
+        assert admin.role == "admin"
+
+    async def test_dependency_with_unicode_in_payload(self):
+        """Test dependencies handle unicode in JWT payloads."""
+        unicode_payload = {
+            "sub": "user-😀-123",
+            "role": "user",
+            "email": "测试@example.com",
+            "name": "日本語",
+        }
+
+        user = await requires_user(unicode_payload)
+        assert "😀" in user.user_id
+        assert user.email == "测试@example.com"
+
+    async def test_dependency_with_null_values(self):
+        """Test dependencies handle null values in payload."""
+        null_payload = {
+            "sub": "user-123",
+            "role": "user",
+            "email": None,
+            "phone": None,
+            "metadata": None,
+        }
+
+        user = await requires_user(null_payload)
+        assert user.user_id == "user-123"
+        assert user.email is None
+
+    async def test_concurrent_requests_isolation(self):
+        """Test that concurrent requests don't interfere with each other."""
+        payload1 = {"sub": "user-1", "role": "user"}
+        payload2 = {"sub": "user-2", "role": "admin"}
+
+        # Simulate concurrent processing
+        user1 = await requires_user(payload1)
+        user2 = await requires_admin_user(payload2)
+
+        assert user1.user_id == "user-1"
+        assert user2.user_id == "user-2"
+        assert user1.role == "user"
+        assert user2.role == "admin"
+
+    @pytest.mark.parametrize(
+        "payload,expected_error,admin_only",
+        [
+            (None, "Authorization header is missing", False),
+            ({}, "User ID not found", False),
+            ({"sub": ""}, "User ID not found", False),
+            ({"role": "user"}, "User ID not found", False),
+            ({"sub": "user", "role": "user"}, "Admin access required", True),
+        ],
+    )
+    async def test_dependency_error_cases(
+        self, payload, expected_error: str, admin_only: bool
+    ):
+        """Test that errors propagate correctly through dependencies."""
+        # Import verify_user to test it directly since dependencies use FastAPI Security
+        from autogpt_libs.auth.jwt_utils import verify_user
+
+        with pytest.raises(HTTPException) as exc_info:
+            verify_user(payload, admin_only=admin_only)
+        assert expected_error in exc_info.value.detail
+
+    async def test_dependency_valid_user(self):
+        """Test valid user case for dependency."""
+        # Import verify_user to test it directly since dependencies use FastAPI Security
+        from autogpt_libs.auth.jwt_utils import verify_user
+
+        # Valid case
+        user = verify_user({"sub": "user", "role": "user"}, admin_only=False)
+        assert user.user_id == "user"

autogpt_platform/autogpt_libs/autogpt_libs/auth/depends.py

@@ -1,46 +0,0 @@
-import fastapi
-
-from .config import settings
-from .middleware import auth_middleware
-from .models import DEFAULT_USER_ID, User
-
-
-def requires_user(payload: dict = fastapi.Depends(auth_middleware)) -> User:
-    return verify_user(payload, admin_only=False)
-
-
-def requires_admin_user(
-    payload: dict = fastapi.Depends(auth_middleware),
-) -> User:
-    return verify_user(payload, admin_only=True)
-
-
-def verify_user(payload: dict | None, admin_only: bool) -> User:
-    if not payload:
-        if settings.ENABLE_AUTH:
-            raise fastapi.HTTPException(
-                status_code=401, detail="Authorization header is missing"
-            )
-        # This handles the case when authentication is disabled
-        payload = {"sub": DEFAULT_USER_ID, "role": "admin"}
-
-    user_id = payload.get("sub")
-
-    if not user_id:
-        raise fastapi.HTTPException(
-            status_code=401, detail="User ID not found in token"
-        )
-
-    if admin_only and payload["role"] != "admin":
-        raise fastapi.HTTPException(status_code=403, detail="Admin access required")
-
-    return User.from_payload(payload)
-
-
-def get_user_id(payload: dict = fastapi.Depends(auth_middleware)) -> str:
-    user_id = payload.get("sub")
-    if not user_id:
-        raise fastapi.HTTPException(
-            status_code=401, detail="User ID not found in token"
-        )
-    return user_id

autogpt_platform/autogpt_libs/autogpt_libs/auth/depends_tests.py

@@ -1,68 +0,0 @@
-import pytest
-
-from .depends import requires_admin_user, requires_user, verify_user
-
-
-def test_verify_user_no_payload():
-    user = verify_user(None, admin_only=False)
-    assert user.user_id == "3e53486c-cf57-477e-ba2a-cb02dc828e1a"
-    assert user.role == "admin"
-
-
-def test_verify_user_no_user_id():
-    with pytest.raises(Exception):
-        verify_user({"role": "admin"}, admin_only=False)
-
-
-def test_verify_user_not_admin():
-    with pytest.raises(Exception):
-        verify_user(
-            {"sub": "3e53486c-cf57-477e-ba2a-cb02dc828e1a", "role": "user"},
-            admin_only=True,
-        )
-
-
-def test_verify_user_with_admin_role():
-    user = verify_user(
-        {"sub": "3e53486c-cf57-477e-ba2a-cb02dc828e1a", "role": "admin"},
-        admin_only=True,
-    )
-    assert user.user_id == "3e53486c-cf57-477e-ba2a-cb02dc828e1a"
-    assert user.role == "admin"
-
-
-def test_verify_user_with_user_role():
-    user = verify_user(
-        {"sub": "3e53486c-cf57-477e-ba2a-cb02dc828e1a", "role": "user"},
-        admin_only=False,
-    )
-    assert user.user_id == "3e53486c-cf57-477e-ba2a-cb02dc828e1a"
-    assert user.role == "user"
-
-
-def test_requires_user():
-    user = requires_user(
-        {"sub": "3e53486c-cf57-477e-ba2a-cb02dc828e1a", "role": "user"}
-    )
-    assert user.user_id == "3e53486c-cf57-477e-ba2a-cb02dc828e1a"
-    assert user.role == "user"
-
-
-def test_requires_user_no_user_id():
-    with pytest.raises(Exception):
-        requires_user({"role": "user"})
-
-
-def test_requires_admin_user():
-    user = requires_admin_user(
-        {"sub": "3e53486c-cf57-477e-ba2a-cb02dc828e1a", "role": "admin"}
-    )
-    assert user.user_id == "3e53486c-cf57-477e-ba2a-cb02dc828e1a"
-    assert user.role == "admin"
-
-
-def test_requires_admin_user_not_admin():
-    with pytest.raises(Exception):
-        requires_admin_user(
-            {"sub": "3e53486c-cf57-477e-ba2a-cb02dc828e1a", "role": "user"}
-        )

autogpt_platform/autogpt_libs/autogpt_libs/auth/helpers.py

@@ -0,0 +1,68 @@
+from fastapi import FastAPI
+from fastapi.openapi.utils import get_openapi
+
+from .jwt_utils import bearer_jwt_auth
+
+
+def add_auth_responses_to_openapi(app: FastAPI) -> None:
+    """
+    Set up custom OpenAPI schema generation that adds 401 responses
+    to all authenticated endpoints.
+
+    This is needed when using HTTPBearer with auto_error=False to get proper
+    401 responses instead of 403, but FastAPI only automatically adds security
+    responses when auto_error=True.
+    """
+
+    def custom_openapi():
+        if app.openapi_schema:
+            return app.openapi_schema
+
+        openapi_schema = get_openapi(
+            title=app.title,
+            version=app.version,
+            description=app.description,
+            routes=app.routes,
+        )
+
+        # Add 401 response to all endpoints that have security requirements
+        for path, methods in openapi_schema["paths"].items():
+            for method, details in methods.items():
+                security_schemas = [
+                    schema
+                    for auth_option in details.get("security", [])
+                    for schema in auth_option.keys()
+                ]
+                if bearer_jwt_auth.scheme_name not in security_schemas:
+                    continue
+
+                if "responses" not in details:
+                    details["responses"] = {}
+
+                details["responses"]["401"] = {
+                    "$ref": "#/components/responses/HTTP401NotAuthenticatedError"
+                }
+
+        # Ensure #/components/responses exists
+        if "components" not in openapi_schema:
+            openapi_schema["components"] = {}
+        if "responses" not in openapi_schema["components"]:
+            openapi_schema["components"]["responses"] = {}
+
+        # Define 401 response
+        openapi_schema["components"]["responses"]["HTTP401NotAuthenticatedError"] = {
+            "description": "Authentication required",
+            "content": {
+                "application/json": {
+                    "schema": {
+                        "type": "object",
+                        "properties": {"detail": {"type": "string"}},
+                    }
+                }
+            },
+        }
+
+        app.openapi_schema = openapi_schema
+        return app.openapi_schema
+
+    app.openapi = custom_openapi

autogpt_platform/autogpt_libs/autogpt_libs/auth/helpers_test.py

@@ -0,0 +1,435 @@
+"""
+Comprehensive tests for auth helpers module to achieve 100% coverage.
+Tests OpenAPI schema generation and authentication response handling.
+"""
+
+from unittest import mock
+
+from fastapi import FastAPI
+from fastapi.openapi.utils import get_openapi
+
+from autogpt_libs.auth.helpers import add_auth_responses_to_openapi
+from autogpt_libs.auth.jwt_utils import bearer_jwt_auth
+
+
+def test_add_auth_responses_to_openapi_basic():
+    """Test adding 401 responses to OpenAPI schema."""
+    app = FastAPI(title="Test App", version="1.0.0")
+
+    # Add some test endpoints with authentication
+    from fastapi import Depends
+
+    from autogpt_libs.auth.dependencies import requires_user
+
+    @app.get("/protected", dependencies=[Depends(requires_user)])
+    def protected_endpoint():
+        return {"message": "Protected"}
+
+    @app.get("/public")
+    def public_endpoint():
+        return {"message": "Public"}
+
+    # Apply the OpenAPI customization
+    add_auth_responses_to_openapi(app)
+
+    # Get the OpenAPI schema
+    schema = app.openapi()
+
+    # Verify basic schema properties
+    assert schema["info"]["title"] == "Test App"
+    assert schema["info"]["version"] == "1.0.0"
+
+    # Verify 401 response component is added
+    assert "components" in schema
+    assert "responses" in schema["components"]
+    assert "HTTP401NotAuthenticatedError" in schema["components"]["responses"]
+
+    # Verify 401 response structure
+    error_response = schema["components"]["responses"]["HTTP401NotAuthenticatedError"]
+    assert error_response["description"] == "Authentication required"
+    assert "application/json" in error_response["content"]
+    assert "schema" in error_response["content"]["application/json"]
+
+    # Verify schema properties
+    response_schema = error_response["content"]["application/json"]["schema"]
+    assert response_schema["type"] == "object"
+    assert "detail" in response_schema["properties"]
+    assert response_schema["properties"]["detail"]["type"] == "string"
+
+
+def test_add_auth_responses_to_openapi_with_security():
+    """Test that 401 responses are added only to secured endpoints."""
+    app = FastAPI()
+
+    # Mock endpoint with security
+    from fastapi import Security
+
+    from autogpt_libs.auth.dependencies import get_user_id
+
+    @app.get("/secured")
+    def secured_endpoint(user_id: str = Security(get_user_id)):
+        return {"user_id": user_id}
+
+    @app.post("/also-secured")
+    def another_secured(user_id: str = Security(get_user_id)):
+        return {"status": "ok"}
+
+    @app.get("/unsecured")
+    def unsecured_endpoint():
+        return {"public": True}
+
+    # Apply OpenAPI customization
+    add_auth_responses_to_openapi(app)
+
+    # Get schema
+    schema = app.openapi()
+
+    # Check that secured endpoints have 401 responses
+    if "/secured" in schema["paths"]:
+        if "get" in schema["paths"]["/secured"]:
+            secured_get = schema["paths"]["/secured"]["get"]
+            if "responses" in secured_get:
+                assert "401" in secured_get["responses"]
+                assert (
+                    secured_get["responses"]["401"]["$ref"]
+                    == "#/components/responses/HTTP401NotAuthenticatedError"
+                )
+
+    if "/also-secured" in schema["paths"]:
+        if "post" in schema["paths"]["/also-secured"]:
+            secured_post = schema["paths"]["/also-secured"]["post"]
+            if "responses" in secured_post:
+                assert "401" in secured_post["responses"]
+
+    # Check that unsecured endpoint does not have 401 response
+    if "/unsecured" in schema["paths"]:
+        if "get" in schema["paths"]["/unsecured"]:
+            unsecured_get = schema["paths"]["/unsecured"]["get"]
+            if "responses" in unsecured_get:
+                assert "401" not in unsecured_get.get("responses", {})
+
+
+def test_add_auth_responses_to_openapi_cached_schema():
+    """Test that OpenAPI schema is cached after first generation."""
+    app = FastAPI()
+
+    # Apply customization
+    add_auth_responses_to_openapi(app)
+
+    # Get schema twice
+    schema1 = app.openapi()
+    schema2 = app.openapi()
+
+    # Should return the same cached object
+    assert schema1 is schema2
+
+
+def test_add_auth_responses_to_openapi_existing_responses():
+    """Test handling endpoints that already have responses defined."""
+    app = FastAPI()
+
+    from fastapi import Security
+
+    from autogpt_libs.auth.jwt_utils import get_jwt_payload
+
+    @app.get(
+        "/with-responses",
+        responses={
+            200: {"description": "Success"},
+            404: {"description": "Not found"},
+        },
+    )
+    def endpoint_with_responses(jwt: dict = Security(get_jwt_payload)):
+        return {"data": "test"}
+
+    # Apply customization
+    add_auth_responses_to_openapi(app)
+
+    schema = app.openapi()
+
+    # Check that existing responses are preserved and 401 is added
+    if "/with-responses" in schema["paths"]:
+        if "get" in schema["paths"]["/with-responses"]:
+            responses = schema["paths"]["/with-responses"]["get"].get("responses", {})
+            # Original responses should be preserved
+            if "200" in responses:
+                assert responses["200"]["description"] == "Success"
+            if "404" in responses:
+                assert responses["404"]["description"] == "Not found"
+            # 401 should be added
+            if "401" in responses:
+                assert (
+                    responses["401"]["$ref"]
+                    == "#/components/responses/HTTP401NotAuthenticatedError"
+                )
+
+
+def test_add_auth_responses_to_openapi_no_security_endpoints():
+    """Test with app that has no secured endpoints."""
+    app = FastAPI()
+
+    @app.get("/public1")
+    def public1():
+        return {"message": "public1"}
+
+    @app.post("/public2")
+    def public2():
+        return {"message": "public2"}
+
+    # Apply customization
+    add_auth_responses_to_openapi(app)
+
+    schema = app.openapi()
+
+    # Component should still be added for consistency
+    assert "HTTP401NotAuthenticatedError" in schema["components"]["responses"]
+
+    # But no endpoints should have 401 responses
+    for path in schema["paths"].values():
+        for method in path.values():
+            if isinstance(method, dict) and "responses" in method:
+                assert "401" not in method["responses"]
+
+
+def test_add_auth_responses_to_openapi_multiple_security_schemes():
+    """Test endpoints with multiple security requirements."""
+    app = FastAPI()
+
+    from fastapi import Security
+
+    from autogpt_libs.auth.dependencies import requires_admin_user, requires_user
+    from autogpt_libs.auth.models import User
+
+    @app.get("/multi-auth")
+    def multi_auth(
+        user: User = Security(requires_user),
+        admin: User = Security(requires_admin_user),
+    ):
+        return {"status": "super secure"}
+
+    # Apply customization
+    add_auth_responses_to_openapi(app)
+
+    schema = app.openapi()
+
+    # Should have 401 response
+    if "/multi-auth" in schema["paths"]:
+        if "get" in schema["paths"]["/multi-auth"]:
+            responses = schema["paths"]["/multi-auth"]["get"].get("responses", {})
+            if "401" in responses:
+                assert (
+                    responses["401"]["$ref"]
+                    == "#/components/responses/HTTP401NotAuthenticatedError"
+                )
+
+
+def test_add_auth_responses_to_openapi_empty_components():
+    """Test when OpenAPI schema has no components section initially."""
+    app = FastAPI()
+
+    # Mock get_openapi to return schema without components
+    original_get_openapi = get_openapi
+
+    def mock_get_openapi(*args, **kwargs):
+        schema = original_get_openapi(*args, **kwargs)
+        # Remove components if it exists
+        if "components" in schema:
+            del schema["components"]
+        return schema
+
+    with mock.patch("autogpt_libs.auth.helpers.get_openapi", mock_get_openapi):
+        # Apply customization
+        add_auth_responses_to_openapi(app)
+
+        schema = app.openapi()
+
+        # Components should be created
+        assert "components" in schema
+        assert "responses" in schema["components"]
+        assert "HTTP401NotAuthenticatedError" in schema["components"]["responses"]
+
+
+def test_add_auth_responses_to_openapi_all_http_methods():
+    """Test that all HTTP methods are handled correctly."""
+    app = FastAPI()
+
+    from fastapi import Security
+
+    from autogpt_libs.auth.jwt_utils import get_jwt_payload
+
+    @app.get("/resource")
+    def get_resource(jwt: dict = Security(get_jwt_payload)):
+        return {"method": "GET"}
+
+    @app.post("/resource")
+    def post_resource(jwt: dict = Security(get_jwt_payload)):
+        return {"method": "POST"}
+
+    @app.put("/resource")
+    def put_resource(jwt: dict = Security(get_jwt_payload)):
+        return {"method": "PUT"}
+
+    @app.patch("/resource")
+    def patch_resource(jwt: dict = Security(get_jwt_payload)):
+        return {"method": "PATCH"}
+
+    @app.delete("/resource")
+    def delete_resource(jwt: dict = Security(get_jwt_payload)):
+        return {"method": "DELETE"}
+
+    # Apply customization
+    add_auth_responses_to_openapi(app)
+
+    schema = app.openapi()
+
+    # All methods should have 401 response
+    if "/resource" in schema["paths"]:
+        for method in ["get", "post", "put", "patch", "delete"]:
+            if method in schema["paths"]["/resource"]:
+                method_spec = schema["paths"]["/resource"][method]
+                if "responses" in method_spec:
+                    assert "401" in method_spec["responses"]
+
+
+def test_bearer_jwt_auth_scheme_config():
+    """Test that bearer_jwt_auth is configured correctly."""
+    assert bearer_jwt_auth.scheme_name == "HTTPBearerJWT"
+    assert bearer_jwt_auth.auto_error is False
+
+
+def test_add_auth_responses_with_no_routes():
+    """Test OpenAPI generation with app that has no routes."""
+    app = FastAPI(title="Empty App")
+
+    # Apply customization to empty app
+    add_auth_responses_to_openapi(app)
+
+    schema = app.openapi()
+
+    # Should still have basic structure
+    assert schema["info"]["title"] == "Empty App"
+    assert "components" in schema
+    assert "responses" in schema["components"]
+    assert "HTTP401NotAuthenticatedError" in schema["components"]["responses"]
+
+
+def test_custom_openapi_function_replacement():
+    """Test that the custom openapi function properly replaces the default."""
+    app = FastAPI()
+
+    # Store original function
+    original_openapi = app.openapi
+
+    # Apply customization
+    add_auth_responses_to_openapi(app)
+
+    # Function should be replaced
+    assert app.openapi != original_openapi
+    assert callable(app.openapi)
+
+
+def test_endpoint_without_responses_section():
+    """Test endpoint that has security but no responses section initially."""
+    app = FastAPI()
+
+    from fastapi import Security
+    from fastapi.openapi.utils import get_openapi as original_get_openapi
+
+    from autogpt_libs.auth.jwt_utils import get_jwt_payload
+
+    # Create endpoint
+    @app.get("/no-responses")
+    def endpoint_without_responses(jwt: dict = Security(get_jwt_payload)):
+        return {"data": "test"}
+
+    # Mock get_openapi to remove responses from the endpoint
+    def mock_get_openapi(*args, **kwargs):
+        schema = original_get_openapi(*args, **kwargs)
+        # Remove responses from our endpoint to trigger line 40
+        if "/no-responses" in schema.get("paths", {}):
+            if "get" in schema["paths"]["/no-responses"]:
+                # Delete responses to force the code to create it
+                if "responses" in schema["paths"]["/no-responses"]["get"]:
+                    del schema["paths"]["/no-responses"]["get"]["responses"]
+        return schema
+
+    with mock.patch("autogpt_libs.auth.helpers.get_openapi", mock_get_openapi):
+        # Apply customization
+        add_auth_responses_to_openapi(app)
+
+        # Get schema and verify 401 was added
+        schema = app.openapi()
+
+        # The endpoint should now have 401 response
+        if "/no-responses" in schema["paths"]:
+            if "get" in schema["paths"]["/no-responses"]:
+                responses = schema["paths"]["/no-responses"]["get"].get("responses", {})
+                assert "401" in responses
+                assert (
+                    responses["401"]["$ref"]
+                    == "#/components/responses/HTTP401NotAuthenticatedError"
+                )
+
+
+def test_components_with_existing_responses():
+    """Test when components already has a responses section."""
+    app = FastAPI()
+
+    # Mock get_openapi to return schema with existing components/responses
+    from fastapi.openapi.utils import get_openapi as original_get_openapi
+
+    def mock_get_openapi(*args, **kwargs):
+        schema = original_get_openapi(*args, **kwargs)
+        # Add existing components/responses
+        if "components" not in schema:
+            schema["components"] = {}
+        schema["components"]["responses"] = {
+            "ExistingResponse": {"description": "An existing response"}
+        }
+        return schema
+
+    with mock.patch("autogpt_libs.auth.helpers.get_openapi", mock_get_openapi):
+        # Apply customization
+        add_auth_responses_to_openapi(app)
+
+        schema = app.openapi()
+
+        # Both responses should exist
+        assert "ExistingResponse" in schema["components"]["responses"]
+        assert "HTTP401NotAuthenticatedError" in schema["components"]["responses"]
+
+        # Verify our 401 response structure
+        error_response = schema["components"]["responses"][
+            "HTTP401NotAuthenticatedError"
+        ]
+        assert error_response["description"] == "Authentication required"
+
+
+def test_openapi_schema_persistence():
+    """Test that modifications to OpenAPI schema persist correctly."""
+    app = FastAPI()
+
+    from fastapi import Security
+
+    from autogpt_libs.auth.jwt_utils import get_jwt_payload
+
+    @app.get("/test")
+    def test_endpoint(jwt: dict = Security(get_jwt_payload)):
+        return {"test": True}
+
+    # Apply customization
+    add_auth_responses_to_openapi(app)
+
+    # Get schema multiple times
+    schema1 = app.openapi()
+
+    # Modify the cached schema (shouldn't affect future calls)
+    schema1["info"]["title"] = "Modified Title"
+
+    # Clear cache and get again
+    app.openapi_schema = None
+    schema2 = app.openapi()
+
+    # Should regenerate with original title
+    assert schema2["info"]["title"] == app.title
+    assert schema2["info"]["title"] != "Modified Title"

autogpt_platform/autogpt_libs/autogpt_libs/auth/jwt_utils.py

@@ -1,11 +1,48 @@
-from typing import Any, Dict
+import logging
+from typing import Any
 
 import jwt
+from fastapi import HTTPException, Security
+from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
 
-from .config import settings
+from .config import get_settings
+from .models import User
+
+logger = logging.getLogger(__name__)
+
+# Bearer token authentication scheme
+bearer_jwt_auth = HTTPBearer(
+    bearerFormat="jwt", scheme_name="HTTPBearerJWT", auto_error=False
+)
 
 
-def parse_jwt_token(token: str) -> Dict[str, Any]:
+async def get_jwt_payload(
+    credentials: HTTPAuthorizationCredentials | None = Security(bearer_jwt_auth),
+) -> dict[str, Any]:
+    """
+    Extract and validate JWT payload from HTTP Authorization header.
+
+    This is the core authentication function that handles:
+    - Reading the `Authorization` header to obtain the JWT token
+    - Verifying the JWT token's signature
+    - Decoding the JWT token's payload
+
+    :param credentials: HTTP Authorization credentials from bearer token
+    :return: JWT payload dictionary
+    :raises HTTPException: 401 if authentication fails
+    """
+    if not credentials:
+        raise HTTPException(status_code=401, detail="Authorization header is missing")
+
+    try:
+        payload = parse_jwt_token(credentials.credentials)
+        logger.debug("Token decoded successfully")
+        return payload
+    except ValueError as e:
+        raise HTTPException(status_code=401, detail=str(e))
+
+
+def parse_jwt_token(token: str) -> dict[str, Any]:
     """
     Parse and validate a JWT token.
 
@@ -13,10 +50,11 @@ def parse_jwt_token(token: str) -> Dict[str, Any]:
     :return: The decoded payload
     :raises ValueError: If the token is invalid or expired
     """
+    settings = get_settings()
     try:
         payload = jwt.decode(
             token,
-            settings.JWT_SECRET_KEY,
+            settings.JWT_VERIFY_KEY,
             algorithms=[settings.JWT_ALGORITHM],
             audience="authenticated",
         )
@@ -25,3 +63,18 @@ def parse_jwt_token(token: str) -> Dict[str, Any]:
         raise ValueError("Token has expired")
     except jwt.InvalidTokenError as e:
         raise ValueError(f"Invalid token: {str(e)}")
+
+
+def verify_user(jwt_payload: dict | None, admin_only: bool) -> User:
+    if jwt_payload is None:
+        raise HTTPException(status_code=401, detail="Authorization header is missing")
+
+    user_id = jwt_payload.get("sub")
+
+    if not user_id:
+        raise HTTPException(status_code=401, detail="User ID not found in token")
+
+    if admin_only and jwt_payload["role"] != "admin":
+        raise HTTPException(status_code=403, detail="Admin access required")
+
+    return User.from_payload(jwt_payload)

autogpt_platform/autogpt_libs/autogpt_libs/auth/jwt_utils_test.py

@@ -0,0 +1,308 @@
+"""
+Comprehensive tests for JWT token parsing and validation.
+Ensures 100% line and branch coverage for JWT security functions.
+"""
+
+import os
+from datetime import datetime, timedelta, timezone
+
+import jwt
+import pytest
+from fastapi import HTTPException
+from fastapi.security import HTTPAuthorizationCredentials
+from pytest_mock import MockerFixture
+
+from autogpt_libs.auth import config, jwt_utils
+from autogpt_libs.auth.config import Settings
+from autogpt_libs.auth.models import User
+
+MOCK_JWT_SECRET = "test-secret-key-with-at-least-32-characters"
+TEST_USER_PAYLOAD = {
+    "sub": "test-user-id",
+    "role": "user",
+    "aud": "authenticated",
+    "email": "test@example.com",
+}
+TEST_ADMIN_PAYLOAD = {
+    "sub": "admin-user-id",
+    "role": "admin",
+    "aud": "authenticated",
+    "email": "admin@example.com",
+}
+
+
+@pytest.fixture(autouse=True)
+def mock_config(mocker: MockerFixture):
+    mocker.patch.dict(os.environ, {"JWT_VERIFY_KEY": MOCK_JWT_SECRET}, clear=True)
+    mocker.patch.object(config, "_settings", Settings())
+    yield
+
+
+def create_token(payload, secret=None, algorithm="HS256"):
+    """Helper to create JWT tokens."""
+    if secret is None:
+        secret = MOCK_JWT_SECRET
+    return jwt.encode(payload, secret, algorithm=algorithm)
+
+
+def test_parse_jwt_token_valid():
+    """Test parsing a valid JWT token."""
+    token = create_token(TEST_USER_PAYLOAD)
+    result = jwt_utils.parse_jwt_token(token)
+
+    assert result["sub"] == "test-user-id"
+    assert result["role"] == "user"
+    assert result["aud"] == "authenticated"
+
+
+def test_parse_jwt_token_expired():
+    """Test parsing an expired JWT token."""
+    expired_payload = {
+        **TEST_USER_PAYLOAD,
+        "exp": datetime.now(timezone.utc) - timedelta(hours=1),
+    }
+    token = create_token(expired_payload)
+
+    with pytest.raises(ValueError) as exc_info:
+        jwt_utils.parse_jwt_token(token)
+    assert "Token has expired" in str(exc_info.value)
+
+
+def test_parse_jwt_token_invalid_signature():
+    """Test parsing a token with invalid signature."""
+    # Create token with different secret
+    token = create_token(TEST_USER_PAYLOAD, secret="wrong-secret")
+
+    with pytest.raises(ValueError) as exc_info:
+        jwt_utils.parse_jwt_token(token)
+    assert "Invalid token" in str(exc_info.value)
+
+
+def test_parse_jwt_token_malformed():
+    """Test parsing a malformed token."""
+    malformed_tokens = [
+        "not.a.token",
+        "invalid",
+        "",
+        # Header only
+        "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9",
+        # No signature
+        "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0In0",
+    ]
+
+    for token in malformed_tokens:
+        with pytest.raises(ValueError) as exc_info:
+            jwt_utils.parse_jwt_token(token)
+        assert "Invalid token" in str(exc_info.value)
+
+
+def test_parse_jwt_token_wrong_audience():
+    """Test parsing a token with wrong audience."""
+    wrong_aud_payload = {**TEST_USER_PAYLOAD, "aud": "wrong-audience"}
+    token = create_token(wrong_aud_payload)
+
+    with pytest.raises(ValueError) as exc_info:
+        jwt_utils.parse_jwt_token(token)
+    assert "Invalid token" in str(exc_info.value)
+
+
+def test_parse_jwt_token_missing_audience():
+    """Test parsing a token without audience claim."""
+    no_aud_payload = {k: v for k, v in TEST_USER_PAYLOAD.items() if k != "aud"}
+    token = create_token(no_aud_payload)
+
+    with pytest.raises(ValueError) as exc_info:
+        jwt_utils.parse_jwt_token(token)
+    assert "Invalid token" in str(exc_info.value)
+
+
+async def test_get_jwt_payload_with_valid_token():
+    """Test extracting JWT payload with valid bearer token."""
+    token = create_token(TEST_USER_PAYLOAD)
+    credentials = HTTPAuthorizationCredentials(scheme="Bearer", credentials=token)
+
+    result = await jwt_utils.get_jwt_payload(credentials)
+    assert result["sub"] == "test-user-id"
+    assert result["role"] == "user"
+
+
+async def test_get_jwt_payload_no_credentials():
+    """Test JWT payload when no credentials provided."""
+    with pytest.raises(HTTPException) as exc_info:
+        await jwt_utils.get_jwt_payload(None)
+    assert exc_info.value.status_code == 401
+    assert "Authorization header is missing" in exc_info.value.detail
+
+
+async def test_get_jwt_payload_invalid_token():
+    """Test JWT payload extraction with invalid token."""
+    credentials = HTTPAuthorizationCredentials(
+        scheme="Bearer", credentials="invalid.token.here"
+    )
+
+    with pytest.raises(HTTPException) as exc_info:
+        await jwt_utils.get_jwt_payload(credentials)
+    assert exc_info.value.status_code == 401
+    assert "Invalid token" in exc_info.value.detail
+
+
+def test_verify_user_with_valid_user():
+    """Test verifying a valid user."""
+    user = jwt_utils.verify_user(TEST_USER_PAYLOAD, admin_only=False)
+    assert isinstance(user, User)
+    assert user.user_id == "test-user-id"
+    assert user.role == "user"
+    assert user.email == "test@example.com"
+
+
+def test_verify_user_with_admin():
+    """Test verifying an admin user."""
+    user = jwt_utils.verify_user(TEST_ADMIN_PAYLOAD, admin_only=True)
+    assert isinstance(user, User)
+    assert user.user_id == "admin-user-id"
+    assert user.role == "admin"
+
+
+def test_verify_user_admin_only_with_regular_user():
+    """Test verifying regular user when admin is required."""
+    with pytest.raises(HTTPException) as exc_info:
+        jwt_utils.verify_user(TEST_USER_PAYLOAD, admin_only=True)
+    assert exc_info.value.status_code == 403
+    assert "Admin access required" in exc_info.value.detail
+
+
+def test_verify_user_no_payload():
+    """Test verifying user with no payload."""
+    with pytest.raises(HTTPException) as exc_info:
+        jwt_utils.verify_user(None, admin_only=False)
+    assert exc_info.value.status_code == 401
+    assert "Authorization header is missing" in exc_info.value.detail
+
+
+def test_verify_user_missing_sub():
+    """Test verifying user with payload missing 'sub' field."""
+    invalid_payload = {"role": "user", "email": "test@example.com"}
+    with pytest.raises(HTTPException) as exc_info:
+        jwt_utils.verify_user(invalid_payload, admin_only=False)
+    assert exc_info.value.status_code == 401
+    assert "User ID not found in token" in exc_info.value.detail
+
+
+def test_verify_user_empty_sub():
+    """Test verifying user with empty 'sub' field."""
+    invalid_payload = {"sub": "", "role": "user"}
+    with pytest.raises(HTTPException) as exc_info:
+        jwt_utils.verify_user(invalid_payload, admin_only=False)
+    assert exc_info.value.status_code == 401
+    assert "User ID not found in token" in exc_info.value.detail
+
+
+def test_verify_user_none_sub():
+    """Test verifying user with None 'sub' field."""
+    invalid_payload = {"sub": None, "role": "user"}
+    with pytest.raises(HTTPException) as exc_info:
+        jwt_utils.verify_user(invalid_payload, admin_only=False)
+    assert exc_info.value.status_code == 401
+    assert "User ID not found in token" in exc_info.value.detail
+
+
+def test_verify_user_missing_role_admin_check():
+    """Test verifying admin when role field is missing."""
+    no_role_payload = {"sub": "user-id"}
+    with pytest.raises(KeyError):
+        # This will raise KeyError when checking payload["role"]
+        jwt_utils.verify_user(no_role_payload, admin_only=True)
+
+
+# ======================== EDGE CASES ======================== #
+
+
+def test_jwt_with_additional_claims():
+    """Test JWT token with additional custom claims."""
+    extra_claims_payload = {
+        "sub": "user-id",
+        "role": "user",
+        "aud": "authenticated",
+        "custom_claim": "custom_value",
+        "permissions": ["read", "write"],
+        "metadata": {"key": "value"},
+    }
+    token = create_token(extra_claims_payload)
+
+    result = jwt_utils.parse_jwt_token(token)
+    assert result["sub"] == "user-id"
+    assert result["custom_claim"] == "custom_value"
+    assert result["permissions"] == ["read", "write"]
+
+
+def test_jwt_with_numeric_sub():
+    """Test JWT token with numeric user ID."""
+    payload = {
+        "sub": 12345,  # Numeric ID
+        "role": "user",
+        "aud": "authenticated",
+    }
+    # Should convert to string internally
+    user = jwt_utils.verify_user(payload, admin_only=False)
+    assert user.user_id == 12345
+
+
+def test_jwt_with_very_long_sub():
+    """Test JWT token with very long user ID."""
+    long_id = "a" * 1000
+    payload = {
+        "sub": long_id,
+        "role": "user",
+        "aud": "authenticated",
+    }
+    user = jwt_utils.verify_user(payload, admin_only=False)
+    assert user.user_id == long_id
+
+
+def test_jwt_with_special_characters_in_claims():
+    """Test JWT token with special characters in claims."""
+    payload = {
+        "sub": "user@example.com/special-chars!@#$%",
+        "role": "admin",
+        "aud": "authenticated",
+        "email": "test+special@example.com",
+    }
+    user = jwt_utils.verify_user(payload, admin_only=True)
+    assert "special-chars!@#$%" in user.user_id
+
+
+def test_jwt_with_future_iat():
+    """Test JWT token with issued-at time in future."""
+    future_payload = {
+        "sub": "user-id",
+        "role": "user",
+        "aud": "authenticated",
+        "iat": datetime.now(timezone.utc) + timedelta(hours=1),
+    }
+    token = create_token(future_payload)
+
+    # PyJWT validates iat claim and should reject future tokens
+    with pytest.raises(ValueError, match="not yet valid"):
+        jwt_utils.parse_jwt_token(token)
+
+
+def test_jwt_with_different_algorithms():
+    """Test that only HS256 algorithm is accepted."""
+    payload = {
+        "sub": "user-id",
+        "role": "user",
+        "aud": "authenticated",
+    }
+
+    # Try different algorithms
+    algorithms = ["HS384", "HS512", "none"]
+    for algo in algorithms:
+        if algo == "none":
+            # Special case for 'none' algorithm (security vulnerability if accepted)
+            token = create_token(payload, "", algorithm="none")
+        else:
+            token = create_token(payload, algorithm=algo)
+
+        with pytest.raises(ValueError) as exc_info:
+            jwt_utils.parse_jwt_token(token)
+        assert "Invalid token" in str(exc_info.value)

autogpt_platform/autogpt_libs/autogpt_libs/auth/middleware.py

@@ -1,139 +0,0 @@
-import inspect
-import logging
-import secrets
-from typing import Any, Callable, Optional
-
-from fastapi import HTTPException, Request, Security
-from fastapi.security import APIKeyHeader, HTTPBearer
-from starlette.status import HTTP_401_UNAUTHORIZED
-
-from .config import settings
-from .jwt_utils import parse_jwt_token
-
-logger = logging.getLogger(__name__)
-bearer_auth = HTTPBearer(auto_error=False)
-
-
-async def auth_middleware(request: Request):
-    if not settings.ENABLE_AUTH:
-        # If authentication is disabled, allow the request to proceed
-        logger.warning("Auth disabled")
-        return {}
-
-    credentials = await bearer_auth(request)
-
-    if not credentials:
-        raise HTTPException(status_code=401, detail="Not authenticated")
-
-    try:
-        payload = parse_jwt_token(credentials.credentials)
-        request.state.user = payload
-        logger.debug("Token decoded successfully")
-    except ValueError as e:
-        raise HTTPException(status_code=401, detail=str(e))
-    return payload
-
-
-class APIKeyValidator:
-    """
-    Configurable API key validator that supports custom validation functions
-    for FastAPI applications.
-
-    This class provides a flexible way to implement API key authentication with optional
-    custom validation logic. It can be used for simple token matching
-    or more complex validation scenarios like database lookups.
-
-    Examples:
-        Simple token validation:
-        ```python
-        validator = APIKeyValidator(
-            header_name="X-API-Key",
-            expected_token="your-secret-token"
-        )
-
-        @app.get("/protected", dependencies=[Depends(validator.get_dependency())])
-        def protected_endpoint():
-            return {"message": "Access granted"}
-        ```
-
-        Custom validation with database lookup:
-        ```python
-        async def validate_with_db(api_key: str):
-            api_key_obj = await db.get_api_key(api_key)
-            return api_key_obj if api_key_obj and api_key_obj.is_active else None
-
-        validator = APIKeyValidator(
-            header_name="X-API-Key",
-            validate_fn=validate_with_db
-        )
-        ```
-
-    Args:
-        header_name (str): The name of the header containing the API key
-        expected_token (Optional[str]): The expected API key value for simple token matching
-        validate_fn (Optional[Callable]): Custom validation function that takes an API key
-            string and returns a boolean or object. Can be async.
-        error_status (int): HTTP status code to use for validation errors
-        error_message (str): Error message to return when validation fails
-    """
-
-    def __init__(
-        self,
-        header_name: str,
-        expected_token: Optional[str] = None,
-        validate_fn: Optional[Callable[[str], bool]] = None,
-        error_status: int = HTTP_401_UNAUTHORIZED,
-        error_message: str = "Invalid API key",
-    ):
-        # Create the APIKeyHeader as a class property
-        self.security_scheme = APIKeyHeader(name=header_name)
-        self.expected_token = expected_token
-        self.custom_validate_fn = validate_fn
-        self.error_status = error_status
-        self.error_message = error_message
-
-    async def default_validator(self, api_key: str) -> bool:
-        if not self.expected_token:
-            raise ValueError(
-                "Expected Token Required to be set when uisng API Key Validator default validation"
-            )
-        return secrets.compare_digest(api_key, self.expected_token)
-
-    async def __call__(
-        self, request: Request, api_key: str = Security(APIKeyHeader)
-    ) -> Any:
-        if api_key is None:
-            raise HTTPException(status_code=self.error_status, detail="Missing API key")
-
-        # Use custom validation if provided, otherwise use default equality check
-        validator = self.custom_validate_fn or self.default_validator
-        result = (
-            await validator(api_key)
-            if inspect.iscoroutinefunction(validator)
-            else validator(api_key)
-        )
-
-        if not result:
-            raise HTTPException(
-                status_code=self.error_status, detail=self.error_message
-            )
-
-        # Store validation result in request state if it's not just a boolean
-        if result is not True:
-            request.state.api_key = result
-
-        return result
-
-    def get_dependency(self):
-        """
-        Returns a callable dependency that FastAPI will recognize as a security scheme
-        """
-
-        async def validate_api_key(
-            request: Request, api_key: str = Security(self.security_scheme)
-        ) -> Any:
-            return await self(request, api_key)
-
-        # This helps FastAPI recognize it as a security dependency
-        validate_api_key.__name__ = f"validate_{self.security_scheme.model.name}"
-        return validate_api_key

autogpt_platform/autogpt_libs/autogpt_libs/logging/config.py

@@ -4,6 +4,7 @@ import logging
 import os
 import socket
 import sys
+from logging.handlers import RotatingFileHandler
 from pathlib import Path
 
 from pydantic import Field, field_validator
@@ -93,42 +94,36 @@ def configure_logging(force_cloud_logging: bool = False) -> None:
     config = LoggingConfig()
     log_handlers: list[logging.Handler] = []
 
+    structured_logging = config.enable_cloud_logging or force_cloud_logging
+
     # Console output handlers
-    stdout = logging.StreamHandler(stream=sys.stdout)
-    stdout.setLevel(config.level)
-    stdout.addFilter(BelowLevelFilter(logging.WARNING))
-    if config.level == logging.DEBUG:
-        stdout.setFormatter(AGPTFormatter(DEBUG_LOG_FORMAT))
-    else:
-        stdout.setFormatter(AGPTFormatter(SIMPLE_LOG_FORMAT))
+    if not structured_logging:
+        stdout = logging.StreamHandler(stream=sys.stdout)
+        stdout.setLevel(config.level)
+        stdout.addFilter(BelowLevelFilter(logging.WARNING))
+        if config.level == logging.DEBUG:
+            stdout.setFormatter(AGPTFormatter(DEBUG_LOG_FORMAT))
+        else:
+            stdout.setFormatter(AGPTFormatter(SIMPLE_LOG_FORMAT))
 
-    stderr = logging.StreamHandler()
-    stderr.setLevel(logging.WARNING)
-    if config.level == logging.DEBUG:
-        stderr.setFormatter(AGPTFormatter(DEBUG_LOG_FORMAT))
-    else:
-        stderr.setFormatter(AGPTFormatter(SIMPLE_LOG_FORMAT))
+        stderr = logging.StreamHandler()
+        stderr.setLevel(logging.WARNING)
+        if config.level == logging.DEBUG:
+            stderr.setFormatter(AGPTFormatter(DEBUG_LOG_FORMAT))
+        else:
+            stderr.setFormatter(AGPTFormatter(SIMPLE_LOG_FORMAT))
 
-    log_handlers += [stdout, stderr]
+        log_handlers += [stdout, stderr]
 
     # Cloud logging setup
-    if config.enable_cloud_logging or force_cloud_logging:
-        import google.cloud.logging
-        from google.cloud.logging.handlers import CloudLoggingHandler
-        from google.cloud.logging_v2.handlers.transports import (
-            BackgroundThreadTransport,
-        )
+    else:
+        # Use Google Cloud Structured Log Handler. Log entries are printed to stdout
+        # in a JSON format which is automatically picked up by Google Cloud Logging.
+        from google.cloud.logging.handlers import StructuredLogHandler
 
-        client = google.cloud.logging.Client()
-        # Use BackgroundThreadTransport to prevent blocking the main thread
-        # and deadlocks when gRPC calls to Google Cloud Logging hang
-        cloud_handler = CloudLoggingHandler(
-            client,
-            name="autogpt_logs",
-            transport=BackgroundThreadTransport,
-        )
-        cloud_handler.setLevel(config.level)
-        log_handlers.append(cloud_handler)
+        structured_log_handler = StructuredLogHandler(stream=sys.stdout)
+        structured_log_handler.setLevel(config.level)
+        log_handlers.append(structured_log_handler)
 
     # File logging setup
     if config.enable_file_logging:
@@ -139,8 +134,13 @@ def configure_logging(force_cloud_logging: bool = False) -> None:
         print(f"Log directory: {config.log_dir}")
 
         # Activity log handler (INFO and above)
-        activity_log_handler = logging.FileHandler(
-            config.log_dir / LOG_FILE, "a", "utf-8"
+        # Security fix: Use RotatingFileHandler with size limits to prevent disk exhaustion
+        activity_log_handler = RotatingFileHandler(
+            config.log_dir / LOG_FILE,
+            mode="a",
+            encoding="utf-8",
+            maxBytes=10 * 1024 * 1024,  # 10MB per file
+            backupCount=3,  # Keep 3 backup files (40MB total)
         )
         activity_log_handler.setLevel(config.level)
         activity_log_handler.setFormatter(
@@ -150,8 +150,13 @@ def configure_logging(force_cloud_logging: bool = False) -> None:
 
         if config.level == logging.DEBUG:
             # Debug log handler (all levels)
-            debug_log_handler = logging.FileHandler(
-                config.log_dir / DEBUG_LOG_FILE, "a", "utf-8"
+            # Security fix: Use RotatingFileHandler with size limits
+            debug_log_handler = RotatingFileHandler(
+                config.log_dir / DEBUG_LOG_FILE,
+                mode="a",
+                encoding="utf-8",
+                maxBytes=10 * 1024 * 1024,  # 10MB per file
+                backupCount=3,  # Keep 3 backup files (40MB total)
             )
             debug_log_handler.setLevel(logging.DEBUG)
             debug_log_handler.setFormatter(
@@ -160,8 +165,13 @@ def configure_logging(force_cloud_logging: bool = False) -> None:
             log_handlers.append(debug_log_handler)
 
         # Error log handler (ERROR and above)
-        error_log_handler = logging.FileHandler(
-            config.log_dir / ERROR_LOG_FILE, "a", "utf-8"
+        # Security fix: Use RotatingFileHandler with size limits
+        error_log_handler = RotatingFileHandler(
+            config.log_dir / ERROR_LOG_FILE,
+            mode="a",
+            encoding="utf-8",
+            maxBytes=10 * 1024 * 1024,  # 10MB per file
+            backupCount=3,  # Keep 3 backup files (40MB total)
         )
         error_log_handler.setLevel(logging.ERROR)
         error_log_handler.setFormatter(AGPTFormatter(DEBUG_LOG_FORMAT, no_color=True))
@@ -169,7 +179,13 @@ def configure_logging(force_cloud_logging: bool = False) -> None:
 
     # Configure the root logger
     logging.basicConfig(
-        format=DEBUG_LOG_FORMAT if config.level == logging.DEBUG else SIMPLE_LOG_FORMAT,
+        format=(
+            "%(levelname)s  %(message)s"
+            if structured_logging
+            else (
+                DEBUG_LOG_FORMAT if config.level == logging.DEBUG else SIMPLE_LOG_FORMAT
+            )
+        ),
         level=config.level,
         handlers=log_handlers,
     )

autogpt_platform/autogpt_libs/autogpt_libs/rate_limit/config.py

@@ -1,3 +1,5 @@
+from typing import Optional
+
 from pydantic import Field
 from pydantic_settings import BaseSettings, SettingsConfigDict
 
@@ -13,8 +15,8 @@ class RateLimitSettings(BaseSettings):
         default="6379", description="Redis port", validation_alias="REDIS_PORT"
     )
 
-    redis_password: str = Field(
-        default="password",
+    redis_password: Optional[str] = Field(
+        default=None,
         description="Redis password",
         validation_alias="REDIS_PASSWORD",
     )

autogpt_platform/autogpt_libs/autogpt_libs/rate_limit/limiter.py

@@ -11,7 +11,7 @@ class RateLimiter:
         self,
         redis_host: str = RATE_LIMIT_SETTINGS.redis_host,
         redis_port: str = RATE_LIMIT_SETTINGS.redis_port,
-        redis_password: str = RATE_LIMIT_SETTINGS.redis_password,
+        redis_password: str | None = RATE_LIMIT_SETTINGS.redis_password,
         requests_per_minute: int = RATE_LIMIT_SETTINGS.requests_per_minute,
     ):
         self.redis = Redis(

autogpt_platform/autogpt_libs/autogpt_libs/utils/cache.py

@@ -1,266 +0,0 @@
-import inspect
-import logging
-import threading
-import time
-from functools import wraps
-from typing import (
-    Awaitable,
-    Callable,
-    ParamSpec,
-    Protocol,
-    Tuple,
-    TypeVar,
-    cast,
-    overload,
-    runtime_checkable,
-)
-
-P = ParamSpec("P")
-R = TypeVar("R")
-
-logger = logging.getLogger(__name__)
-
-
-@overload
-def thread_cached(func: Callable[P, Awaitable[R]]) -> Callable[P, Awaitable[R]]:
-    pass
-
-
-@overload
-def thread_cached(func: Callable[P, R]) -> Callable[P, R]:
-    pass
-
-
-def thread_cached(
-    func: Callable[P, R] | Callable[P, Awaitable[R]],
-) -> Callable[P, R] | Callable[P, Awaitable[R]]:
-    thread_local = threading.local()
-
-    def _clear():
-        if hasattr(thread_local, "cache"):
-            del thread_local.cache
-
-    if inspect.iscoroutinefunction(func):
-
-        async def async_wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
-            cache = getattr(thread_local, "cache", None)
-            if cache is None:
-                cache = thread_local.cache = {}
-            key = (args, tuple(sorted(kwargs.items())))
-            if key not in cache:
-                cache[key] = await cast(Callable[P, Awaitable[R]], func)(
-                    *args, **kwargs
-                )
-            return cache[key]
-
-        setattr(async_wrapper, "clear_cache", _clear)
-        return async_wrapper
-
-    else:
-
-        def sync_wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
-            cache = getattr(thread_local, "cache", None)
-            if cache is None:
-                cache = thread_local.cache = {}
-            key = (args, tuple(sorted(kwargs.items())))
-            if key not in cache:
-                cache[key] = func(*args, **kwargs)
-            return cache[key]
-
-        setattr(sync_wrapper, "clear_cache", _clear)
-        return sync_wrapper
-
-
-def clear_thread_cache(func: Callable) -> None:
-    if clear := getattr(func, "clear_cache", None):
-        clear()
-
-
-FuncT = TypeVar("FuncT")
-
-
-R_co = TypeVar("R_co", covariant=True)
-
-
-@runtime_checkable
-class AsyncCachedFunction(Protocol[P, R_co]):
-    """Protocol for async functions with cache management methods."""
-
-    def cache_clear(self) -> None:
-        """Clear all cached entries."""
-        return None
-
-    def cache_info(self) -> dict[str, int | None]:
-        """Get cache statistics."""
-        return {}
-
-    async def __call__(self, *args: P.args, **kwargs: P.kwargs) -> R_co:
-        """Call the cached function."""
-        return None  # type: ignore
-
-
-def async_ttl_cache(
-    maxsize: int = 128, ttl_seconds: int | None = None
-) -> Callable[[Callable[P, Awaitable[R]]], AsyncCachedFunction[P, R]]:
-    """
-    TTL (Time To Live) cache decorator for async functions.
-
-    Similar to functools.lru_cache but works with async functions and includes optional TTL.
-
-    Args:
-        maxsize: Maximum number of cached entries
-        ttl_seconds: Time to live in seconds. If None, entries never expire (like lru_cache)
-
-    Returns:
-        Decorator function
-
-    Example:
-        # With TTL
-        @async_ttl_cache(maxsize=1000, ttl_seconds=300)
-        async def api_call(param: str) -> dict:
-            return {"result": param}
-
-        # Without TTL (permanent cache like lru_cache)
-        @async_ttl_cache(maxsize=1000)
-        async def expensive_computation(param: str) -> dict:
-            return {"result": param}
-    """
-
-    def decorator(
-        async_func: Callable[P, Awaitable[R]],
-    ) -> AsyncCachedFunction[P, R]:
-        # Cache storage - use union type to handle both cases
-        cache_storage: dict[tuple, R | Tuple[R, float]] = {}
-
-        @wraps(async_func)
-        async def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
-            # Create cache key from arguments
-            key = (args, tuple(sorted(kwargs.items())))
-            current_time = time.time()
-
-            # Check if we have a valid cached entry
-            if key in cache_storage:
-                if ttl_seconds is None:
-                    # No TTL - return cached result directly
-                    logger.debug(
-                        f"Cache hit for {async_func.__name__} with key: {str(key)[:50]}"
-                    )
-                    return cast(R, cache_storage[key])
-                else:
-                    # With TTL - check expiration
-                    cached_data = cache_storage[key]
-                    if isinstance(cached_data, tuple):
-                        result, timestamp = cached_data
-                        if current_time - timestamp < ttl_seconds:
-                            logger.debug(
-                                f"Cache hit for {async_func.__name__} with key: {str(key)[:50]}"
-                            )
-                            return cast(R, result)
-                        else:
-                            # Expired entry
-                            del cache_storage[key]
-                            logger.debug(
-                                f"Cache entry expired for {async_func.__name__}"
-                            )
-
-            # Cache miss or expired - fetch fresh data
-            logger.debug(
-                f"Cache miss for {async_func.__name__} with key: {str(key)[:50]}"
-            )
-            result = await async_func(*args, **kwargs)
-
-            # Store in cache
-            if ttl_seconds is None:
-                cache_storage[key] = result
-            else:
-                cache_storage[key] = (result, current_time)
-
-            # Simple cleanup when cache gets too large
-            if len(cache_storage) > maxsize:
-                # Remove oldest entries (simple FIFO cleanup)
-                cutoff = maxsize // 2
-                oldest_keys = list(cache_storage.keys())[:-cutoff] if cutoff > 0 else []
-                for old_key in oldest_keys:
-                    cache_storage.pop(old_key, None)
-                logger.debug(
-                    f"Cache cleanup: removed {len(oldest_keys)} entries for {async_func.__name__}"
-                )
-
-            return result
-
-        # Add cache management methods (similar to functools.lru_cache)
-        def cache_clear() -> None:
-            cache_storage.clear()
-
-        def cache_info() -> dict[str, int | None]:
-            return {
-                "size": len(cache_storage),
-                "maxsize": maxsize,
-                "ttl_seconds": ttl_seconds,
-            }
-
-        # Attach methods to wrapper
-        setattr(wrapper, "cache_clear", cache_clear)
-        setattr(wrapper, "cache_info", cache_info)
-
-        return cast(AsyncCachedFunction[P, R], wrapper)
-
-    return decorator
-
-
-@overload
-def async_cache(
-    func: Callable[P, Awaitable[R]],
-) -> AsyncCachedFunction[P, R]:
-    pass
-
-
-@overload
-def async_cache(
-    func: None = None,
-    *,
-    maxsize: int = 128,
-) -> Callable[[Callable[P, Awaitable[R]]], AsyncCachedFunction[P, R]]:
-    pass
-
-
-def async_cache(
-    func: Callable[P, Awaitable[R]] | None = None,
-    *,
-    maxsize: int = 128,
-) -> (
-    AsyncCachedFunction[P, R]
-    | Callable[[Callable[P, Awaitable[R]]], AsyncCachedFunction[P, R]]
-):
-    """
-    Process-level cache decorator for async functions (no TTL).
-
-    Similar to functools.lru_cache but works with async functions.
-    This is a convenience wrapper around async_ttl_cache with ttl_seconds=None.
-
-    Args:
-        func: The async function to cache (when used without parentheses)
-        maxsize: Maximum number of cached entries
-
-    Returns:
-        Decorated function or decorator
-
-    Example:
-        # Without parentheses (uses default maxsize=128)
-        @async_cache
-        async def get_data(param: str) -> dict:
-            return {"result": param}
-
-        # With parentheses and custom maxsize
-        @async_cache(maxsize=1000)
-        async def expensive_computation(param: str) -> dict:
-            # Expensive computation here
-            return {"result": param}
-    """
-    if func is None:
-        # Called with parentheses @async_cache() or @async_cache(maxsize=...)
-        return async_ttl_cache(maxsize=maxsize, ttl_seconds=None)
-    else:
-        # Called without parentheses @async_cache
-        decorator = async_ttl_cache(maxsize=maxsize, ttl_seconds=None)
-        return decorator(func)

autogpt_platform/autogpt_libs/autogpt_libs/utils/cache_test.py

@@ -1,705 +0,0 @@
-"""Tests for the @thread_cached decorator.
-
-This module tests the thread-local caching functionality including:
-- Basic caching for sync and async functions
-- Thread isolation (each thread has its own cache)
-- Cache clearing functionality
-- Exception handling (exceptions are not cached)
-- Argument handling (positional vs keyword arguments)
-"""
-
-import asyncio
-import threading
-import time
-from concurrent.futures import ThreadPoolExecutor
-from unittest.mock import Mock
-
-import pytest
-
-from autogpt_libs.utils.cache import (
-    async_cache,
-    async_ttl_cache,
-    clear_thread_cache,
-    thread_cached,
-)
-
-
-class TestThreadCached:
-    def test_sync_function_caching(self):
-        call_count = 0
-
-        @thread_cached
-        def expensive_function(x: int, y: int = 0) -> int:
-            nonlocal call_count
-            call_count += 1
-            return x + y
-
-        assert expensive_function(1, 2) == 3
-        assert call_count == 1
-
-        assert expensive_function(1, 2) == 3
-        assert call_count == 1
-
-        assert expensive_function(1, y=2) == 3
-        assert call_count == 2
-
-        assert expensive_function(2, 3) == 5
-        assert call_count == 3
-
-        assert expensive_function(1) == 1
-        assert call_count == 4
-
-    @pytest.mark.asyncio
-    async def test_async_function_caching(self):
-        call_count = 0
-
-        @thread_cached
-        async def expensive_async_function(x: int, y: int = 0) -> int:
-            nonlocal call_count
-            call_count += 1
-            await asyncio.sleep(0.01)
-            return x + y
-
-        assert await expensive_async_function(1, 2) == 3
-        assert call_count == 1
-
-        assert await expensive_async_function(1, 2) == 3
-        assert call_count == 1
-
-        assert await expensive_async_function(1, y=2) == 3
-        assert call_count == 2
-
-        assert await expensive_async_function(2, 3) == 5
-        assert call_count == 3
-
-    def test_thread_isolation(self):
-        call_count = 0
-        results = {}
-
-        @thread_cached
-        def thread_specific_function(x: int) -> str:
-            nonlocal call_count
-            call_count += 1
-            return f"{threading.current_thread().name}-{x}"
-
-        def worker(thread_id: int):
-            result1 = thread_specific_function(1)
-            result2 = thread_specific_function(1)
-            result3 = thread_specific_function(2)
-            results[thread_id] = (result1, result2, result3)
-
-        with ThreadPoolExecutor(max_workers=3) as executor:
-            futures = [executor.submit(worker, i) for i in range(3)]
-            for future in futures:
-                future.result()
-
-        assert call_count >= 2
-
-        for thread_id, (r1, r2, r3) in results.items():
-            assert r1 == r2
-            assert r1 != r3
-
-    @pytest.mark.asyncio
-    async def test_async_thread_isolation(self):
-        call_count = 0
-        results = {}
-
-        @thread_cached
-        async def async_thread_specific_function(x: int) -> str:
-            nonlocal call_count
-            call_count += 1
-            await asyncio.sleep(0.01)
-            return f"{threading.current_thread().name}-{x}"
-
-        async def async_worker(worker_id: int):
-            result1 = await async_thread_specific_function(1)
-            result2 = await async_thread_specific_function(1)
-            result3 = await async_thread_specific_function(2)
-            results[worker_id] = (result1, result2, result3)
-
-        tasks = [async_worker(i) for i in range(3)]
-        await asyncio.gather(*tasks)
-
-        for worker_id, (r1, r2, r3) in results.items():
-            assert r1 == r2
-            assert r1 != r3
-
-    def test_clear_cache_sync(self):
-        call_count = 0
-
-        @thread_cached
-        def clearable_function(x: int) -> int:
-            nonlocal call_count
-            call_count += 1
-            return x * 2
-
-        assert clearable_function(5) == 10
-        assert call_count == 1
-
-        assert clearable_function(5) == 10
-        assert call_count == 1
-
-        clear_thread_cache(clearable_function)
-
-        assert clearable_function(5) == 10
-        assert call_count == 2
-
-    @pytest.mark.asyncio
-    async def test_clear_cache_async(self):
-        call_count = 0
-
-        @thread_cached
-        async def clearable_async_function(x: int) -> int:
-            nonlocal call_count
-            call_count += 1
-            await asyncio.sleep(0.01)
-            return x * 2
-
-        assert await clearable_async_function(5) == 10
-        assert call_count == 1
-
-        assert await clearable_async_function(5) == 10
-        assert call_count == 1
-
-        clear_thread_cache(clearable_async_function)
-
-        assert await clearable_async_function(5) == 10
-        assert call_count == 2
-
-    def test_simple_arguments(self):
-        call_count = 0
-
-        @thread_cached
-        def simple_function(a: str, b: int, c: str = "default") -> str:
-            nonlocal call_count
-            call_count += 1
-            return f"{a}-{b}-{c}"
-
-        # First call with all positional args
-        result1 = simple_function("test", 42, "custom")
-        assert call_count == 1
-
-        # Same args, all positional - should hit cache
-        result2 = simple_function("test", 42, "custom")
-        assert call_count == 1
-        assert result1 == result2
-
-        # Same values but last arg as keyword - creates different cache key
-        result3 = simple_function("test", 42, c="custom")
-        assert call_count == 2
-        assert result1 == result3  # Same result, different cache entry
-
-        # Different value - new cache entry
-        result4 = simple_function("test", 43, "custom")
-        assert call_count == 3
-        assert result1 != result4
-
-    def test_positional_vs_keyword_args(self):
-        """Test that positional and keyword arguments create different cache entries."""
-        call_count = 0
-
-        @thread_cached
-        def func(a: int, b: int = 10) -> str:
-            nonlocal call_count
-            call_count += 1
-            return f"result-{a}-{b}"
-
-        # All positional
-        result1 = func(1, 2)
-        assert call_count == 1
-        assert result1 == "result-1-2"
-
-        # Same values, but second arg as keyword
-        result2 = func(1, b=2)
-        assert call_count == 2  # Different cache key!
-        assert result2 == "result-1-2"  # Same result
-
-        # Verify both are cached separately
-        func(1, 2)  # Uses first cache entry
-        assert call_count == 2
-
-        func(1, b=2)  # Uses second cache entry
-        assert call_count == 2
-
-    def test_exception_handling(self):
-        call_count = 0
-
-        @thread_cached
-        def failing_function(x: int) -> int:
-            nonlocal call_count
-            call_count += 1
-            if x < 0:
-                raise ValueError("Negative value")
-            return x * 2
-
-        assert failing_function(5) == 10
-        assert call_count == 1
-
-        with pytest.raises(ValueError):
-            failing_function(-1)
-        assert call_count == 2
-
-        with pytest.raises(ValueError):
-            failing_function(-1)
-        assert call_count == 3
-
-        assert failing_function(5) == 10
-        assert call_count == 3
-
-    @pytest.mark.asyncio
-    async def test_async_exception_handling(self):
-        call_count = 0
-
-        @thread_cached
-        async def async_failing_function(x: int) -> int:
-            nonlocal call_count
-            call_count += 1
-            await asyncio.sleep(0.01)
-            if x < 0:
-                raise ValueError("Negative value")
-            return x * 2
-
-        assert await async_failing_function(5) == 10
-        assert call_count == 1
-
-        with pytest.raises(ValueError):
-            await async_failing_function(-1)
-        assert call_count == 2
-
-        with pytest.raises(ValueError):
-            await async_failing_function(-1)
-        assert call_count == 3
-
-    def test_sync_caching_performance(self):
-        @thread_cached
-        def slow_function(x: int) -> int:
-            print(f"slow_function called with x={x}")
-            time.sleep(0.1)
-            return x * 2
-
-        start = time.time()
-        result1 = slow_function(5)
-        first_call_time = time.time() - start
-        print(f"First call took {first_call_time:.4f} seconds")
-
-        start = time.time()
-        result2 = slow_function(5)
-        second_call_time = time.time() - start
-        print(f"Second call took {second_call_time:.4f} seconds")
-
-        assert result1 == result2 == 10
-        assert first_call_time > 0.09
-        assert second_call_time < 0.01
-
-    @pytest.mark.asyncio
-    async def test_async_caching_performance(self):
-        @thread_cached
-        async def slow_async_function(x: int) -> int:
-            print(f"slow_async_function called with x={x}")
-            await asyncio.sleep(0.1)
-            return x * 2
-
-        start = time.time()
-        result1 = await slow_async_function(5)
-        first_call_time = time.time() - start
-        print(f"First async call took {first_call_time:.4f} seconds")
-
-        start = time.time()
-        result2 = await slow_async_function(5)
-        second_call_time = time.time() - start
-        print(f"Second async call took {second_call_time:.4f} seconds")
-
-        assert result1 == result2 == 10
-        assert first_call_time > 0.09
-        assert second_call_time < 0.01
-
-    def test_with_mock_objects(self):
-        mock = Mock(return_value=42)
-
-        @thread_cached
-        def function_using_mock(x: int) -> int:
-            return mock(x)
-
-        assert function_using_mock(1) == 42
-        assert mock.call_count == 1
-
-        assert function_using_mock(1) == 42
-        assert mock.call_count == 1
-
-        assert function_using_mock(2) == 42
-        assert mock.call_count == 2
-
-
-class TestAsyncTTLCache:
-    """Tests for the @async_ttl_cache decorator."""
-
-    @pytest.mark.asyncio
-    async def test_basic_caching(self):
-        """Test basic caching functionality."""
-        call_count = 0
-
-        @async_ttl_cache(maxsize=10, ttl_seconds=60)
-        async def cached_function(x: int, y: int = 0) -> int:
-            nonlocal call_count
-            call_count += 1
-            await asyncio.sleep(0.01)  # Simulate async work
-            return x + y
-
-        # First call
-        result1 = await cached_function(1, 2)
-        assert result1 == 3
-        assert call_count == 1
-
-        # Second call with same args - should use cache
-        result2 = await cached_function(1, 2)
-        assert result2 == 3
-        assert call_count == 1  # No additional call
-
-        # Different args - should call function again
-        result3 = await cached_function(2, 3)
-        assert result3 == 5
-        assert call_count == 2
-
-    @pytest.mark.asyncio
-    async def test_ttl_expiration(self):
-        """Test that cache entries expire after TTL."""
-        call_count = 0
-
-        @async_ttl_cache(maxsize=10, ttl_seconds=1)  # Short TTL
-        async def short_lived_cache(x: int) -> int:
-            nonlocal call_count
-            call_count += 1
-            return x * 2
-
-        # First call
-        result1 = await short_lived_cache(5)
-        assert result1 == 10
-        assert call_count == 1
-
-        # Second call immediately - should use cache
-        result2 = await short_lived_cache(5)
-        assert result2 == 10
-        assert call_count == 1
-
-        # Wait for TTL to expire
-        await asyncio.sleep(1.1)
-
-        # Third call after expiration - should call function again
-        result3 = await short_lived_cache(5)
-        assert result3 == 10
-        assert call_count == 2
-
-    @pytest.mark.asyncio
-    async def test_cache_info(self):
-        """Test cache info functionality."""
-
-        @async_ttl_cache(maxsize=5, ttl_seconds=300)
-        async def info_test_function(x: int) -> int:
-            return x * 3
-
-        # Check initial cache info
-        info = info_test_function.cache_info()
-        assert info["size"] == 0
-        assert info["maxsize"] == 5
-        assert info["ttl_seconds"] == 300
-
-        # Add an entry
-        await info_test_function(1)
-        info = info_test_function.cache_info()
-        assert info["size"] == 1
-
-    @pytest.mark.asyncio
-    async def test_cache_clear(self):
-        """Test cache clearing functionality."""
-        call_count = 0
-
-        @async_ttl_cache(maxsize=10, ttl_seconds=60)
-        async def clearable_function(x: int) -> int:
-            nonlocal call_count
-            call_count += 1
-            return x * 4
-
-        # First call
-        result1 = await clearable_function(2)
-        assert result1 == 8
-        assert call_count == 1
-
-        # Second call - should use cache
-        result2 = await clearable_function(2)
-        assert result2 == 8
-        assert call_count == 1
-
-        # Clear cache
-        clearable_function.cache_clear()
-
-        # Third call after clear - should call function again
-        result3 = await clearable_function(2)
-        assert result3 == 8
-        assert call_count == 2
-
-    @pytest.mark.asyncio
-    async def test_maxsize_cleanup(self):
-        """Test that cache cleans up when maxsize is exceeded."""
-        call_count = 0
-
-        @async_ttl_cache(maxsize=3, ttl_seconds=60)
-        async def size_limited_function(x: int) -> int:
-            nonlocal call_count
-            call_count += 1
-            return x**2
-
-        # Fill cache to maxsize
-        await size_limited_function(1)  # call_count: 1
-        await size_limited_function(2)  # call_count: 2
-        await size_limited_function(3)  # call_count: 3
-
-        info = size_limited_function.cache_info()
-        assert info["size"] == 3
-
-        # Add one more entry - should trigger cleanup
-        await size_limited_function(4)  # call_count: 4
-
-        # Cache size should be reduced (cleanup removes oldest entries)
-        info = size_limited_function.cache_info()
-        assert info["size"] is not None and info["size"] <= 3  # Should be cleaned up
-
-    @pytest.mark.asyncio
-    async def test_argument_variations(self):
-        """Test caching with different argument patterns."""
-        call_count = 0
-
-        @async_ttl_cache(maxsize=10, ttl_seconds=60)
-        async def arg_test_function(a: int, b: str = "default", *, c: int = 100) -> str:
-            nonlocal call_count
-            call_count += 1
-            return f"{a}-{b}-{c}"
-
-        # Different ways to call with same logical arguments
-        result1 = await arg_test_function(1, "test", c=200)
-        assert call_count == 1
-
-        # Same arguments, same order - should use cache
-        result2 = await arg_test_function(1, "test", c=200)
-        assert call_count == 1
-        assert result1 == result2
-
-        # Different arguments - should call function
-        result3 = await arg_test_function(2, "test", c=200)
-        assert call_count == 2
-        assert result1 != result3
-
-    @pytest.mark.asyncio
-    async def test_exception_handling(self):
-        """Test that exceptions are not cached."""
-        call_count = 0
-
-        @async_ttl_cache(maxsize=10, ttl_seconds=60)
-        async def exception_function(x: int) -> int:
-            nonlocal call_count
-            call_count += 1
-            if x < 0:
-                raise ValueError("Negative value not allowed")
-            return x * 2
-
-        # Successful call - should be cached
-        result1 = await exception_function(5)
-        assert result1 == 10
-        assert call_count == 1
-
-        # Same successful call - should use cache
-        result2 = await exception_function(5)
-        assert result2 == 10
-        assert call_count == 1
-
-        # Exception call - should not be cached
-        with pytest.raises(ValueError):
-            await exception_function(-1)
-        assert call_count == 2
-
-        # Same exception call - should call again (not cached)
-        with pytest.raises(ValueError):
-            await exception_function(-1)
-        assert call_count == 3
-
-    @pytest.mark.asyncio
-    async def test_concurrent_calls(self):
-        """Test caching behavior with concurrent calls."""
-        call_count = 0
-
-        @async_ttl_cache(maxsize=10, ttl_seconds=60)
-        async def concurrent_function(x: int) -> int:
-            nonlocal call_count
-            call_count += 1
-            await asyncio.sleep(0.05)  # Simulate work
-            return x * x
-
-        # Launch concurrent calls with same arguments
-        tasks = [concurrent_function(3) for _ in range(5)]
-        results = await asyncio.gather(*tasks)
-
-        # All results should be the same
-        assert all(result == 9 for result in results)
-
-        # Note: Due to race conditions, call_count might be up to 5 for concurrent calls
-        # This tests that the cache doesn't break under concurrent access
-        assert 1 <= call_count <= 5
-
-
-class TestAsyncCache:
-    """Tests for the @async_cache decorator (no TTL)."""
-
-    @pytest.mark.asyncio
-    async def test_basic_caching_no_ttl(self):
-        """Test basic caching functionality without TTL."""
-        call_count = 0
-
-        @async_cache(maxsize=10)
-        async def cached_function(x: int, y: int = 0) -> int:
-            nonlocal call_count
-            call_count += 1
-            await asyncio.sleep(0.01)  # Simulate async work
-            return x + y
-
-        # First call
-        result1 = await cached_function(1, 2)
-        assert result1 == 3
-        assert call_count == 1
-
-        # Second call with same args - should use cache
-        result2 = await cached_function(1, 2)
-        assert result2 == 3
-        assert call_count == 1  # No additional call
-
-        # Third call after some time - should still use cache (no TTL)
-        await asyncio.sleep(0.05)
-        result3 = await cached_function(1, 2)
-        assert result3 == 3
-        assert call_count == 1  # Still no additional call
-
-        # Different args - should call function again
-        result4 = await cached_function(2, 3)
-        assert result4 == 5
-        assert call_count == 2
-
-    @pytest.mark.asyncio
-    async def test_no_ttl_vs_ttl_behavior(self):
-        """Test the difference between TTL and no-TTL caching."""
-        ttl_call_count = 0
-        no_ttl_call_count = 0
-
-        @async_ttl_cache(maxsize=10, ttl_seconds=1)  # Short TTL
-        async def ttl_function(x: int) -> int:
-            nonlocal ttl_call_count
-            ttl_call_count += 1
-            return x * 2
-
-        @async_cache(maxsize=10)  # No TTL
-        async def no_ttl_function(x: int) -> int:
-            nonlocal no_ttl_call_count
-            no_ttl_call_count += 1
-            return x * 2
-
-        # First calls
-        await ttl_function(5)
-        await no_ttl_function(5)
-        assert ttl_call_count == 1
-        assert no_ttl_call_count == 1
-
-        # Wait for TTL to expire
-        await asyncio.sleep(1.1)
-
-        # Second calls after TTL expiry
-        await ttl_function(5)  # Should call function again (TTL expired)
-        await no_ttl_function(5)  # Should use cache (no TTL)
-        assert ttl_call_count == 2  # TTL function called again
-        assert no_ttl_call_count == 1  # No-TTL function still cached
-
-    @pytest.mark.asyncio
-    async def test_async_cache_info(self):
-        """Test cache info for no-TTL cache."""
-
-        @async_cache(maxsize=5)
-        async def info_test_function(x: int) -> int:
-            return x * 3
-
-        # Check initial cache info
-        info = info_test_function.cache_info()
-        assert info["size"] == 0
-        assert info["maxsize"] == 5
-        assert info["ttl_seconds"] is None  # No TTL
-
-        # Add an entry
-        await info_test_function(1)
-        info = info_test_function.cache_info()
-        assert info["size"] == 1
-
-
-class TestTTLOptional:
-    """Tests for optional TTL functionality."""
-
-    @pytest.mark.asyncio
-    async def test_ttl_none_behavior(self):
-        """Test that ttl_seconds=None works like no TTL."""
-        call_count = 0
-
-        @async_ttl_cache(maxsize=10, ttl_seconds=None)
-        async def no_ttl_via_none(x: int) -> int:
-            nonlocal call_count
-            call_count += 1
-            return x**2
-
-        # First call
-        result1 = await no_ttl_via_none(3)
-        assert result1 == 9
-        assert call_count == 1
-
-        # Wait (would expire if there was TTL)
-        await asyncio.sleep(0.1)
-
-        # Second call - should still use cache
-        result2 = await no_ttl_via_none(3)
-        assert result2 == 9
-        assert call_count == 1  # No additional call
-
-        # Check cache info
-        info = no_ttl_via_none.cache_info()
-        assert info["ttl_seconds"] is None
-
-    @pytest.mark.asyncio
-    async def test_cache_options_comparison(self):
-        """Test different cache options work as expected."""
-        ttl_calls = 0
-        no_ttl_calls = 0
-
-        @async_ttl_cache(maxsize=10, ttl_seconds=1)  # With TTL
-        async def ttl_function(x: int) -> int:
-            nonlocal ttl_calls
-            ttl_calls += 1
-            return x * 10
-
-        @async_cache(maxsize=10)  # Process-level cache (no TTL)
-        async def process_function(x: int) -> int:
-            nonlocal no_ttl_calls
-            no_ttl_calls += 1
-            return x * 10
-
-        # Both should cache initially
-        await ttl_function(3)
-        await process_function(3)
-        assert ttl_calls == 1
-        assert no_ttl_calls == 1
-
-        # Immediate second calls - both should use cache
-        await ttl_function(3)
-        await process_function(3)
-        assert ttl_calls == 1
-        assert no_ttl_calls == 1
-
-        # Wait for TTL to expire
-        await asyncio.sleep(1.1)
-
-        # After TTL expiry
-        await ttl_function(3)  # Should call function again
-        await process_function(3)  # Should still use cache
-        assert ttl_calls == 2  # TTL cache expired, called again
-        assert no_ttl_calls == 1  # Process cache never expires

autogpt_platform/autogpt_libs/poetry.lock

@@ -54,7 +54,7 @@ version = "1.2.0"
 description = "Backport of asyncio.Runner, a context manager that controls event loop life cycle."
 optional = false
 python-versions = "<3.11,>=3.8"
-groups = ["main"]
+groups = ["dev"]
 markers = "python_version < \"3.11\""
 files = [
     {file = "backports_asyncio_runner-1.2.0-py3-none-any.whl", hash = "sha256:0da0a936a8aeb554eccb426dc55af3ba63bcdc69fa1a600b5bb305413a4477b5"},
@@ -85,6 +85,87 @@ files = [
     {file = "certifi-2025.7.14.tar.gz", hash = "sha256:8ea99dbdfaaf2ba2f9bac77b9249ef62ec5218e7c2b2e903378ed5fccf765995"},
 ]
 
+[[package]]
+name = "cffi"
+version = "1.17.1"
+description = "Foreign Function Interface for Python calling C code."
+optional = false
+python-versions = ">=3.8"
+groups = ["main"]
+markers = "platform_python_implementation != \"PyPy\""
+files = [
+    {file = "cffi-1.17.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:df8b1c11f177bc2313ec4b2d46baec87a5f3e71fc8b45dab2ee7cae86d9aba14"},
+    {file = "cffi-1.17.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:8f2cdc858323644ab277e9bb925ad72ae0e67f69e804f4898c070998d50b1a67"},
+    {file = "cffi-1.17.1-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:edae79245293e15384b51f88b00613ba9f7198016a5948b5dddf4917d4d26382"},
+    {file = "cffi-1.17.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:45398b671ac6d70e67da8e4224a065cec6a93541bb7aebe1b198a61b58c7b702"},
+    {file = "cffi-1.17.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ad9413ccdeda48c5afdae7e4fa2192157e991ff761e7ab8fdd8926f40b160cc3"},
+    {file = "cffi-1.17.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5da5719280082ac6bd9aa7becb3938dc9f9cbd57fac7d2871717b1feb0902ab6"},
+    {file = "cffi-1.17.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2bb1a08b8008b281856e5971307cc386a8e9c5b625ac297e853d36da6efe9c17"},
+    {file = "cffi-1.17.1-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8"},
+    {file = "cffi-1.17.1-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:6883e737d7d9e4899a8a695e00ec36bd4e5e4f18fabe0aca0efe0a4b44cdb13e"},
+    {file = "cffi-1.17.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:6b8b4a92e1c65048ff98cfe1f735ef8f1ceb72e3d5f0c25fdb12087a23da22be"},
+    {file = "cffi-1.17.1-cp310-cp310-win32.whl", hash = "sha256:c9c3d058ebabb74db66e431095118094d06abf53284d9c81f27300d0e0d8bc7c"},
+    {file = "cffi-1.17.1-cp310-cp310-win_amd64.whl", hash = "sha256:0f048dcf80db46f0098ccac01132761580d28e28bc0f78ae0d58048063317e15"},
+    {file = "cffi-1.17.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:a45e3c6913c5b87b3ff120dcdc03f6131fa0065027d0ed7ee6190736a74cd401"},
+    {file = "cffi-1.17.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:30c5e0cb5ae493c04c8b42916e52ca38079f1b235c2f8ae5f4527b963c401caf"},
+    {file = "cffi-1.17.1-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f75c7ab1f9e4aca5414ed4d8e5c0e303a34f4421f8a0d47a4d019ceff0ab6af4"},
+    {file = "cffi-1.17.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a1ed2dd2972641495a3ec98445e09766f077aee98a1c896dcb4ad0d303628e41"},
+    {file = "cffi-1.17.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:46bf43160c1a35f7ec506d254e5c890f3c03648a4dbac12d624e4490a7046cd1"},
+    {file = "cffi-1.17.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a24ed04c8ffd54b0729c07cee15a81d964e6fee0e3d4d342a27b020d22959dc6"},
+    {file = "cffi-1.17.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:610faea79c43e44c71e1ec53a554553fa22321b65fae24889706c0a84d4ad86d"},
+    {file = "cffi-1.17.1-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:a9b15d491f3ad5d692e11f6b71f7857e7835eb677955c00cc0aefcd0669adaf6"},
+    {file = "cffi-1.17.1-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:de2ea4b5833625383e464549fec1bc395c1bdeeb5f25c4a3a82b5a8c756ec22f"},
+    {file = "cffi-1.17.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b"},
+    {file = "cffi-1.17.1-cp311-cp311-win32.whl", hash = "sha256:85a950a4ac9c359340d5963966e3e0a94a676bd6245a4b55bc43949eee26a655"},
+    {file = "cffi-1.17.1-cp311-cp311-win_amd64.whl", hash = "sha256:caaf0640ef5f5517f49bc275eca1406b0ffa6aa184892812030f04c2abf589a0"},
+    {file = "cffi-1.17.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:805b4371bf7197c329fcb3ead37e710d1bca9da5d583f5073b799d5c5bd1eee4"},
+    {file = "cffi-1.17.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:733e99bc2df47476e3848417c5a4540522f234dfd4ef3ab7fafdf555b082ec0c"},
+    {file = "cffi-1.17.1-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:1257bdabf294dceb59f5e70c64a3e2f462c30c7ad68092d01bbbfb1c16b1ba36"},
+    {file = "cffi-1.17.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:da95af8214998d77a98cc14e3a3bd00aa191526343078b530ceb0bd710fb48a5"},
+    {file = "cffi-1.17.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d63afe322132c194cf832bfec0dc69a99fb9bb6bbd550f161a49e9e855cc78ff"},
+    {file = "cffi-1.17.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:f79fc4fc25f1c8698ff97788206bb3c2598949bfe0fef03d299eb1b5356ada99"},
+    {file = "cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b62ce867176a75d03a665bad002af8e6d54644fad99a3c70905c543130e39d93"},
+    {file = "cffi-1.17.1-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:386c8bf53c502fff58903061338ce4f4950cbdcb23e2902d86c0f722b786bbe3"},
+    {file = "cffi-1.17.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:4ceb10419a9adf4460ea14cfd6bc43d08701f0835e979bf821052f1805850fe8"},
+    {file = "cffi-1.17.1-cp312-cp312-win32.whl", hash = "sha256:a08d7e755f8ed21095a310a693525137cfe756ce62d066e53f502a83dc550f65"},
+    {file = "cffi-1.17.1-cp312-cp312-win_amd64.whl", hash = "sha256:51392eae71afec0d0c8fb1a53b204dbb3bcabcb3c9b807eedf3e1e6ccf2de903"},
+    {file = "cffi-1.17.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:f3a2b4222ce6b60e2e8b337bb9596923045681d71e5a082783484d845390938e"},
+    {file = "cffi-1.17.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:0984a4925a435b1da406122d4d7968dd861c1385afe3b45ba82b750f229811e2"},
+    {file = "cffi-1.17.1-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d01b12eeeb4427d3110de311e1774046ad344f5b1a7403101878976ecd7a10f3"},
+    {file = "cffi-1.17.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:706510fe141c86a69c8ddc029c7910003a17353970cff3b904ff0686a5927683"},
+    {file = "cffi-1.17.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:de55b766c7aa2e2a3092c51e0483d700341182f08e67c63630d5b6f200bb28e5"},
+    {file = "cffi-1.17.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:c59d6e989d07460165cc5ad3c61f9fd8f1b4796eacbd81cee78957842b834af4"},
+    {file = "cffi-1.17.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:dd398dbc6773384a17fe0d3e7eeb8d1a21c2200473ee6806bb5e6a8e62bb73dd"},
+    {file = "cffi-1.17.1-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:3edc8d958eb099c634dace3c7e16560ae474aa3803a5df240542b305d14e14ed"},
+    {file = "cffi-1.17.1-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:72e72408cad3d5419375fc87d289076ee319835bdfa2caad331e377589aebba9"},
+    {file = "cffi-1.17.1-cp313-cp313-win32.whl", hash = "sha256:e03eab0a8677fa80d646b5ddece1cbeaf556c313dcfac435ba11f107ba117b5d"},
+    {file = "cffi-1.17.1-cp313-cp313-win_amd64.whl", hash = "sha256:f6a16c31041f09ead72d69f583767292f750d24913dadacf5756b966aacb3f1a"},
+    {file = "cffi-1.17.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:636062ea65bd0195bc012fea9321aca499c0504409f413dc88af450b57ffd03b"},
+    {file = "cffi-1.17.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c7eac2ef9b63c79431bc4b25f1cd649d7f061a28808cbc6c47b534bd789ef964"},
+    {file = "cffi-1.17.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e221cf152cff04059d011ee126477f0d9588303eb57e88923578ace7baad17f9"},
+    {file = "cffi-1.17.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:31000ec67d4221a71bd3f67df918b1f88f676f1c3b535a7eb473255fdc0b83fc"},
+    {file = "cffi-1.17.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:6f17be4345073b0a7b8ea599688f692ac3ef23ce28e5df79c04de519dbc4912c"},
+    {file = "cffi-1.17.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0e2b1fac190ae3ebfe37b979cc1ce69c81f4e4fe5746bb401dca63a9062cdaf1"},
+    {file = "cffi-1.17.1-cp38-cp38-win32.whl", hash = "sha256:7596d6620d3fa590f677e9ee430df2958d2d6d6de2feeae5b20e82c00b76fbf8"},
+    {file = "cffi-1.17.1-cp38-cp38-win_amd64.whl", hash = "sha256:78122be759c3f8a014ce010908ae03364d00a1f81ab5c7f4a7a5120607ea56e1"},
+    {file = "cffi-1.17.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:b2ab587605f4ba0bf81dc0cb08a41bd1c0a5906bd59243d56bad7668a6fc6c16"},
+    {file = "cffi-1.17.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:28b16024becceed8c6dfbc75629e27788d8a3f9030691a1dbf9821a128b22c36"},
+    {file = "cffi-1.17.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:1d599671f396c4723d016dbddb72fe8e0397082b0a77a4fab8028923bec050e8"},
+    {file = "cffi-1.17.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ca74b8dbe6e8e8263c0ffd60277de77dcee6c837a3d0881d8c1ead7268c9e576"},
+    {file = "cffi-1.17.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87"},
+    {file = "cffi-1.17.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:98e3969bcff97cae1b2def8ba499ea3d6f31ddfdb7635374834cf89a1a08ecf0"},
+    {file = "cffi-1.17.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:cdf5ce3acdfd1661132f2a9c19cac174758dc2352bfe37d98aa7512c6b7178b3"},
+    {file = "cffi-1.17.1-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:9755e4345d1ec879e3849e62222a18c7174d65a6a92d5b346b1863912168b595"},
+    {file = "cffi-1.17.1-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:f1e22e8c4419538cb197e4dd60acc919d7696e5ef98ee4da4e01d3f8cfa4cc5a"},
+    {file = "cffi-1.17.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:c03e868a0b3bc35839ba98e74211ed2b05d2119be4e8a0f224fba9384f1fe02e"},
+    {file = "cffi-1.17.1-cp39-cp39-win32.whl", hash = "sha256:e31ae45bc2e29f6b2abd0de1cc3b9d5205aa847cafaecb8af1476a609a2f6eb7"},
+    {file = "cffi-1.17.1-cp39-cp39-win_amd64.whl", hash = "sha256:d016c76bdd850f3c626af19b0542c9677ba156e4ee4fccfdd7848803533ef662"},
+    {file = "cffi-1.17.1.tar.gz", hash = "sha256:1c39c6016c32bc48dd54561950ebd6836e1670f2ae46128f67cf49e789c52824"},
+]
+
+[package.dependencies]
+pycparser = "*"
+
 [[package]]
 name = "charset-normalizer"
 version = "3.4.2"
@@ -208,12 +289,176 @@ version = "0.4.6"
 description = "Cross-platform colored terminal text."
 optional = false
 python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7"
-groups = ["main"]
+groups = ["main", "dev"]
 files = [
     {file = "colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6"},
     {file = "colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44"},
 ]
 
+[[package]]
+name = "coverage"
+version = "7.10.5"
+description = "Code coverage measurement for Python"
+optional = false
+python-versions = ">=3.9"
+groups = ["dev"]
+files = [
+    {file = "coverage-7.10.5-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:c6a5c3414bfc7451b879141ce772c546985163cf553f08e0f135f0699a911801"},
+    {file = "coverage-7.10.5-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:bc8e4d99ce82f1710cc3c125adc30fd1487d3cf6c2cd4994d78d68a47b16989a"},
+    {file = "coverage-7.10.5-cp310-cp310-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:02252dc1216e512a9311f596b3169fad54abcb13827a8d76d5630c798a50a754"},
+    {file = "coverage-7.10.5-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:73269df37883e02d460bee0cc16be90509faea1e3bd105d77360b512d5bb9c33"},
+    {file = "coverage-7.10.5-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1f8a81b0614642f91c9effd53eec284f965577591f51f547a1cbeb32035b4c2f"},
+    {file = "coverage-7.10.5-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:6a29f8e0adb7f8c2b95fa2d4566a1d6e6722e0a637634c6563cb1ab844427dd9"},
+    {file = "coverage-7.10.5-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:fcf6ab569436b4a647d4e91accba12509ad9f2554bc93d3aee23cc596e7f99c3"},
+    {file = "coverage-7.10.5-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:90dc3d6fb222b194a5de60af8d190bedeeddcbc7add317e4a3cd333ee6b7c879"},
+    {file = "coverage-7.10.5-cp310-cp310-win32.whl", hash = "sha256:414a568cd545f9dc75f0686a0049393de8098414b58ea071e03395505b73d7a8"},
+    {file = "coverage-7.10.5-cp310-cp310-win_amd64.whl", hash = "sha256:e551f9d03347196271935fd3c0c165f0e8c049220280c1120de0084d65e9c7ff"},
+    {file = "coverage-7.10.5-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:c177e6ffe2ebc7c410785307758ee21258aa8e8092b44d09a2da767834f075f2"},
+    {file = "coverage-7.10.5-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:14d6071c51ad0f703d6440827eaa46386169b5fdced42631d5a5ac419616046f"},
+    {file = "coverage-7.10.5-cp311-cp311-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:61f78c7c3bc272a410c5ae3fde7792b4ffb4acc03d35a7df73ca8978826bb7ab"},
+    {file = "coverage-7.10.5-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:f39071caa126f69d63f99b324fb08c7b1da2ec28cbb1fe7b5b1799926492f65c"},
+    {file = "coverage-7.10.5-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:343a023193f04d46edc46b2616cdbee68c94dd10208ecd3adc56fcc54ef2baa1"},
+    {file = "coverage-7.10.5-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:585ffe93ae5894d1ebdee69fc0b0d4b7c75d8007983692fb300ac98eed146f78"},
+    {file = "coverage-7.10.5-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:b0ef4e66f006ed181df29b59921bd8fc7ed7cd6a9289295cd8b2824b49b570df"},
+    {file = "coverage-7.10.5-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:eb7b0bbf7cc1d0453b843eca7b5fa017874735bef9bfdfa4121373d2cc885ed6"},
+    {file = "coverage-7.10.5-cp311-cp311-win32.whl", hash = "sha256:1d043a8a06987cc0c98516e57c4d3fc2c1591364831e9deb59c9e1b4937e8caf"},
+    {file = "coverage-7.10.5-cp311-cp311-win_amd64.whl", hash = "sha256:fefafcca09c3ac56372ef64a40f5fe17c5592fab906e0fdffd09543f3012ba50"},
+    {file = "coverage-7.10.5-cp311-cp311-win_arm64.whl", hash = "sha256:7e78b767da8b5fc5b2faa69bb001edafcd6f3995b42a331c53ef9572c55ceb82"},
+    {file = "coverage-7.10.5-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:c2d05c7e73c60a4cecc7d9b60dbfd603b4ebc0adafaef371445b47d0f805c8a9"},
+    {file = "coverage-7.10.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:32ddaa3b2c509778ed5373b177eb2bf5662405493baeff52278a0b4f9415188b"},
+    {file = "coverage-7.10.5-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:dd382410039fe062097aa0292ab6335a3f1e7af7bba2ef8d27dcda484918f20c"},
+    {file = "coverage-7.10.5-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:7fa22800f3908df31cea6fb230f20ac49e343515d968cc3a42b30d5c3ebf9b5a"},
+    {file = "coverage-7.10.5-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f366a57ac81f5e12797136552f5b7502fa053c861a009b91b80ed51f2ce651c6"},
+    {file = "coverage-7.10.5-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:5f1dc8f1980a272ad4a6c84cba7981792344dad33bf5869361576b7aef42733a"},
+    {file = "coverage-7.10.5-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:2285c04ee8676f7938b02b4936d9b9b672064daab3187c20f73a55f3d70e6b4a"},
+    {file = "coverage-7.10.5-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:c2492e4dd9daab63f5f56286f8a04c51323d237631eb98505d87e4c4ff19ec34"},
+    {file = "coverage-7.10.5-cp312-cp312-win32.whl", hash = "sha256:38a9109c4ee8135d5df5505384fc2f20287a47ccbe0b3f04c53c9a1989c2bbaf"},
+    {file = "coverage-7.10.5-cp312-cp312-win_amd64.whl", hash = "sha256:6b87f1ad60b30bc3c43c66afa7db6b22a3109902e28c5094957626a0143a001f"},
+    {file = "coverage-7.10.5-cp312-cp312-win_arm64.whl", hash = "sha256:672a6c1da5aea6c629819a0e1461e89d244f78d7b60c424ecf4f1f2556c041d8"},
+    {file = "coverage-7.10.5-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:ef3b83594d933020f54cf65ea1f4405d1f4e41a009c46df629dd964fcb6e907c"},
+    {file = "coverage-7.10.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2b96bfdf7c0ea9faebce088a3ecb2382819da4fbc05c7b80040dbc428df6af44"},
+    {file = "coverage-7.10.5-cp313-cp313-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:63df1fdaffa42d914d5c4d293e838937638bf75c794cf20bee12978fc8c4e3bc"},
+    {file = "coverage-7.10.5-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:8002dc6a049aac0e81ecec97abfb08c01ef0c1fbf962d0c98da3950ace89b869"},
+    {file = "coverage-7.10.5-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:63d4bb2966d6f5f705a6b0c6784c8969c468dbc4bcf9d9ded8bff1c7e092451f"},
+    {file = "coverage-7.10.5-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:1f672efc0731a6846b157389b6e6d5d5e9e59d1d1a23a5c66a99fd58339914d5"},
+    {file = "coverage-7.10.5-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:3f39cef43d08049e8afc1fde4a5da8510fc6be843f8dea350ee46e2a26b2f54c"},
+    {file = "coverage-7.10.5-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:2968647e3ed5a6c019a419264386b013979ff1fb67dd11f5c9886c43d6a31fc2"},
+    {file = "coverage-7.10.5-cp313-cp313-win32.whl", hash = "sha256:0d511dda38595b2b6934c2b730a1fd57a3635c6aa2a04cb74714cdfdd53846f4"},
+    {file = "coverage-7.10.5-cp313-cp313-win_amd64.whl", hash = "sha256:9a86281794a393513cf117177fd39c796b3f8e3759bb2764259a2abba5cce54b"},
+    {file = "coverage-7.10.5-cp313-cp313-win_arm64.whl", hash = "sha256:cebd8e906eb98bb09c10d1feed16096700b1198d482267f8bf0474e63a7b8d84"},
+    {file = "coverage-7.10.5-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:0520dff502da5e09d0d20781df74d8189ab334a1e40d5bafe2efaa4158e2d9e7"},
+    {file = "coverage-7.10.5-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:d9cd64aca68f503ed3f1f18c7c9174cbb797baba02ca8ab5112f9d1c0328cd4b"},
+    {file = "coverage-7.10.5-cp313-cp313t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:0913dd1613a33b13c4f84aa6e3f4198c1a21ee28ccb4f674985c1f22109f0aae"},
+    {file = "coverage-7.10.5-cp313-cp313t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:1b7181c0feeb06ed8a02da02792f42f829a7b29990fef52eff257fef0885d760"},
+    {file = "coverage-7.10.5-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:36d42b7396b605f774d4372dd9c49bed71cbabce4ae1ccd074d155709dd8f235"},
+    {file = "coverage-7.10.5-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:b4fdc777e05c4940b297bf47bf7eedd56a39a61dc23ba798e4b830d585486ca5"},
+    {file = "coverage-7.10.5-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:42144e8e346de44a6f1dbd0a56575dd8ab8dfa7e9007da02ea5b1c30ab33a7db"},
+    {file = "coverage-7.10.5-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:66c644cbd7aed8fe266d5917e2c9f65458a51cfe5eeff9c05f15b335f697066e"},
+    {file = "coverage-7.10.5-cp313-cp313t-win32.whl", hash = "sha256:2d1b73023854068c44b0c554578a4e1ef1b050ed07cf8b431549e624a29a66ee"},
+    {file = "coverage-7.10.5-cp313-cp313t-win_amd64.whl", hash = "sha256:54a1532c8a642d8cc0bd5a9a51f5a9dcc440294fd06e9dda55e743c5ec1a8f14"},
+    {file = "coverage-7.10.5-cp313-cp313t-win_arm64.whl", hash = "sha256:74d5b63fe3f5f5d372253a4ef92492c11a4305f3550631beaa432fc9df16fcff"},
+    {file = "coverage-7.10.5-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:68c5e0bc5f44f68053369fa0d94459c84548a77660a5f2561c5e5f1e3bed7031"},
+    {file = "coverage-7.10.5-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:cf33134ffae93865e32e1e37df043bef15a5e857d8caebc0099d225c579b0fa3"},
+    {file = "coverage-7.10.5-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:ad8fa9d5193bafcf668231294241302b5e683a0518bf1e33a9a0dfb142ec3031"},
+    {file = "coverage-7.10.5-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:146fa1531973d38ab4b689bc764592fe6c2f913e7e80a39e7eeafd11f0ef6db2"},
+    {file = "coverage-7.10.5-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6013a37b8a4854c478d3219ee8bc2392dea51602dd0803a12d6f6182a0061762"},
+    {file = "coverage-7.10.5-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:eb90fe20db9c3d930fa2ad7a308207ab5b86bf6a76f54ab6a40be4012d88fcae"},
+    {file = "coverage-7.10.5-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:384b34482272e960c438703cafe63316dfbea124ac62006a455c8410bf2a2262"},
+    {file = "coverage-7.10.5-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:467dc74bd0a1a7de2bedf8deaf6811f43602cb532bd34d81ffd6038d6d8abe99"},
+    {file = "coverage-7.10.5-cp314-cp314-win32.whl", hash = "sha256:556d23d4e6393ca898b2e63a5bca91e9ac2d5fb13299ec286cd69a09a7187fde"},
+    {file = "coverage-7.10.5-cp314-cp314-win_amd64.whl", hash = "sha256:f4446a9547681533c8fa3e3c6cf62121eeee616e6a92bd9201c6edd91beffe13"},
+    {file = "coverage-7.10.5-cp314-cp314-win_arm64.whl", hash = "sha256:5e78bd9cf65da4c303bf663de0d73bf69f81e878bf72a94e9af67137c69b9fe9"},
+    {file = "coverage-7.10.5-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:5661bf987d91ec756a47c7e5df4fbcb949f39e32f9334ccd3f43233bbb65e508"},
+    {file = "coverage-7.10.5-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:a46473129244db42a720439a26984f8c6f834762fc4573616c1f37f13994b357"},
+    {file = "coverage-7.10.5-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:1f64b8d3415d60f24b058b58d859e9512624bdfa57a2d1f8aff93c1ec45c429b"},
+    {file = "coverage-7.10.5-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:44d43de99a9d90b20e0163f9770542357f58860a26e24dc1d924643bd6aa7cb4"},
+    {file = "coverage-7.10.5-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a931a87e5ddb6b6404e65443b742cb1c14959622777f2a4efd81fba84f5d91ba"},
+    {file = "coverage-7.10.5-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:f9559b906a100029274448f4c8b8b0a127daa4dade5661dfd821b8c188058842"},
+    {file = "coverage-7.10.5-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:b08801e25e3b4526ef9ced1aa29344131a8f5213c60c03c18fe4c6170ffa2874"},
+    {file = "coverage-7.10.5-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:ed9749bb8eda35f8b636fb7632f1c62f735a236a5d4edadd8bbcc5ea0542e732"},
+    {file = "coverage-7.10.5-cp314-cp314t-win32.whl", hash = "sha256:609b60d123fc2cc63ccee6d17e4676699075db72d14ac3c107cc4976d516f2df"},
+    {file = "coverage-7.10.5-cp314-cp314t-win_amd64.whl", hash = "sha256:0666cf3d2c1626b5a3463fd5b05f5e21f99e6aec40a3192eee4d07a15970b07f"},
+    {file = "coverage-7.10.5-cp314-cp314t-win_arm64.whl", hash = "sha256:bc85eb2d35e760120540afddd3044a5bf69118a91a296a8b3940dfc4fdcfe1e2"},
+    {file = "coverage-7.10.5-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:62835c1b00c4a4ace24c1a88561a5a59b612fbb83a525d1c70ff5720c97c0610"},
+    {file = "coverage-7.10.5-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:5255b3bbcc1d32a4069d6403820ac8e6dbcc1d68cb28a60a1ebf17e47028e898"},
+    {file = "coverage-7.10.5-cp39-cp39-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:3876385722e335d6e991c430302c24251ef9c2a9701b2b390f5473199b1b8ebf"},
+    {file = "coverage-7.10.5-cp39-cp39-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:8048ce4b149c93447a55d279078c8ae98b08a6951a3c4d2d7e87f4efc7bfe100"},
+    {file = "coverage-7.10.5-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4028e7558e268dd8bcf4d9484aad393cafa654c24b4885f6f9474bf53183a82a"},
+    {file = "coverage-7.10.5-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:03f47dc870eec0367fcdd603ca6a01517d2504e83dc18dbfafae37faec66129a"},
+    {file = "coverage-7.10.5-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:2d488d7d42b6ded7ea0704884f89dcabd2619505457de8fc9a6011c62106f6e5"},
+    {file = "coverage-7.10.5-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:b3dcf2ead47fa8be14224ee817dfc1df98043af568fe120a22f81c0eb3c34ad2"},
+    {file = "coverage-7.10.5-cp39-cp39-win32.whl", hash = "sha256:02650a11324b80057b8c9c29487020073d5e98a498f1857f37e3f9b6ea1b2426"},
+    {file = "coverage-7.10.5-cp39-cp39-win_amd64.whl", hash = "sha256:b45264dd450a10f9e03237b41a9a24e85cbb1e278e5a32adb1a303f58f0017f3"},
+    {file = "coverage-7.10.5-py3-none-any.whl", hash = "sha256:0be24d35e4db1d23d0db5c0f6a74a962e2ec83c426b5cac09f4234aadef38e4a"},
+    {file = "coverage-7.10.5.tar.gz", hash = "sha256:f2e57716a78bc3ae80b2207be0709a3b2b63b9f2dcf9740ee6ac03588a2015b6"},
+]
+
+[package.dependencies]
+tomli = {version = "*", optional = true, markers = "python_full_version <= \"3.11.0a6\" and extra == \"toml\""}
+
+[package.extras]
+toml = ["tomli ; python_full_version <= \"3.11.0a6\""]
+
+[[package]]
+name = "cryptography"
+version = "45.0.6"
+description = "cryptography is a package which provides cryptographic recipes and primitives to Python developers."
+optional = false
+python-versions = "!=3.9.0,!=3.9.1,>=3.7"
+groups = ["main"]
+files = [
+    {file = "cryptography-45.0.6-cp311-abi3-macosx_10_9_universal2.whl", hash = "sha256:048e7ad9e08cf4c0ab07ff7f36cc3115924e22e2266e034450a890d9e312dd74"},
+    {file = "cryptography-45.0.6-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:44647c5d796f5fc042bbc6d61307d04bf29bccb74d188f18051b635f20a9c75f"},
+    {file = "cryptography-45.0.6-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:e40b80ecf35ec265c452eea0ba94c9587ca763e739b8e559c128d23bff7ebbbf"},
+    {file = "cryptography-45.0.6-cp311-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:00e8724bdad672d75e6f069b27970883179bd472cd24a63f6e620ca7e41cc0c5"},
+    {file = "cryptography-45.0.6-cp311-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:7a3085d1b319d35296176af31c90338eeb2ddac8104661df79f80e1d9787b8b2"},
+    {file = "cryptography-45.0.6-cp311-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:1b7fa6a1c1188c7ee32e47590d16a5a0646270921f8020efc9a511648e1b2e08"},
+    {file = "cryptography-45.0.6-cp311-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:275ba5cc0d9e320cd70f8e7b96d9e59903c815ca579ab96c1e37278d231fc402"},
+    {file = "cryptography-45.0.6-cp311-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:f4028f29a9f38a2025abedb2e409973709c660d44319c61762202206ed577c42"},
+    {file = "cryptography-45.0.6-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:ee411a1b977f40bd075392c80c10b58025ee5c6b47a822a33c1198598a7a5f05"},
+    {file = "cryptography-45.0.6-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:e2a21a8eda2d86bb604934b6b37691585bd095c1f788530c1fcefc53a82b3453"},
+    {file = "cryptography-45.0.6-cp311-abi3-win32.whl", hash = "sha256:d063341378d7ee9c91f9d23b431a3502fc8bfacd54ef0a27baa72a0843b29159"},
+    {file = "cryptography-45.0.6-cp311-abi3-win_amd64.whl", hash = "sha256:833dc32dfc1e39b7376a87b9a6a4288a10aae234631268486558920029b086ec"},
+    {file = "cryptography-45.0.6-cp37-abi3-macosx_10_9_universal2.whl", hash = "sha256:3436128a60a5e5490603ab2adbabc8763613f638513ffa7d311c900a8349a2a0"},
+    {file = "cryptography-45.0.6-cp37-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:0d9ef57b6768d9fa58e92f4947cea96ade1233c0e236db22ba44748ffedca394"},
+    {file = "cryptography-45.0.6-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:ea3c42f2016a5bbf71825537c2ad753f2870191134933196bee408aac397b3d9"},
+    {file = "cryptography-45.0.6-cp37-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:20ae4906a13716139d6d762ceb3e0e7e110f7955f3bc3876e3a07f5daadec5f3"},
+    {file = "cryptography-45.0.6-cp37-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:2dac5ec199038b8e131365e2324c03d20e97fe214af051d20c49db129844e8b3"},
+    {file = "cryptography-45.0.6-cp37-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:18f878a34b90d688982e43f4b700408b478102dd58b3e39de21b5ebf6509c301"},
+    {file = "cryptography-45.0.6-cp37-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:5bd6020c80c5b2b2242d6c48487d7b85700f5e0038e67b29d706f98440d66eb5"},
+    {file = "cryptography-45.0.6-cp37-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:eccddbd986e43014263eda489abbddfbc287af5cddfd690477993dbb31e31016"},
+    {file = "cryptography-45.0.6-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:550ae02148206beb722cfe4ef0933f9352bab26b087af00e48fdfb9ade35c5b3"},
+    {file = "cryptography-45.0.6-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:5b64e668fc3528e77efa51ca70fadcd6610e8ab231e3e06ae2bab3b31c2b8ed9"},
+    {file = "cryptography-45.0.6-cp37-abi3-win32.whl", hash = "sha256:780c40fb751c7d2b0c6786ceee6b6f871e86e8718a8ff4bc35073ac353c7cd02"},
+    {file = "cryptography-45.0.6-cp37-abi3-win_amd64.whl", hash = "sha256:20d15aed3ee522faac1a39fbfdfee25d17b1284bafd808e1640a74846d7c4d1b"},
+    {file = "cryptography-45.0.6-pp310-pypy310_pp73-macosx_10_9_x86_64.whl", hash = "sha256:705bb7c7ecc3d79a50f236adda12ca331c8e7ecfbea51edd931ce5a7a7c4f012"},
+    {file = "cryptography-45.0.6-pp310-pypy310_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:826b46dae41a1155a0c0e66fafba43d0ede1dc16570b95e40c4d83bfcf0a451d"},
+    {file = "cryptography-45.0.6-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:cc4d66f5dc4dc37b89cfef1bd5044387f7a1f6f0abb490815628501909332d5d"},
+    {file = "cryptography-45.0.6-pp310-pypy310_pp73-manylinux_2_34_aarch64.whl", hash = "sha256:f68f833a9d445cc49f01097d95c83a850795921b3f7cc6488731e69bde3288da"},
+    {file = "cryptography-45.0.6-pp310-pypy310_pp73-manylinux_2_34_x86_64.whl", hash = "sha256:3b5bf5267e98661b9b888a9250d05b063220dfa917a8203744454573c7eb79db"},
+    {file = "cryptography-45.0.6-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:2384f2ab18d9be88a6e4f8972923405e2dbb8d3e16c6b43f15ca491d7831bd18"},
+    {file = "cryptography-45.0.6-pp311-pypy311_pp73-macosx_10_9_x86_64.whl", hash = "sha256:fc022c1fa5acff6def2fc6d7819bbbd31ccddfe67d075331a65d9cfb28a20983"},
+    {file = "cryptography-45.0.6-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:3de77e4df42ac8d4e4d6cdb342d989803ad37707cf8f3fbf7b088c9cbdd46427"},
+    {file = "cryptography-45.0.6-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:599c8d7df950aa68baa7e98f7b73f4f414c9f02d0e8104a30c0182a07732638b"},
+    {file = "cryptography-45.0.6-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl", hash = "sha256:31a2b9a10530a1cb04ffd6aa1cd4d3be9ed49f7d77a4dafe198f3b382f41545c"},
+    {file = "cryptography-45.0.6-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl", hash = "sha256:e5b3dda1b00fb41da3af4c5ef3f922a200e33ee5ba0f0bc9ecf0b0c173958385"},
+    {file = "cryptography-45.0.6-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:629127cfdcdc6806dfe234734d7cb8ac54edaf572148274fa377a7d3405b0043"},
+    {file = "cryptography-45.0.6.tar.gz", hash = "sha256:5c966c732cf6e4a276ce83b6e4c729edda2df6929083a952cc7da973c539c719"},
+]
+
+[package.dependencies]
+cffi = {version = ">=1.14", markers = "platform_python_implementation != \"PyPy\""}
+
+[package.extras]
+docs = ["sphinx (>=5.3.0)", "sphinx-inline-tabs ; python_full_version >= \"3.8.0\"", "sphinx-rtd-theme (>=3.0.0) ; python_full_version >= \"3.8.0\""]
+docstest = ["pyenchant (>=3)", "readme-renderer (>=30.0)", "sphinxcontrib-spelling (>=7.3.1)"]
+nox = ["nox (>=2024.4.15)", "nox[uv] (>=2024.3.2) ; python_full_version >= \"3.8.0\""]
+pep8test = ["check-sdist ; python_full_version >= \"3.8.0\"", "click (>=8.0.1)", "mypy (>=1.4)", "ruff (>=0.3.6)"]
+sdist = ["build (>=1.0.0)"]
+ssh = ["bcrypt (>=3.1.5)"]
+test = ["certifi (>=2024)", "cryptography-vectors (==45.0.6)", "pretend (>=0.7)", "pytest (>=7.4.0)", "pytest-benchmark (>=4.0)", "pytest-cov (>=2.10.1)", "pytest-xdist (>=3.5.0)"]
+test-randomorder = ["pytest-randomly"]
+
 [[package]]
 name = "deprecation"
 version = "2.1.0"
@@ -235,7 +480,7 @@ version = "1.3.0"
 description = "Backport of PEP 654 (exception groups)"
 optional = false
 python-versions = ">=3.7"
-groups = ["main"]
+groups = ["main", "dev"]
 markers = "python_version < \"3.11\""
 files = [
     {file = "exceptiongroup-1.3.0-py3-none-any.whl", hash = "sha256:4d111e6e0c13d0644cad6ddaa7ed0261a0b36971f6d23e7ec9b4b9097da78a10"},
@@ -710,7 +955,7 @@ version = "2.1.0"
 description = "brain-dead simple config-ini parsing"
 optional = false
 python-versions = ">=3.8"
-groups = ["main"]
+groups = ["dev"]
 files = [
     {file = "iniconfig-2.1.0-py3-none-any.whl", hash = "sha256:9deba5723312380e77435581c6bf4935c94cbfab9b1ed33ef8d238ea168eb760"},
     {file = "iniconfig-2.1.0.tar.gz", hash = "sha256:3abbd2e30b36733fee78f9c7f7308f2d0050e88f0087fd25c2645f63c773e1c7"},
@@ -757,6 +1002,18 @@ dynamodb = ["boto3 (>=1.9.71)"]
 redis = ["redis (>=2.10.5)"]
 test-filesource = ["pyyaml (>=5.3.1)", "watchdog (>=3.0.0)"]
 
+[[package]]
+name = "nodeenv"
+version = "1.9.1"
+description = "Node.js virtual environment builder"
+optional = false
+python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7"
+groups = ["dev"]
+files = [
+    {file = "nodeenv-1.9.1-py2.py3-none-any.whl", hash = "sha256:ba11c9782d29c27c70ffbdda2d7415098754709be8a7056d79a737cd901155c9"},
+    {file = "nodeenv-1.9.1.tar.gz", hash = "sha256:6ec12890a2dab7946721edbfbcd91f3319c6ccc9aec47be7c7e6b7011ee6645f"},
+]
+
 [[package]]
 name = "opentelemetry-api"
 version = "1.35.0"
@@ -779,7 +1036,7 @@ version = "25.0"
 description = "Core utilities for Python packages"
 optional = false
 python-versions = ">=3.8"
-groups = ["main"]
+groups = ["main", "dev"]
 files = [
     {file = "packaging-25.0-py3-none-any.whl", hash = "sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484"},
     {file = "packaging-25.0.tar.gz", hash = "sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f"},
@@ -791,7 +1048,7 @@ version = "1.6.0"
 description = "plugin and hook calling mechanisms for python"
 optional = false
 python-versions = ">=3.9"
-groups = ["main"]
+groups = ["dev"]
 files = [
     {file = "pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746"},
     {file = "pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3"},
@@ -883,6 +1140,19 @@ files = [
 [package.dependencies]
 pyasn1 = ">=0.6.1,<0.7.0"
 
+[[package]]
+name = "pycparser"
+version = "2.22"
+description = "C parser in Python"
+optional = false
+python-versions = ">=3.8"
+groups = ["main"]
+markers = "platform_python_implementation != \"PyPy\""
+files = [
+    {file = "pycparser-2.22-py3-none-any.whl", hash = "sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc"},
+    {file = "pycparser-2.22.tar.gz", hash = "sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6"},
+]
+
 [[package]]
 name = "pydantic"
 version = "2.11.7"
@@ -1047,7 +1317,7 @@ version = "2.19.2"
 description = "Pygments is a syntax highlighting package written in Python."
 optional = false
 python-versions = ">=3.8"
-groups = ["main"]
+groups = ["dev"]
 files = [
     {file = "pygments-2.19.2-py3-none-any.whl", hash = "sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b"},
     {file = "pygments-2.19.2.tar.gz", hash = "sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887"},
@@ -1068,6 +1338,9 @@ files = [
     {file = "pyjwt-2.10.1.tar.gz", hash = "sha256:3cc5772eb20009233caf06e9d8a0577824723b44e6648ee0a2aedb6cf9381953"},
 ]
 
+[package.dependencies]
+cryptography = {version = ">=3.4.0", optional = true, markers = "extra == \"crypto\""}
+
 [package.extras]
 crypto = ["cryptography (>=3.4.0)"]
 dev = ["coverage[toml] (==5.0.4)", "cryptography (>=3.4.0)", "pre-commit", "pytest (>=6.0.0,<7.0.0)", "sphinx", "sphinx-rtd-theme", "zope.interface"]
@@ -1086,13 +1359,34 @@ files = [
     {file = "pyrfc3339-2.0.1.tar.gz", hash = "sha256:e47843379ea35c1296c3b6c67a948a1a490ae0584edfcbdea0eaffb5dd29960b"},
 ]
 
+[[package]]
+name = "pyright"
+version = "1.1.404"
+description = "Command line wrapper for pyright"
+optional = false
+python-versions = ">=3.7"
+groups = ["dev"]
+files = [
+    {file = "pyright-1.1.404-py3-none-any.whl", hash = "sha256:c7b7ff1fdb7219c643079e4c3e7d4125f0dafcc19d253b47e898d130ea426419"},
+    {file = "pyright-1.1.404.tar.gz", hash = "sha256:455e881a558ca6be9ecca0b30ce08aa78343ecc031d37a198ffa9a7a1abeb63e"},
+]
+
+[package.dependencies]
+nodeenv = ">=1.6.0"
+typing-extensions = ">=4.1"
+
+[package.extras]
+all = ["nodejs-wheel-binaries", "twine (>=3.4.1)"]
+dev = ["twine (>=3.4.1)"]
+nodejs = ["nodejs-wheel-binaries"]
+
 [[package]]
 name = "pytest"
 version = "8.4.1"
 description = "pytest: simple powerful testing with Python"
 optional = false
 python-versions = ">=3.9"
-groups = ["main"]
+groups = ["dev"]
 files = [
     {file = "pytest-8.4.1-py3-none-any.whl", hash = "sha256:539c70ba6fcead8e78eebbf1115e8b589e7565830d7d006a8723f19ac8a0afb7"},
     {file = "pytest-8.4.1.tar.gz", hash = "sha256:7c67fd69174877359ed9371ec3af8a3d2b04741818c51e5e99cc1742251fa93c"},
@@ -1116,7 +1410,7 @@ version = "1.1.0"
 description = "Pytest support for asyncio"
 optional = false
 python-versions = ">=3.9"
-groups = ["main"]
+groups = ["dev"]
 files = [
     {file = "pytest_asyncio-1.1.0-py3-none-any.whl", hash = "sha256:5fe2d69607b0bd75c656d1211f969cadba035030156745ee09e7d71740e58ecf"},
     {file = "pytest_asyncio-1.1.0.tar.gz", hash = "sha256:796aa822981e01b68c12e4827b8697108f7205020f24b5793b3c41555dab68ea"},
@@ -1130,13 +1424,33 @@ pytest = ">=8.2,<9"
 docs = ["sphinx (>=5.3)", "sphinx-rtd-theme (>=1)"]
 testing = ["coverage (>=6.2)", "hypothesis (>=5.7.1)"]
 
+[[package]]
+name = "pytest-cov"
+version = "6.2.1"
+description = "Pytest plugin for measuring coverage."
+optional = false
+python-versions = ">=3.9"
+groups = ["dev"]
+files = [
+    {file = "pytest_cov-6.2.1-py3-none-any.whl", hash = "sha256:f5bc4c23f42f1cdd23c70b1dab1bbaef4fc505ba950d53e0081d0730dd7e86d5"},
+    {file = "pytest_cov-6.2.1.tar.gz", hash = "sha256:25cc6cc0a5358204b8108ecedc51a9b57b34cc6b8c967cc2c01a4e00d8a67da2"},
+]
+
+[package.dependencies]
+coverage = {version = ">=7.5", extras = ["toml"]}
+pluggy = ">=1.2"
+pytest = ">=6.2.5"
+
+[package.extras]
+testing = ["fields", "hunter", "process-tests", "pytest-xdist", "virtualenv"]
+
 [[package]]
 name = "pytest-mock"
 version = "3.14.1"
 description = "Thin-wrapper around the mock package for easier use with pytest"
 optional = false
 python-versions = ">=3.8"
-groups = ["main"]
+groups = ["dev"]
 files = [
     {file = "pytest_mock-3.14.1-py3-none-any.whl", hash = "sha256:178aefcd11307d874b4cd3100344e7e2d888d9791a6a1d9bfe90fbc1b74fd1d0"},
     {file = "pytest_mock-3.14.1.tar.gz", hash = "sha256:159e9edac4c451ce77a5cdb9fc5d1100708d2dd4ba3c3df572f14097351af80e"},
@@ -1253,31 +1567,31 @@ pyasn1 = ">=0.1.3"
 
 [[package]]
 name = "ruff"
-version = "0.12.9"
+version = "0.12.11"
 description = "An extremely fast Python linter and code formatter, written in Rust."
 optional = false
 python-versions = ">=3.7"
 groups = ["dev"]
 files = [
-    {file = "ruff-0.12.9-py3-none-linux_armv6l.whl", hash = "sha256:fcebc6c79fcae3f220d05585229463621f5dbf24d79fdc4936d9302e177cfa3e"},
-    {file = "ruff-0.12.9-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:aed9d15f8c5755c0e74467731a007fcad41f19bcce41cd75f768bbd687f8535f"},
-    {file = "ruff-0.12.9-py3-none-macosx_11_0_arm64.whl", hash = "sha256:5b15ea354c6ff0d7423814ba6d44be2807644d0c05e9ed60caca87e963e93f70"},
-    {file = "ruff-0.12.9-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d596c2d0393c2502eaabfef723bd74ca35348a8dac4267d18a94910087807c53"},
-    {file = "ruff-0.12.9-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:1b15599931a1a7a03c388b9c5df1bfa62be7ede6eb7ef753b272381f39c3d0ff"},
-    {file = "ruff-0.12.9-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:3d02faa2977fb6f3f32ddb7828e212b7dd499c59eb896ae6c03ea5c303575756"},
-    {file = "ruff-0.12.9-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:17d5b6b0b3a25259b69ebcba87908496e6830e03acfb929ef9fd4c58675fa2ea"},
-    {file = "ruff-0.12.9-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:72db7521860e246adbb43f6ef464dd2a532ef2ef1f5dd0d470455b8d9f1773e0"},
-    {file = "ruff-0.12.9-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a03242c1522b4e0885af63320ad754d53983c9599157ee33e77d748363c561ce"},
-    {file = "ruff-0.12.9-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9fc83e4e9751e6c13b5046d7162f205d0a7bac5840183c5beebf824b08a27340"},
-    {file = "ruff-0.12.9-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:881465ed56ba4dd26a691954650de6ad389a2d1fdb130fe51ff18a25639fe4bb"},
-    {file = "ruff-0.12.9-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:43f07a3ccfc62cdb4d3a3348bf0588358a66da756aa113e071b8ca8c3b9826af"},
-    {file = "ruff-0.12.9-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:07adb221c54b6bba24387911e5734357f042e5669fa5718920ee728aba3cbadc"},
-    {file = "ruff-0.12.9-py3-none-musllinux_1_2_i686.whl", hash = "sha256:f5cd34fabfdea3933ab85d72359f118035882a01bff15bd1d2b15261d85d5f66"},
-    {file = "ruff-0.12.9-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:f6be1d2ca0686c54564da8e7ee9e25f93bdd6868263805f8c0b8fc6a449db6d7"},
-    {file = "ruff-0.12.9-py3-none-win32.whl", hash = "sha256:cc7a37bd2509974379d0115cc5608a1a4a6c4bff1b452ea69db83c8855d53f93"},
-    {file = "ruff-0.12.9-py3-none-win_amd64.whl", hash = "sha256:6fb15b1977309741d7d098c8a3cb7a30bc112760a00fb6efb7abc85f00ba5908"},
-    {file = "ruff-0.12.9-py3-none-win_arm64.whl", hash = "sha256:63c8c819739d86b96d500cce885956a1a48ab056bbcbc61b747ad494b2485089"},
-    {file = "ruff-0.12.9.tar.gz", hash = "sha256:fbd94b2e3c623f659962934e52c2bea6fc6da11f667a427a368adaf3af2c866a"},
+    {file = "ruff-0.12.11-py3-none-linux_armv6l.whl", hash = "sha256:93fce71e1cac3a8bf9200e63a38ac5c078f3b6baebffb74ba5274fb2ab276065"},
+    {file = "ruff-0.12.11-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:b8e33ac7b28c772440afa80cebb972ffd823621ded90404f29e5ab6d1e2d4b93"},
+    {file = "ruff-0.12.11-py3-none-macosx_11_0_arm64.whl", hash = "sha256:d69fb9d4937aa19adb2e9f058bc4fbfe986c2040acb1a4a9747734834eaa0bfd"},
+    {file = "ruff-0.12.11-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:411954eca8464595077a93e580e2918d0a01a19317af0a72132283e28ae21bee"},
+    {file = "ruff-0.12.11-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:6a2c0a2e1a450f387bf2c6237c727dd22191ae8c00e448e0672d624b2bbd7fb0"},
+    {file = "ruff-0.12.11-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:8ca4c3a7f937725fd2413c0e884b5248a19369ab9bdd850b5781348ba283f644"},
+    {file = "ruff-0.12.11-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:4d1df0098124006f6a66ecf3581a7f7e754c4df7644b2e6704cd7ca80ff95211"},
+    {file = "ruff-0.12.11-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5a8dd5f230efc99a24ace3b77e3555d3fbc0343aeed3fc84c8d89e75ab2ff793"},
+    {file = "ruff-0.12.11-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4dc75533039d0ed04cd33fb8ca9ac9620b99672fe7ff1533b6402206901c34ee"},
+    {file = "ruff-0.12.11-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4fc58f9266d62c6eccc75261a665f26b4ef64840887fc6cbc552ce5b29f96cc8"},
+    {file = "ruff-0.12.11-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:5a0113bd6eafd545146440225fe60b4e9489f59eb5f5f107acd715ba5f0b3d2f"},
+    {file = "ruff-0.12.11-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:0d737b4059d66295c3ea5720e6efc152623bb83fde5444209b69cd33a53e2000"},
+    {file = "ruff-0.12.11-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:916fc5defee32dbc1fc1650b576a8fed68f5e8256e2180d4d9855aea43d6aab2"},
+    {file = "ruff-0.12.11-py3-none-musllinux_1_2_i686.whl", hash = "sha256:c984f07d7adb42d3ded5be894fb4007f30f82c87559438b4879fe7aa08c62b39"},
+    {file = "ruff-0.12.11-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:e07fbb89f2e9249f219d88331c833860489b49cdf4b032b8e4432e9b13e8a4b9"},
+    {file = "ruff-0.12.11-py3-none-win32.whl", hash = "sha256:c792e8f597c9c756e9bcd4d87cf407a00b60af77078c96f7b6366ea2ce9ba9d3"},
+    {file = "ruff-0.12.11-py3-none-win_amd64.whl", hash = "sha256:a3283325960307915b6deb3576b96919ee89432ebd9c48771ca12ee8afe4a0fd"},
+    {file = "ruff-0.12.11-py3-none-win_arm64.whl", hash = "sha256:bae4d6e6a2676f8fb0f98b74594a048bae1b944aab17e9f5d504062303c6dbea"},
+    {file = "ruff-0.12.11.tar.gz", hash = "sha256:c6b09ae8426a65bbee5425b9d0b82796dbb07cb1af045743c79bfb163001165d"},
 ]
 
 [[package]]
@@ -1411,7 +1725,7 @@ version = "2.2.1"
 description = "A lil' TOML parser"
 optional = false
 python-versions = ">=3.8"
-groups = ["main"]
+groups = ["dev"]
 markers = "python_version < \"3.11\""
 files = [
     {file = "tomli-2.2.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:678e4fa69e4575eb77d103de3df8a895e1591b48e740211bd1067378c69e8249"},
@@ -1454,7 +1768,7 @@ version = "4.14.1"
 description = "Backported and Experimental Type Hints for Python 3.9+"
 optional = false
 python-versions = ">=3.9"
-groups = ["main"]
+groups = ["main", "dev"]
 files = [
     {file = "typing_extensions-4.14.1-py3-none-any.whl", hash = "sha256:d1e1e3b58374dc93031d6eda2420a48ea44a36c2b4766a4fdeb3710755731d76"},
     {file = "typing_extensions-4.14.1.tar.gz", hash = "sha256:38b39f4aeeab64884ce9f74c94263ef78f3c22467c8724005483154c26648d36"},
@@ -1615,4 +1929,4 @@ type = ["pytest-mypy"]
 [metadata]
 lock-version = "2.1"
 python-versions = ">=3.10,<4.0"
-content-hash = "4cc687aabe5865665fb8c4ccc0ea7e0af80b41e401ca37919f57efa6e0b5be00"
+content-hash = "0c40b63c3c921846cf05ccfb4e685d4959854b29c2c302245f9832e20aac6954"

autogpt_platform/autogpt_libs/pyproject.toml

@@ -9,21 +9,25 @@ packages = [{ include = "autogpt_libs" }]
 [tool.poetry.dependencies]
 python = ">=3.10,<4.0"
 colorama = "^0.4.6"
+cryptography = "^45.0"
 expiringdict = "^1.2.2"
 fastapi = "^0.116.1"
 google-cloud-logging = "^3.12.1"
 launchdarkly-server-sdk = "^9.12.0"
 pydantic = "^2.11.7"
 pydantic-settings = "^2.10.1"
-pyjwt = "^2.10.1"
-pytest-asyncio = "^1.1.0"
-pytest-mock = "^3.14.1"
+pyjwt = { version = "^2.10.1", extras = ["crypto"] }
 redis = "^6.2.0"
 supabase = "^2.16.0"
 uvicorn = "^0.35.0"
 
 [tool.poetry.group.dev.dependencies]
-ruff = "^0.12.9"
+pyright = "^1.1.404"
+pytest = "^8.4.1"
+pytest-asyncio = "^1.1.0"
+pytest-mock = "^3.14.1"
+pytest-cov = "^6.2.1"
+ruff = "^0.12.11"
 
 [build-system]
 requires = ["poetry-core"]

autogpt_platform/backend/.env.default

@@ -16,13 +16,12 @@ DB_SCHEMA=platform
 DATABASE_URL="postgresql://${DB_USER}:${DB_PASS}@${DB_HOST}:${DB_PORT}/${DB_NAME}?schema=${DB_SCHEMA}&connect_timeout=${DB_CONNECT_TIMEOUT}"
 DIRECT_URL="postgresql://${DB_USER}:${DB_PASS}@${DB_HOST}:${DB_PORT}/${DB_NAME}?schema=${DB_SCHEMA}&connect_timeout=${DB_CONNECT_TIMEOUT}"
 PRISMA_SCHEMA="postgres/schema.prisma"
-ENABLE_AUTH=true
 
 ## ===== REQUIRED SERVICE CREDENTIALS ===== ##
 # Redis Configuration
 REDIS_HOST=localhost
 REDIS_PORT=6379
-REDIS_PASSWORD=password
+# REDIS_PASSWORD=
 
 # RabbitMQ Credentials
 RABBITMQ_DEFAULT_USER=rabbitmq_user_default
@@ -31,7 +30,7 @@ RABBITMQ_DEFAULT_PASS=k0VMxyIJF9S35f3x2uaw5IWAl6Y536O7
 # Supabase Authentication
 SUPABASE_URL=http://localhost:8000
 SUPABASE_SERVICE_ROLE_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyAgCiAgICAicm9sZSI6ICJzZXJ2aWNlX3JvbGUiLAogICAgImlzcyI6ICJzdXBhYmFzZS1kZW1vIiwKICAgICJpYXQiOiAxNjQxNzY5MjAwLAogICAgImV4cCI6IDE3OTk1MzU2MDAKfQ.DaYlNEoUrrEn2Ig7tqibS-PHK5vgusbcbo7X36XVt4Q
-SUPABASE_JWT_SECRET=your-super-secret-jwt-token-with-at-least-32-characters-long
+JWT_VERIFY_KEY=your-super-secret-jwt-token-with-at-least-32-characters-long
 
 ## ===== REQUIRED SECURITY KEYS ===== ##
 # Generate using: from cryptography.fernet import Fernet;Fernet.generate_key().decode()
@@ -67,6 +66,11 @@ NVIDIA_API_KEY=
 GITHUB_CLIENT_ID=
 GITHUB_CLIENT_SECRET=
 
+# Notion OAuth App server credentials - https://developers.notion.com/docs/authorization
+# Configure a public integration
+NOTION_CLIENT_ID=
+NOTION_CLIENT_SECRET=
+
 # Google OAuth App server credentials - https://console.cloud.google.com/apis/credentials, and enable gmail api and set scopes
 # https://console.cloud.google.com/apis/credentials/consent ?project=<your_project_id>
 # You'll need to add/enable the following scopes (minimum):
@@ -175,4 +179,4 @@ SMARTLEAD_API_KEY=
 ZEROBOUNCE_API_KEY=
 
 # Other Services
-AUTOMOD_API_KEY=
+AUTOMOD_API_KEY=

autogpt_platform/backend/.gitignore

@@ -9,4 +9,12 @@ secrets/*
 !secrets/.gitkeep
 
 *.ignore.*
-*.ign.*
+*.ign.*
+
+# Load test results and reports
+load-tests/*_RESULTS.md
+load-tests/*_REPORT.md
+load-tests/results/
+load-tests/*.json
+load-tests/*.log
+load-tests/node_modules/*

autogpt_platform/backend/Dockerfile

@@ -9,8 +9,15 @@ WORKDIR /app
 
 RUN echo 'Acquire::http::Pipeline-Depth 0;\nAcquire::http::No-Cache true;\nAcquire::BrokenProxy true;\n' > /etc/apt/apt.conf.d/99fixbadproxy
 
-# Update package list and install Python and build dependencies
+# Install Node.js repository key and setup
 RUN apt-get update --allow-releaseinfo-change --fix-missing \
+    && apt-get install -y curl ca-certificates gnupg \
+    && mkdir -p /etc/apt/keyrings \
+    && curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg \
+    && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list
+
+# Update package list and install Python, Node.js, and build dependencies
+RUN apt-get update \
     && apt-get install -y \
     python3.13 \
     python3.13-dev \
@@ -20,7 +27,9 @@ RUN apt-get update --allow-releaseinfo-change --fix-missing \
     libpq5 \
     libz-dev \
     libssl-dev \
-    postgresql-client
+    postgresql-client \
+    nodejs \
+    && rm -rf /var/lib/apt/lists/*
 
 ENV POETRY_HOME=/opt/poetry
 ENV POETRY_NO_INTERACTION=1
@@ -38,6 +47,7 @@ RUN poetry install --no-ansi --no-root
 
 # Generate Prisma client
 COPY autogpt_platform/backend/schema.prisma ./
+COPY autogpt_platform/backend/backend/data/partial_types.py ./backend/data/partial_types.py
 RUN poetry run prisma generate
 
 FROM debian:13-slim AS server_dependencies
@@ -54,13 +64,18 @@ ENV PATH=/opt/poetry/bin:$PATH
 # Install Python without upgrading system-managed packages
 RUN apt-get update && apt-get install -y \
     python3.13 \
-    python3-pip
+    python3-pip \
+    && rm -rf /var/lib/apt/lists/*
 
 # Copy only necessary files from builder
 COPY --from=builder /app /app
 COPY --from=builder /usr/local/lib/python3* /usr/local/lib/python3*
 COPY --from=builder /usr/local/bin/poetry /usr/local/bin/poetry
-# Copy Prisma binaries
+# Copy Node.js installation for Prisma
+COPY --from=builder /usr/bin/node /usr/bin/node
+COPY --from=builder /usr/lib/node_modules /usr/lib/node_modules
+COPY --from=builder /usr/bin/npm /usr/bin/npm
+COPY --from=builder /usr/bin/npx /usr/bin/npx
 COPY --from=builder /root/.cache/prisma-python/binaries /root/.cache/prisma-python/binaries
 
 ENV PATH="/app/autogpt_platform/backend/.venv/bin:$PATH"
@@ -78,6 +93,7 @@ FROM server_dependencies AS migrate
 
 # Migration stage only needs schema and migrations - much lighter than full backend
 COPY autogpt_platform/backend/schema.prisma /app/autogpt_platform/backend/
+COPY autogpt_platform/backend/backend/data/partial_types.py /app/autogpt_platform/backend/backend/data/partial_types.py
 COPY autogpt_platform/backend/migrations /app/autogpt_platform/backend/migrations
 
 FROM server_dependencies AS server

autogpt_platform/backend/TESTING.md

@@ -132,17 +132,58 @@ def test_endpoint_success(snapshot: Snapshot):
 
 ### Testing with Authentication
 
+For the main API routes that use JWT authentication, auth is provided by the `autogpt_libs.auth` module. If the test actually uses the `user_id`, the recommended approach for testing is to mock the `get_jwt_payload` function, which underpins all higher-level auth functions used in the API (`requires_user`, `requires_admin_user`, `get_user_id`).
+
+If the test doesn't need the `user_id` specifically, mocking is not necessary as during tests auth is disabled anyway (see `conftest.py`).
+
+#### Using Global Auth Fixtures
+
+Two global auth fixtures are provided by `backend/server/conftest.py`:
+
+- `mock_jwt_user` - Regular user with `test_user_id` ("test-user-id")
+- `mock_jwt_admin` - Admin user with `admin_user_id` ("admin-user-id")
+
+These provide the easiest way to set up authentication mocking in test modules:
+
 ```python
-def override_auth_middleware():
-    return {"sub": "test-user-id"}
+import fastapi
+import fastapi.testclient
+import pytest
+from backend.server.v2.myroute import router
 
-def override_get_user_id():
-    return "test-user-id"
+app = fastapi.FastAPI()
+app.include_router(router)
+client = fastapi.testclient.TestClient(app)
 
-app.dependency_overrides[auth_middleware] = override_auth_middleware
-app.dependency_overrides[get_user_id] = override_get_user_id
+@pytest.fixture(autouse=True)
+def setup_app_auth(mock_jwt_user):
+    """Setup auth overrides for all tests in this module"""
+    from autogpt_libs.auth.jwt_utils import get_jwt_payload
+
+    app.dependency_overrides[get_jwt_payload] = mock_jwt_user['get_jwt_payload']
+    yield
+    app.dependency_overrides.clear()
 ```
 
+For admin-only endpoints, use `mock_jwt_admin` instead:
+
+```python
+@pytest.fixture(autouse=True)
+def setup_app_auth(mock_jwt_admin):
+    """Setup auth overrides for admin tests"""
+    from autogpt_libs.auth.jwt_utils import get_jwt_payload
+
+    app.dependency_overrides[get_jwt_payload] = mock_jwt_admin['get_jwt_payload']
+    yield
+    app.dependency_overrides.clear()
+```
+
+The IDs are also available separately as fixtures:
+
+- `test_user_id`
+- `admin_user_id`
+- `target_user_id` (for admin <-> user operations)
+
 ### Mocking External Services
 
 ```python
@@ -153,10 +194,10 @@ def test_external_api_call(mocker, snapshot):
         "backend.services.external_api.call",
         return_value=mock_response
     )
-    
+
     response = client.post("/api/process")
     assert response.status_code == 200
-    
+
     snapshot.snapshot_dir = "snapshots"
     snapshot.assert_match(
         json.dumps(response.json(), indent=2, sort_keys=True),
@@ -187,6 +228,17 @@ def test_external_api_call(mocker, snapshot):
 - Use `async def` with `@pytest.mark.asyncio` for testing async functions directly
 
 ### 5. Fixtures
+
+#### Global Fixtures (conftest.py)
+
+Authentication fixtures are available globally from `conftest.py`:
+
+- `mock_jwt_user` - Standard user authentication
+- `mock_jwt_admin` - Admin user authentication
+- `configured_snapshot` - Pre-configured snapshot fixture
+
+#### Custom Fixtures
+
 Create reusable fixtures for common test data:
 
 ```python
@@ -202,9 +254,18 @@ def test_create_user(sample_user, snapshot):
     # ... test implementation
 ```
 
+#### Test Isolation
+
+All tests must use fixtures that ensure proper isolation:
+
+- Authentication overrides are automatically cleaned up after each test
+- Database connections are properly managed with cleanup
+- Mock objects are reset between tests
+
 ## CI/CD Integration
 
 The GitHub Actions workflow automatically runs tests on:
+
 - Pull requests
 - Pushes to main branch
 
@@ -216,16 +277,19 @@ Snapshot tests work in CI by:
 ## Troubleshooting
 
 ### Snapshot Mismatches
+
 - Review the diff carefully
 - If changes are expected: `poetry run pytest --snapshot-update`
 - If changes are unexpected: Fix the code causing the difference
 
 ### Async Test Issues
+
 - Ensure async functions use `@pytest.mark.asyncio`
 - Use `AsyncMock` for mocking async functions
 - FastAPI TestClient handles async automatically
 
 ### Import Errors
+
 - Check that all dependencies are in `pyproject.toml`
 - Run `poetry install` to ensure dependencies are installed
 - Verify import paths are correct
@@ -234,4 +298,4 @@ Snapshot tests work in CI by:
 
 Snapshot testing provides a powerful way to ensure API responses remain consistent. Combined with traditional assertions, it creates a robust test suite that catches regressions while remaining maintainable.
 
-Remember: Good tests are as important as good code!
+Remember: Good tests are as important as good code!

autogpt_platform/backend/backend/blocks/__init__.py

@@ -1,4 +1,3 @@
-import functools
 import importlib
 import logging
 import os
@@ -6,6 +5,8 @@ import re
 from pathlib import Path
 from typing import TYPE_CHECKING, TypeVar
 
+from backend.util.cache import cached
+
 logger = logging.getLogger(__name__)
 
 
@@ -15,7 +16,7 @@ if TYPE_CHECKING:
 T = TypeVar("T")
 
 
-@functools.cache
+@cached(ttl_seconds=3600)
 def load_all_blocks() -> dict[str, type["Block"]]:
     from backend.data.block import Block
     from backend.util.settings import Config

autogpt_platform/backend/backend/blocks/agent.py

@@ -1,8 +1,6 @@
 import logging
 from typing import Any, Optional
 
-from pydantic import JsonValue
-
 from backend.data.block import (
     Block,
     BlockCategory,
@@ -12,7 +10,7 @@ from backend.data.block import (
     BlockType,
     get_block,
 )
-from backend.data.execution import ExecutionStatus
+from backend.data.execution import ExecutionStatus, NodesInputMasks
 from backend.data.model import NodeExecutionStats, SchemaField
 from backend.util.json import validate_with_jsonschema
 from backend.util.retry import func_retry
@@ -33,7 +31,7 @@ class AgentExecutorBlock(Block):
         input_schema: dict = SchemaField(description="Input schema for the graph")
         output_schema: dict = SchemaField(description="Output schema for the graph")
 
-        nodes_input_masks: Optional[dict[str, dict[str, JsonValue]]] = SchemaField(
+        nodes_input_masks: Optional[NodesInputMasks] = SchemaField(
             default=None, hidden=True
         )
 

autogpt_platform/backend/backend/blocks/ai_condition.py

@@ -0,0 +1,214 @@
+from typing import Any
+
+from backend.blocks.llm import (
+    TEST_CREDENTIALS,
+    TEST_CREDENTIALS_INPUT,
+    AIBlockBase,
+    AICredentials,
+    AICredentialsField,
+    LlmModel,
+    LLMResponse,
+    llm_call,
+)
+from backend.data.block import BlockCategory, BlockOutput, BlockSchema
+from backend.data.model import APIKeyCredentials, NodeExecutionStats, SchemaField
+
+
+class AIConditionBlock(AIBlockBase):
+    """
+    An AI-powered condition block that uses natural language to evaluate conditions.
+
+    This block allows users to define conditions in plain English (e.g., "the input is an email address",
+    "the input is a city in the USA") and uses AI to determine if the input satisfies the condition.
+    It provides the same yes/no data pass-through functionality as the standard ConditionBlock.
+    """
+
+    class Input(BlockSchema):
+        input_value: Any = SchemaField(
+            description="The input value to evaluate with the AI condition",
+            placeholder="Enter the value to be evaluated (text, number, or any data)",
+        )
+        condition: str = SchemaField(
+            description="A plaintext English description of the condition to evaluate",
+            placeholder="E.g., 'the input is the body of an email', 'the input is a City in the USA', 'the input is an error or a refusal'",
+        )
+        yes_value: Any = SchemaField(
+            description="(Optional) Value to output if the condition is true. If not provided, input_value will be used.",
+            placeholder="Leave empty to use input_value, or enter a specific value",
+            default=None,
+        )
+        no_value: Any = SchemaField(
+            description="(Optional) Value to output if the condition is false. If not provided, input_value will be used.",
+            placeholder="Leave empty to use input_value, or enter a specific value",
+            default=None,
+        )
+        model: LlmModel = SchemaField(
+            title="LLM Model",
+            default=LlmModel.GPT4O,
+            description="The language model to use for evaluating the condition.",
+            advanced=False,
+        )
+        credentials: AICredentials = AICredentialsField()
+
+    class Output(BlockSchema):
+        result: bool = SchemaField(
+            description="The result of the AI condition evaluation (True or False)"
+        )
+        yes_output: Any = SchemaField(
+            description="The output value if the condition is true"
+        )
+        no_output: Any = SchemaField(
+            description="The output value if the condition is false"
+        )
+        error: str = SchemaField(
+            description="Error message if the AI evaluation is uncertain or fails"
+        )
+
+    def __init__(self):
+        super().__init__(
+            id="553ec5b8-6c45-4299-8d75-b394d05f72ff",
+            input_schema=AIConditionBlock.Input,
+            output_schema=AIConditionBlock.Output,
+            description="Uses AI to evaluate natural language conditions and provide conditional outputs",
+            categories={BlockCategory.AI, BlockCategory.LOGIC},
+            test_input={
+                "input_value": "john@example.com",
+                "condition": "the input is an email address",
+                "yes_value": "Valid email",
+                "no_value": "Not an email",
+                "model": LlmModel.GPT4O,
+                "credentials": TEST_CREDENTIALS_INPUT,
+            },
+            test_credentials=TEST_CREDENTIALS,
+            test_output=[
+                ("result", True),
+                ("yes_output", "Valid email"),
+            ],
+            test_mock={
+                "llm_call": lambda *args, **kwargs: LLMResponse(
+                    raw_response="",
+                    prompt=[],
+                    response="true",
+                    tool_calls=None,
+                    prompt_tokens=50,
+                    completion_tokens=10,
+                    reasoning=None,
+                )
+            },
+        )
+
+    async def llm_call(
+        self,
+        credentials: APIKeyCredentials,
+        llm_model: LlmModel,
+        prompt: list,
+        max_tokens: int,
+    ) -> LLMResponse:
+        """Wrapper method for llm_call to enable mocking in tests."""
+        return await llm_call(
+            credentials=credentials,
+            llm_model=llm_model,
+            prompt=prompt,
+            force_json_output=False,
+            max_tokens=max_tokens,
+        )
+
+    async def run(
+        self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
+    ) -> BlockOutput:
+        """
+        Evaluate the AI condition and return appropriate outputs.
+        """
+        # Prepare the yes and no values, using input_value as default
+        yes_value = (
+            input_data.yes_value
+            if input_data.yes_value is not None
+            else input_data.input_value
+        )
+        no_value = (
+            input_data.no_value
+            if input_data.no_value is not None
+            else input_data.input_value
+        )
+
+        # Convert input_value to string for AI evaluation
+        input_str = str(input_data.input_value)
+
+        # Create the prompt for AI evaluation
+        prompt = [
+            {
+                "role": "system",
+                "content": (
+                    "You are an AI assistant that evaluates conditions based on input data. "
+                    "You must respond with only 'true' or 'false' (lowercase) to indicate whether "
+                    "the given condition is met by the input value. Be accurate and consider the "
+                    "context and meaning of both the input and the condition."
+                ),
+            },
+            {
+                "role": "user",
+                "content": (
+                    f"Input value: {input_str}\n"
+                    f"Condition to evaluate: {input_data.condition}\n\n"
+                    f"Does the input value satisfy the condition? Respond with only 'true' or 'false'."
+                ),
+            },
+        ]
+
+        # Call the LLM
+        try:
+            response = await self.llm_call(
+                credentials=credentials,
+                llm_model=input_data.model,
+                prompt=prompt,
+                max_tokens=10,  # We only expect a true/false response
+            )
+
+            # Extract the boolean result from the response
+            response_text = response.response.strip().lower()
+            if response_text == "true":
+                result = True
+            elif response_text == "false":
+                result = False
+            else:
+                # If the response is not clear, try to interpret it using word boundaries
+                import re
+
+                # Use word boundaries to avoid false positives like 'untrue' or '10'
+                tokens = set(re.findall(r"\b(true|false|yes|no|1|0)\b", response_text))
+
+                if tokens == {"true"} or tokens == {"yes"} or tokens == {"1"}:
+                    result = True
+                elif tokens == {"false"} or tokens == {"no"} or tokens == {"0"}:
+                    result = False
+                else:
+                    # Unclear or conflicting response - default to False and yield error
+                    result = False
+                    yield "error", f"Unclear AI response: '{response.response}'"
+
+            # Update internal stats
+            self.merge_stats(
+                NodeExecutionStats(
+                    input_token_count=response.prompt_tokens,
+                    output_token_count=response.completion_tokens,
+                )
+            )
+            self.prompt = response.prompt
+
+        except Exception as e:
+            # In case of any error, default to False to be safe
+            result = False
+            # Log the error but don't fail the block execution
+            import logging
+
+            logger = logging.getLogger(__name__)
+            logger.error(f"AI condition evaluation failed: {str(e)}")
+            yield "error", f"AI evaluation failed: {str(e)}"
+
+        # Yield results
+        yield "result", result
+
+        if result:
+            yield "yes_output", yes_value
+        else:
+            yield "no_output", no_value

autogpt_platform/backend/backend/blocks/ai_image_customizer.py

@@ -0,0 +1,154 @@
+from enum import Enum
+from typing import Literal
+
+from pydantic import SecretStr
+from replicate.client import Client as ReplicateClient
+from replicate.helpers import FileOutput
+
+from backend.data.block import Block, BlockCategory, BlockOutput, BlockSchema
+from backend.data.model import (
+    APIKeyCredentials,
+    CredentialsField,
+    CredentialsMetaInput,
+    SchemaField,
+)
+from backend.integrations.providers import ProviderName
+from backend.util.file import MediaFileType
+
+
+class GeminiImageModel(str, Enum):
+    NANO_BANANA = "google/nano-banana"
+
+
+class OutputFormat(str, Enum):
+    JPG = "jpg"
+    PNG = "png"
+
+
+TEST_CREDENTIALS = APIKeyCredentials(
+    id="01234567-89ab-cdef-0123-456789abcdef",
+    provider="replicate",
+    api_key=SecretStr("mock-replicate-api-key"),
+    title="Mock Replicate API key",
+    expires_at=None,
+)
+
+TEST_CREDENTIALS_INPUT = {
+    "provider": TEST_CREDENTIALS.provider,
+    "id": TEST_CREDENTIALS.id,
+    "type": TEST_CREDENTIALS.type,
+    "title": TEST_CREDENTIALS.title,
+}
+
+
+class AIImageCustomizerBlock(Block):
+    class Input(BlockSchema):
+        credentials: CredentialsMetaInput[
+            Literal[ProviderName.REPLICATE], Literal["api_key"]
+        ] = CredentialsField(
+            description="Replicate API key with permissions for Google Gemini image models",
+        )
+        prompt: str = SchemaField(
+            description="A text description of the image you want to generate",
+            title="Prompt",
+        )
+        model: GeminiImageModel = SchemaField(
+            description="The AI model to use for image generation and editing",
+            default=GeminiImageModel.NANO_BANANA,
+            title="Model",
+        )
+        images: list[MediaFileType] = SchemaField(
+            description="Optional list of input images to reference or modify",
+            default=[],
+            title="Input Images",
+        )
+        output_format: OutputFormat = SchemaField(
+            description="Format of the output image",
+            default=OutputFormat.PNG,
+            title="Output Format",
+        )
+
+    class Output(BlockSchema):
+        image_url: MediaFileType = SchemaField(description="URL of the generated image")
+        error: str = SchemaField(description="Error message if generation failed")
+
+    def __init__(self):
+        super().__init__(
+            id="d76bbe4c-930e-4894-8469-b66775511f71",
+            description=(
+                "Generate and edit custom images using Google's Nano-Banana model from Gemini 2.5. "
+                "Provide a prompt and optional reference images to create or modify images."
+            ),
+            categories={BlockCategory.AI, BlockCategory.MULTIMEDIA},
+            input_schema=AIImageCustomizerBlock.Input,
+            output_schema=AIImageCustomizerBlock.Output,
+            test_input={
+                "prompt": "Make the scene more vibrant and colorful",
+                "model": GeminiImageModel.NANO_BANANA,
+                "images": [],
+                "output_format": OutputFormat.JPG,
+                "credentials": TEST_CREDENTIALS_INPUT,
+            },
+            test_output=[
+                ("image_url", "https://replicate.delivery/generated-image.jpg"),
+            ],
+            test_mock={
+                "run_model": lambda *args, **kwargs: MediaFileType(
+                    "https://replicate.delivery/generated-image.jpg"
+                ),
+            },
+            test_credentials=TEST_CREDENTIALS,
+        )
+
+    async def run(
+        self,
+        input_data: Input,
+        *,
+        credentials: APIKeyCredentials,
+        graph_exec_id: str,
+        user_id: str,
+        **kwargs,
+    ) -> BlockOutput:
+        try:
+            result = await self.run_model(
+                api_key=credentials.api_key,
+                model_name=input_data.model.value,
+                prompt=input_data.prompt,
+                images=input_data.images,
+                output_format=input_data.output_format.value,
+            )
+            yield "image_url", result
+        except Exception as e:
+            yield "error", str(e)
+
+    async def run_model(
+        self,
+        api_key: SecretStr,
+        model_name: str,
+        prompt: str,
+        images: list[MediaFileType],
+        output_format: str,
+    ) -> MediaFileType:
+        client = ReplicateClient(api_token=api_key.get_secret_value())
+
+        input_params: dict = {
+            "prompt": prompt,
+            "output_format": output_format,
+        }
+
+        # Add images to input if provided (API expects "image_input" parameter)
+        if images:
+            input_params["image_input"] = [str(img) for img in images]
+
+        output: FileOutput | str = await client.async_run(  # type: ignore
+            model_name,
+            input=input_params,
+            wait=False,
+        )
+
+        if isinstance(output, FileOutput):
+            return MediaFileType(output.url)
+        if isinstance(output, str):
+            return MediaFileType(output)
+
+        raise ValueError("No output received from the model")

autogpt_platform/backend/backend/blocks/airtable/_api.py

@@ -661,6 +661,167 @@ async def update_field(
 #################################################################
 
 
+async def get_table_schema(
+    credentials: Credentials,
+    base_id: str,
+    table_id_or_name: str,
+) -> dict:
+    """
+    Get the schema for a specific table, including all field definitions.
+
+    Args:
+        credentials: Airtable API credentials
+        base_id: The base ID
+        table_id_or_name: The table ID or name
+
+    Returns:
+        Dict containing table schema with fields information
+    """
+    # First get all tables to find the right one
+    response = await Requests().get(
+        f"https://api.airtable.com/v0/meta/bases/{base_id}/tables",
+        headers={"Authorization": credentials.auth_header()},
+    )
+
+    data = response.json()
+    tables = data.get("tables", [])
+
+    # Find the matching table
+    for table in tables:
+        if table.get("id") == table_id_or_name or table.get("name") == table_id_or_name:
+            return table
+
+    raise ValueError(f"Table '{table_id_or_name}' not found in base '{base_id}'")
+
+
+def get_empty_value_for_field(field_type: str) -> Any:
+    """
+    Return the appropriate empty value for a given Airtable field type.
+
+    Args:
+        field_type: The Airtable field type
+
+    Returns:
+        The appropriate empty value for that field type
+    """
+    # Fields that should be false when empty
+    if field_type == "checkbox":
+        return False
+
+    # Fields that should be empty arrays
+    if field_type in [
+        "multipleSelects",
+        "multipleRecordLinks",
+        "multipleAttachments",
+        "multipleLookupValues",
+        "multipleCollaborators",
+    ]:
+        return []
+
+    # Fields that should be 0 when empty (numeric types)
+    if field_type in [
+        "number",
+        "percent",
+        "currency",
+        "rating",
+        "duration",
+        "count",
+        "autoNumber",
+    ]:
+        return 0
+
+    # Fields that should be empty strings
+    if field_type in [
+        "singleLineText",
+        "multilineText",
+        "email",
+        "url",
+        "phoneNumber",
+        "richText",
+        "barcode",
+    ]:
+        return ""
+
+    # Everything else gets null (dates, single selects, formulas, etc.)
+    return None
+
+
+async def normalize_records(
+    records: list[dict],
+    table_schema: dict,
+    include_field_metadata: bool = False,
+) -> dict:
+    """
+    Normalize Airtable records to include all fields with proper empty values.
+
+    Args:
+        records: List of record objects from Airtable API
+        table_schema: Table schema containing field definitions
+        include_field_metadata: Whether to include field metadata in response
+
+    Returns:
+        Dict with normalized records and optionally field metadata
+    """
+    fields = table_schema.get("fields", [])
+
+    # Normalize each record
+    normalized_records = []
+    for record in records:
+        normalized = {
+            "id": record.get("id"),
+            "createdTime": record.get("createdTime"),
+            "fields": {},
+        }
+
+        # Add existing fields
+        existing_fields = record.get("fields", {})
+
+        # Add all fields from schema, using empty values for missing ones
+        for field in fields:
+            field_name = field["name"]
+            field_type = field["type"]
+
+            if field_name in existing_fields:
+                # Field exists, use its value
+                normalized["fields"][field_name] = existing_fields[field_name]
+            else:
+                # Field is missing, add appropriate empty value
+                normalized["fields"][field_name] = get_empty_value_for_field(field_type)
+
+        normalized_records.append(normalized)
+
+    # Build result dictionary
+    if include_field_metadata:
+        field_metadata = {}
+        for field in fields:
+            metadata = {"type": field["type"], "id": field["id"]}
+
+            # Add type-specific metadata
+            options = field.get("options", {})
+            if field["type"] == "currency" and "symbol" in options:
+                metadata["symbol"] = options["symbol"]
+                metadata["precision"] = options.get("precision", 2)
+            elif field["type"] == "duration" and "durationFormat" in options:
+                metadata["format"] = options["durationFormat"]
+            elif field["type"] == "percent" and "precision" in options:
+                metadata["precision"] = options["precision"]
+            elif (
+                field["type"] in ["singleSelect", "multipleSelects"]
+                and "choices" in options
+            ):
+                metadata["choices"] = [choice["name"] for choice in options["choices"]]
+            elif field["type"] == "rating" and "max" in options:
+                metadata["max"] = options["max"]
+                metadata["icon"] = options.get("icon", "star")
+                metadata["color"] = options.get("color", "yellowBright")
+
+            field_metadata[field["name"]] = metadata
+
+        return {"records": normalized_records, "field_metadata": field_metadata}
+    else:
+        return {"records": normalized_records}
+
+
 async def list_records(
     credentials: Credentials,
     base_id: str,
@@ -1249,3 +1410,26 @@ async def list_bases(
     )
 
     return response.json()
+
+
+async def get_base_tables(
+    credentials: Credentials,
+    base_id: str,
+) -> list[dict]:
+    """
+    Get all tables for a specific base.
+
+    Args:
+        credentials: Airtable API credentials
+        base_id: The ID of the base
+
+    Returns:
+        list[dict]: List of table objects with their schemas
+    """
+    response = await Requests().get(
+        f"https://api.airtable.com/v0/meta/bases/{base_id}/tables",
+        headers={"Authorization": credentials.auth_header()},
+    )
+
+    data = response.json()
+    return data.get("tables", [])

autogpt_platform/backend/backend/blocks/airtable/bases.py

@@ -14,13 +14,13 @@ from backend.sdk import (
     SchemaField,
 )
 
-from ._api import create_base, list_bases
+from ._api import create_base, get_base_tables, list_bases
 from ._config import airtable
 
 
 class AirtableCreateBaseBlock(Block):
     """
-    Creates a new base in an Airtable workspace.
+    Creates a new base in an Airtable workspace, or returns existing base if one with the same name exists.
     """
 
     class Input(BlockSchema):
@@ -31,6 +31,10 @@ class AirtableCreateBaseBlock(Block):
             description="The workspace ID where the base will be created"
         )
         name: str = SchemaField(description="The name of the new base")
+        find_existing: bool = SchemaField(
+            description="If true, return existing base with same name instead of creating duplicate",
+            default=True,
+        )
         tables: list[dict] = SchemaField(
             description="At least one table and field must be specified. Array of table objects to create in the base. Each table should have 'name' and 'fields' properties",
             default=[
@@ -50,14 +54,18 @@ class AirtableCreateBaseBlock(Block):
         )
 
     class Output(BlockSchema):
-        base_id: str = SchemaField(description="The ID of the created base")
+        base_id: str = SchemaField(description="The ID of the created or found base")
         tables: list[dict] = SchemaField(description="Array of table objects")
         table: dict = SchemaField(description="A single table object")
+        was_created: bool = SchemaField(
+            description="True if a new base was created, False if existing was found",
+            default=True,
+        )
 
     def __init__(self):
         super().__init__(
             id="f59b88a8-54ce-4676-a508-fd614b4e8dce",
-            description="Create a new base in Airtable",
+            description="Create or find a base in Airtable",
             categories={BlockCategory.DATA},
             input_schema=self.Input,
             output_schema=self.Output,
@@ -66,6 +74,31 @@ class AirtableCreateBaseBlock(Block):
     async def run(
         self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
     ) -> BlockOutput:
+        # If find_existing is true, check if a base with this name already exists
+        if input_data.find_existing:
+            # List all bases to check for existing one with same name
+            # Note: Airtable API doesn't have a direct search, so we need to list and filter
+            existing_bases = await list_bases(credentials)
+
+            for base in existing_bases.get("bases", []):
+                if base.get("name") == input_data.name:
+                    # Base already exists, return it
+                    base_id = base.get("id")
+                    yield "base_id", base_id
+                    yield "was_created", False
+
+                    # Get the tables for this base
+                    try:
+                        tables = await get_base_tables(credentials, base_id)
+                        yield "tables", tables
+                        for table in tables:
+                            yield "table", table
+                    except Exception:
+                        # If we can't get tables, return empty list
+                        yield "tables", []
+                    return
+
+        # No existing base found or find_existing is false, create new one
         data = await create_base(
             credentials,
             input_data.workspace_id,
@@ -74,6 +107,7 @@ class AirtableCreateBaseBlock(Block):
         )
 
         yield "base_id", data.get("id", None)
+        yield "was_created", True
         yield "tables", data.get("tables", [])
         for table in data.get("tables", []):
             yield "table", table

autogpt_platform/backend/backend/blocks/airtable/records.py

@@ -2,7 +2,7 @@
 Airtable record operation blocks.
 """
 
-from typing import Optional
+from typing import Optional, cast
 
 from backend.sdk import (
     APIKeyCredentials,
@@ -18,7 +18,9 @@ from ._api import (
     create_record,
     delete_multiple_records,
     get_record,
+    get_table_schema,
     list_records,
+    normalize_records,
     update_multiple_records,
 )
 from ._config import airtable
@@ -54,12 +56,24 @@ class AirtableListRecordsBlock(Block):
         return_fields: list[str] = SchemaField(
             description="Specific fields to return (comma-separated)", default=[]
         )
+        normalize_output: bool = SchemaField(
+            description="Normalize output to include all fields with proper empty values (disable to skip schema fetch and get raw Airtable response)",
+            default=True,
+        )
+        include_field_metadata: bool = SchemaField(
+            description="Include field type and configuration metadata (requires normalize_output=true)",
+            default=False,
+        )
 
     class Output(BlockSchema):
         records: list[dict] = SchemaField(description="Array of record objects")
         offset: Optional[str] = SchemaField(
             description="Offset for next page (null if no more records)", default=None
         )
+        field_metadata: Optional[dict] = SchemaField(
+            description="Field type and configuration metadata (only when include_field_metadata=true)",
+            default=None,
+        )
 
     def __init__(self):
         super().__init__(
@@ -73,6 +87,7 @@ class AirtableListRecordsBlock(Block):
     async def run(
         self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
     ) -> BlockOutput:
+
         data = await list_records(
             credentials,
             input_data.base_id,
@@ -88,8 +103,33 @@ class AirtableListRecordsBlock(Block):
             fields=input_data.return_fields if input_data.return_fields else None,
         )
 
-        yield "records", data.get("records", [])
-        yield "offset", data.get("offset", None)
+        records = data.get("records", [])
+
+        # Normalize output if requested
+        if input_data.normalize_output:
+            # Fetch table schema
+            table_schema = await get_table_schema(
+                credentials, input_data.base_id, input_data.table_id_or_name
+            )
+
+            # Normalize the records
+            normalized_data = await normalize_records(
+                records,
+                table_schema,
+                include_field_metadata=input_data.include_field_metadata,
+            )
+
+            yield "records", normalized_data["records"]
+            yield "offset", data.get("offset", None)
+
+            if (
+                input_data.include_field_metadata
+                and "field_metadata" in normalized_data
+            ):
+                yield "field_metadata", normalized_data["field_metadata"]
+        else:
+            yield "records", records
+            yield "offset", data.get("offset", None)
 
 
 class AirtableGetRecordBlock(Block):
@@ -104,11 +144,23 @@ class AirtableGetRecordBlock(Block):
         base_id: str = SchemaField(description="The Airtable base ID")
         table_id_or_name: str = SchemaField(description="Table ID or name")
         record_id: str = SchemaField(description="The record ID to retrieve")
+        normalize_output: bool = SchemaField(
+            description="Normalize output to include all fields with proper empty values (disable to skip schema fetch and get raw Airtable response)",
+            default=True,
+        )
+        include_field_metadata: bool = SchemaField(
+            description="Include field type and configuration metadata (requires normalize_output=true)",
+            default=False,
+        )
 
     class Output(BlockSchema):
         id: str = SchemaField(description="The record ID")
         fields: dict = SchemaField(description="The record fields")
         created_time: str = SchemaField(description="The record created time")
+        field_metadata: Optional[dict] = SchemaField(
+            description="Field type and configuration metadata (only when include_field_metadata=true)",
+            default=None,
+        )
 
     def __init__(self):
         super().__init__(
@@ -122,6 +174,7 @@ class AirtableGetRecordBlock(Block):
     async def run(
         self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
     ) -> BlockOutput:
+
         record = await get_record(
             credentials,
             input_data.base_id,
@@ -129,9 +182,34 @@ class AirtableGetRecordBlock(Block):
             input_data.record_id,
         )
 
-        yield "id", record.get("id", None)
-        yield "fields", record.get("fields", None)
-        yield "created_time", record.get("createdTime", None)
+        # Normalize output if requested
+        if input_data.normalize_output:
+            # Fetch table schema
+            table_schema = await get_table_schema(
+                credentials, input_data.base_id, input_data.table_id_or_name
+            )
+
+            # Normalize the single record (wrap in list and unwrap result)
+            normalized_data = await normalize_records(
+                [record],
+                table_schema,
+                include_field_metadata=input_data.include_field_metadata,
+            )
+
+            normalized_record = normalized_data["records"][0]
+            yield "id", normalized_record.get("id", None)
+            yield "fields", normalized_record.get("fields", None)
+            yield "created_time", normalized_record.get("createdTime", None)
+
+            if (
+                input_data.include_field_metadata
+                and "field_metadata" in normalized_data
+            ):
+                yield "field_metadata", normalized_data["field_metadata"]
+        else:
+            yield "id", record.get("id", None)
+            yield "fields", record.get("fields", None)
+            yield "created_time", record.get("createdTime", None)
 
 
 class AirtableCreateRecordsBlock(Block):
@@ -148,6 +226,10 @@ class AirtableCreateRecordsBlock(Block):
         records: list[dict] = SchemaField(
             description="Array of records to create (each with 'fields' object)"
         )
+        skip_normalization: bool = SchemaField(
+            description="Skip output normalization to get raw Airtable response (faster but may have missing fields)",
+            default=False,
+        )
         typecast: bool = SchemaField(
             description="Automatically convert string values to appropriate types",
             default=False,
@@ -173,7 +255,7 @@ class AirtableCreateRecordsBlock(Block):
     async def run(
         self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
     ) -> BlockOutput:
-        # The create_record API expects records in a specific format
+
         data = await create_record(
             credentials,
             input_data.base_id,
@@ -182,8 +264,22 @@ class AirtableCreateRecordsBlock(Block):
             typecast=input_data.typecast if input_data.typecast else None,
             return_fields_by_field_id=input_data.return_fields_by_field_id,
         )
+        result_records = cast(list[dict], data.get("records", []))
 
-        yield "records", data.get("records", [])
+        # Normalize output unless explicitly disabled
+        if not input_data.skip_normalization and result_records:
+            # Fetch table schema
+            table_schema = await get_table_schema(
+                credentials, input_data.base_id, input_data.table_id_or_name
+            )
+
+            # Normalize the records
+            normalized_data = await normalize_records(
+                result_records, table_schema, include_field_metadata=False
+            )
+            result_records = normalized_data["records"]
+
+        yield "records", result_records
         details = data.get("details", None)
         if details:
             yield "details", details

autogpt_platform/backend/backend/blocks/bannerbear/__init__.py

@@ -0,0 +1,3 @@
+from .text_overlay import BannerbearTextOverlayBlock
+
+__all__ = ["BannerbearTextOverlayBlock"]

autogpt_platform/backend/backend/blocks/bannerbear/_config.py

@@ -0,0 +1,8 @@
+from backend.sdk import BlockCostType, ProviderBuilder
+
+bannerbear = (
+    ProviderBuilder("bannerbear")
+    .with_api_key("BANNERBEAR_API_KEY", "Bannerbear API Key")
+    .with_base_cost(1, BlockCostType.RUN)
+    .build()
+)

autogpt_platform/backend/backend/blocks/bannerbear/text_overlay.py

@@ -0,0 +1,239 @@
+import uuid
+from typing import TYPE_CHECKING, Any, Dict, List
+
+if TYPE_CHECKING:
+    pass
+
+from pydantic import SecretStr
+
+from backend.sdk import (
+    APIKeyCredentials,
+    Block,
+    BlockCategory,
+    BlockOutput,
+    BlockSchema,
+    CredentialsMetaInput,
+    Requests,
+    SchemaField,
+)
+
+from ._config import bannerbear
+
+TEST_CREDENTIALS = APIKeyCredentials(
+    id="01234567-89ab-cdef-0123-456789abcdef",
+    provider="bannerbear",
+    api_key=SecretStr("mock-bannerbear-api-key"),
+    title="Mock Bannerbear API Key",
+)
+
+
+class TextModification(BlockSchema):
+    name: str = SchemaField(
+        description="The name of the layer to modify in the template"
+    )
+    text: str = SchemaField(description="The text content to add to this layer")
+    color: str = SchemaField(
+        description="Hex color code for the text (e.g., '#FF0000')",
+        default="",
+        advanced=True,
+    )
+    font_family: str = SchemaField(
+        description="Font family to use for the text",
+        default="",
+        advanced=True,
+    )
+    font_size: int = SchemaField(
+        description="Font size in pixels",
+        default=0,
+        advanced=True,
+    )
+    font_weight: str = SchemaField(
+        description="Font weight (e.g., 'bold', 'normal')",
+        default="",
+        advanced=True,
+    )
+    text_align: str = SchemaField(
+        description="Text alignment (left, center, right)",
+        default="",
+        advanced=True,
+    )
+
+
+class BannerbearTextOverlayBlock(Block):
+    class Input(BlockSchema):
+        credentials: CredentialsMetaInput = bannerbear.credentials_field(
+            description="API credentials for Bannerbear"
+        )
+        template_id: str = SchemaField(
+            description="The unique ID of your Bannerbear template"
+        )
+        project_id: str = SchemaField(
+            description="Optional: Project ID (required when using Master API Key)",
+            default="",
+            advanced=True,
+        )
+        text_modifications: List[TextModification] = SchemaField(
+            description="List of text layers to modify in the template"
+        )
+        image_url: str = SchemaField(
+            description="Optional: URL of an image to use in the template",
+            default="",
+            advanced=True,
+        )
+        image_layer_name: str = SchemaField(
+            description="Optional: Name of the image layer in the template",
+            default="photo",
+            advanced=True,
+        )
+        webhook_url: str = SchemaField(
+            description="Optional: URL to receive webhook notification when image is ready",
+            default="",
+            advanced=True,
+        )
+        metadata: str = SchemaField(
+            description="Optional: Custom metadata to attach to the image",
+            default="",
+            advanced=True,
+        )
+
+    class Output(BlockSchema):
+        success: bool = SchemaField(
+            description="Whether the image generation was successfully initiated"
+        )
+        image_url: str = SchemaField(
+            description="URL of the generated image (if synchronous) or placeholder"
+        )
+        uid: str = SchemaField(description="Unique identifier for the generated image")
+        status: str = SchemaField(description="Status of the image generation")
+        error: str = SchemaField(description="Error message if the operation failed")
+
+    def __init__(self):
+        super().__init__(
+            id="c7d3a5c2-05fc-450e-8dce-3b0e04626009",
+            description="Add text overlay to images using Bannerbear templates. Perfect for creating social media graphics, marketing materials, and dynamic image content.",
+            categories={BlockCategory.PRODUCTIVITY, BlockCategory.AI},
+            input_schema=self.Input,
+            output_schema=self.Output,
+            test_input={
+                "template_id": "jJWBKNELpQPvbX5R93Gk",
+                "text_modifications": [
+                    {
+                        "name": "headline",
+                        "text": "Amazing Product Launch!",
+                        "color": "#FF0000",
+                    },
+                    {
+                        "name": "subtitle",
+                        "text": "50% OFF Today Only",
+                    },
+                ],
+                "credentials": {
+                    "provider": "bannerbear",
+                    "id": str(uuid.uuid4()),
+                    "type": "api_key",
+                },
+            },
+            test_output=[
+                ("success", True),
+                ("image_url", "https://cdn.bannerbear.com/test-image.jpg"),
+                ("uid", "test-uid-123"),
+                ("status", "completed"),
+            ],
+            test_mock={
+                "_make_api_request": lambda *args, **kwargs: {
+                    "uid": "test-uid-123",
+                    "status": "completed",
+                    "image_url": "https://cdn.bannerbear.com/test-image.jpg",
+                }
+            },
+            test_credentials=TEST_CREDENTIALS,
+        )
+
+    async def _make_api_request(self, payload: dict, api_key: str) -> dict:
+        """Make the actual API request to Bannerbear. This is separated for easy mocking in tests."""
+        headers = {
+            "Authorization": f"Bearer {api_key}",
+            "Content-Type": "application/json",
+        }
+
+        response = await Requests().post(
+            "https://sync.api.bannerbear.com/v2/images",
+            headers=headers,
+            json=payload,
+        )
+
+        if response.status in [200, 201, 202]:
+            return response.json()
+        else:
+            error_msg = f"API request failed with status {response.status}"
+            if response.text:
+                try:
+                    error_data = response.json()
+                    error_msg = (
+                        f"{error_msg}: {error_data.get('message', response.text)}"
+                    )
+                except Exception:
+                    error_msg = f"{error_msg}: {response.text}"
+            raise Exception(error_msg)
+
+    async def run(
+        self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
+    ) -> BlockOutput:
+        # Build the modifications array
+        modifications = []
+
+        # Add text modifications
+        for text_mod in input_data.text_modifications:
+            mod_data: Dict[str, Any] = {
+                "name": text_mod.name,
+                "text": text_mod.text,
+            }
+
+            # Add optional text styling parameters only if they have values
+            if text_mod.color and text_mod.color.strip():
+                mod_data["color"] = text_mod.color
+            if text_mod.font_family and text_mod.font_family.strip():
+                mod_data["font_family"] = text_mod.font_family
+            if text_mod.font_size and text_mod.font_size > 0:
+                mod_data["font_size"] = text_mod.font_size
+            if text_mod.font_weight and text_mod.font_weight.strip():
+                mod_data["font_weight"] = text_mod.font_weight
+            if text_mod.text_align and text_mod.text_align.strip():
+                mod_data["text_align"] = text_mod.text_align
+
+            modifications.append(mod_data)
+
+        # Add image modification if provided and not empty
+        if input_data.image_url and input_data.image_url.strip():
+            modifications.append(
+                {
+                    "name": input_data.image_layer_name,
+                    "image_url": input_data.image_url,
+                }
+            )
+
+        # Build the request payload - only include non-empty optional fields
+        payload = {
+            "template": input_data.template_id,
+            "modifications": modifications,
+        }
+
+        # Add project_id if provided (required for Master API keys)
+        if input_data.project_id and input_data.project_id.strip():
+            payload["project_id"] = input_data.project_id
+
+        if input_data.webhook_url and input_data.webhook_url.strip():
+            payload["webhook_url"] = input_data.webhook_url
+        if input_data.metadata and input_data.metadata.strip():
+            payload["metadata"] = input_data.metadata
+
+        # Make the API request using the private method
+        data = await self._make_api_request(
+            payload, credentials.api_key.get_secret_value()
+        )
+
+        # Synchronous request - image should be ready
+        yield "success", True
+        yield "image_url", data.get("image_url", "")
+        yield "uid", data.get("uid", "")
+        yield "status", data.get("status", "completed")

autogpt_platform/backend/backend/blocks/code_executor.py

@@ -1,8 +1,10 @@
 from enum import Enum
-from typing import Literal
+from typing import Any, Literal, Optional
 
 from e2b_code_interpreter import AsyncSandbox
-from pydantic import SecretStr
+from e2b_code_interpreter import Result as E2BExecutionResult
+from e2b_code_interpreter.charts import Chart as E2BExecutionResultChart
+from pydantic import BaseModel, JsonValue, SecretStr
 
 from backend.data.block import Block, BlockCategory, BlockOutput, BlockSchema
 from backend.data.model import (
@@ -36,14 +38,135 @@ class ProgrammingLanguage(Enum):
     JAVA = "java"
 
 
-class CodeExecutionBlock(Block):
+class MainCodeExecutionResult(BaseModel):
+    """
+    *Pydantic model mirroring `e2b_code_interpreter.Result`*
+
+    Represents the data to be displayed as a result of executing a cell in a Jupyter notebook.
+    The result is similar to the structure returned by ipython kernel: https://ipython.readthedocs.io/en/stable/development/execution.html#execution-semantics
+
+    The result can contain multiple types of data, such as text, images, plots, etc. Each type of data is represented
+    as a string, and the result can contain multiple types of data. The display calls don't have to have text representation,
+    for the actual result the representation is always present for the result, the other representations are always optional.
+    """  # noqa
+
+    class Chart(BaseModel, E2BExecutionResultChart):
+        pass
+
+    text: Optional[str] = None
+    html: Optional[str] = None
+    markdown: Optional[str] = None
+    svg: Optional[str] = None
+    png: Optional[str] = None
+    jpeg: Optional[str] = None
+    pdf: Optional[str] = None
+    latex: Optional[str] = None
+    json: Optional[JsonValue] = None  # type: ignore (reportIncompatibleMethodOverride)
+    javascript: Optional[str] = None
+    data: Optional[dict] = None
+    chart: Optional[Chart] = None
+    extra: Optional[dict] = None
+    """Extra data that can be included. Not part of the standard types."""
+
+
+class CodeExecutionResult(MainCodeExecutionResult):
+    __doc__ = MainCodeExecutionResult.__doc__
+
+    is_main_result: bool = False
+    """Whether this data is the main result of the cell. Data can be produced by display calls of which can be multiple in a cell."""  # noqa
+
+
+class BaseE2BExecutorMixin:
+    """Shared implementation methods for E2B executor blocks."""
+
+    async def execute_code(
+        self,
+        api_key: str,
+        code: str,
+        language: ProgrammingLanguage,
+        template_id: str = "",
+        setup_commands: Optional[list[str]] = None,
+        timeout: Optional[int] = None,
+        sandbox_id: Optional[str] = None,
+        dispose_sandbox: bool = False,
+    ):
+        """
+        Unified code execution method that handles all three use cases:
+        1. Create new sandbox and execute (ExecuteCodeBlock)
+        2. Create new sandbox, execute, and return sandbox_id (InstantiateCodeSandboxBlock)
+        3. Connect to existing sandbox and execute (ExecuteCodeStepBlock)
+        """  # noqa
+        sandbox = None
+        try:
+            if sandbox_id:
+                # Connect to existing sandbox (ExecuteCodeStepBlock case)
+                sandbox = await AsyncSandbox.connect(
+                    sandbox_id=sandbox_id, api_key=api_key
+                )
+            else:
+                # Create new sandbox (ExecuteCodeBlock/InstantiateCodeSandboxBlock case)
+                sandbox = await AsyncSandbox.create(
+                    api_key=api_key, template=template_id, timeout=timeout
+                )
+                if setup_commands:
+                    for cmd in setup_commands:
+                        await sandbox.commands.run(cmd)
+
+            # Execute the code
+            execution = await sandbox.run_code(
+                code,
+                language=language.value,
+                on_error=lambda e: sandbox.kill(),  # Kill the sandbox on error
+            )
+
+            if execution.error:
+                raise Exception(execution.error)
+
+            results = execution.results
+            text_output = execution.text
+            stdout_logs = "".join(execution.logs.stdout)
+            stderr_logs = "".join(execution.logs.stderr)
+
+            return results, text_output, stdout_logs, stderr_logs, sandbox.sandbox_id
+        finally:
+            # Dispose of sandbox if requested to reduce usage costs
+            if dispose_sandbox and sandbox:
+                await sandbox.kill()
+
+    def process_execution_results(
+        self, results: list[E2BExecutionResult]
+    ) -> tuple[dict[str, Any] | None, list[dict[str, Any]]]:
+        """Process and filter execution results."""
+        # Filter out empty formats and convert to dicts
+        processed_results = [
+            {
+                f: value
+                for f in [*r.formats(), "extra", "is_main_result"]
+                if (value := getattr(r, f, None)) is not None
+            }
+            for r in results
+        ]
+        if main_result := next(
+            (r for r in processed_results if r.get("is_main_result")), None
+        ):
+            # Make main_result a copy we can modify & remove is_main_result
+            (main_result := {**main_result}).pop("is_main_result")
+
+        return main_result, processed_results
+
+
+class ExecuteCodeBlock(Block, BaseE2BExecutorMixin):
     # TODO : Add support to upload and download files
-    # Currently, You can customized the CPU and Memory, only by creating a pre customized sandbox template
+    # NOTE: Currently, you can only customize the CPU and Memory
+    #       by creating a pre customized sandbox template
     class Input(BlockSchema):
         credentials: CredentialsMetaInput[
             Literal[ProviderName.E2B], Literal["api_key"]
         ] = CredentialsField(
-            description="Enter your api key for the E2B Sandbox. You can get it in here - https://e2b.dev/docs",
+            description=(
+                "Enter your API key for the E2B platform. "
+                "You can get it in here - https://e2b.dev/docs"
+            ),
         )
 
         # Todo : Option to run commond in background
@@ -76,6 +199,14 @@ class CodeExecutionBlock(Block):
             description="Execution timeout in seconds", default=300
         )
 
+        dispose_sandbox: bool = SchemaField(
+            description=(
+                "Whether to dispose of the sandbox immediately after execution. "
+                "If disabled, the sandbox will run until its timeout expires."
+            ),
+            default=True,
+        )
+
         template_id: str = SchemaField(
             description=(
                 "You can use an E2B sandbox template by entering its ID here. "
@@ -87,7 +218,16 @@ class CodeExecutionBlock(Block):
         )
 
     class Output(BlockSchema):
-        response: str = SchemaField(description="Response from code execution")
+        main_result: MainCodeExecutionResult = SchemaField(
+            title="Main Result", description="The main result from the code execution"
+        )
+        results: list[CodeExecutionResult] = SchemaField(
+            description="List of results from the code execution"
+        )
+        response: str = SchemaField(
+            title="Main Text Output",
+            description="Text output (if any) of the main execution result",
+        )
         stdout_logs: str = SchemaField(
             description="Standard output logs from execution"
         )
@@ -97,10 +237,10 @@ class CodeExecutionBlock(Block):
     def __init__(self):
         super().__init__(
             id="0b02b072-abe7-11ef-8372-fb5d162dd712",
-            description="Executes code in an isolated sandbox environment with internet access.",
+            description="Executes code in a sandbox environment with internet access.",
             categories={BlockCategory.DEVELOPER_TOOLS},
-            input_schema=CodeExecutionBlock.Input,
-            output_schema=CodeExecutionBlock.Output,
+            input_schema=ExecuteCodeBlock.Input,
+            output_schema=ExecuteCodeBlock.Output,
             test_credentials=TEST_CREDENTIALS,
             test_input={
                 "credentials": TEST_CREDENTIALS_INPUT,
@@ -111,91 +251,59 @@ class CodeExecutionBlock(Block):
                 "template_id": "",
             },
             test_output=[
+                ("results", []),
                 ("response", "Hello World"),
                 ("stdout_logs", "Hello World\n"),
             ],
             test_mock={
-                "execute_code": lambda code, language, setup_commands, timeout, api_key, template_id: (
-                    "Hello World",
-                    "Hello World\n",
-                    "",
+                "execute_code": lambda api_key, code, language, template_id, setup_commands, timeout, dispose_sandbox: (  # noqa
+                    [],  # results
+                    "Hello World",  # text_output
+                    "Hello World\n",  # stdout_logs
+                    "",  # stderr_logs
+                    "sandbox_id",  # sandbox_id
                 ),
             },
         )
 
-    async def execute_code(
-        self,
-        code: str,
-        language: ProgrammingLanguage,
-        setup_commands: list[str],
-        timeout: int,
-        api_key: str,
-        template_id: str,
-    ):
-        try:
-            sandbox = None
-            if template_id:
-                sandbox = await AsyncSandbox.create(
-                    template=template_id, api_key=api_key, timeout=timeout
-                )
-            else:
-                sandbox = await AsyncSandbox.create(api_key=api_key, timeout=timeout)
-
-            if not sandbox:
-                raise Exception("Sandbox not created")
-
-            # Running setup commands
-            for cmd in setup_commands:
-                await sandbox.commands.run(cmd)
-
-            # Executing the code
-            execution = await sandbox.run_code(
-                code,
-                language=language.value,
-                on_error=lambda e: sandbox.kill(),  # Kill the sandbox if there is an error
-            )
-
-            if execution.error:
-                raise Exception(execution.error)
-
-            response = execution.text
-            stdout_logs = "".join(execution.logs.stdout)
-            stderr_logs = "".join(execution.logs.stderr)
-
-            return response, stdout_logs, stderr_logs
-
-        except Exception as e:
-            raise e
-
     async def run(
         self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
     ) -> BlockOutput:
         try:
-            response, stdout_logs, stderr_logs = await self.execute_code(
-                input_data.code,
-                input_data.language,
-                input_data.setup_commands,
-                input_data.timeout,
-                credentials.api_key.get_secret_value(),
-                input_data.template_id,
+            results, text_output, stdout, stderr, _ = await self.execute_code(
+                api_key=credentials.api_key.get_secret_value(),
+                code=input_data.code,
+                language=input_data.language,
+                template_id=input_data.template_id,
+                setup_commands=input_data.setup_commands,
+                timeout=input_data.timeout,
+                dispose_sandbox=input_data.dispose_sandbox,
             )
 
-            if response:
-                yield "response", response
-            if stdout_logs:
-                yield "stdout_logs", stdout_logs
-            if stderr_logs:
-                yield "stderr_logs", stderr_logs
+            # Determine result object shape & filter out empty formats
+            main_result, results = self.process_execution_results(results)
+            if main_result:
+                yield "main_result", main_result
+            yield "results", results
+            if text_output:
+                yield "response", text_output
+            if stdout:
+                yield "stdout_logs", stdout
+            if stderr:
+                yield "stderr_logs", stderr
         except Exception as e:
             yield "error", str(e)
 
 
-class InstantiationBlock(Block):
+class InstantiateCodeSandboxBlock(Block, BaseE2BExecutorMixin):
     class Input(BlockSchema):
         credentials: CredentialsMetaInput[
             Literal[ProviderName.E2B], Literal["api_key"]
         ] = CredentialsField(
-            description="Enter your api key for the E2B Sandbox. You can get it in here - https://e2b.dev/docs",
+            description=(
+                "Enter your API key for the E2B platform. "
+                "You can get it in here - https://e2b.dev/docs"
+            )
         )
 
         # Todo : Option to run commond in background
@@ -240,7 +348,10 @@ class InstantiationBlock(Block):
 
     class Output(BlockSchema):
         sandbox_id: str = SchemaField(description="ID of the sandbox instance")
-        response: str = SchemaField(description="Response from code execution")
+        response: str = SchemaField(
+            title="Text Result",
+            description="Text result (if any) of the setup code execution",
+        )
         stdout_logs: str = SchemaField(
             description="Standard output logs from execution"
         )
@@ -250,10 +361,13 @@ class InstantiationBlock(Block):
     def __init__(self):
         super().__init__(
             id="ff0861c9-1726-4aec-9e5b-bf53f3622112",
-            description="Instantiate an isolated sandbox environment with internet access where to execute code in.",
+            description=(
+                "Instantiate a sandbox environment with internet access "
+                "in which you can execute code with the Execute Code Step block."
+            ),
             categories={BlockCategory.DEVELOPER_TOOLS},
-            input_schema=InstantiationBlock.Input,
-            output_schema=InstantiationBlock.Output,
+            input_schema=InstantiateCodeSandboxBlock.Input,
+            output_schema=InstantiateCodeSandboxBlock.Output,
             test_credentials=TEST_CREDENTIALS,
             test_input={
                 "credentials": TEST_CREDENTIALS_INPUT,
@@ -269,11 +383,12 @@ class InstantiationBlock(Block):
                 ("stdout_logs", "Hello World\n"),
             ],
             test_mock={
-                "execute_code": lambda setup_code, language, setup_commands, timeout, api_key, template_id: (
-                    "sandbox_id",
-                    "Hello World",
-                    "Hello World\n",
-                    "",
+                "execute_code": lambda api_key, code, language, template_id, setup_commands, timeout: (  # noqa
+                    [],  # results
+                    "Hello World",  # text_output
+                    "Hello World\n",  # stdout_logs
+                    "",  # stderr_logs
+                    "sandbox_id",  # sandbox_id
                 ),
             },
         )
@@ -282,78 +397,38 @@ class InstantiationBlock(Block):
         self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
     ) -> BlockOutput:
         try:
-            sandbox_id, response, stdout_logs, stderr_logs = await self.execute_code(
-                input_data.setup_code,
-                input_data.language,
-                input_data.setup_commands,
-                input_data.timeout,
-                credentials.api_key.get_secret_value(),
-                input_data.template_id,
+            _, text_output, stdout, stderr, sandbox_id = await self.execute_code(
+                api_key=credentials.api_key.get_secret_value(),
+                code=input_data.setup_code,
+                language=input_data.language,
+                template_id=input_data.template_id,
+                setup_commands=input_data.setup_commands,
+                timeout=input_data.timeout,
             )
             if sandbox_id:
                 yield "sandbox_id", sandbox_id
             else:
                 yield "error", "Sandbox ID not found"
-            if response:
-                yield "response", response
-            if stdout_logs:
-                yield "stdout_logs", stdout_logs
-            if stderr_logs:
-                yield "stderr_logs", stderr_logs
+
+            if text_output:
+                yield "response", text_output
+            if stdout:
+                yield "stdout_logs", stdout
+            if stderr:
+                yield "stderr_logs", stderr
         except Exception as e:
             yield "error", str(e)
 
-    async def execute_code(
-        self,
-        code: str,
-        language: ProgrammingLanguage,
-        setup_commands: list[str],
-        timeout: int,
-        api_key: str,
-        template_id: str,
-    ):
-        try:
-            sandbox = None
-            if template_id:
-                sandbox = await AsyncSandbox.create(
-                    template=template_id, api_key=api_key, timeout=timeout
-                )
-            else:
-                sandbox = await AsyncSandbox.create(api_key=api_key, timeout=timeout)
 
-            if not sandbox:
-                raise Exception("Sandbox not created")
-
-            # Running setup commands
-            for cmd in setup_commands:
-                await sandbox.commands.run(cmd)
-
-            # Executing the code
-            execution = await sandbox.run_code(
-                code,
-                language=language.value,
-                on_error=lambda e: sandbox.kill(),  # Kill the sandbox if there is an error
-            )
-
-            if execution.error:
-                raise Exception(execution.error)
-
-            response = execution.text
-            stdout_logs = "".join(execution.logs.stdout)
-            stderr_logs = "".join(execution.logs.stderr)
-
-            return sandbox.sandbox_id, response, stdout_logs, stderr_logs
-
-        except Exception as e:
-            raise e
-
-
-class StepExecutionBlock(Block):
+class ExecuteCodeStepBlock(Block, BaseE2BExecutorMixin):
     class Input(BlockSchema):
         credentials: CredentialsMetaInput[
             Literal[ProviderName.E2B], Literal["api_key"]
         ] = CredentialsField(
-            description="Enter your api key for the E2B Sandbox. You can get it in here - https://e2b.dev/docs",
+            description=(
+                "Enter your API key for the E2B platform. "
+                "You can get it in here - https://e2b.dev/docs"
+            ),
         )
 
         sandbox_id: str = SchemaField(
@@ -374,8 +449,22 @@ class StepExecutionBlock(Block):
             advanced=False,
         )
 
+        dispose_sandbox: bool = SchemaField(
+            description="Whether to dispose of the sandbox after executing this code.",
+            default=False,
+        )
+
     class Output(BlockSchema):
-        response: str = SchemaField(description="Response from code execution")
+        main_result: MainCodeExecutionResult = SchemaField(
+            title="Main Result", description="The main result from the code execution"
+        )
+        results: list[CodeExecutionResult] = SchemaField(
+            description="List of results from the code execution"
+        )
+        response: str = SchemaField(
+            title="Main Text Output",
+            description="Text output (if any) of the main execution result",
+        )
         stdout_logs: str = SchemaField(
             description="Standard output logs from execution"
         )
@@ -385,10 +474,10 @@ class StepExecutionBlock(Block):
     def __init__(self):
         super().__init__(
             id="82b59b8e-ea10-4d57-9161-8b169b0adba6",
-            description="Execute code in a previously instantiated sandbox environment.",
+            description="Execute code in a previously instantiated sandbox.",
             categories={BlockCategory.DEVELOPER_TOOLS},
-            input_schema=StepExecutionBlock.Input,
-            output_schema=StepExecutionBlock.Output,
+            input_schema=ExecuteCodeStepBlock.Input,
+            output_schema=ExecuteCodeStepBlock.Output,
             test_credentials=TEST_CREDENTIALS,
             test_input={
                 "credentials": TEST_CREDENTIALS_INPUT,
@@ -397,61 +486,43 @@ class StepExecutionBlock(Block):
                 "language": ProgrammingLanguage.PYTHON.value,
             },
             test_output=[
+                ("results", []),
                 ("response", "Hello World"),
                 ("stdout_logs", "Hello World\n"),
             ],
             test_mock={
-                "execute_step_code": lambda sandbox_id, step_code, language, api_key: (
-                    "Hello World",
-                    "Hello World\n",
-                    "",
+                "execute_code": lambda api_key, code, language, sandbox_id, dispose_sandbox: (  # noqa
+                    [],  # results
+                    "Hello World",  # text_output
+                    "Hello World\n",  # stdout_logs
+                    "",  # stderr_logs
+                    sandbox_id,  # sandbox_id
                 ),
             },
         )
 
-    async def execute_step_code(
-        self,
-        sandbox_id: str,
-        code: str,
-        language: ProgrammingLanguage,
-        api_key: str,
-    ):
-        try:
-            sandbox = await AsyncSandbox.connect(sandbox_id=sandbox_id, api_key=api_key)
-            if not sandbox:
-                raise Exception("Sandbox not found")
-
-            # Executing the code
-            execution = await sandbox.run_code(code, language=language.value)
-
-            if execution.error:
-                raise Exception(execution.error)
-
-            response = execution.text
-            stdout_logs = "".join(execution.logs.stdout)
-            stderr_logs = "".join(execution.logs.stderr)
-
-            return response, stdout_logs, stderr_logs
-
-        except Exception as e:
-            raise e
-
     async def run(
         self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
     ) -> BlockOutput:
         try:
-            response, stdout_logs, stderr_logs = await self.execute_step_code(
-                input_data.sandbox_id,
-                input_data.step_code,
-                input_data.language,
-                credentials.api_key.get_secret_value(),
+            results, text_output, stdout, stderr, _ = await self.execute_code(
+                api_key=credentials.api_key.get_secret_value(),
+                code=input_data.step_code,
+                language=input_data.language,
+                sandbox_id=input_data.sandbox_id,
+                dispose_sandbox=input_data.dispose_sandbox,
             )
 
-            if response:
-                yield "response", response
-            if stdout_logs:
-                yield "stdout_logs", stdout_logs
-            if stderr_logs:
-                yield "stderr_logs", stderr_logs
+            # Determine result object shape & filter out empty formats
+            main_result, results = self.process_execution_results(results)
+            if main_result:
+                yield "main_result", main_result
+            yield "results", results
+            if text_output:
+                yield "response", text_output
+            if stdout:
+                yield "stdout_logs", stdout
+            if stderr:
+                yield "stderr_logs", stderr
         except Exception as e:
             yield "error", str(e)

autogpt_platform/backend/backend/blocks/code_extraction_block.py

@@ -90,7 +90,7 @@ class CodeExtractionBlock(Block):
                 for aliases in language_aliases.values()
                 for alias in aliases
             )
-            + r")\s+[\s\S]*?```"
+            + r")[ \t]*\n[\s\S]*?```"
         )
 
         remaining_text = re.sub(pattern, "", input_data.text).strip()
@@ -103,7 +103,9 @@ class CodeExtractionBlock(Block):
         # Escape special regex characters in the language string
         language = re.escape(language)
         # Extract all code blocks enclosed in ```language``` blocks
-        pattern = re.compile(rf"```{language}\s+(.*?)```", re.DOTALL | re.IGNORECASE)
+        pattern = re.compile(
+            rf"```{language}[ \t]*\n(.*?)\n```", re.DOTALL | re.IGNORECASE
+        )
         matches = pattern.finditer(text)
         # Combine all code blocks for this language with newlines between them
         code_blocks = [match.group(1).strip() for match in matches]

autogpt_platform/backend/backend/blocks/data_manipulation.py

@@ -66,6 +66,7 @@ class AddToDictionaryBlock(Block):
         dictionary: dict[Any, Any] = SchemaField(
             default_factory=dict,
             description="The dictionary to add the entry to. If not provided, a new dictionary will be created.",
+            advanced=False,
         )
         key: str = SchemaField(
             default="",

autogpt_platform/backend/backend/blocks/dataforseo/_api.py

@@ -113,6 +113,7 @@ class DataForSeoClient:
         include_serp_info: bool = False,
         include_clickstream_data: bool = False,
         limit: int = 100,
+        depth: Optional[int] = None,
     ) -> List[Dict[str, Any]]:
         """
         Get related keywords from DataForSEO Labs.
@@ -125,6 +126,7 @@ class DataForSeoClient:
             include_serp_info: Include SERP data
             include_clickstream_data: Include clickstream metrics
             limit: Maximum number of results (up to 3000)
+            depth: Keyword search depth (0-4), controls number of returned keywords
 
         Returns:
             API response with related keywords
@@ -148,6 +150,8 @@ class DataForSeoClient:
             task_data["include_clickstream_data"] = include_clickstream_data
         if limit is not None:
             task_data["limit"] = limit
+        if depth is not None:
+            task_data["depth"] = depth
 
         payload = [task_data]
 

autogpt_platform/backend/backend/blocks/dataforseo/keyword_suggestions.py

@@ -90,6 +90,7 @@ class DataForSeoKeywordSuggestionsBlock(Block):
         seed_keyword: str = SchemaField(
             description="The seed keyword used for the query"
         )
+        error: str = SchemaField(description="Error message if the API call failed")
 
     def __init__(self):
         super().__init__(
@@ -161,43 +162,52 @@ class DataForSeoKeywordSuggestionsBlock(Block):
         **kwargs,
     ) -> BlockOutput:
         """Execute the keyword suggestions query."""
-        client = DataForSeoClient(credentials)
+        try:
+            client = DataForSeoClient(credentials)
 
-        results = await self._fetch_keyword_suggestions(client, input_data)
+            results = await self._fetch_keyword_suggestions(client, input_data)
 
-        # Process and format the results
-        suggestions = []
-        if results and len(results) > 0:
-            # results is a list, get the first element
-            first_result = results[0] if isinstance(results, list) else results
-            items = (
-                first_result.get("items", []) if isinstance(first_result, dict) else []
-            )
-            for item in items:
-                # Create the KeywordSuggestion object
-                suggestion = KeywordSuggestion(
-                    keyword=item.get("keyword", ""),
-                    search_volume=item.get("keyword_info", {}).get("search_volume"),
-                    competition=item.get("keyword_info", {}).get("competition"),
-                    cpc=item.get("keyword_info", {}).get("cpc"),
-                    keyword_difficulty=item.get("keyword_properties", {}).get(
-                        "keyword_difficulty"
-                    ),
-                    serp_info=(
-                        item.get("serp_info") if input_data.include_serp_info else None
-                    ),
-                    clickstream_data=(
-                        item.get("clickstream_keyword_info")
-                        if input_data.include_clickstream_data
-                        else None
-                    ),
+            # Process and format the results
+            suggestions = []
+            if results and len(results) > 0:
+                # results is a list, get the first element
+                first_result = results[0] if isinstance(results, list) else results
+                items = (
+                    first_result.get("items", [])
+                    if isinstance(first_result, dict)
+                    else []
                 )
-                yield "suggestion", suggestion
-                suggestions.append(suggestion)
+                if items is None:
+                    items = []
+                for item in items:
+                    # Create the KeywordSuggestion object
+                    suggestion = KeywordSuggestion(
+                        keyword=item.get("keyword", ""),
+                        search_volume=item.get("keyword_info", {}).get("search_volume"),
+                        competition=item.get("keyword_info", {}).get("competition"),
+                        cpc=item.get("keyword_info", {}).get("cpc"),
+                        keyword_difficulty=item.get("keyword_properties", {}).get(
+                            "keyword_difficulty"
+                        ),
+                        serp_info=(
+                            item.get("serp_info")
+                            if input_data.include_serp_info
+                            else None
+                        ),
+                        clickstream_data=(
+                            item.get("clickstream_keyword_info")
+                            if input_data.include_clickstream_data
+                            else None
+                        ),
+                    )
+                    yield "suggestion", suggestion
+                    suggestions.append(suggestion)
 
-        yield "suggestions", suggestions
-        yield "total_count", len(suggestions)
-        yield "seed_keyword", input_data.keyword
+            yield "suggestions", suggestions
+            yield "total_count", len(suggestions)
+            yield "seed_keyword", input_data.keyword
+        except Exception as e:
+            yield "error", f"Failed to fetch keyword suggestions: {str(e)}"
 
 
 class KeywordSuggestionExtractorBlock(Block):

autogpt_platform/backend/backend/blocks/dataforseo/related_keywords.py

@@ -78,6 +78,12 @@ class DataForSeoRelatedKeywordsBlock(Block):
             ge=1,
             le=3000,
         )
+        depth: int = SchemaField(
+            description="Keyword search depth (0-4). Controls the number of returned keywords: 0=1 keyword, 1=~8 keywords, 2=~72 keywords, 3=~584 keywords, 4=~4680 keywords",
+            default=1,
+            ge=0,
+            le=4,
+        )
 
     class Output(BlockSchema):
         related_keywords: List[RelatedKeyword] = SchemaField(
@@ -92,6 +98,7 @@ class DataForSeoRelatedKeywordsBlock(Block):
         seed_keyword: str = SchemaField(
             description="The seed keyword used for the query"
         )
+        error: str = SchemaField(description="Error message if the API call failed")
 
     def __init__(self):
         super().__init__(
@@ -154,6 +161,7 @@ class DataForSeoRelatedKeywordsBlock(Block):
             include_serp_info=input_data.include_serp_info,
             include_clickstream_data=input_data.include_clickstream_data,
             limit=input_data.limit,
+            depth=input_data.depth,
         )
 
     async def run(
@@ -164,50 +172,60 @@ class DataForSeoRelatedKeywordsBlock(Block):
         **kwargs,
     ) -> BlockOutput:
         """Execute the related keywords query."""
-        client = DataForSeoClient(credentials)
+        try:
+            client = DataForSeoClient(credentials)
 
-        results = await self._fetch_related_keywords(client, input_data)
+            results = await self._fetch_related_keywords(client, input_data)
 
-        # Process and format the results
-        related_keywords = []
-        if results and len(results) > 0:
-            # results is a list, get the first element
-            first_result = results[0] if isinstance(results, list) else results
-            items = (
-                first_result.get("items", []) if isinstance(first_result, dict) else []
-            )
-            for item in items:
-                # Extract keyword_data from the item
-                keyword_data = item.get("keyword_data", {})
-
-                # Create the RelatedKeyword object
-                keyword = RelatedKeyword(
-                    keyword=keyword_data.get("keyword", ""),
-                    search_volume=keyword_data.get("keyword_info", {}).get(
-                        "search_volume"
-                    ),
-                    competition=keyword_data.get("keyword_info", {}).get("competition"),
-                    cpc=keyword_data.get("keyword_info", {}).get("cpc"),
-                    keyword_difficulty=keyword_data.get("keyword_properties", {}).get(
-                        "keyword_difficulty"
-                    ),
-                    serp_info=(
-                        keyword_data.get("serp_info")
-                        if input_data.include_serp_info
-                        else None
-                    ),
-                    clickstream_data=(
-                        keyword_data.get("clickstream_keyword_info")
-                        if input_data.include_clickstream_data
-                        else None
-                    ),
+            # Process and format the results
+            related_keywords = []
+            if results and len(results) > 0:
+                # results is a list, get the first element
+                first_result = results[0] if isinstance(results, list) else results
+                items = (
+                    first_result.get("items", [])
+                    if isinstance(first_result, dict)
+                    else []
                 )
-                yield "related_keyword", keyword
-                related_keywords.append(keyword)
+                # Ensure items is never None
+                if items is None:
+                    items = []
+                for item in items:
+                    # Extract keyword_data from the item
+                    keyword_data = item.get("keyword_data", {})
 
-        yield "related_keywords", related_keywords
-        yield "total_count", len(related_keywords)
-        yield "seed_keyword", input_data.keyword
+                    # Create the RelatedKeyword object
+                    keyword = RelatedKeyword(
+                        keyword=keyword_data.get("keyword", ""),
+                        search_volume=keyword_data.get("keyword_info", {}).get(
+                            "search_volume"
+                        ),
+                        competition=keyword_data.get("keyword_info", {}).get(
+                            "competition"
+                        ),
+                        cpc=keyword_data.get("keyword_info", {}).get("cpc"),
+                        keyword_difficulty=keyword_data.get(
+                            "keyword_properties", {}
+                        ).get("keyword_difficulty"),
+                        serp_info=(
+                            keyword_data.get("serp_info")
+                            if input_data.include_serp_info
+                            else None
+                        ),
+                        clickstream_data=(
+                            keyword_data.get("clickstream_keyword_info")
+                            if input_data.include_clickstream_data
+                            else None
+                        ),
+                    )
+                    yield "related_keyword", keyword
+                    related_keywords.append(keyword)
+
+            yield "related_keywords", related_keywords
+            yield "total_count", len(related_keywords)
+            yield "seed_keyword", input_data.keyword
+        except Exception as e:
+            yield "error", f"Failed to fetch related keywords: {str(e)}"
 
 
 class RelatedKeywordExtractorBlock(Block):

autogpt_platform/backend/backend/blocks/discord/bot_blocks.py

@@ -4,13 +4,13 @@ import mimetypes
 from pathlib import Path
 from typing import Any
 
-import aiohttp
 import discord
 from pydantic import SecretStr
 
 from backend.data.block import Block, BlockCategory, BlockOutput, BlockSchema
 from backend.data.model import APIKeyCredentials, SchemaField
 from backend.util.file import store_media_file
+from backend.util.request import Requests
 from backend.util.type import MediaFileType
 
 from ._auth import (
@@ -114,10 +114,9 @@ class ReadDiscordMessagesBlock(Block):
             if message.attachments:
                 attachment = message.attachments[0]  # Process the first attachment
                 if attachment.filename.endswith((".txt", ".py")):
-                    async with aiohttp.ClientSession() as session:
-                        async with session.get(attachment.url) as response:
-                            file_content = response.text()
-                            self.output_data += f"\n\nFile from user: {attachment.filename}\nContent: {file_content}"
+                    response = await Requests().get(attachment.url)
+                    file_content = response.text()
+                    self.output_data += f"\n\nFile from user: {attachment.filename}\nContent: {file_content}"
 
             await client.close()
 
@@ -171,11 +170,11 @@ class SendDiscordMessageBlock(Block):
             description="The content of the message to send"
         )
         channel_name: str = SchemaField(
-            description="The name of the channel the message will be sent to"
+            description="Channel ID or channel name to send the message to"
         )
         server_name: str = SchemaField(
-            description="The name of the server where the channel is located",
-            advanced=True,  # Optional field for server name
+            description="Server name (only needed if using channel name)",
+            advanced=True,
             default="",
         )
 
@@ -231,25 +230,49 @@ class SendDiscordMessageBlock(Block):
         @client.event
         async def on_ready():
             print(f"Logged in as {client.user}")
-            for guild in client.guilds:
-                if server_name and guild.name != server_name:
-                    continue
-                for channel in guild.text_channels:
-                    if channel.name == channel_name:
-                        # Split message into chunks if it exceeds 2000 characters
-                        chunks = self.chunk_message(message_content)
-                        last_message = None
-                        for chunk in chunks:
-                            last_message = await channel.send(chunk)
-                        result["status"] = "Message sent"
-                        result["message_id"] = (
-                            str(last_message.id) if last_message else ""
-                        )
-                        result["channel_id"] = str(channel.id)
-                        await client.close()
-                        return
+            channel = None
 
-            result["status"] = "Channel not found"
+            # Try to parse as channel ID first
+            try:
+                channel_id = int(channel_name)
+                channel = client.get_channel(channel_id)
+            except ValueError:
+                # Not a valid ID, will try name lookup
+                pass
+
+            # If not found by ID (or not an ID), try name lookup
+            if not channel:
+                for guild in client.guilds:
+                    if server_name and guild.name != server_name:
+                        continue
+                    for ch in guild.text_channels:
+                        if ch.name == channel_name:
+                            channel = ch
+                            break
+                    if channel:
+                        break
+
+            if not channel:
+                result["status"] = f"Channel not found: {channel_name}"
+                await client.close()
+                return
+
+            # Type check - ensure it's a text channel that can send messages
+            if not hasattr(channel, "send"):
+                result["status"] = (
+                    f"Channel {channel_name} cannot receive messages (not a text channel)"
+                )
+                await client.close()
+                return
+
+            # Split message into chunks if it exceeds 2000 characters
+            chunks = self.chunk_message(message_content)
+            last_message = None
+            for chunk in chunks:
+                last_message = await channel.send(chunk)  # type: ignore
+            result["status"] = "Message sent"
+            result["message_id"] = str(last_message.id) if last_message else ""
+            result["channel_id"] = str(channel.id)
             await client.close()
 
         await client.start(token)
@@ -675,16 +698,15 @@ class SendDiscordFileBlock(Block):
 
                 elif file.startswith(("http://", "https://")):
                     # URL - download the file
-                    async with aiohttp.ClientSession() as session:
-                        async with session.get(file) as response:
-                            file_bytes = await response.read()
+                    response = await Requests().get(file)
+                    file_bytes = response.content
 
-                            # Try to get filename from URL if not provided
-                            if not filename:
-                                from urllib.parse import urlparse
+                    # Try to get filename from URL if not provided
+                    if not filename:
+                        from urllib.parse import urlparse
 
-                                path = urlparse(file).path
-                                detected_filename = Path(path).name or "download"
+                        path = urlparse(file).path
+                        detected_filename = Path(path).name or "download"
                 else:
                     # Local file path - read from stored media file
                     # This would be a path from a previous block's output

autogpt_platform/backend/backend/blocks/exa/websets.py

@@ -93,11 +93,11 @@ class Webset(BaseModel):
     """
     Set of key-value pairs you want to associate with this object.
     """
-    created_at: Annotated[datetime, Field(alias="createdAt")] | None = None
+    created_at: Annotated[datetime | None, Field(alias="createdAt")] = None
     """
     The date and time the webset was created
     """
-    updated_at: Annotated[datetime, Field(alias="updatedAt")] | None = None
+    updated_at: Annotated[datetime | None, Field(alias="updatedAt")] = None
     """
     The date and time the webset was last updated
     """

autogpt_platform/backend/backend/blocks/firecrawl/_api.py

@@ -0,0 +1,12 @@
+from enum import Enum
+
+
+class ScrapeFormat(Enum):
+    MARKDOWN = "markdown"
+    HTML = "html"
+    RAW_HTML = "rawHtml"
+    LINKS = "links"
+    SCREENSHOT = "screenshot"
+    SCREENSHOT_FULL_PAGE = "screenshot@fullPage"
+    JSON = "json"
+    CHANGE_TRACKING = "changeTracking"

autogpt_platform/backend/backend/blocks/firecrawl/_format_utils.py

@@ -0,0 +1,28 @@
+"""Utility functions for converting between our ScrapeFormat enum and firecrawl FormatOption types."""
+
+from typing import List
+
+from firecrawl.v2.types import FormatOption, ScreenshotFormat
+
+from backend.blocks.firecrawl._api import ScrapeFormat
+
+
+def convert_to_format_options(
+    formats: List[ScrapeFormat],
+) -> List[FormatOption]:
+    """Convert our ScrapeFormat enum values to firecrawl FormatOption types.
+
+    Handles special cases like screenshot@fullPage which needs to be converted
+    to a ScreenshotFormat object.
+    """
+    result: List[FormatOption] = []
+
+    for format_enum in formats:
+        if format_enum.value == "screenshot@fullPage":
+            # Special case: convert to ScreenshotFormat with full_page=True
+            result.append(ScreenshotFormat(type="screenshot", full_page=True))
+        else:
+            # Regular string literals
+            result.append(format_enum.value)
+
+    return result

autogpt_platform/backend/backend/blocks/firecrawl/crawl.py

@@ -1,8 +1,9 @@
-from enum import Enum
 from typing import Any
 
-from firecrawl import FirecrawlApp, ScrapeOptions
+from firecrawl import FirecrawlApp
+from firecrawl.v2.types import ScrapeOptions
 
+from backend.blocks.firecrawl._api import ScrapeFormat
 from backend.sdk import (
     APIKeyCredentials,
     Block,
@@ -14,21 +15,10 @@ from backend.sdk import (
 )
 
 from ._config import firecrawl
-
-
-class ScrapeFormat(Enum):
-    MARKDOWN = "markdown"
-    HTML = "html"
-    RAW_HTML = "rawHtml"
-    LINKS = "links"
-    SCREENSHOT = "screenshot"
-    SCREENSHOT_FULL_PAGE = "screenshot@fullPage"
-    JSON = "json"
-    CHANGE_TRACKING = "changeTracking"
+from ._format_utils import convert_to_format_options
 
 
 class FirecrawlCrawlBlock(Block):
-
     class Input(BlockSchema):
         credentials: CredentialsMetaInput = firecrawl.credentials_field()
         url: str = SchemaField(description="The URL to crawl")
@@ -78,18 +68,17 @@ class FirecrawlCrawlBlock(Block):
     async def run(
         self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
     ) -> BlockOutput:
-
         app = FirecrawlApp(api_key=credentials.api_key.get_secret_value())
 
         # Sync call
-        crawl_result = app.crawl_url(
+        crawl_result = app.crawl(
             input_data.url,
             limit=input_data.limit,
             scrape_options=ScrapeOptions(
-                formats=[format.value for format in input_data.formats],
-                onlyMainContent=input_data.only_main_content,
-                maxAge=input_data.max_age,
-                waitFor=input_data.wait_for,
+                formats=convert_to_format_options(input_data.formats),
+                only_main_content=input_data.only_main_content,
+                max_age=input_data.max_age,
+                wait_for=input_data.wait_for,
             ),
         )
         yield "data", crawl_result.data
@@ -101,7 +90,7 @@ class FirecrawlCrawlBlock(Block):
                 elif f == ScrapeFormat.HTML:
                     yield "html", data.html
                 elif f == ScrapeFormat.RAW_HTML:
-                    yield "raw_html", data.rawHtml
+                    yield "raw_html", data.raw_html
                 elif f == ScrapeFormat.LINKS:
                     yield "links", data.links
                 elif f == ScrapeFormat.SCREENSHOT:
@@ -109,6 +98,6 @@ class FirecrawlCrawlBlock(Block):
                 elif f == ScrapeFormat.SCREENSHOT_FULL_PAGE:
                     yield "screenshot_full_page", data.screenshot
                 elif f == ScrapeFormat.CHANGE_TRACKING:
-                    yield "change_tracking", data.changeTracking
+                    yield "change_tracking", data.change_tracking
                 elif f == ScrapeFormat.JSON:
                     yield "json", data.json

autogpt_platform/backend/backend/blocks/firecrawl/extract.py

@@ -20,7 +20,6 @@ from ._config import firecrawl
 
 @cost(BlockCost(2, BlockCostType.RUN))
 class FirecrawlExtractBlock(Block):
-
     class Input(BlockSchema):
         credentials: CredentialsMetaInput = firecrawl.credentials_field()
         urls: list[str] = SchemaField(
@@ -53,7 +52,6 @@ class FirecrawlExtractBlock(Block):
     async def run(
         self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
     ) -> BlockOutput:
-
         app = FirecrawlApp(api_key=credentials.api_key.get_secret_value())
 
         extract_result = app.extract(

autogpt_platform/backend/backend/blocks/firecrawl/map.py

@@ -1,3 +1,5 @@
+from typing import Any
+
 from firecrawl import FirecrawlApp
 
 from backend.sdk import (
@@ -14,14 +16,16 @@ from ._config import firecrawl
 
 
 class FirecrawlMapWebsiteBlock(Block):
-
     class Input(BlockSchema):
         credentials: CredentialsMetaInput = firecrawl.credentials_field()
 
         url: str = SchemaField(description="The website url to map")
 
     class Output(BlockSchema):
-        links: list[str] = SchemaField(description="The links of the website")
+        links: list[str] = SchemaField(description="List of URLs found on the website")
+        results: list[dict[str, Any]] = SchemaField(
+            description="List of search results with url, title, and description"
+        )
 
     def __init__(self):
         super().__init__(
@@ -35,12 +39,22 @@ class FirecrawlMapWebsiteBlock(Block):
     async def run(
         self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
     ) -> BlockOutput:
-
         app = FirecrawlApp(api_key=credentials.api_key.get_secret_value())
 
         # Sync call
-        map_result = app.map_url(
+        map_result = app.map(
             url=input_data.url,
         )
 
-        yield "links", map_result.links
+        # Convert SearchResult objects to dicts
+        results_data = [
+            {
+                "url": link.url,
+                "title": link.title,
+                "description": link.description,
+            }
+            for link in map_result.links
+        ]
+
+        yield "links", [link.url for link in map_result.links]
+        yield "results", results_data

autogpt_platform/backend/backend/blocks/firecrawl/scrape.py

@@ -1,8 +1,8 @@
-from enum import Enum
 from typing import Any
 
 from firecrawl import FirecrawlApp
 
+from backend.blocks.firecrawl._api import ScrapeFormat
 from backend.sdk import (
     APIKeyCredentials,
     Block,
@@ -14,21 +14,10 @@ from backend.sdk import (
 )
 
 from ._config import firecrawl
-
-
-class ScrapeFormat(Enum):
-    MARKDOWN = "markdown"
-    HTML = "html"
-    RAW_HTML = "rawHtml"
-    LINKS = "links"
-    SCREENSHOT = "screenshot"
-    SCREENSHOT_FULL_PAGE = "screenshot@fullPage"
-    JSON = "json"
-    CHANGE_TRACKING = "changeTracking"
+from ._format_utils import convert_to_format_options
 
 
 class FirecrawlScrapeBlock(Block):
-
     class Input(BlockSchema):
         credentials: CredentialsMetaInput = firecrawl.credentials_field()
         url: str = SchemaField(description="The URL to crawl")
@@ -78,12 +67,11 @@ class FirecrawlScrapeBlock(Block):
     async def run(
         self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
     ) -> BlockOutput:
-
         app = FirecrawlApp(api_key=credentials.api_key.get_secret_value())
 
-        scrape_result = app.scrape_url(
+        scrape_result = app.scrape(
             input_data.url,
-            formats=[format.value for format in input_data.formats],
+            formats=convert_to_format_options(input_data.formats),
             only_main_content=input_data.only_main_content,
             max_age=input_data.max_age,
             wait_for=input_data.wait_for,
@@ -96,7 +84,7 @@ class FirecrawlScrapeBlock(Block):
             elif f == ScrapeFormat.HTML:
                 yield "html", scrape_result.html
             elif f == ScrapeFormat.RAW_HTML:
-                yield "raw_html", scrape_result.rawHtml
+                yield "raw_html", scrape_result.raw_html
             elif f == ScrapeFormat.LINKS:
                 yield "links", scrape_result.links
             elif f == ScrapeFormat.SCREENSHOT:
@@ -104,6 +92,6 @@ class FirecrawlScrapeBlock(Block):
             elif f == ScrapeFormat.SCREENSHOT_FULL_PAGE:
                 yield "screenshot_full_page", scrape_result.screenshot
             elif f == ScrapeFormat.CHANGE_TRACKING:
-                yield "change_tracking", scrape_result.changeTracking
+                yield "change_tracking", scrape_result.change_tracking
             elif f == ScrapeFormat.JSON:
                 yield "json", scrape_result.json

autogpt_platform/backend/backend/blocks/firecrawl/search.py

@@ -1,8 +1,9 @@
-from enum import Enum
 from typing import Any
 
-from firecrawl import FirecrawlApp, ScrapeOptions
+from firecrawl import FirecrawlApp
+from firecrawl.v2.types import ScrapeOptions
 
+from backend.blocks.firecrawl._api import ScrapeFormat
 from backend.sdk import (
     APIKeyCredentials,
     Block,
@@ -14,21 +15,10 @@ from backend.sdk import (
 )
 
 from ._config import firecrawl
-
-
-class ScrapeFormat(Enum):
-    MARKDOWN = "markdown"
-    HTML = "html"
-    RAW_HTML = "rawHtml"
-    LINKS = "links"
-    SCREENSHOT = "screenshot"
-    SCREENSHOT_FULL_PAGE = "screenshot@fullPage"
-    JSON = "json"
-    CHANGE_TRACKING = "changeTracking"
+from ._format_utils import convert_to_format_options
 
 
 class FirecrawlSearchBlock(Block):
-
     class Input(BlockSchema):
         credentials: CredentialsMetaInput = firecrawl.credentials_field()
         query: str = SchemaField(description="The query to search for")
@@ -61,7 +51,6 @@ class FirecrawlSearchBlock(Block):
     async def run(
         self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
     ) -> BlockOutput:
-
         app = FirecrawlApp(api_key=credentials.api_key.get_secret_value())
 
         # Sync call
@@ -69,11 +58,12 @@ class FirecrawlSearchBlock(Block):
             input_data.query,
             limit=input_data.limit,
             scrape_options=ScrapeOptions(
-                formats=[format.value for format in input_data.formats],
-                maxAge=input_data.max_age,
-                waitFor=input_data.wait_for,
+                formats=convert_to_format_options(input_data.formats) or None,
+                max_age=input_data.max_age,
+                wait_for=input_data.wait_for,
             ),
         )
         yield "data", scrape_result
-        for site in scrape_result.data:
-            yield "site", site
+        if hasattr(scrape_result, "web") and scrape_result.web:
+            for site in scrape_result.web:
+                yield "site", site

autogpt_platform/backend/backend/blocks/google/gmail.py

@@ -39,18 +39,6 @@ def serialize_email_recipients(recipients: list[str]) -> str:
     return ", ".join(recipients)
 
 
-def deduplicate_email_addresses(addresses: list[str]) -> list[str]:
-    """Deduplicate email addresses while preserving order.
-
-    Args:
-        addresses: List of email addresses that may contain duplicates or None values
-
-    Returns:
-        List of unique email addresses with None values filtered out
-    """
-    return list(dict.fromkeys(filter(None, addresses)))
-
-
 def _make_mime_text(
     body: str,
     content_type: Optional[Literal["auto", "plain", "html"]] = None,
@@ -1106,6 +1094,117 @@ class GmailGetThreadBlock(GmailBase):
         return thread
 
 
+async def _build_reply_message(
+    service, input_data, graph_exec_id: str, user_id: str
+) -> tuple[str, str]:
+    """
+    Builds a reply MIME message for Gmail threads.
+
+    Returns:
+        tuple: (base64-encoded raw message, threadId)
+    """
+    # Get parent message for reply context
+    parent = await asyncio.to_thread(
+        lambda: service.users()
+        .messages()
+        .get(
+            userId="me",
+            id=input_data.parentMessageId,
+            format="metadata",
+            metadataHeaders=[
+                "Subject",
+                "References",
+                "Message-ID",
+                "From",
+                "To",
+                "Cc",
+                "Reply-To",
+            ],
+        )
+        .execute()
+    )
+
+    # Build headers dictionary, preserving all values for duplicate headers
+    headers = {}
+    for h in parent.get("payload", {}).get("headers", []):
+        name = h["name"].lower()
+        value = h["value"]
+        if name in headers:
+            # For duplicate headers, keep the first occurrence (most relevant for reply context)
+            continue
+        headers[name] = value
+
+    # Determine recipients if not specified
+    if not (input_data.to or input_data.cc or input_data.bcc):
+        if input_data.replyAll:
+            recipients = [parseaddr(headers.get("from", ""))[1]]
+            recipients += [addr for _, addr in getaddresses([headers.get("to", "")])]
+            recipients += [addr for _, addr in getaddresses([headers.get("cc", "")])]
+            # Use dict.fromkeys() for O(n) deduplication while preserving order
+            input_data.to = list(dict.fromkeys(filter(None, recipients)))
+        else:
+            # Check Reply-To header first, fall back to From header
+            reply_to = headers.get("reply-to", "")
+            from_addr = headers.get("from", "")
+            sender = parseaddr(reply_to if reply_to else from_addr)[1]
+            input_data.to = [sender] if sender else []
+
+    # Set subject with Re: prefix if not already present
+    if input_data.subject:
+        subject = input_data.subject
+    else:
+        parent_subject = headers.get("subject", "").strip()
+        # Only add "Re:" if not already present (case-insensitive check)
+        if parent_subject.lower().startswith("re:"):
+            subject = parent_subject
+        else:
+            subject = f"Re: {parent_subject}" if parent_subject else "Re:"
+
+    # Build references header for proper threading
+    references = headers.get("references", "").split()
+    if headers.get("message-id"):
+        references.append(headers["message-id"])
+
+    # Create MIME message
+    msg = MIMEMultipart()
+    if input_data.to:
+        msg["To"] = ", ".join(input_data.to)
+    if input_data.cc:
+        msg["Cc"] = ", ".join(input_data.cc)
+    if input_data.bcc:
+        msg["Bcc"] = ", ".join(input_data.bcc)
+    msg["Subject"] = subject
+    if headers.get("message-id"):
+        msg["In-Reply-To"] = headers["message-id"]
+    if references:
+        msg["References"] = " ".join(references)
+
+    # Use the helper function for consistent content type handling
+    msg.attach(_make_mime_text(input_data.body, input_data.content_type))
+
+    # Handle attachments
+    for attach in input_data.attachments:
+        local_path = await store_media_file(
+            user_id=user_id,
+            graph_exec_id=graph_exec_id,
+            file=attach,
+            return_content=False,
+        )
+        abs_path = get_exec_file_path(graph_exec_id, local_path)
+        part = MIMEBase("application", "octet-stream")
+        with open(abs_path, "rb") as f:
+            part.set_payload(f.read())
+        encoders.encode_base64(part)
+        part.add_header(
+            "Content-Disposition", f"attachment; filename={Path(abs_path).name}"
+        )
+        msg.attach(part)
+
+    # Encode message
+    raw = base64.urlsafe_b64encode(msg.as_bytes()).decode("utf-8")
+    return raw, input_data.threadId
+
+
 class GmailReplyBlock(GmailBase):
     """
     Replies to Gmail threads with intelligent content type detection.
@@ -1242,102 +1341,31 @@ class GmailReplyBlock(GmailBase):
     async def _reply(
         self, service, input_data: Input, graph_exec_id: str, user_id: str
     ) -> dict:
-        parent = await asyncio.to_thread(
-            lambda: service.users()
-            .messages()
-            .get(
-                userId="me",
-                id=input_data.parentMessageId,
-                format="metadata",
-                metadataHeaders=[
-                    "Subject",
-                    "References",
-                    "Message-ID",
-                    "From",
-                    "To",
-                    "Cc",
-                    "Reply-To",
-                ],
-            )
-            .execute()
+        # Build the reply message using the shared helper
+        raw, thread_id = await _build_reply_message(
+            service, input_data, graph_exec_id, user_id
         )
 
-        headers = {
-            h["name"].lower(): h["value"]
-            for h in parent.get("payload", {}).get("headers", [])
-        }
-        if not (input_data.to or input_data.cc or input_data.bcc):
-            if input_data.replyAll:
-                recipients = [parseaddr(headers.get("from", ""))[1]]
-                recipients += [
-                    addr for _, addr in getaddresses([headers.get("to", "")])
-                ]
-                recipients += [
-                    addr for _, addr in getaddresses([headers.get("cc", "")])
-                ]
-                # Deduplicate recipients while preserving order
-                input_data.to = deduplicate_email_addresses(recipients)
-            else:
-                sender = parseaddr(headers.get("reply-to", headers.get("from", "")))[1]
-                input_data.to = [sender] if sender else []
-        subject = input_data.subject or (f"Re: {headers.get('subject', '')}".strip())
-        references = headers.get("references", "").split()
-        if headers.get("message-id"):
-            references.append(headers["message-id"])
-
-        msg = MIMEMultipart()
-        if input_data.to:
-            msg["To"] = ", ".join(input_data.to)
-        if input_data.cc:
-            msg["Cc"] = ", ".join(input_data.cc)
-        if input_data.bcc:
-            msg["Bcc"] = ", ".join(input_data.bcc)
-        msg["Subject"] = subject
-        if headers.get("message-id"):
-            msg["In-Reply-To"] = headers["message-id"]
-        if references:
-            msg["References"] = " ".join(references)
-        # Use the new helper function for consistent content type handling
-        msg.attach(_make_mime_text(input_data.body, input_data.content_type))
-
-        for attach in input_data.attachments:
-            local_path = await store_media_file(
-                user_id=user_id,
-                graph_exec_id=graph_exec_id,
-                file=attach,
-                return_content=False,
-            )
-            abs_path = get_exec_file_path(graph_exec_id, local_path)
-            part = MIMEBase("application", "octet-stream")
-            with open(abs_path, "rb") as f:
-                part.set_payload(f.read())
-            encoders.encode_base64(part)
-            part.add_header(
-                "Content-Disposition", f"attachment; filename={Path(abs_path).name}"
-            )
-            msg.attach(part)
-
-        raw = base64.urlsafe_b64encode(msg.as_bytes()).decode("utf-8")
+        # Send the message
         return await asyncio.to_thread(
             lambda: service.users()
             .messages()
-            .send(userId="me", body={"threadId": input_data.threadId, "raw": raw})
+            .send(userId="me", body={"threadId": thread_id, "raw": raw})
             .execute()
         )
 
 
-class GmailCreateDraftReplyBlock(GmailBase):
+class GmailDraftReplyBlock(GmailBase):
     """
     Creates draft replies to Gmail threads with intelligent content type detection.
 
     Features:
     - Automatic HTML detection: Draft replies containing HTML tags are formatted as text/html
-    - No hard-wrap for plain text: Plain text drafts preserve natural line flow
+    - No hard-wrap for plain text: Plain text draft replies preserve natural line flow
     - Manual content type override: Use content_type parameter to force specific format
-    - Reply-all functionality: Option to draft reply to all original recipients
+    - Reply-all functionality: Option to reply to all original recipients
     - Thread preservation: Maintains proper email threading with headers
     - Full Unicode/emoji support with UTF-8 encoding
-    - Attachment support for multiple files
     """
 
     class Input(BlockSchema):
@@ -1377,31 +1405,31 @@ class GmailCreateDraftReplyBlock(GmailBase):
 
     def __init__(self):
         super().__init__(
-            id="8f2e9d3c-4b1a-4c7e-9a2f-1d3e5f7a9b1c",
-            description="Create draft replies to Gmail threads with automatic HTML detection and proper text formatting. Drafts maintain proper email threading and can be edited before sending.",
+            id="d7a9f3e2-8b4c-4d6f-9e1a-3c5b7f8d2a6e",
+            description="Create draft replies to Gmail threads with automatic HTML detection and proper text formatting. Plain text draft replies maintain natural paragraph flow without 78-character line wrapping. HTML content is automatically detected and formatted correctly.",
             categories={BlockCategory.COMMUNICATION},
-            input_schema=GmailCreateDraftReplyBlock.Input,
-            output_schema=GmailCreateDraftReplyBlock.Output,
+            input_schema=GmailDraftReplyBlock.Input,
+            output_schema=GmailDraftReplyBlock.Output,
             disabled=not GOOGLE_OAUTH_IS_CONFIGURED,
             test_input={
                 "threadId": "t1",
                 "parentMessageId": "m1",
-                "body": "Thanks for your message. I'll draft a response.",
+                "body": "Thanks for your message. I'll review and get back to you.",
                 "replyAll": False,
                 "credentials": TEST_CREDENTIALS_INPUT,
             },
             test_credentials=TEST_CREDENTIALS,
             test_output=[
                 ("draftId", "draft1"),
-                ("messageId", "msg1"),
+                ("messageId", "m2"),
                 ("threadId", "t1"),
-                ("status", "draft_reply_created"),
+                ("status", "draft_created"),
             ],
             test_mock={
                 "_create_draft_reply": lambda *args, **kwargs: {
                     "id": "draft1",
-                    "message": {"id": "msg1", "threadId": "t1"},
-                },
+                    "message": {"id": "m2", "threadId": "t1"},
+                }
             },
         )
 
@@ -1415,117 +1443,26 @@ class GmailCreateDraftReplyBlock(GmailBase):
         **kwargs,
     ) -> BlockOutput:
         service = self._build_service(credentials, **kwargs)
-        result = await self._create_draft_reply(
+        draft = await self._create_draft_reply(
             service,
             input_data,
             graph_exec_id,
             user_id,
         )
-        yield "draftId", result["id"]
-        yield "messageId", result["message"]["id"]
-        yield "threadId", result["message"].get("threadId", input_data.threadId)
-        yield "status", "draft_reply_created"
+        yield "draftId", draft["id"]
+        yield "messageId", draft["message"]["id"]
+        yield "threadId", draft["message"].get("threadId", input_data.threadId)
+        yield "status", "draft_created"
 
     async def _create_draft_reply(
         self, service, input_data: Input, graph_exec_id: str, user_id: str
     ) -> dict:
-        # Fetch parent message metadata
-        parent = await asyncio.to_thread(
-            lambda: service.users()
-            .messages()
-            .get(
-                userId="me",
-                id=input_data.parentMessageId,
-                format="metadata",
-                metadataHeaders=[
-                    "Subject",
-                    "References",
-                    "Message-ID",
-                    "From",
-                    "To",
-                    "Cc",
-                    "Reply-To",
-                ],
-            )
-            .execute()
+        # Build the reply message using the shared helper
+        raw, thread_id = await _build_reply_message(
+            service, input_data, graph_exec_id, user_id
         )
 
-        headers = {
-            h["name"].lower(): h["value"]
-            for h in parent.get("payload", {}).get("headers", [])
-        }
-
-        # Auto-populate recipients if not provided
-        if not (input_data.to or input_data.cc or input_data.bcc):
-            if input_data.replyAll:
-                # Reply all - include all original recipients
-                recipients = [parseaddr(headers.get("from", ""))[1]]
-                recipients += [
-                    addr for _, addr in getaddresses([headers.get("to", "")])
-                ]
-                recipients += [
-                    addr for _, addr in getaddresses([headers.get("cc", "")])
-                ]
-                # Deduplicate recipients
-                dedup: list[str] = []
-                for r in recipients:
-                    if r and r not in dedup:
-                        dedup.append(r)
-                input_data.to = dedup
-            else:
-                # Reply to sender only
-                sender = parseaddr(headers.get("reply-to", headers.get("from", "")))[1]
-                input_data.to = [sender] if sender else []
-
-        # Generate subject with Re: prefix if needed
-        subject = input_data.subject or (f"Re: {headers.get('subject', '')}".strip())
-
-        # Build References header chain
-        references = headers.get("references", "").split()
-        if headers.get("message-id"):
-            references.append(headers["message-id"])
-
-        # Create MIME message with threading headers
-        msg = MIMEMultipart()
-        if input_data.to:
-            msg["To"] = ", ".join(input_data.to)
-        if input_data.cc:
-            msg["Cc"] = ", ".join(input_data.cc)
-        if input_data.bcc:
-            msg["Bcc"] = ", ".join(input_data.bcc)
-        msg["Subject"] = subject
-
-        # Set threading headers for proper conversation grouping
-        if headers.get("message-id"):
-            msg["In-Reply-To"] = headers["message-id"]
-        if references:
-            msg["References"] = " ".join(references)
-
-        # Add body with proper content type handling
-        msg.attach(_make_mime_text(input_data.body, input_data.content_type))
-
-        # Handle attachments if any
-        for attach in input_data.attachments:
-            local_path = await store_media_file(
-                user_id=user_id,
-                graph_exec_id=graph_exec_id,
-                file=attach,
-                return_content=False,
-            )
-            abs_path = get_exec_file_path(graph_exec_id, local_path)
-            part = MIMEBase("application", "octet-stream")
-            with open(abs_path, "rb") as f:
-                part.set_payload(f.read())
-            encoders.encode_base64(part)
-            part.add_header(
-                "Content-Disposition", f"attachment; filename={Path(abs_path).name}"
-            )
-            msg.attach(part)
-
-        # Encode message for Gmail API
-        raw = base64.urlsafe_b64encode(msg.as_bytes()).decode("utf-8")
-
-        # Create draft with threadId to ensure it appears as a reply
+        # Create draft with proper thread association
         draft = await asyncio.to_thread(
             lambda: service.users()
             .drafts()
@@ -1533,7 +1470,7 @@ class GmailCreateDraftReplyBlock(GmailBase):
                 userId="me",
                 body={
                     "message": {
-                        "threadId": input_data.threadId,
+                        "threadId": thread_id,
                         "raw": raw,
                     }
                 },

autogpt_platform/backend/backend/blocks/ideogram.py

@@ -30,6 +30,7 @@ TEST_CREDENTIALS_INPUT = {
 
 
 class IdeogramModelName(str, Enum):
+    V3 = "V_3"
     V2 = "V_2"
     V1 = "V_1"
     V1_TURBO = "V_1_TURBO"
@@ -95,8 +96,8 @@ class IdeogramModelBlock(Block):
             title="Prompt",
         )
         ideogram_model_name: IdeogramModelName = SchemaField(
-            description="The name of the Image Generation Model, e.g., V_2",
-            default=IdeogramModelName.V2,
+            description="The name of the Image Generation Model, e.g., V_3",
+            default=IdeogramModelName.V3,
             title="Image Generation Model",
             advanced=False,
         )
@@ -236,6 +237,111 @@ class IdeogramModelBlock(Block):
         negative_prompt: Optional[str],
         color_palette_name: str,
         custom_colors: Optional[list[str]],
+    ):
+        # Use V3 endpoint for V3 model, legacy endpoint for others
+        if model_name == "V_3":
+            return await self._run_model_v3(
+                api_key,
+                prompt,
+                seed,
+                aspect_ratio,
+                magic_prompt_option,
+                style_type,
+                negative_prompt,
+                color_palette_name,
+                custom_colors,
+            )
+        else:
+            return await self._run_model_legacy(
+                api_key,
+                model_name,
+                prompt,
+                seed,
+                aspect_ratio,
+                magic_prompt_option,
+                style_type,
+                negative_prompt,
+                color_palette_name,
+                custom_colors,
+            )
+
+    async def _run_model_v3(
+        self,
+        api_key: SecretStr,
+        prompt: str,
+        seed: Optional[int],
+        aspect_ratio: str,
+        magic_prompt_option: str,
+        style_type: str,
+        negative_prompt: Optional[str],
+        color_palette_name: str,
+        custom_colors: Optional[list[str]],
+    ):
+        url = "https://api.ideogram.ai/v1/ideogram-v3/generate"
+        headers = {
+            "Api-Key": api_key.get_secret_value(),
+            "Content-Type": "application/json",
+        }
+
+        # Map legacy aspect ratio values to V3 format
+        aspect_ratio_map = {
+            "ASPECT_10_16": "10x16",
+            "ASPECT_16_10": "16x10",
+            "ASPECT_9_16": "9x16",
+            "ASPECT_16_9": "16x9",
+            "ASPECT_3_2": "3x2",
+            "ASPECT_2_3": "2x3",
+            "ASPECT_4_3": "4x3",
+            "ASPECT_3_4": "3x4",
+            "ASPECT_1_1": "1x1",
+            "ASPECT_1_3": "1x3",
+            "ASPECT_3_1": "3x1",
+            # Additional V3 supported ratios
+            "ASPECT_1_2": "1x2",
+            "ASPECT_2_1": "2x1",
+            "ASPECT_4_5": "4x5",
+            "ASPECT_5_4": "5x4",
+        }
+
+        v3_aspect_ratio = aspect_ratio_map.get(
+            aspect_ratio, "1x1"
+        )  # Default to 1x1 if not found
+
+        # Use JSON for V3 endpoint (simpler than multipart/form-data)
+        data: Dict[str, Any] = {
+            "prompt": prompt,
+            "aspect_ratio": v3_aspect_ratio,
+            "magic_prompt": magic_prompt_option,
+            "style_type": style_type,
+        }
+
+        if seed is not None:
+            data["seed"] = seed
+
+        if negative_prompt:
+            data["negative_prompt"] = negative_prompt
+
+        # Note: V3 endpoint may have different color palette support
+        # For now, we'll omit color palettes for V3 to avoid errors
+
+        try:
+            response = await Requests().post(url, headers=headers, json=data)
+            return response.json()["data"][0]["url"]
+        except RequestException as e:
+            raise Exception(f"Failed to fetch image with V3 endpoint: {str(e)}")
+
+    async def _run_model_legacy(
+        self,
+        api_key: SecretStr,
+        model_name: str,
+        prompt: str,
+        seed: Optional[int],
+        aspect_ratio: str,
+        magic_prompt_option: str,
+        style_type: str,
+        negative_prompt: Optional[str],
+        color_palette_name: str,
+        custom_colors: Optional[list[str]],
     ):
         url = "https://api.ideogram.ai/generate"
         headers = {
@@ -249,28 +355,33 @@ class IdeogramModelBlock(Block):
                 "model": model_name,
                 "aspect_ratio": aspect_ratio,
                 "magic_prompt_option": magic_prompt_option,
-                "style_type": style_type,
             }
         }
 
+        # Only add style_type for V2, V2_TURBO, and V3 models (V1 models don't support it)
+        if model_name in ["V_2", "V_2_TURBO", "V_3"]:
+            data["image_request"]["style_type"] = style_type
+
         if seed is not None:
             data["image_request"]["seed"] = seed
 
         if negative_prompt:
             data["image_request"]["negative_prompt"] = negative_prompt
 
-        if color_palette_name != "NONE":
-            data["color_palette"] = {"name": color_palette_name}
-        elif custom_colors:
-            data["color_palette"] = {
-                "members": [{"color_hex": color} for color in custom_colors]
-            }
+        # Only add color palette for V2 and V2_TURBO models (V1 models don't support it)
+        if model_name in ["V_2", "V_2_TURBO"]:
+            if color_palette_name != "NONE":
+                data["color_palette"] = {"name": color_palette_name}
+            elif custom_colors:
+                data["color_palette"] = {
+                    "members": [{"color_hex": color} for color in custom_colors]
+                }
 
         try:
             response = await Requests().post(url, headers=headers, json=data)
             return response.json()["data"][0]["url"]
         except RequestException as e:
-            raise Exception(f"Failed to fetch image: {str(e)}")
+            raise Exception(f"Failed to fetch image with legacy endpoint: {str(e)}")
 
     async def upscale_image(self, api_key: SecretStr, image_url: str):
         url = "https://api.ideogram.ai/upscale"

autogpt_platform/backend/backend/blocks/io.py

@@ -10,7 +10,6 @@ from backend.util.settings import Config
 from backend.util.text import TextFormatter
 from backend.util.type import LongTextType, MediaFileType, ShortTextType
 
-formatter = TextFormatter()
 config = Config()
 
 
@@ -132,6 +131,11 @@ class AgentOutputBlock(Block):
             default="",
             advanced=True,
         )
+        escape_html: bool = SchemaField(
+            default=False,
+            advanced=True,
+            description="Whether to escape special characters in the inserted values to be HTML-safe. Enable for HTML output, disable for plain text.",
+        )
         advanced: bool = SchemaField(
             description="Whether to treat the output as advanced.",
             default=False,
@@ -193,6 +197,7 @@ class AgentOutputBlock(Block):
         """
         if input_data.format:
             try:
+                formatter = TextFormatter(autoescape=input_data.escape_html)
                 yield "output", formatter.format_string(
                     input_data.format, {input_data.name: input_data.value}
                 )
@@ -549,6 +554,89 @@ class AgentToggleInputBlock(AgentInputBlock):
         )
 
 
+class AgentTableInputBlock(AgentInputBlock):
+    """
+    This block allows users to input data in a table format.
+
+    Configure the table columns at build time, then users can input
+    rows of data at runtime. Each row is output as a dictionary
+    with column names as keys.
+    """
+
+    class Input(AgentInputBlock.Input):
+        value: Optional[list[dict[str, Any]]] = SchemaField(
+            description="The table data as a list of dictionaries.",
+            default=None,
+            advanced=False,
+            title="Default Value",
+        )
+        column_headers: list[str] = SchemaField(
+            description="Column headers for the table.",
+            default_factory=lambda: ["Column 1", "Column 2", "Column 3"],
+            advanced=False,
+            title="Column Headers",
+        )
+
+        def generate_schema(self):
+            """Generate schema for the value field with table format."""
+            schema = super().generate_schema()
+            schema["type"] = "array"
+            schema["format"] = "table"
+            schema["items"] = {
+                "type": "object",
+                "properties": {
+                    header: {"type": "string"}
+                    for header in (
+                        self.column_headers or ["Column 1", "Column 2", "Column 3"]
+                    )
+                },
+            }
+            if self.value is not None:
+                schema["default"] = self.value
+            return schema
+
+    class Output(AgentInputBlock.Output):
+        result: list[dict[str, Any]] = SchemaField(
+            description="The table data as a list of dictionaries with headers as keys."
+        )
+
+    def __init__(self):
+        super().__init__(
+            id="5603b273-f41e-4020-af7d-fbc9c6a8d928",
+            description="Block for table data input with customizable headers.",
+            disabled=not config.enable_agent_input_subtype_blocks,
+            input_schema=AgentTableInputBlock.Input,
+            output_schema=AgentTableInputBlock.Output,
+            test_input=[
+                {
+                    "name": "test_table",
+                    "column_headers": ["Name", "Age", "City"],
+                    "value": [
+                        {"Name": "John", "Age": "30", "City": "New York"},
+                        {"Name": "Jane", "Age": "25", "City": "London"},
+                    ],
+                    "description": "Example table input",
+                }
+            ],
+            test_output=[
+                (
+                    "result",
+                    [
+                        {"Name": "John", "Age": "30", "City": "New York"},
+                        {"Name": "Jane", "Age": "25", "City": "London"},
+                    ],
+                )
+            ],
+        )
+
+    async def run(self, input_data: Input, *args, **kwargs) -> BlockOutput:
+        """
+        Yields the table data as a list of dictionaries.
+        """
+        # Pass through the value, defaulting to empty list if None
+        yield "result", input_data.value if input_data.value is not None else []
+
+
 IO_BLOCK_IDs = [
     AgentInputBlock().id,
     AgentOutputBlock().id,
@@ -560,4 +648,5 @@ IO_BLOCK_IDs = [
     AgentFileInputBlock().id,
     AgentDropdownInputBlock().id,
     AgentToggleInputBlock().id,
+    AgentTableInputBlock().id,
 ]

autogpt_platform/backend/backend/blocks/iteration.py

@@ -2,7 +2,7 @@ from typing import Any
 
 from backend.data.block import Block, BlockCategory, BlockOutput, BlockSchema
 from backend.data.model import SchemaField
-from backend.util.json import json
+from backend.util.json import loads
 
 
 class StepThroughItemsBlock(Block):
@@ -54,20 +54,43 @@ class StepThroughItemsBlock(Block):
         )
 
     async def run(self, input_data: Input, **kwargs) -> BlockOutput:
+        # Security fix: Add limits to prevent DoS from large iterations
+        MAX_ITEMS = 10000  # Maximum items to iterate
+        MAX_ITEM_SIZE = 1024 * 1024  # 1MB per item
+
         for data in [input_data.items, input_data.items_object, input_data.items_str]:
             if not data:
                 continue
+
+            # Limit string size before parsing
             if isinstance(data, str):
-                items = json.loads(data)
+                if len(data) > MAX_ITEM_SIZE:
+                    raise ValueError(
+                        f"Input too large: {len(data)} bytes > {MAX_ITEM_SIZE} bytes"
+                    )
+                items = loads(data)
             else:
                 items = data
+
+            # Check total item count
+            if isinstance(items, (list, dict)):
+                if len(items) > MAX_ITEMS:
+                    raise ValueError(f"Too many items: {len(items)} > {MAX_ITEMS}")
+
+            iteration_count = 0
             if isinstance(items, dict):
                 # If items is a dictionary, iterate over its values
-                for item in items.values():
-                    yield "item", item
-                    yield "key", item
+                for key, value in items.items():
+                    if iteration_count >= MAX_ITEMS:
+                        break
+                    yield "item", value
+                    yield "key", key  # Fixed: should yield key, not item
+                    iteration_count += 1
             else:
                 # If items is a list, iterate over the list
                 for index, item in enumerate(items):
+                    if iteration_count >= MAX_ITEMS:
+                        break
                     yield "item", item
                     yield "key", index
+                    iteration_count += 1

autogpt_platform/backend/backend/blocks/jina/fact_checker.py

@@ -1,5 +1,8 @@
+from typing import List
 from urllib.parse import quote
 
+from typing_extensions import TypedDict
+
 from backend.blocks.jina._auth import (
     JinaCredentials,
     JinaCredentialsField,
@@ -10,6 +13,12 @@ from backend.data.model import SchemaField
 from backend.util.request import Requests
 
 
+class Reference(TypedDict):
+    url: str
+    keyQuote: str
+    isSupportive: bool
+
+
 class FactCheckerBlock(Block):
     class Input(BlockSchema):
         statement: str = SchemaField(
@@ -23,6 +32,10 @@ class FactCheckerBlock(Block):
         )
         result: bool = SchemaField(description="The result of the factuality check")
         reason: str = SchemaField(description="The reason for the factuality result")
+        references: List[Reference] = SchemaField(
+            description="List of references supporting or contradicting the statement",
+            default=[],
+        )
         error: str = SchemaField(description="Error message if the check fails")
 
     def __init__(self):
@@ -53,5 +66,11 @@ class FactCheckerBlock(Block):
             yield "factuality", data["factuality"]
             yield "result", data["result"]
             yield "reason", data["reason"]
+
+            # Yield references if present in the response
+            if "references" in data:
+                yield "references", data["references"]
+            else:
+                yield "references", []
         else:
             raise RuntimeError(f"Expected 'data' key not found in response: {data}")

autogpt_platform/backend/backend/blocks/linear/_config.py

@@ -62,10 +62,10 @@ TEST_CREDENTIALS_OAUTH = OAuth2Credentials(
     title="Mock Linear API key",
     username="mock-linear-username",
     access_token=SecretStr("mock-linear-access-token"),
-    access_token_expires_at=None,
+    access_token_expires_at=1672531200,  # Mock expiration time for short-lived token
     refresh_token=SecretStr("mock-linear-refresh-token"),
     refresh_token_expires_at=None,
-    scopes=["mock-linear-scopes"],
+    scopes=["read", "write"],
 )
 
 TEST_CREDENTIALS_API_KEY = APIKeyCredentials(

autogpt_platform/backend/backend/blocks/linear/_oauth.py

@@ -2,7 +2,9 @@
 Linear OAuth handler implementation.
 """
 
+import base64
 import json
+import time
 from typing import Optional
 from urllib.parse import urlencode
 
@@ -38,8 +40,9 @@ class LinearOAuthHandler(BaseOAuthHandler):
         self.client_secret = client_secret
         self.redirect_uri = redirect_uri
         self.auth_base_url = "https://linear.app/oauth/authorize"
-        self.token_url = "https://api.linear.app/oauth/token"  # Correct token URL
+        self.token_url = "https://api.linear.app/oauth/token"
         self.revoke_url = "https://api.linear.app/oauth/revoke"
+        self.migrate_url = "https://api.linear.app/oauth/migrate_old_token"
 
     def get_login_url(
         self, scopes: list[str], state: str, code_challenge: Optional[str]
@@ -82,19 +85,84 @@ class LinearOAuthHandler(BaseOAuthHandler):
 
         return True  # Linear doesn't return JSON on successful revoke
 
+    async def migrate_old_token(
+        self, credentials: OAuth2Credentials
+    ) -> OAuth2Credentials:
+        """
+        Migrate an old long-lived token to a new short-lived token with refresh token.
+
+        This uses Linear's /oauth/migrate_old_token endpoint to exchange current
+        long-lived tokens for short-lived tokens with refresh tokens without
+        requiring users to re-authorize.
+        """
+        if not credentials.access_token:
+            raise ValueError("No access token to migrate")
+
+        request_body = {
+            "client_id": self.client_id,
+            "client_secret": self.client_secret,
+        }
+
+        headers = {
+            "Authorization": f"Bearer {credentials.access_token.get_secret_value()}",
+            "Content-Type": "application/x-www-form-urlencoded",
+        }
+
+        response = await Requests().post(
+            self.migrate_url, data=request_body, headers=headers
+        )
+
+        if not response.ok:
+            try:
+                error_data = response.json()
+                error_message = error_data.get("error", "Unknown error")
+                error_description = error_data.get("error_description", "")
+                if error_description:
+                    error_message = f"{error_message}: {error_description}"
+            except json.JSONDecodeError:
+                error_message = response.text
+            raise LinearAPIException(
+                f"Failed to migrate Linear token ({response.status}): {error_message}",
+                response.status,
+            )
+
+        token_data = response.json()
+
+        # Extract token expiration
+        now = int(time.time())
+        expires_in = token_data.get("expires_in")
+        access_token_expires_at = None
+        if expires_in:
+            access_token_expires_at = now + expires_in
+
+        new_credentials = OAuth2Credentials(
+            provider=self.PROVIDER_NAME,
+            title=credentials.title,
+            username=credentials.username,
+            access_token=token_data["access_token"],
+            scopes=credentials.scopes,  # Preserve original scopes
+            refresh_token=token_data.get("refresh_token"),
+            access_token_expires_at=access_token_expires_at,
+            refresh_token_expires_at=None,
+        )
+
+        new_credentials.id = credentials.id
+        return new_credentials
+
     async def _refresh_tokens(
         self, credentials: OAuth2Credentials
     ) -> OAuth2Credentials:
         if not credentials.refresh_token:
             raise ValueError(
-                "No refresh token available."
-            )  # Linear uses non-expiring tokens
+                "No refresh token available. Token may need to be migrated to the new refresh token system."
+            )
 
         return await self._request_tokens(
             {
                 "refresh_token": credentials.refresh_token.get_secret_value(),
                 "grant_type": "refresh_token",
-            }
+            },
+            current_credentials=credentials,
         )
 
     async def _request_tokens(
@@ -102,16 +170,33 @@ class LinearOAuthHandler(BaseOAuthHandler):
         params: dict[str, str],
         current_credentials: Optional[OAuth2Credentials] = None,
     ) -> OAuth2Credentials:
+        # Determine if this is a refresh token request
+        is_refresh = params.get("grant_type") == "refresh_token"
+
+        # Build request body with appropriate grant_type
         request_body = {
             "client_id": self.client_id,
             "client_secret": self.client_secret,
-            "grant_type": "authorization_code",  # Ensure grant_type is correct
             **params,
         }
 
-        headers = {
-            "Content-Type": "application/x-www-form-urlencoded"
-        }  # Correct header for token request
+        # Set default grant_type if not provided
+        if "grant_type" not in request_body:
+            request_body["grant_type"] = "authorization_code"
+
+        headers = {"Content-Type": "application/x-www-form-urlencoded"}
+
+        # For refresh token requests, support HTTP Basic Authentication as recommended
+        if is_refresh:
+            # Option 1: Use HTTP Basic Auth (preferred by Linear)
+            client_credentials = f"{self.client_id}:{self.client_secret}"
+            encoded_credentials = base64.b64encode(client_credentials.encode()).decode()
+            headers["Authorization"] = f"Basic {encoded_credentials}"
+
+            # Remove client credentials from body when using Basic Auth
+            request_body.pop("client_id", None)
+            request_body.pop("client_secret", None)
+
         response = await Requests().post(
             self.token_url, data=request_body, headers=headers
         )
@@ -120,6 +205,9 @@ class LinearOAuthHandler(BaseOAuthHandler):
             try:
                 error_data = response.json()
                 error_message = error_data.get("error", "Unknown error")
+                error_description = error_data.get("error_description", "")
+                if error_description:
+                    error_message = f"{error_message}: {error_description}"
             except json.JSONDecodeError:
                 error_message = response.text
             raise LinearAPIException(
@@ -129,27 +217,84 @@ class LinearOAuthHandler(BaseOAuthHandler):
 
         token_data = response.json()
 
-        # Note: Linear access tokens do not expire, so we set expires_at to None
+        # Extract token expiration if provided (for new refresh token implementation)
+        now = int(time.time())
+        expires_in = token_data.get("expires_in")
+        access_token_expires_at = None
+        if expires_in:
+            access_token_expires_at = now + expires_in
+
+        # Get username - preserve from current credentials if refreshing
+        username = None
+        if current_credentials and is_refresh:
+            username = current_credentials.username
+        elif "user" in token_data:
+            username = token_data["user"].get("name", "Unknown User")
+        else:
+            # Fetch username using the access token
+            username = await self._request_username(token_data["access_token"])
+
         new_credentials = OAuth2Credentials(
             provider=self.PROVIDER_NAME,
             title=current_credentials.title if current_credentials else None,
-            username=token_data.get("user", {}).get(
-                "name", "Unknown User"
-            ),  # extract name or set appropriate
+            username=username or "Unknown User",
             access_token=token_data["access_token"],
-            scopes=token_data["scope"].split(
-                ","
-            ),  # Linear returns comma-separated scopes
-            refresh_token=token_data.get(
-                "refresh_token"
-            ),  # Linear uses non-expiring tokens so this might be null
-            access_token_expires_at=None,
-            refresh_token_expires_at=None,
+            scopes=(
+                token_data["scope"].split(",")
+                if "scope" in token_data
+                else (current_credentials.scopes if current_credentials else [])
+            ),
+            refresh_token=token_data.get("refresh_token"),
+            access_token_expires_at=access_token_expires_at,
+            refresh_token_expires_at=None,  # Linear doesn't provide refresh token expiration
         )
+
         if current_credentials:
             new_credentials.id = current_credentials.id
+
         return new_credentials
 
+    async def get_access_token(self, credentials: OAuth2Credentials) -> str:
+        """
+        Returns a valid access token, handling migration and refresh as needed.
+
+        This overrides the base implementation to handle Linear's token migration
+        from old long-lived tokens to new short-lived tokens with refresh tokens.
+        """
+        # If token has no expiration and no refresh token, it might be an old token
+        # that needs migration
+        if (
+            credentials.access_token_expires_at is None
+            and credentials.refresh_token is None
+        ):
+            try:
+                # Attempt to migrate the old token
+                migrated_credentials = await self.migrate_old_token(credentials)
+                # Update the credentials store would need to be handled by the caller
+                # For now, use the migrated credentials for this request
+                credentials = migrated_credentials
+            except LinearAPIException:
+                # Migration failed, try to use the old token as-is
+                # This maintains backward compatibility
+                pass
+
+        # Use the standard refresh logic from the base class
+        if self.needs_refresh(credentials):
+            credentials = await self.refresh_tokens(credentials)
+
+        return credentials.access_token.get_secret_value()
+
+    def needs_migration(self, credentials: OAuth2Credentials) -> bool:
+        """
+        Check if credentials represent an old long-lived token that needs migration.
+
+        Old tokens have no expiration time and no refresh token.
+        """
+        return (
+            credentials.access_token_expires_at is None
+            and credentials.refresh_token is None
+        )
+
     async def _request_username(self, access_token: str) -> Optional[str]:
         # Use the LinearClient to fetch user details using GraphQL
         from ._api import LinearClient

autogpt_platform/backend/backend/blocks/linear/models.py

@@ -37,5 +37,5 @@ class Project(BaseModel):
     name: str
     description: str
     priority: int
-    progress: int
-    content: str
+    progress: float
+    content: str | None

autogpt_platform/backend/backend/blocks/llm.py

@@ -1,5 +1,9 @@
+# This file contains a lot of prompt block strings that would trigger "line too long"
+# flake8: noqa: E501
 import ast
 import logging
+import re
+import secrets
 from abc import ABC
 from enum import Enum, EnumMeta
 from json import JSONDecodeError
@@ -27,7 +31,7 @@ from backend.util.prompt import compress_prompt, estimate_token_count
 from backend.util.text import TextFormatter
 
 logger = TruncatedLogger(logging.getLogger(__name__), "[LLM-Block]")
-fmt = TextFormatter()
+fmt = TextFormatter(autoescape=False)
 
 LLMProviderName = Literal[
     ProviderName.AIML_API,
@@ -97,6 +101,8 @@ class LlmModel(str, Enum, metaclass=LlmModelMeta):
     CLAUDE_4_1_OPUS = "claude-opus-4-1-20250805"
     CLAUDE_4_OPUS = "claude-opus-4-20250514"
     CLAUDE_4_SONNET = "claude-sonnet-4-20250514"
+    CLAUDE_4_5_SONNET = "claude-sonnet-4-5-20250929"
+    CLAUDE_4_5_HAIKU = "claude-haiku-4-5-20251001"
     CLAUDE_3_7_SONNET = "claude-3-7-sonnet-20250219"
     CLAUDE_3_5_SONNET = "claude-3-5-sonnet-latest"
     CLAUDE_3_5_HAIKU = "claude-3-5-haiku-latest"
@@ -204,13 +210,19 @@ MODEL_METADATA = {
         "anthropic", 200000, 32000
     ),  # claude-opus-4-1-20250805
     LlmModel.CLAUDE_4_OPUS: ModelMetadata(
-        "anthropic", 200000, 8192
+        "anthropic", 200000, 32000
     ),  # claude-4-opus-20250514
     LlmModel.CLAUDE_4_SONNET: ModelMetadata(
-        "anthropic", 200000, 8192
+        "anthropic", 200000, 64000
     ),  # claude-4-sonnet-20250514
+    LlmModel.CLAUDE_4_5_SONNET: ModelMetadata(
+        "anthropic", 200000, 64000
+    ),  # claude-sonnet-4-5-20250929
+    LlmModel.CLAUDE_4_5_HAIKU: ModelMetadata(
+        "anthropic", 200000, 64000
+    ),  # claude-haiku-4-5-20251001
     LlmModel.CLAUDE_3_7_SONNET: ModelMetadata(
-        "anthropic", 200000, 8192
+        "anthropic", 200000, 64000
     ),  # claude-3-7-sonnet-20250219
     LlmModel.CLAUDE_3_5_SONNET: ModelMetadata(
         "anthropic", 200000, 8192
@@ -382,7 +394,9 @@ def extract_openai_tool_calls(response) -> list[ToolContentBlock] | None:
     return None
 
 
-def get_parallel_tool_calls_param(llm_model: LlmModel, parallel_tool_calls):
+def get_parallel_tool_calls_param(
+    llm_model: LlmModel, parallel_tool_calls: bool | None
+):
     """Get the appropriate parallel_tool_calls parameter for OpenAI-compatible APIs."""
     if llm_model.startswith("o") or parallel_tool_calls is None:
         return openai.NOT_GIVEN
@@ -393,8 +407,8 @@ async def llm_call(
     credentials: APIKeyCredentials,
     llm_model: LlmModel,
     prompt: list[dict],
-    json_format: bool,
     max_tokens: int | None,
+    force_json_output: bool = False,
     tools: list[dict] | None = None,
     ollama_host: str = "localhost:11434",
     parallel_tool_calls=None,
@@ -407,7 +421,7 @@ async def llm_call(
         credentials: The API key credentials to use.
         llm_model: The LLM model to use.
         prompt: The prompt to send to the LLM.
-        json_format: Whether the response should be in JSON format.
+        force_json_output: Whether the response should be in JSON format.
         max_tokens: The maximum number of tokens to generate in the chat completion.
         tools: The tools to use in the chat completion.
         ollama_host: The host for ollama to use.
@@ -446,7 +460,7 @@ async def llm_call(
             llm_model, parallel_tool_calls
         )
 
-        if json_format:
+        if force_json_output:
             response_format = {"type": "json_object"}
 
         response = await oai_client.chat.completions.create(
@@ -559,7 +573,7 @@ async def llm_call(
             raise ValueError("Groq does not support tools.")
 
         client = AsyncGroq(api_key=credentials.api_key.get_secret_value())
-        response_format = {"type": "json_object"} if json_format else None
+        response_format = {"type": "json_object"} if force_json_output else None
         response = await client.chat.completions.create(
             model=llm_model.value,
             messages=prompt,  # type: ignore
@@ -717,7 +731,7 @@ async def llm_call(
         )
 
         response_format = None
-        if json_format:
+        if force_json_output:
             response_format = {"type": "json_object"}
 
         parallel_tool_calls_param = get_parallel_tool_calls_param(
@@ -780,6 +794,17 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
             description="The language model to use for answering the prompt.",
             advanced=False,
         )
+        force_json_output: bool = SchemaField(
+            title="Restrict LLM to pure JSON output",
+            default=False,
+            description=(
+                "Whether to force the LLM to produce a JSON-only response. "
+                "This can increase the block's reliability, "
+                "but may also reduce the quality of the response "
+                "because it prohibits the LLM from reasoning "
+                "before providing its JSON response."
+            ),
+        )
         credentials: AICredentials = AICredentialsField()
         sys_prompt: str = SchemaField(
             title="System Prompt",
@@ -848,17 +873,18 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
                 "llm_call": lambda *args, **kwargs: LLMResponse(
                     raw_response="",
                     prompt=[""],
-                    response=json.dumps(
-                        {
-                            "key1": "key1Value",
-                            "key2": "key2Value",
-                        }
+                    response=(
+                        '<json_output id="test123456">{\n'
+                        '  "key1": "key1Value",\n'
+                        '  "key2": "key2Value"\n'
+                        "}</json_output>"
                     ),
                     tool_calls=None,
                     prompt_tokens=0,
                     completion_tokens=0,
                     reasoning=None,
-                )
+                ),
+                "get_collision_proof_output_tag_id": lambda *args: "test123456",
             },
         )
 
@@ -867,9 +893,9 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
         credentials: APIKeyCredentials,
         llm_model: LlmModel,
         prompt: list[dict],
-        json_format: bool,
-        compress_prompt_to_fit: bool,
         max_tokens: int | None,
+        force_json_output: bool = False,
+        compress_prompt_to_fit: bool = True,
         tools: list[dict] | None = None,
         ollama_host: str = "localhost:11434",
     ) -> LLMResponse:
@@ -882,8 +908,8 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
             credentials=credentials,
             llm_model=llm_model,
             prompt=prompt,
-            json_format=json_format,
             max_tokens=max_tokens,
+            force_json_output=force_json_output,
             tools=tools,
             ollama_host=ollama_host,
             compress_prompt_to_fit=compress_prompt_to_fit,
@@ -895,10 +921,6 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
         logger.debug(f"Calling LLM with input data: {input_data}")
         prompt = [json.to_dict(p) for p in input_data.conversation_history]
 
-        def trim_prompt(s: str) -> str:
-            lines = s.strip().split("\n")
-            return "\n".join([line.strip().lstrip("|") for line in lines])
-
         values = input_data.prompt_values
         if values:
             input_data.prompt = fmt.format_string(input_data.prompt, values)
@@ -907,27 +929,15 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
         if input_data.sys_prompt:
             prompt.append({"role": "system", "content": input_data.sys_prompt})
 
+        # Use a one-time unique tag to prevent collisions with user/LLM content
+        output_tag_id = self.get_collision_proof_output_tag_id()
+        output_tag_start = f'<json_output id="{output_tag_id}">'
         if input_data.expected_format:
-            expected_format = [
-                f'"{k}": "{v}"' for k, v in input_data.expected_format.items()
-            ]
-            if input_data.list_result:
-                format_prompt = (
-                    f'"results": [\n  {{\n  {", ".join(expected_format)}\n  }}\n]'
-                )
-            else:
-                format_prompt = "\n  ".join(expected_format)
-
-            sys_prompt = trim_prompt(
-                f"""
-                  |Reply strictly only in the following JSON format:
-                  |{{
-                  |  {format_prompt}
-                  |}}
-                  |
-                  |Ensure the response is valid JSON. Do not include any additional text outside of the JSON.
-                  |If you cannot provide all the keys, provide an empty string for the values you cannot answer.
-                """
+            sys_prompt = self.response_format_instructions(
+                input_data.expected_format,
+                list_mode=input_data.list_result,
+                pure_json_mode=input_data.force_json_output,
+                output_tag_start=output_tag_start,
             )
             prompt.append({"role": "system", "content": sys_prompt})
 
@@ -945,18 +955,21 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
             except JSONDecodeError as e:
                 return f"JSON decode error: {e}"
 
-        logger.debug(f"LLM request: {prompt}")
-        retry_prompt = ""
+        error_feedback_message = ""
         llm_model = input_data.model
 
         for retry_count in range(input_data.retry):
+            logger.debug(f"LLM request: {prompt}")
             try:
                 llm_response = await self.llm_call(
                     credentials=credentials,
                     llm_model=llm_model,
                     prompt=prompt,
                     compress_prompt_to_fit=input_data.compress_prompt_to_fit,
-                    json_format=bool(input_data.expected_format),
+                    force_json_output=(
+                        input_data.force_json_output
+                        and bool(input_data.expected_format)
+                    ),
                     ollama_host=input_data.ollama_host,
                     max_tokens=input_data.max_tokens,
                 )
@@ -970,16 +983,55 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
                 logger.debug(f"LLM attempt-{retry_count} response: {response_text}")
 
                 if input_data.expected_format:
+                    try:
+                        response_obj = self.get_json_from_response(
+                            response_text,
+                            pure_json_mode=input_data.force_json_output,
+                            output_tag_start=output_tag_start,
+                        )
+                    except (ValueError, JSONDecodeError) as parse_error:
+                        censored_response = re.sub(r"[A-Za-z0-9]", "*", response_text)
+                        response_snippet = (
+                            f"{censored_response[:50]}...{censored_response[-30:]}"
+                        )
+                        logger.warning(
+                            f"Error getting JSON from LLM response: {parse_error}\n\n"
+                            f"Response start+end: `{response_snippet}`"
+                        )
+                        prompt.append({"role": "assistant", "content": response_text})
 
-                    response_obj = json.loads(response_text)
+                        error_feedback_message = self.invalid_response_feedback(
+                            parse_error,
+                            was_parseable=False,
+                            list_mode=input_data.list_result,
+                            pure_json_mode=input_data.force_json_output,
+                            output_tag_start=output_tag_start,
+                        )
+                        prompt.append(
+                            {"role": "user", "content": error_feedback_message}
+                        )
+                        continue
 
+                    # Handle object response for `force_json_output`+`list_result`
                     if input_data.list_result and isinstance(response_obj, dict):
-                        if "results" in response_obj:
-                            response_obj = response_obj.get("results", [])
-                        elif len(response_obj) == 1:
-                            response_obj = list(response_obj.values())
+                        if "results" in response_obj and isinstance(
+                            response_obj["results"], list
+                        ):
+                            response_obj = response_obj["results"]
+                        else:
+                            error_feedback_message = (
+                                "Expected an array of objects in the 'results' key, "
+                                f"but got: {response_obj}"
+                            )
+                            prompt.append(
+                                {"role": "assistant", "content": response_text}
+                            )
+                            prompt.append(
+                                {"role": "user", "content": error_feedback_message}
+                            )
+                            continue
 
-                    response_error = "\n".join(
+                    validation_errors = "\n".join(
                         [
                             validation_error
                             for response_item in (
@@ -991,7 +1043,7 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
                         ]
                     )
 
-                    if not response_error:
+                    if not validation_errors:
                         self.merge_stats(
                             NodeExecutionStats(
                                 llm_call_count=retry_count + 1,
@@ -1001,6 +1053,16 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
                         yield "response", response_obj
                         yield "prompt", self.prompt
                         return
+
+                    prompt.append({"role": "assistant", "content": response_text})
+                    error_feedback_message = self.invalid_response_feedback(
+                        validation_errors,
+                        was_parseable=True,
+                        list_mode=input_data.list_result,
+                        pure_json_mode=input_data.force_json_output,
+                        output_tag_start=output_tag_start,
+                    )
+                    prompt.append({"role": "user", "content": error_feedback_message})
                 else:
                     self.merge_stats(
                         NodeExecutionStats(
@@ -1011,21 +1073,6 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
                     yield "response", {"response": response_text}
                     yield "prompt", self.prompt
                     return
-
-                retry_prompt = trim_prompt(
-                    f"""
-                  |This is your previous error response:
-                  |--
-                  |{response_text}
-                  |--
-                  |
-                  |And this is the error:
-                  |--
-                  |{response_error}
-                  |--
-                """
-                )
-                prompt.append({"role": "user", "content": retry_prompt})
             except Exception as e:
                 logger.exception(f"Error calling LLM: {e}")
                 if (
@@ -1038,9 +1085,133 @@ class AIStructuredResponseGeneratorBlock(AIBlockBase):
                     logger.debug(
                         f"Reducing max_tokens to {input_data.max_tokens} for next attempt"
                     )
-                retry_prompt = f"Error calling LLM: {e}"
+                    # Don't add retry prompt for token limit errors,
+                    # just retry with lower maximum output tokens
 
-        raise RuntimeError(retry_prompt)
+                error_feedback_message = f"Error calling LLM: {e}"
+
+        raise RuntimeError(error_feedback_message)
+
+    def response_format_instructions(
+        self,
+        expected_object_format: dict[str, str],
+        *,
+        list_mode: bool,
+        pure_json_mode: bool,
+        output_tag_start: str,
+    ) -> str:
+        expected_output_format = json.dumps(expected_object_format, indent=2)
+        output_type = "object" if not list_mode else "array"
+        outer_output_type = "object" if pure_json_mode else output_type
+
+        if output_type == "array":
+            indented_obj_format = expected_output_format.replace("\n", "\n  ")
+            expected_output_format = f"[\n  {indented_obj_format},\n  ...\n]"
+            if pure_json_mode:
+                indented_list_format = expected_output_format.replace("\n", "\n  ")
+                expected_output_format = (
+                    "{\n"
+                    '  "reasoning": "... (optional)",\n'  # for better performance
+                    f'  "results": {indented_list_format}\n'
+                    "}"
+                )
+
+        # Preserve indentation in prompt
+        expected_output_format = expected_output_format.replace("\n", "\n|")
+
+        # Prepare prompt
+        if not pure_json_mode:
+            expected_output_format = (
+                f"{output_tag_start}\n{expected_output_format}\n</json_output>"
+            )
+
+        instructions = f"""
+        |In your response you MUST include a valid JSON {outer_output_type} strictly following this format:
+        |{expected_output_format}
+        |
+        |If you cannot provide all the keys, you MUST provide an empty string for the values you cannot answer.
+        """.strip()
+
+        if not pure_json_mode:
+            instructions += f"""
+            |
+            |You MUST enclose your final JSON answer in {output_tag_start}...</json_output> tags, even if the user specifies a different tag.
+            |There MUST be exactly ONE {output_tag_start}...</json_output> block in your response, which MUST ONLY contain the JSON {outer_output_type} and nothing else. Other text outside this block is allowed.
+            """.strip()
+
+        return trim_prompt(instructions)
+
+    def invalid_response_feedback(
+        self,
+        error,
+        *,
+        was_parseable: bool,
+        list_mode: bool,
+        pure_json_mode: bool,
+        output_tag_start: str,
+    ) -> str:
+        outer_output_type = "object" if not list_mode or pure_json_mode else "array"
+
+        if was_parseable:
+            complaint = f"Your previous response did not match the expected {outer_output_type} format."
+        else:
+            complaint = f"Your previous response did not contain a parseable JSON {outer_output_type}."
+
+        indented_parse_error = str(error).replace("\n", "\n|")
+
+        instruction = (
+            f"Please provide a {output_tag_start}...</json_output> block containing a"
+            if not pure_json_mode
+            else "Please provide a"
+        ) + f" valid JSON {outer_output_type} that matches the expected format."
+
+        return trim_prompt(
+            f"""
+            |{complaint}
+            |
+            |{indented_parse_error}
+            |
+            |{instruction}
+        """
+        )
+
+    def get_json_from_response(
+        self, response_text: str, *, pure_json_mode: bool, output_tag_start: str
+    ) -> dict[str, Any] | list[dict[str, Any]]:
+        if pure_json_mode:
+            # Handle pure JSON responses
+            try:
+                return json.loads(response_text)
+            except JSONDecodeError as first_parse_error:
+                # If that didn't work, try finding the { and } to deal with possible ```json fences etc.
+                json_start = response_text.find("{")
+                json_end = response_text.rfind("}")
+                try:
+                    return json.loads(response_text[json_start : json_end + 1])
+                except JSONDecodeError:
+                    # Raise the original error, as it's more likely to be relevant
+                    raise first_parse_error from None
+
+        if output_tag_start not in response_text:
+            raise ValueError(
+                "Response does not contain the expected "
+                f"{output_tag_start}...</json_output> block."
+            )
+        json_output = (
+            response_text.split(output_tag_start, 1)[1]
+            .rsplit("</json_output>", 1)[0]
+            .strip()
+        )
+        return json.loads(json_output)
+
+    def get_collision_proof_output_tag_id(self) -> str:
+        return secrets.token_hex(8)
+
+
+def trim_prompt(s: str) -> str:
+    """Removes indentation up to and including `|` from a multi-line prompt."""
+    lines = s.strip().split("\n")
+    return "\n".join([line.strip().lstrip("|") for line in lines])
 
 
 class AITextGeneratorBlock(AIBlockBase):
@@ -1237,11 +1408,27 @@ class AITextSummarizerBlock(AIBlockBase):
 
     @staticmethod
     def _split_text(text: str, max_tokens: int, overlap: int) -> list[str]:
+        # Security fix: Add validation to prevent DoS attacks
+        # Limit text size to prevent memory exhaustion
+        MAX_TEXT_LENGTH = 1_000_000  # 1MB character limit
+        MAX_CHUNKS = 100  # Maximum number of chunks to prevent excessive memory use
+
+        if len(text) > MAX_TEXT_LENGTH:
+            text = text[:MAX_TEXT_LENGTH]
+
+        # Ensure chunk_size is at least 1 to prevent infinite loops
+        chunk_size = max(1, max_tokens - overlap)
+
+        # Ensure overlap is less than max_tokens to prevent invalid configurations
+        if overlap >= max_tokens:
+            overlap = max(0, max_tokens - 1)
+
         words = text.split()
         chunks = []
-        chunk_size = max_tokens - overlap
 
         for i in range(0, len(words), chunk_size):
+            if len(chunks) >= MAX_CHUNKS:
+                break  # Limit the number of chunks to prevent memory exhaustion
             chunk = " ".join(words[i : i + max_tokens])
             chunks.append(chunk)
 

autogpt_platform/backend/backend/blocks/notion/_api.py

@@ -0,0 +1,536 @@
+"""
+Notion API helper functions and client for making authenticated requests.
+"""
+
+from typing import Any, Dict, List, Optional
+
+from backend.data.model import OAuth2Credentials
+from backend.util.request import Requests
+
+NOTION_VERSION = "2022-06-28"
+
+
+class NotionAPIException(Exception):
+    """Exception raised for Notion API errors."""
+
+    def __init__(self, message: str, status_code: int):
+        super().__init__(message)
+        self.status_code = status_code
+
+
+class NotionClient:
+    """Client for interacting with the Notion API."""
+
+    def __init__(self, credentials: OAuth2Credentials):
+        self.credentials = credentials
+        self.headers = {
+            "Authorization": credentials.auth_header(),
+            "Notion-Version": NOTION_VERSION,
+            "Content-Type": "application/json",
+        }
+        self.requests = Requests()
+
+    async def get_page(self, page_id: str) -> dict:
+        """
+        Fetch a page by ID.
+
+        Args:
+            page_id: The ID of the page to fetch.
+
+        Returns:
+            The page object from Notion API.
+        """
+        url = f"https://api.notion.com/v1/pages/{page_id}"
+        response = await self.requests.get(url, headers=self.headers)
+
+        if not response.ok:
+            raise NotionAPIException(
+                f"Failed to fetch page: {response.status} - {response.text()}",
+                response.status,
+            )
+
+        return response.json()
+
+    async def get_blocks(self, block_id: str, recursive: bool = True) -> List[dict]:
+        """
+        Fetch all blocks from a page or block.
+
+        Args:
+            block_id: The ID of the page or block to fetch children from.
+            recursive: Whether to fetch nested blocks recursively.
+
+        Returns:
+            List of block objects.
+        """
+        blocks = []
+        cursor = None
+
+        while True:
+            url = f"https://api.notion.com/v1/blocks/{block_id}/children"
+            params = {"page_size": 100}
+            if cursor:
+                params["start_cursor"] = cursor
+
+            response = await self.requests.get(url, headers=self.headers, params=params)
+
+            if not response.ok:
+                raise NotionAPIException(
+                    f"Failed to fetch blocks: {response.status} - {response.text()}",
+                    response.status,
+                )
+
+            data = response.json()
+            current_blocks = data.get("results", [])
+
+            # If recursive, fetch children for blocks that have them
+            if recursive:
+                for block in current_blocks:
+                    if block.get("has_children"):
+                        block["children"] = await self.get_blocks(
+                            block["id"], recursive=True
+                        )
+
+            blocks.extend(current_blocks)
+
+            if not data.get("has_more"):
+                break
+            cursor = data.get("next_cursor")
+
+        return blocks
+
+    async def query_database(
+        self,
+        database_id: str,
+        filter_obj: Optional[dict] = None,
+        sorts: Optional[List[dict]] = None,
+        page_size: int = 100,
+    ) -> dict:
+        """
+        Query a database with optional filters and sorts.
+
+        Args:
+            database_id: The ID of the database to query.
+            filter_obj: Optional filter object for the query.
+            sorts: Optional list of sort objects.
+            page_size: Number of results per page.
+
+        Returns:
+            Query results including pages and pagination info.
+        """
+        url = f"https://api.notion.com/v1/databases/{database_id}/query"
+
+        payload: Dict[str, Any] = {"page_size": page_size}
+        if filter_obj:
+            payload["filter"] = filter_obj
+        if sorts:
+            payload["sorts"] = sorts
+
+        response = await self.requests.post(url, headers=self.headers, json=payload)
+
+        if not response.ok:
+            raise NotionAPIException(
+                f"Failed to query database: {response.status} - {response.text()}",
+                response.status,
+            )
+
+        return response.json()
+
+    async def create_page(
+        self,
+        parent: dict,
+        properties: dict,
+        children: Optional[List[dict]] = None,
+        icon: Optional[dict] = None,
+        cover: Optional[dict] = None,
+    ) -> dict:
+        """
+        Create a new page.
+
+        Args:
+            parent: Parent object (page_id or database_id).
+            properties: Page properties.
+            children: Optional list of block children.
+            icon: Optional icon object.
+            cover: Optional cover object.
+
+        Returns:
+            The created page object.
+        """
+        url = "https://api.notion.com/v1/pages"
+
+        payload: Dict[str, Any] = {"parent": parent, "properties": properties}
+
+        if children:
+            payload["children"] = children
+        if icon:
+            payload["icon"] = icon
+        if cover:
+            payload["cover"] = cover
+
+        response = await self.requests.post(url, headers=self.headers, json=payload)
+
+        if not response.ok:
+            raise NotionAPIException(
+                f"Failed to create page: {response.status} - {response.text()}",
+                response.status,
+            )
+
+        return response.json()
+
+    async def update_page(self, page_id: str, properties: dict) -> dict:
+        """
+        Update a page's properties.
+
+        Args:
+            page_id: The ID of the page to update.
+            properties: Properties to update.
+
+        Returns:
+            The updated page object.
+        """
+        url = f"https://api.notion.com/v1/pages/{page_id}"
+
+        response = await self.requests.patch(
+            url, headers=self.headers, json={"properties": properties}
+        )
+
+        if not response.ok:
+            raise NotionAPIException(
+                f"Failed to update page: {response.status} - {response.text()}",
+                response.status,
+            )
+
+        return response.json()
+
+    async def append_blocks(self, block_id: str, children: List[dict]) -> dict:
+        """
+        Append blocks to a page or block.
+
+        Args:
+            block_id: The ID of the page or block to append to.
+            children: List of block objects to append.
+
+        Returns:
+            Response with the created blocks.
+        """
+        url = f"https://api.notion.com/v1/blocks/{block_id}/children"
+
+        response = await self.requests.patch(
+            url, headers=self.headers, json={"children": children}
+        )
+
+        if not response.ok:
+            raise NotionAPIException(
+                f"Failed to append blocks: {response.status} - {response.text()}",
+                response.status,
+            )
+
+        return response.json()
+
+    async def search(
+        self,
+        query: str = "",
+        filter_obj: Optional[dict] = None,
+        sort: Optional[dict] = None,
+        page_size: int = 100,
+    ) -> dict:
+        """
+        Search for pages and databases.
+
+        Args:
+            query: Search query text.
+            filter_obj: Optional filter object.
+            sort: Optional sort object.
+            page_size: Number of results per page.
+
+        Returns:
+            Search results.
+        """
+        url = "https://api.notion.com/v1/search"
+
+        payload: Dict[str, Any] = {"page_size": page_size}
+        if query:
+            payload["query"] = query
+        if filter_obj:
+            payload["filter"] = filter_obj
+        if sort:
+            payload["sort"] = sort
+
+        response = await self.requests.post(url, headers=self.headers, json=payload)
+
+        if not response.ok:
+            raise NotionAPIException(
+                f"Search failed: {response.status} - {response.text()}", response.status
+            )
+
+        return response.json()
+
+
+# Conversion helper functions
+
+
+def parse_rich_text(rich_text_array: List[dict]) -> str:
+    """
+    Extract plain text from a Notion rich text array.
+
+    Args:
+        rich_text_array: Array of rich text objects from Notion.
+
+    Returns:
+        Plain text string.
+    """
+    if not rich_text_array:
+        return ""
+
+    text_parts = []
+    for text_obj in rich_text_array:
+        if "plain_text" in text_obj:
+            text_parts.append(text_obj["plain_text"])
+
+    return "".join(text_parts)
+
+
+def rich_text_to_markdown(rich_text_array: List[dict]) -> str:
+    """
+    Convert Notion rich text array to markdown with formatting.
+
+    Args:
+        rich_text_array: Array of rich text objects from Notion.
+
+    Returns:
+        Markdown formatted string.
+    """
+    if not rich_text_array:
+        return ""
+
+    markdown_parts = []
+
+    for text_obj in rich_text_array:
+        text = text_obj.get("plain_text", "")
+        annotations = text_obj.get("annotations", {})
+
+        # Apply formatting based on annotations
+        if annotations.get("code"):
+            text = f"`{text}`"
+        else:
+            if annotations.get("bold"):
+                text = f"**{text}**"
+            if annotations.get("italic"):
+                text = f"*{text}*"
+            if annotations.get("strikethrough"):
+                text = f"~~{text}~~"
+            if annotations.get("underline"):
+                text = f"<u>{text}</u>"
+
+        # Handle links
+        if text_obj.get("href"):
+            text = f"[{text}]({text_obj['href']})"
+
+        markdown_parts.append(text)
+
+    return "".join(markdown_parts)
+
+
+def block_to_markdown(block: dict, indent_level: int = 0) -> str:
+    """
+    Convert a single Notion block to markdown.
+
+    Args:
+        block: Block object from Notion API.
+        indent_level: Current indentation level for nested blocks.
+
+    Returns:
+        Markdown string representation of the block.
+    """
+    block_type = block.get("type")
+    indent = "  " * indent_level
+    markdown_lines = []
+
+    # Handle different block types
+    if block_type == "paragraph":
+        text = rich_text_to_markdown(block["paragraph"].get("rich_text", []))
+        if text:
+            markdown_lines.append(f"{indent}{text}")
+
+    elif block_type == "heading_1":
+        text = parse_rich_text(block["heading_1"].get("rich_text", []))
+        markdown_lines.append(f"{indent}# {text}")
+
+    elif block_type == "heading_2":
+        text = parse_rich_text(block["heading_2"].get("rich_text", []))
+        markdown_lines.append(f"{indent}## {text}")
+
+    elif block_type == "heading_3":
+        text = parse_rich_text(block["heading_3"].get("rich_text", []))
+        markdown_lines.append(f"{indent}### {text}")
+
+    elif block_type == "bulleted_list_item":
+        text = rich_text_to_markdown(block["bulleted_list_item"].get("rich_text", []))
+        markdown_lines.append(f"{indent}- {text}")
+
+    elif block_type == "numbered_list_item":
+        text = rich_text_to_markdown(block["numbered_list_item"].get("rich_text", []))
+        # Note: This is simplified - proper numbering would need context
+        markdown_lines.append(f"{indent}1. {text}")
+
+    elif block_type == "to_do":
+        text = rich_text_to_markdown(block["to_do"].get("rich_text", []))
+        checked = "x" if block["to_do"].get("checked") else " "
+        markdown_lines.append(f"{indent}- [{checked}] {text}")
+
+    elif block_type == "toggle":
+        text = rich_text_to_markdown(block["toggle"].get("rich_text", []))
+        markdown_lines.append(f"{indent}<details>")
+        markdown_lines.append(f"{indent}<summary>{text}</summary>")
+        markdown_lines.append(f"{indent}")
+        # Process children if they exist
+        if block.get("children"):
+            for child in block["children"]:
+                child_markdown = block_to_markdown(child, indent_level + 1)
+                if child_markdown:
+                    markdown_lines.append(child_markdown)
+        markdown_lines.append(f"{indent}</details>")
+
+    elif block_type == "code":
+        code = parse_rich_text(block["code"].get("rich_text", []))
+        language = block["code"].get("language", "")
+        markdown_lines.append(f"{indent}```{language}")
+        markdown_lines.append(f"{indent}{code}")
+        markdown_lines.append(f"{indent}```")
+
+    elif block_type == "quote":
+        text = rich_text_to_markdown(block["quote"].get("rich_text", []))
+        markdown_lines.append(f"{indent}> {text}")
+
+    elif block_type == "divider":
+        markdown_lines.append(f"{indent}---")
+
+    elif block_type == "image":
+        image = block["image"]
+        url = image.get("external", {}).get("url") or image.get("file", {}).get(
+            "url", ""
+        )
+        caption = parse_rich_text(image.get("caption", []))
+        alt_text = caption if caption else "Image"
+        markdown_lines.append(f"{indent}![{alt_text}]({url})")
+        if caption:
+            markdown_lines.append(f"{indent}*{caption}*")
+
+    elif block_type == "video":
+        video = block["video"]
+        url = video.get("external", {}).get("url") or video.get("file", {}).get(
+            "url", ""
+        )
+        caption = parse_rich_text(video.get("caption", []))
+        markdown_lines.append(f"{indent}[Video]({url})")
+        if caption:
+            markdown_lines.append(f"{indent}*{caption}*")
+
+    elif block_type == "file":
+        file = block["file"]
+        url = file.get("external", {}).get("url") or file.get("file", {}).get("url", "")
+        caption = parse_rich_text(file.get("caption", []))
+        name = caption if caption else "File"
+        markdown_lines.append(f"{indent}[{name}]({url})")
+
+    elif block_type == "bookmark":
+        url = block["bookmark"].get("url", "")
+        caption = parse_rich_text(block["bookmark"].get("caption", []))
+        markdown_lines.append(f"{indent}[{caption if caption else url}]({url})")
+
+    elif block_type == "equation":
+        expression = block["equation"].get("expression", "")
+        markdown_lines.append(f"{indent}$${expression}$$")
+
+    elif block_type == "callout":
+        text = rich_text_to_markdown(block["callout"].get("rich_text", []))
+        icon = block["callout"].get("icon", {})
+        if icon.get("emoji"):
+            markdown_lines.append(f"{indent}> {icon['emoji']} {text}")
+        else:
+            markdown_lines.append(f"{indent}> ℹ️ {text}")
+
+    elif block_type == "child_page":
+        title = block["child_page"].get("title", "Untitled")
+        markdown_lines.append(f"{indent}📄 [{title}](notion://page/{block['id']})")
+
+    elif block_type == "child_database":
+        title = block["child_database"].get("title", "Untitled Database")
+        markdown_lines.append(f"{indent}🗂️ [{title}](notion://database/{block['id']})")
+
+    elif block_type == "table":
+        # Tables are complex - for now just indicate there's a table
+        markdown_lines.append(
+            f"{indent}[Table with {block['table'].get('table_width', 0)} columns]"
+        )
+
+    elif block_type == "column_list":
+        # Process columns
+        if block.get("children"):
+            markdown_lines.append(f"{indent}<div style='display: flex'>")
+            for column in block["children"]:
+                markdown_lines.append(f"{indent}<div style='flex: 1'>")
+                if column.get("children"):
+                    for child in column["children"]:
+                        child_markdown = block_to_markdown(child, indent_level + 1)
+                        if child_markdown:
+                            markdown_lines.append(child_markdown)
+                markdown_lines.append(f"{indent}</div>")
+            markdown_lines.append(f"{indent}</div>")
+
+    # Handle children for blocks that haven't been processed yet
+    elif block.get("children") and block_type not in ["toggle", "column_list"]:
+        for child in block["children"]:
+            child_markdown = block_to_markdown(child, indent_level)
+            if child_markdown:
+                markdown_lines.append(child_markdown)
+
+    return "\n".join(markdown_lines) if markdown_lines else ""
+
+
+def blocks_to_markdown(blocks: List[dict]) -> str:
+    """
+    Convert a list of Notion blocks to a markdown document.
+
+    Args:
+        blocks: List of block objects from Notion API.
+
+    Returns:
+        Complete markdown document as a string.
+    """
+    markdown_parts = []
+
+    for i, block in enumerate(blocks):
+        markdown = block_to_markdown(block)
+        if markdown:
+            markdown_parts.append(markdown)
+            # Add spacing between top-level blocks (except lists)
+            if i < len(blocks) - 1:
+                next_type = blocks[i + 1].get("type", "")
+                current_type = block.get("type", "")
+                # Don't add extra spacing between list items
+                list_types = {"bulleted_list_item", "numbered_list_item", "to_do"}
+                if not (current_type in list_types and next_type in list_types):
+                    markdown_parts.append("")
+
+    return "\n".join(markdown_parts)
+
+
+def extract_page_title(page: dict) -> str:
+    """
+    Extract the title from a Notion page object.
+
+    Args:
+        page: Page object from Notion API.
+
+    Returns:
+        Page title as a string.
+    """
+    properties = page.get("properties", {})
+
+    # Find the title property (it has type "title")
+    for prop_name, prop_value in properties.items():
+        if prop_value.get("type") == "title":
+            return parse_rich_text(prop_value.get("title", []))
+
+    return "Untitled"

autogpt_platform/backend/backend/blocks/notion/_auth.py

@@ -0,0 +1,42 @@
+from typing import Literal
+
+from pydantic import SecretStr
+
+from backend.data.model import CredentialsField, CredentialsMetaInput, OAuth2Credentials
+from backend.integrations.providers import ProviderName
+from backend.util.settings import Secrets
+
+secrets = Secrets()
+NOTION_OAUTH_IS_CONFIGURED = bool(
+    secrets.notion_client_id and secrets.notion_client_secret
+)
+
+NotionCredentials = OAuth2Credentials
+NotionCredentialsInput = CredentialsMetaInput[
+    Literal[ProviderName.NOTION], Literal["oauth2"]
+]
+
+
+def NotionCredentialsField() -> NotionCredentialsInput:
+    """Creates a Notion OAuth2 credentials field."""
+    return CredentialsField(
+        description="Connect your Notion account. Ensure the pages/databases are shared with the integration."
+    )
+
+
+# Test credentials for Notion OAuth2
+TEST_CREDENTIALS = OAuth2Credentials(
+    id="01234567-89ab-cdef-0123-456789abcdef",
+    provider="notion",
+    access_token=SecretStr("test_access_token"),
+    title="Mock Notion OAuth",
+    scopes=["read_content", "insert_content", "update_content"],
+    username="testuser",
+)
+
+TEST_CREDENTIALS_INPUT = {
+    "provider": TEST_CREDENTIALS.provider,
+    "id": TEST_CREDENTIALS.id,
+    "type": TEST_CREDENTIALS.type,
+    "title": TEST_CREDENTIALS.title,
+}

autogpt_platform/backend/backend/blocks/notion/create_page.py

@@ -0,0 +1,360 @@
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional
+
+from pydantic import model_validator
+
+from backend.data.block import Block, BlockCategory, BlockOutput, BlockSchema
+from backend.data.model import OAuth2Credentials, SchemaField
+
+from ._api import NotionClient
+from ._auth import (
+    NOTION_OAUTH_IS_CONFIGURED,
+    TEST_CREDENTIALS,
+    TEST_CREDENTIALS_INPUT,
+    NotionCredentialsField,
+    NotionCredentialsInput,
+)
+
+
+class NotionCreatePageBlock(Block):
+    """Create a new page in Notion with content."""
+
+    class Input(BlockSchema):
+        credentials: NotionCredentialsInput = NotionCredentialsField()
+        parent_page_id: Optional[str] = SchemaField(
+            description="Parent page ID to create the page under. Either this OR parent_database_id is required.",
+            default=None,
+        )
+        parent_database_id: Optional[str] = SchemaField(
+            description="Parent database ID to create the page in. Either this OR parent_page_id is required.",
+            default=None,
+        )
+        title: str = SchemaField(
+            description="Title of the new page",
+        )
+        content: Optional[str] = SchemaField(
+            description="Content for the page. Can be plain text or markdown - will be converted to Notion blocks.",
+            default=None,
+        )
+        properties: Optional[Dict[str, Any]] = SchemaField(
+            description="Additional properties for database pages (e.g., {'Status': 'In Progress', 'Priority': 'High'})",
+            default=None,
+        )
+        icon_emoji: Optional[str] = SchemaField(
+            description="Emoji to use as the page icon (e.g., '📄', '🚀')", default=None
+        )
+
+        @model_validator(mode="after")
+        def validate_parent(self):
+            """Ensure either parent_page_id or parent_database_id is provided."""
+            if not self.parent_page_id and not self.parent_database_id:
+                raise ValueError(
+                    "Either parent_page_id or parent_database_id must be provided"
+                )
+            if self.parent_page_id and self.parent_database_id:
+                raise ValueError(
+                    "Only one of parent_page_id or parent_database_id should be provided, not both"
+                )
+            return self
+
+    class Output(BlockSchema):
+        page_id: str = SchemaField(description="ID of the created page.")
+        page_url: str = SchemaField(description="URL of the created page.")
+        error: str = SchemaField(description="Error message if the operation failed.")
+
+    def __init__(self):
+        super().__init__(
+            id="c15febe0-66ce-4c6f-aebd-5ab351653804",
+            description="Create a new page in Notion. Requires EITHER a parent_page_id OR parent_database_id. Supports markdown content.",
+            categories={BlockCategory.PRODUCTIVITY},
+            input_schema=NotionCreatePageBlock.Input,
+            output_schema=NotionCreatePageBlock.Output,
+            disabled=not NOTION_OAUTH_IS_CONFIGURED,
+            test_input={
+                "parent_page_id": "00000000-0000-0000-0000-000000000000",
+                "title": "Test Page",
+                "content": "This is test content.",
+                "credentials": TEST_CREDENTIALS_INPUT,
+            },
+            test_output=[
+                ("page_id", "12345678-1234-1234-1234-123456789012"),
+                (
+                    "page_url",
+                    "https://notion.so/Test-Page-12345678123412341234123456789012",
+                ),
+            ],
+            test_credentials=TEST_CREDENTIALS,
+            test_mock={
+                "create_page": lambda *args, **kwargs: (
+                    "12345678-1234-1234-1234-123456789012",
+                    "https://notion.so/Test-Page-12345678123412341234123456789012",
+                )
+            },
+        )
+
+    @staticmethod
+    def _markdown_to_blocks(content: str) -> List[dict]:
+        """Convert markdown content to Notion block objects."""
+        if not content:
+            return []
+
+        blocks = []
+        lines = content.split("\n")
+        i = 0
+
+        while i < len(lines):
+            line = lines[i]
+
+            # Skip empty lines
+            if not line.strip():
+                i += 1
+                continue
+
+            # Headings
+            if line.startswith("### "):
+                blocks.append(
+                    {
+                        "type": "heading_3",
+                        "heading_3": {
+                            "rich_text": [
+                                {"type": "text", "text": {"content": line[4:].strip()}}
+                            ]
+                        },
+                    }
+                )
+            elif line.startswith("## "):
+                blocks.append(
+                    {
+                        "type": "heading_2",
+                        "heading_2": {
+                            "rich_text": [
+                                {"type": "text", "text": {"content": line[3:].strip()}}
+                            ]
+                        },
+                    }
+                )
+            elif line.startswith("# "):
+                blocks.append(
+                    {
+                        "type": "heading_1",
+                        "heading_1": {
+                            "rich_text": [
+                                {"type": "text", "text": {"content": line[2:].strip()}}
+                            ]
+                        },
+                    }
+                )
+            # Bullet points
+            elif line.strip().startswith("- "):
+                blocks.append(
+                    {
+                        "type": "bulleted_list_item",
+                        "bulleted_list_item": {
+                            "rich_text": [
+                                {
+                                    "type": "text",
+                                    "text": {"content": line.strip()[2:].strip()},
+                                }
+                            ]
+                        },
+                    }
+                )
+            # Numbered list
+            elif line.strip() and line.strip()[0].isdigit() and ". " in line:
+                content_start = line.find(". ") + 2
+                blocks.append(
+                    {
+                        "type": "numbered_list_item",
+                        "numbered_list_item": {
+                            "rich_text": [
+                                {
+                                    "type": "text",
+                                    "text": {"content": line[content_start:].strip()},
+                                }
+                            ]
+                        },
+                    }
+                )
+            # Code block
+            elif line.strip().startswith("```"):
+                code_lines = []
+                language = line[3:].strip() or "plain text"
+                i += 1
+                while i < len(lines) and not lines[i].strip().startswith("```"):
+                    code_lines.append(lines[i])
+                    i += 1
+                blocks.append(
+                    {
+                        "type": "code",
+                        "code": {
+                            "rich_text": [
+                                {
+                                    "type": "text",
+                                    "text": {"content": "\n".join(code_lines)},
+                                }
+                            ],
+                            "language": language,
+                        },
+                    }
+                )
+            # Quote
+            elif line.strip().startswith("> "):
+                blocks.append(
+                    {
+                        "type": "quote",
+                        "quote": {
+                            "rich_text": [
+                                {
+                                    "type": "text",
+                                    "text": {"content": line.strip()[2:].strip()},
+                                }
+                            ]
+                        },
+                    }
+                )
+            # Horizontal rule
+            elif line.strip() in ["---", "***", "___"]:
+                blocks.append({"type": "divider", "divider": {}})
+            # Regular paragraph
+            else:
+                # Parse for basic markdown formatting
+                text_content = line.strip()
+                rich_text = []
+
+                # Simple bold/italic parsing (this is simplified)
+                if "**" in text_content or "*" in text_content:
+                    # For now, just pass as plain text
+                    # A full implementation would parse and create proper annotations
+                    rich_text = [{"type": "text", "text": {"content": text_content}}]
+                else:
+                    rich_text = [{"type": "text", "text": {"content": text_content}}]
+
+                blocks.append(
+                    {"type": "paragraph", "paragraph": {"rich_text": rich_text}}
+                )
+
+            i += 1
+
+        return blocks
+
+    @staticmethod
+    def _build_properties(
+        title: str, additional_properties: Optional[Dict[str, Any]] = None
+    ) -> Dict[str, Any]:
+        """Build properties object for page creation."""
+        properties: Dict[str, Any] = {
+            "title": {"title": [{"type": "text", "text": {"content": title}}]}
+        }
+
+        if additional_properties:
+            for key, value in additional_properties.items():
+                if key.lower() == "title":
+                    continue  # Skip title as we already have it
+
+                # Try to intelligently map property types
+                if isinstance(value, bool):
+                    properties[key] = {"checkbox": value}
+                elif isinstance(value, (int, float)):
+                    properties[key] = {"number": value}
+                elif isinstance(value, list):
+                    # Assume multi-select
+                    properties[key] = {
+                        "multi_select": [{"name": str(item)} for item in value]
+                    }
+                elif isinstance(value, str):
+                    # Could be select, rich_text, or other types
+                    # For simplicity, try common patterns
+                    if key.lower() in ["status", "priority", "type", "category"]:
+                        properties[key] = {"select": {"name": value}}
+                    elif key.lower() in ["url", "link"]:
+                        properties[key] = {"url": value}
+                    elif key.lower() in ["email"]:
+                        properties[key] = {"email": value}
+                    else:
+                        properties[key] = {
+                            "rich_text": [{"type": "text", "text": {"content": value}}]
+                        }
+
+        return properties
+
+    @staticmethod
+    async def create_page(
+        credentials: OAuth2Credentials,
+        title: str,
+        parent_page_id: Optional[str] = None,
+        parent_database_id: Optional[str] = None,
+        content: Optional[str] = None,
+        properties: Optional[Dict[str, Any]] = None,
+        icon_emoji: Optional[str] = None,
+    ) -> tuple[str, str]:
+        """
+        Create a new Notion page.
+
+        Returns:
+            Tuple of (page_id, page_url)
+        """
+        if not parent_page_id and not parent_database_id:
+            raise ValueError(
+                "Either parent_page_id or parent_database_id must be provided"
+            )
+        if parent_page_id and parent_database_id:
+            raise ValueError(
+                "Only one of parent_page_id or parent_database_id should be provided, not both"
+            )
+
+        client = NotionClient(credentials)
+
+        # Build parent object
+        if parent_page_id:
+            parent = {"type": "page_id", "page_id": parent_page_id}
+        else:
+            parent = {"type": "database_id", "database_id": parent_database_id}
+
+        # Build properties
+        page_properties = NotionCreatePageBlock._build_properties(title, properties)
+
+        # Convert content to blocks if provided
+        children = None
+        if content:
+            children = NotionCreatePageBlock._markdown_to_blocks(content)
+
+        # Build icon if provided
+        icon = None
+        if icon_emoji:
+            icon = {"type": "emoji", "emoji": icon_emoji}
+
+        # Create the page
+        result = await client.create_page(
+            parent=parent, properties=page_properties, children=children, icon=icon
+        )
+
+        page_id = result.get("id", "")
+        page_url = result.get("url", "")
+
+        if not page_id or not page_url:
+            raise ValueError("Failed to get page ID or URL from Notion response")
+
+        return page_id, page_url
+
+    async def run(
+        self,
+        input_data: Input,
+        *,
+        credentials: OAuth2Credentials,
+        **kwargs,
+    ) -> BlockOutput:
+        try:
+            page_id, page_url = await self.create_page(
+                credentials,
+                input_data.title,
+                input_data.parent_page_id,
+                input_data.parent_database_id,
+                input_data.content,
+                input_data.properties,
+                input_data.icon_emoji,
+            )
+            yield "page_id", page_id
+            yield "page_url", page_url
+        except Exception as e:
+            yield "error", str(e) if str(e) else "Unknown error"

autogpt_platform/backend/backend/blocks/notion/read_database.py

@@ -0,0 +1,285 @@
+from __future__ import annotations
+
+from typing import Any, Dict, List, Optional
+
+from backend.data.block import Block, BlockCategory, BlockOutput, BlockSchema
+from backend.data.model import OAuth2Credentials, SchemaField
+
+from ._api import NotionClient, parse_rich_text
+from ._auth import (
+    NOTION_OAUTH_IS_CONFIGURED,
+    TEST_CREDENTIALS,
+    TEST_CREDENTIALS_INPUT,
+    NotionCredentialsField,
+    NotionCredentialsInput,
+)
+
+
+class NotionReadDatabaseBlock(Block):
+    """Query a Notion database and retrieve entries with their properties."""
+
+    class Input(BlockSchema):
+        credentials: NotionCredentialsInput = NotionCredentialsField()
+        database_id: str = SchemaField(
+            description="Notion database ID. Must be accessible by the connected integration.",
+        )
+        filter_property: Optional[str] = SchemaField(
+            description="Property name to filter by (e.g., 'Status', 'Priority')",
+            default=None,
+        )
+        filter_value: Optional[str] = SchemaField(
+            description="Value to filter for in the specified property", default=None
+        )
+        sort_property: Optional[str] = SchemaField(
+            description="Property name to sort by", default=None
+        )
+        sort_direction: Optional[str] = SchemaField(
+            description="Sort direction: 'ascending' or 'descending'",
+            default="ascending",
+        )
+        limit: int = SchemaField(
+            description="Maximum number of entries to retrieve",
+            default=100,
+            ge=1,
+            le=100,
+        )
+
+    class Output(BlockSchema):
+        entries: List[Dict[str, Any]] = SchemaField(
+            description="List of database entries with their properties."
+        )
+        entry: Dict[str, Any] = SchemaField(
+            description="Individual database entry (yields one per entry found)."
+        )
+        entry_ids: List[str] = SchemaField(
+            description="List of entry IDs for batch operations."
+        )
+        entry_id: str = SchemaField(
+            description="Individual entry ID (yields one per entry found)."
+        )
+        count: int = SchemaField(description="Number of entries retrieved.")
+        database_title: str = SchemaField(description="Title of the database.")
+        error: str = SchemaField(description="Error message if the operation failed.")
+
+    def __init__(self):
+        super().__init__(
+            id="fcd53135-88c9-4ba3-be50-cc6936286e6c",
+            description="Query a Notion database with optional filtering and sorting, returning structured entries.",
+            categories={BlockCategory.PRODUCTIVITY},
+            input_schema=NotionReadDatabaseBlock.Input,
+            output_schema=NotionReadDatabaseBlock.Output,
+            disabled=not NOTION_OAUTH_IS_CONFIGURED,
+            test_input={
+                "database_id": "00000000-0000-0000-0000-000000000000",
+                "limit": 10,
+                "credentials": TEST_CREDENTIALS_INPUT,
+            },
+            test_output=[
+                (
+                    "entries",
+                    [{"Name": "Test Entry", "Status": "Active", "_id": "test-123"}],
+                ),
+                ("entry_ids", ["test-123"]),
+                (
+                    "entry",
+                    {"Name": "Test Entry", "Status": "Active", "_id": "test-123"},
+                ),
+                ("entry_id", "test-123"),
+                ("count", 1),
+                ("database_title", "Test Database"),
+            ],
+            test_credentials=TEST_CREDENTIALS,
+            test_mock={
+                "query_database": lambda *args, **kwargs: (
+                    [{"Name": "Test Entry", "Status": "Active", "_id": "test-123"}],
+                    1,
+                    "Test Database",
+                )
+            },
+        )
+
+    @staticmethod
+    def _parse_property_value(prop: dict) -> Any:
+        """Parse a Notion property value into a simple Python type."""
+        prop_type = prop.get("type")
+
+        if prop_type == "title":
+            return parse_rich_text(prop.get("title", []))
+        elif prop_type == "rich_text":
+            return parse_rich_text(prop.get("rich_text", []))
+        elif prop_type == "number":
+            return prop.get("number")
+        elif prop_type == "select":
+            select = prop.get("select")
+            return select.get("name") if select else None
+        elif prop_type == "multi_select":
+            return [item.get("name") for item in prop.get("multi_select", [])]
+        elif prop_type == "date":
+            date = prop.get("date")
+            if date:
+                return date.get("start")
+            return None
+        elif prop_type == "checkbox":
+            return prop.get("checkbox", False)
+        elif prop_type == "url":
+            return prop.get("url")
+        elif prop_type == "email":
+            return prop.get("email")
+        elif prop_type == "phone_number":
+            return prop.get("phone_number")
+        elif prop_type == "people":
+            return [
+                person.get("name", person.get("id"))
+                for person in prop.get("people", [])
+            ]
+        elif prop_type == "files":
+            files = prop.get("files", [])
+            return [
+                f.get(
+                    "name",
+                    f.get("external", {}).get("url", f.get("file", {}).get("url")),
+                )
+                for f in files
+            ]
+        elif prop_type == "relation":
+            return [rel.get("id") for rel in prop.get("relation", [])]
+        elif prop_type == "formula":
+            formula = prop.get("formula", {})
+            return formula.get(formula.get("type"))
+        elif prop_type == "rollup":
+            rollup = prop.get("rollup", {})
+            return rollup.get(rollup.get("type"))
+        elif prop_type == "created_time":
+            return prop.get("created_time")
+        elif prop_type == "created_by":
+            return prop.get("created_by", {}).get(
+                "name", prop.get("created_by", {}).get("id")
+            )
+        elif prop_type == "last_edited_time":
+            return prop.get("last_edited_time")
+        elif prop_type == "last_edited_by":
+            return prop.get("last_edited_by", {}).get(
+                "name", prop.get("last_edited_by", {}).get("id")
+            )
+        else:
+            # Return the raw value for unknown types
+            return prop
+
+    @staticmethod
+    def _build_filter(property_name: str, value: str) -> dict:
+        """Build a simple filter object for a property."""
+        # This is a simplified filter - in reality, you'd need to know the property type
+        # For now, we'll try common filter types
+        return {
+            "or": [
+                {"property": property_name, "rich_text": {"contains": value}},
+                {"property": property_name, "title": {"contains": value}},
+                {"property": property_name, "select": {"equals": value}},
+                {"property": property_name, "multi_select": {"contains": value}},
+            ]
+        }
+
+    @staticmethod
+    async def query_database(
+        credentials: OAuth2Credentials,
+        database_id: str,
+        filter_property: Optional[str] = None,
+        filter_value: Optional[str] = None,
+        sort_property: Optional[str] = None,
+        sort_direction: str = "ascending",
+        limit: int = 100,
+    ) -> tuple[List[Dict[str, Any]], int, str]:
+        """
+        Query a Notion database and parse the results.
+
+        Returns:
+            Tuple of (entries_list, count, database_title)
+        """
+        client = NotionClient(credentials)
+
+        # Build filter if specified
+        filter_obj = None
+        if filter_property and filter_value:
+            filter_obj = NotionReadDatabaseBlock._build_filter(
+                filter_property, filter_value
+            )
+
+        # Build sorts if specified
+        sorts = None
+        if sort_property:
+            sorts = [{"property": sort_property, "direction": sort_direction}]
+
+        # Query the database
+        result = await client.query_database(
+            database_id, filter_obj=filter_obj, sorts=sorts, page_size=limit
+        )
+
+        # Parse the entries
+        entries = []
+        for page in result.get("results", []):
+            entry = {}
+            properties = page.get("properties", {})
+
+            for prop_name, prop_value in properties.items():
+                entry[prop_name] = NotionReadDatabaseBlock._parse_property_value(
+                    prop_value
+                )
+
+            # Add metadata
+            entry["_id"] = page.get("id")
+            entry["_url"] = page.get("url")
+            entry["_created_time"] = page.get("created_time")
+            entry["_last_edited_time"] = page.get("last_edited_time")
+
+            entries.append(entry)
+
+        # Get database title (we need to make a separate call for this)
+        try:
+            database_url = f"https://api.notion.com/v1/databases/{database_id}"
+            db_response = await client.requests.get(
+                database_url, headers=client.headers
+            )
+            if db_response.ok:
+                db_data = db_response.json()
+                db_title = parse_rich_text(db_data.get("title", []))
+            else:
+                db_title = "Unknown Database"
+        except Exception:
+            db_title = "Unknown Database"
+
+        return entries, len(entries), db_title
+
+    async def run(
+        self,
+        input_data: Input,
+        *,
+        credentials: OAuth2Credentials,
+        **kwargs,
+    ) -> BlockOutput:
+        try:
+            entries, count, db_title = await self.query_database(
+                credentials,
+                input_data.database_id,
+                input_data.filter_property,
+                input_data.filter_value,
+                input_data.sort_property,
+                input_data.sort_direction or "ascending",
+                input_data.limit,
+            )
+            # Yield the complete list for batch operations
+            yield "entries", entries
+
+            # Extract and yield IDs as a list for batch operations
+            entry_ids = [entry["_id"] for entry in entries if "_id" in entry]
+            yield "entry_ids", entry_ids
+
+            # Yield each individual entry and its ID for single connections
+            for entry in entries:
+                yield "entry", entry
+                if "_id" in entry:
+                    yield "entry_id", entry["_id"]
+
+            yield "count", count
+            yield "database_title", db_title
+        except Exception as e:
+            yield "error", str(e) if str(e) else "Unknown error"

autogpt_platform/backend/backend/blocks/notion/read_page.py

@@ -0,0 +1,64 @@
+from __future__ import annotations
+
+from backend.data.block import Block, BlockCategory, BlockOutput, BlockSchema
+from backend.data.model import OAuth2Credentials, SchemaField
+
+from ._api import NotionClient
+from ._auth import (
+    NOTION_OAUTH_IS_CONFIGURED,
+    TEST_CREDENTIALS,
+    TEST_CREDENTIALS_INPUT,
+    NotionCredentialsField,
+    NotionCredentialsInput,
+)
+
+
+class NotionReadPageBlock(Block):
+    """Read a Notion page by ID and return its raw JSON."""
+
+    class Input(BlockSchema):
+        credentials: NotionCredentialsInput = NotionCredentialsField()
+        page_id: str = SchemaField(
+            description="Notion page ID. Must be accessible by the connected integration. You can get this from the page URL notion.so/A-Page-586edd711467478da59fe3ce29a1ffab would be 586edd711467478da59fe35e29a1ffab",
+        )
+
+    class Output(BlockSchema):
+        page: dict = SchemaField(description="Raw Notion page JSON.")
+        error: str = SchemaField(description="Error message if the operation failed.")
+
+    def __init__(self):
+        super().__init__(
+            id="5246cc1d-34b7-452b-8fc5-3fb25fd8f542",
+            description="Read a Notion page by its ID and return its raw JSON.",
+            categories={BlockCategory.PRODUCTIVITY},
+            input_schema=NotionReadPageBlock.Input,
+            output_schema=NotionReadPageBlock.Output,
+            disabled=not NOTION_OAUTH_IS_CONFIGURED,
+            test_input={
+                "page_id": "00000000-0000-0000-0000-000000000000",
+                "credentials": TEST_CREDENTIALS_INPUT,
+            },
+            test_output=[("page", dict)],
+            test_credentials=TEST_CREDENTIALS,
+            test_mock={
+                "get_page": lambda *args, **kwargs: {"object": "page", "id": "mocked"}
+            },
+        )
+
+    @staticmethod
+    async def get_page(credentials: OAuth2Credentials, page_id: str) -> dict:
+        client = NotionClient(credentials)
+        return await client.get_page(page_id)
+
+    async def run(
+        self,
+        input_data: Input,
+        *,
+        credentials: OAuth2Credentials,
+        **kwargs,
+    ) -> BlockOutput:
+        try:
+            page = await self.get_page(credentials, input_data.page_id)
+            yield "page", page
+        except Exception as e:
+            yield "error", str(e) if str(e) else "Unknown error"

autogpt_platform/backend/backend/blocks/notion/read_page_markdown.py

@@ -0,0 +1,109 @@
+from __future__ import annotations
+
+from backend.data.block import Block, BlockCategory, BlockOutput, BlockSchema
+from backend.data.model import OAuth2Credentials, SchemaField
+
+from ._api import NotionClient, blocks_to_markdown, extract_page_title
+from ._auth import (
+    NOTION_OAUTH_IS_CONFIGURED,
+    TEST_CREDENTIALS,
+    TEST_CREDENTIALS_INPUT,
+    NotionCredentialsField,
+    NotionCredentialsInput,
+)
+
+
+class NotionReadPageMarkdownBlock(Block):
+    """Read a Notion page and convert it to clean Markdown format."""
+
+    class Input(BlockSchema):
+        credentials: NotionCredentialsInput = NotionCredentialsField()
+        page_id: str = SchemaField(
+            description="Notion page ID. Must be accessible by the connected integration. You can get this from the page URL notion.so/A-Page-586edd711467478da59fe35e29a1ffab would be 586edd711467478da59fe35e29a1ffab",
+        )
+        include_title: bool = SchemaField(
+            description="Whether to include the page title as a header in the markdown",
+            default=True,
+        )
+
+    class Output(BlockSchema):
+        markdown: str = SchemaField(description="Page content in Markdown format.")
+        title: str = SchemaField(description="Page title.")
+        error: str = SchemaField(description="Error message if the operation failed.")
+
+    def __init__(self):
+        super().__init__(
+            id="d1312c4d-fae2-4e70-893d-f4d07cce1d4e",
+            description="Read a Notion page and convert it to Markdown format with proper formatting for headings, lists, links, and rich text.",
+            categories={BlockCategory.PRODUCTIVITY},
+            input_schema=NotionReadPageMarkdownBlock.Input,
+            output_schema=NotionReadPageMarkdownBlock.Output,
+            disabled=not NOTION_OAUTH_IS_CONFIGURED,
+            test_input={
+                "page_id": "00000000-0000-0000-0000-000000000000",
+                "include_title": True,
+                "credentials": TEST_CREDENTIALS_INPUT,
+            },
+            test_output=[
+                ("markdown", "# Test Page\n\nThis is test content."),
+                ("title", "Test Page"),
+            ],
+            test_credentials=TEST_CREDENTIALS,
+            test_mock={
+                "get_page_markdown": lambda *args, **kwargs: (
+                    "# Test Page\n\nThis is test content.",
+                    "Test Page",
+                )
+            },
+        )
+
+    @staticmethod
+    async def get_page_markdown(
+        credentials: OAuth2Credentials, page_id: str, include_title: bool = True
+    ) -> tuple[str, str]:
+        """
+        Get a Notion page and convert it to markdown.
+
+        Args:
+            credentials: OAuth2 credentials for Notion.
+            page_id: The ID of the page to fetch.
+            include_title: Whether to include the page title in the markdown.
+
+        Returns:
+            Tuple of (markdown_content, title)
+        """
+        client = NotionClient(credentials)
+
+        # Get page metadata
+        page = await client.get_page(page_id)
+        title = extract_page_title(page)
+
+        # Get all blocks from the page
+        blocks = await client.get_blocks(page_id, recursive=True)
+
+        # Convert blocks to markdown
+        content_markdown = blocks_to_markdown(blocks)
+
+        # Combine title and content if requested
+        if include_title and title:
+            full_markdown = f"# {title}\n\n{content_markdown}"
+        else:
+            full_markdown = content_markdown
+
+        return full_markdown, title
+
+    async def run(
+        self,
+        input_data: Input,
+        *,
+        credentials: OAuth2Credentials,
+        **kwargs,
+    ) -> BlockOutput:
+        try:
+            markdown, title = await self.get_page_markdown(
+                credentials, input_data.page_id, input_data.include_title
+            )
+            yield "markdown", markdown
+            yield "title", title
+        except Exception as e:
+            yield "error", str(e) if str(e) else "Unknown error"

autogpt_platform/backend/backend/blocks/notion/search.py

@@ -0,0 +1,225 @@
+from __future__ import annotations
+
+from typing import List, Optional
+
+from pydantic import BaseModel
+
+from backend.data.block import Block, BlockCategory, BlockOutput, BlockSchema
+from backend.data.model import OAuth2Credentials, SchemaField
+
+from ._api import NotionClient, extract_page_title, parse_rich_text
+from ._auth import (
+    NOTION_OAUTH_IS_CONFIGURED,
+    TEST_CREDENTIALS,
+    TEST_CREDENTIALS_INPUT,
+    NotionCredentialsField,
+    NotionCredentialsInput,
+)
+
+
+class NotionSearchResult(BaseModel):
+    """Typed model for Notion search results."""
+
+    id: str
+    type: str  # 'page' or 'database'
+    title: str
+    url: str
+    created_time: Optional[str] = None
+    last_edited_time: Optional[str] = None
+    parent_type: Optional[str] = None  # 'page', 'database', or 'workspace'
+    parent_id: Optional[str] = None
+    icon: Optional[str] = None  # emoji icon if present
+    is_inline: Optional[bool] = None  # for databases only
+
+
+class NotionSearchBlock(Block):
+    """Search across your Notion workspace for pages and databases."""
+
+    class Input(BlockSchema):
+        credentials: NotionCredentialsInput = NotionCredentialsField()
+        query: str = SchemaField(
+            description="Search query text. Leave empty to get all accessible pages/databases.",
+            default="",
+        )
+        filter_type: Optional[str] = SchemaField(
+            description="Filter results by type: 'page' or 'database'. Leave empty for both.",
+            default=None,
+        )
+        limit: int = SchemaField(
+            description="Maximum number of results to return", default=20, ge=1, le=100
+        )
+
+    class Output(BlockSchema):
+        results: List[NotionSearchResult] = SchemaField(
+            description="List of search results with title, type, URL, and metadata."
+        )
+        result: NotionSearchResult = SchemaField(
+            description="Individual search result (yields one per result found)."
+        )
+        result_ids: List[str] = SchemaField(
+            description="List of IDs from search results for batch operations."
+        )
+        count: int = SchemaField(description="Number of results found.")
+        error: str = SchemaField(description="Error message if the operation failed.")
+
+    def __init__(self):
+        super().__init__(
+            id="313515dd-9848-46ea-9cd6-3c627c892c56",
+            description="Search your Notion workspace for pages and databases by text query.",
+            categories={BlockCategory.PRODUCTIVITY, BlockCategory.SEARCH},
+            input_schema=NotionSearchBlock.Input,
+            output_schema=NotionSearchBlock.Output,
+            disabled=not NOTION_OAUTH_IS_CONFIGURED,
+            test_input={
+                "query": "project",
+                "limit": 5,
+                "credentials": TEST_CREDENTIALS_INPUT,
+            },
+            test_output=[
+                (
+                    "results",
+                    [
+                        NotionSearchResult(
+                            id="123",
+                            type="page",
+                            title="Project Plan",
+                            url="https://notion.so/Project-Plan-123",
+                        )
+                    ],
+                ),
+                ("result_ids", ["123"]),
+                (
+                    "result",
+                    NotionSearchResult(
+                        id="123",
+                        type="page",
+                        title="Project Plan",
+                        url="https://notion.so/Project-Plan-123",
+                    ),
+                ),
+                ("count", 1),
+            ],
+            test_credentials=TEST_CREDENTIALS,
+            test_mock={
+                "search_workspace": lambda *args, **kwargs: (
+                    [
+                        NotionSearchResult(
+                            id="123",
+                            type="page",
+                            title="Project Plan",
+                            url="https://notion.so/Project-Plan-123",
+                        )
+                    ],
+                    1,
+                )
+            },
+        )
+
+    @staticmethod
+    async def search_workspace(
+        credentials: OAuth2Credentials,
+        query: str = "",
+        filter_type: Optional[str] = None,
+        limit: int = 20,
+    ) -> tuple[List[NotionSearchResult], int]:
+        """
+        Search the Notion workspace.
+
+        Returns:
+            Tuple of (results_list, count)
+        """
+        client = NotionClient(credentials)
+
+        # Build filter if type is specified
+        filter_obj = None
+        if filter_type:
+            filter_obj = {"property": "object", "value": filter_type}
+
+        # Execute search
+        response = await client.search(
+            query=query, filter_obj=filter_obj, page_size=limit
+        )
+
+        # Parse results
+        results = []
+        for item in response.get("results", []):
+            result_data = {
+                "id": item.get("id", ""),
+                "type": item.get("object", ""),
+                "url": item.get("url", ""),
+                "created_time": item.get("created_time"),
+                "last_edited_time": item.get("last_edited_time"),
+                "title": "",  # Will be set below
+            }
+
+            # Extract title based on type
+            if item.get("object") == "page":
+                # For pages, get the title from properties
+                result_data["title"] = extract_page_title(item)
+
+                # Add parent info
+                parent = item.get("parent", {})
+                if parent.get("type") == "page_id":
+                    result_data["parent_type"] = "page"
+                    result_data["parent_id"] = parent.get("page_id")
+                elif parent.get("type") == "database_id":
+                    result_data["parent_type"] = "database"
+                    result_data["parent_id"] = parent.get("database_id")
+                elif parent.get("type") == "workspace":
+                    result_data["parent_type"] = "workspace"
+
+                # Add icon if present
+                icon = item.get("icon")
+                if icon and icon.get("type") == "emoji":
+                    result_data["icon"] = icon.get("emoji")
+
+            elif item.get("object") == "database":
+                # For databases, get title from the title array
+                result_data["title"] = parse_rich_text(item.get("title", []))
+
+                # Add database-specific metadata
+                result_data["is_inline"] = item.get("is_inline", False)
+
+                # Add parent info
+                parent = item.get("parent", {})
+                if parent.get("type") == "page_id":
+                    result_data["parent_type"] = "page"
+                    result_data["parent_id"] = parent.get("page_id")
+                elif parent.get("type") == "workspace":
+                    result_data["parent_type"] = "workspace"
+
+                # Add icon if present
+                icon = item.get("icon")
+                if icon and icon.get("type") == "emoji":
+                    result_data["icon"] = icon.get("emoji")
+
+            results.append(NotionSearchResult(**result_data))
+
+        return results, len(results)
+
+    async def run(
+        self,
+        input_data: Input,
+        *,
+        credentials: OAuth2Credentials,
+        **kwargs,
+    ) -> BlockOutput:
+        try:
+            results, count = await self.search_workspace(
+                credentials, input_data.query, input_data.filter_type, input_data.limit
+            )
+
+            # Yield the complete list for batch operations
+            yield "results", results
+
+            # Extract and yield IDs as a list for batch operations
+            result_ids = [r.id for r in results]
+            yield "result_ids", result_ids
+
+            # Yield each individual result for single connections
+            for result in results:
+                yield "result", result
+
+            yield "count", count
+        except Exception as e:
+            yield "error", str(e) if str(e) else "Unknown error"

autogpt_platform/backend/backend/blocks/perplexity.py

@@ -0,0 +1,226 @@
+# flake8: noqa: E501
+import logging
+from enum import Enum
+from typing import Any, Literal
+
+import openai
+from pydantic import SecretStr
+
+from backend.data.block import Block, BlockCategory, BlockOutput, BlockSchema
+from backend.data.model import (
+    APIKeyCredentials,
+    CredentialsField,
+    CredentialsMetaInput,
+    NodeExecutionStats,
+    SchemaField,
+)
+from backend.integrations.providers import ProviderName
+from backend.util.logging import TruncatedLogger
+
+logger = TruncatedLogger(logging.getLogger(__name__), "[Perplexity-Block]")
+
+
+class PerplexityModel(str, Enum):
+    """Perplexity sonar models available via OpenRouter"""
+
+    SONAR = "perplexity/sonar"
+    SONAR_PRO = "perplexity/sonar-pro"
+    SONAR_DEEP_RESEARCH = "perplexity/sonar-deep-research"
+
+
+PerplexityCredentials = CredentialsMetaInput[
+    Literal[ProviderName.OPEN_ROUTER], Literal["api_key"]
+]
+
+TEST_CREDENTIALS = APIKeyCredentials(
+    id="test-perplexity-creds",
+    provider="open_router",
+    api_key=SecretStr("mock-openrouter-api-key"),
+    title="Mock OpenRouter API key",
+    expires_at=None,
+)
+TEST_CREDENTIALS_INPUT = {
+    "provider": TEST_CREDENTIALS.provider,
+    "id": TEST_CREDENTIALS.id,
+    "type": TEST_CREDENTIALS.type,
+    "title": TEST_CREDENTIALS.title,
+}
+
+
+def PerplexityCredentialsField() -> PerplexityCredentials:
+    return CredentialsField(
+        description="OpenRouter API key for accessing Perplexity models.",
+    )
+
+
+class PerplexityBlock(Block):
+    class Input(BlockSchema):
+        prompt: str = SchemaField(
+            description="The query to send to the Perplexity model.",
+            placeholder="Enter your query here...",
+        )
+        model: PerplexityModel = SchemaField(
+            title="Perplexity Model",
+            default=PerplexityModel.SONAR,
+            description="The Perplexity sonar model to use.",
+            advanced=False,
+        )
+        credentials: PerplexityCredentials = PerplexityCredentialsField()
+        system_prompt: str = SchemaField(
+            title="System Prompt",
+            default="",
+            description="Optional system prompt to provide context to the model.",
+            advanced=True,
+        )
+        max_tokens: int | None = SchemaField(
+            advanced=True,
+            default=None,
+            description="The maximum number of tokens to generate.",
+        )
+
+    class Output(BlockSchema):
+        response: str = SchemaField(
+            description="The response from the Perplexity model."
+        )
+        annotations: list[dict[str, Any]] = SchemaField(
+            description="List of URL citations and annotations from the response."
+        )
+        error: str = SchemaField(description="Error message if the API call failed.")
+
+    def __init__(self):
+        super().__init__(
+            id="c8a5f2e9-8b3d-4a7e-9f6c-1d5e3c9b7a4f",
+            description="Query Perplexity's sonar models with real-time web search capabilities and receive annotated responses with source citations.",
+            categories={BlockCategory.AI, BlockCategory.SEARCH},
+            input_schema=PerplexityBlock.Input,
+            output_schema=PerplexityBlock.Output,
+            test_input={
+                "prompt": "What is the weather today?",
+                "model": PerplexityModel.SONAR,
+                "credentials": TEST_CREDENTIALS_INPUT,
+            },
+            test_credentials=TEST_CREDENTIALS,
+            test_output=[
+                ("response", "The weather varies by location..."),
+                ("annotations", list),
+            ],
+            test_mock={
+                "call_perplexity": lambda *args, **kwargs: {
+                    "response": "The weather varies by location...",
+                    "annotations": [
+                        {
+                            "type": "url_citation",
+                            "url_citation": {
+                                "title": "weather.com",
+                                "url": "https://weather.com",
+                            },
+                        }
+                    ],
+                }
+            },
+        )
+        self.execution_stats = NodeExecutionStats()
+
+    async def call_perplexity(
+        self,
+        credentials: APIKeyCredentials,
+        model: PerplexityModel,
+        prompt: str,
+        system_prompt: str = "",
+        max_tokens: int | None = None,
+    ) -> dict[str, Any]:
+        """Call Perplexity via OpenRouter and extract annotations."""
+        client = openai.AsyncOpenAI(
+            base_url="https://openrouter.ai/api/v1",
+            api_key=credentials.api_key.get_secret_value(),
+        )
+
+        messages = []
+        if system_prompt:
+            messages.append({"role": "system", "content": system_prompt})
+        messages.append({"role": "user", "content": prompt})
+
+        try:
+            response = await client.chat.completions.create(
+                extra_headers={
+                    "HTTP-Referer": "https://agpt.co",
+                    "X-Title": "AutoGPT",
+                },
+                model=model.value,
+                messages=messages,
+                max_tokens=max_tokens,
+            )
+
+            if not response.choices:
+                raise ValueError("No response from Perplexity via OpenRouter.")
+
+            # Extract the response content
+            response_content = response.choices[0].message.content or ""
+
+            # Extract annotations if present in the message
+            annotations = []
+            if hasattr(response.choices[0].message, "annotations"):
+                # If annotations are directly available
+                annotations = response.choices[0].message.annotations
+            else:
+                # Check if there's a raw response with annotations
+                raw = getattr(response.choices[0].message, "_raw_response", None)
+                if isinstance(raw, dict) and "annotations" in raw:
+                    annotations = raw["annotations"]
+
+            if not annotations and hasattr(response, "model_extra"):
+                # Check model_extra for annotations
+                model_extra = response.model_extra
+                if isinstance(model_extra, dict):
+                    # Check in choices
+                    if "choices" in model_extra and len(model_extra["choices"]) > 0:
+                        choice = model_extra["choices"][0]
+                        if "message" in choice and "annotations" in choice["message"]:
+                            annotations = choice["message"]["annotations"]
+
+            # Also check the raw response object for annotations
+            if not annotations:
+                raw = getattr(response, "_raw_response", None)
+                if isinstance(raw, dict):
+                    # Check various possible locations for annotations
+                    if "annotations" in raw:
+                        annotations = raw["annotations"]
+                    elif "choices" in raw and len(raw["choices"]) > 0:
+                        choice = raw["choices"][0]
+                        if "message" in choice and "annotations" in choice["message"]:
+                            annotations = choice["message"]["annotations"]
+
+            # Update execution stats
+            if response.usage:
+                self.execution_stats.input_token_count = response.usage.prompt_tokens
+                self.execution_stats.output_token_count = (
+                    response.usage.completion_tokens
+                )
+
+            return {"response": response_content, "annotations": annotations or []}
+
+        except Exception as e:
+            logger.error(f"Error calling Perplexity: {e}")
+            raise
+
+    async def run(
+        self, input_data: Input, *, credentials: APIKeyCredentials, **kwargs
+    ) -> BlockOutput:
+        logger.debug(f"Running Perplexity block with model: {input_data.model}")
+
+        try:
+            result = await self.call_perplexity(
+                credentials=credentials,
+                model=input_data.model,
+                prompt=input_data.prompt,
+                system_prompt=input_data.system_prompt,
+                max_tokens=input_data.max_tokens,
+            )
+
+            yield "response", result["response"]
+            yield "annotations", result["annotations"]
+
+        except Exception as e:
+            error_msg = f"Error calling Perplexity: {str(e)}"
+            logger.error(error_msg)
+            yield "error", error_msg

autogpt_platform/backend/backend/blocks/rss.py

@@ -1,4 +1,5 @@
 import asyncio
+import logging
 from datetime import datetime, timedelta, timezone
 from typing import Any
 
@@ -7,6 +8,7 @@ import pydantic
 
 from backend.data.block import Block, BlockCategory, BlockOutput, BlockSchema
 from backend.data.model import SchemaField
+from backend.util.request import Requests
 
 
 class RSSEntry(pydantic.BaseModel):
@@ -100,8 +102,33 @@ class ReadRSSFeedBlock(Block):
         )
 
     @staticmethod
-    def parse_feed(url: str) -> dict[str, Any]:
-        return feedparser.parse(url)  # type: ignore
+    async def parse_feed(url: str) -> dict[str, Any]:
+        # Security fix: Add protection against memory exhaustion attacks
+        MAX_FEED_SIZE = 10 * 1024 * 1024  # 10MB limit for RSS feeds
+
+        # Download feed content with size limit
+        try:
+            response = await Requests(raise_for_status=True).get(url)
+
+            # Check content length if available
+            content_length = response.headers.get("Content-Length")
+            if content_length and int(content_length) > MAX_FEED_SIZE:
+                raise ValueError(
+                    f"Feed too large: {content_length} bytes exceeds {MAX_FEED_SIZE} limit"
+                )
+
+            # Get content with size limit
+            content = response.content
+            if len(content) > MAX_FEED_SIZE:
+                raise ValueError(f"Feed too large: exceeds {MAX_FEED_SIZE} byte limit")
+
+            # Parse with feedparser using the validated content
+            # feedparser has built-in protection against XML attacks
+            return feedparser.parse(content)  # type: ignore
+        except Exception as e:
+            # Log error and return empty feed
+            logging.warning(f"Failed to parse RSS feed from {url}: {e}")
+            return {"entries": []}
 
     async def run(self, input_data: Input, **kwargs) -> BlockOutput:
         keep_going = True
@@ -111,7 +138,7 @@ class ReadRSSFeedBlock(Block):
         while keep_going:
             keep_going = input_data.run_continuously
 
-            feed = self.parse_feed(input_data.rss_url)
+            feed = await self.parse_feed(input_data.rss_url)
             all_entries = []
 
             for entry in feed["entries"]:

autogpt_platform/backend/backend/blocks/smart_decision_maker.py

@@ -13,6 +13,11 @@ from backend.data.block import (
     BlockSchema,
     BlockType,
 )
+from backend.data.dynamic_fields import (
+    extract_base_field_name,
+    get_dynamic_field_description,
+    is_dynamic_field,
+)
 from backend.data.model import NodeExecutionStats, SchemaField
 from backend.util import json
 from backend.util.clients import get_database_manager_async_client
@@ -98,6 +103,22 @@ def _create_tool_response(call_id: str, output: Any) -> dict[str, Any]:
     return {"role": "tool", "tool_call_id": call_id, "content": content}
 
 
+def _convert_raw_response_to_dict(raw_response: Any) -> dict[str, Any]:
+    """
+    Safely convert raw_response to dictionary format for conversation history.
+    Handles different response types from different LLM providers.
+    """
+    if isinstance(raw_response, str):
+        # Ollama returns a string, convert to dict format
+        return {"role": "assistant", "content": raw_response}
+    elif isinstance(raw_response, dict):
+        # Already a dict (from tests or some providers)
+        return raw_response
+    else:
+        # OpenAI/Anthropic return objects, convert with json.to_dict
+        return json.to_dict(raw_response)
+
+
 def get_pending_tool_calls(conversation_history: list[Any]) -> dict[str, int]:
     """
     All the tool calls entry in the conversation history requires a response.
@@ -261,6 +282,7 @@ class SmartDecisionMakerBlock(Block):
 
     @staticmethod
     def cleanup(s: str):
+        """Clean up block names for use as tool function names."""
         return re.sub(r"[^a-zA-Z0-9_-]", "_", s).lower()
 
     @staticmethod
@@ -288,41 +310,66 @@ class SmartDecisionMakerBlock(Block):
         }
         sink_block_input_schema = block.input_schema
         properties = {}
+        field_mapping = {}  # clean_name -> original_name
 
         for link in links:
-            sink_name = SmartDecisionMakerBlock.cleanup(link.sink_name)
+            field_name = link.sink_name
+            is_dynamic = is_dynamic_field(field_name)
+            # Clean property key to ensure Anthropic API compatibility for ALL fields
+            clean_field_name = SmartDecisionMakerBlock.cleanup(field_name)
+            field_mapping[clean_field_name] = field_name
 
-            # Handle dynamic fields (e.g., values_#_*, items_$_*, etc.)
-            # These are fields that get merged by the executor into their base field
-            if (
-                "_#_" in link.sink_name
-                or "_$_" in link.sink_name
-                or "_@_" in link.sink_name
-            ):
-                # For dynamic fields, provide a generic string schema
-                # The executor will handle merging these into the appropriate structure
-                properties[sink_name] = {
+            if is_dynamic:
+                # For dynamic fields, use cleaned name but preserve original in description
+                properties[clean_field_name] = {
                     "type": "string",
-                    "description": f"Dynamic value for {link.sink_name}",
+                    "description": get_dynamic_field_description(field_name),
                 }
             else:
-                # For regular fields, use the block's schema
+                # For regular fields, use the block's schema directly
                 try:
-                    properties[sink_name] = sink_block_input_schema.get_field_schema(
-                        link.sink_name
+                    properties[clean_field_name] = (
+                        sink_block_input_schema.get_field_schema(field_name)
                     )
                 except (KeyError, AttributeError):
-                    # If the field doesn't exist in the schema, provide a generic schema
-                    properties[sink_name] = {
+                    # If field doesn't exist in schema, provide a generic one
+                    properties[clean_field_name] = {
                         "type": "string",
-                        "description": f"Value for {link.sink_name}",
+                        "description": f"Value for {field_name}",
                     }
 
+        # Build the parameters schema using a single unified path
+        base_schema = block.input_schema.jsonschema()
+        base_required = set(base_schema.get("required", []))
+
+        # Compute required fields at the leaf level:
+        # - If a linked field is dynamic and its base is required in the block schema, require the leaf
+        # - If a linked field is regular and is required in the block schema, require the leaf
+        required_fields: set[str] = set()
+        for link in links:
+            field_name = link.sink_name
+            is_dynamic = is_dynamic_field(field_name)
+            # Always use cleaned field name for property key (Anthropic API compliance)
+            clean_field_name = SmartDecisionMakerBlock.cleanup(field_name)
+
+            if is_dynamic:
+                base_name = extract_base_field_name(field_name)
+                if base_name in base_required:
+                    required_fields.add(clean_field_name)
+            else:
+                if field_name in base_required:
+                    required_fields.add(clean_field_name)
+
         tool_function["parameters"] = {
-            **block.input_schema.jsonschema(),
+            "type": "object",
             "properties": properties,
+            "additionalProperties": False,
+            "required": sorted(required_fields),
         }
 
+        # Store field mapping for later use in output processing
+        tool_function["_field_mapping"] = field_mapping
+
         return {"type": "function", "function": tool_function}
 
     @staticmethod
@@ -366,13 +413,12 @@ class SmartDecisionMakerBlock(Block):
             sink_block_properties = sink_block_input_schema.get("properties", {}).get(
                 link.sink_name, {}
             )
-            sink_name = SmartDecisionMakerBlock.cleanup(link.sink_name)
             description = (
                 sink_block_properties["description"]
                 if "description" in sink_block_properties
                 else f"The {link.sink_name} of the tool"
             )
-            properties[sink_name] = {
+            properties[link.sink_name] = {
                 "type": "string",
                 "description": description,
                 "default": json.dumps(sink_block_properties.get("default", None)),
@@ -388,24 +434,17 @@ class SmartDecisionMakerBlock(Block):
         return {"type": "function", "function": tool_function}
 
     @staticmethod
-    async def _create_function_signature(node_id: str) -> list[dict[str, Any]]:
+    async def _create_function_signature(
+        node_id: str,
+    ) -> list[dict[str, Any]]:
         """
-        Creates function signatures for tools linked to a specified node within a graph.
-
-        This method filters the graph links to identify those that are tools and are
-        connected to the given node_id. It then constructs function signatures for each
-        tool based on the metadata and input schema of the linked nodes.
+        Creates function signatures for connected tools.
 
         Args:
             node_id: The node_id for which to create function signatures.
 
         Returns:
-            list[dict[str, Any]]: A list of dictionaries, each representing a function signature
-                                  for a tool, including its name, description, and parameters.
-
-        Raises:
-            ValueError: If no tool links are found for the specified node_id, or if a sink node
-                        or its metadata cannot be found.
+            List of function signatures for tools
         """
         db_client = get_database_manager_async_client()
         tools = [
@@ -430,20 +469,116 @@ class SmartDecisionMakerBlock(Block):
                 raise ValueError(f"Sink node not found: {links[0].sink_id}")
 
             if sink_node.block_id == AgentExecutorBlock().id:
-                return_tool_functions.append(
+                tool_func = (
                     await SmartDecisionMakerBlock._create_agent_function_signature(
                         sink_node, links
                     )
                 )
+                return_tool_functions.append(tool_func)
             else:
-                return_tool_functions.append(
+                tool_func = (
                     await SmartDecisionMakerBlock._create_block_function_signature(
                         sink_node, links
                     )
                 )
+                return_tool_functions.append(tool_func)
 
         return return_tool_functions
 
+    async def _attempt_llm_call_with_validation(
+        self,
+        credentials: llm.APIKeyCredentials,
+        input_data: Input,
+        current_prompt: list[dict],
+        tool_functions: list[dict[str, Any]],
+    ):
+        """
+        Attempt a single LLM call with tool validation.
+
+        Returns the response if successful, raises ValueError if validation fails.
+        """
+        resp = await llm.llm_call(
+            credentials=credentials,
+            llm_model=input_data.model,
+            prompt=current_prompt,
+            max_tokens=input_data.max_tokens,
+            tools=tool_functions,
+            ollama_host=input_data.ollama_host,
+            parallel_tool_calls=input_data.multiple_tool_calls,
+        )
+
+        # Track LLM usage stats per call
+        self.merge_stats(
+            NodeExecutionStats(
+                input_token_count=resp.prompt_tokens,
+                output_token_count=resp.completion_tokens,
+                llm_call_count=1,
+            )
+        )
+
+        if not resp.tool_calls:
+            return resp
+        validation_errors_list: list[str] = []
+        for tool_call in resp.tool_calls:
+            tool_name = tool_call.function.name
+            try:
+                tool_args = json.loads(tool_call.function.arguments)
+            except Exception as e:
+                validation_errors_list.append(
+                    f"Tool call '{tool_name}' has invalid JSON arguments: {e}"
+                )
+                continue
+
+            # Find the tool definition to get the expected arguments
+            tool_def = next(
+                (
+                    tool
+                    for tool in tool_functions
+                    if tool["function"]["name"] == tool_name
+                ),
+                None,
+            )
+            if tool_def is None and len(tool_functions) == 1:
+                tool_def = tool_functions[0]
+
+            # Get parameters schema from tool definition
+            if (
+                tool_def
+                and "function" in tool_def
+                and "parameters" in tool_def["function"]
+            ):
+                parameters = tool_def["function"]["parameters"]
+                expected_args = parameters.get("properties", {})
+                required_params = set(parameters.get("required", []))
+            else:
+                expected_args = {arg: {} for arg in tool_args.keys()}
+                required_params = set()
+
+            # Validate tool call arguments
+            provided_args = set(tool_args.keys())
+            expected_args_set = set(expected_args.keys())
+
+            # Check for unexpected arguments (typos)
+            unexpected_args = provided_args - expected_args_set
+            # Only check for missing REQUIRED parameters
+            missing_required_args = required_params - provided_args
+
+            if unexpected_args or missing_required_args:
+                error_msg = f"Tool call '{tool_name}' has parameter errors:"
+                if unexpected_args:
+                    error_msg += f" Unknown parameters: {sorted(unexpected_args)}."
+                if missing_required_args:
+                    error_msg += f" Missing required parameters: {sorted(missing_required_args)}."
+                error_msg += f" Expected parameters: {sorted(expected_args_set)}."
+                if required_params:
+                    error_msg += f" Required parameters: {sorted(required_params)}."
+                validation_errors_list.append(error_msg)
+
+        if validation_errors_list:
+            raise ValueError("; ".join(validation_errors_list))
+
+        return resp
+
     async def run(
         self,
         input_data: Input,
@@ -466,27 +601,19 @@ class SmartDecisionMakerBlock(Block):
         if pending_tool_calls and input_data.last_tool_output is None:
             raise ValueError(f"Tool call requires an output for {pending_tool_calls}")
 
-        # Only assign the last tool output to the first pending tool call
         tool_output = []
         if pending_tool_calls and input_data.last_tool_output is not None:
-            # Get the first pending tool call ID
             first_call_id = next(iter(pending_tool_calls.keys()))
             tool_output.append(
                 _create_tool_response(first_call_id, input_data.last_tool_output)
             )
 
-            # Add tool output to prompt right away
             prompt.extend(tool_output)
-
-            # Check if there are still pending tool calls after handling the first one
             remaining_pending_calls = get_pending_tool_calls(prompt)
 
-            # If there are still pending tool calls, yield the conversation and return early
             if remaining_pending_calls:
                 yield "conversations", prompt
                 return
-
-        # Fallback on adding tool output in the conversation history as user prompt.
         elif input_data.last_tool_output:
             logger.error(
                 f"[SmartDecisionMakerBlock-node_exec_id={node_exec_id}] "
@@ -519,25 +646,33 @@ class SmartDecisionMakerBlock(Block):
         ):
             prompt.append({"role": "user", "content": prefix + input_data.prompt})
 
-        response = await llm.llm_call(
-            credentials=credentials,
-            llm_model=input_data.model,
-            prompt=prompt,
-            json_format=False,
-            max_tokens=input_data.max_tokens,
-            tools=tool_functions,
-            ollama_host=input_data.ollama_host,
-            parallel_tool_calls=input_data.multiple_tool_calls,
-        )
+        current_prompt = list(prompt)
+        max_attempts = max(1, int(input_data.retry))
+        response = None
 
-        # Track LLM usage stats
-        self.merge_stats(
-            NodeExecutionStats(
-                input_token_count=response.prompt_tokens,
-                output_token_count=response.completion_tokens,
-                llm_call_count=1,
+        last_error = None
+        for attempt in range(max_attempts):
+            try:
+                response = await self._attempt_llm_call_with_validation(
+                    credentials, input_data, current_prompt, tool_functions
+                )
+                break
+
+            except ValueError as e:
+                last_error = e
+                error_feedback = (
+                    "Your tool call had parameter errors. Please fix the following issues and try again:\n"
+                    + f"- {str(e)}\n"
+                    + "\nPlease make sure to use the exact parameter names as specified in the function schema."
+                )
+                current_prompt = list(current_prompt) + [
+                    {"role": "user", "content": error_feedback}
+                ]
+
+        if response is None:
+            raise last_error or ValueError(
+                "Failed to get valid response after all retry attempts"
             )
-        )
 
         if not response.tool_calls:
             yield "finished", response.response
@@ -547,7 +682,6 @@ class SmartDecisionMakerBlock(Block):
             tool_name = tool_call.function.name
             tool_args = json.loads(tool_call.function.arguments)
 
-            # Find the tool definition to get the expected arguments
             tool_def = next(
                 (
                     tool
@@ -556,7 +690,6 @@ class SmartDecisionMakerBlock(Block):
                 ),
                 None,
             )
-
             if (
                 tool_def
                 and "function" in tool_def
@@ -564,20 +697,38 @@ class SmartDecisionMakerBlock(Block):
             ):
                 expected_args = tool_def["function"]["parameters"].get("properties", {})
             else:
-                expected_args = tool_args.keys()
+                expected_args = {arg: {} for arg in tool_args.keys()}
 
-            # Yield provided arguments and None for missing ones
-            for arg_name in expected_args:
-                if arg_name in tool_args:
-                    yield f"tools_^_{tool_name}_~_{arg_name}", tool_args[arg_name]
-                else:
-                    yield f"tools_^_{tool_name}_~_{arg_name}", None
+            # Get field mapping from tool definition
+            field_mapping = (
+                tool_def.get("function", {}).get("_field_mapping", {})
+                if tool_def
+                else {}
+            )
+
+            for clean_arg_name in expected_args:
+                # arg_name is now always the cleaned field name (for Anthropic API compliance)
+                # Get the original field name from field mapping for proper emit key generation
+                original_field_name = field_mapping.get(clean_arg_name, clean_arg_name)
+                arg_value = tool_args.get(clean_arg_name)
+
+                sanitized_tool_name = self.cleanup(tool_name)
+                sanitized_arg_name = self.cleanup(original_field_name)
+                emit_key = f"tools_^_{sanitized_tool_name}_~_{sanitized_arg_name}"
+
+                logger.debug(
+                    "[SmartDecisionMakerBlock|geid:%s|neid:%s] emit %s",
+                    graph_exec_id,
+                    node_exec_id,
+                    emit_key,
+                )
+                yield emit_key, arg_value
 
-        # Add reasoning to conversation history if available
         if response.reasoning:
             prompt.append(
                 {"role": "assistant", "content": f"[Reasoning]: {response.reasoning}"}
             )
 
-        prompt.append(response.raw_response)
+        prompt.append(_convert_raw_response_to_dict(response.raw_response))
+
         yield "conversations", prompt

autogpt_platform/backend/backend/blocks/stagehand/_config.py

@@ -0,0 +1,8 @@
+from backend.sdk import BlockCostType, ProviderBuilder
+
+stagehand = (
+    ProviderBuilder("stagehand")
+    .with_api_key("STAGEHAND_API_KEY", "Stagehand API Key")
+    .with_base_cost(1, BlockCostType.RUN)
+    .build()
+)

autogpt_platform/backend/backend/blocks/stagehand/blocks.py

@@ -0,0 +1,393 @@
+import logging
+import signal
+import threading
+from contextlib import contextmanager
+from enum import Enum
+
+# Monkey patch Stagehands to prevent signal handling in worker threads
+import stagehand.main
+from stagehand import Stagehand
+
+from backend.blocks.llm import (
+    MODEL_METADATA,
+    AICredentials,
+    AICredentialsField,
+    LlmModel,
+    ModelMetadata,
+)
+from backend.blocks.stagehand._config import stagehand as stagehand_provider
+from backend.sdk import (
+    APIKeyCredentials,
+    Block,
+    BlockCategory,
+    BlockOutput,
+    BlockSchema,
+    CredentialsMetaInput,
+    SchemaField,
+)
+
+# Store the original method
+original_register_signal_handlers = stagehand.main.Stagehand._register_signal_handlers
+
+
+def safe_register_signal_handlers(self):
+    """Only register signal handlers in the main thread"""
+    if threading.current_thread() is threading.main_thread():
+        original_register_signal_handlers(self)
+    else:
+        # Skip signal handling in worker threads
+        pass
+
+
+# Replace the method
+stagehand.main.Stagehand._register_signal_handlers = safe_register_signal_handlers
+
+
+@contextmanager
+def disable_signal_handling():
+    """Context manager to temporarily disable signal handling"""
+    if threading.current_thread() is not threading.main_thread():
+        # In worker threads, temporarily replace signal.signal with a no-op
+        original_signal = signal.signal
+
+        def noop_signal(*args, **kwargs):
+            pass
+
+        signal.signal = noop_signal
+        try:
+            yield
+        finally:
+            signal.signal = original_signal
+    else:
+        # In main thread, don't modify anything
+        yield
+
+
+logger = logging.getLogger(__name__)
+
+
+class StagehandRecommendedLlmModel(str, Enum):
+    """
+    This is subset of LLModel from autogpt_platform/backend/backend/blocks/llm.py
+
+    It contains only the models recommended by Stagehand
+    """
+
+    # OpenAI
+    GPT41 = "gpt-4.1-2025-04-14"
+    GPT41_MINI = "gpt-4.1-mini-2025-04-14"
+
+    # Anthropic
+    CLAUDE_3_7_SONNET = "claude-3-7-sonnet-20250219"
+
+    @property
+    def provider_name(self) -> str:
+        """
+        Returns the provider name for the model in the required format for Stagehand:
+        provider/model_name
+        """
+        model_metadata = MODEL_METADATA[LlmModel(self.value)]
+        model_name = self.value
+
+        if len(model_name.split("/")) == 1 and not self.value.startswith(
+            model_metadata.provider
+        ):
+            assert (
+                model_metadata.provider != "open_router"
+            ), "Logic failed and open_router provider attempted to be prepended to model name! in stagehand/_config.py"
+            model_name = f"{model_metadata.provider}/{model_name}"
+
+        logger.error(f"Model name: {model_name}")
+        return model_name
+
+    @property
+    def provider(self) -> str:
+        return MODEL_METADATA[LlmModel(self.value)].provider
+
+    @property
+    def metadata(self) -> ModelMetadata:
+        return MODEL_METADATA[LlmModel(self.value)]
+
+    @property
+    def context_window(self) -> int:
+        return MODEL_METADATA[LlmModel(self.value)].context_window
+
+    @property
+    def max_output_tokens(self) -> int | None:
+        return MODEL_METADATA[LlmModel(self.value)].max_output_tokens
+
+
+class StagehandObserveBlock(Block):
+    class Input(BlockSchema):
+        # Browserbase credentials (Stagehand provider) or raw API key
+        stagehand_credentials: CredentialsMetaInput = (
+            stagehand_provider.credentials_field(
+                description="Stagehand/Browserbase API key"
+            )
+        )
+        browserbase_project_id: str = SchemaField(
+            description="Browserbase project ID (required if using Browserbase)",
+        )
+        # Model selection and credentials (provider-discriminated like llm.py)
+        model: StagehandRecommendedLlmModel = SchemaField(
+            title="LLM Model",
+            description="LLM to use for Stagehand (provider is inferred)",
+            default=StagehandRecommendedLlmModel.CLAUDE_3_7_SONNET,
+            advanced=False,
+        )
+        model_credentials: AICredentials = AICredentialsField()
+        url: str = SchemaField(
+            description="URL to navigate to.",
+        )
+        instruction: str = SchemaField(
+            description="Natural language description of elements or actions to discover.",
+        )
+        iframes: bool = SchemaField(
+            description="Whether to search within iframes. If True, Stagehand will search for actions within iframes.",
+            default=True,
+        )
+        domSettleTimeoutMs: int = SchemaField(
+            description="Timeout in milliseconds for DOM settlement.Wait longer for dynamic content",
+            default=45000,
+        )
+
+    class Output(BlockSchema):
+        selector: str = SchemaField(description="XPath selector to locate element.")
+        description: str = SchemaField(description="Human-readable description")
+        method: str | None = SchemaField(description="Suggested action method")
+        arguments: list[str] | None = SchemaField(
+            description="Additional action parameters"
+        )
+
+    def __init__(self):
+        super().__init__(
+            id="d3863944-0eaf-45c4-a0c9-63e0fe1ee8b9",
+            description="Find suggested actions for your workflows",
+            categories={BlockCategory.AI, BlockCategory.DEVELOPER_TOOLS},
+            input_schema=StagehandObserveBlock.Input,
+            output_schema=StagehandObserveBlock.Output,
+        )
+
+    async def run(
+        self,
+        input_data: Input,
+        *,
+        stagehand_credentials: APIKeyCredentials,
+        model_credentials: APIKeyCredentials,
+        **kwargs,
+    ) -> BlockOutput:
+
+        logger.info(f"OBSERVE: Stagehand credentials: {stagehand_credentials}")
+        logger.info(
+            f"OBSERVE: Model credentials: {model_credentials} for provider {model_credentials.provider} secret: {model_credentials.api_key.get_secret_value()}"
+        )
+
+        with disable_signal_handling():
+            stagehand = Stagehand(
+                api_key=stagehand_credentials.api_key.get_secret_value(),
+                project_id=input_data.browserbase_project_id,
+                model_name=input_data.model.provider_name,
+                model_api_key=model_credentials.api_key.get_secret_value(),
+            )
+
+            await stagehand.init()
+
+        page = stagehand.page
+
+        assert page is not None, "Stagehand page is not initialized"
+
+        await page.goto(input_data.url)
+
+        observe_results = await page.observe(
+            input_data.instruction,
+            iframes=input_data.iframes,
+            domSettleTimeoutMs=input_data.domSettleTimeoutMs,
+        )
+        for result in observe_results:
+            yield "selector", result.selector
+            yield "description", result.description
+            yield "method", result.method
+            yield "arguments", result.arguments
+
+
+class StagehandActBlock(Block):
+    class Input(BlockSchema):
+        # Browserbase credentials (Stagehand provider) or raw API key
+        stagehand_credentials: CredentialsMetaInput = (
+            stagehand_provider.credentials_field(
+                description="Stagehand/Browserbase API key"
+            )
+        )
+        browserbase_project_id: str = SchemaField(
+            description="Browserbase project ID (required if using Browserbase)",
+        )
+        # Model selection and credentials (provider-discriminated like llm.py)
+        model: StagehandRecommendedLlmModel = SchemaField(
+            title="LLM Model",
+            description="LLM to use for Stagehand (provider is inferred)",
+            default=StagehandRecommendedLlmModel.CLAUDE_3_7_SONNET,
+            advanced=False,
+        )
+        model_credentials: AICredentials = AICredentialsField()
+        url: str = SchemaField(
+            description="URL to navigate to.",
+        )
+        action: list[str] = SchemaField(
+            description="Action to perform. Suggested actions are: click, fill, type, press, scroll, select from dropdown. For multi-step actions, add an entry for each step.",
+        )
+        variables: dict[str, str] = SchemaField(
+            description="Variables to use in the action. Variables contains data you want the action to use.",
+            default_factory=dict,
+        )
+        iframes: bool = SchemaField(
+            description="Whether to search within iframes. If True, Stagehand will search for actions within iframes.",
+            default=True,
+        )
+        domSettleTimeoutMs: int = SchemaField(
+            description="Timeout in milliseconds for DOM settlement.Wait longer for dynamic content",
+            default=45000,
+        )
+        timeoutMs: int = SchemaField(
+            description="Timeout in milliseconds for DOM ready. Extended timeout for slow-loading forms",
+            default=60000,
+        )
+
+    class Output(BlockSchema):
+        success: bool = SchemaField(
+            description="Whether the action was completed successfully"
+        )
+        message: str = SchemaField(description="Details about the action’s execution.")
+        action: str = SchemaField(description="Action performed")
+
+    def __init__(self):
+        super().__init__(
+            id="86eba68b-9549-4c0b-a0db-47d85a56cc27",
+            description="Interact with a web page by performing actions on a web page. Use it to build self-healing and deterministic automations that adapt to website chang.",
+            categories={BlockCategory.AI, BlockCategory.DEVELOPER_TOOLS},
+            input_schema=StagehandActBlock.Input,
+            output_schema=StagehandActBlock.Output,
+        )
+
+    async def run(
+        self,
+        input_data: Input,
+        *,
+        stagehand_credentials: APIKeyCredentials,
+        model_credentials: APIKeyCredentials,
+        **kwargs,
+    ) -> BlockOutput:
+
+        logger.info(f"ACT: Stagehand credentials: {stagehand_credentials}")
+        logger.info(
+            f"ACT: Model credentials: {model_credentials} for provider {model_credentials.provider} secret: {model_credentials.api_key.get_secret_value()}"
+        )
+
+        with disable_signal_handling():
+            stagehand = Stagehand(
+                api_key=stagehand_credentials.api_key.get_secret_value(),
+                project_id=input_data.browserbase_project_id,
+                model_name=input_data.model.provider_name,
+                model_api_key=model_credentials.api_key.get_secret_value(),
+            )
+
+            await stagehand.init()
+
+        page = stagehand.page
+
+        assert page is not None, "Stagehand page is not initialized"
+
+        await page.goto(input_data.url)
+        for action in input_data.action:
+            action_results = await page.act(
+                action,
+                variables=input_data.variables,
+                iframes=input_data.iframes,
+                domSettleTimeoutMs=input_data.domSettleTimeoutMs,
+                timeoutMs=input_data.timeoutMs,
+            )
+            yield "success", action_results.success
+            yield "message", action_results.message
+            yield "action", action_results.action
+
+
+class StagehandExtractBlock(Block):
+    class Input(BlockSchema):
+        # Browserbase credentials (Stagehand provider) or raw API key
+        stagehand_credentials: CredentialsMetaInput = (
+            stagehand_provider.credentials_field(
+                description="Stagehand/Browserbase API key"
+            )
+        )
+        browserbase_project_id: str = SchemaField(
+            description="Browserbase project ID (required if using Browserbase)",
+        )
+        # Model selection and credentials (provider-discriminated like llm.py)
+        model: StagehandRecommendedLlmModel = SchemaField(
+            title="LLM Model",
+            description="LLM to use for Stagehand (provider is inferred)",
+            default=StagehandRecommendedLlmModel.CLAUDE_3_7_SONNET,
+            advanced=False,
+        )
+        model_credentials: AICredentials = AICredentialsField()
+        url: str = SchemaField(
+            description="URL to navigate to.",
+        )
+        instruction: str = SchemaField(
+            description="Natural language description of elements or actions to discover.",
+        )
+        iframes: bool = SchemaField(
+            description="Whether to search within iframes. If True, Stagehand will search for actions within iframes.",
+            default=True,
+        )
+        domSettleTimeoutMs: int = SchemaField(
+            description="Timeout in milliseconds for DOM settlement.Wait longer for dynamic content",
+            default=45000,
+        )
+
+    class Output(BlockSchema):
+        extraction: str = SchemaField(description="Extracted data from the page.")
+
+    def __init__(self):
+        super().__init__(
+            id="fd3c0b18-2ba6-46ae-9339-fcb40537ad98",
+            description="Extract structured data from a webpage.",
+            categories={BlockCategory.AI, BlockCategory.DEVELOPER_TOOLS},
+            input_schema=StagehandExtractBlock.Input,
+            output_schema=StagehandExtractBlock.Output,
+        )
+
+    async def run(
+        self,
+        input_data: Input,
+        *,
+        stagehand_credentials: APIKeyCredentials,
+        model_credentials: APIKeyCredentials,
+        **kwargs,
+    ) -> BlockOutput:
+
+        logger.info(f"EXTRACT: Stagehand credentials: {stagehand_credentials}")
+        logger.info(
+            f"EXTRACT: Model credentials: {model_credentials} for provider {model_credentials.provider} secret: {model_credentials.api_key.get_secret_value()}"
+        )
+
+        with disable_signal_handling():
+            stagehand = Stagehand(
+                api_key=stagehand_credentials.api_key.get_secret_value(),
+                project_id=input_data.browserbase_project_id,
+                model_name=input_data.model.provider_name,
+                model_api_key=model_credentials.api_key.get_secret_value(),
+            )
+
+            await stagehand.init()
+
+        page = stagehand.page
+
+        assert page is not None, "Stagehand page is not initialized"
+
+        await page.goto(input_data.url)
+        extraction = await page.extract(
+            input_data.instruction,
+            iframes=input_data.iframes,
+            domSettleTimeoutMs=input_data.domSettleTimeoutMs,
+        )
+        yield "extraction", str(extraction.model_dump()["extraction"])

autogpt_platform/backend/backend/blocks/test/test_block.py

@@ -19,7 +19,7 @@ async def test_block_ids_valid(block: Type[Block]):
     # Skip list for blocks with known invalid UUIDs
     skip_blocks = {
         "GetWeatherInformationBlock",
-        "CodeExecutionBlock",
+        "ExecuteCodeBlock",
         "CountdownTimerBlock",
         "TwitterGetListTweetsBlock",
         "TwitterRemoveListMemberBlock",

autogpt_platform/backend/backend/blocks/test/test_blocks_dos_vulnerability.py

@@ -0,0 +1,269 @@
+"""
+Test security fixes for various DoS vulnerabilities.
+"""
+
+import asyncio
+from unittest.mock import patch
+
+import pytest
+
+from backend.blocks.code_extraction_block import CodeExtractionBlock
+from backend.blocks.iteration import StepThroughItemsBlock
+from backend.blocks.llm import AITextSummarizerBlock
+from backend.blocks.text import ExtractTextInformationBlock
+from backend.blocks.xml_parser import XMLParserBlock
+from backend.util.file import store_media_file
+from backend.util.type import MediaFileType
+
+
+class TestCodeExtractionBlockSecurity:
+    """Test ReDoS fixes in CodeExtractionBlock."""
+
+    async def test_redos_protection(self):
+        """Test that the regex patterns don't cause ReDoS."""
+        block = CodeExtractionBlock()
+
+        # Test with input that would previously cause ReDoS
+        malicious_input = "```python" + " " * 10000  # Large spaces
+
+        result = []
+        async for output_name, output_data in block.run(
+            CodeExtractionBlock.Input(text=malicious_input)
+        ):
+            result.append((output_name, output_data))
+
+        # Should complete without hanging
+        assert len(result) >= 1
+        assert any(name == "remaining_text" for name, _ in result)
+
+
+class TestAITextSummarizerBlockSecurity:
+    """Test memory exhaustion fixes in AITextSummarizerBlock."""
+
+    def test_split_text_limits(self):
+        """Test that _split_text has proper limits."""
+        # Test text size limit
+        large_text = "a" * 2_000_000  # 2MB text
+        result = AITextSummarizerBlock._split_text(large_text, 1000, 100)
+
+        # Should be truncated to 1MB
+        total_chars = sum(len(chunk) for chunk in result)
+        assert total_chars <= 1_000_000 + 1000  # Allow for chunk boundary
+
+        # Test chunk count limit
+        result = AITextSummarizerBlock._split_text("word " * 10000, 10, 9)
+        assert len(result) <= 100  # MAX_CHUNKS limit
+
+        # Test parameter validation
+        result = AITextSummarizerBlock._split_text(
+            "test", 10, 15
+        )  # overlap > max_tokens
+        assert len(result) >= 1  # Should still work
+
+
+class TestExtractTextInformationBlockSecurity:
+    """Test ReDoS and memory exhaustion fixes in ExtractTextInformationBlock."""
+
+    async def test_text_size_limits(self):
+        """Test text size limits."""
+        block = ExtractTextInformationBlock()
+
+        # Test with large input
+        large_text = "a" * 2_000_000  # 2MB
+
+        results = []
+        async for output_name, output_data in block.run(
+            ExtractTextInformationBlock.Input(
+                text=large_text, pattern=r"a+", find_all=True, group=0
+            )
+        ):
+            results.append((output_name, output_data))
+
+        # Should complete and have limits applied
+        matched_results = [r for name, r in results if name == "matched_results"]
+        if matched_results:
+            assert len(matched_results[0]) <= 1000  # MAX_MATCHES limit
+
+    async def test_dangerous_pattern_timeout(self):
+        """Test timeout protection for dangerous patterns."""
+        block = ExtractTextInformationBlock()
+
+        # Test with potentially dangerous lookahead pattern
+        test_input = "a" * 1000
+
+        # This should complete quickly due to timeout protection
+        start_time = asyncio.get_event_loop().time()
+        results = []
+        async for output_name, output_data in block.run(
+            ExtractTextInformationBlock.Input(
+                text=test_input, pattern=r"(?=.+)", find_all=True, group=0
+            )
+        ):
+            results.append((output_name, output_data))
+
+        end_time = asyncio.get_event_loop().time()
+        # Should complete within reasonable time (much less than 5s timeout)
+        assert (end_time - start_time) < 10
+
+    async def test_redos_catastrophic_backtracking(self):
+        """Test that ReDoS patterns with catastrophic backtracking are handled."""
+        block = ExtractTextInformationBlock()
+
+        # Pattern that causes catastrophic backtracking: (a+)+b
+        # With input "aaaaaaaaaaaaaaaaaaaaaaaaaaaa" (no 'b'), this causes exponential time
+        dangerous_pattern = r"(a+)+b"
+        test_input = "a" * 30  # 30 'a's without a 'b' at the end
+
+        # This should be handled by timeout protection or pattern detection
+        start_time = asyncio.get_event_loop().time()
+        results = []
+
+        async for output_name, output_data in block.run(
+            ExtractTextInformationBlock.Input(
+                text=test_input, pattern=dangerous_pattern, find_all=True, group=0
+            )
+        ):
+            results.append((output_name, output_data))
+
+        end_time = asyncio.get_event_loop().time()
+        elapsed = end_time - start_time
+
+        # Should complete within timeout (6 seconds to be safe)
+        # The current threading.Timer approach doesn't work, so this will likely fail
+        # demonstrating the need for a fix
+        assert elapsed < 6, f"Regex took {elapsed}s, timeout mechanism failed"
+
+        # Should return empty results on timeout or no match
+        matched_results = [r for name, r in results if name == "matched_results"]
+        assert matched_results[0] == []  # No matches expected
+
+
+class TestStepThroughItemsBlockSecurity:
+    """Test iteration limits in StepThroughItemsBlock."""
+
+    async def test_item_count_limits(self):
+        """Test maximum item count limits."""
+        block = StepThroughItemsBlock()
+
+        # Test with too many items
+        large_list = list(range(20000))  # Exceeds MAX_ITEMS (10000)
+
+        with pytest.raises(ValueError, match="Too many items"):
+            async for _ in block.run(StepThroughItemsBlock.Input(items=large_list)):
+                pass
+
+    async def test_string_size_limits(self):
+        """Test string input size limits."""
+        block = StepThroughItemsBlock()
+
+        # Test with large JSON string
+        large_string = '["item"]' * 200000  # Large JSON string
+
+        with pytest.raises(ValueError, match="Input too large"):
+            async for _ in block.run(
+                StepThroughItemsBlock.Input(items_str=large_string)
+            ):
+                pass
+
+    async def test_normal_iteration_works(self):
+        """Test that normal iteration still works."""
+        block = StepThroughItemsBlock()
+
+        results = []
+        async for output_name, output_data in block.run(
+            StepThroughItemsBlock.Input(items=[1, 2, 3])
+        ):
+            results.append((output_name, output_data))
+
+        # Should have 6 outputs (item, key for each of 3 items)
+        assert len(results) == 6
+        items = [data for name, data in results if name == "item"]
+        assert items == [1, 2, 3]
+
+
+class TestXMLParserBlockSecurity:
+    """Test XML size limits in XMLParserBlock."""
+
+    async def test_xml_size_limits(self):
+        """Test XML input size limits."""
+        block = XMLParserBlock()
+
+        # Test with large XML - need to exceed 10MB limit
+        # Each "<item>data</item>" is 17 chars, need ~620K items for >10MB
+        large_xml = "<root>" + "<item>data</item>" * 620000 + "</root>"
+
+        with pytest.raises(ValueError, match="XML too large"):
+            async for _ in block.run(XMLParserBlock.Input(input_xml=large_xml)):
+                pass
+
+
+class TestStoreMediaFileSecurity:
+    """Test file storage security limits."""
+
+    @patch("backend.util.file.scan_content_safe")
+    @patch("backend.util.file.get_cloud_storage_handler")
+    async def test_file_size_limits(self, mock_cloud_storage, mock_scan):
+        """Test file size limits."""
+        # Mock cloud storage handler - get_cloud_storage_handler is async
+        # but is_cloud_path and parse_cloud_path are sync methods
+        from unittest.mock import MagicMock
+
+        mock_handler = MagicMock()
+        mock_handler.is_cloud_path.return_value = False
+
+        # Make get_cloud_storage_handler an async function that returns the mock handler
+        async def async_get_handler():
+            return mock_handler
+
+        mock_cloud_storage.side_effect = async_get_handler
+        mock_scan.return_value = None
+
+        # Test with large base64 content
+        large_content = "a" * (200 * 1024 * 1024)  # 200MB
+        large_data_uri = f"data:text/plain;base64,{large_content}"
+
+        with pytest.raises(ValueError, match="File too large"):
+            await store_media_file(
+                graph_exec_id="test",
+                file=MediaFileType(large_data_uri),
+                user_id="test_user",
+            )
+
+    @patch("backend.util.file.Path")
+    @patch("backend.util.file.scan_content_safe")
+    @patch("backend.util.file.get_cloud_storage_handler")
+    async def test_directory_size_limits(self, mock_cloud_storage, mock_scan, MockPath):
+        """Test directory size limits."""
+        from unittest.mock import MagicMock
+
+        mock_handler = MagicMock()
+        mock_handler.is_cloud_path.return_value = False
+
+        async def async_get_handler():
+            return mock_handler
+
+        mock_cloud_storage.side_effect = async_get_handler
+        mock_scan.return_value = None
+
+        # Create mock path instance for the execution directory
+        mock_path_instance = MagicMock()
+        mock_path_instance.exists.return_value = True
+
+        # Mock glob to return files that total > 1GB
+        mock_file = MagicMock()
+        mock_file.is_file.return_value = True
+        mock_file.stat.return_value.st_size = 2 * 1024 * 1024 * 1024  # 2GB
+        mock_path_instance.glob.return_value = [mock_file]
+
+        # Make Path() return our mock
+        MockPath.return_value = mock_path_instance
+
+        # Should raise an error when directory size exceeds limit
+        with pytest.raises(ValueError, match="Disk usage limit exceeded"):
+            await store_media_file(
+                graph_exec_id="test",
+                file=MediaFileType(
+                    "data:text/plain;base64,dGVzdA=="
+                ),  # Small test file
+                user_id="test_user",
+            )

autogpt_platform/backend/backend/blocks/test/test_llm.py

@@ -30,7 +30,6 @@ class TestLLMStatsTracking:
                 credentials=llm.TEST_CREDENTIALS,
                 llm_model=llm.LlmModel.GPT4O,
                 prompt=[{"role": "user", "content": "Hello"}],
-                json_format=False,
                 max_tokens=100,
             )
 
@@ -42,6 +41,8 @@ class TestLLMStatsTracking:
     @pytest.mark.asyncio
     async def test_ai_structured_response_block_tracks_stats(self):
         """Test that AIStructuredResponseGeneratorBlock correctly tracks stats."""
+        from unittest.mock import patch
+
         import backend.blocks.llm as llm
 
         block = llm.AIStructuredResponseGeneratorBlock()
@@ -51,7 +52,7 @@ class TestLLMStatsTracking:
             return llm.LLMResponse(
                 raw_response="",
                 prompt=[],
-                response='{"key1": "value1", "key2": "value2"}',
+                response='<json_output id="test123456">{"key1": "value1", "key2": "value2"}</json_output>',
                 tool_calls=None,
                 prompt_tokens=15,
                 completion_tokens=25,
@@ -69,10 +70,12 @@ class TestLLMStatsTracking:
         )
 
         outputs = {}
-        async for output_name, output_data in block.run(
-            input_data, credentials=llm.TEST_CREDENTIALS
-        ):
-            outputs[output_name] = output_data
+        # Mock secrets.token_hex to return consistent ID
+        with patch("secrets.token_hex", return_value="test123456"):
+            async for output_name, output_data in block.run(
+                input_data, credentials=llm.TEST_CREDENTIALS
+            ):
+                outputs[output_name] = output_data
 
         # Check stats
         assert block.execution_stats.input_token_count == 15
@@ -143,7 +146,7 @@ class TestLLMStatsTracking:
                 return llm.LLMResponse(
                     raw_response="",
                     prompt=[],
-                    response='{"wrong": "format"}',
+                    response='<json_output id="test123456">{"wrong": "format"}</json_output>',
                     tool_calls=None,
                     prompt_tokens=10,
                     completion_tokens=15,
@@ -154,7 +157,7 @@ class TestLLMStatsTracking:
                 return llm.LLMResponse(
                     raw_response="",
                     prompt=[],
-                    response='{"key1": "value1", "key2": "value2"}',
+                    response='<json_output id="test123456">{"key1": "value1", "key2": "value2"}</json_output>',
                     tool_calls=None,
                     prompt_tokens=20,
                     completion_tokens=25,
@@ -173,10 +176,12 @@ class TestLLMStatsTracking:
         )
 
         outputs = {}
-        async for output_name, output_data in block.run(
-            input_data, credentials=llm.TEST_CREDENTIALS
-        ):
-            outputs[output_name] = output_data
+        # Mock secrets.token_hex to return consistent ID
+        with patch("secrets.token_hex", return_value="test123456"):
+            async for output_name, output_data in block.run(
+                input_data, credentials=llm.TEST_CREDENTIALS
+            ):
+                outputs[output_name] = output_data
 
         # Check stats - should accumulate both calls
         # For 2 attempts: attempt 1 (failed) + attempt 2 (success) = 2 total
@@ -269,7 +274,8 @@ class TestLLMStatsTracking:
                 mock_response.choices = [
                     MagicMock(
                         message=MagicMock(
-                            content='{"summary": "Test chunk summary"}', tool_calls=None
+                            content='<json_output id="test123456">{"summary": "Test chunk summary"}</json_output>',
+                            tool_calls=None,
                         )
                     )
                 ]
@@ -277,7 +283,7 @@ class TestLLMStatsTracking:
                 mock_response.choices = [
                     MagicMock(
                         message=MagicMock(
-                            content='{"final_summary": "Test final summary"}',
+                            content='<json_output id="test123456">{"final_summary": "Test final summary"}</json_output>',
                             tool_calls=None,
                         )
                     )
@@ -298,11 +304,13 @@ class TestLLMStatsTracking:
                 max_tokens=1000,  # Large enough to avoid chunking
             )
 
-            outputs = {}
-            async for output_name, output_data in block.run(
-                input_data, credentials=llm.TEST_CREDENTIALS
-            ):
-                outputs[output_name] = output_data
+            # Mock secrets.token_hex to return consistent ID
+            with patch("secrets.token_hex", return_value="test123456"):
+                outputs = {}
+                async for output_name, output_data in block.run(
+                    input_data, credentials=llm.TEST_CREDENTIALS
+                ):
+                    outputs[output_name] = output_data
 
             print(f"Actual calls made: {call_count}")
             print(f"Block stats: {block.execution_stats}")
@@ -457,7 +465,7 @@ class TestLLMStatsTracking:
             return llm.LLMResponse(
                 raw_response="",
                 prompt=[],
-                response='{"result": "test"}',
+                response='<json_output id="test123456">{"result": "test"}</json_output>',
                 tool_calls=None,
                 prompt_tokens=10,
                 completion_tokens=20,
@@ -476,10 +484,12 @@ class TestLLMStatsTracking:
 
         # Run the block
         outputs = {}
-        async for output_name, output_data in block.run(
-            input_data, credentials=llm.TEST_CREDENTIALS
-        ):
-            outputs[output_name] = output_data
+        # Mock secrets.token_hex to return consistent ID
+        with patch("secrets.token_hex", return_value="test123456"):
+            async for output_name, output_data in block.run(
+                input_data, credentials=llm.TEST_CREDENTIALS
+            ):
+                outputs[output_name] = output_data
 
         # Block finished - now grab and assert stats
         assert block.execution_stats is not None

autogpt_platform/backend/backend/blocks/test/test_smart_decision_maker.py

@@ -35,20 +35,19 @@ async def execute_graph(
     logger.info("Input data: %s", input_data)
 
     # --- Test adding new executions --- #
-    response = await agent_server.test_execute_graph(
+    graph_exec = await agent_server.test_execute_graph(
         user_id=test_user.id,
         graph_id=test_graph.id,
         graph_version=test_graph.version,
         node_input=input_data,
     )
-    graph_exec_id = response.graph_exec_id
-    logger.info("Created execution with ID: %s", graph_exec_id)
+    logger.info("Created execution with ID: %s", graph_exec.id)
 
     # Execution queue should be empty
     logger.info("Waiting for execution to complete...")
-    result = await wait_execution(test_user.id, graph_exec_id, 30)
+    result = await wait_execution(test_user.id, graph_exec.id, 30)
     logger.info("Execution completed with %d results", len(result))
-    return graph_exec_id
+    return graph_exec.id
 
 
 @pytest.mark.asyncio(loop_scope="session")
@@ -217,8 +216,17 @@ async def test_smart_decision_maker_tracks_llm_stats():
     }
 
     # Mock the _create_function_signature method to avoid database calls
-    with patch("backend.blocks.llm.llm_call", return_value=mock_response), patch.object(
-        SmartDecisionMakerBlock, "_create_function_signature", return_value=[]
+    from unittest.mock import AsyncMock
+
+    with patch(
+        "backend.blocks.llm.llm_call",
+        new_callable=AsyncMock,
+        return_value=mock_response,
+    ), patch.object(
+        SmartDecisionMakerBlock,
+        "_create_function_signature",
+        new_callable=AsyncMock,
+        return_value=[],
     ):
 
         # Create test input
@@ -250,3 +258,471 @@ async def test_smart_decision_maker_tracks_llm_stats():
         # Verify outputs
         assert "finished" in outputs  # Should have finished since no tool calls
         assert outputs["finished"] == "I need to think about this."
+
+
+@pytest.mark.asyncio
+async def test_smart_decision_maker_parameter_validation():
+    """Test that SmartDecisionMakerBlock correctly validates tool call parameters."""
+    from unittest.mock import MagicMock, patch
+
+    import backend.blocks.llm as llm_module
+    from backend.blocks.smart_decision_maker import SmartDecisionMakerBlock
+
+    block = SmartDecisionMakerBlock()
+
+    # Mock tool functions with specific parameter schema
+    mock_tool_functions = [
+        {
+            "type": "function",
+            "function": {
+                "name": "search_keywords",
+                "description": "Search for keywords with difficulty filtering",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "query": {"type": "string", "description": "Search query"},
+                        "max_keyword_difficulty": {
+                            "type": "integer",
+                            "description": "Maximum keyword difficulty (required)",
+                        },
+                        "optional_param": {
+                            "type": "string",
+                            "description": "Optional parameter with default",
+                            "default": "default_value",
+                        },
+                    },
+                    "required": ["query", "max_keyword_difficulty"],
+                },
+            },
+        }
+    ]
+
+    # Test case 1: Tool call with TYPO in parameter name (should retry and eventually fail)
+    mock_tool_call_with_typo = MagicMock()
+    mock_tool_call_with_typo.function.name = "search_keywords"
+    mock_tool_call_with_typo.function.arguments = '{"query": "test", "maximum_keyword_difficulty": 50}'  # TYPO: maximum instead of max
+
+    mock_response_with_typo = MagicMock()
+    mock_response_with_typo.response = None
+    mock_response_with_typo.tool_calls = [mock_tool_call_with_typo]
+    mock_response_with_typo.prompt_tokens = 50
+    mock_response_with_typo.completion_tokens = 25
+    mock_response_with_typo.reasoning = None
+    mock_response_with_typo.raw_response = {"role": "assistant", "content": None}
+
+    from unittest.mock import AsyncMock
+
+    with patch(
+        "backend.blocks.llm.llm_call",
+        new_callable=AsyncMock,
+        return_value=mock_response_with_typo,
+    ) as mock_llm_call, patch.object(
+        SmartDecisionMakerBlock,
+        "_create_function_signature",
+        new_callable=AsyncMock,
+        return_value=mock_tool_functions,
+    ):
+
+        input_data = SmartDecisionMakerBlock.Input(
+            prompt="Search for keywords",
+            model=llm_module.LlmModel.GPT4O,
+            credentials=llm_module.TEST_CREDENTIALS_INPUT,  # type: ignore
+            retry=2,  # Set retry to 2 for testing
+        )
+
+        # Should raise ValueError after retries due to typo'd parameter name
+        with pytest.raises(ValueError) as exc_info:
+            outputs = {}
+            async for output_name, output_data in block.run(
+                input_data,
+                credentials=llm_module.TEST_CREDENTIALS,
+                graph_id="test-graph-id",
+                node_id="test-node-id",
+                graph_exec_id="test-exec-id",
+                node_exec_id="test-node-exec-id",
+                user_id="test-user-id",
+            ):
+                outputs[output_name] = output_data
+
+        # Verify error message contains details about the typo
+        error_msg = str(exc_info.value)
+        assert "Tool call 'search_keywords' has parameter errors" in error_msg
+        assert "Unknown parameters: ['maximum_keyword_difficulty']" in error_msg
+
+        # Verify that LLM was called the expected number of times (retries)
+        assert mock_llm_call.call_count == 2  # Should retry based on input_data.retry
+
+    # Test case 2: Tool call missing REQUIRED parameter (should raise ValueError)
+    mock_tool_call_missing_required = MagicMock()
+    mock_tool_call_missing_required.function.name = "search_keywords"
+    mock_tool_call_missing_required.function.arguments = (
+        '{"query": "test"}'  # Missing required max_keyword_difficulty
+    )
+
+    mock_response_missing_required = MagicMock()
+    mock_response_missing_required.response = None
+    mock_response_missing_required.tool_calls = [mock_tool_call_missing_required]
+    mock_response_missing_required.prompt_tokens = 50
+    mock_response_missing_required.completion_tokens = 25
+    mock_response_missing_required.reasoning = None
+    mock_response_missing_required.raw_response = {"role": "assistant", "content": None}
+
+    from unittest.mock import AsyncMock
+
+    with patch(
+        "backend.blocks.llm.llm_call",
+        new_callable=AsyncMock,
+        return_value=mock_response_missing_required,
+    ), patch.object(
+        SmartDecisionMakerBlock,
+        "_create_function_signature",
+        new_callable=AsyncMock,
+        return_value=mock_tool_functions,
+    ):
+
+        input_data = SmartDecisionMakerBlock.Input(
+            prompt="Search for keywords",
+            model=llm_module.LlmModel.GPT4O,
+            credentials=llm_module.TEST_CREDENTIALS_INPUT,  # type: ignore
+        )
+
+        # Should raise ValueError due to missing required parameter
+        with pytest.raises(ValueError) as exc_info:
+            outputs = {}
+            async for output_name, output_data in block.run(
+                input_data,
+                credentials=llm_module.TEST_CREDENTIALS,
+                graph_id="test-graph-id",
+                node_id="test-node-id",
+                graph_exec_id="test-exec-id",
+                node_exec_id="test-node-exec-id",
+                user_id="test-user-id",
+            ):
+                outputs[output_name] = output_data
+
+        error_msg = str(exc_info.value)
+        assert "Tool call 'search_keywords' has parameter errors" in error_msg
+        assert "Missing required parameters: ['max_keyword_difficulty']" in error_msg
+
+    # Test case 3: Valid tool call with OPTIONAL parameter missing (should succeed)
+    mock_tool_call_valid = MagicMock()
+    mock_tool_call_valid.function.name = "search_keywords"
+    mock_tool_call_valid.function.arguments = '{"query": "test", "max_keyword_difficulty": 50}'  # optional_param missing, but that's OK
+
+    mock_response_valid = MagicMock()
+    mock_response_valid.response = None
+    mock_response_valid.tool_calls = [mock_tool_call_valid]
+    mock_response_valid.prompt_tokens = 50
+    mock_response_valid.completion_tokens = 25
+    mock_response_valid.reasoning = None
+    mock_response_valid.raw_response = {"role": "assistant", "content": None}
+
+    from unittest.mock import AsyncMock
+
+    with patch(
+        "backend.blocks.llm.llm_call",
+        new_callable=AsyncMock,
+        return_value=mock_response_valid,
+    ), patch.object(
+        SmartDecisionMakerBlock,
+        "_create_function_signature",
+        new_callable=AsyncMock,
+        return_value=mock_tool_functions,
+    ):
+
+        input_data = SmartDecisionMakerBlock.Input(
+            prompt="Search for keywords",
+            model=llm_module.LlmModel.GPT4O,
+            credentials=llm_module.TEST_CREDENTIALS_INPUT,  # type: ignore
+        )
+
+        # Should succeed - optional parameter missing is OK
+        outputs = {}
+        async for output_name, output_data in block.run(
+            input_data,
+            credentials=llm_module.TEST_CREDENTIALS,
+            graph_id="test-graph-id",
+            node_id="test-node-id",
+            graph_exec_id="test-exec-id",
+            node_exec_id="test-node-exec-id",
+            user_id="test-user-id",
+        ):
+            outputs[output_name] = output_data
+
+        # Verify tool outputs were generated correctly
+        assert "tools_^_search_keywords_~_query" in outputs
+        assert outputs["tools_^_search_keywords_~_query"] == "test"
+        assert "tools_^_search_keywords_~_max_keyword_difficulty" in outputs
+        assert outputs["tools_^_search_keywords_~_max_keyword_difficulty"] == 50
+        # Optional parameter should be None when not provided
+        assert "tools_^_search_keywords_~_optional_param" in outputs
+        assert outputs["tools_^_search_keywords_~_optional_param"] is None
+
+    # Test case 4: Valid tool call with ALL parameters (should succeed)
+    mock_tool_call_all_params = MagicMock()
+    mock_tool_call_all_params.function.name = "search_keywords"
+    mock_tool_call_all_params.function.arguments = '{"query": "test", "max_keyword_difficulty": 50, "optional_param": "custom_value"}'
+
+    mock_response_all_params = MagicMock()
+    mock_response_all_params.response = None
+    mock_response_all_params.tool_calls = [mock_tool_call_all_params]
+    mock_response_all_params.prompt_tokens = 50
+    mock_response_all_params.completion_tokens = 25
+    mock_response_all_params.reasoning = None
+    mock_response_all_params.raw_response = {"role": "assistant", "content": None}
+
+    from unittest.mock import AsyncMock
+
+    with patch(
+        "backend.blocks.llm.llm_call",
+        new_callable=AsyncMock,
+        return_value=mock_response_all_params,
+    ), patch.object(
+        SmartDecisionMakerBlock,
+        "_create_function_signature",
+        new_callable=AsyncMock,
+        return_value=mock_tool_functions,
+    ):
+
+        input_data = SmartDecisionMakerBlock.Input(
+            prompt="Search for keywords",
+            model=llm_module.LlmModel.GPT4O,
+            credentials=llm_module.TEST_CREDENTIALS_INPUT,  # type: ignore
+        )
+
+        # Should succeed with all parameters
+        outputs = {}
+        async for output_name, output_data in block.run(
+            input_data,
+            credentials=llm_module.TEST_CREDENTIALS,
+            graph_id="test-graph-id",
+            node_id="test-node-id",
+            graph_exec_id="test-exec-id",
+            node_exec_id="test-node-exec-id",
+            user_id="test-user-id",
+        ):
+            outputs[output_name] = output_data
+
+        # Verify all tool outputs were generated correctly
+        assert outputs["tools_^_search_keywords_~_query"] == "test"
+        assert outputs["tools_^_search_keywords_~_max_keyword_difficulty"] == 50
+        assert outputs["tools_^_search_keywords_~_optional_param"] == "custom_value"
+
+
+@pytest.mark.asyncio
+async def test_smart_decision_maker_raw_response_conversion():
+    """Test that SmartDecisionMaker correctly handles different raw_response types with retry mechanism."""
+    from unittest.mock import MagicMock, patch
+
+    import backend.blocks.llm as llm_module
+    from backend.blocks.smart_decision_maker import SmartDecisionMakerBlock
+
+    block = SmartDecisionMakerBlock()
+
+    # Mock tool functions
+    mock_tool_functions = [
+        {
+            "type": "function",
+            "function": {
+                "name": "test_tool",
+                "parameters": {
+                    "type": "object",
+                    "properties": {"param": {"type": "string"}},
+                    "required": ["param"],
+                },
+            },
+        }
+    ]
+
+    # Test case 1: Simulate ChatCompletionMessage raw_response that caused the original error
+    class MockChatCompletionMessage:
+        """Simulate OpenAI's ChatCompletionMessage object that lacks .get() method"""
+
+        def __init__(self, role, content, tool_calls=None):
+            self.role = role
+            self.content = content
+            self.tool_calls = tool_calls or []
+
+        # This is what caused the error - no .get() method
+        # def get(self, key, default=None):  # Intentionally missing
+
+    # First response: has invalid parameter name (triggers retry)
+    mock_tool_call_invalid = MagicMock()
+    mock_tool_call_invalid.function.name = "test_tool"
+    mock_tool_call_invalid.function.arguments = (
+        '{"wrong_param": "test_value"}'  # Invalid parameter name
+    )
+
+    mock_response_retry = MagicMock()
+    mock_response_retry.response = None
+    mock_response_retry.tool_calls = [mock_tool_call_invalid]
+    mock_response_retry.prompt_tokens = 50
+    mock_response_retry.completion_tokens = 25
+    mock_response_retry.reasoning = None
+    # This would cause the original error without our fix
+    mock_response_retry.raw_response = MockChatCompletionMessage(
+        role="assistant", content=None, tool_calls=[mock_tool_call_invalid]
+    )
+
+    # Second response: successful (correct parameter name)
+    mock_tool_call_valid = MagicMock()
+    mock_tool_call_valid.function.name = "test_tool"
+    mock_tool_call_valid.function.arguments = (
+        '{"param": "test_value"}'  # Correct parameter name
+    )
+
+    mock_response_success = MagicMock()
+    mock_response_success.response = None
+    mock_response_success.tool_calls = [mock_tool_call_valid]
+    mock_response_success.prompt_tokens = 50
+    mock_response_success.completion_tokens = 25
+    mock_response_success.reasoning = None
+    mock_response_success.raw_response = MockChatCompletionMessage(
+        role="assistant", content=None, tool_calls=[mock_tool_call_valid]
+    )
+
+    # Mock llm_call to return different responses on different calls
+    from unittest.mock import AsyncMock
+
+    with patch(
+        "backend.blocks.llm.llm_call", new_callable=AsyncMock
+    ) as mock_llm_call, patch.object(
+        SmartDecisionMakerBlock,
+        "_create_function_signature",
+        new_callable=AsyncMock,
+        return_value=mock_tool_functions,
+    ):
+        # First call returns response that will trigger retry due to validation error
+        # Second call returns successful response
+        mock_llm_call.side_effect = [mock_response_retry, mock_response_success]
+
+        input_data = SmartDecisionMakerBlock.Input(
+            prompt="Test prompt",
+            model=llm_module.LlmModel.GPT4O,
+            credentials=llm_module.TEST_CREDENTIALS_INPUT,  # type: ignore
+            retry=2,
+        )
+
+        # Should succeed after retry, demonstrating our helper function works
+        outputs = {}
+        async for output_name, output_data in block.run(
+            input_data,
+            credentials=llm_module.TEST_CREDENTIALS,
+            graph_id="test-graph-id",
+            node_id="test-node-id",
+            graph_exec_id="test-exec-id",
+            node_exec_id="test-node-exec-id",
+            user_id="test-user-id",
+        ):
+            outputs[output_name] = output_data
+
+        # Verify the tool output was generated successfully
+        assert "tools_^_test_tool_~_param" in outputs
+        assert outputs["tools_^_test_tool_~_param"] == "test_value"
+
+        # Verify conversation history was properly maintained
+        assert "conversations" in outputs
+        conversations = outputs["conversations"]
+        assert len(conversations) > 0
+
+        # The conversations should contain properly converted raw_response objects as dicts
+        # This would have failed with the original bug due to ChatCompletionMessage.get() error
+        for msg in conversations:
+            assert isinstance(msg, dict), f"Expected dict, got {type(msg)}"
+            if msg.get("role") == "assistant":
+                # Should have been converted from ChatCompletionMessage to dict
+                assert "role" in msg
+
+        # Verify LLM was called twice (initial + 1 retry)
+        assert mock_llm_call.call_count == 2
+
+    # Test case 2: Test with different raw_response types (Ollama string, dict)
+    # Test Ollama string response
+    mock_response_ollama = MagicMock()
+    mock_response_ollama.response = "I'll help you with that."
+    mock_response_ollama.tool_calls = None
+    mock_response_ollama.prompt_tokens = 30
+    mock_response_ollama.completion_tokens = 15
+    mock_response_ollama.reasoning = None
+    mock_response_ollama.raw_response = (
+        "I'll help you with that."  # Ollama returns string
+    )
+
+    from unittest.mock import AsyncMock
+
+    with patch(
+        "backend.blocks.llm.llm_call",
+        new_callable=AsyncMock,
+        return_value=mock_response_ollama,
+    ), patch.object(
+        SmartDecisionMakerBlock,
+        "_create_function_signature",
+        new_callable=AsyncMock,
+        return_value=[],  # No tools for this test
+    ):
+        input_data = SmartDecisionMakerBlock.Input(
+            prompt="Simple prompt",
+            model=llm_module.LlmModel.GPT4O,
+            credentials=llm_module.TEST_CREDENTIALS_INPUT,  # type: ignore
+        )
+
+        outputs = {}
+        async for output_name, output_data in block.run(
+            input_data,
+            credentials=llm_module.TEST_CREDENTIALS,
+            graph_id="test-graph-id",
+            node_id="test-node-id",
+            graph_exec_id="test-exec-id",
+            node_exec_id="test-node-exec-id",
+            user_id="test-user-id",
+        ):
+            outputs[output_name] = output_data
+
+        # Should finish since no tool calls
+        assert "finished" in outputs
+        assert outputs["finished"] == "I'll help you with that."
+
+    # Test case 3: Test with dict raw_response (some providers/tests)
+    mock_response_dict = MagicMock()
+    mock_response_dict.response = "Test response"
+    mock_response_dict.tool_calls = None
+    mock_response_dict.prompt_tokens = 25
+    mock_response_dict.completion_tokens = 10
+    mock_response_dict.reasoning = None
+    mock_response_dict.raw_response = {
+        "role": "assistant",
+        "content": "Test response",
+    }  # Dict format
+
+    from unittest.mock import AsyncMock
+
+    with patch(
+        "backend.blocks.llm.llm_call",
+        new_callable=AsyncMock,
+        return_value=mock_response_dict,
+    ), patch.object(
+        SmartDecisionMakerBlock,
+        "_create_function_signature",
+        new_callable=AsyncMock,
+        return_value=[],
+    ):
+        input_data = SmartDecisionMakerBlock.Input(
+            prompt="Another test",
+            model=llm_module.LlmModel.GPT4O,
+            credentials=llm_module.TEST_CREDENTIALS_INPUT,  # type: ignore
+        )
+
+        outputs = {}
+        async for output_name, output_data in block.run(
+            input_data,
+            credentials=llm_module.TEST_CREDENTIALS,
+            graph_id="test-graph-id",
+            node_id="test-node-id",
+            graph_exec_id="test-exec-id",
+            node_exec_id="test-node-exec-id",
+            user_id="test-user-id",
+        ):
+            outputs[output_name] = output_data
+
+        assert "finished" in outputs
+        assert outputs["finished"] == "Test response"

autogpt_platform/backend/backend/blocks/test/test_smart_decision_maker_dict.py

@@ -48,16 +48,24 @@ async def test_smart_decision_maker_handles_dynamic_dict_fields():
     assert "parameters" in signature["function"]
     assert "properties" in signature["function"]["parameters"]
 
-    # Check that dynamic fields are handled
+    # Check that dynamic fields are handled with original names
     properties = signature["function"]["parameters"]["properties"]
     assert len(properties) == 3  # Should have all three fields
 
-    # Each dynamic field should have proper schema
-    for prop_value in properties.values():
+    # Check that field names are cleaned (for Anthropic API compatibility)
+    assert "values___name" in properties
+    assert "values___age" in properties
+    assert "values___city" in properties
+
+    # Each dynamic field should have proper schema with descriptive text
+    for field_name, prop_value in properties.items():
         assert "type" in prop_value
         assert prop_value["type"] == "string"  # Dynamic fields get string type
         assert "description" in prop_value
-        assert "Dynamic value for" in prop_value["description"]
+        # Check that descriptions properly explain the dynamic field
+        if field_name == "values___name":
+            assert "Dictionary field 'name'" in prop_value["description"]
+            assert "values['name']" in prop_value["description"]
 
 
 @pytest.mark.asyncio
@@ -96,10 +104,18 @@ async def test_smart_decision_maker_handles_dynamic_list_fields():
     properties = signature["function"]["parameters"]["properties"]
     assert len(properties) == 2  # Should have both list items
 
-    # Each dynamic field should have proper schema
-    for prop_value in properties.values():
+    # Check that field names are cleaned (for Anthropic API compatibility)
+    assert "entries___0" in properties
+    assert "entries___1" in properties
+
+    # Each dynamic field should have proper schema with descriptive text
+    for field_name, prop_value in properties.items():
         assert prop_value["type"] == "string"
-        assert "Dynamic value for" in prop_value["description"]
+        assert "description" in prop_value
+        # Check that descriptions properly explain the list field
+        if field_name == "entries___0":
+            assert "List item 0" in prop_value["description"]
+            assert "entries[0]" in prop_value["description"]
 
 
 @pytest.mark.asyncio

autogpt_platform/backend/backend/blocks/test/test_smart_decision_maker_dynamic_fields.py

@@ -0,0 +1,553 @@
+"""Comprehensive tests for SmartDecisionMakerBlock dynamic field handling."""
+
+import json
+from unittest.mock import AsyncMock, Mock, patch
+
+import pytest
+
+from backend.blocks.data_manipulation import AddToListBlock, CreateDictionaryBlock
+from backend.blocks.smart_decision_maker import SmartDecisionMakerBlock
+from backend.blocks.text import MatchTextPatternBlock
+from backend.data.dynamic_fields import get_dynamic_field_description
+
+
+@pytest.mark.asyncio
+async def test_dynamic_field_description_generation():
+    """Test that dynamic field descriptions are generated correctly."""
+    # Test dictionary field description
+    desc = get_dynamic_field_description("values_#_name")
+    assert "Dictionary field 'name' for base field 'values'" in desc
+    assert "values['name']" in desc
+
+    # Test list field description
+    desc = get_dynamic_field_description("items_$_0")
+    assert "List item 0 for base field 'items'" in desc
+    assert "items[0]" in desc
+
+    # Test object field description
+    desc = get_dynamic_field_description("user_@_email")
+    assert "Object attribute 'email' for base field 'user'" in desc
+    assert "user.email" in desc
+
+    # Test regular field fallback
+    desc = get_dynamic_field_description("regular_field")
+    assert desc == "Value for regular_field"
+
+
+@pytest.mark.asyncio
+async def test_create_block_function_signature_with_dict_fields():
+    """Test that function signatures are created correctly for dictionary dynamic fields."""
+    block = SmartDecisionMakerBlock()
+
+    # Create a mock node for CreateDictionaryBlock
+    mock_node = Mock()
+    mock_node.block = CreateDictionaryBlock()
+    mock_node.block_id = CreateDictionaryBlock().id
+    mock_node.input_default = {}
+
+    # Create mock links with dynamic dictionary fields (source sanitized, sink original)
+    mock_links = [
+        Mock(
+            source_name="tools_^_create_dict_~_values___name",  # Sanitized source
+            sink_name="values_#_name",  # Original sink
+            sink_id="dict_node_id",
+            source_id="smart_decision_node_id",
+        ),
+        Mock(
+            source_name="tools_^_create_dict_~_values___age",  # Sanitized source
+            sink_name="values_#_age",  # Original sink
+            sink_id="dict_node_id",
+            source_id="smart_decision_node_id",
+        ),
+        Mock(
+            source_name="tools_^_create_dict_~_values___email",  # Sanitized source
+            sink_name="values_#_email",  # Original sink
+            sink_id="dict_node_id",
+            source_id="smart_decision_node_id",
+        ),
+    ]
+
+    # Generate function signature
+    signature = await block._create_block_function_signature(mock_node, mock_links)  # type: ignore
+
+    # Verify the signature structure
+    assert signature["type"] == "function"
+    assert "function" in signature
+    assert "parameters" in signature["function"]
+    assert "properties" in signature["function"]["parameters"]
+
+    # Check that dynamic fields are handled with original names
+    properties = signature["function"]["parameters"]["properties"]
+    assert len(properties) == 3
+
+    # Check cleaned field names (for Anthropic API compatibility)
+    assert "values___name" in properties
+    assert "values___age" in properties
+    assert "values___email" in properties
+
+    # Check descriptions mention they are dictionary fields
+    assert "Dictionary field" in properties["values___name"]["description"]
+    assert "values['name']" in properties["values___name"]["description"]
+
+    assert "Dictionary field" in properties["values___age"]["description"]
+    assert "values['age']" in properties["values___age"]["description"]
+
+    assert "Dictionary field" in properties["values___email"]["description"]
+    assert "values['email']" in properties["values___email"]["description"]
+
+
+@pytest.mark.asyncio
+async def test_create_block_function_signature_with_list_fields():
+    """Test that function signatures are created correctly for list dynamic fields."""
+    block = SmartDecisionMakerBlock()
+
+    # Create a mock node for AddToListBlock
+    mock_node = Mock()
+    mock_node.block = AddToListBlock()
+    mock_node.block_id = AddToListBlock().id
+    mock_node.input_default = {}
+
+    # Create mock links with dynamic list fields
+    mock_links = [
+        Mock(
+            source_name="tools_^_add_list_~_0",
+            sink_name="entries_$_0",  # Dynamic list field
+            sink_id="list_node_id",
+            source_id="smart_decision_node_id",
+        ),
+        Mock(
+            source_name="tools_^_add_list_~_1",
+            sink_name="entries_$_1",  # Dynamic list field
+            sink_id="list_node_id",
+            source_id="smart_decision_node_id",
+        ),
+        Mock(
+            source_name="tools_^_add_list_~_2",
+            sink_name="entries_$_2",  # Dynamic list field
+            sink_id="list_node_id",
+            source_id="smart_decision_node_id",
+        ),
+    ]
+
+    # Generate function signature
+    signature = await block._create_block_function_signature(mock_node, mock_links)  # type: ignore
+
+    # Verify the signature structure
+    assert signature["type"] == "function"
+    properties = signature["function"]["parameters"]["properties"]
+
+    # Check cleaned field names (for Anthropic API compatibility)
+    assert "entries___0" in properties
+    assert "entries___1" in properties
+    assert "entries___2" in properties
+
+    # Check descriptions mention they are list items
+    assert "List item 0" in properties["entries___0"]["description"]
+    assert "entries[0]" in properties["entries___0"]["description"]
+
+    assert "List item 1" in properties["entries___1"]["description"]
+    assert "entries[1]" in properties["entries___1"]["description"]
+
+
+@pytest.mark.asyncio
+async def test_create_block_function_signature_with_object_fields():
+    """Test that function signatures are created correctly for object dynamic fields."""
+    block = SmartDecisionMakerBlock()
+
+    # Create a mock node for MatchTextPatternBlock (simulating object fields)
+    mock_node = Mock()
+    mock_node.block = MatchTextPatternBlock()
+    mock_node.block_id = MatchTextPatternBlock().id
+    mock_node.input_default = {}
+
+    # Create mock links with dynamic object fields
+    mock_links = [
+        Mock(
+            source_name="tools_^_extract_~_user_name",
+            sink_name="data_@_user_name",  # Dynamic object field
+            sink_id="extract_node_id",
+            source_id="smart_decision_node_id",
+        ),
+        Mock(
+            source_name="tools_^_extract_~_user_email",
+            sink_name="data_@_user_email",  # Dynamic object field
+            sink_id="extract_node_id",
+            source_id="smart_decision_node_id",
+        ),
+    ]
+
+    # Generate function signature
+    signature = await block._create_block_function_signature(mock_node, mock_links)  # type: ignore
+
+    # Verify the signature structure
+    properties = signature["function"]["parameters"]["properties"]
+
+    # Check cleaned field names (for Anthropic API compatibility)
+    assert "data___user_name" in properties
+    assert "data___user_email" in properties
+
+    # Check descriptions mention they are object attributes
+    assert "Object attribute" in properties["data___user_name"]["description"]
+    assert "data.user_name" in properties["data___user_name"]["description"]
+
+
+@pytest.mark.asyncio
+async def test_create_function_signature():
+    """Test that the mapping between sanitized and original field names is built correctly."""
+    block = SmartDecisionMakerBlock()
+
+    # Mock the database client and connected nodes
+    with patch(
+        "backend.blocks.smart_decision_maker.get_database_manager_async_client"
+    ) as mock_db:
+        mock_client = AsyncMock()
+        mock_db.return_value = mock_client
+
+        # Create mock nodes and links
+        mock_dict_node = Mock()
+        mock_dict_node.block = CreateDictionaryBlock()
+        mock_dict_node.block_id = CreateDictionaryBlock().id
+        mock_dict_node.input_default = {}
+
+        mock_list_node = Mock()
+        mock_list_node.block = AddToListBlock()
+        mock_list_node.block_id = AddToListBlock().id
+        mock_list_node.input_default = {}
+
+        # Mock links with dynamic fields
+        dict_link1 = Mock(
+            source_name="tools_^_create_dictionary_~_name",
+            sink_name="values_#_name",
+            sink_id="dict_node_id",
+            source_id="test_node_id",
+        )
+        dict_link2 = Mock(
+            source_name="tools_^_create_dictionary_~_age",
+            sink_name="values_#_age",
+            sink_id="dict_node_id",
+            source_id="test_node_id",
+        )
+        list_link = Mock(
+            source_name="tools_^_add_to_list_~_0",
+            sink_name="entries_$_0",
+            sink_id="list_node_id",
+            source_id="test_node_id",
+        )
+
+        mock_client.get_connected_output_nodes.return_value = [
+            (dict_link1, mock_dict_node),
+            (dict_link2, mock_dict_node),
+            (list_link, mock_list_node),
+        ]
+
+        # Call the method that builds signatures
+        tool_functions = await block._create_function_signature("test_node_id")
+
+        # Verify we got 2 tool functions (one for dict, one for list)
+        assert len(tool_functions) == 2
+
+        # Verify the tool functions contain the dynamic field names
+        dict_tool = next(
+            (
+                tool
+                for tool in tool_functions
+                if tool["function"]["name"] == "createdictionaryblock"
+            ),
+            None,
+        )
+        assert dict_tool is not None
+        dict_properties = dict_tool["function"]["parameters"]["properties"]
+        assert "values___name" in dict_properties
+        assert "values___age" in dict_properties
+
+        list_tool = next(
+            (
+                tool
+                for tool in tool_functions
+                if tool["function"]["name"] == "addtolistblock"
+            ),
+            None,
+        )
+        assert list_tool is not None
+        list_properties = list_tool["function"]["parameters"]["properties"]
+        assert "entries___0" in list_properties
+
+
+@pytest.mark.asyncio
+async def test_output_yielding_with_dynamic_fields():
+    """Test that outputs are yielded correctly with dynamic field names mapped back."""
+    block = SmartDecisionMakerBlock()
+
+    # No more sanitized mapping needed since we removed sanitization
+
+    # Mock LLM response with tool calls
+    mock_response = Mock()
+    mock_response.tool_calls = [
+        Mock(
+            function=Mock(
+                arguments=json.dumps(
+                    {
+                        "values___name": "Alice",
+                        "values___age": 30,
+                        "values___email": "alice@example.com",
+                    }
+                ),
+            )
+        )
+    ]
+    # Ensure function name is a real string, not a Mock name
+    mock_response.tool_calls[0].function.name = "createdictionaryblock"
+    mock_response.reasoning = "Creating a dictionary with user information"
+    mock_response.raw_response = {"role": "assistant", "content": "test"}
+    mock_response.prompt_tokens = 100
+    mock_response.completion_tokens = 50
+
+    # Mock the LLM call
+    with patch(
+        "backend.blocks.smart_decision_maker.llm.llm_call", new_callable=AsyncMock
+    ) as mock_llm:
+        mock_llm.return_value = mock_response
+
+        # Mock the function signature creation
+        with patch.object(
+            block, "_create_function_signature", new_callable=AsyncMock
+        ) as mock_sig:
+            mock_sig.return_value = [
+                {
+                    "type": "function",
+                    "function": {
+                        "name": "createdictionaryblock",
+                        "parameters": {
+                            "type": "object",
+                            "properties": {
+                                "values___name": {"type": "string"},
+                                "values___age": {"type": "number"},
+                                "values___email": {"type": "string"},
+                            },
+                        },
+                    },
+                }
+            ]
+
+            # Create input data
+            from backend.blocks import llm
+
+            input_data = block.input_schema(
+                prompt="Create a user dictionary",
+                credentials=llm.TEST_CREDENTIALS_INPUT,
+                model=llm.LlmModel.GPT4O,
+            )
+
+            # Run the block
+            outputs = {}
+            async for output_name, output_value in block.run(
+                input_data,
+                credentials=llm.TEST_CREDENTIALS,
+                graph_id="test_graph",
+                node_id="test_node",
+                graph_exec_id="test_exec",
+                node_exec_id="test_node_exec",
+                user_id="test_user",
+            ):
+                outputs[output_name] = output_value
+
+            # Verify the outputs use sanitized field names (matching frontend normalizeToolName)
+            assert "tools_^_createdictionaryblock_~_values___name" in outputs
+            assert outputs["tools_^_createdictionaryblock_~_values___name"] == "Alice"
+
+            assert "tools_^_createdictionaryblock_~_values___age" in outputs
+            assert outputs["tools_^_createdictionaryblock_~_values___age"] == 30
+
+            assert "tools_^_createdictionaryblock_~_values___email" in outputs
+            assert (
+                outputs["tools_^_createdictionaryblock_~_values___email"]
+                == "alice@example.com"
+            )
+
+
+@pytest.mark.asyncio
+async def test_mixed_regular_and_dynamic_fields():
+    """Test handling of blocks with both regular and dynamic fields."""
+    block = SmartDecisionMakerBlock()
+
+    # Create a mock node
+    mock_node = Mock()
+    mock_node.block = Mock()
+    mock_node.block.name = "TestBlock"
+    mock_node.block.description = "A test block"
+    mock_node.block.input_schema = Mock()
+
+    # Mock the get_field_schema to return a proper schema for regular fields
+    def get_field_schema(field_name):
+        if field_name == "regular_field":
+            return {"type": "string", "description": "A regular field"}
+        elif field_name == "values":
+            return {"type": "object", "description": "A dictionary field"}
+        else:
+            raise KeyError(f"Field {field_name} not found")
+
+    mock_node.block.input_schema.get_field_schema = get_field_schema
+    mock_node.block.input_schema.jsonschema = Mock(
+        return_value={"properties": {}, "required": []}
+    )
+
+    # Create links with both regular and dynamic fields
+    mock_links = [
+        Mock(
+            source_name="tools_^_test_~_regular",
+            sink_name="regular_field",  # Regular field
+            sink_id="test_node_id",
+            source_id="smart_decision_node_id",
+        ),
+        Mock(
+            source_name="tools_^_test_~_dict_key",
+            sink_name="values_#_key1",  # Dynamic dict field
+            sink_id="test_node_id",
+            source_id="smart_decision_node_id",
+        ),
+        Mock(
+            source_name="tools_^_test_~_dict_key2",
+            sink_name="values_#_key2",  # Dynamic dict field
+            sink_id="test_node_id",
+            source_id="smart_decision_node_id",
+        ),
+    ]
+
+    # Generate function signature
+    signature = await block._create_block_function_signature(mock_node, mock_links)  # type: ignore
+
+    # Check properties
+    properties = signature["function"]["parameters"]["properties"]
+    assert len(properties) == 3
+
+    # Regular field should have its original schema
+    assert "regular_field" in properties
+    assert properties["regular_field"]["description"] == "A regular field"
+
+    # Dynamic fields should have generated descriptions
+    assert "values___key1" in properties
+    assert "Dictionary field" in properties["values___key1"]["description"]
+
+    assert "values___key2" in properties
+    assert "Dictionary field" in properties["values___key2"]["description"]
+
+
+@pytest.mark.asyncio
+async def test_validation_errors_dont_pollute_conversation():
+    """Test that validation errors are only used during retries and don't pollute the conversation."""
+    block = SmartDecisionMakerBlock()
+
+    # Track conversation history changes
+    conversation_snapshots = []
+
+    # Mock response with invalid tool call (missing required parameter)
+    invalid_response = Mock()
+    invalid_response.tool_calls = [
+        Mock(
+            function=Mock(
+                arguments=json.dumps({"wrong_param": "value"}),  # Wrong parameter name
+            )
+        )
+    ]
+    # Ensure function name is a real string, not a Mock name
+    invalid_response.tool_calls[0].function.name = "test_tool"
+    invalid_response.reasoning = None
+    invalid_response.raw_response = {"role": "assistant", "content": "invalid"}
+    invalid_response.prompt_tokens = 100
+    invalid_response.completion_tokens = 50
+
+    # Mock valid response after retry
+    valid_response = Mock()
+    valid_response.tool_calls = [
+        Mock(function=Mock(arguments=json.dumps({"correct_param": "value"})))
+    ]
+    # Ensure function name is a real string, not a Mock name
+    valid_response.tool_calls[0].function.name = "test_tool"
+    valid_response.reasoning = None
+    valid_response.raw_response = {"role": "assistant", "content": "valid"}
+    valid_response.prompt_tokens = 100
+    valid_response.completion_tokens = 50
+
+    call_count = 0
+
+    async def mock_llm_call(**kwargs):
+        nonlocal call_count
+        # Capture conversation state
+        conversation_snapshots.append(kwargs.get("prompt", []).copy())
+        call_count += 1
+        if call_count == 1:
+            return invalid_response
+        else:
+            return valid_response
+
+    # Mock the LLM call
+    with patch(
+        "backend.blocks.smart_decision_maker.llm.llm_call", new_callable=AsyncMock
+    ) as mock_llm:
+        mock_llm.side_effect = mock_llm_call
+
+        # Mock the function signature creation
+        with patch.object(
+            block, "_create_function_signature", new_callable=AsyncMock
+        ) as mock_sig:
+            mock_sig.return_value = [
+                {
+                    "type": "function",
+                    "function": {
+                        "name": "test_tool",
+                        "parameters": {
+                            "type": "object",
+                            "properties": {
+                                "correct_param": {
+                                    "type": "string",
+                                    "description": "The correct parameter",
+                                }
+                            },
+                            "required": ["correct_param"],
+                        },
+                    },
+                }
+            ]
+
+            # Create input data
+            from backend.blocks import llm
+
+            input_data = block.input_schema(
+                prompt="Test prompt",
+                credentials=llm.TEST_CREDENTIALS_INPUT,
+                model=llm.LlmModel.GPT4O,
+                retry=3,  # Allow retries
+            )
+
+            # Run the block
+            outputs = {}
+            async for output_name, output_value in block.run(
+                input_data,
+                credentials=llm.TEST_CREDENTIALS,
+                graph_id="test_graph",
+                node_id="test_node",
+                graph_exec_id="test_exec",
+                node_exec_id="test_node_exec",
+                user_id="test_user",
+            ):
+                outputs[output_name] = output_value
+
+            # Verify we had 2 LLM calls (initial + retry)
+            assert call_count == 2
+
+            # Check the final conversation output
+            final_conversation = outputs.get("conversations", [])
+
+            # The final conversation should NOT contain the validation error message
+            error_messages = [
+                msg
+                for msg in final_conversation
+                if msg.get("role") == "user"
+                and "parameter errors" in msg.get("content", "")
+            ]
+            assert (
+                len(error_messages) == 0
+            ), "Validation error leaked into final conversation"
+
+            # The final conversation should only have the successful response
+            assert final_conversation[-1]["content"] == "valid"

autogpt_platform/backend/backend/blocks/test/test_table_input.py

@@ -0,0 +1,131 @@
+import pytest
+
+from backend.blocks.io import AgentTableInputBlock
+from backend.util.test import execute_block_test
+
+
+@pytest.mark.asyncio
+async def test_table_input_block():
+    """Test the AgentTableInputBlock with basic input/output."""
+    block = AgentTableInputBlock()
+    await execute_block_test(block)
+
+
+@pytest.mark.asyncio
+async def test_table_input_with_data():
+    """Test AgentTableInputBlock with actual table data."""
+    block = AgentTableInputBlock()
+
+    input_data = block.Input(
+        name="test_table",
+        column_headers=["Name", "Age", "City"],
+        value=[
+            {"Name": "John", "Age": "30", "City": "New York"},
+            {"Name": "Jane", "Age": "25", "City": "London"},
+            {"Name": "Bob", "Age": "35", "City": "Paris"},
+        ],
+    )
+
+    output_data = []
+    async for output_name, output_value in block.run(input_data):
+        output_data.append((output_name, output_value))
+
+    assert len(output_data) == 1
+    assert output_data[0][0] == "result"
+
+    result = output_data[0][1]
+    assert len(result) == 3
+    assert result[0]["Name"] == "John"
+    assert result[1]["Age"] == "25"
+    assert result[2]["City"] == "Paris"
+
+
+@pytest.mark.asyncio
+async def test_table_input_empty_data():
+    """Test AgentTableInputBlock with empty data."""
+    block = AgentTableInputBlock()
+
+    input_data = block.Input(
+        name="empty_table", column_headers=["Col1", "Col2"], value=[]
+    )
+
+    output_data = []
+    async for output_name, output_value in block.run(input_data):
+        output_data.append((output_name, output_value))
+
+    assert len(output_data) == 1
+    assert output_data[0][0] == "result"
+    assert output_data[0][1] == []
+
+
+@pytest.mark.asyncio
+async def test_table_input_with_missing_columns():
+    """Test AgentTableInputBlock passes through data with missing columns as-is."""
+    block = AgentTableInputBlock()
+
+    input_data = block.Input(
+        name="partial_table",
+        column_headers=["Name", "Age", "City"],
+        value=[
+            {"Name": "John", "Age": "30"},  # Missing City
+            {"Name": "Jane", "City": "London"},  # Missing Age
+            {"Age": "35", "City": "Paris"},  # Missing Name
+        ],
+    )
+
+    output_data = []
+    async for output_name, output_value in block.run(input_data):
+        output_data.append((output_name, output_value))
+
+    result = output_data[0][1]
+    assert len(result) == 3
+
+    # Check data is passed through as-is
+    assert result[0] == {"Name": "John", "Age": "30"}
+    assert result[1] == {"Name": "Jane", "City": "London"}
+    assert result[2] == {"Age": "35", "City": "Paris"}
+
+
+@pytest.mark.asyncio
+async def test_table_input_none_value():
+    """Test AgentTableInputBlock with None value returns empty list."""
+    block = AgentTableInputBlock()
+
+    input_data = block.Input(
+        name="none_table", column_headers=["Name", "Age"], value=None
+    )
+
+    output_data = []
+    async for output_name, output_value in block.run(input_data):
+        output_data.append((output_name, output_value))
+
+    assert len(output_data) == 1
+    assert output_data[0][0] == "result"
+    assert output_data[0][1] == []
+
+
+@pytest.mark.asyncio
+async def test_table_input_with_default_headers():
+    """Test AgentTableInputBlock with default column headers."""
+    block = AgentTableInputBlock()
+
+    # Don't specify column_headers, should use defaults
+    input_data = block.Input(
+        name="default_headers_table",
+        value=[
+            {"Column 1": "A", "Column 2": "B", "Column 3": "C"},
+            {"Column 1": "D", "Column 2": "E", "Column 3": "F"},
+        ],
+    )
+
+    output_data = []
+    async for output_name, output_value in block.run(input_data):
+        output_data.append((output_name, output_value))
+
+    assert len(output_data) == 1
+    assert output_data[0][0] == "result"
+
+    result = output_data[0][1]
+    assert len(result) == 2
+    assert result[0]["Column 1"] == "A"
+    assert result[1]["Column 3"] == "F"

autogpt_platform/backend/backend/blocks/text.py

@@ -2,6 +2,8 @@ import re
 from pathlib import Path
 from typing import Any
 
+import regex  # Has built-in timeout support
+
 from backend.data.block import Block, BlockCategory, BlockOutput, BlockSchema
 from backend.data.model import SchemaField
 from backend.util import json, text
@@ -137,6 +139,11 @@ class ExtractTextInformationBlock(Block):
         )
 
     async def run(self, input_data: Input, **kwargs) -> BlockOutput:
+        # Security fix: Add limits to prevent ReDoS and memory exhaustion
+        MAX_TEXT_LENGTH = 1_000_000  # 1MB character limit
+        MAX_MATCHES = 1000  # Maximum number of matches to prevent memory exhaustion
+        MAX_MATCH_LENGTH = 10_000  # Maximum length per match
+
         flags = 0
         if not input_data.case_sensitive:
             flags = flags | re.IGNORECASE
@@ -148,20 +155,85 @@ class ExtractTextInformationBlock(Block):
         else:
             txt = json.dumps(input_data.text)
 
-        matches = [
-            match.group(input_data.group)
-            for match in re.finditer(input_data.pattern, txt, flags)
-            if input_data.group <= len(match.groups())
-        ]
-        if not input_data.find_all:
-            matches = matches[:1]
-        for match in matches:
-            yield "positive", match
-        if not matches:
-            yield "negative", input_data.text
+        # Limit text size to prevent DoS
+        if len(txt) > MAX_TEXT_LENGTH:
+            txt = txt[:MAX_TEXT_LENGTH]
 
-        yield "matched_results", matches
-        yield "matched_count", len(matches)
+        # Validate regex pattern to prevent dangerous patterns
+        dangerous_patterns = [
+            r".*\+.*\+",  # Nested quantifiers
+            r".*\*.*\*",  # Nested quantifiers
+            r"(?=.*\+)",  # Lookahead with quantifier
+            r"(?=.*\*)",  # Lookahead with quantifier
+            r"\(.+\)\+",  # Group with nested quantifier
+            r"\(.+\)\*",  # Group with nested quantifier
+            r"\([^)]+\+\)\+",  # Nested quantifiers like (a+)+
+            r"\([^)]+\*\)\*",  # Nested quantifiers like (a*)*
+        ]
+
+        # Check if pattern is potentially dangerous
+        is_dangerous = any(
+            re.search(dangerous, input_data.pattern) for dangerous in dangerous_patterns
+        )
+
+        # Use regex module with timeout for dangerous patterns
+        # For safe patterns, use standard re module for compatibility
+        try:
+            matches = []
+            match_count = 0
+
+            if is_dangerous:
+                # Use regex module with timeout (5 seconds) for dangerous patterns
+                # The regex module supports timeout parameter in finditer
+                try:
+                    for match in regex.finditer(
+                        input_data.pattern, txt, flags=flags, timeout=5.0
+                    ):
+                        if match_count >= MAX_MATCHES:
+                            break
+                        if input_data.group <= len(match.groups()):
+                            match_text = match.group(input_data.group)
+                            # Limit match length to prevent memory exhaustion
+                            if len(match_text) > MAX_MATCH_LENGTH:
+                                match_text = match_text[:MAX_MATCH_LENGTH]
+                            matches.append(match_text)
+                            match_count += 1
+                except regex.error as e:
+                    # Timeout occurred or regex error
+                    if "timeout" in str(e).lower():
+                        # Timeout - return empty results
+                        pass
+                    else:
+                        # Other regex error
+                        raise
+            else:
+                # Use standard re module for non-dangerous patterns
+                for match in re.finditer(input_data.pattern, txt, flags):
+                    if match_count >= MAX_MATCHES:
+                        break
+                    if input_data.group <= len(match.groups()):
+                        match_text = match.group(input_data.group)
+                        # Limit match length to prevent memory exhaustion
+                        if len(match_text) > MAX_MATCH_LENGTH:
+                            match_text = match_text[:MAX_MATCH_LENGTH]
+                        matches.append(match_text)
+                        match_count += 1
+
+            if not input_data.find_all:
+                matches = matches[:1]
+
+            for match in matches:
+                yield "positive", match
+            if not matches:
+                yield "negative", input_data.text
+
+            yield "matched_results", matches
+            yield "matched_count", len(matches)
+        except Exception:
+            # Return empty results on any regex error
+            yield "negative", input_data.text
+            yield "matched_results", []
+            yield "matched_count", 0
 
 
 class FillTextTemplateBlock(Block):
@@ -172,6 +244,11 @@ class FillTextTemplateBlock(Block):
         format: str = SchemaField(
             description="Template to format the text using `values`. Use Jinja2 syntax."
         )
+        escape_html: bool = SchemaField(
+            default=False,
+            advanced=True,
+            description="Whether to escape special characters in the inserted values to be HTML-safe. Enable for HTML output, disable for plain text.",
+        )
 
     class Output(BlockSchema):
         output: str = SchemaField(description="Formatted text")
@@ -205,6 +282,7 @@ class FillTextTemplateBlock(Block):
         )
 
     async def run(self, input_data: Input, **kwargs) -> BlockOutput:
+        formatter = text.TextFormatter(autoescape=input_data.escape_html)
         yield "output", formatter.format_string(input_data.format, input_data.values)
 
 

autogpt_platform/backend/backend/blocks/time_blocks.py

@@ -270,13 +270,17 @@ class GetCurrentDateBlock(Block):
             test_output=[
                 (
                     "date",
-                    lambda t: abs(datetime.now() - datetime.strptime(t, "%Y-%m-%d"))
-                    < timedelta(days=8),  # 7 days difference + 1 day error margin.
+                    lambda t: abs(
+                        datetime.now().date() - datetime.strptime(t, "%Y-%m-%d").date()
+                    )
+                    <= timedelta(days=8),  # 7 days difference + 1 day error margin.
                 ),
                 (
                     "date",
-                    lambda t: abs(datetime.now() - datetime.strptime(t, "%m/%d/%Y"))
-                    < timedelta(days=8),
+                    lambda t: abs(
+                        datetime.now().date() - datetime.strptime(t, "%m/%d/%Y").date()
+                    )
+                    <= timedelta(days=8),
                     # 7 days difference + 1 day error margin.
                 ),
                 (
@@ -382,7 +386,7 @@ class GetCurrentDateAndTimeBlock(Block):
                     lambda t: abs(
                         datetime.now().date() - datetime.strptime(t, "%Y/%m/%d").date()
                     )
-                    < timedelta(days=1),  # Date format only, no time component
+                    <= timedelta(days=1),  # Date format only, no time component
                 ),
                 (
                     "date_time",

autogpt_platform/backend/backend/blocks/wordpress/_api.py

@@ -4,6 +4,8 @@ from logging import getLogger
 from typing import Any, Dict, List, Union
 from urllib.parse import urlencode
 
+from pydantic import field_serializer
+
 from backend.sdk import BaseModel, Credentials, Requests
 
 logger = getLogger(__name__)
@@ -382,8 +384,9 @@ class CreatePostRequest(BaseModel):
     # Advanced
     metadata: List[Dict[str, Any]] | None = None
 
-    class Config:
-        json_encoders = {datetime: lambda v: v.isoformat()}
+    @field_serializer("date")
+    def serialize_date(self, value: datetime | None) -> str | None:
+        return value.isoformat() if value else None
 
 
 class PostAuthor(BaseModel):

autogpt_platform/backend/backend/blocks/xml_parser.py

@@ -26,6 +26,14 @@ class XMLParserBlock(Block):
         )
 
     async def run(self, input_data: Input, **kwargs) -> BlockOutput:
+        # Security fix: Add size limits to prevent XML bomb attacks
+        MAX_XML_SIZE = 10 * 1024 * 1024  # 10MB limit for XML input
+
+        if len(input_data.input_xml) > MAX_XML_SIZE:
+            raise ValueError(
+                f"XML too large: {len(input_data.input_xml)} bytes > {MAX_XML_SIZE} bytes"
+            )
+
         try:
             tokens = tokenize(input_data.input_xml)
             parser = Parser(tokens)

autogpt_platform/backend/backend/blocks/youtube.py

@@ -1,6 +1,7 @@
 from urllib.parse import parse_qs, urlparse
 
 from youtube_transcript_api._api import YouTubeTranscriptApi
+from youtube_transcript_api._errors import NoTranscriptFound
 from youtube_transcript_api._transcripts import FetchedTranscript
 from youtube_transcript_api.formatters import TextFormatter
 
@@ -64,7 +65,29 @@ class TranscribeYoutubeVideoBlock(Block):
 
     @staticmethod
     def get_transcript(video_id: str) -> FetchedTranscript:
-        return YouTubeTranscriptApi().fetch(video_id=video_id)
+        """
+        Get transcript for a video, preferring English but falling back to any available language.
+
+        :param video_id: The YouTube video ID
+        :return: The fetched transcript
+        :raises: Any exception except NoTranscriptFound for requested languages
+        """
+        api = YouTubeTranscriptApi()
+        try:
+            # Try to get English transcript first (default behavior)
+            return api.fetch(video_id=video_id)
+        except NoTranscriptFound:
+            # If English is not available, get the first available transcript
+            transcript_list = api.list(video_id)
+            # Try manually created transcripts first, then generated ones
+            available_transcripts = list(
+                transcript_list._manually_created_transcripts.values()
+            ) + list(transcript_list._generated_transcripts.values())
+            if available_transcripts:
+                # Fetch the first available transcript
+                return available_transcripts[0].fetch()
+            # If no transcripts at all, re-raise the original error
+            raise
 
     @staticmethod
     def format_transcript(transcript: FetchedTranscript) -> str:

autogpt_platform/backend/backend/conftest.py

@@ -6,8 +6,6 @@ from dotenv import load_dotenv
 
 from backend.util.logging import configure_logging
 
-os.environ["ENABLE_AUTH"] = "false"
-
 load_dotenv()
 
 #  NOTE: You can run tests like with the --log-cli-level=INFO to see the logs

autogpt_platform/backend/backend/data/__init__.py

@@ -1,5 +1,8 @@
+from backend.server.v2.library.model import LibraryAgentPreset
+
 from .graph import NodeModel
 from .integrations import Webhook  # noqa: F401
 
-# Resolve Webhook <- NodeModel forward reference
+# Resolve Webhook forward references
 NodeModel.model_rebuild()
+LibraryAgentPreset.model_rebuild()

autogpt_platform/backend/backend/data/api_key.py

@@ -1,57 +1,32 @@
 import logging
 import uuid
 from datetime import datetime, timezone
-from typing import List, Optional
+from typing import Optional
 
-from autogpt_libs.api_key.key_manager import APIKeyManager
+from autogpt_libs.api_key.keysmith import APIKeySmith
 from prisma.enums import APIKeyPermission, APIKeyStatus
-from prisma.errors import PrismaError
 from prisma.models import APIKey as PrismaAPIKey
-from prisma.types import (
-    APIKeyCreateInput,
-    APIKeyUpdateInput,
-    APIKeyWhereInput,
-    APIKeyWhereUniqueInput,
-)
-from pydantic import BaseModel
+from prisma.types import APIKeyWhereUniqueInput
+from pydantic import BaseModel, Field
 
-from backend.data.db import BaseDbModel
+from backend.data.includes import MAX_USER_API_KEYS_FETCH
+from backend.util.exceptions import NotAuthorizedError, NotFoundError
 
 logger = logging.getLogger(__name__)
+keysmith = APIKeySmith()
 
 
-# Some basic exceptions
-class APIKeyError(Exception):
-    """Base exception for API key operations"""
-
-    pass
-
-
-class APIKeyNotFoundError(APIKeyError):
-    """Raised when an API key is not found"""
-
-    pass
-
-
-class APIKeyPermissionError(APIKeyError):
-    """Raised when there are permission issues with API key operations"""
-
-    pass
-
-
-class APIKeyValidationError(APIKeyError):
-    """Raised when API key validation fails"""
-
-    pass
-
-
-class APIKey(BaseDbModel):
+class APIKeyInfo(BaseModel):
+    id: str
     name: str
-    prefix: str
-    key: str
-    status: APIKeyStatus = APIKeyStatus.ACTIVE
-    permissions: List[APIKeyPermission]
-    postfix: str
+    head: str = Field(
+        description=f"The first {APIKeySmith.HEAD_LENGTH} characters of the key"
+    )
+    tail: str = Field(
+        description=f"The last {APIKeySmith.TAIL_LENGTH} characters of the key"
+    )
+    status: APIKeyStatus
+    permissions: list[APIKeyPermission]
     created_at: datetime
     last_used_at: Optional[datetime] = None
     revoked_at: Optional[datetime] = None
@@ -60,266 +35,215 @@ class APIKey(BaseDbModel):
 
     @staticmethod
     def from_db(api_key: PrismaAPIKey):
-        try:
-            return APIKey(
-                id=api_key.id,
-                name=api_key.name,
-                prefix=api_key.prefix,
-                postfix=api_key.postfix,
-                key=api_key.key,
-                status=APIKeyStatus(api_key.status),
-                permissions=[APIKeyPermission(p) for p in api_key.permissions],
-                created_at=api_key.createdAt,
-                last_used_at=api_key.lastUsedAt,
-                revoked_at=api_key.revokedAt,
-                description=api_key.description,
-                user_id=api_key.userId,
-            )
-        except Exception as e:
-            logger.error(f"Error creating APIKey from db: {str(e)}")
-            raise APIKeyError(f"Failed to create API key object: {str(e)}")
+        return APIKeyInfo(
+            id=api_key.id,
+            name=api_key.name,
+            head=api_key.head,
+            tail=api_key.tail,
+            status=APIKeyStatus(api_key.status),
+            permissions=[APIKeyPermission(p) for p in api_key.permissions],
+            created_at=api_key.createdAt,
+            last_used_at=api_key.lastUsedAt,
+            revoked_at=api_key.revokedAt,
+            description=api_key.description,
+            user_id=api_key.userId,
+        )
 
 
-class APIKeyWithoutHash(BaseModel):
-    id: str
-    name: str
-    prefix: str
-    postfix: str
-    status: APIKeyStatus
-    permissions: List[APIKeyPermission]
-    created_at: datetime
-    last_used_at: Optional[datetime]
-    revoked_at: Optional[datetime]
-    description: Optional[str]
-    user_id: str
+class APIKeyInfoWithHash(APIKeyInfo):
+    hash: str
+    salt: str | None = None  # None for legacy keys
+
+    def match(self, plaintext_key: str) -> bool:
+        """Returns whether the given key matches this API key object."""
+        return keysmith.verify_key(plaintext_key, self.hash, self.salt)
 
     @staticmethod
     def from_db(api_key: PrismaAPIKey):
-        try:
-            return APIKeyWithoutHash(
-                id=api_key.id,
-                name=api_key.name,
-                prefix=api_key.prefix,
-                postfix=api_key.postfix,
-                status=APIKeyStatus(api_key.status),
-                permissions=[APIKeyPermission(p) for p in api_key.permissions],
-                created_at=api_key.createdAt,
-                last_used_at=api_key.lastUsedAt,
-                revoked_at=api_key.revokedAt,
-                description=api_key.description,
-                user_id=api_key.userId,
-            )
-        except Exception as e:
-            logger.error(f"Error creating APIKeyWithoutHash from db: {str(e)}")
-            raise APIKeyError(f"Failed to create API key object: {str(e)}")
+        return APIKeyInfoWithHash(
+            **APIKeyInfo.from_db(api_key).model_dump(),
+            hash=api_key.hash,
+            salt=api_key.salt,
+        )
+
+    def without_hash(self) -> APIKeyInfo:
+        return APIKeyInfo(**self.model_dump(exclude={"hash", "salt"}))
 
 
-async def generate_api_key(
+async def create_api_key(
     name: str,
     user_id: str,
-    permissions: List[APIKeyPermission],
+    permissions: list[APIKeyPermission],
     description: Optional[str] = None,
-) -> tuple[APIKeyWithoutHash, str]:
+) -> tuple[APIKeyInfo, str]:
     """
     Generate a new API key and store it in the database.
     Returns the API key object (without hash) and the plain text key.
     """
-    try:
-        api_manager = APIKeyManager()
-        key = api_manager.generate_api_key()
+    generated_key = keysmith.generate_key()
 
-        api_key = await PrismaAPIKey.prisma().create(
-            data=APIKeyCreateInput(
-                id=str(uuid.uuid4()),
-                name=name,
-                prefix=key.prefix,
-                postfix=key.postfix,
-                key=key.hash,
-                permissions=[p for p in permissions],
-                description=description,
-                userId=user_id,
-            )
-        )
+    saved_key_obj = await PrismaAPIKey.prisma().create(
+        data={
+            "id": str(uuid.uuid4()),
+            "name": name,
+            "head": generated_key.head,
+            "tail": generated_key.tail,
+            "hash": generated_key.hash,
+            "salt": generated_key.salt,
+            "permissions": [p for p in permissions],
+            "description": description,
+            "userId": user_id,
+        }
+    )
 
-        api_key_without_hash = APIKeyWithoutHash.from_db(api_key)
-        return api_key_without_hash, key.raw
-    except PrismaError as e:
-        logger.error(f"Database error while generating API key: {str(e)}")
-        raise APIKeyError(f"Failed to generate API key: {str(e)}")
-    except Exception as e:
-        logger.error(f"Unexpected error while generating API key: {str(e)}")
-        raise APIKeyError(f"Failed to generate API key: {str(e)}")
+    return APIKeyInfo.from_db(saved_key_obj), generated_key.key
 
 
-async def validate_api_key(plain_text_key: str) -> Optional[APIKey]:
+async def get_active_api_keys_by_head(head: str) -> list[APIKeyInfoWithHash]:
+    results = await PrismaAPIKey.prisma().find_many(
+        where={"head": head, "status": APIKeyStatus.ACTIVE}
+    )
+    return [APIKeyInfoWithHash.from_db(key) for key in results]
+
+
+async def validate_api_key(plaintext_key: str) -> Optional[APIKeyInfo]:
     """
-    Validate an API key and return the API key object if valid.
+    Validate an API key and return the API key object if valid and active.
     """
     try:
-        if not plain_text_key.startswith(APIKeyManager.PREFIX):
+        if not plaintext_key.startswith(APIKeySmith.PREFIX):
             logger.warning("Invalid API key format")
             return None
 
-        prefix = plain_text_key[: APIKeyManager.PREFIX_LENGTH]
-        api_manager = APIKeyManager()
+        head = plaintext_key[: APIKeySmith.HEAD_LENGTH]
+        potential_matches = await get_active_api_keys_by_head(head)
 
-        api_key = await PrismaAPIKey.prisma().find_first(
-            where=APIKeyWhereInput(prefix=prefix, status=(APIKeyStatus.ACTIVE))
+        matched_api_key = next(
+            (pm for pm in potential_matches if pm.match(plaintext_key)),
+            None,
         )
-
-        if not api_key:
-            logger.warning(f"No active API key found with prefix {prefix}")
+        if not matched_api_key:
+            # API key not found or invalid
             return None
 
-        is_valid = api_manager.verify_api_key(plain_text_key, api_key.key)
-        if not is_valid:
-            logger.warning("API key verification failed")
-            return None
-
-        return APIKey.from_db(api_key)
-    except Exception as e:
-        logger.error(f"Error validating API key: {str(e)}")
-        raise APIKeyValidationError(f"Failed to validate API key: {str(e)}")
-
-
-async def revoke_api_key(key_id: str, user_id: str) -> Optional[APIKeyWithoutHash]:
-    try:
-        api_key = await PrismaAPIKey.prisma().find_unique(where={"id": key_id})
-
-        if not api_key:
-            raise APIKeyNotFoundError(f"API key with id {key_id} not found")
-
-        if api_key.userId != user_id:
-            raise APIKeyPermissionError(
-                "You do not have permission to revoke this API key."
+        # Migrate legacy keys to secure format on successful validation
+        if matched_api_key.salt is None:
+            matched_api_key = await _migrate_key_to_secure_hash(
+                plaintext_key, matched_api_key
             )
 
-        where_clause: APIKeyWhereUniqueInput = {"id": key_id}
-        updated_api_key = await PrismaAPIKey.prisma().update(
-            where=where_clause,
-            data=APIKeyUpdateInput(
-                status=APIKeyStatus.REVOKED, revokedAt=datetime.now(timezone.utc)
-            ),
-        )
+        return matched_api_key.without_hash()
+    except Exception as e:
+        logger.error(f"Error while validating API key: {e}")
+        raise RuntimeError("Failed to validate API key") from e
 
-        if updated_api_key:
-            return APIKeyWithoutHash.from_db(updated_api_key)
+
+async def _migrate_key_to_secure_hash(
+    plaintext_key: str, key_obj: APIKeyInfoWithHash
+) -> APIKeyInfoWithHash:
+    """Replace the SHA256 hash of a legacy API key with a salted Scrypt hash."""
+    try:
+        new_hash, new_salt = keysmith.hash_key(plaintext_key)
+        await PrismaAPIKey.prisma().update(
+            where={"id": key_obj.id}, data={"hash": new_hash, "salt": new_salt}
+        )
+        logger.info(f"Migrated legacy API key #{key_obj.id} to secure format")
+        # Update the API key object with new values for return
+        key_obj.hash = new_hash
+        key_obj.salt = new_salt
+    except Exception as e:
+        logger.error(f"Failed to migrate legacy API key #{key_obj.id}: {e}")
+
+    return key_obj
+
+
+async def revoke_api_key(key_id: str, user_id: str) -> APIKeyInfo:
+    api_key = await PrismaAPIKey.prisma().find_unique(where={"id": key_id})
+
+    if not api_key:
+        raise NotFoundError(f"API key with id {key_id} not found")
+
+    if api_key.userId != user_id:
+        raise NotAuthorizedError("You do not have permission to revoke this API key.")
+
+    updated_api_key = await PrismaAPIKey.prisma().update(
+        where={"id": key_id},
+        data={
+            "status": APIKeyStatus.REVOKED,
+            "revokedAt": datetime.now(timezone.utc),
+        },
+    )
+    if not updated_api_key:
+        raise NotFoundError(f"API key #{key_id} vanished while trying to revoke.")
+
+    return APIKeyInfo.from_db(updated_api_key)
+
+
+async def list_user_api_keys(
+    user_id: str, limit: int = MAX_USER_API_KEYS_FETCH
+) -> list[APIKeyInfo]:
+    api_keys = await PrismaAPIKey.prisma().find_many(
+        where={"userId": user_id},
+        order={"createdAt": "desc"},
+        take=limit,
+    )
+
+    return [APIKeyInfo.from_db(key) for key in api_keys]
+
+
+async def suspend_api_key(key_id: str, user_id: str) -> APIKeyInfo:
+    selector: APIKeyWhereUniqueInput = {"id": key_id}
+    api_key = await PrismaAPIKey.prisma().find_unique(where=selector)
+
+    if not api_key:
+        raise NotFoundError(f"API key with id {key_id} not found")
+
+    if api_key.userId != user_id:
+        raise NotAuthorizedError("You do not have permission to suspend this API key.")
+
+    updated_api_key = await PrismaAPIKey.prisma().update(
+        where=selector, data={"status": APIKeyStatus.SUSPENDED}
+    )
+    if not updated_api_key:
+        raise NotFoundError(f"API key #{key_id} vanished while trying to suspend.")
+
+    return APIKeyInfo.from_db(updated_api_key)
+
+
+def has_permission(api_key: APIKeyInfo, required_permission: APIKeyPermission) -> bool:
+    return required_permission in api_key.permissions
+
+
+async def get_api_key_by_id(key_id: str, user_id: str) -> Optional[APIKeyInfo]:
+    api_key = await PrismaAPIKey.prisma().find_first(
+        where={"id": key_id, "userId": user_id}
+    )
+
+    if not api_key:
         return None
-    except (APIKeyNotFoundError, APIKeyPermissionError) as e:
-        raise e
-    except PrismaError as e:
-        logger.error(f"Database error while revoking API key: {str(e)}")
-        raise APIKeyError(f"Failed to revoke API key: {str(e)}")
-    except Exception as e:
-        logger.error(f"Unexpected error while revoking API key: {str(e)}")
-        raise APIKeyError(f"Failed to revoke API key: {str(e)}")
 
-
-async def list_user_api_keys(user_id: str) -> List[APIKeyWithoutHash]:
-    try:
-        where_clause: APIKeyWhereInput = {"userId": user_id}
-
-        api_keys = await PrismaAPIKey.prisma().find_many(
-            where=where_clause, order={"createdAt": "desc"}
-        )
-
-        return [APIKeyWithoutHash.from_db(key) for key in api_keys]
-    except PrismaError as e:
-        logger.error(f"Database error while listing API keys: {str(e)}")
-        raise APIKeyError(f"Failed to list API keys: {str(e)}")
-    except Exception as e:
-        logger.error(f"Unexpected error while listing API keys: {str(e)}")
-        raise APIKeyError(f"Failed to list API keys: {str(e)}")
-
-
-async def suspend_api_key(key_id: str, user_id: str) -> Optional[APIKeyWithoutHash]:
-    try:
-        api_key = await PrismaAPIKey.prisma().find_unique(where={"id": key_id})
-
-        if not api_key:
-            raise APIKeyNotFoundError(f"API key with id {key_id} not found")
-
-        if api_key.userId != user_id:
-            raise APIKeyPermissionError(
-                "You do not have permission to suspend this API key."
-            )
-
-        where_clause: APIKeyWhereUniqueInput = {"id": key_id}
-        updated_api_key = await PrismaAPIKey.prisma().update(
-            where=where_clause,
-            data=APIKeyUpdateInput(status=APIKeyStatus.SUSPENDED),
-        )
-
-        if updated_api_key:
-            return APIKeyWithoutHash.from_db(updated_api_key)
-        return None
-    except (APIKeyNotFoundError, APIKeyPermissionError) as e:
-        raise e
-    except PrismaError as e:
-        logger.error(f"Database error while suspending API key: {str(e)}")
-        raise APIKeyError(f"Failed to suspend API key: {str(e)}")
-    except Exception as e:
-        logger.error(f"Unexpected error while suspending API key: {str(e)}")
-        raise APIKeyError(f"Failed to suspend API key: {str(e)}")
-
-
-def has_permission(api_key: APIKey, required_permission: APIKeyPermission) -> bool:
-    try:
-        return required_permission in api_key.permissions
-    except Exception as e:
-        logger.error(f"Error checking API key permissions: {str(e)}")
-        return False
-
-
-async def get_api_key_by_id(key_id: str, user_id: str) -> Optional[APIKeyWithoutHash]:
-    try:
-        api_key = await PrismaAPIKey.prisma().find_first(
-            where=APIKeyWhereInput(id=key_id, userId=user_id)
-        )
-
-        if not api_key:
-            return None
-
-        return APIKeyWithoutHash.from_db(api_key)
-    except PrismaError as e:
-        logger.error(f"Database error while getting API key: {str(e)}")
-        raise APIKeyError(f"Failed to get API key: {str(e)}")
-    except Exception as e:
-        logger.error(f"Unexpected error while getting API key: {str(e)}")
-        raise APIKeyError(f"Failed to get API key: {str(e)}")
+    return APIKeyInfo.from_db(api_key)
 
 
 async def update_api_key_permissions(
-    key_id: str, user_id: str, permissions: List[APIKeyPermission]
-) -> Optional[APIKeyWithoutHash]:
+    key_id: str, user_id: str, permissions: list[APIKeyPermission]
+) -> APIKeyInfo:
     """
     Update the permissions of an API key.
     """
-    try:
-        api_key = await PrismaAPIKey.prisma().find_unique(where={"id": key_id})
+    api_key = await PrismaAPIKey.prisma().find_unique(where={"id": key_id})
 
-        if api_key is None:
-            raise APIKeyNotFoundError("No such API key found.")
+    if api_key is None:
+        raise NotFoundError("No such API key found.")
 
-        if api_key.userId != user_id:
-            raise APIKeyPermissionError(
-                "You do not have permission to update this API key."
-            )
+    if api_key.userId != user_id:
+        raise NotAuthorizedError("You do not have permission to update this API key.")
 
-        where_clause: APIKeyWhereUniqueInput = {"id": key_id}
-        updated_api_key = await PrismaAPIKey.prisma().update(
-            where=where_clause,
-            data=APIKeyUpdateInput(permissions=permissions),
-        )
+    updated_api_key = await PrismaAPIKey.prisma().update(
+        where={"id": key_id},
+        data={"permissions": permissions},
+    )
+    if not updated_api_key:
+        raise NotFoundError(f"API key #{key_id} vanished while trying to update.")
 
-        if updated_api_key:
-            return APIKeyWithoutHash.from_db(updated_api_key)
-        return None
-    except (APIKeyNotFoundError, APIKeyPermissionError) as e:
-        raise e
-    except PrismaError as e:
-        logger.error(f"Database error while updating API key permissions: {str(e)}")
-        raise APIKeyError(f"Failed to update API key permissions: {str(e)}")
-    except Exception as e:
-        logger.error(f"Unexpected error while updating API key permissions: {str(e)}")
-        raise APIKeyError(f"Failed to update API key permissions: {str(e)}")
+    return APIKeyInfo.from_db(updated_api_key)