Merge branch 'dev' into ci/claude-ci-optimizations-SECRT-1950

- Replace manual pnpm caching with setup-node built-in cache
- Add dev environment setup to claude-ci-failure-auto-fix.yml
  so Claude can run lint/tests when fixing CI failures
- Based on PR #12090 patterns

Files updated:
- claude.yml: Simplified pnpm caching
- claude-dependabot.yml: Simplified pnpm caching
- claude-ci-failure-auto-fix.yml: Added Python/Node setup with caching

Fixes SECRT-1950

.github/workflows/claude-ci-failure-auto-fix.yml

@@ -40,6 +40,48 @@ jobs:
           git checkout -b "$BRANCH_NAME"
           echo "branch_name=$BRANCH_NAME" >> $GITHUB_OUTPUT
 
+      # Backend Python/Poetry setup (so Claude can run linting/tests)
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Set up Python dependency cache
+        uses: actions/cache@v5
+        with:
+          path: ~/.cache/pypoetry
+          key: poetry-${{ runner.os }}-${{ hashFiles('autogpt_platform/backend/poetry.lock') }}
+
+      - name: Install Poetry
+        run: |
+          cd autogpt_platform/backend
+          HEAD_POETRY_VERSION=$(python3 ../../.github/workflows/scripts/get_package_version_from_lockfile.py poetry)
+          curl -sSL https://install.python-poetry.org | POETRY_VERSION=$HEAD_POETRY_VERSION python3 -
+          echo "$HOME/.local/bin" >> $GITHUB_PATH
+
+      - name: Install Python dependencies
+        working-directory: autogpt_platform/backend
+        run: poetry install
+
+      - name: Generate Prisma Client
+        working-directory: autogpt_platform/backend
+        run: poetry run prisma generate && poetry run gen-prisma-stub
+
+      # Frontend Node.js/pnpm setup (so Claude can run linting/tests)
+      - name: Enable corepack
+        run: corepack enable
+
+      - name: Set up Node.js
+        uses: actions/setup-node@v6
+        with:
+          node-version: "22"
+          cache: "pnpm"
+          cache-dependency-path: autogpt_platform/frontend/pnpm-lock.yaml
+
+      - name: Install JavaScript dependencies
+        working-directory: autogpt_platform/frontend
+        run: pnpm install --frozen-lockfile
+
       - name: Get CI failure details
         id: failure_details
         uses: actions/github-script@v8

.github/workflows/claude-dependabot.yml

@@ -77,27 +77,15 @@ jobs:
         run: poetry run prisma generate && poetry run gen-prisma-stub
 
       # Frontend Node.js/pnpm setup (mirrors platform-frontend-ci.yml)
+      - name: Enable corepack
+        run: corepack enable
+
       - name: Set up Node.js
         uses: actions/setup-node@v6
         with:
           node-version: "22"
-
-      - name: Enable corepack
-        run: corepack enable
-
-      - name: Set pnpm store directory
-        run: |
-          pnpm config set store-dir ~/.pnpm-store
-          echo "PNPM_HOME=$HOME/.pnpm-store" >> $GITHUB_ENV
-
-      - name: Cache frontend dependencies
-        uses: actions/cache@v5
-        with:
-          path: ~/.pnpm-store
-          key: ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml', 'autogpt_platform/frontend/package.json') }}
-          restore-keys: |
-            ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml') }}
-            ${{ runner.os }}-pnpm-
+          cache: "pnpm"
+          cache-dependency-path: autogpt_platform/frontend/pnpm-lock.yaml
 
       - name: Install JavaScript dependencies
         working-directory: autogpt_platform/frontend

.github/workflows/claude.yml

@@ -93,27 +93,15 @@ jobs:
         run: poetry run prisma generate && poetry run gen-prisma-stub
 
       # Frontend Node.js/pnpm setup (mirrors platform-frontend-ci.yml)
+      - name: Enable corepack
+        run: corepack enable
+
       - name: Set up Node.js
         uses: actions/setup-node@v6
         with:
           node-version: "22"
-
-      - name: Enable corepack
-        run: corepack enable
-
-      - name: Set pnpm store directory
-        run: |
-          pnpm config set store-dir ~/.pnpm-store
-          echo "PNPM_HOME=$HOME/.pnpm-store" >> $GITHUB_ENV
-
-      - name: Cache frontend dependencies
-        uses: actions/cache@v5
-        with:
-          path: ~/.pnpm-store
-          key: ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml', 'autogpt_platform/frontend/package.json') }}
-          restore-keys: |
-            ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml') }}
-            ${{ runner.os }}-pnpm-
+          cache: "pnpm"
+          cache-dependency-path: autogpt_platform/frontend/pnpm-lock.yaml
 
       - name: Install JavaScript dependencies
         working-directory: autogpt_platform/frontend

plans/SECRT-1950-claude-ci-optimizations.md

@@ -0,0 +1,165 @@
+# Implementation Plan: SECRT-1950 - Apply E2E CI Optimizations to Claude Code Workflows
+
+## Ticket
+[SECRT-1950](https://linear.app/autogpt/issue/SECRT-1950)
+
+## Summary
+Apply Pwuts's CI performance optimizations from PR #12090 to Claude Code workflows.
+
+## Reference PR
+https://github.com/Significant-Gravitas/AutoGPT/pull/12090
+
+---
+
+## Analysis
+
+### Current State (claude.yml)
+
+**pnpm caching (lines 104-118):**
+```yaml
+- name: Set up Node.js
+  uses: actions/setup-node@v6
+  with:
+    node-version: "22"
+
+- name: Enable corepack
+  run: corepack enable
+
+- name: Set pnpm store directory
+  run: |
+    pnpm config set store-dir ~/.pnpm-store
+    echo "PNPM_HOME=$HOME/.pnpm-store" >> $GITHUB_ENV
+
+- name: Cache frontend dependencies
+  uses: actions/cache@v5
+  with:
+    path: ~/.pnpm-store
+    key: ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml', 'autogpt_platform/frontend/package.json') }}
+    restore-keys: |
+      ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml') }}
+      ${{ runner.os }}-pnpm-
+```
+
+**Docker setup (lines 134-165):**
+- Uses `docker-buildx-action@v3` 
+- Has manual Docker image caching via `actions/cache`
+- Runs `docker compose up` without buildx bake optimization
+
+### Pwuts's Optimizations (PR #12090)
+
+1. **Simplified pnpm caching** - Use `setup-node` built-in cache:
+```yaml
+- name: Enable corepack
+  run: corepack enable
+
+- name: Set up Node
+  uses: actions/setup-node@v6
+  with:
+    node-version: "22.18.0"
+    cache: "pnpm"
+    cache-dependency-path: autogpt_platform/frontend/pnpm-lock.yaml
+```
+
+2. **Docker build caching via buildx bake**:
+```yaml
+- name: Set up Docker Buildx
+  uses: docker/setup-buildx-action@v3
+  with:
+    driver: docker-container
+    driver-opts: network=host
+
+- name: Expose GHA cache to docker buildx CLI
+  uses: crazy-max/ghaction-github-runtime@v3
+
+- name: Build Docker images (with cache)
+  run: |
+    pip install pyyaml
+    docker compose -f docker-compose.yml config > docker-compose.resolved.yml
+    python ../.github/workflows/scripts/docker-ci-fix-compose-build-cache.py \
+      --source docker-compose.resolved.yml \
+      --cache-from "type=gha" \
+      --cache-to "type=gha,mode=max" \
+      ...
+    docker buildx bake --allow=fs.read=.. -f docker-compose.resolved.yml --load
+```
+
+---
+
+## Proposed Changes
+
+### 1. Update pnpm caching in `claude.yml`
+
+**Before:**
+- Manual cache key generation
+- Separate `actions/cache` step
+- Manual pnpm store directory config
+
+**After:**
+- Use `setup-node` built-in `cache: "pnpm"` option
+- Remove manual cache step
+- Keep `corepack enable` before `setup-node`
+
+### 2. Update Docker build in `claude.yml`
+
+**Before:**
+- Manual Docker layer caching via `actions/cache` with `/tmp/.buildx-cache`
+- Simple `docker compose build`
+
+**After:**
+- Use `crazy-max/ghaction-github-runtime@v3` to expose GHA cache
+- Use `docker-ci-fix-compose-build-cache.py` script
+- Build with `docker buildx bake`
+
+### 3. Apply same changes to other Claude workflows
+
+- `claude-dependabot.yml` - Check if it has similar patterns
+- `claude-ci-failure-auto-fix.yml` - Check if it has similar patterns
+- `copilot-setup-steps.yml` - Reusable workflow, may be the source of truth
+
+---
+
+## Files to Modify
+
+1. `.github/workflows/claude.yml`
+2. `.github/workflows/claude-dependabot.yml` (if applicable)
+3. `.github/workflows/claude-ci-failure-auto-fix.yml` (if applicable)
+
+## Dependencies
+
+- PR #12090 must be merged first (provides the `docker-ci-fix-compose-build-cache.py` script)
+- Backend Dockerfile optimizations (already in PR #12090)
+
+---
+
+## Test Plan
+
+1. Create PR with changes
+2. Trigger Claude workflow manually or via `@claude` mention on a test issue
+3. Compare CI runtime before/after
+4. Verify Claude agent still works correctly (can checkout, build, run tests)
+
+---
+
+## Risk Assessment
+
+**Low risk:**
+- These are CI infrastructure changes, not code changes
+- If caching fails, builds fall back to uncached (slower but works)
+- Changes mirror proven patterns from PR #12090
+
+---
+
+## Questions for Reviewer
+
+1. Should we wait for PR #12090 to merge before creating this PR?
+2. Does `copilot-setup-steps.yml` need updating, or is it a separate concern?
+3. Any concerns about cache key collisions between frontend E2E and Claude workflows?
+
+---
+
+## Verified
+
+- ✅ **`claude-dependabot.yml`**: Has same pnpm caching pattern as `claude.yml` (manual `actions/cache`) — NEEDS UPDATE
+- ✅ **`claude-ci-failure-auto-fix.yml`**: Simple workflow with no pnpm or Docker caching — NO CHANGES NEEDED
+- ✅ **Script path**: `docker-ci-fix-compose-build-cache.py` will be at `.github/workflows/scripts/` after PR #12090 merges
+- ✅ **Test seed caching**: NOT APPLICABLE — Claude workflows spin up a dev environment but don't run E2E tests with pre-seeded data. The seed caching in PR #12090 is specific to the frontend E2E test suite which needs consistent test data. Claude just needs the services running.