mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-02-13 00:05:02 -05:00
ci: apply pnpm caching optimizations to Claude workflows
- Replace manual pnpm caching with setup-node built-in cache - Add dev environment setup to claude-ci-failure-auto-fix.yml so Claude can run lint/tests when fixing CI failures - Based on PR #12090 patterns Files updated: - claude.yml: Simplified pnpm caching - claude-dependabot.yml: Simplified pnpm caching - claude-ci-failure-auto-fix.yml: Added Python/Node setup with caching Fixes SECRT-1950
This commit is contained in:
Otto
42
.github/workflows/claude-ci-failure-auto-fix.yml
vendored
42
.github/workflows/claude-ci-failure-auto-fix.yml
vendored
@@ -40,6 +40,48 @@ jobs:
|
||||
git checkout -b "$BRANCH_NAME"
|
||||
echo "branch_name=$BRANCH_NAME" >> $GITHUB_OUTPUT
|
||||
|
||||
# Backend Python/Poetry setup (so Claude can run linting/tests)
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
- name: Set up Python dependency cache
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ~/.cache/pypoetry
|
||||
key: poetry-${{ runner.os }}-${{ hashFiles('autogpt_platform/backend/poetry.lock') }}
|
||||
|
||||
- name: Install Poetry
|
||||
run: |
|
||||
cd autogpt_platform/backend
|
||||
HEAD_POETRY_VERSION=$(python3 ../../.github/workflows/scripts/get_package_version_from_lockfile.py poetry)
|
||||
curl -sSL https://install.python-poetry.org | POETRY_VERSION=$HEAD_POETRY_VERSION python3 -
|
||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
||||
|
||||
- name: Install Python dependencies
|
||||
working-directory: autogpt_platform/backend
|
||||
run: poetry install
|
||||
|
||||
- name: Generate Prisma Client
|
||||
working-directory: autogpt_platform/backend
|
||||
run: poetry run prisma generate && poetry run gen-prisma-stub
|
||||
|
||||
# Frontend Node.js/pnpm setup (so Claude can run linting/tests)
|
||||
- name: Enable corepack
|
||||
run: corepack enable
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: "22"
|
||||
cache: "pnpm"
|
||||
cache-dependency-path: autogpt_platform/frontend/pnpm-lock.yaml
|
||||
|
||||
- name: Install JavaScript dependencies
|
||||
working-directory: autogpt_platform/frontend
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Get CI failure details
|
||||
id: failure_details
|
||||
uses: actions/github-script@v8
|
||||
|
||||
22
.github/workflows/claude-dependabot.yml
vendored
22
.github/workflows/claude-dependabot.yml
vendored
@@ -77,27 +77,15 @@ jobs:
|
||||
run: poetry run prisma generate && poetry run gen-prisma-stub
|
||||
|
||||
# Frontend Node.js/pnpm setup (mirrors platform-frontend-ci.yml)
|
||||
- name: Enable corepack
|
||||
run: corepack enable
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: "22"
|
||||
|
||||
- name: Enable corepack
|
||||
run: corepack enable
|
||||
|
||||
- name: Set pnpm store directory
|
||||
run: |
|
||||
pnpm config set store-dir ~/.pnpm-store
|
||||
echo "PNPM_HOME=$HOME/.pnpm-store" >> $GITHUB_ENV
|
||||
|
||||
- name: Cache frontend dependencies
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ~/.pnpm-store
|
||||
key: ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml', 'autogpt_platform/frontend/package.json') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml') }}
|
||||
${{ runner.os }}-pnpm-
|
||||
cache: "pnpm"
|
||||
cache-dependency-path: autogpt_platform/frontend/pnpm-lock.yaml
|
||||
|
||||
- name: Install JavaScript dependencies
|
||||
working-directory: autogpt_platform/frontend
|
||||
|
||||
22
.github/workflows/claude.yml
vendored
22
.github/workflows/claude.yml
vendored
@@ -93,27 +93,15 @@ jobs:
|
||||
run: poetry run prisma generate && poetry run gen-prisma-stub
|
||||
|
||||
# Frontend Node.js/pnpm setup (mirrors platform-frontend-ci.yml)
|
||||
- name: Enable corepack
|
||||
run: corepack enable
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: "22"
|
||||
|
||||
- name: Enable corepack
|
||||
run: corepack enable
|
||||
|
||||
- name: Set pnpm store directory
|
||||
run: |
|
||||
pnpm config set store-dir ~/.pnpm-store
|
||||
echo "PNPM_HOME=$HOME/.pnpm-store" >> $GITHUB_ENV
|
||||
|
||||
- name: Cache frontend dependencies
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ~/.pnpm-store
|
||||
key: ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml', 'autogpt_platform/frontend/package.json') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml') }}
|
||||
${{ runner.os }}-pnpm-
|
||||
cache: "pnpm"
|
||||
cache-dependency-path: autogpt_platform/frontend/pnpm-lock.yaml
|
||||
|
||||
- name: Install JavaScript dependencies
|
||||
working-directory: autogpt_platform/frontend
|
||||
|
||||
165
plans/SECRT-1950-claude-ci-optimizations.md
Normal file
165
plans/SECRT-1950-claude-ci-optimizations.md
Normal file
@@ -0,0 +1,165 @@
|
||||
# Implementation Plan: SECRT-1950 - Apply E2E CI Optimizations to Claude Code Workflows
|
||||
|
||||
## Ticket
|
||||
[SECRT-1950](https://linear.app/autogpt/issue/SECRT-1950)
|
||||
|
||||
## Summary
|
||||
Apply Pwuts's CI performance optimizations from PR #12090 to Claude Code workflows.
|
||||
|
||||
## Reference PR
|
||||
https://github.com/Significant-Gravitas/AutoGPT/pull/12090
|
||||
|
||||
---
|
||||
|
||||
## Analysis
|
||||
|
||||
### Current State (claude.yml)
|
||||
|
||||
**pnpm caching (lines 104-118):**
|
||||
```yaml
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: "22"
|
||||
|
||||
- name: Enable corepack
|
||||
run: corepack enable
|
||||
|
||||
- name: Set pnpm store directory
|
||||
run: |
|
||||
pnpm config set store-dir ~/.pnpm-store
|
||||
echo "PNPM_HOME=$HOME/.pnpm-store" >> $GITHUB_ENV
|
||||
|
||||
- name: Cache frontend dependencies
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ~/.pnpm-store
|
||||
key: ${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml', 'autogpt_platform/frontend/package.json') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pnpm-${{ hashFiles('autogpt_platform/frontend/pnpm-lock.yaml') }}
|
||||
${{ runner.os }}-pnpm-
|
||||
```
|
||||
|
||||
**Docker setup (lines 134-165):**
|
||||
- Uses `docker-buildx-action@v3`
|
||||
- Has manual Docker image caching via `actions/cache`
|
||||
- Runs `docker compose up` without buildx bake optimization
|
||||
|
||||
### Pwuts's Optimizations (PR #12090)
|
||||
|
||||
1. **Simplified pnpm caching** - Use `setup-node` built-in cache:
|
||||
```yaml
|
||||
- name: Enable corepack
|
||||
run: corepack enable
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: "22.18.0"
|
||||
cache: "pnpm"
|
||||
cache-dependency-path: autogpt_platform/frontend/pnpm-lock.yaml
|
||||
```
|
||||
|
||||
2. **Docker build caching via buildx bake**:
|
||||
```yaml
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
with:
|
||||
driver: docker-container
|
||||
driver-opts: network=host
|
||||
|
||||
- name: Expose GHA cache to docker buildx CLI
|
||||
uses: crazy-max/ghaction-github-runtime@v3
|
||||
|
||||
- name: Build Docker images (with cache)
|
||||
run: |
|
||||
pip install pyyaml
|
||||
docker compose -f docker-compose.yml config > docker-compose.resolved.yml
|
||||
python ../.github/workflows/scripts/docker-ci-fix-compose-build-cache.py \
|
||||
--source docker-compose.resolved.yml \
|
||||
--cache-from "type=gha" \
|
||||
--cache-to "type=gha,mode=max" \
|
||||
...
|
||||
docker buildx bake --allow=fs.read=.. -f docker-compose.resolved.yml --load
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Proposed Changes
|
||||
|
||||
### 1. Update pnpm caching in `claude.yml`
|
||||
|
||||
**Before:**
|
||||
- Manual cache key generation
|
||||
- Separate `actions/cache` step
|
||||
- Manual pnpm store directory config
|
||||
|
||||
**After:**
|
||||
- Use `setup-node` built-in `cache: "pnpm"` option
|
||||
- Remove manual cache step
|
||||
- Keep `corepack enable` before `setup-node`
|
||||
|
||||
### 2. Update Docker build in `claude.yml`
|
||||
|
||||
**Before:**
|
||||
- Manual Docker layer caching via `actions/cache` with `/tmp/.buildx-cache`
|
||||
- Simple `docker compose build`
|
||||
|
||||
**After:**
|
||||
- Use `crazy-max/ghaction-github-runtime@v3` to expose GHA cache
|
||||
- Use `docker-ci-fix-compose-build-cache.py` script
|
||||
- Build with `docker buildx bake`
|
||||
|
||||
### 3. Apply same changes to other Claude workflows
|
||||
|
||||
- `claude-dependabot.yml` - Check if it has similar patterns
|
||||
- `claude-ci-failure-auto-fix.yml` - Check if it has similar patterns
|
||||
- `copilot-setup-steps.yml` - Reusable workflow, may be the source of truth
|
||||
|
||||
---
|
||||
|
||||
## Files to Modify
|
||||
|
||||
1. `.github/workflows/claude.yml`
|
||||
2. `.github/workflows/claude-dependabot.yml` (if applicable)
|
||||
3. `.github/workflows/claude-ci-failure-auto-fix.yml` (if applicable)
|
||||
|
||||
## Dependencies
|
||||
|
||||
- PR #12090 must be merged first (provides the `docker-ci-fix-compose-build-cache.py` script)
|
||||
- Backend Dockerfile optimizations (already in PR #12090)
|
||||
|
||||
---
|
||||
|
||||
## Test Plan
|
||||
|
||||
1. Create PR with changes
|
||||
2. Trigger Claude workflow manually or via `@claude` mention on a test issue
|
||||
3. Compare CI runtime before/after
|
||||
4. Verify Claude agent still works correctly (can checkout, build, run tests)
|
||||
|
||||
---
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
**Low risk:**
|
||||
- These are CI infrastructure changes, not code changes
|
||||
- If caching fails, builds fall back to uncached (slower but works)
|
||||
- Changes mirror proven patterns from PR #12090
|
||||
|
||||
---
|
||||
|
||||
## Questions for Reviewer
|
||||
|
||||
1. Should we wait for PR #12090 to merge before creating this PR?
|
||||
2. Does `copilot-setup-steps.yml` need updating, or is it a separate concern?
|
||||
3. Any concerns about cache key collisions between frontend E2E and Claude workflows?
|
||||
|
||||
---
|
||||
|
||||
## Verified
|
||||
|
||||
- ✅ **`claude-dependabot.yml`**: Has same pnpm caching pattern as `claude.yml` (manual `actions/cache`) — NEEDS UPDATE
|
||||
- ✅ **`claude-ci-failure-auto-fix.yml`**: Simple workflow with no pnpm or Docker caching — NO CHANGES NEEDED
|
||||
- ✅ **Script path**: `docker-ci-fix-compose-build-cache.py` will be at `.github/workflows/scripts/` after PR #12090 merges
|
||||
- ✅ **Test seed caching**: NOT APPLICABLE — Claude workflows spin up a dev environment but don't run E2E tests with pre-seeded data. The seed caching in PR #12090 is specific to the frontend E2E test suite which needs consistent test data. Claude just needs the services running.
|
||||
Reference in New Issue
Block a user