mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-02-13 16:25:05 -05:00
698dc45146bc06d8adf24ec66affeef370f7c49e
23 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
dependabot[bot]
|
2cd0d4fe0f |
chore(deps): bump actions/checkout from 4 to 6 (#12034)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 6. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/checkout/releases">actions/checkout's releases</a>.</em></p> <blockquote> <h2>v6.0.0</h2> <h2>What's Changed</h2> <ul> <li>Update README to include Node.js 24 support details and requirements by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2248">actions/checkout#2248</a></li> <li>Persist creds to a separate file by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2286">actions/checkout#2286</a></li> <li>v6-beta by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2298">actions/checkout#2298</a></li> <li>update readme/changelog for v6 by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2311">actions/checkout#2311</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v5.0.0...v6.0.0">https://github.com/actions/checkout/compare/v5.0.0...v6.0.0</a></p> <h2>v6-beta</h2> <h2>What's Changed</h2> <p>Updated persist-credentials to store the credentials under <code>$RUNNER_TEMP</code> instead of directly in the local git config.</p> <p>This requires a minimum Actions Runner version of <a href="https://github.com/actions/runner/releases/tag/v2.329.0">v2.329.0</a> to access the persisted credentials for <a href="https://docs.github.com/en/actions/tutorials/use-containerized-services/create-a-docker-container-action">Docker container action</a> scenarios.</p> <h2>v5.0.1</h2> <h2>What's Changed</h2> <ul> <li>Port v6 cleanup to v5 by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2301">actions/checkout#2301</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v5...v5.0.1">https://github.com/actions/checkout/compare/v5...v5.0.1</a></p> <h2>v5.0.0</h2> <h2>What's Changed</h2> <ul> <li>Update actions checkout to use node 24 by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2226">actions/checkout#2226</a></li> <li>Prepare v5.0.0 release by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2238">actions/checkout#2238</a></li> </ul> <h2>⚠️ Minimum Compatible Runner Version</h2> <p><strong>v2.327.1</strong><br /> <a href="https://github.com/actions/runner/releases/tag/v2.327.1">Release Notes</a></p> <p>Make sure your runner is updated to this version or newer to use this release.</p> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v4...v5.0.0">https://github.com/actions/checkout/compare/v4...v5.0.0</a></p> <h2>v4.3.1</h2> <h2>What's Changed</h2> <ul> <li>Port v6 cleanup to v4 by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2305">actions/checkout#2305</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v4...v4.3.1">https://github.com/actions/checkout/compare/v4...v4.3.1</a></p> <h2>v4.3.0</h2> <h2>What's Changed</h2> <ul> <li>docs: update README.md by <a href="https://github.com/motss"><code>@motss</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1971">actions/checkout#1971</a></li> <li>Add internal repos for checking out multiple repositories by <a href="https://github.com/mouismail"><code>@mouismail</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1977">actions/checkout#1977</a></li> <li>Documentation update - add recommended permissions to Readme by <a href="https://github.com/benwells"><code>@benwells</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2043">actions/checkout#2043</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/actions/checkout/blob/main/CHANGELOG.md">actions/checkout's changelog</a>.</em></p> <blockquote> <h1>Changelog</h1> <h2>v6.0.2</h2> <ul> <li>Fix tag handling: preserve annotations and explicit fetch-tags by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2356">actions/checkout#2356</a></li> </ul> <h2>v6.0.1</h2> <ul> <li>Add worktree support for persist-credentials includeIf by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2327">actions/checkout#2327</a></li> </ul> <h2>v6.0.0</h2> <ul> <li>Persist creds to a separate file by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2286">actions/checkout#2286</a></li> <li>Update README to include Node.js 24 support details and requirements by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2248">actions/checkout#2248</a></li> </ul> <h2>v5.0.1</h2> <ul> <li>Port v6 cleanup to v5 by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2301">actions/checkout#2301</a></li> </ul> <h2>v5.0.0</h2> <ul> <li>Update actions checkout to use node 24 by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2226">actions/checkout#2226</a></li> </ul> <h2>v4.3.1</h2> <ul> <li>Port v6 cleanup to v4 by <a href="https://github.com/ericsciple"><code>@ericsciple</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2305">actions/checkout#2305</a></li> </ul> <h2>v4.3.0</h2> <ul> <li>docs: update README.md by <a href="https://github.com/motss"><code>@motss</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1971">actions/checkout#1971</a></li> <li>Add internal repos for checking out multiple repositories by <a href="https://github.com/mouismail"><code>@mouismail</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1977">actions/checkout#1977</a></li> <li>Documentation update - add recommended permissions to Readme by <a href="https://github.com/benwells"><code>@benwells</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2043">actions/checkout#2043</a></li> <li>Adjust positioning of user email note and permissions heading by <a href="https://github.com/joshmgross"><code>@joshmgross</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2044">actions/checkout#2044</a></li> <li>Update README.md by <a href="https://github.com/nebuk89"><code>@nebuk89</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2194">actions/checkout#2194</a></li> <li>Update CODEOWNERS for actions by <a href="https://github.com/TingluoHuang"><code>@TingluoHuang</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2224">actions/checkout#2224</a></li> <li>Update package dependencies by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/2236">actions/checkout#2236</a></li> </ul> <h2>v4.2.2</h2> <ul> <li><code>url-helper.ts</code> now leverages well-known environment variables by <a href="https://github.com/jww3"><code>@jww3</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1941">actions/checkout#1941</a></li> <li>Expand unit test coverage for <code>isGhes</code> by <a href="https://github.com/jww3"><code>@jww3</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1946">actions/checkout#1946</a></li> </ul> <h2>v4.2.1</h2> <ul> <li>Check out other refs/* by commit if provided, fall back to ref by <a href="https://github.com/orhantoy"><code>@orhantoy</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1924">actions/checkout#1924</a></li> </ul> <h2>v4.2.0</h2> <ul> <li>Add Ref and Commit outputs by <a href="https://github.com/lucacome"><code>@lucacome</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1180">actions/checkout#1180</a></li> <li>Dependency updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a>- <a href="https://redirect.github.com/actions/checkout/pull/1777">actions/checkout#1777</a>, <a href="https://redirect.github.com/actions/checkout/pull/1872">actions/checkout#1872</a></li> </ul> <h2>v4.1.7</h2> <ul> <li>Bump the minor-npm-dependencies group across 1 directory with 4 updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1739">actions/checkout#1739</a></li> <li>Bump actions/checkout from 3 to 4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1697">actions/checkout#1697</a></li> <li>Check out other refs/* by commit by <a href="https://github.com/orhantoy"><code>@orhantoy</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1774">actions/checkout#1774</a></li> <li>Pin actions/checkout's own workflows to a known, good, stable version. by <a href="https://github.com/jww3"><code>@jww3</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1776">actions/checkout#1776</a></li> </ul> <h2>v4.1.6</h2> <ul> <li>Check platform to set archive extension appropriately by <a href="https://github.com/cory-miller"><code>@cory-miller</code></a> in <a href="https://redirect.github.com/actions/checkout/pull/1732">actions/checkout#1732</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
deccc26f1f |
chore(deps): bump actions/cache from 4 to 5 (#11665)
Bumps [actions/cache](https://github.com/actions/cache) from 4 to 5. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/cache/releases">actions/cache's releases</a>.</em></p> <blockquote> <h2>v5.0.0</h2> <blockquote> <p>[!IMPORTANT] <strong><code>actions/cache@v5</code> runs on the Node.js 24 runtime and requires a minimum Actions Runner version of <code>2.327.1</code>.</strong></p> <p>If you are using self-hosted runners, ensure they are updated before upgrading.</p> </blockquote> <hr /> <h2>What's Changed</h2> <ul> <li>Upgrade to use node24 by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/cache/pull/1630">actions/cache#1630</a></li> <li>Prepare v5.0.0 release by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/cache/pull/1684">actions/cache#1684</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/cache/compare/v4.3.0...v5.0.0">https://github.com/actions/cache/compare/v4.3.0...v5.0.0</a></p> <h2>v4.3.0</h2> <h2>What's Changed</h2> <ul> <li>Add note on runner versions by <a href="https://github.com/GhadimiR"><code>@GhadimiR</code></a> in <a href="https://redirect.github.com/actions/cache/pull/1642">actions/cache#1642</a></li> <li>Prepare <code>v4.3.0</code> release by <a href="https://github.com/Link"><code>@Link</code></a>- in <a href="https://redirect.github.com/actions/cache/pull/1655">actions/cache#1655</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/GhadimiR"><code>@GhadimiR</code></a> made their first contribution in <a href="https://redirect.github.com/actions/cache/pull/1642">actions/cache#1642</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/cache/compare/v4...v4.3.0">https://github.com/actions/cache/compare/v4...v4.3.0</a></p> <h2>v4.2.4</h2> <h2>What's Changed</h2> <ul> <li>Update README.md by <a href="https://github.com/nebuk89"><code>@nebuk89</code></a> in <a href="https://redirect.github.com/actions/cache/pull/1620">actions/cache#1620</a></li> <li>Upgrade <code>@actions/cache</code> to <code>4.0.5</code> and move <code>@protobuf-ts/plugin</code> to dev depdencies by <a href="https://github.com/Link"><code>@Link</code></a>- in <a href="https://redirect.github.com/actions/cache/pull/1634">actions/cache#1634</a></li> <li>Prepare release <code>4.2.4</code> by <a href="https://github.com/Link"><code>@Link</code></a>- in <a href="https://redirect.github.com/actions/cache/pull/1636">actions/cache#1636</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/nebuk89"><code>@nebuk89</code></a> made their first contribution in <a href="https://redirect.github.com/actions/cache/pull/1620">actions/cache#1620</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/cache/compare/v4...v4.2.4">https://github.com/actions/cache/compare/v4...v4.2.4</a></p> <h2>v4.2.3</h2> <h2>What's Changed</h2> <ul> <li>Update to use <code>@actions/cache</code> 4.0.3 package & prepare for new release by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/cache/pull/1577">actions/cache#1577</a> (SAS tokens for cache entries are now masked in debug logs)</li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> made their first contribution in <a href="https://redirect.github.com/actions/cache/pull/1577">actions/cache#1577</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/cache/compare/v4.2.2...v4.2.3">https://github.com/actions/cache/compare/v4.2.2...v4.2.3</a></p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/actions/cache/blob/main/RELEASES.md">actions/cache's changelog</a>.</em></p> <blockquote> <h1>Releases</h1> <h2>Changelog</h2> <h3>5.0.1</h3> <ul> <li>Update <code>@azure/storage-blob</code> to <code>^12.29.1</code> via <code>@actions/cache@5.0.1</code> <a href="https://redirect.github.com/actions/cache/pull/1685">#1685</a></li> </ul> <h3>5.0.0</h3> <blockquote> <p>[!IMPORTANT] <code>actions/cache@v5</code> runs on the Node.js 24 runtime and requires a minimum Actions Runner version of <code>2.327.1</code>. If you are using self-hosted runners, ensure they are updated before upgrading.</p> </blockquote> <h3>4.3.0</h3> <ul> <li>Bump <code>@actions/cache</code> to <a href="https://redirect.github.com/actions/toolkit/pull/2132">v4.1.0</a></li> </ul> <h3>4.2.4</h3> <ul> <li>Bump <code>@actions/cache</code> to v4.0.5</li> </ul> <h3>4.2.3</h3> <ul> <li>Bump <code>@actions/cache</code> to v4.0.3 (obfuscates SAS token in debug logs for cache entries)</li> </ul> <h3>4.2.2</h3> <ul> <li>Bump <code>@actions/cache</code> to v4.0.2</li> </ul> <h3>4.2.1</h3> <ul> <li>Bump <code>@actions/cache</code> to v4.0.1</li> </ul> <h3>4.2.0</h3> <p>TLDR; The cache backend service has been rewritten from the ground up for improved performance and reliability. <a href="https://github.com/actions/cache">actions/cache</a> now integrates with the new cache service (v2) APIs.</p> <p>The new service will gradually roll out as of <strong>February 1st, 2025</strong>. The legacy service will also be sunset on the same date. Changes in these release are <strong>fully backward compatible</strong>.</p> <p><strong>We are deprecating some versions of this action</strong>. We recommend upgrading to version <code>v4</code> or <code>v3</code> as soon as possible before <strong>February 1st, 2025.</strong> (Upgrade instructions below).</p> <p>If you are using pinned SHAs, please use the SHAs of versions <code>v4.2.0</code> or <code>v3.4.0</code></p> <p>If you do not upgrade, all workflow runs using any of the deprecated <a href="https://github.com/actions/cache">actions/cache</a> will fail.</p> <p>Upgrading to the recommended versions will not break your workflows.</p> <h3>4.1.2</h3> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Swifty
|
5ac941fe2f |
feat(backend): add hybrid search for store listings, docs and blocks (#11721)
This PR adds hybrid search functionality combining semantic embeddings with traditional text search for improved store listing discovery. ### Changes 🏗️ - Add `embeddings.py` - OpenAI-based embedding generation and similarity search - Add `hybrid_search.py` - Combines vector similarity with text matching for better search results - Add `backfill_embeddings.py` - Script to generate embeddings for existing store listings - Update `db.py` - Integrate hybrid search into store database queries - Update `schema.prisma` - Add embedding storage fields and indexes - Add migrations for embedding columns and HNSW index for vector search ### Architecture Decisions 🏛️ **Fail-Fast Approach (No Silent Fallbacks)** We explicitly chose NOT to implement graceful degradation when hybrid search fails. Here's why: ✅ **Benefits:** - Errors surface immediately → faster fixes - Tests verify hybrid search actually works (not just fallback) - Consistent search quality for all users - Forces proper infrastructure setup (API keys, database) ❌ **Why Not Fallback:** - Silent degradation hides production issues - Users get inconsistent results without knowing why - Tests can pass even when hybrid search is broken - Reduces operational visibility **How We Prevent Failures:** 1. Embedding generation in approval flow (db.py:1545) 2. Error logging with `logger.error` (not warning) 3. Clear error messages (ValueError explains what's wrong) 4. Comprehensive test coverage (9/9 tests passing) If embeddings fail, it indicates a real infrastructure issue (missing API key, OpenAI down, database issues) that needs immediate attention, not silent degradation. ### Test Coverage ✅ **All tests passing (1625 total):** - 9/9 hybrid_search tests (including fail-fast validation) - 3/3 db search integration tests - Full schema compatibility (public/platform schemas) - Error handling verification ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Test hybrid search returns relevant results - [x] Test embedding generation for new listings - [x] Test backfill script on existing data - [x] Verify search performance with embeddings - [x] Test fail-fast behavior when embeddings unavailable #### For configuration changes: - [x] `.env.default` is updated or already compatible with my changes - [x] `docker-compose.yml` is updated or already compatible with my changes - [x] Configuration: Requires `openai_internal_api_key` in secrets --------- Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Swifty
|
843c487500 |
feat(backend): add prisma types stub generator for pyright compatibility (#11736)
Prisma's generated `types.py` file is 57,000+ lines with complex recursive TypedDict definitions that exhaust Pyright's type inference budget. This causes random type errors and makes the type checker unreliable. ### Changes 🏗️ - Add `gen_prisma_types_stub.py` script that generates a lightweight `.pyi` stub file - The stub preserves safe types (Literal, TypeVar) while collapsing complex TypedDicts to `dict[str, Any]` - Integrate stub generation into all workflows that run `prisma generate`: - `platform-backend-ci.yml` - `claude.yml` - `claude-dependabot.yml` - `copilot-setup-steps.yml` - `docker-compose.platform.yml` - `Dockerfile` - `Makefile` (migrate & reset-db targets) - `linter.py` (lint & format commands) - Add `gen-prisma-stub` poetry script entry - Fix two pre-existing type errors that were previously masked: - `store/db.py`: Replace private type `_StoreListingVersion_version_OrderByInput` with dict literal - `airtable/_webhook.py`: Add cast for `Serializable` type ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] Run `poetry run format` - passes with 0 errors (down from 57+) - [x] Run `poetry run lint` - passes with 0 errors - [x] Run `poetry run gen-prisma-stub` - generates stub successfully - [x] Verify stub file is created at correct location with proper content #### For configuration changes: - [x] `.env.default` is updated or already compatible with my changes - [x] `docker-compose.yml` is updated or already compatible with my changes - [x] I have included a list of my configuration changes in the PR description (under **Changes**) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Added a lightweight Prisma type-stub generator and integrated it into build, lint, CI/CD, and container workflows. * Build, migration, formatting, and lint steps now generate these stubs to improve type-checking performance and reduce overhead during builds and deployments. * Exposed a project command to run stub generation manually. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Reinier van der Leer
|
a978e91271 |
fix(ci, backend): Update Redis image & amend config to work with it (#10952)
CI is currently broken because Bitnami has pulled all `bitnami/redis` images. The current official Redis image on Docker Hub is `redis`. ### Changes 🏗️ - Replace `bitnami/redis:6.2` by `redis:latest` in Backend CI workflow file - Make `REDIS_PASSWORD` optional in the backend settings ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] CI no longer broken |
||
|
Reinier van der Leer
|
12cdd45551 |
refactor(backend): Improve auth setup & OpenAPI generation (#10720)
Our current auth setup (`autogpt_libs.auth` + its usage) is quite inconsistent and doesn't do all of its jobs properly. The 401 responses you get when unauthenticated are not included in the OpenAPI spec, causing these to be unaccounted for in the generated frontend API client. Usage of the FastAPI dependencies supplied by `autogpt_libs.auth.depends` aren't consistently used the same way, making maintenance on these hard to oversee. API tests use many different ways to get around the auth requirement, making this also hard to maintain and oversee. This pull request aims to fix all of this and give us a consistent, clean, and self-documenting API auth implementation. - Resolves #10715 ### Changes 🏗️ - Homogenize use of `autogpt_libs.auth` security dependencies throughout the backend - Fix OpenAPI schema generation for 401 responses - Handle possible 401 responses in frontend - Tighten validation and add warnings for weak settings in `autogpt_libs.auth.config` - Increase test coverage for `autogpt_libs.auth` to 100% - Standardize auth setup for API tests - Rename `APIKeyValidator` to `APIKeyAuthenticator` and move to its own module in `backend.server` ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] All tests for `autogpt_libs.auth` pass - [x] All tests for `backend.server` pass - [x] @ntindle does a security audit for these changes - [x] OpenAPI spec for authenticated routes is generated with the appropriate `401` response --------- Co-authored-by: Nicholas Tindle <nicholas.tindle@agpt.co> |
||
|
Reinier van der Leer
|
f4538d6f5a |
build(backend): Change base image to debian:13-slim w/ Python 3.13 (#10654)
- Resolves #10653 The objective is to move to a base image with fewer active vulnerabilities. Hence the choice for `debian:13-slim` (0 high, 1 medium, 21 low severity), a huge improvement compared to our current base image `python:3.11.10-slim-bookworm` (4 high, 11 medium, 15 low severity). ### Changes 🏗️ - Change backend base image to `debian:13-slim` - Use Python 3.13 - Fix now-deprecated use of class property in `AppProcess` and `BaseAppService` - Expand backend CI matrix to run with Python 3.11 through 3.13 - Update Python version constraint in `pyproject.toml` to include Python 3.13 Also, unrelated: - Update `autogpt-platform-backend` package version to `v0.6.22`, the latest release ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: - [x] CI passes - [x] No new errors in deployment logs - [x] Everything seems to work normally in deployment |
||
|
Zamil Majdy
|
2f12e8d731 |
feat(platform): Add Host-scoped credentials support for blocks HTTP requests (#10215)
Currently, we don't have a secure way to pass Authorization headers when calling the `SendWebRequestBlock`. This hinders the integration of third-party applications that do not yet have native block support. ### Changes 🏗️ Add Host-scoped credentials support for the newly introduced SendAuthenticatedWebRequestBlock. <img width="1000" alt="image" src="https://github.com/user-attachments/assets/0d3d577a-2b9b-4f0f-9377-0e00a069ba37" /> <img width="1000" alt="image" src="https://github.com/user-attachments/assets/a59b9f16-c89c-453d-a628-1df0dfd60fb5" /> ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: <!-- Put your test plan here: --> - [x] Uses `https://api.openai.com/v1/images/edits` through SendWebRequestBlock by passing the api-key through host-scoped credentials. |
||
|
Zamil Majdy
|
68749a28d4 |
feat(backend): Add ClamAV support for anti-virus scan on file upload to the platform (#10232)
An anti-virus file scan step is added to each file upload step on the platform before the file is sent to cloud storage or local machine storage. ### Changes 🏗️ * Added ClamAV service * Added AV file scan on each upload step * Added tests & documentation * Make the step mandatory even on local development ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: <!-- Put your test plan here: --> - [x] Tried using FileUploadBlock & AgentFileInputBlock |
||
|
Bently
|
d5d613e014 |
chore(backend): Downgrade poetry to 2.1.1 for dependabot (#10079)
Co-authored-by: Reinier van der Leer <pwuts@agpt.co> |
||
|
Nicholas Tindle
|
3e0742f9c5 |
Spike/infra pooling (#9812)
<!-- Clearly explain the need for these changes: --> Swap to pooling supabase connections rather than depending on x number of max open connections ### Changes 🏗️ Adds direct connect URL to be used throughout the system <!-- Concisely describe all of the changes made in this pull request: --> ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: <!-- Put your test plan here: --> - [x] Test thoroughly all of the endpoints in the dev env with switched infra matching pr - [x] Follow the new release plan tests - [x] Follow the old release plan tests #### For configuration changes: - [x] `.env.example` is updated or already compatible with my changes - [x] `docker-compose.yml` is updated or already compatible with my changes - [x] I have included a list of my configuration changes in the PR description (under **Changes**) <details> <summary>configuration changes</summary> - Change how we connect to the database to use direct when configured and database URL when not - update prisma for this - have default matching database and default </details> |
||
|
Reinier van der Leer
|
8e912a016f |
fix(ci/backend): Use Poetry version from lockfile (#9729)
Currently, our CI always uses the latest version of Poetry. This causes issues with the lockfile check whenever a new Poetry version is released, especially if that new version has different lockfile generation behavior. This new mechanism determines the Poetry version to use as follows: - Get Poetry version from backend/poetry.lock in the current branch - Get Poetry version from backend/poetry.lock on the base branch - Use the newest version out of the two found versions This way, we don't automatically update to new Poetry versions, but it is still possible to update to newer versions through pull requests. |
||
|
Zamil Majdy
|
f37d4c2659 |
fix(platform): Set Supabase CLI version (#9619)
<!-- Clearly explain the need for these changes: --> ### Changes 🏗️ <!-- Concisely describe all of the changes made in this pull request: --> ### Checklist 📋 #### For code changes: - [ ] I have clearly listed my changes in the PR description - [ ] I have made a test plan - [ ] I have tested my changes according to the test plan: <!-- Put your test plan here: --> - [ ] ... <details> <summary>Example test plan</summary> - [ ] Create from scratch and execute an agent with at least 3 blocks - [ ] Import an agent from file upload, and confirm it executes correctly - [ ] Upload agent to marketplace - [ ] Import an agent from marketplace and confirm it executes correctly - [ ] Edit an agent from monitor, and confirm it executes correctly </details> #### For configuration changes: - [ ] `.env.example` is updated or already compatible with my changes - [ ] `docker-compose.yml` is updated or already compatible with my changes - [ ] I have included a list of my configuration changes in the PR description (under **Changes**) <details> <summary>Examples of configuration changes</summary> - Changing ports - Adding new services that need to communicate with each other - Secrets or environment variable changes - New or infrastructure changes such as databases </details> |
||
|
Nicholas Tindle
|
15275e2ce1 |
feat(backend): spawn the notifications service + basic test (#9464)
We want to send emails on a schedule, in response to events, and be expandable without being overbearing on the amount of effort to implement. We also want this to use rabbitmq and be easy for other services to send messages into. This PR adds the first use of the service to simply show a log message ### Changes 🏗️ <!-- Concisely describe all of the changes made in this pull request: --> - Adds a new backend service for notifications - Adds first notification into the service -> Agent Execution - Adds spawning the notification service Also - Adds RabbitMQ to CI so we can test stuff - Adds a minor fix for one of the migrations that I thought was causing failures, but isn't but the change is still useful ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [x] I have tested my changes according to the test plan: <!-- Put your test plan here: --> - [x] Built and ran an agent and ensured the following log line appeared which shows the event would have sent an email ``` 2025-02-10 15:52:02,232 INFO Processing notification: user_id='96b8d2f5-a036-437f-bd8e-ba8856028553' type=<NotificationType.AGENT_RUN: 'AGENT_RUN'> data=AgentRunData(agent_name='CalculatorBlock', credits_used=0.0, execution_time=0.0, graph_id='30e5f332-a092-4795-892a-b063a8c7bdd9', node_count=1) created_at=datetime.datetime(2025, 2, 10, 15, 52, 2, 162865) ``` #### For configuration changes: - [ ] `.env.example` is updated or already compatible with my changes - [ ] `docker-compose.yml` is updated or already compatible with my changes - [ ] I have included a list of my configuration changes in the PR description (under **Changes**) None of the other ports are configurable via .env.example listing so left as is <details> <summary>Examples of configuration changes</summary> - Changing ports - Adding new services that need to communicate with each other - Secrets or environment variable changes - New or infrastructure changes such as databases </details> --------- Co-authored-by: Reinier van der Leer <pwuts@agpt.co> |
||
|
Reinier van der Leer
|
d638c1f484 |
Fix Poetry v2.0.0 compatibility (#9197)
Make all changes necessary to make everything work with Poetry v2.0.0. - Resolves #9196 ## Changes - Removed `--no-update` flag from `poetry lock` command in codebase - Removed extra path arguments from `poetry -C [path] run [command]` occurrences - Regenerated all lock files in hierarchical order - Added workaround for Poetry bug where `packages.[i].format` is now suddenly required Additionally: - Fixed up .dockerignore - Fixes .venv being erroneously copied over from local - Fixes build context bloat (300MB -> 2.5MB) - Fixed warnings about entrypoint script not being installed in docker builds ### Relevant (breaking) changes in v2.0.0 - `--no-update` flag no longer exists for `poetry lock` as it has become default behavior - The `-C` option now actually changes the directory, so any path arguments in `poetry run` commands can/must be removed - Poetry v2.0.0 uses the new v2.1 lock file spec, so all lock files have to be regenerated to avoid false-positive lock file updates and checks on future PRs - **BUG:** when specifying `poetry.tool.packages`, `format` is required now - python-poetry/poetry#9961 Full Poetry v2.0.0 release notes and change log: https://python-poetry.org/blog/announcing-poetry-2.0.0 |
||
|
Reinier van der Leer
|
0272d87af3 |
ci(backend): Add poetry.lock check (#8885)
- Resolves #8884 We need to prevent breaking updates to dependency version requirements of `autogpt_libs`. `autogpt_libs/pytroject.toml` and `backend/poetry.lock` are loosely coupled, and to ensure they stay in sync we need an extra check. For now I'm also reverting the breaking update of #8787, otherwise this added CI check will immediately fail. ### Changes - ci(backend): Add `poetry.lock` check - Revert "build(deps): bump pydantic from 2.9.2 to 2.10.2 in /autogpt_platform/autogpt_libs in the production-dependencies group across 1 directory (#8787)" |
||
|
Reinier van der Leer
|
5ccfb8e4c6 |
dx(backend): Fix linting & formatting for autogpt_libs (#8860)
- Resolves #8859 - Follow-up to #8751 ### Changes - Add `autogpt_libs` to the backend CI path filter - Add `ruff format` step for `autogpt_libs` to `linter.py` and `pre-commit` config - Run `poetry run format` with the new setup |
||
|
Reinier van der Leer
|
118fdeeb1d |
ci: Add merge_group triggers
|
||
|
Aarushi
|
a2ef456525 |
fix(ci,platform): Add dev branch trigger to all ci (#8339)
* update ci for dev * update classic * remove duplicate dev |
||
|
Zamil Majdy
|
daa054c79c | feat(backend): Make Redis connection Sync + Use Redis as Distributed Lock (#8197) | ||
|
Reinier van der Leer
|
5e2874c315 |
feat(platform): OAuth support + API key management + GitHub blocks (#8044)
## Config
- For Supabase, the back end needs `SUPABASE_URL`, `SUPABASE_SERVICE_ROLE_KEY`, and `SUPABASE_JWT_SECRET`
- For the GitHub integration to work, the back end needs `GITHUB_CLIENT_ID` and `GITHUB_CLIENT_SECRET`
- For integrations OAuth flows to work in local development, the back end needs `FRONTEND_BASE_URL` to generate login URLs with accurate redirect URLs
## REST API
- Tweak output of OAuth `/login` endpoint: add `state_token` separately in response
- Add `POST /integrations/{provider}/credentials` (for API keys)
- Add `DELETE /integrations/{provider}/credentials/{cred_id}`
## Back end
- Add Supabase support to `AppService`
- Add `FRONTEND_BASE_URL` config option, mainly for local development use
### `autogpt_libs.supabase_integration_credentials_store`
- Add `CredentialsType` alias
- Add `.bearer()` helper methods to `APIKeyCredentials` and `OAuth2Credentials`
### Blocks
- Add `CredentialsField(..) -> CredentialsMetaInput`
## Front end
### UI components
- `CredentialsInput` for use on `CustomNode`: allows user to add/select credentials for a service.
- `APIKeyCredentialsModal`: a dialog for creating API keys
- `OAuth2FlowWaitingModal`: a dialog to indicate that the application is waiting for the user to log in to the 3rd party service in the provided pop-up window
- `NodeCredentialsInput`: wrapper for `CredentialsInput` with the "usual" interface of node input components
- New icons: `IconKey`, `IconKeyPlus`, `IconUser`, `IconUserPlus`
### Data model
- `CredentialsProvider`: introduces the app-level `CredentialsProvidersContext`, which acts as an application-wide store and cache for credentials metadata.
- `useCredentials` for use on `CustomNode`: uses `CredentialsProvidersContext` and provides node-specific credential data and provider-specific data/functions
- `/auth/integrations/oauth_callback` route to close the loop to the `CredentialsInput` after a user completes sign-in to the external service
- Add `BlockIOCredentialsSubSchema`
### API client
- Add `isAuthenticated` method
- Add methods for integration OAuth flow: `oAuthLogin`, `oAuthCallback`
- Add CRD methods for credentials: `createAPIKeyCredentials`, `listCredentials`, `getCredentials`, `deleteCredentials`
- Add mirrored types `CredentialsMetaResponse`, `CredentialsMetaInput`, `OAuth2Credentials`, `APIKeyCredentials`
- Add GitHub blocks + "DEVELOPER_TOOLS" category
- Add `**kwargs` to `Block.run(..)` signature to support additional kwargs
- Add support for loading blocks from nested modules (e.g. `blocks/github/issues.py`)
#### Executor
- Add strict support for `credentials` fields on blocks
- Fetch credentials for graph execution and pass them down through to the node execution
|
||
|
Reinier van der Leer
|
c533044cdc |
ci(backend): Add Supabase (#8123)
- Set up Supabase in the Backend CI - Remove separate Postgres DB setup - Remove unused MinIO service - Remove multi-platform stuff |
||
|
Swifty
|
ef7cfbb860 |
refactor: AutoGPT Platform Stealth Launch Repo Re-Org (#8113)
Restructuring the Repo to make it clear the difference between classic autogpt and the autogpt platform: * Move the "classic" projects `autogpt`, `forge`, `frontend`, and `benchmark` into a `classic` folder * Also rename `autogpt` to `original_autogpt` for absolute clarity * Rename `rnd/` to `autogpt_platform/` * `rnd/autogpt_builder` -> `autogpt_platform/frontend` * `rnd/autogpt_server` -> `autogpt_platform/backend` * Adjust any paths accordingly |