Commit Graph

13 Commits

Author SHA1 Message Date
Otto
582c6cad36 fix(e2e): Make E2E test data deterministic and fix flaky tests (#11890)
## Summary
Fixes flaky E2E marketplace and library tests that were causing PRs to
be removed from the merge queue.

## Root Cause
1. **Test data was probabilistic** - `e2e_test_data.py` used random
chances (40% approve, then 20-50% feature), which could result in 0
featured agents
2. **Library pagination threshold wrong** - Checked `>= 10`, but page
size is 20
3. **Fixed timeouts** - Used `waitForTimeout(2000)` /
`waitForTimeout(10000)` instead of proper waits

## Changes

### Backend (`e2e_test_data.py`)
- Add guaranteed minimums: 8 featured agents, 5 featured creators, 10
top agents
- First N submissions are deterministically approved and featured
- Increase agents per user from 15 → 25 (for pagination with
page_size=20)
- Fix library agent creation to use constants instead of hardcoded `10`

### Frontend Tests
- `library.spec.ts`: Fix pagination threshold to `PAGE_SIZE` (20)
- `library.page.ts`: Replace 2s timeout with `networkidle` +
`waitForFunction`
- `marketplace.page.ts`: Add `networkidle` wait, 30s waits in
`getFirst*` methods
- `marketplace.spec.ts`: Replace 10s timeout with `waitForFunction`
- `marketplace-creator.spec.ts`: Add `networkidle` + element waits

## Related
- Closes SECRT-1848, SECRT-1849
- Should unblock #11841 and other PRs in merge queue

---------

Co-authored-by: Ubbe <hi@ubbe.dev>
2026-01-30 05:12:35 +00:00
Reinier van der Leer
de78d062a9 refactor(backend/api): Clean up API file structure (#11629)
We'll soon be needing a more feature-complete external API. To make way
for this, I'm moving some files around so:
- We can more easily create new versions of our external API
- The file structure of our internal API is more homogeneous

These changes are quite opinionated, but IMO in any case they're better
than the chaotic structure we have now.

### Changes 🏗️

- Move `backend/server` -> `backend/api`
- Move `backend/server/routers` + `backend/server/v2` ->
`backend/api/features`
  - Change absolute sibling imports to relative imports
- Move `backend/server/v2/AutoMod` -> `backend/executor/automod`
- Combine `backend/server/routers/analytics_*test.py` ->
`backend/api/features/analytics_test.py`
- Sort OpenAPI spec file

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - CI tests
  - [x] Clicking around in the app -> no obvious breakage
2025-12-20 20:33:10 +00:00
Reinier van der Leer
3dbc03e488 feat(platform): OAuth API & Single Sign-On (#11617)
We want to provide Single Sign-On for multiple AutoGPT apps that use the
Platform as their backend.

### Changes 🏗️

Backend:
- DB + logic + API for OAuth flow (w/ tests)
  - DB schema additions for OAuth apps, codes, and tokens
  - Token creation/validation/management logic
- OAuth flow endpoints (app info, authorize, token exchange, introspect,
revoke)
  - E2E OAuth API integration tests
- Other OAuth-related endpoints (upload app logo, list owned apps,
external `/me` endpoint)
    - App logo asset management
  - Adjust external API middleware to support auth with access token
  - Expired token clean-up job
    - Add `OAUTH_TOKEN_CLEANUP_INTERVAL_HOURS` setting (optional)
- `poetry run oauth-tool`: dev tool to test the OAuth flows and register
new OAuth apps
- `poetry run export-api-schema`: dev tool to quickly export the OpenAPI
schema (much quicker than spinning up the backend)

Frontend:
- Frontend UI for app authorization (`/auth/authorize`)
  - Re-redirect after login/signup
- Frontend flow to batch-auth integrations on request of the client app
(`/auth/integrations/setup-wizard`)
  - Debug `CredentialInputs` component
- Add `/profile/oauth-apps` management page
- Add `isOurProblem` flag to `ErrorCard` to hide action buttons when the
error isn't our fault
- Add `showTitle` flag to `CredentialsInput` to hide built-in title for
layout reasons

DX:
- Add [API
guide](https://github.com/Significant-Gravitas/AutoGPT/blob/pwuts/sso/docs/content/platform/integrating/api-guide.md)
and [OAuth
guide](https://github.com/Significant-Gravitas/AutoGPT/blob/pwuts/sso/docs/content/platform/integrating/oauth-guide.md)

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Manually verify test coverage of OAuth API tests
  - Test `/auth/authorize` using `poetry run oauth-tool test-server`
    - [x] Works
    - [x] Looks okay
- Test `/auth/integrations/setup-wizard` using `poetry run oauth-tool
test-server`
    - [x] Works
    - [x] Looks okay
  - Test `/profile/oauth-apps` page
    - [x] All owned OAuth apps show up
    - [x] Enabling/disabling apps works
- [ ] ~~Uploading logos works~~ can only test this once deployed to dev

#### For configuration changes:

- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)
2025-12-19 21:05:16 +01:00
Reinier van der Leer
d68dceb9c1 fix(backend/executor): Improve graph execution permission check (#11323)
- Resolves #11316
- Durable fix to replace #11318

### Changes 🏗️

- Expand graph execution permissions check
  - Don't require library membership for execution as sub-graph

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Can run sub-agent with non-latest graph version
- [x] Can run sub-agent that is available in Marketplace but not added
to Library
2025-11-05 17:13:41 +00:00
Zamil Majdy
374f35874c feat(platform): Add LaunchDarkly flag for platform payment system (#11181)
## Summary

Implement selective rollout of payment functionality using LaunchDarkly
feature flags to enable gradual deployment to pilot users.

- Add `ENABLE_PLATFORM_PAYMENT` flag to control credit system behavior
- Update `get_user_credit_model` to use user-specific flag evaluation  
- Replace hardcoded `NEXT_PUBLIC_SHOW_BILLING_PAGE` with LaunchDarkly
flag
- Enable payment UI components only for flagged users
- Maintain backward compatibility with existing beta credit system
- Default to beta monthly credits when flag is disabled
- Fix tests to work with new async credit model function

## Key Changes

### Backend
- **Credit Model Selection**: The `get_user_credit_model()` function now
takes a `user_id` parameter and uses LaunchDarkly to determine which
credit model to return:
- Flag enabled → `UserCredit` (payment system enabled, no monthly
refills)
- Flag disabled → `BetaUserCredit` (current behavior with monthly
refills)
  
- **Flag Integration**: Added `ENABLE_PLATFORM_PAYMENT` flag and
integrated LaunchDarkly evaluation throughout the credit system

- **API Updates**: All credit-related endpoints now use the
user-specific credit model instead of a global instance

### Frontend
- **Dynamic UI**: Payment-related components (billing page, wallet
refill) now show/hide based on the LaunchDarkly flag
- **Removed Environment Variable**: Replaced
`NEXT_PUBLIC_SHOW_BILLING_PAGE` with runtime flag evaluation

### Testing
- **Test Fixes**: Updated all tests that referenced the removed global
`_user_credit_model` to use proper mocking of the new async function

## Deployment Strategy

This implementation enables a controlled rollout:
1. Deploy with flag disabled (default) - no behavior change for existing
users
2. Enable flag for pilot/beta users via LaunchDarkly dashboard
3. Monitor usage and feedback from pilot users
4. Gradually expand to more users
5. Eventually enable for all users once validated

## Test Plan

- [x] Unit tests pass for credit system components
- [x] Payment UI components show/hide correctly based on flag
- [x] Default behavior (flag disabled) maintains current functionality
- [x] Flag enabled users get payment system without monthly refills
- [x] Admin credit operations work correctly
- [x] Backward compatibility maintained

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-17 06:11:39 +00:00
Reinier van der Leer
2ffd249aac fix(backend/external-api): Improve security & reliability of API key storage (#10796)
Our API key generation, storage, and verification system has a couple of
issues that need to be ironed out before full-scale deployment.

### Changes 🏗️

- Move from unsalted SHA256 to salted Scrypt hashing for API keys
- Avoid false-negative API key validation due to prefix collision
- Refactor API key management code for clarity
- [refactor(backend): Clean up API key DB & API code
(#10797)](https://github.com/Significant-Gravitas/AutoGPT/pull/10797)
  - Rename models and properties in `backend.data.api_key` for clarity
- Eliminate redundant/custom/boilerplate error handling/wrapping in API
key endpoint call stack
- Remove redundant/inaccurate `response_model` declarations from API key
endpoints

Dependencies for `autogpt_libs`:
- Add `cryptography` as a dependency
- Add `pyright` as a dev dependency

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - Performing these actions through the UI (still) works:
    - [x] Creating an API key
    - [x] Listing owned API keys
    - [x] Deleting an owned API key
  - [x] Newly created API key can be used in Swagger UI
  - [x] Existing API key can be used in Swagger UI
  - [x] Existing API key is re-encrypted with salt on use
2025-09-10 09:34:49 +00:00
Abhimanyu Yadav
2610c4579f feat(platform/dashboard): Enable editing for agent submissions (#10545)
- resolves -
https://github.com/Significant-Gravitas/AutoGPT/issues/10511

In this PR, I’ve added backend endpoints and a frontend UI for edit
functionality on the Agent Dashboard. Now, users can update their store
submission, if status is `PENDING` or `APPROVED`, but not for `REJECTED`
and `DRAFT`. When users make changes to a pending status submission, the
changes are made to the same version. However, when users make changes
to an approved status submission, a new store listing version is
created.

Backend works something like this: 

<img width="866" height="832" alt="Screenshot 2025-08-15 at 9 39 02 AM"
src="https://github.com/user-attachments/assets/209c60ac-8350-43c1-ba4c-7378d95ecba7"
/>

### Changes
- I’ve updated the `StoreSubmission` view to include `video_url` and
`categories`.
- I’ve added a new frontend UI for editing submissions.
- I’ve created an endpoint for editing submissions.
- I’ve added more end-to-end tests to ensure the edit submission
functionality works as expected.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] I have checked manually, everything is working perfectly.
  - [x] All e2e tests are also passing.

---------

Co-authored-by: Zamil Majdy <zamil.majdy@agpt.co>
Co-authored-by: neo <neo.dowithless@gmail.com>
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
Co-authored-by: Swifty <craigswift13@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Ubbe <hi@ubbe.dev>
Co-authored-by: Lluis Agusti <hi@llu.lu>
2025-08-20 02:49:29 +00:00
Zamil Majdy
89eb5d1189 feat(feature-flag): add LaunchDarkly user context and metadata support (#10595)
## Summary

Enable LaunchDarkly feature flags to use rich user context and metadata
for advanced targeting, including user segments, account age, email
domains, and custom attributes. This unlocks LaunchDarkly's powerful
targeting capabilities beyond simple user ID checks.

## Problem

LaunchDarkly feature flags were only receiving basic user IDs,
preventing the use of:
- **Segment-based targeting** (e.g., "employees", "beta users", "new
accounts")
- **Contextual rules** (e.g., account age, email domain, custom
metadata)
- **Advanced LaunchDarkly features** like percentage rollouts by user
attributes

This limited feature flag flexibility and required manual user ID
management for targeting.

## Solution

### 🎯 **LaunchDarkly Context Enhancement**
- **Rich user context**: Send user metadata, segments, account age,
email domain to LaunchDarkly
- **Automatic segmentation**: Users automatically categorized as
"employee", "new_user", "established_user" etc.
- **Custom metadata support**: Any user metadata becomes available for
LaunchDarkly targeting
- **24-hour caching**: Efficient user context retrieval with TTL cache
to reduce database calls

### 📊 **User Context Data**
```python
# Before: Only user ID
context = Context.builder("user-123").build()

# After: Full context with targeting data
context = {
    "email": "user@agpt.co",
    "created_at": "2023-01-15T10:00:00Z",
    "segments": ["employee", "established_user"],
    "email_domain": "agpt.co", 
    "account_age_days": 365,
    "custom_role": "admin"
}
```

### 🏗️ **Required Infrastructure Changes**

To support proper LaunchDarkly serialization, we needed to implement
clean application models:

#### **Application-Layer User Model**
- Created snake_case User model (`created_at`, `email_verified`) for
proper JSON serialization
- LaunchDarkly expects consistent field naming - camelCase Prisma
objects caused validation errors
- Added `User.from_db()` converter to safely transform database objects

#### **HTTP Client Reliability**  
- Fixed HTTP 4xx retry issue that was causing unnecessary load
- Added layer validation to prevent database objects leaking to external
services

#### **Type Safety**
- Eliminated `Any` types and defensive coding patterns
- Proper typing enables better IDE support and catches errors early

## Technical Implementation

### **Core LaunchDarkly Enhancement**
```python
# autogpt_libs/feature_flag/client.py
@async_ttl_cache(maxsize=1000, ttl_seconds=86400)  # 24h cache
async def _fetch_user_context_data(user_id: str) -> dict[str, Any]:
    user = await get_user_by_id(user_id)
    return _build_launchdarkly_context(user)

def _build_launchdarkly_context(user: User) -> dict[str, Any]:
    return {
        "email": user.email,
        "created_at": user.created_at.isoformat(),  # snake_case for serialization
        "segments": determine_user_segments(user),
        "account_age_days": calculate_account_age(user),
        # ... more context data
    }
```

### **User Segmentation Logic**
- **Role-based**: `admin`, `user`, `system` segments
- **Domain-based**: `employee` for @agpt.co emails  
- **Account age**: `new_user` (<7 days), `recent_user` (7-30 days),
`established_user` (>30 days)
- **Custom metadata**: Any user metadata becomes available for targeting

### **Infrastructure Updates**
- `backend/data/model.py`: Application User model with proper
serialization
- `backend/util/service.py`: HTTP client improvements and layer
validation
- Multiple files: Migration to use application models for consistency

## LaunchDarkly Usage Examples

With this enhancement, you can now create LaunchDarkly rules like:

```yaml
# Target employees only
- variation: true
  targets:
    - values: ["employee"]
      contextKind: "user"
      attribute: "segments"

# Target new users for gradual rollout  
- variation: true
  rollout:
    variations:
      - variation: true
        weight: 25000  # 25% of new users
    contextKind: "user" 
    bucketBy: "segments"
    filters:
      - attribute: "segments"
        op: "contains"
        values: ["new_user"]
```

## Performance & Caching

- **24-hour TTL cache**: Dramatically reduces database calls for user
context
- **Graceful fallbacks**: Simple user ID context if database unavailable
- **Efficient caching**: 1000 entry LRU cache with automatic TTL
expiration

## Testing

- [x] LaunchDarkly context includes all expected user attributes
- [x] Segmentation logic correctly categorizes users
- [x] 24-hour cache reduces database load
- [x] Fallback to simple context works when database unavailable
- [x] All existing feature flag functionality preserved
- [x] HTTP retry improvements work correctly

## Breaking Changes

 **No external API changes** - all existing feature flag usage
continues to work

⚠️ **Internal changes only**:
- `get_user_by_id()` returns application User model instead of Prisma
model
- Test utilities need to import User from `backend.data.model`

## Impact

🎯 **Product Impact**:
- **Advanced targeting**: Product teams can now use sophisticated
LaunchDarkly rules
- **Better user experience**: Gradual rollouts, A/B testing, and
segment-based features
- **Operational efficiency**: Reduced need for manual user ID management

🚀 **Performance Impact**:
- **Reduced database load**: 24-hour caching minimizes repeated user
context queries
- **Improved reliability**: Fixed HTTP retry inefficiencies
- **Better monitoring**: Cleaner logs without 4xx retry noise

---

**Primary goal**: Enable rich LaunchDarkly targeting with user context
and segments
**Infrastructure changes**: Required for proper serialization and
reliability

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-12 05:25:56 +00:00
Abhimanyu Yadav
4f208d262e test(frontend): add e2e tests for agent dashboard page (#10572)
I have added e2e tests for agent dashboard page

It includes, tests like 
- dashboard page loads successfully
- submit agent button works correctly
- agent table displays data correctly
- agent table actions work correctly

I’ve also updated the e2e test script to include some static agent
submissions, so I can test if it loads on the frontend.

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] All tests are working perfectly locally
  
  
<img width="469" height="177" alt="Screenshot 2025-08-08 at 12 13 42 PM"
src="https://github.com/user-attachments/assets/5e37afc3-c151-476a-84de-0a06f44a0722"
/>
2025-08-08 07:29:11 +00:00
Abhimanyu Yadav
9848266474 test(frontend): e2e tests for library page (#10355)
In this PR, I’ve added library page tests.

### Changes

I’ve added 9 tests: 8 for normal flows and 1 for checking edge cases.

Test names are something like:
- Library navigation is accessible from the navbar.
- The library page loads successfully.
- Agents are visible, and cards work correctly.
- Pagination works correctly.
- Sorting works correctly.
- Searching works correctly.
- Pagination while searching works correctly.
- Uploading an agent works correctly.
- Edge case: Search edge cases and error handling behave correctly.

Other than that, I’ve added a new utility that uses the build page to
help us create users at the start, which we could use to test the
library page.

- All tests are passing locally

<img width="514" height="465" alt="Screenshot 2025-07-12 at 11 13 41 AM"
src="https://github.com/user-attachments/assets/7a46c437-7db5-458b-b99a-4fa0d479866f"
/>

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] All library tests are working locally and on CI perfectly.
2025-08-06 08:00:04 +00:00
Abhimanyu Yadav
878f61aaf4 fix(test): Enhance E2E test data script to include featured creators and agents (#10517)
This PR updates the existing E2E test data script to support the
creation of featured creators and featured agents. Previously, these
entities were not included, which limited our ability to fully test
certain flows during Playwright E2E testing.

### Changes
- Added logic to create featured creators
- Added logic to create featured agents

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] All tests are passing locally after updating the data script.
2025-08-01 11:09:39 +00:00
Zamil Majdy
0c9b7334c1 feat(backend): Register agent subgraphs as library entries during agent import (#10409)
Currently, we only create a library entry of the top-most graph when
importing the graph from an exported file.
This can cause some complications, as there is no way to remove the
library entry of it.

### Changes 🏗️

Create the library entry for all the subgraphs during the import
process.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  <!-- Put your test plan here: -->
  - [x] Export an agent with subgraphs and import it back.
2025-07-21 11:54:42 +00:00
Abhimanyu Yadav
e720f92123 feat(tests): Add E2E test data creator script for comprehensive testing (#10368)
### Changes
- Introduced a new script to generate test data for end-to-end (E2E)
tests using API functions, ensuring compatibility with future model
changes.
- The script creates test users, agent blocks, graphs, profiles, library
agents, presets, API keys, and store submissions.
- Utilizes external services for image and video URLs, and includes
error handling for data creation processes.
- Provides a summary of created data upon completion, enhancing the
testing framework for the AutoGPT platform.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
- [x] Test scripts are working perfectly and not breaking anything. Data
is also correctly visible in the database.
2025-07-17 15:47:21 +00:00