Commit Graph

1075 Commits

Author SHA1 Message Date
Nicholas Tindle
270586751b style(backend): format workspace routes and rest_api imports
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:40:12 -06:00
Nicholas Tindle
fa1afd6a6d fix(backend): replace assert with ValueError in FileReadBlock
Asserts can be stripped with -O flag. Use explicit ValueError for
graph_exec_id validation to ensure consistent error handling.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:39:35 -06:00
Nicholas Tindle
af2bcd900a fix(backend): remove dead output_return_type from media blocks
The output_return_type field was defined in Input but never wired up to
store_media_file. The code always used for_block_output. Removed the
misleading field from LoopVideoBlock and AddAudioToVideoBlock.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:39:27 -06:00
Nicholas Tindle
8f7204484d fix(backend): handle race condition in WorkspaceManager.write_file
- Wrap create_workspace_file in try/except for UniqueViolationError
- On conflict with overwrite=True, delete existing and retry
- Remove unused file_exists method (dead code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:39:14 -06:00
Nicholas Tindle
5af4c60b8e fix(backend): use upsert in get_or_create_workspace to prevent race condition
Concurrent first-time requests for the same user could both find no workspace
and attempt to create, causing unique constraint violation. Using upsert
handles this atomically.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:39:01 -06:00
Nicholas Tindle
49b67ccd94 fix(backend): resolve circular import in file.py and remove deprecated params
- Defer WorkspaceManager import to inside store_media_file() to break circular import
- Remove deprecated return_content and save_to_workspace parameters (no callers)
- Make return_format a required parameter
- Update tests to use return_format

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:38:51 -06:00
Nicholas Tindle
efb2e2792d fix(backend): resolve circular import in workspace_storage.py
Defer sanitize_filename import to inside _build_file_path() to break
the circular import chain: workspace_storage → file → workspace → data → blocks → file

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:38:40 -06:00
Nicholas Tindle
d51d811497 fix(backend): security fixes and dead code removal
- routes.py: Sanitize filename in Content-Disposition header to prevent
  header injection (RFC5987 encoding for non-ASCII)
- http.py: Replace assert with explicit ValueError for graph_exec_id check
  (asserts can be stripped with -O)
- workspace.py: Remove unused functions (get_workspace_by_id,
  hard_delete_workspace_file, update_workspace_file)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:14:22 -06:00
Nicholas Tindle
83f93d00f4 fix(backend): add shutdown hook for workspace storage
Add shutdown_workspace_storage() to properly close GCS aiohttp sessions
during application shutdown. Follows the same pattern as cloud_storage.py.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:09:04 -06:00
Nicholas Tindle
c132b6dfa5 fix(backend): prevent path traversal in local workspace storage
- Sanitize filenames using sanitize_filename() before building paths
- Add is_relative_to() check after path resolution for defense in depth
- Replace string comparison with is_relative_to() in _parse_storage_path()
  for robust path containment on case-insensitive filesystems

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:04:53 -06:00
Nicholas Tindle
7eb7b7186f fix: lint 2026-01-28 00:01:24 -06:00
Nicholas Tindle
4b58eac877 fix(backend): prevent race condition in concurrent node execution context
Use model_copy() instead of mutating shared ExecutionContext to prevent
race conditions when multiple nodes execute concurrently. Each node now
gets its own isolated copy with correct node_id and node_exec_id values.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 00:01:05 -06:00
Nicholas Tindle
bae6be915f fix(backend): replace graph_exec_id or "" fallbacks with asserts
The empty string fallback was dead code since store_media_file() validates
graph_exec_id before these lines execute. Replace with explicit asserts
for clearer failure if assumptions are ever violated.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:39:56 -06:00
Nicholas Tindle
0b8c671a27 chore(backend): remove unused workspace REST API routes
Keep only GET /files/{file_id}/download which is used by frontend chat
to render workspace:// images. Remove 10 unused endpoints and models.py.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:31:02 -06:00
Nicholas Tindle
cb074b0076 refactor(backend): extract shared download logic into helper function
Both download_file and download_file_by_path now use _create_file_download_response()
to eliminate ~40 lines of duplicated download handling code.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:27:23 -06:00
Nicholas Tindle
f29dd34f51 fix(backend): include path filter in workspace file count
get_file_count() now accepts path parameter to match list_files() filtering,
fixing pagination totals when filtering by path prefix.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:25:06 -06:00
Nicholas Tindle
581dc337f2 chore(backend): remove unused UploadFileRequest model
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:23:11 -06:00
Nicholas Tindle
f8b041fd63 fix(backend): respect session scoping in workspace file count
get_file_count() now uses the same session scoping logic as list_files(),
ensuring consistent totals when include_all_sessions is false.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:22:27 -06:00
Nicholas Tindle
56248ae7b7 chore(backend): remove unused WORKSPACE_FILE_INFO enum value
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 23:02:37 -06:00
Nicholas Tindle
bec0157f9e Update migration to retain 'search' column
Removed the dropping of the 'search' column and its associated index from the migration script.
2026-01-27 23:01:19 -06:00
Nicholas Tindle
57f44e166a fix(backend): update HTTP block tests for execution_context
Update SendAuthenticatedWebRequestBlock to use execution_context
instead of separate graph_exec_id/user_id parameters, matching
the parent class signature.

Update test_http.py to pass execution_context to all test calls.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 22:59:52 -06:00
Nicholas Tindle
2c678f2658 refactor(backend): rename return_format options for clarity and add auto-fallback
Rename store_media_file() return_format options to make intent clear:
- "local_path" -> "for_local_processing" (ffmpeg, MoviePy, PIL)
- "data_uri" -> "for_external_api" (Replicate, OpenAI APIs)
- "workspace_ref" -> "for_block_output" (auto-adapts to context)

The "for_block_output" format now gracefully handles both contexts:
- CoPilot (has workspace): returns workspace:// reference
- Graph execution (no workspace): falls back to data URI

This prevents blocks from failing in graph execution while still
providing workspace persistence in CoPilot.

Also adds documentation to CLAUDE.md, new_blocks.md, and
block-sdk-guide.md explaining when to use each format.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 21:53:09 -06:00
Nicholas Tindle
669e33d709 chore: remove IDEAS.md from tracking
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 21:15:17 -06:00
Nicholas Tindle
953e7a5afb refactor(backend): replace return_content/save_to_workspace with return_format
Simplify store_media_file API with a single return_format parameter:

- "local_path": Return relative path (for local processing like MoviePy)
- "data_uri": Return base64 data URI (for external APIs like Replicate)
- "workspace_ref": Save to workspace and return workspace://id (for CoPilot)

This replaces the confusing combination of return_content and save_to_workspace
parameters. The old parameters are deprecated but still work via a compatibility
layer.

Updated all blocks to use the new explicit return_format parameter:
- Local processing: return_format="local_path"
- External APIs: return_format="data_uri"
- CoPilot outputs: return_format="workspace_ref"

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 21:12:29 -06:00
Nicholas Tindle
e9c55ed5a3 fix(backend): add save_to_workspace param for external API content
When blocks need to pass content to external APIs (Replicate, Discord),
they need data URIs, not workspace references. Added `save_to_workspace`
parameter to control this:

- save_to_workspace=True (default): save to workspace, return ref
- save_to_workspace=False: don't save, return data URI for API use

Updated blocks:
- AIImageEditorBlock (Flux Kontext) - input for Replicate
- AIImageCustomizerBlock - input for Replicate
- SendDiscordFileBlock - input for Discord

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 21:04:44 -06:00
Nicholas Tindle
ce3b8fa8d8 fix(backend): return data URI when reading workspace files for external APIs
When a block reads from workspace:// and needs content for an external API
(e.g., AIImageEditorBlock sending to Replicate), return data URI instead
of workspace reference.

Logic:
- workspace:// input + return_content=True → data URI (for external APIs)
- workspace:// input + return_content=False → local path (for processing)
- URL/data URI + return_content=True → save to workspace, return ref
- URL/data URI + return_content=False → local path

Fixes AIImageEditorBlock "Does not match format 'uri'" error when input
is a workspace reference.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 20:54:06 -06:00
Nicholas Tindle
ce67b7eca4 fix(backend): resolve workspace:// refs to local paths for file processing
When blocks need to process files locally (MoviePy, ffmpeg, etc.), they call
store_media_file with return_content=False expecting a local file path.

Previously, this always returned a workspace:// reference when workspace
was available, causing errors like:
  "File does not exist: /tmp/exec_file/.../workspace:/abc123"

Now the logic is:
- return_content=True: return workspace:// ref (for CoPilot output persistence)
- return_content=False: return local relative path (for file processing)

Also prevents re-saving when input is already a workspace:// reference,
avoiding unique constraint violations on the (workspaceId, path) index.

Fixes MediaDurationBlock, FileReadBlock, LoopVideoBlock, AddAudioToVideoBlock

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 20:52:35 -06:00
Nicholas Tindle
0e34c7e5c4 Merge branch 'dev' into user-workspace 2026-01-27 20:32:29 -06:00
Nicholas Tindle
759248b7fe refactor(backend/blocks): update block signatures to use ExecutionContext
Update remaining blocks to use the unified ExecutionContext parameter
instead of individual graph_exec_id, user_id, node_exec_id parameters.

Blocks updated:
- FileStoreBlock, FileReadBlock (basic.py, text.py)
- AgentFileInputBlock (io.py)
- MediaDurationBlock, LoopVideoBlock, AddAudioToVideoBlock (media.py)
- SendWebRequestBlock, SendAuthenticatedWebRequestBlock (http.py)
- ScreenshotWebPageBlock (screenshotone.py)
- ReadSpreadsheetBlock (spreadsheet.py)
- SendDiscordFileBlock (discord/bot_blocks.py)
- GmailSendBlock (google/gmail.py)
- Updated test for store_media_file

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 20:22:11 -06:00
Nicholas Tindle
ca5758cce6 feat(backend): pass workspace context through executor
Update executor to propagate workspace context:
- Pass workspace_id in execution kwargs
- Update test utilities with workspace support

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 20:16:34 -06:00
Nicholas Tindle
0db228ed43 feat(backend/blocks): store generated media to workspace
Update media-generating blocks to save outputs to workspace:
- AIImageCustomizerBlock: store customized images
- AIImageGeneratorBlock: store generated images
- AIShortformVideoCreatorBlock (3 blocks): store videos
- BannerbearTextOverlayBlock: store generated images
- AIVideoGeneratorBlock (FAL): store generated videos
- AIImageEditorBlock (Flux Kontext): store edited images
- CreateTalkingAvatarVideoBlock: store avatar videos

All blocks now return workspace:// references instead of
direct URLs, enabling persistent storage and preventing
context bloat from large base64 data URIs.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 20:16:03 -06:00
Nicholas Tindle
590f434d0a feat(backend/chat): update CoPilot execution context mapping
Update run_block.py with proper CoPilot-to-graph context mapping:
- graph_id = copilot-session-{session_id} (agent = session)
- graph_exec_id = copilot-session-{session_id} (run = session)
- graph_version = 1 (versions are 1-indexed)
- Pass workspace_id and session_id for file operations
- Each chat session is its own agent with one continuous run

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 20:15:26 -06:00
Nicholas Tindle
8f171a0537 feat(backend/chat): add CoPilot workspace tools
Add tools for CoPilot to manage workspace files:
- list_workspace_files: list files with session scoping
- read_workspace_file: read file content or metadata
- write_workspace_file: save content to workspace
- delete_workspace_file: remove files
- Session-aware operations (default to current session folder)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 20:15:14 -06:00
Nicholas Tindle
c814a43465 feat(backend/api): add workspace REST endpoints
Add API routes for workspace file management:
- GET /api/workspace - get workspace info
- POST /api/workspace/files - upload file
- GET /api/workspace/files - list files
- GET /api/workspace/files/{id} - get file info
- GET /api/workspace/files/{id}/download - download file
- DELETE /api/workspace/files/{id} - soft delete
- Stream file content when signed URLs unavailable

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 20:15:00 -06:00
Nicholas Tindle
5923041fe8 feat(backend): integrate workspace storage into store_media_file
Update store_media_file to use workspace when available:
- Save files to workspace instead of temp exec_file dir
- Return workspace:// references instead of base64 data URIs
- Handle workspace:// input references (read from workspace)
- Pass session_id to WorkspaceManager for session scoping
- Prevents context bloat (100KB file = ~133KB as base64)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 20:14:44 -06:00
Nicholas Tindle
936a2d70db feat(backend): add workspace_id and session_id to ExecutionContext
Extend ExecutionContext with workspace fields:
- workspace_id: user's workspace for file persistence
- session_id: chat session for file isolation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 20:14:26 -06:00
Nicholas Tindle
80c54b7f46 feat(backend): add workspace storage backend abstraction
Add storage layer for workspace files:
- WorkspaceStorageBackend: abstract interface
- GCSWorkspaceStorage: Google Cloud Storage implementation
- LocalWorkspaceStorage: local filesystem for self-hosted
- WorkspaceManager: high-level file operations with session scoping
- Session-scoped virtual paths: /sessions/{session_id}/{filename}
- Fallback to API proxy when GCS signed URLs unavailable

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 20:14:09 -06:00
Nicholas Tindle
28caf01ca7 feat(backend/db): add UserWorkspace and UserWorkspaceFile models
Add database schema for persistent user workspace storage:
- UserWorkspace: one-per-user workspace container
- UserWorkspaceFile: file metadata with virtual paths, checksums, and source tracking
- WorkspaceFileSource enum: UPLOAD, EXECUTION, COPILOT, IMPORT
- CRUD operations in data/workspace.py

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 20:13:49 -06:00
Nicholas Tindle
ea035224bc feat(copilot): Increase max_agent_runs and max_agent_schedules (#11865)
<!-- Clearly explain the need for these changes: -->
Config change to increase the max times an agent can run in the chat and
the max number of scheduels created by copilot in one chat

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> Increases per-chat operational limits for Copilot.
> 
> - Bumps `max_agent_runs` default from `3` to `30` in `ChatConfig`
> - Bumps `max_agent_schedules` default from `3` to `30` in `ChatConfig`
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
93cbae6d27. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
2026-01-28 01:08:02 +00:00
Bently
67405f7eb9 fix(copilot): ensure tool_call/tool_response pairs stay intact during context compaction (#11863)
## Summary

Fixes context compaction breaking tool_call/tool_response pairs, causing
API validation errors.

## Problem

When context compaction slices messages with `messages[-KEEP_RECENT:]`,
a naive slice can separate an assistant message containing `tool_calls`
from its corresponding tool response messages. This causes API
validation errors like:

```
messages.0.content.1: unexpected 'tool_use_id' found in 'tool_result' blocks: orphan_12345.
Each 'tool_result' block must have a corresponding 'tool_use' block in the previous message.
```

## Solution

Added `_ensure_tool_pairs_intact()` helper function that:
1. Detects orphan tool responses in a slice (tool messages whose
`tool_call_id` has no matching assistant message)
2. Extends the slice backwards to include the missing assistant messages
3. Falls back to removing orphan tool responses if the assistant cannot
be found (edge case)

Applied this safeguard to:
- The initial `KEEP_RECENT` slice (line ~990)
- The progressive fallback slices when still over token limit (line
~1079)

## Testing

- Syntax validated with `python -m py_compile`
- Logic reviewed for correctness

## Linear

Fixes SECRT-1839

---
*Debugged by Toran & Orion in #agpt Discord*
2026-01-28 00:21:54 +00:00
Zamil Majdy
171ff6e776 feat(backend): persist long-running tool results to survive SSE disconnects (#11856)
## Summary

Agent generation (`create_agent`, `edit_agent`) can take 1-5 minutes.
Previously, if the user closed their browser tab during this time:
1. The SSE connection would die
2. The tool execution would be cancelled via `CancelledError`
3. The result would be lost - even if the agent-generator service
completed successfully

This PR ensures long-running tool operations survive SSE disconnections.

### Changes 🏗️

**Backend:**
- **base.py**: Added `is_long_running` property to `BaseTool` for tools
to opt-in to background execution
- **create_agent.py / edit_agent.py**: Set `is_long_running = True`
- **models.py**: Added `OperationStartedResponse`,
`OperationPendingResponse`, `OperationInProgressResponse` types
- **service.py**: Modified `_yield_tool_call()` to:
  - Check if tool is `is_long_running`
  - Save "pending" message to chat history immediately
  - Spawn background task that runs independently of SSE
  - Return `operation_started` immediately (don't wait)
  - Update chat history with result when background task completes
- Track running operations for idempotency (prevents duplicate ops on
refresh)
- **db.py**: Added `update_tool_message_content()` to update pending
messages
- **model.py**: Added `invalidate_session_cache()` to clear Redis after
background completion

**Frontend:**
- **useChatMessage.ts**: Added operation message types
- **helpers.ts**: Handle `operation_started`, `operation_pending`,
`operation_in_progress` response types
- **PendingOperationWidget**: New component to display operation status
with spinner
- **ChatMessage.tsx**: Render `PendingOperationWidget` for operation
messages

### How It Works

```
User Request → Save "pending" message → Spawn background task → Return immediately
                                              ↓
                                     Task runs independently of SSE
                                              ↓
                                     On completion: Update message in chat history
                                              ↓
                                     User refreshes → Loads history → Sees result
```

### User Experience

1. User requests agent creation
2. Sees "Agent creation started. You can close this tab - check your
library in a few minutes."
3. Can close browser tab safely
4. When they return, chat shows the completed result (or error)

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] pyright passes (0 errors)
  - [x] TypeScript checks pass
  - [x] Formatters applied

### Test Plan

1. Start agent creation in copilot
2. Close browser tab immediately after seeing "operation_started" 
3. Wait 2-3 minutes
4. Reopen chat
5. Verify: Chat history shows completion message and agent appears in
library

---------

Co-authored-by: Ubbe <hi@ubbe.dev>
2026-01-28 05:09:34 +07:00
Swifty
2134d777be fix(backend): exclude disabled blocks from chat search and indexing (#11854)
## Summary

Disabled blocks (e.g., webhook blocks without `platform_base_url`
configured) were being indexed and returned in chat tool search results.
This PR ensures they are properly filtered out.

### Changes 🏗️

- **find_block.py**: Skip disabled blocks when enriching search results
- **content_handlers.py**: 
  - Skip disabled blocks during embedding indexing
- Update `get_stats()` to only count enabled blocks for accurate
coverage metrics

### Why

Blocks can be disabled for various reasons (missing OAuth config, no
platform URL for webhooks, etc.). These blocks shouldn't appear in
search results since users cannot use them.

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Verified disabled blocks are filtered from search results
  - [x] Verified disabled blocks are not indexed
  - [x] Verified stats accurately reflect enabled block count
2026-01-27 15:21:13 +00:00
Zamil Majdy
3e9d5d0d50 fix(backend): handle race condition in review processing gracefully (#11845)
## Summary
- Fixes race condition when multiple concurrent requests try to process
the same reviews (e.g., double-click, multiple browser tabs)
- Previously the second request would fail with "Reviews not found,
access denied, or not in WAITING status"
- Now handles this gracefully by treating already-processed reviews with
the same decision as success

## Changes
- Added `get_reviews_by_node_exec_ids()` function that fetches reviews
regardless of status
- Modified `process_all_reviews_for_execution()` to handle
already-processed reviews
- Updated route to use idempotent validation

## Test plan
- [x] Linter passes (`poetry run ruff check`)
- [x] Type checker passes (`poetry run pyright`)
- [x] Formatter passes (`poetry run format`)
- [ ] Manual testing: double-click approve button should not cause
errors

Fixes AUTOGPT-SERVER-7HE
2026-01-27 21:43:31 +07:00
Swifty
fac10c422b fix(backend): add SSE heartbeats to prevent tool execution timeouts (#11855)
## Summary

Long-running chat tools (like `create_agent` and `edit_agent`) were
timing out because no SSE data was sent during tool execution. GCP load
balancers and proxies have idle connection timeouts (~60 seconds), and
when the external Agent Generator service takes longer than this, the
connection would drop.

This PR adds SSE heartbeat comments during tool execution to keep
connections alive.

### Changes 🏗️

- **response_model.py**: Added `StreamHeartbeat` response type that
emits SSE comments (`: heartbeat\n\n`)
- **service.py**: Modified `_yield_tool_call()` to:
  - Run tool execution in a background asyncio task
  - Yield heartbeat events every 15 seconds while waiting
- Handle task failures with explicit error responses (no silent
failures)
  - Handle cancellation gracefully
- **create_agent.py**: Improved error messages with more context and
details
- **edit_agent.py**: Improved error messages with more context and
details

### How It Works

```
Tool Call → Background Task Started
     │
     ├── Every 15 seconds: yield `: heartbeat\n\n` (SSE comment)
     │
     └── Task Complete → yield tool result OR error response
```

SSE comments (`: heartbeat\n\n`) are:
- Ignored by SSE clients (don't trigger events)
- Keep TCP connections alive through proxies/load balancers
- Don't affect the AI SDK data protocol

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] All chat service tests pass (17 tests)
  - [x] Verified heartbeats are sent during long tool execution
  - [x] Verified errors are properly reported to frontend
2026-01-27 15:41:58 +01:00
Bently
91c7896859 fix(backend): implement context window management for long chat sessions (#11848)
## Changes 🏗️

Implements automatic context window management to prevent chat failures
when conversations exceed token limits.

  ### Problem
- **Issue**: [SECRT-1800] Long chat conversations stop working when
context grows beyond model limits (~113k tokens observed)
- **Root Cause**: Chat service sends ALL messages to LLM without
token-aware compression, eventually exceeding Claude Opus 4.5's 200k
context window

  ### Solution
  Implements a sliding window with summarization strategy:
1. Monitors token count before sending to LLM (triggers at 120k tokens)
2. Keeps last 15 messages completely intact (preserves recent
conversation flow)
  3. Summarizes older messages using gpt-4o-mini (fast & cheap)
4. Rebuilds context: `[system_prompt] + [summary] +
[recent_15_messages]`
5. Full history preserved in database (only compresses when sending to
LLM)

  ### Changes Made
- **Added** `_summarize_messages()` helper function to create concise
summaries using gpt-4o-mini
- **Modified** `_stream_chat_chunks()` to implement token counting and
conditional summarization
- **Integrated** existing `estimate_token_count()` utility for accurate
token measurement
- **Added** graceful fallback - continues with original messages if
summarization fails

  ## Motivation and Context 🎯

Without context management, users with long chat sessions (250+
messages) experience:
  - Complete chat failure when hitting 200k token limit
  - Lost conversation context
  - Poor user experience

  This fix enables:
  -  Unlimited conversation length
  -  Transparent operation (no UX changes)
  -  Preserved conversation quality (recent messages intact)
  -  Cost-efficient (~$0.0001 per summarization)

  ## Testing 🧪

  ### Expected Behavior
  - Conversations < 120k tokens: No change (normal operation)
  - Conversations > 120k tokens:
- Log message: `Context summarized: {tokens} tokens, kept last 15
messages + summary`
    - Chat continues working smoothly
    - Recent context remains intact

  ### How to Verify
  1. Start a chat session in copilot
  2. Send 250-600 messages (or 50+ with large code blocks)
  3. Check logs for "Context summarized:" message
  4. Verify chat continues working without errors
  5. Verify conversation quality remains good

  ## Checklist 

  - [x] My code follows the style guidelines of this project
  - [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
  - [x] My changes generate no new warnings
  - [x] I have tested my changes and verified they work as expected
2026-01-27 15:37:17 +01:00
Swifty
bab436231a refactor(backend): remove Langfuse tracing from chat system (#11829)
We are removing Langfuse tracing from the chat/copilot system in favor
of using OpenRouter's broadcast feature, which keeps our codebase
simpler. Langfuse prompt management is retained for fetching system
prompts.

### Changes 🏗️

**Removed Langfuse tracing:**
- Removed `@observe` decorators from all 11 chat tool files
- Removed `langfuse.openai` wrapper (now using standard `openai` client)
- Removed `start_as_current_observation` and `propagate_attributes`
context managers from `service.py`
- Removed `update_current_trace()`, `update_current_span()`,
`span.update()` calls

**Retained Langfuse prompt management:**
- `langfuse.get_prompt()` for fetching system prompts
- `_is_langfuse_configured()` check for prompt availability
- Configuration for `langfuse_prompt_name`

**Files modified:**
- `backend/api/features/chat/service.py`
- `backend/api/features/chat/tools/*.py` (11 tool files)

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Verified `poetry run format` passes
  - [x] Verified no `@observe` decorators remain in chat tools
- [x] Verified Langfuse prompt fetching is still functional (code
preserved)
2026-01-27 13:07:42 +01:00
Swifty
d5c0f5b2df refactor(backend): remove page context from chat service (#11844)
### Background
The chat service previously supported including page context (URL and
content) in user messages. This functionality is being removed.

### Changes 🏗️

- Removed page context handling from `stream_chat_completion` in the
chat service
- User messages are now passed directly without URL/content context
injection
- Removed associated logging for page context

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Verify chat functionality works without page context
  - [x] Confirm no regressions in basic chat message handling
2026-01-26 16:00:48 +00:00
Swifty
75ecc4de92 fix(backend): enforce block disabled flag on execution endpoints (#11839)
## Summary
This PR adds security checks to prevent execution of disabled blocks
across all block execution endpoints.

- Add `disabled` flag check to main web API endpoint
(`/api/blocks/{block_id}/execute`)
- Add `disabled` flag check to external API endpoint
(`/api/blocks/{block_id}/execute`)
- Add `disabled` flag check to chat tool block execution

Previously, block execution endpoints only checked if a block existed
but did not verify the `disabled` flag, allowing any authenticated user
to execute disabled blocks.

## Test plan
- [x] Verify disabled blocks return 403 Forbidden on main API endpoint
- [x] Verify disabled blocks return 403 Forbidden on external API
endpoint
- [x] Verify disabled blocks return error response in chat tool
execution
- [x] Verify enabled blocks continue to execute normally
2026-01-26 13:56:24 +00:00
Swifty
cfb7dc5aca feat(backend): Add PostHog analytics and OpenRouter tracing to chat system (#11828)
Adds analytics tracking to the chat copilot system for better
observability of user interactions and agent operations.

### Changes 🏗️

**PostHog Analytics Integration:**
- Added `posthog` dependency (v7.6.0) to track chat events
- Created new tracking module (`backend/api/features/chat/tracking.py`)
with events:
  - `chat_message_sent` - When a user sends a message
  - `chat_tool_called` - When a tool is called (includes tool name)
  - `chat_agent_run_success` - When an agent runs successfully
  - `chat_agent_scheduled` - When an agent is scheduled
  - `chat_trigger_setup` - When a trigger is set up
- Added PostHog configuration to settings:
  - `POSTHOG_API_KEY` - API key for PostHog
- `POSTHOG_HOST` - PostHog host URL (defaults to
`https://us.i.posthog.com`)

**OpenRouter Tracing:**
- Added `user` and `session_id` fields to chat completion API calls for
OpenRouter tracing
- Added `posthogDistinctId` and `posthogProperties` (with environment)
to API calls

**Files Changed:**
- `backend/api/features/chat/tracking.py` - New PostHog tracking module
- `backend/api/features/chat/service.py` - Added user message tracking
and OpenRouter tracing
- `backend/api/features/chat/tools/__init__.py` - Added tool call
tracking
- `backend/api/features/chat/tools/run_agent.py` - Added agent
run/schedule tracking
- `backend/util/settings.py` - Added PostHog configuration fields
- `pyproject.toml` - Added posthog dependency

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
  - [x] Verified code passes linting and formatting
- [x] Verified PostHog client initializes correctly when API key is
provided
- [x] Verified tracking is gracefully skipped when PostHog is not
configured

#### For configuration changes:

- [ ] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

**New environment variables (optional):**
- `POSTHOG_API_KEY` - PostHog project API key
- `POSTHOG_HOST` - PostHog host URL (optional, defaults to US cloud)
2026-01-26 12:26:15 +00:00
Zamil Majdy
9a6e17ff52 feat(backend): add external Agent Generator service integration (#11819)
## Summary
- Add support for delegating agent generation to an external
microservice when `AGENTGENERATOR_HOST` is configured
- Falls back to built-in LLM-based implementation when not configured
(default behavior)
- Add comprehensive tests for the service client and core integration
(34 tests)

## Changes
- Add `agentgenerator_host`, `agentgenerator_port`,
`agentgenerator_timeout` settings to `backend/util/settings.py`
- Add `service.py` client for external Agent Generator API endpoints:
  - `/api/decompose-description` - Break down goals into steps
  - `/api/generate-agent` - Generate agent from instructions
  - `/api/update-agent` - Generate patches to update existing agents
  - `/api/blocks` - Get available blocks
  - `/health` - Health check
- Update `core.py` to delegate to external service when configured
- Export `is_external_service_configured` and
`check_external_service_health` from the module

## Related PRs
- Infrastructure repo:
https://github.com/Significant-Gravitas/AutoGPT-cloud-infrastructure/pull/273

## Test plan
- [x] All 34 new tests pass (`poetry run pytest test/agent_generator/
-v`)
- [ ] Deploy with `AGENTGENERATOR_HOST` configured and verify external
service is used
- [ ] Verify built-in implementation still works when
`AGENTGENERATOR_HOST` is empty
2026-01-25 04:08:56 +07:00