mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-02-03 19:35:15 -05:00
7668c17d9c
Implements persistent User Workspace storage for CoPilot, enabling
blocks to save and retrieve files across sessions. Files are stored in
session-scoped virtual paths (`/sessions/{session_id}/`).
Fixes SECRT-1833
### Changes 🏗️
**Database & Storage:**
- Add `UserWorkspace` and `UserWorkspaceFile` Prisma models
- Implement `WorkspaceStorageBackend` abstraction (GCS for cloud, local
filesystem for self-hosted)
- Add `workspace_id` and `session_id` fields to `ExecutionContext`
**Backend API:**
- Add REST endpoints: `GET/POST /api/workspace/files`, `GET/DELETE
/api/workspace/files/{id}`, `GET /api/workspace/files/{id}/download`
- Add CoPilot tools: `list_workspace_files`, `read_workspace_file`,
`write_workspace_file`
- Integrate workspace storage into `store_media_file()` - returns
`workspace://file-id` references
**Block Updates:**
- Refactor all file-handling blocks to use unified `ExecutionContext`
parameter
- Update media-generating blocks to persist outputs to workspace
(AIImageGenerator, AIImageCustomizer, FluxKontext, TalkingHead, FAL
video, Bannerbear, etc.)
**Frontend:**
- Render `workspace://` image references in chat via proxy endpoint
- Add "AI cannot see this image" overlay indicator
**CoPilot Context Mapping:**
- Session = Agent (graph_id) = Run (graph_exec_id)
- Files scoped to `/sessions/{session_id}/`
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [ ] I have tested my changes according to the test plan:
- [ ] Create CoPilot session, generate image with AIImageGeneratorBlock
- [ ] Verify image returns `workspace://file-id` (not base64)
- [ ] Verify image renders in chat with visibility indicator
- [ ] Verify workspace files persist across sessions
- [ ] Test list/read/write workspace files via CoPilot tools
- [ ] Test local storage backend for self-hosted deployments
#### For configuration changes:
- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)
🤖 Generated with [Claude Code](https://claude.ai/code)
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Medium Risk**
> Introduces a new persistent file-storage surface area (DB tables,
storage backends, download API, and chat tools) and rewires
`store_media_file()`/block execution context across many blocks, so
regressions could impact file handling, access control, or storage
costs.
>
> **Overview**
> Adds a **persistent per-user Workspace** (new
`UserWorkspace`/`UserWorkspaceFile` models plus `WorkspaceManager` +
`WorkspaceStorageBackend` with GCS/local implementations) and wires it
into the API via a new `/api/workspace/files/{file_id}/download` route
(including header-sanitized `Content-Disposition`) and shutdown
lifecycle hooks.
>
> Extends `ExecutionContext` to carry execution identity +
`workspace_id`/`session_id`, updates executor tooling to clone
node-specific contexts, and updates `run_block` (CoPilot) to create a
session-scoped workspace and synthetic graph/run/node IDs.
>
> Refactors `store_media_file()` to require `execution_context` +
`return_format` and to support `workspace://` references; migrates many
media/file-handling blocks and related tests to the new API and to
persist generated media as `workspace://...` (or fall back to data URIs
outside CoPilot), and adds CoPilot chat tools for
listing/reading/writing/deleting workspace files with safeguards against
context bloat.
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
6abc70f793. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
118 lines
4.2 KiB
Markdown
118 lines
4.2 KiB
Markdown
# Multimedia
|
|
<!-- MANUAL: file_description -->
|
|
Blocks for processing and manipulating video and audio files.
|
|
<!-- END MANUAL -->
|
|
|
|
## Add Audio To Video
|
|
|
|
### What it is
|
|
Block to attach an audio file to a video file using moviepy.
|
|
|
|
### How it works
|
|
<!-- MANUAL: how_it_works -->
|
|
This block combines a video file with an audio file using the moviepy library. The audio track is attached to the video, optionally with volume adjustment via the volume parameter (1.0 = original volume).
|
|
|
|
Input files can be URLs, data URIs, or local paths. The output format is automatically determined: `workspace://` URLs in CoPilot, data URIs in graph executions.
|
|
<!-- END MANUAL -->
|
|
|
|
### Inputs
|
|
|
|
| Input | Description | Type | Required |
|
|
|-------|-------------|------|----------|
|
|
| video_in | Video input (URL, data URI, or local path). | str (file) | Yes |
|
|
| audio_in | Audio input (URL, data URI, or local path). | str (file) | Yes |
|
|
| volume | Volume scale for the newly attached audio track (1.0 = original). | float | No |
|
|
|
|
### Outputs
|
|
|
|
| Output | Description | Type |
|
|
|--------|-------------|------|
|
|
| error | Error message if the operation failed | str |
|
|
| video_out | Final video (with attached audio), as a path or data URI. | str (file) |
|
|
|
|
### Possible use case
|
|
<!-- MANUAL: use_case -->
|
|
**Add Voiceover**: Combine generated voiceover audio with video content for narrated videos.
|
|
|
|
**Background Music**: Add music tracks to silent videos or replace existing audio.
|
|
|
|
**Audio Replacement**: Swap the audio track of a video for localization or accessibility.
|
|
<!-- END MANUAL -->
|
|
|
|
---
|
|
|
|
## Loop Video
|
|
|
|
### What it is
|
|
Block to loop a video to a given duration or number of repeats.
|
|
|
|
### How it works
|
|
<!-- MANUAL: how_it_works -->
|
|
This block extends a video by repeating it to reach a target duration or number of loops. Set duration to specify the total length in seconds, or use n_loops to repeat the video a specific number of times.
|
|
|
|
The looped video is seamlessly concatenated. The output format is automatically determined: `workspace://` URLs in CoPilot, data URIs in graph executions.
|
|
<!-- END MANUAL -->
|
|
|
|
### Inputs
|
|
|
|
| Input | Description | Type | Required |
|
|
|-------|-------------|------|----------|
|
|
| video_in | The input video (can be a URL, data URI, or local path). | str (file) | Yes |
|
|
| duration | Target duration (in seconds) to loop the video to. If omitted, defaults to no looping. | float | No |
|
|
| n_loops | Number of times to repeat the video. If omitted, defaults to 1 (no repeat). | int | No |
|
|
|
|
### Outputs
|
|
|
|
| Output | Description | Type |
|
|
|--------|-------------|------|
|
|
| error | Error message if the operation failed | str |
|
|
| video_out | Looped video returned either as a relative path or a data URI. | str |
|
|
|
|
### Possible use case
|
|
<!-- MANUAL: use_case -->
|
|
**Background Videos**: Loop short clips to match the duration of longer audio or content.
|
|
|
|
**GIF-Like Content**: Create seamlessly looping video content for social media.
|
|
|
|
**Filler Content**: Extend short video clips to meet minimum duration requirements.
|
|
<!-- END MANUAL -->
|
|
|
|
---
|
|
|
|
## Media Duration
|
|
|
|
### What it is
|
|
Block to get the duration of a media file.
|
|
|
|
### How it works
|
|
<!-- MANUAL: how_it_works -->
|
|
This block analyzes a media file and returns its duration in seconds. Set is_video to true for video files or false for audio files to ensure proper parsing.
|
|
|
|
The input can be a URL, data URI, or local file path. The duration is returned as a float for precise timing calculations.
|
|
<!-- END MANUAL -->
|
|
|
|
### Inputs
|
|
|
|
| Input | Description | Type | Required |
|
|
|-------|-------------|------|----------|
|
|
| media_in | Media input (URL, data URI, or local path). | str (file) | Yes |
|
|
| is_video | Whether the media is a video (True) or audio (False). | bool | No |
|
|
|
|
### Outputs
|
|
|
|
| Output | Description | Type |
|
|
|--------|-------------|------|
|
|
| error | Error message if the operation failed | str |
|
|
| duration | Duration of the media file (in seconds). | float |
|
|
|
|
### Possible use case
|
|
<!-- MANUAL: use_case -->
|
|
**Video Processing Prep**: Get video duration before deciding how to loop, trim, or synchronize it.
|
|
|
|
**Audio Matching**: Determine audio length to generate matching-length video content.
|
|
|
|
**Content Validation**: Verify that uploaded media meets duration requirements.
|
|
<!-- END MANUAL -->
|
|
|
|
---
|