Files
AutoGPT/docs/integrations/block-integrations/multimedia.md
Nicholas Tindle 7668c17d9c feat(platform): add User Workspace for persistent CoPilot file storage (#11867)
Implements persistent User Workspace storage for CoPilot, enabling
blocks to save and retrieve files across sessions. Files are stored in
session-scoped virtual paths (`/sessions/{session_id}/`).

Fixes SECRT-1833

### Changes 🏗️

**Database & Storage:**
- Add `UserWorkspace` and `UserWorkspaceFile` Prisma models
- Implement `WorkspaceStorageBackend` abstraction (GCS for cloud, local
filesystem for self-hosted)
- Add `workspace_id` and `session_id` fields to `ExecutionContext`

**Backend API:**
- Add REST endpoints: `GET/POST /api/workspace/files`, `GET/DELETE
/api/workspace/files/{id}`, `GET /api/workspace/files/{id}/download`
- Add CoPilot tools: `list_workspace_files`, `read_workspace_file`,
`write_workspace_file`
- Integrate workspace storage into `store_media_file()` - returns
`workspace://file-id` references

**Block Updates:**
- Refactor all file-handling blocks to use unified `ExecutionContext`
parameter
- Update media-generating blocks to persist outputs to workspace
(AIImageGenerator, AIImageCustomizer, FluxKontext, TalkingHead, FAL
video, Bannerbear, etc.)

**Frontend:**
- Render `workspace://` image references in chat via proxy endpoint
- Add "AI cannot see this image" overlay indicator

**CoPilot Context Mapping:**
- Session = Agent (graph_id) = Run (graph_exec_id)
- Files scoped to `/sessions/{session_id}/`

### Checklist 📋

#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [ ] I have tested my changes according to the test plan:
- [ ] Create CoPilot session, generate image with AIImageGeneratorBlock
  - [ ] Verify image returns `workspace://file-id` (not base64)
  - [ ] Verify image renders in chat with visibility indicator
  - [ ] Verify workspace files persist across sessions
  - [ ] Test list/read/write workspace files via CoPilot tools
  - [ ] Test local storage backend for self-hosted deployments

#### For configuration changes:
- [x] `.env.default` is updated or already compatible with my changes
- [x] `docker-compose.yml` is updated or already compatible with my
changes
- [x] I have included a list of my configuration changes in the PR
description (under **Changes**)

🤖 Generated with [Claude Code](https://claude.ai/code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Introduces a new persistent file-storage surface area (DB tables,
storage backends, download API, and chat tools) and rewires
`store_media_file()`/block execution context across many blocks, so
regressions could impact file handling, access control, or storage
costs.
> 
> **Overview**
> Adds a **persistent per-user Workspace** (new
`UserWorkspace`/`UserWorkspaceFile` models plus `WorkspaceManager` +
`WorkspaceStorageBackend` with GCS/local implementations) and wires it
into the API via a new `/api/workspace/files/{file_id}/download` route
(including header-sanitized `Content-Disposition`) and shutdown
lifecycle hooks.
> 
> Extends `ExecutionContext` to carry execution identity +
`workspace_id`/`session_id`, updates executor tooling to clone
node-specific contexts, and updates `run_block` (CoPilot) to create a
session-scoped workspace and synthetic graph/run/node IDs.
> 
> Refactors `store_media_file()` to require `execution_context` +
`return_format` and to support `workspace://` references; migrates many
media/file-handling blocks and related tests to the new API and to
persist generated media as `workspace://...` (or fall back to data URIs
outside CoPilot), and adds CoPilot chat tools for
listing/reading/writing/deleting workspace files with safeguards against
context bloat.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
6abc70f793. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Reinier van der Leer <pwuts@agpt.co>
2026-01-29 05:49:47 +00:00

4.2 KiB

Multimedia

Blocks for processing and manipulating video and audio files.

Add Audio To Video

What it is

Block to attach an audio file to a video file using moviepy.

How it works

This block combines a video file with an audio file using the moviepy library. The audio track is attached to the video, optionally with volume adjustment via the volume parameter (1.0 = original volume).

Input files can be URLs, data URIs, or local paths. The output format is automatically determined: workspace:// URLs in CoPilot, data URIs in graph executions.

Inputs

Input Description Type Required
video_in Video input (URL, data URI, or local path). str (file) Yes
audio_in Audio input (URL, data URI, or local path). str (file) Yes
volume Volume scale for the newly attached audio track (1.0 = original). float No

Outputs

Output Description Type
error Error message if the operation failed str
video_out Final video (with attached audio), as a path or data URI. str (file)

Possible use case

Add Voiceover: Combine generated voiceover audio with video content for narrated videos.

Background Music: Add music tracks to silent videos or replace existing audio.

Audio Replacement: Swap the audio track of a video for localization or accessibility.


Loop Video

What it is

Block to loop a video to a given duration or number of repeats.

How it works

This block extends a video by repeating it to reach a target duration or number of loops. Set duration to specify the total length in seconds, or use n_loops to repeat the video a specific number of times.

The looped video is seamlessly concatenated. The output format is automatically determined: workspace:// URLs in CoPilot, data URIs in graph executions.

Inputs

Input Description Type Required
video_in The input video (can be a URL, data URI, or local path). str (file) Yes
duration Target duration (in seconds) to loop the video to. If omitted, defaults to no looping. float No
n_loops Number of times to repeat the video. If omitted, defaults to 1 (no repeat). int No

Outputs

Output Description Type
error Error message if the operation failed str
video_out Looped video returned either as a relative path or a data URI. str

Possible use case

Background Videos: Loop short clips to match the duration of longer audio or content.

GIF-Like Content: Create seamlessly looping video content for social media.

Filler Content: Extend short video clips to meet minimum duration requirements.


Media Duration

What it is

Block to get the duration of a media file.

How it works

This block analyzes a media file and returns its duration in seconds. Set is_video to true for video files or false for audio files to ensure proper parsing.

The input can be a URL, data URI, or local file path. The duration is returned as a float for precise timing calculations.

Inputs

Input Description Type Required
media_in Media input (URL, data URI, or local path). str (file) Yes
is_video Whether the media is a video (True) or audio (False). bool No

Outputs

Output Description Type
error Error message if the operation failed str
duration Duration of the media file (in seconds). float

Possible use case

Video Processing Prep: Get video duration before deciding how to loop, trim, or synchronize it.

Audio Matching: Determine audio length to generate matching-length video content.

Content Validation: Verify that uploaded media meets duration requirements.