mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-04-30 03:00:41 -04:00
refactor(backend): rename return_format options for clarity and add auto-fallback
Rename store_media_file() return_format options to make intent clear: - "local_path" -> "for_local_processing" (ffmpeg, MoviePy, PIL) - "data_uri" -> "for_external_api" (Replicate, OpenAI APIs) - "workspace_ref" -> "for_block_output" (auto-adapts to context) The "for_block_output" format now gracefully handles both contexts: - CoPilot (has workspace): returns workspace:// reference - Graph execution (no workspace): falls back to data URI This prevents blocks from failing in graph execution while still providing workspace persistence in CoPilot. Also adds documentation to CLAUDE.md, new_blocks.md, and block-sdk-guide.md explaining when to use each format. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -277,6 +277,50 @@ async def run(
|
||||
token = credentials.api_key.get_secret_value()
|
||||
```
|
||||
|
||||
### Handling Files
|
||||
|
||||
When your block works with files (images, videos, documents), use `store_media_file()`:
|
||||
|
||||
```python
|
||||
from backend.data.execution import ExecutionContext
|
||||
from backend.util.file import store_media_file
|
||||
from backend.util.type import MediaFileType
|
||||
|
||||
async def run(
|
||||
self,
|
||||
input_data: Input,
|
||||
*,
|
||||
execution_context: ExecutionContext,
|
||||
**kwargs,
|
||||
):
|
||||
# PROCESSING: Need local file path for tools like ffmpeg, MoviePy, PIL
|
||||
local_path = await store_media_file(
|
||||
file=input_data.video,
|
||||
execution_context=execution_context,
|
||||
return_format="for_local_processing",
|
||||
)
|
||||
|
||||
# EXTERNAL API: Need base64 content for APIs like Replicate, OpenAI
|
||||
image_b64 = await store_media_file(
|
||||
file=input_data.image,
|
||||
execution_context=execution_context,
|
||||
return_format="for_external_api",
|
||||
)
|
||||
|
||||
# OUTPUT: Return to user/next block (auto-adapts to context)
|
||||
result = await store_media_file(
|
||||
file=generated_url,
|
||||
execution_context=execution_context,
|
||||
return_format="for_block_output", # workspace:// in CoPilot, data URI in graphs
|
||||
)
|
||||
yield "image_url", result
|
||||
```
|
||||
|
||||
**Return format options:**
|
||||
- `"for_local_processing"` - Local file path for processing tools
|
||||
- `"for_external_api"` - Data URI for external APIs needing base64
|
||||
- `"for_block_output"` - **Always use for outputs** - automatically picks best format
|
||||
|
||||
## Testing Your Block
|
||||
|
||||
```bash
|
||||
|
||||
@@ -111,6 +111,71 @@ Follow these steps to create and test a new block:
|
||||
- `graph_exec_id`: The ID of the execution of the agent. This changes every time the agent has a new "run"
|
||||
- `node_exec_id`: The ID of the execution of the node. This changes every time the node is executed
|
||||
- `node_id`: The ID of the node that is being executed. It changes every version of the graph, but not every time the node is executed.
|
||||
- `execution_context`: An `ExecutionContext` object containing user_id, graph_exec_id, workspace_id, and session_id. Required for file handling.
|
||||
|
||||
### Handling Files in Blocks
|
||||
|
||||
When your block needs to work with files (images, videos, documents), use `store_media_file()` from `backend.util.file`. This function handles downloading, validation, virus scanning, and storage.
|
||||
|
||||
**Import:**
|
||||
```python
|
||||
from backend.data.execution import ExecutionContext
|
||||
from backend.util.file import store_media_file
|
||||
from backend.util.type import MediaFileType
|
||||
```
|
||||
|
||||
**The `return_format` parameter determines what you get back:**
|
||||
|
||||
| Format | Use When | Returns |
|
||||
|--------|----------|---------|
|
||||
| `"for_local_processing"` | Processing with local tools (ffmpeg, MoviePy, PIL) | Local file path (e.g., `"image.png"`) |
|
||||
| `"for_external_api"` | Sending content to external APIs (Replicate, OpenAI) | Data URI (e.g., `"data:image/png;base64,..."`) |
|
||||
| `"for_block_output"` | Returning output from your block | Smart: `workspace://` in CoPilot, data URI in graphs |
|
||||
|
||||
**Examples:**
|
||||
|
||||
```python
|
||||
async def run(
|
||||
self,
|
||||
input_data: Input,
|
||||
*,
|
||||
execution_context: ExecutionContext,
|
||||
**kwargs,
|
||||
) -> BlockOutput:
|
||||
# PROCESSING: Need to work with file locally (ffmpeg, MoviePy, PIL)
|
||||
local_path = await store_media_file(
|
||||
file=input_data.video,
|
||||
execution_context=execution_context,
|
||||
return_format="for_local_processing",
|
||||
)
|
||||
# local_path = "video.mp4" - use with Path, ffmpeg, subprocess, etc.
|
||||
full_path = get_exec_file_path(execution_context.graph_exec_id, local_path)
|
||||
|
||||
# EXTERNAL API: Need to send content to an API like Replicate
|
||||
image_b64 = await store_media_file(
|
||||
file=input_data.image,
|
||||
execution_context=execution_context,
|
||||
return_format="for_external_api",
|
||||
)
|
||||
# image_b64 = "data:image/png;base64,iVBORw0..." - send to external API
|
||||
|
||||
# OUTPUT: Returning result from block to user/next block
|
||||
result_url = await store_media_file(
|
||||
file=generated_image_url,
|
||||
execution_context=execution_context,
|
||||
return_format="for_block_output",
|
||||
)
|
||||
yield "image_url", result_url
|
||||
# In CoPilot: result_url = "workspace://abc123" (persistent, context-efficient)
|
||||
# In graphs: result_url = "data:image/png;base64,..." (for next block/display)
|
||||
```
|
||||
|
||||
**Key points:**
|
||||
|
||||
- `for_block_output` is the **only** format that auto-adapts to execution context
|
||||
- Always use `for_block_output` for block outputs unless you have a specific reason not to
|
||||
- Never manually check for `workspace_id` - let `for_block_output` handle the logic
|
||||
- The function handles URLs, data URIs, `workspace://` references, and local paths as input
|
||||
|
||||
### Field Types
|
||||
|
||||
|
||||
Reference in New Issue
Block a user