github/InvokeAI
1
1
Fork 0
mirror of https://github.com/invoke-ai/InvokeAI.git synced 2026-04-23 03:00:31 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat(recall): support model-free reference images in recall API

Browse Source 
The recall parameters API previously exposed only `loras`, `control_layers`,
and `ip_adapters`. This meant reference images used by architectures that
feed images directly into the main model — FLUX.2 Klein, FLUX Kontext, and
Qwen Image Edit — could not be sent through the recall endpoint at all:
they have no adapter model to resolve, so they could not ride in the
`ip_adapters` list.

This change adds a new `reference_images` field on RecallParameter that
carries only an `image_name`. The backend validates the file exists in
outputs/images and forwards the resolved metadata (width/height) in the
broadcast event. The frontend's recall handler picks the right config type
(`flux2_reference_image` / `flux_kontext_reference_image` / `ip_adapter`
fallback) via getDefaultRefImageConfig() based on the currently-selected
main model, matching the behavior of a manual drag-and-drop, and dispatches
`refImagesRecalled` with replace:false so these append rather than clobber
any adapters already applied in the same event.

Also consolidates the two existing docs under docs/contributing/RECALL_PARAMETERS/
(RECALL_PARAMETERS_API.md and RECALL_API_LORAS_CONTROLNETS_IMAGES.md) into
a single RECALL_PARAMETERS_API.md that documents the full request schema
including the new field.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Lincoln Stein
2026-04-11 22:08:53 -04:00
parent a3507121da
commit 4da9fab888
5 changed files with 660 additions and 461 deletions
Download Patch File Download Diff File Expand all files Collapse all files

375
docs/contributing/RECALL_PARAMETERS/RECALL_API_LORAS_CONTROLNETS_IMAGES.md
View File

@@ -1,375 +0,0 @@
# Recall Parameters API - LoRAs, ControlNets, and IP Adapters with Images
## Overview
The Recall Parameters API supports recalling LoRAs, ControlNets (including T2I Adapters and Control LoRAs), and IP Adapters along with their associated weights and settings. Control Layers and IP Adapters can now include image references from the `INVOKEAI_ROOT/outputs/images` directory for fully functional control and image prompt functionality.
## Key Features
✅ **LoRAs**: Fully functional - adds to UI, queries model configs, applies weights
✅ **Control Layers**: Full support with optional images from outputs/images
✅ **IP Adapters**: Full support with optional reference images from outputs/images
✅ **Model Name Resolution**: Automatic lookup from human-readable names to internal keys
✅ **Image Validation**: Backend validates that image files exist before sending
## Endpoints
### POST `/api/v1/recall/{queue_id}`
Updates recallable parameters for the frontend, including LoRAs, control adapters, and IP adapters with optional images.
**Path Parameters:**
- `queue_id` (string): The queue ID to associate parameters with (typically "default")
**Request Body:**
All fields are optional. Include only the parameters you want to update.
```typescript
{
// Standard parameters
positive_prompt?: string;
negative_prompt?: string;
model?: string; // Model name or key
steps?: number;
cfg_scale?: number;
width?: number;
height?: number;
seed?: number;
// ... other standard parameters
// LoRAs
loras?: Array<{
model_name: string; // LoRA model name
weight?: number; // Default: 0.75, Range: -10 to 10
is_enabled?: boolean; // Default: true
}>;
// Control Layers (ControlNet, T2I Adapter, Control LoRA)
control_layers?: Array<{
model_name: string; // Control adapter model name
image_name?: string; // Optional image filename from outputs/images
weight?: number; // Default: 1.0, Range: -1 to 2
begin_step_percent?: number; // Default: 0.0, Range: 0 to 1
end_step_percent?: number; // Default: 1.0, Range: 0 to 1
control_mode?: "balanced" | "more_prompt" | "more_control"; // ControlNet only
}>;
// IP Adapters
ip_adapters?: Array<{
model_name: string; // IP Adapter model name
image_name?: string; // Optional reference image filename from outputs/images
weight?: number; // Default: 1.0, Range: -1 to 2
begin_step_percent?: number; // Default: 0.0, Range: 0 to 1
end_step_percent?: number; // Default: 1.0, Range: 0 to 1
method?: "full" | "style" | "composition"; // Default: "full"
influence?: "Lowest" | "Low" | "Medium" | "High" | "Highest"; // Flux Redux only; default: "highest"
}>;
}
```
## Model Name Resolution
The backend automatically resolves model names to their internal keys:
1. **Main Models**: Resolved from the name to the model key
2. **LoRAs**: Searched in the LoRA model database
3. **Control Adapters**: Tried in order - ControlNet → T2I Adapter → Control LoRA
4. **IP Adapters**: Searched in the IP Adapter model database
Models that cannot be resolved are skipped with a warning in the logs.
## Image File Handling
### Image Path Resolution
When you specify an `image_name`, the backend:
1. Constructs the full path: `{INVOKEAI_ROOT}/outputs/images/{image_name}`
2. Validates that the file exists
3. Includes the image reference in the event sent to the frontend
4. Logs whether the image was found or not
### Image Naming
Images should be referenced by their filename as it appears in the outputs/images directory:
- ✅ Correct: `"image_name": "example.png"`
- ✅ Correct: `"image_name": "my_control_image_20240110.jpg"`
- ❌ Incorrect: `"image_name": "outputs/images/example.png"` (use relative filename only)
- ❌ Incorrect: `"image_name": "/full/path/to/example.png"` (use relative filename only)
## Frontend Behavior
### LoRAs
- **Fully Supported**: LoRAs are immediately added to the LoRA list in the UI
- Existing LoRAs are cleared before adding new ones
- Each LoRA's model config is fetched and applied with the specified weight
- LoRAs appear in the LoRA selector panel
### Control Layers with Images
- **Fully Supported**: Control layers now support images from outputs/images
- Configuration includes model, weights, step percentages, and image reference
- Image availability is logged in frontend console
- Images can be used to create actual control layers through the UI
### IP Adapters with Images
- **Fully Supported**: IP Adapters now support reference images from outputs/images
- Configuration includes model, weights, step percentages, method, and image reference
- Image availability is logged in frontend console
- Images can be used to create actual reference image layers through the UI
## Examples
### 1. Add LoRAs Only
```bash
curl -X POST http://localhost:9090/api/v1/recall/default \
-H "Content-Type: application/json" \
-d '{
"loras": [
{
"model_name": "add-detail-xl",
"weight": 0.8,
"is_enabled": true
},
{
"model_name": "sd_xl_offset_example-lora_1.0",
"weight": 0.5,
"is_enabled": true
}
]
}'
```
### 2. Configure Control Layers with Image
Replace `my_control_image.png` with an actual image filename from your outputs/images directory.
```bash
curl -X POST http://localhost:9090/api/v1/recall/default \
-H "Content-Type: application/json" \
-d '{
"control_layers": [
{
"model_name": "controlnet-canny-sdxl-1.0",
"image_name": "my_control_image.png",
"weight": 0.75,
"begin_step_percent": 0.0,
"end_step_percent": 0.8,
"control_mode": "balanced"
}
]
}'
```
### 3. Configure IP Adapters with Reference Image
Replace `reference_face.png` with an actual image filename from your outputs/images directory.
```bash
curl -X POST http://localhost:9090/api/v1/recall/default \
-H "Content-Type: application/json" \
-d '{
"ip_adapters": [
{
"model_name": "ip-adapter-plus-face_sd15",
"image_name": "reference_face.png",
"weight": 0.7,
"begin_step_percent": 0.0,
"end_step_percent": 1.0,
"method": "composition"
}
]
}'
```
### 4. Complete Configuration with All Features
```bash
curl -X POST http://localhost:9090/api/v1/recall/default \
-H "Content-Type: application/json" \
-d '{
"positive_prompt": "masterpiece, detailed photo with specific style",
"negative_prompt": "blurry, low quality",
"model": "FLUX Schnell",
"steps": 25,
"cfg_scale": 8.0,
"width": 1024,
"height": 768,
"seed": 42,
"loras": [
{
"model_name": "add-detail-xl",
"weight": 0.6,
"is_enabled": true
}
],
"control_layers": [
{
"model_name": "controlnet-depth-sdxl-1.0",
"image_name": "depth_map.png",
"weight": 1.0,
"begin_step_percent": 0.0,
"end_step_percent": 0.7
}
],
"ip_adapters": [
{
"model_name": "ip-adapter-plus-face_sd15",
"image_name": "style_reference.png",
"weight": 0.5,
"begin_step_percent": 0.0,
"end_step_percent": 1.0,
"method": "style"
}
]
}'
```
## Response Format
```json
{
"status": "success",
"queue_id": "default",
"updated_count": 15,
"parameters": {
"positive_prompt": "...",
"steps": 25,
"loras": [
{
"model_key": "abc123...",
"weight": 0.6,
"is_enabled": true
}
],
"control_layers": [
{
"model_key": "controlnet-xyz...",
"weight": 1.0,
"image": {
"image_name": "depth_map.png"
}
}
],
"ip_adapters": [
{
"model_key": "ip-adapter-xyz...",
"weight": 0.5,
"image": {
"image_name": "style_reference.png"
}
}
]
}
}
```
## WebSocket Events
When parameters are updated, a `recall_parameters_updated` event is emitted via WebSocket to the queue room. The frontend automatically:
1. Applies standard parameters (prompts, steps, dimensions, etc.)
2. Loads and adds LoRAs to the LoRA list
3. Logs control layer and IP adapter configurations with image information
4. Makes image references available for manual canvas/reference image creation
## Logging
### Backend Logs
Backend logs show:
- Model name → key resolution (success/failure)
- Image file validation (found/not found)
- Parameter storage confirmation
- Event emission status
Example log messages:
```
INFO: Resolved ControlNet model name 'controlnet-canny-sdxl-1.0' to key 'controlnet-xyz...'
INFO: Found image file: depth_map.png
INFO: Updated 12 recall parameters for queue default
INFO: Resolved 1 LoRA(s)
INFO: Resolved 1 control layer(s)
INFO: Resolved 1 IP adapter(s)
```
### Frontend Logs
Frontend logs (check browser console):
- Set `localStorage.ROARR_FILTER = 'debug'` to see all debug messages
- Look for messages from the `events` namespace
- LoRA loading, model resolution, and parameter application are logged
Example log messages:
```
INFO: Applied 5 recall parameters to store
INFO: Received 1 control layer(s) with image support
INFO: Control layer 1: controlnet-xyz... (weight: 0.75, image: depth_map.png)
DEBUG: Control layer 1 image available at: outputs/images/depth_map.png
INFO: Received 1 IP adapter(s) with image support
INFO: IP adapter 1: ip-adapter-xyz... (weight: 0.7, image: style_reference.png)
DEBUG: IP adapter 1 image available at: outputs/images/style_reference.png
```
## Limitations
1. **Canvas Integration**: Control layers and IP adapters with images are currently logged but not automatically added to canvas layers
- Users can view the configuration and manually create canvas layers with the provided images
- Future enhancement: Auto-create canvas layers with stored images
2. **Model Availability**: Models must be installed in InvokeAI before they can be recalled
3. **Image Availability**: Images must exist in the outputs/images directory
- Missing images are logged as warnings but don't fail the request
- Other parameters are still applied even if images are missing
4. **Image URLs**: Only local filenames from outputs/images are supported
- Remote image URLs are not currently supported
## Testing
Use the provided test script:
```bash
./test_recall_loras_controlnets.sh
```
This will test:
- LoRA addition with multiple models
- Control layer configuration with image references
- IP adapter configuration with image references
- Combined parameter updates with all features
Note: Update the image names in the test script to match actual images in your outputs/images directory.
## Troubleshooting
### Images Not Found
If you see "Image file not found" in the logs:
1. Verify the image filename matches exactly (case-sensitive)
2. Ensure the image is in `{INVOKEAI_ROOT}/outputs/images/`
3. Check that the filename doesn't include the `outputs/images/` prefix
### Models Not Found
If you see "Could not find model" messages:
1. Verify the model name matches exactly (case-sensitive)
2. Ensure the model is installed in InvokeAI
3. Check the model name using the models browser in the UI
### Event Not Received
If the frontend doesn't receive the event:
1. Check browser console for connection errors
2. Verify the queue_id matches the frontend's queue (usually "default")
3. Check backend logs for event emission errors
## Future Enhancements
Potential improvements:
1. Auto-create canvas layers with provided control layer images
2. Auto-create reference image layers with provided IP adapter images
3. Support for image URLs
4. Batch operations for multiple queue IDs
5. Image upload capability (accept base64 or file upload)

475
docs/contributing/RECALL_PARAMETERS/RECALL_PARAMETERS_API.md
View File

@@ -2,29 +2,54 @@
## Overview
A new REST API endpoint has been added to the InvokeAI backend that allows programmatic updates to recallable parameters from another process. This enables external applications or scripts to modify frontend parameters like prompts, models, and step counts via HTTP requests.
The Recall Parameters API is a REST endpoint on the InvokeAI backend that
lets external processes set recallable generation parameters on the
frontend. Supported parameters include:
When parameters are updated via the API, the backend automatically broadcasts a WebSocket event to all connected frontend clients subscribed to that queue, causing them to update immediately.
- Core text and numeric parameters (prompts, model, steps, CFG, dimensions, seed, ...)
- LoRAs
- Control Layers (ControlNet, T2I Adapter, Control LoRA) with optional control images
- IP Adapters and FLUX Redux reference images with optional images
- Model-free reference images (FLUX.2 Klein, FLUX Kontext, Qwen Image Edit)
When parameters are updated via the API, the backend stores them in client
state persistence for the target queue and broadcasts a `recall_parameters_updated`
WebSocket event. Any frontend client subscribed to that queue applies the
new values immediately — no manual reload required.
Typical use cases:
- An external image browser that wants to "recall" or "remix" the
generation parameters saved into a PNG's metadata.
- A script that pre-populates parameters before the user runs generation.
- Automated testing or batch workflows that want to reuse existing model
and adapter configurations.
## How It Works
1. **API Request**: External application sends a POST request with parameters to update
2. **Storage**: Parameters are stored in client state persistence, associated with a queue ID
3. **Broadcast**: A WebSocket event (`recall_parameters_updated`) is emitted to all frontend clients listening to that queue
4. **Frontend Update**: Connected frontend clients receive the event and can process the updated parameters
5. **Immediate Display**: The frontend UI updates automatically with the new values
This means if you have the InvokeAI frontend open in a browser, updating parameters via the API will instantly reflect on the screen without any manual action needed.
1. **API request** — your client POSTs a JSON body of parameters to
`/api/v1/recall/{queue_id}`.
2. **Storage** — non-null parameters are stored under
`recall_*` keys in the client state persistence service, scoped to the
given `queue_id`.
3. **Resolution** — models are resolved from human-readable names to the
internal model keys used by the frontend, and image filenames are
validated against `{INVOKEAI_ROOT}/outputs/images`.
4. **Broadcast** — a `recall_parameters_updated` event is emitted on the
websocket room for `queue_id`.
5. **Frontend update** — any connected client subscribed to that queue
applies the update to its Redux store, so UI fields, LoRAs, control
layers, IP adapters, and reference images all populate immediately.
## Endpoint
**Base URL**: `http://localhost:9090/api/v1/recall/{queue_id}`
**Base URL:** `http://localhost:9090/api/v1/recall/{queue_id}`
## POST - Update Recall Parameters
The queue id is usually `default`.
Updates recallable parameters for a given queue ID.
### POST — Update Recall Parameters
### Request
Updates recallable parameters for the given `queue_id`.
```http
POST /api/v1/recall/{queue_id}
@@ -42,11 +67,25 @@ Content-Type: application/json
}
```
The queue id is usually "default".
All parameters are optional — only send the fields you want to update.
### Parameters
### GET — Retrieve Recall Parameters
All parameters are optional. Only provide the parameters you want to update:
```http
GET /api/v1/recall/{queue_id}
```
```json
{
"status": "success",
"queue_id": "queue_123",
"note": "Use the frontend to access stored recall parameters, or set specific parameters using POST"
}
```
## Request Schema
### Core parameters
| Parameter | Type | Description |
|-----------|------|-------------|
@@ -65,60 +104,130 @@ All parameters are optional. Only provide the parameters you want to update:
| `width` | integer | Image width in pixels (≥64) |
| `height` | integer | Image height in pixels (≥64) |
| `seed` | integer | Random seed (≥0) |
| `denoise_strength` | number | Denoising strength (0-1) |
| `refiner_denoise_start` | number | Refiner denoising start (0-1) |
| `denoise_strength` | number | Denoising strength (0–1) |
| `refiner_denoise_start` | number | Refiner denoising start (0–1) |
| `clip_skip` | integer | CLIP skip layers (≥0) |
| `seamless_x` | boolean | Enable seamless X tiling |
| `seamless_y` | boolean | Enable seamless Y tiling |
| `refiner_positive_aesthetic_score` | number | Refiner positive aesthetic score |
| `refiner_negative_aesthetic_score` | number | Refiner negative aesthetic score |
### Response
### Collection parameters
```json
```typescript
{
"status": "success",
"queue_id": "queue_123",
"updated_count": 7,
"parameters": {
"positive_prompt": "a beautiful landscape",
"negative_prompt": "blurry, low quality",
"model": "sd-1.5",
"steps": 20,
"cfg_scale": 7.5,
"width": 512,
"height": 512,
"seed": 12345
}
// LoRAs
loras?: Array<{
model_name: string; // LoRA model name
weight?: number; // Default: 0.75, Range: -10 to 10
is_enabled?: boolean; // Default: true
}>;
// Control Layers (ControlNet, T2I Adapter, Control LoRA)
control_layers?: Array<{
model_name: string; // Control adapter model name
image_name?: string; // Optional image filename from outputs/images
weight?: number; // Default: 1.0, Range: -1 to 2
begin_step_percent?: number; // Default: 0.0, Range: 0 to 1
end_step_percent?: number; // Default: 1.0, Range: 0 to 1
control_mode?: "balanced" | "more_prompt" | "more_control"; // ControlNet only
}>;
// IP Adapters (includes FLUX Redux)
ip_adapters?: Array<{
model_name: string; // IP Adapter / FLUX Redux model name
image_name?: string; // Optional reference image filename from outputs/images
weight?: number; // Default: 1.0, Range: -1 to 2
begin_step_percent?: number; // Default: 0.0, Range: 0 to 1
end_step_percent?: number; // Default: 1.0, Range: 0 to 1
method?: "full" | "style" | "composition"; // Default: "full"
image_influence?: "lowest" | "low" | "medium" | "high" | "highest"; // FLUX Redux only
}>;
// Model-free reference images (FLUX.2 Klein, FLUX Kontext, Qwen Image Edit)
reference_images?: Array<{
image_name: string; // Reference image filename from outputs/images
}>;
}
```
## GET - Retrieve Recall Parameters
## Model Name Resolution
Retrieves metadata about stored recall parameters.
The backend resolves model names to their internal keys:
### Request
1. **Main models** — resolved from the name to the model key.
2. **LoRAs** — searched in the LoRA model database.
3. **Control adapters** — tried in order: ControlNet → T2I Adapter → Control LoRA.
4. **IP Adapters** — searched in the IP Adapter database; falls back to FLUX Redux.
```http
GET /api/v1/recall/{queue_id}
```
Models that cannot be resolved are skipped with a warning in the logs —
the rest of the parameters are still applied.
### Response
## Image File Handling
```json
{
"status": "success",
"queue_id": "queue_123",
"note": "Use the frontend to access stored recall parameters, or set specific parameters using POST"
}
```
When an `image_name` is supplied, the backend:
1. Resolves `{INVOKEAI_ROOT}/outputs/images/{image_name}` via the image
files service (which also validates the path).
2. Opens the image to extract width/height.
3. Includes the image metadata in the event sent to the frontend.
4. Logs whether the image was found.
Images must be referenced by their filename as it appears in the
outputs/images directory:
- ✅ `"image_name": "example.png"`
- ✅ `"image_name": "my_control_image_20240110.jpg"`
- ❌ `"image_name": "outputs/images/example.png"` (no prefix)
- ❌ `"image_name": "/full/path/to/example.png"` (no absolute paths)
Missing images are logged as warnings but **do not** fail the request —
remaining parameters are still applied.
## Feature Details
### LoRAs
- Existing LoRAs are cleared before new ones are added.
- Each LoRA's model config is fetched and applied with the specified weight.
- LoRAs appear in the LoRA selector panel.
### Control Layers
- Fully supported with optional images from `outputs/images`.
- Configuration includes model, weights, step percentages, control mode,
and an image reference.
- Image availability is logged in the frontend console.
### IP Adapters / FLUX Redux
- Reference images loaded from `outputs/images` are validated and passed
through.
- Configuration includes model, weights, step percentages, method, and an
image reference.
- FLUX Redux uses `image_influence` instead of a numeric weight.
### Model-free reference images
Used by architectures that consume a reference image directly, with no
separate adapter model:
- **FLUX.2 Klein** — built-in reference image support.
- **FLUX Kontext** — reference image associated with the main model.
- **Qwen Image Edit** — reference image associated with the main model.
Because there is no adapter model to resolve, these entries carry only
`image_name`. When the frontend receives them, it picks the appropriate
config flavor (`flux2_reference_image`, `flux_kontext_reference_image`,
or `qwen_image_reference_image`) based on the currently-selected main
model, matching the behavior of a manual drag-and-drop.
## Usage Examples
### Using cURL
### cURL
```bash
# Update prompts and model
# Core parameters
curl -X POST http://localhost:9090/api/v1/recall/default \
-H "Content-Type: application/json" \
-d '{
@@ -128,81 +237,275 @@ curl -X POST http://localhost:9090/api/v1/recall/default \
"steps": 30
}'
# Update just the seed
# Just the seed
curl -X POST http://localhost:9090/api/v1/recall/default \
-H "Content-Type: application/json" \
-d '{"seed": 99999}'
```
### Using Python
### LoRAs only
```bash
curl -X POST http://localhost:9090/api/v1/recall/default \
-H "Content-Type: application/json" \
-d '{
"loras": [
{"model_name": "add-detail-xl", "weight": 0.8, "is_enabled": true},
{"model_name": "sd_xl_offset_example-lora_1.0", "weight": 0.5}
]
}'
```
### Control layers with an image
```bash
curl -X POST http://localhost:9090/api/v1/recall/default \
-H "Content-Type: application/json" \
-d '{
"control_layers": [
{
"model_name": "controlnet-canny-sdxl-1.0",
"image_name": "my_control_image.png",
"weight": 0.75,
"begin_step_percent": 0.0,
"end_step_percent": 0.8,
"control_mode": "balanced"
}
]
}'
```
### IP adapters with a reference image
```bash
curl -X POST http://localhost:9090/api/v1/recall/default \
-H "Content-Type: application/json" \
-d '{
"ip_adapters": [
{
"model_name": "ip-adapter-plus-face_sd15",
"image_name": "reference_face.png",
"weight": 0.7,
"method": "composition"
}
]
}'
```
### Model-free reference images (FLUX.2 Klein / FLUX Kontext / Qwen Image Edit)
```bash
curl -X POST http://localhost:9090/api/v1/recall/default \
-H "Content-Type: application/json" \
-d '{
"model": "FLUX.2 Klein",
"reference_images": [
{"image_name": "style_reference.png"}
]
}'
```
### Complete configuration
```bash
curl -X POST http://localhost:9090/api/v1/recall/default \
-H "Content-Type: application/json" \
-d '{
"positive_prompt": "masterpiece, detailed photo with specific style",
"negative_prompt": "blurry, low quality",
"model": "FLUX Schnell",
"steps": 25,
"cfg_scale": 8.0,
"width": 1024,
"height": 768,
"seed": 42,
"loras": [
{"model_name": "add-detail-xl", "weight": 0.6}
],
"control_layers": [
{
"model_name": "controlnet-depth-sdxl-1.0",
"image_name": "depth_map.png",
"weight": 1.0,
"end_step_percent": 0.7
}
],
"ip_adapters": [
{
"model_name": "ip-adapter-plus-face_sd15",
"image_name": "style_reference.png",
"weight": 0.5,
"method": "style"
}
]
}'
```
### Python
```python
import requests
import json
# Configuration
API_URL = "http://localhost:9090/api/v1/recall/default"
# Update multiple parameters
params = {
"positive_prompt": "a serene forest",
"negative_prompt": "people, buildings",
"steps": 25,
"cfg_scale": 7.0,
"seed": 42
"seed": 42,
}
response = requests.post(API_URL, json=params)
result = response.json()
print(f"Status: {result['status']}")
print(f"Updated {result['updated_count']} parameters")
print(json.dumps(result['parameters'], indent=2))
```
### Using Node.js/JavaScript
### JavaScript
```javascript
const API_URL = 'http://localhost:9090/api/v1/recall/default';
const params = {
positive_prompt: 'a beautiful sunset',
negative_prompt: 'blurry',
steps: 20,
width: 768,
height: 768,
seed: 12345
};
fetch(API_URL, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(params)
body: JSON.stringify({
positive_prompt: 'a beautiful sunset',
steps: 20,
width: 768,
height: 768,
seed: 12345,
}),
})
.then(res => res.json())
.then(data => console.log(data));
.then((res) => res.json())
.then((data) => console.log(data));
```
## Response Format
```json
{
"status": "success",
"queue_id": "default",
"updated_count": 15,
"parameters": {
"positive_prompt": "...",
"steps": 25,
"loras": [
{"model_key": "abc123...", "weight": 0.6, "is_enabled": true}
],
"control_layers": [
{
"model_key": "controlnet-xyz...",
"weight": 1.0,
"image": {"image_name": "depth_map.png", "width": 1024, "height": 768}
}
],
"ip_adapters": [
{
"model_key": "ip-adapter-xyz...",
"weight": 0.5,
"image": {"image_name": "style_reference.png", "width": 1024, "height": 1024}
}
],
"reference_images": [
{"image": {"image_name": "style_reference.png", "width": 1024, "height": 1024}}
]
}
}
```
## WebSocket Events
Parameter updates emit a `recall_parameters_updated` event to the queue
room. Connected frontend clients automatically:
1. Apply standard parameters (prompts, steps, dimensions, etc.).
2. Load and add LoRAs to the LoRA list.
3. Apply control-layer configurations.
4. Apply IP Adapter / FLUX Redux configurations with their images.
5. Append model-free reference images, using the config flavor that
matches the currently-selected main model.
## Error Handling
- **400 Bad Request** — invalid parameters or parameter values.
- **500 Internal Server Error** — server-side storage or retrieval failure.
Errors include detailed messages. Missing images and unresolved model
names are **not** errors — they are logged and the remaining parameters
are still applied.
## Logging
### Backend
```
INFO: Resolved ControlNet model name 'controlnet-canny-sdxl-1.0' to key 'controlnet-xyz...'
INFO: Found image file: depth_map.png (1024x768)
INFO: Updated 12 recall parameters for queue default
INFO: Resolved 1 LoRA(s)
INFO: Resolved 1 control layer(s)
INFO: Resolved 1 IP adapter(s)
INFO: Resolved 1 reference image(s)
```
### Frontend
Set `localStorage.ROARR_FILTER = 'debug'` in the browser to see all debug
messages under the `events` namespace.
```
INFO: Applied 5 recall parameters to store
INFO: Applied 1 IP adapter(s), replacing existing list
INFO: Applied 1 model-free reference image(s)
DEBUG: Built IP adapter ref image state: ip-adapter-xyz... (weight: 0.7)
DEBUG: IP adapter image: outputs/images/depth_map.png (1024x768)
```
## Implementation Details
- Parameters are stored in the client state persistence service, using keys prefixed with `recall_`
- The parameters are associated with a `queue_id`, allowing multiple concurrent sessions to maintain separate parameter sets
- Only non-null parameters are processed and stored
- The endpoint provides validation for numeric ranges (e.g., steps ≥ 1, dimensions ≥ 64)
- All parameter values are JSON-serialized for storage
- When parameter values are changed, the backend generates a web sockets event that the frontend listens to.
- Parameters are stored in the client state persistence service under
`recall_*` keys, scoped to the `queue_id`.
- Numeric validation runs at the FastAPI layer (e.g. `steps ≥ 1`, `width ≥ 64`).
- Only non-null parameters are processed, stored, and broadcast.
- Model-key resolution runs **after** the raw parameters are stored, so
an unresolvable model name simply drops out of the broadcast but does
not corrupt the persisted state.
- The broadcast payload contains resolved model keys and image metadata
(width/height) so the frontend can populate its store without extra
round-trips.
## Integration with Frontend
## Troubleshooting
The stored parameters can be accessed by the frontend through the
existing client state API or by implementing hooks that read from the
recall parameter storage. This allows external applications to
pre-populate generation parameters before the user initiates image
generation.
### Image not found
## Error Handling
If you see "Image file not found" in the logs:
- **400 Bad Request**: Invalid parameters or parameter values
- **500 Internal Server Error**: Server-side error storing or retrieving parameters
1. Verify the filename matches exactly (case-sensitive).
2. Ensure the image is in `{INVOKEAI_ROOT}/outputs/images/`.
3. Check that the filename does not include the `outputs/images/` prefix.
Errors include detailed messages explaining what went wrong.
### Model not found
If you see "Could not find model":
1. Verify the model name matches exactly (case-sensitive).
2. Ensure the model is installed.
3. Check the name via the Models Manager panel.
### Event not received
1. Check the browser console for socket connection errors.
2. Verify the `queue_id` matches the frontend's queue (usually `default`).
3. Check backend logs for event emission errors.
## Limitations
- **Model availability** — models referenced in the payload must be installed.
- **Image availability** — images must exist in `outputs/images`; remote
URLs are not supported.
- **Canvas auto-layer creation** — control layers and IP adapters with
images populate the recall state, but creating a canvas layer from
them still happens through the UI.

62
invokeai/app/api/routers/recall_parameters.py
View File

@@ -57,6 +57,20 @@ class IPAdapterRecallParameter(BaseModel):
)
class ReferenceImageRecallParameter(BaseModel):
"""Global reference-image configuration for recall.
Used for reference images that feed directly into the main model rather
than through a separate IP-Adapter / ControlNet model — for example
FLUX.2 Klein, FLUX Kontext, and Qwen Image Edit. The receiving frontend
picks the correct config type (``flux2_reference_image`` /
``qwen_image_reference_image`` / ``flux_kontext_reference_image``) based
on the currently-selected main model.
"""
image_name: str = Field(description="The filename of the reference image in outputs/images")
class RecallParameter(BaseModel):
"""Request model for updating recallable parameters."""
@@ -104,6 +118,14 @@ class RecallParameter(BaseModel):
ip_adapters: Optional[list[IPAdapterRecallParameter]] = Field(
None, description="List of IP Adapters with their settings"
)
reference_images: Optional[list[ReferenceImageRecallParameter]] = Field(
None,
description=(
"List of model-free reference images for architectures that consume reference "
"images directly (FLUX.2 Klein, FLUX Kontext, Qwen Image Edit). The frontend "
"picks the correct config type based on the currently-selected main model."
),
)
def resolve_model_name_to_key(model_name: str, model_type: ModelType = ModelType.Main) -> Optional[str]:
@@ -291,6 +313,38 @@ def resolve_ip_adapter_models(ip_adapters: list[IPAdapterRecallParameter]) -> li
return resolved_adapters
def resolve_reference_images(
reference_images: list[ReferenceImageRecallParameter],
) -> list[dict[str, Any]]:
"""
Validate model-free reference images and build the configuration list.
Unlike IP Adapters and ControlNets, these reference images are consumed
directly by the main model (FLUX.2 Klein, FLUX Kontext, Qwen Image Edit),
so there is no adapter-model name to resolve. We simply verify that each
referenced file exists in ``outputs/images`` and pass the image metadata
through to the frontend.
Args:
reference_images: List of reference-image recall parameters
Returns:
List of reference-image configurations with resolved image metadata.
Entries whose image file cannot be loaded are dropped with a warning.
"""
logger = ApiDependencies.invoker.services.logger
resolved: list[dict[str, Any]] = []
for ref in reference_images:
image_data = load_image_file(ref.image_name)
if image_data is None:
logger.warning(f"Skipping reference image '{ref.image_name}' - file not found")
continue
resolved.append({"image": image_data})
return resolved
@recall_parameters_router.post(
"/{queue_id}",
operation_id="update_recall_parameters",
@@ -391,6 +445,14 @@ async def update_recall_parameters(
provided_params["ip_adapters"] = resolved_adapters
logger.info(f"Resolved {len(resolved_adapters)} IP adapter(s)")
# Process model-free reference images if provided
if "reference_images" in provided_params:
reference_images_param = parameters.reference_images
if reference_images_param is not None:
resolved_refs = resolve_reference_images(reference_images_param)
provided_params["reference_images"] = resolved_refs
logger.info(f"Resolved {len(resolved_refs)} reference image(s)")
# Emit event to notify frontend of parameter updates
try:
logger.info(

50
invokeai/frontend/web/src/services/events/setEventListeners.tsx
View File

@@ -4,6 +4,7 @@ import { socketConnected } from 'app/store/middleware/listenerMiddleware/listene
import type { AppStore } from 'app/store/store';
import { deepClone } from 'common/util/deepClone';
import { forEach, isNil, round } from 'es-toolkit/compat';
import { getDefaultRefImageConfig } from 'features/controlLayers/hooks/addLayerHooks';
import { allEntitiesDeleted, controlLayerRecalled } from 'features/controlLayers/store/canvasSlice';
import { canvasWorkflowIntegrationProcessingCompleted } from 'features/controlLayers/store/canvasWorkflowIntegrationSlice';
import { loraAllDeleted, loraRecalled } from 'features/controlLayers/store/lorasSlice';
@@ -844,6 +845,55 @@ export const setEventListeners = ({ socket, store, setIsConnected }: SetEventLis
});
}
}
// Handle model-free reference images (FLUX.2 Klein, FLUX Kontext, Qwen Image Edit).
// These feed the reference image directly into the main model rather than going
// through an IP Adapter, so the backend sends them without a model_key and we
// pick the right config type via getDefaultRefImageConfig() based on the main
// model that is currently selected in the UI.
if (data.parameters.reference_images !== undefined && Array.isArray(data.parameters.reference_images)) {
log.debug(`Processing ${data.parameters.reference_images.length} reference image(s)`);
const referenceImagePromises = data.parameters.reference_images
.filter((cfg) => cfg.image?.image_name && typeof cfg.image.image_name === 'string')
.map(async (refConfig) => {
const imageName = refConfig.image.image_name as string;
try {
// Pre-fetch the image DTO so ref image validation succeeds.
await dispatch(imagesApi.endpoints.getImageDTO.initiate(imageName)).unwrap();
} catch (imageError) {
log.warn(`Could not pre-fetch reference image ${imageName}, continuing anyway: ${imageError}`);
}
// Pick the config flavor (flux2 / flux_kontext / ip_adapter fallback) that
// matches the currently-selected main model.
const baseConfig = getDefaultRefImageConfig(getState);
const imageData = {
original: {
image: {
image_name: imageName,
width: typeof refConfig.image.width === 'number' ? refConfig.image.width : 512,
height: typeof refConfig.image.height === 'number' ? refConfig.image.height : 512,
},
},
};
return getReferenceImageState(`recalled-ref-image-${Date.now()}-${Math.random()}`, {
isEnabled: true,
config: { ...baseConfig, image: imageData },
});
});
// Append reference images to whatever the ip_adapters branch already
// dispatched — replace:false so we don't clobber IP adapters.
Promise.all(referenceImagePromises).then((refImageStates) => {
const validStates = refImageStates.filter((state): state is RefImageState => state !== null);
if (validStates.length > 0) {
dispatch(refImagesRecalled({ entities: validStates, replace: false }));
log.info(`Applied ${validStates.length} model-free reference image(s)`);
}
});
}
}
});

159
tests/app/routers/test_recall_parameters.py Normal file
View File

@@ -0,0 +1,159 @@
"""Tests for the recall parameters router.
Focused on the ``reference_images`` field added for model-free reference
images (FLUX.2 Klein, FLUX Kontext, Qwen Image Edit). The existing
``loras`` / ``control_layers`` / ``ip_adapters`` paths are exercised via
integration tests elsewhere; this file pins down the new field's
request-validation, resolver behavior, and event payload.
"""
from typing import Any
import pytest
from fastapi.testclient import TestClient
from invokeai.app.api.dependencies import ApiDependencies
from invokeai.app.api.routers import recall_parameters as recall_module
from invokeai.app.api_app import app
from invokeai.app.services.invoker import Invoker
@pytest.fixture
def client() -> TestClient:
return TestClient(app)
class MockApiDependencies(ApiDependencies):
"""Minimal ApiDependencies stand-in that only wires up an invoker."""
invoker: Invoker
def __init__(self, invoker: Invoker) -> None:
self.invoker = invoker
@pytest.fixture
def patched_dependencies(monkeypatch: Any, mock_invoker: Invoker) -> MockApiDependencies:
"""Install a mock ApiDependencies for the recall_parameters router.
The router persists each parameter via ``client_state_persistence.set_by_key``,
whose ``user_id`` column has a FOREIGN KEY constraint back to the users
table. The mock invoker uses an in-memory SQLite database that is not
pre-populated with any users, so persistence would fail with "FOREIGN
KEY constraint failed" — that's an orthogonal concern to the reference-
images resolver under test, so we stub it out.
"""
dependencies = MockApiDependencies(mock_invoker)
monkeypatch.setattr("invokeai.app.api.routers.recall_parameters.ApiDependencies", dependencies)
monkeypatch.setattr(
mock_invoker.services.client_state_persistence,
"set_by_key",
lambda user_id, key, value: value,
)
return dependencies
class TestReferenceImagesRecall:
def test_reference_images_forwarded_when_image_exists(
self, monkeypatch: Any, patched_dependencies: MockApiDependencies, client: TestClient
) -> None:
"""Reference images whose files exist should flow through to the event payload."""
# Stub load_image_file so we don't need a real outputs/images directory.
def fake_load_image_file(image_name: str) -> dict[str, Any] | None:
return {"image_name": image_name, "width": 1024, "height": 768}
monkeypatch.setattr(recall_module, "load_image_file", fake_load_image_file)
response = client.post(
"/api/v1/recall/default",
json={
"reference_images": [
{"image_name": "cat.png"},
{"image_name": "dog.png"},
]
},
)
assert response.status_code == 200
body = response.json()
assert body["status"] == "success"
assert body["queue_id"] == "default"
# Both references came through, in order.
resolved = body["parameters"]["reference_images"]
assert len(resolved) == 2
assert resolved[0]["image"]["image_name"] == "cat.png"
assert resolved[1]["image"]["image_name"] == "dog.png"
assert resolved[0]["image"]["width"] == 1024
def test_missing_reference_images_are_dropped_without_failing(
self, monkeypatch: Any, patched_dependencies: MockApiDependencies, client: TestClient
) -> None:
"""An image that can't be loaded should be skipped — never 500."""
def fake_load_image_file(image_name: str) -> dict[str, Any] | None:
if image_name == "present.png":
return {"image_name": image_name, "width": 512, "height": 512}
return None
monkeypatch.setattr(recall_module, "load_image_file", fake_load_image_file)
response = client.post(
"/api/v1/recall/default",
json={
"reference_images": [
{"image_name": "missing.png"},
{"image_name": "present.png"},
]
},
)
assert response.status_code == 200
resolved = response.json()["parameters"]["reference_images"]
assert len(resolved) == 1
assert resolved[0]["image"]["image_name"] == "present.png"
def test_reference_images_do_not_require_model_name(
self, monkeypatch: Any, patched_dependencies: MockApiDependencies, client: TestClient
) -> None:
"""The schema must accept a reference image entry with only ``image_name``.
This pins down the "model-free" contract: unlike ``ip_adapters``,
these entries are for FLUX.2 Klein / FLUX Kontext / Qwen Image Edit,
where the reference image feeds the main model directly and there is
no adapter model to name. Callers should be able to omit every
field except ``image_name``.
"""
monkeypatch.setattr(
recall_module,
"load_image_file",
lambda image_name: {"image_name": image_name, "width": 64, "height": 64},
)
response = client.post(
"/api/v1/recall/default",
json={"reference_images": [{"image_name": "ok.png"}]},
)
assert response.status_code == 200
resolved = response.json()["parameters"]["reference_images"]
assert resolved == [{"image": {"image_name": "ok.png", "width": 64, "height": 64}}]
def test_empty_reference_images_is_noop_for_other_fields(
self, monkeypatch: Any, patched_dependencies: MockApiDependencies, client: TestClient
) -> None:
"""Sending an empty reference_images list should not break other fields."""
monkeypatch.setattr(
recall_module,
"load_image_file",
lambda image_name: {"image_name": image_name, "width": 1, "height": 1},
)
response = client.post(
"/api/v1/recall/default",
json={
"positive_prompt": "hello",
"reference_images": [],
},
)
assert response.status_code == 200
params = response.json()["parameters"]
assert params["positive_prompt"] == "hello"
assert params["reference_images"] == []