- Allow `uploadImage` util to accept `metadata` to embed in the image
- Update compositor to support `metadata` field when uploading rasterized composite layer
- Add async zod refiner to `zImageWithDims` which fetches the image as part of validation
- Add `zServerValidatedModelIdentifierField`, a zod-refined version of `zModelIdentifierField` which fetches the model as part of validation
- Add `zCanvasMetadata` zod schema, which contains only canvas entities - no bbox, and no `isHidden` flags
- Renamed "Send to Canvas" -> "New Layer from Image"
- Added "New Canvas from Image"
This clarifies the purpose of the menu items and gives tablet users a way to easily add images tot he canvas.
Also update the verbiage for the alerts:
- "Sending to Canvas" -> "Staging Generations on Canvas"
- "Sending to Gallery" -> "Sending Generations to Gallery"
- Add buttons to zoom in/out
- Update hotkeys for fit & 100% to match affinity (e.g. ctrl+0, ctrl+1)
- Add hotkeys for 200%, 400%, 800%
- Update tooltips
This mirrors affinity/photoshop's default `d` hotkey, which sets the fg/bg to white/black. We don't have a concept of "background color", and white is more useful for control images, so it sets to white.
- Rework hotkey data to include the keys for each hotkey action.
- Add wrapper for `useHotkeys` that accepts a hotkey category and id. Automatically selects the key from the hotkey data.
- Add handling for macOS (cmd vs ctrl, option vs alt).
- Redo all hotkey descriptions, deleting nonexistant ones.
- Some `esc` hotkeys that just close whatever you are currently in are omitted due to their relative simplicity and intuitiveness.
This was caused by allowing the stage to be set to fractional coordinates. For example, the stage might be positioned at `x: 142.22255, y: 488.79`.
When positioned like this, the canvas will be slightly misaligned with its native pixel grid. The browser does its best, but this causes tiny scaling artifacts throughout the image. It's most noticeable where there is a sharp contrast.
This behaviour was introduced while troubleshooting an issue with degraded quality when saving canvas to gallery. Turned out the stage position was unrelated to that issue, but I didn't realize that the change would cause this other type of problem.
The fix is super simple - ensure we floor stage coords when setting the manually. Konva never sets the position to fractional coordinates itself. For example, while dragging the stage, Konva sets the stage coordiantes itself, and they are always integers.
## Summary
This PR adds support for FLUX LoRA models on both quantized and
non-quantized base models.
Supported formats:
- diffusers
- kohya
Full changelist:
- Consolidated LoRA handling code in `invokeai/backend/lora`
- Add support for FLUX kohya and FLUX diffusers LoRA model loading
- Add ability to either patch LoRAs or run as a sidecar model (the
latter enables LoRAs to be applied to a wide range of quantized models).
## QA Instructions
Note to reviewers: I tested everything in this checklist. Feel free to
re-verify any of this, but also test any LoRAs that you have. There are
many small LoRA format variations, and there's a risk of breaking one of
them with this change.
FLUX LoRA
- [x] Import / probe of kohya FLUX LoRA
(https://civitai.com/models/159333/pokemon-trainer-sprite-pixelart?modelVersionId=779247)
- [x] Import / probe of Diffusers FLUX LoRA
(https://civitai.com/models/200255/hands-xl-sd-15-flux1-dev?modelVersionId=781855)
- [x] kohya with non-quantized base model
- [x] kohya with quantized base model (should roughly match the
non-quantized case)
- [x] diffusers with non-quantized base model
- [x] diffusers with quantized base model (should roughly match the
non-quantized case)
- [x] Sidecar LoRA patching speed (<0.1secs after model is loaded)
- [x] Stacking multiple fused LoRA models (i.e. on top on non-quantized
model)
- [x] Stacking multiple sidecar LoRA models (i.e. on top of quantized
model)
Regression Tests
- [x] SD1.5 LoRA (check output, speed and memory)
- [x] SDXL LoRA (check output, speed and memory)
- [x] `USE_MODULAR_DENOISE=1` smoke test with LoRA
Test for output regression with the following LoRA formats:
- [x] LoRA
- [x] LoHA
- [x] LoKr
- [x] IA3
## Checklist
- [x] _The PR has a short but descriptive title, suitable for a
changelog_
- [x] _Tests added / updated (if applicable)_
- [x] _Documentation added / updated (if applicable)_
