psychedelicious
23d59abbd7
chore: ruff
2025-08-12 10:51:05 +10:00
psychedelicious
cf7fa5bce8
perf(backend): clear torch cache after encoding each image in kontext extension
...
Slightly reduces VRAM allocations.
2025-08-12 10:51:05 +10:00
psychedelicious
c6eff71b74
fix(backend): bug in kontext canvas dimension tracking when concating in latent space
...
We weren't tracking the canvas dimensions properly which coudl result in
FLUX not "seeing" ref images after the first very well
2025-08-12 10:51:05 +10:00
psychedelicious
7d86f00d82
feat(mm): implement working memory estimation for VAE encode for all models
...
Tell the model manager that we need some extra working memory for VAE
encoding operations to prevent OOMs.
See previous commit for investigation and determination of the magic
numbers used.
This safety measure is especially relevant now that we have FLUX Kontext
and may be encoding rather large ref images. Without the working memory
estimation we can OOM as we prepare for denoising.
See #8405 for an example of this issue on a very low VRAM system. It's
possible we can have the same issue on any GPU, though - just a matter
of hitting the right combination of models loaded.
2025-08-12 10:51:05 +10:00
psychedelicious
e522de33f8
refactor(nodes): roll back latent-space resizing of kontext images
2025-08-04 23:03:12 +10:00
psychedelicious
56d75e1c77
feat(backend): use VAE mean encoding for Kontext reference images
...
Use distribution mean without sampling noise for more stable and
consistent reference image encoding, matching ComfyUI implementation
2025-08-04 23:03:12 +10:00
psychedelicious
df77a12efe
refactor(backend): use torchvision transforms for Kontext image preprocessing
...
Replace numpy-based normalization with torchvision transforms for
consistency with other image processing in the codebase
2025-08-04 23:03:12 +10:00
psychedelicious
faf662d12e
refactor(backend): use BICUBIC resampling for Kontext images
...
Switch from LANCZOS to BICUBIC for smoother image resizing to reduce
artifacts in reference image processing
2025-08-04 23:03:12 +10:00
psychedelicious
44a7dfd486
fix(backend): use consistent idx_offset=1 for all Kontext images
...
Changes from per-image index offsets to a consistent value of 1 for
all reference images, matching the ComfyUI implementation
2025-08-04 23:03:12 +10:00
psychedelicious
bb15e5cf06
feat(backend): add spatial tiling for multiple Kontext reference images
...
Implements intelligent spatial tiling that arranges multiple reference
images in a virtual canvas, choosing between horizontal and vertical
placement to maintain a square-like aspect ratio
2025-08-04 23:03:12 +10:00
psychedelicious
1a1c846be3
feat(backend): include reference images in negative CFG pass for Kontext
...
Maintains consistency between positive and negative passes to prevent
CFG artifacts when using Kontext reference images
2025-08-04 23:03:12 +10:00
psychedelicious
93c896a370
fix(backend): use img_cond_seq to check for Kontext slicing
...
Was incorrectly checking img_input_ids instead of img_cond_seq
2025-08-04 23:03:12 +10:00
psychedelicious
1ed1fefa60
feat(nodes): support multiple kontext ref images
...
Images are concatenated in latent space.
2025-07-30 12:26:48 +10:00
psychedelicious
68571ece8f
tidy(app): remove unused methods
2025-07-04 20:35:29 +10:00
psychedelicious
4887424ca3
chore: ruff
2025-07-04 20:35:29 +10:00
Kent Keirsey
28f6a20e71
format import block
2025-07-04 20:35:29 +10:00
Kent Keirsey
c4142e75b2
fix import
2025-07-04 20:35:29 +10:00
Kent Keirsey
fefe563127
fix resizing and versioning
2025-07-04 20:35:29 +10:00
Kent Keirsey
e7ce08cffa
ruff format
2025-07-04 19:24:44 +10:00
Kent Keirsey
983cb5ebd2
ruff ruff
2025-07-04 19:24:44 +10:00
Kent Keirsey
52dbdb7118
ruff
2025-07-04 19:24:44 +10:00
Kent Keirsey
71e6f00e10
test fixes
...
fix
test
fix 2
fix 3
fix 4
yet another
attempt new fix
pray
more pray
lol
2025-07-04 19:24:44 +10:00
Kent Keirsey
51e1c56636
ruff
2025-06-27 18:27:46 +10:00
Kent Keirsey
ca1df60e54
Explain the Magic
2025-06-27 18:27:46 +10:00
Cursor Agent
7549c1250d
Add FLUX Kontext conditioning support for reference images
...
Co-authored-by: kent <kent@invoke.ai >
Fix Kontext sequence length handling in Flux denoise invocation
Co-authored-by: kent <kent@invoke.ai >
Fix Kontext step callback to handle combined token sequences
Co-authored-by: kent <kent@invoke.ai >
fix ruff
Fix Flux Kontext
2025-06-27 18:27:46 +10:00
Ryan Dick
7e894ffe83
Consolidate InpaintExtension implementations for SD3 and FLUX.
2025-04-10 10:50:13 +10:00
psychedelicious
a44bfb4658
fix(mm): handle FLUX models w/ diff in_channels keys
...
Before FLUX Fill was merged, we didn't do any checks for the model variant. We always returned "normal".
To determine if a model is a FLUX Fill model, we need to check the state dict for a specific key. Initially, this logic was too strict and rejected quantized FLUX models. This issue was resolved, but it turns out there is another failure mode - some fine-tunes use a different key.
This change further reduces the strictness, handling the alternate key and also falling back to "normal" if we don't see either key. This effectively restores the previous probing behaviour for all FLUX models.
Closes #7856
Closes #7859
2025-03-31 12:32:55 +11:00
Ryan Dick
5ea3ec5cc8
Get FLUX Fill working. Note: To use FLUX Fill, set guidance to ~30.
2025-03-19 14:45:18 +11:00
Ryan Dick
4c02ba908a
Add support for FLUX Redux masks.
2025-03-06 10:31:17 +11:00
Ryan Dick
f1fde792ee
Get FLUX Redux working: model loading and inference.
2025-03-06 10:31:17 +11:00
Ryan Dick
d5211a8088
Add FluxRedux model type and probing logic.
2025-03-06 10:31:17 +11:00
Ryan Dick
b301785dc8
Normalize the T5 model identifiers so that a FLUX T5 or an SD3 T5 model can be used interchangeably.
2025-01-16 08:33:58 +11:00
Ryan Dick
607d19f4dd
We should not trust the value of since the model could be partially-loaded.
2025-01-07 19:22:31 -05:00
Ryan Dick
bcd29c5d74
Remove all cases where we check the 'model.device'. This is no longer trustworthy now that partial loading is permitted.
2025-01-07 00:31:00 +00:00
Ryan Dick
d84adfd39f
Clean up FLUX control LoRA pre-processing logic.
2024-12-17 07:28:45 -05:00
Brandon Rising
70811d0bd0
Remove unexpected artifacts in output images
2024-12-17 07:28:45 -05:00
Brandon Rising
046d19446c
Rename Structural Lora to Control Lora
2024-12-17 07:28:45 -05:00
Brandon Rising
f53da60b84
Lots of updates centered around using the lora patcher rather than changing the modules in the transformer model
2024-12-17 07:28:45 -05:00
Brandon Rising
5a035dd19f
Support bnb quantized nf4 flux models, Use controlnet vae, only support 1 structural lora per transformer. various other refractors and bugfixes
2024-12-17 07:28:45 -05:00
Brandon Rising
f3b253987f
Initial setup for flux tools control loras
2024-12-17 07:28:45 -05:00
Ryan Dick
0bff4ace1b
Revert performance improvement, because it caused flux inference to fail on Mac: https://github.com/invoke-ai/InvokeAI/issues/7422
2024-12-03 15:18:58 +00:00
Ryan Dick
021552fd81
Avoid unnecessary dtype conversions with rope encodings.
2024-11-29 12:32:50 -05:00
Ryan Dick
be73dbba92
Use view() instead of rearrange() for better performance.
2024-11-29 12:32:50 -05:00
Ryan Dick
db9c0cad7c
Replace custom RMSNorm implementation with torch.nn.functional.rms_norm(...) for improved speed.
2024-11-29 12:32:50 -05:00
Ryan Dick
5d8dd6e26e
Fix FLUX regional negative prompts.
2024-11-28 18:49:29 +00:00
Ryan Dick
64364e7911
Short-circuit if there are no region masks in FLUX and don't apply attention masking.
2024-11-27 22:40:10 +00:00
Ryan Dick
6565cea039
Comment unused _prepare_unrestricted_attn_mask(...) for future reference.
2024-11-27 22:16:44 +00:00
Ryan Dick
3ebd8d6c07
Delete outdated TODO comment.
2024-11-27 22:13:25 +00:00
Ryan Dick
e970185161
Tweak flux regional prompting attention scheme based on latest experimentation.
2024-11-27 22:13:07 +00:00
Ryan Dick
b54463d294
Allow regional prompting background regions to attend to themselves and to the entire txt embedding.
2024-11-26 17:57:31 +00:00
