Commit Graph

13416 Commits

Author SHA1 Message Date
psychedelicious
d927b631c5 feat(ui): add raster layer rendering and interaction (WIP) 2024-09-06 22:56:24 +10:00
psychedelicious
17dc5d98d1 feat(ui): scaffold out raster layers
Raster layers may have images, lines and shapes. These will replace initial image layers and provide sketching functionality like we have on canvas.
2024-09-06 22:56:24 +10:00
psychedelicious
cda086093d refactor(ui): revise types for line and rect objects
- Create separate object types for brush and eraser lines, instead of a single type that has a `tool` field.
- Create new object type for rect shapes.
- Add logic to schemas to migrate old object types to new.
- Update renderers & reducers.
2024-09-06 22:56:24 +10:00
Brandon Rising
bda579577c chore: 4.2.9 version bump v4.2.9 2024-09-05 16:17:48 -04:00
Brandon Rising
a16b555d47 Simplify flux model dtype conversion in model loader 2024-09-05 15:47:14 -04:00
Brandon Rising
6667c39c73 Remove dependency of asizeof 2024-09-05 15:47:14 -04:00
Brandon Rising
5219ac12a6 Add comment explaining the cache make room call 2024-09-05 15:47:14 -04:00
Brandon Rising
445f813fb9 Update flux transformer loader to more efficiently use and release memory during upcasting 2024-09-05 15:47:14 -04:00
Brandon Rising
87f9e59cfb Cast tensors in unquantized flux models to bfloat16 during loading 2024-09-05 15:47:14 -04:00
Phrixus2023
8b03b39aa8 translationBot(ui): update translation (Chinese (Simplified Han script))
Currently translated at 97.6% (1342 of 1374 strings)

Co-authored-by: Phrixus2023 <920414016@qq.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/zh_Hans/
Translation: InvokeAI/Web UI
2024-09-05 15:34:13 -04:00
Tobias Lechner
e59b6bb971 translationBot(ui): update translation (German)
Currently translated at 63.3% (870 of 1374 strings)

Co-authored-by: Tobias Lechner <me@tobias-lechner.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/de/
Translation: InvokeAI/Web UI
2024-09-05 15:34:13 -04:00
Riccardo Giovanetti
24a7ed467c translationBot(ui): update translation (Italian)
Currently translated at 98.2% (1350 of 1374 strings)

translationBot(ui): update translation (Italian)

Currently translated at 98.2% (1350 of 1374 strings)

translationBot(ui): update translation (Italian)

Currently translated at 98.2% (1350 of 1374 strings)

translationBot(ui): update translation (Italian)

Currently translated at 98.4% (1349 of 1370 strings)

translationBot(ui): update translation (Italian)

Currently translated at 98.4% (1348 of 1369 strings)

Co-authored-by: Riccardo Giovanetti <riccardo.giovanetti@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/it/
Translation: InvokeAI/Web UI
2024-09-05 15:34:13 -04:00
Васянатор
f01f1033ac translationBot(ui): update translation (Russian)
Currently translated at 100.0% (1370 of 1370 strings)

translationBot(ui): update translation (Russian)

Currently translated at 100.0% (1369 of 1369 strings)

Co-authored-by: Васянатор <ilabulanov339@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/ru/
Translation: InvokeAI/Web UI
2024-09-05 15:34:13 -04:00
smk-e
d35f515413 translationBot(ui): update translation (Spanish)
Currently translated at 33.0% (452 of 1369 strings)

Co-authored-by: smk-e <jit-r8@outlook.com>
Translate-URL: https://hosted.weblate.org/projects/invokeai/web-ui/es/
Translation: InvokeAI/Web UI
2024-09-05 15:34:13 -04:00
Brandon Rising
125b459e56 chore: 4.2.9rc2 version bump v4.2.9rc2 2024-09-04 10:42:16 -04:00
Brandon Rising
33edee1ba6 Delete all flux bundle state dict keys when extracting the transformer state dict 2024-09-04 09:36:23 -04:00
Brandon Rising
d20335dabc convert_bundle_to_flux_transformer_checkpoint now removes processed keys to decrease memory usage 2024-09-04 09:36:23 -04:00
Brandon Rising
d10d258213 Add a comment for why we're converting scale tensors in flux models to bfloat16 2024-09-04 09:36:23 -04:00
Brandon
d57ba1ed8b Update invokeai/backend/model_manager/probe.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
2024-09-04 09:36:23 -04:00
Brandon Rising
2d0e34e57b Support non-quantized bundles 2024-09-04 09:36:23 -04:00
Brandon Rising
a005d06255 feat: support checkpoint bundles containing more than just the transformer 2024-09-04 09:36:23 -04:00
Eugene Brodsky
a301ef5a5a chore(ci): update github action version pins in container build workflow 2024-09-03 16:01:58 -04:00
Eugene Brodsky
9422df2737 feat(ci): enable a checkbox to push the container image when manually building via workflow dispatch 2024-09-03 16:01:58 -04:00
Lincoln Stein
6dabe4d3ca assign T5 encoder to base type "Any" 2024-09-03 15:55:51 -04:00
Lincoln Stein
00e4652d30 add more reliable fallback method for determining BnbQuantizedLlmInt8b 2024-09-03 15:55:51 -04:00
Lincoln Stein
b6434c5318 correct modelformat probe for t5 encoders 2024-09-03 15:55:51 -04:00
Lincoln Stein
3f7f9f8d61 add probes for T5_encoder and ClipTextModel 2024-09-03 15:55:51 -04:00
Brandon Rising
f3bb592544 Update latents used for preview images in flux 2024-09-03 14:04:16 -04:00
Brandon Rising
69f080fb75 Move flux step callback code into the step_callback util scripts, use other services within the invocation context 2024-09-03 14:04:16 -04:00
Brandon Rising
04272a7cc8 Initial attempt at preview images 2024-09-03 14:04:16 -04:00
Lincoln Stein
8d35af946e [MM] add API routes for getting & setting MM cache sizes (#6523)
* [MM] add API routes for getting & setting MM cache sizes, and retrieving MM stats

* Update invokeai/app/api/routers/model_manager.py

Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>

* code cleanup after @ryand review

* Update invokeai/app/api/routers/model_manager.py

Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>

* fix merge conflicts; tested and working

---------

Co-authored-by: Lincoln Stein <lstein@gmail.com>
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com>
2024-09-02 12:18:21 -04:00
Ryan Dick
24065ec6b6 Add FLUX image-to-image and inpainting (#6798)
## Summary

This PR adds support for Image-to-Image and inpainting workflows with
the FLUX model.

Full changelog:
- Split out `FLUX VAE Encode` and `FLUX VAE Decode` nodes
- Renamed `FLUX Text-to-Image` node to `FLUX Denoise` (since it now
supports image-to-image too). This is a workflow-breaking change.
- Added support for FLUX image-to-image via the `Latents` param on the
FLUX denoising node.
- Added support for FLUX masked inpainting via the `Denoise Mask` param
on the FLUX denoising node.
- Added "Denoise Start" and "Denoise End" params to the "FLUX Denoise"
node.
- Updated the "FLUX Text to Image" default workflow.
- Added a "FLUX Image to Image" default workflow.

### Example

FLUX inpainting workflow
<img width="1282" alt="image"
src="https://github.com/user-attachments/assets/86fc1170-e620-4412-8fd8-e119f875fc2e">

Input image

![image](https://github.com/user-attachments/assets/9c381b86-9f87-4257-bd2e-da22c56ca26c)

Mask

![image](https://github.com/user-attachments/assets/8f774c5c-2a25-45fe-9d4b-b233e3d58d2c)

Output image

![image](https://github.com/user-attachments/assets/8576a630-24ce-4a00-8052-e86bab59c855)


### Callouts for reviewers:
- I renamed FLUXTextToImageInvocation -> FLUXDenoisingInvocation. This
is, of course, a breaking change. It feels like the right move and now
is the right time to do it. Any objection?
- I added new `FLUX VAE Encode` and `FLUX VAE Decode` nodes.
Alternatively, I could have tried to match these names to the
corresponding SD nodes (e.g. `FLUX Image to Latents`, `FLUX Latents to
Image`). Personally, I prefer the current names, but want to hear other
opinions.

### Usage notes:
- With the default dev timestep scheduler, the image structure is
largely determined in the first ~3 steps. A consequence of this is that
the denoise_start parameter provides limited 'granularity' of control.
This will likely be improved in the future as we add more scheduler
options. In the meantime, you will likely want to use small values for
`denoise_start` (e.g. 0.03) to start denoising on step ~1-4 out of ~30.
- Currently, there is no 'noise' parameter on the `FLUX Denoise` node,
so the `denoise_end` parameter has limited utility. This will be added
in the future.

## QA Instructions

Test the following workflows:
- [x] Vanilla FLUX text-to-image behaviour is unchanged
- [x] Image-to-image with FLUX dev, no mask
- [x] Image-to-image with FLUX dev, with mask
- [x] Image-to-image with FLUX schnell, no mask (smoke test, not
expected to work well)

## Merge Plan

No special instructions.

## Checklist

- [x] _The PR has a short but descriptive title, suitable for a
changelog_
- [x] _Tests added / updated (if applicable)_
- [x] _Documentation added / updated (if applicable)_
2024-09-02 09:50:31 -04:00
Ryan Dick
627b0bf644 Expose all FLUX model params in the default FLUX models. 2024-09-02 09:38:17 -04:00
Ryan Dick
b43da46b82 Rename 'FLUX VAE Encode'/'FLUX VAE Decode' to 'FLUX Image to Latents'/'FLUX Latents to Image' 2024-09-02 09:38:17 -04:00
Ryan Dick
4255a01c64 Restore line that was accidentally removed during development. 2024-09-02 09:38:17 -04:00
Ryan Dick
23adbd4002 Update schema.ts. 2024-09-02 09:38:17 -04:00
Ryan Dick
fb5a24fcc6 Update default workflows for FLUX. 2024-09-02 09:38:17 -04:00
Ryan Dick
cfdd5a1900 Rename flux_text_to_image.py -> flex_denoise.py 2024-09-02 09:38:17 -04:00
Ryan Dick
2313f326df Add denoise_end param to FluxDenoiseInvocation. 2024-09-02 09:38:17 -04:00
Ryan Dick
2e092a2313 Rename FluxTextToImageInvocation -> FluxDenoiseInvocation. 2024-09-02 09:38:17 -04:00
Ryan Dick
763ef06c18 Use the existence of initial latents to decide whether we are doing image-to-image in the FLUX denoising node. Previously we were using the denoising_start value, but in some cases with an inpaintin mask you may want to run image-to-image from densoising_start=0. 2024-09-02 09:38:17 -04:00
Ryan Dick
8292f6cd42 Code cleanup and documentation around FLUX inpainting. 2024-09-02 09:38:17 -04:00
Ryan Dick
278bba499e Split FLUX VAE decoding out into its own node from LatentsToImageInvocation. 2024-09-02 09:38:17 -04:00
Ryan Dick
dd99ed28e0 Split FLUX VAE encoding out into its own node from ImageToLatentsInvocation. 2024-09-02 09:38:17 -04:00
Ryan Dick
9a8aca69bf Get a rough version of FLUX inpainting working. 2024-09-02 09:38:17 -04:00
Ryan Dick
7ad62512eb Update MaskTensorToImageInvocation to support input mask tensors with or without a channel dimension. 2024-09-02 09:38:17 -04:00
Ryan Dick
bd466661ec Remove unused vae field from FLUXTextToImageInvocation. 2024-09-02 09:38:17 -04:00
Ryan Dick
7ebb509d05 Bump FLUX node versions after splitting out VAE encode/decode. 2024-09-02 09:38:17 -04:00
Ryan Dick
0aa13c046c Split VAE decoding out from the FLUXTextToImageInvocation. 2024-09-02 09:38:17 -04:00
Ryan Dick
a7a33d73f5 Get FLUX non-masked image-to-image working - still rough. 2024-09-02 09:38:17 -04:00