Lincoln Stein
3f7f9f8d61
add probes for T5_encoder and ClipTextModel
2024-09-03 15:55:51 -04:00
Brandon Rising
f3bb592544
Update latents used for preview images in flux
2024-09-03 14:04:16 -04:00
Brandon Rising
69f080fb75
Move flux step callback code into the step_callback util scripts, use other services within the invocation context
2024-09-03 14:04:16 -04:00
Brandon Rising
04272a7cc8
Initial attempt at preview images
2024-09-03 14:04:16 -04:00
Lincoln Stein
8d35af946e
[MM] add API routes for getting & setting MM cache sizes ( #6523 )
...
* [MM] add API routes for getting & setting MM cache sizes, and retrieving MM stats
* Update invokeai/app/api/routers/model_manager.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com >
* code cleanup after @ryand review
* Update invokeai/app/api/routers/model_manager.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com >
* fix merge conflicts; tested and working
---------
Co-authored-by: Lincoln Stein <lstein@gmail.com >
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com >
2024-09-02 12:18:21 -04:00
Ryan Dick
2313f326df
Add denoise_end param to FluxDenoiseInvocation.
2024-09-02 09:38:17 -04:00
Ryan Dick
8292f6cd42
Code cleanup and documentation around FLUX inpainting.
2024-09-02 09:38:17 -04:00
Ryan Dick
9a8aca69bf
Get a rough version of FLUX inpainting working.
2024-09-02 09:38:17 -04:00
Ryan Dick
a7a33d73f5
Get FLUX non-masked image-to-image working - still rough.
2024-09-02 09:38:17 -04:00
Ryan Dick
e85c3bc465
Add FLUX VAE support to ImageToLatentsInvocation.
2024-09-02 09:38:17 -04:00
Ryan Dick
4e4b6c6dbc
Tidy variable management and dtype handling in FluxTextToImageInvocation.
2024-08-29 19:08:18 +00:00
Ryan Dick
29fe1533f2
Fix bug in InvokeLinear8bitLt that was causing old state information to persist after loading from a state dict. This manifested as state tensors being left on the GPU even when a model had been offloaded to the CPU cache.
2024-08-29 19:08:18 +00:00
Ryan Dick
77090070bd
Check the size of a model on disk and make room for it in the cache before loading it.
2024-08-29 19:08:18 +00:00
Ryan Dick
6ba9b1b6b0
Tidy up GIG -> GB and remove unused GIG constant.
2024-08-29 19:08:18 +00:00
Ryan Dick
c578b8df1e
Improve ModelCache docs.
2024-08-29 19:08:18 +00:00
Ryan Dick
cad9a41433
Remove unused MOdelCache.exists(...) function.
2024-08-29 19:08:18 +00:00
Ryan Dick
5fefb3b0f4
Remove unused param from ModelCache.
2024-08-29 19:08:18 +00:00
Ryan Dick
5284a870b0
Remove unused constructor params from ModelCache.
2024-08-29 19:08:18 +00:00
Ryan Dick
e064377c05
Remove default model cache sizes from model_cache_default.py. These defaults were misleading, because the config defaults take precedence over them.
2024-08-29 19:08:18 +00:00
Ryan Dick
50085b40bb
Update starter model size estimates.
2024-08-26 20:17:50 -04:00
Brandon Rising
54d54d1bf2
Run ruff
2024-08-26 20:17:50 -04:00
Brandon Rising
65bb46bcca
Rename params for flux and flux vae, add comments explaining use of the config_path in model config
2024-08-26 20:17:50 -04:00
Brandon Rising
2d185fb766
Run ruff
2024-08-26 20:17:50 -04:00
Brandon Rising
849da67cc7
Remove no longer used code in the flux denoise function
2024-08-26 20:17:50 -04:00
Ryan Dick
bbf934d980
Remove outdated TODO.
2024-08-26 20:17:50 -04:00
Ryan Dick
620f733110
ruff format
2024-08-26 20:17:50 -04:00
Ryan Dick
635d2f480d
ruff
2024-08-26 20:17:50 -04:00
Brandon Rising
70c278c810
Remove dependency on flux config files
2024-08-26 20:17:50 -04:00
Brandon Rising
56b9906e2e
Setup scaffolding for in progress images and add ability to cancel the flux node
2024-08-26 20:17:50 -04:00
Ryan Dick
a808ce81fd
Replace swish() with torch.nn.functional.silu(h). They are functionally equivalent, but in my test VAE deconding was ~8% faster after the change.
2024-08-26 20:17:50 -04:00
Ryan Dick
83f82c5ddf
Switch the CLIP-L start model to use our hosted version - which is much smaller.
2024-08-26 20:17:50 -04:00
Brandon Rising
101de8c25d
Update t5 encoder formats to accurately reflect the quantization strategy and data type
2024-08-26 20:17:50 -04:00
Ryan Dick
3339a4baf0
Downgrade revert torch version after removing optimum-qanto, and other minor version-related fixes.
2024-08-26 20:17:50 -04:00
Ryan Dick
dff4a88baa
Move quantization scripts to a scripts/ subdir.
2024-08-26 20:17:50 -04:00
Ryan Dick
a21f6c4964
Update docs for T5 quantization script.
2024-08-26 20:17:50 -04:00
Ryan Dick
97562504b7
Remove all references to optimum-quanto and downgrade diffusers.
2024-08-26 20:17:50 -04:00
Ryan Dick
75d8ac378c
Update the T5 8-bit quantized starter model to use the BnB LLM.int8() variant.
2024-08-26 20:17:50 -04:00
Ryan Dick
b9dd354e2b
Fixes to the T5XXL quantization script.
2024-08-26 20:17:50 -04:00
Ryan Dick
33c2fbd201
Add script for quantizing a T5 model.
2024-08-26 20:17:50 -04:00
Brandon Rising
1047584b3e
Only import bnb quantize file if bitsandbytes is installed
2024-08-26 20:17:50 -04:00
Ryan Dick
a0bf20bcee
Run FLUX VAE decoding in the user's preferred dtype rather than float32. Tested, and seems to work well at float16.
2024-08-26 20:17:50 -04:00
Ryan Dick
14ab339b33
Move prepare_latent_image_patches(...) to sampling.py with all of the related FLUX inference code.
2024-08-26 20:17:50 -04:00
Ryan Dick
1c1f2c6664
Add comment about incorrect T5 Tokenizer size calculation.
2024-08-26 20:17:50 -04:00
Ryan Dick
185f2a395f
Make FLUX get_noise(...) consistent across devices/dtypes.
2024-08-26 20:17:50 -04:00
Brandon Rising
94aba5892a
Attribute black-forest-labs/flux for much of the flux code
2024-08-26 20:17:50 -04:00
maryhipp
34451e5f27
added FLUX dev to starter models
2024-08-26 20:17:50 -04:00
Brandon Rising
c27d59baf7
Run ruff
2024-08-26 20:17:50 -04:00
Ryan Dick
b66f19d4d1
Add docs to the quantization scripts.
2024-08-26 20:17:50 -04:00
Ryan Dick
4105a78b83
Update load_flux_model_bnb_llm_int8.py to work with a single-file FLUX transformer checkpoint.
2024-08-26 20:17:50 -04:00
Ryan Dick
19a68afb3a
Fix bug in InvokeInt8Params that was causing it to use double the necessary VRAM.
2024-08-26 20:17:50 -04:00
