Brandon Rising
a16b555d47
Simplify flux model dtype conversion in model loader
2024-09-05 15:47:14 -04:00
Brandon Rising
6667c39c73
Remove dependency of asizeof
2024-09-05 15:47:14 -04:00
Brandon Rising
5219ac12a6
Add comment explaining the cache make room call
2024-09-05 15:47:14 -04:00
Brandon Rising
445f813fb9
Update flux transformer loader to more efficiently use and release memory during upcasting
2024-09-05 15:47:14 -04:00
Brandon Rising
87f9e59cfb
Cast tensors in unquantized flux models to bfloat16 during loading
2024-09-05 15:47:14 -04:00
Brandon Rising
33edee1ba6
Delete all flux bundle state dict keys when extracting the transformer state dict
2024-09-04 09:36:23 -04:00
Brandon Rising
d20335dabc
convert_bundle_to_flux_transformer_checkpoint now removes processed keys to decrease memory usage
2024-09-04 09:36:23 -04:00
Brandon Rising
d10d258213
Add a comment for why we're converting scale tensors in flux models to bfloat16
2024-09-04 09:36:23 -04:00
Brandon
d57ba1ed8b
Update invokeai/backend/model_manager/probe.py
...
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com >
2024-09-04 09:36:23 -04:00
Brandon Rising
2d0e34e57b
Support non-quantized bundles
2024-09-04 09:36:23 -04:00
Brandon Rising
a005d06255
feat: support checkpoint bundles containing more than just the transformer
2024-09-04 09:36:23 -04:00
Lincoln Stein
6dabe4d3ca
assign T5 encoder to base type "Any"
2024-09-03 15:55:51 -04:00
Lincoln Stein
00e4652d30
add more reliable fallback method for determining BnbQuantizedLlmInt8b
2024-09-03 15:55:51 -04:00
Lincoln Stein
b6434c5318
correct modelformat probe for t5 encoders
2024-09-03 15:55:51 -04:00
Lincoln Stein
3f7f9f8d61
add probes for T5_encoder and ClipTextModel
2024-09-03 15:55:51 -04:00
Lincoln Stein
8d35af946e
[MM] add API routes for getting & setting MM cache sizes ( #6523 )
...
* [MM] add API routes for getting & setting MM cache sizes, and retrieving MM stats
* Update invokeai/app/api/routers/model_manager.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com >
* code cleanup after @ryand review
* Update invokeai/app/api/routers/model_manager.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com >
* fix merge conflicts; tested and working
---------
Co-authored-by: Lincoln Stein <lstein@gmail.com >
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com >
2024-09-02 12:18:21 -04:00
Ryan Dick
77090070bd
Check the size of a model on disk and make room for it in the cache before loading it.
2024-08-29 19:08:18 +00:00
Ryan Dick
6ba9b1b6b0
Tidy up GIG -> GB and remove unused GIG constant.
2024-08-29 19:08:18 +00:00
Ryan Dick
c578b8df1e
Improve ModelCache docs.
2024-08-29 19:08:18 +00:00
Ryan Dick
cad9a41433
Remove unused MOdelCache.exists(...) function.
2024-08-29 19:08:18 +00:00
Ryan Dick
5fefb3b0f4
Remove unused param from ModelCache.
2024-08-29 19:08:18 +00:00
Ryan Dick
5284a870b0
Remove unused constructor params from ModelCache.
2024-08-29 19:08:18 +00:00
Ryan Dick
e064377c05
Remove default model cache sizes from model_cache_default.py. These defaults were misleading, because the config defaults take precedence over them.
2024-08-29 19:08:18 +00:00
Ryan Dick
50085b40bb
Update starter model size estimates.
2024-08-26 20:17:50 -04:00
Brandon Rising
65bb46bcca
Rename params for flux and flux vae, add comments explaining use of the config_path in model config
2024-08-26 20:17:50 -04:00
Ryan Dick
bbf934d980
Remove outdated TODO.
2024-08-26 20:17:50 -04:00
Ryan Dick
635d2f480d
ruff
2024-08-26 20:17:50 -04:00
Brandon Rising
70c278c810
Remove dependency on flux config files
2024-08-26 20:17:50 -04:00
Ryan Dick
83f82c5ddf
Switch the CLIP-L start model to use our hosted version - which is much smaller.
2024-08-26 20:17:50 -04:00
Brandon Rising
101de8c25d
Update t5 encoder formats to accurately reflect the quantization strategy and data type
2024-08-26 20:17:50 -04:00
Ryan Dick
75d8ac378c
Update the T5 8-bit quantized starter model to use the BnB LLM.int8() variant.
2024-08-26 20:17:50 -04:00
Brandon Rising
1047584b3e
Only import bnb quantize file if bitsandbytes is installed
2024-08-26 20:17:50 -04:00
Ryan Dick
a0bf20bcee
Run FLUX VAE decoding in the user's preferred dtype rather than float32. Tested, and seems to work well at float16.
2024-08-26 20:17:50 -04:00
Ryan Dick
1c1f2c6664
Add comment about incorrect T5 Tokenizer size calculation.
2024-08-26 20:17:50 -04:00
maryhipp
34451e5f27
added FLUX dev to starter models
2024-08-26 20:17:50 -04:00
Brandon Rising
c27d59baf7
Run ruff
2024-08-26 20:17:50 -04:00
maryhipp
e210c96485
add FLUX schnell starter models and submodels as dependenices or adhoc download options
2024-08-26 20:17:50 -04:00
maryhipp
5f567f41f4
add case for clip embed models in probe
2024-08-26 20:17:50 -04:00
Brandon Rising
72398350b4
More flux loader cleanup
2024-08-26 20:17:50 -04:00
Brandon Rising
df9445c351
Various styling and exception type updates
2024-08-26 20:17:50 -04:00
Brandon Rising
87b7a2e39b
Switch inheritance class of flux model loaders
2024-08-26 20:17:50 -04:00
Brandon Rising
57168d719b
Fix styling/lint
2024-08-26 20:17:50 -04:00
Brandon Rising
dee6d2c98e
Fix support for 8b quantized t5 encoders, update exception messages in flux loaders
2024-08-26 20:17:50 -04:00
Ryan Dick
0c5e11f521
Fix FLUX output image clamping. And a few other minor fixes to make inference work with the full bfloat16 FLUX transformer model.
2024-08-26 20:17:50 -04:00
Brandon Rising
a63f842a13
Select dev/schnell based on state dict, use correct max seq len based on dev/schnell, and shift in inference, separate vae flux params into separate config
2024-08-26 20:17:50 -04:00
Brandon Rising
4bd7fda694
Install sub directories with folders correctly, ensure consistent dtype of tensors in flux pipeline and vae
2024-08-26 20:17:50 -04:00
Brandon Rising
81f0886d6f
Working inference node with quantized bnb nf4 checkpoint
2024-08-26 20:17:50 -04:00
Brandon Rising
723f3ab0a9
Add nf4 bnb quantized format
2024-08-26 20:17:50 -04:00
Brandon Rising
1bd90e0fd4
Run ruff, setup initial text to image node
2024-08-26 20:17:50 -04:00
Brandon Rising
436f18ff55
Add backend functions and classes for Flux implementation, Update the way flux encoders/tokenizers are loaded for prompt encoding, Update way flux vae is loaded
2024-08-26 20:17:50 -04:00
