Ryan Dick
3d4bd71098
Update test_probe_handles_state_dict_with_integer_keys() to make sure that it is still testing what it's intended to test. Previously, we were skipping an important part of the test by using a fake file path.
2024-10-02 18:33:05 -04:00
Brandon Rising
814be44cd7
Ignore paths that don't exist in probe for unit tests
2024-10-02 18:33:05 -04:00
Brandon Rising
0f333388bb
Add comment describing why we're not using the meta device during probing of gguf files
2024-10-02 18:33:05 -04:00
Ryan Dick
bc63e2acc5
Add workaround for FLUX GGUF models with incorrect img_in.weight shape.
2024-10-02 18:33:05 -04:00
Ryan Dick
ec7e771942
Add a compute_dtype field to GGMLTensor.
2024-10-02 18:33:05 -04:00
Brandon Rising
446e2884bc
Remove no longer used code paths, general cleanup of new dequantization code, update probe
2024-10-02 18:33:05 -04:00
Brandon Rising
7d9f125232
Run ruff and update imports
2024-10-02 18:33:05 -04:00
Brandon Rising
0875e861f5
Various updates to gguf performance
2024-10-02 18:33:05 -04:00
Brandon
0267d73dfc
Update invokeai/backend/model_manager/load/model_loaders/flux.py
...
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com >
2024-10-02 18:33:05 -04:00
Ryan Dick
f06765dfba
Get alternative GGUF implementation working... barely.
2024-10-02 18:33:05 -04:00
Lincoln Stein
c665cf3525
recognize .gguf files when scanning a folder for import
2024-10-02 18:33:05 -04:00
Brandon Rising
2bfb0ddff5
Initial GGUF support for flux models
2024-10-02 18:33:05 -04:00
Mary Hipp
c224971cb4
feat(ui,api): add guidance as a default setting option for FLUX models
2024-09-30 17:15:33 -04:00
Ryan Dick
e88d3cf2f7
Assume alpha=rank for FLUX diffusers PEFT LoRA models.
2024-09-16 13:57:07 +00:00
Ryan Dick
81fbaf2b8b
Assume LoRA alpha=8 for FLUX diffusers PEFT LoRAs.
2024-09-15 04:39:56 +03:00
Ryan Dick
2ff4dae5ce
Add util functions calc_tensor_size(...) and calc_tensors_size(...).
2024-09-15 04:39:56 +03:00
Ryan Dick
5800e60b06
Add model probe support for FLUX LoRA models in Diffusers format.
2024-09-15 04:39:56 +03:00
Ryan Dick
cf9f30cc56
Rename flux_kohya_lora_conversion_utils.py
2024-09-15 04:39:56 +03:00
Ryan Dick
50c9410121
WIP
2024-09-15 04:39:56 +03:00
Ryan Dick
01a15b4d80
WIP - add invocations to support FLUX LORAs.
2024-09-15 04:39:56 +03:00
Ryan Dick
db61ec4322
Get probing of FLUX LoRA kohya models working.
2024-09-15 04:39:56 +03:00
Ryan Dick
04b37e64ea
Move the responsibilities of 1) state_dict loading from file, and 2) SDXL lora key conversions, out of LoRAModelRaw and into LoRALoader.
2024-09-15 04:39:56 +03:00
Ryan Dick
2b3e4e123d
Split LoRA layer implementations into separate files.
2024-09-12 15:53:30 +00:00
psychedelicious
53792fafb3
feat(nodes): add DWOpenposeDetectionInvocation
...
Similar to the existing node, but without any resizing. The backend logic was consolidated and modified so that it the model loading can be managed by the model manager.
The ONNX Runtime `InferenceSession` class was added to the `AnyModel` union to satisfy the type checker.
2024-09-11 08:12:48 -04:00
Brandon Rising
a16b555d47
Simplify flux model dtype conversion in model loader
2024-09-05 15:47:14 -04:00
Brandon Rising
6667c39c73
Remove dependency of asizeof
2024-09-05 15:47:14 -04:00
Brandon Rising
5219ac12a6
Add comment explaining the cache make room call
2024-09-05 15:47:14 -04:00
Brandon Rising
445f813fb9
Update flux transformer loader to more efficiently use and release memory during upcasting
2024-09-05 15:47:14 -04:00
Brandon Rising
87f9e59cfb
Cast tensors in unquantized flux models to bfloat16 during loading
2024-09-05 15:47:14 -04:00
Brandon Rising
33edee1ba6
Delete all flux bundle state dict keys when extracting the transformer state dict
2024-09-04 09:36:23 -04:00
Brandon Rising
d20335dabc
convert_bundle_to_flux_transformer_checkpoint now removes processed keys to decrease memory usage
2024-09-04 09:36:23 -04:00
Brandon Rising
d10d258213
Add a comment for why we're converting scale tensors in flux models to bfloat16
2024-09-04 09:36:23 -04:00
Brandon
d57ba1ed8b
Update invokeai/backend/model_manager/probe.py
...
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com >
2024-09-04 09:36:23 -04:00
Brandon Rising
2d0e34e57b
Support non-quantized bundles
2024-09-04 09:36:23 -04:00
Brandon Rising
a005d06255
feat: support checkpoint bundles containing more than just the transformer
2024-09-04 09:36:23 -04:00
Lincoln Stein
6dabe4d3ca
assign T5 encoder to base type "Any"
2024-09-03 15:55:51 -04:00
Lincoln Stein
00e4652d30
add more reliable fallback method for determining BnbQuantizedLlmInt8b
2024-09-03 15:55:51 -04:00
Lincoln Stein
b6434c5318
correct modelformat probe for t5 encoders
2024-09-03 15:55:51 -04:00
Lincoln Stein
3f7f9f8d61
add probes for T5_encoder and ClipTextModel
2024-09-03 15:55:51 -04:00
Lincoln Stein
8d35af946e
[MM] add API routes for getting & setting MM cache sizes ( #6523 )
...
* [MM] add API routes for getting & setting MM cache sizes, and retrieving MM stats
* Update invokeai/app/api/routers/model_manager.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com >
* code cleanup after @ryand review
* Update invokeai/app/api/routers/model_manager.py
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com >
* fix merge conflicts; tested and working
---------
Co-authored-by: Lincoln Stein <lstein@gmail.com >
Co-authored-by: Ryan Dick <ryanjdick3@gmail.com >
2024-09-02 12:18:21 -04:00
Ryan Dick
77090070bd
Check the size of a model on disk and make room for it in the cache before loading it.
2024-08-29 19:08:18 +00:00
Ryan Dick
6ba9b1b6b0
Tidy up GIG -> GB and remove unused GIG constant.
2024-08-29 19:08:18 +00:00
Ryan Dick
c578b8df1e
Improve ModelCache docs.
2024-08-29 19:08:18 +00:00
Ryan Dick
cad9a41433
Remove unused MOdelCache.exists(...) function.
2024-08-29 19:08:18 +00:00
Ryan Dick
5fefb3b0f4
Remove unused param from ModelCache.
2024-08-29 19:08:18 +00:00
Ryan Dick
5284a870b0
Remove unused constructor params from ModelCache.
2024-08-29 19:08:18 +00:00
Ryan Dick
e064377c05
Remove default model cache sizes from model_cache_default.py. These defaults were misleading, because the config defaults take precedence over them.
2024-08-29 19:08:18 +00:00
Ryan Dick
50085b40bb
Update starter model size estimates.
2024-08-26 20:17:50 -04:00
Brandon Rising
65bb46bcca
Rename params for flux and flux vae, add comments explaining use of the config_path in model config
2024-08-26 20:17:50 -04:00
Ryan Dick
bbf934d980
Remove outdated TODO.
2024-08-26 20:17:50 -04:00
