Ryan Dick
|
ef970a1cdc
|
Add support for FluxControlLoRALayer in CustomLinear layers and add a unit test for it.
|
2024-12-27 21:00:47 +00:00 |
|
Ryan Dick
|
e24e386a27
|
Add support for patches to CustomModuleMixin and add a single unit test (more to come).
|
2024-12-27 18:57:13 +00:00 |
|
Ryan Dick
|
b06d61e3c0
|
Improve custom layer wrap/unwrap logic.
|
2024-12-27 16:29:48 +00:00 |
|
Ryan Dick
|
7d6ab0ceb2
|
Add a CustomModuleMixin class with a flag for enabling/disabling autocasting (since it incurs some runtime speed overhead.)
|
2024-12-26 20:08:30 +00:00 |
|
Ryan Dick
|
a8b2c4c3d2
|
Add inference tests for all custom module types (i.e. to test autocasting from cpu to device).
|
2024-12-26 18:33:46 +00:00 |
|
Ryan Dick
|
987c9ae076
|
Move custom autocast modules to separate files in a custom_modules/ directory.
|
2024-12-24 22:21:31 +00:00 |
|
Ryan Dick
|
6d7314ac0a
|
Consolidate the LayerPatching patching modes into a single implementation.
|
2024-12-24 15:57:54 +00:00 |
|
Ryan Dick
|
80db9537ff
|
Rename model_patcher.py -> layer_patcher.py.
|
2024-12-24 15:57:54 +00:00 |
|
Ryan Dick
|
6f926f05b0
|
Update apply_smart_model_patches() so that layer restore matches the behavior of non-smart mode.
|
2024-12-24 15:57:54 +00:00 |
|
Ryan Dick
|
cefcb340d9
|
Add LoRAPatcher.smart_apply_lora_patches()
|
2024-12-24 15:57:54 +00:00 |
|
Ryan Dick
|
7214d4969b
|
Workaround a weird quirk of QuantState.to() and add a unit test to exercise it.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
f8a6accf8a
|
Fix bitsandbytes imports to avoid ImportErrors on MacOS.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
f8ab414f99
|
Add CachedModelOnlyFullLoad to mirror the CachedModelWithPartialLoad for models that cannot or should not be partially loaded.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
c6795a1b47
|
Make CachedModelWithPartialLoad work with models that have non-persistent buffers.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
0a8fc74ae9
|
Add CachedModelWithPartialLoad to manage partially-loaded models using the new autocast modules.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
dc54e8763b
|
Add CustomInvokeLinearNF4 to enable CPU -> GPU streaming for InvokeLinearNF4 layers.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
1b56020876
|
Add CustomInvokeLinear8bitLt layer for device streaming with InvokeLinear8bitLt layers.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
3f990393a1
|
Simplify the state management in InvokeLinear8bitLt and add unit tests. This is in preparation for wrapping it to support streaming of weights from cpu to gpu.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
fe0ef2c27c
|
Add torch module autocast utilities.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
65fcbf5f60
|
Bump bitsandbytes. The new verson contains improvements to state_dict loading/saving for LLM.int8 and promises improved speed on some HW.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
55b13c1da3
|
(minor) Add TODO comment regarding the location of get_model_cache_key().
|
2024-12-24 14:23:19 +00:00 |
|
Ryan Dick
|
7dc3e0fdbe
|
Get rid of ModelLocker. It was an unnecessary layer of indirection.
|
2024-12-24 14:23:18 +00:00 |
|
Ryan Dick
|
a39bcf7e85
|
Move lock(...) and unlock(...) logic from ModelLocker to the ModelCache and make a bunch of ModelCache properties/methods private.
|
2024-12-24 14:23:18 +00:00 |
|
Ryan Dick
|
a7c72992a6
|
Pull get_model_cache_key(...) out of ModelCache. The ModelCache should not be concerned with implementation details like the submodel_type.
|
2024-12-24 14:23:18 +00:00 |
|
Ryan Dick
|
d30a9ced38
|
Rename model_cache_default.py -> model_cache.py.
|
2024-12-24 14:23:18 +00:00 |
|
Ryan Dick
|
e0bfa6157b
|
Remove ModelCacheBase.
|
2024-12-24 14:23:18 +00:00 |
|
Ryan Dick
|
83ea6420e2
|
Move CacheStats to its own file.
|
2024-12-24 14:23:18 +00:00 |
|
Ryan Dick
|
ce11a1952e
|
Move CacheRecord out to its own file.
|
2024-12-24 14:23:18 +00:00 |
|
Ryan Dick
|
e48dee4c4a
|
Rip out ModelLockerBase.
|
2024-12-24 14:23:18 +00:00 |
|
Mary Hipp
|
0c2f96daf1
|
add probe for ControlLoRA x diffusers
|
2024-12-17 14:01:41 -05:00 |
|
Brandon Rising
|
c9b2cce627
|
Add diffusers config object for control loras
|
2024-12-17 14:01:41 -05:00 |
|
Mary Hipp
|
401fb392b8
|
add FLUX control loras to starter models
|
2024-12-17 09:29:21 -05:00 |
|
Ryan Dick
|
b272d46056
|
Enable ability to control the weight of FLUX Control LoRAs.
|
2024-12-17 13:36:10 +00:00 |
|
Ryan Dick
|
dd09509dbd
|
Rename ModelPatcher -> LayerPatcher to avoid conflicts with another ModelPatcher definition.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
7fad4c9491
|
Rename LoRAModelRaw to ModelPatchRaw.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
b820862eab
|
Rename ModelPatcher methods to reflect that they are general model patching methods and are not LoRA-specific.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
c604a0956e
|
Rename LoRAPatcher -> ModelPatcher.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
9369b39a12
|
Add GGMLTensor op.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
80f64abd1e
|
Use a FluxControlLoRALayer when loading FLUX control LoRAs.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
37e3089457
|
Push LoRA layer reshaping down into the patch layers and add a new FluxControlLoRALayer type.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
fe09f2d27a
|
Move handling of LoRA scale and patch weight down into the layer patch classes.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
e7e3f7e144
|
Ensure that patches are on the correct device when used in sidecar wrappers.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
606d58d7db
|
Add sidecar wrapper for FLUX RMSNorm layers to support SetParameterLayers used by FLUX structural control LoRAs.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
c76a448846
|
Delete old sidecar_layers/ dir.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
46133b5656
|
Switch LoRAPatcher to use the new sidecar_wrappers/ rather than sidecar_layers/.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
ac28370fd2
|
Break up functions in LoRAPatcher in preparation for more refactoring.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
1e0552c813
|
Add optimized implementations for the LinearSidecarWrapper when using LoRALayer or ConcatenatedLoRALayer patch types (since these are the most common).
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
e2451ef5ca
|
A unit tests for LinearSidecarWrapper (and fix a bug).
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
443d838fd0
|
Add initial basic implementation of sidecar wrappers.
|
2024-12-17 13:20:19 +00:00 |
|
Ryan Dick
|
808e3770d3
|
Remove AnyLoRALayer type definition in favor of using BaseLayerPatch base class.
|
2024-12-17 13:20:19 +00:00 |
|