211 Commits

Author	SHA1	Message	Date
Ryan Dick	8b4b0ff0cf	Fix bug in CustomConv1d and CustomConv2d patch calculations.	2024-12-29 19:10:19 +00:00
Ryan Dick	a8bef59699	First pass at making custom layer patches work with weights streamed from the CPU to the GPU.	2024-12-29 17:01:37 +00:00
Ryan Dick	6d49ee839c	Switch the LayerPatcher to use 'custom modules' to manage layer patching.	2024-12-29 01:18:30 +00:00
Ryan Dick	0525f967c2	Fix the _autocast_forward_with_patches() function for CustomConv1d and CustomConv2d.	2024-12-29 00:22:37 +00:00
Ryan Dick	2855bb6b41	Update BaseLayerPatch.get_parameters(...) to accept a dict of orig_parameters rather than orig_module. This will enable compatibility between patching and cpu->gpu streaming.	2024-12-28 21:12:53 +00:00
Ryan Dick	20acfc9a00	Raise in CustomEmbedding and CustomGroupNorm if a patch is applied.	2024-12-28 20:49:17 +00:00
Ryan Dick	918f541af8	Add unit test for a SetParameterLayer patch applied to a CustomFluxRMSNorm layer.	2024-12-28 20:44:48 +00:00
Ryan Dick	93e76b61d6	Add CustomFluxRMSNorm layer.	2024-12-28 20:33:38 +00:00
Ryan Dick	f692e217ea	Add patch support to CustomConv1d and CustomConv2d (no unit tests yet).	2024-12-27 22:23:17 +00:00
Ryan Dick	f2981979f9	Get custom layer patches working with all quantized linear layer types.	2024-12-27 22:00:22 +00:00
Ryan Dick	ef970a1cdc	Add support for FluxControlLoRALayer in CustomLinear layers and add a unit test for it.	2024-12-27 21:00:47 +00:00
Ryan Dick	e24e386a27	Add support for patches to CustomModuleMixin and add a single unit test (more to come).	2024-12-27 18:57:13 +00:00
Ryan Dick	b06d61e3c0	Improve custom layer wrap/unwrap logic.	2024-12-27 16:29:48 +00:00
Ryan Dick	7d6ab0ceb2	Add a CustomModuleMixin class with a flag for enabling/disabling autocasting (since it incurs some runtime speed overhead.)	2024-12-26 20:08:30 +00:00
Ryan Dick	987c9ae076	Move custom autocast modules to separate files in a custom_modules/ directory.	2024-12-24 22:21:31 +00:00
Ryan Dick	7214d4969b	Workaround a weird quirk of QuantState.to() and add a unit test to exercise it.	2024-12-24 14:32:11 +00:00
Ryan Dick	f8a6accf8a	Fix bitsandbytes imports to avoid ImportErrors on MacOS.	2024-12-24 14:32:11 +00:00
Ryan Dick	f8ab414f99	Add CachedModelOnlyFullLoad to mirror the CachedModelWithPartialLoad for models that cannot or should not be partially loaded.	2024-12-24 14:32:11 +00:00
Ryan Dick	c6795a1b47	Make CachedModelWithPartialLoad work with models that have non-persistent buffers.	2024-12-24 14:32:11 +00:00
Ryan Dick	0a8fc74ae9	Add CachedModelWithPartialLoad to manage partially-loaded models using the new autocast modules.	2024-12-24 14:32:11 +00:00
Ryan Dick	dc54e8763b	Add CustomInvokeLinearNF4 to enable CPU -> GPU streaming for InvokeLinearNF4 layers.	2024-12-24 14:32:11 +00:00
Ryan Dick	1b56020876	Add CustomInvokeLinear8bitLt layer for device streaming with InvokeLinear8bitLt layers.	2024-12-24 14:32:11 +00:00
Ryan Dick	fe0ef2c27c	Add torch module autocast utilities.	2024-12-24 14:32:11 +00:00
Ryan Dick	55b13c1da3	(minor) Add TODO comment regarding the location of get_model_cache_key().	2024-12-24 14:23:19 +00:00
Ryan Dick	7dc3e0fdbe	Get rid of ModelLocker. It was an unnecessary layer of indirection.	2024-12-24 14:23:18 +00:00
Ryan Dick	a39bcf7e85	Move lock(...) and unlock(...) logic from ModelLocker to the ModelCache and make a bunch of ModelCache properties/methods private.	2024-12-24 14:23:18 +00:00
Ryan Dick	a7c72992a6	Pull get_model_cache_key(...) out of ModelCache. The ModelCache should not be concerned with implementation details like the submodel_type.	2024-12-24 14:23:18 +00:00
Ryan Dick	d30a9ced38	Rename model_cache_default.py -> model_cache.py.	2024-12-24 14:23:18 +00:00
Ryan Dick	e0bfa6157b	Remove ModelCacheBase.	2024-12-24 14:23:18 +00:00
Ryan Dick	83ea6420e2	Move CacheStats to its own file.	2024-12-24 14:23:18 +00:00
Ryan Dick	ce11a1952e	Move CacheRecord out to its own file.	2024-12-24 14:23:18 +00:00
Ryan Dick	e48dee4c4a	Rip out ModelLockerBase.	2024-12-24 14:23:18 +00:00
Brandon Rising	c9b2cce627	Add diffusers config object for control loras	2024-12-17 14:01:41 -05:00
Ryan Dick	7fad4c9491	Rename LoRAModelRaw to ModelPatchRaw.	2024-12-17 13:20:19 +00:00
Ryan Dick	41664f88db	Rename backend/patches/conversions/ to backend/patches/lora_conversions/	2024-12-17 13:20:19 +00:00
Ryan Dick	42f8d6aa11	Rename backend/lora/ to backend/patches	2024-12-17 13:20:19 +00:00
Brandon Rising	046d19446c	Rename Structural Lora to Control Lora	2024-12-17 07:28:45 -05:00
Brandon Rising	f3b253987f	Initial setup for flux tools control loras	2024-12-17 07:28:45 -05:00
David Burnett	bb3cedddd5	Rework change based on comments	2024-11-08 10:27:47 +00:00
David Burnett	7b5efc2203	Flux Vae broke for float16, force bfloat16 or float32 were compatible	2024-11-06 17:47:22 -05:00
Brandon Rising	ebabf4f7a8	Setup Model and T5 Encoder selection fields for sd3 nodes	2024-11-04 12:42:09 -05:00
Ryan Dick	c620581699	Bug fixes to get SD3 text-to-image workflow running.	2024-11-04 12:42:09 -05:00
Ryan Dick	586c00bc02	(minor) Remove unused dict.	2024-11-04 12:42:09 -05:00
Ryan Dick	a2486a5f06	Remove unused prediction_type and upcast_attention from from_single_file(...) calls.	2024-10-28 13:05:17 -04:00
Ryan Dick	07ab116efb	Remove load_safety_checker=False from calls to from_single_file(...). ... This param has been deprecated, and by including it (even when set to False) the safety checker automatically gets downloaded.	2024-10-28 13:05:17 -04:00
David Burnett	24f9b46fbc	ruff fix	2024-10-23 10:09:24 +11:00
David Burnett	54b3aa1d01	load t5 model in the same format as it is saved, seems to load as float32 on Macs	2024-10-23 10:09:24 +11:00
Ryan Dick	e545f18a45	(minor) Fix ruff.	2024-10-21 22:38:06 +00:00
Ryan Dick	f70a8e2c1a	A bunch of HACKS to get ViT-L CLIP vision encoder working for FLUX IP-Adapter. Need to revisit how to clean this all up long term.	2024-10-21 15:43:00 +00:00
Ryan Dick	c2a8fbd8d6	(minor) Move infer_xlabs_ip_adapter_params_from_state_dict(...) to state_dict_utils.py.	2024-10-21 15:38:50 +00:00