Ryan Dick
|
ef970a1cdc
|
Add support for FluxControlLoRALayer in CustomLinear layers and add a unit test for it.
|
2024-12-27 21:00:47 +00:00 |
|
Ryan Dick
|
5ee7405f97
|
Add more unit tests for custom module LoRA patching: multiple LoRAs and ConcatenatedLoRALayers.
|
2024-12-27 19:47:21 +00:00 |
|
Ryan Dick
|
e24e386a27
|
Add support for patches to CustomModuleMixin and add a single unit test (more to come).
|
2024-12-27 18:57:13 +00:00 |
|
Ryan Dick
|
b06d61e3c0
|
Improve custom layer wrap/unwrap logic.
|
2024-12-27 16:29:48 +00:00 |
|
Ryan Dick
|
7d6ab0ceb2
|
Add a CustomModuleMixin class with a flag for enabling/disabling autocasting (since it incurs some runtime speed overhead.)
|
2024-12-26 20:08:30 +00:00 |
|
Ryan Dick
|
9692a36dd6
|
Use a fixture to parameterize tests in test_all_custom_modules.py so that a fresh instance of the layer under test is initialized for each test.
|
2024-12-26 19:41:25 +00:00 |
|
Ryan Dick
|
b0b699a01f
|
Add unit test to test that isinstance(...) behaves as expected with custom module types.
|
2024-12-26 18:45:56 +00:00 |
|
Ryan Dick
|
a8b2c4c3d2
|
Add inference tests for all custom module types (i.e. to test autocasting from cpu to device).
|
2024-12-26 18:33:46 +00:00 |
|
Ryan Dick
|
03944191db
|
Split test_autocast_modules.py into separate test files to mirror the source file structure.
|
2024-12-24 22:29:11 +00:00 |
|
Ryan Dick
|
987c9ae076
|
Move custom autocast modules to separate files in a custom_modules/ directory.
|
2024-12-24 22:21:31 +00:00 |
|
Ryan Dick
|
0fc538734b
|
Skip flaky test when running on Github Actions, and further reduce peak unit test memory.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
7214d4969b
|
Workaround a weird quirk of QuantState.to() and add a unit test to exercise it.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
a83a999b79
|
Reduce peak memory used for unit tests.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
f8a6accf8a
|
Fix bitsandbytes imports to avoid ImportErrors on MacOS.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
f8ab414f99
|
Add CachedModelOnlyFullLoad to mirror the CachedModelWithPartialLoad for models that cannot or should not be partially loaded.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
c6795a1b47
|
Make CachedModelWithPartialLoad work with models that have non-persistent buffers.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
0a8fc74ae9
|
Add CachedModelWithPartialLoad to manage partially-loaded models using the new autocast modules.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
dc54e8763b
|
Add CustomInvokeLinearNF4 to enable CPU -> GPU streaming for InvokeLinearNF4 layers.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
1b56020876
|
Add CustomInvokeLinear8bitLt layer for device streaming with InvokeLinear8bitLt layers.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
97d56f7dc9
|
Add torch module autocast unit test for GGUF-quantized models.
|
2024-12-24 14:32:11 +00:00 |
|
Ryan Dick
|
fe0ef2c27c
|
Add torch module autocast utilities.
|
2024-12-24 14:32:11 +00:00 |
|