psychedelicious
b9c7bc8b0e
chore: ruff
2025-03-07 11:45:49 +11:00
psychedelicious
0f45ee04a2
tests: fix test_extract_valid_metadata_from_image to accomodate prev commit
2025-03-07 11:45:49 +11:00
psychedelicious
8e46b03f09
tests: add tests for extract_metadata_from_image
2025-03-07 11:45:49 +11:00
psychedelicious
cf0cbaf0ae
chore: ruff (more)
2025-03-06 10:57:54 +11:00
psychedelicious
ac6fc6eccb
chore: ruff
2025-03-06 10:57:54 +11:00
Mary Hipp
201d7f1fdb
fix test
2025-03-06 10:41:47 +11:00
Mary Hipp
17a5b1bd28
fix test
2025-03-06 10:41:47 +11:00
Ryan Dick
f1fde792ee
Get FLUX Redux working: model loading and inference.
2025-03-06 10:31:17 +11:00
Billy
f2689598c0
Formatting
2025-03-06 09:11:00 +11:00
psychedelicious
d2f8db9745
tidy: remove unused utils
2025-03-06 07:49:35 +11:00
psychedelicious
c1cf01a038
tests: use dangerously_run_function_in_subprocess to fix configure_torch_cuda_allocator tests
2025-03-06 07:49:35 +11:00
psychedelicious
2bfb4fc79c
tests: add util to run a function in separate process
...
This allows our tests to run in an isolated environment. For tests taht implicitly depend on import behaviour, this can prevent side-effects.
The function should only be used for tests.
2025-03-06 07:49:35 +11:00
psychedelicious
d037d8f9aa
tests: update tests for configure_torch_cuda_allocator
2025-03-06 07:49:35 +11:00
psychedelicious
d5401e8443
tests: add testing utils to set/unset env var
2025-03-06 07:49:35 +11:00
Ryan Dick
49191709a0
Mark test_configure_torch_cuda_allocator_raises_if_torch_is_already_imported() to only run if CUDA is available.
2025-02-28 21:39:09 +00:00
Ryan Dick
a36a627f83
Switch from use_cuda_malloc flag to a general pytorch_cuda_alloc_conf config field that allows full customization of the CUDA allocator.
2025-02-28 21:39:09 +00:00
Ryan Dick
b31c71f302
Simplify is_torch_cuda_malloc_enabled() implementation and add unit tests.
2025-02-28 21:39:09 +00:00
psychedelicious
047c643295
tidy(app): document & clean up batch prep logic
2025-02-26 21:04:23 +11:00
psychedelicious
1bb8edf57e
perf(app): optimise batch prep logic even more
...
Found another place where we deepcopy a dict, but it is safe to mutate.
Restructured the prep logic a bit to support this. Updated tests to use the new structure.
2025-02-26 21:04:23 +11:00
psychedelicious
a3e78f0db6
perf(app): optimise batch prep logic
...
- Avoid pydantic models when dict manipulation works
- Avoid extraneous deep copies when we can safely mutate
- Avoid NamedTuple construct and its overhead
- Fix tests to use altered function signatures
- Remove extraneous populate_graph function
2025-02-26 21:04:23 +11:00
psychedelicious
d66cd4e81b
tests(nodes): add test for collect -> iterate type validation
2025-02-06 07:57:26 +11:00
Ryan Dick
5357d6e08e
Rename ConcatenatedLoRALayer to MergedLayerPatch. And other minor cleanup.
2025-01-28 14:51:35 +00:00
Ryan Dick
28514ba59a
Update ConcatenatedLoRALayer to work with all sub-layer types.
2025-01-28 14:51:35 +00:00
Ryan Dick
206f261e45
Add utils for loading FLUX OneTrainer DoRA models.
2025-01-28 14:51:35 +00:00
Ryan Dick
dfa253e75b
Add utils for working with Kohya LoRA keys.
2025-01-28 14:51:35 +00:00
Ryan Dick
faa4fa02c0
Expand unit tests to test for confusion between FLUX LoRA formats.
2025-01-28 14:51:35 +00:00
Ryan Dick
5bd6428fdd
Add is_state_dict_likely_in_flux_onetrainer_format() util function.
2025-01-28 14:51:35 +00:00
Ryan Dick
8b4f411f7b
Add a test state dict for the OneTrainer DoRA format.
2025-01-28 14:51:35 +00:00
Ryan Dick
e2f05d0800
Add unit tests for LoKR patch layers. The new tests trigger a bug when LoKR layers are applied to BnB-quantized layers (also impacts several other LoRA variant types).
2025-01-22 09:20:40 +11:00
psychedelicious
db5f016826
fix(nodes): allow batch datum items to mix ints and floats
...
Unfortunately we cannot do strict floats or ints.
The batch data models don't specify the value types, it instead relies on pydantic parsing. JSON doesn't differentiate between float and int, so a float `1.0` gets parsed as `1` in python.
As a result, we _must_ accept mixed floats and ints for BatchDatum.items.
Tests and validation updated to handle this.
Maybe we should update the BatchDatum model to have a `type` field? Then we could parse as float or int, depending on the inputs...
2025-01-17 12:19:04 +11:00
Ryan Dick
36a3869af0
Add keep_ram_copy_of_weights config option.
2025-01-16 15:35:25 +00:00
Ryan Dick
c76d08d1fd
Add keep_ram_copy option to CachedModelOnlyFullLoad.
2025-01-16 15:08:23 +00:00
Ryan Dick
04087c38ce
Add keep_ram_copy option to CachedModelWithPartialLoad.
2025-01-16 14:51:44 +00:00
Ryan Dick
974b4671b1
Deprecate the ram and vram configs to make the migration to dynamic
...
memory limits smoother for users who had previously overriden these
values.
2025-01-07 16:45:29 +00:00
Ryan Dick
d7ab464176
Offload the current model when locking if it is already partially loaded and we have insufficient VRAM.
2025-01-07 02:53:44 +00:00
Ryan Dick
5eafe1ec7a
Fix ModelCache execution device selection in unit tests.
2025-01-07 01:20:15 +00:00
Ryan Dick
a167632f09
Calculate model cache size limits dynamically based on the available RAM / VRAM.
2025-01-07 01:14:20 +00:00
Ryan Dick
402dd840a1
Add seed to flaky unit test.
2025-01-07 00:31:00 +00:00
Ryan Dick
d0bfa019be
Add 'enable_partial_loading' config flag.
2025-01-07 00:31:00 +00:00
Ryan Dick
535e45cedf
First pass at adding partial loading support to the ModelCache.
2025-01-07 00:30:58 +00:00
psychedelicious
868e06eb8b
tests: fix test_model_install.py
2025-01-03 11:21:23 -05:00
Ryan Dick
9a0a226ce1
Fix bitsandbytes imports in unit tests on MacOS.
2024-12-30 10:41:48 -05:00
Ryan Dick
6fd9b0a274
Delete old sidecar wrapper implementation. This functionality has moved into the custom layers.
2024-12-29 17:33:08 +00:00
Ryan Dick
52fc5a64d4
Add a unit test for a LoRA patch applied to a quantized linear layer with weights streamed from CPU to GPU.
2024-12-29 17:14:55 +00:00
Ryan Dick
a8bef59699
First pass at making custom layer patches work with weights streamed from the CPU to the GPU.
2024-12-29 17:01:37 +00:00
Ryan Dick
6d49ee839c
Switch the LayerPatcher to use 'custom modules' to manage layer patching.
2024-12-29 01:18:30 +00:00
Ryan Dick
2855bb6b41
Update BaseLayerPatch.get_parameters(...) to accept a dict of orig_parameters rather than orig_module. This will enable compatibility between patching and cpu->gpu streaming.
2024-12-28 21:12:53 +00:00
Ryan Dick
918f541af8
Add unit test for a SetParameterLayer patch applied to a CustomFluxRMSNorm layer.
2024-12-28 20:44:48 +00:00
Ryan Dick
93e76b61d6
Add CustomFluxRMSNorm layer.
2024-12-28 20:33:38 +00:00
Ryan Dick
f2981979f9
Get custom layer patches working with all quantized linear layer types.
2024-12-27 22:00:22 +00:00
