Commit Graph

2646 Commits

Author SHA1 Message Date
Ryan Dick
6a5cee61be Tune the working memory estimate for FLUX VAE decoding. 2025-01-07 01:20:15 +00:00
Ryan Dick
bd8017ecd5 Update working memory estimate for VAE decoding when tiling is being applied. 2025-01-07 01:20:15 +00:00
Ryan Dick
299eb94a05 Estimate the working memory required for VAE decoding, since this operations tends to be memory intensive. 2025-01-07 01:20:15 +00:00
Ryan Dick
a167632f09 Calculate model cache size limits dynamically based on the available RAM / VRAM. 2025-01-07 01:14:20 +00:00
Ryan Dick
1321fac8f2 Remove get_cache_size() and set_cache_size() endpoints. These were unused by the frontend and refer to cache fields that are no longer accessible. 2025-01-07 01:06:20 +00:00
Ryan Dick
bcd29c5d74 Remove all cases where we check the 'model.device'. This is no longer trustworthy now that partial loading is permitted. 2025-01-07 00:31:00 +00:00
Ryan Dick
d0bfa019be Add 'enable_partial_loading' config flag. 2025-01-07 00:31:00 +00:00
Ryan Dick
535e45cedf First pass at adding partial loading support to the ModelCache. 2025-01-07 00:30:58 +00:00
Riku
f4f7415a3b fix(app): remove obsolete DEFAULT_PRECISION variable 2025-01-06 11:14:58 +11:00
Mary Hipp
7d6c443d6f fix(api): limit board_name length to 300 characters 2025-01-06 10:49:49 +11:00
psychedelicious
b6d7a44004 refactor(events): include full model source in model install events
This is required to fix an issue with the MM UI's error handling.

Previously, we only included the model source as a string. That could be an arbitrary URL, file path or HF repo id, but the frontend has no parsing logic to differentiate between these different model sources.

Without access to the type of model source, it is difficult to determine how the user should proceed. For example, if it's HF URL with an HTTP unauthorized error, we should direct the user to log in to HF. But if it's a civitai URL with the same error, we should not direct the user to HF.

There are a variety of related edge cases.

With this change, the full `ModelSource` object is included in each model install event, including error events.

I had to fix some circular import issues, hence the import changes to files other than `events_common.py`.
2025-01-03 11:21:23 -05:00
Ryan Dick
477d87ec31 Fix layer patch dtype selection for CLIP text encoder models. 2024-12-29 21:48:51 +00:00
Ryan Dick
6d7314ac0a Consolidate the LayerPatching patching modes into a single implementation. 2024-12-24 15:57:54 +00:00
Ryan Dick
80db9537ff Rename model_patcher.py -> layer_patcher.py. 2024-12-24 15:57:54 +00:00
Ryan Dick
61253b91f1 Enable LoRAPatcher.apply_smart_lora_patches(...) throughout the stack. 2024-12-24 15:57:54 +00:00
Ryan Dick
7dc3e0fdbe Get rid of ModelLocker. It was an unnecessary layer of indirection. 2024-12-24 14:23:18 +00:00
Ryan Dick
d30a9ced38 Rename model_cache_default.py -> model_cache.py. 2024-12-24 14:23:18 +00:00
Ryan Dick
e0bfa6157b Remove ModelCacheBase. 2024-12-24 14:23:18 +00:00
Ryan Dick
83ea6420e2 Move CacheStats to its own file. 2024-12-24 14:23:18 +00:00
Riku
525cb38c71 fix(app): fixed InputField default values 2024-12-20 09:30:56 +11:00
psychedelicious
a9a6720bad feat(app): change queue item execution log from debug to info
This provides useful context for subsequent logs during queue item execution.
2024-12-20 09:19:04 +11:00
psychedelicious
858bf9cf8c feat(api): less verbose uvicorn logs
Uvicorn's logging is rather verbose. This change adds a `log_level_network` config setting to independently control uvicorn's log outputs. The setting defaults to warning.

The change hides the helpful startup message that says the host and port we are running on.

For example: `Uvicorn running on http://0.0.0.0:9090 (Press CTRL+C to quit`

The ASGI lifespan handler is updated to log an equivalent message on startup, regardless of log level settings.

Besides being helpful, the launcher relies on a message like this to launch the app. So, previously, if the user set their log level to anything above info (e.g. warning or error), the launcher would fail to open the app. This change prevents that edge case.
2024-12-20 09:19:04 +11:00
David Hauptman
74a29c3735 re-format to fix ruff error 2024-12-19 22:33:17 +11:00
David Hauptman
6fc6be3aa0 Fix error message when adding a local path with quotes around the string 2024-12-19 22:33:17 +11:00
Mary Hipp
c154d833b9 raise error if control lora used with schnell 2024-12-18 10:19:28 -05:00
Ryan Dick
a463e97269 Bump FluxControlLoRALoaderInvocation version. 2024-12-17 13:36:10 +00:00
Ryan Dick
b272d46056 Enable ability to control the weight of FLUX Control LoRAs. 2024-12-17 13:36:10 +00:00
Ryan Dick
dd09509dbd Rename ModelPatcher -> LayerPatcher to avoid conflicts with another ModelPatcher definition. 2024-12-17 13:20:19 +00:00
Ryan Dick
7fad4c9491 Rename LoRAModelRaw to ModelPatchRaw. 2024-12-17 13:20:19 +00:00
Ryan Dick
b820862eab Rename ModelPatcher methods to reflect that they are general model patching methods and are not LoRA-specific. 2024-12-17 13:20:19 +00:00
Ryan Dick
c604a0956e Rename LoRAPatcher -> ModelPatcher. 2024-12-17 13:20:19 +00:00
Ryan Dick
41664f88db Rename backend/patches/conversions/ to backend/patches/lora_conversions/ 2024-12-17 13:20:19 +00:00
Ryan Dick
42f8d6aa11 Rename backend/lora/ to backend/patches 2024-12-17 13:20:19 +00:00
Ryan Dick
a4bed7aee3 Minor tidy of FLUX control LoRA implementation. (mostly documentation) 2024-12-17 07:28:45 -05:00
Ryan Dick
d84adfd39f Clean up FLUX control LoRA pre-processing logic. 2024-12-17 07:28:45 -05:00
Ryan Dick
ac82f73dbe Make FluxControlLoRALoaderOutput.control_lora non-optional. 2024-12-17 07:28:45 -05:00
Brandon Rising
046d19446c Rename Structural Lora to Control Lora 2024-12-17 07:28:45 -05:00
Brandon Rising
f53da60b84 Lots of updates centered around using the lora patcher rather than changing the modules in the transformer model 2024-12-17 07:28:45 -05:00
Brandon Rising
5a035dd19f Support bnb quantized nf4 flux models, Use controlnet vae, only support 1 structural lora per transformer. various other refractors and bugfixes 2024-12-17 07:28:45 -05:00
Brandon Rising
f3b253987f Initial setup for flux tools control loras 2024-12-17 07:28:45 -05:00
Eugene Brodsky
4c94d41fa9 (chore) ruff format 2024-12-04 17:02:08 +00:00
Eugene Brodsky
4036244ee9 (app) clarify log message when migrating old .cache 2024-12-04 17:02:08 +00:00
Eugene Brodsky
d06232d9ba (config) ensure legacy model configs and node template are writable by the user even if the source files are read-only 2024-12-04 17:02:08 +00:00
psychedelicious
3305bad0c2 fix(app): queue item id check before setting cancel flag should use != instead of is not
The `is` operator compares references, not values. Thanks to a wonderfully unintuitive quirk of python, `is` works on integers from `-5` to `256`, inclusive.

Whenever integers in this range are used for a value, internally python returns a reference to a stable object in memory. When integers outside this range are used as a value, python creates a new object in memory for that integer.

See `PyLong_FromLong` documentation here: https://docs.python.org/3/c-api/long.html

Tying this back to our session processor, we were using `is` to compare the queue item ids for equality. Our queue item ids start at 0, and each queue item created increments this by one. So this comparison works only for the first 256 queue items on the machine.

Starting with the 257th queue item, the comparison starts returning `False`, and cancelation gets weird.

Easy fix - use `!=` instead of `is not`.
2024-12-02 23:22:58 -05:00
Mary Hipp
a6be26e174 fix(worker): only apply processor cancel logic if cancel event is for current queue item 2024-12-02 14:03:05 -08:00
Riku
159b00a490 fix(app): adjust session queue api type 2024-12-01 20:06:05 -08:00
Jonathan
6012b0f912 Update flux_text_encoder.py
Updated version number for FLUX Text Encoding.
2024-11-30 08:29:21 -05:00
Jonathan
bb0ed5dc8a Update flux_denoise.py
Updated node version for FLUX Denoise.
2024-11-30 08:29:21 -05:00
psychedelicious
5910892c33 Merge remote-tracking branch 'origin/main' into ryan/flux-regional-prompting 2024-11-29 15:19:39 +10:00
Ryan Dick
8cfb032051 Add utility ImagePanelLayoutInvocation for working with In-Context LoRA workflows. 2024-11-26 20:58:31 -08:00