2115 Commits

Author	SHA1	Message	Date
Ryan Dick	040551d4fb	Fixes to get FLUX Control LoRA working.	2024-12-17 07:28:45 -05:00
Brandon Rising	f53da60b84	Lots of updates centered around using the lora patcher rather than changing the modules in the transformer model	2024-12-17 07:28:45 -05:00
Brandon Rising	5a035dd19f	Support bnb quantized nf4 flux models, Use controlnet vae, only support 1 structural lora per transformer. various other refractors and bugfixes	2024-12-17 07:28:45 -05:00
Brandon Rising	f3b253987f	Initial setup for flux tools control loras	2024-12-17 07:28:45 -05:00
Ryan Dick	0bff4ace1b	Revert performance improvement, because it caused flux inference to fail on Mac: https://github.com/invoke-ai/InvokeAI/issues/7422	2024-12-03 15:18:58 +00:00
Jonathan	c094838c6a	Update model_util.py	2024-12-02 14:35:02 -08:00
Ryan Dick	021552fd81	Avoid unnecessary dtype conversions with rope encodings.	2024-11-29 12:32:50 -05:00
Ryan Dick	be73dbba92	Use view() instead of rearrange() for better performance.	2024-11-29 12:32:50 -05:00
Ryan Dick	db9c0cad7c	Replace custom RMSNorm implementation with torch.nn.functional.rms_norm(...) for improved speed.	2024-11-29 12:32:50 -05:00
psychedelicious	5910892c33	Merge remote-tracking branch 'origin/main' into ryan/flux-regional-prompting	2024-11-29 15:19:39 +10:00
Ryan Dick	5d8dd6e26e	Fix FLUX regional negative prompts.	2024-11-28 18:49:29 +00:00
Ryan Dick	64364e7911	Short-circuit if there are no region masks in FLUX and don't apply attention masking.	2024-11-27 22:40:10 +00:00
Ryan Dick	6565cea039	Comment unused _prepare_unrestricted_attn_mask(...) for future reference.	2024-11-27 22:16:44 +00:00
Ryan Dick	3ebd8d6c07	Delete outdated TODO comment.	2024-11-27 22:13:25 +00:00
Ryan Dick	e970185161	Tweak flux regional prompting attention scheme based on latest experimentation.	2024-11-27 22:13:07 +00:00
Brandon Rising	ed46acee79	fix: Fail scan on InvalidMagicError in picklescan, update default for read_checkpoint_meta to scan unless explicitly told not to	2024-11-26 16:17:12 -05:00
Ryan Dick	b54463d294	Allow regional prompting background regions to attend to themselves and to the entire txt embedding.	2024-11-26 17:57:31 +00:00
Ryan Dick	faee79dc95	Distinguish between restricted and unrestricted attn masks in FLUX regional prompting.	2024-11-26 16:55:52 +00:00
Ryan Dick	e01f66b026	Apply regional attention masks in the single stream blocks in addition to the double stream blocks.	2024-11-25 22:40:08 +00:00
Ryan Dick	53abdde242	Update Flux RegionalPromptingExtension to prepare both a mask with restricted image self-attention and a mask with unrestricted image self attention.	2024-11-25 22:04:23 +00:00
Ryan Dick	94c088300f	Be smarter about selecting the global CLIP embedding for FLUX regional prompting.	2024-11-25 20:15:04 +00:00
Ryan Dick	3741a6f5e0	Fix device handling for regional masks and apply the attention mask in the FLUX double stream block.	2024-11-25 16:02:03 +00:00
Ryan Dick	2c23b8414c	Use a single global CLIP embedding for FLUX regional guidance.	2024-11-22 23:01:43 +00:00
Ryan Dick	20356c0746	Fixup the logic for preparing FLUX regional prompt attention masks.	2024-11-21 22:46:25 +00:00
Ryan Dick	bad1149504	WIP - add rough logic for preparing the FLUX regional prompting attention mask.	2024-11-20 22:29:36 +00:00
Ryan Dick	fda7aaa7ca	Pass RegionalPromptingExtension down to the CustomDoubleStreamBlockProcessor in FLUX.	2024-11-20 19:48:04 +00:00
Ryan Dick	85c616fa34	WIP - Pass prompt masks to FLUX model during denoising.	2024-11-20 18:51:43 +00:00
Mary Hipp	1948ffe106	make sure Soft Edge Detection has preprocessor applied	2024-11-20 08:46:02 -05:00
Brandon Rising	a69c5ff9ef	Add copyright notice for CIELab_to_UPLab.icc	2024-11-19 07:02:37 -08:00
Brandon Rising	1fd80d54a4	Run Ruff	2024-11-19 07:02:37 -08:00
Brandon Rising	991f63e455	Store CIELab_to_UPLab.icc within the repo	2024-11-19 07:02:37 -08:00
Brandon Rising	0eadc0dd9e	feat: Support a subset of composition nodes within base invokeai	2024-11-19 07:02:37 -08:00
Ryan Dick	1a0885bfb1	Update FLUX IP-Adapter starter model from XLabs v1 to XLabs v2.	2024-11-18 17:06:53 -08:00
Ryan Dick	c6fc82f756	Infer the clip_extra_context_tokens param from the state dict for FLUX XLabs IP-Adapter V2 models.	2024-11-18 17:06:53 -08:00
Ryan Dick	1cf7600f5b	Merge branch 'main' into ryan/sd3-image-to-image	2024-11-14 09:25:23 -08:00
Ryan Dick	50897ba066	Add flag to optionally allow missing layer keys in FLUX lora loader.	2024-11-14 16:53:49 +00:00
Ryan Dick	3510643870	Support FLUX LoRAs without .proj_mlp layers.	2024-11-14 16:53:49 +00:00
Ryan Dick	b89caa02bd	Merge branch 'main' into flux_vae_fp16_broke	2024-11-13 15:33:43 -08:00
Darrell	fb19621361	Updated link to flux ip adapter model	2024-11-12 08:11:40 -05:00
Mary Hipp	13cb5f0ba2	Merge remote-tracking branch 'origin/main' into ryan/sd3-image-to-image	2024-11-08 20:29:56 -05:00
Vargol	83609d5008	Merge branch 'invoke-ai:main' into flux_vae_fp16_broke	2024-11-08 10:37:31 +00:00
David Burnett	bb3cedddd5	Rework change based on comments	2024-11-08 10:27:47 +00:00
Ryan Dick	0e6cb91863	Update SD3 InpaintExtension with gradient adjustment to match FLUX.	2024-11-07 22:55:30 +00:00
Ryan Dick	a0fefcd43f	Switch to using a custom scheduler implementation for SD3 rather than the diffusers FlowMatchEulerDiscreteScheduler. It is easier to work with and enables us to re-use the clip_timestep_schedule_fractional() utility from FLUX.	2024-11-07 22:46:52 +00:00
psychedelicious	29d63d5dea	fix(app): silence pydantic protected namespace warning ... Closes #7287	2024-11-08 07:36:50 +11:00
Ryan Dick	a5f8c23dee	Add inpainting support for SD3.	2024-11-07 20:21:43 +00:00
Brandon Rising	f4b0b6a93d	fix: Look in known subfolders for configs for clip variants	2024-11-07 14:02:20 +00:00
Brandon Rising	1337c33ad3	fix: Avoid downloading unsafe .bin files if a safetensors file is available	2024-11-07 14:02:20 +00:00
Brandon Rising	c064257759	fix: Look in known subfolders for configs for clip variants	2024-11-07 12:01:02 +11:00
Brandon Rising	8a4c629576	fix: Avoid downloading unsafe .bin files if a safetensors file is available	2024-11-06 19:31:18 -05:00