wip

.github/pull_request_template.md

@@ -19,4 +19,3 @@
 - [ ] _The PR has a short but descriptive title, suitable for a changelog_
 - [ ] _Tests added / updated (if applicable)_
 - [ ] _Documentation added / updated (if applicable)_
-- [ ] _Updated `What's New` copy (if doing a release after this PR)_

docker/Dockerfile

@@ -38,7 +38,7 @@ RUN --mount=type=cache,target=/root/.cache/pip \
     if [ "$TARGETPLATFORM" = "linux/arm64" ] || [ "$GPU_DRIVER" = "cpu" ]; then \
         extra_index_url_arg="--extra-index-url https://download.pytorch.org/whl/cpu"; \
     elif [ "$GPU_DRIVER" = "rocm" ]; then \
-        extra_index_url_arg="--extra-index-url https://download.pytorch.org/whl/rocm6.1"; \
+        extra_index_url_arg="--extra-index-url https://download.pytorch.org/whl/rocm5.6"; \
     else \
         extra_index_url_arg="--extra-index-url https://download.pytorch.org/whl/cu124"; \
     fi &&\

docs/contributing/contribution_guides/newContributorChecklist.md

@@ -5,7 +5,7 @@ If you're a new contributor to InvokeAI or Open Source Projects, this is the gui
 ## New Contributor Checklist
 
 - [x] Set up your local development environment & fork of InvokAI by following [the steps outlined here](../dev-environment.md)
-- [x] Set up your local tooling with [this guide](../LOCAL_DEVELOPMENT.md). Feel free to skip this step if you already have tooling you're comfortable with.
+- [x] Set up your local tooling with [this guide](InvokeAI/contributing/LOCAL_DEVELOPMENT/#developing-invokeai-in-vscode). Feel free to skip this step if you already have tooling you're comfortable with.
 - [x] Familiarize yourself with [Git](https://www.atlassian.com/git) & our project structure by reading through the [development documentation](development.md)
 - [x] Join the [#dev-chat](https://discord.com/channels/1020123559063990373/1049495067846524939) channel of the Discord
 - [x] Choose an issue to work on! This can be achieved by asking in the #dev-chat channel, tackling a [good first issue](https://github.com/invoke-ai/InvokeAI/contribute) or finding an item on the [roadmap](https://github.com/orgs/invoke-ai/projects/7). If nothing in any of those places catches your eye, feel free to work on something of interest to you!

docs/contributing/dev-environment.md

@@ -17,49 +17,46 @@ If you just want to use Invoke, you should use the [installer][installer link].
 ## Setup
 
 1. Run through the [requirements][requirements link].
-2. [Fork and clone][forking link] the [InvokeAI repo][repo link].
-3. Create an directory for user data (images, models, db, etc). This is typically at `~/invokeai`, but if you already have a non-dev install, you may want to create a separate directory for the dev install.
-4. Create a python virtual environment inside the directory you just created:
+1. [Fork and clone][forking link] the [InvokeAI repo][repo link].
+1. Create an directory for user data (images, models, db, etc). This is typically at `~/invokeai`, but if you already have a non-dev install, you may want to create a separate directory for the dev install.
+1. Create a python virtual environment inside the directory you just created:
 
-      ```sh
-      python3 -m venv .venv --prompt InvokeAI-Dev
-      ```
+   ```sh
+   python3 -m venv .venv --prompt InvokeAI-Dev
+   ```
 
-5. Activate the venv (you'll need to do this every time you want to run the app):
+1. Activate the venv (you'll need to do this every time you want to run the app):
 
-        ```sh
-        source .venv/bin/activate
-        ```
+   ```sh
+   source .venv/bin/activate
+   ```
 
-6. Install the repo as an [editable install][editable install link]:
+1. Install the repo as an [editable install][editable install link]:
 
-      ```sh
-      pip install -e ".[dev,test,xformers]" --use-pep517 --extra-index-url https://download.pytorch.org/whl/cu121
-      ```
+   ```sh
+   pip install -e ".[dev,test,xformers]" --use-pep517 --extra-index-url https://download.pytorch.org/whl/cu121
+   ```
 
-      Refer to the [manual installation][manual install link]] instructions for more determining the correct install options. `xformers` is optional, but `dev` and `test` are not.
+   Refer to the [manual installation][manual install link]] instructions for more determining the correct install options. `xformers` is optional, but `dev` and `test` are not.
 
-7. Install the frontend dev toolchain:
+1. Install the frontend dev toolchain:
 
    - [`nodejs`](https://nodejs.org/) (recommend v20 LTS)
-   - [`pnpm`](https://pnpm.io/8.x/installation) (must be v8 - not v9!)
+   - [`pnpm`](https://pnpm.io/installation#installing-a-specific-version) (must be v8 - not v9!)
 
-8. Do a production build of the frontend:
+1. Do a production build of the frontend:
 
-      ```sh
-      cd PATH_TO_INVOKEAI_REPO/invokeai/frontend/web
-      pnpm i
-      pnpm build
-      ```
+   ```sh
+   pnpm build
+   ```
 
-9. Start the application:
+1. Start the application:
 
-      ```sh
-      cd PATH_TO_INVOKEAI_REPO
-      python scripts/invokeai-web.py
-      ```
+   ```sh
+   python scripts/invokeai-web.py
+   ```
 
-10. Access the UI at `localhost:9090`.
+1. Access the UI at `localhost:9090`.
 
 ## Updating the UI
 

docs/faq.md

@@ -209,7 +209,7 @@ checkpoint models.
 
 To solve this, go to the Model Manager tab (the cube), select the
 checkpoint model that's giving you trouble, and press the "Convert"
-button in the upper right of your browser window. This will convert the
+button in the upper right of your browser window. This will conver the
 checkpoint into a diffusers model, after which loading should be
 faster and less memory-intensive.
 

docs/installation/patchmatch.md

@@ -97,16 +97,16 @@ Prior to installing PyPatchMatch, you need to take the following steps:
    sudo pacman -S --needed base-devel
    ```
 
-2. Install `opencv`, `blas`, and required dependencies:
+2. Install `opencv` and `blas`:
 
    ```sh
-   sudo pacman -S opencv blas fmt glew vtk hdf5
+   sudo pacman -S opencv blas
    ```
 
    or for CUDA support
 
    ```sh
-   sudo pacman -S opencv-cuda blas fmt glew vtk hdf5
+   sudo pacman -S opencv-cuda blas
    ```
 
 3. Fix the naming of the `opencv` package configuration file:

installer/install.sh.in

@@ -12,7 +12,7 @@ MINIMUM_PYTHON_VERSION=3.10.0
 MAXIMUM_PYTHON_VERSION=3.11.100
 PYTHON=""
 for candidate in python3.11 python3.10 python3 python ; do
-    if ppath=`which $candidate 2>/dev/null`; then
+    if ppath=`which $candidate`; then
         # when using `pyenv`, the executable for an inactive Python version will exist but will not be operational
         # we check that this found executable can actually run
         if [ $($candidate --version &>/dev/null; echo ${PIPESTATUS}) -gt 0 ]; then continue; fi
@@ -30,11 +30,10 @@ done
 if [ -z "$PYTHON" ]; then
     echo "A suitable Python interpreter could not be found"
     echo "Please install Python $MINIMUM_PYTHON_VERSION or higher (maximum $MAXIMUM_PYTHON_VERSION) before running this script. See instructions at $INSTRUCTIONS for help."
+    echo "For the best user experience we suggest enlarging or maximizing this window now."
     read -p "Press any key to exit"
     exit -1
 fi
 
-echo "For the best user experience we suggest enlarging or maximizing this window now."
-
 exec $PYTHON ./lib/main.py ${@}
 read -p "Press any key to exit"

installer/lib/installer.py

@@ -245,9 +245,6 @@ class InvokeAiInstance:
 
         pip = local[self.pip]
 
-        # Uninstall xformers if it is present; the correct version of it will be reinstalled if needed
-        _ = pip["uninstall", "-yqq", "xformers"] & FG
-
         pipeline = pip[
             "install",
             "--require-virtualenv",
@@ -410,7 +407,7 @@ def get_torch_source() -> Tuple[str | None, str | None]:
     optional_modules: str | None = None
     if OS == "Linux":
         if device == GpuType.ROCM:
-            url = "https://download.pytorch.org/whl/rocm6.1"
+            url = "https://download.pytorch.org/whl/rocm5.6"
         elif device == GpuType.CPU:
             url = "https://download.pytorch.org/whl/cpu"
         elif device == GpuType.CUDA:

installer/lib/messages.py

@@ -259,7 +259,7 @@ def select_gpu() -> GpuType:
                     [
                         f"Detected the [gold1]{OS}-{ARCH}[/] platform",
                         "",
-                        "See [deep_sky_blue1]https://invoke-ai.github.io/InvokeAI/installation/requirements/[/] to ensure your system meets the minimum requirements.",
+                        "See [deep_sky_blue1]https://invoke-ai.github.io/InvokeAI/#system[/] to ensure your system meets the minimum requirements.",
                         "",
                         "[red3]🠶[/] [b]Your GPU drivers must be correctly installed before using InvokeAI![/] [red3]🠴[/]",
                     ]

installer/templates/invoke.sh.in

@@ -68,7 +68,7 @@ do_line_input() {
     printf "2: Open the developer console\n"
     printf "3: Command-line help\n"
     printf "Q: Quit\n\n"
-    printf "To update, download and run the installer from https://github.com/invoke-ai/InvokeAI/releases/latest\n\n"
+    printf "To update, download and run the installer from https://github.com/invoke-ai/InvokeAI/releases/latest.\n\n"
     read -p "Please enter 1-4, Q: [1] " yn
     choice=${yn:='1'}
     do_choice $choice

invokeai/app/api/routers/app_info.py

@@ -40,8 +40,6 @@ class AppVersion(BaseModel):
 
     version: str = Field(description="App version")
 
-    highlights: Optional[list[str]] = Field(default=None, description="Highlights of release")
-
 
 class AppDependencyVersions(BaseModel):
     """App depencency Versions Response"""

invokeai/app/api/routers/model_manager.py

@@ -1,7 +1,6 @@
 # Copyright (c) 2023 Lincoln D. Stein
 """FastAPI route for model configuration records."""
 
-import contextlib
 import io
 import pathlib
 import shutil
@@ -11,7 +10,6 @@ from enum import Enum
 from tempfile import TemporaryDirectory
 from typing import List, Optional, Type
 
-import huggingface_hub
 from fastapi import Body, Path, Query, Response, UploadFile
 from fastapi.responses import FileResponse, HTMLResponse
 from fastapi.routing import APIRouter
@@ -29,7 +27,6 @@ from invokeai.app.services.model_records import (
     ModelRecordChanges,
     UnknownModelException,
 )
-from invokeai.app.util.suppress_output import SuppressOutput
 from invokeai.backend.model_manager.config import (
     AnyModelConfig,
     BaseModelType,
@@ -41,12 +38,7 @@ from invokeai.backend.model_manager.load.model_cache.model_cache_base import Cac
 from invokeai.backend.model_manager.metadata.fetch.huggingface import HuggingFaceMetadataFetch
 from invokeai.backend.model_manager.metadata.metadata_base import ModelMetadataWithFiles, UnknownMetadataException
 from invokeai.backend.model_manager.search import ModelSearch
-from invokeai.backend.model_manager.starter_models import (
-    STARTER_BUNDLES,
-    STARTER_MODELS,
-    StarterModel,
-    StarterModelWithoutDependencies,
-)
+from invokeai.backend.model_manager.starter_models import STARTER_MODELS, StarterModel, StarterModelWithoutDependencies
 
 model_manager_router = APIRouter(prefix="/v2/models", tags=["model_manager"])
 
@@ -751,7 +743,7 @@ async def convert_model(
 
     with TemporaryDirectory(dir=ApiDependencies.invoker.services.configuration.models_path) as tmpdir:
         convert_path = pathlib.Path(tmpdir) / pathlib.Path(model_config.path).stem
-        converted_model = loader.load_model(model_config, queue_id="default")
+        converted_model = loader.load_model(model_config)
         # write the converted file to the convert path
         raw_model = converted_model.model
         assert hasattr(raw_model, "save_pretrained")
@@ -800,52 +792,22 @@ async def convert_model(
     return new_config
 
 
-class StarterModelResponse(BaseModel):
-    starter_models: list[StarterModel]
-    starter_bundles: dict[str, list[StarterModel]]
-
-
-def get_is_installed(
-    starter_model: StarterModel | StarterModelWithoutDependencies, installed_models: list[AnyModelConfig]
-) -> bool:
-    for model in installed_models:
-        if model.source == starter_model.source:
-            return True
-        if (
-            (model.name == starter_model.name or model.name in starter_model.previous_names)
-            and model.base == starter_model.base
-            and model.type == starter_model.type
-        ):
-            return True
-    return False
-
-
-@model_manager_router.get("/starter_models", operation_id="get_starter_models", response_model=StarterModelResponse)
-async def get_starter_models() -> StarterModelResponse:
+@model_manager_router.get("/starter_models", operation_id="get_starter_models", response_model=list[StarterModel])
+async def get_starter_models() -> list[StarterModel]:
     installed_models = ApiDependencies.invoker.services.model_manager.store.search_by_attr()
+    installed_model_sources = {m.source for m in installed_models}
     starter_models = deepcopy(STARTER_MODELS)
-    starter_bundles = deepcopy(STARTER_BUNDLES)
     for model in starter_models:
-        model.is_installed = get_is_installed(model, installed_models)
+        if model.source in installed_model_sources:
+            model.is_installed = True
         # Remove already-installed dependencies
         missing_deps: list[StarterModelWithoutDependencies] = []
-
         for dep in model.dependencies or []:
-            if not get_is_installed(dep, installed_models):
+            if dep.source not in installed_model_sources:
                 missing_deps.append(dep)
         model.dependencies = missing_deps
 
-    for bundle in starter_bundles.values():
-        for model in bundle:
-            model.is_installed = get_is_installed(model, installed_models)
-            # Remove already-installed dependencies
-            missing_deps: list[StarterModelWithoutDependencies] = []
-            for dep in model.dependencies or []:
-                if not get_is_installed(dep, installed_models):
-                    missing_deps.append(dep)
-            model.dependencies = missing_deps
-
-    return StarterModelResponse(starter_models=starter_models, starter_bundles=starter_bundles)
+    return starter_models
 
 
 @model_manager_router.get(
@@ -926,51 +888,3 @@ async def get_stats() -> Optional[CacheStats]:
     """Return performance statistics on the model manager's RAM cache. Will return null if no models have been loaded."""
 
     return ApiDependencies.invoker.services.model_manager.load.ram_cache.stats
-
-
-class HFTokenStatus(str, Enum):
-    VALID = "valid"
-    INVALID = "invalid"
-    UNKNOWN = "unknown"
-
-
-class HFTokenHelper:
-    @classmethod
-    def get_status(cls) -> HFTokenStatus:
-        try:
-            if huggingface_hub.get_token_permission(huggingface_hub.get_token()):
-                # Valid token!
-                return HFTokenStatus.VALID
-            # No token set
-            return HFTokenStatus.INVALID
-        except Exception:
-            return HFTokenStatus.UNKNOWN
-
-    @classmethod
-    def set_token(cls, token: str) -> HFTokenStatus:
-        with SuppressOutput(), contextlib.suppress(Exception):
-            huggingface_hub.login(token=token, add_to_git_credential=False)
-        return cls.get_status()
-
-
-@model_manager_router.get("/hf_login", operation_id="get_hf_login_status", response_model=HFTokenStatus)
-async def get_hf_login_status() -> HFTokenStatus:
-    token_status = HFTokenHelper.get_status()
-
-    if token_status is HFTokenStatus.UNKNOWN:
-        ApiDependencies.invoker.services.logger.warning("Unable to verify HF token")
-
-    return token_status
-
-
-@model_manager_router.post("/hf_login", operation_id="do_hf_login", response_model=HFTokenStatus)
-async def do_hf_login(
-    token: str = Body(description="Hugging Face token to use for login", embed=True),
-) -> HFTokenStatus:
-    HFTokenHelper.set_token(token)
-    token_status = HFTokenHelper.get_status()
-
-    if token_status is HFTokenStatus.UNKNOWN:
-        ApiDependencies.invoker.services.logger.warning("Unable to verify HF token")
-
-    return token_status

invokeai/app/api/sockets.py

@@ -31,7 +31,6 @@ from invokeai.app.services.events.events_common import (
     ModelInstallErrorEvent,
     ModelInstallStartedEvent,
     ModelLoadCompleteEvent,
-    ModelLoadEventBase,
     ModelLoadStartedEvent,
     QueueClearedEvent,
     QueueEventBase,
@@ -54,13 +53,6 @@ class BulkDownloadSubscriptionEvent(BaseModel):
     bulk_download_id: str
 
 
-class ModelLoadSubscriptionEvent(BaseModel):
-    """Event data for subscribing to the socket.io model loading room.
-    This is a pydantic model to ensure the data is in the correct format."""
-
-    queue_id: str
-
-
 QUEUE_EVENTS = {
     InvocationStartedEvent,
     InvocationProgressEvent,
@@ -77,6 +69,8 @@ MODEL_EVENTS = {
     DownloadErrorEvent,
     DownloadProgressEvent,
     DownloadStartedEvent,
+    ModelLoadStartedEvent,
+    ModelLoadCompleteEvent,
     ModelInstallDownloadProgressEvent,
     ModelInstallDownloadsCompleteEvent,
     ModelInstallStartedEvent,
@@ -85,11 +79,6 @@ MODEL_EVENTS = {
     ModelInstallErrorEvent,
 }
 
-MODEL_LOAD_EVENTS = {
-    ModelLoadStartedEvent,
-    ModelLoadCompleteEvent,
-}
-
 BULK_DOWNLOAD_EVENTS = {BulkDownloadStartedEvent, BulkDownloadCompleteEvent, BulkDownloadErrorEvent}
 
 
@@ -112,7 +101,6 @@ class SocketIO:
 
         register_events(QUEUE_EVENTS, self._handle_queue_event)
         register_events(MODEL_EVENTS, self._handle_model_event)
-        register_events(MODEL_LOAD_EVENTS, self._handle_model_load_event)
         register_events(BULK_DOWNLOAD_EVENTS, self._handle_bulk_image_download_event)
 
     async def _handle_sub_queue(self, sid: str, data: Any) -> None:
@@ -127,18 +115,9 @@ class SocketIO:
     async def _handle_unsub_bulk_download(self, sid: str, data: Any) -> None:
         await self._sio.leave_room(sid, BulkDownloadSubscriptionEvent(**data).bulk_download_id)
 
-    async def _handle_sub_model_load(self, sid: str, data: Any) -> None:
-        await self._sio.enter_room(sid, ModelLoadSubscriptionEvent(**data).queue_id)
-
-    async def _handle_unsub_model_load(self, sid: str, data: Any) -> None:
-        await self._sio.leave_room(sid, ModelLoadSubscriptionEvent(**data).queue_id)
-
     async def _handle_queue_event(self, event: FastAPIEvent[QueueEventBase]):
         await self._sio.emit(event=event[0], data=event[1].model_dump(mode="json"), room=event[1].queue_id)
 
-    async def _handle_model_load_event(self, event: FastAPIEvent[ModelLoadEventBase]) -> None:
-        await self._sio.emit(event=event[0], data=event[1].model_dump(mode="json"), room=event[1].queue_id)
-
     async def _handle_model_event(self, event: FastAPIEvent[ModelEventBase | DownloadEventBase]) -> None:
         await self._sio.emit(event=event[0], data=event[1].model_dump(mode="json"))
 

invokeai/app/invocations/baseinvocation.py

@@ -4,7 +4,6 @@ from __future__ import annotations
 
 import inspect
 import re
-import sys
 import warnings
 from abc import ABC, abstractmethod
 from enum import Enum
@@ -193,19 +192,12 @@ class BaseInvocation(ABC, BaseModel):
         """Gets a pydantc TypeAdapter for the union of all invocation types."""
         if not cls._typeadapter or cls._typeadapter_needs_update:
             AnyInvocation = TypeAliasType(
-                "AnyInvocation", Annotated[Union[tuple(cls.get_invocations())], Field(discriminator="type")]
+                "AnyInvocation", Annotated[Union[tuple(cls._invocation_classes)], Field(discriminator="type")]
             )
             cls._typeadapter = TypeAdapter(AnyInvocation)
             cls._typeadapter_needs_update = False
         return cls._typeadapter
 
-    @classmethod
-    def invalidate_typeadapter(cls) -> None:
-        """Invalidates the typeadapter, forcing it to be rebuilt on next access. If the invocation allowlist or
-        denylist is changed, this should be called to ensure the typeadapter is updated and validation respects
-        the updated allowlist and denylist."""
-        cls._typeadapter_needs_update = True
-
     @classmethod
     def get_invocations(cls) -> Iterable[BaseInvocation]:
         """Gets all invocations, respecting the allowlist and denylist."""
@@ -487,26 +479,6 @@ def invocation(
             title="type", default=invocation_type, json_schema_extra={"field_kind": FieldKind.NodeAttribute}
         )
 
-        # Validate the `invoke()` method is implemented
-        if "invoke" in cls.__abstractmethods__:
-            raise ValueError(f'Invocation "{invocation_type}" must implement the "invoke" method')
-
-        # And validate that `invoke()` returns a subclass of `BaseInvocationOutput
-        invoke_return_annotation = signature(cls.invoke).return_annotation
-
-        try:
-            # TODO(psyche): If `invoke()` is not defined, `return_annotation` ends up as the string "BaseInvocationOutput"
-            # instead of the class `BaseInvocationOutput`. This may be a pydantic bug: https://github.com/pydantic/pydantic/issues/7978
-            if isinstance(invoke_return_annotation, str):
-                invoke_return_annotation = getattr(sys.modules[cls.__module__], invoke_return_annotation)
-
-            assert invoke_return_annotation is not BaseInvocationOutput
-            assert issubclass(invoke_return_annotation, BaseInvocationOutput)
-        except Exception:
-            raise ValueError(
-                f'Invocation "{invocation_type}" must have a return annotation of a subclass of BaseInvocationOutput (got "{invoke_return_annotation}")'
-            )
-
         docstring = cls.__doc__
         cls = create_model(
             cls.__qualname__,

invokeai/app/invocations/compel.py

@@ -63,12 +63,12 @@ class CompelInvocation(BaseInvocation):
 
     @torch.no_grad()
     def invoke(self, context: InvocationContext) -> ConditioningOutput:
-        tokenizer_info = context.models.load(self.clip.tokenizer, queue_id=context.util.get_queue_id())
-        text_encoder_info = context.models.load(self.clip.text_encoder, queue_id=context.util.get_queue_id())
+        tokenizer_info = context.models.load(self.clip.tokenizer)
+        text_encoder_info = context.models.load(self.clip.text_encoder)
 
         def _lora_loader() -> Iterator[Tuple[LoRAModelRaw, float]]:
             for lora in self.clip.loras:
-                lora_info = context.models.load(lora.lora, queue_id=context.util.get_queue_id())
+                lora_info = context.models.load(lora.lora)
                 assert isinstance(lora_info.model, LoRAModelRaw)
                 yield (lora_info.model, lora.weight)
                 del lora_info
@@ -137,8 +137,8 @@ class SDXLPromptInvocationBase:
         lora_prefix: str,
         zero_on_empty: bool,
     ) -> Tuple[torch.Tensor, Optional[torch.Tensor]]:
-        tokenizer_info = context.models.load(clip_field.tokenizer, queue_id=context.util.get_queue_id())
-        text_encoder_info = context.models.load(clip_field.text_encoder, queue_id=context.util.get_queue_id())
+        tokenizer_info = context.models.load(clip_field.tokenizer)
+        text_encoder_info = context.models.load(clip_field.text_encoder)
 
         # return zero on empty
         if prompt == "" and zero_on_empty:
@@ -163,7 +163,7 @@ class SDXLPromptInvocationBase:
 
         def _lora_loader() -> Iterator[Tuple[LoRAModelRaw, float]]:
             for lora in clip_field.loras:
-                lora_info = context.models.load(lora.lora, context.util.get_queue_id())
+                lora_info = context.models.load(lora.lora)
                 lora_model = lora_info.model
                 assert isinstance(lora_model, LoRAModelRaw)
                 yield (lora_model, lora.weight)

invokeai/app/invocations/controlnet_image_processors.py

@@ -649,9 +649,7 @@ class DepthAnythingImageProcessorInvocation(ImageProcessorInvocation):
             return DepthAnythingPipeline(depth_anything_pipeline)
 
         with self._context.models.load_remote_model(
-            source=DEPTH_ANYTHING_MODELS[self.model_size],
-            queue_id=self._context.util.get_queue_id(),
-            loader=load_depth_anything,
+            source=DEPTH_ANYTHING_MODELS[self.model_size], loader=load_depth_anything
         ) as depth_anything_detector:
             assert isinstance(depth_anything_detector, DepthAnythingPipeline)
             depth_map = depth_anything_detector.generate_depth(image)

invokeai/app/invocations/create_denoise_mask.py

@@ -60,7 +60,7 @@ class CreateDenoiseMaskInvocation(BaseInvocation):
         )
 
         if image_tensor is not None:
-            vae_info = context.models.load(self.vae.vae, context.util.get_queue_id())
+            vae_info = context.models.load(self.vae.vae)
 
             img_mask = tv_resize(mask, image_tensor.shape[-2:], T.InterpolationMode.BILINEAR, antialias=False)
             masked_image = image_tensor * torch.where(img_mask < 0.5, 0.0, 1.0)

invokeai/app/invocations/create_gradient_mask.py

@@ -124,7 +124,7 @@ class CreateGradientMaskInvocation(BaseInvocation):
             assert isinstance(main_model_config, MainConfigBase)
             if main_model_config.variant is ModelVariantType.Inpaint:
                 mask = blur_tensor
-                vae_info: LoadedModel = context.models.load(self.vae.vae, context.util.get_queue_id())
+                vae_info: LoadedModel = context.models.load(self.vae.vae)
                 image = context.images.get_pil(self.image.image_name)
                 image_tensor = image_resized_to_grid_as_tensor(image.convert("RGB"))
                 if image_tensor.dim() == 3:

invokeai/app/invocations/denoise_latents.py

@@ -13,7 +13,6 @@ from diffusers.models.unets.unet_2d_condition import UNet2DConditionModel
 from diffusers.schedulers.scheduling_dpmsolver_sde import DPMSolverSDEScheduler
 from diffusers.schedulers.scheduling_tcd import TCDScheduler
 from diffusers.schedulers.scheduling_utils import SchedulerMixin as Scheduler
-from PIL import Image
 from pydantic import field_validator
 from torchvision.transforms.functional import resize as tv_resize
 from transformers import CLIPVisionModelWithProjection
@@ -88,7 +87,7 @@ def get_scheduler(
     # TODO(ryand): Silently falling back to ddim seems like a bad idea. Look into why this was added and remove if
     # possible.
     scheduler_class, scheduler_extra_config = SCHEDULER_MAP.get(scheduler_name, SCHEDULER_MAP["ddim"])
-    orig_scheduler_info = context.models.load(scheduler_info, context.util.get_queue_id())
+    orig_scheduler_info = context.models.load(scheduler_info)
     with orig_scheduler_info as orig_scheduler:
         scheduler_config = orig_scheduler.config
 
@@ -435,9 +434,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
 
         controlnet_data: list[ControlNetData] = []
         for control_info in control_list:
-            control_model = exit_stack.enter_context(
-                context.models.load(control_info.control_model, context.util.get_queue_id())
-            )
+            control_model = exit_stack.enter_context(context.models.load(control_info.control_model))
             assert isinstance(control_model, ControlNetModel)
 
             control_image_field = control_info.image
@@ -494,9 +491,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
             raise ValueError(f"Unexpected control_input type: {type(control_input)}")
 
         for control_info in control_list:
-            model = exit_stack.enter_context(
-                context.models.load(control_info.control_model, context.util.get_queue_id())
-            )
+            model = exit_stack.enter_context(context.models.load(control_info.control_model))
             ext_manager.add_extension(
                 ControlNetExt(
                     model=model,
@@ -515,7 +510,6 @@ class DenoiseLatentsInvocation(BaseInvocation):
         context: InvocationContext,
         t2i_adapters: Optional[Union[T2IAdapterField, list[T2IAdapterField]]],
         ext_manager: ExtensionsManager,
-        bgr_mode: bool = False,
     ) -> None:
         if t2i_adapters is None:
             return
@@ -525,10 +519,6 @@ class DenoiseLatentsInvocation(BaseInvocation):
             t2i_adapters = [t2i_adapters]
 
         for t2i_adapter_field in t2i_adapters:
-            image = context.images.get_pil(t2i_adapter_field.image.image_name)
-            if bgr_mode:  # SDXL t2i trained on cv2's BGR outputs, but PIL won't convert straight to BGR
-                r, g, b = image.split()
-                image = Image.merge("RGB", (b, g, r))
             ext_manager.add_extension(
                 T2IAdapterExt(
                     node_context=context,
@@ -549,21 +539,15 @@ class DenoiseLatentsInvocation(BaseInvocation):
         """Run the IPAdapter CLIPVisionModel, returning image prompt embeddings."""
         image_prompts = []
         for single_ip_adapter in ip_adapters:
-            with context.models.load(
-                single_ip_adapter.ip_adapter_model, context.util.get_queue_id()
-            ) as ip_adapter_model:
+            with context.models.load(single_ip_adapter.ip_adapter_model) as ip_adapter_model:
                 assert isinstance(ip_adapter_model, IPAdapter)
-                image_encoder_model_info = context.models.load(
-                    single_ip_adapter.image_encoder_model, context.util.get_queue_id()
-                )
+                image_encoder_model_info = context.models.load(single_ip_adapter.image_encoder_model)
                 # `single_ip_adapter.image` could be a list or a single ImageField. Normalize to a list here.
                 single_ipa_image_fields = single_ip_adapter.image
                 if not isinstance(single_ipa_image_fields, list):
                     single_ipa_image_fields = [single_ipa_image_fields]
 
-                single_ipa_images = [
-                    context.images.get_pil(image.image_name, mode="RGB") for image in single_ipa_image_fields
-                ]
+                single_ipa_images = [context.images.get_pil(image.image_name) for image in single_ipa_image_fields]
                 with image_encoder_model_info as image_encoder_model:
                     assert isinstance(image_encoder_model, CLIPVisionModelWithProjection)
                     # Get image embeddings from CLIP and ImageProjModel.
@@ -589,9 +573,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
         for single_ip_adapter, (image_prompt_embeds, uncond_image_prompt_embeds) in zip(
             ip_adapters, image_prompts, strict=True
         ):
-            ip_adapter_model = exit_stack.enter_context(
-                context.models.load(single_ip_adapter.ip_adapter_model, context.util.get_queue_id())
-            )
+            ip_adapter_model = exit_stack.enter_context(context.models.load(single_ip_adapter.ip_adapter_model))
 
             mask_field = single_ip_adapter.mask
             mask = context.tensors.load(mask_field.tensor_name) if mask_field is not None else None
@@ -631,20 +613,14 @@ class DenoiseLatentsInvocation(BaseInvocation):
         t2i_adapter_data = []
         for t2i_adapter_field in t2i_adapter:
             t2i_adapter_model_config = context.models.get_config(t2i_adapter_field.t2i_adapter_model.key)
-            t2i_adapter_loaded_model = context.models.load(
-                t2i_adapter_field.t2i_adapter_model, context.util.get_queue_id()
-            )
-            image = context.images.get_pil(t2i_adapter_field.image.image_name, mode="RGB")
+            t2i_adapter_loaded_model = context.models.load(t2i_adapter_field.t2i_adapter_model)
+            image = context.images.get_pil(t2i_adapter_field.image.image_name)
 
             # The max_unet_downscale is the maximum amount that the UNet model downscales the latent image internally.
             if t2i_adapter_model_config.base == BaseModelType.StableDiffusion1:
                 max_unet_downscale = 8
             elif t2i_adapter_model_config.base == BaseModelType.StableDiffusionXL:
                 max_unet_downscale = 4
-
-                # SDXL adapters are trained on cv2's BGR outputs
-                r, g, b = image.split()
-                image = Image.merge("RGB", (b, g, r))
             else:
                 raise ValueError(f"Unexpected T2I-Adapter base model type: '{t2i_adapter_model_config.base}'.")
 
@@ -652,39 +628,29 @@ class DenoiseLatentsInvocation(BaseInvocation):
             with t2i_adapter_loaded_model as t2i_adapter_model:
                 total_downscale_factor = t2i_adapter_model.total_downscale_factor
 
+                # Resize the T2I-Adapter input image.
+                # We select the resize dimensions so that after the T2I-Adapter's total_downscale_factor is applied, the
+                # result will match the latent image's dimensions after max_unet_downscale is applied.
+                t2i_input_height = latents_shape[2] // max_unet_downscale * total_downscale_factor
+                t2i_input_width = latents_shape[3] // max_unet_downscale * total_downscale_factor
+
                 # Note: We have hard-coded `do_classifier_free_guidance=False`. This is because we only want to prepare
                 # a single image. If CFG is enabled, we will duplicate the resultant tensor after applying the
                 # T2I-Adapter model.
                 #
                 # Note: We re-use the `prepare_control_image(...)` from ControlNet for T2I-Adapter, because it has many
                 # of the same requirements (e.g. preserving binary masks during resize).
-
-                # Assuming fixed dimensional scaling of LATENT_SCALE_FACTOR.
-                _, _, latent_height, latent_width = latents_shape
-                control_height_resize = latent_height * LATENT_SCALE_FACTOR
-                control_width_resize = latent_width * LATENT_SCALE_FACTOR
                 t2i_image = prepare_control_image(
                     image=image,
                     do_classifier_free_guidance=False,
-                    width=control_width_resize,
-                    height=control_height_resize,
+                    width=t2i_input_width,
+                    height=t2i_input_height,
                     num_channels=t2i_adapter_model.config["in_channels"],  # mypy treats this as a FrozenDict
                     device=t2i_adapter_model.device,
                     dtype=t2i_adapter_model.dtype,
                     resize_mode=t2i_adapter_field.resize_mode,
                 )
 
-                # Resize the T2I-Adapter input image.
-                # We select the resize dimensions so that after the T2I-Adapter's total_downscale_factor is applied, the
-                # result will match the latent image's dimensions after max_unet_downscale is applied.
-                # We crop the image to this size so that the positions match the input image on non-standard resolutions
-                t2i_input_height = latents_shape[2] // max_unet_downscale * total_downscale_factor
-                t2i_input_width = latents_shape[3] // max_unet_downscale * total_downscale_factor
-                if t2i_image.shape[2] > t2i_input_height or t2i_image.shape[3] > t2i_input_width:
-                    t2i_image = t2i_image[
-                        :, :, : min(t2i_image.shape[2], t2i_input_height), : min(t2i_image.shape[3], t2i_input_width)
-                    ]
-
                 adapter_state = t2i_adapter_model(t2i_image)
 
             if do_classifier_free_guidance:
@@ -932,13 +898,12 @@ class DenoiseLatentsInvocation(BaseInvocation):
             #    ext = extension_field.to_extension(exit_stack, context, ext_manager)
             #    ext_manager.add_extension(ext)
             self.parse_controlnet_field(exit_stack, context, self.control, ext_manager)
-            bgr_mode = self.unet.unet.base == BaseModelType.StableDiffusionXL
-            self.parse_t2i_adapter_field(exit_stack, context, self.t2i_adapter, ext_manager, bgr_mode)
+            self.parse_t2i_adapter_field(exit_stack, context, self.t2i_adapter, ext_manager)
 
             # ext: t2i/ip adapter
             ext_manager.run_callback(ExtensionCallbackType.SETUP, denoise_ctx)
 
-            unet_info = context.models.load(self.unet.unet, context.util.get_queue_id())
+            unet_info = context.models.load(self.unet.unet)
             assert isinstance(unet_info.model, UNet2DConditionModel)
             with (
                 unet_info.model_on_device() as (cached_weights, unet),
@@ -1001,13 +966,13 @@ class DenoiseLatentsInvocation(BaseInvocation):
 
         def _lora_loader() -> Iterator[Tuple[LoRAModelRaw, float]]:
             for lora in self.unet.loras:
-                lora_info = context.models.load(lora.lora, context.util.get_queue_id())
+                lora_info = context.models.load(lora.lora)
                 assert isinstance(lora_info.model, LoRAModelRaw)
                 yield (lora_info.model, lora.weight)
                 del lora_info
             return
 
-        unet_info = context.models.load(self.unet.unet, context.util.get_queue_id())
+        unet_info = context.models.load(self.unet.unet)
         assert isinstance(unet_info.model, UNet2DConditionModel)
         with (
             ExitStack() as exit_stack,

invokeai/app/invocations/depth_anything.py

@@ -35,9 +35,7 @@ class DepthAnythingDepthEstimationInvocation(BaseInvocation, WithMetadata, WithB
         model_url = DEPTH_ANYTHING_MODELS[self.model_size]
         image = context.images.get_pil(self.image.image_name, "RGB")
 
-        loaded_model = context.models.load_remote_model(
-            model_url, context.util.get_queue_id(), DepthAnythingPipeline.load_model
-        )
+        loaded_model = context.models.load_remote_model(model_url, DepthAnythingPipeline.load_model)
 
         with loaded_model as depth_anything_detector:
             assert isinstance(depth_anything_detector, DepthAnythingPipeline)

invokeai/app/invocations/dw_openpose.py

@@ -29,10 +29,10 @@ class DWOpenposeDetectionInvocation(BaseInvocation, WithMetadata, WithBoard):
         onnx_pose_path = context.models.download_and_cache_model(DWOpenposeDetector2.get_model_url_pose())
 
         loaded_session_det = context.models.load_local_model(
-            onnx_det_path, context.util.get_queue_id(), DWOpenposeDetector2.create_onnx_inference_session
+            onnx_det_path, DWOpenposeDetector2.create_onnx_inference_session
         )
         loaded_session_pose = context.models.load_local_model(
-            onnx_pose_path, context.util.get_queue_id(), DWOpenposeDetector2.create_onnx_inference_session
+            onnx_pose_path, DWOpenposeDetector2.create_onnx_inference_session
         )
 
         with loaded_session_det as session_det, loaded_session_pose as session_pose:

invokeai/app/invocations/fields.py

@@ -41,7 +41,6 @@ class UIType(str, Enum, metaclass=MetaEnum):
     # region Model Field Types
     MainModel = "MainModelField"
     FluxMainModel = "FluxMainModelField"
-    SD3MainModel = "SD3MainModelField"
     SDXLMainModel = "SDXLMainModelField"
     SDXLRefinerModel = "SDXLRefinerModelField"
     ONNXModel = "ONNXModelField"
@@ -53,8 +52,6 @@ class UIType(str, Enum, metaclass=MetaEnum):
     T2IAdapterModel = "T2IAdapterModelField"
     T5EncoderModel = "T5EncoderModelField"
     CLIPEmbedModel = "CLIPEmbedModelField"
-    CLIPLEmbedModel = "CLIPLEmbedModelField"
-    CLIPGEmbedModel = "CLIPGEmbedModelField"
     SpandrelImageToImageModel = "SpandrelImageToImageModelField"
     # endregion
 
@@ -134,10 +131,8 @@ class FieldDescriptions:
     clip = "CLIP (tokenizer, text encoder, LoRAs) and skipped layer count"
     t5_encoder = "T5 tokenizer and text encoder"
     clip_embed_model = "CLIP Embed loader"
-    clip_g_model = "CLIP-G Embed loader"
     unet = "UNet (scheduler, LoRAs)"
     transformer = "Transformer"
-    mmditx = "MMDiTX"
     vae = "VAE"
     cond = "Conditioning tensor"
     controlnet_model = "ControlNet model to load"
@@ -145,7 +140,6 @@ class FieldDescriptions:
     lora_model = "LoRA model to load"
     main_model = "Main model (UNet, VAE, CLIP) to load"
     flux_model = "Flux model (Transformer) to load"
-    sd3_model = "SD3 model (MMDiTX) to load"
     sdxl_main_model = "SDXL Main model (UNet, VAE, CLIP1, CLIP2) to load"
     sdxl_refiner_model = "SDXL Refiner Main Modde (UNet, VAE, CLIP2) to load"
     onnx_main_model = "ONNX Main model (UNet, VAE, CLIP) to load"
@@ -252,12 +246,6 @@ class FluxConditioningField(BaseModel):
     conditioning_name: str = Field(description="The name of conditioning tensor")
 
 
-class SD3ConditioningField(BaseModel):
-    """A conditioning tensor primitive value"""
-
-    conditioning_name: str = Field(description="The name of conditioning tensor")
-
-
 class ConditioningField(BaseModel):
     """A conditioning tensor primitive value"""
 

invokeai/app/invocations/flux_denoise.py

@@ -1,19 +1,15 @@
 from contextlib import ExitStack
 from typing import Callable, Iterator, Optional, Tuple
 
-import numpy as np
-import numpy.typing as npt
 import torch
 import torchvision.transforms as tv_transforms
 from torchvision.transforms.functional import resize as tv_resize
-from transformers import CLIPImageProcessor, CLIPVisionModelWithProjection
 
 from invokeai.app.invocations.baseinvocation import BaseInvocation, Classification, invocation
 from invokeai.app.invocations.fields import (
     DenoiseMaskField,
     FieldDescriptions,
     FluxConditioningField,
-    ImageField,
     Input,
     InputField,
     LatentsField,
@@ -21,7 +17,6 @@ from invokeai.app.invocations.fields import (
     WithMetadata,
 )
 from invokeai.app.invocations.flux_controlnet import FluxControlNetField
-from invokeai.app.invocations.ip_adapter import IPAdapterField
 from invokeai.app.invocations.model import TransformerField, VAEField
 from invokeai.app.invocations.primitives import LatentsOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
@@ -31,8 +26,6 @@ from invokeai.backend.flux.denoise import denoise
 from invokeai.backend.flux.extensions.inpaint_extension import InpaintExtension
 from invokeai.backend.flux.extensions.instantx_controlnet_extension import InstantXControlNetExtension
 from invokeai.backend.flux.extensions.xlabs_controlnet_extension import XLabsControlNetExtension
-from invokeai.backend.flux.extensions.xlabs_ip_adapter_extension import XLabsIPAdapterExtension
-from invokeai.backend.flux.ip_adapter.xlabs_ip_adapter_flux import XlabsIpAdapterFlux
 from invokeai.backend.flux.model import Flux
 from invokeai.backend.flux.sampling_utils import (
     clip_timestep_schedule_fractional,
@@ -56,7 +49,7 @@ from invokeai.backend.util.devices import TorchDevice
     title="FLUX Denoise",
     tags=["image", "flux"],
     category="image",
-    version="3.2.0",
+    version="3.1.0",
     classification=Classification.Prototype,
 )
 class FluxDenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
@@ -89,24 +82,6 @@ class FluxDenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
     positive_text_conditioning: FluxConditioningField = InputField(
         description=FieldDescriptions.positive_cond, input=Input.Connection
     )
-    negative_text_conditioning: FluxConditioningField | None = InputField(
-        default=None,
-        description="Negative conditioning tensor. Can be None if cfg_scale is 1.0.",
-        input=Input.Connection,
-    )
-    cfg_scale: float | list[float] = InputField(default=1.0, description=FieldDescriptions.cfg_scale, title="CFG Scale")
-    cfg_scale_start_step: int = InputField(
-        default=0,
-        title="CFG Scale Start Step",
-        description="Index of the first step to apply cfg_scale. Negative indices count backwards from the "
-        + "the last step (e.g. a value of -1 refers to the final step).",
-    )
-    cfg_scale_end_step: int = InputField(
-        default=-1,
-        title="CFG Scale End Step",
-        description="Index of the last step to apply cfg_scale. Negative indices count backwards from the "
-        + "last step (e.g. a value of -1 refers to the final step).",
-    )
     width: int = InputField(default=1024, multiple_of=16, description="Width of the generated image.")
     height: int = InputField(default=1024, multiple_of=16, description="Height of the generated image.")
     num_steps: int = InputField(
@@ -121,15 +96,10 @@ class FluxDenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
         default=None, input=Input.Connection, description="ControlNet models."
     )
     controlnet_vae: VAEField | None = InputField(
-        default=None,
         description=FieldDescriptions.vae,
         input=Input.Connection,
     )
 
-    ip_adapter: IPAdapterField | list[IPAdapterField] | None = InputField(
-        description=FieldDescriptions.ip_adapter, title="IP-Adapter", default=None, input=Input.Connection
-    )
-
     @torch.no_grad()
     def invoke(self, context: InvocationContext) -> LatentsOutput:
         latents = self._run_diffusion(context)
@@ -138,19 +108,6 @@ class FluxDenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
         name = context.tensors.save(tensor=latents)
         return LatentsOutput.build(latents_name=name, latents=latents, seed=None)
 
-    def _load_text_conditioning(
-        self, context: InvocationContext, conditioning_name: str, dtype: torch.dtype
-    ) -> Tuple[torch.Tensor, torch.Tensor]:
-        # Load the conditioning data.
-        cond_data = context.conditioning.load(conditioning_name)
-        assert len(cond_data.conditionings) == 1
-        flux_conditioning = cond_data.conditionings[0]
-        assert isinstance(flux_conditioning, FLUXConditioningInfo)
-        flux_conditioning = flux_conditioning.to(dtype=dtype)
-        t5_embeddings = flux_conditioning.t5_embeds
-        clip_embeddings = flux_conditioning.clip_embeds
-        return t5_embeddings, clip_embeddings
-
     def _run_diffusion(
         self,
         context: InvocationContext,
@@ -158,15 +115,13 @@ class FluxDenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
         inference_dtype = torch.bfloat16
 
         # Load the conditioning data.
-        pos_t5_embeddings, pos_clip_embeddings = self._load_text_conditioning(
-            context, self.positive_text_conditioning.conditioning_name, inference_dtype
-        )
-        neg_t5_embeddings: torch.Tensor | None = None
-        neg_clip_embeddings: torch.Tensor | None = None
-        if self.negative_text_conditioning is not None:
-            neg_t5_embeddings, neg_clip_embeddings = self._load_text_conditioning(
-                context, self.negative_text_conditioning.conditioning_name, inference_dtype
-            )
+        cond_data = context.conditioning.load(self.positive_text_conditioning.conditioning_name)
+        assert len(cond_data.conditionings) == 1
+        flux_conditioning = cond_data.conditionings[0]
+        assert isinstance(flux_conditioning, FLUXConditioningInfo)
+        flux_conditioning = flux_conditioning.to(dtype=inference_dtype)
+        t5_embeddings = flux_conditioning.t5_embeds
+        clip_embeddings = flux_conditioning.clip_embeds
 
         # Load the input latents, if provided.
         init_latents = context.tensors.load(self.latents.latents_name) if self.latents else None
@@ -183,7 +138,7 @@ class FluxDenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
             seed=self.seed,
         )
 
-        transformer_info = context.models.load(self.transformer.transformer, context.util.get_queue_id())
+        transformer_info = context.models.load(self.transformer.transformer)
         is_schnell = "schnell" in transformer_info.config.config_path
 
         # Calculate the timestep schedule.
@@ -227,16 +182,8 @@ class FluxDenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
         b, _c, latent_h, latent_w = x.shape
         img_ids = generate_img_ids(h=latent_h, w=latent_w, batch_size=b, device=x.device, dtype=x.dtype)
 
-        pos_bs, pos_t5_seq_len, _ = pos_t5_embeddings.shape
-        pos_txt_ids = torch.zeros(
-            pos_bs, pos_t5_seq_len, 3, dtype=inference_dtype, device=TorchDevice.choose_torch_device()
-        )
-        neg_txt_ids: torch.Tensor | None = None
-        if neg_t5_embeddings is not None:
-            neg_bs, neg_t5_seq_len, _ = neg_t5_embeddings.shape
-            neg_txt_ids = torch.zeros(
-                neg_bs, neg_t5_seq_len, 3, dtype=inference_dtype, device=TorchDevice.choose_torch_device()
-            )
+        bs, t5_seq_len, _ = t5_embeddings.shape
+        txt_ids = torch.zeros(bs, t5_seq_len, 3, dtype=inference_dtype, device=TorchDevice.choose_torch_device())
 
         # Pack all latent tensors.
         init_latents = pack(init_latents) if init_latents is not None else None
@@ -257,21 +204,6 @@ class FluxDenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
                 noise=noise,
             )
 
-        # Compute the IP-Adapter image prompt clip embeddings.
-        # We do this before loading other models to minimize peak memory.
-        # TODO(ryand): We should really do this in a separate invocation to benefit from caching.
-        ip_adapter_fields = self._normalize_ip_adapter_fields()
-        pos_image_prompt_clip_embeds, neg_image_prompt_clip_embeds = self._prep_ip_adapter_image_prompt_clip_embeds(
-            ip_adapter_fields, context
-        )
-
-        cfg_scale = self.prep_cfg_scale(
-            cfg_scale=self.cfg_scale,
-            timesteps=timesteps,
-            cfg_scale_start_step=self.cfg_scale_start_step,
-            cfg_scale_end_step=self.cfg_scale_end_step,
-        )
-
         with ExitStack() as exit_stack:
             # Prepare ControlNet extensions.
             # Note: We do this before loading the transformer model to minimize peak memory (see implementation).
@@ -320,88 +252,23 @@ class FluxDenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
             else:
                 raise ValueError(f"Unsupported model format: {config.format}")
 
-            # Prepare IP-Adapter extensions.
-            pos_ip_adapter_extensions, neg_ip_adapter_extensions = self._prep_ip_adapter_extensions(
-                pos_image_prompt_clip_embeds=pos_image_prompt_clip_embeds,
-                neg_image_prompt_clip_embeds=neg_image_prompt_clip_embeds,
-                ip_adapter_fields=ip_adapter_fields,
-                context=context,
-                exit_stack=exit_stack,
-                dtype=inference_dtype,
-            )
-
             x = denoise(
                 model=transformer,
                 img=x,
                 img_ids=img_ids,
-                txt=pos_t5_embeddings,
-                txt_ids=pos_txt_ids,
-                vec=pos_clip_embeddings,
-                neg_txt=neg_t5_embeddings,
-                neg_txt_ids=neg_txt_ids,
-                neg_vec=neg_clip_embeddings,
+                txt=t5_embeddings,
+                txt_ids=txt_ids,
+                vec=clip_embeddings,
                 timesteps=timesteps,
                 step_callback=self._build_step_callback(context),
                 guidance=self.guidance,
-                cfg_scale=cfg_scale,
                 inpaint_extension=inpaint_extension,
                 controlnet_extensions=controlnet_extensions,
-                pos_ip_adapter_extensions=pos_ip_adapter_extensions,
-                neg_ip_adapter_extensions=neg_ip_adapter_extensions,
             )
 
         x = unpack(x.float(), self.height, self.width)
         return x
 
-    @classmethod
-    def prep_cfg_scale(
-        cls, cfg_scale: float | list[float], timesteps: list[float], cfg_scale_start_step: int, cfg_scale_end_step: int
-    ) -> list[float]:
-        """Prepare the cfg_scale schedule.
-
-        - Clips the cfg_scale schedule based on cfg_scale_start_step and cfg_scale_end_step.
-        - If cfg_scale is a list, then it is assumed to be a schedule and is returned as-is.
-        - If cfg_scale is a scalar, then a linear schedule is created from cfg_scale_start_step to cfg_scale_end_step.
-        """
-        # num_steps is the number of denoising steps, which is one less than the number of timesteps.
-        num_steps = len(timesteps) - 1
-
-        # Normalize cfg_scale to a list if it is a scalar.
-        cfg_scale_list: list[float]
-        if isinstance(cfg_scale, float):
-            cfg_scale_list = [cfg_scale] * num_steps
-        elif isinstance(cfg_scale, list):
-            cfg_scale_list = cfg_scale
-        else:
-            raise ValueError(f"Unsupported cfg_scale type: {type(cfg_scale)}")
-        assert len(cfg_scale_list) == num_steps
-
-        # Handle negative indices for cfg_scale_start_step and cfg_scale_end_step.
-        start_step_index = cfg_scale_start_step
-        if start_step_index < 0:
-            start_step_index = num_steps + start_step_index
-        end_step_index = cfg_scale_end_step
-        if end_step_index < 0:
-            end_step_index = num_steps + end_step_index
-
-        # Validate the start and end step indices.
-        if not (0 <= start_step_index < num_steps):
-            raise ValueError(f"Invalid cfg_scale_start_step. Out of range: {cfg_scale_start_step}.")
-        if not (0 <= end_step_index < num_steps):
-            raise ValueError(f"Invalid cfg_scale_end_step. Out of range: {cfg_scale_end_step}.")
-        if start_step_index > end_step_index:
-            raise ValueError(
-                f"cfg_scale_start_step ({cfg_scale_start_step}) must be before cfg_scale_end_step "
-                + f"({cfg_scale_end_step})."
-            )
-
-        # Set values outside the start and end step indices to 1.0. This is equivalent to disabling cfg_scale for those
-        # steps.
-        clipped_cfg_scale = [1.0] * num_steps
-        clipped_cfg_scale[start_step_index : end_step_index + 1] = cfg_scale_list[start_step_index : end_step_index + 1]
-
-        return clipped_cfg_scale
-
     def _prep_inpaint_mask(self, context: InvocationContext, latents: torch.Tensor) -> torch.Tensor | None:
         """Prepare the inpaint mask.
 
@@ -468,9 +335,7 @@ class FluxDenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
         # minimize peak memory.
 
         # First, load the ControlNet models so that we can determine the ControlNet types.
-        controlnet_models = [
-            context.models.load(controlnet.control_model, context.util.get_queue_id()) for controlnet in controlnets
-        ]
+        controlnet_models = [context.models.load(controlnet.control_model) for controlnet in controlnets]
 
         # Calculate the controlnet conditioning tensors.
         # We do this before loading the ControlNet models because it may require running the VAE, and we are trying to
@@ -481,7 +346,7 @@ class FluxDenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
             if isinstance(controlnet_model.model, InstantXControlNetFlux):
                 if self.controlnet_vae is None:
                     raise ValueError("A ControlNet VAE is required when using an InstantX FLUX ControlNet.")
-                vae_info = context.models.load(self.controlnet_vae.vae, context.util.get_queue_id())
+                vae_info = context.models.load(self.controlnet_vae.vae)
                 controlnet_conds.append(
                     InstantXControlNetExtension.prepare_controlnet_cond(
                         controlnet_image=image,
@@ -543,119 +408,9 @@ class FluxDenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
 
         return controlnet_extensions
 
-    def _normalize_ip_adapter_fields(self) -> list[IPAdapterField]:
-        if self.ip_adapter is None:
-            return []
-        elif isinstance(self.ip_adapter, IPAdapterField):
-            return [self.ip_adapter]
-        elif isinstance(self.ip_adapter, list):
-            return self.ip_adapter
-        else:
-            raise ValueError(f"Unsupported IP-Adapter type: {type(self.ip_adapter)}")
-
-    def _prep_ip_adapter_image_prompt_clip_embeds(
-        self,
-        ip_adapter_fields: list[IPAdapterField],
-        context: InvocationContext,
-    ) -> tuple[list[torch.Tensor], list[torch.Tensor]]:
-        """Run the IPAdapter CLIPVisionModel, returning image prompt embeddings."""
-        clip_image_processor = CLIPImageProcessor()
-
-        pos_image_prompt_clip_embeds: list[torch.Tensor] = []
-        neg_image_prompt_clip_embeds: list[torch.Tensor] = []
-        for ip_adapter_field in ip_adapter_fields:
-            # `ip_adapter_field.image` could be a list or a single ImageField. Normalize to a list here.
-            ipa_image_fields: list[ImageField]
-            if isinstance(ip_adapter_field.image, ImageField):
-                ipa_image_fields = [ip_adapter_field.image]
-            elif isinstance(ip_adapter_field.image, list):
-                ipa_image_fields = ip_adapter_field.image
-            else:
-                raise ValueError(f"Unsupported IP-Adapter image type: {type(ip_adapter_field.image)}")
-
-            if len(ipa_image_fields) != 1:
-                raise ValueError(
-                    f"FLUX IP-Adapter only supports a single image prompt (received {len(ipa_image_fields)})."
-                )
-
-            ipa_images = [context.images.get_pil(image.image_name, mode="RGB") for image in ipa_image_fields]
-
-            pos_images: list[npt.NDArray[np.uint8]] = []
-            neg_images: list[npt.NDArray[np.uint8]] = []
-            for ipa_image in ipa_images:
-                assert ipa_image.mode == "RGB"
-                pos_image = np.array(ipa_image)
-                # We use a black image as the negative image prompt for parity with
-                # https://github.com/XLabs-AI/x-flux-comfyui/blob/45c834727dd2141aebc505ae4b01f193a8414e38/nodes.py#L592-L593
-                # An alternative scheme would be to apply zeros_like() after calling the clip_image_processor.
-                neg_image = np.zeros_like(pos_image)
-                pos_images.append(pos_image)
-                neg_images.append(neg_image)
-
-            with context.models.load(
-                ip_adapter_field.image_encoder_model, context.util.get_queue_id()
-            ) as image_encoder_model:
-                assert isinstance(image_encoder_model, CLIPVisionModelWithProjection)
-
-                clip_image: torch.Tensor = clip_image_processor(images=pos_images, return_tensors="pt").pixel_values
-                clip_image = clip_image.to(device=image_encoder_model.device, dtype=image_encoder_model.dtype)
-                pos_clip_image_embeds = image_encoder_model(clip_image).image_embeds
-
-                clip_image = clip_image_processor(images=neg_images, return_tensors="pt").pixel_values
-                clip_image = clip_image.to(device=image_encoder_model.device, dtype=image_encoder_model.dtype)
-                neg_clip_image_embeds = image_encoder_model(clip_image).image_embeds
-
-            pos_image_prompt_clip_embeds.append(pos_clip_image_embeds)
-            neg_image_prompt_clip_embeds.append(neg_clip_image_embeds)
-
-        return pos_image_prompt_clip_embeds, neg_image_prompt_clip_embeds
-
-    def _prep_ip_adapter_extensions(
-        self,
-        ip_adapter_fields: list[IPAdapterField],
-        pos_image_prompt_clip_embeds: list[torch.Tensor],
-        neg_image_prompt_clip_embeds: list[torch.Tensor],
-        context: InvocationContext,
-        exit_stack: ExitStack,
-        dtype: torch.dtype,
-    ) -> tuple[list[XLabsIPAdapterExtension], list[XLabsIPAdapterExtension]]:
-        pos_ip_adapter_extensions: list[XLabsIPAdapterExtension] = []
-        neg_ip_adapter_extensions: list[XLabsIPAdapterExtension] = []
-        for ip_adapter_field, pos_image_prompt_clip_embed, neg_image_prompt_clip_embed in zip(
-            ip_adapter_fields, pos_image_prompt_clip_embeds, neg_image_prompt_clip_embeds, strict=True
-        ):
-            ip_adapter_model = exit_stack.enter_context(
-                context.models.load(ip_adapter_field.ip_adapter_model, context.util.get_queue_id())
-            )
-            assert isinstance(ip_adapter_model, XlabsIpAdapterFlux)
-            ip_adapter_model = ip_adapter_model.to(dtype=dtype)
-            if ip_adapter_field.mask is not None:
-                raise ValueError("IP-Adapter masks are not yet supported in Flux.")
-            ip_adapter_extension = XLabsIPAdapterExtension(
-                model=ip_adapter_model,
-                image_prompt_clip_embed=pos_image_prompt_clip_embed,
-                weight=ip_adapter_field.weight,
-                begin_step_percent=ip_adapter_field.begin_step_percent,
-                end_step_percent=ip_adapter_field.end_step_percent,
-            )
-            ip_adapter_extension.run_image_proj(dtype=dtype)
-            pos_ip_adapter_extensions.append(ip_adapter_extension)
-
-            ip_adapter_extension = XLabsIPAdapterExtension(
-                model=ip_adapter_model,
-                image_prompt_clip_embed=neg_image_prompt_clip_embed,
-                weight=ip_adapter_field.weight,
-                begin_step_percent=ip_adapter_field.begin_step_percent,
-                end_step_percent=ip_adapter_field.end_step_percent,
-            )
-            ip_adapter_extension.run_image_proj(dtype=dtype)
-            neg_ip_adapter_extensions.append(ip_adapter_extension)
-
-        return pos_ip_adapter_extensions, neg_ip_adapter_extensions
-
     def _lora_iterator(self, context: InvocationContext) -> Iterator[Tuple[LoRAModelRaw, float]]:
         for lora in self.transformer.loras:
-            lora_info = context.models.load(lora.lora, context.util.get_queue_id())
+            lora_info = context.models.load(lora.lora)
             assert isinstance(lora_info.model, LoRAModelRaw)
             yield (lora_info.model, lora.weight)
             del lora_info

invokeai/app/invocations/flux_ip_adapter.py

@@ -1,89 +0,0 @@
-from builtins import float
-from typing import List, Literal, Union
-
-from pydantic import field_validator, model_validator
-from typing_extensions import Self
-
-from invokeai.app.invocations.baseinvocation import BaseInvocation, Classification, invocation
-from invokeai.app.invocations.fields import InputField, UIType
-from invokeai.app.invocations.ip_adapter import (
-    CLIP_VISION_MODEL_MAP,
-    IPAdapterField,
-    IPAdapterInvocation,
-    IPAdapterOutput,
-)
-from invokeai.app.invocations.model import ModelIdentifierField
-from invokeai.app.invocations.primitives import ImageField
-from invokeai.app.invocations.util import validate_begin_end_step, validate_weights
-from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.model_manager.config import (
-    IPAdapterCheckpointConfig,
-    IPAdapterInvokeAIConfig,
-)
-
-
-@invocation(
-    "flux_ip_adapter",
-    title="FLUX IP-Adapter",
-    tags=["ip_adapter", "control"],
-    category="ip_adapter",
-    version="1.0.0",
-    classification=Classification.Prototype,
-)
-class FluxIPAdapterInvocation(BaseInvocation):
-    """Collects FLUX IP-Adapter info to pass to other nodes."""
-
-    # FLUXIPAdapterInvocation is based closely on IPAdapterInvocation, but with some unsupported features removed.
-
-    image: ImageField = InputField(description="The IP-Adapter image prompt(s).")
-    ip_adapter_model: ModelIdentifierField = InputField(
-        description="The IP-Adapter model.", title="IP-Adapter Model", ui_type=UIType.IPAdapterModel
-    )
-    # Currently, the only known ViT model used by FLUX IP-Adapters is ViT-L.
-    clip_vision_model: Literal["ViT-L"] = InputField(description="CLIP Vision model to use.", default="ViT-L")
-    weight: Union[float, List[float]] = InputField(
-        default=1, description="The weight given to the IP-Adapter", title="Weight"
-    )
-    begin_step_percent: float = InputField(
-        default=0, ge=0, le=1, description="When the IP-Adapter is first applied (% of total steps)"
-    )
-    end_step_percent: float = InputField(
-        default=1, ge=0, le=1, description="When the IP-Adapter is last applied (% of total steps)"
-    )
-
-    @field_validator("weight")
-    @classmethod
-    def validate_ip_adapter_weight(cls, v: float) -> float:
-        validate_weights(v)
-        return v
-
-    @model_validator(mode="after")
-    def validate_begin_end_step_percent(self) -> Self:
-        validate_begin_end_step(self.begin_step_percent, self.end_step_percent)
-        return self
-
-    def invoke(self, context: InvocationContext) -> IPAdapterOutput:
-        # Lookup the CLIP Vision encoder that is intended to be used with the IP-Adapter model.
-        ip_adapter_info = context.models.get_config(self.ip_adapter_model.key)
-        assert isinstance(ip_adapter_info, (IPAdapterInvokeAIConfig, IPAdapterCheckpointConfig))
-
-        # Note: There is a IPAdapterInvokeAIConfig.image_encoder_model_id field, but it isn't trustworthy.
-        image_encoder_starter_model = CLIP_VISION_MODEL_MAP[self.clip_vision_model]
-        image_encoder_model_id = image_encoder_starter_model.source
-        image_encoder_model_name = image_encoder_starter_model.name
-        image_encoder_model = IPAdapterInvocation.get_clip_image_encoder(
-            context, image_encoder_model_id, image_encoder_model_name
-        )
-
-        return IPAdapterOutput(
-            ip_adapter=IPAdapterField(
-                image=self.image,
-                ip_adapter_model=self.ip_adapter_model,
-                image_encoder_model=ModelIdentifierField.from_config(image_encoder_model),
-                weight=self.weight,
-                target_blocks=[],  # target_blocks is currently unused for FLUX IP-Adapters.
-                begin_step_percent=self.begin_step_percent,
-                end_step_percent=self.end_step_percent,
-                mask=None,  # mask is currently unused for FLUX IP-Adapters.
-            ),
-        )

invokeai/app/invocations/flux_model_loader.py

@@ -1,89 +0,0 @@
-from typing import Literal
-
-from invokeai.app.invocations.baseinvocation import (
-    BaseInvocation,
-    BaseInvocationOutput,
-    Classification,
-    invocation,
-    invocation_output,
-)
-from invokeai.app.invocations.fields import FieldDescriptions, Input, InputField, OutputField, UIType
-from invokeai.app.invocations.model import CLIPField, ModelIdentifierField, T5EncoderField, TransformerField, VAEField
-from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.flux.util import max_seq_lengths
-from invokeai.backend.model_manager.config import (
-    CheckpointConfigBase,
-    SubModelType,
-)
-
-
-@invocation_output("flux_model_loader_output")
-class FluxModelLoaderOutput(BaseInvocationOutput):
-    """Flux base model loader output"""
-
-    transformer: TransformerField = OutputField(description=FieldDescriptions.transformer, title="Transformer")
-    clip: CLIPField = OutputField(description=FieldDescriptions.clip, title="CLIP")
-    t5_encoder: T5EncoderField = OutputField(description=FieldDescriptions.t5_encoder, title="T5 Encoder")
-    vae: VAEField = OutputField(description=FieldDescriptions.vae, title="VAE")
-    max_seq_len: Literal[256, 512] = OutputField(
-        description="The max sequence length to used for the T5 encoder. (256 for schnell transformer, 512 for dev transformer)",
-        title="Max Seq Length",
-    )
-
-
-@invocation(
-    "flux_model_loader",
-    title="Flux Main Model",
-    tags=["model", "flux"],
-    category="model",
-    version="1.0.4",
-    classification=Classification.Prototype,
-)
-class FluxModelLoaderInvocation(BaseInvocation):
-    """Loads a flux base model, outputting its submodels."""
-
-    model: ModelIdentifierField = InputField(
-        description=FieldDescriptions.flux_model,
-        ui_type=UIType.FluxMainModel,
-        input=Input.Direct,
-    )
-
-    t5_encoder_model: ModelIdentifierField = InputField(
-        description=FieldDescriptions.t5_encoder, ui_type=UIType.T5EncoderModel, input=Input.Direct, title="T5 Encoder"
-    )
-
-    clip_embed_model: ModelIdentifierField = InputField(
-        description=FieldDescriptions.clip_embed_model,
-        ui_type=UIType.CLIPEmbedModel,
-        input=Input.Direct,
-        title="CLIP Embed",
-    )
-
-    vae_model: ModelIdentifierField = InputField(
-        description=FieldDescriptions.vae_model, ui_type=UIType.FluxVAEModel, title="VAE"
-    )
-
-    def invoke(self, context: InvocationContext) -> FluxModelLoaderOutput:
-        for key in [self.model.key, self.t5_encoder_model.key, self.clip_embed_model.key, self.vae_model.key]:
-            if not context.models.exists(key):
-                raise ValueError(f"Unknown model: {key}")
-
-        transformer = self.model.model_copy(update={"submodel_type": SubModelType.Transformer})
-        vae = self.vae_model.model_copy(update={"submodel_type": SubModelType.VAE})
-
-        tokenizer = self.clip_embed_model.model_copy(update={"submodel_type": SubModelType.Tokenizer})
-        clip_encoder = self.clip_embed_model.model_copy(update={"submodel_type": SubModelType.TextEncoder})
-
-        tokenizer2 = self.t5_encoder_model.model_copy(update={"submodel_type": SubModelType.Tokenizer2})
-        t5_encoder = self.t5_encoder_model.model_copy(update={"submodel_type": SubModelType.TextEncoder2})
-
-        transformer_config = context.models.get_config(transformer)
-        assert isinstance(transformer_config, CheckpointConfigBase)
-
-        return FluxModelLoaderOutput(
-            transformer=TransformerField(transformer=transformer, loras=[]),
-            clip=CLIPField(tokenizer=tokenizer, text_encoder=clip_encoder, loras=[], skipped_layers=0),
-            t5_encoder=T5EncoderField(tokenizer=tokenizer2, text_encoder=t5_encoder),
-            vae=VAEField(vae=vae),
-            max_seq_len=max_seq_lengths[transformer_config.config_path],
-        )

invokeai/app/invocations/flux_text_encoder.py

@@ -57,8 +57,8 @@ class FluxTextEncoderInvocation(BaseInvocation):
         return FluxConditioningOutput.build(conditioning_name)
 
     def _t5_encode(self, context: InvocationContext) -> torch.Tensor:
-        t5_tokenizer_info = context.models.load(self.t5_encoder.tokenizer, context.util.get_queue_id())
-        t5_text_encoder_info = context.models.load(self.t5_encoder.text_encoder, context.util.get_queue_id())
+        t5_tokenizer_info = context.models.load(self.t5_encoder.tokenizer)
+        t5_text_encoder_info = context.models.load(self.t5_encoder.text_encoder)
 
         prompt = [self.prompt]
 
@@ -77,8 +77,8 @@ class FluxTextEncoderInvocation(BaseInvocation):
         return prompt_embeds
 
     def _clip_encode(self, context: InvocationContext) -> torch.Tensor:
-        clip_tokenizer_info = context.models.load(self.clip.tokenizer, context.util.get_queue_id())
-        clip_text_encoder_info = context.models.load(self.clip.text_encoder, context.util.get_queue_id())
+        clip_tokenizer_info = context.models.load(self.clip.tokenizer)
+        clip_text_encoder_info = context.models.load(self.clip.text_encoder)
 
         prompt = [self.prompt]
 
@@ -118,7 +118,7 @@ class FluxTextEncoderInvocation(BaseInvocation):
 
     def _clip_lora_iterator(self, context: InvocationContext) -> Iterator[Tuple[LoRAModelRaw, float]]:
         for lora in self.clip.loras:
-            lora_info = context.models.load(lora.lora, context.util.get_queue_id())
+            lora_info = context.models.load(lora.lora)
             assert isinstance(lora_info.model, LoRAModelRaw)
             yield (lora_info.model, lora.weight)
             del lora_info

invokeai/app/invocations/flux_vae_decode.py

@@ -52,7 +52,7 @@ class FluxVaeDecodeInvocation(BaseInvocation, WithMetadata, WithBoard):
     @torch.no_grad()
     def invoke(self, context: InvocationContext) -> ImageOutput:
         latents = context.tensors.load(self.latents.latents_name)
-        vae_info = context.models.load(self.vae.vae, context.util.get_queue_id())
+        vae_info = context.models.load(self.vae.vae)
         image = self._vae_decode(vae_info=vae_info, latents=latents)
 
         TorchDevice.empty_cache()

invokeai/app/invocations/flux_vae_encode.py

@@ -54,7 +54,7 @@ class FluxVaeEncodeInvocation(BaseInvocation):
     def invoke(self, context: InvocationContext) -> LatentsOutput:
         image = context.images.get_pil(self.image.image_name)
 
-        vae_info = context.models.load(self.vae.vae, context.util.get_queue_id())
+        vae_info = context.models.load(self.vae.vae)
 
         image_tensor = image_resized_to_grid_as_tensor(image.convert("RGB"))
         if image_tensor.dim() == 3:

invokeai/app/invocations/grounding_dino.py

@@ -94,9 +94,7 @@ class GroundingDinoInvocation(BaseInvocation):
         labels = [label if label.endswith(".") else label + "." for label in labels]
 
         with context.models.load_remote_model(
-            source=GROUNDING_DINO_MODEL_IDS[self.model],
-            queue_id=context.util.get_queue_id(),
-            loader=GroundingDinoInvocation._load_grounding_dino,
+            source=GROUNDING_DINO_MODEL_IDS[self.model], loader=GroundingDinoInvocation._load_grounding_dino
         ) as detector:
             assert isinstance(detector, GroundingDinoPipeline)
             return detector.detect(image=image, candidate_labels=labels, threshold=threshold)

invokeai/app/invocations/hed.py

@@ -22,9 +22,7 @@ class HEDEdgeDetectionInvocation(BaseInvocation, WithMetadata, WithBoard):
 
     def invoke(self, context: InvocationContext) -> ImageOutput:
         image = context.images.get_pil(self.image.image_name, "RGB")
-        loaded_model = context.models.load_remote_model(
-            HEDEdgeDetector.get_model_url(), context.util.get_queue_id(), HEDEdgeDetector.load_model
-        )
+        loaded_model = context.models.load_remote_model(HEDEdgeDetector.get_model_url(), HEDEdgeDetector.load_model)
 
         with loaded_model as model:
             assert isinstance(model, ControlNetHED_Apache2)

invokeai/app/invocations/image_to_latents.py

@@ -111,7 +111,7 @@ class ImageToLatentsInvocation(BaseInvocation):
     def invoke(self, context: InvocationContext) -> LatentsOutput:
         image = context.images.get_pil(self.image.image_name)
 
-        vae_info = context.models.load(self.vae.vae, context.util.get_queue_id())
+        vae_info = context.models.load(self.vae.vae)
 
         image_tensor = image_resized_to_grid_as_tensor(image.convert("RGB"))
         if image_tensor.dim() == 3:

invokeai/app/invocations/infill.py

@@ -36,7 +36,7 @@ class InfillImageProcessorInvocation(BaseInvocation, WithMetadata, WithBoard):
     image: ImageField = InputField(description="The image to process")
 
     @abstractmethod
-    def infill(self, image: Image.Image, queue_id: str) -> Image.Image:
+    def infill(self, image: Image.Image) -> Image.Image:
         """Infill the image with the specified method"""
         pass
 
@@ -56,7 +56,7 @@ class InfillImageProcessorInvocation(BaseInvocation, WithMetadata, WithBoard):
             return ImageOutput.build(context.images.get_dto(self.image.image_name))
 
         # Perform Infill action
-        infilled_image = self.infill(input_image, context.util.get_queue_id())
+        infilled_image = self.infill(input_image)
 
         # Create ImageDTO for Infilled Image
         infilled_image_dto = context.images.save(image=infilled_image)
@@ -74,7 +74,7 @@ class InfillColorInvocation(InfillImageProcessorInvocation):
         description="The color to use to infill",
     )
 
-    def infill(self, image: Image.Image, queue_id: str):
+    def infill(self, image: Image.Image):
         solid_bg = Image.new("RGBA", image.size, self.color.tuple())
         infilled = Image.alpha_composite(solid_bg, image.convert("RGBA"))
         infilled.paste(image, (0, 0), image.split()[-1])
@@ -93,7 +93,7 @@ class InfillTileInvocation(InfillImageProcessorInvocation):
         description="The seed to use for tile generation (omit for random)",
     )
 
-    def infill(self, image: Image.Image, queue_id: str):
+    def infill(self, image: Image.Image):
         output = infill_tile(image, seed=self.seed, tile_size=self.tile_size)
         return output.infilled
 
@@ -107,7 +107,7 @@ class InfillPatchMatchInvocation(InfillImageProcessorInvocation):
     downscale: float = InputField(default=2.0, gt=0, description="Run patchmatch on downscaled image to speedup infill")
     resample_mode: PIL_RESAMPLING_MODES = InputField(default="bicubic", description="The resampling mode")
 
-    def infill(self, image: Image.Image, queue_id: str):
+    def infill(self, image: Image.Image):
         resample_mode = PIL_RESAMPLING_MAP[self.resample_mode]
 
         width = int(image.width / self.downscale)
@@ -131,10 +131,9 @@ class InfillPatchMatchInvocation(InfillImageProcessorInvocation):
 class LaMaInfillInvocation(InfillImageProcessorInvocation):
     """Infills transparent areas of an image using the LaMa model"""
 
-    def infill(self, image: Image.Image, queue_id: str):
+    def infill(self, image: Image.Image):
         with self._context.models.load_remote_model(
             source="https://github.com/Sanster/models/releases/download/add_big_lama/big-lama.pt",
-            queue_id=queue_id,
             loader=LaMA.load_jit_model,
         ) as model:
             lama = LaMA(model)
@@ -145,7 +144,7 @@ class LaMaInfillInvocation(InfillImageProcessorInvocation):
 class CV2InfillInvocation(InfillImageProcessorInvocation):
     """Infills transparent areas of an image using OpenCV Inpainting"""
 
-    def infill(self, image: Image.Image, queue_id: str):
+    def infill(self, image: Image.Image):
         return cv2_inpaint(image)
 
 
@@ -167,5 +166,5 @@ class MosaicInfillInvocation(InfillImageProcessorInvocation):
         description="The max threshold for color",
     )
 
-    def infill(self, image: Image.Image, queue_id: str):
+    def infill(self, image: Image.Image):
         return infill_mosaic(image, (self.tile_width, self.tile_height), self.min_color.tuple(), self.max_color.tuple())

invokeai/app/invocations/ip_adapter.py

@@ -9,7 +9,6 @@ from invokeai.app.invocations.fields import FieldDescriptions, InputField, Outpu
 from invokeai.app.invocations.model import ModelIdentifierField
 from invokeai.app.invocations.primitives import ImageField
 from invokeai.app.invocations.util import validate_begin_end_step, validate_weights
-from invokeai.app.services.model_records.model_records_base import ModelRecordChanges
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.backend.model_manager.config import (
     AnyModelConfig,
@@ -18,12 +17,6 @@ from invokeai.backend.model_manager.config import (
     IPAdapterInvokeAIConfig,
     ModelType,
 )
-from invokeai.backend.model_manager.starter_models import (
-    StarterModel,
-    clip_vit_l_image_encoder,
-    ip_adapter_sd_image_encoder,
-    ip_adapter_sdxl_image_encoder,
-)
 
 
 class IPAdapterField(BaseModel):
@@ -62,14 +55,10 @@ class IPAdapterOutput(BaseInvocationOutput):
     ip_adapter: IPAdapterField = OutputField(description=FieldDescriptions.ip_adapter, title="IP-Adapter")
 
 
-CLIP_VISION_MODEL_MAP: dict[Literal["ViT-L", "ViT-H", "ViT-G"], StarterModel] = {
-    "ViT-L": clip_vit_l_image_encoder,
-    "ViT-H": ip_adapter_sd_image_encoder,
-    "ViT-G": ip_adapter_sdxl_image_encoder,
-}
+CLIP_VISION_MODEL_MAP = {"ViT-H": "ip_adapter_sd_image_encoder", "ViT-G": "ip_adapter_sdxl_image_encoder"}
 
 
-@invocation("ip_adapter", title="IP-Adapter", tags=["ip_adapter", "control"], category="ip_adapter", version="1.5.0")
+@invocation("ip_adapter", title="IP-Adapter", tags=["ip_adapter", "control"], category="ip_adapter", version="1.4.1")
 class IPAdapterInvocation(BaseInvocation):
     """Collects IP-Adapter info to pass to other nodes."""
 
@@ -81,7 +70,7 @@ class IPAdapterInvocation(BaseInvocation):
         ui_order=-1,
         ui_type=UIType.IPAdapterModel,
     )
-    clip_vision_model: Literal["ViT-H", "ViT-G", "ViT-L"] = InputField(
+    clip_vision_model: Literal["ViT-H", "ViT-G"] = InputField(
         description="CLIP Vision model to use. Overrides model settings. Mandatory for checkpoint models.",
         default="ViT-H",
         ui_order=2,
@@ -122,11 +111,9 @@ class IPAdapterInvocation(BaseInvocation):
             image_encoder_model_id = ip_adapter_info.image_encoder_model_id
             image_encoder_model_name = image_encoder_model_id.split("/")[-1].strip()
         else:
-            image_encoder_starter_model = CLIP_VISION_MODEL_MAP[self.clip_vision_model]
-            image_encoder_model_id = image_encoder_starter_model.source
-            image_encoder_model_name = image_encoder_starter_model.name
+            image_encoder_model_name = CLIP_VISION_MODEL_MAP[self.clip_vision_model]
 
-        image_encoder_model = self.get_clip_image_encoder(context, image_encoder_model_id, image_encoder_model_name)
+        image_encoder_model = self._get_image_encoder(context, image_encoder_model_name)
 
         if self.method == "style":
             if ip_adapter_info.base == "sd-1":
@@ -160,10 +147,7 @@ class IPAdapterInvocation(BaseInvocation):
             ),
         )
 
-    @classmethod
-    def get_clip_image_encoder(
-        cls, context: InvocationContext, image_encoder_model_id: str, image_encoder_model_name: str
-    ) -> AnyModelConfig:
+    def _get_image_encoder(self, context: InvocationContext, image_encoder_model_name: str) -> AnyModelConfig:
         image_encoder_models = context.models.search_by_attrs(
             name=image_encoder_model_name, base=BaseModelType.Any, type=ModelType.CLIPVision
         )
@@ -175,11 +159,7 @@ class IPAdapterInvocation(BaseInvocation):
             )
 
             installer = context._services.model_manager.install
-            # Note: We hard-code the type to CLIPVision here because if the model contains both a CLIPVision and a
-            # CLIPText model, the probe may treat it as a CLIPText model.
-            job = installer.heuristic_import(
-                image_encoder_model_id, ModelRecordChanges(name=image_encoder_model_name, type=ModelType.CLIPVision)
-            )
+            job = installer.heuristic_import(f"InvokeAI/{image_encoder_model_name}")
             installer.wait_for_job(job, timeout=600)  # Wait for up to 10 minutes
             image_encoder_models = context.models.search_by_attrs(
                 name=image_encoder_model_name, base=BaseModelType.Any, type=ModelType.CLIPVision

invokeai/app/invocations/latents_to_image.py

@@ -57,7 +57,7 @@ class LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
     def invoke(self, context: InvocationContext) -> ImageOutput:
         latents = context.tensors.load(self.latents.latents_name)
 
-        vae_info = context.models.load(self.vae.vae, context.util.get_queue_id())
+        vae_info = context.models.load(self.vae.vae)
         assert isinstance(vae_info.model, (AutoencoderKL, AutoencoderTiny))
         with SeamlessExt.static_patch_model(vae_info.model, self.vae.seamless_axes), vae_info as vae:
             assert isinstance(vae, (AutoencoderKL, AutoencoderTiny))

invokeai/app/invocations/lineart.py

@@ -23,9 +23,7 @@ class LineartEdgeDetectionInvocation(BaseInvocation, WithMetadata, WithBoard):
     def invoke(self, context: InvocationContext) -> ImageOutput:
         image = context.images.get_pil(self.image.image_name, "RGB")
         model_url = LineartEdgeDetector.get_model_url(self.coarse)
-        loaded_model = context.models.load_remote_model(
-            model_url, context.util.get_queue_id(), LineartEdgeDetector.load_model
-        )
+        loaded_model = context.models.load_remote_model(model_url, LineartEdgeDetector.load_model)
 
         with loaded_model as model:
             assert isinstance(model, Generator)

invokeai/app/invocations/lineart_anime.py

@@ -20,9 +20,7 @@ class LineartAnimeEdgeDetectionInvocation(BaseInvocation, WithMetadata, WithBoar
     def invoke(self, context: InvocationContext) -> ImageOutput:
         image = context.images.get_pil(self.image.image_name, "RGB")
         model_url = LineartAnimeEdgeDetector.get_model_url()
-        loaded_model = context.models.load_remote_model(
-            model_url, context.util.get_queue_id(), LineartAnimeEdgeDetector.load_model
-        )
+        loaded_model = context.models.load_remote_model(model_url, LineartAnimeEdgeDetector.load_model)
 
         with loaded_model as model:
             assert isinstance(model, UnetGenerator)

invokeai/app/invocations/mask.py

@@ -5,7 +5,6 @@ from PIL import Image
 from invokeai.app.invocations.baseinvocation import BaseInvocation, Classification, InvocationContext, invocation
 from invokeai.app.invocations.fields import ImageField, InputField, TensorField, WithBoard, WithMetadata
 from invokeai.app.invocations.primitives import ImageOutput, MaskOutput
-from invokeai.backend.image_util.util import pil_to_np
 
 
 @invocation(
@@ -149,55 +148,3 @@ class MaskTensorToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
         mask_pil = Image.fromarray(mask_np, mode="L")
         image_dto = context.images.save(image=mask_pil)
         return ImageOutput.build(image_dto)
-
-
-@invocation(
-    "apply_tensor_mask_to_image",
-    title="Apply Tensor Mask to Image",
-    tags=["mask"],
-    category="mask",
-    version="1.0.0",
-)
-class ApplyMaskTensorToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
-    """Applies a tensor mask to an image.
-
-    The image is converted to RGBA and the mask is applied to the alpha channel."""
-
-    mask: TensorField = InputField(description="The mask tensor to apply.")
-    image: ImageField = InputField(description="The image to apply the mask to.")
-    invert: bool = InputField(default=False, description="Whether to invert the mask.")
-
-    def invoke(self, context: InvocationContext) -> ImageOutput:
-        image = context.images.get_pil(self.image.image_name, mode="RGBA")
-        mask = context.tensors.load(self.mask.tensor_name)
-
-        # Squeeze the channel dimension if it exists.
-        if mask.dim() == 3:
-            mask = mask.squeeze(0)
-
-        # Ensure that the mask is binary.
-        if mask.dtype != torch.bool:
-            mask = mask > 0.5
-        mask_np = (mask.float() * 255).byte().cpu().numpy().astype(np.uint8)
-
-        if self.invert:
-            mask_np = 255 - mask_np
-
-        # Apply the mask only to the alpha channel where the original alpha is non-zero. This preserves the original
-        # image's transparency - else the transparent regions would end up as opaque black.
-
-        # Separate the image into R, G, B, and A channels
-        image_np = pil_to_np(image)
-        r, g, b, a = np.split(image_np, 4, axis=-1)
-
-        # Apply the mask to the alpha channel
-        new_alpha = np.where(a.squeeze() > 0, mask_np, a.squeeze())
-
-        # Stack the RGB channels with the modified alpha
-        masked_image_np = np.dstack([r.squeeze(), g.squeeze(), b.squeeze(), new_alpha])
-
-        # Convert back to an image (RGBA)
-        masked_image = Image.fromarray(masked_image_np.astype(np.uint8), "RGBA")
-        image_dto = context.images.save(image=masked_image)
-
-        return ImageOutput.build(image_dto)

invokeai/app/invocations/metadata.py

@@ -40,7 +40,7 @@ class IPAdapterMetadataField(BaseModel):
 
     image: ImageField = Field(description="The IP-Adapter image prompt.")
     ip_adapter_model: ModelIdentifierField = Field(description="The IP-Adapter model.")
-    clip_vision_model: Literal["ViT-L", "ViT-H", "ViT-G"] = Field(description="The CLIP Vision model")
+    clip_vision_model: Literal["ViT-H", "ViT-G"] = Field(description="The CLIP Vision model")
     method: Literal["full", "style", "composition"] = Field(description="Method to apply IP Weights with")
     weight: Union[float, list[float]] = Field(description="The weight given to the IP-Adapter")
     begin_step_percent: float = Field(description="When the IP-Adapter is first applied (% of total steps)")

invokeai/app/invocations/mlsd.py

@@ -28,9 +28,7 @@ class MLSDDetectionInvocation(BaseInvocation, WithMetadata, WithBoard):
 
     def invoke(self, context: InvocationContext) -> ImageOutput:
         image = context.images.get_pil(self.image.image_name, "RGB")
-        loaded_model = context.models.load_remote_model(
-            MLSDDetector.get_model_url(), context.util.get_queue_id(), MLSDDetector.load_model
-        )
+        loaded_model = context.models.load_remote_model(MLSDDetector.get_model_url(), MLSDDetector.load_model)
 
         with loaded_model as model:
             assert isinstance(model, MobileV2_MLSD_Large)

invokeai/app/invocations/model.py

@@ -1,5 +1,5 @@
 import copy
-from typing import List, Optional
+from typing import List, Literal, Optional
 
 from pydantic import BaseModel, Field
 
@@ -13,9 +13,11 @@ from invokeai.app.invocations.baseinvocation import (
 from invokeai.app.invocations.fields import FieldDescriptions, Input, InputField, OutputField, UIType
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.shared.models import FreeUConfig
+from invokeai.backend.flux.util import max_seq_lengths
 from invokeai.backend.model_manager.config import (
     AnyModelConfig,
     BaseModelType,
+    CheckpointConfigBase,
     ModelType,
     SubModelType,
 )
@@ -137,6 +139,78 @@ class ModelIdentifierInvocation(BaseInvocation):
         return ModelIdentifierOutput(model=self.model)
 
 
+@invocation_output("flux_model_loader_output")
+class FluxModelLoaderOutput(BaseInvocationOutput):
+    """Flux base model loader output"""
+
+    transformer: TransformerField = OutputField(description=FieldDescriptions.transformer, title="Transformer")
+    clip: CLIPField = OutputField(description=FieldDescriptions.clip, title="CLIP")
+    t5_encoder: T5EncoderField = OutputField(description=FieldDescriptions.t5_encoder, title="T5 Encoder")
+    vae: VAEField = OutputField(description=FieldDescriptions.vae, title="VAE")
+    max_seq_len: Literal[256, 512] = OutputField(
+        description="The max sequence length to used for the T5 encoder. (256 for schnell transformer, 512 for dev transformer)",
+        title="Max Seq Length",
+    )
+
+
+@invocation(
+    "flux_model_loader",
+    title="Flux Main Model",
+    tags=["model", "flux"],
+    category="model",
+    version="1.0.4",
+    classification=Classification.Prototype,
+)
+class FluxModelLoaderInvocation(BaseInvocation):
+    """Loads a flux base model, outputting its submodels."""
+
+    model: ModelIdentifierField = InputField(
+        description=FieldDescriptions.flux_model,
+        ui_type=UIType.FluxMainModel,
+        input=Input.Direct,
+    )
+
+    t5_encoder_model: ModelIdentifierField = InputField(
+        description=FieldDescriptions.t5_encoder, ui_type=UIType.T5EncoderModel, input=Input.Direct, title="T5 Encoder"
+    )
+
+    clip_embed_model: ModelIdentifierField = InputField(
+        description=FieldDescriptions.clip_embed_model,
+        ui_type=UIType.CLIPEmbedModel,
+        input=Input.Direct,
+        title="CLIP Embed",
+    )
+
+    vae_model: ModelIdentifierField = InputField(
+        description=FieldDescriptions.vae_model, ui_type=UIType.FluxVAEModel, title="VAE"
+    )
+
+    def invoke(self, context: InvocationContext) -> FluxModelLoaderOutput:
+        for key in [self.model.key, self.t5_encoder_model.key, self.clip_embed_model.key, self.vae_model.key]:
+            if not context.models.exists(key):
+                raise ValueError(f"Unknown model: {key}")
+
+        transformer = self.model.model_copy(update={"submodel_type": SubModelType.Transformer})
+        vae = self.vae_model.model_copy(update={"submodel_type": SubModelType.VAE})
+
+        tokenizer = self.clip_embed_model.model_copy(update={"submodel_type": SubModelType.Tokenizer})
+        clip_encoder = self.clip_embed_model.model_copy(update={"submodel_type": SubModelType.TextEncoder})
+
+        tokenizer2 = self.t5_encoder_model.model_copy(update={"submodel_type": SubModelType.Tokenizer2})
+        t5_encoder = self.t5_encoder_model.model_copy(update={"submodel_type": SubModelType.TextEncoder2})
+
+        transformer_config = context.models.get_config(transformer)
+        assert isinstance(transformer_config, CheckpointConfigBase)
+
+        return FluxModelLoaderOutput(
+            transformer=TransformerField(transformer=transformer, loras=[]),
+            clip=CLIPField(tokenizer=tokenizer, text_encoder=clip_encoder, loras=[], skipped_layers=0),
+            t5_encoder=T5EncoderField(tokenizer=tokenizer2, text_encoder=t5_encoder),
+            vae=VAEField(vae=vae),
+            max_seq_len=max_seq_lengths[transformer_config.config_path],
+        )
+
+
 @invocation(
     "main_model_loader",
     title="Main Model",

invokeai/app/invocations/normal_bae.py

@@ -20,9 +20,7 @@ class NormalMapInvocation(BaseInvocation, WithMetadata, WithBoard):
 
     def invoke(self, context: InvocationContext) -> ImageOutput:
         image = context.images.get_pil(self.image.image_name, "RGB")
-        loaded_model = context.models.load_remote_model(
-            NormalMapDetector.get_model_url(), context.util.get_queue_id(), NormalMapDetector.load_model
-        )
+        loaded_model = context.models.load_remote_model(NormalMapDetector.get_model_url(), NormalMapDetector.load_model)
 
         with loaded_model as model:
             assert isinstance(model, NNET)

invokeai/app/invocations/pidi.py

@@ -22,9 +22,7 @@ class PiDiNetEdgeDetectionInvocation(BaseInvocation, WithMetadata, WithBoard):
 
     def invoke(self, context: InvocationContext) -> ImageOutput:
         image = context.images.get_pil(self.image.image_name, "RGB")
-        loaded_model = context.models.load_remote_model(
-            PIDINetDetector.get_model_url(), context.util.get_queue_id(), PIDINetDetector.load_model
-        )
+        loaded_model = context.models.load_remote_model(PIDINetDetector.get_model_url(), PIDINetDetector.load_model)
 
         with loaded_model as model:
             assert isinstance(model, PiDiNet)

invokeai/app/invocations/primitives.py

@@ -18,7 +18,6 @@ from invokeai.app.invocations.fields import (
     InputField,
     LatentsField,
     OutputField,
-    SD3ConditioningField,
     TensorField,
     UIComponent,
 )
@@ -427,17 +426,6 @@ class FluxConditioningOutput(BaseInvocationOutput):
         return cls(conditioning=FluxConditioningField(conditioning_name=conditioning_name))
 
 
-@invocation_output("sd3_conditioning_output")
-class SD3ConditioningOutput(BaseInvocationOutput):
-    """Base class for nodes that output a single SD3 conditioning tensor"""
-
-    conditioning: SD3ConditioningField = OutputField(description=FieldDescriptions.cond)
-
-    @classmethod
-    def build(cls, conditioning_name: str) -> "SD3ConditioningOutput":
-        return cls(conditioning=SD3ConditioningField(conditioning_name=conditioning_name))
-
-
 @invocation_output("conditioning_output")
 class ConditioningOutput(BaseInvocationOutput):
     """Base class for nodes that output a single conditioning tensor"""

invokeai/app/invocations/sd3_denoise.py

@@ -1,260 +0,0 @@
-from typing import Callable, Tuple
-
-import torch
-from diffusers.models.transformers.transformer_sd3 import SD3Transformer2DModel
-from diffusers.schedulers.scheduling_flow_match_euler_discrete import FlowMatchEulerDiscreteScheduler
-from tqdm import tqdm
-
-from invokeai.app.invocations.baseinvocation import BaseInvocation, Classification, invocation
-from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR
-from invokeai.app.invocations.fields import (
-    FieldDescriptions,
-    Input,
-    InputField,
-    SD3ConditioningField,
-    WithBoard,
-    WithMetadata,
-)
-from invokeai.app.invocations.model import TransformerField
-from invokeai.app.invocations.primitives import LatentsOutput
-from invokeai.app.invocations.sd3_text_encoder import SD3_T5_MAX_SEQ_LEN
-from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.model_manager.config import BaseModelType
-from invokeai.backend.stable_diffusion.diffusers_pipeline import PipelineIntermediateState
-from invokeai.backend.stable_diffusion.diffusion.conditioning_data import SD3ConditioningInfo
-from invokeai.backend.util.devices import TorchDevice
-
-
-@invocation(
-    "sd3_denoise",
-    title="SD3 Denoise",
-    tags=["image", "sd3"],
-    category="image",
-    version="1.0.0",
-    classification=Classification.Prototype,
-)
-class SD3DenoiseInvocation(BaseInvocation, WithMetadata, WithBoard):
-    """Run denoising process with a SD3 model."""
-
-    transformer: TransformerField = InputField(
-        description=FieldDescriptions.sd3_model,
-        input=Input.Connection,
-        title="Transformer",
-    )
-    positive_conditioning: SD3ConditioningField = InputField(
-        description=FieldDescriptions.positive_cond, input=Input.Connection
-    )
-    negative_conditioning: SD3ConditioningField = InputField(
-        description=FieldDescriptions.negative_cond, input=Input.Connection
-    )
-    cfg_scale: float | list[float] = InputField(default=3.5, description=FieldDescriptions.cfg_scale, title="CFG Scale")
-    width: int = InputField(default=1024, multiple_of=16, description="Width of the generated image.")
-    height: int = InputField(default=1024, multiple_of=16, description="Height of the generated image.")
-    steps: int = InputField(default=10, gt=0, description=FieldDescriptions.steps)
-    seed: int = InputField(default=0, description="Randomness seed for reproducibility.")
-
-    @torch.no_grad()
-    def invoke(self, context: InvocationContext) -> LatentsOutput:
-        latents = self._run_diffusion(context)
-        latents = latents.detach().to("cpu")
-
-        name = context.tensors.save(tensor=latents)
-        return LatentsOutput.build(latents_name=name, latents=latents, seed=None)
-
-    def _load_text_conditioning(
-        self,
-        context: InvocationContext,
-        conditioning_name: str,
-        joint_attention_dim: int,
-        dtype: torch.dtype,
-        device: torch.device,
-    ) -> Tuple[torch.Tensor, torch.Tensor]:
-        # Load the conditioning data.
-        cond_data = context.conditioning.load(conditioning_name)
-        assert len(cond_data.conditionings) == 1
-        sd3_conditioning = cond_data.conditionings[0]
-        assert isinstance(sd3_conditioning, SD3ConditioningInfo)
-        sd3_conditioning = sd3_conditioning.to(dtype=dtype, device=device)
-
-        t5_embeds = sd3_conditioning.t5_embeds
-        if t5_embeds is None:
-            t5_embeds = torch.zeros(
-                (1, SD3_T5_MAX_SEQ_LEN, joint_attention_dim),
-                device=device,
-                dtype=dtype,
-            )
-
-        clip_prompt_embeds = torch.cat([sd3_conditioning.clip_l_embeds, sd3_conditioning.clip_g_embeds], dim=-1)
-        clip_prompt_embeds = torch.nn.functional.pad(
-            clip_prompt_embeds, (0, t5_embeds.shape[-1] - clip_prompt_embeds.shape[-1])
-        )
-
-        prompt_embeds = torch.cat([clip_prompt_embeds, t5_embeds], dim=-2)
-        pooled_prompt_embeds = torch.cat(
-            [sd3_conditioning.clip_l_pooled_embeds, sd3_conditioning.clip_g_pooled_embeds], dim=-1
-        )
-
-        return prompt_embeds, pooled_prompt_embeds
-
-    def _get_noise(
-        self,
-        num_samples: int,
-        num_channels_latents: int,
-        height: int,
-        width: int,
-        dtype: torch.dtype,
-        device: torch.device,
-        seed: int,
-    ) -> torch.Tensor:
-        # We always generate noise on the same device and dtype then cast to ensure consistency across devices/dtypes.
-        rand_device = "cpu"
-        rand_dtype = torch.float16
-
-        return torch.randn(
-            num_samples,
-            num_channels_latents,
-            int(height) // LATENT_SCALE_FACTOR,
-            int(width) // LATENT_SCALE_FACTOR,
-            device=rand_device,
-            dtype=rand_dtype,
-            generator=torch.Generator(device=rand_device).manual_seed(seed),
-        ).to(device=device, dtype=dtype)
-
-    def _prepare_cfg_scale(self, num_timesteps: int) -> list[float]:
-        """Prepare the CFG scale list.
-
-        Args:
-            num_timesteps (int): The number of timesteps in the scheduler. Could be different from num_steps depending
-            on the scheduler used (e.g. higher order schedulers).
-
-        Returns:
-            list[float]: _description_
-        """
-        if isinstance(self.cfg_scale, float):
-            cfg_scale = [self.cfg_scale] * num_timesteps
-        elif isinstance(self.cfg_scale, list):
-            assert len(self.cfg_scale) == num_timesteps
-            cfg_scale = self.cfg_scale
-        else:
-            raise ValueError(f"Invalid CFG scale type: {type(self.cfg_scale)}")
-
-        return cfg_scale
-
-    def _run_diffusion(
-        self,
-        context: InvocationContext,
-    ):
-        inference_dtype = TorchDevice.choose_torch_dtype()
-        device = TorchDevice.choose_torch_device()
-
-        transformer_info = context.models.load(self.transformer.transformer, context.util.get_queue_id())
-
-        # Load/process the conditioning data.
-        # TODO(ryand): Make CFG optional.
-        do_classifier_free_guidance = True
-        pos_prompt_embeds, pos_pooled_prompt_embeds = self._load_text_conditioning(
-            context=context,
-            conditioning_name=self.positive_conditioning.conditioning_name,
-            joint_attention_dim=transformer_info.model.config.joint_attention_dim,
-            dtype=inference_dtype,
-            device=device,
-        )
-        neg_prompt_embeds, neg_pooled_prompt_embeds = self._load_text_conditioning(
-            context=context,
-            conditioning_name=self.negative_conditioning.conditioning_name,
-            joint_attention_dim=transformer_info.model.config.joint_attention_dim,
-            dtype=inference_dtype,
-            device=device,
-        )
-        # TODO(ryand): Support both sequential and batched CFG inference.
-        prompt_embeds = torch.cat([neg_prompt_embeds, pos_prompt_embeds], dim=0)
-        pooled_prompt_embeds = torch.cat([neg_pooled_prompt_embeds, pos_pooled_prompt_embeds], dim=0)
-
-        # Prepare the scheduler.
-        scheduler = FlowMatchEulerDiscreteScheduler()
-        scheduler.set_timesteps(num_inference_steps=self.steps, device=device)
-        timesteps = scheduler.timesteps
-        assert isinstance(timesteps, torch.Tensor)
-
-        # Prepare the CFG scale list.
-        cfg_scale = self._prepare_cfg_scale(len(timesteps))
-
-        # Generate initial latent noise.
-        num_channels_latents = transformer_info.model.config.in_channels
-        assert isinstance(num_channels_latents, int)
-        noise = self._get_noise(
-            num_samples=1,
-            num_channels_latents=num_channels_latents,
-            height=self.height,
-            width=self.width,
-            dtype=inference_dtype,
-            device=device,
-            seed=self.seed,
-        )
-        latents: torch.Tensor = noise
-
-        total_steps = len(timesteps)
-        step_callback = self._build_step_callback(context)
-
-        step_callback(
-            PipelineIntermediateState(
-                step=0,
-                order=1,
-                total_steps=total_steps,
-                timestep=int(timesteps[0]),
-                latents=latents,
-            ),
-        )
-
-        with transformer_info.model_on_device() as (cached_weights, transformer):
-            assert isinstance(transformer, SD3Transformer2DModel)
-
-            # 6. Denoising loop
-            for step_idx, t in tqdm(list(enumerate(timesteps))):
-                # Expand the latents if we are doing CFG.
-                latent_model_input = torch.cat([latents] * 2) if do_classifier_free_guidance else latents
-                # Expand the timestep to match the latent model input.
-                timestep = t.expand(latent_model_input.shape[0])
-
-                noise_pred = transformer(
-                    hidden_states=latent_model_input,
-                    timestep=timestep,
-                    encoder_hidden_states=prompt_embeds,
-                    pooled_projections=pooled_prompt_embeds,
-                    joint_attention_kwargs=None,
-                    return_dict=False,
-                )[0]
-
-                # Apply CFG.
-                if do_classifier_free_guidance:
-                    noise_pred_uncond, noise_pred_cond = noise_pred.chunk(2)
-                    noise_pred = noise_pred_uncond + cfg_scale[step_idx] * (noise_pred_cond - noise_pred_uncond)
-
-                # Compute the previous noisy sample x_t -> x_t-1.
-                latents_dtype = latents.dtype
-                latents = scheduler.step(model_output=noise_pred, timestep=t, sample=latents, return_dict=False)[0]
-
-                # TODO(ryand): This MPS dtype handling was copied from diffusers, I haven't tested to see if it's
-                # needed.
-                if latents.dtype != latents_dtype:
-                    if torch.backends.mps.is_available():
-                        # some platforms (eg. apple mps) misbehave due to a pytorch bug: https://github.com/pytorch/pytorch/pull/99272
-                        latents = latents.to(latents_dtype)
-
-                step_callback(
-                    PipelineIntermediateState(
-                        step=step_idx + 1,
-                        order=1,
-                        total_steps=total_steps,
-                        timestep=int(t),
-                        latents=latents,
-                    ),
-                )
-
-        return latents
-
-    def _build_step_callback(self, context: InvocationContext) -> Callable[[PipelineIntermediateState], None]:
-        def step_callback(state: PipelineIntermediateState) -> None:
-            context.util.sd_step_callback(state, BaseModelType.StableDiffusion3)
-
-        return step_callback

invokeai/app/invocations/sd3_latents_to_image.py

@@ -1,73 +0,0 @@
-from contextlib import nullcontext
-
-import torch
-from diffusers.models.autoencoders.autoencoder_kl import AutoencoderKL
-from einops import rearrange
-from PIL import Image
-
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import (
-    FieldDescriptions,
-    Input,
-    InputField,
-    LatentsField,
-    WithBoard,
-    WithMetadata,
-)
-from invokeai.app.invocations.model import VAEField
-from invokeai.app.invocations.primitives import ImageOutput
-from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.stable_diffusion.extensions.seamless import SeamlessExt
-from invokeai.backend.util.devices import TorchDevice
-
-
-@invocation(
-    "sd3_l2i",
-    title="SD3 Latents to Image",
-    tags=["latents", "image", "vae", "l2i", "sd3"],
-    category="latents",
-    version="1.3.0",
-)
-class SD3LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
-    """Generates an image from latents."""
-
-    latents: LatentsField = InputField(
-        description=FieldDescriptions.latents,
-        input=Input.Connection,
-    )
-    vae: VAEField = InputField(
-        description=FieldDescriptions.vae,
-        input=Input.Connection,
-    )
-
-    @torch.no_grad()
-    def invoke(self, context: InvocationContext) -> ImageOutput:
-        latents = context.tensors.load(self.latents.latents_name)
-
-        vae_info = context.models.load(self.vae.vae, context.util.get_queue_id())
-        assert isinstance(vae_info.model, (AutoencoderKL))
-        with SeamlessExt.static_patch_model(vae_info.model, self.vae.seamless_axes), vae_info as vae:
-            assert isinstance(vae, (AutoencoderKL))
-            latents = latents.to(vae.device)
-
-            vae.disable_tiling()
-
-            tiling_context = nullcontext()
-
-            # clear memory as vae decode can request a lot
-            TorchDevice.empty_cache()
-
-            with torch.inference_mode(), tiling_context:
-                # copied from diffusers pipeline
-                latents = latents / vae.config.scaling_factor
-                img = vae.decode(latents, return_dict=False)[0]
-
-            img = img.clamp(-1, 1)
-            img = rearrange(img[0], "c h w -> h w c")  # noqa: F821
-            img_pil = Image.fromarray((127.5 * (img + 1.0)).byte().cpu().numpy())
-
-        TorchDevice.empty_cache()
-
-        image_dto = context.images.save(image=img_pil)
-
-        return ImageOutput.build(image_dto)

invokeai/app/invocations/sd3_model_loader.py

@@ -1,108 +0,0 @@
-from typing import Optional
-
-from invokeai.app.invocations.baseinvocation import (
-    BaseInvocation,
-    BaseInvocationOutput,
-    Classification,
-    invocation,
-    invocation_output,
-)
-from invokeai.app.invocations.fields import FieldDescriptions, Input, InputField, OutputField, UIType
-from invokeai.app.invocations.model import CLIPField, ModelIdentifierField, T5EncoderField, TransformerField, VAEField
-from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.model_manager.config import SubModelType
-
-
-@invocation_output("sd3_model_loader_output")
-class Sd3ModelLoaderOutput(BaseInvocationOutput):
-    """SD3 base model loader output."""
-
-    transformer: TransformerField = OutputField(description=FieldDescriptions.transformer, title="Transformer")
-    clip_l: CLIPField = OutputField(description=FieldDescriptions.clip, title="CLIP L")
-    clip_g: CLIPField = OutputField(description=FieldDescriptions.clip, title="CLIP G")
-    t5_encoder: T5EncoderField = OutputField(description=FieldDescriptions.t5_encoder, title="T5 Encoder")
-    vae: VAEField = OutputField(description=FieldDescriptions.vae, title="VAE")
-
-
-@invocation(
-    "sd3_model_loader",
-    title="SD3 Main Model",
-    tags=["model", "sd3"],
-    category="model",
-    version="1.0.0",
-    classification=Classification.Prototype,
-)
-class Sd3ModelLoaderInvocation(BaseInvocation):
-    """Loads a SD3 base model, outputting its submodels."""
-
-    model: ModelIdentifierField = InputField(
-        description=FieldDescriptions.sd3_model,
-        ui_type=UIType.SD3MainModel,
-        input=Input.Direct,
-    )
-
-    t5_encoder_model: Optional[ModelIdentifierField] = InputField(
-        description=FieldDescriptions.t5_encoder,
-        ui_type=UIType.T5EncoderModel,
-        input=Input.Direct,
-        title="T5 Encoder",
-        default=None,
-    )
-
-    clip_l_model: Optional[ModelIdentifierField] = InputField(
-        description=FieldDescriptions.clip_embed_model,
-        ui_type=UIType.CLIPLEmbedModel,
-        input=Input.Direct,
-        title="CLIP L Encoder",
-        default=None,
-    )
-
-    clip_g_model: Optional[ModelIdentifierField] = InputField(
-        description=FieldDescriptions.clip_g_model,
-        ui_type=UIType.CLIPGEmbedModel,
-        input=Input.Direct,
-        title="CLIP G Encoder",
-        default=None,
-    )
-
-    vae_model: Optional[ModelIdentifierField] = InputField(
-        description=FieldDescriptions.vae_model, ui_type=UIType.VAEModel, title="VAE", default=None
-    )
-
-    def invoke(self, context: InvocationContext) -> Sd3ModelLoaderOutput:
-        transformer = self.model.model_copy(update={"submodel_type": SubModelType.Transformer})
-        vae = (
-            self.vae_model.model_copy(update={"submodel_type": SubModelType.VAE})
-            if self.vae_model
-            else self.model.model_copy(update={"submodel_type": SubModelType.VAE})
-        )
-        tokenizer_l = self.model.model_copy(update={"submodel_type": SubModelType.Tokenizer})
-        clip_encoder_l = (
-            self.clip_l_model.model_copy(update={"submodel_type": SubModelType.TextEncoder})
-            if self.clip_l_model
-            else self.model.model_copy(update={"submodel_type": SubModelType.TextEncoder})
-        )
-        tokenizer_g = self.model.model_copy(update={"submodel_type": SubModelType.Tokenizer2})
-        clip_encoder_g = (
-            self.clip_g_model.model_copy(update={"submodel_type": SubModelType.TextEncoder2})
-            if self.clip_g_model
-            else self.model.model_copy(update={"submodel_type": SubModelType.TextEncoder2})
-        )
-        tokenizer_t5 = (
-            self.t5_encoder_model.model_copy(update={"submodel_type": SubModelType.Tokenizer3})
-            if self.t5_encoder_model
-            else self.model.model_copy(update={"submodel_type": SubModelType.Tokenizer3})
-        )
-        t5_encoder = (
-            self.t5_encoder_model.model_copy(update={"submodel_type": SubModelType.TextEncoder3})
-            if self.t5_encoder_model
-            else self.model.model_copy(update={"submodel_type": SubModelType.TextEncoder3})
-        )
-
-        return Sd3ModelLoaderOutput(
-            transformer=TransformerField(transformer=transformer, loras=[]),
-            clip_l=CLIPField(tokenizer=tokenizer_l, text_encoder=clip_encoder_l, loras=[], skipped_layers=0),
-            clip_g=CLIPField(tokenizer=tokenizer_g, text_encoder=clip_encoder_g, loras=[], skipped_layers=0),
-            t5_encoder=T5EncoderField(tokenizer=tokenizer_t5, text_encoder=t5_encoder),
-            vae=VAEField(vae=vae),
-        )

invokeai/app/invocations/sd3_text_encoder.py

@@ -1,199 +0,0 @@
-from contextlib import ExitStack
-from typing import Iterator, Tuple
-
-import torch
-from transformers import (
-    CLIPTextModel,
-    CLIPTextModelWithProjection,
-    CLIPTokenizer,
-    T5EncoderModel,
-    T5Tokenizer,
-    T5TokenizerFast,
-)
-
-from invokeai.app.invocations.baseinvocation import BaseInvocation, Classification, invocation
-from invokeai.app.invocations.fields import FieldDescriptions, Input, InputField
-from invokeai.app.invocations.model import CLIPField, T5EncoderField
-from invokeai.app.invocations.primitives import SD3ConditioningOutput
-from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.lora.conversions.flux_lora_constants import FLUX_LORA_CLIP_PREFIX
-from invokeai.backend.lora.lora_model_raw import LoRAModelRaw
-from invokeai.backend.lora.lora_patcher import LoRAPatcher
-from invokeai.backend.model_manager.config import ModelFormat
-from invokeai.backend.stable_diffusion.diffusion.conditioning_data import ConditioningFieldData, SD3ConditioningInfo
-
-# The SD3 T5 Max Sequence Length set based on the default in diffusers.
-SD3_T5_MAX_SEQ_LEN = 256
-
-
-@invocation(
-    "sd3_text_encoder",
-    title="SD3 Text Encoding",
-    tags=["prompt", "conditioning", "sd3"],
-    category="conditioning",
-    version="1.0.0",
-    classification=Classification.Prototype,
-)
-class Sd3TextEncoderInvocation(BaseInvocation):
-    """Encodes and preps a prompt for a SD3 image."""
-
-    clip_l: CLIPField = InputField(
-        title="CLIP L",
-        description=FieldDescriptions.clip,
-        input=Input.Connection,
-    )
-    clip_g: CLIPField = InputField(
-        title="CLIP G",
-        description=FieldDescriptions.clip,
-        input=Input.Connection,
-    )
-
-    # The SD3 models were trained with text encoder dropout, so the T5 encoder can be omitted to save time/memory.
-    t5_encoder: T5EncoderField | None = InputField(
-        title="T5Encoder",
-        default=None,
-        description=FieldDescriptions.t5_encoder,
-        input=Input.Connection,
-    )
-    prompt: str = InputField(description="Text prompt to encode.")
-
-    @torch.no_grad()
-    def invoke(self, context: InvocationContext) -> SD3ConditioningOutput:
-        # Note: The text encoding model are run in separate functions to ensure that all model references are locally
-        # scoped. This ensures that earlier models can be freed and gc'd before loading later models (if necessary).
-
-        clip_l_embeddings, clip_l_pooled_embeddings = self._clip_encode(context, self.clip_l)
-        clip_g_embeddings, clip_g_pooled_embeddings = self._clip_encode(context, self.clip_g)
-
-        t5_embeddings: torch.Tensor | None = None
-        if self.t5_encoder is not None:
-            t5_embeddings = self._t5_encode(context, SD3_T5_MAX_SEQ_LEN)
-
-        conditioning_data = ConditioningFieldData(
-            conditionings=[
-                SD3ConditioningInfo(
-                    clip_l_embeds=clip_l_embeddings,
-                    clip_l_pooled_embeds=clip_l_pooled_embeddings,
-                    clip_g_embeds=clip_g_embeddings,
-                    clip_g_pooled_embeds=clip_g_pooled_embeddings,
-                    t5_embeds=t5_embeddings,
-                )
-            ]
-        )
-
-        conditioning_name = context.conditioning.save(conditioning_data)
-        return SD3ConditioningOutput.build(conditioning_name)
-
-    def _t5_encode(self, context: InvocationContext, max_seq_len: int) -> torch.Tensor:
-        assert self.t5_encoder is not None
-        t5_tokenizer_info = context.models.load(self.t5_encoder.tokenizer, context.util.get_queue_id())
-        t5_text_encoder_info = context.models.load(self.t5_encoder.text_encoder, context.util.get_queue_id())
-
-        prompt = [self.prompt]
-
-        with (
-            t5_text_encoder_info as t5_text_encoder,
-            t5_tokenizer_info as t5_tokenizer,
-        ):
-            assert isinstance(t5_text_encoder, T5EncoderModel)
-            assert isinstance(t5_tokenizer, (T5Tokenizer, T5TokenizerFast))
-
-            text_inputs = t5_tokenizer(
-                prompt,
-                padding="max_length",
-                max_length=max_seq_len,
-                truncation=True,
-                add_special_tokens=True,
-                return_tensors="pt",
-            )
-            text_input_ids = text_inputs.input_ids
-            untruncated_ids = t5_tokenizer(prompt, padding="longest", return_tensors="pt").input_ids
-            assert isinstance(text_input_ids, torch.Tensor)
-            assert isinstance(untruncated_ids, torch.Tensor)
-            if untruncated_ids.shape[-1] >= text_input_ids.shape[-1] and not torch.equal(
-                text_input_ids, untruncated_ids
-            ):
-                removed_text = t5_tokenizer.batch_decode(untruncated_ids[:, max_seq_len - 1 : -1])
-                context.logger.warning(
-                    "The following part of your input was truncated because `max_sequence_length` is set to "
-                    f" {max_seq_len} tokens: {removed_text}"
-                )
-
-            prompt_embeds = t5_text_encoder(text_input_ids.to(t5_text_encoder.device))[0]
-
-        assert isinstance(prompt_embeds, torch.Tensor)
-        return prompt_embeds
-
-    def _clip_encode(
-        self, context: InvocationContext, clip_model: CLIPField, tokenizer_max_length: int = 77
-    ) -> Tuple[torch.Tensor, torch.Tensor]:
-        clip_tokenizer_info = context.models.load(clip_model.tokenizer, context.util.get_queue_id())
-        clip_text_encoder_info = context.models.load(clip_model.text_encoder, context.util.get_queue_id())
-
-        prompt = [self.prompt]
-
-        with (
-            clip_text_encoder_info.model_on_device() as (cached_weights, clip_text_encoder),
-            clip_tokenizer_info as clip_tokenizer,
-            ExitStack() as exit_stack,
-        ):
-            assert isinstance(clip_text_encoder, (CLIPTextModel, CLIPTextModelWithProjection))
-            assert isinstance(clip_tokenizer, CLIPTokenizer)
-
-            clip_text_encoder_config = clip_text_encoder_info.config
-            assert clip_text_encoder_config is not None
-
-            # Apply LoRA models to the CLIP encoder.
-            # Note: We apply the LoRA after the transformer has been moved to its target device for faster patching.
-            if clip_text_encoder_config.format in [ModelFormat.Diffusers]:
-                # The model is non-quantized, so we can apply the LoRA weights directly into the model.
-                exit_stack.enter_context(
-                    LoRAPatcher.apply_lora_patches(
-                        model=clip_text_encoder,
-                        patches=self._clip_lora_iterator(context, clip_model),
-                        prefix=FLUX_LORA_CLIP_PREFIX,
-                        cached_weights=cached_weights,
-                    )
-                )
-            else:
-                # There are currently no supported CLIP quantized models. Add support here if needed.
-                raise ValueError(f"Unsupported model format: {clip_text_encoder_config.format}")
-
-            clip_text_encoder = clip_text_encoder.eval().requires_grad_(False)
-
-            text_inputs = clip_tokenizer(
-                prompt,
-                padding="max_length",
-                max_length=tokenizer_max_length,
-                truncation=True,
-                return_tensors="pt",
-            )
-
-            text_input_ids = text_inputs.input_ids
-            untruncated_ids = clip_tokenizer(prompt, padding="longest", return_tensors="pt").input_ids
-            assert isinstance(text_input_ids, torch.Tensor)
-            assert isinstance(untruncated_ids, torch.Tensor)
-            if untruncated_ids.shape[-1] >= text_input_ids.shape[-1] and not torch.equal(
-                text_input_ids, untruncated_ids
-            ):
-                removed_text = clip_tokenizer.batch_decode(untruncated_ids[:, tokenizer_max_length - 1 : -1])
-                context.logger.warning(
-                    "The following part of your input was truncated because CLIP can only handle sequences up to"
-                    f" {tokenizer_max_length} tokens: {removed_text}"
-                )
-            prompt_embeds = clip_text_encoder(
-                input_ids=text_input_ids.to(clip_text_encoder.device), output_hidden_states=True
-            )
-            pooled_prompt_embeds = prompt_embeds[0]
-            prompt_embeds = prompt_embeds.hidden_states[-2]
-
-            return prompt_embeds, pooled_prompt_embeds
-
-    def _clip_lora_iterator(
-        self, context: InvocationContext, clip_model: CLIPField
-    ) -> Iterator[Tuple[LoRAModelRaw, float]]:
-        for lora in clip_model.loras:
-            lora_info = context.models.load(lora.lora, context.util.get_queue_id())
-            assert isinstance(lora_info.model, LoRAModelRaw)
-            yield (lora_info.model, lora.weight)
-            del lora_info

invokeai/app/invocations/segment_anything.py

@@ -1,11 +1,9 @@
-from enum import Enum
 from pathlib import Path
 from typing import Literal
 
 import numpy as np
 import torch
 from PIL import Image
-from pydantic import BaseModel, Field
 from transformers import AutoModelForMaskGeneration, AutoProcessor
 from transformers.models.sam import SamModel
 from transformers.models.sam.processing_sam import SamProcessor
@@ -25,31 +23,12 @@ SEGMENT_ANYTHING_MODEL_IDS: dict[SegmentAnythingModelKey, str] = {
 }
 
 
-class SAMPointLabel(Enum):
-    negative = -1
-    neutral = 0
-    positive = 1
-
-
-class SAMPoint(BaseModel):
-    x: int = Field(..., description="The x-coordinate of the point")
-    y: int = Field(..., description="The y-coordinate of the point")
-    label: SAMPointLabel = Field(..., description="The label of the point")
-
-
-class SAMPointsField(BaseModel):
-    points: list[SAMPoint] = Field(..., description="The points of the object")
-
-    def to_list(self) -> list[list[int]]:
-        return [[point.x, point.y, point.label.value] for point in self.points]
-
-
 @invocation(
     "segment_anything",
     title="Segment Anything",
     tags=["prompt", "segmentation"],
     category="segmentation",
-    version="1.1.0",
+    version="1.0.0",
 )
 class SegmentAnythingInvocation(BaseInvocation):
     """Runs a Segment Anything Model."""
@@ -61,13 +40,7 @@ class SegmentAnythingInvocation(BaseInvocation):
 
     model: SegmentAnythingModelKey = InputField(description="The Segment Anything model to use.")
     image: ImageField = InputField(description="The image to segment.")
-    bounding_boxes: list[BoundingBoxField] | None = InputField(
-        default=None, description="The bounding boxes to prompt the SAM model with."
-    )
-    point_lists: list[SAMPointsField] | None = InputField(
-        default=None,
-        description="The list of point lists to prompt the SAM model with. Each list of points represents a single object.",
-    )
+    bounding_boxes: list[BoundingBoxField] = InputField(description="The bounding boxes to prompt the SAM model with.")
     apply_polygon_refinement: bool = InputField(
         description="Whether to apply polygon refinement to the masks. This will smooth the edges of the masks slightly and ensure that each mask consists of a single closed polygon (before merging).",
         default=True,
@@ -82,12 +55,7 @@ class SegmentAnythingInvocation(BaseInvocation):
         # The models expect a 3-channel RGB image.
         image_pil = context.images.get_pil(self.image.image_name, mode="RGB")
 
-        if self.point_lists is not None and self.bounding_boxes is not None:
-            raise ValueError("Only one of point_lists or bounding_box can be provided.")
-
-        if (not self.bounding_boxes or len(self.bounding_boxes) == 0) and (
-            not self.point_lists or len(self.point_lists) == 0
-        ):
+        if len(self.bounding_boxes) == 0:
             combined_mask = torch.zeros(image_pil.size[::-1], dtype=torch.bool)
         else:
             masks = self._segment(context=context, image=image_pil)
@@ -115,23 +83,22 @@ class SegmentAnythingInvocation(BaseInvocation):
         assert isinstance(sam_processor, SamProcessor)
         return SegmentAnythingPipeline(sam_model=sam_model, sam_processor=sam_processor)
 
-    def _segment(self, context: InvocationContext, image: Image.Image) -> list[torch.Tensor]:
+    def _segment(
+        self,
+        context: InvocationContext,
+        image: Image.Image,
+    ) -> list[torch.Tensor]:
         """Use Segment Anything (SAM) to generate masks given an image + a set of bounding boxes."""
         # Convert the bounding boxes to the SAM input format.
-        sam_bounding_boxes = (
-            [[bb.x_min, bb.y_min, bb.x_max, bb.y_max] for bb in self.bounding_boxes] if self.bounding_boxes else None
-        )
-        sam_points = [p.to_list() for p in self.point_lists] if self.point_lists else None
+        sam_bounding_boxes = [[bb.x_min, bb.y_min, bb.x_max, bb.y_max] for bb in self.bounding_boxes]
 
         with (
             context.models.load_remote_model(
-                source=SEGMENT_ANYTHING_MODEL_IDS[self.model],
-                queue_id=context.util.get_queue_id(),
-                loader=SegmentAnythingInvocation._load_sam_model,
+                source=SEGMENT_ANYTHING_MODEL_IDS[self.model], loader=SegmentAnythingInvocation._load_sam_model
             ) as sam_pipeline,
         ):
             assert isinstance(sam_pipeline, SegmentAnythingPipeline)
-            masks = sam_pipeline.segment(image=image, bounding_boxes=sam_bounding_boxes, point_lists=sam_points)
+            masks = sam_pipeline.segment(image=image, bounding_boxes=sam_bounding_boxes)
 
         masks = self._process_masks(masks)
         if self.apply_polygon_refinement:
@@ -174,10 +141,9 @@ class SegmentAnythingInvocation(BaseInvocation):
 
         return masks
 
-    def _filter_masks(
-        self, masks: list[torch.Tensor], bounding_boxes: list[BoundingBoxField] | None
-    ) -> list[torch.Tensor]:
+    def _filter_masks(self, masks: list[torch.Tensor], bounding_boxes: list[BoundingBoxField]) -> list[torch.Tensor]:
         """Filter the detected masks based on the specified mask filter."""
+        assert len(masks) == len(bounding_boxes)
 
         if self.mask_filter == "all":
             return masks
@@ -185,10 +151,6 @@ class SegmentAnythingInvocation(BaseInvocation):
             # Find the largest mask.
             return [max(masks, key=lambda x: float(x.sum()))]
         elif self.mask_filter == "highest_box_score":
-            assert (
-                bounding_boxes is not None
-            ), "Bounding boxes must be provided to use the 'highest_box_score' mask filter."
-            assert len(masks) == len(bounding_boxes)
             # Find the index of the bounding box with the highest score.
             # Note that we fallback to -1.0 if the score is None. This is mainly to satisfy the type checker. In most
             # cases the scores should all be non-None when using this filtering mode. That being said, -1.0 is a

invokeai/app/invocations/spandrel_image_to_image.py

@@ -158,7 +158,7 @@ class SpandrelImageToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
         image = context.images.get_pil(self.image.image_name, mode="RGB")
 
         # Load the model.
-        spandrel_model_info = context.models.load(self.image_to_image_model, context.util.get_queue_id())
+        spandrel_model_info = context.models.load(self.image_to_image_model)
 
         def step_callback(step: int, total_steps: int) -> None:
             context.util.signal_progress(
@@ -207,7 +207,7 @@ class SpandrelImageToImageAutoscaleInvocation(SpandrelImageToImageInvocation):
         image = context.images.get_pil(self.image.image_name, mode="RGB")
 
         # Load the model.
-        spandrel_model_info = context.models.load(self.image_to_image_model, context.util.get_queue_id())
+        spandrel_model_info = context.models.load(self.image_to_image_model)
 
         # The target size of the image, determined by the provided scale. We'll run the upscaler until we hit this size.
         # Later, we may mutate this value if the model doesn't upscale the image or if the user requested a multiple of 8.

invokeai/app/invocations/tiled_multi_diffusion_denoise_latents.py

@@ -196,13 +196,13 @@ class TiledMultiDiffusionDenoiseLatents(BaseInvocation):
         # Prepare an iterator that yields the UNet's LoRA models and their weights.
         def _lora_loader() -> Iterator[Tuple[LoRAModelRaw, float]]:
             for lora in self.unet.loras:
-                lora_info = context.models.load(lora.lora, context.util.get_queue_id())
+                lora_info = context.models.load(lora.lora)
                 assert isinstance(lora_info.model, LoRAModelRaw)
                 yield (lora_info.model, lora.weight)
                 del lora_info
 
         # Load the UNet model.
-        unet_info = context.models.load(self.unet.unet, context.util.get_queue_id())
+        unet_info = context.models.load(self.unet.unet)
 
         with (
             ExitStack() as exit_stack,

invokeai/app/invocations/upscale.py

@@ -90,7 +90,7 @@ class ESRGANInvocation(BaseInvocation, WithMetadata, WithBoard):
             raise ValueError(msg)
 
         loadnet = context.models.load_remote_model(
-            source=ESRGAN_MODEL_URLS[self.model_name], queue_id=context.util.get_queue_id()
+            source=ESRGAN_MODEL_URLS[self.model_name],
         )
 
         with loadnet as loadnet_model:

invokeai/app/services/events/events_base.py

@@ -131,17 +131,15 @@ class EventServiceBase:
 
     # region Model loading
 
-    def emit_model_load_started(
-        self, config: "AnyModelConfig", queue_id: str, submodel_type: Optional["SubModelType"] = None
-    ) -> None:
+    def emit_model_load_started(self, config: "AnyModelConfig", submodel_type: Optional["SubModelType"] = None) -> None:
         """Emitted when a model load is started."""
-        self.dispatch(ModelLoadStartedEvent.build(config, queue_id, submodel_type))
+        self.dispatch(ModelLoadStartedEvent.build(config, submodel_type))
 
     def emit_model_load_complete(
-        self, config: "AnyModelConfig", queue_id: str, submodel_type: Optional["SubModelType"] = None
+        self, config: "AnyModelConfig", submodel_type: Optional["SubModelType"] = None
     ) -> None:
         """Emitted when a model load is complete."""
-        self.dispatch(ModelLoadCompleteEvent.build(config, queue_id, submodel_type))
+        self.dispatch(ModelLoadCompleteEvent.build(config, submodel_type))
 
     # endregion
 

invokeai/app/services/events/events_common.py

@@ -383,14 +383,12 @@ class DownloadErrorEvent(DownloadEventBase):
         return cls(source=str(job.source), error_type=job.error_type, error=job.error)
 
 
-class ModelLoadEventBase(EventBase):
-    """Base class for queue events"""
-
-    queue_id: str = Field(description="The ID of the queue")
+class ModelEventBase(EventBase):
+    """Base class for events associated with a model"""
 
 
 @payload_schema.register
-class ModelLoadStartedEvent(ModelLoadEventBase):
+class ModelLoadStartedEvent(ModelEventBase):
     """Event model for model_load_started"""
 
     __event_name__ = "model_load_started"
@@ -399,14 +397,12 @@ class ModelLoadStartedEvent(ModelLoadEventBase):
     submodel_type: Optional[SubModelType] = Field(default=None, description="The submodel type, if any")
 
     @classmethod
-    def build(
-        cls, config: AnyModelConfig, queue_id: str, submodel_type: Optional[SubModelType] = None
-    ) -> "ModelLoadStartedEvent":
-        return cls(config=config, queue_id=queue_id, submodel_type=submodel_type)
+    def build(cls, config: AnyModelConfig, submodel_type: Optional[SubModelType] = None) -> "ModelLoadStartedEvent":
+        return cls(config=config, submodel_type=submodel_type)
 
 
 @payload_schema.register
-class ModelLoadCompleteEvent(ModelLoadEventBase):
+class ModelLoadCompleteEvent(ModelEventBase):
     """Event model for model_load_complete"""
 
     __event_name__ = "model_load_complete"
@@ -415,14 +411,8 @@ class ModelLoadCompleteEvent(ModelLoadEventBase):
     submodel_type: Optional[SubModelType] = Field(default=None, description="The submodel type, if any")
 
     @classmethod
-    def build(
-        cls, config: AnyModelConfig, queue_id: str, submodel_type: Optional[SubModelType] = None
-    ) -> "ModelLoadCompleteEvent":
-        return cls(config=config, queue_id=queue_id, submodel_type=submodel_type)
-
-
-class ModelEventBase(EventBase):
-    """Base class for model events"""
+    def build(cls, config: AnyModelConfig, submodel_type: Optional[SubModelType] = None) -> "ModelLoadCompleteEvent":
+        return cls(config=config, submodel_type=submodel_type)
 
 
 @payload_schema.register

invokeai/app/services/image_files/image_files_disk.py

@@ -110,26 +110,15 @@ class DiskImageFileStorage(ImageFileStorageBase):
         except Exception as e:
             raise ImageFileDeleteException from e
 
+    # TODO: make this a bit more flexible for e.g. cloud storage
     def get_path(self, image_name: str, thumbnail: bool = False) -> Path:
-        base_folder = self.__thumbnails_folder if thumbnail else self.__output_folder
-        filename = get_thumbnail_name(image_name) if thumbnail else image_name
+        path = self.__output_folder / image_name
 
-        # Strip any path information from the filename
-        basename = Path(filename).name
+        if thumbnail:
+            thumbnail_name = get_thumbnail_name(image_name)
+            path = self.__thumbnails_folder / thumbnail_name
 
-        if basename != filename:
-            raise ValueError("Invalid image name, potential directory traversal detected")
-
-        image_path = base_folder / basename
-
-        # Ensure the image path is within the base folder to prevent directory traversal
-        resolved_base = base_folder.resolve()
-        resolved_image_path = image_path.resolve()
-
-        if not resolved_image_path.is_relative_to(resolved_base):
-            raise ValueError("Image path outside outputs folder, potential directory traversal detected")
-
-        return resolved_image_path
+        return path
 
     def validate_path(self, path: Union[str, Path]) -> bool:
         """Validates the path given for an image or thumbnail."""

invokeai/app/services/model_load/model_load_base.py

@@ -14,9 +14,7 @@ class ModelLoadServiceBase(ABC):
     """Wrapper around AnyModelLoader."""
 
     @abstractmethod
-    def load_model(
-        self, model_config: AnyModelConfig, queue_id: str, submodel_type: Optional[SubModelType] = None
-    ) -> LoadedModel:
+    def load_model(self, model_config: AnyModelConfig, submodel_type: Optional[SubModelType] = None) -> LoadedModel:
         """
         Given a model's configuration, load it and return the LoadedModel object.
 
@@ -31,7 +29,7 @@ class ModelLoadServiceBase(ABC):
 
     @abstractmethod
     def load_model_from_path(
-        self, model_path: Path, queue_id: str, loader: Optional[Callable[[Path], AnyModel]] = None
+        self, model_path: Path, loader: Optional[Callable[[Path], AnyModel]] = None
     ) -> LoadedModelWithoutConfig:
         """
         Load the model file or directory located at the indicated Path.

invokeai/app/services/model_load/model_load_default.py

@@ -49,9 +49,7 @@ class ModelLoadService(ModelLoadServiceBase):
         """Return the RAM cache used by this loader."""
         return self._ram_cache
 
-    def load_model(
-        self, model_config: AnyModelConfig, queue_id: str, submodel_type: Optional[SubModelType] = None
-    ) -> LoadedModel:
+    def load_model(self, model_config: AnyModelConfig, submodel_type: Optional[SubModelType] = None) -> LoadedModel:
         """
         Given a model's configuration, load it and return the LoadedModel object.
 
@@ -62,7 +60,7 @@ class ModelLoadService(ModelLoadServiceBase):
         # We don't have an invoker during testing
         # TODO(psyche): Mock this method on the invoker in the tests
         if hasattr(self, "_invoker"):
-            self._invoker.services.events.emit_model_load_started(model_config, queue_id, submodel_type)
+            self._invoker.services.events.emit_model_load_started(model_config, submodel_type)
 
         implementation, model_config, submodel_type = self._registry.get_implementation(model_config, submodel_type)  # type: ignore
         loaded_model: LoadedModel = implementation(
@@ -72,12 +70,12 @@ class ModelLoadService(ModelLoadServiceBase):
         ).load_model(model_config, submodel_type)
 
         if hasattr(self, "_invoker"):
-            self._invoker.services.events.emit_model_load_complete(model_config, queue_id, submodel_type)
+            self._invoker.services.events.emit_model_load_complete(model_config, submodel_type)
 
         return loaded_model
 
     def load_model_from_path(
-        self, model_path: Path, queue_id: str, loader: Optional[Callable[[Path], AnyModel]] = None
+        self, model_path: Path, loader: Optional[Callable[[Path], AnyModel]] = None
     ) -> LoadedModelWithoutConfig:
         cache_key = str(model_path)
         ram_cache = self.ram_cache

invokeai/app/services/model_records/model_records_base.py

@@ -15,7 +15,6 @@ from invokeai.app.util.model_exclude_null import BaseModelExcludeNull
 from invokeai.backend.model_manager.config import (
     AnyModelConfig,
     BaseModelType,
-    ClipVariantType,
     ControlAdapterDefaultSettings,
     MainModelDefaultSettings,
     ModelFormat,
@@ -86,7 +85,7 @@ class ModelRecordChanges(BaseModelExcludeNull):
 
     # Checkpoint-specific changes
     # TODO(MM2): Should we expose these? Feels footgun-y...
-    variant: Optional[ModelVariantType | ClipVariantType] = Field(description="The variant of the model.", default=None)
+    variant: Optional[ModelVariantType] = Field(description="The variant of the model.", default=None)
     prediction_type: Optional[SchedulerPredictionType] = Field(
         description="The prediction type of the model.", default=None
     )

invokeai/app/services/shared/invocation_context.py

@@ -1,4 +1,3 @@
-from copy import deepcopy
 from dataclasses import dataclass
 from pathlib import Path
 from typing import TYPE_CHECKING, Callable, Optional, Union
@@ -222,7 +221,7 @@ class ImagesInterface(InvocationContextInterface):
         )
 
     def get_pil(self, image_name: str, mode: IMAGE_MODES | None = None) -> Image:
-        """Gets an image as a PIL Image object. This method returns a copy of the image.
+        """Gets an image as a PIL Image object.
 
         Args:
             image_name: The name of the image to get.
@@ -234,15 +233,11 @@ class ImagesInterface(InvocationContextInterface):
         image = self._services.images.get_pil_image(image_name)
         if mode and mode != image.mode:
             try:
-                # convert makes a copy!
                 image = image.convert(mode)
             except ValueError:
                 self._services.logger.warning(
                     f"Could not convert image from {image.mode} to {mode}. Using original mode instead."
                 )
-        else:
-            # copy the image to prevent the user from modifying the original
-            image = image.copy()
         return image
 
     def get_metadata(self, image_name: str) -> Optional[MetadataField]:
@@ -295,15 +290,15 @@ class TensorsInterface(InvocationContextInterface):
         return name
 
     def load(self, name: str) -> Tensor:
-        """Loads a tensor by name. This method returns a copy of the tensor.
+        """Loads a tensor by name.
 
         Args:
             name: The name of the tensor to load.
 
         Returns:
-            The tensor.
+            The loaded tensor.
         """
-        return self._services.tensors.load(name).clone()
+        return self._services.tensors.load(name)
 
 
 class ConditioningInterface(InvocationContextInterface):
@@ -321,16 +316,16 @@ class ConditioningInterface(InvocationContextInterface):
         return name
 
     def load(self, name: str) -> ConditioningFieldData:
-        """Loads conditioning data by name. This method returns a copy of the conditioning data.
+        """Loads conditioning data by name.
 
         Args:
             name: The name of the conditioning data to load.
 
         Returns:
-            The conditioning data.
+            The loaded conditioning data.
         """
 
-        return deepcopy(self._services.conditioning.load(name))
+        return self._services.conditioning.load(name)
 
 
 class ModelsInterface(InvocationContextInterface):
@@ -351,10 +346,7 @@ class ModelsInterface(InvocationContextInterface):
             return self._services.model_manager.store.exists(identifier.key)
 
     def load(
-        self,
-        identifier: Union[str, "ModelIdentifierField"],
-        queue_id: str,
-        submodel_type: Optional[SubModelType] = None,
+        self, identifier: Union[str, "ModelIdentifierField"], submodel_type: Optional[SubModelType] = None
     ) -> LoadedModel:
         """Load a model.
 
@@ -371,19 +363,14 @@ class ModelsInterface(InvocationContextInterface):
 
         if isinstance(identifier, str):
             model = self._services.model_manager.store.get_model(identifier)
-            return self._services.model_manager.load.load_model(model, queue_id, submodel_type)
+            return self._services.model_manager.load.load_model(model, submodel_type)
         else:
             _submodel_type = submodel_type or identifier.submodel_type
             model = self._services.model_manager.store.get_model(identifier.key)
-            return self._services.model_manager.load.load_model(model, queue_id, _submodel_type)
+            return self._services.model_manager.load.load_model(model, _submodel_type)
 
     def load_by_attrs(
-        self,
-        name: str,
-        base: BaseModelType,
-        type: ModelType,
-        queue_id: str,
-        submodel_type: Optional[SubModelType] = None,
+        self, name: str, base: BaseModelType, type: ModelType, submodel_type: Optional[SubModelType] = None
     ) -> LoadedModel:
         """Load a model by its attributes.
 
@@ -405,7 +392,7 @@ class ModelsInterface(InvocationContextInterface):
         if len(configs) > 1:
             raise ValueError(f"More than one model found with name {name}, base {base}, and type {type}")
 
-        return self._services.model_manager.load.load_model(configs[0], queue_id, submodel_type)
+        return self._services.model_manager.load.load_model(configs[0], submodel_type)
 
     def get_config(self, identifier: Union[str, "ModelIdentifierField"]) -> AnyModelConfig:
         """Get a model's config.
@@ -480,7 +467,6 @@ class ModelsInterface(InvocationContextInterface):
     def load_local_model(
         self,
         model_path: Path,
-        queue_id: str,
         loader: Optional[Callable[[Path], AnyModel]] = None,
     ) -> LoadedModelWithoutConfig:
         """
@@ -498,14 +484,11 @@ class ModelsInterface(InvocationContextInterface):
         Returns:
             A LoadedModelWithoutConfig object.
         """
-        return self._services.model_manager.load.load_model_from_path(
-            model_path=model_path, queue_id=queue_id, loader=loader
-        )
+        return self._services.model_manager.load.load_model_from_path(model_path=model_path, loader=loader)
 
     def load_remote_model(
         self,
         source: str | AnyHttpUrl,
-        queue_id: str,
         loader: Optional[Callable[[Path], AnyModel]] = None,
     ) -> LoadedModelWithoutConfig:
         """
@@ -526,9 +509,7 @@ class ModelsInterface(InvocationContextInterface):
             A LoadedModelWithoutConfig object.
         """
         model_path = self._services.model_manager.install.download_and_cache_model(source=str(source))
-        return self._services.model_manager.load.load_model_from_path(
-            model_path=model_path, queue_id=queue_id, loader=loader
-        )
+        return self._services.model_manager.load.load_model_from_path(model_path=model_path, loader=loader)
 
 
 class ConfigInterface(InvocationContextInterface):
@@ -549,14 +530,6 @@ class UtilInterface(InvocationContextInterface):
         super().__init__(services, data)
         self._is_canceled = is_canceled
 
-    def get_queue_id(self) -> str:
-        """Checks if the current session has been canceled.
-
-        Returns:
-            True if the current session has been canceled, False if not.
-        """
-        return self._data.queue_item.queue_id
-
     def is_canceled(self) -> bool:
         """Checks if the current session has been canceled.
 

invokeai/app/services/workflow_records/default_workflows/SD3.5 Text to Image.json

@@ -1,382 +0,0 @@
-{
-    "name": "SD3.5 Text to Image",
-   "author": "InvokeAI",
-    "description": "Sample text to image workflow for Stable Diffusion 3.5",
-    "version": "1.0.0",
-    "contact": "invoke@invoke.ai",
-    "tags": "text2image, SD3.5, default",
-  "notes": "",
-    "exposedFields": [
-      {
-        "nodeId": "3f22f668-0e02-4fde-a2bb-c339586ceb4c",
-        "fieldName": "model"
-      },
-      {
-        "nodeId": "e17d34e7-6ed1-493c-9a85-4fcd291cb084",
-        "fieldName": "prompt"
-      }
-    ],
-    "meta": {
-      "version": "3.0.0",
-      "category": "default"
-    },
-    "id": "e3a51d6b-8208-4d6d-b187-fcfe8b32934c",
-    "nodes": [
-      {
-        "id": "3f22f668-0e02-4fde-a2bb-c339586ceb4c",
-        "type": "invocation",
-        "data": {
-          "id": "3f22f668-0e02-4fde-a2bb-c339586ceb4c",
-          "type": "sd3_model_loader",
-          "version": "1.0.0",
-          "label": "",
-          "notes": "",
-          "isOpen": true,
-          "isIntermediate": true,
-          "useCache": true,
-          "nodePack": "invokeai",
-          "inputs": {
-            "model": {
-              "name": "model",
-              "label": "",
-              "value": {
-                "key": "f7b20be9-92a8-4cfb-bca4-6c3b5535c10b",
-                "hash": "placeholder",
-                "name": "stable-diffusion-3.5-medium",
-                "base": "sd-3",
-                "type": "main"
-              }
-            },
-            "t5_encoder_model": {
-              "name": "t5_encoder_model",
-              "label": ""
-            },
-            "clip_l_model": {
-              "name": "clip_l_model",
-              "label": ""
-            },
-            "clip_g_model": {
-              "name": "clip_g_model",
-              "label": ""
-            },
-            "vae_model": {
-              "name": "vae_model",
-              "label": ""
-            }
-          }
-        },
-        "position": {
-          "x": -55.58689609637031,
-          "y": -111.53602444662268
-        }
-      },
-      {
-        "id": "f7e394ac-6394-4096-abcb-de0d346506b3",
-        "type": "invocation",
-        "data": {
-          "id": "f7e394ac-6394-4096-abcb-de0d346506b3",
-          "type": "rand_int",
-          "version": "1.0.1",
-          "label": "",
-          "notes": "",
-          "isOpen": true,
-          "isIntermediate": true,
-          "useCache": false,
-          "nodePack": "invokeai",
-          "inputs": {
-            "low": {
-              "name": "low",
-              "label": "",
-              "value": 0
-            },
-            "high": {
-              "name": "high",
-              "label": "",
-              "value": 2147483647
-            }
-          }
-        },
-        "position": {
-          "x": 470.45870147220353,
-          "y": 350.3141781644303
-        }
-      },
-      {
-        "id": "9eb72af0-dd9e-4ec5-ad87-d65e3c01f48b",
-        "type": "invocation",
-        "data": {
-          "id": "9eb72af0-dd9e-4ec5-ad87-d65e3c01f48b",
-          "type": "sd3_l2i",
-          "version": "1.3.0",
-          "label": "",
-          "notes": "",
-          "isOpen": true,
-          "isIntermediate": false,
-          "useCache": true,
-          "nodePack": "invokeai",
-          "inputs": {
-            "board": {
-              "name": "board",
-              "label": ""
-            },
-            "metadata": {
-              "name": "metadata",
-              "label": ""
-            },
-            "latents": {
-              "name": "latents",
-              "label": ""
-            },
-            "vae": {
-              "name": "vae",
-              "label": ""
-            }
-          }
-        },
-        "position": {
-          "x": 1192.3097009334897,
-          "y": -366.0994675072209
-        }
-      },
-      {
-        "id": "3b4f7f27-cfc0-4373-a009-99c5290d0cd6",
-        "type": "invocation",
-        "data": {
-          "id": "3b4f7f27-cfc0-4373-a009-99c5290d0cd6",
-          "type": "sd3_text_encoder",
-          "version": "1.0.0",
-          "label": "",
-          "notes": "",
-          "isOpen": true,
-          "isIntermediate": true,
-          "useCache": true,
-          "nodePack": "invokeai",
-          "inputs": {
-            "clip_l": {
-              "name": "clip_l",
-              "label": ""
-            },
-            "clip_g": {
-              "name": "clip_g",
-              "label": ""
-            },
-            "t5_encoder": {
-              "name": "t5_encoder",
-              "label": ""
-            },
-            "prompt": {
-              "name": "prompt",
-              "label": "",
-              "value": ""
-            }
-          }
-        },
-        "position": {
-          "x": 408.16054647924784,
-          "y": 65.06415352118786
-        }
-      },
-      {
-        "id": "e17d34e7-6ed1-493c-9a85-4fcd291cb084",
-        "type": "invocation",
-        "data": {
-          "id": "e17d34e7-6ed1-493c-9a85-4fcd291cb084",
-          "type": "sd3_text_encoder",
-          "version": "1.0.0",
-          "label": "",
-          "notes": "",
-          "isOpen": true,
-          "isIntermediate": true,
-          "useCache": true,
-          "nodePack": "invokeai",
-          "inputs": {
-            "clip_l": {
-              "name": "clip_l",
-              "label": ""
-            },
-            "clip_g": {
-              "name": "clip_g",
-              "label": ""
-            },
-            "t5_encoder": {
-              "name": "t5_encoder",
-              "label": ""
-            },
-            "prompt": {
-              "name": "prompt",
-              "label": "",
-              "value": ""
-            }
-          }
-        },
-        "position": {
-          "x": 378.9283412440941,
-          "y": -302.65777497352553
-        }
-      },
-      {
-        "id": "c7539f7b-7ac5-49b9-93eb-87ede611409f",
-        "type": "invocation",
-        "data": {
-          "id": "c7539f7b-7ac5-49b9-93eb-87ede611409f",
-          "type": "sd3_denoise",
-          "version": "1.0.0",
-          "label": "",
-          "notes": "",
-          "isOpen": true,
-          "isIntermediate": true,
-          "useCache": true,
-          "nodePack": "invokeai",
-          "inputs": {
-            "board": {
-              "name": "board",
-              "label": ""
-            },
-            "metadata": {
-              "name": "metadata",
-              "label": ""
-            },
-            "transformer": {
-              "name": "transformer",
-              "label": ""
-            },
-            "positive_conditioning": {
-              "name": "positive_conditioning",
-              "label": ""
-            },
-            "negative_conditioning": {
-              "name": "negative_conditioning",
-              "label": ""
-            },
-            "cfg_scale": {
-              "name": "cfg_scale",
-              "label": "",
-              "value": 3.5
-            },
-            "width": {
-              "name": "width",
-              "label": "",
-              "value": 1024
-            },
-            "height": {
-              "name": "height",
-              "label": "",
-              "value": 1024
-            },
-            "steps": {
-              "name": "steps",
-              "label": "",
-              "value": 30
-            },
-            "seed": {
-              "name": "seed",
-              "label": "",
-              "value": 0
-            }
-          }
-        },
-        "position": {
-          "x": 813.7814762740603,
-          "y": -142.20529727605867
-        }
-      }
-    ],
-    "edges": [
-      {
-        "id": "reactflow__edge-3f22f668-0e02-4fde-a2bb-c339586ceb4cvae-9eb72af0-dd9e-4ec5-ad87-d65e3c01f48bvae",
-        "type": "default",
-        "source": "3f22f668-0e02-4fde-a2bb-c339586ceb4c",
-        "target": "9eb72af0-dd9e-4ec5-ad87-d65e3c01f48b",
-        "sourceHandle": "vae",
-        "targetHandle": "vae"
-      },
-      {
-        "id": "reactflow__edge-3f22f668-0e02-4fde-a2bb-c339586ceb4ct5_encoder-3b4f7f27-cfc0-4373-a009-99c5290d0cd6t5_encoder",
-        "type": "default",
-        "source": "3f22f668-0e02-4fde-a2bb-c339586ceb4c",
-        "target": "3b4f7f27-cfc0-4373-a009-99c5290d0cd6",
-        "sourceHandle": "t5_encoder",
-        "targetHandle": "t5_encoder"
-      },
-      {
-        "id": "reactflow__edge-3f22f668-0e02-4fde-a2bb-c339586ceb4ct5_encoder-e17d34e7-6ed1-493c-9a85-4fcd291cb084t5_encoder",
-        "type": "default",
-        "source": "3f22f668-0e02-4fde-a2bb-c339586ceb4c",
-        "target": "e17d34e7-6ed1-493c-9a85-4fcd291cb084",
-        "sourceHandle": "t5_encoder",
-        "targetHandle": "t5_encoder"
-      },
-      {
-        "id": "reactflow__edge-3f22f668-0e02-4fde-a2bb-c339586ceb4cclip_g-3b4f7f27-cfc0-4373-a009-99c5290d0cd6clip_g",
-        "type": "default",
-        "source": "3f22f668-0e02-4fde-a2bb-c339586ceb4c",
-        "target": "3b4f7f27-cfc0-4373-a009-99c5290d0cd6",
-        "sourceHandle": "clip_g",
-        "targetHandle": "clip_g"
-      },
-      {
-        "id": "reactflow__edge-3f22f668-0e02-4fde-a2bb-c339586ceb4cclip_g-e17d34e7-6ed1-493c-9a85-4fcd291cb084clip_g",
-        "type": "default",
-        "source": "3f22f668-0e02-4fde-a2bb-c339586ceb4c",
-        "target": "e17d34e7-6ed1-493c-9a85-4fcd291cb084",
-        "sourceHandle": "clip_g",
-        "targetHandle": "clip_g"
-      },
-      {
-        "id": "reactflow__edge-3f22f668-0e02-4fde-a2bb-c339586ceb4cclip_l-3b4f7f27-cfc0-4373-a009-99c5290d0cd6clip_l",
-        "type": "default",
-        "source": "3f22f668-0e02-4fde-a2bb-c339586ceb4c",
-        "target": "3b4f7f27-cfc0-4373-a009-99c5290d0cd6",
-        "sourceHandle": "clip_l",
-        "targetHandle": "clip_l"
-      },
-      {
-        "id": "reactflow__edge-3f22f668-0e02-4fde-a2bb-c339586ceb4cclip_l-e17d34e7-6ed1-493c-9a85-4fcd291cb084clip_l",
-        "type": "default",
-        "source": "3f22f668-0e02-4fde-a2bb-c339586ceb4c",
-        "target": "e17d34e7-6ed1-493c-9a85-4fcd291cb084",
-        "sourceHandle": "clip_l",
-        "targetHandle": "clip_l"
-      },
-      {
-        "id": "reactflow__edge-3f22f668-0e02-4fde-a2bb-c339586ceb4ctransformer-c7539f7b-7ac5-49b9-93eb-87ede611409ftransformer",
-        "type": "default",
-        "source": "3f22f668-0e02-4fde-a2bb-c339586ceb4c",
-        "target": "c7539f7b-7ac5-49b9-93eb-87ede611409f",
-        "sourceHandle": "transformer",
-        "targetHandle": "transformer"
-      },
-      {
-        "id": "reactflow__edge-f7e394ac-6394-4096-abcb-de0d346506b3value-c7539f7b-7ac5-49b9-93eb-87ede611409fseed",
-        "type": "default",
-        "source": "f7e394ac-6394-4096-abcb-de0d346506b3",
-        "target": "c7539f7b-7ac5-49b9-93eb-87ede611409f",
-        "sourceHandle": "value",
-        "targetHandle": "seed"
-      },
-      {
-        "id": "reactflow__edge-c7539f7b-7ac5-49b9-93eb-87ede611409flatents-9eb72af0-dd9e-4ec5-ad87-d65e3c01f48blatents",
-        "type": "default",
-        "source": "c7539f7b-7ac5-49b9-93eb-87ede611409f",
-        "target": "9eb72af0-dd9e-4ec5-ad87-d65e3c01f48b",
-        "sourceHandle": "latents",
-        "targetHandle": "latents"
-      },
-      {
-        "id": "reactflow__edge-e17d34e7-6ed1-493c-9a85-4fcd291cb084conditioning-c7539f7b-7ac5-49b9-93eb-87ede611409fpositive_conditioning",
-        "type": "default",
-        "source": "e17d34e7-6ed1-493c-9a85-4fcd291cb084",
-        "target": "c7539f7b-7ac5-49b9-93eb-87ede611409f",
-        "sourceHandle": "conditioning",
-        "targetHandle": "positive_conditioning"
-      },
-      {
-        "id": "reactflow__edge-3b4f7f27-cfc0-4373-a009-99c5290d0cd6conditioning-c7539f7b-7ac5-49b9-93eb-87ede611409fnegative_conditioning",
-        "type": "default",
-        "source": "3b4f7f27-cfc0-4373-a009-99c5290d0cd6",
-        "target": "c7539f7b-7ac5-49b9-93eb-87ede611409f",
-        "sourceHandle": "conditioning",
-        "targetHandle": "negative_conditioning"
-      }
-    ]
-  }

invokeai/app/util/step_callback.py

@@ -34,25 +34,6 @@ SD1_5_LATENT_RGB_FACTORS = [
     [-0.1307, -0.1874, -0.7445],  # L4
 ]
 
-SD3_5_LATENT_RGB_FACTORS = [
-    [-0.05240681, 0.03251581, 0.0749016],
-    [-0.0580572, 0.00759826, 0.05729818],
-    [0.16144888, 0.01270368, -0.03768577],
-    [0.14418615, 0.08460266, 0.15941818],
-    [0.04894035, 0.0056485, -0.06686988],
-    [0.05187166, 0.19222395, 0.06261094],
-    [0.1539433, 0.04818359, 0.07103094],
-    [-0.08601796, 0.09013458, 0.10893912],
-    [-0.12398469, -0.06766567, 0.0033688],
-    [-0.0439737, 0.07825329, 0.02258823],
-    [0.03101129, 0.06382551, 0.07753657],
-    [-0.01315361, 0.08554491, -0.08772475],
-    [0.06464487, 0.05914605, 0.13262741],
-    [-0.07863674, -0.02261737, -0.12761454],
-    [-0.09923835, -0.08010759, -0.06264447],
-    [-0.03392309, -0.0804029, -0.06078822],
-]
-
 FLUX_LATENT_RGB_FACTORS = [
     [-0.0412, 0.0149, 0.0521],
     [0.0056, 0.0291, 0.0768],
@@ -129,9 +110,6 @@ def stable_diffusion_step_callback(
         sdxl_latent_rgb_factors = torch.tensor(SDXL_LATENT_RGB_FACTORS, dtype=sample.dtype, device=sample.device)
         sdxl_smooth_matrix = torch.tensor(SDXL_SMOOTH_MATRIX, dtype=sample.dtype, device=sample.device)
         image = sample_to_lowres_estimated_image(sample, sdxl_latent_rgb_factors, sdxl_smooth_matrix)
-    elif base_model == BaseModelType.StableDiffusion3:
-        sd3_latent_rgb_factors = torch.tensor(SD3_5_LATENT_RGB_FACTORS, dtype=sample.dtype, device=sample.device)
-        image = sample_to_lowres_estimated_image(sample, sd3_latent_rgb_factors)
     else:
         v1_5_latent_rgb_factors = torch.tensor(SD1_5_LATENT_RGB_FACTORS, dtype=sample.dtype, device=sample.device)
         image = sample_to_lowres_estimated_image(sample, v1_5_latent_rgb_factors)

invokeai/app/util/ti_utils.py

@@ -22,7 +22,7 @@ def generate_ti_list(
     for trigger in extract_ti_triggers_from_prompt(prompt):
         name_or_key = trigger[1:-1]
         try:
-            loaded_model = context.models.load(name_or_key, queue_id=context.util.get_queue_id())
+            loaded_model = context.models.load(name_or_key)
             model = loaded_model.model
             assert isinstance(model, TextualInversionModelRaw)
             assert loaded_model.config.base == base
@@ -30,7 +30,7 @@ def generate_ti_list(
         except UnknownModelException:
             try:
                 loaded_model = context.models.load_by_attrs(
-                    name=name_or_key, base=base, type=ModelType.TextualInversion, queue_id=context.util.get_queue_id()
+                    name=name_or_key, base=base, type=ModelType.TextualInversion
                 )
                 model = loaded_model.model
                 assert isinstance(model, TextualInversionModelRaw)

invokeai/backend/flux/custom_block_processor.py

@@ -1,83 +0,0 @@
-import einops
-import torch
-
-from invokeai.backend.flux.extensions.xlabs_ip_adapter_extension import XLabsIPAdapterExtension
-from invokeai.backend.flux.math import attention
-from invokeai.backend.flux.modules.layers import DoubleStreamBlock
-
-
-class CustomDoubleStreamBlockProcessor:
-    """A class containing a custom implementation of DoubleStreamBlock.forward() with additional features
-    (IP-Adapter, etc.).
-    """
-
-    @staticmethod
-    def _double_stream_block_forward(
-        block: DoubleStreamBlock, img: torch.Tensor, txt: torch.Tensor, vec: torch.Tensor, pe: torch.Tensor
-    ) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
-        """This function is a direct copy of DoubleStreamBlock.forward(), but it returns some of the intermediate
-        values.
-        """
-        img_mod1, img_mod2 = block.img_mod(vec)
-        txt_mod1, txt_mod2 = block.txt_mod(vec)
-
-        # prepare image for attention
-        img_modulated = block.img_norm1(img)
-        img_modulated = (1 + img_mod1.scale) * img_modulated + img_mod1.shift
-        img_qkv = block.img_attn.qkv(img_modulated)
-        img_q, img_k, img_v = einops.rearrange(img_qkv, "B L (K H D) -> K B H L D", K=3, H=block.num_heads)
-        img_q, img_k = block.img_attn.norm(img_q, img_k, img_v)
-
-        # prepare txt for attention
-        txt_modulated = block.txt_norm1(txt)
-        txt_modulated = (1 + txt_mod1.scale) * txt_modulated + txt_mod1.shift
-        txt_qkv = block.txt_attn.qkv(txt_modulated)
-        txt_q, txt_k, txt_v = einops.rearrange(txt_qkv, "B L (K H D) -> K B H L D", K=3, H=block.num_heads)
-        txt_q, txt_k = block.txt_attn.norm(txt_q, txt_k, txt_v)
-
-        # run actual attention
-        q = torch.cat((txt_q, img_q), dim=2)
-        k = torch.cat((txt_k, img_k), dim=2)
-        v = torch.cat((txt_v, img_v), dim=2)
-
-        attn = attention(q, k, v, pe=pe)
-        txt_attn, img_attn = attn[:, : txt.shape[1]], attn[:, txt.shape[1] :]
-
-        # calculate the img bloks
-        img = img + img_mod1.gate * block.img_attn.proj(img_attn)
-        img = img + img_mod2.gate * block.img_mlp((1 + img_mod2.scale) * block.img_norm2(img) + img_mod2.shift)
-
-        # calculate the txt bloks
-        txt = txt + txt_mod1.gate * block.txt_attn.proj(txt_attn)
-        txt = txt + txt_mod2.gate * block.txt_mlp((1 + txt_mod2.scale) * block.txt_norm2(txt) + txt_mod2.shift)
-        return img, txt, img_q
-
-    @staticmethod
-    def custom_double_block_forward(
-        timestep_index: int,
-        total_num_timesteps: int,
-        block_index: int,
-        block: DoubleStreamBlock,
-        img: torch.Tensor,
-        txt: torch.Tensor,
-        vec: torch.Tensor,
-        pe: torch.Tensor,
-        ip_adapter_extensions: list[XLabsIPAdapterExtension],
-    ) -> tuple[torch.Tensor, torch.Tensor]:
-        """A custom implementation of DoubleStreamBlock.forward() with additional features:
-        - IP-Adapter support
-        """
-        img, txt, img_q = CustomDoubleStreamBlockProcessor._double_stream_block_forward(block, img, txt, vec, pe)
-
-        # Apply IP-Adapter conditioning.
-        for ip_adapter_extension in ip_adapter_extensions:
-            img = ip_adapter_extension.run_ip_adapter(
-                timestep_index=timestep_index,
-                total_num_timesteps=total_num_timesteps,
-                block_index=block_index,
-                block=block,
-                img_q=img_q,
-                img=img,
-            )
-
-        return img, txt

invokeai/backend/flux/denoise.py

@@ -1,4 +1,3 @@
-import math
 from typing import Callable
 
 import torch
@@ -8,7 +7,6 @@ from invokeai.backend.flux.controlnet.controlnet_flux_output import ControlNetFl
 from invokeai.backend.flux.extensions.inpaint_extension import InpaintExtension
 from invokeai.backend.flux.extensions.instantx_controlnet_extension import InstantXControlNetExtension
 from invokeai.backend.flux.extensions.xlabs_controlnet_extension import XLabsControlNetExtension
-from invokeai.backend.flux.extensions.xlabs_ip_adapter_extension import XLabsIPAdapterExtension
 from invokeai.backend.flux.model import Flux
 from invokeai.backend.stable_diffusion.diffusers_pipeline import PipelineIntermediateState
 
@@ -18,23 +16,15 @@ def denoise(
     # model input
     img: torch.Tensor,
     img_ids: torch.Tensor,
-    # positive text conditioning
     txt: torch.Tensor,
     txt_ids: torch.Tensor,
     vec: torch.Tensor,
-    # negative text conditioning
-    neg_txt: torch.Tensor | None,
-    neg_txt_ids: torch.Tensor | None,
-    neg_vec: torch.Tensor | None,
     # sampling parameters
     timesteps: list[float],
     step_callback: Callable[[PipelineIntermediateState], None],
     guidance: float,
-    cfg_scale: list[float],
     inpaint_extension: InpaintExtension | None,
     controlnet_extensions: list[XLabsControlNetExtension | InstantXControlNetExtension],
-    pos_ip_adapter_extensions: list[XLabsIPAdapterExtension],
-    neg_ip_adapter_extensions: list[XLabsIPAdapterExtension],
 ):
     # step 0 is the initial state
     total_steps = len(timesteps) - 1
@@ -47,9 +37,10 @@ def denoise(
             latents=img,
         ),
     )
+    step = 1
     # guidance_vec is ignored for schnell.
     guidance_vec = torch.full((img.shape[0],), guidance, device=img.device, dtype=img.dtype)
-    for step_index, (t_curr, t_prev) in tqdm(list(enumerate(zip(timesteps[:-1], timesteps[1:], strict=True)))):
+    for t_curr, t_prev in tqdm(list(zip(timesteps[:-1], timesteps[1:], strict=True))):
         t_vec = torch.full((img.shape[0],), t_curr, dtype=img.dtype, device=img.device)
 
         # Run ControlNet models.
@@ -57,7 +48,7 @@ def denoise(
         for controlnet_extension in controlnet_extensions:
             controlnet_residuals.append(
                 controlnet_extension.run_controlnet(
-                    timestep_index=step_index,
+                    timestep_index=step - 1,
                     total_num_timesteps=total_steps,
                     img=img,
                     img_ids=img_ids,
@@ -70,7 +61,7 @@ def denoise(
             )
 
         # Merge the ControlNet residuals from multiple ControlNets.
-        # TODO(ryand): We may want to calculate the sum just-in-time to keep peak memory low. Keep in mind, that the
+        # TODO(ryand): We may want to alculate the sum just-in-time to keep peak memory low. Keep in mind, that the
         # controlnet_residuals datastructure is efficient in that it likely contains multiple references to the same
         # tensors. Calculating the sum materializes each tensor into its own instance.
         merged_controlnet_residuals = sum_controlnet_flux_outputs(controlnet_residuals)
@@ -83,39 +74,10 @@ def denoise(
             y=vec,
             timesteps=t_vec,
             guidance=guidance_vec,
-            timestep_index=step_index,
-            total_num_timesteps=total_steps,
             controlnet_double_block_residuals=merged_controlnet_residuals.double_block_residuals,
             controlnet_single_block_residuals=merged_controlnet_residuals.single_block_residuals,
-            ip_adapter_extensions=pos_ip_adapter_extensions,
         )
 
-        step_cfg_scale = cfg_scale[step_index]
-
-        # If step_cfg_scale, is 1.0, then we don't need to run the negative prediction.
-        if not math.isclose(step_cfg_scale, 1.0):
-            # TODO(ryand): Add option to run positive and negative predictions in a single batch for better performance
-            # on systems with sufficient VRAM.
-
-            if neg_txt is None or neg_txt_ids is None or neg_vec is None:
-                raise ValueError("Negative text conditioning is required when cfg_scale is not 1.0.")
-
-            neg_pred = model(
-                img=img,
-                img_ids=img_ids,
-                txt=neg_txt,
-                txt_ids=neg_txt_ids,
-                y=neg_vec,
-                timesteps=t_vec,
-                guidance=guidance_vec,
-                timestep_index=step_index,
-                total_num_timesteps=total_steps,
-                controlnet_double_block_residuals=None,
-                controlnet_single_block_residuals=None,
-                ip_adapter_extensions=neg_ip_adapter_extensions,
-            )
-            pred = neg_pred + step_cfg_scale * (pred - neg_pred)
-
         preview_img = img - t_curr * pred
         img = img + (t_prev - t_curr) * pred
 
@@ -125,12 +87,13 @@ def denoise(
 
         step_callback(
             PipelineIntermediateState(
-                step=step_index + 1,
+                step=step,
                 order=1,
                 total_steps=total_steps,
                 timestep=int(t_curr),
                 latents=preview_img,
             ),
         )
+        step += 1
 
     return img

invokeai/backend/flux/extensions/xlabs_ip_adapter_extension.py

@@ -1,89 +0,0 @@
-import math
-from typing import List, Union
-
-import einops
-import torch
-from PIL import Image
-from transformers import CLIPImageProcessor, CLIPVisionModelWithProjection
-
-from invokeai.backend.flux.ip_adapter.xlabs_ip_adapter_flux import XlabsIpAdapterFlux
-from invokeai.backend.flux.modules.layers import DoubleStreamBlock
-
-
-class XLabsIPAdapterExtension:
-    def __init__(
-        self,
-        model: XlabsIpAdapterFlux,
-        image_prompt_clip_embed: torch.Tensor,
-        weight: Union[float, List[float]],
-        begin_step_percent: float,
-        end_step_percent: float,
-    ):
-        self._model = model
-        self._image_prompt_clip_embed = image_prompt_clip_embed
-        self._weight = weight
-        self._begin_step_percent = begin_step_percent
-        self._end_step_percent = end_step_percent
-
-        self._image_proj: torch.Tensor | None = None
-
-    def _get_weight(self, timestep_index: int, total_num_timesteps: int) -> float:
-        first_step = math.floor(self._begin_step_percent * total_num_timesteps)
-        last_step = math.ceil(self._end_step_percent * total_num_timesteps)
-
-        if timestep_index < first_step or timestep_index > last_step:
-            return 0.0
-
-        if isinstance(self._weight, list):
-            return self._weight[timestep_index]
-
-        return self._weight
-
-    @staticmethod
-    def run_clip_image_encoder(
-        pil_image: List[Image.Image], image_encoder: CLIPVisionModelWithProjection
-    ) -> torch.Tensor:
-        clip_image_processor = CLIPImageProcessor()
-        clip_image: torch.Tensor = clip_image_processor(images=pil_image, return_tensors="pt").pixel_values
-        clip_image = clip_image.to(device=image_encoder.device, dtype=image_encoder.dtype)
-        clip_image_embeds = image_encoder(clip_image).image_embeds
-        return clip_image_embeds
-
-    def run_image_proj(self, dtype: torch.dtype):
-        image_prompt_clip_embed = self._image_prompt_clip_embed.to(dtype=dtype)
-        self._image_proj = self._model.image_proj(image_prompt_clip_embed)
-
-    def run_ip_adapter(
-        self,
-        timestep_index: int,
-        total_num_timesteps: int,
-        block_index: int,
-        block: DoubleStreamBlock,
-        img_q: torch.Tensor,
-        img: torch.Tensor,
-    ) -> torch.Tensor:
-        """The logic in this function is based on:
-        https://github.com/XLabs-AI/x-flux/blob/47495425dbed499be1e8e5a6e52628b07349cba2/src/flux/modules/layers.py#L245-L301
-        """
-        weight = self._get_weight(timestep_index=timestep_index, total_num_timesteps=total_num_timesteps)
-        if weight < 1e-6:
-            return img
-
-        ip_adapter_block = self._model.ip_adapter_double_blocks.double_blocks[block_index]
-
-        ip_key = ip_adapter_block.ip_adapter_double_stream_k_proj(self._image_proj)
-        ip_value = ip_adapter_block.ip_adapter_double_stream_v_proj(self._image_proj)
-
-        # Reshape projections for multi-head attention.
-        ip_key = einops.rearrange(ip_key, "B L (H D) -> B H L D", H=block.num_heads)
-        ip_value = einops.rearrange(ip_value, "B L (H D) -> B H L D", H=block.num_heads)
-
-        # Compute attention between IP projections and the latent query.
-        ip_attn = torch.nn.functional.scaled_dot_product_attention(
-            img_q, ip_key, ip_value, dropout_p=0.0, is_causal=False
-        )
-        ip_attn = einops.rearrange(ip_attn, "B H L D -> B L (H D)", H=block.num_heads)
-
-        img = img + weight * ip_attn
-
-        return img

invokeai/backend/flux/ip_adapter/ip_double_stream_block_processor.py

@@ -1,93 +0,0 @@
-# This file is based on:
-# https://github.com/XLabs-AI/x-flux/blob/47495425dbed499be1e8e5a6e52628b07349cba2/src/flux/modules/layers.py#L221
-import einops
-import torch
-
-from invokeai.backend.flux.math import attention
-from invokeai.backend.flux.modules.layers import DoubleStreamBlock
-
-
-class IPDoubleStreamBlockProcessor(torch.nn.Module):
-    """Attention processor for handling IP-adapter with double stream block."""
-
-    def __init__(self, context_dim: int, hidden_dim: int):
-        super().__init__()
-
-        # Ensure context_dim matches the dimension of image_proj
-        self.context_dim = context_dim
-        self.hidden_dim = hidden_dim
-
-        # Initialize projections for IP-adapter
-        self.ip_adapter_double_stream_k_proj = torch.nn.Linear(context_dim, hidden_dim, bias=True)
-        self.ip_adapter_double_stream_v_proj = torch.nn.Linear(context_dim, hidden_dim, bias=True)
-
-        torch.nn.init.zeros_(self.ip_adapter_double_stream_k_proj.weight)
-        torch.nn.init.zeros_(self.ip_adapter_double_stream_k_proj.bias)
-
-        torch.nn.init.zeros_(self.ip_adapter_double_stream_v_proj.weight)
-        torch.nn.init.zeros_(self.ip_adapter_double_stream_v_proj.bias)
-
-    def __call__(
-        self,
-        attn: DoubleStreamBlock,
-        img: torch.Tensor,
-        txt: torch.Tensor,
-        vec: torch.Tensor,
-        pe: torch.Tensor,
-        image_proj: torch.Tensor,
-        ip_scale: float = 1.0,
-    ):
-        # Prepare image for attention
-        img_mod1, img_mod2 = attn.img_mod(vec)
-        txt_mod1, txt_mod2 = attn.txt_mod(vec)
-
-        img_modulated = attn.img_norm1(img)
-        img_modulated = (1 + img_mod1.scale) * img_modulated + img_mod1.shift
-        img_qkv = attn.img_attn.qkv(img_modulated)
-        img_q, img_k, img_v = einops.rearrange(
-            img_qkv, "B L (K H D) -> K B H L D", K=3, H=attn.num_heads, D=attn.head_dim
-        )
-        img_q, img_k = attn.img_attn.norm(img_q, img_k, img_v)
-
-        txt_modulated = attn.txt_norm1(txt)
-        txt_modulated = (1 + txt_mod1.scale) * txt_modulated + txt_mod1.shift
-        txt_qkv = attn.txt_attn.qkv(txt_modulated)
-        txt_q, txt_k, txt_v = einops.rearrange(
-            txt_qkv, "B L (K H D) -> K B H L D", K=3, H=attn.num_heads, D=attn.head_dim
-        )
-        txt_q, txt_k = attn.txt_attn.norm(txt_q, txt_k, txt_v)
-
-        q = torch.cat((txt_q, img_q), dim=2)
-        k = torch.cat((txt_k, img_k), dim=2)
-        v = torch.cat((txt_v, img_v), dim=2)
-
-        attn1 = attention(q, k, v, pe=pe)
-        txt_attn, img_attn = attn1[:, : txt.shape[1]], attn1[:, txt.shape[1] :]
-
-        # print(f"txt_attn shape: {txt_attn.size()}")
-        # print(f"img_attn shape: {img_attn.size()}")
-
-        img = img + img_mod1.gate * attn.img_attn.proj(img_attn)
-        img = img + img_mod2.gate * attn.img_mlp((1 + img_mod2.scale) * attn.img_norm2(img) + img_mod2.shift)
-
-        txt = txt + txt_mod1.gate * attn.txt_attn.proj(txt_attn)
-        txt = txt + txt_mod2.gate * attn.txt_mlp((1 + txt_mod2.scale) * attn.txt_norm2(txt) + txt_mod2.shift)
-
-        # IP-adapter processing
-        ip_query = img_q  # latent sample query
-        ip_key = self.ip_adapter_double_stream_k_proj(image_proj)
-        ip_value = self.ip_adapter_double_stream_v_proj(image_proj)
-
-        # Reshape projections for multi-head attention
-        ip_key = einops.rearrange(ip_key, "B L (H D) -> B H L D", H=attn.num_heads, D=attn.head_dim)
-        ip_value = einops.rearrange(ip_value, "B L (H D) -> B H L D", H=attn.num_heads, D=attn.head_dim)
-
-        # Compute attention between IP projections and the latent query
-        ip_attention = torch.nn.functional.scaled_dot_product_attention(
-            ip_query, ip_key, ip_value, dropout_p=0.0, is_causal=False
-        )
-        ip_attention = einops.rearrange(ip_attention, "B H L D -> B L (H D)", H=attn.num_heads, D=attn.head_dim)
-
-        img = img + ip_scale * ip_attention
-
-        return img, txt

invokeai/backend/flux/ip_adapter/state_dict_utils.py

@@ -1,50 +0,0 @@
-from typing import Any, Dict
-
-import torch
-
-from invokeai.backend.flux.ip_adapter.xlabs_ip_adapter_flux import XlabsIpAdapterParams
-
-
-def is_state_dict_xlabs_ip_adapter(sd: Dict[str, Any]) -> bool:
-    """Is the state dict for an XLabs FLUX IP-Adapter model?
-
-    This is intended to be a reasonably high-precision detector, but it is not guaranteed to have perfect precision.
-    """
-    # If all of the expected keys are present, then this is very likely an XLabs IP-Adapter model.
-    expected_keys = {
-        "double_blocks.0.processor.ip_adapter_double_stream_k_proj.bias",
-        "double_blocks.0.processor.ip_adapter_double_stream_k_proj.weight",
-        "double_blocks.0.processor.ip_adapter_double_stream_v_proj.bias",
-        "double_blocks.0.processor.ip_adapter_double_stream_v_proj.weight",
-        "ip_adapter_proj_model.norm.bias",
-        "ip_adapter_proj_model.norm.weight",
-        "ip_adapter_proj_model.proj.bias",
-        "ip_adapter_proj_model.proj.weight",
-    }
-
-    if expected_keys.issubset(sd.keys()):
-        return True
-    return False
-
-
-def infer_xlabs_ip_adapter_params_from_state_dict(state_dict: dict[str, torch.Tensor]) -> XlabsIpAdapterParams:
-    num_double_blocks = 0
-    context_dim = 0
-    hidden_dim = 0
-
-    # Count the number of double blocks.
-    double_block_index = 0
-    while f"double_blocks.{double_block_index}.processor.ip_adapter_double_stream_k_proj.weight" in state_dict:
-        double_block_index += 1
-    num_double_blocks = double_block_index
-
-    hidden_dim = state_dict["double_blocks.0.processor.ip_adapter_double_stream_k_proj.weight"].shape[0]
-    context_dim = state_dict["double_blocks.0.processor.ip_adapter_double_stream_k_proj.weight"].shape[1]
-    clip_embeddings_dim = state_dict["ip_adapter_proj_model.proj.weight"].shape[1]
-
-    return XlabsIpAdapterParams(
-        num_double_blocks=num_double_blocks,
-        context_dim=context_dim,
-        hidden_dim=hidden_dim,
-        clip_embeddings_dim=clip_embeddings_dim,
-    )

invokeai/backend/flux/ip_adapter/xlabs_ip_adapter_flux.py

@@ -1,67 +0,0 @@
-from dataclasses import dataclass
-
-import torch
-
-from invokeai.backend.ip_adapter.ip_adapter import ImageProjModel
-
-
-class IPDoubleStreamBlock(torch.nn.Module):
-    def __init__(self, context_dim: int, hidden_dim: int):
-        super().__init__()
-
-        self.context_dim = context_dim
-        self.hidden_dim = hidden_dim
-
-        self.ip_adapter_double_stream_k_proj = torch.nn.Linear(context_dim, hidden_dim, bias=True)
-        self.ip_adapter_double_stream_v_proj = torch.nn.Linear(context_dim, hidden_dim, bias=True)
-
-
-class IPAdapterDoubleBlocks(torch.nn.Module):
-    def __init__(self, num_double_blocks: int, context_dim: int, hidden_dim: int):
-        super().__init__()
-        self.double_blocks = torch.nn.ModuleList(
-            [IPDoubleStreamBlock(context_dim, hidden_dim) for _ in range(num_double_blocks)]
-        )
-
-
-@dataclass
-class XlabsIpAdapterParams:
-    num_double_blocks: int
-    context_dim: int
-    hidden_dim: int
-
-    clip_embeddings_dim: int
-
-
-class XlabsIpAdapterFlux(torch.nn.Module):
-    def __init__(self, params: XlabsIpAdapterParams):
-        super().__init__()
-        self.image_proj = ImageProjModel(
-            cross_attention_dim=params.context_dim, clip_embeddings_dim=params.clip_embeddings_dim
-        )
-        self.ip_adapter_double_blocks = IPAdapterDoubleBlocks(
-            num_double_blocks=params.num_double_blocks, context_dim=params.context_dim, hidden_dim=params.hidden_dim
-        )
-
-    def load_xlabs_state_dict(self, state_dict: dict[str, torch.Tensor], assign: bool = False):
-        """We need this custom function to load state dicts rather than using .load_state_dict(...) because the model
-        structure does not match the state_dict structure.
-        """
-        # Split the state_dict into the image projection model and the double blocks.
-        image_proj_sd: dict[str, torch.Tensor] = {}
-        double_blocks_sd: dict[str, torch.Tensor] = {}
-        for k, v in state_dict.items():
-            if k.startswith("ip_adapter_proj_model."):
-                image_proj_sd[k] = v
-            elif k.startswith("double_blocks."):
-                double_blocks_sd[k] = v
-            else:
-                raise ValueError(f"Unexpected key: {k}")
-
-        # Initialize the image projection model.
-        image_proj_sd = {k.replace("ip_adapter_proj_model.", ""): v for k, v in image_proj_sd.items()}
-        self.image_proj.load_state_dict(image_proj_sd, assign=assign)
-
-        # Initialize the double blocks.
-        double_blocks_sd = {k.replace("processor.", ""): v for k, v in double_blocks_sd.items()}
-        self.ip_adapter_double_blocks.load_state_dict(double_blocks_sd, assign=assign)

invokeai/backend/flux/model.py

@@ -5,8 +5,6 @@ from dataclasses import dataclass
 import torch
 from torch import Tensor, nn
 
-from invokeai.backend.flux.custom_block_processor import CustomDoubleStreamBlockProcessor
-from invokeai.backend.flux.extensions.xlabs_ip_adapter_extension import XLabsIPAdapterExtension
 from invokeai.backend.flux.modules.layers import (
     DoubleStreamBlock,
     EmbedND,
@@ -90,11 +88,8 @@ class Flux(nn.Module):
         timesteps: Tensor,
         y: Tensor,
         guidance: Tensor | None,
-        timestep_index: int,
-        total_num_timesteps: int,
         controlnet_double_block_residuals: list[Tensor] | None,
         controlnet_single_block_residuals: list[Tensor] | None,
-        ip_adapter_extensions: list[XLabsIPAdapterExtension],
     ) -> Tensor:
         if img.ndim != 3 or txt.ndim != 3:
             raise ValueError("Input img and txt tensors must have 3 dimensions.")
@@ -116,19 +111,7 @@ class Flux(nn.Module):
         if controlnet_double_block_residuals is not None:
             assert len(controlnet_double_block_residuals) == len(self.double_blocks)
         for block_index, block in enumerate(self.double_blocks):
-            assert isinstance(block, DoubleStreamBlock)
-
-            img, txt = CustomDoubleStreamBlockProcessor.custom_double_block_forward(
-                timestep_index=timestep_index,
-                total_num_timesteps=total_num_timesteps,
-                block_index=block_index,
-                block=block,
-                img=img,
-                txt=txt,
-                vec=vec,
-                pe=pe,
-                ip_adapter_extensions=ip_adapter_extensions,
-            )
+            img, txt = block(img=img, txt=txt, vec=vec, pe=pe)
 
             if controlnet_double_block_residuals is not None:
                 img += controlnet_double_block_residuals[block_index]

invokeai/backend/flux/sampling_utils.py

@@ -168,17 +168,8 @@ def generate_img_ids(h: int, w: int, batch_size: int, device: torch.device, dtyp
     Returns:
         torch.Tensor: Image position ids.
     """
-
-    if device.type == "mps":
-        orig_dtype = dtype
-        dtype = torch.float16
-
     img_ids = torch.zeros(h // 2, w // 2, 3, device=device, dtype=dtype)
     img_ids[..., 1] = img_ids[..., 1] + torch.arange(h // 2, device=device, dtype=dtype)[:, None]
     img_ids[..., 2] = img_ids[..., 2] + torch.arange(w // 2, device=device, dtype=dtype)[None, :]
     img_ids = repeat(img_ids, "h w c -> b (h w) c", b=batch_size)
-
-    if device.type == "mps":
-        img_ids.to(orig_dtype)
-
     return img_ids

invokeai/backend/image_util/segment_anything/segment_anything_pipeline.py

@@ -1,4 +1,4 @@
-from typing import Optional, TypeAlias
+from typing import Optional
 
 import torch
 from PIL import Image
@@ -7,14 +7,6 @@ from transformers.models.sam.processing_sam import SamProcessor
 
 from invokeai.backend.raw_model import RawModel
 
-# Type aliases for the inputs to the SAM model.
-ListOfBoundingBoxes: TypeAlias = list[list[int]]
-"""A list of bounding boxes. Each bounding box is in the format [xmin, ymin, xmax, ymax]."""
-ListOfPoints: TypeAlias = list[list[int]]
-"""A list of points. Each point is in the format [x, y]."""
-ListOfPointLabels: TypeAlias = list[int]
-"""A list of SAM point labels. Each label is an integer where -1 is background, 0 is neutral, and 1 is foreground."""
-
 
 class SegmentAnythingPipeline(RawModel):
     """A wrapper class for the transformers SAM model and processor that makes it compatible with the model manager."""
@@ -35,53 +27,20 @@ class SegmentAnythingPipeline(RawModel):
 
         return calc_module_size(self._sam_model)
 
-    def segment(
-        self,
-        image: Image.Image,
-        bounding_boxes: list[list[int]] | None = None,
-        point_lists: list[list[list[int]]] | None = None,
-    ) -> torch.Tensor:
+    def segment(self, image: Image.Image, bounding_boxes: list[list[int]]) -> torch.Tensor:
         """Run the SAM model.
 
-        Either bounding_boxes or point_lists must be provided. If both are provided, bounding_boxes will be used and
-        point_lists will be ignored.
-
         Args:
             image (Image.Image): The image to segment.
             bounding_boxes (list[list[int]]): The bounding box prompts. Each bounding box is in the format
                 [xmin, ymin, xmax, ymax].
-            point_lists (list[list[list[int]]]): The points prompts. Each point is in the format [x, y, label].
-                `label` is an integer where -1 is background, 0 is neutral, and 1 is foreground.
 
         Returns:
             torch.Tensor: The segmentation masks. dtype: torch.bool. shape: [num_masks, channels, height, width].
         """
-
-        # Prep the inputs:
-        # - Create a list of bounding boxes or points and labels.
-        # - Add a batch dimension of 1 to the inputs.
-        if bounding_boxes:
-            input_boxes: list[ListOfBoundingBoxes] | None = [bounding_boxes]
-            input_points: list[ListOfPoints] | None = None
-            input_labels: list[ListOfPointLabels] | None = None
-        elif point_lists:
-            input_boxes: list[ListOfBoundingBoxes] | None = None
-            input_points: list[ListOfPoints] | None = []
-            input_labels: list[ListOfPointLabels] | None = []
-            for point_list in point_lists:
-                input_points.append([[p[0], p[1]] for p in point_list])
-                input_labels.append([p[2] for p in point_list])
-
-        else:
-            raise ValueError("Either bounding_boxes or points and labels must be provided.")
-
-        inputs = self._sam_processor(
-            images=image,
-            input_boxes=input_boxes,
-            input_points=input_points,
-            input_labels=input_labels,
-            return_tensors="pt",
-        ).to(self._sam_model.device)
+        # Add batch dimension of 1 to the bounding boxes.
+        boxes = [bounding_boxes]
+        inputs = self._sam_processor(images=image, input_boxes=boxes, return_tensors="pt").to(self._sam_model.device)
         outputs = self._sam_model(**inputs)
         masks = self._sam_processor.post_process_masks(
             masks=outputs.pred_masks,

invokeai/backend/model_manager/config.py

@@ -53,7 +53,6 @@ class BaseModelType(str, Enum):
     Any = "any"
     StableDiffusion1 = "sd-1"
     StableDiffusion2 = "sd-2"
-    StableDiffusion3 = "sd-3"
     StableDiffusionXL = "sdxl"
     StableDiffusionXLRefiner = "sdxl-refiner"
     Flux = "flux"
@@ -84,10 +83,8 @@ class SubModelType(str, Enum):
     Transformer = "transformer"
     TextEncoder = "text_encoder"
     TextEncoder2 = "text_encoder_2"
-    TextEncoder3 = "text_encoder_3"
     Tokenizer = "tokenizer"
     Tokenizer2 = "tokenizer_2"
-    Tokenizer3 = "tokenizer_3"
     VAE = "vae"
     VAEDecoder = "vae_decoder"
     VAEEncoder = "vae_encoder"
@@ -95,13 +92,6 @@ class SubModelType(str, Enum):
     SafetyChecker = "safety_checker"
 
 
-class ClipVariantType(str, Enum):
-    """Variant type."""
-
-    L = "large"
-    G = "gigantic"
-
-
 class ModelVariantType(str, Enum):
     """Variant type."""
 
@@ -157,15 +147,6 @@ class ModelSourceType(str, Enum):
 DEFAULTS_PRECISION = Literal["fp16", "fp32"]
 
 
-AnyVariant: TypeAlias = Union[ModelVariantType, ClipVariantType, None]
-
-
-class SubmodelDefinition(BaseModel):
-    path_or_prefix: str
-    model_type: ModelType
-    variant: AnyVariant = None
-
-
 class MainModelDefaultSettings(BaseModel):
     vae: str | None = Field(default=None, description="Default VAE for this model (model key)")
     vae_precision: DEFAULTS_PRECISION | None = Field(default=None, description="Default VAE precision for this model")
@@ -212,9 +193,6 @@ class ModelConfigBase(BaseModel):
         schema["required"].extend(["key", "type", "format"])
 
     model_config = ConfigDict(validate_assignment=True, json_schema_extra=json_schema_extra)
-    submodels: Optional[Dict[SubModelType, SubmodelDefinition]] = Field(
-        description="Loadable submodels in this model", default=None
-    )
 
 
 class CheckpointConfigBase(ModelConfigBase):
@@ -357,7 +335,7 @@ class MainConfigBase(ModelConfigBase):
     default_settings: Optional[MainModelDefaultSettings] = Field(
         description="Default settings for this model", default=None
     )
-    variant: AnyVariant = ModelVariantType.Normal
+    variant: ModelVariantType = ModelVariantType.Normal
 
 
 class MainCheckpointConfig(CheckpointConfigBase, MainConfigBase):
@@ -416,8 +394,6 @@ class IPAdapterBaseConfig(ModelConfigBase):
 class IPAdapterInvokeAIConfig(IPAdapterBaseConfig):
     """Model config for IP Adapter diffusers format models."""
 
-    # TODO(ryand): Should we deprecate this field? From what I can tell, it hasn't been probed correctly for a long
-    # time. Need to go through the history to make sure I'm understanding this fully.
     image_encoder_model_id: str
     format: Literal[ModelFormat.InvokeAI]
 
@@ -441,33 +417,12 @@ class CLIPEmbedDiffusersConfig(DiffusersConfigBase):
 
     type: Literal[ModelType.CLIPEmbed] = ModelType.CLIPEmbed
     format: Literal[ModelFormat.Diffusers] = ModelFormat.Diffusers
-    variant: ClipVariantType = ClipVariantType.L
 
     @staticmethod
     def get_tag() -> Tag:
         return Tag(f"{ModelType.CLIPEmbed.value}.{ModelFormat.Diffusers.value}")
 
 
-class CLIPGEmbedDiffusersConfig(CLIPEmbedDiffusersConfig):
-    """Model config for CLIP-G Embeddings."""
-
-    variant: ClipVariantType = ClipVariantType.G
-
-    @staticmethod
-    def get_tag() -> Tag:
-        return Tag(f"{ModelType.CLIPEmbed.value}.{ModelFormat.Diffusers.value}.{ClipVariantType.G}")
-
-
-class CLIPLEmbedDiffusersConfig(CLIPEmbedDiffusersConfig):
-    """Model config for CLIP-L Embeddings."""
-
-    variant: ClipVariantType = ClipVariantType.L
-
-    @staticmethod
-    def get_tag() -> Tag:
-        return Tag(f"{ModelType.CLIPEmbed.value}.{ModelFormat.Diffusers.value}.{ClipVariantType.L}")
-
-
 class CLIPVisionDiffusersConfig(DiffusersConfigBase):
     """Model config for CLIPVision."""
 
@@ -544,8 +499,6 @@ AnyModelConfig = Annotated[
         Annotated[SpandrelImageToImageConfig, SpandrelImageToImageConfig.get_tag()],
         Annotated[CLIPVisionDiffusersConfig, CLIPVisionDiffusersConfig.get_tag()],
         Annotated[CLIPEmbedDiffusersConfig, CLIPEmbedDiffusersConfig.get_tag()],
-        Annotated[CLIPLEmbedDiffusersConfig, CLIPLEmbedDiffusersConfig.get_tag()],
-        Annotated[CLIPGEmbedDiffusersConfig, CLIPGEmbedDiffusersConfig.get_tag()],
     ],
     Discriminator(get_model_discriminator_value),
 ]

invokeai/backend/model_manager/load/model_loaders/clip_vision.py

@@ -1,41 +0,0 @@
-from pathlib import Path
-from typing import Optional
-
-from transformers import CLIPVisionModelWithProjection
-
-from invokeai.backend.model_manager.config import (
-    AnyModel,
-    AnyModelConfig,
-    BaseModelType,
-    DiffusersConfigBase,
-    ModelFormat,
-    ModelType,
-    SubModelType,
-)
-from invokeai.backend.model_manager.load.load_default import ModelLoader
-from invokeai.backend.model_manager.load.model_loader_registry import ModelLoaderRegistry
-
-
-@ModelLoaderRegistry.register(base=BaseModelType.Any, type=ModelType.CLIPVision, format=ModelFormat.Diffusers)
-class ClipVisionLoader(ModelLoader):
-    """Class to load CLIPVision models."""
-
-    def _load_model(
-        self,
-        config: AnyModelConfig,
-        submodel_type: Optional[SubModelType] = None,
-    ) -> AnyModel:
-        if not isinstance(config, DiffusersConfigBase):
-            raise ValueError("Only DiffusersConfigBase models are currently supported here.")
-
-        if submodel_type is not None:
-            raise Exception("There are no submodels in CLIP Vision models.")
-
-        model_path = Path(config.path)
-
-        model = CLIPVisionModelWithProjection.from_pretrained(
-            model_path, torch_dtype=self._torch_dtype, local_files_only=True
-        )
-        assert isinstance(model, CLIPVisionModelWithProjection)
-
-        return model

invokeai/backend/model_manager/load/model_loaders/flux.py

@@ -19,10 +19,6 @@ from invokeai.backend.flux.controlnet.state_dict_utils import (
     is_state_dict_xlabs_controlnet,
 )
 from invokeai.backend.flux.controlnet.xlabs_controlnet_flux import XLabsControlNetFlux
-from invokeai.backend.flux.ip_adapter.state_dict_utils import infer_xlabs_ip_adapter_params_from_state_dict
-from invokeai.backend.flux.ip_adapter.xlabs_ip_adapter_flux import (
-    XlabsIpAdapterFlux,
-)
 from invokeai.backend.flux.model import Flux
 from invokeai.backend.flux.modules.autoencoder import AutoEncoder
 from invokeai.backend.flux.util import ae_params, params
@@ -39,7 +35,6 @@ from invokeai.backend.model_manager.config import (
     CLIPEmbedDiffusersConfig,
     ControlNetCheckpointConfig,
     ControlNetDiffusersConfig,
-    IPAdapterCheckpointConfig,
     MainBnbQuantized4bCheckpointConfig,
     MainCheckpointConfig,
     MainGGUFCheckpointConfig,
@@ -128,9 +123,9 @@ class BnbQuantizedLlmInt8bCheckpointModel(ModelLoader):
                 "The bnb modules are not available. Please install bitsandbytes if available on your platform."
             )
         match submodel_type:
-            case SubModelType.Tokenizer2 | SubModelType.Tokenizer3:
+            case SubModelType.Tokenizer2:
                 return T5Tokenizer.from_pretrained(Path(config.path) / "tokenizer_2", max_length=512)
-            case SubModelType.TextEncoder2 | SubModelType.TextEncoder3:
+            case SubModelType.TextEncoder2:
                 te2_model_path = Path(config.path) / "text_encoder_2"
                 model_config = AutoConfig.from_pretrained(te2_model_path)
                 with accelerate.init_empty_weights():
@@ -172,10 +167,10 @@ class T5EncoderCheckpointModel(ModelLoader):
             raise ValueError("Only T5EncoderConfig models are currently supported here.")
 
         match submodel_type:
-            case SubModelType.Tokenizer2 | SubModelType.Tokenizer3:
+            case SubModelType.Tokenizer2:
                 return T5Tokenizer.from_pretrained(Path(config.path) / "tokenizer_2", max_length=512)
-            case SubModelType.TextEncoder2 | SubModelType.TextEncoder3:
-                return T5EncoderModel.from_pretrained(Path(config.path) / "text_encoder_2", torch_dtype="auto")
+            case SubModelType.TextEncoder2:
+                return T5EncoderModel.from_pretrained(Path(config.path) / "text_encoder_2")
 
         raise ValueError(
             f"Only Tokenizer and TextEncoder submodels are currently supported. Received: {submodel_type.value if submodel_type else 'None'}"
@@ -357,26 +352,3 @@ class FluxControlnetModel(ModelLoader):
 
         model.load_state_dict(sd, assign=True)
         return model
-
-
-@ModelLoaderRegistry.register(base=BaseModelType.Flux, type=ModelType.IPAdapter, format=ModelFormat.Checkpoint)
-class FluxIpAdapterModel(ModelLoader):
-    """Class to load FLUX IP-Adapter models."""
-
-    def _load_model(
-        self,
-        config: AnyModelConfig,
-        submodel_type: Optional[SubModelType] = None,
-    ) -> AnyModel:
-        if not isinstance(config, IPAdapterCheckpointConfig):
-            raise ValueError(f"Unexpected model config type: {type(config)}.")
-
-        sd = load_file(Path(config.path))
-
-        params = infer_xlabs_ip_adapter_params_from_state_dict(sd)
-
-        with accelerate.init_empty_weights():
-            model = XlabsIpAdapterFlux(params=params)
-
-        model.load_xlabs_state_dict(sd, assign=True)
-        return model

invokeai/backend/model_manager/load/model_loaders/generic_diffusers.py

@@ -22,6 +22,7 @@ from invokeai.backend.model_manager.load.load_default import ModelLoader
 from invokeai.backend.model_manager.load.model_loader_registry import ModelLoaderRegistry
 
 
+@ModelLoaderRegistry.register(base=BaseModelType.Any, type=ModelType.CLIPVision, format=ModelFormat.Diffusers)
 @ModelLoaderRegistry.register(base=BaseModelType.Any, type=ModelType.T2IAdapter, format=ModelFormat.Diffusers)
 class GenericDiffusersLoader(ModelLoader):
     """Class to load simple diffusers models."""

invokeai/backend/model_manager/load/model_loaders/stable_diffusion.py

@@ -42,7 +42,6 @@ VARIANT_TO_IN_CHANNEL_MAP = {
 @ModelLoaderRegistry.register(
     base=BaseModelType.StableDiffusionXLRefiner, type=ModelType.Main, format=ModelFormat.Diffusers
 )
-@ModelLoaderRegistry.register(base=BaseModelType.StableDiffusion3, type=ModelType.Main, format=ModelFormat.Diffusers)
 @ModelLoaderRegistry.register(base=BaseModelType.StableDiffusion1, type=ModelType.Main, format=ModelFormat.Checkpoint)
 @ModelLoaderRegistry.register(base=BaseModelType.StableDiffusion2, type=ModelType.Main, format=ModelFormat.Checkpoint)
 @ModelLoaderRegistry.register(base=BaseModelType.StableDiffusionXL, type=ModelType.Main, format=ModelFormat.Checkpoint)
@@ -52,6 +51,13 @@ VARIANT_TO_IN_CHANNEL_MAP = {
 class StableDiffusionDiffusersModel(GenericDiffusersLoader):
     """Class to load main models."""
 
+    model_base_to_model_type = {
+        BaseModelType.StableDiffusion1: "FrozenCLIPEmbedder",
+        BaseModelType.StableDiffusion2: "FrozenOpenCLIPEmbedder",
+        BaseModelType.StableDiffusionXL: "SDXL",
+        BaseModelType.StableDiffusionXLRefiner: "SDXL-Refiner",
+    }
+
     def _load_model(
         self,
         config: AnyModelConfig,
@@ -111,6 +117,8 @@ class StableDiffusionDiffusersModel(GenericDiffusersLoader):
             load_class = load_classes[config.base][config.variant]
         except KeyError as e:
             raise Exception(f"No diffusers pipeline known for base={config.base}, variant={config.variant}") from e
+        prediction_type = config.prediction_type.value
+        upcast_attention = config.upcast_attention
 
         # Without SilenceWarnings we get log messages like this:
         # site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
@@ -121,7 +129,13 @@ class StableDiffusionDiffusersModel(GenericDiffusersLoader):
         # ['text_model.embeddings.position_ids']
 
         with SilenceWarnings():
-            pipeline = load_class.from_single_file(config.path, torch_dtype=self._torch_dtype)
+            pipeline = load_class.from_single_file(
+                config.path,
+                torch_dtype=self._torch_dtype,
+                prediction_type=prediction_type,
+                upcast_attention=upcast_attention,
+                load_safety_checker=False,
+            )
 
         if not submodel_type:
             return pipeline

invokeai/backend/model_manager/metadata/fetch/huggingface.py

@@ -20,7 +20,7 @@ from typing import Optional
 
 import requests
 from huggingface_hub import HfApi, configure_http_backend, hf_hub_url
-from huggingface_hub.errors import RepositoryNotFoundError, RevisionNotFoundError
+from huggingface_hub.utils._errors import RepositoryNotFoundError, RevisionNotFoundError
 from pydantic.networks import AnyHttpUrl
 from requests.sessions import Session
 

invokeai/backend/model_manager/probe.py

@@ -1,7 +1,7 @@
 import json
 import re
 from pathlib import Path
-from typing import Any, Callable, Dict, Literal, Optional, Union
+from typing import Any, Dict, Literal, Optional, Union
 
 import safetensors.torch
 import spandrel
@@ -14,7 +14,6 @@ from invokeai.backend.flux.controlnet.state_dict_utils import (
     is_state_dict_instantx_controlnet,
     is_state_dict_xlabs_controlnet,
 )
-from invokeai.backend.flux.ip_adapter.state_dict_utils import is_state_dict_xlabs_ip_adapter
 from invokeai.backend.lora.conversions.flux_diffusers_lora_conversion_utils import (
     is_state_dict_likely_in_flux_diffusers_format,
 )
@@ -22,7 +21,6 @@ from invokeai.backend.lora.conversions.flux_kohya_lora_conversion_utils import i
 from invokeai.backend.model_hash.model_hash import HASHING_ALGORITHMS, ModelHash
 from invokeai.backend.model_manager.config import (
     AnyModelConfig,
-    AnyVariant,
     BaseModelType,
     ControlAdapterDefaultSettings,
     InvalidModelConfigException,
@@ -34,15 +32,8 @@ from invokeai.backend.model_manager.config import (
     ModelType,
     ModelVariantType,
     SchedulerPredictionType,
-    SubmodelDefinition,
-    SubModelType,
-)
-from invokeai.backend.model_manager.load.model_loaders.generic_diffusers import ConfigLoader
-from invokeai.backend.model_manager.util.model_util import (
-    get_clip_variant_type,
-    lora_token_vector_length,
-    read_checkpoint_meta,
 )
+from invokeai.backend.model_manager.util.model_util import lora_token_vector_length, read_checkpoint_meta
 from invokeai.backend.quantization.gguf.ggml_tensor import GGMLTensor
 from invokeai.backend.quantization.gguf.loaders import gguf_sd_loader
 from invokeai.backend.spandrel_image_to_image_model import SpandrelImageToImageModel
@@ -120,7 +111,6 @@ class ModelProbe(object):
         "StableDiffusionXLPipeline": ModelType.Main,
         "StableDiffusionXLImg2ImgPipeline": ModelType.Main,
         "StableDiffusionXLInpaintPipeline": ModelType.Main,
-        "StableDiffusion3Pipeline": ModelType.Main,
         "LatentConsistencyModelPipeline": ModelType.Main,
         "AutoencoderKL": ModelType.VAE,
         "AutoencoderTiny": ModelType.VAE,
@@ -131,12 +121,8 @@ class ModelProbe(object):
         "CLIPTextModel": ModelType.CLIPEmbed,
         "T5EncoderModel": ModelType.T5Encoder,
         "FluxControlNetModel": ModelType.ControlNet,
-        "SD3Transformer2DModel": ModelType.Main,
-        "CLIPTextModelWithProjection": ModelType.CLIPEmbed,
     }
 
-    TYPE2VARIANT: Dict[ModelType, Callable[[str], Optional[AnyVariant]]] = {ModelType.CLIPEmbed: get_clip_variant_type}
-
     @classmethod
     def register_probe(
         cls, format: Literal["diffusers", "checkpoint", "onnx"], model_type: ModelType, probe_class: type[ProbeBase]
@@ -183,10 +169,7 @@ class ModelProbe(object):
         fields["path"] = model_path.as_posix()
         fields["type"] = fields.get("type") or model_type
         fields["base"] = fields.get("base") or probe.get_base_type()
-        variant_func = cls.TYPE2VARIANT.get(fields["type"], None)
-        fields["variant"] = (
-            fields.get("variant") or (variant_func and variant_func(model_path.as_posix())) or probe.get_variant_type()
-        )
+        fields["variant"] = fields.get("variant") or probe.get_variant_type()
         fields["prediction_type"] = fields.get("prediction_type") or probe.get_scheduler_prediction_type()
         fields["image_encoder_model_id"] = fields.get("image_encoder_model_id") or probe.get_image_encoder_model_id()
         fields["name"] = fields.get("name") or cls.get_model_name(model_path)
@@ -233,10 +216,6 @@ class ModelProbe(object):
                 and fields["prediction_type"] == SchedulerPredictionType.VPrediction
             )
 
-        get_submodels = getattr(probe, "get_submodels", None)
-        if fields["base"] == BaseModelType.StableDiffusion3 and callable(get_submodels):
-            fields["submodels"] = get_submodels()
-
         model_info = ModelConfigFactory.make_config(fields)  # , key=fields.get("key", None))
         return model_info
 
@@ -264,6 +243,8 @@ class ModelProbe(object):
                     "cond_stage_model.",
                     "first_stage_model.",
                     "model.diffusion_model.",
+                    # FLUX models in the official BFL format contain keys with the "double_blocks." prefix.
+                    "double_blocks.",
                     # Some FLUX checkpoint files contain transformer keys prefixed with "model.diffusion_model".
                     # This prefix is typically used to distinguish between multiple models bundled in a single file.
                     "model.diffusion_model.double_blocks.",
@@ -271,10 +252,6 @@ class ModelProbe(object):
             ):
                 # Keys starting with double_blocks are associated with Flux models
                 return ModelType.Main
-            # FLUX models in the official BFL format contain keys with the "double_blocks." prefix, but we must be
-            # careful to avoid false positives on XLabs FLUX IP-Adapter models.
-            elif key.startswith("double_blocks.") and "ip_adapter" not in key:
-                return ModelType.Main
             elif key.startswith(("encoder.conv_in", "decoder.conv_in")):
                 return ModelType.VAE
             elif key.startswith(("lora_te_", "lora_unet_")):
@@ -297,14 +274,7 @@ class ModelProbe(object):
                 )
             ):
                 return ModelType.ControlNet
-            elif key.startswith(
-                (
-                    "image_proj.",
-                    "ip_adapter.",
-                    # XLabs FLUX IP-Adapter models have keys startinh with "ip_adapter_proj_model.".
-                    "ip_adapter_proj_model.",
-                )
-            ):
+            elif key.startswith(("image_proj.", "ip_adapter.")):
                 return ModelType.IPAdapter
             elif key in {"emb_params", "string_to_param"}:
                 return ModelType.TextualInversion
@@ -482,9 +452,8 @@ MODEL_NAME_TO_PREPROCESSOR = {
     "normal": "normalbae_image_processor",
     "sketch": "pidi_image_processor",
     "scribble": "lineart_image_processor",
-    "lineart anime": "lineart_anime_image_processor",
-    "lineart_anime": "lineart_anime_image_processor",
     "lineart": "lineart_image_processor",
+    "lineart_anime": "lineart_anime_image_processor",
     "softedge": "hed_image_processor",
     "hed": "hed_image_processor",
     "shuffle": "content_shuffle_image_processor",
@@ -703,10 +672,6 @@ class IPAdapterCheckpointProbe(CheckpointProbeBase):
 
     def get_base_type(self) -> BaseModelType:
         checkpoint = self.checkpoint
-
-        if is_state_dict_xlabs_ip_adapter(checkpoint):
-            return BaseModelType.Flux
-
         for key in checkpoint.keys():
             if not key.startswith(("image_proj.", "ip_adapter.")):
                 continue
@@ -767,33 +732,18 @@ class FolderProbeBase(ProbeBase):
 
 class PipelineFolderProbe(FolderProbeBase):
     def get_base_type(self) -> BaseModelType:
-        # Handle pipelines with a UNet (i.e SD 1.x, SD2, SDXL).
-        config_path = self.model_path / "unet" / "config.json"
-        if config_path.exists():
-            with open(config_path) as file:
-                unet_conf = json.load(file)
-            if unet_conf["cross_attention_dim"] == 768:
-                return BaseModelType.StableDiffusion1
-            elif unet_conf["cross_attention_dim"] == 1024:
-                return BaseModelType.StableDiffusion2
-            elif unet_conf["cross_attention_dim"] == 1280:
-                return BaseModelType.StableDiffusionXLRefiner
-            elif unet_conf["cross_attention_dim"] == 2048:
-                return BaseModelType.StableDiffusionXL
-            else:
-                raise InvalidModelConfigException(f"Unknown base model for {self.model_path}")
-
-        # Handle pipelines with a transformer (i.e. SD3).
-        config_path = self.model_path / "transformer" / "config.json"
-        if config_path.exists():
-            with open(config_path) as file:
-                transformer_conf = json.load(file)
-            if transformer_conf["_class_name"] == "SD3Transformer2DModel":
-                return BaseModelType.StableDiffusion3
-            else:
-                raise InvalidModelConfigException(f"Unknown base model for {self.model_path}")
-
-        raise InvalidModelConfigException(f"Unknown base model for {self.model_path}")
+        with open(self.model_path / "unet" / "config.json", "r") as file:
+            unet_conf = json.load(file)
+        if unet_conf["cross_attention_dim"] == 768:
+            return BaseModelType.StableDiffusion1
+        elif unet_conf["cross_attention_dim"] == 1024:
+            return BaseModelType.StableDiffusion2
+        elif unet_conf["cross_attention_dim"] == 1280:
+            return BaseModelType.StableDiffusionXLRefiner
+        elif unet_conf["cross_attention_dim"] == 2048:
+            return BaseModelType.StableDiffusionXL
+        else:
+            raise InvalidModelConfigException(f"Unknown base model for {self.model_path}")
 
     def get_scheduler_prediction_type(self) -> SchedulerPredictionType:
         with open(self.model_path / "scheduler" / "scheduler_config.json", "r") as file:
@@ -805,23 +755,6 @@ class PipelineFolderProbe(FolderProbeBase):
         else:
             raise InvalidModelConfigException("Unknown scheduler prediction type: {scheduler_conf['prediction_type']}")
 
-    def get_submodels(self) -> Dict[SubModelType, SubmodelDefinition]:
-        config = ConfigLoader.load_config(self.model_path, config_name="model_index.json")
-        submodels: Dict[SubModelType, SubmodelDefinition] = {}
-        for key, value in config.items():
-            if key.startswith("_") or not (isinstance(value, list) and len(value) == 2):
-                continue
-            model_loader = str(value[1])
-            if model_type := ModelProbe.CLASS2TYPE.get(model_loader):
-                variant_func = ModelProbe.TYPE2VARIANT.get(model_type, None)
-                submodels[SubModelType(key)] = SubmodelDefinition(
-                    path_or_prefix=(self.model_path / key).resolve().as_posix(),
-                    model_type=model_type,
-                    variant=variant_func and variant_func((self.model_path / key).as_posix()),
-                )
-
-        return submodels
-
     def get_variant_type(self) -> ModelVariantType:
         # This only works for pipelines! Any kind of
         # exception results in our returning the

invokeai/backend/model_manager/util/model_util.py

@@ -8,7 +8,6 @@ import safetensors
 import torch
 from picklescan.scanner import scan_file_path
 
-from invokeai.backend.model_manager.config import ClipVariantType
 from invokeai.backend.quantization.gguf.loaders import gguf_sd_loader
 
 
@@ -166,23 +165,3 @@ def convert_bundle_to_flux_transformer_checkpoint(
         del transformer_state_dict[k]
 
     return original_state_dict
-
-
-def get_clip_variant_type(location: str) -> Optional[ClipVariantType]:
-    try:
-        path = Path(location)
-        config_path = path / "config.json"
-        if not config_path.exists():
-            return ClipVariantType.L
-        with open(config_path) as file:
-            clip_conf = json.load(file)
-            hidden_size = clip_conf.get("hidden_size", -1)
-            match hidden_size:
-                case 1280:
-                    return ClipVariantType.G
-                case 768:
-                    return ClipVariantType.L
-                case _:
-                    return ClipVariantType.L
-    except Exception:
-        return ClipVariantType.L

invokeai/backend/model_manager/util/select_hf_files.py

@@ -129,11 +129,9 @@ def _filter_by_variant(files: List[Path], variant: ModelRepoVariant) -> Set[Path
 
             # Some special handling is needed here if there is not an exact match and if we cannot infer the variant
             # from the file name. In this case, we only give this file a point if the requested variant is FP32 or DEFAULT.
-            if (
-                variant is not ModelRepoVariant.Default
-                and candidate_variant_label
-                and candidate_variant_label.startswith(f".{variant.value}")
-            ) or (not candidate_variant_label and variant in [ModelRepoVariant.FP32, ModelRepoVariant.Default]):
+            if candidate_variant_label == f".{variant}" or (
+                not candidate_variant_label and variant in [ModelRepoVariant.FP32, ModelRepoVariant.Default]
+            ):
                 score += 1
 
             if parent not in subfolder_weights:
@@ -148,7 +146,7 @@ def _filter_by_variant(files: List[Path], variant: ModelRepoVariant) -> Set[Path
         # Check if at least one of the files has the explicit fp16 variant.
         at_least_one_fp16 = False
         for candidate in candidate_list:
-            if len(candidate.path.suffixes) == 2 and candidate.path.suffixes[0].startswith(".fp16"):
+            if len(candidate.path.suffixes) == 2 and candidate.path.suffixes[0] == ".fp16":
                 at_least_one_fp16 = True
                 break
 
@@ -164,16 +162,7 @@ def _filter_by_variant(files: List[Path], variant: ModelRepoVariant) -> Set[Path
         # candidate.
         highest_score_candidate = max(candidate_list, key=lambda candidate: candidate.score)
         if highest_score_candidate:
-            pattern = r"^(.*?)-\d+-of-\d+(\.\w+)$"
-            match = re.match(pattern, highest_score_candidate.path.as_posix())
-            if match:
-                for candidate in candidate_list:
-                    if candidate.path.as_posix().startswith(match.group(1)) and candidate.path.as_posix().endswith(
-                        match.group(2)
-                    ):
-                        result.add(candidate.path)
-            else:
-                result.add(highest_score_candidate.path)
+            result.add(highest_score_candidate.path)
 
     # If one of the architecture-related variants was specified and no files matched other than
     # config and text files then we return an empty list

invokeai/backend/quantization/gguf/ggml_tensor.py

@@ -54,11 +54,6 @@ GGML_TENSOR_OP_TABLE = {
     torch.ops.aten.mul.Tensor: dequantize_and_run,  # pyright: ignore
 }
 
-if torch.backends.mps.is_available():
-    GGML_TENSOR_OP_TABLE.update(
-        {torch.ops.aten.linear.default: dequantize_and_run}  # pyright: ignore
-    )
-
 
 class GGMLTensor(torch.Tensor):
     """A torch.Tensor sub-class holding a quantized GGML tensor.

invokeai/backend/stable_diffusion/diffusers_pipeline.py

@@ -499,22 +499,6 @@ class StableDiffusionGeneratorPipeline(StableDiffusionPipeline):
                     for idx, value in enumerate(single_t2i_adapter_data.adapter_state):
                         accum_adapter_state[idx] += value * t2i_adapter_weight
 
-            # Hack: force compatibility with irregular resolutions by padding the feature map with zeros
-            for idx, tensor in enumerate(accum_adapter_state):
-                # The tensor size is supposed to be some integer downscale factor of the latents size.
-                # Internally, the unet will pad the latents before downscaling between levels when it is no longer divisible by its downscale factor.
-                # If the latent size does not scale down evenly, we need to pad the tensor so that it matches the the downscaled padded latents later on.
-                scale_factor = latents.size()[-1] // tensor.size()[-1]
-                required_padding_width = math.ceil(latents.size()[-1] / scale_factor) - tensor.size()[-1]
-                required_padding_height = math.ceil(latents.size()[-2] / scale_factor) - tensor.size()[-2]
-                tensor = torch.nn.functional.pad(
-                    tensor,
-                    (0, required_padding_width, 0, required_padding_height, 0, 0, 0, 0),
-                    mode="constant",
-                    value=0,
-                )
-                accum_adapter_state[idx] = tensor
-
             down_intrablock_additional_residuals = accum_adapter_state
 
         # Handle inpainting models.

invokeai/backend/stable_diffusion/diffusion/conditioning_data.py

@@ -49,32 +49,9 @@ class FLUXConditioningInfo:
         return self
 
 
-@dataclass
-class SD3ConditioningInfo:
-    clip_l_pooled_embeds: torch.Tensor
-    clip_l_embeds: torch.Tensor
-    clip_g_pooled_embeds: torch.Tensor
-    clip_g_embeds: torch.Tensor
-    t5_embeds: torch.Tensor | None
-
-    def to(self, device: torch.device | None = None, dtype: torch.dtype | None = None):
-        self.clip_l_pooled_embeds = self.clip_l_pooled_embeds.to(device=device, dtype=dtype)
-        self.clip_l_embeds = self.clip_l_embeds.to(device=device, dtype=dtype)
-        self.clip_g_pooled_embeds = self.clip_g_pooled_embeds.to(device=device, dtype=dtype)
-        self.clip_g_embeds = self.clip_g_embeds.to(device=device, dtype=dtype)
-        if self.t5_embeds is not None:
-            self.t5_embeds = self.t5_embeds.to(device=device, dtype=dtype)
-        return self
-
-
 @dataclass
 class ConditioningFieldData:
-    conditionings: (
-        List[BasicConditioningInfo]
-        | List[SDXLConditioningInfo]
-        | List[FLUXConditioningInfo]
-        | List[SD3ConditioningInfo]
-    )
+    conditionings: List[BasicConditioningInfo] | List[SDXLConditioningInfo] | List[FLUXConditioningInfo]
 
 
 @dataclass

invokeai/backend/stable_diffusion/extensions/lora.py

@@ -29,7 +29,7 @@ class LoRAExt(ExtensionBase):
 
     @contextmanager
     def patch_unet(self, unet: UNet2DConditionModel, original_weights: OriginalWeightsStorage):
-        lora_model = self._node_context.models.load(self._model_id, self._node_context.util.get_queue_id()).model
+        lora_model = self._node_context.models.load(self._model_id).model
         assert isinstance(lora_model, LoRAModelRaw)
         LoRAPatcher.apply_lora_patch(
             model=unet,

invokeai/backend/stable_diffusion/extensions/preview.py

@@ -33,7 +33,7 @@ class PreviewExt(ExtensionBase):
     def initial_preview(self, ctx: DenoiseContext):
         self.callback(
             PipelineIntermediateState(
-                step=0,
+                step=-1,
                 order=ctx.scheduler.order,
                 total_steps=len(ctx.inputs.timesteps),
                 timestep=int(ctx.scheduler.config.num_train_timesteps),  # TODO: is there any code which uses it?

invokeai/backend/stable_diffusion/extensions/t2i_adapter.py

@@ -54,7 +54,7 @@ class T2IAdapterExt(ExtensionBase):
     @callback(ExtensionCallbackType.SETUP)
     def setup(self, ctx: DenoiseContext):
         t2i_model: T2IAdapter
-        with self._node_context.models.load(self._model_id, self._node_context.util.get_queue_id()) as t2i_model:
+        with self._node_context.models.load(self._model_id) as t2i_model:
             _, _, latents_height, latents_width = ctx.inputs.orig_latents.shape
 
             self._adapter_state = self._run_model(

invokeai/backend/util/hotfixes.py

@@ -3,7 +3,7 @@ from typing import Any, Dict, List, Optional, Tuple, Union
 import diffusers
 import torch
 from diffusers.configuration_utils import ConfigMixin, register_to_config
-from diffusers.loaders.single_file_model import FromOriginalModelMixin
+from diffusers.loaders import FromOriginalControlNetMixin
 from diffusers.models.attention_processor import AttentionProcessor, AttnProcessor
 from diffusers.models.controlnet import ControlNetConditioningEmbedding, ControlNetOutput, zero_module
 from diffusers.models.embeddings import (
@@ -32,9 +32,7 @@ from invokeai.backend.util.logging import InvokeAILogger
 logger = InvokeAILogger.get_logger(__name__)
 
 
-# NOTE(ryand): I'm not the origina author of this code, but for future reference, it appears that this class was copied
-# from diffusers in order to add support for the encoder_attention_mask argument.
-class ControlNetModel(ModelMixin, ConfigMixin, FromOriginalModelMixin):
+class ControlNetModel(ModelMixin, ConfigMixin, FromOriginalControlNetMixin):
     """
     A ControlNet model.
 

invokeai/frontend/web/knip.ts

@@ -9,7 +9,6 @@ const config: KnipConfig = {
     'src/services/api/schema.ts',
     'src/features/nodes/types/v1/**',
     'src/features/nodes/types/v2/**',
-    'src/features/parameters/types/parameterSchemas.ts',
     // TODO(psyche): maybe we can clean up these utils after canvas v2 release
     'src/features/controlLayers/konva/util.ts',
     // TODO(psyche): restore HRF functionality?

invokeai/frontend/web/package.json

@@ -58,7 +58,7 @@
     "@dnd-kit/sortable": "^8.0.0",
     "@dnd-kit/utilities": "^3.2.2",
     "@fontsource-variable/inter": "^5.1.0",
-    "@invoke-ai/ui-library": "^0.0.43",
+    "@invoke-ai/ui-library": "^0.0.42",
     "@nanostores/react": "^0.7.3",
     "@reduxjs/toolkit": "2.2.3",
     "@roarr/browser-log-writer": "^1.3.0",
@@ -114,7 +114,8 @@
   },
   "peerDependencies": {
     "react": "^18.2.0",
-    "react-dom": "^18.2.0"
+    "react-dom": "^18.2.0",
+    "ts-toolbelt": "^9.6.0"
   },
   "devDependencies": {
     "@invoke-ai/eslint-config-react": "^0.0.14",
@@ -148,8 +149,8 @@
     "prettier": "^3.3.3",
     "rollup-plugin-visualizer": "^5.12.0",
     "storybook": "^8.3.4",
+    "ts-toolbelt": "^9.6.0",
     "tsafe": "^1.7.5",
-    "type-fest": "^4.26.1",
     "typescript": "^5.6.2",
     "vite": "^5.4.8",
     "vite-plugin-css-injected-by-js": "^3.5.2",

invokeai/frontend/web/pnpm-lock.yaml

@@ -24,8 +24,8 @@ dependencies:
     specifier: ^5.1.0
     version: 5.1.0
   '@invoke-ai/ui-library':
-    specifier: ^0.0.43
-    version: 0.0.43(@chakra-ui/form-control@2.2.0)(@chakra-ui/icon@3.2.0)(@chakra-ui/media-query@3.3.0)(@chakra-ui/menu@2.2.1)(@chakra-ui/spinner@2.1.0)(@chakra-ui/system@2.6.2)(@fontsource-variable/inter@5.1.0)(@types/react@18.3.11)(i18next@23.15.1)(react-dom@18.3.1)(react@18.3.1)
+    specifier: ^0.0.42
+    version: 0.0.42(@chakra-ui/form-control@2.2.0)(@chakra-ui/icon@3.2.0)(@chakra-ui/media-query@3.3.0)(@chakra-ui/menu@2.2.1)(@chakra-ui/spinner@2.1.0)(@chakra-ui/system@2.6.2)(@fontsource-variable/inter@5.1.0)(@types/react@18.3.11)(i18next@23.15.1)(react-dom@18.3.1)(react@18.3.1)
   '@nanostores/react':
     specifier: ^0.7.3
     version: 0.7.3(nanostores@0.11.3)(react@18.3.1)
@@ -277,12 +277,12 @@ devDependencies:
   storybook:
     specifier: ^8.3.4
     version: 8.3.4
+  ts-toolbelt:
+    specifier: ^9.6.0
+    version: 9.6.0
   tsafe:
     specifier: ^1.7.5
     version: 1.7.5
-  type-fest:
-    specifier: ^4.26.1
-    version: 4.26.1
   typescript:
     specifier: ^5.6.2
     version: 5.6.2
@@ -1696,20 +1696,20 @@ packages:
       prettier: 3.3.3
     dev: true
 
-  /@invoke-ai/ui-library@0.0.43(@chakra-ui/form-control@2.2.0)(@chakra-ui/icon@3.2.0)(@chakra-ui/media-query@3.3.0)(@chakra-ui/menu@2.2.1)(@chakra-ui/spinner@2.1.0)(@chakra-ui/system@2.6.2)(@fontsource-variable/inter@5.1.0)(@types/react@18.3.11)(i18next@23.15.1)(react-dom@18.3.1)(react@18.3.1):
-    resolution: {integrity: sha512-t3fPYyks07ue3dEBPJuTHbeDLnDckDCOrtvc07mMDbLOnlPEZ0StaeiNGH+oO8qLzAuMAlSTdswgHfzTc2MmPw==}
+  /@invoke-ai/ui-library@0.0.42(@chakra-ui/form-control@2.2.0)(@chakra-ui/icon@3.2.0)(@chakra-ui/media-query@3.3.0)(@chakra-ui/menu@2.2.1)(@chakra-ui/spinner@2.1.0)(@chakra-ui/system@2.6.2)(@fontsource-variable/inter@5.1.0)(@types/react@18.3.11)(i18next@23.15.1)(react-dom@18.3.1)(react@18.3.1):
+    resolution: {integrity: sha512-OuDXRipBO5mu+Nv4qN8cd8MiwiGBdq6h4PirVgPI9/ltbdcIzePgUJ0dJns26lflHSTRWW38I16wl4YTw3mNWA==}
     peerDependencies:
       '@fontsource-variable/inter': ^5.0.16
       react: ^18.2.0
       react-dom: ^18.2.0
     dependencies:
-      '@chakra-ui/anatomy': 2.3.4
+      '@chakra-ui/anatomy': 2.2.2
       '@chakra-ui/icons': 2.2.4(@chakra-ui/react@2.10.2)(react@18.3.1)
       '@chakra-ui/layout': 2.3.1(@chakra-ui/system@2.6.2)(react@18.3.1)
       '@chakra-ui/portal': 2.1.0(react-dom@18.3.1)(react@18.3.1)
       '@chakra-ui/react': 2.10.2(@emotion/react@11.13.3)(@emotion/styled@11.13.0)(@types/react@18.3.11)(framer-motion@11.10.0)(react-dom@18.3.1)(react@18.3.1)
-      '@chakra-ui/styled-system': 2.11.2(react@18.3.1)
-      '@chakra-ui/theme-tools': 2.2.6(@chakra-ui/styled-system@2.11.2)(react@18.3.1)
+      '@chakra-ui/styled-system': 2.9.2
+      '@chakra-ui/theme-tools': 2.1.2(@chakra-ui/styled-system@2.9.2)
       '@emotion/react': 11.13.3(@types/react@18.3.11)(react@18.3.1)
       '@emotion/styled': 11.13.0(@emotion/react@11.13.3)(@types/react@18.3.11)(react@18.3.1)
       '@fontsource-variable/inter': 5.1.0
@@ -8830,6 +8830,10 @@ packages:
     resolution: {integrity: sha512-tLJxacIQUM82IR7JO1UUkKlYuUTmoY9HBJAmNWFzheSlDS5SPMcNIepejHJa4BpPQLAcbRhRf3GDJzyj6rbKvA==}
     dev: false
 
+  /ts-toolbelt@9.6.0:
+    resolution: {integrity: sha512-nsZd8ZeNUzukXPlJmTBwUAuABDe/9qtVDelJeT/qW0ow3ZS3BsQJtNkan1802aM9Uf68/Y8ljw86Hu0h5IUW3w==}
+    dev: true
+
   /tsafe@1.7.5:
     resolution: {integrity: sha512-tbNyyBSbwfbilFfiuXkSOj82a6++ovgANwcoqBAcO9/REPoZMEQoE8kWPeO0dy5A2D/2Lajr8Ohue5T0ifIvLQ==}
     dev: true

invokeai/frontend/web/public/locales/de.json

@@ -93,10 +93,7 @@
         "placeholderSelectAModel": "Modell auswählen",
         "reset": "Zurücksetzen",
         "none": "Keine",
-        "new": "Neu",
-        "ok": "OK",
-        "close": "Schließen",
-        "clipboard": "Zwischenablage"
+        "new": "Neu"
     },
     "gallery": {
         "galleryImageSize": "Bildgröße",
@@ -159,11 +156,7 @@
         "displayBoardSearch": "Board durchsuchen",
         "displaySearch": "Bild suchen",
         "go": "Los",
-        "jump": "Springen",
-        "assetsTab": "Dateien, die Sie zur Verwendung in Ihren Projekten hochgeladen haben.",
-        "imagesTab": "Bilder, die Sie in Invoke erstellt und gespeichert haben.",
-        "boardsSettings": "Ordnereinstellungen",
-        "imagesSettings": "Galeriebildereinstellungen"
+        "jump": "Springen"
     },
     "hotkeys": {
         "noHotkeysFound": "Kein Hotkey gefunden",
@@ -274,18 +267,6 @@
             "applyFilter": {
                 "title": "Filter anwenden",
                 "desc": "Wende den ausstehenden Filter auf die ausgewählte Ebene an."
-            },
-            "cancelFilter": {
-                "title": "Filter abbrechen",
-                "desc": "Den ausstehenden Filter abbrechen."
-            },
-            "applyTransform": {
-                "desc": "Die ausstehende Transformation auf die ausgewählte Ebene anwenden.",
-                "title": "Transformation anwenden"
-            },
-            "cancelTransform": {
-                "title": "Transformation abbrechen",
-                "desc": "Die ausstehende Transformation abbrechen."
             }
         },
         "viewer": {
@@ -536,12 +517,14 @@
         "addModels": "Model hinzufügen",
         "deleteModelImage": "Lösche Model Bild",
         "huggingFaceRepoID": "HuggingFace Repo ID",
+        "hfToken": "HuggingFace Schlüssel",
         "huggingFacePlaceholder": "besitzer/model-name",
         "modelSettings": "Modelleinstellungen",
         "typePhraseHere": "Phrase hier eingeben",
         "spandrelImageToImage": "Bild zu Bild (Spandrel)",
         "starterModels": "Einstiegsmodelle",
         "t5Encoder": "T5-Kodierer",
+        "useDefaultSettings": "Standardeinstellungen verwenden",
         "uploadImage": "Bild hochladen",
         "urlOrLocalPath": "URL oder lokaler Pfad",
         "install": "Installieren",
@@ -580,18 +563,7 @@
         "scanResults": "Ergebnisse des Scans",
         "urlOrLocalPathHelper": "URLs sollten auf eine einzelne Datei deuten. Lokale Pfade können zusätzlich auch auf einen Ordner für ein einzelnes Diffusers-Modell hinweisen.",
         "inplaceInstallDesc": "Installieren Sie Modelle, ohne die Dateien zu kopieren. Wenn Sie das Modell verwenden, wird es direkt von seinem Speicherort geladen. Wenn deaktiviert, werden die Dateien während der Installation in das von Invoke verwaltete Modellverzeichnis kopiert.",
-        "scanFolderHelper": "Der Ordner wird rekursiv nach Modellen durchsucht. Dies kann bei sehr großen Ordnern etwas dauern.",
-        "includesNModels": "Enthält {{n}} Modelle und deren Abhängigkeiten",
-        "starterBundles": "Starterpakete",
-        "installingXModels_one": "{{count}} Modell wird installiert",
-        "installingXModels_other": "{{count}} Modelle werden installiert",
-        "skippingXDuplicates_one": ", überspringe {{count}} Duplikat",
-        "skippingXDuplicates_other": ", überspringe {{count}} Duplikate",
-        "installingModel": "Modell wird installiert",
-        "loraTriggerPhrases": "LoRA-Auslösephrasen",
-        "installingBundle": "Bündel wird installiert",
-        "triggerPhrases": "Auslösephrasen",
-        "mainModelTriggerPhrases": "Hauptmodell-Auslösephrasen"
+        "scanFolderHelper": "Der Ordner wird rekursiv nach Modellen durchsucht. Dies kann bei sehr großen Ordnern etwas dauern."
     },
     "parameters": {
         "images": "Bilder",
@@ -677,41 +649,10 @@
     "toast": {
         "uploadFailed": "Hochladen fehlgeschlagen",
         "imageCopied": "Bild kopiert",
-        "parametersNotSet": "Parameter nicht zurückgerufen",
+        "parametersNotSet": "Parameter nicht festgelegt",
         "addedToBoard": "Dem Board hinzugefügt",
         "loadedWithWarnings": "Workflow mit Warnungen geladen",
-        "imageSaved": "Bild gespeichert",
-        "linkCopied": "Link kopiert",
-        "problemCopyingLayer": "Ebene kann nicht kopiert werden",
-        "problemSavingLayer": "Ebene kann nicht gespeichert werden",
-        "parameterSetDesc": "{{parameter}} zurückgerufen",
-        "imageUploaded": "Bild hochgeladen",
-        "problemCopyingImage": "Bild kann nicht kopiert werden",
-        "parameterNotSetDesc": "{{parameter}} kann nicht zurückgerufen werden",
-        "prunedQueue": "Warteschlange bereinigt",
-        "modelAddedSimple": "Modell zur Warteschlange hinzugefügt",
-        "parametersSet": "Parameter zurückgerufen",
-        "imageNotLoadedDesc": "Bild konnte nicht gefunden werden",
-        "setControlImage": "Als Kontrollbild festlegen",
-        "sentToUpscale": "An Vergrößerung gesendet",
-        "parameterNotSetDescWithMessage": "{{parameter}} kann nicht zurückgerufen werden: {{message}}",
-        "unableToLoadImageMetadata": "Bildmetadaten können nicht geladen werden",
-        "unableToLoadImage": "Bild kann nicht geladen werden",
-        "serverError": "Serverfehler",
-        "parameterNotSet": "Parameter nicht zurückgerufen",
-        "sessionRef": "Sitzung: {{sessionId}}",
-        "problemDownloadingImage": "Bild kann nicht heruntergeladen werden",
-        "parameters": "Parameter",
-        "parameterSet": "Parameter zurückgerufen",
-        "importFailed": "Import fehlgeschlagen",
-        "importSuccessful": "Import erfolgreich",
-        "setNodeField": "Als Knotenfeld festlegen",
-        "somethingWentWrong": "Etwas ist schief gelaufen",
-        "workflowLoaded": "Arbeitsablauf geladen",
-        "workflowDeleted": "Arbeitsablauf gelöscht",
-        "errorCopied": "Fehler kopiert",
-        "layerCopiedToClipboard": "Ebene in die Zwischenablage kopiert",
-        "sentToCanvas": "An Leinwand gesendet"
+        "imageSaved": "Bild gespeichert"
     },
     "accessibility": {
         "uploadImage": "Bild hochladen",
@@ -726,8 +667,7 @@
         "about": "Über",
         "submitSupportTicket": "Support-Ticket senden",
         "toggleRightPanel": "Rechtes Bedienfeld umschalten (G)",
-        "toggleLeftPanel": "Linkes Bedienfeld umschalten (T)",
-        "uploadImages": "Bild(er) hochladen"
+        "toggleLeftPanel": "Linkes Bedienfeld umschalten (T)"
     },
     "boards": {
         "autoAddBoard": "Board automatisch erstellen",
@@ -762,7 +702,7 @@
         "shared": "Geteilte Ordner",
         "archiveBoard": "Ordner archivieren",
         "archived": "Archiviert",
-        "noBoards": "Kein {{boardType}} Ordner",
+        "noBoards": "Kein {boardType}} Ordner",
         "hideBoards": "Ordner verstecken",
         "viewBoards": "Ordner ansehen",
         "deletedPrivateBoardsCannotbeRestored": "Gelöschte Boards können nicht wiederhergestellt werden. Wenn Sie „Nur Board löschen“ wählen, werden die Bilder in einen privaten, nicht kategorisierten Status für den Ersteller des Bildes versetzt.",
@@ -855,6 +795,7 @@
         "width": "Breite",
         "createdBy": "Erstellt von",
         "steps": "Schritte",
+        "seamless": "Nahtlos",
         "positivePrompt": "Positiver Prompt",
         "generationMode": "Generierungsmodus",
         "Threshold": "Rauschen-Schwelle",
@@ -870,8 +811,7 @@
         "parameterSet": "Parameter {{parameter}} setzen",
         "recallParameter": "{{label}} Abrufen",
         "parsingFailed": "Parsing Fehlgeschlagen",
-        "canvasV2Metadata": "Leinwand",
-        "guidance": "Führung"
+        "canvasV2Metadata": "Leinwand"
     },
     "popovers": {
         "noiseUseCPU": {
@@ -1197,21 +1137,7 @@
         "workflowNotes": "Notizen",
         "workflowTags": "Tags",
         "workflowVersion": "Version",
-        "saveToGallery": "In Galerie speichern",
-        "noWorkflows": "Keine Arbeitsabläufe",
-        "noMatchingWorkflows": "Keine passenden Arbeitsabläufe",
-        "unknownErrorValidatingWorkflow": "Unbekannter Fehler beim Validieren des Arbeitsablaufes",
-        "inputFieldTypeParseError": "Typ des Eingabefelds {{node}}.{{field}} kann nicht analysiert werden ({{message}})",
-        "workflowSettings": "Arbeitsablauf Editor Einstellungen",
-        "unableToLoadWorkflow": "Arbeitsablauf kann nicht geladen werden",
-        "viewMode": "In linearen Ansicht verwenden",
-        "unableToValidateWorkflow": "Arbeitsablauf kann nicht validiert werden",
-        "outputFieldTypeParseError": "Typ des Ausgabefelds {{node}}.{{field}} kann nicht analysiert werden ({{message}})",
-        "unableToGetWorkflowVersion": "Version des Arbeitsablaufschemas kann nicht bestimmt werden",
-        "unknownFieldType": "$t(nodes.unknownField) Typ: {{type}}",
-        "unknownField": "Unbekanntes Feld",
-        "unableToUpdateNodes_one": "{{count}} Knoten kann nicht aktualisiert werden",
-        "unableToUpdateNodes_other": "{{count}} Knoten können nicht aktualisiert werden"
+        "saveToGallery": "In Galerie speichern"
     },
     "hrf": {
         "enableHrf": "Korrektur für hohe Auflösungen",
@@ -1341,7 +1267,15 @@
         "enableLogging": "Protokollierung aktivieren"
     },
     "whatsNew": {
-        "whatsNewInInvoke": "Was gibt's Neues"
+        "whatsNewInInvoke": "Was gibt's Neues",
+        "canvasV2Announcement": {
+            "fluxSupport": "Unterstützung für Flux-Modelle",
+            "newCanvas": "Eine leistungsstarke neue Kontrollfläche",
+            "newLayerTypes": "Neue Ebenentypen für noch mehr Kontrolle",
+            "readReleaseNotes": "Anmerkungen zu dieser Version lesen",
+            "watchReleaseVideo": "Video über diese Version anzeigen",
+            "watchUiUpdatesOverview": "Interface-Updates Übersicht"
+        }
     },
     "stylePresets": {
         "name": "Name",

invokeai/frontend/web/public/locales/en.json

@@ -12,8 +12,7 @@
         "resetUI": "$t(accessibility.reset) UI",
         "toggleRightPanel": "Toggle Right Panel (G)",
         "toggleLeftPanel": "Toggle Left Panel (T)",
-        "uploadImage": "Upload Image",
-        "uploadImages": "Upload Image(s)"
+        "uploadImage": "Upload Image"
     },
     "boards": {
         "addBoard": "Add Board",
@@ -94,7 +93,6 @@
         "close": "Close",
         "copy": "Copy",
         "copyError": "$t(gallery.copy) Error",
-        "clipboard": "Clipboard",
         "on": "On",
         "off": "Off",
         "or": "or",
@@ -682,8 +680,7 @@
         "recallParameters": "Recall Parameters",
         "recallParameter": "Recall {{label}}",
         "scheduler": "Scheduler",
-        "seamlessXAxis": "Seamless X Axis",
-        "seamlessYAxis": "Seamless Y Axis",
+        "seamless": "Seamless",
         "seed": "Seed",
         "steps": "Steps",
         "strength": "Image to image strength",
@@ -714,12 +711,8 @@
         "convertToDiffusersHelpText4": "This is a one time process only. It might take around 30s-60s depending on the specifications of your computer.",
         "convertToDiffusersHelpText5": "Please make sure you have enough disk space. Models generally vary between 2GB-7GB in size.",
         "convertToDiffusersHelpText6": "Do you wish to convert this model?",
-        "noDefaultSettings": "No default settings configured for this model. Visit the Model Manager to add default settings.",
         "defaultSettings": "Default Settings",
         "defaultSettingsSaved": "Default Settings Saved",
-        "defaultSettingsOutOfSync": "Some settings do not match the model's defaults:",
-        "restoreDefaultSettings": "Click to use the model's default settings.",
-        "usingDefaultSettings": "Using model's default settings",
         "delete": "Delete",
         "deleteConfig": "Delete Config",
         "deleteModel": "Delete Model",
@@ -733,19 +726,8 @@
         "huggingFacePlaceholder": "owner/model-name",
         "huggingFaceRepoID": "HuggingFace Repo ID",
         "huggingFaceHelper": "If multiple models are found in this repo, you will be prompted to select one to install.",
-        "hfTokenLabel": "HuggingFace Token (Required for some models)",
-        "hfTokenHelperText": "A HF token is required to use some models. Click here to create or get your token.",
-        "hfTokenInvalid": "Invalid or Missing HF Token",
-        "hfForbidden": "You do not have access to this HF model",
-        "hfForbiddenErrorMessage": "We recommend visiting the repo page on HuggingFace.com. The owner may require acceptance of terms in order to download.",
-        "hfTokenInvalidErrorMessage": "Invalid or missing HuggingFace token.",
-        "hfTokenRequired": "You are trying to download a model that requires a valid HuggingFace Token.",
-        "hfTokenInvalidErrorMessage2": "Update it in the ",
-        "hfTokenUnableToVerify": "Unable to Verify HF Token",
-        "hfTokenUnableToVerifyErrorMessage": "Unable to verify HuggingFace token. This is likely due to a network error. Please try again later.",
-        "hfTokenSaved": "HF Token Saved",
+        "hfToken": "HuggingFace Token",
         "imageEncoderModelId": "Image Encoder Model ID",
-        "includesNModels": "Includes {{n}} models and their dependencies",
         "installQueue": "Install Queue",
         "inplaceInstall": "In-place install",
         "inplaceInstallDesc": "Install models without copying the files. When using the model, it will be loaded from its this location. If disabled, the model file(s) will be copied into the Invoke-managed models directory during installation.",
@@ -799,8 +781,6 @@
         "simpleModelPlaceholder": "URL or path to a local file or diffusers folder",
         "source": "Source",
         "spandrelImageToImage": "Image to Image (Spandrel)",
-        "starterBundles": "Starter Bundles",
-        "starterBundleHelpText": "Easily install all models needed to get started with a base model, including a main model, controlnets, IP adapters, and more. Selecting a bundle will skip any models that you already have installed.",
         "starterModels": "Starter Models",
         "starterModelsInModelManager": "Starter Models can be found in Model Manager",
         "syncModels": "Sync Models",
@@ -814,16 +794,11 @@
         "uploadImage": "Upload Image",
         "urlOrLocalPath": "URL or Local Path",
         "urlOrLocalPathHelper": "URLs should point to a single file. Local paths can point to a single file or folder for a single diffusers model.",
+        "useDefaultSettings": "Use Default Settings",
         "vae": "VAE",
         "vaePrecision": "VAE Precision",
         "variant": "Variant",
-        "width": "Width",
-        "installingBundle": "Installing Bundle",
-        "installingModel": "Installing Model",
-        "installingXModels_one": "Installing {{count}} model",
-        "installingXModels_other": "Installing {{count}} models",
-        "skippingXDuplicates_one": ", skipping {{count}} duplicate",
-        "skippingXDuplicates_other": ", skipping {{count}} duplicates"
+        "width": "Width"
     },
     "models": {
         "addLora": "Add LoRA",
@@ -997,7 +972,6 @@
         "controlNetControlMode": "Control Mode",
         "copyImage": "Copy Image",
         "denoisingStrength": "Denoising Strength",
-        "noRasterLayers": "No Raster Layers",
         "downloadImage": "Download Image",
         "general": "General",
         "guidance": "Guidance",
@@ -1048,7 +1022,6 @@
         "patchmatchDownScaleSize": "Downscale",
         "perlinNoise": "Perlin Noise",
         "positivePromptPlaceholder": "Positive Prompt",
-        "recallMetadata": "Recall Metadata",
         "iterations": "Iterations",
         "scale": "Scale",
         "scaleBeforeProcessing": "Scale Before Processing",
@@ -1125,9 +1098,6 @@
         "enableInformationalPopovers": "Enable Informational Popovers",
         "informationalPopoversDisabled": "Informational Popovers Disabled",
         "informationalPopoversDisabledDesc": "Informational popovers have been disabled. Enable them in Settings.",
-        "enableModelDescriptions": "Enable Model Descriptions in Dropdowns",
-        "modelDescriptionsDisabled": "Model Descriptions in Dropdowns Disabled",
-        "modelDescriptionsDisabledDesc": "Model descriptions in dropdowns have been disabled. Enable them in Settings.",
         "enableInvisibleWatermark": "Enable Invisible Watermark",
         "enableNSFWChecker": "Enable NSFW Checker",
         "general": "General",
@@ -1152,8 +1122,7 @@
         "reloadingIn": "Reloading in"
     },
     "toast": {
-        "addedToBoard": "Added to board {{name}}'s assets",
-        "addedToUncategorized": "Added to board $t(boards.uncategorized)'s assets",
+        "addedToBoard": "Added to board",
         "baseModelChanged": "Base Model Changed",
         "baseModelChangedCleared_one": "Cleared or disabled {{count}} incompatible submodel",
         "baseModelChangedCleared_other": "Cleared or disabled {{count}} incompatible submodels",
@@ -1202,10 +1171,7 @@
         "setNodeField": "Set as node field",
         "somethingWentWrong": "Something Went Wrong",
         "uploadFailed": "Upload failed",
-        "imagesWillBeAddedTo": "Uploaded images will be added to board {{boardName}}'s assets.",
-        "uploadFailedInvalidUploadDesc_withCount_one": "Must be maximum of 1 PNG or JPEG image.",
-        "uploadFailedInvalidUploadDesc_withCount_other": "Must be maximum of {{count}} PNG or JPEG images.",
-        "uploadFailedInvalidUploadDesc": "Must be PNG or JPEG images.",
+        "uploadFailedInvalidUploadDesc": "Must be PNG or JPEG images",
         "workflowLoaded": "Workflow Loaded",
         "problemRetrievingWorkflow": "Problem Retrieving Workflow",
         "workflowDeleted": "Workflow Deleted",
@@ -1271,33 +1237,6 @@
             "heading": "Mask Adjustments",
             "paragraphs": ["Adjust the mask."]
         },
-        "inpainting": {
-            "heading": "Inpainting",
-            "paragraphs": ["Controls which area is modified, guided by Denoising Strength."]
-        },
-        "rasterLayer": {
-            "heading": "Raster Layer",
-            "paragraphs": ["Pixel-based content of your canvas, used during image generation."]
-        },
-        "regionalGuidance": {
-            "heading": "Regional Guidance",
-            "paragraphs": ["Brush to guide where elements from global prompts should appear."]
-        },
-        "regionalGuidanceAndReferenceImage": {
-            "heading": "Regional Guidance and Regional Reference Image",
-            "paragraphs": [
-                "For Regional Guidance, brush to guide where elements from global prompts should appear.",
-                "For Regional Reference Image, brush to apply a reference image to specific areas."
-            ]
-        },
-        "globalReferenceImage": {
-            "heading": "Global Reference Image",
-            "paragraphs": ["Applies a reference image to influence the entire generation."]
-        },
-        "regionalReferenceImage": {
-            "heading": "Regional Reference Image",
-            "paragraphs": ["Brush to apply a reference image to specific areas."]
-        },
         "controlNet": {
             "heading": "ControlNet",
             "paragraphs": [
@@ -1413,9 +1352,8 @@
         "paramDenoisingStrength": {
             "heading": "Denoising Strength",
             "paragraphs": [
-                "Controls how much the generated image varies from the raster layer(s).",
-                "Lower strength stays closer to the combined visible raster layers. Higher strength relies more on the global prompt.",
-                "When there are no raster layers with visible content, this setting is ignored."
+                "How much noise is added to the input image.",
+                "0 will result in an identical image, while 1 will result in a completely new image."
             ]
         },
         "paramHeight": {
@@ -1654,17 +1592,14 @@
         "newControlLayerError": "Problem Creating Control Layer",
         "newRasterLayerOk": "Created Raster Layer",
         "newRasterLayerError": "Problem Creating Raster Layer",
-        "newFromImage": "New from Image",
         "pullBboxIntoLayerOk": "Bbox Pulled Into Layer",
         "pullBboxIntoLayerError": "Problem Pulling BBox Into Layer",
         "pullBboxIntoReferenceImageOk": "Bbox Pulled Into ReferenceImage",
         "pullBboxIntoReferenceImageError": "Problem Pulling BBox Into ReferenceImage",
         "regionIsEmpty": "Selected region is empty",
         "mergeVisible": "Merge Visible",
-        "mergeDown": "Merge Down",
-        "mergeVisibleOk": "Merged layers",
-        "mergeVisibleError": "Error merging layers",
-        "mergingLayers": "Merging layers",
+        "mergeVisibleOk": "Merged visible layers",
+        "mergeVisibleError": "Error merging visible layers",
         "clearHistory": "Clear History",
         "bboxOverlay": "Show Bbox Overlay",
         "resetCanvas": "Reset Canvas",
@@ -1699,8 +1634,6 @@
         "controlLayer": "Control Layer",
         "inpaintMask": "Inpaint Mask",
         "regionalGuidance": "Regional Guidance",
-        "canvasAsRasterLayer": "$t(controlLayers.canvas) as $t(controlLayers.rasterLayer)",
-        "canvasAsControlLayer": "$t(controlLayers.canvas) as $t(controlLayers.controlLayer)",
         "referenceImage": "Reference Image",
         "regionalReferenceImage": "Regional Reference Image",
         "globalReferenceImage": "Global Reference Image",
@@ -1741,18 +1674,8 @@
         "layer_other": "Layers",
         "layer_withCount_one": "Layer ({{count}})",
         "layer_withCount_other": "Layers ({{count}})",
-        "convertRasterLayerTo": "Convert $t(controlLayers.rasterLayer) To",
-        "convertControlLayerTo": "Convert $t(controlLayers.controlLayer) To",
-        "convertInpaintMaskTo": "Convert $t(controlLayers.inpaintMask) To",
-        "convertRegionalGuidanceTo": "Convert $t(controlLayers.regionalGuidance) To",
-        "copyRasterLayerTo": "Copy $t(controlLayers.rasterLayer) To",
-        "copyControlLayerTo": "Copy $t(controlLayers.controlLayer) To",
-        "copyInpaintMaskTo": "Copy $t(controlLayers.inpaintMask) To",
-        "copyRegionalGuidanceTo": "Copy $t(controlLayers.regionalGuidance) To",
-        "newRasterLayer": "New $t(controlLayers.rasterLayer)",
-        "newControlLayer": "New $t(controlLayers.controlLayer)",
-        "newInpaintMask": "New $t(controlLayers.inpaintMask)",
-        "newRegionalGuidance": "New $t(controlLayers.regionalGuidance)",
+        "convertToControlLayer": "Convert to Control Layer",
+        "convertToRasterLayer": "Convert to Raster Layer",
         "transparency": "Transparency",
         "enableTransparencyEffect": "Enable Transparency Effect",
         "disableTransparencyEffect": "Disable Transparency Effect",
@@ -1776,11 +1699,9 @@
         "newGallerySessionDesc": "This will clear the canvas and all settings except for your model selection. Generations will be sent to the gallery.",
         "newCanvasSession": "New Canvas Session",
         "newCanvasSessionDesc": "This will clear the canvas and all settings except for your model selection. Generations will be staged on the canvas.",
-        "replaceCurrent": "Replace Current",
-        "controlLayerEmptyState": "<UploadButton>Upload an image</UploadButton>, drag an image from the <GalleryButton>gallery</GalleryButton> onto this layer, or draw on the canvas to get started.",
         "controlMode": {
             "controlMode": "Control Mode",
-            "balanced": "Balanced (recommended)",
+            "balanced": "Balanced",
             "prompt": "Prompt",
             "control": "Control",
             "megaControl": "Mega Control"
@@ -1819,9 +1740,6 @@
             "process": "Process",
             "apply": "Apply",
             "cancel": "Cancel",
-            "advanced": "Advanced",
-            "processingLayerWith": "Processing layer with the {{type}} filter.",
-            "forMoreControl": "For more control, click Advanced below.",
             "spandrel_filter": {
                 "label": "Image-to-Image Model",
                 "description": "Run an image-to-image model on the selected layer.",
@@ -1834,7 +1752,7 @@
                 "label": "Canny Edge Detection",
                 "description": "Generates an edge map from the selected layer using the Canny edge detection algorithm.",
                 "low_threshold": "Low Threshold",
-                "high_threshold": "High Threshold"
+                "high_threshold": "Hight Threshold"
             },
             "color_map": {
                 "label": "Color Map",
@@ -1910,25 +1828,6 @@
             "apply": "Apply",
             "cancel": "Cancel"
         },
-        "selectObject": {
-            "selectObject": "Select Object",
-            "pointType": "Point Type",
-            "invertSelection": "Invert Selection",
-            "include": "Include",
-            "exclude": "Exclude",
-            "neutral": "Neutral",
-            "apply": "Apply",
-            "reset": "Reset",
-            "saveAs": "Save As",
-            "cancel": "Cancel",
-            "process": "Process",
-            "help1": "Select a single target object. Add <Bold>Include</Bold> and <Bold>Exclude</Bold> points to indicate which parts of the layer are part of the target object.",
-            "help2": "Start with one <Bold>Include</Bold> point within the target object. Add more points to refine the selection. Fewer points typically produce better results.",
-            "help3": "Invert the selection to select everything except the target object.",
-            "clickToAdd": "Click on the layer to add a point",
-            "dragToMove": "Drag a point to move it",
-            "clickToRemove": "Click on a point to remove it"
-        },
         "settings": {
             "snapToGrid": {
                 "label": "Snap to Grid",
@@ -1939,10 +1838,10 @@
                 "label": "Preserve Masked Region",
                 "alert": "Preserving Masked Region"
             },
-            "isolatedStagingPreview": "Isolated Staging Preview",
             "isolatedPreview": "Isolated Preview",
-            "isolatedLayerPreview": "Isolated Layer Preview",
-            "isolatedLayerPreviewDesc": "Whether to show only this layer when performing operations like filtering or transforming.",
+            "isolatedStagingPreview": "Isolated Staging Preview",
+            "isolatedFilteringPreview": "Isolated Filtering Preview",
+            "isolatedTransformingPreview": "Isolated Transforming Preview",
             "invertBrushSizeScrollDirection": "Invert Scroll for Brush Size",
             "pressureSensitivity": "Pressure Sensitivity"
         },
@@ -1968,8 +1867,6 @@
             "newRegionalReferenceImage": "New Regional Reference Image",
             "newControlLayer": "New Control Layer",
             "newRasterLayer": "New Raster Layer",
-            "newInpaintMask": "New Inpaint Mask",
-            "newRegionalGuidance": "New Regional Guidance",
             "cropCanvasToBbox": "Crop Canvas to Bbox"
         },
         "stagingArea": {
@@ -2093,19 +1990,18 @@
         }
     },
     "newUserExperience": {
-        "toGetStartedLocal": "To get started, make sure to download or import models needed to run Invoke. Then, enter a prompt in the box and click <StrongComponent>Invoke</StrongComponent> to generate your first image. Select a prompt template to improve results. You can choose to save your images directly to the <StrongComponent>Gallery</StrongComponent> or edit them to the <StrongComponent>Canvas</StrongComponent>.",
         "toGetStarted": "To get started, enter a prompt in the box and click <StrongComponent>Invoke</StrongComponent> to generate your first image. Select a prompt template to improve results. You can choose to save your images directly to the <StrongComponent>Gallery</StrongComponent> or edit them to the <StrongComponent>Canvas</StrongComponent>.",
-        "gettingStartedSeries": "Want more guidance? Check out our <LinkComponent>Getting Started Series</LinkComponent> for tips on unlocking the full potential of the Invoke Studio.",
-        "downloadStarterModels": "Download Starter Models",
-        "importModels": "Import Models",
-        "noModelsInstalled": "It looks like you don't have any models installed"
+        "gettingStartedSeries": "Want more guidance? Check out our <LinkComponent>Getting Started Series</LinkComponent> for tips on unlocking the full potential of the Invoke Studio."
     },
     "whatsNew": {
         "whatsNewInInvoke": "What's New in Invoke",
-        "line1": "<StrongComponent>Layer Merging</StrongComponent>: New <StrongComponent>Merge Down</StrongComponent> and improved <StrongComponent>Merge Visible</StrongComponent> for all layers, with special handling for Regional Guidance and Control Layers.",
-        "line2": "<StrongComponent>HF Token Support</StrongComponent>: Upload models that require Hugging Face authentication.",
-        "readReleaseNotes": "Read Release Notes",
-        "watchRecentReleaseVideos": "Watch Recent Release Videos",
-        "watchUiUpdatesOverview": "Watch UI Updates Overview"
+        "canvasV2Announcement": {
+            "newCanvas": "A powerful new control canvas",
+            "newLayerTypes": "New layer types for even more control",
+            "fluxSupport": "Support for the Flux family of models",
+            "readReleaseNotes": "Read Release Notes",
+            "watchReleaseVideo": "Watch Release Video",
+            "watchUiUpdatesOverview": "Watch UI Updates Overview"
+        }
     }
 }

invokeai/frontend/web/public/locales/es.json

@@ -224,9 +224,7 @@
         "createIssue": "Crear un problema",
         "resetUI": "Interfaz de usuario $t(accessibility.reset)",
         "mode": "Modo",
-        "submitSupportTicket": "Enviar Ticket de Soporte",
-        "toggleRightPanel": "Activar o desactivar el panel derecho (G)",
-        "toggleLeftPanel": "Activar o desactivar el panel izquierdo (T)"
+        "submitSupportTicket": "Enviar Ticket de Soporte"
     },
     "nodes": {
         "zoomInNodes": "Acercar",
@@ -275,12 +273,7 @@
         "addSharedBoard": "Agregar Panel Compartido",
         "boards": "Paneles",
         "archiveBoard": "Archivar Panel",
-        "archived": "Archivado",
-        "selectedForAutoAdd": "Seleccionado para agregar automáticamente",
-        "unarchiveBoard": "Desarchivar el tablero",
-        "noBoards": "No hay tableros {{boardType}}",
-        "shared": "Carpetas compartidas",
-        "deletedPrivateBoardsCannotbeRestored": "Los tableros eliminados no se pueden restaurar. Al elegir \"Eliminar solo tablero\", las imágenes se colocan en un estado privado y sin categoría para el creador de la imagen."
+        "archived": "Archivado"
     },
     "accordions": {
         "compositing": {
@@ -323,13 +316,5 @@
         "inviteTeammates": "Invitar compañeros de equipo",
         "shareAccess": "Compartir acceso",
         "professionalUpsell": "Disponible en la edición profesional de Invoke. Haz clic aquí o visita invoke.com/pricing para obtener más detalles."
-    },
-    "controlLayers": {
-        "layer_one": "Capa",
-        "layer_many": "Capas",
-        "layer_other": "Capas",
-        "layer_withCount_one": "({{count}}) capa",
-        "layer_withCount_many": "({{count}}) capas",
-        "layer_withCount_other": "({{count}}) capas"
     }
 }

invokeai/frontend/web/public/locales/it.json

@@ -91,10 +91,7 @@
         "reset": "Reimposta",
         "none": "Niente",
         "new": "Nuovo",
-        "view": "Vista",
-        "close": "Chiudi",
-        "clipboard": "Appunti",
-        "ok": "Ok"
+        "view": "Vista"
     },
     "gallery": {
         "galleryImageSize": "Dimensione dell'immagine",
@@ -160,9 +157,7 @@
         "openViewer": "Apri visualizzatore",
         "closeViewer": "Chiudi visualizzatore",
         "imagesTab": "Immagini create e salvate in Invoke.",
-        "assetsTab": "File che hai caricato per usarli nei tuoi progetti.",
-        "boardsSettings": "Impostazioni Bacheche",
-        "imagesSettings": "Impostazioni Immagini Galleria"
+        "assetsTab": "File che hai caricato per usarli nei tuoi progetti."
     },
     "hotkeys": {
         "searchHotkeys": "Cerca tasti di scelta rapida",
@@ -544,6 +539,7 @@
         "defaultSettingsSaved": "Impostazioni predefinite salvate",
         "defaultSettings": "Impostazioni predefinite",
         "metadata": "Metadati",
+        "useDefaultSettings": "Usa le impostazioni predefinite",
         "triggerPhrases": "Frasi Trigger",
         "deleteModelImage": "Elimina l'immagine del modello",
         "localOnly": "solo locale",
@@ -578,26 +574,7 @@
         "noMatchingModels": "Nessun modello corrispondente",
         "starterModelsInModelManager": "I modelli iniziali possono essere trovati in Gestione Modelli",
         "spandrelImageToImage": "Immagine a immagine (Spandrel)",
-        "learnMoreAboutSupportedModels": "Scopri di più sui modelli che supportiamo",
-        "starterBundles": "Pacchetti per iniziare",
-        "installingBundle": "Installazione del pacchetto",
-        "skippingXDuplicates_one": ", saltando {{count}} duplicato",
-        "skippingXDuplicates_many": ", saltando {{count}} duplicati",
-        "skippingXDuplicates_other": ", saltando {{count}} duplicati",
-        "installingModel": "Installazione del modello",
-        "installingXModels_one": "Installazione di {{count}} modello",
-        "installingXModels_many": "Installazione di {{count}} modelli",
-        "installingXModels_other": "Installazione di {{count}} modelli",
-        "includesNModels": "Include {{n}} modelli e le loro dipendenze",
-        "starterBundleHelpText": "Installa facilmente tutti i modelli necessari per iniziare con un modello base, tra cui un modello principale, controlnet, adattatori IP e altro. Selezionando un pacchetto salterai tutti i modelli che hai già installato.",
-        "noDefaultSettings": "Nessuna impostazione predefinita configurata per questo modello. Visita Gestione Modelli per aggiungere impostazioni predefinite.",
-        "defaultSettingsOutOfSync": "Alcune impostazioni non corrispondono a quelle predefinite del modello:",
-        "restoreDefaultSettings": "Fare clic per utilizzare le impostazioni predefinite del modello.",
-        "usingDefaultSettings": "Utilizzo delle impostazioni predefinite del modello",
-        "huggingFace": "HuggingFace",
-        "huggingFaceRepoID": "HuggingFace Repository ID",
-        "clipEmbed": "CLIP Embed",
-        "t5Encoder": "T5 Encoder"
+        "learnMoreAboutSupportedModels": "Scopri di più sui modelli che supportiamo"
     },
     "parameters": {
         "images": "Immagini",
@@ -698,8 +675,7 @@
         "boxBlur": "Sfocatura Box",
         "staged": "Maschera espansa",
         "optimizedImageToImage": "Immagine-a-immagine ottimizzata",
-        "sendToCanvas": "Invia alla Tela",
-        "coherenceMinDenoise": "Riduzione minima del rumore"
+        "sendToCanvas": "Invia alla Tela"
     },
     "settings": {
         "models": "Modelli",
@@ -734,10 +710,7 @@
         "reloadingIn": "Ricaricando in",
         "informationalPopoversDisabled": "Testo informativo a comparsa disabilitato",
         "informationalPopoversDisabledDesc": "I testi informativi a comparsa sono disabilitati. Attivali nelle impostazioni.",
-        "confirmOnNewSession": "Conferma su nuova sessione",
-        "enableModelDescriptions": "Abilita le descrizioni dei modelli nei menu a discesa",
-        "modelDescriptionsDisabled": "Descrizioni dei modelli nei menu a discesa disabilitate",
-        "modelDescriptionsDisabledDesc": "Le descrizioni dei modelli nei menu a discesa sono state disabilitate. Abilitale nelle Impostazioni."
+        "confirmOnNewSession": "Conferma su nuova sessione"
     },
     "toast": {
         "uploadFailed": "Caricamento fallito",
@@ -746,7 +719,7 @@
         "serverError": "Errore del Server",
         "connected": "Connesso al server",
         "canceled": "Elaborazione annullata",
-        "uploadFailedInvalidUploadDesc": "Devono essere immagini PNG o JPEG.",
+        "uploadFailedInvalidUploadDesc": "Deve essere una singola immagine PNG o JPEG",
         "parameterSet": "Parametro richiamato",
         "parameterNotSet": "Parametro non richiamato",
         "problemCopyingImage": "Impossibile copiare l'immagine",
@@ -755,7 +728,7 @@
         "baseModelChangedCleared_other": "Cancellati o disabilitati {{count}} sottomodelli incompatibili",
         "loadedWithWarnings": "Flusso di lavoro caricato con avvisi",
         "imageUploaded": "Immagine caricata",
-        "addedToBoard": "Aggiunto alle risorse della bacheca {{name}}",
+        "addedToBoard": "Aggiunto alla bacheca",
         "modelAddedSimple": "Modello aggiunto alla Coda",
         "imageUploadFailed": "Caricamento immagine non riuscito",
         "setControlImage": "Imposta come immagine di controllo",
@@ -793,13 +766,7 @@
         "imageSaved": "Immagine salvata",
         "imageSavingFailed": "Salvataggio dell'immagine non riuscito",
         "layerCopiedToClipboard": "Livello copiato negli appunti",
-        "imageNotLoadedDesc": "Impossibile trovare l'immagine",
-        "linkCopied": "Collegamento copiato",
-        "addedToUncategorized": "Aggiunto alle risorse della bacheca $t(boards.uncategorized)",
-        "imagesWillBeAddedTo": "Le immagini caricate verranno aggiunte alle risorse della bacheca {{boardName}}.",
-        "uploadFailedInvalidUploadDesc_withCount_one": "Devi caricare al massimo 1 immagine PNG o JPEG.",
-        "uploadFailedInvalidUploadDesc_withCount_many": "Devi caricare al massimo {{count}} immagini PNG o JPEG.",
-        "uploadFailedInvalidUploadDesc_withCount_other": "Devi caricare al massimo {{count}} immagini PNG o JPEG."
+        "imageNotLoadedDesc": "Impossibile trovare l'immagine"
     },
     "accessibility": {
         "invokeProgressBar": "Barra di avanzamento generazione",
@@ -814,8 +781,7 @@
         "about": "Informazioni",
         "submitSupportTicket": "Invia ticket di supporto",
         "toggleLeftPanel": "Attiva/disattiva il pannello sinistro (T)",
-        "toggleRightPanel": "Attiva/disattiva il pannello destro (G)",
-        "uploadImages": "Carica immagine(i)"
+        "toggleRightPanel": "Attiva/disattiva il pannello destro (G)"
     },
     "nodes": {
         "zoomOutNodes": "Rimpicciolire",
@@ -956,7 +922,7 @@
         "saveToGallery": "Salva nella Galleria",
         "noMatchingWorkflows": "Nessun flusso di lavoro corrispondente",
         "noWorkflows": "Nessun flusso di lavoro",
-        "workflowHelpText": "Hai bisogno di aiuto? Consulta la nostra guida <LinkComponent>Introduzione ai flussi di lavoro</LinkComponent>."
+        "workflowHelpText": "Hai bisogno di aiuto? Consulta la nostra guida <LinkComponent>Introduzione ai flussi di lavoro</LinkComponent>"
     },
     "boards": {
         "autoAddBoard": "Aggiungi automaticamente bacheca",
@@ -1089,8 +1055,7 @@
         "noLoRAsInstalled": "Nessun LoRA installato",
         "addLora": "Aggiungi LoRA",
         "defaultVAE": "VAE predefinito",
-        "concepts": "Concetti",
-        "lora": "LoRA"
+        "concepts": "Concetti"
     },
     "invocationCache": {
         "disable": "Disabilita",
@@ -1147,8 +1112,7 @@
             "paragraphs": [
                 "Scegli quanti livelli del modello CLIP saltare.",
                 "Alcuni modelli funzionano meglio con determinate impostazioni di CLIP Skip."
-            ],
-            "heading": "CLIP Skip"
+            ]
         },
         "compositingCoherencePass": {
             "heading": "Passaggio di Coerenza",
@@ -1507,42 +1471,6 @@
                 "Controlla quanto il prompt influenza il processo di generazione.",
                 "Valori di guida elevati possono causare sovrasaturazione e una guida elevata o bassa può causare risultati di generazione distorti. La guida si applica solo ai modelli FLUX DEV."
             ]
-        },
-        "regionalReferenceImage": {
-            "paragraphs": [
-                "Pennello per applicare un'immagine di riferimento ad aree specifiche."
-            ],
-            "heading": "Immagine di riferimento Regionale"
-        },
-        "rasterLayer": {
-            "paragraphs": [
-                "Contenuto basato sui pixel della tua tela, utilizzato durante la generazione dell'immagine."
-            ],
-            "heading": "Livello Raster"
-        },
-        "regionalGuidance": {
-            "heading": "Guida Regionale",
-            "paragraphs": [
-                "Pennello per guidare la posizione in cui devono apparire gli elementi dei prompt globali."
-            ]
-        },
-        "regionalGuidanceAndReferenceImage": {
-            "heading": "Guida regionale e immagine di riferimento regionale",
-            "paragraphs": [
-                "Per la Guida Regionale, utilizzare il pennello per indicare dove devono apparire gli elementi dei prompt globali.",
-                "Per l'immagine di riferimento regionale, utilizzare il pennello per applicare un'immagine di riferimento ad aree specifiche."
-            ]
-        },
-        "globalReferenceImage": {
-            "heading": "Immagine di riferimento Globale",
-            "paragraphs": [
-                "Applica un'immagine di riferimento per influenzare l'intera generazione."
-            ]
-        },
-        "inpainting": {
-            "paragraphs": [
-                "Controlla quale area viene modificata, in base all'intensità di riduzione del rumore."
-            ]
         }
     },
     "sdxl": {
@@ -1564,6 +1492,7 @@
         "refinerSteps": "Passi Affinamento"
     },
     "metadata": {
+        "seamless": "Senza giunture",
         "positivePrompt": "Prompt positivo",
         "negativePrompt": "Prompt negativo",
         "generationMode": "Modalità generazione",
@@ -1590,11 +1519,7 @@
         "parameterSet": "Parametro {{parameter}} impostato",
         "parsingFailed": "Analisi non riuscita",
         "recallParameter": "Richiama {{label}}",
-        "canvasV2Metadata": "Tela",
-        "guidance": "Guida",
-        "seamlessXAxis": "Asse X senza giunte",
-        "seamlessYAxis": "Asse Y senza giunte",
-        "vae": "VAE"
+        "canvasV2Metadata": "Tela"
     },
     "hrf": {
         "enableHrf": "Abilita Correzione Alta Risoluzione",
@@ -1645,12 +1570,7 @@
         "defaultWorkflows": "Flussi di lavoro predefiniti",
         "uploadAndSaveWorkflow": "Carica nella libreria",
         "chooseWorkflowFromLibrary": "Scegli il flusso di lavoro dalla libreria",
-        "deleteWorkflow2": "Vuoi davvero eliminare questo flusso di lavoro? Questa operazione non può essere annullata.",
-        "edit": "Modifica",
-        "download": "Scarica",
-        "copyShareLink": "Copia Condividi Link",
-        "copyShareLinkForWorkflow": "Copia Condividi Link del Flusso di lavoro",
-        "delete": "Elimina"
+        "deleteWorkflow2": "Vuoi davvero eliminare questo flusso di lavoro? Questa operazione non può essere annullata."
     },
     "accordions": {
         "compositing": {
@@ -1691,11 +1611,11 @@
         "regionalGuidance": "Guida regionale",
         "opacity": "Opacità",
         "mergeVisible": "Fondi il visibile",
-        "mergeVisibleOk": "Livelli uniti",
+        "mergeVisibleOk": "Livelli visibili uniti",
         "deleteReferenceImage": "Elimina l'immagine di riferimento",
         "referenceImage": "Immagine di riferimento",
         "fitBboxToLayers": "Adatta il riquadro di delimitazione ai livelli",
-        "mergeVisibleError": "Errore durante l'unione dei livelli",
+        "mergeVisibleError": "Errore durante l'unione dei livelli visibili",
         "regionalReferenceImage": "Immagine di riferimento Regionale",
         "newLayerFromImage": "Nuovo livello da immagine",
         "newCanvasFromImage": "Nuova tela da immagine",
@@ -1787,7 +1707,7 @@
             "composition": "Solo Composizione",
             "ipAdapterMethod": "Metodo Adattatore IP"
         },
-        "showingType": "Mostra {{type}}",
+        "showingType": "Mostrare {{type}}",
         "dynamicGrid": "Griglia dinamica",
         "tool": {
             "view": "Muovi",
@@ -1915,6 +1835,8 @@
         "layer_withCount_one": "Livello ({{count}})",
         "layer_withCount_many": "Livelli ({{count}})",
         "layer_withCount_other": "Livelli ({{count}})",
+        "convertToControlLayer": "Converti in livello di controllo",
+        "convertToRasterLayer": "Converti in livello raster",
         "unlocked": "Sbloccato",
         "enableTransparencyEffect": "Abilita l'effetto trasparenza",
         "replaceLayer": "Sostituisci livello",
@@ -1927,7 +1849,9 @@
         "newCanvasSession": "Nuova sessione Tela",
         "deleteSelected": "Elimina selezione",
         "settings": {
+            "isolatedFilteringPreview": "Anteprima del filtraggio isolata",
             "isolatedStagingPreview": "Anteprima di generazione isolata",
+            "isolatedTransformingPreview": "Anteprima di trasformazione isolata",
             "isolatedPreview": "Anteprima isolata",
             "invertBrushSizeScrollDirection": "Inverti scorrimento per dimensione pennello",
             "snapToGrid": {
@@ -1939,20 +1863,14 @@
             "preserveMask": {
                 "alert": "Preservare la regione mascherata",
                 "label": "Preserva la regione mascherata"
-            },
-            "isolatedLayerPreview": "Anteprima livello isolato",
-            "isolatedLayerPreviewDesc": "Se visualizzare solo questo livello quando si eseguono operazioni come il filtraggio o la trasformazione."
+            }
         },
         "transform": {
             "reset": "Reimposta",
             "fitToBbox": "Adatta al Riquadro",
             "transform": "Trasforma",
             "apply": "Applica",
-            "cancel": "Annulla",
-            "fitMode": "Adattamento",
-            "fitModeContain": "Contieni",
-            "fitModeFill": "Riempi",
-            "fitModeCover": "Copri"
+            "cancel": "Annulla"
         },
         "stagingArea": {
             "next": "Successiva",
@@ -1986,46 +1904,8 @@
             "canvasGroup": "Tela",
             "newRasterLayer": "Nuovo Livello Raster",
             "saveCanvasToGallery": "Salva la Tela nella Galleria",
-            "saveToGalleryGroup": "Salva nella Galleria",
-            "newInpaintMask": "Nuova maschera Inpaint",
-            "newRegionalGuidance": "Nuova Guida Regionale"
-        },
-        "newImg2ImgCanvasFromImage": "Nuova Immagine da immagine",
-        "copyRasterLayerTo": "Copia $t(controlLayers.rasterLayer) in",
-        "copyControlLayerTo": "Copia $t(controlLayers.controlLayer) in",
-        "copyInpaintMaskTo": "Copia $t(controlLayers.inpaintMask) in",
-        "selectObject": {
-            "dragToMove": "Trascina un punto per spostarlo",
-            "clickToAdd": "Fare clic sul livello per aggiungere un punto",
-            "clickToRemove": "Clicca su un punto per rimuoverlo",
-            "help3": "Inverte la selezione per selezionare tutto tranne l'oggetto di destinazione.",
-            "pointType": "Tipo punto",
-            "apply": "Applica",
-            "reset": "Reimposta",
-            "cancel": "Annulla",
-            "selectObject": "Seleziona oggetto",
-            "invertSelection": "Inverti selezione",
-            "exclude": "Escludi",
-            "include": "Includi",
-            "neutral": "Neutro",
-            "saveAs": "Salva come",
-            "process": "Elabora",
-            "help1": "Seleziona un singolo oggetto di destinazione. Aggiungi i punti <Bold>Includi</Bold> e <Bold>Escludi</Bold> per indicare quali parti del livello fanno parte dell'oggetto di destinazione.",
-            "help2": "Inizia con un punto <Bold>Include</Bold> all'interno dell'oggetto di destinazione. Aggiungi altri punti per perfezionare la selezione. Meno punti in genere producono risultati migliori."
-        },
-        "convertControlLayerTo": "Converti $t(controlLayers.controlLayer) in",
-        "newRasterLayer": "Nuovo $t(controlLayers.rasterLayer)",
-        "newRegionalGuidance": "Nuova $t(controlLayers.regionalGuidance)",
-        "canvasAsRasterLayer": "$t(controlLayers.canvas) come $t(controlLayers.rasterLayer)",
-        "canvasAsControlLayer": "$t(controlLayers.canvas) come $t(controlLayers.controlLayer)",
-        "convertInpaintMaskTo": "Converti $t(controlLayers.inpaintMask) in",
-        "copyRegionalGuidanceTo": "Copia $t(controlLayers.regionalGuidance) in",
-        "convertRasterLayerTo": "Converti $t(controlLayers.rasterLayer) in",
-        "convertRegionalGuidanceTo": "Converti $t(controlLayers.regionalGuidance) in",
-        "newControlLayer": "Nuovo $t(controlLayers.controlLayer)",
-        "newInpaintMask": "Nuova $t(controlLayers.inpaintMask)",
-        "replaceCurrent": "Sostituisci corrente",
-        "mergeDown": "Unire in basso"
+            "saveToGalleryGroup": "Salva nella Galleria"
+        }
     },
     "ui": {
         "tabs": {
@@ -2111,20 +1991,18 @@
     },
     "newUserExperience": {
         "gettingStartedSeries": "Desideri maggiori informazioni? Consulta la nostra <LinkComponent>Getting Started Series</LinkComponent> per suggerimenti su come sfruttare appieno il potenziale di Invoke Studio.",
-        "toGetStarted": "Per iniziare, inserisci un prompt nella casella e fai clic su <StrongComponent>Invoke</StrongComponent> per generare la tua prima immagine. Seleziona un modello di prompt per migliorare i risultati. Puoi scegliere di salvare le tue immagini direttamente nella <StrongComponent>Galleria</StrongComponent> o modificarle nella <StrongComponent>Tela</StrongComponent>.",
-        "importModels": "Importa modelli",
-        "downloadStarterModels": "Scarica i modelli per iniziare",
-        "noModelsInstalled": "Sembra che tu non abbia installato alcun modello",
-        "toGetStartedLocal": "Per iniziare, assicurati di scaricare o importare i modelli necessari per eseguire Invoke. Quindi, inserisci un prompt nella casella e fai clic su <StrongComponent>Invoke</StrongComponent> per generare la tua prima immagine. Seleziona un modello di prompt per migliorare i risultati. Puoi scegliere di salvare le tue immagini direttamente nella <StrongComponent>Galleria</StrongComponent> o modificarle nella <StrongComponent>Tela</StrongComponent>."
+        "toGetStarted": "Per iniziare, inserisci un prompt nella casella e fai clic su <StrongComponent>Invoke</StrongComponent> per generare la tua prima immagine. Seleziona un modello di prompt per migliorare i risultati. Puoi scegliere di salvare le tue immagini direttamente nella <StrongComponent>Galleria</StrongComponent> o modificarle nella <StrongComponent>Tela</StrongComponent>."
     },
     "whatsNew": {
-        "whatsNewInInvoke": "Novità in Invoke",
-        "line2": "Supporto Flux esteso, ora con immagini di riferimento globali",
-        "line3": "Tooltip e menu contestuali migliorati",
-        "readReleaseNotes": "Leggi le note di rilascio",
-        "watchRecentReleaseVideos": "Guarda i video su questa versione",
-        "line1": "Strumento <ItalicComponent>Seleziona oggetto</ItalicComponent> per la selezione e la modifica precise degli oggetti",
-        "watchUiUpdatesOverview": "Guarda le novità dell'interfaccia"
+        "canvasV2Announcement": {
+            "readReleaseNotes": "Leggi le Note di Rilascio",
+            "fluxSupport": "Supporto per la famiglia di modelli Flux",
+            "newCanvas": "Una nuova potente tela di controllo",
+            "watchReleaseVideo": "Guarda il video di rilascio",
+            "watchUiUpdatesOverview": "Guarda le novità dell'interfaccia",
+            "newLayerTypes": "Nuovi tipi di livello per un miglior controllo"
+        },
+        "whatsNewInInvoke": "Novità in Invoke"
     },
     "system": {
         "logLevel": {

invokeai/frontend/web/public/locales/ja.json

@@ -229,6 +229,7 @@
         "submitSupportTicket": "サポート依頼を送信する"
     },
     "metadata": {
+        "seamless": "シームレス",
         "Threshold": "ノイズ閾値",
         "seed": "シード",
         "width": "幅",

invokeai/frontend/web/public/locales/nl.json

@@ -155,6 +155,7 @@
         "path": "Pad",
         "triggerPhrases": "Triggerzinnen",
         "typePhraseHere": "Typ zin hier in",
+        "useDefaultSettings": "Gebruik standaardinstellingen",
         "modelImageDeleteFailed": "Fout bij verwijderen modelafbeelding",
         "modelImageUpdated": "Modelafbeelding bijgewerkt",
         "modelImageUpdateFailed": "Fout bij bijwerken modelafbeelding",
@@ -665,6 +666,7 @@
         }
     },
     "metadata": {
+        "seamless": "Naadloos",
         "positivePrompt": "Positieve prompt",
         "negativePrompt": "Negatieve prompt",
         "generationMode": "Genereermodus",

invokeai/frontend/web/public/locales/ru.json

@@ -94,8 +94,7 @@
         "reset": "Сброс",
         "none": "Ничего",
         "new": "Новый",
-        "ok": "Ok",
-        "close": "Закрыть"
+        "ok": "Ok"
     },
     "gallery": {
         "galleryImageSize": "Размер изображений",
@@ -161,9 +160,7 @@
         "openViewer": "Открыть просмотрщик",
         "closeViewer": "Закрыть просмотрщик",
         "imagesTab": "Изображения, созданные и сохраненные в Invoke.",
-        "assetsTab": "Файлы, которые вы загрузили для использования в своих проектах.",
-        "boardsSettings": "Настройки доски",
-        "imagesSettings": "Настройки галереи изображений"
+        "assetsTab": "Файлы, которые вы загрузили для использования в своих проектах."
     },
     "hotkeys": {
         "searchHotkeys": "Поиск горячих клавиш",
@@ -544,6 +541,7 @@
         "scanResults": "Результаты сканирования",
         "source": "Источник",
         "triggerPhrases": "Триггерные фразы",
+        "useDefaultSettings": "Использовать стандартные настройки",
         "modelName": "Название модели",
         "modelSettings": "Настройки модели",
         "upcastAttention": "Внимание",
@@ -572,6 +570,7 @@
         "simpleModelPlaceholder": "URL или путь к локальному файлу или папке diffusers",
         "urlOrLocalPath": "URL или локальный путь",
         "urlOrLocalPathHelper": "URL-адреса должны указывать на один файл. Локальные пути могут указывать на один файл или папку для одной модели диффузоров.",
+        "hfToken": "Токен HuggingFace",
         "starterModels": "Стартовые модели",
         "textualInversions": "Текстовые инверсии",
         "loraModels": "LoRAs",
@@ -584,18 +583,7 @@
         "learnMoreAboutSupportedModels": "Подробнее о поддерживаемых моделях",
         "t5Encoder": "T5 энкодер",
         "spandrelImageToImage": "Image to Image (Spandrel)",
-        "clipEmbed": "CLIP Embed",
-        "installingXModels_one": "Установка {{count}} модели",
-        "installingXModels_few": "Установка {{count}} моделей",
-        "installingXModels_many": "Установка {{count}} моделей",
-        "installingBundle": "Установка пакета",
-        "installingModel": "Установка модели",
-        "starterBundles": "Стартовые пакеты",
-        "skippingXDuplicates_one": ", пропуская {{count}} дубликат",
-        "skippingXDuplicates_few": ", пропуская {{count}} дубликата",
-        "skippingXDuplicates_many": ", пропуская {{count}} дубликатов",
-        "includesNModels": "Включает в себя {{n}} моделей и их зависимостей",
-        "starterBundleHelpText": "Легко установите все модели, необходимые для начала работы с базовой моделью, включая основную модель, сети управления, IP-адаптеры и многое другое. При выборе комплекта все уже установленные модели будут пропущены."
+        "clipEmbed": "CLIP Embed"
     },
     "parameters": {
         "images": "Изображения",
@@ -742,7 +730,7 @@
         "serverError": "Ошибка сервера",
         "connected": "Подключено к серверу",
         "canceled": "Обработка отменена",
-        "uploadFailedInvalidUploadDesc": "Это должны быть изображения PNG или JPEG.",
+        "uploadFailedInvalidUploadDesc": "Должно быть одно изображение в формате PNG или JPEG",
         "parameterNotSet": "Параметр не задан",
         "parameterSet": "Параметр задан",
         "problemCopyingImage": "Не удается скопировать изображение",
@@ -754,7 +742,7 @@
         "setNodeField": "Установить как поле узла",
         "invalidUpload": "Неверная загрузка",
         "imageUploaded": "Изображение загружено",
-        "addedToBoard": "Добавлено в активы доски {{name}}",
+        "addedToBoard": "Добавлено на доску",
         "workflowLoaded": "Рабочий процесс загружен",
         "problemDeletingWorkflow": "Проблема с удалением рабочего процесса",
         "modelAddedSimple": "Модель добавлена в очередь",
@@ -789,13 +777,7 @@
         "unableToLoadStylePreset": "Невозможно загрузить предустановку стиля",
         "layerCopiedToClipboard": "Слой скопирован в буфер обмена",
         "sentToUpscale": "Отправить на увеличение",
-        "layerSavedToAssets": "Слой сохранен в активах",
-        "linkCopied": "Ссылка скопирована",
-        "addedToUncategorized": "Добавлено в активы доски $t(boards.uncategorized)",
-        "imagesWillBeAddedTo": "Загруженные изображения будут добавлены в активы доски {{boardName}}.",
-        "uploadFailedInvalidUploadDesc_withCount_one": "Должно быть не более {{count}} изображения в формате PNG или JPEG.",
-        "uploadFailedInvalidUploadDesc_withCount_few": "Должно быть не более {{count}} изображений в формате PNG или JPEG.",
-        "uploadFailedInvalidUploadDesc_withCount_many": "Должно быть не более {{count}} изображений в формате PNG или JPEG."
+        "layerSavedToAssets": "Слой сохранен в активах"
     },
     "accessibility": {
         "uploadImage": "Загрузить изображение",
@@ -810,8 +792,7 @@
         "about": "Об этом",
         "submitSupportTicket": "Отправить тикет в службу поддержки",
         "toggleRightPanel": "Переключить правую панель (G)",
-        "toggleLeftPanel": "Переключить левую панель (T)",
-        "uploadImages": "Загрузить изображения"
+        "toggleLeftPanel": "Переключить левую панель (T)"
     },
     "nodes": {
         "zoomInNodes": "Увеличьте масштаб",
@@ -952,7 +933,7 @@
         "saveToGallery": "Сохранить в галерею",
         "noWorkflows": "Нет рабочих процессов",
         "noMatchingWorkflows": "Нет совпадающих рабочих процессов",
-        "workflowHelpText": "Нужна помощь? Ознакомьтесь с нашим руководством <LinkComponent>Getting Started with Workflows</LinkComponent>."
+        "workflowHelpText": "Нужна помощь? Ознакомьтесь с нашим руководством <LinkComponent>Getting Started with Workflows</LinkComponent>"
     },
     "boards": {
         "autoAddBoard": "Авто добавление Доски",
@@ -1400,6 +1381,7 @@
         }
     },
     "metadata": {
+        "seamless": "Бесшовность",
         "positivePrompt": "Запрос",
         "negativePrompt": "Негативный запрос",
         "generationMode": "Режим генерации",
@@ -1427,8 +1409,7 @@
         "recallParameter": "Отозвать {{label}}",
         "allPrompts": "Все запросы",
         "imageDimensions": "Размеры изображения",
-        "canvasV2Metadata": "Холст",
-        "guidance": "Точность"
+        "canvasV2Metadata": "Холст"
     },
     "queue": {
         "status": "Статус",
@@ -1580,12 +1561,7 @@
         "defaultWorkflows": "Стандартные рабочие процессы",
         "deleteWorkflow2": "Вы уверены, что хотите удалить этот рабочий процесс? Это нельзя отменить.",
         "chooseWorkflowFromLibrary": "Выбрать рабочий процесс из библиотеки",
-        "uploadAndSaveWorkflow": "Загрузить в библиотеку",
-        "edit": "Редактировать",
-        "download": "Скачать",
-        "copyShareLink": "Скопировать ссылку на общий доступ",
-        "copyShareLinkForWorkflow": "Скопировать ссылку на общий доступ для рабочего процесса",
-        "delete": "Удалить"
+        "uploadAndSaveWorkflow": "Загрузить в библиотеку"
     },
     "hrf": {
         "enableHrf": "Включить исправление высокого разрешения",
@@ -1833,12 +1809,14 @@
         },
         "settings": {
             "isolatedPreview": "Изолированный предпросмотр",
+            "isolatedTransformingPreview": "Изолированный предпросмотр преобразования",
             "invertBrushSizeScrollDirection": "Инвертировать прокрутку для размера кисти",
             "snapToGrid": {
                 "label": "Привязка к сетке",
                 "on": "Вкл",
                 "off": "Выкл"
             },
+            "isolatedFilteringPreview": "Изолированный предпросмотр фильтрации",
             "pressureSensitivity": "Чувствительность к давлению",
             "isolatedStagingPreview": "Изолированный предпросмотр на промежуточной стадии",
             "preserveMask": {
@@ -1860,6 +1838,7 @@
         "enableAutoNegative": "Включить авто негатив",
         "maskFill": "Заполнение маски",
         "viewProgressInViewer": "Просматривайте прогресс и результаты в <Btn>Просмотрщике изображений</Btn>.",
+        "convertToRasterLayer": "Конвертировать в растровый слой",
         "tool": {
             "move": "Двигать",
             "bbox": "Ограничительная рамка",
@@ -1911,10 +1890,7 @@
             "fitToBbox": "Вместить в рамку",
             "reset": "Сбросить",
             "apply": "Применить",
-            "cancel": "Отменить",
-            "fitModeContain": "Уместить",
-            "fitMode": "Режим подгонки",
-            "fitModeFill": "Заполнить"
+            "cancel": "Отменить"
         },
         "disableAutoNegative": "Отключить авто негатив",
         "deleteReferenceImage": "Удалить эталонное изображение",
@@ -1927,6 +1903,7 @@
         "newGallerySession": "Новая сессия галереи",
         "sendToCanvasDesc": "Нажатие кнопки Invoke отображает вашу текущую работу на холсте.",
         "globalReferenceImages_withCount_hidden": "Глобальные эталонные изображения ({{count}} скрыто)",
+        "convertToControlLayer": "Конвертировать в контрольный слой",
         "layer_withCount_one": "Слой ({{count}})",
         "layer_withCount_few": "Слои ({{count}})",
         "layer_withCount_many": "Слои ({{count}})",
@@ -1943,8 +1920,7 @@
         "globalReferenceImage": "Глобальное эталонное изображение",
         "sendToGallery": "Отправить в галерею",
         "referenceImage": "Эталонное изображение",
-        "addGlobalReferenceImage": "Добавить $t(controlLayers.globalReferenceImage)",
-        "newImg2ImgCanvasFromImage": "Новое img2img из изображения"
+        "addGlobalReferenceImage": "Добавить $t(controlLayers.globalReferenceImage)"
     },
     "ui": {
         "tabs": {
@@ -2056,6 +2032,14 @@
         }
     },
     "whatsNew": {
+        "canvasV2Announcement": {
+            "newLayerTypes": "Новые типы слоев для еще большего контроля",
+            "readReleaseNotes": "Прочитать информацию о выпуске",
+            "watchReleaseVideo": "Смотреть видео о выпуске",
+            "fluxSupport": "Поддержка семейства моделей Flux",
+            "newCanvas": "Новый мощный холст управления",
+            "watchUiUpdatesOverview": "Обзор обновлений пользовательского интерфейса"
+        },
         "whatsNewInInvoke": "Что нового в Invoke"
     },
     "newUserExperience": {