Change tiling strategy to make TiledStableDiffusionRefineInvocation work with more tile shapes and overlaps.

- Fix type errors
- Enable SilenceWarnings to be used as both a context manager and a decorator
- Remove duplicate implementation
- Check the initial verbosity on __enter__() rather than __init__()

.github/actions/install-frontend-deps/action.yml

@@ -9,9 +9,9 @@ runs:
         node-version: '18'
 
     - name: setup pnpm
-      uses: pnpm/action-setup@v4
+      uses: pnpm/action-setup@v2
       with:
-        version: 8.15.6
+        version: 8
         run_install: false
 
     - name: get pnpm store directory

.github/pull_request_template.md

@@ -8,7 +8,7 @@
 
 ## QA Instructions
 
-<!--WHEN APPLICABLE: Describe how you have tested the changes in this PR. Provide enough detail that a reviewer can reproduce your tests.-->
+<!--WHEN APPLICABLE: Describe how we can test the changes in this PR.-->
 
 ## Merge Plan
 

README.md

@@ -12,24 +12,12 @@
 
 Invoke is a leading creative engine built to empower professionals and enthusiasts alike. Generate and create stunning visual media using the latest AI-driven technologies. Invoke offers an industry leading web-based UI, and serves as the foundation for multiple commercial products.
 
-Invoke is available in two editions:
-
-| **Community Edition**                                                                                                      | **Professional Edition**                                                                            |
-|----------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------|
-| **For users looking for a locally installed, self-hosted and self-managed service**                                         | **For users or teams looking for a cloud-hosted, fully managed service**                            |
-| - Free to use under a commercially-friendly license                                                                         | - Monthly subscription fee with three different plan levels                                         |
-| - Download and install on compatible hardware                                                                               | - Offers additional benefits, including multi-user support, improved model training, and more                          |
-| - Includes all core studio features: generate, refine, iterate on images, and build workflows                               | - Hosted in the cloud for easy, secure model access and scalability                                               |
-| Quick Start -> [Installation and Updates][installation docs]                                                                     | More Information -> [www.invoke.com/pricing](https://www.invoke.com/pricing)                        |
+[Installation and Updates][installation docs] - [Documentation and Tutorials][docs home] - [Bug Reports][github issues] - [Contributing][contributing docs]
 
+<div align="center">
 
 ![Highlighted Features - Canvas and Workflows](https://github.com/invoke-ai/InvokeAI/assets/31807370/708f7a82-084f-4860-bfbe-e2588c53548d)
 
-# Documentation
-| **Quick Links**                                                                                                      | 
-|----------------------------------------------------------------------------------------------------------------------------|
-|  [Installation and Updates][installation docs] - [Documentation and Tutorials][docs home] - [Bug Reports][github issues] - [Contributing][contributing docs]  | 
-
 </div>
 
 ## Quick Start
@@ -49,33 +37,6 @@ Invoke is available in two editions:
 
 More detail, including hardware requirements and manual install instructions, are available in the [installation documentation][installation docs].
 
-## Docker Container
-
-We publish official container images in Github Container Registry: https://github.com/invoke-ai/InvokeAI/pkgs/container/invokeai. Both CUDA and ROCm images are available. Check the above link for relevant tags.
-
-> [!IMPORTANT]
-> Ensure that Docker is set up to use the GPU. Refer to [NVIDIA][nvidia docker docs] or [AMD][amd docker docs] documentation.
-
-### Generate!
-
-Run the container, modifying the command as necessary:
-
-```bash
-docker run --runtime=nvidia --gpus=all --publish 9090:9090 ghcr.io/invoke-ai/invokeai
-```
-
-Then open `http://localhost:9090` and install some models using the Model Manager tab to begin generating.
-
-For ROCm, add `--device /dev/kfd --device /dev/dri` to the `docker run` command.
-
-### Persist your data
-
-You will likely want to persist your workspace outside of the container. Use the `--volume /home/myuser/invokeai:/invokeai` flag to mount some local directory (using its **absolute** path) to the `/invokeai` path inside the container. Your generated images and models will reside there. You can use this directory with other InvokeAI installations, or switch between runtime directories as needed.
-
-### DIY
-
-Build your own image and customize the environment to match your needs using our `docker-compose` stack. See [README.md](./docker/README.md) in the [docker](./docker) directory.
-
 ## Troubleshooting, FAQ and Support
 
 Please review our [FAQ][faq] for solutions to common installation problems and other issues.
@@ -153,5 +114,3 @@ Original portions of the software are Copyright © 2024 by respective contributo
 [latest release link]: https://github.com/invoke-ai/InvokeAI/releases/latest
 [translation status badge]: https://hosted.weblate.org/widgets/invokeai/-/svg-badge.svg
 [translation status link]: https://hosted.weblate.org/engage/invokeai/
-[nvidia docker docs]: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
-[amd docker docs]: https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html

docker/.env.sample

@@ -19,9 +19,8 @@
 ## INVOKEAI_PORT is the port on which the InvokeAI web interface will be available
 # INVOKEAI_PORT=9090
 
-## GPU_DRIVER can be set to either `cuda` or `rocm` to enable GPU support in the container accordingly.
-# GPU_DRIVER=cuda #| rocm
+## GPU_DRIVER can be set to either `nvidia` or `rocm` to enable GPU support in the container accordingly.
+# GPU_DRIVER=nvidia #| rocm
 
 ## CONTAINER_UID can be set to the UID of the user on the host system that should own the files in the container.
-## It is usually not necessary to change this. Use `id -u` on the host system to find the UID.
 # CONTAINER_UID=1000

docker/README.md

@@ -1,75 +1,41 @@
-# Invoke in Docker
+# InvokeAI Containerized
 
-- Ensure that Docker can use the GPU on your system
-- This documentation assumes Linux, but should work similarly under Windows with WSL2
-- We don't recommend running Invoke in Docker on macOS at this time. It works, but very slowly.
+All commands should be run within the `docker` directory: `cd docker`
 
-## Quickstart :lightning:
+## Quickstart :rocket:
 
-No `docker compose`, no persistence, just a simple one-liner using the official images:
+On a known working Linux+Docker+CUDA (Nvidia) system, execute `./run.sh` in this directory. It will take a few minutes - depending on your internet speed - to install the core models. Once the application starts up, open `http://localhost:9090` in your browser to Invoke!
 
-**CUDA:**
+For more configuration options (using an AMD GPU, custom root directory location, etc): read on.
 
-```bash
-docker run --runtime=nvidia --gpus=all --publish 9090:9090 ghcr.io/invoke-ai/invokeai
-```
-
-**ROCm:**
-
-```bash
-docker run --device /dev/kfd --device /dev/dri --publish 9090:9090 ghcr.io/invoke-ai/invokeai:main-rocm
-```
-
-Open `http://localhost:9090` in your browser once the container finishes booting, install some models, and generate away!
-
-> [!TIP]
-> To persist your data (including downloaded models) outside of the container, add a `--volume/-v` flag to the above command, e.g.: `docker run --volume /some/local/path:/invokeai <...the rest of the command>`
-
-## Customize the container
-
-We ship the `run.sh` script, which is a convenient wrapper around `docker compose` for cases where custom image build args are needed. Alternatively, the familiar `docker compose` commands work just as well.
-
-```bash
-cd docker
-cp .env.sample .env
-# edit .env to your liking if you need to; it is well commented.
-./run.sh
-```
-
-It will take a few minutes to build the image the first time. Once the application starts up, open `http://localhost:9090` in your browser to invoke!
-
-## Docker setup in detail
+## Detailed setup
 
 #### Linux
 
 1. Ensure builkit is enabled in the Docker daemon settings (`/etc/docker/daemon.json`)
 2. Install the `docker compose` plugin using your package manager, or follow a [tutorial](https://docs.docker.com/compose/install/linux/#install-using-the-repository).
-    - The deprecated `docker-compose` (hyphenated) CLI probably won't work. Update to a recent version.
+    - The deprecated `docker-compose` (hyphenated) CLI continues to work for now.
 3. Ensure docker daemon is able to access the GPU.
-    - [NVIDIA docs](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
-    - [AMD docs](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html)
+    - You may need to install [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
 
 #### macOS
 
-> [!TIP]
-> You'll be better off installing Invoke directly on your system, because Docker can not use the GPU on macOS.
-
-If you are still reading:
-
 1. Ensure Docker has at least 16GB RAM
 2. Enable VirtioFS for file sharing
 3. Enable `docker compose` V2 support
 
-This is done via Docker Desktop preferences.
+This is done via Docker Desktop preferences
 
-### Configure the Invoke Environment
+### Configure Invoke environment
 
-1. Make a copy of `.env.sample` and name it `.env` (`cp .env.sample .env` (Mac/Linux) or `copy example.env .env` (Windows)). Make changes as necessary. Set `INVOKEAI_ROOT` to an absolute path to the desired location of the InvokeAI runtime directory. It may be an existing directory from a previous installation (post 4.0.0).
+1. Make a copy of `.env.sample` and name it `.env` (`cp .env.sample .env` (Mac/Linux) or `copy example.env .env` (Windows)). Make changes as necessary. Set `INVOKEAI_ROOT` to an absolute path to:
+    a. the desired location of the InvokeAI runtime directory, or
+    b. an existing, v3.0.0 compatible runtime directory.
 1. Execute `run.sh`
 
 The image will be built automatically if needed.
 
-The runtime directory (holding models and outputs) will be created in the location specified by `INVOKEAI_ROOT`. The default location is `~/invokeai`. Navigate to the Model Manager tab and install some models before generating.
+The runtime directory (holding models and outputs) will be created in the location specified by `INVOKEAI_ROOT`. The default location is `~/invokeai`. The runtime directory will be populated with the base configs and models necessary to start generating.
 
 ### Use a GPU
 
@@ -77,9 +43,9 @@ The runtime directory (holding models and outputs) will be created in the locati
 - WSL2 is *required* for Windows.
 - only `x86_64` architecture is supported.
 
-The Docker daemon on the system must be already set up to use the GPU. In case of Linux, this involves installing `nvidia-docker-runtime` and configuring the `nvidia` runtime as default. Steps will be different for AMD. Please see Docker/NVIDIA/AMD documentation for the most up-to-date instructions for using your GPU with Docker.
+The Docker daemon on the system must be already set up to use the GPU. In case of Linux, this involves installing `nvidia-docker-runtime` and configuring the `nvidia` runtime as default. Steps will be different for AMD. Please see Docker documentation for the most up-to-date instructions for using your GPU with Docker.
 
-To use an AMD GPU, set `GPU_DRIVER=rocm` in your `.env` file before running `./run.sh`.
+To use an AMD GPU, set `GPU_DRIVER=rocm` in your `.env` file.
 
 ## Customize
 
@@ -93,10 +59,10 @@ Values are optional, but setting `INVOKEAI_ROOT` is highly recommended. The defa
 INVOKEAI_ROOT=/Volumes/WorkDrive/invokeai
 HUGGINGFACE_TOKEN=the_actual_token
 CONTAINER_UID=1000
-GPU_DRIVER=cuda
+GPU_DRIVER=nvidia
 ```
 
-Any environment variables supported by InvokeAI can be set here. See the [Configuration docs](https://invoke-ai.github.io/InvokeAI/features/CONFIGURATION/) for further detail.
+Any environment variables supported by InvokeAI can be set here - please see the [Configuration docs](https://invoke-ai.github.io/InvokeAI/features/CONFIGURATION/) for further detail.
 
 ## Even More Customizing!
 

docker/docker-compose.yml

@@ -1,5 +1,7 @@
 # Copyright (c) 2023 Eugene Brodsky https://github.com/ebr
 
+version: '3.8'
+
 x-invokeai: &invokeai
     image: "local/invokeai:latest"
     build:
@@ -30,7 +32,7 @@ x-invokeai: &invokeai
 
 
 services:
-  invokeai-cuda:
+  invokeai-nvidia:
     <<: *invokeai
     deploy:
       resources:

docker/docker-entrypoint.sh

@@ -23,18 +23,18 @@ usermod -u ${USER_ID} ${USER} 1>/dev/null
 # but it is useful to have the full SSH server e.g. on Runpod.
 # (use SCP to copy files to/from the image, etc)
 if [[ -v "PUBLIC_KEY" ]] && [[ ! -d "${HOME}/.ssh" ]]; then
-  apt-get update
-  apt-get install -y openssh-server
-  pushd "$HOME"
-  mkdir -p .ssh
-  echo "${PUBLIC_KEY}" >.ssh/authorized_keys
-  chmod -R 700 .ssh
-  popd
-  service ssh start
+    apt-get update
+    apt-get install -y openssh-server
+    pushd "$HOME"
+    mkdir -p .ssh
+    echo "${PUBLIC_KEY}" > .ssh/authorized_keys
+    chmod -R 700 .ssh
+    popd
+    service ssh start
 fi
 
 mkdir -p "${INVOKEAI_ROOT}"
-chown --recursive ${USER} "${INVOKEAI_ROOT}" || true
+chown --recursive ${USER} "${INVOKEAI_ROOT}"
 cd "${INVOKEAI_ROOT}"
 
 # Run the CMD as the Container User (not root).

docker/run.sh

@@ -8,15 +8,11 @@ run() {
   local build_args=""
   local profile=""
 
-  # create .env file if it doesn't exist, otherwise docker compose will fail
   touch .env
-
-  # parse .env file for build args
   build_args=$(awk '$1 ~ /=[^$]/ && $0 !~ /^#/ {print "--build-arg " $0 " "}' .env) &&
   profile="$(awk -F '=' '/GPU_DRIVER/ {print $2}' .env)"
 
-  # default to 'cuda' profile
-  [[ -z "$profile" ]] && profile="cuda"
+  [[ -z "$profile" ]] && profile="nvidia"
 
   local service_name="invokeai-$profile"
 

docs/contributing/DOWNLOAD_QUEUE.md

@@ -128,8 +128,7 @@ The queue operates on a series of download job objects. These objects
 specify the source and destination of the download, and keep track of
 the progress of the download.
 
-Two job types are defined. `DownloadJob` and
-`MultiFileDownloadJob`. The former is a pydantic object with the
+The only job type currently implemented is `DownloadJob`, a pydantic object with the
 following fields:
 
 | **Field**      | **Type**        |  **Default**  | **Description** |
@@ -139,7 +138,7 @@ following fields:
 | `dest`           | Path            |               | Where to download to              |
 | `access_token`   | str             |               | [optional] string containing authentication token for access |
 | `on_start`       | Callable        |               | [optional] callback when the download starts |
-| `on_progress` | Callable | | [optional] callback called at intervals during download progress |
+| `on_progress`    | Callable        |               | [optional] callback called at intervals during download progress |
 | `on_complete`    | Callable        |               | [optional] callback called after successful download completion |
 | `on_error`       | Callable        |               | [optional] callback called after an error occurs  |
 | `id`             | int             | auto assigned | Job ID, an integer >= 0           |
@@ -191,33 +190,6 @@ A cancelled job will have status `DownloadJobStatus.ERROR` and an
 `error_type` field of "DownloadJobCancelledException". In addition,
 the job's `cancelled` property will be set to True.
 
-The `MultiFileDownloadJob` is used for diffusers model downloads,
-which contain multiple files and directories under a common root:
-
-| **Field**      | **Type**        |  **Default**  | **Description** |
-|----------------|-----------------|---------------|-----------------|
-| _Fields passed in at job creation time_                               |
-| `download_parts` | Set[DownloadJob]|               | Component download jobs |
-| `dest`           | Path            |               | Where to download to              |
-| `on_start`       | Callable        |               | [optional] callback when the download starts |
-| `on_progress` | Callable | | [optional] callback called at intervals during download progress |
-| `on_complete`    | Callable        |               | [optional] callback called after successful download completion |
-| `on_error`       | Callable        |               | [optional] callback called after an error occurs  |
-| `id`             | int             | auto assigned | Job ID, an integer >= 0           |
-| _Fields updated over the course of the download task_
-| `status`         | DownloadJobStatus|              | Status code                                |
-| `download_path`  | Path |              | Path to the root of the downloaded files |
-| `bytes`          | int              | 0            | Bytes downloaded so far   |
-| `total_bytes`    | int              | 0            | Total size of the file at the remote site  |
-| `error_type`     | str              |              | String version of the exception that caused an error during download |
-| `error`          | str              |              | String version of the traceback associated with an error |
-| `cancelled`      | bool             | False        | Set to true if the job was cancelled by the caller|
-
-Note that the MultiFileDownloadJob does not support the `priority`,
-`job_started`, `job_ended` or `content_type` attributes. You can get
-these from the individual download jobs in `download_parts`.
-
-
 ### Callbacks
 
 Download jobs can be associated with a series of callbacks, each with
@@ -279,40 +251,11 @@ jobs using `list_jobs()`, fetch a single job by its with
 running jobs with `cancel_all_jobs()`, and wait for all jobs to finish
 with `join()`.
 
-#### job = queue.download(source, dest, priority, access_token, on_start, on_progress, on_complete, on_cancelled, on_error)
+#### job = queue.download(source, dest, priority, access_token)
 
 Create a new download job and put it on the queue, returning the
 DownloadJob object.
 
-#### multifile_job = queue.multifile_download(parts, dest, access_token, on_start, on_progress, on_complete, on_cancelled, on_error)
-
-This is similar to download(), but instead of taking a single source,
-it accepts a `parts` argument consisting of a list of
-`RemoteModelFile` objects. Each part corresponds to a URL/Path pair,
-where the URL is the location of the remote file, and the Path is the
-destination.
-
-`RemoteModelFile` can be imported from `invokeai.backend.model_manager.metadata`, and
-consists of a url/path pair. Note that the path *must* be relative.
-
-The method returns a `MultiFileDownloadJob`.
-
-
-```
-from invokeai.backend.model_manager.metadata import RemoteModelFile
-remote_file_1 = RemoteModelFile(url='http://www.foo.bar/my/pytorch_model.safetensors'',
-                                path='my_model/textencoder/pytorch_model.safetensors'
-			 			  )
-remote_file_2 = RemoteModelFile(url='http://www.bar.baz/vae.ckpt',
-                                path='my_model/vae/diffusers_model.safetensors'
-			 			  )
-job = queue.multifile_download(parts=[remote_file_1, remote_file_2],
-                               dest='/tmp/downloads',
-                               on_progress=TqdmProgress().update)
-queue.wait_for_job(job)
-print(f"The files were downloaded to {job.download_path}")
-```
-
 #### jobs = queue.list_jobs()
 
 Return a list of all active and inactive `DownloadJob`s.

docs/contributing/MODEL_MANAGER.md

@@ -397,25 +397,26 @@ In the event you wish to create a new installer, you may use the
 following initialization pattern:
 
 ```
-from invokeai.app.services.config import get_config
+from invokeai.app.services.config import InvokeAIAppConfig
 from invokeai.app.services.model_records import ModelRecordServiceSQL
 from invokeai.app.services.model_install import ModelInstallService
 from invokeai.app.services.download import DownloadQueueService
-from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
+from invokeai.app.services.shared.sqlite import SqliteDatabase
 from invokeai.backend.util.logging import InvokeAILogger
 
-config = get_config()
+config = InvokeAIAppConfig.get_config()
+config.parse_args()
 
 logger = InvokeAILogger.get_logger(config=config)
-db = SqliteDatabase(config.db_path, logger)
+db = SqliteDatabase(config, logger)
 record_store = ModelRecordServiceSQL(db)
 queue = DownloadQueueService()
 queue.start()
 
-installer = ModelInstallService(app_config=config,
+installer = ModelInstallService(app_config=config, 
                                 record_store=record_store,
-                                download_queue=queue
-                                )
+              download_queue=queue
+           )
 installer.start()
 ```
 
@@ -1366,20 +1367,12 @@ the in-memory loaded model:
 | `model`        | AnyModel               | The instantiated model (details below) |
 | `locker`       | ModelLockerBase        | A context manager that mediates the movement of the model into VRAM |
 
-### get_model_by_key(key, [submodel]) -> LoadedModel
-
-The `get_model_by_key()` method will retrieve the model using its
-unique database key. For example:
-
-loaded_model = loader.get_model_by_key('f13dd932c0c35c22dcb8d6cda4203764', SubModelType('vae'))
-
-`get_model_by_key()` may raise any of the following exceptions:
-
-* `UnknownModelException`   -- key not in database
-* `ModelNotFoundException`  -- key in database but model not found at path
-* `NotImplementedException` -- the loader doesn't know how to load this type of model
-
-### Using the Loaded Model in Inference
+Because the loader can return multiple model types, it is typed to
+return `AnyModel`, a Union `ModelMixin`, `torch.nn.Module`,
+`IAIOnnxRuntimeModel`, `IPAdapter`, `IPAdapterPlus`, and
+`EmbeddingModelRaw`. `ModelMixin` is the base class of all diffusers
+models, `EmbeddingModelRaw` is used for LoRA and TextualInversion
+models. The others are obvious.
 
 `LoadedModel` acts as a context manager. The context loads the model
 into the execution device (e.g. VRAM on CUDA systems), locks the model
@@ -1387,33 +1380,17 @@ in the execution device for the duration of the context, and returns
 the model. Use it like this:
 
 ```
-loaded_model_= loader.get_model_by_key('f13dd932c0c35c22dcb8d6cda4203764', SubModelType('vae'))
-with loaded_model as vae:
+model_info = loader.get_model_by_key('f13dd932c0c35c22dcb8d6cda4203764', SubModelType('vae'))
+with model_info as vae:
  image = vae.decode(latents)[0]
 ```
 
-The object returned by the LoadedModel context manager is an
-`AnyModel`, which is a Union of `ModelMixin`, `torch.nn.Module`,
-`IAIOnnxRuntimeModel`, `IPAdapter`, `IPAdapterPlus`, and
-`EmbeddingModelRaw`. `ModelMixin` is the base class of all diffusers
-models, `EmbeddingModelRaw` is used for LoRA and TextualInversion
-models. The others are obvious.
-
-In addition, you may call `LoadedModel.model_on_device()`, a context
-manager that returns a tuple of the model's state dict in CPU and the
-model itself in VRAM. It is used to optimize the LoRA patching and
-unpatching process:
-
-```
-loaded_model_= loader.get_model_by_key('f13dd932c0c35c22dcb8d6cda4203764', SubModelType('vae'))
-with loaded_model.model_on_device() as (state_dict, vae):
- image = vae.decode(latents)[0]
-```
-
-Since not all models have state dicts, the `state_dict` return value
-can be None.
-
+`get_model_by_key()` may raise any of the following exceptions:
 
+* `UnknownModelException`   -- key not in database
+* `ModelNotFoundException`  -- key in database but model not found at path
+* `NotImplementedException` -- the loader doesn't know how to load this type of model
+  
 ### Emitting model loading events
 
 When the `context` argument is passed to `load_model_*()`, it will
@@ -1601,59 +1578,3 @@ This method takes a model key, looks it up using the
 `ModelRecordServiceBase` object in `mm.store`, and passes the returned
 model configuration to `load_model_by_config()`.  It may raise a
 `NotImplementedException`.
-
-## Invocation Context Model Manager API
-
-Within invocations, the following methods are available from the
-`InvocationContext` object:
-
-### context.download_and_cache_model(source) -> Path
-
-This method accepts a `source` of a remote model, downloads and caches
-it locally, and then returns a Path to the local model. The source can
-be a direct download URL or a HuggingFace repo_id.
-
-In the case of HuggingFace repo_id, the following variants are
-recognized:
-
-* stabilityai/stable-diffusion-v4           -- default model
-* stabilityai/stable-diffusion-v4:fp16      -- fp16 variant
-* stabilityai/stable-diffusion-v4:fp16:vae  -- the fp16 vae subfolder
-* stabilityai/stable-diffusion-v4:onnx:vae  -- the onnx variant vae subfolder
-
-You can also point at an arbitrary individual file within a repo_id
-directory using this syntax:
-
-* stabilityai/stable-diffusion-v4::/checkpoints/sd4.safetensors
-
-### context.load_local_model(model_path, [loader]) -> LoadedModel
-
-This method loads a local model from the indicated path, returning a
-`LoadedModel`. The optional loader is a Callable that accepts a Path
-to the object, and returns a `AnyModel` object. If no loader is
-provided, then the method will use `torch.load()` for a .ckpt or .bin
-checkpoint file, `safetensors.torch.load_file()` for a safetensors
-checkpoint file, or `cls.from_pretrained()` for a directory that looks
-like a diffusers directory.
-
-### context.load_remote_model(source, [loader]) -> LoadedModel
-
-This method accepts a `source` of a remote model, downloads and caches
-it locally, loads it, and returns a `LoadedModel`. The source can be a
-direct download URL or a HuggingFace repo_id.
-
-In the case of HuggingFace repo_id, the following variants are
-recognized:
-
-* stabilityai/stable-diffusion-v4           -- default model
-* stabilityai/stable-diffusion-v4:fp16      -- fp16 variant
-* stabilityai/stable-diffusion-v4:fp16:vae  -- the fp16 vae subfolder
-* stabilityai/stable-diffusion-v4:onnx:vae  -- the onnx variant vae subfolder
-
-You can also point at an arbitrary individual file within a repo_id
-directory using this syntax:
-
-* stabilityai/stable-diffusion-v4::/checkpoints/sd4.safetensors
-
-
-

docs/installation/040_INSTALL_DOCKER.md

@@ -4,37 +4,50 @@ title: Installing with Docker
 
 # :fontawesome-brands-docker: Docker
 
-!!! warning "macOS users"
+!!! warning "macOS and AMD GPU Users"
 
-    Docker can not access the GPU on macOS, so your generation speeds will be slow. [Install InvokeAI](INSTALLATION.md) instead.
+    We highly recommend to Install InvokeAI locally using [these instructions](INSTALLATION.md),
+    because Docker containers can not access the GPU on macOS.
+
+!!! warning "AMD GPU Users"
+
+    Container support for AMD GPUs has been reported to work by the community, but has not received
+    extensive testing. Please make sure to set the `GPU_DRIVER=rocm` environment variable (see below), and
+    use the `build.sh` script to build the image for this to take effect at build time.
 
 !!! tip "Linux and Windows Users"
 
-    Configure Docker to access your machine's GPU.
+    For optimal performance, configure your Docker daemon to access your machine's GPU.
     Docker Desktop on Windows [includes GPU support](https://www.docker.com/blog/wsl-2-gpu-support-for-docker-desktop-on-nvidia-gpus/).
-    Linux users should follow the [NVIDIA](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) or [AMD](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html) documentation.
+    Linux users should install and configure the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
+
+## Why containers?
+
+They provide a flexible, reliable way to build and deploy InvokeAI.
+See [Processes](https://12factor.net/processes) under the Twelve-Factor App
+methodology for details on why running applications in such a stateless fashion is important.
+
+The container is configured for CUDA by default, but can be built to support AMD GPUs
+by setting the `GPU_DRIVER=rocm` environment variable at Docker image build time.
+
+Developers on Apple silicon (M1/M2/M3): You
+[can't access your GPU cores from Docker containers](https://github.com/pytorch/pytorch/issues/81224)
+and performance is reduced compared with running it directly on macOS but for
+development purposes it's fine. Once you're done with development tasks on your
+laptop you can build for the target platform and architecture and deploy to
+another environment with NVIDIA GPUs on-premises or in the cloud.
 
 ## TL;DR
 
-Ensure your Docker setup is able to use your GPU. Then:
-
-    ```bash
-    docker run --runtime=nvidia --gpus=all --publish 9090:9090 ghcr.io/invoke-ai/invokeai
-    ```
-
-Once the container starts up, open http://localhost:9090 in your browser, install some models, and start generating.
-
-## Build-It-Yourself
-
-All the docker materials are located inside the [docker](https://github.com/invoke-ai/InvokeAI/tree/main/docker) directory in the Git repo.
+This assumes properly configured Docker on Linux or Windows/WSL2. Read on for detailed customization options.
 
     ```bash
+    # docker compose commands should be run from the `docker` directory
     cd docker
-    cp .env.sample .env
     docker compose up
     ```
 
-We also ship the `run.sh` convenience script. See the `docker/README.md` file for detailed instructions on how to customize the docker setup to your needs.
+## Installation in a Linux container (desktop)
 
 ### Prerequisites
 
@@ -45,9 +58,18 @@ Preferences, Resources, Advanced. Increase the CPUs and Memory to avoid this
 [Issue](https://github.com/invoke-ai/InvokeAI/issues/342). You may need to
 increase Swap and Disk image size too.
 
+#### Get a Huggingface-Token
+
+Besides the Docker Agent you will need an Account on
+[huggingface.co](https://huggingface.co/join).
+
+After you succesfully registered your account, go to
+[huggingface.co/settings/tokens](https://huggingface.co/settings/tokens), create
+a token and copy it, since you will need in for the next step.
+
 ### Setup
 
-Set up your environment variables. In the `docker` directory, make a copy of `.env.sample` and name it `.env`. Make changes as necessary.
+Set up your environmnent variables. In the `docker` directory, make a copy of `.env.sample` and name it `.env`. Make changes as necessary.
 
 Any environment variables supported by InvokeAI can be set here - please see the [CONFIGURATION](../features/CONFIGURATION.md) for further detail.
 
@@ -81,9 +103,10 @@ Once the container starts up (and configures the InvokeAI root directory if this
 ## Troubleshooting / FAQ
 
 - Q: I am running on Windows under WSL2, and am seeing a "no such file or directory" error.
-- A: Your `docker-entrypoint.sh` might have has Windows (CRLF) line endings, depending how you cloned the repository.
-    To solve this, change the line endings in the `docker-entrypoint.sh` file to `LF`. You can do this in VSCode
+- A: Your `docker-entrypoint.sh` file likely has Windows (CRLF) as opposed to Unix (LF) line endings,
+    and you may have cloned this repository before the issue was fixed. To solve this, please change
+    the line endings in the `docker-entrypoint.sh` file to `LF`. You can do this in VSCode
     (`Ctrl+P` and search for "line endings"), or by using the `dos2unix` utility in WSL.
     Finally, you may delete `docker-entrypoint.sh` followed by  `git pull; git checkout docker/docker-entrypoint.sh`
     to reset the file to its most recent version.
-    For more information on this issue, see [Docker Desktop documentation](https://docs.docker.com/desktop/troubleshoot/topics/#avoid-unexpected-syntax-errors-use-unix-style-line-endings-for-files-in-containers)
+    For more information on this issue, please see the [Docker Desktop documentation](https://docs.docker.com/desktop/troubleshoot/topics/#avoid-unexpected-syntax-errors-use-unix-style-line-endings-for-files-in-containers)

installer/templates/invoke.bat.in

@@ -13,7 +13,7 @@ echo 2. Open the developer console
 echo 3. Command-line help
 echo Q - Quit
 echo.
-echo To update, download and run the installer from https://github.com/invoke-ai/InvokeAI/releases/latest
+echo To update, download and run the installer from https://github.com/invoke-ai/InvokeAI/releases/latest.
 echo.
 set /P choice="Please enter 1-4, Q: [1] "
 if not defined choice set choice=1

invokeai/app/api/dependencies.py

@@ -4,39 +4,37 @@ from logging import Logger
 
 import torch
 
-from invokeai.app.services.board_image_records.board_image_records_sqlite import SqliteBoardImageRecordStorage
-from invokeai.app.services.board_images.board_images_default import BoardImagesService
-from invokeai.app.services.board_records.board_records_sqlite import SqliteBoardRecordStorage
-from invokeai.app.services.boards.boards_default import BoardService
-from invokeai.app.services.bulk_download.bulk_download_default import BulkDownloadService
-from invokeai.app.services.config.config_default import InvokeAIAppConfig
-from invokeai.app.services.download.download_default import DownloadQueueService
-from invokeai.app.services.events.events_fastapievents import FastAPIEventService
-from invokeai.app.services.image_files.image_files_disk import DiskImageFileStorage
-from invokeai.app.services.image_records.image_records_sqlite import SqliteImageRecordStorage
-from invokeai.app.services.images.images_default import ImageService
-from invokeai.app.services.invocation_cache.invocation_cache_memory import MemoryInvocationCache
-from invokeai.app.services.invocation_services import InvocationServices
-from invokeai.app.services.invocation_stats.invocation_stats_default import InvocationStatsService
-from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.model_images.model_images_default import ModelImageFileStorageDisk
-from invokeai.app.services.model_manager.model_manager_default import ModelManagerService
-from invokeai.app.services.model_records.model_records_sql import ModelRecordServiceSQL
-from invokeai.app.services.names.names_default import SimpleNameService
 from invokeai.app.services.object_serializer.object_serializer_disk import ObjectSerializerDisk
 from invokeai.app.services.object_serializer.object_serializer_forward_cache import ObjectSerializerForwardCache
-from invokeai.app.services.session_processor.session_processor_default import (
-    DefaultSessionProcessor,
-    DefaultSessionRunner,
-)
-from invokeai.app.services.session_queue.session_queue_sqlite import SqliteSessionQueue
 from invokeai.app.services.shared.sqlite.sqlite_util import init_db
-from invokeai.app.services.urls.urls_default import LocalUrlService
-from invokeai.app.services.workflow_records.workflow_records_sqlite import SqliteWorkflowRecordsStorage
 from invokeai.backend.stable_diffusion.diffusion.conditioning_data import ConditioningFieldData
 from invokeai.backend.util.logging import InvokeAILogger
 from invokeai.version.invokeai_version import __version__
 
+from ..services.board_image_records.board_image_records_sqlite import SqliteBoardImageRecordStorage
+from ..services.board_images.board_images_default import BoardImagesService
+from ..services.board_records.board_records_sqlite import SqliteBoardRecordStorage
+from ..services.boards.boards_default import BoardService
+from ..services.bulk_download.bulk_download_default import BulkDownloadService
+from ..services.config import InvokeAIAppConfig
+from ..services.download import DownloadQueueService
+from ..services.events.events_fastapievents import FastAPIEventService
+from ..services.image_files.image_files_disk import DiskImageFileStorage
+from ..services.image_records.image_records_sqlite import SqliteImageRecordStorage
+from ..services.images.images_default import ImageService
+from ..services.invocation_cache.invocation_cache_memory import MemoryInvocationCache
+from ..services.invocation_services import InvocationServices
+from ..services.invocation_stats.invocation_stats_default import InvocationStatsService
+from ..services.invoker import Invoker
+from ..services.model_images.model_images_default import ModelImageFileStorageDisk
+from ..services.model_manager.model_manager_default import ModelManagerService
+from ..services.model_records import ModelRecordServiceSQL
+from ..services.names.names_default import SimpleNameService
+from ..services.session_processor.session_processor_default import DefaultSessionProcessor, DefaultSessionRunner
+from ..services.session_queue.session_queue_sqlite import SqliteSessionQueue
+from ..services.urls.urls_default import LocalUrlService
+from ..services.workflow_records.workflow_records_sqlite import SqliteWorkflowRecordsStorage
+
 
 # TODO: is there a better way to achieve this?
 def check_internet() -> bool:
@@ -95,7 +93,7 @@ class ApiDependencies:
         conditioning = ObjectSerializerForwardCache(
             ObjectSerializerDisk[ConditioningFieldData](output_folder / "conditioning", ephemeral=True)
         )
-        download_queue_service = DownloadQueueService(app_config=configuration, event_bus=events)
+        download_queue_service = DownloadQueueService(event_bus=events)
         model_images_service = ModelImageFileStorageDisk(model_images_folder / "model_images")
         model_manager = ModelManagerService.build_model_manager(
             app_config=configuration,

invokeai/app/api/routers/app_info.py

@@ -10,13 +10,14 @@ from fastapi import Body
 from fastapi.routing import APIRouter
 from pydantic import BaseModel, Field
 
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.invocations.upscale import ESRGAN_MODELS
 from invokeai.app.services.invocation_cache.invocation_cache_common import InvocationCacheStatus
 from invokeai.backend.image_util.infill_methods.patchmatch import PatchMatch
 from invokeai.backend.util.logging import logging
 from invokeai.version import __version__
 
+from ..dependencies import ApiDependencies
+
 
 class LogLevel(int, Enum):
     NotSet = logging.NOTSET

invokeai/app/api/routers/board_images.py

@@ -2,7 +2,7 @@ from fastapi import Body, HTTPException
 from fastapi.routing import APIRouter
 from pydantic import BaseModel, Field
 
-from invokeai.app.api.dependencies import ApiDependencies
+from ..dependencies import ApiDependencies
 
 board_images_router = APIRouter(prefix="/v1/board_images", tags=["boards"])
 

invokeai/app/api/routers/boards.py

@@ -4,11 +4,12 @@ from fastapi import Body, HTTPException, Path, Query
 from fastapi.routing import APIRouter
 from pydantic import BaseModel, Field
 
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.services.board_records.board_records_common import BoardChanges
 from invokeai.app.services.boards.boards_common import BoardDTO
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
 
+from ..dependencies import ApiDependencies
+
 boards_router = APIRouter(prefix="/v1/boards", tags=["boards"])
 
 
@@ -31,7 +32,6 @@ class DeleteBoardResult(BaseModel):
 )
 async def create_board(
     board_name: str = Query(description="The name of the board to create"),
-    is_private: bool = Query(default=False, description="Whether the board is private"),
 ) -> BoardDTO:
     """Creates a board"""
     try:
@@ -118,13 +118,15 @@ async def list_boards(
     all: Optional[bool] = Query(default=None, description="Whether to list all boards"),
     offset: Optional[int] = Query(default=None, description="The page offset"),
     limit: Optional[int] = Query(default=None, description="The number of boards per page"),
-    include_archived: bool = Query(default=False, description="Whether or not to include archived boards in list"),
 ) -> Union[OffsetPaginatedResults[BoardDTO], list[BoardDTO]]:
     """Gets a list of boards"""
     if all:
-        return ApiDependencies.invoker.services.boards.get_all(include_archived)
+        return ApiDependencies.invoker.services.boards.get_all()
     elif offset is not None and limit is not None:
-        return ApiDependencies.invoker.services.boards.get_many(offset, limit, include_archived)
+        return ApiDependencies.invoker.services.boards.get_many(
+            offset,
+            limit,
+        )
     else:
         raise HTTPException(
             status_code=400,

invokeai/app/api/routers/download_queue.py

@@ -8,12 +8,13 @@ from fastapi.routing import APIRouter
 from pydantic.networks import AnyHttpUrl
 from starlette.exceptions import HTTPException
 
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.services.download import (
     DownloadJob,
     UnknownJobIDException,
 )
 
+from ..dependencies import ApiDependencies
+
 download_queue_router = APIRouter(prefix="/v1/download_queue", tags=["download_queue"])
 
 

invokeai/app/api/routers/images.py

@@ -8,16 +8,12 @@ from fastapi.routing import APIRouter
 from PIL import Image
 from pydantic import BaseModel, Field, JsonValue
 
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.invocations.fields import MetadataField
-from invokeai.app.services.image_records.image_records_common import (
-    ImageCategory,
-    ImageRecordChanges,
-    ResourceOrigin,
-)
+from invokeai.app.services.image_records.image_records_common import ImageCategory, ImageRecordChanges, ResourceOrigin
 from invokeai.app.services.images.images_common import ImageDTO, ImageUrlsDTO
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_common import SQLiteDirection
+
+from ..dependencies import ApiDependencies
 
 images_router = APIRouter(prefix="/v1/images", tags=["images"])
 
@@ -320,14 +316,16 @@ async def list_image_dtos(
     ),
     offset: int = Query(default=0, description="The page offset"),
     limit: int = Query(default=10, description="The number of images per page"),
-    order_dir: SQLiteDirection = Query(default=SQLiteDirection.Descending, description="The order of sort"),
-    starred_first: bool = Query(default=True, description="Whether to sort by starred images first"),
-    search_term: Optional[str] = Query(default=None, description="The term to search for"),
 ) -> OffsetPaginatedResults[ImageDTO]:
     """Gets a list of image DTOs"""
 
     image_dtos = ApiDependencies.invoker.services.images.get_many(
-        offset, limit, starred_first, order_dir, image_origin, categories, is_intermediate, board_id, search_term
+        offset,
+        limit,
+        image_origin,
+        categories,
+        is_intermediate,
+        board_id,
     )
 
     return image_dtos

invokeai/app/api/routers/model_manager.py

@@ -3,23 +3,23 @@
 
 import io
 import pathlib
+import shutil
 import traceback
 from copy import deepcopy
-from tempfile import TemporaryDirectory
 from typing import Any, Dict, List, Optional, Type
 
 from fastapi import Body, Path, Query, Response, UploadFile
-from fastapi.responses import FileResponse, HTMLResponse
+from fastapi.responses import FileResponse
 from fastapi.routing import APIRouter
 from PIL import Image
 from pydantic import AnyHttpUrl, BaseModel, ConfigDict, Field
 from starlette.exceptions import HTTPException
 from typing_extensions import Annotated
 
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.services.model_images.model_images_common import ModelImageFileNotFoundException
 from invokeai.app.services.model_install.model_install_common import ModelInstallJob
 from invokeai.app.services.model_records import (
+    DuplicateModelException,
     InvalidModelException,
     ModelRecordChanges,
     UnknownModelException,
@@ -30,12 +30,15 @@ from invokeai.backend.model_manager.config import (
     MainCheckpointConfig,
     ModelFormat,
     ModelType,
+    SubModelType,
 )
 from invokeai.backend.model_manager.metadata.fetch.huggingface import HuggingFaceMetadataFetch
 from invokeai.backend.model_manager.metadata.metadata_base import ModelMetadataWithFiles, UnknownMetadataException
 from invokeai.backend.model_manager.search import ModelSearch
 from invokeai.backend.model_manager.starter_models import STARTER_MODELS, StarterModel, StarterModelWithoutDependencies
 
+from ..dependencies import ApiDependencies
+
 model_manager_router = APIRouter(prefix="/v2/models", tags=["model_manager"])
 
 # images are immutable; set a high max-age
@@ -171,6 +174,18 @@ async def get_model_record(
         raise HTTPException(status_code=404, detail=str(e))
 
 
+# @model_manager_router.get("/summary", operation_id="list_model_summary")
+# async def list_model_summary(
+#     page: int = Query(default=0, description="The page to get"),
+#     per_page: int = Query(default=10, description="The number of models per page"),
+#     order_by: ModelRecordOrderBy = Query(default=ModelRecordOrderBy.Default, description="The attribute to order by"),
+# ) -> PaginatedResults[ModelSummary]:
+#     """Gets a page of model summary data."""
+#     record_store = ApiDependencies.invoker.services.model_manager.store
+#     results: PaginatedResults[ModelSummary] = record_store.list_models(page=page, per_page=per_page, order_by=order_by)
+#     return results
+
+
 class FoundModel(BaseModel):
     path: str = Field(description="Path to the model")
     is_installed: bool = Field(description="Whether or not the model is already installed")
@@ -487,133 +502,6 @@ async def install_model(
     return result
 
 
-@model_manager_router.get(
-    "/install/huggingface",
-    operation_id="install_hugging_face_model",
-    responses={
-        201: {"description": "The model is being installed"},
-        400: {"description": "Bad request"},
-        409: {"description": "There is already a model corresponding to this path or repo_id"},
-    },
-    status_code=201,
-    response_class=HTMLResponse,
-)
-async def install_hugging_face_model(
-    source: str = Query(description="HuggingFace repo_id to install"),
-) -> HTMLResponse:
-    """Install a Hugging Face model using a string identifier."""
-
-    def generate_html(title: str, heading: str, repo_id: str, is_error: bool, message: str | None = "") -> str:
-        if message:
-            message = f"<p>{message}</p>"
-        title_class = "error" if is_error else "success"
-        return f"""
-            <html>
-
-            <head>
-                <title>{title}</title>
-                <style>
-                    body {{
-                        text-align: center;
-                        background-color: hsl(220 12% 10% / 1);
-                        font-family: Helvetica, sans-serif;
-                        color: hsl(220 12% 86% / 1);
-                    }}
-
-                    .repo-id {{
-                        color: hsl(220 12% 68% / 1);
-                    }}
-
-                    .error {{
-                        color: hsl(0 42% 68% / 1)
-                    }}
-
-                    .message-box {{
-                        display: inline-block;
-                        border-radius: 5px;
-                        background-color: hsl(220 12% 20% / 1);
-                        padding-inline-end: 30px;
-                        padding: 20px;
-                        padding-inline-start: 30px;
-                        padding-inline-end: 30px;
-                    }}
-
-                    .container {{
-                        display: flex;
-                        width: 100%;
-                        height: 100%;
-                        align-items: center;
-                        justify-content: center;
-                    }}
-
-                    a {{
-                        color: inherit
-                    }}
-
-                    a:visited {{
-                        color: inherit
-                    }}
-
-                    a:active {{
-                        color: inherit
-                    }}
-                </style>
-            </head>
-
-            <body style="background-color: hsl(220 12% 10% / 1);">
-                <div class="container">
-                    <div class="message-box">
-                        <h2 class="{title_class}">{heading}</h2>
-                        {message}
-                        <p class="repo-id">Repo ID: {repo_id}</p>
-                    </div>
-                </div>
-            </body>
-
-            </html>
-        """
-
-    try:
-        metadata = HuggingFaceMetadataFetch().from_id(source)
-        assert isinstance(metadata, ModelMetadataWithFiles)
-    except UnknownMetadataException:
-        title = "Unable to Install Model"
-        heading = "No HuggingFace repository found with that repo ID."
-        message = "Ensure the repo ID is correct and try again."
-        return HTMLResponse(content=generate_html(title, heading, source, True, message), status_code=400)
-
-    logger = ApiDependencies.invoker.services.logger
-
-    try:
-        installer = ApiDependencies.invoker.services.model_manager.install
-        if metadata.is_diffusers:
-            installer.heuristic_import(
-                source=source,
-                inplace=False,
-            )
-        elif metadata.ckpt_urls is not None and len(metadata.ckpt_urls) == 1:
-            installer.heuristic_import(
-                source=str(metadata.ckpt_urls[0]),
-                inplace=False,
-            )
-        else:
-            title = "Unable to Install Model"
-            heading = "This HuggingFace repo has multiple models."
-            message = "Please use the Model Manager to install this model."
-            return HTMLResponse(content=generate_html(title, heading, source, True, message), status_code=200)
-
-        title = "Model Install Started"
-        heading = "Your HuggingFace model is installing now."
-        message = "You can close this tab and check the Model Manager for installation progress."
-        return HTMLResponse(content=generate_html(title, heading, source, False, message), status_code=201)
-    except Exception as e:
-        logger.error(str(e))
-        title = "Unable to Install Model"
-        heading = "There was an problem installing this model."
-        message = 'Please use the Model Manager directly to install this model. If the issue persists, ask for help on <a href="https://discord.gg/ZmtBAhwWhy">discord</a>.'
-        return HTMLResponse(content=generate_html(title, heading, source, True, message), status_code=500)
-
-
 @model_manager_router.get(
     "/install",
     operation_id="list_model_installs",
@@ -731,36 +619,39 @@ async def convert_model(
         logger.error(f"The model with key {key} is not a main checkpoint model.")
         raise HTTPException(400, f"The model with key {key} is not a main checkpoint model.")
 
-    with TemporaryDirectory(dir=ApiDependencies.invoker.services.configuration.models_path) as tmpdir:
-        convert_path = pathlib.Path(tmpdir) / pathlib.Path(model_config.path).stem
-        converted_model = loader.load_model(model_config)
-        # write the converted file to the convert path
-        raw_model = converted_model.model
-        assert hasattr(raw_model, "save_pretrained")
-        raw_model.save_pretrained(convert_path)
-        assert convert_path.exists()
+    # loading the model will convert it into a cached diffusers file
+    try:
+        cc_size = loader.convert_cache.max_size
+        if cc_size == 0:  # temporary set the convert cache to a positive number so that cached model is written
+            loader._convert_cache.max_size = 1.0
+        loader.load_model(model_config, submodel_type=SubModelType.Scheduler)
+    finally:
+        loader._convert_cache.max_size = cc_size
 
-        # temporarily rename the original safetensors file so that there is no naming conflict
-        original_name = model_config.name
-        model_config.name = f"{original_name}.DELETE"
-        changes = ModelRecordChanges(name=model_config.name)
-        store.update_model(key, changes=changes)
+    # Get the path of the converted model from the loader
+    cache_path = loader.convert_cache.cache_path(key)
+    assert cache_path.exists()
 
-        # install the diffusers
-        try:
-            new_key = installer.install_path(
-                convert_path,
-                config={
-                    "name": original_name,
-                    "description": model_config.description,
-                    "hash": model_config.hash,
-                    "source": model_config.source,
-                },
-            )
-        except Exception as e:
-            logger.error(str(e))
-            store.update_model(key, changes=ModelRecordChanges(name=original_name))
-            raise HTTPException(status_code=409, detail=str(e))
+    # temporarily rename the original safetensors file so that there is no naming conflict
+    original_name = model_config.name
+    model_config.name = f"{original_name}.DELETE"
+    changes = ModelRecordChanges(name=model_config.name)
+    store.update_model(key, changes=changes)
+
+    # install the diffusers
+    try:
+        new_key = installer.install_path(
+            cache_path,
+            config={
+                "name": original_name,
+                "description": model_config.description,
+                "hash": model_config.hash,
+                "source": model_config.source,
+            },
+        )
+    except DuplicateModelException as e:
+        logger.error(str(e))
+        raise HTTPException(status_code=409, detail=str(e))
 
     # Update the model image if the model had one
     try:
@@ -773,8 +664,8 @@ async def convert_model(
     # delete the original safetensors file
     installer.delete(key)
 
-    # delete the temporary directory
-    # shutil.rmtree(cache_path)
+    # delete the cached version
+    shutil.rmtree(cache_path)
 
     # return the config record for the new diffusers directory
     new_config = store.get_model(new_key)

invokeai/app/api/routers/session_queue.py

@@ -4,7 +4,6 @@ from fastapi import Body, Path, Query
 from fastapi.routing import APIRouter
 from pydantic import BaseModel
 
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.services.session_processor.session_processor_common import SessionProcessorStatus
 from invokeai.app.services.session_queue.session_queue_common import (
     QUEUE_ITEM_STATUS,
@@ -20,6 +19,8 @@ from invokeai.app.services.session_queue.session_queue_common import (
 )
 from invokeai.app.services.shared.pagination import CursorPaginatedResults
 
+from ..dependencies import ApiDependencies
+
 session_queue_router = APIRouter(prefix="/v1/queue", tags=["queue"])
 
 

invokeai/app/api_app.py

@@ -20,9 +20,14 @@ from torch.backends.mps import is_available as is_mps_available
 # noinspection PyUnresolvedReferences
 import invokeai.backend.util.hotfixes  # noqa: F401 (monkeypatching on import)
 import invokeai.frontend.web as web_dir
-from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.api.no_cache_staticfiles import NoCacheStaticFiles
-from invokeai.app.api.routers import (
+from invokeai.app.services.config.config_default import get_config
+from invokeai.app.util.custom_openapi import get_openapi_func
+from invokeai.backend.util.devices import TorchDevice
+
+from ..backend.util.logging import InvokeAILogger
+from .api.dependencies import ApiDependencies
+from .api.routers import (
     app_info,
     board_images,
     boards,
@@ -33,11 +38,7 @@ from invokeai.app.api.routers import (
     utilities,
     workflows,
 )
-from invokeai.app.api.sockets import SocketIO
-from invokeai.app.services.config.config_default import get_config
-from invokeai.app.util.custom_openapi import get_openapi_func
-from invokeai.backend.util.devices import TorchDevice
-from invokeai.backend.util.logging import InvokeAILogger
+from .api.sockets import SocketIO
 
 app_config = get_config()
 

invokeai/app/invocations/baseinvocation.py

@@ -40,7 +40,7 @@ from invokeai.app.util.misc import uuid_string
 from invokeai.backend.util.logging import InvokeAILogger
 
 if TYPE_CHECKING:
-    from invokeai.app.services.invocation_services import InvocationServices
+    from ..services.invocation_services import InvocationServices
 
 logger = InvokeAILogger.get_logger()
 

invokeai/app/invocations/collections.py

@@ -4,12 +4,13 @@
 import numpy as np
 from pydantic import ValidationInfo, field_validator
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import InputField
 from invokeai.app.invocations.primitives import IntegerCollectionOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.misc import SEED_MAX
 
+from .baseinvocation import BaseInvocation, invocation
+from .fields import InputField
+
 
 @invocation(
     "range", title="Integer Range", tags=["collection", "integer", "range"], category="collections", version="1.0.0"

invokeai/app/invocations/compel.py

@@ -5,7 +5,6 @@ from compel import Compel, ReturnedEmbeddingsType
 from compel.prompt_parser import Blend, Conjunction, CrossAttentionControlSubstitute, FlattenedPrompt, Fragment
 from transformers import CLIPTextModel, CLIPTextModelWithProjection, CLIPTokenizer
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
 from invokeai.app.invocations.fields import (
     ConditioningField,
     FieldDescriptions,
@@ -15,7 +14,6 @@ from invokeai.app.invocations.fields import (
     TensorField,
     UIComponent,
 )
-from invokeai.app.invocations.model import CLIPField
 from invokeai.app.invocations.primitives import ConditioningOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.ti_utils import generate_ti_list
@@ -28,6 +26,9 @@ from invokeai.backend.stable_diffusion.diffusion.conditioning_data import (
 )
 from invokeai.backend.util.devices import TorchDevice
 
+from .baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
+from .model import CLIPField
+
 # unconditioned: Optional[torch.Tensor]
 
 
@@ -80,13 +81,9 @@ class CompelInvocation(BaseInvocation):
 
         with (
             # apply all patches while the model is on the target device
-            text_encoder_info.model_on_device() as (model_state_dict, text_encoder),
+            text_encoder_info as text_encoder,
             tokenizer_info as tokenizer,
-            ModelPatcher.apply_lora_text_encoder(
-                text_encoder,
-                loras=_lora_loader(),
-                model_state_dict=model_state_dict,
-            ),
+            ModelPatcher.apply_lora_text_encoder(text_encoder, _lora_loader()),
             # Apply CLIP Skip after LoRA to prevent LoRA application from failing on skipped layers.
             ModelPatcher.apply_clip_skip(text_encoder, self.clip.skipped_layers),
             ModelPatcher.apply_ti(tokenizer, text_encoder, ti_list) as (
@@ -175,14 +172,9 @@ class SDXLPromptInvocationBase:
 
         with (
             # apply all patches while the model is on the target device
-            text_encoder_info.model_on_device() as (state_dict, text_encoder),
+            text_encoder_info as text_encoder,
             tokenizer_info as tokenizer,
-            ModelPatcher.apply_lora(
-                text_encoder,
-                loras=_lora_loader(),
-                prefix=lora_prefix,
-                model_state_dict=state_dict,
-            ),
+            ModelPatcher.apply_lora(text_encoder, _lora_loader(), lora_prefix),
             # Apply CLIP Skip after LoRA to prevent LoRA application from failing on skipped layers.
             ModelPatcher.apply_clip_skip(text_encoder, clip_field.skipped_layers),
             ModelPatcher.apply_ti(tokenizer, text_encoder, ti_list) as (

invokeai/app/invocations/constants.py

@@ -1,5 +1,6 @@
 from typing import Literal
 
+from invokeai.backend.stable_diffusion.schedulers import SCHEDULER_MAP
 from invokeai.backend.util.devices import TorchDevice
 
 LATENT_SCALE_FACTOR = 8
@@ -10,6 +11,9 @@ factor is hard-coded to a literal '8' rather than using this constant.
 The ratio of image:latent dimensions is LATENT_SCALE_FACTOR:1, or 8:1.
 """
 
+SCHEDULER_NAME_VALUES = Literal[tuple(SCHEDULER_MAP.keys())]
+"""A literal type representing the valid scheduler names."""
+
 IMAGE_MODES = Literal["L", "RGB", "RGBA", "CMYK", "YCbCr", "LAB", "HSV", "I", "F"]
 """A literal type for PIL image modes supported by Invoke"""
 

invokeai/app/invocations/controlnet_image_processors.py

@@ -2,7 +2,6 @@
 # initial implementation by Gregg Helt, 2023
 # heavily leverages controlnet_aux package: https://github.com/patrickvonplaten/controlnet_aux
 from builtins import bool, float
-from pathlib import Path
 from typing import Dict, List, Literal, Union
 
 import cv2
@@ -22,13 +21,6 @@ from controlnet_aux.util import HWC3, ade_palette
 from PIL import Image
 from pydantic import BaseModel, Field, field_validator, model_validator
 
-from invokeai.app.invocations.baseinvocation import (
-    BaseInvocation,
-    BaseInvocationOutput,
-    Classification,
-    invocation,
-    invocation_output,
-)
 from invokeai.app.invocations.fields import (
     FieldDescriptions,
     ImageField,
@@ -44,13 +36,14 @@ from invokeai.app.invocations.util import validate_begin_end_step, validate_weig
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.controlnet_utils import CONTROLNET_MODE_VALUES, CONTROLNET_RESIZE_VALUES, heuristic_resize
 from invokeai.backend.image_util.canny import get_canny_edges
-from invokeai.backend.image_util.depth_anything import DEPTH_ANYTHING_MODELS, DepthAnythingDetector
-from invokeai.backend.image_util.dw_openpose import DWPOSE_MODELS, DWOpenposeDetector
+from invokeai.backend.image_util.depth_anything import DepthAnythingDetector
+from invokeai.backend.image_util.dw_openpose import DWOpenposeDetector
 from invokeai.backend.image_util.hed import HEDProcessor
 from invokeai.backend.image_util.lineart import LineartProcessor
 from invokeai.backend.image_util.lineart_anime import LineartAnimeProcessor
 from invokeai.backend.image_util.util import np_to_pil, pil_to_np
-from invokeai.backend.util.devices import TorchDevice
+
+from .baseinvocation import BaseInvocation, BaseInvocationOutput, Classification, invocation, invocation_output
 
 
 class ControlField(BaseModel):
@@ -146,7 +139,6 @@ class ImageProcessorInvocation(BaseInvocation, WithMetadata, WithBoard):
         return context.images.get_pil(self.image.image_name, "RGB")
 
     def invoke(self, context: InvocationContext) -> ImageOutput:
-        self._context = context
         raw_image = self.load_image(context)
         # image type should be PIL.PngImagePlugin.PngImageFile ?
         processed_image = self.run_processor(raw_image)
@@ -292,8 +284,7 @@ class MidasDepthImageProcessorInvocation(ImageProcessorInvocation):
     # depth_and_normal not supported in controlnet_aux v0.0.3
     # depth_and_normal: bool = InputField(default=False, description="whether to use depth and normal mode")
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
-        # TODO: replace from_pretrained() calls with context.models.download_and_cache() (or similar)
+    def run_processor(self, image):
         midas_processor = MidasDetector.from_pretrained("lllyasviel/Annotators")
         processed_image = midas_processor(
             image,
@@ -320,7 +311,7 @@ class NormalbaeImageProcessorInvocation(ImageProcessorInvocation):
     detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
     image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
+    def run_processor(self, image):
         normalbae_processor = NormalBaeDetector.from_pretrained("lllyasviel/Annotators")
         processed_image = normalbae_processor(
             image, detect_resolution=self.detect_resolution, image_resolution=self.image_resolution
@@ -339,7 +330,7 @@ class MlsdImageProcessorInvocation(ImageProcessorInvocation):
     thr_v: float = InputField(default=0.1, ge=0, description="MLSD parameter `thr_v`")
     thr_d: float = InputField(default=0.1, ge=0, description="MLSD parameter `thr_d`")
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
+    def run_processor(self, image):
         mlsd_processor = MLSDdetector.from_pretrained("lllyasviel/Annotators")
         processed_image = mlsd_processor(
             image,
@@ -362,7 +353,7 @@ class PidiImageProcessorInvocation(ImageProcessorInvocation):
     safe: bool = InputField(default=False, description=FieldDescriptions.safe_mode)
     scribble: bool = InputField(default=False, description=FieldDescriptions.scribble_mode)
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
+    def run_processor(self, image):
         pidi_processor = PidiNetDetector.from_pretrained("lllyasviel/Annotators")
         processed_image = pidi_processor(
             image,
@@ -390,7 +381,7 @@ class ContentShuffleImageProcessorInvocation(ImageProcessorInvocation):
     w: int = InputField(default=512, ge=0, description="Content shuffle `w` parameter")
     f: int = InputField(default=256, ge=0, description="Content shuffle `f` parameter")
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
+    def run_processor(self, image):
         content_shuffle_processor = ContentShuffleDetector()
         processed_image = content_shuffle_processor(
             image,
@@ -414,7 +405,7 @@ class ContentShuffleImageProcessorInvocation(ImageProcessorInvocation):
 class ZoeDepthImageProcessorInvocation(ImageProcessorInvocation):
     """Applies Zoe depth processing to image"""
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
+    def run_processor(self, image):
         zoe_depth_processor = ZoeDetector.from_pretrained("lllyasviel/Annotators")
         processed_image = zoe_depth_processor(image)
         return processed_image
@@ -435,7 +426,7 @@ class MediapipeFaceProcessorInvocation(ImageProcessorInvocation):
     detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
     image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
+    def run_processor(self, image):
         mediapipe_face_processor = MediapipeFaceDetector()
         processed_image = mediapipe_face_processor(
             image,
@@ -463,7 +454,7 @@ class LeresImageProcessorInvocation(ImageProcessorInvocation):
     detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
     image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
+    def run_processor(self, image):
         leres_processor = LeresDetector.from_pretrained("lllyasviel/Annotators")
         processed_image = leres_processor(
             image,
@@ -505,8 +496,8 @@ class TileResamplerProcessorInvocation(ImageProcessorInvocation):
         np_img = cv2.resize(np_img, (W, H), interpolation=cv2.INTER_AREA)
         return np_img
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
-        np_img = np.array(image, dtype=np.uint8)
+    def run_processor(self, img):
+        np_img = np.array(img, dtype=np.uint8)
         processed_np_image = self.tile_resample(
             np_img,
             # res=self.tile_size,
@@ -529,7 +520,7 @@ class SegmentAnythingProcessorInvocation(ImageProcessorInvocation):
     detect_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.detect_res)
     image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
+    def run_processor(self, image):
         # segment_anything_processor = SamDetector.from_pretrained("ybelkada/segment-anything", subfolder="checkpoints")
         segment_anything_processor = SamDetectorReproducibleColors.from_pretrained(
             "ybelkada/segment-anything", subfolder="checkpoints"
@@ -575,7 +566,7 @@ class ColorMapImageProcessorInvocation(ImageProcessorInvocation):
 
     color_map_tile_size: int = InputField(default=64, ge=1, description=FieldDescriptions.tile_size)
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
+    def run_processor(self, image: Image.Image):
         np_image = np.array(image, dtype=np.uint8)
         height, width = np_image.shape[:2]
 
@@ -610,18 +601,12 @@ class DepthAnythingImageProcessorInvocation(ImageProcessorInvocation):
     )
     resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
-        def loader(model_path: Path):
-            return DepthAnythingDetector.load_model(
-                model_path, model_size=self.model_size, device=TorchDevice.choose_torch_device()
-            )
+    def run_processor(self, image: Image.Image):
+        depth_anything_detector = DepthAnythingDetector()
+        depth_anything_detector.load_model(model_size=self.model_size)
 
-        with self._context.models.load_remote_model(
-            source=DEPTH_ANYTHING_MODELS[self.model_size], loader=loader
-        ) as model:
-            depth_anything_detector = DepthAnythingDetector(model, TorchDevice.choose_torch_device())
-            processed_image = depth_anything_detector(image=image, resolution=self.resolution)
-            return processed_image
+        processed_image = depth_anything_detector(image=image, resolution=self.resolution)
+        return processed_image
 
 
 @invocation(
@@ -639,11 +624,8 @@ class DWOpenposeImageProcessorInvocation(ImageProcessorInvocation):
     draw_hands: bool = InputField(default=False)
     image_resolution: int = InputField(default=512, ge=1, description=FieldDescriptions.image_res)
 
-    def run_processor(self, image: Image.Image) -> Image.Image:
-        onnx_det = self._context.models.download_and_cache_model(DWPOSE_MODELS["yolox_l.onnx"])
-        onnx_pose = self._context.models.download_and_cache_model(DWPOSE_MODELS["dw-ll_ucoco_384.onnx"])
-
-        dw_openpose = DWOpenposeDetector(onnx_det=onnx_det, onnx_pose=onnx_pose)
+    def run_processor(self, image: Image.Image):
+        dw_openpose = DWOpenposeDetector()
         processed_image = dw_openpose(
             image,
             draw_face=self.draw_face,

invokeai/app/invocations/cv.py

@@ -5,11 +5,13 @@ import cv2 as cv
 import numpy
 from PIL import Image, ImageOps
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import ImageField, InputField, WithBoard, WithMetadata
+from invokeai.app.invocations.fields import ImageField
 from invokeai.app.invocations.primitives import ImageOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 
+from .baseinvocation import BaseInvocation, invocation
+from .fields import InputField, WithBoard, WithMetadata
+
 
 @invocation("cv_inpaint", title="OpenCV Inpaint", tags=["opencv", "inpaint"], category="inpaint", version="1.3.1")
 class CvInpaintInvocation(BaseInvocation, WithMetadata, WithBoard):

invokeai/app/invocations/fields.py

@@ -160,8 +160,6 @@ class FieldDescriptions:
     fp32 = "Whether or not to use full float32 precision"
     precision = "Precision to use"
     tiled = "Processing using overlapping tiles (reduce memory consumption)"
-    vae_tile_size = "The tile size for VAE tiling in pixels (image space). If set to 0, the default tile size for the "
-    "model will be used. Larger tile sizes generally produce better results at the cost of higher memory usage."
     detect_res = "Pixel resolution for detection"
     image_res = "Pixel resolution for output image"
     safe_mode = "Whether or not to use safe mode"

invokeai/app/invocations/image.py

@@ -6,7 +6,6 @@ import cv2
 import numpy
 from PIL import Image, ImageChops, ImageFilter, ImageOps
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, Classification, invocation
 from invokeai.app.invocations.constants import IMAGE_MODES
 from invokeai.app.invocations.fields import (
     ColorField,
@@ -22,6 +21,8 @@ from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.backend.image_util.invisible_watermark import InvisibleWatermark
 from invokeai.backend.image_util.safety_checker import SafetyChecker
 
+from .baseinvocation import BaseInvocation, Classification, invocation
+
 
 @invocation("show_image", title="Show Image", tags=["image"], category="image", version="1.0.1")
 class ShowImageInvocation(BaseInvocation):

invokeai/app/invocations/image_to_latents.py

@@ -1,4 +1,3 @@
-from contextlib import nullcontext
 from functools import singledispatchmethod
 
 import einops
@@ -13,7 +12,7 @@ from diffusers.models.autoencoders.autoencoder_kl import AutoencoderKL
 from diffusers.models.autoencoders.autoencoder_tiny import AutoencoderTiny
 
 from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.constants import DEFAULT_PRECISION, LATENT_SCALE_FACTOR
+from invokeai.app.invocations.constants import DEFAULT_PRECISION
 from invokeai.app.invocations.fields import (
     FieldDescriptions,
     ImageField,
@@ -25,7 +24,6 @@ from invokeai.app.invocations.primitives import LatentsOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.backend.model_manager import LoadedModel
 from invokeai.backend.stable_diffusion.diffusers_pipeline import image_resized_to_grid_as_tensor
-from invokeai.backend.stable_diffusion.vae_tiling import patch_vae_tiling_params
 
 
 @invocation(
@@ -33,7 +31,7 @@ from invokeai.backend.stable_diffusion.vae_tiling import patch_vae_tiling_params
     title="Image to Latents",
     tags=["latents", "image", "vae", "i2l"],
     category="latents",
-    version="1.1.0",
+    version="1.0.2",
 )
 class ImageToLatentsInvocation(BaseInvocation):
     """Encodes an image into latents."""
@@ -46,17 +44,12 @@ class ImageToLatentsInvocation(BaseInvocation):
         input=Input.Connection,
     )
     tiled: bool = InputField(default=False, description=FieldDescriptions.tiled)
-    # NOTE: tile_size = 0 is a special value. We use this rather than `int | None`, because the workflow UI does not
-    # offer a way to directly set None values.
-    tile_size: int = InputField(default=0, multiple_of=8, description=FieldDescriptions.vae_tile_size)
     fp32: bool = InputField(default=DEFAULT_PRECISION == torch.float32, description=FieldDescriptions.fp32)
 
     @staticmethod
-    def vae_encode(
-        vae_info: LoadedModel, upcast: bool, tiled: bool, image_tensor: torch.Tensor, tile_size: int = 0
-    ) -> torch.Tensor:
+    def vae_encode(vae_info: LoadedModel, upcast: bool, tiled: bool, image_tensor: torch.Tensor) -> torch.Tensor:
         with vae_info as vae:
-            assert isinstance(vae, (AutoencoderKL, AutoencoderTiny))
+            assert isinstance(vae, torch.nn.Module)
             orig_dtype = vae.dtype
             if upcast:
                 vae.to(dtype=torch.float32)
@@ -88,18 +81,9 @@ class ImageToLatentsInvocation(BaseInvocation):
             else:
                 vae.disable_tiling()
 
-            tiling_context = nullcontext()
-            if tile_size > 0:
-                tiling_context = patch_vae_tiling_params(
-                    vae,
-                    tile_sample_min_size=tile_size,
-                    tile_latent_min_size=tile_size // LATENT_SCALE_FACTOR,
-                    tile_overlap_factor=0.25,
-                )
-
             # non_noised_latents_from_image
             image_tensor = image_tensor.to(device=vae.device, dtype=vae.dtype)
-            with torch.inference_mode(), tiling_context:
+            with torch.inference_mode():
                 latents = ImageToLatentsInvocation._encode_to_tensor(vae, image_tensor)
 
             latents = vae.config.scaling_factor * latents
@@ -117,9 +101,7 @@ class ImageToLatentsInvocation(BaseInvocation):
         if image_tensor.dim() == 3:
             image_tensor = einops.rearrange(image_tensor, "c h w -> 1 c h w")
 
-        latents = self.vae_encode(
-            vae_info=vae_info, upcast=self.fp32, tiled=self.tiled, image_tensor=image_tensor, tile_size=self.tile_size
-        )
+        latents = self.vae_encode(vae_info, self.fp32, self.tiled, image_tensor)
 
         latents = latents.to("cpu")
         name = context.tensors.save(tensor=latents)

invokeai/app/invocations/infill.py

@@ -3,9 +3,7 @@ from typing import Literal, get_args
 
 from PIL import Image
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import ColorField, ImageField, InputField, WithBoard, WithMetadata
-from invokeai.app.invocations.image import PIL_RESAMPLING_MAP, PIL_RESAMPLING_MODES
+from invokeai.app.invocations.fields import ColorField, ImageField
 from invokeai.app.invocations.primitives import ImageOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.misc import SEED_MAX
@@ -16,6 +14,10 @@ from invokeai.backend.image_util.infill_methods.patchmatch import PatchMatch, in
 from invokeai.backend.image_util.infill_methods.tile import infill_tile
 from invokeai.backend.util.logging import InvokeAILogger
 
+from .baseinvocation import BaseInvocation, invocation
+from .fields import InputField, WithBoard, WithMetadata
+from .image import PIL_RESAMPLING_MAP, PIL_RESAMPLING_MODES
+
 logger = InvokeAILogger.get_logger()
 
 
@@ -40,16 +42,15 @@ class InfillImageProcessorInvocation(BaseInvocation, WithMetadata, WithBoard):
         """Infill the image with the specified method"""
         pass
 
-    def load_image(self) -> tuple[Image.Image, bool]:
+    def load_image(self, context: InvocationContext) -> tuple[Image.Image, bool]:
         """Process the image to have an alpha channel before being infilled"""
-        image = self._context.images.get_pil(self.image.image_name)
+        image = context.images.get_pil(self.image.image_name)
         has_alpha = True if image.mode == "RGBA" else False
         return image, has_alpha
 
     def invoke(self, context: InvocationContext) -> ImageOutput:
-        self._context = context
         # Retrieve and process image to be infilled
-        input_image, has_alpha = self.load_image()
+        input_image, has_alpha = self.load_image(context)
 
         # If the input image has no alpha channel, return it
         if has_alpha is False:
@@ -132,12 +133,8 @@ class LaMaInfillInvocation(InfillImageProcessorInvocation):
     """Infills transparent areas of an image using the LaMa model"""
 
     def infill(self, image: Image.Image):
-        with self._context.models.load_remote_model(
-            source="https://github.com/Sanster/models/releases/download/add_big_lama/big-lama.pt",
-            loader=LaMA.load_jit_model,
-        ) as model:
-            lama = LaMA(model)
-            return lama(image)
+        lama = LaMA()
+        return lama(image)
 
 
 @invocation("infill_cv2", title="CV2 Infill", tags=["image", "inpaint"], category="inpaint", version="1.2.2")

invokeai/app/invocations/latent.py

@@ -17,7 +17,7 @@ from torchvision.transforms.functional import resize as tv_resize
 from transformers import CLIPVisionModelWithProjection
 
 from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR
+from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR, SCHEDULER_NAME_VALUES
 from invokeai.app.invocations.controlnet_image_processors import ControlField
 from invokeai.app.invocations.fields import (
     ConditioningField,
@@ -54,9 +54,7 @@ from invokeai.backend.stable_diffusion.diffusion.conditioning_data import (
     TextConditioningRegions,
 )
 from invokeai.backend.stable_diffusion.schedulers import SCHEDULER_MAP
-from invokeai.backend.stable_diffusion.schedulers.schedulers import SCHEDULER_NAME_VALUES
 from invokeai.backend.util.devices import TorchDevice
-from invokeai.backend.util.hotfixes import ControlNetModel
 from invokeai.backend.util.mask import to_standard_float_mask
 from invokeai.backend.util.silence_warnings import SilenceWarnings
 
@@ -89,9 +87,6 @@ def get_scheduler(
 
     scheduler = scheduler_class.from_config(scheduler_config)
 
-    # hack copied over from generate.py
-    if not hasattr(scheduler, "uses_inpainting_model"):
-        scheduler.uses_inpainting_model = lambda: False
     assert isinstance(scheduler, Scheduler)
     return scheduler
 
@@ -388,38 +383,38 @@ class DenoiseLatentsInvocation(BaseInvocation):
             requires_safety_checker=False,
         )
 
-    @staticmethod
     def prep_control_data(
+        self,
         context: InvocationContext,
-        control_input: ControlField | list[ControlField] | None,
+        control_input: Optional[Union[ControlField, List[ControlField]]],
         latents_shape: List[int],
         exit_stack: ExitStack,
         do_classifier_free_guidance: bool = True,
-    ) -> list[ControlNetData] | None:
-        # Normalize control_input to a list.
-        control_list: list[ControlField]
-        if isinstance(control_input, ControlField):
-            control_list = [control_input]
-        elif isinstance(control_input, list):
-            control_list = control_input
-        elif control_input is None:
-            control_list = []
-        else:
-            raise ValueError(f"Unexpected control_input type: {type(control_input)}")
-
-        if len(control_list) == 0:
-            return None
-
+    ) -> Optional[List[ControlNetData]]:
         # Assuming fixed dimensional scaling of LATENT_SCALE_FACTOR.
-        _, _, latent_height, latent_width = latents_shape
-        control_height_resize = latent_height * LATENT_SCALE_FACTOR
-        control_width_resize = latent_width * LATENT_SCALE_FACTOR
+        control_height_resize = latents_shape[2] * LATENT_SCALE_FACTOR
+        control_width_resize = latents_shape[3] * LATENT_SCALE_FACTOR
+        if control_input is None:
+            control_list = None
+        elif isinstance(control_input, list) and len(control_input) == 0:
+            control_list = None
+        elif isinstance(control_input, ControlField):
+            control_list = [control_input]
+        elif isinstance(control_input, list) and len(control_input) > 0 and isinstance(control_input[0], ControlField):
+            control_list = control_input
+        else:
+            control_list = None
+        if control_list is None:
+            return None
+        # After above handling, any control that is not None should now be of type list[ControlField].
 
-        controlnet_data: list[ControlNetData] = []
+        # FIXME: add checks to skip entry if model or image is None
+        #        and if weight is None, populate with default 1.0?
+        controlnet_data = []
         for control_info in control_list:
             control_model = exit_stack.enter_context(context.models.load(control_info.control_model))
-            assert isinstance(control_model, ControlNetModel)
 
+            # control_models.append(control_model)
             control_image_field = control_info.image
             input_image = context.images.get_pil(control_image_field.image_name)
             # self.image.image_type, self.image.image_name
@@ -440,7 +435,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
                 resize_mode=control_info.resize_mode,
             )
             control_item = ControlNetData(
-                model=control_model,
+                model=control_model,  # model object
                 image_tensor=control_image,
                 weight=control_info.control_weight,
                 begin_step_percent=control_info.begin_step_percent,
@@ -602,7 +597,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
         denoising_start: float,
         denoising_end: float,
         seed: int,
-    ) -> Tuple[torch.Tensor, torch.Tensor, Dict[str, Any]]:
+    ) -> Tuple[int, List[int], int, Dict[str, Any]]:
         assert isinstance(scheduler, ConfigMixin)
         if scheduler.config.get("cpu_only", False):
             scheduler.set_timesteps(steps, device="cpu")
@@ -628,6 +623,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
 
         init_timestep = timesteps[t_start_idx : t_start_idx + 1]
         timesteps = timesteps[t_start_idx : t_start_idx + t_end_idx]
+        num_inference_steps = len(timesteps) // scheduler.order
 
         scheduler_step_kwargs: Dict[str, Any] = {}
         scheduler_step_signature = inspect.signature(scheduler.step)
@@ -649,7 +645,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
         if isinstance(scheduler, TCDScheduler):
             scheduler_step_kwargs.update({"eta": 1.0})
 
-        return timesteps, init_timestep, scheduler_step_kwargs
+        return num_inference_steps, timesteps, init_timestep, scheduler_step_kwargs
 
     def prep_inpaint_mask(
         self, context: InvocationContext, latents: torch.Tensor
@@ -670,18 +666,6 @@ class DenoiseLatentsInvocation(BaseInvocation):
     def prepare_noise_and_latents(
         context: InvocationContext, noise_field: LatentsField | None, latents_field: LatentsField | None
     ) -> Tuple[int, torch.Tensor | None, torch.Tensor]:
-        """Depending on the workflow, we expect different combinations of noise and latents to be provided. This
-        function handles preparing these values accordingly.
-
-        Expected workflows:
-        - Text-to-Image Denoising: `noise` is provided, `latents` is not. `latents` is initialized to zeros.
-        - Image-to-Image Denoising: `noise` and `latents` are both provided.
-        - Text-to-Image SDXL Refiner Denoising: `latents` is provided, `noise` is not.
-        - Image-to-Image SDXL Refiner Denoising: `latents` is provided, `noise` is not.
-
-        NOTE(ryand): I wrote this docstring, but I am not the original author of this code. There may be other workflows
-        I haven't considered.
-        """
         noise = None
         if noise_field is not None:
             noise = context.tensors.load(noise_field.latents_name)
@@ -694,7 +678,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
             raise ValueError("'latents' or 'noise' must be provided!")
 
         if noise is not None and noise.shape[1:] != latents.shape[1:]:
-            raise ValueError(f"Incompatible 'noise' and 'latents' shapes: {latents.shape=} {noise.shape=}")
+            raise ValueError(f"Incompatable 'noise' and 'latents' shapes: {latents.shape=} {noise.shape=}")
 
         # The seed comes from (in order of priority): the noise field, the latents field, or 0.
         seed = 0
@@ -737,7 +721,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
         # The image prompts are then passed to prep_ip_adapter_data().
         image_prompts = self.prep_ip_adapter_image_prompts(context=context, ip_adapters=ip_adapters)
 
-        # get the unet's config so that we can pass the base to sd_step_callback()
+        # get the unet's config so that we can pass the base to dispatch_progress()
         unet_config = context.models.get_config(self.unet.unet.key)
 
         def step_callback(state: PipelineIntermediateState) -> None:
@@ -755,15 +739,11 @@ class DenoiseLatentsInvocation(BaseInvocation):
         assert isinstance(unet_info.model, UNet2DConditionModel)
         with (
             ExitStack() as exit_stack,
-            unet_info.model_on_device() as (model_state_dict, unet),
+            unet_info as unet,
             ModelPatcher.apply_freeu(unet, self.unet.freeu_config),
             set_seamless(unet, self.unet.seamless_axes),  # FIXME
             # Apply the LoRA after unet has been moved to its target device for faster patching.
-            ModelPatcher.apply_lora_unet(
-                unet,
-                loras=_lora_loader(),
-                model_state_dict=model_state_dict,
-            ),
+            ModelPatcher.apply_lora_unet(unet, _lora_loader()),
         ):
             assert isinstance(unet, UNet2DConditionModel)
             latents = latents.to(device=unet.device, dtype=unet.dtype)
@@ -815,7 +795,7 @@ class DenoiseLatentsInvocation(BaseInvocation):
                 dtype=unet.dtype,
             )
 
-            timesteps, init_timestep, scheduler_step_kwargs = self.init_scheduler(
+            num_inference_steps, timesteps, init_timestep, scheduler_step_kwargs = self.init_scheduler(
                 scheduler,
                 device=unet.device,
                 steps=self.steps,
@@ -832,7 +812,8 @@ class DenoiseLatentsInvocation(BaseInvocation):
                 seed=seed,
                 mask=mask,
                 masked_latents=masked_latents,
-                is_gradient_mask=gradient_mask,
+                gradient_mask=gradient_mask,
+                num_inference_steps=num_inference_steps,
                 scheduler_step_kwargs=scheduler_step_kwargs,
                 conditioning_data=conditioning_data,
                 control_data=controlnet_data,

invokeai/app/invocations/latents_to_image.py

@@ -1,5 +1,3 @@
-from contextlib import nullcontext
-
 import torch
 from diffusers.image_processor import VaeImageProcessor
 from diffusers.models.attention_processor import (
@@ -10,9 +8,10 @@ from diffusers.models.attention_processor import (
 )
 from diffusers.models.autoencoders.autoencoder_kl import AutoencoderKL
 from diffusers.models.autoencoders.autoencoder_tiny import AutoencoderTiny
+from PIL import Image
 
 from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.constants import DEFAULT_PRECISION, LATENT_SCALE_FACTOR
+from invokeai.app.invocations.constants import DEFAULT_PRECISION
 from invokeai.app.invocations.fields import (
     FieldDescriptions,
     Input,
@@ -24,8 +23,8 @@ from invokeai.app.invocations.fields import (
 from invokeai.app.invocations.model import VAEField
 from invokeai.app.invocations.primitives import ImageOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.backend.model_manager.load.load_base import LoadedModel
 from invokeai.backend.stable_diffusion import set_seamless
-from invokeai.backend.stable_diffusion.vae_tiling import patch_vae_tiling_params
 from invokeai.backend.util.devices import TorchDevice
 
 
@@ -34,7 +33,7 @@ from invokeai.backend.util.devices import TorchDevice
     title="Latents to Image",
     tags=["latents", "image", "vae", "l2i"],
     category="latents",
-    version="1.3.0",
+    version="1.2.2",
 )
 class LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
     """Generates an image from latents."""
@@ -48,21 +47,22 @@ class LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
         input=Input.Connection,
     )
     tiled: bool = InputField(default=False, description=FieldDescriptions.tiled)
-    # NOTE: tile_size = 0 is a special value. We use this rather than `int | None`, because the workflow UI does not
-    # offer a way to directly set None values.
-    tile_size: int = InputField(default=0, multiple_of=8, description=FieldDescriptions.vae_tile_size)
     fp32: bool = InputField(default=DEFAULT_PRECISION == torch.float32, description=FieldDescriptions.fp32)
 
-    @torch.no_grad()
-    def invoke(self, context: InvocationContext) -> ImageOutput:
-        latents = context.tensors.load(self.latents.latents_name)
-
-        vae_info = context.models.load(self.vae.vae)
+    @staticmethod
+    def vae_decode(
+        context: InvocationContext,
+        vae_info: LoadedModel,
+        seamless_axes: list[str],
+        latents: torch.Tensor,
+        use_fp32: bool,
+        use_tiling: bool,
+    ) -> Image.Image:
         assert isinstance(vae_info.model, (AutoencoderKL, AutoencoderTiny))
-        with set_seamless(vae_info.model, self.vae.seamless_axes), vae_info as vae:
+        with set_seamless(vae_info.model, seamless_axes), vae_info as vae:
             assert isinstance(vae, (AutoencoderKL, AutoencoderTiny))
             latents = latents.to(vae.device)
-            if self.fp32:
+            if use_fp32:
                 vae.to(dtype=torch.float32)
 
                 use_torch_2_0_or_xformers = hasattr(vae.decoder, "mid_block") and isinstance(
@@ -87,24 +87,15 @@ class LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
                 vae.to(dtype=torch.float16)
                 latents = latents.half()
 
-            if self.tiled or context.config.get().force_tiled_decode:
+            if use_tiling or context.config.get().force_tiled_decode:
                 vae.enable_tiling()
             else:
                 vae.disable_tiling()
 
-            tiling_context = nullcontext()
-            if self.tile_size > 0:
-                tiling_context = patch_vae_tiling_params(
-                    vae,
-                    tile_sample_min_size=self.tile_size,
-                    tile_latent_min_size=self.tile_size // LATENT_SCALE_FACTOR,
-                    tile_overlap_factor=0.25,
-                )
-
             # clear memory as vae decode can request a lot
             TorchDevice.empty_cache()
 
-            with torch.inference_mode(), tiling_context:
+            with torch.inference_mode():
                 # copied from diffusers pipeline
                 latents = latents / vae.config.scaling_factor
                 image = vae.decode(latents, return_dict=False)[0]
@@ -116,6 +107,21 @@ class LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
 
         TorchDevice.empty_cache()
 
+        return image
+
+    @torch.no_grad()
+    def invoke(self, context: InvocationContext) -> ImageOutput:
+        latents = context.tensors.load(self.latents.latents_name)
+        vae_info = context.models.load(self.vae.vae)
+
+        image = self.vae_decode(
+            context=context,
+            vae_info=vae_info,
+            seamless_axes=self.vae.seamless_axes,
+            latents=latents,
+            use_fp32=self.fp32,
+            use_tiling=self.tiled,
+        )
         image_dto = context.images.save(image=image)
 
         return ImageOutput.build(image_dto)

invokeai/app/invocations/math.py

@@ -5,11 +5,12 @@ from typing import Literal
 import numpy as np
 from pydantic import ValidationInfo, field_validator
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
 from invokeai.app.invocations.fields import FieldDescriptions, InputField
 from invokeai.app.invocations.primitives import FloatOutput, IntegerOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 
+from .baseinvocation import BaseInvocation, invocation
+
 
 @invocation("add", title="Add Integers", tags=["math", "add"], category="math", version="1.0.1")
 class AddInvocation(BaseInvocation):

invokeai/app/invocations/metadata.py

@@ -14,7 +14,8 @@ from invokeai.app.invocations.fields import (
 from invokeai.app.invocations.model import ModelIdentifierField
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.controlnet_utils import CONTROLNET_MODE_VALUES, CONTROLNET_RESIZE_VALUES
-from invokeai.version.invokeai_version import __version__
+
+from ...version import __version__
 
 
 class MetadataItemField(BaseModel):

invokeai/app/invocations/model.py

@@ -3,17 +3,18 @@ from typing import List, Optional
 
 from pydantic import BaseModel, Field
 
-from invokeai.app.invocations.baseinvocation import (
+from invokeai.app.invocations.fields import FieldDescriptions, Input, InputField, OutputField, UIType
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.app.shared.models import FreeUConfig
+from invokeai.backend.model_manager.config import AnyModelConfig, BaseModelType, ModelType, SubModelType
+
+from .baseinvocation import (
     BaseInvocation,
     BaseInvocationOutput,
     Classification,
     invocation,
     invocation_output,
 )
-from invokeai.app.invocations.fields import FieldDescriptions, Input, InputField, OutputField, UIType
-from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.app.shared.models import FreeUConfig
-from invokeai.backend.model_manager.config import AnyModelConfig, BaseModelType, ModelType, SubModelType
 
 
 class ModelIdentifierField(BaseModel):

invokeai/app/invocations/noise.py

@@ -4,12 +4,18 @@
 import torch
 from pydantic import field_validator
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
 from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR
 from invokeai.app.invocations.fields import FieldDescriptions, InputField, LatentsField, OutputField
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.app.util.misc import SEED_MAX
-from invokeai.backend.util.devices import TorchDevice
+
+from ...backend.util.devices import TorchDevice
+from .baseinvocation import (
+    BaseInvocation,
+    BaseInvocationOutput,
+    invocation,
+    invocation_output,
+)
 
 """
 Utilities

invokeai/app/invocations/param_easing.py

@@ -39,11 +39,12 @@ from easing_functions import (
 )
 from matplotlib.ticker import MaxNLocator
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import InputField
 from invokeai.app.invocations.primitives import FloatCollectionOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 
+from .baseinvocation import BaseInvocation, invocation
+from .fields import InputField
+
 
 @invocation(
     "float_range",

invokeai/app/invocations/primitives.py

@@ -4,7 +4,6 @@ from typing import Optional
 
 import torch
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
 from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR
 from invokeai.app.invocations.fields import (
     ColorField,
@@ -22,6 +21,13 @@ from invokeai.app.invocations.fields import (
 from invokeai.app.services.images.images_common import ImageDTO
 from invokeai.app.services.shared.invocation_context import InvocationContext
 
+from .baseinvocation import (
+    BaseInvocation,
+    BaseInvocationOutput,
+    invocation,
+    invocation_output,
+)
+
 """
 Primitives: Boolean, Integer, Float, String, Image, Latents, Conditioning, Color
 - primitive nodes

invokeai/app/invocations/prompt.py

@@ -5,11 +5,12 @@ import numpy as np
 from dynamicprompts.generators import CombinatorialPromptGenerator, RandomPromptGenerator
 from pydantic import field_validator
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import InputField, UIComponent
 from invokeai.app.invocations.primitives import StringCollectionOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 
+from .baseinvocation import BaseInvocation, invocation
+from .fields import InputField, UIComponent
+
 
 @invocation(
     "dynamic_prompt",

invokeai/app/invocations/scheduler.py

@@ -1,4 +1,5 @@
 from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
+from invokeai.app.invocations.constants import SCHEDULER_NAME_VALUES
 from invokeai.app.invocations.fields import (
     FieldDescriptions,
     InputField,
@@ -6,7 +7,6 @@ from invokeai.app.invocations.fields import (
     UIType,
 )
 from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.stable_diffusion.schedulers.schedulers import SCHEDULER_NAME_VALUES
 
 
 @invocation_output("scheduler_output")

invokeai/app/invocations/sdxl.py

@@ -1,9 +1,15 @@
-from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
 from invokeai.app.invocations.fields import FieldDescriptions, InputField, OutputField, UIType
-from invokeai.app.invocations.model import CLIPField, ModelIdentifierField, UNetField, VAEField
 from invokeai.app.services.shared.invocation_context import InvocationContext
 from invokeai.backend.model_manager import SubModelType
 
+from .baseinvocation import (
+    BaseInvocation,
+    BaseInvocationOutput,
+    invocation,
+    invocation_output,
+)
+from .model import CLIPField, ModelIdentifierField, UNetField, VAEField
+
 
 @invocation_output("sdxl_model_loader_output")
 class SDXLModelLoaderOutput(BaseInvocationOutput):

invokeai/app/invocations/strings.py

@@ -2,11 +2,17 @@
 
 import re
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput, invocation, invocation_output
-from invokeai.app.invocations.fields import InputField, OutputField, UIComponent
-from invokeai.app.invocations.primitives import StringOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
 
+from .baseinvocation import (
+    BaseInvocation,
+    BaseInvocationOutput,
+    invocation,
+    invocation_output,
+)
+from .fields import InputField, OutputField, UIComponent
+from .primitives import StringOutput
+
 
 @invocation_output("string_pos_neg_output")
 class StringPosNegOutput(BaseInvocationOutput):

invokeai/app/invocations/tiled_multi_diffusion_denoise_latents.py

@@ -1,282 +0,0 @@
-import copy
-from contextlib import ExitStack
-from typing import Iterator, Tuple
-
-import torch
-from diffusers.models.unets.unet_2d_condition import UNet2DConditionModel
-from diffusers.schedulers.scheduling_utils import SchedulerMixin
-from pydantic import field_validator
-
-from invokeai.app.invocations.baseinvocation import BaseInvocation, Classification, invocation
-from invokeai.app.invocations.constants import LATENT_SCALE_FACTOR
-from invokeai.app.invocations.controlnet_image_processors import ControlField
-from invokeai.app.invocations.denoise_latents import DenoiseLatentsInvocation, get_scheduler
-from invokeai.app.invocations.fields import (
-    ConditioningField,
-    FieldDescriptions,
-    Input,
-    InputField,
-    LatentsField,
-    UIType,
-)
-from invokeai.app.invocations.model import UNetField
-from invokeai.app.invocations.primitives import LatentsOutput
-from invokeai.app.services.shared.invocation_context import InvocationContext
-from invokeai.backend.lora import LoRAModelRaw
-from invokeai.backend.model_patcher import ModelPatcher
-from invokeai.backend.stable_diffusion.diffusers_pipeline import ControlNetData, PipelineIntermediateState
-from invokeai.backend.stable_diffusion.multi_diffusion_pipeline import (
-    MultiDiffusionPipeline,
-    MultiDiffusionRegionConditioning,
-)
-from invokeai.backend.stable_diffusion.schedulers.schedulers import SCHEDULER_NAME_VALUES
-from invokeai.backend.tiles.tiles import (
-    calc_tiles_min_overlap,
-)
-from invokeai.backend.tiles.utils import TBLR
-from invokeai.backend.util.devices import TorchDevice
-
-
-def crop_controlnet_data(control_data: ControlNetData, latent_region: TBLR) -> ControlNetData:
-    """Crop a ControlNetData object to a region."""
-    # Create a shallow copy of the control_data object.
-    control_data_copy = copy.copy(control_data)
-    # The ControlNet reference image is the only attribute that needs to be cropped.
-    control_data_copy.image_tensor = control_data.image_tensor[
-        :,
-        :,
-        latent_region.top * LATENT_SCALE_FACTOR : latent_region.bottom * LATENT_SCALE_FACTOR,
-        latent_region.left * LATENT_SCALE_FACTOR : latent_region.right * LATENT_SCALE_FACTOR,
-    ]
-    return control_data_copy
-
-
-@invocation(
-    "tiled_multi_diffusion_denoise_latents",
-    title="Tiled Multi-Diffusion Denoise Latents",
-    tags=["upscale", "denoise"],
-    category="latents",
-    classification=Classification.Beta,
-    version="1.0.0",
-)
-class TiledMultiDiffusionDenoiseLatents(BaseInvocation):
-    """Tiled Multi-Diffusion denoising.
-
-    This node handles automatically tiling the input image, and is primarily intended for global refinement of images
-    in tiled upscaling workflows. Future Multi-Diffusion nodes should allow the user to specify custom regions with
-    different parameters for each region to harness the full power of Multi-Diffusion.
-
-    This node has a similar interface to the `DenoiseLatents` node, but it has a reduced feature set (no IP-Adapter,
-    T2I-Adapter, masking, etc.).
-    """
-
-    positive_conditioning: ConditioningField = InputField(
-        description=FieldDescriptions.positive_cond, input=Input.Connection
-    )
-    negative_conditioning: ConditioningField = InputField(
-        description=FieldDescriptions.negative_cond, input=Input.Connection
-    )
-    noise: LatentsField | None = InputField(
-        default=None,
-        description=FieldDescriptions.noise,
-        input=Input.Connection,
-    )
-    latents: LatentsField | None = InputField(
-        default=None,
-        description=FieldDescriptions.latents,
-        input=Input.Connection,
-    )
-    tile_height: int = InputField(
-        default=1024, gt=0, multiple_of=LATENT_SCALE_FACTOR, description="Height of the tiles in image space."
-    )
-    tile_width: int = InputField(
-        default=1024, gt=0, multiple_of=LATENT_SCALE_FACTOR, description="Width of the tiles in image space."
-    )
-    tile_overlap: int = InputField(
-        default=32,
-        multiple_of=LATENT_SCALE_FACTOR,
-        gt=0,
-        description="The overlap between adjacent tiles in pixel space. (Of course, tile merging is applied in latent "
-        "space.) Tiles will be cropped during merging (if necessary) to ensure that they overlap by exactly this "
-        "amount.",
-    )
-    steps: int = InputField(default=18, gt=0, description=FieldDescriptions.steps)
-    cfg_scale: float | list[float] = InputField(default=6.0, description=FieldDescriptions.cfg_scale, title="CFG Scale")
-    denoising_start: float = InputField(
-        default=0.0,
-        ge=0,
-        le=1,
-        description=FieldDescriptions.denoising_start,
-    )
-    denoising_end: float = InputField(default=1.0, ge=0, le=1, description=FieldDescriptions.denoising_end)
-    scheduler: SCHEDULER_NAME_VALUES = InputField(
-        default="euler",
-        description=FieldDescriptions.scheduler,
-        ui_type=UIType.Scheduler,
-    )
-    unet: UNetField = InputField(
-        description=FieldDescriptions.unet,
-        input=Input.Connection,
-        title="UNet",
-    )
-    cfg_rescale_multiplier: float = InputField(
-        title="CFG Rescale Multiplier", default=0, ge=0, lt=1, description=FieldDescriptions.cfg_rescale_multiplier
-    )
-    control: ControlField | list[ControlField] | None = InputField(
-        default=None,
-        input=Input.Connection,
-    )
-
-    @field_validator("cfg_scale")
-    def ge_one(cls, v: list[float] | float) -> list[float] | float:
-        """Validate that all cfg_scale values are >= 1"""
-        if isinstance(v, list):
-            for i in v:
-                if i < 1:
-                    raise ValueError("cfg_scale must be greater than 1")
-        else:
-            if v < 1:
-                raise ValueError("cfg_scale must be greater than 1")
-        return v
-
-    @staticmethod
-    def create_pipeline(
-        unet: UNet2DConditionModel,
-        scheduler: SchedulerMixin,
-    ) -> MultiDiffusionPipeline:
-        # TODO(ryand): Get rid of this FakeVae hack.
-        class FakeVae:
-            class FakeVaeConfig:
-                def __init__(self) -> None:
-                    self.block_out_channels = [0]
-
-            def __init__(self) -> None:
-                self.config = FakeVae.FakeVaeConfig()
-
-        return MultiDiffusionPipeline(
-            vae=FakeVae(),
-            text_encoder=None,
-            tokenizer=None,
-            unet=unet,
-            scheduler=scheduler,
-            safety_checker=None,
-            feature_extractor=None,
-            requires_safety_checker=False,
-        )
-
-    @torch.no_grad()
-    def invoke(self, context: InvocationContext) -> LatentsOutput:
-        # Convert tile image-space dimensions to latent-space dimensions.
-        latent_tile_height = self.tile_height // LATENT_SCALE_FACTOR
-        latent_tile_width = self.tile_width // LATENT_SCALE_FACTOR
-        latent_tile_overlap = self.tile_overlap // LATENT_SCALE_FACTOR
-
-        seed, noise, latents = DenoiseLatentsInvocation.prepare_noise_and_latents(context, self.noise, self.latents)
-        _, _, latent_height, latent_width = latents.shape
-
-        # Calculate the tile locations to cover the latent-space image.
-        tiles = calc_tiles_min_overlap(
-            image_height=latent_height,
-            image_width=latent_width,
-            tile_height=latent_tile_height,
-            tile_width=latent_tile_width,
-            min_overlap=latent_tile_overlap,
-        )
-
-        # Get the unet's config so that we can pass the base to sd_step_callback().
-        unet_config = context.models.get_config(self.unet.unet.key)
-
-        def step_callback(state: PipelineIntermediateState) -> None:
-            context.util.sd_step_callback(state, unet_config.base)
-
-        # Prepare an iterator that yields the UNet's LoRA models and their weights.
-        def _lora_loader() -> Iterator[Tuple[LoRAModelRaw, float]]:
-            for lora in self.unet.loras:
-                lora_info = context.models.load(lora.lora)
-                assert isinstance(lora_info.model, LoRAModelRaw)
-                yield (lora_info.model, lora.weight)
-                del lora_info
-
-        # Load the UNet model.
-        unet_info = context.models.load(self.unet.unet)
-
-        with ExitStack() as exit_stack, unet_info as unet, ModelPatcher.apply_lora_unet(unet, _lora_loader()):
-            assert isinstance(unet, UNet2DConditionModel)
-            latents = latents.to(device=unet.device, dtype=unet.dtype)
-            if noise is not None:
-                noise = noise.to(device=unet.device, dtype=unet.dtype)
-            scheduler = get_scheduler(
-                context=context,
-                scheduler_info=self.unet.scheduler,
-                scheduler_name=self.scheduler,
-                seed=seed,
-            )
-            pipeline = self.create_pipeline(unet=unet, scheduler=scheduler)
-
-            # Prepare the prompt conditioning data. The same prompt conditioning is applied to all tiles.
-            conditioning_data = DenoiseLatentsInvocation.get_conditioning_data(
-                context=context,
-                positive_conditioning_field=self.positive_conditioning,
-                negative_conditioning_field=self.negative_conditioning,
-                unet=unet,
-                latent_height=latent_tile_height,
-                latent_width=latent_tile_width,
-                cfg_scale=self.cfg_scale,
-                steps=self.steps,
-                cfg_rescale_multiplier=self.cfg_rescale_multiplier,
-            )
-
-            controlnet_data = DenoiseLatentsInvocation.prep_control_data(
-                context=context,
-                control_input=self.control,
-                latents_shape=list(latents.shape),
-                # do_classifier_free_guidance=(self.cfg_scale >= 1.0))
-                do_classifier_free_guidance=True,
-                exit_stack=exit_stack,
-            )
-
-            # Split the controlnet_data into tiles.
-            # controlnet_data_tiles[t][c] is the c'th control data for the t'th tile.
-            controlnet_data_tiles: list[list[ControlNetData]] = []
-            for tile in tiles:
-                tile_controlnet_data = [crop_controlnet_data(cn, tile.coords) for cn in controlnet_data or []]
-                controlnet_data_tiles.append(tile_controlnet_data)
-
-            # Prepare the MultiDiffusionRegionConditioning list.
-            multi_diffusion_conditioning: list[MultiDiffusionRegionConditioning] = []
-            for tile, tile_controlnet_data in zip(tiles, controlnet_data_tiles, strict=True):
-                multi_diffusion_conditioning.append(
-                    MultiDiffusionRegionConditioning(
-                        region=tile,
-                        text_conditioning_data=conditioning_data,
-                        control_data=tile_controlnet_data,
-                    )
-                )
-
-            timesteps, init_timestep, scheduler_step_kwargs = DenoiseLatentsInvocation.init_scheduler(
-                scheduler,
-                device=unet.device,
-                steps=self.steps,
-                denoising_start=self.denoising_start,
-                denoising_end=self.denoising_end,
-                seed=seed,
-            )
-
-            # Run Multi-Diffusion denoising.
-            result_latents = pipeline.multi_diffusion_denoise(
-                multi_diffusion_conditioning=multi_diffusion_conditioning,
-                target_overlap=latent_tile_overlap,
-                latents=latents,
-                scheduler_step_kwargs=scheduler_step_kwargs,
-                noise=noise,
-                timesteps=timesteps,
-                init_timestep=init_timestep,
-                callback=step_callback,
-            )
-
-        result_latents = result_latents.to("cpu")
-        # TODO(ryand): I copied this from DenoiseLatentsInvocation. I'm not sure if it's actually important.
-        TorchDevice.empty_cache()
-
-        name = context.tensors.save(tensor=result_latents)
-        return LatentsOutput.build(latents_name=name, latents=result_latents, seed=None)

invokeai/app/invocations/tiled_stable_diffusion_refine.py

@@ -0,0 +1,384 @@
+from contextlib import ExitStack
+from typing import Iterator, Tuple
+
+import numpy as np
+import numpy.typing as npt
+import torch
+from diffusers.models.unets.unet_2d_condition import UNet2DConditionModel
+from PIL import Image
+from pydantic import field_validator
+
+from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
+from invokeai.app.invocations.constants import DEFAULT_PRECISION, LATENT_SCALE_FACTOR, SCHEDULER_NAME_VALUES
+from invokeai.app.invocations.fields import (
+    ConditioningField,
+    FieldDescriptions,
+    ImageField,
+    Input,
+    InputField,
+    UIType,
+)
+from invokeai.app.invocations.image_to_latents import ImageToLatentsInvocation
+from invokeai.app.invocations.latent import DenoiseLatentsInvocation, get_scheduler
+from invokeai.app.invocations.latents_to_image import LatentsToImageInvocation
+from invokeai.app.invocations.model import ModelIdentifierField, UNetField, VAEField
+from invokeai.app.invocations.noise import get_noise
+from invokeai.app.invocations.primitives import ImageOutput
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.app.util.controlnet_utils import CONTROLNET_MODE_VALUES, CONTROLNET_RESIZE_VALUES, prepare_control_image
+from invokeai.backend.lora import LoRAModelRaw
+from invokeai.backend.model_patcher import ModelPatcher
+from invokeai.backend.stable_diffusion.diffusers_pipeline import ControlNetData, image_resized_to_grid_as_tensor
+from invokeai.backend.tiles.tiles import calc_tiles_with_overlap, merge_tiles_with_linear_blending
+from invokeai.backend.tiles.utils import Tile
+from invokeai.backend.util.devices import TorchDevice
+from invokeai.backend.util.hotfixes import ControlNetModel
+
+
+@invocation(
+    "tiled_stable_diffusion_refine",
+    title="Tiled Stable Diffusion Refine",
+    tags=["upscale", "denoise"],
+    category="latents",
+    version="1.0.0",
+)
+class TiledStableDiffusionRefineInvocation(BaseInvocation):
+    """A tiled Stable Diffusion pipeline for refining high resolution images. This invocation is intended to be used to
+    refine an image after upscaling i.e. it is the second step in a typical "tiled upscaling" workflow.
+    """
+
+    image: ImageField = InputField(description="Image to be refined.")
+
+    positive_conditioning: ConditioningField = InputField(
+        description=FieldDescriptions.positive_cond, input=Input.Connection
+    )
+    negative_conditioning: ConditioningField = InputField(
+        description=FieldDescriptions.negative_cond, input=Input.Connection
+    )
+    # TODO(ryand): Add multiple-of validation.
+    tile_height: int = InputField(default=512, gt=0, description="Height of the tiles.")
+    tile_width: int = InputField(default=512, gt=0, description="Width of the tiles.")
+    tile_overlap: int = InputField(
+        default=16,
+        gt=0,
+        description="Target overlap between adjacent tiles (the last row/column may overlap more than this).",
+    )
+    steps: int = InputField(default=18, gt=0, description=FieldDescriptions.steps)
+    cfg_scale: float | list[float] = InputField(default=6.0, description=FieldDescriptions.cfg_scale, title="CFG Scale")
+    denoising_start: float = InputField(
+        default=0.65,
+        ge=0,
+        le=1,
+        description=FieldDescriptions.denoising_start,
+    )
+    denoising_end: float = InputField(default=1.0, ge=0, le=1, description=FieldDescriptions.denoising_end)
+    scheduler: SCHEDULER_NAME_VALUES = InputField(
+        default="euler",
+        description=FieldDescriptions.scheduler,
+        ui_type=UIType.Scheduler,
+    )
+    unet: UNetField = InputField(
+        description=FieldDescriptions.unet,
+        input=Input.Connection,
+        title="UNet",
+    )
+    cfg_rescale_multiplier: float = InputField(
+        title="CFG Rescale Multiplier", default=0, ge=0, lt=1, description=FieldDescriptions.cfg_rescale_multiplier
+    )
+    vae: VAEField = InputField(
+        description=FieldDescriptions.vae,
+        input=Input.Connection,
+    )
+    vae_fp32: bool = InputField(
+        default=DEFAULT_PRECISION == torch.float32, description="Whether to use float32 precision when running the VAE."
+    )
+    # HACK(ryand): We probably want to allow the user to control all of the parameters in ControlField. But, we akwardly
+    # don't want to use the image field. Figure out how best to handle this.
+    # TODO(ryand): Currently, there is no ControlNet preprocessor applied to the tile images. In other words, we pretty
+    # much assume that it is a tile ControlNet. We need to decide how we want to handle this. E.g. find a way to support
+    # CN preprocessors, raise a clear warning when a non-tile CN model is selected, hardcode the supported CN models,
+    # etc.
+    control_model: ModelIdentifierField = InputField(
+        description=FieldDescriptions.controlnet_model, ui_type=UIType.ControlNetModel
+    )
+    control_weight: float = InputField(default=0.6)
+
+    @field_validator("cfg_scale")
+    def ge_one(cls, v: list[float] | float) -> list[float] | float:
+        """Validate that all cfg_scale values are >= 1"""
+        if isinstance(v, list):
+            for i in v:
+                if i < 1:
+                    raise ValueError("cfg_scale must be greater than 1")
+        else:
+            if v < 1:
+                raise ValueError("cfg_scale must be greater than 1")
+        return v
+
+    @staticmethod
+    def crop_latents_to_tile(latents: torch.Tensor, image_tile: Tile) -> torch.Tensor:
+        """Crop the latent-space tensor to the area corresponding to the image-space tile.
+        The tile coordinates must be divisible by the LATENT_SCALE_FACTOR.
+        """
+        for coord in [image_tile.coords.top, image_tile.coords.left, image_tile.coords.right, image_tile.coords.bottom]:
+            if coord % LATENT_SCALE_FACTOR != 0:
+                raise ValueError(
+                    f"The tile coordinates must all be divisible by the latent scale factor"
+                    f" ({LATENT_SCALE_FACTOR}). {image_tile.coords=}."
+                )
+        assert latents.dim() == 4  # We expect: (batch_size, channels, height, width).
+
+        top = image_tile.coords.top // LATENT_SCALE_FACTOR
+        left = image_tile.coords.left // LATENT_SCALE_FACTOR
+        bottom = image_tile.coords.bottom // LATENT_SCALE_FACTOR
+        right = image_tile.coords.right // LATENT_SCALE_FACTOR
+        return latents[..., top:bottom, left:right]
+
+    def run_controlnet(
+        self,
+        image: Image.Image,
+        controlnet_model: ControlNetModel,
+        weight: float,
+        do_classifier_free_guidance: bool,
+        width: int,
+        height: int,
+        device: torch.device,
+        dtype: torch.dtype,
+        control_mode: CONTROLNET_MODE_VALUES = "balanced",
+        resize_mode: CONTROLNET_RESIZE_VALUES = "just_resize_simple",
+    ) -> ControlNetData:
+        control_image = prepare_control_image(
+            image=image,
+            do_classifier_free_guidance=do_classifier_free_guidance,
+            width=width,
+            height=height,
+            device=device,
+            dtype=dtype,
+            control_mode=control_mode,
+            resize_mode=resize_mode,
+        )
+        return ControlNetData(
+            model=controlnet_model,
+            image_tensor=control_image,
+            weight=weight,
+            begin_step_percent=0.0,
+            end_step_percent=1.0,
+            control_mode=control_mode,
+            # Any resizing needed should currently be happening in prepare_control_image(), but adding resize_mode to
+            # ControlNetData in case needed in the future.
+            resize_mode=resize_mode,
+        )
+
+    @torch.no_grad()
+    def invoke(self, context: InvocationContext) -> ImageOutput:
+        # TODO(ryand): Expose the seed parameter.
+        seed = 0
+
+        # Load the input image.
+        input_image = context.images.get_pil(self.image.image_name)
+
+        # Calculate the tile locations to cover the image.
+        # We have selected this tiling strategy to make it easy to achieve tile coords that are multiples of 8. This
+        # facilitates conversions between image space and latent space.
+        # TODO(ryand): Expose these tiling parameters. (Keep in mind the multiple-of constraints on these params.)
+        tiles = calc_tiles_with_overlap(
+            image_height=input_image.height,
+            image_width=input_image.width,
+            tile_height=self.tile_height,
+            tile_width=self.tile_width,
+            overlap=self.tile_overlap,
+        )
+
+        # Convert the input image to a torch.Tensor.
+        input_image_torch = image_resized_to_grid_as_tensor(input_image.convert("RGB"), multiple_of=LATENT_SCALE_FACTOR)
+        input_image_torch = input_image_torch.unsqueeze(0)  # Add a batch dimension.
+        # Validate our assumptions about the shape of input_image_torch.
+        assert input_image_torch.dim() == 4  # We expect: (batch_size, channels, height, width).
+        assert input_image_torch.shape[:2] == (1, 3)
+
+        # Split the input image into tiles in torch.Tensor format.
+        image_tiles_torch: list[torch.Tensor] = []
+        for tile in tiles:
+            image_tile = input_image_torch[
+                :,
+                :,
+                tile.coords.top : tile.coords.bottom,
+                tile.coords.left : tile.coords.right,
+            ]
+            image_tiles_torch.append(image_tile)
+
+        # Split the input image into tiles in numpy format.
+        # TODO(ryand): We currently maintain both np.ndarray and torch.Tensor tiles. Ideally, all operations should work
+        # with torch.Tensor tiles.
+        input_image_np = np.array(input_image)
+        image_tiles_np: list[npt.NDArray[np.uint8]] = []
+        for tile in tiles:
+            image_tile_np = input_image_np[
+                tile.coords.top : tile.coords.bottom,
+                tile.coords.left : tile.coords.right,
+                :,
+            ]
+            image_tiles_np.append(image_tile_np)
+
+        # VAE-encode each image tile independently.
+        # TODO(ryand): Is there any advantage to VAE-encoding the entire image before splitting it into tiles? What
+        # about for decoding?
+        vae_info = context.models.load(self.vae.vae)
+        latent_tiles: list[torch.Tensor] = []
+        for image_tile_torch in image_tiles_torch:
+            latent_tiles.append(
+                ImageToLatentsInvocation.vae_encode(
+                    vae_info=vae_info, upcast=self.vae_fp32, tiled=False, image_tensor=image_tile_torch
+                )
+            )
+
+        # Generate noise with dimensions corresponding to the full image in latent space.
+        # It is important that the noise tensor is generated at the full image dimension and then tiled, rather than
+        # generating for each tile independently. This ensures that overlapping regions between tiles use the same
+        # noise.
+        assert input_image_torch.shape[2] % LATENT_SCALE_FACTOR == 0
+        assert input_image_torch.shape[3] % LATENT_SCALE_FACTOR == 0
+        global_noise = get_noise(
+            width=input_image_torch.shape[3],
+            height=input_image_torch.shape[2],
+            device=TorchDevice.choose_torch_device(),
+            seed=seed,
+            downsampling_factor=LATENT_SCALE_FACTOR,
+            use_cpu=True,
+        )
+
+        # Crop the global noise into tiles.
+        noise_tiles = [self.crop_latents_to_tile(latents=global_noise, image_tile=t) for t in tiles]
+
+        # Prepare an iterator that yields the UNet's LoRA models and their weights.
+        def _lora_loader() -> Iterator[Tuple[LoRAModelRaw, float]]:
+            for lora in self.unet.loras:
+                lora_info = context.models.load(lora.lora)
+                assert isinstance(lora_info.model, LoRAModelRaw)
+                yield (lora_info.model, lora.weight)
+                del lora_info
+
+        # Load the UNet model.
+        unet_info = context.models.load(self.unet.unet)
+
+        refined_latent_tiles: list[torch.Tensor] = []
+        with ExitStack() as exit_stack, unet_info as unet, ModelPatcher.apply_lora_unet(unet, _lora_loader()):
+            assert isinstance(unet, UNet2DConditionModel)
+            scheduler = get_scheduler(
+                context=context,
+                scheduler_info=self.unet.scheduler,
+                scheduler_name=self.scheduler,
+                seed=seed,
+            )
+            pipeline = DenoiseLatentsInvocation.create_pipeline(unet=unet, scheduler=scheduler)
+
+            # Prepare the prompt conditioning data. The same prompt conditioning is applied to all tiles.
+            # Assume that all tiles have the same shape.
+            _, _, latent_height, latent_width = latent_tiles[0].shape
+            conditioning_data = DenoiseLatentsInvocation.get_conditioning_data(
+                context=context,
+                positive_conditioning_field=self.positive_conditioning,
+                negative_conditioning_field=self.negative_conditioning,
+                unet=unet,
+                latent_height=latent_height,
+                latent_width=latent_width,
+                cfg_scale=self.cfg_scale,
+                steps=self.steps,
+                cfg_rescale_multiplier=self.cfg_rescale_multiplier,
+            )
+
+            # Load the ControlNet model.
+            # TODO(ryand): Support multiple ControlNet models.
+            controlnet_model = exit_stack.enter_context(context.models.load(self.control_model))
+            assert isinstance(controlnet_model, ControlNetModel)
+
+            # Denoise (i.e. "refine") each tile independently.
+            for image_tile_np, latent_tile, noise_tile in zip(image_tiles_np, latent_tiles, noise_tiles, strict=True):
+                assert latent_tile.shape == noise_tile.shape
+
+                # Prepare a PIL Image for ControlNet processing.
+                # TODO(ryand): This is a bit awkward that we have to prepare both torch.Tensor and PIL.Image versions of
+                # the tiles. Ideally, the ControlNet code should be able to work with Tensors.
+                image_tile_pil = Image.fromarray(image_tile_np)
+
+                # Run the ControlNet on the image tile.
+                height, width, _ = image_tile_np.shape
+                # The height and width must be evenly divisible by LATENT_SCALE_FACTOR. This is enforced earlier, but we
+                # validate this assumption here.
+                assert height % LATENT_SCALE_FACTOR == 0
+                assert width % LATENT_SCALE_FACTOR == 0
+                controlnet_data = self.run_controlnet(
+                    image=image_tile_pil,
+                    controlnet_model=controlnet_model,
+                    weight=self.control_weight,
+                    do_classifier_free_guidance=True,
+                    width=width,
+                    height=height,
+                    device=controlnet_model.device,
+                    dtype=controlnet_model.dtype,
+                    control_mode="balanced",
+                    resize_mode="just_resize_simple",
+                )
+
+                num_inference_steps, timesteps, init_timestep, scheduler_step_kwargs = (
+                    DenoiseLatentsInvocation.init_scheduler(
+                        scheduler,
+                        device=unet.device,
+                        steps=self.steps,
+                        denoising_start=self.denoising_start,
+                        denoising_end=self.denoising_end,
+                        seed=seed,
+                    )
+                )
+
+                # TODO(ryand): Think about when/if latents/noise should be moved off of the device to save VRAM.
+                latent_tile = latent_tile.to(device=unet.device, dtype=unet.dtype)
+                noise_tile = noise_tile.to(device=unet.device, dtype=unet.dtype)
+                refined_latent_tile = pipeline.latents_from_embeddings(
+                    latents=latent_tile,
+                    timesteps=timesteps,
+                    init_timestep=init_timestep,
+                    noise=noise_tile,
+                    seed=seed,
+                    mask=None,
+                    masked_latents=None,
+                    gradient_mask=None,
+                    num_inference_steps=num_inference_steps,
+                    scheduler_step_kwargs=scheduler_step_kwargs,
+                    conditioning_data=conditioning_data,
+                    control_data=[controlnet_data],
+                    ip_adapter_data=None,
+                    t2i_adapter_data=None,
+                    callback=lambda x: None,
+                )
+                refined_latent_tiles.append(refined_latent_tile)
+
+        # VAE-decode each refined latent tile independently.
+        refined_image_tiles: list[Image.Image] = []
+        for refined_latent_tile in refined_latent_tiles:
+            refined_image_tile = LatentsToImageInvocation.vae_decode(
+                context=context,
+                vae_info=vae_info,
+                seamless_axes=self.vae.seamless_axes,
+                latents=refined_latent_tile,
+                use_fp32=self.vae_fp32,
+                use_tiling=False,
+            )
+            refined_image_tiles.append(refined_image_tile)
+
+        # TODO(ryand): I copied this from DenoiseLatentsInvocation. I'm not sure if it's actually important.
+        TorchDevice.empty_cache()
+
+        # Merge the refined image tiles back into a single image.
+        refined_image_tiles_np = [np.array(t) for t in refined_image_tiles]
+        merged_image_np = np.zeros(shape=(input_image.height, input_image.width, 3), dtype=np.uint8)
+        # TODO(ryand): Tune the blend_amount. Should this be exposed as a parameter?
+        merge_tiles_with_linear_blending(
+            dst_image=merged_image_np, tiles=tiles, tile_images=refined_image_tiles_np, blend_amount=self.tile_overlap
+        )
+
+        # Save the refined image and return its reference.
+        merged_image_pil = Image.fromarray(merged_image_np)
+        image_dto = context.images.save(image=merged_image_pil)
+
+        return ImageOutput.build(image_dto)

invokeai/app/invocations/upscale.py

@@ -1,4 +1,5 @@
 # Copyright (c) 2022 Kyle Schouviller (https://github.com/kyle0654) & the InvokeAI Team
+from pathlib import Path
 from typing import Literal
 
 import cv2
@@ -6,12 +7,16 @@ import numpy as np
 from PIL import Image
 from pydantic import ConfigDict
 
-from invokeai.app.invocations.baseinvocation import BaseInvocation, invocation
-from invokeai.app.invocations.fields import ImageField, InputField, WithBoard, WithMetadata
+from invokeai.app.invocations.fields import ImageField
 from invokeai.app.invocations.primitives import ImageOutput
 from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.app.util.download_with_progress import download_with_progress_bar
 from invokeai.backend.image_util.basicsr.rrdbnet_arch import RRDBNet
 from invokeai.backend.image_util.realesrgan.realesrgan import RealESRGAN
+from invokeai.backend.util.devices import TorchDevice
+
+from .baseinvocation import BaseInvocation, invocation
+from .fields import InputField, WithBoard, WithMetadata
 
 # TODO: Populate this from disk?
 # TODO: Use model manager to load?
@@ -47,6 +52,7 @@ class ESRGANInvocation(BaseInvocation, WithMetadata, WithBoard):
 
         rrdbnet_model = None
         netscale = None
+        esrgan_model_path = None
 
         if self.model_name in [
             "RealESRGAN_x4plus.pth",
@@ -89,25 +95,28 @@ class ESRGANInvocation(BaseInvocation, WithMetadata, WithBoard):
             context.logger.error(msg)
             raise ValueError(msg)
 
-        loadnet = context.models.load_remote_model(
-            source=ESRGAN_MODEL_URLS[self.model_name],
+        esrgan_model_path = Path(context.config.get().models_path, f"core/upscaling/realesrgan/{self.model_name}")
+
+        # Downloads the ESRGAN model if it doesn't already exist
+        download_with_progress_bar(
+            name=self.model_name, url=ESRGAN_MODEL_URLS[self.model_name], dest_path=esrgan_model_path
         )
 
-        with loadnet as loadnet_model:
-            upscaler = RealESRGAN(
-                scale=netscale,
-                loadnet=loadnet_model,
-                model=rrdbnet_model,
-                half=False,
-                tile=self.tile_size,
-            )
+        upscaler = RealESRGAN(
+            scale=netscale,
+            model_path=esrgan_model_path,
+            model=rrdbnet_model,
+            half=False,
+            tile=self.tile_size,
+        )
 
-            # prepare image - Real-ESRGAN uses cv2 internally, and cv2 uses BGR vs RGB for PIL
-            # TODO: This strips the alpha... is that okay?
-            cv2_image = cv2.cvtColor(np.array(image.convert("RGB")), cv2.COLOR_RGB2BGR)
-            upscaled_image = upscaler.upscale(cv2_image)
+        # prepare image - Real-ESRGAN uses cv2 internally, and cv2 uses BGR vs RGB for PIL
+        # TODO: This strips the alpha... is that okay?
+        cv2_image = cv2.cvtColor(np.array(image.convert("RGB")), cv2.COLOR_RGB2BGR)
+        upscaled_image = upscaler.upscale(cv2_image)
+        pil_image = Image.fromarray(cv2.cvtColor(upscaled_image, cv2.COLOR_BGR2RGB)).convert("RGBA")
 
-            pil_image = Image.fromarray(cv2.cvtColor(upscaled_image, cv2.COLOR_BGR2RGB)).convert("RGBA")
+        TorchDevice.empty_cache()
 
         image_dto = context.images.save(image=pil_image)
 

invokeai/app/services/board_image_records/board_image_records_sqlite.py

@@ -2,11 +2,12 @@ import sqlite3
 import threading
 from typing import Optional, cast
 
-from invokeai.app.services.board_image_records.board_image_records_base import BoardImageRecordStorageBase
 from invokeai.app.services.image_records.image_records_common import ImageRecord, deserialize_image_record
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
 from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
 
+from .board_image_records_base import BoardImageRecordStorageBase
+
 
 class SqliteBoardImageRecordStorage(BoardImageRecordStorageBase):
     _conn: sqlite3.Connection

invokeai/app/services/board_images/board_images_default.py

@@ -1,8 +1,9 @@
 from typing import Optional
 
-from invokeai.app.services.board_images.board_images_base import BoardImagesServiceABC
 from invokeai.app.services.invoker import Invoker
 
+from .board_images_base import BoardImagesServiceABC
+
 
 class BoardImagesService(BoardImagesServiceABC):
     __invoker: Invoker

invokeai/app/services/board_records/board_records_base.py

@@ -1,8 +1,9 @@
 from abc import ABC, abstractmethod
 
-from invokeai.app.services.board_records.board_records_common import BoardChanges, BoardRecord
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
 
+from .board_records_common import BoardChanges, BoardRecord
+
 
 class BoardRecordStorageBase(ABC):
     """Low-level service responsible for interfacing with the board record store."""
@@ -39,12 +40,16 @@ class BoardRecordStorageBase(ABC):
 
     @abstractmethod
     def get_many(
-        self, offset: int = 0, limit: int = 10, include_archived: bool = False
+        self,
+        offset: int = 0,
+        limit: int = 10,
     ) -> OffsetPaginatedResults[BoardRecord]:
         """Gets many board records."""
         pass
 
     @abstractmethod
-    def get_all(self, include_archived: bool = False) -> list[BoardRecord]:
+    def get_all(
+        self,
+    ) -> list[BoardRecord]:
         """Gets all board records."""
         pass

invokeai/app/services/board_records/board_records_common.py

@@ -22,10 +22,6 @@ class BoardRecord(BaseModelExcludeNull):
     """The updated timestamp of the image."""
     cover_image_name: Optional[str] = Field(default=None, description="The name of the cover image of the board.")
     """The name of the cover image of the board."""
-    archived: bool = Field(description="Whether or not the board is archived.")
-    """Whether or not the board is archived."""
-    is_private: Optional[bool] = Field(default=None, description="Whether the board is private.")
-    """Whether the board is private."""
 
 
 def deserialize_board_record(board_dict: dict) -> BoardRecord:
@@ -39,8 +35,6 @@ def deserialize_board_record(board_dict: dict) -> BoardRecord:
     created_at = board_dict.get("created_at", get_iso_timestamp())
     updated_at = board_dict.get("updated_at", get_iso_timestamp())
     deleted_at = board_dict.get("deleted_at", get_iso_timestamp())
-    archived = board_dict.get("archived", False)
-    is_private = board_dict.get("is_private", False)
 
     return BoardRecord(
         board_id=board_id,
@@ -49,15 +43,12 @@ def deserialize_board_record(board_dict: dict) -> BoardRecord:
         created_at=created_at,
         updated_at=updated_at,
         deleted_at=deleted_at,
-        archived=archived,
-        is_private=is_private,
     )
 
 
 class BoardChanges(BaseModel, extra="forbid"):
     board_name: Optional[str] = Field(default=None, description="The board's new name.")
     cover_image_name: Optional[str] = Field(default=None, description="The name of the board's new cover image.")
-    archived: Optional[bool] = Field(default=None, description="Whether or not the board is archived")
 
 
 class BoardRecordNotFoundException(Exception):

invokeai/app/services/board_records/board_records_sqlite.py

@@ -2,8 +2,12 @@ import sqlite3
 import threading
 from typing import Union, cast
 
-from invokeai.app.services.board_records.board_records_base import BoardRecordStorageBase
-from invokeai.app.services.board_records.board_records_common import (
+from invokeai.app.services.shared.pagination import OffsetPaginatedResults
+from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
+from invokeai.app.util.misc import uuid_string
+
+from .board_records_base import BoardRecordStorageBase
+from .board_records_common import (
     BoardChanges,
     BoardRecord,
     BoardRecordDeleteException,
@@ -11,9 +15,6 @@ from invokeai.app.services.board_records.board_records_common import (
     BoardRecordSaveException,
     deserialize_board_record,
 )
-from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
-from invokeai.app.util.misc import uuid_string
 
 
 class SqliteBoardRecordStorage(BoardRecordStorageBase):
@@ -124,17 +125,6 @@ class SqliteBoardRecordStorage(BoardRecordStorageBase):
                     (changes.cover_image_name, board_id),
                 )
 
-            # Change the archived status of a board
-            if changes.archived is not None:
-                self._cursor.execute(
-                    """--sql
-                    UPDATE boards
-                    SET archived = ?
-                    WHERE board_id = ?;
-                    """,
-                    (changes.archived, board_id),
-                )
-
             self._conn.commit()
         except sqlite3.Error as e:
             self._conn.rollback()
@@ -144,49 +134,35 @@ class SqliteBoardRecordStorage(BoardRecordStorageBase):
         return self.get(board_id)
 
     def get_many(
-        self, offset: int = 0, limit: int = 10, include_archived: bool = False
+        self,
+        offset: int = 0,
+        limit: int = 10,
     ) -> OffsetPaginatedResults[BoardRecord]:
         try:
             self._lock.acquire()
 
-            # Build base query
-            base_query = """
+            # Get all the boards
+            self._cursor.execute(
+                """--sql
                 SELECT *
                 FROM boards
-                {archived_filter}
                 ORDER BY created_at DESC
                 LIMIT ? OFFSET ?;
-            """
-
-            # Determine archived filter condition
-            if include_archived:
-                archived_filter = ""
-            else:
-                archived_filter = "WHERE archived = 0"
-
-            final_query = base_query.format(archived_filter=archived_filter)
-
-            # Execute query to fetch boards
-            self._cursor.execute(final_query, (limit, offset))
+                """,
+                (limit, offset),
+            )
 
             result = cast(list[sqlite3.Row], self._cursor.fetchall())
             boards = [deserialize_board_record(dict(r)) for r in result]
 
-            # Determine count query
-            if include_archived:
-                count_query = """
-                    SELECT COUNT(*)
-                    FROM boards;
+            # Get the total number of boards
+            self._cursor.execute(
+                """--sql
+                SELECT COUNT(*)
+                FROM boards
+                WHERE 1=1;
                 """
-            else:
-                count_query = """
-                    SELECT COUNT(*)
-                    FROM boards
-                    WHERE archived = 0;
-                """
-
-            # Execute count query
-            self._cursor.execute(count_query)
+            )
 
             count = cast(int, self._cursor.fetchone()[0])
 
@@ -198,25 +174,20 @@ class SqliteBoardRecordStorage(BoardRecordStorageBase):
         finally:
             self._lock.release()
 
-    def get_all(self, include_archived: bool = False) -> list[BoardRecord]:
+    def get_all(
+        self,
+    ) -> list[BoardRecord]:
         try:
             self._lock.acquire()
 
-            base_query = """
+            # Get all the boards
+            self._cursor.execute(
+                """--sql
                 SELECT *
                 FROM boards
-                {archived_filter}
                 ORDER BY created_at DESC
-            """
-
-            if include_archived:
-                archived_filter = ""
-            else:
-                archived_filter = "WHERE archived = 0"
-
-            final_query = base_query.format(archived_filter=archived_filter)
-
-            self._cursor.execute(final_query)
+                """
+            )
 
             result = cast(list[sqlite3.Row], self._cursor.fetchall())
             boards = [deserialize_board_record(dict(r)) for r in result]

invokeai/app/services/boards/boards_base.py

@@ -1,9 +1,10 @@
 from abc import ABC, abstractmethod
 
 from invokeai.app.services.board_records.board_records_common import BoardChanges
-from invokeai.app.services.boards.boards_common import BoardDTO
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
 
+from .boards_common import BoardDTO
+
 
 class BoardServiceABC(ABC):
     """High-level service for board management."""
@@ -43,12 +44,16 @@ class BoardServiceABC(ABC):
 
     @abstractmethod
     def get_many(
-        self, offset: int = 0, limit: int = 10, include_archived: bool = False
+        self,
+        offset: int = 0,
+        limit: int = 10,
     ) -> OffsetPaginatedResults[BoardDTO]:
         """Gets many boards."""
         pass
 
     @abstractmethod
-    def get_all(self, include_archived: bool = False) -> list[BoardDTO]:
+    def get_all(
+        self,
+    ) -> list[BoardDTO]:
         """Gets all boards."""
         pass

invokeai/app/services/boards/boards_common.py

@@ -2,7 +2,7 @@ from typing import Optional
 
 from pydantic import Field
 
-from invokeai.app.services.board_records.board_records_common import BoardRecord
+from ..board_records.board_records_common import BoardRecord
 
 
 class BoardDTO(BoardRecord):

invokeai/app/services/boards/boards_default.py

@@ -1,9 +1,11 @@
 from invokeai.app.services.board_records.board_records_common import BoardChanges
-from invokeai.app.services.boards.boards_base import BoardServiceABC
-from invokeai.app.services.boards.boards_common import BoardDTO, board_record_to_dto
+from invokeai.app.services.boards.boards_common import BoardDTO
 from invokeai.app.services.invoker import Invoker
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
 
+from .boards_base import BoardServiceABC
+from .boards_common import board_record_to_dto
+
 
 class BoardService(BoardServiceABC):
     __invoker: Invoker
@@ -46,10 +48,8 @@ class BoardService(BoardServiceABC):
     def delete(self, board_id: str) -> None:
         self.__invoker.services.board_records.delete(board_id)
 
-    def get_many(
-        self, offset: int = 0, limit: int = 10, include_archived: bool = False
-    ) -> OffsetPaginatedResults[BoardDTO]:
-        board_records = self.__invoker.services.board_records.get_many(offset, limit, include_archived)
+    def get_many(self, offset: int = 0, limit: int = 10) -> OffsetPaginatedResults[BoardDTO]:
+        board_records = self.__invoker.services.board_records.get_many(offset, limit)
         board_dtos = []
         for r in board_records.items:
             cover_image = self.__invoker.services.image_records.get_most_recent_image_for_board(r.board_id)
@@ -63,8 +63,8 @@ class BoardService(BoardServiceABC):
 
         return OffsetPaginatedResults[BoardDTO](items=board_dtos, offset=offset, limit=limit, total=len(board_dtos))
 
-    def get_all(self, include_archived: bool = False) -> list[BoardDTO]:
-        board_records = self.__invoker.services.board_records.get_all(include_archived)
+    def get_all(self) -> list[BoardDTO]:
+        board_records = self.__invoker.services.board_records.get_all()
         board_dtos = []
         for r in board_records:
             cover_image = self.__invoker.services.image_records.get_most_recent_image_for_board(r.board_id)

invokeai/app/services/bulk_download/bulk_download_default.py

@@ -4,7 +4,6 @@ from typing import Optional, Union
 from zipfile import ZipFile
 
 from invokeai.app.services.board_records.board_records_common import BoardRecordNotFoundException
-from invokeai.app.services.bulk_download.bulk_download_base import BulkDownloadBase
 from invokeai.app.services.bulk_download.bulk_download_common import (
     DEFAULT_BULK_DOWNLOAD_ID,
     BulkDownloadException,
@@ -16,6 +15,8 @@ from invokeai.app.services.images.images_common import ImageDTO
 from invokeai.app.services.invoker import Invoker
 from invokeai.app.util.misc import uuid_string
 
+from .bulk_download_base import BulkDownloadBase
+
 
 class BulkDownloadService(BulkDownloadBase):
     def start(self, invoker: Invoker) -> None:

invokeai/app/services/config/__init__.py

@@ -1,6 +1,7 @@
 """Init file for InvokeAI configure package."""
 
 from invokeai.app.services.config.config_common import PagingArgumentParser
-from invokeai.app.services.config.config_default import InvokeAIAppConfig, get_config
+
+from .config_default import InvokeAIAppConfig, get_config
 
 __all__ = ["InvokeAIAppConfig", "get_config", "PagingArgumentParser"]

invokeai/app/services/config/config_default.py

@@ -3,7 +3,6 @@
 
 from __future__ import annotations
 
-import copy
 import locale
 import os
 import re
@@ -26,13 +25,14 @@ DB_FILE = Path("invokeai.db")
 LEGACY_INIT_FILE = Path("invokeai.init")
 DEFAULT_RAM_CACHE = 10.0
 DEFAULT_VRAM_CACHE = 0.25
+DEFAULT_CONVERT_CACHE = 20.0
 DEVICE = Literal["auto", "cpu", "cuda", "cuda:1", "mps"]
 PRECISION = Literal["auto", "float16", "bfloat16", "float32"]
 ATTENTION_TYPE = Literal["auto", "normal", "xformers", "sliced", "torch-sdp"]
 ATTENTION_SLICE_SIZE = Literal["auto", "balanced", "max", 1, 2, 3, 4, 5, 6, 7, 8]
 LOG_FORMAT = Literal["plain", "color", "syslog", "legacy"]
 LOG_LEVEL = Literal["debug", "info", "warning", "error", "critical"]
-CONFIG_SCHEMA_VERSION = "4.0.2"
+CONFIG_SCHEMA_VERSION = "4.0.1"
 
 
 def get_default_ram_cache_size() -> float:
@@ -85,8 +85,7 @@ class InvokeAIAppConfig(BaseSettings):
         log_tokenization: Enable logging of parsed prompt tokens.
         patchmatch: Enable patchmatch inpaint code.
         models_dir: Path to the models directory.
-        convert_cache_dir: Path to the converted models cache directory (DEPRECATED, but do not delete because it is needed for migration from previous versions).
-        download_cache_dir: Path to the directory that contains dynamically downloaded models.
+        convert_cache_dir: Path to the converted models cache directory. When loading a non-diffusers model, it will be converted and store on disk at this location.
         legacy_conf_dir: Path to directory of legacy checkpoint config files.
         db_dir: Path to InvokeAI databases directory.
         outputs_dir: Path to directory for outputs.
@@ -102,6 +101,7 @@ class InvokeAIAppConfig(BaseSettings):
         profiles_dir: Path to profiles output directory.
         ram: Maximum memory amount used by memory model cache for rapid switching (GB).
         vram: Amount of VRAM reserved for model storage (GB).
+        convert_cache: Maximum size of on-disk converted models cache (GB).
         lazy_offload: Keep models in VRAM until their space is needed.
         log_memory_usage: If True, a memory snapshot will be captured before and after every model cache operation, and the result will be logged (at debug level). There is a time cost to capturing the memory snapshots, so it is recommended to only enable this feature if you are actively inspecting the model cache's behaviour.
         device: Preferred execution device. `auto` will choose the device depending on the hardware platform and the installed torch capabilities.<br>Valid values: `auto`, `cpu`, `cuda`, `cuda:1`, `mps`
@@ -112,7 +112,6 @@ class InvokeAIAppConfig(BaseSettings):
         force_tiled_decode: Whether to enable tiled VAE decode (reduces memory consumption with some performance penalty).
         pil_compress_level: The compress_level setting of PIL.Image.save(), used for PNG encoding. All settings are lossless. 0 = no compression, 1 = fastest with slightly larger filesize, 9 = slowest with smallest filesize. 1 is typically the best setting.
         max_queue_size: Maximum number of items in the session queue.
-        clear_queue_on_startup: Empties session queue on startup.
         allow_nodes: List of nodes to allow. Omit to allow all.
         deny_nodes: List of nodes to deny. Omit to deny none.
         node_cache_size: How many cached nodes to keep in memory.
@@ -147,8 +146,7 @@ class InvokeAIAppConfig(BaseSettings):
 
     # PATHS
     models_dir:                    Path = Field(default=Path("models"),     description="Path to the models directory.")
-    convert_cache_dir:             Path = Field(default=Path("models/.convert_cache"), description="Path to the converted models cache directory (DEPRECATED, but do not delete because it is needed for migration from previous versions).")
-    download_cache_dir:            Path = Field(default=Path("models/.download_cache"), description="Path to the directory that contains dynamically downloaded models.")
+    convert_cache_dir:             Path = Field(default=Path("models/.cache"), description="Path to the converted models cache directory. When loading a non-diffusers model, it will be converted and store on disk at this location.")
     legacy_conf_dir:               Path = Field(default=Path("configs"), description="Path to directory of legacy checkpoint config files.")
     db_dir:                        Path = Field(default=Path("databases"),  description="Path to InvokeAI databases directory.")
     outputs_dir:                   Path = Field(default=Path("outputs"),    description="Path to directory for outputs.")
@@ -169,8 +167,9 @@ class InvokeAIAppConfig(BaseSettings):
     profiles_dir:                  Path = Field(default=Path("profiles"),   description="Path to profiles output directory.")
 
     # CACHE
-    ram:                           float = Field(default_factory=get_default_ram_cache_size, gt=0, description="Maximum memory amount used by memory model cache for rapid switching (GB).")
-    vram:                          float = Field(default=DEFAULT_VRAM_CACHE, ge=0, description="Amount of VRAM reserved for model storage (GB).")
+    ram:                          float = Field(default_factory=get_default_ram_cache_size, gt=0, description="Maximum memory amount used by memory model cache for rapid switching (GB).")
+    vram:                         float = Field(default=DEFAULT_VRAM_CACHE, ge=0, description="Amount of VRAM reserved for model storage (GB).")
+    convert_cache:                float = Field(default=DEFAULT_CONVERT_CACHE, ge=0, description="Maximum size of on-disk converted models cache (GB).")
     lazy_offload:                  bool = Field(default=True,               description="Keep models in VRAM until their space is needed.")
     log_memory_usage:              bool = Field(default=False,              description="If True, a memory snapshot will be captured before and after every model cache operation, and the result will be logged (at debug level). There is a time cost to capturing the memory snapshots, so it is recommended to only enable this feature if you are actively inspecting the model cache's behaviour.")
 
@@ -185,7 +184,6 @@ class InvokeAIAppConfig(BaseSettings):
     force_tiled_decode:            bool = Field(default=False,              description="Whether to enable tiled VAE decode (reduces memory consumption with some performance penalty).")
     pil_compress_level:             int = Field(default=1,                  description="The compress_level setting of PIL.Image.save(), used for PNG encoding. All settings are lossless. 0 = no compression, 1 = fastest with slightly larger filesize, 9 = slowest with smallest filesize. 1 is typically the best setting.")
     max_queue_size:                 int = Field(default=10000, gt=0,        description="Maximum number of items in the session queue.")
-    clear_queue_on_startup:        bool = Field(default=False,              description="Empties session queue on startup.")
 
     # NODES
     allow_nodes:    Optional[list[str]] = Field(default=None,               description="List of nodes to allow. Omit to allow all.")
@@ -305,11 +303,6 @@ class InvokeAIAppConfig(BaseSettings):
         """Path to the converted cache models directory, resolved to an absolute path.."""
         return self._resolve(self.convert_cache_dir)
 
-    @property
-    def download_cache_path(self) -> Path:
-        """Path to the downloaded models directory, resolved to an absolute path.."""
-        return self._resolve(self.download_cache_dir)
-
     @property
     def custom_nodes_path(self) -> Path:
         """Path to the custom nodes directory, resolved to an absolute path.."""
@@ -355,14 +348,14 @@ class DefaultInvokeAIAppConfig(InvokeAIAppConfig):
         return (init_settings,)
 
 
-def migrate_v3_config_dict(config_dict: dict[str, Any]) -> dict[str, Any]:
-    """Migrate a v3 config dictionary to a v4.0.0.
+def migrate_v3_config_dict(config_dict: dict[str, Any]) -> InvokeAIAppConfig:
+    """Migrate a v3 config dictionary to a current config object.
 
     Args:
         config_dict: A dictionary of settings from a v3 config file.
 
     Returns:
-        An `InvokeAIAppConfig` config dict.
+        An instance of `InvokeAIAppConfig` with the migrated settings.
 
     """
     parsed_config_dict: dict[str, Any] = {}
@@ -396,41 +389,32 @@ def migrate_v3_config_dict(config_dict: dict[str, Any]) -> dict[str, Any]:
             elif k in InvokeAIAppConfig.model_fields:
                 # skip unknown fields
                 parsed_config_dict[k] = v
-    parsed_config_dict["schema_version"] = "4.0.0"
-    return parsed_config_dict
+    # When migrating the config file, we should not include currently-set environment variables.
+    config = DefaultInvokeAIAppConfig.model_validate(parsed_config_dict)
+
+    return config
 
 
-def migrate_v4_0_0_to_4_0_1_config_dict(config_dict: dict[str, Any]) -> dict[str, Any]:
-    """Migrate v4.0.0 config dictionary to a v4.0.1 config dictionary
+def migrate_v4_0_0_config_dict(config_dict: dict[str, Any]) -> InvokeAIAppConfig:
+    """Migrate v4.0.0 config dictionary to a current config object.
 
     Args:
         config_dict: A dictionary of settings from a v4.0.0 config file.
 
     Returns:
-        A config dict with the settings migrated to v4.0.1.
+        An instance of `InvokeAIAppConfig` with the migrated settings.
     """
-    parsed_config_dict: dict[str, Any] = copy.deepcopy(config_dict)
-    # precision "autocast" was replaced by "auto" in v4.0.1
-    if parsed_config_dict.get("precision") == "autocast":
-        parsed_config_dict["precision"] = "auto"
-    parsed_config_dict["schema_version"] = "4.0.1"
-    return parsed_config_dict
-
-
-def migrate_v4_0_1_to_4_0_2_config_dict(config_dict: dict[str, Any]) -> dict[str, Any]:
-    """Migrate v4.0.1 config dictionary to a v4.0.2 config dictionary.
-
-    Args:
-        config_dict: A dictionary of settings from a v4.0.1 config file.
-
-    Returns:
-        An config dict with the settings migrated to v4.0.2.
-    """
-    parsed_config_dict: dict[str, Any] = copy.deepcopy(config_dict)
-    # convert_cache was removed in 4.0.2
-    parsed_config_dict.pop("convert_cache", None)
-    parsed_config_dict["schema_version"] = "4.0.2"
-    return parsed_config_dict
+    parsed_config_dict: dict[str, Any] = {}
+    for k, v in config_dict.items():
+        # autocast was removed from precision in v4.0.1
+        if k == "precision" and v == "autocast":
+            parsed_config_dict["precision"] = "auto"
+        else:
+            parsed_config_dict[k] = v
+        if k == "schema_version":
+            parsed_config_dict[k] = CONFIG_SCHEMA_VERSION
+    config = DefaultInvokeAIAppConfig.model_validate(parsed_config_dict)
+    return config
 
 
 def load_and_migrate_config(config_path: Path) -> InvokeAIAppConfig:
@@ -444,31 +428,27 @@ def load_and_migrate_config(config_path: Path) -> InvokeAIAppConfig:
     """
     assert config_path.suffix == ".yaml"
     with open(config_path, "rt", encoding=locale.getpreferredencoding()) as file:
-        loaded_config_dict: dict[str, Any] = yaml.safe_load(file)
+        loaded_config_dict = yaml.safe_load(file)
 
     assert isinstance(loaded_config_dict, dict)
 
-    migrated = False
     if "InvokeAI" in loaded_config_dict:
-        migrated = True
-        loaded_config_dict = migrate_v3_config_dict(loaded_config_dict)  # pyright: ignore [reportUnknownArgumentType]
-    if loaded_config_dict["schema_version"] == "4.0.0":
-        migrated = True
-        loaded_config_dict = migrate_v4_0_0_to_4_0_1_config_dict(loaded_config_dict)
-    if loaded_config_dict["schema_version"] == "4.0.1":
-        migrated = True
-        loaded_config_dict = migrate_v4_0_1_to_4_0_2_config_dict(loaded_config_dict)
-
-    if migrated:
+        # This is a v3 config file, attempt to migrate it
         shutil.copy(config_path, config_path.with_suffix(".yaml.bak"))
         try:
-            # load and write without environment variables
-            migrated_config = DefaultInvokeAIAppConfig.model_validate(loaded_config_dict)
-            migrated_config.write_file(config_path)
+            # loaded_config_dict could be the wrong shape, but we will catch all exceptions below
+            migrated_config = migrate_v3_config_dict(loaded_config_dict)  # pyright: ignore [reportUnknownArgumentType]
         except Exception as e:
             shutil.copy(config_path.with_suffix(".yaml.bak"), config_path)
             raise RuntimeError(f"Failed to load and migrate v3 config file {config_path}: {e}") from e
+        migrated_config.write_file(config_path)
+        return migrated_config
 
+    if loaded_config_dict["schema_version"] == "4.0.0":
+        loaded_config_dict = migrate_v4_0_0_config_dict(loaded_config_dict)
+        loaded_config_dict.write_file(config_path)
+
+    # Attempt to load as a v4 config file
     try:
         # Meta is not included in the model fields, so we need to validate it separately
         config = InvokeAIAppConfig.model_validate(loaded_config_dict)

invokeai/app/services/download/__init__.py

@@ -1,17 +1,10 @@
 """Init file for download queue."""
 
-from invokeai.app.services.download.download_base import (
-    DownloadJob,
-    DownloadJobStatus,
-    DownloadQueueServiceBase,
-    MultiFileDownloadJob,
-    UnknownJobIDException,
-)
-from invokeai.app.services.download.download_default import DownloadQueueService, TqdmProgress
+from .download_base import DownloadJob, DownloadJobStatus, DownloadQueueServiceBase, UnknownJobIDException
+from .download_default import DownloadQueueService, TqdmProgress
 
 __all__ = [
     "DownloadJob",
-    "MultiFileDownloadJob",
     "DownloadQueueServiceBase",
     "DownloadQueueService",
     "TqdmProgress",

invokeai/app/services/download/download_base.py

@@ -5,13 +5,11 @@ from abc import ABC, abstractmethod
 from enum import Enum
 from functools import total_ordering
 from pathlib import Path
-from typing import Any, Callable, List, Optional, Set, Union
+from typing import Any, Callable, List, Optional
 
 from pydantic import BaseModel, Field, PrivateAttr
 from pydantic.networks import AnyHttpUrl
 
-from invokeai.backend.model_manager.metadata import RemoteModelFile
-
 
 class DownloadJobStatus(str, Enum):
     """State of a download job."""
@@ -35,23 +33,30 @@ class ServiceInactiveException(Exception):
     """This exception is raised when user attempts to initiate a download before the service is started."""
 
 
-SingleFileDownloadEventHandler = Callable[["DownloadJob"], None]
-SingleFileDownloadExceptionHandler = Callable[["DownloadJob", Optional[Exception]], None]
-MultiFileDownloadEventHandler = Callable[["MultiFileDownloadJob"], None]
-MultiFileDownloadExceptionHandler = Callable[["MultiFileDownloadJob", Optional[Exception]], None]
-DownloadEventHandler = Union[SingleFileDownloadEventHandler, MultiFileDownloadEventHandler]
-DownloadExceptionHandler = Union[SingleFileDownloadExceptionHandler, MultiFileDownloadExceptionHandler]
+DownloadEventHandler = Callable[["DownloadJob"], None]
+DownloadExceptionHandler = Callable[["DownloadJob", Optional[Exception]], None]
 
 
-class DownloadJobBase(BaseModel):
-    """Base of classes to monitor and control downloads."""
+@total_ordering
+class DownloadJob(BaseModel):
+    """Class to monitor and control a model download request."""
 
+    # required variables to be passed in on creation
+    source: AnyHttpUrl = Field(description="Where to download from. Specific types specified in child classes.")
+    dest: Path = Field(description="Destination of downloaded model on local disk; a directory or file path")
+    access_token: Optional[str] = Field(default=None, description="authorization token for protected resources")
     # automatically assigned on creation
     id: int = Field(description="Numeric ID of this job", default=-1)  # default id is a sentinel
+    priority: int = Field(default=10, description="Queue priority; lower values are higher priority")
 
-    dest: Path = Field(description="Initial destination of downloaded model on local disk; a directory or file path")
-    download_path: Optional[Path] = Field(default=None, description="Final location of downloaded file or directory")
+    # set internally during download process
     status: DownloadJobStatus = Field(default=DownloadJobStatus.WAITING, description="Status of the download")
+    download_path: Optional[Path] = Field(default=None, description="Final location of downloaded file")
+    job_started: Optional[str] = Field(default=None, description="Timestamp for when the download job started")
+    job_ended: Optional[str] = Field(
+        default=None, description="Timestamp for when the download job ende1d (completed or errored)"
+    )
+    content_type: Optional[str] = Field(default=None, description="Content type of downloaded file")
     bytes: int = Field(default=0, description="Bytes downloaded so far")
     total_bytes: int = Field(default=0, description="Total file size (bytes)")
 
@@ -69,6 +74,14 @@ class DownloadJobBase(BaseModel):
     _on_cancelled: Optional[DownloadEventHandler] = PrivateAttr(default=None)
     _on_error: Optional[DownloadExceptionHandler] = PrivateAttr(default=None)
 
+    def __hash__(self) -> int:
+        """Return hash of the string representation of this object, for indexing."""
+        return hash(str(self))
+
+    def __le__(self, other: "DownloadJob") -> bool:
+        """Return True if this job's priority is less than another's."""
+        return self.priority <= other.priority
+
     def cancel(self) -> None:
         """Call to cancel the job."""
         self._cancelled = True
@@ -85,11 +98,6 @@ class DownloadJobBase(BaseModel):
         """Return true if job completed without errors."""
         return self.status == DownloadJobStatus.COMPLETED
 
-    @property
-    def waiting(self) -> bool:
-        """Return true if the job is waiting to run."""
-        return self.status == DownloadJobStatus.WAITING
-
     @property
     def running(self) -> bool:
         """Return true if the job is running."""
@@ -146,37 +154,6 @@ class DownloadJobBase(BaseModel):
         self._on_cancelled = on_cancelled
 
 
-@total_ordering
-class DownloadJob(DownloadJobBase):
-    """Class to monitor and control a model download request."""
-
-    # required variables to be passed in on creation
-    source: AnyHttpUrl = Field(description="Where to download from. Specific types specified in child classes.")
-    access_token: Optional[str] = Field(default=None, description="authorization token for protected resources")
-    priority: int = Field(default=10, description="Queue priority; lower values are higher priority")
-
-    # set internally during download process
-    job_started: Optional[str] = Field(default=None, description="Timestamp for when the download job started")
-    job_ended: Optional[str] = Field(
-        default=None, description="Timestamp for when the download job ende1d (completed or errored)"
-    )
-    content_type: Optional[str] = Field(default=None, description="Content type of downloaded file")
-
-    def __hash__(self) -> int:
-        """Return hash of the string representation of this object, for indexing."""
-        return hash(str(self))
-
-    def __le__(self, other: "DownloadJob") -> bool:
-        """Return True if this job's priority is less than another's."""
-        return self.priority <= other.priority
-
-
-class MultiFileDownloadJob(DownloadJobBase):
-    """Class to monitor and control multifile downloads."""
-
-    download_parts: Set[DownloadJob] = Field(default_factory=set, description="List of download parts.")
-
-
 class DownloadQueueServiceBase(ABC):
     """Multithreaded queue for downloading models via URL."""
 
@@ -224,48 +201,6 @@ class DownloadQueueServiceBase(ABC):
         """
         pass
 
-    @abstractmethod
-    def multifile_download(
-        self,
-        parts: List[RemoteModelFile],
-        dest: Path,
-        access_token: Optional[str] = None,
-        submit_job: bool = True,
-        on_start: Optional[DownloadEventHandler] = None,
-        on_progress: Optional[DownloadEventHandler] = None,
-        on_complete: Optional[DownloadEventHandler] = None,
-        on_cancelled: Optional[DownloadEventHandler] = None,
-        on_error: Optional[DownloadExceptionHandler] = None,
-    ) -> MultiFileDownloadJob:
-        """
-        Create and enqueue a multifile download job.
-
-        :param parts: Set of URL / filename pairs
-        :param dest: Path to download to. See below.
-        :param access_token: Access token to download the indicated files. If not provided,
-         each file's URL may be matched to an access token using the config file matching
-         system.
-        :param submit_job: If true [default] then submit the job for execution. Otherwise,
-         you will need to pass the job to submit_multifile_download().
-        :param on_start, on_progress, on_complete, on_error: Callbacks for the indicated
-         events.
-        :returns: A MultiFileDownloadJob object for monitoring the state of the download.
-
-        The `dest` argument is a Path object pointing to a directory. All downloads
-        with be placed inside this directory. The callbacks will receive the
-        MultiFileDownloadJob.
-        """
-        pass
-
-    @abstractmethod
-    def submit_multifile_download(self, job: MultiFileDownloadJob) -> None:
-        """
-        Enqueue a previously-created multi-file download job.
-
-        :param job: A MultiFileDownloadJob created with multifile_download()
-        """
-        pass
-
     @abstractmethod
     def submit_download_job(
         self,
@@ -317,7 +252,7 @@ class DownloadQueueServiceBase(ABC):
         pass
 
     @abstractmethod
-    def cancel_job(self, job: DownloadJobBase) -> None:
+    def cancel_job(self, job: DownloadJob) -> None:
         """Cancel the job, clearing partial downloads and putting it into ERROR state."""
         pass
 
@@ -327,7 +262,7 @@ class DownloadQueueServiceBase(ABC):
         pass
 
     @abstractmethod
-    def wait_for_job(self, job: DownloadJobBase, timeout: int = 0) -> DownloadJobBase:
+    def wait_for_job(self, job: DownloadJob, timeout: int = 0) -> DownloadJob:
         """Wait until the indicated download job has reached a terminal state.
 
         This will block until the indicated install job has completed,

invokeai/app/services/download/download_default.py

@@ -8,30 +8,29 @@ import time
 import traceback
 from pathlib import Path
 from queue import Empty, PriorityQueue
-from typing import Any, Dict, List, Literal, Optional, Set
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set
 
 import requests
 from pydantic.networks import AnyHttpUrl
 from requests import HTTPError
 from tqdm import tqdm
 
-from invokeai.app.services.config import InvokeAIAppConfig, get_config
-from invokeai.app.services.download.download_base import (
+from invokeai.app.util.misc import get_iso_timestamp
+from invokeai.backend.util.logging import InvokeAILogger
+
+from .download_base import (
     DownloadEventHandler,
     DownloadExceptionHandler,
     DownloadJob,
-    DownloadJobBase,
     DownloadJobCancelledException,
     DownloadJobStatus,
     DownloadQueueServiceBase,
-    MultiFileDownloadJob,
     ServiceInactiveException,
     UnknownJobIDException,
 )
-from invokeai.app.services.events.events_base import EventServiceBase
-from invokeai.app.util.misc import get_iso_timestamp
-from invokeai.backend.model_manager.metadata import RemoteModelFile
-from invokeai.backend.util.logging import InvokeAILogger
+
+if TYPE_CHECKING:
+    from invokeai.app.services.events.events_base import EventServiceBase
 
 # Maximum number of bytes to download during each call to requests.iter_content()
 DOWNLOAD_CHUNK_SIZE = 100000
@@ -43,24 +42,20 @@ class DownloadQueueService(DownloadQueueServiceBase):
     def __init__(
         self,
         max_parallel_dl: int = 5,
-        app_config: Optional[InvokeAIAppConfig] = None,
         event_bus: Optional["EventServiceBase"] = None,
         requests_session: Optional[requests.sessions.Session] = None,
     ):
         """
         Initialize DownloadQueue.
 
-        :param app_config: InvokeAIAppConfig object
         :param max_parallel_dl: Number of simultaneous downloads allowed [5].
         :param requests_session: Optional requests.sessions.Session object, for unit tests.
         """
-        self._app_config = app_config or get_config()
         self._jobs: Dict[int, DownloadJob] = {}
-        self._download_part2parent: Dict[AnyHttpUrl, MultiFileDownloadJob] = {}
         self._next_job_id = 0
         self._queue: PriorityQueue[DownloadJob] = PriorityQueue()
         self._stop_event = threading.Event()
-        self._job_terminated_event = threading.Event()
+        self._job_completed_event = threading.Event()
         self._worker_pool: Set[threading.Thread] = set()
         self._lock = threading.Lock()
         self._logger = InvokeAILogger.get_logger("DownloadQueueService")
@@ -112,16 +107,18 @@ class DownloadQueueService(DownloadQueueServiceBase):
             raise ServiceInactiveException(
                 "The download service is not currently accepting requests. Please call start() to initialize the service."
             )
-        job.id = self._next_id()
-        job.set_callbacks(
-            on_start=on_start,
-            on_progress=on_progress,
-            on_complete=on_complete,
-            on_cancelled=on_cancelled,
-            on_error=on_error,
-        )
-        self._jobs[job.id] = job
-        self._queue.put(job)
+        with self._lock:
+            job.id = self._next_job_id
+            self._next_job_id += 1
+            job.set_callbacks(
+                on_start=on_start,
+                on_progress=on_progress,
+                on_complete=on_complete,
+                on_cancelled=on_cancelled,
+                on_error=on_error,
+            )
+            self._jobs[job.id] = job
+            self._queue.put(job)
 
     def download(
         self,
@@ -144,7 +141,7 @@ class DownloadQueueService(DownloadQueueServiceBase):
             source=source,
             dest=dest,
             priority=priority,
-            access_token=access_token or self._lookup_access_token(source),
+            access_token=access_token,
         )
         self.submit_download_job(
             job,
@@ -156,63 +153,10 @@ class DownloadQueueService(DownloadQueueServiceBase):
         )
         return job
 
-    def multifile_download(
-        self,
-        parts: List[RemoteModelFile],
-        dest: Path,
-        access_token: Optional[str] = None,
-        submit_job: bool = True,
-        on_start: Optional[DownloadEventHandler] = None,
-        on_progress: Optional[DownloadEventHandler] = None,
-        on_complete: Optional[DownloadEventHandler] = None,
-        on_cancelled: Optional[DownloadEventHandler] = None,
-        on_error: Optional[DownloadExceptionHandler] = None,
-    ) -> MultiFileDownloadJob:
-        mfdj = MultiFileDownloadJob(dest=dest, id=self._next_id())
-        mfdj.set_callbacks(
-            on_start=on_start,
-            on_progress=on_progress,
-            on_complete=on_complete,
-            on_cancelled=on_cancelled,
-            on_error=on_error,
-        )
-
-        for part in parts:
-            url = part.url
-            path = dest / part.path
-            assert path.is_relative_to(dest), "only relative download paths accepted"
-            job = DownloadJob(
-                source=url,
-                dest=path,
-                access_token=access_token or self._lookup_access_token(url),
-            )
-            mfdj.download_parts.add(job)
-            self._download_part2parent[job.source] = mfdj
-        if submit_job:
-            self.submit_multifile_download(mfdj)
-        return mfdj
-
-    def submit_multifile_download(self, job: MultiFileDownloadJob) -> None:
-        for download_job in job.download_parts:
-            self.submit_download_job(
-                download_job,
-                on_start=self._mfd_started,
-                on_progress=self._mfd_progress,
-                on_complete=self._mfd_complete,
-                on_cancelled=self._mfd_cancelled,
-                on_error=self._mfd_error,
-            )
-
     def join(self) -> None:
         """Wait for all jobs to complete."""
         self._queue.join()
 
-    def _next_id(self) -> int:
-        with self._lock:
-            id = self._next_job_id
-            self._next_job_id += 1
-        return id
-
     def list_jobs(self) -> List[DownloadJob]:
         """List all the jobs."""
         return list(self._jobs.values())
@@ -234,14 +178,14 @@ class DownloadQueueService(DownloadQueueServiceBase):
         except KeyError as excp:
             raise UnknownJobIDException("Unrecognized job") from excp
 
-    def cancel_job(self, job: DownloadJobBase) -> None:
+    def cancel_job(self, job: DownloadJob) -> None:
         """
         Cancel the indicated job.
 
         If it is running it will be stopped.
         job.status will be set to DownloadJobStatus.CANCELLED
         """
-        if job.status in [DownloadJobStatus.WAITING, DownloadJobStatus.RUNNING]:
+        with self._lock:
             job.cancel()
 
     def cancel_all_jobs(self) -> None:
@@ -250,12 +194,12 @@ class DownloadQueueService(DownloadQueueServiceBase):
             if not job.in_terminal_state:
                 self.cancel_job(job)
 
-    def wait_for_job(self, job: DownloadJobBase, timeout: int = 0) -> DownloadJobBase:
+    def wait_for_job(self, job: DownloadJob, timeout: int = 0) -> DownloadJob:
         """Block until the indicated job has reached terminal state, or when timeout limit reached."""
         start = time.time()
         while not job.in_terminal_state:
-            if self._job_terminated_event.wait(timeout=0.25):  # in case we miss an event
-                self._job_terminated_event.clear()
+            if self._job_completed_event.wait(timeout=0.25):  # in case we miss an event
+                self._job_completed_event.clear()
             if timeout > 0 and time.time() - start > timeout:
                 raise TimeoutError("Timeout exceeded")
         return job
@@ -284,25 +228,22 @@ class DownloadQueueService(DownloadQueueServiceBase):
                 job.job_started = get_iso_timestamp()
                 self._do_download(job)
                 self._signal_job_complete(job)
-            except DownloadJobCancelledException:
-                self._signal_job_cancelled(job)
-                self._cleanup_cancelled_job(job)
-            except Exception as excp:
+            except (OSError, HTTPError) as excp:
                 job.error_type = excp.__class__.__name__ + f"({str(excp)})"
                 job.error = traceback.format_exc()
                 self._signal_job_error(job, excp)
+            except DownloadJobCancelledException:
+                self._signal_job_cancelled(job)
+                self._cleanup_cancelled_job(job)
+
             finally:
                 job.job_ended = get_iso_timestamp()
-                self._job_terminated_event.set()  # signal a change to terminal state
-                self._download_part2parent.pop(job.source, None)  # if this is a subpart of a multipart job, remove it
-                self._job_terminated_event.set()
+                self._job_completed_event.set()  # signal a change to terminal state
                 self._queue.task_done()
-
         self._logger.debug(f"Download queue worker thread {threading.current_thread().name} exiting.")
 
     def _do_download(self, job: DownloadJob) -> None:
         """Do the actual download."""
-
         url = job.source
         header = {"Authorization": f"Bearer {job.access_token}"} if job.access_token else {}
         open_mode = "wb"
@@ -394,29 +335,38 @@ class DownloadQueueService(DownloadQueueServiceBase):
     def _in_progress_path(self, path: Path) -> Path:
         return path.with_name(path.name + ".downloading")
 
-    def _lookup_access_token(self, source: AnyHttpUrl) -> Optional[str]:
-        # Pull the token from config if it exists and matches the URL
-        token = None
-        for pair in self._app_config.remote_api_tokens or []:
-            if re.search(pair.url_regex, str(source)):
-                token = pair.token
-                break
-        return token
-
     def _signal_job_started(self, job: DownloadJob) -> None:
         job.status = DownloadJobStatus.RUNNING
-        self._execute_cb(job, "on_start")
+        if job.on_start:
+            try:
+                job.on_start(job)
+            except Exception as e:
+                self._logger.error(
+                    f"An error occurred while processing the on_start callback: {traceback.format_exception(e)}"
+                )
         if self._event_bus:
             self._event_bus.emit_download_started(job)
 
     def _signal_job_progress(self, job: DownloadJob) -> None:
-        self._execute_cb(job, "on_progress")
+        if job.on_progress:
+            try:
+                job.on_progress(job)
+            except Exception as e:
+                self._logger.error(
+                    f"An error occurred while processing the on_progress callback: {traceback.format_exception(e)}"
+                )
         if self._event_bus:
             self._event_bus.emit_download_progress(job)
 
     def _signal_job_complete(self, job: DownloadJob) -> None:
         job.status = DownloadJobStatus.COMPLETED
-        self._execute_cb(job, "on_complete")
+        if job.on_complete:
+            try:
+                job.on_complete(job)
+            except Exception as e:
+                self._logger.error(
+                    f"An error occurred while processing the on_complete callback: {traceback.format_exception(e)}"
+                )
         if self._event_bus:
             self._event_bus.emit_download_complete(job)
 
@@ -424,21 +374,26 @@ class DownloadQueueService(DownloadQueueServiceBase):
         if job.status not in [DownloadJobStatus.RUNNING, DownloadJobStatus.WAITING]:
             return
         job.status = DownloadJobStatus.CANCELLED
-        self._execute_cb(job, "on_cancelled")
+        if job.on_cancelled:
+            try:
+                job.on_cancelled(job)
+            except Exception as e:
+                self._logger.error(
+                    f"An error occurred while processing the on_cancelled callback: {traceback.format_exception(e)}"
+                )
         if self._event_bus:
             self._event_bus.emit_download_cancelled(job)
 
-        # if multifile download, then signal the parent
-        if parent_job := self._download_part2parent.get(job.source, None):
-            if not parent_job.in_terminal_state:
-                parent_job.status = DownloadJobStatus.CANCELLED
-                self._execute_cb(parent_job, "on_cancelled")
-
     def _signal_job_error(self, job: DownloadJob, excp: Optional[Exception] = None) -> None:
         job.status = DownloadJobStatus.ERROR
         self._logger.error(f"{str(job.source)}: {traceback.format_exception(excp)}")
-        self._execute_cb(job, "on_error", excp)
-
+        if job.on_error:
+            try:
+                job.on_error(job, excp)
+            except Exception as e:
+                self._logger.error(
+                    f"An error occurred while processing the on_error callback: {traceback.format_exception(e)}"
+                )
         if self._event_bus:
             self._event_bus.emit_download_error(job)
 
@@ -451,97 +406,6 @@ class DownloadQueueService(DownloadQueueServiceBase):
         except OSError as excp:
             self._logger.warning(excp)
 
-    ########################################
-    # callbacks used for multifile downloads
-    ########################################
-    def _mfd_started(self, download_job: DownloadJob) -> None:
-        self._logger.info(f"File download started: {download_job.source}")
-        with self._lock:
-            mf_job = self._download_part2parent[download_job.source]
-            if mf_job.waiting:
-                mf_job.total_bytes = sum(x.total_bytes for x in mf_job.download_parts)
-                mf_job.status = DownloadJobStatus.RUNNING
-                assert download_job.download_path is not None
-                path_relative_to_destdir = download_job.download_path.relative_to(mf_job.dest)
-                mf_job.download_path = (
-                    mf_job.dest / path_relative_to_destdir.parts[0]
-                )  # keep just the first component of the path
-                self._execute_cb(mf_job, "on_start")
-
-    def _mfd_progress(self, download_job: DownloadJob) -> None:
-        with self._lock:
-            mf_job = self._download_part2parent[download_job.source]
-            if mf_job.cancelled:
-                for part in mf_job.download_parts:
-                    self.cancel_job(part)
-            elif mf_job.running:
-                mf_job.total_bytes = sum(x.total_bytes for x in mf_job.download_parts)
-                mf_job.bytes = sum(x.total_bytes for x in mf_job.download_parts)
-                self._execute_cb(mf_job, "on_progress")
-
-    def _mfd_complete(self, download_job: DownloadJob) -> None:
-        self._logger.info(f"Download complete: {download_job.source}")
-        with self._lock:
-            mf_job = self._download_part2parent[download_job.source]
-
-            # are there any more active jobs left in this task?
-            if mf_job.running and all(x.complete for x in mf_job.download_parts):
-                mf_job.status = DownloadJobStatus.COMPLETED
-                self._execute_cb(mf_job, "on_complete")
-
-            # we're done with this sub-job
-            self._job_terminated_event.set()
-
-    def _mfd_cancelled(self, download_job: DownloadJob) -> None:
-        with self._lock:
-            mf_job = self._download_part2parent[download_job.source]
-            assert mf_job is not None
-
-            if not mf_job.in_terminal_state:
-                self._logger.warning(f"Download cancelled: {download_job.source}")
-                mf_job.cancel()
-
-            for s in mf_job.download_parts:
-                self.cancel_job(s)
-
-    def _mfd_error(self, download_job: DownloadJob, excp: Optional[Exception] = None) -> None:
-        with self._lock:
-            mf_job = self._download_part2parent[download_job.source]
-            assert mf_job is not None
-            if not mf_job.in_terminal_state:
-                mf_job.status = download_job.status
-                mf_job.error = download_job.error
-                mf_job.error_type = download_job.error_type
-                self._execute_cb(mf_job, "on_error", excp)
-                self._logger.error(
-                    f"Cancelling {mf_job.dest} due to an error while downloading {download_job.source}: {str(excp)}"
-                )
-                for s in [x for x in mf_job.download_parts if x.running]:
-                    self.cancel_job(s)
-                self._download_part2parent.pop(download_job.source)
-                self._job_terminated_event.set()
-
-    def _execute_cb(
-        self,
-        job: DownloadJob | MultiFileDownloadJob,
-        callback_name: Literal[
-            "on_start",
-            "on_progress",
-            "on_complete",
-            "on_cancelled",
-            "on_error",
-        ],
-        excp: Optional[Exception] = None,
-    ) -> None:
-        if callback := getattr(job, callback_name, None):
-            args = [job, excp] if excp else [job]
-            try:
-                callback(*args)
-            except Exception as e:
-                self._logger.error(
-                    f"An error occurred while processing the {callback_name} callback: {traceback.format_exception(e)}"
-                )
-
 
 def get_pc_name_max(directory: str) -> int:
     if hasattr(os, "pathconf"):

invokeai/app/services/events/events_base.py

@@ -22,7 +22,6 @@ from invokeai.app.services.events.events_common import (
     ModelInstallCompleteEvent,
     ModelInstallDownloadProgressEvent,
     ModelInstallDownloadsCompleteEvent,
-    ModelInstallDownloadStartedEvent,
     ModelInstallErrorEvent,
     ModelInstallStartedEvent,
     ModelLoadCompleteEvent,
@@ -35,6 +34,7 @@ from invokeai.backend.stable_diffusion.diffusers_pipeline import PipelineInterme
 if TYPE_CHECKING:
     from invokeai.app.invocations.baseinvocation import BaseInvocation, BaseInvocationOutput
     from invokeai.app.services.download.download_base import DownloadJob
+    from invokeai.app.services.events.events_common import EventBase
     from invokeai.app.services.model_install.model_install_common import ModelInstallJob
     from invokeai.app.services.session_processor.session_processor_common import ProgressImage
     from invokeai.app.services.session_queue.session_queue_common import (
@@ -145,10 +145,6 @@ class EventServiceBase:
 
     # region Model install
 
-    def emit_model_install_download_started(self, job: "ModelInstallJob") -> None:
-        """Emitted at intervals while the install job is started (remote models only)."""
-        self.dispatch(ModelInstallDownloadStartedEvent.build(job))
-
     def emit_model_install_download_progress(self, job: "ModelInstallJob") -> None:
         """Emitted at intervals while the install job is in progress (remote models only)."""
         self.dispatch(ModelInstallDownloadProgressEvent.build(job))

invokeai/app/services/events/events_common.py

@@ -417,42 +417,6 @@ class ModelLoadCompleteEvent(ModelEventBase):
         return cls(config=config, submodel_type=submodel_type)
 
 
-@payload_schema.register
-class ModelInstallDownloadStartedEvent(ModelEventBase):
-    """Event model for model_install_download_started"""
-
-    __event_name__ = "model_install_download_started"
-
-    id: int = Field(description="The ID of the install job")
-    source: str = Field(description="Source of the model; local path, repo_id or url")
-    local_path: str = Field(description="Where model is downloading to")
-    bytes: int = Field(description="Number of bytes downloaded so far")
-    total_bytes: int = Field(description="Total size of download, including all files")
-    parts: list[dict[str, int | str]] = Field(
-        description="Progress of downloading URLs that comprise the model, if any"
-    )
-
-    @classmethod
-    def build(cls, job: "ModelInstallJob") -> "ModelInstallDownloadStartedEvent":
-        parts: list[dict[str, str | int]] = [
-            {
-                "url": str(x.source),
-                "local_path": str(x.download_path),
-                "bytes": x.bytes,
-                "total_bytes": x.total_bytes,
-            }
-            for x in job.download_parts
-        ]
-        return cls(
-            id=job.id,
-            source=str(job.source),
-            local_path=job.local_path.as_posix(),
-            parts=parts,
-            bytes=job.bytes,
-            total_bytes=job.total_bytes,
-        )
-
-
 @payload_schema.register
 class ModelInstallDownloadProgressEvent(ModelEventBase):
     """Event model for model_install_download_progress"""

invokeai/app/services/events/events_fastapievents.py

@@ -6,11 +6,12 @@ from queue import Empty, Queue
 
 from fastapi_events.dispatcher import dispatch
 
-from invokeai.app.services.events.events_base import EventServiceBase
 from invokeai.app.services.events.events_common import (
     EventBase,
 )
 
+from .events_base import EventServiceBase
+
 
 class FastAPIEventService(EventServiceBase):
     def __init__(self, event_handler_id: int) -> None:

invokeai/app/services/image_files/image_files_disk.py

@@ -7,15 +7,12 @@ from PIL import Image, PngImagePlugin
 from PIL.Image import Image as PILImageType
 from send2trash import send2trash
 
-from invokeai.app.services.image_files.image_files_base import ImageFileStorageBase
-from invokeai.app.services.image_files.image_files_common import (
-    ImageFileDeleteException,
-    ImageFileNotFoundException,
-    ImageFileSaveException,
-)
 from invokeai.app.services.invoker import Invoker
 from invokeai.app.util.thumbnails import get_thumbnail_name, make_thumbnail
 
+from .image_files_base import ImageFileStorageBase
+from .image_files_common import ImageFileDeleteException, ImageFileNotFoundException, ImageFileSaveException
+
 
 class DiskImageFileStorage(ImageFileStorageBase):
     """Stores images on disk"""

invokeai/app/services/image_records/image_records_base.py

@@ -3,14 +3,9 @@ from datetime import datetime
 from typing import Optional
 
 from invokeai.app.invocations.fields import MetadataField
-from invokeai.app.services.image_records.image_records_common import (
-    ImageCategory,
-    ImageRecord,
-    ImageRecordChanges,
-    ResourceOrigin,
-)
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_common import SQLiteDirection
+
+from .image_records_common import ImageCategory, ImageRecord, ImageRecordChanges, ResourceOrigin
 
 
 class ImageRecordStorageBase(ABC):
@@ -42,13 +37,10 @@ class ImageRecordStorageBase(ABC):
         self,
         offset: int = 0,
         limit: int = 10,
-        starred_first: bool = True,
-        order_dir: SQLiteDirection = SQLiteDirection.Descending,
         image_origin: Optional[ResourceOrigin] = None,
         categories: Optional[list[ImageCategory]] = None,
         is_intermediate: Optional[bool] = None,
         board_id: Optional[str] = None,
-        search_term: Optional[str] = None,
     ) -> OffsetPaginatedResults[ImageRecord]:
         """Gets a page of image records."""
         pass

invokeai/app/services/image_records/image_records_sqlite.py

@@ -4,8 +4,11 @@ from datetime import datetime
 from typing import Optional, Union, cast
 
 from invokeai.app.invocations.fields import MetadataField, MetadataFieldValidator
-from invokeai.app.services.image_records.image_records_base import ImageRecordStorageBase
-from invokeai.app.services.image_records.image_records_common import (
+from invokeai.app.services.shared.pagination import OffsetPaginatedResults
+from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
+
+from .image_records_base import ImageRecordStorageBase
+from .image_records_common import (
     IMAGE_DTO_COLS,
     ImageCategory,
     ImageRecord,
@@ -16,9 +19,6 @@ from invokeai.app.services.image_records.image_records_common import (
     ResourceOrigin,
     deserialize_image_record,
 )
-from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_common import SQLiteDirection
-from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
 
 
 class SqliteImageRecordStorage(ImageRecordStorageBase):
@@ -144,13 +144,10 @@ class SqliteImageRecordStorage(ImageRecordStorageBase):
         self,
         offset: int = 0,
         limit: int = 10,
-        starred_first: bool = True,
-        order_dir: SQLiteDirection = SQLiteDirection.Descending,
         image_origin: Optional[ResourceOrigin] = None,
         categories: Optional[list[ImageCategory]] = None,
         is_intermediate: Optional[bool] = None,
         board_id: Optional[str] = None,
-        search_term: Optional[str] = None,
     ) -> OffsetPaginatedResults[ImageRecord]:
         try:
             self._lock.acquire()
@@ -211,21 +208,9 @@ class SqliteImageRecordStorage(ImageRecordStorageBase):
                 """
                 query_params.append(board_id)
 
-            # Search term condition
-            if search_term:
-                query_conditions += """--sql
-                AND images.metadata LIKE ?
-                """
-                query_params.append(f"%{search_term.lower()}%")
-
-            if starred_first:
-                query_pagination = f"""--sql
-                ORDER BY images.starred DESC, images.created_at {order_dir.value} LIMIT ? OFFSET ?
-                """
-            else:
-                query_pagination = f"""--sql
-                ORDER BY images.created_at {order_dir.value} LIMIT ? OFFSET ?
-                """
+            query_pagination = """--sql
+            ORDER BY images.starred DESC, images.created_at DESC LIMIT ? OFFSET ?
+            """
 
             # Final images query with pagination
             images_query += query_conditions + query_pagination + ";"

invokeai/app/services/images/images_base.py

@@ -12,7 +12,6 @@ from invokeai.app.services.image_records.image_records_common import (
 )
 from invokeai.app.services.images.images_common import ImageDTO
 from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_common import SQLiteDirection
 
 
 class ImageServiceABC(ABC):
@@ -117,13 +116,10 @@ class ImageServiceABC(ABC):
         self,
         offset: int = 0,
         limit: int = 10,
-        starred_first: bool = True,
-        order_dir: SQLiteDirection = SQLiteDirection.Descending,
         image_origin: Optional[ResourceOrigin] = None,
         categories: Optional[list[ImageCategory]] = None,
         is_intermediate: Optional[bool] = None,
         board_id: Optional[str] = None,
-        search_term: Optional[str] = None,
     ) -> OffsetPaginatedResults[ImageDTO]:
         """Gets a paginated list of image DTOs."""
         pass

invokeai/app/services/images/images_default.py

@@ -3,12 +3,15 @@ from typing import Optional
 from PIL.Image import Image as PILImageType
 
 from invokeai.app.invocations.fields import MetadataField
-from invokeai.app.services.image_files.image_files_common import (
+from invokeai.app.services.invoker import Invoker
+from invokeai.app.services.shared.pagination import OffsetPaginatedResults
+
+from ..image_files.image_files_common import (
     ImageFileDeleteException,
     ImageFileNotFoundException,
     ImageFileSaveException,
 )
-from invokeai.app.services.image_records.image_records_common import (
+from ..image_records.image_records_common import (
     ImageCategory,
     ImageRecord,
     ImageRecordChanges,
@@ -19,11 +22,8 @@ from invokeai.app.services.image_records.image_records_common import (
     InvalidOriginException,
     ResourceOrigin,
 )
-from invokeai.app.services.images.images_base import ImageServiceABC
-from invokeai.app.services.images.images_common import ImageDTO, image_record_to_dto
-from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.shared.pagination import OffsetPaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_common import SQLiteDirection
+from .images_base import ImageServiceABC
+from .images_common import ImageDTO, image_record_to_dto
 
 
 class ImageService(ImageServiceABC):
@@ -73,12 +73,7 @@ class ImageService(ImageServiceABC):
                 session_id=session_id,
             )
             if board_id is not None:
-                try:
-                    self.__invoker.services.board_image_records.add_image_to_board(
-                        board_id=board_id, image_name=image_name
-                    )
-                except Exception as e:
-                    self.__invoker.services.logger.warn(f"Failed to add image to board {board_id}: {str(e)}")
+                self.__invoker.services.board_image_records.add_image_to_board(board_id=board_id, image_name=image_name)
             self.__invoker.services.image_files.save(
                 image_name=image_name, image=image, metadata=metadata, workflow=workflow, graph=graph
             )
@@ -207,25 +202,19 @@ class ImageService(ImageServiceABC):
         self,
         offset: int = 0,
         limit: int = 10,
-        starred_first: bool = True,
-        order_dir: SQLiteDirection = SQLiteDirection.Descending,
         image_origin: Optional[ResourceOrigin] = None,
         categories: Optional[list[ImageCategory]] = None,
         is_intermediate: Optional[bool] = None,
         board_id: Optional[str] = None,
-        search_term: Optional[str] = None,
     ) -> OffsetPaginatedResults[ImageDTO]:
         try:
             results = self.__invoker.services.image_records.get_many(
                 offset,
                 limit,
-                starred_first,
-                order_dir,
                 image_origin,
                 categories,
                 is_intermediate,
                 board_id,
-                search_term,
             )
 
             image_dtos = [

invokeai/app/services/invocation_services.py

@@ -10,28 +10,29 @@ if TYPE_CHECKING:
 
     import torch
 
-    from invokeai.app.services.board_image_records.board_image_records_base import BoardImageRecordStorageBase
-    from invokeai.app.services.board_images.board_images_base import BoardImagesServiceABC
-    from invokeai.app.services.board_records.board_records_base import BoardRecordStorageBase
-    from invokeai.app.services.boards.boards_base import BoardServiceABC
-    from invokeai.app.services.bulk_download.bulk_download_base import BulkDownloadBase
-    from invokeai.app.services.config import InvokeAIAppConfig
-    from invokeai.app.services.download import DownloadQueueServiceBase
-    from invokeai.app.services.events.events_base import EventServiceBase
-    from invokeai.app.services.image_files.image_files_base import ImageFileStorageBase
-    from invokeai.app.services.image_records.image_records_base import ImageRecordStorageBase
-    from invokeai.app.services.images.images_base import ImageServiceABC
-    from invokeai.app.services.invocation_cache.invocation_cache_base import InvocationCacheBase
-    from invokeai.app.services.invocation_stats.invocation_stats_base import InvocationStatsServiceBase
-    from invokeai.app.services.model_images.model_images_base import ModelImageFileStorageBase
-    from invokeai.app.services.model_manager.model_manager_base import ModelManagerServiceBase
-    from invokeai.app.services.names.names_base import NameServiceBase
-    from invokeai.app.services.session_processor.session_processor_base import SessionProcessorBase
-    from invokeai.app.services.session_queue.session_queue_base import SessionQueueBase
-    from invokeai.app.services.urls.urls_base import UrlServiceBase
-    from invokeai.app.services.workflow_records.workflow_records_base import WorkflowRecordsStorageBase
     from invokeai.backend.stable_diffusion.diffusion.conditioning_data import ConditioningFieldData
 
+    from .board_image_records.board_image_records_base import BoardImageRecordStorageBase
+    from .board_images.board_images_base import BoardImagesServiceABC
+    from .board_records.board_records_base import BoardRecordStorageBase
+    from .boards.boards_base import BoardServiceABC
+    from .bulk_download.bulk_download_base import BulkDownloadBase
+    from .config import InvokeAIAppConfig
+    from .download import DownloadQueueServiceBase
+    from .events.events_base import EventServiceBase
+    from .image_files.image_files_base import ImageFileStorageBase
+    from .image_records.image_records_base import ImageRecordStorageBase
+    from .images.images_base import ImageServiceABC
+    from .invocation_cache.invocation_cache_base import InvocationCacheBase
+    from .invocation_stats.invocation_stats_base import InvocationStatsServiceBase
+    from .model_images.model_images_base import ModelImageFileStorageBase
+    from .model_manager.model_manager_base import ModelManagerServiceBase
+    from .names.names_base import NameServiceBase
+    from .session_processor.session_processor_base import SessionProcessorBase
+    from .session_queue.session_queue_base import SessionQueueBase
+    from .urls.urls_base import UrlServiceBase
+    from .workflow_records.workflow_records_base import WorkflowRecordsStorageBase
+
 
 class InvocationServices:
     """Services that can be used by invocations"""

invokeai/app/services/invocation_stats/invocation_stats_default.py

@@ -9,8 +9,11 @@ import torch
 
 import invokeai.backend.util.logging as logger
 from invokeai.app.invocations.baseinvocation import BaseInvocation
-from invokeai.app.services.invocation_stats.invocation_stats_base import InvocationStatsServiceBase
-from invokeai.app.services.invocation_stats.invocation_stats_common import (
+from invokeai.app.services.invoker import Invoker
+from invokeai.backend.model_manager.load.model_cache import CacheStats
+
+from .invocation_stats_base import InvocationStatsServiceBase
+from .invocation_stats_common import (
     GESStatsNotFoundError,
     GraphExecutionStats,
     GraphExecutionStatsSummary,
@@ -19,8 +22,6 @@ from invokeai.app.services.invocation_stats.invocation_stats_common import (
     NodeExecutionStats,
     NodeExecutionStatsSummary,
 )
-from invokeai.app.services.invoker import Invoker
-from invokeai.backend.model_manager.load.model_cache import CacheStats
 
 # Size of 1GB in bytes.
 GB = 2**30

invokeai/app/services/invoker.py

@@ -1,7 +1,7 @@
 # Copyright (c) 2022 Kyle Schouviller (https://github.com/kyle0654)
 
 
-from invokeai.app.services.invocation_services import InvocationServices
+from .invocation_services import InvocationServices
 
 
 class Invoker:

invokeai/app/services/model_images/model_images_default.py

@@ -5,14 +5,15 @@ from PIL.Image import Image as PILImageType
 from send2trash import send2trash
 
 from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.model_images.model_images_base import ModelImageFileStorageBase
-from invokeai.app.services.model_images.model_images_common import (
+from invokeai.app.util.misc import uuid_string
+from invokeai.app.util.thumbnails import make_thumbnail
+
+from .model_images_base import ModelImageFileStorageBase
+from .model_images_common import (
     ModelImageFileDeleteException,
     ModelImageFileNotFoundException,
     ModelImageFileSaveException,
 )
-from invokeai.app.util.misc import uuid_string
-from invokeai.app.util.thumbnails import make_thumbnail
 
 
 class ModelImageFileStorageDisk(ModelImageFileStorageBase):

invokeai/app/services/model_install/__init__.py

@@ -1,7 +1,9 @@
 """Initialization file for model install service package."""
 
-from invokeai.app.services.model_install.model_install_base import ModelInstallServiceBase
-from invokeai.app.services.model_install.model_install_common import (
+from .model_install_base import (
+    ModelInstallServiceBase,
+)
+from .model_install_common import (
     HFModelSource,
     InstallStatus,
     LocalModelSource,
@@ -10,7 +12,7 @@ from invokeai.app.services.model_install.model_install_common import (
     UnknownInstallJobException,
     URLModelSource,
 )
-from invokeai.app.services.model_install.model_install_default import ModelInstallService
+from .model_install_default import ModelInstallService
 
 __all__ = [
     "ModelInstallServiceBase",

invokeai/app/services/model_install/model_install_base.py

@@ -13,7 +13,7 @@ from invokeai.app.services.events.events_base import EventServiceBase
 from invokeai.app.services.invoker import Invoker
 from invokeai.app.services.model_install.model_install_common import ModelInstallJob, ModelSource
 from invokeai.app.services.model_records import ModelRecordServiceBase
-from invokeai.backend.model_manager import AnyModelConfig
+from invokeai.backend.model_manager.config import AnyModelConfig
 
 
 class ModelInstallServiceBase(ABC):
@@ -243,11 +243,12 @@ class ModelInstallServiceBase(ABC):
         """
 
     @abstractmethod
-    def download_and_cache_model(self, source: str | AnyHttpUrl) -> Path:
+    def download_and_cache(self, source: Union[str, AnyHttpUrl], access_token: Optional[str] = None) -> Path:
         """
         Download the model file located at source to the models cache and return its Path.
 
-        :param source: A string representing a URL or repo_id.
+        :param source: A Url or a string that can be converted into one.
+        :param access_token: Optional access token to access restricted resources.
 
         The model file will be downloaded into the system-wide model cache
         (`models/.cache`) if it isn't already there. Note that the model cache

invokeai/app/services/model_install/model_install_common.py

@@ -8,7 +8,7 @@ from pydantic import BaseModel, Field, PrivateAttr, field_validator
 from pydantic.networks import AnyHttpUrl
 from typing_extensions import Annotated
 
-from invokeai.app.services.download import DownloadJob, MultiFileDownloadJob
+from invokeai.app.services.download import DownloadJob
 from invokeai.backend.model_manager import AnyModelConfig, ModelRepoVariant
 from invokeai.backend.model_manager.config import ModelSourceType
 from invokeai.backend.model_manager.metadata import AnyModelRepoMetadata
@@ -26,6 +26,13 @@ class InstallStatus(str, Enum):
     CANCELLED = "cancelled"  # terminated with an error message
 
 
+class ModelInstallPart(BaseModel):
+    url: AnyHttpUrl
+    path: Path
+    bytes: int = 0
+    total_bytes: int = 0
+
+
 class UnknownInstallJobException(Exception):
     """Raised when the status of an unknown job is requested."""
 
@@ -162,7 +169,6 @@ class ModelInstallJob(BaseModel):
     )
     # internal flags and transitory settings
     _install_tmpdir: Optional[Path] = PrivateAttr(default=None)
-    _multifile_job: Optional[MultiFileDownloadJob] = PrivateAttr(default=None)
     _exception: Optional[Exception] = PrivateAttr(default=None)
 
     def set_error(self, e: Exception) -> None:

invokeai/app/services/model_install/model_install_default.py

@@ -5,34 +5,23 @@ import os
 import re
 import threading
 import time
+from hashlib import sha256
 from pathlib import Path
 from queue import Empty, Queue
 from shutil import copyfile, copytree, move, rmtree
 from tempfile import mkdtemp
-from typing import Any, Dict, List, Optional, Tuple, Type, Union
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Union
 
 import torch
 import yaml
 from huggingface_hub import HfFolder
 from pydantic.networks import AnyHttpUrl
-from pydantic_core import Url
 from requests import Session
 
 from invokeai.app.services.config import InvokeAIAppConfig
-from invokeai.app.services.download import DownloadQueueServiceBase, MultiFileDownloadJob
-from invokeai.app.services.events.events_base import EventServiceBase
+from invokeai.app.services.download import DownloadJob, DownloadQueueServiceBase, TqdmProgress
 from invokeai.app.services.invoker import Invoker
 from invokeai.app.services.model_install.model_install_base import ModelInstallServiceBase
-from invokeai.app.services.model_install.model_install_common import (
-    MODEL_SOURCE_TO_TYPE_MAP,
-    HFModelSource,
-    InstallStatus,
-    LocalModelSource,
-    ModelInstallJob,
-    ModelSource,
-    StringLikeSource,
-    URLModelSource,
-)
 from invokeai.app.services.model_records import DuplicateModelException, ModelRecordServiceBase
 from invokeai.app.services.model_records.model_records_base import ModelRecordChanges
 from invokeai.backend.model_manager.config import (
@@ -55,10 +44,23 @@ from invokeai.backend.model_manager.search import ModelSearch
 from invokeai.backend.util import InvokeAILogger
 from invokeai.backend.util.catch_sigint import catch_sigint
 from invokeai.backend.util.devices import TorchDevice
-from invokeai.backend.util.util import slugify
+
+from .model_install_common import (
+    MODEL_SOURCE_TO_TYPE_MAP,
+    HFModelSource,
+    InstallStatus,
+    LocalModelSource,
+    ModelInstallJob,
+    ModelSource,
+    StringLikeSource,
+    URLModelSource,
+)
 
 TMPDIR_PREFIX = "tmpinstall_"
 
+if TYPE_CHECKING:
+    from invokeai.app.services.events.events_base import EventServiceBase
+
 
 class ModelInstallService(ModelInstallServiceBase):
     """class for InvokeAI model installation."""
@@ -89,7 +91,7 @@ class ModelInstallService(ModelInstallServiceBase):
         self._downloads_changed_event = threading.Event()
         self._install_completed_event = threading.Event()
         self._download_queue = download_queue
-        self._download_cache: Dict[int, ModelInstallJob] = {}
+        self._download_cache: Dict[AnyHttpUrl, ModelInstallJob] = {}
         self._running = False
         self._session = session
         self._install_thread: Optional[threading.Thread] = None
@@ -208,12 +210,33 @@ class ModelInstallService(ModelInstallServiceBase):
         access_token: Optional[str] = None,
         inplace: Optional[bool] = False,
     ) -> ModelInstallJob:
-        """Install a model using pattern matching to infer the type of source."""
-        source_obj = self._guess_source(source)
-        if isinstance(source_obj, LocalModelSource):
-            source_obj.inplace = inplace
-        elif isinstance(source_obj, HFModelSource) or isinstance(source_obj, URLModelSource):
-            source_obj.access_token = access_token
+        variants = "|".join(ModelRepoVariant.__members__.values())
+        hf_repoid_re = f"^([^/:]+/[^/:]+)(?::({variants})?(?::/?([^:]+))?)?$"
+        source_obj: Optional[StringLikeSource] = None
+
+        if Path(source).exists():  # A local file or directory
+            source_obj = LocalModelSource(path=Path(source), inplace=inplace)
+        elif match := re.match(hf_repoid_re, source):
+            source_obj = HFModelSource(
+                repo_id=match.group(1),
+                variant=match.group(2) if match.group(2) else None,  # pass None rather than ''
+                subfolder=Path(match.group(3)) if match.group(3) else None,
+                access_token=access_token,
+            )
+        elif re.match(r"^https?://[^/]+", source):
+            # Pull the token from config if it exists and matches the URL
+            _token = access_token
+            if _token is None:
+                for pair in self.app_config.remote_api_tokens or []:
+                    if re.search(pair.url_regex, source):
+                        _token = pair.token
+                        break
+            source_obj = URLModelSource(
+                url=AnyHttpUrl(source),
+                access_token=_token,
+            )
+        else:
+            raise ValueError(f"Unsupported model source: '{source}'")
         return self.import_model(source_obj, config)
 
     def import_model(self, source: ModelSource, config: Optional[Dict[str, Any]] = None) -> ModelInstallJob:  # noqa D102
@@ -274,9 +297,8 @@ class ModelInstallService(ModelInstallServiceBase):
     def cancel_job(self, job: ModelInstallJob) -> None:
         """Cancel the indicated job."""
         job.cancel()
-        self._logger.warning(f"Cancelling {job.source}")
-        if dj := job._multifile_job:
-            self._download_queue.cancel_job(dj)
+        with self._lock:
+            self._cancel_download_parts(job)
 
     def prune_jobs(self) -> None:
         """Prune all completed and errored jobs."""
@@ -324,7 +346,7 @@ class ModelInstallService(ModelInstallServiceBase):
                     legacy_config_path = stanza.get("config")
                     if legacy_config_path:
                         # In v3, these paths were relative to the root. Migrate them to be relative to the legacy_conf_dir.
-                        legacy_config_path = self._app_config.root_path / legacy_config_path
+                        legacy_config_path: Path = self._app_config.root_path / legacy_config_path
                         if legacy_config_path.is_relative_to(self._app_config.legacy_conf_path):
                             legacy_config_path = legacy_config_path.relative_to(self._app_config.legacy_conf_path)
                         config["config_path"] = str(legacy_config_path)
@@ -364,95 +386,38 @@ class ModelInstallService(ModelInstallServiceBase):
             rmtree(model_path)
         self.unregister(key)
 
-    @classmethod
-    def _download_cache_path(cls, source: Union[str, AnyHttpUrl], app_config: InvokeAIAppConfig) -> Path:
-        escaped_source = slugify(str(source))
-        return app_config.download_cache_path / escaped_source
-
-    def download_and_cache_model(
+    def download_and_cache(
         self,
-        source: str | AnyHttpUrl,
+        source: Union[str, AnyHttpUrl],
+        access_token: Optional[str] = None,
+        timeout: int = 0,
     ) -> Path:
         """Download the model file located at source to the models cache and return its Path."""
-        model_path = self._download_cache_path(str(source), self._app_config)
+        model_hash = sha256(str(source).encode("utf-8")).hexdigest()[0:32]
+        model_path = self._app_config.convert_cache_path / model_hash
 
-        # We expect the cache directory to contain one and only one downloaded file or directory.
+        # We expect the cache directory to contain one and only one downloaded file.
         # We don't know the file's name in advance, as it is set by the download
         # content-disposition header.
         if model_path.exists():
-            contents: List[Path] = list(model_path.iterdir())
+            contents = [x for x in model_path.iterdir() if x.is_file()]
             if len(contents) > 0:
                 return contents[0]
 
         model_path.mkdir(parents=True, exist_ok=True)
-        model_source = self._guess_source(str(source))
-        remote_files, _ = self._remote_files_from_source(model_source)
-        job = self._multifile_download(
+        job = self._download_queue.download(
+            source=AnyHttpUrl(str(source)),
             dest=model_path,
-            remote_files=remote_files,
-            subfolder=model_source.subfolder if isinstance(model_source, HFModelSource) else None,
+            access_token=access_token,
+            on_progress=TqdmProgress().update,
         )
-        files_string = "file" if len(remote_files) == 1 else "files"
-        self._logger.info(f"Queuing model download: {source} ({len(remote_files)} {files_string})")
-        self._download_queue.wait_for_job(job)
+        self._download_queue.wait_for_job(job, timeout)
         if job.complete:
             assert job.download_path is not None
             return job.download_path
         else:
             raise Exception(job.error)
 
-    def _remote_files_from_source(
-        self, source: ModelSource
-    ) -> Tuple[List[RemoteModelFile], Optional[AnyModelRepoMetadata]]:
-        metadata = None
-        if isinstance(source, HFModelSource):
-            metadata = HuggingFaceMetadataFetch(self._session).from_id(source.repo_id, source.variant)
-            assert isinstance(metadata, ModelMetadataWithFiles)
-            return (
-                metadata.download_urls(
-                    variant=source.variant or self._guess_variant(),
-                    subfolder=source.subfolder,
-                    session=self._session,
-                ),
-                metadata,
-            )
-
-        if isinstance(source, URLModelSource):
-            try:
-                fetcher = self.get_fetcher_from_url(str(source.url))
-                kwargs: dict[str, Any] = {"session": self._session}
-                metadata = fetcher(**kwargs).from_url(source.url)
-                assert isinstance(metadata, ModelMetadataWithFiles)
-                return metadata.download_urls(session=self._session), metadata
-            except ValueError:
-                pass
-
-            return [RemoteModelFile(url=source.url, path=Path("."), size=0)], None
-
-        raise Exception(f"No files associated with {source}")
-
-    def _guess_source(self, source: str) -> ModelSource:
-        """Turn a source string into a ModelSource object."""
-        variants = "|".join(ModelRepoVariant.__members__.values())
-        hf_repoid_re = f"^([^/:]+/[^/:]+)(?::({variants})?(?::/?([^:]+))?)?$"
-        source_obj: Optional[StringLikeSource] = None
-
-        if Path(source).exists():  # A local file or directory
-            source_obj = LocalModelSource(path=Path(source))
-        elif match := re.match(hf_repoid_re, source):
-            source_obj = HFModelSource(
-                repo_id=match.group(1),
-                variant=ModelRepoVariant(match.group(2)) if match.group(2) else None,  # pass None rather than ''
-                subfolder=Path(match.group(3)) if match.group(3) else None,
-            )
-        elif re.match(r"^https?://[^/]+", source):
-            source_obj = URLModelSource(
-                url=Url(source),
-            )
-        else:
-            raise ValueError(f"Unsupported model source: '{source}'")
-        return source_obj
-
     # --------------------------------------------------------------------------------------------
     # Internal functions that manage the installer threads
     # --------------------------------------------------------------------------------------------
@@ -513,19 +478,16 @@ class ModelInstallService(ModelInstallServiceBase):
         job.config_out = self.record_store.get_model(key)
         self._signal_job_completed(job)
 
-    def _set_error(self, install_job: ModelInstallJob, excp: Exception) -> None:
-        multifile_download_job = install_job._multifile_job
-        if multifile_download_job and any(
-            x.content_type is not None and "text/html" in x.content_type for x in multifile_download_job.download_parts
-        ):
-            install_job.set_error(
+    def _set_error(self, job: ModelInstallJob, excp: Exception) -> None:
+        if any(x.content_type is not None and "text/html" in x.content_type for x in job.download_parts):
+            job.set_error(
                 InvalidModelConfigException(
-                    f"At least one file in {install_job.local_path} is an HTML page, not a model. This can happen when an access token is required to download."
+                    f"At least one file in {job.local_path} is an HTML page, not a model. This can happen when an access token is required to download."
                 )
             )
         else:
-            install_job.set_error(excp)
-        self._signal_job_errored(install_job)
+            job.set_error(excp)
+        self._signal_job_errored(job)
 
     # --------------------------------------------------------------------------------------------
     # Internal functions that manage the models directory
@@ -551,6 +513,7 @@ class ModelInstallService(ModelInstallServiceBase):
         This is typically only used during testing with a new DB or when using the memory DB, because those are the
         only situations in which we may have orphaned models in the models directory.
         """
+
         installed_model_paths = {
             (self._app_config.models_path / x.path).resolve() for x in self.record_store.all_models()
         }
@@ -562,13 +525,8 @@ class ModelInstallService(ModelInstallServiceBase):
             if resolved_path in installed_model_paths:
                 return True
             # Skip core models entirely - these aren't registered with the model manager.
-            for special_directory in [
-                self.app_config.models_path / "core",
-                self.app_config.convert_cache_dir,
-                self.app_config.download_cache_dir,
-            ]:
-                if resolved_path.is_relative_to(special_directory):
-                    return False
+            if str(resolved_path).startswith(str(self.app_config.models_path / "core")):
+                return False
             try:
                 model_id = self.register_path(model_path)
                 self._logger.info(f"Registered {model_path.name} with id {model_id}")
@@ -683,15 +641,20 @@ class ModelInstallService(ModelInstallServiceBase):
             inplace=source.inplace or False,
         )
 
-    def _import_from_hf(
-        self,
-        source: HFModelSource,
-        config: Optional[Dict[str, Any]] = None,
-    ) -> ModelInstallJob:
+    def _import_from_hf(self, source: HFModelSource, config: Optional[Dict[str, Any]]) -> ModelInstallJob:
         # Add user's cached access token to HuggingFace requests
-        if source.access_token is None:
-            source.access_token = HfFolder.get_token()
-        remote_files, metadata = self._remote_files_from_source(source)
+        source.access_token = source.access_token or HfFolder.get_token()
+        if not source.access_token:
+            self._logger.info("No HuggingFace access token present; some models may not be downloadable.")
+
+        metadata = HuggingFaceMetadataFetch(self._session).from_id(source.repo_id, source.variant)
+        assert isinstance(metadata, ModelMetadataWithFiles)
+        remote_files = metadata.download_urls(
+            variant=source.variant or self._guess_variant(),
+            subfolder=source.subfolder,
+            session=self._session,
+        )
+
         return self._import_remote_model(
             source=source,
             config=config,
@@ -699,12 +662,22 @@ class ModelInstallService(ModelInstallServiceBase):
             metadata=metadata,
         )
 
-    def _import_from_url(
-        self,
-        source: URLModelSource,
-        config: Optional[Dict[str, Any]],
-    ) -> ModelInstallJob:
-        remote_files, metadata = self._remote_files_from_source(source)
+    def _import_from_url(self, source: URLModelSource, config: Optional[Dict[str, Any]]) -> ModelInstallJob:
+        # URLs from HuggingFace will be handled specially
+        metadata = None
+        fetcher = None
+        try:
+            fetcher = self.get_fetcher_from_url(str(source.url))
+        except ValueError:
+            pass
+        kwargs: dict[str, Any] = {"session": self._session}
+        if fetcher is not None:
+            metadata = fetcher(**kwargs).from_url(source.url)
+        self._logger.debug(f"metadata={metadata}")
+        if metadata and isinstance(metadata, ModelMetadataWithFiles):
+            remote_files = metadata.download_urls(session=self._session)
+        else:
+            remote_files = [RemoteModelFile(url=source.url, path=Path("."), size=0)]
         return self._import_remote_model(
             source=source,
             config=config,
@@ -719,9 +692,12 @@ class ModelInstallService(ModelInstallServiceBase):
         metadata: Optional[AnyModelRepoMetadata],
         config: Optional[Dict[str, Any]],
     ) -> ModelInstallJob:
+        # TODO: Replace with tempfile.tmpdir() when multithreading is cleaned up.
+        # Currently the tmpdir isn't automatically removed at exit because it is
+        # being held in a daemon thread.
         if len(remote_files) == 0:
             raise ValueError(f"{source}: No downloadable files found")
-        destdir = Path(
+        tmpdir = Path(
             mkdtemp(
                 dir=self._app_config.models_path,
                 prefix=TMPDIR_PREFIX,
@@ -732,28 +708,55 @@ class ModelInstallService(ModelInstallServiceBase):
             source=source,
             config_in=config or {},
             source_metadata=metadata,
-            local_path=destdir,  # local path may change once the download has started due to content-disposition handling
+            local_path=tmpdir,  # local path may change once the download has started due to content-disposition handling
             bytes=0,
             total_bytes=0,
         )
-        # remember the temporary directory for later removal
-        install_job._install_tmpdir = destdir
-        install_job.total_bytes = sum((x.size or 0) for x in remote_files)
+        # In the event that there is a subfolder specified in the source,
+        # we need to remove it from the destination path in order to avoid
+        # creating unwanted subfolders
+        if isinstance(source, HFModelSource) and source.subfolder:
+            root = Path(remote_files[0].path.parts[0])
+            subfolder = root / source.subfolder
+        else:
+            root = Path(".")
+            subfolder = Path(".")
 
-        multifile_job = self._multifile_download(
-            remote_files=remote_files,
-            dest=destdir,
-            subfolder=source.subfolder if isinstance(source, HFModelSource) else None,
-            access_token=source.access_token,
-            submit_job=False,  # Important! Don't submit the job until we have set our _download_cache dict
-        )
-        self._download_cache[multifile_job.id] = install_job
-        install_job._multifile_job = multifile_job
+        # we remember the path up to the top of the tmpdir so that it may be
+        # removed safely at the end of the install process.
+        install_job._install_tmpdir = tmpdir
+        assert install_job.total_bytes is not None  # to avoid type checking complaints in the loop below
 
-        files_string = "file" if len(remote_files) == 1 else "files"
-        self._logger.info(f"Queueing model install: {source} ({len(remote_files)} {files_string})")
+        files_string = "file" if len(remote_files) == 1 else "file"
+        self._logger.info(f"Queuing model install: {source} ({len(remote_files)} {files_string})")
         self._logger.debug(f"remote_files={remote_files}")
-        self._download_queue.submit_multifile_download(multifile_job)
+        for model_file in remote_files:
+            url = model_file.url
+            path = root / model_file.path.relative_to(subfolder)
+            self._logger.debug(f"Downloading {url} => {path}")
+            install_job.total_bytes += model_file.size
+            assert hasattr(source, "access_token")
+            dest = tmpdir / path.parent
+            dest.mkdir(parents=True, exist_ok=True)
+            download_job = DownloadJob(
+                source=url,
+                dest=dest,
+                access_token=source.access_token,
+            )
+            self._download_cache[download_job.source] = install_job  # matches a download job to an install job
+            install_job.download_parts.add(download_job)
+
+        # only start the jobs once install_job.download_parts is fully populated
+        for download_job in install_job.download_parts:
+            self._download_queue.submit_download_job(
+                download_job,
+                on_start=self._download_started_callback,
+                on_progress=self._download_progress_callback,
+                on_complete=self._download_complete_callback,
+                on_error=self._download_error_callback,
+                on_cancelled=self._download_cancelled_callback,
+            )
+
         return install_job
 
     def _stat_size(self, path: Path) -> int:
@@ -765,104 +768,87 @@ class ModelInstallService(ModelInstallServiceBase):
                 size += sum(self._stat_size(Path(root, x)) for x in files)
         return size
 
-    def _multifile_download(
-        self,
-        remote_files: List[RemoteModelFile],
-        dest: Path,
-        subfolder: Optional[Path] = None,
-        access_token: Optional[str] = None,
-        submit_job: bool = True,
-    ) -> MultiFileDownloadJob:
-        # HuggingFace repo subfolders are a little tricky. If the name of the model is "sdxl-turbo", and
-        # we are installing the "vae" subfolder, we do not want to create an additional folder level, such
-        # as "sdxl-turbo/vae", nor do we want to put the contents of the vae folder directly into "sdxl-turbo".
-        # So what we do is to synthesize a folder named "sdxl-turbo_vae" here.
-        if subfolder:
-            top = Path(remote_files[0].path.parts[0])  # e.g. "sdxl-turbo/"
-            path_to_remove = top / subfolder.parts[-1]  # sdxl-turbo/vae/
-            path_to_add = Path(f"{top}_{subfolder}")
-        else:
-            path_to_remove = Path(".")
-            path_to_add = Path(".")
-
-        parts: List[RemoteModelFile] = []
-        for model_file in remote_files:
-            assert model_file.size is not None
-            parts.append(
-                RemoteModelFile(
-                    url=model_file.url,  # if a subfolder, then sdxl-turbo_vae/config.json
-                    path=path_to_add / model_file.path.relative_to(path_to_remove),
-                )
-            )
-
-        return self._download_queue.multifile_download(
-            parts=parts,
-            dest=dest,
-            access_token=access_token,
-            submit_job=submit_job,
-            on_start=self._download_started_callback,
-            on_progress=self._download_progress_callback,
-            on_complete=self._download_complete_callback,
-            on_error=self._download_error_callback,
-            on_cancelled=self._download_cancelled_callback,
-        )
-
     # ------------------------------------------------------------------
     # Callbacks are executed by the download queue in a separate thread
     # ------------------------------------------------------------------
-    def _download_started_callback(self, download_job: MultiFileDownloadJob) -> None:
+    def _download_started_callback(self, download_job: DownloadJob) -> None:
+        self._logger.info(f"Model download started: {download_job.source}")
         with self._lock:
-            if install_job := self._download_cache.get(download_job.id, None):
-                install_job.status = InstallStatus.DOWNLOADING
+            install_job = self._download_cache[download_job.source]
+            install_job.status = InstallStatus.DOWNLOADING
 
-                if install_job.local_path == install_job._install_tmpdir:  # first time
-                    assert download_job.download_path
-                    install_job.local_path = download_job.download_path
-                install_job.download_parts = download_job.download_parts
-                install_job.bytes = sum(x.bytes for x in download_job.download_parts)
-                install_job.total_bytes = download_job.total_bytes
-                self._signal_job_download_started(install_job)
+            assert download_job.download_path
+            if install_job.local_path == install_job._install_tmpdir:
+                partial_path = download_job.download_path.relative_to(install_job._install_tmpdir)
+                dest_name = partial_path.parts[0]
+                install_job.local_path = install_job._install_tmpdir / dest_name
 
-    def _download_progress_callback(self, download_job: MultiFileDownloadJob) -> None:
+            # Update the total bytes count for remote sources.
+            if not install_job.total_bytes:
+                install_job.total_bytes = sum(x.total_bytes for x in install_job.download_parts)
+
+    def _download_progress_callback(self, download_job: DownloadJob) -> None:
         with self._lock:
-            if install_job := self._download_cache.get(download_job.id, None):
-                if install_job.cancelled:  # This catches the case in which the caller directly calls job.cancel()
-                    self._download_queue.cancel_job(download_job)
-                else:
-                    # update sizes
-                    install_job.bytes = sum(x.bytes for x in download_job.download_parts)
-                    install_job.total_bytes = sum(x.total_bytes for x in download_job.download_parts)
-                    self._signal_job_downloading(install_job)
+            install_job = self._download_cache[download_job.source]
+            if install_job.cancelled:  # This catches the case in which the caller directly calls job.cancel()
+                self._cancel_download_parts(install_job)
+            else:
+                # update sizes
+                install_job.bytes = sum(x.bytes for x in install_job.download_parts)
+                self._signal_job_downloading(install_job)
 
-    def _download_complete_callback(self, download_job: MultiFileDownloadJob) -> None:
+    def _download_complete_callback(self, download_job: DownloadJob) -> None:
+        self._logger.info(f"Model download complete: {download_job.source}")
         with self._lock:
-            if install_job := self._download_cache.pop(download_job.id, None):
+            install_job = self._download_cache[download_job.source]
+
+            # are there any more active jobs left in this task?
+            if install_job.downloading and all(x.complete for x in install_job.download_parts):
                 self._signal_job_downloads_done(install_job)
-                self._put_in_queue(install_job)  # this starts the installation and registration
+                self._put_in_queue(install_job)
 
-                # Let other threads know that the number of downloads has changed
-                self._downloads_changed_event.set()
+            # Let other threads know that the number of downloads has changed
+            self._download_cache.pop(download_job.source, None)
+            self._downloads_changed_event.set()
 
-    def _download_error_callback(self, download_job: MultiFileDownloadJob, excp: Optional[Exception] = None) -> None:
+    def _download_error_callback(self, download_job: DownloadJob, excp: Optional[Exception] = None) -> None:
         with self._lock:
-            if install_job := self._download_cache.pop(download_job.id, None):
-                assert excp is not None
-                self._set_error(install_job, excp)
-                self._download_queue.cancel_job(download_job)
+            install_job = self._download_cache.pop(download_job.source, None)
+            assert install_job is not None
+            assert excp is not None
+            install_job.set_error(excp)
+            self._logger.error(
+                f"Cancelling {install_job.source} due to an error while downloading {download_job.source}: {str(excp)}"
+            )
+            self._cancel_download_parts(install_job)
 
-                # Let other threads know that the number of downloads has changed
-                self._downloads_changed_event.set()
+            # Let other threads know that the number of downloads has changed
+            self._downloads_changed_event.set()
 
-    def _download_cancelled_callback(self, download_job: MultiFileDownloadJob) -> None:
+    def _download_cancelled_callback(self, download_job: DownloadJob) -> None:
         with self._lock:
-            if install_job := self._download_cache.pop(download_job.id, None):
-                self._downloads_changed_event.set()
-                # if install job has already registered an error, then do not replace its status with cancelled
-                if not install_job.errored:
-                    install_job.cancel()
+            install_job = self._download_cache.pop(download_job.source, None)
+            if not install_job:
+                return
+            self._downloads_changed_event.set()
+            self._logger.warning(f"Model download canceled: {download_job.source}")
+            # if install job has already registered an error, then do not replace its status with cancelled
+            if not install_job.errored:
+                install_job.cancel()
+            self._cancel_download_parts(install_job)
 
-                # Let other threads know that the number of downloads has changed
-                self._downloads_changed_event.set()
+            # Let other threads know that the number of downloads has changed
+            self._downloads_changed_event.set()
+
+    def _cancel_download_parts(self, install_job: ModelInstallJob) -> None:
+        # on multipart downloads, _cancel_components() will get called repeatedly from the download callbacks
+        # do not lock here because it gets called within a locked context
+        for s in install_job.download_parts:
+            self._download_queue.cancel_job(s)
+
+        if all(x.in_terminal_state for x in install_job.download_parts):
+            # When all parts have reached their terminal state, we finalize the job to clean up the temporary directory and other resources
+            self._put_in_queue(install_job)
 
     # ------------------------------------------------------------------------------------------------
     # Internal methods that put events on the event bus
@@ -873,18 +859,8 @@ class ModelInstallService(ModelInstallServiceBase):
         if self._event_bus:
             self._event_bus.emit_model_install_started(job)
 
-    def _signal_job_download_started(self, job: ModelInstallJob) -> None:
-        if self._event_bus:
-            assert job._multifile_job is not None
-            assert job.bytes is not None
-            assert job.total_bytes is not None
-            self._event_bus.emit_model_install_download_started(job)
-
     def _signal_job_downloading(self, job: ModelInstallJob) -> None:
         if self._event_bus:
-            assert job._multifile_job is not None
-            assert job.bytes is not None
-            assert job.total_bytes is not None
             self._event_bus.emit_model_install_download_progress(job)
 
     def _signal_job_downloads_done(self, job: ModelInstallJob) -> None:
@@ -899,8 +875,6 @@ class ModelInstallService(ModelInstallServiceBase):
         self._logger.info(f"Model install complete: {job.source}")
         self._logger.debug(f"{job.local_path} registered key {job.config_out.key}")
         if self._event_bus:
-            assert job.local_path is not None
-            assert job.config_out is not None
             self._event_bus.emit_model_install_complete(job)
 
     def _signal_job_errored(self, job: ModelInstallJob) -> None:
@@ -916,13 +890,7 @@ class ModelInstallService(ModelInstallServiceBase):
             self._event_bus.emit_model_install_cancelled(job)
 
     @staticmethod
-    def get_fetcher_from_url(url: str) -> Type[ModelMetadataFetchBase]:
-        """
-        Return a metadata fetcher appropriate for provided url.
-
-        This used to be more useful, but the number of supported model
-        sources has been reduced to HuggingFace alone.
-        """
+    def get_fetcher_from_url(url: str) -> ModelMetadataFetchBase:
         if re.match(r"^https?://huggingface.co/[^/]+/[^/]+$", url.lower()):
             return HuggingFaceMetadataFetch
         raise ValueError(f"Unsupported model source: '{url}'")

invokeai/app/services/model_load/__init__.py

@@ -1,6 +1,6 @@
 """Initialization file for model load service module."""
 
-from invokeai.app.services.model_load.model_load_base import ModelLoadServiceBase
-from invokeai.app.services.model_load.model_load_default import ModelLoadService
+from .model_load_base import ModelLoadServiceBase
+from .model_load_default import ModelLoadService
 
 __all__ = ["ModelLoadServiceBase", "ModelLoadService"]

invokeai/app/services/model_load/model_load_base.py

@@ -2,11 +2,11 @@
 """Base class for model loader."""
 
 from abc import ABC, abstractmethod
-from pathlib import Path
-from typing import Callable, Optional
+from typing import Optional
 
 from invokeai.backend.model_manager import AnyModel, AnyModelConfig, SubModelType
-from invokeai.backend.model_manager.load import LoadedModel, LoadedModelWithoutConfig
+from invokeai.backend.model_manager.load import LoadedModel
+from invokeai.backend.model_manager.load.convert_cache import ModelConvertCacheBase
 from invokeai.backend.model_manager.load.model_cache.model_cache_base import ModelCacheBase
 
 
@@ -27,25 +27,7 @@ class ModelLoadServiceBase(ABC):
     def ram_cache(self) -> ModelCacheBase[AnyModel]:
         """Return the RAM cache used by this loader."""
 
+    @property
     @abstractmethod
-    def load_model_from_path(
-        self, model_path: Path, loader: Optional[Callable[[Path], AnyModel]] = None
-    ) -> LoadedModelWithoutConfig:
-        """
-        Load the model file or directory located at the indicated Path.
-
-        This will load an arbitrary model file into the RAM cache. If the optional loader
-        argument is provided, the loader will be invoked to load the model into
-        memory. Otherwise the method will call safetensors.torch.load_file() or
-        torch.load() as appropriate to the file suffix.
-
-        Be aware that this returns a LoadedModelWithoutConfig object, which is the same as
-        LoadedModel, but without the config attribute.
-
-        Args:
-          model_path: A pathlib.Path to a checkpoint-style models file
-          loader: A Callable that expects a Path and returns a Dict[str, Tensor]
-
-        Returns:
-          A LoadedModel object.
-        """
+    def convert_cache(self) -> ModelConvertCacheBase:
+        """Return the checkpoint convert cache used by this loader."""

invokeai/app/services/model_load/model_load_default.py

@@ -1,28 +1,22 @@
 # Copyright (c) 2024 Lincoln D. Stein and the InvokeAI Team
 """Implementation of model loader service."""
 
-from pathlib import Path
-from typing import Callable, Optional, Type
-
-from picklescan.scanner import scan_file_path
-from safetensors.torch import load_file as safetensors_load_file
-from torch import load as torch_load
+from typing import Optional, Type
 
 from invokeai.app.services.config import InvokeAIAppConfig
 from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.model_load.model_load_base import ModelLoadServiceBase
 from invokeai.backend.model_manager import AnyModel, AnyModelConfig, SubModelType
 from invokeai.backend.model_manager.load import (
     LoadedModel,
-    LoadedModelWithoutConfig,
     ModelLoaderRegistry,
     ModelLoaderRegistryBase,
 )
+from invokeai.backend.model_manager.load.convert_cache import ModelConvertCacheBase
 from invokeai.backend.model_manager.load.model_cache.model_cache_base import ModelCacheBase
-from invokeai.backend.model_manager.load.model_loaders.generic_diffusers import GenericDiffusersLoader
-from invokeai.backend.util.devices import TorchDevice
 from invokeai.backend.util.logging import InvokeAILogger
 
+from .model_load_base import ModelLoadServiceBase
+
 
 class ModelLoadService(ModelLoadServiceBase):
     """Wrapper around ModelLoaderRegistry."""
@@ -31,6 +25,7 @@ class ModelLoadService(ModelLoadServiceBase):
         self,
         app_config: InvokeAIAppConfig,
         ram_cache: ModelCacheBase[AnyModel],
+        convert_cache: ModelConvertCacheBase,
         registry: Optional[Type[ModelLoaderRegistryBase]] = ModelLoaderRegistry,
     ):
         """Initialize the model load service."""
@@ -39,6 +34,7 @@ class ModelLoadService(ModelLoadServiceBase):
         self._logger = logger
         self._app_config = app_config
         self._ram_cache = ram_cache
+        self._convert_cache = convert_cache
         self._registry = registry
 
     def start(self, invoker: Invoker) -> None:
@@ -49,6 +45,11 @@ class ModelLoadService(ModelLoadServiceBase):
         """Return the RAM cache used by this loader."""
         return self._ram_cache
 
+    @property
+    def convert_cache(self) -> ModelConvertCacheBase:
+        """Return the checkpoint convert cache used by this loader."""
+        return self._convert_cache
+
     def load_model(self, model_config: AnyModelConfig, submodel_type: Optional[SubModelType] = None) -> LoadedModel:
         """
         Given a model's configuration, load it and return the LoadedModel object.
@@ -67,47 +68,10 @@ class ModelLoadService(ModelLoadServiceBase):
             app_config=self._app_config,
             logger=self._logger,
             ram_cache=self._ram_cache,
+            convert_cache=self._convert_cache,
         ).load_model(model_config, submodel_type)
 
         if hasattr(self, "_invoker"):
             self._invoker.services.events.emit_model_load_complete(model_config, submodel_type)
 
         return loaded_model
-
-    def load_model_from_path(
-        self, model_path: Path, loader: Optional[Callable[[Path], AnyModel]] = None
-    ) -> LoadedModelWithoutConfig:
-        cache_key = str(model_path)
-        ram_cache = self.ram_cache
-        try:
-            return LoadedModelWithoutConfig(_locker=ram_cache.get(key=cache_key))
-        except IndexError:
-            pass
-
-        def torch_load_file(checkpoint: Path) -> AnyModel:
-            scan_result = scan_file_path(checkpoint)
-            if scan_result.infected_files != 0:
-                raise Exception("The model at {checkpoint} is potentially infected by malware. Aborting load.")
-            result = torch_load(checkpoint, map_location="cpu")
-            return result
-
-        def diffusers_load_directory(directory: Path) -> AnyModel:
-            load_class = GenericDiffusersLoader(
-                app_config=self._app_config,
-                logger=self._logger,
-                ram_cache=self._ram_cache,
-                convert_cache=self.convert_cache,
-            ).get_hf_load_class(directory)
-            return load_class.from_pretrained(model_path, torch_dtype=TorchDevice.choose_torch_dtype())
-
-        loader = loader or (
-            diffusers_load_directory
-            if model_path.is_dir()
-            else torch_load_file
-            if model_path.suffix.endswith((".ckpt", ".pt", ".pth", ".bin"))
-            else lambda path: safetensors_load_file(path, device="cpu")
-        )
-        assert loader is not None
-        raw_model = loader(model_path)
-        ram_cache.put(key=cache_key, model=raw_model)
-        return LoadedModelWithoutConfig(_locker=ram_cache.get(key=cache_key))

invokeai/app/services/model_manager/__init__.py

@@ -1,9 +1,10 @@
 """Initialization file for model manager service."""
 
-from invokeai.app.services.model_manager.model_manager_default import ModelManagerService, ModelManagerServiceBase
 from invokeai.backend.model_manager import AnyModel, AnyModelConfig, BaseModelType, ModelType, SubModelType
 from invokeai.backend.model_manager.load import LoadedModel
 
+from .model_manager_default import ModelManagerService, ModelManagerServiceBase
+
 __all__ = [
     "ModelManagerServiceBase",
     "ModelManagerService",

invokeai/app/services/model_manager/model_manager_base.py

@@ -5,13 +5,14 @@ from abc import ABC, abstractmethod
 import torch
 from typing_extensions import Self
 
-from invokeai.app.services.config.config_default import InvokeAIAppConfig
-from invokeai.app.services.download.download_base import DownloadQueueServiceBase
-from invokeai.app.services.events.events_base import EventServiceBase
 from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.model_install.model_install_base import ModelInstallServiceBase
-from invokeai.app.services.model_load.model_load_base import ModelLoadServiceBase
-from invokeai.app.services.model_records.model_records_base import ModelRecordServiceBase
+
+from ..config import InvokeAIAppConfig
+from ..download import DownloadQueueServiceBase
+from ..events.events_base import EventServiceBase
+from ..model_install import ModelInstallServiceBase
+from ..model_load import ModelLoadServiceBase
+from ..model_records import ModelRecordServiceBase
 
 
 class ModelManagerServiceBase(ABC):

invokeai/app/services/model_manager/model_manager_default.py

@@ -6,20 +6,19 @@ from typing import Optional
 import torch
 from typing_extensions import Self
 
-from invokeai.app.services.config.config_default import InvokeAIAppConfig
-from invokeai.app.services.download.download_base import DownloadQueueServiceBase
-from invokeai.app.services.events.events_base import EventServiceBase
 from invokeai.app.services.invoker import Invoker
-from invokeai.app.services.model_install.model_install_base import ModelInstallServiceBase
-from invokeai.app.services.model_install.model_install_default import ModelInstallService
-from invokeai.app.services.model_load.model_load_base import ModelLoadServiceBase
-from invokeai.app.services.model_load.model_load_default import ModelLoadService
-from invokeai.app.services.model_manager.model_manager_base import ModelManagerServiceBase
-from invokeai.app.services.model_records.model_records_base import ModelRecordServiceBase
-from invokeai.backend.model_manager.load import ModelCache, ModelLoaderRegistry
+from invokeai.backend.model_manager.load import ModelCache, ModelConvertCache, ModelLoaderRegistry
 from invokeai.backend.util.devices import TorchDevice
 from invokeai.backend.util.logging import InvokeAILogger
 
+from ..config import InvokeAIAppConfig
+from ..download import DownloadQueueServiceBase
+from ..events.events_base import EventServiceBase
+from ..model_install import ModelInstallService, ModelInstallServiceBase
+from ..model_load import ModelLoadService, ModelLoadServiceBase
+from ..model_records import ModelRecordServiceBase
+from .model_manager_base import ModelManagerServiceBase
+
 
 class ModelManagerService(ModelManagerServiceBase):
     """
@@ -87,9 +86,11 @@ class ModelManagerService(ModelManagerServiceBase):
             logger=logger,
             execution_device=execution_device or TorchDevice.choose_torch_device(),
         )
+        convert_cache = ModelConvertCache(cache_path=app_config.convert_cache_path, max_size=app_config.convert_cache)
         loader = ModelLoadService(
             app_config=app_config,
             ram_cache=ram_cache,
+            convert_cache=convert_cache,
             registry=ModelLoaderRegistry,
         )
         installer = ModelInstallService(

invokeai/app/services/model_records/model_records_base.py

@@ -12,13 +12,15 @@ from pydantic import BaseModel, Field
 
 from invokeai.app.services.shared.pagination import PaginatedResults
 from invokeai.app.util.model_exclude_null import BaseModelExcludeNull
-from invokeai.backend.model_manager.config import (
+from invokeai.backend.model_manager import (
     AnyModelConfig,
     BaseModelType,
-    ControlAdapterDefaultSettings,
-    MainModelDefaultSettings,
     ModelFormat,
     ModelType,
+)
+from invokeai.backend.model_manager.config import (
+    ControlAdapterDefaultSettings,
+    MainModelDefaultSettings,
     ModelVariantType,
     SchedulerPredictionType,
 )

invokeai/app/services/model_records/model_records_sql.py

@@ -45,16 +45,7 @@ from math import ceil
 from pathlib import Path
 from typing import List, Optional, Union
 
-from invokeai.app.services.model_records.model_records_base import (
-    DuplicateModelException,
-    ModelRecordChanges,
-    ModelRecordOrderBy,
-    ModelRecordServiceBase,
-    ModelSummary,
-    UnknownModelException,
-)
 from invokeai.app.services.shared.pagination import PaginatedResults
-from invokeai.app.services.shared.sqlite.sqlite_database import SqliteDatabase
 from invokeai.backend.model_manager.config import (
     AnyModelConfig,
     BaseModelType,
@@ -63,6 +54,16 @@ from invokeai.backend.model_manager.config import (
     ModelType,
 )
 
+from ..shared.sqlite.sqlite_database import SqliteDatabase
+from .model_records_base import (
+    DuplicateModelException,
+    ModelRecordChanges,
+    ModelRecordOrderBy,
+    ModelRecordServiceBase,
+    ModelSummary,
+    UnknownModelException,
+)
+
 
 class ModelRecordServiceSQL(ModelRecordServiceBase):
     """Implementation of the ModelConfigStore ABC using a SQL database."""

invokeai/app/services/names/names_default.py

@@ -1,6 +1,7 @@
-from invokeai.app.services.names.names_base import NameServiceBase
 from invokeai.app.util.misc import uuid_string
 
+from .names_base import NameServiceBase
+
 
 class SimpleNameService(NameServiceBase):
     """Creates image names from UUIDs."""

invokeai/app/services/session_processor/session_processor_default.py

@@ -13,24 +13,24 @@ from invokeai.app.services.events.events_common import (
     register_events,
 )
 from invokeai.app.services.invocation_stats.invocation_stats_common import GESStatsNotFoundError
-from invokeai.app.services.invoker import Invoker
 from invokeai.app.services.session_processor.session_processor_base import (
-    InvocationServices,
     OnAfterRunNode,
     OnAfterRunSession,
     OnBeforeRunNode,
     OnBeforeRunSession,
     OnNodeError,
     OnNonFatalProcessorError,
-    SessionProcessorBase,
-    SessionRunnerBase,
 )
-from invokeai.app.services.session_processor.session_processor_common import CanceledException, SessionProcessorStatus
+from invokeai.app.services.session_processor.session_processor_common import CanceledException
 from invokeai.app.services.session_queue.session_queue_common import SessionQueueItem, SessionQueueItemNotFoundError
 from invokeai.app.services.shared.graph import NodeInputError
 from invokeai.app.services.shared.invocation_context import InvocationContextData, build_invocation_context
 from invokeai.app.util.profiler import Profiler
 
+from ..invoker import Invoker
+from .session_processor_base import InvocationServices, SessionProcessorBase, SessionRunnerBase
+from .session_processor_common import SessionProcessorStatus
+
 
 class DefaultSessionRunner(SessionRunnerBase):
     """Processes a single session's invocations."""

invokeai/app/services/session_queue/session_queue_sqlite.py

@@ -37,14 +37,10 @@ class SqliteSessionQueue(SessionQueueBase):
     def start(self, invoker: Invoker) -> None:
         self.__invoker = invoker
         self._set_in_progress_to_canceled()
-        if self.__invoker.services.configuration.clear_queue_on_startup:
-            clear_result = self.clear(DEFAULT_QUEUE_ID)
-            if clear_result.deleted > 0:
-                self.__invoker.services.logger.info(f"Cleared all {clear_result.deleted} queue items")
-        else:
-            prune_result = self.prune(DEFAULT_QUEUE_ID)
-            if prune_result.deleted > 0:
-                self.__invoker.services.logger.info(f"Pruned {prune_result.deleted} finished queue items")
+        prune_result = self.prune(DEFAULT_QUEUE_ID)
+
+        if prune_result.deleted > 0:
+            self.__invoker.services.logger.info(f"Pruned {prune_result.deleted} finished queue items")
 
     def __init__(self, db: SqliteDatabase) -> None:
         super().__init__()

invokeai/app/services/shared/graph.py

@@ -652,7 +652,7 @@ class Graph(BaseModel):
         output_fields = [get_input_field(self.get_node(e.node_id), e.field) for e in outputs]
 
         # Input type must be a list
-        if get_origin(input_field) is not list:
+        if get_origin(input_field) != list:
             return False
 
         # Validate that all outputs match the input type

invokeai/app/services/shared/invocation_context.py

@@ -3,7 +3,6 @@ from pathlib import Path
 from typing import TYPE_CHECKING, Callable, Optional, Union
 
 from PIL.Image import Image
-from pydantic.networks import AnyHttpUrl
 from torch import Tensor
 
 from invokeai.app.invocations.constants import IMAGE_MODES
@@ -15,15 +14,8 @@ from invokeai.app.services.images.images_common import ImageDTO
 from invokeai.app.services.invocation_services import InvocationServices
 from invokeai.app.services.model_records.model_records_base import UnknownModelException
 from invokeai.app.util.step_callback import stable_diffusion_step_callback
-from invokeai.backend.model_manager.config import (
-    AnyModel,
-    AnyModelConfig,
-    BaseModelType,
-    ModelFormat,
-    ModelType,
-    SubModelType,
-)
-from invokeai.backend.model_manager.load.load_base import LoadedModel, LoadedModelWithoutConfig
+from invokeai.backend.model_manager.config import AnyModelConfig, BaseModelType, ModelFormat, ModelType, SubModelType
+from invokeai.backend.model_manager.load.load_base import LoadedModel
 from invokeai.backend.stable_diffusion.diffusers_pipeline import PipelineIntermediateState
 from invokeai.backend.stable_diffusion.diffusion.conditioning_data import ConditioningFieldData
 
@@ -328,10 +320,8 @@ class ConditioningInterface(InvocationContextInterface):
 
 
 class ModelsInterface(InvocationContextInterface):
-    """Common API for loading, downloading and managing models."""
-
     def exists(self, identifier: Union[str, "ModelIdentifierField"]) -> bool:
-        """Check if a model exists.
+        """Checks if a model exists.
 
         Args:
             identifier: The key or ModelField representing the model.
@@ -341,13 +331,13 @@ class ModelsInterface(InvocationContextInterface):
         """
         if isinstance(identifier, str):
             return self._services.model_manager.store.exists(identifier)
-        else:
-            return self._services.model_manager.store.exists(identifier.key)
+
+        return self._services.model_manager.store.exists(identifier.key)
 
     def load(
         self, identifier: Union[str, "ModelIdentifierField"], submodel_type: Optional[SubModelType] = None
     ) -> LoadedModel:
-        """Load a model.
+        """Loads a model.
 
         Args:
             identifier: The key or ModelField representing the model.
@@ -371,7 +361,7 @@ class ModelsInterface(InvocationContextInterface):
     def load_by_attrs(
         self, name: str, base: BaseModelType, type: ModelType, submodel_type: Optional[SubModelType] = None
     ) -> LoadedModel:
-        """Load a model by its attributes.
+        """Loads a model by its attributes.
 
         Args:
             name: Name of the model.
@@ -394,7 +384,7 @@ class ModelsInterface(InvocationContextInterface):
         return self._services.model_manager.load.load_model(configs[0], submodel_type)
 
     def get_config(self, identifier: Union[str, "ModelIdentifierField"]) -> AnyModelConfig:
-        """Get a model's config.
+        """Gets a model's config.
 
         Args:
             identifier: The key or ModelField representing the model.
@@ -404,11 +394,11 @@ class ModelsInterface(InvocationContextInterface):
         """
         if isinstance(identifier, str):
             return self._services.model_manager.store.get_model(identifier)
-        else:
-            return self._services.model_manager.store.get_model(identifier.key)
+
+        return self._services.model_manager.store.get_model(identifier.key)
 
     def search_by_path(self, path: Path) -> list[AnyModelConfig]:
-        """Search for models by path.
+        """Searches for models by path.
 
         Args:
             path: The path to search for.
@@ -425,7 +415,7 @@ class ModelsInterface(InvocationContextInterface):
         type: Optional[ModelType] = None,
         format: Optional[ModelFormat] = None,
     ) -> list[AnyModelConfig]:
-        """Search for models by attributes.
+        """Searches for models by attributes.
 
         Args:
             name: The name to search for (exact match).
@@ -444,72 +434,6 @@ class ModelsInterface(InvocationContextInterface):
             model_format=format,
         )
 
-    def download_and_cache_model(
-        self,
-        source: str | AnyHttpUrl,
-    ) -> Path:
-        """
-        Download the model file located at source to the models cache and return its Path.
-
-        This can be used to single-file install models and other resources of arbitrary types
-        which should not get registered with the database. If the model is already
-        installed, the cached path will be returned. Otherwise it will be downloaded.
-
-        Args:
-            source: A URL that points to the model, or a huggingface repo_id.
-
-        Returns:
-            Path to the downloaded model
-        """
-        return self._services.model_manager.install.download_and_cache_model(source=source)
-
-    def load_local_model(
-        self,
-        model_path: Path,
-        loader: Optional[Callable[[Path], AnyModel]] = None,
-    ) -> LoadedModelWithoutConfig:
-        """
-        Load the model file located at the indicated path
-
-        If a loader callable is provided, it will be invoked to load the model. Otherwise,
-        `safetensors.torch.load_file()` or `torch.load()` will be called to load the model.
-
-        Be aware that the LoadedModelWithoutConfig object has no `config` attribute
-
-        Args:
-            path: A model Path
-            loader: A Callable that expects a Path and returns a dict[str|int, Any]
-
-        Returns:
-            A LoadedModelWithoutConfig object.
-        """
-        return self._services.model_manager.load.load_model_from_path(model_path=model_path, loader=loader)
-
-    def load_remote_model(
-        self,
-        source: str | AnyHttpUrl,
-        loader: Optional[Callable[[Path], AnyModel]] = None,
-    ) -> LoadedModelWithoutConfig:
-        """
-        Download, cache, and load the model file located at the indicated URL or repo_id.
-
-        If the model is already downloaded, it will be loaded from the cache.
-
-        If the a loader callable is provided, it will be invoked to load the model. Otherwise,
-        `safetensors.torch.load_file()` or `torch.load()` will be called to load the model.
-
-        Be aware that the LoadedModelWithoutConfig object has no `config` attribute
-
-        Args:
-            source: A URL or huggingface repoid.
-            loader: A Callable that expects a Path and returns a dict[str|int, Any]
-
-        Returns:
-            A LoadedModelWithoutConfig object.
-        """
-        model_path = self._services.model_manager.install.download_and_cache_model(source=str(source))
-        return self._services.model_manager.load.load_model_from_path(model_path=model_path, loader=loader)
-
 
 class ConfigInterface(InvocationContextInterface):
     def get(self) -> InvokeAIAppConfig:

invokeai/app/services/shared/sqlite/sqlite_util.py

@@ -13,9 +13,6 @@ from invokeai.app.services.shared.sqlite_migrator.migrations.migration_7 import
 from invokeai.app.services.shared.sqlite_migrator.migrations.migration_8 import build_migration_8
 from invokeai.app.services.shared.sqlite_migrator.migrations.migration_9 import build_migration_9
 from invokeai.app.services.shared.sqlite_migrator.migrations.migration_10 import build_migration_10
-from invokeai.app.services.shared.sqlite_migrator.migrations.migration_11 import build_migration_11
-from invokeai.app.services.shared.sqlite_migrator.migrations.migration_12 import build_migration_12
-from invokeai.app.services.shared.sqlite_migrator.migrations.migration_13 import build_migration_13
 from invokeai.app.services.shared.sqlite_migrator.sqlite_migrator_impl import SqliteMigrator
 
 
@@ -46,9 +43,6 @@ def init_db(config: InvokeAIAppConfig, logger: Logger, image_files: ImageFileSto
     migrator.register_migration(build_migration_8(app_config=config))
     migrator.register_migration(build_migration_9())
     migrator.register_migration(build_migration_10())
-    migrator.register_migration(build_migration_11(app_config=config, logger=logger))
-    migrator.register_migration(build_migration_12(app_config=config))
-    migrator.register_migration(build_migration_13())
     migrator.run_migrations()
 
     return db

invokeai/app/services/shared/sqlite_migrator/migrations/migration_11.py

@@ -1,75 +0,0 @@
-import shutil
-import sqlite3
-from logging import Logger
-
-from invokeai.app.services.config import InvokeAIAppConfig
-from invokeai.app.services.shared.sqlite_migrator.sqlite_migrator_common import Migration
-
-LEGACY_CORE_MODELS = [
-    # OpenPose
-    "any/annotators/dwpose/yolox_l.onnx",
-    "any/annotators/dwpose/dw-ll_ucoco_384.onnx",
-    # DepthAnything
-    "any/annotators/depth_anything/depth_anything_vitl14.pth",
-    "any/annotators/depth_anything/depth_anything_vitb14.pth",
-    "any/annotators/depth_anything/depth_anything_vits14.pth",
-    # Lama inpaint
-    "core/misc/lama/lama.pt",
-    # RealESRGAN upscale
-    "core/upscaling/realesrgan/RealESRGAN_x4plus.pth",
-    "core/upscaling/realesrgan/RealESRGAN_x4plus_anime_6B.pth",
-    "core/upscaling/realesrgan/ESRGAN_SRx4_DF2KOST_official-ff704c30.pth",
-    "core/upscaling/realesrgan/RealESRGAN_x2plus.pth",
-]
-
-
-class Migration11Callback:
-    def __init__(self, app_config: InvokeAIAppConfig, logger: Logger) -> None:
-        self._app_config = app_config
-        self._logger = logger
-
-    def __call__(self, cursor: sqlite3.Cursor) -> None:
-        self._remove_convert_cache()
-        self._remove_downloaded_models()
-        self._remove_unused_core_models()
-
-    def _remove_convert_cache(self) -> None:
-        """Rename models/.cache to models/.convert_cache."""
-        self._logger.info("Removing .cache directory. Converted models will now be cached in .convert_cache.")
-        legacy_convert_path = self._app_config.root_path / "models" / ".cache"
-        shutil.rmtree(legacy_convert_path, ignore_errors=True)
-
-    def _remove_downloaded_models(self) -> None:
-        """Remove models from their old locations; they will re-download when needed."""
-        self._logger.info(
-            "Removing legacy just-in-time models. Downloaded models will now be cached in .download_cache."
-        )
-        for model_path in LEGACY_CORE_MODELS:
-            legacy_dest_path = self._app_config.models_path / model_path
-            legacy_dest_path.unlink(missing_ok=True)
-
-    def _remove_unused_core_models(self) -> None:
-        """Remove unused core models and their directories."""
-        self._logger.info("Removing defunct core models.")
-        for dir in ["face_restoration", "misc", "upscaling"]:
-            path_to_remove = self._app_config.models_path / "core" / dir
-            shutil.rmtree(path_to_remove, ignore_errors=True)
-        shutil.rmtree(self._app_config.models_path / "any" / "annotators", ignore_errors=True)
-
-
-def build_migration_11(app_config: InvokeAIAppConfig, logger: Logger) -> Migration:
-    """
-    Build the migration from database version 10 to 11.
-
-    This migration does the following:
-    - Moves "core" models previously downloaded with download_with_progress_bar() into new
-      "models/.download_cache" directory.
-    - Renames "models/.cache" to "models/.convert_cache".
-    """
-    migration_11 = Migration(
-        from_version=10,
-        to_version=11,
-        callback=Migration11Callback(app_config=app_config, logger=logger),
-    )
-
-    return migration_11

invokeai/app/services/shared/sqlite_migrator/migrations/migration_12.py

@@ -1,35 +0,0 @@
-import shutil
-import sqlite3
-
-from invokeai.app.services.config import InvokeAIAppConfig
-from invokeai.app.services.shared.sqlite_migrator.sqlite_migrator_common import Migration
-
-
-class Migration12Callback:
-    def __init__(self, app_config: InvokeAIAppConfig) -> None:
-        self._app_config = app_config
-
-    def __call__(self, cursor: sqlite3.Cursor) -> None:
-        self._remove_model_convert_cache_dir()
-
-    def _remove_model_convert_cache_dir(self) -> None:
-        """
-        Removes unused model convert cache directory
-        """
-        convert_cache = self._app_config.convert_cache_path
-        shutil.rmtree(convert_cache, ignore_errors=True)
-
-
-def build_migration_12(app_config: InvokeAIAppConfig) -> Migration:
-    """
-    Build the migration from database version 11 to 12.
-
-    This migration removes the now-unused model convert cache directory.
-    """
-    migration_12 = Migration(
-        from_version=11,
-        to_version=12,
-        callback=Migration12Callback(app_config),
-    )
-
-    return migration_12

invokeai/app/services/shared/sqlite_migrator/migrations/migration_13.py

@@ -1,31 +0,0 @@
-import sqlite3
-
-from invokeai.app.services.shared.sqlite_migrator.sqlite_migrator_common import Migration
-
-
-class Migration13Callback:
-    def __call__(self, cursor: sqlite3.Cursor) -> None:
-        self._add_archived_col(cursor)
-
-    def _add_archived_col(self, cursor: sqlite3.Cursor) -> None:
-        """
-        - Adds `archived` columns to the board table.
-        """
-
-        cursor.execute("ALTER TABLE boards ADD COLUMN archived BOOLEAN DEFAULT FALSE;")
-
-
-def build_migration_13() -> Migration:
-    """
-    Build the migration from database version 12 to 13..
-
-    This migration does the following:
-    - Adds `archived` columns to the board table.
-    """
-    migration_13 = Migration(
-        from_version=12,
-        to_version=13,
-        callback=Migration13Callback(),
-    )
-
-    return migration_13

invokeai/app/services/urls/urls_default.py

@@ -1,6 +1,6 @@
 import os
 
-from invokeai.app.services.urls.urls_base import UrlServiceBase
+from .urls_base import UrlServiceBase
 
 
 class LocalUrlService(UrlServiceBase):

invokeai/app/util/download_with_progress.py

@@ -0,0 +1,51 @@
+from pathlib import Path
+from urllib import request
+
+from tqdm import tqdm
+
+from invokeai.backend.util.logging import InvokeAILogger
+
+
+class ProgressBar:
+    """Simple progress bar for urllib.request.urlretrieve using tqdm."""
+
+    def __init__(self, model_name: str = "file"):
+        self.pbar = None
+        self.name = model_name
+
+    def __call__(self, block_num: int, block_size: int, total_size: int):
+        if not self.pbar:
+            self.pbar = tqdm(
+                desc=self.name,
+                initial=0,
+                unit="iB",
+                unit_scale=True,
+                unit_divisor=1000,
+                total=total_size,
+            )
+        self.pbar.update(block_size)
+
+
+def download_with_progress_bar(name: str, url: str, dest_path: Path) -> bool:
+    """Download a file from a URL to a destination path, with a progress bar.
+    If the file already exists, it will not be downloaded again.
+
+    Exceptions are not caught.
+
+    Args:
+        name (str): Name of the file being downloaded.
+        url (str): URL to download the file from.
+        dest_path (Path): Destination path to save the file to.
+
+    Returns:
+        bool: True if the file was downloaded, False if it already existed.
+    """
+    if dest_path.exists():
+        return False  # already downloaded
+
+    InvokeAILogger.get_logger().info(f"Downloading {name}...")
+
+    dest_path.parent.mkdir(parents=True, exist_ok=True)
+    request.urlretrieve(url, dest_path, ProgressBar(name))
+
+    return True

invokeai/app/util/step_callback.py

@@ -5,8 +5,9 @@ from PIL import Image
 
 from invokeai.app.services.session_processor.session_processor_common import CanceledException, ProgressImage
 from invokeai.backend.model_manager.config import BaseModelType
-from invokeai.backend.stable_diffusion.diffusers_pipeline import PipelineIntermediateState
-from invokeai.backend.util.util import image_to_dataURL
+
+from ...backend.stable_diffusion import PipelineIntermediateState
+from ...backend.util.util import image_to_dataURL
 
 if TYPE_CHECKING:
     from invokeai.app.services.events.events_base import EventServiceBase

invokeai/backend/image_util/__init__.py

@@ -2,11 +2,6 @@
 Initialization file for invokeai.backend.image_util methods.
 """
 
-from invokeai.backend.image_util.infill_methods.patchmatch import PatchMatch  # noqa: F401
-from invokeai.backend.image_util.pngwriter import (  # noqa: F401
-    PngWriter,
-    PromptFormatter,
-    retrieve_metadata,
-    write_metadata,
-)
-from invokeai.backend.image_util.util import InitImageResizer, make_grid  # noqa: F401
+from .infill_methods.patchmatch import PatchMatch  # noqa: F401
+from .pngwriter import PngWriter, PromptFormatter, retrieve_metadata, write_metadata  # noqa: F401
+from .util import InitImageResizer, make_grid  # noqa: F401

invokeai/backend/image_util/basicsr/rrdbnet_arch.py

@@ -2,7 +2,7 @@ import torch
 from torch import nn as nn
 from torch.nn import functional as F
 
-from invokeai.backend.image_util.basicsr.arch_util import default_init_weights, make_layer, pixel_unshuffle
+from .arch_util import default_init_weights, make_layer, pixel_unshuffle
 
 
 class ResidualDenseBlock(nn.Module):

invokeai/backend/image_util/depth_anything/__init__.py

@@ -1,5 +1,5 @@
-from pathlib import Path
-from typing import Literal
+import pathlib
+from typing import Literal, Union
 
 import cv2
 import numpy as np
@@ -10,17 +10,28 @@ from PIL import Image
 from torchvision.transforms import Compose
 
 from invokeai.app.services.config.config_default import get_config
+from invokeai.app.util.download_with_progress import download_with_progress_bar
 from invokeai.backend.image_util.depth_anything.model.dpt import DPT_DINOv2
 from invokeai.backend.image_util.depth_anything.utilities.util import NormalizeImage, PrepareForNet, Resize
+from invokeai.backend.util.devices import TorchDevice
 from invokeai.backend.util.logging import InvokeAILogger
 
 config = get_config()
 logger = InvokeAILogger.get_logger(config=config)
 
 DEPTH_ANYTHING_MODELS = {
-    "large": "https://huggingface.co/spaces/LiheYoung/Depth-Anything/resolve/main/checkpoints/depth_anything_vitl14.pth?download=true",
-    "base": "https://huggingface.co/spaces/LiheYoung/Depth-Anything/resolve/main/checkpoints/depth_anything_vitb14.pth?download=true",
-    "small": "https://huggingface.co/spaces/LiheYoung/Depth-Anything/resolve/main/checkpoints/depth_anything_vits14.pth?download=true",
+    "large": {
+        "url": "https://huggingface.co/spaces/LiheYoung/Depth-Anything/resolve/main/checkpoints/depth_anything_vitl14.pth?download=true",
+        "local": "any/annotators/depth_anything/depth_anything_vitl14.pth",
+    },
+    "base": {
+        "url": "https://huggingface.co/spaces/LiheYoung/Depth-Anything/resolve/main/checkpoints/depth_anything_vitb14.pth?download=true",
+        "local": "any/annotators/depth_anything/depth_anything_vitb14.pth",
+    },
+    "small": {
+        "url": "https://huggingface.co/spaces/LiheYoung/Depth-Anything/resolve/main/checkpoints/depth_anything_vits14.pth?download=true",
+        "local": "any/annotators/depth_anything/depth_anything_vits14.pth",
+    },
 }
 
 
@@ -42,27 +53,36 @@ transform = Compose(
 
 
 class DepthAnythingDetector:
-    def __init__(self, model: DPT_DINOv2, device: torch.device) -> None:
-        self.model = model
-        self.device = device
+    def __init__(self) -> None:
+        self.model = None
+        self.model_size: Union[Literal["large", "base", "small"], None] = None
+        self.device = TorchDevice.choose_torch_device()
 
-    @staticmethod
-    def load_model(
-        model_path: Path, device: torch.device, model_size: Literal["large", "base", "small"] = "small"
-    ) -> DPT_DINOv2:
-        match model_size:
-            case "small":
-                model = DPT_DINOv2(encoder="vits", features=64, out_channels=[48, 96, 192, 384])
-            case "base":
-                model = DPT_DINOv2(encoder="vitb", features=128, out_channels=[96, 192, 384, 768])
-            case "large":
-                model = DPT_DINOv2(encoder="vitl", features=256, out_channels=[256, 512, 1024, 1024])
+    def load_model(self, model_size: Literal["large", "base", "small"] = "small"):
+        DEPTH_ANYTHING_MODEL_PATH = config.models_path / DEPTH_ANYTHING_MODELS[model_size]["local"]
+        download_with_progress_bar(
+            pathlib.Path(DEPTH_ANYTHING_MODELS[model_size]["url"]).name,
+            DEPTH_ANYTHING_MODELS[model_size]["url"],
+            DEPTH_ANYTHING_MODEL_PATH,
+        )
 
-        model.load_state_dict(torch.load(model_path.as_posix(), map_location="cpu"))
-        model.eval()
+        if not self.model or model_size != self.model_size:
+            del self.model
+            self.model_size = model_size
 
-        model.to(device)
-        return model
+            match self.model_size:
+                case "small":
+                    self.model = DPT_DINOv2(encoder="vits", features=64, out_channels=[48, 96, 192, 384])
+                case "base":
+                    self.model = DPT_DINOv2(encoder="vitb", features=128, out_channels=[96, 192, 384, 768])
+                case "large":
+                    self.model = DPT_DINOv2(encoder="vitl", features=256, out_channels=[256, 512, 1024, 1024])
+
+            self.model.load_state_dict(torch.load(DEPTH_ANYTHING_MODEL_PATH.as_posix(), map_location="cpu"))
+            self.model.eval()
+
+        self.model.to(self.device)
+        return self.model
 
     def __call__(self, image: Image.Image, resolution: int = 512) -> Image.Image:
         if not self.model:

invokeai/backend/image_util/depth_anything/model/dpt.py

@@ -4,7 +4,7 @@ import torch
 import torch.nn as nn
 import torch.nn.functional as F
 
-from invokeai.backend.image_util.depth_anything.model.blocks import FeatureFusionBlock, _make_scratch
+from .blocks import FeatureFusionBlock, _make_scratch
 
 torchhub_path = Path(__file__).parent.parent / "torchhub"