Clean up erroniously added lines

## What type of PR is this? (check all applicable)

- [x] Refactor
- [x] Feature
- [x] Bug Fix
- [?] Optimization


## Have you discussed this change with the InvokeAI team?
- [x] No

     
## Description

- Fixed filter type select using `images` instead of `all` -- Probably
some merge conflict.
- Added loading state for model lists. Handy when the model list takes
longer than a second for any reason to fetch. Better to show this than
an empty screen.
- Refactored the render model list function so we modify the display
component in one area rather than have repeated code.

### Other Issues

- The list can get a bit laggy on initial load when you have a hundreds
of models / loras. This needs to be fixed. Will make another PR for
this.

.github/workflows/lint-frontend.yml

@@ -2,8 +2,6 @@ name: Lint frontend
 
 on:
   pull_request:
-    paths:
-      - 'invokeai/frontend/web/**'
     types:
       - 'ready_for_review'
       - 'opened'
@@ -11,8 +9,6 @@ on:
   push:
     branches:
       - 'main'
-    paths:
-      - 'invokeai/frontend/web/**'
   merge_group:
   workflow_dispatch:
 

docs/contributing/contribution_guides/development.md

@@ -16,7 +16,7 @@ If you don't feel ready to make a code contribution yet, no problem! You can als
 There are two paths to making a development contribution: 
 
 1. Choosing an open issue to address. Open issues can be found in the [Issues](https://github.com/invoke-ai/InvokeAI/issues?q=is%3Aissue+is%3Aopen) section of the InvokeAI repository. These are tagged by the issue type (bug, enhancement, etc.) along with the “good first issues” tag denoting if they are suitable for first time contributors.
-    1. Additional items can be found on our roadmap <******************************link to roadmap>******************************. The roadmap is organized in terms of priority, and contains features of varying size and complexity. If there is an inflight item you’d like to help with, reach out to the contributor assigned to the item to see how you can help. 
+    1. Additional items can be found on our [roadmap](https://github.com/orgs/invoke-ai/projects/7). The roadmap is organized in terms of priority, and contains features of varying size and complexity. If there is an inflight item you’d like to help with, reach out to the contributor assigned to the item to see how you can help. 
 2. Opening a new issue or feature to add. **Please make sure you have searched through existing issues before creating new ones.**
 
 *Regardless of what you choose, please post in the  [#dev-chat](https://discord.com/channels/1020123559063990373/1049495067846524939) channel of the Discord before you start development in order to confirm that the issue or feature is aligned with the current direction of the project. We value our contributors time and effort and want to ensure that no one’s time is being misspent.*

docs/features/index.md

@@ -4,6 +4,9 @@ title: Overview
 
 Here you can find the documentation for InvokeAI's various features.
 
+## The [Getting Started Guide](../help/gettingStartedWithAI) 
+A getting started guide for those new to AI image generation. 
+
 ## The Basics
 ### * The [Web User Interface](WEB.md)
 Guide to the Web interface. Also see the [WebUI Hotkeys Reference Guide](WEBUIHOTKEYS.md)
@@ -46,7 +49,7 @@ Personalize models by adding your own style or subjects.
 
 ## Other Features
 
-### * [The NSFW Checker](NSFW.md)
+### * [The NSFW Checker](WATERMARK+NSFW.md)
 Prevent InvokeAI from displaying unwanted racy images.
 
 ### * [Controlling Logging](LOGGING.md)

docs/help/gettingStartedWithAI.md

@@ -0,0 +1,95 @@
+# Getting Started with AI Image Generation
+
+New to image generation with AI? You’re in the right place! 
+
+This is a high level walkthrough of some of the concepts and terms you’ll see as you start using InvokeAI. Please note, this is not an exhaustive guide and may be out of date due to the rapidly changing nature of the space. 
+
+## Using InvokeAI
+
+### **Prompt Crafting**
+
+- Prompts are the basis of using InvokeAI, providing the models directions on what to generate. As a general rule of thumb, the more detailed your prompt is, the better your result will be.
+
+ *To get started, here’s an easy template to use for structuring your prompts:*
+
+- Subject, Style, Quality, Aesthetic
+    - **Subject:** What your image will be about. E.g. “a futuristic city with trains”,  “penguins floating on icebergs”, “friends sharing beers”
+    - **Style:** The style or medium in which your image will be in. E.g. “photograph”, “pencil sketch”, “oil paints”, or “pop art”, “cubism”, “abstract”
+    - **Quality:** A particular aspect or trait that you would like to see emphasized in your image. E.g. "award-winning", "featured in {relevant set of high quality works}", "professionally acclaimed". Many people often use "masterpiece".
+    - **Aesthetics:** The visual impact and design of the artwork. This can be colors, mood, lighting, setting, etc.
+- There are two prompt boxes: *Positive Prompt* & *Negative Prompt*.
+    - A **Positive** Prompt includes words you want the model to reference when creating an image.
+    - Negative Prompt is for anything you want the model to eliminate when creating an image. It doesn’t always interpret things exactly the way you would, but helps control the generation process. Always try to include a few terms - you can typically use lower quality image terms like “blurry” or “distorted” with good success.
+- Some examples prompts you can try on your own:
+    - A detailed oil painting of a tranquil forest at sunset with vibrant+ colors and soft, golden light filtering through the trees
+    - friends sharing beers in a busy city, realistic colored pencil sketch, twilight, masterpiece, bright, lively
+
+### Generation Workflows
+
+- Invoke offers a number of different workflows for interacting with models to produce images. Each is extremely powerful on its own, but together provide you an unparalleled way of producing high quality creative outputs that align with your vision.
+    - **Text to Image:** The text to image tab focuses on the key workflow of using a prompt to generate a new image. It includes other features that help control the generation process as well.
+    - **Image to Image:** With image to image, you provide an image as a reference (called the “initial image”), which provides more guidance around color and structure to the AI as it generates a new image. This is provided alongside the same features as Text to Image.
+    - **Unified Canvas:** The Unified Canvas is an advanced AI-first image editing tool that is easy to use, but hard to master. Drag an image onto the canvas from your gallery in order to regenerate certain elements, edit content or colors (known as inpainting), or extend the image with an exceptional degree of consistency and clarity (called outpainting).
+
+### Improving Image Quality
+
+- Fine tuning your prompt - the more specific you are, the closer the image will turn out to what is in your head!  Adding more details in the Positive Prompt or Negative Prompt can help add / remove pieces of your image to improve it - You can also use advanced techniques like upweighting and downweighting to control the influence of certain words. [Learn more here](https://invoke-ai.github.io/InvokeAI/features/PROMPTS/#prompt-syntax-features).
+    - **Tip: If you’re seeing poor results, try adding the things you don’t like about the image to your negative prompt may help. E.g. distorted, low quality, unrealistic, etc.**
+- Explore different models - Other models can produce different results due to the data they’ve been trained on. Each model has specific language and settings it works best with; a model’s documentation is your friend here.  Play around with some and see what works best for you!
+- Increasing Steps - The number of steps used controls how much time the model is given to produce an image, and depends on the “Scheduler” used. The schedule controls how each step is processed by the model. More steps tends to mean better results, but will take longer - We recommend at least 30 steps for most
+- Tweak and Iterate - Remember, it’s best to change one thing at a time so you know what is working and what isn't. Sometimes you just need to try a new image, and other times using a new prompt might be the ticket. For testing, consider turning off the “random” Seed - Using the same seed with the same settings will produce the same image, which makes it the perfect way to learn exactly what your changes are doing.
+- Explore Advanced Settings - InvokeAI has a full suite of tools available to allow you complete control over your image creation process - Check out our [docs if you want to learn more](https://invoke-ai.github.io/InvokeAI/features/).
+
+
+## Terms & Concepts
+
+If you're interested in learning more, check out [this presentation](https://docs.google.com/presentation/d/1IO78i8oEXFTZ5peuHHYkVF-Y3e2M6iM5tCnc-YBfcCM/edit?usp=sharing) from one of our maintainers (@lstein). 
+
+### Stable Diffusion
+
+Stable Diffusion is deep learning, text-to-image model that is the foundation of the capabilities found in InvokeAI. Since the release of Stable Diffusion, there have been many subsequent models created based on Stable Diffusion that are designed to generate specific types of images. 
+
+### Prompts
+
+Prompts provide the models directions on what to generate. As a general rule of thumb, the more detailed your prompt is, the better your result will be.
+
+### Models
+
+Models are the magic that power InvokeAI. These files represent the output of training a machine on understanding massive amounts of images - providing them with the capability to generate new images using just a text description of what you’d like to see. (Like Stable Diffusion!)
+
+Invoke offers a simple way to download several different models upon installation, but many more can be discovered online, including at ****. Each model can produce a unique style of output, based on the images it was trained on - Try out different models to see which best fits your creative vision!
+
+- *Models that contain “inpainting” in the name are designed for use with the inpainting feature of the Unified Canvas*
+
+### Scheduler
+
+Schedulers guide the process of removing noise (de-noising) from data. They determine:
+
+1. The number of steps to take to remove the noise.
+2. Whether the steps are random (stochastic) or predictable (deterministic).
+3. The specific method (algorithm) used for de-noising.
+
+Experimenting with different schedulers is recommended as each will produce different outputs!
+
+### Steps
+
+The number of de-noising steps each generation through. 
+
+Schedulers can be intricate and there's often a balance to strike between how quickly they can de-noise data and how well they can do it. It's typically advised to experiment with different schedulers to see which one gives the best results. There has been a lot written on the internet about different schedulers, as well as exploring what the right level of "steps" are for each. You can save generation time by reducing the number of steps used, but you'll want to make sure that you are satisfied with the quality of images produced!
+
+### Low-Rank Adaptations / LoRAs
+
+Low-Rank Adaptations (LoRAs) are like a smaller, more focused version of models, intended to focus on training a better understanding of how a specific character, style, or concept looks.
+
+### Textual Inversion Embeddings
+
+Textual Inversion Embeddings, like LoRAs, assist with more easily prompting for certain characters, styles, or concepts. However, embeddings are trained to update the relationship between a specific word (known as the “trigger”) and the intended output. 
+
+### ControlNet
+
+ControlNets are neural network models that are able to extract key features from an existing image and use these features to guide the output of the image generation model. 
+
+### VAE
+
+Variational auto-encoder (VAE) is a encode/decode model that translates the "latents" image produced during the image generation procees to the large pixel images that we see. 
+

docs/index.md

@@ -11,6 +11,33 @@ title: Home
   ```
 -->
 
+<!-- CSS styling -->
+<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@fortawesome/fontawesome-free@6.2.1/css/fontawesome.min.css">
+<style>
+    .button {
+      width: 300px;
+      height: 50px;
+      background-color: #448AFF;
+      color: #fff;
+      font-size: 16px;
+      border: none;
+      cursor: pointer;
+      border-radius: 0.2rem;
+    }
+
+    .button-container {
+      display: grid;
+      grid-template-columns: repeat(3, 300px);
+      gap: 20px;
+    }
+
+    .button:hover {
+      background-color: #526CFE;
+    }
+</style>
+
+
+
 <div align="center" markdown>
 
 
@@ -70,63 +97,23 @@ image-to-image generator. It provides a streamlined process with various new
 features and options to aid the image generation process. It runs on Windows,
 Mac and Linux machines, and runs on GPU cards with as little as 4 GB of RAM.
 
-**Quick links**: [<a href="https://discord.gg/ZmtBAhwWhy">Discord Server</a>]
-[<a href="https://github.com/invoke-ai/InvokeAI/">Code and Downloads</a>] [<a
-href="https://github.com/invoke-ai/InvokeAI/issues">Bug Reports</a>] [<a
-href="https://github.com/invoke-ai/InvokeAI/discussions">Discussion, Ideas &
-Q&A</a>]
-
 <div align="center"><img src="assets/invoke-web-server-1.png" width=640></div>
 
-!!! note
+!!! Note
 
-    This software is rapidly evolving. Please use the [Issues tab](https://github.com/invoke-ai/InvokeAI/issues) to report bugs and make feature requests. Be sure to use the provided templates. They will help aid diagnose issues faster.
+    This project is rapidly evolving. Please use the [Issues tab](https://github.com/invoke-ai/InvokeAI/issues) to report bugs and make feature requests. Be sure to use the provided templates as it will help aid response time.
 
-## :octicons-package-dependencies-24: Installation
+## :octicons-link-24: Quick Links
 
-This software is supported across Linux, Windows and Macintosh. Linux users can use
-either an Nvidia-based card (with CUDA support) or an AMD card (using the ROCm
-driver).
-
-### [Installation Getting Started Guide](installation)
-#### **[Automated Installer](installation/010_INSTALL_AUTOMATED.md)**
-✅ This is the recommended installation method for first-time users. 
-#### [Manual Installation](installation/020_INSTALL_MANUAL.md)
-This method is recommended for experienced users and developers
-#### [Docker Installation](installation/040_INSTALL_DOCKER.md)
-This method is recommended for those familiar with running Docker containers
-#### [Installation Troubleshooting](installation/010_INSTALL_AUTOMATED.md#troubleshooting)
-Installation troubleshooting guide.
-### Other Installation Guides
-  - [PyPatchMatch](installation/060_INSTALL_PATCHMATCH.md)
-  - [XFormers](installation/070_INSTALL_XFORMERS.md)
-  - [CUDA and ROCm Drivers](installation/030_INSTALL_CUDA_AND_ROCM.md)
-  - [Installing New Models](installation/050_INSTALLING_MODELS.md)
-
-## :fontawesome-solid-computer: Hardware Requirements
-
-### :octicons-cpu-24: System
-
-You wil need one of the following:
-
-- :simple-nvidia: An NVIDIA-based graphics card with 4 GB or more VRAM memory.
-- :simple-amd: An AMD-based graphics card with 4 GB or more VRAM memory (Linux
-  only)
-- :fontawesome-brands-apple: An Apple computer with an M1 chip.
-
-We do **not recommend** the following video cards due to issues with their
-running in half-precision mode and having insufficient VRAM to render 512x512
-images in full-precision mode:
-
-- NVIDIA 10xx series cards such as the 1080ti
-- GTX 1650 series cards
-- GTX 1660 series cards
-
-### :fontawesome-solid-memory: Memory and Disk
-
-- At least 12 GB Main Memory RAM.
-- At least 18 GB of free disk space for the machine learning model, Python, and
-  all its dependencies.
+<div class="button-container">
+    <a href="installation/INSTALLATION"> <button class="button">Installation</button> </a>
+    <a href="features/"> <button class="button">Features</button> </a>
+    <a href="help/gettingStartedWithAI/"> <button class="button">Getting Started</button> </a>
+    <a href="contributing/CONTRIBUTING/"> <button class="button">Contributing</button> </a>
+    <a href="https://github.com/invoke-ai/InvokeAI/"> <button class="button">Code and Downloads</button> </a>
+    <a href="https://github.com/invoke-ai/InvokeAI/issues"> <button class="button">Bug Reports </button> </a>
+    <a href="https://discord.gg/ZmtBAhwWhy"> <button class="button"> Join the Discord Server!</button> </a>
+</div>
 
 
 ## :octicons-gift-24: InvokeAI Features

docs/installation/INSTALLATION.md

@@ -1,6 +1,4 @@
----
-title: Overview
----
+# Overview
 
 We offer several ways to install InvokeAI, each one suited to your
 experience and preferences. We suggest that everyone start by
@@ -15,6 +13,56 @@ See the [troubleshooting
 section](010_INSTALL_AUTOMATED.md#troubleshooting) of the automated
 install guide for frequently-encountered installation issues.
 
+This fork is supported across Linux, Windows and Macintosh. Linux users can use
+either an Nvidia-based card (with CUDA support) or an AMD card (using the ROCm
+driver).
+
+### [Installation Getting Started Guide](installation)
+#### **[Automated Installer](010_INSTALL_AUTOMATED.md)**
+✅ This is the recommended installation method for first-time users. 
+#### [Manual Installation](020_INSTALL_MANUAL.md)
+This method is recommended for experienced users and developers
+#### [Docker Installation](040_INSTALL_DOCKER.md)
+This method is recommended for those familiar with running Docker containers
+### Other Installation Guides
+  - [PyPatchMatch](installation/060_INSTALL_PATCHMATCH.md)
+  - [XFormers](installation/070_INSTALL_XFORMERS.md)
+  - [CUDA and ROCm Drivers](installation/030_INSTALL_CUDA_AND_ROCM.md)
+  - [Installing New Models](installation/050_INSTALLING_MODELS.md)
+
+## :fontawesome-solid-computer: Hardware Requirements
+
+### :octicons-cpu-24: System
+
+You wil need one of the following:
+
+- :simple-nvidia: An NVIDIA-based graphics card with 4 GB or more VRAM memory.
+- :simple-amd: An AMD-based graphics card with 4 GB or more VRAM memory (Linux
+  only)
+- :fontawesome-brands-apple: An Apple computer with an M1 chip.
+
+** SDXL 1.0 Requirements*
+To use SDXL, user must have one of the following: 
+- :simple-nvidia: An NVIDIA-based graphics card with 8 GB or more VRAM memory.
+- :simple-amd: An AMD-based graphics card with 16 GB or more VRAM memory (Linux
+  only)
+- :fontawesome-brands-apple: An Apple computer with an M1 chip.
+
+
+### :fontawesome-solid-memory: Memory and Disk
+
+- At least 12 GB Main Memory RAM.
+- At least 18 GB of free disk space for the machine learning model, Python, and
+  all its dependencies.
+
+We do **not recommend** the following video cards due to issues with their
+running in half-precision mode and having insufficient VRAM to render 512x512
+images in full-precision mode:
+
+- NVIDIA 10xx series cards such as the 1080ti
+- GTX 1650 series cards
+- GTX 1660 series cards
+
 ## Installation options
 
 1. [Automated Installer](010_INSTALL_AUTOMATED.md)

docs/nodes/communityNodes.md

@@ -35,7 +35,7 @@ The nodes linked below have been developed and contributed by members of the Inv
 **Node Link:** https://github.com/JPPhoto/ideal-size-node
 
 --------------------------------
-### Super Cool Node Template
+### Example Node Template
 
 **Description:** This node allows you to do super cool things with InvokeAI.
 
@@ -45,7 +45,9 @@ The nodes linked below have been developed and contributed by members of the Inv
 
 **Output Examples** 
 
-![Invoke AI](https://invoke-ai.github.io/InvokeAI/assets/invoke_ai_banner.png)
+![Example Image](https://invoke-ai.github.io/InvokeAI/assets/invoke_ai_banner.png){: style="height:115px;width:240px"}
 
 ## Help
 If you run into any issues with a node, please post in the [InvokeAI Discord](https://discord.gg/ZmtBAhwWhy). 
+
+

flake.nix

@@ -34,6 +34,10 @@
             cudaPackages.cudnn
             cudaPackages.cuda_nvrtc
             cudatoolkit
+            pkgconfig
+            libconfig
+            cmake
+            blas
             freeglut
             glib
             gperf
@@ -42,6 +46,12 @@
             libGLU
             linuxPackages.nvidia_x11
             python
+            (opencv4.override {
+              enableGtk3 = true;
+              enableFfmpeg = true;
+              enableCuda = true;
+              enableUnfree = true;
+            })
             stdenv.cc
             stdenv.cc.cc.lib
             xorg.libX11

installer/lib/installer.py

@@ -455,7 +455,7 @@ def get_torch_source() -> (Union[str, None], str):
     device = graphical_accelerator()
 
     url = None
-    optional_modules = None
+    optional_modules = "[onnx]"
     if OS == "Linux":
         if device == "rocm":
             url = "https://download.pytorch.org/whl/rocm5.4.2"
@@ -464,7 +464,10 @@ def get_torch_source() -> (Union[str, None], str):
 
     if device == "cuda":
         url = "https://download.pytorch.org/whl/cu117"
-        optional_modules = "[xformers]"
+        optional_modules = "[xformers,onnx-cuda]"
+    if device == "cuda_and_dml":
+        url = "https://download.pytorch.org/whl/cu117"
+        optional_modules = "[xformers,onnx-directml]"
 
     # in all other cases, Torch wheels should be coming from PyPi as of Torch 1.13
 

installer/lib/messages.py

@@ -167,6 +167,10 @@ def graphical_accelerator():
         "an [gold1 b]NVIDIA[/] GPU (using CUDA™)",
         "cuda",
     )
+    nvidia_with_dml = (
+        "an [gold1 b]NVIDIA[/] GPU (using CUDA™, and DirectML™ for ONNX) -- ALPHA",
+        "cuda_and_dml",
+    )
     amd = (
         "an [gold1 b]AMD[/] GPU (using ROCm™)",
         "rocm",
@@ -181,7 +185,7 @@ def graphical_accelerator():
     )
 
     if OS == "Windows":
-        options = [nvidia, cpu]
+        options = [nvidia, nvidia_with_dml, cpu]
     if OS == "Linux":
         options = [nvidia, amd, cpu]
     elif OS == "Darwin":

invokeai/app/api/dependencies.py

@@ -1,5 +1,6 @@
 # Copyright (c) 2022 Kyle Schouviller (https://github.com/kyle0654)
 
+from typing import Optional
 from logging import Logger
 import os
 from invokeai.app.services.board_image_record_storage import (
@@ -29,6 +30,8 @@ from ..services.invoker import Invoker
 from ..services.processor import DefaultInvocationProcessor
 from ..services.sqlite import SqliteItemStorage
 from ..services.model_manager_service import ModelManagerService
+from ..services.batch_manager import BatchManager
+from ..services.batch_manager_storage import SqliteBatchProcessStorage
 from .events import FastAPIEventService
 
 
@@ -54,7 +57,7 @@ logger = InvokeAILogger.getLogger()
 class ApiDependencies:
     """Contains and initializes all dependencies for the API"""
 
-    invoker: Invoker = None
+    invoker: Optional[Invoker] = None
 
     @staticmethod
     def initialize(config: InvokeAIAppConfig, event_handler_id: int, logger: Logger = logger):
@@ -115,11 +118,15 @@ class ApiDependencies:
             )
         )
 
+        batch_manager_storage = SqliteBatchProcessStorage(db_location)
+        batch_manager = BatchManager(batch_manager_storage)
+
         services = InvocationServices(
             model_manager=ModelManagerService(config, logger),
             events=events,
             latents=latents,
             images=images,
+            batch_manager=batch_manager,
             boards=boards,
             board_images=board_images,
             queue=MemoryInvocationQueue(),

invokeai/app/api/routers/sessions.py

@@ -15,6 +15,7 @@ from ...services.graph import (
     GraphExecutionState,
     NodeAlreadyExecutedError,
 )
+from ...services.batch_manager import Batch, BatchProcess
 from ...services.item_storage import PaginatedResults
 from ..dependencies import ApiDependencies
 
@@ -37,6 +38,37 @@ async def create_session(
     return session
 
 
+@session_router.post(
+    "/batch",
+    operation_id="create_batch",
+    responses={
+        200: {"model": BatchProcess},
+        400: {"description": "Invalid json"},
+    },
+)
+async def create_batch(
+    graph: Optional[Graph] = Body(default=None, description="The graph to initialize the session with"),
+    batches: list[Batch] = Body(description="Batch config to apply to the given graph"),
+) -> BatchProcess:
+    """Creates and starts a new new batch process"""
+    batch_id = ApiDependencies.invoker.services.batch_manager.create_batch_process(batches, graph)
+    ApiDependencies.invoker.services.batch_manager.run_batch_process(batch_id)
+    return {"batch_id":batch_id}
+
+
+@session_router.delete(
+    "{batch_process_id}/batch",
+    operation_id="cancel_batch",
+    responses={202: {"description": "The batch is canceled"}},
+)
+async def cancel_batch(
+    batch_process_id: str = Path(description="The id of the batch process to cancel"),
+) -> Response:
+    """Creates and starts a new new batch process"""
+    ApiDependencies.invoker.services.batch_manager.cancel_batch_process(batch_process_id)
+    return Response(status_code=202)
+
+
 @session_router.get(
     "/",
     operation_id="list_sessions",

invokeai/app/cli_app.py

@@ -37,6 +37,8 @@ from invokeai.app.services.image_record_storage import SqliteImageRecordStorage
 from invokeai.app.services.images import ImageService, ImageServiceDependencies
 from invokeai.app.services.resource_name import SimpleNameService
 from invokeai.app.services.urls import LocalUrlService
+from invokeai.app.services.batch_manager import BatchManager
+from invokeai.app.services.batch_manager_storage import SqliteBatchProcessStorage
 from .services.default_graphs import default_text_to_image_graph_id, create_system_graphs
 from .services.latent_storage import DiskLatentsStorage, ForwardCacheLatentsStorage
 
@@ -300,12 +302,16 @@ def invoke_cli():
         )
     )
 
+    batch_manager_storage = SqliteBatchProcessStorage(db_location)
+    batch_manager = BatchManager(batch_manager_storage)
+
     services = InvocationServices(
         model_manager=model_manager,
         events=events,
         latents=ForwardCacheLatentsStorage(DiskLatentsStorage(f"{output_folder}/latents")),
         images=images,
         boards=boards,
+        batch_manager=batch_manager,
         board_images=board_images,
         queue=MemoryInvocationQueue(),
         graph_library=SqliteItemStorage[LibraryGraph](filename=db_location, table_name="graphs"),

invokeai/app/invocations/compel.py

@@ -1,6 +1,14 @@
 from typing import Literal, Optional, Union, List, Annotated
 from pydantic import BaseModel, Field
 import re
+
+from .baseinvocation import BaseInvocation, BaseInvocationOutput, InvocationContext, InvocationConfig
+from .model import ClipField
+
+from ...backend.util.devices import torch_dtype
+from ...backend.stable_diffusion.diffusion import InvokeAIDiffuserComponent
+from ...backend.model_management import BaseModelType, ModelType, SubModelType, ModelPatcher
+
 import torch
 from compel import Compel, ReturnedEmbeddingsType
 from compel.prompt_parser import Blend, Conjunction, CrossAttentionControlSubstitute, FlattenedPrompt, Fragment

invokeai/app/invocations/latent.py

@@ -24,6 +24,7 @@ from ...backend.stable_diffusion.diffusers_pipeline import (
 )
 from ...backend.stable_diffusion.diffusion.shared_invokeai_diffusion import PostprocessingSettings
 from ...backend.stable_diffusion.schedulers import SCHEDULER_MAP
+from ...backend.model_management import ModelPatcher
 from ...backend.util.devices import choose_torch_device, torch_dtype, choose_precision
 from ..models.image import ImageCategory, ImageField, ResourceOrigin
 from .baseinvocation import BaseInvocation, BaseInvocationOutput, InvocationConfig, InvocationContext

invokeai/app/invocations/model.py

@@ -53,6 +53,7 @@ class MainModelField(BaseModel):
 
     model_name: str = Field(description="Name of the model")
     base_model: BaseModelType = Field(description="Base model")
+    model_type: ModelType = Field(description="Model Type")
 
 
 class LoRAModelField(BaseModel):

invokeai/app/invocations/onnx.py

@@ -0,0 +1,578 @@
+# Copyright (c) 2023 Borisov Sergey (https://github.com/StAlKeR7779)
+
+from contextlib import ExitStack
+from typing import List, Literal, Optional, Union
+
+import re
+import inspect
+
+from pydantic import BaseModel, Field, validator
+import torch
+import numpy as np
+from diffusers import ControlNetModel, DPMSolverMultistepScheduler
+from diffusers.image_processor import VaeImageProcessor
+from diffusers.schedulers import SchedulerMixin as Scheduler
+
+from ..models.image import ImageCategory, ImageField, ResourceOrigin
+from ...backend.model_management import ONNXModelPatcher
+from ...backend.util import choose_torch_device
+from .baseinvocation import BaseInvocation, BaseInvocationOutput, InvocationConfig, InvocationContext
+from .compel import ConditioningField
+from .controlnet_image_processors import ControlField
+from .image import ImageOutput
+from .model import ModelInfo, UNetField, VaeField
+
+from invokeai.app.invocations.metadata import CoreMetadata
+from invokeai.backend import BaseModelType, ModelType, SubModelType
+from invokeai.app.util.step_callback import stable_diffusion_step_callback
+from ...backend.stable_diffusion import PipelineIntermediateState
+
+from tqdm import tqdm
+from .model import ClipField
+from .latent import LatentsField, LatentsOutput, build_latents_output, get_scheduler, SAMPLER_NAME_VALUES
+from .compel import CompelOutput
+
+
+ORT_TO_NP_TYPE = {
+    "tensor(bool)": np.bool_,
+    "tensor(int8)": np.int8,
+    "tensor(uint8)": np.uint8,
+    "tensor(int16)": np.int16,
+    "tensor(uint16)": np.uint16,
+    "tensor(int32)": np.int32,
+    "tensor(uint32)": np.uint32,
+    "tensor(int64)": np.int64,
+    "tensor(uint64)": np.uint64,
+    "tensor(float16)": np.float16,
+    "tensor(float)": np.float32,
+    "tensor(double)": np.float64,
+}
+
+PRECISION_VALUES = Literal[tuple(list(ORT_TO_NP_TYPE.keys()))]
+
+
+class ONNXPromptInvocation(BaseInvocation):
+    type: Literal["prompt_onnx"] = "prompt_onnx"
+
+    prompt: str = Field(default="", description="Prompt")
+    clip: ClipField = Field(None, description="Clip to use")
+
+    def invoke(self, context: InvocationContext) -> CompelOutput:
+        tokenizer_info = context.services.model_manager.get_model(
+            **self.clip.tokenizer.dict(),
+        )
+        text_encoder_info = context.services.model_manager.get_model(
+            **self.clip.text_encoder.dict(),
+        )
+        with tokenizer_info as orig_tokenizer, text_encoder_info as text_encoder, ExitStack() as stack:
+            # loras = [(stack.enter_context(context.services.model_manager.get_model(**lora.dict(exclude={"weight"}))), lora.weight) for lora in self.clip.loras]
+            loras = [
+                (context.services.model_manager.get_model(**lora.dict(exclude={"weight"})).context.model, lora.weight)
+                for lora in self.clip.loras
+            ]
+
+            ti_list = []
+            for trigger in re.findall(r"<[a-zA-Z0-9., _-]+>", self.prompt):
+                name = trigger[1:-1]
+                try:
+                    ti_list.append(
+                        # stack.enter_context(
+                        #    context.services.model_manager.get_model(
+                        #        model_name=name,
+                        #        base_model=self.clip.text_encoder.base_model,
+                        #        model_type=ModelType.TextualInversion,
+                        #    )
+                        # )
+                        context.services.model_manager.get_model(
+                            model_name=name,
+                            base_model=self.clip.text_encoder.base_model,
+                            model_type=ModelType.TextualInversion,
+                        ).context.model
+                    )
+                except Exception:
+                    # print(e)
+                    # import traceback
+                    # print(traceback.format_exc())
+                    print(f'Warn: trigger: "{trigger}" not found')
+            if loras or ti_list:
+                text_encoder.release_session()
+            with ONNXModelPatcher.apply_lora_text_encoder(text_encoder, loras), ONNXModelPatcher.apply_ti(
+                orig_tokenizer, text_encoder, ti_list
+            ) as (tokenizer, ti_manager):
+                text_encoder.create_session()
+
+                # copy from
+                # https://github.com/huggingface/diffusers/blob/3ebbaf7c96801271f9e6c21400033b6aa5ffcf29/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion.py#L153
+                text_inputs = tokenizer(
+                    self.prompt,
+                    padding="max_length",
+                    max_length=tokenizer.model_max_length,
+                    truncation=True,
+                    return_tensors="np",
+                )
+                text_input_ids = text_inputs.input_ids
+                """
+                untruncated_ids = tokenizer(prompt, padding="max_length", return_tensors="np").input_ids
+
+                if not np.array_equal(text_input_ids, untruncated_ids):
+                    removed_text = self.tokenizer.batch_decode(
+                        untruncated_ids[:, self.tokenizer.model_max_length - 1 : -1]
+                    )
+                    logger.warning(
+                        "The following part of your input was truncated because CLIP can only handle sequences up to"
+                        f" {self.tokenizer.model_max_length} tokens: {removed_text}"
+                    )
+                """
+
+                prompt_embeds = text_encoder(input_ids=text_input_ids.astype(np.int32))[0]
+
+        conditioning_name = f"{context.graph_execution_state_id}_{self.id}_conditioning"
+
+        # TODO: hacky but works ;D maybe rename latents somehow?
+        context.services.latents.save(conditioning_name, (prompt_embeds, None))
+
+        return CompelOutput(
+            conditioning=ConditioningField(
+                conditioning_name=conditioning_name,
+            ),
+        )
+
+
+# Text to image
+class ONNXTextToLatentsInvocation(BaseInvocation):
+    """Generates latents from conditionings."""
+
+    type: Literal["t2l_onnx"] = "t2l_onnx"
+
+    # Inputs
+    # fmt: off
+    positive_conditioning: Optional[ConditioningField] = Field(description="Positive conditioning for generation")
+    negative_conditioning: Optional[ConditioningField] = Field(description="Negative conditioning for generation")
+    noise: Optional[LatentsField] = Field(description="The noise to use")
+    steps:       int = Field(default=10, gt=0, description="The number of steps to use to generate the image")
+    cfg_scale: Union[float, List[float]] = Field(default=7.5, ge=1, description="The Classifier-Free Guidance, higher values may result in a result closer to the prompt", )
+    scheduler: SAMPLER_NAME_VALUES = Field(default="euler", description="The scheduler to use" )
+    precision: PRECISION_VALUES = Field(default = "tensor(float16)", description="The precision to use when generating latents")
+    unet: UNetField = Field(default=None, description="UNet submodel")
+    control: Union[ControlField, list[ControlField]] = Field(default=None, description="The control to use")
+    # seamless:   bool = Field(default=False, description="Whether or not to generate an image that can tile without seams", )
+    # seamless_axes: str = Field(default="", description="The axes to tile the image on, 'x' and/or 'y'")
+    # fmt: on
+
+    @validator("cfg_scale")
+    def ge_one(cls, v):
+        """validate that all cfg_scale values are >= 1"""
+        if isinstance(v, list):
+            for i in v:
+                if i < 1:
+                    raise ValueError("cfg_scale must be greater than 1")
+        else:
+            if v < 1:
+                raise ValueError("cfg_scale must be greater than 1")
+        return v
+
+    # Schema customisation
+    class Config(InvocationConfig):
+        schema_extra = {
+            "ui": {
+                "tags": ["latents"],
+                "type_hints": {
+                    "model": "model",
+                    "control": "control",
+                    # "cfg_scale": "float",
+                    "cfg_scale": "number",
+                },
+            },
+        }
+
+    # based on
+    # https://github.com/huggingface/diffusers/blob/3ebbaf7c96801271f9e6c21400033b6aa5ffcf29/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion.py#L375
+    def invoke(self, context: InvocationContext) -> LatentsOutput:
+        c, _ = context.services.latents.get(self.positive_conditioning.conditioning_name)
+        uc, _ = context.services.latents.get(self.negative_conditioning.conditioning_name)
+        graph_execution_state = context.services.graph_execution_manager.get(context.graph_execution_state_id)
+        source_node_id = graph_execution_state.prepared_source_mapping[self.id]
+        if isinstance(c, torch.Tensor):
+            c = c.cpu().numpy()
+        if isinstance(uc, torch.Tensor):
+            uc = uc.cpu().numpy()
+        device = torch.device(choose_torch_device())
+        prompt_embeds = np.concatenate([uc, c])
+
+        latents = context.services.latents.get(self.noise.latents_name)
+        if isinstance(latents, torch.Tensor):
+            latents = latents.cpu().numpy()
+
+        # TODO: better execution device handling
+        latents = latents.astype(ORT_TO_NP_TYPE[self.precision])
+
+        # get the initial random noise unless the user supplied it
+        do_classifier_free_guidance = True
+        # latents_dtype = prompt_embeds.dtype
+        # latents_shape = (batch_size * num_images_per_prompt, 4, height // 8, width // 8)
+        # if latents.shape != latents_shape:
+        #    raise ValueError(f"Unexpected latents shape, got {latents.shape}, expected {latents_shape}")
+
+        scheduler = get_scheduler(
+            context=context,
+            scheduler_info=self.unet.scheduler,
+            scheduler_name=self.scheduler,
+        )
+
+        def torch2numpy(latent: torch.Tensor):
+            return latent.cpu().numpy()
+
+        def numpy2torch(latent, device):
+            return torch.from_numpy(latent).to(device)
+
+        def dispatch_progress(
+            self, context: InvocationContext, source_node_id: str, intermediate_state: PipelineIntermediateState
+        ) -> None:
+            stable_diffusion_step_callback(
+                context=context,
+                intermediate_state=intermediate_state,
+                node=self.dict(),
+                source_node_id=source_node_id,
+            )
+
+        scheduler.set_timesteps(self.steps)
+        latents = latents * np.float64(scheduler.init_noise_sigma)
+
+        extra_step_kwargs = dict()
+        if "eta" in set(inspect.signature(scheduler.step).parameters.keys()):
+            extra_step_kwargs.update(
+                eta=0.0,
+            )
+
+        unet_info = context.services.model_manager.get_model(**self.unet.unet.dict())
+
+        with unet_info as unet, ExitStack() as stack:
+            # loras = [(stack.enter_context(context.services.model_manager.get_model(**lora.dict(exclude={"weight"}))), lora.weight) for lora in self.unet.loras]
+            loras = [
+                (context.services.model_manager.get_model(**lora.dict(exclude={"weight"})).context.model, lora.weight)
+                for lora in self.unet.loras
+            ]
+
+            if loras:
+                unet.release_session()
+            with ONNXModelPatcher.apply_lora_unet(unet, loras):
+                # TODO:
+                _, _, h, w = latents.shape
+                unet.create_session(h, w)
+
+                timestep_dtype = next(
+                    (input.type for input in unet.session.get_inputs() if input.name == "timestep"), "tensor(float16)"
+                )
+                timestep_dtype = ORT_TO_NP_TYPE[timestep_dtype]
+                for i in tqdm(range(len(scheduler.timesteps))):
+                    t = scheduler.timesteps[i]
+                    # expand the latents if we are doing classifier free guidance
+                    latent_model_input = np.concatenate([latents] * 2) if do_classifier_free_guidance else latents
+                    latent_model_input = scheduler.scale_model_input(numpy2torch(latent_model_input, device), t)
+                    latent_model_input = latent_model_input.cpu().numpy()
+
+                    # predict the noise residual
+                    timestep = np.array([t], dtype=timestep_dtype)
+                    noise_pred = unet(sample=latent_model_input, timestep=timestep, encoder_hidden_states=prompt_embeds)
+                    noise_pred = noise_pred[0]
+
+                    # perform guidance
+                    if do_classifier_free_guidance:
+                        noise_pred_uncond, noise_pred_text = np.split(noise_pred, 2)
+                        noise_pred = noise_pred_uncond + self.cfg_scale * (noise_pred_text - noise_pred_uncond)
+
+                    # compute the previous noisy sample x_t -> x_t-1
+                    scheduler_output = scheduler.step(
+                        numpy2torch(noise_pred, device), t, numpy2torch(latents, device), **extra_step_kwargs
+                    )
+                    latents = torch2numpy(scheduler_output.prev_sample)
+
+                    state = PipelineIntermediateState(
+                        run_id="test", step=i, timestep=timestep, latents=scheduler_output.prev_sample
+                    )
+                    dispatch_progress(self, context=context, source_node_id=source_node_id, intermediate_state=state)
+
+                    # call the callback, if provided
+                    # if callback is not None and i % callback_steps == 0:
+                    #    callback(i, t, latents)
+
+        torch.cuda.empty_cache()
+
+        name = f"{context.graph_execution_state_id}__{self.id}"
+        context.services.latents.save(name, latents)
+        return build_latents_output(latents_name=name, latents=torch.from_numpy(latents))
+
+
+# Latent to image
+class ONNXLatentsToImageInvocation(BaseInvocation):
+    """Generates an image from latents."""
+
+    type: Literal["l2i_onnx"] = "l2i_onnx"
+
+    # Inputs
+    latents: Optional[LatentsField] = Field(description="The latents to generate an image from")
+    vae: VaeField = Field(default=None, description="Vae submodel")
+    metadata: Optional[CoreMetadata] = Field(
+        default=None, description="Optional core metadata to be written to the image"
+    )
+    # tiled: bool = Field(default=False, description="Decode latents by overlaping tiles(less memory consumption)")
+
+    # Schema customisation
+    class Config(InvocationConfig):
+        schema_extra = {
+            "ui": {
+                "tags": ["latents", "image"],
+            },
+        }
+
+    def invoke(self, context: InvocationContext) -> ImageOutput:
+        latents = context.services.latents.get(self.latents.latents_name)
+
+        if self.vae.vae.submodel != SubModelType.VaeDecoder:
+            raise Exception(f"Expected vae_decoder, found: {self.vae.vae.model_type}")
+
+        vae_info = context.services.model_manager.get_model(
+            **self.vae.vae.dict(),
+        )
+
+        # clear memory as vae decode can request a lot
+        torch.cuda.empty_cache()
+
+        with vae_info as vae:
+            vae.create_session()
+
+            # copied from
+            # https://github.com/huggingface/diffusers/blob/3ebbaf7c96801271f9e6c21400033b6aa5ffcf29/src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion.py#L427
+            latents = 1 / 0.18215 * latents
+            # image = self.vae_decoder(latent_sample=latents)[0]
+            # it seems likes there is a strange result for using half-precision vae decoder if batchsize>1
+            image = np.concatenate([vae(latent_sample=latents[i : i + 1])[0] for i in range(latents.shape[0])])
+
+            image = np.clip(image / 2 + 0.5, 0, 1)
+            image = image.transpose((0, 2, 3, 1))
+            image = VaeImageProcessor.numpy_to_pil(image)[0]
+
+        torch.cuda.empty_cache()
+
+        image_dto = context.services.images.create(
+            image=image,
+            image_origin=ResourceOrigin.INTERNAL,
+            image_category=ImageCategory.GENERAL,
+            node_id=self.id,
+            session_id=context.graph_execution_state_id,
+            is_intermediate=self.is_intermediate,
+            metadata=self.metadata.dict() if self.metadata else None,
+        )
+
+        return ImageOutput(
+            image=ImageField(image_name=image_dto.image_name),
+            width=image_dto.width,
+            height=image_dto.height,
+        )
+
+
+class ONNXModelLoaderOutput(BaseInvocationOutput):
+    """Model loader output"""
+
+    # fmt: off
+    type: Literal["model_loader_output_onnx"] = "model_loader_output_onnx"
+
+    unet: UNetField = Field(default=None, description="UNet submodel")
+    clip: ClipField = Field(default=None, description="Tokenizer and text_encoder submodels")
+    vae_decoder: VaeField = Field(default=None, description="Vae submodel")
+    vae_encoder: VaeField = Field(default=None, description="Vae submodel")
+    # fmt: on
+
+
+class ONNXSD1ModelLoaderInvocation(BaseInvocation):
+    """Loading submodels of selected model."""
+
+    type: Literal["sd1_model_loader_onnx"] = "sd1_model_loader_onnx"
+
+    model_name: str = Field(default="", description="Model to load")
+    # TODO: precision?
+
+    # Schema customisation
+    class Config(InvocationConfig):
+        schema_extra = {
+            "ui": {"tags": ["model", "loader"], "type_hints": {"model_name": "model"}},  # TODO: rename to model_name?
+        }
+
+    def invoke(self, context: InvocationContext) -> ONNXModelLoaderOutput:
+        model_name = "stable-diffusion-v1-5"
+        base_model = BaseModelType.StableDiffusion1
+
+        # TODO: not found exceptions
+        if not context.services.model_manager.model_exists(
+            model_name=model_name,
+            base_model=BaseModelType.StableDiffusion1,
+            model_type=ModelType.ONNX,
+        ):
+            raise Exception(f"Unkown model name: {model_name}!")
+
+        return ONNXModelLoaderOutput(
+            unet=UNetField(
+                unet=ModelInfo(
+                    model_name=model_name,
+                    base_model=base_model,
+                    model_type=ModelType.ONNX,
+                    submodel=SubModelType.UNet,
+                ),
+                scheduler=ModelInfo(
+                    model_name=model_name,
+                    base_model=base_model,
+                    model_type=ModelType.ONNX,
+                    submodel=SubModelType.Scheduler,
+                ),
+                loras=[],
+            ),
+            clip=ClipField(
+                tokenizer=ModelInfo(
+                    model_name=model_name,
+                    base_model=base_model,
+                    model_type=ModelType.ONNX,
+                    submodel=SubModelType.Tokenizer,
+                ),
+                text_encoder=ModelInfo(
+                    model_name=model_name,
+                    base_model=base_model,
+                    model_type=ModelType.ONNX,
+                    submodel=SubModelType.TextEncoder,
+                ),
+                loras=[],
+            ),
+            vae_decoder=VaeField(
+                vae=ModelInfo(
+                    model_name=model_name,
+                    base_model=base_model,
+                    model_type=ModelType.ONNX,
+                    submodel=SubModelType.VaeDecoder,
+                ),
+            ),
+            vae_encoder=VaeField(
+                vae=ModelInfo(
+                    model_name=model_name,
+                    base_model=base_model,
+                    model_type=ModelType.ONNX,
+                    submodel=SubModelType.VaeEncoder,
+                ),
+            ),
+        )
+
+
+class OnnxModelField(BaseModel):
+    """Onnx model field"""
+
+    model_name: str = Field(description="Name of the model")
+    base_model: BaseModelType = Field(description="Base model")
+    model_type: ModelType = Field(description="Model Type")
+
+
+class OnnxModelLoaderInvocation(BaseInvocation):
+    """Loads a main model, outputting its submodels."""
+
+    type: Literal["onnx_model_loader"] = "onnx_model_loader"
+
+    model: OnnxModelField = Field(description="The model to load")
+
+    # Schema customisation
+    class Config(InvocationConfig):
+        schema_extra = {
+            "ui": {
+                "title": "Onnx Model Loader",
+                "tags": ["model", "loader"],
+                "type_hints": {"model": "model"},
+            },
+        }
+
+    def invoke(self, context: InvocationContext) -> ONNXModelLoaderOutput:
+        base_model = self.model.base_model
+        model_name = self.model.model_name
+        model_type = ModelType.ONNX
+
+        # TODO: not found exceptions
+        if not context.services.model_manager.model_exists(
+            model_name=model_name,
+            base_model=base_model,
+            model_type=model_type,
+        ):
+            raise Exception(f"Unknown {base_model} {model_type} model: {model_name}")
+
+        """
+        if not context.services.model_manager.model_exists(
+            model_name=self.model_name,
+            model_type=SDModelType.Diffusers,
+            submodel=SDModelType.Tokenizer,
+        ):
+            raise Exception(
+                f"Failed to find tokenizer submodel in {self.model_name}! Check if model corrupted"
+            )
+
+        if not context.services.model_manager.model_exists(
+            model_name=self.model_name,
+            model_type=SDModelType.Diffusers,
+            submodel=SDModelType.TextEncoder,
+        ):
+            raise Exception(
+                f"Failed to find text_encoder submodel in {self.model_name}! Check if model corrupted"
+            )
+
+        if not context.services.model_manager.model_exists(
+            model_name=self.model_name,
+            model_type=SDModelType.Diffusers,
+            submodel=SDModelType.UNet,
+        ):
+            raise Exception(
+                f"Failed to find unet submodel from {self.model_name}! Check if model corrupted"
+            )
+        """
+
+        return ONNXModelLoaderOutput(
+            unet=UNetField(
+                unet=ModelInfo(
+                    model_name=model_name,
+                    base_model=base_model,
+                    model_type=model_type,
+                    submodel=SubModelType.UNet,
+                ),
+                scheduler=ModelInfo(
+                    model_name=model_name,
+                    base_model=base_model,
+                    model_type=model_type,
+                    submodel=SubModelType.Scheduler,
+                ),
+                loras=[],
+            ),
+            clip=ClipField(
+                tokenizer=ModelInfo(
+                    model_name=model_name,
+                    base_model=base_model,
+                    model_type=model_type,
+                    submodel=SubModelType.Tokenizer,
+                ),
+                text_encoder=ModelInfo(
+                    model_name=model_name,
+                    base_model=base_model,
+                    model_type=model_type,
+                    submodel=SubModelType.TextEncoder,
+                ),
+                loras=[],
+                skipped_layers=0,
+            ),
+            vae_decoder=VaeField(
+                vae=ModelInfo(
+                    model_name=model_name,
+                    base_model=base_model,
+                    model_type=model_type,
+                    submodel=SubModelType.VaeDecoder,
+                ),
+            ),
+            vae_encoder=VaeField(
+                vae=ModelInfo(
+                    model_name=model_name,
+                    base_model=base_model,
+                    model_type=model_type,
+                    submodel=SubModelType.VaeEncoder,
+                ),
+            ),
+        )

invokeai/app/services/batch_manager.py

@@ -0,0 +1,139 @@
+import networkx as nx
+import copy
+
+from abc import ABC, abstractmethod
+from itertools import product
+from pydantic import BaseModel, Field
+from fastapi_events.handlers.local import local_handler
+from fastapi_events.typing import Event
+
+from invokeai.app.services.events import EventServiceBase
+from invokeai.app.services.graph import Graph, GraphExecutionState
+from invokeai.app.services.invoker import Invoker
+from invokeai.app.services.batch_manager_storage import (
+    BatchProcessStorageBase,
+    BatchSessionNotFoundException,
+    Batch,
+    BatchProcess,
+    BatchSession,
+    BatchSessionChanges,
+)
+
+
+class BatchManagerBase(ABC):
+    @abstractmethod
+    def start(self, invoker: Invoker):
+        pass
+
+    @abstractmethod
+    def create_batch_process(self, batches: list[Batch], graph: Graph) -> str:
+        pass
+
+    @abstractmethod
+    def run_batch_process(self, batch_id: str):
+        pass
+
+    @abstractmethod
+    def cancel_batch_process(self, batch_process_id: str):
+        pass
+
+
+class BatchManager(BatchManagerBase):
+    """Responsible for managing currently running and scheduled batch jobs"""
+
+    __invoker: Invoker
+    __batches: list[BatchProcess]
+    __batch_process_storage: BatchProcessStorageBase
+
+    def __init__(self, batch_process_storage: BatchProcessStorageBase) -> None:
+        super().__init__()
+        self.__batch_process_storage = batch_process_storage
+
+    def start(self, invoker: Invoker) -> None:
+        # if we do want multithreading at some point, we could make this configurable
+        self.__invoker = invoker
+        self.__batches = list()
+        local_handler.register(event_name=EventServiceBase.session_event, _func=self.on_event)
+
+    async def on_event(self, event: Event):
+        event_name = event[1]["event"]
+
+        match event_name:
+            case "graph_execution_state_complete":
+                await self.process(event, False)
+            case "invocation_error":
+                await self.process(event, True)
+
+        return event
+
+    async def process(self, event: Event, err: bool):
+        data = event[1]["data"]
+        batch_session = self.__batch_process_storage.get_session(data["graph_execution_state_id"])
+        if not batch_session:
+            return
+        updateSession = BatchSessionChanges(
+            state='error' if err else 'completed'
+        )
+        batch_session = self.__batch_process_storage.update_session_state(
+            batch_session.batch_id,
+            batch_session.session_id,
+            updateSession,
+        )
+        self.run_batch_process(batch_session.batch_id)
+
+    def _create_batch_session(self, batch_process: BatchProcess, batch_indices: list[int]) -> GraphExecutionState:
+        graph = copy.deepcopy(batch_process.graph)
+        batches = batch_process.batches
+        g = graph.nx_graph_flat()
+        sorted_nodes = nx.topological_sort(g)
+        for npath in sorted_nodes:
+            node = graph.get_node(npath)
+            (index, batch) = next(((i, b) for i, b in enumerate(batches) if b.node_id in node.id), (None, None))
+            if batch:
+                batch_index = batch_indices[index]
+                datum = batch.data[batch_index]
+                for key in datum:
+                    node.__dict__[key] = datum[key]
+                graph.update_node(npath, node)
+
+        return GraphExecutionState(graph=graph)
+
+    def run_batch_process(self, batch_id: str):
+        try: 
+            created_session = self.__batch_process_storage.get_created_session(batch_id)
+        except BatchSessionNotFoundException:
+            return
+        ges = self.__invoker.services.graph_execution_manager.get(created_session.session_id)
+        self.__invoker.invoke(ges, invoke_all=True)
+    
+    def _valid_batch_config(self, batch_process: BatchProcess) -> bool:
+        return True
+    
+    def create_batch_process(self, batches: list[Batch], graph: Graph) -> str:
+        batch_process = BatchProcess(
+            batches=batches,
+            graph=graph,
+        )
+        if not self._valid_batch_config(batch_process):
+            return None
+        batch_process = self.__batch_process_storage.save(batch_process)
+        self._create_sessions(batch_process)
+        return batch_process.batch_id
+    
+    def _create_sessions(self, batch_process: BatchProcess):
+        batch_indices = list()
+        for batch in batch_process.batches:
+            batch_indices.append(list(range(len(batch.data))))
+        all_batch_indices = product(*batch_indices)
+        for bi in all_batch_indices:
+            ges = self._create_batch_session(batch_process, bi)
+            self.__invoker.services.graph_execution_manager.set(ges)
+            batch_session = BatchSession(
+                batch_id=batch_process.batch_id,
+                session_id=ges.id,
+                state="created"
+            )
+            self.__batch_process_storage.create_session(batch_session)
+
+    def cancel_batch_process(self, batch_process_id: str):
+        self.__batches = [batch for batch in self.__batches if batch.id != batch_process_id]

invokeai/app/services/batch_manager_storage.py

@@ -0,0 +1,505 @@
+from abc import ABC, abstractmethod
+from typing import cast
+import uuid
+import sqlite3
+import threading
+from typing import (
+    Any,
+    List,
+    Literal,
+    Optional,
+    Union,
+)
+import json
+
+from invokeai.app.invocations.baseinvocation import (
+    BaseInvocation,
+)
+from invokeai.app.services.graph import Graph
+from invokeai.app.models.image import ImageField
+
+from pydantic import BaseModel, Field, Extra, parse_raw_as
+
+invocations = BaseInvocation.get_invocations()
+InvocationsUnion = Union[invocations]  # type: ignore
+
+BatchDataType = Union[str, int, float, ImageField]
+
+class Batch(BaseModel):
+    data: list[dict[str, BatchDataType]] = Field(description="Mapping of node field to data value")
+    node_id: str = Field(description="ID of the node to batch")
+
+
+class BatchSession(BaseModel):
+    batch_id: str = Field(description="Identifier for which batch this Index belongs to")
+    session_id: str = Field(description="Session ID Created for this Batch Index")
+    state: Literal["created", "completed", "inprogress", "error"] = Field(
+        description="Is this session created, completed, in progress, or errored?"
+    )
+
+
+def uuid_string():
+    res = uuid.uuid4()
+    return str(res)
+
+class BatchProcess(BaseModel):
+    batch_id: Optional[str] = Field(default_factory=uuid_string, description="Identifier for this batch")
+    batches: List[Batch] = Field(
+        description="List of batch configs to apply to this session",
+        default_factory=list,
+    )
+    graph: Graph = Field(description="The graph being executed")
+
+
+class BatchSessionChanges(BaseModel, extra=Extra.forbid):
+    state: Literal["created", "completed", "inprogress", "error"] = Field(
+        description="Is this session created, completed, in progress, or errored?"
+    )
+
+
+class BatchProcessNotFoundException(Exception):
+    """Raised when an Batch Process record is not found."""
+
+    def __init__(self, message="BatchProcess record not found"):
+        super().__init__(message)
+
+
+class BatchProcessSaveException(Exception):
+    """Raised when an Batch Process record cannot be saved."""
+
+    def __init__(self, message="BatchProcess record not saved"):
+        super().__init__(message)
+
+
+class BatchProcessDeleteException(Exception):
+    """Raised when an Batch Process record cannot be deleted."""
+
+    def __init__(self, message="BatchProcess record not deleted"):
+        super().__init__(message)
+
+
+class BatchSessionNotFoundException(Exception):
+    """Raised when an Batch Session record is not found."""
+
+    def __init__(self, message="BatchSession record not found"):
+        super().__init__(message)
+
+
+class BatchSessionSaveException(Exception):
+    """Raised when an Batch Session record cannot be saved."""
+
+    def __init__(self, message="BatchSession record not saved"):
+        super().__init__(message)
+
+
+class BatchSessionDeleteException(Exception):
+    """Raised when an Batch Session record cannot be deleted."""
+
+    def __init__(self, message="BatchSession record not deleted"):
+        super().__init__(message)
+
+
+class BatchProcessStorageBase(ABC):
+    """Low-level service responsible for interfacing with the Batch Process record store."""
+
+    @abstractmethod
+    def delete(self, batch_id: str) -> None:
+        """Deletes a Batch Process record."""
+        pass
+
+    @abstractmethod
+    def save(
+        self,
+        batch_process: BatchProcess,
+    ) -> BatchProcess:
+        """Saves a Batch Process record."""
+        pass
+
+    @abstractmethod
+    def get(
+        self,
+        batch_id: str,
+    ) -> BatchProcess:
+        """Gets a Batch Process record."""
+        pass
+
+    @abstractmethod
+    def create_session(
+            self,
+            session: BatchSession,
+    ) -> BatchSession:
+        """Creates a Batch Session attached to a Batch Process."""
+        pass
+    
+    @abstractmethod
+    def get_session(
+        self,
+        session_id: str
+    ) -> BatchSession:
+        """Gets session by session_id"""
+        pass
+
+    @abstractmethod
+    def get_created_session(
+        self,
+        batch_id: str
+    ) -> BatchSession:
+        """Gets all created Batch Sessions for a given Batch Process id."""
+        pass
+
+    @abstractmethod
+    def get_created_sessions(
+        self,
+        batch_id: str
+    ) -> List[BatchSession]:
+        """Gets all created Batch Sessions for a given Batch Process id."""
+        pass
+
+    @abstractmethod
+    def update_session_state(
+        self,
+        batch_id: str,
+        session_id: str,
+        changes: BatchSessionChanges,
+    ) -> BatchSession:
+        """Updates the state of a Batch Session record."""
+        pass
+
+
+class SqliteBatchProcessStorage(BatchProcessStorageBase):
+    _filename: str
+    _conn: sqlite3.Connection
+    _cursor: sqlite3.Cursor
+    _lock: threading.Lock
+
+    def __init__(self, filename: str) -> None:
+        super().__init__()
+        self._filename = filename
+        self._conn = sqlite3.connect(filename, check_same_thread=False)
+        # Enable row factory to get rows as dictionaries (must be done before making the cursor!)
+        self._conn.row_factory = sqlite3.Row
+        self._cursor = self._conn.cursor()
+        self._lock = threading.Lock()
+
+        try:
+            self._lock.acquire()
+            # Enable foreign keys
+            self._conn.execute("PRAGMA foreign_keys = ON;")
+            self._create_tables()
+            self._conn.commit()
+        finally:
+            self._lock.release()
+
+    def _create_tables(self) -> None:
+        """Creates the `batch_process` table and `batch_session` junction table."""
+
+        # Create the `batch_process` table.
+        self._cursor.execute(
+            """--sql
+            CREATE TABLE IF NOT EXISTS batch_process (
+                batch_id TEXT NOT NULL PRIMARY KEY,
+                batches TEXT NOT NULL,
+                graph TEXT NOT NULL,
+                created_at DATETIME NOT NULL DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')),
+                -- Updated via trigger
+                updated_at DATETIME NOT NULL DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')),
+                -- Soft delete, currently unused
+                deleted_at DATETIME
+            );
+            """
+        )
+
+        self._cursor.execute(
+            """--sql
+            CREATE INDEX IF NOT EXISTS idx_batch_process_created_at ON batch_process (created_at);
+            """
+        )
+
+        # Add trigger for `updated_at`.
+        self._cursor.execute(
+            """--sql
+            CREATE TRIGGER IF NOT EXISTS tg_batch_process_updated_at
+            AFTER UPDATE
+            ON batch_process FOR EACH ROW
+            BEGIN
+                UPDATE batch_process SET updated_at = current_timestamp
+                    WHERE batch_id = old.batch_id;
+            END;
+            """
+        )
+
+        # Create the `batch_session` junction table.
+        self._cursor.execute(
+            """--sql
+            CREATE TABLE IF NOT EXISTS batch_session (
+                batch_id TEXT NOT NULL,
+                session_id TEXT NOT NULL,
+                state TEXT NOT NULL,
+                created_at DATETIME NOT NULL DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')),
+                -- updated via trigger
+                updated_at DATETIME NOT NULL DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')),
+                -- Soft delete, currently unused
+                deleted_at DATETIME,
+                -- enforce one-to-many relationship between batch_process and batch_session using PK
+                -- (we can extend this to many-to-many later)
+                PRIMARY KEY (batch_id,session_id),
+                FOREIGN KEY (batch_id) REFERENCES batch_process (batch_id) ON DELETE CASCADE
+            );
+            """
+        )
+
+        # Add index for batch id
+        self._cursor.execute(
+            """--sql
+            CREATE INDEX IF NOT EXISTS idx_batch_session_batch_id ON batch_session (batch_id);
+            """
+        )
+
+        # Add index for batch id, sorted by created_at
+        self._cursor.execute(
+            """--sql
+            CREATE INDEX IF NOT EXISTS idx_batch_session_batch_id_created_at ON batch_session (batch_id,created_at);
+            """
+        )
+
+        # Add trigger for `updated_at`.
+        self._cursor.execute(
+            """--sql
+            CREATE TRIGGER IF NOT EXISTS tg_batch_session_updated_at
+            AFTER UPDATE
+            ON batch_session FOR EACH ROW
+            BEGIN
+                UPDATE batch_session SET updated_at = STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')
+                    WHERE batch_id = old.batch_id AND session_id = old.session_id;
+            END;
+            """
+        )
+
+    def delete(self, batch_id: str) -> None:
+        try:
+            self._lock.acquire()
+            self._cursor.execute(
+                """--sql
+                DELETE FROM batch_process
+                WHERE batch_id = ?;
+                """,
+                (batch_id,),
+            )
+            self._conn.commit()
+        except sqlite3.Error as e:
+            self._conn.rollback()
+            raise BatchProcessDeleteException from e
+        except Exception as e:
+            self._conn.rollback()
+            raise BatchProcessDeleteException from e
+        finally:
+            self._lock.release()
+
+    def save(
+        self,
+        batch_process: BatchProcess,
+    ) -> BatchProcess:
+        try:
+            self._lock.acquire()
+            batches = [batch.json() for batch in batch_process.batches]
+            self._cursor.execute(
+                """--sql
+                INSERT OR IGNORE INTO batch_process (batch_id, batches, graph)
+                VALUES (?, ?, ?);
+                """,
+                (batch_process.batch_id, json.dumps(batches), batch_process.graph.json()),
+            )
+            self._conn.commit()
+        except sqlite3.Error as e:
+            self._conn.rollback()
+            raise BatchProcessSaveException from e
+        finally:
+            self._lock.release()
+        return self.get(batch_process.batch_id)
+
+    def _deserialize_batch_process(self, session_dict: dict) -> BatchProcess:
+        """Deserializes a batch session."""
+
+        # Retrieve all the values, setting "reasonable" defaults if they are not present.
+
+        batch_id = session_dict.get("batch_id", "unknown")
+        batches_raw = session_dict.get("batches", "unknown")
+        graph_raw = session_dict.get("graph", "unknown")
+        batches = json.loads(batches_raw)
+        batches = [parse_raw_as(Batch, batch) for batch in batches]
+        return BatchProcess(
+            batch_id=batch_id,
+            batches=batches,
+            graph=parse_raw_as(Graph, graph_raw),
+        )
+
+    def get(
+        self,
+        batch_id: str,
+    ) -> BatchProcess:
+        try:
+            self._lock.acquire()
+            self._cursor.execute(
+                """--sql
+                SELECT *
+                FROM batch_process
+                WHERE batch_id = ?;
+                """,
+                (batch_id,)
+            )
+
+            result = cast(Union[sqlite3.Row, None], self._cursor.fetchone())
+        except sqlite3.Error as e:
+            self._conn.rollback()
+            raise BatchProcessNotFoundException from e
+        finally:
+            self._lock.release()
+        if result is None:
+            raise BatchProcessNotFoundException
+        return self._deserialize_batch_process(dict(result))
+
+    def create_session(
+            self,
+            session: BatchSession,
+    ) -> BatchSession:
+        try:
+            self._lock.acquire()
+            self._cursor.execute(
+                """--sql
+                INSERT OR IGNORE INTO batch_session (batch_id, session_id, state)
+                VALUES (?, ?, ?);
+                """,
+                (session.batch_id, session.session_id, session.state),
+            )
+            self._conn.commit()
+        except sqlite3.Error as e:
+            self._conn.rollback()
+            raise BatchSessionSaveException from e
+        finally:
+            self._lock.release()
+        return self.get_session(session.session_id)
+
+
+    def get_session(
+        self,
+        session_id: str
+    ) -> BatchSession:
+        try:
+            self._lock.acquire()
+            self._cursor.execute(
+                """--sql
+                SELECT *
+                FROM batch_session
+                WHERE session_id= ?;
+                """,
+                (session_id,),
+            )
+
+            result = cast(Union[sqlite3.Row, None], self._cursor.fetchone())
+        except sqlite3.Error as e:
+            self._conn.rollback()
+            raise BatchSessionNotFoundException from e
+        finally:
+            self._lock.release()
+        if result is None:
+            raise BatchSessionNotFoundException
+        return self._deserialize_batch_session(dict(result))
+
+    def _deserialize_batch_session(self, session_dict: dict) -> BatchSession:
+        """Deserializes a batch session."""
+
+        # Retrieve all the values, setting "reasonable" defaults if they are not present.
+
+        batch_id = session_dict.get("batch_id", "unknown")
+        session_id = session_dict.get("session_id", "unknown")
+        state = session_dict.get("state", "unknown")
+
+        return BatchSession(
+            batch_id=batch_id,
+            session_id=session_id,
+            state=state,
+        )
+
+
+    def get_created_session(
+        self,
+        batch_id: str
+    ) -> BatchSession:
+        try:
+            self._lock.acquire()
+            self._cursor.execute(
+                """--sql
+                SELECT *
+                FROM batch_session
+                WHERE batch_id = ? AND state = 'created';
+                """,
+                (batch_id,),
+            )
+
+            result = cast(list[sqlite3.Row], self._cursor.fetchone())
+        except sqlite3.Error as e:
+            self._conn.rollback()
+            raise BatchSessionNotFoundException from e
+        finally:
+            self._lock.release()
+        if result is None:
+            raise BatchSessionNotFoundException
+        session = self._deserialize_batch_session(dict(result))
+        return session
+
+
+    def get_created_sessions(
+        self,
+        batch_id: str
+    ) -> List[BatchSession]:
+        try:
+            self._lock.acquire()
+            self._cursor.execute(
+                """--sql
+                SELECT *
+                FROM batch_session
+                WHERE batch_id = ? AND state = created;
+                """,
+                (batch_id,),
+            )
+
+
+            result = cast(list[sqlite3.Row], self._cursor.fetchall())
+        except sqlite3.Error as e:
+            self._conn.rollback()
+            raise BatchSessionNotFoundException from e
+        finally:
+            self._lock.release()
+        if result is None:
+            raise BatchSessionNotFoundException
+        sessions = list(map(lambda r: self._deserialize_batch_session(dict(r)), result))
+        return sessions
+            
+
+    def update_session_state(
+        self,
+        batch_id: str,
+        session_id: str,
+        changes: BatchSessionChanges,
+    ) -> BatchSession:
+        try:
+            self._lock.acquire()
+
+            # Change the state of a batch session
+            if changes.state is not None:
+                self._cursor.execute(
+                    f"""--sql
+                    UPDATE batch_session
+                    SET state = ?
+                    WHERE batch_id = ? AND session_id = ?;
+                    """,
+                    (changes.state, batch_id, session_id),
+                )
+
+            self._conn.commit()
+        except sqlite3.Error as e:
+            self._conn.rollback()
+            raise BatchSessionSaveException from e
+        finally:
+            self._lock.release()
+        return self.get_session(session_id)

invokeai/app/services/invocation_services.py

@@ -4,6 +4,7 @@ from typing import TYPE_CHECKING
 
 if TYPE_CHECKING:
     from logging import Logger
+    from invokeai.app.services.batch_manager import BatchManagerBase
     from invokeai.app.services.board_images import BoardImagesServiceABC
     from invokeai.app.services.boards import BoardServiceABC
     from invokeai.app.services.images import ImageServiceABC
@@ -21,6 +22,7 @@ class InvocationServices:
     """Services that can be used by invocations"""
 
     # TODO: Just forward-declared everything due to circular dependencies. Fix structure.
+    batch_manager: "BatchManagerBase"
     board_images: "BoardImagesServiceABC"
     boards: "BoardServiceABC"
     configuration: "InvokeAIAppConfig"
@@ -36,6 +38,7 @@ class InvocationServices:
 
     def __init__(
         self,
+        batch_manager: "BatchManagerBase",
         board_images: "BoardImagesServiceABC",
         boards: "BoardServiceABC",
         configuration: "InvokeAIAppConfig",
@@ -49,6 +52,7 @@ class InvocationServices:
         processor: "InvocationProcessorABC",
         queue: "InvocationQueueABC",
     ):
+        self.batch_manager = batch_manager
         self.board_images = board_images
         self.boards = boards
         self.boards = boards

invokeai/app/services/sqlite.py

@@ -29,6 +29,7 @@ class SqliteItemStorage(ItemStorageABC, Generic[T]):
         self._conn = sqlite3.connect(
             self._filename, check_same_thread=False
         )  # TODO: figure out a better threading solution
+        self._conn.execute('pragma journal_mode=wal')
         self._cursor = self._conn.cursor()
 
         self._create_table()

invokeai/backend/install/model_install_backend.py

@@ -7,11 +7,12 @@ import warnings
 from dataclasses import dataclass, field
 from pathlib import Path
 from tempfile import TemporaryDirectory
-from typing import List, Dict, Callable, Union, Set, Optional
+from typing import Optional, List, Dict, Callable, Union, Set
 
 import requests
 from diffusers import DiffusionPipeline
 from diffusers import logging as dlogging
+import onnx
 from huggingface_hub import hf_hub_url, HfFolder, HfApi
 from omegaconf import OmegaConf
 from tqdm import tqdm
@@ -86,8 +87,8 @@ class ModelLoadInfo:
     name: str
     model_type: ModelType
     base_type: BaseModelType
-    path: Path = None
-    repo_id: str = None
+    path: Optional[Path] = None
+    repo_id: Optional[str] = None
     description: str = ""
     installed: bool = False
     recommended: bool = False
@@ -302,8 +303,10 @@ class ModelInstall(object):
 
         with TemporaryDirectory(dir=self.config.models_path) as staging:
             staging = Path(staging)
-            if "model_index.json" in files:
+            if "model_index.json" in files and "unet/model.onnx" not in files:
                 location = self._download_hf_pipeline(repo_id, staging)  # pipeline
+            elif "unet/model.onnx" in files:
+                location = self._download_hf_model(repo_id, files, staging)
             else:
                 for suffix in ["safetensors", "bin"]:
                     if f"pytorch_lora_weights.{suffix}" in files:
@@ -368,7 +371,7 @@ class ModelInstall(object):
             model_format=info.format,
         )
         legacy_conf = None
-        if info.model_type == ModelType.Main:
+        if info.model_type == ModelType.Main or info.model_type == ModelType.ONNX:
             attributes.update(
                 dict(
                     variant=info.variant_type,
@@ -433,8 +436,13 @@ class ModelInstall(object):
         location = staging / name
         paths = list()
         for filename in files:
+            filePath = Path(filename)
             p = hf_download_with_resume(
-                repo_id, model_dir=location, model_name=filename, access_token=self.access_token
+                repo_id,
+                model_dir=location / filePath.parent,
+                model_name=filePath.name,
+                access_token=self.access_token,
+                subfolder=filePath.parent,
             )
             if p:
                 paths.append(p)
@@ -482,11 +490,12 @@ def hf_download_with_resume(
     model_name: str,
     model_dest: Path = None,
     access_token: str = None,
+    subfolder: str = None,
 ) -> Path:
     model_dest = model_dest or Path(os.path.join(model_dir, model_name))
     os.makedirs(model_dir, exist_ok=True)
 
-    url = hf_hub_url(repo_id, model_name)
+    url = hf_hub_url(repo_id, model_name, subfolder=subfolder)
 
     header = {"Authorization": f"Bearer {access_token}"} if access_token else {}
     open_mode = "wb"

invokeai/backend/model_management/__init__.py

@@ -3,6 +3,7 @@ Initialization file for invokeai.backend.model_management
 """
 from .model_manager import ModelManager, ModelInfo, AddModelResult, SchedulerPredictionType
 from .model_cache import ModelCache
+from .lora import ModelPatcher, ONNXModelPatcher
 from .models import (
     BaseModelType,
     ModelType,

invokeai/backend/model_management/lora.py

@@ -6,11 +6,22 @@ from typing import Optional, Dict, Tuple, Any, Union, List
 from pathlib import Path
 
 import torch
+from safetensors.torch import load_file
+from torch.utils.hooks import RemovableHandle
+
+from diffusers.models import UNet2DConditionModel
+from transformers import CLIPTextModel
+from onnx import numpy_helper
+from onnxruntime import OrtValue
+import numpy as np
+
 from compel.embeddings_provider import BaseTextualInversionManager
 from diffusers.models import UNet2DConditionModel
 from safetensors.torch import load_file
 from transformers import CLIPTextModel, CLIPTokenizer
 
+# TODO: rename and split this file
+
 
 class LoRALayerBase:
     # rank: Optional[int]
@@ -698,3 +709,186 @@ class TextualInversionManager(BaseTextualInversionManager):
                 new_token_ids.extend(self.pad_tokens[token_id])
 
         return new_token_ids
+
+
+class ONNXModelPatcher:
+    from .models.base import IAIOnnxRuntimeModel, OnnxRuntimeModel
+
+    @classmethod
+    @contextmanager
+    def apply_lora_unet(
+        cls,
+        unet: OnnxRuntimeModel,
+        loras: List[Tuple[LoRAModel, float]],
+    ):
+        with cls.apply_lora(unet, loras, "lora_unet_"):
+            yield
+
+    @classmethod
+    @contextmanager
+    def apply_lora_text_encoder(
+        cls,
+        text_encoder: OnnxRuntimeModel,
+        loras: List[Tuple[LoRAModel, float]],
+    ):
+        with cls.apply_lora(text_encoder, loras, "lora_te_"):
+            yield
+
+    # based on
+    # https://github.com/ssube/onnx-web/blob/ca2e436f0623e18b4cfe8a0363fcfcf10508acf7/api/onnx_web/convert/diffusion/lora.py#L323
+    @classmethod
+    @contextmanager
+    def apply_lora(
+        cls,
+        model: IAIOnnxRuntimeModel,
+        loras: List[Tuple[LoraModel, float]],
+        prefix: str,
+    ):
+        from .models.base import IAIOnnxRuntimeModel
+
+        if not isinstance(model, IAIOnnxRuntimeModel):
+            raise Exception("Only IAIOnnxRuntimeModel models supported")
+
+        orig_weights = dict()
+
+        try:
+            blended_loras = dict()
+
+            for lora, lora_weight in loras:
+                for layer_key, layer in lora.layers.items():
+                    if not layer_key.startswith(prefix):
+                        continue
+
+                    layer.to(dtype=torch.float32)
+                    layer_key = layer_key.replace(prefix, "")
+                    layer_weight = layer.get_weight().detach().cpu().numpy() * lora_weight
+                    if layer_key is blended_loras:
+                        blended_loras[layer_key] += layer_weight
+                    else:
+                        blended_loras[layer_key] = layer_weight
+
+            node_names = dict()
+            for node in model.nodes.values():
+                node_names[node.name.replace("/", "_").replace(".", "_").lstrip("_")] = node.name
+
+            for layer_key, lora_weight in blended_loras.items():
+                conv_key = layer_key + "_Conv"
+                gemm_key = layer_key + "_Gemm"
+                matmul_key = layer_key + "_MatMul"
+
+                if conv_key in node_names or gemm_key in node_names:
+                    if conv_key in node_names:
+                        conv_node = model.nodes[node_names[conv_key]]
+                    else:
+                        conv_node = model.nodes[node_names[gemm_key]]
+
+                    weight_name = [n for n in conv_node.input if ".weight" in n][0]
+                    orig_weight = model.tensors[weight_name]
+
+                    if orig_weight.shape[-2:] == (1, 1):
+                        if lora_weight.shape[-2:] == (1, 1):
+                            new_weight = orig_weight.squeeze((3, 2)) + lora_weight.squeeze((3, 2))
+                        else:
+                            new_weight = orig_weight.squeeze((3, 2)) + lora_weight
+
+                        new_weight = np.expand_dims(new_weight, (2, 3))
+                    else:
+                        if orig_weight.shape != lora_weight.shape:
+                            new_weight = orig_weight + lora_weight.reshape(orig_weight.shape)
+                        else:
+                            new_weight = orig_weight + lora_weight
+
+                    orig_weights[weight_name] = orig_weight
+                    model.tensors[weight_name] = new_weight.astype(orig_weight.dtype)
+
+                elif matmul_key in node_names:
+                    weight_node = model.nodes[node_names[matmul_key]]
+                    matmul_name = [n for n in weight_node.input if "MatMul" in n][0]
+
+                    orig_weight = model.tensors[matmul_name]
+                    new_weight = orig_weight + lora_weight.transpose()
+
+                    orig_weights[matmul_name] = orig_weight
+                    model.tensors[matmul_name] = new_weight.astype(orig_weight.dtype)
+
+                else:
+                    # warn? err?
+                    pass
+
+            yield
+
+        finally:
+            # restore original weights
+            for name, orig_weight in orig_weights.items():
+                model.tensors[name] = orig_weight
+
+    @classmethod
+    @contextmanager
+    def apply_ti(
+        cls,
+        tokenizer: CLIPTokenizer,
+        text_encoder: IAIOnnxRuntimeModel,
+        ti_list: List[Any],
+    ) -> Tuple[CLIPTokenizer, TextualInversionManager]:
+        from .models.base import IAIOnnxRuntimeModel
+
+        if not isinstance(text_encoder, IAIOnnxRuntimeModel):
+            raise Exception("Only IAIOnnxRuntimeModel models supported")
+
+        orig_embeddings = None
+
+        try:
+            ti_tokenizer = copy.deepcopy(tokenizer)
+            ti_manager = TextualInversionManager(ti_tokenizer)
+
+            def _get_trigger(ti, index):
+                trigger = ti.name
+                if index > 0:
+                    trigger += f"-!pad-{i}"
+                return f"<{trigger}>"
+
+            # modify tokenizer
+            new_tokens_added = 0
+            for ti in ti_list:
+                for i in range(ti.embedding.shape[0]):
+                    new_tokens_added += ti_tokenizer.add_tokens(_get_trigger(ti, i))
+
+            # modify text_encoder
+            orig_embeddings = text_encoder.tensors["text_model.embeddings.token_embedding.weight"]
+
+            embeddings = np.concatenate(
+                (np.copy(orig_embeddings), np.zeros((new_tokens_added, orig_embeddings.shape[1]))),
+                axis=0,
+            )
+
+            for ti in ti_list:
+                ti_tokens = []
+                for i in range(ti.embedding.shape[0]):
+                    embedding = ti.embedding[i].detach().numpy()
+                    trigger = _get_trigger(ti, i)
+
+                    token_id = ti_tokenizer.convert_tokens_to_ids(trigger)
+                    if token_id == ti_tokenizer.unk_token_id:
+                        raise RuntimeError(f"Unable to find token id for token '{trigger}'")
+
+                    if embeddings[token_id].shape != embedding.shape:
+                        raise ValueError(
+                            f"Cannot load embedding for {trigger}. It was trained on a model with token dimension {embedding.shape[0]}, but the current model has token dimension {embeddings[token_id].shape[0]}."
+                        )
+
+                    embeddings[token_id] = embedding
+                    ti_tokens.append(token_id)
+
+                if len(ti_tokens) > 1:
+                    ti_manager.pad_tokens[ti_tokens[0]] = ti_tokens[1:]
+
+            text_encoder.tensors["text_model.embeddings.token_embedding.weight"] = embeddings.astype(
+                orig_embeddings.dtype
+            )
+
+            yield ti_tokenizer, ti_manager
+
+        finally:
+            # restore
+            if orig_embeddings is not None:
+                text_encoder.tensors["text_model.embeddings.token_embedding.weight"] = orig_embeddings

invokeai/backend/model_management/model_cache.py

@@ -360,7 +360,8 @@ class ModelCache(object):
             # 2 refs:
             # 1 from cache_entry
             # 1 from getrefcount function
-            if not cache_entry.locked and refs <= 2:
+            # 1 from onnx runtime object
+            if not cache_entry.locked and refs <= 3 if "onnx" in model_key else 2:
                 self.logger.debug(
                     f"Unloading model {model_key} to free {(model_size/GIG):.2f} GB (-{(cache_entry.size/GIG):.2f} GB)"
                 )

invokeai/backend/model_management/model_manager.py

@@ -276,7 +276,7 @@ class ModelInfo:
     hash: str
     location: Union[Path, str]
     precision: torch.dtype
-    _cache: ModelCache = None
+    _cache: Optional[ModelCache] = None
 
     def __enter__(self):
         return self.context.__enter__()

invokeai/backend/model_management/model_probe.py

@@ -27,7 +27,7 @@ class ModelProbeInfo(object):
     variant_type: ModelVariantType
     prediction_type: SchedulerPredictionType
     upcast_attention: bool
-    format: Literal["diffusers", "checkpoint", "lycoris"]
+    format: Literal["diffusers", "checkpoint", "lycoris", "olive", "onnx"]
     image_size: int
 
 
@@ -41,6 +41,7 @@ class ModelProbe(object):
     PROBES = {
         "diffusers": {},
         "checkpoint": {},
+        "onnx": {},
     }
 
     CLASS2TYPE = {
@@ -53,7 +54,9 @@ class ModelProbe(object):
     }
 
     @classmethod
-    def register_probe(cls, format: Literal["diffusers", "checkpoint"], model_type: ModelType, probe_class: ProbeBase):
+    def register_probe(
+        cls, format: Literal["diffusers", "checkpoint", "onnx"], model_type: ModelType, probe_class: ProbeBase
+    ):
         cls.PROBES[format][model_type] = probe_class
 
     @classmethod
@@ -95,6 +98,7 @@ class ModelProbe(object):
                 if format_type == "diffusers"
                 else cls.get_model_type_from_checkpoint(model_path, model)
             )
+            format_type = "onnx" if model_type == ModelType.ONNX else format_type
             probe_class = cls.PROBES[format_type].get(model_type)
             if not probe_class:
                 return None
@@ -168,6 +172,8 @@ class ModelProbe(object):
         if model:
             class_name = model.__class__.__name__
         else:
+            if (folder_path / "unet/model.onnx").exists():
+                return ModelType.ONNX
             if (folder_path / "learned_embeds.bin").exists():
                 return ModelType.TextualInversion
 
@@ -460,6 +466,17 @@ class TextualInversionFolderProbe(FolderProbeBase):
         return TextualInversionCheckpointProbe(None, checkpoint=checkpoint).get_base_type()
 
 
+class ONNXFolderProbe(FolderProbeBase):
+    def get_format(self) -> str:
+        return "onnx"
+
+    def get_base_type(self) -> BaseModelType:
+        return BaseModelType.StableDiffusion1
+
+    def get_variant_type(self) -> ModelVariantType:
+        return ModelVariantType.Normal
+
+
 class ControlNetFolderProbe(FolderProbeBase):
     def get_base_type(self) -> BaseModelType:
         config_file = self.folder_path / "config.json"
@@ -497,3 +514,4 @@ ModelProbe.register_probe("checkpoint", ModelType.Vae, VaeCheckpointProbe)
 ModelProbe.register_probe("checkpoint", ModelType.Lora, LoRACheckpointProbe)
 ModelProbe.register_probe("checkpoint", ModelType.TextualInversion, TextualInversionCheckpointProbe)
 ModelProbe.register_probe("checkpoint", ModelType.ControlNet, ControlNetCheckpointProbe)
+ModelProbe.register_probe("onnx", ModelType.ONNX, ONNXFolderProbe)

invokeai/backend/model_management/models/__init__.py

@@ -23,8 +23,11 @@ from .lora import LoRAModel
 from .controlnet import ControlNetModel  # TODO:
 from .textual_inversion import TextualInversionModel
 
+from .stable_diffusion_onnx import ONNXStableDiffusion1Model, ONNXStableDiffusion2Model
+
 MODEL_CLASSES = {
     BaseModelType.StableDiffusion1: {
+        ModelType.ONNX: ONNXStableDiffusion1Model,
         ModelType.Main: StableDiffusion1Model,
         ModelType.Vae: VaeModel,
         ModelType.Lora: LoRAModel,
@@ -32,6 +35,7 @@ MODEL_CLASSES = {
         ModelType.TextualInversion: TextualInversionModel,
     },
     BaseModelType.StableDiffusion2: {
+        ModelType.ONNX: ONNXStableDiffusion2Model,
         ModelType.Main: StableDiffusion2Model,
         ModelType.Vae: VaeModel,
         ModelType.Lora: LoRAModel,
@@ -45,6 +49,7 @@ MODEL_CLASSES = {
         ModelType.Lora: LoRAModel,
         ModelType.ControlNet: ControlNetModel,
         ModelType.TextualInversion: TextualInversionModel,
+        ModelType.ONNX: ONNXStableDiffusion2Model,
     },
     BaseModelType.StableDiffusionXLRefiner: {
         ModelType.Main: StableDiffusionXLModel,
@@ -53,6 +58,7 @@ MODEL_CLASSES = {
         ModelType.Lora: LoRAModel,
         ModelType.ControlNet: ControlNetModel,
         ModelType.TextualInversion: TextualInversionModel,
+        ModelType.ONNX: ONNXStableDiffusion2Model,
     },
     # BaseModelType.Kandinsky2_1: {
     #    ModelType.Main: Kandinsky2_1Model,

invokeai/backend/model_management/models/base.py

@@ -8,13 +8,23 @@ from abc import ABCMeta, abstractmethod
 from pathlib import Path
 from picklescan.scanner import scan_file_path
 import torch
+import numpy as np
 import safetensors.torch
-from diffusers import DiffusionPipeline, ConfigMixin
+from pathlib import Path
+from diffusers import DiffusionPipeline, ConfigMixin, OnnxRuntimeModel
 
 from contextlib import suppress
 from pydantic import BaseModel, Field
 from typing import List, Dict, Optional, Type, Literal, TypeVar, Generic, Callable, Any, Union
 
+import onnx
+from onnx import numpy_helper
+from onnxruntime import (
+    InferenceSession,
+    SessionOptions,
+    get_available_providers,
+)
+
 
 class DuplicateModelException(Exception):
     pass
@@ -37,6 +47,7 @@ class BaseModelType(str, Enum):
 
 
 class ModelType(str, Enum):
+    ONNX = "onnx"
     Main = "main"
     Vae = "vae"
     Lora = "lora"
@@ -51,6 +62,8 @@ class SubModelType(str, Enum):
     Tokenizer = "tokenizer"
     Tokenizer2 = "tokenizer_2"
     Vae = "vae"
+    VaeDecoder = "vae_decoder"
+    VaeEncoder = "vae_encoder"
     Scheduler = "scheduler"
     SafetyChecker = "safety_checker"
     # MoVQ = "movq"
@@ -362,6 +375,8 @@ def calc_model_size_by_data(model) -> int:
         return _calc_pipeline_by_data(model)
     elif isinstance(model, torch.nn.Module):
         return _calc_model_by_data(model)
+    elif isinstance(model, IAIOnnxRuntimeModel):
+        return _calc_onnx_model_by_data(model)
     else:
         return 0
 
@@ -382,6 +397,12 @@ def _calc_model_by_data(model) -> int:
     return mem
 
 
+def _calc_onnx_model_by_data(model) -> int:
+    tensor_size = model.tensors.size() * 2  # The session doubles this
+    mem = tensor_size  # in bytes
+    return mem
+
+
 def _fast_safetensors_reader(path: str):
     checkpoint = dict()
     device = torch.device("meta")
@@ -449,3 +470,208 @@ class SilenceWarnings(object):
         transformers_logging.set_verbosity(self.transformers_verbosity)
         diffusers_logging.set_verbosity(self.diffusers_verbosity)
         warnings.simplefilter("default")
+
+
+ONNX_WEIGHTS_NAME = "model.onnx"
+
+
+class IAIOnnxRuntimeModel:
+    class _tensor_access:
+        def __init__(self, model):
+            self.model = model
+            self.indexes = dict()
+            for idx, obj in enumerate(self.model.proto.graph.initializer):
+                self.indexes[obj.name] = idx
+
+        def __getitem__(self, key: str):
+            value = self.model.proto.graph.initializer[self.indexes[key]]
+            return numpy_helper.to_array(value)
+
+        def __setitem__(self, key: str, value: np.ndarray):
+            new_node = numpy_helper.from_array(value)
+            # set_external_data(new_node, location="in-memory-location")
+            new_node.name = key
+            # new_node.ClearField("raw_data")
+            del self.model.proto.graph.initializer[self.indexes[key]]
+            self.model.proto.graph.initializer.insert(self.indexes[key], new_node)
+            # self.model.data[key] = OrtValue.ortvalue_from_numpy(value)
+
+        # __delitem__
+
+        def __contains__(self, key: str):
+            return self.indexes[key] in self.model.proto.graph.initializer
+
+        def items(self):
+            raise NotImplementedError("tensor.items")
+            # return [(obj.name, obj) for obj in self.raw_proto]
+
+        def keys(self):
+            return self.indexes.keys()
+
+        def values(self):
+            raise NotImplementedError("tensor.values")
+            # return [obj for obj in self.raw_proto]
+
+        def size(self):
+            bytesSum = 0
+            for node in self.model.proto.graph.initializer:
+                bytesSum += sys.getsizeof(node.raw_data)
+            return bytesSum
+
+    class _access_helper:
+        def __init__(self, raw_proto):
+            self.indexes = dict()
+            self.raw_proto = raw_proto
+            for idx, obj in enumerate(raw_proto):
+                self.indexes[obj.name] = idx
+
+        def __getitem__(self, key: str):
+            return self.raw_proto[self.indexes[key]]
+
+        def __setitem__(self, key: str, value):
+            index = self.indexes[key]
+            del self.raw_proto[index]
+            self.raw_proto.insert(index, value)
+
+        # __delitem__
+
+        def __contains__(self, key: str):
+            return key in self.indexes
+
+        def items(self):
+            return [(obj.name, obj) for obj in self.raw_proto]
+
+        def keys(self):
+            return self.indexes.keys()
+
+        def values(self):
+            return [obj for obj in self.raw_proto]
+
+    def __init__(self, model_path: str, provider: Optional[str]):
+        self.path = model_path
+        self.session = None
+        self.provider = provider
+        """
+        self.data_path = self.path + "_data"
+        if not os.path.exists(self.data_path):
+            print(f"Moving model tensors to separate file: {self.data_path}")
+            tmp_proto = onnx.load(model_path, load_external_data=True)
+            onnx.save_model(tmp_proto, self.path, save_as_external_data=True, all_tensors_to_one_file=True, location=os.path.basename(self.data_path), size_threshold=1024, convert_attribute=False)
+            del tmp_proto
+            gc.collect()
+
+        self.proto = onnx.load(model_path, load_external_data=False)
+        """
+
+        self.proto = onnx.load(model_path, load_external_data=True)
+        # self.data = dict()
+        # for tensor in self.proto.graph.initializer:
+        #     name = tensor.name
+
+        #     if tensor.HasField("raw_data"):
+        #         npt = numpy_helper.to_array(tensor)
+        #         orv = OrtValue.ortvalue_from_numpy(npt)
+        #         # self.data[name] = orv
+        #         # set_external_data(tensor, location="in-memory-location")
+        #         tensor.name = name
+        #         # tensor.ClearField("raw_data")
+
+        self.nodes = self._access_helper(self.proto.graph.node)
+        # self.initializers = self._access_helper(self.proto.graph.initializer)
+        # print(self.proto.graph.input)
+        # print(self.proto.graph.initializer)
+
+        self.tensors = self._tensor_access(self)
+
+    # TODO: integrate with model manager/cache
+    def create_session(self, height=None, width=None):
+        if self.session is None or self.session_width != width or self.session_height != height:
+            # onnx.save(self.proto, "tmp.onnx")
+            # onnx.save_model(self.proto, "tmp.onnx", save_as_external_data=True, all_tensors_to_one_file=True, location="tmp.onnx_data", size_threshold=1024, convert_attribute=False)
+            # TODO: something to be able to get weight when they already moved outside of model proto
+            # (trimmed_model, external_data) = buffer_external_data_tensors(self.proto)
+            sess = SessionOptions()
+            # self._external_data.update(**external_data)
+            # sess.add_external_initializers(list(self.data.keys()), list(self.data.values()))
+            # sess.enable_profiling = True
+
+            # sess.intra_op_num_threads = 1
+            # sess.inter_op_num_threads = 1
+            # sess.execution_mode = ExecutionMode.ORT_SEQUENTIAL
+            # sess.graph_optimization_level = GraphOptimizationLevel.ORT_ENABLE_ALL
+            # sess.enable_cpu_mem_arena = True
+            # sess.enable_mem_pattern = True
+            # sess.add_session_config_entry("session.intra_op.use_xnnpack_threadpool", "1") ########### It's the key code
+            self.session_height = height
+            self.session_width = width
+            if height and width:
+                sess.add_free_dimension_override_by_name("unet_sample_batch", 2)
+                sess.add_free_dimension_override_by_name("unet_sample_channels", 4)
+                sess.add_free_dimension_override_by_name("unet_hidden_batch", 2)
+                sess.add_free_dimension_override_by_name("unet_hidden_sequence", 77)
+                sess.add_free_dimension_override_by_name("unet_sample_height", self.session_height)
+                sess.add_free_dimension_override_by_name("unet_sample_width", self.session_width)
+                sess.add_free_dimension_override_by_name("unet_time_batch", 1)
+            providers = []
+            if self.provider:
+                providers.append(self.provider)
+            else:
+                providers = get_available_providers()
+            if "TensorrtExecutionProvider" in providers:
+                providers.remove("TensorrtExecutionProvider")
+            try:
+                self.session = InferenceSession(self.proto.SerializeToString(), providers=providers, sess_options=sess)
+            except Exception as e:
+                raise e
+            # self.session = InferenceSession("tmp.onnx", providers=[self.provider], sess_options=self.sess_options)
+            # self.io_binding = self.session.io_binding()
+
+    def release_session(self):
+        self.session = None
+        import gc
+
+        gc.collect()
+        return
+
+    def __call__(self, **kwargs):
+        if self.session is None:
+            raise Exception("You should call create_session before running model")
+
+        inputs = {k: np.array(v) for k, v in kwargs.items()}
+        output_names = self.session.get_outputs()
+        # for k in inputs:
+        #     self.io_binding.bind_cpu_input(k, inputs[k])
+        # for name in output_names:
+        #     self.io_binding.bind_output(name.name)
+        # self.session.run_with_iobinding(self.io_binding, None)
+        # return self.io_binding.copy_outputs_to_cpu()
+        return self.session.run(None, inputs)
+
+    # compatability with diffusers load code
+    @classmethod
+    def from_pretrained(
+        cls,
+        model_id: Union[str, Path],
+        subfolder: Union[str, Path] = None,
+        file_name: Optional[str] = None,
+        provider: Optional[str] = None,
+        sess_options: Optional["SessionOptions"] = None,
+        **kwargs,
+    ):
+        file_name = file_name or ONNX_WEIGHTS_NAME
+
+        if os.path.isdir(model_id):
+            model_path = model_id
+            if subfolder is not None:
+                model_path = os.path.join(model_path, subfolder)
+            model_path = os.path.join(model_path, file_name)
+
+        else:
+            model_path = model_id
+
+        # load model from local directory
+        if not os.path.isfile(model_path):
+            raise Exception(f"Model not found: {model_path}")
+
+        # TODO: session options
+        return cls(model_path, provider=provider)

invokeai/backend/model_management/models/stable_diffusion_onnx.py

@@ -0,0 +1,157 @@
+import os
+import json
+from enum import Enum
+from pydantic import Field
+from pathlib import Path
+from typing import Literal, Optional, Union
+from .base import (
+    ModelBase,
+    ModelConfigBase,
+    BaseModelType,
+    ModelType,
+    SubModelType,
+    ModelVariantType,
+    DiffusersModel,
+    SchedulerPredictionType,
+    SilenceWarnings,
+    read_checkpoint_meta,
+    classproperty,
+    OnnxRuntimeModel,
+    IAIOnnxRuntimeModel,
+)
+from invokeai.app.services.config import InvokeAIAppConfig
+
+
+class StableDiffusionOnnxModelFormat(str, Enum):
+    Olive = "olive"
+    Onnx = "onnx"
+
+
+class ONNXStableDiffusion1Model(DiffusersModel):
+    class Config(ModelConfigBase):
+        model_format: Literal[StableDiffusionOnnxModelFormat.Onnx]
+        variant: ModelVariantType
+
+    def __init__(self, model_path: str, base_model: BaseModelType, model_type: ModelType):
+        assert base_model == BaseModelType.StableDiffusion1
+        assert model_type == ModelType.ONNX
+        super().__init__(
+            model_path=model_path,
+            base_model=BaseModelType.StableDiffusion1,
+            model_type=ModelType.ONNX,
+        )
+
+        for child_name, child_type in self.child_types.items():
+            if child_type is OnnxRuntimeModel:
+                self.child_types[child_name] = IAIOnnxRuntimeModel
+
+            # TODO: check that no optimum models provided
+
+    @classmethod
+    def probe_config(cls, path: str, **kwargs):
+        model_format = cls.detect_format(path)
+        in_channels = 4  # TODO:
+
+        if in_channels == 9:
+            variant = ModelVariantType.Inpaint
+        elif in_channels == 4:
+            variant = ModelVariantType.Normal
+        else:
+            raise Exception("Unkown stable diffusion 1.* model format")
+
+        return cls.create_config(
+            path=path,
+            model_format=model_format,
+            variant=variant,
+        )
+
+    @classproperty
+    def save_to_config(cls) -> bool:
+        return True
+
+    @classmethod
+    def detect_format(cls, model_path: str):
+        # TODO: Detect onnx vs olive
+        return StableDiffusionOnnxModelFormat.Onnx
+
+    @classmethod
+    def convert_if_required(
+        cls,
+        model_path: str,
+        output_path: str,
+        config: ModelConfigBase,
+        base_model: BaseModelType,
+    ) -> str:
+        return model_path
+
+
+class ONNXStableDiffusion2Model(DiffusersModel):
+    # TODO: check that configs overwriten properly
+    class Config(ModelConfigBase):
+        model_format: Literal[StableDiffusionOnnxModelFormat.Onnx]
+        variant: ModelVariantType
+        prediction_type: SchedulerPredictionType
+        upcast_attention: bool
+
+    def __init__(self, model_path: str, base_model: BaseModelType, model_type: ModelType):
+        assert base_model == BaseModelType.StableDiffusion2
+        assert model_type == ModelType.ONNX
+        super().__init__(
+            model_path=model_path,
+            base_model=BaseModelType.StableDiffusion2,
+            model_type=ModelType.ONNX,
+        )
+
+        for child_name, child_type in self.child_types.items():
+            if child_type is OnnxRuntimeModel:
+                self.child_types[child_name] = IAIOnnxRuntimeModel
+            # TODO: check that no optimum models provided
+
+    @classmethod
+    def probe_config(cls, path: str, **kwargs):
+        model_format = cls.detect_format(path)
+        in_channels = 4  # TODO:
+
+        if in_channels == 9:
+            variant = ModelVariantType.Inpaint
+        elif in_channels == 5:
+            variant = ModelVariantType.Depth
+        elif in_channels == 4:
+            variant = ModelVariantType.Normal
+        else:
+            raise Exception("Unkown stable diffusion 2.* model format")
+
+        if variant == ModelVariantType.Normal:
+            prediction_type = SchedulerPredictionType.VPrediction
+            upcast_attention = True
+
+        else:
+            prediction_type = SchedulerPredictionType.Epsilon
+            upcast_attention = False
+
+        return cls.create_config(
+            path=path,
+            model_format=model_format,
+            variant=variant,
+            prediction_type=prediction_type,
+            upcast_attention=upcast_attention,
+        )
+
+    @classproperty
+    def save_to_config(cls) -> bool:
+        return True
+
+    @classmethod
+    def detect_format(cls, model_path: str):
+        # TODO: Detect onnx vs olive
+        return StableDiffusionOnnxModelFormat.Onnx
+
+    @classmethod
+    def convert_if_required(
+        cls,
+        model_path: str,
+        output_path: str,
+        config: ModelConfigBase,
+        base_model: BaseModelType,
+    ) -> str:
+        return model_path

invokeai/frontend/web/config/vite.package.config.ts

@@ -21,7 +21,7 @@ export const packageConfig: UserConfig = {
       fileName: (format) => `invoke-ai-ui.${format}.js`,
     },
     rollupOptions: {
-      external: ['react', 'react-dom', '@emotion/react'],
+      external: ['react', 'react-dom', '@emotion/react', '@chakra-ui/react'],
       output: {
         globals: {
           react: 'React',

invokeai/frontend/web/dist/assets/ThemeLocaleProvider-a4dc3f38.js

@@ -1,4 +1,4 @@
-import{A as m,f_ as Je,z as y,a4 as Ka,f$ as Xa,af as va,aj as d,g0 as b,g1 as t,g2 as Ya,g3 as h,g4 as ua,g5 as Ja,g6 as Qa,aI as Za,g7 as et,ad as rt,g8 as at}from"./index-9bb68e3a.js";import{s as fa,n as o,t as tt,o as ha,p as ot,q as ma,v as ga,w as ya,x as it,y as Sa,z as pa,A as xr,B as nt,D as lt,E as st,F as xa,G as $a,H as ka,J as dt,K as _a,L as ct,M as bt,N as vt,O as ut,Q as wa,R as ft,S as ht,T as mt,U as gt,V as yt,W as St,e as pt,X as xt}from"./MantineProvider-ae002ae6.js";var za=String.raw,Ca=za`
+import{A as m,f$ as Je,z as y,a4 as Ka,g0 as Xa,af as va,aj as d,g1 as b,g2 as t,g3 as Ya,g4 as h,g5 as ua,g6 as Ja,g7 as Qa,aI as Za,g8 as et,ad as rt,g9 as at}from"./index-18f2f740.js";import{s as fa,n as o,t as tt,o as ha,p as ot,q as ma,v as ga,w as ya,x as it,y as Sa,z as pa,A as xr,B as nt,D as lt,E as st,F as xa,G as $a,H as ka,J as dt,K as _a,L as ct,M as bt,N as vt,O as ut,Q as wa,R as ft,S as ht,T as mt,U as gt,V as yt,W as St,e as pt,X as xt}from"./MantineProvider-b20a2267.js";var za=String.raw,Ca=za`
   :root,
   :host {
     --chakra-vh: 100vh;

invokeai/frontend/web/dist/index.html

@@ -12,7 +12,7 @@
         margin: 0;
       }
     </style>
-    <script type="module" crossorigin src="./assets/index-9bb68e3a.js"></script>
+    <script type="module" crossorigin src="./assets/index-18f2f740.js"></script>
   </head>
 
   <body dir="ltr">

invokeai/frontend/web/dist/locales/en.json

@@ -342,6 +342,8 @@
         "diffusersModels": "Diffusers",
         "loraModels": "LoRAs",
         "safetensorModels": "SafeTensors",
+        "onnxModels": "Onnx",
+        "oliveModels": "Olives",
         "modelAdded": "Model Added",
         "modelUpdated": "Model Updated",
         "modelUpdateFailed": "Model Update Failed",

invokeai/frontend/web/package.json

@@ -116,6 +116,7 @@
   },
   "peerDependencies": {
     "@chakra-ui/cli": "^2.4.0",
+    "@chakra-ui/react": "^2.8.0",
     "react": "^18.2.0",
     "react-dom": "^18.2.0",
     "ts-toolbelt": "^9.6.0"

invokeai/frontend/web/public/locales/en.json

@@ -342,6 +342,8 @@
         "diffusersModels": "Diffusers",
         "loraModels": "LoRAs",
         "safetensorModels": "SafeTensors",
+        "onnxModels": "Onnx",
+        "oliveModels": "Olives",
         "modelAdded": "Model Added",
         "modelUpdated": "Model Updated",
         "modelUpdateFailed": "Model Update Failed",

invokeai/frontend/web/src/app/components/InvokeAIUI.tsx

@@ -1,3 +1,4 @@
+import { store } from 'app/store/store';
 import React, {
   lazy,
   memo,
@@ -6,18 +7,17 @@ import React, {
   useEffect,
 } from 'react';
 import { Provider } from 'react-redux';
-import { store } from 'app/store/store';
 
-import Loading from '../../common/components/Loading/Loading';
-import { addMiddleware, resetMiddlewares } from 'redux-dynamic-middlewares';
 import { PartialAppConfig } from 'app/types/invokeai';
+import { addMiddleware, resetMiddlewares } from 'redux-dynamic-middlewares';
+import Loading from '../../common/components/Loading/Loading';
 
-import '../../i18n';
-import { socketMiddleware } from 'services/events/middleware';
 import { Middleware } from '@reduxjs/toolkit';
-import ImageDndContext from './ImageDnd/ImageDndContext';
-import { AddImageToBoardContextProvider } from '../contexts/AddImageToBoardContext';
 import { $authToken, $baseUrl } from 'services/api/client';
+import { socketMiddleware } from 'services/events/middleware';
+import '../../i18n';
+import { AddImageToBoardContextProvider } from '../contexts/AddImageToBoardContext';
+import ImageDndContext from './ImageDnd/ImageDndContext';
 
 const App = lazy(() => import('./App'));
 const ThemeLocaleProvider = lazy(() => import('./ThemeLocaleProvider'));
@@ -28,6 +28,7 @@ interface Props extends PropsWithChildren {
   config?: PartialAppConfig;
   headerComponent?: ReactNode;
   middleware?: Middleware[];
+  projectId?: string;
 }
 
 const InvokeAIUI = ({

invokeai/frontend/web/src/app/store/middleware/listenerMiddleware/listeners/modelSelected.ts

@@ -6,7 +6,7 @@ import {
   modelChanged,
   vaeSelected,
 } from 'features/parameters/store/generationSlice';
-import { zMainModel } from 'features/parameters/types/parameterSchemas';
+import { zMainOrOnnxModel } from 'features/parameters/types/parameterSchemas';
 import { addToast } from 'features/system/store/systemSlice';
 import { makeToast } from 'features/system/util/makeToast';
 import { forEach } from 'lodash-es';
@@ -19,7 +19,7 @@ export const addModelSelectedListener = () => {
       const log = logger('models');
 
       const state = getState();
-      const result = zMainModel.safeParse(action.payload);
+      const result = zMainOrOnnxModel.safeParse(action.payload);
 
       if (!result.success) {
         log.error(

invokeai/frontend/web/src/app/store/middleware/listenerMiddleware/listeners/modelsLoaded.ts

@@ -6,7 +6,8 @@ import {
   vaeSelected,
 } from 'features/parameters/store/generationSlice';
 import {
-  zMainModel,
+  zMainOrOnnxModel,
+  zSDXLRefinerModel,
   zVaeModel,
 } from 'features/parameters/types/parameterSchemas';
 import {
@@ -36,7 +37,8 @@ export const addModelsLoadedListener = () => {
         action.payload.entities,
         (m) =>
           m?.model_name === currentModel?.model_name &&
-          m?.base_model === currentModel?.base_model
+          m?.base_model === currentModel?.base_model &&
+          m?.model_type === currentModel?.model_type
       );
 
       if (isCurrentModelAvailable) {
@@ -52,7 +54,7 @@ export const addModelsLoadedListener = () => {
         return;
       }
 
-      const result = zMainModel.safeParse(firstModel);
+      const result = zMainOrOnnxModel.safeParse(firstModel);
 
       if (!result.success) {
         log.error(
@@ -83,7 +85,8 @@ export const addModelsLoadedListener = () => {
         action.payload.entities,
         (m) =>
           m?.model_name === currentModel?.model_name &&
-          m?.base_model === currentModel?.base_model
+          m?.base_model === currentModel?.base_model &&
+          m?.model_type === currentModel?.model_type
       );
 
       if (isCurrentModelAvailable) {
@@ -100,7 +103,7 @@ export const addModelsLoadedListener = () => {
         return;
       }
 
-      const result = zMainModel.safeParse(firstModel);
+      const result = zSDXLRefinerModel.safeParse(firstModel);
 
       if (!result.success) {
         log.error(

invokeai/frontend/web/src/app/store/middleware/listenerMiddleware/listeners/tabChanged.ts

@@ -47,9 +47,9 @@ export const addTabChangedListener = () => {
         }
 
         // only store the model name and base model in redux
-        const { base_model, model_name } = firstValidCanvasModel;
+        const { base_model, model_name, model_type } = firstValidCanvasModel;
 
-        dispatch(modelChanged({ base_model, model_name }));
+        dispatch(modelChanged({ base_model, model_name, model_type }));
       }
     },
   });

invokeai/frontend/web/src/features/nodes/components/fields/ModelInputFieldComponent.tsx

@@ -14,8 +14,11 @@ import SyncModelsButton from 'features/ui/components/tabs/ModelManager/subpanels
 import { forEach } from 'lodash-es';
 import { memo, useCallback, useMemo } from 'react';
 import { useTranslation } from 'react-i18next';
+import {
+  useGetMainModelsQuery,
+  useGetOnnxModelsQuery,
+} from 'services/api/endpoints/models';
 import { NON_REFINER_BASE_MODELS } from 'services/api/constants';
-import { useGetMainModelsQuery } from 'services/api/endpoints/models';
 import { FieldComponentProps } from './types';
 import { useFeatureStatus } from '../../../system/hooks/useFeatureStatus';
 
@@ -28,6 +31,7 @@ const ModelInputFieldComponent = (
   const { t } = useTranslation();
   const isSyncModelEnabled = useFeatureStatus('syncModels').isFeatureEnabled;
 
+  const { data: onnxModels } = useGetOnnxModelsQuery(NON_REFINER_BASE_MODELS);
   const { data: mainModels, isLoading } = useGetMainModelsQuery(
     NON_REFINER_BASE_MODELS
   );
@@ -51,17 +55,39 @@ const ModelInputFieldComponent = (
       });
     });
 
+    if (onnxModels) {
+      forEach(onnxModels.entities, (model, id) => {
+        if (!model) {
+          return;
+        }
+
+        data.push({
+          value: id,
+          label: model.model_name,
+          group: MODEL_TYPE_MAP[model.base_model],
+        });
+      });
+    }
     return data;
-  }, [mainModels]);
+  }, [mainModels, onnxModels]);
 
   // grab the full model entity from the RTK Query cache
   // TODO: maybe we should just store the full model entity in state?
   const selectedModel = useMemo(
     () =>
-      mainModels?.entities[
+      (mainModels?.entities[
         `${field.value?.base_model}/main/${field.value?.model_name}`
-      ] ?? null,
-    [field.value?.base_model, field.value?.model_name, mainModels?.entities]
+      ] ||
+        onnxModels?.entities[
+          `${field.value?.base_model}/onnx/${field.value?.model_name}`
+        ]) ??
+      null,
+    [
+      field.value?.base_model,
+      field.value?.model_name,
+      mainModels?.entities,
+      onnxModels?.entities,
+    ]
   );
 
   const handleChangeModel = useCallback(

invokeai/frontend/web/src/features/nodes/util/graphBuilders/addLoRAsToGraph.ts

@@ -9,6 +9,7 @@ import {
   CLIP_SKIP,
   LORA_LOADER,
   MAIN_MODEL_LOADER,
+  ONNX_MODEL_LOADER,
   METADATA_ACCUMULATOR,
   NEGATIVE_CONDITIONING,
   POSITIVE_CONDITIONING,
@@ -17,7 +18,8 @@ import {
 export const addLoRAsToGraph = (
   state: RootState,
   graph: NonNullableGraph,
-  baseNodeId: string
+  baseNodeId: string,
+  modelLoaderNodeId: string = MAIN_MODEL_LOADER
 ): void => {
   /**
    * LoRA nodes get the UNet and CLIP models from the main model loader and apply the LoRA to them.
@@ -40,6 +42,10 @@ export const addLoRAsToGraph = (
         !(
           e.source.node_id === MAIN_MODEL_LOADER &&
           ['unet'].includes(e.source.field)
+        ) &&
+        !(
+          e.source.node_id === ONNX_MODEL_LOADER &&
+          ['unet'].includes(e.source.field)
         )
     );
     // Remove CLIP_SKIP connections to conditionings to feed it through LoRAs
@@ -75,12 +81,11 @@ export const addLoRAsToGraph = (
 
     // add to graph
     graph.nodes[currentLoraNodeId] = loraLoaderNode;
-
     if (currentLoraIndex === 0) {
       // first lora = start the lora chain, attach directly to model loader
       graph.edges.push({
         source: {
-          node_id: MAIN_MODEL_LOADER,
+          node_id: modelLoaderNodeId,
           field: 'unet',
         },
         destination: {

invokeai/frontend/web/src/features/nodes/util/graphBuilders/addVAEToGraph.ts

@@ -9,13 +9,15 @@ import {
   LATENTS_TO_IMAGE,
   MAIN_MODEL_LOADER,
   METADATA_ACCUMULATOR,
+  ONNX_MODEL_LOADER,
   TEXT_TO_IMAGE_GRAPH,
   VAE_LOADER,
 } from './constants';
 
 export const addVAEToGraph = (
   state: RootState,
-  graph: NonNullableGraph
+  graph: NonNullableGraph,
+  modelLoaderNodeId: string = MAIN_MODEL_LOADER
 ): void => {
   const { vae } = state.generation;
 
@@ -32,12 +34,12 @@ export const addVAEToGraph = (
       vae_model: vae,
     };
   }
-
+  const isOnnxModel = modelLoaderNodeId == ONNX_MODEL_LOADER;
   if (graph.id === TEXT_TO_IMAGE_GRAPH || graph.id === IMAGE_TO_IMAGE_GRAPH) {
     graph.edges.push({
       source: {
-        node_id: isAutoVae ? MAIN_MODEL_LOADER : VAE_LOADER,
-        field: 'vae',
+        node_id: isAutoVae ? modelLoaderNodeId : VAE_LOADER,
+        field: isAutoVae && isOnnxModel ? 'vae_decoder' : 'vae',
       },
       destination: {
         node_id: LATENTS_TO_IMAGE,
@@ -49,8 +51,8 @@ export const addVAEToGraph = (
   if (graph.id === IMAGE_TO_IMAGE_GRAPH) {
     graph.edges.push({
       source: {
-        node_id: isAutoVae ? MAIN_MODEL_LOADER : VAE_LOADER,
-        field: 'vae',
+        node_id: isAutoVae ? modelLoaderNodeId : VAE_LOADER,
+        field: isAutoVae && isOnnxModel ? 'vae_decoder' : 'vae',
       },
       destination: {
         node_id: IMAGE_TO_LATENTS,
@@ -62,8 +64,8 @@ export const addVAEToGraph = (
   if (graph.id === INPAINT_GRAPH) {
     graph.edges.push({
       source: {
-        node_id: isAutoVae ? MAIN_MODEL_LOADER : VAE_LOADER,
-        field: 'vae',
+        node_id: isAutoVae ? modelLoaderNodeId : VAE_LOADER,
+        field: isAutoVae && isOnnxModel ? 'vae_decoder' : 'vae',
       },
       destination: {
         node_id: INPAINT,

invokeai/frontend/web/src/features/nodes/util/graphBuilders/buildCanvasTextToImageGraph.ts

@@ -12,6 +12,7 @@ import {
   CLIP_SKIP,
   LATENTS_TO_IMAGE,
   MAIN_MODEL_LOADER,
+  ONNX_MODEL_LOADER,
   METADATA_ACCUMULATOR,
   NEGATIVE_CONDITIONING,
   NOISE,
@@ -19,6 +20,10 @@ import {
   TEXT_TO_IMAGE_GRAPH,
   TEXT_TO_LATENTS,
 } from './constants';
+import {
+  ONNXTextToLatentsInvocation,
+  TextToLatentsInvocation,
+} from 'services/api/types';
 
 /**
  * Builds the Canvas tab's Text to Image graph.
@@ -52,7 +57,31 @@ export const buildCanvasTextToImageGraph = (
   const use_cpu = shouldUseNoiseSettings
     ? shouldUseCpuNoise
     : initialGenerationState.shouldUseCpuNoise;
-
+  const isUsingOnnxModel = model.model_type === 'onnx';
+  const modelLoaderNodeId = isUsingOnnxModel
+    ? ONNX_MODEL_LOADER
+    : MAIN_MODEL_LOADER;
+  const modelLoaderNodeType = isUsingOnnxModel
+    ? 'onnx_model_loader'
+    : 'main_model_loader';
+  const t2lNode: TextToLatentsInvocation | ONNXTextToLatentsInvocation =
+    isUsingOnnxModel
+      ? {
+          type: 't2l_onnx',
+          id: TEXT_TO_LATENTS,
+          is_intermediate: true,
+          cfg_scale,
+          scheduler,
+          steps,
+        }
+      : {
+          type: 't2l',
+          id: TEXT_TO_LATENTS,
+          is_intermediate: true,
+          cfg_scale,
+          scheduler,
+          steps,
+        };
   /**
    * The easiest way to build linear graphs is to do it in the node editor, then copy and paste the
    * full graph here as a template. Then use the parameters from app state and set friendlier node
@@ -63,17 +92,18 @@ export const buildCanvasTextToImageGraph = (
    */
 
   // copy-pasted graph from node editor, filled in with state values & friendly node ids
+  // TODO: Actually create the graph correctly for ONNX
   const graph: NonNullableGraph = {
     id: TEXT_TO_IMAGE_GRAPH,
     nodes: {
       [POSITIVE_CONDITIONING]: {
-        type: 'compel',
+        type: isUsingOnnxModel ? 'prompt_onnx' : 'compel',
         id: POSITIVE_CONDITIONING,
         is_intermediate: true,
         prompt: positivePrompt,
       },
       [NEGATIVE_CONDITIONING]: {
-        type: 'compel',
+        type: isUsingOnnxModel ? 'prompt_onnx' : 'compel',
         id: NEGATIVE_CONDITIONING,
         is_intermediate: true,
         prompt: negativePrompt,
@@ -86,17 +116,10 @@ export const buildCanvasTextToImageGraph = (
         height,
         use_cpu,
       },
-      [TEXT_TO_LATENTS]: {
-        type: 't2l',
-        id: TEXT_TO_LATENTS,
-        is_intermediate: true,
-        cfg_scale,
-        scheduler,
-        steps,
-      },
-      [MAIN_MODEL_LOADER]: {
-        type: 'main_model_loader',
-        id: MAIN_MODEL_LOADER,
+      [t2lNode.id]: t2lNode,
+      [modelLoaderNodeId]: {
+        type: modelLoaderNodeType,
+        id: modelLoaderNodeId,
         is_intermediate: true,
         model,
       },
@@ -107,7 +130,7 @@ export const buildCanvasTextToImageGraph = (
         skipped_layers: clipSkip,
       },
       [LATENTS_TO_IMAGE]: {
-        type: 'l2i',
+        type: isUsingOnnxModel ? 'l2i_onnx' : 'l2i',
         id: LATENTS_TO_IMAGE,
         is_intermediate: !shouldAutoSave,
       },
@@ -135,7 +158,7 @@ export const buildCanvasTextToImageGraph = (
       },
       {
         source: {
-          node_id: MAIN_MODEL_LOADER,
+          node_id: modelLoaderNodeId,
           field: 'clip',
         },
         destination: {
@@ -165,7 +188,7 @@ export const buildCanvasTextToImageGraph = (
       },
       {
         source: {
-          node_id: MAIN_MODEL_LOADER,
+          node_id: modelLoaderNodeId,
           field: 'unet',
         },
         destination: {
@@ -229,10 +252,10 @@ export const buildCanvasTextToImageGraph = (
   });
 
   // add LoRA support
-  addLoRAsToGraph(state, graph, TEXT_TO_LATENTS);
+  addLoRAsToGraph(state, graph, TEXT_TO_LATENTS, modelLoaderNodeId);
 
   // optionally add custom VAE
-  addVAEToGraph(state, graph);
+  addVAEToGraph(state, graph, modelLoaderNodeId);
 
   // add dynamic prompts - also sets up core iteration and seed
   addDynamicPromptsToGraph(state, graph);

invokeai/frontend/web/src/features/nodes/util/graphBuilders/buildLinearTextToImageGraph.ts

@@ -12,6 +12,7 @@ import {
   CLIP_SKIP,
   LATENTS_TO_IMAGE,
   MAIN_MODEL_LOADER,
+  ONNX_MODEL_LOADER,
   METADATA_ACCUMULATOR,
   NEGATIVE_CONDITIONING,
   NOISE,
@@ -19,6 +20,10 @@ import {
   TEXT_TO_IMAGE_GRAPH,
   TEXT_TO_LATENTS,
 } from './constants';
+import {
+  ONNXTextToLatentsInvocation,
+  TextToLatentsInvocation,
+} from 'services/api/types';
 
 export const buildLinearTextToImageGraph = (
   state: RootState
@@ -48,6 +53,31 @@ export const buildLinearTextToImageGraph = (
     throw new Error('No model found in state');
   }
 
+  const isUsingOnnxModel = model.model_type === 'onnx';
+  const modelLoaderNodeId = isUsingOnnxModel
+    ? ONNX_MODEL_LOADER
+    : MAIN_MODEL_LOADER;
+  const modelLoaderNodeType = isUsingOnnxModel
+    ? 'onnx_model_loader'
+    : 'main_model_loader';
+  const t2lNode: TextToLatentsInvocation | ONNXTextToLatentsInvocation =
+    isUsingOnnxModel
+      ? {
+          type: 't2l_onnx',
+          id: TEXT_TO_LATENTS,
+          is_intermediate: true,
+          cfg_scale,
+          scheduler,
+          steps,
+        }
+      : {
+          type: 't2l',
+          id: TEXT_TO_LATENTS,
+          is_intermediate: true,
+          cfg_scale,
+          scheduler,
+          steps,
+        };
   /**
    * The easiest way to build linear graphs is to do it in the node editor, then copy and paste the
    * full graph here as a template. Then use the parameters from app state and set friendlier node
@@ -58,28 +88,22 @@ export const buildLinearTextToImageGraph = (
    */
 
   // copy-pasted graph from node editor, filled in with state values & friendly node ids
+
+  // TODO: Actually create the graph correctly for ONNX
   const graph: NonNullableGraph = {
     id: TEXT_TO_IMAGE_GRAPH,
     nodes: {
-      [MAIN_MODEL_LOADER]: {
-        type: 'main_model_loader',
-        id: MAIN_MODEL_LOADER,
-        model,
-      },
-      [CLIP_SKIP]: {
-        type: 'clip_skip',
-        id: CLIP_SKIP,
-        skipped_layers: clipSkip,
-      },
       [POSITIVE_CONDITIONING]: {
-        type: 'compel',
+        type: isUsingOnnxModel ? 'prompt_onnx' : 'compel',
         id: POSITIVE_CONDITIONING,
         prompt: positivePrompt,
+        is_intermediate: true,
       },
       [NEGATIVE_CONDITIONING]: {
-        type: 'compel',
+        type: isUsingOnnxModel ? 'prompt_onnx' : 'compel',
         id: NEGATIVE_CONDITIONING,
         prompt: negativePrompt,
+        is_intermediate: true,
       },
       [NOISE]: {
         type: 'noise',
@@ -87,16 +111,23 @@ export const buildLinearTextToImageGraph = (
         width,
         height,
         use_cpu,
+        is_intermediate: true,
       },
-      [TEXT_TO_LATENTS]: {
-        type: 't2l',
-        id: TEXT_TO_LATENTS,
-        cfg_scale,
-        scheduler,
-        steps,
+      [t2lNode.id]: t2lNode,
+      [modelLoaderNodeId]: {
+        type: modelLoaderNodeType,
+        id: modelLoaderNodeId,
+        is_intermediate: true,
+        model,
+      },
+      [CLIP_SKIP]: {
+        type: 'clip_skip',
+        id: CLIP_SKIP,
+        skipped_layers: clipSkip,
+        is_intermediate: true,
       },
       [LATENTS_TO_IMAGE]: {
-        type: 'l2i',
+        type: isUsingOnnxModel ? 'l2i_onnx' : 'l2i',
         id: LATENTS_TO_IMAGE,
         fp32: vaePrecision === 'fp32' ? true : false,
       },
@@ -104,7 +135,7 @@ export const buildLinearTextToImageGraph = (
     edges: [
       {
         source: {
-          node_id: MAIN_MODEL_LOADER,
+          node_id: modelLoaderNodeId,
           field: 'clip',
         },
         destination: {
@@ -114,7 +145,7 @@ export const buildLinearTextToImageGraph = (
       },
       {
         source: {
-          node_id: MAIN_MODEL_LOADER,
+          node_id: modelLoaderNodeId,
           field: 'unet',
         },
         destination: {
@@ -218,10 +249,10 @@ export const buildLinearTextToImageGraph = (
   });
 
   // add LoRA support
-  addLoRAsToGraph(state, graph, TEXT_TO_LATENTS);
+  addLoRAsToGraph(state, graph, TEXT_TO_LATENTS, modelLoaderNodeId);
 
   // optionally add custom VAE
-  addVAEToGraph(state, graph);
+  addVAEToGraph(state, graph, modelLoaderNodeId);
 
   // add dynamic prompts - also sets up core iteration and seed
   addDynamicPromptsToGraph(state, graph);

invokeai/frontend/web/src/features/nodes/util/graphBuilders/constants.ts

@@ -10,6 +10,7 @@ export const RANDOM_INT = 'rand_int';
 export const RANGE_OF_SIZE = 'range_of_size';
 export const ITERATE = 'iterate';
 export const MAIN_MODEL_LOADER = 'main_model_loader';
+export const ONNX_MODEL_LOADER = 'onnx_model_loader';
 export const VAE_LOADER = 'vae_loader';
 export const LORA_LOADER = 'lora_loader';
 export const CLIP_SKIP = 'clip_skip';

invokeai/frontend/web/src/features/parameters/components/Parameters/MainModel/ParamMainModelSelect.tsx

@@ -15,8 +15,11 @@ import { modelIdToMainModelParam } from 'features/parameters/util/modelIdToMainM
 import SyncModelsButton from 'features/ui/components/tabs/ModelManager/subpanels/ModelManagerSettingsPanel/SyncModelsButton';
 import { activeTabNameSelector } from 'features/ui/store/uiSelectors';
 import { forEach } from 'lodash-es';
+import {
+  useGetMainModelsQuery,
+  useGetOnnxModelsQuery,
+} from 'services/api/endpoints/models';
 import { NON_REFINER_BASE_MODELS } from 'services/api/constants';
-import { useGetMainModelsQuery } from 'services/api/endpoints/models';
 import { useFeatureStatus } from '../../../../system/hooks/useFeatureStatus';
 
 const selector = createSelector(
@@ -35,6 +38,9 @@ const ParamMainModelSelect = () => {
   const { data: mainModels, isLoading } = useGetMainModelsQuery(
     NON_REFINER_BASE_MODELS
   );
+  const { data: onnxModels, isLoading: onnxLoading } = useGetOnnxModelsQuery(
+    NON_REFINER_BASE_MODELS
+  );
 
   const activeTabName = useAppSelector(activeTabNameSelector);
 
@@ -59,17 +65,35 @@ const ParamMainModelSelect = () => {
         group: MODEL_TYPE_MAP[model.base_model],
       });
     });
+    forEach(onnxModels?.entities, (model, id) => {
+      if (
+        !model ||
+        activeTabName === 'unifiedCanvas' ||
+        activeTabName === 'img2img'
+      ) {
+        return;
+      }
+
+      data.push({
+        value: id,
+        label: model.model_name,
+        group: MODEL_TYPE_MAP[model.base_model],
+      });
+    });
 
     return data;
-  }, [mainModels, activeTabName]);
+  }, [mainModels, onnxModels, activeTabName]);
 
   // grab the full model entity from the RTK Query cache
   // TODO: maybe we should just store the full model entity in state?
   const selectedModel = useMemo(
     () =>
-      mainModels?.entities[`${model?.base_model}/main/${model?.model_name}`] ??
+      (mainModels?.entities[`${model?.base_model}/main/${model?.model_name}`] ||
+        onnxModels?.entities[
+          `${model?.base_model}/onnx/${model?.model_name}`
+        ]) ??
       null,
-    [mainModels?.entities, model]
+    [mainModels?.entities, model, onnxModels?.entities]
   );
 
   const handleChangeModel = useCallback(
@@ -89,7 +113,7 @@ const ParamMainModelSelect = () => {
     [dispatch]
   );
 
-  return isLoading ? (
+  return isLoading || onnxLoading ? (
     <IAIMantineSearchableSelect
       label={t('modelManager.model')}
       placeholder="Loading..."

invokeai/frontend/web/src/features/parameters/hooks/useRecallParameters.ts

@@ -35,6 +35,7 @@ import {
   isValidSDXLNegativeStylePrompt,
   isValidSDXLPositiveStylePrompt,
   isValidSDXLRefinerAestheticScore,
+  isValidSDXLRefinerModel,
   isValidSDXLRefinerStart,
   isValidScheduler,
   isValidSeed,
@@ -381,7 +382,7 @@ export const useRecallParameters = () => {
         dispatch(setNegativeStylePromptSDXL(negative_style_prompt));
       }
 
-      if (isValidMainModel(refiner_model)) {
+      if (isValidSDXLRefinerModel(refiner_model)) {
         dispatch(refinerModelChanged(refiner_model));
       }
 

invokeai/frontend/web/src/features/parameters/store/actions.ts

@@ -1,10 +1,10 @@
 import { createAction } from '@reduxjs/toolkit';
-import { ImageDTO, MainModelField } from 'services/api/types';
+import { ImageDTO, MainModelField, OnnxModelField } from 'services/api/types';
 
 export const initialImageSelected = createAction<ImageDTO | undefined>(
   'generation/initialImageSelected'
 );
 
-export const modelSelected = createAction<MainModelField>(
+export const modelSelected = createAction<MainModelField | OnnxModelField>(
   'generation/modelSelected'
 );

invokeai/frontend/web/src/features/parameters/store/generationSlice.ts

@@ -3,13 +3,14 @@ import { createSlice } from '@reduxjs/toolkit';
 import { roundToMultiple } from 'common/util/roundDownToMultiple';
 import { configChanged } from 'features/system/store/configSlice';
 import { clamp } from 'lodash-es';
-import { ImageDTO, MainModelField } from 'services/api/types';
+import { ImageDTO } from 'services/api/types';
 import { clipSkipMap } from '../types/constants';
 import {
   CfgScaleParam,
   HeightParam,
   MainModelParam,
   NegativePromptParam,
+  OnnxModelParam,
   PositivePromptParam,
   PrecisionParam,
   SchedulerParam,
@@ -50,7 +51,7 @@ export interface GenerationState {
   shouldUseSymmetry: boolean;
   horizontalSymmetrySteps: number;
   verticalSymmetrySteps: number;
-  model: MainModelField | null;
+  model: MainModelParam | OnnxModelParam | null;
   vae: VaeModelParam | null;
   vaePrecision: PrecisionParam;
   seamlessXAxis: boolean;
@@ -229,7 +230,10 @@ export const generationSlice = createSlice({
       const { image_name, width, height } = action.payload;
       state.initialImage = { imageName: image_name, width, height };
     },
-    modelChanged: (state, action: PayloadAction<MainModelParam | null>) => {
+    modelChanged: (
+      state,
+      action: PayloadAction<MainModelParam | OnnxModelParam | null>
+    ) => {
       state.model = action.payload;
 
       if (state.model === null) {
@@ -272,11 +276,12 @@ export const generationSlice = createSlice({
       const defaultModel = action.payload.sd?.defaultModel;
 
       if (defaultModel && !state.model) {
-        const [base_model, _model_type, model_name] = defaultModel.split('/');
+        const [base_model, model_type, model_name] = defaultModel.split('/');
 
         const result = zMainModel.safeParse({
           model_name,
           base_model,
+          model_type,
         });
 
         if (result.success) {

invokeai/frontend/web/src/features/parameters/types/parameterSchemas.ts

@@ -215,23 +215,65 @@ const zBaseModel = z.enum(['sd-1', 'sd-2', 'sdxl', 'sdxl-refiner']);
 export type BaseModelParam = z.infer<typeof zBaseModel>;
 
 /**
- * Zod schema for model parameter
+ * Zod schema for main model parameter
  * TODO: Make this a dynamically generated enum?
  */
 export const zMainModel = z.object({
   model_name: z.string().min(1),
   base_model: zBaseModel,
+  model_type: z.literal('main'),
 });
-
 /**
- * Type alias for model parameter, inferred from its zod schema
+ * Type alias for main model parameter, inferred from its zod schema
  */
 export type MainModelParam = z.infer<typeof zMainModel>;
 /**
- * Validates/type-guards a value as a model parameter
+ * Validates/type-guards a value as a main model parameter
  */
 export const isValidMainModel = (val: unknown): val is MainModelParam =>
   zMainModel.safeParse(val).success;
+
+/**
+ * Zod schema for SDXL refiner model parameter
+ * TODO: Make this a dynamically generated enum?
+ */
+export const zSDXLRefinerModel = z.object({
+  model_name: z.string().min(1),
+  base_model: z.literal('sdxl-refiner'),
+  model_type: z.literal('main'),
+});
+/**
+ * Type alias for SDXL refiner model parameter, inferred from its zod schema
+ */
+export type SDXLRefinerModelParam = z.infer<typeof zSDXLRefinerModel>;
+/**
+ * Validates/type-guards a value as a SDXL refiner model parameter
+ */
+export const isValidSDXLRefinerModel = (
+  val: unknown
+): val is SDXLRefinerModelParam => zSDXLRefinerModel.safeParse(val).success;
+
+/**
+ * Zod schema for Onnx model parameter
+ * TODO: Make this a dynamically generated enum?
+ */
+export const zOnnxModel = z.object({
+  model_name: z.string().min(1),
+  base_model: zBaseModel,
+  model_type: z.literal('onnx'),
+});
+/**
+ * Type alias for Onnx model parameter, inferred from its zod schema
+ */
+export type OnnxModelParam = z.infer<typeof zOnnxModel>;
+/**
+ * Validates/type-guards a value as a Onnx model parameter
+ */
+export const isValidOnnxModel = (val: unknown): val is OnnxModelParam =>
+  zOnnxModel.safeParse(val).success;
+
+export const zMainOrOnnxModel = z.union([zMainModel, zOnnxModel]);
+
 /**
  * Zod schema for VAE parameter
  */

invokeai/frontend/web/src/features/parameters/util/modelIdToMainModelParam.ts

@@ -1,18 +1,20 @@
 import { logger } from 'app/logging/logger';
 import {
   MainModelParam,
-  zMainModel,
+  OnnxModelParam,
+  zMainOrOnnxModel,
 } from 'features/parameters/types/parameterSchemas';
 
 export const modelIdToMainModelParam = (
   mainModelId: string
-): MainModelParam | undefined => {
+): OnnxModelParam | MainModelParam | undefined => {
   const log = logger('models');
-  const [base_model, _model_type, model_name] = mainModelId.split('/');
+  const [base_model, model_type, model_name] = mainModelId.split('/');
 
-  const result = zMainModel.safeParse({
+  const result = zMainOrOnnxModel.safeParse({
     base_model,
     model_name,
+    model_type,
   });
 
   if (!result.success) {

invokeai/frontend/web/src/features/parameters/util/modelIdToSDXLRefinerModelParam.ts

@@ -0,0 +1,31 @@
+import { logger } from 'app/logging/logger';
+import {
+  SDXLRefinerModelParam,
+  zSDXLRefinerModel,
+} from 'features/parameters/types/parameterSchemas';
+
+export const modelIdToSDXLRefinerModelParam = (
+  mainModelId: string
+): SDXLRefinerModelParam | undefined => {
+  const log = logger('models');
+  const [base_model, model_type, model_name] = mainModelId.split('/');
+
+  const result = zSDXLRefinerModel.safeParse({
+    base_model,
+    model_name,
+    model_type,
+  });
+
+  if (!result.success) {
+    log.error(
+      {
+        mainModelId,
+        errors: result.error.format(),
+      },
+      'Failed to parse main model id'
+    );
+    return;
+  }
+
+  return result.data;
+};

invokeai/frontend/web/src/features/sdxl/components/SDXLRefiner/ParamSDXLRefinerModelSelect.tsx

@@ -6,7 +6,7 @@ import { useAppDispatch, useAppSelector } from 'app/store/storeHooks';
 import { defaultSelectorOptions } from 'app/store/util/defaultMemoizeOptions';
 import IAIMantineSearchableSelect from 'common/components/IAIMantineSearchableSelect';
 import { MODEL_TYPE_MAP } from 'features/parameters/types/constants';
-import { modelIdToMainModelParam } from 'features/parameters/util/modelIdToMainModelParam';
+import { modelIdToSDXLRefinerModelParam } from 'features/parameters/util/modelIdToSDXLRefinerModelParam';
 import { refinerModelChanged } from 'features/sdxl/store/sdxlSlice';
 import { useFeatureStatus } from 'features/system/hooks/useFeatureStatus';
 import SyncModelsButton from 'features/ui/components/tabs/ModelManager/subpanels/ModelManagerSettingsPanel/SyncModelsButton';
@@ -68,7 +68,7 @@ const ParamSDXLRefinerModelSelect = () => {
         return;
       }
 
-      const newModel = modelIdToMainModelParam(v);
+      const newModel = modelIdToSDXLRefinerModelParam(v);
 
       if (!newModel) {
         return;

invokeai/frontend/web/src/features/sdxl/store/sdxlSlice.ts

@@ -1,11 +1,10 @@
 import { PayloadAction, createSlice } from '@reduxjs/toolkit';
 import {
-  MainModelParam,
   NegativeStylePromptSDXLParam,
   PositiveStylePromptSDXLParam,
+  SDXLRefinerModelParam,
   SchedulerParam,
 } from 'features/parameters/types/parameterSchemas';
-import { MainModelField } from 'services/api/types';
 
 type SDXLInitialState = {
   positiveStylePrompt: PositiveStylePromptSDXLParam;
@@ -13,7 +12,7 @@ type SDXLInitialState = {
   shouldConcatSDXLStylePrompt: boolean;
   shouldUseSDXLRefiner: boolean;
   sdxlImg2ImgDenoisingStrength: number;
-  refinerModel: MainModelField | null;
+  refinerModel: SDXLRefinerModelParam | null;
   refinerSteps: number;
   refinerCFGScale: number;
   refinerScheduler: SchedulerParam;
@@ -56,7 +55,7 @@ const sdxlSlice = createSlice({
     },
     refinerModelChanged: (
       state,
-      action: PayloadAction<MainModelParam | null>
+      action: PayloadAction<SDXLRefinerModelParam | null>
     ) => {
       state.refinerModel = action.payload;
     },

invokeai/frontend/web/src/features/ui/components/tabs/ModelManager/subpanels/ModelManagerPanel/ModelList.tsx

@@ -1,4 +1,4 @@
-import { ButtonGroup, Flex, Text } from '@chakra-ui/react';
+import { ButtonGroup, Flex, Spinner, Text } from '@chakra-ui/react';
 import { EntityState } from '@reduxjs/toolkit';
 import IAIButton from 'common/components/IAIButton';
 import IAIInput from 'common/components/IAIInput';
@@ -6,23 +6,25 @@ import { forEach } from 'lodash-es';
 import type { ChangeEvent, PropsWithChildren } from 'react';
 import { useCallback, useState } from 'react';
 import { useTranslation } from 'react-i18next';
+import { ALL_BASE_MODELS } from 'services/api/constants';
 import {
-  MainModelConfigEntity,
-  useGetMainModelsQuery,
-  useGetLoRAModelsQuery,
   LoRAModelConfigEntity,
+  MainModelConfigEntity,
+  OnnxModelConfigEntity,
+  useGetLoRAModelsQuery,
+  useGetMainModelsQuery,
+  useGetOnnxModelsQuery,
 } from 'services/api/endpoints/models';
 import ModelListItem from './ModelListItem';
-import { ALL_BASE_MODELS } from 'services/api/constants';
 
 type ModelListProps = {
   selectedModelId: string | undefined;
   setSelectedModelId: (name: string | undefined) => void;
 };
 
-type ModelFormat = 'images' | 'checkpoint' | 'diffusers';
+type ModelFormat = 'all' | 'checkpoint' | 'diffusers' | 'olive' | 'onnx';
 
-type ModelType = 'main' | 'lora';
+type ModelType = 'main' | 'lora' | 'onnx';
 
 type CombinedModelFormat = ModelFormat | 'lora';
 
@@ -31,35 +33,63 @@ const ModelList = (props: ModelListProps) => {
   const { t } = useTranslation();
   const [nameFilter, setNameFilter] = useState<string>('');
   const [modelFormatFilter, setModelFormatFilter] =
-    useState<CombinedModelFormat>('images');
+    useState<CombinedModelFormat>('all');
 
-  const { filteredDiffusersModels } = useGetMainModelsQuery(ALL_BASE_MODELS, {
-    selectFromResult: ({ data }) => ({
-      filteredDiffusersModels: modelsFilter(
-        data,
-        'main',
-        'diffusers',
-        nameFilter
-      ),
-    }),
-  });
+  const { filteredDiffusersModels, isLoadingDiffusersModels } =
+    useGetMainModelsQuery(ALL_BASE_MODELS, {
+      selectFromResult: ({ data, isLoading }) => ({
+        filteredDiffusersModels: modelsFilter(
+          data,
+          'main',
+          'diffusers',
+          nameFilter
+        ),
+        isLoadingDiffusersModels: isLoading,
+      }),
+    });
 
-  const { filteredCheckpointModels } = useGetMainModelsQuery(ALL_BASE_MODELS, {
-    selectFromResult: ({ data }) => ({
-      filteredCheckpointModels: modelsFilter(
-        data,
-        'main',
-        'checkpoint',
-        nameFilter
-      ),
-    }),
-  });
+  const { filteredCheckpointModels, isLoadingCheckpointModels } =
+    useGetMainModelsQuery(ALL_BASE_MODELS, {
+      selectFromResult: ({ data, isLoading }) => ({
+        filteredCheckpointModels: modelsFilter(
+          data,
+          'main',
+          'checkpoint',
+          nameFilter
+        ),
+        isLoadingCheckpointModels: isLoading,
+      }),
+    });
 
-  const { filteredLoraModels } = useGetLoRAModelsQuery(undefined, {
-    selectFromResult: ({ data }) => ({
-      filteredLoraModels: modelsFilter(data, 'lora', undefined, nameFilter),
-    }),
-  });
+  const { filteredLoraModels, isLoadingLoraModels } = useGetLoRAModelsQuery(
+    undefined,
+    {
+      selectFromResult: ({ data, isLoading }) => ({
+        filteredLoraModels: modelsFilter(data, 'lora', undefined, nameFilter),
+        isLoadingLoraModels: isLoading,
+      }),
+    }
+  );
+
+  const { filteredOnnxModels, isLoadingOnnxModels } = useGetOnnxModelsQuery(
+    ALL_BASE_MODELS,
+    {
+      selectFromResult: ({ data, isLoading }) => ({
+        filteredOnnxModels: modelsFilter(data, 'onnx', 'onnx', nameFilter),
+        isLoadingOnnxModels: isLoading,
+      }),
+    }
+  );
+
+  const { filteredOliveModels, isLoadingOliveModels } = useGetOnnxModelsQuery(
+    ALL_BASE_MODELS,
+    {
+      selectFromResult: ({ data, isLoading }) => ({
+        filteredOliveModels: modelsFilter(data, 'onnx', 'olive', nameFilter),
+        isLoadingOliveModels: isLoading,
+      }),
+    }
+  );
 
   const handleSearchFilter = useCallback((e: ChangeEvent<HTMLInputElement>) => {
     setNameFilter(e.target.value);
@@ -70,8 +100,8 @@ const ModelList = (props: ModelListProps) => {
       <Flex flexDirection="column" gap={4} paddingInlineEnd={4}>
         <ButtonGroup isAttached>
           <IAIButton
-            onClick={() => setModelFormatFilter('images')}
-            isChecked={modelFormatFilter === 'images'}
+            onClick={() => setModelFormatFilter('all')}
+            isChecked={modelFormatFilter === 'all'}
             size="sm"
           >
             {t('modelManager.allModels')}
@@ -90,6 +120,20 @@ const ModelList = (props: ModelListProps) => {
           >
             {t('modelManager.checkpointModels')}
           </IAIButton>
+          <IAIButton
+            size="sm"
+            onClick={() => setModelFormatFilter('onnx')}
+            isChecked={modelFormatFilter === 'onnx'}
+          >
+            {t('modelManager.onnxModels')}
+          </IAIButton>
+          <IAIButton
+            size="sm"
+            onClick={() => setModelFormatFilter('olive')}
+            isChecked={modelFormatFilter === 'olive'}
+          >
+            {t('modelManager.oliveModels')}
+          </IAIButton>
           <IAIButton
             size="sm"
             onClick={() => setModelFormatFilter('lora')}
@@ -111,59 +155,76 @@ const ModelList = (props: ModelListProps) => {
           maxHeight={window.innerHeight - 280}
           overflow="scroll"
         >
-          {['images', 'diffusers'].includes(modelFormatFilter) &&
+          {/* Diffusers List */}
+          {isLoadingDiffusersModels && (
+            <FetchingModelsLoader loadingMessage="Loading Diffusers..." />
+          )}
+          {['all', 'diffusers'].includes(modelFormatFilter) &&
+            !isLoadingDiffusersModels &&
             filteredDiffusersModels.length > 0 && (
-              <StyledModelContainer>
-                <Flex sx={{ gap: 2, flexDir: 'column' }}>
-                  <Text variant="subtext" fontSize="sm">
-                    Diffusers
-                  </Text>
-                  {filteredDiffusersModels.map((model) => (
-                    <ModelListItem
-                      key={model.id}
-                      model={model}
-                      isSelected={selectedModelId === model.id}
-                      setSelectedModelId={setSelectedModelId}
-                    />
-                  ))}
-                </Flex>
-              </StyledModelContainer>
+              <ModelListWrapper
+                title="Diffusers"
+                modelList={filteredDiffusersModels}
+                selected={{ selectedModelId, setSelectedModelId }}
+                key="diffusers"
+              />
             )}
-          {['images', 'checkpoint'].includes(modelFormatFilter) &&
+          {/* Checkpoints List */}
+          {isLoadingCheckpointModels && (
+            <FetchingModelsLoader loadingMessage="Loading Checkpoints..." />
+          )}
+          {['all', 'checkpoint'].includes(modelFormatFilter) &&
+            !isLoadingCheckpointModels &&
             filteredCheckpointModels.length > 0 && (
-              <StyledModelContainer>
-                <Flex sx={{ gap: 2, flexDir: 'column' }}>
-                  <Text variant="subtext" fontSize="sm">
-                    Checkpoints
-                  </Text>
-                  {filteredCheckpointModels.map((model) => (
-                    <ModelListItem
-                      key={model.id}
-                      model={model}
-                      isSelected={selectedModelId === model.id}
-                      setSelectedModelId={setSelectedModelId}
-                    />
-                  ))}
-                </Flex>
-              </StyledModelContainer>
+              <ModelListWrapper
+                title="Checkpoints"
+                modelList={filteredCheckpointModels}
+                selected={{ selectedModelId, setSelectedModelId }}
+                key="checkpoints"
+              />
             )}
-          {['images', 'lora'].includes(modelFormatFilter) &&
+
+          {/* LoRAs List */}
+          {isLoadingLoraModels && (
+            <FetchingModelsLoader loadingMessage="Loading LoRAs..." />
+          )}
+          {['all', 'lora'].includes(modelFormatFilter) &&
+            !isLoadingLoraModels &&
             filteredLoraModels.length > 0 && (
-              <StyledModelContainer>
-                <Flex sx={{ gap: 2, flexDir: 'column' }}>
-                  <Text variant="subtext" fontSize="sm">
-                    LoRAs
-                  </Text>
-                  {filteredLoraModels.map((model) => (
-                    <ModelListItem
-                      key={model.id}
-                      model={model}
-                      isSelected={selectedModelId === model.id}
-                      setSelectedModelId={setSelectedModelId}
-                    />
-                  ))}
-                </Flex>
-              </StyledModelContainer>
+              <ModelListWrapper
+                title="LoRAs"
+                modelList={filteredLoraModels}
+                selected={{ selectedModelId, setSelectedModelId }}
+                key="loras"
+              />
+            )}
+          {/* Olive List */}
+          {isLoadingOliveModels && (
+            <FetchingModelsLoader loadingMessage="Loading Olives..." />
+          )}
+          {['all', 'olive'].includes(modelFormatFilter) &&
+            !isLoadingOliveModels &&
+            filteredOliveModels.length > 0 && (
+              <ModelListWrapper
+                title="Olives"
+                modelList={filteredOliveModels}
+                selected={{ selectedModelId, setSelectedModelId }}
+                key="olive"
+              />
+            )}
+          {/* Onnx List */}
+          {isLoadingOnnxModels && (
+            <FetchingModelsLoader loadingMessage="Loading ONNX..." />
+          )}
+          {['all', 'onnx'].includes(modelFormatFilter) &&
+            !isLoadingOnnxModels &&
+            filteredOnnxModels.length > 0 && (
+              <ModelListWrapper
+                title="ONNX"
+                modelList={filteredOnnxModels}
+                selected={{ selectedModelId, setSelectedModelId }}
+                key="onnx"
+              />
             )}
         </Flex>
       </Flex>
@@ -173,7 +234,12 @@ const ModelList = (props: ModelListProps) => {
 
 export default ModelList;
 
-const modelsFilter = <T extends MainModelConfigEntity | LoRAModelConfigEntity>(
+const modelsFilter = <
+  T extends
+    | MainModelConfigEntity
+    | LoRAModelConfigEntity
+    | OnnxModelConfigEntity
+>(
   data: EntityState<T> | undefined,
   model_type: ModelType,
   model_format: ModelFormat | undefined,
@@ -218,3 +284,52 @@ const StyledModelContainer = (props: PropsWithChildren) => {
     </Flex>
   );
 };
+
+type ModelListWrapperProps = {
+  title: string;
+  modelList:
+    | MainModelConfigEntity[]
+    | LoRAModelConfigEntity[]
+    | OnnxModelConfigEntity[];
+  selected: ModelListProps;
+};
+
+function ModelListWrapper(props: ModelListWrapperProps) {
+  const { title, modelList, selected } = props;
+  return (
+    <StyledModelContainer>
+      <Flex sx={{ gap: 2, flexDir: 'column' }}>
+        <Text variant="subtext" fontSize="sm">
+          {title}
+        </Text>
+        {modelList.map((model) => (
+          <ModelListItem
+            key={model.id}
+            model={model}
+            isSelected={selected.selectedModelId === model.id}
+            setSelectedModelId={selected.setSelectedModelId}
+          />
+        ))}
+      </Flex>
+    </StyledModelContainer>
+  );
+}
+
+function FetchingModelsLoader({ loadingMessage }: { loadingMessage?: string }) {
+  return (
+    <StyledModelContainer>
+      <Flex
+        justifyContent="center"
+        alignItems="center"
+        flexDirection="column"
+        p={4}
+        gap={8}
+      >
+        <Spinner />
+        <Text variant="subtext">
+          {loadingMessage ? loadingMessage : 'Fetching...'}
+        </Text>
+      </Flex>
+    </StyledModelContainer>
+  );
+}

invokeai/frontend/web/src/features/ui/components/tabs/ModelManager/subpanels/ModelManagerPanel/ModelListItem.tsx

@@ -15,10 +15,11 @@ import {
   LoRAModelConfigEntity,
   useDeleteMainModelsMutation,
   useDeleteLoRAModelsMutation,
+  OnnxModelConfigEntity,
 } from 'services/api/endpoints/models';
 
 type ModelListItemProps = {
-  model: MainModelConfigEntity | LoRAModelConfigEntity;
+  model: MainModelConfigEntity | OnnxModelConfigEntity | LoRAModelConfigEntity;
   isSelected: boolean;
   setSelectedModelId: (v: string | undefined) => void;
 };
@@ -37,9 +38,12 @@ export default function ModelListItem(props: ModelListItemProps) {
   }, [model.id, setSelectedModelId]);
 
   const handleModelDelete = useCallback(() => {
-    const method = { main: deleteMainModel, lora: deleteLoRAModel }[
-      model.model_type
-    ];
+    const method = {
+      main: deleteMainModel,
+      lora: deleteLoRAModel,
+      onnx: deleteMainModel,
+    }[model.model_type];
+
     method(model)
       .unwrap()
       .then((_) => {

invokeai/frontend/web/src/index.ts

@@ -6,3 +6,4 @@ export { default as ParamMainModelSelect } from './features/parameters/component
 export { default as InvokeAiLogoComponent } from './features/system/components/InvokeAILogoComponent';
 export { default as SettingsModal } from './features/system/components/SettingsModal/SettingsModal';
 export { default as StatusIndicator } from './features/system/components/StatusIndicator';
+export { theme as theme } from './theme/theme';

invokeai/frontend/web/src/services/api/endpoints/models.ts

@@ -10,9 +10,11 @@ import {
   ImportModelConfig,
   LoRAModelConfig,
   MainModelConfig,
+  OnnxModelConfig,
   MergeModelConfig,
   TextualInversionModelConfig,
   VaeModelConfig,
+  ModelType,
 } from 'services/api/types';
 
 import queryString from 'query-string';
@@ -27,6 +29,8 @@ export type MainModelConfigEntity =
   | DiffusersModelConfigEntity
   | CheckpointModelConfigEntity;
 
+export type OnnxModelConfigEntity = OnnxModelConfig & { id: string };
+
 export type LoRAModelConfigEntity = LoRAModelConfig & { id: string };
 
 export type ControlNetModelConfigEntity = ControlNetModelConfig & {
@@ -41,6 +45,7 @@ export type VaeModelConfigEntity = VaeModelConfig & { id: string };
 
 type AnyModelConfigEntity =
   | MainModelConfigEntity
+  | OnnxModelConfigEntity
   | LoRAModelConfigEntity
   | ControlNetModelConfigEntity
   | TextualInversionModelConfigEntity
@@ -66,6 +71,7 @@ type UpdateLoRAModelResponse = UpdateMainModelResponse;
 type DeleteMainModelArg = {
   base_model: BaseModelType;
   model_name: string;
+  model_type: ModelType;
 };
 
 type DeleteMainModelResponse = void;
@@ -119,6 +125,10 @@ type SearchFolderArg = operations['search_for_models']['parameters']['query'];
 const mainModelsAdapter = createEntityAdapter<MainModelConfigEntity>({
   sortComparer: (a, b) => a.model_name.localeCompare(b.model_name),
 });
+
+const onnxModelsAdapter = createEntityAdapter<OnnxModelConfigEntity>({
+  sortComparer: (a, b) => a.model_name.localeCompare(b.model_name),
+});
 const loraModelsAdapter = createEntityAdapter<LoRAModelConfigEntity>({
   sortComparer: (a, b) => a.model_name.localeCompare(b.model_name),
 });
@@ -156,6 +166,49 @@ const createModelEntities = <T extends AnyModelConfigEntity>(
 
 export const modelsApi = api.injectEndpoints({
   endpoints: (build) => ({
+    getOnnxModels: build.query<
+      EntityState<OnnxModelConfigEntity>,
+      BaseModelType[]
+    >({
+      query: (base_models) => {
+        const params = {
+          model_type: 'onnx',
+          base_models,
+        };
+
+        const query = queryString.stringify(params, { arrayFormat: 'none' });
+        return `models/?${query}`;
+      },
+      providesTags: (result, error, arg) => {
+        const tags: ApiFullTagDescription[] = [
+          { type: 'OnnxModel', id: LIST_TAG },
+        ];
+
+        if (result) {
+          tags.push(
+            ...result.ids.map((id) => ({
+              type: 'OnnxModel' as const,
+              id,
+            }))
+          );
+        }
+
+        return tags;
+      },
+      transformResponse: (
+        response: { models: OnnxModelConfig[] },
+        meta,
+        arg
+      ) => {
+        const entities = createModelEntities<OnnxModelConfigEntity>(
+          response.models
+        );
+        return onnxModelsAdapter.setAll(
+          onnxModelsAdapter.getInitialState(),
+          entities
+        );
+      },
+    }),
     getMainModels: build.query<
       EntityState<MainModelConfigEntity>,
       BaseModelType[]
@@ -213,6 +266,7 @@ export const modelsApi = api.injectEndpoints({
       invalidatesTags: [
         { type: 'MainModel', id: LIST_TAG },
         { type: 'SDXLRefinerModel', id: LIST_TAG },
+        { type: 'OnnxModel', id: LIST_TAG },
       ],
     }),
     importMainModels: build.mutation<
@@ -229,6 +283,7 @@ export const modelsApi = api.injectEndpoints({
       invalidatesTags: [
         { type: 'MainModel', id: LIST_TAG },
         { type: 'SDXLRefinerModel', id: LIST_TAG },
+        { type: 'OnnxModel', id: LIST_TAG },
       ],
     }),
     addMainModels: build.mutation<AddMainModelResponse, AddMainModelArg>({
@@ -242,21 +297,23 @@ export const modelsApi = api.injectEndpoints({
       invalidatesTags: [
         { type: 'MainModel', id: LIST_TAG },
         { type: 'SDXLRefinerModel', id: LIST_TAG },
+        { type: 'OnnxModel', id: LIST_TAG },
       ],
     }),
     deleteMainModels: build.mutation<
       DeleteMainModelResponse,
       DeleteMainModelArg
     >({
-      query: ({ base_model, model_name }) => {
+      query: ({ base_model, model_name, model_type }) => {
         return {
-          url: `models/${base_model}/main/${model_name}`,
+          url: `models/${base_model}/${model_type}/${model_name}`,
           method: 'DELETE',
         };
       },
       invalidatesTags: [
         { type: 'MainModel', id: LIST_TAG },
         { type: 'SDXLRefinerModel', id: LIST_TAG },
+        { type: 'OnnxModel', id: LIST_TAG },
       ],
     }),
     convertMainModels: build.mutation<
@@ -273,6 +330,7 @@ export const modelsApi = api.injectEndpoints({
       invalidatesTags: [
         { type: 'MainModel', id: LIST_TAG },
         { type: 'SDXLRefinerModel', id: LIST_TAG },
+        { type: 'OnnxModel', id: LIST_TAG },
       ],
     }),
     mergeMainModels: build.mutation<MergeMainModelResponse, MergeMainModelArg>({
@@ -286,6 +344,7 @@ export const modelsApi = api.injectEndpoints({
       invalidatesTags: [
         { type: 'MainModel', id: LIST_TAG },
         { type: 'SDXLRefinerModel', id: LIST_TAG },
+        { type: 'OnnxModel', id: LIST_TAG },
       ],
     }),
     syncModels: build.mutation<SyncModelsResponse, void>({
@@ -298,6 +357,7 @@ export const modelsApi = api.injectEndpoints({
       invalidatesTags: [
         { type: 'MainModel', id: LIST_TAG },
         { type: 'SDXLRefinerModel', id: LIST_TAG },
+        { type: 'OnnxModel', id: LIST_TAG },
       ],
     }),
     getLoRAModels: build.query<EntityState<LoRAModelConfigEntity>, void>({
@@ -494,6 +554,7 @@ export const modelsApi = api.injectEndpoints({
 
 export const {
   useGetMainModelsQuery,
+  useGetOnnxModelsQuery,
   useGetControlNetModelsQuery,
   useGetLoRAModelsQuery,
   useGetTextualInversionModelsQuery,

invokeai/frontend/web/src/services/api/schema.d.ts

@@ -1381,7 +1381,7 @@ export type components = {
        * @description The nodes in this graph
        */
       nodes?: {
-        [key: string]: (components["schemas"]["ControlNetInvocation"] | components["schemas"]["ImageProcessorInvocation"] | components["schemas"]["MainModelLoaderInvocation"] | components["schemas"]["LoraLoaderInvocation"] | components["schemas"]["VaeLoaderInvocation"] | components["schemas"]["MetadataAccumulatorInvocation"] | components["schemas"]["RangeInvocation"] | components["schemas"]["RangeOfSizeInvocation"] | components["schemas"]["RandomRangeInvocation"] | components["schemas"]["ImageCollectionInvocation"] | components["schemas"]["CompelInvocation"] | components["schemas"]["SDXLCompelPromptInvocation"] | components["schemas"]["SDXLRefinerCompelPromptInvocation"] | components["schemas"]["SDXLRawPromptInvocation"] | components["schemas"]["SDXLRefinerRawPromptInvocation"] | components["schemas"]["ClipSkipInvocation"] | components["schemas"]["LoadImageInvocation"] | components["schemas"]["ShowImageInvocation"] | components["schemas"]["ImageCropInvocation"] | components["schemas"]["ImagePasteInvocation"] | components["schemas"]["MaskFromAlphaInvocation"] | components["schemas"]["ImageMultiplyInvocation"] | components["schemas"]["ImageChannelInvocation"] | components["schemas"]["ImageConvertInvocation"] | components["schemas"]["ImageBlurInvocation"] | components["schemas"]["ImageResizeInvocation"] | components["schemas"]["ImageScaleInvocation"] | components["schemas"]["ImageLerpInvocation"] | components["schemas"]["ImageInverseLerpInvocation"] | components["schemas"]["ImageNSFWBlurInvocation"] | components["schemas"]["ImageWatermarkInvocation"] | components["schemas"]["CvInpaintInvocation"] | components["schemas"]["TextToLatentsInvocation"] | components["schemas"]["LatentsToImageInvocation"] | components["schemas"]["ResizeLatentsInvocation"] | components["schemas"]["ScaleLatentsInvocation"] | components["schemas"]["ImageToLatentsInvocation"] | components["schemas"]["InpaintInvocation"] | components["schemas"]["InfillColorInvocation"] | components["schemas"]["InfillTileInvocation"] | components["schemas"]["InfillPatchMatchInvocation"] | components["schemas"]["AddInvocation"] | components["schemas"]["SubtractInvocation"] | components["schemas"]["MultiplyInvocation"] | components["schemas"]["DivideInvocation"] | components["schemas"]["RandomIntInvocation"] | components["schemas"]["NoiseInvocation"] | components["schemas"]["DynamicPromptInvocation"] | components["schemas"]["PromptsFromFileInvocation"] | components["schemas"]["ParamIntInvocation"] | components["schemas"]["ParamFloatInvocation"] | components["schemas"]["ParamStringInvocation"] | components["schemas"]["ParamPromptInvocation"] | components["schemas"]["FloatLinearRangeInvocation"] | components["schemas"]["StepParamEasingInvocation"] | components["schemas"]["SDXLModelLoaderInvocation"] | components["schemas"]["SDXLRefinerModelLoaderInvocation"] | components["schemas"]["SDXLTextToLatentsInvocation"] | components["schemas"]["SDXLLatentsToLatentsInvocation"] | components["schemas"]["ESRGANInvocation"] | components["schemas"]["GraphInvocation"] | components["schemas"]["IterateInvocation"] | components["schemas"]["CollectInvocation"] | components["schemas"]["CannyImageProcessorInvocation"] | components["schemas"]["HedImageProcessorInvocation"] | components["schemas"]["LineartImageProcessorInvocation"] | components["schemas"]["LineartAnimeImageProcessorInvocation"] | components["schemas"]["OpenposeImageProcessorInvocation"] | components["schemas"]["MidasDepthImageProcessorInvocation"] | components["schemas"]["NormalbaeImageProcessorInvocation"] | components["schemas"]["MlsdImageProcessorInvocation"] | components["schemas"]["PidiImageProcessorInvocation"] | components["schemas"]["ContentShuffleImageProcessorInvocation"] | components["schemas"]["ZoeDepthImageProcessorInvocation"] | components["schemas"]["MediapipeFaceProcessorInvocation"] | components["schemas"]["LeresImageProcessorInvocation"] | components["schemas"]["TileResamplerProcessorInvocation"] | components["schemas"]["SegmentAnythingProcessorInvocation"] | components["schemas"]["LatentsToLatentsInvocation"]) | undefined;
+        [key: string]: (components["schemas"]["ControlNetInvocation"] | components["schemas"]["ImageProcessorInvocation"] | components["schemas"]["MainModelLoaderInvocation"] | components["schemas"]["LoraLoaderInvocation"] | components["schemas"]["VaeLoaderInvocation"] | components["schemas"]["MetadataAccumulatorInvocation"] | components["schemas"]["CompelInvocation"] | components["schemas"]["SDXLCompelPromptInvocation"] | components["schemas"]["SDXLRefinerCompelPromptInvocation"] | components["schemas"]["SDXLRawPromptInvocation"] | components["schemas"]["SDXLRefinerRawPromptInvocation"] | components["schemas"]["ClipSkipInvocation"] | components["schemas"]["LoadImageInvocation"] | components["schemas"]["ShowImageInvocation"] | components["schemas"]["ImageCropInvocation"] | components["schemas"]["ImagePasteInvocation"] | components["schemas"]["MaskFromAlphaInvocation"] | components["schemas"]["ImageMultiplyInvocation"] | components["schemas"]["ImageChannelInvocation"] | components["schemas"]["ImageConvertInvocation"] | components["schemas"]["ImageBlurInvocation"] | components["schemas"]["ImageResizeInvocation"] | components["schemas"]["ImageScaleInvocation"] | components["schemas"]["ImageLerpInvocation"] | components["schemas"]["ImageInverseLerpInvocation"] | components["schemas"]["ImageNSFWBlurInvocation"] | components["schemas"]["ImageWatermarkInvocation"] | components["schemas"]["TextToLatentsInvocation"] | components["schemas"]["LatentsToImageInvocation"] | components["schemas"]["ResizeLatentsInvocation"] | components["schemas"]["ScaleLatentsInvocation"] | components["schemas"]["ImageToLatentsInvocation"] | components["schemas"]["SDXLModelLoaderInvocation"] | components["schemas"]["SDXLRefinerModelLoaderInvocation"] | components["schemas"]["SDXLTextToLatentsInvocation"] | components["schemas"]["SDXLLatentsToLatentsInvocation"] | components["schemas"]["ONNXPromptInvocation"] | components["schemas"]["ONNXTextToLatentsInvocation"] | components["schemas"]["ONNXLatentsToImageInvocation"] | components["schemas"]["ONNXSD1ModelLoaderInvocation"] | components["schemas"]["OnnxModelLoaderInvocation"] | components["schemas"]["DynamicPromptInvocation"] | components["schemas"]["PromptsFromFileInvocation"] | components["schemas"]["AddInvocation"] | components["schemas"]["SubtractInvocation"] | components["schemas"]["MultiplyInvocation"] | components["schemas"]["DivideInvocation"] | components["schemas"]["RandomIntInvocation"] | components["schemas"]["ParamIntInvocation"] | components["schemas"]["ParamFloatInvocation"] | components["schemas"]["ParamStringInvocation"] | components["schemas"]["ParamPromptInvocation"] | components["schemas"]["CvInpaintInvocation"] | components["schemas"]["RangeInvocation"] | components["schemas"]["RangeOfSizeInvocation"] | components["schemas"]["RandomRangeInvocation"] | components["schemas"]["ImageCollectionInvocation"] | components["schemas"]["FloatLinearRangeInvocation"] | components["schemas"]["StepParamEasingInvocation"] | components["schemas"]["NoiseInvocation"] | components["schemas"]["ESRGANInvocation"] | components["schemas"]["InpaintInvocation"] | components["schemas"]["InfillColorInvocation"] | components["schemas"]["InfillTileInvocation"] | components["schemas"]["InfillPatchMatchInvocation"] | components["schemas"]["GraphInvocation"] | components["schemas"]["IterateInvocation"] | components["schemas"]["CollectInvocation"] | components["schemas"]["CannyImageProcessorInvocation"] | components["schemas"]["HedImageProcessorInvocation"] | components["schemas"]["LineartImageProcessorInvocation"] | components["schemas"]["LineartAnimeImageProcessorInvocation"] | components["schemas"]["OpenposeImageProcessorInvocation"] | components["schemas"]["MidasDepthImageProcessorInvocation"] | components["schemas"]["NormalbaeImageProcessorInvocation"] | components["schemas"]["MlsdImageProcessorInvocation"] | components["schemas"]["PidiImageProcessorInvocation"] | components["schemas"]["ContentShuffleImageProcessorInvocation"] | components["schemas"]["ZoeDepthImageProcessorInvocation"] | components["schemas"]["MediapipeFaceProcessorInvocation"] | components["schemas"]["LeresImageProcessorInvocation"] | components["schemas"]["TileResamplerProcessorInvocation"] | components["schemas"]["SegmentAnythingProcessorInvocation"] | components["schemas"]["LatentsToLatentsInvocation"]) | undefined;
       };
       /**
        * Edges 
@@ -1424,7 +1424,7 @@ export type components = {
        * @description The results of node executions
        */
       results: {
-        [key: string]: (components["schemas"]["ImageOutput"] | components["schemas"]["MaskOutput"] | components["schemas"]["ControlOutput"] | components["schemas"]["ModelLoaderOutput"] | components["schemas"]["LoraLoaderOutput"] | components["schemas"]["VaeLoaderOutput"] | components["schemas"]["MetadataAccumulatorOutput"] | components["schemas"]["IntCollectionOutput"] | components["schemas"]["FloatCollectionOutput"] | components["schemas"]["ImageCollectionOutput"] | components["schemas"]["CompelOutput"] | components["schemas"]["ClipSkipInvocationOutput"] | components["schemas"]["LatentsOutput"] | components["schemas"]["IntOutput"] | components["schemas"]["FloatOutput"] | components["schemas"]["NoiseOutput"] | components["schemas"]["PromptOutput"] | components["schemas"]["PromptCollectionOutput"] | components["schemas"]["StringOutput"] | components["schemas"]["SDXLModelLoaderOutput"] | components["schemas"]["SDXLRefinerModelLoaderOutput"] | components["schemas"]["GraphInvocationOutput"] | components["schemas"]["IterateInvocationOutput"] | components["schemas"]["CollectInvocationOutput"]) | undefined;
+        [key: string]: (components["schemas"]["ImageOutput"] | components["schemas"]["MaskOutput"] | components["schemas"]["ControlOutput"] | components["schemas"]["ModelLoaderOutput"] | components["schemas"]["LoraLoaderOutput"] | components["schemas"]["VaeLoaderOutput"] | components["schemas"]["MetadataAccumulatorOutput"] | components["schemas"]["CompelOutput"] | components["schemas"]["ClipSkipInvocationOutput"] | components["schemas"]["LatentsOutput"] | components["schemas"]["SDXLModelLoaderOutput"] | components["schemas"]["SDXLRefinerModelLoaderOutput"] | components["schemas"]["ONNXModelLoaderOutput"] | components["schemas"]["PromptOutput"] | components["schemas"]["PromptCollectionOutput"] | components["schemas"]["IntOutput"] | components["schemas"]["FloatOutput"] | components["schemas"]["StringOutput"] | components["schemas"]["IntCollectionOutput"] | components["schemas"]["FloatCollectionOutput"] | components["schemas"]["ImageCollectionOutput"] | components["schemas"]["NoiseOutput"] | components["schemas"]["GraphInvocationOutput"] | components["schemas"]["IterateInvocationOutput"] | components["schemas"]["CollectInvocationOutput"]) | undefined;
       };
       /**
        * Errors 
@@ -3173,6 +3173,8 @@ export type components = {
       model_name: string;
       /** @description Base model */
       base_model: components["schemas"]["BaseModelType"];
+      /** @description Model Type */
+      model_type: components["schemas"]["ModelType"];
     };
     /**
      * MainModelLoaderInvocation 
@@ -3618,7 +3620,7 @@ export type components = {
      * @description An enumeration. 
      * @enum {string}
      */
-    ModelType: "main" | "vae" | "lora" | "controlnet" | "embedding";
+    ModelType: "onnx" | "main" | "vae" | "lora" | "controlnet" | "embedding";
     /**
      * ModelVariantType 
      * @description An enumeration. 
@@ -3628,7 +3630,7 @@ export type components = {
     /** ModelsList */
     ModelsList: {
       /** Models */
-      models: (components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"])[];
+      models: (components["schemas"]["ONNXStableDiffusion1ModelConfig"] | components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["ONNXStableDiffusion2ModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"])[];
     };
     /**
      * MultiplyInvocation 
@@ -3778,6 +3780,261 @@ export type components = {
        */
       image_resolution?: number;
     };
+    /**
+     * ONNXLatentsToImageInvocation 
+     * @description Generates an image from latents.
+     */
+    ONNXLatentsToImageInvocation: {
+      /**
+       * Id 
+       * @description The id of this node. Must be unique among all nodes.
+       */
+      id: string;
+      /**
+       * Is Intermediate 
+       * @description Whether or not this node is an intermediate node. 
+       * @default false
+       */
+      is_intermediate?: boolean;
+      /**
+       * Type 
+       * @default l2i_onnx 
+       * @enum {string}
+       */
+      type?: "l2i_onnx";
+      /**
+       * Latents 
+       * @description The latents to generate an image from
+       */
+      latents?: components["schemas"]["LatentsField"];
+      /**
+       * Vae 
+       * @description Vae submodel
+       */
+      vae?: components["schemas"]["VaeField"];
+      /**
+       * Metadata 
+       * @description Optional core metadata to be written to the image
+       */
+      metadata?: components["schemas"]["CoreMetadata"];
+    };
+    /**
+     * ONNXModelLoaderOutput 
+     * @description Model loader output
+     */
+    ONNXModelLoaderOutput: {
+      /**
+       * Type 
+       * @default model_loader_output_onnx 
+       * @enum {string}
+       */
+      type?: "model_loader_output_onnx";
+      /**
+       * Unet 
+       * @description UNet submodel
+       */
+      unet?: components["schemas"]["UNetField"];
+      /**
+       * Clip 
+       * @description Tokenizer and text_encoder submodels
+       */
+      clip?: components["schemas"]["ClipField"];
+      /**
+       * Vae Decoder 
+       * @description Vae submodel
+       */
+      vae_decoder?: components["schemas"]["VaeField"];
+      /**
+       * Vae Encoder 
+       * @description Vae submodel
+       */
+      vae_encoder?: components["schemas"]["VaeField"];
+    };
+    /**
+     * ONNXPromptInvocation 
+     * @description A node to process inputs and produce outputs.
+     * May use dependency injection in __init__ to receive providers.
+     */
+    ONNXPromptInvocation: {
+      /**
+       * Id 
+       * @description The id of this node. Must be unique among all nodes.
+       */
+      id: string;
+      /**
+       * Is Intermediate 
+       * @description Whether or not this node is an intermediate node. 
+       * @default false
+       */
+      is_intermediate?: boolean;
+      /**
+       * Type 
+       * @default prompt_onnx 
+       * @enum {string}
+       */
+      type?: "prompt_onnx";
+      /**
+       * Prompt 
+       * @description Prompt 
+       * @default
+       */
+      prompt?: string;
+      /**
+       * Clip 
+       * @description Clip to use
+       */
+      clip?: components["schemas"]["ClipField"];
+    };
+    /**
+     * ONNXSD1ModelLoaderInvocation 
+     * @description Loading submodels of selected model.
+     */
+    ONNXSD1ModelLoaderInvocation: {
+      /**
+       * Id 
+       * @description The id of this node. Must be unique among all nodes.
+       */
+      id: string;
+      /**
+       * Is Intermediate 
+       * @description Whether or not this node is an intermediate node. 
+       * @default false
+       */
+      is_intermediate?: boolean;
+      /**
+       * Type 
+       * @default sd1_model_loader_onnx 
+       * @enum {string}
+       */
+      type?: "sd1_model_loader_onnx";
+      /**
+       * Model Name 
+       * @description Model to load 
+       * @default
+       */
+      model_name?: string;
+    };
+    /** ONNXStableDiffusion1ModelConfig */
+    ONNXStableDiffusion1ModelConfig: {
+      /** Model Name */
+      model_name: string;
+      base_model: components["schemas"]["BaseModelType"];
+      /**
+       * Model Type 
+       * @enum {string}
+       */
+      model_type: "onnx";
+      /** Path */
+      path: string;
+      /** Description */
+      description?: string;
+      /**
+       * Model Format 
+       * @enum {string}
+       */
+      model_format: "onnx";
+      error?: components["schemas"]["ModelError"];
+      variant: components["schemas"]["ModelVariantType"];
+    };
+    /** ONNXStableDiffusion2ModelConfig */
+    ONNXStableDiffusion2ModelConfig: {
+      /** Model Name */
+      model_name: string;
+      base_model: components["schemas"]["BaseModelType"];
+      /**
+       * Model Type 
+       * @enum {string}
+       */
+      model_type: "onnx";
+      /** Path */
+      path: string;
+      /** Description */
+      description?: string;
+      /**
+       * Model Format 
+       * @enum {string}
+       */
+      model_format: "onnx";
+      error?: components["schemas"]["ModelError"];
+      variant: components["schemas"]["ModelVariantType"];
+      prediction_type: components["schemas"]["SchedulerPredictionType"];
+      /** Upcast Attention */
+      upcast_attention: boolean;
+    };
+    /**
+     * ONNXTextToLatentsInvocation 
+     * @description Generates latents from conditionings.
+     */
+    ONNXTextToLatentsInvocation: {
+      /**
+       * Id 
+       * @description The id of this node. Must be unique among all nodes.
+       */
+      id: string;
+      /**
+       * Is Intermediate 
+       * @description Whether or not this node is an intermediate node. 
+       * @default false
+       */
+      is_intermediate?: boolean;
+      /**
+       * Type 
+       * @default t2l_onnx 
+       * @enum {string}
+       */
+      type?: "t2l_onnx";
+      /**
+       * Positive Conditioning 
+       * @description Positive conditioning for generation
+       */
+      positive_conditioning?: components["schemas"]["ConditioningField"];
+      /**
+       * Negative Conditioning 
+       * @description Negative conditioning for generation
+       */
+      negative_conditioning?: components["schemas"]["ConditioningField"];
+      /**
+       * Noise 
+       * @description The noise to use
+       */
+      noise?: components["schemas"]["LatentsField"];
+      /**
+       * Steps 
+       * @description The number of steps to use to generate the image 
+       * @default 10
+       */
+      steps?: number;
+      /**
+       * Cfg Scale 
+       * @description The Classifier-Free Guidance, higher values may result in a result closer to the prompt 
+       * @default 7.5
+       */
+      cfg_scale?: number | (number)[];
+      /**
+       * Scheduler 
+       * @description The scheduler to use 
+       * @default euler 
+       * @enum {string}
+       */
+      scheduler?: "ddim" | "ddpm" | "deis" | "lms" | "lms_k" | "pndm" | "heun" | "heun_k" | "euler" | "euler_k" | "euler_a" | "kdpm_2" | "kdpm_2_a" | "dpmpp_2s" | "dpmpp_2s_k" | "dpmpp_2m" | "dpmpp_2m_k" | "dpmpp_2m_sde" | "dpmpp_2m_sde_k" | "dpmpp_sde" | "dpmpp_sde_k" | "unipc";
+      /**
+       * Precision 
+       * @description The precision to use when generating latents 
+       * @default tensor(float16) 
+       * @enum {string}
+       */
+      precision?: "tensor(bool)" | "tensor(int8)" | "tensor(uint8)" | "tensor(int16)" | "tensor(uint16)" | "tensor(int32)" | "tensor(uint32)" | "tensor(int64)" | "tensor(uint64)" | "tensor(float16)" | "tensor(float)" | "tensor(double)";
+      /**
+       * Unet 
+       * @description UNet submodel
+       */
+      unet?: components["schemas"]["UNetField"];
+      /**
+       * Control 
+       * @description The control to use
+       */
+      control?: components["schemas"]["ControlField"] | (components["schemas"]["ControlField"])[];
+    };
     /**
      * OffsetPaginatedResults[BoardDTO] 
      * @description Offset-paginated results
@@ -3830,6 +4087,49 @@ export type components = {
        */
       total: number;
     };
+    /**
+     * OnnxModelField 
+     * @description Onnx model field
+     */
+    OnnxModelField: {
+      /**
+       * Model Name 
+       * @description Name of the model
+       */
+      model_name: string;
+      /** @description Base model */
+      base_model: components["schemas"]["BaseModelType"];
+      /** @description Model Type */
+      model_type: components["schemas"]["ModelType"];
+    };
+    /**
+     * OnnxModelLoaderInvocation 
+     * @description Loads a main model, outputting its submodels.
+     */
+    OnnxModelLoaderInvocation: {
+      /**
+       * Id 
+       * @description The id of this node. Must be unique among all nodes.
+       */
+      id: string;
+      /**
+       * Is Intermediate 
+       * @description Whether or not this node is an intermediate node. 
+       * @default false
+       */
+      is_intermediate?: boolean;
+      /**
+       * Type 
+       * @default onnx_model_loader 
+       * @enum {string}
+       */
+      type?: "onnx_model_loader";
+      /**
+       * Model 
+       * @description The model to load
+       */
+      model: components["schemas"]["OnnxModelField"];
+    };
     /**
      * OpenposeImageProcessorInvocation 
      * @description Applies Openpose processing to image
@@ -4963,6 +5263,12 @@ export type components = {
        */
       antialias?: boolean;
     };
+    /**
+     * SchedulerPredictionType 
+     * @description An enumeration. 
+     * @enum {string}
+     */
+    SchedulerPredictionType: "epsilon" | "v_prediction" | "sample";
     /**
      * SegmentAnythingProcessorInvocation 
      * @description Applies segment anything processing to image
@@ -5273,7 +5579,7 @@ export type components = {
      * @description An enumeration. 
      * @enum {string}
      */
-    SubModelType: "unet" | "text_encoder" | "text_encoder_2" | "tokenizer" | "tokenizer_2" | "vae" | "scheduler" | "safety_checker";
+    SubModelType: "unet" | "text_encoder" | "text_encoder_2" | "tokenizer" | "tokenizer_2" | "vae" | "vae_decoder" | "vae_encoder" | "scheduler" | "safety_checker";
     /**
      * SubtractInvocation 
      * @description Subtracts two numbers
@@ -5586,29 +5892,35 @@ export type components = {
       image?: components["schemas"]["ImageField"];
     };
     /**
-     * StableDiffusion2ModelFormat 
+     * ControlNetModelFormat 
      * @description An enumeration. 
      * @enum {string}
      */
-    StableDiffusion2ModelFormat: "checkpoint" | "diffusers";
+    ControlNetModelFormat: "checkpoint" | "diffusers";
     /**
      * StableDiffusionXLModelFormat 
      * @description An enumeration. 
      * @enum {string}
      */
     StableDiffusionXLModelFormat: "checkpoint" | "diffusers";
+    /**
+     * StableDiffusionOnnxModelFormat 
+     * @description An enumeration. 
+     * @enum {string}
+     */
+    StableDiffusionOnnxModelFormat: "olive" | "onnx";
+    /**
+     * StableDiffusion2ModelFormat 
+     * @description An enumeration. 
+     * @enum {string}
+     */
+    StableDiffusion2ModelFormat: "checkpoint" | "diffusers";
     /**
      * StableDiffusion1ModelFormat 
      * @description An enumeration. 
      * @enum {string}
      */
     StableDiffusion1ModelFormat: "checkpoint" | "diffusers";
-    /**
-     * ControlNetModelFormat 
-     * @description An enumeration. 
-     * @enum {string}
-     */
-    ControlNetModelFormat: "checkpoint" | "diffusers";
   };
   responses: never;
   parameters: never;
@@ -5719,7 +6031,7 @@ export type operations = {
     };
     requestBody: {
       content: {
-        "application/json": components["schemas"]["ControlNetInvocation"] | components["schemas"]["ImageProcessorInvocation"] | components["schemas"]["MainModelLoaderInvocation"] | components["schemas"]["LoraLoaderInvocation"] | components["schemas"]["VaeLoaderInvocation"] | components["schemas"]["MetadataAccumulatorInvocation"] | components["schemas"]["RangeInvocation"] | components["schemas"]["RangeOfSizeInvocation"] | components["schemas"]["RandomRangeInvocation"] | components["schemas"]["ImageCollectionInvocation"] | components["schemas"]["CompelInvocation"] | components["schemas"]["SDXLCompelPromptInvocation"] | components["schemas"]["SDXLRefinerCompelPromptInvocation"] | components["schemas"]["SDXLRawPromptInvocation"] | components["schemas"]["SDXLRefinerRawPromptInvocation"] | components["schemas"]["ClipSkipInvocation"] | components["schemas"]["LoadImageInvocation"] | components["schemas"]["ShowImageInvocation"] | components["schemas"]["ImageCropInvocation"] | components["schemas"]["ImagePasteInvocation"] | components["schemas"]["MaskFromAlphaInvocation"] | components["schemas"]["ImageMultiplyInvocation"] | components["schemas"]["ImageChannelInvocation"] | components["schemas"]["ImageConvertInvocation"] | components["schemas"]["ImageBlurInvocation"] | components["schemas"]["ImageResizeInvocation"] | components["schemas"]["ImageScaleInvocation"] | components["schemas"]["ImageLerpInvocation"] | components["schemas"]["ImageInverseLerpInvocation"] | components["schemas"]["ImageNSFWBlurInvocation"] | components["schemas"]["ImageWatermarkInvocation"] | components["schemas"]["CvInpaintInvocation"] | components["schemas"]["TextToLatentsInvocation"] | components["schemas"]["LatentsToImageInvocation"] | components["schemas"]["ResizeLatentsInvocation"] | components["schemas"]["ScaleLatentsInvocation"] | components["schemas"]["ImageToLatentsInvocation"] | components["schemas"]["InpaintInvocation"] | components["schemas"]["InfillColorInvocation"] | components["schemas"]["InfillTileInvocation"] | components["schemas"]["InfillPatchMatchInvocation"] | components["schemas"]["AddInvocation"] | components["schemas"]["SubtractInvocation"] | components["schemas"]["MultiplyInvocation"] | components["schemas"]["DivideInvocation"] | components["schemas"]["RandomIntInvocation"] | components["schemas"]["NoiseInvocation"] | components["schemas"]["DynamicPromptInvocation"] | components["schemas"]["PromptsFromFileInvocation"] | components["schemas"]["ParamIntInvocation"] | components["schemas"]["ParamFloatInvocation"] | components["schemas"]["ParamStringInvocation"] | components["schemas"]["ParamPromptInvocation"] | components["schemas"]["FloatLinearRangeInvocation"] | components["schemas"]["StepParamEasingInvocation"] | components["schemas"]["SDXLModelLoaderInvocation"] | components["schemas"]["SDXLRefinerModelLoaderInvocation"] | components["schemas"]["SDXLTextToLatentsInvocation"] | components["schemas"]["SDXLLatentsToLatentsInvocation"] | components["schemas"]["ESRGANInvocation"] | components["schemas"]["GraphInvocation"] | components["schemas"]["IterateInvocation"] | components["schemas"]["CollectInvocation"] | components["schemas"]["CannyImageProcessorInvocation"] | components["schemas"]["HedImageProcessorInvocation"] | components["schemas"]["LineartImageProcessorInvocation"] | components["schemas"]["LineartAnimeImageProcessorInvocation"] | components["schemas"]["OpenposeImageProcessorInvocation"] | components["schemas"]["MidasDepthImageProcessorInvocation"] | components["schemas"]["NormalbaeImageProcessorInvocation"] | components["schemas"]["MlsdImageProcessorInvocation"] | components["schemas"]["PidiImageProcessorInvocation"] | components["schemas"]["ContentShuffleImageProcessorInvocation"] | components["schemas"]["ZoeDepthImageProcessorInvocation"] | components["schemas"]["MediapipeFaceProcessorInvocation"] | components["schemas"]["LeresImageProcessorInvocation"] | components["schemas"]["TileResamplerProcessorInvocation"] | components["schemas"]["SegmentAnythingProcessorInvocation"] | components["schemas"]["LatentsToLatentsInvocation"];
+        "application/json": components["schemas"]["ControlNetInvocation"] | components["schemas"]["ImageProcessorInvocation"] | components["schemas"]["MainModelLoaderInvocation"] | components["schemas"]["LoraLoaderInvocation"] | components["schemas"]["VaeLoaderInvocation"] | components["schemas"]["MetadataAccumulatorInvocation"] | components["schemas"]["CompelInvocation"] | components["schemas"]["SDXLCompelPromptInvocation"] | components["schemas"]["SDXLRefinerCompelPromptInvocation"] | components["schemas"]["SDXLRawPromptInvocation"] | components["schemas"]["SDXLRefinerRawPromptInvocation"] | components["schemas"]["ClipSkipInvocation"] | components["schemas"]["LoadImageInvocation"] | components["schemas"]["ShowImageInvocation"] | components["schemas"]["ImageCropInvocation"] | components["schemas"]["ImagePasteInvocation"] | components["schemas"]["MaskFromAlphaInvocation"] | components["schemas"]["ImageMultiplyInvocation"] | components["schemas"]["ImageChannelInvocation"] | components["schemas"]["ImageConvertInvocation"] | components["schemas"]["ImageBlurInvocation"] | components["schemas"]["ImageResizeInvocation"] | components["schemas"]["ImageScaleInvocation"] | components["schemas"]["ImageLerpInvocation"] | components["schemas"]["ImageInverseLerpInvocation"] | components["schemas"]["ImageNSFWBlurInvocation"] | components["schemas"]["ImageWatermarkInvocation"] | components["schemas"]["TextToLatentsInvocation"] | components["schemas"]["LatentsToImageInvocation"] | components["schemas"]["ResizeLatentsInvocation"] | components["schemas"]["ScaleLatentsInvocation"] | components["schemas"]["ImageToLatentsInvocation"] | components["schemas"]["SDXLModelLoaderInvocation"] | components["schemas"]["SDXLRefinerModelLoaderInvocation"] | components["schemas"]["SDXLTextToLatentsInvocation"] | components["schemas"]["SDXLLatentsToLatentsInvocation"] | components["schemas"]["ONNXPromptInvocation"] | components["schemas"]["ONNXTextToLatentsInvocation"] | components["schemas"]["ONNXLatentsToImageInvocation"] | components["schemas"]["ONNXSD1ModelLoaderInvocation"] | components["schemas"]["OnnxModelLoaderInvocation"] | components["schemas"]["DynamicPromptInvocation"] | components["schemas"]["PromptsFromFileInvocation"] | components["schemas"]["AddInvocation"] | components["schemas"]["SubtractInvocation"] | components["schemas"]["MultiplyInvocation"] | components["schemas"]["DivideInvocation"] | components["schemas"]["RandomIntInvocation"] | components["schemas"]["ParamIntInvocation"] | components["schemas"]["ParamFloatInvocation"] | components["schemas"]["ParamStringInvocation"] | components["schemas"]["ParamPromptInvocation"] | components["schemas"]["CvInpaintInvocation"] | components["schemas"]["RangeInvocation"] | components["schemas"]["RangeOfSizeInvocation"] | components["schemas"]["RandomRangeInvocation"] | components["schemas"]["ImageCollectionInvocation"] | components["schemas"]["FloatLinearRangeInvocation"] | components["schemas"]["StepParamEasingInvocation"] | components["schemas"]["NoiseInvocation"] | components["schemas"]["ESRGANInvocation"] | components["schemas"]["InpaintInvocation"] | components["schemas"]["InfillColorInvocation"] | components["schemas"]["InfillTileInvocation"] | components["schemas"]["InfillPatchMatchInvocation"] | components["schemas"]["GraphInvocation"] | components["schemas"]["IterateInvocation"] | components["schemas"]["CollectInvocation"] | components["schemas"]["CannyImageProcessorInvocation"] | components["schemas"]["HedImageProcessorInvocation"] | components["schemas"]["LineartImageProcessorInvocation"] | components["schemas"]["LineartAnimeImageProcessorInvocation"] | components["schemas"]["OpenposeImageProcessorInvocation"] | components["schemas"]["MidasDepthImageProcessorInvocation"] | components["schemas"]["NormalbaeImageProcessorInvocation"] | components["schemas"]["MlsdImageProcessorInvocation"] | components["schemas"]["PidiImageProcessorInvocation"] | components["schemas"]["ContentShuffleImageProcessorInvocation"] | components["schemas"]["ZoeDepthImageProcessorInvocation"] | components["schemas"]["MediapipeFaceProcessorInvocation"] | components["schemas"]["LeresImageProcessorInvocation"] | components["schemas"]["TileResamplerProcessorInvocation"] | components["schemas"]["SegmentAnythingProcessorInvocation"] | components["schemas"]["LatentsToLatentsInvocation"];
       };
     };
     responses: {
@@ -5756,7 +6068,7 @@ export type operations = {
     };
     requestBody: {
       content: {
-        "application/json": components["schemas"]["ControlNetInvocation"] | components["schemas"]["ImageProcessorInvocation"] | components["schemas"]["MainModelLoaderInvocation"] | components["schemas"]["LoraLoaderInvocation"] | components["schemas"]["VaeLoaderInvocation"] | components["schemas"]["MetadataAccumulatorInvocation"] | components["schemas"]["RangeInvocation"] | components["schemas"]["RangeOfSizeInvocation"] | components["schemas"]["RandomRangeInvocation"] | components["schemas"]["ImageCollectionInvocation"] | components["schemas"]["CompelInvocation"] | components["schemas"]["SDXLCompelPromptInvocation"] | components["schemas"]["SDXLRefinerCompelPromptInvocation"] | components["schemas"]["SDXLRawPromptInvocation"] | components["schemas"]["SDXLRefinerRawPromptInvocation"] | components["schemas"]["ClipSkipInvocation"] | components["schemas"]["LoadImageInvocation"] | components["schemas"]["ShowImageInvocation"] | components["schemas"]["ImageCropInvocation"] | components["schemas"]["ImagePasteInvocation"] | components["schemas"]["MaskFromAlphaInvocation"] | components["schemas"]["ImageMultiplyInvocation"] | components["schemas"]["ImageChannelInvocation"] | components["schemas"]["ImageConvertInvocation"] | components["schemas"]["ImageBlurInvocation"] | components["schemas"]["ImageResizeInvocation"] | components["schemas"]["ImageScaleInvocation"] | components["schemas"]["ImageLerpInvocation"] | components["schemas"]["ImageInverseLerpInvocation"] | components["schemas"]["ImageNSFWBlurInvocation"] | components["schemas"]["ImageWatermarkInvocation"] | components["schemas"]["CvInpaintInvocation"] | components["schemas"]["TextToLatentsInvocation"] | components["schemas"]["LatentsToImageInvocation"] | components["schemas"]["ResizeLatentsInvocation"] | components["schemas"]["ScaleLatentsInvocation"] | components["schemas"]["ImageToLatentsInvocation"] | components["schemas"]["InpaintInvocation"] | components["schemas"]["InfillColorInvocation"] | components["schemas"]["InfillTileInvocation"] | components["schemas"]["InfillPatchMatchInvocation"] | components["schemas"]["AddInvocation"] | components["schemas"]["SubtractInvocation"] | components["schemas"]["MultiplyInvocation"] | components["schemas"]["DivideInvocation"] | components["schemas"]["RandomIntInvocation"] | components["schemas"]["NoiseInvocation"] | components["schemas"]["DynamicPromptInvocation"] | components["schemas"]["PromptsFromFileInvocation"] | components["schemas"]["ParamIntInvocation"] | components["schemas"]["ParamFloatInvocation"] | components["schemas"]["ParamStringInvocation"] | components["schemas"]["ParamPromptInvocation"] | components["schemas"]["FloatLinearRangeInvocation"] | components["schemas"]["StepParamEasingInvocation"] | components["schemas"]["SDXLModelLoaderInvocation"] | components["schemas"]["SDXLRefinerModelLoaderInvocation"] | components["schemas"]["SDXLTextToLatentsInvocation"] | components["schemas"]["SDXLLatentsToLatentsInvocation"] | components["schemas"]["ESRGANInvocation"] | components["schemas"]["GraphInvocation"] | components["schemas"]["IterateInvocation"] | components["schemas"]["CollectInvocation"] | components["schemas"]["CannyImageProcessorInvocation"] | components["schemas"]["HedImageProcessorInvocation"] | components["schemas"]["LineartImageProcessorInvocation"] | components["schemas"]["LineartAnimeImageProcessorInvocation"] | components["schemas"]["OpenposeImageProcessorInvocation"] | components["schemas"]["MidasDepthImageProcessorInvocation"] | components["schemas"]["NormalbaeImageProcessorInvocation"] | components["schemas"]["MlsdImageProcessorInvocation"] | components["schemas"]["PidiImageProcessorInvocation"] | components["schemas"]["ContentShuffleImageProcessorInvocation"] | components["schemas"]["ZoeDepthImageProcessorInvocation"] | components["schemas"]["MediapipeFaceProcessorInvocation"] | components["schemas"]["LeresImageProcessorInvocation"] | components["schemas"]["TileResamplerProcessorInvocation"] | components["schemas"]["SegmentAnythingProcessorInvocation"] | components["schemas"]["LatentsToLatentsInvocation"];
+        "application/json": components["schemas"]["ControlNetInvocation"] | components["schemas"]["ImageProcessorInvocation"] | components["schemas"]["MainModelLoaderInvocation"] | components["schemas"]["LoraLoaderInvocation"] | components["schemas"]["VaeLoaderInvocation"] | components["schemas"]["MetadataAccumulatorInvocation"] | components["schemas"]["CompelInvocation"] | components["schemas"]["SDXLCompelPromptInvocation"] | components["schemas"]["SDXLRefinerCompelPromptInvocation"] | components["schemas"]["SDXLRawPromptInvocation"] | components["schemas"]["SDXLRefinerRawPromptInvocation"] | components["schemas"]["ClipSkipInvocation"] | components["schemas"]["LoadImageInvocation"] | components["schemas"]["ShowImageInvocation"] | components["schemas"]["ImageCropInvocation"] | components["schemas"]["ImagePasteInvocation"] | components["schemas"]["MaskFromAlphaInvocation"] | components["schemas"]["ImageMultiplyInvocation"] | components["schemas"]["ImageChannelInvocation"] | components["schemas"]["ImageConvertInvocation"] | components["schemas"]["ImageBlurInvocation"] | components["schemas"]["ImageResizeInvocation"] | components["schemas"]["ImageScaleInvocation"] | components["schemas"]["ImageLerpInvocation"] | components["schemas"]["ImageInverseLerpInvocation"] | components["schemas"]["ImageNSFWBlurInvocation"] | components["schemas"]["ImageWatermarkInvocation"] | components["schemas"]["TextToLatentsInvocation"] | components["schemas"]["LatentsToImageInvocation"] | components["schemas"]["ResizeLatentsInvocation"] | components["schemas"]["ScaleLatentsInvocation"] | components["schemas"]["ImageToLatentsInvocation"] | components["schemas"]["SDXLModelLoaderInvocation"] | components["schemas"]["SDXLRefinerModelLoaderInvocation"] | components["schemas"]["SDXLTextToLatentsInvocation"] | components["schemas"]["SDXLLatentsToLatentsInvocation"] | components["schemas"]["ONNXPromptInvocation"] | components["schemas"]["ONNXTextToLatentsInvocation"] | components["schemas"]["ONNXLatentsToImageInvocation"] | components["schemas"]["ONNXSD1ModelLoaderInvocation"] | components["schemas"]["OnnxModelLoaderInvocation"] | components["schemas"]["DynamicPromptInvocation"] | components["schemas"]["PromptsFromFileInvocation"] | components["schemas"]["AddInvocation"] | components["schemas"]["SubtractInvocation"] | components["schemas"]["MultiplyInvocation"] | components["schemas"]["DivideInvocation"] | components["schemas"]["RandomIntInvocation"] | components["schemas"]["ParamIntInvocation"] | components["schemas"]["ParamFloatInvocation"] | components["schemas"]["ParamStringInvocation"] | components["schemas"]["ParamPromptInvocation"] | components["schemas"]["CvInpaintInvocation"] | components["schemas"]["RangeInvocation"] | components["schemas"]["RangeOfSizeInvocation"] | components["schemas"]["RandomRangeInvocation"] | components["schemas"]["ImageCollectionInvocation"] | components["schemas"]["FloatLinearRangeInvocation"] | components["schemas"]["StepParamEasingInvocation"] | components["schemas"]["NoiseInvocation"] | components["schemas"]["ESRGANInvocation"] | components["schemas"]["InpaintInvocation"] | components["schemas"]["InfillColorInvocation"] | components["schemas"]["InfillTileInvocation"] | components["schemas"]["InfillPatchMatchInvocation"] | components["schemas"]["GraphInvocation"] | components["schemas"]["IterateInvocation"] | components["schemas"]["CollectInvocation"] | components["schemas"]["CannyImageProcessorInvocation"] | components["schemas"]["HedImageProcessorInvocation"] | components["schemas"]["LineartImageProcessorInvocation"] | components["schemas"]["LineartAnimeImageProcessorInvocation"] | components["schemas"]["OpenposeImageProcessorInvocation"] | components["schemas"]["MidasDepthImageProcessorInvocation"] | components["schemas"]["NormalbaeImageProcessorInvocation"] | components["schemas"]["MlsdImageProcessorInvocation"] | components["schemas"]["PidiImageProcessorInvocation"] | components["schemas"]["ContentShuffleImageProcessorInvocation"] | components["schemas"]["ZoeDepthImageProcessorInvocation"] | components["schemas"]["MediapipeFaceProcessorInvocation"] | components["schemas"]["LeresImageProcessorInvocation"] | components["schemas"]["TileResamplerProcessorInvocation"] | components["schemas"]["SegmentAnythingProcessorInvocation"] | components["schemas"]["LatentsToLatentsInvocation"];
       };
     };
     responses: {
@@ -6020,14 +6332,14 @@ export type operations = {
     };
     requestBody: {
       content: {
-        "application/json": components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
+        "application/json": components["schemas"]["ONNXStableDiffusion1ModelConfig"] | components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["ONNXStableDiffusion2ModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
       };
     };
     responses: {
       /** @description The model was updated successfully */
       200: {
         content: {
-          "application/json": components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
+          "application/json": components["schemas"]["ONNXStableDiffusion1ModelConfig"] | components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["ONNXStableDiffusion2ModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
         };
       };
       /** @description Bad request */
@@ -6058,7 +6370,7 @@ export type operations = {
       /** @description The model imported successfully */
       201: {
         content: {
-          "application/json": components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
+          "application/json": components["schemas"]["ONNXStableDiffusion1ModelConfig"] | components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["ONNXStableDiffusion2ModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
         };
       };
       /** @description The model could not be found */
@@ -6084,14 +6396,14 @@ export type operations = {
   add_model: {
     requestBody: {
       content: {
-        "application/json": components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
+        "application/json": components["schemas"]["ONNXStableDiffusion1ModelConfig"] | components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["ONNXStableDiffusion2ModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
       };
     };
     responses: {
       /** @description The model added successfully */
       201: {
         content: {
-          "application/json": components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
+          "application/json": components["schemas"]["ONNXStableDiffusion1ModelConfig"] | components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["ONNXStableDiffusion2ModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
         };
       };
       /** @description The model could not be found */
@@ -6131,7 +6443,7 @@ export type operations = {
       /** @description Model converted successfully */
       200: {
         content: {
-          "application/json": components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
+          "application/json": components["schemas"]["ONNXStableDiffusion1ModelConfig"] | components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["ONNXStableDiffusion2ModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
         };
       };
       /** @description Bad request */
@@ -6220,7 +6532,7 @@ export type operations = {
       /** @description Model converted successfully */
       200: {
         content: {
-          "application/json": components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
+          "application/json": components["schemas"]["ONNXStableDiffusion1ModelConfig"] | components["schemas"]["StableDiffusion1ModelCheckpointConfig"] | components["schemas"]["StableDiffusion1ModelDiffusersConfig"] | components["schemas"]["VaeModelConfig"] | components["schemas"]["LoRAModelConfig"] | components["schemas"]["ControlNetModelCheckpointConfig"] | components["schemas"]["ControlNetModelDiffusersConfig"] | components["schemas"]["TextualInversionModelConfig"] | components["schemas"]["ONNXStableDiffusion2ModelConfig"] | components["schemas"]["StableDiffusion2ModelCheckpointConfig"] | components["schemas"]["StableDiffusion2ModelDiffusersConfig"] | components["schemas"]["StableDiffusionXLModelCheckpointConfig"] | components["schemas"]["StableDiffusionXLModelDiffusersConfig"];
         };
       };
       /** @description Incompatible models */

invokeai/frontend/web/src/services/api/types.d.ts

@@ -32,6 +32,7 @@ export type ModelType = components['schemas']['ModelType'];
 export type SubModelType = components['schemas']['SubModelType'];
 export type BaseModelType = components['schemas']['BaseModelType'];
 export type MainModelField = components['schemas']['MainModelField'];
+export type OnnxModelField = components['schemas']['OnnxModelField'];
 export type VAEModelField = components['schemas']['VAEModelField'];
 export type LoRAModelField = components['schemas']['LoRAModelField'];
 export type ControlNetModelField =
@@ -59,13 +60,16 @@ export type CheckpointModelConfig =
   | components['schemas']['StableDiffusion1ModelCheckpointConfig']
   | components['schemas']['StableDiffusion2ModelCheckpointConfig']
   | components['schemas']['StableDiffusionXLModelCheckpointConfig'];
+export type OnnxModelConfig =
+  components['schemas']['ONNXStableDiffusion1ModelConfig'];
 export type MainModelConfig = DiffusersModelConfig | CheckpointModelConfig;
 export type AnyModelConfig =
   | LoRAModelConfig
   | VaeModelConfig
   | ControlNetModelConfig
   | TextualInversionModelConfig
-  | MainModelConfig;
+  | MainModelConfig
+  | OnnxModelConfig;
 
 export type MergeModelConfig = components['schemas']['Body_merge_models'];
 export type ConvertModelConfig = components['schemas']['Body_convert_model'];
@@ -112,6 +116,9 @@ export type NoiseInvocation = TypeReq<components['schemas']['NoiseInvocation']>;
 export type TextToLatentsInvocation = TypeReq<
   components['schemas']['TextToLatentsInvocation']
 >;
+export type ONNXTextToLatentsInvocation = TypeReq<
+  components['schemas']['ONNXTextToLatentsInvocation']
+>;
 export type LatentsToLatentsInvocation = TypeReq<
   components['schemas']['LatentsToLatentsInvocation']
 >;
@@ -127,6 +134,9 @@ export type ImageCollectionInvocation = TypeReq<
 export type MainModelLoaderInvocation = TypeReq<
   components['schemas']['MainModelLoaderInvocation']
 >;
+export type OnnxModelLoaderInvocation = TypeReq<
+  components['schemas']['OnnxModelLoaderInvocation']
+>;
 export type LoraLoaderInvocation = TypeReq<
   components['schemas']['LoraLoaderInvocation']
 >;

invokeai/frontend/web/yarn.lock

@@ -6447,11 +6447,16 @@ tslib@^1.8.1:
   resolved "https://registry.yarnpkg.com/tslib/-/tslib-1.14.1.tgz#cf2d38bdc34a134bcaf1091c41f6619e2f672d00"
   integrity sha512-Xni35NKzjgMrwevysHTCArtLDpPvye8zV/0E4EyYn43P7/7qvQwPh9BGkHewbMulVntbigmcT7rdX3BNo9wRJg==
 
-tslib@^2.0.0, tslib@^2.0.3, tslib@^2.1.0, tslib@^2.3.0, tslib@^2.4.0:
+tslib@^2.0.0, tslib@^2.1.0, tslib@^2.3.0, tslib@^2.4.0:
   version "2.6.0"
   resolved "https://registry.yarnpkg.com/tslib/-/tslib-2.6.0.tgz#b295854684dbda164e181d259a22cd779dcd7bc3"
   integrity sha512-7At1WUettjcSRHXCyYtTselblcHl9PJFFVKiCAy/bY97+BPZXSQ2wbq0P9s8tK2G7dFQfNnlJnPAiArVBVBsfA==
 
+tslib@^2.0.3:
+  version "2.6.1"
+  resolved "https://registry.yarnpkg.com/tslib/-/tslib-2.6.1.tgz#fd8c9a0ff42590b25703c0acb3de3d3f4ede0410"
+  integrity sha512-t0hLfiEKfMUoqhG+U1oid7Pva4bbDPHYfJNiB7BiIjRkj1pyC++4N3huJfqY6aRH6VTB0rvtzQwjM4K6qpfOig==
+
 tsutils@^3.21.0:
   version "3.21.0"
   resolved "https://registry.yarnpkg.com/tsutils/-/tsutils-3.21.0.tgz#b48717d394cea6c1e096983eed58e9d61715b623"

mkdocs.yml

@@ -36,7 +36,6 @@ theme:
     - navigation.instant
     - navigation.tabs
     - navigation.tabs.sticky
-    - navigation.top
     - navigation.tracking
     - navigation.indexes
     - navigation.path
@@ -102,9 +101,9 @@ plugins:
 nav:
     - Home: 'index.md'
     - Installation:
-      - Overview: 'installation/index.md'
+      - Overview: 'installation/INSTALLATION.md'
       - Installing with the Automated Installer: 'installation/010_INSTALL_AUTOMATED.md'
-      - Installing manually: 'installation/020_INSTALL_MANUAL.md'
+      - Installing Manually: 'installation/020_INSTALL_MANUAL.md'
       - NVIDIA Cuda / AMD ROCm: 'installation/030_INSTALL_CUDA_AND_ROCM.md'
       - Installing with Docker: 'installation/040_INSTALL_DOCKER.md'
       - Installing Models: 'installation/050_INSTALLING_MODELS.md'
@@ -124,6 +123,7 @@ nav:
       - Overview: 'nodes/overview.md'
     - Features:
       - Overview: 'features/index.md'
+      - New to InvokeAI?: 'help/gettingStartedWithAI.md'
       - Concepts: 'features/CONCEPTS.md'
       - Configuration: 'features/CONFIGURATION.md'
       - ControlNet: 'features/CONTROLNET.md'
@@ -157,6 +157,7 @@ nav:
       - Inpainting: 'deprecated/INPAINTING.md'
       - Outpainting: 'deprecated/OUTPAINTING.md'
     - Help:
+      - Getting Started: 'help/gettingStartedWithAI.md'
       - Sampler Convergence: 'help/SAMPLER_CONVERGENCE.md'
     - Other:
       - Contributors: 'other/CONTRIBUTORS.md'

pyproject.toml

@@ -61,6 +61,8 @@ dependencies = [
   "numpy",
   "npyscreen",
   "omegaconf",
+  "onnx",
+  "onnxruntime",
   "opencv-python",
   "pydantic==1.*",
   "picklescan",
@@ -103,6 +105,15 @@ dependencies = [
 	   "xformers~=0.0.19; sys_platform!='darwin'",
 	   "triton; sys_platform=='linux'",
 ]
+"onnx" = [
+  "onnxruntime",
+]
+"onnx-cuda" = [
+  "onnxruntime-gpu",
+]
+"onnx-directml" = [
+  "onnxruntime-directml",
+]
 
 [project.scripts]
 
@@ -180,4 +191,4 @@ output = "coverage/index.xml"
 max-line-length = 120
 
 [tool.black]
-line-length = 120
+line-length = 120

tests/nodes/test_graph_execution_state.py

@@ -48,6 +48,7 @@ def mock_services() -> InvocationServices:
         images=None,  # type: ignore
         latents=None,  # type: ignore
         boards=None,  # type: ignore
+        batch_manager=None,  # type: ignore
         board_images=None,  # type: ignore
         queue=MemoryInvocationQueue(),
         graph_library=SqliteItemStorage[LibraryGraph](filename=sqlite_memory, table_name="graphs"),

tests/nodes/test_invoker.py

@@ -40,6 +40,7 @@ def mock_services() -> InvocationServices:
         logger=None,  # type: ignore
         images=None,  # type: ignore
         latents=None,  # type: ignore
+        batch_manager=None,  # type: ignore
         boards=None,  # type: ignore
         board_images=None,  # type: ignore
         queue=MemoryInvocationQueue(),