chore: bump version to v5.7.2rc1

Fixes an issue where fields like control weight on ControlNet nodes and image on IP Adapter nodes didn't render.

These are "single or collection" fields. They accept a single input object, or collection. They are supposed to render the UI input for a single object.

In a7a71ca935 a performance optimisation for a hot code-path inadvertently broke this.

The determination of which UI component to render for a given field was done using a type guard function for the field's template. Previously, this used a zod schema to parse the template. This is very slow, especially when the template was not the expected type.

The optimization changed the type guards to check the field name (aka its type, integer, image, etc) and cardinality directly, without any zod parsing.

It's much faster, but subtly changed the behaviour because it was a bit stricter. For some fields, it rejected "single or collection" cardinalities when it should have accepted them.

When these fields - like the aforementioned Control Weight and Image - were being rendered, none of the type guards passed and they rendered nothing.

The fix here updates the type guard functions to support multiple cardinalities. So now, when we go to render a "single or collection" field, we will render the "single" input component as it should be.

.github/workflows/build-container.yml

@@ -76,9 +76,6 @@ jobs:
             latest=${{ matrix.gpu-driver == 'cuda' && github.ref == 'refs/heads/main' }}
             suffix=-${{ matrix.gpu-driver }},onlatest=false
 
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v3
-
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v3
         with:
@@ -103,7 +100,7 @@ jobs:
           push: ${{ github.ref == 'refs/heads/main' || github.ref_type == 'tag' || github.event.inputs.push-to-registry }}
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
-          cache-from: |
-            type=gha,scope=${{ github.ref_name }}-${{ matrix.gpu-driver }}
-            type=gha,scope=main-${{ matrix.gpu-driver }}
-          cache-to: type=gha,mode=max,scope=${{ github.ref_name }}-${{ matrix.gpu-driver }}
+          # cache-from: |
+          #   type=gha,scope=${{ github.ref_name }}-${{ matrix.gpu-driver }}
+          #   type=gha,scope=main-${{ matrix.gpu-driver }}
+          # cache-to: type=gha,mode=max,scope=${{ github.ref_name }}-${{ matrix.gpu-driver }}

docs/RELEASE.md

@@ -1,41 +1,50 @@
 # Release Process
 
-The app is published in twice, in different build formats.
+The Invoke application is published as a python package on [PyPI]. This includes both a source distribution and built distribution (a wheel).
 
-- A [PyPI] distribution. This includes both a source distribution and built distribution (a wheel). Users install with `pip install invokeai`. The updater uses this build.
-- An installer on the [InvokeAI Releases Page]. This is a zip file with install scripts and a wheel. This is only used for new installs.
+Most users install it with the [Launcher](https://github.com/invoke-ai/launcher/), others with `pip`.
+
+The launcher uses GitHub as the source of truth for available releases.
+
+## Broad Strokes
+
+- Merge all changes and bump the version in the codebase.
+- Tag the release commit.
+- Wait for the release workflow to complete.
+- Approve the PyPI publish jobs.
+- Write GH release notes.
 
 ## General Prep
 
-Make a developer call-out for PRs to merge. Merge and test things out.
-
-While the release workflow does not include end-to-end tests, it does pause before publishing so you can download and test the final build.
+Make a developer call-out for PRs to merge. Merge and test things out. Bump the version by editing `invokeai/version/invokeai_version.py`.
 
 ## Release Workflow
 
 The `release.yml` workflow runs a number of jobs to handle code checks, tests, build and publish on PyPI.
 
-It is triggered on **tag push**, when the tag matches `v*`. It doesn't matter if you've prepped a release branch like `release/v3.5.0` or are releasing from `main` - it works the same.
-
-> Because commits are reference-counted, it is safe to create a release branch, tag it, let the workflow run, then delete the branch. So long as the tag exists, that commit will exist.
+It is triggered on **tag push**, when the tag matches `v*`.
 
 ### Triggering the Workflow
 
-Run `make tag-release` to tag the current commit and kick off the workflow.
+Ensure all commits that should be in the release are merged, and you have pulled them locally.
 
-The release may also be dispatched [manually].
+Double-check that you have checked out the commit that will represent the release (typically the latest commit on `main`).
+
+Run `make tag-release` to tag the current commit and kick off the workflow. You will be prompted to provide a message - use the version specifier.
+
+If this version's tag already exists for some reason (maybe you had to make a last minute change), the script will overwrite it.
+
+> In case you cannot use the Make target, the release may also be dispatched [manually] via GH.
 
 ### Workflow Jobs and Process
 
-The workflow consists of a number of concurrently-run jobs, and two final publish jobs.
+The workflow consists of a number of concurrently-run checks and tests, then two final publish jobs.
 
 The publish jobs require manual approval and are only run if the other jobs succeed.
 
 #### `check-version` Job
 
-This job checks that the git ref matches the app version. It matches the ref against the `__version__` variable in `invokeai/version/invokeai_version.py`.
-
-When the workflow is triggered by tag push, the ref is the tag. If the workflow is run manually, the ref is the target selected from the **Use workflow from** dropdown.
+This job ensures that the `invokeai` python package version specifier matches the tag for the release. The version specifier is pulled from the `__version__` variable in `invokeai/version/invokeai_version.py`.
 
 This job uses [samuelcolvin/check-python-version].
 
@@ -43,62 +52,52 @@ This job uses [samuelcolvin/check-python-version].
 
 #### Check and Test Jobs
 
+Next, these jobs run and must pass. They are the same jobs that are run for every PR.
+
 - **`python-tests`**: runs `pytest` on matrix of platforms
 - **`python-checks`**: runs `ruff` (format and lint)
 - **`frontend-tests`**: runs `vitest`
 - **`frontend-checks`**: runs `prettier` (format), `eslint` (lint), `dpdm` (circular refs), `tsc` (static type check) and `knip` (unused imports)
-
-> **TODO** We should add `mypy` or `pyright` to the **`check-python`** job.
-
-> **TODO** We should add an end-to-end test job that generates an image.
+- **`typegen-checks`**: ensures the frontend and backend types are synced
 
 #### `build-installer` Job
 
 This sets up both python and frontend dependencies and builds the python package. Internally, this runs `installer/create_installer.sh` and uploads two artifacts:
 
 - **`dist`**: the python distribution, to be published on PyPI
-- **`InvokeAI-installer-${VERSION}.zip`**: the installer to be included in the GitHub release
+- **`InvokeAI-installer-${VERSION}.zip`**: the legacy install scripts
+
+You don't need to download either of these files.
+
+> The legacy install scripts are no longer used, but we haven't updated the workflow to skip building them.
 
 #### Sanity Check & Smoke Test
 
-At this point, the release workflow pauses as the remaining publish jobs require approval. Time to test the installer.
+At this point, the release workflow pauses as the remaining publish jobs require approval.
 
-Because the installer pulls from PyPI, and we haven't published to PyPI yet, you will need to install from the wheel:
+It's possible to test the python package before it gets published to PyPI. We've never had problems with it, so it's not necessary to do this.
 
-- Download and unzip `dist.zip` and the installer from the **Summary** tab of the workflow
-- Run the installer script using the `--wheel` CLI arg, pointing at the wheel:
+But, if you want to be extra-super careful, here's how to test it:
 
-  ```sh
-  ./install.sh --wheel ../InvokeAI-4.0.0rc6-py3-none-any.whl
-  ```
-
-- Install to a temporary directory so you get the new user experience
-- Download a model and generate
-
-> The same wheel file is bundled in the installer and in the `dist` artifact, which is uploaded to PyPI. You should end up with the exactly the same installation as if the installer got the wheel from PyPI.
+- Download the `dist.zip` build artifact from the `build-installer` job
+- Unzip it and find the wheel file
+- Create a fresh Invoke install by following the [manual install guide](https://invoke-ai.github.io/InvokeAI/installation/manual/) - but instead of installing from PyPI, install from the wheel
+- Test the app
 
 ##### Something isn't right
 
-If testing reveals any issues, no worries. Cancel the workflow, which will cancel the pending publish jobs (you didn't approve them prematurely, right?).
-
-Now you can start from the top:
-
-- Fix the issues and PR the fixes per usual
-- Get the PR approved and merged per usual
-- Switch to `main` and pull in the fixes
-- Run `make tag-release` to move the tag to `HEAD` (which has the fixes) and kick off the release workflow again
-- Re-do the sanity check
+If testing reveals any issues, no worries. Cancel the workflow, which will cancel the pending publish jobs (you didn't approve them prematurely, right?) and start over.
 
 #### PyPI Publish Jobs
 
-The publish jobs will run if any of the previous jobs fail.
+The publish jobs will not run if any of the previous jobs fail.
 
 They use [GitHub environments], which are configured as [trusted publishers] on PyPI.
 
-Both jobs require a maintainer to approve them from the workflow's **Summary** tab.
+Both jobs require a @hipsterusername or @psychedelicious to approve them from the workflow's **Summary** tab.
 
 - Click the **Review deployments** button
-- Select the environment (either `testpypi` or `pypi`)
+- Select the environment (either `testpypi` or `pypi` - typically you select both)
 - Click **Approve and deploy**
 
 > **If the version already exists on PyPI, the publish jobs will fail.** PyPI only allows a given version to be published once - you cannot change it. If version published on PyPI has a problem, you'll need to "fail forward" by bumping the app version and publishing a followup release.
@@ -113,46 +112,33 @@ If there are no incidents, contact @hipsterusername or @lstein, who have owner a
 
 Publishes the distribution on the [Test PyPI] index, using the `testpypi` GitHub environment.
 
-This job is not required for the production PyPI publish, but included just in case you want to test the PyPI release.
+This job is not required for the production PyPI publish, but included just in case you want to test the PyPI release for some reason:
 
-If approved and successful, you could try out the test release like this:
-
-```sh
-# Create a new virtual environment
-python -m venv ~/.test-invokeai-dist --prompt test-invokeai-dist
-# Install the distribution from Test PyPI
-pip install --index-url https://test.pypi.org/simple/ invokeai
-# Run and test the app
-invokeai-web
-# Cleanup
-deactivate
-rm -rf ~/.test-invokeai-dist
-```
+- Approve this publish job without approving the prod publish
+- Let it finish
+- Create a fresh Invoke install by following the [manual install guide](https://invoke-ai.github.io/InvokeAI/installation/manual/), making sure to use the Test PyPI index URL: `https://test.pypi.org/simple/`
+- Test the app
 
 #### `publish-pypi` Job
 
 Publishes the distribution on the production PyPI index, using the `pypi` GitHub environment.
 
-## Publish the GitHub Release with installer
+It's a good idea to wait to approve and run this job until you have the release notes ready!
 
-Once the release is published to PyPI, it's time to publish the GitHub release.
+## Prep and publish the GitHub Release
 
 1. [Draft a new release] on GitHub, choosing the tag that triggered the release.
-1. Write the release notes, describing important changes. The **Generate release notes** button automatically inserts the changelog and new contributors, and you can copy/paste the intro from previous releases.
-1. Use `scripts/get_external_contributions.py` to get a list of external contributions to shout out in the release notes.
-1. Upload the zip file created in **`build`** job into the Assets section of the release notes.
-1. Check **Set as a pre-release** if it's a pre-release.
-1. Check **Create a discussion for this release**.
-1. Publish the release.
-1. Announce the release in Discord.
-
-> **TODO** Workflows can create a GitHub release from a template and upload release assets. One popular action to handle this is [ncipollo/release-action]. A future enhancement to the release process could set this up.
-
-## Manual Build
-
-The `build installer` workflow can be dispatched manually. This is useful to test the installer for a given branch or tag.
-
-No checks are run, it just builds.
+2. The **Generate release notes** button automatically inserts the changelog and new contributors. Make sure to select the correct tags for this release and the last stable release. GH often selects the wrong tags - do this manually.
+3. Write the release notes, describing important changes. Contributions from community members should be shouted out. Use the GH-generated changelog to see all contributors. If there are Weblate translation updates, open that PR and shout out every person who contributed a translation.
+4. Check **Set as a pre-release** if it's a pre-release.
+5. Approve and wait for the `publish-pypi` job to finish if you haven't already.
+6. Publish the GH release.
+7. Post the release in Discord in the [releases](https://discord.com/channels/1020123559063990373/1149260708098359327) channel with abbreviated notes. For example:
+   > Invoke v5.7.0 (stable): <https://github.com/invoke-ai/InvokeAI/releases/tag/v5.7.0>
+   >
+   > It's a pretty big one - Form Builder, Metadata Nodes (thanks @SkunkWorxDark!), and much more.
+8. Right click the message in releases and copy the link to it. Then, post that link in the [new-release-discussion](https://discord.com/channels/1020123559063990373/1149506274971631688) channel. For example:
+   > Invoke v5.7.0 (stable): <https://discord.com/channels/1020123559063990373/1149260708098359327/1344521744916021248>
 
 ## Manual Release
 
@@ -160,12 +146,10 @@ The `release` workflow can be dispatched manually. You must dispatch the workflo
 
 This functionality is available as a fallback in case something goes wonky. Typically, releases should be triggered via tag push as described above.
 
-[InvokeAI Releases Page]: https://github.com/invoke-ai/InvokeAI/releases
 [PyPI]: https://pypi.org/
 [Draft a new release]: https://github.com/invoke-ai/InvokeAI/releases/new
 [Test PyPI]: https://test.pypi.org/
 [version specifier]: https://packaging.python.org/en/latest/specifications/version-specifiers/
-[ncipollo/release-action]: https://github.com/ncipollo/release-action
 [GitHub environments]: https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment
 [trusted publishers]: https://docs.pypi.org/trusted-publishers/
 [samuelcolvin/check-python-version]: https://github.com/samuelcolvin/check-python-version

docs/features/low-vram.md

@@ -31,6 +31,7 @@ It is possible to fine-tune the settings for best performance or if you still ge
 Low-VRAM mode involves 4 features, each of which can be configured or fine-tuned:
 
 - Partial model loading (`enable_partial_loading`)
+- PyTorch CUDA allocator config (`pytorch_cuda_alloc_conf`)
 - Dynamic RAM and VRAM cache sizes (`max_cache_ram_gb`, `max_cache_vram_gb`)
 - Working memory (`device_working_mem_gb`)
 - Keeping a RAM weight copy (`keep_ram_copy_of_weights`)
@@ -51,6 +52,16 @@ As described above, you can enable partial model loading by adding this line to
 enable_partial_loading: true
 ```
 
+### PyTorch CUDA allocator config
+
+The PyTorch CUDA allocator's behavior can be configured using the `pytorch_cuda_alloc_conf` config. Tuning the allocator configuration can help to reduce the peak reserved VRAM. The optimal configuration is dependent on many factors (e.g. device type, VRAM, CUDA driver version, etc.), but switching from PyTorch's native allocator to using CUDA's built-in allocator works well on many systems. To try this, add the following line to your `invokeai.yaml` file:
+
+```yaml
+pytorch_cuda_alloc_conf: "backend:cudaMallocAsync"
+```
+
+A more complete explanation of the available configuration options is [here](https://pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf).
+
 ### Dynamic RAM and VRAM cache sizes
 
 Loading models from disk is slow and can be a major bottleneck for performance. Invoke uses two model caches - RAM and VRAM - to reduce loading from disk to a minimum.
@@ -75,24 +86,26 @@ But, if your GPU has enough VRAM to hold models fully, you might get a perf boos
 # As an example, if your system has 32GB of RAM and no other heavy processes, setting the `max_cache_ram_gb` to 28GB
 # might be a good value to achieve aggressive model caching.
 max_cache_ram_gb: 28
+
 # The default max cache VRAM size is adjusted dynamically based on the amount of available VRAM (taking into
 # consideration the VRAM used by other processes).
-# You can override the default value by setting `max_cache_vram_gb`. Note that this value takes precedence over the
-# `device_working_mem_gb`.
-# It is recommended to set the VRAM cache size to be as large as possible while leaving enough room for the working
-# memory of the tasks you will be doing. For example, on a 24GB GPU that will be running unquantized FLUX without any
-# auxiliary models, 18GB might be a good value.
-max_cache_vram_gb: 18
+# You can override the default value by setting `max_cache_vram_gb`.
+# CAUTION: Most users should not manually set this value. See warning below.
+max_cache_vram_gb: 16
 ```
 
-!!! tip "Max safe value for `max_cache_vram_gb`"
+!!! warning "Max safe value for `max_cache_vram_gb`"
 
-    To determine the max safe value for `max_cache_vram_gb`, subtract `device_working_mem_gb` from your GPU's VRAM. As described below, the default for `device_working_mem_gb` is 3GB.
+    Most users should not manually configure the `max_cache_vram_gb`. This configuration value takes precedence over the `device_working_mem_gb` and any operations that explicitly reserve additional working memory (e.g. VAE decode). As such, manually configuring it increases the likelihood of encountering out-of-memory errors.
+
+    For users who wish to configure `max_cache_vram_gb`, the max safe value can be determined by subtracting `device_working_mem_gb` from your GPU's VRAM. As described below, the default for `device_working_mem_gb` is 3GB.
 
     For example, if you have a 12GB GPU, the max safe value for `max_cache_vram_gb` is `12GB - 3GB = 9GB`.
 
     If you had increased `device_working_mem_gb` to 4GB, then the max safe value for `max_cache_vram_gb` is `12GB - 4GB = 8GB`.
 
+    Most users who override `max_cache_vram_gb` are doing so because they wish to use significantly less VRAM, and should be setting `max_cache_vram_gb` to a value significantly less than the 'max safe value'.
+
 ### Working memory
 
 Invoke cannot use _all_ of your VRAM for model caching and loading. It requires some VRAM to use as working memory for various operations.

invokeai/app/api_app.py

@@ -1,12 +1,8 @@
 import asyncio
 import logging
-import mimetypes
-import socket
 from contextlib import asynccontextmanager
 from pathlib import Path
 
-import torch
-import uvicorn
 from fastapi import FastAPI, Request
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.middleware.gzip import GZipMiddleware
@@ -15,11 +11,7 @@ from fastapi.responses import HTMLResponse, RedirectResponse
 from fastapi_events.handlers.local import local_handler
 from fastapi_events.middleware import EventHandlerASGIMiddleware
 from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint
-from torch.backends.mps import is_available as is_mps_available
 
-# for PyCharm:
-# noinspection PyUnresolvedReferences
-import invokeai.backend.util.hotfixes  # noqa: F401 (monkeypatching on import)
 import invokeai.frontend.web as web_dir
 from invokeai.app.api.dependencies import ApiDependencies
 from invokeai.app.api.no_cache_staticfiles import NoCacheStaticFiles
@@ -36,39 +28,15 @@ from invokeai.app.api.routers import (
     workflows,
 )
 from invokeai.app.api.sockets import SocketIO
-from invokeai.app.invocations.load_custom_nodes import load_custom_nodes
 from invokeai.app.services.config.config_default import get_config
 from invokeai.app.util.custom_openapi import get_openapi_func
-from invokeai.backend.util.devices import TorchDevice
 from invokeai.backend.util.logging import InvokeAILogger
 
 app_config = get_config()
-
-
-if is_mps_available():
-    import invokeai.backend.util.mps_fixes  # noqa: F401 (monkeypatching on import)
-
-
 logger = InvokeAILogger.get_logger(config=app_config)
-# fix for windows mimetypes registry entries being borked
-# see https://github.com/invoke-ai/InvokeAI/discussions/3684#discussioncomment-6391352
-mimetypes.add_type("application/javascript", ".js")
-mimetypes.add_type("text/css", ".css")
-
-torch_device_name = TorchDevice.get_torch_device_name()
-logger.info(f"Using torch device: {torch_device_name}")
 
 loop = asyncio.new_event_loop()
 
-# We may change the port if the default is in use, this global variable is used to store the port so that we can log
-# the correct port when the server starts in the lifespan handler.
-port = app_config.port
-
-# Load custom nodes. This must be done after importing the Graph class, which itself imports all modules from the
-# invocations module. The ordering here is implicit, but important - we want to load custom nodes after all the
-# core nodes have been imported so that we can catch when a custom node clobbers a core node.
-load_custom_nodes(custom_nodes_path=app_config.custom_nodes_path)
-
 
 @asynccontextmanager
 async def lifespan(app: FastAPI):
@@ -77,7 +45,7 @@ async def lifespan(app: FastAPI):
 
     # Log the server address when it starts - in case the network log level is not high enough to see the startup log
     proto = "https" if app_config.ssl_certfile else "http"
-    msg = f"Invoke running on {proto}://{app_config.host}:{port} (Press CTRL+C to quit)"
+    msg = f"Invoke running on {proto}://{app_config.host}:{app_config.port} (Press CTRL+C to quit)"
 
     # Logging this way ignores the logger's log level and _always_ logs the message
     record = logger.makeRecord(
@@ -192,73 +160,3 @@ except RuntimeError:
 app.mount(
     "/static", NoCacheStaticFiles(directory=Path(web_root_path, "static/")), name="static"
 )  # docs favicon is in here
-
-
-def check_cudnn(logger: logging.Logger) -> None:
-    """Check for cuDNN issues that could be causing degraded performance."""
-    if torch.backends.cudnn.is_available():
-        try:
-            # Note: At the time of writing (torch 2.2.1), torch.backends.cudnn.version() only raises an error the first
-            # time it is called. Subsequent calls will return the version number without complaining about a mismatch.
-            cudnn_version = torch.backends.cudnn.version()
-            logger.info(f"cuDNN version: {cudnn_version}")
-        except RuntimeError as e:
-            logger.warning(
-                "Encountered a cuDNN version issue. This may result in degraded performance. This issue is usually "
-                "caused by an incompatible cuDNN version installed in your python environment, or on the host "
-                f"system. Full error message:\n{e}"
-            )
-
-
-def invoke_api() -> None:
-    def find_port(port: int) -> int:
-        """Find a port not in use starting at given port"""
-        # Taken from https://waylonwalker.com/python-find-available-port/, thanks Waylon!
-        # https://github.com/WaylonWalker
-        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
-            s.settimeout(1)
-            if s.connect_ex(("localhost", port)) == 0:
-                return find_port(port=port + 1)
-            else:
-                return port
-
-    if app_config.dev_reload:
-        try:
-            import jurigged
-        except ImportError as e:
-            logger.error(
-                'Can\'t start `--dev_reload` because jurigged is not found; `pip install -e ".[dev]"` to include development dependencies.',
-                exc_info=e,
-            )
-        else:
-            jurigged.watch(logger=InvokeAILogger.get_logger(name="jurigged").info)
-
-    global port
-    port = find_port(app_config.port)
-    if port != app_config.port:
-        logger.warn(f"Port {app_config.port} in use, using port {port}")
-
-    check_cudnn(logger)
-
-    config = uvicorn.Config(
-        app=app,
-        host=app_config.host,
-        port=port,
-        loop="asyncio",
-        log_level=app_config.log_level_network,
-        ssl_certfile=app_config.ssl_certfile,
-        ssl_keyfile=app_config.ssl_keyfile,
-    )
-    server = uvicorn.Server(config)
-
-    # replace uvicorn's loggers with InvokeAI's for consistent appearance
-    uvicorn_logger = InvokeAILogger.get_logger("uvicorn")
-    uvicorn_logger.handlers.clear()
-    for hdlr in logger.handlers:
-        uvicorn_logger.addHandler(hdlr)
-
-    loop.run_until_complete(server.serve())
-
-
-if __name__ == "__main__":
-    invoke_api()

invokeai/app/invocations/flux_vae_decode.py

@@ -41,16 +41,11 @@ class FluxVaeDecodeInvocation(BaseInvocation, WithMetadata, WithBoard):
 
     def _estimate_working_memory(self, latents: torch.Tensor, vae: AutoEncoder) -> int:
         """Estimate the working memory required by the invocation in bytes."""
-        # It was found experimentally that the peak working memory scales linearly with the number of pixels and the
-        # element size (precision).
         out_h = LATENT_SCALE_FACTOR * latents.shape[-2]
         out_w = LATENT_SCALE_FACTOR * latents.shape[-1]
         element_size = next(vae.parameters()).element_size()
-        scaling_constant = 1090  # Determined experimentally.
+        scaling_constant = 2200  # Determined experimentally.
         working_memory = out_h * out_w * element_size * scaling_constant
-
-        # We add a 20% buffer to the working memory estimate to be safe.
-        working_memory = working_memory * 1.2
         return int(working_memory)
 
     def _vae_decode(self, vae_info: LoadedModel, latents: torch.Tensor) -> Image.Image:

invokeai/app/invocations/latents_to_image.py

@@ -60,7 +60,7 @@ class LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
         # It was found experimentally that the peak working memory scales linearly with the number of pixels and the
         # element size (precision). This estimate is accurate for both SD1 and SDXL.
         element_size = 4 if self.fp32 else 2
-        scaling_constant = 960  # Determined experimentally.
+        scaling_constant = 2200  # Determined experimentally.
 
         if use_tiling:
             tile_size = self.tile_size
@@ -84,9 +84,7 @@ class LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
             # If we are running in FP32, then we should account for the likely increase in model size (~250MB).
             working_memory += 250 * 2**20
 
-        # We add 20% to the working memory estimate to be safe.
-        working_memory = int(working_memory * 1.2)
-        return working_memory
+        return int(working_memory)
 
     @torch.no_grad()
     def invoke(self, context: InvocationContext) -> ImageOutput:

invokeai/app/invocations/sd3_latents_to_image.py

@@ -43,16 +43,11 @@ class SD3LatentsToImageInvocation(BaseInvocation, WithMetadata, WithBoard):
 
     def _estimate_working_memory(self, latents: torch.Tensor, vae: AutoencoderKL) -> int:
         """Estimate the working memory required by the invocation in bytes."""
-        # It was found experimentally that the peak working memory scales linearly with the number of pixels and the
-        # element size (precision).
         out_h = LATENT_SCALE_FACTOR * latents.shape[-2]
         out_w = LATENT_SCALE_FACTOR * latents.shape[-1]
         element_size = next(vae.parameters()).element_size()
-        scaling_constant = 1230  # Determined experimentally.
+        scaling_constant = 2200  # Determined experimentally.
         working_memory = out_h * out_w * element_size * scaling_constant
-
-        # We add a 20% buffer to the working memory estimate to be safe.
-        working_memory = working_memory * 1.2
         return int(working_memory)
 
     @torch.no_grad()

invokeai/app/run_app.py

@@ -1,12 +1,82 @@
-"""This is a wrapper around the main app entrypoint, to allow for CLI args to be parsed before running the app."""
+import uvicorn
+
+from invokeai.app.invocations.load_custom_nodes import load_custom_nodes
+from invokeai.app.services.config.config_default import get_config
+from invokeai.app.util.torch_cuda_allocator import configure_torch_cuda_allocator
+from invokeai.backend.util.logging import InvokeAILogger
+from invokeai.frontend.cli.arg_parser import InvokeAIArgs
+
+
+def get_app():
+    """Import the app and event loop. We wrap this in a function to more explicitly control when it happens, because
+    importing from api_app does a bunch of stuff - it's more like calling a function than importing a module.
+    """
+    from invokeai.app.api_app import app, loop
+
+    return app, loop
 
 
 def run_app() -> None:
-    # Before doing _anything_, parse CLI args!
-    from invokeai.frontend.cli.arg_parser import InvokeAIArgs
-
+    """The main entrypoint for the app."""
+    # Parse the CLI arguments.
     InvokeAIArgs.parse_args()
 
-    from invokeai.app.api_app import invoke_api
+    # Load config.
+    app_config = get_config()
 
-    invoke_api()
+    logger = InvokeAILogger.get_logger(config=app_config)
+
+    # Configure the torch CUDA memory allocator.
+    # NOTE: It is important that this happens before torch is imported.
+    if app_config.pytorch_cuda_alloc_conf:
+        configure_torch_cuda_allocator(app_config.pytorch_cuda_alloc_conf, logger)
+
+    # Import from startup_utils here to avoid importing torch before configure_torch_cuda_allocator() is called.
+    from invokeai.app.util.startup_utils import (
+        apply_monkeypatches,
+        check_cudnn,
+        enable_dev_reload,
+        find_open_port,
+        register_mime_types,
+    )
+
+    # Find an open port, and modify the config accordingly.
+    orig_config_port = app_config.port
+    app_config.port = find_open_port(app_config.port)
+    if orig_config_port != app_config.port:
+        logger.warning(f"Port {orig_config_port} is already in use. Using port {app_config.port}.")
+
+    # Miscellaneous startup tasks.
+    apply_monkeypatches()
+    register_mime_types()
+    if app_config.dev_reload:
+        enable_dev_reload()
+    check_cudnn(logger)
+
+    # Initialize the app and event loop.
+    app, loop = get_app()
+
+    # Load custom nodes. This must be done after importing the Graph class, which itself imports all modules from the
+    # invocations module. The ordering here is implicit, but important - we want to load custom nodes after all the
+    # core nodes have been imported so that we can catch when a custom node clobbers a core node.
+    load_custom_nodes(custom_nodes_path=app_config.custom_nodes_path)
+
+    # Start the server.
+    config = uvicorn.Config(
+        app=app,
+        host=app_config.host,
+        port=app_config.port,
+        loop="asyncio",
+        log_level=app_config.log_level_network,
+        ssl_certfile=app_config.ssl_certfile,
+        ssl_keyfile=app_config.ssl_keyfile,
+    )
+    server = uvicorn.Server(config)
+
+    # replace uvicorn's loggers with InvokeAI's for consistent appearance
+    uvicorn_logger = InvokeAILogger.get_logger("uvicorn")
+    uvicorn_logger.handlers.clear()
+    for hdlr in logger.handlers:
+        uvicorn_logger.addHandler(hdlr)
+
+    loop.run_until_complete(server.serve())

invokeai/app/services/config/config_default.py

@@ -91,6 +91,7 @@ class InvokeAIAppConfig(BaseSettings):
         ram: DEPRECATED: This setting is no longer used. It has been replaced by `max_cache_ram_gb`, but most users will not need to use this config since automatic cache size limits should work well in most cases. This config setting will be removed once the new model cache behavior is stable.
         vram: DEPRECATED: This setting is no longer used. It has been replaced by `max_cache_vram_gb`, but most users will not need to use this config since automatic cache size limits should work well in most cases. This config setting will be removed once the new model cache behavior is stable.
         lazy_offload: DEPRECATED: This setting is no longer used. Lazy-offloading is enabled by default. This config setting will be removed once the new model cache behavior is stable.
+        pytorch_cuda_alloc_conf: Configure the Torch CUDA memory allocator. This will impact peak reserved VRAM usage and performance. Setting to "backend:cudaMallocAsync" works well on many systems. The optimal configuration is highly dependent on the system configuration (device type, VRAM, CUDA driver version, etc.), so must be tuned experimentally.
         device: Preferred execution device. `auto` will choose the device depending on the hardware platform and the installed torch capabilities.<br>Valid values: `auto`, `cpu`, `cuda`, `cuda:1`, `mps`
         precision: Floating point precision. `float16` will consume half the memory of `float32` but produce slightly lower-quality images. The `auto` setting will guess the proper precision based on your video card and operating system.<br>Valid values: `auto`, `float16`, `bfloat16`, `float32`
         sequential_guidance: Whether to calculate guidance in serial instead of in parallel, lowering memory requirements.
@@ -169,6 +170,9 @@ class InvokeAIAppConfig(BaseSettings):
     vram:               Optional[float] = Field(default=None, ge=0,         description="DEPRECATED: This setting is no longer used. It has been replaced by `max_cache_vram_gb`, but most users will not need to use this config since automatic cache size limits should work well in most cases. This config setting will be removed once the new model cache behavior is stable.")
     lazy_offload:                  bool = Field(default=True,               description="DEPRECATED: This setting is no longer used. Lazy-offloading is enabled by default. This config setting will be removed once the new model cache behavior is stable.")
 
+    # PyTorch Memory Allocator
+    pytorch_cuda_alloc_conf: Optional[str] = Field(default=None,            description="Configure the Torch CUDA memory allocator. This will impact peak reserved VRAM usage and performance. Setting to \"backend:cudaMallocAsync\" works well on many systems. The optimal configuration is highly dependent on the system configuration (device type, VRAM, CUDA driver version, etc.), so must be tuned experimentally.")
+
     # DEVICE
     device:                      DEVICE = Field(default="auto",             description="Preferred execution device. `auto` will choose the device depending on the hardware platform and the installed torch capabilities.")
     precision:                PRECISION = Field(default="auto",             description="Floating point precision. `float16` will consume half the memory of `float32` but produce slightly lower-quality images. The `auto` setting will guess the proper precision based on your video card and operating system.")

invokeai/app/util/startup_utils.py

@@ -0,0 +1,64 @@
+import logging
+import mimetypes
+import socket
+
+import torch
+
+
+def find_open_port(port: int) -> int:
+    """Find a port not in use starting at given port"""
+    # Taken from https://waylonwalker.com/python-find-available-port/, thanks Waylon!
+    # https://github.com/WaylonWalker
+    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+        s.settimeout(1)
+        if s.connect_ex(("localhost", port)) == 0:
+            return find_open_port(port=port + 1)
+        else:
+            return port
+
+
+def check_cudnn(logger: logging.Logger) -> None:
+    """Check for cuDNN issues that could be causing degraded performance."""
+    if torch.backends.cudnn.is_available():
+        try:
+            # Note: At the time of writing (torch 2.2.1), torch.backends.cudnn.version() only raises an error the first
+            # time it is called. Subsequent calls will return the version number without complaining about a mismatch.
+            cudnn_version = torch.backends.cudnn.version()
+            logger.info(f"cuDNN version: {cudnn_version}")
+        except RuntimeError as e:
+            logger.warning(
+                "Encountered a cuDNN version issue. This may result in degraded performance. This issue is usually "
+                "caused by an incompatible cuDNN version installed in your python environment, or on the host "
+                f"system. Full error message:\n{e}"
+            )
+
+
+def enable_dev_reload() -> None:
+    """Enable hot reloading on python file changes during development."""
+    from invokeai.backend.util.logging import InvokeAILogger
+
+    try:
+        import jurigged
+    except ImportError as e:
+        raise RuntimeError(
+            'Can\'t start `--dev_reload` because jurigged is not found; `pip install -e ".[dev]"` to include development dependencies.'
+        ) from e
+    else:
+        jurigged.watch(logger=InvokeAILogger.get_logger(name="jurigged").info)
+
+
+def apply_monkeypatches() -> None:
+    """Apply monkeypatches to fix issues with third-party libraries."""
+
+    import invokeai.backend.util.hotfixes  # noqa: F401 (monkeypatching on import)
+
+    if torch.backends.mps.is_available():
+        import invokeai.backend.util.mps_fixes  # noqa: F401 (monkeypatching on import)
+
+
+def register_mime_types() -> None:
+    """Register additional mime types for windows."""
+    # Fix for windows mimetypes registry entries being borked.
+    # see https://github.com/invoke-ai/InvokeAI/discussions/3684#discussioncomment-6391352
+    mimetypes.add_type("application/javascript", ".js")
+    mimetypes.add_type("text/css", ".css")

invokeai/app/util/torch_cuda_allocator.py

@@ -0,0 +1,42 @@
+import logging
+import os
+
+
+def configure_torch_cuda_allocator(pytorch_cuda_alloc_conf: str, logger: logging.Logger | None = None):
+    """Configure the PyTorch CUDA memory allocator. See
+    https://pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf for supported
+    configurations.
+    """
+
+    # Raise if the PYTORCH_CUDA_ALLOC_CONF environment variable is already set.
+    prev_cuda_alloc_conf = os.environ.get("PYTORCH_CUDA_ALLOC_CONF", None)
+    if prev_cuda_alloc_conf is not None:
+        raise RuntimeError(
+            f"Attempted to configure the PyTorch CUDA memory allocator, but PYTORCH_CUDA_ALLOC_CONF is already set to "
+            f"'{prev_cuda_alloc_conf}'."
+        )
+
+    # Configure the PyTorch CUDA memory allocator.
+    # NOTE: It is important that this happens before torch is imported.
+    os.environ["PYTORCH_CUDA_ALLOC_CONF"] = pytorch_cuda_alloc_conf
+
+    import torch
+
+    # Relevant docs: https://pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf
+    if not torch.cuda.is_available():
+        raise RuntimeError(
+            "Attempted to configure the PyTorch CUDA memory allocator, but no CUDA devices are available."
+        )
+
+    # Verify that the torch allocator was properly configured.
+    allocator_backend = torch.cuda.get_allocator_backend()
+    expected_backend = "cudaMallocAsync" if "cudaMallocAsync" in pytorch_cuda_alloc_conf else "native"
+    if allocator_backend != expected_backend:
+        raise RuntimeError(
+            f"Failed to configure the PyTorch CUDA memory allocator. Expected backend: '{expected_backend}', but got "
+            f"'{allocator_backend}'. Verify that 1) the pytorch_cuda_alloc_conf is set correctly, and 2) that torch is "
+            "not imported before calling configure_torch_cuda_allocator()."
+        )
+
+    if logger is not None:
+        logger.info(f"PyTorch CUDA memory allocator: {torch.cuda.get_allocator_backend()}")

invokeai/frontend/web/public/locales/en.json

@@ -921,6 +921,7 @@
         "currentImage": "Current Image",
         "currentImageDescription": "Displays the current image in the Node Editor",
         "downloadWorkflow": "Download Workflow JSON",
+        "downloadWorkflowError": "Error downloading workflow",
         "edge": "Edge",
         "edit": "Edit",
         "editMode": "Edit in Workflow Editor",

invokeai/frontend/web/src/common/hooks/useImageUploadButton.tsx

@@ -128,7 +128,11 @@ export const useImageUploadButton = ({ onUpload, isDisabled, allowMultiple }: Us
     getInputProps: getUploadInputProps,
     open: openUploader,
   } = useDropzone({
-    accept: { 'image/png': ['.png'], 'image/jpeg': ['.jpg', '.jpeg', '.png'] },
+    accept: {
+      'image/png': ['.png'],
+      'image/jpeg': ['.jpg', '.jpeg', '.png'],
+      'image/webp': ['.webp'],
+    },
     onDropAccepted,
     onDropRejected,
     disabled: isDisabled,

invokeai/frontend/web/src/features/dnd/FullscreenDropzone.tsx

@@ -22,8 +22,8 @@ import { useBoardName } from 'services/api/hooks/useBoardName';
 import type { UploadImageArg } from 'services/api/types';
 import { z } from 'zod';
 
-const ACCEPTED_IMAGE_TYPES = ['image/png', 'image/jpg', 'image/jpeg'];
-const ACCEPTED_FILE_EXTENSIONS = ['.png', '.jpg', '.jpeg'];
+const ACCEPTED_IMAGE_TYPES = ['image/png', 'image/jpg', 'image/jpeg', 'image/webp'];
+const ACCEPTED_FILE_EXTENSIONS = ['.png', '.jpg', '.jpeg', '.webp'];
 
 // const MAX_IMAGE_SIZE = 4; //In MegaBytes
 // const sizeInMB = (sizeInBytes: number, decimalsNum = 2) => {

invokeai/frontend/web/src/features/modelManagerV2/subpanels/ModelPanel/Fields/ModelImageUpload.tsx

@@ -72,7 +72,11 @@ const ModelImageUpload = ({ model_key, model_image }: Props) => {
   }, [model_key, t, deleteModelImage]);
 
   const { getInputProps, getRootProps } = useDropzone({
-    accept: { 'image/png': ['.png'], 'image/jpeg': ['.jpg', '.jpeg', '.png'] },
+    accept: {
+      'image/png': ['.png'],
+      'image/jpeg': ['.jpg', '.jpeg', '.png'],
+      'image/webp': ['.webp'],
+    },
     onDropAccepted,
     noDrag: true,
     multiple: false,

invokeai/frontend/web/src/features/nodes/components/sidePanel/WorkflowListMenu/WorkflowListItem.tsx

@@ -6,7 +6,7 @@ import dateFormat, { masks } from 'dateformat';
 import { selectWorkflowId } from 'features/nodes/store/workflowSlice';
 import { useDeleteWorkflow } from 'features/workflowLibrary/components/DeleteLibraryWorkflowConfirmationAlertDialog';
 import { useLoadWorkflow } from 'features/workflowLibrary/components/LoadWorkflowConfirmationAlertDialog';
-import { useDownloadWorkflow } from 'features/workflowLibrary/hooks/useDownloadWorkflow';
+import { useDownloadWorkflowById } from 'features/workflowLibrary/hooks/useDownloadWorkflowById';
 import type { MouseEvent } from 'react';
 import { useCallback, useMemo, useState } from 'react';
 import { useTranslation } from 'react-i18next';
@@ -30,7 +30,7 @@ export const WorkflowListItem = ({ workflow }: { workflow: WorkflowRecordListIte
   }, []);
 
   const workflowId = useAppSelector(selectWorkflowId);
-  const downloadWorkflow = useDownloadWorkflow();
+  const { downloadWorkflow, isLoading: isLoadingDownloadWorkflow } = useDownloadWorkflowById();
   const shareWorkflow = useShareWorkflow();
   const deleteWorkflow = useDeleteWorkflow();
   const loadWorkflow = useLoadWorkflow();
@@ -71,9 +71,9 @@ export const WorkflowListItem = ({ workflow }: { workflow: WorkflowRecordListIte
     (e: MouseEvent<HTMLButtonElement>) => {
       e.stopPropagation();
       setIsHovered(false);
-      downloadWorkflow();
+      downloadWorkflow(workflow.workflow_id);
     },
-    [downloadWorkflow]
+    [downloadWorkflow, workflow.workflow_id]
   );
 
   return (
@@ -144,6 +144,7 @@ export const WorkflowListItem = ({ workflow }: { workflow: WorkflowRecordListIte
             aria-label={t('workflows.download')}
             onClick={handleClickDownload}
             icon={<PiDownloadSimpleBold />}
+            isLoading={isLoadingDownloadWorkflow}
           />
         </Tooltip>
         {!!projectUrl && workflow.workflow_id && workflow.category !== 'user' && (

invokeai/frontend/web/src/features/nodes/types/field.ts

@@ -331,14 +331,25 @@ const buildInstanceTypeGuard = <T extends z.ZodTypeAny>(schema: T) => {
   return (val: unknown): val is z.infer<T> => schema.safeParse(val).success;
 };
 
+/**
+ * Builds a type guard for a specific field input template type.
+ *
+ * The output type guards are primarily used for determining which input component to render for fields in the
+ * <InputFieldRenderer/> component.
+ *
+ * @param name The name of the field type.
+ * @param cardinalities The allowed cardinalities for the field type. If omitted, all cardinalities are allowed.
+ *
+ *  @returns A type guard for the specified field type.
+ */
 const buildTemplateTypeGuard =
-  <T extends FieldInputTemplate>(name: string, cardinality?: 'SINGLE' | 'COLLECTION' | 'SINGLE_OR_COLLECTION') =>
+  <T extends FieldInputTemplate>(name: string, cardinalities?: FieldType['cardinality'][]) =>
   (template: FieldInputTemplate): template is T => {
     if (template.type.name !== name) {
       return false;
     }
-    if (cardinality) {
-      return template.type.cardinality === cardinality;
+    if (cardinalities) {
+      return cardinalities.includes(template.type.cardinality);
     }
     return true;
   };
@@ -366,7 +377,10 @@ export type IntegerFieldValue = z.infer<typeof zIntegerFieldValue>;
 export type IntegerFieldInputInstance = z.infer<typeof zIntegerFieldInputInstance>;
 export type IntegerFieldInputTemplate = z.infer<typeof zIntegerFieldInputTemplate>;
 export const isIntegerFieldInputInstance = buildInstanceTypeGuard(zIntegerFieldInputInstance);
-export const isIntegerFieldInputTemplate = buildTemplateTypeGuard<IntegerFieldInputTemplate>('IntegerField', 'SINGLE');
+export const isIntegerFieldInputTemplate = buildTemplateTypeGuard<IntegerFieldInputTemplate>('IntegerField', [
+  'SINGLE',
+  'SINGLE_OR_COLLECTION',
+]);
 // #endregion
 
 // #region IntegerField Collection
@@ -406,7 +420,7 @@ export type IntegerFieldCollectionInputTemplate = z.infer<typeof zIntegerFieldCo
 export const isIntegerFieldCollectionInputInstance = buildInstanceTypeGuard(zIntegerFieldCollectionInputInstance);
 export const isIntegerFieldCollectionInputTemplate = buildTemplateTypeGuard<IntegerFieldCollectionInputTemplate>(
   'IntegerField',
-  'COLLECTION'
+  ['COLLECTION']
 );
 // #endregion
 
@@ -432,7 +446,10 @@ export type FloatFieldValue = z.infer<typeof zFloatFieldValue>;
 export type FloatFieldInputInstance = z.infer<typeof zFloatFieldInputInstance>;
 export type FloatFieldInputTemplate = z.infer<typeof zFloatFieldInputTemplate>;
 export const isFloatFieldInputInstance = buildInstanceTypeGuard(zFloatFieldInputInstance);
-export const isFloatFieldInputTemplate = buildTemplateTypeGuard<FloatFieldInputTemplate>('FloatField', 'SINGLE');
+export const isFloatFieldInputTemplate = buildTemplateTypeGuard<FloatFieldInputTemplate>('FloatField', [
+  'SINGLE',
+  'SINGLE_OR_COLLECTION',
+]);
 // #endregion
 
 // #region FloatField Collection
@@ -471,7 +488,7 @@ export type FloatFieldCollectionInputTemplate = z.infer<typeof zFloatFieldCollec
 export const isFloatFieldCollectionInputInstance = buildInstanceTypeGuard(zFloatFieldCollectionInputInstance);
 export const isFloatFieldCollectionInputTemplate = buildTemplateTypeGuard<FloatFieldCollectionInputTemplate>(
   'FloatField',
-  'COLLECTION'
+  ['COLLECTION']
 );
 // #endregion
 
@@ -504,7 +521,10 @@ export type StringFieldValue = z.infer<typeof zStringFieldValue>;
 export type StringFieldInputInstance = z.infer<typeof zStringFieldInputInstance>;
 export type StringFieldInputTemplate = z.infer<typeof zStringFieldInputTemplate>;
 export const isStringFieldInputInstance = buildInstanceTypeGuard(zStringFieldInputInstance);
-export const isStringFieldInputTemplate = buildTemplateTypeGuard<StringFieldInputTemplate>('StringField', 'SINGLE');
+export const isStringFieldInputTemplate = buildTemplateTypeGuard<StringFieldInputTemplate>('StringField', [
+  'SINGLE',
+  'SINGLE_OR_COLLECTION',
+]);
 // #endregion
 
 // #region StringField Collection
@@ -550,7 +570,7 @@ export type StringFieldCollectionInputTemplate = z.infer<typeof zStringFieldColl
 export const isStringFieldCollectionInputInstance = buildInstanceTypeGuard(zStringFieldCollectionInputInstance);
 export const isStringFieldCollectionInputTemplate = buildTemplateTypeGuard<StringFieldCollectionInputTemplate>(
   'StringField',
-  'COLLECTION'
+  ['COLLECTION']
 );
 // #endregion
 
@@ -613,7 +633,10 @@ export type ImageFieldValue = z.infer<typeof zImageFieldValue>;
 export type ImageFieldInputInstance = z.infer<typeof zImageFieldInputInstance>;
 export type ImageFieldInputTemplate = z.infer<typeof zImageFieldInputTemplate>;
 export const isImageFieldInputInstance = buildInstanceTypeGuard(zImageFieldInputInstance);
-export const isImageFieldInputTemplate = buildTemplateTypeGuard<ImageFieldInputTemplate>('ImageField', 'SINGLE');
+export const isImageFieldInputTemplate = buildTemplateTypeGuard<ImageFieldInputTemplate>('ImageField', [
+  'SINGLE',
+  'SINGLE_OR_COLLECTION',
+]);
 // #endregion
 
 // #region ImageField Collection
@@ -648,7 +671,7 @@ export type ImageFieldCollectionInputTemplate = z.infer<typeof zImageFieldCollec
 export const isImageFieldCollectionInputInstance = buildInstanceTypeGuard(zImageFieldCollectionInputInstance);
 export const isImageFieldCollectionInputTemplate = buildTemplateTypeGuard<ImageFieldCollectionInputTemplate>(
   'ImageField',
-  'COLLECTION'
+  ['COLLECTION']
 );
 // #endregion
 

invokeai/frontend/web/src/features/workflowLibrary/components/WorkflowLibraryMenu/DownloadWorkflowMenuItem.tsx

@@ -1,12 +1,12 @@
 import { MenuItem } from '@invoke-ai/ui-library';
-import { useDownloadWorkflow } from 'features/workflowLibrary/hooks/useDownloadWorkflow';
+import { useDownloadCurrentlyLoadedWorkflow } from 'features/workflowLibrary/hooks/useDownloadCurrentlyLoadedWorkflow';
 import { memo } from 'react';
 import { useTranslation } from 'react-i18next';
 import { PiDownloadSimpleBold } from 'react-icons/pi';
 
 const DownloadWorkflowMenuItem = () => {
   const { t } = useTranslation();
-  const downloadWorkflow = useDownloadWorkflow();
+  const downloadWorkflow = useDownloadCurrentlyLoadedWorkflow();
 
   return (
     <MenuItem as="button" icon={<PiDownloadSimpleBold />} onClick={downloadWorkflow}>

invokeai/frontend/web/src/features/workflowLibrary/hooks/useDownloadCurrentlyLoadedWorkflow.ts

@@ -3,7 +3,7 @@ import { $builtWorkflow } from 'features/nodes/hooks/useWorkflowWatcher';
 import { workflowDownloaded } from 'features/workflowLibrary/store/actions';
 import { useCallback } from 'react';
 
-export const useDownloadWorkflow = () => {
+export const useDownloadCurrentlyLoadedWorkflow = () => {
   const dispatch = useAppDispatch();
 
   const downloadWorkflow = useCallback(() => {

invokeai/frontend/web/src/features/workflowLibrary/hooks/useDownloadWorkflowById.ts

@@ -0,0 +1,42 @@
+import { useAppDispatch } from 'app/store/storeHooks';
+import { toast } from 'features/toast/toast';
+import { workflowDownloaded } from 'features/workflowLibrary/store/actions';
+import { useCallback } from 'react';
+import { useTranslation } from 'react-i18next';
+import { useLazyGetWorkflowQuery } from 'services/api/endpoints/workflows';
+
+export const useDownloadWorkflowById = () => {
+  const { t } = useTranslation();
+  const dispatch = useAppDispatch();
+  const [trigger, query] = useLazyGetWorkflowQuery();
+
+  const toastError = useCallback(() => {
+    toast({ status: 'error', description: t('nodes.downloadWorkflowError') });
+  }, [t]);
+
+  const downloadWorkflow = useCallback(
+    async (workflowId: string) => {
+      try {
+        const { data } = await trigger(workflowId);
+        if (!data) {
+          toastError();
+          return;
+        }
+        const { workflow } = data;
+        const blob = new Blob([JSON.stringify(workflow, null, 2)]);
+        const a = document.createElement('a');
+        a.href = URL.createObjectURL(blob);
+        a.download = `${workflow.name || 'My Workflow'}.json`;
+        document.body.appendChild(a);
+        a.click();
+        a.remove();
+        dispatch(workflowDownloaded());
+      } catch {
+        toastError();
+      }
+    },
+    [dispatch, toastError, trigger]
+  );
+
+  return { downloadWorkflow, isLoading: query.isLoading };
+};

invokeai/version/invokeai_version.py

@@ -1 +1 @@
-__version__ = "5.7.1"
+__version__ = "5.7.2rc1"

tests/app/util/test_torch_cuda_allocator.py

@@ -0,0 +1,13 @@
+import pytest
+import torch
+
+from invokeai.app.util.torch_cuda_allocator import configure_torch_cuda_allocator
+
+
+@pytest.mark.skipif(not torch.cuda.is_available(), reason="Requires CUDA device.")
+def test_configure_torch_cuda_allocator_raises_if_torch_is_already_imported():
+    """Test that configure_torch_cuda_allocator() raises a RuntimeError if torch is already imported."""
+    import torch  # noqa: F401
+
+    with pytest.raises(RuntimeError, match="Failed to configure the PyTorch CUDA memory allocator."):
+        configure_torch_cuda_allocator("backend:cudaMallocAsync")