Configure the NSFW checker at install time with default on (#1624)

* configure the NSFW checker at install time with default on

1. Changes the --safety_checker argument to --nsfw_checker and
--no-nsfw_checker. The original argument is recognized for backward
compatibility.

2. The configure script asks users whether to enable the checker
(default yes). Also offers users ability to select default sampler and
number of generation steps.

3.Enables the pasting of the caution icon on blurred images when
InvokeAI is installed into the package directory.

4. Adds documentation for the NSFW checker, including caveats about
accuracy, memory requirements, and intermediate image dispaly.

* use better fitting icon

* NSFW defaults false for testing

* set default back to nsfw active

Co-authored-by: Matthias Wild <40327258+mauwii@users.noreply.github.com>
Co-authored-by: mauwii <Mauwii@outlook.de>

.dev_scripts/diff_images.py

@@ -0,0 +1,32 @@
+import argparse
+
+import numpy as np
+from PIL import Image
+
+
+def read_image_int16(image_path):
+    image = Image.open(image_path)
+    return np.array(image).astype(np.int16)
+
+
+def calc_images_mean_L1(image1_path, image2_path):
+    image1 = read_image_int16(image1_path)
+    image2 = read_image_int16(image2_path)
+    assert image1.shape == image2.shape
+
+    mean_L1 = np.abs(image1 - image2).mean()
+    return mean_L1
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument('image1_path')
+    parser.add_argument('image2_path')
+    args = parser.parse_args()
+    return args
+
+
+if __name__ == '__main__':
+    args = parse_args()
+    mean_L1 = calc_images_mean_L1(args.image1_path, args.image2_path)
+    print(mean_L1)

.dev_scripts/sample_command.txt

@@ -0,0 +1 @@
+"a photograph of an astronaut riding a horse" -s50 -S42

.dev_scripts/test_regression_txt2img_dream_v1_4.sh

@@ -0,0 +1,19 @@
+# generate an image
+PROMPT_FILE=".dev_scripts/sample_command.txt"
+OUT_DIR="outputs/img-samples/test_regression_txt2img_v1_4"
+SAMPLES_DIR=${OUT_DIR}
+python scripts/dream.py \
+    --from_file ${PROMPT_FILE} \
+    --outdir ${OUT_DIR} \
+    --sampler plms
+
+# original output by CompVis/stable-diffusion
+IMAGE1=".dev_scripts/images/v1_4_astronaut_rides_horse_plms_step50_seed42.png"
+# new output
+IMAGE2=`ls -A ${SAMPLES_DIR}/*.png | sort | tail -n 1`
+
+echo ""
+echo "comparing the following two images"
+echo "IMAGE1: ${IMAGE1}"
+echo "IMAGE2: ${IMAGE2}"
+python .dev_scripts/diff_images.py ${IMAGE1} ${IMAGE2}

.dev_scripts/test_regression_txt2img_v1_4.sh

@@ -0,0 +1,23 @@
+# generate an image
+PROMPT="a photograph of an astronaut riding a horse"
+OUT_DIR="outputs/txt2img-samples/test_regression_txt2img_v1_4"
+SAMPLES_DIR="outputs/txt2img-samples/test_regression_txt2img_v1_4/samples"
+python scripts/orig_scripts/txt2img.py \
+    --prompt "${PROMPT}" \
+    --outdir ${OUT_DIR} \
+    --plms \
+    --ddim_steps 50 \
+    --n_samples 1 \
+    --n_iter 1 \
+    --seed 42
+
+# original output by CompVis/stable-diffusion
+IMAGE1=".dev_scripts/images/v1_4_astronaut_rides_horse_plms_step50_seed42.png"
+# new output
+IMAGE2=`ls -A ${SAMPLES_DIR}/*.png | sort | tail -n 1`
+
+echo ""
+echo "comparing the following two images"
+echo "IMAGE1: ${IMAGE1}"
+echo "IMAGE2: ${IMAGE2}"
+python .dev_scripts/diff_images.py ${IMAGE1} ${IMAGE2}

.dockerignore

@@ -0,0 +1,12 @@
+*
+!backend
+!configs
+!environments-and-requirements
+!frontend
+!installer
+!ldm
+!main.py
+!scripts
+!server
+!static
+!setup.py

.editorconfig

@@ -0,0 +1,12 @@
+# All files
+[*]
+charset = utf-8
+end_of_line = lf
+indent_size = 2
+indent_style = space
+insert_final_newline = true
+trim_trailing_whitespace = true
+
+# Python
+[*.py]
+indent_size = 4

.gitattributes

@@ -0,0 +1,4 @@
+# Auto normalizes line endings on commit so devs don't need to change local settings.
+# Only affects text files and ignores other file types. 
+# For more info see: https://www.aleksandrhovhannisyan.com/blog/crlf-vs-lf-normalizing-line-endings-in-git/
+* text=auto

.github/CODEOWNERS

@@ -0,0 +1,7 @@
+ldm/invoke/pngwriter.py @CapableWeb
+ldm/invoke/server_legacy.py @CapableWeb
+scripts/legacy_api.py @CapableWeb
+tests/legacy_tests.sh @CapableWeb
+installer/ @tildebyte
+.github/workflows/ @mauwii
+docker_build/ @mauwii

.github/ISSUE_TEMPLATE/BUG_REPORT.yml

@@ -0,0 +1,102 @@
+name: 🐞 Bug Report
+
+description: File a bug report
+
+title: '[bug]: '
+
+labels: ['bug']
+
+# assignees:
+#   - moderator_bot
+#   - lstein
+
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for taking the time to fill out this Bug Report!
+
+  - type: checkboxes
+    attributes:
+      label: Is there an existing issue for this?
+      description: |
+        Please use the [search function](https://github.com/invoke-ai/InvokeAI/issues?q=is%3Aissue+is%3Aopen+label%3Abug)
+        irst to see if an issue already exists for the bug you encountered.
+      options:
+        - label: I have searched the existing issues
+          required: true
+
+  - type: markdown
+    attributes:
+      value: __Describe your environment__
+
+  - type: dropdown
+    id: os_dropdown
+    attributes:
+      label: OS
+      description: Which operating System did you use when the bug occured
+      multiple: false
+      options:
+        - 'Linux'
+        - 'Windows'
+        - 'macOS'
+    validations:
+      required: true
+
+  - type: dropdown
+    id: gpu_dropdown
+    attributes:
+      label: GPU
+      description: Which kind of Graphic-Adapter is your System using
+      multiple: false
+      options:
+        - 'cuda'
+        - 'amd'
+        - 'mps'
+        - 'cpu'
+    validations:
+      required: true
+
+  - type: input
+    id: vram
+    attributes:
+      label: VRAM
+      description: Size of the VRAM if known
+      placeholder: 8GB
+    validations:
+      required: false
+
+  - type: textarea
+    id: what-happened
+    attributes:
+      label: What happened?
+      description: |
+        Briefly describe what happened, what you expected to happen and how to reproduce this bug.
+      placeholder: When using the webinterface and right-clicking on button X instead of the popup-menu there error Y appears
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: Screenshots
+      description: If applicable, add screenshots to help explain your problem
+      placeholder: this is what the result looked like <screenshot>
+    validations:
+      required: false
+
+  - type: textarea
+    attributes:
+      label: Additional context
+      description: Add any other context about the problem here
+      placeholder: Only happens when there is full moon and Friday the 13th on Christmas Eve 🎅🏻
+    validations:
+      required: false
+
+  - type: input
+    id: contact
+    attributes:
+      label: Contact Details
+      description: __OPTIONAL__ How can we get in touch with you if we need more info (besides this issue)?
+      placeholder: ex. email@example.com, discordname, twitter, ...
+    validations:
+      required: false

.github/ISSUE_TEMPLATE/FEATURE_REQUEST.yml

@@ -0,0 +1,56 @@
+name: Feature Request
+description: Commit a idea or Request a new feature
+title: '[enhancement]: '
+labels: ['enhancement']
+# assignees:
+#   - lstein
+#   - tildebyte
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for taking the time to fill out this Feature request!
+
+  - type: checkboxes
+    attributes:
+      label: Is there an existing issue for this?
+      description: |
+        Please make use of the [search function](https://github.com/invoke-ai/InvokeAI/labels/enhancement)
+        to see if a simmilar issue already exists for the feature you want to request
+      options:
+        - label: I have searched the existing issues
+          required: true
+
+  - type: input
+    id: contact
+    attributes:
+      label: Contact Details
+      description: __OPTIONAL__ How could we get in touch with you if we need more info (besides this issue)?
+      placeholder: ex. email@example.com, discordname, twitter, ...
+    validations:
+      required: false
+
+  - type: textarea
+    id: whatisexpected
+    attributes:
+      label: What should this feature add?
+      description: Please try to explain the functionality this feature should add
+      placeholder: |
+        Instead of one huge textfield, it would be nice to have forms for bug-reports, feature-requests, ...
+        Great benefits with automatic labeling, assigning and other functionalitys not available in that form
+        via old-fashioned markdown-templates. I would also love to see the use of a moderator bot 🤖 like
+        https://github.com/marketplace/actions/issue-moderator-with-commands to auto close old issues and other things
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: Alternatives
+      description: Describe alternatives you've considered
+      placeholder: A clear and concise description of any alternative solutions or features you've considered.
+
+  - type: textarea
+    attributes:
+      label: Aditional Content
+      description: Add any other context or screenshots about the feature request here.
+      placeholder: This is a Mockup of the design how I imagine it <screenshot>

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,14 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Project-Documentation
+    url: https://invoke-ai.github.io/InvokeAI/
+    about: Should be your first place to go when looking for manuals/FAQs regarding our InvokeAI Toolkit
+  - name: Discord
+    url: https://discord.gg/ZmtBAhwWhy
+    about: Our Discord Community could maybe help you out via live-chat
+  - name: GitHub Community Support
+    url: https://github.com/orgs/community/discussions
+    about: Please ask and answer questions regarding the GitHub Platform here.
+  - name: GitHub Security Bug Bounty
+    url: https://bounty.github.com/
+    about: Please report security vulnerabilities of the GitHub Platform here.

.github/workflows/build-container.yml

@@ -0,0 +1,43 @@
+# Building the Image without pushing to confirm it is still buildable
+# confirum functionality would unfortunately need way more resources
+name: build container image
+on:
+  push:
+    branches:
+      - 'main'
+      - 'development'
+      - 'update-dockerfile'
+
+jobs:
+  docker:
+    strategy:
+      fail-fast: false
+      matrix:
+        arch:
+          - x86_64
+          - aarch64
+        pip-requirements:
+          - requirements-lin-amd.txt
+          - requirements-lin-cuda.txt
+    runs-on: ubuntu-latest
+    name: ${{ matrix.pip-requirements }} ${{ matrix.arch }}
+    steps:
+      - name: prepare docker-tag
+        env:
+          repository: ${{ github.repository }}
+        run: echo "dockertag=${repository,,}" >> $GITHUB_ENV
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v2
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+      - name: Build container
+        uses: docker/build-push-action@v3
+        with:
+          context: .
+          file: docker-build/Dockerfile
+          platforms: Linux/${{ matrix.arch }}
+          push: false
+          tags: ${{ env.dockertag }}:${{ matrix.pip-requirements }}-${{ matrix.arch }}
+          build-args: pip_requirements=${{ matrix.pip-requirements }}

.github/workflows/mkdocs-material.yml

@@ -0,0 +1,40 @@
+name: mkdocs-material
+on:
+  push:
+    branches:
+      - 'main'
+      - 'development'
+
+jobs:
+  mkdocs-material:
+    runs-on: ubuntu-latest
+    steps:
+      - name: checkout sources
+        uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+
+      - name: setup python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.10'
+
+      - name: install requirements
+        run: |
+          python -m \
+            pip install -r docs/requirements-mkdocs.txt
+
+      - name: confirm buildability
+        run: |
+          python -m \
+            mkdocs build \
+            --clean \
+            --verbose
+
+      - name: deploy to gh-pages
+        if: ${{ github.ref == 'refs/heads/main' }}
+        run: |
+          python -m \
+            mkdocs gh-deploy \
+            --clean \
+            --force

.github/workflows/test-invoke-conda.yml

@@ -0,0 +1,135 @@
+name: Test invoke.py
+on:
+  push:
+    branches:
+      - 'main'
+      - 'development'
+      - 'fix-gh-actions-fork'
+  pull_request:
+    branches:
+      - 'main'
+      - 'development'
+
+jobs:
+  matrix:
+    strategy:
+      matrix:
+        stable-diffusion-model:
+          - 'stable-diffusion-1.5'
+        environment-yaml:
+          - environment-lin-amd.yml
+          - environment-lin-cuda.yml
+          - environment-mac.yml
+        include:
+          - environment-yaml: environment-lin-amd.yml
+            os: ubuntu-latest
+            default-shell: bash -l {0}
+          - environment-yaml: environment-lin-cuda.yml
+            os: ubuntu-latest
+            default-shell: bash -l {0}
+          - environment-yaml: environment-mac.yml
+            os: macos-12
+            default-shell: bash -l {0}
+          - stable-diffusion-model: stable-diffusion-1.5
+            stable-diffusion-model-url: https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.ckpt
+            stable-diffusion-model-dl-path: models/ldm/stable-diffusion-v1
+            stable-diffusion-model-dl-name: v1-5-pruned-emaonly.ckpt
+    name: ${{ matrix.environment-yaml }} on ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    env:
+      CONDA_ENV_NAME: invokeai
+      INVOKEAI_ROOT: '${{ github.workspace }}/invokeai'
+    defaults:
+      run:
+        shell: ${{ matrix.default-shell }}
+    steps:
+      - name: Checkout sources
+        id: checkout-sources
+        uses: actions/checkout@v3
+
+      - name: create models.yaml from example
+        run: |
+          mkdir -p ${{ env.INVOKEAI_ROOT }}/configs
+          cp configs/models.yaml.example ${{ env.INVOKEAI_ROOT }}/configs/models.yaml
+
+      - name: create environment.yml
+        run: cp "environments-and-requirements/${{ matrix.environment-yaml }}" environment.yml
+
+      - name: Use cached conda packages
+        id: use-cached-conda-packages
+        uses: actions/cache@v3
+        with:
+          path: ~/conda_pkgs_dir
+          key: conda-pkgs-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles(matrix.environment-yaml) }}
+
+      - name: Activate Conda Env
+        id: activate-conda-env
+        uses: conda-incubator/setup-miniconda@v2
+        with:
+          activate-environment: ${{ env.CONDA_ENV_NAME }}
+          environment-file: environment.yml
+          miniconda-version: latest
+
+      - name: set test prompt to main branch validation
+        if: ${{ github.ref == 'refs/heads/main' }}
+        run: echo "TEST_PROMPTS=tests/preflight_prompts.txt" >> $GITHUB_ENV
+
+      - name: set test prompt to development branch validation
+        if: ${{ github.ref == 'refs/heads/development' }}
+        run: echo "TEST_PROMPTS=tests/dev_prompts.txt" >> $GITHUB_ENV
+
+      - name: set test prompt to Pull Request validation
+        if: ${{ github.ref != 'refs/heads/main' && github.ref != 'refs/heads/development' }}
+        run: echo "TEST_PROMPTS=tests/validate_pr_prompt.txt" >> $GITHUB_ENV
+
+      - name: Use Cached Stable Diffusion Model
+        id: cache-sd-model
+        uses: actions/cache@v3
+        env:
+          cache-name: cache-${{ matrix.stable-diffusion-model }}
+        with:
+          path: ${{ env.INVOKEAI_ROOT }}/${{ matrix.stable-diffusion-model-dl-path }}
+          key: ${{ env.cache-name }}
+
+      - name: Download ${{ matrix.stable-diffusion-model }}
+        id: download-stable-diffusion-model
+        if: ${{ steps.cache-sd-model.outputs.cache-hit != 'true' }}
+        run: |
+          mkdir -p "${{ env.INVOKEAI_ROOT }}/${{ matrix.stable-diffusion-model-dl-path }}"
+          curl \
+            -H "Authorization: Bearer ${{ secrets.HUGGINGFACE_TOKEN }}" \
+            -o "${{ env.INVOKEAI_ROOT }}/${{ matrix.stable-diffusion-model-dl-path }}/${{ matrix.stable-diffusion-model-dl-name }}" \
+            -L ${{ matrix.stable-diffusion-model-url }}
+
+      - name: run configure_invokeai.py
+        id: run-preload-models
+        run: |
+          python scripts/configure_invokeai.py --no-interactive --yes
+
+      - name: cat ~/.invokeai
+        id: cat-invokeai
+        run: cat ~/.invokeai
+
+      - name: Run the tests
+        id: run-tests
+        run: |
+          time python scripts/invoke.py \
+            --no-patchmatch \
+            --no-nsfw_checker \
+            --model ${{ matrix.stable-diffusion-model }} \
+            --from_file ${{ env.TEST_PROMPTS }} \
+            --root="${{ env.INVOKEAI_ROOT }}" \
+            --outdir="${{ env.INVOKEAI_ROOT }}/outputs"
+
+      - name: export conda env
+        id: export-conda-env
+        run: |
+          mkdir -p outputs/img-samples
+          conda env export --name ${{ env.CONDA_ENV_NAME }} > outputs/img-samples/environment-${{ runner.os }}-${{ runner.arch }}.yml
+
+      - name: Archive results
+        id: archive-results
+        uses: actions/upload-artifact@v3
+        with:
+          name: results_${{ matrix.requirements-file }}_${{ matrix.python-version }}
+          path: ${{ env.INVOKEAI_ROOT }}/outputs

.github/workflows/test-invoke-pip.yml

@@ -0,0 +1,128 @@
+name: Test invoke.py pip
+on:
+  push:
+    branches:
+      - 'main'
+      - 'development'
+  pull_request:
+    branches:
+      - 'main'
+      - 'development'
+
+jobs:
+  matrix:
+    strategy:
+      matrix:
+        stable-diffusion-model:
+          - stable-diffusion-1.5
+        requirements-file:
+          - requirements-lin-cuda.txt
+          - requirements-lin-amd.txt
+          - requirements-mac-mps-cpu.txt
+        python-version:
+          # - '3.9'
+          - '3.10'
+        include:
+          - requirements-file: requirements-lin-cuda.txt
+            os: ubuntu-latest
+            default-shell: bash -l {0}
+          - requirements-file: requirements-lin-amd.txt
+            os: ubuntu-latest
+            default-shell: bash -l {0}
+          - requirements-file: requirements-mac-mps-cpu.txt
+            os: macOS-12
+            default-shell: bash -l {0}
+          - stable-diffusion-model: stable-diffusion-1.5
+            stable-diffusion-model-url: https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.ckpt
+            stable-diffusion-model-dl-path: models/ldm/stable-diffusion-v1
+            stable-diffusion-model-dl-name: v1-5-pruned-emaonly.ckpt
+    name: ${{ matrix.requirements-file }} on ${{ matrix.python-version }}
+    runs-on: ${{ matrix.os }}
+    defaults:
+      run:
+        shell: ${{ matrix.default-shell }}
+    env:
+      INVOKEAI_ROOT: '${{ github.workspace }}/invokeai'
+    steps:
+      - name: Checkout sources
+        id: checkout-sources
+        uses: actions/checkout@v3
+
+      - name: create models.yaml from example
+        run: |
+          mkdir -p ${{ env.INVOKEAI_ROOT }}/configs
+          cp configs/models.yaml.example ${{ env.INVOKEAI_ROOT }}/configs/models.yaml
+
+      - name: set test prompt to main branch validation
+        if: ${{ github.ref == 'refs/heads/main' }}
+        run: echo "TEST_PROMPTS=tests/preflight_prompts.txt" >> $GITHUB_ENV
+
+      - name: set test prompt to development branch validation
+        if: ${{ github.ref == 'refs/heads/development' }}
+        run: echo "TEST_PROMPTS=tests/dev_prompts.txt" >> $GITHUB_ENV
+
+      - name: set test prompt to Pull Request validation
+        if: ${{ github.ref != 'refs/heads/main' && github.ref != 'refs/heads/development' }}
+        run: echo "TEST_PROMPTS=tests/validate_pr_prompt.txt" >> $GITHUB_ENV
+
+      - name: create requirements.txt
+        run: cp 'environments-and-requirements/${{ matrix.requirements-file }}' '${{ matrix.requirements-file }}'
+
+      - name: setup python
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: 'pip'
+          cache-dependency-path: ${{ matrix.requirements-file }}
+
+      # - name: install dependencies
+      #   run: ${{ env.pythonLocation }}/bin/pip install --upgrade pip setuptools wheel
+
+      - name: install requirements
+        run: ${{ env.pythonLocation }}/bin/pip install -r '${{ matrix.requirements-file }}'
+
+      - name: Use Cached Stable Diffusion Model
+        id: cache-sd-model
+        uses: actions/cache@v3
+        env:
+          cache-name: cache-${{ matrix.stable-diffusion-model }}
+        with:
+          path: ${{ env.INVOKEAI_ROOT }}/${{ matrix.stable-diffusion-model-dl-path }}
+          key: ${{ env.cache-name }}
+
+      - name: Download ${{ matrix.stable-diffusion-model }}
+        id: download-stable-diffusion-model
+        if: ${{ steps.cache-sd-model.outputs.cache-hit != 'true' }}
+        run: |
+          mkdir -p "${{ env.INVOKEAI_ROOT }}/${{ matrix.stable-diffusion-model-dl-path }}"
+          curl \
+            -H "Authorization: Bearer ${{ secrets.HUGGINGFACE_TOKEN }}" \
+            -o "${{ env.INVOKEAI_ROOT }}/${{ matrix.stable-diffusion-model-dl-path }}/${{ matrix.stable-diffusion-model-dl-name }}" \
+            -L ${{ matrix.stable-diffusion-model-url }}
+
+      - name: run configure_invokeai.py
+        id: run-preload-models
+        run: |
+          ${{ env.pythonLocation }}/bin/python scripts/configure_invokeai.py --no-interactive --yes
+
+      - name: cat ~/.invokeai
+        id: cat-invokeai
+        run: cat ~/.invokeai
+
+      - name: Run the tests
+        id: run-tests
+        run: |
+          time ${{ env.pythonLocation }}/bin/python scripts/invoke.py \
+            --no-patchmatch \
+            --no-nsfw_checker \
+            --model ${{ matrix.stable-diffusion-model }} \
+            --from_file ${{ env.TEST_PROMPTS }} \
+            --root="${{ env.INVOKEAI_ROOT }}" \
+            --outdir="${{ env.INVOKEAI_ROOT }}/outputs"
+
+      - name: Archive results
+        id: archive-results
+        uses: actions/upload-artifact@v3
+        with:
+          name: results_${{ matrix.requirements-file }}_${{ matrix.python-version }}
+          path: ${{ env.INVOKEAI_ROOT }}/outputs

.gitignore

@@ -0,0 +1,236 @@
+# ignore default image save location and model symbolic link
+outputs/
+models/ldm/stable-diffusion-v1/model.ckpt
+**/restoration/codeformer/weights
+
+# ignore user models config
+configs/models.user.yaml
+config/models.user.yml
+
+# ignore the Anaconda/Miniconda installer used while building Docker image
+anaconda.sh
+
+# ignore a directory which serves as a place for initial images
+inputs/
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# emacs autosave and recovery files
+*~
+.#*
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+src
+**/__pycache__/
+outputs
+
+# Logs and associated folders
+# created from generated embeddings.
+logs
+testtube
+checkpoints
+# If it's a Mac
+.DS_Store
+
+# Let the frontend manage its own gitignore
+!frontend/*
+
+# Scratch folder
+.scratch/
+.vscode/
+gfpgan/
+models/ldm/stable-diffusion-v1/*.sha256
+
+
+# GFPGAN model files
+gfpgan/
+
+# config file (will be created by installer)
+configs/models.yaml
+
+# weights (will be created by installer)
+models/ldm/stable-diffusion-v1/*.ckpt
+models/clipseg
+models/gfpgan
+
+# ignore initfile
+.invokeai
+
+# ignore environment.yml and requirements.txt
+# these are links to the real files in environments-and-requirements
+environment.yml
+requirements.txt
+
+# source installer files
+source_installer/*zip
+source_installer/invokeAI
+install.bat
+install.sh
+update.bat
+update.sh
+
+# this may be present if the user created a venv
+invokeai
+
+# no longer stored in source directory
+models

.prettierrc.yaml

@@ -0,0 +1,13 @@
+endOfLine: lf
+tabWidth: 2
+useTabs: false
+singleQuote: true
+quoteProps: as-needed
+embeddedLanguageFormatting: auto
+overrides:
+  - files: '*.md'
+    options:
+      proseWrap: always
+      printWidth: 80
+      parser: markdown
+      cursorOffset: -1

CODE_OF_CONDUCT.md

@@ -0,0 +1,128 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior
+may be reported to the community leaders responsible for enforcement
+at https://github.com/invoke-ai/InvokeAI/issues.  All complaints will
+be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
+
+Community Impact Guidelines were inspired by [Mozilla's code of conduct
+enforcement ladder](https://github.com/mozilla/diversity).
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see the FAQ at
+https://www.contributor-covenant.org/faq. Translations are available at
+https://www.contributor-covenant.org/translations.

InvokeAI_Statement_of_Values.md

@@ -0,0 +1,85 @@
+<img src="docs/assets/invoke_ai_banner.png" align="center">
+
+Invoke-AI is a community of software developers, researchers, and user
+interface experts who have come together on a voluntary basis to build
+software tools which support cutting edge AI text-to-image
+applications. This community is open to anyone who wishes to
+contribute to the effort and has the skill and time to do so.
+
+# Our Values
+
+The InvokeAI team is a diverse community which includes individuals
+from various parts of the world and many walks of life. Despite our
+differences, we share a number of core values which we ask prospective
+contributors to understand and respect. We believe:
+
+1. That Open Source Software is a positive force in the world. We
+create software that can be used, reused, and redistributed, without
+restrictions, under a straightforward Open Source license (MIT). We
+believe that Open Source benefits society as a whole by increasing the
+availability of high quality software to all.
+
+2. That those who create software should receive proper attribution
+for their creative work. While we support the exchange and reuse of
+Open Source Software, we feel strongly that the original authors of a
+piece of code should receive credit for their contribution, and we
+endeavor to do so whenever possible.
+
+3. That there is moral ambiguity surrounding AI-assisted art. We are
+aware of the moral and ethical issues surrounding the release of the
+Stable Diffusion model and similar products. We are aware that, due to
+the composition of their training sets, current AI-generated image
+models are biased against certain ethnic groups, cultural concepts of
+beauty, ethnic stereotypes, and gender roles.
+
+      1. We recognize the potential for harm to these groups that these biases
+       represent and trust that future AI models will take steps towards
+       reducing or eliminating the biases noted above, respect and give due
+       credit to the artists whose work is sourced, and call on developers
+       and users to favor these models over the older ones as they become
+       available.
+
+4. We are deeply committed to ensuring that this technology benefits
+everyone, including artists. We see AI art not as a replacement for
+the artist, but rather as a tool to empower them. With that
+in mind, we are constantly debating how to build systems that put
+artists’ needs first: tools which can be readily integrated into an
+artist’s existing workflows and practices, enhancing their work and
+helping them to push it further. Every decision we take as a team,
+which includes several artists, aims to build towards that goal.
+
+5. That artificial intelligence can be a force for good in the world,
+but must be used responsibly. Artificial intelligence technologies
+have the potential to improve society, in everything from cancer care,
+to customer service, to creative writing.
+
+     1. While we do not believe that software should arbitrarily limit what
+     users can do with it, we recognize that when used irresponsibly, AI
+     has the potential to do much harm. Our Discord server is actively
+     moderated in order to minimize the potential of harm from
+     user-contributed images. In addition, we ask users of our software to
+     refrain from using it in any way that would cause mental, emotional or
+     physical harm to individuals and vulnerable populations including (but
+     not limited to) women; minors; ethnic minorities; religious groups;
+     members of LGBTQIA communities; and people with disabilities or
+     impairments.
+
+     2. Note that some of the image generation AI models which the Invoke-AI
+     toolkit supports carry licensing agreements which impose restrictions
+     on how the model is used. We ask that our users read and agree to
+     these terms if they wish to make use of these models. These agreements
+     are distinct from the MIT license which applies to the InvokeAI
+     software and source code.
+
+6. That mutual respect is key to a healthy software development
+community. Members of the InvokeAI community are expected to treat
+each other with respect, beneficence, and empathy. Each of us has a
+different background and a unique set of skills. We strive to help
+each other grow and gain new skills, and we apportion expectations in
+a way that balances the members' time, skillset, and interest
+area. Disputes are resolved by open and honest communication.
+
+## Signature
+
+This document has been collectively crafted and approved by the current InvokeAI team members, as of 28 Nov 2022: **lstein** (Lincoln Stein), **blessedcoolant**, **hipsterusername** (Kent Keirsey), **Kyle0654** (Kyle Schouviller), **damian0815**, **mauwii** (Matthias Wild), **Netsvetaev** (Artur Netsvetaev), **psychedelicious**, **tildebyte**, and **keturn**. Although individuals within the group may hold differing views on particular details and/or their implications, we are all in agreement about its fundamental statements, as well as their significance and importance to this project moving forward.  
+

LICENSE

@@ -1,9 +1,16 @@
-All rights reserved by the authors.
-You must not distribute the weights provided to you directly or indirectly without explicit consent of the authors.
-You must not distribute harmful, offensive, dehumanizing content or otherwise harmful representations of people or their environments, cultures, religions, etc. produced with the model weights
-or other generated content described in the "Misuse and Malicious Use" section in the model card.
-The model weights are provided for research purposes only.
+MIT License
 
+Copyright (c) 2022 InvokeAI Team
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
 
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
@@ -11,4 +18,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+SOFTWARE.

LICENSE-ModelWeights.txt

@@ -0,0 +1,294 @@
+Copyright (c) 2022 Robin Rombach and Patrick Esser and contributors
+
+CreativeML Open RAIL-M
+dated August 22, 2022
+
+Section I: PREAMBLE
+
+Multimodal generative models are being widely adopted and used, and
+have the potential to transform the way artists, among other
+individuals, conceive and benefit from AI or ML technologies as a tool
+for content creation.
+
+Notwithstanding the current and potential benefits that these
+artifacts can bring to society at large, there are also concerns about
+potential misuses of them, either due to their technical limitations
+or ethical considerations.
+
+In short, this license strives for both the open and responsible
+downstream use of the accompanying model. When it comes to the open
+character, we took inspiration from open source permissive licenses
+regarding the grant of IP rights. Referring to the downstream
+responsible use, we added use-based restrictions not permitting the
+use of the Model in very specific scenarios, in order for the licensor
+to be able to enforce the license in case potential misuses of the
+Model may occur. At the same time, we strive to promote open and
+responsible research on generative models for art and content
+generation.
+
+Even though downstream derivative versions of the model could be
+released under different licensing terms, the latter will always have
+to include - at minimum - the same use-based restrictions as the ones
+in the original license (this license). We believe in the intersection
+between open and responsible AI development; thus, this License aims
+to strike a balance between both in order to enable responsible
+open-science in the field of AI.
+
+This License governs the use of the model (and its derivatives) and is
+informed by the model card associated with the model.
+
+NOW THEREFORE, You and Licensor agree as follows:
+
+1. Definitions
+
+- "License" means the terms and conditions for use, reproduction, and
+  Distribution as defined in this document.
+
+- "Data" means a collection of information and/or content extracted
+  from the dataset used with the Model, including to train, pretrain,
+  or otherwise evaluate the Model. The Data is not licensed under this
+  License.
+
+- "Output" means the results of operating a Model as embodied in
+  informational content resulting therefrom.
+
+- "Model" means any accompanying machine-learning based assemblies
+  (including checkpoints), consisting of learnt weights, parameters
+  (including optimizer states), corresponding to the model
+  architecture as embodied in the Complementary Material, that have
+  been trained or tuned, in whole or in part on the Data, using the
+  Complementary Material.
+
+- "Derivatives of the Model" means all modifications to the Model,
+  works based on the Model, or any other model which is created or
+  initialized by transfer of patterns of the weights, parameters,
+  activations or output of the Model, to the other model, in order to
+  cause the other model to perform similarly to the Model, including -
+  but not limited to - distillation methods entailing the use of
+  intermediate data representations or methods based on the generation
+  of synthetic data by the Model for training the other model.
+
+- "Complementary Material" means the accompanying source code and
+  scripts used to define, run, load, benchmark or evaluate the Model,
+  and used to prepare data for training or evaluation, if any. This
+  includes any accompanying documentation, tutorials, examples, etc,
+  if any.
+
+- "Distribution" means any transmission, reproduction, publication or
+  other sharing of the Model or Derivatives of the Model to a third
+  party, including providing the Model as a hosted service made
+  available by electronic or other remote means - e.g. API-based or
+  web access.
+
+- "Licensor" means the copyright owner or entity authorized by the
+  copyright owner that is granting the License, including the persons
+  or entities that may have rights in the Model and/or distributing
+  the Model.
+
+- "You" (or "Your") means an individual or Legal Entity exercising
+  permissions granted by this License and/or making use of the Model
+  for whichever purpose and in any field of use, including usage of
+  the Model in an end-use application - e.g. chatbot, translator,
+  image generator.
+
+- "Third Parties" means individuals or legal entities that are not
+  under common control with Licensor or You.
+
+- "Contribution" means any work of authorship, including the original
+  version of the Model and any modifications or additions to that
+  Model or Derivatives of the Model thereof, that is intentionally
+  submitted to Licensor for inclusion in the Model by the copyright
+  owner or by an individual or Legal Entity authorized to submit on
+  behalf of the copyright owner. For the purposes of this definition,
+  "submitted" means any form of electronic, verbal, or written
+  communication sent to the Licensor or its representatives, including
+  but not limited to communication on electronic mailing lists, source
+  code control systems, and issue tracking systems that are managed
+  by, or on behalf of, the Licensor for the purpose of discussing and
+  improving the Model, but excluding communication that is
+  conspicuously marked or otherwise designated in writing by the
+  copyright owner as "Not a Contribution."
+
+- "Contributor" means Licensor and any individual or Legal Entity on
+  behalf of whom a Contribution has been received by Licensor and
+  subsequently incorporated within the Model.
+
+Section II: INTELLECTUAL PROPERTY RIGHTS
+
+Both copyright and patent grants apply to the Model, Derivatives of
+the Model and Complementary Material. The Model and Derivatives of the
+Model are subject to additional terms as described in Section III.
+
+2. Grant of Copyright License. Subject to the terms and conditions of
+this License, each Contributor hereby grants to You a perpetual,
+worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+copyright license to reproduce, prepare, publicly display, publicly
+perform, sublicense, and distribute the Complementary Material, the
+Model, and Derivatives of the Model.
+
+3. Grant of Patent License. Subject to the terms and conditions of
+this License and where and as applicable, each Contributor hereby
+grants to You a perpetual, worldwide, non-exclusive, no-charge,
+royalty-free, irrevocable (except as stated in this paragraph) patent
+license to make, have made, use, offer to sell, sell, import, and
+otherwise transfer the Model and the Complementary Material, where
+such license applies only to those patent claims licensable by such
+Contributor that are necessarily infringed by their Contribution(s)
+alone or by combination of their Contribution(s) with the Model to
+which such Contribution(s) was submitted. If You institute patent
+litigation against any entity (including a cross-claim or counterclaim
+in a lawsuit) alleging that the Model and/or Complementary Material or
+a Contribution incorporated within the Model and/or Complementary
+Material constitutes direct or contributory patent infringement, then
+any patent licenses granted to You under this License for the Model
+and/or Work shall terminate as of the date such litigation is asserted
+or filed.
+
+Section III: CONDITIONS OF USAGE, DISTRIBUTION AND REDISTRIBUTION
+
+4. Distribution and Redistribution. You may host for Third Party
+remote access purposes (e.g. software-as-a-service), reproduce and
+distribute copies of the Model or Derivatives of the Model thereof in
+any medium, with or without modifications, provided that You meet the
+following conditions: Use-based restrictions as referenced in
+paragraph 5 MUST be included as an enforceable provision by You in any
+type of legal agreement (e.g. a license) governing the use and/or
+distribution of the Model or Derivatives of the Model, and You shall
+give notice to subsequent users You Distribute to, that the Model or
+Derivatives of the Model are subject to paragraph 5. This provision
+does not apply to the use of Complementary Material.  You must give
+any Third Party recipients of the Model or Derivatives of the Model a
+copy of this License; You must cause any modified files to carry
+prominent notices stating that You changed the files; You must retain
+all copyright, patent, trademark, and attribution notices excluding
+those notices that do not pertain to any part of the Model,
+Derivatives of the Model.  You may add Your own copyright statement to
+Your modifications and may provide additional or different license
+terms and conditions - respecting paragraph 4.a. - for use,
+reproduction, or Distribution of Your modifications, or for any such
+Derivatives of the Model as a whole, provided Your use, reproduction,
+and Distribution of the Model otherwise complies with the conditions
+stated in this License.
+
+5. Use-based restrictions. The restrictions set forth in Attachment A
+are considered Use-based restrictions. Therefore You cannot use the
+Model and the Derivatives of the Model for the specified restricted
+uses. You may use the Model subject to this License, including only
+for lawful purposes and in accordance with the License. Use may
+include creating any content with, finetuning, updating, running,
+training, evaluating and/or reparametrizing the Model. You shall
+require all of Your users who use the Model or a Derivative of the
+Model to comply with the terms of this paragraph (paragraph 5).
+
+6. The Output You Generate. Except as set forth herein, Licensor
+claims no rights in the Output You generate using the Model. You are
+accountable for the Output you generate and its subsequent uses. No
+use of the output can contravene any provision as stated in the
+License.
+
+Section IV: OTHER PROVISIONS
+
+7. Updates and Runtime Restrictions. To the maximum extent permitted
+by law, Licensor reserves the right to restrict (remotely or
+otherwise) usage of the Model in violation of this License, update the
+Model through electronic means, or modify the Output of the Model
+based on updates. You shall undertake reasonable efforts to use the
+latest version of the Model.
+
+8. Trademarks and related. Nothing in this License permits You to make
+use of Licensors’ trademarks, trade names, logos or to otherwise
+suggest endorsement or misrepresent the relationship between the
+parties; and any rights not expressly granted herein are reserved by
+the Licensors.
+
+9. Disclaimer of Warranty. Unless required by applicable law or agreed
+to in writing, Licensor provides the Model and the Complementary
+Material (and each Contributor provides its Contributions) on an "AS
+IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+express or implied, including, without limitation, any warranties or
+conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR
+A PARTICULAR PURPOSE. You are solely responsible for determining the
+appropriateness of using or redistributing the Model, Derivatives of
+the Model, and the Complementary Material and assume any risks
+associated with Your exercise of permissions under this License.
+
+10. Limitation of Liability. In no event and under no legal theory,
+whether in tort (including negligence), contract, or otherwise, unless
+required by applicable law (such as deliberate and grossly negligent
+acts) or agreed to in writing, shall any Contributor be liable to You
+for damages, including any direct, indirect, special, incidental, or
+consequential damages of any character arising as a result of this
+License or out of the use or inability to use the Model and the
+Complementary Material (including but not limited to damages for loss
+of goodwill, work stoppage, computer failure or malfunction, or any
+and all other commercial damages or losses), even if such Contributor
+has been advised of the possibility of such damages.
+
+11. Accepting Warranty or Additional Liability. While redistributing
+the Model, Derivatives of the Model and the Complementary Material
+thereof, You may choose to offer, and charge a fee for, acceptance of
+support, warranty, indemnity, or other liability obligations and/or
+rights consistent with this License. However, in accepting such
+obligations, You may act only on Your own behalf and on Your sole
+responsibility, not on behalf of any other Contributor, and only if
+You agree to indemnify, defend, and hold each Contributor harmless for
+any liability incurred by, or claims asserted against, such
+Contributor by reason of your accepting any such warranty or
+additional liability.
+
+12. If any provision of this License is held to be invalid, illegal or
+unenforceable, the remaining provisions shall be unaffected thereby
+and remain valid as if such provision had not been set forth herein.
+
+END OF TERMS AND CONDITIONS
+
+
+
+
+Attachment A
+
+Use Restrictions
+
+You agree not to use the Model or Derivatives of the Model:
+
+- In any way that violates any applicable national, federal, state,
+  local or international law or regulation;
+
+- For the purpose of exploiting, harming or attempting to exploit or
+  harm minors in any way;
+
+- To generate or disseminate verifiably false information and/or
+  content with the purpose of harming others;
+
+- To generate or disseminate personal identifiable information that
+  can be used to harm an individual;
+
+- To defame, disparage or otherwise harass others;
+
+- For fully automated decision making that adversely impacts an
+  individual’s legal rights or otherwise creates or modifies a
+  binding, enforceable obligation;
+
+pp- For any use intended to or which has the effect of discriminating
+  against or harming individuals or groups based on online or offline
+  social behavior or known or predicted personal or personality
+  characteristics;
+
+- To exploit any of the vulnerabilities of a specific group of persons
+  based on their age, social, physical or mental characteristics, in
+  order to materially distort the behavior of a person pertaining to
+  that group in a manner that causes or is likely to cause that person
+  or another person physical or psychological harm;
+
+- For any use intended to or which has the effect of discriminating
+  against individuals or groups based on legally protected
+  characteristics or categories;
+
+- To provide medical advice and medical results interpretation;
+
+- To generate or disseminate information for the purpose to be used
+  for administration of justice, law enforcement, immigration or
+  asylum processes, such as predicting an individual will commit
+  fraud/crime commitment (e.g. by text profiling, drawing causal
+  relationships between assertions made in documents, indiscriminate
+  and arbitrarily-targeted use).

README.md

@@ -1,513 +1,208 @@
-# Stable Diffusion Dream Script
+<div align="center">
 
-This is a fork of CompVis/stable-diffusion, the wonderful open source
-text-to-image generator.
+# InvokeAI: A Stable Diffusion Toolkit
 
-The original has been modified in several ways:
+_Formerly known as lstein/stable-diffusion_
 
-## Interactive command-line interface similar to the Discord bot
+![project logo](docs/assets/logo.png)
 
-The *dream.py* script, located in scripts/dream.py, 
-provides an interactive interface to image generation similar to
-the "dream mothership" bot that Stable AI provided on its Discord
-server. Unlike the txt2img.py and img2img.py scripts provided in the
-original CompViz/stable-diffusion source code repository, the
-time-consuming initialization of the AI model
-initialization only happens once. After that image generation 
-from the command-line interface is very fast.
+[![discord badge]][discord link]
 
-The script uses the readline library to allow for in-line editing,
-command history (up and down arrows), autocompletion, and more. To help
-keep track of which prompts generated which images, the script writes a
-log file of image names and prompts to the selected output directory.
-In addition, as of version 1.02, it also writes the prompt into the PNG
-file's metadata where it can be retrieved using scripts/images2prompt.py
+[![latest release badge]][latest release link] [![github stars badge]][github stars link] [![github forks badge]][github forks link]
 
-The script is confirmed to work on Linux and Windows systems. It should
-work on MacOSX as well, but this is not confirmed. Note that this script
-runs from the command-line (CMD or Terminal window), and does not have a GUI.
+[![CI checks on main badge]][CI checks on main link] [![CI checks on dev badge]][CI checks on dev link] [![latest commit to dev badge]][latest commit to dev link]
 
-~~~~
-(ldm) ~/stable-diffusion$ python3 ./scripts/dream.py
-* Initializing, be patient...
-Loading model from models/ldm/text2img-large/model.ckpt
-LatentDiffusion: Running in eps-prediction mode
-DiffusionWrapper has 872.30 M params.
-making attention of type 'vanilla' with 512 in_channels
-Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
-making attention of type 'vanilla' with 512 in_channels
-Loading Bert tokenizer from "models/bert"
-setting sampler to plms
+[![github open issues badge]][github open issues link] [![github open prs badge]][github open prs link]
 
-* Initialization done! Awaiting your command...
-dream> ashley judd riding a camel -n2 -s150
-Outputs:
-   outputs/img-samples/00009.png: "ashley judd riding a camel" -n2 -s150 -S 416354203
-   outputs/img-samples/00010.png: "ashley judd riding a camel" -n2 -s150 -S 1362479620
+[CI checks on dev badge]: https://flat.badgen.net/github/checks/invoke-ai/InvokeAI/development?label=CI%20status%20on%20dev&cache=900&icon=github
+[CI checks on dev link]: https://github.com/invoke-ai/InvokeAI/actions?query=branch%3Adevelopment
+[CI checks on main badge]: https://flat.badgen.net/github/checks/invoke-ai/InvokeAI/main?label=CI%20status%20on%20main&cache=900&icon=github
+[CI checks on main link]: https://github.com/invoke-ai/InvokeAI/actions/workflows/test-invoke-conda.yml
+[discord badge]: https://flat.badgen.net/discord/members/ZmtBAhwWhy?icon=discord
+[discord link]: https://discord.gg/ZmtBAhwWhy
+[github forks badge]: https://flat.badgen.net/github/forks/invoke-ai/InvokeAI?icon=github
+[github forks link]: https://useful-forks.github.io/?repo=invoke-ai%2FInvokeAI
+[github open issues badge]: https://flat.badgen.net/github/open-issues/invoke-ai/InvokeAI?icon=github
+[github open issues link]: https://github.com/invoke-ai/InvokeAI/issues?q=is%3Aissue+is%3Aopen
+[github open prs badge]: https://flat.badgen.net/github/open-prs/invoke-ai/InvokeAI?icon=github
+[github open prs link]: https://github.com/invoke-ai/InvokeAI/pulls?q=is%3Apr+is%3Aopen
+[github stars badge]: https://flat.badgen.net/github/stars/invoke-ai/InvokeAI?icon=github
+[github stars link]: https://github.com/invoke-ai/InvokeAI/stargazers
+[latest commit to dev badge]: https://flat.badgen.net/github/last-commit/invoke-ai/InvokeAI/development?icon=github&color=yellow&label=last%20dev%20commit&cache=900
+[latest commit to dev link]: https://github.com/invoke-ai/InvokeAI/commits/development
+[latest release badge]: https://flat.badgen.net/github/release/invoke-ai/InvokeAI/development?icon=github
+[latest release link]: https://github.com/invoke-ai/InvokeAI/releases
+</div>
 
-dream> "there's a fly in my soup" -n6 -g
-    outputs/img-samples/00011.png: "there's a fly in my soup" -n6 -g -S 2685670268
-    seeds for individual rows: [2685670268, 1216708065, 2335773498, 822223658, 714542046, 3395302430]
-dream> q
+This is a fork of
+[CompVis/stable-diffusion](https://github.com/CompVis/stable-diffusion),
+the open source text-to-image generator. It provides a streamlined
+process with various new features and options to aid the image
+generation process. It runs on Windows, Mac and Linux machines, with
+GPU cards with as little as 4 GB of RAM. It provides both a polished
+Web interface (see below), and an easy-to-use command-line interface.
 
-# this shows how to retrieve the prompt stored in the saved image's metadata
-(ldm) ~/stable-diffusion$ python3 ./scripts/images2prompt.py outputs/img_samples/*.png
-00009.png: "ashley judd riding a camel" -s150 -S 416354203
-00010.png: "ashley judd riding a camel" -s150 -S 1362479620
-00011.png: "there's a fly in my soup" -n6 -g -S 2685670268
-~~~~
+**Quick links**: [<a href="https://discord.gg/ZmtBAhwWhy">Discord Server</a>] [<a href="https://invoke-ai.github.io/InvokeAI/">Documentation and Tutorials</a>] [<a href="https://github.com/invoke-ai/InvokeAI/">Code and Downloads</a>] [<a href="https://github.com/invoke-ai/InvokeAI/issues">Bug Reports</a>] [<a href="https://github.com/invoke-ai/InvokeAI/discussions">Discussion, Ideas & Q&A</a>]
 
-The dream> prompt's arguments are pretty much
-identical to those used in the Discord bot, except you don't need to
-type "!dream" (it doesn't hurt if you do). A significant change is that creation of individual images 
-is now the default
-unless --grid (-g) is given. For backward compatibility, the -i switch is recognized.
-For command-line help type -h (or --help) at the dream> prompt.
-
-The script itself also recognizes a series of command-line switches that will change
-important global defaults, such as the directory for image outputs and the location
-of the model weight files.
-
-## Image-to-Image
-
-This script also provides an img2img feature that lets you seed your
-creations with a drawing or photo. This is a really cool feature that tells
-stable diffusion to build the prompt on top of the image you provide, preserving
-the original's basic shape and layout. To use it, provide the --init_img 
-option as shown here:
-
-~~~~
-dream> "waterfall and rainbow" --init_img=./init-images/crude_drawing.png --strength=0.5 -s100 -n4
-~~~~
-
-The --init_img (-I) option gives the path to the seed picture. --strength (-f) controls how much
-the original will be modified, ranging from 0.0 (keep the original intact), to 1.0 (ignore the original
-completely). The default is 0.75, and ranges from 0.25-0.75 give interesting results.
-
-## Changes
-
-* v1.03 (22 August 2022)
-   * The original txt2img and img2img scripts from the CompViz repository have been moved into
-     a subfolder named "orig_scripts", to reduce confusion.
-   
-* v1.02 (21 August 2022)
-    * A copy of the prompt and all of its switches and options is now stored in the corresponding
-    image in a tEXt metadata field named "Dream". You can read the prompt using scripts/images2prompt.py,
-    or an image editor that allows you to explore the full metadata.
-        **Please run "conda env update -f environment.yaml" to load the k_lms dependencies!!**
-
-* v1.01 (21 August 2022)
-    * added k_lms sampling. 
-      **Please run "conda env update -f environment.yaml" to load the k_lms dependencies!!**
-    * use half precision arithmetic by default, resulting in faster execution and lower memory requirements
-    Pass argument --full_precision to dream.py to get slower but more accurate image generation
+<div align="center"><img src="docs/assets/invoke-web-server-1.png" width=640></div>
 
 
-## Installation
+_Note: This fork is rapidly evolving. Please use the
+[Issues](https://github.com/invoke-ai/InvokeAI/issues) tab to report bugs and make feature
+requests. Be sure to use the provided templates. They will help aid diagnose issues faster._
 
-### Linux/Mac
+## Table of Contents
 
-1. You will need to install the following prerequisites if they are not already available. Use your
-operating system's preferred installer
-* Python (version 3.8.5 recommended; higher may work)
-* git
+1. [Installation](#installation)
+2. [Hardware Requirements](#hardware-requirements)
+3. [Features](#features)
+4. [Latest Changes](#latest-changes)
+5. [Troubleshooting](#troubleshooting)
+6. [Contributing](#contributing)
+7. [Contributors](#contributors)
+8. [Support](#support)
+9. [Further Reading](#further-reading)
 
-2. Install the Python Anaconda environment manager using pip3.
-```
-~$ pip3 install anaconda
-```
-After installing anaconda, you should log out of your system and log back in. If the installation
-worked, your command prompt will be prefixed by the name of the current anaconda environment, "(base)".
+### Installation
 
-3. Copy the stable-diffusion source code from GitHub:
-```
-(base) ~$ git clone https://github.com/lstein/stable-diffusion.git
-```
-This will create stable-diffusion folder where you will follow the rest of the steps.
+This fork is supported across Linux, Windows and Macintosh. Linux
+users can use either an Nvidia-based card (with CUDA support) or an
+AMD card (using the ROCm driver). For full installation and upgrade
+instructions, please see:
+[InvokeAI Installation Overview](https://invoke-ai.github.io/InvokeAI/installation/)
 
-4. Enter the newly-created stable-diffusion folder. From this step forward make sure that you are working in the stable-diffusion directory!
-```
-(base) ~$ cd stable-diffusion
-(base) ~/stable-diffusion$
-```
-5. Use anaconda to copy necessary python packages, create a new python environment named "ldm",
-and activate the environment. 
-```
-(base) ~/stable-diffusion$ conda env create -f environment.yaml
-(base) ~/stable-diffusion$ conda activate ldm
-(ldm) ~/stable-diffusion$
-```
-After these steps, your command prompt will be prefixed by "(ldm)" as shown above.
+### Hardware Requirements
 
-6. Load a couple of small machine-learning models required by stable diffusion:
-```
-(ldm) ~/stable-diffusion$ python3 scripts/preload_models.py
+#### System
+
+You wil need one of the following:
+
+- An NVIDIA-based graphics card with 4 GB or more VRAM memory.
+- An Apple computer with an M1 chip.
+
+#### Memory
+
+- At least 12 GB Main Memory RAM.
+
+#### Disk
+
+- At least 12 GB of free disk space for the machine learning model, Python, and all its dependencies.
+
+**Note**
+
+If you have a Nvidia 10xx series card (e.g. the 1080ti), please
+run the dream script in full-precision mode as shown below.
+
+Similarly, specify full-precision mode on Apple M1 hardware.
+
+Precision is auto configured based on the device. If however you encounter
+errors like 'expected type Float but found Half' or 'not implemented for Half'
+you can try starting `invoke.py` with the `--precision=float32` flag:
+
+```bash
+(invokeai) ~/InvokeAI$ python scripts/invoke.py --precision=float32
 ```
 
-7. Now you need to install the weights for the stable diffusion model.
-
-For testing prior to the release of the real weights, you can use an older weight file that produces low-quality images. Create a directory within stable-diffusion named "models/ldm/text2img-large", and use the wget URL downloader tool to copy the weight file into it:
-```
-(ldm) ~/stable-diffusion$ mkdir -p models/ldm/text2img-large
-(ldm) ~/stable-diffusion$ wget -O models/ldm/text2img-large/model.ckpt https://ommer-lab.com/files/latent-diffusion/nitro/txt2img-f8-large/model.ckpt
-```
-For testing with the released weighs, you will do something similar, but with a directory named "models/ldm/stable-diffusion-v1"
-```
-(ldm) ~/stable-diffusion$ mkdir -p models/ldm/stable-diffusion-v1
-(ldm) ~/stable-diffusion$ wget -O models/ldm/stable-diffusion-v1/model.ckpt <ENTER URL HERE>
-```
-These weight files are ~5 GB in size, so downloading may take a while.
-
-8. Start generating images!
-```
-# for the pre-release weights use the -l or --liaon400m switch
-(ldm) ~/stable-diffusion$ python3 scripts/dream.py -l
-
-# for the post-release weights do not use the switch
-(ldm) ~/stable-diffusion$ python3 scripts/dream.py
-
-# for additional configuration switches and arguments, use -h or --help
-(ldm) ~/stable-diffusion$ python3 scripts/dream.py -h
-```
-9. Subsequently, to relaunch the script, be sure to run "conda activate ldm" (step 5, second command), enter the "stable-diffusion" 
-directory, and then launch the dream script (step 8). If you forget to activate the ldm environment, the script will fail with multiple ModuleNotFound errors.
-
-### Updating to newer versions of the script
-
-This distribution is changing rapidly. If you used the "git clone" method (step 5) to download the stable-diffusion directory, then to update to the latest and greatest version, launch the Anaconda window, enter "stable-diffusion", and type:
-```
-(ldm) ~/stable-diffusion$ git pull
-```
-This will bring your local copy into sync with the remote one.
-
-### Windows
-
-1. Install Python version 3.8.5 from here: https://www.python.org/downloads/windows/
-   (note that several users have reported that later versions do not work properly)
-
-2. Install Anaconda3 (miniconda3 version) from here: https://docs.anaconda.com/anaconda/install/windows/
-
-3. Install Git from here: https://git-scm.com/download/win
-
-4. Launch Anaconda from the Windows Start menu. This will bring up a command window. Type all the remaining commands in this window.
-
-5. Run the command:
-```
-git clone https://github.com/lstein/stable-diffusion.git
-```
-This will create stable-diffusion folder where you will follow the rest of the steps.
-
-6. Enter the newly-created stable-diffusion folder. From this step forward make sure that you are working in the stable-diffusion directory!
-```
-cd stable-diffusion
-```
-
-7. Run the following two commands:
-```
-conda env create -f environment.yaml    (step 7a)
-conda activate ldm                      (step 7b)
-```
-This will install all python requirements and activate the "ldm" environment which sets PATH and other environment variables properly.
-
-8. Run the command:
-```
-python scripts\preload_models.py
-```
-This installs two machine learning models that stable diffusion requires.
-
-9. Now you need to install the weights for the big stable diffusion model.
-
-For testing prior to the release of the real weights, create a directory within stable-diffusion named "models\ldm\text2img-large".
-
-For testing with the released weights, create a directory within stable-diffusion named "models\ldm\stable-diffusion-v1".
-
-Then use a web browser to copy model.ckpt into the appropriate directory. For the text2img-large (pre-release) model, the weights are at https://ommer-lab.com/files/latent-diffusion/nitro/txt2img-f8-large/model.ckpt. Check back here later for the release URL.
-
-10. Start generating images!
-```
-# for the pre-release weights
-python scripts\dream.py -l
-
-# for the post-release weights
-python scripts\dream.py
-```
-11. Subsequently, to relaunch the script, first activate the Anaconda command window (step 4), enter the stable-diffusion directory (step 6, "cd \path\to\stable-diffusion"), run "conda activate ldm" (step 7b), and then launch the dream script (step 10).
-
-### Updating to newer versions of the script
-
-This distribution is changing rapidly. If you used the "git clone" method (step 5) to download the stable-diffusion directory, then to update to the latest and greatest version, launch the Anaconda window, enter "stable-diffusion", and type:
-```
-git pull
-```
-This will bring your local copy into sync with the remote one.
-
-## Simplified API for text to image generation
-
-For programmers who wish to incorporate stable-diffusion into other
-products, this repository includes a simplified API for text to image generation, which
-lets you create images from a prompt in just three lines of code:
-
-~~~~
-from ldm.simplet2i import T2I
-model   = T2I()
-outputs = model.text2image("a unicorn in manhattan")
-~~~~
-
-Outputs is a list of lists in the format [[filename1,seed1],[filename2,seed2]...]
-Please see ldm/simplet2i.py for more information.
-
-
-## Workaround for machines with limited internet connectivity
-
-My development machine is a GPU node in a high-performance compute
-cluster which has no connection to the internet. During model
-initialization, stable-diffusion tries to download the Bert tokenizer
-and a file needed by the kornia library.  This obviously didn't work
-for me.
-
-To work around this, I have modified ldm/modules/encoders/modules.py
-to look for locally cached Bert files rather than attempting to
-download them. For this to work, you must run
-"scripts/preload_models.py" once from an internet-connected machine
-prior to running the code on an isolated one. This assumes that both
-machines share a common network-mounted filesystem with a common
-.cache directory.
-
-~~~~
-(ldm) ~/stable-diffusion$ python3 ./scripts/preload_models.py
-preloading bert tokenizer...
-Downloading: 100%|██████████████████████████████████| 28.0/28.0 [00:00<00:00, 49.3kB/s]
-Downloading: 100%|██████████████████████████████████| 226k/226k [00:00<00:00, 2.79MB/s]
-Downloading: 100%|██████████████████████████████████| 455k/455k [00:00<00:00, 4.36MB/s]
-Downloading: 100%|██████████████████████████████████| 570/570 [00:00<00:00, 477kB/s]
-...success
-preloading kornia requirements...
-Downloading: "https://github.com/DagnyT/hardnet/raw/master/pretrained/train_liberty_with_aug/checkpoint_liberty_with_aug.pth" to /u/lstein/.cache/torch/hub/checkpoints/checkpoint_liberty_with_aug.pth
-100%|███████████████████████████████████████████████| 5.10M/5.10M [00:00<00:00, 101MB/s]
-...success
-~~~~
-
-If you don't need this change and want to download the files just in
-time, copy over the file ldm/modules/encoders/modules.py from the
-CompVis/stable-diffusion repository. Or you can run preload_models.py
-on the target machine.
-
-## Support
-
-For support,
-please use this repository's GitHub Issues tracking service. Feel free
-to send me an email if you use and like the script.
-
-*Original Author:* Lincoln D. Stein <lincoln.stein@gmail.com>
-
-*Contributions by:* [Peter Kowalczyk](https://github.com/slix), [Henry Harrison](https://github.com/hwharrison), [xraxra](https://github.com/xraxra), and [bmaltais](https://github.com/bmaltais)
-
-# Original README from CompViz/stable-diffusion
-*Stable Diffusion was made possible thanks to a collaboration with [Stability AI](https://stability.ai/) and [Runway](https://runwayml.com/) and builds upon our previous work:*
-
-[**High-Resolution Image Synthesis with Latent Diffusion Models**](https://ommer-lab.com/research/latent-diffusion-models/)<br/>
-[Robin Rombach](https://github.com/rromb)\*,
-[Andreas Blattmann](https://github.com/ablattmann)\*,
-[Dominik Lorenz](https://github.com/qp-qp)\,
-[Patrick Esser](https://github.com/pesser),
-[Björn Ommer](https://hci.iwr.uni-heidelberg.de/Staff/bommer)<br/>
-
-**CVPR '22 Oral**
-
-which is available on [GitHub](https://github.com/CompVis/latent-diffusion). PDF at [arXiv](https://arxiv.org/abs/2112.10752). Please also visit our [Project page](https://ommer-lab.com/research/latent-diffusion-models/).
-
-![txt2img-stable2](assets/stable-samples/txt2img/merged-0006.png)
-[Stable Diffusion](#stable-diffusion-v1) is a latent text-to-image diffusion
-model.
-Thanks to a generous compute donation from [Stability AI](https://stability.ai/) and support from [LAION](https://laion.ai/), we were able to train a Latent Diffusion Model on 512x512 images from a subset of the [LAION-5B](https://laion.ai/blog/laion-5b/) database. 
-Similar to Google's [Imagen](https://arxiv.org/abs/2205.11487), 
-this model uses a frozen CLIP ViT-L/14 text encoder to condition the model on text prompts.
-With its 860M UNet and 123M text encoder, the model is relatively lightweight and runs on a GPU with at least 10GB VRAM.
-See [this section](#stable-diffusion-v1) below and the [model card](https://huggingface.co/CompVis/stable-diffusion).
-
-  
-## Requirements
-
-A suitable [conda](https://conda.io/) environment named `ldm` can be created
-and activated with:
-
-```
-conda env create -f environment.yaml
-conda activate ldm
-```
-
-You can also update an existing [latent diffusion](https://github.com/CompVis/latent-diffusion) environment by running
-
-```
-conda install pytorch torchvision -c pytorch
-pip install transformers==4.19.2
-pip install -e .
-```
-
-## Stable Diffusion v1
-
-Stable Diffusion v1 refers to a specific configuration of the model
-architecture that uses a downsampling-factor 8 autoencoder with an 860M UNet
-and CLIP ViT-L/14 text encoder for the diffusion model. The model was pretrained on 256x256 images and 
-then finetuned on 512x512 images.
-
-*Note: Stable Diffusion v1 is a general text-to-image diffusion model and therefore mirrors biases and (mis-)conceptions that are present
-in its training data. 
-Details on the training procedure and data, as well as the intended use of the model can be found in the corresponding [model card](https://huggingface.co/CompVis/stable-diffusion).
-Research into the safe deployment of general text-to-image models is an ongoing effort. To prevent misuse and harm, we currently provide access to the checkpoints only for [academic research purposes upon request](https://stability.ai/academia-access-form).
-**This is an experiment in safe and community-driven publication of a capable and general text-to-image model. We are working on a public release with a more permissive license that also incorporates ethical considerations.***
-
-[Request access to Stable Diffusion v1 checkpoints for academic research](https://stability.ai/academia-access-form) 
-
-### Weights
-
-We currently provide three checkpoints, `sd-v1-1.ckpt`, `sd-v1-2.ckpt` and `sd-v1-3.ckpt`,
-which were trained as follows,
-
-- `sd-v1-1.ckpt`: 237k steps at resolution `256x256` on [laion2B-en](https://huggingface.co/datasets/laion/laion2B-en).
-  194k steps at resolution `512x512` on [laion-high-resolution](https://huggingface.co/datasets/laion/laion-high-resolution) (170M examples from LAION-5B with resolution `>= 1024x1024`).
-- `sd-v1-2.ckpt`: Resumed from `sd-v1-1.ckpt`.
-  515k steps at resolution `512x512` on "laion-improved-aesthetics" (a subset of laion2B-en,
-filtered to images with an original size `>= 512x512`, estimated aesthetics score `> 5.0`, and an estimated watermark probability `< 0.5`. The watermark estimate is from the LAION-5B metadata, the aesthetics score is estimated using an [improved aesthetics estimator](https://github.com/christophschuhmann/improved-aesthetic-predictor)).
-- `sd-v1-3.ckpt`: Resumed from `sd-v1-2.ckpt`. 195k steps at resolution `512x512` on "laion-improved-aesthetics" and 10\% dropping of the text-conditioning to improve [classifier-free guidance sampling](https://arxiv.org/abs/2207.12598).
-
-Evaluations with different classifier-free guidance scales (1.5, 2.0, 3.0, 4.0,
-5.0, 6.0, 7.0, 8.0) and 50 PLMS sampling
-steps show the relative improvements of the checkpoints:
-![sd evaluation results](assets/v1-variants-scores.jpg)
-
-
-
-### Text-to-Image with Stable Diffusion
-![txt2img-stable2](assets/stable-samples/txt2img/merged-0005.png)
-![txt2img-stable2](assets/stable-samples/txt2img/merged-0007.png)
-
-Stable Diffusion is a latent diffusion model conditioned on the (non-pooled) text embeddings of a CLIP ViT-L/14 text encoder.
-
-
-#### Sampling Script
-
-After [obtaining the weights](#weights), link them
-```
-mkdir -p models/ldm/stable-diffusion-v1/
-ln -s <path/to/model.ckpt> models/ldm/stable-diffusion-v1/model.ckpt 
-```
-and sample with
-```
-python scripts/txt2img.py --prompt "a photograph of an astronaut riding a horse" --plms 
-```
-
-By default, this uses a guidance scale of `--scale 7.5`, [Katherine Crowson's implementation](https://github.com/CompVis/latent-diffusion/pull/51) of the [PLMS](https://arxiv.org/abs/2202.09778) sampler, 
-and renders images of size 512x512 (which it was trained on) in 50 steps. All supported arguments are listed below (type `python scripts/txt2img.py --help`).
-
-```commandline
-usage: txt2img.py [-h] [--prompt [PROMPT]] [--outdir [OUTDIR]] [--skip_grid] [--skip_save] [--ddim_steps DDIM_STEPS] [--plms] [--laion400m] [--fixed_code] [--ddim_eta DDIM_ETA] [--n_iter N_ITER] [--H H] [--W W] [--C C] [--f F] [--n_samples N_SAMPLES] [--n_rows N_ROWS]
-                  [--scale SCALE] [--from-file FROM_FILE] [--config CONFIG] [--ckpt CKPT] [--seed SEED] [--precision {full,autocast}]
-
-optional arguments:
-  -h, --help            show this help message and exit
-  --prompt [PROMPT]     the prompt to render
-  --outdir [OUTDIR]     dir to write results to
-  --skip_grid           do not save a grid, only individual samples. Helpful when evaluating lots of samples
-  --skip_save           do not save individual samples. For speed measurements.
-  --ddim_steps DDIM_STEPS
-                        number of ddim sampling steps
-  --plms                use plms sampling
-  --laion400m           uses the LAION400M model
-  --fixed_code          if enabled, uses the same starting code across samples
-  --ddim_eta DDIM_ETA   ddim eta (eta=0.0 corresponds to deterministic sampling
-  --n_iter N_ITER       sample this often
-  --H H                 image height, in pixel space
-  --W W                 image width, in pixel space
-  --C C                 latent channels
-  --f F                 downsampling factor
-  --n_samples N_SAMPLES
-                        how many samples to produce for each given prompt. A.k.a. batch size
-  --n_rows N_ROWS       rows in the grid (default: n_samples)
-  --scale SCALE         unconditional guidance scale: eps = eps(x, empty) + scale * (eps(x, cond) - eps(x, empty))
-  --from-file FROM_FILE
-                        if specified, load prompts from this file
-  --config CONFIG       path to config which constructs model
-  --ckpt CKPT           path to checkpoint of model
-  --seed SEED           the seed (for reproducible sampling)
-  --precision {full,autocast}
-                        evaluate at this precision
-
-```
-Note: The inference config for all v1 versions is designed to be used with EMA-only checkpoints. 
-For this reason `use_ema=False` is set in the configuration, otherwise the code will try to switch from
-non-EMA to EMA weights. If you want to examine the effect of EMA vs no EMA, we provide "full" checkpoints
-which contain both types of weights. For these, `use_ema=False` will load and use the non-EMA weights.
-
-
-#### Diffusers Integration
-
-Another way to download and sample Stable Diffusion is by using the [diffusers library](https://github.com/huggingface/diffusers/tree/main#new--stable-diffusion-is-now-fully-compatible-with-diffusers)
-```py
-# make sure you're logged in with `huggingface-cli login`
-from torch import autocast
-from diffusers import StableDiffusionPipeline, LMSDiscreteScheduler
-
-pipe = StableDiffusionPipeline.from_pretrained(
-	"CompVis/stable-diffusion-v1-3-diffusers", 
-	use_auth_token=True
-)
-
-prompt = "a photo of an astronaut riding a horse on mars"
-with autocast("cuda"):
-    image = pipe(prompt)["sample"][0]  
-    
-image.save("astronaut_rides_horse.png")
-```
-
-
-
-### Image Modification with Stable Diffusion
-
-By using a diffusion-denoising mechanism as first proposed by [SDEdit](https://arxiv.org/abs/2108.01073), the model can be used for different 
-tasks such as text-guided image-to-image translation and upscaling. Similar to the txt2img sampling script, 
-we provide a script to perform image modification with Stable Diffusion.  
-
-The following describes an example where a rough sketch made in [Pinta](https://www.pinta-project.com/) is converted into a detailed artwork.
-```
-python scripts/img2img.py --prompt "A fantasy landscape, trending on artstation" --init-img <path-to-img.jpg> --strength 0.8
-```
-Here, strength is a value between 0.0 and 1.0, that controls the amount of noise that is added to the input image. 
-Values that approach 1.0 allow for lots of variations but will also produce images that are not semantically consistent with the input. See the following example.
-
-**Input**
-
-![sketch-in](assets/stable-samples/img2img/sketch-mountains-input.jpg)
-
-**Outputs**
-
-![out3](assets/stable-samples/img2img/mountains-3.png)
-![out2](assets/stable-samples/img2img/mountains-2.png)
-
-This procedure can, for example, also be used to upscale samples from the base model.
-
-
-## Comments 
-
-- Our codebase for the diffusion models builds heavily on [OpenAI's ADM codebase](https://github.com/openai/guided-diffusion)
-and [https://github.com/lucidrains/denoising-diffusion-pytorch](https://github.com/lucidrains/denoising-diffusion-pytorch). 
-Thanks for open-sourcing!
-
-- The implementation of the transformer encoder is from [x-transformers](https://github.com/lucidrains/x-transformers) by [lucidrains](https://github.com/lucidrains?tab=repositories). 
-
-
-## BibTeX
-
-```
-@misc{rombach2021highresolution,
-      title={High-Resolution Image Synthesis with Latent Diffusion Models}, 
-      author={Robin Rombach and Andreas Blattmann and Dominik Lorenz and Patrick Esser and Björn Ommer},
-      year={2021},
-      eprint={2112.10752},
-      archivePrefix={arXiv},
-      primaryClass={cs.CV}
-}
-
-```
-
-
+### Features
+
+#### Major Features
+
+- [Web Server](https://invoke-ai.github.io/InvokeAI/features/WEB/)
+- [Interactive Command Line Interface](https://invoke-ai.github.io/InvokeAI/features/CLI/)
+- [Image To Image](https://invoke-ai.github.io/InvokeAI/features/IMG2IMG/)
+- [Inpainting Support](https://invoke-ai.github.io/InvokeAI/features/INPAINTING/)
+- [Outpainting Support](https://invoke-ai.github.io/InvokeAI/features/OUTPAINTING/)
+- [Upscaling, face-restoration and outpainting](https://invoke-ai.github.io/InvokeAI/features/POSTPROCESS/)
+- [Reading Prompts From File](https://invoke-ai.github.io/InvokeAI/features/PROMPTS/#reading-prompts-from-a-file)
+- [Prompt Blending](https://invoke-ai.github.io/InvokeAI/features/PROMPTS/#prompt-blending)
+- [Thresholding and Perlin Noise Initialization Options](https://invoke-ai.github.io/InvokeAI/features/OTHER/#thresholding-and-perlin-noise-initialization-options)
+- [Negative/Unconditioned Prompts](https://invoke-ai.github.io/InvokeAI/features/PROMPTS/#negative-and-unconditioned-prompts)
+- [Variations](https://invoke-ai.github.io/InvokeAI/features/VARIATIONS/)
+- [Personalizing Text-to-Image Generation](https://invoke-ai.github.io/InvokeAI/features/TEXTUAL_INVERSION/)
+- [Simplified API for text to image generation](https://invoke-ai.github.io/InvokeAI/features/OTHER/#simplified-api)
+
+#### Other Features
+
+- [Google Colab](https://invoke-ai.github.io/InvokeAI/features/OTHER/#google-colab)
+- [Seamless Tiling](https://invoke-ai.github.io/InvokeAI/features/OTHER/#seamless-tiling)
+- [Shortcut: Reusing Seeds](https://invoke-ai.github.io/InvokeAI/features/OTHER/#shortcuts-reusing-seeds)
+- [Preload Models](https://invoke-ai.github.io/InvokeAI/features/OTHER/#preload-models)
+
+### Latest Changes
+
+- v2.0.1 (13 October 2022)
+  - fix noisy images at high step count when using k* samplers
+  - dream.py script now calls invoke.py module directly rather than
+    via a new python process (which could break the environment)
+
+- v2.0.0 (9 October 2022)
+
+  - `dream.py` script renamed `invoke.py`. A `dream.py` script wrapper remains
+    for backward compatibility.
+  - Completely new WebGUI - launch with `python3 scripts/invoke.py --web`
+  - Support for <a href="https://invoke-ai.github.io/InvokeAI/features/INPAINTING/">inpainting</a> and <a href="https://invoke-ai.github.io/InvokeAI/features/OUTPAINTING/">outpainting</a>
+  - img2img runs on all k* samplers
+  - Support for <a href="https://invoke-ai.github.io/InvokeAI/features/PROMPTS/#negative-and-unconditioned-prompts">negative prompts</a>
+  - Support for CodeFormer face reconstruction
+  - Support for Textual Inversion on Macintoshes
+  - Support in both WebGUI and CLI for <a href="https://invoke-ai.github.io/InvokeAI/features/POSTPROCESS/">post-processing of previously-generated images</a>
+    using facial reconstruction, ESRGAN upscaling, outcropping (similar to DALL-E infinite canvas),
+    and "embiggen" upscaling. See the `!fix` command.
+  - New `--hires` option on `invoke>` line allows <a href="https://invoke-ai.github.io/InvokeAI/features/CLI/#txt2img">larger images to be created without duplicating elements</a>, at the cost of some performance.
+  - New `--perlin` and `--threshold` options allow you to add and control variation
+    during image generation (see <a href="https://github.com/invoke-ai/InvokeAI/blob/main/docs/features/OTHER.md#thresholding-and-perlin-noise-initialization-options">Thresholding and Perlin Noise Initialization</a>
+  - Extensive metadata now written into PNG files, allowing reliable regeneration of images
+    and tweaking of previous settings.
+  - Command-line completion in `invoke.py` now works on Windows, Linux and Mac platforms.
+  - Improved <a href="https://invoke-ai.github.io/InvokeAI/features/CLI/">command-line completion behavior</a>.
+    New commands added:
+    - List command-line history with `!history`
+    - Search command-line history with `!search`
+    - Clear history with `!clear`
+  - Deprecated `--full_precision` / `-F`. Simply omit it and `invoke.py` will auto
+    configure. To switch away from auto use the new flag like `--precision=float32`.
+
+For older changelogs, please visit the **[CHANGELOG](https://invoke-ai.github.io/InvokeAI/CHANGELOG#v114-11-september-2022)**.
+
+### Troubleshooting
+
+Please check out our **[Q&A](https://invoke-ai.github.io/InvokeAI/help/TROUBLESHOOT/#faq)** to get solutions for common installation
+problems and other issues.
+
+# Contributing
+
+Anyone who wishes to contribute to this project, whether documentation, features, bug fixes, code
+cleanup, testing, or code reviews, is very much encouraged to do so. To join, just raise your hand on the InvokeAI
+Discord server or discussion board. 
+
+If you are unfamiliar with how
+to contribute to GitHub projects, here is a
+[Getting Started Guide](https://opensource.com/article/19/7/create-pull-request-github). A full set of contribution guidelines, along with templates, are in progress, but for now the most
+important thing is to **make your pull request against the "development" branch**, and not against
+"main". This will help keep public breakage to a minimum and will allow you to propose more radical
+changes.
+
+We hope you enjoy using our software as much as we enjoy creating it,
+and we hope that some of those of you who are reading this will elect
+to become part of our community.
+
+Welcome to InvokeAI!
+
+### Contributors
+
+This fork is a combined effort of various people from across the world.
+[Check out the list of all these amazing people](https://invoke-ai.github.io/InvokeAI/other/CONTRIBUTORS/). We thank them for
+their time, hard work and effort.
+
+### Support
+
+For support, please use this repository's GitHub Issues tracking service. Feel free to send me an
+email if you use and like the script.
+
+Original portions of the software are Copyright (c) 2020
+[Lincoln D. Stein](https://github.com/lstein)
+
+### Further Reading
+
+Please see the original README for more information on this software and underlying algorithm,
+located in the file [README-CompViz.md](https://invoke-ai.github.io/InvokeAI/other/README-CompViz/).

backend/modules/create_cmd_parser.py

@@ -0,0 +1,55 @@
+import argparse
+import os
+from ldm.invoke.args import PRECISION_CHOICES
+
+
+def create_cmd_parser():
+    parser = argparse.ArgumentParser(description="InvokeAI web UI")
+    parser.add_argument(
+        "--host",
+        type=str,
+        help="The host to serve on",
+        default="localhost",
+    )
+    parser.add_argument("--port", type=int, help="The port to serve on", default=9090)
+    parser.add_argument(
+        "--cors",
+        nargs="*",
+        type=str,
+        help="Additional allowed origins, comma-separated",
+    )
+    parser.add_argument(
+        "--embedding_path",
+        type=str,
+        help="Path to a pre-trained embedding manager checkpoint - can only be set on command line",
+    )
+    # TODO: Can't get flask to serve images from any dir (saving to the dir does work when specified)
+    # parser.add_argument(
+    #     "--output_dir",
+    #     default="outputs/",
+    #     type=str,
+    #     help="Directory for output images",
+    # )
+    parser.add_argument(
+        "-v",
+        "--verbose",
+        action="store_true",
+        help="Enables verbose logging",
+    )
+    parser.add_argument(
+        "--precision",
+        dest="precision",
+        type=str,
+        choices=PRECISION_CHOICES,
+        metavar="PRECISION",
+        help=f'Set model precision. Defaults to auto selected based on device. Options: {", ".join(PRECISION_CHOICES)}',
+        default="auto",
+    )
+    parser.add_argument(
+        '--free_gpu_mem',
+        dest='free_gpu_mem',
+        action='store_true',
+        help='Force free gpu memory before final decoding',
+    )
+
+    return parser

backend/modules/get_canvas_generation_mode.py

@@ -0,0 +1,117 @@
+from PIL import Image, ImageChops
+from PIL.Image import Image as ImageType
+from typing import Union, Literal
+
+# https://stackoverflow.com/questions/43864101/python-pil-check-if-image-is-transparent
+def check_for_any_transparency(img: Union[ImageType, str]) -> bool:
+    if type(img) is str:
+        img = Image.open(str)
+
+    if img.info.get("transparency", None) is not None:
+        return True
+    if img.mode == "P":
+        transparent = img.info.get("transparency", -1)
+        for _, index in img.getcolors():
+            if index == transparent:
+                return True
+    elif img.mode == "RGBA":
+        extrema = img.getextrema()
+        if extrema[3][0] < 255:
+            return True
+    return False
+
+
+def get_canvas_generation_mode(
+    init_img: Union[ImageType, str], init_mask: Union[ImageType, str]
+) -> Literal["txt2img", "outpainting", "inpainting", "img2img",]:
+    if type(init_img) is str:
+        init_img = Image.open(init_img)
+
+    if type(init_mask) is str:
+        init_mask = Image.open(init_mask)
+
+    init_img = init_img.convert("RGBA")
+
+    # Get alpha from init_img
+    init_img_alpha = init_img.split()[-1]
+    init_img_alpha_mask = init_img_alpha.convert("L")
+    init_img_has_transparency = check_for_any_transparency(init_img)
+
+    if init_img_has_transparency:
+        init_img_is_fully_transparent = (
+            True if init_img_alpha_mask.getbbox() is None else False
+        )
+
+    """
+    Mask images are white in areas where no change should be made, black where changes 
+    should be made.
+    """
+
+    # Fit the mask to init_img's size and convert it to greyscale
+    init_mask = init_mask.resize(init_img.size).convert("L")
+
+    """
+    PIL.Image.getbbox() returns the bounding box of non-zero areas of the image, so we first
+    invert the mask image so that masked areas are white and other areas black == zero.
+    getbbox() now tells us if the are any masked areas.
+    """
+    init_mask_bbox = ImageChops.invert(init_mask).getbbox()
+    init_mask_exists = False if init_mask_bbox is None else True
+
+    if init_img_has_transparency:
+        if init_img_is_fully_transparent:
+            return "txt2img"
+        else:
+            return "outpainting"
+    else:
+        if init_mask_exists:
+            return "inpainting"
+        else:
+            return "img2img"
+
+
+def main():
+    # Testing
+    init_img_opaque = "test_images/init-img_opaque.png"
+    init_img_partial_transparency = "test_images/init-img_partial_transparency.png"
+    init_img_full_transparency = "test_images/init-img_full_transparency.png"
+    init_mask_no_mask = "test_images/init-mask_no_mask.png"
+    init_mask_has_mask = "test_images/init-mask_has_mask.png"
+
+    print(
+        "OPAQUE IMAGE, NO MASK, expect img2img, got ",
+        get_canvas_generation_mode(init_img_opaque, init_mask_no_mask),
+    )
+
+    print(
+        "IMAGE WITH TRANSPARENCY, NO MASK, expect outpainting, got ",
+        get_canvas_generation_mode(
+            init_img_partial_transparency, init_mask_no_mask
+        ),
+    )
+
+    print(
+        "FULLY TRANSPARENT IMAGE NO MASK, expect txt2img, got ",
+        get_canvas_generation_mode(init_img_full_transparency, init_mask_no_mask),
+    )
+
+    print(
+        "OPAQUE IMAGE, WITH MASK, expect inpainting, got ",
+        get_canvas_generation_mode(init_img_opaque, init_mask_has_mask),
+    )
+
+    print(
+        "IMAGE WITH TRANSPARENCY, WITH MASK, expect outpainting, got ",
+        get_canvas_generation_mode(
+            init_img_partial_transparency, init_mask_has_mask
+        ),
+    )
+
+    print(
+        "FULLY TRANSPARENT IMAGE WITH MASK, expect txt2img, got ",
+        get_canvas_generation_mode(init_img_full_transparency, init_mask_has_mask),
+    )
+
+
+if __name__ == "__main__":
+    main()

backend/modules/parameters.py

@@ -0,0 +1,71 @@
+from backend.modules.parse_seed_weights import parse_seed_weights
+import argparse
+
+SAMPLER_CHOICES = [
+    "ddim",
+    "k_dpm_2_a",
+    "k_dpm_2",
+    "k_dpmpp_2_a",
+    "k_dpmpp_2",
+    "k_euler_a",
+    "k_euler",
+    "k_heun",
+    "k_lms",
+    "plms",
+]
+
+
+def parameters_to_command(params):
+    """
+    Converts dict of parameters into a `invoke.py` REPL command.
+    """
+
+    switches = list()
+
+    if "prompt" in params:
+        switches.append(f'"{params["prompt"]}"')
+    if "steps" in params:
+        switches.append(f'-s {params["steps"]}')
+    if "seed" in params:
+        switches.append(f'-S {params["seed"]}')
+    if "width" in params:
+        switches.append(f'-W {params["width"]}')
+    if "height" in params:
+        switches.append(f'-H {params["height"]}')
+    if "cfg_scale" in params:
+        switches.append(f'-C {params["cfg_scale"]}')
+    if "sampler_name" in params:
+        switches.append(f'-A {params["sampler_name"]}')
+    if "seamless" in params and params["seamless"] == True:
+        switches.append(f"--seamless")
+    if "hires_fix" in params and params["hires_fix"] == True:
+        switches.append(f"--hires")
+    if "init_img" in params and len(params["init_img"]) > 0:
+        switches.append(f'-I {params["init_img"]}')
+    if "init_mask" in params and len(params["init_mask"]) > 0:
+        switches.append(f'-M {params["init_mask"]}')
+    if "init_color" in params and len(params["init_color"]) > 0:
+        switches.append(f'--init_color {params["init_color"]}')
+    if "strength" in params and "init_img" in params:
+        switches.append(f'-f {params["strength"]}')
+        if "fit" in params and params["fit"] == True:
+            switches.append(f"--fit")
+    if "facetool" in params:
+        switches.append(f'-ft {params["facetool"]}')
+    if "facetool_strength" in params and params["facetool_strength"]:
+        switches.append(f'-G {params["facetool_strength"]}')
+    elif "gfpgan_strength" in params and params["gfpgan_strength"]:
+        switches.append(f'-G {params["gfpgan_strength"]}')
+    if "codeformer_fidelity" in params:
+        switches.append(f'-cf {params["codeformer_fidelity"]}')
+    if "upscale" in params and params["upscale"]:
+        switches.append(f'-U {params["upscale"][0]} {params["upscale"][1]}')
+    if "variation_amount" in params and params["variation_amount"] > 0:
+        switches.append(f'-v {params["variation_amount"]}')
+        if "with_variations" in params:
+            seed_weight_pairs = ",".join(
+                f"{seed}:{weight}" for seed, weight in params["with_variations"]
+            )
+            switches.append(f"-V {seed_weight_pairs}")
+
+    return " ".join(switches)

backend/modules/parse_seed_weights.py

@@ -0,0 +1,47 @@
+def parse_seed_weights(seed_weights):
+    """
+    Accepts seed weights as string in "12345:0.1,23456:0.2,3456:0.3" format
+    Validates them
+    If valid: returns as [[12345, 0.1], [23456, 0.2], [3456, 0.3]]
+    If invalid: returns False
+    """
+
+    # Must be a string
+    if not isinstance(seed_weights, str):
+        return False
+    # String must not be empty
+    if len(seed_weights) == 0:
+        return False
+
+    pairs = []
+
+    for pair in seed_weights.split(","):
+        split_values = pair.split(":")
+
+        # Seed and weight are required
+        if len(split_values) != 2:
+            return False
+
+        if len(split_values[0]) == 0 or len(split_values[1]) == 1:
+            return False
+
+        # Try casting the seed to int and weight to float
+        try:
+            seed = int(split_values[0])
+            weight = float(split_values[1])
+        except ValueError:
+            return False
+
+        # Seed must be 0 or above
+        if not seed >= 0:
+            return False
+
+        # Weight must be between 0 and 1
+        if not (weight >= 0 and weight <= 1):
+            return False
+
+        # This pair is valid
+        pairs.append([seed, weight])
+
+    # All pairs are valid
+    return pairs

configs/INITIAL_MODELS.yaml

@@ -0,0 +1,80 @@
+stable-diffusion-1.5:
+   description: The newest Stable Diffusion version 1.5 weight file (4.27 GB)
+   repo_id: runwayml/stable-diffusion-v1-5
+   config: v1-inference.yaml
+   file: v1-5-pruned-emaonly.ckpt
+   recommended: true
+   width: 512
+   height: 512
+inpainting-1.5:
+   description: RunwayML SD 1.5 model optimized for inpainting (4.27 GB)
+   repo_id: runwayml/stable-diffusion-inpainting
+   config: v1-inpainting-inference.yaml
+   file: sd-v1-5-inpainting.ckpt
+   recommended: True
+   width: 512
+   height: 512
+ft-mse-improved-autoencoder-840000:
+   description: StabilityAI improved autoencoder fine-tuned for human faces (recommended; 335 MB)
+   repo_id: stabilityai/sd-vae-ft-mse-original
+   config: VAE/default
+   file: vae-ft-mse-840000-ema-pruned.ckpt
+   recommended: True
+   width: 512
+   height: 512
+stable-diffusion-1.4:
+   description: The original Stable Diffusion version 1.4 weight file (4.27 GB)
+   repo_id: CompVis/stable-diffusion-v-1-4-original
+   config: v1-inference.yaml
+   file: sd-v1-4.ckpt
+   recommended: False
+   width: 512
+   height: 512
+waifu-diffusion-1.3:
+   description: Stable Diffusion 1.4 fine tuned on anime-styled images (4.27)
+   repo_id: hakurei/waifu-diffusion-v1-3
+   config: v1-inference.yaml
+   file: model-epoch09-float32.ckpt
+   recommended: False
+   width: 512
+   height: 512
+trinart-2.0:
+   description: An SD model finetuned with ~40,000 assorted high resolution manga/anime-style pictures (2.13 GB)
+   repo_id: naclbit/trinart_stable_diffusion_v2
+   config: v1-inference.yaml
+   file: trinart2_step95000.ckpt
+   recommended: False
+   width: 512
+   height: 512
+trinart_characters-1.0:
+   description: An SD model finetuned with 19.2M anime/manga style images (2.13 GB)
+   repo_id: naclbit/trinart_characters_19.2m_stable_diffusion_v1
+   config: v1-inference.yaml
+   file: trinart_characters_it4_v1.ckpt
+   recommended: False
+   width: 512
+   height: 512
+trinart_vae:
+   description: Custom autoencoder for trinart_characters
+   repo_id: naclbit/trinart_characters_19.2m_stable_diffusion_v1
+   config: VAE/trinart
+   file: autoencoder_fix_kl-f8-trinart_characters.ckpt
+   recommended: False
+   width: 512
+   height: 512
+papercut-1.0:
+   description: SD 1.5 fine-tuned for papercut art (use "PaperCut" in your prompts) (2.13 GB)
+   repo_id: Fictiverse/Stable_Diffusion_PaperCut_Model
+   config: v1-inference.yaml
+   file: PaperCut_v1.ckpt
+   recommended: False
+   width: 512
+   height: 512
+voxel_art-1.0:
+   description: Stable Diffusion trained on voxel art (use "VoxelArt" in your prompts) (4.27 GB)
+   repo_id: Fictiverse/Stable_Diffusion_VoxelArt_Model
+   config: v1-inference.yaml
+   file: VoxelArt_v1.ckpt
+   recommended: False
+   width: 512
+   height: 512

configs/autoencoder/autoencoder_kl_16x16x16.yaml

@@ -1,54 +0,0 @@
-model:
-  base_learning_rate: 4.5e-6
-  target: ldm.models.autoencoder.AutoencoderKL
-  params:
-    monitor: "val/rec_loss"
-    embed_dim: 16
-    lossconfig:
-      target: ldm.modules.losses.LPIPSWithDiscriminator
-      params:
-        disc_start: 50001
-        kl_weight: 0.000001
-        disc_weight: 0.5
-
-    ddconfig:
-      double_z: True
-      z_channels: 16
-      resolution: 256
-      in_channels: 3
-      out_ch: 3
-      ch: 128
-      ch_mult: [ 1,1,2,2,4]  # num_down = len(ch_mult)-1
-      num_res_blocks: 2
-      attn_resolutions: [16]
-      dropout: 0.0
-
-
-data:
-  target: main.DataModuleFromConfig
-  params:
-    batch_size: 12
-    wrap: True
-    train:
-      target: ldm.data.imagenet.ImageNetSRTrain
-      params:
-        size: 256
-        degradation: pil_nearest
-    validation:
-      target: ldm.data.imagenet.ImageNetSRValidation
-      params:
-        size: 256
-        degradation: pil_nearest
-
-lightning:
-  callbacks:
-    image_logger:
-      target: main.ImageLogger
-      params:
-        batch_frequency: 1000
-        max_images: 8
-        increase_log_steps: True
-
-  trainer:
-    benchmark: True
-    accumulate_grad_batches: 2

configs/autoencoder/autoencoder_kl_32x32x4.yaml

@@ -1,53 +0,0 @@
-model:
-  base_learning_rate: 4.5e-6
-  target: ldm.models.autoencoder.AutoencoderKL
-  params:
-    monitor: "val/rec_loss"
-    embed_dim: 4
-    lossconfig:
-      target: ldm.modules.losses.LPIPSWithDiscriminator
-      params:
-        disc_start: 50001
-        kl_weight: 0.000001
-        disc_weight: 0.5
-
-    ddconfig:
-      double_z: True
-      z_channels: 4
-      resolution: 256
-      in_channels: 3
-      out_ch: 3
-      ch: 128
-      ch_mult: [ 1,2,4,4 ]  # num_down = len(ch_mult)-1
-      num_res_blocks: 2
-      attn_resolutions: [ ]
-      dropout: 0.0
-
-data:
-  target: main.DataModuleFromConfig
-  params:
-    batch_size: 12
-    wrap: True
-    train:
-      target: ldm.data.imagenet.ImageNetSRTrain
-      params:
-        size: 256
-        degradation: pil_nearest
-    validation:
-      target: ldm.data.imagenet.ImageNetSRValidation
-      params:
-        size: 256
-        degradation: pil_nearest
-
-lightning:
-  callbacks:
-    image_logger:
-      target: main.ImageLogger
-      params:
-        batch_frequency: 1000
-        max_images: 8
-        increase_log_steps: True
-
-  trainer:
-    benchmark: True
-    accumulate_grad_batches: 2

configs/autoencoder/autoencoder_kl_64x64x3.yaml

@@ -1,54 +0,0 @@
-model:
-  base_learning_rate: 4.5e-6
-  target: ldm.models.autoencoder.AutoencoderKL
-  params:
-    monitor: "val/rec_loss"
-    embed_dim: 3
-    lossconfig:
-      target: ldm.modules.losses.LPIPSWithDiscriminator
-      params:
-        disc_start: 50001
-        kl_weight: 0.000001
-        disc_weight: 0.5
-
-    ddconfig:
-      double_z: True
-      z_channels: 3
-      resolution: 256
-      in_channels: 3
-      out_ch: 3
-      ch: 128
-      ch_mult: [ 1,2,4 ]  # num_down = len(ch_mult)-1
-      num_res_blocks: 2
-      attn_resolutions: [ ]
-      dropout: 0.0
-
-
-data:
-  target: main.DataModuleFromConfig
-  params:
-    batch_size: 12
-    wrap: True
-    train:
-      target: ldm.data.imagenet.ImageNetSRTrain
-      params:
-        size: 256
-        degradation: pil_nearest
-    validation:
-      target: ldm.data.imagenet.ImageNetSRValidation
-      params:
-        size: 256
-        degradation: pil_nearest
-
-lightning:
-  callbacks:
-    image_logger:
-      target: main.ImageLogger
-      params:
-        batch_frequency: 1000
-        max_images: 8
-        increase_log_steps: True
-
-  trainer:
-    benchmark: True
-    accumulate_grad_batches: 2

configs/autoencoder/autoencoder_kl_8x8x64.yaml

@@ -1,53 +0,0 @@
-model:
-  base_learning_rate: 4.5e-6
-  target: ldm.models.autoencoder.AutoencoderKL
-  params:
-    monitor: "val/rec_loss"
-    embed_dim: 64
-    lossconfig:
-      target: ldm.modules.losses.LPIPSWithDiscriminator
-      params:
-        disc_start: 50001
-        kl_weight: 0.000001
-        disc_weight: 0.5
-
-    ddconfig:
-      double_z: True
-      z_channels: 64
-      resolution: 256
-      in_channels: 3
-      out_ch: 3
-      ch: 128
-      ch_mult: [ 1,1,2,2,4,4]  # num_down = len(ch_mult)-1
-      num_res_blocks: 2
-      attn_resolutions: [16,8]
-      dropout: 0.0
-
-data:
-  target: main.DataModuleFromConfig
-  params:
-    batch_size: 12
-    wrap: True
-    train:
-      target: ldm.data.imagenet.ImageNetSRTrain
-      params:
-        size: 256
-        degradation: pil_nearest
-    validation:
-      target: ldm.data.imagenet.ImageNetSRValidation
-      params:
-        size: 256
-        degradation: pil_nearest
-
-lightning:
-  callbacks:
-    image_logger:
-      target: main.ImageLogger
-      params:
-        batch_frequency: 1000
-        max_images: 8
-        increase_log_steps: True
-
-  trainer:
-    benchmark: True
-    accumulate_grad_batches: 2

configs/latent-diffusion/celebahq-ldm-vq-4.yaml

@@ -1,86 +0,0 @@
-model:
-  base_learning_rate: 2.0e-06
-  target: ldm.models.diffusion.ddpm.LatentDiffusion
-  params:
-    linear_start: 0.0015
-    linear_end: 0.0195
-    num_timesteps_cond: 1
-    log_every_t: 200
-    timesteps: 1000
-    first_stage_key: image
-    image_size: 64
-    channels: 3
-    monitor: val/loss_simple_ema
-
-    unet_config:
-      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
-      params:
-        image_size: 64
-        in_channels: 3
-        out_channels: 3
-        model_channels: 224
-        attention_resolutions:
-        # note: this isn\t actually the resolution but
-        # the downsampling factor, i.e. this corresnponds to
-        # attention on spatial resolution 8,16,32, as the
-        # spatial reolution of the latents is 64 for f4
-        - 8
-        - 4
-        - 2
-        num_res_blocks: 2
-        channel_mult:
-        - 1
-        - 2
-        - 3
-        - 4
-        num_head_channels: 32
-    first_stage_config:
-      target: ldm.models.autoencoder.VQModelInterface
-      params:
-        embed_dim: 3
-        n_embed: 8192
-        ckpt_path: models/first_stage_models/vq-f4/model.ckpt
-        ddconfig:
-          double_z: false
-          z_channels: 3
-          resolution: 256
-          in_channels: 3
-          out_ch: 3
-          ch: 128
-          ch_mult:
-          - 1
-          - 2
-          - 4
-          num_res_blocks: 2
-          attn_resolutions: []
-          dropout: 0.0
-        lossconfig:
-          target: torch.nn.Identity
-    cond_stage_config: __is_unconditional__
-data:
-  target: main.DataModuleFromConfig
-  params:
-    batch_size: 48
-    num_workers: 5
-    wrap: false
-    train:
-      target: taming.data.faceshq.CelebAHQTrain
-      params:
-        size: 256
-    validation:
-      target: taming.data.faceshq.CelebAHQValidation
-      params:
-        size: 256
-
-
-lightning:
-  callbacks:
-    image_logger:
-      target: main.ImageLogger
-      params:
-        batch_frequency: 5000
-        max_images: 8
-        increase_log_steps: False
-
-  trainer:
-    benchmark: True

configs/latent-diffusion/cin-ldm-vq-f8.yaml

@@ -1,98 +0,0 @@
-model:
-  base_learning_rate: 1.0e-06
-  target: ldm.models.diffusion.ddpm.LatentDiffusion
-  params:
-    linear_start: 0.0015
-    linear_end: 0.0195
-    num_timesteps_cond: 1
-    log_every_t: 200
-    timesteps: 1000
-    first_stage_key: image
-    cond_stage_key: class_label
-    image_size: 32
-    channels: 4
-    cond_stage_trainable: true
-    conditioning_key: crossattn
-    monitor: val/loss_simple_ema
-    unet_config:
-      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
-      params:
-        image_size: 32
-        in_channels: 4
-        out_channels: 4
-        model_channels: 256
-        attention_resolutions:
-        #note: this isn\t actually the resolution but
-        # the downsampling factor, i.e. this corresnponds to
-        # attention on spatial resolution 8,16,32, as the
-        # spatial reolution of the latents is 32 for f8
-        - 4
-        - 2
-        - 1
-        num_res_blocks: 2
-        channel_mult:
-        - 1
-        - 2
-        - 4
-        num_head_channels: 32
-        use_spatial_transformer: true
-        transformer_depth: 1
-        context_dim: 512
-    first_stage_config:
-      target: ldm.models.autoencoder.VQModelInterface
-      params:
-        embed_dim: 4
-        n_embed: 16384
-        ckpt_path: configs/first_stage_models/vq-f8/model.yaml
-        ddconfig:
-          double_z: false
-          z_channels: 4
-          resolution: 256
-          in_channels: 3
-          out_ch: 3
-          ch: 128
-          ch_mult:
-          - 1
-          - 2
-          - 2
-          - 4
-          num_res_blocks: 2
-          attn_resolutions:
-          - 32
-          dropout: 0.0
-        lossconfig:
-          target: torch.nn.Identity
-    cond_stage_config:
-      target: ldm.modules.encoders.modules.ClassEmbedder
-      params:
-        embed_dim: 512
-        key: class_label
-data:
-  target: main.DataModuleFromConfig
-  params:
-    batch_size: 64
-    num_workers: 12
-    wrap: false
-    train:
-      target: ldm.data.imagenet.ImageNetTrain
-      params:
-        config:
-          size: 256
-    validation:
-      target: ldm.data.imagenet.ImageNetValidation
-      params:
-        config:
-          size: 256
-
-
-lightning:
-  callbacks:
-    image_logger:
-      target: main.ImageLogger
-      params:
-        batch_frequency: 5000
-        max_images: 8
-        increase_log_steps: False
-
-  trainer:
-    benchmark: True

configs/latent-diffusion/cin256-v2.yaml

@@ -1,68 +0,0 @@
-model:
-  base_learning_rate: 0.0001
-  target: ldm.models.diffusion.ddpm.LatentDiffusion
-  params:
-    linear_start: 0.0015
-    linear_end: 0.0195
-    num_timesteps_cond: 1
-    log_every_t: 200
-    timesteps: 1000
-    first_stage_key: image
-    cond_stage_key: class_label
-    image_size: 64
-    channels: 3
-    cond_stage_trainable: true
-    conditioning_key: crossattn
-    monitor: val/loss
-    use_ema: False
-    
-    unet_config:
-      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
-      params:
-        image_size: 64
-        in_channels: 3
-        out_channels: 3
-        model_channels: 192
-        attention_resolutions:
-        - 8
-        - 4
-        - 2
-        num_res_blocks: 2
-        channel_mult:
-        - 1
-        - 2
-        - 3
-        - 5
-        num_heads: 1
-        use_spatial_transformer: true
-        transformer_depth: 1
-        context_dim: 512
-    
-    first_stage_config:
-      target: ldm.models.autoencoder.VQModelInterface
-      params:
-        embed_dim: 3
-        n_embed: 8192
-        ddconfig:
-          double_z: false
-          z_channels: 3
-          resolution: 256
-          in_channels: 3
-          out_ch: 3
-          ch: 128
-          ch_mult:
-          - 1
-          - 2
-          - 4
-          num_res_blocks: 2
-          attn_resolutions: []
-          dropout: 0.0
-        lossconfig:
-          target: torch.nn.Identity
-    
-    cond_stage_config:
-      target: ldm.modules.encoders.modules.ClassEmbedder
-      params:
-        n_classes: 1001
-        embed_dim: 512
-        key: class_label

configs/latent-diffusion/ffhq-ldm-vq-4.yaml

@@ -1,85 +0,0 @@
-model:
-  base_learning_rate: 2.0e-06
-  target: ldm.models.diffusion.ddpm.LatentDiffusion
-  params:
-    linear_start: 0.0015
-    linear_end: 0.0195
-    num_timesteps_cond: 1
-    log_every_t: 200
-    timesteps: 1000
-    first_stage_key: image
-    image_size: 64
-    channels: 3
-    monitor: val/loss_simple_ema
-    unet_config:
-      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
-      params:
-        image_size: 64
-        in_channels: 3
-        out_channels: 3
-        model_channels: 224
-        attention_resolutions:
-        # note: this isn\t actually the resolution but
-        # the downsampling factor, i.e. this corresnponds to
-        # attention on spatial resolution 8,16,32, as the
-        # spatial reolution of the latents is 64 for f4
-        - 8
-        - 4
-        - 2
-        num_res_blocks: 2
-        channel_mult:
-        - 1
-        - 2
-        - 3
-        - 4
-        num_head_channels: 32
-    first_stage_config:
-      target: ldm.models.autoencoder.VQModelInterface
-      params:
-        embed_dim: 3
-        n_embed: 8192
-        ckpt_path: configs/first_stage_models/vq-f4/model.yaml
-        ddconfig:
-          double_z: false
-          z_channels: 3
-          resolution: 256
-          in_channels: 3
-          out_ch: 3
-          ch: 128
-          ch_mult:
-          - 1
-          - 2
-          - 4
-          num_res_blocks: 2
-          attn_resolutions: []
-          dropout: 0.0
-        lossconfig:
-          target: torch.nn.Identity
-    cond_stage_config: __is_unconditional__
-data:
-  target: main.DataModuleFromConfig
-  params:
-    batch_size: 42
-    num_workers: 5
-    wrap: false
-    train:
-      target: taming.data.faceshq.FFHQTrain
-      params:
-        size: 256
-    validation:
-      target: taming.data.faceshq.FFHQValidation
-      params:
-        size: 256
-
-
-lightning:
-  callbacks:
-    image_logger:
-      target: main.ImageLogger
-      params:
-        batch_frequency: 5000
-        max_images: 8
-        increase_log_steps: False
-
-  trainer:
-    benchmark: True

configs/latent-diffusion/lsun_bedrooms-ldm-vq-4.yaml

@@ -1,85 +0,0 @@
-model:
-  base_learning_rate: 2.0e-06
-  target: ldm.models.diffusion.ddpm.LatentDiffusion
-  params:
-    linear_start: 0.0015
-    linear_end: 0.0195
-    num_timesteps_cond: 1
-    log_every_t: 200
-    timesteps: 1000
-    first_stage_key: image
-    image_size: 64
-    channels: 3
-    monitor: val/loss_simple_ema
-    unet_config:
-      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
-      params:
-        image_size: 64
-        in_channels: 3
-        out_channels: 3
-        model_channels: 224
-        attention_resolutions:
-        # note: this isn\t actually the resolution but
-        # the downsampling factor, i.e. this corresnponds to
-        # attention on spatial resolution 8,16,32, as the
-        # spatial reolution of the latents is 64 for f4
-        - 8
-        - 4
-        - 2
-        num_res_blocks: 2
-        channel_mult:
-        - 1
-        - 2
-        - 3
-        - 4
-        num_head_channels: 32
-    first_stage_config:
-      target: ldm.models.autoencoder.VQModelInterface
-      params:
-        ckpt_path: configs/first_stage_models/vq-f4/model.yaml
-        embed_dim: 3
-        n_embed: 8192
-        ddconfig:
-          double_z: false
-          z_channels: 3
-          resolution: 256
-          in_channels: 3
-          out_ch: 3
-          ch: 128
-          ch_mult:
-          - 1
-          - 2
-          - 4
-          num_res_blocks: 2
-          attn_resolutions: []
-          dropout: 0.0
-        lossconfig:
-          target: torch.nn.Identity
-    cond_stage_config: __is_unconditional__
-data:
-  target: main.DataModuleFromConfig
-  params:
-    batch_size: 48
-    num_workers: 5
-    wrap: false
-    train:
-      target: ldm.data.lsun.LSUNBedroomsTrain
-      params:
-        size: 256
-    validation:
-      target: ldm.data.lsun.LSUNBedroomsValidation
-      params:
-        size: 256
-
-
-lightning:
-  callbacks:
-    image_logger:
-      target: main.ImageLogger
-      params:
-        batch_frequency: 5000
-        max_images: 8
-        increase_log_steps: False
-
-  trainer:
-    benchmark: True

configs/latent-diffusion/lsun_churches-ldm-kl-8.yaml

@@ -1,91 +0,0 @@
-model:
-  base_learning_rate: 5.0e-5   # set to target_lr by starting main.py with '--scale_lr False'
-  target: ldm.models.diffusion.ddpm.LatentDiffusion
-  params:
-    linear_start: 0.0015
-    linear_end: 0.0155
-    num_timesteps_cond: 1
-    log_every_t: 200
-    timesteps: 1000
-    loss_type: l1
-    first_stage_key: "image"
-    cond_stage_key: "image"
-    image_size: 32
-    channels: 4
-    cond_stage_trainable: False
-    concat_mode: False
-    scale_by_std: True
-    monitor: 'val/loss_simple_ema'
-
-    scheduler_config: # 10000 warmup steps
-      target: ldm.lr_scheduler.LambdaLinearScheduler
-      params:
-        warm_up_steps: [10000]
-        cycle_lengths: [10000000000000]
-        f_start: [1.e-6]
-        f_max: [1.]
-        f_min: [ 1.]
-
-    unet_config:
-      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
-      params:
-        image_size: 32
-        in_channels: 4
-        out_channels: 4
-        model_channels: 192
-        attention_resolutions: [ 1, 2, 4, 8 ]   # 32, 16, 8, 4
-        num_res_blocks: 2
-        channel_mult: [ 1,2,2,4,4 ]  # 32, 16, 8, 4, 2
-        num_heads: 8
-        use_scale_shift_norm: True
-        resblock_updown: True
-
-    first_stage_config:
-      target: ldm.models.autoencoder.AutoencoderKL
-      params:
-        embed_dim: 4
-        monitor: "val/rec_loss"
-        ckpt_path: "models/first_stage_models/kl-f8/model.ckpt"
-        ddconfig:
-          double_z: True
-          z_channels: 4
-          resolution: 256
-          in_channels: 3
-          out_ch: 3
-          ch: 128
-          ch_mult: [ 1,2,4,4 ]  # num_down = len(ch_mult)-1
-          num_res_blocks: 2
-          attn_resolutions: [ ]
-          dropout: 0.0
-        lossconfig:
-          target: torch.nn.Identity
-
-    cond_stage_config: "__is_unconditional__"
-
-data:
-  target: main.DataModuleFromConfig
-  params:
-    batch_size: 96
-    num_workers: 5
-    wrap: False
-    train:
-      target: ldm.data.lsun.LSUNChurchesTrain
-      params:
-        size: 256
-    validation:
-      target: ldm.data.lsun.LSUNChurchesValidation
-      params:
-        size: 256
-
-lightning:
-  callbacks:
-    image_logger:
-      target: main.ImageLogger
-      params:
-        batch_frequency: 5000
-        max_images: 8
-        increase_log_steps: False
-
-
-  trainer:
-    benchmark: True

configs/latent-diffusion/txt2img-1p4B-eval.yaml

@@ -1,71 +0,0 @@
-model:
-  base_learning_rate: 5.0e-05
-  target: ldm.models.diffusion.ddpm.LatentDiffusion
-  params:
-    linear_start: 0.00085
-    linear_end: 0.012
-    num_timesteps_cond: 1
-    log_every_t: 200
-    timesteps: 1000
-    first_stage_key: image
-    cond_stage_key: caption
-    image_size: 32
-    channels: 4
-    cond_stage_trainable: true
-    conditioning_key: crossattn
-    monitor: val/loss_simple_ema
-    scale_factor: 0.18215
-    use_ema: False
-
-    unet_config:
-      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
-      params:
-        image_size: 32
-        in_channels: 4
-        out_channels: 4
-        model_channels: 320
-        attention_resolutions:
-        - 4
-        - 2
-        - 1
-        num_res_blocks: 2
-        channel_mult:
-        - 1
-        - 2
-        - 4
-        - 4
-        num_heads: 8
-        use_spatial_transformer: true
-        transformer_depth: 1
-        context_dim: 1280
-        use_checkpoint: true
-        legacy: False
-
-    first_stage_config:
-      target: ldm.models.autoencoder.AutoencoderKL
-      params:
-        embed_dim: 4
-        monitor: val/rec_loss
-        ddconfig:
-          double_z: true
-          z_channels: 4
-          resolution: 256
-          in_channels: 3
-          out_ch: 3
-          ch: 128
-          ch_mult:
-          - 1
-          - 2
-          - 4
-          - 4
-          num_res_blocks: 2
-          attn_resolutions: []
-          dropout: 0.0
-        lossconfig:
-          target: torch.nn.Identity
-
-    cond_stage_config:
-      target: ldm.modules.encoders.modules.BERTEmbedder
-      params:
-        n_embed: 1280
-        n_layer: 32

configs/models.yaml.example

@@ -0,0 +1,27 @@
+# This file describes the alternative machine learning models
+# available to InvokeAI script.
+#
+# To add a new model, follow the examples below. Each
+# model requires a model config file, a weights file,
+# and the width and height of the images it
+# was trained on.
+stable-diffusion-1.5:
+  description: The newest Stable Diffusion version 1.5 weight file (4.27 GB)
+  weights: models/ldm/stable-diffusion-v1/v1-5-pruned-emaonly.ckpt
+  config: configs/stable-diffusion/v1-inference.yaml
+  width: 512
+  height: 512
+  vae: ./models/ldm/stable-diffusion-v1/vae-ft-mse-840000-ema-pruned.ckpt
+  default: true
+stable-diffusion-1.4:
+  description: Stable Diffusion inference model version 1.4
+  config: configs/stable-diffusion/v1-inference.yaml
+  weights: models/ldm/stable-diffusion-v1/sd-v1-4.ckpt
+  vae: models/ldm/stable-diffusion-v1/vae-ft-mse-840000-ema-pruned.ckpt
+  width: 512
+  height: 512
+inpainting-1.5:
+  weights: models/ldm/stable-diffusion-v1/sd-v1-5-inpainting.ckpt
+  config: configs/stable-diffusion/v1-inpainting-inference.yaml
+  vae: models/ldm/stable-diffusion-v1/vae-ft-mse-840000-ema-pruned.ckpt
+  description: RunwayML SD 1.5 model optimized for inpainting

configs/retrieval-augmented-diffusion/768x768.yaml

@@ -1,68 +0,0 @@
-model:
-  base_learning_rate: 0.0001
-  target: ldm.models.diffusion.ddpm.LatentDiffusion
-  params:
-    linear_start: 0.0015
-    linear_end: 0.015
-    num_timesteps_cond: 1
-    log_every_t: 200
-    timesteps: 1000
-    first_stage_key: jpg
-    cond_stage_key: nix
-    image_size: 48
-    channels: 16
-    cond_stage_trainable: false
-    conditioning_key: crossattn
-    monitor: val/loss_simple_ema
-    scale_by_std: false
-    scale_factor: 0.22765929
-    unet_config:
-      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
-      params:
-        image_size: 48
-        in_channels: 16
-        out_channels: 16
-        model_channels: 448
-        attention_resolutions:
-        - 4
-        - 2
-        - 1
-        num_res_blocks: 2
-        channel_mult:
-        - 1
-        - 2
-        - 3
-        - 4
-        use_scale_shift_norm: false
-        resblock_updown: false
-        num_head_channels: 32
-        use_spatial_transformer: true
-        transformer_depth: 1
-        context_dim: 768
-        use_checkpoint: true
-    first_stage_config:
-      target: ldm.models.autoencoder.AutoencoderKL
-      params:
-        monitor: val/rec_loss
-        embed_dim: 16
-        ddconfig:
-          double_z: true
-          z_channels: 16
-          resolution: 256
-          in_channels: 3
-          out_ch: 3
-          ch: 128
-          ch_mult:
-          - 1
-          - 1
-          - 2
-          - 2
-          - 4
-          num_res_blocks: 2
-          attn_resolutions:
-          - 16
-          dropout: 0.0
-        lossconfig:
-          target: torch.nn.Identity
-    cond_stage_config:
-      target: torch.nn.Identity

configs/sd-concepts.txt

@@ -0,0 +1,803 @@
+sd-concepts-library/001glitch-core
+sd-concepts-library/2814-roth
+sd-concepts-library/3d-female-cyborgs
+sd-concepts-library/4tnght
+sd-concepts-library/80s-anime-ai
+sd-concepts-library/80s-anime-ai-being
+sd-concepts-library/852style-girl
+sd-concepts-library/8bit
+sd-concepts-library/8sconception
+sd-concepts-library/Aflac-duck
+sd-concepts-library/Akitsuki
+sd-concepts-library/Atako
+sd-concepts-library/Exodus-Styling
+sd-concepts-library/RINGAO
+sd-concepts-library/a-female-hero-from-the-legend-of-mir
+sd-concepts-library/a-hat-kid
+sd-concepts-library/a-tale-of-two-empires
+sd-concepts-library/aadhav-face
+sd-concepts-library/aavegotchi
+sd-concepts-library/abby-face
+sd-concepts-library/abstract-concepts
+sd-concepts-library/accurate-angel
+sd-concepts-library/agm-style-nao
+sd-concepts-library/aj-fosik
+sd-concepts-library/alberto-mielgo
+sd-concepts-library/alex-portugal
+sd-concepts-library/alex-thumbnail-object-2000-steps
+sd-concepts-library/aleyna-tilki
+sd-concepts-library/alf
+sd-concepts-library/alicebeta
+sd-concepts-library/alien-avatar
+sd-concepts-library/alisa
+sd-concepts-library/all-rings-albuns
+sd-concepts-library/altvent
+sd-concepts-library/altyn-helmet
+sd-concepts-library/amine
+sd-concepts-library/amogus
+sd-concepts-library/anders-zorn
+sd-concepts-library/angus-mcbride-style
+sd-concepts-library/animalve3-1500seq
+sd-concepts-library/anime-background-style
+sd-concepts-library/anime-background-style-v2
+sd-concepts-library/anime-boy
+sd-concepts-library/anime-girl
+sd-concepts-library/anyXtronXredshift
+sd-concepts-library/anya-forger
+sd-concepts-library/apex-wingman
+sd-concepts-library/apulian-rooster-v0-1
+sd-concepts-library/arcane-face
+sd-concepts-library/arcane-style-jv
+sd-concepts-library/arcimboldo-style
+sd-concepts-library/armando-reveron-style
+sd-concepts-library/armor-concept
+sd-concepts-library/arq-render
+sd-concepts-library/art-brut
+sd-concepts-library/arthur1
+sd-concepts-library/artist-yukiko-kanagai
+sd-concepts-library/arwijn
+sd-concepts-library/ashiok
+sd-concepts-library/at-wolf-boy-object
+sd-concepts-library/atm-ant
+sd-concepts-library/atm-ant-2
+sd-concepts-library/axe-tattoo
+sd-concepts-library/ayush-spider-spr
+sd-concepts-library/azura-from-vibrant-venture
+sd-concepts-library/ba-shiroko
+sd-concepts-library/babau
+sd-concepts-library/babs-bunny
+sd-concepts-library/babushork
+sd-concepts-library/backrooms
+sd-concepts-library/bad_Hub_Hugh
+sd-concepts-library/bada-club
+sd-concepts-library/baldi
+sd-concepts-library/baluchitherian
+sd-concepts-library/bamse
+sd-concepts-library/bamse-og-kylling
+sd-concepts-library/bee
+sd-concepts-library/beholder
+sd-concepts-library/beldam
+sd-concepts-library/belen
+sd-concepts-library/bella-goth
+sd-concepts-library/belle-delphine
+sd-concepts-library/bert-muppet
+sd-concepts-library/better-collage3
+sd-concepts-library/between2-mt-fade
+sd-concepts-library/birb-style
+sd-concepts-library/black-and-white-design
+sd-concepts-library/black-waifu
+sd-concepts-library/bloo
+sd-concepts-library/blue-haired-boy
+sd-concepts-library/blue-zombie
+sd-concepts-library/blue-zombiee
+sd-concepts-library/bluebey
+sd-concepts-library/bluebey-2
+sd-concepts-library/bobs-burgers
+sd-concepts-library/boissonnard
+sd-concepts-library/bonzi-monkey
+sd-concepts-library/borderlands
+sd-concepts-library/bored-ape-textual-inversion
+sd-concepts-library/boris-anderson
+sd-concepts-library/bozo-22
+sd-concepts-library/breakcore
+sd-concepts-library/brittney-williams-art
+sd-concepts-library/bruma
+sd-concepts-library/brunnya
+sd-concepts-library/buddha-statue
+sd-concepts-library/bullvbear
+sd-concepts-library/button-eyes
+sd-concepts-library/canadian-goose
+sd-concepts-library/canary-cap
+sd-concepts-library/cancer_style
+sd-concepts-library/captain-haddock
+sd-concepts-library/captainkirb
+sd-concepts-library/car-toy-rk
+sd-concepts-library/carasibana
+sd-concepts-library/carlitos-el-mago
+sd-concepts-library/carrascharacter
+sd-concepts-library/cartoona-animals
+sd-concepts-library/cat-toy
+sd-concepts-library/centaur
+sd-concepts-library/cgdonny1
+sd-concepts-library/cham
+sd-concepts-library/chandra-nalaar
+sd-concepts-library/char-con
+sd-concepts-library/character-pingu
+sd-concepts-library/cheburashka
+sd-concepts-library/chen-1
+sd-concepts-library/child-zombie
+sd-concepts-library/chillpill
+sd-concepts-library/chonkfrog
+sd-concepts-library/chop
+sd-concepts-library/christo-person
+sd-concepts-library/chuck-walton
+sd-concepts-library/chucky
+sd-concepts-library/chungus-poodl-pet
+sd-concepts-library/cindlop
+sd-concepts-library/collage-cutouts
+sd-concepts-library/collage14
+sd-concepts-library/collage3
+sd-concepts-library/collage3-hubcity
+sd-concepts-library/cologne
+sd-concepts-library/color-page
+sd-concepts-library/colossus
+sd-concepts-library/command-and-conquer-remastered-cameos
+sd-concepts-library/concept-art
+sd-concepts-library/conner-fawcett-style
+sd-concepts-library/conway-pirate
+sd-concepts-library/coop-himmelblau
+sd-concepts-library/coraline
+sd-concepts-library/cornell-box
+sd-concepts-library/cortana
+sd-concepts-library/covid-19-rapid-test
+sd-concepts-library/cow-uwu
+sd-concepts-library/cowboy
+sd-concepts-library/crazy-1
+sd-concepts-library/crazy-2
+sd-concepts-library/crb-portraits
+sd-concepts-library/crb-surrealz
+sd-concepts-library/crbart
+sd-concepts-library/crested-gecko
+sd-concepts-library/crinos-form-garou
+sd-concepts-library/cry-baby-style
+sd-concepts-library/crybaby-style-2-0
+sd-concepts-library/csgo-awp-object
+sd-concepts-library/csgo-awp-texture-map
+sd-concepts-library/cubex
+sd-concepts-library/cumbia-peruana
+sd-concepts-library/cute-bear
+sd-concepts-library/cute-cat
+sd-concepts-library/cute-game-style
+sd-concepts-library/cyberpunk-lucy
+sd-concepts-library/dabotap
+sd-concepts-library/dan-mumford
+sd-concepts-library/dan-seagrave-art-style
+sd-concepts-library/dark-penguin-pinguinanimations
+sd-concepts-library/darkpenguinanimatronic
+sd-concepts-library/darkplane
+sd-concepts-library/david-firth-artstyle
+sd-concepts-library/david-martinez-cyberpunk
+sd-concepts-library/david-martinez-edgerunners
+sd-concepts-library/david-moreno-architecture
+sd-concepts-library/daycare-attendant-sun-fnaf
+sd-concepts-library/ddattender
+sd-concepts-library/degods
+sd-concepts-library/degodsheavy
+sd-concepts-library/depthmap
+sd-concepts-library/depthmap-style
+sd-concepts-library/design
+sd-concepts-library/detectivedinosaur1
+sd-concepts-library/diaosu-toy
+sd-concepts-library/dicoo
+sd-concepts-library/dicoo2
+sd-concepts-library/dishonored-portrait-styles
+sd-concepts-library/disquieting-muses
+sd-concepts-library/ditko
+sd-concepts-library/dlooak
+sd-concepts-library/doc
+sd-concepts-library/doener-red-line-art
+sd-concepts-library/dog
+sd-concepts-library/dog-django
+sd-concepts-library/doge-pound
+sd-concepts-library/dong-ho
+sd-concepts-library/dong-ho2
+sd-concepts-library/doose-s-realistic-art-style
+sd-concepts-library/dq10-anrushia
+sd-concepts-library/dr-livesey
+sd-concepts-library/dr-strange
+sd-concepts-library/dragonborn
+sd-concepts-library/dreamcore
+sd-concepts-library/dreamy-painting
+sd-concepts-library/drive-scorpion-jacket
+sd-concepts-library/dsmuses
+sd-concepts-library/dtv-pkmn
+sd-concepts-library/dullboy-caricature
+sd-concepts-library/duranduran
+sd-concepts-library/durer-style
+sd-concepts-library/dyoudim-style
+sd-concepts-library/early-mishima-kurone
+sd-concepts-library/eastward
+sd-concepts-library/eddie
+sd-concepts-library/edgerunners-style
+sd-concepts-library/edgerunners-style-v2
+sd-concepts-library/el-salvador-style-style
+sd-concepts-library/elegant-flower
+sd-concepts-library/elspeth-tirel
+sd-concepts-library/eru-chitanda-casual
+sd-concepts-library/erwin-olaf-style
+sd-concepts-library/ettblackteapot
+sd-concepts-library/explosions-cat
+sd-concepts-library/eye-of-agamotto
+sd-concepts-library/f-22
+sd-concepts-library/facadeplace
+sd-concepts-library/fairy-tale-painting-style
+sd-concepts-library/fairytale
+sd-concepts-library/fang-yuan-001
+sd-concepts-library/faraon-love-shady
+sd-concepts-library/fasina
+sd-concepts-library/felps
+sd-concepts-library/female-kpop-singer
+sd-concepts-library/fergal-cat
+sd-concepts-library/filename-2
+sd-concepts-library/fileteado-porteno
+sd-concepts-library/final-fantasy-logo
+sd-concepts-library/fireworks-over-water
+sd-concepts-library/fish
+sd-concepts-library/flag-ussr
+sd-concepts-library/flatic
+sd-concepts-library/floral
+sd-concepts-library/fluid-acrylic-jellyfish-creatures-style-of-carl-ingram-art
+sd-concepts-library/fnf-boyfriend
+sd-concepts-library/fold-structure
+sd-concepts-library/fox-purple
+sd-concepts-library/fractal
+sd-concepts-library/fractal-flame
+sd-concepts-library/fractal-temple-style
+sd-concepts-library/frank-frazetta
+sd-concepts-library/franz-unterberger
+sd-concepts-library/freddy-fazbear
+sd-concepts-library/freefonix-style
+sd-concepts-library/furrpopasthetic
+sd-concepts-library/fursona
+sd-concepts-library/fzk
+sd-concepts-library/galaxy-explorer
+sd-concepts-library/ganyu-genshin-impact
+sd-concepts-library/garcon-the-cat
+sd-concepts-library/garfield-pizza-plush
+sd-concepts-library/garfield-pizza-plush-v2
+sd-concepts-library/gba-fe-class-cards
+sd-concepts-library/gba-pokemon-sprites
+sd-concepts-library/geggin
+sd-concepts-library/ggplot2
+sd-concepts-library/ghost-style
+sd-concepts-library/ghostproject-men
+sd-concepts-library/gibasachan-v0
+sd-concepts-library/gim
+sd-concepts-library/gio
+sd-concepts-library/giygas
+sd-concepts-library/glass-pipe
+sd-concepts-library/glass-prism-cube
+sd-concepts-library/glow-forest
+sd-concepts-library/goku
+sd-concepts-library/gram-tops
+sd-concepts-library/green-blue-shanshui
+sd-concepts-library/green-tent
+sd-concepts-library/grifter
+sd-concepts-library/grisstyle
+sd-concepts-library/grit-toy
+sd-concepts-library/gt-color-paint-2
+sd-concepts-library/gta5-artwork
+sd-concepts-library/guttestreker
+sd-concepts-library/gymnastics-leotard-v2
+sd-concepts-library/half-life-2-dog
+sd-concepts-library/handstand
+sd-concepts-library/hanfu-anime-style
+sd-concepts-library/happy-chaos
+sd-concepts-library/happy-person12345
+sd-concepts-library/happy-person12345-assets
+sd-concepts-library/harley-quinn
+sd-concepts-library/harmless-ai-1
+sd-concepts-library/harmless-ai-house-style-1
+sd-concepts-library/hd-emoji
+sd-concepts-library/heather
+sd-concepts-library/henjo-techno-show
+sd-concepts-library/herge-style
+sd-concepts-library/hiten-style-nao
+sd-concepts-library/hitokomoru-style-nao
+sd-concepts-library/hiyuki-chan
+sd-concepts-library/hk-bamboo
+sd-concepts-library/hk-betweenislands
+sd-concepts-library/hk-bicycle
+sd-concepts-library/hk-blackandwhite
+sd-concepts-library/hk-breakfast
+sd-concepts-library/hk-buses
+sd-concepts-library/hk-clouds
+sd-concepts-library/hk-goldbuddha
+sd-concepts-library/hk-goldenlantern
+sd-concepts-library/hk-hkisland
+sd-concepts-library/hk-leaves
+sd-concepts-library/hk-market
+sd-concepts-library/hk-oldcamera
+sd-concepts-library/hk-opencamera
+sd-concepts-library/hk-peach
+sd-concepts-library/hk-phonevax
+sd-concepts-library/hk-streetpeople
+sd-concepts-library/hk-vintage
+sd-concepts-library/hoi4
+sd-concepts-library/hoi4-leaders
+sd-concepts-library/homestuck-sprite
+sd-concepts-library/homestuck-troll
+sd-concepts-library/hours-sentry-fade
+sd-concepts-library/hours-style
+sd-concepts-library/hrgiger-drmacabre
+sd-concepts-library/huang-guang-jian
+sd-concepts-library/huatli
+sd-concepts-library/huayecai820-greyscale
+sd-concepts-library/hub-city
+sd-concepts-library/hubris-oshri
+sd-concepts-library/huckleberry
+sd-concepts-library/hydrasuit
+sd-concepts-library/i-love-chaos
+sd-concepts-library/ibere-thenorio
+sd-concepts-library/ic0n
+sd-concepts-library/ie-gravestone
+sd-concepts-library/ikea-fabler
+sd-concepts-library/illustration-style
+sd-concepts-library/ilo-kunst
+sd-concepts-library/ilya-shkipin
+sd-concepts-library/im-poppy
+sd-concepts-library/ina-art
+sd-concepts-library/indian-watercolor-portraits
+sd-concepts-library/indiana
+sd-concepts-library/ingmar-bergman
+sd-concepts-library/insidewhale
+sd-concepts-library/interchanges
+sd-concepts-library/inuyama-muneto-style-nao
+sd-concepts-library/irasutoya
+sd-concepts-library/iridescent-illustration-style
+sd-concepts-library/iridescent-photo-style
+sd-concepts-library/isabell-schulte-pv-pvii-3000steps
+sd-concepts-library/isabell-schulte-pviii-1-image-style
+sd-concepts-library/isabell-schulte-pviii-1024px-1500-steps-style
+sd-concepts-library/isabell-schulte-pviii-12tiles-3000steps-style
+sd-concepts-library/isabell-schulte-pviii-4-tiles-1-lr-3000-steps-style
+sd-concepts-library/isabell-schulte-pviii-4-tiles-3-lr-5000-steps-style
+sd-concepts-library/isabell-schulte-pviii-4tiles-500steps
+sd-concepts-library/isabell-schulte-pviii-4tiles-6000steps
+sd-concepts-library/isabell-schulte-pviii-style
+sd-concepts-library/isometric-tile-test
+sd-concepts-library/jacqueline-the-unicorn
+sd-concepts-library/james-web-space-telescope
+sd-concepts-library/jamie-hewlett-style
+sd-concepts-library/jamiels
+sd-concepts-library/jang-sung-rak-style
+sd-concepts-library/jetsetdreamcastcovers
+sd-concepts-library/jin-kisaragi
+sd-concepts-library/jinjoon-lee-they
+sd-concepts-library/jm-bergling-monogram
+sd-concepts-library/joe-mad
+sd-concepts-library/joe-whiteford-art-style
+sd-concepts-library/joemad
+sd-concepts-library/john-blanche
+sd-concepts-library/johnny-silverhand
+sd-concepts-library/jojo-bizzare-adventure-manga-lineart
+sd-concepts-library/jos-de-kat
+sd-concepts-library/junji-ito-artstyle
+sd-concepts-library/kaleido
+sd-concepts-library/kaneoya-sachiko
+sd-concepts-library/kanovt
+sd-concepts-library/kanv1
+sd-concepts-library/karan-gloomy
+sd-concepts-library/karl-s-lzx-1
+sd-concepts-library/kasumin
+sd-concepts-library/kawaii-colors
+sd-concepts-library/kawaii-girl-plus-object
+sd-concepts-library/kawaii-girl-plus-style
+sd-concepts-library/kawaii-girl-plus-style-v1-1
+sd-concepts-library/kay
+sd-concepts-library/kaya-ghost-assasin
+sd-concepts-library/ki
+sd-concepts-library/kinda-sus
+sd-concepts-library/kings-quest-agd
+sd-concepts-library/kiora
+sd-concepts-library/kira-sensei
+sd-concepts-library/kirby
+sd-concepts-library/klance
+sd-concepts-library/kodakvision500t
+sd-concepts-library/kogatan-shiny
+sd-concepts-library/kogecha
+sd-concepts-library/kojima-ayami
+sd-concepts-library/koko-dog
+sd-concepts-library/kuvshinov
+sd-concepts-library/kysa-v-style
+sd-concepts-library/laala-character
+sd-concepts-library/larrette
+sd-concepts-library/lavko
+sd-concepts-library/lazytown-stephanie
+sd-concepts-library/ldr
+sd-concepts-library/ldrs
+sd-concepts-library/led-toy
+sd-concepts-library/lego-astronaut
+sd-concepts-library/leica
+sd-concepts-library/leif-jones
+sd-concepts-library/lex
+sd-concepts-library/liliana
+sd-concepts-library/liliana-vess
+sd-concepts-library/liminal-spaces-2-0
+sd-concepts-library/liminalspaces
+sd-concepts-library/line-art
+sd-concepts-library/line-style
+sd-concepts-library/linnopoke
+sd-concepts-library/liquid-light
+sd-concepts-library/liqwid-aquafarmer
+sd-concepts-library/lizardman
+sd-concepts-library/loab-character
+sd-concepts-library/loab-style
+sd-concepts-library/lofa
+sd-concepts-library/logo-with-face-on-shield
+sd-concepts-library/lolo
+sd-concepts-library/looney-anime
+sd-concepts-library/lost-rapper
+sd-concepts-library/lphr-style
+sd-concepts-library/lucario
+sd-concepts-library/lucky-luke
+sd-concepts-library/lugal-ki-en
+sd-concepts-library/luinv2
+sd-concepts-library/lula-13
+sd-concepts-library/lumio
+sd-concepts-library/lxj-o4
+sd-concepts-library/m-geo
+sd-concepts-library/m-geoo
+sd-concepts-library/madhubani-art
+sd-concepts-library/mafalda-character
+sd-concepts-library/magic-pengel
+sd-concepts-library/malika-favre-art-style
+sd-concepts-library/manga-style
+sd-concepts-library/marbling-art
+sd-concepts-library/margo
+sd-concepts-library/marty
+sd-concepts-library/marty6
+sd-concepts-library/mass
+sd-concepts-library/masyanya
+sd-concepts-library/masyunya
+sd-concepts-library/mate
+sd-concepts-library/matthew-stone
+sd-concepts-library/mattvidpro
+sd-concepts-library/maurice-quentin-de-la-tour-style
+sd-concepts-library/maus
+sd-concepts-library/max-foley
+sd-concepts-library/mayor-richard-irvin
+sd-concepts-library/mechasoulall
+sd-concepts-library/medazzaland
+sd-concepts-library/memnarch-mtg
+sd-concepts-library/metagabe
+sd-concepts-library/meyoco
+sd-concepts-library/meze-audio-elite-headphones
+sd-concepts-library/midjourney-style
+sd-concepts-library/mikako-method
+sd-concepts-library/mikako-methodi2i
+sd-concepts-library/miko-3-robot
+sd-concepts-library/milady
+sd-concepts-library/mildemelwe-style
+sd-concepts-library/million-live-akane-15k
+sd-concepts-library/million-live-akane-3k
+sd-concepts-library/million-live-akane-shifuku-3k
+sd-concepts-library/million-live-spade-q-object-3k
+sd-concepts-library/million-live-spade-q-style-3k
+sd-concepts-library/minecraft-concept-art
+sd-concepts-library/mishima-kurone
+sd-concepts-library/mizkif
+sd-concepts-library/moeb-style
+sd-concepts-library/moebius
+sd-concepts-library/mokoko
+sd-concepts-library/mokoko-seed
+sd-concepts-library/monster-girl
+sd-concepts-library/monster-toy
+sd-concepts-library/monte-novo
+sd-concepts-library/moo-moo
+sd-concepts-library/morino-hon-style
+sd-concepts-library/moxxi
+sd-concepts-library/msg
+sd-concepts-library/mtg-card
+sd-concepts-library/mtl-longsky
+sd-concepts-library/mu-sadr
+sd-concepts-library/munch-leaks-style
+sd-concepts-library/museum-by-coop-himmelblau
+sd-concepts-library/muxoyara
+sd-concepts-library/my-hero-academia-style
+sd-concepts-library/my-mug
+sd-concepts-library/mycat
+sd-concepts-library/mystical-nature
+sd-concepts-library/naf
+sd-concepts-library/nahiri
+sd-concepts-library/namine-ritsu
+sd-concepts-library/naoki-saito
+sd-concepts-library/nard-style
+sd-concepts-library/naruto
+sd-concepts-library/natasha-johnston
+sd-concepts-library/nathan-wyatt
+sd-concepts-library/naval-portrait
+sd-concepts-library/nazuna
+sd-concepts-library/nebula
+sd-concepts-library/ned-flanders
+sd-concepts-library/neon-pastel
+sd-concepts-library/new-priests
+sd-concepts-library/nic-papercuts
+sd-concepts-library/nikodim
+sd-concepts-library/nissa-revane
+sd-concepts-library/nixeu
+sd-concepts-library/noggles
+sd-concepts-library/nomad
+sd-concepts-library/nouns-glasses
+sd-concepts-library/obama-based-on-xi
+sd-concepts-library/obama-self-2
+sd-concepts-library/og-mox-style
+sd-concepts-library/ohisashiburi-style
+sd-concepts-library/oleg-kuvaev
+sd-concepts-library/olli-olli
+sd-concepts-library/on-kawara
+sd-concepts-library/one-line-drawing
+sd-concepts-library/onepunchman
+sd-concepts-library/onzpo
+sd-concepts-library/orangejacket
+sd-concepts-library/ori
+sd-concepts-library/ori-toor
+sd-concepts-library/orientalist-art
+sd-concepts-library/osaka-jyo
+sd-concepts-library/osaka-jyo2
+sd-concepts-library/osrsmini2
+sd-concepts-library/osrstiny
+sd-concepts-library/other-mother
+sd-concepts-library/ouroboros
+sd-concepts-library/outfit-items
+sd-concepts-library/overprettified
+sd-concepts-library/owl-house
+sd-concepts-library/painted-by-silver-of-999
+sd-concepts-library/painted-by-silver-of-999-2
+sd-concepts-library/painted-student
+sd-concepts-library/painting
+sd-concepts-library/pantone-milk
+sd-concepts-library/paolo-bonolis
+sd-concepts-library/party-girl
+sd-concepts-library/pascalsibertin
+sd-concepts-library/pastelartstyle
+sd-concepts-library/paul-noir
+sd-concepts-library/pen-ink-portraits-bennorthen
+sd-concepts-library/phan
+sd-concepts-library/phan-s-collage
+sd-concepts-library/phc
+sd-concepts-library/phoenix-01
+sd-concepts-library/pineda-david
+sd-concepts-library/pink-beast-pastelae-style
+sd-concepts-library/pintu
+sd-concepts-library/pion-by-august-semionov
+sd-concepts-library/piotr-jablonski
+sd-concepts-library/pixel-mania
+sd-concepts-library/pixel-toy
+sd-concepts-library/pjablonski-style
+sd-concepts-library/plant-style
+sd-concepts-library/plen-ki-mun
+sd-concepts-library/pokemon-conquest-sprites
+sd-concepts-library/pool-test
+sd-concepts-library/poolrooms
+sd-concepts-library/poring-ragnarok-online
+sd-concepts-library/poutine-dish
+sd-concepts-library/princess-knight-art
+sd-concepts-library/progress-chip
+sd-concepts-library/puerquis-toy
+sd-concepts-library/purplefishli
+sd-concepts-library/pyramidheadcosplay
+sd-concepts-library/qpt-atrium
+sd-concepts-library/quiesel
+sd-concepts-library/r-crumb-style
+sd-concepts-library/rahkshi-bionicle
+sd-concepts-library/raichu
+sd-concepts-library/rail-scene
+sd-concepts-library/rail-scene-style
+sd-concepts-library/ralph-mcquarrie
+sd-concepts-library/ransom
+sd-concepts-library/rayne-weynolds
+sd-concepts-library/rcrumb-portraits-style
+sd-concepts-library/rd-chaos
+sd-concepts-library/rd-paintings
+sd-concepts-library/red-glasses
+sd-concepts-library/reeducation-camp
+sd-concepts-library/reksio-dog
+sd-concepts-library/rektguy
+sd-concepts-library/remert
+sd-concepts-library/renalla
+sd-concepts-library/repeat
+sd-concepts-library/retro-girl
+sd-concepts-library/retro-mecha-rangers
+sd-concepts-library/retropixelart-pinguin
+sd-concepts-library/rex-deno
+sd-concepts-library/rhizomuse-machine-bionic-sculpture
+sd-concepts-library/ricar
+sd-concepts-library/rickyart
+sd-concepts-library/rico-face
+sd-concepts-library/riker-doll
+sd-concepts-library/rikiart
+sd-concepts-library/rikiboy-art
+sd-concepts-library/rilakkuma
+sd-concepts-library/rishusei-style
+sd-concepts-library/rj-palmer
+sd-concepts-library/rl-pkmn-test
+sd-concepts-library/road-to-ruin
+sd-concepts-library/robertnava
+sd-concepts-library/roblox-avatar
+sd-concepts-library/roy-lichtenstein
+sd-concepts-library/ruan-jia
+sd-concepts-library/russian
+sd-concepts-library/s1m-naoto-ohshima
+sd-concepts-library/saheeli-rai
+sd-concepts-library/sakimi-style
+sd-concepts-library/salmonid
+sd-concepts-library/sam-yang
+sd-concepts-library/sanguo-guanyu
+sd-concepts-library/sas-style
+sd-concepts-library/scarlet-witch
+sd-concepts-library/schloss-mosigkau
+sd-concepts-library/scrap-style
+sd-concepts-library/scratch-project
+sd-concepts-library/sculptural-style
+sd-concepts-library/sd-concepts-library-uma-meme
+sd-concepts-library/seamless-ground
+sd-concepts-library/selezneva-alisa
+sd-concepts-library/sem-mac2n
+sd-concepts-library/senneca
+sd-concepts-library/seraphimmoonshadow-art
+sd-concepts-library/sewerslvt
+sd-concepts-library/she-hulk-law-art
+sd-concepts-library/she-mask
+sd-concepts-library/sherhook-painting
+sd-concepts-library/sherhook-painting-v2
+sd-concepts-library/shev-linocut
+sd-concepts-library/shigure-ui-style
+sd-concepts-library/shiny-polyman
+sd-concepts-library/shrunken-head
+sd-concepts-library/shu-doll
+sd-concepts-library/shvoren-style
+sd-concepts-library/sims-2-portrait
+sd-concepts-library/singsing
+sd-concepts-library/singsing-doll
+sd-concepts-library/sintez-ico
+sd-concepts-library/skyfalls
+sd-concepts-library/slm
+sd-concepts-library/smarties
+sd-concepts-library/smiling-friend-style
+sd-concepts-library/smooth-pencils
+sd-concepts-library/smurf-style
+sd-concepts-library/smw-map
+sd-concepts-library/society-finch
+sd-concepts-library/sorami-style
+sd-concepts-library/spider-gwen
+sd-concepts-library/spritual-monsters
+sd-concepts-library/stable-diffusion-conceptualizer
+sd-concepts-library/star-tours-posters
+sd-concepts-library/stardew-valley-pixel-art
+sd-concepts-library/starhavenmachinegods
+sd-concepts-library/sterling-archer
+sd-concepts-library/stretch-re1-robot
+sd-concepts-library/stuffed-penguin-toy
+sd-concepts-library/style-of-marc-allante
+sd-concepts-library/summie-style
+sd-concepts-library/sunfish
+sd-concepts-library/super-nintendo-cartridge
+sd-concepts-library/supitcha-mask
+sd-concepts-library/sushi-pixel
+sd-concepts-library/swamp-choe-2
+sd-concepts-library/t-skrang
+sd-concepts-library/takuji-kawano
+sd-concepts-library/tamiyo
+sd-concepts-library/tangles
+sd-concepts-library/tb303
+sd-concepts-library/tcirle
+sd-concepts-library/teelip-ir-landscape
+sd-concepts-library/teferi
+sd-concepts-library/tela-lenca
+sd-concepts-library/tela-lenca2
+sd-concepts-library/terraria-style
+sd-concepts-library/tesla-bot
+sd-concepts-library/test
+sd-concepts-library/test-epson
+sd-concepts-library/test2
+sd-concepts-library/testing
+sd-concepts-library/thalasin
+sd-concepts-library/thegeneral
+sd-concepts-library/thorneworks
+sd-concepts-library/threestooges
+sd-concepts-library/thunderdome-cover
+sd-concepts-library/thunderdome-covers
+sd-concepts-library/ti-junglepunk-v0
+sd-concepts-library/tili-concept
+sd-concepts-library/titan-robot
+sd-concepts-library/tnj
+sd-concepts-library/toho-pixel
+sd-concepts-library/tomcat
+sd-concepts-library/tonal1
+sd-concepts-library/tony-diterlizzi-s-planescape-art
+sd-concepts-library/towerplace
+sd-concepts-library/toy
+sd-concepts-library/toy-bonnie-plush
+sd-concepts-library/toyota-sera
+sd-concepts-library/transmutation-circles
+sd-concepts-library/trash-polka-artstyle
+sd-concepts-library/travis-bedel
+sd-concepts-library/trigger-studio
+sd-concepts-library/trust-support
+sd-concepts-library/trypophobia
+sd-concepts-library/ttte
+sd-concepts-library/tubby
+sd-concepts-library/tubby-cats
+sd-concepts-library/tudisco
+sd-concepts-library/turtlepics
+sd-concepts-library/type
+sd-concepts-library/ugly-sonic
+sd-concepts-library/uliana-kudinova
+sd-concepts-library/uma
+sd-concepts-library/uma-clean-object
+sd-concepts-library/uma-meme
+sd-concepts-library/uma-meme-style
+sd-concepts-library/uma-style-classic
+sd-concepts-library/unfinished-building
+sd-concepts-library/urivoldemort
+sd-concepts-library/uzumaki
+sd-concepts-library/valorantstyle
+sd-concepts-library/vb-mox
+sd-concepts-library/vcr-classique
+sd-concepts-library/venice
+sd-concepts-library/vespertine
+sd-concepts-library/victor-narm
+sd-concepts-library/vietstoneking
+sd-concepts-library/vivien-reid
+sd-concepts-library/vkuoo1
+sd-concepts-library/vraska
+sd-concepts-library/w3u
+sd-concepts-library/walter-wick-photography
+sd-concepts-library/warhammer-40k-drawing-style
+sd-concepts-library/waterfallshadow
+sd-concepts-library/wayne-reynolds-character
+sd-concepts-library/wedding
+sd-concepts-library/wedding-HandPainted
+sd-concepts-library/werebloops
+sd-concepts-library/wheatland
+sd-concepts-library/wheatland-arknight
+sd-concepts-library/wheelchair
+sd-concepts-library/wildkat
+sd-concepts-library/willy-hd
+sd-concepts-library/wire-angels
+sd-concepts-library/wish-artist-stile
+sd-concepts-library/wlop-style
+sd-concepts-library/wojak
+sd-concepts-library/wojaks-now
+sd-concepts-library/wojaks-now-now-now
+sd-concepts-library/xatu
+sd-concepts-library/xatu2
+sd-concepts-library/xbh
+sd-concepts-library/xi
+sd-concepts-library/xidiversity
+sd-concepts-library/xioboma
+sd-concepts-library/xuna
+sd-concepts-library/xyz
+sd-concepts-library/yb-anime
+sd-concepts-library/yerba-mate
+sd-concepts-library/yesdelete
+sd-concepts-library/yf21
+sd-concepts-library/yilanov2
+sd-concepts-library/yinit
+sd-concepts-library/yoji-shinkawa-style
+sd-concepts-library/yolandi-visser
+sd-concepts-library/yoshi
+sd-concepts-library/youpi2
+sd-concepts-library/youtooz-candy
+sd-concepts-library/yuji-himukai-style
+sd-concepts-library/zaney
+sd-concepts-library/zaneypixelz
+sd-concepts-library/zdenek-art
+sd-concepts-library/zero
+sd-concepts-library/zero-bottle
+sd-concepts-library/zero-suit-samus
+sd-concepts-library/zillertal-can
+sd-concepts-library/zizigooloo
+sd-concepts-library/zk
+sd-concepts-library/zoroark

configs/stable-diffusion/v1-finetune.yaml

@@ -0,0 +1,110 @@
+model:
+  base_learning_rate: 5.0e-03
+  target: ldm.models.diffusion.ddpm.LatentDiffusion
+  params:
+    linear_start: 0.00085
+    linear_end: 0.0120
+    num_timesteps_cond: 1
+    log_every_t: 200
+    timesteps: 1000
+    first_stage_key: image
+    cond_stage_key: caption
+    image_size: 64
+    channels: 4
+    cond_stage_trainable: true   # Note: different from the one we trained before
+    conditioning_key: crossattn
+    monitor: val/loss_simple_ema
+    scale_factor: 0.18215
+    use_ema: False
+    embedding_reg_weight: 0.0
+
+    personalization_config:
+      target: ldm.modules.embedding_manager.EmbeddingManager
+      params:
+        placeholder_strings: ["*"]
+        initializer_words: ["sculpture"]
+        per_image_tokens: false
+        num_vectors_per_token: 1
+        progressive_words: False
+
+    unet_config:
+      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
+      params:
+        image_size: 32 # unused
+        in_channels: 4
+        out_channels: 4
+        model_channels: 320
+        attention_resolutions: [ 4, 2, 1 ]
+        num_res_blocks: 2
+        channel_mult: [ 1, 2, 4, 4 ]
+        num_heads: 8
+        use_spatial_transformer: True
+        transformer_depth: 1
+        context_dim: 768
+        use_checkpoint: True
+        legacy: False
+
+    first_stage_config:
+      target: ldm.models.autoencoder.AutoencoderKL
+      params:
+        embed_dim: 4
+        monitor: val/rec_loss
+        ddconfig:
+          double_z: true
+          z_channels: 4
+          resolution: 256
+          in_channels: 3
+          out_ch: 3
+          ch: 128
+          ch_mult:
+          - 1
+          - 2
+          - 4
+          - 4
+          num_res_blocks: 2
+          attn_resolutions: []
+          dropout: 0.0
+        lossconfig:
+          target: torch.nn.Identity
+
+    cond_stage_config:
+      target: ldm.modules.encoders.modules.FrozenCLIPEmbedder
+
+data:
+  target: main.DataModuleFromConfig
+  params:
+    batch_size: 1
+    num_workers: 2
+    wrap: false
+    train:
+      target: ldm.data.personalized.PersonalizedBase
+      params:
+        size: 512
+        set: train
+        per_image_tokens: false
+        repeats: 100
+    validation:
+      target: ldm.data.personalized.PersonalizedBase
+      params:
+        size: 512
+        set: val
+        per_image_tokens: false
+        repeats: 10
+
+lightning:
+  modelcheckpoint:
+    params:
+      every_n_train_steps: 500
+  callbacks:
+    image_logger:
+      target: main.ImageLogger
+      params:
+        batch_frequency: 500
+        max_images: 8
+        increase_log_steps: False
+
+  trainer:
+    benchmark: True
+    max_steps: 4000000
+#    max_steps: 4000
+    

configs/stable-diffusion/v1-finetune_style.yaml

@@ -0,0 +1,103 @@
+model:
+  base_learning_rate: 5.0e-03
+  target: ldm.models.diffusion.ddpm.LatentDiffusion
+  params:
+    linear_start: 0.00085
+    linear_end: 0.0120
+    num_timesteps_cond: 1
+    log_every_t: 200
+    timesteps: 1000
+    first_stage_key: image
+    cond_stage_key: caption
+    image_size: 64
+    channels: 4
+    cond_stage_trainable: true   # Note: different from the one we trained before
+    conditioning_key: crossattn
+    monitor: val/loss_simple_ema
+    scale_factor: 0.18215
+    use_ema: False
+    embedding_reg_weight: 0.0
+
+    personalization_config:
+      target: ldm.modules.embedding_manager.EmbeddingManager
+      params:
+        placeholder_strings: ["*"]
+        initializer_words: ["painting"]
+        per_image_tokens: false
+        num_vectors_per_token: 1
+
+    unet_config:
+      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
+      params:
+        image_size: 32 # unused
+        in_channels: 4
+        out_channels: 4
+        model_channels: 320
+        attention_resolutions: [ 4, 2, 1 ]
+        num_res_blocks: 2
+        channel_mult: [ 1, 2, 4, 4 ]
+        num_heads: 8
+        use_spatial_transformer: True
+        transformer_depth: 1
+        context_dim: 768
+        use_checkpoint: True
+        legacy: False
+
+    first_stage_config:
+      target: ldm.models.autoencoder.AutoencoderKL
+      params:
+        embed_dim: 4
+        monitor: val/rec_loss
+        ddconfig:
+          double_z: true
+          z_channels: 4
+          resolution: 256
+          in_channels: 3
+          out_ch: 3
+          ch: 128
+          ch_mult:
+          - 1
+          - 2
+          - 4
+          - 4
+          num_res_blocks: 2
+          attn_resolutions: []
+          dropout: 0.0
+        lossconfig:
+          target: torch.nn.Identity
+
+    cond_stage_config:
+      target: ldm.modules.encoders.modules.FrozenCLIPEmbedder
+
+data:
+  target: main.DataModuleFromConfig
+  params:
+    batch_size: 2
+    num_workers: 16
+    wrap: false
+    train:
+      target: ldm.data.personalized_style.PersonalizedBase
+      params:
+        size: 512
+        set: train
+        per_image_tokens: false
+        repeats: 100
+    validation:
+      target: ldm.data.personalized_style.PersonalizedBase
+      params:
+        size: 512
+        set: val
+        per_image_tokens: false
+        repeats: 10
+
+lightning:
+  callbacks:
+    image_logger:
+      target: main.ImageLogger
+      params:
+        batch_frequency: 500
+        max_images: 8
+        increase_log_steps: False
+
+  trainer:
+    benchmark: True

configs/stable-diffusion/v1-inference.yaml

@@ -26,6 +26,15 @@ model:
         f_max: [ 1. ]
         f_min: [ 1. ]
 
+    personalization_config:
+      target: ldm.modules.embedding_manager.EmbeddingManager
+      params:
+        placeholder_strings: ["*"]
+        initializer_words: ['sculpture']
+        per_image_tokens: false
+        num_vectors_per_token: 8
+        progressive_words: False
+
     unet_config:
       target: ldm.modules.diffusionmodules.openaimodel.UNetModel
       params:
@@ -67,4 +76,4 @@ model:
           target: torch.nn.Identity
 
     cond_stage_config:
-      target: ldm.modules.encoders.modules.FrozenCLIPEmbedder
+      target: ldm.modules.encoders.modules.WeightedFrozenCLIPEmbedder

configs/stable-diffusion/v1-inpainting-inference.yaml

@@ -0,0 +1,79 @@
+model:
+  base_learning_rate: 7.5e-05
+  target: ldm.models.diffusion.ddpm.LatentInpaintDiffusion
+  params:
+    linear_start: 0.00085
+    linear_end: 0.0120
+    num_timesteps_cond: 1
+    log_every_t: 200
+    timesteps: 1000
+    first_stage_key: "jpg"
+    cond_stage_key: "txt"
+    image_size: 64
+    channels: 4
+    cond_stage_trainable: false   # Note: different from the one we trained before
+    conditioning_key: hybrid   # important
+    monitor: val/loss_simple_ema
+    scale_factor: 0.18215
+    finetune_keys: null
+
+    scheduler_config: # 10000 warmup steps
+      target: ldm.lr_scheduler.LambdaLinearScheduler
+      params:
+        warm_up_steps: [ 2500 ] # NOTE for resuming. use 10000 if starting from scratch
+        cycle_lengths: [ 10000000000000 ] # incredibly large number to prevent corner cases
+        f_start: [ 1.e-6 ]
+        f_max: [ 1. ]
+        f_min: [ 1. ]
+
+    personalization_config:
+      target: ldm.modules.embedding_manager.EmbeddingManager
+      params:
+        placeholder_strings: ["*"]
+        initializer_words: ['sculpture']
+        per_image_tokens: false
+        num_vectors_per_token: 8
+        progressive_words: False
+
+    unet_config:
+      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
+      params:
+        image_size: 32 # unused
+        in_channels: 9  # 4 data + 4 downscaled image + 1 mask
+        out_channels: 4
+        model_channels: 320
+        attention_resolutions: [ 4, 2, 1 ]
+        num_res_blocks: 2
+        channel_mult: [ 1, 2, 4, 4 ]
+        num_heads: 8
+        use_spatial_transformer: True
+        transformer_depth: 1
+        context_dim: 768
+        use_checkpoint: True
+        legacy: False
+
+    first_stage_config:
+      target: ldm.models.autoencoder.AutoencoderKL
+      params:
+        embed_dim: 4
+        monitor: val/rec_loss
+        ddconfig:
+          double_z: true
+          z_channels: 4
+          resolution: 256
+          in_channels: 3
+          out_ch: 3
+          ch: 128
+          ch_mult:
+          - 1
+          - 2
+          - 4
+          - 4
+          num_res_blocks: 2
+          attn_resolutions: []
+          dropout: 0.0
+        lossconfig:
+          target: torch.nn.Identity
+
+    cond_stage_config:
+      target: ldm.modules.encoders.modules.WeightedFrozenCLIPEmbedder

configs/stable-diffusion/v1-m1-finetune.yaml

@@ -0,0 +1,110 @@
+model:
+  base_learning_rate: 5.0e-03
+  target: ldm.models.diffusion.ddpm.LatentDiffusion
+  params:
+    linear_start: 0.00085
+    linear_end: 0.0120
+    num_timesteps_cond: 1
+    log_every_t: 200
+    timesteps: 1000
+    first_stage_key: image
+    cond_stage_key: caption
+    image_size: 64
+    channels: 4
+    cond_stage_trainable: true   # Note: different from the one we trained before
+    conditioning_key: crossattn
+    monitor: val/loss_simple_ema
+    scale_factor: 0.18215
+    use_ema: False
+    embedding_reg_weight: 0.0
+
+    personalization_config:
+      target: ldm.modules.embedding_manager.EmbeddingManager
+      params:
+        placeholder_strings: ["*"]
+        initializer_words: ['sculpture']
+        per_image_tokens: false
+        num_vectors_per_token: 6
+        progressive_words: False
+
+    unet_config:
+      target: ldm.modules.diffusionmodules.openaimodel.UNetModel
+      params:
+        image_size: 32 # unused
+        in_channels: 4
+        out_channels: 4
+        model_channels: 320
+        attention_resolutions: [ 4, 2, 1 ]
+        num_res_blocks: 2
+        channel_mult: [ 1, 2, 4, 4 ]
+        num_heads: 8
+        use_spatial_transformer: True
+        transformer_depth: 1
+        context_dim: 768
+        use_checkpoint: True
+        legacy: False
+
+    first_stage_config:
+      target: ldm.models.autoencoder.AutoencoderKL
+      params:
+        embed_dim: 4
+        monitor: val/rec_loss
+        ddconfig:
+          double_z: true
+          z_channels: 4
+          resolution: 256
+          in_channels: 3
+          out_ch: 3
+          ch: 128
+          ch_mult:
+          - 1
+          - 2
+          - 4
+          - 4
+          num_res_blocks: 2
+          attn_resolutions: []
+          dropout: 0.0
+        lossconfig:
+          target: torch.nn.Identity
+
+    cond_stage_config:
+      target: ldm.modules.encoders.modules.FrozenCLIPEmbedder
+
+data:
+  target: main.DataModuleFromConfig
+  params:
+    batch_size: 1
+    num_workers: 2
+    wrap: false
+    train:
+      target: ldm.data.personalized.PersonalizedBase
+      params:
+        size: 512
+        set: train
+        per_image_tokens: false
+        repeats: 100
+    validation:
+      target: ldm.data.personalized.PersonalizedBase
+      params:
+        size: 512
+        set: val
+        per_image_tokens: false
+        repeats: 10
+
+lightning:
+  modelcheckpoint:
+    params:
+      every_n_train_steps: 500
+  callbacks:
+    image_logger:
+      target: main.ImageLogger
+      params:
+        batch_frequency: 500
+        max_images: 5
+        increase_log_steps: False
+
+  trainer:
+    benchmark: False
+    max_steps: 6200
+#    max_steps: 4000
+    

docker-build/Dockerfile

@@ -0,0 +1,34 @@
+FROM ubuntu:22.10
+
+# use bash
+SHELL [ "/bin/bash", "-c" ]
+
+# Install necesarry packages
+RUN apt-get update \
+  && apt-get install -y \
+    --no-install-recommends \
+    build-essential \
+    gcc \
+    git \
+    libgl1-mesa-glx \
+    libglib2.0-0 \
+    pip \
+    python3 \
+    python3-dev \
+  && apt-get clean \
+  && rm -rf /var/lib/apt/lists/*
+
+# set workdir and copy sources
+WORKDIR /invokeai
+ARG PIP_REQUIREMENTS=requirements-lin-cuda.txt
+COPY . ./environments-and-requirements/${PIP_REQUIREMENTS} ./
+
+# install requirements and link outputs folder
+RUN pip install \
+  --no-cache-dir \
+  -r ${PIP_REQUIREMENTS}
+
+# set Environment, Entrypoint and default CMD
+ENV INVOKEAI_ROOT /data
+ENTRYPOINT [ "python3", "scripts/invoke.py", "--outdir=/data/outputs" ]
+CMD [ "--web", "--host=0.0.0.0" ]

docker-build/build.sh

@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+set -e
+
+# IMPORTANT: You need to have a token on huggingface.co to be able to download the checkpoints!!!
+# configure values by using env when executing build.sh f.e. `env ARCH=aarch64 ./build.sh`
+
+source ./docker-build/env.sh \
+  || echo "please execute docker-build/build.sh from repository root" \
+  || exit 1
+
+pip_requirements=${PIP_REQUIREMENTS:-requirements-lin-cuda.txt}
+dockerfile=${INVOKE_DOCKERFILE:-docker-build/Dockerfile}
+
+# print the settings
+echo "You are using these values:"
+echo -e "Dockerfile:\t\t ${dockerfile}"
+echo -e "requirements:\t\t ${pip_requirements}"
+echo -e "volumename:\t\t ${volumename}"
+echo -e "arch:\t\t\t ${arch}"
+echo -e "platform:\t\t ${platform}"
+echo -e "invokeai_tag:\t\t ${invokeai_tag}\n"
+
+if [[ -n "$(docker volume ls -f name="${volumename}" -q)" ]]; then
+  echo "Volume already exists"
+  echo
+else
+  echo -n "createing docker volume "
+  docker volume create "${volumename}"
+fi
+
+# Build Container
+docker build \
+  --platform="${platform}" \
+  --tag="${invokeai_tag}" \
+  --build-arg="PIP_REQUIREMENTS=${pip_requirements}" \
+  --file="${dockerfile}" \
+  .
+
+docker run \
+  --rm \
+  --platform="$platform" \
+  --name="$project_name" \
+  --hostname="$project_name" \
+  --mount="source=$volumename,target=/data" \
+  --mount="type=bind,source=$HOME/.huggingface,target=/root/.huggingface" \
+  --env="HUGGINGFACE_TOKEN=${HUGGINGFACE_TOKEN}" \
+  --entrypoint="python3" \
+  "${invokeai_tag}" \
+  scripts/configure_invokeai.py --yes

docker-build/env.sh

@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+
+project_name=${PROJECT_NAME:-invokeai}
+volumename=${VOLUMENAME:-${project_name}_data}
+arch=${ARCH:-x86_64}
+platform=${PLATFORM:-Linux/${arch}}
+invokeai_tag=${INVOKEAI_TAG:-${project_name}:${arch}}
+
+export project_name
+export volumename
+export arch
+export platform
+export invokeai_tag

docker-build/run.sh

@@ -0,0 +1,15 @@
+#!/usr/bin/env bash
+set -e
+
+source ./docker-build/env.sh || echo "please run from repository root" || exit 1
+
+docker run \
+  --interactive \
+  --tty \
+  --rm \
+  --platform="$platform" \
+  --name="$project_name" \
+  --hostname="$project_name" \
+  --mount="source=$volumename,target=/data" \
+  --publish=9090:9090 \
+  "$invokeai_tag" ${1:+$@}

docs/CHANGELOG.md

@@ -0,0 +1,390 @@
+---
+title: Changelog
+---
+
+# :octicons-log-16: **Changelog**
+
+## v2.1.0 <small>(2 November 2022)</small>
+
+- update mac instructions to use invokeai for env name by @willwillems in
+  https://github.com/invoke-ai/InvokeAI/pull/1030
+- Update .gitignore by @blessedcoolant in
+  https://github.com/invoke-ai/InvokeAI/pull/1040
+- reintroduce fix for m1 from https://github.com/invoke-ai/InvokeAI/pull/579
+  missing after merge by @skurovec in
+  https://github.com/invoke-ai/InvokeAI/pull/1056
+- Update Stable_Diffusion_AI_Notebook.ipynb (Take 2) by @ChloeL19 in
+  https://github.com/invoke-ai/InvokeAI/pull/1060
+- Print out the device type which is used by @manzke in
+  https://github.com/invoke-ai/InvokeAI/pull/1073
+- Hires Addition by @hipsterusername in
+  https://github.com/invoke-ai/InvokeAI/pull/1063
+- fix for "1 leaked semaphore objects to clean up at shutdown" on M1 by
+  @skurovec in https://github.com/invoke-ai/InvokeAI/pull/1081
+- Forward dream.py to invoke.py using the same interpreter, add deprecation
+  warning by @db3000 in https://github.com/invoke-ai/InvokeAI/pull/1077
+- fix noisy images at high step counts by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1086
+- Generalize facetool strength argument by @db3000 in
+  https://github.com/invoke-ai/InvokeAI/pull/1078
+- Enable fast switching among models at the invoke> command line by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1066
+- Fix Typo, committed changing ldm environment to invokeai by @jdries3 in
+  https://github.com/invoke-ai/InvokeAI/pull/1095
+- Update generate.py by @unreleased in
+  https://github.com/invoke-ai/InvokeAI/pull/1109
+- Update 'ldm' env to 'invokeai' in troubleshooting steps by @19wolf in
+  https://github.com/invoke-ai/InvokeAI/pull/1125
+- Fixed documentation typos and resolved merge conflicts by @rupeshs in
+  https://github.com/invoke-ai/InvokeAI/pull/1123
+- Fix broken doc links, fix malaprop in the project subtitle by @majick in
+  https://github.com/invoke-ai/InvokeAI/pull/1131
+- Only output facetool parameters if enhancing faces by @db3000 in
+  https://github.com/invoke-ai/InvokeAI/pull/1119
+- Update gitignore to ignore codeformer weights at new location by
+  @spezialspezial in https://github.com/invoke-ai/InvokeAI/pull/1136
+- fix links to point to invoke-ai.github.io #1117 by @mauwii in
+  https://github.com/invoke-ai/InvokeAI/pull/1143
+- Rework-mkdocs by @mauwii in https://github.com/invoke-ai/InvokeAI/pull/1144
+- add option to CLI and pngwriter that allows user to set PNG compression level
+  by @lstein in https://github.com/invoke-ai/InvokeAI/pull/1127
+- Fix img2img DDIM index out of bound by @wfng92 in
+  https://github.com/invoke-ai/InvokeAI/pull/1137
+- Fix gh actions by @mauwii in https://github.com/invoke-ai/InvokeAI/pull/1128
+- update mac instructions to use invokeai for env name by @willwillems in
+  https://github.com/invoke-ai/InvokeAI/pull/1030
+- Update .gitignore by @blessedcoolant in
+  https://github.com/invoke-ai/InvokeAI/pull/1040
+- reintroduce fix for m1 from https://github.com/invoke-ai/InvokeAI/pull/579
+  missing after merge by @skurovec in
+  https://github.com/invoke-ai/InvokeAI/pull/1056
+- Update Stable_Diffusion_AI_Notebook.ipynb (Take 2) by @ChloeL19 in
+  https://github.com/invoke-ai/InvokeAI/pull/1060
+- Print out the device type which is used by @manzke in
+  https://github.com/invoke-ai/InvokeAI/pull/1073
+- Hires Addition by @hipsterusername in
+  https://github.com/invoke-ai/InvokeAI/pull/1063
+- fix for "1 leaked semaphore objects to clean up at shutdown" on M1 by
+  @skurovec in https://github.com/invoke-ai/InvokeAI/pull/1081
+- Forward dream.py to invoke.py using the same interpreter, add deprecation
+  warning by @db3000 in https://github.com/invoke-ai/InvokeAI/pull/1077
+- fix noisy images at high step counts by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1086
+- Generalize facetool strength argument by @db3000 in
+  https://github.com/invoke-ai/InvokeAI/pull/1078
+- Enable fast switching among models at the invoke> command line by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1066
+- Fix Typo, committed changing ldm environment to invokeai by @jdries3 in
+  https://github.com/invoke-ai/InvokeAI/pull/1095
+- Fixed documentation typos and resolved merge conflicts by @rupeshs in
+  https://github.com/invoke-ai/InvokeAI/pull/1123
+- Only output facetool parameters if enhancing faces by @db3000 in
+  https://github.com/invoke-ai/InvokeAI/pull/1119
+- add option to CLI and pngwriter that allows user to set PNG compression level
+  by @lstein in https://github.com/invoke-ai/InvokeAI/pull/1127
+- Fix img2img DDIM index out of bound by @wfng92 in
+  https://github.com/invoke-ai/InvokeAI/pull/1137
+- Add text prompt to inpaint mask support by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1133
+- Respect http[s] protocol when making socket.io middleware by @damian0815 in
+  https://github.com/invoke-ai/InvokeAI/pull/976
+- WebUI: Adds Codeformer support by @psychedelicious in
+  https://github.com/invoke-ai/InvokeAI/pull/1151
+- Skips normalizing prompts for web UI metadata by @psychedelicious in
+  https://github.com/invoke-ai/InvokeAI/pull/1165
+- Add Asymmetric Tiling by @carson-katri in
+  https://github.com/invoke-ai/InvokeAI/pull/1132
+- Web UI: Increases max CFG Scale to 200 by @psychedelicious in
+  https://github.com/invoke-ai/InvokeAI/pull/1172
+- Corrects color channels in face restoration; Fixes #1167 by @psychedelicious
+  in https://github.com/invoke-ai/InvokeAI/pull/1175
+- Flips channels using array slicing instead of using OpenCV by @psychedelicious
+  in https://github.com/invoke-ai/InvokeAI/pull/1178
+- Fix typo in docs: s/Formally/Formerly by @noodlebox in
+  https://github.com/invoke-ai/InvokeAI/pull/1176
+- fix clipseg loading problems by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1177
+- Correct color channels in upscale using array slicing by @wfng92 in
+  https://github.com/invoke-ai/InvokeAI/pull/1181
+- Web UI: Filters existing images when adding new images; Fixes #1085 by
+  @psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/1171
+- fix a number of bugs in textual inversion by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1190
+- Improve !fetch, add !replay command by @ArDiouscuros in
+  https://github.com/invoke-ai/InvokeAI/pull/882
+- Fix generation of image with s>1000 by @holstvoogd in
+  https://github.com/invoke-ai/InvokeAI/pull/951
+- Web UI: Gallery improvements by @psychedelicious in
+  https://github.com/invoke-ai/InvokeAI/pull/1198
+- Update CLI.md by @krummrey in https://github.com/invoke-ai/InvokeAI/pull/1211
+- outcropping improvements by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1207
+- add support for loading VAE autoencoders by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1216
+- remove duplicate fix_func for MPS by @wfng92 in
+  https://github.com/invoke-ai/InvokeAI/pull/1210
+- Metadata storage and retrieval fixes by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1204
+- nix: add shell.nix file by @Cloudef in
+  https://github.com/invoke-ai/InvokeAI/pull/1170
+- Web UI: Changes vite dist asset paths to relative by @psychedelicious in
+  https://github.com/invoke-ai/InvokeAI/pull/1185
+- Web UI: Removes isDisabled from PromptInput by @psychedelicious in
+  https://github.com/invoke-ai/InvokeAI/pull/1187
+- Allow user to generate images with initial noise as on M1 / mps system by
+  @ArDiouscuros in https://github.com/invoke-ai/InvokeAI/pull/981
+- feat: adding filename format template by @plucked in
+  https://github.com/invoke-ai/InvokeAI/pull/968
+- Web UI: Fixes broken bundle by @psychedelicious in
+  https://github.com/invoke-ai/InvokeAI/pull/1242
+- Support runwayML custom inpainting model by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1243
+- Update IMG2IMG.md by @talitore in
+  https://github.com/invoke-ai/InvokeAI/pull/1262
+- New dockerfile - including a build- and a run- script as well as a GH-Action
+  by @mauwii in https://github.com/invoke-ai/InvokeAI/pull/1233
+- cut over from karras to model noise schedule for higher steps by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1222
+- Prompt tweaks by @lstein in https://github.com/invoke-ai/InvokeAI/pull/1268
+- Outpainting implementation by @Kyle0654 in
+  https://github.com/invoke-ai/InvokeAI/pull/1251
+- fixing aspect ratio on hires by @tjennings in
+  https://github.com/invoke-ai/InvokeAI/pull/1249
+- Fix-build-container-action by @mauwii in
+  https://github.com/invoke-ai/InvokeAI/pull/1274
+- handle all unicode characters by @damian0815 in
+  https://github.com/invoke-ai/InvokeAI/pull/1276
+- adds models.user.yml to .gitignore by @JakeHL in
+  https://github.com/invoke-ai/InvokeAI/pull/1281
+- remove debug branch, set fail-fast to false by @mauwii in
+  https://github.com/invoke-ai/InvokeAI/pull/1284
+- Protect-secrets-on-pr by @mauwii in
+  https://github.com/invoke-ai/InvokeAI/pull/1285
+- Web UI: Adds initial inpainting implementation by @psychedelicious in
+  https://github.com/invoke-ai/InvokeAI/pull/1225
+- fix environment-mac.yml - tested on x64 and arm64 by @mauwii in
+  https://github.com/invoke-ai/InvokeAI/pull/1289
+- Use proper authentication to download model by @mauwii in
+  https://github.com/invoke-ai/InvokeAI/pull/1287
+- Prevent indexing error for mode RGB by @spezialspezial in
+  https://github.com/invoke-ai/InvokeAI/pull/1294
+- Integrate sd-v1-5 model into test matrix (easily expandable), remove
+  unecesarry caches by @mauwii in
+  https://github.com/invoke-ai/InvokeAI/pull/1293
+- add --no-interactive to preload_models step by @mauwii in
+  https://github.com/invoke-ai/InvokeAI/pull/1302
+- 1-click installer and updater. Uses micromamba to install git and conda into a
+  contained environment (if necessary) before running the normal installation
+  script by @cmdr2 in https://github.com/invoke-ai/InvokeAI/pull/1253
+- preload_models.py script downloads the weight files by @lstein in
+  https://github.com/invoke-ai/InvokeAI/pull/1290
+
+## v2.0.1 <small>(13 October 2022)</small>
+
+- fix noisy images at high step count when using k\* samplers
+- dream.py script now calls invoke.py module directly rather than via a new
+  python process (which could break the environment)
+
+## v2.0.0 <small>(9 October 2022)</small>
+
+- `dream.py` script renamed `invoke.py`. A `dream.py` script wrapper remains for
+  backward compatibility.
+- Completely new WebGUI - launch with `python3 scripts/invoke.py --web`
+- Support for [inpainting](features/INPAINTING.md) and
+  [outpainting](features/OUTPAINTING.md)
+- img2img runs on all k\* samplers
+- Support for
+  [negative prompts](features/PROMPTS.md#negative-and-unconditioned-prompts)
+- Support for CodeFormer face reconstruction
+- Support for Textual Inversion on Macintoshes
+- Support in both WebGUI and CLI for
+  [post-processing of previously-generated images](features/POSTPROCESS.md)
+  using facial reconstruction, ESRGAN upscaling, outcropping (similar to DALL-E
+  infinite canvas), and "embiggen" upscaling. See the `!fix` command.
+- New `--hires` option on `invoke>` line allows
+  [larger images to be created without duplicating elements](features/CLI.md#this-is-an-example-of-txt2img),
+  at the cost of some performance.
+- New `--perlin` and `--threshold` options allow you to add and control
+  variation during image generation (see
+  [Thresholding and Perlin Noise Initialization](features/OTHER.md#thresholding-and-perlin-noise-initialization-options))
+- Extensive metadata now written into PNG files, allowing reliable regeneration
+  of images and tweaking of previous settings.
+- Command-line completion in `invoke.py` now works on Windows, Linux and Mac
+  platforms.
+- Improved [command-line completion behavior](features/CLI.md) New commands
+  added:
+  - List command-line history with `!history`
+  - Search command-line history with `!search`
+  - Clear history with `!clear`
+- Deprecated `--full_precision` / `-F`. Simply omit it and `invoke.py` will auto
+  configure. To switch away from auto use the new flag like
+  `--precision=float32`.
+
+## v1.14 <small>(11 September 2022)</small>
+
+- Memory optimizations for small-RAM cards. 512x512 now possible on 4 GB GPUs.
+- Full support for Apple hardware with M1 or M2 chips.
+- Add "seamless mode" for circular tiling of image. Generates beautiful effects.
+  ([prixt](https://github.com/prixt)).
+- Inpainting support.
+- Improved web server GUI.
+- Lots of code and documentation cleanups.
+
+## v1.13 <small>(3 September 2022)</small>
+
+- Support image variations (see [VARIATIONS](features/VARIATIONS.md)
+  ([Kevin Gibbons](https://github.com/bakkot) and many contributors and
+  reviewers)
+- Supports a Google Colab notebook for a standalone server running on Google
+  hardware [Arturo Mendivil](https://github.com/artmen1516)
+- WebUI supports GFPGAN/ESRGAN facial reconstruction and upscaling
+  [Kevin Gibbons](https://github.com/bakkot)
+- WebUI supports incremental display of in-progress images during generation
+  [Kevin Gibbons](https://github.com/bakkot)
+- A new configuration file scheme that allows new models (including upcoming
+  stable-diffusion-v1.5) to be added without altering the code.
+  ([David Wager](https://github.com/maddavid12))
+- Can specify --grid on invoke.py command line as the default.
+- Miscellaneous internal bug and stability fixes.
+- Works on M1 Apple hardware.
+- Multiple bug fixes.
+
+---
+
+## v1.12 <small>(28 August 2022)</small>
+
+- Improved file handling, including ability to read prompts from standard input.
+  (kudos to [Yunsaki](https://github.com/yunsaki)
+- The web server is now integrated with the invoke.py script. Invoke by adding
+  --web to the invoke.py command arguments.
+- Face restoration and upscaling via GFPGAN and Real-ESGAN are now automatically
+  enabled if the GFPGAN directory is located as a sibling to Stable Diffusion.
+  VRAM requirements are modestly reduced. Thanks to both
+  [Blessedcoolant](https://github.com/blessedcoolant) and
+  [Oceanswave](https://github.com/oceanswave) for their work on this.
+- You can now swap samplers on the invoke> command line.
+  [Blessedcoolant](https://github.com/blessedcoolant)
+
+---
+
+## v1.11 <small>(26 August 2022)</small>
+
+- NEW FEATURE: Support upscaling and face enhancement using the GFPGAN module.
+  (kudos to [Oceanswave](https://github.com/Oceanswave)
+- You now can specify a seed of -1 to use the previous image's seed, -2 to use
+  the seed for the image generated before that, etc. Seed memory only extends
+  back to the previous command, but will work on all images generated with the
+  -n# switch.
+- Variant generation support temporarily disabled pending more general solution.
+- Created a feature branch named **yunsaki-morphing-invoke** which adds
+  experimental support for iteratively modifying the prompt and its parameters.
+  Please
+  see[Pull Request #86](https://github.com/lstein/stable-diffusion/pull/86) for
+  a synopsis of how this works. Note that when this feature is eventually added
+  to the main branch, it will may be modified significantly.
+
+---
+
+## v1.10 <small>(25 August 2022)</small>
+
+- A barebones but fully functional interactive web server for online generation
+  of txt2img and img2img.
+
+---
+
+## v1.09 <small>(24 August 2022)</small>
+
+- A new -v option allows you to generate multiple variants of an initial image
+  in img2img mode. (kudos to [Oceanswave](https://github.com/Oceanswave).
+  [ See this discussion in the PR for examples and details on use](https://github.com/lstein/stable-diffusion/pull/71#issuecomment-1226700810))
+- Added ability to personalize text to image generation (kudos to
+  [Oceanswave](https://github.com/Oceanswave) and
+  [nicolai256](https://github.com/nicolai256))
+- Enabled all of the samplers from k_diffusion
+
+---
+
+## v1.08 <small>(24 August 2022)</small>
+
+- Escape single quotes on the invoke> command before trying to parse. This
+  avoids parse errors.
+- Removed instruction to get Python3.8 as first step in Windows install.
+  Anaconda3 does it for you.
+- Added bounds checks for numeric arguments that could cause crashes.
+- Cleaned up the copyright and license agreement files.
+
+---
+
+## v1.07 <small>(23 August 2022)</small>
+
+- Image filenames will now never fill gaps in the sequence, but will be assigned
+  the next higher name in the chosen directory. This ensures that the alphabetic
+  and chronological sort orders are the same.
+
+---
+
+## v1.06 <small>(23 August 2022)</small>
+
+- Added weighted prompt support contributed by
+  [xraxra](https://github.com/xraxra)
+- Example of using weighted prompts to tweak a demonic figure contributed by
+  [bmaltais](https://github.com/bmaltais)
+
+---
+
+## v1.05 <small>(22 August 2022 - after the drop)</small>
+
+- Filenames now use the following formats: 000010.95183149.png -- Two files
+  produced by the same command (e.g. -n2), 000010.26742632.png -- distinguished
+  by a different seed.
+
+  000011.455191342.01.png -- Two files produced by the same command using
+  000011.455191342.02.png -- a batch size>1 (e.g. -b2). They have the same seed.
+
+  000011.4160627868.grid#1-4.png -- a grid of four images (-g); the whole grid
+  can be regenerated with the indicated key
+
+- It should no longer be possible for one image to overwrite another
+- You can use the "cd" and "pwd" commands at the invoke> prompt to set and
+  retrieve the path of the output directory.
+
+---
+
+## v1.04 <small>(22 August 2022 - after the drop)</small>
+
+- Updated README to reflect installation of the released weights.
+- Suppressed very noisy and inconsequential warning when loading the frozen CLIP
+  tokenizer.
+
+---
+
+## v1.03 <small>(22 August 2022)</small>
+
+- The original txt2img and img2img scripts from the CompViz repository have been
+  moved into a subfolder named "orig_scripts", to reduce confusion.
+
+---
+
+## v1.02 <small>(21 August 2022)</small>
+
+- A copy of the prompt and all of its switches and options is now stored in the
+  corresponding image in a tEXt metadata field named "Dream". You can read the
+  prompt using scripts/images2prompt.py, or an image editor that allows you to
+  explore the full metadata. **Please run "conda env update" to load the k_lms
+  dependencies!!**
+
+---
+
+## v1.01 <small>(21 August 2022)</small>
+
+- added k_lms sampling. **Please run "conda env update" to load the k_lms
+  dependencies!!**
+- use half precision arithmetic by default, resulting in faster execution and
+  lower memory requirements Pass argument --full_precision to invoke.py to get
+  slower but more accurate image generation
+
+---
+
+## Links
+
+- **[Read Me](index.md)**