Commit Graph

6 Commits

Author SHA1 Message Date
Ryan Dick
0e632dbc5c (minor) typo 2025-02-28 21:39:09 +00:00
Ryan Dick
a36a627f83 Switch from use_cuda_malloc flag to a general pytorch_cuda_alloc_conf config field that allows full customization of the CUDA allocator. 2025-02-28 21:39:09 +00:00
Ryan Dick
5302d4890f Add use_cuda_malloc config option. 2025-02-28 21:39:09 +00:00
Ryan Dick
1e2c7c51b5 Move load_custom_nodes() to run_app() entrypoint. 2025-02-28 20:54:26 +00:00
Ryan Dick
68d14de3ee Split run_app.py and api_app.py so that api_app.py is more narrowly responsible for just initializing the FastAPI app. This also gives clearer control over the order of the initialization steps, which will be important as we add planned torch configurations that must be applied before torch is imported. 2025-02-28 20:10:24 +00:00
psychedelicious
ce9aeeece3 feat: single app entrypoint with CLI arg parsing
We have two problems with how argparse is being utilized:
- We parse CLI args as the `api_app.py` file is read. This causes a problem pytest, which has an incompatible set of CLI args. Some tests import the FastAPI app, which triggers the config to parse CLI args, which receives the pytest args and fails.
- We've repeatedly had problems when something that uses the config is imported before the CLI args are parsed. When this happens, the root dir may not be set correctly, so we attempt to operate on incorrect paths.

To resolve these issues, we need to lift CLI arg parsing outside of the application code, but still let the application access the CLI args. We can create a external app entrypoint to do this.

- `InvokeAIArgs` is a simple helper class that parses CLI args and stores the result.
- `run_app()` is the new entrypoint. It first parses CLI args, then runs `invoke_api` to start the app.

The `invokeai-web` project script and `invokeai-web.py` dev script now call `run_app()` instead of `invoke_api()`.

The first time `get_config()` is called to get the singleton config object, it retrieves the args from `InvokeAIArgs`, sets the root dir if provided, then merges settings in from `invokeai.yaml`.

CLI arg parsing is now safely insulated from application code, but still accessible. And we don't need to worry about import order having an impact on anything, because by the time the app is running, we have already parsed CLI args. Whew!
2024-03-19 09:24:28 +11:00