* Enable caching allocator by default
Going to toggle VMA off too and this is required for performance. Will have to monitor in the wild reports.
* Disable VMA
Disable VMA
* Optimize device enumeration overhead and log details on long operations.
* Various fixes to add `@functools.cache` to what should be one time, expensive, device enumeration and setup activities. Cuts several seconds off of initialization on my machine.
* Add detailed tracing to actual invocations if they exceed a certain timeout or have an exception.
* Add detailed tracing to loading status.
* By default detail logging is only printed if an operation takes an excessive amount of time. All logging/timing can be printed by setting the variable `$env:SHARK_DETAIL_TRACE = "1"`
* Remove cache from unhashable functions
* [MiniGPT4] Add MiniGPT4 to SHARK
-- This is the first installment of MiniGPT4 in SHARK.
Signed-off-by: Abhishek Varma <abhishek@nod-labs.com>
* Add int8 support for MiniGPT4
-- This commit adds int8 support for MiniGPT4.
Signed-off-by: Abhishek Varma <abhishek@nod-lab.com>
* Update .spec for MiniGPT4's config files
* black format MiniGPT4
---------
Signed-off-by: Abhishek Varma <abhishek@nod-labs.com>
Signed-off-by: Abhishek Varma <abhishek@nod-lab.com>
* Adding metal_utils for iree_utils
* Add patch for making compile API work for both MEGABYTE and MiniGPT4 (#1559)
-- It also modifies the mega_test.py script
Signed-off-by: Abhishek Varma <abhishek@nod-labs.com>
* [SD] Update unet in_channels API and add PIL metadata to spec. (#1560)
* Fix deprecation warning for unet config.
* Include PIL metadata instead of hidden imports in SD spec.
* Fixing iree-metal-target-platform
* adding metal to txt2img pipeline
* Fixing Copyright date
* removing debug prints
* black lint formating
* fixing device dump
---------
Signed-off-by: Abhishek Varma <abhishek@nod-labs.com>
Co-authored-by: Abhishek Varma <avarma094@gmail.com>
Co-authored-by: Ean Garvey <87458719+monorimet@users.noreply.github.com>
Co-authored-by: powderluv <powderluv@users.noreply.github.com>
single endpoint in apps/language/models/scripts/vicuna.py
removed main functions from pipelines
replaced divergent utils compile with shark_importer
adds support for different precisions
* The two with blocks in add_upcast appear to be underindented making
SD 1.4 break on rdna3, I've pushed them out one more tab, and then
everything appears to work again.
Model config generator takes a PyTorch model as input and generates a JSON file with model layers and other propperties that define sharding on a particular hardware.
