This allows to pass more arguemnts to the IREE compiler
Example:
python my-app.py --additional_compile_args="--mlir-pretty-debuginfo --mlir-timing"
Co-authored-by: Boian Petkantchin <boian@nod-labs.com>
* Enable caching allocator by default
Going to toggle VMA off too and this is required for performance. Will have to monitor in the wild reports.
* Disable VMA
Disable VMA
* Only xfail windows models in CI
* downloader: make model updates more robust.
* Separate baseline and native benchmarks in pytest.
* Fix native benchmarks
* Fix torchvision model utils.
Example:
$ python my_app.py --device_allocator caching debug
This will wrap the device allocator with first caching allocator then
debug allocator.
$ python my_app.py --device_allocator caching
Only wrap with caching allocator.
Co-authored-by: Boian Petkantchin <boian@nod-labs.com>
-Adds date variable back to nightly.yml so shark_tank uploads are dated again
-added specification for nightly pytests to not run tests on metal (vulkan is sufficient)
-added some paths/filetypes to be ignored when triggering workflow runs. (no test-models on changes to .md files or anything in the shark/examples/ directory or its subdirectories.
-pytest only picks up tank/test_models.py, so no need to specify which file to run when running pytest from SHARK base directory.
-Cleaned up xfails so that they can be added to models as csv entries. Columns 7-9 in all_models.csv trigger xfails with cpu, cuda, vulkan, respectively, and row 10 can be populated with a reason for the xfails.
-Fixed a few defaults for shark_args and pytest args (defined in conftest.py)
-Fixes --update_tank option in shark_downloader
removes some multiprocessing in pytest / TF+CUDA support because it breaks pytest and false passes, leaving regressions at large.
-Adds xfails for and removes albert torch from gen_sharktank list (tank/torch_model_list.csv).
-Cleans up xfails for cpu, cuda, vulkan (removing old ones)
* Move most xfails to entries in tank/all_models.csv
* enable usage of pytest without specifying tank/test_models.py
* add dict_configs.py to gitignore.
* Pin versions for runtimes and torch-mlir for setup.
To produce benchmarks of individual dispatches, you can add --dispatch_benchmarks=All --dispatch_benchmarks_dir=<output_dir> to your command line argument.
Co-authored-by: Elias Joseph <elias@nod-labs.com>
* Fix validation for nightly builds.
* Add option to generate shark_tank inside SHARK project
Add shark_arg for updating tank on mismatched hash (downloader)
* Fixup CI tank dir option.
* Fixup work directory variable
individual tests will require implementation of the flag
alternatively, simply passing shark_default_sha in your
individual app's download function will allow for this behavior
* Add ONNX env var flags for venv setup.
* Setup arguments for ONNX benchmarking via pytest.
* Enable ONNX benchmarking on MiniLM via pytest (experimental)
* Fix sequence lengths to 128 for TF model creation and fix issue with benchmarks.
* Disable CI CPU benchmarks on A100, change some default args.
* add xfails for roberta TF model tests on GPU.
