* Only xfail windows models in CI
* downloader: make model updates more robust.
* Separate baseline and native benchmarks in pytest.
* Fix native benchmarks
* Fix torchvision model utils.
* Fix sharktank generation and add batch_size pytest option for torch.
* Disable torch dynamo until py3.11 supported
* Compile torchmodel without dynamo if torch.compile fails
* Use release versions of TF/Keras for importer.
* Pin torchvision and remove debug prints.
* Remove duplicates from torch model list.
* Update generate_sharktank.py
* xfail a few models that fail sharktank generation/ numerics
* Minor improvements to test-models workflow
- cleaned up pytest command line args in Validate Models job scripts.
- Removed -s flag to provide more readable logs
- Changed shark_cache location to within github workspace and removed --update_tank flag from Linux workflows.
* Use pytest-forked for managing pytest memory usage.
-Adds date variable back to nightly.yml so shark_tank uploads are dated again
-added specification for nightly pytests to not run tests on metal (vulkan is sufficient)
-added some paths/filetypes to be ignored when triggering workflow runs. (no test-models on changes to .md files or anything in the shark/examples/ directory or its subdirectories.
-pytest only picks up tank/test_models.py, so no need to specify which file to run when running pytest from SHARK base directory.
-Cleaned up xfails so that they can be added to models as csv entries. Columns 7-9 in all_models.csv trigger xfails with cpu, cuda, vulkan, respectively, and row 10 can be populated with a reason for the xfails.
-Fixed a few defaults for shark_args and pytest args (defined in conftest.py)
-Fixes --update_tank option in shark_downloader
removes some multiprocessing in pytest / TF+CUDA support because it breaks pytest and false passes, leaving regressions at large.
-Adds xfails for and removes albert torch from gen_sharktank list (tank/torch_model_list.csv).
-Cleans up xfails for cpu, cuda, vulkan (removing old ones)
* Move most xfails to entries in tank/all_models.csv
* enable usage of pytest without specifying tank/test_models.py
* add dict_configs.py to gitignore.
* Pin versions for runtimes and torch-mlir for setup.
* Use IREE tf tools to save .mlir modules when generating shark_tank.
* Add option to pytest for enabling auto-updates to local shark tank.
* xfail mobilenet torch on cpu, cuda and fix CI macos setup
* Update test-models.yml to disable macos vulkan CI.
* Add ONNX env var flags for venv setup.
* Setup arguments for ONNX benchmarking via pytest.
* Enable ONNX benchmarking on MiniLM via pytest (experimental)
* Fix sequence lengths to 128 for TF model creation and fix issue with benchmarks.
* Disable CI CPU benchmarks on A100, change some default args.
* add xfails for roberta TF model tests on GPU.
