* Only xfail windows models in CI
* downloader: make model updates more robust.
* Separate baseline and native benchmarks in pytest.
* Fix native benchmarks
* Fix torchvision model utils.
* Fix sharktank generation and add batch_size pytest option for torch.
* Disable torch dynamo until py3.11 supported
* Compile torchmodel without dynamo if torch.compile fails
* Use release versions of TF/Keras for importer.
* Pin torchvision and remove debug prints.
* Remove duplicates from torch model list.
* Update generate_sharktank.py
* xfail a few models that fail sharktank generation/ numerics
* Minor improvements to test-models workflow
- cleaned up pytest command line args in Validate Models job scripts.
- Removed -s flag to provide more readable logs
- Changed shark_cache location to within github workspace and removed --update_tank flag from Linux workflows.
* Use pytest-forked for managing pytest memory usage.
-Adds date variable back to nightly.yml so shark_tank uploads are dated again
-added specification for nightly pytests to not run tests on metal (vulkan is sufficient)
-added some paths/filetypes to be ignored when triggering workflow runs. (no test-models on changes to .md files or anything in the shark/examples/ directory or its subdirectories.
-pytest only picks up tank/test_models.py, so no need to specify which file to run when running pytest from SHARK base directory.
-Cleaned up xfails so that they can be added to models as csv entries. Columns 7-9 in all_models.csv trigger xfails with cpu, cuda, vulkan, respectively, and row 10 can be populated with a reason for the xfails.
-Fixed a few defaults for shark_args and pytest args (defined in conftest.py)
-Fixes --update_tank option in shark_downloader
removes some multiprocessing in pytest / TF+CUDA support because it breaks pytest and false passes, leaving regressions at large.
-Adds xfails for and removes albert torch from gen_sharktank list (tank/torch_model_list.csv).
-Cleans up xfails for cpu, cuda, vulkan (removing old ones)
