Ean Garvey
f508c80c23
Add workflow for GH pages releases and release scraping script. ( #394 )
...
* Add workflow for GH pages releases and release scraping script.
* Update test_models.py and change tokens for gh pages.
2022-10-11 22:03:33 -05:00
Daniel Garvey
d38e37bd99
seperate importer and benchmark deps ( #393 )
2022-10-08 23:31:20 -05:00
Daniel Garvey
c54815de17
edit assets path ( #376 )
2022-09-28 16:42:36 -05:00
Ean Garvey
9035a2eed3
Add --local_tank_cache flag and update requirements. ( #368 )
...
* Add --local_tank_cache flag and update requirements.
* Update requirements-importer.txt
2022-09-28 03:02:59 -05:00
Ean Garvey
c0f3a09a40
Include SHA in path to failure reproducers. Add --save_fails option. ( #352 )
2022-09-21 17:55:06 -05:00
Ean Garvey
c061a8897d
Add pytest options to save reproducers. ( #350 )
...
* Add pytest options to save and/or upload reproducers.
* pass shark_module to benchmark method.
2022-09-20 20:29:46 -05:00
Daniel Garvey
5df582e7e8
creates abstract test case class ( #333 )
2022-09-20 07:06:38 -07:00
powderluv
44d8f08bfc
Fix Torch-MLIR release page
2022-09-17 00:50:39 -07:00
Ean Garvey
6cf5564c84
Remove "gpu" device alias and migrate to using "cuda" for NVIDIA GPU. ( #325 )
...
* Replace instances of "gpu" alias for devices with "cuda"
2022-09-13 01:16:56 -05:00
Ean Garvey
7143902a90
Update test-models.yml ( #323 )
2022-09-12 22:47:40 -05:00
Ean Garvey
6aeda3670f
Split nightly workflow by backend (IREE / SHARK) ( #313 )
...
* Fix validation for nightly builds.
* Add option to generate shark_tank inside SHARK project
Add shark_arg for updating tank on mismatched hash (downloader)
* Fixup CI tank dir option.
* Fixup work directory variable
2022-09-09 22:51:30 +05:30
Ean Garvey
d453f2e49d
Enable CPU benchmarks on test-models workflows. ( #299 )
...
* Update test-models.yml
* Update README.md
2022-09-07 01:22:58 -05:00
powderluv
4ee164c66f
remove a100 cpu
2022-08-31 12:59:47 -07:00
powderluv
bf84c033bb
add icelake
2022-08-31 12:58:40 -07:00
powderluv
57f73dfbc9
Update nightly.yml
2022-08-28 23:59:03 -07:00
powderluv
50b2b9638d
Update nightly.yml
2022-08-28 23:43:32 -07:00
Phaneesh Barwaria
1322ec5935
Simplified Testing Interface ( #289 )
2022-08-24 23:54:56 -05:00
Prashant Kumar
7dc27a7477
Don't remove the latest .whl package from CI. ( #290 )
...
Previously, the CI was removing the latest package and pointing to the
stale package.
2022-08-24 09:03:48 -07:00
Daniel Garvey
1a85550879
fix nightly upload check ( #277 )
2022-08-17 14:31:15 -05:00
Ean Garvey
3514822cac
Improvements to pytest benchmarks. ( #267 )
...
* Add ONNX env var flags for venv setup.
* Setup arguments for ONNX benchmarking via pytest.
* Enable ONNX benchmarking on MiniLM via pytest (experimental)
* Fix sequence lengths to 128 for TF model creation and fix issue with benchmarks.
* Disable CI CPU benchmarks on A100, change some default args.
* add xfails for roberta TF model tests on GPU.
2022-08-17 02:29:48 -05:00
Daniel Garvey
5e931debd5
Sharktank-ci ( #262 )
2022-08-15 13:32:24 -05:00
Ean Garvey
22ff92c48b
Add config.VmModule argument to from_flatbuffer call. ( #266 )
2022-08-14 15:11:19 -07:00
powderluv
7f5aaa3477
Update nightly.yml
2022-08-14 12:22:50 -07:00
powderluv
904e0e1444
Update nightly.yml
2022-08-14 09:57:10 -07:00
Daniel Garvey
e8482d47f5
split nightly pytest commands ( #259 )
...
prevents oom
2022-08-12 16:11:46 -07:00
powderluv
c2aa451767
Update test-models.yml
2022-08-09 10:12:59 -07:00
powderluv
274650fd43
Update nightly.yml
...
Add tests for USE_IREE=1
2022-08-07 00:06:11 -07:00
Ean Garvey
6f5ceb4e61
Update test-models.yml ( #244 )
2022-08-04 21:56:08 -07:00
Ean Garvey
6c22139ac9
Upload benchmark results for every test-models workflow (excl. Vulkan) ( #241 )
...
* Upload benchmark results for every test-models workflow (excl. Vulkan)
2022-08-04 14:43:07 -07:00
powderluv
c6b4ad1e26
Update test-models.yml
2022-08-02 13:19:10 -07:00
Phaneesh Barwaria
198755788c
Check for MacStudio Target Value ( #233 )
...
* Identify Apple M2 for Vulkan
* Check for MacOS Target
2022-08-02 13:18:36 -07:00
Ean Garvey
d556c0d6ef
Update nightly.yml to remove IMPORTER=1 in package validation. ( #226 )
...
* Update nightly.yml to fix wheel building issues.
* Update setup_venv.sh to only install torch+cu116 if IMPORTER=1 is set.
2022-07-31 20:39:59 -07:00
Ean Garvey
0ee515a7be
Fix GPU benchmarks on PyTorch model tests. ( #218 )
...
* Add CUDA_BENCHMARKS option to setup.venv to enable PyTorch benchmarks on CUDA.
* Fix PyTorch GPU benchmarks for tank models.
2022-07-29 15:17:55 -07:00
Daniel Garvey
4c0deb9899
remove gpu script ( #220 )
...
No longer needed after fixes in iree
2022-07-29 15:20:03 -05:00
Ean Garvey
921ccdc40b
Force TensorFlow to use CPU when generating model and golden values. ( #199 )
...
Split up gpu pytest runs temporarily
2022-07-22 20:21:16 -07:00
Ean Garvey
d047eff086
Update test-models.yml
2022-07-16 16:38:32 -05:00
monorimet
5da32601c0
Ensure CPU workflow job suite only collects CPU tests.
2022-07-16 13:50:36 -05:00
Prashant Kumar
0dcf387089
Add shark_importer for torch_models. ( #183 )
...
All the torch_models are imported to gs::shark_tank.
Scripts have been updated.
2022-07-12 20:38:19 -07:00
powderluv
c351bb50b6
Update test-models.yml
2022-07-03 13:56:45 -07:00
powderluv
ca36d40f0a
Update test-models.yml
2022-07-01 12:27:49 -07:00
powderluv
cc11a71ec8
Update test-models.yml
2022-06-30 18:55:38 -07:00
powderluv
1d1a1b131f
Update test-models.yml
2022-06-30 18:54:50 -07:00
powderluv
84a2969381
Update test-models.yml
2022-06-30 15:57:07 -07:00
powderluv
8199ea15e8
Update test-models.yml
2022-06-30 14:42:41 -07:00
powderluv
958e0100bf
Update test-models.yml
2022-06-30 14:41:26 -07:00
powderluv
6ca726615e
Update test-models.yml
2022-06-30 14:40:25 -07:00
powderluv
d1b398cb9b
Update test-models.yml
2022-06-30 14:28:37 -07:00
powderluv
6b0bd7181f
Update test-models.yml
2022-06-30 14:20:30 -07:00
powderluv
b24ce861d0
Update test-models.yml
2022-06-30 14:14:49 -07:00
powderluv
79e392d85f
enable pip cache
2022-06-30 14:11:00 -07:00
