Ean Garvey
a3654f33da
Fix sourcing for canonical MiniLM shark_tank model artifacts. ( #278 )
...
* Fix generation of MiniLM artifacts.
* Fix miniLM output for validation. Xfail numerics failure on mpnet.
* Update distilbert-base-uncased_tf_test.py
* try-except for transition of minilm model
2022-08-17 23:03:47 -05:00
Daniel Garvey
82c541dfb8
fix missing model download path ( #281 )
2022-08-17 23:02:50 -05:00
Stanley Winata
55bcb2eb3c
Level Zero Backend ( #280 )
2022-08-17 19:19:27 -07:00
Daniel Garvey
1a85550879
fix nightly upload check ( #277 )
2022-08-17 14:31:15 -05:00
Ean Garvey
334f2f76c4
Update README.md ( #273 )
...
* Update README.md
* Update README.md
* Update README.md
* Update README.md
Co-authored-by: powderluv <powderluv@users.noreply.github.com >
2022-08-17 10:38:27 -07:00
Vivek Khandelwal
03601ccdd6
Add v_diffusion_pytorch model in shark/tank ( #271 )
2022-08-17 22:53:31 +05:30
Prashant Kumar
88b0dec0ee
Update unet_model to run on shark.
...
-- Verified unet_model runs on the cpu/gpu/vulkan backend.
2022-08-17 13:16:02 +05:30
Ean Garvey
3514822cac
Improvements to pytest benchmarks. ( #267 )
...
* Add ONNX env var flags for venv setup.
* Setup arguments for ONNX benchmarking via pytest.
* Enable ONNX benchmarking on MiniLM via pytest (experimental)
* Fix sequence lengths to 128 for TF model creation and fix issue with benchmarks.
* Disable CI CPU benchmarks on A100, change some default args.
* add xfails for roberta TF model tests on GPU.
2022-08-17 02:29:48 -05:00
Ean Garvey
a8b021dc8d
Add benchmarks to MHLO miniLM and resnet50 and add dialect, num_iterations ( #264 )
2022-08-16 13:55:40 -05:00
Daniel Garvey
5e931debd5
Sharktank-ci ( #262 )
2022-08-15 13:32:24 -05:00
Ean Garvey
22ff92c48b
Add config.VmModule argument to from_flatbuffer call. ( #266 )
2022-08-14 15:11:19 -07:00
powderluv
7f5aaa3477
Update nightly.yml
2022-08-14 12:22:50 -07:00
powderluv
904e0e1444
Update nightly.yml
2022-08-14 09:57:10 -07:00
powderluv
db6e2207ed
Update _common.py
2022-08-13 13:49:01 -07:00
Daniel Garvey
7975087ee2
change backend name ( #265 )
2022-08-13 12:01:12 -07:00
Daniel Garvey
e8482d47f5
split nightly pytest commands ( #259 )
...
prevents oom
2022-08-12 16:11:46 -07:00
Ean Garvey
3e900d2b25
Change Resnet50 directory names. ( #263 )
2022-08-12 16:10:59 -07:00
Ean Garvey
4b5d09fc6c
Add TF ResNet50 to tank tests. ( #261 )
...
* Add TensorFlow Resnet50 test to shark tank.
2022-08-12 09:20:43 -07:00
Prashant Kumar
02b1e7ac36
Update torch_mlir.compile API.
...
torch_mlir.compile API is updated and verified by compiling all the
torch models via generate_sharktank script.
2022-08-10 22:50:15 +05:30
Ean Garvey
23619068eb
Disable passing of sm_arch to iree-compile CL args by default. ( #253 )
...
* Disable passing of sm_arch to iree-compile CL args by default.
* Fix formatting.
2022-08-10 01:19:24 -07:00
powderluv
f7f24dc4d9
Revert "Add Debug log of torch_model_blacklist.txt ( #242 )" ( #249 )
...
This reverts commit 7023d556b5 .
2022-08-09 10:23:14 -07:00
powderluv
c2aa451767
Update test-models.yml
2022-08-09 10:12:59 -07:00
Chi_Liu
7023d556b5
Add Debug log of torch_model_blacklist.txt ( #242 )
...
* Add debug log of torch_model_blacklist.txt
* Add make_fx for torch model
* Update torch_model_blacklists.txt
* Add some Xfails
2022-08-09 17:54:02 +05:30
powderluv
274650fd43
Update nightly.yml
...
Add tests for USE_IREE=1
2022-08-07 00:06:11 -07:00
Prashant Kumar
d934765b1d
Add mobilenet_v3_small torch model to the test_suite.
...
-- The model doesn't validate with the correct results on the GPU.
-- The model passes on CPU and levelzero.
-- The static version of the model gets stuck for vulkan.
2022-08-05 14:10:43 +05:30
Ean Garvey
6f5ceb4e61
Update test-models.yml ( #244 )
2022-08-04 21:56:08 -07:00
Ean Garvey
6c22139ac9
Upload benchmark results for every test-models workflow (excl. Vulkan) ( #241 )
...
* Upload benchmark results for every test-models workflow (excl. Vulkan)
2022-08-04 14:43:07 -07:00
powderluv
1c4f5e0c34
Add M1 Max and Pro variants
2022-08-04 13:45:34 -07:00
Daniel Garvey
7dc0a4f74d
fine tune with shark ( #211 )
2022-08-04 13:14:57 -05:00
Chi_Liu
90fddc6cb0
Add more torch hg model tests ( #238 )
2022-08-03 18:00:04 -07:00
Quinn Dawkins
934f15ebb7
Fix IREE eager backend device string ( #237 )
2022-08-03 12:09:52 -07:00
Ean Garvey
38664a4c68
Update README.md ( #239 )
2022-08-03 11:39:00 -07:00
Chi_Liu
abce0b1c91
Move torch tests up to /tank ( #234 )
2022-08-03 10:50:53 -07:00
Phaneesh Barwaria
189466bbe4
Mark XFail for M1 Vulkan Failures ( #235 )
2022-08-02 19:56:02 -07:00
powderluv
c6b4ad1e26
Update test-models.yml
2022-08-02 13:19:10 -07:00
Phaneesh Barwaria
198755788c
Check for MacStudio Target Value ( #233 )
...
* Identify Apple M2 for Vulkan
* Check for MacOS Target
2022-08-02 13:18:36 -07:00
Ean Garvey
c1a14a8db1
Update wide_resnet50_2_test.py ( #229 )
...
Removed old xfails.
2022-08-01 20:38:53 -05:00
Chi_Liu
0e42c19f33
Test tflite quantization_info fail on cloud ( #228 )
2022-08-01 18:21:04 -07:00
Chi_Liu
0c31bb82cd
Add hash of mlir for tf/tflite ( #225 )
2022-08-01 10:40:49 -07:00
Prashant Kumar
315ec72984
Add check for hash.npy in the models folder.
2022-08-01 22:40:25 +05:30
Ean Garvey
d556c0d6ef
Update nightly.yml to remove IMPORTER=1 in package validation. ( #226 )
...
* Update nightly.yml to fix wheel building issues.
* Update setup_venv.sh to only install torch+cu116 if IMPORTER=1 is set.
2022-07-31 20:39:59 -07:00
Ean Garvey
0ee515a7be
Fix GPU benchmarks on PyTorch model tests. ( #218 )
...
* Add CUDA_BENCHMARKS option to setup.venv to enable PyTorch benchmarks on CUDA.
* Fix PyTorch GPU benchmarks for tank models.
2022-07-29 15:17:55 -07:00
Daniel Garvey
4c0deb9899
remove gpu script ( #220 )
...
No longer needed after fixes in iree
2022-07-29 15:20:03 -05:00
Chi_Liu
94d9275515
Move tf tests to tank/ ( #223 )
2022-07-29 09:34:50 -07:00
Chi_Liu
c1cb7bb3fd
Rename tank tflite/torch model dir ( #219 )
2022-07-29 01:56:05 -05:00
Prashant Kumar
ff20ddeb97
Add hash of the mlir for checking upstream changes. ( #217 )
...
-- hash of the mlir files have been added in the shark_tank. (If there
is upstream changes, the model will be downloaded otherwise not).
-- The models have been placed in `~/.local/shark_tank/` folder as
opposed to the current folder not to redownload the files again.
2022-07-28 21:54:50 -05:00
Chi_Liu
2c7d879a4e
Add tf image classification auto model tests ( #216 )
2022-07-27 18:38:13 -07:00
Prashant Kumar
4e5c592094
Enable --devices to pass through CLI.
2022-07-27 22:13:52 +05:30
Chi_Liu
af4257d05f
Add tf image classification auto model ( #213 )
2022-07-26 23:18:42 -07:00
Chi_Liu
dc1a283ab7
Remove tf package dep for SharkDownloader tflite tests ( #212 )
2022-07-26 12:41:12 -07:00