Commit Graph

512 Commits

Author SHA1 Message Date
Ean Garvey
a3654f33da Fix sourcing for canonical MiniLM shark_tank model artifacts. (#278)
* Fix generation of MiniLM artifacts.

* Fix miniLM output for validation. Xfail numerics failure on mpnet.

* Update distilbert-base-uncased_tf_test.py

* try-except for transition of minilm model
2022-08-17 23:03:47 -05:00
Daniel Garvey
82c541dfb8 fix missing model download path (#281) 2022-08-17 23:02:50 -05:00
Stanley Winata
55bcb2eb3c Level Zero Backend (#280) 2022-08-17 19:19:27 -07:00
Daniel Garvey
1a85550879 fix nightly upload check (#277) 2022-08-17 14:31:15 -05:00
Ean Garvey
334f2f76c4 Update README.md (#273)
* Update README.md

* Update README.md

* Update README.md

* Update README.md

Co-authored-by: powderluv <powderluv@users.noreply.github.com>
2022-08-17 10:38:27 -07:00
Vivek Khandelwal
03601ccdd6 Add v_diffusion_pytorch model in shark/tank (#271) 2022-08-17 22:53:31 +05:30
Prashant Kumar
88b0dec0ee Update unet_model to run on shark.
-- Verified unet_model runs on the cpu/gpu/vulkan backend.
2022-08-17 13:16:02 +05:30
Ean Garvey
3514822cac Improvements to pytest benchmarks. (#267)
* Add ONNX env var flags for venv setup.

* Setup arguments for ONNX benchmarking via pytest.

* Enable ONNX benchmarking on MiniLM via pytest (experimental)

* Fix sequence lengths to 128 for TF model creation and fix issue with benchmarks.

* Disable CI CPU benchmarks on A100, change some default args.

* add xfails for roberta TF model tests on GPU.
2022-08-17 02:29:48 -05:00
Ean Garvey
a8b021dc8d Add benchmarks to MHLO miniLM and resnet50 and add dialect, num_iterations (#264) 2022-08-16 13:55:40 -05:00
Daniel Garvey
5e931debd5 Sharktank-ci (#262) 2022-08-15 13:32:24 -05:00
Ean Garvey
22ff92c48b Add config.VmModule argument to from_flatbuffer call. (#266) 2022-08-14 15:11:19 -07:00
powderluv
7f5aaa3477 Update nightly.yml 2022-08-14 12:22:50 -07:00
powderluv
904e0e1444 Update nightly.yml 2022-08-14 09:57:10 -07:00
powderluv
db6e2207ed Update _common.py 2022-08-13 13:49:01 -07:00
Daniel Garvey
7975087ee2 change backend name (#265) 2022-08-13 12:01:12 -07:00
Daniel Garvey
e8482d47f5 split nightly pytest commands (#259)
prevents oom
2022-08-12 16:11:46 -07:00
Ean Garvey
3e900d2b25 Change Resnet50 directory names. (#263) 2022-08-12 16:10:59 -07:00
Ean Garvey
4b5d09fc6c Add TF ResNet50 to tank tests. (#261)
* Add TensorFlow Resnet50 test to shark tank.
2022-08-12 09:20:43 -07:00
Prashant Kumar
02b1e7ac36 Update torch_mlir.compile API.
torch_mlir.compile API is updated and verified by compiling all the
torch models via generate_sharktank script.
2022-08-10 22:50:15 +05:30
Ean Garvey
23619068eb Disable passing of sm_arch to iree-compile CL args by default. (#253)
* Disable passing of sm_arch to iree-compile CL args by default.

* Fix formatting.
2022-08-10 01:19:24 -07:00
powderluv
f7f24dc4d9 Revert "Add Debug log of torch_model_blacklist.txt (#242)" (#249)
This reverts commit 7023d556b5.
2022-08-09 10:23:14 -07:00
powderluv
c2aa451767 Update test-models.yml 2022-08-09 10:12:59 -07:00
Chi_Liu
7023d556b5 Add Debug log of torch_model_blacklist.txt (#242)
* Add debug log of torch_model_blacklist.txt

* Add make_fx for torch model

* Update torch_model_blacklists.txt

* Add some Xfails
2022-08-09 17:54:02 +05:30
powderluv
274650fd43 Update nightly.yml
Add tests for USE_IREE=1
2022-08-07 00:06:11 -07:00
Prashant Kumar
d934765b1d Add mobilenet_v3_small torch model to the test_suite.
-- The model doesn't validate with the correct results on the GPU.
-- The model passes on CPU and levelzero.
-- The static version of the model gets stuck for vulkan.
2022-08-05 14:10:43 +05:30
Ean Garvey
6f5ceb4e61 Update test-models.yml (#244) 2022-08-04 21:56:08 -07:00
Ean Garvey
6c22139ac9 Upload benchmark results for every test-models workflow (excl. Vulkan) (#241)
* Upload benchmark results for every test-models workflow (excl. Vulkan)
2022-08-04 14:43:07 -07:00
powderluv
1c4f5e0c34 Add M1 Max and Pro variants 2022-08-04 13:45:34 -07:00
Daniel Garvey
7dc0a4f74d fine tune with shark (#211) 2022-08-04 13:14:57 -05:00
Chi_Liu
90fddc6cb0 Add more torch hg model tests (#238) 2022-08-03 18:00:04 -07:00
Quinn Dawkins
934f15ebb7 Fix IREE eager backend device string (#237) 2022-08-03 12:09:52 -07:00
Ean Garvey
38664a4c68 Update README.md (#239) 2022-08-03 11:39:00 -07:00
Chi_Liu
abce0b1c91 Move torch tests up to /tank (#234) 2022-08-03 10:50:53 -07:00
Phaneesh Barwaria
189466bbe4 Mark XFail for M1 Vulkan Failures (#235) 2022-08-02 19:56:02 -07:00
powderluv
c6b4ad1e26 Update test-models.yml 2022-08-02 13:19:10 -07:00
Phaneesh Barwaria
198755788c Check for MacStudio Target Value (#233)
* Identify Apple M2 for Vulkan

* Check for MacOS Target
2022-08-02 13:18:36 -07:00
Ean Garvey
c1a14a8db1 Update wide_resnet50_2_test.py (#229)
Removed old xfails.
2022-08-01 20:38:53 -05:00
Chi_Liu
0e42c19f33 Test tflite quantization_info fail on cloud (#228) 2022-08-01 18:21:04 -07:00
Chi_Liu
0c31bb82cd Add hash of mlir for tf/tflite (#225) 2022-08-01 10:40:49 -07:00
Prashant Kumar
315ec72984 Add check for hash.npy in the models folder. 2022-08-01 22:40:25 +05:30
Ean Garvey
d556c0d6ef Update nightly.yml to remove IMPORTER=1 in package validation. (#226)
* Update nightly.yml to fix wheel building issues.

* Update setup_venv.sh to only install torch+cu116 if IMPORTER=1 is set.
2022-07-31 20:39:59 -07:00
Ean Garvey
0ee515a7be Fix GPU benchmarks on PyTorch model tests. (#218)
* Add CUDA_BENCHMARKS option to setup.venv to enable PyTorch benchmarks on CUDA.

* Fix PyTorch GPU benchmarks for tank models.
2022-07-29 15:17:55 -07:00
Daniel Garvey
4c0deb9899 remove gpu script (#220)
No longer needed after fixes in iree
2022-07-29 15:20:03 -05:00
Chi_Liu
94d9275515 Move tf tests to tank/ (#223) 2022-07-29 09:34:50 -07:00
Chi_Liu
c1cb7bb3fd Rename tank tflite/torch model dir (#219) 2022-07-29 01:56:05 -05:00
Prashant Kumar
ff20ddeb97 Add hash of the mlir for checking upstream changes. (#217)
-- hash of the mlir files have been added in the shark_tank. (If there
is upstream changes, the model will be downloaded otherwise not).
-- The models have been placed in `~/.local/shark_tank/` folder as
opposed to the current folder not to redownload the files again.
2022-07-28 21:54:50 -05:00
Chi_Liu
2c7d879a4e Add tf image classification auto model tests (#216) 2022-07-27 18:38:13 -07:00
Prashant Kumar
4e5c592094 Enable --devices to pass through CLI. 2022-07-27 22:13:52 +05:30
Chi_Liu
af4257d05f Add tf image classification auto model (#213) 2022-07-26 23:18:42 -07:00
Chi_Liu
dc1a283ab7 Remove tf package dep for SharkDownloader tflite tests (#212) 2022-07-26 12:41:12 -07:00