24 Commits

Author SHA1 Message Date
Ean Garvey
2191fc8952 Separate pytest benchmark modes and fix model updates for SHARK downloader / pytest. (#1264)
* Only xfail windows models in CI

* downloader: make model updates more robust.

* Separate baseline and native benchmarks in pytest.

* Fix native benchmarks

* Fix torchvision model utils.
2023-04-03 08:24:21 -07:00
Ean Garvey
96b1560da5 Make batch size configurable via pytest and fix sharktank generation. (#1227)
* Fix sharktank generation and add batch_size pytest option for torch.

* Disable torch dynamo until py3.11 supported

* Compile torchmodel without dynamo if torch.compile fails

* Use release versions of TF/Keras for importer.

* Pin torchvision and remove debug prints.

* Remove duplicates from torch model list.

* Update generate_sharktank.py

* xfail a few models that fail sharktank generation/ numerics
2023-03-28 14:33:39 -05:00
mariecwhite
90c958bca2 Add T5-base and T5-large Torch and TF Models (#1116) 2023-03-20 17:32:50 -05:00
mariecwhite
f99903e023 Add EfficientNet B0 and B7 Torch and TF models 2023-03-21 09:22:05 +11:00
mariecwhite
c6f44ef1b3 Add EfficientNet B0 and B7 Torch and TF models 2023-03-21 09:14:45 +11:00
mariecwhite
8dcd4d5aeb Make batch size configurable 2023-03-20 18:03:17 -04:00
mariecwhite
c3a4fdcbfc Add bert-large-uncased TF model 2023-02-15 21:42:44 -08:00
mariecwhite
eb8114ece8 Initialize TF models locally (#610) 2022-12-12 11:35:34 +11:00
mariecwhite
ff649b52ef Add TF EfficientNet Model (#502) 2022-12-06 13:51:59 -06:00
mariecwhite
6393dc5d14 Use correct TF device depending on configuration (#492) 2022-12-02 11:33:56 -06:00
Ean Garvey
45677c1e23 Install torch version required by torch-mlir when setting up importer venv. (#486) 2022-11-14 14:01:01 +05:30
mariecwhite
ec461a4456 Enable XLA compiler for TF models (#484) 2022-11-13 20:10:47 -08:00
Daniel Garvey
5df582e7e8 creates abstract test case class (#333) 2022-09-20 07:06:38 -07:00
Ean Garvey
3514822cac Improvements to pytest benchmarks. (#267)
* Add ONNX env var flags for venv setup.

* Setup arguments for ONNX benchmarking via pytest.

* Enable ONNX benchmarking on MiniLM via pytest (experimental)

* Fix sequence lengths to 128 for TF model creation and fix issue with benchmarks.

* Disable CI CPU benchmarks on A100, change some default args.

* add xfails for roberta TF model tests on GPU.
2022-08-17 02:29:48 -05:00
Ean Garvey
a8b021dc8d Add benchmarks to MHLO miniLM and resnet50 and add dialect, num_iterations (#264) 2022-08-16 13:55:40 -05:00
Ean Garvey
4b5d09fc6c Add TF ResNet50 to tank tests. (#261)
* Add TensorFlow Resnet50 test to shark tank.
2022-08-12 09:20:43 -07:00
Chi_Liu
2c7d879a4e Add tf image classification auto model tests (#216) 2022-07-27 18:38:13 -07:00
Ean Garvey
921ccdc40b Force TensorFlow to use CPU when generating model and golden values. (#199)
Split up gpu pytest runs temporarily
2022-07-22 20:21:16 -07:00
monorimet
08448a013f Refactor TF tests for importer split, update pytorch tests. 2022-07-14 12:59:21 -05:00
Prashant Kumar
0dcf387089 Add shark_importer for torch_models. (#183)
All the torch_models are imported to gs::shark_tank.
Scripts have been updated.
2022-07-12 20:38:19 -07:00
Anush Elangovan
a7435973d9 Fix black formatting 2022-06-30 20:42:02 +00:00
Prashant Kumar
e8aa105b2a Divide iree_utils and do module imports on function calls. 2022-06-22 14:17:33 +05:30
Prashant Kumar
4d3acff484 Add tf masked lm models (17 models - 12 pass, 5 fail). (#135) 2022-06-16 07:28:00 -07:00
Ean Garvey
acdd1d1785 Update model testing utilities. (#119) 2022-06-13 02:16:30 -07:00