AMD-SHARK-Studio/tank/all_models.csv at d6f740b9981612640071a413dce1b140459a3e9f

mirror of https://github.com/nod-ai/AMD-SHARK-Studio.git synced 2026-04-25 03:00:12 -04:00

Files

Ean Garvey d6f740b998 allow pytest to retry getting model artifacts + disable autotuning for pytorch benchmarks (#1257 )

* Adds a few xfails to enable macOS builder

* Convert string batch sizes to ints where needed.

* allow pytest to retry getting model artifacts

* Reduce attempts and add assert msg.

2023-03-28 23:38:45 -05:00

4.3 KiB

Raw Blame History

1	resnet50	mhlo	tf	1e-2	1e-3	default	nhcw-nhwc	False	False	False		macos
2	albert-base-v2	mhlo	tf	1e-2	1e-2	default	None	False	False	False
3	roberta-base	mhlo	tf	1e-02	1e-3	default	nhcw-nhwc	True	True	True		macos
4	bert-base-uncased	mhlo	tf	1e-2	1e-3	default	None	False	False	False		enabled_windows
5	camembert-base	mhlo	tf	1e-2	1e-3	default	None	True	True	True
6	dbmdz/convbert-base-turkish-cased	mhlo	tf	1e-2	1e-3	default	nhcw-nhwc	True	True	False	https://github.com/iree-org/iree/issues/9971
7	distilbert-base-uncased	mhlo	tf	1e-2	1e-3	default	None	False	False	False
8	facebook/convnext-tiny-224	mhlo	tf	1e-2	1e-3	tf_vit	nhcw-nhwc	True	True	False	https://github.com/nod-ai/SHARK/issues/311 & https://github.com/nod-ai/SHARK/issues/342	macos
9	funnel-transformer/small	mhlo	tf	1e-2	1e-3	default	None	True	True	False	https://github.com/nod-ai/SHARK/issues/201
10	google/electra-small-discriminator	mhlo	tf	1e-2	1e-3	default	None	False	False	False
11	google/mobilebert-uncased	mhlo	tf	1e-2	1e-3	default	None	True	False	False	Fails during iree-compile
12	google/vit-base-patch16-224	mhlo	tf	1e-2	1e-3	tf_vit	nhcw-nhwc	False	False	False
13	microsoft/MiniLM-L12-H384-uncased	mhlo	tf	1e-2	1e-3	tf_hf	None	True	False	False	Fails during iree-compile.
14	microsoft/layoutlm-base-uncased	mhlo	tf	1e-2	1e-3	default	None	False	False	False
15	microsoft/mpnet-base	mhlo	tf	1e-2	1e-2	default	None	True	True	True
16	albert-base-v2	linalg	torch	1e-2	1e-3	default	None	True	True	True	issue with aten.tanh in torch-mlir
17	alexnet	linalg	torch	1e-2	1e-3	default	None	True	True	False	https://github.com/nod-ai/SHARK/issues/879
18	bert-base-cased	linalg	torch	1e-2	1e-3	default	None	False	False	False
19	bert-base-uncased	linalg	torch	1e-2	1e-3	default	None	False	False	False
20	bert-base-uncased_fp16	linalg	torch	1e-1	1e-1	default	None	True	False	True
21	bert-large-uncased	linalg	torch	1e-2	1e-3	default	None	False	False	False
22	bert-large-uncased	mhlo	tf	1e-2	1e-3	default	None	False	False	False
23	facebook/deit-small-distilled-patch16-224	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	False	True	False	Fails during iree-compile.
24	google/vit-base-patch16-224	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	False	True	False	https://github.com/nod-ai/SHARK/issues/311
25	microsoft/beit-base-patch16-224-pt22k-ft22k	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	False	True	False	https://github.com/nod-ai/SHARK/issues/390	macos
26	microsoft/MiniLM-L12-H384-uncased	linalg	torch	1e-2	1e-3	default	None	False	False	False
27	google/mobilebert-uncased	linalg	torch	1e-2	1e-3	default	None	False	False	False	https://github.com/nod-ai/SHARK/issues/344
28	mobilenet_v3_small	linalg	torch	1e-1	1e-2	default	nhcw-nhwc	False	True	False	https://github.com/nod-ai/SHARK/issues/388	macos
29	nvidia/mit-b0	linalg	torch	1e-2	1e-3	default	None	True	True	False	https://github.com/nod-ai/SHARK/issues/343	macos
30	resnet101	linalg	torch	1e-2	1e-3	default	nhcw-nhwc/img2col	False	False	False		macos
31	resnet18	linalg	torch	1e-2	1e-3	default	None	True	True	False		macos
32	resnet50	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	False	False	False		macos
33	resnet50_fp16	linalg	torch	1e-2	1e-2	default	nhcw-nhwc/img2col	True	False	True
34	squeezenet1_0	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	False	False	False		macos
35	wide_resnet50_2	linalg	torch	1e-2	1e-3	default	nhcw-nhwc/img2col	False	False	False		macos
36	efficientnet-v2-s	mhlo	tf	1e-02	1e-3	default	nhcw-nhwc	False	False	False		macos
37	mnasnet1_0	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	True	True	True		macos
38	efficientnet_b0	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	True	True	False	https://github.com/nod-ai/SHARK/issues/1243
39	efficientnet_b7	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	True	False	False	Torchvision imports issue
40	efficientnet_b0	mhlo	tf	1e-2	1e-3	default	None	nhcw-nhwc	False	False	False
41	efficientnet_b7	mhlo	tf	1e-2	1e-3	default	None	nhcw-nhwc	False	False	False
42	gpt2	mhlo	tf	1e-2	1e-3	default	None	True	False	False
43	t5-base	linalg	torch	1e-2	1e-3	default	None	True	True	True	Inputs for seq2seq models in torch currently unsupported.
44	t5-base	mhlo	tf	1e-2	1e-3	default	None	False	False	False
45	t5-large	linalg	torch	1e-2	1e-3	default	None	True	True	True	Inputs for seq2seq models in torch currently unsupported
46	t5-large	mhlo	tf	1e-2	1e-3	default	None	False	False	False

4.3 KiB Raw Blame History

4.3 KiB

Raw Blame History