AMD-SHARK-Studio/tank/all_models.csv at 20230829.921

mirror of https://github.com/nod-ai/AMD-SHARK-Studio.git synced 2026-04-03 03:00:17 -04:00

Files

Ean Garvey cb509343d9 Fix pytest benchmarks and shark_tank generation. (#1632 )

- fix setup_venv.sh for benchmarks/imports etc.
- fix torch benchmarks in SharkBenchmarkRunner
- generate SD artifacts using build_tools/stable_diffusion_testing.py and --import_mlir
- decouple SD gen from tank/generate_sharktank for now

2023-08-16 17:48:47 -05:00

4.5 KiB

Raw Permalink Blame History

1	resnet50	stablehlo	tf	1e-2	1e-3	default	nhcw-nhwc	False	False	False		macos
2	albert-base-v2	stablehlo	tf	1e-2	1e-2	default	None	False	False	False
3	roberta-base	stablehlo	tf	1e-02	1e-3	default	nhcw-nhwc	True	True	True		macos
4	bert-base-uncased	stablehlo	tf	1e-2	1e-3	default	None	False	False	False		enabled_windows
5	camembert-base	stablehlo	tf	1e-2	1e-3	default	None	True	True	True
6	dbmdz/convbert-base-turkish-cased	stablehlo	tf	1e-2	1e-3	default	nhcw-nhwc	True	True	False	https://github.com/iree-org/iree/issues/9971
7	distilbert-base-uncased	stablehlo	tf	1e-2	1e-3	default	None	False	False	False
8	facebook/convnext-tiny-224	stablehlo	tf	1e-2	1e-3	tf_vit	nhcw-nhwc	True	True	False	https://github.com/nod-ai/SHARK/issues/311 & https://github.com/nod-ai/SHARK/issues/342	macos
9	funnel-transformer/small	stablehlo	tf	1e-2	1e-3	default	None	True	True	False	https://github.com/nod-ai/SHARK/issues/201
10	google/electra-small-discriminator	stablehlo	tf	1e-2	1e-3	default	None	False	False	False
11	google/mobilebert-uncased	stablehlo	tf	1e-2	1e-3	default	None	True	False	False	Fails during iree-compile	macos
12	google/vit-base-patch16-224	stablehlo	tf	1e-2	1e-3	tf_vit	nhcw-nhwc	False	False	False
13	microsoft/MiniLM-L12-H384-uncased	stablehlo	tf	1e-2	1e-3	tf_hf	None	True	False	False	Fails during iree-compile.
14	microsoft/layoutlm-base-uncased	stablehlo	tf	1e-2	1e-3	default	None	False	False	False
15	microsoft/mpnet-base	stablehlo	tf	1e-2	1e-2	default	None	True	True	True
16	alexnet	linalg	torch	1e-2	1e-3	default	None	True	True	False	https://github.com/nod-ai/SHARK/issues/879
17	bert-base-cased	linalg	torch	1e-2	1e-3	default	None	False	True	False
18	bert-base-uncased	linalg	torch	1e-2	1e-3	default	None	False	True	False
19	bert-base-uncased_fp16	linalg	torch	1e-1	1e-1	default	None	True	True	True
20	bert-large-uncased	linalg	torch	1e-2	1e-3	default	None	False	True	False
21	bert-large-uncased	stablehlo	tf	1e-2	1e-3	default	None	False	False	False
22	facebook/deit-small-distilled-patch16-224	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	False	True	False	Fails during iree-compile.
23	google/vit-base-patch16-224	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	False	True	False	https://github.com/nod-ai/SHARK/issues/311
24	microsoft/beit-base-patch16-224-pt22k-ft22k	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	False	True	False	https://github.com/nod-ai/SHARK/issues/390	macos
25	microsoft/MiniLM-L12-H384-uncased	linalg	torch	1e-2	1e-3	default	None	False	True	False
26	google/mobilebert-uncased	linalg	torch	1e-2	1e-3	default	None	False	True	False	https://github.com/nod-ai/SHARK/issues/344	macos
27	mobilenet_v3_small	linalg	torch	1e-1	1e-2	default	nhcw-nhwc	True	True	True	https://github.com/nod-ai/SHARK/issues/388, https://github.com/nod-ai/SHARK/issues/1487	macos
28	nvidia/mit-b0	linalg	torch	1e-2	1e-3	default	None	True	True	True	https://github.com/nod-ai/SHARK/issues/343,https://github.com/nod-ai/SHARK/issues/1487	macos
29	resnet101	linalg	torch	1e-2	1e-3	default	nhcw-nhwc/img2col	True	False	False		macos
30	resnet18	linalg	torch	1e-2	1e-3	default	None	True	True	False		macos
31	resnet50	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	False	False	False		macos
32	resnet50_fp16	linalg	torch	1e-2	1e-2	default	nhcw-nhwc/img2col	True	True	True	Numerics issues, awaiting cuda-independent fp16 integration
33	squeezenet1_0	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	False	False	False		macos
34	wide_resnet50_2	linalg	torch	1e-2	1e-3	default	nhcw-nhwc/img2col	True	False	False		macos
35	efficientnet-v2-s	stablehlo	tf	1e-02	1e-3	default	nhcw-nhwc	False	False	False		macos
36	mnasnet1_0	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	True	True	True		macos
37	efficientnet_b0	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	True	True	True	https://github.com/nod-ai/SHARK/issues/1487	macos
38	efficientnet_b7	linalg	torch	1e-2	1e-3	default	nhcw-nhwc	True	True	True	https://github.com/nod-ai/SHARK/issues/1487	macos
39	efficientnet_b0	stablehlo	tf	1e-2	1e-3	default	nhcw-nhwc	False	False	False
40	efficientnet_b7	stablehlo	tf	1e-2	1e-3	default	nhcw-nhwc	False	False	False	Fails on MacOS builder, VK device lost	macos
41	gpt2	stablehlo	tf	1e-2	1e-3	default	None	True	False	False		macos
42	t5-base	linalg	torch	1e-2	1e-3	default	None	True	True	True	Inputs for seq2seq models in torch currently unsupported.	macos
43	t5-base	stablehlo	tf	1e-2	1e-3	default	None	False	False	False		macos
44	t5-large	linalg	torch	1e-2	1e-3	default	None	True	True	True	Inputs for seq2seq models in torch currently unsupported	macos
45	t5-large	stablehlo	tf	1e-2	1e-3	default	None	False	False	False		macos

4.5 KiB Raw Permalink Blame History

4.5 KiB

Raw Permalink Blame History