mirror of
https://github.com/nod-ai/AMD-SHARK-Studio.git
synced 2026-02-19 11:56:43 -05:00
07f6f4a2f7
* Small changes to OPT example. * Update opt README. * Add a few modes to batch script. * Update README.md
1.1 KiB
1.1 KiB
Run OPT for sentence completion through SHARK
From base SHARK directory, follow instructions to set up a virtual environment with SHARK. (./setup_venv.sh or ./setup_venv.ps1)
Then, you may run opt_causallm.py to get a very simple sentence completion application running through SHARK
python opt_causallm.py
Run OPT performance comparison on SHARK vs. PyTorch
python opt_perf_comparison.py --max-seq-len=512 --model-name=facebook/opt-1.3b \
--platform=shark
Any OPT model from huggingface should work with this script, and you can choose between --platform=shark or --platform=huggingface to generate benchmarks of OPT inference on SHARK / PyTorch.
Run a small suite of OPT models through the benchmark script
python opt_perf_comparison_batch.py
This script will run benchmarks from a suite of OPT configurations:
- Sequence Lengths: 32, 128, 256, 512
- Parameter Counts: 125m, 350m, 1.3b
note: Most of these scripts are written for use on CPU, as perf comparisons against pytorch can be problematic across platforms otherwise.