Fix incorrect throughput benchmark command in inference/vllm-benchmark.rst (#4723)

* update inference index to include pyt inference

* fix incorrect command in throughput benchmark

* wording
This commit is contained in:
Peter Park
2025-05-08 09:24:51 -04:00
committed by GitHub
parent 8ef1bb0139
commit bb7af3351a
2 changed files with 6 additions and 4 deletions

View File

@@ -20,6 +20,8 @@ training, fine-tuning, and inference. It leverages popular machine learning fram
- :doc:`LLM inference frameworks <llm-inference-frameworks>`
- :doc:`Performance testing <vllm-benchmark>`
- :doc:`vLLM inference performance testing <vllm-benchmark>`
- :doc:`PyTorch inference performance testing <pytorch-inference-benchmark>`
- :doc:`Deploying your model <deploy-your-model>`

View File

@@ -276,7 +276,7 @@ vLLM inference performance testing
* Latency benchmark
Use this command to benchmark the latency of the {{model.model}} model on eight GPUs with the ``{{model.precision}}`` data type.
Use this command to benchmark the latency of the {{model.model}} model on eight GPUs with ``{{model.precision}}`` precision.
.. code-block::
@@ -286,11 +286,11 @@ vLLM inference performance testing
* Throughput benchmark
Use this command to throughput the latency of the {{model.model}} model on eight GPUs with the ``{{model.precision}}`` data type.
Use this command to benchmark the throughput of the {{model.model}} model on eight GPUs with ``{{model.precision}}`` precision.
.. code-block:: shell
./vllm_benchmark_report.sh -s latency -m {{model.model_repo}} -g 8 -d {{model.precision}}
./vllm_benchmark_report.sh -s throughput -m {{model.model_repo}} -g 8 -d {{model.precision}}
Find the throughput report at ``./reports_{{model.precision}}_vllm_rocm{{unified_docker.rocm_version}}/summary/{{model.model_repo.split('/', 1)[1] if '/' in model.model_repo else model.model_repo}}_throughput_report.csv``.