Benchmark suites of vLLM

Contents

Benchmark suites of vLLM#

vLLM contains two sets of benchmarks:

Performance benchmarks: benchmark vLLM’s performance under various workloads at a high frequency (when a pull request (PR for short) of vLLM is being merged). See vLLM performance dashboard for the latest performance results.
Nightly benchmarks: compare vLLM’s performance against alternatives (tgi, trt-llm, and lmdeploy) when there are major updates of vLLM (e.g., bumping up to a new version). The latest results are available in the vLLM GitHub README.

Trigger a benchmark#

The performance benchmarks and nightly benchmarks can be triggered by submitting a PR to vLLM, and label the PR with perf-benchmarks and nightly-benchmarks.

Note

Please refer to vLLM performance benchmark descriptions and vLLM nightly benchmark descriptions for detailed descriptions on benchmark environment, workload and metrics.