Skip to content

vllm bench

Overview

vLLM bench subcommand.

Subcommands

Command Description
mm-processor Benchmark multimodal processor latency across different configurations.
latency Benchmark the latency of a single batch of requests.
serve Benchmark the online serving throughput.
startup Benchmark the startup time of vLLM models.
sweep Benchmark for a parameter sweep.
throughput Benchmark offline inference throughput.