Installation with ROCm#

vLLM supports AMD GPUs with ROCm 5.7 and 6.0.


  • OS: Linux

  • Python: 3.8 – 3.11

  • GPU: MI200s (gfx90a), MI300 (gfx942), Radeon RX 7900 series (gfx1100)

  • ROCm 6.0 and ROCm 5.7

Installation options:

  1. Build from source with docker

  2. Build from source

Option 2: Build from source#

  1. Install prerequisites (skip if you are already in an environment/docker with the following installed):

For installing PyTorch, you can start from a fresh docker image, e.g, rocm6.0.2_ubuntu22.04_py3.10_pytorch_2.1.2, rocm/pytorch:rocm6.0_ubuntu20.04_py3.9_pytorch_2.1.1, rocm/pytorch-nightly.

Alternatively, you can install pytorch using pytorch wheels. You can check Pytorch installation guild in Pytorch Getting Started

For rocm6.0:

$ pip3 install torch --index-url

For rocm5.7:

$ pip install torch --index-url
  1. Install Triton flash attention for ROCm

Install ROCm’s Triton flash attention (the default triton-mlir branch) following the instructions from ROCm/triton

  1. Optionally, if you choose to use CK flash attention, you can install flash attention for ROCm

Install ROCm’s flash attention (v2.0.4) following the instructions from ROCm/flash-attention


  • If you are using rocm5.7 with pytorch 2.1.0 onwards, you don’t need to apply the hipify_python.patch. You can build the ROCm flash attention directly.

  • If you fail to install ROCm/flash-attention, try cloning from the commit 6fd2f8e572805681cd67ef8596c7e2ce521ed3c6.

  • ROCm’s Flash-attention-2 (v2.0.4) does not support sliding windows attention.

  • You might need to downgrade the “ninja” version to 1.10 it is not used when compiling flash-attention-2 (e.g. pip install ninja==

  1. Build vLLM.

$ cd vllm
$ pip install -U -r requirements-rocm.txt
$ python install # This may take 5-10 minutes. Currently, `pip install .`` does not work for ROCm installation


  • You may need to turn on the --enforce-eager flag if you experience process hang when running the script to test your installation.

  • Triton flash attention is used by default. For benchmarking purposes, it is recommended to run a warm up step before collecting perf numbers.

  • To use CK flash-attention, please use this flag export VLLM_USE_FLASH_ATTN_TRITON=0 to turn off triton flash attention.

  • The ROCm version of pytorch, ideally, should match the ROCm driver version.