FlashInfer on ROCm installation

FlashInfer on ROCm installation#

2026-10-30

2 min read time

Applies to Linux

System requirements#

To use FlashInfer 0.6.18, you need the following prerequisites:

  • ROCm version: 10.1.0

  • Operating system: Ubuntu 24.04

  • GPU platform: AMD Instinct™ MI300X, MI325X, MI350X, MI355X

  • PyTorch: 2.12.0

  • Python: 3.12

  • AITER: 0.1.21.post2 (source build)

Install FlashInfer#

FlashInfer on ROCm can be run directly by setting up a Docker container from scratch.

Build FlashInfer from source#

  1. Clone the AMD-Ecosystem/flashinfer repository.

    git clone https://github.com/AMD-Ecosystem/flashinfer.git

  2. Enter the directory and build the Dockerfile (AMD-Ecosystem/flashinfer) to create a Docker image.

    cd flashinfer
docker build \
  -t rocm-flashinfer-dev \
  -f docker/Dockerfile.rocm . \
  --build-arg USERNAME=$USER \
  --build-arg USER_UID=$(id -u) \
  --build-arg USER_GID=$(id -g)

  3. Start a Docker container using the image.

    docker run -it --rm \
--privileged --network=host --device=/dev/kfd \
--device=/dev/dri --group-add video \
--group-add "$(getent group render | cut -d: -f3)" \
--cap-add=SYS_PTRACE \
--security-opt seccomp=unconfined \
--shm-size 16G \
-v $PWD:/workspace \
rocm-flashinfer-dev

    Pass render as the host’s numeric GID, as shown. The image runs as a non-root user, so --privileged on its own does not grant access, and passing the group by name picks up the image’s own render group, whose GID does not match the device.

  4. Once you are inside the container, you can now install FlashInfer inside it.

    cd /workspace
python -m pip install --no-build-isolation -ve .

Test the FlashInfer installation#

Verify that FlashInfer is installed correctly.

python -c "import flashinfer; print(flashinfer.__version__)"

Expected output:

0.6.18+amd.2

If you see the version string above, FlashInfer 0.6.18 has been installed successfully. You can now use FlashInfer in your projects.

AITER Support#

FlashInfer on ROCm supports ROCm/aiter as a backend. AITER is a library for efficient attention operations.

The development image built above already includes the validated AITER build; no separate AITER installation is needed.

Contents