Install AMD LLM Extension

2026-07-27

1 min read time

Applies to Linux

This topic provides brief guidance and recommendations on setting up your environment for LLM workloads. This includes verifying prerequisites before installing AMD LLM Extension frameworks and libraries.

System requirements

The AMD LLM Extension components are all supported on AMD Instinct MI300X GPUs. Individual components also support additional hardware configurations.

Before installing any AMD LLM Extension component, verify that your system meets the hardware and software prerequisites for each component, as outlined in the AMD LLM Extension compatibility matrix page.

Install AMD LLM Extension components

Each component has distinct prerequisite and environment requirements. To avoid dependency conflicts and configuration errors, components must be installed individually rather than through a unified installation process.

The installation instructions for each component on ROCm can be found as follows:

• ComfyUI

• FlashInfer on ROCm

• llama.cpp on ROCm

• Ray on ROCm

• ROCm-RAG

• Triton Inference Server on ROCm

• verl on ROCm