Install ROCm-LLMExt

2026-03-20

1 min read time

Applies to Linux

This topic provides brief guidance and recommendations on setting up your environment for LLM workloads. This includes verifying prerequisites before installing ROCm-LLMExt frameworks and libraries.

System requirements

The ROCm-LLMExt components are all supported on AMD Instinct MI300X GPUs. Individual components also support additional hardware configurations.

Before installing any ROCm-LLMExt component, verify that your system meets the hardware and software prerequisites for each component, as outlined in the ROCm-LLMExt compatibility matrix page.

Install ROCm-LLMExt components

Each component has distinct prerequisite and environment requirements. To avoid dependency conflicts and configuration errors, components must be installed individually rather than through a unified installation process.

The installation instructions for each component, on ROCm can be found as follows:

• FlashInfer on ROCm

• llama.cpp on ROCm

• Megablocks on ROCm

• Ray on ROCm

• Stanford Megatron-LM on ROCm

• verl on ROCm