AMD LLM Extension documentation

AMD LLM Extension documentation#

2026-08-24

1 min read time

Applies to Linux

AMD LLM Extension is an open-source software toolkit built on the ROCm platform for large language model (LLM) extensions, integrations, and performance enablement on AMD GPUs. The domain brings together training, post-training, inference, and orchestration components to make modern LLM stacks practical and reproducible on AMD hardware.

LLM Task

Features

Training

  • Large-scale transformer training

  • Distributed parallelism (data, tensor, and pipeline)

  • Mixed precision and performance tuning

  • Mixture-of-Experts (MoE) enablement

Post-training and alignment

  • Reinforcement learning and post-training workflows

  • Scalable experimentation

  • Reproducible configurations

Inference and serving

  • High-throughput decoding and low-latency serving

  • Optimized attention and inference operators

  • Lightweight and edge-friendly inference paths

Distributed execution

  • Multi-node orchestration

  • Cluster bring-up and scheduling

  • Batch and online inference pipelines

The AMD LLM Extension source code is hosted on GitHub at AMD-Ecosystem/ROCm-LLMExt.

Note

AMD LLM Extension 26.06 introduces updates to two components (verl and Ray) as part of the toolkit; other components remain unchanged (ComfyUI, FlashInfer, llama.cpp, ROCm-RAG, and Triton Inference Server).

AMD LLM Extension documentation is organized into the following categories:

Install
Components
Resources