AMD LLM Extension documentation

AMD LLM Extension documentation#

2026-10-14

1 min read time

Applies to Linux

AMD LLM Extension is an open-source software toolkit built on the ROCm platform for large language model (LLM) extensions, integrations, and performance enablement on AMD GPUs. The domain brings together training, post-training, inference, and orchestration components to make modern LLM stacks practical and reproducible on AMD hardware.

LLM Task

Features

Training

  • Large-scale transformer training

  • Distributed parallelism (data, tensor, and pipeline)

  • Mixed precision and performance tuning

  • Mixture-of-Experts (MoE) enablement

Post-training and alignment

  • Reinforcement learning and post-training workflows

  • Scalable experimentation

  • Reproducible configurations

Inference and serving

  • High-throughput decoding and low-latency serving

  • Optimized attention and inference operators

  • Lightweight and edge-friendly inference paths

Distributed execution

  • Multi-node orchestration

  • Cluster bring-up and scheduling

  • Batch and online inference pipelines

The AMD LLM Extension source code is hosted on GitHub at AMD-Ecosystem/llmextension.

Note

AMD LLM Extension 25.09 includes targeted updates to one component (llama.cpp) and introduces another (FlashInfer); four components remain unchanged (verl, Stanford Megatron-LM, Megablocks, and Ray).

AMD LLM Extension documentation is organized into the following categories:

Install
Components
Resources

To contribute to the documentation, see Contributing to ROCm.

You can find licensing information on the Licensing page.