AMD LLM Extension 26.10 release notes

AMD LLM Extension 26.10 release notes#

2 min read time

Applies to Linux

This is the eighth release of the AMD LLM Extension toolkit, an open-source software toolkit built on the ROCm platform for large language model (LLM) extensions, integrations, and performance enablement on AMD GPUs. The domain brings together training, post-training, inference, and orchestration components to make modern LLM stacks practical and reproducible on AMD hardware.

Release highlights#

Note

AMD LLM Extension 26.10 introduces updates to two components (FlashInfer and Ray) as part of the toolkit; other components remain unchanged (verl, ROCm-RAG, and Triton Inference Server).

This release updates the following components with support for ROCm 10.1.0:

  • FlashInfer is a library and kernel generator that provides high-performance LLM GPU kernels, including attention and decoding operators.

  • Ray is an open-source framework for scaling Python and AI workloads, providing the distributed compute and orchestration layer used by verl for multi-GPU and multi-node training.

System requirements#

AMD LLM Extension components span a range of ROCm version requirements depending on the specific component. Ensure you follow the installation instructions for each component, which list the exact ROCm dependencies, or refer to the Compatibility matrix to verify the supported ROCm versions.

AMD LLM Extension components#

The following table lists AMD LLM Extension component versions for the 26.10 release. Click to go to the component’s source on GitHub.

Name Version Source
FlashInfer 0.5.3 ⇒ 0.6.18
Ray 2.58.0 ⇒ 2.59.0
verl 0.9.0
Triton Inference Server 25.12
ROCm-RAG 1.0.0

Detailed component changelogs#

FlashInfer 0.6.18#

This release updates FlashInfer in AMD LLM Extension from 0.5.3 to 0.6.18, and adds support for ROCm 10.1.0 on AMD Instinct MI300X, MI325X, MI350X, and MI355X GPUs with PyTorch 2.12.0 and Python 3.12 on Ubuntu 24.04. FlashInfer 0.6.18 requires AITER 0.1.21.post2, built from source.

Ray 2.59.0#

This release updates Ray in AMD LLM Extension from 2.58.0 to 2.59.0, and adds support for ROCm 10.1.0 on AMD Instinct MI300X, MI325X, MI350X, and MI355X GPUs with PyTorch 2.13.0 and Python 3.14 on Ubuntu 24.04. Ray 2.59.0 requires vLLM 0.29.0.

Known issues#

  • RayTrain: It is recommended to use the legacy V1 API by explicitly setting os.environ["RAY_TRAIN_V2_ENABLED"] = "0".

Contents