# AMD LLM Extension 26.10 release notes This is the eighth release of the AMD LLM Extension toolkit, an open-source software toolkit built on the ROCm platform for large language model (LLM) extensions, integrations, and performance enablement on AMD GPUs. The domain brings together training, post-training, inference, and orchestration components to make modern LLM stacks practical and reproducible on AMD hardware. - [Release highlights](#release-highlights) - [System requirements](#system-requirements) - [AMD LLM Extension components](#amd-llm-extension-components) - [Detailed component changelogs](#detailed-component-changelogs) ## Release highlights ```{note} AMD LLM Extension 26.10 introduces updates to two components (FlashInfer and Ray) as part of the toolkit; other components remain unchanged (verl, ROCm-RAG, and Triton Inference Server). ``` This release updates the following components with support for [ROCm 10.1.0](https://rocm.docs.amd.com/en/docs-10.1.0/): * [FlashInfer](https://rocm.docs.amd.com/projects/flashinfer/en/docs-26.10/) is a library and kernel generator that provides high-performance LLM GPU kernels, including attention and decoding operators. * [Ray](https://rocm.docs.amd.com/projects/ray/en/docs-26.10/) is an open-source framework for scaling Python and AI workloads, providing the distributed compute and orchestration layer used by verl for multi-GPU and multi-node training. ## System requirements AMD LLM Extension components span a range of ROCm version requirements depending on the specific component. Ensure you follow the installation instructions for each component, which list the exact ROCm dependencies, or refer to the [Compatibility matrix](https://rocm.docs.amd.com/projects/llmextension/en/docs-26.10/about/compatibility-matrix.html) to verify the supported ROCm versions. ## AMD LLM Extension components The following table lists AMD LLM Extension component versions for the 26.10 release. Click {fab}`github` to go to the component's source on GitHub.
Name Version Source
FlashInfer 0.5.3 ⇒ 0.6.18
Ray 2.58.0 ⇒ 2.59.0
verl 0.9.0
Triton Inference Server 25.12
ROCm-RAG 1.0.0
## Detailed component changelogs ### FlashInfer 0.6.18 This release updates FlashInfer in AMD LLM Extension from 0.5.3 to 0.6.18, and adds support for ROCm 10.1.0 on AMD Instinct MI300X, MI325X, MI350X, and MI355X GPUs with PyTorch 2.12.0 and Python 3.12 on Ubuntu 24.04. FlashInfer 0.6.18 requires [AITER 0.1.21.post2](https://github.com/ROCm/aiter/releases/tag/v0.1.21.post2), built from source. ### Ray 2.59.0 This release updates Ray in AMD LLM Extension from 2.58.0 to 2.59.0, and adds support for ROCm 10.1.0 on AMD Instinct MI300X, MI325X, MI350X, and MI355X GPUs with PyTorch 2.13.0 and Python 3.14 on Ubuntu 24.04. Ray 2.59.0 requires [vLLM 0.29.0](https://github.com/vllm-project/vllm/releases/tag/v0.29.0). #### Known issues - **RayTrain**: It is recommended to use the legacy V1 API by explicitly setting `os.environ["RAY_TRAIN_V2_ENABLED"] = "0"`.