ROCm-LLMExt 25.08 release notes

ROCm-LLMExt 25.08 release notes#

3 min read time

Applies to Linux

This is the first release of the AMD ROCm LLMExt toolkit (ROCm-LLMExt), an open-source software toolkit built on the ROCm platform for large language model (LLM) extensions, integrations, and performance enablement on AMD GPUs. The domain brings together training, post-training, inference, and orchestration components to make modern LLM stacks practical and reproducible on AMD hardware.

Release highlights#

This release introduces support for ROCm 6.4.1 for the following component:

  • Ray is a unified framework for scaling AI and Python applications from your laptop to a full cluster, without changing your code. Ray consists of a core distributed runtime and a set of AI libraries for simplifying machine learning computations. Ray is a general-purpose framework that runs many types of workloads efficiently. Any Python application can be scaled with Ray, without extra infrastructure.

This release introduces support for ROCm 6.4.0 for the following component:

  • llama.cpp is an open-source inference library and framework for large language model (LLM) inference that runs on both central processing units (CPUs) and graphics processing units (GPUs). It is written in plain C/C++, providing a simple, dependency-free setup.

This release introduces support for ROCm 6.3.0 for the following components:

  • Megablocks is a lightweight library for mixture-of-experts (MoE) training. The core of the system is efficient “dropless-MoE” and standard MoE layers.

  • Stanford Megatron-LM is a large-scale language model training framework. It provides efficient tensor, pipeline, and sequence-based model parallelism for pre-training transformer-based language models such as GPT (Decoder Only), BERT (Encoder Only), and T5 (Encoder-Decoder).

This release introduces support for ROCm 6.2.0 for the following component:

  • verl is a flexible, efficient and production-ready RL training library designed for large language models (LLMs) post-training.

System requirements#

For the 25.08 release, the ROCm‑LLMExt components span a range of ROCm version requirements depending on the specific extension. Ensure you follow the installation instructions for each individual component, where the exact ROCm dependency is listed or refer to the compatibility matrix to verify supported ROCm versions.

ROCm-LLMExt components#

The following table lists the ROCm-LLMExt component version for the 25.08 release. Click to go to the component’s source on GitHub.

Name Version Source
verl 0.3.0.post0
Stanford Megatron-LM 85f95ae
Megablocks 0.7.0
Ray 2.48.0.post0
llama.cpp b5997
Contents