verl compatibility#
2025-10-21
4 min read time
Volcano Engine Reinforcement Learning for LLMs (verl) is a reinforcement learning framework designed for large language models (LLMs). verl offers a scalable, open-source fine-tuning solution by using a hybrid programming model that makes it easy to define and run complex post-training dataflows efficiently.
Its modular APIs separate computation from data, allowing smooth integration with other frameworks. It also supports flexible model placement across GPUs for efficient scaling on different cluster sizes. verl achieves high training and generation throughput by building on existing LLM frameworks. Its 3D-HybridEngine reduces memory use and communication overhead when switching between training and inference, improving overall performance.
Support overview#
The ROCm-supported version of verl is maintained in the official ROCm/verl repository, which differs from the volcengine/verl upstream repository.
To get started and install verl on ROCm, use the prebuilt Docker image, which includes ROCm, verl, and all required dependencies.
See the ROCm verl installation guide for installation and setup instructions.
You can also consult the upstream verl documentation for additional context.
Version support#
verl is supported on ROCm 6.2.0.
Supported devices#
Officially Supported: AMD Instinct™ MI300X
Use cases and recommendations#
The benefits of verl in large-scale reinforcement learning from human feedback (RLHF) are discussed in the Reinforcement Learning from Human Feedback on AMD GPUs with verl and ROCm Integration blog. The blog post outlines how the Volcano Engine Reinforcement Learning (verl) framework integrates with the AMD ROCm platform to optimize training on Instinct™ MI300X GPUs. The guide details the process of building a Docker image, setting up single-node and multi-node training environments, and highlights performance benchmarks demonstrating improved throughput and convergence accuracy. This resource serves as a comprehensive starting point for deploying verl on AMD GPUs, facilitating efficient RLHF training workflows.
Supported features#
The following table shows verl on ROCm support for GPU-accelerated modules.
Module |
Description |
verl version |
ROCm version |
|---|---|---|---|
|
Training engine |
0.3.0.post0 |
6.2.0 |
|
Inference engine |
0.3.0.post0 |
6.2.0 |
Docker image compatibility#
AMD validates and publishes ready-made verl Docker images with ROCm backends on Docker Hub. The following Docker image tag and associated inventories represent the latest verl version from the official Docker Hub. Click to view the image on Docker Hub.