Ray on ROCm installation#
2026-10-02
4 min read time
System requirements#
To use Ray 2.58.0, you need the following prerequisites:
Install Ray#
To install Ray on ROCm, you have the following options:
Build your own Docker image#
Clone the AMD-Ecosystem/ray repository:
git clone https://github.com/AMD-Ecosystem/ray.git -b release/2.58.0
Build the Docker container using the Dockerfile in the
ray/dockerdirectory:cd ray docker build -f docker/Dockerfile.rocm -t my-rocm-ray .
Launch and connect to the container:
docker run -it -d \ --network=host \ --device=/dev/kfd \ --device=/dev/dri \ --ipc=host \ --shm-size 128G \ --group-add video \ --cap-add=SYS_PTRACE \ --security-opt seccomp=unconfined \ -p 8265:8265 \ -v $(pwd):/host_dir \ -w /app \ --name rocm_ray \ my-rocm-ray \ /bin/bash docker attach rocm_ray
Note
The
--shm-sizeparameter allocates shared memory for the container. It can be adjusted based on your system’s resources. Replace$(pwd)with the absolute path to the directory you want to mount inside the container.Warning
--cap-add=SYS_PTRACEand--security-opt seccomp=unconfinedreduce container isolation.--network=hostand--ipc=hostshare the host network and IPC namespace with the container. Use this configuration on a trusted development host. On a shared or production host, drop the flags that your workload does not require.Verify the installed Ray version:
pip3 freeze | grep ray
Expected output:
ray==2.58.0
Install Ray on bare metal or a custom container#
Follow these steps if you prefer to install ROCm manually on your host system or in a custom container.
Install ROCm. Follow the ROCm installation guide to install ROCm on your system. That page is parameterized for AMD Instinct MI355X. Select your GPU on the page if you use a different Instinct platform.
Once installed, verify your ROCm installation using:
amd-smi
Expected output:
+------------------------------------------------------------------------------+ | AMD-SMI 27.0.0+6b0e43f3 | | amdgpu Version: 7.1.0.0 | | ROCm Version: 10.0.0 | | VBIOS Version: 00182096 | | FW PLDM: 00.25.06.05 | | Platform: Linux Baremetal | |-------------------------------------+----------------------------------------| | BDF GPU-Name | Mem-Uti Temp UEC Power-Usage | | GPU HIP-ID OAM-ID Partition-Mode | GFX-Uti Fan Mem-Usage | |=====================================+========================================| | 0000:1b:00.0 AMD Instinct MI300X | 0 % 43 °C 0 157/750 W | | 0 0 1 SPX/NPS1 | 0 % N/A 285/196592 MB | |-------------------------------------+----------------------------------------| | 0000:3d:00.0 AMD Instinct MI300X | 0 % 43 °C 0 154/750 W | | 1 1 0 SPX/NPS1 | 0 % N/A 285/196592 MB | |-------------------------------------+----------------------------------------| | 0000:4e:00.0 AMD Instinct MI300X | 0 % 42 °C 0 156/750 W | | 2 2 3 SPX/NPS1 | 0 % N/A 285/196592 MB | |-------------------------------------+----------------------------------------| | 0000:5f:00.0 AMD Instinct MI300X | 0 % 43 °C 0 152/750 W | | 3 3 2 SPX/NPS1 | 0 % N/A 285/196592 MB | |-------------------------------------+----------------------------------------| | 0000:9d:00.0 AMD Instinct MI300X | 0 % 43 °C 0 154/750 W | | 4 4 6 SPX/NPS1 | 0 % N/A 286/196592 MB | |-------------------------------------+----------------------------------------| | 0000:bd:00.0 AMD Instinct MI300X | 0 % 44 °C 0 151/750 W | | 5 5 7 SPX/NPS1 | 0 % N/A 285/196592 MB | |-------------------------------------+----------------------------------------| | 0000:cd:00.0 AMD Instinct MI300X | 0 % 43 °C 0 155/750 W | | 6 6 5 SPX/NPS1 | 0 % N/A 285/196592 MB | |-------------------------------------+----------------------------------------| | 0000:dd:00.0 AMD Instinct MI300X | 0 % 40 °C 0 152/750 W | | 7 7 4 SPX/NPS1 | 0 % N/A 285/196592 MB | +-------------------------------------+----------------------------------------+ +------------------------------------------------------------------------------+ | Processes: | | GPU PID Process Name GTT_MEM VRAM_MEM MEM_USAGE CU % SDMA | |==============================================================================| | No running processes found | +------------------------------------------------------------------------------+
Install PyTorch 2.12.0 with ROCm support. Follow the PyTorch on ROCm installation guide. That page is parameterized for AMD Instinct MI355X. Select your GPU on the page if you use a different Instinct platform.
Install vLLM 0.27.0 with ROCm support. Follow the vLLM on ROCm installation guide. That page is parameterized for AMD Instinct MI355X. Select your GPU on the page if you use a different Instinct platform.
Install the required version of Ray with ROCm support using pip:
pip install -U ray[default,serve]==2.58.0
Verify the installed Ray version:
pip3 freeze | grep ray
Expected output:
ray==2.58.0
Build Ray from source#
Follow the Building Ray from source guide to build Ray with ROCm support from source.
Test the Ray installation#
To run unit tests manually and validate your installation fully, follow these steps:
After launching the container, test whether Ray detects ROCm devices as expected.
python3 -c "import ray; ray.init(); print(ray.cluster_resources())"
If the setup is successful, the output should list all available ROCm devices.
Expected output (for example, on the MI300X node):
{'memory': 1420360912896.0, 'GPU': 8.0, 'accelerator_type:AMD-Instinct-MI300X-OAM': 1.0, 'node:10.7.39.110': 1.0, 'CPU': 384.0, 'node:__internal_head__': 1.0, 'object_store_memory': 200000000000.0}