---
myst:
  html_meta:
    "description lang=en": "How to run ROCm workloads in Docker containers on AMD GPUs."
    "keywords": "Docker, Podman, AMD, ROCm, GPU, container, amd-container,runtime, CDI"
---

# Run ROCm Docker containers

Docker is a popular way to run ROCm workloads in a consistent, reproducible environment.
There are two ways to expose AMD GPUs to Docker containers:

* {ref}`AMD Container Runtime Toolkit <docker-with-toolkit>` — simplifies
  GPU access within Docker environments, enhances device discovery, and enables
  better integration with modern container technologies.

* {ref}`Manual Docker device passthrough <docker-manual>` — passes GPU device nodes
  directly to the container with `--device`. No toolkit required.

## Prerequisites

Regardless of which approach you use, the following are required on the host system:

* On Linux, the AMD GPU driver (`amdgpu-dkms`). See {doc}`/install/rocm` for
  driver installation guidance for your environment.

* [Docker Engine](https://docs.docker.com/engine/install/).

(docker-with-toolkit)=

## With the AMD Container Runtime Toolkit

The [AMD Container Runtime
Toolkit](https://instinct.docs.amd.com/projects/container-toolkit/en/latest/container-runtime/overview.html)
provides two mechanisms for GPU injection: CDI (recommended) and the
amd-container-runtime. Both require installing the toolkit first.

1. Install the toolkit by following the [Quick Start Guide (AMD Container
   Runtime Toolkit
   docs)](https://instinct.docs.amd.com/projects/container-toolkit/en/latest/container-runtime/quick-start-guide.html).

2. See [Running Workloads (AMD Container Runtime Toolkit
   docs)](https://instinct.docs.amd.com/projects/container-toolkit/en/latest/container-runtime/running-workloads.html)
   to get started running containerized ROCm applications on AMD GPUs.

(docker-manual)=

## Without the AMD Container Runtime Toolkit

If you prefer not to install the AMD Container Runtime Toolkit, or you're using
older versions of Docker Engine, you can pass GPU device nodes directly to the
container using Docker's `--device` flag. This approach requires no additional
software beyond Docker.

```bash
 docker run -it --rm \
     --device /dev/kfd \
     --device /dev/dri \
     --security-opt seccomp=unconfined \
     <image>
```

The purpose of each option:

* `--device /dev/kfd`

  The main compute interface, shared by all GPUs.

* `--device /dev/dri`

  Contains the Direct Rendering Interface (DRI) device nodes for each GPU.
  Passing the whole directory grants access to all GPUs. To restrict access
  to specific GPUs, see {ref}`docker-restrict-gpus`.

* `--security-opt seccomp=unconfined` (optional)

  Enables memory mapping. Recommended for HPC workloads that use `numactl`
  for GPU/CPU affinity mappings.
  See [Optional security options (Docker docs)](https://docs.docker.com/reference/cli/docker/container/run/#security-opt).

(docker-restrict-gpus)=

### Restricting GPU access

```{tip}
[AMD Container Runtime
Toolkit](https://instinct.docs.amd.com/projects/container-toolkit/en/latest/container-runtime/overview.html)
simplifies GPU selection, removing the need to manually map the render nodes.
This is recommended if your use case needs fine-grained GPU selection. See
[Running Workloads (AMD Container Toolkit Runtime
docs)](https://instinct.docs.amd.com/projects/container-toolkit/en/latest/container-runtime/running-workloads.html#running-workloads)
for more information.
```

By default, `--device /dev/dri` grants access to all GPUs on the system. To
limit a container to specific GPUs, pass their individual render nodes instead.

List available render nodes on the host:

```bash
ls /dev/dri/render*
```

GPU render nodes are typically named `renderD128`, `renderD129`, and so on.
Pass them individually alongside `/dev/kfd`:

```bash
docker run --device /dev/kfd \
    --device /dev/dri/renderD128 \
    --device /dev/dri/renderD129 \
    <image>
```

```{note}
When GPUs are partitioned (for example, an Instinct MI300X or MI350X in DPX,
QPX, or CPX mode), each partition appears as a separate render node. In CPX
mode, `renderD128` through `renderD136` are partitions of the first physical
GPU, and `renderD137` is the second GPU. Account for this when selecting render
nodes. See [GPU
partitioning](https://instinct.docs.amd.com/projects/amdgpu-docs/en/latest/gpu-partitioning/mi300x/overview.html).
for details.
```

## Docker Compose

Docker Compose can simplify complex Docker invocations. See [Docker Compose
usage](https://instinct.docs.amd.com/projects/container-toolkit/en/latest/container-runtime/docker-compose.html)
for AMD Container Toolkit configuration examples.

For manual Docker device passthrough, use the `devices` key in your Compose file:

```yaml
services:
  myapp:
    image: <image>
    devices:
      - /dev/kfd
      - /dev/dri
```

## Verifying GPU access

Inside any container with ROCm installed, `rocminfo` and `amd-smi list`
enumerate only the GPUs passed into that container. On the host, they enumerate
all ROCm-capable GPUs.

{doc}`rocminfo <rocminfo:index>` and {doc}`amd-smi <amdsmi:index>` are provided
by the ROCm Core SDK. Run either tool to confirm the expected GPUs are
visible.

## Docker images

The [ROCm Docker repository](https://github.com/ROCm/ROCm-docker) hosts Dockerfiles
for building ROCm-capable containers. Pre-built images are available on
[Docker Hub (`u/rocm`)](https://hub.docker.com/u/rocm):

* `rocm/rocm-terminal` — minimal image with prerequisites to build HIP
  applications, without any libraries.

* [ROCm dev images](https://hub.docker.com/u/rocm?page=1&search=dev-) — a variety
  of OS and ROCm version combinations, suitable as a base for building applications.

* The [ROCm AI
  ecosystem](https://rocm.docs.amd.com/projects/ai-ecosystem/en/latest/)
  provides Docker images for popular inference frameworks.
