Getting started with ROCm CLI

Getting started with ROCm CLI#

Launch ROCm CLI with no arguments:

rocm

With no arguments on an interactive terminal, rocm opens the launcher — a small front-door menu that gets you to the common tasks:

  • Set up this system — install or update ROCm

  • Serve a model — run a model on your GPU

  • Diagnose & fix — check GPU, driver, and ROCm

  • Chat — talk to a local or API-backed model

  • Open full dashboard → — escalate into the live dashboard (rocm dash)

Pick a row with the arrow keys and Enter; press q — or Ctrl-C, which quits from the launcher and the dashboard alike and restores your terminal — to quit. The one exception is the dashboard’s console for a running job, where Ctrl-C keeps its existing meaning of “cancel this job” and does not quit; once that job finishes, Ctrl-C quits there too. On a non-interactive terminal (or piped output), rocm prints a one-shot status summary instead of opening the launcher.

Configure ROCm and serve a model#

Before serving a model, ensure a managed ROCm runtime is configured:

rocm install sdk

This downloads TheRock ROCm wheels and a matching PyTorch stack into a managed environment. On machines with an existing ROCm install, rocm examine will show it as legacy_rocm_status: detected_unmanaged — running rocm install sdk creates a separate managed runtime alongside it. Running the command when a managed runtime is already the active default asks first, because the new install takes over as the active default; see ROCm installation for that gate and the flags that approve it without a prompt.

Then serve a model:

rocm serve qwen

qwen is a built-in alias for a small assistant model that serves out of the box.

You can also serve any compatible Hugging Face model directly — see Model serving for the GGUF-vs-safetensors rule, since which form works depends on the engine your GPU selects.

Quick reference#

Command

Description

rocm

Open the launcher menu (setup, serve, diagnose, chat, dashboard)

rocm examine

Check GPU, ROCm install, engines, and managed folders

rocm diagnose

Match this machine against known ROCm/PyTorch/llama.cpp failure modes

rocm fix [<fix-id>]

Apply a fix reported by rocm diagnose

rocm install sdk

Install TheRock ROCm wheels into a managed Python environment

rocm install driver

Install the AMD kernel driver on Linux

rocm serve <model>

Start a local OpenAI-compatible model server

rocm dash

Open the full-screen telemetry dashboard

rocm bench load --endpoint <url>

Load-test a local OpenAI-compatible endpoint

rocm setup status

Show first-time setup state

rocm version

Print the rocm-cli version, release tag or branch, and commit hash, plus the ROCm SDK and GPU driver in use

rocm completions <shell>

Print a shell completion script (bash, zsh, fish, elvish, powershell)

Interactive interfaces#

rocm ships two terminal UIs built on ratatui:

The launcher (rocm)#

The lightweight hub described above. It runs the guided Set up, Serve, Diagnose, and Chat flows in place, and hands off to the full dashboard when you need live instruments. This is the default surface for everyday use; the legacy full-screen setup assistant has been retired.

The dashboard (rocm dash)#

The full-screen telemetry dashboard — every instrument and action on one screen. It auto-starts an embedded rocmd daemon when none is running, then presents five tabs (switch with Tab/Shift+Tab or number keys 1–5):

Tab

What it shows

Home

At-a-glance status: GPU, active runtime, running servers

ROCm

Guided ROCm and runtime actions with inline details

Serving

Start, inspect, and manage model servers

Observe

Live GPU utilization, instances, and benchmark telemetry

Chat

Assistant chat backed by a local server or configured provider

Live mode reads telemetry over a Unix domain socket, so it requires Linux or WSL. Use rocm dash --demo for a synthetic session that runs anywhere without a GPU or daemon.

Contents