Docker images for AI inference

2025-09-08

1 min read time

Applies to Linux

This page accompanies preview Docker images designed to reproduce inference performance on AMD Instinct™ MI355X, MI350X, and MI300X series accelerators. The images provide access to preview versions of the ROCm 7.0 software stack and are targeted at early-access users evaluating AI inference workloads using next-generation AMD accelerators.

Important

The following AI workload benchmarks use the ROCm 7.0 release candidate preview on AMD Instinct MI355X, MI350X, and MI300X series accelerators.

If you’re looking for production-level workloads for MI300X series accelerators, see Infinity Hub.

Inference

• Benchmark Llama 3.1 405B FP4 inference with vLLM

• Benchmark Llama 3.3 70B FP8 inference with vLLM

• Benchmark GPT OSS 120B inference with vLLM

• Benchmark DeepSeek R1 FP4 inference with SGLang

• Benchmark DeepSeek R1 FP8 inference with SGLang