AMD ROCm AI ecosystem

ROCm is AMD’s open software platform for GPU computing. This documentation covers the AI ecosystem built on ROCm — from framework setup and training to inference serving and performance optimization.

Deep learning frameworks

Install PyTorch and JAX on AMD GPUs. Includes hardware-specific instructions for AMD Instinct and Radeon GPUs and Ryzen APUs across Linux and Windows using pip.

• Install PyTorch

• Install JAX

Training

Scale model training across multiple AMD GPUs using PyTorch distributed primitives (DDP, RPC, collective communication) for large models that exceed single-GPU memory.

• Scale model training

Inference

Serve LLMs and generative AI models using high-performance inference frameworks. Covers single-node and distributed multi-GPU deployments.

• vLLM

• SGLang

• MIGraphX

• ONNX Runtime

• xDiT

• ComfyUI

Distributed inference

Multi-node prefill-decode disaggregated serving over RDMA networking using MoRI (Modular RDMA Interface) on MI355X clusters.

• vLLM with MoRI recipe

• SGLang with MoRI recipe

Optimization

Improve throughput, latency, and memory efficiency for AI workloads on AMD Instinct GPUs.

• Workload optimization

• vLLM V1 performance

• Model quantization

• Model acceleration libraries

• Triton kernels

• Composable Kernel

Tutorials

Hands-on guides and recipes for building AI applications on AMD hardware.

• AI Playbooks

• AI Developer Hub