AMD ROCm AI ecosystem

ROCm is AMD’s open software platform for GPU computing. This documentation covers the AI ecosystem built on ROCm — from framework setup and training to inference serving and performance optimization.

Deep learning frameworks

Install PyTorch and JAX on AMD GPUs. Includes hardware-specific instructions for AMD Instinct and Radeon GPUs and Ryzen APUs across Linux and Windows using pip.

• Install PyTorch

• Install JAX

Training

Train and scale models on AMD GPUs. Primus provides end-to-end training infrastructure for multiple popular backend frameworks, including Megatron, TorchTitan, and MaxText.

• Primus

• Scale model training

Inference

Serve LLMs and generative AI models using high-performance inference frameworks. Covers single-node and distributed multi-GPU deployments.

• vLLM

• SGLang

• ATOM

• MIGraphX

• ONNX Runtime

• xDiT

• ComfyUI

Distributed inference

Infera orchestrates vLLM, SGLang, or ATOM workers and adds AMD-native tiered KV cache. The MoRI recipes cover prefill-decode disaggregated serving over RDMA networking on MI355X clusters.

• Infera

• vLLM with MoRI recipe

• SGLang with MoRI recipe

Optimization

Improve throughput, latency, and memory efficiency for AI workloads on AMD Instinct GPUs.

• Workload optimization

• vLLM V1 performance

• Model quantization

• Model acceleration libraries

• Triton kernels

• Composable Kernel

Tutorials

Hands-on guides and recipes for building AI applications on AMD hardware.

• AI Playbooks

• AI Developer Hub