xDiT diffusion inference

2025-12-17

17 min read time

Applies to Linux

The rocm/pytorch-xdit Docker image offers a prebuilt, optimized environment based on xDiT for benchmarking diffusion model video and image generation on AMD Instinct MI355X, MI350X (gfx950), MI325X, and MI300X (gfx942) GPUs.

The image runs ROCm 7.10.0 (preview) based on TheRock and includes the following components:

Software components

Software component	Version
TheRock	3e3f834
rccl	d23d18f
composable_kernel	2570462
rocm-libraries	0588f07
rocm-systems	473025a
torch	73adac
torchvision	f5c6c2e
triton	7416ffc
accelerate	34c1779
aiter	de14bec
diffusers	40528e9
xfuser	ccba9d5
yunchang	2c9b712

Follow this guide to pull the required image, spin up a container, download the model, and run a benchmark. For preview and development releases, see amdsiloai/pytorch-xdit.

What’s new

• Adds T2V and TI2V support for Wan models.

• Adds support for SD-3.5 T2I model.

Supported models

The following models are supported for inference performance benchmarking. Some instructions, commands, and recommendations in this documentation might vary by model – select one to get started.

Model

Hunyuan Video

Wan-AI

FLUX

Stable Diffusion

Variant

Hunyuan Video

Wan2.1

Wan2.2

FLUX.1

stable-diffusion-3.5-large

Note

To learn more about your specific model see the Hunyuan Video model card on Hugging Face or visit the GitHub page. Note that some models require access authorization before use via an external license agreement through a third party.

Note

To learn more about your specific model see the Wan2.1 model card on Hugging Face or visit the GitHub page. Note that some models require access authorization before use via an external license agreement through a third party.

Note

To learn more about your specific model see the Wan2.2 model card on Hugging Face or visit the GitHub page. Note that some models require access authorization before use via an external license agreement through a third party.

Note

To learn more about your specific model see the FLUX.1 model card on Hugging Face or visit the GitHub page. Note that some models require access authorization before use via an external license agreement through a third party.

Note

To learn more about your specific model see the stable-diffusion-3.5-large model card on Hugging Face or visit the GitHub page. Note that some models require access authorization before use via an external license agreement through a third party.

System validation

Before running AI workloads, it’s important to validate that your AMD hardware is configured correctly and performing optimally.

If you have already validated your system settings, including aspects like NUMA auto-balancing, you can skip this step. Otherwise, complete the procedures in the System validation and optimization guide to properly configure your system settings before starting.

To test for optimal performance, consult the recommended System health benchmarks. This suite of tests will help you verify and fine-tune your system’s configuration.

Pull the Docker image

For this tutorial, it’s recommended to use the latest rocm/pytorch-xdit:v25.12 Docker image. Pull the image using the following command:

docker pull rocm/pytorch-xdit:v25.12

Validate and benchmark

Once the image has been downloaded you can follow these steps to run benchmarks and generate outputs.

The following commands are written for Hunyuan Video. See Supported models to switch to another available model.

The following commands are written for Wan2.1. See Supported models to switch to another available model.

The following commands are written for Wan2.2. See Supported models to switch to another available model.

The following commands are written for FLUX.1. See Supported models to switch to another available model.

The following commands are written for stable-diffusion-3.5-large. See Supported models to switch to another available model.

Choose your setup method

You can either use an existing Hugging Face cache or download the model fresh inside the container.

Option 1: Use existing Hugging Face cache

If you already have models downloaded on your host system, you can mount your existing cache.

1. Set your Hugging Face cache location.

   export HF_HOME=/your/hf_cache/location
   

2. Download the model (if not already cached).

   huggingface-cli download tencent/HunyuanVideo  --revision refs/pr/18 
   

3. Launch the container with mounted cache.

   docker run \
       -it --rm \
       --cap-add=SYS_PTRACE \
       --security-opt seccomp=unconfined \
       --user root \
       --device=/dev/kfd \
       --device=/dev/dri \
       --group-add video \
       --ipc=host \
       --network host \
       --privileged \
       --shm-size 128G \
       --name pytorch-xdit \
       -e HSA_NO_SCRATCH_RECLAIM=1 \
       -e OMP_NUM_THREADS=16 \
       -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
       -e HF_HOME=/app/huggingface_models \
       -v $HF_HOME:/app/huggingface_models \
       rocm/pytorch-xdit:v25.12
   

Option 2: Download inside container

If you prefer to keep the container self-contained or don’t have an existing cache.

1. Launch the container

   docker run \
       -it --rm \
       --cap-add=SYS_PTRACE \
       --security-opt seccomp=unconfined \
       --user root \
       --device=/dev/kfd \
       --device=/dev/dri \
       --group-add video \
       --ipc=host \
       --network host \
       --privileged \
       --shm-size 128G \
       --name pytorch-xdit \
       -e HSA_NO_SCRATCH_RECLAIM=1 \
       -e OMP_NUM_THREADS=16 \
       -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
       rocm/pytorch-xdit:v25.12
   

2. Inside the container, set the Hugging Face cache location and download the model.

   export HF_HOME=/app/huggingface_models
   huggingface-cli download tencent/HunyuanVideo  --revision refs/pr/18 
   

   Warning

   Models will be downloaded to the container’s filesystem and will be lost when the container is removed unless you persist the data with a volume.

Option 1: Use existing Hugging Face cache

If you already have models downloaded on your host system, you can mount your existing cache.

1. Set your Hugging Face cache location.

   export HF_HOME=/your/hf_cache/location
   

2. Download the model (if not already cached).

   huggingface-cli download Wan-AI/Wan2.1-I2V-14B-720P-Diffusers 
   

3. Launch the container with mounted cache.

   docker run \
       -it --rm \
       --cap-add=SYS_PTRACE \
       --security-opt seccomp=unconfined \
       --user root \
       --device=/dev/kfd \
       --device=/dev/dri \
       --group-add video \
       --ipc=host \
       --network host \
       --privileged \
       --shm-size 128G \
       --name pytorch-xdit \
       -e HSA_NO_SCRATCH_RECLAIM=1 \
       -e OMP_NUM_THREADS=16 \
       -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
       -e HF_HOME=/app/huggingface_models \
       -v $HF_HOME:/app/huggingface_models \
       rocm/pytorch-xdit:v25.12
   

Option 2: Download inside container

If you prefer to keep the container self-contained or don’t have an existing cache.

1. Launch the container

   docker run \
       -it --rm \
       --cap-add=SYS_PTRACE \
       --security-opt seccomp=unconfined \
       --user root \
       --device=/dev/kfd \
       --device=/dev/dri \
       --group-add video \
       --ipc=host \
       --network host \
       --privileged \
       --shm-size 128G \
       --name pytorch-xdit \
       -e HSA_NO_SCRATCH_RECLAIM=1 \
       -e OMP_NUM_THREADS=16 \
       -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
       rocm/pytorch-xdit:v25.12
   

2. Inside the container, set the Hugging Face cache location and download the model.

   export HF_HOME=/app/huggingface_models
   huggingface-cli download Wan-AI/Wan2.1-I2V-14B-720P-Diffusers 
   

   Warning

   Models will be downloaded to the container’s filesystem and will be lost when the container is removed unless you persist the data with a volume.

Option 1: Use existing Hugging Face cache

If you already have models downloaded on your host system, you can mount your existing cache.

1. Set your Hugging Face cache location.

   export HF_HOME=/your/hf_cache/location
   

2. Download the model (if not already cached).

   huggingface-cli download Wan-AI/Wan2.2-I2V-A14B-Diffusers 
   

3. Launch the container with mounted cache.

   docker run \
       -it --rm \
       --cap-add=SYS_PTRACE \
       --security-opt seccomp=unconfined \
       --user root \
       --device=/dev/kfd \
       --device=/dev/dri \
       --group-add video \
       --ipc=host \
       --network host \
       --privileged \
       --shm-size 128G \
       --name pytorch-xdit \
       -e HSA_NO_SCRATCH_RECLAIM=1 \
       -e OMP_NUM_THREADS=16 \
       -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
       -e HF_HOME=/app/huggingface_models \
       -v $HF_HOME:/app/huggingface_models \
       rocm/pytorch-xdit:v25.12
   

Option 2: Download inside container

If you prefer to keep the container self-contained or don’t have an existing cache.

1. Launch the container

   docker run \
       -it --rm \
       --cap-add=SYS_PTRACE \
       --security-opt seccomp=unconfined \
       --user root \
       --device=/dev/kfd \
       --device=/dev/dri \
       --group-add video \
       --ipc=host \
       --network host \
       --privileged \
       --shm-size 128G \
       --name pytorch-xdit \
       -e HSA_NO_SCRATCH_RECLAIM=1 \
       -e OMP_NUM_THREADS=16 \
       -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
       rocm/pytorch-xdit:v25.12
   

2. Inside the container, set the Hugging Face cache location and download the model.

   export HF_HOME=/app/huggingface_models
   huggingface-cli download Wan-AI/Wan2.2-I2V-A14B-Diffusers 
   

   Warning

   Models will be downloaded to the container’s filesystem and will be lost when the container is removed unless you persist the data with a volume.

Option 1: Use existing Hugging Face cache

If you already have models downloaded on your host system, you can mount your existing cache.

1. Set your Hugging Face cache location.

   export HF_HOME=/your/hf_cache/location
   

2. Download the model (if not already cached).

   huggingface-cli download black-forest-labs/FLUX.1-dev 
   

3. Launch the container with mounted cache.

   docker run \
       -it --rm \
       --cap-add=SYS_PTRACE \
       --security-opt seccomp=unconfined \
       --user root \
       --device=/dev/kfd \
       --device=/dev/dri \
       --group-add video \
       --ipc=host \
       --network host \
       --privileged \
       --shm-size 128G \
       --name pytorch-xdit \
       -e HSA_NO_SCRATCH_RECLAIM=1 \
       -e OMP_NUM_THREADS=16 \
       -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
       -e HF_HOME=/app/huggingface_models \
       -v $HF_HOME:/app/huggingface_models \
       rocm/pytorch-xdit:v25.12
   

Option 2: Download inside container

If you prefer to keep the container self-contained or don’t have an existing cache.

1. Launch the container

   docker run \
       -it --rm \
       --cap-add=SYS_PTRACE \
       --security-opt seccomp=unconfined \
       --user root \
       --device=/dev/kfd \
       --device=/dev/dri \
       --group-add video \
       --ipc=host \
       --network host \
       --privileged \
       --shm-size 128G \
       --name pytorch-xdit \
       -e HSA_NO_SCRATCH_RECLAIM=1 \
       -e OMP_NUM_THREADS=16 \
       -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
       rocm/pytorch-xdit:v25.12
   

2. Inside the container, set the Hugging Face cache location and download the model.

   export HF_HOME=/app/huggingface_models
   huggingface-cli download black-forest-labs/FLUX.1-dev 
   

   Warning

   Models will be downloaded to the container’s filesystem and will be lost when the container is removed unless you persist the data with a volume.

Option 1: Use existing Hugging Face cache

If you already have models downloaded on your host system, you can mount your existing cache.

1. Set your Hugging Face cache location.

   export HF_HOME=/your/hf_cache/location
   

2. Download the model (if not already cached).

   huggingface-cli download stabilityai/stable-diffusion-3.5-large 
   

3. Launch the container with mounted cache.

   docker run \
       -it --rm \
       --cap-add=SYS_PTRACE \
       --security-opt seccomp=unconfined \
       --user root \
       --device=/dev/kfd \
       --device=/dev/dri \
       --group-add video \
       --ipc=host \
       --network host \
       --privileged \
       --shm-size 128G \
       --name pytorch-xdit \
       -e HSA_NO_SCRATCH_RECLAIM=1 \
       -e OMP_NUM_THREADS=16 \
       -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
       -e HF_HOME=/app/huggingface_models \
       -v $HF_HOME:/app/huggingface_models \
       rocm/pytorch-xdit:v25.12
   

Option 2: Download inside container

If you prefer to keep the container self-contained or don’t have an existing cache.

1. Launch the container

   docker run \
       -it --rm \
       --cap-add=SYS_PTRACE \
       --security-opt seccomp=unconfined \
       --user root \
       --device=/dev/kfd \
       --device=/dev/dri \
       --group-add video \
       --ipc=host \
       --network host \
       --privileged \
       --shm-size 128G \
       --name pytorch-xdit \
       -e HSA_NO_SCRATCH_RECLAIM=1 \
       -e OMP_NUM_THREADS=16 \
       -e CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
       rocm/pytorch-xdit:v25.12
   

2. Inside the container, set the Hugging Face cache location and download the model.

   export HF_HOME=/app/huggingface_models
   huggingface-cli download stabilityai/stable-diffusion-3.5-large 
   

   Warning

   Models will be downloaded to the container’s filesystem and will be lost when the container is removed unless you persist the data with a volume.

Run inference

MAD-integrated benchmarking

1. Clone the ROCm Model Automation and Dashboarding (ROCm/MAD) repository to a local directory and install the required packages on the host machine.

   git clone https://github.com/ROCm/MAD
   cd MAD
   pip install -r requirements.txt
   

2. On the host machine, use this command to run the performance benchmark test on the Hunyuan Video model using one node.

   export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
   madengine run \
       --tags pyt_xdit_hunyuanvideo \
       --keep-model-dir \
       --live-output
   

MAD launches a Docker container with the name container_ci-pyt_xdit_hunyuanvideo. The throughput and serving reports of the model are collected in the following paths: pyt_xdit_hunyuanvideo_throughput.csv and pyt_xdit_hunyuanvideo_serving.csv.

Standalone benchmarking

To run the benchmarks for Hunyuan Video, use the following command:

cd /app/Hunyuanvideo
mkdir results

torchrun --nproc_per_node=8 run.py \
   --model tencent/HunyuanVideo \
   --prompt "In the large cage, two puppies were wagging their tails at each other." \
   --height 720 --width 1280 --num_frames 129 \
   --num_inference_steps 50 --warmup_steps 1 --n_repeats 1 \
   --ulysses_degree 8 \
   --enable_tiling --enable_slicing \
   --use_torch_compile \
   --bench_output results

The generated video will be stored under the results directory. For the actual benchmark step runtimes, see stdout.

MAD-integrated benchmarking

1. Clone the ROCm Model Automation and Dashboarding (ROCm/MAD) repository to a local directory and install the required packages on the host machine.

   git clone https://github.com/ROCm/MAD
   cd MAD
   pip install -r requirements.txt
   

2. On the host machine, use this command to run the performance benchmark test on the Wan2.1 model using one node.

   export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
   madengine run \
       --tags pyt_xdit_wan_2_1 \
       --keep-model-dir \
       --live-output
   

MAD launches a Docker container with the name container_ci-pyt_xdit_wan_2_1. The throughput and serving reports of the model are collected in the following paths: pyt_xdit_wan_2_1_throughput.csv and pyt_xdit_wan_2_1_serving.csv.

Standalone benchmarking

To run the benchmarks for Wan2.1, use the following command:

cd Wan
mkdir results

torchrun --nproc_per_node=8 /app/Wan/run.py \
   --task i2v \
   --height 720 \
   --width 1280 \
   --model Wan-AI/Wan2.1-I2V-14B-720P-Diffusers \
   --img_file_path /app/Wan/i2v_input.JPG \
   --ulysses_degree 8 \
   --seed 42 \
   --num_frames 81 \
   --prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline's intricate details and the refreshing atmosphere of the seaside." \
   --num_repetitions 1 \
   --num_inference_steps 40 \
   --use_torch_compile

The generated video will be stored under the results directory. For the actual benchmark step runtimes, see results/outputs/rank0_*.json

MAD-integrated benchmarking

1. Clone the ROCm Model Automation and Dashboarding (ROCm/MAD) repository to a local directory and install the required packages on the host machine.

   git clone https://github.com/ROCm/MAD
   cd MAD
   pip install -r requirements.txt
   

2. On the host machine, use this command to run the performance benchmark test on the Wan2.2 model using one node.

   export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
   madengine run \
       --tags pyt_xdit_wan_2_2 \
       --keep-model-dir \
       --live-output
   

MAD launches a Docker container with the name container_ci-pyt_xdit_wan_2_2. The throughput and serving reports of the model are collected in the following paths: pyt_xdit_wan_2_2_throughput.csv and pyt_xdit_wan_2_2_serving.csv.

Standalone benchmarking

To run the benchmarks for Wan2.2, use the following command:

cd Wan
mkdir results

torchrun --nproc_per_node=8 /app/Wan/run.py \
   --task i2v \
   --height 720 \
   --width 1280 \
   --model Wan-AI/Wan2.2-I2V-A14B-Diffusers \
   --img_file_path /app/Wan/i2v_input.JPG \
   --ulysses_degree 8 \
   --seed 42 \
   --num_frames 81 \
   --prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline's intricate details and the refreshing atmosphere of the seaside." \
   --num_repetitions 1 \
   --num_inference_steps 40 \
   --use_torch_compile

The generated video will be stored under the results directory. For the actual benchmark step runtimes, see results/outputs/rank0_*.json

MAD-integrated benchmarking

1. Clone the ROCm Model Automation and Dashboarding (ROCm/MAD) repository to a local directory and install the required packages on the host machine.

   git clone https://github.com/ROCm/MAD
   cd MAD
   pip install -r requirements.txt
   

2. On the host machine, use this command to run the performance benchmark test on the FLUX.1 model using one node.

   export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
   madengine run \
       --tags pyt_xdit_flux \
       --keep-model-dir \
       --live-output
   

MAD launches a Docker container with the name container_ci-pyt_xdit_flux. The throughput and serving reports of the model are collected in the following paths: pyt_xdit_flux_throughput.csv and pyt_xdit_flux_serving.csv.

Standalone benchmarking

To run the benchmarks for FLUX.1, use the following command:

cd Flux
mkdir results

torchrun --nproc_per_node=8 /app/Flux/run.py \
   --model black-forest-labs/FLUX.1-dev \
   --seed 42 \
   --prompt "A small cat" \
   --height 1024 \
   --width 1024 \
   --num_inference_steps 25 \
   --max_sequence_length 256 \
   --warmup_steps 5 \
   --no_use_resolution_binning \
   --ulysses_degree 8 \
   --use_torch_compile \
   --num_repetitions 50

The generated video will be stored under the results directory. For the actual benchmark step runtimes, see results/timing.json

You may also use run_usp.py which implements USP without modifying the default diffusers pipeline.

MAD-integrated benchmarking

1. Clone the ROCm Model Automation and Dashboarding (ROCm/MAD) repository to a local directory and install the required packages on the host machine.

   git clone https://github.com/ROCm/MAD
   cd MAD
   pip install -r requirements.txt
   

2. On the host machine, use this command to run the performance benchmark test on the stable-diffusion-3.5-large model using one node.

   export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
   madengine run \
       --tags pyt_xdit_sd_3_5 \
       --keep-model-dir \
       --live-output
   

MAD launches a Docker container with the name container_ci-pyt_xdit_sd_3_5. The throughput and serving reports of the model are collected in the following paths: pyt_xdit_sd_3_5_throughput.csv and pyt_xdit_sd_3_5_serving.csv.

Standalone benchmarking

To run the benchmarks for stable-diffusion-3.5-large, use the following command:

cd StableDiffusion3.5
mkdir results

torchrun --nproc_per_node=8 /app/StableDiffusion3.5/run.py \
   --model stabilityai/stable-diffusion-3.5-large \
   --num_inference_steps 28 \
   --prompt "A capybara holding a sign that reads Hello World" \
   --use_torch_compile \
   --pipefusion_parallel_degree 4 \
   --use_cfg_parallel \
   --num_repetitions 50 \
   --dtype torch.float16 \
   --output_path results

The generated video will be stored under the results directory. For the actual benchmark step runtimes, see benchmark_results.csv

Previous versions

See xDiT diffusion inference performance testing version history to find documentation for previous releases of xDiT diffusion inference performance testing.