Clone the ROCm Model Automation and Dashboarding (ROCm/MAD) repository to a local directory and install the required packages on the host machine.

git clone https://github.com/ROCm/MAD
cd MAD
pip install -r requirements.txt

For example, use this command to run the performance benchmark test on the Llama 3.1 8B model using one GPU with the float16 data type on the host machine.

export MAD_SECRETS_HFTOKEN="your personal Hugging Face token to access gated models"
python3 tools/run_models.py --tags pyt_train_llama-3.1-8b --keep-model-dir --live-output --timeout 28800

The available models for MAD-integrated benchmarking are:

• pyt_train_llama-3.3-70b

• pyt_train_llama-3.1-8b

• pyt_train_llama-3.1-70b

• pyt_train_flux

MAD launches a Docker container with the name container_ci-pyt_train_llama-3.1-8b, for example. The latency and throughput reports of the model are collected in the following path: ~/MAD/perf.csv.

Download the Docker image and required packages

Use the following command to pull the Docker image from Docker Hub.

docker pull rocm/pytorch-training:v25.5

Run the Docker container.

docker run -it --device /dev/dri --device /dev/kfd --network host --ipc host --group-add video --cap-add SYS_PTRACE --security-opt seccomp=unconfined --privileged -v $HOME:$HOME -v  $HOME/.ssh:/root/.ssh --shm-size 64G --name training_env rocm/pytorch-training:v25.5

Use these commands if you exit the training_env container and need to return to it.

docker start training_env
docker exec -it training_env bash

In the Docker container, clone the ROCm/MAD repository and navigate to the benchmark scripts directory /workspace/MAD/scripts/pytorch_train.

git clone https://github.com/ROCm/MAD
cd MAD/scripts/pytorch_train

Prepare training datasets and dependencies

The following benchmarking examples require downloading models and datasets from Hugging Face. To ensure successful access to gated repos, set your HF_TOKEN.

export HF_TOKEN=$your_personal_hugging_face_access_token

Run the setup script to install libraries and datasets needed for benchmarking.

./pytorch_benchmark_setup.sh

pytorch_benchmark_setup.sh installs the following libraries:

pytorch_benchmark_setup.sh downloads the following models from Hugging Face:

• meta-llama/Llama-3.1-70B-Instruct

• black-forest-labs/FLUX.1-dev

Along with the following datasets:

• WikiText

• UltraChat 200k

• bghira/pseudo-camera-10k

Pretraining

To start the pretraining benchmark, use the following command with the appropriate options. See the following list of options and their descriptions.

./pytorch_benchmark_report.sh -t $training_mode -m $model_repo -p $datatype -s $sequence_length

Fine-tuning

To start the fine-tuning benchmark, use the following command. It will run the benchmarking example of Llama 3.1 70B with the WikiText dataset using the AMD fork of torchtune.

./pytorch_benchmark_report.sh -t {finetune_fw, finetune_lora} -p BF16 -m Llama-3.1-70B

Use the following command to run the benchmarking example of Llama 2 70B with the UltraChat 200k dataset using Hugging Face PEFT.

./pytorch_benchmark_report.sh -t HF_finetune_lora -p BF16 -m Llama-2-70B

Benchmarking examples

Here are some example commands to get started pretraining and fine-tuning with various model configurations.

• Example 1: Llama 3.1 70B with BF16 precision with torchtitan.

  ./pytorch_benchmark_report.sh -t pretrain -p BF16 -m Llama-3.1-70B -s 8192
  

• Example 2: Llama 3.1 8B with FP8 precision using Transformer Engine (TE) and Hugging Face Accelerator.

  ./pytorch_benchmark_report.sh -t pretrain -p FP8 -m Llama-3.1-70B -s 8192
  

• Example 3: FLUX.1-dev with BF16 precision with FluxBenchmark.

  ./pytorch_benchmark_report.sh -t pretrain -p BF16 -m Flux
  

• Example 4: Torchtune full weight fine-tuning with Llama 3.1 70B

  ./pytorch_benchmark_report.sh -t finetune_fw -p BF16 -m Llama-3.1-70B
  

• Example 5: Torchtune LoRA fine-tuning with Llama 3.1 70B

  ./pytorch_benchmark_report.sh -t finetune_lora -p BF16 -m Llama-3.1-70B
  

• Example 6: Torchtune full weight fine-tuning with Llama-3.3-70B

  ./pytorch_benchmark_report.sh -t finetune_fw -p BF16 -m Llama-3.3-70B
  

• Example 7: Torchtune LoRA fine-tuning with Llama-3.3-70B

  ./pytorch_benchmark_report.sh -t finetune_lora -p BF16 -m Llama-3.3-70B
  

• Example 8: Torchtune QLoRA fine-tuning with Llama-3.3-70B

  ./pytorch_benchmark_report.sh -t finetune_qlora -p BF16 -m Llama-3.3-70B
  

• Example 9: Hugging Face PEFT LoRA fine-tuning with Llama 2 70B

  ./pytorch_benchmark_report.sh -t HF_finetune_lora -p BF16 -m Llama-2-70B