Run AGFHC, TransferBench, and RVS health burn-in tests in Cluster Validation Suite (CVS)

Run AGFHC, TransferBench, and RVS health burn-in tests in Cluster Validation Suite (CVS)#

2026-10-26

4 min read time

Applies to Linux

Health burn-in tests run AMD GPU Field Health Check (AGFHC), TransferBench, and ROCm Validation Suite (RVS) on every node in the cluster. Use these tests to validate GPU hardware, memory bandwidth, and compute throughput before production workloads.

Set up config#

  1. Copy the health configuration file:

    cvs config copy health/health_config.json --output ~/cvs_workspace/health/health_config.json

  2. Edit paths and any remaining <changeme> placeholders:

    Section

    Fields to set

    Notes

    agfhc

    path, package_tar_ball, install_dir, env, log_dir

    A path other than /opt/amd/agfhc installs with ./install --rocm-tar -o and no sudo; its parent and log_dir must be writable on the nodes. Update env when path changes.

    transferbench

    git_install_path, rocm_path

    Leave rocm_path as <changeme> to auto-detect. For test_transfer_bench_a2a, set results.gpu_to_gpu_a2a_per_link_min (default 32.9). For test_transfer_bench_a2asweep, set a2asweep_timeout (default 1500) or omit it to derive the timeout from sweep size. See Health test configuration file reference for Cluster Validation Suite (CVS).

    rvs

    git_install_path, path, rocm_path

For the complete field reference including install paths and threshold values, see Health test configuration file reference for Cluster Validation Suite (CVS).

Run tests#

Use these tests to validate single-node diagnostics for GPU cluster health. For the performance validation, use the reference bandwidth or latency numbers provided as part of the input config_file for the relevant test.

Use these scripts to run each health test. These CVS test scripts have two parts: installing the functionality and running the tests.

Note

These health suites honor orchestrator in the cluster file. With container, the workload runs inside the per-host container (docker exec). See Run Cluster Validation Suite (CVS) test suites with a per-host Docker container backend.

AGFHC#

See the AGFHC (AMD GPU Field Health Check) docs for more information.

You can list all available AGFHC test cases using the CLI:

cvs list agfhc_cvs

Available tests in agfhc_cvs:
  - test_agfhc_hbm
  - test_agfhc_hbm1_lvl5
  - test_agfhc_hbm2_lvl5
  - test_agfhc_hbm3_lvl3
  - test_agfhc_dma_all_lvl1
  - test_agfhc_dma_lvl1
  - test_agfhc_gfx_lvl1
  - test_agfhc_pcie_lvl1
  - test_agfhc_pcie_lvl3
  - test_agfhc_xgmi_lvl1
  - test_agfhc_all_perf
  - test_agfhc_all_lvl5

Run the AGFHC test:

  1. Run the installation:

    cvs run install_agfhc \
  --cluster_file ~/cvs_workspace/cluster.json \
  --config_file ~/cvs_workspace/health/health_config.json \
  --capture=tee-sys -vvv -s

  2. Run the AGFHC test:

    cvs run agfhc_cvs \
  --cluster_file ~/cvs_workspace/cluster.json \
  --config_file ~/cvs_workspace/health/health_config.json \
  --capture=tee-sys -vvv -s

  3. Run the CSP qualification test:

    cvs run csp_qual_agfhc \
  --cluster_file ~/cvs_workspace/cluster.json \
  --config_file ~/cvs_workspace/health/health_config.json \
  --capture=tee-sys -vvv -s

TransferBench#

See the TransferBench docs for more information.

You can list all available TransferBench test cases using the CLI:

cvs list transferbench_cvs

Available tests in transferbench_cvs:
  - test_transfer_bench_a2a
  - test_transfer_bench_p2p
  - test_transfer_bench_healthcheck
  - test_transfer_bench_a2asweep
  - test_transfer_bench_scaling
  - test_transfer_bench_schmoo

Run the TransferBench test:

  1. Run the installation:

    cvs run install_transferbench \
  --cluster_file ~/cvs_workspace/cluster.json \
  --config_file ~/cvs_workspace/health/health_config.json \
  --capture=tee-sys -vvv -s

  2. Start the TransferBench test:

    cvs run transferbench_cvs \
  --cluster_file ~/cvs_workspace/cluster.json \
  --config_file ~/cvs_workspace/health/health_config.json \
  --capture=tee-sys -vvv -s

RVS#

See the ROCm Validation Suite (RVS) docs for more information.

You can list all available RVS test cases using the CLI:

cvs list rvs_cvs

Available tests in rvs_cvs:
  - test_rvs_level_config
  - test_rvs_gpu_enumeration
  - test_rvs_gpup_single
  - test_rvs_mem_test
  - test_rvs_gst_single
  - test_rvs_iet_single
  - test_rvs_pebb_single
  - test_rvs_pbqt_single
  - test_rvs_peqt_single
  - test_rvs_rcqt_single
  - test_rvs_tst_single
  - test_rvs_babel_stream

Run the RVS test:

  1. Run the installation:

    cvs run install_rvs \
  --cluster_file ~/cvs_workspace/cluster.json \
  --config_file ~/cvs_workspace/health/health_config.json \
  --capture=tee-sys -vvv -s

  2. Start the RVS test:

    cvs run rvs_cvs \
  --cluster_file ~/cvs_workspace/cluster.json \
  --config_file ~/cvs_workspace/health/health_config.json \
  --capture=tee-sys -vvv -s

Run on managed compute#

See Run Cluster Validation Suite (CVS) on managed compute for guidance on running these health suites on managed compute.

Contents