Basic usage#

The following section outlines basic ROCm Compute Profiler workflows, modes, options, and operations.

Command line profiler#

Launch and profile the target application using the command line profiler.

The command line profiler launches the target application and collects hardware performance counter data for the specified kernels, dispatches, and hardware components. By default, ROCm Compute Profiler collects all available counters for all kernels and dispatches launched by the executable. Profiling is performed using ROCprofiler-SDK; see Configuring the environment for profiling for details on backend selection and the native counter collection tool.

To collect the default set of data for all kernels in the target application, launch, for example:

$ rocprof-compute profile -n vcopy_data -- ./vcopy -n 1048576 -b 256

This runs the app, launches each kernel, and generates profiling results. By default, results are written to a subdirectory with your accelerator’s name; for example, ./workloads/vcopy_data/MI200/, where name is configurable via the -n argument. When an MPI rank is detected, the default output directory appends the rank (./workloads/vcopy_data/<rank>/) instead of the gpu model. Use -d, --output-directory to override the output location.

Note

To collect all requested profile information, ROCm Compute Profiler might replay kernels multiple times.

Customize data collection#

Options are available to specify for which kernels and metrics data should be collected. Note that you can apply filtering in either the profiling or analysis stage. Filtering at profiling collection often speeds up your aggregate profiling run time.

Common filters to customize data collection include:

-k, --kernel

Enables filtering kernels by name.

--kernel-iteration-range

In profile mode, selects which iterations of each kernel to profile.

-d, --dispatch

In analyze mode, selects which already-profiled dispatches to report on.

-b, --block

Enables collection metrics for only the specified analysis report blocks.

See Filtering for an in-depth walkthrough.

List blocks and metrics#

To view available metrics by hardware block, use the --list-metrics option. It lists the blocks of the current GPU and their aliases. Add block or metric IDs or aliases to list their metrics with units and descriptions, or all to list every metric. Each ID or alias adds to the list: rocprof-compute --list-metrics sol 12.1 lists all of block 2 (sol) and table 12.1.

$ rocprof-compute --list-metrics
$ rocprof-compute --list-metrics sol 12 11.1.0
$ rocprof-compute --list-metrics all

To list the blocks and metrics of another GPU, for example on a machine without a GPU, add the debug option --debug-list-metrics-gfx-id <arch> to --list-metrics.

The first command lists the blocks of gfx90a. The second lists the metrics of block 2 (sol) on gfx90a:

$ rocprof-compute --debug-list-metrics-gfx-id gfx90a --list-metrics
$ rocprof-compute --debug-list-metrics-gfx-id gfx90a --list-metrics sol

To list the blocks and metrics of a profiled workload, use analyze mode:

$ rocprof-compute analyze -p <workload_path> --list-metrics

The block table shows each block’s ID and alias. Each metric table shows the metric’s ID, unit, and description. For example, on an MI300 system:

$ rocprof-compute --list-metrics

Platform: gfx942 (MI300)

+----------+----------------+------------------------------------------------------+
| Block_ID | Alias          | Block                                                |
+----------+----------------+------------------------------------------------------+
| 0        | topstats       | Top Stats                                            |
+----------+----------------+------------------------------------------------------+
| 1        | sysinfo        | System Info                                          |
+----------+----------------+------------------------------------------------------+
| 2        | sol            | System Speed-of-Light                                |
+----------+----------------+------------------------------------------------------+
...

$ rocprof-compute --list-metrics spi

Platform: gfx942 (MI300)

6. Workgroup Manager (SPI) (alias: spi)
6.1 Workgroup manager utilizations
+-----------+-------------------------------+-------------+------------------------------------------+
| Metric_ID | Metric                        | Unit        | Description                              |
+-----------+-------------------------------+-------------+------------------------------------------+
| 6.1.0     | Accelerator Utilization       | Percent     | The percent of cycles in the kernel      |
|           |                               |             | where the accelerator was actively doing |
|           |                               |             | any work.                                |
+-----------+-------------------------------+-------------+------------------------------------------+
| 6.1.1     | Scheduler-Pipe Utilization    | Percent     | The percent of total scheduler-pipe      |
|           |                               |             | cycles in the kernel where the           |
|           |                               |             | scheduler-pipes were actively doing any  |
|           |                               |             | work.                                    |
+-----------+-------------------------------+-------------+------------------------------------------+
...

Note

--list-blocks and --list-available-metrics are deprecated and will be removed in a future release. Use --list-metrics instead.

Analyze in the command line#

After generating a local output folder (for example, ./workloads/vcopy_data/MI200), use the command line tool to quickly interface with profiling results. View different metrics derived from your profiled results and get immediate access all metrics organized by hardware blocks.

If you don’t apply kernel, dispatch, or analysis report block filters at this stage, analysis is reflective of the entirety of the profiling data.

To interact with profiling results from a different session, provide the workload path.

-p, --path

Enables you to analyze existing profiling data in the ROCm Compute Profiler CLI.

See CLI analysis for more detailed information.

Modes#

Modes change the fundamental behavior of the ROCm Compute Profiler command line tool. Depending on which mode you choose, different command line options become available.

Profile mode#

profile

Launches the target application on the local system using ROCprofiler-SDK. Depending on the profiling options chosen, selected kernels, dispatches, and or hardware components used by the application are profiled. It stores results locally in an output folder: ./workloads/\<name> (or rank-specific subdirectories when using MPI).

$ rocprof-compute profile --help

See Profile mode to learn about this mode in depth and to get started profiling with ROCm Compute Profiler.

Analyze mode#

analyze

Loads profiling data from the --path (-p) directory into the ROCm Compute Profiler CLI analyzer where you have immediate access to profiling results and generated metrics. It generates metrics from the entirety of your profiled application or a subset identified through the ROCm Compute Profiler CLI analysis filters.

$ rocprof-compute analyze --help

See Analyze mode to learn about this mode in depth and to get started with analysis using ROCm Compute Profiler.

Global options#

The ROCm Compute Profiler command line tool has a set of global utility options that are available across all modes.

-v, --version

Prints the ROCm Compute Profiler version and exits.

-V, --verbose

Increases output verbosity. Use multiple times for higher levels of verbosity.

-q, --quiet

Reduces output verbosity and runs quietly.

-s, --specs

Prints system specs and exits.

Note

ROCm Compute Profiler also recognizes the project variable, ROCPROFCOMPUTE_COLOR should you choose to disable colorful output. To disable default colorful behavior, set this variable to 0.

Basic operations#

The following table lists ROCm Compute Profiler’s basic operations, their modes, and required arguments.

Operation description

Mode

Required arguments

Profile a workload

profile

--name or --output-directory, -- <profile_cmd>

Standalone roofline analysis

profile

--name or --output-directory, --roofline, -- <profile_cmd>

Interact with profiling results from CLI

analyze

--path