Using CMake#
2024-07-11
16 min read time
Most components in ROCm support CMake. Projects depending on header-only or library components typically require CMake 3.5 or higher whereas those wanting to make use of the CMake HIP language support will require CMake 3.21 or higher.
Finding dependencies#
Note
For a complete reference on how to deal with dependencies in CMake, refer to the CMake docs on find_package and the Using Dependencies Guide to get an overview of CMake related facilities.
In short, CMake supports finding dependencies in two ways:
In Module mode, it consults a file
Find<PackageName>.cmake
which tries to find the component in typical install locations and layouts. CMake ships a few dozen such scripts, but users and projects may ship them as well.In Config mode, it locates a file named
<packagename>-config.cmake
or<PackageName>Config.cmake
which describes the installed component in all regards needed to consume it.
ROCm predominantly relies on Config mode, one notable exception being the Module
driving the compilation of HIP programs on NVIDIA runtimes. As such, when
dependencies are not found in standard system locations, one either has to
instruct CMake to search for package config files in additional folders using
the CMAKE_PREFIX_PATH
variable (a semi-colon separated list of file system
paths), or using <PackageName>_ROOT
variable on a project-specific basis.
There are nearly a dozen ways to set these variables. One may be more convenient
over the other depending on your workflow. Conceptually the simplest is adding
it to your CMake configuration command on the command line via
-D CMAKE_PREFIX_PATH=....
. AMD packaged ROCm installs can typically be
added to the config file search paths such as:
Windows:
-D CMAKE_PREFIX_PATH=${env:HIP_PATH}
Linux:
-D CMAKE_PREFIX_PATH=/opt/rocm
ROCm provides the respective config-file packages, and this enables
find_package
to be used directly. ROCm does not require any Find module as
the config-file packages are shipped with the upstream projects, such as
rocPRIM and other ROCm libraries.
For a complete guide on where and how ROCm may be installed on a system, refer to the installation guides for Linux and Windows.
Using HIP in CMake#
ROCm components providing a C/C++ interface support consumption via any C/C++ toolchain that CMake knows how to drive. ROCm also supports the CMake HIP language features, allowing users to program using the HIP single-source programming model. When a program (or translation-unit) uses the HIP API without compiling any GPU device code, HIP can be treated in CMake as a simple C/C++ library.
Using the HIP single-source programming model#
Source code written in the HIP dialect of C++ typically uses the .hip extension. When the HIP CMake language is enabled, it will automatically associate such source files with the HIP toolchain being used.
cmake_minimum_required(VERSION 3.21) # HIP language support requires 3.21
cmake_policy(VERSION 3.21.3...3.27)
project(MyProj LANGUAGES HIP)
add_executable(MyApp Main.hip)
Should you have existing CUDA code that is from the source compatible subset of HIP, you can tell CMake that despite their .cu extension, they’re HIP sources. Do note that this mostly facilitates compiling kernel code-only source files, as host-side CUDA API won’t compile in this fashion.
add_library(MyLib MyLib.cu)
set_source_files_properties(MyLib.cu PROPERTIES LANGUAGE HIP)
CMake itself only hosts part of the HIP language support, such as defining
HIP-specific properties, etc. while the other half ships with the HIP
implementation, such as ROCm. CMake will search for a file
hip-lang-config.cmake describing how the the properties defined by CMake
translate to toolchain invocations. If one installs ROCm using non-standard
methods or layouts and CMake can’t locate this file or detect parts of the SDK,
there’s a catch-all, last resort variable consulted locating this file,
-D CMAKE_HIP_COMPILER_ROCM_ROOT:PATH=
which should be set the root of the
ROCm installation.
Note
Imported targets defined by hip-lang-config.cmake are for internal use only.
If the user doesn’t provide a semi-colon delimited list of device architectures
via CMAKE_HIP_ARCHITECTURES
, CMake will select some sensible default. It is
advised though that if a user knows what devices they wish to target, then set
this variable explicitly.
Consuming ROCm C/C++ libraries#
Libraries such as rocBLAS, rocFFT, MIOpen, etc. behave as C/C++ libraries.
Illustrated in the example below is a C++ application using MIOpen from CMake.
It calls find_package(miopen)
, which provides the MIOpen
imported
target. This can be linked with target_link_libraries
cmake_minimum_required(VERSION 3.5) # find_package(miopen) requires 3.5
cmake_policy(VERSION 3.5...3.27)
project(MyProj LANGUAGES CXX)
find_package(miopen)
add_library(MyLib ...)
target_link_libraries(MyLib PUBLIC MIOpen)
Note
Most libraries are designed as host-only API, so using a GPU device compiler is not necessary for downstream projects unless they use GPU device code.
Consuming the HIP API in C++ code#
Consuming the HIP API without compiling single-source GPU device code can be
done using any C++ compiler. The find_package(hip)
provides the
hip::host
imported target to use HIP in this scenario.
cmake_minimum_required(VERSION 3.5) # find_package(hip) requires 3.5
cmake_policy(VERSION 3.5...3.27)
project(MyProj LANGUAGES CXX)
find_package(hip REQUIRED)
add_executable(MyApp ...)
target_link_libraries(MyApp PRIVATE hip::host)
When mixing such CXX
sources with HIP
sources holding device-code, link
only to hip::host. If HIP sources don’t have .hip as their extension, use
set_source_files_properties(<hip_sources>… PROPERTIES LANGUAGE HIP) on them.
Linking to hip::host will set all the necessary flags for the CXX
sources
while HIP
sources inherit all flags from the built-in language support.
Having HIP sources in a target will turn the LINKER_LANGUAGE
into HIP
.
Compiling device code in C++ language mode#
Attention
The workflow detailed here is considered legacy and is shown for understanding’s sake. It pre-dates the existence of HIP language support in CMake. If source code has HIP device code in it, it is a HIP source file and should be compiled as such. Only resort to the method below if your HIP-enabled CMake code path can’t mandate CMake version 3.21.
If code uses the HIP API and compiles GPU device code, it requires using a
device compiler. The compiler for CMake can be set using either the
CMAKE_C_COMPILER
and CMAKE_CXX_COMPILER
variable or using the CC
and CXX
environment variables. This can be set when configuring CMake or
put into a CMake toolchain file. The device compiler must be set to a
compiler that supports AMD GPU targets, which is usually Clang.
The find_package(hip)
provides the hip::device
imported target to add
all the flags necessary for device compilation.
cmake_minimum_required(VERSION 3.8) # cxx_std_11 requires 3.8
cmake_policy(VERSION 3.8...3.27)
project(MyProj LANGUAGES CXX)
find_package(hip REQUIRED)
add_library(MyLib ...)
target_link_libraries(MyLib PRIVATE hip::device)
target_compile_features(MyLib PRIVATE cxx_std_11)
Note
Compiling for the GPU device requires at least C++11.
This project can then be configured with the following CMake commands:
Windows:
cmake -D CMAKE_CXX_COMPILER:PATH=${env:HIP_PATH}\bin\clang++.exe
Linux:
cmake -D CMAKE_CXX_COMPILER:PATH=/opt/rocm/bin/amdclang++
Which use the device compiler provided from the binary packages of ROCm HIP SDK and repo.radeon.com respectively.
When using the CXX
language support to compile HIP device code, selecting the
target GPU architectures is done via setting the GPU_TARGETS
variable.
CMAKE_HIP_ARCHITECTURES
only exists when the HIP language is enabled. By
default, this is set to some subset of the currently supported architectures of
AMD ROCm. It can be set to the CMake option -D GPU_TARGETS="gfx1032;gfx1035"
.
ROCm CMake packages#
Component |
Package |
Targets |
---|---|---|
HIP |
hip |
|
rocPRIM |
rocprim |
|
rocThrust |
rocthrust |
|
hipCUB |
hipcub |
|
rocRAND |
rocrand |
|
rocBLAS |
rocblas |
|
rocSOLVER |
rocsolver |
|
hipBLAS |
hipblas |
|
rocFFT |
rocfft |
|
hipFFT |
hipfft |
|
rocSPARSE |
rocsparse |
|
hipSPARSE |
hipsparse |
|
rocALUTION |
rocalution |
|
RCCL |
rccl |
|
MIOpen |
miopen |
|
MIGraphX |
migraphx |
|
Using CMake presets#
CMake command lines depending on how specific users like to be when compiling
code can grow to unwieldy lengths. This is the primary reason why projects tend
to bake script snippets into their build definitions controlling compiler
warning levels, changing CMake defaults (CMAKE_BUILD_TYPE
or
BUILD_SHARED_LIBS
just to name a few) and all sorts anti-patterns, all in
the name of convenience.
Load on the command-line interface (CLI) starts immediately by selecting a
toolchain, the set of utilities used to compile programs. To ease some of the
toolchain related pains, CMake does consult the CC
and CXX
environmental
variables when setting a default CMAKE_C[XX]_COMPILER
respectively, but that
is just the tip of the iceberg. There’s a fair number of variables related to
just the toolchain itself (typically supplied using
toolchain files
), and then we still haven’t talked about user preference or project-specific
options.
IDEs supporting CMake (Visual Studio, Visual Studio Code, CLion, etc.) all came up with their own way to register command-line fragments of different purpose in a setup-and-forget fashion for quick assembly using graphical front-ends. This is all nice, but configurations aren’t portable, nor can they be reused in Continuous Integration (CI) pipelines. CMake has condensed existing practice into a portable JSON format that works in all IDEs and can be invoked from any command line. This is CMake Presets.
There are two types of preset files: one supplied by the project, called
CMakePresets.json
which is meant to be committed to version control,
typically used to drive CI; and one meant for the user to provide, called
CMakeUserPresets.json
, typically used to house user preference and adapting
the build to the user’s environment. These JSON files are allowed to include
other JSON files and the user presets always implicitly includes the non-user
variant.
Using HIP with presets#
Following is an example CMakeUserPresets.json
file which actually compiles
the amd/rocm-examples suite of sample
applications on a typical ROCm installation:
{
"version": 3,
"cmakeMinimumRequired": {
"major": 3,
"minor": 21,
"patch": 0
},
"configurePresets": [
{
"name": "layout",
"hidden": true,
"binaryDir": "${sourceDir}/build/${presetName}",
"installDir": "${sourceDir}/install/${presetName}"
},
{
"name": "generator-ninja-multi-config",
"hidden": true,
"generator": "Ninja Multi-Config"
},
{
"name": "toolchain-makefiles-c/c++-amdclang",
"hidden": true,
"cacheVariables": {
"CMAKE_C_COMPILER": "/opt/rocm/bin/amdclang",
"CMAKE_CXX_COMPILER": "/opt/rocm/bin/amdclang++",
"CMAKE_HIP_COMPILER": "/opt/rocm/bin/amdclang++"
}
},
{
"name": "clang-strict-iso-high-warn",
"hidden": true,
"cacheVariables": {
"CMAKE_C_FLAGS": "-Wall -Wextra -pedantic",
"CMAKE_CXX_FLAGS": "-Wall -Wextra -pedantic",
"CMAKE_HIP_FLAGS": "-Wall -Wextra -pedantic"
}
},
{
"name": "ninja-mc-rocm",
"displayName": "Ninja Multi-Config ROCm",
"inherits": [
"layout",
"generator-ninja-multi-config",
"toolchain-makefiles-c/c++-amdclang",
"clang-strict-iso-high-warn"
]
}
],
"buildPresets": [
{
"name": "ninja-mc-rocm-debug",
"displayName": "Debug",
"configuration": "Debug",
"configurePreset": "ninja-mc-rocm"
},
{
"name": "ninja-mc-rocm-release",
"displayName": "Release",
"configuration": "Release",
"configurePreset": "ninja-mc-rocm"
},
{
"name": "ninja-mc-rocm-debug-verbose",
"displayName": "Debug (verbose)",
"configuration": "Debug",
"configurePreset": "ninja-mc-rocm",
"verbose": true
},
{
"name": "ninja-mc-rocm-release-verbose",
"displayName": "Release (verbose)",
"configuration": "Release",
"configurePreset": "ninja-mc-rocm",
"verbose": true
}
],
"testPresets": [
{
"name": "ninja-mc-rocm-debug",
"displayName": "Debug",
"configuration": "Debug",
"configurePreset": "ninja-mc-rocm",
"execution": {
"jobs": 0
}
},
{
"name": "ninja-mc-rocm-release",
"displayName": "Release",
"configuration": "Release",
"configurePreset": "ninja-mc-rocm",
"execution": {
"jobs": 0
}
}
]
}
Note
Getting presets to work reliably on Windows requires some CMake improvements and/or support from compiler vendors. (Refer to Add support to the Visual Studio generators and Sourcing environment scripts .)