GPU Isolation Techniques#
6 min read time
Restricting the access of applications to a subset of GPUs, aka isolating GPUs allows users to hide GPU resources from programs. The programs by default will only use the “exposed” GPUs ignoring other (hidden) GPUs in the system.
There are multiple ways to achieve isolation of GPUs in the ROCm software stack, differing in which applications they apply to and the security they provide. This page serves as an overview of the techniques.
The runtimes in the ROCm software stack read these environment variables to select the exposed or default device to present to applications using them.
Environment variables shouldn’t be used for isolating untrusted applications, as an application can reset them before initializing the runtime.
A list of device indices or UUIDs that will be exposed to applications.
Runtime : ROCm Platform Runtime. Applies to all applications using the user mode ROCm software stack.
Devices indices exposed to OpenCL and HIP applications.
: ROCm Common Language Runtime (
ROCclr). Applies to applications and runtimes
ROCclr abstraction layer including HIP and OpenCL applications.
Device indices exposed to HIP applications.
Runtime : HIP Runtime. Applies only to applications using HIP on the AMD platform.
Provided for CUDA compatibility, has the same effect as
on the AMD platform.
Runtime : HIP or CUDA Runtime. Applies to HIP applications on the AMD or NVIDIA platform and CUDA applications.
Default device used for OpenMP target offloading.
Runtime : OpenMP Runtime. Applies only to applications using OpenMP offloading.
Docker uses Linux kernel namespaces to provide isolated environments for applications. This isolation applies to most devices by default, including GPUs. To access them in containers explicit access must be granted, please see Accessing GPUs in containers for details. Specifically refer to Restricting a container to a subset of the GPUs on exposing just a subset of all GPUs.
Docker isolation is more secure than environment variables, and applies
to all programs that use the
amdgpu kernel module interfaces.
Even programs that don’t use the ROCm runtime, like graphics applications
using OpenGL or Vulkan, can only access the GPUs exposed to the container.
GPU Passthrough to Virtual Machines#
Virtual machines achieve the highest level of isolation, because even the kernel of the virtual machine is isolated from the host. Devices physically installed in the host system can be passed to the virtual machine using PCIe passthrough. This allows for using the GPU with a different operating systems like a Windows guest from a Linux host.
Setting up PCIe passthrough is specific to the hypervisor used. ROCm officially supports VMware ESXi for select GPUs.