RDNA2 Workstation Tuning Guide

RDNA2 Workstation Tuning Guide#

Applies to Linux

2023-06-01

6 min read time

System Settings#

This chapter reviews system settings that are required to configure the system for ROCm virtualization on RDNA2-based AMD Radeon™ PRO GPUs. Installing ROCm on Bare Metal follows the routine ROCm installation procedure.

To enable ROCm virtualization on V620, one has to setup Single Root I/O Virtualization (SR-IOV) in the BIOS via setting found in the following (System BIOS Settings). A tested configuration can be followed in (Operating System Settings).

Attention

SR-IOV is supported on V620 and unsupported on W6800.

System BIOS Settings#

Table 12 Settings for the system BIOS in an ASrock platform.#

Advanced / North Bridge Configuration

IOMMU

Enabled

Input-output Memory Management Unit

Advanced / North Bridge Configuration

ACS Enable

Enabled

Access Control Service

Advanced / PCIe/PCI/PnP Configuration

SR-IOV Support

Enabled

Single Root I/O Virtualization

Advanced / ACPI settings

PCI AER Support

Enabled

Advanced Error Reporting

To set up the host, update SBIOS to version 1.2a.

Operating System Settings#

Table 13 System Configuration Prerequisites#

Server

SMC 4124 [AS -4124GS-TNR]

Host OS

Ubuntu 20.04.3 LTS

Host Kernel

5.4.0-97-generic

CPU

AMD EPYC 7552 48-Core Processor

GPU

RDNA2 V620 (D603GLXE)

SBIOS

Version SMC_r_1.2a

VBIOS

113-D603GLXE-077

Guest OS 1

Ubuntu 20.04.5 LTS

Guest OS 2

RHEL 9.0

GIM Driver

gim-dkms_1.0.0.1234577_all

VM CPU Cores

32

VM RAM

64 GB

Install the following Kernel-based Virtual Machine (KVM) Hypervisor packages:

sudo apt-get -y install qemu-kvm qemu-utils  bridge-utils virt-manager  gir1.2-spiceclientgtk*  gir1.2-spice-client-gtk* libvirt-daemon-system dnsmasq-base
sudo virsh net-start default /*to enable Virtual network by default

Enable IOMMU in GRUB settings by adding the following line to /etc/default/grub:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=on" for AMD CPU

Update grub and reboot

sudo update=grub
sudo reboot

Install the GPU-IOV Module (GIM, where IOV is I/O Virtualization) driver and follow the steps below. To obtain the GIM driver, write to us here:

sudo dpkg -i <gim_driver>
sudo reboot
# Load Host Driver to Create 1VF
sudo modprobe gim vf_num=1
# Note: If GIM driver loaded successfully, we could see "gim info:(gim_init:213) *****Running GIM*****" in dmesg
lspci -d 1002:

Which should output something like:

01:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Device 1478
02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Device 1479
03:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Device 73a1
03:02.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Device 73ae → VF

Guest OS installation#

First, assign GPU virtual function (VF) to VM using the following steps.

  1. Shut down the VM.

  2. Run virt-manager

  3. In the Virtual Machine Manager GUI, select the VM and click Open.

    Virtual Machine Manager

    Fig. 24 Virtual Machine Manager#

  4. In the VM GUI, go to Show Virtual Hardware Details > Add Hardware to configure hardware.

    Show Virtual Hardware Details

    Fig. 25 Virtual Machine Manager#

  5. Go to Add Hardware > PCI Host Device > VF and click Finish.

    VF Selection

    Fig. 26 VF Selection#

Then start the VM.

Finally install ROCm on the virtual machine (VM). For detailed instructions, refer to the ROCm Installation Guide. For any issue encountered during installation, write to us here.

Contents