---
myst:
  html_meta:
    "description": "Learn about system settings and performance tuning for RDNA2-based GPUs."
    "keywords": "RDNA2, workstation, desktop, BIOS, installation, Radeon, pro, v620, w6000"
---

# AMD RDNA2 system optimization

The topic covers specific software requirements and processes needed to use
these GPUs for Single Root I/O Virtualization (SR-IOV) and machine learning
tasks.

The main purpose of this document is to help users utilize the RDNA™ 2 GPUs to
their full potential.

## Workstation workloads

Workstation workloads, much like those for HPC, have a unique set of
requirements: a blend of both graphics and compute, certification, stability,
and more.

For more information about RDNA2, see the [RDNA 2 Instruction Set Architecture: Reference Guide](https://www.amd.com/system/files/TechDocs/rdna2-shader-instruction-set-architecture.pdf).

## System settings

This section discusses system settings used to configure the system
for ROCm virtualization on RDNA2-based AMD Radeon™ PRO GPUs. Installing ROCm on
Bare Metal follows the routine {doc}`ROCm installation procedure <../../install/rocm>`.

To enable ROCm virtualization on V620, set up Single Root I/O
Virtualization (SR-IOV) in the BIOS via settings found in
{ref}`bios-settings`. A tested configuration can be followed in
{ref}`os-settings`.

:::{attention}
SR-IOV is supported on V620 and unsupported on W6800.
:::

(bios-settings)=

### System BIOS settings

```{list-table}
:header-rows: 1
:name: v620-bios

*
  - Advanced / North Bridge Configuration
  - IOMMU
  - Enabled
  - Input-output Memory Management Unit
*
  - Advanced / North Bridge Configuration
  - ACS Enable
  - Enabled
  - Access Control Service
*
  - Advanced / PCIe/PCI/PnP Configuration
  - SR-IOV Support
  - Enabled
  - Single Root I/O Virtualization
*
  - Advanced / ACPI settings
  - PCI AER Support
  - Enabled
  - Advanced Error Reporting
```

To set up the host, update SBIOS to version 1.2a.

(os-settings)=

### Operating system settings

```{list-table}
:header-rows: 1
:name: v620-prereq

*
  - Server
  - [SMC 4124](https://www.supermicro.com/en/Aplus/system/4U/4124/AS-4124GS-TNR.cfm) [AS -4124GS-TNR]
*
  - Host OS
  - Ubuntu 20.04.3 LTS
*
  - Host Kernel
  - 5.4.0-97-generic
*
  - CPU
  - AMD EPYC 7552 48-Core Processor
*
  - GPU
  - RDNA2 V620 (D603GLXE)
*
  - SBIOS
  - Version SMC_r_1.2a
*
  - VBIOS
  - 113-D603GLXE-077
*
  - Guest OS 1
  - Ubuntu 20.04.5 LTS
*
  - Guest OS 2
  - RHEL 9.0
*
  - GIM Driver
  - gim-dkms_1.0.0.1234577_all
*
  - VM CPU Cores
  - 32
*
  - VM RAM
  - 64 GB
```

Install the following Kernel-based Virtual Machine (KVM) Hypervisor packages:

```shell
sudo apt-get -y install qemu-kvm qemu-utils  bridge-utils virt-manager  gir1.2-spiceclientgtk*  gir1.2-spice-client-gtk* libvirt-daemon-system dnsmasq-base
sudo virsh net-start default /*to enable Virtual network by default
```

Enable input-output memory management unit (IOMMU) in GRUB settings by adding the following line to `/etc/default/grub`:

```none
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" for AMD CPU
```

Update grub and reboot

```shell
sudo update=grub
sudo reboot
```

Install the GPU-IOV Module (GIM, where IOV is I/O Virtualization) driver and
follow the steps below.z

```shell
sudo dpkg -i <gim_driver>
sudo reboot
# Load Host Driver to Create 1VF
sudo modprobe gim vf_num=1
# Note: If GIM driver loaded successfully, we could see "gim info:(gim_init:213) *****Running GIM*****" in dmesg
lspci -d 1002:
```

Which should output something like:

```none
01:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Device 1478
02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Device 1479
03:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Device 73a1
03:02.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Device 73ae → VF
```

### Guest OS installation

First, assign GPU virtual function (VF) to VM using the following steps.

1. Stop the VM.

2. Run `virt-manager`

3. In the **Virtual Machine Manager** GUI, select the **VM** and click **Open**.

   ![Virtual Machine Manager](./images/tuning014.png "Virtual Machine Manager")

4. In the VM GUI, go to **Show Virtual Hardware Details > Add Hardware** to
  configure hardware.

   ![Show virtual hardware details](./images/tuning015.png "Virtual Machine Manager: show virtual hardware details")

5. Go to **Add Hardware > PCI Host Device > VF** and click **Finish**.

    ![VF Selection](./images/tuning016.png "VF Selection")

Then start the VM.

Lastly, install ROCm on the virtual machine (VM). For detailed instructions,
refer to the {doc}`Linux install guide </install/rocm>`.