<head>
  <meta charset="UTF-8">
  <meta name="description" content="RDNA2 workstation tuning guide">
  <meta name="keywords" content="RDNA2, workstation, BIOS settings, installation, AMD,
  ROCm">
</head>

# AMD RDNA2 system optimization

## System settings

This chapter reviews system settings that are required to configure the system
for ROCm virtualization on RDNA2-based AMD Radeon™ PRO GPUs. Installing ROCm on
Bare Metal follows the routine ROCm
{doc}`installation procedure<rocm-install-on-linux:install/native-install/index>`.

To enable ROCm virtualization on V620, one has to setup Single Root I/O
Virtualization (SR-IOV) in the BIOS via setting found in the following
({ref}`bios-settings`). A tested configuration can be followed in
({ref}`os-settings`).

:::{attention}
SR-IOV is supported on V620 and unsupported on W6800.
:::

(bios-settings)=

### System BIOS settings

```{list-table} Settings for the system BIOS in an ASrock platform.
:header-rows: 1
:name: v620-bios

*
  - Advanced / North Bridge Configuration
  - IOMMU
  - Enabled
  - Input-output Memory Management Unit
*
  - Advanced / North Bridge Configuration
  - ACS Enable
  - Enabled
  - Access Control Service
*
  - Advanced / PCIe/PCI/PnP Configuration
  - SR-IOV Support
  - Enabled
  - Single Root I/O Virtualization
*
  - Advanced / ACPI settings
  - PCI AER Support
  - Enabled
  - Advanced Error Reporting
```

To set up the host, update SBIOS to version 1.2a.

(os-settings)=

### Operating system settings

```{list-table} System Configuration Prerequisites
:header-rows: 1
:name: v620-prereq

*
  - Server
  - [SMC 4124](https://www.supermicro.com/en/Aplus/system/4U/4124/AS-4124GS-TNR.cfm) [AS -4124GS-TNR]
*
  - Host OS
  - Ubuntu 20.04.3 LTS
*
  - Host Kernel
  - 5.4.0-97-generic
*
  - CPU
  - AMD EPYC 7552 48-Core Processor
*
  - GPU
  - RDNA2 V620 (D603GLXE)
*
  - SBIOS
  - Version SMC_r_1.2a
*
  - VBIOS
  - 113-D603GLXE-077
*
  - Guest OS 1
  - Ubuntu 20.04.5 LTS
*
  - Guest OS 2
  - RHEL 9.0
*
  - GIM Driver
  - gim-dkms_1.0.0.1234577_all
*
  - VM CPU Cores
  - 32
*
  - VM RAM
  - 64 GB
```

Install the following Kernel-based Virtual Machine (KVM) Hypervisor packages:

```shell
sudo apt-get -y install qemu-kvm qemu-utils  bridge-utils virt-manager  gir1.2-spiceclientgtk*  gir1.2-spice-client-gtk* libvirt-daemon-system dnsmasq-base
sudo virsh net-start default /*to enable Virtual network by default
```

Enable input-output memory management unit (IOMMU) in GRUB settings by adding the following line to `/etc/default/grub`:

```none
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=on" for AMD CPU
```

Update grub and reboot

```shell
sudo update=grub
sudo reboot
```

Install the GPU-IOV Module (GIM, where IOV is I/O Virtualization) driver and
follow the steps below.z

```shell
sudo dpkg -i <gim_driver>
sudo reboot
# Load Host Driver to Create 1VF
sudo modprobe gim vf_num=1
# Note: If GIM driver loaded successfully, we could see "gim info:(gim_init:213) *****Running GIM*****" in dmesg
lspci -d 1002:
```

Which should output something like:

```none
01:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Device 1478
02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Device 1479
03:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Device 73a1
03:02.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Device 73ae → VF
```

### Guest OS installation

First, assign GPU virtual function (VF) to VM using the following steps.

1. Shut down the VM.

2. Run `virt-manager`

3. In the **Virtual Machine Manager** GUI, select the **VM** and click **Open**.

   ![Virtual Machine Manager](../../data/how-to/tuning-guides/tuning014.png "Virtual Machine Manager")

4. In the VM GUI, go to **Show Virtual Hardware Details > Add Hardware** to
  configure hardware.

   ![Show virtual hardware details](../../data/how-to/tuning-guides/tuning015.png "Virtual Machine Manager: show virtual hardware details")

5. Go to **Add Hardware > PCI Host Device > VF** and click **Finish**.

    ![VF Selection](../../data/how-to/tuning-guides/tuning016.png "VF Selection")

Then start the VM.

Finally install ROCm on the virtual machine (VM). For detailed instructions,
refer to the {doc}`Linux install guide<rocm-install-on-linux:install/native-install/index>`.
