gdesatrigraha/vfio-fedora-34-identical-gpu.md

## vfio-fedora-34-identical-gpu.md

      
    Raw
  

              vfio-fedora-34-identical-gpu.md
            
          
    c2:d1:05:16:b0:b5
VFIO on Fedora 34

Reference: https://qubitrenegade.com/virtualization/kvm/vfio/2019/07/17/VFIO-Fedora-Notes.html

This method of VFIO supports identical GPU as described in the original reference.

This gist is only to fix some typo or error I encountered when applying the steps decribed in the original reference, while also adding my own customization.
Machine Spec:
CPU: AMD Ryzen 1700
RAM: 64GiB (20GiB will be dedicated to the Guest)
GPU1: Nvidia GT710 (Host)
GPU2: Nvidia GTX1060 (Guest)

Install KVM & Libvirt

sudo dnf -y install bridge-utils libvirt virt-install qemu-kvm virt-top libguestfs-tools virt-manager

Networking

export BR_NAME=br0
export BR_INTERFACE=enp5s0

export BR_INT_OG_UUID=$(nmcli -g GENERAL.CON-UUID device show "${BR_INTERFACE}")

nmcli c add type bridge autoconnect yes con-name "${BR_NAME}" ifname "${BR_NAME}"
nmcli c modify "${BR_NAME}" bridge.stp no

# I need this because I have static DHCP mapping on my home network
nmcli c modify br0 bridge.mac-address '<mac-address>'

nmcli c add type bridge-slave autoconnect yes con-name ${BR_INTERFACE} ifname ${BR_INTERFACE} master ${BR_NAME}
nmcli c down "${BR_INT_OG_UUID}"
nmcli c up "${BR_NAME}"
nmcli c delete "${BR_INT_OG_UUID}"

GRUB Kernel Parameter

sudo sed -i .old 's|\(GRUB_CMDLINE_LINUX.*\)[XZ]*"|\1 amd_iommu=on iommu=pt rd.driver.pre=vfio-pci default_hugepagesz=1G hugepagesz=1G hugepages=20"|' /etc/default/grub

Modprobe Config

echo 'install vfio-pci /usr/local/bin/vfio-pci-override.sh' | sudo tee /etc/modprobe.d/vfio.conf > /dev/null
echo 'options kvm ignore_msrs=1' | sudo tee /etc/modprobe.d/kvm.conf > /dev/null

Dracut Config

cat << EOF | sudo tee /etc/dracut.conf.d/vfio.conf > /dev/null
add_drivers+=" vfio vfio_iommu_type1 vfio_pci vfio_virqfd "
install_items+=" /usr/local/bin/vfio-pci-override.sh /usr/bin/dirname "
EOF

VGA Passthrough Script


save content of vfio-pci-override.sh to /usr/local/bin/vfio-pci-override.sh
run sudo chmod +x /usr/local/bin/vfio-pci-override.sh

Update Boot System

sudo dracut -f --kver $(uname -r)
sudo grub2-mkconfig -o /etc/grub2-efi.cfg


## vfio-pci-override.sh
#!/bin/sh

set -u

for boot_vga in /sys/bus/pci/devices/*/boot_vga; do
  echo "Found vga device: ${boot_vga}"
  if [ $(<"${boot_vga}") -eq 0 ]; then
    echo "Found Boot VGA Device - false: ${boot_vga}"

    dir=$(dirname -- "${boot_vga}")
    for dev in "${dir::-1}"*; do
      echo "Registering Devices: ${dev}"
      echo 'vfio-pci' > "${dev}/driver_override"
    done
  else
    echo "Found Boot VGA Device - true: ${boot_vga}"
  fi
done

modprobe -i vfio-pci
	#!/bin/sh

	set -u

	for boot_vga in /sys/bus/pci/devices/*/boot_vga; do
	echo "Found vga device: ${boot_vga}"
	if [ $(<"${boot_vga}") -eq 0 ]; then
	echo "Found Boot VGA Device - false: ${boot_vga}"

	dir=$(dirname -- "${boot_vga}")
	for dev in "${dir::-1}"*; do
	echo "Registering Devices: ${dev}"
	echo 'vfio-pci' > "${dev}/driver_override"
	done
	else
	echo "Found Boot VGA Device - true: ${boot_vga}"
	fi
	done

	modprobe -i vfio-pci