Running GPU Accelerated Games with OpenShift Virtualization

README.md

Running GPU Accelerated Games with OpenShift Virtualization

Introduction

The following document describes at a high level how one can use OpenShift Virtualization, an NVIDIA GPU, and VirtualGL to run hardware accelerated remote games.

Requirements

• OpenShift 4.8 or Higher
• OpenShift Virtualization 4.8 or higher
• OpenShift Worker Nodes with NVIDIA Graphics Card Installed
  • NVIDIA P4 used in this document

OpenShift Setup

You will need an OpenShift 4.8 (or higher) cluster available that is running on Bare Metal. See: https://docs.openshift.com/container-platform/4.8/installing/installing_bare_metal/preparing-to-install-on-bare-metal.html

Once your BM cluster is up and running, install OpenShift Virtualization. See: https://docs.openshift.com/container-platform/4.8/virt/install/preparing-cluster-for-virt.html

Once OpenShift Virtualization is intalled, configure OpenShift Virtualization for GPU Passthrough. See: https://docs.openshift.com/container-platform/4.8/virt/virtual_machines/advanced_vm_management/virt-configuring-pci-passthrough.html

Finally Create an instance of a Fedora34 VM using the fedorawk1.yml file below. See: https://docs.openshift.com/container-platform/4.8/virt/virtual_machines/virt-create-vms.html NOTE: The yaml below will create a vm with a user of "fedora:fedora", be sure to update the password field if you do not want this.

VM Setup

Log into the VM you created and run the following commands:

$ sudo su -
$ dnf update -y
$ dnf install -y VirtualGL supertuxkart

Configure VirtualGL per the following two documents:

1. Install NVIDIA Drivers: https://ask.fedoraproject.org/t/installing-nvidia-drivers-on-fedora-34/14071
2. Configure X for Headless configuration: https://virtualgl.org/Documentation/HeadlessNV
3. Configure and start VirtualGL: https://cdn.rawgit.com/VirtualGL/virtualgl/2.6.5/doc/index.html#hd006

Remote Workstation Setup

Log into the workstation you want to display the remote workload on. Example instructions assume Fedora34 workstation.

$ sudo su -
$ dnf update -y
$ dnf install -y VirtualGL

Running the Demo

Once your remote VM and display workstation are setup you can run the remote workload. VirtualGL can stream the application and 3D accelerated video stream over SSH. We will use this process to connect and run the SuperTuxCart application.

From the workstation you want to display the application on run:

$ vglconnect -s <IP address> -p <NodePort>
# you will be prompted for your password twice. This sets up the inital connection and then the VirtualGL tunnel
$ vglrun supertuxcart --demo-mode=1

At this point SuperTuxCart should start on your workstation, using the Graphics Card from your remote virtual machine.

fedorawk1.yml

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  annotations:
  name: fedorawk1
  namespace: myvms
spec:
  dataVolumeTemplates:
    - metadata:
        creationTimestamp: null
        name: fedorawk1-rootdisk-em0yd
      spec:
        pvc:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 60Gi
          storageClassName: hostpath-provisioner
          volumeMode: Filesystem
        source:
          http:
            url: >-
              https://download.fedoraproject.org/pub/fedora/linux/releases/34/Cloud/x86_64/images/Fedora-Cloud-Base-34-1.2.x86_64.raw.xz
  running: false
  template:
    metadata:
      annotations:
        vm.kubevirt.io/flavor: large
        vm.kubevirt.io/os: fedora
        vm.kubevirt.io/workload: desktop
      creationTimestamp: null
      labels:
        flavor.template.kubevirt.io/large: 'true'
        kubevirt.io/domain: fedorawk1
        kubevirt.io/size: large
        os.template.kubevirt.io/fedora34: 'true'
        vm.kubevirt.io/name: fedorawk1
        workload.template.kubevirt.io/desktop: 'true'
    spec:
      domain:
        cpu:
          cores: 1
          sockets: 2
          threads: 1
        devices:
          disks:
            - disk:
                bus: virtio
              name: cloudinitdisk
            - bootOrder: 1
              disk:
                bus: virtio
              name: rootdisk
          gpus:
            - deviceName: nvidia.com/P4
              name: gpu1
          inputs:
            - bus: virtio
              name: tablet
              type: tablet
          interfaces:
            - masquerade: {}
              model: virtio
              name: default
          networkInterfaceMultiqueue: true
          rng: {}
        machine:
          type: pc-q35-rhel8.4.0
        resources:
          requests:
            memory: 8Gi
      evictionStrategy: LiveMigrate
      hostname: fedorawk1
      networks:
        - name: default
          pod: {}
      terminationGracePeriodSeconds: 180
      volumes:
        - cloudInitNoCloud:
            userData: |
              #cloud-config
              user: fedora
              password: fedora
              chpasswd:
                expire: false
          name: cloudinitdisk
        - dataVolume:
            name: fedorawk1-rootdisk-em0yd
          name: rootdisk

gistfile1.txt

kind: Service
apiVersion: v1
metadata:
  name: fedorawk1-vnc-service
  namespace: myvms
spec:
  ports:
    - protocol: TCP
      port: 25901
      targetPort: 5901
  type: NodePort
  ipFamilyPolicy: SingleStack
  sessionAffinity: None
  selector:
    vm.kubevirt.io/name: fedorawk1

sshNodePortService.yml

kind: Service
apiVersion: v1
metadata:
  name: fedorawk1-ssh-service
  namespace: myvms
spec:
  ports:
    - protocol: TCP
      port: 22000
      targetPort: 22
  type: NodePort
  ipFamilyPolicy: SingleStack
  sessionAffinity: None
  selector:
    vm.kubevirt.io/name: fedorawk1