Skip to content

Instantly share code, notes, and snippets.

@blenessy

blenessy/jammy-amd-sev-snp.md

Created December 4, 2022 11:40
Show Gist options
  • Save blenessy/415e30d341bc115edec5b3704d4f4fe9 to your computer and use it in GitHub Desktop.
Save blenessy/415e30d341bc115edec5b3704d4f4fe9 to your computer and use it in GitHub Desktop.
Download ZIP
Building the 5.19 kernel with AMD SEV-SNP patches on/for Ubuntu 22.04 and later
Raw
jammy-amd-sev-snp.md

Getting AMD SEV-SNP to work on Ubuntu 22.04

If you opened the champagne after the AMD SEV-SNP Finally Being Merged In Linux 5.19 article like I did, you started celebrating too early it seems. You are likely missing essential patches from AMD's sev-snp-iommu-avic_5.19-rc6_v4 branch.

This GIST will properly enable AMD SEV-SNP support on 22.04 and later (works with 22.10 as well).

Before we begin - this is how success looks like on my Dell R6515 (with AMD EPYC 7313P). If this is NOT what you are getting then you should keep reading:

# dmesg | grep -i 'SEV\|SME\|SNP\|ccp'
[    0.074408] Kernel command line: BOOT_IMAGE=/vmlinuz-5.19.0-rc6-v4-sev-snp-iommu-avic root=UUID=a47fbf65-1545-4923-9a07-53822cc6ba89 ro mem_encrypt=on kvm_amd.sev=1
[    0.074455] Unknown kernel command line parameters "BOOT_IMAGE=/vmlinuz-5.19.0-rc6-v4-sev-snp-iommu-avic mem_encrypt=on", will be passed to user space.
[    0.173965] Memory Encryption Features active: AMD SME
[    0.470146] AMD-Vi: SNP enabled
[    0.470148] SEV-SNP: RMP table physical address 0x0000000046000000 - 0x00000000568fffff
[    0.497163] AMD-Vi: Extended features (0x841f77e022094ace, 0x0): PPR X2APIC NX IA GA PC SNP
[    0.497171] AMD-Vi: Extended features (0x841f77e022094ace, 0x0): PPR X2APIC NX IA GA PC SNP
[    0.497176] AMD-Vi: Extended features (0x841f77e022094ace, 0x0): PPR X2APIC NX IA GA PC SNP
[    0.497181] AMD-Vi: Extended features (0x841f77e022094ace, 0x0): PPR X2APIC NX IA GA PC SNP
[    1.283684]     BOOT_IMAGE=/vmlinuz-5.19.0-rc6-v4-sev-snp-iommu-avic
[    1.852179] usb usb1: Manufacturer: Linux 5.19.0-rc6-v4-sev-snp-iommu-avic xhci-hcd
[    1.971238] usb usb2: Manufacturer: Linux 5.19.0-rc6-v4-sev-snp-iommu-avic xhci-hcd
[    2.152875] usb usb3: Manufacturer: Linux 5.19.0-rc6-v4-sev-snp-iommu-avic xhci-hcd
[    2.235459] usb usb4: Manufacturer: Linux 5.19.0-rc6-v4-sev-snp-iommu-avic xhci-hcd
[    4.281607] ccp 0000:46:00.1: no command queues available
[    4.282549] ccp 0000:46:00.1: sev enabled
[    4.282554] ccp 0000:46:00.1: psp enabled
[    5.488544] ccp 0000:46:00.1: SEV API:1.52 build:4
[    5.488550] ccp 0000:46:00.1: SEV-SNP API:1.52 build:4
[    5.501067] SEV supported: 310 ASIDs
[    5.501068] SEV-ES and SEV-SNP supported: 199 ASIDs

Building and installing AMD's SEV-SNP enabled kernel

Check this great guide for more in-depth explation of what we are doing.

This is the bare minimum dependencies to build the Kernel in question:

apt install --no-install-recommends wget build-essential bison flex libncurses-dev libssl-dev libelf-dev fakeroot

Download the and unpack the contents of the sev-snp-iommu-avic_5.19-rc6_v4 branch:

mkdir ~/amd-sev-snp-kernel
cd ~/amd-sev-snp-kernel
wget 'https://github.com/AMDESE/linux/archive/refs/heads/sev-snp-iommu-avic_5.19-rc6_v4.tar.gz'
tar -xaf sev-snp-iommu-avic_5.19-rc6_v4.tar.gz
cd ~/amd-sev-snp-kernel/linux-sev-snp-iommu-avic_5.19-rc6_v4

Copy in your current kernel config and merge with the defaults from the downloaded kernel:

cp "/boot/config-$(uname -r)" .config
make olddefconfig

Disable some configs that are in the way:

scripts/config --set-str SYSTEM_TRUSTED_KEYS ""
scripts/config --set-str SYSTEM_REVOCATION_KEYS ""

Optionally, disable some debug configs to save space and build faster:

scripts/config --undefine DEBUG_INFO
scripts/config --undefine DEBUG_INFO_COMPRESSED
scripts/config --undefine DEBUG_INFO_REDUCED
scripts/config --undefine DEBUG_INFO_SPLIT
scripts/config --undefine GDB_SCRIPTS
scripts/config --set-val  DEBUG_INFO_DWARF5     n
scripts/config --set-val  DEBUG_INFO_NONE       y

Build the kernel and package it into installable DEB packages:

make -j "$(nproc)" bindeb-pkg LOCALVERSION=-v4-sev-snp-iommu-avic

Install your kernel and supporting packages with:

dpkg -i ~/amd-sev-snp-kernel/*.deb

Enabling SEV/SME support in Kernel/KVM

According to the libvirt guide on SEV you kernel cmdline needs to include mem_encrypt=on kvm_amd.sev=1. You can set this in /etc/default/grub:

GRUB_CMDLINE_LINUX_DEFAULT="mem_encrypt=on kvm_amd.sev=1"

Before rebooting don't forget to regenerate grub:

sudo update-grub

Enable SEV/SNP support in your BIOS

By default SEV/SNP support is disabled on my Dell R6515.

  • Update your BIOS (I have 2.8.5 on my Dell R6515).
  • Turn on SEV, SME and SNP, but leave TSME disabled. More specifically, for Dell R6515 I had to enable/set the following:
    • Secure Memory Encryption (SME)
    • Secured Nested Paging (SNP)
    • SNP Memory Coverage
    • Minimum SEV non-ES ASID=200

Regarding the Minimum SEV non-ES ASID setting... There at 509 ASID in total on Milan CPUs, and it seems like you partition these 509 ASIDs between plain SEV and SEV-SE/SNP guest VMs. The default setting of 1 will not allow you to run any SEV-ES or SEV-SNP guest so you need to crank this up a bit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment