Skip to content

Instantly share code, notes, and snippets.

@smoser
Last active January 12, 2024 20:07
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save smoser/41dd513b35c9e78eedd6713b30cc8acf to your computer and use it in GitHub Desktop.
Save smoser/41dd513b35c9e78eedd6713b30cc8acf to your computer and use it in GitHub Desktop.
stubby talk at All Systems Go conference September 2023.

All Systems Go 2023: Kernel command line and UKI; systemd-stub and the ‘stubby’ alternative

This talk is was given 2023-09-14 in Berlin at the All Systems Go 2023 conference. It is available online from all-systems-go conference here.

Abstract

Modification of the kernel command line has historically been one of the easiest ways to customize system behavior. Bootloaders allow for persistent changes via config-files and on-the-fly changes interactively during system boot.

System behavior changes made via the kernel command line are not limited to the kernel itself. Userspace applications from installers to init systems and beyond also take input from /proc/cmdline.

It is clear that some kernel command line options are desirable (console=ttyS0 verbose) and possibly even necessary. Others, such as the cromulent 'init=/bin/sh', can allow circumvention of benefits that Secureboot and TPM provide. How to control access to kernel command line modification is a non-trivial subject. A recent pull request to systemd that added "command-line addons" garnered hundreds of comments.

This talk will cover:

  • The stub loader 'stubby' and its allowed-list approach to kernel command line options.
  • Systemd-stub’s solution for command line customization
  • System changes that can be made through kernel command line.
  • Alternative channels such as smbios oem strings, or qemu 'fw_cfg'

Format

The talk.mdp file can be viewed with mdp.

%title: Secure/Measured boot, Kernel command line and stubby %footer: Scott Moser / All Systems Go %author: Scott Moser %date: 2023-09-14

-> # Me


-> # Intro

  • Kernel command line
  • Secure / measured boot + kernel command line (Yikes!)
  • systemd-stub
  • stubby
  • questions

-> # Kernel command line

  • Why?
    • its a command line
    • flexibilty - kernels are big, UKI even bigger. ^
  • Very easy to edit in bootloaders and pxeboot ^
  • What does it do?
    • 1 part system specific configuration (console=, crashkernel, root=, ip=)
    • 1 part workaround (init_blacklist, module_blacklist)
    • 1 part rootkit^H^H^H^H^H debug (init=/bin/bash, rdinit=, selinux=disabled)
    • kernel and userspace

-> # Kernel command line - kernel

  • widely used - over 900 documented in kernel-parameters.txt
  • kernel parameters have no namespace (verbose, console=, vga=, blacklist)
  • module parameters namespace via <modname.varname> (usbcore.blinkenlights=1)
  • key/value type can be quoted (key="value here")
  • generally pretty consistent, improvements over time. ^
    • Ignoring arguments after --
    • "Unknown kernel command line parameters" (flags BOOT_IMAGE=/boot/vmlinuz)

-> # Kernel command line - userspace

  • consumers:
    • systemd (systemd.mask, systemd.debug-shell)
    • cloud-init (cloud-config-url=http://)
    • initramfs (rd.break, rd.shell)
    • installers (ksurl=)
    • ????
  • generally just read /proc/cmdline

-> # Kernel command line - the kitchen sink

  • No central doc, not consistent, consumer specific implementation.
  • confusing - sometimes kernel handles, sometimes initramfs (root=, ip=) ^

The kernel parses parameters from the kernel command line up to "--"; if it doesn't recognize a parameter and it doesn't contain a '.', the parameter gets passed to init: parameters with '=' go into init's environment, others are passed as command line arguments to init. Everything after "--" is passed as an argument to init. ^

  • later values generally override earlier values (console=ttyS0 console=tty1)
  • '-- foo=bar' != 'foo=bar --'
  • Can change init's environment (SYSTEMD_PROC_CMDLINE, PATH, LD_PRELOAD...)

-> # Secureboot / TPM based secrets - Yikes

  • Limit access to TPM secrets based on PCR values.
  • Arbitrary access to the kernel command line becomes attack vector

^


-> # Shim / Stub / EFI loading

  • EFI stub - a stub of an EFI application to load a kernel and initramfs
  • UKI - Unified Kernel Image - an kernel, initramfs, cmdline added to a stub
  • UKI can be loaded via bootloader such as sd-boot, grub or directly by EFI ^
  • You can build in a full kernel command line, but that seems too rigid even for an appliance.

-> # sd-stub

  • If you're using a stub you're probably using this

  • Build with ukify

    ukify build --linux=my-vmlinuz --initrd=my-initrd.img --cmdline='quiet rw' --sign-kernel

  • Very active development > 75 'boot:' commits to systemd in 2023

  • Loads of functionality, could easily over-fill a dedicated talk

  • good doc


-> # sd-stub details

  • companion files along side (kernel.efi -> kernel.efi.d)
  • dynamically generated cpio archives

^

  • command line additions supprted via signed efi addons

    • .efi.extra.d/*.addon.efi
    • sections
      • cmdline
      • sbat
      • uname
  • order of tokens is supplied by filename in ESP


-> # stubby

  • Why?
    • Internal trusted boot appliance with secrets stored in TPM
    • Could not have init=/bin/bash on kernel command line

^

  • simple allowed-list token by token matching
  • no quote/spaces support.
  • exact matches or "begins with" via ^
  • Does not measure the kernel command line to any PCR ^
  • examples
    • verbose, quiet - full token matches
    • ^console=, ^root= - prefix match tokens
    • ^namespace.

-> # stubby - substitution

  • additions are added wherever STUBBY_RT_CLI1 occurs in signed command line

    • builtin: console=ttyS0 root=/dev/vda -- STUBBY_RT_CLI1 runtime: rdinit=/bin/sh result: console=ttyS0 root=/dev/vda -- rdinit=/bin/sh

    • builtin: console=ttyS0 STUBBY_RT_CLI1 runtime: -- systemd.mask rd.break result: console=ttyS0 -- systemd.mask rd.break

    • builtin: console=ttyS0 runtime: anything result: illegal

  • if runtime contains illegal tokens,

    • secure-boot is rejected
    • insecure-boot warns and allows

-> # comparison

┌────────────────────────────────────────────────────────────────────────┐ │ │ │ sd-stub │ stubby │ │ ───────────────────────────────┼─────────────────────────────── │ │ │ │ │ individually signed kernel │ allowed list with prefix │ │ parameters │ matching │ │ │ │ │ external .efi files │ builtin to UKI │ │ │ │ │ parameters appended in user │ parameters substituted into │ │ supplied order │ cmdline in user supplied order │ │ │ │ │ new paramter easy to ship │ extending the allowed-list │ │ (just sign and ship file) │ means shipping new UKI │ │ │ │ │ signing key required to sign │ signing-key required only │ │ UKI and individual params │ at UKI built-time │ └────────────────────────────────────────────────────────────────────────┘


-> # stubby - Going forward

  • Support allowed-list being inserted at smash timeframe.

  • Would like to clean up substitution

  • Fix bug with aarch64 ^

  • Get allowed-list support into sd-stub

    • issue 24539 discusses whitelist solution.
  • systemd only read cmdline things after a '--'?


-> # Thanks / Questions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment