Skip to content

Instantly share code, notes, and snippets.

@InsanePrawn
Last active March 7, 2024 18:11
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save InsanePrawn/83d12bff45695a287c0a815baebbf953 to your computer and use it in GitHub Desktop.
Save InsanePrawn/83d12bff45695a287c0a815baebbf953 to your computer and use it in GitHub Desktop.
block device passthrough into systemd-nspawn for testing the munin smartctl plugin
[Match]
Name=host*
[Network]
DHCP=yes

My quick (incomplete) notes and gotchas on how to setup an nspawn container that can query the host's smart data. I guess LXC also works but that would've been way too easy. Praise the systemd overlords!

create a chroot

(Feel free to substitute this with debootstrap or yum or whatever you fancy. You need a somewhat recent systemd though, so no debian 7, sorry.) based on the arch wiki:

# feel free to not select anything that you won't need in your container,
# for example pci-related stuff and the linux kernel package
# optional stuff i like in my systems in square brackets
pacstrap -i -c -d ~/munin-chroot base smartmontools munin-node [zsh sudo]

if you want this as a service unit, you should probably move the chroot to /var/lib/machines/munin or bind mount it there. The arch wiki states systemd can't handle symlinks there, but the info is about 10 systemd versions old. I haven't tested whether that's still correct, a bind mount definitely works.

You should probably give yourself a user and make it comfy for yourself at this point.

Test the container

# try root, it will log you in without asking for a password.
# in theory, `smartctl -a /dev/sda` should work at this point, unless your host has no /dev/sda ...
sudo systemd-nspawn -nbD $YOUR_CHROOT --capability=CAP_SYS_ADMIN,CAP_SYS_RAWIO --bind=/dev/sd*

if anything doesn't work at this point, fix it. if it's smartctl failing with permission denied, you have my prayers.

Optional configurations

SSH to make your live easier:

configure DHCP in the container

we have added host-networking to the container with the -n switch, but it doesn't automatically request an IP, at least not on arch. Configure your favourite dhcp client to request an ip on host0. That's right, the interface is called host0. Since I'm feeling Poettering today, I've attached a systemd-networkd .network config file for you to put in /etc/systemd/network/ in the chroot. Don't forget to ```systemctl enable systemd-networkd` on at least the host, even if you use a different dhcp client in your container.

resolve container name via DNS:

if you want to reach your container via its name, edit your /etc/nsswitch.conf like so: hosts: files mymachines dns myhostname great, you should now immediatly be able to ping munin if the container is running (and has aquired a DHCP lease...)

Enable sshd

Install openssh, then systemctl enable sshd

Enable DNS resolution in the container

Your container probably has access to external IPs already (test by pinging your favorite online service's ip), but lacks DNS resolution. Fix this by putting your favorite DNS server (possibly your host if you run a DNS server on it and bind to all IPs) into resolv.conf. You can now install software from inside the container with ease!

Make it a service

Enable the initial service file

systemctl enable systemd-nspawn@munin

Allow the service to access the block device files

Now you've got your systemd-nspawn@munin.service file, but it restricts itself to a very limited subset of your /dev/ tree. Now, we could just edit the file directly, but it's a symlink to the systemd-nspawn@.service template file, so we'd be editing the template. The clean way is to issue a systemctl edit systemd-nspawn@munin.service. This will drop you into an empty text editor. It's worth nothing that when you try overriding directives, you have to specify the section header first. I've attached my override file for you.

Not all the configuration options for the nspawn container go into the service file, the above line just makes systemd configure the slice's cgroup allow us acces to the block device files, we still have to create the bind mounts for the actual files somehow. This is where /etc/systemd/nspawn/ (I actually had to mkdir that) comes in: nspawn parses .nspawn files named after the container in that directory when the container is started by name.

GOTCHA: The nspawn cli uses comma-separated input for capacities, the .nspawn file uses space separated values. I've attached my munin.nspawn, which contains the bind mount and capacity directives.

Finally, start the systemd-nspawn@munin.service and attach to it either via machinectl or ssh.

Happy smartctl-ing!

[Exec]
Capability=CAP_SYS_ADMIN CAP_SYS_RAWIO
[Files]
# I have yet to figure out how to make this one take a wildcard or something equivalent, sorry :/
# I guess mayb you could automate setting up a directory containing hard links
# to the block device files in the host's /dev fs, then bind(mount) that... ugh.
Bind=/dev/sda
[Service]
# sadly, wildcards don't seem to work :(
#DeviceAllow=/dev/sd* rwm
# you could instead specify the devices directly, that works!
#DeviceAllow=/dev/sda rwm
#DeviceAllow=/dev/sdb rwm
# my solution is a bit broad, but i use the container to prevent cluttering my host, not for security purposes:
DeviceAllow=block-sd rwm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment