Testing zfs zone'd datasets with Linux Containers under PVE

zfs_zoned_pve.md

For debian12base2 CT under PVE

Container

• Debian 12 (bookworm) with PVE/PBS APT sources added, primarily for matching ZFS userspace distrib, but also to optionally install any PVE/PBS components

APT Sources Host and CT

/etc/apt/sources.list

deb https://mirrors.xmission.com/debian bookworm main contrib non-free non-free-firmware
deb-src https://mirrors.xmission.com/debian bookworm main contrib non-free non-free-firmware

deb https://mirrors.xmission.com/debian bookworm-backports main contrib non-free non-free-firmware
deb-src https://mirrors.xmission.com/debian bookworm-backports main contrib non-free non-free-firmware

deb https://mirrors.xmission.com/debian bookworm-backports-sloppy main contrib non-free non-free-firmware
deb-src https://mirrors.xmission.com/debian bookworm-backports-sloppy main contrib non-free non-free-firmware

deb https://mirrors.xmission.com/debian bookworm-proposed-updates main contrib non-free non-free-firmware
deb-src https://mirrors.xmission.com/debian bookworm-proposed-updates main contrib non-free non-free-firmware

deb https://mirrors.xmission.com/debian bookworm-updates main contrib non-free non-free-firmware
deb-src https://mirrors.xmission.com/debian bookworm-updates main contrib non-free non-free-firmware

# security updates
deb https://security.debian.org bookworm-security main contrib non-free non-free-firmware

/etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg

wget https://enterprise.proxmox.com/debian/proxmox-release-bookworm.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg

/etc/apt/sources.list.d/pve-enterprise.list

#deb https://enterprise.proxmox.com/debian/pve bookworm pve-enterprise

/etc/apt/sources.list.d/pve-no-subscription.list

deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription

/etc/apt/sources.list.d/pbs-enterprise.list

#deb https://enterprise.proxmox.com/debian/pbs bookworm pbs-enterprise

/etc/apt/sources.list.d/pbs-no-subscription.list

deb http://download.proxmox.com/debian/pbs bookworm pbs-no-subscription

LXC /dev/zfs Mapping

root@soldier10:~# cat /etc/pve/nodes/soldier10/lxc/103.conf
...
lxc.cgroup2.devices.allow: c 10 249 rwm
lxc.mount.entry: /dev/zfs dev/zfs none bind,create=file

ZFS Dataset Settings on Host

# Create datasets to namespace into containers
zfs create -o mountpoint=none rpool/ct
zfs create rpool/ct/103

# Find init process of container (Started after /dev/zfs mapping)
ps faux
...
root      123684  0.0  0.0   5024  2816 ?        Ss    2023   2:07 /usr/bin/lxc-start -F -n 103
100000    123700  0.0  0.0 167728  9472 ?        Ss    2023   0:11  \_ /sbin/init
...

# Verify user namespace exists (unprivileged container)
root@soldier10:~# lsns | grep 123700
...
4026532835 user       22 123700 100000    ├─/sbin/init
...

# Set namespace for dataset
zfs set zoned=on rpool/ct/103
zfs zone /proc/123700/ns/user rpool/ct/103

ZFS Settings in CT

• Set APT Sources
• apt install zfsutils-linux to match host

View in CT

root@debian12base2:~# zfs list
NAME                USED  AVAIL  REFER  MOUNTPOINT
rpool              63.9G   386G    96K  /rpool
rpool/ct            480K   386G    96K  none
rpool/ct/103        192K   386G    96K  none

root@debian12base2:~# zfs create -o mountpoint=/tank rpool/ct/103/tank
root@debian12base2:~# zfs list
NAME                USED  AVAIL  REFER  MOUNTPOINT
rpool              63.9G   386G    96K  /rpool
rpool/ct            480K   386G    96K  none
rpool/ct/103        192K   386G    96K  none
rpool/ct/103/tank    96K   386G    96K  /tank

root@debian12base2:~# mount | grep tank
rpool/ct/103/tank on /tank type zfs (rw,noatime,xattr,noacl,casesensitive)

Next

• Lookup environment variables for PVE hookscript and see if I can automate zfs zone and zfs unzone
• Try ZFS Docker driver in CT

This is really great work.

If your interested I've created a quick hook script to automate zfs zone.

#!/bin/bash

vmId="$1"
runPhase="$2"
echo "Running $runPhase on VM=$vmId"

case "$runPhase" in
    post-start)
        pid=$( lxc-info -n $1 -p | awk '/PID:/{print $NF}' )
        zfs zone /proc/$pid/ns/user rpool/ct/$1
        ;;
esac

If your interested I've created a quick hook script to automate zfs zone.

Thanks! I'll give this a try!