Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Fedora with Root on ZFS - installation notes

Fedora: Root on ZFS

  • Fedora: 28
  • ZFS on Linux: 0.7.9

A. Prepare source

  1. Download Fedora Workstation network installer image.

  2. Boot live media by adding inst.gpt at the end of the boot line and set up with:

    • a boot partition /boot/efi of size 512MB
    • a root partition / of size rest of the disk
    • minimal installation
  3. Reboot and:

    • run dnf -y update
    • set hostname, networking, idmapd
    • output of rpm -qa|grep "grub2-efi-x64\|shim-x64" should contain:
    grub2-efi-x64
     shim-x64
    • install UEFI modules and some required packages:
    dnf -y install grub2-efi-x64-modules kernel-devel gdisk rsync bridge-utils
    • add system wide profile customization in /etc/profile.d/
    • run init 6
  4. Create scripts:

    • to run dracut and make GRUB ZFS aware:
    #!/bin/bash
     kver=$1
     sh -x /usr/bin/dracut -fv --kver $kver
     mount /boot/efi
     grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg
     grubby --set-default-index 0
     mkdir -p /boot/efi/EFI/fedora/x86_64-efi
     cp -a /usr/lib/grub/x86_64-efi/zfs* /boot/efi/EFI/fedora/x86_64-efi
     umount /boot/efi
    • to enter chroot:
    #!/bin/bash
     target=$1
     mount -t proc  proc $target/proc
     mount -t sysfs sys $target/sys
     mount -o bind /dev $target/dev
     mount -o bind /dev/pts $target/dev/pts
     chroot $target /bin/env -i HOME=/root \
         TERM="$TERM" PS1='[\u@chroot \W]\$ ' \
         PATH=/bin:/usr/bin:/sbin:/usr/sbin \
         /bin/bash --login
     echo "Exiting chroot environment..."
     umount $target/dev/pts
     umount $target/dev/
     umount $target/sys/
     umount $target/proc/
    • make both scripts executable
  5. Add ZFS support to GRUB in /etc/default/grub:

GRUB_PRELOADED_MODULES="zfs"

so that it can detect the ZFS filesystem; and

GRUB_DISABLE_OS_PROBER=true

so that grub2-mkconfig does not throw errors.

  1. Fix GRUB's handling of pool device path names by creating /etc/profile.d/grub2_zpool_fix.sh containing
export ZPOOL_VDEV_NAME_PATH=YES

The effect is that zpool status will list the full path of the block device: /dev/disk/by-id/nvme0n1-part2 instead of nvme0n1-part2.

  1. Prepare for future kernel updates:
ln -s /usr/local/sbin/zmogrify /etc/kernel/postinst.d
  1. Add ZFS on Linux repository then install and load ZFS:
dnf install zfs zfs-dracut
modprobe zfs
  1. Disable cache and enable scan:
systemctl disable zfs-import-cache
systemctl enable zfs-import-scan
  1. Regenerate initramfs:
dracut -fv --kver `uname -r`
  1. Reboot

B. Prepare target

  1. Wipe disks:
sgdisk --zap-all /dev/disk/by-id/nvme0n1
sgdisk --zap-all /dev/disk/by-id/nvme1n1
  1. Create boot partition (UEFI) and dedicate the rest to ZFS:
sgdisk -n1:1M:+512M -t1:EF00 /dev/disk/by-id/nvme0n1
sgdisk -n2:0:0 -t2:BF01 /dev/disk/by-id/nvme0n1
sgdisk -n1:1M:+512M -t1:EF00 /dev/disk/by-id/nvme1n1
sgdisk -n2:0:0 -t2:BF01 /dev/disk/by-id/nvme1n1
  1. Format boot partition:
mkfs.fat -F32 /dev/disk/by-id/nvme0n1-part1
mkfs.fat -F32 /dev/disk/by-id/nvme1n1-part1
  1. Create the pool:
zpool create -m none -O canmount=off -o ashift=12 -o cachefile=none rpool mirror /dev/disk/by-id/nvme0n1-part2 /dev/disk/by-id/nvme1n1-part2
  1. Create a container dataset:
zfs create -o canmount=off -o mountpoint=none rpool/ROOT
  1. Create the root dataset:
zfs create rpool/ROOT/default
zfs set acltype=posixacl rpool/ROOT/default # systemd-journald's /var/log/journal
zfs set xattr=sa rpool/ROOT/default
zfs set compression=on rpool/ROOT/default
zfs set atime=off rpool/ROOT/default
  1. Create the home dataset:
zfs create -o compression=on -o atime=off -o xattr=sa -o acltype=posixacl rpool/home
  1. Create a dataset for storing data:
zfs create -o compression=on -o atime=off -o xattr=sa -o acltype=posixacl rpool/local
  1. Create a swap volume:
zfs create -V 16G -b $(getconf PAGESIZE) -o compression=zle -o logbias=throughput -o sync=always -o primarycache=metadata -o secondarycache=none -o com.sun:auto-snapshot=false rpool/swap
mkswap /dev/zvol/rpool/swap
  1. Export pool:
zpool export rpool
  1. Import pool into an alternate mount point:
zpool import rpool -d /dev/disk/by-id -o altroot=/sysroot
  1. Set the mountpoint for the root and for home and local datasets:
zfs set mountpoint=/ rpool/ROOT/default
zfs set mountpoint=/home rpool/home # directory is empty
zfs set mountpoint=/local rpool/local # directory is empty
  1. Check mountpoints.

  2. Transfer data:

rsync -WSHAXhavx / /sysroot/
  1. Copy EFI partition from the source to the target:
cd
mkdir here
mount /dev/disk/by-id/nvme0n1-part1 here
cp -a /boot/efi/* here
umount here
rmdir here
  1. Add boot and swap partitions to fstab:
blkid /dev/disk/by-id/nvme0n1-part1

Replace content of /sysroot/etc/fstab with:

UUID=<UUID from blkid>  /boot/efi  vfat  umask=0077,shortname=winnt  0  2
/dev/zvol/rpool/swap    swap       swap  defaults                    0  0
  1. Enter chroot:
zenter /sysroot
  1. Needed by F28's GRUB is a line containing:
GRUB_DEVICE_BOOT=/dev/disk/by-id/nvme0n1-part2

in /etc/default/grub.

  1. Transform target into a ZFS aware system:
zmogrify \`uname  -r\`
  1. Confirm that GRUB can recognize the ZFS filesystem:
grub2-probe /

should output zfs.

  1. Exit chroot

  2. Export pool:

zpool export rpool
  1. Reboot

  2. Take an initial snapshot:

zfs snapshot rpool/ROOT/default@initial
  1. Create user and group

  2. Complete installation:

dnf -y groupinstall "Fedora Workstation"
systemctl set-default graphical.target
systemctl enable gdm.service
  1. Reboot

References:

@kpfleming

This comment has been minimized.

Copy link

@kpfleming kpfleming commented Apr 26, 2020

If you change your sgdisk commands to be like this you won't have to 'fix' the partition numbering:

sgdisk -n1:1M:+512M -t1:EF00 /dev/disk/by-id/nvme0n1
sgdisk -n2:0:0 -t2:BF01 /dev/disk/by-id/nvme0n1
sgdisk -n1:1M:+512M -t1:EF00 /dev/disk/by-id/nvme1n1
sgdisk -n2:0:0 -t2:BF01 /dev/disk/by-id/nvme1n1
@p7cq

This comment has been minimized.

Copy link
Owner Author

@p7cq p7cq commented Apr 26, 2020

Indeed, thank you,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.