I like ZFS and its features, and I don't want to mess with GRUB.
- Fedora 32+ (Probably) bootable image
- Dracut 050+ (for improved uefi support and features)
- ZFS 0.8.4
Boot into your fedora image through whatever means you prefer. You'll want to make sure that the live image has the running
kernel verions of kernel-devel and kernel-headers which can be done with:
dnf install -y kernel-headers-$(uname -r) kernel-devel-$(uname -r)
this will allow dkms to build the zfs modules when you install it.
Next, install the current zfs release rpm for the version of Fedora that you're installing.
At the time of writing that's 32 so:
dnf install -y http://download.zfsonlinux.org/fedora/zfs-release.fc32.noarch.rpm
and then install zfs dnf install -y zfs and then modprobe zfs afterward.
This should complete succesfully in most cases, but since the live image doesn't have a large amount of storage it may fail.
If it does fail you can usually free up some working space from the earlier dnf steps by doing dnf clean all
to clean up all the dnf cache files, and then manually trigger the dkms install process for the running kernel
dkms install zfs/0.8.4 -k $(uname -r) and load the zfs module modprobe zfs.
I'm assuming that the disks you're using here will be used entirely for your root pool (and esp). If this isn't the case you'll want to modify to fit your needs. Additionally I'm assuming you've got two disks and you're going to make a mirror pool. Again if not, just ignore the bits regarding multiple disks.
You need to partition your disks to allow for the creation of a uefi system partition. You can do this with the gdisk utility which should be included with the live image, if it's not just install it with dnf. You'll want to create a new gpt partition table on the disks. Next create two new partitions on each disk, one 500M for the esp with type ef00 (important), and the second the size of the remaining disk space.
Now we'll make a raid1 with mdadm to mirror the esp. This works with a surprising number of uefi motherboards, but it might
not for yours. If it doesn't you can always just use one of the esps as /boot/efi instead, it's not perfect, but eh.
mdadm --create /dev/md0 --level 1 --raid-disks 2 --metadata 1.0 /dev/nvme0n1p1 /dev/nvme1n1p1 the metadata argument is
important here. If you don't include that it puts the metadata at the front of the partition instead of the end which messes
up most firmware's detection of the esp. Now you should create the fat32 filesystem on the md device mkfs.vfat -F32 /dev/md0.
Now let's make the zpool. We don't need anything special here other than what you would probably want for a zpool on a linux
host zpool create -o ashift=12 -O xattr=sa -O acltype=posixacl -O compression=on -O atime=off -O relatime=off -O mountpoint=none -R /mnt/sysroot rpool mirror /dev/nvme0n1p2 /dev/nvme1n1p2
This will create a pool rpool that is a mirror of the two nvme devices at an altroot located at /mnt/sysroot.
Obviously feel free to change your ashfit value as necessary, or enable other features like encryption.
Since we're not messing around with grub we can turn on whatever we please. Finally, let's set the fedora dataset as
the bootfs zpool set bootfs=rpool/fedora rpool
At this point you'll have the pool created.
Let's create a dataset for the fedora root to go in. You can do whatever
hierachy you like here, I'm lazy so I'm just going to make a rpool/fedora dataset. zfs create -o mountpoint=/ rpool/fedora
I'm also going to create a dedicated var and /var/log. You don't have to do this if you don't want to. zfs create -o mountpoint=legacy rpool/fedora/var
zfs create -o mountpoint=legacy rpool/fedora/var/log
Let's also create the mountpoint for the esp. mkdir -p /mnt/sysroot/boot/efi and mount it there
mount /dev/md0 /mnt/sysroot/boot/efi .
Now we'll actually install fedora dnf --installroot=/mnt/sysroot --releasever=32 group install "Xfce Desktop" this
will pull in all the packages needed for the Xfce spin of fedora (more or less). You can choose from any of the other
groups available, or even just "Minimal Install" or "Fedora Custom Operating System" whatever you want.
At this point we've got a fedora install on our rpool, but we still need to do some stuff to finish up, so let's get things set up to chroot into it.
mount -t proc proc /mnt/sysroot/proc
mount -t sysfs sys /mnt/sysroot/sys
mount -o rbind /dev /mnt/sysroot/dev
cp /etc/resolv.conf /mnt/sysroot/etc
chroot /mnt/sysroot /usr/bin/bash
now we're in our freshly installed fedora root. At this point let's make sure we've got the same kernel and junk as the live
environemnt, for no real reason than consistency.
dnf install -y kernel-$(uname -r) kernel-headers-$(uname -r) kernel-devel-$(uname -r) http://download.zfsonlinux.org/fedora/zfs-release.fc32.noarch.rpm busybox efivar efivar-libs mokutil efibootmgr
and we can install zfs in the root dnf install -y zfs This should go a head and install and build the modules for
the kernel versions that are installed, but just in case it doesn't we can manually run the dkms install for all the installed kernels
dkms install zfs/0.8.4 -k <kernel version here>
Now it's time for the actual special sauce that makes this possible, dracut. Recent versions of dracut have added the ability
to create efi images with the stub kernel, kernel, and initramfs all packaged up nicely. This eliminates the need for a dedicated
bootloader, like grub, or even refind. But we need to create some configuration to tell dracut to do this.
uefi=yes
hostonly=yes
hostonly_cmdline=no
use_fstab=yes
early_microcode=yes
add_drivers+=" zfs "
add+=" busybox "
kernel_cmdline="rw rd.shell root=zfs:AUTO"
You can add any other kernel command line arguments that you want to kernel_cmdline. Now put that in /etc/dracut.conf.d/zfs-uefi.conf
and then run dracut --force --regenerate-all which should then recreate initrds for all installed kernels, pull in zfs support
and then pull in the kernel, stub, and resulting initrd into an efi executable located (by default) at /boot/efi/EFI/Linux.
Assuming everything works out your firmware should be able to see these automagically, but you may have to add them using
efibootmgr. efibootmgr --create --disk /dev/nvme0n1p1 --label "Fedora 32 <kernel version>" /EFI/Linux/<Image name here>
You'll probably want to set a root password, you'll probably want to also run a restorecon -Rv / to make sure everything is labeled
correctly for selinux. You may even need to boot with enforcing=0 on the kernel command line the first time and make a custom
policy to cover things that aren't assumed by the default policy. If things take forever to start at boot, or never start,
or logins fail, it's probably selinux. Also make sure to add mounts for /boot/efi and any other legacy mounts (like the var/log dataset) to /etc/fstab.
Then you'll want to exit the chroot, unmount all the directories you mounted into the chroot umount /mnt/sysroot/{dev,sys,proc,boot/efi}
and then export the pool. Exporting the pool is important here, otherwise the pool will fail to import at boot and drop you to a recovery shell.
If that still happens you can always force import the pool in the recovery shell and then continue booting.
At this point we should be done. Ideally we just reboot, the firmware sees the esp and efi image, and boots.
raid1 might not work for your mobo, if nothing shows up that's probably what's happened. If this happens, boot in the live environment, copy the esp contents somewhere, stop the md device, wipefs the partitions that made up the md device, mkfs.vfat the individual partitions, and copy the esp contents back. It should show up now.
Panic at boot, not able to find root. If you have rd.shell on the command line it should drop you to a shell and hopefully give you an idea of why it failed. Usually it's the pool has failed to import. You can force it and then exit the shell and the boot process should continue.
You can't easily modify your kernel command line arguments. You could add rd.cmdline=ask to your kernel_cmdline in the dracut config, which can prompt you for additional arguments. Though I'm not sure if that times out without interaction, I've not played with it yet, so that may not be a good solution.