Skip to content

Instantly share code, notes, and snippets.

@kdwinter

kdwinter/Arch-ZFSRoot-on-dm-crypt-UEFI.md

Forked from aaronlauterer/Arch-ZFSRoot-on-dm-crypt-UEFI
Last active July 24, 2023 21:31
Show Gist options
  • Star 12 You must be signed in to star a gist
  • Fork 7 You must be signed in to fork a gist
  • Save kdwinter/2e779abab2e25f8a0bdea7928860fbb5 to your computer and use it in GitHub Desktop.
Save kdwinter/2e779abab2e25f8a0bdea7928860fbb5 to your computer and use it in GitHub Desktop.
Download ZIP
ZFSRoot installation over a dm-crypt volume for Arch Linux (UEFI)
Raw
Arch-ZFSRoot-on-dm-crypt-UEFI.md

Install Arch Linux on root ZFS filesystem

Pre-installation

Download Arch linux

https://archlinux.org

Create bootable USB (don't add partition number!)

# dd bs=4M if=/path/to/archlinux.iso of=/dev/sdx status=progress && sync

Reboot into newly created Arch USB

Installation

Load keymap

# loadkeys be-latin1

Connect to the internet (if not using wired cable)

# wifi-menu

Add the ZFS repo to pacman configuration

# vim /etc/pacman.conf
----------------------
[archzfs]
Server = http://archzfs.com/$repo/$arch

Add the ZFS repo gpg keys to pacman's keyring

# pacman-key -r 0ee7a126
# pacman-key --lsign-key 0ee7a126

Install the zfs-linux and spl-linux packages inside archiso environment

# pacman -Sy linux-headers zfs-dkms spl-dkms

Load the ZFS kernel module and create empty ZFS pool cache file

# modprobe zfs
# touch /etc/zfs/zpool.cache

Partition the disk

# parted /dev/sdx
-----------------
(parted) mklabel gpt
(parted) mkpart ESP fat32 1MiB 513MiB
(parted) set 1 boot on
(parted) mkpart primary ext2 513MiB 99%

Create and open encrypted container on system partition

# cryptsetup luksFormat --type luks2 -v -s 512 /dev/sdx2
# cryptsetup open /dev/sdx2 cryptroot

Create the ZFS pool and base datasets onto encrypted container

canmount=off makes it so actual data is stored inside the parent dataset; but the hierarchical structure exists for possible children.

Double check before you run any of these; not all are necessarily required either

# zpool create -o ashift=12 -o cachefile=/etc/zfs/zpool.cache -m none -R /mnt zroot /dev/mapper/cryptroot
# zfs create -o mountpoint=none -o compression=lz4 zroot/ROOT
# zfs create -o mountpoint=/ zroot/ROOT/default
# zfs create -o canmount=off -o mountpoint=/var -o xattr=sa zroot/ROOT/var
# zfs create -o canmount=off -o mountpoint=/var/lib zroot/ROOT/var/lib
# zfs create -o canmount=off -o mountpoint=/var/lib/systemd zroot/ROOT/var/lib/systemd
# zfs create -o canmount=off -o mountpoint=/usr zroot/ROOT/usr
# zfs create -o mountpoint=/opt zroot/ROOT/opt
# zfs create -o mountpoint=/var/lib/systemd/coredump zroot/ROOT/var/lib/systemd/coredump
# zfs create -o mountpoint=/var/log zroot/ROOT/var/log
# zfs create -o acltype=posixacl -o mountpoint=/var/log/journal zroot/ROOT/var/log/journal
# zfs create -o mountpoint=/home zroot/home
# zfs create -o mountpoint=/root zroot/home/root
# zfs create -o mountpoint=/home/<username> zroot/home/<username>
# zpool set bootfs=zroot zroot
# zfs set relatime=on zroot
# zfs set compression=lz4 zroot

Create swap zvol

# zfs create -V 16G -b 4096 -o logbias=throughput -o sync=always -o primarycache=metadata -o com.sun:auto-snapshot=false zroot/swap
# mkswap -f /dev/zvol/zroot/swap

Export and import (and mount) the new ZFS filesystem into /mnt

# zpool export zroot
# zpool import -R /mnt zroot

Mount /boot into /mnt/boot

# mkdir /mnt/boot
# mount /dev/sdx1 /mnt/boot

Install base Arch Linux system into /mnt

# pacstrap -i /mnt base base-devel

Generate /boot mount options and pipe them into fstab

# loadkeys us
# genfstab -U -p /mnt | grep boot >> /mnt/etc/fstab
# loadkeys be-latin1

Add the swap volume to fstab

# vim /mnt/etc/fstab
--------------------
/dev/zvol/zroot/swap none swap discard 0 0

Chroot into the Arch installation on /mnt

# arch-chroot /mnt /bin/bash

Set locale

# vi /etc/locale.gen
--------------------
Uncomment en_US.UTF-8 UTF-8

# locale-gen

# vi /etc/locale.conf
---------------------
LANG=en_US.UTF-8

Set timezone and time

# rm /etc/localtime
# ln -s /usr/share/zoneinfo/Europe/Brussels /etc/localtime
# hwclock --systohc --utc
# pacman -S ntp
# ntpd -q
# hwclock -w

Install vim

# pacman -S vim

Set console font and keymap more permanently

# pacman -S terminus-font
# vim /etc/vconsole.conf
------------------------
KEYMAP=be-latin1
FONT=ter-112n

Add the ZFS repository to pacman configuration in the actual install too

# vim /etc/pacman.conf
----------------------
[archzfs]
Server = http://archzfs.com/$repo/$arch

Add the ZFS repository gpg keys to pacman's keyring

# pacman-key -r 0ee7a126
# pacman-key --lsign-key 0ee7a126

Update pacman's mirrorlist and prioritize closer mirrors

# vim /etc/pacman.d/mirrorlist
------------------------------
Move i3d.net up to top (with https)

Update package repositories

# pacman -Syu

Install ZFS into new install

# pacman -S linux-headers zfs-dkms spl-dkms

Enable the ZFS system services

# systemctl enable zfs.target
# systemctl enable zfs-import-cache
# systemctl enable zfs-mount

Set max ZFS arc memory usage to 2GB

# vim /etc/modprobe.d/zfs.conf
------------------------------
options zfs zfs_arc_max="2147483648"

Update mkinitcpio hooks to include LUKS disk encryption and ZFS

# vim /etc/mkinitcpio.conf
--------------------------
HOOKS=(base udev autodetect modconf keyboard keymap consolefont block encrypt zfs filesystems)

Regenerate kernel image with the new hooks

# mkinitcpio -p linux

Set root user password

# passwd

Set machine hostname

# vim /etc/hostname
-------------------
<hostname>

Install bootloader

# bootctl --path=/boot install

# vim /boot/loader/loader.conf
------------------------------
default arch
timeout 4
editor 0

Temporarily save system partition UUID into a file for easy copying in vim

# loadkeys us
# blkid /dev/sdx2 > partitionid.txt
# loadkeys be-latin1

Create the main boot entry (add UUID from partitionid.txt)

# vim /boot/loader/entries/arch.conf
------------------------------------
title Arch Linux
linux /vmlinuz-linux
initrd /initramfs-linux.img
options cryptdevice=UUID=<UUID>:cryptroot:allow-discards zfs=zroot/ROOT/default rw pcie_aspm=force

Create fallback boot entry (add UUID from partitionid.txt)

# vim /boot/loader/entries/arch-fallback.conf
---------------------------------------------
title Arch Linux Fallback
linux /vmlinuz-linux
initrd /initramfs-linux-fallback.img
options cryptdevice=UUID=<UUID>:cryptroot:allow-discards zfs=zroot/ROOT/default rw

Install some other necessary packages for network, etc, and enable their services

# pacman -S networkmanager openssh tlp smartmontools ethtool networkmanager-openvpn x86_energy_perf_policy
# systemctl enable NetworkManager
# systemctl enable tlp
# systemctl enable tlp-sleep
# systemctl enable fstrim.timer

Fix XPS 13 9343 sound (otherwise linux loads the HDMI sound driver)

# vim /etc/modprobe.d/alsa-base.conf
------------------------------------
options snd_hda_intel index=1,0

Leave chroot environment

# exit

Copy the ZFS cache file into system

# cp /etc/zfs/zpool.cache /mnt/etc/zfs/

Unmount /boot and export the ZFS filesystem

# umount /mnt/boot
# zpool export zroot

Reboot into new install

# reboot

Post-installation

Set up NetworkManager wifi connection

# nmcli d wifi list
# nmcli dev wifi connect <SSID> password <password>
# ping google.com

Install zsh (optional) and create user account

# pacman -S zsh
# useradd -m -G wheel -s /bin/zsh <username>
# zfs allow <username> create,mount,mountpoint,snapshot zroot/home/<username>
# chown -R <username> /home/<username>

Set password for new user

# passwd <username>

Allow users in wheel group to use sudo

# visudo
--------
Uncomment %wheel ALL=(ALL) ALL

Install xorg and other useful packages

# pacman -S rsync xorg-server xf86-video-intel xorg-xinit xorg-xrandr i3 pulseaudio pavucontrol termite conky git libyaml gcc6 imagemagick xorg-xbacklight mpv notify-osd ttf-inconsolata ttf-bitstream-vera ttf-dejavu ttf-droid ttf-fira-mono ttf-fira-sans ttf-freefont ttf-liberation ttf-roboto ttf-ubuntu-font-family

Log out as root and log back in as user

# exit

Copy system xinitrc to home directory and run i3 instead

$ cp /etc/X11/xinit/xinitrc .xinitrc

$ vim .xinitrc
--------------
exec i3

Fix HiDPI scaling

$ gsettings set org.gnome.desktop.interface scaling-factor 2

$ vim .Xresources
-----------------
Xft.dpi: 166
Xft.autohint: false
Xft.hintstyle: hintslight
Xft.lcdfilter: lcddefault
Xft.hinting: true
Xft.antialias: true
Xft.rgba: rgb

Start X11

$ startx

Start a terminal with Alt+Enter

Set keyboard layout

$ setxkbmap be

etc.

@kraem
Copy link

kraem commented Apr 16, 2019 •
edited

Great! Thank you so much!

I forked it and headers with info on; creating your own archiso to have the latest kernel when installing zfs-dkms, couldn't get your keys to work so changed them to the official ones on the ZOL github, formatting boot partition, using reflector package to update the mirrors.

I'm stuck at the last step though. I get systemd boot to launch and display my "Arch Linux" entry, but nothing happens when choosing it. Would you know what that's about? Added the hooks in mkinitcpio and rebuilt the kernel.

EDIT Did "solve" the issue. Everything starts up normally but the screen is black. If i type in the LUKS-password blindly and press enter, everything boots OK after.

Again; Thank you for the detailed instructions!

@ArchGalileu
Copy link

Hello

I have my system like this howto, great howto ;)

I want to add +1 disk 1Tb.... how to do it? as another pool, only for data, video editing data.

Any advices?

Best regards
ArchGalileu

@kdwinter
Copy link
Author

Hello

I have my system like this howto, great howto ;)

I want to add +1 disk 1Tb.... how to do it? as another pool, only for data, video editing data.

Any advices?

Hello,

It should be fairly straightforward to create a new pool with this disk. First figure out the ID of your disk in /dev/disk/by-id/, and then create a pool with it:

zpool create -f -o ashift=12 -o cachefile=/etc/zfs/zpool.cache -m none storage ata-WDC_WD30EFRX-......etc
zfs set relatime=on storage
zfs set compression=lz4 storage

If you want to encrypt it you could either first create a new LUKS device and create the pool on that instead, or use native ZFS encryption.
Just make sure to rebuild your initramfs after you're done so that the updated /etc/zfs/zpool.cache file is part of it on next boot.

@ArchGalileu
Copy link

Thank you ;)

Best regards
ArchGalileu

@ArchGalileu
Copy link

Hello kdwinter

My primary drive is an M.2 nvme 512Gb (1800Mb write)

The added one, 1Tb ssd, usual 450Mb write

When I'm copying from USB or sdcard big files, from 6Gb, I also need to copy between 50Gb and 200Gb, it freezes my laptop entirely;

Any advice?

I have read, and read, all over but with no success.

I have 8Gb Ram

Ps. I encrypted the 1Tb with zfs_encryption but can't mount at boot, I mount it manually ;)

Best regards
ArchGalileu

@ArchGalileu
Copy link

Solved ;)

I disabled SWAP and then voila ;)

Best regards
ArchGalileu

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment