NixOS root partition RAID1 using mdadm + LUKS cryptsetup encryption

README.md

1. Boot into NixOS Live USB

Insert your NixOS live USB and boot into the system.

2. Partition disks

Note: This guide assumes you're using UEFI. If you're using BIOS, the partitioning scheme will differ.

You can use cfdisk to manually partition your disks.

cfdisk /dev/sda
cfdisk /dev/sdb

1. Create a boot partition and set the type to EFI System.
2. Allocate the remaining disk space for a Linux RAID partition.

3. Format EFI Partitions

Note: UEFI doesn't support booting from RAID. We'll designate one drive as the "primary" boot drive that holds the actual EFI files. The secondary drive can be easily configured to become the new primary in case of failure.

Format the EFI partitions and label the primary one:

mkfs.fat -F32 -n BOOT /dev/sda1
mkfs.fat -F32         /dev/sdb1

4. Setup mdadm array

Create an array using the RAID partitions:

mdadm --create --verbose --level=1 --raid-devices=2 /dev/md0 /dev/sda2 /dev/sdb2

5. Setup encryption with LUKS

cryptsetup luksFormat /dev/md0
cryptsetup open       /dev/md0 nixos

6. Format the root parition

mkfs.ext4 -L nixos /dev/mapper/nixos

7. Mount the root parititon

mount /dev/mapper/nixos /mnt

8. Continue with typical NixOS installation steps

Proceed until you're about to run nixos-install.

9. Dump mdadm configuration

mdadm --detail --scan --verbose

You should see output like:

ARRAY /dev/md0 level=raid1 num-devices=2 metadata=1.2 name=nixos:0 UUID=... devices=/dev/sda2,/dev/sdb2

10. Edit configuration.nix / hardware-configuration.nix

Note: The following instructions are based on NixOS 23.05. Future releases might change some of these options (e.g., boot.initrd.services.swraid will be renamed to boot.swraid).

  # systemd stage-1 loader (required for swraid)
  boot.initrd.systemd.enable = true;

  # Software RAID
  boot.initrd.services.swraid = {
    enable = true;
    
    # Replace with the output from step 9.
    mdadmConf = ''
      ARRAY /dev/md0 level=raid1 num-devices=2 metadata=1.2 name=nixos:0 UUID=... devices=/dev/sda2,/dev/sdb2
    '';
  };

  # LUKS disk encryption
  boot.initrd.luks.devices."nixos".device = "/dev/md0";

  # File system mounts
  fileSystems = {
    "/" = {
      device = "/dev/disk/by-label/nixos";
      fsType = "ext4";
    };
    "/boot/efi" = {
      device = "/dev/disk/by-label/BOOT";
      fsType = "vfat";
    };
  };

11. Install NixOS

Run nixos-install and finish the installation.

---

Your NixOS system should now be set up with RAID1 and LUKS encryption.

whats also useful is

    postDeviceCommands = ''
      mkfifo /tmp/continue
      cat /tmp/continue
    '';

if you are unlocking via ssh with headers placed on a remote location or just want to essentially halt the boot, issue some commands via ssh and upon doing

echo blablastuff > /tmp/continue

let the boot continue after the device commands

note: you may want to set forceLuksSupportInInitrd = true; if you wish to have a complex unlock scheme over ssh thats not just uploading a keyfile. That way you can in combination with the commands above just ensure that your /dev/root is present after executing a bunch of ssh commands without nix failing to "locate" anything. Its hacky but i wanted to share it so someone may benefit from it and not spend hours debugging this like me