Skip to content

Instantly share code, notes, and snippets.

@ladinu
Last active December 8, 2024 12:43
Show Gist options
  • Save ladinu/bfebdd90a5afd45dec811296016b2a3f to your computer and use it in GitHub Desktop.
Save ladinu/bfebdd90a5afd45dec811296016b2a3f to your computer and use it in GitHub Desktop.
NixOS install with encrypted /boot /root with single password unlock

Requirements

  1. Encrypt everthing including /boot and /root
  2. Enter password once
  3. Support UEFI

Installation media setup

Download NixOS minimal iso and copy to USB stick. For example on Mac OSX

$ diskutil list
$ diskutil unmountDisk /dev/disk1 # Make sure you got right device
$ dd if=nixos-minimal-20.09.2405.e065200fc90-x86_64-linux.iso of=/dev/disk1

NixOS install

Boot from the USB stick and setup networking. (optionally setup SSH if you want to complete the install from another computer)

$ wpa_passphrase SSID 'PASSWORD' > /etc/wpa_supplicant.conf
$ systemctl start wpa_supplicant
$ systemctl start sshd
$ passwd # So we can login via SSH

Partitioning

I have 2 drives on my system: NVME SSD and a regular SATA drive. This is what the drives should look like after install. I use NVME SSD for booting and SATA drive as a /data mount.

NAME             MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                8:0    0 931.5G  0 disk
└─sda1             8:1    0 931.5G  0 part
  └─crypted-data 254:3    0 931.5G  0 crypt /data
nvme0n1          259:0    0 465.8G  0 disk
├─nvme0n1p1      259:1    0   549M  0 part  /boot/efi
└─nvme0n1p2      259:2    0 465.2G  0 part
  └─root         254:0    0 465.2G  0 crypt
    ├─vg-swap    254:1    0     4G  0 lvm   [SWAP]
    └─vg-root    254:2    0 461.2G  0 lvm   /

The only unecrypted partition is nvme0n1p1, which is mounted on /boot/efi. But rest of /boot is encrypted along with swap and root. /dev/sda1 is also encrypted and mounted as /data (Note: If you plan on extending /data with more drives, you should do LUKS-on-LVM )

Use Use gdisk to partition the drives

$ gdisk /dev/nvme0n1
  • o Create partition table
  • n Create new partition of size 550M and of type ef00
  • n Create another partition of type 8300 and use remainig space
  • p Show what gdisk will write
  • w Write to disk an exit
$ gdisk /dev/sda
  • o Create partition table
  • n Create another partition of type 8300 and use all space
  • p Show what gdisk will write
  • w Write to disk an exit

Generate keys for single password unlock

$ dd if=/dev/urandom of=./keyfile0.bin bs=1024 count=4
$ dd if=/dev/urandom of=./keyfile1.bin bs=1024 count=4

Setup LUKS and add the keys

# Enter the passphrase which is used to unlock disk. You will enter this in grub on every boot
$ cryptsetup luksFormat --type luks1 -c aes-xts-plain64 -s 256 -h sha512 /dev/nvme0n1p2

# Add a second key which will be used by nixos. You will need to enter the pasphrase from previous step
$ cryptsetup luksAddKey /dev/nvme0n1p2 keyfile0.bin
$ cryptsetup luksOpen /dev/nvme0n1p2 crypted-nixos -d keyfile0.bin

# For /data just the second random generated key
$ cryptsetup luksFormat -c aes-xts-plain64 -s 256 -h sha512 /dev/sda1 -d keyfile1.bin
$ cryptsetup luksOpen /dev/sda1 crypted-data -d keyfile1.bin

Setup LVM

$ pvcreate /dev/mapper/crypted-nixos
$ vgcreate vg /dev/mapper/crypted-nixos
$ lvcreate -L 4G -n swap vg
$ lvcreate -l '100%FREE' -n root vg

Format the partitions and mount

$ mkfs.fat -F 32 /dev/nvme0n1p1
$ mkswap -L swap /dev/vg/swap
$ mkfs.ext4 -L root /dev/vg/root
$ mkfs.ext4 -L data /dev/mapper/crypted-data
$ mount /dev/vg/root /mnt
$ mkdir -p /mnt/boot/efi
$ mount /dev/nvme0n1p1 /mnt/boot/efi
$ swapon /dev/vg/swap

Copy the keys

$ mkdir -p /mnt/etc/secrets/initrd/
$ cp keyfile0.bin keyfile1.bin /mnt/etc/secrets/initrd
$ chmod 000 /mnt/etc/secrets/initrd/keyfile*.bin

Generate and edit configuration

$ nixos-generate-config --root /mnt

Add the following to /mnt/etc/nixos/configuration.nix

  boot.loader.efi.canTouchEfiVariables = true;
  boot.loader.efi.efiSysMountPoint = "/boot/efi";
  boot.loader.grub = {
    enable = true;
    device = "nodev";
    version = 2;
    efiSupport = true;
    enableCryptodisk = true;
  };
  
  boot.initrd = {
    luks.devices."root" = {
      device = "/dev/disk/by-uuid/a8b302cf-5296-4a2e-a7ba-707e6fa75123"; # UUID for /dev/nvme01np2 
      preLVM = true;
      keyFile = "/keyfile0.bin";
      allowDiscards = true;
    };
    secrets = {
      # Create /mnt/etc/secrets/initrd directory and copy keys to it
      "keyfile0.bin" = "/etc/secrets/initrd/keyfile0.bin";
      "keyfile1.bin" = "/etc/secrets/initrd/keyfile1.bin";
    };
};

# Data mount
fileSystems."/data" = {
  device = "/dev/disk/by-uuid/79630267-5766-4c7d-85a5-1d5f1dcd58ad"; # UUID for /dev/mapper/crypted-data
  encrypted = {
    enable = true;
    label = "crypted-data";
    blkDev = "/dev/disk/by-uuid/3476cb09-b3c4-4301-9ec9-84f60f32828a"; # UUID for /dev/sda1
    keyFile = "/keyfile1.bin";
  };
};
  

You can get the UUIDs by running

$ blkid

Install NixOS and reboot

$ nixos-install
$ reboot

Thats it! Once you reboot, GRUB will ask for the password. If password is correct, GRUB will show you the NixOS system profiles menu. After that, your system will boot without asking for the disk password.

Future work

If I enter an incorrect disk password, GRUB does not handle it gracefully. It will drop me into a shell without re-prompting for a password. I have to run the following commands for it to prompt again. Need to figure out if GRUB has an option for it to re-prompt.

cryptomount hdX,gptY    # Device to mount: drive X, GPT partition Y, this forces the re-prompt.
insmod normal           # Load the normal mode boot module.
normal                  # Enter normal mode and display the GRUB menu.

Credits

@ElvishJerricco
Copy link

ElvishJerricco commented Sep 29, 2018

  extraInitrd = /boot/initrd.keys.gz;

I believe the use of path literal syntax will cause initrd.keys.gz to be placed in your nix store, allowing all users to read the file. Just putting the path in a string literal will have the same functionality but won't put it in the nix store.

Anyway thanks for this. Very useful!

@whoizit
Copy link

whoizit commented Apr 6, 2019

you should not do LVM-on-LUKS for /data disk, cause you can extend your /data disks array with another disks (LVM spanning) only with LUKS-on-LVM
https://wiki.archlinux.org/index.php/Dm-crypt/Encrypting_an_entire_system#LUKS_on_LVM
https://wiki.archlinux.org/index.php/Dm-crypt/Encrypting_an_entire_system#LVM_on_LUKS

@starfys
Copy link

starfys commented Nov 25, 2019

Ran into some issues getting this working recently, because cryptsetup now defaults to LUKS2 when formatting, which is not compatible with GRUB. The fix is to modify the format line for the root partition as follows:

cryptsetup luksFormat --type luks1 -c aes-xts-plain64 -s 256 -h sha512 /dev/nvme0n1p2

The /data partition should not need this

@WolfangAukang
Copy link

Hey, just wanted to provide some updates (Dec 2020) to anyone that came to this guide:

  • First, @starfys comment is totally accurate. Remember to specify the --type luks1 when encrypting the partition with root, as GRUB still does not support luks2, which is the default type used by cryptsetup.
  • Second, to avoid putting the password too many times, you can mount the encrypted partition with cryptsetup luksOpen /dev/sda1 -d keyfile1.bin crypted-name. Just remember to use the corresponding key.
  • The initrd part is obsolete. Now NixOS uses the boot.initrd.secrets section for this. So, for example, if I try to set up this guide with this new setting (plus other minimal fixes), it will look like this:
boot.initrd = {
  luks.devices."root" = {
    device = "/dev/disk/by-uuid/a8b302cf-5296-4a2e-a7ba-707e6fa75123"; # UUID for /dev/nvme01np2 
    preLVM = true;
    keyFile = "/keyfile0.bin";
    allowDiscards = true;
  };
  secrets = {
    # Create /mnt/etc/secrets/initrd directory and copy keys to it
    "keyfile0.bin" = "/etc/secrets/initrd/keyfile0.bin";
    "keyfile1.bin" = "/etc/secrets/initrd/keyfile1.bin";
  };
};

# Data mount
fileSystems."/data" = {
  device = "/dev/disk/by-uuid/79630267-5766-4c7d-85a5-1d5f1dcd58ad"; # UUID for /dev/mapper/crypted-data
  encrypted = {
    enable = true;
    label = "crypted-data";
    blkDev = "/dev/disk/by-uuid/3476cb09-b3c4-4301-9ec9-84f60f32828a"; # UUID for /dev/sda1
    keyFile = "/keyfile1.bin";
  };
};

@bryanasdev000
Copy link

Hey, just wanted to provide some updates (Dec 2020) to anyone that came to this guide:

  • First, @starfys comment is totally accurate. Remember to specify the --type luks1 when encrypting the partition with root, as GRUB still does not support luks2, which is the default type used by cryptsetup.
  • Second, to avoid putting the password too many times, you can mount the encrypted partition with cryptsetup luksOpen /dev/sda1 -d keyfile1.bin crypted-name. Just remember to use the corresponding key.
  • The initrd part is obsolete. Now NixOS uses the boot.initrd.secrets section for this. So, for example, if I try to set up this guide with this new setting (plus other minimal fixes), it will look like this:
boot.initrd = {
  luks.devices."root" = {
    device = "/dev/disk/by-uuid/a8b302cf-5296-4a2e-a7ba-707e6fa75123"; # UUID for /dev/nvme01np2 
    preLVM = true;
    keyFile = "/keyfile0.bin";
    allowDiscards = true;
  };
  secrets = {
    # Create /mnt/etc/secrets/initrd directory and copy keys to it
    "keyfile0.bin" = "/etc/secrets/initrd/keyfile0.bin";
    "keyfile1.bin" = "/etc/secrets/initrd/keyfile1.bin";
  };
};

# Data mount
fileSystems."/data" = {
  device = "/dev/disk/by-uuid/79630267-5766-4c7d-85a5-1d5f1dcd58ad"; # UUID for /dev/mapper/crypted-data
  encrypted = {
    enable = true;
    label = "crypted-data";
    blkDev = "/dev/disk/by-uuid/3476cb09-b3c4-4301-9ec9-84f60f32828a"; # UUID for /dev/sda1
    keyFile = "/keyfile1.bin";
  };
};

Thanks! Worked like a charm with ZFS native encryption.

@ladinu
Copy link
Author

ladinu commented Jan 2, 2021

I've updated the doc. Thank you everyone for the feedback!

@YellowOnion
Copy link

grub merged LUKS2 support in Jan 2020, It should work in theory.

@bryanasdev000
Copy link

grub merged LUKS2 support in Jan 2020, It should work in theory.

Thanks for the heads up!

Here is ref:

https://git.savannah.gnu.org/cgit/grub.git/log/

https://git.savannah.gnu.org/cgit/grub.git/commit/?id=ec46685ed4ea7bf0dd81c8254c2d9e0e8514976c (last commit with luks2 message)

@ja0nz
Copy link

ja0nz commented Mar 21, 2021

grub merged LUKS2 support in Jan 2020, It should work in theory.

Yet at time of writing GRUB is still in version 2.04 from 2019! So no, LUKS2 is not working at the moment.
The next major release of GRUB 2.06 is expected in April 2021 and hopefully lands of NixOS soon after.

@maralorn
Copy link

maralorn commented Jul 4, 2021

I just wanted to share that we have now grub 2.06~rc1 in nixos stable. Sadly booting from a luks2 header failed for me without helpful error message. (It just doesn‘t decrypt on boot.) With luks1 header this guide works fine.

EDIT: Maybe this is because I had an argon2i header on the disk, but the key I wanted grub to unlock was pkdf2.

@Dom-R
Copy link

Dom-R commented Aug 30, 2021

Tried with luks2 on 21.05 and grub still fails.

Btw, what do you think about switching on the readme the path to the configuration file to /mnt/etc/nixos/configuration.nix? I got confused and ended up actually changing /etc instead of /mnt/etc.

Let's just say that I spent a stupid amount of time trying to fix an error caused by my stupidity. 🤦

@ladinu
Copy link
Author

ladinu commented Sep 4, 2021

Good suggestion @Dom-R. Updated the doc

@WolfangAukang
Copy link

FYI, for the people that is following this guide and using the 21.11 image as of now (Dec 2021), you might find with this issue. In summary, the error mktemp: failed to create directory via template ‘/mnt/tmp.kYftcsVwDN/initrd-secrets.XXXXXXXXXX’: No such file or directory might show up.

Just run nixos-install normally, then (as nixos user) run sudo nixos-enter and finally nixos-install --root /. There might be some warnings, but as long as it asks you for a new password, everything is fine (or at least it worked for me).

Finally, one addition to the guide if someone has no problem in using systemd-boot instead of grub:

boot.loader = {
  efi.canTouchEfiVariables = true;
  systemd-boot = {
    enable = true;
    configurationLimit = 20; # You can leave it null for no limit, but it is not recommended, as it can fill your boot partition.
  };
};

@vroad
Copy link

vroad commented Mar 28, 2022

I had to use /dev/disk/by-partuuid/UUID instead of /dev/disk/by-uuid/UUID, in my case LUKS partition created on external SSD didn't show up in /dev/disk/by-uuid/.
I was able to install NixOS 21.11, using the same workaround @WolfangAukang mentioned.

@WolfangAukang
Copy link

@sarveshrulz this is weird. Last weekend I ran a nixos-install from a flake, using this guide, and it worked normally. Here is my config if you want a reference.

@sarveshrulz
Copy link

@sarveshrulz this is weird. Last weekend I ran a nixos-install from a flake, using this guide, and it worked normally. Here is my config if you want a reference.

My bad, it was cuz my initrd was lz4, sorry and thank you

@leg7
Copy link

leg7 commented Apr 10, 2023

@WolfangAukang afaik systemd boot can't work with encrypted /boot so following this guide for systemd-boot would be terrible because your keyfiles will reside unencrypted in /boot making FDE completely useless.

@WolfangAukang
Copy link

@WolfangAukang afaik systemd boot can't work with encrypted /boot so following this guide for systemd-boot would be terrible because your keyfiles will reside unencrypted in /boot making FDE completely useless.

I've only provided an alternative as there has been people reluctant in using GRUB, but it is their responsibility in investigating and knowing the consequences before doing this. The guide is originally using GRUB.

@leg7
Copy link

leg7 commented Apr 20, 2023

@WolfangAukang afaik systemd boot can't work with encrypted /boot so following this guide for systemd-boot would be terrible because your keyfiles will reside unencrypted in /boot making FDE completely useless.

I've only provided an alternative as there has been people reluctant in using GRUB, but it is their responsibility in investigating and knowing the consequences before doing this. The guide is originally using GRUB.

I understand but in this case it is not an alternative and IMO it's dangerous to suggest using it since it could lead people into believing they have FDE when in reality their keyfile is sitting unencrypted in /boot. I think you should add this information as a disclaimer in the guide to keep people safe.

@IQ8QI
Copy link

IQ8QI commented Mar 1, 2024

There is no grub1 in NixOS anymore?
I got an error that:
boot.loader.grub.version does not have any effect anymore, please remove it from your configuration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment