kongkrit/void-install.md

## void-install.md

      
    Raw
  

              void-install.md
            
          
    Void Linux Root on ZFS Installation - glibc version

Part 1: Create a ZFS-Capable Installation

Create a void installation medium with ZFS (either a virtual hard disk or a USB stick)

Download ISO from here


choose void-live-x86_64-YYYYMMDD.iso - as of this writing (2020-03-20) the newest one is void-live-x86_64-20191109.iso
tested on VirtualBox 6.1 with the folllowing parameters:

system: chipset: ICH9, enable EFI, RAM: 2GB, 2 CPUs, enable nested VT-x/AMD-V
3 SATA hard disks hanging of AHCI controller - 2 for mirror and the other 10GB for temp installation
IDE-CD-Rom on IDE PIIX4 controller.
bridged network adaptor Intel PRO/1000 MT Desktopm, Promiscuous Mode: Allow VMs


Best to remove all hard disks, except the installation target
Boot the default option, login as root password voidlinux
start bash bash
refresh xbps repo:
xbps-install -Sy


update void installer and install gpt disk utils:
xbps-install -y void-installer gptfdisk


find the USB drive:
fdisk -l       # list all drives


and zap it (all data will be gone)
sgdisk --zap-all /dev/sdc # <-- here /dev/sdc is the USB hard disk


run void-installer
void-installer


keyboard: us
network: select interfaces and use dhcp
source: network
hostname: void-zfs
locale: en_US.UTF-8
timezone: Asia/Bangkok
rootpassword: whatever you want
useraccount: void with whatever password you want

note the groups: wheel, floppy, audio, video, cdrom, optical, kvm, xbuilder


bootloader: select the temp disk/USB disk, no graphical terminal
partition: choose the temp disk/USB disk: label-type gpt

new partition: size 512M type: EFI System
new partition: size (whatever is left) type: Linux filesystem
choose write and confirm writing, then quit


filesystem:

choose first 512M partition and use vfat FAT32 filesystem, mounted at /boot/efi
choose the second partition and use 'ext4filesystem, mounted at/`


Install. Wait a while...
Exit (do not reboot) the installer.


start sshd and SSH into the machine
sv start sshd
ip addr show


SSH in as user anon with password voidlinux and become root
sudo bash


mount /dev /proc sys / and /boot/efi from the installation
fdisk -l  # check the device & partition numbers (like /dev/sdc1 & /dev/sdc2)
rm -rf /mnt/void
mkdir -p /mnt/void/{dev,proc,sys,boot/efi}
mount /dev/sda1 /mnt/void/boot/efi
mount /dev/sda2 /mnt/void
mount --rbind --make-rslave /dev /mnt/void/dev
mount --rbind --make-rslave /proc /mnt/void/proc
mount --rbind --make-rslave /sys /mnt/void/sys


chroot into installation
chroot /mnt/void /bin/bash


check the mount points:
mount | grep -E "^/dev/sd"

output should have 2 lines, as follows:

/dev/sda2 on / type ext4 (rw,relatime)
/dev/sda1 on /boot/efi type vfat (rw,relatime,...


if there is any partition missing, mount it:
if /dev/sda1 is missing: mount /dev/sda1 /boot/efi
if /dev/sda2 is missing: mount /dev/sda2 /


Install grub for removable drive. The key is --removable flag for grub-install
BOOTLOADER_ID=$(ls /boot/efi/EFI/ || echo GRUB)
echo $BOOTLOADER_ID
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=$BOOTLOADER_ID --removable
grub-mkconfig -o /boot/grub/grub.cfg


Check for UUID consistency of /boot/grub/grub.cfg
grep -E "[[:space:]]/boot/.*[[:space:]]root=UUID=" /boot/grub/grub.cfg

should return several lines with root=UUID=xxxxxxxxxxxx but all UUID's are the same. compare the UUID with UUID returned by
ls -l /dev/disk/by-uuid/

The UUID in /boot/grub/grub.cfg should match the UUID of the root partition of the installation.
put in generic DNS servers in /etc/resolv.conf:
cat > /etc/resolv.conf << EOF
nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 1.1.1.1
EOF


exit chroot:
exit


Reboot into this new installation:
login as root with the password you set earlier
sync repo and install openssh for ease of access:
xbps-install -S
xbps-install -y openssh


modify sshd so root can log in:
cat >> /etc/ssh/sshd_config << EOF
PermitRootLogin yes
EOF


start sshd service
ln -s /etc/sv/sshd /var/service
sv up sshd
ip addr show # show the ip address to SSH in


Now you can ssh in the box and copy & paste easier: login as root and your password
bash                # something less painful than sh
xbps-install -y zfs # put ZFS on this new installation


check the zfs installation
modprobe zfs
lsmod | grep zfs
zfs --version


install gpt disk utils and nano:
xbps-install -y gptfdisk nano


change default shell for root to bash:
usermod -s /bin/bash root


prolong life of USB disk

by making mounted filesystems with noatime:

sed -i -E 's/^(UUID.*)defaults(.*)$/\1noatime\2/g' /etc/fstab


by mounting /tmp /var/tmp /var/log on tmpfs:

cat >> /etc/fstab << EOF
tmpfs /var/tmp tmpfs defaults,nosuid,nodev 0 0
tmpfs /var/log tmpfs defaults,nosuid,nodev 0 0
EOF


make sudo run without typing in password:
sed -E 's|^#?[ \t]*(\%wheel ALL=\(ALL\) ALL.*)$|#\1|g' /etc/sudoers | EDITOR=tee visudo
sed -E 's|^#?[ \t]*(\%wheel ALL=\(ALL\) NOPASSWD.*)$|\1|g' /etc/sudoers | EDITOR=tee visudo


make note for users when boot
cat >> /etc/issue << EOF
Void-ZFS-Installer
------------------
Login as root:<yourRootPasswordHere>
      or void:<yourVoidPasswordHere>

EOF


poweroff. this media is ready for the next step
poweroff


Void Linux Root on ZFS Installation - glibc version

Part 2: Use Void Linux Created from Part 1 to Install ZFS on root


boot from void linux installation generated from part 1
log in as root and use password set from installation
get the ip address
ip addr show


now we can go in via SSH and do copy & paste! SSH in and login as root
check that zfs module is running:
modprobe zfs     # should return nothing
lsmod | grep zfs # should return many modules, especially zfs and spl
zfs --version    # should return version >= 0.8.3 as of 2020-03-20


update repo
xbps-install -S


list disks by id ls -lF /dev/disk/by-id and look for the target hard disks

lrwxrwxrwx 1 root root  9 Mar 20 21:01 ata-VBOX_HARDDISK_VB33cf0a30-97d0c313 -> ../../sdb
lrwxrwxrwx 1 root root  9 Mar 20 21:01 ata-VBOX_HARDDISK_VBb1ae8a4d-06eb7d1d -> ../../sda


We will use /dev/sda id as $DISK1 and /dev/sdb id as $DISK2 for the following example:
DISK1=/dev/disk/by-id/ata-VBOX_HARDDISK_VBb1ae8a4d-06eb7d1d
DISK2=/dev/disk/by-id/ata-VBOX_HARDDISK_VB33cf0a30-97d0c313


wipe both disks (sgdisk installed by gptfdisk package):
sgdisk --zap-all $DISK1 ; udevadm settle
sgdisk --zap-all $DISK2 ; udevadm settle

each command should return:

Creating new GPT entries in memory.
GPT data structures destroyed! You may now partition the disk using fdisk or other utilities. 


create UEFI partition for booting (there's a report can't find reference that UEFI partition < 512M causes problem with some motherboard:
sgdisk -n1:1M:+512M -t1:EF00 $DISK1
udevadm settle
mkfs.fat -F 32 -n EFI ${DISK1}-part1
udevadm settle
sgdisk -n1:1M:+512M -t1:EF00 $DISK2
udevadm settle
mkfs.fat -F 32 -n EFI ${DISK2}-part1
udevadm settle


create ZFS partitions for boot:
sgdisk -n2:0:+1G -t2:BF00 $DISK1
sgdisk -n2:0:+1G -t2:BF00 $DISK2
udevadm settle


create ZFS partition for root:
sgdisk -n3:0:0 -t3:BF00 $DISK1
sgdisk -n3:0:0 -t3:BF00 $DISK2
udevadm settle


ZFS pool names cannot contain - because of this Dracut bug
create boot pool (bpool) with features incompatible for grub disabled. (see step 2.4) for parameters compatible with GRUB:
BPOOL=bpool # must not contain -
zpool create -o ashift=12 -d \
  -o feature@async_destroy=enabled \
  -o feature@bookmarks=enabled \
  -o feature@embedded_data=enabled \
  -o feature@empty_bpobj=enabled \
  -o feature@enabled_txg=enabled \
  -o feature@extensible_dataset=enabled \
  -o feature@filesystem_limits=enabled \
  -o feature@hole_birth=enabled \
  -o feature@large_blocks=enabled \
  -o feature@lz4_compress=enabled \
  -o feature@spacemap_histogram=enabled \
  -o feature@userobj_accounting=enabled \
  -o feature@zpool_checkpoint=enabled \
  -o feature@spacemap_v2=enabled \
  -o feature@project_quota=enabled \
  -o feature@resilver_defer=enabled \
  -o feature@allocation_classes=enabled \
  -O acltype=posixacl -O canmount=off -O compression=lz4 -O devices=off \
  -O normalization=formD -O atime=off -O xattr=sa \
  -O mountpoint=none -R /mnt -f $BPOOL mirror ${DISK1}-part2 ${DISK2}-part2


create root pool (rpool) (use ashift=13 for Samsung SSD):

note, there's a choice between -O relatime=on or -O atime=off here. Read more about it [ here | here | here ]

RPOOL=rpool # must not contain -
zpool create -o ashift=12 \
  -O acltype=posixacl -O canmount=off -O compression=lz4 \
  -O dnodesize=auto -O normalization=formD -O atime=off -O xattr=sa \
  -O mountpoint=none -R /mnt \
  -f $RPOOL mirror ${DISK1}-part3 ${DISK2}-part3


create containers for $BPOOL and $RPOOL
zfs create -o mountpoint=none -o canmount=off $BPOOL/BOOT
zfs create -o mountpoint=none -o canmount=off $RPOOL/ROOT


read create your datasets
zfs create -o mountpoint=/               -o atime=off $RPOOL/ROOT/default
zfs create -o mountpoint=/home           -o atime=off $RPOOL/home
zfs create -o mountpoint=/root           -o atime=off $RPOOL/home/root
zfs create -o mountpoint=/var/cache/xbps -o atime=off $RPOOL/pkgcache

#zfs create -o mountpoint=legacy -o atime=off $BPOOL/BOOT/default  
zfs create -o mountpoint=/boot -o atime=off $BPOOL/BOOT/default


Configure root filesystem
zfs unmount -a
zfs set mountpoint=/               $RPOOL/ROOT/default
zfs set mountpoint=/home           $RPOOL/home
zfs set mountpoint=/root           $RPOOL/home/root
zfs set mountpoint=/boot           $BPOOL/BOOT/default
zfs set mountpoint=/var/cache/xbps $RPOOL/pkgcache


Export pools:
zpool export $BPOOL
zpool export $RPOOL


Re-import pools:
rm -rf /mnt
zpool import -d /dev/disk/by-id -R /mnt $RPOOL
zpool import -d /dev/disk/by-id -R /mnt $BPOOL


Disable cache files:
zpool set cachefile=none $BPOOL
zpool set cachefile=none $RPOOL


make efi mount points and mount them:
mkdir -p /mnt/{efi1,efi2}
mount -t vfat ${DISK1}-part1 /mnt/efi1
mount -t vfat ${DISK2}-part1 /mnt/efi2


check the work:
findmnt | grep /mnt  # check that /mnt, /mnt/boot, /mnt/root, /mnt/home, /mnt/ef1, /mnt/efi2 are mounted


make the /mnt folders:
mkdir -p /mnt/{boot,dev,proc,run,sys}  


mount /dev /proc /run /sys in /mnt
mount --rbind /dev /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /run /mnt/run
mount --rbind /sys /mnt/sys


Install! Can choose a different repo from this list. Add /current after the URL from the list.
xbps-install -S -R http://alpha.de.repo.voidlinux.org/current -r /mnt base-system


install some additional stuff as well here:
xbps-install -S -R http://alpha.de.repo.voidlinux.org/current -r /mnt \
  nano bash-completion wget curl htop zstd bridge-utils


save environment variables for use later, if necessary:
cat > /mnt/root/vars.sh << EOF
DISK1=$DISK1
DISK2=$DISK2
RPOOL=$RPOOL
BPOOL=$BPOOL
EOF


chroot into /mnt:
chroot /mnt /usr/bin/env DISK1=$DISK1 DISK2=$DISK2 BPOOL=$BPOOL RPOOL=$RPOOL /bin/bash


get resolv.conf up and test the network
cat > /etc/resolv.conf << EOF
nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 1.1.1.1
EOF
ping -c 3 google.com # should work now


sync repo and install ZFS
xbps-install -S
xbps-install -y zfs


get zfs bash-completion installed:
wget https://raw.githubusercontent.com/openzfs/zfs/master/contrib/bash_completion.d/zfs -O /usr/share/bash-completion/completions/zfs


set timezone: list timezones by ls -F /usr/share/zoneinfo/
rm -f /etc/localtime
ln -sf /usr/share/zoneinfo/Asia/Bangkok /etc/localtime
hwclock --systohc # generate /etc/adjtime and set HW RTC to UTC


Select NTP client, from either chrony or openntpd (choose only one)


[ comparing-all-ntp-clients ]
looks like both chrony and openntpd is are good [ void-wiki-ntp | manage-chrony ]

Set up openntpd (and don't install chrony)


set up openntpd NTP client and tune it
xbps-install -y openntpd
ln -s /etc/sv/ntpd /var/service/


tune /etc/ntpd.conf according to ntpd.conf(5)
cat >> /etc/ntpd.conf << EOF

server time.google.com
server time.windows.com
EOF


Set up chrony (don't do this if you installed openntpd)


set up chrony NTP client and tune it.
xbps-install -y chrony
ln -s /etc/sv/chronyd/ /var/service/


tune /etc/chrony.conf according to chrony.conf(5)
sed -i -E 's/^(pool[ \t]+.*)$/\1\nserver time.google.com iburst prefer\nserver time.windows.com iburst prefer/g' /etc/chrony.conf


Locales


set locale [ void-wiki-locale ]:
sed -i -E 's/^(LANG=).*$/\1en_US.UTF-8/g' /etc/locale.conf


uncomment the needed locale from /etc/default/libc-locales
sed -i -E 's/^#?[ \t]*(en_US\.UTF-8.*)$/\1/g' /etc/default/libc-locales


rebuild GNU lib locales:
xbps-reconfigure -f glibc-locales


export LANG and LC_COLLATE environment vars:
export LANG=en_US.UTF-8
export LC_COLLATE=C


set timezone (void-wiki-timezone), keymap, and font in /etc/rc.conf
# set timezone 
sed -i -E 's|#?(TIMEZONE=).*|\1"Asia\/Bangkok"|g' /etc/rc.conf
# set keymap to us
sed -i -E 's|#?(KEYMAP=).*|\1"us"|g' /etc/rc.conf
# set font only for 4k display -- see fonts in /usr/share/kbd/consolefonts/
sed -i -E 's|#?(FONT=).*|\1"latarcyrheb-sun32"|g' /etc/rc.conf


set hostname and edit /etc/hosts file:
HOSTNAME=void-zfs  # change hostname to whatever you want
echo $HOSTNAME > /etc/hostname
# sed -i -E 's|(127\.0\.0\.1).*|\1\t\tlocalhost|g' /etc/hosts
# sed -i -E 's|(::1).*|\1\t\t\tlocalhost|g' /etc/hosts
sed -i -E 's|^(::1.*)$|\1\n127.0.1.1\t\t'"$HOSTNAME"'|g' /etc/hosts


enable dhcpcd service
ln -s /etc/sv/dhcpcd /var/service/
# sv up dhcpcd
sv status dhcpcd


enable sshd service
ln -s /etc/sv/sshd /var/service


add useful aliases to /etc/skel/.bashrc
cat >> /etc/skel/.bashrc << EOF

if [[ \$USER != "root" ]]; then
  PS1="\[\e[01;32m\]\u@\h\[\e[01;34m\] \w \\\\$\[\e[0m\] "
else
  PS1="\[\e[01;31m\]\u@\h\[\e[01;34m\] \w \\\\$\[\e[0m\] "
fi
alias egrep='egrep --color=auto'
alias fgrep='fgrep --color=auto'
alias grep='grep --color=auto'
alias l='ls -CF'
alias la='ls -A'
alias ll='ls -alF'
alias ls='ls --color=auto'

export EDITOR=nano
EOF


copy /etc/skel/.bash* to /root and make root user bin/bash as shell
cp /etc/skel/.bash* /root
usermod -s /bin/bash root


set root password
# echo "root:abc" | chpasswd # DON'T USE - chpasswd is broken in void linux
passwd root


add another user, just in case root can't log in:
useradd -m -s /bin/bash user
usermod -aG wheel user
# echo "user:pass" | chpasswd # DON'T USE - chpasswd is broken in void linux
passwd user


check the /etc/shadow file to see that the password is changed
run visudo for the new user:

Choose only 1 option:
option 1: to allow wheel group users to sudo without password, do the following:
sed -E 's|^#?[ \t]*(\%wheel ALL=\(ALL\) NOPASSWD.*)$|\1|g' /etc/sudoers | EDITOR=tee visudo


option 2: to allow wheel group users to sudo with password, do the following:
sed -E 's|^#?[ \t]*(\%wheel ALL=\(ALL\) ALL.*)$|\1|g' /etc/sudoers | EDITOR=tee visudo


add /var/tmp as tmpfs in /etc/fstab
cat >> /etc/fstab << EOF
tmpfs /var/tmp tmpfs defaults,nosuid,nodev 0 0
EOF


add 2 EFI partitions to /etc/fstab
echo PARTUUID=$(blkid -s PARTUUID -o value ${DISK1}-part1) \
  /efi1 vfat nofail,noatime,defaults 0 0 >> /etc/fstab
echo PARTUUID=$(blkid -s PARTUUID -o value ${DISK2}-part1) \
  /efi2 vfat nofail,noatime,defaults 0 0 >> /etc/fstab


install grub and efibootmgr
xbps-install -y grub-x86_64-efi


test that grub can read /boot
ZPOOL_VDEV_NAME_PATH=1 grub-probe /boot # should return "zfs"


install grub
BOOTLOADER_ID=GRUB
ZPOOL_VDEV_NAME_PATH=1 grub-install --target=x86_64-efi --efi-directory=/efi1 --bootloader-id=$BOOTLOADER_ID


configure /etc/dracut.conf with hostonly="no" and remove btrfs module
cat >> /etc/dracut.conf << EOF
hostonly="no"
omit_dracutmodules+="btrfs"
# compress="lz4"
EOF


modify /etc/default/grub - add root=ZFS=$RPOOL/ROOT/default
[[ -f /etc/default/grub.original ]] && cp /etc/default/grub.original /etc/default/grub
cp /etc/default/grub /etc/default/grub.original

# add GRUB_CMDLINE_LINUX="root=ZFS=$RPOOL/ROOT/default"
if grep -qE "^GRUB_CMDLINE_LINUX=" /etc/default/grub ; then
  echo "found CMDLINE_LINUX="
  sed -i -E 's/^(GRUB_CMDLINE_LINUX=")/\1root=ZFS='"$RPOOL"'\/ROOT\/default /g' /etc/default/grub
elif grep -qE "^GRUB_CMDLINE_LINUX_DEFAULT=" /etc/default/grub ; then
  echo "not found CMDLINE_LINUX= found CMDLINE_LINUX_DEFAULT="
  sed -i -E 's|^(GRUB_CMDLINE_LINUX_DEFAULT=.*)$|\1\nGRUB_CMDLINE_LINUX="root=ZFS='"$RPOOL"'/ROOT/default"|g' /etc/default/grub
elif grep -qE "^#?[[:space:]]*GRUB_DISTRIBUTOR=" /etc/default/grub ; then
  echo "not found CMDLINE_LINUX= not found CMDLINE_LINUX_DEFAULT= found DISTRIBUTOR="
  sed -i -E 's|^(#?[ \t]*GRUB_DISTRIBUTOR=.*)$|\1\nGRUB_CMDLINE_LINUX="root=ZFS='"$RPOOL"'/ROOT/default"|g' /etc/default/grub
else
  echo "***ERROR!**** no GRUB_DISTRIBUTOR found!"
fi

# add GRUB_RECORDFAIL_TIMEOUT=5
sed -i -E 's/(^GRUB_TIMEOUT=)[0-9]+$/\15\nGRUB_RECORDFAIL_TIMEOUT=5/g' /etc/default/grub
# remove "quiet" from GRUB_CMDLINE_LINUX_DEFAULT
sed -i -E 's/(^GRUB_CMDLINE_LINUX_DEFAULT=")(.*)(quiet)(.*)(")/\1\2\4\5/g' /etc/default/grub


grub-mkconfig:
mkdir -p /boot/grub
ZPOOL_VDEV_NAME_PATH=1 grub-mkconfig -o /boot/grub/grub.cfg


set zpool cachefile
zpool set cachefile=/etc/zfs/zpool.cache $RPOOL
zpool set cachefile=/etc/zfs/zpool.cache $BPOOL


force update dracut efibootmgr grub
KERNELPKG=$(xbps-query -l | grep "Linux kernel and modules" | sed -E '/source headers/d; s/^ii[ \t]+(linux[^ \t]+)[ \t]+.*$/\1/g')
xbps-reconfigure -f $KERNELPKG
KERNELVER=$(echo $KERNELPKG | sed -E 's/linux[0-9\.]+-//g')
dracut --force --kver $KERNELVER
ZPOOL_VDEV_NAME_PATH=1 grub-mkconfig -o /boot/grub/grub.cfg


sanity check: lsinitrd should contain both zfs.ko and zpool.cache, and /boot/grub/grub.cfg contains root=ZFS=$RPOOL/ROOT/default
lsinitrd /boot/initramfs-${KERNELVER}.img | grep -E "zfs\.ko|zpool\.cache"  # output should have 2 lines with zfs.ko.gz and zpool.cache
grep -E "linux[[:space:]]+/BOOT/.*[[:space:]]+root=ZFS=""$RPOOL""/ROOT/default" /boot/grub/grub.cfg  # shouldn't be empty


rsync /efi1 and /efi2 and install efibootmgr
xbps-install -y rsync
rsync -Rai --stats --human-readable --delete --verbose --progress /efi1/./ /efi2
efibootmgr -c -g -d $DISK2 -p 1 -L "${BOOTLOADER_ID}-2" -l '\EFI\'"$BOOTLOADER_ID"'\grubx64.efi'


save environment variables:
cat > ~/vars.sh << EOF
# installation vars
export DISK1=$DISK1
export DISK2=$DISK2
export BPOOL=$BPOOL
export RPOOL=$RPOOL
export BOOTLOADER_ID=$BOOTLOADER_ID
export KERNELPKG=$KERNELPKG
export KERNELVER=$KERNELVER
EOF


exit chroot
exit


unmount filesystem:
umount -R /mnt
zfs unmount -a


export $RPOOL and $BPOOL
zpool export $RPOOL
zpool export $BPOOL


poweroff, remove USB device, and restart
poweroff


login as root and remove extraneous entry from grub boot menu
source ~/vars.sh
ZPOOL_VDEV_NAME_PATH=1 grub-mkconfig -o /boot/grub/grub.cfg


after first boot, become root
zfs list 
# if $BPOOL is not mounted or missing, then
source ~/vars.sh
zpool import $BPOOL

this has to be done only once.
after $RPOOL and $BPOOL are mounted, take snapshot of the clean system
source ~root/vars.sh
zfs snapshot $BPOOL/BOOT/default@clean
zfs snapshot $RPOOL/ROOT/default@clean