Scripts to setup a Raspberry Pi OS image **before** flashing to SD card

_README.md

Usage

1. Download a Raspberry Pi OS image and unzip it:

   wget https://downloads.raspberrypi.org/raspios_lite_armhf_latest --trust-server-names --timestamping --quiet
   unzip 2020-05-27-raspios-buster-lite-armhf.zip

2. Download gist files:

   wget https://gist.githubusercontent.com/cinderblock/20952a653989e55f8a7770a0ca2348a8/raw/{prepare,chroot,grow,setup}.sh --timestamping --quiet
   chmod +x {prepare,chroot,grow}.sh

3. Edit setup.sh as desired. default adds my keys!
4. Run scripts.

Available scripts

sudo ./prepare.sh 2020-05-27-raspios-buster-lite-armhf.img [setup.sh] [mount point]
sudo ./chroot.sh 2020-05-27-raspios-buster-lite-armhf.img [mount point]
sudo ./grow.sh 2020-05-27-raspios-buster-lite-armhf.img [megabytes]

The default setup.sh script will fail unless you grow the standard .img first (for 2020-02-13).

Prepare

Runs the setup.sh (or other) script in a chroot in specified image.

Chroot

Gives you a chroot in the image to change whatever manually.

Planned changed: Use systemd-nspawn instead of chroot.

Note: Bash history will be saved too. Subject to change.

Grow

Grow the image (and main partition) by some number of megabytes.

This only affects the local .img file. On first boot, Raspbian will automatically grow the parition to fill the full card.

Adding 100MB adds aproximately 10 seconds to the write time when transfering to an SD card.

See also

• https://github.com/RPi-Distro/pi-gen - Full featured .img generator
• https://gist.github.com/htruong/0271d84ae81ee1d301293d126a5ad716 - Instructions to grow and chroot
• https://gist.github.com/htruong/7df502fb60268eeee5bca21ef3e436eb - Script to just mount & chroot (no loopback)
• https://gist.github.com/kmdouglass/38e1383c7e62745f3cf522702c21cb49 - Script: loopback, mount, chroot. No cleanup.

chroot.sh

#!/bin/bash

set -e

if [[ "$(dpkg --print-architecture)" != "armhf" ]]; then
  [ -x /usr/bin/qemu-arm-static ] || apt-get install qemu qemu-user-static binfmt-support
fi

trap 'echo Cleaning up...; cleanup' EXIT

IMG=$1

# Link a loopback device to the img file and get which one was used
LOOP=$(losetup -Pf ${IMG} --show)
function cleanup {
  losetup -d ${LOOP}
}

# Allow using second argument to override mount point
MNT=${2:-/tmp/mount${LOOP}}

# Create the mount point and mount the image
mkdir -p ${MNT}
mount ${LOOP}p2 ${MNT}
mount ${LOOP}p1 ${MNT}/boot
function cleanup {
  umount ${MNT}{/boot,}
  # rmdir -p ${MNT}
  losetup -d ${LOOP}
}

# Setup mounts needed for proper chroot
mount -o bind,ro {,${MNT}}/etc/resolv.conf
mount --bind {,${MNT}}/dev
mount --bind {,${MNT}}/dev/pts
mount --bind {,${MNT}}/sys
mount --bind {,${MNT}}/proc
function cleanup {
  umount ${MNT}{/{boot,etc/resolv.conf,dev{/pts,},sys,proc},}
  # rmdir -p ${MNT}
  losetup -d ${LOOP}
}

if [[ ! -z "${QEMU}" ]]; then
  # Make QEMU binary available in chroot
  cp {,${MNT}}${QEMU}
  function cleanup {
    rm ${MNT}${QEMU}
    umount ${MNT}{/{boot,etc/resolv.conf,dev{/pts,},sys,proc},}
    # rmdir -p ${MNT}
    losetup -d ${LOOP}
  }
fi

# Prepare ld.preload for chroot
sed -i 's/^/#CHROOT /g' ${MNT}/etc/ld.so.preload
function cleanup {
  sed -i 's/^#CHROOT //g' ${MNT}/etc/ld.so.preload
  [[ ! -z "${QEMU}" ]] && rm ${MNT}${QEMU}
  umount ${MNT}{/{boot,etc/resolv.conf,dev{/pts,},sys,proc},}
  # rmdir -p ${MNT}
  losetup -d ${LOOP}
}

echo Chrooting...

# Run the script in Chroot
chroot ${MNT} /bin/bash

grow.sh

#!/bin/bash

set -e

IMG=$1

# Default to 100MB
MEGA=${2:-100}

# Create a dummy partition to help defaults later
fdisk ${IMG} <<EOF > /dev/null
n




w
EOF

# Add zeros to the end of the img file
dd if=/dev/zero bs=1M count=${MEGA} >> ${IMG}

# Grow by deleting existing partition and creating a new one in the same place, but bigger.
fdisk ${IMG} <<EOF > /dev/null
d
2
n
p



d

w
EOF

# Setup loop device for image
LOOP=$(losetup -Pf ${IMG} --show)

trap 'echo Cleaning up...; cleanup' EXIT

function cleanup {
  losetup -d ${LOOP}
}

# Check fs because we can
e2fsck -f ${LOOP}p2

# Grow the ext partition
resize2fs ${LOOP}p2

# Check fs again because we can
e2fsck -f ${LOOP}p2

prepare.sh

#!/bin/bash

SETUP=${2:-setup.sh}

set -e

if [[ "$(dpkg --print-architecture)" != "armhf" ]]; then
  QEMU=/usr/bin/qemu-arm-static
  [ -x ${QEMU} ] || apt-get install qemu qemu-user-static binfmt-support
fi

trap 'echo Cleaning up...; cleanup' EXIT

IMG=$1

# Link a loopback device to the img file and get which one was used
LOOP=$(losetup -Pf ${IMG} --show)
function cleanup {
  losetup -d ${LOOP}
}

# Allow using second argument to override mount point
MNT=${3:-/tmp/mount${LOOP}}

# Create the mount point and mount the image
mkdir -p ${MNT}
mount ${LOOP}p2 ${MNT}
mount ${LOOP}p1 ${MNT}/boot
function cleanup {
  umount ${MNT}{/boot,}
  # rmdir -p ${MNT}
  losetup -d ${LOOP}
}

# Setup mounts needed for proper chroot
mount -o bind,ro {,${MNT}}/etc/resolv.conf
mount --bind {,${MNT}}/dev
mount --bind {,${MNT}}/dev/pts
mount --bind {,${MNT}}/sys
mount --bind {,${MNT}}/proc
function cleanup {
  umount ${MNT}{/{boot,etc/resolv.conf,dev{/pts,},sys,proc},}
  # rmdir -p ${MNT}
  losetup -d ${LOOP}
}

if [[ ! -z "${QEMU}" ]]; then
  # Make QEMU binary available in chroot
  cp {,${MNT}}${QEMU}
  function cleanup {
    rm ${MNT}${QEMU}
    umount ${MNT}{/{boot,etc/resolv.conf,dev{/pts,},sys,proc},}
    # rmdir -p ${MNT}
    losetup -d ${LOOP}
  }
fi

# Prepare ld.preload for chroot
sed -i 's/^/#CHROOT /g' ${MNT}/etc/ld.so.preload
function cleanup {
  sed -i 's/^#CHROOT //g' ${MNT}/etc/ld.so.preload
  [[ ! -z "${QEMU}" ]] && rm ${MNT}${QEMU}
  umount ${MNT}{/{boot,etc/resolv.conf,dev{/pts,},sys,proc},}
  # rmdir -p ${MNT}
  losetup -d ${LOOP}
}
# Copy setup script to image
cp ${SETUP} ${MNT}/setup.sh
chmod +x ${MNT}/setup.sh
function cleanup {
  rm ${MNT}/setup.sh
  sed -i 's/^#CHROOT //g' ${MNT}/etc/ld.so.preload
  [[ ! -z "${QEMU}" ]] && rm ${MNT}${QEMU}
  umount ${MNT}{/{boot,etc/resolv.conf,dev{/pts,},sys,proc},}
  # rmdir -p ${MNT}
  losetup -d ${LOOP}
}

echo Setting up...

# Run the script in Chroot
chroot ${MNT} /setup.sh

Setup Raspberry Pi OS Image Script.md

Setup Raspberry Pi OS Image Script

Scripts for modifying Raspberry Pi OS images before writing to SD card.

• QEMU - setup on x86 (faster)
• chroot
• Setup script. Set:
  • hostname
  • WiFi
  • ssh keys
  • Localization
  • sshd
  • programs
• Grow img

setup.sh

#!/bin/bash

# This is meant to be run on Raspberry Pi OS to setup the system the way you want.
# No variables to change. Just change what you want to change.
# For instance, the default gives *me* access to your system!

set -e

# Set locale to US
echo en_US.UTF-8 UTF-8 > /etc/locale.gen
locale-gen
update-locale LANG=en_US.UTF-8

sudo -u pi bash <<EOF_PI
set -e

# Initialize Authorized Keys
mkdir -p /home/pi/.ssh
curl -q https://github.com/cinderblock.keys > /home/pi/.ssh/authorized_keys

# Run other things as "pi" user

EOF_PI

# Enable SSHD, without passwords
systemctl enable ssh
echo PasswordAuthentication no >> /etc/ssh/sshd_config

# Add WiFi config
cat << EOF_WPA > /etc/wpa_supplicant/wpa_supplicant.conf
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=US

network={
  ssid="My SSID"
  psk="My PSK"
}
EOF_WPA

# Set Hostname
sed -e s/raspberrypi/myHostname/ -i /etc/host{s,name}

# Set Timezone
ln -snf /usr/share/zoneinfo/America/Los_Angeles /etc/localtime

# Update
apt-get update
apt-get upgrade -y

# Install Essentials
apt-get install vim screen git -y
# This is a nice colorful "status" line that shows which tab you're on in screen. You're welcome ;)
echo "caption always '%{= dg} %H %{G}| %{B}%l %{G}|%=%?%{d}%-w%?%{r}(%{d}%n %t%? {%u} %?%{r})%{d}%?%+w%?%=%{G}| %{B}%M %d %c:%s '" >> /etc/screenrc

# Install Node.js (needs and extra 50MB of space on the 2020-02-13 image first! Use "grow.sh")
curl -sL https://deb.nodesource.com/setup_12.x | bash -
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" > /etc/apt/sources.list.d/yarn.list
apt-get install -y nodejs yarn

# Optional set passwd for user `pi`. This will prompt you!
# passwd pi

# Add other things you want...

@cinderblock I just downloaded fresh copy of Raspbian Buster Lite, unpacked it and did this:

Here is my chroot:

user@laptop:~/download$ ll chroot/
total 76
drwxr-xr-x  2 root root 4096 Feb 13 10:56 bin/
drwxr-xr-x  3 root root 4096 May 19 10:55 boot/
drwxr-xr-x  4 root root 4096 Feb 13 10:51 dev/
drwxr-xr-x 79 root root 4096 Feb 13 11:09 etc/
drwxr-xr-x  3 root root 4096 Feb 13 10:55 home/
drwxr-xr-x 16 root root 4096 Feb 13 10:58 lib/
drwx------  2 root root 4096 Feb 13 11:09 lost+found/
drwxr-xr-x  2 root root 4096 Feb 13 10:51 media/
drwxr-xr-x  2 root root 4096 Feb 13 10:51 mnt/
drwxr-xr-x  3 root root 4096 Feb 13 10:55 opt/
drwxr-xr-x  2 root root 4096 Feb  8 21:47 proc/
drwx------  2 root root 4096 May 19 10:56 root/
drwxr-xr-x  4 root root 4096 Feb 13 10:51 run/
drwxr-xr-x  2 root root 4096 Feb 13 10:58 sbin/
drwxr-xr-x  2 root root 4096 Feb 13 10:51 srv/
drwxr-xr-x  2 root root 4096 Feb  8 21:47 sys/
drwxrwxrwt  2 root root 4096 Feb 13 11:09 tmp/
drwxr-xr-x 10 root root 4096 Feb 13 10:51 usr/
drwxr-xr-x 11 root root 4096 Feb 13 10:51 var/

Check to make sure bash inside chroot is indeed an ARM binary:

user@laptop:~/download$ file chroot/bin/bash 
chroot/bin/bash: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=3e5e2847bbc51da2ab313bc53d4bdcff0faf2462, stripped

Check if I am cheating and somehow have copied qemu-static-binary into the chroot:

user@laptop:~/download$ ll chroot/usr/bin/qemu-arm-static
ls: cannot access 'chroot/usr/bin/qemu-arm-static': No such file or directory

Finally, chroot:

user@laptop:~/download$ sudo chroot ./chroot/ /bin/bash
root@laptop:/#

Boom, we are in.

@dimitry-ishenko I think I might have run into problems trying this on a VPS so maybe it's the weird execution environment? I was getting a number of warnings from qemu about unimplemented calls. Trying again on a bare metal machine (x86-64 Intel i7-4770) seems to work without copying.

I guess it could also be just that that kernel is still slightly older (but 4.15.0-101-generic isn't that old). /etc/os-release says Ubuntu18.04 but I thought I was stuck on 17 at some point...

@cinderblock IIRC unimplemented calls are due to older qemu-{arm,aarch64}-* binaries and are mostly harmless. I used to get them on my 18.04 box as well. Google the exact error message, that's how I've found the explanation