Skip to content

Instantly share code, notes, and snippets.

@kmdouglass
Created March 25, 2018 12:06
  • Star 8 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save kmdouglass/38e1383c7e62745f3cf522702c21cb49 to your computer and use it in GitHub Desktop.
Creates a chroot environment for creating custom Raspbian images and cross-compiling programs for the Raspberry Pi.
#!/bin/bash
# Creates a custom Raspbian image and cross-compilation environment.
#
# USAGE: ./create_image.sh
#
# This script must be run as either root or sudo.
#
# After creating the chroot environment, the script specified in the
# *script* variable will be executed from within the chroot. Your
# custom system setup commands should be located here. For example,
# this file may contain commands for creating new users, configuring
# the firewall, or compiling custom code.
#
# Prior to running this script, you will need the qemu,
# qemu-user-static, and binfmt-support Debian/Ubuntu packages (or
# their equivalent on other distributions). To install them on a
# Debian/Ubuntu system, use the command:
#
# sudo apt-get install qemu qemu-user-static binfmt-support
#
# This script has been tested in Ubuntu 16.04.4 LTS.
#
# Credit for much of this script goes to Michael Daffin:
# https://disconnected.systems/blog/custom-rpi-image-with-github-travis/
#
# Kyle M. Douglass, 2018
# https://kmdouglass.github.io
#
# Setup script error handling. See
# https://disconnected.systems/blog/another-bash-strict-mode for
# details.
set -uo pipefail
trap 's=$?; echo "$0: Error on line "$LINENO": $BASH_COMMAND"; exit $s' ERR
IFS=$'\n\t'
# Ensure root.
if [[ $EUID -ne 0 ]]; then
echo "Error: This script must be run as root or sudo."
exit 1
fi
# User-defined variables. Adjust these to your needs.
mount="/mnt/alphapi"
host_home="/home/kmdouglass"
script="setup"
rpi_zip="raspbian_lite_latest.zip"
rpi_url="https://downloads.raspberrypi.org/raspbian_lite_latest"
img_size="4G"
tmp_img="tmp.img"
loop_dev="/dev/loop2"
# Create the mount directory if it does not exist.
mkdir -p "${mount}"
# Download Raspbian only if we have not already done so.
[ ! -f "${rpi_zip}" ] && wget "${rpi_url}" -O "${rpi_zip}"
export orig_img_name=$(unzip -l raspbian_lite_latest.zip | grep .img | awk -F" " '{print $4}')
# Tasks to run when the shell exits for any reason, unmount the image
# and general cleanup.
cleanup() {
[[ -f "${tmp_img}" ]] && rm "${tmp_img}"
if [[ -d "${mount}" ]]; then
umount "${mount}/dev/pts" || true
umount "${mount}/dev" || true
umount "${mount}/proc" || true
umount "${mount}/sys" || true
umount "${mount}/boot" || true
umount "${mount}${host_home}" || true
umount "${mount}" || true
rm -r "${mount}" || true
fi
}
trap cleanup EXIT
# Extract the image.
unzip raspbian_lite_latest.zip
export root_start_sector=$(fdisk -l $orig_img_name | grep .img2 | awk -F" " '{print $2}')
export boot_start_sector=$(fdisk -l $orig_img_name | grep .img1 | awk -F" " '{print $2}')
export sector_size=$(fdisk -l $orig_img_name | grep "Sector size" | awk -F" " '{print $4}')
echo -e "Start sector: ${root_start_sector}\nSector size: ${sector_size}"
# Increase the size of the image by first appending zeros and then
# expanding the partition.
echo "Expanding the size of the image file..."
orig_size=$(du -h ${orig_img_name} | awk -F" " '{print $1}')
diff_size=$[$(numfmt --from=iec ${img_size}) - \
$(numfmt --from=iec ${orig_size})]
dd if=/dev/zero bs=1 count=1 seek=${diff_size} of=${tmp_img}
cat ${tmp_img} >> ${orig_img_name}
parted ${orig_img_name} resizepart 2 100%
# Mount the root image, check it, expand it, then unmount it.
losetup --offset=$[root_start_sector * sector_size] ${loop_dev} ${orig_img_name}
e2fsck -f ${loop_dev}
resize2fs -f ${loop_dev}
losetup -d ${loop_dev}
# Mount the images. Mount parts of the original image by using the
# offset option and the previously-determined sectors.
[ ! -d "${mount}" ] && mkdir "${mount}"
mount -o loop,offset=$[root_start_sector * sector_size] ${orig_img_name} ${mount}
[ ! -d "${mount}/boot" ] && mkdir "${mount}/boot"
mount -o loop,offset=$[boot_start_sector * sector_size] ${orig_img_name} ${mount}/boot
# Copy the image setup script.
install -Dm755 "${script}" "${mount}/tmp/${script}"
# Prep the chroot.
mount -t proc none ${mount}/proc
mount -t sysfs none ${mount}/sys
mount -o bind /dev ${mount}/dev
mount -o bind /dev/pts ${mount}/dev/pts
# Provide network access to the chroot.
rm ${mount}/etc/resolv.conf
cp /etc/resolv.conf ${mount}/etc/resolv.conf
cp /usr/bin/qemu-arm-static ${mount}/usr/bin/
# Mount my current home directory which contains the software to build.
echo "Mounting host home directory..."
echo "Mount point: ${mount}${host_home}"
mkdir -p "${mount}${host_home}"
mount -o bind ${host_home} ${mount}${host_home}
chroot ${mount} "/tmp/${script}"
@hadim
Copy link

hadim commented Mar 29, 2018

That looks very useful @kmdouglass !

A Docker image running it would make it even more accessible :-D

@kmdouglass
Copy link
Author

That's the next step ;p I just got a new Pi Model 3B+ and when I get time I'll test to verify that the Micro-Manager build is working correctly. If it is, I'll put all the scripts inside the Docker image.

I also need to tinker a bit with properly setting up the firewall and user security settings before I post the setup script...

@Marvin-Brouwer
Copy link

Marvin-Brouwer commented Jul 13, 2022

Hi,

First of all, thanks for sharing this.
I'm messing around with this script for a bit and I found a couple of minor tweaks.

  • line 58
    the line export orig_img_name=$(unzip -l raspbian_lite_latest.zip | grep .img | awk -F" " '{print $4}')
    should be export orig_img_name=$(unzip -l ${rpi_zip} | grep .img | awk -F" " '{print $4}')
    just for consistency
  • line 96
    the line e2fsck -f ${loop_dev}
    should be e2fsck -f -p ${loop_dev} to work in automated scripts

However, I'm having a hard time getting this to work.
It's currently failing on this error:

Resizing the filesystem on /dev/loop3 to 961740 (4k) blocks.
The filesystem on /dev/loop3 is now 961740 (4k) blocks long.
mount: /mnt/raspbian/boot: overlapping loop device exists for /home/runner/work/.../2020-02-13-raspbian-buster-lite.img.
./.github/workflows/scripts/create_image.sh: Error on line 105: mount -o loop,offset=$[boot_start_sector * sector_size] ${orig_img_name} ${mount}/boot

Do you think you know what might cause this?

Basically, all I want to do is preload a raspbian image with some files and a setup script so I don't need to reflash the sd-card and retest everything manually every time I change something in the setup.
I'm doing this in a GitHub pipeline using ubuntu-latest if that helps

@kmdouglass
Copy link
Author

Hi @Marvin-Brouwer ,
Thanks for the tweaks! I haven't worked on this for a few years and, to be honest, I don't remember the details of this script.

Looking at the error message, my guess is that the mount command is failing because either the boot_start_sector or root_start_sector values are wrong, which would mean that you're trying to mount part of the same image twice. This is just a guess, however.

@Marvin-Brouwer
Copy link

Hi @kmdouglass,

I see, well I did manage to figure it out.
I ran into this: https://forums.raspberrypi.com/viewtopic.php?t=190154
With some trial and error I managed to use fstab to get the limit sizes and that part works now!

I am running into this now :(

install: cannot stat 'setup.sh': No such file or directory
./.github/workflows/scripts/create_image.sh: Error on line 153: install -Dm755 "${script}" "${mount}/tmp/${script}"

Could this just simply be a pathing issue?

@Marvin-Brouwer
Copy link

Nice, got that to work by splitting the actual script path and the file I wanted to mount.
Anyway, I know you haven't touched this for a long time but I have one remaining question.

I now have a script that mounts(chroots?) the iso, copies a folder and runs an install script.
How do I now turn that into a new ISO?
Is that just implicitly done by unmounting because of the chroot?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment