Skip to content

Instantly share code, notes, and snippets.

@davidmpage
Created April 2, 2021 19:14
Show Gist options
  • Save davidmpage/aaefe8375312f489f99e985bde2b127b to your computer and use it in GitHub Desktop.
Save davidmpage/aaefe8375312f489f99e985bde2b127b to your computer and use it in GitHub Desktop.
Run a Raspberry Pi 4B from an SSD connected to a USB-3 port

Run Raspberry Pi 4 from SSD

Deprecation notice (2021-03-08)

The material in this gist is a bit out-of-date and will soon be withdrawn.


This gist builds on the excellent work done by Graham Garner, Andreas Spiess, and Peter Scargill to make it easy for Makers to install IOTstack on a Raspberry Pi.

One concern with any out-of-the-box Raspberry Pi is the limited lifespan of its SD card. You'd be pretty annoyed if you got your IOTstack up and running, only to find that you were one of the unlucky ones whose SD card wore out in less than six months!

SSDs, on the other hand, while not immune to the same kinds of failure, tend to have projected life expectancies in the hundreds of years.

The problem is that, while a Raspberry Pi 3B+ can boot and run from an external USB drive, the Raspberry Pi 4B can't yet do this. Given its two USB-3 ports, that's a real pity.

The Internet contains a huge amount of how-to on getting Raspberry Pi's to talk to external storage devices but I was unable to find a step-by-step guide to take you from bare-metal to IOTstack running on an SSD attached to a USB-3 port of a Raspberry Pi 4B. This gist is an attempt to remedy that.

Although this gist assumes installing IOTstack is the primary task goal, it is not essential. If you simply want to get Raspbian running on an SSD, just skip the bits that talk about IOTstack and Docker.

Perform a bare-metal installation of Raspbian on a Raspberry Pi 4B such that the system:

  1. Boots from an SD card.

  2. Runs from a larger-capacity SSD attached to a USB-3 port.

  3. Advertises itself using the multicast-DNS name "iot-hub.local"

  4. Supports headless operation via SSH and VNC.

  5. Supports SQLite3.

  6. Has a base install of IOTstack and Docker with:

    • Mosquitto
    • Node-Red
    • InfluxDB
    • Grafana
    • Portainer

Caveat: I am using an iMac running macOS Mojave 10.14 as my support platform. Some applications and Terminal commands are specific to that platform. You may need to adapt this gist if you are using MS Windows or Linux as your support platform. There are occasional uses of brew which is a third-party package manager (Homebrew installation instructions).

Open a browser page at www.raspberrypi.org/downloads/raspbian/ and download an appropriate image ("Raspbian Buster with desktop"). This arrives as something like:

~/Downloads/2020-02-05-raspbian-buster.zip

Calculate the digital signature of the downloaded file and compare it with the signature published on the site to assure no tampering or corruption. This assumes OpenSSL is installed (hint: "brew install openssl").

$ openssl dgst -sha256 ~/Downloads/2020-02-05-raspbian-buster.zip

Insert the SD card in the Mac. Or, more precisely, insert the Raspberry Pi's microSD card into an SD card adapter and insert the adapter into your Mac. Use BalenaEtcher to write 2020-02-05-raspbian-buster.zip to the SD card. macOS will prompt for your administrator password. BalenaEtcher creates two partitions on the SD card:

  • boot is mountable on the Mac
  • the second partition (ext4) is not mountable on the Mac.

Depending on BalenaEtcher's user preferences, the boot partition may be unmounted automatically at the end of the etching process. If it is, just pull out the SD card adapter, re-insert it, and wait for the boot partition to appear on the desktop.

In macOS Terminal, instruct Raspbian to permit access via SSH.

$ touch /Volumes/boot/ssh

Next, set up your WiFi credentials. Throughout these instructions I use the VI text editor. Feel free to use the text editor of your choice.

$ vi /Volumes/boot/wpa_supplicant.conf

Copy the text below to the clipboard.

country=«CC»
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
	ssid="«SSID»"
	psk="«PSK»"
}

Switch back to the Terminal session, and paste the text into VI. Replace the embedded «delimited» fields with values appropriate to your situation:

  • «CC» = a two-character upper-case country code, eg "AU"
  • «SSID» = WiFi Network Name
  • «PSK» = Password for WiFi network

Save and quit. Confirm that the WiFi credentials are in place.

$ cat /Volumes/boot/wpa_supplicant.conf

Eject the boot volume by dragging to the trash. Although this leaves some macOS Spotlight support files on the volume, it does not seem to worry Raspbian.

Remove the SD card from the Mac.

Insert the SD card into the Raspberry Pi. Apply power. Once it comes up and connects to WiFi it will advertise itself with the multicast-DNS name "raspberrypi.local". You can test for its presence with:

$ ping raspberrypi.local

Among other things, executing "ping" will give you the IP address assigned to the Raspberry Pi by DHCP. You can then find the MAC address of the WiFi interface via:

$ arp -a

and, in turn, use the MAC address to tell your DHCP server to assign a static IP address. While you can rely on a multicast-DNS name, you normally want a predictable IP address for an IoT server. The alternative is to configure the Raspberry Pi with a static IP address internally but a static assignment in a DHCP server is usually simpler and better for long-term maintenance.

Make sure SSH forgets any previous fingerprints for "raspberrypi.local". This may or may not return an error (either way, ignore the result).

$ ssh-keygen -R raspberrypi.local

Connect to the Raspberry Pi via SSH.

$ ssh pi@raspberrypi.local

Respond "yes" to fingerprint prompts from SSH and supply the Raspberry Pi's default password ("raspberry") when requested.

If you plan to set up a DHCP static assignment for the Ethernet port, you can identify its MAC address using:

$ ifconfig eth0

It is good practice to bring everything up-to-date.

$ sudo apt update
$ sudo apt full-upgrade -y

Basic configuration of the Raspberry Pi is via "raspi-config".

$ sudo raspi-config

Work through the following options:

8	Update
1	Change User Password
	⎣ «new password»
2	Network Options
	⎣	N1 Hostname
		⎣ «host name» eg "iot-hub"
4	Localisation Options
	⎣	I2 Change Timezone
		⎣ «set country and city»
	⎣	I4 Change WLAN Country
		⎣ «two-character upper-case country code» eg "AU"
5	Interfacing Options
	⎣	P3 VNC
		⎣ enable VNC server
7	Advanced Options
	⎣	A5 Resolution
		⎣ (DMT Mode 82 1920x1080 60Hz 16:9)
Finish (hint: [TAB] [TAB] )

The wisdom of always changing the host name will make sense, come the day you get another Raspberry Pi. Having two devices on your network using the same multicast-DNS name ("raspberrypi.local") is a recipe for a mess.

If raspi-config suggests rebooting, do that. If it does not, perform a manual reboot (ignore any error referring to "raspberrypi.local").

$ sudo reboot

Use the macOS Terminal to re-connect with the Raspberry Pi under its new name (which, from this point on, is assumed to be "iot-hub"). The first two lines below clear any previous fingerprints from the SSH known hosts file.

$ ssh-keygen -R raspberrypi.local
$ ssh-keygen -R iot-hub.local
$ ssh pi@iot-hub.local

Again, respond "yes" to fingerprint prompts from SSH but this time supply the new password you just set in "raspi-config".

Worst case, in the event of problems, is the need to start over from BalenaEtcher.

Depending on the Raspbian image you chose as your starting point, some of the tools you might need later may not be installed. Feel free to pick and choose the packages you want. It also does no harm to "install" something that is already installed.

Required software

$ sudo apt install -y git curl

Recommended software

$ sudo apt install -y acl jq sqlite3 uuid-runtime wget

Useful software

$ sudo apt install -y dnsutils iotop iperf mosquitto-clients ruby subversion sysstat

Generate a password for VNC.

$ sudo vncpasswd -print

You will be prompted for a password and to verify it. Use the same password as set in "raspi-config" (or you'll go mad). You will wind up with three lines like this:

Password:
Verify:
Password=«password hash»

The value in the «password hash» field above will be needed later. Copy the following lines to the clipboard.

Encryption=PreferOn
Authentication=VncAuth
Password=«password hash»

Create the VNC configuration file.

$ sudo vi /etc/vnc/config.d/common.custom

Paste the three lines copied to the clipboard in the previous step into VI and then replace the «password hash» with the value returned by "vncpassword" in the preceding step. Save and exit.

Confirm that the changes have been applied.

$ cat /etc/vnc/config.d/common.custom

Restart the VNC server (which was set running by the earlier "raspi_config").

$ sudo systemctl restart vncserver-x11-serviced

It should now be possible to connect to the Raspberry Pi from the Mac via VNC. In the macOS Finder, press Command+K and use this URL:

vnc://pi@iot-hub.local

Follow all GUI prompts to the bitter end. If prompted for another password change, use the same password as in previous steps (or you'll go mad). The system will suggest restarting. Accept that suggestion and close the VNC window.

This step is optional. If you want to run from the SD card you have just built, skip to Hass.io support.

The next few steps assume that:

  • the SSD you want to use for Raspbian has been connected to your Mac and formatted via Disk Utility as a Master Boot Record FAT volume with the name "EXTSSD". Pre-formatting of the SSD is not essential but having known characteristics makes it easy to identify the SSD when it is first attached to the Raspberry Pi.
  • The drive has been ejected from the Mac.

Connect the SSD to a USB-3 port on the Raspberry Pi.

In the macOS Terminal window, reconnect to the Raspberry Pi via SSH.

$ ssh pi@iot-hub.local

On the Raspberry Pi, locate the available block devices.

$ lsblk

The output should look similar to this:

NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda           8:0    1 14.6G  0 disk
└─sda1        8:1    1 14.6G  0 part /media/pi/EXTSSD
mmcblk0     179:0    0 14.6G  0 disk
├─mmcblk0p1 179:1    0  256M  0 part /boot
└─mmcblk0p2 179:2    0 14.3G  0 part /

Each entry in the "NAME" column implies a "/dev/" prefix. We are interested in:

  • the external SSD physical disk: /dev/sda,
  • the external SSD logical volume: /dev/sda1, and
  • the current root volume /dev/mmcblk0p2 on the SD card.

Notice how the external SSD logical volume has an entry in the "MOUNTPOINT" column. This means the volume is mounted. The last part of the mount point path is "EXTSSD" which is the name the volume was given when it was formatted on the Mac, and confirms that this is the right device. Unmount it.

$ sudo umount /dev/sda1

Prepare the external SSD physical disk for use by Raspbian. The glyph is used throughout this documentation to mean "press the Return key".

$ sudo fdisk /dev/sda
d
n
p
1
⏎
⏎
(respond Y if asked to remove a prior signature)
w

Construct an ext4 type file system on the external SSD logical volume.

$ sudo mkfs -t ext4 /dev/sda1

Clone the root file system from the SD onto the SSD (refer back to the "lsblk" output as the source of the /dev/mmcblk0p2 and /dev/sda1 parameter values).

$ sudo dd if=/dev/mmcblk0p2 of=/dev/sda1 bs=64K conv=noerror,sync

Create a mount point for the SSD on the SD card.

$ sudo mkdir /ssd

Mount the cloned volume.

$ sudo mount /dev/sda1 /ssd

Edit the fstab for the running system (ie the SD) so that it knows about and will auto-mount the SSD at /ssd

$ sudo vi /etc/fstab

The final result should look something like this (the critical change is adding the 4th line).

proc                  /proc           proc    defaults          0       0
PARTUUID=d9b3f436-01  /boot           vfat    defaults          0       2
PARTUUID=d9b3f436-02  /               ext4    defaults,noatime  0       1
/dev/sda1             /ssd            ext4    defaults          0       2
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that

Make a mount-point for the SD card in the (non-running) SSD system.

$ sudo mkdir /ssd/sd

Edit the fstab for the new system (ie the clone on the SSD) so that it knows about and will auto-mount the SD at /sd

$ sudo vi /ssd/etc/fstab

The key changes are:

  • 3rd line:
    • change the mount point from "/" to "/sd". Remember, this is editing the fstab on the SSD which is not yet the running system.
    • remove the ",noatime" mount option
    • change the value of the right-most parameter (fsck order) from "1" to "2"
  • Add the 4th line, "as is".

The final result should look something like the following:

proc                  /proc           proc    defaults          0       0
PARTUUID=d9b3f436-01  /boot           vfat    defaults          0       2
PARTUUID=d9b3f436-02  /sd             ext4    defaults          0       2
/dev/sda1             /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that

What we have just done is to say:

  • When the Raspberry Pi runs from the SD card, the SSD will be auto-mounted at /ssd
  • When the Raspberry Pi runs from the SSD, the SD card will be auto-mounted at /sd

The Raspberry Pi will always boot from the /boot volume on the SD card but will then run Raspbian from whichever volume we choose in the next step.

Change into the boot partition.

$ cd /boot

At the moment, cmdline.txt is instructing the Raspberry Pi to run Raspbian from the SD. Make a backup of cmdline.txt which preserves that.

$ sudo cp cmdline.txt cmdline.run-from-sd-card.txt

Edit cmdline.txt to tell the Raspberry Pi to run Raspbian from the SSD.

$ sudo vi cmdline.txt

Replace the right hand side of "root=" with "/dev/sda1" so that the final result looks something like this:

console=serial0,115200 console=tty1 root=/dev/sda1 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles

Make a copy of that newly-edited file to preserve the instruction to run Raspbian from the SSD.

$ sudo cp cmdline.txt cmdline.run-from-ssd.txt

The presence of these two files both documents what you have done and makes it easy to swap back and forth between the two run methods by executing one of the following:

$ sudo cp cmdline.run-from-ssd.txt cmdline.txt
$ sudo cp cmdline.run-from-sd-card.txt cmdline.txt

Right now, the "run from SSD" variant is ready to take effect so let's do that.

$ sudo reboot

Once the Raspberry Pi comes back up, use the macOS Terminal to re-connect.

$ ssh pi@iot-hub.local

Because we cloned from a 16GB SD card, the current file system is limited to that size. You can prove that to yourself.

$ df -h

This line shows a 15GB partition (the rest of the space is in the boot partition):

/dev/root        15G  3.0G   11G  22% /

Tell the system to use all of the partition on the SSD.

$ sudo resize2fs /dev/sda1

Confirm that the expansion has taken effect.

$ df -h

This line shows a much larger partition:

/dev/root       441G  3.0G  420G   1% /

Reboot (prophylactic).

$ sudo reboot

Hass.io support (optional)

This step is optional. If you have no plans to install Hass.io, skip to Install IOTstack & Docker.

Do not make the mistake of reasoning like this:

I can't think of any reason why I need Hass.io now but, who knows, I might want it someday, therefore I'll install the support now, just to be safe.

Hass.io creates a conundrum:

  • If you are definitely going to install Hass.io then you must install its dependencies before you install IOTstack and Docker.
  • One of Hass.io's dependencies is Network Manager. Network Manager makes serious changes to your operating system, with side-effects you may not expect such as giving your Raspberry Pi's WiFi interface a random MAC address both during the installation and, then, each time you reboot. You are in for a world of pain if you install Network Manager without first understanding what is going to happen and planning accordingly.
  • If you don't install Hass.io's dependencies before you install IOTstack and Docker, you will probably have to rebuild your system if you change your mind and decide to install Hass.io. This is because both Docker and Network Manager futz with your Raspberry Pi's networking. Docker is happy to install after Network Manager, but the reverse is not true.

Assuming you still want to proceed, install the following:

$ sudo apt install -y apparmor apparmor-profiles apparmor-utils

A post at community.home-assistant.io suggests also installing:

$ sudo apt install -y software-properties-common apt-transport-https ca-certificates dbus

but it is not clear whether those are strictly necessary.

If you are running headless (SSH or VNC), connect your Raspberry Pi to Ethernet. When the Ethernet interface initialises, work out its IP address:

$ ifconfig eth0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.132.9  netmask 255.255.255.0  broadcast 192.168.132.255
        ether ab:cd:ef:12:34:56  txqueuelen 1000  (Ethernet)
        RX packets 4166292  bytes 3545370373 (3.3 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2086814  bytes 2024386593 (1.8 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

In the above, the IP address assigned to the Ethernet interface is 192.168.132.9.

Drop out of your existing session (SSH or VNC) and re-connect to your Raspberry Pi using the IP address assigned to its Ethernet interface:

$ ssh pi@192.168.132.9

or:

vnc://pi@192.168.132.9

If you ignore the advice about connecting via Ethernet and install Network Manager while your session is connected via WiFi, your connection will freeze part way through the installation (when Network Manager starts running and unconditionally changes your Raspberry Pi's WiFi MAC address).

You may be able to re-connect after the WiFi interface acquires a new IP address and advertises that via multicast DNS associated with the name of your device (eg iot-hub.local), but you may also find that the only way to regain control is to power-cycle your Raspberry Pi.

The advice about using Ethernet is well-intentioned. You should heed this advice even if means you need to temporarily relocate your Raspberry Pi just so you can attach it via Ethernet for the next few steps. You can go back to WiFi later, once everything is set up. You have been warned!

Install Network Manager:

$ sudo apt install -y network-manager

According to @steveatk on Discord, you can stop Network Manager from allocating random MAC addresses to your WiFi interface by editing:

$ sudo vi /etc/NetworkManager/NetworkManager.conf

and adding:

[device]
wifi.scan-rand-mac-address=no

This needs to be done twice:

  • after Network Manager is installed; and again
  • after Hass.io is installed via the IOTstack menu.

In both cases, NetworkManager.conf is replaced with a version that enables random MAC allocation.

Complete instructions for installing IOTstack on the Raspberry Pi are available at sensorsiot.github.io/IOTstack/. This is a summary.

Previously, the instructions were at github.com/gcgarner/IOTstack

key assumptions

It is important to understand that there are some assumptions built into IOTstack:

  1. It is running on a Raspberry Pi 3 or 4.
  2. You are logged-in as the user "pi".
  3. User "pi" has the user ID 1000.
  4. The home directory for user "pi" is /home/pi/.
  5. IOTstack is installed at /home/pi/IOTstack.

This is not to say that IOTstack won't run on other platforms, including other Raspberry Pi variants. It will. It's just that IOTstack gets more testing on RPi3 and RPi4 platforms than other hardware…

The other assumptions are immutable. You can expect trouble if you get all creative and rename the default user and/or change the user ID or the installation location. Should it be like that? Probably not. It just is.

installation

Connect to the Raspberry Pi from macOS Terminal.

$ ssh pi@iot-hub.local

Clone the IOTstack GIT repository.

$ cd
$ git clone https://github.com/SensorsIot/IOTstack.git IOTstack

Previously, the URL was "https://github.com/gcgarner/IOTstack.git".

Install Docker.

$ cd ~/IOTstack
$ ./menu.sh
Choose "Install Docker"

Installing Docker involves a reboot of the Raspberry Pi so reconnect from macOS Terminal.

$ ssh pi@iot-hub.local

Back on the Raspberry Pi, select the Docker containers you want to install.

$ cd ~/IOTstack
$ ./menu.sh 
Choose "Build Stack"


This is the basic set you should probably include:

  • Portainer
  • Node-Red
  • InfluxDB
  • Grafana and
  • Eclipse-Mosquitto

To change what is installed, navigate with the arrow keys and press the space-bar to opt in or out. For example, to add Pi-Hole:

[x] Pi-Hole

When you are ready, press [TAB] to select "<Ok>" and press ⏎.

A list of Node-RED nodes is shown. Select each one you require by navigating with the arrow keys and pressing the space-bar. Example:

[x] node-red-node-sqlite

Note: Installing "node-red-node-sqlite" produces a massive number of compiler warnings during the subsequent "docker-compose up -d" step. While they seem alarming, these warnings can be ignored.

To complete the process press [TAB] to select "<Ok>" and press ⏎ to generate the YAML file.

Instruct Docker to download, install and activate all selected container services.

$ docker-compose up -d

Have a cup of coffee. Once the process completes, confirm that your chosen containers are running.

$ docker ps

Periodically, do this:

$ sudo apt update
$ sudo apt upgrade
$ sudo reboot

This will also update docker and docker-compose as needed. If you suspect that a package you depend on is not being updated because of a dependency issue, you can consider replacing "sudo apt upgrade" with:

$ sudo apt full-upgrade

Bring the project up-to-date:

$ cd ~/IOTstack
$ git status

Review any changes flagged by git and take appropriate action (eg making backup copies). Then:

$ git pull origin master

If any changes applied by the pull appear to affect your stack (eg a template for one of your containerised processes has been updated), re-run the menu and follow your nose:

$ ./menu.sh

To update everything EXCEPT Node-Red:

$ cd ~/IOTstack
$ docker-compose pull
$ docker-compose up -d

To update Node-Red:

$ cd ~/IOTstack
$ docker-compose build --no-cache --pull nodered

Notes:

  1. Remember that a rebuild of Node-Red takes a long time if SQLite needs to be recompiled and chucks up a lot of compiler warnings and even seems to stall. Be patient!
  2. There is no need to take your stack down before you start the update process. Your existing containers continue to run while Docker prepares the updated versions. A new-for-old swap occurs at the last moment with barely any downtime.

After any update, check if any extraneous images have been left behind:

$ docker images

Remove extraneous images using their "Image ID", as in:

$ docker rmi «imageID»

If you get an error saying the image is in use by something else, try removing that container using the ID in the error message:

$ docker rm «containerID»

and then re-try removing the image. Keep iterating until the image is gone.

From time to time it is also useful to a general cleanup:

$ docker system prune

The material explaining how to set up SSHFS has been moved to a dedicated gist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment