Skip to content

Instantly share code, notes, and snippets.

@avoidik
Last active December 12, 2024 08:06
Show Gist options
  • Save avoidik/d8fc39a372db680090edd5322d60848f to your computer and use it in GitHub Desktop.
Save avoidik/d8fc39a372db680090edd5322d60848f to your computer and use it in GitHub Desktop.
Migrate Raspberry Pi from SD to USB

Raspberry Pi from SD to USB

How to switch over from SD-card to USB-attached device (USB thumbstick, USB enclosed SSD or HDD, etc.) to have more durable storage option.

Steps

  1. Connect USB device to your RPI

  2. Make sure USB device visible by the system

    sudo fdisk -l
    sudo lsblk

    Usually mmcblk0 is your SD-card, and sda, sdb, sdX are attached USB devices, in my case it was sda. Make sure there is no sensitive data stored on the target USB device. If it's new USB device there should be no partitions reported by fdisk for it. You may want to clean target device using sudo sfdisk --delete /dev/sda (be very careful, it will delete everything on your target device)

  3. Install rpi-clone

    git clone https://github.com/billw2/rpi-clone.git 
    cd rpi-clone
    sudo cp rpi-clone rpi-clone-setup /usr/local/sbin
    sudo rpi-clone-setup -t $(hostname -s)
    sudo rpi-clone -s
  4. Clone SD-card to USB device

    sudo rpi-clone sda # replace sda here with your device name

    This may take some time depending on how much data you have, storage performance, running applications, clients connected, etc. During the process few questions will be asked, read carefully and answer. On the last step where it's asked to examine mounted clone run another connection to RPI, keep existing intact.

    If you've accidentally skipped it just remount it manually:

    • sudo mount /dev/sda6 /mnt/clone/boot.
    • sudo mount /dev/sda7 /mnt/clone
  5. Identify partitions

    Target device will have the same partitions layout as in source device. We need to find out boot and last partition ids.

    sudo lsblk -o name,mountpoint,size,partuuid /dev/sda6 # clone of mmcblk0p6 mounted as /boot
    sudo lsblk -o name,mountpoint,size,partuuid /dev/sda7 # clone of mmcblk0p7 mounted as / or root

    Write down PARTUUID column, in my case these were 3b4344a4-06 for boot and 3b4344a4-07 for root partition respectively

  6. Change cmdline.txt

    To be able to boot from the target device we have to change cmdline.txt file:

    • change root=/dev/mmcblk0p7 to root=PARTUUID=3b4344a4-07 which is the root partition id
    • append at the end of the line rootdelay=5 option to wait while attached device is initialized at boot time

    Complete example:

    console=serial0,115200 console=tty1 root=PARTUUID=3b4344a4-07 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait rootdelay=5
    

    If you'll try to boot without this change you'll get endless Waiting for root message.

    Note: there is one interesting caveat, if you'd like to use LABEL or UUID instead of PARTUUID you must create initramfs/initrd file, hence additional step will be required. Do not use PARTUUID with initrd.

    If you're going to use PARTUUID then you don't need this step (I repeat, skip it and follow the next step).

    • make sure root= set to something like root=LABEL=USB-ROOT
    • cd /mnt/clone/boot/
    • generate initrd file with mkinitramfs -o initrd.sda
    • append initramfs initrd.sda followkernel after rootdelay=5

    Complete example:

    console=serial0,115200 console=tty1 root=LABEL=USB-ROOT rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait rootdelay=5 initramfs initrd.sda followkernel
    
  7. Change fstab

    After the boot process has been completed we would need to mount correct partitions.

    Open /mnt/clone/etc/fstab and do the following:

    • change device /dev/mmcblk0p6 to PARTUUID=3b4344a4-06 (remember to use your own id)
    • change device /dev/mmcblk0p7 to PARTUUID=3b4344a4-07 (remember to use your own id)

    Complete example:

    proc            /proc           proc    defaults          0       0
    PARTUUID=3b4344a4-06  /boot           vfat    defaults          0       2
    PARTUUID=3b4344a4-07  /               ext4    defaults,noatime  0       1
    
  8. Unmount

    Now return back to the initial terminal where we've left rpi-clone process open and hit enter to unmount all partitions. Make sure you don't have the second terminal open somewhere at the mounted paths.

  9. Test

    Shutdown RPI with sudo shutdown -h now, disconnect PSU, unplug SD card, plug PSU in, verify

@bnhf
Copy link

bnhf commented Sep 19, 2022

A couple of notes on the above:

Make sure you have your boot order set correctly BEFORE you start the steps listed. This can be done with raspi-config utility, though that doesn't show what your current boot order settings are. You'll want to set your boot order to be USB-MSD first, followed by SD-CARD as a backup. I ended up using a utility called rpi-eeprom-mgr, to both update the bootloader to the latest stable version and to confirm the desired boot order. The utility is discussed here, and can be downloaded from there too:

https://forums.raspberrypi.com/viewtopic.php?t=283347

After updating the BOOTLOADER EEPROM, you should get a display like this in rpi-eeprom-mgr:

Screenshot 2022-09-19 153931

And if you've successfully changed your boot order, you can confirm that in rpi-eeprom-mgr:

Screenshot 2022-09-19 154331

Reboots are required after each one of these changes, and also after the:

sudo sfdisk --delete /dev/sda

in Step #2 above, if your USB Flash drive has been previously partitioned.

The cmdline.txt referenced in Step #6 is somewhat ambiguous, as there are two such files after running rpi-clone. I modified the one under /mnt/clone/boot, as this is the file that will be used when booting from the new USB drive. I did not need to modify anything under /mnt/clone/etc/fstab as the PARTUUID entries were already correct.

Everything seems to be working fine, after completing this process. I do get a brief error message on the console now:

mmc1: Controller never released inhibit bit(s)

This does not appear to be an issue, and the only solutions I've found involve flashing the USB drive from scratch and re-loading everything -- which I'd very much prefer not to do.

For reference, I'm using an RPi4 8GB board with the Raspberry Pi OS (64-bit) -- originally the Lite version, but updated later with Desktop. USB 3.1 flash drive is a 256GB Samsung Fit Plus.

@avoidik
Copy link
Author

avoidik commented Oct 27, 2022

@bnhf much appreciated for your valuable input, I've just finished this setup on a brand new pi 400, can confirm that steps 5, 6 (excluding rootdelay flag), 7 will be optional if we'd be using rpi-eeprom-mgr at the beginning

I've also commented out the following condition in rpi-eeprom-mgr around line #563, it seems I'm dealing with newer cpu revision

if [[ $(cat /proc/cpuinfo | grep -c "^Revision\s\+:\s\+[abcd]0311[1245]$") -eq 0 && $(cat /proc/cpuinfo | grep -c "^Revision\s\+:\s\+c03130$") -eq 0 ]]; then
  errexit "$0 must be run on a Raspberry Pi 4/400"
fi

@avoidik
Copy link
Author

avoidik commented Oct 27, 2022

for non-UAS SSD enclosures we have to add its USB VID & PID to usb-quirks, ref.

@mariotaku
Copy link

Thank you. I have finished migration with this guide, and booted successfully in one shot.

@thomasvjohansen
Copy link

Thank you for the guide, it worked after I got the boot to USB mode configured:

I have a rpi 3B+ and I didnt have the boot option in the raspi-config.

So I had to do:
echo program_usb_boot_mode=1 | sudo tee -a /boot/config.txt

and check:
vcgencmd otp_dump | grep 17: 17:3020000a
which should give output "0x3020000a"

source: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#usb-boot-modes

@kahancock
Copy link

Thanks, this worked great for my octopi install. I had previously tried just cloning the SD card as many others had reported success with but had no luck. This worked first shot!

@picard12
Copy link

I have been using an emmc with my raspberries for several years. I've never used such a procedure to use it, just used "dd" to clone hdd, sd etc, or copy an iso to the emmc.

@mg2c
Copy link

mg2c commented Feb 10, 2023

Thank you for developing this - I was faithful I could clone my SD Card to a 2.5" drive and exactly followed the instructions. But I failed several times, even after using the abovementioned EEPROM Bootloader utility. The RPi4 never boots from USB, when I remove the SD Card.

Could these be two reasons for my problem:?

  • I use a 5TB drive (I get an error message during the cloning prozess about GPT and size limits of 2TB). But "sda" is recognized, several partitions are created, I get the different PARTUUIDs, the last cloned partition has 2 TB, so the process itself seems to work.
  • I use an Geekworm enclosure, that has a Raid Controller on board (https://geekworm.com/products/naspi-gemini - newest V2 version)

As for the editing of the cmdline.txt - I do it via this command:
sudo nano /mnt/clone/boot/cmdline.txt

The strange thing is that I do get an output, that seems, as if the 2.5" drive is active as the bootdrive, when the SD-Card is back in and the RPi boots:

xxxxxx:~ $ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 4,6T 0 disk
├─sda1 8:1 0 2,4G 0 part
├─sda2 8:2 0 1K 0 part
├─sda5 8:5 0 32M 0 part
├─sda6 8:6 0 256M 0 part /boot
└─sda7 8:7 0 2T 0 part /
mmcblk0 179:0 0 29,7G 0 disk
├─mmcblk0p1 179:1 0 2,4G 0 part
├─mmcblk0p2 179:2 0 1K 0 part
├─mmcblk0p5 179:5 0 32M 0 part
├─mmcblk0p6 179:6 0 256M 0 part
└─mmcblk0p7 179:7 0 27,1G 0 part
xxxxxx:~ $ df -h
Dateisystem Größe Benutzt Verf. Verw% Eingehängt auf
/dev/root 2,0T 6,0G 1,9T 1% /
devtmpfs 775M 0 775M 0% /dev
tmpfs 936M 12K 936M 1% /dev/shm
tmpfs 936M 18M 919M 2% /run
tmpfs 5,0M 4,0K 5,0M 1% /run/lock
tmpfs 936M 0 936M 0% /sys/fs/cgroup
/dev/sda6 253M 50M 203M 20% /boot
tmpfs 188M 0 188M 0% /run/user/1000

When searching for the edited cmdline.txt (the one I altered in /mnt/clone/boot/cmdline.txt), I am able to find it with the appropriate PARTUUID in /boot/cmdline.txt. Strange, that the RPi does not boot... I even set rootdelay to 20 seconds, just in case...

Another hint may be, that software, that I did install on the SD Card prior to cloning (Samsung Smartcontol to monitor drive health), is not working , when booting with the SD Card put back in. So some stuff seemed to be cloned, other stuff not?

Would you be so kind and give me some hints on what to do?
Thanks

@avoidik
Copy link
Author

avoidik commented Feb 10, 2023

@mg2c this enclosure is indeed interesting, unfortunately, I don't know much about their specific raid controller implementation nor required drivers, in a world of enterprise raid controllers I was deliberately using hybrid booting (usb+raid), also, partition layout of such a huge drives for sure must be using gpt (as mbr is limited and uefi is not an option), have you tried any other alternative like usb-boot?

@mg2c
Copy link

mg2c commented Feb 10, 2023

@mg2c have you tried any other alternative like usb-boot?
Thanks for the quick answer - I actually did try usb-boot. I ended up with just two partitions on the HDD. Felt a bit strange, coming from 5 partitions on the SD-card and 4 partitions on the HDD, when using rpi-clone.
There is no way, that you software can deal with GPT and support large drives?

@anuragthehatter
Copy link

@avoidik Thanks a lot. Booted successfully in one shot. Wondering if this would work from USB to USB?

@avoidik
Copy link
Author

avoidik commented Jun 8, 2023

hey @anuragthehatter it should work, just make sure that partition ids are up to date

@anuragthehatter
Copy link

hey @anuragthehatter it should work, just make sure that partition ids are up to date

Thanks @avoidik I will check that and update.

@anuragthehatter
Copy link

anuragthehatter commented Jun 16, 2023

hey @anuragthehatter it should work, just make sure that partition ids are up to date

Thanks @avoidik I will check that and update.

That worked like a charm. Just make sure we don't connect both USBs together and boot paritition can be taken from any of one while booting. So I have backup one connected with powered usb hub with its switch off :)

@avoidik
Copy link
Author

avoidik commented Jun 16, 2023

@anuragthehatter very good, thanks for sharing 👍

@octocron
Copy link

I ran this to set up a cluster of pi's 8/13/23. The rpi-clone now nicely takes care of the cmdline.txt and fstab edit steps and additionally tells you the PARTUUID, eliminating doing those steps manually. This has worked perfectly while now being even easier.

@andychess
Copy link

This worked for me on a Pi4 with 512Gb SSD. Thanks to everyone involved, you have saved me hours of research!

@vseibt
Copy link

vseibt commented Dec 27, 2023

Worked not that good. rpi-clone copied everything from root-partition but nothing from boot partition.
So I had to copy boot partition manually using dd (done on a different computer) and had to edit cmdline.txt there accordingly.
At least /etc/fstab was automatically rewritten correctly.
Without this thread I would never have figured out to edit cmdline.txt in the boot partition.
I was cloning from an SD card to an USB-SSD-128 GB harddrive on a raspi 4.

@int48
Copy link

int48 commented Jan 16, 2024

Copy from boot and root partition is perfect, use it with a raspi 4, micro sd card and Sandisk Extreme USB Stick.
The programm and instructions is perfect, thanks a lot.

@Madben82
Copy link

Worked flawlessly to switch from SD to USB NVME on my RPi4! Thank you very much.

Was a bit confused that the cmdline.txt in the firmware folder was hidden and the last two chages to the fstab file were already changed to the correct values of the new SSD. But I am not that experienced in all this Raspberry stuff, so thats my fault I guess.

@jorycz
Copy link

jorycz commented Feb 9, 2024

Just finished this on Raspberry Pi4 without issue on first try. Thanks. /boot/firmware/ was not mounted so before you start, mount it with
mount /dev/mmcblk0p1 /boot/firmware/

@DevinTDHa
Copy link

DevinTDHa commented Dec 9, 2024

For anyone using this guide to try to boot their Raspberry Pi 3b+ from an external drive:

Firstly, I used a more recent maintained version here: https://github.com/geerlingguy/rpi-clone

Then I tried to follow this guide, but the pi would not boot just from the attached external drive. My solution was to leave the SD card plugged in but copy /boot/cmdline.txt and /etc/fstab from the cloned partitions to the SD card. I suppose as long as /boot is ok it will boot from the external drive.

For example:

# Just in case
sudo cp /boot/cmdline.txt /boot/bak_cmdline.txt
sudo cp /etc/fstab /etc/bak_fstab

# Copy the files from sda to sd card
sudo cp /mnt/clone/boot/cmdline.txt /boot/cmdline.txt
sudo cp /mnt/clone/etc/fstab /etc/fstab

# After Reboot
sudo lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0 931.5G  0 disk
├─sda1        8:1    0   256M  0 part /boot
└─sda2        8:2    0 931.3G  0 part /
mmcblk0     179:0    0  59.5G  0 disk
├─mmcblk0p1 179:1    0   256M  0 part
└─mmcblk0p2 179:2    0  59.2G  0 part

The idea is, that the pi will still initially boot from the SD card, but after the kernel is loaded, mount all the files from the external drive. I read this in the forum post Running Raspbian from USB Devices : Made Easy

The easiest and most reliable way to run Raspberry Pi OS on a USB device with any Raspberry Pi prior to the model 4 or 5 is to leave an SD card containing Raspberry Pi OS in place, but use it only for starting the Raspberry Pi OS that is residing on a USB device.

Works for me so far, as I did this because of the SD card going bad.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment