Back up your computer

backup.md

Preface

There are many ways to back up your computer. In order from least effective to most effective, we have:

• Copy all the files manually to another disk periodically. (Optionally, use an automated tool for this, like rsync.)

  This sucks. Most of the time you don't even preserve permissions. Plus it's horribly error-prone. And you also can't restore a working system from it.

• Image your entire boot drive.

  This also sucks. It takes up loads of space. What if your boot drive is 1TB? Each backup becomes 1TB! This has the upside of being able to restore a fully working system, though.

• Image only the first x (GB/TB/etc) of your boot drive. (UNSAFE)

  This is foolish. If you know for a fact that data isn't stored past a certain point, still don't try this. There is absolutely no protection against you screwing this up. You won't know you've lost data until you try to restore and your system won't boot because the filesystem is corrupted.

• Image your entire boot drive and then compress it.

  You have to first produce a full-size image in order to compress it, but this is slightly better than the other options. This still restores a fully working system but it's smaller than a full image. The downside is that it also has to store all free space.

• Use the technique outlined in this guide

  This only stores allocated space, doesn't require producing a full image (it comes out pre-compressed), and also expands to a lossless, fully working system when restored. It's slightly more complex, but it is worth it.

As you can see, there are either serious restorability tradeoffs (files? how do you restore a working computer from files?) or serious space tradeoffs (1TB of backup for 150GB of used space?) for every method other than the one outlined here.

Hoever, there are drawbacks:

• Linux is required for the backup, but that is true for many of the methods outlined here. That means Intel RST or any other weird gimmicks (that prevent Linux from seeing your drive) must be disabled in the BIOS configuration.
• Backups done this way are slow. Not only do you have to read the entirety of your boot drive, but you have to compress it as well, which uses a decent amount of CPU power. The compression ratio can be tuned for good speed, but that hurts the final output size a bit.

Step-by-step

What we're going to do is first fill the free space with zeros and then make a compressed backup. The zeros will crunch down to almost nothing, so effectively you'll only pay space for what you actually have on the drive. Have a second drive handy to put the backup on, it will have to be formatted as FAT or something that Linux can read. You'll also need a USB drive with some Linux on it, I used Alpine (but wouldn't recommend that for beginners). For beginners you can just use Ubuntu or whatever as long as you have access to a terminal.

• From Windows:

  1. Use Task Manager to locate any programs that are using the disk, and close as many as you can. Many programs will completely wipe your settings if you run out of disk space, even if the space comes back later, so make sure those are all closed.

  2. Clear as much space as you can. You can use a helper program like WizTree for this, which is basically a much faster version of WinDirStat. I deleted my Steam games library which took up around 500GB of space. It can be redownloaded later.

  3. Run this command in WSL1, preferably in /mnt/c or a subdirectory thereof:

     cat /dev/zero > zeros.bin
     

     DO NOT RUN IT IN WSL2 or else you will NOT be able to get your free space back. WSL2 uses a virtual machine hard disk, which does not shrink (without a lot of manual kerfuffery).

     You can use Windows Explorer to monitor your free space. The command will automatically exit when you are completely out.

  4. You can then delete the file:

     rm zeros.bin
     

     This will take a minute.

  5. Immediately reboot into your Linux live USB.

• From your Linux live USB:

  1. Install pv (progress bar) and zstd (compression utility), which will be needed for the backup. (pv isn't strictly needed, but a progress bar is very nice)

  2. Use blkid to locate your boot drive (/dev/nvme0n1 for me) and external drive partition (/dev/sdb1 for me).

  3. Mount your external drive where the backup will be stored:

     mkdir /media/wd
     mount /dev/sdb1 /media/wd
     

  4. Then, run this command to backup your entire boot drive, bit-for-bit, replacing /dev/nvme0n1 with the drive being backed up, and /media/wd/... with a file on the drive where the backup will be stored:

     pv /dev/nvme0n1 | zstd -T12 -5 > /media/wd/ASUS.nvme0n1.5.25.2022.zst
     

     (Try setting -T to the amount of threads your processor has. -5 is the compression level, in my testing 5 is still fast enough to essentially saturate most connections.)

  5. The output file was about 113GB, down from around 350GB of used space on my drive, which has a total capacity of 1TB (!).

Restoring from your backup

Oh noes! Something has gone wrong. Backups are utterly useless if you can't restore from them. Luckily, in this case, it's as simple as booting up into your Linux USB, mounting your backup drive, then running this command:

pv /media/wd/ASUS.nvme0n1.5.25.2022.zst | unzstd > /dev/nvme0n1

(substituting /media/wd/ASUS.nvme0n1.5.25.2022.zst and /dev/nvme0n1 with your backup file and boot drive respectively)

Wait for it to finish, then you'll be able to boot right back up just like nothing ever happened. Then you can either try again, or do something else, or just continue on with your life.