Creating a raspbian image to duplicate on large SD cards

Small-Raspbian-Image-for-Large-SD-Cards.md

(This is compiled from several very helpful stackexchange posts. I spent a while figuring this out, and I don't want to lose it.)

We recently needed several identical raspbian images, all with the same libraries installed, same content in the home directory and the same setup configuration.

The obvious thing to try is to just boot a base raspbian image, set it up as you want it, then remove the card from the Pi and use another machine to copy the contents of the card to create an image, then flash that image onto the other cards you have. There are two problems with this:

1. If your SD cards are large (ours were 32G), this process becomes a bit cumbersome. Flashing the cards takes longer, creating an image from the card can take a really long time, and if you have limited space on your non-pi machine, you might not have anywhere to put a 32G image.
2. Even if two SD cards are nominally the same size, they might not have exactly the same amount of space. This means that if you create your image from a card that's just a little bit bigger, when you put the image on another card, you'll lose some of the filesystem, which will break the image.

So, what we want is to create an image which is set up in the way we want, but which takes up as little space on the SD card as possible.

Even though the base raspbian image is pretty small (we were using stretch-lite, which is less than 2G), when you boot the Pi, it automatically expands the Linux partition and the filesystem to take up the rest of the space on the card. The tiny amount of data for the root filesystem is now dispersed through the whole 32G, and this process is very difficult to reverse.

So, the first thing to do is to prevent the Pi from automatically expanding the filesystem. There are two things to change to prevent this from happening:

1. The first thing is located on the boot partition on the image. The file /boot/cmdline.txt contains the following: init=/usr/lib/raspi-config/init_resize.sh. Delete it.
2. On the root partition, there are two files to delete:

• /etc/init.d/resizefs_once
• /etc/rc3.d/S01resizefs_once, which is a symlink to the first file.

This process shouldn't interfere with your ability to expand the filesystem later using raspi-config. I found that it was easier to make these modifications to the .img file before flashing it onto a card, but I think you can do it either way. This was done on linux, and relies on several command line tools.

In order to modify content of each partition, we have to mount them. To do that, we have to find where in the image each partition starts. This is done with fdisk -l raspbian-image.img, which should output something like:

Device                                Boot Start     End Sectors  Size Id Type
2018-06-27-raspbian-stretch-lite.img1       8192   96663   88472 43.2M  c W95 FAT32 (LBA)
2018-06-27-raspbian-stretch-lite.img2      98304 3637247 3538944  1.7G 83 Linux

This tells us how large the partitions in the image are and where they start and end in terms of sector size, which is also listed in the output of fdisk: Units: sectors of 1 * 512 = 512 bytes.

Using this, along with the start locations, we can figure out the offset of each partition from the start of the image file in terms of bytes, which is what we need to know for mounting.

• for the boot filesystem, the start point would be 512*8192=4194304
• for the linux filesystem, the start point would be 512*98304=50331648

Create mount points for each filesystem, then mount them:

mount -v -o offset=4194304 -t vfat raspbian-image.img /mnt/img/one
mount -v -o offset=50331648 -t ext4 raspbian.img /mnt/img/two

Now that the systems are mounted, you can cd into the mount points and make the changes to prevent automatic filesystem expansion. Once that's done, unmount everything, and you should be able to flash the image onto an SD card.

Now you should be able to set up your pi as you want. However, since the filesystem hasn't expanded, you might run out of space pretty quickly. The solution is to manually expand the partition and then the filesystem just enough so that you have space to set up your Pi, but not so much that it creates an inconveniently large image. I chose to make mine a little under 4GB, but that was mostly a guess about how much space I might need. Also, I was using the stretch-lite image, so if you've got a graphical image, keep in mind that it will need more space.

We can expand the partition manually using fdisk. This is done by deleting the existing partition (scary, I know), and creating a new partition with exactly the same start point, and a new (larger) end point. Use lsblk to check the name of the device with your partitions. For most RPis I think it should be mmcblk0. fdisk is an interactive process, so to start it, run sudo fdisk /dev/mmcblk0.

• First, use p to list the current partitions, so that you know where the linux partition starts.
• use d to delete the current linux partition (it should be ext4)
• use n to create a new partition. Set it's start location to be the same as the old partition, and calculate the end position using the start position and the size you want for the partition do not use the default by not entering a value for the end position-- this will expand to take up the whole SD card!
• (At this point, I think there was some sort of warning popup about whether or not to delete an ext4 something or other. I deleted it. Also, the instructions this is based on said to use a to set the partition to bootable, but I'm pretty sure I forgot to do that, and it's working fine.)
• use w to write the changes

Then reboot your Pi, and run sudo resize2fs /dev/mmcblk0p2 (you should check to make you use the name of your linux partition) to expand the filesystem to take up all of the new space.

Once you've completed all of your setup, you can remove the SD card and save the finalized image as a file on your non-Pi computer. Note: if you're using Mac or Windows and you use an SD card copying utility, it might only recognize the boot partition, but not the linux partition which is ext4. Check the size of the image to make sure it isn't suspiciously small. We used dd to copy the image to a file, but if you use dd and specify only the input and output without additional options, it will attempt to copy the whole card-- even the part that's just empty space. To fix this, we can tell dd exactly how much to copy from the image into the new file. We should be able to check where the partitions end on the card by using fdisk -l on it. Then we can determine the end location in terms of bytes. When you run dd, you can specify a block size. We can also use the count= argument to specify how many blocks to copy. So you'll have to do some math to figure out how many blocks you need for the block size you want and the length of the partitions in bytes. Make sure to round up!

Once that's done, check the size of the image file to make sure it's what you expect. Then you can use it to flash as many SD cards as you need!