Skip to content

Instantly share code, notes, and snippets.

@jowagner
Created February 23, 2022 17:13
Show Gist options
  • Save jowagner/4fa63bfedf0b3d9389550995505871c9 to your computer and use it in GitHub Desktop.
Save jowagner/4fa63bfedf0b3d9389550995505871c9 to your computer and use it in GitHub Desktop.
Helper for punching holes into a disk image where partclone says that blocks are not in use
#!/usr/bin/env python
import sys
offset = int(sys.argv[1])
img_name = sys.argv[2]
# skip current_pos
while True:
line = sys.stdin.readline()
if not line: break
if line.startswith('#'): continue
# reached first non-comment line
# --> this is the current_pos line
break
# process ranges
while True:
line = sys.stdin.readline()
if not line: break
if line.startswith('#'): continue
fields = line.split()
if fields[2] == '+': continue
assert fields[2] == '?'
# found free area
start = int(fields[0], 16)
length = int(fields[1], 16)
sys.stdout.write('fallocate -p -l %d -o %d %s\n' %(
length, start + offset, img_name
))
@jowagner
Copy link
Author

jowagner commented Feb 24, 2022

Creates fallocate --punch-hole commands for each byte range reported as unused (status ?) by a domain mapfile (provided on stdin) such as produced by partclone.[fstype] -D. A non-zero offset can be specified to account for headers, e.g. when punching holes into a LUKS-encrypted container. If using with LVM or software RAID carefully check that the payload is indeed stored sequentially.

This can be used to make file areas sparse that are not in use by the filesystem in the image file, freeing disk space. For plain images, a frequently suggested alternative is to first write zero bytes to the free space (by writing large, non-sparse files), then using fallocate --dig-holes or cp --sparse=always to create a sparse image and finally deleting the zero files inside. For LUKS containers, this approach cannot work as one does not know what data one must write in order for a block of zero bytes to come out of the encryption.

Example:

losetup --find --show filesystem.img 
partclone.btrfs -D -s /dev/loop0 -o filesystem.map
../domain-map-2-punch-holes.py 0 filesystem.img < filesystem.map > punch-holes.sh
losetup -d /dev/loop0
bash ./punch-holes.sh 
losetup --find --show filesystem.img 
btrfs check /dev/loop0
mount /dev/loop0 /mnt
btrfs scrub /mnt

Known issue: Thomas-Tsai/partclone#180

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment