Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save rmi1974/2e82e59971529d13c97ba463f250ca0c to your computer and use it in GitHub Desktop.
Save rmi1974/2e82e59971529d13c97ba463f250ca0c to your computer and use it in GitHub Desktop.
How to optimize disk space usage with BTRFS #commandlinefu #btrfs #rmlint

How to optimize disk space usage with BTRFS

Enable zstd compression

For general overview see here: BTRFS Compression.

Check zstd compression compression ratio using compsize:

$ sudo compsize $HOME

Processed 4521850 files, 6020362 regular extents (6190661 refs), 1797190 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       77%      945G         1.1T         1.1T       
none       100%      800G         800G         797G       
zstd        33%      144G         424G         429G       
prealloc   100%      8.0K         8.0K         3.8M

Enabling compression does not re-compress existing files. Instead you have to use btrfs filesystem defrag to re-compress them. To force recompression of existing data:

sudo btrfs fi defrag -v -czstd -r $HOME

Statistics after force compress:

$ sudo compsize $HOME

Processed 4521262 files, 5599695 regular extents (5599704 refs), 1797206 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       75%      931G         1.1T         1.1T       
none       100%      778G         778G         778G       
zstd        34%      152G         448G         448G       
prealloc   100%      5.0M         5.0M         3.8M

Note, normal compression has a feature in that if it detects that compression has little to no benefit. It doesn't bother continuing the compression for that particular file (thereby saving wasted CPU). By using compress-force, it attempts compression anyway. Good examples where this applies are for files that are already compressed (multimedia, zip files, etc). So the benefits might be limited. In previous case the savings are still noticeable.

Deduplicate on file level

See BRTFS Deduplication for details. rmlint is a duplicate file finder with btrfs support.

Example which finds duplicates for specific directory and generates shell script for deduplication.

rmlint -g -vv --types="duplicates" --config=sh:handler=clone $HOME

bash rmlint.sh

rm rmlint.sh

Links

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment