Skip to content

Instantly share code, notes, and snippets.

@AnatomicJC

AnatomicJC/xfs-dedup.md

Last active November 23, 2023 17:21
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save AnatomicJC/d51072e09f4f17c05042f639e7b1f4c6 to your computer and use it in GitHub Desktop.
Save AnatomicJC/d51072e09f4f17c05042f639e7b1f4c6 to your computer and use it in GitHub Desktop.
Download ZIP
XFS reflinks deduplication with duperemove and fdupes
Raw
xfs-dedup.md

First, read this:

http://dashohoxha.fs.al/deduplicating-data-with-xfs-and-reflinks/

Then, some adds:

I recommand the use of fdupes to really save space used by all duplicated files. duperemove can miss some of them.

fdupes -r . | duperemove --fdupes

Finally, once duplicated files deduplicated, you can use same and block options to save more place.

duperemove -hdr --hashfile=/tmp/test.hash --dedupe-options=same,block .

I saved 15Go on a 40Go qemu windows image qcow file with this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment