Skip to content

Instantly share code, notes, and snippets.

@makew0rld
Created January 30, 2025 15:54
Show Gist options
  • Select an option

  • Save makew0rld/766602d3c0e3f61be4bee40865df796c to your computer and use it in GitHub Desktop.

Select an option

Save makew0rld/766602d3c0e3f61be4bee40865df796c to your computer and use it in GitHub Desktop.

Locking in your data for archival

  • Hashing protects the integrity of your files
  • Signing proves they came from you, and allows you to author updates

Hashing your files

Use modern secure hash algorithms like SHA-256 or BLAKE3.

To hash many files at once, you can use Bash:

find . -type f -exec b3sum {} \; > checkfile.txt

The output file now contains hashes for all files in the current directory.

Another option is to use my tool merkdir, which will give you a single root hash to represent a directory.

$ merkdir gen -o documents_tree.merkdir ~/Documents
Finding files...
Found 2339 files. Starting hashing...
 100% |████████████████████████████████████████| (1.6/1.6 GB, 5.0 GB/s)        
Root hash: 3e1db8e48dd101bed67ccd117ad011fa76aca26c38ce1ab1612010d5140618b1

Signing

The easiest way to digitally sign files is with Minisign. Create a keypair with minisign -G and publish your public key on official channels (like a website) while keeping your private key secure and private.

You can use this to:

  • Sign individual files: minisign -Sm my_file.txt
  • Sign a group of hashes: minisign -Sm checkfile.txt
  • Sign the root hash of a Merkle tree: merkdir root documents_tree.merkdir > documents.root && minisign -Sm documents.root

The latter two options are much more efficient, because you only need to sign once to attach authorship to potentially hundreds of thousands of files.

Verifying

You can verify hashes by hashing the file and seeing if it matches the stored hash. Most hashing tools have built-in ways to verify checkfiles as well, such as b3sum --check.

Signatures can be verified like so:

$ minisign -Vm myfile.txt -P RWQf6LRCGA9i53mlYecO4IzT51TGPpvWucNSCh1CBM0QTaLn73Y7GFO3
$ minisign -Vm myfile.txt -p signature.pub

This requires the signature myfile.txt.minisig to be present in the same directory.

Note that public verification requires public release of some data. For example, allowing a checkfile to be verified means the verifier must have access to the checkfile, which exposes the hashes and filenames of all your files. The main advantage of merkdir is that individual files can be verified without exposing the others, even though only one signature was made.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment