Skip to content

Instantly share code, notes, and snippets.

@badsyntax
Forked from emersonf/s3etag.sh
Last active December 16, 2021 13:47
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save badsyntax/1bfd3a47a33bbe7a810a3cbf2542f8b5 to your computer and use it in GitHub Desktop.
Save badsyntax/1bfd3a47a33bbe7a810a3cbf2542f8b5 to your computer and use it in GitHub Desktop.
Generate an S3 ETAG for a multipart upload file, on MacOS
#!/bin/bash
# Generate an S3 ETAG for a multipart upload file.
#
# From: https://gist.github.com/emersonf/7413337
# Author: Emerson Farrugia <https://github.com/emersonf>
# Modified by: Richard Willis <https://github.com/badsyntax>
# Modifications: Work with bytes instead of MB.
#
# MacOS only!
if [ $# -ne 2 ]; then
echo "Usage: $0 file partSizeInBytes"
exit 0
fi
file=$1
if [ ! -f "$file" ]; then
echo "Error: $file not found."
exit 1
fi
partSizeInBytes=$2
fileSizeInBytes=$(stat -f%z "$file")
parts=$((fileSizeInBytes / partSizeInBytes))
if [[ $((fileSizeInBytes % partSizeInBytes)) -gt 0 ]]; then
parts=$((parts + 1))
fi
checksumFile=$(mktemp -t s3md5)
for ((part = 0; part < $parts; part++)); do
skip=$((partSizeInBytes * part))
$(dd bs=1 count=$partSizeInBytes skip=$skip if="$file" 2>/dev/null | md5 >>$checksumFile)
done
echo $(xxd -r -p $checksumFile | md5)-$parts
rm $checksumFile
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment