Skip to content

Instantly share code, notes, and snippets.

@mleukering
Forked from emersonf/s3etag.sh
Last active December 18, 2018 16:36
Show Gist options
  • Save mleukering/26368718908e3c4a0680087d91a8250e to your computer and use it in GitHub Desktop.
Save mleukering/26368718908e3c4a0680087d91a8250e to your computer and use it in GitHub Desktop.
A Bash script to compute ETag values for S3 multipart uploads on OS X.
#!/bin/bash
if [ $# -ne 2 ]; then
echo "Usage: $0 file partSizeInMb";
exit 0;
fi
file=$1
if [ ! -f "$file" ]; then
echo "Error: $file not found."
exit 1;
fi
partSizeInMb=$2
fileSizeInMb=$(du -m "$file" | cut -f 1)
parts=$((fileSizeInMb / partSizeInMb))
if [[ $((fileSizeInMb % partSizeInMb)) -gt 0 ]]; then
parts=$((parts + 1));
fi
checksumFile=$(mktemp -t s3md5.XXXX)
for (( part=0; part<$parts; part++ ))
do
skip=$((partSizeInMb * $part))
$(dd bs=1M count=$partSizeInMb skip=$skip if="$file" 2>/dev/null | md5sum >>$checksumFile)
done
echo $(xxd -r -p $checksumFile | md5sum | tr -d '[:space:]')$parts
rm $checksumFile
@mleukering
Copy link
Author

Modified to work with linux. Tested on Ubuntu 18.04.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment