Skip to content

Instantly share code, notes, and snippets.

@tiborvass
Forked from emersonf/s3etag.sh
Created January 31, 2020 00:49
Show Gist options
  • Save tiborvass/24e5e90380f16b63ac90b4671c7dcaaa to your computer and use it in GitHub Desktop.
Save tiborvass/24e5e90380f16b63ac90b4671c7dcaaa to your computer and use it in GitHub Desktop.
A Bash script to compute ETag values for S3 multipart uploads on OS X.
#!/bin/bash
if [ $# -ne 2 ]; then
echo "Usage: $0 file partSizeInMb";
exit 0;
fi
file=$1
if [ ! -f "$file" ]; then
echo "Error: $file not found."
exit 1;
fi
partSizeInMb=$2
fileSizeInMb=$(du -m "$file" | cut -f 1)
parts=$((fileSizeInMb / partSizeInMb))
if [[ $((fileSizeInMb % partSizeInMb)) -gt 0 ]]; then
parts=$((parts + 1));
fi
if [[ "$parts" = "1" ]]; then
cat "$file" | md5
else
checksumFile=$(mktemp -t s3md5)
for (( part=0; part<$parts; part++ ))
do
skip=$((partSizeInMb * part))
$(dd bs=1m count=$partSizeInMb skip=$skip if="$file" 2>/dev/null | md5 >>$checksumFile)
done
echo $(xxd -r -p $checksumFile | md5)-$parts
rm $checksumFile
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment