Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
Command line utility to sync the current directory with an S3 bucket, compressing and metadata'ing files as we go...
TMP_DIR=`mktemp -d -t gzsync`
# check that we have a trailing slash
[[ $BUCKET != */ ]] && BUCKET="$BUCKET"/
printf "Copying files to temporary directory... "
cp -R . $TMP_DIR && cd $TMP_DIR
printf "Done\n"
if [ "$PWD" = "$TMP_DIR" ]
for filename in $(find . -type f | sed "s/\.\///g" | grep -v git | grep -v .DS_Store)
printf "${filename}... "
gzip -t ${filename} > /dev/null 2>&1 || { gzip -9 ${filename}; mv ${filename}.gz ${filename}; }
s3cmd put ${filename} ${BUCKET}${filename} --add-header "Content-Encoding: gzip" --acl-public > /dev/null 2>&1
printf "Done\n"
printf "Cleaning up... "
cd && rm -rf ${TMP_DIR}
printf "Done\n"
echo "ERROR: Failed to change to current directory, exiting" 1>&2

This comment has been minimized.

Copy link

commented Mar 14, 2015

This is really slick! It was a huge help in getting compression going on my Jekyll site. Thanks!

I posted a forked version where I added the ability to exclude files from compression based on a file extension:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.