Skip to content

Instantly share code, notes, and snippets.

@rachejazz

rachejazz/s3_transfer.sh

Created August 22, 2022 22:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rachejazz/9f611b76ee12a0d8ba671a6221dd60bf to your computer and use it in GitHub Desktop.
Save rachejazz/9f611b76ee12a0d8ba671a6221dd60bf to your computer and use it in GitHub Desktop.
Download ZIP
transfers log files from an s3 bucket to another
Raw
s3_transfer.sh
#this script transfers log files from s3 bucket to desired remote server
DIR = '~/var/logs'
COPIED_TO = ''
ssh remote_user@<IP> find $DIR -maxdepth <max depth here> -mtime -$1 -mtime +$2 -type f > loglist.txt
while read -r line
do
rsync -chavzP --stats user@remote.host:$line $COPIED_TO
done < loglist.txt
aws s3 cp $COPIED_TO s3://bucket/ --recursive \
--exclude "*" --include "*.txt"
rm loglist.txt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment