Skip to content

Instantly share code, notes, and snippets.

@willpower232
Created December 13, 2019 10:06
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save willpower232/e088832f59e8a5841d6ed4e4997bcfbb to your computer and use it in GitHub Desktop.
Save willpower232/e088832f59e8a5841d6ed4e4997bcfbb to your computer and use it in GitHub Desktop.
How to loop through folders in AWS S3, select a file, and download it
#!/bin/bash
target="s3://your-AWS-bucket-here/"
file=""
# if you're looking for a specific file extension
# you can do something like this instead
# while [[ "$target" != *.tgz ]]; do
# while there is a slash at the end of the path
# i.e. it isn't a file
while [[ "$target" == */ ]]; do
list=$(aws s3 ls "$target")
# create an array of options using the
# last "word" in each line of the
# output from aws
options=()
while IFS= read; do
options+=( $(echo "$REPLY" | awk '{print $(NF)}') )
done <<< "$list"
# select an option and append it
select option in "${options[@]}"; do
[ -z "$option" ] && exit 1
target="$target$option"
file="$option"
break
done
done
aws s3 cp $target $file
# do something with $file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment