Skip to content

Instantly share code, notes, and snippets.

@holmberd
Created July 15, 2019 16:57
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save holmberd/a714040e1ad70c4cfbd6e8743130cca2 to your computer and use it in GitHub Desktop.
Save holmberd/a714040e1ad70c4cfbd6e8743130cca2 to your computer and use it in GitHub Desktop.
AWS S3 Bucket - List records by date

AWS S3 Bucket - List files by date

With s3 ls

ls-s3.sh:

#!/bin/bash
DATE=$1 || $(date +%Y-%m-%d)
echo "List records for date $DATE"
aws s3 ls s3://bucket-name/directory/ | grep ${DATE}

Usage

  • ./ls-s3.sh - Defaults to todays date ./ls-s3.sh 2019-07-15 - Argument date

With s3api --query

All records for a specific date

aws s3api list-objects-v2 --bucket "bucket-name" --prefix "record-prefix" --query "Contents[?contains(LastModified, '2019-07-15')].{key: Key, date: LastModified}"

All records on and after a specific date

aws s3api list-objects-v2 --bucket "bucket-name" --prefix "record-prefix" --query "Contents[?LastModified>='2019-07-15'].{key: Key, date: LastModified}"
@holmberd
Copy link
Author

@pandeyvinod Yes it will work for large number of objects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment