Skip to content

Instantly share code, notes, and snippets.

@holmberd
Created July 15, 2019 16:57
Show Gist options
  • Save holmberd/a714040e1ad70c4cfbd6e8743130cca2 to your computer and use it in GitHub Desktop.
Save holmberd/a714040e1ad70c4cfbd6e8743130cca2 to your computer and use it in GitHub Desktop.
AWS S3 Bucket - List records by date

AWS S3 Bucket - List files by date

With s3 ls

ls-s3.sh:

#!/bin/bash
DATE=$1 || $(date +%Y-%m-%d)
echo "List records for date $DATE"
aws s3 ls s3://bucket-name/directory/ | grep ${DATE}

Usage

  • ./ls-s3.sh - Defaults to todays date ./ls-s3.sh 2019-07-15 - Argument date

With s3api --query

All records for a specific date

aws s3api list-objects-v2 --bucket "bucket-name" --prefix "record-prefix" --query "Contents[?contains(LastModified, '2019-07-15')].{key: Key, date: LastModified}"

All records on and after a specific date

aws s3api list-objects-v2 --bucket "bucket-name" --prefix "record-prefix" --query "Contents[?LastModified>='2019-07-15'].{key: Key, date: LastModified}"
@pandeyvinod
Copy link

it helped.

will this query will also works when you have large numbers of objects( crossing the limit where s3 will stop sorting in s3 bucket in GUI mode) ?

@holmberd
Copy link
Author

@pandeyvinod Yes it will work for large number of objects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment