Skip to content

Instantly share code, notes, and snippets.

@huytv593
Forked from brianhempel/all_s3_objects.rb
Created June 28, 2017 09:49
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save huytv593/90a290a93f1bc1235b6062dc847b7203 to your computer and use it in GitHub Desktop.
Save huytv593/90a290a93f1bc1235b6062dc847b7203 to your computer and use it in GitHub Desktop.
List/fetch all objects in a bucket with AWS::S3 Ruby gem
# by default you only get 1000 objects at a time
# so you have to roll your own cursor
S3.connect!
objects = []
last_key = nil
begin
new_objects = AWS::S3::Bucket.objects(bucket_name, :marker => last_key)
objects += new_objects
last_key = objects.last.key
end while new_objects.size > 0
# you can easily define the above as an all_objects method on AWS::S3::Bucket
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment