Skip to content

Instantly share code, notes, and snippets.

@ranman
Created May 7, 2018 21:28
Show Gist options
  • Save ranman/739ddaae3b81766381f150391a2147ec to your computer and use it in GitHub Desktop.
Save ranman/739ddaae3b81766381f150391a2147ec to your computer and use it in GitHub Desktop.
import json
import time
import boto3
s3 = boto3.resource('s3')
sqs = boto3.resource('sqs')
queue = sqs.get_queue_by_name(QueueName="S3Example")
while True:
for message in queue.receive_messages(
WaitTimeSeconds=10,
VisibilityTimeout=60,
MaxNumberOfMessages=10
):
records = json.loads(message.body).get('Records', [])
for record in records:
s3.meta.client.download_file(
record['s3']['bucket']['name'],
record['s3']['object']['key'],
"/tmp/"+record['s3']['object']['key']
)
message.delete()
time.sleep(1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment