Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save GithubOllie/ec61167c32c59656276ce45452ad76d3 to your computer and use it in GitHub Desktop.
Save GithubOllie/ec61167c32c59656276ce45452ad76d3 to your computer and use it in GitHub Desktop.
Cloud Formation: S3 Queue Notification
AWSTemplateFormatVersion: "2010-09-09"
Description: Video Processing Queue Pipeline, S3 -> SQS -> Lambda -> EC2 (GPU instance)
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: Stack Variables
Parameters:
- Environment
- AppName
Parameters:
AppName:
Type: String
Default: swingitbetter-processing-queue
Environment:
Type: String
Default: dev
Resources:
SourceBucket:
Type: AWS::S3::Bucket
DependsOn:
- StandardQueue
- QueuePolicy
Properties:
BucketName: !Join [ "-", [ !Ref AppName, !Ref Environment ] ]
NotificationConfiguration:
QueueConfigurations:
- Event: s3:ObjectCreated:*
Queue: !GetAtt StandardQueue.Arn
QueuePolicy:
Type: AWS::SQS::QueuePolicy
DependsOn:
- StandardQueue
Properties:
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
AWS: '*'
Action:
- SQS:SendMessage
# Target a wildcard resource name based on the same format as QueueName
Resource: !Join [ "", [ "arn:aws:sqs:", !Ref "AWS::Region", ":", !Ref "AWS::AccountId", ":", !Join [ "-", [ !Ref AppName, !Ref Environment ] ], "*" ] ]
Condition:
ArnLike:
# Static BucketName used to avoid circular dependency with S3 bucket
aws:SourceArn: !Join [ "", ["arn:aws:s3:*:*:", !Join [ "-", [ !Ref AppName, !Ref Environment ] ] ] ]
StringEquals:
aws:SourceAccount: !Ref "AWS::AccountId"
Queues:
- !Ref StandardQueue
StandardQueue:
Type: AWS::SQS::Queue
Properties:
DelaySeconds: 0
MaximumMessageSize: 262144
MessageRetentionPeriod: 864000
QueueName: !Join [ "-", [ !Ref AppName, !Ref Environment, standard ] ]
ReceiveMessageWaitTimeSeconds: 0
RedrivePolicy:
deadLetterTargetArn: !GetAtt FailureQueue.Arn
maxReceiveCount: 10
VisibilityTimeout: 90
FailureQueue:
Type: AWS::SQS::Queue
Properties:
DelaySeconds: 0
MaximumMessageSize: 262144
MessageRetentionPeriod: 864000
QueueName: !Join [ "-", [ !Ref AppName, !Ref Environment, "failure" ] ]
ReceiveMessageWaitTimeSeconds: 0
VisibilityTimeout: 500
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment