Skip to content

Instantly share code, notes, and snippets.

@cm-kajiwara-taishi
Created November 19, 2019 11:41
Show Gist options
  • Save cm-kajiwara-taishi/73c8e796f40b318715e6354772bba782 to your computer and use it in GitHub Desktop.
Save cm-kajiwara-taishi/73c8e796f40b318715e6354772bba782 to your computer and use it in GitHub Desktop.
cfn-kinesisfirehose-waflog.yml
AWSTemplateFormatVersion: '2010-09-09'
Description: Kinesis Firehoses for WAF logs
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: "Kinesis Firehose"
Parameters:
- SizeInMBs
- IntervalInSeconds
- CompressionFormat
- Label:
default: "S3"
Parameters:
- BucketName
- ExpirationInDays
Parameters:
BucketName:
Description: "A name for logs bucket. "
Type: String
ExpirationInDays:
Description: "Indicates the number of days after creation when objects are deleted from Amazon S3"
Type: Number
Default: 90
SizeInMBs:
Description: "The size of the buffer, in MBs, that Kinesis Data Firehose uses for incoming data before delivering it to the destination."
Type: Number
Default: 5
MinValue: 1
MaxValue: 128
IntervalInSeconds:
Description: The length of time, in seconds, that Kinesis Data Firehose buffers incoming data before delivering it to the destination.
Type: Number
Default: 300
MinValue: 60
MaxValue: 900
CompressionFormat:
Description: "The type of compression that Kinesis Data Firehose uses to compress the data that it delivers to the Amazon S3 bucket. "
Type: String
Default: 'ZIP'
AllowedValues: ['GZIP','Snappy','UNCOMPRESSED','ZIP']
Resources:
S3Bucket:
Type: AWS::S3::Bucket
DeletionPolicy: Retain
Properties:
BucketName: !Ref BucketName
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: AES256
LifecycleConfiguration:
Rules:
- Id: !Sub 'ExpirationIn-${ExpirationInDays}Days'
ExpirationInDays: !Ref 'ExpirationInDays'
Status: Enabled
WAFLogDeliveryStream:
Type: AWS::KinesisFirehose::DeliveryStream
Properties:
DeliveryStreamName: !Sub 'aws-waf-logs-${AWS::StackName}'
DeliveryStreamType: DirectPut
S3DestinationConfiguration:
BucketARN: !Sub '${S3Bucket.Arn}'
BufferingHints:
SizeInMBs: !Ref SizeInMBs
IntervalInSeconds: !Ref IntervalInSeconds
CloudWatchLoggingOptions:
Enabled: true
LogGroupName: !Sub '/aws/kinesisfirehose/aws-waf-logs-${AWS::StackName}'
LogStreamName: S3Delivery
CompressionFormat: !Ref CompressionFormat
EncryptionConfiguration:
NoEncryptionConfig: NoEncryption
ErrorOutputPrefix: ''
Prefix: ''
RoleARN: !Sub '${FirehoseRole.Arn}'
WAFLogDeliveryStreamLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub '/aws/kinesisfirehose/aws-waf-logs-${AWS::StackName}'
WAFLogDeliveryStreamLogStream:
Type: AWS::Logs::LogStream
Properties:
LogGroupName: !Ref WAFLogDeliveryStreamLogGroup
LogStreamName: S3Delivery
FirehoseRole:
Type: AWS::IAM::Role
DeletionPolicy: Retain
Properties:
RoleName: !Sub '${AWS::StackName}-FirehoseRole'
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: firehose.amazonaws.com
Policies:
- PolicyDocument:
Version: '2012-10-17'
Statement:
- Action:
- glue:GetTable
- glue:GetTableVersion
- glue:GetTableVersions
Effect: Allow
Resource: '*'
- Action:
- s3:AbortMultipartUpload
- s3:GetBucketLocation
- s3:GetObject
- s3:ListBucket
- s3:ListBucketMultipartUploads
- s3:PutObject
Effect: Allow
Resource:
- !Sub '${S3Bucket.Arn}'
- !Sub '${S3Bucket.Arn}/*'
- arn:aws:s3:::%FIREHOSE_BUCKET_NAME%
- arn:aws:s3:::%FIREHOSE_BUCKET_NAME%/*
- Action: kms:Decrypt
Effect: Allow
Resource: !Sub 'arn:aws:kms:${AWS::Region}:${AWS::AccountId}:key/%SSE_KEY_ID%'
- Action:
- lambda:InvokeFunction
- lambda:GetFunctionConfiguration
Effect: Allow
Resource: !Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:%FIREHOSE_DEFAULT_FUNCTION%:%FIREHOSE_DEFAULT_VERSION%'
- Action: logs:PutLogEvents
Effect: Allow
Resource: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/kinesisfirehose/${S3Bucket}:log-stream:*'
- Action:
- kinesis:DescribeStream
- kinesis:GetShardIterator
- kinesis:GetRecords
Effect: Allow
Resource: !Sub 'arn:aws:kinesis:${AWS::Region}:${AWS::AccountId}:stream/%FIREHOSE_STREAM_NAME%'
PolicyName: firehose_delivery_role_policy
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment