Skip to content

Instantly share code, notes, and snippets.

@qi-qi

qi-qi/firehose-s3-prefix.txt

Last active April 17, 2019 18:27
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save qi-qi/902f78b728bfe83a13e4430d86d60094 to your computer and use it in GitHub Desktop.
Save qi-qi/902f78b728bfe83a13e4430d86d60094 to your computer and use it in GitHub Desktop.
Download ZIP
AWS Firehose Prefix
Raw
firehose-s3-prefix.txt
aws firehose update-destination --delivery-stream-name 'test' --current-delivery-stream-version-id '7' --destination-id 'destinationId-000000000001' --extended-s3-destination-update '{"Prefix":"test_hose/dt=!{timestamp:yyyy'-'MM'-'dd}/h=!{timestamp:HH}/","ErrorOutputPrefix":"test_hose/processing-failed/dt=!{timestamp:yyyy'-'MM'-'dd}/h=!{timestamp:HH}/!{firehose:error-output-type}"}'
aws firehose update-destination --delivery-stream-name 'requests-batch-all' --current-delivery-stream-version-id '16' --destination-id 'destinationId-000000000001' --extended-s3-destination-update '{"Prefix":"dt=!{timestamp:yyyy'-'MM'-'dd}/hh=!{timestamp:HH}/","ErrorOutputPrefix":"processing-failed/dt=!{timestamp:yyyy'-'MM'-'dd}/hh=!{timestamp:HH}/!{firehose:error-output-type}"}'
aws firehose describe-delivery-stream --delivery-stream-name test
https://docs.aws.amazon.com/firehose/latest/dev/s3-prefixes.html
https://aws.amazon.com/about-aws/whats-new/2019/02/amazon-kinesis-data-firehose-announces-support-for-custom-amazon-s3-prefixes/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment