Skip to content

Instantly share code, notes, and snippets.

@onyxraven
Last active September 12, 2018 21:25
Show Gist options
  • Save onyxraven/166395ae5cd5223d765a5aaa22d02ebf to your computer and use it in GitHub Desktop.
Save onyxraven/166395ae5cd5223d765a5aaa22d02ebf to your computer and use it in GitHub Desktop.
Stream a Kinesis shard on the Commandline
#!/usr/bin/env bash
# prereqs: awscli, awslocal (for now), jq
set -e
streamname=$1
iteratorType=${2:-LATEST}
shard=$(awslocal kinesis describe-stream --stream-name $streamname --query 'StreamDescription.Shards[0].ShardId' --output text)
iterator=$(awslocal kinesis get-shard-iterator --stream-name $streamname --shard-id $shard --shard-iterator-type $iteratorType --output text)
while output=$(awslocal kinesis get-records --shard-iterator $iterator); do
iterator=$(echo $output | jq -r '.NextShardIterator')
records=$(echo $output | jq -r '.Records[].Data')
for record in $records; do
echo "----START RECORD----"
echo $(echo $record | base64 -D)
echo "----END RECORD----"
done
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment