Skip to content

Instantly share code, notes, and snippets.

@khebbie
Created November 29, 2016 08:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save khebbie/95283577240903a716794b1bf27eb803 to your computer and use it in GitHub Desktop.
Save khebbie/95283577240903a716794b1bf27eb803 to your computer and use it in GitHub Desktop.
Read from kinesis stream
#!/bin/bash
stream_name=$1
while :
do
shardIds=$(aws kinesis describe-stream --stream-name $stream_name | jq '.StreamDescription.Shards[].ShardId' | tr -d '"')
for shard in $shardIds;do
iterator=$(aws kinesis get-shard-iterator --stream-name $stream_name --shard-id $shard --shard-iterator-type LATEST | jq '.ShardIterator' | tr -d '"')
data=$(aws kinesis get-records --shard-iterator "$iterator" | jq ".Records[].Data" | tr -d '""')
for dat in $data; do
echo $dat | base64 -d
done
done
echo "Press [CTRL+C] to stop.."
sleep 1
done
@khebbie
Copy link
Author

khebbie commented Nov 29, 2016

To run this script you need to have the aws cli and jq installed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment