Skip to content

Instantly share code, notes, and snippets.

@mpneuried
Last active December 27, 2019 07:20
Show Gist options
  • Save mpneuried/3f6e4cdb50e624c437929cf6b22863a0 to your computer and use it in GitHub Desktop.
Save mpneuried/3f6e4cdb50e624c437929cf6b22863a0 to your computer and use it in GitHub Desktop.
AWS SQS Message pump

AWS SQS Message pump

This script will pump AWS SQS Messages from one queue into another queue.

Usage

./pump_sqs.sh {profile} {source-sqs-url} {target-sqs-url} [{iterations}]
  • profile: a aws-cli profile
  • source-sqs-url: the source sqs url. _Example: https://sqs.{ region }.amazonaws.com/{ account-id }/{ queue_name }
  • target-sqs-url: the target sqs url. _Example: https://sqs.{ region }.amazonaws.com/{ account-id }/{ queue_name }
  • iterations (default:1): the iterations

Example

Pump 10 messages from the queue my_source_queue into my_target_queue

./pump_sqs.sh aws_customer_profile https://sqs.eu-central-1.amazonaws.com/666230815013/my_source_queue https://sqs.eu-central-1.amazonaws.com/666230815013/my_target_queue 10
#!/bin/sh
###
# USAGE
# ./pump_sqs.sh {profile} {source-sqs-url} {target-sqs-url} [{iterations}]
#
# Example:
# ./pump_sqs.sh milon https://sqs.eu-central-1.amazonaws.com/405710347540/milon_redrive https://sqs.eu-central-1.amazonaws.com/405710347540/milon 10
###
AWSPROFILE=$1
SQSSOURCEURL=$2
SQSTARGETURL=$3
COUNT=${4:-1}
i=0
while [ $i -lt $COUNT ]
do
echo "\nIteration: $i"
MSG=$( aws --profile $AWSPROFILE sqs receive-message --queue-url $SQSSOURCEURL )
BODY=$(echo $MSG | jq -M -c -r '.Messages[0].Body' )
RHD=$(echo $MSG | jq -M -c -r '.Messages[0].ReceiptHandle' )
MID=$(echo $MSG | jq -M -c -r '.Messages[0].MessageId' )
if [ "$MID" = "null" ]
then
echo "error read message"
echo $MSG
exit 1
else
echo "got message: $MID"
fi
CHECKBODY=${#BODY}
echo "body length $CHECKBODY"
if [ $CHECKBODY -le 5 ]
then
echo "error - empty body.\nProbably empty queue, so wait for 5sec!"
sleep 5
else
WRITE=$(aws sqs --profile=$AWSPROFILE send-message --queue-url $SQSTARGETURL --message-body $BODY )
WMID=$(echo $WRITE| jq '.MessageId')
echo "written msg to target: $WMID"
if [ "$WMID" = "null" ]
then
echo "error on write"
else
$(aws sqs --profile=$AWSPROFILE delete-message --queue-url $SQSSOURCEURL --receipt-handle $RHD ) | jq '.MessageId'
echo "delete message $MID"
fi
fi
true $(( i++ ))
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment