Skip to content

Instantly share code, notes, and snippets.

@mpneuried

mpneuried/README.md

Last active Dec 27, 2019
Embed
What would you like to do?
AWS SQS Message pump

AWS SQS Message pump

This script will pump AWS SQS Messages from one queue into another queue.

Usage

./pump_sqs.sh {profile} {source-sqs-url} {target-sqs-url} [{iterations}]
  • profile: a aws-cli profile
  • source-sqs-url: the source sqs url. _Example: https://sqs.{ region }.amazonaws.com/{ account-id }/{ queue_name }
  • target-sqs-url: the target sqs url. _Example: https://sqs.{ region }.amazonaws.com/{ account-id }/{ queue_name }
  • iterations (default:1): the iterations

Example

Pump 10 messages from the queue my_source_queue into my_target_queue

./pump_sqs.sh aws_customer_profile https://sqs.eu-central-1.amazonaws.com/666230815013/my_source_queue https://sqs.eu-central-1.amazonaws.com/666230815013/my_target_queue 10
#!/bin/sh
###
# USAGE
# ./pump_sqs.sh {profile} {source-sqs-url} {target-sqs-url} [{iterations}]
#
# Example:
# ./pump_sqs.sh milon https://sqs.eu-central-1.amazonaws.com/405710347540/milon_redrive https://sqs.eu-central-1.amazonaws.com/405710347540/milon 10
###
AWSPROFILE=$1
SQSSOURCEURL=$2
SQSTARGETURL=$3
COUNT=${4:-1}
i=0
while [ $i -lt $COUNT ]
do
echo "\nIteration: $i"
MSG=$( aws --profile $AWSPROFILE sqs receive-message --queue-url $SQSSOURCEURL )
BODY=$(echo $MSG | jq -M -c -r '.Messages[0].Body' )
RHD=$(echo $MSG | jq -M -c -r '.Messages[0].ReceiptHandle' )
MID=$(echo $MSG | jq -M -c -r '.Messages[0].MessageId' )
if [ "$MID" = "null" ]
then
echo "error read message"
echo $MSG
exit 1
else
echo "got message: $MID"
fi
CHECKBODY=${#BODY}
echo "body length $CHECKBODY"
if [ $CHECKBODY -le 5 ]
then
echo "error - empty body.\nProbably empty queue, so wait for 5sec!"
sleep 5
else
WRITE=$(aws sqs --profile=$AWSPROFILE send-message --queue-url $SQSTARGETURL --message-body $BODY )
WMID=$(echo $WRITE| jq '.MessageId')
echo "written msg to target: $WMID"
if [ "$WMID" = "null" ]
then
echo "error on write"
else
$(aws sqs --profile=$AWSPROFILE delete-message --queue-url $SQSSOURCEURL --receipt-handle $RHD ) | jq '.MessageId'
echo "delete message $MID"
fi
fi
true $(( i++ ))
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment