Skip to content

Instantly share code, notes, and snippets.

@kjnilsson
Last active April 9, 2020 20:45
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save kjnilsson/159c643fb34604f8ea20be336109261b to your computer and use it in GitHub Desktop.
Save kjnilsson/159c643fb34604f8ea20be336109261b to your computer and use it in GitHub Desktop.
Shovelling a message from RabbitMQ to Azure ServiceBus

Basics

Azure ServiceBus can be used with the AMQP 1.0 protocol. Since version 3.7 RabbitMQ supports shovels where either the source or destination (or both) uses AMQP 1.0. Hence it is possible to connect the two systems together using shovels. In this brief tutorial we are going to shovel a message from a RabbitMQ queue to a queue in Azure ServiceBus (SB).

First get the SB connection string from the Microsoft Azure portal. Typically it will look something like:

Endpoint=sb://shoveltest.servicebus.windows.net/;SharedAccessKeyName=TheUser;SharedAccessKey=Some/String=;EntityPath=aqueue1

The RabbitMQ shovel is normally configured using an amqp uri. SB reuquires TLS and uses port 5671, it also has restrictions on the versions of TLS it supports which may or may not be a problem depending on the version of erlang RabbitMQ is running under.

Given this we can translate the connection string into an amqp URI as follows:

amqps://TheUser:Some%2FString=@shoveltest.servicebus.windows.net:5671=versions=tlsv1.0,tlsv1.1,tlsv1.2

Note we restricted the tls versions it should attempt to use as well as URI encoded the UserName and Password as Azure kindly include a lot of forward slashes into these.

This URI can now be used to configure the shovel URI together with the queue name as either the source or destination assuming the SB queue is not configured to use sessions or partitions.

Sessions and Partitions

To use SB sessions and partitions with the shovel some additional configuration is required. The current best way to do so is by using the static shovel configuration to add the required properties and/or message annotations.

Sessions

To use sessions the group-id property needs to be set apropriately. E.g:

{destination,
   [{protocol, amqp10},
    {uris, ["amqps://TheUser:Some%2FString=@shoveltest.servicebus.windows.net:5671?versions=tlsv1.0,tlsv1.1,tlsv1.2"]},
    {target_address, <<"aqueue1">>}
    {properties, [{group_id, <<"some-group">>}]},

Partitions

To use a partition the x-opt-partition-key property needs to be set apropriately. E.g:

{destination,
   [{protocol, amqp10},
    {uris, ["amqps://TheUser:Some%2FString=@shoveltest.servicebus.windows.net:5671?versions=tlsv1.0,tlsv1.1,tlsv1.2"]},
    {target_address, <<"aqueue1">>}
    {message_annotations, [{"x-opt-partition-key", 12345}]},     
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment