Skip to content

Instantly share code, notes, and snippets.

@Gsantomaggio
Last active October 25, 2022 20:33
Show Gist options
  • Save Gsantomaggio/149b604113bf9d2e044a0f89531c101f to your computer and use it in GitHub Desktop.
Save Gsantomaggio/149b604113bf9d2e044a0f89531c101f to your computer and use it in GitHub Desktop.

Stream-Core and Stream-Plugin


Stream queues are divided into two sections:

Stream-Core queues

New built-in queue type. Available with RabbitMQ 3.9. It is possible to use the standard AMQP 0.9.1 clients to read/write messages. See the documentation and here some example.

Interoperabilty

Stream queues store the messages in AMQP 1.0 format. With a AMQP 0.9.1 client:

  • Publish: it pushes 0.9.1 messages and converted into 1.0 and stored.
  • Consumers: converted from 1.0 to 0.9.1 and send to the conumsers

Stream Plugin

Can be enabled with:

rabbitmq-plugins enable rabbitmq_stream
rabbitmq-plugins enable rabbitmq_stream_management

With the plugin is possible to use the RabbitMQ Stream protocol with dedicated clients:

Core-Stream vs Stream-Plugin

Feature Stream-Core Stream-Plugin
Activation Built-in Must be enabled
Protocol AMQP 0.9.1 RabbitMQ Stream
Client library AMQP 0.9.1 RabbitMQ Stream
Port 5672 5552
Format consume AMQP 0.9.1/1.0 messages consume AMQP 0.9.1/1.0 messages
Sub-Entry batching Not supported Supported
Offset tracking Use external store Built-in server side
Publishing deduplication Not supported Supported
Throughput Hundreds of thousands per second Millions messages per second
TLS Supported (5671-Port) Supported (5551-Port)

See here for all the clients: https://www.rabbitmq.com/devtools.html

Interoperabilty

AMQP 0.9.1 and Stream clients can read and write to same same stream queue with some limitation:

  • Sub-Entry messages: only not compressed

  • AMQP 1.0 format only:

       1. ApplicationData
    
       2. MessageAnnotations
    
       3. MessageProperties
    
       4. ApplicationProperties 
    

AmqpValue is not supported.

Limitations

https://www.rabbitmq.com/streams.html#limitations

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment