vhost serve para separar ambientes, um ambiente rabbit para dev, outro prod, etc.
Também pode servir para separar projetos no mesmo servidor.
vhost separa aplicações usando o mesmo RabbitMQ instance.
Different users can have different access privileges to different vhosts and queues,
and exchanges can be created so that they only exist in one vhost.
Acknowledgments and Confirms podem ser emitidos tanto pelos consumidores para o servidor, como vice-versa, do servidor para o producer.
4 tipos de exchanges: Direct, Topic, Fanout, Headers
Direct envia as mensagens diretamente para as queues baseado numa routing key.
Topic filtra as mensagens conforme alguma expressão
Fanout envia para todas as Queues
Headers aplica rotas conforme informações contindas no Header
Durable Exchanges sobrevivem após restarts do servidor, e duram até serem deletadas.
Temporary Exchanges existem até a instância RabbitMQ sofrer shut down.
Auto-deleted Exchanges são removidas assim que o último objeto é desassociado da exchange.
Think of the routing key as an "address" that the exchange uses to decide on how to route the message.
A message goes to the queue(s) that have or has the exact match in binding key to the routing key of the message.
Note: If the message routing key does not match any binding key, the message is discarded.
Default exchange
A default exchange é uma direct exchange pré-declarada, sem nome.
Quando usando a exchange default, a mensagem é entregue para a queue com um nome igual a routing key da mensagem.
Toda queue é automaticamente associada a uma exchange default com uma routing key que corresponda ao nome da queue.
Ou seja, será criada uma routing key com o nome igual ao nome da queue
TOPIC EXCHANGE
Topic exchanges roteiam as mensagens para uma queue baseado num filtro com wildcard
Para rotear precisa uma correspondência entre routing key e routing pattern que é especificado pelo queue binding
Mensagens podem ser roteadas para uma ou mais queues.
The routing key must be a list of words, delimited by a period (.).
The routing patterns may contain an asterisk ("*") to match a word in a specific position of the routing key
A pound symbol ("#") indicates a match on zero or more words
(e.g., a routing pattern of agreements.*.*.b.* only match routing keys where the first word is agreements and the fourth word is "b").
(e.g., a routing pattern of agreements.eu.berlin.# matches any routing keys beginning with agreements.eu.berlin).
O consumidor indica qual tópico está interessado.
O consumidor que cria uma queue.
O consumidor cria a fila e configura um binding com um routing pattern para a exchange.
The consumers indicate which topics they are interested in (like subscribing to a feed of an individual tag).
The consumer creates a queue and sets up a binding with a given routing pattern to the exchange.
All messages with a routing key that match the routing pattern are routed to the queue and stay there until the consumer handles the message.
routing pattern é o filtro, que terá os wildcards, e fica entre a queue e a exchange.
routing key é a informação na mensagem.
Headers Exchange
A special argument named "x-match", added in the binding between exchange and queue, specifies if all headers must match or just one.
The "x-match" property can have two different values: "any" or "all", where "all" is the default value.
A value of "all" means all header pairs (key, value) must match, while value of "any" means at least one of the header pairs must match.
It's worth noting that in a header exchange, the actual order of the key-value pairs in the message is irrelevant.
Dead Letter Exchange
A message is considered "dead" when it has reached the end of it's time-to-live or,
the queue exceeds the max length (messages or bytes) configured for the queue or,
the message has been rejected by the queue or marked by the consumer for some reason and
is not marked for re-queueing.
A dead-lettered message can be republished to an exchange called dead letter exchange.
O roteamento pode ser usando uma routing key especificada para a dead letter exchange ou com a mesma routing key original.
A client may accidentally or maliciously route messages using non-existent routing keys.
To avoid complications from lost information, collecting unroutable messages in a RabbitMQ alternate exchange is an easy, safe backup.
RabbitMQ handles unroutable messages in two ways based on the mandatory flag setting within the message header.
The server either returns the message when the flag is set to "true" or silently drops the message when set to "false".
O servidor retorna a mensagem quando o sinalizador é definido como "verdadeiro" ou elimina silenciosamente a mensagem quando definido como "falso".
Alternate Exchange
RabbitMQ let you define an alternate exchange to apply logic to unroutable messages.
Confirmation Mode
A channel in confirmation mode requires each published message to be ‘acked’ or ‘nacked’ by the server,
thereby indicating that it has been handled.
UI Interface
Unacked are the number of messages for which the server is waiting for acknowledgment.
Features show the parameters for the exchange (e.g D stands for durable, and AD for auto-delete)
Message TTL - The time a message published to a queue can live before being discarded.
o que ocorre com mensagem descartada? se perde?
Auto-expire - The time a queue can be unused before it is automatically deleted.
Max length - How many ready messages a queue can hold before it starts to drop them.
se passar do limite as mensagens serao perdidas?
Max length bytes - The total size of ready messages a queue can hold before it starts to drop them.
Shovel?
Enable lazy queues
Lazy queues are able to support long queues (millions of messages).
Lazy queues, added in RabbitMQ 3.6, write messages to disk immediately.
Messages are only loaded into memory when needed, thereby minimizing the RAM usage, but increasing the throughput time.
É um recurso que evita o servidor ficar sobrecarregado se vier muitas mensagens de uma vez,
com o custo de toda mensagem ser gravada em disco, minimiza o uso de RAM.
We recommend using lazy queues when you know that you will have large queues from time to time
Sending many messages at once (e.g., processing batch jobs) or
if there is a risk consumers will not keep up with the speed of the publishers consistently,
lazy queues are recommended.
Disable lazy queues if high performance is required,
if queues are always short, or if a max-length policy exists.
Queues
Queues são single-thread.
Queues suportam até 50.000 mensagens.
RabbitMQ queues are bound to the node where they are first declared.
All messages routed to a specific queue will end up on the node where that queue resides.
Manually splitting queues evenly between nodes is possible,
but the downside is to remember where each queue is located manually.
Queues são ligadas a um node ao serem criadas, todas as mensagens para essa queue serao roteadas para esse node.
É possível dividir as queues manualmente, mas terá que lembrar onde cada queue está localizada.
Multiple Cores
Two plugins that help if there are multiple nodes or a single node cluster with multiple
cores are the consistent hash exchange plugin and RabbitMQ Sharding.
Plugin Constisten Hash Exchange
O plug-in consistent hash exchange permite o uso de uma exchange para fazer load-balance de mensagens entre as filas.
As mensagens enviadas para essa exchange são distribuídas de forma consistente
e igual em muitas filas com base na chave de roteamento da mensagem.
O plugin cria um hash da chave de roteamento e espalha as mensagens entre as filas que têm uma binding a essa exchange.
Fazer isso manualmente pode se tornar problemático rapidamente
sem adicionar muitas informações sobre o número de filas e suas bindings no publisher.
RabbitMQ sharding
https://github.com/rabbitmq/rabbitmq-sharding.
A queue name starting with amq. is reserved for internal use by the broker.
TTL para fila (time-to-live) policy on the queue.
A TTL policy of 28 days will delete queues that have not had messages consumed from them in the last 28 days.
auto-delete queue é deletada quando o último consumer é cancelado ou quando o channel/connection é fechada.
exclusive queue é usada apenas pela connection que a declarou, e será deletada quando a connection for fechada.
Sending multiple small messages might be a bad alternative.
The better idea could be to bundle the small messages into one larger message and let the consumer split it up.
However, if bundling multiple messages remember that this might affect the processing time.
Se uma das mensagens agrupadas falhar, todas elas precisarão ser reprocessadas?
A largura de banda e a arquitetura ditarão a melhor maneira de configurar as mensagens para considerar a carga útil.
Each connection uses about 100 KB of RAM (and even more, if TLS is used).
Connections should be long-lived so that channels can be opened and closed more frequently, if required.
Even channels should be long-lived if possible.
Do not open a channel every time a message is published.
Custos:
• AMQP connections: 7 TCP packages
• AMQP channel: 2 TCP packages
• AMQP publish: 1 TCP package (more for larger messages)
• AMQP close channel: 2 TCP packages
• AMQP close connection: 2 TCP packages
The client can either ack(knowledge) the message when it receives it, or when the client has completely processed the message.
Se as mensagens forem coisa que precisa ser processado até o fim, então o ack deve ser emitido pelo consumer só após o processamento.
Messages, exchanges, and queues that are not durable and persistent are lost during a broker restart.
Mensagens, exchanges, e queues não são duráveis por default, e são perdidas num restart.
Durable Queues and Persistent Messages
Queues should be declared as "durable" and messages should be sent with delivery mode "persistent".
Use persistent messages and durable queues
If you cannot afford to lose any messages,
make sure that your queue is declared as "durable",
and your messages are sent with delivery mode "persistent"
(delivery_mode=2)
Encriptação
Connecting to RabbitMQ over AMQPS is the AMQP protocol wrapped in TLS.
AMPQS tem problema de performance, é possível VPC no lugar.
PRE-FETCH
The RabbitMQ default pre-fetch setting gives clients an unlimited buffer,
meaning that RabbitMQ by default sends as many messages as it can to any consumer that looks ready to accept them.
Messages that are sent are cached by the RabbitMQ client library in the consumer until processed.
Pre-fetch limits how many messages the client can receive before acknowledging a message.
A large pre-fetch count, can deliver many messages to one single consumer,
and keep other consumers in an idling state.
Creating a CloudAMQP instance with one node. Um RabbitMQ só num node é muito rápido,
visto que as mensagens não são sincronizadas entre nodes.
Creating a CloudAMQP instance with two nodes results in half the performance.
With three nodes created in a CloudAMQP instance. Um Rabbit com 3 nodes será um quarto da performance.
The nodes are located in different availability zones. Quando se cria cluster, cada node precisa ficar em diferentes zonas.
Pause minority is a partition handling strategy in a three node cluster that protects data from being inconsistent due to net-splits.
Pause minority???
If the system has many consumers, and/or a long processing time,
the best practice is to set the pre-fetch count to one (1) to evenly distribute the messages among all resources.
Please note that if a client is set up to auto-ack messages, the pre-fetch value has no effect.
HIPE
Habilitando HIPE, RabbitMQ terá maior throughput, porém, o tempo de startup será maior.
Enabling HiPE means that RabbitMQ is compiled at startup.
REMEMBER TO ENABLE HA ON NEW VHOSTS
Ao criar os clusters, é necessário ter a política HA, para que os nodes sejam sincronizados.
A common mistake on CloudAMQP clusters is that users create a new vhost but forget
to enable an HA-policy for it.
Messages will therefore not be synced between nodes.
Direct Exchange are the fastest
Dead Letter
A queue that is declared with the x-dead-letter-exchange property sends messages
which are either rejected, nacked (negative acknowledged) or expired (with TTL),
to the specified dead-letter-exchange.
Specifying x-dead-letter-routing-key in the routing
key of the message will change it when dead lettered.
Message TTL
Declaring a queue with the x-message-ttl property means that messages will be
discarded from the queue if they haven’t been consumed within the time specified.
Set max-length if needed
Drop or dead-letter messages from the front of the queue (i.e. the oldest messages in the queue).
Split queues over different cores
Better performance is achieved by splitting queues into different cores,
and into different nodes if possible, as queue performance is limited to one CPU core.
Enable lazy queues
Lazy queues write messages to disk immediately,
which spreads the work out over time instead of risking a performance hit somewhere down the line.
The result of a lazy queue is a more predictable, smooth performance curve without sudden drops.
Lazy queues gravam no disco.
O que espalha a carga ao longo do tempo,
ao invés de correr o risco de ter quedas bruscas de performance em um momento no tempo.
O resultado de uma lazy queue é uma curva mais suave e previsível de performance sem quedas repentinas.
RabbitMQ HA – Two (2) nodes
Dois (2) nós são ideais para alta disponibilidade
Quando se trata de mensageria não precisa mais que isso, não há o problema de split brain.
CloudAMQP locates each node in a cluster in different availability zones (AWS).
Additionally, queues are automatically mirrored and replicated (HA) between availability zones.
Optional federation use between clouds
Um Broker de mensageria, um Load Balancer não serão gargalos, não serão pontos de fragilidade do sistema, pois não processam, não possuem regras, então quebrar um broker ou um load balancer é quase impossível, é disponibilidade quase 100%.
Clustering is not recommended between clouds or regions,
and therefore there is no plan at CloudAMQP to spread nodes across regions or datacenters.
If one cloud region goes down, the CloudAMQP cluster also goes down, but this scenario would be extremely rare.
Instead, cluster nodes are spread across availability zones within the same region.
Fazer cluster de mensageria em cloud, ou em regiões muito distante vai ter throughput ou latência, o que torna inviável.
Pode-se utilizar federation para alta disponibilidade, que é um mecanismo diferente do que clusterizar com réplicas.
Management statistics rate mode
Setting the rate mode of RabbitMQ management statistics to ‘detailed’ has a serious impact on performance.
This setting is not recommended in production.
Limited use of priority queues
Each priority level uses an internal queue on the Erlang VM, which consumes resources.
In most uses it is sufficient to have no more than five (5) priority levels.
Policy
A policy is a pattern against which queue names are matched.
Policy is a regular expression used to match queue (or exchange) names.
A policy can apply to an upstream set or a single upstream of exchanges and/or queues.
Protocols
One protocol can be used when publishing while another can be used to consume.
It has no impact on the message itself.
The MQTT protocol, with its minimal design, is perfect for built-in
systems, mobile phones, and other memory and bandwidth sensitive applications.
MQTT
MQ Telemetry Transport is a publish-subscribe, pattern-based "lightweight" messaging protocol.
The protocol is often used in the IoT (Internet of Things) world of connected devices.
MQTT lacks authorization and error notifications from the server to clients,
which are significant limitations in some scenarios.
STOMP
STOMP does not deal with queues and topics; instead, it uses a SEND semantic with a destination string.
HTTP
RabbitMQ can transmit messages over HTTP.
The RabbitMQ-management plugin provides an HTTP-based API for management and monitoring of the RabbitMQ server.
CloudAMQP HTTP assigned port number is 443.
