Skip to content

Instantly share code, notes, and snippets.

View kjnilsson's full-sized avatar

Karl Nilsson kjnilsson

View GitHub Profile
@kjnilsson
kjnilsson / xheaders.md
Last active June 13, 2023 16:27
RabbitMQ x- headers
Name of RabbitMQ header App Type Purpose
x-received-from Federation internal record hop information for cycle detection
x-death Dead lettering internal record dead lettering information (somewhat compacted)
x-stream-offset Streams outgoing annotation include the offset for a given message consumed using AMQP
x-delivery-count Quorum outgoing annotation include the number of delivery attemps for messages consumed from a quorum queue
x-stream-filter Streams incoming annotation include a message filter string used for stream filtering feature
CC, BCC Routing incoming annotation used for including additional routes, not evaluated beyond routing, really ought to be x- headers
x-first-death-reason DLX outgoing annotation dead lettering info
x-first-death-queue DLX outgoing annotation dead lettering queue name

Introduction

There are two types of consumer processing failures:

  1. Poison messages. Messages that are badly formatted and either cause the consuming application to crash or otherwise fail to process the message.

  2. Downstream dependency failures. I.e. when a message is valid but a downstream dependency such as a database isn't available. This category can further be subdived into:

    a. All messages on the queue are destined to the same unavailable downstream.

b. The downstream dependencies may be different depending on the message. (e.g. different databases or integration endpoints).

@kjnilsson
kjnilsson / 10tips.md
Last active December 20, 2021 09:15
10 (8) RabbitMQ tips you may not be aware of

10 RabbitMQ tips you didn't know

  1. Don't retry publishes to Quorum Queues or Streams if the channel is still live.

    Normally we recommend to retry a publish operation if a publisher confirm isn't received within some timeframe. This isn't actually necessary if you know you are only publishing to quorum queues and/or streams and the channel used for publishing is still live. These queue types have their own reliability/resend protocol between the channel and the queue and will ensure publish operations get there once the situation that caused the delay resolves itself (e.g. a temporary unavailability

@kjnilsson
kjnilsson / async_consumer.cs
Last active January 24, 2021 07:17
RabbitMQ .NET client async consumer example
using System;
using System.Threading.Tasks;
using RabbitMQ.Client;
using RabbitMQ.Client.Events;
namespace ConsoleApplication
{
public class Program
{
public static void Main(string[] args)
add two fields to append_entries_rpc:
read_index :: non_neg_integer()
echo :: boolean()
and one field to append_entries_reply:
read_index :: non_neg_integer()
@kjnilsson
kjnilsson / handy.fs
Last active April 6, 2018 17:49
my current favourite handy fsharp helpers
//invaluable for all those Choice<'a, exn> flows
let exnf f = Printf.ksprintf (fun s -> exn s) f
//combine paths
let (</>) x y = System.IO.Path.Combine(x, y)
//allows you to pattern match on values in the current scope rather than just literals
let (|Eq|_|) expected value =
if expected = value then Some ()
else None
@kjnilsson
kjnilsson / otp module sigs
Last active March 21, 2018 18:45
module sigs for otp
module type GenServerType =
type State
val init : ErlangTerm list -> State
...
module MyGenServer : GenServerType =
type State = {Name : string}
let init args =
@kjnilsson
kjnilsson / ra_thw.erl
Last active January 4, 2018 16:09
spreading sampler module
-module(ra_thw).
-export([
% new/1, % size,
new/2, % size, distance
append/3,
to_list/1,
get/2, % kv lookup
truncate/2 % state, first entry to keep
]).
@kjnilsson
kjnilsson / -
Created December 21, 2017 16:52
Checking whether the PLT /Users/knilsson/code/rabbitmq/rabbitmq-public-umbrella-master/deps/rabbit_common/.rabbit_common.plt is up-to-date... yes
Proceeding with analysis...
app_utils.erl:44: The created fun only terminates with explicit exception
app_utils.erl:50: The created fun only terminates with explicit exception
file_handle_cache.erl:1015: The pattern {MemUse, MemLimit} can never match since previous clauses completely covered the type {non_neg_integer(),'infinity'}
file_handle_cache.erl:1016: The pattern {MemUse, MemLimit} can never match since previous clauses completely covered the type {non_neg_integer(),'infinity'}
file_handle_cache.erl:1021: Function reduce_read_cache/2 will never be called
file_handle_cache.erl:1029: Guard test Freed::any() >= MemToFree::none() can never succeed
file_handle_cache_stats.erl:33: Expression produces a value of type atom() | ets:tid(), but this value is unmatched
file_handle_cache_stats.erl:44: Expression produces a value of type [integer()] | integer(), but th
module xmas
open Fez.Core
let greet who =
sprintf "Hi %s" who
type Tid = Tid
// this is simply to create an atom