Benchmark scripts behind 'The Same Matching Engine Measured 130,000/s and 106/s'

README.md

Matching engine benchmark scripts

The scripts behind the numbers in The Same Matching Engine Measured 130,000/s and 106/s. They are copied verbatim from the exchange backend, with one local Docker default credential replaced by guest:guest. results-2026-08-20.md is the run log the article quotes.

Script	What is inside the timer	Needs
matching-domain-performance.js	calculateDeal() in a loop, no I/O. Warms up with 10,000 throwaway iterations first.	The private matching.domain.ts; included to show the harness, not runnable standalone
conductor-performance.js	Publishes N alternating limit bids and asks to matching:commands.p0, waits for the conductor's /metrics counter to reach N, fails on any command failure, reports remaining book depth. BENCH_WAIT_OUTBOX=1 also waits for the event stream to drain.	RabbitMQ, Redis, a running conductor (amqplib, ioredis, superjson)
outbox-drain-performance.js	Times the outbox draining an existing backlog by polling XLEN every 10 ms.	Redis, a running outbox (ioredis)
redis-command-performance.js	MULTI; HSET; ZADD; XADD; EXEC in a loop, sequential per worker, N workers on N connections.	Any Redis-compatible datastore (ioredis)
redis-multicell-performance.js	Same transaction spread over N logical partitions across the URLs given. BENCH_WAITAOF=1 issues WAITAOF 1 1 2000 after every transaction.	Valkey or Redis with AOF and a replica (ioredis)

Running

npm install ioredis amqplib superjson

node matching-domain-performance.js 100000
node conductor-performance.js 1000
BENCH_WAIT_OUTBOX=1 node conductor-performance.js 5000
node outbox-drain-performance.js

BENCH_REDIS_URL=redis://localhost:6379 node redis-command-performance.js 20000 16
BENCH_REDIS_URLS=redis://127.0.0.1:6580,redis://127.0.0.1:6581 \
  BENCH_WAITAOF=1 node redis-multicell-performance.js 10000 16

Overrides: BENCH_RABBIT_URL, BENCH_CONDUCTOR_URL, BENCH_OUTBOX_URL, BENCH_REDIS_HOST, BENCH_REDIS_PORT, BENCH_REDIS_URL, BENCH_REDIS_URLS.

What these numbers are not

Laptop Docker numbers: no network latency, one hot pair, no downstream consumers, no resource limits. The article states the same caveat. Every completed run asserts zero command failures, zero outbox retries, zero dead letters, the expected event count, and an empty book at the end.

conductor-performance.js

#!/usr/bin/env node

const amqp = require('amqplib');
const Redis = require('ioredis');
const SuperJSON = require('superjson').default;
const { randomUUID } = require('crypto');

const count = Number.parseInt(process.argv[2] || '1000', 10);
const rabbitUrl =
  process.env.BENCH_RABBIT_URL ||
  'amqp://guest:guest@localhost:5672';
const conductorUrl =
  process.env.BENCH_CONDUCTOR_URL || 'http://host.docker.internal:3002';
const redisHost = process.env.BENCH_REDIS_HOST || 'host.docker.internal';
const redisPort = Number.parseInt(process.env.BENCH_REDIS_PORT || '6381', 10);
const exchange = 'matching:commands';
const queue = 'matching:commands.p0';
const eventExchanges = [
  'messages:order:create',
  'messages:order:delete',
  'messages:deal:create',
];

if (!Number.isInteger(count) || count <= 0 || count % 2 !== 0) {
  throw new Error('Command count must be a positive even integer');
}

const percentile = (sorted, fraction) =>
  sorted[Math.min(sorted.length - 1, Math.floor(sorted.length * fraction))];

async function matchingKeys(redis) {
  let cursor = '0';
  const keys = [];
  do {
    const response = await redis.scan(
      cursor,
      'MATCH',
      'matching:*',
      'COUNT',
      1000,
    );
    cursor = response[0];
    keys.push(...response[1]);
  } while (cursor !== '0');
  return keys;
}

function command(index, submittedAt) {
  const commandId = randomUUID();
  const side = index % 2 === 0 ? 'bid' : 'ask';
  return {
    schemaVersion: 1,
    commandId,
    correlationId: commandId,
    type: 'order.place',
    submittedAt,
    pairId: 1,
    partition: 0,
    payload: {
      id: `perf-${commandId}`,
      userId: 1,
      quoteVolume: '0',
      quoteDecimals: 8,
      amountDecimals: 4,
      priceDecimals: 2,
      side,
      type: 'limit',
      pairId: 1,
      timestamp: Date.now() + index,
      price: '100',
      role: 'taker',
      volume: '1',
      volumeTotal: '1',
    },
  };
}

async function main() {
  const redis = new Redis({
    host: redisHost,
    port: redisPort,
    maxRetriesPerRequest: 2,
  });
  const staleKeys = await matchingKeys(redis);
  if (staleKeys.length) await redis.del(...staleKeys);

  const before = await (await fetch(`${conductorUrl}/metrics`)).json();
  const connection = await amqp.connect(rabbitUrl);
  const channel = await connection.createConfirmChannel();
  await channel.assertExchange(exchange, 'direct', { durable: true });
  await channel.assertQueue(queue, {
    durable: true,
    arguments: { 'x-single-active-consumer': true },
  });
  await channel.bindQueue(queue, exchange, queue);
  for (const eventExchange of eventExchanges) {
    const sinkQueue = `performance:${eventExchange}`;
    await channel.assertExchange(eventExchange, 'direct', { durable: true });
    await channel.assertQueue(sinkQueue, { durable: true });
    await channel.bindQueue(sinkQueue, eventExchange, eventExchange);
    await channel.purgeQueue(sinkQueue);
  }

  const submittedAt = new Date().toISOString();
  const start = process.hrtime.bigint();
  const publishLatencies = [];
  for (let index = 0; index < count; index += 1) {
    const publishStart = process.hrtime.bigint();
    const writable = channel.publish(
      exchange,
      queue,
      Buffer.from(SuperJSON.stringify(command(index, submittedAt))),
      { persistent: true, contentType: 'application/json' },
    );
    publishLatencies.push(Number(process.hrtime.bigint() - publishStart) / 1e6);
    if (!writable)
      await new Promise((resolve) => channel.once('drain', resolve));
  }
  await channel.waitForConfirms();
  const confirmed = process.hrtime.bigint();

  const target = before.matching_commands_processed_total + count;
  let metrics;
  const deadline = Date.now() + 120_000;
  do {
    metrics = await (await fetch(`${conductorUrl}/metrics`)).json();
    if (
      metrics.matching_command_failures_total >
      before.matching_command_failures_total
    ) {
      throw new Error(
        `Conductor reported ${
          metrics.matching_command_failures_total -
          before.matching_command_failures_total
        } failures`,
      );
    }
    if (metrics.matching_commands_processed_total >= target) break;
    if (Date.now() >= deadline)
      throw new Error(
        `Timed out at ${
          metrics.matching_commands_processed_total -
          before.matching_commands_processed_total
        }/${count}`,
      );
    await new Promise((resolve) => setTimeout(resolve, 20));
  } while (true);

  if (process.env.BENCH_WAIT_OUTBOX === '1') {
    do {
      if ((await redis.xlen('matching:{matching-p0}:events')) === 0) break;
      if (Date.now() >= deadline)
        throw new Error('Timed out waiting for Outbox');
      await new Promise((resolve) => setTimeout(resolve, 20));
    } while (true);
  }

  const end = process.hrtime.bigint();
  const totalMs = Number(end - start) / 1e6;
  const confirmMs = Number(confirmed - start) / 1e6;
  publishLatencies.sort((left, right) => left - right);
  const streamLength = await redis.xlen('matching:{matching-p0}:events');
  const askDepth = await redis.zcard('matching:{matching-p0}:book:1:ask:rates');
  const bidDepth = await redis.zcard('matching:{matching-p0}:book:1:bid:rates');

  console.log(
    JSON.stringify(
      {
        commands: count,
        seconds: totalMs / 1000,
        commandsPerSecond: count / (totalMs / 1000),
        brokerConfirmMs: confirmMs,
        publishCallMs: {
          p50: percentile(publishLatencies, 0.5),
          p95: percentile(publishLatencies, 0.95),
          p99: percentile(publishLatencies, 0.99),
          max: publishLatencies[publishLatencies.length - 1],
        },
        commandFailures:
          metrics.matching_command_failures_total -
          before.matching_command_failures_total,
        eventsCreated:
          metrics.matching_events_created_total -
          before.matching_events_created_total,
        streamLength,
        remainingBookDepth: askDepth + bidDepth,
      },
      null,
      2,
    ),
  );

  await channel.close();
  await connection.close();
  await redis.quit();
}

main().catch((error) => {
  console.error(error);
  process.exitCode = 1;
});

matching-domain-performance.js

#!/usr/bin/env node

require('ts-node/register/transpile-only');

const { calculateDeal } = require('../apps/conductor/src/matching.domain');
const {
  OrderRole,
  OrderSide,
  OrderType,
} = require('../apps/conductor/src/interfaces');

const iterations = Number.parseInt(process.argv[2] || '100000', 10);

const makeOrder = (id, side, role, volume) => ({
  id,
  userId: 1,
  quoteVolume: '0',
  quoteDecimals: 8,
  amountDecimals: 4,
  priceDecimals: 2,
  side,
  type: OrderType.Limit,
  pairId: 1,
  timestamp: 1,
  price: '100',
  role,
  volume,
  volumeTotal: volume,
});

function run(count) {
  let checksum = 0;
  const start = process.hrtime.bigint();
  for (let index = 0; index < count; index += 1) {
    const result = calculateDeal(
      makeOrder(`maker-${index}`, OrderSide.Ask, OrderRole.Maker, '1'),
      makeOrder(`taker-${index}`, OrderSide.Bid, OrderRole.Taker, '1'),
      1_725_000_000_000,
    );
    if (!result.matched) throw new Error('Unexpected no-match result');
    checksum += result.deal.makerVolumeChange === '1' ? 1 : 0;
  }
  return { elapsedNs: process.hrtime.bigint() - start, checksum };
}

run(Math.min(10_000, iterations));
const { elapsedNs, checksum } = run(iterations);
const seconds = Number(elapsedNs) / 1e9;
console.log(
  JSON.stringify(
    {
      calculations: iterations,
      seconds,
      calculationsPerSecond: iterations / seconds,
      checksum,
    },
    null,
    2,
  ),
);

outbox-drain-performance.js

#!/usr/bin/env node

const Redis = require('ioredis');

const redisHost = process.env.BENCH_REDIS_HOST || 'host.docker.internal';
const redisPort = Number.parseInt(process.env.BENCH_REDIS_PORT || '6381', 10);
const outboxUrl =
  process.env.BENCH_OUTBOX_URL || 'http://host.docker.internal:3035';
const streamKey = 'matching:{matching-p0}:events';

async function main() {
  const redis = new Redis({
    host: redisHost,
    port: redisPort,
    maxRetriesPerRequest: 2,
  });
  const initialEvents = await redis.xlen(streamKey);
  if (initialEvents === 0) throw new Error('Outbox stream has no backlog');

  const deadline = Date.now() + 120_000;
  let started;
  let remaining = initialEvents;
  while (remaining > 0) {
    const current = await redis.xlen(streamKey);
    if (!started && current < initialEvents) started = process.hrtime.bigint();
    remaining = current;
    if (Date.now() >= deadline)
      throw new Error(`Timed out with ${remaining} events remaining`);
    if (remaining > 0) await new Promise((resolve) => setTimeout(resolve, 10));
  }
  const end = process.hrtime.bigint();
  const activeDrainMs = Number(end - started) / 1e6;
  const metrics = await (await fetch(`${outboxUrl}/metrics`)).json();
  console.log(
    JSON.stringify(
      {
        events: initialEvents,
        activeDrainSeconds: activeDrainMs / 1000,
        confirmedEventsPerSecond: initialEvents / (activeDrainMs / 1000),
        retries: metrics.matching_outbox_publish_retries_total,
        deadLetters: metrics.matching_outbox_dead_letter_total,
        pending: metrics.matching_outbox_pending,
        remainingStreamLength: metrics.matching_outbox_stream_length,
      },
      null,
      2,
    ),
  );
  await redis.quit();
}

main().catch((error) => {
  console.error(error);
  process.exitCode = 1;
});

redis-command-performance.js

#!/usr/bin/env node

const Redis = require('ioredis');
const { randomUUID } = require('node:crypto');
const { performance } = require('node:perf_hooks');

const url = process.env.BENCH_REDIS_URL || 'redis://host.docker.internal:6379';
const operations = Number.parseInt(process.argv[2] || '10000', 10);
const concurrency = Number.parseInt(process.argv[3] || '1', 10);

if (!Number.isInteger(operations) || operations <= 0) {
  throw new Error('Operations must be a positive integer');
}
if (!Number.isInteger(concurrency) || concurrency <= 0) {
  throw new Error('Concurrency must be a positive integer');
}

async function worker(workerId, count, runId) {
  const redis = new Redis(url, { maxRetriesPerRequest: 2 });
  const tag = `{bench-${runId}-${workerId}}`;
  const ordersKey = `matching:${tag}:orders`;
  const bookKey = `matching:${tag}:book`;
  const streamKey = `matching:${tag}:events`;

  try {
    // Sequential transactions intentionally model one ordered matching partition.
    for (let index = 0; index < count; index += 1) {
      const id = `${workerId}-${index}`;
      await redis
        .multi()
        .hset(ordersKey, id, JSON.stringify({ id, price: '100', volume: '1' }))
        .zadd(bookKey, 100, id)
        .xadd(
          streamKey,
          '*',
          'event',
          JSON.stringify({ type: 'order.opened', id }),
        )
        .exec();
    }
  } finally {
    await redis.del(ordersKey, bookKey, streamKey);
    await redis.quit();
  }
}

async function main() {
  const runId = randomUUID();
  const base = Math.floor(operations / concurrency);
  const remainder = operations % concurrency;
  const started = performance.now();

  await Promise.all(
    Array.from({ length: concurrency }, (_, workerId) =>
      worker(workerId, base + (workerId < remainder ? 1 : 0), runId),
    ),
  );

  const seconds = (performance.now() - started) / 1000;
  console.log(
    JSON.stringify(
      {
        operations,
        concurrency,
        seconds,
        transactionsPerSecond: operations / seconds,
        redisCommandsPerSecond: (operations * 3) / seconds,
      },
      null,
      2,
    ),
  );
}

main().catch((error) => {
  console.error(error);
  process.exitCode = 1;
});

redis-multicell-performance.js

#!/usr/bin/env node

const Redis = require('ioredis');
const { randomUUID } = require('node:crypto');
const { performance } = require('node:perf_hooks');

const urls = (process.env.BENCH_REDIS_URLS ||
  'redis://127.0.0.1:6480,redis://127.0.0.1:6481')
  .split(',')
  .map((url) => url.trim())
  .filter(Boolean);
const operations = Number.parseInt(process.argv[2] || '20000', 10);
const logicalPartitions = Number.parseInt(process.argv[3] || '16', 10);
const waitForDurability = process.env.BENCH_WAITAOF === '1';

if (urls.length === 0) throw new Error('BENCH_REDIS_URLS must contain a URL');
if (!Number.isInteger(operations) || operations < 1) {
  throw new Error('operations must be a positive integer');
}
if (!Number.isInteger(logicalPartitions) || logicalPartitions < 1) {
  throw new Error('logicalPartitions must be a positive integer');
}

async function partitionWorker(partition, count, runId) {
  const redis = new Redis(urls[partition % urls.length], {
    maxRetriesPerRequest: 2,
  });
  const tag = `{matching-bench-${runId}-p${partition}}`;
  const orderKey = `matching:${tag}:orders`;
  const bookKey = `matching:${tag}:book`;
  const eventKey = `matching:${tag}:events`;
  try {
    for (let index = 0; index < count; index += 1) {
      const id = `${partition}-${index}`;
      const result = await redis
        .multi()
        .hset(orderKey, id, `100:${index}`)
        .zadd(bookKey, 100, id)
        .xadd(eventKey, '*', 'eventId', id, 'payload', '{}')
        .exec();
      if (!result || result.some(([error]) => error)) {
        throw new Error(`transaction failed in partition ${partition}`);
      }
      if (waitForDurability) {
        const durable = await redis.call('WAITAOF', 1, 1, 2000);
        if (
          !Array.isArray(durable) ||
          Number(durable[0]) < 1 ||
          Number(durable[1]) < 1
        ) {
          throw new Error(`durability acknowledgement failed: ${durable}`);
        }
      }
    }
  } finally {
    await redis.del(orderKey, bookKey, eventKey);
    await redis.quit();
  }
}

async function main() {
  const runId = randomUUID();
  const base = Math.floor(operations / logicalPartitions);
  const remainder = operations % logicalPartitions;
  const started = performance.now();
  await Promise.all(
    Array.from({ length: logicalPartitions }, (_, partition) =>
      partitionWorker(
        partition,
        base + (partition < remainder ? 1 : 0),
        runId,
      ),
    ),
  );
  const seconds = (performance.now() - started) / 1000;
  process.stdout.write(
    `${JSON.stringify(
      {
        cells: urls.length,
        logicalPartitions,
        operations,
        waitForDurability,
        seconds,
        transactionsPerSecond: operations / seconds,
        redisCommandsPerSecond: (operations * 3) / seconds,
      },
      null,
      2,
    )}\n`,
  );
}

main().catch((error) => {
  console.error(error);
  process.exitCode = 1;
});

results-2026-08-20.md

Conductor performance results — 2026-08-20

Executive result

On an isolated local Docker stack, the single-partition Conductor sustained approximately 308 commands/s across repeated 1,000-command samples and approximately 380 commands/s across two 5,000-command samples. A 10,000 event Outbox backlog drained at 517 confirmed events/s. With Conductor and Outbox active concurrently, 5,000 commands producing 10,000 confirmed events completed at 106 commands/s end to end.

All completed runs had zero command failures, zero Outbox retries, zero dead letters, the expected event count, and zero remaining order-book depth.

Environment

• Apple M3 Max, 14 logical CPUs, 36 GiB RAM
• macOS/Darwin arm64
• Docker Desktop engine 28.1.1
• Node.js 20.20.2 containers
• RabbitMQ 3.9 Management container
• Redis 7 Alpine container
• Conductor and Outbox built from the working tree under review

The host Node.js 26 runtime cannot start this legacy dependency tree because buffer-equal-constant-time expects the removed SlowBuffer API. Benchmarks therefore use Node 20, which is a compatible runtime for the repository.

Workload and measurement

The broker workload alternates equal-price, equal-volume limit bids and asks on one pair. Every two commands fully match, leaving no resting order. This tests the maximum-contention single-writer path rather than parallel unrelated pairs.

The producer uses one RabbitMQ confirm channel and starts the clock before the first publish. Completion is observed from Conductor metrics, not merely from the broker queue becoming empty. Each command creates two events on average. For the end-to-end run, completion additionally requires the Redis event stream to reach zero after RabbitMQ publisher confirmation.

Downstream exchanges are bound to durable benchmark sink queues, but downstream business consumers are not run. Results therefore include Outbox-to-RabbitMQ confirmation, but not consumer database work.

Results

Test	Volume	Time	Throughput	Failures
Warm-up	100 commands	0.409 s	245 commands/s	0
Conductor sample 1	1,000 commands	3.172 s	315 commands/s	0
Conductor sample 2	1,000 commands	3.421 s	292 commands/s	0
Conductor sample 3	1,000 commands	3.171 s	315 commands/s	0
Conductor sustained	5,000 commands	13.089 s	382 commands/s	0
Conductor sustained repeat	5,000 commands	13.239 s	378 commands/s	0
Outbox backlog drain	10,000 events	19.359 s	517 events/s	0 retries
Full pipeline	5,000 commands / 10,000 events	47.211 s	106 commands/s	0

RabbitMQ accepted and confirmed each 5,000-command producer burst in roughly 296–303 ms when Outbox was inactive. In the concurrent full-pipeline run this rose to 1,088 ms. Producer call latency in that run was 0.029 ms p50, 0.342 ms p95, 2.590 ms p99, and 12.296 ms maximum.

Pure domain baseline

Three samples of 100,000 complete calculateDeal calls produced:

• 129,912 calculations/s
• 107,233 calculations/s
• 134,894 calculations/s

This is orders of magnitude above system throughput. Decimal arithmetic and the pure matching function are not the current limiting factors.

Finding discovered by fault injection

Resetting the isolated Redis matching namespace while Outbox was alive removed its stream consumer group. The worker originally retried NOGROUP forever and its metrics endpoint returned HTTP 500. The benchmark exposed this recovery gap. Outbox now recreates a missing group automatically, and its pending metric returns zero while the group is being restored. The failed full-pipeline test was discarded and rerun after rebuilding.

Interpretation

The main bottleneck is confirmed event publication, particularly while Conductor and Outbox share Redis and RabbitMQ. Outbox deliberately publishes and confirms events sequentially to preserve partition order. That is correct but limits one worker to a few hundred events per second. Because this workload creates two events per command, it caps full-pipeline command throughput below the Conductor-only rate.

The current result is a baseline, not a production capacity claim. Docker Desktop loopback, no network latency, one hot pair, no downstream consumers, and no resource limits differ materially from deployment conditions.

Recommended next experiments

1. Add timestamped command completion observations to measure p50/p95/p99 end-to-end latency, not just throughput.
2. Profile Redis command time and RabbitMQ confirm time independently.
3. Test a safe ordered confirm window: publish a bounded contiguous batch, await all confirms, and ACK only the confirmed prefix.
4. Repeat with multiple partitions and unrelated pairs after partition count is configurable.
5. Add downstream consumer/database workloads and duplicate-delivery injection.
6. Run 15–30 minute soak tests with CPU, RSS, Redis memory, queue age, and event loop delay recorded.

Reproduction

With compatible Redis, RabbitMQ, Conductor, and Outbox endpoints available:

node scripts/matching-domain-performance.js 100000
node scripts/conductor-performance.js 1000
BENCH_WAIT_OUTBOX=1 node scripts/conductor-performance.js 5000
node scripts/outbox-drain-performance.js

The scripts accept BENCH_RABBIT_URL, BENCH_CONDUCTOR_URL, BENCH_OUTBOX_URL, BENCH_REDIS_HOST, and BENCH_REDIS_PORT overrides.