Scalaz RingBuffer adds a high-performance low-latency multi-producer/multi-consumer task queue to RTS for faster and smarter concurrency in ZIO.
ZIO RTS runs on a thread pool executor backed by LinkedBlockingQueue. This has the following consequences:
- Performance and latency are inhibited because: a) waiting on locks cause threads to be de-scheduled, b) traversing a linked list is not cache-friendly.
- Optimizations to Fiber cooperative multi-tasking strategy is limited because the thread pool doesn't expose any metrics on its queue remaining capacity\processing times.
Another approach is to use a lock-free concurrent queue backed by a RingBuffer which would act as a multi-producer\multi-consumer (MPMC) task queue for ZIO RTS. Main objectives are to:
- increase throughput,
- reduce latency,
- expose metrics on size, capacity, processing times.
There are two non-direct competing solutions in the Java world, which can actually be used as an inspiration and case-study:
- CoralQueue is a proprietary "ultra-low-latency, high-performance, lock-free, garbage-free, concurrent queue, demultiplexer, multiplexer, mpmc queue, and splitter".
- LMAX Disruptor is an open-source "high-performance inter-thread messaging library".
The list below is not exhaustive, but is a good starting point:
- Article: The LMAX Architecture by Martin Fowler.
- Article: Yet another implementation of a lock-free circular array queue by Faustino Frechilla is a deep dive into the subject with examples in C++.
- Video: LMAX - How to do 100K TPS at less than 1ms latency by Martin Thompson is the first LMAX talk by one of the authors of LMAX.
- Video: Locks? We Don't Need No Stinkin' Locks! by Michael Barker is about v3 of LMAX that received a proper MPMC support.
- Paper: Draft: Lock-Free Ringbuffer Designs.
- Paper: Simple, Fast, and Practical Non-Blocking and Blocking Concurrent Queue Algorithms