Onyx 0.7.0 Benchmark Specification
Onyx SHA was 4530f76fa4644b48f2baef5d47fe103c4c0b0fec. Benchmark SHA was 3c5040dd89f3c52f5ef3b0f9a5783b44f6d76c1f. We are running on AWS to test Onyx's raw throughput independent of applications that run on top of it. We ran 1 job with 6 tasks and performance tuned it.
The job in question has 1 input task, 4 processing tasks, and 1 output task. Input messages were fabricated by a special plugin generating a 104 byte payload in each message. 4 bytes were allocated for a random integer, and 100 additional bytes were allocated as an opaque payload to simulate a medium sized message. These two values were put into a single map, which creates an Onyx segment. The segments flow through the workflow. The four processing tasks increment the integer by 1. When the message hits the output task, it is discarded. We do this because we're measuring Onyx's throughput independent of any other I/O. All benchmarks used the Aeron transport implementation with an external media driver.
Each benchmark used a 1 node ZooKeeper cluster (c4.large) and a 1 node metrics server that hosted Postgres, Riemann, Graphite, and Grafana (c4.large). Measurements were aggregated locally on the Onyx peer and sent to Riemann every 1,000 milliseconds, after which they were flushed from the peer. Riemann then pushed the furthered aggregated values out to Graphite.
Measurements taken were segments processed per second by all tasks, number of timed-out retries per second, VM memory usage, and VM processor usage.
All servers ran with JVM opts: -server -Xmx10g -XX:+UnlockCommercialFeatures -XX:+FlightRecorder -XX:StartFlightRecording=duration=240s,filename=myrecording.jfr, and an anti-affinity for colocating acking daemons on the same physical machine.
The job used the following workflow:
[[:in :inc1]
[:inc1 :inc2]
[:inc2 :inc3]
[:inc3 :inc4]
[:inc4 :no-op]]And the catalog:
[{:onyx/name :in
:onyx/ident :generator
:onyx/type :input
:onyx/max-pending pending-size
:onyx/medium :generator
:onyx/max-peers n-parallel-input-streams
:onyx/batch-size batch-size
:onyx/doc "Fabricates messages"}
{:onyx/name :inc1
:onyx/fn :onyx-benchmark.peer/my-inc
:onyx/type :function
:onyx/batch-size batch-size}
{:onyx/name :inc2
:onyx/fn :onyx-benchmark.peer/my-inc
:onyx/type :function
:onyx/batch-size batch-size}
{:onyx/name :inc3
:onyx/fn :onyx-benchmark.peer/my-inc
:onyx/type :function
:onyx/batch-size batch-size}
{:onyx/name :inc4
:onyx/fn :onyx-benchmark.peer/my-inc
:onyx/type :function
:onyx/batch-size batch-size}
{:onyx/name :no-op
:onyx/ident :core.async/write-to-chan
:onyx/type :output
:onyx/medium :core.async
:onyx/max-peers n-parallel-output-streams
:onyx/batch-size batch-size
:onyx/doc "Discards messages"}]Further specifications include:
| Attribute | Value |
|---|---|
| Region | us-east-1 |
| Availability Zone | us-east-1e |
| Tenancy | shared |
| Spot pricing | yes |
| Acker percentage | 10 |
| CPU Cores | 8 |
| Virtual Peers per machine | 8 |
| Total number of virtual peers | 40 |
| Subscriber Threads per peer | 1 |
| Log level | WARN |
| Clojure | 1.7.0 |
| Flight Recorder enabled | yes |
| Batch size | 20 |
:onyx/max-pending |
50,000 segments |
:onyx/pending-timeout |
60,000 ms |
| Parallel input streams | 7 |