Kafka benchmarks - Confluent

$ kafka-topics --zookeeper localhost:2181 --create --topic test-rep-one -partitions 6 --replication-factor 1
Created topic "test-rep-one".

kafka-producer-perf-test --topic test-rep-one --num-records 50000 --record-size 1000 --throughput 1000 --producer-props acks=-1 bootstrap.servers=$HOSTNAME:9092
5000 records sent, 1000.0 records/sec (0.95 MB/sec), 4.3 ms avg latency, 222.0 max latency.
5005 records sent, 1000.8 records/sec (0.95 MB/sec), 1.6 ms avg latency, 10.0 max latency.
5002 records sent, 1000.2 records/sec (0.95 MB/sec), 1.4 ms avg latency, 23.0 max latency.
5002 records sent, 1000.2 records/sec (0.95 MB/sec), 1.3 ms avg latency, 8.0 max latency.
5001 records sent, 1000.2 records/sec (0.95 MB/sec), 1.2 ms avg latency, 6.0 max latency.
5000 records sent, 1000.0 records/sec (0.95 MB/sec), 1.2 ms avg latency, 5.0 max latency.
5003 records sent, 1000.4 records/sec (0.95 MB/sec), 1.1 ms avg latency, 12.0 max latency.
5002 records sent, 1000.2 records/sec (0.95 MB/sec), 1.1 ms avg latency, 7.0 max latency.
5001 records sent, 1000.0 records/sec (0.95 MB/sec), 1.0 ms avg latency, 7.0 max latency.
50000 records sent, 999.840026 records/sec (0.95 MB/sec), 1.50 ms avg latency, 222.00 ms max latency, 1 ms 50th, 2 ms 95th, 4 ms 99th, 60 ms 99.9th.

kafka-producer-perf-test --topic test-rep-one --num-records 50000 --record-size 1000 --throughput -1 --producer-props acks=1 bootstrap.servers=$HOSTNAME:9092
50000 records sent, 51282.051282 records/sec (48.91 MB/sec), 145.74 ms avg latency, 326.00 ms max latency, 148 ms 50th, 323 ms 95th, 325 ms 99th, 326 ms 99.9th.

kafka-topics --zookeeper localhost:2181 --create --topic test6 --partitions 6 --replication-factor 3
Created topic "test6".

kafka-producer-perf-test --topic test6 --num-records 50000 --record-size 1000 --throughput -1 --producer-props acks=1 bootstrap.servers=$HOSTNAME:9092 buffer.memory=67108864 batch.size=64000
50000 records sent, 59453.032105 records/sec (56.70 MB/sec), 39.12 ms avg latency, 180.00 ms max latency, 14 ms 50th, 142 ms 95th, 149 ms 99th, 151 ms 99.9th.

[lkandrakota@lvn-bi-dvkfka01 ~]$ kafka-producer-perf-test --topic test6 --num-records 5000000 --record-size 1000 --throughput -1 --producer-props acks=1 bootstrap.servers=$HOSTNAME:9092 buffer.memory=67108864 batch.size=64000
905860 records sent, 181172.0 records/sec (172.78 MB/sec), 275.2 ms avg latency, 988.0 max latency.
org.apache.kafka.common.errors.NetworkException: The server disconnected before a response was received.
org.apache.kafka.common.errors.NetworkException: The server disconnected before a response was received.
org.apache.kafka.common.errors.NetworkException: The server disconnected before a response was received.
816662 records sent, 74350.1 records/sec (70.91 MB/sec), 252.2 ms avg latency, 8477.0 max latency.

[lkandrakota@lvn-bi-dvkfka01 ~]$ kafka-producer-perf-test --topic test6 --num-records 5000000 --record-size 1000 --throughput -1 --producer-props acks=-1 bootstrap.servers=$HOSTNAME:9092 buffer.memory=67108864 batch.size=64000
905860 records sent, 181172.0 records/sec (172.78 MB/sec), 275.2 ms avg latency, 988.0 max latency.
816662 records sent, 74350.1 records/sec (70.91 MB/sec), 252.2 ms avg latency, 8477.0 max latency.

$ kafka-producer-perf-test --topic test6 --num-records 5000000 --record-size 1000 --throughput -1 --producer-props acks=-1 bootstrap.servers=$HOSTNAME:9092 buffer.memory=67108864 batch.size=8192
$ kafka-producer-perf-test --topic test6 --num-records 5000000 --record-size 10 --throughput -1 --producer-props acks=1 bootstrap.servers=$HOSTNAME:9092 buffer.memory=67108864 batch.size=8192
$ kafka-producer-perf-test --topic test6 --num-records 5000000 --record-size 10 --throughput -1 --producer-props acks=1 bootstrap.servers=$HOSTNAME:9092 buffer.memory=67108864 batch.size=8192