Record size 10, snappy, 256 partitions:
gp@manager:~/kafka_2.11-0.9.0.1$ ./bin/kafka-producer-perf-test.sh --topic t2-p256-r3 --num-records 10000000 --record-size 10 --throughput -1 --producer-props bootstrap.servers=kafka-broker-1:9092 acks=1 compression.type=snappy
1541930 records sent, 307157.4 records/sec (2.93 MB/sec), 233.3 ms avg latency, 623.0 max latency.
2725936 records sent, 544642.6 records/sec (5.19 MB/sec), 478.3 ms avg latency, 1642.0 max latency.
2769899 records sent, 549256.2 records/sec (5.24 MB/sec), 726.2 ms avg latency, 2077.0 max latency.
2837082 records sent, 565493.7 records/sec (5.39 MB/sec), 456.6 ms avg latency, 1181.0 max latency.
10000000 records sent, 495049.504950 records/sec (4.72 MB/sec), 504.86 ms avg latency, 2077.00 ms max latency, 295 ms 50th, 1406 ms 95th, 1835 ms 99th, 2034 ms 99.9th.
Record size 100, snappy, 256 partitions:
gp@manager:~/kafka_2.11-0.9.0.1$ ./bin/kafka-producer-perf-test.sh --topic t2-p256-r3 --num-records 10000000 --record-size 100 --throughput -1 --producer-props bootstrap.servers=kafka-broker-1:9092 acks=1 compression.type=snappy
1372126 records sent, 272193.2 records/sec (25.96 MB/sec), 671.0 ms avg latency, 1784.0 max latency.
1878686 records sent, 373867.9 records/sec (35.65 MB/sec), 895.3 ms avg latency, 2665.0 max latency.
2589506 records sent, 499711.7 records/sec (47.66 MB/sec), 584.3 ms avg latency, 1737.0 max latency.
2184084 records sent, 435423.4 records/sec (41.53 MB/sec), 402.5 ms avg latency, 1188.0 max latency.
10000000 records sent, 416059.912627 records/sec (39.68 MB/sec), 537.59 ms avg latency, 2665.00 ms max latency, 389 ms 50th, 1453 ms 95th, 1836 ms 99th, 2241 ms 99.9th.
Record size 500, snappy, 256 partitions:
gp@manager:~/kafka_2.11-0.9.0.1$ time ./bin/kafka-producer-perf-test.sh --topic t2-p256-r3 --num-records 10000000 --record-size 500 --throughput -1 --producer-props bootstrap.servers=kafka-broker-1:9092 acks=1 compression.type=snappy
964979 records sent, 192649.0 records/sec (91.86 MB/sec), 132.2 ms avg latency, 599.0 max latency.
1051553 records sent, 210184.5 records/sec (100.22 MB/sec), 76.6 ms avg latency, 258.0 max latency.
1305574 records sent, 260906.1 records/sec (124.41 MB/sec), 87.0 ms avg latency, 197.0 max latency.
1828852 records sent, 365770.4 records/sec (174.41 MB/sec), 118.8 ms avg latency, 301.0 max latency.
1855329 records sent, 370251.2 records/sec (176.55 MB/sec), 114.6 ms avg latency, 383.0 max latency.
1876883 records sent, 375151.5 records/sec (178.89 MB/sec), 107.2 ms avg latency, 266.0 max latency.
10000000 records sent, 300525.920361 records/sec (143.30 MB/sec), 110.43 ms avg latency, 720.00 ms max latency, 94 ms 50th, 222 ms 95th, 382 ms 99th, 561 ms 99.9th.
real 0m34.038s
user 0m47.776s
sys 0m4.812s
Record size 500, lz4, 256 partitions:
gp@manager:~/kafka_2.11-0.9.0.1$ time ./bin/kafka-producer-perf-test.sh --topic t2-p256-r3 --num-records 10000000 --record-size 500 --throughput -1 --producer-props bootstrap.servers=kafka-broker-1:9092 acks=1 compression.type=lz4
1194578 records sent, 237537.9 records/sec (113.27 MB/sec), 710.8 ms avg latency, 1186.0 max latency.
2386432 records sent, 473780.4 records/sec (225.92 MB/sec), 751.0 ms avg latency, 1165.0 max latency.
1940992 records sent, 386960.1 records/sec (184.52 MB/sec), 521.0 ms avg latency, 964.0 max latency.
1378816 records sent, 265719.0 records/sec (126.70 MB/sec), 442.1 ms avg latency, 1094.0 max latency.
2044239 records sent, 405201.0 records/sec (193.21 MB/sec), 846.8 ms avg latency, 1596.0 max latency.
10000000 records sent, 334918.614777 records/sec (159.70 MB/sec), 632.01 ms avg latency, 1596.00 ms max latency, 606 ms 50th, 1148 ms 95th, 1382 ms 99th, 1477 ms 99.9th.
real 0m30.672s
user 0m45.184s
sys 0m3.624s
Record size 500, no compression, 256 partitions:
gp@manager:~/kafka_2.11-0.9.0.1$ time ./bin/kafka-producer-perf-test.sh --topic t2-p256-r3 --num-records 10000000 --record-size 500 --throughput -1 --producer-props bootstrap.servers=kafka-broker-1:9092 acks=1
437196 records sent, 87038.8 records/sec (41.50 MB/sec), 568.2 ms avg latency, 864.0 max latency.
728128 records sent, 145625.6 records/sec (69.44 MB/sec), 417.1 ms avg latency, 679.0 max latency.
654720 records sent, 127700.4 records/sec (60.89 MB/sec), 424.2 ms avg latency, 1038.0 max latency.
486080 records sent, 97060.7 records/sec (46.28 MB/sec), 677.4 ms avg latency, 1453.0 max latency.
813440 records sent, 162525.5 records/sec (77.50 MB/sec), 364.4 ms avg latency, 523.0 max latency.
700352 records sent, 138519.0 records/sec (66.05 MB/sec), 400.4 ms avg latency, 841.0 max latency.
140864 records sent, 28172.8 records/sec (13.43 MB/sec), 1069.2 ms avg latency, 4144.0 max latency.
261888 records sent, 52325.3 records/sec (24.95 MB/sec), 1805.3 ms avg latency, 4139.0 max latency.
480128 records sent, 95987.2 records/sec (45.77 MB/sec), 594.2 ms avg latency, 1171.0 max latency.
511872 records sent, 98989.0 records/sec (47.20 MB/sec), 590.6 ms avg latency, 1555.0 max latency.
91264 records sent, 17972.4 records/sec (8.57 MB/sec), 1826.8 ms avg latency, 4563.0 max latency.
136896 records sent, 27357.3 records/sec (13.04 MB/sec), 2806.3 ms avg latency, 5936.0 max latency.
162688 records sent, 30794.6 records/sec (14.68 MB/sec), 1551.1 ms avg latency, 3279.0 max latency.
123008 records sent, 24048.5 records/sec (11.47 MB/sec), 2243.3 ms avg latency, 4267.0 max latency.
61504 records sent, 12076.2 records/sec (5.76 MB/sec), 4915.9 ms avg latency, 7082.0 max latency.
414656 records sent, 82798.7 records/sec (39.48 MB/sec), 1164.8 ms avg latency, 7067.0 max latency.
486080 records sent, 88716.9 records/sec (42.30 MB/sec), 397.3 ms avg latency, 2967.0 max latency.
87296 records sent, 17448.7 records/sec (8.32 MB/sec), 4817.8 ms avg latency, 7135.0 max latency.
75392 records sent, 15003.4 records/sec (7.15 MB/sec), 1776.0 ms avg latency, 4946.0 max latency.
51584 records sent, 10138.4 records/sec (4.83 MB/sec), 5986.6 ms avg latency, 8129.0 max latency.
97216 records sent, 17503.8 records/sec (8.35 MB/sec), 3031.4 ms avg latency, 6667.0 max latency.
825344 records sent, 164739.3 records/sec (78.55 MB/sec), 612.3 ms avg latency, 5140.0 max latency.
531712 records sent, 106278.6 records/sec (50.68 MB/sec), 502.0 ms avg latency, 1210.0 max latency.
226176 records sent, 43781.6 records/sec (20.88 MB/sec), 1318.5 ms avg latency, 4076.0 max latency.
69440 records sent, 13769.6 records/sec (6.57 MB/sec), 3583.2 ms avg latency, 5331.0 max latency.
41664 records sent, 8167.8 records/sec (3.89 MB/sec), 5133.2 ms avg latency, 6918.0 max latency.
91264 records sent, 17523.8 records/sec (8.36 MB/sec), 3445.0 ms avg latency, 6568.0 max latency.
99200 records sent, 19191.3 records/sec (9.15 MB/sec), 4837.0 ms avg latency, 8332.0 max latency.
539648 records sent, 107800.2 records/sec (51.40 MB/sec), 592.3 ms avg latency, 2886.0 max latency.
10000000 records sent, 65892.213517 records/sec (31.42 MB/sec), 904.06 ms avg latency, 8332.00 ms max latency, 409 ms 50th, 4224 ms 95th, 5969 ms 99th, 7745 ms 99.9th.
real 2m32.616s
user 0m50.376s
sys 0m9.196s
Record size 500, snappy, 3 partitions:
gp@manager:~/kafka_2.11-0.9.0.1$ time ./bin/kafka-producer-perf-test.sh --topic t2-p3-r3 --num-records 10000000 --record-size 500 --throughput -1 --producer-props bootstrap.servers=kafka-broker-1:9092 acks=1 compression.type=snappy
788833 records sent, 157766.6 records/sec (75.23 MB/sec), 58.1 ms avg latency, 303.0 max latency.
1032838 records sent, 206567.6 records/sec (98.50 MB/sec), 26.3 ms avg latency, 171.0 max latency.
1147534 records sent, 229506.8 records/sec (109.44 MB/sec), 14.9 ms avg latency, 88.0 max latency.
1330342 records sent, 266068.4 records/sec (126.87 MB/sec), 13.2 ms avg latency, 122.0 max latency.
1326866 records sent, 265373.2 records/sec (126.54 MB/sec), 10.6 ms avg latency, 76.0 max latency.
1268826 records sent, 253562.4 records/sec (120.91 MB/sec), 11.8 ms avg latency, 63.0 max latency.
1315925 records sent, 263185.0 records/sec (125.50 MB/sec), 10.4 ms avg latency, 78.0 max latency.
1402240 records sent, 280448.0 records/sec (133.73 MB/sec), 7.9 ms avg latency, 58.0 max latency.
10000000 records sent, 240830.383161 records/sec (114.84 MB/sec), 16.56 ms avg latency, 303.00 ms max latency, 7 ms 50th, 56 ms 95th, 160 ms 99th, 213 ms 99.9th.
real 0m42.292s
user 0m58.272s
sys 0m14.560s
Record size 500, snappy, 1 partition (total ordering):
gp@manager:~/kafka_2.11-0.9.0.1$ time ./bin/kafka-producer-perf-test.sh --topic t2-p1-r3 --num-records 10000000 --record-size 500 --throughput -1 --producer-props bootstrap.servers=kafka-broker-1:9092 acks=1 compression.type=snappy
758905 records sent, 151781.0 records/sec (72.37 MB/sec), 23.6 ms avg latency, 226.0 max latency.
1088212 records sent, 217642.4 records/sec (103.78 MB/sec), 12.0 ms avg latency, 41.0 max latency.
1088358 records sent, 217628.1 records/sec (103.77 MB/sec), 11.5 ms avg latency, 41.0 max latency.
1047351 records sent, 209428.3 records/sec (99.86 MB/sec), 18.9 ms avg latency, 129.0 max latency.
1084884 records sent, 216976.8 records/sec (103.46 MB/sec), 13.2 ms avg latency, 61.0 max latency.
1122237 records sent, 224447.4 records/sec (107.02 MB/sec), 11.3 ms avg latency, 48.0 max latency.
1115563 records sent, 223112.6 records/sec (106.39 MB/sec), 12.1 ms avg latency, 62.0 max latency.
1128711 records sent, 225697.1 records/sec (107.62 MB/sec), 9.9 ms avg latency, 40.0 max latency.
1105356 records sent, 221027.0 records/sec (105.39 MB/sec), 12.2 ms avg latency, 56.0 max latency.
10000000 records sent, 212729.748128 records/sec (101.44 MB/sec), 13.27 ms avg latency, 226.00 ms max latency, 9 ms 50th, 36 ms 95th, 93 ms 99th, 120 ms 99.9th.
real 0m47.738s
user 0m58.988s
sys 0m19.660s
Record size 1000, snappy, 256 partitions:
gp@manager:~/kafka_2.11-0.9.0.1$ ./bin/kafka-producer-perf-test.sh --topic t2-p256-r3 --num-records 10000000 --record-size 1000 --throughput -1 --producer-props bootstrap.servers=kafka-broker-1:9092 acks=1 compression.type=snappy
505446 records sent, 100867.3 records/sec (96.19 MB/sec), 227.1 ms avg latency, 797.0 max latency.
989451 records sent, 197771.5 records/sec (188.61 MB/sec), 163.7 ms avg latency, 431.0 max latency.
665430 records sent, 132979.6 records/sec (126.82 MB/sec), 101.9 ms avg latency, 345.0 max latency.
1016261 records sent, 203049.2 records/sec (193.64 MB/sec), 109.1 ms avg latency, 360.0 max latency.
1070331 records sent, 213809.6 records/sec (203.90 MB/sec), 112.1 ms avg latency, 343.0 max latency.
1099843 records sent, 219968.6 records/sec (209.78 MB/sec), 102.9 ms avg latency, 290.0 max latency.
1067603 records sent, 213349.9 records/sec (203.47 MB/sec), 162.2 ms avg latency, 883.0 max latency.
1156771 records sent, 230708.2 records/sec (220.02 MB/sec), 128.2 ms avg latency, 790.0 max latency.
1132119 records sent, 226242.8 records/sec (215.76 MB/sec), 108.1 ms avg latency, 377.0 max latency.
1121638 records sent, 224237.9 records/sec (213.85 MB/sec), 115.8 ms avg latency, 276.0 max latency.
10000000 records sent, 196378.775382 records/sec (187.28 MB/sec), 127.81 ms avg latency, 883.00 ms max latency, 102 ms 50th, 295 ms 95th, 550 ms 99th, 711 ms 99.9th.
Record size 1000, no compression, 256 partitions:
gp@manager:~/kafka_2.11-0.9.0.1$ ./bin/kafka-producer-perf-test.sh --topic t2-p256-r3 --num-records 10000000 --record-size 1000 --throughput -1 --producer-props acks=1 bootstrap.servers=kafka-broker-1:9092
446749 records sent, 88764.0 records/sec (84.65 MB/sec), 279.6 ms avg latency, 834.0 max latency.
431430 records sent, 85397.9 records/sec (81.44 MB/sec), 300.1 ms avg latency, 1491.0 max latency.
574305 records sent, 114608.9 records/sec (109.30 MB/sec), 264.8 ms avg latency, 2109.0 max latency.
493153 records sent, 98218.1 records/sec (93.67 MB/sec), 267.8 ms avg latency, 1294.0 max latency.
254181 records sent, 50836.2 records/sec (48.48 MB/sec), 556.1 ms avg latency, 3780.0 max latency.
256107 records sent, 51129.4 records/sec (48.76 MB/sec), 485.7 ms avg latency, 1890.0 max latency.
362094 records sent, 72418.8 records/sec (69.06 MB/sec), 434.4 ms avg latency, 1904.0 max latency.
341635 records sent, 68313.3 records/sec (65.15 MB/sec), 393.5 ms avg latency, 1551.0 max latency.
332433 records sent, 66433.5 records/sec (63.36 MB/sec), 405.3 ms avg latency, 1783.0 max latency.
223555 records sent, 44612.9 records/sec (42.55 MB/sec), 493.9 ms avg latency, 1992.0 max latency.
60887 records sent, 11742.9 records/sec (11.20 MB/sec), 2675.9 ms avg latency, 6014.0 max latency.
140571 records sent, 27913.2 records/sec (26.62 MB/sec), 1047.2 ms avg latency, 4541.0 max latency.
82595 records sent, 15660.8 records/sec (14.94 MB/sec), 1749.7 ms avg latency, 5085.0 max latency.
84523 records sent, 16441.0 records/sec (15.68 MB/sec), 1641.5 ms avg latency, 4169.0 max latency.
79943 records sent, 14649.6 records/sec (13.97 MB/sec), 1596.7 ms avg latency, 4340.0 max latency.
219230 records sent, 43741.0 records/sec (41.71 MB/sec), 826.6 ms avg latency, 5473.0 max latency.
268112 records sent, 45667.2 records/sec (43.55 MB/sec), 471.2 ms avg latency, 2006.0 max latency.
207696 records sent, 41539.2 records/sec (39.61 MB/sec), 809.2 ms avg latency, 2750.0 max latency.
209163 records sent, 41832.6 records/sec (39.89 MB/sec), 718.5 ms avg latency, 2406.0 max latency.
242179 records sent, 46943.0 records/sec (44.77 MB/sec), 512.1 ms avg latency, 1787.0 max latency.
155591 records sent, 31118.2 records/sec (29.68 MB/sec), 985.4 ms avg latency, 2473.0 max latency.
175509 records sent, 34809.4 records/sec (33.20 MB/sec), 784.5 ms avg latency, 1926.0 max latency.
142797 records sent, 28142.9 records/sec (26.84 MB/sec), 960.9 ms avg latency, 2237.0 max latency.
125529 records sent, 24158.8 records/sec (23.04 MB/sec), 1104.2 ms avg latency, 3718.0 max latency.
136592 records sent, 27247.6 records/sec (25.99 MB/sec), 965.2 ms avg latency, 3210.0 max latency.
192674 records sent, 38450.2 records/sec (36.67 MB/sec), 836.8 ms avg latency, 2857.0 max latency.
183822 records sent, 36764.4 records/sec (35.06 MB/sec), 733.1 ms avg latency, 2302.0 max latency.
270156 records sent, 54009.6 records/sec (51.51 MB/sec), 600.0 ms avg latency, 2028.0 max latency.
147122 records sent, 29424.4 records/sec (28.06 MB/sec), 677.5 ms avg latency, 2439.0 max latency.
190760 records sent, 38113.9 records/sec (36.35 MB/sec), 949.4 ms avg latency, 2765.0 max latency.
167659 records sent, 33525.1 records/sec (31.97 MB/sec), 776.6 ms avg latency, 2278.0 max latency.
107386 records sent, 21417.2 records/sec (20.43 MB/sec), 1077.6 ms avg latency, 3945.0 max latency.
190079 records sent, 36455.5 records/sec (34.77 MB/sec), 947.8 ms avg latency, 2672.0 max latency.
155314 records sent, 31062.8 records/sec (29.62 MB/sec), 796.0 ms avg latency, 2539.0 max latency.
130159 records sent, 26011.0 records/sec (24.81 MB/sec), 1214.2 ms avg latency, 4197.0 max latency.
156426 records sent, 27828.9 records/sec (26.54 MB/sec), 768.8 ms avg latency, 2852.0 max latency.
176376 records sent, 31261.3 records/sec (29.81 MB/sec), 915.2 ms avg latency, 3955.0 max latency.
159487 records sent, 31432.2 records/sec (29.98 MB/sec), 883.0 ms avg latency, 2798.0 max latency.
167222 records sent, 33119.8 records/sec (31.59 MB/sec), 915.3 ms avg latency, 3395.0 max latency.
243047 records sent, 48609.4 records/sec (46.36 MB/sec), 636.0 ms avg latency, 2749.0 max latency.
204717 records sent, 40085.6 records/sec (38.23 MB/sec), 681.3 ms avg latency, 2491.0 max latency.
35904 records sent, 5400.7 records/sec (5.15 MB/sec), 1489.7 ms avg latency, 7914.0 max latency.
261365 records sent, 52075.1 records/sec (49.66 MB/sec), 1103.0 ms avg latency, 8418.0 max latency.
