Skip to content

Instantly share code, notes, and snippets.

View RaulGracia's full-sized avatar

Raúl Gracia RaulGracia

25 followers · 1 following
View GitHub Profile
@RaulGracia
RaulGracia / variability.csv
Last active September 28, 2020 17:35
100B events Pravega (1 Seg.) 100B events Pravega (1 Seg.)
Requested TP Measured TP P50 P95 P99 Requested TP Measured TP P50 P95 P99
(10/09/2020) 1000 1000.5 1.9 2.8 3 (10/09/2020) Unthrottled 995020.3 5.4 7.5 8.7
(20/08/2020) 1000 1000.5 1.8 2.7 3 (20/08/2020) Unthrottled 1088080.4 5.4 7.7 11.7
(19/08/2020) 1000 1000.5 1.9 2.9 4 (19/08/2020) Unthrottled 953208.7 5.1 7.1 8
(18/08/2020) 1000 1000.5 1.9 3 5 (18/08/2020) Unthrottled 1008320.5 5.8 7.5 9.4
@RaulGracia
RaulGracia / confluent_kafka_driver.patch
Last active September 16, 2020 08:46
From 530c0ac8847720180f857ff58846d181a4cb3cec Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ra=C3=BAl=20Gracia?= <raul.gracia@emc.com>
Date: Wed, 16 Sep 2020 10:41:41 +0200
Subject: [PATCH] Adapt Confluent Kafka driver to our benchmark methodology
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Signed-off-by: Raúl Gracia <raul.gracia@emc.com>
---
@RaulGracia
RaulGracia / kafka_pulsar_pravega_nokey_read_latency_vs_throughput.csv
Created September 11, 2020 09:30
We can make this file beautiful and searchable if this error is corrected: It looks like row 8 should actually have 26 columns, instead of 2. in line 7.
100B events,"Pravega (16 Seg., no keys)",,,,"Kafka ""no flush"" (16 Seg., no keys)",,,,"Pulsar ""batch"" (16 Seg., no keys)",,,,10KB events,"Pravega (16 Seg., no keys)",,,,"Kafka ""no flush"" (16 Seg., no keys)",,,,"Pulsar ""batch"" (16 Seg., no keys)",,,
Requested TP,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Requested TP,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99
100,100.1,2,3,3,100.057,2.535,3.025,3.165,100.1,3,4,5,1000,1000.6,2,3,3,1000.976,2.159,3.061,3.231,1000.6,3,4,5
1000,1000.4,2,2,3,1000.768,2.004,2.914,3.089,1000.6,3,4,4,3000,3004.5,2,3,6,3004.829,2.595,3.569,3.828,3004.5,3,4,5
5000,5005.2,2,3,5,5004.553,2.042,2.951,3.087,5003.6,3,4,4,6000,6014.1,3,5,14,6009.826,3.293,4.458,4.863,6010.3,3,5,5
10000,10010.6,2,6,9,10006.471,2.201,2.998,3.148,10005.6,3,4,5,9000,9008.6,3,8,36,9011.107,3.58,4.644,5.166,9013.9,4,5,5
50000,50027.4,3,7,11,50037.684,2.17,3.138,3.273,50029,3,5,7,12000,12016.2,4,10,35,12013.116,4.038,5.268,6.345,12019.2,4,5,6
100000,10
@RaulGracia
RaulGracia / kafka_pulsar_pravega_read_latency_vs_throughput.csv
Last active September 15, 2020 14:15
We can make this file beautiful and searchable if this error is corrected: It looks like row 8 should actually have 25 columns, instead of 18. in line 7.
100B events,Pravega (1 Segment),,,,"Kafka ""no flush"" (1 Segment)",,,,"Pulsar ""batch"" (1 Segment)",,,,Pravega (16 Segments),,,,"Kafka ""no flush"" (16 Segments)",,,,"Pulsar ""batch"" (16 Segments)",,,
Requested TP,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99
100,100,2,2,3,100.05,2.495,2.954,3.084,100.1,8,12,13,100.1,2,3,3,100.057,2.633,3.136,3.289,100.1,3,4,4
1000,1000.5,2,3,3,1000.648,1.829,2.748,2.846,1000.7,9,13,14,1000.6,2,2,3,1000.726,2.116,2.779,3.006,1000.6,4,12,13
5000,5002.8,2,4,4,5003.864,1.993,2.725,2.854,5003.6,9,13,14,5005,2,3,4,5004.349,2.147,2.737,3.048,5003.4,8,13,14
10000,10005.4,2,4,4,10006.805,2.088,2.736,2.863,10006.4,9,13,14,10010.6,2,5,9,10007.772,2.316,4.218,5.994,10006.6,8,13,14
50000,50026.2,3,4,5,50034.84,2.126,3.142,3.204,50032.1,9,13,14,50037.7,3,7,10,50053.76,5.332,10.006,41.082,50027.7,9,13,14
100000,100055,3,4,13,100065.408,2.129,3.145,3.195,100064.4,9,14,14,100062,4,7,12,100086
@RaulGracia
RaulGracia / kafka_pulsar_large_events_latency_vs_throughput.csv
Created September 11, 2020 09:15
We can make this file beautiful and searchable if this error is corrected: It looks like row 10 should actually have 17 columns, instead of 8. in line 9.
10KB events,"Kafka ""no flush"" (1 Segment)",,,,"Pulsar ""batch"" (1 Segment)",,,,"Kafka ""no flush"" (16 Segments)",,,,"Pulsar ""batch"" (16 Segments)",,,
Requested TP,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99
1000,1000.719,1.654,2.228,2.615,1000.7,3.1,4.1,4.2,1001.009,1.827,2.248,2.485,1000.6,2.8,4,4.1
3000,3005.268,1.661,2.982,3.265,3004.8,3.2,4.2,4.4,3004.548,1.746,2.406,2.623,3004.6,2.7,4,4.1
6000,6010.615,3.611,6.998,39.828,6010.5,3.6,4.6,4.6,6010.466,1.929,2.633,2.923,6009.8,2.8,4.1,4.3
9000,7925.023,419.805,428.361,434.649,9013.5,2.5,3.1,3.3,9010.368,2.34,4.233,6.434,9011.3,2.9,4.2,4.5
12000,6952.854,480.799,489.525,494.703,12019.3,2.7,3.3,3.4,12014.142,5.479,8.988,10.321,12016.8,3,4.4,6
15000,6641.302,500.855,510.113,513.617,15029.4,2,3,3.5,15025.525,5.951,9.212,11.119,15025,3.1,4.7,10.7
20000,7526.797,442.719,452.957,459.579,20014.5,2.5,2.9,6.6,20019.101,7.122,9.95,13.338,20013.7,3.3,5.1,18.4
25000,7784.583,427.959,436.305,443.111,25019.2,2.4,3.3
@RaulGracia
RaulGracia / kafka_pulsar_batching_latency_vs_throughput.csv
Created September 11, 2020 09:12
We can make this file beautiful and searchable if this error is corrected: It looks like row 7 should actually have 25 columns, instead of 18. in line 6.
100B events,"Pulsar ""batch"" (1 Segment)",,,,"Pulsar ""no batch"" (1 Segment)",,,,"Kafka ""large batch"" (1 Segment)",,,,"Pulsar ""batch"" (16 Segments)",,,,"Pulsar ""no batch"" (16 Segments)",,,,"Kafka ""large batch"" (16 Segments)",,,
Requested TP,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99
100,100.1,2.8,3.8,3.9,100.1,1.8,2,2.1,100.084,9.91,10.961,11.085,100.1,2.8,3.8,4,100.1,1.9,2.1,2.2,100.054,10.885,11.226,11.408
1000,1000.7,2.9,3.9,4,1000.6,1.8,2,2.1,1000.457,6.075,10.884,11.41,1000.6,2.7,3.8,4,1000.6,1.8,2,2.1,1000.623,7.416,11.694,11.908
5000,5003.5,2.8,3.9,3.9,5004.7,1.8,1.9,2,5003.036,6.29,11.054,11.615,5003.4,2.6,3.9,4,5004.1,1.8,1.9,2,5003.157,6.543,11.064,11.998
10000,10006.4,2.9,3.9,4,10007.5,1.9,2,2.1,10006.099,6.194,10.953,11.46,10006.7,2.6,3.9,4.1,10009.1,1.9,2,2.1,10009.711,6.614,11.083,12.043
50000,50032.5,2.9,3.9,4,42135.5,235.5,331.8,375,50039.013,6.947,11.743,12.163,50027.7,2.7,4,4.2,5004
@RaulGracia
RaulGracia / kafka_durability_latency_vs_throughput.csv
Created September 11, 2020 09:07
We can make this file beautiful and searchable if this error is corrected: It looks like row 9 should actually have 17 columns, instead of 9. in line 8.
100B events,"Kafka ""no flush"" (1 Segment)",,,,"Kafka ""flush"" (1 Segment)",,,,"Kafka ""no flush"" (16 Segments)",,,,"Kafka ""flush"" (16 Segments)",,,
Requested TP,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99,Measured TP,P50,P95,P99
100,100.05,1.958,2.121,2.38,100.051,2.307,2.605,2.776,100.057,2.109,2.382,2.469,100.07,2.45,2.778,2.956
1000,1000.648,1.579,1.928,2.01,1000.696,2.083,2.964,3.293,1000.731,1.765,2.166,2.396,1001.045,2.248,2.832,3.111
5000,5003.864,1.509,2.369,2.623,5003.655,2.028,2.985,3.67,5004.348,1.65,2.345,2.552,5003.219,6.22,9.853,11.773
10000,10006.805,1.59,2.478,2.645,10006.985,2.149,3.163,32.774,10007.813,1.943,3.873,5.669,10009.559,12.214,14.945,17.111
50000,50034.84,1.642,2.699,2.961,50032.592,3.016,67.607,143.491,50053.777,4.941,9.769,10.555,50050.535,16.191,18.976,24.22
100000,100065.408,1.693,2.773,2.905,100081.746,4.765,63.905,130.282,100086.764,3.423,6.913,7.884,100114.133,17.455,20.021,30.274
200000,200134.702,1.796,3.026,3.196,200200.645,8.745,69.96,17
@RaulGracia
RaulGracia / testgen-pravega-autoscale.py
Created August 19, 2020 13:04
#!/usr/bin/env python
from __future__ import print_function
import json
import sys
def add_test():
driver = {
'name': 'Pravega',
@RaulGracia
RaulGracia / testgen-pulsar-historical.py
Created August 14, 2020 10:54
#!/usr/bin/env python
from __future__ import print_function
import json
import sys
test_list = []
localWorker = False
tarball = 'package/target/openmessaging-benchmark-0.0.1-SNAPSHOT-bin.tar.gz'
@RaulGracia
RaulGracia / pravega_latency_vs_throughput.csv
Last active September 11, 2020 08:58
100B events Pravega (1 Segment) Pravega (16 Segments) 10KB events Pravega (1 Segment) Pravega (16 Segments)
Requested TP Measured TP P50 P95 P99 Measured TP P50 P95 P99 Requested TP Measured TP P50 P95 P99 Measured TP P50 P95 P99
100 100 1.9 2.1 2.1 100.1 2 2.2 2.5 1000 1000.5 2 3 4.5 1000.7 2 2.2 2.5
1000 1000.5 1.8 2.7 3 1000.6 1.9 2 2.2 3000 3004.7 2.5 3.6 6.8 3004.9 2.1 2.6 5
5000 5002.8 2 3.5 4.1 5005 1.9 2.8 3.7 6000 6009.7 3.5 4.9 6.2 6010.2 2.2 4.1 7.2
10000 10005.4 2 3.5 4 10008.9 2.1 4.4 7.4 9000 9007.8 3.9 5.7 6.9 9008.9 2.5 5.4 9.1
50000 50026.4 2.5 3.8 4.1 50033.9 3.2 6.3 9.1 12000 12013.5 5 7.1 8.3 12015.3 2.8 6.4 10.7
100000 100054.8 2.8 3.8 4.5 100062 3.3 6.4 9.4 15000 15025.5 4.3 6.3 9.2 15025.6 3.1 6.7 11
200000 200117.4 3.1 3.9 4.3 200142.9 3.6 7 10.2 20000 18434.3 6.4 1357.1 2484.2 20013.3 3.5 7.3 13.8
300000 300424.2 3.2 4.2 5.9 300423.3 3.7 7 10.2 25000 17111.8 233.6 2881.9 3312.7 25036.8 3.8 8.3 20.4