Skip to content

Instantly share code, notes, and snippets.

View cdbartholomew's full-sized avatar

Chris Bartholomew cdbartholomew

13 followers · 3 following
View GitHub Profile
@cdbartholomew
cdbartholomew / gist:88a419a222b631fc2cd0a3759fb5b279
Created November 9, 2019 19:46
Shared consumer 2
2019-11-09 13:48:55.257 INFO ConsumerImpl:170 | [persistent://azure-kafkaesque-io/local-uscentral1-azure/subscription-demo, shared, 0] Created consumer on broker [172.16.0.117:47342 -> 52.242.224.253:6651]
2019-11-09 13:58:54.959 INFO ConsumerStatsImpl:64 | Consumer [persistent://azure-kafkaesque-io/local-uscentral1-azure/subscription-demo, shared, 0] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 0, receivedMsgMap_ = {[Key: TimeOut, Value: 200], }, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {[Key: TimeOut, Value: 200], }, totalAckedMsgMap_ = {})
Received message 'b'Hello World! 1'' id='(71,1,-1,-1)'
Received message 'b'Hello World! 3'' id='(71,3,-1,-1)'
Received message 'b'Hello World! 5'' id='(71,5,-1,-1)'
@cdbartholomew
cdbartholomew / gist:27cf67f28efbdaac89d34f425a162e70
Created November 9, 2019 19:46
Shared consumer 1
2019-11-09 13:48:37.658 INFO ConsumerImpl:170 | [persistent://azure-kafkaesque-io/local-uscentral1-azure/subscription-demo, shared, 0] Created consumer on broker [172.16.0.117:46688 -> 52.242.224.253:6651]
2019-11-09 13:58:37.361 INFO ConsumerStatsImpl:64 | Consumer [persistent://azure-kafkaesque-io/local-uscentral1-azure/subscription-demo, shared, 0] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 0, receivedMsgMap_ = {[Key: TimeOut, Value: 200], }, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {[Key: TimeOut, Value: 200], }, totalAckedMsgMap_ = {})
Received message 'b'Hello World! 0'' id='(71,0,-1,-1)'
Received message 'b'Hello World! 2'' id='(71,2,-1,-1)'
Received message 'b'Hello World! 4'' id='(71,4,-1,-1)'
@cdbartholomew
cdbartholomew / gist:adf0b23231735260bb44eab042a7917a
Created November 9, 2019 19:44
Exclusive consumer
2019-11-09 13:58:14.122 INFO ConsumerStatsImpl:64 | Consumer [persistent://azure-kafkaesque-io/local-uscentral1-azure/subscription-demo, exclusive, 0] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 0, receivedMsgMap_ = {[Key: TimeOut, Value: 200], }, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {[Key: TimeOut, Value: 200], }, totalAckedMsgMap_ = {})
Received message 'b'Hello World! 0'' id='(71,0,-1,-1)'
Received message 'b'Hello World! 1'' id='(71,1,-1,-1)'
Received message 'b'Hello World! 2'' id='(71,2,-1,-1)'
Received message 'b'Hello World! 3'' id='(71,3,-1,-1)'
Received message 'b'Hello World! 4'' id='(71,4,-1,-1)'
Received message 'b'Hello World! 5'' id='(71,5,-1,-1)'
@cdbartholomew
cdbartholomew / sub-demo-prod.py
Created November 9, 2019 19:42
import pulsar
service_url = 'pulsar+ssl://uscentral1.azure.kafkaesque.io:6651'
trust_certs='/etc/ssl/certs/ca-bundle.crt'
token='***************************************************************************'
client = pulsar.Client(service_url,
authentication=pulsar.AuthenticationToken(token),
tls_trust_certs_file_path=trust_certs)
producer = client.create_producer('persistent://azure-kafkaesque-io/local-uscentral1-azure/subscription-demo')
@cdbartholomew
cdbartholomew / sub_demo_shared.py
Last active November 9, 2019 20:27
import pulsar,time,_pulsar
service_url = 'pulsar+ssl://uscentral1.azure.kafkaesque.io:6651'
trust_certs='/etc/ssl/certs/ca-bundle.crt'
token='***********************************************************************************************'
client = pulsar.Client(service_url,
authentication=pulsar.AuthenticationToken(token),
tls_trust_certs_file_path=trust_certs)
consumer = client.subscribe('persistent://azure-kafkaesque-io/local-uscentral1-azure/subscription-demo',
@cdbartholomew
cdbartholomew / sub_demo.py
Last active November 9, 2019 19:44
import pulsar,time,_pulsar
service_url = 'pulsar+ssl://uscentral1.azure.kafkaesque.io:6651'
trust_certs='/etc/ssl/certs/ca-bundle.crt'
token='***************************************************************************************************'
client = pulsar.Client(service_url,
authentication=pulsar.AuthenticationToken(token),
tls_trust_certs_file_path=trust_certs)
consumer = client.subscribe('persistent://azure-kafkaesque-io/local-uscentral1-azure/subscription-demo',
@cdbartholomew
cdbartholomew / 1topic-6partition-100b-flush-End-to-endLatencyAvg.png
Last active December 11, 2019 19:40
Kafka/Pulsar Latency combined charts
1topic-6partition-100b-flush-End-to-endLatencyAvg.png
@cdbartholomew
cdbartholomew / 1topic-16partition-100b-ConsumeRate.svg
Last active December 11, 2019 19:44
Test
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@cdbartholomew
cdbartholomew / table.csv
Created August 30, 2019 19:16
Kafka with fsync, 1 topic, 16 partition, 100 byte
Latency Type Average 50th 75th 95th 99th 99.9th 99.99th Maximum
Publishing (milliseconds) 18.454 17.935 19.815 22.075 23.404 123.801 222.137 290.615
End-to-end (milliseconds) 21.119 20.0 22.0 25.0 33.0 199.0 259.0 334.0
@cdbartholomew
cdbartholomew / table.csv
Created August 30, 2019 19:15
Kafka with fsync, 1 topic, 6 partition, 100 byte
Latency Type Average 50th 75th 95th 99th 99.9th 99.99th Maximum
Publishing (milliseconds) 11.125 10.823 11.33 12.081 13.517 132.025 212.062 225.853
End-to-end (milliseconds) 13.857 13.0 13.0 15.0 32.0 208.0 239.0 287.0