Skip to content

Instantly share code, notes, and snippets.

@manuzhang

manuzhang/kafka_source_benchmark_commands

Created February 10, 2015 04:46
Show Gist options
  • Save manuzhang/a60df427af85cc70bb3a to your computer and use it in GitHub Desktop.
Save manuzhang/a60df427af85cc70bb3a to your computer and use it in GitHub Desktop.
Download ZIP
kafka_source_benchmark_commands
Raw
kafka_source_benchmark_commands
Setup
bin/kafka-topics.sh --create --topic consumer --zookeeper 192.168.1.73:2181/kafka --replication-factor 3 --partitions 6
Produce data
bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance consumer 500000000 100 -1 acks=1 bootstrap.servers=192.168.1.73:9092 buffer.memory=67108864 batch.size=8196
Single KafkaSource
bin/gear app -jar examples/gearpump-examples-assembly-0.2.4-SNAPSHOT.jar org.apache.gearpump.streaming.examples.kafka.consumer.KafkaConsumerPerf -master 192.168.1.71:3000 -kafka_stream_producer 1 -runseconds 360
KafkaSource + StreamProcessor
bin/gear app -jar examples/gearpump-examples-assembly-0.2.4-SNAPSHOT.jar org.apache.gearpump.streaming.examples.kafka.consumer.KafkaConsumerPerfSOL -master 192.168.1.71:3000 -kafka_stream_producer 1 -bolt 1 -runseconds 360
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment