dongjinleekr

#!/usr/bin/env bash

JSHELL=$(which jshell)

if [[ -z ${JSHELL} ]]; then
  echo "jshell not installed; exit."
  exit 1
fi

GRADLE=$(which gradle)

dongjinleekr

This gist describes how to create a Kafka cluster on Kubernetes with minimal effort.

Dislike to incubator/kafka helm chart, this approach uses wurstmeister/kafka Docker image or its GraalVM equivalent, dongjinleekr/kafka.

Note: This configuration is intended for dev or testing purpose; it may be used in production environment, but I can't give any guarantees in that respect.

Prerequisites

Install Zookeeper with the following helm command:

dongjinleekr

Tested on Hadoop 2.7.7 + HBase 2.0.5.

1. Add following dependencies:

• org.apache.hbase:hbase-client:${hbase.version}
• org.apache.hbase:hbase-common:${hbase.version}
• org.apache.hbase:hbase-testing-util:test:${hbase.version}

2. Use HbaseTestUtil to startup a mini cluster in setup/teardown of the suite.

dongjinleekr

Keybase proof

I hereby claim:

• I am dongjinleekr on github.
• I am dongjinleekr (https://keybase.io/dongjinleekr) on keybase.
• I have a public key ASBY5pir8DtYwtcEP9Ybe3NLsqGgdKjBk_zYOmRoJMdJhAo

To claim this, I am signing this object:

dongjinleekr

public class Numbers {
  private Numbers() {
  }
 
  public static boolean valueOf(Boolean b) {
    return valueOf(b, false);
  }
 
  public static boolean valueOf(Boolean b1, boolean b2) {
    if (null == b1) {

dongjinleekr

// Uses a function instance with internal state, along with mapping the map values only.
// see: http://stackoverflow.com/questions/23213891/how-to-map-values-in-a-map-in-java-8

import com.google.common.collect.ImmutableMap;

import java.util.Map;
import java.util.function.Function;

import static java.util.stream.Collectors.toMap;

dongjinleekr

## Consumer Throughput: Single consumer thread, no compression
## Consumer Throughput: 3 consumer thread, no compression

bin/kafka-consumer-perf-test.sh --topic benchmark-3-3-none \
--zookeeper kafka-zk-1:2181,kafka-zk-2:2181,kafka-zk-3:2181 \
--messages 15000000 \
--threads 1