ernestas-poskus

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

1. List the topics to which the group is subscribed

kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

2. Reset the consumer offset for a topic (preview)

ernestas-poskus

# knife cheat

## Search Examples

	knife search "name:ip*" 
	knife search "platform:ubuntu*"
	knife search "platform:*" -a macaddress
	knife search "platform:ubuntu*" -a uptime
	knife search "platform:ubuntu*" -a virtualization.system
	knife search "platform:ubuntu*" -a network.default_gateway

ernestas-poskus

#[derive(Clone)]
pub struct KillSwitch {
    kill: Arc<AtomicBool>,
}
impl KillSwitch {
    pub fn new() -> Self {
        KillSwitch {
            kill: Arc::new(AtomicBool::new(false)),
        }
    }

ernestas-poskus

Latency Comparison Numbers (~2012)
----------------------------------
L1 cache reference                           0.5 ns
Branch mispredict                            5   ns
L2 cache reference                           7   ns                      14x L1 cache
Mutex lock/unlock                           25   ns
Main memory reference                      100   ns                      20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy             3,000   ns        3 us
Send 1K bytes over 1 Gbps network       10,000   ns       10 us
Read 4K randomly from SSD*             150,000   ns      150 us          ~1GB/sec SSD

ernestas-poskus

obj-m += rootkit.o

all:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

ernestas-poskus

Moved to git repository: https://github.com/denji/nginx-tuning

NGINX Tuning For Best Performance

For this configuration you can use web server you like, i decided, because i work mostly with it to use nginx.

Generally, properly configured nginx can handle up to 400K to 500K requests per second (clustered), most what i saw is 50K to 80K (non-clustered) requests per second and 30% CPU load, course, this was 2 x Intel Xeon with HyperThreading enabled, but it can work without problem on slower machines.

You must understand that this config is used in testing environment and not in production so you will need to find a way to implement most of those features best possible for your servers.

ernestas-poskus

DATA_NODES_COUNT = 96
DATA_NODE_PER_SERVER = 4
SHARDS_PER_DATA_NODE = 2 * 1000 * 1000
REPLICA_COUNT = 2

def servers(s, r = REPLICA_COUNT)
  s * r + s
end

s = 4

ernestas-poskus

m = []

size = 15

def shards(s, r)
  s * r + s
end

size.times { |y| m << Array(0..size).map { |x| shards(y, x) }.join(' | ') }

ernestas-poskus

# to generate your dhparam.pem file, run in the terminal
openssl dhparam -out /etc/nginx/ssl/dhparam.pem 2048

ernestas-poskus

###
# Based on:
# jQuery JavaScript Library v1.4.2
# http://jquery.com/
#
# Copyright 2010, John Resig
# Dual licensed under the MIT or GPL Version 2 licenses.
# http://jquery.org/license
###