Skip to content

Instantly share code, notes, and snippets.

@davewat
davewat / gist:b3c5c4a383d1d9d26df33ecbb6ddd668
Last active March 30, 2022 19:03
Kafka commands and scripts
# Calculate the latest and earliest offsets, and provide the total # of messages in the topic (the difference):
#!/bin/bash
if [[ $# -eq 0 ]] ; then
echo 'provide the name of a topic to get totals against'
exit 0
fi
latest=`bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic $1 --time -1 --offsets 1 | awk -F ":" '{sum += $3} END {print sum}'`
earliest=`bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic $1 --time -2 --offsets 1 | awk -F ":" '{sum2 += $3} END {print sum2}'`
total=`expr $latest - $earliest`
@davewat
davewat / latency.markdown
Created November 17, 2021 06:14 — forked from hellerbarde/latency.markdown
Latency numbers every programmer should know

Latency numbers every programmer should know

L1 cache reference ......................... 0.5 ns
Branch mispredict ............................ 5 ns
L2 cache reference ........................... 7 ns
Mutex lock/unlock ........................... 25 ns
Main memory reference ...................... 100 ns             
Compress 1K bytes with Zippy ............. 3,000 ns  =   3 µs
Send 2K bytes over 1 Gbps network ....... 20,000 ns  =  20 µs
SSD random read ........................ 150,000 ns  = 150 µs

Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs