Skip to content

Instantly share code, notes, and snippets.

View Dieterbe's full-sized avatar

Dieter Plaetinck Dieterbe

View GitHub Profile
@Dieterbe
Dieterbe / font-test.sh
Created December 19, 2021 19:00
font-test.sh
#!/bin/bash
read -r -d '' text <<'EOF'
'" <> ?+| /=\ -_ ,.:;
ABCDEFGHIJKLMNOPQRSTUVWXYZ
abcdefghijklmnopqrstuvwxyz
! @ # $ % ^ & * ( ) { }
1 2 3 4 5 6 7 8 9 0 [ ]
EOF
echo -e "Regular:\n$text"
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"limit": 100,
#!/bin/bash
# useful articles:
# https://vstinner.github.io/journey-to-stable-benchmark-system.html
# http://perf.readthedocs.io/en/latest/system.html
# https://baiweiblog.wordpress.com/2017/11/02/how-to-set-processor-affinity-in-linux-using-taskset/
# https://wiki.archlinux.org/index.php/CPU_frequency_scaling#CPU_frequency_driver
if [[ $EUID > 0 ]]; then
echo "Please run as root"
exit 1
$ go test -count=10 -run='^$' -bench=. -benchtime=100ms ./...
goos: linux
goarch: amd64
pkg: github.com/grafana/metrictank/api
BenchmarkFix1M-8 16 9189930 ns/op 1305.78 MB/s
BenchmarkFix1M-8 14 8155018 ns/op 1471.49 MB/s
BenchmarkFix1M-8 13 12086950 ns/op 992.81 MB/s
BenchmarkFix1M-8 13 9010493 ns/op 1331.78 MB/s
BenchmarkFix1M-8 16 10516333 ns/op 1141.08 MB/s
BenchmarkFix1M-8 16 9929228 ns/op 1208.55 MB/s

as described in devdocs/expr.md, metrictank resets the consolidation function when it passes a "special" function. a special function is one that changes the nature of the data on a fundamental level. Such as summarize, perSecond, derivative, integral We explain the reasoning for that behavior with a few examples.

example 1

summarize(consolidateBy(statsd.count,"sum"),"24h","sum") 
  • the inputs to summarize, if we need to consolidate, can be summed, because we'll sum the data anyway
  • but if we want to see daily sums, and maxdatapoints dictates that we must consolidate at runtime we don't want the points summed together into buckets of arbitrary N-day-size buckets.
sarama/8 rtm 3424 -> 0.030103
sarama/8 id 687540 -> 0.002537
sarama/8 ops 1009213 -> 0.004118
sarama/8 fng 3024891 -> 0.001198
sarama/8 fake-false 1000 -> 0.032249
sarama/8 fake-false 10000 -> 0.006223
sarama/8 fake-false 100000 -> 0.002262
sarama/8 fake-false 1000000 -> 0.000801
sarama/8 fake-false 10000000 -> 0.000325
sarama/8 fake-true 1000 -> 1.088978
taskset --cpu-list 1,2 go test . -run='^$' -test.benchmem -bench .
goos: linux
goarch: amd64
pkg: github.com/grafana/metrictank/cluster/partitioner
BenchmarkPartitionerSarama-2 10000000 205 ns/op 0 B/op 0 allocs/op
BenchmarkJumpPartitionerMauro-2 5000000 271 ns/op 210 B/op 1 allocs/op
BenchmarkJumpPartitionerFnv-2 5000000 323 ns/op 8 B/op 1 allocs/op
BenchmarkJumpPartitionerMetro-2 20000000 101 ns/op 0 B/op 0 allocs/op
BenchmarkJumpPartitionerSip-2 10000000 145 ns/op 0 B/op 0 allocs/op
BenchmarkJumpPartitionerXxhash-2 20000000 99.4 ns/op 0 B/op 0 allocs/op
unused ./...
Unused has been deprecated. Please use staticcheck instead.
api/config.go:20:2: var logMinDur is unused (U1000)
cmd/mt-store-cat/match_test.go:5:6: type matchCase is unused (U1000)
cmd/mt-store-cat/match_test.go:40:6: func testmatch is unused (U1000)
cmd/mt-update-ttl/main_test.go:5:6: type testCase is unused (U1000)
cmd/mt-update-ttl/main_test.go:28:6: type completenessCase is unused (U1000)
cmd/mt-whisper-importer-reader/conversion_test.go:10:6: func testIncResolution is unused (U1000)
cmd/mt-whisper-importer-reader/conversion_test.go:397:6: func testDecResolution is unused (U1000)
cmd/mt-whisper-importer-reader/conversion_test.go:422:6: func getSimpleInData is unused (U1000)
#!/bin/bash
style=$1
if [ "$style" != "LIGHT" -a "$style" != light -a "$style" != "dark" ]; then
echo "arg1 must be LIGHT, light or dark" >&2
exit 2
fi
# shell
# if we source this in terminals we can't just reload termite with proper colors,
* etsy/statsd is the official statsd implementation, yet hasn't been maintained for more than 2 years.
(see also https://github.com/etsy/statsd/issues/649)
* dozens of forks and re-implementations exist in the community.
usually they bring one or more of extra features (e.g. graphite tags, security, ...),
easier deployment and better performance,
with various degrees of backwards compatibility
Which implementation is "best" depends on many factors, but often enough, it's not the official reference implementation.
IMHO, most of a statsd server's responsibility is catering to an organisations need of how they organize/name/.. their metrics.
So any implementation is very opinionated, and for etsy/statsd trying to cater to many different styles leads to a ballooning complexity