Skip to content

Instantly share code, notes, and snippets.

Kelly Sommers kellabyte

Block or report user

Report or block kellabyte

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@kellabyte
kellabyte / hash_benchmarks_test.txt
Last active Mar 29, 2018
Go hashing algorithm benchmarks
View hash_benchmarks_test.txt
BenchmarkCRC32/1-8 100000000 15.80 ns/op 63.46 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/2-8 100000000 15.80 ns/op 126.58 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/4-8 100000000 15.90 ns/op 250.87 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/8-8 100000000 16.00 ns/op 498.77 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/32-8 100000000 18.40 ns/op 1738.69 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/64-8 100000000 21.00 ns/op 3053.22 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/128-8 50000000 26.50 ns/op 4823.16 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/256-8 50000000 38.80 ns/op 6596.60 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/512-8 30000000 51.00 ns/op 10037.68 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/1024-8 20000000 84.90 ns/op 12055.07 MB/s
View haywire_bottleneck.md

If I run 20 haywire processes using tcmalloc haywire reaches 6.3 million requests/second.

killall hello_world; for i in `seq 20`; do LD_PRELOAD="./lib/gperftools/.libs/libtcmalloc.so" ./build/hello_world --balancer reuseport & done

perf top
   7.42%  hello_world              [.] http_request_buffer_pin
   6.94%  hello_world              [.] http_request_buffer_reassign_pin
   6.63%  hello_world              [.] http_parser_execute
   6.23%  libtcmalloc.so.4.3.0     [.] tc_deletearray_nothrow
   4.94%  hello_world              [.] http_request_buffer_locate
View SO_REUSEPORT.md

3 million requests/second

./build/hello_world --threads 20 --balancer reuseport
perf top

   9.76%  hello_world              [.] http_parser_execute
   7.85%  libc-2.21.so             [.] malloc
   4.50%  libc-2.21.so             [.] free
   3.43%  libc-2.21.so             [.] __libc_calloc
View fsqual.md

FSQual run on Linux subsystem for Windows

fsqual - file system qualification tool for asynchonus I/O
https://github.com/avikivity/fsqual

./fsqual
context switch per appending io (iodepth 1): 0 (GOOD)
context switch per appending io (iodepth 3): 0 (GOOD)
context switch per appending io (iodepth 3): 0 (GOOD)
View haywire_run.md

Max throughput benchmark

Max throughput in master

./bin/wrk/wrk --script ./pipelined_get.lua --latency -d 5m -t 40 -c 760 http://server:8000 -- 32
Running 5m test @ http://server:8000
  40 threads and 760 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.16ms    6.70ms 223.68ms   93.20%
    Req/Sec   114.20k    28.04k  410.56k    72.96%
View ruby_regex.md

Why does this work on Rubular.com but not in Ruby code? What am I doing wrong?

Source string

Running 1s test @ http://192.168.0.2:8000\n  8 threads and 256 connections\n  Thread Stats   Avg      Stdev     Max   +/- Stdev\n    Latency     2.08ms    3.91ms  61.86ms   92.89%\n    Req/Sec    23.80k    10.45k   60.63k    74.70%\n  Latency Distribution\n     50%    1.10ms\n     75%    1.59ms\n     90%    4.03ms\n     99%   22.04ms\n  197510 requests in 1.10s, 29.76MB read\nRequests/sec: 179561.40\nTransfer/sec:     27.06MB

Ruby regex

output.match("Requests\/sec: (.*)\\n")
View haywire_vs_kestrel.md

Setup

Kestrel only supports 1 thread currently. I've included single threaded mode Haywire benchmarks and multi-threaded mode benchmarks for comparisons.

Kestrel 1 thread HTTP pipelining enabled

Running 10s test @ http://192.168.0.101:5000
  8 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    71.50ms   41.78ms 212.21ms   39.50%
    Req/Sec     1.70k   534.07     3.80k    77.39%
View hellcat.cpp
#include <iostream>
#include <sstream>
#include <fstream>
#include <haywire.h>
#include "lmdb.h"
#include "common.h"
#include "storage/store.h"
#include "storage/lmdb_store.h"
#include "haywire.h"
#include "hellcat.h"
View hellcat_perf_callgraph.txt
- 10.26% hellcat [kernel.kallsyms] [k] _raw_spin_lock ◆
+ start_thread ▒
+ uv__thread_start ▒
- connection_consumer_start ▒
- 100.00% uv_run ▒
- 100.00% uv__io_poll ▒
- 99.97% uv__stream_io ▒
- 99.79% uv__read ▒
- 99.98% http_stream_on_read ▒
- 99.98% http_parser_execute ▒
View client.txt
git clone https://github.com/kellabyte/Haywire.git
cd Haywire
./build.sh -c release
cd benchmark
../bin/wrk/wrk --script ./pipelined_get.lua --latency -d 10s -t 16 -c 32 http://127.0.0.1:8000 -- 64
../bin/wrk/wrk --script ./pipelined_get.lua --latency -d 10s -t 16 -c 32 http://127.0.0.1:8001 -- 64
You can’t perform that action at this time.