Skip to content

Instantly share code, notes, and snippets.

View kellabyte's full-sized avatar

Kelly Sommers kellabyte

791 followers · 1 following
View GitHub Profile
@kellabyte
kellabyte / recoil.md
Last active January 31, 2023 14:52

History

For a long time I've been really impacted by the ease of use Cassandra and CockroachDB bring to operating a data store at scale. While these systems have very different tradeoffs what they have in common is how easy it is to deploy and operate a cluster. I have experience with them with cluster sizes in the dozens, hundreds, or even thousands of nodes and in comparison to some other clustered technologies they get you far pretty fast. They have sane defaults that provide scale and high availability to people that wouldn't always understand how to achieve it with more complex systems. People can get pretty far before they have to become experts. When you start needing more extreme usage you will need to become an expert of the system just like any other piece of infrastructure. But what I really love about these systems is it makes geo-aware data placement, GDPR concerns potentially simplified and data replication and movement a breeze most of the time.

Several years ago the great [Andy Gross](ht

@kellabyte
kellabyte / hash_benchmarks_test.txt
Last active March 29, 2018 07:02
Go hashing algorithm benchmarks
BenchmarkCRC32/1-8 100000000 15.80 ns/op 63.46 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/2-8 100000000 15.80 ns/op 126.58 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/4-8 100000000 15.90 ns/op 250.87 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/8-8 100000000 16.00 ns/op 498.77 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/32-8 100000000 18.40 ns/op 1738.69 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/64-8 100000000 21.00 ns/op 3053.22 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/128-8 50000000 26.50 ns/op 4823.16 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/256-8 50000000 38.80 ns/op 6596.60 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/512-8 30000000 51.00 ns/op 10037.68 MB/s 0 B/op 0 allocs/op
BenchmarkCRC32/1024-8 20000000 84.90 ns/op 12055.07 MB/s
@kellabyte
kellabyte / http_server.c
Created November 25, 2014 04:47
#pragma comment (lib, "libuv.lib")
#pragma comment (lib, "ws2_32.lib")
#pragma comment(lib, "psapi.lib")
#pragma comment(lib, "Iphlpapi.lib")
#ifdef PLATFORM_POSIX
#include <signal.h>
#endif // PLATFORM_POSIX
#include <stdio.h>
@kellabyte
kellabyte / twitter_datamining.cs
Created January 7, 2012 20:29
Data mining from the Twitter public stream and storing the JSON into Cassandra
#
# Commands executed in Cassandra
#
/*
connect 127.0.0.1/9160;
create keyspace twitter;
use twitter;
# Create a column family to contain the rowcount of how many tweets I have.
CREATE COLUMN FAMILY rowcounts
@kellabyte
kellabyte / CQS_and_CQRS.cs
Created March 3, 2012 03:19
CQS_and_CQRS
// CQS (Command-Query Separation)
// Bertrand Meyer devised the CQS principle
// It states that every method should either be a command that performs an action, or a
// query that returns data to the caller, but not both. In other words, asking a question
// should not change the answer. More formally, methods should return a value only if they
// are referentially transparent and hence possess no side effects.
//
// Example:
public class CustomerService
@kellabyte
kellabyte / haywire_bottleneck.md
Last active November 25, 2016 04:49

If I run 20 haywire processes using tcmalloc haywire reaches 6.3 million requests/second.

killall hello_world; for i in `seq 20`; do LD_PRELOAD="./lib/gperftools/.libs/libtcmalloc.so" ./build/hello_world --balancer reuseport & done

perf top
   7.42%  hello_world              [.] http_request_buffer_pin
   6.94%  hello_world              [.] http_request_buffer_reassign_pin
   6.63%  hello_world              [.] http_parser_execute
   6.23%  libtcmalloc.so.4.3.0     [.] tc_deletearray_nothrow
   4.94%  hello_world              [.] http_request_buffer_locate
@kellabyte
kellabyte / SO_REUSEPORT.md
Last active November 24, 2016 20:30

3 million requests/second

./build/hello_world --threads 20 --balancer reuseport
perf top

   9.76%  hello_world              [.] http_parser_execute
   7.85%  libc-2.21.so             [.] malloc
   4.50%  libc-2.21.so             [.] free
   3.43%  libc-2.21.so             [.] __libc_calloc
@kellabyte
kellabyte / go_vs_haywire_baremetal.md
Last active May 29, 2016 22:06

Plaintext 8 threads

Connections: 8		Haywire		Requests/sec:  57334.75
Connections: 8		Go		    Requests/sec:  42901.57
-------------------------------------------------------
Connections: 16		Haywire		Requests/sec:  71460.01
Connections: 16		Go		    Requests/sec:  71891.07
-------------------------------------------------------
Connections: 32		Haywire		Requests/sec:  78421.15
Connections: 32		Go		    Requests/sec:  89202.96
@kellabyte
kellabyte / fsqual.md
Last active April 11, 2016 19:27

FSQual run on Linux subsystem for Windows

fsqual - file system qualification tool for asynchonus I/O
https://github.com/avikivity/fsqual

./fsqual
context switch per appending io (iodepth 1): 0 (GOOD)
context switch per appending io (iodepth 3): 0 (GOOD)
context switch per appending io (iodepth 3): 0 (GOOD)
@kellabyte
kellabyte / hellcat_cmake.txt
Last active January 4, 2016 15:09
cmake_minimum_required(VERSION 2.6)
# create_source_group(relativeSourcePath sourceGroupName files)
#
# Creates a source group with the specified name relative to the relative path
# specified.
#
# Parameters:
# - sourceGroupName: Name of the source group to create.
# - relativeSourcePath: Relative path to the files.