Skip to content

Instantly share code, notes, and snippets.

View kellabyte's full-sized avatar

Kelly Sommers kellabyte

792 followers · 1 following
View GitHub Profile
@kellabyte
kellabyte / segmented_indexes.md
Last active August 29, 2015 13:55

Indexes are 8 (to make it small enough to read) sorted 32bit integer segments. Each integer represents the record_id that matches the term. Each segment is stored in a Key/Value store.

Index A represents the rows that have the term "Canada" in a Country column. Index B represents the rows that have the term "Ontario" in a Province column.

Segments from both indexes will be read off disk using a Key/Value store and intersected to evaluate a conjunction query.

Index A | Index B
-----------------
     Segment 1
@kellabyte
kellabyte / dstat.csv
Created February 12, 2014 02:56
Dstat 0.7.2 CSV output
Author: Dag Wieers <dag@wieers.com> URL: http://dag.wieers.com/home-made/dstat/
Host: bm1 User: azureuser
Cmdline: dstat -tcmrd --disk-util -ny --output stats.out Date: 12 Aug 2013 02:49:03 UTC
system total cpu usage memory usage io/total dsk/total sda sdb sr0 net/total system
time usr sys idl wai hiq siq used buff cach free read writ read writ util util util recv send int csw
12/8/2014 2:49 2.296 0.901 94.702 1.99 0 0.112 157990912 44568576 550817792 6556995584 2.465 20.54 54106.362 1821659.921 7.757 0.756 0.001 0 0 1155.05 333.637
12/8/2014 2:49 0 0.249 99.751 0 0 0 158093312 44568576 550805504 6556905472 0 0 0 0 0 0 0 252 1444 1040 144
12/8/2014 2:49 0 0 100 0 0 0 158220288 44568576 550805504 6556778496 0 0 0 0 0 0 0 66 530 1029 96
@kellabyte
kellabyte / CMakeLists.txt
Created April 15, 2014 22:08
cmake_minimum_required(VERSION 2.8)
# create_source_group(relativeSourcePath sourceGroupName files)
#
# Creates a source group with the specified name relative to the relative path
# specified.
#
# Parameters:
# - sourceGroupName: Name of the source group to create.
# - relativeSourcePath: Relative path to the files.
@kellabyte
kellabyte / prelimary_haywire_native_vs_managed.md
Last active August 29, 2015 14:01

#Setup

  • Benchmark client: Linux, gigabit LAN, running Wrk benchmark tool.
  • Haywire server: Windows 7, Intel Core i7 930. Haywire running in Release mode.

Haywire is running in single threaded mode (using a IOCP pool for IO) with Haywire event-loop fan-out disabled (broken right now). So this means Haywire won't use all the available cores yet.

#Native results

  • CPU: Peak 5%
  • RAM: 2.5MB
@kellabyte
kellabyte / benchmark.sh
Created June 18, 2014 14:15
#!/bin/bash
function run {
NAME=$1
BENCHMARK=$2
RECORDS=$3
VALUE_SIZE=$4
CACHE_SIZE=$5
REUSE_DB=$6
BENCHMARK_NAME=${NAME}_${BENCHMARK}
@kellabyte
kellabyte / get_sequential.lua
Last active August 29, 2015 14:05
counter = 0
arguments = {}
init = function(args)
wrk.init(args)
arguments = args
end
request = function()
pipeline_length = tonumber(arguments[1]) or 1
@kellabyte
kellabyte / hellcat_linker_errors.txt
Created August 19, 2014 13:16
/share/git/hellcat$ ./compile_make.sh
mkdir: cannot create directory âbuildâ: File exists
-- Configuring done
-- Generating done
-- Build files have been written to: /share/git/hellcat/build
/usr/bin/cmake -H/share/git/hellcat -B/share/git/hellcat/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start /share/git/hellcat/build/CMakeFiles /share/git/hellcat/build/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory `/share/git/hellcat/build'
make -f CMakeFiles/hellcat.dir/build.make CMakeFiles/hellcat.dir/depend
@kellabyte
kellabyte / h2o_vs_haywire.md
Last active August 29, 2015 14:08

Hardware setup

Client and servers are run on independent machines. Each one has the following.

OS: Ubuntu
Kernel: 3.14.5-031405
RAM: 24GB
CPU: 2 sockets, each socket has a quad core 2.5ghz processor.

Processor topology: https://gist.github.com/kellabyte/8546939

@kellabyte
kellabyte / wrk2_pipeline.txt
Created November 12, 2014 15:19
./wrk --script ./get.lua --latency -d 30s -t 8 -c 32 -R 30000 http://server:8000 -- 64
Running 30s test @ http://server:8000
8 threads and 32 connections
Thread calibration: mean lat.: 776.155ms, rate sampling interval: 2844ms
Thread calibration: mean lat.: 2560.296ms, rate sampling interval: 9338ms
Thread calibration: mean lat.: 1820.471ms, rate sampling interval: 6823ms
Thread calibration: mean lat.: 467.598ms, rate sampling interval: 1831ms
Thread calibration: mean lat.: 2186.486ms, rate sampling interval: 8626ms
Thread calibration: mean lat.: 2698.650ms, rate sampling interval: 9633ms
Thread calibration: mean lat.: 707.204ms, rate sampling interval: 2703ms
@kellabyte
kellabyte / haywire_1024connection_crash.txt
Created November 13, 2014 04:07
haywire_hello_world: lib/libuv/src/unix/stream.c:1459: uv_read_start: Assertion `((stream)->io_watcher.fd) >= 0' failed.
Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff65f2700 (LWP 6445)]
0x00007ffff762b425 in __GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff762b425 in __GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1 0x00007ffff762eb8b in __GI_abort () at abort.c:91
#2 0x00007ffff76240ee in __assert_fail_base (fmt=<optimized out>, assertion=0x428368 "((stream)->io_watcher.fd) >= 0",