hailelagi

Latency Comparison Numbers (~2012)
----------------------------------
L1 cache reference                           0.5 ns
Branch mispredict                            5   ns
L2 cache reference                           7   ns                      14x L1 cache
Mutex lock/unlock                           25   ns
Main memory reference                      100   ns                      20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy             3,000   ns        3 us
Send 1K bytes over 1 Gbps network       10,000   ns       10 us
Read 4K randomly from SSD*             150,000   ns      150 us          ~1GB/sec SSD

hailelagi

Big-O Time Complexities for Elixir data structures

Map [1]

Operation	Time Complexity
Access	O(log n)
Search	O(log n)
Insertion	O(n) for <= 32 elements, O(log n) for > 32 elements [2]
Deletion	O(n) for <= 32 elements, O(log n) for > 32 elements

hailelagi

https://web.archive.org/web/20110219163448/http://howtohft.wordpress.com/2011/02/15/how-to-build-a-fast-limit-order-book/

The response to my first few posts has been much larger than I’d imagined and I’d like to thank everyone for the encouragement.

If you’re interested in building a trading system I recommend first reading my previous post on general ideas to keep in mind.

My first really technical post will be on how to build a limit order book, probably the single most important component of a trading system. Because the data structure chosen to represent the limit order book will be the primary source of market information for trading models, it is important to make it both absolutely correct and extremely fast.

To give some idea of the data volumes, the Nasdaq TotalView ITCH feed, which is every event in every instrument traded on the Nasdaq, can have data rates of 20+ gigabytes/day with spikes of 3 megabytes/second or more. The individual messages average about 20 bytes each so this means handling

hailelagi

Introduction

The computer driven markets for instruments like stocks and exchange traded stock options, have transformed finance and the flow of capital. These markets are enabled by order matching engines (and the infrastructure that supports this software). Before computer trading networks and matching engines, stocks where traded on cavernous exchange floors and transaction costs where high. When electronic trading fully matured, floor traders were a fading anachronism and transaction costs had been reduced to pennies a share in many cases. Electronic trading could not exist without advanced network infrastructure, but without the software matching engines no shares would change hands. The computer trading networks, the matching engine software has also created a concentrated nexus of potential failure. Failures in these systems have increased as the frequency and volume on the electronic networks has increased. The position of order matching engines in the trading infrastructure makes these systems o

hailelagi

sysctl -w fs.file-max=12000500
sysctl -w fs.nr_open=20000500
ulimit -n 4000000
sysctl -w net.ipv4.tcp_mem='10000000 10000000 10000000'
sysctl -w net.ipv4.tcp_rmem='1024 4096 16384'
sysctl -w net.ipv4.tcp_wmem='1024 4096 16384'
sysctl -w net.core.rmem_max=16384
sysctl -w net.core.wmem_max=16384
wget http://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb
sudo dpkg -i erlang-solutions_1.0_all.deb

hailelagi

// these aren't _quite_ functional tests,
// and should all be compile_fail,
// but may be illustrative

#[test]
fn concurrent_set() {
    use std::sync::Arc;
    let x = Arc::new(Cell::new(42));
    let x1 = Arc::clone(&x);
    std::thread::spawn(move || {

hailelagi

Recon and Attack Vectors from My Logs

This document contains excerpts from my web server logs collected over a period of 7 years that shows various kinds of recon and attack vectors.

There were a total of 37.2 million lines of logs out of which 1.1 million unique HTTP requests (Method + URI) were found.

$ sed 's/^.* - - \[.*\] "\(.*\) HTTP\/.*" .*/\1/' access.log > requests.txt

hailelagi

How to think about Monads

Initially, Monads are the biggest, scariest thing about Functional Programming and especially Haskell. I've used monads for quite some time now, but I didn't have a very good model for what they really are. I read Philip Wadler's paper Monads for functional programming and I still didnt quite see the pattern.

It wasn't until I read the blog post You Could Have Invented Monads! (And Maybe You Already Have.) that I started to see things more clearly.

This is a distillation of those works and most likely an oversimplification in an attempt to make things easier to understand. Nuance can come later. What we need when first learning something is a simple, if inaccurate, model.

This document assumes a beginner's knowledge of pure functional programming and Haskell with some brief encounters of Monads, e.g. [Functors, Applicatives, And

hailelagi

// is Just(..) a monad? Well, it's a monad constructor.
// Its instances are certainly monads.
function Just(v) {
  return { map, chain, ap };
  function map(fn) {
    return Just(fn(v));
  }
  function chain(fn) {
    return fn(v);
  }

hailelagi

// connect() is a function that injects Redux-related props into your component.
// You can inject data and callbacks that change that data by dispatching actions.
function connect(mapStateToProps, mapDispatchToProps) {
  // It lets us inject component as the last step so people can use it as a decorator.
  // Generally you don't need to worry about it.
  return function (WrappedComponent) {
    // It returns a component
    return class extends React.Component {
      render() {
        return (