4u9u5tsong

Useful Spacemacs commands

• SPC q q - quit
• SPC w / - split window vertically
• SPC w - - split window horizontally
• SPC 1 - switch to window 1
• SPC 2 - switch to window 2
• SPC w c - delete current window
• SPC TAB - switch to previous buffer
• SPC b b - switch buffers

4u9u5tsong

Latency Comparison Numbers
--------------------------
L1 cache reference                           0.5 ns
Branch mispredict                            5   ns
L2 cache reference                           7   ns                      14x L1 cache
Mutex lock/unlock                           25   ns
Main memory reference                      100   ns                      20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy             3,000   ns        3 us
Send 1K bytes over 1 Gbps network       10,000   ns       10 us
Read 4K randomly from SSD*             150,000   ns      150 us          ~1GB/sec SSD

4u9u5tsong

1. General Background and Overview

• Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
• Models and Issues in Data Stream Systems
• Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
• Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
• [Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep

4u9u5tsong

compile

1. in DRAMSim2 folder: make libdramsim.so
2. in Marss: scons dramsim=/full/path/to/DRAMSim2

flags

DRAMSim2/Makefile: comment -DLOG_OUTPUT for extra logs

4u9u5tsong

ssh -f -L 24800:127.0.0.1:24800 128.151.160.230 -N

4u9u5tsong

def float64Tobinary(num):
    return ''.join(bin(ord(c)).replace('0b', '').rjust(8, '0') for c in
                   struct.pack('!d', num))


def convertTo1DIntArray(d, start, end, patternwidth, f):
    sview = 'S' + str(patternwidth)
    return f(d[start:end]).view(sview)

4u9u5tsong

# Got an answer from the Scipy user group:
# http://stackoverflow.com/questions/8955448/save-load-scipy-sparse-csr-matrix-in-portable-data-format
# A csr_matrix has 3 data attributes that matter: .data, .indices, and .indptr. All are simple ndarrays, so numpy.save will work on them. Save the three arrays with numpy.save or numpy.savez, load them back with numpy.load, and then recreate the sparse matrix object with:

# new_csr = csr_matrix((data, indices, indptr), shape=(M, N))
# So for example:

def save_sparse_csr(filename,array):
    np.savez(filename,data = array.data ,indices=array.indices,
             indptr =array.indptr, shape=array.shape )

4u9u5tsong

inpstream = subprocess.check_output(cmdstr).split('\n')
data = sp.genfromtxt(inpstream, dtype=np.uint64)

4u9u5tsong

#!/usr/bin/env python

import struct
import scipy as sp
import numpy as np


def binary(num):
    return ''.join(bin(ord(c)).replace('0b', '').rjust(8, '0') for c in struct.pack('!d', num))

4u9u5tsong

    ifstream pbinfile(argv[2]);
    
    vector<unordered_set<int> > numbers;
    while(getline(pbinfile, stemp)){
      istringstream buffer(stemp);
      vector<int> line((istream_iterator<int>(buffer)),
                       istream_iterator<int>());
      unordered_set<int> s(line.begin(), line.end());
      numbers.push_back(s);
    }