Skip to content

Instantly share code, notes, and snippets.

View robertobarreda's full-sized avatar

Roberto Barreda robertobarreda

View GitHub Profile
@robertobarreda
robertobarreda / latency_numbers.txt
Last active November 17, 2016 14:23
Latency numbers every programmer should know
### Latency numbers every programmer should know
L1 cache reference ......................... 0.5 ns
Branch mispredict ............................ 5 ns
L2 cache reference ........................... 7 ns
Mutex lock/unlock ........................... 25 ns
Main memory reference ...................... 100 ns
Compress 1K bytes with Zippy ............. 3,000 ns = 3 µs
Send 2K bytes over 1 Gbps network ....... 20,000 ns = 20 µs
SSD random read ........................ 150,000 ns = 150 µs
Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs
@robertobarreda
robertobarreda / is_mobile.py
Created October 3, 2013 14:09
django.request is_mobile?
import re
USER_AGENT_REGEX = re.compile(
r"android|avantgo|blackberry|blazer|compal|elaine|fennec|hiptop|iemobile|"
r"ip(hone|od)|iris|kindle|lge |maemo|midp|mmp|opera m(ob|in)i|palm( os)?|"
r"phone|p(ixi|re)\\/|plucker|pocket|psp|symbian|treo|up\\.(browser|link)|"
r"vodafone|wap|indows (ce|phone)|xda|xiino", re.M)
def is_mobile(request):
ua = request.META.get('HTTP_USER_AGENT', '').lower()
@robertobarreda
robertobarreda / mysql_to_redshift.py
Created December 19, 2014 10:34
mysql to postresql schema converter
#!/usr/bin/env python
"""
Fixes a MySQL dump made with the right format so it can be
imported to a Redshift database.
Dump using:
mysqldump --compatible=postgresql --default-character-set=utf8 -r databasename.mysql -u root databasename tablename
"""
#!/usr/bin/env python
import argparse
import csv
import gzip
import sys
import re
from contextlib import closing
# This prevents prematurely closed pipes from raising
# an exception in Python
from signal import signal, SIGPIPE, SIG_DFL
/* crc64.c -- compute CRC-64
* Copyright (C) 2013 Mark Adler
* Version 1.4 16 Dec 2013 Mark Adler
*/
/*
This software is provided 'as-is', without any express or implied
warranty. In no event will the author be held liable for any damages
arising from the use of this software.
@robertobarreda
robertobarreda / quantile.py
Last active December 28, 2021 13:02
Python Implementation of Graham Cormode and S. Muthukrishnan's Effective Computation of Biased Quantiles over Data Streams in ICDE’05 (https://github.com/matttproud/python_quantile_estimation)
#!/usr/bin/python
"""
Copyright 2013 matt.proud@gmail.com (Matt T. Proud)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@robertobarreda
robertobarreda / count_min_sketch.py
Created July 22, 2015 13:17
CountMinSketch is an implementation of the count min sketch algorithm that probabilistically counts string frequencies.
from __future__ import division
from xxhash import xxh32
import numpy as np
DTYPE = np.int64
class CountMinSketch(object):
@robertobarreda
robertobarreda / tdigest.py
Last active September 29, 2015 12:41
Efficient percentile estimation of streaming or distributed data - https://github.com/CamDavidsonPilon/tdigest
from __future__ import print_function
from random import choice
from bintrees import FastRBTree as RBTree
import pyudorandom
from itertools import chain
class Centroid(object):
def __init__(self, mean, count):
@robertobarreda
robertobarreda / Vagrantfile
Last active June 27, 2020 15:18
JEDI SI PAC1 (system security)
Vagrant.configure("2") do |config|
config.vm.box = "debian/buster64"
# config.vm.synced_folder ".", "/vagrant", disabled: true
config.vm.provider "virtualbox" do |vb|
vb.memory = "256"
end
config.vm.define :proxy, primary: true do |proxy|
proxy.vm.hostname = "proxy"
@robertobarreda
robertobarreda / Vagrantfile
Last active July 3, 2020 07:16
JEDI SI PAC3 (Web Vulnerabilities)
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.box = "debian/buster64"
config.vm.network "forwarded_port", guest: 9090, host: 9090
# config.vm.synced_folder ".", "/vagrant", disabled: true
config.vm.provider "virtualbox" do |vb|
vb.memory = "256"
end