Skip to content

Instantly share code, notes, and snippets.

View jseabold's full-sized avatar

Skipper Seabold jseabold

740 followers · 12 following
View GitHub Profile
@jseabold
jseabold / frisch_waugh_lovell.py
Last active December 25, 2015 05:09
"""
Frisch-Waugh-Lovell Theorem.
In a multiple regression, the effect of any single covariate can be obtained
by removing the effects of the other covariates from both the dependent and
independent variables.
"""
import numpy as np
import statsmodels.api as sm
@jseabold
jseabold / chroot32.md
Last active December 26, 2015 10:49
Create a 32-bit chroot environment

Create the chroot

sudo apt-get install debootstrap schroot
CHROOT=/tmp/saucy-i386; mkdir -p $CHROOT; sudo debootstrap --arch=i386 --include=python-pandas,sudo,wget --components=main,universe,multiverse saucy $CHROOT http://debian.lcs.mit.edu/ubuntu

echo -e "[saucy-i386]\ndescription=Ubuntu saucy  i386 architecture\ndirectory=$CHROOT\ntype=directory\nusers=$USER\nroot-groups=root\n" | sudo bash -c 'cat - > /etc/schroot/chroot.d/saucy-i386'

Enter the chroot

@jseabold
jseabold / write_dates.R
Created January 3, 2014 20:46
Write dates to file in R. Must be a better way. What is this? SAS?
library(fpp)
data(usmelec)
write.csv(as.data.frame(list(date=as.Date(usmelec), usmelec=coredata(usmelec))), "usmelec.csv", row.names=FALSE)
@jseabold
jseabold / math.tex
Created January 9, 2014 23:36
Cheat sheet for stochastic independence and conditional probability in latex.
% stochastic independence
\newcommand\independent{\protect\mathpalette{\protect\independenT}{\perp}}
\def\independenT#1#2{\mathrel{\rlap{$#1#2$}\mkern2mu{#1#2}}}
% conditional probability pipe
\mid
@jseabold
jseabold / sm_install.sh
Last active August 29, 2015 13:56
Minimal working environment for virtualenv for statsmodels
export BLAS=/home/skipper/.local/lib/libopenblas.a
export LAPACK=/home/skipper/.local/lib/libopenblas.a
pip install numpy
pip install scipy
pip install nose
pip install patsy
pip install Cython
pip install pandas
@jseabold
jseabold / AT_Austria.csv
Created February 14, 2014 20:32
Example for failing GLM
We can make this file beautiful and searchable if this error is corrected: It looks like row 10 should actually have 28 columns, instead of 11. in line 9.
Origin,Destination,Data,Oi,Dj,Dij,Offset,beta,OrigAT11,OrigAT12,OrigAT13,OrigAT21,OrigAT22,OrigAT31,OrigAT32,OrigAT33,OrigAT34,DestAT11,DestAT12,DestAT13,DestAT21,DestAT22,DestAT31,DestAT32,DestAT33,DestAT34,Oi2007,Dj2007
AT11,AT11,0,4016,5146,9.999999999999994e-301,9.999999999999994e-301,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4320,5452
AT11,AT12,1131,4016,25741,103.00184499999999,1.0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,4320,27169
AT11,AT13,1887,4016,26980,84.204666,1.0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,4320,28710
AT11,AT21,69,4016,4117,220.811933,1.0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,4320,4354
AT11,AT22,738,4016,8634,132.00748000000002,1.0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,4320,9069
AT11,AT31,98,4016,8193,214.51181400000002,1.0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,4320,8577
AT11,AT32,31,4016,4902,246.933305,1.0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,4320,4963
AT11,AT33,43,4016,3952,390.85611,1.0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,4320,3923
AT11,AT34,19,4016,1910,505.08953899999995,1.0,1,0,0,0
@jseabold
jseabold / logging_loops.py
Last active August 29, 2015 13:56
Multiple logs in a loop
import logging
for i in range(5):
for j in range(10, 15):
logger = logging.getLogger("log_{}_{}.log".format(i, j))
handler = logging.FileHandler("log_{}_{}.log".format(i, j))
logger.setLevel(logging.INFO)
logger.addHandler(handler)
logger.info("I'm on this loop")
@jseabold
jseabold / read_scan.py
Last active August 29, 2015 13:57
Quick no-features version of R's scan to be used for URLs only
from urllib2 import urlopen
from itertools import chain
def scan(url, sep=" ", skiprows=0):
f = urlopen(url).readlines()[skiprows:]
return map(float, chain(*map(lambda x : x.strip().split(sep), f)))
if __name__ == "__main__":
url = "http://robjhyndman.com/tsdldata/hurst/precip1.dat"
scan(url, 1)
@jseabold
jseabold / memmap_sum.py
Created March 26, 2014 21:11
Just messing around with memmap arrays in response to a mailing list question.
N = 2*10**7
chunk_size = 100000
farr1 = 'scratch/arr1'
farr2 = 'scratch/arr2'
arr1 = np.memmap(farr1, dtype='uint8', mode='w+', shape=(N, 4))
arr2 = np.memmap(farr2, dtype='uint8', mode='w+', shape=(N, 4))
for i in xrange(0, N, chunk_size):
@jseabold
jseabold / build.log
Created April 2, 2014 14:21
Failure of Cython build. `cython3 -a mwe.pyx && gcc -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing -I/usr/include/python3.3m -I/home/skipper/.local/lib/python3.3/site-packages/numpy/core/include -o mwe.so mwe.c`
In file included from /home/skipper/.local/lib/python3.3/site-packages/numpy/core/include/numpy/ndarraytypes.h:1761:0,
from /home/skipper/.local/lib/python3.3/site-packages/numpy/core/include/numpy/ndarrayobject.h:17,
from /home/skipper/.local/lib/python3.3/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from mwe.c:346:
/home/skipper/.local/lib/python3.3/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it by " \
^
In file included from /usr/include/python3.3m/pytime.h:6:0,
from /usr/include/python3.3m/Python.h:65,
from mwe.c:16: