seanickle

given a dataframe, scrub cols

• two options:
  • either fill col with random data
  • or replace w random data, but by hashing the values instead so that repete are consistent

import uuid
import pandas as pd

def make_random_phone_dict(phones_list):

seanickle

Basic string matching tools

import nltk
from copy import deepcopy

def best_match_terms(from_pair, end_list, cutoff=10, include_distances=False):
    '''
    from_pair: [from_identifier, from_term] 
    '''
    [from_identifier, from_term] = from_pair

seanickle

1 tail vs 2 tail

• hmm, https://help.optimizely.com/Analyze_Results/False_discovery_rate_control , another way to choose?

how large does the sample size need to be for a test

• well, it comes down to the requirement of the confidence level, that is, reaching statistical significance,
• per [1] , this is phrased as, what rate of false positive are we willing to accept

Baseline conversion rate

seanickle

```

digraph structs {
    node [shape=plaintext];

    struct1 [label=<<TABLE>
    <TR><TD><IMG SRC="jpegs/Screen Shot 2017-07-19 at 11.08.56 AM.jpg"/></TD></TR>
    <TR><TD>caption1 </TD></TR>

seanickle

Downsize image sizes

• per https://superuser.com/questions/571003/how-to-convert-gif-files-to-png-or-jpeg-in-os-x-with-command-line#571068
• Take all .png and convert them to jpg

mkdir jpegs
sips -s format jpeg ./*.png --out jpegs

• Also, we can downgrade the quality as

sips -s format jpeg -s formatOptions low ./*.png --out jpegs

seanickle

set ignorecase
syntax on

set number

set softtabstop=4 
set tabstop=4 
set shiftwidth=4 
set expandtab 

seanickle

# from multiprocessing import Process
# from multiprocessing import Pool
from multiprocessing import Process, Pipe

import cPickle

 
def foo(multiproc, chunk_ids, feature_names, child_conn):
    child_conn.send([chunk_ids, feature_names])

seanickle

• foo

last_n=15
# tags=($(git tag | egrep '^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$'))
tags=($(git tag --sort version:refname | egrep '^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$'))
begin=`expr ${#tags[@]} - ${last_n}`
subset_tags=(${tags[@]:${begin}})
# for (( i=`expr ${#subset_tags[@]} - 1`; i>=0 ; i-=1 )) ;   # reverse
for (( i=0; i< `expr ${#subset_tags[@]} - 1` ; i+=1 )) ; # forwards

seanickle

* numpy bool 
```python
ipdb> pp outdf
      a  b        c    d      e
0  None  a       []  foo   True
1   3.4  b   [asdf]   no  False
2    34  0  [{}, 0]  meh  False
...
...
ipdb> pp type(outdf.ix[0,'e'])

seanickle

• Can modify query search with __iregex for some regex based querying.

In [713]: BlahModel.objects.filter(phone__iregex=r"\d\d\d\d\d\d\d\d\d\d\d\d").val
     ...: ues_list('phone')
Out[713]: [(u'51615415135135',), (u'3424234234234',), (u'0971557335940',), (u' 923006306666',), (u' 923006306666',), (u' 923006306666',), (u'491725152799',), (u'4921195799170',), (u'527821398854',), (u'860290084656',)]