This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from sys import argv | |
import re | |
name, file_path = argv | |
p = re.compile(r"[\s]\(\((.*?[)]{0,1})\)\)[\s]{0,1}") | |
# The tricky part here is to match all text between (()), including as many as | |
# one set of (), which may even terminate ))). The {0,1} captures as many as | |
# one ). The trailing space is there because I often surrounded the (()) with | |
# a space to make it clear in the Wordpress editor. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
modal_person_attribute <- function(df, attribute){ | |
# df: rbind of all person tables from all years | |
# attribute: vector name to calculate the modal value | |
# Calculate the number of instances an attributed is associated with an id | |
mode <- do.call(rbind, | |
tapply(as.character(df[[attribute]]), df$sasid, | |
function(x) data.frame(attribute=rle(x)$values, | |
counts=rle(x)$lengths))) | |
names(mode) <- c(as.character(attribute), 'counts') | |
# Clean up |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
modal_person_attribute <- function(df, attribute){ | |
# df: rbind of all person tables from all years | |
# attribute: vector name to calculate the modal value | |
# Calculate the number of instances an attributed is associated with an id | |
dt <- data.table(df) | |
mode <- dt[, rle(as.character(.SD[[attribute]])), by=sasid] | |
setnames(mode, c('sasid', 'counts', as.character(attribute))) | |
setkeyv(mode, c('sasid', 'counts')) | |
# Only include attributes with the maximum values. This is equivalent to the | |
# mode with two records when there is a tie. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cutoff_matrix <- function(df, classifier, outcome, breaks=seq(1,0,-.01)){ | |
results <- data.frame(cutoff=vector(mode='numeric', length=length(breaks)), | |
true_pos=vector(mode='numeric', length=length(breaks)), | |
true_neg=vector(mode='numeric', length=length(breaks)), | |
false_pos=vector(mode='numeric', length=length(breaks)), | |
false_neg=vector(mode='numeric', length=length(breaks))) | |
for(i in seq(1, length(breaks))){ | |
value <- breaks[i] | |
j <- data.frame(table(df[[classifier]]>value, df[[outcome]])) | |
results[i,1] <- value |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<!-- In your content.md meta data include a link attribute like this --> | |
Link: http://www.getpelican.com | |
<!-- In your articles.html template --> | |
{% block content %} | |
{% if article.link %} <!-- checks if article has link as a metadata attribute --> | |
<h1 class="title"><a href='{{ article.link }}' title="Link">{{ article.title }}</a></h1> | |
{% else %} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import sys | |
import os, shutil, time | |
import subprocess | |
import os.path | |
import exifread | |
from datetime import datetime | |
def photoDateSIPS(f): | |
"Return the date/time on which the given photo was taken." |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/python | |
import requests | |
import sys | |
from re import search | |
from subprocess import check_output | |
url = check_output('pbpaste') | |
r = requests.get(url) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
install.packages('rvest') | |
library(rvest) | |
library(dplyr) | |
page <- html('http://app.providencejournal.com/topics/special-reports/tables/firefigher-salaries-fy2014/firefighter-salaries.htm') | |
data <- page %>% | |
html_node('table') %>% | |
html_table() %>% | |
.[c(-1,-2,-3),] %>% | |
as.data.frame |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Current pattern | |
function(myList){ | |
do_stuff_with(mylist$attribute1, myList$attribute2) | |
do_stuff_with(mylist$attribute3, myList$attribute4) | |
} | |
## Desired pattern | |
function(myList){ | |
do_stuff_with(attribute1, attribute2) | |
do_stuff_with(attribute4, attribute4) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Ever want to log how long someting takes? I know I do. timing() | |
# isn't about benchmarking, it's about getting feedback from long running | |
# tasks, especially if they are scheduled in production. Anyway, I love | |
# this function because it's a simple wrapper that can go around anything. | |
# I use it inside our tools at Allovue when extracting data to get things | |
# like this: | |
#> accounts <- extract_data(config_file$accounts) | |
# Starting queries/accounts.sql at: Mon Sep 19 15:27:34 2016 | |
# Completed at: Mon Sep 19 15:28:14 2016 |
OlderNewer