Skip to content

Instantly share code, notes, and snippets.

@justgrimes
justgrimes / opm-fedscope-workforce-data.r
Last active February 21, 2017 22:06
OPM Fedscope Data - Cleaning and Loading
## OPM Workforce Data
## FedScope Employment Cube RAW Data Sets for general public consumption
## Justin Grimes
# Load necesary packages
#install.packages("curl")
#library(curl)
#install.packages("RCurl")
#library(RCurl)
@justgrimes
justgrimes / BuildingSMSappExample.md
Last active August 29, 2015 14:24
Simple SMS Application using Twilio, Lua, & webscripts.io

Building SMS Applications with Twilio

With over 3.5 billion active users, SMS (Short Message Service) is one of the most widely used data applications in the world. In the United States over 91% of adults own cell phone. Text messaging can be an incredibly useful way for delivering services and information to people.

So today, I'm going to show you how you can create a super simple light weight SMS application with just a few lines of code using Twilio, Lua, webscript.io, and Socrata's Open Data API (SODA). Twilio is a wonderful service that provides a HTTP API that allows people to build quick and easy SMS & voice applications. Lua is lightweight scripting language and [webscript.io](https://www.webscri

@justgrimes
justgrimes / gist:71fd98441d865714899a
Created July 7, 2015 22:52
awk one liner for checking lines for quote mismatches (aka odd quote counts by line)
awk -F'|' 'BEGIN{print "COUNT,", "LINE"}{print gsub(/"/,"") "," NR}' file
@justgrimes
justgrimes / plsmap.r
Created May 22, 2015 17:57
pls mapping
#PLS Dot Density Map
library(maps)
library(mapproj)
library(maptools)
library(geosphere)
library(splancs)
library(foreign)
# Map just the points
@justgrimes
justgrimes / pls12nbmcai14.r
Last active August 29, 2015 14:21
Public library broadband data - Loads, cleans, and merges Public Library Survey data with National Broadband Map Community Anchor Institution data
# Script to load public public library data from FY 2012 Public Library Survey (PLS) from Institute of Museum and Library Services
# and merge it with most recent broadband data from National Broadband Map Community Anchor Institution data
#
# FOR REFERENCE
# CAIID - Unique federal ID's associated with the CAICAT
# CAICAT -these are the unique federal ID's associated with the CAICAT
# MAXADDOWN - Maximum Advertised Download Speed subscribed to at this CAI
# MAXADUP - Maximum Advertised Upload Speed subscribed to at this CAI
# BBSERVICE - Does institution subscribe to broadband service at location (ex. Y=Yes; N=No; U=Unknown),
@justgrimes
justgrimes / SonifyPLS.R
Last active August 29, 2015 14:20
sonify public library survey data using R and audiolyzR
# Script to load public public library data from the most recent
# FY 2012 Public Library Survey (PLS) from Institute of Museum and Library Services
# Data documentation -> http://www.imls.gov/assets/1/AssetManager/fy2012_pls_data_file_documentation.pdf
# More info about data & survey -> http://www.imls.gov/research/public_libraries_in_the_united_states_survey.aspx
install.packages("audiolyzR")
library("audiolyzR")
# Load PLS 2012 directly from the web
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@justgrimes
justgrimes / gist:07079a03c66f2ad054d3
Created December 10, 2014 16:24
Introduction to R - SDC Annual Meeting 2014
#SDC Annual Meeting 2014 Introduction to R, R Studio, and Statistical Computing
# Load downloaded data from PLS 2012 survey, CSV format (comma seperated value)
pupld12a_csv <- read.csv("pupld12a_csv/pupld12a.csv", header = TRUE, sep=",", quote = "\"")
puout12a_csv <- read.csv("pupld12a_csv/puout12a.csv", header = TRUE, sep=",", quote = "\"")
pusum12a_csv <- read.csv("pupld12a_csv/pusum12a.csv", header = TRUE, sep=",", quote = "\"")
# Load downloaded data from PLS 2012 survey, Excel format (xls)
#install.packages("gdata")
library(gdata) #library to load xls data; if loading xlsx files use XLConnect library
@justgrimes
justgrimes / imls_pls_small_multiple.r
Created January 13, 2014 19:34
Downloads FY 2011 Public Library Survey data files from Institute of Museum and Library Services website and constructs a small multiple map of NCES locale codes.
# Load packages
library(maps) # To draw map
library(maptools)
# download data files from IMLS website
temp <- tempfile()
download.file("http://www.imls.gov/assets/1/AssetManager/pupld11b_csv.zip",temp)
pupld11b <- read.csv(unz(temp, "pupld11b.csv")) #file with library systems (admin buildings)
puout11b <- read.csv(unz(temp, "puout11b.csv")) #file with library outlets (all buildings)
unlink(temp)
@justgrimes
justgrimes / gist:3308085
Created August 9, 2012 21:07
text mining in r snippet
require(tm)
a <- Corpus(DirSource("C:/Users/jgrimes/Desktop/text/"), readerControl = list(language="lat"))
#summary(a)
a <- tm_map(a, function(x) iconv(enc2utf8(x), sub = "byte"))
a <- tm_map(a, removePunctuation)
a <- tm_map(a, removeNumbers)
a <- tm_map(a, stripWhitespace)
a <- tm_map(a, tolower)
a <- tm_map(a, removeWords, stopwords("english"))
a <- tm_map(a, stemDocument, language = "english")