Skip to content

Instantly share code, notes, and snippets.

View potterzot's full-sized avatar

Nicholas Potter potterzot

74 followers · 18 following
View GitHub Profile
@potterzot
potterzot / README.md
Last active August 26, 2015 18:33
Pesticide use map through time

##Pesticide Use Map The USDA surveys pesticide use across time. This is a demonstration of mapping pesticide use, but no effort has been made to interpolate interpolate years between surveys.

@potterzot
potterzot / index.html
Last active August 27, 2015 15:19
Create a simple web comic with p5js.
<!DOCTYPE html>
<html>
<head>
<title>p5js Example</title>
<script src="http://cdnjs.cloudflare.com/ajax/libs/p5.js/0.4.8/p5.js"></script>
<body>
<h1>Processing.js Test</h1>
<p>Using p5js to create simple canvas comic graphics... (sorry for the embarrassingly bad joke.)</p>
</body>
@potterzot
potterzot / keybase.md
Last active August 29, 2015 14:10
keybase.io proof

Keybase proof

I hereby claim:

  • I am potterzot on github.
  • I am potterzot (https://keybase.io/potterzot) on keybase.
  • I have a public key whose fingerprint is 9C73 3AAA 6D99 86F1 F8F4 601D 298F 8713 1BE6 5941

To claim this, I am signing this object:

@potterzot
potterzot / 20city_PctOfHomesSellingForLoss_AllHomes.csv
Last active August 29, 2015 14:11
City Housing Loss Through Time
We can make this file beautiful and searchable if this error is corrected: Unclosed quoted field in line 1.
"RegionName","State","Metro","CountyName","1998-01","1998-02","1998-03","1998-04","1998-05","1998-06","1998-07","1998-08","1998-09","1998-10","1998-11","1998-12","1999-01","1999-02","1999-03","1999-04","1999-05","1999-06","1999-07","1999-08","1999-09","1999-10","1999-11","1999-12","2000-01","2000-02","2000-03","2000-04","2000-05","2000-06","2000-07","2000-08","2000-09","2000-10","2000-11","2000-12","2001-01","2001-02","2001-03","2001-04","2001-05","2001-06","2001-07","2001-08","2001-09","2001-10","2001-11","2001-12","2002-01","2002-02","2002-03","2002-04","2002-05","2002-06","2002-07","2002-08","2002-09","2002-10","2002-11","2002-12","2003-01","2003-02","2003-03","2003-04","2003-05","2003-06","2003-07","2003-08","2003-09","2003-10","2003-11","2003-12","2004-01","2004-02","2004-03","2004-04","2004-05","2004-06","2004-07","2004-08","2004-09","2004-10","2004-11","2004-12","2005-01","2005-02","2005-03","2005-04","2005-05","2005-06","2005-07","2005-08","2005-09","2005-10","2005-11","2005-12","2006-01","2006-02","2
@potterzot
potterzot / SamplingDistributionAndSamplingRates
Last active August 29, 2015 14:23
Relationship between sampling distribution and sampling rates
## Motivation
Let's simulate the relationship between sampling rates and the sample distribution variation.
```{r }
pop = seq(1,1000,1)
sat.scores = rnorm(pop, mean=500, sd=100)
head(sat.scores)
df = data.frame(pop, sat.scores)
@potterzot
potterzot / global.R
Last active August 29, 2015 14:26
Sample of toggling ggplot2 layers in R shiny
library(ggplot2)
#Some large data that might take a lot of time to plot
set.seed(1225)
df = data.frame(run=1:10000, temp.hi = 100+10*rnorm(100000), temp.low = 30+20*rnorm(100000))
df$temp.avg = with(df, (temp.hi + temp.low)/2)
df$temp.75pct = with(df, temp.avg*1.5)
df$temp.25pct = with(df, temp.avg*0.5)
layerBase <- function() {
@potterzot
potterzot / ggplot2_theme.R
Created August 18, 2015 18:40
A consistent plot theme for ggplot2
#A theme for ggplot2 plots for consistent look and feel of graphics.
@potterzot
potterzot / README.md
Created September 10, 2015 19:14
Steps to create an sqlite3-compatible EC2 instance.

##Amazon EC2 and simple web server deployment

Often I want to create a simple web site that is nearly static with the exception of a simple SQL database (mostly to serve data of some form, occassionally to allow collecting it). These days with static data I'd probably do something like mongodb, but legacy has it's effects...

Anyway, steps (see a guide like this one:

  1. Spin up a micro instance of the free variety
  2. Install updates (sudo yum update) and httpd (sudo yum install httpd)
  3. Add a security rule to allow http access from the web
  4. Edit /etc/httpd/conf/httpd.conf to change DocumentRoot AllowAccessFrom to All
@potterzot
potterzot / README.Rmd
Last active September 17, 2015 21:40
Reproducible examples in R
```
devtools::install_github("jennybc/reprex")
library(reprex)
reprex({mean(1:4)})
```
If you get an error like:
>pandoc: loadlocale.c:131: _nl_intern_locale_data: Assertion `cnt < (sizeof (_nl_value_type_LC_COLLATE) / sizeof (_nl_value_type_LC_COLLATE[0]))' failed.
Error in reprex_(r_file, venue, show, upload.fun) :
@potterzot
potterzot / README.md
Last active September 27, 2015 19:07
Convert large zipped csv files to compressed RDS files for use in R.

##Convert zipped csv files to gzipped RDS files for use in R.

Note that at the moment, while we can read in in chunks and save without a problem, and the file size grows with each chunk, that the final RDS object only contains the first chunk saved. The connection stays open and so the writes of later chunks continue to happen, but aren't loaded when you load the RDS file. If someone knows of a way to make this work I'd love to hear about it!

Convert zipped csv files that are too large to read directly into memory in R into RDS files, which are equivalent in size to a zipped csv and can be read in directly and much more quickly in R. A 2GB csv file can crash things on your laptop, but with this function you can convert it to a ~250MB RDS file and read that directly with no problem.

The function uses chunks to avoid reading a lot of data into R at once. Right now the number of chunks is limited to the letters in the alphabet, so csv files larger than 10 or GB might not work well, but then you should pro