Neil Best nbest937
nbest937
/ sparkSessionOnEMR.scala
Last active
October 3, 2018 22:11
— forked from dyno/sparkSessionOnEMR.scala
Working with SparkSession with Ammonite .on AWS EMR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // ============================================================================= | |
| // emr.sc -- import it in predef.sc | |
| import ammonite.ops._ | |
| // add emr jars and config dir to interp class path | |
| // | |
| def loadEmrJars() = { | |
| // /etc/zeppelin/conf/zeppelin-env.sh | |
| val libDirs = List( // | |
| "/usr/lib/hadoop-lzo/lib", | |
| "/usr/lib/spark/jars", |
nbest937
/ GreatCircleFlights.R
Created
February 20, 2014 23:20
— forked from cengel/GreatCircleFlights.R
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| library(maps) | |
| library(geosphere) | |
| library(plyr) | |
| library(ggplot2) | |
| library(sp) | |
| airports <- read.csv("http://www.stanford.edu/~cengel/cgi-bin/anthrospace/wp-content/uploads/2012/03/airports.csv", as.is=TRUE, header=TRUE) | |
| flights <- read.csv("http://www.stanford.edu/~cengel/cgi-bin/anthrospace/wp-content/uploads/2012/03/PEK-openflights-export-2012-03-19.csv", as.is=TRUE, header=TRUE) | |
| # aggregate nunber of flights |