Skip to content

Instantly share code, notes, and snippets.

View mtreg's full-sized avatar

Mike Treglia mtreg

22 followers · 9 following
View GitHub Profile
@mtreg
mtreg / gist:1cdcd5bc4fb246e2b667
Last active August 29, 2015 14:11
Download, Unzip, Mosaic, and Reproject National Elevation Dataset 1/3 arcsecond
#All of this tested in in Cygwin on Windows 7 x64
#Requires GDAL, Cygwin (or other *nix terminal) with wget and unzip installed; also requires an installation of Python
#GDAL should be in system PATH
#To download files based on a hand-made text file called "URLZipList.txt" - list of all the .zip filenames needed (files from here: ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/NED/13/IMG/):
#Example file name: "ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/NED/1/IMG/n39w096.zip"
wget -i URLZipList.txt
#Unzip all files to respective folders
for z in *.zip; do
@mtreg
mtreg / gist:b44b2892170487895a3b
Created May 25, 2015 03:32
Round Time to Nearest 10 minutes - No dependencies (ignoring seconds)
et <- example_time <- Sys.time()
#This code will work on list of times, or can easily be set up to work on a column of a data frame
#The seconds are ignored and actually set to 0 (which was adequate for my needs)
for(i in et){
if(as.POSIXlt(et)$min >= 10)
{
if((as.POSIXlt(et)$min/5) %% 1 == 0)
{
et <- et + 60
@mtreg
mtreg / makefile_syllabusHTML.sh
Created January 7, 2016 22:31
Markdown to HTML via Pandoc with appropriate header and footer
#!bin/bash
pandoc ../CourseInfo/Syllabus.md -o syllabus_tmp.html;
cat html_head.html syllabus_tmp.html html_foot.html > ../Syllabus.html;
rm syllabus_tmp.html
@mtreg
mtreg / Download_Unzip.R
Last active August 16, 2016 18:54
Issue Unzipping NYC OpenData Download File in R
#Initiate Download of file, naming it as test.zip
download.file(url='https://data.cityofnewyork.us/api/geospatial/tqmj-j8zm?method=export&format=Original', destfile="test.zip")
#unzip the file using built in functionality
unzip('test.zip')
#Throws error when running on Windows:
#Warning message:
#In unzip("test.zip") : error 1 in extracting from zip file
@mtreg
mtreg / RODBC_Install_Output.R
Created March 2, 2017 17:39
Output of installing RODBC on ubuntu 14.04
> install.packages("RODBC")
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
--- Please select a CRAN mirror for use in this session ---
trying URL 'https://cloud.r-project.org/src/contrib/RODBC_1.3-14.tar.gz'
Content type 'unknown' length 1163799 bytes (1.1 MB)
==================================================
downloaded 1.1 MB
* installing *source* package ‘RODBC’ ...
@mtreg
mtreg / NYHOPS_netCDF_Example.R
Created March 3, 2017 04:21
Load Stevens Institute NYHOPS netCDF data into R
#Sample Code
#Gets netCDF data for lat/long points from Stevens Institute NYHOPS Data
library(ncdf4)
#Connect to netCDF
nyhops.nc <- nc_open("http://colossus.dl.stevens-tech.edu/thredds/dodsC/LISS/Hindcast/all_nb_mon_81.nc")
##To get 'simple', 3D variale (e.g., depth)
#Load depth, lat, and long
@mtreg
mtreg / FacDB_pgtable_unique.txt
Last active May 8, 2017 15:37
FacDB pgtable Unique Entries
pgtable
usnps_facilities_parks
usdot_facilities_airports
dcas_facilities_colp;dot_facilities_parkingfacilities
dcas_facilities_colp;omb_facilities_libraryvisits
bic_facilities_tradewaste
dfta_facilities_contracts;hhs_facilities_proposals
acs_facilities_daycareheadstart;doe_facilities_universalprek;dohmh_facilities_daycare
doe_facilities_schoolsbluebook;hhs_facilities_proposals
nycdoc_facilities_corrections
@mtreg
mtreg / TopoWx_Daily_CroppedSubset.R
Created September 16, 2017 14:57
Acquire and Crop Specific TopoWx Subsets of Daily Data in R
####This code allows pulling of daily TopoWx data from an OpenDap server, coarsely selecting specific days
####Opendap Access form at: https://cida.usgs.gov/thredds/dodsC/topowx.html
####for ncdf4 to pull from OpenDap, cannot be run on windows unless specifically compiled for such;
####see notes in this thread: https://stat.ethz.ch/pipermail/r-help/2015-July/430161.html
#Load necessary packages
library(raster)
library(ncdf4)
#Set call to dataset via OpenDap
@mtreg
mtreg / rmapzen_NYCGeosearch.R
Created February 13, 2018 00:25
Troubleshooting rmapzen with NYC Geosearch
#############
#Windows
#############
devtools::install_github("tarakc02/rmapzen", ref = "devel")
library(rmapzen)
options(RMAPZEN_SEARCH_HOST = "http://geosearch.planninglabs.nyc")
Sys.setenv("MAPZEN_KEY" = NA)
mz_search("1220 Fifth Ave, New York, NY")
@mtreg
mtreg / import_map_nyc_cdprofiles.R
Last active February 15, 2018 18:09
Import , NYC Comm Dist boundaries, Community Profiles, Join them, and Map them
#Import community district boundaries
#relies on sf package
#ggplot2 plotting should also work, but needs dev version to deal with sf objects
cds <- sf::st_read("https://data.cityofnewyork.us/resource/jp9i-3b7y.GeoJSON")
#import community profile data - note, you can adjust this line to select specific variables [all are imported by default as written]
cdprofiles <- read.csv("https://planninglabs.carto.com/api/v2/sql?format=csv&q=SELECT%20cartodb_id,the_geom,the_geom_webmercator,shared_puma,future_fp_mhhi,current_fp_mhhi,cd_tot_resunits,future_fp_resunits,future_fp_bldg,current_fp_resunits,cd_tot_bldgs,current_fp_bldg,moe_under18_rate_nyc,moe_under18_rate_boro,total_lot_area,lot_area_other_no_data,lot_area_parking,lot_area_open_space,lot_area_public_facility_institution,lot_area_transportation_utility,lot_area_industrial_manufacturing,lot_area_commercial_office,lot_area_mixed_use,lot_area_res_multifamily_elevator,lot_area_res_multifamily_walkup,lot_area_res_1_2_family_bldg,lots_total,lots_other_no_data,lots_