Skip to content

Instantly share code, notes, and snippets.

View maptastik's full-sized avatar

Ryan Cooper maptastik

View GitHub Profile
@maptastik
maptastik / remoteGeoJSONToGDF.py
Last active February 2, 2020 15:42
Pull GeoJSON from the toobz and create a GeoDataFrame
import requests, json, geopandas as gpd
def remoteGeoJSONToGDF(url, display = False):
"""Import remote GeoJSON to a GeoDataFrame
Keyword arguments:
url -- URL to GeoJSON resource on web
display -- Displays geometries upon loading (default: False)
"""
r = requests.get(url)
data = r.json()
@maptastik
maptastik / arcgis_to_csv.py
Created November 26, 2019 16:30
A quick method for writing out some rows and a subset of columns from a feature class to a CSV. This is particularly well-suited for when you're working in ArcGIS Desktop or Pro and need to quickly export some rows and a subset of columns from a vect
import arcpy
import csv
with open('output.csv', 'w', newline = '') as csvfile:
writer = csv.writer(csvfile) # Create writer object
writer.writerow(['Column 1', 'Column 2', 'Column 3', 'Column 4']) # Define the header row
with arcpy.da.SearchCursor('input_layer', ['Field1', 'Field2', 'Field3', 'Field4']) as cursor: # Define cursor object
for row in cursor:
writer.writerow([row[0], row[1], row[2], row[3]]) # Write values from each item in cursor to csv
@maptastik
maptastik / geojson_csv_join.bat
Created November 22, 2019 17:15
An example of an ETL script that allows you to query a remote spatial data source and join the query result with a CSV. For now, this requires csvs-to-sqlite as ogr2ogr doesn't seem to like working with CSVs with the SQLite dialect for SQL queries. O
csvs-to-sqlite tbl.csv tbl.db && \
curl "<URL to enpoint and/or query that returns GeoJSON>" | \
ogr2ogr -f geojson -nln input /vsistdout/ /vsistdin/ | \
ogr2ogr -f geojson \
-dialect sqlite \
-sql "select input.*, tbl.* from inputjoin 'tbl.db'.tbl AS tbl on input.<primary key> = tbl.<foreign key>" \
output.geojson /vsistdin/ && \
del tbl.db
@maptastik
maptastik / ebpa_routes_sample.geojson
Created October 28, 2019 13:19
EBPA sample routes for a single census block
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@maptastik
maptastik / citrix_bike_routes_sample10.geojson
Last active October 28, 2019 13:17
Sample Citrix Cycle route data
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
ffmpeg -i animated.gif -movflags faststart -pix_fmt yuv420p -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" video.mp4
import pandas as pd
df = pd.DataFrame({'a': [1,2,3], 'b': [2,3,4], 'c':['dd','ee','ff'], 'd':[5,9,1]})
df['e'] = df[['a', 'b', 'd']].sum(axis = 1)
#########################
## RESULT ##
#########################
# a b c d e # COLUMNS
# 0 1 2 dd 5 8 # ROW
def quantile_bins(quantiles, min = 0, max = 100):
"""Return a list of quantiles including floor and ceiling values
Args:
quantile (list): Quantile values in order
min (numeric): Floor value of data range
max (numeric): Ceiling value of data range
Returns:
list: An inclusive list of quantile bin values
def quantile_percentiles(quantile_count: int = 5) -> list:
"""Return quantile percentile values for a given number of quantiles.
Args:
quantile_count (int): Number of quantiles to generate percentile values for (default 5)
Returns:
list: Equally spaced quantile percentile values based on quantile_count
Examples:
-- ABOUT -----------------------------------------------------------------------
-- This script is meant to outline a procedure for updating a PostgreSQL table
-- with the latest features even if there are views that depend on the table.
-- In other words, this method allows you to update the table without deleting
-- it and recreating it with the latest data. There are definitely some
-- opportunities for improvement - there may be a bit too much redundancy - but
-- this should get the job done, especially on datasets where size is trivial.
--------------------------------------------------------------------------------
-- NOTES -----------------------------------------------------------------------