Skip to content

Instantly share code, notes, and snippets.

Chris Whong chriswhong

Block or report user

Report or block chriswhong

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@chriswhong
chriswhong / gist:762ceac7fb8a1420e7e7adceb770b707
Last active Sep 18, 2018
Using ST_AsMVT() and ST_AsMVTGeom() in express to build a vector tile endpoint
View gist:762ceac7fb8a1420e7e7adceb770b707
/* GET /tiles/:z/:x/:y.mvt */
/* Retreive a vector tile by tileid */
router.get('/tiles/:z/:x/:y.mvt', async (req, res) => {
const { z, x, y } = req.params;
// calculate the bounding polygon for this tile
const bbox = mercator.bbox(x, y, z, false);
// Query the database, using ST_AsMVTGeom() to clip the geometries
// Wrap the whole query with ST_AsMVT(), which will create a protocol buffer
@chriswhong
chriswhong / RadiusMode.js
Created Mar 1, 2018
RadiusMode, a custom mode for mapbox-gl-draw for drawing a radius
View RadiusMode.js
// custom mapbopx-gl-draw mode that modifies draw_line_string
// shows a center point, radius line, and circle polygon while drawing
// forces draw.create on creation of second vertex
import MapboxDraw from 'mapbox-gl-draw';
import numeral from 'numeral';
import lineDistance from 'npm:@turf/line-distance';
const RadiusMode = MapboxDraw.modes.draw_line_string;
@chriswhong
chriswhong / README.md
Created Feb 27, 2018
Consuming USGS OrthoImagery in MapboxGL
View README.md

MapboxGL Layer Definition for consuming tiles from the USGS Imagery MapServer

{
  id: 'usgs-aerials',
  type: 'raster',
  source: {
    type: 'raster',
    tiles: [
      'https://basemap.nationalmap.gov/arcgis/rest/services/USGSImageryOnly/MapServer/tile/{z}/{y}/{x}',
    ],
@chriswhong
chriswhong / geosupport-autocomplete.md
Last active Nov 6, 2017
An autocomplete web API for geosupport
View geosupport-autocomplete.md

Problem

DCP maintains geosupport, the city's official geocoder, which can be accessed by using the libraries directly, or via various client-facing application/services including GOAT, GeoClient, GeoBat, etc.

The NYC Planning Labs team would like to incorporate geosupport results into web mapping applications, but geosupport's architecture does not work well with the "autocomplete" search style that is popular in modern web applications. Traditionally, to use geosupport, the user must provide at least a house number, street name, and borough, and will either git a hit, or a list of possible alternatives.

Below we propose some high-level specifications for a GeoSupport Autocomplete Web API that would be useful for search in modern web applications.

A Model to Emulate: Mapzen Search's Autocomplete API

Mapzen Search has an autocomplete API, which is well-documente, open source, and provides the type of experience developers want in a modern g

@chriswhong
chriswhong / README.md
Created Aug 8, 2017
A Vector Tile microservice for MapPLUTO (or any other data)
View README.md

Background

We like building mapping applications with carto as the backend, because it gets us an instant map tiler and SQL API. However, carto's tiler is raster-based, and we would like to take advantage of mapboxGL for it's vector-based rendering. We discovered an undocumented vector tile endpoint in carto, which has enabled us to use vector tiles without any additional work

Problem

Using the undocumented MVT API has worked well enough to date, but when faced with a dataset with lots of very small contiguous polygons, the tiler seems to choke at low zoom levels, and does not include all data that should be rendered in a tile.

Solution

We have contemplated setting up a microservice to serve vector tiles of MapPLUTO, either cutting static tiles, or using some other tile server.

TileStrata is a good candidate for this task. It's an extensible nodejs-based tile server.

@chriswhong
chriswhong / README.md
Last active Jun 21, 2017
Logging Maryland MTA Real-time bus data
View README.md

Logging Maryland MTA Real-time bus data

Technical considerations for logging real-time bus data

Overview

The Maryland MTA has made available real-time bus location data in GTFS-RT format, available at http://mta.maryland.gov/content/developer-resources. Various stakeholders wish to use historical real-time data to assess the timeliness of the bus system. Successful analysis would require a consistent dataset of archived real time locations.

Database vs Static Files

A major consideration is whether to use a database to store the data, or to store it in a well-organized hierarchical tree of static files.

Storing the data in database means a slightly more complex hosting environment. New data are committed to the database, but the data would not be easily accessible without more work to expose certain queries as bulk downloads, or allow users to build their own queries and retreive data on demand (basically, we must build a web api on top of the database in order to make it accessible).

@chriswhong
chriswhong / README.md
Last active Sep 15, 2018
Node.js proxy endpoint to access TMS tiles via XYZ url
View README.md

TMS vs XYZ

Web map raster tile URL templates usally look like this: //{servername}/{somepath}/{z}/{x}/{y}.png

Z is the zoom level (0 being zoomed all the way out so the earth fits on a single 256px x 256px tile, 18 or higher getting you down to street level) X and Y are the tiles coordinates, but there are two different standards for where the origin of the Y coordinate is.

If you look at [http://www.maptiler.org/google-maps-coordinates-tile-bounds-projection/](this site) which shows the various tile coordinates, you'll see that the X's are identical for 'Google' and 'TMS', but the Ys are different. 'Google', aka 'XYZ' aka a few other names that nobody seems to agree on places the Y origin at the north end of the earth, while TMS places it in the south.

Re-serve a TMS tileset as an XYZ tileset using express.js

@chriswhong
chriswhong / choropleth.js
Created Mar 23, 2017
Choropleth for mapboxGL
View choropleth.js
// choropleth.js
// given polygon geojson data and options, returns a GL fill style
// adaptation of Tim Wisniewskis' Leaflet Choropleth
import chroma from 'chroma-js';
const Choropleth = (geojson, opts) => {
opts = opts || {};
@chriswhong
chriswhong / chunk.sh
Last active Mar 10, 2017
Chunk a csv into many files
View chunk.sh
#!/bin/bash
FILENAME=cpdb_spending.csv
HDR=$(head -1 $FILENAME) # Pick up CSV header line to apply to each file
split -l 200000 $FILENAME xyz # Split the file into chunks of 20 lines each
n=1
for f in xyz* # Go through all newly created chunks
do
if [n -gt 1]
then
echo $HDR > Part${n}.csv # Write out header to new file called "Part(n)"
@chriswhong
chriswhong / pluto-carto.md
Last active Jun 15, 2018
Loading PLUTO into Carto
View pluto-carto.md

To load MapPLUTO into carto, the best approach is to upload the five borough shapefiles together, then UNION ALL them together.

  • Upload all five zipped borough shapefiles from Bytes of the Big Apple. Be sure to uncheck 'Allow Carto to guess column types" when uploading, or you'll get column type mismatches
  • UNION ALL the tables together with the following query. We can't just SELECT * because we'd have duplicate cartodb_ids in the result set, and saving as a new table would fail.
SELECT the_geom,the_geom_webmercator,borough,block,lot,cd,ct2010,cb2010,schooldist,council,zipcode,firecomp,policeprct,healthcent,healtharea,sanitboro,sanitdistr,sanitsub,address,zonedist1,zonedist2,zonedist3,zonedist4,overlay1,overlay2,spdist1,spdist2,spdist3,ltdheight,splitzone,bldgclass,landuse,easements,ownertype,ownername,lotarea,bldgarea,comarea,resarea,officearea,retailarea,garagearea,strgearea,factryarea,otherarea,areasource,numbldgs,numfloors,unitsres,unitstotal,lotfront,lotdepth,bldgfront,bldgdepth,ext,pro
You can’t perform that action at this time.