Skip to content

Instantly share code, notes, and snippets.

@veltman
Last active October 8, 2016 23:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save veltman/4f3a55a779175ded6760774040997376 to your computer and use it in GitHub Desktop.
Save veltman/4f3a55a779175ded6760774040997376 to your computer and use it in GitHub Desktop.
Geosupport w/ JS

Using NYC's Geosupport library with Node

Following on Chris Whong's excellent writeup of how to make calls directly to NYC's geosupport client, here's a basic way to call it as a child process in Node.js.

  1. Follow Chris's instructions on installing the desktop edition on Linux.
  2. Instead of his test.c, use this geocode.c, which is modified to treat the first command line argument as the contents of Work Area 1, and the (optional) second argument as the contents of Work Area 2. It prints the two resulting work areas to stdout, separated by a newline.
  3. Compile geocode.c with the same instructions.
  4. Use something like this geocode.js to call it repeatedly as a child process, constructing the proper work area string as needed by following the user guide.

This geocode.js constructs the working area for geocoding an address into a lat/lng, census tract, and census block based on address components (Geosupport function 1). But by changing the input string you send (line 20), and by how you parse the output (lines 27 - 46), you could do any other function.

Spawning a new child process over and over introduces a lot of unnecessary overhead, but in a Docker container on my machine this still does ~250 addresses per second. There at least two alternatives that would be much faster:

  • Use node-ffi to link the library directly. I'll give this a shot at some point.
  • Do the work natively in C instead. I bet this would be one or two orders of magnitude faster. But manipulating strings in C makes me want to tear my hair out.

Notes

  • This script also reprojects the results, which are in NY Long Island State Plane, into latitude/longitude
  • If the first attempt returns support code EE, which suggests a corrected street name (e.g. if you supplied BRADWAY it might suggest you try BROADWAY instead), this script tries again with the suggested name.
#include "/version-16c_16.3/include/foruser/geo.h"
#include <stdio.h>
#include <string.h>
int main(int argc, char *argv[])
{
char wa1[1200]="";
char wa2[1200]="";
strncpy(wa1, argv[1], strlen(argv[1]));
if (argc > 2) {
strncpy(wa2, argv[2], strlen(argv[2]));
}
char *wa1ptr;
char *wa2ptr;
wa1ptr=wa1;
wa2ptr=wa2;
geo(wa1ptr, wa2ptr);
printf("%s",wa1ptr);
printf("\n");
printf("%s",wa2ptr);
return 0;
}
var fs = require("fs"),
JSONStream = require("JSONStream"),
child_process = require("child_process"),
proj4 = require("proj4"),
through2 = require("through2");
var reproject = proj4('PROJCS["NAD_1983_StatePlane_New_York_Long_Island_FIPS_3104_Feet",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Lambert_Conformal_Conic"],PARAMETER["False_Easting",984250.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-74.0],PARAMETER["Standard_Parallel_1",40.66666666666666],PARAMETER["Standard_Parallel_2",41.03333333333333],PARAMETER["Latitude_Of_Origin",40.16666666666666],UNIT["Foot_US",0.3048006096012192]]').inverse;
fs.createReadStream("parsed/QUEENS.json", "utf8")
.pipe(JSONStream.parse("*"))
.pipe(through2.obj(function(address, enc, cb){
geocode(address, cb);
}))
.pipe(JSONStream.stringify())
.pipe(fs.createWriteStream("geocoded/QUEENS.json"));
function geocode(address, cb) {
// Construct COW string for WA1
var input = ("1 " + rightpad(address.HouseNumber, 16) + rightpad("", 38) + address.BoroughCode + rightpad("", 10) + rightpad(address.AltStreetName || address.StreetName, 32) + rightpad("", 113) + "C" + rightpad(address.ZipCode, 5)).toUpperCase();
// Pass working area str as command line arg
child_process.exec("./geocode \"" + input + "\"", function(err, stdout){
if (err) throw err;
var i = stdout.indexOf("\n");
var returnCode = stdout.substring(716, 718),
x,
y;
// Success
if (returnCode === "00") {
x = +(stdout.substring(i + 126, i + 126 + 7));
y = +(stdout.substring(i + 133, i + 133 + 7));
address.tract = +(stdout.substring(i + 224, i + 224 + 6));
address.block = +(stdout.substring(i + 230, i + 230 + 4));
address.lngLat = reproject([x, y]);
// Support code EE, suggested alternate street name
} else if (!address.AltStreetName && returnCode === "EE") {
address.AltStreetName = stdout.substring(880, 880 + 32);
return geocode(address, cb);
}
cb(null, address);
});
}
function rightpad(str, width) {
str = str.toString();
while (str.length < width) {
str += " ";
}
if (str.length > width) {
return str.slice(0, width);
}
return str;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment