iandees

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "s3:ListObjects",
            "Resource": "*"
        },
        {

iandees

These are my notes for taking the Microsoft US Building Footprints and splitting them into more manageable chunks based on US Census Tracts.

All of this happened on an m5.xlarge in AWS and used up about ~300GB of EBS over the course of a few hours.

1. Make a filesystem on the EBS volume and mount it:

   sudo mkfs.xfs /dev/nvme1n1
   mount /dev/nvme1n1 /mnt
   

iandees

This CSV uses the census_geo_containment table of CensusReporter's TIGER dumps to show the relationship between zipcodes (represented here as ZCTAs) and counties. It's important to note that ZCTAs are not zipcodes. Further, the ZCTA dataset haven't been updated since 2010, and are out of date. But this is a pretty good place to start and is still useful.

Geographies are represented here using extended Census geoids. If you chop off the first 7 characters of the child_zip values you'll get the zipcode and if you chop off the first 7 characters of the parent_county you'll get the county's FIPS code. For quick debugging, you can pass these geoids into CensusReporter URLs like so: https://censusreporter.org/profiles/<geoid> (e.g. https://censusreporter.org/profiles/05000US72099).

The `percent_cov

iandees

I've been interested in computer vision for a long time, but I haven't had any free time to make any progress until this holiday season. Over Christmas and the New Years I experimented with various methodologies in OpenCV to detect road signs and other objects of interest to OpenStreetMap. After some failed experiments with thresholding and feature detection, the excellent /r/computervision suggested using the dlib C++ module because it has more consistently-good documentation and the pre-built tools are faster.

After a day or two figuring out how to compile the examples, I finally made some progress:

Compiling dlib C++ on a Mac with Homebrew

1. Clone dlib from Github to your local machine:

iandees

iandees

My goal here was the create a routable graph for Chicago.

I used Ubuntu 12.04LTS on an m2.4xlarge just to be sure memory wasn't too much of an issue.

Install

Start a screen session and start downloading an OSM extract (I used this extract of Illinois). In another screen I did the following:

  sudo apt-get update && sudo apt-get upgrade -y

sudo apt-get -y install git make cmake build-essential \

iandees

http://data.pgc.umn.edu/elev/dem/setsm/ArcticDEM/mosaic/v2.0/07_40/07_40_2_2_5m_v2.0.tar.gz
http://data.pgc.umn.edu/elev/dem/setsm/ArcticDEM/mosaic/v2.0/07_41/07_41_1_1_5m_v2.0.tar.gz
http://data.pgc.umn.edu/elev/dem/setsm/ArcticDEM/mosaic/v2.0/07_41/07_41_1_2_5m_v2.0.tar.gz
http://data.pgc.umn.edu/elev/dem/setsm/ArcticDEM/mosaic/v2.0/07_41/07_41_2_1_5m_v2.0.tar.gz
http://data.pgc.umn.edu/elev/dem/setsm/ArcticDEM/mosaic/v2.0/07_41/07_41_2_2_5m_v2.0.tar.gz
http://data.pgc.umn.edu/elev/dem/setsm/ArcticDEM/mosaic/v2.0/07_42/07_42_1_2_5m_v2.0.tar.gz
http://data.pgc.umn.edu/elev/dem/setsm/ArcticDEM/mosaic/v2.0/08_39/08_39_1_2_5m_v2.0.tar
http://data.pgc.umn.edu/elev/dem/setsm/ArcticDEM/mosaic/v2.0/08_39/08_39_2_2_5m_v2.0.tar
http://data.pgc.umn.edu/elev/dem/setsm/ArcticDEM/mosaic/v2.0/08_40/08_40_1_1_5m_v2.0.tar.gz
http://data.pgc.umn.edu/elev/dem/setsm/ArcticDEM/mosaic/v2.0/08_40/08_40_1_2_5m_v2.0.tar.gz

iandees

#!/bin/python

import sys
import urllib2
import xml.etree.cElementTree as ElementTree
import simplejson as json
from datetime import datetime
import StringIO
import gzip
import time

iandees

iandees

import unicodecsv
import argparse
import sys
import simplejson as json
from shapely import wkt
from shapely.geometry import mapping
import shapely.speedups
shapely.speedups.enable()

parser = argparse.ArgumentParser()