Skip to content

Instantly share code, notes, and snippets.

@aaronbinns
Last active September 3, 2019 21:39
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save aaronbinns/2c2173fc57818ffc106e to your computer and use it in GitHub Desktop.
Save aaronbinns/2c2173fc57818ffc106e to your computer and use it in GitHub Desktop.
Build GeoJSON shapes for all US cities.
#!/bin/bash
# Extract the US zip-code shapes and convert to a single US-wide
# GeoJSON file
echo -n "Extracting US-wide GeoJSON file (takes ~10 mins)..."
mkdir tl_2014_us_zcta510
unzip -qq tl_2014_us_zcta510.zip -d tl_2014_us_zcta510
ogr2ogr -f GeoJSON -t_srs crs:84 tl_2014_us_zcta510.geojson tl_2014_us_zcta510/tl_2014_us_zcta510.shp
rm -r tl_2014_us_zcta510
echo "done."
# Join the {zip->city} mapping to create per-city files with the
# GeoJSON sub-documents for each zip-code in the city.
echo -n "Creating per-city partial GeoJSON files (takes ~15 mins)..."
mkdir per_city_partials
join -1 2 -2 1 <(sort -k 2,2 us_city_zips.tsv) <(cat tl_2014_us_zcta510.geojson | gawk '{ match( $0 , /ZCTA5CE10\": \"([0-9]+)\"/, arr ); if ( arr[1] != "" ) print arr[1], $0 }' | sort) | sed 's/,$//g' | gawk '{ city="per_city_partials/" $2; $1=$2=""; print $0 >> city; close(city) }'
echo "done"
# For each city, combine the zip-code GeoJSON sub-documents into a
# single city-level GeoJSON document. Also annotate the resulting
# GeoJSON document with properties for the city ID and city name.
echo "Combining partial GeoJSON files into complete per-city GeoJSON files with annotations (takes ~2 hours)..."
mkdir per_city_geojson
for i in per_city_partials/*
do
id="${i#*/}"
state="${id%%_*}"
city="${id#*_}"
city="${city//_/ }"
cat <( echo '{ "type":"FeatureCollection", "features":[' ; gawk '{ if ( NR > 1 ) print "," ; print $0 }' ${i} ; echo "]}" ) | mapshaper - -dissolve2 -simplify 10% -each "\$.properties = { id: \"${id}\", state: \"${state}\", city: \"${city}\" }" -o per_city_geojson/${id}.json
done
echo "Done generating complete per-city GeoJSON files."
# Catenate the per-city GeoJSON files into a single US-wide GeoJSON
# file.
echo -n "Combine per-city GeoJSON files into single US-wide GeoJSON file..."
echo '{"type":"FeatureCollection","features":[' > us_cities.geojson
for i in per_city_geojson/*
do
cat ${i}
echo
done | sed 's/^[{]"type"[:]"FeatureCollection"[,]"features"[:][[]//' | sed 's/\]\}$//' | awk '{ if ( NR > 1 ) printf( "%s", ", ") ; print $0 }' >> us_cities.geojson
echo ']}' >> us_cities.geojson
echo "done."
# Last, but not least, convert GeoJSON into a Shapefile .zip
echo "Convert US-wide GeoJSON file to shapefile .zip..."
mkdir shapefile
cd shapefile
ogr2ogr -F "ESRI Shapefile" us_cities.shp ../us_cities.geojson OGRGeoJSON
zip -qq ../us_cities.zip us_cities.dbf us_cities.prj us_cities.shp us_cities.shx
cd ..
rm -rf shapefile
echo "done."
echo ; echo "All done!"
@joshuarabe
Copy link

Have you actually done this? Do you have geojson data for all us cities?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment