Skip to content

Instantly share code, notes, and snippets.

@sevko sevko/README.md
Last active Aug 29, 2015

Embed
What would you like to do?
Import Quattroshapes into PostgreSQL.

import quattroshapes into PostgreSQL

The import_quattroshapes_pgsql.sh shell-script will import all Quattroshapes shapefiles into a PostgreSQL database. The process has some gotchas and is generally painful to do manually. Before running the script, ensure that you are logged in as a user with permissions to access/write to PostgreSQL. Then:

bash import_quattroshapes_pgsql.sh

Note that the script will create a Postgres table quattroshapes, and download all Quattroshapes shapefiles into quattroshapes/.

#! /bin/bash
# Description:
# A script to download, unzip, and import all Quattroshapes shapefiles into
# Postgres. You must have created a database called `quattroshapes` with the
# `postgis` extension.
#
# Use:
# # must be run by a user with read/write access to Postgres
# ./import_quattroshapes_pgsql.sh
shapefiles=(
qs_adm0
qs_adm1
qs_adm2
qs_localities
qs_localadmin
qs_neighborhoods
)
echo_err(){
echo "$@" 1>&2
}
import_shapefile(){
shp2pgsql -k -D -s 4326 ${@:2} $1/$1 |\
psql -v ON_ERROR_STOP=1 -d quattroshapes > /dev/null 2>&1
}
check_dependencies(){
# Verify that dependencies are installed before proceeding with
# the download/import.
local dependencies=(
shp2pgsql
psql
)
local status=0
for dep in ${dependencies[@]}; do
which $dep > /dev/null
if [ $? -ne 0 ]; then
echo "Dependency $dep unmet."
status=1
fi
done
if [ $status -eq 1 ]; then
echo "Please install missing dependencies, then re-run the script."
exit 1
fi
mkdir quattroshapes
cd quattroshapes
}
download_files(){
# Download and unzip all shapefiles into respective directories.
local baseUrl=http://static.quattroshapes.com
for shapefile in ${shapefiles[@]}; do
echo_err "Downloading $shapefile."
wget --quiet $baseUrl/$shapefile.zip
mkdir $shapefile
mv $shapefile.zip $shapefile
cd $shapefile
echo_err "Unzipping $shapefile."
unzip $shapefile.zip > /dev/null
rm $shapefile.zip
cd ..
done
}
import_files(){
# Import all shapefiles into a PostgreSQL database called `quattroshapes`.
# Must have the Postgis extension.
createdb quattroshapes
echo "create extension postgis;" | psql -d quattroshapes
for shapefile in qs_adm0 qs_adm1 qs_localadmin qs_neighborhoods; do
echo_err "Importing $shapefile."
import_shapefile $shapefile
done
# `qs_localities.dbf` has a bad data error: one attribute column is one
# byte too long, shifting every single one that follows it over by one.
# Fix by incrementing the byte that encodes the length of that column.
# See https://github.com/mbloch/mapshaper/issues/59 .
printf "\xff" |\
dd of=qs_localities/qs_localities.dbf \
bs=1 seek=144 count=1 conv=notrunc
# `qs_adm2` and `qs_localities` have a different encoding.
for shapefile in qs_adm2 qs_localities; do
echo_err "Importing $shapefile."
import_shapefile $shapefile -W LATIN1
done
}
main(){
check_dependencies
download_files
import_files
}
main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.