Skip to content

Instantly share code, notes, and snippets.

@diogok
Last active November 10, 2017 04:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save diogok/87c669acd4f3842b4e9c to your computer and use it in GitHub Desktop.
Save diogok/87c669acd4f3842b4e9c to your computer and use it in GitHub Desktop.
Comparing file size for geospatial data for GeoJSON, TopoJSON, KML, CSV with WKT, SQLite/Spatialite and original shapefile. All with and without compression, none simplified.
88K ucs_mma_28012015.attributes.csv.gz
352K ucs_mma_28012015.topo.json.gz
460K ucs_mma_28012015.attributes.csv
2,8M ucs_mma_28012015.topo.json
16M ucs_mma_28012015.spatial.deflate.db
26M ucs_mma_28012015.spatial.db
31M ucs_mma_28012015.wkt.csv.gz
31M ucs_mma_28012015.kml.gz
31M ucs_mma_28012015.geo.json.gz
38M ucs_mma_28012015.wkb.db
41M ucs_mma_28012015.tar.gz
89M ucs_mma_28012015.wkt.csv
90M ucs_mma_28012015.wkt.db
91M ucs_mma_28012015.kml
103M ucs_mma_28012015.geo.json
#!/bin/bash
## Usage: run.sh source.shp
## All files will be created at folder named target
## Comparing file size for geospatial data for GeoJSON, TopoJSON, KML, CSV with WKT, SQLite/Spatialite and shapefile. All with and without compression.
# sudo npm install -g topojson
# sudo aptitude install gdal-bin gdal-contrib libspatialite5
FULLFILE=$1
FILE=$(basename $FULLFILE)
FILENAME="${FILE%.*}"
mkdir -p target
# Shape compressed
tar -cvf target/$FILENAME.tar.gz $(dirname $FULLFILE)
# GeoJSON
ogr2ogr -f GeoJSON target/$FILENAME.geo.json $FULLFILE
cat target/$FILENAME.geo.json | gzip > target/$FILENAME.geo.json.gz
# TopoJSON
topojson -o target/$FILENAME.topo.json target/$FILENAME.geo.json
cat target/$FILENAME.topo.json | gzip > target/$FILENAME.topo.json.gz
# WKT CSV
ogr2ogr -f CSV target/$FILENAME.wkt.csv $FULLFILE -lco GEOMETRY=AS_WKT
cat target/$FILENAME.wkt.csv | gzip > target/$FILENAME.wkt.csv.gz
# Attributes only CSV
ogr2ogr -f CSV target/$FILENAME.attributes.csv $FULLFILE
cat target/$FILENAME.attributes.csv | gzip > target/$FILENAME.attributes.csv.gz
# KML
ogr2ogr -f KML target/$FILENAME.kml $FULLFILE
cat target/$FILENAME.kml | gzip > target/$FILENAME.kml.gz
# SQLite WKT
OGR_SQLITE_SYNCHRONOUS=OFF ogr2ogr -f SQLite target/$FILENAME.wkt.db -dsco SPATIALITE=NO $FULLFILE -lco FORMAT=WKT
# SQLite WKB
OGR_SQLITE_SYNCHRONOUS=OFF ogr2ogr -f SQLite -dsco SPATIALITE=NO target/$FILENAME.wkb.db $FULLFILE
# SpatialLite
OGR_SQLITE_SYNCHRONOUS=OFF ogr2ogr -f SQLite target/$FILENAME.spatial.db -dsco SPATIALITE=yes $FULLFILE -lco SPATIAL_INDEX=YES -skipfailures
# SpatialLite compressed geom
OGR_SQLITE_SYNCHRONOUS=OFF ogr2ogr -f SQLite target/$FILENAME.spatial.deflate.db -dsco SPATIALITE=yes $FULLFILE -lco SPATIAL_INDEX=YES -lco COMPRESS_GEOM=yes -skipfailures
ls -sh1Sr target
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment