Skip to content

Instantly share code, notes, and snippets.

View stevenvandervalk's full-sized avatar

Steve Vandervalk stevenvandervalk

View GitHub Profile

Finessing Excel's stupid line endings

I am sheepish to admit a certain type of routine Microsoft Excel use.

Current example: I am marking for STAT 545. I use R to create a comma delimited marking sheet, by joining the official class list and peer reviews. The sheet contains variables, initially set to NA, where the TAs and I enter official marks and optional comments.

This is where Excel comes in. I like its visual organization of this comma delimited file much more than, say, using a plain text editor. I use the ability to hide columns, resize columns, wrap text, and (gasp!) even fill rows with grey to indicate I am done.

I keep saving the file as comma delimited and I put up with Excel's incessant freak out about "losing features". This is not a one time thing. I need to save and commit this file many times before it is considered done.

Sometimes, for whatever reason, QGIS hates your CSV of point data and mangles it so it can't be converted to points. Sometimes, it gets worse and command line GDAL also messes things up. In these cases you can fall back on a .vrt
A .vrt is a really simple file that tells GDAL *exactly* where your lat/long columns are and how to render them (e.g. as point geometry). You can also use this for non-point geometry if you want to get fancy and have columns of polygon data (or a non-csv file type that holds them in another fashion).
A .vrt is basically formatted as below:
<OGRVRTDataSource>
<OGRVRTLayer name="[Filename]">
<SrcDataSource>[Filename].csv</SrcDataSource>
<GeometryType>wkbPoint</GeometryType>
@stevenvandervalk
stevenvandervalk / geojson-conversion.sh
Created May 30, 2017 06:29 — forked from benbalter/geojson-conversion.sh
Bulk convert shapefiles to geojson using ogr2ogr
# Bulk convert shapefiles to geojson using ogr2ogr
# For more information, see http://ben.balter.com/2013/06/26/how-to-convert-shapefiles-to-geojson-for-use-on-github/
# Note: Assumes you're in a folder with one or more zip files containing shape files
# and Outputs as geojson with the crs:84 SRS (for use on GitHub or elsewhere)
#geojson conversion
function shp2geojson() {
ogr2ogr -f GeoJSON -t_srs crs:84 "$1.geojson" "$1.shp"
}
@stevenvandervalk
stevenvandervalk / setup-certbot-script.sh
Created February 8, 2017 11:38
A bash script to install certbot and add crontab to renew the cert
#!/bin/bash
#
# This sets up Let's Encrypt SSL certificates and automatic renewal
# using certbot: https://certbot.eff.org
#
# - Run this script as root.
# - A webserver must be up and running.
#
# Certificate files are placed into subdirectories under
# /etc/letsencrypt/live/*.