Skip to content

Instantly share code, notes, and snippets.

View jczaplew's full-sized avatar

John J Czaplewski jczaplew

View GitHub Profile
@jczaplew
jczaplew / error.sql
Created September 17, 2015 22:27
pg_dump raster error
CREATE DATABASE test;
CREATE EXTENSION postgis;
CREATE SCHEMA rasters;
CREATE TABLE rasters.my_rasters (
id serial NOT NULL,
rast raster,
CONSTRAINT enforce_pixel_types_rast CHECK ((public._raster_constraint_pixel_types(rast) = '{16BSI}'::text[]))
);
@jczaplew
jczaplew / nlp2json.py
Last active August 29, 2015 14:24
nlp2json.py
from bs4 import BeautifulSoup
import json
xml = ""
with open("dump/sentences.txt.xml") as input:
xml = input.read()
soup = BeautifulSoup(xml, "xml")
output = {"sentences": [{
@jczaplew
jczaplew / census_places.py
Created April 27, 2015 21:15
US Census incorporated places in PostGIS
import urllib2
import sys, os
from StringIO import StringIO
from zipfile import ZipFile
import psycopg2
import subprocess
# Connect to Postgres
pg_conn = psycopg2.connect(dbname="database", user="you", host="localhost", port=5432)
pg_cur = pg_conn.cursor()
@jczaplew
jczaplew / pgFormatDate.js
Created April 24, 2015 14:11
Javascript Date to Postgres-acceptable format
// Convert Javascript date to Pg YYYY MM DD HH MI SS
function pgFormatDate(date) {
/* Via http://stackoverflow.com/questions/3605214/javascript-add-leading-zeroes-to-date */
function zeroPad(d) {
return ("0" + d).slice(-2)
}
var parsed = new Date(date)
@jczaplew
jczaplew / README.md
Last active August 29, 2015 14:19
Rolling tiles with mapnik + tilestache instead of Tilemill

Rolling tiles sans Tilemill

Perhaps you've had problems programmatically creating tiles with Tilemill, or are concerned about the future stability of project (or you just like playing with geo software).

Install mapnik and TileStache

brew install mapnik --with-gdal --with-postgresql
sudo pip install -U PIL modestmaps simplejson werkzeug
git clone https://github.com/TileStache/TileStache.git TileStache
cd TileStache
@jczaplew
jczaplew / test.sql
Created March 19, 2015 15:02
Postgres array union of result set
CREATE TABLE test (
some_arrays int[]
);
INSERT INTO test (some_arrays) VALUES (array[1,2,3]), (array[3,4,5]), (array[5,6,7]), (array[7,8,9]);
WITH first_query AS (
SELECT 1 AS arbitrary_group_by, unnest(some_arrays) AS my_numbers
FROM test
)
@jczaplew
jczaplew / response.md
Created March 12, 2015 13:49
Packt Response

Hi ---- person sending email ---,

That does sounds interesting - I love --- book title ---. However, it is my understanding that Packt Publishing is a for-profit corporation and will be selling this book, yet wants me to contribute pro bono. Because there are so many nonprofits that are in desperate need of people, I cannot justify donating my spare time to an entity that will turn around and profit from it.

Unless you are willing to pay me (with money, not a book), or donate all profits from the sale of this book to a charity, I am not interested in reviewing this book. Time is too precious.

Best,

--- you ---

@jczaplew
jczaplew / readme.md
Last active August 29, 2015 14:16
Postgres management tools
@jczaplew
jczaplew / README.md
Last active August 29, 2015 14:15
Get Macrostrat columns + GMUS geometry for Michigan Basin
  1. Save get_data.sh
  2. chmod +x get_data.sh
  3. ./get_data.sh
  4. This will produce a folder columns that contains a shapefile of all Macrostrat column geometry, and a folder mibasin that contains a shapefile of the GMUS data for the Michigan Basin
@jczaplew
jczaplew / match.sql
Last active August 29, 2015 14:14
Postgres string matching in WHERE between two fields
CREATE TABLE pattern_test (field1 text, field2 text);
INSERT INTO pattern_test VALUES ('abc', '123 abc 546'), ('def', '123def546'), ('xyz', 'xyz something'), ('abo', 'something about something');
-- We want to find all records where field2 contains field1 as a distinct string (i.e. get records 0 and 2)
-- ~* is similar to ILIKE, but accepts a regular expression
-- '\y' will make sure that field1 is always a distinct object in field2
SELECT * FROM pattern_test WHERE field2 ~* concat('\y',field1,'\y');