Skip to content

Instantly share code, notes, and snippets.

View darkblue-b's full-sized avatar
👁️
.

Brian M Hamlin darkblue-b

👁️
.
View GitHub Profile
@darkblue-b
darkblue-b / import_master_20121yr_8.py
Last active December 24, 2015 22:59
imports ACS 2012 1yr extension - rev8: first cut at geodata; imports multiple states as previously; needs more QA
#!/usr/bin/python
##===========================================================
## Import ACS 2012 1 year extension
## GPLv3 Oct 2013 -dbb
##
## Useage: python import_master_20121yr_8.py Sequence_Number_and_Table_Number_Lookup.txt work_dir
## assumes an existing database 'census_acs_2012' ; currently writes to public schema
##
## Note: results in ~1480 tables plus 1478 more per state, so
## max_locks_per_transaction must be manually increased
@darkblue-b
darkblue-b / parse_log24.py
Last active December 20, 2015 01:29
next rev to parse build log
import sys,re
# more hacks
# remove the last two lines from the saved output
# psql
# > create database pkgs1;
# > \c pkgs1
# pkgs1=# create table raw_parse (script_name text, action text, pkg_name text);
# pkgs1=# copy raw_parse from '/home/user/out1.csv' with CSV delimiter E'\t';
# pkgs1=# select pkg_name from raw_parse group by pkg_name having every(action <> 'REMOVE') order by pkg_name;