Skip to content

Instantly share code, notes, and snippets.

Ewan Oglethorpe eoglethorpe

  • In the general biosphere
Block or report user

Report or block eoglethorpe

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View Classify keywords by sector.md

Creating a system for identifying keywords in textual humanitarian data

This is a brief crash course for creating a system to categorize key words in sets of data with known classifications of segments of text. Although it is written in a humanitarian context, it can be flexibly used elsewhere.

Overview: What's in the data?

A sample dataset could include the following format where it is normalized:

View Classify keywords by sector.md

Creating a system for identifying keywords in textual humanitarian data

This is a brief crash course for creating a system to categorize key words in sets of data with known classifications of segments of text. Although it is written in a humanitarian context, it can be flexibly used elsewhere.

Overview: What's in the data?

A sample dataset could include the following format where it is normalized:

View Classify keywords by sector
Creating a system for identifying keywords in textual humanitarian data
===================================================
This is a brief crash course for creating a system to categorize key words in sets of data with known classifications of segments of text. Although it is written in a humanitarian context, it can be flexibly used elsewhere.
Overview: What's in the data?
-------------------
A sample dataset could include the following format where it is normalized:
View 538_results_mar_10.json
[{
"date": "Feb. 1",
"state": "Iowa",
"del_won_hill": "23",
"num_del_tot": "44",
"del_won_bern": "21"
}, {
"date": "Feb. 9",
"state": "New Hampshire",
"del_won_hill": "9",
View parse_538_results.py
__author__ = 'eoglethorpe'
"""take election info from
http://projects.fivethirtyeight.com/election-2016/delegate-targets/democrats/
and convert it to JSON
"""
import json
import requests
View winter.sql
--add winterisation data
drop table if exists winterisation_db;
CREATE TABLE winterisation_db
("dist_code" varchar(11), "vdc_code" varchar(17), "district" varchar(15), "vdc" varchar(32), "rep_ag" varchar(26), "imp_ag" varchar(21), "target_hh" int, "type" varchar(61), "full" varchar(3), "comment" varchar(362), "act_state" varchar(9), "edit_dt" timestamp, "comments" varchar(119))
;
INSERT INTO winterisation_db
("dist_code", "vdc_code", "district", "vdc", "rep_ag", "imp_ag", "target_hh", "type", "full", "comment", "act_state", "edit_dt", "comments")
View clean.sql
--planned and neither date > delete (74)
delete from distributions
where act_desc in ('Tents', 'Tarpaulin','tarpaulin','Cash for Shelter (15,000 NPR)', 'CGI Bundle (72ft/bundle)')
and
act_status like 'Planned (funded)' and (start_dt is null and comp_dt is null)
--no status > delete (52)
delete from distributions
where act_desc in ('Tents', 'Tarpaulin','tarpaulin','Cash for Shelter (15,000 NPR)', 'CGI Bundle (72ft/bundle)')
and
View gist:558693203df79607554a
# Root logger option
log4j.rootLogger=INFO, syslog, console
#console
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console=org.apache.log4j.ConsoleAppender
#papertrail
log4j.appender.syslog=org.apache.log4j.net.SyslogAppender
log4j.appender.syslog.Facility=LOCAL7
View gist:8d8391fb1c27c01551b2
Properties props = new Properties();
Logger log4jLogger = Logger.getLogger(S3Sink.class);
props.load(getClass().getResourceAsStream("/log4j.properties"));
PropertyConfigurator.configure(props);
log4jLogger.info("log content")
You can’t perform that action at this time.