Skip to content

Instantly share code, notes, and snippets.

@ccattuto
ccattuto / timelines5
Created May 15, 2012 14:56
compute timelines of daily hashtag activity
REGISTER lib/com.googlecode.json-simple-json-simple-1.1.jar;
REGISTER lib/joda-time-1.6.jar;
REGISTER eb.jar;
REGISTER piggybank.jar;
DEFINE LOWER org.apache.pig.piggybank.evaluation.string.LOWER;
DEFINE ISOToDay org.apache.pig.piggybank.evaluation.datetime.truncate.ISOToDay;
DEFINE CustomFormatToISO org.apache.pig.piggybank.evaluation.datetime.convert.CustomFormatToISO;
tweets = LOAD '/twitter/*.json.lzo' using com.twitter.elephantbird.pig.load.LzoJsonLoader() AS (json: map[]);