Skip to content

Instantly share code, notes, and snippets.

@bsmedberg
Created August 9, 2012 18:56
Show Gist options
  • Save bsmedberg/3307099 to your computer and use it in GitHub Desktop.
Save bsmedberg/3307099 to your computer and use it in GitHub Desktop.
pig problem, can I do scalar assignments?
REGISTER 'socorro-toolbox-0.1-SNAPSHOT.jar'
REGISTER 'lib/akela-0.4-SNAPSHOT.jar'
SET pig.logfile saveonerecord.log;
SET default_parallel 2;
SET pig.tmpfilecompression true;
SET pig.tmpfilecompression.codec lzo;
uuid = '$uuid';
date = SUBSTRING(uuid, 30, 36);
hbaseid = SUBSTRING(uuid, 0, 1) + date + uuid;
raw = LOAD 'hbase://crash_reports' USING com.mozilla.pig.load.HBaseMultiScanLoader(date, date,
'yyMMdd',
'meta_data:json,processed_data:json',
'true') AS
(k:bytearray, meta_json:chararray, processed_json:chararray);
f = FILTER raw BY k == uuid;
STORE f INTO 'saveonerecord-$uuid' USING PigStorage();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment