infectious

Sample of 100 rows extraced when the select condition is absent, but are excluded when condition is applied.
(taken from the difference between tables loaded with and without the condition)
Yet all segments have active=1, obsolete=0...
ie these are the ones being excluded, but shouldn't be.

Hour: 2013_05_24_00

period user_id segment_id
380376  4712986594389400462	444291
380376	7074347584413598423	380725

infectious

## Question: I want to use this same ETL for a few different dimensions. How can I use a parameter (cols) to employ the 'columns' option in the load block (line 28/29 below)?

parameter :today do
 8.day.ago.to_date
end

parameter :dimension do
 "placement"
end

infectious

register /home/hadoop/.versions/pig-0.11.1.1/lib/pig/piggybank-0.11.1.1-amzn.jar;
define ISOToUnix org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix();

data = load 'data_file.txt' using PigStorage('\t') as (user_id:int, datetime:chararray);

data = limit data 10000;

pdata = foreach generate
  user_id,

infectious

<script>(function()
	{
	var flashAd='<OBJECT id="0" data="http://cdn.adnxs.com/p/22/8d/1e/99/228d1e996f864b393b49976c5db81a12.swf" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" WIDTH="300" HEIGHT="250" flashvars="clickTAG=http%3A%2F%2Fnym1.ib.adnxs.com%2Fclick%3FAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA___________NmJRSAAAAAAEAAAAAAAAAAAAAAAAAAABiuI4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgUAAAAAAAEA9QsTUwAAAAA.%2Fclickenc%3Dhttp%253A%252F%252Fwww.johnlewis.com%253Ftmad%253Dc%2526tmcampid%253D77%2526s_dsuid%253D%2526s_dscid%253DDSINF_P9_OUTERO_300x250_C_%2524%257BCP_ID%257D&clickTAG1=http%3A%2F%2Fnym1.ib.adnxs.com%2Fclick%3FAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA___________NmJRSAAAAAAEAAAAAAAAAAAAAAAAAAABiuI4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgUAAAAAAAEA9QsTUwAAAAA.%2Fclickenc%3Dhttp%253A%252F%252Fwww.johnlewis.com%252Fmen%252Fmen%255C%2527s-jackets-coats%252Fc600001512%253Ftmad%253Dc%2526tmcampid%253D77%2526s_dsuid%253D%2526s_dscid%253DDSINF_P9_OUTERO_3