Skip to content

Instantly share code, notes, and snippets.

@milindjagre

milindjagre/post16.pig

Last active April 21, 2017 02:43
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save milindjagre/5259bedac6572e5a639b8e7da90dd218 to your computer and use it in GitHub Desktop.
Save milindjagre/5259bedac6572e5a639b8e7da90dd218 to your computer and use it in GitHub Desktop.
Download ZIP
this pig script is used for removing records with NULL values
Raw
post16.pig
--Removing records with NULL values in pig relation
--loading the data in input_data relation
input_data = LOAD '/hdpcd/input/post16/post16.csv' USING PigStorage() AS (line:chararray);
--performing filter operation to remove records with NULL values
filtered_data = FILTER input_data BY line IS NOT NULL;
--storing the final output in HDFS
STORE filtered_data INTO '/hdpcd/output/post16';
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment