Skip to content

Instantly share code, notes, and snippets.

@AndyCross

AndyCross/Job.cs

Created November 27, 2012 15:14
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save AndyCross/4154736 to your computer and use it in GitHub Desktop.
Save AndyCross/4154736 to your computer and use it in GitHub Desktop.
Download ZIP
Hadoop Compression
Raw
inputflag.txt
-D "io.compression.codecs=org.apache.hadoop.io.compress.GzipCodec"
Raw
Job.cs
//public override HadoopJobConfiguration Configure(ExecutorContext context)
HadoopJobConfiguration config = new HadoopJobConfiguration();
config.InputPath = "/input/data.log.gz";
config.OutputFolder = "/output/output" + DateTime.Now.ToString("yyyyMMddhhmmss");
config.AdditionalGenericArguments.Add("-D \"io.compression.codecs=org.apache.hadoop.io.compress.GzipCodec\"");
return config;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment