Skip to content

Instantly share code, notes, and snippets.

@okram
Created May 25, 2014 02:02
Show Gist options
  • Save okram/2bb14ad1d3009ce4361c to your computer and use it in GitHub Desktop.
Save okram/2bb14ad1d3009ce4361c to your computer and use it in GitHub Desktop.
public DataOutputStream getDataOuputStream(final TaskAttemptContext job) throws IOException, InterruptedException {
final Configuration conf = job.getConfiguration();
boolean isCompressed = getCompressOutput(job);
CompressionCodec codec = null;
String extension = "";
if (isCompressed) {
final Class<? extends CompressionCodec> codecClass = getOutputCompressorClass(job, DefaultCodec.class);
codec = ReflectionUtils.newInstance(codecClass, conf);
extension = codec.getDefaultExtension();
}
final Path file = super.getDefaultWorkFile(job, extension);
final FileSystem fs = file.getFileSystem(conf);
if (!isCompressed) {
return new DataOutputStream(fs.create(file, false));
} else {
return new DataOutputStream(codec.createOutputStream(fs.create(file, false)));
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment