Skip to content

Instantly share code, notes, and snippets.

@stefanw
Created August 14, 2010 19:21
Show Gist options
  • Save stefanw/524625 to your computer and use it in GitHub Desktop.
Save stefanw/524625 to your computer and use it in GitHub Desktop.
// create job, set properties etc.
// instead of job.waitForCompletion(true), just do:
job.submit();
// set time out:
long maxDuration = 1000 * 120; // 120 seconds
String lastReport = null;
long start = System.currentTimeMillis();
long current = start;
while (current - start < maxDuration){
if (job.isComplete()) {
break;
}
Thread.sleep(1000);
current = System.currentTimeMillis();
// the reporting is from Hadoop's JobClient
String report = " map " + StringUtils.formatPercent(job.mapProgress(), 0)+
" reduce " + StringUtils.formatPercent(job.reduceProgress(), 0);
if (!report.equals(lastReport)) {
System.out.println(report + " time "+ (current - start) / 1000);
lastReport = report;
}
}
if (!job.isComplete()){
job.killJob();
System.err.println("Job took too long, it was killed!");
return 1;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment