public
Last active

Hive merge small files bug (?) (when using HiveInputFormat and not CombineHiveInputFormat)

  • Download Gist
hive_merge_small_files.java
Java
1 2 3 4 5 6 7 8 9 10 11 12 13
diff --git ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
index a3e40f7..7674af4 100644
--- ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
+++ ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
@@ -381,7 +381,7 @@ public class MapRedTask extends ExecDriver implements Serializable {
.printInfo("Number of reduce tasks is set to 0 since there's no reduce operator");
work.setNumReduceTasks(Integer.valueOf(0));
} else {
- if (numReducersFromWork >= 0) {
+ if (numReducersFromWork > 0) {
console.printInfo("Number of reduce tasks determined at compile time: "
+ work.getNumReduceTasks());
} else if (job.getNumReduceTasks() > 0) {

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.