Skip to content

Instantly share code, notes, and snippets.

@shrijeet

shrijeet/hive_merge_small_files.java

Created March 12, 2012 23:09
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save shrijeet/2025303 to your computer and use it in GitHub Desktop.
Save shrijeet/2025303 to your computer and use it in GitHub Desktop.
Download ZIP
Hive merge small files bug (?) (when using HiveInputFormat and not CombineHiveInputFormat)
Raw
hive_merge_small_files.java
diff --git ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
index a3e40f7..7674af4 100644
--- ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
+++ ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
@@ -381,7 +381,7 @@ public class MapRedTask extends ExecDriver implements Serializable {
.printInfo("Number of reduce tasks is set to 0 since there's no reduce operator");
work.setNumReduceTasks(Integer.valueOf(0));
} else {
- if (numReducersFromWork >= 0) {
+ if (numReducersFromWork > 0) {
console.printInfo("Number of reduce tasks determined at compile time: "
+ work.getNumReduceTasks());
} else if (job.getNumReduceTasks() > 0) {
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment