Skip to content

Instantly share code, notes, and snippets.

@meyarivan

meyarivan/Hive+S3 Tuning

Last active May 4, 2016 12:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save meyarivan/9cc3f4938c0a28440b3abd9a132addae to your computer and use it in GitHub Desktop.
Save meyarivan/9cc3f4938c0a28440b3abd9a132addae to your computer and use it in GitHub Desktop.
Download ZIP
Hive + S3 tuning
Raw
Hive+S3 Tuning
Tunables:
- mapreduce.input.fileinputformat.list-status.num-threads (hive-site.xml)
- fs.s3a.block.size
JIRAs:
- https://issues.apache.org/jira/browse/HADOOP-12810
- https://issues.apache.org/jira/browse/HADOOP-9565
- https://issues.apache.org/jira/browse/HADOOP-12878
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment