Skip to content

Instantly share code, notes, and snippets.

@bepcyc

bepcyc/spark2_hive_hbase.sh

Created October 19, 2017 16:50
Show Gist options
  • Save bepcyc/fff0711f8ece6fbc9645d869e74fb6bb to your computer and use it in GitHub Desktop.
Save bepcyc/fff0711f8ece6fbc9645d869e74fb6bb to your computer and use it in GitHub Desktop.
Download ZIP
Make Spark 2.x work with Hive mapped HBase tables on Cloudera CDH 5.12
Raw
spark2_hive_hbase.sh
HBASE_JARS=$(ls -1 /opt/cloudera/parcels/CDH/jars/*hbase*.jar|grep -v test|tr '\n' ',')/opt/cloudera/parcels/CDH/lib/hbase/lib/htrace-core.jar
# the part with --files is crucial. Otherwise all queries freeze with no error.
spark2-shell --jars $HBASE_JARS --files /etc/hbase/conf/hbase-site.xml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment