- Below are sample steps on how to build LLAP using Gopal's tez-autobuild repo on HDP 2.3.x cluster.
- Note that due to HIVE-12446 for now a slightly older version of tez (with tag: 0.8.1-alpha) was built.
- SSH into HDP cluster
- Install C/C++ compiler
yum install -y gcc gcc-cpp gcc-c++
- Install maven
curl -o /etc/yum.repos.d/epel-apache-maven.repo https://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo
yum -y install apache-maven-3.2*
- Point maven to hwx Nexus repo
mkdir ~/.m2
wget https://gist.githubusercontent.com/abajwa-hw/7cdfc0bf6b2774ae7ccf/raw/16abe28ceb9452d2cf3a69405b1ad3cd74fc04dc/settings.xml -O ~/.m2/settings.xml
- Pull llap branch of Gopal's tez-autobuild repo
cd
git clone -b llap https://github.com/t3rmin4t0r/tez-autobuild.git
- Due to HIVE-12446, we will compile slightly older version of tez (tag: 0.8.1-alpha). The below changes to the Makefile allow us to build using tez 0.8.1-alpha tag of tez branch while skipping tez-ui component
vi /root/tez-autobuild/Makefile
#near the top of the file modify as below (you will need to add TEZ_TAG)
TEZ_VERSION=0.8.1-alpha
TEZ_BRANCH=master
TEZ_TAG=release-0.8.1-alpha
#change line 87 to below to checkout the $TEZ_TAG and remove the tez-ui component from pom.xml:
cd tez/; [ -n "$TEZ_TAG" ] && git checkout tags/$(TEZ_TAG) ; sed -i.bak '/tez-ui/d' pom.xml; . /etc/profile; \
- Compile tez and hive
cd /root/tez-autobuild
make clean dist install
- Once complete, build llap slider
make run
- Run llap daemon on YARN via Slider (you must have Slider installed)
/root/tez-autobuild/llap-slider-*/run.sh
-
At this point LLAP app master should appear in YARN RM UI (e.g. http://sandbox.hortonworks.com:8088/cluster/apps)
-
Launch hive CLI
/root/tez-autobuild/dist/hive/bin/hive
After build this when I try to use the cli I get this erros:
$ /home/hdfs/tez-autobuild/dist/hive/bin/hive
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hdfs/tez-autobuild/dist/hive/lib/hive-exec-2.1.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdfs/tez-autobuild/dist/hive/lib/hive-jdbc-2.1.0-SNAPSHOT-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdfs/tez-autobuild/dist/hive/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdfs/tez-autobuild/dist/tez/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/share/apache-maven/lib/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.4.0-3485/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Logging initialized using configuration in file:/home/hdfs/tez-autobuild/dist/hive/conf/hive-log4j2.properties
hive> select b_no_int from test.spins_orc where betperline>50 limit 3;
Query ID = hdfs_20160218095021_7ef9c899-24aa-4395-b8d4-2c478e863d23
Total jobs = 1
Launching Job 1 out of 1
Failed to get session
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
Any tips about that?
Thanks!