Build LLAP on HDP 2.3

Build LLAP on HDP 2.3.md

Build LLAP on HDP 2.3.x

• Below are sample steps on how to build LLAP using Gopal's tez-autobuild repo on HDP 2.3.x cluster.
• Note that due to HIVE-12446 for now a slightly older version of tez (with tag: 0.8.1-alpha) was built.

Steps

• SSH into HDP cluster
• Install C/C++ compiler

yum install -y gcc gcc-cpp gcc-c++

• Install maven

curl -o /etc/yum.repos.d/epel-apache-maven.repo https://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo
yum -y install apache-maven-3.2*

• Point maven to hwx Nexus repo

mkdir ~/.m2
wget https://gist.githubusercontent.com/abajwa-hw/7cdfc0bf6b2774ae7ccf/raw/16abe28ceb9452d2cf3a69405b1ad3cd74fc04dc/settings.xml -O ~/.m2/settings.xml

• Pull llap branch of Gopal's tez-autobuild repo

cd
git clone -b llap https://github.com/t3rmin4t0r/tez-autobuild.git

• Due to HIVE-12446, we will compile slightly older version of tez (tag: 0.8.1-alpha). The below changes to the Makefile allow us to build using tez 0.8.1-alpha tag of tez branch while skipping tez-ui component
  • vi /root/tez-autobuild/Makefile

#near the top of the file modify as below (you will need to add TEZ_TAG)
TEZ_VERSION=0.8.1-alpha
TEZ_BRANCH=master
TEZ_TAG=release-0.8.1-alpha

#change line 87 to below to checkout the $TEZ_TAG and remove the tez-ui component from pom.xml:
        cd tez/; [ -n "$TEZ_TAG" ] && git checkout tags/$(TEZ_TAG) ; sed -i.bak '/tez-ui/d' pom.xml; . /etc/profile; \

• Compile tez and hive

cd /root/tez-autobuild
make clean dist install

• Once complete, build llap slider

make run

• Run llap daemon on YARN via Slider (you must have Slider installed)

/root/tez-autobuild/llap-slider-*/run.sh

• At this point LLAP app master should appear in YARN RM UI (e.g. http://sandbox.hortonworks.com:8088/cluster/apps)

• Launch hive CLI

/root/tez-autobuild/dist/hive/bin/hive

After build this when I try to use the cli I get this erros:
$ /home/hdfs/tez-autobuild/dist/hive/bin/hive
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hdfs/tez-autobuild/dist/hive/lib/hive-exec-2.1.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdfs/tez-autobuild/dist/hive/lib/hive-jdbc-2.1.0-SNAPSHOT-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdfs/tez-autobuild/dist/hive/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdfs/tez-autobuild/dist/tez/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/share/apache-maven/lib/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.4.0-3485/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in file:/home/hdfs/tez-autobuild/dist/hive/conf/hive-log4j2.properties
hive> select b_no_int from test.spins_orc where betperline>50 limit 3;
Query ID = hdfs_20160218095021_7ef9c899-24aa-4395-b8d4-2c478e863d23
Total jobs = 1
Launching Job 1 out of 1
Failed to get session
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

Any tips about that?

Thanks!