abajwa-hw

Phoenix/Spark demo

• These demo is part of a webinar on 'HBase for Mission Critical Applications'
• The webinar recording /slides are available at http://hortonworks.com/partners/learn/#hbase2
• The Ambari service for OpenTSDB is available at https://github.com/abajwa-hw/opentsdb-service
• The full list of available workshops is available at http://hortonworks.com/partners/learn

Option 1: prebuilt VM

There is a prebuilt Centos 6.5 VM with the below components installed:

• HDP 2.3.0.0-1754
• Spark 1.3.1

abajwa-hw

Hbase indexing to solr in HDP 2.3

• Background:

The HBase Indexer provides the ability to stream events from HBase to Solr for near real time searching. The HBase indexer is included with HDPSearch as an additional service. The indexer works by acting as an HBase replication sink. As updates are written to HBase, the events are asynchronously replicated to the HBase Indexer processes, which in turn creates Solr documents and pushes them to Solr.

• References:
  • http://doc.lucidworks.com/hdpsearch23/JobJarGuide.html#_hbase_indexer
  • https://github.com/NGDATA/hbase-indexer/wiki/Tutorial

abajwa-hw

Index documents using HDPSearch

Lab Overview

In this lab, we will learn to:

• Configure Solr to store indexes in HDFS
• Create a solr cluster of 2 solr instances running on port 8983 and 8984
• Index documents in HDFS using the Hadoop connectors
• Use Solr to search documents

abajwa-hw

LAB

This lab is part of a 'Sql on Hadoop' webinar. The recording and slides can be found here

Purpose

How/when to use Hive vs Phoenix vs SparkSQL

abajwa-hw

HDF workshop

Setup already done

• Download and import latest HDP 2.3 sandbox VM from http://hortonworks.com/sandbox and import into VMware Fusion
• Deployed and install VNC ambari service (to be able to 'remote desktop' in and use eclipse)

  sudo git clone https://github.com/hortonworks-gallery/ambari-vnc-service.git /var/lib/ambari-server/resources/stacks/HDP/2.3/services/VNCSERVER
  
  

abajwa-hw

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                          http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <localRepository/>
  <interactiveMode/>
  <usePluginRegistry/>
  <offline/>
  <pluginGroups/>
  <servers/>

abajwa-hw

Build LLAP on HDP 2.3.x

• Below are sample steps on how to build LLAP using Gopal's tez-autobuild repo on HDP 2.3.x cluster.
• Note that due to HIVE-12446 for now a slightly older version of tez (with tag: 0.8.1-alpha) was built.

Steps

• SSH into HDP cluster
• Install C/C++ compiler

yum install -y gcc gcc-cpp gcc-c++

abajwa-hw

EDW optimization and Single view lab

EDW optimization

• Goal: demonstrate how you can bulk import data from EDW/RDBMS into Hive and then incrementally keep the Hive tables updated periodically

Pre-requisites

• 1. Download contoso data set and MSSQL schema from here into /tmp on sandbox

abajwa-hw

#find jar containing class org.apache.hive.jdbc.HiveStatement
find /  -iname '*.jar' | xargs -i bash -c "jar -tvf {} | tr / . | grep  org.apache.hive.jdbc.HiveStatement && echo {}"

abajwa-hw

Add HDB (HAWQ) to HDP 2.4.2 with Zeppelin

Goals:

• Install a 4 node cluster running HDP 2.4.2 using Ambari 2.2.2.0 (including Zeppelin and HDB) using Ambari bootstrap via blueprints or Ambari install wizard
• Configure HAWQ for Zeppelin
• Configure Zeppelin for HAWQ
• Run HAWQ queries via Zeppelin

Notes: