Skip to content

Instantly share code, notes, and snippets.

@tonycox
Last active February 12, 2021 10:18
Show Gist options
  • Star 7 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save tonycox/322e8ffa584123f1b498c705cf5d972f to your computer and use it in GitHub Desktop.
Save tonycox/322e8ffa584123f1b498c705cf5d972f to your computer and use it in GitHub Desktop.
how i install bigtop 1.1.0 on ubuntu 16.04
sudo apt-get update && apt-get upgrade
# install java (homedir - /usr/lib/jvm/java-8-oracle/)
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer
# or use this to set as default -> sudo apt install oracle-java8-set-default
# install maven (homedir - /usr/share/maven)
sudo apt-get install maven
# set env
echo "export JAVA_HOME=/usr/lib/jvm/java-8-oracle/" >> /etc/bash.bashrc
echo "export M2_HOME=/usr/share/maven" >> /etc/bash.bashrc
echo "export PATH=$M2_HOME/bin:$JAVA_HOME/bin:$PATH" >> /etc/bash.bashrc
source /etc/bash.bashrc
mvn -v
# Install the Apache Bigtop GPG key
wget -O- http://archive.apache.org/dist/bigtop/bigtop-1.1.0/repos/GPG-KEY-bigtop | sudo apt-key add -
# Make sure to grab the repo file:
sudo wget -O /etc/apt/sources.list.d/bigtop-1.1.0.list http://archive.apache.org/dist/bigtop/bigtop-1.1.0/repos/trusty/bigtop.list
# Update the apt cache
sudo apt-get update
# Install bigtop-utils
sudo apt-get install bigtop-utils
# Install the full Hadoop stack (or parts of it)
sudo apt-get install hadoop\* flume-* oozie\* hive\*
## Running
# Format the namenode
sudo /etc/init.d/hadoop-hdfs-namenode init
# Start the necessary Hadoop services
for i in hadoop-hdfs-namenode hadoop-hdfs-datanode ; do sudo service $i start ; done
sudo service hadoop-yarn-resourcemanager start
sudo service hadoop-yarn-nodemanager start
# Check hadoop fs
sudo hdfs dfs -ls /
# Run hive
hadoop fs -mkdir /tmp
hadoop fs -mkdir /user/hive/warehouse
hadoop -chmod g+x /tmp
hadoop -chmod g+x /user/hive/warehouse
# if dirs have not been created on local fs
# sudo mkdir /var/run/hive
# sudo mkdir /var/lock/subsys
sudo /etc/init.d/hive-server start
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment