Skip to content

Instantly share code, notes, and snippets.

@xterat
Last active October 8, 2018 12:08
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save xterat/bf3877d59cd41ffa1738caa0b5c67d92 to your computer and use it in GitHub Desktop.
Save xterat/bf3877d59cd41ffa1738caa0b5c67d92 to your computer and use it in GitHub Desktop.
Compile Hadoop-2.8

Compile Hadoop-2.8 on Ubuntu 16.04

  1. Install some dependencies:

    sudo apt -y install gcc*
    sudo apt -y install cmake
    sudo apt -y install glibc-headers
    sudo apt -y install gcc-c++
    sudo apt -y install zip-devel
    sudo apt -y install openssl-devel
    sudo apt -y install svn
    sudo apt -y install git
    sudo apt -y install ncurses-devel 
    sudo apt -y install lzo-devel
    sudo apt -y install autoconf
    sudo apt -y install libtool
    sudo apt -y install automake
  2. Install protobuf:

    wget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz
    tar zxvf protobuf-2.5.0.tar.gz
    cd protobuf-2.5.0
    ./configure
    make 
    make check
    sudo make install
    sudo ldconfig
  3. Compile Hadoop:

    wget http://apache.spinellicreations.com/hadoop/common/hadoop-2.8.1/hadoop-2.8.1-src.tar.gz
    tar -zxvf hadoop-2.8.1-src.tar.gz
    cd hadoop-2.8.1-src
    mvn package -Dmaven.javadoc.skip=true -Pdist,native -DskipTests -Dtar
  4. The compiled hadoop tarball locates at hadoop-2.8.1-src/hadoop-dist/target/hadoop-2.8.1.tar.gz

@dertin
Copy link

dertin commented Oct 8, 2018

Hi, thanks for the contribution, I made a fork for version 3.1. I'm going to test if it works later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment