Skip to content

Instantly share code, notes, and snippets.

@josiah14
Last active October 29, 2017 20:48
Show Gist options
  • Save josiah14/aa1c72aa337252cb483863490cb85f26 to your computer and use it in GitHub Desktop.
Save josiah14/aa1c72aa337252cb483863490cb85f26 to your computer and use it in GitHub Desktop.
Problems encountered when compiling Hadoop ecosystem libraries.

HDFS

  • Need libprotoc 2.5.0 exactly, not compatible with newer systems (mine is running 2.6.1, which is causing a compile error).
    • FIX: Do the following steps
      wget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz
      tar -xzf protobuf-2.5.0.tar.gz
      cd protobuf-2.5.0
      ./configure
      make
      make check
      make install
      protoc --version
      
      • Issues with Fix
        • It installs protoc to /usr/local/bin, but by default, protoc is installed to /usr/bin on Ubuntu, which naturally takes precedence. I could screw with my system, but instead, this would be better-off done in a Docker container so that I don't ruin other applications that might require the newer protoc.

Spark

  • No Issues encountered on Ubuntu 16.04 for me.

Flume

  • No Issues encountered on Ubuntu 16.04 for me.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment