Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Big Data tools installations on Linux - Suggested by Isaac Arnault
________ ________ ___ __ ___
|\_____ \|\ __ \|\ \|\ \ |\ \
\|___/ /\ \ \|\ \ \ \/ /|\ \ \
/ / /\ \ __ \ \ ___ \ \ \
/ /_/__\ \ \ \ \ \ \\ \ \ \ \
|\________\ \__\ \__\ \__\\ \__\ \__\
\|_______|\|__|\|__|\|__| \|__|\|__|
Non-Linux Operating Systems
Installations requiring the use of tools such as Docker / Vagrant

Big Data and Data Science - Tools installation (Linux)

Project Status: Concept – Minimal or no implementation has been done yet, or the repository is only intended to be a limited example, demo, or proof-of-concept.

I'd like to provide some effective directions regarding installations you may need to perform to try Big Data major tools.
The following gist is exclusively intended to Linux users.
I performed my installations on Ubuntu 18.04.2 LTS.
To check your OS version, execute $ lsb_release -a in your Terminal.

Installations

  1. MongoDB - done
  2. Neo4j - done
  3. Apache Nifi - done
  4. Apache Zeppelin- done
  5. Talend - upcoming
  6. Apache Cassandra - upcoming ...

Prerequisites

First, make sure Oracle jdk is installed. I recommend java 1.8.0
To uninstall effectively your current jdk, perform this:
$ sudo apt-get remove openjdk*
$ sudo apt-get remove --auto-remove openjdk*
$ sudo apt-get purge openjdk*
$ sudo apt-get purge --auto-remove openjdk*

To install java 1.8.0, Open Terminal Ctrl+Alt+T and run the command:
$ sudo add-apt-repository ppa:webupd8team/java // adds PPA repository
$ sudo apt-get update // updates package list
$ sudo apt-get install openjdk-8-jdk // installs openjdk

java-8.png
$ javac -version // shows your new java version

Author

  • Isaac Arnault - Suggesting installations of major Big Data tools.

MongoDB

$ mkdir mongodb // creates a new directory
$ cd mongodb // opens your directory
$ pwd # gives you the path of your directory
$ curl -O https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-4.0.8.tgz // imports the tarball
$ tar xvf mongodb-linux-x86_64-4.0.8.tgz // opens the tarball
$ mv mongodb-linux-x86_64-4.0.8 mongodb // adds MongoDB bin directory to PATH variable
$ cd mongodb // opens your directory
$ echo $PATH // displays current path
$ export PATH=$PATH:/home/zaki/Desktop/softs/mongodb/mongodb/bin // add MongoDB bin directory to PATH variable
$ mkdir data // creates new directory
$ cd bin // opens your directory
$ ./mongod --dbpath /home/zaki/Desktop/softs/mongodb/mongodb/data & // starts MongoDB

isaac-arnault-mongo-1.png Open a new tab in your Terminal and start using MongoDB:
$ ps -eaf | grep mongo // confirms MongoDB is running $ ./mongo // starts MongoDB

isaac-arnault-mongo-2.png

Neo4J

Before you start installing Neo4j, perform as follows:

$ javac -version // make sure you have Oracle jdk is installed. I recommend java 1.8.0. See Prerequisites for install
$ wget -O - https://debian.neo4j.org/neotechnology.gpg.key | sudo apt-key add - // fetching package & adding key
$ echo 'deb http://debian.neo4j.org/repo stable/' >/tmp/neo4j.list - // echoing to the repository
$ sudo mv /tmp/neo4j.list /etc/apt/sources.list.d - // moving the package
$ sudo apt-get update - // updates all dependencies
Now, you are ready to install Neo4j.
$ mkdir neo4j // creates a new directory
$ cd neo4j // opens your directory
$ pwd # gives you the path of your directory
$ sudo apt-get install neo4j=3.1.4 // stable version I am using - community edition
$ sudo service neo4j restart // restarts the service
You can access Neo4j from the following url: http://localhost:7474/browser/

isaac-arnault-neo4j.png

Nifi

Before you start installing Nifi, perform as follows:

$ javac -version // make sure you have Oracle jdk is installed.
I recommend java 1.8.0. See Prerequisites for install
Now, you are ready to install Nifi.
$ mkdir nifi // creates a new directory
$ cd nifi // opens your directory
$ pwd // gives you the path of your directory
$ wget http://apache.crihan.fr/dist/nifi/1.9.2/nifi-1.9.2-bin.tar.gz // downloads the tarball
$ tar -xvf /home/zaki/Desktop/softs/nifi/nifi-1.9.2-bin.tar.gz // extracts the tarball
$ bin/nifi.sh run // runs the service

isaac-arnault-nifi-1.png

You can access Nifi from the following url: http://localhost:8080/nifi // wait 2 minutes and refresh the page

isaac-arnault-nifi-2.png
Useful commands :
$ bin/nifi.sh status // status of the service - run it in another tab
$ bin/nifi.sh status // stops the service - run it in another tab
$ bin/nifi.sh start // starts the service - run it in another tab

Zeppelin

$ mkdir zeppelin // creates a new directory
$ cd zeppelin // opens your directory
$ pwd // gives you the path of your directory
$ curl -O http://apache.crihan.fr/dist/zeppelin/zeppelin-0.8.1/zeppelin-0.8.1-bin-all.tgz // downloads tgz
$ tar xvf zeppelin-0.8.1-bin-all.tgz // extracts the tarball

isaac-arnault-zeppelin-1.png $ cd zeppelin-0.8.1-bin-all // access to binary files

isaac-arnault-zeppelin-2.png $ bin/zeppelin-daemon.sh start // starting the service

Access zeppelin from http://localhost:8080.

isaac-arnault-zeppelin.png

MIT License
Copyright (c) 2019 Isaac Arnault
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.