Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save albertospelta/12f461bd3db5c949ac78ae454321e0fd to your computer and use it in GitHub Desktop.
Save albertospelta/12f461bd3db5c949ac78ae454321e0fd to your computer and use it in GitHub Desktop.
Install Spark standalone and Anaconda distribution on Ubuntu 18.04 LTS
#!/bin/bash
# Install Spark and Anaconda distribution on Ubuntu 18.04 LTS
sudo apt-get update && sudo apt-get upgrade
# install oracle jdk
sudo add-apt-repository ppa:webupd8team/java
sudo apt update && sudo apt install oracle-java8-set-default
# install spark
wget -c http://www-us.apache.org/dist/spark/spark-2.3.1/spark-2.3.1-bin-hadoop2.7.tgz
tar -xvf spark-2.3.1-bin-hadoop2.7.tgz
sudo mv spark-2.3.1-bin-hadoop2.7 /usr/local/
sudo ln -s /usr/local/spark-2.3.1-bin-hadoop2.7/ /usr/local/spark
echo 'export SPARK_HOME=/usr/local/spark' >> ~/.bashrc
echo 'export PATH=$PATH:$SPARK_HOME/bin' >> ~/.bashrc
source ~/.bashrc
# verify install
# deafult spark context web UI - http://<IP>:4040/
spark-shell
# install anaconda distribution
wget https://repo.anaconda.com/archive/Anaconda3-5.2.0-Linux-x86_64.sh
bash Anaconda3-5.2.0-Linux-x86_64.sh # confirm default location
# verify install
source ~/.bashrc
conda list
# install python-pip
sudo apt install python-pip
sudo pip install --upgrade pip
# install jupiter kernel - apache toree https://github.com/apache/incubator-toree
wget https://dist.apache.org/repos/dist/dev/incubator/toree/0.2.0-incubating-rc6/toree-pip/toree-0.2.0.tar.gz
sudo pip install toree-0.2.0.tar.gz
sudo jupyter toree install --spark_home=$SPARK_HOME --interpreters=Scala,PySpark,SQL
# start jupyter notebook
# default jupyter notebook web UI - http://<IP>:8888/?token=<TOKEN>
jupyter notebook --ip=0.0.0.0 --port=8888 --no-browser
@Srikantsaec15
Copy link

Hi,
jupyter notebook is not opening. while opening this in the browser it is giving error "This site can’t be reached". Kindly tell me how to resolve this and getting it work.
Thanks in advance.
capture

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment