Skip to content

Instantly share code, notes, and snippets.

@pat-humphreys
Last active August 10, 2016 15:19
Show Gist options
  • Save pat-humphreys/33667566f23c53baa57b8652a1ae172c to your computer and use it in GitHub Desktop.
Save pat-humphreys/33667566f23c53baa57b8652a1ae172c to your computer and use it in GitHub Desktop.
Quick setup for Spark+Scala+Sbt on Ubuntu. Ensure you have the Oracle Java8 installed
#Install Spark+Scala+Sbt on Ubuntu
scalaVersion=2.11.8
sparkVersion=2.0.0
sbtVersion=0.13.12
hadoopVersion=2.7
sudo apt-get remove scala-library scala
sudo wget www.scala-lang.org/files/archive/scala-$scalaVersion.deb
sudo dpkg -i $scalaVersion.deb
sudo apt-get update
sudo apt-get install scala
wget https://bintray.com/artifact/download/sbt/debian/sbt-$sbtVersion.deb
sudo dpkg -i sbt-$sbtVersion.deb
sudo apt-get update
sudo apt-get install sbt
wget http://d3kbcqa49mib13.cloudfront.net/spark-$sparkVersion-bin-hadoop$hadoopVersion.tgz
tar xvf spark-$sparkVersion-bin-hadoop$hadoopVersion.tgz
sudo mv spark-$sparkVersion-bin-hadoop$hadoopVersion /opt/
echo export PATH="/opt/spark-$sparkVersion-bin-hadoop$hadoopVersion/bin/:$PATH" > ~/.bashrc
exit
#Test Sbt,Scala,Spark,
sbt
scala
spark-shell
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment