Skip to content

Instantly share code, notes, and snippets.

@domderen
Last active April 17, 2022 06:28
Show Gist options
  • Star 13 You must be signed in to star a gist
  • Fork 29 You must be signed in to fork a gist
  • Save domderen/27caeb7eb02118d71279 to your computer and use it in GitHub Desktop.
Save domderen/27caeb7eb02118d71279 to your computer and use it in GitHub Desktop.
Installation of apache spark on ubuntu machine.
#!/bin/sh
# installation of Oracle Java JDK.
sudo apt-get -y update
sudo apt-get -y install python-software-properties
sudo add-apt-repository -y ppa:webupd8team/java
sudo apt-get -y update
sudo apt-get -y install oracle-java7-installer
# Installation of commonly used python scipy tools
sudo apt-get -y install python-numpy python-scipy python-matplotlib ipython ipython-notebook python-pandas python-sympy python-nose
# Installation of scala
wget http://www.scala-lang.org/files/archive/scala-2.11.1.deb
sudo dpkg -i scala-2.11.1.deb
sudo apt-get -y update
sudo apt-get -y install scala
# Installation of sbt
wget http://scalasbt.artifactoryonline.com/scalasbt/sbt-native-packages/org/scala-sbt/sbt//0.12.3/sbt.deb
sudo dpkg -i sbt.deb
sudo apt-get -y update
sudo apt-get -y install sbt
# Downloading spark
wget http://d3kbcqa49mib13.cloudfront.net/spark-1.0.0.tgz
tar -zxf spark-1.0.0.tgz
cd spark-1.0.0
# Building spark
./sbt/sbt assembly
# Clean-up
rm scala-2.11.1.deb
rm sbt.deb
rm spark-1.0.0.tgz
rm install.sh
@sypher123
Copy link

installation of sbt , getting 404 error

@pltrdy
Copy link

pltrdy commented Feb 23, 2016

@sypher123: FYI I forked using spark-1.6 (and adding set -e to stop execution on error)

Thx for this script tho @domderen

@aidanasr
Copy link

aidanasr commented Aug 1, 2017

Hi, everyone
I want to know how to download apache spark in eclipse as open source code for editing its scheduling algorithm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment