Skip to content

Instantly share code, notes, and snippets.

@rlaverde

rlaverde/install-spark-ts

Last active May 24, 2018 18:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rlaverde/ef00140ff01f4808fcb89a2c6da542a6 to your computer and use it in GitHub Desktop.
Save rlaverde/ef00140ff01f4808fcb89a2c6da542a6 to your computer and use it in GitHub Desktop.
Download ZIP
Install spark-ts in EMR (Amazon Linux)
Raw
install-spark-ts
# This is not a script, just instructions to install it
sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo
sudo sed -i s/\$releasever/6/g /etc/yum.repos.d/epel-apache-maven.repo
sudo yum install -y apache-maven
sudo update-alternatives --config java #pick java 1.8
sudo update-alternatives --config javac #pick java 1.8
git clone https://github.com/sryza/spark-timeseries.git
cd spark-timeseries/
mvn package
cp target/sparkts-0.4.0-SNAPSHOT-jar-with-dependencies.jar python/sparkts/
cd python
sudo python setup.py install
pyspark --jars spark-timeseries/target/sparkts-0.4.0-SNAPSHOT-jar-with-dependencies.jar
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment