Skip to content

Instantly share code, notes, and snippets.

@yujikiriki

yujikiriki/Dockerfile

Created September 17, 2015 13:14
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save yujikiriki/0825037d72a5293e2008 to your computer and use it in GitHub Desktop.
Save yujikiriki/0825037d72a5293e2008 to your computer and use it in GitHub Desktop.
Download ZIP
Mi primer aporte a SparkMDE. Con mucho cariño.
Raw
Dockerfile
FROM nimmis/java:openjdk-8-jdk
MAINTAINER yujikiriki
# expose spark ports
EXPOSE 8080
EXPOSE 7077
EXPOSE 8081
EXPOSE 6066
WORKDIR /usr/local
# add python
RUN apt-get install -y python
#add scala
RUN wget http://www.scala-lang.org/files/archive/scala-2.11.7.deb
RUN dpkg -i scala-2.11.7.deb
RUN rm -f scala-2.11.7.deb
# build spark
RUN export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=512m"
RUN curl -s http://mirror.metrocast.net/apache/spark/spark-1.4.1/spark-1.4.1.tgz| tar -xz ; ln -s spark-1.4.1 spark
RUN cd spark ; sbt/sbt assembly
# start spark standalone master with one slave
RUN /usr/local/spark/sbin/start-master.sh
RUN env
RUN /usr/local/spark/sbin/start-slave.sh spark://$HOSTNAME:7077
CMD /usr/local/spark/bin/spark-shell --master spark://$HOSTNAME:7077
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment