Skip to content

Instantly share code, notes, and snippets.

@jbu
Last active August 29, 2015 14:19
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jbu/1d950d698e8b9113a8e2 to your computer and use it in GitHub Desktop.
Save jbu/1d950d698e8b9113a8e2 to your computer and use it in GitHub Desktop.
spark docker
#FROM errordeveloper/oracle-jdk
FROM debian:jessie
RUN apt-get update && apt-get -y install curl
ENV JAVA_HOME /usr/jdk1.8.0_31
RUN curl \
# --silent \
--location \
--retry 3 \
--header "Cookie: oraclelicense=accept-securebackup-cookie;" \
"http://download.oracle.com/otn-pub/java/jdk/8u31-b13/jdk-8u31-linux-x64.tar.gz" \
| gunzip \
| tar x -C /usr/ \
&& ln -s $JAVA_HOME /usr/java \
&& rm -rf $JAVA_HOME/src.zip $JAVA_HOME/javafx-src.zip $JAVA_HOME/man
ENV PATH ${PATH}:${JAVA_HOME}/bin
RUN curl \
# --silent \
http://d3kbcqa49mib13.cloudfront.net/spark-1.3.0-bin-hadoop2.4.tgz \
| gunzip \
| tar x -C / && \
cd /spark-1.3.0-bin-hadoop2.4/lib && \
curl \
# --silent \
-O \
https://storage.googleapis.com/hadoop-lib/gcs/gcs-connector-latest-hadoop2.jar && \
echo export SPARK_CLASSPATH=$SPARK_CLASSPATH:/spark-1.3.0-bin-hadoop2.4/lib/gcs-connector-latest-hadoop2.jar > /spark-1.3.0-bin-hadoop2.4/conf/spark-env.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment