Skip to content

Instantly share code, notes, and snippets.

@backslash112

backslash112/install-spark-with-docker.md

Last active April 18, 2018 18:42
Show Gist options
  • Save backslash112/7829f1e0f79a7805745c1ec16c50b574 to your computer and use it in GitHub Desktop.
Save backslash112/7829f1e0f79a7805745c1ec16c50b574 to your computer and use it in GitHub Desktop.
Download ZIP
QuickStart: Try Spark with Docker
Raw
install-spark-with-docker.md

Environment:

  1. Ubuntu 16.04 x64

Requirments:

  1. Docker install-docker-ubuntu-16-04.md
  2. Docker image for Spark: andypetrella/spark-notebook

Commands:

docker pull andypetrella/spark-notebook:0.9.0-SNAPSHOT-scala-2.11.8-spark-2.2.1-hadoop-2.7.2-with-hive

docker run \
-p 9000:9000 \
-p 4040-4045:4040-4045 \
andypetrella/spark-notebook:0.9.0-SNAPSHOT-scala-2.11.8-spark-2.2.1-hadoop-2.7.2-with-hive

Then you can using Spark by visit 9000 port. For example: 127.0.0.1:9000

Run a hello world with Scala

  1. Create a new notebook, named hello-world:

screen shot 2018-04-18 at 1 33 14 pm

2. Write some code and hit the run button:

screen shot 2018-04-18 at 1 34 24 pm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment