Skip to content

Instantly share code, notes, and snippets.

@odewahn
Last active October 22, 2015 15:29
Show Gist options
  • Save odewahn/84dee8bbbe0259c7e56a to your computer and use it in GitHub Desktop.
Save odewahn/84dee8bbbe0259c7e56a to your computer and use it in GitHub Desktop.
export TOKEN=$( head -c 30 /dev/urandom | xxd -p )
export POOL_SIZE=5
export OVERPROVISION_FACTOR=2
export CPU_SHARES=$(( (1024*${OVERPROVISION_FACTOR})/${POOL_SIZE} ))
export TMPNB_NODE=bb0f0ccc-2dcb-410b-94cf-808c99324ab6-n1
export DOCKER_HOST=tcp://104.130.0.25:2376
docker run -d \
-P \
-p 80 \
-h sparkdemo.tmpnb-oreilly.com \
-e CONFIGPROXY_AUTH_TOKEN=$TOKEN \
--restart=always \
-e constraint:node==$TMPNB_NODE \
--name=proxy \
-u root \
jupyter/configurable-http-proxy \
--default-target http://127.0.0.1:9999 \
--port 80 \
--api-ip 127.0.0.1 \
--api-port 8001
docker run --rm --volumes-from swarm-data \
-e constraint:node==$TMPNB_NODE \
busybox \
sh -c "cp /etc/docker/server-cert.pem /etc/docker/cert.pem && cp /etc/docker/server-key.pem /etc/docker/key.pem"
docker run -d --restart=always \
--net=container:proxy \
-e CONFIGPROXY_AUTH_TOKEN=$TOKEN \
-e constraint:node==$TMPNB_NODE \
--volumes-from swarm-data \
--name=tmpnb \
-e DOCKER_HOST=$DOCKER_HOST \
-e DOCKER_TLS_VERIFY=1 \
-e DOCKER_CERT_PATH=/etc/docker \
jupyter/tmpnb python orchestrate.py \
--image='zischwartz/sparkdemo' \
--container_ip=0.0.0.0 \
—-allow_origin="*" \
--command="/bin/bash -c 'IPYTHON_OPTS=\"notebook --NotebookApp.base_url=/{base_path} --ip=0.0.0.0 --NotebookApp.allow_origin=*\" pyspark --packages com.databricks:spark-csv_2.10:1.2.0'" \
--pool_size=$POOL_SIZE \
--mem_limit='128m' \
--cpu_shares=$CPU_SHARES
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment