jgwerner

#DRAFT

#Introduction

JupyterHub was developed to offer additional choices when running Jupyter Notebook servers (formerly known as IPython Notebooks). Basically, three options may be configured:

1. Single user environments: one user needs one notebook to be spawned on their local or remote machine
2. Temporary multiple user environments: multiple users need notebooks spawned for each user with temporary home paths
3. Permanent multiple user environments: multiple users need notebooks spawned for each user with permanent home paths

jgwerner

#!/bin/bash
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

sudo apt-get purge maven maven2 maven3
sudo apt-add-repository ppa:andrei-pozolotin/maven3
sudo apt-get update
sudo apt-get install maven3

jgwerner

worker_processes  1;

events {
	worker_connections 1024;
}

http {

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

jgwerner

# Configuration file for Jupyter Hub
c = get_config()

import os
import sys
sys.path.insert(0, '/srv/jupyterhub_config')

# Base configuration
c.JupyterHub.log_level = "INFO"
c.JupyterHub.db_url = "postgresql://{}:{}@{}:{}/{}".format(

jgwerner

Overlay Network on local machine with VirtualBox

Set up K/V server

set up k/v store with virtualbox

docker-machine create -d virtualbox mh-keystore

switch docker to k/v machine context

eval "$(docker-machine env mh-keystore)"

jgwerner

# Jupyter Notebook Python, Spark, Mesos Stack

## What it Gives You

* Jupyter Notebook 4.2.x
* Conda Python 3.x and Python 2.7.x environments
* pyspark, pandas, matplotlib, scipy, seaborn, scikit-learn pre-installed
* Spark 1.6.0 for use in local mode or to connect to a cluster of Spark workers
* Mesos client 0.22 binary that can communicate with a Mesos master

jgwerner

Maintenance and troubleshooting commands

Installing Docker

Docker install script is probably the best way to install docker with Ubuntu, as it's the official bash script maintained by Docker and is updated with latest features:

$ curl -sSL https://get.docker.com | sh

Uninstalling Docker

jgwerner

Advanced Topics

• first-section

First Section

jgwerner

from jupyter_core.paths import jupyter_data_dir
import subprocess
import os
import errno
import stat

c = get_config()
c.NotebookApp.ip = '*'
c.NotebookApp.port = 8888
c.NotebookApp.open_browser = False

jgwerner

Data Science Workflow

Define the Problem

• What is the problem? Provide formal and informal definitions.
• Why does the problem need to be solved? Motivation, benefits, how it will be used.
• How would I solve the problem? Describe how the problem would be solved manually to flush domain knowledge.

Prepare Data