Simple notebook to set up ipython notebook on remote servers and access it easily over ssh from the local computer.

SetupRemoteNotebookServers.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:fc41aade06f35220860ede3c666a85e17cef81744859db0ba0ec7b1fe5d8aa1b"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Prerequisites"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This notebook installs various bash scripts to make running and accessing remote ipython notebook sessions easier.  It works over `ssh` using `autossh`, and assumes that the remote computer(s) can successfully run `ipython notebook`, even if it can't open a browser itself.  So the most unusual part is `autossh`.  But if you have that installed, just try to run this script, and if something fails, look at the following subsection for a little more help."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "More detailed explanation of prerequisites"
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Local computer"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Just to use this script, you need\n",
      "\n",
      "  * `bash` as the default user shell\n",
      "  * [`python`](http://www.python.org/) (just to run this script)\n",
      "  \n",
      "To connect to a remote notebook server, you also need\n",
      "\n",
      "  * [`nc`](http://netcat.sourceforge.net/)\n",
      "  * [`autossh`](http://www.harding.motd.ca/autossh/)\n",
      "  * One open port accessible by the user\n",
      "\n",
      "Most linux varieties should have these installed and available.  Mac users will need to install the optional free [Xcode](https://itunes.apple.com/us/app/xcode/id497799835?mt=12), then install `autossh` via [macports](http://www.macports.org/) or [homebrew](http://brew.sh/).  Windows users should [reconsider using Windows](http://www.ubuntu.com/desktop)."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Any computer running the ipython notebook itself"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "\n",
      "  * [`screen`](http://www.gnu.org/software/screen/)\n",
      "  * [`ipython notebook`](http://ipython.org/notebook.html)\n",
      "  * One open port accessible by the user\n",
      "\n",
      "The ipython notebook, of course, requires python to be installed.  Once python is installed, the easiest way to install ipython (and a couple other nice packages) is to run\n",
      "\n",
      "    easy_install pip\n",
      "    pip install --user \"matplotlib scipy ipython[notebook]\"\n",
      "\n",
      "The `--user` flag installs it in the user's directory.  You don't necessarily need it if you manage the computer you're running this on.\n",
      "\n",
      "If the remote computer is managed by someone else, you should just ask the admin to install matplotlib, scipy, and ipython notebook."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Both computers when using notebooks remotely"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "\n",
      "  * [`ssh`](http://www.openssh.com/)\n",
      "  * Easy `ssh` access from the local to the remote computer\n",
      "\n",
      "This last point means that you should be able to run `ssh remote@server.org` if your remote computer is named `remote` and in the `server.org` domain.  You almost certainly want to [set this up using ssh keys](https://wiki.archlinux.org/index.php/SSH_Keys).  And be sure to enter a passphrase when it asks you.\n",
      "\n",
      "[On Mac OS X](https://help.github.com/articles/working-with-ssh-key-passphrases), this means that a box will pop up to ask for your passphrase whenever it's used; check the box telling it to remember the password in your keychain, and it will be automatic from there on out.  On other types of linux, you might need to [do something more complicated](https://wiki.archlinux.org/index.php/SSH_Keys#SSH_agents), but you're using linux, so you'll figure it out.  On Windows, you should think seriously about what you're doing to yourself and those around you by using Windows."
     ]
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Configuration options (Change things here!)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `remote_servers` will be hosting your notebook servers.  Each must be accessible via ssh.  In particular, if you can run `ssh myserver`, then you can put `myserver` in the list.  These three are, of course, just examples."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "remote_servers = ['remote1@somewhere.com', 'remote2@elsewhere.edu', 'remote3@nowhere.gov']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 32
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "For each of the servers you list above, list the port number you can serve the ipython notebook from.  Usually, you can just pick some number above 1024 and get lucky.  If you don't get lucky, you might want to stay away from the 8000s.  These ports can be the same or different; they have no relation to each other.  The only requirement is that you have to be able to use them on the remote computer."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "remote_ports = [4000, 4000, 4000] # Can be the same or different"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 8
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You will also need two consecutive ports on this (local) computer for each server.  These ports must all be different, and separated by at least one spare port."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "local_ports = [4000, 4002, 5000] # Separate by at least 2"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You will need one more port on this (local) computer to serve a local copy of ipython notebook.  If you don't intend to ever serve ipython notebook from this computer, set this to `None`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "local_server_port = 3000 # Separated from any of the `local_ports`"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 62
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Finally, list the startup files that will read in the bash functions created below.  Because different systems behave inconsistently, it's probably safest just to leave these both as is.  Note that these files will have a single line appended to them, saying to read in the shell scripts that will be created by this script."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "source_functions_from_startup_files = ['~/.bashrc', '~/.bash_profile']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 10
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Set password"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You will also need to set a password to use in the web browser when connecting to these servers.  Out of laziness, we'll just use the same password for all our servers.  If you don't like this, you'll have to go through and change/set the passwords individually once you've got things set up.  Otherwise, just run the cell below, and enter the password and verify it in the box that pops up.\n",
      "\n",
      "For security purposes, only the \"hash\" of the password will be saved by the script, so be sure you remember the original password itself."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from IPython.lib import passwd\n",
      "from IPython.display import HTML\n",
      "\n",
      "for i in range(3):\n",
      "    display(HTML(\"\"\"\n",
      "    <script>\n",
      "    function myFunction() {\n",
      "      var p0=prompt(\"Enter a password for web connections to remote servers.\",\"\");\n",
      "      if (p0!=null) {\n",
      "        var p1=prompt(\"Please verify password for web connections to remote servers.  Remember this password for later use\",\"\");\n",
      "        if (p1!=null && p0==p1) {\n",
      "          var kernel = IPython.notebook.kernel;\n",
      "          kernel.execute(\"password_hash=passwd('\"+p1+\"')\");\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "    myFunction()\n",
      "    </script>\n",
      "    \"\"\"))\n",
      "    try:\n",
      "        password_hash\n",
      "        print(\"That worked!\")\n",
      "        break\n",
      "    except NameError:\n",
      "        pass\n",
      "else:\n",
      "    raise UsageError('No matching passwords found. Giving up.')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "That worked!\n"
       ]
      }
     ],
     "prompt_number": 35
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Check configuration options"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now, let's just check to make sure that the options you've given above are okay.  You don't need to change anything here, just run the code, and fix things if you get an error."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Check that we got enough remote_ports\n",
      "if(len(remote_ports)!=len(remote_servers)):\n",
      "    raise ValueError(\"The `remote_ports` and `remote_servers` lists must have the same lengths.\")\n",
      "# Check that we got enough local_ports\n",
      "if(len(local_ports)!=len(remote_servers)):\n",
      "    raise ValueError(\"The `local_ports` and `remote_servers` lists must have the same lengths.\")\n",
      "# Check that all the local_ports are separated by at least 2\n",
      "for i in range(len(local_ports)):\n",
      "    for j in range(i+1,len(local_ports)):\n",
      "        if(abs(local_ports[i]-local_ports[j])<2):\n",
      "            raise ValueError(\"local_ports[{0}]={1} and local_ports[{2}]={3}\".format(i,local_ports[i],j,local_ports[j])\n",
      "                             +\" should be separated by at least 2.\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 11
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Bash function definitions"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The following defines the bash function used to start a local notebook.  If there is nothing listening on the chosen port, it starts a new instance of ipython notebook in a `screen` session.  With no arguments, the function then simply exits.  If there are arguments, they are assumed to be file names for ipython notebooks to be opened."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "notebook_local_start = r\"\"\"notebook () {{\n",
      "    # Usage: notebook [file1.ipynb [file2.ipynb [...]]]\n",
      "    nc -z localhost {Notebook_port} &> /dev/null # Check if port {Notebook_port} is listening\n",
      "    if [ $? -ne 0 ]; then\n",
      "        # If not, open a new ipython notebook server in screen\n",
      "        screen -dmS \"ipython_notebook_server\" bash -l -c \"cd {Notebook_dir}; ipython notebook --profile=nbserver\"\n",
      "        for retry_count in {{1..100}}; do\n",
      "            nc -z localhost {Notebook_port} &> /dev/null # Wait until the port is listening\n",
      "            if [ $? -ne 0 ]; then\n",
      "                sleep 0.1\n",
      "            else\n",
      "                notebook \"$@\" # Now recurse once, with the port listening\n",
      "                return $?\n",
      "            fi\n",
      "        done\n",
      "        echo \"Failed to open the notebook server on port {Notebook_port}\"\n",
      "        return 1\n",
      "    fi\n",
      "    for ARG in \"$@\"; do\n",
      "        # Get the full path to the file, then strip the base of the notebook directory\n",
      "        ARG=$(echo $(cd $(dirname $ARG); pwd)/$(basename $ARG) | \\\n",
      "                     sed \"s/$(echo $(cd $(dirname {Notebook_dir})) | sed 's/\\//\\\\\\//g')//\")\n",
      "        # Now just open that file in the already running notebook server\n",
      "        open \"http://127.0.0.1:{Notebook_port}/notebooks$ARG\"\n",
      "    done\n",
      "    return 0\n",
      "}}\n",
      "\"\"\" # This string is intended to be used with `format` later"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 45
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now we define the bash function used to connect to a remote computer and start the ipython notebook server there.  There are three required arguments to this function:\n",
      "\n",
      "  1. The local port number (this port and the following must be open)\n",
      "  2. The remote port number used by ipython notebook server\n",
      "  3. The remote server name to connect via ssh\n",
      "\n",
      "If anything is listening on the local port, the function simply assumes that it is `autossh`, and that the remote server is running"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "notebook_remote_start = r\"\"\"remotenotebook () {\n",
      "    # Usage: remotenotebook <local_port_number> <remote_port_number> <remote_server_name>\n",
      "    nc -z localhost $1 &> /dev/null # Check if the localhost port is already listening\n",
      "    if [ $? -ne 0 ]; then\n",
      "        # If not, try to tunnel and connect the local port to the remote port\n",
      "        autossh -M $(($1+1)) -N -f -L localhost:$1:localhost:$2 $3\n",
      "        if [ $? -ne 0]; then\n",
      "            echo \"remotenotebook failed to run `autossh -M $(($1+1)) -N -f -L localhost:$1:localhost:$2 $3`.\"\n",
      "            echo \"Check your port availabilities and ssh connection.\"\n",
      "        else\n",
      "            sleep 3 # Give it a few seconds to sort itself out\n",
      "            ssh $3 'notebook' # Make sure the remote has started the notebook server\n",
      "            sleep 5 # Give it a few more seconds to start the notebook server\n",
      "            open https://localhost:$1/ # Open on the local computer's browser\n",
      "        fi\n",
      "    else\n",
      "        # If so, just try to open the browser\n",
      "        open https://localhost:$1/\n",
      "    fi\n",
      "}\n",
      "\"\"\""
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 46
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The following bash function closes the relevant notebook server.  With no arguments, it just closes the server on the computer on which it is run.  With two arguments, it first closes the remote server, and then kills the autossh process that was listening to it.  In this case, the first argument is taken as the local port that autossh is running over, and the second is taken as the ssh name for the remote server."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "notebook_closer = r\"\"\"closenotebook () {\n",
      "    # Usage: closenotebook [<local_port_number> <remote_server_name>]\n",
      "    if [ $# == 2 ]; then\n",
      "        ssh $2 closenotebook\n",
      "        autossh_PID=$(ps x | awk \"(/autossh -M $(($1+1))/)\"' && (!/ awk /){print $1}')\n",
      "        [ -z \"$autossh_PID\" ] && echo \"Cannot find PID of command 'autossh -M $(($1+1))'.  Maybe it's already dead.\" || kill $autossh_PID\n",
      "    else\n",
      "        # Send two Ctrl+c signals to the server, separated in time slightly\n",
      "        screen -S \"ipython_notebook_server\" -p 0 -X stuff $'\\cc'\n",
      "        sleep 0.1\n",
      "        screen -S \"ipython_notebook_server\" -p 0 -X stuff $'\\cc'\n",
      "    fi\n",
      "}\n",
      "\"\"\""
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 47
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Finally, the local bash shell will have a few aliases to start up and close remote servers and connections to them.  For example, for server `remote1@somewhere.com`, we will get the aliases `remote1_notebook` to open the server and `remote1_close` to close the server and the `autossh` connection.  These aliases will only exist locally."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "alias_creation = \"\"\n",
      "for local,remote,server in zip(local_ports, remote_ports, remote_servers):\n",
      "    nice_server_name = server.split('@')[0]\n",
      "    alias_creation += \"\"\"\n",
      "alias {0}_notebook='remotenootebook {1} {2} {3}'\n",
      "alias {0}_close='closenootebook {1} {3}'\"\"\".format(nice_server_name, local, remote, server)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 16
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Set up the remote servers"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import os\n",
      "\n",
      "for local_port,remote_port,remote_server in zip(local_ports, remote_ports, remote_servers):\n",
      "    # Write the temporary shell script that will set things up for us\n",
      "    remote_work_script=r\"\"\"\n",
      "ssh -T {remote_server} <<\\EOF\n",
      "# Create a new profile and switch to that directory\n",
      "mv $(ipython locate profile nbserver) $(ipython locate profile nbserver)_old &> /dev/null\n",
      "ipython profile create nbserver\n",
      "cd $(ipython locate profile nbserver)\n",
      "\n",
      "# Create a new SSL certificate\n",
      "openssl req -batch -x509 -nodes -days 730 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem\n",
      "\n",
      "# Append the following lines to the config file\n",
      "echo >> ipython_notebook_config.py\n",
      "echo \\# Customizations >> ipython_notebook_config.py\n",
      "echo c.IPKernelApp.pylab = \\'inline\\' >> ipython_notebook_config.py\n",
      "echo c.NotebookApp.certfile = u\\'$(ipython locate profile nbserver)/mycert.pem\\' >> ipython_notebook_config.py\n",
      "echo c.NotebookApp.ip = \\'*\\' >> ipython_notebook_config.py\n",
      "echo c.NotebookApp.open_browser = False >> ipython_notebook_config.py\n",
      "echo c.NotebookApp.password = u\\'{password_hash}\\' >> ipython_notebook_config.py\n",
      "echo c.NotebookApp.port = {remote_port} >> ipython_notebook_config.py\n",
      "\n",
      "# Append any requested startup files\n",
      "cd\n",
      "{startup_appends}\n",
      "EOF\n",
      "\"\"\".format(password_hash=password_hash, remote_port=remote_port, remote_server=remote_server,\n",
      "           startup_appends=''.join([\"echo '. ~/.bash_nbserver' >> {0}\\n\".format(startup_file)\n",
      "                                      for startup_file in source_functions_from_startup_files]))\n",
      "    with open(\"nbserver_tmp1.sh\", \"w\") as script_file:\n",
      "        script_file.write(remote_work_script)\n",
      "\n",
      "    # Write the ~/.bash_nbserver file\n",
      "    with open(\"nbserver_tmp2.sh\", \"w\") as script_file:\n",
      "        script_file.write(notebook_local_start.format(Notebook_dir='',Notebook_port=remote_port))\n",
      "        #script_file.write(notebook_remote_start)\n",
      "        script_file.write(notebook_closer)\n",
      "        #script_file.write(alias_creation)\n",
      "    \n",
      "    # Copy the files, run the script, and delete the temp files\n",
      "    ! scp nbserver_tmp2.sh {remote_server}:~/.bash_nbserver\n",
      "    os.remove('nbserver_tmp2.sh')\n",
      "    ! bash nbserver_tmp1.sh\n",
      "    os.remove('nbserver_tmp1.sh')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 57
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Set up this computer"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "if(local_server_port):\n",
      "    pwd=os.getcwd()\n",
      "\n",
      "    # Write the temporary shell script that will set things up for us\n",
      "    local_work_script=r\"\"\"\n",
      "    # Create a new profile and switch to that directory\n",
      "    mv $(ipython locate profile nbserver) $(ipython locate profile nbserver)_old &> /dev/null\n",
      "    ipython profile create nbserver\n",
      "    cd $(ipython locate profile nbserver)\n",
      "\n",
      "    # Create a new SSL certificate\n",
      "    openssl req -batch -x509 -nodes -days 730 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem\n",
      "\n",
      "    # Append the following lines to the config file\n",
      "    echo >> ipython_notebook_config.py\n",
      "    echo \\# Customizations >> ipython_notebook_config.py\n",
      "    echo c.IPKernelApp.pylab = \\'inline\\' >> ipython_notebook_config.py\n",
      "    #echo c.NotebookApp.certfile = u\\'$(ipython locate profile nbserver)/mycert.pem\\' >> ipython_notebook_config.py\n",
      "    echo c.NotebookApp.ip = \\'*\\' >> ipython_notebook_config.py\n",
      "    echo c.NotebookApp.open_browser = False >> ipython_notebook_config.py\n",
      "    #echo c.NotebookApp.password = u\\'{password_hash}\\' >> ipython_notebook_config.py\n",
      "    echo c.NotebookApp.port = {local_port} >> ipython_notebook_config.py\n",
      "\n",
      "    # Append any requested startup files\n",
      "    cd\n",
      "    {startup_appends}\n",
      "    \"\"\".format(password_hash=password_hash, local_port=local_server_port,\n",
      "               startup_appends=''.join([\"echo '. ~/.bash_nbserver' >> {0}\\n\".format(startup_file)\n",
      "                                          for startup_file in source_functions_from_startup_files]))\n",
      "    with open(\"nbserver_tmp1.sh\", \"w\") as script_file:\n",
      "        script_file.write(local_work_script)\n",
      "    ! bash nbserver_tmp1.sh\n",
      "    os.remove('nbserver_tmp1.sh')\n",
      "\n",
      "\n",
      "# Write the ~/.bash_nbserver file\n",
      "with open(\"nbserver_tmp2.sh\", \"w\") as script_file:\n",
      "    script_file.write(notebook_local_start.format(Notebook_dir='',Notebook_port=remote_port))\n",
      "    script_file.write(notebook_remote_start)\n",
      "    script_file.write(notebook_closer)\n",
      "    script_file.write(alias_creation)\n",
      "\n",
      "# Copy the files, run the script, and delete the temp files\n",
      "! cp nbserver_tmp2.sh ~/.bash_nbserver\n",
      "os.remove('nbserver_tmp2.sh')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 67
    }
   ],
   "metadata": {}
  }
 ]
}