A node.js kernel for IPython notebook. You can see the explanation of the ipynb rendered in http://nbviewer.ipython.org

kernel.js

zmq = require("zmq")
fs = require("fs")

var config = JSON.parse(fs.readFileSync(process.argv[2]))

var connexion = "tcp://"+config.ip+":"
var shell_conn = connexion+config.shell_port
var pub_conn = connexion+config.iopub_port
var hb_conn = connexion+config.hb_port



var util = require('util'),
    vm = require('vm'),
    initSandbox = {},
    context = vm.createContext(initSandbox);


var hb_socket = zmq.createSocket('rep');
hb_socket.bind(hb_conn)

hb_socket.on('message',
        function(data){
            console.log("wtf ?");
            hb_socket.send(data);
        });

var pub_socket = zmq.createSocket('pub');
pub_socket.bind(pub_conn);


var reply_socket = zmq.createSocket('xrep')
reply_socket.bind(shell_conn)

reply_socket.on('message',
        function(data){
            for(i in arguments){
               console.log("["+i+"]: "+arguments[i].toString())
            }
            
            var parent_header = JSON.parse(arguments[3].toString());
            
            var unparsed_content = arguments[6];
            if(unparsed_content != undefined ) {
                var content = JSON.parse(unparsed_content.toString());
            }

            var code = content?content.code:undefined;
            var result
            if(code != undefined){
                result = vm.runInContext(code , context, '<kernel>');
            } else {
                result = 'undefined'
            }
            
            var header_reply ={
                msg_id:1,
                session:parent_header.session,
                msg_type:"execute_reply",
            } 


            var ident = "";
            var delim =  "<IDS|MSG>"
            var signature = ""
            var metadata = {}

            var content = JSON.stringify({
                    execution_count:1,
                    data:{
                        "text/plain":result?result.toString():"undefined"
                    }
                })

            

            var header_pub ={
                msg_id:1,
                session:parent_header.session,
                msg_type:"pyout",
            }

            pub_socket.send([
                    ident,
                    delim,
                    signature,
                    JSON.stringify(header_pub),
                    JSON.stringify(parent_header),
                    JSON.stringify(metadata),
                    content])

            reply_socket.send([
                    ident,
                    delim,
                    signature,
                    JSON.stringify(header_reply),
                    JSON.stringify(parent_header),
                    JSON.stringify(metadata),
                    content
                ]);

            })


reply_socket.on('error',
        function(data){
            console.log('error',data)
            })

node-kernel.ipynb

{
 "metadata": {
  "name": "node-kernel"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Writing a Node.js kernel for IPyton in half an hour."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This will show you how to write a native kernel for the [IPython notebook server](http://ipython.org/ipython-doc/dev/interactive/htmlnotebook.html) using [node.js](http://nodejs.org/).\n",
      "\n",
      "For those who don't know what node is, we can look at the small description on their website :\n",
      "\n",
      "![](http://upload.wikimedia.org/wikipedia/en/a/a7/Nodejs_logo_light.png)\n",
      "\n",
      "> Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.\n",
      "\n",
      "For those who don't know IPython and in particular its Notebook frontend, you are probably reading this in a static version of the notebook in which I wrote this. The IPython notebook is a web-base frontend which allows interactive programming and display of rich media type like images, video, $\\LaTeX$ thanks to [Mathjax](http://www.mathjax.org/). Python is for know the only native kernel supported, but we wish to improve this. \n",
      "\n",
      "![](http://ipython.org/ipython-doc/dev/_static/logo.png)\n",
      "\n",
      "A prototype of [ruby]() [kernel]() as already been developped, and I am working on dooing the same with [Julia](https://github.com/JuliaLang)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "---\n",
      "### Let's go.\n",
      "\n",
      "First you will need the content of [this pull request](https://github.com/ipython/ipython/pull/2643) if it have not been merged yet.\n",
      "\n",
      "Make sure you can start the IPython notebook and that everythong works.\n",
      "\n",
      "Create an IPython profile, I'll call it \"node\"\n",
      "```\n",
      "$ ipython profile create node \n",
      "```\n",
      "\n",
      "edit your notebook profile (`~/.ipython/profile_node/ipython_notebook_config.py`) and add the following :\n",
      "\n",
      "```\n",
      "c.SubprocessKernelManager.kernel_launch_program = ['node','~/node-kernel/kernel.js','{connection_file_name}']\n",
      "c.MappingKernelManager.kernel_manager_class= 'IPython.zmq.subprocesskernel.SubprocessKernelManager'\n",
      "c.Session.key=''\n",
      "```\n",
      "\n",
      "where `~/node-kernel/kernel.js` is where your node kernel will live.\n",
      "\n",
      "you might want to install node also, it will be helpfull.\n",
      "\n",
      "### install node-zmq\n",
      "\n",
      "simply use\n",
      "\n",
      "```\n",
      "$ npm install zmq\n",
      "```\n",
      "\n",
      "if you don't know what ZMQ is, *Clark's third law* state that\n",
      "\n",
      "> Any sufficiently advanced technology is indistinguishable from magic\n",
      "\n",
      "So ZMQ is a magic messaging library. \n",
      "\n",
      "### Start to code\n",
      "Our program will have to read its configuration from the filename given as the first argument and connect 3 ZMQ channel.\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "zmq = require(\"zmq\")\n",
      "fs = require(\"fs\")\n",
      "\n",
      "var config = JSON.parse(fs.readFileSync(process.argv[2]))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": "*"
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "I'll let you look at the config object yourself.\n",
      "Setup the connexion strings :"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "var connexion = \"tcp://\"+config.ip+\":\"\n",
      "var shell_conn = connexion+config.shell_port\n",
      "var pub_conn = connexion+config.iopub_port\n",
      "var hb_conn = connexion+config.hb_port"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": "*"
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Create a context to evaluate the code we will receive later. (copy pasted from docs)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "var util = require('util'),\n",
      "    vm = require('vm'),\n",
      "    initSandbox = {},\n",
      "    context = vm.createContext(initSandbox);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": "*"
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "First to avoid the message telling us that the kernel has died, we setup an echo server on the heartbeat channel which is a simple req/rep in zmq dialect.\n",
      "For whatever reason we have to log something on the console otherwise it does not work (no idea why, I'll figure out later)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "var hb_socket = zmq.createSocket('rep');\n",
      "hb_socket.bind(hb_conn)\n",
      "\n",
      "hb_socket.on('message',\n",
      "        function(data){\n",
      "            console.log(\"Dummy console log.\");\n",
      "            hb_socket.send(data);\n",
      "        });\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": "*"
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now we create and bind the two other channels, `pub_socket` should be of type `pub` and `reply_socket` of type `xrep`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "var pub_socket = zmq.createSocket('pub');\n",
      "pub_socket.bind(pub_conn);\n",
      "\n",
      "var reply_socket = zmq.createSocket('xrep')\n",
      "reply_socket.bind(shell_conn)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now most of the code will be on the callback which is called when the `reply_socket` receive a message"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "reply_socket.on('message',\n",
      "                function(data){\n",
      "                               //rest of the code\n",
      "                });"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": "*"
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Well, actually we will not use `data`. As we send messages in part, the callback will receive a varaible number of arguments.\n",
      "We are only interested here at the 3rd and 6th argument. Respectively header of arriving message and content. \n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "var parent_header = JSON.parse(arguments[3].toString());\n",
      "            \n",
      "var unparsed_content = arguments[6];"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": "*"
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We take care of extracting the code that should be executed if there is some. "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "if(unparsed_content != undefined ) {\n",
      "    var content = JSON.parse(unparsed_content.toString());\n",
      "}\n",
      "\n",
      "var code = content?content.code:undefined;\n",
      "    "
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": "*"
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "And run it in a separate context that we've created before."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "var result\n",
      "if(code != undefined){\n",
      "    result = vm.runInContext(code , context, '<kernel>');\n",
      "} else {\n",
      "    result = 'undefined'\n",
      "}"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "To construct the minimal messages that are accepted by the server we will need the following.\n",
      "It does not respect the specification but should be enough as an example."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "var ident = \"\";\n",
      "var delim =  \"<IDS|MSG>\"\n",
      "var signature = \"\"\n",
      "var metadata = {}\n",
      "\n",
      "var header_reply ={\n",
      "    msg_id:1,\n",
      "    session:parent_header.session,\n",
      "    msg_type:\"execute_reply\",\n",
      "}\n",
      "\n",
      "var header_pub ={\n",
      "    msg_id:1,\n",
      "    session:parent_header.session,\n",
      "    msg_type:\"pyout\",\n",
      "}"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": "*"
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The part that depends on the result is he following. Here I explicitely set the `execution_count` to 1, but a correct kernel would count the number of execution done by the user. \n",
      "\n",
      "As the frontend support several representation, I choose here to the usual one which is 'text/plain', and set it to the stringified result returned from the previous evaluation."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "var content = JSON.stringify({\n",
      "    execution_count:1,\n",
      "    data:{\n",
      "        \"text/plain\":result?result.toString():\"undefined\"\n",
      "    }\n",
      "})"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": "*"
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "I can now send the reply to both the `reply_socket` and the `iopub_socket`.\n",
      "here the content are similar except for 4th one (header) build above."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "pub_socket.send([\n",
      "    ident,\n",
      "    delim,\n",
      "    signature,\n",
      "    JSON.stringify(header_pub),\n",
      "    JSON.stringify(parent_header),\n",
      "    JSON.stringify(metadata),\n",
      "    content])\n",
      "\n",
      "reply_socket.send([\n",
      "    ident,\n",
      "    delim,\n",
      "    signature,\n",
      "    JSON.stringify(header_reply),\n",
      "    JSON.stringify(parent_header),\n",
      "    JSON.stringify(metadata),\n",
      "    content\n",
      "]);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": "*"
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "I can now close my callback, save my file and run IPython notebook."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "```\n",
      "$ ipython notebook --profile=node\n",
      "```\n",
      "If everything went allright, I should now be able to write how I did it, and execute js on server side :"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "[\"hello from \",\" to IPython Notebook\"].join(\"Node.js\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 1,
       "text": [
        "hello from Node.js to IPython Notebook"
       ]
      }
     ],
     "prompt_number": "*"
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "###Closing thought. \n",
      "\n",
      "There are of course many gotchas with current implementation, mainly the one of [evaluating code in the vm](http://nodejs.org/api/vm.html), of course the kernel should catch the error of user to display traceback and register a display hook to redirect stdin/stdout to the frontend.\n",
      "\n",
      "My knowledge of Node is really limited as this is my first program using it, so my guess is more competent people will certainly be able to do much more things with this as a starting point. \n",
      "\n",
      "I will certainly post this as [a gist](https://gist.github.com/4279371) and [directly](http://nbviewer.ipython.org/4279371/) [nbviewer](http://nbviewer.ipython.org) with a link to [the kernel.js file](https://gist.github.com/4279371/). \n",
      "\n",
      "Please send me any correction or improvement you wish."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}