IPython Notebook Git revision control with Pre-commit hook, Netcat and Python Daemon

attributes

# This goes in .git/info

*.ipynb    filter=dropoutput_ipynb

initalize.sh

#!/bin/bash

# Tell the repository to use the strip filter

git config filter.dropoutput_ipynb.clean /home/change_this_path/bin/stripfilter.sh
git config filter.dropoutput_ipynb.smudge cat

# Your .git/config will have the following lines added to it
#
# filter "dropoutput_ipynb"]
# 	clean = /home/change_this_path/bin/stripfilter.sh
# 	smudge = cat

nbstripdaemon.py

#!/usr/bin/python

import socket
from IPython.nbformat.current import reads, writes

TCP_IP = '127.0.0.1'
TCP_PORT = 5005
BUFFER_SIZE = 1024

def processIPython(strin):
    json_in = reads(strin, 'json')

    for sheet in json_in.worksheets:
        for cell in sheet.cells:
            if "outputs" in cell:
                cell.outputs = []
            if "prompt_number" in cell:
                cell.prompt_number = None

    data = writes(json_in, 'json')
    return data

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((TCP_IP, TCP_PORT))
s.listen(1)

while True:
    try:
        conn, addr = s.accept()
        #print 'Connection address:', addr
        buffer = []
        while 1:
            data = conn.recv(BUFFER_SIZE)
            if not data: break
            buffer.append(data)

        data = "".join(buffer)

        data = processIPython(data)
            
        conn.send(data)  # echo
        conn.close()
    except:
        print "weird exception, handle to taste" 
        

stripfilter.sh

#!/bin/bash
# This is my all time favorite script

nc 127.0.0.1 5005

Python is slow as a filter. The script is invoked multiple times by git tools like tig, git diff, etc.

Launching a tiny python daemon and using Netcat to twist the ends together works well.

Good info from: stackoverflow though make sure to use 'None instead of '' in the Json dict.