Last active

Embed URL

HTTPS clone URL

SSH clone URL

You can clone with HTTPS or SSH.

Download Gist

git pre-commit hook for stripping output from IPython notebooks

View nbstripout
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61
#!/usr/bin/env python
"""strip outputs from an IPython Notebook
Opens a notebook, strips its output, and writes the outputless version to the original file.
Useful mainly as a git filter or pre-commit hook for users who don't want to track output in VCS.
This does mostly the same thing as the `Clear All Output` command in the notebook UI.
"""
 
import io
import sys
 
try:
# Jupyter >= 4
from nbformat import read, write, NO_CONVERT
except ImportError:
# IPython 3
try:
from IPython.nbformat import read, write, NO_CONVERT
except ImportError:
# IPython < 3
from IPython.nbformat import current
def read(f, as_version):
return current.read(f, 'json')
def write(nb, f):
return current.write(nb, f, 'json')
 
 
def _cells(nb):
"""Yield all cells in an nbformat-insensitive manner"""
if nb.nbformat < 4:
for ws in nb.worksheets:
for cell in ws.cells:
yield cell
else:
for cell in nb.cells:
yield cell
 
 
def strip_output(nb):
"""strip the outputs from a notebook object"""
nb.metadata.pop('signature', None)
for cell in _cells(nb):
if 'outputs' in cell:
cell['outputs'] = []
if 'prompt_number' in cell:
cell['prompt_number'] = None
return nb
 
 
if __name__ == '__main__':
filename = sys.argv[1]
with io.open(filename, 'r', encoding='utf8') as f:
nb = read(f, as_version=NO_CONVERT)
nb = strip_output(nb)
with io.open(filename, 'w', encoding='utf8') as f:
write(nb, f)
 
View nbstripout
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
#!/bin/sh
#
# strip output of IPython Notebooks
# add this as `.git/hooks/pre-commit`
# to run every time you commit a notebook
#
# requires `nbstripout` to be available on your PATH
#
 
if git rev-parse --verify HEAD >/dev/null 2>&1; then
against=HEAD
else
# Initial commit: diff against an empty tree object
against=4b825dc642cb6eb9a060e54bf8d69288fbee4904
fi
# Find notebooks to be committed
(
IFS='
'
NBS=`git diff-index -z --cached $against --name-only | grep '.ipynb$' | uniq`
 
for NB in $NBS ; do
echo "Removing outputs from $NB"
nbstripout "$NB"
git add "$NB"
done
)
 
exec git diff-index --check --cached $against --
Owner

Or as a git filter instead:

(from @JanShulz)

Add this to your .git/config:

[filter "stripoutput"]
    clean = "/path/to/nbconvert/nbstripout"

and a .gitattributes file with

*.ipynb filter=stripoutput

for the git filter, I made the following changes to nbstripout: https://github.com/cfriedline/ipynb_template/blob/master/nbstripout

nb = current.read(sys.stdin, 'json')
nb = strip_output(nb)
current.write(nb, sys.stdout, 'json')

To get it to work for me I hat to add -a to grep and remove the $ in the git hook. Dunno why, but now it works.

NBS=`git diff-index -z --cached $against --name-only | grep -a '.ipynb' | uniq`

Line 23 of pre-commit, one needs to replace
for NB in NBS
with
for NB in $NBS
and also to make the change of @sotte.

Any chance this will become part of nbconvert?

Is there any way to commit the stripped version but leave output in your working directory

On OS X 10.10, I couldn't get NBS=`git diff-index -z ... | grep ... to work with null character separators, so here's one workaround in bash:

#!/bin/bash

...

(
pat='\.ipynb$'
while IFS= read -r -d '' file; do
    if [[ "$file" =~ $pat ]]; then
        printf 'Removing outputs from %q\n' "$file";
        nbstripout "$file"
        git add "$file"
    fi
done < <(git diff-index -z --cached $against --name-only)
)

In nbstripout I also made the following changes, though this probably depends on individual taste. Cell toggling isn't reset by clearing output in the notebook GUI, so toggle states may get versioned even if no output is present. Popping prompt_number matches notebook gui behavior (in IPython 2.4.1).

if 'prompt_number' in cell:
    cell.pop('prompt_number')
if 'collapsed' in cell:
    cell['collapsed'] = False

the pre-commit hook approach didn't work for me (the grep somehow found .py files, but only if there was a .ipynb in the commit..) but filter seems cleaner anyway. Here's what I did to get it working:

I modified cfriedline's nbstripout file slightly to give an informative error when you can't import the latest IPython:
https://github.com/petered/plato/blob/fb2f4e252f50c79768920d0e47b870a8d799e92b/notebooks/config/strip_notebook_output
And added it to my repo, lets say in ./relative/path/to/nbstripout

Also added the file .gitattributes file to the root of the repo, containing:

*.ipynb filter=stripoutput

And created a setup_git_filters.sh containing

git config filter.stripoutput.clean "$(git rev-parse --show-toplevel)/relative/path/to/nbstripout" 
git config filter.stripoutput.smudge cat
git config filter.stripoutput.required true

And ran source setup_git_filters.sh. The fancy $(git rev-parse...) thing is to find the local path of your repo on any (Unix) machine.

Slightly modified method that works with the new notebook format (v4) used in iPython 3

https://gist.github.com/waylonflinn/010f0a1a66760adf914f

The essential difference is an added check for the presence of the worksheets object on the root.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.