Skip to content

@minrk /nbstripout
Last active

Embed URL

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
git pre-commit hook for stripping output from IPython notebooks
#!/usr/bin/env python
"""strip outputs from an IPython Notebook
Opens a notebook, strips its output, and writes the outputless version to the original file.
Useful mainly as a git filter or pre-commit hook for users who don't want to track output in VCS.
This does mostly the same thing as the `Clear All Output` command in the notebook UI.
"""
import io
import sys
try:
# Jupyter >= 4
from nbformat import read, write, NO_CONVERT
except ImportError:
# IPython 3
try:
from IPython.nbformat import read, write, NO_CONVERT
except ImportError:
# IPython < 3
from IPython.nbformat import current
def read(f, as_version):
return current.read(f, 'json')
def write(nb, f):
return current.write(nb, f, 'json')
def _cells(nb):
"""Yield all cells in an nbformat-insensitive manner"""
if nb.nbformat < 4:
for ws in nb.worksheets:
for cell in ws.cells:
yield cell
else:
for cell in nb.cells:
yield cell
def strip_output(nb):
"""strip the outputs from a notebook object"""
nb.metadata.pop('signature', None)
for cell in _cells(nb):
if 'outputs' in cell:
cell['outputs'] = []
if 'prompt_number' in cell:
cell['prompt_number'] = None
return nb
if __name__ == '__main__':
filename = sys.argv[1]
with io.open(filename, 'r', encoding='utf8') as f:
nb = read(f, as_version=NO_CONVERT)
nb = strip_output(nb)
with io.open(filename, 'w', encoding='utf8') as f:
write(nb, f)
#!/bin/sh
#
# strip output of IPython Notebooks
# add this as `.git/hooks/pre-commit`
# to run every time you commit a notebook
#
# requires `nbstripout` to be available on your PATH
#
if git rev-parse --verify HEAD >/dev/null 2>&1; then
against=HEAD
else
# Initial commit: diff against an empty tree object
against=4b825dc642cb6eb9a060e54bf8d69288fbee4904
fi
# Find notebooks to be committed
(
IFS='
'
NBS=`git diff-index -z --cached $against --name-only | grep '.ipynb$' | uniq`
for NB in $NBS ; do
echo "Removing outputs from $NB"
nbstripout "$NB"
git add "$NB"
done
)
exec git diff-index --check --cached $against --
@minrk
Owner

Or as a git filter instead:

(from @JanShulz)

Add this to your .git/config:

[filter "stripoutput"]
    clean = "/path/to/nbconvert/nbstripout"

and a .gitattributes file with

*.ipynb filter=stripoutput
@cfriedline

for the git filter, I made the following changes to nbstripout: https://github.com/cfriedline/ipynb_template/blob/master/nbstripout

nb = current.read(sys.stdin, 'json')
nb = strip_output(nb)
current.write(nb, sys.stdout, 'json')
@sotte

To get it to work for me I hat to add -a to grep and remove the $ in the git hook. Dunno why, but now it works.

NBS=`git diff-index -z --cached $against --name-only | grep -a '.ipynb' | uniq`
@SylvainCorlay

Line 23 of pre-commit, one needs to replace
for NB in NBS
with
for NB in $NBS
and also to make the change of @sotte.

@ketch

Any chance this will become part of nbconvert?

@justmytwospence

Is there any way to commit the stripped version but leave output in your working directory

@aarontran

On OS X 10.10, I couldn't get NBS=`git diff-index -z ... | grep ... to work with null character separators, so here's one workaround in bash:

#!/bin/bash

...

(
pat='\.ipynb$'
while IFS= read -r -d '' file; do
    if [[ "$file" =~ $pat ]]; then
        printf 'Removing outputs from %q\n' "$file";
        nbstripout "$file"
        git add "$file"
    fi
done < <(git diff-index -z --cached $against --name-only)
)

In nbstripout I also made the following changes, though this probably depends on individual taste. Cell toggling isn't reset by clearing output in the notebook GUI, so toggle states may get versioned even if no output is present. Popping prompt_number matches notebook gui behavior (in IPython 2.4.1).

if 'prompt_number' in cell:
    cell.pop('prompt_number')
if 'collapsed' in cell:
    cell['collapsed'] = False
@petered

the pre-commit hook approach didn't work for me (the grep somehow found .py files, but only if there was a .ipynb in the commit..) but filter seems cleaner anyway. Here's what I did to get it working:

I modified cfriedline's nbstripout file slightly to give an informative error when you can't import the latest IPython:
https://github.com/petered/plato/blob/fb2f4e252f50c79768920d0e47b870a8d799e92b/notebooks/config/strip_notebook_output
And added it to my repo, lets say in ./relative/path/to/nbstripout

Also added the file .gitattributes file to the root of the repo, containing:

*.ipynb filter=stripoutput

And created a setup_git_filters.sh containing

git config filter.stripoutput.clean "$(git rev-parse --show-toplevel)/relative/path/to/nbstripout" 
git config filter.stripoutput.smudge cat
git config filter.stripoutput.required true

And ran source setup_git_filters.sh. The fancy $(git rev-parse...) thing is to find the local path of your repo on any (Unix) machine.

@waylonflinn

Slightly modified method that works with the new notebook format (v4) used in iPython 3

https://gist.github.com/waylonflinn/010f0a1a66760adf914f

The essential difference is an added check for the presence of the worksheets object on the root.

@dietmarw

I've created a version that removes the whole cell. Although I have to admit the way I track the index is not at all optimal and there might be better ways making proper use of the API. Feedback welcome:
https://gist.github.com/dietmarw/dc0cf089d8d6211136d5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.