Skip to content

Instantly share code, notes, and snippets.

@jfeist
Last active December 20, 2017 13:26
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jfeist/cd00aa3b681092e1d5dc to your computer and use it in GitHub Desktop.
Save jfeist/cd00aa3b681092e1d5dc to your computer and use it in GitHub Desktop.

Small script to flatten an IPython notebook (much like https://gist.github.com/takluyver/bc8f3275c7d34abb68bf), using jq instead of python. It is significantly faster. Put it somewhere in your path and make it executable. You can call it with nbflatten.jq --arg show_output 0 to suppress showing outputs (by default they are shown).

#!/bin/sh
jq -r 'def banner: "\(.) " + (28-(.|length))*"-";
("Non-cell info" | banner), del(.cells), "",
(.cells[] | ("\(.cell_type) cell" | banner),
"\(.source|add)",
if ($show_output == "1") then
"",
( select(.cell_type=="code" and (.outputs|length)>0) |
("output" | banner),
(.outputs[] |
(select(.text) | "\(.text|add)" | rtrimstr("\n")),
(select(.traceback) | (.traceback|join("\n"))),
(select(.text or .traceback|not) | "(Non-plaintext output)")
),
""
)
else ""
end
)' "$@" --arg show_output 1
@jankatins
Copy link

I'm currently using the following, called with jq -r -f nbflatten.jq:

def banner: "\(.) " + (28-(.|length))*"-";
# metadata
("Non-cell info" | banner), del(.cells), "",
# content
(.cells[] | (
     ("\(.cell_type) cell" | banner), 
     (.source[] | rtrimstr("\n")), # output source
     if ($show_output == "1") then # the cell output only when it is requested..
       "",
       (select(.cell_type=="code" and (.outputs|length)>0) | (
         ("output" | banner),
         (.outputs[] | (
            (select(.text) | "\(.text|add)" | rtrimstr("\n")),
            (select(.traceback) | (.traceback|join("\n"))),
            (select(.text or .traceback|not) | "(Non-plaintext output)")
           )
         ),
         ""
        )
       )
     else 
       ""
     end
  )
)

Main change:

  • -r will remove the quotes from each line
  • using (.source[] | rtrimstr("\n") instead of "\(.source|add)" will output each line in a codecell on it's own instead of in one long string.

@axsk
Copy link

axsk commented Feb 24, 2016

this is amazing, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment