Create a gist now

Instantly share code, notes, and snippets.

@jfeist /nbflatten.jq
Last active Aug 27, 2016

What would you like to do?

Small script to flatten an IPython notebook (much like https://gist.github.com/takluyver/bc8f3275c7d34abb68bf), using jq instead of python. It is significantly faster. Put it somewhere in your path and make it executable. You can call it with nbflatten.jq --arg show_output 0 to suppress showing outputs (by default they are shown).

#!/bin/sh
jq -r 'def banner: "\(.) " + (28-(.|length))*"-";
("Non-cell info" | banner), del(.cells), "",
(.cells[] | ("\(.cell_type) cell" | banner),
"\(.source|add)",
if ($show_output == "1") then
"",
( select(.cell_type=="code" and (.outputs|length)>0) |
("output" | banner),
(.outputs[] |
(select(.text) | "\(.text|add)" | rtrimstr("\n")),
(select(.traceback) | (.traceback|join("\n"))),
(select(.text or .traceback|not) | "(Non-plaintext output)")
),
""
)
else ""
end
)' "$@" --arg show_output 1

I'm currently using the following, called with jq -r -f nbflatten.jq:

def banner: "\(.) " + (28-(.|length))*"-";
# metadata
("Non-cell info" | banner), del(.cells), "",
# content
(.cells[] | (
     ("\(.cell_type) cell" | banner), 
     (.source[] | rtrimstr("\n")), # output source
     if ($show_output == "1") then # the cell output only when it is requested..
       "",
       (select(.cell_type=="code" and (.outputs|length)>0) | (
         ("output" | banner),
         (.outputs[] | (
            (select(.text) | "\(.text|add)" | rtrimstr("\n")),
            (select(.traceback) | (.traceback|join("\n"))),
            (select(.text or .traceback|not) | "(Non-plaintext output)")
           )
         ),
         ""
        )
       )
     else 
       ""
     end
  )
)

Main change:

  • -r will remove the quotes from each line
  • using (.source[] | rtrimstr("\n") instead of "\(.source|add)" will output each line in a codecell on it's own instead of in one long string.

axsk commented Feb 24, 2016

this is amazing, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment