Create a gist now

Instantly share code, notes, and snippets.

@jfeist /nbflatten.jq
Last active Dec 20, 2017

Embed

Small script to flatten an IPython notebook (much like https://gist.github.com/takluyver/bc8f3275c7d34abb68bf), using jq instead of python. It is significantly faster. Put it somewhere in your path and make it executable. You can call it with nbflatten.jq --arg show_output 0 to suppress showing outputs (by default they are shown).

#!/bin/sh
jq -r 'def banner: "\(.) " + (28-(.|length))*"-";
("Non-cell info" | banner), del(.cells), "",
(.cells[] | ("\(.cell_type) cell" | banner),
"\(.source|add)",
if ($show_output == "1") then
"",
( select(.cell_type=="code" and (.outputs|length)>0) |
("output" | banner),
(.outputs[] |
(select(.text) | "\(.text|add)" | rtrimstr("\n")),
(select(.traceback) | (.traceback|join("\n"))),
(select(.text or .traceback|not) | "(Non-plaintext output)")
),
""
)
else ""
end
)' "$@" --arg show_output 1
@jankatins

This comment has been minimized.

Show comment
Hide comment
@jankatins

jankatins Jan 26, 2016

I'm currently using the following, called with jq -r -f nbflatten.jq:

def banner: "\(.) " + (28-(.|length))*"-";
# metadata
("Non-cell info" | banner), del(.cells), "",
# content
(.cells[] | (
     ("\(.cell_type) cell" | banner), 
     (.source[] | rtrimstr("\n")), # output source
     if ($show_output == "1") then # the cell output only when it is requested..
       "",
       (select(.cell_type=="code" and (.outputs|length)>0) | (
         ("output" | banner),
         (.outputs[] | (
            (select(.text) | "\(.text|add)" | rtrimstr("\n")),
            (select(.traceback) | (.traceback|join("\n"))),
            (select(.text or .traceback|not) | "(Non-plaintext output)")
           )
         ),
         ""
        )
       )
     else 
       ""
     end
  )
)

Main change:

  • -r will remove the quotes from each line
  • using (.source[] | rtrimstr("\n") instead of "\(.source|add)" will output each line in a codecell on it's own instead of in one long string.

I'm currently using the following, called with jq -r -f nbflatten.jq:

def banner: "\(.) " + (28-(.|length))*"-";
# metadata
("Non-cell info" | banner), del(.cells), "",
# content
(.cells[] | (
     ("\(.cell_type) cell" | banner), 
     (.source[] | rtrimstr("\n")), # output source
     if ($show_output == "1") then # the cell output only when it is requested..
       "",
       (select(.cell_type=="code" and (.outputs|length)>0) | (
         ("output" | banner),
         (.outputs[] | (
            (select(.text) | "\(.text|add)" | rtrimstr("\n")),
            (select(.traceback) | (.traceback|join("\n"))),
            (select(.text or .traceback|not) | "(Non-plaintext output)")
           )
         ),
         ""
        )
       )
     else 
       ""
     end
  )
)

Main change:

  • -r will remove the quotes from each line
  • using (.source[] | rtrimstr("\n") instead of "\(.source|add)" will output each line in a codecell on it's own instead of in one long string.
@axsk

This comment has been minimized.

Show comment
Hide comment
@axsk

axsk Feb 24, 2016

this is amazing, thank you!

axsk commented Feb 24, 2016

this is amazing, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment