IPython extension for system calls with file like return value. Good for e.g. reading awk output to pandas.

pxmagic.py

def pipex(line):
    "Execute command and return stdout as file like object"
    import subprocess as sp
    cmd = get_ipython().var_expand(line)
    p = sp.Popen(cmd,shell=True,stdout=sp.PIPE)
    return p.stdout

def load_ipython_extension(ipython):
    """This function is called when the extension is loaded.
    It accepts an IPython InteractiveShell instance.
    We can register the magic with the `register_magic_function`
    method of the shell instance."""
    ipython.register_magic_function(pipex, 'line')

Install by downloading the above snippet of code

%install_ext https://gist.githubusercontent.com/kpalin/ec5264445dbb56616d6e/raw/fc4ba1d168fddcd0d15ffa2baef55800b8cba329/pxmagic.py

Set up for current IPython session

%load_ext pxmagic

or for default session by adding it to c.InteractiveShellApp.extensions list.

Usage for e.g. reading awk output to pandas DataFrame

a=%pipex wc -l * |awk '{print $1,$2**2;}'
import pandas as pd
b=pd.read_table(a,header=None,sep=" ",names=["lines","linesSquared"])
b.head()