Profiling Python 3 Code with cProfile and CProfileV

ProfilingPython3CodeWithcProfieAndCProfileV.md

First, profile your code with cProfile:

$ python3 -m cProfile -o profile_output search.py

But output of that is not very helpful. To make sense of the cProfiler's output install CProfileV:

# install cprofilev
$ pip3 install cprofilev

# convert it to py3k
2to3 -w /path/to/python3.4/site-packages/cprofilev.py

# call it with your cprofile output
$ cprofilev /path/to/cprofile/output

# navigate to http://localhost:4000

And here's how it looks in browser (it is actually sortable by different columns AND you can drill down the function stack):

![Image of CProfileV in action] (http://i.imgur.com/II7SyxC.png)

Here is an excellent article on profiling from CProfileV developer himself: http://ymichael.com/2014/03/08/profiling-python-with-cprofile.html

I actually created a bash function for profiling python scripts. On OSX this function should be added to ~/.bash_profile, but it should work on other OSes as well.

# profile python script
pyprofile() {
    # check if python 3 is installed
    if ! hash python3 2>/dev/null; then
        printf "\n"
        printf "ERROR: Python 3 is not installed or is not part of \$PATH\n"
        printf "\n"
        
        return 1
    fi
    
    # check if cprofilev is installed
    HAS_CPROFILEV=$(python3 -c "import importlib.util;loader=importlib.util.find_spec('cprofilev');print(loader is not None)")
    if [ $HAS_CPROFILEV == "False" ]; then
        printf "\n"
        printf "ERROR: CProfileV is not installed. Please install it first (e.g., pip3 install cprofilev)\n"
        
        # check for virtual environments
        if [ -z "$VIRTUALENV_PYTHON" ] || [ -z "$VIRTUALENVWRAPPER_PYTHON" ]; then
            printf "\n"
            printf "Additionaly, no virtual Python environments where found. Are you working inside one?\n"
        fi
        
        printf "\n"
        
        return 1
    fi
    
    # check for script name
    if [ -z "$1" ]; then
        printf "\n"
        printf "ERROR: Please specify python script to profile\n"
        printf "\n"
        
        return 1
    fi
    
    # everything is ok
    printf "Running profiler... "
    PROFILER_OUTPUT=$(python3 -m cProfile -o cprofile_output.bin $1)
    if [ -z "$PROFILER_OUTPUT" ]; then
        printf "OK\n"
    fi  
    
    # run CProfileV
    printf "CTRL+C to quit, "
    cprofilev cprofile_output.bin
    
    # cleanup
    rm -f cprofile_output.bin
    
    return 0
}

After that run pyprofile <script_name> and everything magically works :D