code_profiling.md

PyInstrument

from pyinstrument import Profiler
profiler = Profiler()

profiler.start()

# --- code to profile : start ---
# e.g. :-
# np.random.randint(1, 7, 1000)
# --- code to profile : end ---

profiler.stop()

# --- view profiling result ---
print(profiler.output_text(unicode=True, color=True, show_all=True))

cProfiler

import cProfile
import pstats

command_to_profile = '<cmd to run>'
dump_file = '</path/to/dump/file>'

cProfile.run(command_to_profile, dump_file)
p = pstats.Stats(dump_file)
p.sort_stats('cumulative').print_stats(50)

Python

• pdb : The Python Debugger
• cProfiler : Get profile for running time of various functions
• KCacheGrind : Generate call graph of functions
• valgrind : Profile memory usage by program
• gperftools : Set of tools for performance profiling and memory checking. Generates a call graph output
• pylint : Source code syntax & linting checker
• pyreverse : Creates UML diagrams for python code. Included in pylint
• pyflakes : Similar to pylint

C++, Python/C extension, Python C API

• gdb
• gprof
• pprof
• KCacheGrind
• yep

Links

• Python tools : https://wiki.python.org/moin/PythonTestingToolsTaxonomy
• C++ tools : http://www.programmerworld.net/resources/c_plus/c_tool.php
• gprof, valgrind, gperftools : http://gernotklingler.com/blog/gprof-valgrind-gperftools-evaluation-tools-application-level-cpu-profiling-linux/
• GDB docs : https://sourceware.org/gdb/onlinedocs/gdb/index.html
• GDB demo : https://stripe.com/blog/exploring-python-using-gdb
• On gperftools : https://gperftools.github.io/gperftools/cpuprofile.html