Skip to content

Instantly share code, notes, and snippets.

@myselfhimself

myselfhimself/cpython_gdb_debugging.md

Last active October 30, 2023 09:31
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save myselfhimself/aa25e69f1f07b74795112e2574dfc0bd to your computer and use it in GitHub Desktop.
Save myselfhimself/aa25e69f1f07b74795112e2574dfc0bd to your computer and use it in GitHub Desktop.
Download ZIP
Debugging a C Python extension with GDB and CLion
Raw
cpython_gdb_debugging.md

Assuming that you are writing a compiled Python module (ie. that compiles into a .dll .so or .dynlib..) in C/C++, here are tips to setup real-time debugging, both with your own C/C++ source code and within Python's C source.

Instructions here are for Linux only. You should find your way for any different OS..

Debugging your own C/C++ code

  • have your compiled module (with no optimization -O0 and debug symbols -g) is available in your current (virtual environment):
    • for this, make a wheel for your project and load it in your virtual environment
    • OR compile your project as a .so only and make sure you set LD_LIBRARY_PATH= to the directory where your .so is located
  • test your proper module loading:
    • run import <your module name>
    • look at <your module name>.__spec__ to understand which .so file was loaded if this variable is available

Debugging with gdb in command-line

  • run gdb python
  • in the GDB shell, you can then run r if you want the regular Python shell, or r somefilename.py to execute a script
  • you can set breakpoints from there, using gdb's commands eg. b mymodulefile.cpp:29
  • normally, you module .so file should have .c/.cpp file paths info for your own module source.. if not try to use the directory <your-module-source-dir> first
  • once a breakpoint has been reached a crash happens, you can use c for continuing, bt for backtrace investigation, l and ll for displaying where you are, n for next line, s for step, up, down, p someCvariable for variable printing or evaluation etc

Debugging with CLion

  • run python in a shell, note that your python executable does not need to be compiled with debug symbols, only your module
  • go to Run > Attach to process and try to find Python process with a very fresh/high PID, either at the very top suggestion list, or at the very bottom.
    • for the very first time, you may need to go first at the very bottom and click the > on the process you would like to attach to, and choose GDB instead of LLDB
    • from then on, CLion will pick GDB for you as a connecting method

More details for this "Attach to process" feature on CLion's help pages.

Debugging Python core source code

Imagine you have an ugly segfault when the Python terminal shuts down, possibly because of refcount Py_INCREF/Py_DECREF in your own module's code... but your module is silent.. To investigate this, despite having the above setup in place and working, you will just see adresses and function names of Python's interpreter, but no way to see which line in Python's core source brought the crash to you, frame by frame.

Using pure GDB

The solution to also do step-by-step debugging is to provide Python's source for your python-compiled-with-debug-symbols executable to GDB. This article by Alex Dzyoba shows the idea and you should read it till step "2. Change GDB source path". It boils down to:

  1. see if your python executable does have debug symbols using objdump -h and objdump -g (if not you could sudo apt-get install python3.7-dbg or something like this depending on your distribution)
  2. if you miss debug symbols but have installed or compiled yourself a python executable with debug symbols, use it and if you have virtual environments management, use it for your virtual environment, eg. mkvirtualenv python-dbg-module-debug -p /usr/bin/python3.7-dbg)
  3. git clone or download a proper version of Python's source and get the full path of the build/ directory inside it (readlink -f <somerelativepath> or pwd are your friends)
  4. inside the GDB shell (for gdb <your python dbg executable>) type directory <the full path of the build directory from previous step
  5. make things run and see that backtraces (bt command) now display your Python's C source path!!!

Here is what I obtain for debugging the G'MIC Python with Python 3.7.5-dbg using a Python-3.7.4 source clone:

(numpy291-dbg) :~/Productions/GMIC/gmic-py$ 
$ gdb python
GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...done.
(gdb) directory /export/home/AAA/Productions/Python-3.7.4/build
Source directories searched: /export/home/AAA/Productions/Python-3.7.4/build:$cdir:$cwd
(gdb) r
Starting program: /export/home/AAA/.virtualenvs/numpy291-dbg/bin/python 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
/export/home/AAA/.virtualenvs/numpy291-dbg/lib/python3.7/site.py:165: DeprecationWarning: 'U' mode is deprecated
  f = open(fullname, "rU")
Python 3.7.5 (default, Nov  7 2019, 10:50:52) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gmic
[New Thread 0x7fffce7a0700 (LWP 29172)]
[New Thread 0x7fffcbf9f700 (LWP 29173)]
[New Thread 0x7fffc979e700 (LWP 29174)]
[New Thread 0x7fffc8f9d700 (LWP 29175)]
[New Thread 0x7fffc479c700 (LWP 29176)]
[New Thread 0x7fffc3f9b700 (LWP 29177)]
[New Thread 0x7fffbf79a700 (LWP 29178)]
[New Thread 0x7fffbcf99700 (LWP 29179)]
[New Thread 0x7fffba798700 (LWP 29180)]
[New Thread 0x7fffb7f97700 (LWP 29181)]
[New Thread 0x7fffb7796700 (LWP 29182)]
[New Thread 0x7fffb2f95700 (LWP 29183)]
>>> import gmic
>>> import PIL.Image
>>> PIL_apples_filename = "PIL_apples.png"
>>> gmic.run("sp apples output " + PIL_apples_filename)
[New Thread 0x7fffb137f700 (LWP 29202)]
[New Thread 0x7fffb0b7e700 (LWP 29203)]
[New Thread 0x7fffb037d700 (LWP 29204)]
[New Thread 0x7fffafb7c700 (LWP 29205)]
[New Thread 0x7fffaf37b700 (LWP 29206)]
[New Thread 0x7fffaeb7a700 (LWP 29207)]
[New Thread 0x7fffae379700 (LWP 29208)]
[New Thread 0x7fffadb78700 (LWP 29209)]
[New Thread 0x7fffad377700 (LWP 29210)]
[New Thread 0x7fffacb76700 (LWP 29211)]
[New Thread 0x7fffac375700 (LWP 29212)]
>>> PIL_apples = PIL.Image.open(PIL_apples_filename)
>>> apples = gmic.GmicImage.from_PIL(PIL_apples)
>>> gmic.run("display", apples)
[gmic]-1./ Display image [0] = '[unnamed]', from point (320,200,0).
[0] = '[unnamed]':
  size = (640,400,1,3) [3000 Kio of floats].
  data = (20,22,20,20,20,22,22,22,22,22,22,20,(...),1,1,1,1,1,1,1,1,1,1,1,1).
  min = 1, max = 250, mean = 58.5602, std = 59.8916, coords_min = (317,306,0,1), coords_max = (430,135,0,0).
[New Thread 0x7fffa8e75700 (LWP 29213)]
>>> 

Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x0000000000534a6a in visit_decref (op=0x7fffb21c37d0, data=0x0) at ../Modules/gcmodule.c:271
271	    if (PyObject_IS_GC(op)) {
(gdb) bt
#0  0x0000000000534a6a in visit_decref (op=0x7fffb21c37d0, data=0x0) at ../Modules/gcmodule.c:271
#1  0x0000000000458835 in dict_traverse (op=0x7ffff7e98ae0, visit=0x534a5d <visit_decref>, arg=0x0) at ../Objects/dictobject.c:2987
#2  0x000000000053467e in subtract_refs (containers=containers@entry=0xacf540 <_PyRuntime+448>) at ../Modules/gcmodule.c:296
#3  0x0000000000535971 in collect (generation=generation@entry=2, n_collected=n_collected@entry=0x7fffffffd520, n_uncollectable=n_uncollectable@entry=0x7fffffffd528, nofail=nofail@entry=0) at ../Modules/gcmodule.c:853
#4  0x0000000000535e75 in collect_with_callback (generation=generation@entry=2) at ../Modules/gcmodule.c:1028
#5  0x0000000000536445 in PyGC_Collect () at ../Modules/gcmodule.c:1573
#6  0x0000000000536480 in _PyGC_CollectIfEnabled () at ../Modules/gcmodule.c:1587
#7  0x0000000000511f24 in Py_FinalizeEx () at ../Python/pylifecycle.c:1185
#8  0x0000000000429409 in pymain_main (pymain=pymain@entry=0x7fffffffd5d0) at ../Modules/main.c:3030
#9  0x00000000004294e7 in _Py_UnixMain (argc=<optimized out>, argv=<optimized out>) at ../Modules/main.c:3063
#10 0x0000000000422d1b in main (argc=<optimized out>, argv=<optimized out>) at ../Programs/python.c:15
(gdb)

Using CLion

  • a shorthand for the pure GDB way of seeing Python's source instead of CLion showing the disassembled code of Python is to
    1. set a breakpoint early in your code
    2. attach to process and run python process to debug (no need to use gdb there, CLion will use its own gdb)
    3. once you hit the early breakpoint, in your debug panel, go into the GDB shell tab next the Variables tab
    4. in this GDB shell, type directory /export/home/AAA/Productions/Python-3.7.4/build or a Python C source path that fits you
    5. hit the play button so that debugging continues, now CLion will show you proper paths and C source code from Python's core!!

:)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment