Debugging Mixed Python/C++ code in Visual Studio Code

Debugging Mixed Python C++ code in Visual Studio Code

I've tested it on Fedora 23 and Ubuntu 16.04. I'm using gcc-5.3.1, python-3.4, VS Code-1.14.0
You can debug mixed Python/C++ in the same GUI. It also works for MPI applications. You can switch between the debuggers and corresponding call stacks.

1. Packages needed
  1) Visual Studio Code

  2) Extensions for VS Code:
      "Python" from Don Jayamanne (I'm using 0.6.7)
        This allows VS Code act as the front end to debug python.
        This gives VS Code ability to attach to a python script that uses module "ptvsd". 
  
      "C/C++" from Microsoft (I'm using 0.12.0)
        This allows VS Code act as the front end for gdb/lldb
  
  3) python module:
      ptvsd-3.0.0
        When use this module in your python script, VS Code can attach to the script.
        The lastest version (3.1.0) doesn't work with the VS Code extension "Python"
        you can install it using pip
  

2. Setting up VS Code for Python/C++ debugging
  1) Senario 1: For non-MPI application
    a) First, in VS Code, launch a python script with python debugger:
      See launch.json
      Start debugger mode "Python launch"

    b) Then, in VS code, attach GDB to the python process:
      See launch.json
      Start debugger mode "GDB Attach proc 0", see Appendix below

  2) Senario 2: For MPI application, attach python debugger and GDB to each process
    a) In the python script your want to run, use ptvsd. See "enable_ptvsd_in_python_script.txt"
      ptvsd allow a debugger from remote machine to attach to a python script running on the local machine
      We are going to run both the python script and the debugger on local machine. So ip address is ip address of the local machine
      For the port number, if you are using multiple MPI process, assign a unused port to each process

    b) Then, in terminal, use mpiexec to launch the python script
      mpiexec -np 2 xterm -e python3 your_python_script.py
      2 MPI processes will be launched. The processes would wait on "ptvsd.wait_for_attach()", I use port number 3000 for process 0, and 3001 for process 1
    
    c) Then in VS Code, attach python debugger to the python process
      See launch.json
      Start debugger mode "Python Attach (local) proc 0". This python debugger will attach to the MPI process 0 
      Start debugger mode "Python Attach (local) proc 1". This python debugger will attach to the MPI process 1
    
    d) Then in VS Code, attach GDB to the python process
      See launch.json
      Start debugger mode "GDB Attach proc 0". VS Code will ask you which process you want to attach to, select the one that is MPI process 0
      Start debugger mode "GDB Attach proc 1". VS Code will ask you which process you want to attach to, select the one that is MPI process 1


You only need to add ptvsd in the top most python script.
If you only want to debug one MPI process, you only need to use ptvsd on that process. 
If you want to debug more MPI processes than 2 in the example, add more debugger configurations in the launch.json file.
Remember to use unique port number for each MPI process you want to debug (in python script and also in launch.json file)

enable_ptvsd_in_python_script.txt

import ptvsd
port=3000
ptvsd.enable_attach(secret='my_secret', address =('127.0.0.1', port))
ptvsd.wait_for_attach()

launch.json

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python launch",
            "type": "python",
            "request": "launch",
            "stopOnEntry": true,
            "pythonPath": "/usr/bin/python3",
            "program": "${workspaceRoot}/your_python_script.py",
            "cwd": "",
            "console": "externalTerminal",
            "env": {},
            "envFile": "${workspaceRoot}/.env",
            "debugOptions": [
                "WaitOnAbnormalExit",
                "WaitOnNormalExit"
            ]
        },
        {
            "name": "Python Attach (local) proc 0",
            "type": "python",
            "request": "attach",
            "localRoot": "${workspaceRoot}",
            "remoteRoot": "${workspaceRoot}",
            "port": 3000,
            "secret": "my_secret",
            "host": "localhost"
        },
        {
            "name": "Python Attach (local) proc 1",
            "type": "python",
            "request": "attach",
            "localRoot": "${workspaceRoot}",
            "remoteRoot": "${workspaceRoot}",
            "port": 3001,
            "secret": "my_secret",
            "host": "localhost"
        },
        { 
            "name": "GDB Attach proc 0",
            "type": "cppdbg",
            "request": "attach",
            "program": "/usr/bin/python3",
            "processId": "${command:pickProcess}",
            "MIMode": "gdb"
        },
        { 
            "name": "GDB Attach proc 1",
            "type": "cppdbg",
            "request": "attach",
            "program": "/usr/bin/python3",
            "processId": "${command:pickProcess}",
            "MIMode": "gdb"
        }
    ]
}

I have a python code calling C++ function, I tried following your instructions, however I get this error :
`E-0000.520: Exception escaped from start_client

        Traceback (most recent call last):
          File "/Users/sujithshivaprakash/.vscode/extensions/ms-python.python-2019.3.6139/pythonFiles/lib/python/ptvsd/log.py", line 110, in g
            return f(*args, **kwargs)
          File "/Users/sujithshivaprakash/.vscode/extensions/ms-python.python-2019.3.6139/pythonFiles/lib/python/ptvsd/pydevd_hooks.py", line 74, in start_client
            sock, start_session = daemon.start_client((host, port))
          File "/Users/sujithshivaprakash/.vscode/extensions/ms-python.python-2019.3.6139/pythonFiles/lib/python/ptvsd/daemon.py", line 214, in start_client
            with self.started():
          File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
            return self.gen.next()
          File "/Users/sujithshivaprakash/.vscode/extensions/ms-python.python-2019.3.6139/pythonFiles/lib/python/ptvsd/daemon.py", line 110, in started
            self.start()
          File "/Users/sujithshivaprakash/.vscode/extensions/ms-python.python-2019.3.6139/pythonFiles/lib/python/ptvsd/daemon.py", line 145, in start
            raise RuntimeError('already started')
        RuntimeError: already started`

My c/c++ debugger has been attached but was not stopping at the breakpoints at all. Then I realized that I did not have a .dSYM file for the shared library that I am debugging.

Now it all works. thanks for this gist

Hi @asroy , concerning Scenario 1, do you think it is possible to make the process more 'automatic'? I mean, knowing that the Python debugger spawned the process, the C++ debugger might find out the PID without human action, no?

In case you are using containers as I did just add --cap-add sys_ptrace capbility while you give run command. Details in below post.

https://stackoverflow.com/questions/45171339/gdb-cannot-attach-to-process/45171694#45171694

@asroy Attach to Python process does not work. Any known issues on MacOS? (Catalina; VSCode 1.41.1; Anaconda Python 3.6.8; ptvsd 4.3.2)

As far as I'm concernd, vscode can't launch two debug configuration at the same time.

1. Senario 1: For non-MPI application
   a) First, in VS Code, launch a python script with python debugger:
   See launch.json
   Start debugger mode "Python launch"

b) Then, in VS code, attach GDB to the python process:
  See launch.json
  Start debugger mode "GDB Attach proc 0", see Appendix below

Are these two configuration launch in one vscode window?

@BrightXiaoHan yes, both debuggers coexist and work simultaneously in the same vscode window. You have a selector for debuggers. Once the Python debugger is stopped at some breakpoint you can use the selector to choose the C++ one and launch it.

Hi @asroy,

When I try to attach the gdb to python, I need to specify the number process but when I select it, the terminal writes:

Python Exception <class 'ImportError'> This package should not be accessible on Python 3. Either you are trying to run from the python-future src folder or your installation of python-future is corrupted.:
/usr/bin/gdb: warning:
Could not load the Python gdb module from `/usr/share/gdb/python'.
Limited Python support is available from the _gdb module.
Suggest passing --data-directory=/path/to/gdb/data-directory.

What I do wrong? Help is very much appreciated.