Debugging Django apps running in Docker using ptvsd - Visual Studio (Code)

docker_debugging.md

Remote debugging in Docker (for Django apps)

In order to enable debugging for your Django app running in a Docker container, follow these steps using Visual Studio (Code):

1. Add ptvsd to your requirements.txt file

ptvsd == 4.3.2

2. To your launch.json, add this:

  {
      "name": "Remote Django App",
      "type": "python",
      "request": "attach",
      "pathMappings": [
          {
              "localRoot": "${workspaceFolder}",
              "remoteRoot": "/remote_root/of/your/app"
          }
      ],
      "port": 3000,
      "host": "localhost"
  }

(Edit the remoteRoot option to reflect your app).

3. To your manage.py, add this:

  if __name__ == "__main__":                                                    # This already exists
      os.environ.setdefault("DJANGO_SETTINGS_MODULE", "your_project.settings")  # This already exists

      from django.core.management import execute_from_command_line              # This already exists
      from django.conf import settings

      if settings.DEBUG:
          if os.environ.get('RUN_MAIN') or os.environ.get('WERKZEUG_RUN_MAIN'):
              import ptvsd
              ptvsd.enable_attach(address = ('0.0.0.0', 3000))
              print "Attached remote debugger"

      execute_from_command_line(sys.argv)                                         # This already exists

Note: The third if statement here ensures the debugger does not get attached again after a live reload.

4. Be sure to open port 3000 in your docker command or docker-compose.yml

5. Run your app:

  python manage.py runserver 0.0.0.0:8000

Line-by-line debugging

Note: In some (non-Django) cases line-by-line debugging does not work, unless you use double backslashes (\) in your remoteRoot parameter (Viscual Studio Code), even though the remote server runs on Linux. E.g. "remoteRoot": "\\remote_root\\of\\your\\app"

Gotchas

• When running a (non-Django) app using docker-compose run instead of docker-compose up, you need to add the additional --service-ports flag to open the ports defined in your docker-compose.yml:

docker-compose run --service-ports your_app

Otherwise, you will see a Connection refused error when trying to attach, as the required debug port (usually 3000) is not opened.

Shoutout

Shoutout to this exellent blog post describing how you can keep live reload turned on while remote debugging. (Previously we had to disable Django's live reload function because it would attach the debugger twice, thus throwing an error). We really can have our cake and eat it too.

Note: Be sure to use the latest version of ptvsd, as some old versions will throw an exceptions.SystemExit when you are attached from VS Code and the server gets reloaded. This happened to me using `ptvsd == 4.1.3.

Update 2019-08-22: Outdated issue/solution.

There have been some problems reported regarding remote debugging from Visual Studio Code using ptvsd > 3.0.0.
The issue mentioned most often is that VSC 'just hangs' when trying to connect to your remote debugger.

For now, use ptvsd 3.0.0:
pip install ptvsd==3.0.0

Another problem can be caused by using ptvsd.wait_for_attach().
This can lead to the same problem as described above, where VSC just hangs while connecting to the remote debugger.

Try replacing wait_for_attach() with a simple time.sleep(10) and connect within the 10 seconds.

Yet another problem can occur when using forward slashes in your launch.json.

E.g. change "remoteRoot": "/remote_root/of/your/app",
into "remoteRoot": "\\remote_root\of\your\app",

Thanks a lot for sharing @veuncent , it really helped me figure out how to debug my django container within vscode

Here's what worked for me:

1. add ptvsd == 4.1.4 to your requirements.txt file

2. add the following inside your manage.py file (as suggeste in above gist)

#!/usr/bin/env python
import os
import sys


if __name__ == "__main__":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "api.settings")

    if os.environ.get("DJANGO_DEBUGGER", False):
        import ptvsd

        ptvsd.enable_attach(address=("0.0.0.0", 3000))

    from django.core.management import execute_from_command_line

    execute_from_command_line(sys.argv)

3. install vscode extension ms-python.python
   

4. add the following configuration inside your launch.json file:

"configurations": [
    {
        // requires vscode extension: ms-python.python
        "name": "runserver",
        "type": "python",
        "request": "attach",
        "pathMappings": [
            {
                "localRoot": "${workspaceFolder}",
                "remoteRoot": "/home"
            }
        ],
        "port": 3000,
        "host": "localhost",
    }
]

5. execute the following in your terminal:

docker-compose run -p 3000:3000 -p 8080:8001 --rm web sh -c 'DJANGO_DEBUGGER=True python api/manage.py runserver --noreload --nothreading 0.0.0.0:8001'

6. once your server is running, start debugging by selecting runserver launch configuration in vscode's debug menu
   

7. that's it, you are debugging in vscode !
   

Thanks @lebnic, that was helpful too. The approach I've settled with is calling python -m ptvsd from the command line instead of embedding it in manage.py.

My tasks.json looks like this (note the sleep 5 at the end of the command, this is needed to allow the debugger time to attach):

{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "start_container",
            "type": "shell",
            "command": "docker-compose run -d -p 3000:3000 -p 8000:8000 --rm web bash -c 'python -m ptvsd --host 0.0.0.0 --port 3000 --wait manage.py runserver --noreload --nothreading 0.0.0.0:8000' && sleep 5"
        },
        {
            "label": "stop_container",
            "type": "shell",
            "command": "docker rm -f $(docker ps -a -q --filter ancestor=web --format='{{.ID}}')",
        }
    ]
}

@christippett I'll also need to implement a similar waiting in order to execute my unit tests; Your comment will surely help me, thanks :-)

thanks a lot's. my debuger is worked.
btw,I use python manage.py runserver 0.0.0.0:8000 make debuger connected,but add option --noreload --nothreading not work

Hi @callect , what is the error you get when running with --noreload --nothreading? One of the problems you may run into without those settings is that, when you edit your python code, Django will reload the development server and run ptvsd.enable_attach() (in your manage.py) again, which could throw an error.

hi,@veuncent
I'm try add those options again, it's worked from now on.
I don't know why it doesn't work, and can not refresh page at yesterday. I'm sorry that I can't reproduce right now :(

No worries @callect! Glad it is solved.

@veuncent thanks

I updated this gist to reflect some really useful suggestions from this blog post, enabling us to keep live reload on while remote debugging. I also edited step 1 where we add ptvsd to our requirements.txt, as suggested bij @lebnic

Thanks, @veuncent! Can confirm, current gist works perfectly as-is.

I had issues getting this to work when using the ${workspaceFolder}, but it started working perfectly when I hardcoded the path to the project.

Also, when running with the --nothreading argument, everything is quite slow, so is there any way to debug without that argument?

Hi @Moulde , if you follow the latest version of this gist, you don't need the --nothreading and --noreload arguments. The third if statement in step 3 makes that possible (also see the shout-out paragraph at the end). I hope this helps!

@veuncent Ah thanks! Completely missed that :)

Hello @veuncent ! The above didn't work in my case. However I found out an alternative. Please take a look at this solution. And it works as a charm as well!

Hi @drigio , thanks for posting the alternative solution! Was there anything in particular that didn't work for you in my solution? Explaining that might help others with the same problem. Thanks!

Sure @veuncent ! Actually the project that I am working right now is kind of a large Django app, in which they had a wrapper around ./manage.py. So that wrapper used to call many sub-processes (so that they can keep a track of their instantiations to destroy them at a later point) among which one was ./manage.py. So while it used to call the attach_debug more than once (even with your code above don't understand why) hence giving OSError. It was one of the sub-processes and I think the breakpoints were not hitting because the request used to go to other threads (Maybe debugger was getting attached to the Django's Watcher process instead of application one ?).

I have the exact same problem as @drigio. However no success debugging that so far...

Update:
enabling checks with:
os.environ.get("RUN_MAIN") or os.environ.get("WERKZEUG_RUN_MAIN")
seems to fix this problem.

However now im getting error ECONNREFUSED from visual studio

@an0o0nym Did you exposed right port in docker-compose ?

@matiuszka I am not quite to sure at this point whether I was exposing the right port at the time when I posted my comment.

However now its working. I am not sure what I did exactly to make it work, because I tried many many things. Although I post few steps that I did along the way to make it work that hopefuly can help in some way or another.

1. I remember reading on SO about requirement to have the exact same version ptvsd installed on your machine (make sure its not on virtualenv level but rather system user level) as well as inside docker container.

2. Another thing was to add inside my manage.py try-catch block to prevent ptvsd from throwing error. Sample code may look like :

#!/usr/bin/env python3
import os
import sys

if __name__ == "__main__":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "your_project_name.settings")

    if (
        os.environ.get("RUN_MAIN") or os.environ.get("WERKZEUG_RUN_MAIN")
    ) and os.environ.get("VSCODE_DEBUGGER", False):
        import ptvsd

        ptvsd_port = os.environ.get("PORT_PTVSD", 5678)

        try:
            ptvsd.enable_attach(address=("0.0.0.0", ptvsd_port))
            print("Started ptvsd at port %s." % ptvsd_port)
        except OSError:
            print("ptvsd port %s already in use." % ptvsd_port)
    from django.core.management import execute_from_command_line

    execute_from_command_line(sys.argv)

Just for reference my docker-compose.debug.yml which overrides specific items from a regular docker-compose.yml looks like:

version: '2'

services:
  web:
    expose:
      - "5678"
    ports:
      - 8000:8000
      - 5678:5678
    environment: 
      - VSCODE_DEBUGGER=True

3. Make sure you run docker-compose -f docker-compose.yml -f docker-compose.debug.yml up web to apply those overrides :)

I can make ptvsd work using manage.py calls and command line python -m ptvsd --host 0.0.0.0 --port 5678 manage.py runserver --noreload but when i try and use either within context of docker container using ubuntu:18.04 the 5678 port listener isn't created.

If i docker -it exec django_webservice bin/bash into the container and do a pip list i see ptvsd and if i do a netstat -tunlp it shows nginx web server tcp/80 listener but not the ptvsd tcp/5678 port so it doesn't matter that i exposed port in "docker run -p 8000:80 -p 5678:5678 . . . " command it seems something about initializing ptvsd from within that environment isn't working. i tried adding Dockerfile EXPOSE 5678 in addition to EXPOSE 80 that was already there and that makes no difference and docs suggest EXPOSE is optional documentation syntax only.

Any thoughts on how to debug "docker run . . . " to determine what's causing failure?