Skip to content

Instantly share code, notes, and snippets.

@EtsuNDmA
Last active April 4, 2024 17:54
Show Gist options
  • Star 22 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save EtsuNDmA/dd8949061783bf593706559374c8f635 to your computer and use it in GitHub Desktop.
Save EtsuNDmA/dd8949061783bf593706559374c8f635 to your computer and use it in GitHub Desktop.
Two ways to integrate Django with Jupyter (JupyterLab)

⚠️ This gist was the first time written in 2018. It was tested with django==2.2.x. It's likely to be relevant for the newer versions of django and jupyter, but I did't check that.

2023-03-15: checked it for python == 3.11, django == 4.1.7, jupyterlab = 3.6.1, django-extensions == 3.2.1, jupyter-server <= 2.0.0

Django_jupyter

Using Django project in Jupyter or JupyterLab

Method 1 (using shell_plus)

We assume that django and jupyter are already installed

  1. Install django-extensions

    pip install django-extensions
  2. Add django-extensions app into Django settings

    INSTALLED_APPS += ['django_extensions']
  3. Add jupyter setting into Django settings

    NOTEBOOK_ARGUMENTS = [
        '--ip', '0.0.0.0',
        '--port', '8888',
    ]
    IPYTHON_KERNEL_DISPLAY_NAME = 'Django Kernel'

    All params you can find in jupyter help

    jupyter --help-all
  4. Install JupyterLab (optional). To start JupyterLab by default add this to Django settings

    try:
        import jupyterlab
        NOTEBOOK_DEFAULT_URL = '/lab'  # Using JupyterLab
    except ImportError:
        NOTEBOOK_DEFAULT_URL = '/tree'  # Using Jupyter
    
    NOTEBOOK_DIR = BASE_DIR / "notebooks"
    
    NOTEBOOK_ARGUMENTS = [
        '--ip', '0.0.0.0',
        '--port', '8888',
        '--notebook-dir', NOTEBOOK_DIR,
        '--NotebookApp.default_url', NOTEBOOK_DEFAULT_URL,
    ]
    IPYTHON_KERNEL_DISPLAY_NAME = 'Django Kernel'
    
    # if you want to use Chrome by default
    # os.environ.setdefault('BROWSER', 'google-chrome')
  5. Run jupyter

    PYTHONPATH=$(pwd):$PYTHONPATH python manage.py shell_plus --notebook

    for notebook>=7.0.0 (django-extensions/django-extensions#1835)

    PYTHONPATH=$(pwd):$PYTHONPATH python manage.py shell_plus --lab

    Browser will be opened at http://[ip]:[port]/tree

  6. Create new notebook using Django Kernel

    We can see that all Django models are already imported

    locals()

    WARNING Model names collisions will be resolved by (collision resolver)

  7. Be careful with async. Read https://ipython.readthedocs.io/en/stable/interactive/autoawait.html#difference-between-terminal-ipython-and-ipykernel and https://docs.djangoproject.com/en/4.1/topics/async/#async-safety

    TL;DR

    os.environ["DJANGO_ALLOW_ASYNC_UNSAFE"] = "true"

Method 2

In this case you don't need to install django-extensions. However it is less convenient

Initialize Django in the first cell of each notebook

import os, sys
import django
PROJECTPATH = '/my/django/project/path'
sys.path.insert(0, PROJECTPATH)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my.django.settings.module")
os.environ["DJANGO_ALLOW_ASYNC_UNSAFE"] = "true"  # https://docs.djangoproject.com/en/4.1/topics/async/#async-safety
os.chdir(PROJECTPATH)
django.setup()

Than you can import models

from myapp.models import Vehicle
Vehicle.objects.count()
@x0lani
Copy link

x0lani commented Feb 28, 2023

Method 2 throws an async error when accessing a Django model:
**SynchronousOnlyOperation:** You cannot call this from an async context - use a thread or sync_to_async.

Looking at the error stack it looks like you can set the environment variable DJANGO_ALLOW_ASYNC_UNSAFE but that looks unsafe...

@x0lani
Copy link

x0lani commented Feb 28, 2023

Modifying your snippet for Method 2 with the following prevents the SynchronousOnlyOperation error:

import os, sys
import django
PROJECTPATH = '/my/django/project/path'
sys.path.insert(0, PROJECTPATH)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my.django.settings.module")
os.environ["DJANGO_ALLOW_ASYNC_UNSAFE"] = "true"
os.chdir(PROJECTPATH)
django.setup()

@EtsuNDmA
Copy link
Author

@x0lani thanks! It's good to know that this gist is still relevant even after 5 years ☺️

@flo-85
Copy link

flo-85 commented Aug 24, 2023

@x0lani @EtsuNDmA thanks for sharing this solution.

Method 2 works for me.

Method 1 returns this:

ModuleNotFoundError: No module named 'notebook.notebookapp'
CommandError: Could not load shell runner: 'IPython Notebook'.

I am using django 4.2 abd python 3.10.
I am using Windows WSL (Linux subsystem on a windows machine)
Jupyter notebook is installed. not sure why method 1 is failing in my case.

This works:

./manage.py shell_plus --ipython

But not this:

./manage.py shell_plus --notebook

@EtsuNDmA
Copy link
Author

Hi @flo-85! Please ensure that you have jupyterlab installed http://jupyter.org/install

@jozekuhar
Copy link

Hey @flo-85, i had the same problem. I installed different version of notebook (not 7+) and it helped.

notebook==6.4.13

@azegas
Copy link

azegas commented Mar 5, 2024

Hi @flo-85! Please ensure that you have jupyterlab installed http://jupyter.org/install

I am getting the same error message even with jupyterlab installed:

(shipping) C:\Users\XXX\Desktop\GIT\shipping\code\backend\app>pip show ipython
Name: ipython
Version: 8.22.2
Summary: IPython: Productive Interactive Computing
Home-page:
Author: The IPython Development Team
Author-email: ipython-dev@python.org
License: BSD-3-Clause
Location: c:\users\XXX\envs\shipping\lib\site-packages
Requires: colorama, decorator, exceptiongroup, jedi, matplotlib-inline, prompt-toolkit, pygments, stack-data, traitlets
Required-by: ipykernel, ipywidgets, jupyter-console

(shipping) C:\Users\XXX\Desktop\GIT\shipping\code\backend\app>pip show jupyter
Name: jupyter
Version: 1.0.0
Summary: Jupyter metapackage. Install all the Jupyter components in one go.
Home-page: http://jupyter.org
Author: Jupyter Development Team
Author-email: jupyter@googlegroups.org
License: BSD
Location: c:\users\XXX\envs\shipping\lib\site-packages
Requires: ipykernel, ipywidgets, jupyter-console, nbconvert, notebook, qtconsole
Required-by:

(shipping) C:\Users\XXX\Desktop\GIT\shipping\code\backend\app>pip show notebook
Name: notebook
Version: 7.1.1
Summary: Jupyter Notebook - A web-based notebook environment for interactive computing
Home-page: 
Author: 
Author-email: Jupyter Development Team <jupyter@googlegroups.com>
License: BSD 3-Clause License
        
        - Copyright (c) 2001-2015, IPython Development Team
        - Copyright (c) 2015-, Jupyter Development Team
        
        All rights reserved.
        
        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions are met:

        1. Redistributions of source code must retain the above copyright notice, this
           list of conditions and the following disclaimer.

        2. Redistributions in binary form must reproduce the above copyright notice,
           this list of conditions and the following disclaimer in the documentation
           and/or other materials provided with the distribution.

        3. Neither the name of the copyright holder nor the names of its
           contributors may be used to endorse or promote products derived from
           this software without specific prior written permission.

        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
        AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
        IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
        DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
        FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
        DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
        SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
        CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
        OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Location: c:\users\XXX\envs\shipping\lib\site-packages
Requires: jupyter-server, jupyterlab, jupyterlab-server, notebook-shim, tornado
Required-by: jupyter

image

If I simply try to run jupyter notebook command, it runs just fine, browser window opens and I am able to use the notebook.

EDIT: used --lab instead of --notebook and it worked. Source - django-extensions/django-extensions#1835

@EtsuNDmA
Copy link
Author

EtsuNDmA commented Apr 4, 2024

@azegas thanks, I updated the gist

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment