Circular imports in Python 2 and Python 3: when are they fatal? When do they work?

README.md

When are Python circular imports fatal?

In your Python package, you have:

• an __init__.py that designates this as a Python package
• a module_a.py, containing a function action_a() that references an attribute (like a function or variable) in module_b.py, and
• a module_b.py, containing a function action_b() that references an attribute (like a function or variable) in module_a.py.

This situation can introduce a circular import error: module_a attempts to import module_b, but can't, because module_b needs to import module_a, which is in the process of being interpreted.

But, sometimes Python is magic, and code that looks like it should cause this circular import error works just fine!

When does it work and when does it not? Why does it work when it does?

When it works

Top of module; no from; Python 2 only

Imports at top of module. import MODULE. Function references MODULE.ATTRIBUTE. Since import MODULE is an implicit relative import from the current directory, this might not work for importing from other packages. Since import MODULE is a syntax error in Python 3 this works in Python 2 only so is not future-proof.

# pkg/module_a.py                             # pkg/module_b.py
import module_b                               import module_a
def action_a():                               def action_b():
    print(module_b.action_b.__name__)             print(module_a.action_a.__name__)

Top of module; no from; no relative

Imports at top of module. import PACKAGE.MODULE. Function references PACKAGE.MODULE.ATTRIBUTE. Since import .MODULE is a syntax error, relative imports can't be used here; modules must know their containing package's name.

# pkg/module_a.py                             # pkg/module_b.py
import pkg.module_b                           import pkg.module_a
def action_a():                               def action_b():
    print(pkg.module_b.action_b.__name__)         print(pkg.module_a.action_a.__name__)

Bottom of module; import attribute, not module; from okay.

Imports at bottom of module (or at least after referenced attribute). from PACKAGE.MODULE import ATTRIBUTE. May be relative, like from .MODULE import ATTRIBUTE. Function references ATTRIBUTE.

# pkg/module_a.py                             # pkg/module_b.py
def action_a():                               def action_b():
    print(action_b.__name__)                      print(action_a.__name__)
from .module_b import action_b                from .module_a import action_a

Top of function; from okay

Imports at top of function. from PACKAGE import MODULE. PACKAGE may be relative, like .. Function references MODULE.ATTRIBUTE. This becomes ugly (but still works) if you have several functions that all refer to the same MODULE.

# pkg/module_a.py                             # pkg/module_b.py                
def action_a():                               def action_b():
    from . import module_b                        from . import module_a
    print(module_b.action_b.__name__)             print(module_a.action_a.__name__)

Consider the following example python package where a.py and b.py depend on each other:

/package
    __init__.py
    a.py
    b.py

There are several ways to import a module in python

import package.a           # Absolute import
import package.a as a_mod  # Absolute import bound to different name
from package import a      # Alternate absolute import
import a                   # Implicit relative import (deprecated, py2 only)
from . import a            # Explicit relative import

Unfortunately, only the 1st and 4th options actually work when you have circular dependencies (the rest all raise ImportError or AttributeError). In general, you shouldn't be using the 4th syntax, since it only works in python2 and runs the risk of clashing with other 3rd party modules. So really, only the first syntax is guaranteed to work. However, you still have several options when dealing with circular dependencies.

EDIT: The ImportError and AttributeError issues only occur in python 2. In python 3 the import machinery has been rewritten and all of these import statements (with the exception of 4) will work, even with circular dependencies.
Use Absolute Imports

Just use the first import syntax above. The downside to this method is that the import names can get super long for large packages.

In a.py

import package.b
In b.py

import package.a
Defer import until later

I've seen this method used in lots of packages, but it still feels hacky to me, and I dislike that I can't look at the top of a module and see all its dependencies, I have to go searching through all the functions as well.

In a.py

def func():
    from package import b

In b.py

def func():
    from package import a

Put all imports in a central module

This also works, but has the same problem as the first method, where all the package and submodule calls get super long. It also has two major flaws -- it forces all the submodules to be imported, even if you're only using one or two, and you still can't look at any of the submodules and quickly see their dependencies at the top, you have to go sifting through functions.

In init.py

from . import a
from . import b

In a.py

import package

def func():
    package.b.some_object()

In b.py

import package

def func():
    package.a.some_object()

So those are your options (and they all kinda suck IMO). Frankly, this seems to be a glaring bug in the python import machinery, but thats just my opinion.

Add parent directory in the classpath

import sys
import os
# set sytem path to be directory above so that a can be a 
# package namespace
DIRECTORY_SCRIPT = os.path.dirname(os.path.realpath(__file__)) 
sys.path.insert(0,DIRECTORY_SCRIPT+"/..")