python-packaging.md

Python notes

First notes from https://docs.python.org/3/tutorial/modules.html

Modules

Show search path for modules:

$ python3
>>> import sys
>>> print(sys.path)
['', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages']

The two interesting directories seem to be /usr/lib/python3/dist-packages and /usr/lib/python3.6

Display names provided by module:

>>> import sys
>>> dir(sys)
['__displayhook__', '__doc__', ...

All currently available names:

>>> dir()
['__annotations__', ...

By default builtins aren't listed by dir() but you can list them by importing them explicitly:

>>> import builtins
>>> dir(builtins)  
['ArithmeticError', 'AssertionError', ...

A module is just a file - the names defined in a module can be imported into another module.

>>> import fibo

The names in fibo.py are imported but need to be qualified with fibo, e.g. fibo.x.

Import names explicitly, these names don't need to be qualified:

>>> from fibo import x
>>> print(x)

You can import all names but this isn't recommended:

>>> from fibo import *

You can choose your own qualifier for names imported from a given module:

>>> import fibo as fib

And rename specific names:

>>> from fibo import fib as fibonacci

You can get the name of your containing module is available as __name__, however if you're being invoked as a script rather than as a module then __name__ will be __main__:

$ python3 fibo.py
>>> print(__name__)
__main__

Packages

A package is a hierarchy of modules.

Oddly the current Python3 package documentation seems to state that an __init__.py file (empty or containing initialization code) is needed in a directory for it to be considered a package.

This seems to have been true up until Python 3.3 which introduced namespace packages. The older style of packages are called regular packages.

I can't find anything that makes clear if there's any situation where you might still chose regular packages over namespace packages.

See this SO answer and the first and second answers on another SO question.

A non-empty __init__.py is still definitely useful for e.g. for exporting a set of names from the root of a package such that users of the package don't have to know where in the underlying hierarchy the name was defined (as shown in this SO answer.

You can include whatever you want in __init__.py:

$ mkdir -p alpha/beta
$ cat > alpha/beta/beta.py << EOF
> def b():
>     print("b")
> EOF
$ cat > alpha/beta/__init__.py << EOF
> print(f"initializing {__name__}")
> EOF
$ python3
>>> from alpha.beta.beta import b
initializing alpha.beta
>>> b()
b

For modules within a given package you can refer to other modules with absolute imports:

$ mkdir -p p/q/r
$ mkdir -p p/q/s
$ cat > p/q/r/alpha.py << EOF
def alpha():
    print("alpha")
EOF
$ cat > p/q/s/beta.py << EOF
from p.q.r.alpha import alpha

def beta():
    alpha()
    print("beta")
EOF
$ python3
>>> from p.q.s.beta import beta
>>> beta()
alpha
beta

Note that p is resolved against you Python path (sys.path above) and your current directory is generally the first directory in this path.

But you can also use relative paths, e.g. the import in beta.py can be changed to:

from ..r.alpha import alpha

Note that relative imports are based on the name of the current module. Since the name of the main module is always "__main__", modules intended for use as the main module of a Python application must always use absolute imports.

Wheel packaging

The Wheel format appears to be the modern replacement for the Egg format (see "wheel vs egg" on Python Packaging).

TODO: go through Wheel creation tutorial.

See also https://pythonwheels.com/ - why does it say setup.py isn't required when it's clearly in the tutorial (maybe it's whether the creator or the installer has to run setup.py)?

TODO: go through the full Python.org Python tutorial.

From "wheel vs egg":

Note: Wheel is a distribution format, i.e a packaging format. Egg was both a distribution format and a runtime installation format (if left zipped), and was designed to be importable.

I'm not sure what this means.

Virtual environments

See https://docs.python.org/3/library/venv.html and https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/