Skip to content

Instantly share code, notes, and snippets.

@jigi-33
Created October 29, 2021 06:37
Show Gist options
  • Save jigi-33/e09e9d6b29f998716ba1c22cb5df4faa to your computer and use it in GitHub Desktop.
Save jigi-33/e09e9d6b29f998716ba1c22cb5df4faa to your computer and use it in GitHub Desktop.
Python3 typing hints and class declaration features [code examples]

Type hints cheat sheet (Python 3)

This document is a quick cheat sheet showing how the PEP 484 type annotation notation represents various common types in Python 3.

Note

Technically many of the type annotations shown below are redundant, because mypy can derive them from the type of the expression. So many of the examples have a dual purpose: show how to write the annotation, and show the inferred types.

Variables

Python 3.6 introduced a syntax for annotating variables in PEP 526 and we use it in most examples.

# This is how you declare the type of a variable type in Python 3.6
age: int = 1

# In Python 3.5 and earlier you can use a type comment instead
# (equivalent to the previous definition)
age = 1  # type: int

# You don't need to initialize a variable to annotate it
a: int  # Ok (no value at runtime until assigned)

# The latter is useful in conditional branches
child: bool
if age < 18:
    child = True
else:
    child = False

Built-in types

from typing import List, Set, Dict, Tuple, Optional

# For simple built-in types, just use the name of the type
x: int = 1
x: float = 1.0
x: bool = True
x: str = "test"
x: bytes = b"test"

# For collections, the type of the collection item is in brackets
# (Python 3.9+)
x: list[int] = [1]
x: set[int] = {6, 7}

# In Python 3.8 and earlier, the name of the collection type is
# capitalized, and the type is imported from 'typing'
x: List[int] = [1]
x: Set[int] = {6, 7}

# Same as above, but with type comment syntax (Python 3.5 and earlier)
x = [1]  # type: List[int]

# For mappings, we need the types of both keys and values
x: dict[str, float] = {'field': 2.0}  # Python 3.9+
x: Dict[str, float] = {'field': 2.0}

# For tuples of fixed size, we specify the types of all the elements
x: tuple[int, str, float] = (3, "yes", 7.5)  # Python 3.9+
x: Tuple[int, str, float] = (3, "yes", 7.5)

# For tuples of variable size, we use one type and ellipsis
x: tuple[int, ...] = (1, 2, 3)  # Python 3.9+
x: Tuple[int, ...] = (1, 2, 3)

# Use Optional[] for values that could be None
x: Optional[str] = some_function()
# Mypy understands a value can't be None in an if-statement
if x is not None:
    print(x.upper())
# If a value can never be None due to some invariants, use an assert
assert x is not None
print(x.upper())

Functions

Python 3 supports an annotation syntax for function declarations.

from typing import Callable, Iterator, Union, Optional, List

# This is how you annotate a function definition
def stringify(num: int) -> str:
    return str(num)

# And here's how you specify multiple arguments
def plus(num1: int, num2: int) -> int:
    return num1 + num2

# Add default value for an argument after the type annotation
def f(num1: int, my_float: float = 3.5) -> float:
    return num1 + my_float

# This is how you annotate a callable (function) value
x: Callable[[int, float], float] = f

# A generator function that yields ints is secretly just a function that
# returns an iterator of ints, so that's how we annotate it
def g(n: int) -> Iterator[int]:
    i = 0
    while i < n:
        yield i
        i += 1

# You can of course split a function annotation over multiple lines
def send_email(address: Union[str, List[str]],
               sender: str,
               cc: Optional[List[str]],
               bcc: Optional[List[str]],
               subject='',
               body: Optional[List[str]] = None
               ) -> bool:
    ...

# An argument can be declared positional-only by giving it a name
# starting with two underscores:
def quux(__x: int) -> None:
    pass

quux(3)  # Fine
quux(__x=3)  # Error

When you're puzzled or when things are complicated

from typing import Union, Any, List, Optional, cast

# To find out what type mypy infers for an expression anywhere in
# your program, wrap it in reveal_type().  Mypy will print an error
# message with the type; remove it again before running the code.
reveal_type(1)  # -> Revealed type is "builtins.int"

# Use Union when something could be one of a few types
x: List[Union[int, str]] = [3, 5, "test", "fun"]

# Use Any if you don't know the type of something or it's too
# dynamic to write a type for
x: Any = mystery_function()

# If you initialize a variable with an empty container or "None"
# you may have to help mypy a bit by providing a type annotation
x: List[str] = []
x: Optional[str] = None

# This makes each positional arg and each keyword arg a "str"
def call(self, *args: str, **kwargs: str) -> str:
    request = make_request(*args, **kwargs)
    return self.do_api_query(request)

# Use a "type: ignore" comment to suppress errors on a given line,
# when your code confuses mypy or runs into an outright bug in mypy.
# Good practice is to comment every "ignore" with a bug link
# (in mypy, typeshed, or your own code) or an explanation of the issue.
x = confusing_function()  # type: ignore  # https://github.com/python/mypy/issues/1167

# "cast" is a helper function that lets you override the inferred
# type of an expression. It's only for mypy -- there's no runtime check.
a = [4]
b = cast(List[int], a)  # Passes fine
c = cast(List[str], a)  # Passes fine (no runtime check)
reveal_type(c)  # -> Revealed type is "builtins.list[builtins.str]"
print(c)  # -> [4]; the object is not cast

# If you want dynamic attributes on your class, have it override "__setattr__"
# or "__getattr__" in a stub or in your source code.
#
# "__setattr__" allows for dynamic assignment to names
# "__getattr__" allows for dynamic access to names
class A:
    # This will allow assignment to any A.x, if x is the same type as "value"
    # (use "value: Any" to allow arbitrary types)
    def __setattr__(self, name: str, value: int) -> None: ...

    # This will allow access to any A.x, if x is compatible with the return type
    def __getattr__(self, name: str) -> int: ...

a.foo = 42  # Works
a.bar = 'Ex-parrot'  # Fails type checking

Standard "duck types"

In typical Python code, many functions that can take a list or a dict as an argument only need their argument to be somehow "list-like" or "dict-like". A specific meaning of "list-like" or "dict-like" (or something-else-like) is called a "duck type", and several duck types that are common in idiomatic Python are standardized.

from typing import Mapping, MutableMapping, Sequence, Iterable, List, Set

# Use Iterable for generic iterables (anything usable in "for"),
# and Sequence where a sequence (supporting "len" and "__getitem__") is
# required
def f(ints: Iterable[int]) -> List[str]:
    return [str(x) for x in ints]

f(range(1, 3))

# Mapping describes a dict-like object (with "__getitem__") that we won't
# mutate, and MutableMapping one (with "__setitem__") that we might
def f(my_mapping: Mapping[int, str]) -> List[int]:
    my_mapping[5] = 'maybe'  # if we try this, mypy will throw an error...
    return list(my_mapping.keys())

f({3: 'yes', 4: 'no'})

def f(my_mapping: MutableMapping[int, str]) -> Set[str]:
    my_mapping[5] = 'maybe'  # ...but mypy is OK with this.
    return set(my_mapping.values())

f({3: 'yes', 4: 'no'})

You can even make your own duck types using :ref:`protocol-types`.

Classes

class MyClass:
    # You can optionally declare instance variables in the class body
    attr: int
    # This is an instance variable with a default value
    charge_percent: int = 100

    # The "__init__" method doesn't return anything, so it gets return
    # type "None" just like any other method that doesn't return anything
    def __init__(self) -> None:
        ...

    # For instance methods, omit type for "self"
    def my_method(self, num: int, str1: str) -> str:
        return num * str1

# User-defined classes are valid as types in annotations
x: MyClass = MyClass()

# You can use the ClassVar annotation to declare a class variable
class Car:
    seats: ClassVar[int] = 4
    passengers: ClassVar[List[str]]

# You can also declare the type of an attribute in "__init__"
class Box:
    def __init__(self) -> None:
        self.items: List[str] = []

Coroutines and asyncio

See :ref:`async-and-await` for the full detail on typing coroutines and asynchronous code.

import asyncio

# A coroutine is typed like a normal function
async def countdown35(tag: str, count: int) -> str:
    while count > 0:
        print('T-minus {} ({})'.format(count, tag))
        await asyncio.sleep(0.1)
        count -= 1
    return "Blastoff!"

Miscellaneous

import sys
import re
from typing import Match, AnyStr, IO

# "typing.Match" describes regex matches from the re module
x: Match[str] = re.match(r'[0-9]+', "15")

# Use IO[] for functions that should accept or return any
# object that comes from an open() call (IO[] does not
# distinguish between reading, writing or other modes)
def get_sys_IO(mode: str = 'w') -> IO[str]:
    if mode == 'w':
        return sys.stdout
    elif mode == 'r':
        return sys.stdin
    else:
        return sys.stdout

# Forward references are useful if you want to reference a class before
# it is defined
def f(foo: A) -> int:  # This will fail
    ...

class A:
    ...

# If you use the string literal 'A', it will pass as long as there is a
# class of that name later on in the file
def f(foo: 'A') -> int:  # Ok
    ...

Decorators

Decorator functions can be expressed via generics. See :ref:`declaring-decorators` for more details.

from typing import Any, Callable, TypeVar

F = TypeVar('F', bound=Callable[..., Any])

def bare_decorator(func: F) -> F:
    ...

def decorator_args(url: str) -> Callable[[F], F]:
    ...

Classes basics

This section will help get you started annotating your classes. Built-in classes such as int also follow these same rules.

Instance and class attributes

The type checker detects if you are trying to access a missing attribute, which is a very common programming error. For this to work correctly, instance and class attributes must be defined or initialized within the class. Python infers the types of attributes:

class A:
    def __init__(self, x: int) -> None:
        self.x = x  # Aha, attribute 'x' of type 'int'

a = A(1)
a.x = 2  # OK!
a.y = 3  # Error: "A" has no attribute "y"

This is a bit like each class having an implicitly defined :py:data:`__slots__ <object.__slots__>` attribute. This is only enforced during type checking and not when your program is running.

You can declare types of variables in the class body explicitly using a type annotation:

class A:
    x: List[int]  # Declare attribute 'x' of type List[int] w/o value

a = A()
a.x = [1]     # OK

As in Python generally, a variable defined in the class body can be used as a class or an instance variable. (As discussed in the next section, you can override this with a :py:data:`~typing.ClassVar` annotation.)

Type comments work as well, if you need to support Python versions earlier than 3.6:

class A:
    x = None  # type: List[int]  # Declare attribute 'x' of type List[int]

Note that attribute definitions in the class body that use a type comment are special: a None value is valid as the initializer, even though the declared type is not optional. This should be used sparingly, as this can result in None-related runtime errors that mypy can't detect.

Similarly, you can give explicit types to instance variables defined in a method:

class A:
    def __init__(self) -> None:
        self.x: List[int] = []

    def f(self) -> None:
        self.y: Any = 0

You can only define an instance variable within a method if you assign to it explicitly using self:

class A:
    def __init__(self) -> None:
        self.y = 1   # Define 'y'
        a = self
        a.x = 1      # Error: 'x' not defined

Annotating __init__ methods

The :py:meth:`__init__ <object.__init__>` method is somewhat special -- it doesn't return a value. This is best expressed as -> None. However, since many feel this is redundant, it is allowed to omit the return type declaration on :py:meth:`__init__ <object.__init__>` methods if at least one argument is annotated. For example, in the following classes :py:meth:`__init__ <object.__init__>` is considered fully annotated:

class C1:
    def __init__(self) -> None:
        self.var = 42

class C2:
    def __init__(self, arg: int):
        self.var = arg

However, if :py:meth:`__init__ <object.__init__>` has no annotated arguments and no return type annotation, it is considered an untyped method:

class C3:
    def __init__(self):
        # This body is not type checked
        self.var = 42 + 'abc'

Class attribute annotations

You can use a :py:data:`ClassVar[t] <typing.ClassVar>` annotation to explicitly declare that a particular attribute should not be set on instances:

from typing import ClassVar

class A:
    x: ClassVar[int] = 0  # Class variable only

A.x += 1  # OK

a = A()
a.x = 1  # Error: Cannot assign to class variable "x" via instance
print(a.x)  # OK -- can be read through an instance

Note

If you need to support Python 3 versions 3.5.2 or earlier, you have to import ClassVar from typing_extensions instead (available on PyPI). If you use Python 2.7, you can import it from typing.

It's not necessary to annotate all class variables using :py:data:`~typing.ClassVar`. An attribute without the :py:data:`~typing.ClassVar` annotation can still be used as a class variable. However, mypy won't prevent it from being used as an instance variable, as discussed previously:

class A:
    x = 0  # Can be used as a class or instance variable

A.x += 1  # OK

a = A()
a.x = 1  # Also OK

Note that :py:data:`~typing.ClassVar` is not a class, and you can't use it with :py:func:`isinstance` or :py:func:`issubclass`. It does not change Python runtime behavior -- it's only for type checkers such as mypy (and also helpful for human readers).

You can also omit the square brackets and the variable type in a :py:data:`~typing.ClassVar` annotation, but this might not do what you'd expect:

class A:
    y: ClassVar = 0  # Type implicitly Any!

In this case the type of the attribute will be implicitly Any. This behavior will change in the future, since it's surprising.

Note

A :py:data:`~typing.ClassVar` type parameter cannot include type variables: ClassVar[T] and ClassVar[List[T]] are both invalid if T is a type variable (see :ref:`generic-classes` for more about type variables).

Overriding statically typed methods

When overriding a statically typed method, mypy checks that the override has a compatible signature:

class Base:
    def f(self, x: int) -> None:
        ...

class Derived1(Base):
    def f(self, x: str) -> None:   # Error: type of 'x' incompatible
        ...

class Derived2(Base):
    def f(self, x: int, y: int) -> None:  # Error: too many arguments
        ...

class Derived3(Base):
    def f(self, x: int) -> None:   # OK
        ...

class Derived4(Base):
    def f(self, x: float) -> None:   # OK: mypy treats int as a subtype of float
        ...

class Derived5(Base):
    def f(self, x: int, y: int = 0) -> None:   # OK: accepts more than the base
        ...                                    #     class method

Note

You can also vary return types covariantly in overriding. For example, you could override the return type Iterable[int] with a subtype such as List[int]. Similarly, you can vary argument types contravariantly -- subclasses can have more general argument types.

You can also override a statically typed method with a dynamically typed one. This allows dynamically typed code to override methods defined in library classes without worrying about their type signatures.

As always, relying on dynamically typed code can be unsafe. There is no runtime enforcement that the method override returns a value that is compatible with the original return type, since annotations have no effect at runtime:

class Base:
    def inc(self, x: int) -> int:
        return x + 1

class Derived(Base):
    def inc(self, x):   # Override, dynamically typed
        return 'hello'  # Incompatible with 'Base', but no mypy error

Abstract base classes and multiple inheritance

Mypy supports Python :doc:`abstract base classes <library/abc>` (ABCs). Abstract classes have at least one abstract method or property that must be implemented by any concrete (non-abstract) subclass. You can define abstract base classes using the :py:class:`abc.ABCMeta` metaclass and the :py:func:`@abc.abstractmethod <abc.abstractmethod>` function decorator. Example:

from abc import ABCMeta, abstractmethod

class Animal(metaclass=ABCMeta):
    @abstractmethod
    def eat(self, food: str) -> None: pass

    @property
    @abstractmethod
    def can_walk(self) -> bool: pass

class Cat(Animal):
    def eat(self, food: str) -> None:
        ...  # Body omitted

    @property
    def can_walk(self) -> bool:
        return True

x = Animal()  # Error: 'Animal' is abstract due to 'eat' and 'can_walk'
y = Cat()     # OK
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment