Skip to content

Instantly share code, notes, and snippets.

@inaz2
Created March 19, 2023 18:04
Show Gist options
  • Save inaz2/d436e4273e4f3051dcbd7b7d990d408e to your computer and use it in GitHub Desktop.
Save inaz2/d436e4273e4f3051dcbd7b7d990d408e to your computer and use it in GitHub Desktop.
Comparison of Python type-checkers

Test code

from typing import List
import requests

def ident(x):
    return x

def add_1(x):
    return x + 1

def and_y(x, y):
    return x and y

def or_y(x, y):
    return x or y

def implicit_none(x):
    if x:
        return x

def list_any_unannotated():
    lst = ["PyCon"]
    lst.append(2019)
    return [str(x) for x in lst]

def list_any_annotated() -> List[str]:
    lst = ["PyCon"]
    lst.append(2019)
    return [str(x) for x in lst]

def third_party_module():
    r = requests.get('/api/users/profile')
    return r.status_code

def test():
    add_1("str").upper()
    add_1(ident("str")).upper()
    and_y("str", 1).upper()
    or_y(1, "str").upper()
    implicit_none("").upper()
    list_any_unannotated()
    list_any_annotated()
    third_party_module().upper()

Mypy

$ mypy tests/
tests/tests.py:2: error: Library stubs not installed for "requests"  [import]
tests/tests.py:2: note: Hint: "python3 -m pip install types-requests"
tests/tests.py:2: note: (or run "mypy --install-types" to install all missing stub packages)
tests/tests.py:2: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
tests/tests.py:27: error: Argument 1 to "append" of "list" has incompatible type "int"; expected "str"  [arg-type]
Found 2 errors in 1 file (checked 1 source file)
$ mypy --no-site-packages --ignore-missing-imports tests/
tests/tests.py:27: error: Argument 1 to "append" of "list" has incompatible type "int"; expected "str"  [arg-type]
Found 1 error in 1 file (checked 1 source file)

Pyright

$ pyright tests/
No configuration file found.
No pyproject.toml file found.
stubPath /home/user/work/typechecker/typings is not a valid directory.
Assuming Python platform Linux
Searching for source files
Found 1 source file
pyright 1.1.299
/home/user/work/typechecker/tests/tests.py
  /home/user/work/typechecker/tests/tests.py:22:16 - error: Argument of type "Literal[2019]" cannot be assigned to parameter "__object" of type "str" in function "append"
    "Literal[2019]" is incompatible with "str" (reportGeneralTypeIssues)
  /home/user/work/typechecker/tests/tests.py:27:16 - error: Argument of type "Literal[2019]" cannot be assigned to parameter "__object" of type "str" in function "append"
    "Literal[2019]" is incompatible with "str" (reportGeneralTypeIssues)
  /home/user/work/typechecker/tests/tests.py:37:21 - error: Cannot access member "upper" for type "Literal[1]"
    Member "upper" is unknown (reportGeneralTypeIssues)
  /home/user/work/typechecker/tests/tests.py:38:20 - error: Cannot access member "upper" for type "Literal[1]"
    Member "upper" is unknown (reportGeneralTypeIssues)
  /home/user/work/typechecker/tests/tests.py:39:17 - error: Cannot access member "upper" for type "None"
    Member "upper" is unknown (reportGeneralTypeIssues)
  /home/user/work/typechecker/tests/tests.py:42:26 - error: Cannot access member "upper" for type "int"
    Member "upper" is unknown (reportGeneralTypeIssues)
6 errors, 0 warnings, 0 informations 
Completed in 0.811sec

Pytype

$ pytype tests/
Computing dependencies
/home/user/work/typechecker/venv/lib/python3.10/site-packages/urllib3/contrib/socks.py:50: DependencyWarning: SOCKS support in urllib3 requires the installation of optional dependencies: specifically, PySocks.  For more information, see https://urllib3.readthedocs.io/en/1.26.x/contrib.html#socks-proxies
  warnings.warn(
Analyzing 1 sources with 0 local dependencies
ninja: Entering directory `.pytype'
[1/1] check tests
FAILED: /home/user/work/typechecker/.pytype/pyi/tests.pyi 
/home/user/work/typechecker/venv/bin/python3 -m pytype.single --imports_info /home/user/work/typechecker/.pytype/imports/tests.imports --module-name tests --platform linux -V 3.10 -o /home/user/work/typechecker/.pytype/pyi/tests.pyi --analyze-annotated --nofail --quick /home/user/work/typechecker/tests/tests.py
File "/home/user/work/typechecker/tests/tests.py", line 8, in add_1: unsupported operand type(s) for +: str and int [unsupported-operands]
  Function __add__ on str expects str
Called from (traceback):
  line 35, in test
File "/home/user/work/typechecker/tests/tests.py", line 37, in test: No attribute 'upper' on int [attribute-error]
File "/home/user/work/typechecker/tests/tests.py", line 38, in test: No attribute 'upper' on int [attribute-error]
File "/home/user/work/typechecker/tests/tests.py", line 39, in test: No attribute 'upper' on None [attribute-error]
File "/home/user/work/typechecker/tests/tests.py", line 42, in test: No attribute 'upper' on int [attribute-error]

For more details, see https://google.github.io/pytype/errors.html
ninja: build stopped: subcommand failed.
Leaving directory '.pytype'
File "/home/user/work/typechecker/tests/tests.py", line 8, in add_1: unsupported operand type(s) for +: str and int [unsupported-operands]
  Function __add__ on str expects str
Called from (traceback):
  line 36, in test
$ cat .pytype/pyi/tests.pyi
# (generated with --quick)

import requests
from typing import Any, List, Optional, TypeVar, Union

_T0 = TypeVar('_T0')
_T1 = TypeVar('_T1')

def add_1(x) -> Any: ...
def and_y(x: _T0, y: _T1) -> Union[_T0, _T1]: ...
def ident(x: _T0) -> _T0: ...
def implicit_none(x: _T0) -> Optional[_T0]: ...
def list_any_annotated() -> List[str]: ...
def list_any_unannotated() -> List[str]: ...
def or_y(x: _T0, y: _T1) -> Union[_T0, _T1]: ...
def test() -> None: ...
def third_party_module() -> int: ...

Pyre

$ pyre init
ƛ Which directory(ies) should pyre analyze? (Default: `.`): tests
ƛ Successfully initialized pyre!
  You can view the configuration at `/home/user/work/typechecker/.pyre_configuration`.
  You can now run the type checker with `pyre`.
$ pyre
ƛ No watchman binary found. 
To enable pyre incremental, you can install watchman: https://facebook.github.io/watchman/docs/install
ƛ Defaulting to non-incremental check.
ƛ Found 1 type error!
tests/tests.py:27:15 Incompatible parameter type [6]: In call `list.append`, for 1st positional argument, expected `str` but got `int`.
$ pyre --strict
ƛ No watchman binary found. 
To enable pyre incremental, you can install watchman: https://facebook.github.io/watchman/docs/install
ƛ Defaulting to non-incremental check.
ƛ Found 17 type errors!
tests/tests.py:4:0 Missing return annotation [3]: Return type is not specified.
tests/tests.py:4:10 Missing parameter annotation [2]: Parameter `x` has no type specified.
tests/tests.py:7:0 Missing return annotation [3]: Return type is not specified.
tests/tests.py:7:10 Missing parameter annotation [2]: Parameter `x` has no type specified.
tests/tests.py:10:0 Missing return annotation [3]: Return type is not specified.
tests/tests.py:10:10 Missing parameter annotation [2]: Parameter `x` has no type specified.
tests/tests.py:10:13 Missing parameter annotation [2]: Parameter `y` has no type specified.
tests/tests.py:13:0 Missing return annotation [3]: Return type is not specified.
tests/tests.py:13:9 Missing parameter annotation [2]: Parameter `x` has no type specified.
tests/tests.py:13:12 Missing parameter annotation [2]: Parameter `y` has no type specified.
tests/tests.py:16:0 Missing return annotation [3]: Return type is not specified.
tests/tests.py:16:12 Missing parameter annotation [2]: Parameter `x` has no type specified.
tests/tests.py:20:0 Missing return annotation [3]: Returning `List[str]` but no return type is specified.
tests/tests.py:22:15 Incompatible parameter type [6]: In call `list.append`, for 1st positional argument, expected `str` but got `int`.
tests/tests.py:27:15 Incompatible parameter type [6]: In call `list.append`, for 1st positional argument, expected `str` but got `int`.
tests/tests.py:30:0 Missing return annotation [3]: Returning `int` but no return type is specified.
tests/tests.py:34:0 Missing return annotation [3]: Returning `None` but no return type is specified.

Summary

Mypy Pyright Pytype Pyre Pyre (strict)
add_1() x x
add_1(ident()) x x
and_y() x x x
or_y() x x x
implicit_none() x x x
list_any_unannotated() x x
list_any_annotated() x x x x
third_party_module() x x x
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment