Skip to content

Instantly share code, notes, and snippets.

@masahitojp

masahitojp/test.md

Created Oct 10, 2019
Embed
What would you like to do?
for hiroshima
marp
true

footer: Python と型ヒントとその使い方 - PyCon Kyushu in Okinawa 2019 May 18th slidenumbers: true

Python と型ヒント

とその使い方

PyCon Kyushu in Okinawa 2019 May 18th

in Ryukyu University, Okinawa


[fit] こんにちは


Who are you(お前誰よ)?

  • 中村真人(masahito)
  • 株式会社ヌーラボ Typetalk チーム所属 Software Developer
    • use Scala and JavaScript and Python 🐍
  • Python ❤️ and ❤️

^ May I take a picture from the stage? ^ How many people come from Japan?


Masato Nakamura(masahito)

  • Speaker
    • Python Conference Hong Kong 2018
    • Python Conference Kyushu 2018 in Japan
    • Python Conference Taiwan 2017

Nulab provide collaboration softwares.

inline 80%


Today's Goal

Type Hints とは?について説明できる Type Hints を今日から使えるように Python2 to 3 の話も少しだけします


Not covering today

型とは? Technical Detail


Today's Outline

Ⅰ. Introduction of Type Hints Ⅱ. Usage of Type Hints Ⅲ. Benefits of Type Hints


Today's Outline

👉 Ⅰ. Introduction of Type Hints Ⅱ. Usage of Type Hints Ⅲ. Benefits of Type Hints


Introduction of Type Hints

  • A. What is Type Hints function
  • B. History of Type Hints
  • C. typing module

What is Type Hints function

We can write more Readable Codes.

We can annotate our codes only.

^ I want to explain using examples to everyone


What is Type Hints function

  • Traditional Python code
def twice(num):
    return num * 2
  • using Type Hints
def twice(num: int) -> int:
    return num * 2

What is Type Hints function

also we can use Class!

class Car:
    def __init__(self) -> None:
        self.seats = 4
c: Car = Car()

What is Type Hints function

  • using Type Hints
def twice(num: int) -> int:
    return num * 2
  • C-language Code
int twice(int num) {
   return num * 2;
}

Difference of Type Hints and C code

  • If you set wrong Type,We cannot show runtime error
def twice(num: int) -> int:
    return num * 2.0

# expected `int` but use `str` 😓
print(twice("hello"))
$ python set_wrong_type.py
$ hellohello ## 😑

[fit] hmm, What is TypeHints 🤔???


[fit] We can annotate our codes only.


But You use library, e.g. mypy

$ mypy --python-version 3.7 example.py
example.py:5: error: Argument 1 to "twice" has incompatible type
"str"; expected "int"

[fit] What is good point for checking type🤔???


[fit] We can check our code without run codes.

  • so we can re-write our Python2 code to Python3 😄 more easily
  • We can do it !!

History of Type Hints


History of Type Hints

implement PEP
Python 3.0 3107
Python 3.5 484
Ptyhon 3.6 526
Python 3.7 560, 561, 562,563

Traditional Python code(Python2)

def greeting(name):
    return 'Hello ' + name

greet = greeting("Guido")
  • What is name🤔?
  • What does greeting return🤔?

PEP 3107(Python 3.0)

パラメータと戻り値の両方で、オプションで関数アノテーションを追加することができます。フリーテキストを書くことができます。

def compile(source: "something compilable",
            filename: "where the compilable thing comes from",
            mode: "is this a single statement or a suite?"):

セマンティクスは意図的に未定義に


PEP 484(Python 3.5)

コミュニティが標準ライブラリ内の標準化された語彙およびベースラインツールから恩恵を受けることができるように、静的タイプ分析のための十分なサードパーティの使用方法を用意した

def greeting(name: str) -> str:
    return 'Hello ' + name

Points in PEP484

Python は dynamically typed language のままであり、作者は型のヒントを必須にしたいという欲求を全く持っていません。

  • コード生成ではありません
  • コンパイラがコードをどのように準拠しているかには影響しません。
  • 型チェック後、実行時にコードが壊れる可能性があります。
  • Python を静的型にするつもりはない

inconvinient of PEP484

def greeting(name: str) -> str:
    return 'Hello ' + name

greet = greeting("Masato") # type: str # make greeting message!

Hmm , でももうちょい簡潔に定義を書きたい


PEP 526(Python 3.6)

# PEP 484
def greeting(name: str) -> str:
    return 'Hello ' + name

greet = greeting("Masato") # type: str # make greeting message!
# pep 526
def greeting(name: str) -> str:
    return 'Hello ' + name

greet:str = greeting("Masato") # make greeting message!

typing module


What is the typing module

  • PEP 484 が 3.5 で実装されたときに導入されたモジュール
  • List,Dict,Union などのよく使われる型を Python に提供します
  • namedtupleのようなデータ構造を簡単に導入できる
from typing import List

a: List[int] = [1,2,3]
path: Optional[str] = None # Path to module sourde

  • Union
i = 1           # Infer type "int" for i
l = [1, 2]      # Infer type "List[int]" for l


from typing import Union

x: Union[int, str] = 1
x: Union[int, str] = 1.1  # Error

  • NamedTuple

In Python 3.5(PEP 484)

Point = namedtuple('Point', ['x', 'y'])
p = Point(x=1, y=2)

In Python 3.6(PEP 526)

from typing import NamedTuple

class Point(NamedTuple):
    x: int
    y: int
p = Point(x=1, y=2)

Today's Outline

Ⅰ. Introduction of Type Hints 👉 Ⅱ. Usage of Type Hints Ⅲ. Benefits of Type Hints


Usage Of Type Hints

  • A. How to write
  • B. How to get Type Hints Info

How to write

PEP526 style

def greeting(name: str) -> str:
    return 'Hello ' + name

greet: str = greeting("Masato")

difference of PEP484 and PEP526

# pep 484
child # type: bool # cannot allow
if age < 18:
    child = True  # type: bool # It's OK
else:
    child = False # type: bool # It's OK
# pep 526
child: bool # OK
if age < 18:
    child = True # No need to Write bool
else:
    child = False

backward compatibility

# We can write pep484 style code in 3.6 backward compatiblity
hour = 24 # type: int

# PEP 526 style
hour: int; hour = 24

hour: int = 24

recommend using Type Hints

PEP3107 style is also OK

>>> alice: 'well done' = 'A+'
>>> bob: 'what a shame' = 'F-'
>>> __annotations__
{'alice': 'well done', 'bob': 'what a shame'}

でもなるべくこういう書き方は Type Hints でのみ使いましょうね.


How to get Type Hints Info

Q. where is variable annotation

A. at __annotaitons__


How to get Type Hints Info

example

>>> answer:int = 42
>>> __annotations__
{'answer': <class 'int'>}

Note for Class

We can only find Class variables.

>>> class Car:
...     stats: ClassVar[Dict[str, int]] = {}
...     def __init__(self) -> None:
...         self.seats = 4
>>> c: Car = Car()
>>> c.__annotations__
{'stats': typing.ClassVar[typing.Dict[str, int]]}
# only ClassVar!

We cannot get instance's variables!


Today's Outline

Ⅰ. Introduction of Type Hints Ⅱ. Usage of Type Hints 👉 Ⅲ. Benefits of Type Hints


Benefits of Type Hints

A. Code Style B. code completion C. statistic type analysis D. Our story


Code Style

  • Simple is best
  • Explicit is better than Implicit.
def greeting(name: str) -> str:
    return 'Hello ' + name

We can check this by Type Hints

def greeting(name: str) -> str:
    return 42

Code Completion

We can use code completion in

  • Editor
    • Visual Studio code
  • IDE
    • PyCharm(IntelliJ)

example

inline 100%


statistic type analysis


statistic type analysis

  • for check variable type(no runable code)
  • python has some tools
    • mypy
    • pytypes (not talk)

example

inline 200%


mypy


How to install mypy

$ pip install mypy

Notice requires Python 3.5 or later to run.


mypy example

  • code(example.py)
from typing import NamedTuple

class Employee(NamedTuple):
    name: str
    id: int

emp: Employee = Employee(name='Guido', id='x')
  • run
$ mypy --python-version 3.7 example.py
example.py:16: error: Argument 2 to "Employee" has incompatible type
"str"; expected "int"

mypy can use in Python2

class MyClass(object):

    # For instance methods, omit `self`.
    def my_method(self, num, str1):
        # type: (int, str) -> str
        return num * str1

    def __init__(self):
        # type: () -> None
        pass

x = MyClass() # type: MyClass

Missing imports

import しているライブラリが type annotation を提供してないとエラーが出る 😢

main.py:1: error: No library stub file for standard library module 'antigravity'
main.py:2: error: No library stub file for module 'flask'
main.py:3: error: Cannot find module named 'this_module_does_not_exist'

このエラーは抑えられる(現状だとほぼ必須 😢)

mypy --ignore-missing-imports

Continuous Integration

  • Jenkins
  • Travis CI
  • CircleCi

Continuous Integration

  • we can run mypy on CI systems
  • we can get results
    • If you develop in a team, there are many benefits
    • ex: If someone issues a pull request for your project, you can review their code

sample for CI

Sample


example(error)

inline


example(success)

inline


Benefits for us(Nulab)

  • We use Fabric v1 and Ansible for deployment
  • But it is for Only Python2
  • Python2 will finished at 2020(Tokyo Olympic Year)
  • We thought we should convert Python2 to Python3

inline

Recently, that date has been updated to January 1, 2020.


We want to use Python3


We(Nulab) use Python3 & Fabric3 & mypy

  • We use TypeHints for rewriting our code
  • We got it! -> Only 2days
  • We deploy the new version of Out App every week.
  • beta version every day!

Python2 to 3 で主にやったこと

  • まずは Python2.7 で動くか検証する
  • six を入れる(まずは Python2 と 3 で動くように)
  • 2to3 コマンドで何が変わるかをまずは確認する

Python2 to 3 で主にやったこと

  • 重要な処理は リファクタリングを入れて、テスト追加、関数化を進める
    • デプロイスクリプトはべたっとコードを書きがち
    • なるべく関数に分割していく
    • ここで型ヒントを入れる-> 型チェックするテスト書くの辛い
      • でも頑張りすぎない

def is_modified(path, backup='bak'):
    # type: (Text, str) -> bool
    """
    該当の path とバックアップファイルを比較し変更があったかを返す
    """
    with settings(hide('warnings'), warn_only=True):
        result = sudo('diff -u %s.%s %s' % (path, backup, path))
    return result.return_code == 1
```

---

# Those are the Benefits of Type Hints

---

# Today's Outline1. Introduction of Type Hints2. Usage of Type Hints3. Benefits of Type Hints

---

# 型ヒントってどんなものか理解できました?

---

# Thank you!

@masahito

---

## おまけ

- type-hints 周りの 今議論されている機能 pep
  - PEP 586 -- Literal Types
  - PEP 591 -- Adding a final qualifier to typing
- mypy の機能拡張 (plugin)
- typing module の拡張の話

---

## PEP 586 -- Literal Types

```python
from typing import Literal

def accepts_only_four(x: Literal[4]) -> None:
    pass

accepts_only_four(4)   # OK
accepts_only_four(19)  # Rejected

PEP 591 -- Adding a final qualifier to typing

from typing import Final

RATE: Final = 3000

class Base:
    DEFAULT_ID: Final = 0

RATE = 300  # Error: can't assign to final attribute
Base.DEFAULT_ID = 1  # Error: can't override a final attribute

mypy plugin

  • mypy に色々機能拡張する方法が議論され、ている
  • plugin として mypy が機能を提供、拡張したいときは plugin を書いて読み込ませる
  • 将来的に sqlAlchemy のテーブル定義を取り出したり etc 用の plugin を書いたりとかもできそう

mypy plugin


mypy と PEP561

  • PEP 561 is
    • 各ライブラリごとの型情報を外だしできるようにするための取り決め
    • これのおかげで numpy の stub も提供されている
    • https://github.com/numpy/numpy-stubs

typing extentions


User Defined Protocol

from typing import Iterable
from typing_extensions import Protocol

class SupportsClose(Protocol):
    def close(self) -> None:
       ...  # Empty method body (explicit '...')

class Resource:  # No SupportsClose base class!
    # ... some methods ...

    def close(self) -> None:
       self.resource.release()

def close_all(items: Iterable[SupportsClose]) -> None:
    for item in items:
        item.close()

close_all([Resource(), open('some/file')])  # Okay!

DuckTyping 的なメソッドのチェックができるように!!!


Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.