masahitojp/test.md

## test.md

      
    Raw
  

              test.md
            
          
  marp
  
  
  true
  
  
footer: Python と型ヒントとその使い方 - PyCon Kyushu in Okinawa 2019 May 18th
slidenumbers: true
Python と型ヒント

とその使い方

PyCon Kyushu in Okinawa 2019 May 18th

in Ryukyu University, Okinawa


[fit] こんにちは


Who are you(お前誰よ)?


中村真人(masahito)
株式会社ヌーラボ Typetalk チーム所属 Software Developer

use Scala and JavaScript and Python 🐍


Python ❤️ and ☕ ❤️

^ May I take a picture from the stage?
^ How many people come from Japan?

Masato Nakamura(masahito)


Speaker

Python Conference Hong Kong 2018
Python Conference Kyushu 2018 in Japan
Python Conference Taiwan 2017


Nulab provide collaboration softwares.


Today's Goal


Type Hints とは?について説明できる
Type Hints を今日から使えるように
Python2 to 3 の話も少しだけします


Not covering today


型とは?
Technical Detail


Today's Outline

Ⅰ. Introduction of Type Hints
Ⅱ. Usage of Type Hints
Ⅲ. Benefits of Type Hints

Today's Outline

👉 Ⅰ. Introduction of Type Hints
Ⅱ. Usage of Type Hints
Ⅲ. Benefits of Type Hints

Introduction of Type Hints


A. What is Type Hints function
B. History of Type Hints
C. typing module


What is Type Hints function

We can write more Readable Codes.

We can annotate our codes only.

^ I want to explain using examples to everyone

What is Type Hints function


Traditional Python code

def twice(num):
    return num * 2

using Type Hints

def twice(num: int) -> int:
    return num * 2

What is Type Hints function

also we can use Class!
class Car:
    def __init__(self) -> None:
        self.seats = 4
c: Car = Car()

What is Type Hints function


using Type Hints

def twice(num: int) -> int:
    return num * 2

C-language Code

int twice(int num) {
   return num * 2;
}

Difference of Type Hints and C code


If you set wrong Type,We cannot show runtime error

def twice(num: int) -> int:
    return num * 2.0

# expected `int` but use `str` 😓
print(twice("hello"))
$ python set_wrong_type.py
$ hellohello ## 😑

[fit] hmm, What is TypeHints 🤔???


[fit] We can annotate our codes only.


But You use library, e.g. mypy

$ mypy --python-version 3.7 example.py
example.py:5: error: Argument 1 to "twice" has incompatible type
"str"; expected "int"

[fit] What is good point for checking type🤔???


[fit] We can check our code without run codes.


so we can re-write our Python2 code to Python3 😄 more easily
We can do it !!


History of Type Hints


History of Type Hints


implement
PEP


Python 3.0
3107


Python 3.5
484


Ptyhon 3.6
526


Python 3.7
560, 561, 562,563


Traditional Python code(Python2)

def greeting(name):
    return 'Hello ' + name

greet = greeting("Guido")

What is name🤔?
What does greeting return🤔?


PEP 3107(Python 3.0)

パラメータと戻り値の両方で、オプションで関数アノテーションを追加することができます。フリーテキストを書くことができます。
def compile(source: "something compilable",
            filename: "where the compilable thing comes from",
            mode: "is this a single statement or a suite?"):
セマンティクスは意図的に未定義に

PEP 484(Python 3.5)


コミュニティが標準ライブラリ内の標準化された語彙およびベースラインツールから恩恵を受けることができるように、静的タイプ分析のための十分なサードパーティの使用方法を用意した

def greeting(name: str) -> str:
    return 'Hello ' + name

PEP-484 Type Hints


Points in PEP484


Python は dynamically typed language のままであり、作者は型のヒントを必須にしたいという欲求を全く持っていません。


コード生成ではありません
コンパイラがコードをどのように準拠しているかには影響しません。
型チェック後、実行時にコードが壊れる可能性があります。
Python を静的型にするつもりはない


inconvinient of PEP484

def greeting(name: str) -> str:
    return 'Hello ' + name

greet = greeting("Masato") # type: str # make greeting message!
Hmm , でももうちょい簡潔に定義を書きたい

PEP 526(Python 3.6)

# PEP 484
def greeting(name: str) -> str:
    return 'Hello ' + name

greet = greeting("Masato") # type: str # make greeting message!
# pep 526
def greeting(name: str) -> str:
    return 'Hello ' + name

greet:str = greeting("Masato") # make greeting message!

typing module


What is the typing module


PEP 484 が 3.5 で実装されたときに導入されたモジュール
List,Dict,Union などのよく使われる型を Python に提供します
namedtupleのようなデータ構造を簡単に導入できる

from typing import List

a: List[int] = [1,2,3]
path: Optional[str] = None # Path to module sourde


Union

i = 1           # Infer type "int" for i
l = [1, 2]      # Infer type "List[int]" for l


from typing import Union

x: Union[int, str] = 1
x: Union[int, str] = 1.1  # Error


NamedTuple

In Python 3.5(PEP 484)
Point = namedtuple('Point', ['x', 'y'])
p = Point(x=1, y=2)
In Python 3.6(PEP 526)
from typing import NamedTuple

class Point(NamedTuple):
    x: int
    y: int
p = Point(x=1, y=2)


Today's Outline

Ⅰ. Introduction of Type Hints
👉 Ⅱ. Usage of Type Hints
Ⅲ. Benefits of Type Hints

Usage Of Type Hints


A. How to write
B. How to get Type Hints Info


How to write

PEP526 style
def greeting(name: str) -> str:
    return 'Hello ' + name

greet: str = greeting("Masato")

difference of PEP484 and PEP526

# pep 484
child # type: bool # cannot allow
if age < 18:
    child = True  # type: bool # It's OK
else:
    child = False # type: bool # It's OK
# pep 526
child: bool # OK
if age < 18:
    child = True # No need to Write bool
else:
    child = False

backward compatibility

# We can write pep484 style code in 3.6 backward compatiblity
hour = 24 # type: int

# PEP 526 style
hour: int; hour = 24

hour: int = 24

recommend using Type Hints

PEP3107 style is also OK
>>> alice: 'well done' = 'A+'
>>> bob: 'what a shame' = 'F-'
>>> __annotations__
{'alice': 'well done', 'bob': 'what a shame'}
でもなるべくこういう書き方は Type Hints でのみ使いましょうね.

How to get Type Hints Info

Q. where is variable annotation

A. at __annotaitons__


How to get Type Hints Info

example
>>> answer:int = 42
>>> __annotations__
{'answer': <class 'int'>}

Note for Class

We can only find Class variables.
>>> class Car:
...     stats: ClassVar[Dict[str, int]] = {}
...     def __init__(self) -> None:
...         self.seats = 4
>>> c: Car = Car()
>>> c.__annotations__
{'stats': typing.ClassVar[typing.Dict[str, int]]}
# only ClassVar!
We cannot get instance's variables!

Today's Outline

Ⅰ. Introduction of Type Hints
Ⅱ. Usage of Type Hints
👉 Ⅲ. Benefits of Type Hints

Benefits of Type Hints

A. Code Style
B. code completion
C. statistic type analysis
D. Our story

Code Style


Simple is best
Explicit is better than Implicit.

def greeting(name: str) -> str:
    return 'Hello ' + name
We can check this by Type Hints
def greeting(name: str) -> str:
    return 42

Code Completion

We can use code completion in

Editor

Visual Studio code


IDE

PyCharm(IntelliJ)


example


statistic type analysis


statistic type analysis


for check variable type(no runable code)
python has some tools

mypy
pytypes (not talk)


example


mypy


https://github.com/python/mypy
made by JukkaL & Guido
can output report
We can use!(PEP526 style)


How to install mypy

$ pip install mypy
Notice requires Python 3.5 or later to run.

mypy example


code(example.py)

from typing import NamedTuple

class Employee(NamedTuple):
    name: str
    id: int

emp: Employee = Employee(name='Guido', id='x')

run

$ mypy --python-version 3.7 example.py
example.py:16: error: Argument 2 to "Employee" has incompatible type
"str"; expected "int"

mypy can use in Python2

class MyClass(object):

    # For instance methods, omit `self`.
    def my_method(self, num, str1):
        # type: (int, str) -> str
        return num * str1

    def __init__(self):
        # type: () -> None
        pass

x = MyClass() # type: MyClass

Missing imports

import しているライブラリが type annotation を提供してないとエラーが出る 😢
main.py:1: error: No library stub file for standard library module 'antigravity'
main.py:2: error: No library stub file for module 'flask'
main.py:3: error: Cannot find module named 'this_module_does_not_exist'

このエラーは抑えられる(現状だとほぼ必須 😢)
mypy --ignore-missing-imports


Continuous Integration


Jenkins
Travis CI
CircleCi


Continuous Integration


we can run mypy on CI systems
we can get results

If you develop in a team, there are many benefits
ex: If someone issues a pull request for your project, you can review their code


sample for CI

Sample

example(error)


example(success)


Benefits for us(Nulab)


We use Fabric v1 and Ansible for deployment
But it is for Only Python2
Python2 will finished at 2020(Tokyo Olympic Year)
We thought we should convert Python2 to Python3


Recently, that date has been updated to January 1, 2020.


We want to use Python3


We use Fabric3

It's Fabric for Python2.7 and Python3
https://pypi.python.org/pypi/Fabric3


We try to check this, That's good for run


We(Nulab) use Python3 & Fabric3 & mypy


We use TypeHints for rewriting our code
We got it! -> Only 2days
We deploy the new version of Out App every week.
beta version every day!


Python2 to 3 で主にやったこと


まずは Python2.7 で動くか検証する
six を入れる(まずは Python2 と 3 で動くように)
2to3 コマンドで何が変わるかをまずは確認する


Python2 to 3 で主にやったこと


重要な処理は リファクタリングを入れて、テスト追加、関数化を進める

デプロイスクリプトはべたっとコードを書きがち
なるべく関数に分割していく
ここで型ヒントを入れる-> 型チェックするテスト書くの辛い

でも頑張りすぎない


def is_modified(path, backup='bak'):
    # type: (Text, str) -> bool
    """
    該当の path とバックアップファイルを比較し変更があったかを返す
    """
    with settings(hide('warnings'), warn_only=True):
        result = sudo('diff -u %s.%s %s' % (path, backup, path))
    return result.return_code == 1
```

---

# Those are the Benefits of Type Hints

---

# Today's Outline

✅ 1. Introduction of Type Hints
✅ 2. Usage of Type Hints
✅ 3. Benefits of Type Hints

---

# 型ヒントってどんなものか理解できました?

---

# Thank you！

@masahito

---

## おまけ

- type-hints 周りの 今議論されている機能 pep
  - PEP 586 -- Literal Types
  - PEP 591 -- Adding a final qualifier to typing
- mypy の機能拡張 (plugin)
- typing module の拡張の話

---

## PEP 586 -- Literal Types

```python
from typing import Literal

def accepts_only_four(x: Literal[4]) -> None:
    pass

accepts_only_four(4)   # OK
accepts_only_four(19)  # Rejected

PEP 591 -- Adding a final qualifier to typing

from typing import Final

RATE: Final = 3000

class Base:
    DEFAULT_ID: Final = 0

RATE = 300  # Error: can't assign to final attribute
Base.DEFAULT_ID = 1  # Error: can't override a final attribute

mypy plugin


mypy に色々機能拡張する方法が議論され、ている
plugin として mypy が機能を提供、拡張したいときは plugin を書いて読み込ませる
将来的に sqlAlchemy のテーブル定義を取り出したり etc 用の plugin を書いたりとかもできそう


mypy plugin


dropbox が SqlAlchemy 用の mypy-stub を提供してくれてます

python/mypy#3538
https://github.com/dropbox/sqlalchemy-stubs


mypy と PEP561


PEP 561 is

各ライブラリごとの型情報を外だしできるようにするための取り決め
これのおかげで numpy の stub も提供されている
https://github.com/numpy/numpy-stubs


typing extentions


議論されている typing の機能を追加して検証するための場所
https://github.com/python/typing/tree/master/typing_extensions
Protocol とかとか


User Defined Protocol

from typing import Iterable
from typing_extensions import Protocol

class SupportsClose(Protocol):
    def close(self) -> None:
       ...  # Empty method body (explicit '...')

class Resource:  # No SupportsClose base class!
    # ... some methods ...

    def close(self) -> None:
       self.resource.release()

def close_all(items: Iterable[SupportsClose]) -> None:
    for item in items:
        item.close()

close_all([Resource(), open('some/file')])  # Okay!
DuckTyping 的なメソッドのチェックができるように！！！

Thanks again!
implement	PEP
Python 3.0	3107
Python 3.5	484
Ptyhon 3.6	526
Python 3.7	560, 561, 562,563