esho

Keybase proof

I hereby claim:

• I am esho on github.
• I am esho2019 (https://keybase.io/esho2019) on keybase.
• I have a public key ASCe6ExZNyS6Rq7Gds4aUcPOYLx55oDJuCU_WFZR3sScJwo

To claim this, I am signing this object:

esho

import pytest
import tokenizer


def test_string_with_spaces():
    results = tokenizer.tokenize('a string of words')
    assert results[0].token == 'a' and results[0].type == tokenizer.TokenType.STRING
    assert results[1].token == 'string' and results[1].type == tokenizer.TokenType.STRING
    assert results[2].token == 'of' and results[2].type == tokenizer.TokenType.STRING
    assert results[3].token == 'words' and results[3].type == tokenizer.TokenType.STRING