Skip to content

Instantly share code, notes, and snippets.

@halfak

halfak/line_breaks.py

Last active August 6, 2020 15:29
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save halfak/138be6c0edbcd2d987be5c403ab62437 to your computer and use it in GitHub Desktop.
Save halfak/138be6c0edbcd2d987be5c403ab62437 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
line_breaks.py
self._cjk_processing(tokenized_text, language=max_char_lang_frac, token_class=token_class)
# TO
self._cjk_processing(
tokenized_text, language=max_char_lang_frac, token_class=token_class)
# OR
self._cjk_processing(
tokenized_text, language=max_char_lang_frac,
token_class=token_class)
Raw
list_comprehension.py
text = "".join([tok[:] for tok in tokenized_text if tok.type == 'cjk_word'])
# TO
text = "".join([tok[:]
for tok in tokenized_text
if tok.type == 'cjk_word'])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment