Skip to content

Instantly share code, notes, and snippets.

@richashworth

richashworth/spacy-tokenisation-example.py

Last active September 9, 2021 18:24
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save richashworth/51686d8dad069f428a901e8a145e78f0 to your computer and use it in GitHub Desktop.
Save richashworth/51686d8dad069f428a901e8a145e78f0 to your computer and use it in GitHub Desktop.
Download ZIP
Tokenisation Example
Raw
spacy-tokenisation-example.py
text = " ".join(list(jonathan_ross_segments))
sentences = nlp(text).sents
HTML(DataFrame(map(lambda x: str(x), sentences), columns=["Sentences"]).to_html(index=False)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment