Skip to content

Instantly share code, notes, and snippets.

Avatar

amnrzv

  • London
View GitHub Profile
@amnrzv
amnrzv / nltk_tokenize.py
Last active Nov 1, 2017
A little example of NLTK's word and sentence tokenization. Output here: https://gist.github.com/amnrzv/2cbaad89e016acc0db410ec79a5ff40f
View nltk_tokenize.py
from nltk.tokenize import word_tokenize, sent_tokenize
text = "Hello, Mr. Jacobs. Nice to meet you!"
sentences = sent_tokenize(text)
words = word_tokenize(text)
print (sentences)
print (words)
You can’t perform that action at this time.