Skip to content

Instantly share code, notes, and snippets.

@megha444

megha444/nltk.py

Created October 13, 2020 16:28
Show Gist options
  • Save megha444/5ea336b8201fd8b536d0754b43bc7472 to your computer and use it in GitHub Desktop.
Save megha444/5ea336b8201fd8b536d0754b43bc7472 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
nltk.py
import nltk
from nltk.tokenize import word_tokenize
nltk.download ('punkt')
text = "This is a sample text to show word tokenization"
print(word_tokenize(text))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment