Skip to content

Instantly share code, notes, and snippets.

@amankharwal

amankharwal/nltk_tutorial.py Secret

Created April 11, 2021 07:19
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save amankharwal/eef1e4ff66d87b51b7622f5301384cf4 to your computer and use it in GitHub Desktop.
Save amankharwal/eef1e4ff66d87b51b7622f5301384cf4 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
nltk_tutorial.py
import nltk
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize
nltk.download('stopwords')
nltk.download('punkt')
text = "Hi, My name is Aman Kharwal, I am here to guide you to your journey in Machine Learning for free."
tokens = word_tokenize(text)
tokenization = [word for word in tokens if not word in stopwords.words('english')]
print(tokens)
print(tokenization)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment