Skip to content

Instantly share code, notes, and snippets.

@amankharwal

amankharwal/stopwords.py Secret

Created March 28, 2021 07:13
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save amankharwal/bb81013488a4c1221ed734a5dbc0375b to your computer and use it in GitHub Desktop.
Save amankharwal/bb81013488a4c1221ed734a5dbc0375b to your computer and use it in GitHub Desktop.
Download ZIP
Raw
stopwords.py
import nltk
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize
nltk.download('stopwords')
nltk.download('punkt')
text = "Hi, My name is Aman Kharwal, I am here to guide you to your journey in Machine Learning for free."
tokens = word_tokenize(text)
tokenization = [word for word in tokens if not word in stopwords.words('english')]
print(tokens)
print(tokenization)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment