Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
"""
@ github.com/ahmedshahriar
This code will remove remove non-English words from text
"""
import nltk
# download nltk english corpus
nltk.download('wordnet')
wordnet = set(nltk.corpus.wordnet.words())
words = "love ে ী ে া ে ু ্ া োঁ ে"
words_cleaned = " ".join(w for w in nltk.wordpunct_tokenize(words) if w.lower() in wordnet )
print(words_cleaned)
# output 'love'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment