Skip to content

Instantly share code, notes, and snippets.

@tejas-kr

tejas-kr/text_processing.py

Created April 23, 2021 05:05
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tejas-kr/836796fd18470f84650af95d950a2048 to your computer and use it in GitHub Desktop.
Save tejas-kr/836796fd18470f84650af95d950a2048 to your computer and use it in GitHub Desktop.
Download ZIP
text processing for nlp tasks (using nltk) [for spacy i will create soon]
Raw
text_processing.py
from nltk.corpus import stopwords
import string
def text_process(text):
nopunc = [char for char in text if char not in string.punctuation]
nopunc = ''.join(nopunc)
return [word for word in nopunc.split() if word.lower() not in stopwords.words('english')]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment