Skip to content

Instantly share code, notes, and snippets.

@abhishek-shrm

abhishek-shrm/eta_on_text_data-14.py

Created April 10, 2020 12:04
Show Gist options
  • Save abhishek-shrm/6020543d77392b3bdfd3a4d5dc41b0d5 to your computer and use it in GitHub Desktop.
Save abhishek-shrm/6020543d77392b3bdfd3a4d5dc41b0d5 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
eta_on_text_data-14.py
# Importing spacy
import spacy
# Loading model
nlp = spacy.load('en_core_web_sm',disable=['parser', 'ner'])
# Lemmatization with stopwords removal
df['lemmatized']=df['cleaned'].apply(lambda x: ' '.join([token.lemma_ for token in list(nlp(x)) if (token.is_stop==False)]))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment