Skip to content

Instantly share code, notes, and snippets.

@prateekjoshi565

prateekjoshi565/text_normalization_elmo.py

Last active March 24, 2019 06:16
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save prateekjoshi565/55481ca27e0e2225881e59a13e6c7f6f to your computer and use it in GitHub Desktop.
Save prateekjoshi565/55481ca27e0e2225881e59a13e6c7f6f to your computer and use it in GitHub Desktop.
Download ZIP
text normalization elmo
Raw
text_normalization_elmo.py
# import spaCy's language model
nlp = spacy.load('en', disable=['parser', 'ner'])
# function to lemmatize text
def lemmatization(texts):
output = []
for i in texts:
s = [token.lemma_ for token in nlp(i)]
output.append(' '.join(s))
return output
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment