Skip to content

Instantly share code, notes, and snippets.

@James-McNeill

James-McNeill/03_nlp_token_lemma_stopword.py

Created July 16, 2021 16:02
Show Gist options
  • Save James-McNeill/142deb6f9772557a41ec2d85ccb6d06d to your computer and use it in GitHub Desktop.
Save James-McNeill/142deb6f9772557a41ec2d85ccb6d06d to your computer and use it in GitHub Desktop.
Download ZIP
Raw
03_nlp_token_lemma_stopword.py
# Reviewing the token, lemma and stopword for each token (item)
print(f"Token \t\tLemma \t\tStopword".format('Token', 'Lemma', 'Stopword'))
print("-"*40)
# Review the first 20 values to test the output
for token in doc[:20]:
print(f"{str(token)}\t\t{token.lemma_}\t\t{token.is_stop}\t\t{len(token)}")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment