Skip to content

Instantly share code, notes, and snippets.

@mlai-demo

mlai-demo/te6.py

Created August 11, 2019 22:42
Show Gist options
  • Save mlai-demo/93e21578748786b44fef4206540a3eb0 to your computer and use it in GitHub Desktop.
Save mlai-demo/93e21578748786b44fef4206540a3eb0 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
te6.py
unique = set(words)
print("The tokenized text is {} words long, has {} unique words and {} letters on average".format
(len(words), len(unique), round(sum(len(word) for word in words)/len(words),2)))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment