Skip to content

Instantly share code, notes, and snippets.

@shubham-singh-ss

shubham-singh-ss/gensim1.py

Last active July 5, 2019 12:25
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save shubham-singh-ss/6d56faa5c742e6d2d58c865029386e8e to your computer and use it in GitHub Desktop.
Save shubham-singh-ss/6d56faa5c742e6d2d58c865029386e8e to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gensim1.py
from gensim.utils import tokenize
text = """Founded in 2002, SpaceX’s mission is to enable humans to become a spacefaring civilization and a multi-planet
species by building a self-sustaining city on Mars. In 2008, SpaceX’s Falcon 1 became the first privately developed
liquid-fuel launch vehicle to orbit the Earth."""
list(tokenize(text))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment