Skip to content

Instantly share code, notes, and snippets.

@Yuktha-Majella
Last active August 7, 2021 16:13
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Embed
What would you like to do?
Creating Bigrams and Trigrams in Gensim
import gensim.downloader as api
from gensim.models.phrases import Phrases
dataset = api.load("text8")
tokens = [word for word in dataset]
bigram_model = Phrases(tokens, min_count = 3, threshold = 10)
print(bigram_model[tokens[0]])
trigram_model = Phrases(bigram_model[data], threshold = 10)
print(trigram_model[bigram_model[data[0]]])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment