Skip to content

Instantly share code, notes, and snippets.

@Yuktha-Majella

Yuktha-Majella/create_bigrams_trigrams

Last active August 7, 2021 16:13
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Yuktha-Majella/a91725a89df8f29663489462ae50de3a to your computer and use it in GitHub Desktop.
Save Yuktha-Majella/a91725a89df8f29663489462ae50de3a to your computer and use it in GitHub Desktop.
Download ZIP
Creating Bigrams and Trigrams in Gensim
Raw
create_bigrams_trigrams
import gensim.downloader as api
from gensim.models.phrases import Phrases
dataset = api.load("text8")
tokens = [word for word in dataset]
bigram_model = Phrases(tokens, min_count = 3, threshold = 10)
print(bigram_model[tokens[0]])
trigram_model = Phrases(bigram_model[data], threshold = 10)
print(trigram_model[bigram_model[data[0]]])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment