Skip to content

Instantly share code, notes, and snippets.

@MaartenGr

MaartenGr/tfidf_embeddings.py

Created January 5, 2021 12:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save MaartenGr/7b0f1ab542a3360b9e9b73c3331178d9 to your computer and use it in GitHub Desktop.
Save MaartenGr/7b0f1ab542a3360b9e9b73c3331178d9 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
tfidf_embeddings.py
from bertopic import BERTopic
from sklearn.feature_extraction.text import TfidfVectorizer
# Create TF-IDF sparse matrix
vectorizer = TfidfVectorizer(min_df=5)
embeddings = vectorizer.fit_transform(docs)
# Model
model = BERTopic(stop_words="english")
topics, probabilities = model.fit_transform(docs, embeddings)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment