Created
December 9, 2019 11:33
-
-
Save lakshay-arora/f4241030c5d6eff434e30833d3d2096a to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# define stage 1: tokenize the tweet text | |
stage_1 = RegexTokenizer(inputCol= 'tweet' , outputCol= 'tokens', pattern= '\\W') | |
# define stage 2: remove the stop words | |
stage_2 = StopWordsRemover(inputCol= 'tokens', outputCol= 'filtered_words') | |
# define stage 3: create a word vector of the size 100 | |
stage_3 = Word2Vec(inputCol= 'filtered_words', outputCol= 'vector', vectorSize= 100) | |
# define stage 4: Logistic Regression Model | |
model = LogisticRegression(featuresCol= 'vector', labelCol= 'label') |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment