Skip to content

Instantly share code, notes, and snippets.

@bgweber
Created May 15, 2019 03:27
Show Gist options
  • Save bgweber/ed92ae2eb1f4613e33d76cdb24240f3e to your computer and use it in GitHub Desktop.
Save bgweber/ed92ae2eb1f4613e33d76cdb24240f3e to your computer and use it in GitHub Desktop.
# train a model, but first, pull everything to the driver node
df = spark_df.toPandas().drop(['user_id', 'partition_id'], axis = 1)
y_train = df['label']
x_train = df.drop(['label'], axis=1)
# use logistic regression
model = LogisticRegression()
model.fit(x_train, y_train)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment