Skip to content

Instantly share code, notes, and snippets.

@amankharwal
Created Sep 26, 2020
Embed
What would you like to do?
from pyspark.ml.evaluation import RegressionEvaluator
from pyspark.ml.recommendation import ALS
from pyspark.sql import Row
lines = spark.read.text("TEXT_FILE.txt").rdd
parts = lines.map(lambda row: row.value.split("::"))
liked_data = parts.map(lambda p: Row(COLUMN_1, COLUMN_2, like=like)
liked_df = spark.createDataFrame(liked_data)
(training, test) = liked_df.randomSplit([0.75, 0.25])
als = ALS(maxIter=4, regParam=0.05, users="COLUMN_1", items="COLUMN_2", ratingCol="like",
coldStartStrategy="drop")
model = als.fit(training)
predictions = model.transform(test)
evaluator = RegressionEvaluator(metricName="rmse", labelCol="like",
predictionCol="prediction")
rmse = evaluator.evaluate(predictions)
print("RMSE = " + str(rmse))
videoRecs = model.recommendForUsers(20)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment