Skip to content

Instantly share code, notes, and snippets.

@icexelloss
Last active November 20, 2017 15:53
Show Gist options
  • Save icexelloss/cbca325fdc2456b7a3f05f713c1b1a36 to your computer and use it in GitHub Desktop.
Save icexelloss/cbca325fdc2456b7a3f05f713c1b1a36 to your computer and use it in GitHub Desktop.
from pyspark.sql.functions import udf
# Use udf to define a row-at-a-time udf
@udf('double')
# Input/output are both a single double value
def plus_one(v):
return v + 1
df.withColumn('v2', plus_one(df.v))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment