Skip to content

Instantly share code, notes, and snippets.

@jonhilgart22

jonhilgart22/create column from UDF from withColumn pyspark

Created March 12, 2018 23:05
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jonhilgart22/7507cea1b2deadb6e9b6df42ff2e26d9 to your computer and use it in GitHub Desktop.
Save jonhilgart22/7507cea1b2deadb6e9b6df42ff2e26d9 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
create column from UDF from withColumn pyspark
from pyspark.sql.types import StringType
from pyspark.sql.functions import udf
maturity_udf = udf(lambda age: "adult" if age >=18 else "child", StringType())
df = sqlContext.createDataFrame([{'name': 'Alice', 'age': 1}])
df.withColumn("maturity", maturity_udf(df.age))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment