Skip to content

Instantly share code, notes, and snippets.

@sabaelhilo

sabaelhilo/unique_id.py Secret

Created June 7, 2019 17:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sabaelhilo/8624ecfb0142905ce895f2015c91a12b to your computer and use it in GitHub Desktop.
Save sabaelhilo/8624ecfb0142905ce895f2015c91a12b to your computer and use it in GitHub Desktop.
Download ZIP
unique id demo
Raw
unique_id.py
uuid_udf = udf(lambda: uuid_generator(), StringType())
df = skewed_df.withColumn("u_id", uuid_udf())\
.repartition(num_partitions, 'u_id')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment