Skip to content

Instantly share code, notes, and snippets.

@1ambda

1ambda/df-array-collect-list.py

Created December 21, 2021 23:22
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save 1ambda/dc778d34024e1ef57fe1c4af862140ce to your computer and use it in GitHub Desktop.
Save 1ambda/dc778d34024e1ef57fe1c4af862140ce to your computer and use it in GitHub Desktop.
Download ZIP
Raw
df-array-collect-list.py
df\
    .where(col("brand").isNotNull())\
    .groupBy("brand")\
    .agg(
        collect_list("product_id").alias("product_id_list"),
        size(collect_list("product_id")).alias("product_id_list_count"),
        collect_set("product_id").alias("product_id_set"),
        size(collect_set("product_id")).alias("product_id_set_count"),
    )\
    .show()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment