Skip to content

Instantly share code, notes, and snippets.

@NeerajBhadani

NeerajBhadani/arr_arrays_zip.scala

Created May 25, 2020 10:06
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save NeerajBhadani/c92285efcc0a8b64cac5e673204b550e to your computer and use it in GitHub Desktop.
Save NeerajBhadani/c92285efcc0a8b64cac5e673204b550e to your computer and use it in GitHub Desktop.
Download ZIP
arrays_zip
Raw
arr_arrays_zip.scala
// remove element "2" from array column "array_col2"
val temp_df = full_df
.withColumn("new_array_col", array_remove($"array_col2",2))
// zip column "array_col1" with newly created column "new_array_col"
val arr_zip_df = temp_df
.withColumn("result", arrays_zip($"array_col1", $"new_array_col"))
.select("array_col1", "new_array_col", "result")
arr_zip_df.show(truncate=false)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment