Skip to content

Instantly share code, notes, and snippets.

@dmateusp

dmateusp/ExampleWithTransform.scala

Last active June 4, 2019 06:49
Show Gist options
  • Save dmateusp/93b3c7d97fac34fb4fa0945c37663a69 to your computer and use it in GitHub Desktop.
Save dmateusp/93b3c7d97fac34fb4fa0945c37663a69 to your computer and use it in GitHub Desktop.
Download ZIP
DataFrame.transform - Spark Function Composition - Methods with transform
Raw
ExampleWithTransform.scala
dfTransactions
.transform(extractPayerBeneficiary("details", _))
.transform(sumAmounts(_, date_trunc("day", col("ts")), col("details_beneficiary")))
.filter(col("sum(amount)") > 25)
.show
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment