Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
filter by days prior to today
val nDays = 450
df
.withColumn("date_merge", concat_ws("-", $"year", $"month", $"day"))
.withColumn("date", to_date(unix_timestamp($"date_merge", "yyyy-MM-dd").cast("timestamp")))
.drop("date_merge")
.where($"date" >= date_add(current_date(), -nDays)))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.