View win_rowsBetween_agg.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val rows_between_df = empsalary.withColumn("max_salary", max("salary").over(winSpec)) | |
rows_between_df.show() |
View win_rowsBetween_spec.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val winSpec = Window.partitionBy("depName") | |
.orderBy("salary").rowsBetween(-1, 1) |
View win_rangeBetween_agg_boundary.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val winSpec = Window.partitionBy("depName").orderBy("salary") | |
.rangeBetween(300L, Window.unboundedFollowing) | |
val range_unbounded_df = empsalary.withColumn("max_salary", max("salary").over(winSpec)) | |
range_unbounded_df.show() |
View win_rangeBetween_agg.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val range_between_df = empsalary.withColumn("max_salary", max("salary").over(winSpec)) | |
range_between_df.show() |
View win_rangeBetween.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val winSpec = Window.partitionBy("depName") | |
.orderBy("salary") | |
.rangeBetween(100L, 300L) |
View win_lead.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val winSpec = Window.partitionBy("depName").orderBy("salary") | |
val lead_df = | |
empsalary.withColumn("lead", lead("salary", 2).over(winSpec)) | |
lead_df.show() |
View win_lag.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val winSpec = Window.partitionBy("depName").orderBy("salary") | |
val lag_df = | |
empsalary.withColumn("lag", lag("salary", 2).over(winSpec)) | |
lag_df.show() |
View win_cume_dist.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val winSpec = Window.partitionBy("depName").orderBy("salary") | |
val cume_dist_df = | |
empsalary.withColumn("cume_dist",cume_dist().over(winSpec)) | |
cume_dist_df.show() |
View win_ntile.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val ntile_df = empsalary.withColumn("ntile", ntile(3).over(winSpec)) | |
ntile_df.show() |
View win_percent_rank.scala
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val percent_rank_df = empsalary.withColumn("percent_rank", percent_rank().over(winSpec)) | |
percent_rank_df.show() |
NewerOlder