Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save AayushSameerShah/b72be52fe2ff4d64ea572a9569afabd1 to your computer and use it in GitHub Desktop.
Save AayushSameerShah/b72be52fe2ff4d64ea572a9569afabd1 to your computer and use it in GitHub Desktop.
This will help adding a column from a table to another when we "just" want to glue it without the same column. Here, we would need to use the "row_index" which will behave like a same column.
import static org.apache.spark.sql.functions.*;
result = result.withColumn("row_index", row_number().over(Window.orderBy(monotonically_increasing_id())));
DF = DF.withColumn("row_index", row_number().over(Window.orderBy(monotonically_increasing_id())));
result = result.join(DF.withColumn("realBMI",col("BMI")).select("row_index", "realBMI"), result.col("row_index").equalTo(DF.col("row_index")));
result = result.drop("row_index");
result.show();
@AayushSameerShah
Copy link
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment