Skip to content

Instantly share code, notes, and snippets.

@aialenti
Created October 28, 2019 00:28
Show Gist options
  • Save aialenti/353f9b67354c3fb3f737aa45c8048866 to your computer and use it in GitHub Desktop.
Save aialenti/353f9b67354c3fb3f737aa45c8048866 to your computer and use it in GitHub Desktop.
al t0 = System.nanoTime()
// Do repartitioning
fact_table = fact_table.repartition(200, col("uniformly_distributed_column"))
// Create the Execution Plan
fact_table = fact_table.join(broadcast(dimension_table2),
fact_table.col("dimension_2_id") === dimension_table.col("id"))
// Perform an action to run the execution
fact_table.count
val t1 = System.nanoTime()
println("Elapsed time: " + (t1 - t0)/10e8 + "s")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment