Skip to content

Instantly share code, notes, and snippets.

@ijan10

ijan10/left_join_with_skew_key_example.py

Created February 28, 2019 12:07
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ijan10/b09841023327f989525007133bb5397f to your computer and use it in GitHub Desktop.
Save ijan10/b09841023327f989525007133bb5397f to your computer and use it in GitHub Desktop.
Download ZIP
Raw
left_join_with_skew_key_example.py
join_columns = ['year_week','line_item_id']
df_join = bin_packing.left_join_with_skew_key(spark_session=spark_session,
df_left_src=demand_supply_df,
df_right_src=li_sum_predict_requests_df,
left_col_names_list=join_columns,
right_col_names_list=join_columns,
brodcast_hint=True,
num_of_partitions=None)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment