Skip to content

Instantly share code, notes, and snippets.

@1ambda
Created January 2, 2022 01:35
Show Gist options
  • Save 1ambda/c17fd85d0052d84df4c80414f00aa4f0 to your computer and use it in GitHub Desktop.
Save 1ambda/c17fd85d0052d84df4c80414f00aa4f0 to your computer and use it in GitHub Desktop.
# repartition 이 컬럼 기준 없이 되었으므로
# 동일한 property_id (e.g., 2101) 가 여러 Partition = Connection 에 나누어 Insert
df\
.repartition(10)\
.write\
.mode("append")\
.format("jdbc")\
.option("numPartitions", "10")
# property_id 기준으로 Partition 이 나뉘고
# 서로 다른 Partition = Connection 들은 동일한 property_id 값을 가지지 않습니다.
df\
.repartition(10, "property_id")\
.write\
.mode("append")\
.format("jdbc")\
.option("numPartitions", "10")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment