Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save DianGermishuizen/86f63e7b2dbf417ce27362d3496e9525 to your computer and use it in GitHub Desktop.
Save DianGermishuizen/86f63e7b2dbf417ce27362d3496e9525 to your computer and use it in GitHub Desktop.
# pDataLakeContainer: "silver"
# pTableName": "Product"
# vDeltaTablePath: "/mnt/silver/WorldWideImporters/Batch/Product/"
vDeltaTableCreateStatement = 'CREATE TABLE IF NOT EXISTS ' \
+ pDataLakeContainer + '.' + pTableName + ' \n' \
+ 'USING DELTA ' + '\n' \
+ 'LOCATION \'' + vDeltaTablePath + '\' ' + '\n' \
+ 'PARTITIONED BY ( ProductCategory )' + '\n' \
+ 'AS' + '\n' \
+ 'SELECT * FROM sourceTemporaryView'
#Check final output
print('vDeltaTableCreateStatement: ' + vDeltaTableCreateStatement)
#Execute the SQL
spark.sql(vDeltaTableCreateStatement)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment