Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Code snippet for demonstrating Delta Lake
%python
# Create a temparory dataset
data = spark.range(0, 50)
data.write.format("delta").save("/tmp/myfirst-delta-table")
# Read the data
df = spark.read.format("delta").load("/tmp/myfirst-delta-table")
df.show()
# Updating the dataset
data = spark.range(51, 100)
data.write.format("delta").mode("overwrite").save("/tmp/myfirst-delta-table")
# Read the data
df = spark.read.format("delta").load("/tmp/myfirst-delta-table")
df.show()
# Read the older version of data
df = spark.read.format("delta").option("versionAsOf", 0).load("/tmp/myfirst-delta-table")
df.show()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.