Skip to content

Instantly share code, notes, and snippets.

@francisco-ltech

francisco-ltech/deelta-duckdb.py

Created November 7, 2022 10:53
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save francisco-ltech/5fda45f57ca9eb072db0c4c2172aa047 to your computer and use it in GitHub Desktop.
Save francisco-ltech/5fda45f57ca9eb072db0c4c2172aa047 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
deelta-duckdb.py
import duckdb
from deltalake import DeltaTable
# Read delta table using delta lake Python package
dt = DeltaTable(“abfs://container/path-to-delta-table/my-table”, storage_options=storage_options)
# Convert to PyArrow dataset
ds = dt.to_pyarrow_dataset()
# Connect to in-memory duckdb database
conn = duckdb.connect()
# Execute a query against the PyArrow dataset
results = conn.execute("SELECT … FROM ds").arrow()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment