Skip to content

Instantly share code, notes, and snippets.

View apurv06's full-sized avatar

Apurv apurv06

View GitHub Profile
# Databricks notebook source
# =============================================================================
# RELTIO → ADLS EXPORT PIPELINE
# =============================================================================
# Responsibility : Authenticate with Reltio, trigger export, poll until
# complete, stage .gz files to ADLS, read into PySpark,
# write to final ADLS path as Delta / Parquet / CSV.
# Auth : Storage account key configures Spark at runtime.
# No Azure SDK needed — dbutils.fs.cp handles staging,
# dbutils.fs.rm handles cleanup.