Martin Zurita Gorgoras
Gorgoras
/ checkIdleDatasets.py
Last active
November 25, 2020 18:15
Check for Datasets not being used in any Pipeline in Azure Data Factory. Just clone the repo, then set repo_path, ADF name, and run!
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import os | |
| # set repo location and ADF name | |
| repo_path = "D:/Work/Python/Azure" | |
| dataFactory_name = "Highwayman-ADFv2" | |
| full_path = "/".join([repo_path, dataFactory_name]) | |
| # get list of dataset and pipeline files | |
| datasets = os.listdir("{}/dataset".format(full_path)) | |
| pipelines = os.listdir("{}/pipeline".format(full_path)) |