Skip to content

Instantly share code, notes, and snippets.

@JedIV
Last active January 29, 2019 00:20
Show Gist options
  • Save JedIV/bc02ee7364bebf166017c35e96590601 to your computer and use it in GitHub Desktop.
Save JedIV/bc02ee7364bebf166017c35e96590601 to your computer and use it in GitHub Desktop.
Checks the metrics of a dataset and conditionally runs a scenario
from dataiku.scenario import Scenario
import dataiku
### Set up objects
# The Scenario object is the main handle from which you initiate steps
scenario = Scenario()
# The Project that we'll be working with
# set up an api client
client = dataiku.api_client()
# Get the current project
proj = client.get_project(dataiku.default_project_key())
###
### Set up input variables
# dataset we care about
ds = "customers_copy_entire"
# Scenario we want to conditionally run
scen = "partition_update"
# Metric ID
metric = 'records:COUNT_RECORDS'
###
### Use APIs to build and run scenarios
# Building a dataset
scenario.build_dataset(ds)
# Compute metrics for accurate checks
scenario.compute_dataset_metrics(ds)
# Run checks
# get metric from dataset
client = dataiku.api_client()
metrics = proj.get_dataset(ds).get_last_metric_values()
check_info = metrics.get_metric_by_id(metric)['lastValues'][0]['value']
if (check_info < 1000):
scenario.run_scenario("partition_update")
elif(check_info < 10000):
print("do something else")
else:
print("Adios")
###
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment