Skip to content

Instantly share code, notes, and snippets.

@mrchristine
Created June 29, 2016 19:55
Show Gist options
  • Save mrchristine/7dcf68380896894fd30e94775f9f2509 to your computer and use it in GitHub Desktop.
Save mrchristine/7dcf68380896894fd30e94775f9f2509 to your computer and use it in GitHub Desktop.
Databricks Rest API to delete job schedules.
#!/bin/bash
# catch ctrl+c handler
trap ctrl_c_cleanup INT
function ctrl_c_cleanup() {
echo "** Interrupt handler caught"
rm -rf $job_file
}
IFS=$'\n' # make newlines the only separator
while getopts ":z" opt; do
case $opt in
z)
dryrun=true
;;
\?)
echo "Running deletion job.\n"
;;
esac
done
# Define input parameters
user=`echo $DBCLOUD_USER`
pass=`echo $DBCLOUD_PASSWORD`
# Format: https://shardname.cloud.databricks.com
shard=`echo $DBCLOUD_SHARD`
# Fetch the job list and use jq to parse the resulting json
job_file="dbc_reset_jobid_list_$$.txt"
## Find jobs that have a defined schedule
curl --silent -u $user:$pass $shard/api/2.0/jobs/list | jq '.jobs[] | select( .settings.schedule ) | .job_id' > $job_file
for j in `cat $job_file`
do
echo "Current job_id: $j"
## Use the /2.0/jobs/reset API to reset the schedule policy for these jobs
## This query uses jq to delete the schedule and created timestamp to reapply these settings
newSettingJson=`curl --silent -u $user:$pass $shard/api/2.0/jobs/get?job_id=$j | jq '. | del(.settings.schedule) | del(.created_time)' | sed 's/settings/new_settings/' | tr -d '\r\n' `
if [ "$dryrun" = true ]; then
echo "'$newSettingJson'"
sleep 2
else
echo "'$newSettingJson'"
curl -u $user:$pass -H "Content-Type: application/json" -d "$newSettingJson" $shard/api/2.0/jobs/reset
fi
done
rm -rf $job_file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment