Skip to content

Instantly share code, notes, and snippets.

View DaveRuijter's full-sized avatar

Dave Ruijter DaveRuijter

49 followers · 1 following
View GitHub Profile
@DaveRuijter
DaveRuijter / ADF_Scale_Azure_AS.json
Last active December 24, 2018 10:58
Azure Data Factory pipeline definition with a Web Activity to scale an Azure Analysis Services server. The pipeline is parameterized. The call to AAS is synchronous, hence it waits for the scale operation to complete.
{
"name": "ADF_Scale_Azure_AS",
"properties": {
"activities": [
{
"name": "00_Scale_Azure_AS",
"type": "WebActivity",
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
@DaveRuijter
DaveRuijter / RemoveServicePrincipalFromPowerBIWorkspaces.ps1
Created September 21, 2020 11:23
Script to remove Service Principal from Power BI workspaces
# =================================================================================================================================================
## This script will remove the given Service Principal from Power BI workspaces
## It will first ask for the (correct) ObjectId of the Service Principal
## Then it will ask for the credentials of a Power BI Service Administrator
# =================================================================================================================================================
## Parameters
# Remove the Service Principal from workspaces that are in Premium capacity?
$RemoveFromPremiumCapacityWorkspaces = $true
@DaveRuijter
DaveRuijter / PowerBIEmbeddedResumeCapacity.ps1
Created November 19, 2018 22:28
PowerShell script that enables resuming of an Azure Power BI Embedded Capacity.
<#
.SYNOPSIS
Resume an Azure Power BI Embedded Capacity using Azure Automation.
.DESCRIPTION
This Azure Automation runbook enables resuming of an Azure Power BI Embedded Capacity.
.PARAMETER resourceGroupName
Name of the resource group to which the capacity is assigned.
@DaveRuijter
DaveRuijter / pull_request_template.md
Created October 3, 2021 19:12
Azure DevOps Pull Request template with type of update

Type of update

  • Fix
  • Feature
  • Big feature
@DaveRuijter
DaveRuijter / pipeline-pullrequest-administration.yml
Created October 3, 2021 19:13
Azure DevOps YAML pipeline for Pull Request administration, with SemVer based on the PR description
trigger:
- none
pool:
vmImage: ubuntu-latest
## Job to calculate semantic version
jobs:
- job: CalculateVersion
displayName: Semantic versioning
@DaveRuijter
DaveRuijter / release-notes-template.md
Created October 3, 2021 20:23
Template file for automatic release notes

🚀 Release v{{buildDetails.buildNumber}}

Associated Pull Requests ({{pullRequests.length}})

{{#forEach pullRequests}}

  • Pull Request [#{{this.pullRequestId}} {{this.title}}]({{replace (replace this.url "_apis/git/repositories" "_git") "pullRequests" "pullRequest"}}) {{/forEach}}

Associated Azure Board Items ({{workItems.length}})

{{#forEach this.workItems}}

  • Item [#{{this.id}} {{lookup this.fields 'System.Title'}}]({{replace this.url "_apis/wit/workItems" "_workitems/edit"}})
@DaveRuijter
DaveRuijter / vacuum_all_databricks_delta_tables.py
Created October 21, 2021 19:21
Script that iterates all tables in all databases that match the given database name filter, and performs the VACUUM command on each table.
database_names_filter = "20_silver*"
try:
dbs = spark.sql(f"SHOW DATABASES LIKE '{database_names_filter}'").select("databaseName").collect()
dbs = [(row.databaseName) for row in dbs]
for database_name in dbs:
print(f"Found database: {database_name}, performing actions on all its tables..")
tables = spark.sql(f"SHOW TABLES FROM `{database_name}`").select("tableName").collect()
tables = [(row.tableName) for row in tables]
for table_name in tables:
#spark.sql(f"ALTER TABLE `{database_name}`.`{table_name}` SET TBLPROPERTIES ('delta.logRetentionDuration'='interval 30 days', 'delta.deletedFileRetentionDuration'='interval 7 days')")
@DaveRuijter
DaveRuijter / backup_sta_lifecycle_policy_rules.json
Created October 21, 2021 19:27
This policy is used on the storage account that contains the backup copies of the Data Lake. This will apply retention of 60 days to the weekly backups, and retention of 30 days to the daily (incremental) backups.
{
"rules": [
{
"enabled": true,
"name": "weeklybackupsrule",
"type": "Lifecycle",
"definition": {
"actions": {
"baseBlob": {
"delete": {
@DaveRuijter
DaveRuijter / data_lake_sta_lifecycle_policy_rules.json
Created October 21, 2021 19:28
This policy is used on the storage account of the Data Lake. This will ensure new data in the dls/05_store/_archive folder of the lake is automatically assigned to the cool access tier.
{
"rules": [
{
"enabled": true,
"name": "daily-moving-data-lake-store-archive-to-cool",
"type": "Lifecycle",
"definition": {
"actions": {
"baseBlob": {
"tierToCool": {
@DaveRuijter
DaveRuijter / stage-backup-dls.yml
Created October 21, 2021 20:28
This YAML file is part of the Data Lake Backup Strategy
parameters:
- name: dependsOnStage
type: string
- name: triggerPeriod
type: string
- name: environment
type: string
- name: backupStore
type: boolean
- name: backupBronze