Skip to content

Instantly share code, notes, and snippets.

@syedhassaanahmed
Created April 15, 2019 21:40
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save syedhassaanahmed/3ac3b31a22f6c920a4e2e0f1c3997867 to your computer and use it in GitHub Desktop.
Save syedhassaanahmed/3ac3b31a22f6c920a4e2e0f1c3997867 to your computer and use it in GitHub Desktop.
Kusto query to generate N partition tables
let tablePrefix = "TimeseriesTable";
let partitions = 2;
//
let partitionTables = materialize(
range i from 0 to partitions - 1 step 1
| extend t = strcat(tablePrefix, "_", i)
);
//
let dropTables = toscalar(
partitionTables
| summarize makelist(t)
| project strcat(".drop tables (\n", strcat_array(list_t, ",\n"), "\n) ifexists")
);
//
let createTables = toscalar(
partitionTables
| project t = strcat(" ", t, " (Sensor:string, Timestamp:datetime, Value:real)")
| summarize makelist(t)
| project strcat(".create tables\n", strcat_array(list_t, ",\n"))
);
//
let alterFolder = toscalar(
partitionTables
| project t = strcat(".alter table ", t, " folder \"Partitioned Tables\"")
| summarize makelist(t)
| project strcat_array(list_t, "\n\n")
);
//
let updatePolicy = toscalar(
partitionTables
| project t = strcat(".alter-merge table ", t, " policy update @'[{\"IsEnabled\": true, \"Source\": \"", tablePrefix, "_Staging\", \"Query\": \"", tablePrefix, "_UpdateFunction(", i, ")\", \"IsTransactional\": true, \"PropagateIngestionProperties\": true}]'")
| summarize makelist(t)
| project strcat_array(list_t, "\n\n")
);
//
let unionFunction = toscalar(
partitionTables
| project t = strcat(" (", t, " | where hash(Sensor, 100) == ", i, ")")
| summarize makelist(t)
| project strcat(".create-or-alter function f", tablePrefix, "()", "\n{\n union\n", strcat_array(list_t, ",\n"), "\n}")
);
//
let scriptArray=pack_array(
dropTables,
createTables,
alterFolder,
updatePolicy,
unionFunction);
//
print strcat_array(scriptArray, "\n\n")
@syedhassaanahmed
Copy link
Author

Output of above query is

.drop tables (
TimeseriesTable_0,
TimeseriesTable_1
) ifexists

.create tables
  TimeseriesTable_0 (Sensor:string, Timestamp:datetime, Value:real),
  TimeseriesTable_1 (Sensor:string, Timestamp:datetime, Value:real)

.alter table TimeseriesTable_0 folder "Partitioned Tables"

.alter table TimeseriesTable_1 folder "Partitioned Tables"

.alter-merge table TimeseriesTable_0 policy update @'[{"IsEnabled": true, "Source": "TimeseriesTable_Staging", "Query": "TimeseriesTable_UpdateFunction(0)", "IsTransactional": true, "PropagateIngestionProperties": true}]'

.alter-merge table TimeseriesTable_1 policy update @'[{"IsEnabled": true, "Source": "TimeseriesTable_Staging", "Query": "TimeseriesTable_UpdateFunction(1)", "IsTransactional": true, "PropagateIngestionProperties": true}]'

.create-or-alter function fTimeseriesTable()
{
  union
  (TimeseriesTable_0 | where hash(Sensor, 100) == 0),
  (TimeseriesTable_1 | where hash(Sensor, 100) == 1)
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment