Skip to content

Instantly share code, notes, and snippets.

View jalockie's full-sized avatar

Jason Lockie jalockie

View GitHub Profile
@jalockie
jalockie / index.html
Last active May 3, 2021 22:16
Schemaless Object Builder using HTML and Javascript
<html>
<head>Schemaless Object Builder</head>
<body>
<div>
<div>
<input id="new_field_name"></input>
<button onclick="add_field()">Add Field</button>
<input id="new_calculated_field_name"></input>
<button onclick="add_calculated_field()">Add Calculated Field</button>
</div>
@jalockie
jalockie / table_match.lua
Last active May 3, 2021 22:02
Lua Helper for Table Matching
-- The "#" operator in Lua does not count the
-- number of items in an non-continous numbered list
-- i.e tbl={[1]="i1", [9]="i2"} print(#tbl) -- returns 1 (not 2)
function len(...)
if not arg[1] then return 0 end
if arg.n == 1 and type(arg[1]) == 'table' then
local c=0
for i, _ in pairs(arg[1]) do c=c+1 end
return c
@jalockie
jalockie / azure_table_storage.py
Last active April 29, 2021 09:38
Collating News Articles from RSS Feeds into Azure Table Storage
import os, uuid
from datetime import datetime
from azure.cosmosdb.table.tableservice import TableService
from azure.common import AzureMissingResourceHttpError
from azure.core.exceptions import ResourceNotFoundError
# resolve table connection
account_name = os.getenv('AZURE_STORAGE_ACCOUNT_NAME')
@jalockie
jalockie / Expression 1: lookup column mapping activity
Last active April 29, 2021 08:55
Dynamic Schema Mapping in Azure Data Factory Copy Data Activity
;with cols as (
select
REPLACE(column_name, (' ', '__wspc__') as new_name,
column_name as old_name
from INFORMATION_SCHEMA.columns
where table_name = '@{pipeline().parameters.SOURCE_TABLE}'
and table_schema = '@{pipeline().parameters.SOURCE_SCHEMA}'
)
select ' |||'+old_name+'||'+new_name+'|' as mapping
from cols;