Skip to content

Instantly share code, notes, and snippets.

View raven-rock's full-sized avatar

Levi Gillette raven-rock

View GitHub Profile
colorscheme jellybeans
filetype plugin indent on " This does though, including ruby.
syntax on " turn on syntax hilighting
set hidden " allow hidden buffers, i.e. unsaved buffers in background
" set mouse +=a " enable mouse support in terminal -> http://stackoverflow.com/questions/2149456/reasons-for-using-command-line-vim-versus-gui-vim
" set clipboard=unnamedplus
set clipboard=unnamed " enable yanks to go to OS clipboard.
set nu " line numbers on
set ignorecase " ignore case in searches
set smartcase " ignore case if search pattern is all lowercase, case-sensitive otherwise
@raven-rock
raven-rock / core.rb
Created July 19, 2021 20:35
Deterministic Key (DK) generation algorithm for scalars, rows, and row sets (batches)
require 'set'
require 'digest/md5'
# Introducing the concept of the *determistic key* (DK)!
# The DK is a deterministic, order-independent (when needed), fixed-length hash
# digest of a scalar value or a collection of values. Utilizing hashing
# algorithms allows for deterministic key generation in distributed systems, and
# thus very parallel-friendly asychronous generation by multiple cores or
# machines.
@raven-rock
raven-rock / pit-psa-postgres-trigger.sql
Created June 21, 2021 14:54
Trigger-powered Point-In-Time table (PIT) for persistent staging area (PSA)
-- Thanks to: https://stackoverflow.com/questions/58655766/create-trigger-for-update-another-row-automatically-on-same-table-using-postgres
/*
Features:
- DONE Auto-updating xtime.
- DONE Insert custom vtime.
- DONE Insert custom vtime & xtime.
- DONE Idempotency.
- DONE Vtime insertion order is independent.
- DONE Do-not-insert new row when attrs is identical to previous row.
@raven-rock
raven-rock / pit-psa-via-cte_example_postgres.sql
Last active June 21, 2021 14:57
Example of PIT-PSA (Point-In-Time Persistent Staging Area) system in PostgreSQL, implemented via Common Table Expressions (CTE), for efficient ingestion and change data capture (CDC). Suitable for as base for a scriptable star schema.
/* NOTES
This file contains an efficient, script-ready algorithm to upsert data into a
persistent, point-in-time (PIT) staging table (PSA) in PostgreSQL.
It is assumed that the PSA will provide a virtually immutable datastore of all
ingested data in it's original form, e.g., as it appeared in a source CSV files
or source tables at certain points in time. The PSA in turn provides the basis
for dynamically deriving useful analytical datasets, such as star/snowflake
schemas, which can be blown away and rebuilt as needed to support ever changing
@raven-rock
raven-rock / postgresql_temporal_table_stage_update_example.sql
Created June 2, 2021 06:24
Example of how to update a temporal database table (aka Type 2 slowly changing dimension) from staged data using PostgreSQL
-- Adapted from my MySQL SCD2 version here: https://gist.github.com/raven-rock/ff715978ae75fdf67e42f764ed1566c3.
--
-- Meant to be a more general https://en.wikipedia.org/wiki/Temporal_database technique, not limited
-- to dimensional modeling's SCD2 use case.
-- Set up scd
drop table if exists scd_widget;
create temporary table scd_widget
@raven-rock
raven-rock / postgresql_hashes_stored_as_binary_cheatsheet.sql
Created May 4, 2021 18:45
postgresql_hashes_stored_as_binary_cheatsheet.sql
WITH xs as (
select '' as x -- empty string
union all select 'a' -- lowercase letter
union all select 'A' -- uppercase
union all select 'aa' -- two ascii letters
union all select 'á' -- 'a' with accent
union all select '的' -- one CJK character, stored as multiple bytes
union all select 'deadbeef'
union all select 'DEADBEEF'
)
@raven-rock
raven-rock / README.md
Created March 16, 2021 16:01
Windows 10: showing seconds in the system tray clock

Showing seconds in the system tray clock in Windows 10 is accomplished via a registry hack. It is based a boolean DWORD entry called ShowSecondsInSystemClock. The entry doesn't exist by default, so must created by hand, or you can use the code below to create two convenient registry script files, one for enabling seconds and another for disabling.

@raven-rock
raven-rock / movies-dmenu
Created September 22, 2020 22:09
movies-dmenu: Script to let user pick a movie to watch via dmenu in their ~/Movies folder, sorted newest videos first based on file modification time
#!/usr/bin/env bash
# handy to bind to global shortcut like Control-Shift-m
cd ~
nohup find Movies/ -type f |
ruby -e '
STDIN.each_line.map(&:chomp)
.select{|f| File.extname(f) =~ /(?:mp3|mp4|wmv|mov|webm|m4v|mkv|avi|flv|xspf)/i}.sort_by{|f| File.mtime f }
@raven-rock
raven-rock / mysql_scd2_stage_update_example.sql
Last active March 26, 2020 02:00
Example of how to update a Type 2 slowly changing dimension table from staged data using MySQL
/*
In this example,
- rowid is simply a rowid for the scd. Perhaps to avoid confusion we actually always name this "rowid".
- widget_code is the durable natural key (could be a composite set of columns)
- widget_name is the change-tracked attribute (could be set of columns)
- widget_color is another the change-tracked attribute (could be set of columns)
*/
@raven-rock
raven-rock / timezones.org
Created February 8, 2020 01:23
Time Zone data
tz_database_name(pk)country_codelatitude_longitudeportion_of_country_coveredstatusutc_offsetutc_dst_offsetnotes
Africa/AbidjanCI+0519−00402Canonical+00:00+00:00