Skip to content

Instantly share code, notes, and snippets.

View robcowie's full-sized avatar

Rob Cowie robcowie

  • Recycleye
  • Leeds/London, United Kingdom
View GitHub Profile
@robcowie
robcowie / git-branching-diagram.md
Created May 4, 2021 17:42 — forked from bryanbraun/git-branching-diagram.md
Example Git Branching Diagram

Example Git Branching Diagram

You can use this diagram as a template to create your own git branching diagrams. Here's how:

  1. Create a new diagram with diagrams.net (formerly draw.io)
  2. Go to File > Open From > URL
  3. Insert this url (it points to the xml data below): https://gist.githubusercontent.com/bryanbraun/8c93e154a93a08794291df1fcdce6918/raw/0378fda86483def0a9680270c02907a9cbaf7615/template-data.xml
  4. Customize as needed for your team.

@robcowie
robcowie / README.md
Created April 16, 2021 10:36 — forked from sandys/Fastapi-sqlalchemy-pydantic-dataclasses-reloadable-logging.md
fastapi with python 3.7 dataclasses - used to create both sqlalchemy and pydantic models simultaneously

cmdline

poetry run gunicorn testpg:app -p 8080 --preload --reload --reload-engine inotify -w 10 -k uvicorn.workers.UvicornWorker --log-level debug --access-logfile - --error-logfile - --access-logformat "SSSS - %(h)s %(l)s %(u)s %(t)s \"%(r)s\" %(s)s %(b)s \"%(f)s\" \"%(a)s"

How to quickly run postgres (using docker)

docker run --network="host" -it --rm --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -e PGDATA=/var/lib/postgresql/data/pgdata -v /tmp/pgdata2:/var/lib/postgresql/data -e POSTGRES_USER=test postgres

This command will quickly start postgres on port 5432 and create a database test with user test and password mysecretpassword

@robcowie
robcowie / python_dict_donfig_example.py
Created November 17, 2020 16:22
Short dict config example
import logging
LOGGING_CONFIG = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'sane': {
'format': '%(levelname)-8s [%(asctime)s] %(name)s: %(message)s'
},
},
@robcowie
robcowie / versioning_trigger.sql
Last active November 13, 2020 16:19
Postgresql trigger to write snapshots to a history table
CREATE OR REPLACE FUNCTION versioning() RETURNS TRIGGER AS $$
DECLARE
hist_tbl text := TG_TABLE_NAME || '_history';
BEGIN
IF (TG_OP = 'UPDATE') THEN
NEW.sys_period = tstzrange(CURRENT_TIMESTAMP, NULL);
OLD.sys_period = tstzrange(lower(OLD.sys_period), CURRENT_TIMESTAMP);
EXECUTE format('INSERT INTO %I SELECT ($1).*', hist_tbl) USING OLD;
RETURN NEW;
@robcowie
robcowie / rounded_div_function.sql
Created October 12, 2020 13:58
Postgresql division with rounding function
-- Handle zero-division by returning null. Round result to N decimal places
CREATE OR REPLACE FUNCTION rounded_safe_div(a numeric, b numeric, dp int = 2) RETURNS numeric AS
$$ SELECT round($1::numeric / nullif($2, 0), $3) $$
LANGUAGE SQL
IMMUTABLE
RETURNS NULL ON NULL INPUT;
@robcowie
robcowie / amazon_allowed_chars.py
Created July 22, 2020 16:59
Allowed characters for Amazon entities and keywords
# From https://advertising.amazon.com/API/docs/en-us/get-started/how-to-use-api#Keyword-bid-constraints-by-marketplace
# Broken decoding fixed, mostly. We've lost a few characters but they seem obscure.
# See http://www.i18nqa.com/debug/utf8-debug.html for explanation
ALLOWED_ENTITY_CHARS = {
"'",
' ',
'!',
'"',
'#',
@robcowie
robcowie / local_cloud_function.py
Created May 7, 2020 08:03
Tiny wrapper to serve a cloud function handler locally with Flask
import os, sys
from pathlib import Path
from flask import Flask, request
sys.path.append(Path(".").absolute().as_posix())
from main import handler
app = Flask("name")
@robcowie
robcowie / dag_duration.sql
Created May 4, 2020 13:08
Airflow Task & DAG Duration Queries
-- duration of task instances
select
date(execution_date) as dt,
avg(duration) as avg_duration,
min(duration) as min_duration,
max(duration) as max_duration
from
task_instance
group by
date(execution_date)
@robcowie
robcowie / random_strings.sql
Created April 30, 2020 10:44
Generate random strings in Postgresql
-- Fixed-length ascii strings
CREATE OR REPLACE FUNCTION pg_temp.random_string(int) RETURNS text
AS $$ SELECT
array_to_string(ARRAY(SELECT chr(ascii('B') + round(random() * 25)::integer)
FROM
generate_series(1, $1)), '') $$
LANGUAGE sql;
select pg_temp.random_string(8);
@robcowie
robcowie / gcloud_configurations.md
Created April 20, 2020 22:27
gcloud named configurations

Manage Local GCP Config

To see help for gcloud named configurations

gcloud topic configurations

To list & describe configurations