Skip to content

Instantly share code, notes, and snippets.

Avatar

Tan Duong tanduong

View GitHub Profile
View composer_airflow_worker_patch.yaml
spec:
template:
spec:
containers:
- name: airflow-worker
resources:
requests:
memory: 2Gi
limits:
memory: 2Gi
View composer_airflow_worker_hpa.yaml
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
name: airflow-worker
namespace: # use ${AIRFLOW_WORKER_NS} variable defined above
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: airflow-worker
View cloud_composer_autoscale.sh
#!/usr/bin/env bash
set -e
PROJECT=[provide your gcp project id]
COMPOSER_NAME=[provide your composer environment name]
COMPOSER_LOCATION=[provide the selected composer’s location e.g. us-central]
CLUSTER_ZONE=[provide the selected composer’s zone e.g. us-central1-a]
# Enable autoscaling in node level
GKE_CLUSTER=$(gcloud composer environments describe \
View zshrc
# Luke's config for the Zoomer Shell
# Enable colors and change prompt:
autoload -U colors && colors
PS1="%B%{$fg[red]%}[%{$fg[yellow]%}%n%{$fg[green]%}@%{$fg[blue]%}%M %{$fg[magenta]%}%~%{$fg[red]%}]%{$reset_color%}$%b "
# History in cache directory:
HISTSIZE=10000
SAVEHIST=10000
HISTFILE=~/.cache/zsh/history
View creating_a_date_dimension_table_in_postgresql.sql
DROP TABLE if exists d_date;
CREATE TABLE d_date
(
date_dim_id INT NOT NULL,
date_actual DATE NOT NULL,
epoch BIGINT NOT NULL,
day_suffix VARCHAR(4) NOT NULL,
day_name VARCHAR(9) NOT NULL,
day_of_week INT NOT NULL,
@tanduong
tanduong / odbc_setup_macos.md
Created Sep 24, 2019 — forked from arvsrao/odbc_setup_macos.md
Guide to accessing MS SQL Server in Mac OS X via PyODBC
View odbc_setup_macos.md

Since I spent essentially two full days figuring out how to access a corporate MS SQL database pythonicly, I figured I should leave some notes, for future reference and to aid other souls looking to do the same.

These instructions and the commands that follow, were executed on a MAC OS 10.8.3 system. Additionally, I found this blog [post][1] especially helpful during the debugging process.

On mac os, there is a default ODBC manager, iODBC. Other Unix based systems tend to use [unixODBC][2]. Look elsewhere for a discussion about the differences between these driver managers. The only feature we care about is being able to connect to SQL databases through [pyodbc][3], and at the time of this writing [pyodbc][3] requires iODBC as its manager.

Start by installing freeTDS libraries. FreeTDS allows unix programs to talk natively with MS SQL and SyBase databases.

brew intsall freetds
@tanduong
tanduong / odbc_setup_macos.md
Created Sep 24, 2019 — forked from arvsrao/odbc_setup_macos.md
Guide to accessing MS SQL Server in Mac OS X via PyODBC
View odbc_setup_macos.md

Since I spent essentially two full days figuring out how to access a corporate MS SQL database pythonicly, I figured I should leave some notes, for future reference and to aid other souls looking to do the same.

These instructions and the commands that follow, were executed on a MAC OS 10.8.3 system. Additionally, I found this blog [post][1] especially helpful during the debugging process.

On mac os, there is a default ODBC manager, iODBC. Other Unix based systems tend to use [unixODBC][2]. Look elsewhere for a discussion about the differences between these driver managers. The only feature we care about is being able to connect to SQL databases through [pyodbc][3], and at the time of this writing [pyodbc][3] requires iODBC as its manager.

Start by installing freeTDS libraries. FreeTDS allows unix programs to talk natively with MS SQL and SyBase databases.

brew intsall freetds
@tanduong
tanduong / Middleware.js
Created Jul 3, 2019 — forked from darrenscerri/Middleware.js
A very minimal Javascript (ES5 & ES6) Middleware Pattern Implementation
View Middleware.js
var Middleware = function() {};
Middleware.prototype.use = function(fn) {
var self = this;
this.go = (function(stack) {
return function(next) {
stack.call(self, function() {
fn.call(self, next.bind(self));
});
@tanduong
tanduong / jwtRS256.sh
Created May 20, 2019 — forked from ygotthilf/jwtRS256.sh
How to generate JWT RS256 key
View jwtRS256.sh
ssh-keygen -t rsa -b 4096 -m PEM -f jwtRS256.key
# Don't add passphrase
openssl rsa -in jwtRS256.key -pubout -outform PEM -out jwtRS256.key.pub
cat jwtRS256.key
cat jwtRS256.key.pub
You can’t perform that action at this time.