Skip to content

Instantly share code, notes, and snippets.

@eidosam
eidosam / snowflake_cost_per_user
Created May 6, 2024 11:22
Compute Snowflake cost per user
SELECT
user_name,
warehouse_name,
warehouse_size,
database_name,
run_date,
ROUND(t/1000/3600*credits_per_hour, 2) credits
FROM (
SELECT
import json
import boto3
def clone_iam_role(original_role_name,
new_role_name):
iam = boto3.client("iam")
@eidosam
eidosam / clone_instance_profile.py
Created April 5, 2024 21:16
Clone AWS IAM instance profile
import json
import boto3
original_instance_profile_name = 'original-role'
new_instance_profile_name = 'new-role'
def clone_instance_profile(original_instance_profile_name,
new_instance_profile_name):
@eidosam
eidosam / BigQuery_re2_udfs
Created January 23, 2023 12:05
BigQuery RegExp functions in Spark
https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_contains
https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_extract
https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#regexp_replace
import copy
def merge_dict_deep(dest, src):
merged_dict = copy.deepcopy(dest)
for key in src:
if key not in dest:
merged_dict[key] = src[key]
elif isinstance(dest[key], dict) and isinstance(src[key], dict):
@eidosam
eidosam / create-transfer-job.sh
Created March 7, 2019 14:03
Start a Google Cloud Storage Transfer job from command-line
#!/usr/bin/env bash
### ------ Preparation ------ ###
# Acquire new user credentials to use for Application Default Credentials
# Run: gcloud auth application-default login
### ------ -------------- --- ###
function printUsage() {
echo -e "
\rUsage: bash ./create-transfer-job.sh [options] <s3-source-data> <gcs-bucket-name>
from boto3.session import Session
import base64
from botocore.exceptions import ClientError
def get_secret(secret_name, region_name='us-west-2'):
secret_value_response = {}
try:
secret_value_response = (
function newlineDelimitedKeyValue(obj, parentKey = '') {
return Object.entries(obj)
.map(([k, v]) => {
if (typeof v === 'object') {
return newlineDelimitedKeyValue(v, `${parentKey}${k}_`);
}
const kUpper = `${parentKey}${k}`.toUpperCase();
return `${kUpper}=${v}`;
@eidosam
eidosam / crc16-xmodem.awk
Last active February 20, 2022 23:14
CRC16 XMODEM ZMODEM implementation in AWK, it is the model used by Redis Cluster for key distribution
BEGIN {
cmpl = 0xffff;
initial = 0x0000;
TABLE[ 0] = 0x0000;
TABLE[ 1] = 0x1021;
TABLE[ 2] = 0x2042;
TABLE[ 3] = 0x3063;
TABLE[ 4] = 0x4084;
@eidosam
eidosam / scala.Makefile
Created November 2, 2020 13:05
Default lifecycle phases of Maven build as Makefile
# https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
# Validate the project is correct and all necessary information is available.
validate:
mvn validate
# Compile the source code of the project.
compile:
mvn compile
# Test the compiled source code using a suitable unit testing framework.