Skip to content

Instantly share code, notes, and snippets.

View ian-whitestone's full-sized avatar
🐍
Exit code 143

Ian Whitestone ian-whitestone

🐍
Exit code 143
View GitHub Profile
@ian-whitestone
ian-whitestone / example_cte_query.sql
Created March 18, 2023 12:29
Example Snowflake CTE query
View example_cte_query.sql
with sample_data as (
select *
from snowflake_sample_data.tpch_sf1.customer
),
nation_14_customers as (
select *
from sample_data
where c_nationkey = 14
),
@ian-whitestone
ian-whitestone / query_tags.sql
Created February 23, 2023 14:11
Query tag with no node refs
View query_tags.sql
{% macro set_query_tag() -%}
{# Start with any model-configured dict #}
{% set tag_dict = config.get('query_tag', default={}) %}
{# Regardless of resource type, we can always access the config via the 'model' variable #}
{%- do tag_dict.update(
dbt_snowflake_query_tags_version='1.1.3',
app='dbt',
dbt_version=dbt_version,
project_name=project_name,
@ian-whitestone
ian-whitestone / mode_confettis.js
Created January 17, 2023 16:42
Give the people confetti 🎉
View mode_confettis.js
<script src="https://cdn.jsdelivr.net/npm/canvas-confetti@1.4.0/dist/confetti.browser.min.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js"></script>
<div id="bottom_of_page">Thanks for scrolling to the bottom 🙌</div>
<div>Enjoy some confetti 🎉</div>
<script>
@ian-whitestone
ian-whitestone / main.py
Last active May 3, 2021 12:54
Code for the "Testing SQL" blog post: http://ianwhitestone.work/testing-sql
View main.py
import os
from jinja2 import Environment, meta, Template
import pandas as pd
from pandas.testing import assert_frame_equal
import pandas.io.sql as psql
import psycopg2
# All these constants would likely live in separate files
CONNECTION = psycopg2.connect(
View simulations_for_post.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@ian-whitestone
ian-whitestone / a_a_sim.py
Last active February 16, 2021 02:15
Bayesian conversion test A/A simulations
View a_a_sim.py
import numpy as np
def bayesian_rate_comparison(
control_successes,
control_samples,
test_successes,
test_samples,
prior_a=0,
prior_b=0
):
@ian-whitestone
ian-whitestone / simulations.ipynb
Last active April 18, 2021 22:16
Code for the choosing your randomization unit post - https://ianwhitestone.work/choosing-randomization-unit/
View simulations.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@ian-whitestone
ian-whitestone / helpers.py
Created December 21, 2020 01:51
Code for the Dask cluster on demand blog post
View helpers.py
import logging
import time
from dataclasses import dataclass
from typing import Any, Optional, Tuple
import requests
import yaml
from distributed import Client
from distributed.security import Security
from googleapiclient import discovery
@ian-whitestone
ian-whitestone / Dockerfile
Created September 29, 2020 02:03
Code snippets for single node Dask cluster on GCP blog post
View Dockerfile
FROM python:3.7.6-buster
# Set the working directory
RUN mkdir /opt/app
WORKDIR /opt/app
# Copy poetry files into docker image
COPY pyproject.toml .
COPY poetry.lock .
@ian-whitestone
ian-whitestone / parse_unixtime.py
Last active August 5, 2020 22:27
Parse a unix timestamp with proper handling for milliseconds
View parse_unixtime.py
from __future__ import unicode_literals
import pytest
from starscream.pipeline.stage import TransformStage
from pyspark.sql import functions as F, types as T
from starscream.contract import Contract
from starscream.utils.dataframe import as_dicts, from_dicts