Skip to content

Instantly share code, notes, and snippets.

View ryancollingwood's full-sized avatar
🙌
Grateful for the codes

Ryan Collingwood ryancollingwood

🙌
Grateful for the codes
View GitHub Profile
@ryancollingwood
ryancollingwood / sql_parse_column_query_lineage.py
Created November 2, 2023 06:43
For column that is a composite of other columns in SQL query (sub queries or CTEs), get the sql that makes up that column
from typing import Dict
import re
from dataclasses import dataclass, field
import sqlparse
@dataclass
class SQLColumnLineage():
sql: str
target_column: str
aliased_columns: Dict[str, str] = field(default_factory=lambda: dict())
from typing import List
from pydantic import BaseModel
class CharacterClass(BaseModel):
name: str
is_ranged: bool
is_magic: bool
class Item(BaseModel):
name: str
@ryancollingwood
ryancollingwood / flatten_json_doc_to_lines.py
Created July 26, 2022 04:14
flattens a JSON parsed obj into multiple lines. Last Element [-1] in each line is the value. Elements [0:-1] is the path to that value. line_prefix if specified will prefix a value to the begining of each outputted line. add_key_index will add 0-based index position when encoutering lists or tuples. remove_keys if a key (any point in the path) m…
def flatten_json_object_to_lines(obj, remove_keys=None, line_prefix=None, add_key_index=True):
'''
flattens a JSON parsed obj into multiple lines
Last Element [-1] in each line is the value
Elements [0:-1] is the path to that value
line_prefix if specified will prefix a value to the begining of each outputted line
add_key_index will add 0-based index position when encoutering lists or tuples
remove_keys if a key (any point in the path) matches then remove it
adapted from: https://thispointer.com/python-how-to-iterate-over-nested-dictionary-dict-of-dicts/
'''
@ryancollingwood
ryancollingwood / index.html
Created January 31, 2019 12:01
Simple 404 Page
<div id="main">
<div class="fof">
<h1>Error 404</h1>
</div>
</div>
import arcade
"""
A silly demo with the python arcade library
"""
SCREEN_WIDTH = 640
SCREEN_HEIGHT = 480
HALF_SQUARE_WIDTH = 2.5
import pandas as pd
# docs can be found at http://gspread.readthedocs.io/en/latest/
import gspread
from oauth2client.service_account import ServiceAccountCredentials
def get_googlesheet_worksheet(json_keyfile_name, sheet_name, worksheet_name):
scope = ['https://spreadsheets.google.com/feeds']
creds = ServiceAccountCredentials.from_json_keyfile_name(json_keyfile_name, scope)
# Credit for this: Nicholas Swift
# as found at https://medium.com/@nicholas.w.swift/easy-a-star-pathfinding-7e6689c7f7b2
from warnings import warn
import heapq
class Node:
"""
A node class for A* Pathfinding
"""
@ryancollingwood
ryancollingwood / compare_dfs.py
Last active September 11, 2018 02:11 — forked from yassineAlouini/compare_dfs.py
Compare two Pandas DataFrames
import pandas as pd
import numpy as np
def compare_two_dfs(input_df_1, input_df_2):
# explicitly calling fillna with ""
# as if you've used np.nan it has the
# property of nevery being able to be equals
# i.e. `np.nan == np.nan` will always be False
df_1, df_2 = input_df_1.copy().fillna(""), input_df_2.copy().fillna("")
@ryancollingwood
ryancollingwood / swaggerhub_validation.py
Created February 15, 2018 05:21
Validating data against model as defined in Swaggerhub. Mostly a wrapper around https://github.com/Yelp/bravado-core/
import requests
import yaml
from bravado_core.spec import Spec
from bravado_core.validate import validate_schema_object, validate_object
from jsonschema.exceptions import ValidationError
# get a formatted swagger url for yaml definition
def get_swagger_yaml_url(organisation, api, version):
return "https://api.swaggerhub.com/apis/{}/{}/{}/swagger.yaml".format(organisation ,api, version)
@ryancollingwood
ryancollingwood / GoogleDocsLauncher.cs
Created August 29, 2016 03:22
If you use the google drive desktop application on Windows, and want to launch your Google Docs, Sheets, etc. from explorer - then this could help.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.IO;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
namespace GoogleDocsLauncher