Skip to content

Instantly share code, notes, and snippets.

View mmeasic's full-sized avatar

Saint Shuttles-worth mmeasic

  • Meta
  • Barcelona
View GitHub Profile
@Wittline
Wittline / Uber_tracking_expenses.py
Last active September 14, 2021 10:21
Uber_tracking_expenses DAG
import logging
import datetime
from airflow import DAG
from airflow.models import Variable
from airflow.models.connection import Connection
from airflow.contrib.hooks.aws_hook import AwsHook
from airflow.providers.amazon.aws.hooks.s3 import S3Hook
from airflow.hooks.postgres_hook import PostgresHook
from airflow.operators.dummy_operator import DummyOperator
from airflow.operators.postgres_operator import PostgresOperator
from pyspark.sql.types import *
from pyspark.sql.functions import *
#Flatten array of structs and structs
def flatten(df):
# compute Complex Fields (Lists and Structs) in Schema
complex_fields = dict([(field.name, field.dataType)
for field in df.schema.fields
if type(field.dataType) == ArrayType or type(field.dataType) == StructType])
while len(complex_fields)!=0:
@adamhaney
adamhaney / dag.py
Created June 14, 2017 18:10
DBT Airflow DAG with model/graph introspection
from datetime import datetime, timedelta
import networkx as nx
from airflow import DAG
from airflow.operators import BashOperator, SubDagOperator
start_date = datetime(year=2017, month=6, day=13, hour=19, minute=0)
schedule_interval = '0 * * * 1-5'
default_args = {
@bobbydennett
bobbydennett / redshift_performance_tuning.sql
Created July 9, 2016 00:22
Redshift performance tuning-related queries
--------------------
-- Incorrect column encoding
--------------------
SELECT database, schema || '.' || "table" AS "table", encoded, size
FROM svv_table_info
WHERE encoded='N'
ORDER BY 2;
SELECT trim(n.nspname || '.' || c.relname) AS "table",trim(a.attname) AS "column",format_type(a.atttypid, a.atttypmod) AS "type",
@bgusach
bgusach / multireplace.py
Last active February 16, 2024 02:52
Python string multireplacement
def multireplace(string, replacements, ignore_case=False):
"""
Given a string and a replacement map, it returns the replaced string.
:param str string: string to execute replacements on
:param dict replacements: replacement dictionary {value to find: value to replace}
:param bool ignore_case: whether the match should be case insensitive
:rtype: str
"""
@switchtrue
switchtrue / oracle_1Z0-061_notes.md
Last active September 29, 2023 13:07
Notes for taking the 1Z0-061 Oracle Exam - Oracle Database 12c: SQL Fundamentals

Introduction

I have compiled these notes whilst revising for the Oracle 1Z0-061 Exam - Oracle Database 12c: SQL Fundamentals. They should also be relevant to the 1Z0-051 - Oracle Database 11g: SQL Fundamentals exam. Revision was most conducted using the excellent and highly recommended "OCA Oracle Database 12c SQL Fundamentals I Exam Guide" by Roopesh Ramklass.

I have aimed to include include in these notes common "gotchas" and easy to forget functionality rather than documenting everything required for the exam. This can then be used as a quick refresher before walking into the exam.

The content is broken up into sections with each heading mapping to the relevant [Oracle 1Z0-061 exam topics](https://edu

@mmasashi
mmasashi / num_commits_per_day.sql
Created December 2, 2014 23:32
Number of commits per day (Amazon Redshift)
select trunc(startwork), count(distinct xid) from stl_commit_stats where xid > 0 group by trunc(startwork) order by 1;