Skip to content

Instantly share code, notes, and snippets.

@mstrother
mstrother / gist:4e507d5d02f650f020c0a223eaa79e20
Created March 18, 2019 19:13
Export All K8s yml from cluster
#!/bin/bash
i=$((0))
for n in $(kubectl get -o=custom-columns=NAMESPACE:.metadata.namespace,KIND:.kind,NAME:.metadata.name pv,pvc,configmap,ingress,service,secret,deployment,statefulset,hpa,job,cronjob --all-namespaces | grep -v 'secrets/default-token')
do
if (( $i < 1 )); then
namespace=$n
i=$(($i+1))
if [[ "$namespace" == "PersistentVolume" ]]; then
kind=$n
@dennyglee
dennyglee / spark-to-sql-validation-sample.py
Created April 4, 2018 18:54
Validate Spark DataFrame data and schema prior to loading into SQL
'''
Example Schema Validation
Assumes the DataFrame `df` is already populated with schema:
{id : int, day_cd : 8-digit code representing date, category : varchar(24), type : varchar(10), ind : varchar(1), purchase_amt : decimal(18,6) }
Runs various checks to ensure data is valid (e.g. no NULL id and day_cd fields) and schema is valid (e.g. [category] cannot be larger than varchar(24))
'''
@cpswan
cpswan / nginx.conf
Last active May 29, 2023 12:20
Using nginx to proxy to an AWS ELB
daemon off;
worker_processes 1;
events { worker_connections 1024; }
http{
sendfile on;
@clintongormley
clintongormley / load_test_data.sh
Last active January 5, 2024 07:32
Run these commands in your shell to setup the test data for Chapter 5
curl -XPUT 'http://localhost:9200/us/user/1?pretty=1' -d '
{
"email" : "john@smith.com",
"name" : "John Smith",
"username" : "@john"
}
'
curl -XPUT 'http://localhost:9200/gb/user/2?pretty=1' -d '
{