Skip to content

Instantly share code, notes, and snippets.

View dstreev's full-sized avatar

David W. Streever dstreev

  • Cloudera
  • Atlanta, GA
View GitHub Profile
@dstreev
dstreev / phx_overloading.sql
Created July 8, 2015 20:07
Example of Phoenix Overloading
DROP VIEW
IF EXISTS phoenix_overload.profile;
DROP VIEW
IF EXISTS phoenix_overload.relationship;
DROP VIEW
IF EXISTS phoenix_overload.primary_residence;
DROP VIEW
IF EXISTS phoenix_overload.demographic;
DROP VIEW
IF EXISTS phoenix_overload.entity;
@dstreev
dstreev / hdp_log_archive.sh
Created May 12, 2016 13:07
HDP Log Archive Script
#!/bin/sh
# Command Line params
# ACTUAL at this time:
# $1 = Interval Days to remove.
DAY_ARCHIVE_THRESHOLD=$1
#!/bin/bash
#!/usr/bin/env bash
#########################
#
# WARNING: I make NO guarantees, test it first. This can be VERY destructive!! You've been warned... again..
#
# Usage: ./remove-hdp.sh <hdp-version>
#
# This will yum erase the hdp version specified on the host. To use this on the cluster,
# try copying to an Ambari Servers resources directory. /var/lib/ambari-server/resources
@dstreev
dstreev / cluster_compare.py
Created November 17, 2016 11:46
HDP Cluster Compare Tool
#!/usr/bin/python
import sys
import json
import cStringIO
import difflib
import random
import time
import getpass
import collections
#!/usr/bin/env bash
################################################################################################
#
# Use this to establish a baseline for a cluster by running a variety of Teragen/sort processes.
#
# With the input parameters, the script will make some basic calculations to determine a good
# load profile that can will utilize the cluster to its fullest potential. In order for these
# tests to perform consistently, we're making the assumption that the cluster will be FULLY
# available for the tests.
#!/usr/bin/env bash
# Expecting 2 parameters.
# - Snapshoted Directory
# - Snapshots older then x days.
while [ $# -gt 0 ]; do
case "$1" in
--snapshot-dir)