Skip to content

Instantly share code, notes, and snippets.

@ewhauser
ewhauser / csshx.rb
Created May 1, 2011 22:06
Knife csshx
# Use csshX to open terminals on the nodes found in a search
# knife exec csshx.rb 'role:the_role'
abort("usage: knife exec csshx SEARCH_TERM") unless ARGV[2]
nodes = search(:node, "#{ARGV[2].to_s}").collect { |n| n.fqdn }
exec('csshX ' + nodes.join(' ')) unless nodes.length < 1
exit 0
package com.exacttarget.bloomin.jedis;
import com.google.common.base.Preconditions;
import com.twitter.common.net.pool.Connection;
import redis.clients.jedis.Jedis;
public class JedisConnection implements Connection<Jedis, Jedis> {
private final Jedis jedis;
package org.apache.flume.kafka;
import com.cloudera.flume.core.Event;
import com.cloudera.flume.core.EventImpl;
import com.cloudera.util.Clock;
import kafka.api.FetchRequest;
import kafka.javaapi.consumer.SimpleConsumer;
import kafka.message.Message;
import kafka.server.KafkaConfig;
import kafka.server.KafkaServer;
@ewhauser
ewhauser / flume_monitor.rb
Created September 17, 2011 03:39
Pull some metrics from flume
#!/usr/bin/env ruby
require 'rubygems'
require 'getoptlong'
require 'socket'
require 'json'
require 'timeout'
require 'open-uri'
host = "localhost"
@ewhauser
ewhauser / gist:5727229
Created June 7, 2013 05:31
Notes on Facebook's Presto
These notes are from Facebook's Analytics @ Scale conference. I didn't take notes from the presentation or discussions with developers, so feel free to correct any inconsistencies:
- Presto is an ANSI SQL engine built on top of HDFS
- Similar functionality to Cloudera's Impala
- Facebook started developing this project prior to the Impala annoucement, some different design choices
- Implemented in Java
- Queries execute around 10x faster than Hive, aggregation based queries can be 100x times faster
- Byte code generation is used for efficient predicate processing
- Efficient fixed memory data structures are used (very low GC overhead)
- Presto daemons do not have to run on data nodes

Keybase proof

I hereby claim:

  • I am ewhauser on github.
  • I am ewhauser (https://keybase.io/ewhauser) on keybase.
  • I have a public key ASDgt9UmpXcLznvoNPS0LsfUbzCUQPcbaceAg8QC2CAZlgo

To claim this, I am signing this object:

@ewhauser
ewhauser / export_audit_logs.py
Created February 3, 2019 21:03
Export LaunchDarkly audit logs to CSV
#!/usr/bin/env python3
"""
Requirements:
pip install requests python-dateutil
"""
import dateutil.parser as dp
import csv
import sys
from optparse import OptionParser
SELECT repo.name, JSON_EXTRACT(payload, '$.issue.number') as issue_id, COUNT(*) as count, JSON_EXTRACT(payload, '$.issue.title') as title
FROM `githubarchive.month.*`
WHERE type = 'IssueCommentEvent'
and repo.name = 'angular/angular'
and (_TABLE_SUFFIX like '2019%' or _TABLE_SUFFIX like '2018%' or _TABLE_SUFFIX like '2017%')
GROUP BY repo.name, issue_id, title
ORDER BY count desc
@ewhauser
ewhauser / date_dim.sql
Created December 24, 2019 22:54
Generate Date Dimension Table for BigQuery
SELECT
FORMAT_DATE('%F', d) as id,
d AS full_date,
EXTRACT(YEAR FROM d) AS year,
EXTRACT(WEEK FROM d) AS year_week,
EXTRACT(DAY FROM d) AS year_day,
EXTRACT(YEAR FROM d) AS fiscal_year,
FORMAT_DATE('%Q', d) as fiscal_qtr,
EXTRACT(MONTH FROM d) AS month,
FORMAT_DATE('%B', d) as month_name,
name: "Terraform"
on:
push:
branches:
- main
paths:
- terraform/**
pull_request:
branches:
- main