mdagost

#!/usr/bin/env python

EMR_COMMAND = os.path.expanduser('~/elastic-mapreduce/elastic-mapreduce')
EMR_LOGGING_DIR = "s3://songkick/emr-logs"

def create_pig_job_flow(pigscript, num_instances=1, extraArguments=[]):
  jobname = "Pig_Daily_" + datetime.datetime.now().strftime('%Y%m%d-%H%M%S')
  print "Creating pig job flow", jobname, pigscript
  args = [EMR_COMMAND,
    "--create",

mdagost

Studying for a Tech Interview Sucks, so Here's a Cheat Sheet to Help

This list is meant to be a both a quick guide and reference for further research into these topics. It's basically a summary of that comp sci course you never took or forgot about, so there's no way it can cover everything in depth. It also will be available as a gist on Github for everyone to edit and add to.

Data Structure Basics

###Array ####Definition:

• Stores data elements based on an sequential, most commonly 0 based, index.
• Based on tuples from set theory.

patr1ckm

#!/usr/local/bin/Rscript

'usage: my_prog.R [-a -r -m <msg>]

options:
-a        Add
-r        Remote
-m <msg>  Message' -> doc

library(docopt)

chrishamant

#!/usr/bin/env python
"""Split large file into multiple pieces for upload to S3.

S3 only supports 5Gb files for uploading directly, so for larger CloudBioLinux
box images we need to use boto's multipart file support.

This parallelizes the task over available cores using multiprocessing.

Usage:
  s3_multipart_upload.py <file_to_transfer> <bucket_name> [<s3_key_name>]

glamp

library(XML)
library(uuid)
library(stringr)
library(plyr)
library(reshape2)
library(ggplot2)

f <- "https://raw.githubusercontent.com/chris-taylor/USElection/master/data/electoral-college-votes.csv"
electoral.college <- read.csv(f, header=FALSE)
names(electoral.college) <- c("state", "electoral_votes")

rspeer

chriddyp

# See official docs at https://dash.plotly.com
# pip install dash pandas

from dash import Dash, dcc, html, Input, Output
import plotly.express as px

import pandas as pd

df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/gapminderDataFiveYear.csv')

barnes7td

Setup Terminal for Sublime Shorcut "subl":

Open terminal and type:

1. Create a directory at ~/bin:

mkdir ~/bin

2. Copy sublime executable to your ~/bin directory:

jasonrudolph

5 entertaining things you can find with the GitHub Search API

Let's have some command-line fun with curl, [jq][1], and the [new GitHub Search API][2].

Today we're looking for:

• The hottest repositories created in the last week.
• The oldest user accounts with zero followers. (So sad.)
• Issue trends by language.
• Clojure projects defying convention.

rgreenjr

-- show running queries (pre 9.2)
SELECT procpid, age(clock_timestamp(), query_start), usename, current_query 
FROM pg_stat_activity 
WHERE current_query != '<IDLE>' AND current_query NOT ILIKE '%pg_stat_activity%' 
ORDER BY query_start desc;

-- show running queries (9.2)
SELECT pid, age(clock_timestamp(), query_start), usename, query 
FROM pg_stat_activity 
WHERE query != '<IDLE>' AND query NOT ILIKE '%pg_stat_activity%'