Skip to content

Instantly share code, notes, and snippets.

View lizturp's full-sized avatar

Elizabeth lizturp

View GitHub Profile
@lizturp
lizturp / qubole-bastion-host-cf.template
Created May 4, 2018 06:42
A Cloudformation template for spinning up a bastion host to use with Qubole clusters in an AWS private subnet
{
"AWSTemplateFormatVersion" : "2012-10-17",
"Description" : "Launch EC2 Instance as a bastion host for Qubole. http://docs.qubole.com/en/latest/user-guide/clusters/clusters-in-vpcs.html.",
"Parameters" : {
"AccountName" : {
"Description" : "For tagging purposes - example : BI Sales Department",
"Type" : "String"
#!/usr/bin/env python
import boto3
import argparse
# Available args:
# Environment
# --env dev, stage or prod
#
# run the script from a command line like so:
# python boto-s3-things.py --env=stage
@lizturp
lizturp / cloudformation-kinesis-fh-delivery-stream.json
Last active November 3, 2022 03:34
AWS Cloudformation template to build a firehose delivery stream to S3, with a kinesis stream as the source. JSON, but it's fine.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "The AWS CloudFormation template for Kinesis Stream",
"Parameters": {
"Environment": {
"Description": "dev, stage, or prod - this is for bucket tags",
"Type": "String",
"MinLength": "3",
"MaxLength": "5"
}
@lizturp
lizturp / setup-example-glue
Created September 28, 2017 15:17
Pulling an mlb example data set and setting up in AWS Glue to query with Athena
# finish dis later
sudo pip install kaggle-cli
kg download -u 'myUsername' -p 'myPassword' -d DigitRecognizer -f test.csv
# https://www.kaggle.com/seanlahman/the-history-of-baseball/downloads/the-history-of-baseball.zip
# https://www.kaggle.com/seanlahman/the-history-of-baseball
@lizturp
lizturp / pull-omniture-report.R
Created July 25, 2017 23:11
Basic script to test pulling a report from omniture.
#!/usr/bin/Rscript
library("RSiteCatalyst")
project_name <- "testing-omniture"
rsid <- "REPORT-SUITE-ID"
rsid_friendly <- "FRIENDLY-NAME"
today_dt <- Sys.Date()
date.1.start <- Sys.Date()-3
@lizturp
lizturp / upload-file-to-s3-with-encryption.R
Created July 24, 2017 20:55
Basic script to test upload to s3 functionality is working in R
#!/usr/bin/Rscript
library(aws.s3)
library(stringi)
source("./credentials/aws.txt")
#bucketlist()
s3conn <- rawConnection(raw(0), "r+")
#this is not writing anything out....
tempfile <- paste0("hello this is a test.")
put_object(tempfile,
object = ("liz-example/redshift/tempfile"),
@lizturp
lizturp / launch-pipe.bash
Last active November 18, 2015 16:43
Launch AWS Datapipeline from Bash Script
#!/bin/bash
usage() {
echo "
Usage:
-e Environment (dev, stage, prod)
"
}
while getopts "e:" flag