This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env ruby | |
# Copy a heroku app (buildpacks, add-ons, labs, config, users). | |
# This script is idempotent so it can run against an existing app. | |
# | |
# Usage: | |
# $> clone-heroku-app source-app target-app | |
require 'json' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
let i = 0 | |
const start = Date.now() | |
setInterval(() => console.log(++i, Date.now() - start), 1000) | |
console.time('bigcalc') | |
for (let i = 0; i < 10000000000; ++i) { | |
Math.cos(37863443 * Math.exp(23749374)) | |
} | |
console.timeEnd('bigcalc') |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
##### READ SPARK DATAFRAME | |
df = spark.read.option("header", "true").option("inferSchema", "true").csv(fname) | |
# store the schema from the CSV w/ the header in the first file, and infer the types for the columns | |
df_schema = df.schema | |
##### SAVE JSON SCHEMA INTO S3 / BLOB STORAGE | |
# save the schema to load from the streaming job, which we will load during the next job | |
dbutils.fs.rm("/home/mwc/airline_schema.json", True) | |
with open("/dbfs/home/mwc/airline_schema.json", "w") as f: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
##### Taken from https://github.com/bosswissam/pysize | |
import sys | |
def get_size(obj, seen=None): | |
"""Recursively finds size of objects""" | |
size = sys.getsizeof(obj) | |
if seen is None: | |
seen = set() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Python logger in AWS Lambda has a preset format. To change the format of the logging statement, | |
# remove the logging handler & add a new handler with the required format | |
import logging | |
import sys | |
def setup_logging(): | |
logger = logging.getLogger() | |
for h in logger.handlers: | |
logger.removeHandler(h) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
'use strict'; | |
var config = { | |
sass : { | |
errLogToConsole: true, | |
outputStyle: 'compressed' | |
}, | |
bower: './bower_components', | |
source: { | |
main: '.src/', | |
html:{ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
###### | |
# Author: Marcello de Sales (marcello.desales@gmail.com) | |
# Description: Create Create Environment Variables in EC2 Hosts from EC2 Host Tags | |
# | |
### Requirements: | |
# * Install jq library (sudo apt-get install -y jq) | |
# * Install the EC2 Instance Metadata Query Tool (http://aws.amazon.com/code/1825) | |
# | |
### Installation: | |
# * Add the Policy EC2:DescribeTags to a User |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/python | |
# Send messages to an Azure Event Hub using the Apache Qpid Proton AMQP library. | |
import sys | |
import commands | |
from proton import * | |
# Event Hub address & credentials | |
# amqps://<keyname>:<key>@<namespace>.servicebus.windows.net/<eventhubname> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"title": "Nginx Web", | |
"services": { | |
"query": { | |
"list": { | |
"0": { | |
"query": "response:[200 TO 299]", | |
"alias": "OK", | |
"color": "#7EB26D", | |
"id": 0, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
'use strict'; | |
var gulp = require('gulp'); | |
var gutil = require('gulp-util'); | |
var del = require('del'); | |
var uglify = require('gulp-uglify'); | |
var gulpif = require('gulp-if'); | |
var exec = require('child_process').exec; | |
var notify = require('gulp-notify'); |
NewerOlder