Skip to content

Instantly share code, notes, and snippets.

View eddies's full-sized avatar

Edwin Shin eddies

25 followers · 5 following
  • TRA
  • in the space-time continuum
View GitHub Profile
@eddies
eddies / apache-spark.rb
Created January 2, 2019 04:22
Apache Spark 2.2.0 Homebrew
class ApacheSpark < Formula
desc "Engine for large-scale data processing"
homepage "https://spark.apache.org/"
url "https://archive.apache.org/dist/spark/spark-2.2.0/spark-2.2.0-bin-hadoop2.7.tgz"
version "2.2.0"
sha256 "97fd2cc58e08975d9c4e4ffa8d7f8012c0ac2792bcd9945ce2a561cf937aebcc"
head "https://github.com/apache/spark.git"
bottle :unneeded
@eddies
eddies / os-google.xml
Created January 24, 2018 11:50
Google US English OpenSearch Plugin for Firefox
<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/"
xmlns:moz="http://www.mozilla.org/2006/browser/search/">
<ShortName>Google US</ShortName>
<Description>Google US</Description>
<InputEncoding>UTF-8</InputEncoding>
<moz:SearchForm>https://www.google.com/ncr</moz:SearchForm>
<Url type="text/html" method="get" template="https://www.google.com/search">
<Param name="gl" value="en"/>
<Param name="hl" value="en"/>
@eddies
eddies / trelloCardNumbers.js
Last active January 24, 2018 03:00
Trello Card Numbers bookmarklet
javascript:void (function () {
let o = $('.card-short-id');
o.hasClass('hide') ? o.removeClass('hide').css({
'margin-right': '5px',
color: 'black',
'font-weight': 'bold'
}) : o.addClass('hide');
}());
@eddies
eddies / setup-notes.md
Created July 29, 2016 08:00
Spark 2.0.0 and Hadoop 2.7 with s3a setup

Standalone Spark 2.0.0 with s3

###Tested with:

  • Spark 2.0.0 pre-built for Hadoop 2.7
  • Mac OS X 10.11
  • Python 3.5.2

Goal

Use s3 within pyspark with minimal hassle.

@eddies
eddies / gtb
Created April 5, 2016 09:58 — forked from dschuetz/gtb
Hackish script to compute Google Authenticator tokencodes and launch a Tunnelblick VPN with the appropriate computed password+tokencode
#!/usr/bin/python
import hmac, struct, time, base64, hashlib # for totp generation
import re, sys, subprocess # for general stuff
from getopt import getopt, GetoptError # pretending to be easy-to-use
#
# gtb - Google(auth) + Tunnelblick
#
@eddies
eddies / reactive_map.js
Created March 31, 2016 02:26 — forked from granturing/reactive_map.js
Sample reactive Leaflet code for Zeppelin
<!-- place this in an %angular paragraph -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/0.7.5/leaflet.css" />
<div id="map" style="height: 800px; width: 100%"></div>
<script type="text/javascript">
function initMap() {
var map = L.map('map').setView([30.00, -30.00], 3);
L.tileLayer('http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
@eddies
eddies / GIF-Screencast-OSX.md
Created February 29, 2016 07:07 — forked from dergachev/GIF-Screencast-OSX.md
OS X Screencast to animated GIF

OS X Screencast to animated GIF

This gist shows how to create a GIF screencast using only free OS X tools: QuickTime, ffmpeg, and gifsicle.

Screencapture GIF

Instructions

To capture the video (filesize: 19MB), using the free "QuickTime Player" application:

@eddies
eddies / leaflet_prep.scala
Created February 19, 2016 08:54
import org.apache.commons.io.IOUtils
import java.net.URL
import java.nio.charset.Charset
// Zeppelin creates and injects sc (SparkContext) and sqlContext (HiveContext or SqlContext)
// So you don't need create them manually
// load map data
val myMapText = sc.parallelize(
IOUtils.toString(
@eddies
eddies / test_spark.py
Created November 4, 2015 07:32
py.test sanity test of spark
import pytest
def test_spark_sanity_check(sc, tmpdir):
csv = tmpdir.mkdir(__name__).join('one_liner.csv')
csv.write("foo,bar,baz")
data = sc.textFile(str(csv))
line_count = data.count()
assert line_count == 1
@eddies
eddies / snsToSlack.js
Last active October 7, 2015 04:51 — forked from terranware/snsToSlack.js
AWS Lambda function to Slack Channel hookup
var https = require('https');
var util = require('util');
exports.handler = function(event, context) {
console.log(JSON.stringify(event, null, 2));
console.log('From SNS:', event.Records[0].Sns.Message);
var postData = {
"channel": "#aws-sns",
"username": "AWS SNS via Lamda :: DevQa Cloud",