Skip to content

Instantly share code, notes, and snippets.

View tdas's full-sized avatar

Tathagata Das tdas

View GitHub Profile
package test
import java.lang.management.ManagementFactory
object MemoryTest {
def main(args: Array[String]) {
val runtime = Runtime.getRuntime()
val runtimeMXBean = ManagementFactory.getRuntimeMXBean()
println("JVM arguments: " + runtimeMXBean.getInputArguments())
@tdas
tdas / gist:a0abcd9b481cbde7caf1
Created February 2, 2015 05:23
Get memory dump periodically
PID=${1:-$(jps | grep SparkSubmit | cut -d ' ' -f 1 )}
INTERVAL=${2:-"1"}
if [ -z $PID ]; then
echo "Usage: get-memory-stats.sh <PID>"
exit -1
fi
echo "Started monitoring $PID"
while [ 1 ]; do
currenttime=$(date)
echo
@tdas
tdas / Spark PR to JIRA Linker [Tamper Monkey]
Created January 16, 2015 20:36
TamperMonkey script to convert JIRA number of Spark Github PR title to actual JIRA webpage
// ==UserScript==
// @name Spark Github PR to JIRA Linker
// @namespace http://your.homepage/
// @version 0.1
// @description Links JIRA number in Spark Github PR title to the actual JIRA page
// @author You
// @match https://github.com/apache/spark/pull/*
// @grant none
// ==/UserScript==
var elements = $('.js-issue-title');
@tdas
tdas / cluster-tests.py
Last active August 29, 2015 13:58
cluster-tests.py
import os
import re
import subprocess
import sys
import time
""" Test basic properties of a cluster configuration.
This script verifies basic functioning of the Spark/Shark/Hadoop setup in a cluster created
with the Spark EC2 scripts.
@tdas
tdas / generate-changelist.py
Last active August 29, 2015 13:57
generate-changelist.py
#!/usr/bin/python
import os
import sys
import subprocess
import time
import traceback
SPARK_HOME = os.environ["SPARK_HOME"]
NEW_RELEASE_VERSION = "0.9.1"