Skip to content

Instantly share code, notes, and snippets.

View prasanthj's full-sized avatar

Prasanth Jayachandran prasanthj

99 followers · 46 following
View GitHub Profile
@prasanthj
prasanthj / publicSubnets.go
Last active December 17, 2019 00:01 — forked from stavxyz/publicSubnets.go
finding public subnets in go
package main
import (
"fmt"
"sort"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/aws/aws-sdk-go/service/ec2/ec2iface"
@prasanthj
prasanthj / AKSClusterConfigurationReader.json
Created October 15, 2019 19:04 — forked from brbarnett/AKSClusterConfigurationReader.json
Create Azure custom role for az aks get-credentials
{
"Name":"AKS Cluster Configuration Reader",
"Id":"{{ create a unique guid }}",
"IsCustom":true,
"Description":"Can get AKS configuration.",
"Actions":[
"Microsoft.ContainerService/managedClusters/accessProfiles/listCredential/action",
"Microsoft.ContainerService/managedClusters/listClusterUserCredential/action"
],
"NotActions":[
@prasanthj
prasanthj / java-async-profiler.txt
Created December 13, 2017 23:41
Java Async Profiler Usage
# Pre-req: gcc, gcc-c++, git (to clone), openjdk with debug symbols (centos: java-1.8.0-openjdk-debug.x86_64, ubuntu: openjdk-8-dbg)
# Git clone async-profiler https://github.com/jvm-profiling-tools/async-profiler
# Make sure JAVA_HOME is set and you are logged as same user running the java process
cd async-profiler
make all
# Also git clone FlameGraph in the same directory as you clone async-profiler
# sysctl changes
echo 1 > /proc/sys/kernel/perf_event_paranoid
echo 0 > /proc/sys/kernel/kptr_restrict
@prasanthj
prasanthj / workload-management-examples.txt
Last active March 21, 2019 20:38
Hive Workload Management Examples For Tez Container Mode
SHOW RESOURCE PLANS;
CREATE RESOURCE PLAN llap;
CREATE RESOURCE PLAN global;
set mapred.min.split.size=10000000;
set mapred.max.split.size=10000000;
source /work/queries/tpcds/q55.sql;
ALTER RESOURCE PLAN llap ACTIVATE;
ALTER RESOURCE PLAN global DISABLE;
CREATE TRIGGER global.highly_parallel WHEN TOTAL_TASKS > 40 DO KILL;
@prasanthj
prasanthj / jquery-test-url-with-retry.js
Created December 15, 2014 19:59
jQuery test liveness of URL
function testURL(url, retryInterval, maxRetryInterval) {
$.ajax({
url: url,
type: "GET",
timeout: 2000,
success: function(resp) {
console.log(url + " is alive!")
},
error: function(x, t, m) {
if (t == 'timeout') {
@prasanthj
prasanthj / s3-get-speed-private.sh
Last active March 1, 2019 17:17
S3 GET Speed Private Bucket
#!/bin/bash
set -e
: ${AWS_ACCESS_KEY_ID:?"AWS_ACCESS_KEY_ID should be set in script or exported"}
: ${AWS_SECRET_ACCESS_KEY:?"AWS_SECRET_ACCESS_KEY should be set in script or exported"}
if [[ $# -eq 0 ]] ; then
echo 'S3 object URL expected as argument. Usage: ./s3-get-speed-private.sh <s3-private-object-uri>'
exit 0
fi
@prasanthj
prasanthj / orc-file-dump-total-row-count.txt
Created February 21, 2019 03:06
Total row count from orc file dumps
hive --orcfiledump <orc-table-path> | grep "Rows:" | cut -f2 -d":" | awk '{s+=$1}END{print s}'
@prasanthj
prasanthj / hive-site.xml
Created December 11, 2014 08:34
hive configs
<configuration>
<!-- Metastore configs -->
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/apps/hive/warehouse</value>
</property>
<property>
<name>hive.metastore.cache.pinobjtypes</name>
<value>Table,Database,Type,FieldSchema,Order</value>
</property>
@prasanthj
prasanthj / mvn-subminute-hive-build.txt
Last active October 16, 2017 23:44
Apache Hive sub-minute maven dev build
Build times for Apache Hive master on my Mac Book Pro
# No source code changes (first build not considered as mvn will download all dependencies and cache it locally)
# Base: Clean offline quite build
$ time mvn clean install -DskipTests -o -q
real 3m9.005s
user 7m14.864s
sys 0m40.295s
@prasanthj
prasanthj / ats-extract-plan.py
Last active August 4, 2017 20:58 — forked from t3rmin4t0r/ats-extract-plan.py
ATS to Hive query plan extraction
import json
import os,sys
class ATSFile(object):
def __init__(self, name):
self.data = json.load(open(name))
self.name = name
def dump(self):
if "hive_query_id" in self.data:
info = self.data["hive_query_id"]["otherinfo"]