Skip to content

Instantly share code, notes, and snippets.

Prasanth Jayachandran prasanthj

Block or report user

Report or block prasanthj

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@prasanthj
prasanthj / eclipse-mat-offline.txt
Last active Feb 2, 2020
Eclipse MAT offline analysis
View eclipse-mat-offline.txt
Download Eclipse MAT from https://www.eclipse.org/mat/
Change Xmx depending on heap dump size being analyzed in ini file. If GC errors are observed then add -XX:-UseGCOverheadLimit option too
vim ~/home/pkgs/mat/MemoryAnalyzer.ini
~/home/pkgs/mat/ParseHeapDump.sh heapdump.hprof
@prasanthj
prasanthj / publicSubnets.go
Last active Dec 17, 2019 — forked from samstav/publicSubnets.go
finding public subnets in go
View publicSubnets.go
package main
import (
"fmt"
"sort"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/aws/aws-sdk-go/service/ec2/ec2iface"
@prasanthj
prasanthj / AKSClusterConfigurationReader.json
Created Oct 15, 2019 — forked from brbarnett/AKSClusterConfigurationReader.json
Create Azure custom role for az aks get-credentials
View AKSClusterConfigurationReader.json
{
"Name":"AKS Cluster Configuration Reader",
"Id":"{{ create a unique guid }}",
"IsCustom":true,
"Description":"Can get AKS configuration.",
"Actions":[
"Microsoft.ContainerService/managedClusters/accessProfiles/listCredential/action",
"Microsoft.ContainerService/managedClusters/listClusterUserCredential/action"
],
"NotActions":[
View s3-sync-debian
apt-get update
apt-get install -y python curl vim
curl -O https://bootstrap.pypa.io/get-pip.py
export PATH=~/.local/bin:$PATH
python get-pip.py --user
pip install awscli --upgrade --user
mkdir -p ~/.aws
vim ~/.aws/config
[default]
@prasanthj
prasanthj / workload-management-examples.txt
Last active Mar 21, 2019
Hive Workload Management Examples For Tez Container Mode
View workload-management-examples.txt
SHOW RESOURCE PLANS;
CREATE RESOURCE PLAN llap;
CREATE RESOURCE PLAN global;
set mapred.min.split.size=10000000;
set mapred.max.split.size=10000000;
source /work/queries/tpcds/q55.sql;
ALTER RESOURCE PLAN llap ACTIVATE;
ALTER RESOURCE PLAN global DISABLE;
CREATE TRIGGER global.highly_parallel WHEN TOTAL_TASKS > 40 DO KILL;
@prasanthj
prasanthj / s3-get-speed-private.sh
Last active Mar 1, 2019
S3 GET Speed Private Bucket
View s3-get-speed-private.sh
#!/bin/bash
set -e
: ${AWS_ACCESS_KEY_ID:?"AWS_ACCESS_KEY_ID should be set in script or exported"}
: ${AWS_SECRET_ACCESS_KEY:?"AWS_SECRET_ACCESS_KEY should be set in script or exported"}
if [[ $# -eq 0 ]] ; then
echo 'S3 object URL expected as argument. Usage: ./s3-get-speed-private.sh <s3-private-object-uri>'
exit 0
fi
@prasanthj
prasanthj / orc-file-dump-total-row-count.txt
Created Feb 21, 2019
Total row count from orc file dumps
View orc-file-dump-total-row-count.txt
hive --orcfiledump <orc-table-path> | grep "Rows:" | cut -f2 -d":" | awk '{s+=$1}END{print s}'
@prasanthj
prasanthj / ec2-nvme-count.txt
Created Jan 18, 2019
Mount NVMe SSDs in AWS EC2 instances
View ec2-nvme-count.txt
Mount Separately
----------------
# mount nvme SSDs to /dataN when available for specific instance types
# There are only 4 nvme slots in r5 instances. /dev/nvme0n1 is root EBS volume.
for i in $(seq 1 4); do
if [ -e "/dev/nvme${i}n1" ]
then
echo "Mounting /dev/nvme${i}n1 to /data${i}"
sudo mkfs.ext4 -E nodiscard /dev/nvme${i}n1
sudo mkdir -p /data${i}
@prasanthj
prasanthj / java-async-profiler.txt
Created Dec 13, 2017
Java Async Profiler Usage
View java-async-profiler.txt
# Pre-req: gcc, gcc-c++, git (to clone), openjdk with debug symbols (centos: java-1.8.0-openjdk-debug.x86_64, ubuntu: openjdk-8-dbg)
# Git clone async-profiler https://github.com/jvm-profiling-tools/async-profiler
# Make sure JAVA_HOME is set and you are logged as same user running the java process
cd async-profiler
make all
# Also git clone FlameGraph in the same directory as you clone async-profiler
# sysctl changes
echo 1 > /proc/sys/kernel/perf_event_paranoid
echo 0 > /proc/sys/kernel/kptr_restrict
@prasanthj
prasanthj / mvn-subminute-hive-build.txt
Last active Oct 16, 2017
Apache Hive sub-minute maven dev build
View mvn-subminute-hive-build.txt
Build times for Apache Hive master on my Mac Book Pro
# No source code changes (first build not considered as mvn will download all dependencies and cache it locally)
# Base: Clean offline quite build
$ time mvn clean install -DskipTests -o -q
real 3m9.005s
user 7m14.864s
sys 0m40.295s
You can’t perform that action at this time.