Skip to content

Instantly share code, notes, and snippets.


Rajkumar Singh rajkrrsingh

View GitHub Profile
rajkrrsingh / Google protobuf installation on Mac
Created Nov 27, 2016
Steps to Install google protobuf on Mac
View Google protobuf installation on Mac
$tar xvf protobuf-2.5.0.tar.bz2
$cd protobuf-2.5.0
$./configure CC=clang CXX=clang++ CXXFLAGS='-std=c++11 -stdlib=libc++ -O3 -g' LDFLAGS='-stdlib=libc++' LIBS="-lc++ -lc++abi"
$make -j 4
$sudo make install
$protoc --version
rajkrrsingh / Java DNS
Last active Sep 4, 2022
how the java dns resolution work
View Java DNS

lets take a sample code to demonste what happen when java tries to resolve the ip for the give hostname.


public class DNSClient {
    public static void main(String[] args) {
 InetAddress address = null;
rajkrrsingh /
Last active Jul 23, 2022
Monitoring Kafka Broker JMX using jolokia JVM Agent
Download Jolokia JVM Agent from following location

mv jolokia-jvm-1.3.7-agent.jar agent.jar
here is the small shell script to get metrics you are intersted in
rajkrrsingh /
Last active Jun 4, 2022
reference guide to run kafka command on CDP-DC cluster

CDP-DC kafka useful kafka commands

-- list topics
kafka-topics --list --bootstrap-server `hostname -f`:9092

-- create topic 
kafka-topics --create --bootstrap-server `hostname -f`:9092 --replication-factor 1 --partitions 1 --topic kafkatopic

-- produce messages
kafka-console-producer --broker-list `hostname -f`:9092 --topic kafkatopic
rajkrrsingh / Hive Replication V2 startup
Last active Nov 4, 2021
Jump start guide for Hive Replication V2 - to know more about hive replication please refer
View Hive Replication V2 startup

Prerequisite hive settings:

set hive.server2.logging.operation.level=execution;
set hive.metastore.transactional.event.listeners=org.apache.hive.hcatalog.listener.DbNotificationListener;

Setup database and tables

rajkrrsingh / Hive Thrift Client in Java
Last active Jul 6, 2021
sample java code to connect hive server using thrift client
View Hive Thrift Client in Java
package com.rajkrrsingh.thrift.client;
import org.apache.hive.service.auth.HiveAuthFactory;
import org.apache.hive.service.auth.PlainSaslHelper;
import org.apache.hive.service.cli.thrift.*;
import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.transport.TTransport;
rajkrrsingh /
Created May 28, 2021
Configuring Impyla for CDW Hive LLAP Cluster
import logging
import os
import pandas
from impala.dbapi import connect
from impala.util import as_pandas

conn = connect(host=HIVE_HS2_HOST,port=443,
rajkrrsingh / Interaction with zookeeper using java code
Last active May 5, 2021
Zookeeper : create,list,update zknodes using java
View Interaction with zookeeper using java code
package com.rajkrrsingh.zk;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
rajkrrsingh /
Last active Mar 3, 2021
aws cli commands for day to day work


List users

aws iam list-users

Current user

aws iam get-user

List access key


workflow dir @hdfs

 hadoop fs -ls /tmp/sparkOozieShellAction/
Found 4 items
-rw-r--r--   3 oozie hdfs        178 2017-05-08 07:00 /tmp/sparkOozieShellAction/
drwxr-xr-x   - oozie hdfs          0 2017-05-08 07:01 /tmp/sparkOozieShellAction/lib
-rw-r--r--   3 oozie hdfs        279 2017-05-08 07:12 /tmp/sparkOozieShellAction/
-rw-r--r--   3 oozie hdfs        712 2017-05-08 07:34 /tmp/sparkOozieShellAction/workflow.xml