Skip to content

Instantly share code, notes, and snippets.

View rajkrrsingh's full-sized avatar

Rajkumar Singh rajkrrsingh

View GitHub Profile
rajkrrsingh / Create MapR-Db Table
Created August 9, 2015 03:42
code snippet to create maprdb table
private void createTable(String tableName, List<String> cfList)
throws IOException {
final String table = tableName;
final List<String> cfs = cfList;
try {
ugi.doAs(new PrivilegedExceptionAction<Void>() {
public Void run() throws Exception {
if (!admin.tableExists(table)) {
rajkrrsingh / Reading parquet files using the parquet tools
Created August 10, 2015 22:04
reading parquet files and know meta information of parquet file
// Building a parquet tools
git clone
cd parquet-mr/parquet-tools/
mvn clean package -Plocal
// know the schema of the parquet file
java -jar parquet-tools-1.6.0rc3-SNAPSHOT.jar schema sample.parquet
// Read parquet file
java -jar parquet-tools-1.6.0rc3-SNAPSHOT.jar cat sample.parquet
rajkrrsingh / Interaction with zookeeper using java code
Last active May 5, 2021 09:23
Zookeeper : create,list,update zknodes using java
package com.rajkrrsingh.zk;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
--get storage plugins
--create storage plugin
curl --request POST --header "ContentType: application/json" --data '{"name": "dfs1", "config": { "type": "file", "enabled": false, "connection": "maprfs:///", "workspaces": null, "formats": null }}'
"result" : "success"
--get specific storage plugin
rajkrrsingh / gist:a86ff86f73e351bdf86c
Created November 23, 2015 16:56
Redis Server installation on CentOs
# tar xzf redis-3.0.2.tar.gz
# cd redis-3.0.2
# make
Run Redis Server
rajkrrsingh / Hive Thrift Client in Java
Last active July 6, 2021 11:49
sample java code to connect hive server using thrift client
package com.rajkrrsingh.thrift.client;
import org.apache.hive.service.auth.HiveAuthFactory;
import org.apache.hive.service.auth.PlainSaslHelper;
import org.apache.hive.service.cli.thrift.*;
import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.transport.TTransport;
rajkrrsingh / HiveServer2 connection with MySQL DB as metastore over
Last active July 18, 2016 06:07
quick guide to connect HS2 to MySQL DB metastore over SSL

Setting up MySQL SSL

# Create clean environment
shell> rm -rf newcerts
shell> mkdir newcerts && cd newcerts

# Create CA certificate
shell> openssl genrsa 2048 > ca-key.pem
shell> openssl req -new -x509 -nodes -days 3600 \
         -key ca-key.pem -out ca.pem
rajkrrsingh /
Last active February 20, 2018 13:18
LLAP Application demo on HDP2.5 using tpc-ds dataset
su - hdfs 
cd hive-testbench-hive14/
vi hive-testbench-hive14/settings/load-partitioned.sql -- remove G1GC and use Parrallel GC scheme
cd hive-testbench-hive14/
yum install gcc
echo 'export JAVA_HOME=/usr/jdk64/jdk1.8.0_77' >> ~/.bashrc
echo 'PATH=$PATH:$JAVA_HOME/bin' &gt;&gt; ~/.bashrc
  • install wget
  • su - hive
cd /tmp

open hive cli

-- create table
rajkrrsingh /
Created November 22, 2016 09:46
java program to run Sqoop Command using google SSHXCUTE framework
import net.neoremind.sshxcute.core.SSHExec;
import net.neoremind.sshxcute.core.ConnBean;
import net.neoremind.sshxcute.task.CustomTask;
import net.neoremind.sshxcute.task.impl.ExecCommand;
public class RunSqoopCommand {
public static void main(String args[]) throws Exception{