Skip to content

Instantly share code, notes, and snippets.

Avatar

Rajkumar Singh rajkrrsingh

21 followers · 27 following
View GitHub Profile
@rajkrrsingh
rajkrrsingh / Create MapR-Db Table
Created August 9, 2015 03:42
code snippet to create maprdb table
View Create MapR-Db Table
private void createTable(String tableName, List<String> cfList)
throws IOException {
final String table = tableName;
final List<String> cfs = cfList;
try {
ugi.doAs(new PrivilegedExceptionAction<Void>() {
@Override
public Void run() throws Exception {
if (!admin.tableExists(table)) {
@rajkrrsingh
rajkrrsingh / Reading parquet files using the parquet tools
Created August 10, 2015 22:04
reading parquet files and know meta information of parquet file
View Reading parquet files using the parquet tools
// Building a parquet tools
git clone https://github.com/Parquet/parquet-mr.git
cd parquet-mr/parquet-tools/
mvn clean package -Plocal
// know the schema of the parquet file
java -jar parquet-tools-1.6.0rc3-SNAPSHOT.jar schema sample.parquet
// Read parquet file
java -jar parquet-tools-1.6.0rc3-SNAPSHOT.jar cat sample.parquet
@rajkrrsingh
rajkrrsingh / Interaction with zookeeper using java code
Last active May 5, 2021 09:23
Zookeeper : create,list,update zknodes using java
View Interaction with zookeeper using java code
package com.rajkrrsingh.zk;
import java.io.IOException;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
@rajkrrsingh
rajkrrsingh / Apache Drill REST API reference
Created October 15, 2015 15:01
View Apache Drill REST API reference
--get storage plugins
curl http://node3.mapr.com:8047/storage.json
--create storage plugin
curl --request POST --header "ContentType: application/json" --data '{"name": "dfs1", "config": { "type": "file", "enabled": false, "connection": "maprfs:///", "workspaces": null, "formats": null }}' http://node3.mapr.com:8047/storage/dfs1.json
{
"result" : "success"
}
--get specific storage plugin
curl http://10.10.71.14:8047/storage/dfs1.json
{
@rajkrrsingh
rajkrrsingh / gist:a86ff86f73e351bdf86c
Created November 23, 2015 16:56
Redis Server installation on CentOs
View gist:a86ff86f73e351bdf86c
#wget http://download.redis.io/releases/redis-3.0.2.tar.gz
# tar xzf redis-3.0.2.tar.gz
# cd redis-3.0.2
# make
Run Redis Server
src/redis-server
@rajkrrsingh
rajkrrsingh / Hive Thrift Client in Java
Last active July 6, 2021 11:49
sample java code to connect hive server using thrift client
View Hive Thrift Client in Java
package com.rajkrrsingh.thrift.client;
import org.apache.hive.service.auth.HiveAuthFactory;
import org.apache.hive.service.auth.PlainSaslHelper;
import org.apache.hive.service.cli.thrift.*;
import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.transport.TTransport;
import javax.security.sasl.SaslException;
@rajkrrsingh
rajkrrsingh / HiveServer2 connection with MySQL DB as metastore over SSL.md
Last active July 18, 2016 06:07
quick guide to connect HS2 to MySQL DB metastore over SSL
View HiveServer2 connection with MySQL DB as metastore over SSL.md

Setting up MySQL SSL

# Create clean environment
shell> rm -rf newcerts
shell> mkdir newcerts && cd newcerts

# Create CA certificate
shell> openssl genrsa 2048 > ca-key.pem
shell> openssl req -new -x509 -nodes -days 3600 \
         -key ca-key.pem -out ca.pem
@rajkrrsingh
rajkrrsingh / LLAP_DEMO.md
Last active February 20, 2018 13:18
LLAP Application demo on HDP2.5 using tpc-ds dataset
View LLAP_DEMO.md 
su - hdfs 
wget https://github.com/hortonworks/hive-testbench/archive/hive14.zip
unzip hive14.zip
cd hive-testbench-hive14/
vi hive-testbench-hive14/settings/load-partitioned.sql -- remove G1GC and use Parrallel GC scheme
cd hive-testbench-hive14/
yum install gcc
echo 'export JAVA_HOME=/usr/jdk64/jdk1.8.0_77' >> ~/.bashrc
echo 'PATH=$PATH:$JAVA_HOME/bin' &gt;&gt; ~/.bashrc
@rajkrrsingh
rajkrrsingh / Hive-Overview.md
Last active October 14, 2018 20:05
View Hive-Overview.md
  • install wget
  • su - hive
cd /tmp
wget https://raw.githubusercontent.com/HortonworksUniversity/Security_Labs/master/labdata/sample_07.csv
wget https://raw.githubusercontent.com/HortonworksUniversity/Security_Labs/master/labdata/sample_08.csv

open hive cli

-- create table
@rajkrrsingh
rajkrrsingh / RunSqoopCommand.java
Created November 22, 2016 09:46
java program to run Sqoop Command using google SSHXCUTE framework
View RunSqoopCommand.java
import net.neoremind.sshxcute.core.SSHExec;
import net.neoremind.sshxcute.core.ConnBean;
import net.neoremind.sshxcute.task.CustomTask;
import net.neoremind.sshxcute.task.impl.ExecCommand;
public class RunSqoopCommand {
public static void main(String args[]) throws Exception{