Skip to content

Instantly share code, notes, and snippets.

Avatar

Rajkumar Singh rajkrrsingh

View GitHub Profile
@rajkrrsingh
rajkrrsingh / Create MapR-Db Table
Created Aug 9, 2015
code snippet to create maprdb table
View Create MapR-Db Table
private void createTable(String tableName, List<String> cfList)
throws IOException {
final String table = tableName;
final List<String> cfs = cfList;
try {
ugi.doAs(new PrivilegedExceptionAction<Void>() {
@Override
public Void run() throws Exception {
if (!admin.tableExists(table)) {
@rajkrrsingh
rajkrrsingh / Reading parquet files using the parquet tools
Created Aug 10, 2015
reading parquet files and know meta information of parquet file
View Reading parquet files using the parquet tools
// Building a parquet tools
git clone https://github.com/Parquet/parquet-mr.git
cd parquet-mr/parquet-tools/
mvn clean package -Plocal
// know the schema of the parquet file
java -jar parquet-tools-1.6.0rc3-SNAPSHOT.jar schema sample.parquet
// Read parquet file
java -jar parquet-tools-1.6.0rc3-SNAPSHOT.jar cat sample.parquet
@rajkrrsingh
rajkrrsingh / Interaction with zookeeper using java code
Last active May 5, 2021
Zookeeper : create,list,update zknodes using java
View Interaction with zookeeper using java code
package com.rajkrrsingh.zk;
import java.io.IOException;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
View Apache Drill REST API reference
--get storage plugins
curl http://node3.mapr.com:8047/storage.json
--create storage plugin
curl --request POST --header "ContentType: application/json" --data '{"name": "dfs1", "config": { "type": "file", "enabled": false, "connection": "maprfs:///", "workspaces": null, "formats": null }}' http://node3.mapr.com:8047/storage/dfs1.json
{
"result" : "success"
}
--get specific storage plugin
curl http://10.10.71.14:8047/storage/dfs1.json
{
@rajkrrsingh
rajkrrsingh / gist:a86ff86f73e351bdf86c
Created Nov 23, 2015
Redis Server installation on CentOs
View gist:a86ff86f73e351bdf86c
#wget http://download.redis.io/releases/redis-3.0.2.tar.gz
# tar xzf redis-3.0.2.tar.gz
# cd redis-3.0.2
# make
Run Redis Server
src/redis-server
@rajkrrsingh
rajkrrsingh / Hive Thrift Client in Java
Last active Jul 6, 2021
sample java code to connect hive server using thrift client
View Hive Thrift Client in Java
package com.rajkrrsingh.thrift.client;
import org.apache.hive.service.auth.HiveAuthFactory;
import org.apache.hive.service.auth.PlainSaslHelper;
import org.apache.hive.service.cli.thrift.*;
import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.transport.TTransport;
import javax.security.sasl.SaslException;
@rajkrrsingh
rajkrrsingh / HiveServer2 connection with MySQL DB as metastore over SSL.md
Last active Jul 18, 2016
quick guide to connect HS2 to MySQL DB metastore over SSL
View HiveServer2 connection with MySQL DB as metastore over SSL.md

Setting up MySQL SSL

# Create clean environment
shell> rm -rf newcerts
shell> mkdir newcerts && cd newcerts

# Create CA certificate
shell> openssl genrsa 2048 > ca-key.pem
shell> openssl req -new -x509 -nodes -days 3600 \
         -key ca-key.pem -out ca.pem
@rajkrrsingh
rajkrrsingh / LLAP_DEMO.md
Last active Feb 20, 2018
LLAP Application demo on HDP2.5 using tpc-ds dataset
View LLAP_DEMO.md
su - hdfs 
wget https://github.com/hortonworks/hive-testbench/archive/hive14.zip
unzip hive14.zip
cd hive-testbench-hive14/
vi hive-testbench-hive14/settings/load-partitioned.sql -- remove G1GC and use Parrallel GC scheme
cd hive-testbench-hive14/
yum install gcc
echo 'export JAVA_HOME=/usr/jdk64/jdk1.8.0_77' >> ~/.bashrc
echo 'PATH=$PATH:$JAVA_HOME/bin' >> ~/.bashrc
View Hive-Overview.md
  • install wget
  • su - hive
cd /tmp
wget https://raw.githubusercontent.com/HortonworksUniversity/Security_Labs/master/labdata/sample_07.csv
wget https://raw.githubusercontent.com/HortonworksUniversity/Security_Labs/master/labdata/sample_08.csv

open hive cli

-- create table
@rajkrrsingh
rajkrrsingh / RunSqoopCommand.java
Created Nov 22, 2016
java program to run Sqoop Command using google SSHXCUTE framework
View RunSqoopCommand.java
import net.neoremind.sshxcute.core.SSHExec;
import net.neoremind.sshxcute.core.ConnBean;
import net.neoremind.sshxcute.task.CustomTask;
import net.neoremind.sshxcute.task.impl.ExecCommand;
public class RunSqoopCommand {
public static void main(String args[]) throws Exception{