rajkrrsingh

package org.example;


import java.net.URI;
import java.net.URLEncoder;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;

rajkrrsingh

JPam is a Java-PAM bridge. PAM, or Pluggable Authentication Modules, is a standard security architecture used on Linux, Mac OS X, Solaris, HP-UX and other Unix systems. JPam is the missing link between the two.

JPAM permits the use of PAM authentication facilities by Java applications running on those platforms.

These facilities include:

account
auth
password
session

rajkrrsingh

$wget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.bz2
$tar xvf protobuf-2.5.0.tar.bz2
$cd protobuf-2.5.0
$./configure CC=clang CXX=clang++ CXXFLAGS='-std=c++11 -stdlib=libc++ -O3 -g' LDFLAGS='-stdlib=libc++' LIBS="-lc++ -lc++abi"
$make -j 4 
$sudo make install
$protoc --version

rajkrrsingh

CDP-DC kafka useful kafka commands

-- list topics
kafka-topics --list --bootstrap-server `hostname -f`:9092

-- create topic 
kafka-topics --create --bootstrap-server `hostname -f`:9092 --replication-factor 1 --partitions 1 --topic kafkatopic

-- produce messages
kafka-console-producer --broker-list `hostname -f`:9092 --topic kafkatopic

rajkrrsingh

lets take a sample code to demonste what happen when java tries to resolve the ip for the give hostname.

import java.net.InetAddress;
import java.net.UnknownHostException;

public class DNSClient {
    public static void main(String[] args) {
 InetAddress address = null;

rajkrrsingh

Download Jolokia JVM Agent from following location

https://jolokia.org/download.html
wget http://search.maven.org/remotecontent?filepath=org/jolokia/jolokia-jvm/1.3.7/jolokia-jvm-1.3.7-agent.jar

mv jolokia-jvm-1.3.7-agent.jar agent.jar

here is the small shell script to get metrics you are intersted in

rajkrrsingh

Prerequisite hive settings:

set hive.server2.logging.operation.level=execution;
set hive.metastore.transactional.event.listeners=org.apache.hive.hcatalog.listener.DbNotificationListener;
set hive.metastore.dml.events=true;

Setup database and tables

rajkrrsingh

package com.rajkrrsingh.thrift.client;

import org.apache.hive.service.auth.HiveAuthFactory;
import org.apache.hive.service.auth.PlainSaslHelper;
import org.apache.hive.service.cli.thrift.*;
import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.transport.TTransport;

import javax.security.sasl.SaslException;

rajkrrsingh

import logging
import os
import pandas
from impala.dbapi import connect
from impala.util import as_pandas
logging.basicConfig(level=logging.DEBUG)

HIVE_HS2_HOST='HIVE_HOST_ENDPOINT'
conn = connect(host=HIVE_HS2_HOST,port=443,

rajkrrsingh

package com.rajkrrsingh.zk;

import java.io.IOException;
import java.util.List;
import java.util.concurrent.CountDownLatch;

import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;