Skip to content

Instantly share code, notes, and snippets.

View ashishb888's full-sized avatar

Ashish Bhosle ashishb888

7 followers · 16 following
  • Navi Mumbai
View GitHub Profile
@ashishb888
ashishb888 / HelloWorld.scala
Created August 14, 2019 09:54
Scala "Hello world!" example
object HelloWorld {
def main(args: Array[String]) {
println("Hello world!")
}
}
@ashishb888
ashishb888 / spark-stand-alone-cluster
Last active August 24, 2019 06:34
Apache Spark stand alone cluster
./sbin/start-master.sh
---------------------------
Spark Command: /opt/ngs/setups/jdk1.8.0_191/bin/java -cp /opt/ngs/ashishb/packages/spark-2.4.3-bin-hadoop2.7/conf/:/opt/ngs/ashishb/packages/spark-2.4.3-bin-hadoop2.7/jars/* -Xmx1g org.apache.spark.deploy.master.Master --host hdpdev6 --port 7077 --webui-port 8080
========================================
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
19/08/14 19:18:48 INFO Master: Started daemon with process name: 51863@hdpdev6
19/08/14 19:18:48 INFO SignalUtils: Registered signal handler for TERM
19/08/14 19:18:48 INFO SignalUtils: Registered signal handler for HUP
@ashishb888
ashishb888 / kafka-commands.sh
Last active September 20, 2019 07:21
Apache Kafka [2.3.0] commands
#Create a topic
$KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic streams-plaintext-input
#Create a topic with config
$KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic streams-wordcount-output --config cleanup.policy=compact
#Run an example
$KAFKA_HOME/bin/kafka-run-class.sh org.apache.kafka.streams.examples.wordcount.WordCountDemo
#Start console producer
@ashishb888
ashishb888 / advanced-jvm-tuning
Created September 9, 2019 13:04
Advanced JVM tuning | JVM profiling
#Garbage Collection Tuning
-server
-Xms10g
-Xmx10g
-XX:+AlwaysPreTouch
-XX:+UseG1GC
-XX:+ScavengeBeforeFullGC
-XX:+DisableExplicitGC
#Getting Heap Dump on Out of Memory Errors
@ashishb888
ashishb888 / elasticsearch.yml
Last active September 21, 2019 07:17
Elastic stack
cluster.name: elasticsearch-dev
node.name: node-1
network.host: 0.0.0.0
discovery.seed_hosts: ["172.17.5.31", "172.17.5.35", "172.17.5.36"]
discovery.zen.minimum_master_nodes: 2
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
@ashishb888
ashishb888 / stage1.js
Created October 8, 2019 10:25
Thoughtworks REST online test
var xmlhttp = new XMLHttpRequest();
var url = "https://http-hunt.thoughtworks-labs.net/challenge/input";
xmlhttp.onreadystatechange = function () {
if (this.readyState == 4 && this.status == 200) {
var resp = JSON.parse(this.responseText);
console.log("resp: ", resp);
var count = resp.text.length;
console.log("count: " + count);
@ashishb888
ashishb888 / Java HTTPS connection with SSL certificate
Created November 11, 2019 12:15
Java HTTPS connection with SSL certificate
Get SSL certificate from server (Site URL). You can refer below URL
https://www.shellhacks.com/get-ssl-certificate-from-server-site-url-export-download/
Use Java's keytool utility to add downloaded certificate to Java's keystore
keytool -importcert -file <path-of-certificate> -keystore <keystore-file> -alias <certificate-alias>
i.e. keytool -importcert -file E:\certs\some-site.cer -keystore "E:\sw\jdk1.8.0_112\jre\lib\security\cacerts" -alias "some-site"
Now you will be able to access HTTPS URLs
Common issues when downloading dependencies from repositories URLs
@ashishb888
ashishb888 / kafka-connect
Last active December 13, 2019 07:01
Example of File sink & source connector
[hdpusr@hdpdev6 kafka_2.12-2.3.0]$ cat config/connect-standalone.properties
bootstrap.servers=localhost:7092
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=true
value.converter.schemas.enable=true
offset.storage.file.filename=/tmp/connect.offsets
offset.flush.interval.ms=10000
@ashishb888
ashishb888 / ignite-persistence-data-node1.log
Last active December 31, 2019 08:05
Ignite persistence logs
2019-12-31 13:23:28.035 INFO 50839 --- [main] poc.ignite.IgniteApp : Starting IgniteApp v0.0.1-SNAPSHOT on hdpdev6 with PID 50839 (/opt/ngs/ashishb/apps/ignite/ignite-persistence-data-node/ignite-persistence-data-node-0.0.1-SNAPSHOT.jar started by hdpusr in /opt/ngs/ashishb/apps/ignite/ignite-persistence-data-node)
2019-12-31 13:23:28.038 DEBUG 50839 --- [main] poc.ignite.IgniteApp : Running with Spring Boot v2.1.6.RELEASE, Spring v5.1.8.RELEASE
2019-12-31 13:23:28.038 INFO 50839 --- [main] poc.ignite.IgniteApp : No active profile set, falling back to default profiles: default
2019-12-31 13:23:28.740 INFO 50839 --- [main] poc.ignite.config.BeansConfig : ignite bean service
2019-12-31 13:23:28.956 WARN 50839 --- [main] o.apache.ignite.internal.util.typedef.G : Ignite work directory is not provided, automatically resolved to: /opt/ngs/setups/apache-ignite-fabric-2.6.0-bin/work
2019-12-31 13:23:29.108 INFO 50839 --- [main] org.apache.ig
@ashishb888
ashishb888 / npm ssl
Created January 7, 2020 11:54
NPM Config
npm config set https-proxy http://172.20.7.21:8080
npm config set http-proxy http://172.20.7.21:8080
npm config set proxy http://172.20.7.21:8080
npm config set registry "http://registry.npmjs.org"
npm config set strict-ssl false