sathish-io

Edit /etc/sudoers file add below, where <username> is the user name to be enabled

 <username> ALL=(ALL:ALL) NOPASSWD: ALL

sathish-io

Run Spark Application from SBT console:
----------------------------------------
run-main -Dspark.master=local[*]  com.greenline.data.spark.app.CollegeDataIndexerSparkApp

Add Eclipse plugin support:
-----------------------------
in project/plugins.sbt add,
addSbtPlugin("com.typesafe.sbteclipse" % "sbteclipse-plugin" % "2.5.0")

Then run 'sbt eclipse' or 'activator eclipse'

sathish-io

// Use Gists to store code you would like to remember later on
console.log(window); // log the "window" object to the console

sathish-io

Home: /Users/skandasa/installs/kafka_2.10-0.8.2.0

1) start a ZooKeeper server 
bin/zookeeper-server-start.sh config/zookeeper.properties
		binding to port 0.0.0.0/0.0.0.0:2181

2) Start brokers
a) bin/kafka-server-start.sh config/server.properties 
	- starts with port 9092
b) bin/kafka-server-start.sh config/server-1.properties 

sathish-io

java -jar app.jar --server.port=8180 --debug --spring.profiles.active=production --logging.file=myapp.log --logging.path=./

sathish-io

<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true">
    <appender name="consoleAppender" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <charset>UTF-8</charset>
            <Pattern>%d %-4relative [%thread] %-5level %logger{35} - %msg%n</Pattern>
        </encoder>
    </appender>
 
    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">

sathish-io

import java.io.IOException;
import java.io.UnsupportedEncodingException;

import org.apache.commons.httpclient.Credentials;
import org.apache.commons.httpclient.DefaultHttpMethodRetryHandler;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.UsernamePasswordCredentials;
import org.apache.commons.httpclient.auth.AuthScope;

sathish-io

import javax.jms.Connection;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageProducer;
import javax.jms.Queue;
import javax.jms.Session;
import javax.jms.TextMessage;

sathish-io

AWK script to get min, max, sum, average, standard deviation for numbers:

cat data.txt | awk -F ":" 'BEGIN {max = 0; min = 999999; sum = 0; sumsq = 0} { x= ($NF + 0); sum += x; sumsq += x*x; if(x > max)max = x; if(x < min) min = x}  END { print "Min: ", min,  " Max: ",max, " Sum: ", sum, " Average: ", sum/NR , " Std Dev: ", sqrt(sumsq/NR - (sum/NR)^2) }'  

Assume you have data.txt that has following data set,
2
3
6
8
11

sathish-io

Basic postgresql commands:


\l  - to list all database.

\db - to list all tables in a db.

\d <tablename> - to describe a table

Create new database