Skip to content

Instantly share code, notes, and snippets.

Avatar

Neeraj nsabharwal

View GitHub Profile
View servicedeftest.txt
[ec2-user@ip-172-31-3-208 nifi]$ cat privacera_nifi.json
{
"isEnabled": "true",
"type": "nifi",
"configs": {
"username": "nifi"
},
"name": "privacera_nifi",
"description": "nifi repo"
}
View hadoopcleanup
ambari-server stop
ambari-agent stop
pkill -9 java
#################################
# Remove Packages
################################
yum -y remove ambari-\*
yum -y remove hcatalog\*
yum -y remove hive\*
View gist:58d2379023407ed2857d4957ad29fb77
neeraj_mac:emrfs-cf neerajsab$ ssh -i ~/.ssh/ns-privacera-prod.pem centos@34.229.52.168
[centos@ip-172-31-41-233 ~]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/centos/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/centos/.ssh/id_rsa.
Your public key has been saved in /home/centos/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:Y3A4qhJucOe2XXxzpdtD6OFcnlCRAYYPPNArIniyUBs centos@ip-172-31-41-233.ec2.internal
View S3demo
import subprocess,json,sys
def runAndGet(cmd,parse=False):
print(cmd)
dummy=raw_input()
try:
data = subprocess.check_output(cmd, shell=True)
except:
sys.exit(0)
View gist:5061353405510bcaa0e6c4f0408ff8d8
import subprocess,json,sys
def runAndGet(cmd,parse=False):
print(cmd)
dummy=raw_input()
try:
data = subprocess.check_output(cmd, shell=True)
except:
View Cluster Install info
——————————————————————————————————————
*** Initial setup before Ambari setup ***
——————————————————————————————————————
OS checks:
yum install wget
View Zeppelin and SparkR
#Install R and then following packages
#repr failed to create
yum install R-*
install.packages("evaluate", dependencies = TRUE)
install.packages("base64enc", dependencies = TRUE)
install.packages("devtools", dependencies = TRUE)
install_github('IRkernel/repr')
install.packages("dplyr", dependencies = TRUE)
install.packages("caret", dependencies = TRUE)
install.packages("repr", dependencies = TRUE)
View cassandra advrep
##Start cluster
/Users/neeraj/dse/bin/dse cassandra
## - Stop cluster
/Users/neeraj/dse/bin/dse cassandra-stop
## enable advanced replication
vi /Users/neeraj/dse/resources/dse/conf/dse.yaml
./nodetool status
export CQLSH_HOST=192.168.1.185
./cqlsh
# run this in edge and hub cluster
View HawqHive test
su - hdfs
git clone https://github.com/pivotalsoftware/pivotal-samples.git
# I copied the files on /mnt
cd /mnt/pivotal-samples/sample-data
# load data into HDFS
sh -x load_data_to_HDFS.sh ## it will show you the output of the commands the script is running
hdfs dfs -ls /retail_demo
#### Hive table #####
https://github.com/pivotalsoftware/pivotal-samples/blob/master/hive/create_hive_tables.sql
View Druid setup
## git clone https://github.com/Banno/druid-docker
## git clone https://github.com/zcox/druid-pageviews
cd druid-docker
fig kill && fig rm --force
./build.sh
fig up -d druid
#### docker
## `boot2docker shellinit`
## docker ps
##
You can’t perform that action at this time.