Skip to content

Instantly share code, notes, and snippets.

View pm-hwks's full-sized avatar
:octocat:

pm-hwks pm-hwks

:octocat:
View GitHub Profile
@pm-hwks
pm-hwks / hdf30_sandbox_startup.sh
Last active October 23, 2017 23:16
[HDF 3.0 Docker container startup script] use this script to start HDF 3.0 docker container #docker #hdf #sandbox
#!/bin/bash
echo "Deploying HDF Sandbox Container:"
until docker ps 2>&1| grep STATUS>/dev/null; do sleep 1; done; >/dev/null
docker ps -a | grep sandbox-hdf
if [ $? -eq 0 ]; then
docker start sandbox-hdf
else
docker run --name sandbox-hdf --hostname "sandbox-hdf.hortonworks.com" --privileged -d \
-p 9999:9999 \
-p 12181:2181 \
@pm-hwks
pm-hwks / start_sandbox-hdp.sh
Last active September 7, 2021 11:23
[HDP sandbox startup] Use this script to start HDP sandbox #docker #hdp #sandbox
#!/bin/bash
echo "Waiting for docker daemon to start up:"
until docker ps 2>&1| grep STATUS>/dev/null; do sleep 1; done; >/dev/null
docker ps -a | grep sandbox-hdp
if [ $? -eq 0 ]; then
docker start sandbox-hdp
else
docker run --name sandbox-hdp --hostname "sandbox.hortonworks.com" --privileged -d \
-p 1111:111 \
-p 1000:1000 \
@pm-hwks
pm-hwks / install_pip_package.py
Created October 23, 2017 23:19
[Install pip packages using code] use this script to install python packages using nothin but python code and also list installed packages #python #pip #packages #install
# Install requests package and list packages
import pip
# install request package using pip
pip.main(['install', 'requests'])
# List installed packges
installed_packages = pip.get_installed_distributions()
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
for i in installed_packages])
@pm-hwks
pm-hwks / config.txt
Created October 24, 2017 18:16
[gmail SMTP info] smtp info to be used configuring outgoing emails #gmail #smtp #notifications
Gmail SMTP server address: smtp.gmail.com
Gmail SMTP username: Your Gmail address (e.g. example@gmail.com)
Gmail SMTP password: Your Gmail password
Gmail SMTP port (TLS): 587
Gmail SMTP port (SSL): 465
Gmail SMTP TLS/SSL required: yes
@pm-hwks
pm-hwks / hdf31_docker_setup.sh
Last active June 9, 2018 20:26
[Download and setup HDF 3.1 docker container] Use this script to download and setup HDF docker container #hdf #docker #setup #hdf31
### Download and run HDF 3.1 docker image
#!/bin/bash
echo "Waiting for docker daemon to start up:"
until docker ps 2>&1| grep STATUS>/dev/null; do sleep 1; done; >/dev/null
docker ps -a | grep sandbox-hdf
if [ $? -eq 0 ]; then
docker start sandbox-hdf
else
docker pull hortonworks/sandbox-hdf-standalone:3.1.0
docker run --privileged --name sandbox-hdf -h sandbox-hdf.hortonworks.com -itd \
@pm-hwks
pm-hwks / hdp_docker_setup.sh
Last active June 9, 2018 20:25
[Download and setup HDP 2.6 docker container] Use this script to download and setup HDP docker container #hdp #docker #setup #hdp26
### Download and run HDP 2.6.4 docker image
#!/bin/bash
echo "Waiting for docker daemon to start up:"
until docker ps 2>&1| grep STATUS>/dev/null; do sleep 1; done; >/dev/null
docker ps -a | grep sandbox-hdp
if [ $? -eq 0 ]; then
docker start sandbox-hdp
else
docker pull hortonworks/sandbox-hdp-standalone:2.6.4
docker run --name sandbox-hdp --hostname "sandbox-hdp.hortonworks.com" --privileged -d \
@pm-hwks
pm-hwks / setup.sh
Created October 24, 2017 21:26
[Run NiFi on macos] Running NiFi on OSX #nifi #macos
## Change to directory where you downloaded and extracted the latest version of NiFi
# Setup Java Home
export JAVA_HOME=$(/usr/libexec/java_home)
export PATH=${JAVA_HOME}/bin:$PATH
#Start NiFi
bin/nifi.sh start
## Access NiFi at http://localhost:8080/nifi
@pm-hwks
pm-hwks / dd.sh
Last active January 9, 2020 21:47
[macOS disk utility and dd commands] Use this create USB disk images and restore from images #usb #macos #disk_utility #dd #image #restore #wip
## **** This is a work in progress snippit **** ##
## Link https://www.cyberciti.biz/faq/how-to-create-disk-image-on-mac-os-x-with-dd-command/
# Step 1: Erase & format the usb drive with exFAT using the macos disk utility app
# step 2: list all the disk and identify the usb disk
diskutil list
# step 3: unmount disk using the disk utility macos app
@pm-hwks
pm-hwks / dd.sh
Last active February 25, 2018 21:12
[Generate Test data] DD command to generate test file #disk_utility #dd #generate #test_file #test_data
# Generate test file
dd if=/dev/zero of=/tmp/test2.img bs=512 count=1000 oflag=dsync
@pm-hwks
pm-hwks / iperf.sh
Created February 25, 2018 21:55
[Network benchmarking - iperf] iperf3 network speed testing and benchmarking #iperf #nw #nw-test #nw-benchmark #benchmark
## Install iperf3 using package manager
## Start iperf server
iperf3 -s <serverip>
## Start client and test speed against server
iperf3 -c <serverip>