Commands for CLI and bash scripting

Useful_bash_commands.md

Cleanup after jenkins

	rm -rf $(ls -la | grep -E 'nisal.*slave*' | awk '{print $9}')
	rm -rf $(ls -la | grep -E 'nisal.tmp' | awk '{print $9}')
	rm -rf $(ls -la | grep -E '*nisal*.*._.*' | awk '{print $9}')
	rm -rf $(ls -la | grep -E '*nisal*.*tool*' | awk '{print $9}')
	
	kill $(ps aux | grep -E 'nisal.*java -jar /tmp*' | awk '{print $2}')
	kill $(ps aux | grep -E 'nisal.*slave*' | awk '{print $2}')
	kill $(ps aux | grep -E '/usr/lib/jvm/java.*TomcatInstance*' | awk '{print $2}')

##Bash commands I encountered throughout my bash-scripting-experience ###SED ####Add spaces: simply insert “backslash \ followed by spaces”

####Remove directory and contents: rm -rf

$ apt-get --purge remove <package> 
FILE_LIST="`ls *.html`"

####Vim replace > :%s/foo/bar/g #####Find each occurrence of 'foo' (in all lines), and replace it with 'bar'.

:s/foo/bar/g 

#####Find each occurrence of 'foo' (in the current line only), and replace it with 'bar'.

:%s/foo/bar/gc

#####Change each 'foo' to 'bar', but ask for confirmation first.

:%s/\<foo\>/bar/gc

#####Change only whole words exactly matching 'foo' to 'bar'; ask for confirmation.

####RSYNC

rsync -a user@infinity.st.cs.uni-saarland.de:/home/user/DIRECTORY /Users/adityauser/Desktop/DIRECTORY

#####list files according to installation date:

ls -alt

#####starting with specific letter (case-sensitive) :

ls -d D* / ls -d d* (directories)

#####In case of all files/directories starting with h :

ls h*

#####Show directory size :

du -h your_directory

####Install maven

$ wget http://mirror.olnevhost.net/pub/apache/maven/maven-3/3.3.3/binaries/apache-maven-3.3.3-bin.tar.gz
$ sudo tar -zxf apache-maven-3.3.3-bin.tar.gz -C /usr/local/
$ sudo ln -s /usr/local/apache-maven-3.3.3/bin/mvn /usr/bin/mvn
$ export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-i386
export MAVEN_OPTS="-Xmx1024M"

####Install JDK8

$ sudo add-apt-repository ppa:webupd8team/java 
$ sudo apt-get update 
$ sudo apt-get install oracle-java8-installer 
Set default $ sudo apt-get install oracle-java8-set-default 

####VNC server : screen sharing

COPY FOLD ER and ALL contents recursive 
$ cp -R /src /dest 

INSTALL LAMP SERVER

$ sudo apt-get install lamp-server^ 

####SSH keyless Authentication

http://coolestguidesontheplanet.com/make-passwordless-ssh-connection-osx-10-9-mavericks-linux/

####CREATE ALIASES

$ cd home 
$ vim .bash_profile
$ sudo chmod 700 ~/.bash_profile
$ infinity=’ssh user@someremoteserverIP’
$ source ~/.bash_profile
$ infinity
$ exit #exits ssh session

####SSH into guest VM

Change VBox config and add host-only adapter from preferences
open Ubuntu VM and add following lines to /etc/network/interfaces

\# interfaces(5) file used by ifup(8) and ifdown(8)
\# The loopback network interface
auto lo
iface lo inet loopback

\# The primary network interface
auto eth0
iface eth0 inet dhcp

\# The secondary network interface (static IP)
auto eth1
iface eth1 inet static
address 192.168.56.10
netmask 255.255.255.0

3. sudo reboot

####VIMRC

if has("mouse")
        set mouse=a
    endif

set mouse=a
if has("mouse_sgr")
    set ttymouse=sgr
else
    set ttymouse=xterm2
end

####TMUX Detach session:

ctrl-B + D (caps on)
tmux detach

Create a new tmux session named session_name: tmux new -s session_name

Create a new tmux session IF already present ELSE attach to existing session of same name : tmux new -A -s session_name

Switch to an existing session named session_name: tmux s -t session_name

List existing tmux sessions tmux list-sessions

Kill session tmux kill-session -t session_name

####Current directory:

CURRENT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

####Find a directory find <path-where-you-wanna-find OR "/" for everywhere OR "." for-current-dir> -type d -name '<dir-name>'

###Find a file find <path-where-you-wanna-find OR "/" for-everywhere OR "." for-current-dir> (-type f) -name '<file-name>'

####Multi-hop SSH port forwarding If you want to forward a port (say tomcat's 8080) from infinity (a remote server) on my home machine

[adi@home] $ ssh -L8888:localhost:8888 adi@chair
[adi@chair]$ ssh -L8888:localhost:8080 adi@infinity

Above example by default binds the port to localhost (127.0.1.1 -> loopback interface).

For wildcard IP address port forwarding,

1. Enable GatewayPorts yes under sshd_config

ssh -g -L *:8000:localhost:8000 adi@chair
ssh -g -L *:8000:localhost:8000 adi@infinity

####Check if you can connect to a particular port on particular IP telnet 134.96.111.96 80

####Create your OWN bash-completion in home directory (without root privs)

mkdir -p ~/.bash_completion.d

Then,

vim ~/.bash_completion.d/tma
        _tma() {
                TMUX_SESSIONS=$(tmux ls -F '#S' | xargs)
                local cur=${COMP_WORDS[COMP_CWORD]}
                COMPREPLY=( $(compgen -W "$TMUX_SESSIONS" -- $cur) )
        }
        complete -F _tma tma

Then, create an alias in .bashrc or .bash_profile and source the file

alias tma='tmux attach -t $1'
if [ -f ~/.bash_completion.d/tma ]; then
. ~/.bash_completion.d/tma
fi

source ~/.bash_completion.d/tma

For loop iteration over array

#!/bin/bash

declare -a arr=("version1" "version2")

## now loop through the above array
for JenkinsVersion in "${arr[@]}"
do
   	if [ ! -f  /home/$user/JenkinsWarFiles/jenkins"$JenkinsVersion".war ]; then
	wget https://updates.jenkins-ci.org/download/war/$JenkinsVersion/jenkins.war -O /home/$user/JenkinsWarFiles/jenkins"$JenkinsVersion".war
	fi
   echo "$JenkinsVersion"
done

Using the lovely grep command

Grep a process (eg tomcat) with pid

ps aux | grep tomcat | awk '{print $2}'

Kill a process (eg tomcat) with pid

kill $(ps aux | grep tomcat | awk '{print $2}')

Grep a string from a directory

grep -R 'string' dir/

Java heap size printing

java -XX:+PrintFlagsFinal -version | grep -iE 'HeapSize|PermSize|ThreadStackSize'

Check if a PORT is listening

netstat -a -n | grep 4444

Delete all files in a directory for MY user

rm -rf `ls -la | grep 'nisal' | awk ' { print $9 } '`

Delete all files in a directory for MY user and matching specific pattern

This command will delete files such as slave.jar jenkinsslave.jar slavejenkins.jar etc.

rm -rf $(ls -la | grep -E 'nisal.*slave*' | awk '{print $9}')

Kill processes matching two patterns

kill $(ps aux | grep -E '/usr/lib/jvm/java.*TomcatInstance'$startupPort'*' | awk '{print $2}')

Kill CLOSE_WAIT processes

 netstat -tulnap | grep CLOSE_WAIT | grep 8887 | awk '{print $7}' | cut -d \/ -f1 | grep -oE "[[:digit:]]{1,}" | xargs kill

Grep multiple lines matching a pattern AND enumerate them AND ignore some lines matching another pattern

grep -E 'tests/desktop/' fireplaceTests_MV2_2015_02_10.log
grep -E 'tests/desktop/' fireplaceTests_MV2_2015_02_10.log | nl -v 0
grep -E 'tests/desktop/' fireplaceTests_MV2_2015_02_10.log | grep -v 'SKIPPED' | nl -v 0
grep -E 'tests/desktop/' fireplaceTests_MV2_2015_02_10.log | grep -v 'SKIPPED' | nl -v 0 | awk '{print $3}'

Sampling logged data

head Adi_conversions24thJune_final.csv | sed 's/\(.*\)    .*:/\1  /'
cat xaa.gznon_split.tsv.gz | head -n 1 | tr '\t' '\n' | nl
cat 160610-deliveries.gz | cut -f 10,1,44,45,51 | grep --color=auto -v NULL | grep --color=auto 2016-05-24 > ADI_conversions24thJune_NEW.csv

👍