Snippet shell

awk.md

AWK COMMAND

• List of commands you use most often

history | awk '{a[$2]++}END{for(i in a){print a[i] " " i}}' | sort -rn | head

• Remove duplicate entries in a file without sorting.

Using awk, find duplicates in a file without sorting, which reorders the contents. awk will not reorder them, and still find and remove duplicates which you can then redirect into another file.

awk '!x[$0]++' <file>

• Analyse an Apache access log for the most common IP addresses

This uses awk to grab the IP address from each request and then sorts and summarises the top 10.

tail -10000 access_log | awk '{print $1}' | sort | uniq -c | sort -n | tail

• Using awk to sum/count a column of numbers. Takes a input file (count.txt) that looks like: 1 2 3 4 5 It will add/sum the first column of numbers.

cat count.txt | awk '{ sum+=$1} END {print sum}'

• list all file extensions in a directory

find . -type f | awk -F'.' '{print $NF}' | sort| uniq -c | sort -g

netstat.md

NETSTAT command

• Graph # of connections for each hosts

$ netstat -an | grep ESTABLISHED | awk '{print $5}' | awk -F: '{print $1}' | sort | uniq -c | awk '{ printf("%s\t%s\t",$2,$1) ; for (i = 0; i < $1; i++) {printf("*")}; print "" }'

• List the number and type of active network connections

$ netstat -ant | awk '{print $NF}' | grep -v '[a-z]' | sort | uniq -c

• List the number of active network connections

$ netstat -ntu | awk '{print $5}' | cut -d: -f1 -s | sort | uniq -c | sort -nk1 -r

OR

$ netstat -ntu | awk -F"[ :]+" 'NR>2{print $6}'| sort | uniq -c |sort -nr

OR

netstat -ntu | awk ' $5 ~ /^[0-9]/ {print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

OR

netstat -an | grep ESTABLISHED | awk '{print $5}' | awk -F: '{print $1}' | sort | uniq -c | awk '{ printf("%s\t%s\t",$2,$1); for (i = 0; i < $1; i++) {printf("*")}; print ""}'

• check open ports (both ipv4 and ipv6)

netstat -plnt

• Show apps that use internet connection at the moment. (Multi-Language)

netstat -lantp | grep -i stab | awk -F/ '{print $2}' | sort | uniq

storage.md

COMMAND FOR CHECK STORAGE

• du with colored bar graph

du -x --max-depth=1|sort -rn|awk -F / -v c=$COLUMNS 'NR==1{t=$1} NR>1{r=int($1/t*c+.5); b="\033[1;31m"; for (i=0; i<r; i++) b=b"#"; printf " %5.2f%% %s\033[0m %s\n", $1/t*100, b, $2}'|tac

• easily find megabyte eating files or directories

alias dush="du -sm *|sort -n|tail"

• copy working directory and compress it on-the-fly while showing progress

tar -cf - . | pv -s $(du -sb . | awk '{print $1}') | gzip > out.tgz

• Recursively find top 20 largest files (> 1MB) sort human readable format

find . -type f -print0 | xargs -0 du -h | sort -hr | head -20

• Sum file sizes

du -scb