Linux/Unix Cheat Sheet

BashCheatSheet.md

Bash Cheat Sheet

If

If Directory Exists

Stack Overflow

if [ -d "$DIRECTORY" ]; then
  # Control will enter here if $DIRECTORY exists.
fi

Or to check if a directory doesn't exist:

if [ ! -d "$DIRECTORY" ]; then
  # Control will enter here if $DIRECTORY doesn't exist.
fi

Note:

Symbolic links may have to be treated differently, if subsequent commands expect directories:

if [ -d "$LINK_OR_DIR" ]; then 
  if [ -L "$LINK_OR_DIR" ]; then
    # It is a symlink!
    # Symbolic link specific commands go here.
    rm "$LINK_OR_DIR"
  else
    # It's a directory!
    # Directory command goes here.
    rmdir "$LINK_OR_DIR"
  fi
fi

Take particular note of the double-quotes used to wrap the variables, the reason for this is explained by 8jean in another answer.

If the variables contain spaces or other unusual characters it will probably cause the script to fail.

If Environment Variable is Set

[ -z "$VARIABLE" ] && VARIABLE="abc"

if env | grep -q ^VARIABLE=
then
  echo env variable is already exported
else
  echo env variable was not exported, but now it is
  export VARIABLE
fi

Note:

I want to stress that [ -z $VARIABLE ] is not enough, because you can have VARIABLE but it was not exported. That means that it is not an environment variable at all.

DateCheatSheet.md

Date

Examples

Datetime in local timezone

echo "$(date +'%Z'): $(date +'%Y-%m-%dT%H:%M:%S.%N%:z')"

Datetime in specified timezone

echo "$(TZ=UTC date +'%Z'): $(TZ=UTC date +'%Y-%m-%dT%H:%M:%S.%N%:z')"

HelpfulLinuxCommands.md

Helpful Unix Commands

File Manipulation

Email File

cat readme.txt | uuencode readme.txt | mail damc.dev@gmail.com

Requires

• uuencode
• mail

File Length

Show count of lines for each files in current directory

find . -type f -exec wc -l {} \;

Show total count of lines for files in current directory

find . -type f -exec wc -l {} \; | awk '{total = total + $1}END{print total}'

Processes

View Command Lines of Java Processes Running as User

ps -fp $(pgrep -x java -d, -u david)

View Full Command Line for Process

ps -fwwp <PID>

Get Pid for Process Matching Arguments

pgrep -f "app.name=my-application"

Get Process Listening on Port

ps -fp $(netstat -nlp 2>/dev/null | grep :8080 | awk '{print $7}' | cut -d'/' -f1)

Parsing

Grep For Substring (Perl Regex)

echo "java -Dapp.version=1.0.0-SNAPSHOT -jar hello.jar" | grep -oP "(?<=app.version=).*?(?= )"

Stack Overflow: Can grep output only specified groupings that match?

Expanded Grep

Grep for 2 lines before match

printf "1. Hello,\n2. how\n3. are\n4. you\n5. today\n" | grep "you" -B 2 

Grep for 3 lines after match

printf "1. Hello,\n2. how\n3. are\n4. you\n5. today\n" | grep "Hello" -A 3 

JSON Objects From Log File into Array

grep -o \{\s*.*\\} application.log | awk '{print $0, ","}' | echo -e "[$(cat -)]" > application.json

Note:

Will need to remove a comma from the last line for valid JSON array

String Replace in File(s)

Replacing all occurrences of one string with another in all files in the current directory

These are for cases where you know that the directory contains only regular files and that you want to process all non-hidden files. If that is not the case, use the approaches in 2.

All sed solutions in this answer assume GNU sed. If using FreeBSD or OS/X, replace -i with -i ''.

Non recursive, files in this directory only:

sed -i -- 's/foo/bar/g' *
perl -Ti -pe 's/foo/bar/g' ./*

(the perl one will fail for file names ending in | or space)).

Recursive, regular files (including hidden ones) in this and all subdirectories

find . -type f -exec sed -i 's/foo/bar/g' {} +

If you are using bash, bash having no support for glob qualifiers, you can't check for regular files:

shopt -s globstar
shopt -s dotglob

Then:

sed -i -- 's/foo/bar/g' **/*

Replace only if the file name matches another string / has a specific extension / is of a certain type etc

Replace only if the string is found in a certain context

[Multiple replace operations: replace with different strings](http://unix.stackexchange.com/questions/112023/how-can-i-replace-a-string-in-a-files#3. Replace)

[Multiple replace operations: replace multiple patterns with the same string](http://unix.stackexchange.com/questions/112023/how-can-i-replace-a-string-in-a-files#3. Replace)

File Transfer

SCP

Local-to-Remote

scp /path/file-on-local david@my-remote-host:~/

Remote-to-Local

scp david@my-remote-host:/path/file-on-remote ~/

Specify Private Key

scp -i ~/.ssh/my-private-key.id_rsa /path/file-on-local david@my-remote-host:~/

SSH

SSH Multiple Commands

ssh otherhost << EOF
  ls some_folder;
  ./someaction.sh 'some params'
  pwd
  ./some_other_action 'other params'
EOF

Note:

If you get: "Pseudo-terminal will not be allocated because stdin is not a terminal. – "

You may be able to (depending what you're doing on the remote site) get away with replacing the first line with ssh otherhost /bin/bash << EOF

Specify Private Key

ssh -i ~/.ssh/my-private-key.id_rsa david@my-remote-host echo \$HOSTNAME

TarCheatSheet.md

TAR

Usage

Type tar then -option(s)

Options list:

• -c --- create.
• -v --- verbose, give more output, show what files are being worked with (extracted or added).
• -f --- file (create or extract from file) - should always be the last option otherwise the command will not work.
• -z --- put the file though gzip or use gunzip on the file first.
• -x --- extract the files from the tarball.
• -p --- preserves dates, permissions of the original files.
• -j --- send archive through bzip2.
• --exclude=pattern --- this will stop certain files from being archived (using a standard wild-card pattern) or a single file name.

Create a tape archive (no compressing)

tar -cvpf name_of_file.tar files_to_be_backed_up 

Extract files (verbosely) from a gzipped tape archive

 tar -zxvpf my_tar_file.tar.gz