Useful Linux commands

Compression
------------
http://linux.dsplabs.com.au/tar-how-to-create-and-extract-tar-gz-and-bz2-compressed-archives-under-linux-p39/

**Creating archives tar
tar cf dir.tar dir/ 
tar czf dir.tar.gz dir/ 
tar cjf dir.tar.bz2 dir/

**Extracting archives
tar xf dir.tar 
tar xzf dir.tar.gz 
tar xjf dir.tar.bz2 

How to locally install 7zip in home directory on linux
http://isezen.com/2011/08/31/how-to-locally-install-7zip-in-home-directory-on-linux/

**Creating archives 7z
7zr a directory.7z directory
7zr a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on archive.7z dir1 

**Extracting archives 7z
7zr x -so directory.tar.7z | tar xf -

**Creating archives gzip
gzip -c in > out.gz

Screen
-------
http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to/
Ctrl-A + ?		screen help page
Ctrl-A + c		new window
Ctrl-A + n		next window
Ctrl-A + p		previous window
Ctrl-A + k		kill window or exit
Ctrl-A + d		detach screen
screen -ls		screen listing
screen -r cod	re-attach to a session

Sed
----
String replace http://www.brunolinux.com/02-The_Terminal/Find_and%20Replace_with_Sed.html
$ sed -i 's/ugly/beautiful/g' /home/bruno/old-friends/sue.txt


Intersection between two files
http://www.commandlinefu.com/commands/view/5710/intersection-between-two-files
----
grep -Fx -f file1 file2


Randomly Pick Lines From a File Without Slurping It With Unix
----
if you have that many lines, are you sure you want exactly 1% or a statistical estimate would be enough?
In that second case, just randomize at 1% at each line...
awk 'BEGIN {srand()} !/^$/ { if (rand() <= .01) print $0}'
perl -ne 'print if (rand() < .01)' your_file.txt 
rl

Split a file
-----
Each split with an specific number of lines (e.g., 100)
split -d -l 100 triples-all-uniq-new-400-sample-00002 triples-all-uniq-new-400-sample-00002-
Split into n parts


Shuffle lines
http://www.commandlinefu.com/commands/view/2754/pick-a-random-line-from-a-file
-----
shuf file.txt | head -n 1


Display the standard deviation of a column of numbers with awk
-----
awk '{sum+=$1; sumsq+=$1*$1} END {print sqrt(sumsq/NR - (sum/NR)**2)}' file.dat
more betterleave-ten-out-c-10-normalized | awk -F ":" '$0~/^Accuracy/ {sum+=$2; sumsq+=$2*$2; count+=1} END {print "Avg.=" sum/count " +-sd="  sqrt(sumsq/count - (sum/count)**2)}'

Merge column from different files
-----
Option 1:
*http://www.techrepublic.com/article/lesser-known-linux-commands-join-paste-and-sort/5031653
paste triples-predictors-01 triples-all-uniq-new-normalized-01 > triples-all-uniq-new-classified-01
Option 2:
awk -f syb.awk file1.txt file2.txt

BEGIN {
  FS=":"
  OFS=" "
}

FNR==NR {
  arr[FNR] = $1 OFS $2
  next
}
{
   print arr[FNR], $2
}

Option 3:
*http://www.linuxquestions.org/questions/linux-newbie-8/awk-command-to-merge-columns-from-two-separate-files-into-single-file-522016/
pr -m -t -s\  file1 file2 | gawk '{print $4,$5,$6,$1}'

Option 4:
*http://stackoverflow.com/questions/5467690/how-to-merge-two-files-using-awk
awk 'FNR==NR{a[$1]=$2 FS $3;next}{ print $0, a[$1]}' file2 file1



5) Command editing Tricks

There are many but i find these are the most useful

Ctrl + a = moves to beginning of line

Ctrl + e = moves to end of line

Ctrl + w = deletes the word before the cursor

Ctrl + u = deletes to beginning of line

Ctrl + k = deletes to end of line

Ctrl + y = undo a deletion

Alt + b = moves to beginning of line word by word

Alt + f = moves to end of line word by word


VI
------------------------
Command	Description
q	Quit
q!	Quit without saving changes i.e. discard changes
r fileName	Read data from file called fileName
wq	Write and quit (save and exit)
w fileName	Write to file called fileName (save as)
w! fileName	Overwrite to file called fileName (save as forcefully)