Linux learnings

gistfile1.txt

Part 1
Linux Command Line

As a web developer you are assigned a task to administer Linux based web server or your company says we don't like Windows you must have to choose Mac or Linux based system. Things get worse when you are afraid of command line and hesitate to write commands yourself. This series of articles will help you jump start with command line quickly,  easily, and after reading the whole series, you will not fear to run commands yourself.

Prerequisite
I assume you have some basic understanding of Unix/Linux based systems and at least must be comfortable working with the GUI environment of the OS.

Bash
Bash (Bourne Again SHell) is a shell program that interprets the command you enter and decides whether to execute itself, or execute using other binary, or execute a script. Bash is part of GNU project and it's was developed as a replace for sh (Bourne shell) used in Unix. Bash not only comes with Linux but also the default shell for Mac OS X. So the commands which you will learn in this series will work in both Linux and Mac OS X.

Terminal Emulator
Today almost every client OS comes with GUI so we need some terminal emulation program which allows us to emulate Bash shell in a terminal window. Default terminal in GNOME is gnome-terminal in GNOME, and konsole in KDE based environments. Similarly default terminal emulator in Mac OS X is Terminal.app.

I am a Linux user and will use gnome-terminal in Ubuntu for running commands. Don't worry all the commands will also work in other Linux distributions and may work in Mac OS X as well but output may vary a bit. I will only show you the most commonly used options for each command. As it's not a reference guide. My point is to give you basic understanding of each command so you can further explore it on your own.

User, Machine, Privilege Info
Open your terminal emulator and there you will see info related to logged in user, machine name, and a sign which represents super user or normal user. On my notebook it shows:

ifadey@Inspiron-1564:~$

ifadey is my username separated by @ symbol and then Inspiron-1564 is the computer name. Then a $ (dollar sign) means I am a standard user and don't have root/admin privileges. If your terminal is displaying # (hash) instead of $, it means your user have root privileges.

Command
What exactly is a command? When you type a command in the terminal and hit enter. What's the thing that makes it perform some operation? Actually the command can be an executable binary, or a Bash built-in command (a command integrated with Bash shell) or a script or an alias to some existing command. You can check the command type using type command. For example pwd, and cd are shell built-in commands and you can check that by typing the following commands in the terminal.

ifadey@Inspiron-1564:~$ type pwd 
pwd is a shell builtin

ifadey@Inspiron-1564:~$ type cd 
cd is a shell builtin


Similarly cp and mkdir are executable binaries. Type following commands to check it.

ifadey@ifadey-Inspiron-1564:~$ type cp 
cp is /bin/cp 

ifadey@ifadey-Inspiron-1564:~$ type mkdir 
mkdir is /bin/mkdir 

Don't worry what these commands are. For now just remember there are four types of commands and you can check that using type command.

Navigating the File System
In GUI, you may navigate the file system using Nautilus (default file manager in GNOME) or Dolphin (default in KDE). You browser your file system using these graphical file managers or may be some other. But in command line you can navigate the file system using three commands.

1. ls - List directory contents
2. cd - Change the shell working directory.
3. pwd - Print the name of the current working directory.

pwd
pwd command prints Present or Current Working Directory. It's the currently targeted directory where you can run commands like ls, cp, rm, mkdir, etc without providing them the path as an argument. You will see later what I mean by command argument.

Open your terminal and type pwd.

ifadey@Inspiron-1564:~$ pwd 
/home/ifadey

ls Command
This command is used to display contents of directory. Type ls at your terminal.

ifadey@Inspiron-1564:~$ ls 
Desktop    Downloads  examples.desktop  Public     Ubuntu One 
Documents  Music      Pictures          Templates  Videos 

You will see the list of files and folders in current working directory. Now type “ls /” command to display contents of root directory.

ifadey@Inspiron-1564:~$ ls /

Even though our current directory is home but you can display contents of other directories using command arguments. In the above command / is the argument passed to ls command. Try another command.

ifadey@Inspiron-1564:~$ ls /usr

Absolute Path
Above command display contents of usr directory under root. Note that the argument passed in above command is /usr. This is an absolute path or full path because it's not relative to some directory. Also absolute paths start always start from root directory / (forward slash). Using absolute paths will always give same output no matter what your current working directory is.

Relative Path
Relative path define a location relative to current working directory. Let's see some commands some examples of ls command with relative paths.

ifadey@Inspiron-1564:~$ ls .

Above command is not simple ls. It's ls (dot). In CLI, . (dot) symbol means current directory. So when you ran the above command, it actually printed the contents of current working directory as it did when ran without (dot) argument.

Similarly .. two dots represent the parent directory of current directory. Let's try it.

ifadey@Inspiron-1564:~$ ls ..
asalam  ifadey

Above command print the contents of /home directory because it's the parent directory of my current directory (/home/ifadey).  Now above path is a relative path because it targets a directory relative to present directory. Here some more examples of relative paths.

ifadey@Inspiron-1564:~$ ls ../asalam/Desktop/
ifadey@Inspiron-1564:~$ ls ./Downloads/
ifadey@Inspiron-1564:~$ ls Downloads/
First path passed to ls command moves to its parent directory because of .. (two dots). And then it navigates to asalam and finally to Desktop in asalam.

As mentioned above . (dot) represents the current directory. So in second command, path targets Downloads directory in current directory.

Third command will produce the same output as second command did. If you write a directory name, or file name without any prefix, will try to find it in the current directory.

Command Options
Every command has options with which you can change its behavior. Try ls command with -l option.

ifadey@Inspiron-1564:~$ ls -l
drwxr-xr-x 2 ifadey ifadey 4096 Jun 23 00:33 Documents 
drwxr-xr-x 4 ifadey ifadey 4096 Jul 19 20:11 Downloads 
drwxr-xr-x 2 ifadey ifadey 4096 Jun 30 16:24 dwhelper 
-rw-r--r-- 1 ifadey ifadey 8445 Jun 23 00:27 examples.desktop

-l option display the contents of directory in long detailed format. This format displays information in following manner.

1. Note the first alphabet d or – (dash) in each row. If it displays d, it means it's a directory and – (dash) means it's a file.
2. Then first rwx means owner of this file/directory have read, write, and execute privileges. More info on file rights in later articles.
3. Second rwx is for all the users in group to which this file/directory belongs.
4. Third rwx is for everyone else. If you find – (dash) in rwx, it means that particular privilege is missing. For example r-x means read, no write access, and execute.

5. Second column display numbers (2, 4, 2, 1). These numbers represent how many hard links exist for this file/directory. More info on links in later articles.

6. Third column display usernames who own the file/directory.

7. Fourth column display the group name to which this file/directory belongs.

8. Then size of  file/directory in bytes. Use -h option to display sizes in human readable format. For example ls -lh.

9. Sixth column displays the Last modified date/time.

10. Finally the name of  file/directory.

In *nix based operating systems, hidden files are prefixed with . (dot). For example .filename is a hidden file. Use -a option to display all files including the hidden ones as well.

ifadey@Inspiron-1564:~$ ls -a

Use ls --help option to view all the options available for ls command.

cd command
cd command is used to change current working directory to some other directory according to the path passed as argument to it. When you open the terminal, default directory is your home directory (/home/username). You can always check your current directory using pwd command. Here's an example using both cd and pwd commands.

ifadey@Inspiron-1564:~$ pwd 
/home/ifadey 
ifadey@Inspiron-1564:~$ cd Desktop/ 
ifadey@Inspiron-1564:~/Desktop$ pwd 
/home/ifadey/Desktop 

Above example changes my current directory from ifadey to Desktop. This command can work with both absolute and relative paths like ls command. Here are some more examples.

ifadey@Inspiron-1564:~$ cd /usr/share/ 
ifadey@Inspiron-1564:/usr/share$ cd ../bin

cd command is easy to use but there are some interesting shortcuts which you can use with this command. First no matter what your current directory is, simply type cd without passing any argument. It will take you to your home directory.

ifadey@Inspiron-1564:/usr/share$ pwd 
/usr/share 
ifadey@Inspiron-1564:/usr/share$ cd 
ifadey@Inspiron-1564:~$ pwd
/home/ifadey 

Second shortcut is cd - (dash). This will take you to previous working directory. It's similar to Back button in GUI based file manager. Here's an example usage.

ifadey@Inspiron-1564:~$ cd Pictures/ 
ifadey@Inspiron-1564:~/Pictures$ pwd 
/home/ifadey/Pictures 
ifadey@Inspiron-1564:~/Pictures$ cd - 
/home/ifadey 

Another useful shortcut is cd ~username. It will navigate or change your current directory to home directory of user whose username is entered after ~ (Tilde) sign.

ifadey@Inspiron-1564:~$ pwd 
/home/ifadey 
ifadey@Inspiron-1564:~$ cd ~asalam 
ifadey@Inspiron-1564:/home/asalam$ pwd 
/home/asalam 

My home directory is /home/ifadey and I moved to other user's home directory using command cd ~asalam.


Need more Help?
Most commands have --help option with which you can view additional details regarding command usage. But you can view more detailed reference for each command using man and help commands.

help command
help command is used to display detailed information along command options for shell built-in commands. If you are not sure about command type, you can check it using type command as explained above. Here's an example usage:

ifadey@ifadey-Inspiron-1564:~$ type cd 
cd is a shell builtin 
ifadey@ifadey-Inspiron-1564:~$ help cd

You will see a long description with command options after running the help command.

man command
man (manual) command is used to display reference documents for executable commands. Actually it contains reference material for other things as well like System Calls, C library APIs, and much more. But for the time being remember this command can give you reference documents for executable commands. type command will print the path to binary/executable file used to run that particular command.

ifadey@ifadey-Inspiron-1564:~$ type cp 
cp is /bin/cp 
ifadey@ifadey-Inspiron-1564:~$ man cp

Pressing q will exist man pages.

Review
In this first part you learned about command types and the type command. Then you learned how to navigate the file system using pwd, cd, and ls commands. In the end you learned commands to get help regarding the command usage. help command for shell built-in and man for binaries.

=================== END ===================


























Linux Command Line (Part 2)

In first part you learned about navigating file system using Bash shell and you also learned how to get help for a particular command. In this part you will learn about

* Filesystem Hierarchy Standard.
* Create, Copy, Move, and Delete files or directories.
* Rename file or directories.

Previous Parts
* Part 1 (Link to Part 1)


Filesystem Hierarchy Standard
In previous part you learned to navigate file system but what if you don't know where to find the files you want or in other words what is in different directories (bin, usr, var, etc, dev, home, and many more) that exist in root of the file system?

Let me give you an example of Windows so you understand what I mean by Filesystem Hierarchy. In Windows all of your partitions (C, D, E, etc) are mounted in Computer. Typically Windows is installed in C drive and with in that drive you find some important directories like Windows, Program Files, All Users, etc. You may know that Windows directory contains the important system files of the OS. Program Files is where user programs are installed. Similarly All Users contain the home directories of all user accounts. So it make up a tree like structure and regular user of any OS understand what different folders/directories contain in this tree structure that exist on the file system. This tree like structure is actually Filesystem Hierarchy.

The good news is that file system hierarchy for Linux is standardized and this standard is known as FHS (Filesystem Hierarchy Standard). Modern distros have tried a lot to get compliant with this standard but still not all of them are 100% compliant with it. So you may find some variations in your Linux distro. Here's a high level view of FHS.

TABLE GOES HERE...

Above table is created with the help of FHS 2.3 document.

Mac OS X is based on Darwin and the Filesystem Hierarchy in Darwin looks pretty similar to that of Linux. OS X users must take look at this link for Filesystem Hierarchy.

Create
Files
There's no specific command for creating a file but you can achieve this task using touch command. Actually touch command is used to modify timestamps of file. But the file will automatically gets created if it doesn't exist. Here is an example which creates a text file on Desktop.

ifadey@Inspiron-1564:~/Desktop$ touch abc.txt

You can also create multiple files by passing more than one argument.

ifadey@Inspiron-1564:~/Desktop$ touch abc.txt xyz.txt

Above command will create two files abc.txt, xyz.txt. Also remember you can use both relative and absolute paths with this command.

Now you can start editing this file using vim, nano, gedit or some other editor. Check touch documentation using man command (man touch).

Directories
Creating directory is pretty simple task. mkdir (Make directory) is the command used to create directories. Here are some examples.

ifadey@Inspiron-1564:~/Desktop$ mkdir myDir
ifadey@Inspiron-1564:~/Desktop$ mkdir dir1 dir2 dir3
ifadey@Inspiron-1564:~/Desktop$ mkdir /home/ifadey/linux

First command creates myDir on Desktop. Second one create three directories (dir1, dir2, dir3). Third one uses absolute path and it creates linux directory in my home directory ifadey.

Remember mkdir command creates a new directory only if it doesn't exist. Check mkdir –help for more info.

Delete
Files
rm (remove) command is used for deleting both files and directories. Here's how you can delete a file.

ifadey@Inspiron-1564:~/Desktop$ rm someFile.txt

CAUTION: Remember that rm command doesn't move files/directories to Trash. It deletes them permanently. So be careful when using it.

You can use both absolute and relative paths with rm command. Here are some more examples.

ifadey@Inspiron-1564:~/Desktop$ rm file1 ../file2 /home/ifadey/file3

Above command removes three files. First from present working directory (which is Desktop in above example). Second file (using relative path) from parent directory (which is ifadey) and third one from the same directory but using absolute path.

Directories
Now create a new directory in your home directory using mkdir and try removing it using rm command.

ifadey@Inspiron-1564:~$ mkdir newDir 
ifadey@Inspiron-1564:~$ rm newDir/ 
rm: cannot remove `newDir/': Is a directory

As you can see rm was unable to remove a directory and gave an error “Is a directory”. It's because you need -r option to delete a directory and its contents recursively. Here's how you can delete the directory.

ifadey@Inspiron-1564:~$ rm -r newDir/ 

Useful Options
Force
Force option -f, when used won't give you error “No such file or directory” if file or directory doesn't exist. It's useful option specially when you write scripts.

ifadey@ifadey-Inspiron-1564:~$ rm -f fileNotExist

Interactive
Interactive option -i, when used will prompt you before removing file or directory.

ifadey@ifadey-Inspiron-1564:~$ rm -ir someDir
rm: remove directory `someDir'? y

Type y for yes and n for no. Note that two options interactive and recursive (-ir) are used together in above command.

Verbose
Finally verbose option -v is useful when deleting multiple files or directory containing multiple files. Verbose option will show you each file/directory that rm deleted.

ifadey@Inspiron-1564:~/Desktop$ rm -rv tstDir/ 
removed `tstDir/file1' 
removed `tstDir/file3' 
removed `tstDir/file2' 
removed directory: `tstDir'

Copy
Files
cp command is used to copy files and directories. cp command accept list of files/directories which you want to copy and then the destination directory as final argument. Here are some examples.

ifadey@Inspiron-1564:~$ cp Desktop/tst.htm .
ifadey@Inspiron-1564:~$ cp readme.txt script.php /var/www
ifadey@Inspiron-1564:~$ cp /var/www/launch.py /media/myStorageDevice

In first example, tst.htm file is copied from Desktop to current working directory. Note the arguments in the first command. First argument (SOURCE) is Desktop/tst.htm which is a relative path and refers to tst.htm file in Desktop directory which is in current directory. Second argument (DESTINATION) is . (dot) which represents current directory.

Similarly in second example, two files are copied from current directory to www directory. In this example first two arguments are the source files. In short multiple sources are allowed and destination must be single directory.

Final command uses absolute paths to copy launch.py (Python script file) from www directory to some external storage device.

NOTE: In modern Linux distributions, external devices (CD/DVD Discs, USB HDD or SSD, etc) are automatically mounted in /media directory. More info on directory structure of Linux coming in later parts.
Directories
Copying directories is similar to copying files with one exception and i.e. recursive option -r is required to copy directories just like rm command required it for deleting directories. Rest of the rules are same as for copying files.

ifadey@Inspiron-1564:~$ cp -r dir1 copy_of_dir1
ifadey@Inspiron-1564:~$ cp -r Pictures Desktop
ifadey@Inspiron-1564:~$ cp -r dir2 dir3 /home
ifadey@Inspiron-1564:~$ cp -r /home/ifadey/Downloads/books eBooks

First example copy dir1 in the same directory (current directory) but with different name copy_of_dir1.

Second example copy Pictures directory from current directory to Desktop directory.

Third example copy two directories from current directory to home directory. Note Both dir2 and dir3 are relative paths (relative to current directory) and /home is absolute path.

Final example uses absolute path to copy books directory to eBooks directory (relative path) in current directory.

cp command have similar important options as rm command have.

Useful Options
Force
Force option -f with cp command works bit differently than it did in rm command. By default cp command overwrite existing files but if it's not able to overwrite any file and -f option is used then it will remove the existing file and try to copy the new one. Here's an example:

ifadey@ifadey-Inspiron-1564:~$ cp -f Documents/article.odt ./

Above command copies article.odt from Documents to current directory.

Interactive
Interactive option -i will prompt before overwrite.

ifadey@ifadey-Inspiron-1564:~$ cp -i examples.txt dwhelper/tst 
cp: overwrite `dwhelper/tst'? y

This option works exactly in same way as it did for rm command.

Update
Update option -u is used for copying only when the SOURCE file is newer than the destination file or when the destination file is missing.

ifadey@ifadey-Inspiron-1564:~$ cp -u Documents/article.odt ./

Verbose
Verbose option -v again works in same way it did for rm command. When coping files, verbose option will show you each file/directory that cp copied.

ifadey@Inspiron-1564:~/Desktop$ mv -v dir ../ 
`dir' -> `../dir'

Above command moves dir directory from Desktop to parent directory and display details after performing its operation because of verbose option -v.

Move
Files
mv command is used to move files around file system and it works in a similar way as cp command did. The only difference is that mv command removes the SOURCE files after completing its operation (just like Cut/Paste in GUI). Here are the same examples from copying files section but with mv command.

ifadey@Inspiron-1564:~$ mv Desktop/tst.htm .
ifadey@Inspiron-1564:~$ mv readme.txt script.php /var/www
ifadey@Inspiron-1564:~$ mv /var/www/launch.py /media/myStorageDevice

Above commands as same as in copy section so I am not going to explain the arguments used with them.
Directories
Moving directories is also same as copying except with one variation. mv command doesn't require recursive option -r to move directories.

ifadey@Inspiron-1564:~$ mv dir1 copy_of_dir1
ifadey@Inspiron-1564:~$ mv Pictures Desktop
ifadey@Inspiron-1564:~$ mv dir2 dir3 /home
ifadey@Inspiron-1564:~$ mv /home/ifadey/Downloads/books eBooks

Useful Options
Force
Force option -f when used with mv command will not prompt before overwriting existing file.

ifadey@ifadey-Inspiron-1564:~$ mv -f Documents/article.odt ./

Above command moves article.odt from Documents to current directory and will not prompt even if article.odt already exist.

Interactive
Interactive option -i is opposite to -f option. It will prompt before overwriting.

ifadey@ifadey-Inspiron-1564:~$ mv -i examples.txt dwhelper/tst 
mv: overwrite `dwhelper/tst'? y

Update
Update option -u will move only when the SOURCE file is newer than the destination file or when the destination file is missing.

ifadey@ifadey-Inspiron-1564:~$ mv -u Documents/article.odt ./

Verbose
Remember verbose option -v works almost same with every command. It purpose it to simply display details regarding each option currently performed.

ifadey@Inspiron-1564:~/Desktop$ mv -v nodeServer/ ../
`nodeServer/' -> `../nodeServer'

Rename File/Directories
Another important use of mv command is to rename files and directories. Here's an example for renaming file.

ifadey@Inspiron-1564:~/Desktop$ mv tst.htm index.htm

Above command renames tst.htm file to index.htm. Following example is for renaming directory.

ifadey@Inspiron-1564:~/Desktop$ mv dir1 dir2

Above command renames dir1 to dir2.

Review
In this part you learned about the purpose of each folder in Linux (Filesystem Hierarchy Standard). Then you learned to create (touch and mkdir commands), delete (rm command), copy (cp command), and move (mv command) files and directories. Finally you learned to renames files/directories using mv command.

=================== END ===================


























Linux Command Line – Part 3

* Binaries and File Types in Linux.
* Text File Viewer – less command.

Binaries and File Types in Linux
Every OS has its own way to determine file type like Windows uses file extension for identifying type of file. But Linux don't care about file extensions so it's optional. Actually Linux looks at file header information (stored in the file itself) of a binary/executable to see how to run this file. Typically, executables or binaries in Linux don't have file extension.

As I mentioned above file extension mean nothing to Linux so to determine the type of file, there's a handy command called file which is used to check the type of any file. Here's an example usage:

ifadey@Inspiron-1564:~$ file examples.desktop 
examples.desktop: UTF-8 Unicode text

ifadey@Inspiron-1564:~$ file /usr/lib/firefox/firefox.sh
/usr/lib/firefox/firefox.sh: POSIX shell script, ASCII text executable

ifadey@Inspiron-1564:/bin$ file cp 
cp: ELF 64-bit LSB executable, x86-64, ...

First file is UTF-8 based text file. Second is a shell script used to launch Firefox. Third is 64-bit binary file. As you may have realized the power of file command. It give detail more than type of file.

Creating a Link/Shortcut
Shortcut is known as link in Linux and there are two types of link (Hard Link and Soft Link).

Hard Link
Following command is used to create hard link:

ln file link
There are some differences/limitations you must remember.

1. Hard link cannot refer files outside its partition.
2. Hard link cannot refer directory.
3. You cannot distinguish between original file and newly created hard link.
4. Deleting original file will not be removed from file system if it has one or more hard links pointing to it.
5. Hard link works even if the original file's location is changed.

Let's take a look at an example. To create a hard link of file powertop.webm, use this command:
ln powertop.web linux-powertop.webm

A new hard link linux-powertop.webm is created. Use ls -li command to check whether a file has hard link pointing to it or not. Here's the output of ls -lih.

5243698 -rw-r--r-- 1 ifadey ifadey 1.7K Nov 27 18:51 backboneTest.js 
5244571 -rw-rw-r-- 1 ifadey ifadey 1008 Nov 23 17:26 font-test.htm 
5243934 -rw-rw-r-- 2 ifadey ifadey  64M Dec 19 22:15 linux-powertop.webm 
5243927 -rw-rw-r-- 1 ifadey ifadey  221 Dec 17 00:28 note.txt 
5243777 -rw-rw-r-- 1 ifadey ifadey  36K Dec 12 13:15 style.css 
5243934 -rw-rw-r-- 2 ifadey ifadey  64M Dec 19 22:15 powertop.webm 

linux-powertop.webm is a hard link created from powertop.webm. There are two ways to prove it. First both files have same inode (5243934) in first column of output. Secondly the count in third column is 2 which means there's one hard link pointing to it. You maybe wondering why it's displaying 2 if one hard link exist for the original file? It's because the original file is also a hard link to the data in file system. That's why it's showing 2 as the count of hard links that exist for that particular file.

Note that there's no way to tell from the output which file is the original one.

Symbolic/Soft Link
Following command is used to create Symlink.

ln -s file/dir link

Symlink is actually very similar to shortcut in Windows. It has following differences when compared to hard link.

1. Symlinks can point to both files and directories.
2. Symlinks can point to file/directory even if it's in different partition.
3. Symlinks can be identified by look at the ls -l output. In other words you can actually distinguish b/w original file and sym link.
4. Symlinks will stop working if original file is deleted or its location is changed. In such a case it's called dangling or broken link.

Use following command to create a symlink for that same powertop.webm file.

ln -s powertop.webm s-lnk-powertop.webm

Now run ls -lih command.

5243934 -rw-rw-r-- 1 ifadey ifadey  64M Dec 19 22:15 powertop.webm
5244048 lrwxrwxrwx 1 ifadey ifadey   13 Dec 24 01:27 s-lnk-powertop.webm -> powertop.webm
5243934 -rw-rw-r-- 2 ifadey ifadey  64M Dec 19 22:15 linux-powertop.webm

Note the visual representation of symlink shown by ls command. It shows the link name and an arrow which points to original file.

You can create hard/soft link in any directory using absolute or relative path. Here are some more examples of ln command creating both hard/soft links.

ln -s note.txt ~/Desktop/note.txt
ln -s note.txt ~/Desktop/note.txt

Text File Viewer – less Command