meeting-minutes-sh-posix-sell-session-day-3-july-2020

2020-07-08 16:58:48	+mohan43u-we	good evening to all
2020-07-08 16:58:58	+mohan43u-we	welcome to day 3 of 'sh' Posix Shell session
2020-07-08 16:59:27	+mohan43u-we	before we begin, if you are using 'webchat.freenode.net' you can hide join/part messages
2020-07-08 16:59:41	+mohan43u-we	the instruction is provided in the 'training.ilugc.in' page
2020-07-08 16:59:58	+mohan43u-we	also we have a new practice server. IP address got changed
2020-07-08 17:00:17	+mohan43u-we	so make sure you login to the correct server to practice
2020-07-08 17:00:29	+mohan43u-we	without further delay, lets start
2020-07-08 17:01:18	+mohan43u-we	yesterday we learnt  about man command, manual pages and howto execute more than one command with ';', '&&' and '||'combination
2020-07-08 17:01:28	+mohan43u-we	today, we will start with redirection
2020-07-08 17:02:02	+mohan43u-we	as I explained earlier, a command have "--options", "[args]' arguments, as well as it give returncode when it exits
2020-07-08 17:02:37	+mohan43u-we	apart from that. a command have input and output and error-output
2020-07-08 17:03:30	+mohan43u-we	these three are called default file descriptors for a command
2020-07-08 17:04:08	+mohan43u-we	as you see in presenter terminal, I executed a command 'ls -l /proc/self/fd'
2020-07-08 17:04:23	+mohan43u-we	the 'ls' is the command
2020-07-08 17:04:41	+mohan43u-we	I asked to show me the directory listing of /proc/self/fd directory
2020-07-08 17:04:49	+mohan43u-we	with detailed listing '-l'
2020-07-08 17:04:52	+mohan43u-we	so it showed
2020-07-08 17:05:15	+mohan43u-we	what special about this is, /proc is a special directory in unix filesystem
2020-07-08 17:05:51	+mohan43u-we	hi
2020-07-08 17:06:28	+mohan43u-we	this /proc directory is the way we see details about every process in the unix system
2020-07-08 17:07:06	+mohan43u-we	the directory /proc/self basically points to the directory which kernel maintains about a particular process
2020-07-08 17:07:43	+mohan43u-we	in our case 'ls' is that particular process
2020-07-08 17:08:19	+mohan43u-we	this may be too deep, but just assume /proc/self/fd directory show all the file descriptors opened by the command
2020-07-08 17:09:03	+mohan43u-we	I just now switched to 'sh' the posix shell :)
2020-07-08 17:09:12	+mohan43u-we	previously I was operating in 'bash'
2020-07-08 17:09:25	+mohan43u-we	coming to the talk, so as you see, I just typed 'echo $$'
2020-07-08 17:09:53	+mohan43u-we	$$ is a special variable which shows the pid of the current shell
2020-07-08 17:10:05	+mohan43u-we	the current shell pid is 710059
2020-07-08 17:10:31	+mohan43u-we	pid means process id, every process in unix have process-id
2020-07-08 17:11:06	+mohan43u-we	now, I'm going to use this pid to show you the file descriptors this shell is currently having
2020-07-08 17:11:54	+mohan43u-we	see, the listing show that "0 -> /dev/pts/1", "1 -> /dev/pts/1" and "2 -> /dev/pts/1"
2020-07-08 17:12:12	+mohan43u-we	just ignore "10 -> /dev/tty" for simplicity
2020-07-08 17:12:44	+mohan43u-we	so it tell that the shell is using /dev/pts/1 file as its standard input (number 0)
2020-07-08 17:13:12	+mohan43u-we	it also tells that the same file is used as standard output as well as standard error output (number 1, and 2)
2020-07-08 17:13:55	+mohan43u-we	so it means /dev/pts/1 is the STDIN, STDOUT and STDERR for my current shell
2020-07-08 17:14:22	+mohan43u-we	see, my current terminal is actually /dev/pts/1
2020-07-08 17:14:41	+mohan43u-we	every terminal in unix is a file which resides in /dev/pts directory
2020-07-08 17:15:01	+mohan43u-we	earlier I said right 'everything in unix is a file', this is one example
2020-07-08 17:15:11	+mohan43u-we	/dev/pts/1 is a special file
2020-07-08 17:15:39	+mohan43u-we	its a character special file
2020-07-08 17:16:00	+mohan43u-we	it is represented by 'c' in 'crw--w----' permission string
2020-07-08 17:16:32	+mohan43u-we	so basically, the shell is using the special terminal file called /dev/pts/1 for its STDIN, STDOUT and STDERR
2020-07-08 17:16:52	+mohan43u-we	so, redirection means, changing these default file descriptors to something else
2020-07-08 17:17:46	+mohan43u-we	before that, I'm going to tell about 'cat' command
2020-07-08 17:18:04	+mohan43u-we	this is a very simple command, which reads input from STDIN and shows output to STDOUT
2020-07-08 17:18:31	+mohan43u-we	as we all aware now, every command in unix will get default STDIN, STDOUT and STDERR mapped to the current terminal
2020-07-08 17:18:53	+mohan43u-we	in that way, 'cat' will also get the current terminal as STDIN, STDOUT and STDERR
2020-07-08 17:19:38	+mohan43u-we	I just typed some lines
2020-07-08 17:19:48	+mohan43u-we	'cat' just repeated that line
2020-07-08 17:20:10	+mohan43u-we	because it is a dump command which does nothing but reading from STDIN and throwing out to STDOUT
2020-07-08 17:20:25	+mohan43u-we	so when I typed "hi" and press enter, it just write back the "hi"
2020-07-08 17:21:20	+mohan43u-we	the pid of cat command we currently executing is 712280
2020-07-08 17:21:28	+mohan43u-we	I just listed its file descriptors
2020-07-08 17:21:41	+mohan43u-we	all points to /dev/pts/1 the same terminal where we run 'sh' shell
2020-07-08 17:22:35	+mohan43u-we	see this time, I redirected STDOUT of the cat command to output.txt
2020-07-08 17:22:51	+mohan43u-we	now we see what fd the current cat command have
2020-07-08 17:23:40	+mohan43u-we	see, the STDOUT (file descriptor 1) is now mapped to /home/trainer/output.txt instead of the default mapping of /dev/pts/1
2020-07-08 17:23:58	+mohan43u-we	thats why the previous echo didn't happen this time
2020-07-08 17:24:47	+mohan43u-we	to end the input to 'cat' command, we need to press 'ctrl-d' which means EOF (or end of file
2020-07-08 17:24:58	+mohan43u-we	now we got the shell prompt back
2020-07-08 17:25:30	+mohan43u-we	just like how we redirected STDOUT, we are now going to redirect STDIN
2020-07-08 17:26:25	+mohan43u-we	see, this time, cat didn't waited for our input, becuase we did a STDIN redirect
2020-07-08 17:26:46	+mohan43u-we	so 'cat' command instead of waiting for our input it just straignt to read output.txt file
2020-07-08 17:27:16	+mohan43u-we	thus, we did STDIN and STDOUT redirection
2020-07-08 17:27:40	+mohan43u-we	there is STDERR, lets do the redirection of that also
2020-07-08 17:28:20	+mohan43u-we	we told 'cat' to read a file which is not available in the filesystem
2020-07-08 17:28:41	+mohan43u-we	because that file is not available, 'cat' command throwed error message like 'No such file"
2020-07-08 17:29:25	+mohan43u-we	cat command uses STDOUT for its output and STDERR to throw error messages
2020-07-08 17:29:40	+mohan43u-we	now lets redirect the stderr
2020-07-08 17:31:00	+mohan43u-we	cat can also read file given as first argument instead of using STDIN, lets use that method
2020-07-08 17:31:32	+mohan43u-we	ok, now cat is throwing error message instead of 'sh' shell throwing error message
2020-07-08 17:32:11	+mohan43u-we	see, that error message disappared, because of '2>/dev/null'
2020-07-08 17:32:21	+mohan43u-we	'2' represents the stderr file descriptor
2020-07-08 17:33:32	+mohan43u-we	instead of just doing '>' we can also use '1>' to do stdout redirection, '2>' to do stderr redirection
2020-07-08 17:33:48	+mohan43u-we	so this is how redirection works
2020-07-08 17:34:12	+mohan43u-we	ok, move on to the next topic
2020-07-08 17:34:15	+mohan43u-we	before going
2020-07-08 17:34:31	+mohan43u-we	I would like to say people who is using webchat.freenode.net
2020-07-08 17:34:51	+mohan43u-we	you can disable join/part messages in the web irc client by
2020-07-08 17:35:28	+mohan43u-we	clicking 'settings' icon on the top right corner (near to x), then uncheck "Show when People Join" option
2020-07-08 17:35:43	+mohan43u-we	https://pasteboard.co/JgGXsrf.gif
2020-07-08 17:35:46	+mohan43u-we	here is a gif
2020-07-08 17:36:29	+mohan43u-we	ok, now move to the next topic
2020-07-08 17:36:47	+mohan43u-we	pipes
2020-07-08 17:37:33	+mohan43u-we	in redirection, we used a file in the filesystem as endpoint, like '>/to/some/file'
2020-07-08 17:37:48	+mohan43u-we	but in pipes, instead of using a file, we use a process as its endpoint
2020-07-08 17:38:58	+mohan43u-we	as a example, I executed 'top' command in the 'sh' shell
2020-07-08 17:39:16	+mohan43u-we	I switched to another shell and typed 'ps axf'
2020-07-08 17:39:34	+mohan43u-we	'ps' command is used to list processes in unix system
2020-07-08 17:39:46	+mohan43u-we	'ps axf' will show every running process in unix system
2020-07-08 17:41:11	+mohan43u-we	ps command will list all process details in STDOUT, instead of doing 'ps axf >myprocesses.txt' to store the process details, I used 'grep' command to filter a like containing 'top' word
2020-07-08 17:41:37	+mohan43u-we	so this is one of the way you can search for a process id in unix
2020-07-08 17:41:52	+mohan43u-we	the first like is the details about the top process
2020-07-08 17:42:03	+mohan43u-we	*like*line*
2020-07-08 17:42:35	+mohan43u-we	the second line provides details about the grep command we used to search the top process id
2020-07-08 17:43:10	+mohan43u-we	ignore the second like for simplicity, so there are two matches from the 'ps axf' output
2020-07-08 17:43:24	+mohan43u-we	we need the actual top commands details, so we take the first like
2020-07-08 17:43:31	+mohan43u-we	*like*line*
2020-07-08 17:43:59	+mohan43u-we	the first field in the first line is the process id of top command which we are running in the 'sh' shell
2020-07-08 17:44:17	+mohan43u-we	so this is how pipe works
2020-07-08 17:44:38	+mohan43u-we	we redirect one command's standard output to another commands standard input
2020-07-08 17:45:11	+mohan43u-we	grep by default read everything from standard input and look for <regular-expression> provided as first argument to grep
2020-07-08 17:46:27	+mohan43u-we	we redirected standard input of the grep command to use output.txt file as STDIN, and asked grep to search for lines containing word 'hi'
2020-07-08 17:46:38	+mohan43u-we	so it throwed only one line
2020-07-08 17:47:31	+mohan43u-we	see, now I used 'how' word, it throwed three lines containing how word
2020-07-08 17:48:01	+mohan43u-we	with this functionality of grep and the functionality of 'ps', I'm combining both to find out the pid of the top command
2020-07-08 17:48:15	+mohan43u-we	simple but yet powerful
2020-07-08 17:48:25	+mohan43u-we	this is how unix works
2020-07-08 17:49:17	+mohan43u-we	unix dont have big big kitchen sink GUI applications to do work, but small small tools like 'ps', 'grep', 'cat' etc
2020-07-08 17:49:30	+mohan43u-we	these are like tools, we are goldsmiths
2020-07-08 17:49:48	+mohan43u-we	we use these tools to combine them to create golds called 'shell scripts'
2020-07-08 17:51:00	+mohan43u-we	one of the famous and important ideology of unix is 'do one thing and do it well'
2020-07-08 17:52:01	+mohan43u-we	so the unix commands are all tools and we combine them to achieve out goals
2020-07-08 17:53:29	+mohan43u-we	the command which I currently executed shows that there are 161 people watching the live sesion through 'Presenter Terminal'
2020-07-08 17:54:07	+mohan43u-we	now, move on to the next topic
2020-07-08 17:54:12	+mohan43u-we	background execution
2020-07-08 17:54:40	+mohan43u-we	till now we executed commands and waited for output
2020-07-08 17:55:43	+mohan43u-we	ok guys, I think today we have lot to cover, I'm going to take another half hour to complete today's session
2020-07-08 17:56:00	+mohan43u-we	take 5 min break and comback at 18.00 PM to continue
2020-07-08 17:56:04	+mohan43u-we	we have lot to cover
2020-07-08 17:56:16	 *	mohan43u-we taking 5 min break
2020-07-08 18:00:38	 *	mohan43u-we back from 5 min break
2020-07-08 18:01:07	+mohan43u-we	so, lets continue our session
2020-07-08 18:01:48	+mohan43u-we	for anyone joined new, and struggling to see the chat because of join/part messages, see this https://pasteboard.co/JgGXsrf.gif link to disable it
2020-07-08 18:02:33	+mohan43u-we	so lets continue
2020-07-08 18:03:10	+mohan43u-we	background execution means, we dont have to wait for a command to complete to get a shell prompt
2020-07-08 18:04:06	+mohan43u-we	see, I just executed one for look
2020-07-08 18:04:29	+mohan43u-we	that for look basically does nothing but slep for 100 seconds
2020-07-08 18:04:44	+mohan43u-we	I dont have patient to wait for 100 seconds to get back the command prompt
2020-07-08 18:05:01	+mohan43u-we	so I just used '&' symbol at the end of the command line
2020-07-08 18:05:20	+mohan43u-we	thus I get 'command prompt' immediately to type the next command
2020-07-08 18:05:26	+mohan43u-we	this is called background execution
2020-07-08 18:05:40	+mohan43u-we	to see whether the command still running, use bg command
2020-07-08 18:06:03	+mohan43u-we	see that background for look completed
2020-07-08 18:06:32	+mohan43u-we	now I increased the time
2020-07-08 18:07:51	+mohan43u-we	ok, 'sh' dont provide range expansion, for simplicity I'm switching myself to bash, which provides range expansion
2020-07-08 18:08:30	+mohan43u-we	also, the command to check the background commands is called 'jobs' not 'bg'
2020-07-08 18:08:50	+mohan43u-we	so I typed the for look and put it in background
2020-07-08 18:09:09	+mohan43u-we	now I'm checking whether it is running or not
2020-07-08 18:09:33	+mohan43u-we	the first line of 'job' command output shows that the for loop is still running
2020-07-08 18:10:17	+mohan43u-we	to bring back the background executed command to front, we use 'fg' command
2020-07-08 18:10:37	+mohan43u-we	see, '%1' here represents the job number
2020-07-08 18:11:04	+mohan43u-we	'fg %1' means, big %1 background job to front and wait till that command completes
2020-07-08 18:11:41	+mohan43u-we	I just canceled that for look in the mid way using 'ctrl+c' keystroke
2020-07-08 18:11:48	+mohan43u-we	and I got command prompt immediately
2020-07-08 18:12:10	+mohan43u-we	then I run 'jobs', it show there are no background jobs currently executing
2020-07-08 18:12:16	+mohan43u-we	that all about background execution
2020-07-08 18:12:54	+mohan43u-we	I'll tell about subshell execution after we  talk about variables
2020-07-08 18:13:01	+mohan43u-we	because that will make sense
2020-07-08 18:13:15	+mohan43u-we	now coming to the 'variables' topic
2020-07-08 18:13:29	+mohan43u-we	till now we saw commands, and how we can combine them
2020-07-08 18:13:46	+mohan43u-we	shell also provides way to define variable
2020-07-08 18:14:54	+mohan43u-we	to create a variable, assign something to it and start using
2020-07-08 18:15:12	+mohan43u-we	now I'm in 'sh' posix shell
2020-07-08 18:15:34	+mohan43u-we	I typed 'var1=val1' thats it, var1 variable got created
2020-07-08 18:15:48	+mohan43u-we	to get back the value of 'var', use echo
2020-07-08 18:16:09	+mohan43u-we	the $ symbol plays important part here
2020-07-08 18:16:58	+mohan43u-we	it means instead of showing the word 'var1' search for a variable called 'var1' in the current shell environment and give the value of that variable
2020-07-08 18:17:21	+mohan43u-we	thats how you assign and retrive
2020-07-08 18:17:47	+mohan43u-we	you can also use the same way to assign the value of var1 to var2
2020-07-08 18:17:53	+mohan43u-we	now both contains val1
2020-07-08 18:18:39	+mohan43u-we	these simple variables are called shell variables which you create and use
2020-07-08 18:18:51	+mohan43u-we	there are some predefined variables called ENVIRONMENT variables
2020-07-08 18:19:16	+mohan43u-we	these variables are inheried from parent process to process
2020-07-08 18:19:27	+mohan43u-we	to see those variables, we use env
2020-07-08 18:19:31	+mohan43u-we	env command
2020-07-08 18:19:52	+mohan43u-we	env command will provide all the predefined environment variables
2020-07-08 18:20:24	+mohan43u-we	I just used pipe method to see the 'env' command output
2020-07-08 18:20:46	+mohan43u-we	'less' command is a pager, its main funcitonality is to show standard input page by page
2020-07-08 18:21:14	+mohan43u-we	so I combined 'env' and 'less' achieve this current look in presenter terminal
2020-07-08 18:21:46	+mohan43u-we	in 'less', it will show one full page of output and it will wait for your response
2020-07-08 18:21:58	+mohan43u-we	to see the next full page, you have to press 'space' key
2020-07-08 18:22:27	+mohan43u-we	now I'm at the end of the output, I can go back using 'b' key
2020-07-08 18:22:41	+mohan43u-we	see, now I'm top of the output
2020-07-08 18:23:10	+mohan43u-we	coming back to Environment variables, you can see that there are lot of environment variables 'sh' inherited from its parent process
2020-07-08 18:23:18	+mohan43u-we	one such variable is PATH
2020-07-08 18:23:36	+mohan43u-we	another is HOME
2020-07-08 18:23:45	+mohan43u-we	another is USER
2020-07-08 18:24:12	+mohan43u-we	another is PWD which is now shown currently but you can look at the output
2020-07-08 18:24:40	+mohan43u-we	so the 'env' command provides all the environment variables
2020-07-08 18:25:02	+mohan43u-we	you may ask that
2020-07-08 18:25:21	+mohan43u-we	where is var1 and var2 which we defined here
2020-07-08 18:25:28	+mohan43u-we	in the output of env
2020-07-08 18:25:43	+mohan43u-we	there is big difference between local variable and exported environment variable
2020-07-08 18:26:11	+mohan43u-we	normal variables like 'var1' or 'var2' dont get exported to the child shells
2020-07-08 18:26:21	+mohan43u-we	only exported variabled go to child shells
2020-07-08 18:27:18	+mohan43u-we	see, I just exported var1 which is a normal variable to make it as environment variable
2020-07-08 18:27:40	+mohan43u-we	now  when I typed 'env' command, it now shows 'var1' which has value 'val1'
2020-07-08 18:27:53	+mohan43u-we	now this is the right time to say about subshell
2020-07-08 18:29:38	+mohan43u-we	there are currently two variables var1 and var2
2020-07-08 18:29:48	+mohan43u-we	I exported var1 but didn' texport var2
2020-07-08 18:30:06	+mohan43u-we	I start another 'sh' shell
2020-07-08 18:30:23	+mohan43u-we	now I type 'echo $var1' it will show you the value of var1
2020-07-08 18:30:38	+mohan43u-we	but when I type 'echo $var2' it will not show
2020-07-08 18:31:03	+mohan43u-we	see, this is what I mean by subshell
2020-07-08 18:31:30	+mohan43u-we	so, when you program shell scripts, you need to be very careful about normal variables and environment variables
2020-07-08 18:31:56	+mohan43u-we	only environment variables pass through subshells, or child commands, or child processes
2020-07-08 18:32:09	+mohan43u-we	normal variables are not accessable from subshells
2020-07-08 18:32:49	+mohan43u-we	that all from today. we will continue the rest in the next session
2020-07-08 18:33:06	--	Mode #ilugc [-m] by mohan43u
2020-07-08 18:33:19	+mohan43u-we	the channel is open  for questions
2020-07-08 18:33:30	+mohan43u-we	you can ask any doubts from todays session
2020-07-08 18:34:29	--	Mode #ilugc [-v mohan43u-we] by mohan43u
2020-07-08 18:34:31	SATHISHKUMARPVR	Thank you
2020-07-08 18:34:39	--	Mode #ilugc [-v shrini] by mohan43u
2020-07-08 18:35:20	Nero007	Was a subshell created when you typed sh
2020-07-08 18:35:26	@mohan43u	Nero007: yes
2020-07-08 18:36:09	Nero007	Had it the same priority as the previous shell or was it inferior and thats why the name subshell
2020-07-08 18:37:02	linuxbaskar	how many subshells  can be created, and how the user know whether the shell is subshell or not
2020-07-08 18:37:04	@mohan43u	Nero007: you can use the word subshell in various context, I used it to show the relashionship between parent process and child process
2020-07-08 18:37:36	@mohan43u	Nero007: and also to explain the importance of using 'export' to export environment variables
2020-07-08 18:37:49	hellboy212	mohan43u-we: can you plz tell once again the command to export a normal variable as an environment variable in the chat? i had missed it
2020-07-08 18:38:06	@mohan43u	hellboy212: 'export var=value'
2020-07-08 18:38:21	hellboy212	mohan43u: thank you
2020-07-08 18:38:31	@mohan43u	hellboy212: instead of just assigning value to variable, type 'export' before so that variable become environment variable
2020-07-08 18:38:55	hellboy212	ohh okay
2020-07-08 18:38:59	Nero007	mohan43u: Thanks
2020-07-08 18:41:48	hellboy212	can we delete an environment variable from shell?
2020-07-08 18:42:09	@mohan43u	linuxbaskar: 'ulimit' command can provide you various limits, one of the limit is maximum processes, you can use  'ulimit -p' to see the maximum number of child processes you can create
2020-07-08 18:43:59	linuxbaskar	I have invoked multiple subshell. how can i find the pid of the current subshell
2020-07-08 18:46:55	linuxbaskar	echo $$  displays the pid for the current subshell
2020-07-08 18:47:14	@mohan43u	hellboy212: unset <variable-name>
2020-07-08 18:48:39	@mohan43u	hellboy212: 'unset' command unsets variable (means deletes variable from current environment), if the variable is a environment variable, then it also removes the environment variable from the environment
2020-07-08 18:49:39	Nero007	mohan43u:It's a request to please update the logs
2020-07-08 18:50:31	@mohan43u	Nero007: will do it mate. dont worry.
2020-07-08 18:53:31	shrini	we got 92 users logged in the remote server
2020-07-08 19:05:07	humachine	mohan43u: The part on file descriptors was very good.  Like the way you really getting behind the scenes in explaining the concept.:)
2020-07-08 19:06:21	mohan43u	humachine: thanks. After nobody asked questions, I thought I screwed somewhat and nobody understand. you gave me hope :)
2020-07-08 19:07:05	humachine	mohan43u: :)