meeting-minutes-sh-posix-sell-session-day-5-july-2020

2020-07-10 17:00:20	+mohan43u-we	good evening all
2020-07-10 17:00:37	+mohan43u-we	welcome to Day 5 of 'sh' Posix shell Session
2020-07-10 17:01:17	+mohan43u-we	I hope all of you who joined #ilugc through webchat followed https://pasteboard.co/JgGXsrf.gif link to disable join/part messages
2020-07-10 17:02:10	+mohan43u-we	yesterday we saw variables, if-elif, case, for, while control structures
2020-07-10 17:02:42	+mohan43u-we	also we saw how variable get expanded by shell before executing command, different types of variable expansion
2020-07-10 17:02:55	+mohan43u-we	we also saw arithmetic expansion, command substitution
2020-07-10 17:03:23	+mohan43u-we	we also looked into special built-in variables shell provides like $$, $!, $? etc.,
2020-07-10 17:04:10	+mohan43u-we	today, we will look into functions, positional parameters
2020-07-10 17:04:15	+mohan43u-we	let see function
2020-07-10 17:04:46	+mohan43u-we	a function in shell is nothing but another built-in command which you write, instead of shell providing by default
2020-07-10 17:05:13	+mohan43u-we	we all know 'cd' is a built-in command
2020-07-10 17:05:42	+mohan43u-we	this built-in command is provided by 'sh' shell itself
2020-07-10 17:05:52	+mohan43u-we	like 'cd', we can write our own function
2020-07-10 17:07:07	+mohan43u-we	see, in practice terminal, I created a function called replicate_cd()
2020-07-10 17:07:32	+mohan43u-we	which does nothing but replicates actual cd command
2020-07-10 17:07:47	+mohan43u-we	I'm currently in /home/trainer directory
2020-07-10 17:08:19	+mohan43u-we	see, instead of using actual 'cd' command, I used my custom function 'replicate_cd' to switch to '/' directory
2020-07-10 17:08:55	+mohan43u-we	the definition of 'replicate_cd' is nothing but 'cd "${@}"'
2020-07-10 17:09:14	+mohan43u-we	I'll explain what "${@}" means in short time
2020-07-10 17:09:41	+mohan43u-we	but as far as functions goes, they are nothing but custom built-in commands we define for the current shell
2020-07-10 17:10:08	+mohan43u-we	they act just like built-in commands, in functions, we define what the command need to do instead of shell already defined the functionality
2020-07-10 17:10:23	+mohan43u-we	thats all about function
2020-07-10 17:10:29	+mohan43u-we	now, moving to the next topic
2020-07-10 17:10:34	+mohan43u-we	parameters
2020-07-10 17:11:24	+mohan43u-we	here when we called replicate_cd, we provided argument as '/'
2020-07-10 17:11:31	+mohan43u-we	this is called parameter to the function
2020-07-10 17:12:25	+mohan43u-we	it is also called as positional parameters
2020-07-10 17:12:42	+mohan43u-we	lets define replicate_function again
2020-07-10 17:13:52	+mohan43u-we	see, I redefined replicate_cd() again, this time instead of changing directory, I used echo to print positional parameters
2020-07-10 17:14:58	+mohan43u-we	in the body of replicate_cd, '${0}' maps to whatever the name of the function (or executable command name)
2020-07-10 17:15:11	+mohan43u-we	'${1}" is the first parameter
2020-07-10 17:15:24	+mohan43u-we	"${2}" is the second parameter
2020-07-10 17:15:36	+mohan43u-we	this can go on
2020-07-10 17:15:58	+mohan43u-we	so in the body, I just echo these positional parameters one by one
2020-07-10 17:16:52	+mohan43u-we	thats why the first line of the output shows 'sh' the shell name (the executable which our function currently resides)
2020-07-10 17:17:15	+mohan43u-we	the second line is the first argument '/' I passed to replicate_cd
2020-07-10 17:17:30	+mohan43u-we	there is no second argument, so the empty third line
2020-07-10 17:18:23	+mohan43u-we	see, I redefined 'replicate_cd()' again, but in the body i used '${@}'
2020-07-10 17:18:50	+mohan43u-we	"${@}" means, all the positional parameters except "${0}"
2020-07-10 17:19:18	+mohan43u-we	when I run 'replicate_cd' with 'one two three four' it just showed all parameters
2020-07-10 17:19:48	+mohan43u-we	thats the meaning of this special '${@}' positional parameter
2020-07-10 17:20:21	+mohan43u-we	you can also use '${*}' to get the same like '${@}', but there is difference in expanding "${@}" and "${*}"
2020-07-10 17:21:29	+mohan43u-we	see using "${*}" is also worked same as "${@}"
2020-07-10 17:23:24	+mohan43u-we	forget it. I thought of explaining the actual difference between these two, but not coming up with proper example
2020-07-10 17:23:45	+mohan43u-we	but there is difference. dont assume that '${@}' and '${*}' are same
2020-07-10 17:23:55	+mohan43u-we	there is one more special parameter
2020-07-10 17:24:20	+mohan43u-we	this '${#}'
2020-07-10 17:24:42	+mohan43u-we	it provides how many positional parameters are present, instead of giving the positional parameters
2020-07-10 17:25:19	+mohan43u-we	this is very useful to check whether user is providing required input or not in shell scripts
2020-07-10 17:25:47	+mohan43u-we	thats all about positional parameters
2020-07-10 17:25:58	+mohan43u-we	now,moving to the next
2020-07-10 17:26:05	+mohan43u-we	shell patterns
2020-07-10 17:26:47	+mohan43u-we	remember the for loop we used yesterday?
2020-07-10 17:28:17	+mohan43u-we	we used $(ls /) command substitution to give dynamic values for iteration, instead we can just use /*
2020-07-10 17:29:36	+mohan43u-we	both expanded dynamically to provide values for each iteration
2020-07-10 17:29:55	+mohan43u-we	here the '/*' means expand to all the entries under '/' directory
2020-07-10 17:30:15	+mohan43u-we	we can also use '*' to match anywhere in the path
2020-07-10 17:31:12	+mohan43u-we	see the last for command expanded to the files inside /home/trainer/ directory
2020-07-10 17:31:31	+mohan43u-we	'*' is called shell glob pattern matching
2020-07-10 17:32:57	+mohan43u-we	when I used '*' for 'ls' command the 'sh' shell expanded '*' to the files and directories under /home/trainer and executed 'ls' command with the expanded values
2020-07-10 17:33:20	+mohan43u-we	I can also limit the glob expansion with specific prefix or suffix
2020-07-10 17:34:02	+mohan43u-we	see instead of using plain '*' I used 'out*' which expands only to filenames or directories starting with 'out' not all
2020-07-10 17:34:40	+mohan43u-we	see, I used '*put*' which means, any file or directory name which contains 'put' word
2020-07-10 17:34:52	+mohan43u-we	see the last one uses suffix
2020-07-10 17:35:08	+mohan43u-we	'*.txt' means any file or directory name which ends with '.txt'
2020-07-10 17:35:21	+mohan43u-we	so this is how we expand the filenames using shell glob pattern
2020-07-10 17:35:58	+mohan43u-we	we can also use range of characters instead of only matching constant words
2020-07-10 17:36:44	+mohan43u-we	I created a file called testfile.txt
2020-07-10 17:36:55	+mohan43u-we	now I can match with range
2020-07-10 17:37:45	+mohan43u-we	see, I asked to shell to expand any file which starts with either 'o' or 't', so it expands to 'output.txt' and 'testfile.txt'
2020-07-10 17:38:23	+mohan43u-we	I also included 'P', so it expanded to 'Prod' directory
2020-07-10 17:38:32	+mohan43u-we	so this is called shell glob pattern matching
2020-07-10 17:38:47	+mohan43u-we	you can use this shell glob patterns in anywhere
2020-07-10 17:39:19	+mohan43u-we	se, I used 'ls -l /proc/*/cwd' command
2020-07-10 17:40:06	+mohan43u-we	which expanded to all the directories under /proc directory but it specifically listed 'cwd' file under each expanded directory
2020-07-10 17:40:28	+mohan43u-we	thats all about shell glob patterns
2020-07-10 17:40:39	+mohan43u-we	moving to next topic
2020-07-10 17:40:58	+mohan43u-we	'find' command
2020-07-10 17:41:16	+mohan43u-we	it is one important command to nagivate through the filesystem in unix
2020-07-10 17:41:47	+mohan43u-we	by default 'find' command recurse through each and every directory in a path and provide their full path
2020-07-10 17:41:49	+mohan43u-we	lets try
2020-07-10 17:42:50	+mohan43u-we	see, it started going through each and every directory under /home/trainer and started giving full path of those file and directories
2020-07-10 17:43:26	+mohan43u-we	now, just regulate find, find have -maxdepth option which restricts how many innder depths it can go
2020-07-10 17:44:10	+mohan43u-we	when I gave maxdepth 0, it didn't traverse, because I asked not to traverse
2020-07-10 17:44:16	+mohan43u-we	by setting maxdepth to 0
2020-07-10 17:44:44	+mohan43u-we	when I increase maxdepth to 1, it only lists all the files and directories immediately under '/'
2020-07-10 17:45:10	+mohan43u-we	see, I asked find to only go through 2 depth
2020-07-10 17:45:19	+mohan43u-we	this is one of the way to control find
2020-07-10 17:45:42	+mohan43u-we	another is to use shell glob pattern matching with -name
2020-07-10 17:47:19	+mohan43u-we	see, I asked find to only match fine which ends with '*.html', under /home/trainer/Prod/meetty directory, so it recurses through that directory and printed full path of filenames which ends with .html
2020-07-10 17:47:42	+mohan43u-we	find have very advanced filtering, it will take another day to explain about all the options it has
2020-07-10 17:48:04	+mohan43u-we	for simplicity, just read 'man find', it give you every option available
2020-07-10 17:48:38	+mohan43u-we	I'm going to tell you how you can use find to get filenams and do further processing
2020-07-10 17:48:52	+mohan43u-we	there are two methods to do post processing
2020-07-10 17:49:33	+mohan43u-we	using find's in-built '-exec' option, or piping out and using some other command to do post processing
2020-07-10 17:51:36	+mohan43u-we	see, I used 'find' to get all the 'html' files and used 'exec' to pass the filenames to 'grep' command, then asked 'grep' command to search for string '52.24.74.253' and print the matching lines
2020-07-10 17:52:08	+mohan43u-we	so grep showed that file Prod/meetty/src/meetty/templates/index.html contains line matching 52.24.74.253
2020-07-10 17:52:14	+mohan43u-we	lets do it in other way
2020-07-10 17:53:19	+mohan43u-we	see, I used 'read' command along with 'while' command to iterate through the filenames provided by find command
2020-07-10 17:54:12	+mohan43u-we	'read' command simply read one line from standard input and store it in the variable name provided as argument, so for each line from find output goes to 'file' variable, thanks to 'read'
2020-07-10 17:54:54	+mohan43u-we	we used while command to iterate till 'read' command returns false returncode
2020-07-10 17:55:21	+mohan43u-we	in the body of the while command we simply print the filename
2020-07-10 17:57:08	+mohan43u-we	see,in the body of the while,, we further do some grep apart from capturing the filename to get the lines form the html files which contains 52.24.74.253
2020-07-10 17:57:26	+mohan43u-we	so these are two way to do postprocessing from find ouput
2020-07-10 17:58:02	+mohan43u-we	this is how we use 'find' command
2020-07-10 17:58:55	+mohan43u-we	lets take a break for 5 mins before moving to the topic
2020-07-10 17:59:16	 *	mohan43u-we taking 5 mins break
2020-07-10 18:04:30	 *	mohan43u-we back from 5 min break
2020-07-10 18:05:24	+mohan43u-we	to all, apart from this 5 day session, we have regular monthly meet tomorrow at 3.00 pm IST
2020-07-10 18:05:42	+mohan43u-we	we will be sending invite mail today after conforming talks
2020-07-10 18:06:52	+mohan43u-we	we are conducting monthly meet every month 2nd saturday
2020-07-10 18:07:24	+mohan43u-we	this is happening for the last 20 years, I mean more than 20 years, from 1998
2020-07-10 18:08:30	+mohan43u-we	ILGUC is one of the oldest special interest groups in India who promote Free and Open Source Software around Chennai and Tamil Nadu
2020-07-10 18:09:22	+mohan43u-we	we welcome everyone to join ILUGC learn about Linux and FOSS and share your know to everyone
2020-07-10 18:09:37	+mohan43u-we	you can get more details bout ilugc in 'https://ilugc.in'
2020-07-10 18:10:16	+mohan43u-we	ok, enough promotion, lets get back to shell session
2020-07-10 18:10:46	+mohan43u-we	the next topic is regular expressions
2020-07-10 18:11:00	+mohan43u-we	remember the grep command?
2020-07-10 18:11:34	+mohan43u-we	when I introduced 'grep' I told that it not only takes static strings as input, but also you can give 'Basic Regular Expressions'
2020-07-10 18:11:56	+mohan43u-we	let do some grep in the output.txt
2020-07-10 18:12:21	+mohan43u-we	I just typed 'hi', which is a static string I passed to grep
2020-07-10 18:12:39	+mohan43u-we	it searched for line containing 'hi' returned the matching line
2020-07-10 18:12:52	+mohan43u-we	lets try to match this line in a different way
2020-07-10 18:13:41	+mohan43u-we	this time, I used regular expression '^h' with grep,
2020-07-10 18:14:41	+mohan43u-we	'^' in '^h' represents start of the line, so, '^h' means, match any line starting with 'h'
2020-07-10 18:15:02	+mohan43u-we	so grep matched not only 'hi', it also matched 'how'
2020-07-10 18:15:13	+mohan43u-we	thats why we got more than one line
2020-07-10 18:16:05	+mohan43u-we	'$'in 'life$' represents end of line, so 'life$' means match any line ending with 'life'
2020-07-10 18:16:16	+mohan43u-we	thus we got line ends with life
2020-07-10 18:16:35	+mohan43u-we	'^' and '$' are anchors in regular expression
2020-07-10 18:17:39	+mohan43u-we	see, this time I didn't give any anchors, but simply said 'you', so grep matched 'you' as well as 'yours'
2020-07-10 18:18:32	+mohan43u-we	this time I used another set of anchors '\<' and '\>', which means,beginning of word and end of word
2020-07-10 18:18:43	+mohan43u-we	so this time grep exactly matched 'you'
2020-07-10 18:19:15	+mohan43u-we	these are just drops in the vast ocean of regular expressions
2020-07-10 18:20:00	+mohan43u-we	for proper beginning, read 'Regular Expression' topic in this link https://www.freebsd.org/cgi/man.cgi?query=ed&apropos=0&sektion=0&manpath=FreeBSD+12.1-RELEASE+and+Ports&arch=default&format=html
2020-07-10 18:20:27	+mohan43u-we	'ed' is the first original unix command which introduced 'Regular Expressions'
2020-07-10 18:20:44	+mohan43u-we	'grep' is basically one of the functionality of ed
2020-07-10 18:21:14	+mohan43u-we	I can only provide this much about regular expressions this time
2020-07-10 18:21:22	+mohan43u-we	and 'grep' command.
2020-07-10 18:22:01	+mohan43u-we	learning regular expressions will help you vast in programming (not just shell scripts)
2020-07-10 18:22:54	+mohan43u-we	next topic, 'sed' , it is basically manuplating standard input to get desired standard output
2020-07-10 18:24:36	+mohan43u-we	see, we asked find command to find files with '.html' output, but we piped find output to 'sed' to cut off 'html' from the filenames
2020-07-10 18:24:48	+mohan43u-we	thus we end up with this kind of output
2020-07-10 18:25:03	+mohan43u-we	'sed' is also one of the functionality of 'ed' command
2020-07-10 18:25:25	+mohan43u-we	you can read more about 'grep' and 'sed' from their manual pages 'man grep' and 'man sed'
2020-07-10 18:25:54	+mohan43u-we	due to time limit, I'm skipping about 'awk', it is also like 'sed' but more advanced
2020-07-10 18:25:59	+mohan43u-we	read about 'awk' from 'man awk'
2020-07-10 18:26:03	+mohan43u-we	now. the final topic
2020-07-10 18:27:16	+mohan43u-we	a basic structure of shell script
2020-07-10 18:27:26	+mohan43u-we	the first line is very important
2020-07-10 18:27:52	+mohan43u-we	#! means 'magic number' which represents that this file is a shell script
2020-07-10 18:28:10	+mohan43u-we	we also need to mention which 'shell' this script this compatable with
2020-07-10 18:28:25	+mohan43u-we	thats why we start with '#!/bin/sh' which is Posix 'sh' shell
2020-07-10 18:28:41	+mohan43u-we	the rest of the body is basically how we defined function and executed that function
2020-07-10 18:29:06	+mohan43u-we	one important thing is, you need to make this file as executable before running, otherwise unix will throw error
2020-07-10 18:29:27	+mohan43u-we	see, os throwed that 'test.sh' is not excutable
2020-07-10 18:30:05	+mohan43u-we	see, 'chmod +x' will change the file permission of 'test.sh' to make it as executable
2020-07-10 18:30:14	+mohan43u-we	so 'test.sh' become executable
2020-07-10 18:30:41	+mohan43u-we	see after that, when we run './test.sh' it run and gave us ouput
2020-07-10 18:31:02	+mohan43u-we	thats all about a simple shell script structure
2020-07-10 18:31:16	+mohan43u-we	shell scripts are nothing but pre defined commands and functions
2020-07-10 18:31:45	+mohan43u-we	whatever we execute in commandline, we just put it as a file and start executing that file
2020-07-10 18:32:04	+mohan43u-we	it will run whatever the commands we listed in that file line by line
2020-07-10 18:32:20	+mohan43u-we	thus, the 5 day course coming to end
2020-07-10 18:32:32	--	Mode #ilugc [-m] by mohan43u
2020-07-10 18:33:07	+mohan43u-we	I thank each and everyone who had patent to listen to my ramblinglings for these past 5 days
2020-07-10 18:33:22	+mohan43u-we	*patent*patient*whatever*
2020-07-10 18:33:48	+mohan43u-we	thanks for attending this 5 day session
2020-07-10 18:34:20	Jayaram_T	thankyou sir It was wonderful session
2020-07-10 18:34:23	+mohan43u-we	#ilugc channel is relaxed already, if anyone have doubts, querys, suggestions, please feel free to communicate in #ilugc channel
2020-07-10 18:34:58	Guest15811	Thanks mohan43u-we
2020-07-10 18:34:59	MITHULKIRUTHIK_K	Thank you sir
2020-07-10 18:35:06	MATHAN_181CS196	thank you sir
2020-07-10 18:35:31	+mohan43u-we	also do attend tomorrows regular #ilugc monthly meet at 03.00 PM India time here at #ilugc irc channel
2020-07-10 18:36:12	+mohan43u-we	it may not be through training.ilugc.in, it can be through jitsi, it will be decided by the speaker who take tomorrow's session
2020-07-10 18:36:13	Sankar	thank you sir
2020-07-10 18:37:17	+mohan43u-we	thanks you all. see you in next #ilugc session or meet, until then 'peace'
2020-07-10 18:37:50	humachine	mohan:  Thanks for taking time to conduct this session :-D awesome
2020-07-10 18:37:51	+mohan43u-we	shrini: over to shrini
2020-07-10 18:38:00	infoFarmer	mohan43u-we:if we practice sh+regex well, we will be one of the terminal magician. Thank you so much indeed for this 5 days.. most impotant like our right hand 5 FINGERsshrini:thx 4 your outreach.Closing here to expect your screencast about it.Thanks indeed to all whom are behind the scene. bye4now
2020-07-10 18:40:28	Subathra	Thank you ilugc team for this wonderful & infomative sessions.
2020-07-10 19:06:03	+shrini	we got a very good feedback on irc training from ranjith kumar on ilugc list
2020-07-10 19:06:06	+shrini	thanks ranjith
2020-07-10 19:06:17	+shrini	yes. IRC is slow compared to video meets
2020-07-10 19:06:33	+shrini	but this slowness gives good opportunity to read and understand clearly
2020-07-10 19:19:32	+mbuf	shrini, IRC is not slow. People typing speeds are slow.
2020-07-10 19:20:08	+mbuf	shrini, text-based communication can never be slow even on a dial-up modem.
2020-07-10 19:33:56	linuxbaskar	yes.  Have a look at #ubuntu.  At any time, there are 1K+ members.  Agreed with mbuf, even in late 90s when using dial-up modem, we did not face any slow problem.  All we faced is only frequent disconnection problem due to noice
2020-07-10 19:51:17	+shrini	mbuf: yes. I mean the slowness of the incoming data by human typing
2020-07-10 19:58:49	linuxbaskar	:b  it's due to sickness caused by MS$