ilugc-meet-10-07-2021.chatlog

2021-07-10 15:05:22	mohan43u-we	hi all
2021-07-10 15:05:39	mohan43u-we	welcome to ILUGC july month meet for the year 2021
2021-07-10 15:06:06	mohan43u-we	Im mohan
2021-07-10 15:06:24	mohan43u-we	the first talk is about rootless containers
2021-07-10 15:06:41	mohan43u-we	lets first talk about containers
2021-07-10 15:07:07	mohan43u-we	to learn containers, we need to first learn about an old command called chroot
2021-07-10 15:07:52	mohan43u-we	if you read 'man chroot' you will learn how it changes the root directory of a process
2021-07-10 15:08:14	mohan43u-we	every process have a root file system view (the word view is important)
2021-07-10 15:08:56	mohan43u-we	normally in unix, every process see root as same
2021-07-10 15:09:41	mohan43u-we	I hope everyone joined to this meet can see my terminal either through https://meet.ilugc.in or trough jitsi
2021-07-10 15:09:58	mohan43u-we	if not please use either any one to see what Im showing
2021-07-10 15:10:35	mohan43u-we	if you see the presenter terminal, you can see te root directory of my system
2021-07-10 15:10:58	mohan43u-we	this root directory normally will not change for different process
2021-07-10 15:11:41	mohan43u-we	as you see, using proc filesytem, we can see each process exact root filesystem
2021-07-10 15:12:20	mohan43u-we	as you see in the presenter terminal, I showed two process's root file system, both pointing to /
2021-07-10 15:15:55	mohan43u-we	if you see the presenter terminal now, you can see that I executed one chroot
2021-07-10 15:16:08	mohan43u-we	that process id is 177265
2021-07-10 15:16:29	mohan43u-we	as you see, the root of the first command (ls) is /
2021-07-10 15:16:55	mohan43u-we	but if you see the root of process id 177265, you can see that it is changed to /home/presenter/Prod/ubuntu
2021-07-10 15:17:06	mohan43u-we	this is what chroot does
2021-07-10 15:17:22	mohan43u-we	it changes the root file system view of a process
2021-07-10 15:17:50	mohan43u-we	this is one of the oldest way of isolating a process
2021-07-10 15:18:50	mohan43u-we	so any process which runs inside chroot see a different view then the host process
2021-07-10 15:19:12	mohan43u-we	as you see, wen I executed /etc/hostname inside chroot, it showed localhost.localdomain
2021-07-10 15:19:34	mohan43u-we	but when I executed it in host, I see hetzner0-host0
2021-07-10 15:19:43	mohan43u-we	this is called process isonation
2021-07-10 15:19:58	mohan43u-we	and chroot is one of the oldest method
2021-07-10 15:20:22	mohan43u-we	in 2000's, linux kernel modernized process isolation through namespaces
2021-07-10 15:20:37	mohan43u-we	this concept is inspired by plan9 os from bell labs
2021-07-10 15:20:55	mohan43u-we	and this process isolation is also main point in fuscia
2021-07-10 15:21:04	mohan43u-we	coming to namespace
2021-07-10 15:21:24	mohan43u-we	so what the oldest chroot does is called filesystem namespace
2021-07-10 15:21:35	mohan43u-we	if you guys have any doubt. please ask
2021-07-10 15:21:40	mohan43u-we	Ill will try to reply
2021-07-10 15:22:08	mohan43u-we	now going to next
2021-07-10 15:22:46	mohan43u-we	as I said, modern linux not only capable of isonating filesystem, it is capable of isolating process view (pid namespace)
2021-07-10 15:23:07	mohan43u-we	capable of isonating network view (network namespace)
2021-07-10 15:23:23	mohan43u-we	isolating ipc view (ipc namespace, sharedmemory)
2021-07-10 15:23:43	mohan43u-we	you can get all the details through 'man 7 namespaces'
2021-07-10 15:24:12	mohan43u-we	as you see in presenter terminal, there are 7 namespaces in modern linux
2021-07-10 15:24:28	mohan43u-we	cgroup namespace is to isolate cgroups
2021-07-10 15:24:36	mohan43u-we	ipc as I already explained
2021-07-10 15:24:42	mohan43u-we	network namespace
2021-07-10 15:25:00	mohan43u-we	mount namespace (or filesystem namespace, which is what chroot does in the old days)
2021-07-10 15:25:08	mohan43u-we	pid namespace
2021-07-10 15:25:13	mohan43u-we	ok
2021-07-10 15:25:54	mohan43u-we	using combination of these namespaces, we can created fully isolated view to a process from the original host process view
2021-07-10 15:26:16	mohan43u-we	when a process have a isolated view, then that process is called containered process
2021-07-10 15:26:20	mohan43u-we	or simply container
2021-07-10 15:26:45	mohan43u-we	to these combination of isolations, we need some tools to do the job
2021-07-10 15:27:26	mohan43u-we	here comes the tools like systemd-nspawn, runc, crun, firejail, bubblewrap etc.,
2021-07-10 15:27:55	mohan43u-we	there is also one more tool called unshare
2021-07-10 15:28:26	mohan43u-we	this is just like all the tools I said above, but comes build-in in every linux distro
2021-07-10 15:28:41	mohan43u-we	ok.
2021-07-10 15:29:08	mohan43u-we	now. these tools primary job is to isolate and create containerized processes
2021-07-10 15:29:26	mohan43u-we	but to organize more, we needed much better managing tools
2021-07-10 15:29:39	mohan43u-we	here comes, docker, podman
2021-07-10 15:29:51	mohan43u-we	Im not going to talk about docker here.
2021-07-10 15:29:59	mohan43u-we	so we will focus podman
2021-07-10 15:30:25	mohan43u-we	lets install podman
2021-07-10 15:30:55	mohan43u-we	as you see in the presenter terminal, Im installing both podman and crun
2021-07-10 15:31:21	mohan43u-we	podman is a container manager, and crun is the actual containerizer (or engine in podman terms)
2021-07-10 15:31:45	mohan43u-we	by default, podman uses another containerizer called runc, which is writter in golang
2021-07-10 15:32:07	mohan43u-we	runc is having some problem with rootless namespace, thats why I use crun
2021-07-10 15:32:21	mohan43u-we	so, Im installing both podman and crun
2021-07-10 15:32:42	mohan43u-we	done
2021-07-10 15:32:47	mohan43u-we	I installed podman and crun
2021-07-10 15:33:05	mohan43u-we	now, lets start a container
2021-07-10 15:33:52	mohan43u-we	as you can see, I just executed podman and asked to give me a plain alpine container
2021-07-10 15:34:48	mohan43u-we	-t option to podman's run command means, not to go background and keep the stdout,stderr,stdin intact (dont close)
2021-07-10 15:35:11	mohan43u-we	sorry, the -i option means the above explanation
2021-07-10 15:35:23	mohan43u-we	-t means open a new tty for the container
2021-07-10 15:35:43	mohan43u-we	--rm option means, when the container stop running, delete that container
2021-07-10 15:36:09	mohan43u-we	--rmi option means, when there is no container for an image, delete that image
2021-07-10 15:36:29	mohan43u-we	--name option means, im giving a neme to that container
2021-07-10 15:36:58	mohan43u-we	finally 'alpine' means, fetch this container image from docker's container registry
2021-07-10 15:37:33	mohan43u-we	if you see te first line in the output, you cansee that podman recognizes 'alpine' as an alias to docker.io/library/alpine:latest
2021-07-10 15:37:48	mohan43u-we	this is te actual url to alpine container image in docker
2021-07-10 15:37:55	mohan43u-we	so it pulls that image from there
2021-07-10 15:38:46	mohan43u-we	as you see in the presenter terminal, I listed all te containers
2021-07-10 15:38:57	mohan43u-we	you can see that there is a container called container0 running
2021-07-10 15:39:15	mohan43u-we	as you can see, there is a alpine image available
2021-07-10 15:39:38	mohan43u-we	If I exit now from container, both container0 and the corresponding image will be gone
2021-07-10 15:40:06	mohan43u-we	so for a container,  we firs tneed to fetch a image form registry
2021-07-10 15:42:07	mohan43u-we	as you see, we pulled one alpine image from docker
2021-07-10 15:42:22	mohan43u-we	with this image, we can now create one container
2021-07-10 15:43:32	mohan43u-we	done. when I exit, te containers will not be deleted
2021-07-10 15:43:50	mohan43u-we	see, the container is still present
2021-07-10 15:44:01	mohan43u-we	the image also still present
2021-07-10 15:44:12	mohan43u-we	we can create any number of containers from a single image
2021-07-10 15:44:43	mohan43u-we	as you see, I created another container, now I have two containers
2021-07-10 15:44:51	mohan43u-we	both are from single alpine image
2021-07-10 15:45:16	mohan43u-we	lets delete the containers and images
2021-07-10 15:45:44	mohan43u-we	since I didnt run any containers, I was able to delete easily
2021-07-10 15:45:59	mohan43u-we	if Im running a container, I need to first stop that container
2021-07-10 15:46:05	mohan43u-we	ten only I can delete
2021-07-10 15:46:26	mohan43u-we	since there is no containers for alpine image, I can now go and delete the images
2021-07-10 15:46:58	mohan43u-we	as you see, I dont have any image or container
2021-07-10 15:47:24	mohan43u-we	if you notice, in all the podman commands I executed, I never used sudo
2021-07-10 15:47:33	mohan43u-we	I never used root capability
2021-07-10 15:48:09	mohan43u-we	so, the containers which I created are all rootless containers
2021-07-10 15:48:25	mohan43u-we	let see where podman store the container image when it runs in rootless mode
2021-07-10 15:49:29	mohan43u-we	/home/mohan/.local/share/containers/storage this is where podman stores all the images when it runs in rootless mode
2021-07-10 15:49:50	mohan43u-we	if I delete this location, podman will not show anything
2021-07-10 15:50:09	mohan43u-we	see, gone
2021-07-10 15:50:14	mohan43u-we	no image
2021-07-10 15:50:43	mohan43u-we	ok, now lets see how we can compose a custom image
2021-07-10 15:51:09	mohan43u-we	for that, we need a Containerfile
2021-07-10 15:51:29	mohan43u-we	Containerfile in podman is equal to Dockerfile in docker
2021-07-10 15:51:48	mohan43u-we	podman works with oci images and it is capable of generating oci images
2021-07-10 15:52:01	mohan43u-we	nowadays, every docker image is a oci image
2021-07-10 15:52:09	mohan43u-we	and every Dockerfile is a Containerfile
2021-07-10 15:52:49	mohan43u-we	this is my custom docker file
2021-07-10 15:53:11	mohan43u-we	the first FROM line tells podman to pull alpine image from a registry
2021-07-10 15:53:26	mohan43u-we	usually podman pulls from docker.io or quay.io
2021-07-10 15:53:43	mohan43u-we	there are way to configure podman to fetch from a particular registry
2021-07-10 15:54:13	mohan43u-we	the third line RUN tells podman to execute that small script after pulling the image
2021-07-10 15:54:31	mohan43u-we	my script basically install python3 and starts one http server
2021-07-10 15:55:34	mohan43u-we	the line with EXPOSE 80 means, port 80 inside container, should be accessable from outside container
2021-07-10 15:55:46	mohan43u-we	because on port 80, we will be running a http server
2021-07-10 15:56:05	mohan43u-we	the last line ENTRYPOINT means, the first command to run after starting the container
2021-07-10 15:56:29	mohan43u-we	Im running /usr/bin/python3 -m http.server --directory /var/www/helloworld 80
2021-07-10 15:56:51	mohan43u-we	this command basically runs a static website which is inside /var/www/helloworld
2021-07-10 15:57:05	mohan43u-we	if you look at my script, I just put one file called index.html saying hellowlrd
2021-07-10 15:57:15	mohan43u-we	^s/hellowlrd/helloworld/g
2021-07-10 15:57:27	mohan43u-we	ok, lets create my custom image
2021-07-10 15:57:59	mohan43u-we	-t option means a new name for that image
2021-07-10 15:58:10	mohan43u-we	-f option means the name of the docker file
2021-07-10 15:58:24	mohan43u-we	as you see, te image is already composed
2021-07-10 15:58:36	mohan43u-we	now, lets see the images
2021-07-10 15:59:01	mohan43u-we	as you see, podman pulls generic alpine image from registry and saves it locally
2021-07-10 15:59:13	mohan43u-we	ten it created my custom image called image0
2021-07-10 15:59:35	mohan43u-we	lets create one container from image0 and run
2021-07-10 16:00:03	mohan43u-we	see, my python http server started running
2021-07-10 16:00:54	mohan43u-we	people who already familier with containers will have a question like port 80 is inside the container, I made a mistake not routing outside port to inside port
2021-07-10 16:01:12	mohan43u-we	yes, I deliberately didn't expose the container yet to the outside world
2021-07-10 16:01:36	mohan43u-we	well I have a bit of explanation to do about rootless container's networking
2021-07-10 16:02:13	mohan43u-we	before networking, I need to tell that rootless containers uses fuse-overlayfs to do mounting overlay filesystem without root capability
2021-07-10 16:02:55	mohan43u-we	as you see pid 178954 is running as mohan user
2021-07-10 16:02:58	mohan43u-we	not root
2021-07-10 16:03:11	mohan43u-we	fuse-overlayfs does the overlay
2021-07-10 16:03:16	mohan43u-we	now come to network
2021-07-10 16:03:33	mohan43u-we	rootless containers uses slirp4ns for usermode networking
2021-07-10 16:03:53	mohan43u-we	if you are familier with qemu, you might have heard usermode networking
2021-07-10 16:04:08	mohan43u-we	the default network mode in qemu
2021-07-10 16:04:18	mohan43u-we	podman uses the same mechanism
2021-07-10 16:04:31	mohan43u-we	it uses slirp4ns for usermode networking
2021-07-10 16:05:14	mohan43u-we	now, lets get inside the current running container
2021-07-10 16:08:11	mohan43u-we	ok. now Im inside the container
2021-07-10 16:08:15	mohan43u-we	the running container
2021-07-10 16:08:22	mohan43u-we	with all the isolation
2021-07-10 16:08:50	mohan43u-we	as you see, inside the container, there is only a tap0 interface
2021-07-10 16:09:38	mohan43u-we	slirp4ns creates this interface inside container, whenever there is a packet comes into this interface, it is redirected to outside world just like a normal process in te ost
2021-07-10 16:09:42	mohan43u-we	the host
2021-07-10 16:10:55	mohan43u-we	see, inside the container, there is only one processes listening on port 80
2021-07-10 16:11:22	mohan43u-we	inside container, the ip address of that container is 10.0.2.100
2021-07-10 16:11:49	mohan43u-we	as you see, my custom http server is capable of serving helloworld
2021-07-10 16:12:09	mohan43u-we	now, I checked that everything is working
2021-07-10 16:12:18	mohan43u-we	now, lets expose this container to outside workd
2021-07-10 16:12:21	mohan43u-we	workd
2021-07-10 16:12:24	mohan43u-we	world
2021-07-10 16:12:34	mohan43u-we	for that. I need to stop and restart
2021-07-10 16:13:19	mohan43u-we	done. container0 now stopped
2021-07-10 16:13:27	mohan43u-we	lets start with port redirection
2021-07-10 16:14:09	mohan43u-we	I asked podman to route all packets from port 8001 in the host to port 80 in the container
2021-07-10 16:14:27	mohan43u-we	now you can try http://meet.ilugc.in:8001 to get my greetings
2021-07-10 16:15:25	mohan43u-we	probably, you guys are using chrome
2021-07-10 16:15:48	mohan43u-we	which remembers that http://meet.ilugc.in means https://meet.ilugc.in
2021-07-10 16:15:52	mohan43u-we	so it uses https
2021-07-10 16:16:02	mohan43u-we	and wrongly connects to port 443
2021-07-10 16:16:12	mohan43u-we	thats why I see lot of bad requests
2021-07-10 16:16:27	mohan43u-we	try opening chrome or firefox in incognito mode
2021-07-10 16:16:35	mohan43u-we	and type http://meet.ilugc.in:8001
2021-07-10 16:16:54	mohan43u-we	now you will get my greetings
2021-07-10 16:17:43	mohan43u-we	so, this is how people run rootless containers for web services
2021-07-10 16:18:01	mohan43u-we	rootless containers provides better protection than containers running as root
2021-07-10 16:18:23	mohan43u-we	also, development is much easy in rootless containers
2021-07-10 16:18:32	mohan43u-we	no need for root permission to run development containers
2021-07-10 16:19:52	mohan43u-we	as you noticed, this command 'containers-rootlessport' is the one which listens on port 8001 in the host
2021-07-10 16:20:20	mohan43u-we	then it redirects that packets to port 80 inside contianer0 through slirpns
2021-07-10 16:20:42	mohan43u-we	thats all for today
2021-07-10 16:21:07	mohan43u-we	I hope this talk gives insight to how containers works and all the layers involved
2021-07-10 16:21:30	mohan43u-we	there is one more upper layer to manage all the container managers like podman
2021-07-10 16:21:47	mohan43u-we	that manager is called kubernates
2021-07-10 16:22:12	mohan43u-we	it sits in one place and manages these podman container managers.
2021-07-10 16:22:47	mohan43u-we	in kubernates terms, this machine (where I'm running podman) is called node
2021-07-10 16:23:33	mohan43u-we	not sure I should go inside kubernates. but this is how kubernates creates containers
2021-07-10 16:24:13	mohan43u-we	so kubernates->podman(or docker)->crun(or runc)->namespaces->kernel
2021-07-10 16:24:27	mohan43u-we	thats all from my talk
2021-07-10 16:24:45	mohan43u-we	thank you.
2021-07-10 16:24:57	mohan43u-we	if you have any questions let discuss
2021-07-10 16:26:16	mohan43u-we	the talk is over today. lets join https://meet.jit.si/ILUGCMeet10072021 for audio chat for general discussion