Created
May 6, 2021 05:45
-
-
Save bikram20/493f685c4fa1f6e453f1db7053acbe4f to your computer and use it in GitHub Desktop.
DOKS worker nodes accessibility from Internet
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Use "brew install nmap" or any other way to get nmap. | |
| # Have doctl and kubectl configured. | |
| # | |
| bgupta@C02CC1EGMD6M employeeapp % doctl compute droplet list --tag-name 'k8s' --format 'Name' | |
| Name | |
| pool-4fz85fgrm-8rqqw | |
| pool-4fz85fgrm-8yde3 | |
| bgupta@C02CC1EGMD6M employeeapp % doctl compute droplet list --tag-name 'k8s' --format 'Name','PublicIPv4' | |
| Name Public IPv4 | |
| pool-4fz85fgrm-8rqqw 161.35.114.243 | |
| pool-4fz85fgrm-8yde3 134.209.219.51 | |
| bgupta@C02CC1EGMD6M employeeapp % | |
| bgupta@C02CC1EGMD6M employeeapp % nmap -F 161.35.114.243 | |
| Starting Nmap 7.91 ( https://nmap.org ) at 2021-05-05 20:01 PDT | |
| Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn | |
| Nmap done: 1 IP address (0 hosts up) scanned in 3.08 seconds | |
| bgupta@C02CC1EGMD6M employeeapp % nmap -Pn 161.35.114.243 | |
| Host discovery disabled (-Pn). All addresses will be marked 'up' and scan times will be slower. | |
| Starting Nmap 7.91 ( https://nmap.org ) at 2021-05-05 20:02 PDT | |
| Nmap scan report for 161.35.114.243 | |
| Host is up. | |
| All 1000 scanned ports on 161.35.114.243 are filtered | |
| Nmap done: 1 IP address (1 host up) scanned in 402.30 seconds | |
| bgupta@C02CC1EGMD6M employeeapp % | |
| bgupta@C02CC1EGMD6M employeeapp % nmap -A 161.35.114.243 | |
| Starting Nmap 7.91 ( https://nmap.org ) at 2021-05-05 20:14 PDT | |
| Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn | |
| Nmap done: 1 IP address (0 hosts up) scanned in 3.41 seconds | |
| bgupta@C02CC1EGMD6M employeeapp % nmap -sV 161.35.114.243 | |
| Starting Nmap 7.91 ( https://nmap.org ) at 2021-05-05 20:14 PDT | |
| Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn | |
| Nmap done: 1 IP address (0 hosts up) scanned in 3.30 seconds | |
| bgupta@C02CC1EGMD6M employeeapp % nmap -p- 161.35.114.243 | |
| Starting Nmap 7.91 ( https://nmap.org ) at 2021-05-05 20:14 PDT | |
| Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn | |
| Nmap done: 1 IP address (0 hosts up) scanned in 3.10 seconds | |
| bgupta@C02CC1EGMD6M employeeapp % | |
| # So all the ports are blocked from outside. The IP addresses are not not exposed to Internet. | |
| # We can check the firewalls that apply to the worker nodes | |
| # Now let us connect to a worker node and inspect the firewall on the kernel | |
| bgupta@C02CC1EGMD6M employeeapp % git clone https://github.com/digitalocean/doks-debug.git | |
| Cloning into 'doks-debug'... | |
| remote: Enumerating objects: 91, done. | |
| remote: Total 91 (delta 0), reused 0 (delta 0), pack-reused 91 | |
| Receiving objects: 100% (91/91), 16.66 KiB | 550.00 KiB/s, done. | |
| Resolving deltas: 100% (45/45), done. | |
| bgupta@C02CC1EGMD6M employeeapp % cd doks-debug | |
| bgupta@C02CC1EGMD6M doks-debug % ls | |
| Dockerfile LICENSE README.md k8s script | |
| bgupta@C02CC1EGMD6M doks-debug % kubectl apply -f k8s/daemonset.yaml | |
| daemonset.apps/doks-debug created | |
| bgupta@C02CC1EGMD6M doks-debug % kgpoall | |
| NAMESPACE NAME READY STATUS RESTARTS AGE | |
| default kube-bench-2wn9v 0/1 Completed 0 16d | |
| kube-system cilium-7nzwm 1/1 Running 0 34m | |
| kube-system cilium-fwqdt 1/1 Running 0 22d | |
| kube-system cilium-operator-84bdd6f7b6-4hvm2 1/1 Running 2 22d | |
| kube-system cilium-operator-84bdd6f7b6-nlgth 1/1 Running 2 22d | |
| kube-system coredns-55ff57f948-2lpbt 1/1 Running 0 22d | |
| kube-system coredns-55ff57f948-2x44z 1/1 Running 0 22d | |
| kube-system csi-do-node-b6gdh 2/2 Running 0 34m | |
| kube-system csi-do-node-q6vrt 2/2 Running 0 22d | |
| kube-system do-node-agent-q9jbg 1/1 Running 0 22d | |
| kube-system do-node-agent-zwwdk 1/1 Running 0 34m | |
| kube-system doks-debug-gxwkm 1/1 Running 0 25s | |
| kube-system doks-debug-tlrlc 1/1 Running 0 25s | |
| kube-system kube-proxy-4kw5d 1/1 Running 0 34m | |
| kube-system kube-proxy-nt6p2 1/1 Running 0 22d | |
| bgupta@C02CC1EGMD6M doks-debug % | |
| bgupta@C02CC1EGMD6M doks-debug % kubectl -n kube-system exec -it doks-debug-gxwkm bash | |
| kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. | |
| root@pool-4fz85fgrm-8yde3:~# | |
| root@pool-4fz85fgrm-8yde3:~# ifconfig eth0 | |
| eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 | |
| inet 134.209.219.51 netmask 255.255.240.0 broadcast 134.209.223.255 | |
| inet6 fe80::dccc:aaff:fe8f:adf0 prefixlen 64 scopeid 0x20<link> | |
| ether de:cc:aa:8f:ad:f0 txqueuelen 1000 (Ethernet) | |
| RX packets 10974086 bytes 2489372603 (2.4 GB) | |
| RX errors 0 dropped 0 overruns 0 frame 0 | |
| TX packets 10535154 bytes 1350196311 (1.3 GB) | |
| TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 | |
| root@pool-4fz85fgrm-8yde3:~# | |
| # From another terminal, ping to the worker node from your laptop | |
| bgupta@C02CC1EGMD6M src % curl icanhazip.com | |
| 73.70.228.24 | |
| bgupta@C02CC1EGMD6M src % ping 134.209.219.51 | |
| PING 134.209.219.51 (134.209.219.51): 56 data bytes | |
| Request timeout for icmp_seq 0 | |
| Request timeout for icmp_seq 1 | |
| Request timeout for icmp_seq 2 | |
| # Come back to the worker node and do a tcpdump for your source IP | |
| root@pool-4fz85fgrm-8yde3:~# tcpdump host 73.70.228.24 | |
| tcpdump: verbose output suppressed, use -v or -vv for full protocol decode | |
| listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes | |
| ^C | |
| 0 packets captured | |
| 0 packets received by filter | |
| 0 packets dropped by kernel | |
| root@pool-4fz85fgrm-8yde3:~# | |
| # Now let us verify if you can ping one worker node from another. Connect to another worker node and do the ping, while doing a tcpdump. | |
| bgupta@C02CC1EGMD6M src % kubectl -n kube-system exec -it doks-debug-tlrlc bash | |
| kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. | |
| root@pool-4fz85fgrm-8rqqw:~# | |
| root@pool-4fz85fgrm-8rqqw:~# ifconfig eth0 | |
| eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 | |
| inet 161.35.114.243 netmask 255.255.240.0 broadcast 161.35.127.255 | |
| inet6 fe80::bc4d:4dff:fe9c:94df prefixlen 64 scopeid 0x20<link> | |
| ether be:4d:4d:9c:94:df txqueuelen 1000 (Ethernet) | |
| RX packets 14322 bytes 64587231 (64.5 MB) | |
| RX errors 0 dropped 0 overruns 0 frame 0 | |
| TX packets 8494 bytes 855588 (855.5 KB) | |
| TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 | |
| root@pool-4fz85fgrm-8rqqw:~# ifconfig eth1 | |
| eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 | |
| inet 10.116.0.4 netmask 255.255.240.0 broadcast 10.116.15.255 | |
| inet6 fe80::98ce:85ff:fe72:93f3 prefixlen 64 scopeid 0x20<link> | |
| ether 9a:ce:85:72:93:f3 txqueuelen 1000 (Ethernet) | |
| RX packets 31264 bytes 10668107 (10.6 MB) | |
| RX errors 0 dropped 0 overruns 0 frame 0 | |
| TX packets 32720 bytes 4007955 (4.0 MB) | |
| TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 | |
| root@pool-4fz85fgrm-8rqqw:~# | |
| root@pool-4fz85fgrm-8rqqw:~# route -n | |
| Kernel IP routing table | |
| Destination Gateway Genmask Flags Metric Ref Use Iface | |
| 0.0.0.0 161.35.112.1 0.0.0.0 UG 0 0 0 eth0 | |
| 10.10.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 | |
| 10.116.0.0 0.0.0.0 255.255.240.0 U 0 0 0 eth1 | |
| 10.244.0.0 10.116.0.3 255.255.255.128 UG 0 0 0 eth1 | |
| 10.244.1.0 10.244.1.19 255.255.255.128 UG 0 0 0 cilium_host | |
| 10.244.1.19 0.0.0.0 255.255.255.255 UH 0 0 0 cilium_host | |
| 161.35.112.0 0.0.0.0 255.255.240.0 U 0 0 0 eth0 | |
| 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 | |
| root@pool-4fz85fgrm-8rqqw:~# | |
| root@pool-4fz85fgrm-8rqqw:~# ping 134.209.219.51 | |
| PING 134.209.219.51 (134.209.219.51) 56(84) bytes of data. | |
| ^C | |
| --- 134.209.219.51 ping statistics --- | |
| 3 packets transmitted, 0 received, 100% packet loss, time 2033ms | |
| root@pool-4fz85fgrm-8rqqw:~# ping 10.116.0.3 | |
| PING 10.116.0.3 (10.116.0.3) 56(84) bytes of data. | |
| 64 bytes from 10.116.0.3: icmp_seq=1 ttl=64 time=1.10 ms | |
| ^C | |
| --- 10.116.0.3 ping statistics --- | |
| 1 packets transmitted, 1 received, 0% packet loss, time 0ms | |
| rtt min/avg/max/mdev = 1.102/1.102/1.102/0.000 ms | |
| root@pool-4fz85fgrm-8rqqw:~# ping 134.209.219.51 --help | |
| ping: invalid option -- '-' | |
| Usage: ping [-aAbBdDfhLnOqrRUvV64] [-c count] [-i interval] [-I interface] | |
| [-m mark] [-M pmtudisc_option] [-l preload] [-p pattern] [-Q tos] | |
| [-s packetsize] [-S sndbuf] [-t ttl] [-T timestamp_option] | |
| [-w deadline] [-W timeout] [hop1 ...] destination | |
| Usage: ping -6 [-aAbBdDfhLnOqrRUvV] [-c count] [-i interval] [-I interface] | |
| [-l preload] [-m mark] [-M pmtudisc_option] | |
| [-N nodeinfo_option] [-p pattern] [-Q tclass] [-s packetsize] | |
| [-S sndbuf] [-t ttl] [-T timestamp_option] [-w deadline] | |
| [-W timeout] destination | |
| root@pool-4fz85fgrm-8rqqw:~# ping 134.209.219.51 -I eth1 | |
| PING 134.209.219.51 (134.209.219.51) from 10.116.0.4 eth1: 56(84) bytes of data. | |
| ^C | |
| --- 134.209.219.51 ping statistics --- | |
| 3 packets transmitted, 0 received, 100% packet loss, time 2054ms | |
| root@pool-4fz85fgrm-8rqqw:~# ping 10.116.0.3 -I eth1 | |
| PING 10.116.0.3 (10.116.0.3) from 10.116.0.4 eth1: 56(84) bytes of data. | |
| 64 bytes from 10.116.0.3: icmp_seq=1 ttl=64 time=12.0 ms | |
| ^C | |
| --- 10.116.0.3 ping statistics --- | |
| 1 packets transmitted, 1 received, 0% packet loss, time 0ms | |
| rtt min/avg/max/mdev = 12.009/12.009/12.009/0.000 ms | |
| root@pool-4fz85fgrm-8rqqw:~# ping 10.116.0.3 -I eth0 | |
| PING 10.116.0.3 (10.116.0.3) from 161.35.114.243 eth0: 56(84) bytes of data. | |
| ^C | |
| --- 10.116.0.3 ping statistics --- | |
| 1 packets transmitted, 0 received, 100% packet loss, time 0ms | |
| root@pool-4fz85fgrm-8rqqw:~# ping 10.116.0.3 -I eth1 | |
| PING 10.116.0.3 (10.116.0.3) from 10.116.0.4 eth1: 56(84) bytes of data. | |
| 64 bytes from 10.116.0.3: icmp_seq=1 ttl=64 time=3.45 ms | |
| 64 bytes from 10.116.0.3: icmp_seq=2 ttl=64 time=1.36 ms | |
| 64 bytes from 10.116.0.3: icmp_seq=3 ttl=64 time=0.941 ms | |
| 64 bytes from 10.116.0.3: icmp_seq=4 ttl=64 time=1.07 ms | |
| # Notice that all pings pass ONLY to the internal IP of the worker node. Everything directed to external IP is blocked by the DO firewall. | |
| # You can run tcpdump on the worker node to verify that the traffic to the public IP is not even received by the node. | |
| root@pool-4fz85fgrm-8yde3:~# tcpdump -i eth1 host 10.116.0.4 icmp | |
| tcpdump: syntax error in filter expression: syntax error | |
| root@pool-4fz85fgrm-8yde3:~# tcpdump -i eth1 icmp | |
| tcpdump: verbose output suppressed, use -v or -vv for full protocol decode | |
| listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes | |
| 05:42:34.124708 IP 10.116.0.4 > 10.116.0.3: ICMP echo request, id 9367, seq 106, length 64 | |
| 05:42:34.124788 IP 10.116.0.3 > 10.116.0.4: ICMP echo reply, id 9367, seq 106, length 64 | |
| 05:42:35.126041 IP 10.116.0.4 > 10.116.0.3: ICMP echo request, id 9367, seq 107, length 64 | |
| 05:42:35.126107 IP 10.116.0.3 > 10.116.0.4: ICMP echo reply, id 9367, seq 107, length 64 | |
| 05:42:36.127037 IP 10.116.0.4 > 10.116.0.3: ICMP echo request, id 9367, seq 108, length 64 | |
| 05:42:36.127097 IP 10.116.0.3 > 10.116.0.4: ICMP echo reply, id 9367, seq 108, length 64 | |
| ^C | |
| 6 packets captured | |
| 8 packets received by filter | |
| 0 packets dropped by kernel | |
| root@pool-4fz85fgrm-8yde3:~# | |
| # You have the option to harden the worker nodes further through DO firewall. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment