Created
October 3, 2017 11:26
-
-
Save janjaapbos/b67f97f2f32d7cf09c066fa5eaf50e89 to your computer and use it in GitHub Desktop.
docker compose for ZeroTier 6plane
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| version: '2.1' | |
| # run with IPv6 network of the docker container as enviroment variable | |
| # e.g. ZT6PLANE=fc7b:59ab:4811:901c:40ea docker-compose up | |
| networks: | |
| zerotier: | |
| driver: bridge | |
| enable_ipv6: true | |
| internal: false | |
| ipam: | |
| config: | |
| - subnet: ${ZT6PLANE}::/80 | |
| volumes: | |
| zerotier_var: | |
| services: | |
| zerotier: | |
| image: zerotier/zerotier-containerized | |
| devices: | |
| - /dev/net/tun | |
| network_mode: host | |
| cap_add: | |
| - NET_ADMIN | |
| - SYS_ADMIN | |
| volumes: | |
| - zerotier_var:/var/lib/zerotier-one/ | |
| # this only exists so that the networks get created | |
| alpine: | |
| image: bwstitt/alpine | |
| command: tail -f /dev/null | |
| # uncomment this once the zerotier container is running | |
| networks: | |
| zerotier: | |
| ipv6_address: ${ZT6PLANE}::2 |
Yes, the tcpdump was done inside my haproxy container from this command: docker run --rm -it --net container:frontend_haproxy_zt_1 nicolaka/netshoot
I am able to ping between some containers on the same host (::b37e:f2a9 -> ::2):
# docker run --rm -it --net container:ethereum_parity_1 nicolaka/netshoot ping6 -c2 fcf0:a9af:17a3:c742:eb37::2
PING fcf0:a9af:17a3:c742:eb37::2(fcf0:a9af:17a3:c742:eb37::2) 56 data bytes
64 bytes from fcf0:a9af:17a3:c742:eb37::2: icmp_seq=1 ttl=64 time=0.192 ms
64 bytes from fcf0:a9af:17a3:c742:eb37::2: icmp_seq=2 ttl=64 time=0.085 ms
--- fcf0:a9af:17a3:c742:eb37::2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.085/0.138/0.192/0.054 ms
# docker run --rm -it --net container:ethereum_parity_1 nicolaka/netshoot traceroute6 fcf0:a9af:17a3:c742:eb37::2
traceroute to fcf0:a9af:17a3:c742:eb37::2 (fcf0:a9af:17a3:c742:eb37::2), 30 hops max, 72 byte packets
1 shared_alpine_zt_1.shared_zerotier (fcf0:a9af:17a3:c742:eb37::2) 0.011 ms 0.005 ms 0.002 ms
It fails for this other host though (::b37e:f2a9 -> ::c4d:421f):
# docker run --rm -it --net container:ethereum_parity_1 nicolaka/netshoot traceroute6 fcf0:a9af:17a3:c742:eb37::0c4d:421f
traceroute to fcf0:a9af:17a3:c742:eb37::0c4d:421f (fcf0:a9af:17a3:c742:eb37::c4d:421f), 30 hops max, 72 byte packets
1 ethereum_parity_1.shared_zerotier (fcf0:a9af:17a3:c742:eb37::b37e:f2a9) 3075.682 ms !H 3071.421 ms !H 3071.836 ms !H
# docker run --rm -it --net container:ethereum_parity_1 nicolaka/netshoot ping6 fcf0:a9af:17a3:c742:eb37::0c4d:421f
PING fcf0:a9af:17a3:c742:eb37::0c4d:421f(fcf0:a9af:17a3:c742:eb37::c4d:421f) 56 data bytes
From fcf0:a9af:17a3:c742:eb37::b37e:f2a9 icmp_seq=1 Destination unreachable: Address unreachable
From fcf0:a9af:17a3:c742:eb37::b37e:f2a9 icmp_seq=5 Destination unreachable: Address unreachable
From fcf0:a9af:17a3:c742:eb37::b37e:f2a9 icmp_seq=6 Destination unreachable: Address unreachable
^C
--- fcf0:a9af:17a3:c742:eb37::0c4d:421f ping statistics ---
8 packets transmitted, 0 received, +3 errors, 100% packet loss, time 7148ms
Host OS is Fedora 26 with the latest ce version of docker (installed via docker-machine). I've disabled the firewall to simplify the testing.
Regarding NDP on the host, perhaps this helps:
sysctl -w net.ipv6.conf.all.proxy_ndp=1
Looks promising!
[bwstitt@tank:~] $ sysctl net.ipv6.conf.all.proxy_ndp
net.ipv6.conf.all.proxy_ndp = 0
[admin@aws:~] $ sudo sysctl net.ipv6.conf.all.proxy_ndp
net.ipv6.conf.all.proxy_ndp = 0
But changing it to 1 doesn't seem to have made any difference. Pings still fail with the same errors
Latest tcpdump output
[bwstitt:~] $ docker run -it --net host nicolaka/netshoot sysctl net.ipv6.conf.all.proxy_ndp
net.ipv6.conf.all.proxy_ndp = 1
[bwstitt:~] $ docker run -it --net host nicolaka/netshoot traceroute6 fcf0:a9af:17a3:c742:eb37::4
traceroute to fcf0:a9af:17a3:c742:eb37::4 (fcf0:a9af:17a3:c742:eb37::4), 30 hops max, 72 byte packets
1 fcf0:a9af:17a3:c742:eb37::1 (fcf0:a9af:17a3:c742:eb37::1) 299.604 ms 312.555 ms 338.743 ms
2 * * *
3 * * *
4 * * *
5 * * *^C
[root@tank] # docker run -it --net host nicolaka/netshoot sysctl nev6.conf.all.proxy_ndp
net.ipv6.conf.all.proxy_ndp = 1
[root@tank] # docker run -it --net host nicolaka/netshoot tcpdump -i zt0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on zt0, link-type EN10MB (Ethernet), capture size 262144 bytes
02:05:36.083679 IP6 fcf0:a9af:17ea:c412:57de::1.47406 > fcf0:a9af:17a3:c742:eb37::4.33435: UDP, length 24
02:05:36.083873 IP6 fcf0:a9af:17a3:c742:eb37::1 > fcf0:a9af:17ea:c412:57de::1: ICMP6, time exceeded in-transit for fcf0:a9af:17a3:c742:eb37::4, length 80
02:05:36.327194 IP6 fcf0:a9af:17ea:c412:57de::1.47406 > fcf0:a9af:17a3:c742:eb37::4.33436: UDP, length 24
02:05:36.327332 IP6 fcf0:a9af:17a3:c742:eb37::1 > fcf0:a9af:17ea:c412:57de::1: ICMP6, time exceeded in-transit for fcf0:a9af:17a3:c742:eb37::4, length 80
02:05:39.686508 IP 10.242.176.103.54421 > 10.242.255.255.21027: UDP, length 69
02:05:41.450627 IP6 fe80::4ca2:c1ff:fe21:b299 > fcf0:a9af:17ea:c412:57de::1: ICMP6, neighbor solicitation, who has fcf0:a9af:17ea:c412:57de::1, length 32
02:05:41.562750 IP6 fcf0:a9af:17ea:c412:57de::1.47406 > fcf0:a9af:17a3:c742:eb37::4.33438: UDP, length 24
02:05:41.756616 IP6 fcf0:a9af:17ea:c412:57de::1 > fe80::4ca2:c1ff:fe21:b299: ICMP6, neighbor advertisement, tgt is fcf0:a9af:17ea:c412:57de::1, length 24
02:05:46.577343 IP6 fcf0:a9af:17ea:c412:57de::1.47406 > fcf0:a9af:17a3:c742:eb37::4.33439: UDP, length 24
02:05:46.634252 IP6 fe80::4ceb:c2ff:fe71:e70 > fcf0:a9af:17a3:c742:eb37::4: ICMP6, neighbor solicitation, who has fcf0:a9af:17a3:c742:eb37::4, length 32
02:05:46.634393 IP6 fe80::4ca2:c1ff:fe21:b299 > fe80::4ceb:c2ff:fe71:e70: ICMP6, destination unreachable, beyond scope fcf0:a9af:17a3:c742:eb37::4, source address fe80::4ceb:c2ff:fe71:e70, length 80
02:05:46.790118 IP6 fe80::4ceb:c2ff:fe71:e70 > fe80::4ca2:c1ff:fe21:b299: ICMP6, neighbor solicitation, who has fe80::4ca2:c1ff:fe21:b299, length 32
02:05:46.790142 IP6 fe80::4ca2:c1ff:fe21:b299 > fe80::4ceb:c2ff:fe71:e70: ICMP6, neighbor advertisement, tgt is fe80::4ca2:c1ff:fe21:b299, length 24
02:05:47.710453 IP6 fe80::4ceb:c2ff:fe71:e70 > fcf0:a9af:17a3:c742:eb37::4: ICMP6, neighbor solicitation, who has fcf0:a9af:17a3:c742:eb37::4, length 32
02:05:47.710635 IP6 fe80::4ca2:c1ff:fe21:b299 > fe80::4ceb:c2ff:fe71:e70: ICMP6, destination unreachable, beyond scope fcf0:a9af:17a3:c742:eb37::4, source address fe80::4ceb:c2ff:fe71:e70, length 80
02:05:48.733475 IP6 fe80::4ceb:c2ff:fe71:e70 > fcf0:a9af:17a3:c742:eb37::4: ICMP6, neighbor solicitation, who has fcf0:a9af:17a3:c742:eb37::4, length 32
02:05:48.733608 IP6 fe80::4ca2:c1ff:fe21:b299 > fe80::4ceb:c2ff:fe71:e70: ICMP6, destination unreachable, beyond scope fcf0:a9af:17a3:c742:eb37::4, source address fe80::4ceb:c2ff:fe71:e70, length 80
02:05:51.690628 IP6 fe80::4ca2:c1ff:fe21:b299 > fe80::4ceb:c2ff:fe71:e70: ICMP6, neighbor solicitation, who has fe80::4ceb:c2ff:fe71:e70, length 32
02:05:51.909489 IP6 fe80::4ceb:c2ff:fe71:e70 > fe80::4ca2:c1ff:fe21:b299: ICMP6, neighbor advertisement, tgt is fe80::4ceb:c2ff:fe71:e70, length 24
And here is a successful traceroute for a different container on the same host:
[bwstitt@laptop] $ docker run -it --net host nicolaka/netshoot traceroute6 fcf0:a9af:17a3:c742:eb37::2
traceroute to fcf0:a9af:17a3:c742:eb37::2 (fcf0:a9af:17a3:c742:eb37::2), 30 hops max, 72 byte packets
1 fcf0:a9af:17a3:c742:eb37::1 (fcf0:a9af:17a3:c742:eb37::1) 245.199 ms * 324.855 ms
2 fcf0:a9af:17a3:c742:eb37::2 (fcf0:a9af:17a3:c742:eb37::2) 305.673 ms 309.446 ms 309.176 ms
[root@tank] # docker run -it --net host nicolaka/netshoot tcpdump -i zt0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on zt0, link-type EN10MB (Ethernet), capture size 262144 bytes
02:13:54.875535 IP6 fcf0:a9af:17ea:c412:57de::1.49226 > fcf0:a9af:17a3:c742:eb37::2.33435: UDP, length 24
02:13:54.875678 IP6 fcf0:a9af:17a3:c742:eb37::1 > fcf0:a9af:17ea:c412:57de::1: ICMP6, time exceeded in-transit for fcf0:a9af:17a3:c742:eb37::2, length 80
02:14:00.145734 IP6 fcf0:a9af:17ea:c412:57de::1.49226 > fcf0:a9af:17a3:c742:eb37::2.33437: UDP, length 24
02:14:00.145908 IP6 fcf0:a9af:17a3:c742:eb37::1 > fcf0:a9af:17ea:c412:57de::1: ICMP6, time exceeded in-transit for fcf0:a9af:17a3:c742:eb37::2, length 80
02:14:00.459565 IP6 fcf0:a9af:17ea:c412:57de::1.49226 > fcf0:a9af:17a3:c742:eb37::2.33438: UDP, length 24
02:14:00.459606 IP6 fcf0:a9af:17a3:c742:eb37::2 > fcf0:a9af:17ea:c412:57de::1: ICMP6, destination unreachable, unreachable port, fcf0:a9af:17a3:c742:eb37::2 udp port 33438, length 80
02:14:00.763468 IP6 fcf0:a9af:17ea:c412:57de::1.49226 > fcf0:a9af:17a3:c742:eb37::2.33439: UDP, length 24
02:14:00.763519 IP6 fcf0:a9af:17a3:c742:eb37::2 > fcf0:a9af:17ea:c412:57de::1: ICMP6, destination unreachable, unreachable port, fcf0:a9af:17a3:c742:eb37::2 udp port 33439, length 80
02:14:01.081223 IP6 fcf0:a9af:17ea:c412:57de::1.49226 > fcf0:a9af:17a3:c742:eb37::2.33440: UDP, length 24
02:14:01.081259 IP6 fcf0:a9af:17a3:c742:eb37::2 > fcf0:a9af:17ea:c412:57de::1: ICMP6, destination unreachable, unreachable port, fcf0:a9af:17a3:c742:eb37::2 udp port 33440, length 80
Looks like zerotier/zerotier-containerized is gone :(
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Ok, so if on the server it already does not work to ping its container, you can focus on getting that to work before testing across hosts. So the tcpdump is done on the container where the ping is directed? You see NDP request but no responses.
Can you ping between containers on the same host?
What is the host OS / distro?
Is there a host firewall active?