Last active
November 20, 2019 00:30
-
-
Save christf/2600c456ac5d72986e768b8dad9ccbab to your computer and use it in GitHub Desktop.
ipv4 packet loss problem
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Freifunk CLAT Issue | |
| * We have a client c connected to a Freifunk node n | |
| * C obtains an ipv4address 10.139.12.13 from n via dhcp | |
| * n has an ipv4 next-node-address of 10.139.0.1 | |
| * n is connected to the rest of the freifunk infrastructure via VPN with MTU m | |
| * The freifunk infrastructure is ipv6-only. | |
| * n runs a clat kernel module translating ipv4 to ipv6 such that c can connect | |
| anywhere in ipv6-land. | |
| # Problem #1 | |
| When C sends a packet that is larger than the MTU of the vpn tunnel, an ICMP | |
| message (packet too big) is created at n and sent to the client. | |
| However the client discards this packet because its source address is set to | |
| the client's ipv4 address. | |
| # Unsuccessful Attemts for Resolution | |
| * place a host-route on n for the client as this: | |
| ip r a 10.139.12.13/32 dev local-node src 10.139.0.1 t local | |
| ip r a 10.139.12.13/32 dev local-node src 10.139.0.1 | |
| * Looked at iptables rewrite src-IP but that will only work on POSTROUTING, not | |
| in the OUTPUT Chain | |
| # Solution part 1 | |
| Adding another clat configuration will lead to icmp packets with correct src-address however causes another issue (see below) | |
| echo insert clat local.v4 10.139.0.1/32 local.v6 :: local.style NONE local.ea-len 0 local.psid-offset 0 remote.v4 10.139.0.1/32 remote.v6 fda9:26e:5805:bab2:5054:ff:feac:7a2/128 remote.style NONE remote.ea-len 0 remote.psid-offset 0 > control | |
| # Problem #2 | |
| ## Situation | |
| tcpdump shows icmp packets to be created with the correct src-address on device clat: | |
| tcpdump: verbose output suppressed, use -v or -vv for full protocol decode | |
| listening on clat, link-type RAW (Raw IP), capture size 262144 bytes | |
| 01:03:08.039119 IP 10.139.22.188 > 8.8.8.8: ICMP echo request, id 5058, seq 1, length 1408 | |
| 01:03:08.039192 IP 10.139.0.1 > 10.139.22.188: ICMP type-#1, length 1220 | |
| However the client does not receive the packet. | |
| tcpdump -pni local-node icmp does not show the packet on the node | |
| tcpdump -pni any icmp | |
| tcpdump: verbose output suppressed, use -v or -vv for full protocol decode | |
| listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes | |
| 01:28:58.551763 IP 10.139.22.188 > 8.8.8.8: ICMP echo request, id 5074, seq 1, length 1408 | |
| 01:28:58.551777 IP 10.139.22.188 > 8.8.8.8: ICMP echo request, id 5074, seq 1, length 1408 | |
| 01:28:58.551790 IP 10.139.22.188 > 8.8.8.8: ICMP echo request, id 5074, seq 1, length 1408 | |
| 01:28:58.551826 IP 10.139.22.188 > 8.8.8.8: ICMP echo request, id 5074, seq 1, length 1408 | |
| 01:28:58.551894 IP 10.139.0.1 > 10.139.22.188: ICMP 8.8.8.8 unreachable - need to frag (mtu 1354), length 1220 | |
| root@3xxxx-525400ee0043:/# ip r s t clat | |
| default dev clat proto static scope link metric 2048 | |
| 10.139.0.0/17 dev local-node proto static scope link | |
| 10.139.22.188 dev local-node scope link metric 1024 | |
| root@3xxxx-525400ee0043:/# ^C | |
| root@3xxxx-525400ee0043:/# ip ru s | |
| 0: from all lookup local | |
| 9999: from 10.139.0.0/17 lookup clat | |
| 10000: from 10.139.0.1 lookup clat | |
| 20000: from all to 10.139.0.1/17 lookup clat | |
| 32766: from all lookup main | |
| 32767: from all lookup default | |
| 90009: from all iif lo lookup clat | |
| 90010: from all iif lo lookup clat | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Topology: | |
| +--------------------------+ +--------------------------+ +--------------------------+ +--------------------------+ | |
| | Client | | Freifunk AP | | VPN GW | | Exit | | |
| | 10.139.x.y | | 10.139.0.1 | | Jool, Stateful NAT64 | | 10.11.12.5/30 | | |
| | | | NAT64 | | 10.11.12.6/30 | | Debian Linux 4.19 | | |
| | | WLAN-Verbindung| | Wireguard VPN | Debian Linux 4.19 | GRE Tunnel | | | |
| | iperf3 -c 10.11.12.6 -R +----------------> +----------------+ +----------------+ iperf3 -s | | |
| | iperf3 -c 10.11.12.6 | MTU 1500 | | MTU 1374 | | MTU 1400 | | | |
| | | IPv4 and IPV6 | | IPv6-Only | | IPv4-Only | | | |
| | | | | | | | | | |
| | | | | | | | | | |
| +--------------------------+ +--------------------------+ +--------------------------+ +--------------------------+ | |
| Symptom: | |
| * iperf3 is fast (multiple mbit, ok for this internet connection) | |
| * iperf3 -R is slow (a few kbit) | |
| * tcpdump on VPN GW shows icmp "need to frag" packets towards 10.11.12.5 on | |
| * tcpdump on Exit show icmp "need to frag" packets towards 10.11.12.5 | |
| * rp_filter = 0 or all interfaces and default | |
| * mss clamping is active: iptables -v -n -L FORWARD -t mangle | |
| Chain FORWARD (policy ACCEPT 85M packets, 72G bytes) | |
| pkts bytes target prot opt in out source destination | |
| 210K 13M TCPMSS tcp -- br-ffmd ffrl+ anywhere anywhere tcp flags:SYN,RST/SYN TCPMSS clamp to PMTU | |
| 187K 11M TCPMSS tcp -- ffrl+ br-ffmd anywhere anywhere tcp flags:SYN,RST/SYN TCPMSS clamp to PMTU | |
| 210K 13M TCPMSS tcp -- br-ffmd ffrl+ anywhere anywhere tcp flags:SYN,RST/SYN TCPMSS clamp to PMTU | |
| 187K 11M TCPMSS tcp -- ffrl+ br-ffmd anywhere anywhere tcp flags:SYN,RST/SYN TCPMSS clamp to PMTU | |
| 3395 208K TCPMSS tcp -- backend+ any anywhere anywhere tcp flags:SYN,RST/SYN TCPMSS clamp to PMTU | |
| 2665 158K TCPMSS tcp -- any backend+ anywhere anywhere tcp flags:SYN,RST/SYN TCPMSS clamp to PMTU | |
| 2665 158K TCPMSS tcp -- any backend-babel1 anywhere anywhere tcp flags:SYN,RST/SYN TCPMSS clamp to PMTU | |
| => dito für OUTBOUND | |
| Problem: | |
| * no fragmentation seems to happen, packets are dropped between VPN GW and Exit and the VPN-host sends "need to frag" messages towards EXIT | |
| Solution: | |
| * Upgrade to jool 4.0.5 | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment