Skip to content

Instantly share code, notes, and snippets.

@thockin
Created November 18, 2015 19:21
Show Gist options
  • Save thockin/e67399d19757d7bb403b to your computer and use it in GitHub Desktop.
Save thockin/e67399d19757d7bb403b to your computer and use it in GitHub Desktop.
ipvsadm trouble
```
# hostA exists
root@kubernetes-minion-32zi:/home/thockin# curl 10.244.1.27:9376
hostA
# no ipvs yet
root@kubernetes-minion-32zi:/home/thockin# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
# VIP 10.9.8.7:9376
root@kubernetes-minion-32zi:/home/thockin# ipvsadm -A -t 10.9.8.7:9376 -s rr
# VIP -> hostA
root@kubernetes-minion-32zi:/home/thockin# ipvsadm -a -t 10.9.8.7:9376 -r 10.244.1.27
# proof
root@kubernetes-minion-32zi:/home/thockin# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.9.8.7:9376 rr
-> 10.244.1.27:9376 Route 1 0 0
# no workie
root@kubernetes-minion-32zi:/home/thockin# curl 10.9.8.7:9376
^C
# delete it and add as masq
root@kubernetes-minion-32zi:/home/thockin# ipvsadm -d -t 10.9.8.7:9376 -r 10.244.1.27
root@kubernetes-minion-32zi:/home/thockin# ipvsadm -a -t 10.9.8.7:9376 -r 10.244.1.27 -m
root@kubernetes-minion-32zi:/home/thockin# curl 10.9.8.7:9376
hostA
## What is going wrong here?
```
# Same as above
root@kubernetes-minion-32zi:/home/thockin# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.9.8.7:9376 rr
-> 10.244.1.27:9376 Masq 1 0 0
root@kubernetes-minion-32zi:/home/thockin# curl 10.244.1.27:9376
hostA
root@kubernetes-minion-32zi:/home/thockin# curl 10.9.8.7:9376
hostA
root@kubernetes-minion-32zi:/home/thockin# docker run -ti busybox
/ # wget -qO- 10.244.1.27:9376
hostA
/ # wget -qO- 10.9.8.7:9376
^C
@kobolog
Copy link

kobolog commented Nov 18, 2015

For DR/IPIP you need to set up some more stuff on backends:
– for DR, you need to set up an IP alias for the LB host on any non-ARPabe interface (dummy or loopback), since the LB will forward packets without doing anything to them except rewriting MAC, i.e. something like ip addr add $VIP dev lo && ip route add $VIP dev lo and both hosts have to be on the same L2.
– for IPIP, basically the same, except that you need to do that on the tunl0 device which is created by modprobe ipip
– ip_forward should be enabled on both, of course.
– Also note that usually $VIP != $LOAD_BALANCER_IP.

As for namespaces, I never tried this specific configuration (accessing a local ipvs from inside a netns), so need more research here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment