- kind (v0.8.0+, Tested with v0.8.1)
- kubectl
- helm (v3+, tested with v3.3.0)
- Vagrant w/ vagrant-libvirt
- Setup a new docker network to share between libvirt and kind
docker network create -d=bridge --subnet 172.30.0.0/16 --ip-range 172.30.100.0/24 \
-o com.docker.network.bridge.name=tink-dev \
-o com.docker.network.bridge.enable_icc=1 \
-o com.docker.network.bridge.host_binding_ipv4=0.0.0.0 \
-o com.docker.network.bridge.enable_ip_masquerade=true \
tink-dev
- Bring up a new kind cluster
KIND_EXPERIMENTAL_DOCKER_NETWORK=tink-dev kind create cluster --name tink-dev
- Install MetalLB (from metallb.universe.tf/installation/)
# create the metal-lb config
cat << EOF | kubectl create -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: metallb-config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 172.30.10.0-172.30.10.255
EOF
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install lb --set existingConfigMap=metallb-config bitnami/metallb
- Deploy the source-ip-app and create the service of type loadbalancer
kubectl create deployment source-ip-app --image=k8s.gcr.io/echoserver:1.4
kubectl expose deployment source-ip-app --name=loadbalancer --port=80 --target-port=8080 --type=LoadBalancer
kubectl patch svc loadbalancer -p '{"spec":{"externalTrafficPolicy":"Local"}}'
- Bring up the test instance on the tink-dev network and make a request to the source-ip-app through the load balancer
vagrant up test
vagrant ssh test
curl 172.30.10.0
If setting bridge-nf-call-iptables
to 0, then the client-ip correctly returns 172.30.0.100 (the IP address of the libvirt instance). However, if it is set to 1 (the default), the client-ip is reported as 172.30.100.0 (the IP assigned to the bridge device and returned as the docker host IP by kind entrypoint)
So, it seems that the behavior I'm seeing exists on Fedora 32 as a base host, but not Ubuntu 20.04...
Currently investigating some of the things described in https://vincent.bernat.ch/en/blog/2017-linux-bridge-isolation#bridge-processing, since the iptables rules between Fedora 32 and Ubuntu appear identical.