zOrg1331/vxlan.md

## vxlan.md

      
    Raw
  

              vxlan.md
            
          
    VXLAN Configuration Examples for Linux

Here you can find several simple (static VTEP binding) configurations for VXLAN
networks which can be built using Linux-based box.
VXLAN over flat private network

Input data:

Two hosts on the same network: HostA (192.168.222.5/24) <--> HostB
(192.168.222.9/24). Both use eth0 interface.
Virtual machine on the HostA: VMa (192.168.56.101/24), bridged to HostA's
network interface 192.168.56.20/24
Virtual machine on the HostB: VMb (192.168.58.101/24), bridged to HostB's
network interface 192.168.58.20/24

Our target:

Provide network connectivity of VMa and VMb using VXLAN.

This is a bit more complicated setup. To demonstrate working VXLAN network we do
not need those virtual machines.
HostA setup:
# ip link add vxlan0 type vxlan id 123 dstport 4789 dev eth0
# bridge fdb append to 00:00:00:00:00:00 dst 192.168.222.9 dev vxlan0
# ip addr add 192.168.200.2/24 dev vxlan0
# ip link set dev vxlan0 up

HostB setup:
# ip link add vxlan0 type vxlan id 123 dstport 4789 dev eth0
# bridge fdb append to 00:00:00:00:00:00 dst 192.168.222.5 dev vxlan0
# ip addr add 192.168.200.3/24 dev vxlan0
# ip link set dev vxlan0 up

At this point HostA can ping HostB using IP address 192.168.200.3. This
traffic, including ARP broadcasts will be encapsulated into VXLAN with VNI 123.
To enable network connectivity between virtual machines we need additional
steps.
HostA:
# ip route add 192.168.58.0/24 via 192.168.200.3
# echo 1 > /proc/sys/net/ipv4/conf/vxlan0/proxy_arp

HostB:
# ip route add 192.168.56.0/24 via 192.168.200.2
# echo 1 > /proc/sys/net/ipv4/conf/vxlan0/proxy_arp

Now VMa can ping VMb using IP address 192.168.58.101. This traffic will be
encapsulated into VXLAN with VNI 123.
VXLAN between Internet-based host and machine on private network

Input data:

HostA on the Internet: IP 85.85.85.85 (interface name eth0).
HostB on some private network: IP 192.168.222.5/24.
GatewayA on the private network: internal IP 192.168.222.1/24. Masquarades all
outgoing traffic.

Our target:

Between HostA and HostB should be network connectivity using network
192.168.200.0/24 over VXLAN.

HostA setup:
# ip link add vxlan0 type vxlan id 123 dstport 4789 dev eth0
# ip addr add 192.168.200.2/24 dev vxlan0
# ip link set dev vxlan0 up

HostB setup:
# ip link add vxlan0 type vxlan id 123 dstport 4789 dev eth0
# bridge fdb append to 00:00:00:00:00:00 dst 85.85.85.85 dev vxlan0
# ip addr add 192.168.200.3/24 dev vxlan0
# ip link set dev vxlan0 up

Gateway setup:
# iptables -t nat -A PREROUTING -p udp -s 85.85.85.85 --dport 4789 -j DNAT --to-destination 192.168.222.5

At this point HostB can ping HostA using IP address 192.168.200.2.
If you want to avoid port forwarding setup on the gateway, set source port of
outgoing VXLAN UDP packets to 4789 too. By default Linux sends VXLAN packets
with random (determined using hash function) source port. VXLAN source port can
be specified during interface creation:
# ip link add vxlan0 type vxlan id 123 dstport 4789 srcport 4789 4790 dev eth0

VXLAN over Internet, weird way

I once faced with strange setup which I was required to repeat using Linux
tools. This describes how it was accomplished.
What we want to achieve:

Traffic between host with private IP and Internet-based should be encapsulated
into VXLAN.
Remote VTEP address: 85.85.85.85.
Traffic inside VXLAN should target the same IP: 85.85.85.85.

In fact, the last item is the most problematic.
Input data:

Internet-based HostA: IP 85.85.85.85, VNI 123. There is no access to this host.
Test machine HostB with private IP address 192.168.222.5/24 which accesses the
Internet via gateway.
Virtual machine VMa, bridged with HostB vxlan0 (!!!) interface: IP 192.168.200.101/24.

VMa setup:
# ip route add default via 192.168.200.100

HostB setup:
# sysctl -w net.ipv4.ip_forward=1
# ip link add vxlan0 type vxlan id 123 dstport 4789 srcport 4789 4790 dev eth0
# bridge fdb append to 00:00:00:00:00:00 dst 85.85.85.85 dev vxlan0
# ip addr add 192.168.200.100/24 dev vxlan0
# ip link set dev vxlan0 up

Important moment was to set source port of VXLAN packets to static value of 4789.
At this moment when VMa sends packets to 85.85.85.85 they arrive on HostB
192.168.200.100 interface and then routed via eth0 as normal traffic,
bypassing VXLAN encapsulation. We need to force traffic from VMa to go via
vxlan0 interface. For this we configure source routing policy.
HostB setup:
# echo "200 myvxlan" >> /etc/iproute2/rt_tables
# ip rule add from 192.168.200.101 table myvxlan

# ip route add default via 192.168.200.100 dev vxlan0 table myvxlan

Once source routing is configured and traffic goes inside VXLAN channel, HostB
will start with ARP broadcasts to understand where to direct packets. In my
specific case, ARP was disabled. So I added static entry linking upstream IP to
some random MAC address:
# arp -i vxlan0 -s 85.85.85.85 2a:5e:9e:99:39:96

At this moment we have bidirectional communication channel between VMa and HostA
over the Internet encapsulated into VXLAN. In this case all outgoing packets
from VMa will be transmitted inside VXLAN.
If you intercept this traffic quite strange picture appears. Outgoing packet to
85.85.85.85 from VMa will have the following headers:

Ethernet headers: src -- MAC of HostB's eth0, dst -- MAC of the gateway.
IP header: src -- IP of HostB 192.168.222.5, dst -- 85.85.85.85.
UDP: src port 4789, dst port 4789, VXLAN with VNI 123.
VXLAN Ethernet: src -- MAC of HostB's vxlan0, dst MAC -- the random one we set.
VXLAN IP header: src -- IP of VMa 192.168.200.101, dst -- 85.85.85.85.
VXLAN the rest as was created by VMa.