wireguard, wireguard layer 2, wireguard over TCP

wireguard_layer2.md

Intro

This note describes how to connect two networks/devices/VMs over public network using Wireguard with Layer 2 support (ARP, IPv6 link-local, etc).

This can also be achieved using SSH and its "tap" tunnel, however, it does not provide the same level of latency and bandwidth as full-blown VPN such as Wireguard.

In addition, this note describes how to tunnel Wireguard over TCP connection. This may be of use if you encounter firewall in-between so, for instance, you can use TCP port 443 only.

Objective

Let's draw a network diagram of what we would like to have in as a result.

 Host on private LAN                                         Host on the Internet
 +---------------------------------+                       +-------------------------------+
 |   brtun bridge                  |                       |                  brtun bridge |
 | +-------------+                 |                       |                 +--------+    |
 | | ethX gretun |<->wg0<->udp2raw | <-Internet, TCP 443-> | udp2raw<->wg0<->| gretun |    |
 | +-------------+                 |                       |                 +--------+    |
 +---------------------------------+                       +-------------------------------+
 
 brtun: 192.168.0.200/24                                    brtun: 192.168.0.50/24
 wg0: 12.12.12.2/24                                         wg0: 12.12.12.1/24
 gretun: 12.12.12.2->12.12.12.1                             gretun: 12.12.12.1->12.12.12.2
 

Setting things up

Distinguishing between server and client here is based on the answer to the question: who can connect to Wireguard listener socket.

Server

Generating a key:

wg genkey | tee wgkeyprivs | wg pubkey > wgkeypubs

Creating a Wireguard interface, setting the private key and a unique private IPv4 address:

ip l a wg0 type wireguard
wg set wg0 private-key ./wgkeyprivs
ip a a 12.12.12.1/24 dev wg0

In case we want Wireguard over TCP, we have to decrease MTU:

ip l set dev wg0 mtu 1200

Configuring peer's public key and setting interface up (you can restrict allowed-ips to the range you want):

ip l set dev wg0 up
wg set wg0 listen-port 51820 peer $(cat wgkeypubc) allowed-ips 0.0.0.0/0

Creating GRETAP interface which will provide Layer 2 over our Wireguard tunnel:

ip l a gretun type gretap local 12.12.12.1 remote 12.12.12.2
ip l s gretun up

Configuring a bridge which includes GRETAP interface. This step is in fact optional, but it allows you some flexibility: you can assign an IP address to this bridge interface and/or you can add additional interfaces to this bridge.

ip l add name brtun type bridge
ip l set dev brtun up
ip l set gretun master brtun

Let's assign an IP address from the target network to this bridge:

sudo ip a a 192.168.0.51/24 dev brtun

If we want Wireguard over TCP, launch UDP2RAW listener which will do its magic. You can get UDP2RAW from this Git repo: https://github.com/wangyu-/udp2raw-tunnel/releases/

./udp2raw_amd64 -s -l YOUR_PUBLIC_IP:443 -r 127.0.0.1:51820 -a --fix-gro

Client

Client is very similar to the server described above.

Generating a key:

wg genkey | tee wgkeyprivc | wg pubkey > wgkeypubc

Creating a Wireguard interface, setting the private key and a unique private IPv4 address:

ip l a wg0 type wireguard
wg set wg0 private-key ./wgkeyprivc
ip a a 12.12.12.2/24 dev wg0

In case we want Wireguard over TCP, we have to decrease MTU:

ip l set dev wg0 mtu 1200

Configuring peer's public key, endpoint address and setting interface up (you can restrict allowed-ips to the range you want):

ip l set dev wg0 up
wg set wg0 listen-port 51820 peer $(cat wgkeypubs) allowed-ips 0.0.0.0/0 endpoint YOUR_PUBLIC_IP:51820 persistent-keepalive 15

In case you want Wireguard over TCP, set endpoint address to a localhost interface:

wg set wg0 listen-port 51820 peer $(cat wgkeypubs) allowed-ips 0.0.0.0/0 endpoint 127.0.0.1:7777 persistent-keepalive 15

Creating GRETAP interface which will provide Layer 2 over our Wireguard tunnel:

ip l a gretun type gretap local 12.12.12.2 remote 12.12.12.1
ip l s gretun up

Configuring a bridge which includes GRETAP interface.

ip l add name brtun type bridge
ip l set dev brtun up
ip l set gretun master brtun

Let's add a physical interface belonging to a LAN network to the bridge and start DHCP client:

ip l set ethX master brtun
dhcpcd ethX

If we want Wireguard over TCP, launch UDP2RAW listener on the localhost interface which connects to the corresponding public listener.

./udp2raw_amd64 -c -l 127.0.0.1:7777 -r YOUR_PUBLIC_IP:443 -a --fix-gro

Results

That's it. Now you will have Layer 2 connectivity between two separated machines over Wireguard VPN (optionally, over TCP). Enjoy your ARP and IPv6 link-local experience.

References

• https://notes.superlogical.ch/pages/note_wg/nolayer2/
• https://lafibre.info/vpn/tunnel-gre-sur-wireguard/&prev=search&pto=aue

for fake tcp,try phantun or mimc(debian 12 or kernel 6.1+),faster if you want.And I prefer use vxlan via wg.

@AndrewL733 - Your guide is really good, and it works. :) However, I do have an issue, and I'm a bit lost.

I have tried it between 2 data centers. When I install 2 Windows servers, one on each site (Site A and Site B), they do timeout when pinging each other. I did manage to get a speed test with over a Gigabit connection through the tunnel, but after the test, the connection dies. Then I need to reboot the WireGuard "switches" and wait 3-10 minutes to have a somewhat ok connection again. I have tried to change the MTU and also did an MTU test through the tunnel, but with no luck.

I can see some have commented on the high throughput and suggested changing the MTU. Has anyone figured it out? :)

The point of this for me is because I need to migrate some servers from one data center to another, and if I could make a big "switch", it would be nice to just take parts of the environment at a time.

I will paste my configuration down here, for Site A and Site B, and I hope you, or maybe someone else, have experienced the same issue and maybe have a solution.

Thank you for your time :)

Site A

#Ubuntu 22.04 LTS
#Remember to have Promiscuous mode, MAC address changes, and Forged transmits enabled on the interfaces.
#Change to root.
sudo su -
apt purge snapd -y;apt update -y ; apt full-upgrade -y;apt install wireguard bridge-utils openvswitch-switch-dpdk traceroute net-tools -y
reboot now
#Login again
sudo su -
wg genkey | tee wgkeyprivs | wg pubkey > wgkeypubs
cat wgkeyprivs
cat wgkeypubs
Private: SiteAPrivateKey
Public: SiteAPublicKey

nano /etc/netplan/00-installer-config.yaml
#Change the netplan configuration. Modify it for the respective interfaces, IPs, and WireGuard keys.
# This is the network config written by 'subiquity'
network:
  ethernets:
    #Uplink/switching interface 10.76.0.0/24
    ens192:
      dhcp4: false
      dhcp6: false

  bridges:
    br1:
      dhcp4: false
      dhcp6: false
      #IP to Wireguard VM
      addresses: [10.76.1.10/24]
      #DNS - This can be changed to the Domain Controller if needed
      nameservers:
        addresses:
          - 1.1.1.1
          - 1.0.0.1
      routes:
        - to: default
          via: 10.76.1.1
      interfaces:
        - ens192
      parameters:
        stp: false

  tunnels:
    wg0:
      mode: wireguard
      port: 51820
      #Site A Privatekey
      key: SiteAPrivateKey
      addresses:
        - 172.16.0.1/24
      peers:
        - allowed-ips: [172.16.0.0/24]
          endpoint: SiteB_WAN:51820
          #Peers public key
          keys:
            public: SiteBPublicKey
          keepalive: 90
  version: 2

#Activate Layer2
#Create bash script, with execute permission.
nano ~/gretun_cmd
  #!/bin/bash
  ip link add gretun1 type gretap local 172.16.0.1 remote 172.16.0.2 ignore-df nopmtudisc
  ip link set gretun1 up
  ip link set gretun1 master br1
  ip link set gretun1 mtu 1500
chmod u+rwx,g+r-wx,o-rwx ~/gretun_cmd
bash ~/gretun_cmd

#Create bash script, to Block DHCP Req over WireGuard VPN. 
nano ~/block_dhcp_cmd
  #!/bin/bash
  ebtables -A INPUT --in-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
  ebtables -A INPUT --in-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP
  ebtables -A FORWARD --out-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
  ebtables -A FORWARD --out-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP
chmod u+rwx,g+r-wx,o-rwx ~/block_dhcp_cmd
bash ~/block_dhcp_cmd

#Now Layer2 is active

Site B

#Ubuntu 22.04 LTS
#Remember to have Promiscuous mode, MAC address changes, and Forged transmits enabled on the interfaces.
#Change to root.
sudo su -
apt purge snapd -y;apt update -y ; apt full-upgrade -y;apt install wireguard bridge-utils openvswitch-switch-dpdk traceroute net-tools -y
reboot now
#Login again
sudo su -
wg genkey | tee wgkeyprivs | wg pubkey > wgkeypubs
cat wgkeyprivs
cat wgkeypubs
Private: SiteBPrivateKey
Public: SiteBPublicKey

nano /etc/netplan/00-installer-config.yaml
#Change the netplan configuration. Modify it for the respective interfaces, IPs, and WireGuard keys.
# This is the network config written by 'subiquity'
network:
  ethernets:
    #Uplink interface 10.76.2.0/24. This allows connection through WireGuard, with traffic passing through this interface.
    ens7:
      dhcp4: true
      dhcp6: false

    #Switching interface for bridge br1 10.76.1.0/24 
    #This is a dummy interface, BUT STILL REQUIRING Promiscuous mode, MAC address changes, and Forged transmits TO BE ACTIVE ON THE INTERFACE.
    #No gateway should be configured, as it goes through the WireGuard tunnel.
    ens3:
      dhcp4: false
      dhcp6: false

  bridges:
    br1:
      dhcp4: false
      dhcp6: false
      addresses: [10.76.1.11/24]
      #DNS - This can be changed to the Domain Controller if needed
      nameservers:
        addresses:
          - 1.1.1.1
          - 1.0.0.1
      #No routing needs to be configured; everything is routed through the WireGuard tunnel.
#      routes:
#        - to: default
#          via: 10.76.1.1
      interfaces:
        - ens3
      parameters:
        stp: false

  tunnels:
    wg0:
      mode: wireguard
      port: 51820
      #Site B Private key
      key: SiteBPrivateKey
      addresses:
        - 172.16.0.2/24
      peers:
        - allowed-ips: [172.16.0.0/24]
          endpoint: SiteA_WAN:51820
          #Peers public key
          keys:
            public: SiteAPublicKey
          keepalive: 90
  version: 2

#Activate Layer2
#Create bash script, with execute permission.
nano ~/gretun_cmd
  #!/bin/bash
  ip link add gretun1 type gretap local 172.16.0.2 remote 172.16.0.1 ignore-df nopmtudisc
  ip link set gretun1 up
  ip link set gretun1 master br1
  ip link set gretun1 mtu 1500
chmod u+rwx,g+r-wx,o-rwx ~/gretun_cmd
bash ~/gretun_cmd

#Create bash script, to Block DHCP Req over WireGuard VPN.
nano ~/block_dhcp_cmd
  #!/bin/bash
  ebtables -A INPUT --in-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
  ebtables -A INPUT --in-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP
  ebtables -A FORWARD --out-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
  ebtables -A FORWARD --out-interface gretun1 --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP
chmod u+rwx,g+r-wx,o-rwx ~/block_dhcp_cmd
bash ~/block_dhcp_cmd

#Now Layer2 is active