Policy Routing VPN with Tinc and Nftables

README.md

Policy Routing VPN with Tinc and Nftables

This is just a personal backup, but if it helps you, that's even better.

Target

1. Make linux work as a router.
2. For domestic targets, connect them directly.
3. For traffic that need to bypass the firewall, use VPN.

Setup

1. Setup tinc or other vpn as usual(don't forget to enable ip forward sysctl, also use correct user group because I use this to determine if traffic should go direct, in my case its proxy).
2. Write the main.nft with additional ipsets.
3. Setup nft on tinc-up and node-up.
4. Enable nftables service(The debian comes with this) and start it.
5. Configure tinc service to run after nftables, also restart with it.

Why not iptables

Using nftables is more clear and efficient.

Advanced Usage

You can setup multiple nodes in tinc, so the whole service will be a reliable tunnel. You can move the node-up and down to tinc-up and down. After setting weights of the nodes, when part of nodes are down, your network connectivity should not be affected.

How to Debug

Add counter inside nft configuration and use nft list ruleset to watch it.

Known Issues

1. After setting up interfaces in tinc-up, apply nft configuration instantly may fail. I guess the interface information has not been updated? My workaround is to sleep 5 before applying nft.

etc-nftables.conf

#!/usr/sbin/nft -f

include "/opt/netfilter/main.nft";

main.nft

#!/usr/sbin/nft -f

flush ruleset
table ip tun {
    include "/opt/netfilter/cnip4.nft";
    include "/opt/netfilter/skip4.nft";

    chain tun {
        meta skgid proxy return
        fib daddr type { unspec, local, broadcast, anycast, multicast } return
        meta mark set ct mark
        meta mark and 0x10 == 0x10 return
        ip daddr @cnip4 ct mark set 0x12 meta mark set 0x12 return
        ip daddr @skip4 ct mark set 0x12 meta mark set 0x12 return
        ct mark set 0x11 meta mark set 0x11 counter
    }

    chain output {
        type route hook output priority mangle; policy accept;
        meta protocol ip ct direction original jump tun
    }

    chain prerouting {
        type filter hook prerouting priority mangle; policy accept;

        iif != ens192 return
        meta protocol ip ct direction original jump tun
    }
}

table ip forward {
    chain postrouting {
        type nat hook postrouting priority srcnat; policy accept;
        oif ens192 masquerade
    }
    chain prerouting {
        type nat hook prerouting priority dstnat; policy accept;
    }
}

table ip x {
    flowtable f {
        hook ingress priority filter
        devices = { ens192 }
        #flags offload
    }
    chain forward {
        type filter hook forward priority filter; policy accept;
        flow offload @f
    }
}

node-down

#!/bin/sh
ip rule flush table 17

node-up

#!/bin/sh
ip route del default table 17
ip route add default dev $INTERFACE table 17
ip rule flush table 17
ip rule add fwmark 17 lookup 17

skip4.nft

set skip4 {
    typeof ip daddr
    flags interval
    auto-merge
    elements = {
        10.0.0.0/8,
        192.168.0.0/16
    }
}

tinc-down

#!/bin/sh
ip route del 192.168.90.0/24 dev $INTERFACE
ip addr del 192.168.90.3 dev $INTERFACE
ip link set $INTERFACE down

# Remove MASQUERADE
nft delete chain forward postrouting-tinc

tinc-up

#!/bin/sh
ip link set $INTERFACE up
ip addr add 192.168.90.3 dev $INTERFACE
ip route add 192.168.90.0/24 dev $INTERFACE

# MASQUERADE
nft -f - <<EOF
table ip forward {
    chain postrouting-tinc {
        type nat hook postrouting priority srcnat; policy accept;
        oif $INTERFACE masquerade
    }
}
EOF

tinc.service

[Unit]
Description=Tinc VPN
Documentation=info:tinc
Documentation=man:tinc(8) man:tinc.conf(5)
Documentation=http://tinc-vpn.org/docs/
After=network.target nftables.service
Wants=network.target
Requires=nftables.service
PartOf=nftables.service

[Service]
Group=proxy
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/true
ExecReload=/bin/true
WorkingDirectory=/etc/tinc

[Install]
WantedBy=multi-user.target

Special thanks to @dyxushuai for help!