Skip to content

Instantly share code, notes, and snippets.

@thepatrick
Last active September 30, 2018 12:07
Show Gist options
  • Save thepatrick/75693dcb676c8fe81f67d7567245bde2 to your computer and use it in GitHub Desktop.
Save thepatrick/75693dcb676c8fe81f67d7567245bde2 to your computer and use it in GitHub Desktop.
Experiments with MLVPN

This is how I got MLVPN working for me. I say working, but it's a work in progress really.

Based heavily on Linux with two ADSL uplinks for agregation and failover.

0. Assumptions:

  1. Server & Client are both running Ubuntu 18.04, 64bit intel.
  2. Server is on AWS. It's public IP is 52.62.1.227, private IP is 172.31.19.54 & the VPC is 172.31.0.0/16
  3. The client has three networks:
    1. 10.0.111.0/24 (upstream 1)
    2. 10.0.113.0/24 (upstream 2)
    3. 10.18.0.1/16 (we want to send traffic from here through the server!)
  4. You have another computer/vm that is connected to the 10.18.0.1/16 interface with a static IP and can ping 10.18.0.1, and has DNS (e.g. 1.1.1.1 & 1.0.0.1).

1. Compiling it

  1. Grabbed the source (git clone https://github.com/zehome/MLVPN.git) & cd in to it.
  2. Put build.sh in to this folder and run it.
  3. You can copy src/mlvpn to the other end

2. Get ready

  1. Enable routing on both theserver & the client: echo 1 > /proc/sys/net/ipv4/ip_forward
  2. The client needs 3 new route tables, add 101 up1 102 up2 150 mlvpn to /etc/iproute2/rt_tables
  3. Setup source routing (put source_routing.sh in /usr/local/sbin, run it and then setup networkd-dispatcher to run it...)
  4. Create an mlvpn user: adduser --quiet --system --no-create-home --home /var/run/mlvpn --shell /usr/sbin/nologin mlvpn
  5. You might need to create /var/run/mlvpn & chown -R mlvpn /var/run/mlvpn

3. Configure mlvpn

  1. Client: put client-mlvpn.conf and client-mlvpn_updown.sh into /etc/mlvpn/mlvpn.conf & /etc/mlvpn/mlvpn_updown.sh. Make them both root. mlvpn.conf needs to be 0600 and mlvpn_updown.sh needs to be 0700.
  2. Server: put server-mlvpn.conf and server-mlvpn_updown.sh into /etc/mlvpn/mlvpn.conf & /etc/mlvpn/mlvpn_updown.sh. Make them both root. mlvpn.conf needs to be 0600 and mlvpn_updown.sh needs to be 0700.
  3. Decide on a shared secret. Put it as the password in mlvpn.conf on both the client AND the server.

4. Try it out

  1. On the server run sudo ./mlvpn --user mlvpn -c /etc/mlvpn/mlvpn.conf --debug
  2. On the client run sudo ./mlvpn --user mlvpn -c /etc/mlvpn/mlvpn.conf --debug

99. TODO

  1. Actually run /usr/local/sbin/source_routing.sh again on reboot (I haven't figured out networkd-dispatcher yet)
  2. Do echo 1 > /proc/sys/net/ipv4/ip_forward on boot
  3. Basically make all of this survive a reboot.
  4. init.d-mlvpn is the initd script created by exracting the static build (which is x86 only), I need to test this and see if it's usable as is. It will definitely need defaults-mlvpn (put in /etc/defaults/mlpvn) too.
  5. Set up a DHCP server on the client machine & have it hand out IPs & DNS.
#!/bin/bash
set -e
set -x
sudo apt-get install flex bison build-essential pkg-config make autoconf libev-dev libsodium-dev libpcap-dev
EV_VERSION=4.22
LIBSODIUM_VERSION=1.0.8
PCAP_VERSION=1.7.4
wget http://dist.schmorp.de/libev/Attic/libev-${EV_VERSION}.tar.gz
wget https://github.com/jedisct1/libsodium/releases/download/1.0.8/libsodium-${LIBSODIUM_VERSION}.tar.gz
wget http://www.tcpdump.org/release/libpcap-${PCAP_VERSION}.tar.gz
tar xzf libev-${EV_VERSION}.tar.gz
tar xzf libsodium-${LIBSODIUM_VERSION}.tar.gz
tar xzf libpcap-${PCAP_VERSION}.tar.gz
echo libev
(cd libev-${EV_VERSION}
./configure --enable-static --disable-shared --prefix $HOME/libev/
make -j4 install)
echo libsodium
(cd libsodium-${LIBSODIUM_VERSION}
./configure --enable-static --disable-shared --prefix=$HOME/libsodium/
make -j4 install)
echo libpcap
(cd libpcap-${PCAP_VERSION}
./configure --disable-shared --prefix $HOME/libpcap/
make -j4 install)
libpcap_LIBS="-L${HOME}/libpcap/lib -lpcap" libpcap_CFLAGS="-I${HOME}/libpcap/include" libsodium_LIBS="-L${HOME}/libsodium/lib -lsodium" libsodium_CFLAGS=-I${HOME}/libsodium/include libev_LIBS="-L${HOME}/libev/lib -lev" libev_CFLAGS=-I${HOME}/libev/include ./configure --enable-filters LDFLAGS="-Wl,-Bdynamic" --prefix=${HOME}/mlvpn/
make
[general]
statuscommand = "/etc/mlvpn/mlvpn_updown.sh"
mode = "client"
tuntap = "tun"
interface_name = "mlvpn0"
timeout = 30
password = "SET THIS"
reorder_buffer_size = 64
loss_tolerance = 50
mtu = 1444
cleartext_data = 0
[filters]
[dsl1]
bindhost = "10.0.111.106"
remotehost = "52.62.1.227"
remoteport = 5080
bandwidth_upload = 5120000
[dsl2]
bindhost = "10.0.113.64"
remotehost = "52.62.1.227"
remoteport = 5081
#!/bin/bash
error=0; trap "error=$((error|1))" ERR
tuntap_intf="$1"
newstatus="$2"
rtun="$3"
[ -z "$newstatus" ] && exit 1
(
if [ "$newstatus" = "tuntap_up" ]; then
echo "$tuntap_intf setup"
/sbin/ip link set dev $tuntap_intf mtu 1400 up
/sbin/ip route add 172.31.0.0/16 dev $tuntap_intf scope link table mlvpn
/sbin/ip route add default via 172.31.19.54 dev $tuntap_intf table mlvpn
elif [ "$newstatus" = "tuntap_down" ]; then
echo "$tuntap_intf shutdown"
/sbin/ip route del 172.31.0.0/16 dev $tuntap_intf scope link table mlvpn
/sbin/ip route del default via 172.31.19.54 dev $tuntap_intf table mlvpn
elif [ "$newstatus" = "rtun_up" ]; then
echo "rtun [${rtun}] is up"
elif [ "$newstatus" = "rtun_down" ]; then
echo "rtun [${rtun}] is down"
fi
) >> /var/log/mlvpn_commands.log 2>&1
exit $errors
# This is the configuration file for /etc/init.d/mlvpn
# Start only theses VPN automatically via init script.
# Allowed values are "all", "none", or space separated list of
# names of VPNs.
# Empty meens "all".
# The VPN name refers to /etc/mlvpn/{VPNNAME}.conf without the brackets.
# On which unprivileged user the service should run
USER=mlvpn
#AUTOSTART="all"
#AUTOSTART="none"
#AUTOSTART="paris berlin"
# Set to "yes" if you don't want MLVPN to change it's proc title
DISABLE_PROCTITLE="no"
#!/bin/sh -e
### BEGIN INIT INFO
# Provides: mlvpn
# Required-Start: $network $remote_fs $syslog
# Required-Stop: $network $remote_fs $syslog
# Should-Start: network-manager
# Should-Stop: network-manager
# X-Start-Before: $x-display-manager gdm kdm xdm wdm ldm sdm nodm
# X-Interactive: true
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: MLVPN Link Aggregator
# Description: This script will start MLVPN tunnels as specified
# in /etc/default/mlvpn and /etc/mlvpn/*.conf
### END INIT INFO
# Original version by Robert Leslie
# <rob@mars.org>, edited by iwj and cs
# Modified for openvpn by Alberto Gonzalez Iniesta <agi@inittab.org>
# Modified for restarting / starting / stopping single tunnels by Richard Mueller <mueller@teamix.net>
# Modified for mlvpn by Laurent Coustet <ed@zehome.com>
. /lib/lsb/init-functions
test $DEBIAN_SCRIPT_DEBUG && set -v -x
DAEMON=/usr/sbin/mlvpn
RUNDIR=/var/run/mlvpn
USER=mlvpn
DESC="virtual private network daemon"
CONFIG_DIR=/etc/mlvpn
test -x $DAEMON || exit 0
test -d $CONFIG_DIR || exit 0
[ -d $RUNDIR ] || mkdir $RUNDIR
# Source defaults file; edit that file to configure this script.
AUTOSTART="all"
if test -e /etc/default/mlvpn ; then
. /etc/default/mlvpn
fi
start_vpn () {
log_progress_msg "$NAME"
STATUS=0
start-stop-daemon --start --quiet --oknodo \
--pidfile /var/run/mlvpn.$NAME.pid \
--background \
--make-pidfile \
--exec $DAEMON -- -c $CONFIG_DIR/$NAME.conf --user=$USER --name=$NAME || STATUS=1
}
stop_vpn () {
kill `cat $PIDFILE` || true
rm -f $PIDFILE
}
case "$1" in
start)
log_daemon_msg "Starting $DESC"
# autostart VPNs
if test -z "$2" ; then
# check if automatic startup is disabled by AUTOSTART=none
if test "x$AUTOSTART" = "xnone" -o -z "$AUTOSTART" ; then
log_warning_msg " Autostart disabled."
exit 0
fi
if test -z "$AUTOSTART" -o "x$AUTOSTART" = "xall" ; then
# all VPNs shall be started automatically
for CONFIG in `cd $CONFIG_DIR; ls *.conf 2> /dev/null`; do
NAME=${CONFIG%%.conf}
start_vpn
done
else
# start only specified VPNs
for NAME in $AUTOSTART ; do
if test -e $CONFIG_DIR/$NAME.conf ; then
start_vpn
else
log_failure_msg "No such VPN: $NAME"
STATUS=1
fi
done
fi
#start VPNs from command line
else
while shift ; do
[ -z "$1" ] && break
if test -e $CONFIG_DIR/$1.conf ; then
NAME=$1
start_vpn
else
log_failure_msg " No such VPN: $1"
STATUS=1
fi
done
fi
log_end_msg ${STATUS:-0}
;;
stop)
log_daemon_msg "Stopping $DESC"
if test -z "$2" ; then
for PIDFILE in `ls /var/run/mlvpn.*.pid 2> /dev/null`; do
NAME=$(basename $PIDFILE | cut -f2- -d'.')
NAME=${NAME%%.pid}
stop_vpn
log_progress_msg "$NAME"
done
else
while shift ; do
[ -z "$1" ] && break
if test -e /var/run/mlvpn.$1.pid ; then
PIDFILE=`ls /var/run/mlvpn.$1.pid 2> /dev/null`
NAME=$(basename $PIDFILE | cut -f2- -d'.')
NAME=${NAME%%.pid}
stop_vpn
log_progress_msg "$NAME"
else
log_failure_msg " (failure: No such VPN is running: $1)"
fi
done
fi
log_end_msg 0
;;
# Only 'reload' running VPNs. New ones will only start with 'start' or 'restart'.
reload|force-reload)
log_daemon_msg "Reloading $DESC"
if test -z "$2" ; then
for PIDFILE in `ls /var/run/mlvpn.*.pid 2> /dev/null`; do
NAME=$(basename $PIDFILE | cut -f2- -d'.')
NAME=${NAME%%.pid}
kill -HUP $(cat $PIDFILE)
log_progress_msg "$NAME (reloaded)"
done
else
while shift ; do
[ -z "$1" ] && break
if test -e /var/run/mlvpn.$1.pid ; then
PIDFILE=`ls /var/run/mlvpn.$1.pid 2> /dev/null`
NAME=$(basename $PIDFILE | cut -f2- -d'.')
NAME=${NAME%%.pid}
kill -HUP $(cat $PIDFILE)
log_progress_msg "$NAME (reloaded)"
fi
done
fi
log_end_msg 0
;;
cond-restart)
# Do nothing. This is used in apt-get upgrade for example.
;;
restart)
shift
$0 stop ${@}
sleep 1
$0 start ${@}
;;
status)
GLOBAL_STATUS=0
if test -z "$2" ; then
# We want status for all defined VPNs.
# Returns success if all autostarted VPNs are defined and running
if test "x$AUTOSTART" = "xnone" ; then
# Consider it a failure if AUTOSTART=none
log_warning_msg "No VPN autostarted"
GLOBAL_STATUS=1
else
if ! test -z "$AUTOSTART" -o "x$AUTOSTART" = "xall" ; then
# Consider it a failure if one of the autostarted VPN is not defined
for VPN in $AUTOSTART ; do
if ! test -f $CONFIG_DIR/$VPN.conf ; then
log_warning_msg "VPN '$VPN' is in AUTOSTART but is not defined"
GLOBAL_STATUS=1
fi
done
fi
fi
for CONFIG in `cd $CONFIG_DIR; ls *.conf 2> /dev/null`; do
NAME=${CONFIG%%.conf}
# Is it an autostarted VPN ?
if test -z "$AUTOSTART" -o "x$AUTOSTART" = "xall" ; then
AUTOVPN=1
else
if test "x$AUTOSTART" = "xnone" ; then
AUTOVPN=0
else
AUTOVPN=0
for VPN in $AUTOSTART; do
if test "x$VPN" = "x$NAME" ; then
AUTOVPN=1
fi
done
fi
fi
if test "x$AUTOVPN" = "x1" ; then
# If it is autostarted, then it contributes to global status
status_of_proc -p /var/run/mlvpn.${NAME}.pid mlvpn "VPN '${NAME}'" || GLOBAL_STATUS=1
else
status_of_proc -p /var/run/mlvpn.${NAME}.pid mlvpn "VPN '${NAME}' (non autostarted)" || true
fi
done
else
# We just want status for specified VPNs.
# Returns success if all specified VPNs are defined and running
while shift ; do
[ -z "$1" ] && break
NAME=$1
if test -e $CONFIG_DIR/$NAME.conf ; then
# Config exists
status_of_proc -p /var/run/mlvpn.${NAME}.pid mlvpn "VPN '${NAME}'" || GLOBAL_STATUS=1
else
# Config does not exist
log_warning_msg "VPN '$NAME': missing $CONFIG_DIR/$NAME.conf file !"
GLOBAL_STATUS=1
fi
done
fi
exit $GLOBAL_STATUS
;;
*)
echo "Usage: $0 {start|stop|reload|restart|status}" >&2
exit 1
;;
esac
exit 0
# vim:set ai sts=2 sw=2 tw=0:
[general]
statuscommand = "/etc/mlvpn/mlvpn_updown.sh"
mode = "server"
tuntap = "tun"
interface_name = "mlvpn0"
timeout = 30
password = ""
reorder_buffer_size = 64
loss_tolerance = 50
mtu = 1444
cleartext_data = 0
#[filters]
[dsl1]
bindhost = "0.0.0.0"
bindport = 5080
[dsl2]
bindhost = "0.0.0.0"
bindport = 5081
#!/bin/sh
REMOTE_IP_RANGE="10.18.0.0/16"
LOCAL_ETH="ens5"
error=0; trap "error=$((error|1))" ERR
tuntap_intf="$1"
newstatus="$2"
rtun="$3"
[ -z "$newstatus" ] && exit 1
(
if [ "$newstatus" = "tuntap_up" ]; then
echo "$tuntap_intf setup"
/sbin/ip link set dev $tuntap_intf mtu 1400 up
# NAT thru our server ($LOCAL_ETH is our output interface on the server)
# LAN $REMOTE_IP_RANGE from "client"
/sbin/ip route add $REMOTE_IP_RANGE dev $tuntap_intf
/sbin/iptables -t nat -A POSTROUTING -o $LOCAL_ETH -s $REMOTE_IP_RANGE -j MASQUERADE
elif [ "$newstatus" = "tuntap_down" ]; then
/sbin/iptables -t nat -D POSTROUTING -o $LOCAL_ETH -s $REMOTE_IP_RANGE -j MASQUERADE
fi
) >> /var/log/mlvpn_commands.log 2>&1
exit $errors
#!/bin/bash
# Inserting routes for upstream 1
ip route add 10.0.111.0/24 dev ens224 scope link table up1
ip route add default via 10.0.111.1 dev ens224 table up1
# Inserting routes for upstream 2
ip route add 10.0.113.0/24 dev ens192 scope link table up2
ip route add default via 10.0.113.1 dev ens192 table up2
# Inserting routes for mlvpn
ip route add 10.18.0.0/16 dev ens160 scope link table mlvpn
# And then do the magic to send to the right table
ip rule add from 10.0.111.0/24 table up1
ip rule add from 10.0.113.0/24 table up2
ip rule add from 10.18.0.0/16 table mlvpn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment