Skip to content

Instantly share code, notes, and snippets.

@eqhmcow
Last active June 18, 2018 09:14
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save eqhmcow/9967292 to your computer and use it in GitHub Desktop.
Save eqhmcow/9967292 to your computer and use it in GitHub Desktop.
use multiple internet connections, both simultaneously and transparently, both from the router itself and via boxes behind the router through NAT
#!/bin/bash
# As the "bufferbloat" folks have recently re-discovered and/or more widely
# publicized, congestion avoidance algorithms (such as those found in TCP) do
# a great job of allowing network endpoints to negotiate transfer rates that
# maximize a link's bandwidth usage without unduly penalizing any particular
# stream. This allows bulk transfer streams to use the maximum available
# bandwidth without affecting the latency of non-bulk (e.g. interactive)
# streams.
# In other words, TCP lets you have your cake and eat it too -- both fast
# downloads and low latency all at the same time.
# However, this only works if TCP's afore-mentioned congestion avoidance
# algorithms actually kick in. The most reliable method of signaling
# congestion is to drop packets. (There are other ways, such as ECN, but
# unfortunately they're still not in wide use.)
# Dropping packets to make the network work better is kinda counter-intuitive.
# But, that's how TCP works. And if you take advantage of that, you can make
# TCP work great.
# Dropping packets gets TCP's attention and fast. The sending endpoint
# throttles back to avoid further network congestion. In other words, your
# fast download slows down. Then, as long as there's no further congestion,
# the sending endpoint gradually increases the transfer rate. Then the cycle
# repeats. It can get a lot more complex than that simple explanation, but the
# main point is: dropping packets when there's congestion is good.
# Traffic shaping is all about slowing down and/or dropping (or ECN marking)
# packets. The thing is, it's much better for latency to simply drop packets
# than it is to slow them down. Linux has a couple of traffic shapers that
# aren't afraid to drop packets. One of the most well-known is TBF, the Token
# Bucket Filter. Normally it slows down packets to a specific rate. But, it
# also accepts a "limit" option to specify the maximum number of packets to
# queue. When the limit is exceeded, packets are dropped.
# TBF's simple "tail-drop" algorithm is actually one of the worst kinds of
# "active queue management" (AQM) that you can do. But even still, it can make
# a huge difference. Applying TBF alone (with a short enough limit) can make a
# maddeningly high-latency link usable again in short order.
# TBF's big disadvantage is that it's a "classless" shaper. That means you
# can't prioritize one TCP stream over another. That's where HTB, the
# Hierarchical Token Bucket, comes in. HTB uses the same general algorithm as
# TBF while also allowing you to filter specific traffic to prioritized queues.
# But HTB has a big weakness: it doesn't have a good, easy way of specifying a
# queue limit like TBF does. That means, compared to TBF, HTB is much more
# inclined to slow packets rather than to drop them. That hurts latency, bad.
# So now we come to Linux traffic shaping's best kept secret: the HFSC shaper.
# HFSC stands for Hierarchical Fair Service Curve. The linux implementation is
# a complex beast, enough so to have a 9 part question about it on serverfault
# ( http://serverfault.com/questions/105014/does-anyone-really-understand-how-hfsc-scheduling-in-linux-bsd-works ).
# Nonetheless, HFSC can be understood in a simplified way as HTB with limits.
# HFSC allows you to classify traffic (like HTB, unlike TBF), but it also has
# no fear of dropping packets (unlike HTB, like TBF).
# The results of HFSC are simply amazing. With it, it's possible to fully
# saturate a link while maintaining perfect non-bulk session interactivity.
# It is the holy grail of traffic shaping, and it's in the stock kernel.
# To get the best results, HFSC should be combined with SFQ (Stochastic
# Fairness Queueing) and optionally an ingress filter. If all three are used,
# it's possible to maintain low-latency interactive sessions even without any
# traffic prioritization. Further adding prioritization then maximizes
# interactivity.
# Here's how it's done:
# set this to your internet-facing network interface:
#WAN_INTERFACE=ppp0
#WAN_INTERFACE2=eth2
WAN_INTERFACE=eth2
# set this to your local network interface:
LAN_INTERFACE=lan
# how fast is your downlink?
#MAX_DOWNRATE=5900kbit
MAX_DOWNRATE=9900kbit
# how close should we get to max down? e.g. 90%
USE_DOWNPERCENT=0.95
# how fast is your uplink?
MAX_UPRATE=990kbit
#MAX_UPRATE=300kbit
#MAX_UPRATE=512kbit
# how close should we get to max up? e.g. 80%
USE_UPPERCENT=0.90
# what port do you want to prioritize? e.g. for ssh, use 22
# 3074 for xbox
INTERACTIVE_PORT=3074
## now for the magic
# remove any existing qdiscs
/sbin/tc qdisc del dev $WAN_INTERFACE root 2> /dev/null
/sbin/tc qdisc del dev $WAN_INTERFACE ingress 2> /dev/null
#/sbin/tc qdisc del dev $WAN_INTERFACE2 root 2> /dev/null
#/sbin/tc qdisc del dev $WAN_INTERFACE2 ingress 2> /dev/null
/sbin/tc qdisc del dev $LAN_INTERFACE root 2> /dev/null
/sbin/tc qdisc del dev $LAN_INTERFACE ingress 2> /dev/null
# computations
MAX_UPNUM=`echo $MAX_UPRATE | sed 's/[^0-9]//g'`
MAX_UPBASE=`echo $MAX_UPRATE | sed 's/[0-9]//g'`
MAX_DOWNNUM=`echo $MAX_DOWNRATE | sed 's/[^0-9]//g'`
MAX_DOWNBASE=`echo $MAX_DOWNRATE | sed 's/[0-9]//g'`
NEAR_MAX_UPNUM=`echo "$MAX_UPNUM * $USE_UPPERCENT" | bc | xargs printf "%.0f"`
NEAR_MAX_UPRATE="${NEAR_MAX_UPNUM}${MAX_UPBASE}"
NEAR_MAX_DOWNNUM=`echo "$MAX_DOWNNUM * $USE_DOWNPERCENT" | bc | xargs printf "%.0f"`
NEAR_MAX_DOWNRATE="${NEAR_MAX_DOWNNUM}${MAX_DOWNBASE}"
HALF_MAXUPNUM=$(( $MAX_UPNUM / 2 ))
HALF_MAXUP="${HALF_MAXUPNUM}${MAX_UPBASE}"
HALF_MAXDOWNNUM=$(( $MAX_DOWNNUM / 2 ))
HALF_MAXDOWN="${HALF_MAXDOWNNUM}${MAX_DOWNBASE}"
# install HFSC under WAN to limit upload
/sbin/tc qdisc add dev $WAN_INTERFACE root handle 1: hfsc default 11
/sbin/tc class add dev $WAN_INTERFACE parent 1: classid 1:1 hfsc sc rate $NEAR_MAX_UPRATE ul rate $NEAR_MAX_UPRATE
/sbin/tc class add dev $WAN_INTERFACE parent 1:1 classid 1:10 hfsc sc umax 1540 dmax 5ms rate $HALF_MAXUP ul rate $NEAR_MAX_UPRATE
/sbin/tc class add dev $WAN_INTERFACE parent 1:1 classid 1:11 hfsc sc umax 1540 dmax 5ms rate $HALF_MAXUP ul rate $HALF_MAXUP
# prioritize interactive port blocks: xbox gaming
for i in $INTERACTIVE_PORT 30000 31000 ; do
/sbin/tc filter add dev $WAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip sport $i 0xfc00 flowid 1:10
/sbin/tc filter add dev $WAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip dport $i 0xfc00 flowid 1:10
done
# additional single ports: ssh, VPN, webex, google hangouts
for i in 22 4500 9000 19302 19303 19304 19305 19306 19307 19308 19309; do
/sbin/tc filter add dev $WAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip sport $i 0xffff flowid 1:10
/sbin/tc filter add dev $WAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip dport $i 0xffff flowid 1:10
done
# also VPN IP range
/sbin/tc filter add dev $WAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip src 65.205.190.0/24 flowid 1:10
/sbin/tc filter add dev $WAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip dst 65.205.190.0/24 flowid 1:10
# also icmp
/sbin/tc filter add dev $WAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip protocol 1 0xff flowid 1:10
# prioritize ack
/sbin/tc filter add dev $WAN_INTERFACE protocol ip parent 1:0 prio 2 handle 2 fw flowid 1:10
# add SFQ
/sbin/tc qdisc add dev $WAN_INTERFACE parent 1:10 handle 30: sfq perturb 10
/sbin/tc qdisc add dev $WAN_INTERFACE parent 1:11 handle 40: sfq perturb 10
## wan2
#/sbin/tc qdisc add dev $WAN_INTERFACE2 root handle 1: hfsc default 11
##/sbin/tc class add dev $WAN_INTERFACE2 parent 1: classid 1:1 hfsc sc rate 899kbit ul rate 899kbit
##/sbin/tc class add dev $WAN_INTERFACE2 parent 1:1 classid 1:10 hfsc sc umax 1540 dmax 5ms rate 512kbit ul rate 899kbit
###/sbin/tc class add dev $WAN_INTERFACE2 parent 1:1 classid 1:11 hfsc sc umax 1540 dmax 5ms rate $HALF_MAXUP ul rate $NEAR_MAX_UPRATE
##/sbin/tc class add dev $WAN_INTERFACE2 parent 1:1 classid 1:11 hfsc sc umax 1540 dmax 5ms rate 512kbit ul rate 819kbit
#
#/sbin/tc class add dev $WAN_INTERFACE2 parent 1: classid 1:1 hfsc sc rate 4096kbit ul rate 4096kbit
#/sbin/tc class add dev $WAN_INTERFACE2 parent 1:1 classid 1:10 hfsc sc umax 1540 dmax 5ms rate 2048kbit ul rate 4096kbit
#/sbin/tc class add dev $WAN_INTERFACE2 parent 1:1 classid 1:11 hfsc sc umax 1540 dmax 5ms rate 2048kbit ul rate 3072kbit
#
# prioritize interactive port blocks: xbox gaming
#for i in $INTERACTIVE_PORT 30000 31000 ; do
# /sbin/tc filter add dev $WAN_INTERFACE2 protocol ip parent 1:0 prio 1 u32 match ip sport $i 0xfc00 flowid 1:10
# /sbin/tc filter add dev $WAN_INTERFACE2 protocol ip parent 1:0 prio 1 u32 match ip dport $i 0xfc00 flowid 1:10
#done
#
## additional single ports: ssh, VPN, webex, google hangouts
#for i in 22 4500 9000 19302 19303 19304 19305 19306 19307 19308 19309; do
# /sbin/tc filter add dev $WAN_INTERFACE2 protocol ip parent 1:0 prio 1 u32 match ip sport $i 0xffff flowid 1:10
# /sbin/tc filter add dev $WAN_INTERFACE2 protocol ip parent 1:0 prio 1 u32 match ip dport $i 0xffff flowid 1:10
#done
#
## also VPN IP range
#/sbin/tc filter add dev $WAN_INTERFACE2 protocol ip parent 1:0 prio 1 u32 match ip src 65.205.190.0/24 flowid 1:10
#/sbin/tc filter add dev $WAN_INTERFACE2 protocol ip parent 1:0 prio 1 u32 match ip dst 65.205.190.0/24 flowid 1:10
#
## also icmp
#/sbin/tc filter add dev $WAN_INTERFACE2 protocol ip parent 1:0 prio 1 u32 match ip protocol 1 0xff flowid 1:10
#
## prioritize ack
#/sbin/tc filter add dev $WAN_INTERFACE2 protocol ip parent 1:0 prio 2 handle 2 fw flowid 1:10
#
## add SFQ
#/sbin/tc qdisc add dev $WAN_INTERFACE2 parent 1:10 handle 30: sfq perturb 10
#/sbin/tc qdisc add dev $WAN_INTERFACE2 parent 1:11 handle 40: sfq perturb 10
# install ingress filter to limit download to 97% max
MAX_DOWNRATE_INGRESSNUM=`echo "$MAX_DOWNNUM * 0.97" | bc | xargs printf "%.0f"`
MAX_DOWNRATE_INGRESS="${MAX_DOWNRATE_INGRESSNUM}${MAX_DOWNBASE}"
/sbin/tc qdisc add dev $WAN_INTERFACE handle ffff: ingress
# prioritize interactive port blocks: xbox gaming
for i in $INTERACTIVE_PORT 30000 31000 ; do
/sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 1 u32 match ip sport $i 0xfc00 flowid :1
/sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 1 u32 match ip dport $i 0xfc00 flowid :1
done
# additional single ports: ssh, VPN, webex, google hangouts
for i in 22 4500 9000 19302 19303 19304 19305 19306 19307 19308 19309; do
/sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 1 u32 match ip sport $i 0xffff flowid :1
/sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 1 u32 match ip dport $i 0xffff flowid :1
done
# also VPN IP range
/sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 1 u32 match ip src 65.205.190.0/24 flowid :1
/sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 1 u32 match ip dst 65.205.190.0/24 flowid :1
# also icmp
/sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 1 u32 match ip protocol 1 0xff flowid :1
# police
/sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 50 u32 match ip src 0.0.0.0/0 police rate $MAX_DOWNRATE_INGRESS burst 20k drop flowid :2
# install HFSC under LAN to limit download
/sbin/tc qdisc add dev $LAN_INTERFACE root handle 1: hfsc default 11
/sbin/tc class add dev $LAN_INTERFACE parent 1: classid 1:1 hfsc sc rate 1000mbit ul rate 1000mbit
/sbin/tc class add dev $LAN_INTERFACE parent 1:1 classid 1:10 hfsc sc umax 1540 dmax 5ms rate 900mbit ul rate 900mbit
/sbin/tc class add dev $LAN_INTERFACE parent 1:1 classid 1:11 hfsc sc umax 1540 dmax 5ms rate $HALF_MAXDOWN ul rate $NEAR_MAX_DOWNRATE
# prioritize local LAN traffic
/sbin/tc filter add dev $LAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip src 10.0.0.0/24 match ip dst 10.0.0.0/24 flowid 1:10
# prioritize interactive port blocks: xbox gaming
for i in $INTERACTIVE_PORT 30000 31000 ; do
/sbin/tc filter add dev $LAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip sport $i 0xfc00 flowid 1:10
/sbin/tc filter add dev $LAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip dport $i 0xfc00 flowid 1:10
done
# additional single ports: ssh, VPN, webex, google hangouts
for i in 22 4500 9000 19302 19303 19304 19305 19306 19307 19308 19309; do
/sbin/tc filter add dev $LAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip sport $i 0xffff flowid 1:10
/sbin/tc filter add dev $LAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip dport $i 0xffff flowid 1:10
done
# also VPN IP range
/sbin/tc filter add dev $LAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip src 65.205.190.0/24 flowid 1:10
/sbin/tc filter add dev $LAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip dst 65.205.190.0/24 flowid 1:10
# also icmp
/sbin/tc filter add dev $LAN_INTERFACE protocol ip parent 1:0 prio 1 u32 match ip protocol 1 0xff flowid 1:10
## prioritize marked packets
#/sbin/tc filter add dev $LAN_INTERFACE protocol ip parent 1:0 prio 2 handle 1 fw flowid 1:10
# add SFQ
/sbin/tc qdisc add dev $LAN_INTERFACE parent 1:10 handle 30: sfq perturb 10
/sbin/tc qdisc add dev $LAN_INTERFACE parent 1:11 handle 40: sfq perturb 10
#!/bin/bash
# This script lets you use multiple internet connections, both simultaneously and transparently,
# both from the router itself and via boxes behind the router through NAT. For this to actually
# work with NAT, you most definitely need to recompile your linux kernel with the "Routing and
# Netfilter" patches from http://www.ssi.bg/~ja/#routes . (Using the latest full patchset is fine.)
# See also http://lartc.org/howto/lartc.rpdb.multiple-links.html
# You will need to modify this script so it actually works in your environment. Specifically,
# the IP parsing code directly below will probably not work for you without tweaking.
# Also, you'll want to change the interfaces (IF0, IF1 and IF2) to match your LAN and WAN interface names,
# and you'll need to change the gateway (P1, P2) and NET variables to match your current networks.
# You'll also need to update the hard-coded DNS IPs near the bottom of the script to match your ISP's DNS boxes.
IF0=lan
P0_NET=10.0.0.0/24
IF1=ppp0
IP1=`ip addr | grep $IF1 | grep inet | perl -p -e 's/^.*inet (\S+)\s+.*$/$1/'`
P1=98.69.164.1
P1_NET=98.69.164.1/32
IF2=eth2
IP2=`ip addr | grep $IF2 | grep inet | perl -p -e 's/^.*inet ([^\/]+)(\/\d+)?\s+.*$/$1/'`
P2=65.190.0.1
P2_NET=65.190.0.1/20
ip route flush table T1
ip route flush table T2
ip route add $P1_NET dev $IF1 src $IP1 table T1
ip route add default via $P1 table T1
ip route add $P2_NET dev $IF2 src $IP2 table T2
ip route add default via $P2 table T2
ip route add $P1_NET dev $IF1 src $IP1
ip route add $P2_NET dev $IF2 src $IP2
ip rule add from $IP1 table T1
ip rule add from $IP2 table T2
ip route add $P0_NET dev $IF0 table T1
ip route add $P2_NET dev $IF2 table T1
ip route add 127.0.0.0/8 dev lo table T1
ip route add $P0_NET dev $IF0 table T2
ip route add $P1_NET dev $IF1 table T2
ip route add 127.0.0.0/8 dev lo table T2
# hardcode dns routes
ip route add 205.152.128.23/32 dev $IF1 src $IP1
ip route add 205.152.37.23/32 dev $IF1 src $IP1
ip route add 209.18.47.61/32 dev $IF2 src $IP2
ip route add 209.18.47.62/32 dev $IF2 src $IP2
ip route replace default scope global nexthop via $P1 dev $IF1 weight 1 nexthop via $P2 dev $IF2 weight 1
# no cable modem (dsl only):
#ip route replace default scope global nexthop via $P1 dev $IF1 weight 1
# no dsl (cable modem only):
#ip route replace default scope global nexthop via $P2 dev $IF2 weight 1
@manickamk
Copy link

Hi,

I want to bandwidth shape on UDP port wise can you please give some example for like below

50002 = 2 Mbps

50003 = 3 Mbps

50004 = 4 Mbps

50005 = 5 Mbps

50006 = 6 Mbps

50007 = 7 Mbps

50008 = 8 Mbps

50009 = 9 Mbps

50010 = 10 Mbps

50011 = 11 Mbps

50012 = 12 Mbps

50013 = 13 Mbps

50014 = 14 Mbps

50015 = 15 Mbps

50016 = 16 Mbps

50017 = 17 Mbps

50018 = 18 Mbps

50019 = 19 Mbps

50020 = 20 Mbps ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment