Skip to content

Instantly share code, notes, and snippets.

@rmounce
Last active January 25, 2017 07:56
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rmounce/80738b892e81e54065034a08cef8e1cb to your computer and use it in GitHub Desktop.
Save rmounce/80738b892e81e54065034a08cef8e1cb to your computer and use it in GitHub Desktop.
Thoughts on precise shaping of constrained DOCSIS upstream links with cake AQM

Theoretical vs experimental results for Telstra EuroDOCSIS 3.0 (with "Ultimate Speed Boost" add-on) upstream rates at various packet sizes

Telstra's supplied modem (CG3100D-2BPAUS) appears to be shaping (through a ~94KiB FIFO) based on ethernet frames (inc. FCS, excl. preamble etc). The 6-byte DOCSIS MAC header and 4-byte "Upstream Privacy EH version 2 Element" are not a factor.

Theoretical vs actual throughput at various UDP payload sizes (with iperf)

12 byte payload

Naive:
(12 / (12 + 8 + 20 + 14 + 4))*2500 = 517
Accounting for minimum datagram size of 64 bytes:
(12 / (12 + 8 + 20 + 14 + 4 + 6))*2500 = 468
Actual ~= 470kbps

18 to 1472 byte payload

(18 / (18 + 8 + 20 + 14 + 4))*2500 = 703
Actual ~= 703kbps

(22 / (22 + 8 + 20 + 14 + 4))*2500 = 808
Actual ~= 810kbps

(24 / (24 + 8 + 20 + 14 + 4))*2500 = 857
Actual ~= 860kbps

(1472 / (1472 + 8 + 20 + 14 + 4))*2500 = 2424
Actual ~= 2424kbps

The advertised "2Mbps" service appears to be configured for precisely 2.5Mbps

Linear optimisation of 'naive' overhead compensator for TCP ACKs / full size packets

line rate (inc. ether + fcs overhead) = 2500kbps
desired TCP ACK (12 byte udp payload equiv) throughput = 468kbps
desired full size (1472 byte udp payload equiv) throughput = 2424kbps

(12 / (12 + 8 + 20 + 14 + x))*y = 468
(1472 / (1472 + 8 + 20 + 14 + x))*y = 2424

solve for x (spoofed overhead in bytes) and y (spoofed line rate in kbps)

x = 10.3649 (round up for safety) y = 2510.23 (round down for safety)

previous variables based on trial and error x = 10 y = 2500

The use of this linear optimisation may result in link saturation for very small packets (smaller than TCP ACK, rare), and slightly reduced link utilisation for mid-size packets.

Example cake configuration on LEDE

/etc/hotplug.d/iface/99-cake

#!/bin/sh

[ "$ACTION" = ifup ] && [ "$DEVICE" = "eth0.2" ] && {
  logger -t Cake Device: $DEVICE / Action: $ACTION

  tc qdisc del dev eth0.2 root
  tc qdisc add dev eth0.2 root cake bandwidth 2510Kbit triple-isolate overhead 11

}

Ideally, a shaper with a parameter for the minimum packet size should be used. I have prepared a patched verion of cake with a simple "eth" framing compensation option (with a static minimum frame size of 64 bytes), in the same vein as the atm and ptm options. A more generic option to define the minimum frame size may be more appropriate, as well as being less similarly named to the existing "ether-" overhead macros.

The configuration of my patched cake is as follows. This allows for 99.96% upstream link utilisation across all possible packet sizes.

/etc/hotplug.d/iface/99-cake

#!/bin/sh

[ "$ACTION" = ifup ] && [ "$DEVICE" = "eth0.2" ] && {
  logger -t Cake Device: $DEVICE / Action: $ACTION

  tc qdisc del dev eth0.2 root
  tc qdisc add dev eth0.2 root cake bandwidth 2499Kbit triple-isolate overhead 4 eth
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment