I'm tired of massive delay (500-1000 ms pings) while uploading large files. QoS helps some, but a significant portion remains due to bufferbloat (100-150ms). This leads to delay in VoIP calls and generally sluggish web browsing while uploading.
The new Codel algorithm in OpenWRT / Attitude Adjustment should help a lot. The results below show only ~5-6ms of added latency during uploads (i.e. 14ms vs 500+ms before!). Insane.
These instructions are for the TP-Link TL-WDR4300, because I got a good deal. If you have the money, buy a Netgear WNDR3800 and install CeroWRT, you'll probably see even better results.
In short, its fully supported in OpenWRT Attitude Adjustment and works well for my needs:
- Cheap
- Simultaneous dual-band (2.4/5GHz)
- Well supported in OpenWRT
- Atheros chipsets:
- Up to date drivers (req'd for Attitude Adjustment / Codel)
- Very advanced capabilities via the ath9k driver
- Want to standardize on Atheros since I often need WDS (Any combination of Broadcom WDS / RALink WDS / Atheros WDS does not work well together)
- Detachable antennas / cheap availability of high-gain dual-band antennas.
- (Hoping I can get by with a single router without needing WDS or repeaters).
The TL-WDR3600 is almost identical (2x2 MIMO on 5GHz instead of 3x3 on the TL-WDR4300). Setup / functionality should be the same.
Netgear WNDR3800 or anything supported in Attitude Adjustment.
I believe that currently rules out all Broadcom devices (for now?). I think they only have binary drivers and they only work on 2.4/2.6. The Codel work requires 3.3 or higher. Most Cisco/Linksys, Asus routers are using Broadcom. :-/
I think most of them are stuck on old Linux kernels (2.4/2.6), but Codel only works on 3.3+.
I think its primarily so they can support Broadcom drivers.
- Yes, codel is doing its job. Magnificently. Results below.
- I can barely even tell when I'm uploading. Latency is great. Browsing used to be very sluggish during uploads ("ughh" every time Google Music uploader starts). So far its seems much snappier, almost like I can't tell I'm uploading.
- Jitter during uploads is less than my sub-20ms target (one single g.711u packet). This means a shorter jitter buffer resulting in less delay.
- OpenWRT flashed fine right from the default TP-Link firmware. No tftp or serial cable required.
- OpenWRT is weird (but awesome!). Default install does not include web interface. Must:
- Flash firmware
- Telnet to 192.168.1.1 (ssh not enabled yet)
- Set up
/etc/config/network
for your internet connection only if your ISP not using DHCP (e.g. you're using DSL/PPPoE). - Once you have internet access, then you can use 'opkg install luci' to install the web interface.
- Most (all?) other config can happen via web interface (http://192.168.1.1) after that.
- No multiple SSID support so far (atleast in the GUI). I'm surprised honestly. Would have thought these Atheros chipsets supported that. Might be a bug with newer builds of OpenWRT (looking into it).
- Wireless (RF) output seems a little low. I've heard there's a flag you can set so OpenWRT will let you set it higher. Requires compiling your own OpenWRT build.
- Sample pings to my "first hop" during a large upload, listed cronologically (newest at bottom):
Reply from 207.118.111.1: bytes=32 time=9ms TTL=254 <- QoS neither installed, nor enabled. No uploads in progress
Reply from 207.118.111.1: bytes=32 time=9ms TTL=254
Reply from 207.118.111.1: bytes=32 time=9ms TTL=254
Reply from 207.118.111.1: bytes=32 time=9ms TTL=254
Reply from 207.118.111.1: bytes=32 time=336ms TTL=254 <-- Upload started. 300-500ms (no QoS)
Reply from 207.118.111.1: bytes=32 time=542ms TTL=254
Reply from 207.118.111.1: bytes=32 time=446ms TTL=254
Reply from 207.118.111.1: bytes=32 time=342ms TTL=254
Reply from 198.125.244.1: bytes=32 time=22ms TTL=254 <-- QoS (via Codel) Enabled, Upload in progress, this ICMP (ping) packets are not prioritized via QoS. Avg 16.09ms.
Reply from 198.125.244.1: bytes=32 time=12ms TTL=254
Reply from 198.125.244.1: bytes=32 time=19ms TTL=254
Reply from 198.125.244.1: bytes=32 time=21ms TTL=254
Reply from 198.125.244.1: bytes=32 time=14ms TTL=254
Reply from 198.125.244.1: bytes=32 time=21ms TTL=254
Reply from 198.125.244.1: bytes=32 time=16ms TTL=254
Reply from 198.125.244.1: bytes=32 time=21ms TTL=254
Reply from 198.125.244.1: bytes=32 time=18ms TTL=254
Reply from 198.125.244.1: bytes=32 time=17ms TTL=254
Reply from 198.125.244.1: bytes=32 time=19ms TTL=254
Reply from 198.125.244.1: bytes=32 time=15ms TTL=254
Reply from 198.125.244.1: bytes=32 time=9ms TTL=254
Reply from 198.125.244.1: bytes=32 time=14ms TTL=254
Reply from 198.125.244.1: bytes=32 time=14ms TTL=254
Reply from 198.125.244.1: bytes=32 time=9ms TTL=254
Reply from 198.125.244.1: bytes=32 time=9ms TTL=254
Reply from 198.125.244.1: bytes=32 time=10ms TTL=254
Reply from 198.125.244.1: bytes=32 time=24ms TTL=254
Reply from 198.125.244.1: bytes=32 time=14ms TTL=254
Reply from 198.125.244.1: bytes=32 time=16ms TTL=254
Reply from 198.125.244.1: bytes=32 time=20ms TTL=254
Reply from 98.125.244.1: bytes=32 time=10ms TTL=254 <-- QoS (via Codel) Enabled, Upload in progress, ICMP (ping) packets prioritized via QoS ('priority' queue). Avg 13.74ms.
Reply from 98.125.244.1: bytes=32 time=13ms TTL=254
Reply from 98.125.244.1: bytes=32 time=11ms TTL=254
Reply from 98.125.244.1: bytes=32 time=9ms TTL=254
Reply from 98.125.244.1: bytes=32 time=9ms TTL=254
Reply from 98.125.244.1: bytes=32 time=22ms TTL=254
Reply from 98.125.244.1: bytes=32 time=9ms TTL=254
Reply from 98.125.244.1: bytes=32 time=19ms TTL=254
Reply from 98.125.244.1: bytes=32 time=9ms TTL=254
Reply from 98.125.244.1: bytes=32 time=12ms TTL=254
Reply from 98.125.244.1: bytes=32 time=12ms TTL=254
Reply from 98.125.244.1: bytes=32 time=9ms TTL=254
Reply from 98.125.244.1: bytes=32 time=15ms TTL=254
Reply from 98.125.244.1: bytes=32 time=18ms TTL=254
Reply from 98.125.244.1: bytes=32 time=10ms TTL=254
Reply from 98.125.244.1: bytes=32 time=13ms TTL=254
Reply from 98.125.244.1: bytes=32 time=9ms TTL=254
Reply from 98.125.244.1: bytes=32 time=21ms TTL=254
Reply from 98.125.244.1: bytes=32 time=18ms TTL=254
Reply from 98.125.244.1: bytes=32 time=10ms TTL=254
Reply from 98.125.244.1: bytes=32 time=17ms TTL=254
Reply from 98.125.244.1: bytes=32 time=19ms TTL=254
Reply from 98.125.244.1: bytes=32 time=22ms TTL=254
- Codel is definitely in use (notice the
fq_codel
mentions below):
root@OpenWrt:/etc/config# tc -s -d qdisc show
qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 44528859 bytes 64845 pkt (dropped 0, overlimits 0 requeues 1)
backlog 0b 0p requeues 1
qdisc hfsc 1: dev pppoe-wan root refcnt 2 default 30
Sent 22510040 bytes 18630 pkt (dropped 3, overlimits 33984 requeues 0)
backlog 368b 1p requeues 0
qdisc fq_codel 100: dev pppoe-wan parent 1:10 limit 10240p flows 1024 quantum 1518 target 5.0ms interval 100.0ms ecn
Sent 82016 bytes 1684 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 429 drop_overlimit 0 new_flow_count 915 ecn_mark 0
new_flows_len 0 old_flows_len 1
qdisc fq_codel 200: dev pppoe-wan parent 1:20 limit 10240p flows 1024 quantum 1518 target 5.0ms interval 100.0ms ecn
Sent 8685 bytes 82 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 256 drop_overlimit 0 new_flow_count 82 ecn_mark 0
new_flows_len 1 old_flows_len 0
qdisc fq_codel 300: dev pppoe-wan parent 1:30 limit 10240p flows 1024 quantum 1518 target 5.0ms interval 100.0ms ecn
Sent 22411576 bytes 16803 pkt (dropped 2156, overlimits 0 requeues 0)
backlog 368b 1p requeues 0
maxpacket 1492 drop_overlimit 0 new_flow_count 863 ecn_mark 0
new_flows_len 0 old_flows_len 1
qdisc fq_codel 400: dev pppoe-wan parent 1:40 limit 10240p flows 1024 quantum 1518 target 5.0ms interval 100.0ms ecn
Sent 4917 bytes 59 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 477 drop_overlimit 0 new_flow_count 22 ecn_mark 0
new_flows_len 1 old_flows_len 0
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
Sent 2874823 bytes 17490 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev wlan0 root
Sent 39473 bytes 262 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev wlan1 root
Sent 37948 bytes 250 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc hfsc 1: dev ifb0 root refcnt 2 default 30
Sent 3329407 bytes 17488 pkt (dropped 1, overlimits 1729 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 100: dev ifb0 parent 1:10 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn
Sent 57011 bytes 399 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 560 drop_overlimit 0 new_flow_count 116 ecn_mark 0
new_flows_len 0 old_flows_len 1
qdisc fq_codel 200: dev ifb0 parent 1:20 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 300: dev ifb0 parent 1:30 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn
Sent 3272224 bytes 17087 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 1518 drop_overlimit 0 new_flow_count 1439 ecn_mark 0
new_flows_len 1 old_flows_len 1
qdisc fq_codel 400: dev ifb0 parent 1:40 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
-
Load the following webpages so you have access while offline: http://wiki.openwrt.org/doc/uci/network
-
Download openwrt for the tl-wdr4300. You want the
squashfs-factory
version. http://downloads.openwrt.org/snapshots/trunk/ar71xx/ http://downloads.openwrt.org/snapshots/trunk/ar71xx/openwrt-ar71xx-generic-tl-wdr4300-v1-squashfs-factory.bin -
Flash using web admin
-
Telnet to 192.168.1.1
-
Bring up the network so we can install LuCI (the web interface)
Optional (if your WAN connection doesn't use DHCP): Config per http://wiki.openwrt.org/doc/uci/network (edit /etc/config/network) when done, restart the network to make changes take effect: /etc/init.d/network restart
-
Install LuCI: http://wiki.openwrt.org/doc/howto/luci.essentials#installation opkg update opkg install luci
Enable the LuCI service (so that it will be started at every boot) and start uHTTPd: /etc/init.d/uhttpd enable /etc/init.d/uhttpd start
-
Install QoS web interface module for LuCI: opkg install luci-app-qos
-
Follow the QoS Quick Start Guide: http://wiki.openwrt.org/doc/uci/qos#quick.start.guide:
root@OpenWrt:/etc/config# opkg install qos-scripts root@OpenWrt:/etc/config# uci set qos.wan.upload=700 # Choose an appropriate number for your connection. My upload speed it ~768kbps upload speed, but its good to be a little conservative. root@OpenWrt:/etc/config# uci set qos.wan.download=10000 # Choose an appropriate number for your connection. My download speed is ~10500kbps, but its good to be a little conservative. root@OpenWrt:/etc/config# uci commit qos
root@OpenWrt:/etc/config# /etc/init.d/qos start root@OpenWrt:/etc/config# /etc/init.d/qos enable
-
Log into LuCI web interface (http://192.168.1.1)
10 LuCI > System > Administration * Set password * Enable SSH Access ("Dropbear Instance") on lan interface * Save & Apply
-
Reboot router (if no LuCI > Network > QoS tab. It didn't show up for me until after a reboot)
-
LuCI > Network > QoS
- Interfaces > WAN > Enable: checked
-
LuCI > Network > Wifi > Radio0 > Edit
-
Note: radio0 seems to be 2.4GHz
-
General Setup > Enable
-
General Setup
-
Device Configuration > General Setup tab:
- Channel
- Transmit Power
-
Device Configuration > Advanced Settings tab:
-
Interface Configuration > General Setup
- ESSID (Network Name)
- Mode (probably Access Point, but whatever you want)
- Network: lan
-
Interface Configuration > Wireless Security tab
- Encryption: WPA2-PSK
- Cipher: Force CCPM (AES) (TKIP is less secure, but you may need it if you have an old device that can't connect)
- Key:
-
Save & Apply
-
Thanks for posting your notes, this helped me decide to buy one.