Skip to content

Instantly share code, notes, and snippets.

@ihameed

ihameed/Notes on Lenovo ThinkCentre M90n IoT as a FreeBSD router.md Secret

Created February 10, 2021 19:12
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save ihameed/c7537e0c61ba01e415d276a0ac1c122e to your computer and use it in GitHub Desktop.
Save ihameed/c7537e0c61ba01e415d276a0ac1c122e to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Notes on Lenovo ThinkCentre M90n IoT as a FreeBSD router.md

System Specs

Lenovo sold these on eBay near the end of 2020 for $200. I wouldn't spend more than that for one of these. As of 2021-01-09, Lenovo's MSRP for this MPN is $819.00. At that price you could get significantly better hardware from another vendor with better NICs, or you could build your own machine.

A WiFi card is preinstalled on this machine. It's easy to remove. There are two screws on the bottom of the case. Once removed, the bottom panel slides off. The WiFi card and the M.2 SSD are both easy to access. Example: https://www.youtube.com/watch?v=JsRa51G-Ec4&t=1m40s

With pfSense 2.4.5, the M90n IoT can handle filtering, NAT, and AQM for a 1 Gbit/sec symmetric link. I haven't measured how many packets per second this can forward with 84 byte packets. Speedtest.net measures ~922 Mbit/sec down and ~925 Mbit/sec up when using multiple connections. DSLReports measures 648 Mbit/sec down, 933 Mbit/sec up, and awards an "A" for bufferbloat.

Realtek NICs

See https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=166724. The network adapters on the M90n IoT exhibit this problem, and an easy way to reproduce this is to run an iperf3 bidirectional test between the M90n IoT and another computer. The net/realtek-re-kmod port fixes this problem. This port is available as a pkg package for pfSense 2.5 but not pfSense 2.4.5.

This port may nevertheless be built for pfSense 2.4.x by installing FreeBSD 11.3-RELEASE in a VM. Build net/realtek-re-kmod and then copy /boot/modules/if_re.ko from the FreeBSD VM to /boot/modules/if_re.ko on your pfSense machine. Follow the directions given in pkg-message:

Add the following lines to your /boot/loader.conf
to override the built-in FreeBSD re(4) driver.

if_re_load="YES"
if_re_name="/boot/modules/if_re.ko"

By default, the size of allocated mbufs is enough
to receive the largest Ethernet frame supported
by the card.  If your memory is highly fragmented,
trying to allocate contiguous pages (more than
4096 bytes) may result in driver hangs.
For this reason the value is tunable at boot time,
e.g. if you don't need Jumbo frames you can lower
the memory requirements and avoid this issue with:

hw.re.max_rx_mbuf_sz="2048"

pfSense 2.4.5 uses FreeBSD 11.3-STABLE r357046, not FreeBSD 11.3-RELEASE, but the resulting .ko still loads fine and works fine.

Alternatively, if you're into this sort of thing, there's a precompiled .ko available here: https://forums.serverbuilds.net/t/guide-resolve-realtek-nic-stability-issues-on-freebsd-pfsense-2-4-4-2-4-5-2-5-0-opnsense-use-2-5gb-realtek/3555.

One thing to note is that realtek-ke-kmod uses much more CPU time than base if_re. Expect to see 60-90% of a single core loaded (with dev.cpu.0.freq=2301) when running a single-connection 1Gbit/sec bandwidth test.

DisplayPort Hotplug

Neither FreeBSD 12.2 nor pfSense 2.4.5 will display vt consoles on DisplayPort monitors that are plugged in after headless booting. On vanilla FreeBSD, i915kms.ko from graphics/drm-kmod can work around this, although this is of questionable utility when attaching a monitor to a formerly-headless machine that has panicked--I have no idea if anything useful would happen. Some random links:

It should be possible to build this port on a vanilla FreeBSD machine and then copy the results to a pfSense machine. I tried doing this, but kldload complained with this message:

i915kms.ko: depends on drmn - not available or version mismatch

So I guess the "drmn" ABI changed between 11.3-RELEASE and 11.3-STABLE r357046. I didn't have the patience to pursue this further.

Bufferbloat/Active Queue Management (AQM):

FreeBSD's dummynet supports fq_codel. Problem: FreeBSD PF does not support dummynet. IPFW does. Options:

1. pfSense:

pfSense 2.4.4+ supports dummynet fq_codel with PF [1]. This is done via a patch to FreeBSD: Ermal Luçi authored a patch to integrate pf and dummynet in ~2007 [2]. He later maintained this patch for pfSense [4] [5]. Netgate still maintains this patch [6]. Sadly, neither the original patch nor pfSense's updated patch were merged into FreeBSD [2] [3].

  1. https://www.youtube.com/watch?v=o8nL81DzTlU&t=8m28s, https://www.slideshare.net/NetgateUSA/pfsense-244-short-topic-miscellany-pfsense-hangout-august-2018
  2. https://lists.freebsd.org/pipermail/freebsd-pf/2007-October/003849.html
  3. https://lists.freebsd.org/pipermail/freebsd-pf/2013-June/007066.html
  4. https://github.com/unexpectedBy/pfsense-tools/blob/master/patches/RELENG_10_0/dummynet.RELENG_10.diff
  5. https://github.com/pfsense/FreeBSD-src/commit/8d1444d3a535c4ccadf0cf2388883c078fa097e7
  6. https://github.com/pfsense/FreeBSD-src/commit/bc2eae151f8881f9c9262366adb5769922b45a29

There are some reported issues with pfSense 2.5 and dummynet limiters (which is necessary for fq_codel):

2. FreeBSD with IPFW:

IPFW in-kernel NAT uses a small fixed-size (4001 buckets) separate chaining hash table for NAT. Right now I have 1743 concurrent connections open, which would leave this table with a load factor of 0.43.

I decided to just use pfSense 2.4.5 with a hand-built if_re.ko for now, even though I'd really prefer to use vanilla FreeBSD. I don't currently have time to try to understand pf, dummynet, and Ermal's patch.

PF Trivia

PF + dummynet

IPFW Trivia

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment