flowchart LR
a[Cellular Dongle<br />Huawei E3372h-608] --> |USB| b[Raspberry Pi]
b-->|Ethernet| c[Billion Router<br />BiPAC 7800NXL]
c-->|Ethernet/WiFi| d[Home Broadband]
I have a temporary need for cellular broadband at home, and don't want to throw more e-waste at the problem. Alas it turns out my dusty old stalwart, the USB 3G/LTE compatible 7800NXL, doesn't want to play ball with the HiLink based E3372h dongle. The dongle works fine in a MacBook.
If only I could put a mini-MacBook between the dongle and the router... sounds like a job for one of the Pi's I've got stashed away!
All the guides I found were either out of date, clumsy or just plain ineffective. So here's a what worked for me. I've included explanations that should help you adjust to your own needs.
The starting point is a fresh Raspberry Pi OS Lite install, specifically 2023-05-03-raspios-bullseye-arm64-lite, configured for SSH access and connected over Ethernet.
Plug the cellular dongle into one of the Pi's USB ports. Contrary to expectations, the USB modeswitch worked for me without any effort. Once the dongle connected to cellular, the Pi had Internet access through a newly formed eth1 interface.
WiFi is no longer needed, so it can be turned off.
sudo rfkill block wifi
Then install dnsmasq. It seems like the easiest, right-sized solution to give the router what it needs to consider the Pi an Internet source.
sudo apt update
sudo apt install dnsmasq
Then add to the bottom of /etc/dhcpcd.conf:
interface eth0
static ip_address=192.168.2.1/24
nogateway
The nogateway prevents the Pi from prioritising Ethernet as a potential route to the Internet, since it's now strictly a downstream port. Obviously static sets a static IP for the Ethernet port. This will play well with dnsmasq and provide a predictable gateway address for the router.
The choice is fairly arbitrary, but I picked a subnet different from the 192.168.1.1/24 of the router and the 192.168.8.1/24 of the cellular dongle to make it easier to distinguish them.
Being a downstream port we don't need to bother with gateway and DNS settings for the Ethernet port.
By the way, I used dhcpcd.conf because while I find /etc/network/interfaces highly familiar, it feels like it's time to put it on ice. It's safe to leave it untouched. Raspberry Pi OS currently makes dhcpcd the default over NetworkManager, which suits me just fine. So using dhcpcd.conf to configure the ethernet port seems like the right thing to do.
Now on to dnsmasq. At the bottom of /etc/dnsmasq.conf add:
interface=eth0
#bind-interfaces # Causes silent failure on first boot because there's no eth0 to bind to at that time. Luckily, seems unnecessary.
#listen-address=192.168.2.1 # fails with “cannot assign requested address” because it’s redundant: https://forums.raspberrypi.com/viewtopic.php?p=1704404#p1704404
dhcp-range=192.168.2.50,192.168.2.100,12h
#server=1.1.1.1 # Forward DNS requests to Cloudflare. Not necessary at this level.
domain-needed # Don't forward short names
bogus-priv # Never forward addresses in the non-routed address spaces.
Honestly, the dnsmasq config was the hardest bit. I find Linux's boot behaviour progressively unpredictable over the last 10 years or so, and the various mechanisms to wait on network availability feel like working with a patchwork of bandaids. This configuration was the best I could find to ensure that a good working state would be established regardless of the sequence of power on and cable connections.
So finally, on to packet forwarding.
In /etc/sysctl.conf uncomment net.ipv4.ip_forward=1. This configures the kernel to be happy to redirect packets that arrive on one interface to another. Otherwise the kernel acts like a normal host and just drops packets not intended for itself.
There's an equivalent for IPv6: net.ipv6.conf.all.forwarding. But even though the cellular dongle establishes a IPv6 address, I haven't explored this side of things yet. It's been 25 years since IPv6 become the essential way forward, so I'm hoping I can squeeze a little more out of IPv4.
Okay, now the kernel is happy, set up the forwarding!
sudo iptables -F # Flush all the chains (ie. delete all the rules) in the default "filter" table
sudo iptables -t nat -F # Do the same for the "nat" table
sudo iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE # perform SNAT "the liberal way" by pretending to be the source of every outgoing packet
sudo iptables -A FORWARD -i eth1 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT # then forward anything we inititiated coming back in on Cellular, out to Ethernet, as if weren't here
sudo iptables -A FORWARD -i eth0 -o eth1 -j ACCEPT # and vice versa
Obviously eth0 and eth1 need to be exactly the destination and source interfaces respectively. But the rest seems to work for a variety of forwarding scenarios.
Aside: a question for the audience - I initially misspelled eth0 in the last rule (I still haven't managed to eradicate the scourge that is auto-correct from my life!). Everything still seemed to work just fine, despite the rule referencing a non-existent interface. Why is it so?
Now make sure the forwarding sticks through reboots by saving the configuration and repeating it when the network interface comes up.
sudo iptables-save > pi-gateway.conf
sudo mv pi-gateway.conf /etc/pi-gateway.conf
Create /etc/network/if-up.d/iptables with contents:
#!/bin/sh
iptables-restore < /etc/pi-gateway.conf
Make it executable, and the Pi is cooked!
sudo chmod +x /etc/network/if-up.d/iptables
Given my original intention was to take advantage of the 7800NXL's 3G/LTE USB port, it seems a trifle redundant to insert another network between the LAN and the Internet. But it's just a temporary cellular broadband thing, not some cloud infrastructure thing, and the following seems to work just fine.
In the router's web admin page, click Configuration --> WAN. Add a service and pick "Ethernet". Then configure it like so:
I mostly accepted the defaults, except I turned off IPv6 as explained in the section above. The important setting is "IP over Ethernet" which seems to simply mean "DHCP client". This is easy to understand, and works just fine, though I do wonder if there's a way to take advantage of the "Bridging" option. I find Billion's powerfully capable routers to be a little let down by obscure nomenclature and lack of explanatory docs.
Anyway, that's all that needs to be done on the router side - the Pi can now be plugged into the WAN port and away you go!
Reboot all the equipment, test the network and run journalctl -u dnsmasq on the Pi to make sure everything is happy.
Frankenrouter.
I've since replaced the generic RPi3 with an exquisitely fit for purpose Rock Pi S. Raxda is doing a fine job of advancing on the Linux SBC theme, and I can't really blame them for making Debian the path of least resistance. On the down side, well, we just have a bit more work to (un)do. And in the case of the Pi S, it's Buster-era work.
Oh, and because computers are involved, "a bit more work" was grossly optimistic.
First do the Raxda dance.
sudo apt update # expect apt.radxa.com to fail at this point, but we need *some* updates for the following to work. Otherwise we have neither chicken nor egg.
sudo apt install -y wget # get the chicken (or the egg, depending on your perspective)
export DISTRO=buster-stable
#echo "deb http://apt.radxa.com/$DISTRO/ ${DISTRO%-*} main" | sudo tee -a /etc/apt/sources.list.d/apt-radxa-com.list # already present, so don't bother
wget -O - apt.radxa.com/$DISTRO/public.key | sudo apt-key add - # and there's our egg (or chicken)
sudo apt update # should be smooth sailing from here
sudo apt install -y rockchip-overlay
sudo apt install -y linux-4.4-rock-pi-s-latest rockpis-dtbo
sudo apt install -y rtl8723ds-firmware rockchip-adbd resize-assistant
Now get to work. Start by uninstalling NetworkManager. It's kryptonite for unattended installations and this is about as far from a desktop scenario as you could imagine. Fortunately, Debian has no relevant dependencies (reconsider if libbluetooth3 is important to you), so it's safe to purge.
sudo apt purge network-manager
And because I'm not a caveman, install a modern pager and add the missing ZeroConf support so the Rock Pi answers to rockpis.local and I can find it again.
sudo apt install less avahi-daemon
Lastly, the essentials:
sudo apt install iptables dnsmasq
Then, optionally, clean up after ourselves.
sudo apt autoremove
Okay, on to configuration. dhcpcd is unavailable, but systemd-networkd is installed and networking is not. So let's work with the /etc/systemd/network files instead of /etc/network/interfaces.
Create a file /etc/systemd/network/eth0.network with contents:
[Match]
Name=eth0
[Network]
Address=192.168.2.1/24
This mimics the dhcpcd config from the RPi3. Now before we forget, make sure it takes effect on reboot!
sudo systemctl enable systemd-networkd
Next, just like before, at the bottom of /etc/dnsmasq.conf add:
interface=eth0
#bind-interfaces # Causes silent failure on first boot because there's no eth0 to bind to at that time. Luckily, seems unnecessary.
#listen-address=192.168.2.1 # fails with “cannot assign requested address” because it’s redundant: https://forums.raspberrypi.com/viewtopic.php?p=1704404#p1704404
dhcp-range=192.168.2.50,192.168.2.100,12h
server=1.1.1.1 # Forward DNS requests to Cloudflare. Sadly, this is likely much, much faster than managing it ourselves.
domain-needed # Don't forward short names
bogus-priv # Never forward addresses in the non-routed address spaces.
Moving on to the packet forwarding. The kernel flag is the same, but instead of editing /etc/sysctl.conf lets do it properly. Create a file called /etc/sysctl.d/30-ipforward.conf with contents:
net.ipv4.ip_forward=1
By inspecting the output of update-alternatives --config iptables and sudo ls -l /sbin/ipt* I can see iptables-save and friends are expecting to be working with iptables-nft. Thinking this was my opportunity to get on board with nftables I developed some equivalent rules using nft. Alas, I couldn't get iptables to successfully save and restore the rules, whether they were created with iptables or nft. In the end, I found a combination that worked:
| Create rules | Save rules | Restore rules |
|---|---|---|
iptables |
iptables-save |
iptables-legacy-restore |
But the mishmash left me a bit unsettled, so I decided to go all in on nftables. Here's how to enable nftables:
sudo systemctl enable nftables
And set the rules:
# Enable NAT. Note I choose the "ip" address family which is IPv4 only.
sudo nft add table ip nat
sudo nft add chain ip nat prerouting '{ type nat hook prerouting priority 100; }' # OMG, this requirement was hard earned: https://unix.stackexchange.com/questions/403389/using-linux-as-router-with-nftables-masquerade-not-forwarding-reply-back#comment1448354_439000
sudo nft add chain ip nat postrouting '{ type nat hook postrouting priority 100; }'
# Now we can masquerade as before
sudo nft add rule ip nat postrouting oifname eth1 masquerade
# and then forward packets as before
sudo nft add table ip filter
sudo nft add chain ip filter forward '{ type filter hook forward priority 0; policy drop; }'
sudo nft add rule ip filter forward ct state related,established accept
sudo nft add rule ip filter forward iifname eth0 oifname eth1 accept
Save the rules using nft too. Although now that I've read a few more man pages on iptables, lets draw on the standard name/path from that world:
echo "flush ruleset" > nftables.up.rules # start from scratch
sudo nft -s list ruleset > nftables.up.rules
sudo mkdir /etc/network
sudo mv nftables.up.rules /etc/network/
But more importantly, now that we've switched to systemd-networkd let's also switch from ifupdown to networkd-dispatcher to do the auto-loading on boot. Install it first:
sudo apt install networkd-dispatcher
Now we need to do the interface and state detection in the script itself (though I suspect the latter is redundant - both mind and logs seem to concur). Put a script in /etc/networkd-dispatcher/routable.d/nftables, like so:
#!/bin/sh
if [ "$IFACE" = lo ]; then
echo "$0: ignoring $IFACE for \`$STATE'"
exit 0
fi
case "$STATE" in
routable)
echo "$0: restoring nftables rules for $IFACE"
/usr/sbin/nft -f /etc/network/nftables.up.rules
;;
*)
echo "$0: nothing to do with $IFACE for \`$STATE'"
;;
esac
Echoing logging information to stdout means it appears in journalctl -u networkd-dispatcher, which is convenient, and only as painful to work with as the rest of the journalctl world.
Make it executable, and the (Rock) Pi is cooked!
sudo chmod +x /etc/networkd-dispatcher/routable.d/nftables
Just kidding... there's one more obstacle. Unlike Raspberry Pi OS, usb-modeswitch is not installed in this version of Debian. So when I plug in the dongle, I get this in dmesg:
[170317.837453] usb 2-1: new high-speed USB device number 2 using ehci-platform
[170317.971587] usb 2-1: New USB device found, idVendor=12d1, idProduct=1f01
[170317.972310] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[170317.973078] usb 2-1: Product: HUAWEI_MOBILE
[170317.973615] usb 2-1: Manufacturer: HUAWEI_MOBILE
[170317.974121] usb 2-1: SerialNumber: 0123456789ABCDEF
[170318.038013] usb-storage 2-1:1.0: USB Mass Storage device detected
[170318.040182] scsi host0: usb-storage 2-1:1.0
[170318.041831] usbcore: registered new interface driver usb-storage
[170318.047625] usbcore: registered new interface driver uas
[170319.043885] scsi 0:0:0:0: CD-ROM HUAWEI Mass Storage 2.31 PQ: 0 ANSI: 2
[170319.050697] sr 0:0:0:0: [sr0] scsi-1 drive
[170319.051183] cdrom: Uniform CD-ROM driver Revision: 3.20
[170319.053169] sr 0:0:0:0: Attached scsi CD-ROM sr0
[170319.058914] sr 0:0:0:0: Attached scsi generic sg0 type 5
[170321.422932] sr 0:0:0:0: [sr0] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[170321.423847] sr 0:0:0:0: [sr0] tag#0 Sense Key : Medium Error [current]
[170321.424537] sr 0:0:0:0: [sr0] tag#0 Add. Sense: Unrecovered read error
[170321.425221] sr 0:0:0:0: [sr0] tag#0 CDB: Read(10) 28 00 00 00 0d fc 00 00 02 00
[170321.426067] blk_update_request: critical medium error, dev sr0, sector 14320
And so on. It gets stuck enumerating it as USB mass storage instead of switching to a CDC Ethernet device. Easy fix, just install usb-modeswitch:
sudo apt install usb-modeswitch
That creates the file /lib/udev/rules.d/40-usb_modeswitch.rules which includes the lines:
# Generic entry for most Huawei devices, excluding Android phones
ATTRS{idVendor}=="12d1", ATTRS{manufacturer}!="Android", ATTR{bInterfaceNumber}=="00", ATTR{bInterfaceClass}=="08", RUN+="usb_modeswitch '/%k'"
Which is all that's needed to make the magic happen:
[12265.480426] usb 2-1: new high-speed USB device number 2 using ehci-platform
[12265.614505] usb 2-1: New USB device found, idVendor=12d1, idProduct=1f01
[12265.615221] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[12265.615943] usb 2-1: Product: HUAWEI_MOBILE
[12265.616475] usb 2-1: Manufacturer: HUAWEI_MOBILE
[12265.616970] usb 2-1: SerialNumber: 0123456789ABCDEF
[12266.034203] usb-storage 2-1:1.0: USB Mass Storage device detected
[12266.035579] scsi host0: usb-storage 2-1:1.0
[12266.036773] usbcore: registered new interface driver usb-storage
[12266.043948] usbcore: registered new interface driver uas
[12266.710726] usb 2-1: USB disconnect, device number 2
[12267.148497] usb 2-1: new high-speed USB device number 3 using ehci-platform
[12267.282325] usb 2-1: New USB device found, idVendor=12d1, idProduct=14db
[12267.283037] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[12267.283760] usb 2-1: Product: HUAWEI_MOBILE
[12267.284211] usb 2-1: Manufacturer: HUAWEI_MOBILE
[12267.402735] cdc_ether 2-1:1.0 eth1: register 'cdc_ether' at usb-ff440000.usb-1, CDC Ethernet Device, 00:1e:10:1f:00:00
[12267.404141] usbcore: registered new interface driver cdc_ether
[12267.438204] cdc_ether 2-1:1.0 enx001e101f0000: renamed from eth1
Oh yeah, and as that last line indicates, it's also not called eth1 anymore. Either go to war with Predictable Network Interface Names, or adjust accordingly.
And a mere handful or dozen of late nights by the glow of a screen later, we're ready to kick things off with a reboot.
sudo reboot
