Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
IOTstack tutorial: Quick and Dirty WireGuard

IOTstack tutorial: Quick and Dirty WireGuard

Have you been tearing your hair out trying to get WireGuard to work under IOTstack? If so, you've (probably — hopefully) come to the right place.

This gist has been tested on both a Raspberry Pi 3B+ and 4B running Raspberry Pi OS (aka Raspbian). Your mileage may vary on other hardware or operating systems.


Installation procedure

  1. Make sure Raspberry Pi OS is up-to-date:

    $ sudo apt update
    $ sudo apt upgrade -y
    

    This is a SERIOUS instruction.

    Please don't think, "it's OK to skip this because my Raspbian is reasonably up-to-date." You need to bring it fully up-to-date for Wireguard. If you want to understand why, see the read only (:ro) flag … .

  2. Define this useful alias (optional):

    $ alias WGPS='docker ps --format "table {{.Names}}\t{{.RunningFor}}\t{{.Status}}" --filter "name=wireguard"'
    
  3. Start in the correct directory:

    $ cd ~/IOTstack
    
  4. Double-check that WireGuard is not running:

    $ WGPS
    NAMES       CREATED         STATUS
    

    If all you get is the titles ("NAMES" etc) then WireGuard is not running. If it is running, take it down with:

    $ docker-compose stop wireguard
    $ docker-compose rm -f wireguard
    
  5. This is the service definition that you should use for WireGuard:

      wireguard:
        image: ghcr.io/linuxserver/wireguard
        container_name: wireguard
        cap_add:
          - NET_ADMIN
          - SYS_MODULE
        environment:
          - PUID=1000
          - PGID=1000
          - TZ=«country/city»
          - SERVERURL=«your.dynamic.dns.name»
          - SERVERPORT=51820
          - PEERS=«my,device,list»
          - PEERDNS=auto
          - ALLOWEDIPS=0.0.0.0/0
        volumes:
          - ./volumes/wireguard:/config
          - /lib/modules:/lib/modules:ro
        ports:
          - "51820:51820/udp"
        sysctls:
          - net.ipv4.conf.all.src_valid_mark=1
        restart: unless-stopped
    

    Notes:

    • this WireGuard service definition is different to what you get if you run the menu:

      1. If your docker-compose.yml contains an existing WireGuard service definition, remove it.

      2. Copy the service definition above into a text editor. Then:

        • replace «country/city» with your timezone. Example:

           - TZ=Australia/Sydney
          
        • replace «your.dynamic.dns.name» with the DynamicDNS domain name you have registered with duckdns.org or noip.com etc. Example:

           - SERVERURL=myplace.hopto.org
          
        • replace «my,device,list» with an appropriate comma-separated list of your client devices (all the phones, tablets, laptops, desktops you want to use remotely to get back into your home network). Example:

           - PEERS=iPad,iPhone,laptop
          

          Many examples on the web use "PEERS=n" where "n" is a number. Don't do that. It's fragile. Use names!

        • see Understanding the port numbers then change the ports if necessary. It's perfectly OK to leave them all at 51820.

      3. Paste the result into your docker-compose.yml.

    Heed this advice:

    • If you find yourself worrying about the differences between the service definition above and the version provided by the menu, the question you should be asking yourself is, "do you actually want WireGuard to work reliably?"

    • If you haven't set up an account with a Dynamic DNS service provider and registered a domain name for your router, do that first.

    • Everything from "SERVERURL=" down to "PEERDNS=auto" (inclusive) affects the generated configurations (the QR codes):

      • You must think about SERVERURL=, PEERS= and the port numbers now.
      • Do not think, "I'll fire it up, see what happens, then come back and fix these things later." In my experience, WireGuard doesn't always propagate changes correctly so "fixing things later" often means "breaking things later".
  6. Ensure that there is no prior configuration (essential):

    $ sudo rm -rf ./volumes/wireguard
    
  7. Start WireGuard:

    $ docker-compose up -d wireguard
    Creating wireguard ... done
    
  8. Confirm that WireGuard is running:

    $ WGPS
    NAMES       CREATED        STATUS
    wireguard   2 minutes ago  Up About a minute
    

    Repeat the WGPS command a few times with a short delay in between. You are looking for signs that the container is restarting. If it is then this command is your friend:

    $ docker logs wireguard
    

    See also discussion of the read-only flag.

  9. Confirm that WireGuard has generated the configurations:

    $ tree -pu ./volumes/wireguard
    ./volumes/wireguard
    ├── [drwxr-xr-x pi      ]  coredns
    │   └── [-rw------- pi      ]  Corefile
    ├── [drwx------ pi      ]  peer_iPad
    │   ├── [-rw------- pi      ]  peer_iPad.conf
    │   ├── [-rw------- pi      ]  peer_iPad.png
    │   ├── [-rw------- pi      ]  privatekey-peer_iPad
    │   └── [-rw------- pi      ]  publickey-peer_iPad
    ├── [drwx------ pi      ]  peer_iPhone
    │   ├── [-rw------- pi      ]  peer_iPhone.conf
    │   ├── [-rw------- pi      ]  peer_iPhone.png
    │   ├── [-rw------- pi      ]  privatekey-peer_iPhone
    │   └── [-rw------- pi      ]  publickey-peer_iPhone
    ├── [drwx------ pi      ]  peer_laptop
    │   ├── [-rw------- pi      ]  peer_laptop.conf
    │   ├── [-rw------- pi      ]  peer_laptop.png
    │   ├── [-rw------- pi      ]  privatekey-peer_laptop
    │   └── [-rw------- pi      ]  publickey-peer_laptop
    ├── [drwxr-xr-x pi      ]  server
    │   ├── [-rw------- pi      ]  privatekey-server
    │   └── [-rw------- pi      ]  publickey-server
    ├── [drwxr-xr-x pi      ]  templates
    │   ├── [-rw-r--r-- pi      ]  peer.conf
    │   └── [-rw-r--r-- pi      ]  server.conf
    └── [-rw------- pi      ]  wg0.conf
    
    6 directories, 18 files
    

    Notice how this line in the service definition:

    PEERS=iPad,iPhone,laptop
    

    has become three sub-directories named peer_iPad and so on. You should expect the same pattern for your peers.

  10. If you want to get the QR codes out of there, try something like:

    $ find ./volumes/wireguard -name "*.png" -exec scp {} user@hostorip:. \;
    

Understanding the three port numbers

The WireGuard service definition above follows the convention of using "51820" in three places. To understand what each port number does, it is better to think of them like this:

      - SERVERPORT=«public»
 
    ports:
      - "«external»:«internal»/udp"

The «public» port is the port number that your clients (iDevice, laptop etc) will try to reach. This is the port number that your router needs to expose to the outside world.

The «external» port is the port number that your Raspberry Pi will be listening on. Your router needs to forward traffic to this port.

The «internal» port is the port number that wireguard (the process) will be listening to inside the WireGuard container. Docker handles forwarding between the «external» and «internal» port.

Rule #1:

  • You can change the «public» and «external» ports but you can't change the «internal» port unless you are prepared to do a lot more work.

Rule #2:

  • The «public» port forms part of the QR codes. If you decide to change the «public» port after you generate the QR codes, you will have to start over.

Rule #3:

  • Your router needs to know about both the «public» and «external» ports so, if you decide to change either of those, you must also reconfigure your router.

Configuring your router

Your router needs to be set up to forward WireGuard traffic. Routers have wildly different user interfaces but the concepts will be the same.

  1. The router sub-process you need to configure is called Network Address Translation (NAT) but it's not unheard of for this functionality to be grouped with FireWall.

  2. The NAT component you are looking for probably has a name like "Port Redirection", "Port Forwarding" or "NAT Forwarding".

    • It might also be under "Open Ports" but those are usually one-to-one mappings (ie incomingPort=outgoingPort), apply to port ranges, and are intended to target a single DMZ host.
  3. The configuration screen will contain at least the following fields:

    Field Value
    Protocol UDP
    Interface router's WAN interface
    Public Port «public»
    Private IP x.x.x.x
    Private Port «external»

    The fields may be in a different order and may have different names:

    • Protocol will usually default to "TCP" but you must change it to "UDP".
    • Interface is typically a popup menu. Generally it will either default to the name of the physical port on your router that connects to the outside world, or be some other sensible default like "All".
    • Public Port or External Port needs to be the value you chose for «public» in the WireGuard service definition (51820 if you didn't change it).
    • Private IP or Internal IP is the IP address of your Raspberry Pi. Note that this pretty much forces you to give your Raspberry Pi a statically-configured IP address (either a static binding in your DHCP server or a hard-coded address in the Raspberry Pi itself).
    • Private Port or Internal Port needs to be the value you chose for «external» in the WireGuard service definition (51820 if you didn't change it).

Telling the world

There are two parts to a Dynamic DNS service:

  1. You have to register with a service provider like DuckDNS or NoIP and choose a domain name that is not already taken.
  2. Something on your side of the network needs to tell your chosen service provider when the public IP address assigned to your router's WAN by your ISP changes.

Your router is in the best position to know when its IP address changes so, if your router can be configured to do that, use it!

IOTstack also provides a solution for DuckDNS at:

~/IOTstack/duck/duck.sh 

You need to configure that script with your credentials and arrange to have it run periodically to make sure your chosen service provider knows which public IP address your router is using.

The common arrangement is to invoke duck.sh from cron but you need to be aware that if you set cron to do that bang on every five minutes (0, 5, 10 etc) then it will probably fail. Either "everyone in the whole world" is hammering on the same schedule causing the AWS servers to stick their fingers in their ears and hum the DDoS song, or there is some periodic server reset occurring on the same time interval.

This is why I recommend using your router instead of the duck.sh script, if you can figure out how to make it work. Your router will only send updates when necessary and will retry when necessary until it is sure your Dynamic DNS service provider's system is up-to-date.

The read only (:ro) flag …

The :ro at the end of the following line in WireGuard's service definition means "read only":

- /lib/modules:/lib/modules:ro

If that flag is omitted then WireGuard may try to update the /lib/modules path in your operating system. To be clear, /lib/modules is both outside the WireGuard container and outside the normal persistent storage area in the ./volumes directory.

In my view this is very poor form for any container. The basic idea of containers is that processes are contained, include all their own dependencies, can be added and removed cleanly, and don't futz with the underlying operating system.

Writing into /lib/modules is also not needed on a Raspberry Pi running Raspberry Pi OS, providing that Raspberry Pi OS is up-to-date. That is why the first step in the installation procedure tells you to bring the system up-to-date.

If WireGuard refuses to install and you have good reason to suspect that WireGuard may be trying to write to /lib/modules then you can consider removing the :ro flag and re-trying. Just be aware that WireGuard will likely be modifying your operating system.

If WireGuard turns to custard …

If WireGuard stops working (usually because you changed something and it didn't propagate correctly), just:

  • Stop WireGuard
  • Erase its persistent storage area
  • Start WireGuard
  • Use the new QR codes on your devices.

IOTstack follow-through

If you want to make these changes IOTstack "menu friendly" (meaning "they have some chance of not being overwritten if you re-run the menu") then also do the following:

$ cd ~/IOTstack/services
$ rm -rf wireguard
$ mkdir wireguard

Note:

  • none of those commands should need sudo. It's OK to use sudo for the rm command but you should investigate further if the mkdir hits a permissions error because services should be owned by pi:pi.

New Menu

If you are running "new menu" (master branch), open the following path in your favourite text editor:

~/IOTstack/services/docker-compose.save.yml

The structure of that file is:

services:

followed by the service definition of each service you have chosen in the menu. Remove any existing service definition for WireGuard and then insert your working service definition from docker-compose.yml.

Hey - don't new-menu service definitions normally need networks?

Yes, but WireGuard is an exception. There's no need to add iotstack_nw.

Old Menu

If you are running "old menu" (old-menu branch):

  1. The file at this path:

    ~/IOTstack/services/selection.txt
    

    has a list of services the menu thinks of as "active". If "wireguard" is not in the list, add it.

  2. Copy your working WireGuard service definition from docker-compose.yml to this path:

    ~/IOTstack/services/wireguard/service.yml
    

Postscript

I'm working on a Pull Request to change the IOTstack WireGuard template to adopt the service definition described above.

In the meantime, I'd appreciate feedback if anything here doesn't work on your system or if you think something could be improved or clarified.

@ksetdekov

This comment has been minimized.

Copy link

@ksetdekov ksetdekov commented Mar 4, 2021

Thank you for a more detailed breakdown of the setup. Will try to recreate that.

@ksetdekov

This comment has been minimized.

Copy link

@ksetdekov ksetdekov commented Mar 8, 2021

Thanks for the code, unfortunately doesn't work for me(((
Did similar step, once starting as a regular user, once as a root user on Debian 10 on intel 6th gen i3.
root@deb:/home/kirill/IOTstack# WGPS NAMES CREATED STATUS wireguard 7 minutes ago Up 7 minutes

But no configuration are created:
root@deb:/home/kirill/IOTstack# tree -pu ./volumes/wireguard ./volumes/wireguard ├── [drwxr-xr-x root ] coredns ├── [drwxr-xr-x root ] templates └── [-rw------- root ] wg0.conf

And docker install fails(

root@deb:/home/kirill/IOTstack# docker logs wireguard
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-envfile: executing... 
[cont-init.d] 01-envfile: exited 0.
[cont-init.d] 10-adduser: executing... 

-------------------------------------
          _         ()
         | |  ___   _    __
         | | / __| | |  /  \ 
         | | \__ \ | | | () |
         |_| |___/ |_|  \__/


Brought to you by linuxserver.io
-------------------------------------

To support the app dev(s) visit:
WireGuard: https://www.wireguard.com/donations/

To support LSIO projects visit:
https://www.linuxserver.io/donate/
-------------------------------------
GID/UID
-------------------------------------

User uid:    1000
User gid:    1000
-------------------------------------

[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 30-config: executing... 
Uname info: Linux dd5392ed71fd 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64 x86_64 x86_64 GNU/Linux
RTNETLINK answers: Operation not supported
**** The wireguard module is not active, will attempt kernel header install and module compilation. ****
**** Attempting kernel header install ****
Get:1 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic/multiverse Sources [216 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic/restricted Sources [5,823 B]
Get:6 http://archive.ubuntu.com/ubuntu bionic/universe Sources [11.5 MB]
Get:7 http://archive.ubuntu.com/ubuntu bionic/main Sources [1,063 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1,344 kB]
Get:10 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]
Get:11 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Sources [21.8 kB]
Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe Sources [568 kB]
Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Sources [12.2 kB]
Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main Sources [634 kB]
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [353 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2,163 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [31.4 kB]
Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [2,394 kB]
Get:20 http://archive.ubuntu.com/ubuntu bionic-security/restricted Sources [18.4 kB]
Get:21 http://archive.ubuntu.com/ubuntu bionic-security/main Sources [313 kB]
Get:22 http://archive.ubuntu.com/ubuntu bionic-security/multiverse Sources [4,361 B]
Get:23 http://archive.ubuntu.com/ubuntu bionic-security/universe Sources [345 kB]
Get:24 http://archive.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [24.5 kB]
Get:25 http://archive.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,396 kB]
Get:26 http://archive.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [324 kB]
Get:27 http://archive.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,964 kB]
Fetched 36.7 MB in 4s (8,482 kB/s)
Reading package lists...
**** Debian host detected, attempting to install kernel headers from Debian Buster repo ****
Warning: apt-key output should not be parsed (stdout is not a terminal)
OK
Warning: apt-key output should not be parsed (stdout is not a terminal)
OK
Get:1 http://deb.debian.org/debian buster InRelease [122 kB]
Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease
Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit:4 http://archive.ubuntu.com/ubuntu bionic-security InRelease
Get:5 http://deb.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:6 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:7 http://deb.debian.org/debian buster-backports InRelease [46.7 kB]
Get:8 http://deb.debian.org/debian buster/contrib Sources [50.3 kB]
Get:9 http://deb.debian.org/debian buster/main Sources [10.4 MB]
Get:10 http://deb.debian.org/debian buster/non-free Sources [104 kB]
Get:11 http://deb.debian.org/debian buster/main amd64 Packages [10.7 MB]
Get:12 http://deb.debian.org/debian buster/contrib amd64 Packages [60.2 kB]
Get:13 http://deb.debian.org/debian buster/non-free amd64 Packages [107 kB]
Get:14 http://deb.debian.org/debian-security buster/updates/non-free Sources [610 B]
Get:15 http://deb.debian.org/debian-security buster/updates/main Sources [250 kB]
Get:16 http://deb.debian.org/debian-security buster/updates/main amd64 Packages [335 kB]
Get:17 http://deb.debian.org/debian-security buster/updates/non-free amd64 Packages [486 B]
Get:18 http://deb.debian.org/debian buster-updates/non-free Sources [638 B]
Get:19 http://deb.debian.org/debian buster-updates/main Sources [4,562 B]
Get:20 http://deb.debian.org/debian buster-updates/main amd64 Packages [10.8 kB]
Get:21 http://deb.debian.org/debian buster-updates/non-free amd64 Packages [529 B]
Get:22 http://deb.debian.org/debian buster-backports/non-free Sources [9,124 B]
Get:23 http://deb.debian.org/debian buster-backports/contrib Sources [5,072 B]
Get:24 http://deb.debian.org/debian buster-backports/main Sources [434 kB]
Get:25 http://deb.debian.org/debian buster-backports/main amd64 Packages [453 kB]
Get:26 http://deb.debian.org/debian buster-backports/contrib amd64 Packages [9,688 B]
Get:27 http://deb.debian.org/debian buster-backports/non-free amd64 Packages [34.5 kB]
Fetched 23.3 MB in 3s (6,827 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  cpp-8 gcc-8 libasan5 libgcc-8-dev libubsan1 linux-compiler-gcc-8-x86
  linux-headers-4.19.0-14-common linux-kbuild-4.19
Suggested packages:
  gcc-8-locales gcc-8-multilib gcc-8-doc libgcc1-dbg libgomp1-dbg libitm1-dbg
  libatomic1-dbg libasan5-dbg liblsan0-dbg libtsan0-dbg libubsan1-dbg
  libmpx2-dbg libquadmath0-dbg
The following NEW packages will be installed:
  cpp-8 gcc-8 libasan5 libgcc-8-dev libubsan1 linux-compiler-gcc-8-x86
  linux-headers-4.19.0-14-amd64 linux-headers-4.19.0-14-common
  linux-kbuild-4.19
0 upgraded, 9 newly installed, 0 to remove and 182 not upgraded.
Need to get 28.8 MB of archives.
After this operation, 128 MB of additional disk space will be used.
Get:1 http://deb.debian.org/debian buster/main amd64 linux-compiler-gcc-8-x86 amd64 4.19.171-2 [511 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 cpp-8 amd64 8.4.0-1ubuntu1~18.04 [7,225 kB]
Get:3 http://deb.debian.org/debian buster/main amd64 linux-headers-4.19.0-14-common all 4.19.171-2 [8,457 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libasan5 amd64 8.4.0-1ubuntu1~18.04 [366 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libubsan1 amd64 8.4.0-1ubuntu1~18.04 [122 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libgcc-8-dev amd64 8.4.0-1ubuntu1~18.04 [2,305 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 gcc-8 amd64 8.4.0-1ubuntu1~18.04 [8,044 kB]
Get:8 http://deb.debian.org/debian buster/main amd64 linux-kbuild-4.19 amd64 4.19.171-2 [744 kB]
Get:9 http://deb.debian.org/debian buster/main amd64 linux-headers-4.19.0-14-amd64 amd64 4.19.171-2 [994 kB]
Fetched 28.8 MB in 7s (4,217 kB/s)
Selecting previously unselected package cpp-8.
(Reading database ... 14292 files and directories currently installed.)
Preparing to unpack .../0-cpp-8_8.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking cpp-8 (8.4.0-1ubuntu1~18.04) ...
Selecting previously unselected package libasan5:amd64.
Preparing to unpack .../1-libasan5_8.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking libasan5:amd64 (8.4.0-1ubuntu1~18.04) ...
Selecting previously unselected package libubsan1:amd64.
Preparing to unpack .../2-libubsan1_8.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking libubsan1:amd64 (8.4.0-1ubuntu1~18.04) ...
Selecting previously unselected package libgcc-8-dev:amd64.
Preparing to unpack .../3-libgcc-8-dev_8.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking libgcc-8-dev:amd64 (8.4.0-1ubuntu1~18.04) ...
Selecting previously unselected package gcc-8.
Preparing to unpack .../4-gcc-8_8.4.0-1ubuntu1~18.04_amd64.deb ...
Unpacking gcc-8 (8.4.0-1ubuntu1~18.04) ...
Selecting previously unselected package linux-compiler-gcc-8-x86.
Preparing to unpack .../5-linux-compiler-gcc-8-x86_4.19.171-2_amd64.deb ...
Unpacking linux-compiler-gcc-8-x86 (4.19.171-2) ...
Selecting previously unselected package linux-headers-4.19.0-14-common.
Preparing to unpack .../6-linux-headers-4.19.0-14-common_4.19.171-2_all.deb ...
Unpacking linux-headers-4.19.0-14-common (4.19.171-2) ...
Selecting previously unselected package linux-kbuild-4.19.
Preparing to unpack .../7-linux-kbuild-4.19_4.19.171-2_amd64.deb ...
Unpacking linux-kbuild-4.19 (4.19.171-2) ...
Selecting previously unselected package linux-headers-4.19.0-14-amd64.
Preparing to unpack .../8-linux-headers-4.19.0-14-amd64_4.19.171-2_amd64.deb ...
Unpacking linux-headers-4.19.0-14-amd64 (4.19.171-2) ...
Setting up cpp-8 (8.4.0-1ubuntu1~18.04) ...
Setting up linux-headers-4.19.0-14-common (4.19.171-2) ...
Setting up linux-kbuild-4.19 (4.19.171-2) ...
Setting up libasan5:amd64 (8.4.0-1ubuntu1~18.04) ...
Setting up libubsan1:amd64 (8.4.0-1ubuntu1~18.04) ...
Setting up libgcc-8-dev:amd64 (8.4.0-1ubuntu1~18.04) ...
Setting up gcc-8 (8.4.0-1ubuntu1~18.04) ...
Setting up linux-compiler-gcc-8-x86 (4.19.171-2) ...
Setting up linux-headers-4.19.0-14-amd64 (4.19.171-2) ...
/etc/kernel/header_postinst.d/dkms:
 * dkms: running auto installation service for kernel 4.19.0-14-amd64
   ...done.
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
**** Kernel headers seem to be present, attempting to build the wireguard module. . . ****
**** Generating signing key ****
Generating a RSA private key
....++++
..............................................................................................++++
writing new private key to 'signing_key.pem'
-----
**** Building the module ****
make: Entering directory '/app/wireguard-linux-compat/src'
  CC [M]  /app/wireguard-linux-compat/src/main.o
  CC [M]  /app/wireguard-linux-compat/src/noise.o
  CC [M]  /app/wireguard-linux-compat/src/device.o
  CC [M]  /app/wireguard-linux-compat/src/peer.o
  CC [M]  /app/wireguard-linux-compat/src/timers.o
  CC [M]  /app/wireguard-linux-compat/src/queueing.o
  CC [M]  /app/wireguard-linux-compat/src/send.o
  CC [M]  /app/wireguard-linux-compat/src/receive.o
  CC [M]  /app/wireguard-linux-compat/src/socket.o
  CC [M]  /app/wireguard-linux-compat/src/peerlookup.o
  CC [M]  /app/wireguard-linux-compat/src/allowedips.o
  CC [M]  /app/wireguard-linux-compat/src/ratelimiter.o
  CC [M]  /app/wireguard-linux-compat/src/cookie.o
  CC [M]  /app/wireguard-linux-compat/src/netlink.o
  CC [M]  /app/wireguard-linux-compat/src/crypto/zinc/chacha20/chacha20.o
  PERLASM /app/wireguard-linux-compat/src/crypto/zinc/chacha20/chacha20-x86_64.S
  CC [M]  /app/wireguard-linux-compat/src/crypto/zinc/poly1305/poly1305.o
  PERLASM /app/wireguard-linux-compat/src/crypto/zinc/poly1305/poly1305-x86_64.S
  CC [M]  /app/wireguard-linux-compat/src/crypto/zinc/chacha20poly1305.o
  CC [M]  /app/wireguard-linux-compat/src/crypto/zinc/blake2s/blake2s.o
  AS [M]  /app/wireguard-linux-compat/src/crypto/zinc/blake2s/blake2s-x86_64.o
  CC [M]  /app/wireguard-linux-compat/src/crypto/zinc/curve25519/curve25519.o
  AS [M]  /app/wireguard-linux-compat/src/crypto/zinc/chacha20/chacha20-x86_64.o
  AS [M]  /app/wireguard-linux-compat/src/crypto/zinc/poly1305/poly1305-x86_64.o
  LD [M]  /app/wireguard-linux-compat/src/wireguard.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /app/wireguard-linux-compat/src/wireguard.mod.o
  LD [M]  /app/wireguard-linux-compat/src/wireguard.ko
make: Leaving directory '/app/wireguard-linux-compat/src'
make: Entering directory '/app/wireguard-linux-compat/src'
  INSTALL /app/wireguard-linux-compat/src/wireguard.ko
cp: cannot create regular file '/lib/modules/4.19.0-14-amd64/extra/wireguard.ko': Read-only file system
  DEPMOD  4.19.0-14-amd64
Warning: modules_install: missing 'System.map' file. Skipping depmod.
depmod -b "/" -a 4.19.0-14-amd64
depmod: ERROR: openat(//lib/modules/4.19.0-14-amd64, modules.dep.tmp, 1101, 644): Read-only file system
depmod: ERROR: openat(//lib/modules/4.19.0-14-amd64, modules.dep.bin.tmp, 1101, 644): Read-only file system
depmod: ERROR: openat(//lib/modules/4.19.0-14-amd64, modules.alias.tmp, 1101, 644): Read-only file system
depmod: ERROR: openat(//lib/modules/4.19.0-14-amd64, modules.alias.bin.tmp, 1101, 644): Read-only file system
depmod: ERROR: openat(//lib/modules/4.19.0-14-amd64, modules.softdep.tmp, 1101, 644): Read-only file system
depmod: ERROR: openat(//lib/modules/4.19.0-14-amd64, modules.symbols.tmp, 1101, 644): Read-only file system
depmod: ERROR: openat(//lib/modules/4.19.0-14-amd64, modules.symbols.bin.tmp, 1101, 644): Read-only file system
depmod: ERROR: openat(//lib/modules/4.19.0-14-amd64, modules.builtin.bin.tmp, 1101, 644): Read-only file system
depmod: ERROR: openat(//lib/modules/4.19.0-14-amd64, modules.devname.tmp, 1101, 644): Read-only file system
make: Leaving directory '/app/wireguard-linux-compat/src'
**** Let's test our new module. ****
RTNETLINK answers: Operation not supported
**** The module is not active, review the logs. Sleeping now. . . ****
@ksetdekov

This comment has been minimized.

Copy link

@ksetdekov ksetdekov commented Mar 8, 2021

I might be stupid and not linux literate enough yet, but are you setting own rights for "pi" user twice? And are all other commands ran as a su or root or a pi user?

$ sudo chown pi:pi ./volumes/wireguard
$ touch ./volumes/wireguard/wg0.conf
$ chmod 600 ./volumes/wireguard/wg0.conf
@Paraphraser

This comment has been minimized.

Copy link
Owner Author

@Paraphraser Paraphraser commented Mar 8, 2021

@ksetdekov

I suspect this could well be a case of "you might have been staring at it too hard and possibly seen 'chown' twice instead of a 'chown' and a 'chmod'". But you are the only one who will be able to say whether my guess is correct or not.

Let me go through it blow-by-blow. The first command in the sequence is to get a clean slate:

$ sudo rm -rf ./volumes/wireguard

wireguard directory all gone.

$ sudo mkdir -p ./volumes/wireguard

sudo needed because ./volumes directory is owned by root. Result is that wireguard directory will be created and owned by root.

$ sudo chown pi:pi ./volumes/wireguard

Now the wireguard directory is owned by pi.

$ touch ./volumes/wireguard/wg0.conf

sudo not needed to create a file inside wireguard directory because of previous step to change its ownership to pi. Result is wg0.conf file created. It will be owned by pi because it was created by pi inside a directory owned by pi. However, its default permissions are 644 (-rw-r--r--). That needs to be reduced so that only pi can read the file.

$ chmod 600 ./volumes/wireguard/wg0.conf 

wg0.conf now has permissions -rw-------.


Having said all that, I now think that steps 6 and 7 can be reduced to:

  • Step 6: ERASE the persistent storage area:

     $ sudo rm -rf ./volumes/wireguard
    
  • Step 7: there is no step 7

Here's why:

IMG_1078

You'll see that erasing the folder and then bringing up Wireguard is sufficient to get the entire job done correctly.

There's a long story here. I've been focusing on the problems Mosquitto has if it is not pre-initialised before the container is brought up for the first time. I thought Wireguard had the same kind of problem and that wg0.conf had to be in place beforehand. That's what directoryfix.sh (old menu) and build.sh (new menu) do, except that they do it inside ./services/wireguard/config. Using a Dockerfile I fixed Mosquitto so that you can just erase ./volumes/mosquitto, launch Mosquitto and it will self-repair. I turned my attention to Wireguard intending to fix the wg0.conf problem with a Dockerfile, only to realise it already creates its persistent storage folder and wg0.conf properly without any help from me.

What I don't know is whether Wireguard has always done this (ie directoryfix.sh and build.sh were never needed) or if it is a recent development. Either way, the bottom line is that Wireguard behaves like any other well-behaved container (eg NodeRed, Influx, Grafana, PiHole) and correctly self-initialises.

@Paraphraser

This comment has been minimized.

Copy link
Owner Author

@Paraphraser Paraphraser commented Mar 8, 2021

@ksetdekov

On your earlier post, it gets back to this line in Wireguard's fragment in docker-compose.yml:

- /lib/modules:/lib/modules:ro

The first time I installed Wireguard, I saw that it was "updating" my operating system by adding stuff to /lib/modules. In other words, Wireguard is updating your operating system outside of the container.

I reckon that's really, really, really, really BAD behaviour. The whole idea of containerisation is that effects are contained to a container. No container should ever futz with your operating system. I reacted to that knowledge by:

  1. Rebuilding the Pi from the ground up; and
  2. Adding the :ro (read-only) flag on the end of that line so that Wireguard would keep its hands to itself.

I also realised that Wireguard was actually telling me that my Raspbian needed to be updated. The problem went away after the rebuild and testing on another system showed that a sudo apt update; sudo apt upgrade also stopped Wireguard from trying to update Raspbian behind my back.

I left the :ro in place so that, if Wireguard stopped working in future and the reason turned out to be the same as the one shown in your traceback, I would know that I had to update Raspbian outside of container-space.

When I see root@deb in your traceback I assume "Debian". I was testing on Raspbian. Close, I know, but not guaranteed to be the same. I don't have a Debian system to test on so I can't say whether it makes any difference.

It's possible that it will work on your system if you either update Debian, or if you take that :ro off and try again. If you decide to remove the :ro, just be aware of the fact that the container is modifying your operating system.

@Paraphraser

This comment has been minimized.

Copy link
Owner Author

@Paraphraser Paraphraser commented Mar 9, 2021

@ksetdekov - gist updated. Now just says to erase the persistent storage area. Added a section on the read-only flag. Also added a table of contents to make navigation a bit easier.

Thanks for all the feedback. I'll be really interested to hear if either a full OS update avoids the need to remove the read-only flag or the only way to get going on Debian is by removing the read-only flag and letting WireGuard install stuff into your OS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment