Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Installing Docker for IOTstack

Installing Docker for IOTstack

A lot of issues raised on the IOTstack Discord channel turn out to have improper installation of docker and/or docker-compose as their underlying cause. The hows and whys of this happening are not really important. This gist is intended to help you diagnose and understand the problem, and then recover without having to start from a clean image.

Symptoms of prior installation

The symptoms usually appear when trying to run menu.sh. You get messages:

  • Suggesting docker and/or docker-compose are already installed when you are reasonably sure they should not be.

  • Implying a possible need to use sudo:

     Error getting docker version. Received permission denied error…
    
  • Complaining about groups:

     User is NOT in 'docker' group
    

Most of the time these "already installed" indications are correct. Either IOTstack has been installed before, or something else has needed docker and pulled it in as a prerequisite, or it's a straightforward misunderstanding where you thought docker had to be installed before IOTstack.

TL;DR

If you just want to convince menu.sh to re-install docker and docker-compose properly, feel free to jump straight to Uninstalling docker and docker-compose.

How IOTstack installs docker and docker-compose

Assuming a clean installation of Raspberry Pi OS (Raspbian), the starting point is:

  • neither docker nor docker-compose is installed
  • the docker group does not exist in /etc/group
  • the bluetooth group exists in /etc/group but user "pi" is not a member.

Under old-menu, you are told to:

  1. Run the commands:

    $ cd ~/IOTstack
    $ ./menu.sh
    
  2. Choose "Install Docker".

    "Install Docker" runs these commands:

    $ curl -fsSL https://get.docker.com | sh
    $ sudo usermod -G docker -a $USER
    $ sudo usermod -G bluetooth -a $USER
    $ sudo apt install -y docker-compose
    $ sudo reboot
    

Notes:

  1. The new menu automatically detects the absence of docker and/or docker-compose and takes you straight to the installation.
  2. The script downloaded by the curl command contains its own sudo commands, which is why it does not need to be invoked with sudo.
  3. The reboot is needed so that the usermod commands can take effect.

IOTstack installs an obsolete version of docker-compose

The menu (old or new) installs version 1.21.0 of docker-compose:

$ docker-compose version
docker-compose version 1.21.0, build unknown

This is quite an old version. It appears to have been compiled in 2018:

$ ls -l $(which docker-compose)
-rwxr-xr-x 1 root root 420 Oct  4  2018 /usr/bin/docker-compose

The latest compose-file version supported by docker-compose v1.21.0 is v3.6, which is why the compose-files produced by the IOTstack menu are pinned to that version:

$ head -1 ~/IOTstack/docker-compose.yml
version: '3.6'

It is not clear why the version of docker-compose installed by apt is so ancient. It just is. It also does not matter that it is ancient, providing that you don't need features that only appeared in later compose-file versions.

If you do need recent compose-file features, you must upgrade docker-compose by hand.

Upgrading docker-compose to the current version

IOTstack doesn't care which version of docker-compose you are using so you should not worry about breaking anything by upgrading.

Even if you don't actually need recent compose-file features, there is no reason not to upgrade. The latest docker-compose has been improved in other ways too (like progress messages during pull commands).

Start by making sure that your IOTstack is not running:

$ cd ~/IOTstack
$ docker-compose down

Remove the version of docker-compose installed by the menu:

$ sudo apt -y remove docker-compose

Install the current version of docker-compose:

$ sudo pip3 install -U docker-compose

Notes:

  • pip3 will work without the sudo but docker-compose will be installed in a user folder in your PATH such as ~/bin or ~/.local/bin. That will "work" but may cause problems down the track if an older copy in the user folder gets in the way of a later version installed with sudo.
  • The -U flag means "update docker-compose even if it is already installed".

Logout, then login again. You can check the version of docker-compose installed by pip3:

$ docker-compose version
docker-compose version 1.28.5, build unknown
$ ls -l $(which docker-compose)
-rwxr-xr-x 1 root root 218 Mar  1 23:15 /usr/local/bin/docker-compose

Finally, bring up your stack again:

$ cd ~/IOTstack
$ docker-compose up -d

Uninstalling docker and docker-compose

Uninstalling docker and docker-compose is a good way to convince menu.sh that it needs to reinstall both – and do it properly.

Start by making sure that your IOTstack is not running:

$ cd ~/IOTstack
$ docker-compose down

Run these commands:

$ sudo apt -y purge docker-ce docker-ce-cli containerd.io
$ sudo apt -y remove docker-compose
$ sudo pip3 uninstall -y docker-compose

Both forms of removing docker-compose are included here to be on the safe side.

If you notice a hint to run autoremove you can do that:

$ sudo apt autoremove

The next step is a reboot:

$ sudo reboot

After that, you have a clean slate as far as IOTstack is concerned. Running the old menu and choosing "Install Docker", or running the new menu and having it auto-detect the absence of docker and docker-compose will result in the kind of installation that IOTstack expects and assumes.

Notes:

  • This kind of "uninstallation" does not remove your Docker images, stopped containers or interfere with your ~/IOTstack folder. They will all still be there when you bring the stack up again. The objective of this "uninstallation" is not to nuke everything in sight. It is only to convince menu.sh to reinstall docker and docker-compose properly.
  • Please see the first command in the next section (begins sudo bash -c) and the associated footnote. No matter how you install Docker, this patch is highly recommended.

Scripting IOTstack installations

If you want to automate your IOTstack installations, you can try the following at the end of your standard Raspberry Pi build process:

  1. If you prefer old-menu:

    $ sudo bash -c '[ $(egrep -c "^allowinterfaces eth*,wlan*" /etc/dhcpcd.conf) -eq 0 ] && echo "allowinterfaces eth*,wlan*" >> /etc/dhcpcd.conf'
    $ sudo apt install -y git curl
    $ git clone -b old-menu https://github.com/SensorsIot/IOTstack.git ~/IOTstack 
    $ curl -fsSL https://get.docker.com | sh
    $ sudo usermod -G docker -a $USER
    $ sudo usermod -G bluetooth -a $USER
    $ sudo apt install -y python3-pip python3-dev
    $ sudo pip3 install -U docker-compose
    $ sudo pip3 install -U ruamel.yaml==0.16.12 blessed
    $ sudo reboot
    

    Note:

    • The first line (adding "allowinterfaces eth*,wlan*" to /etc/dhcpcd.conf) is highly recommended. It appears to prevent Docker from duelling with DHCPCD. Refer issues 219 and 253 for the history of this.
  2. If you prefer new-menu, use the same commands but remove -b old-menu from the git clone command.

After the reboot, the IOTstack folder is in a state where you can either:

  • run the menu and build your stack; or
  • restore a backup.

A word about ruamel.yaml & blessed

If you install IOTstack via either the install.sh script method or by cloning the repository and running menu.sh for the first time, ruamel.yaml and blessed are installed like this:

$ pip3 install -U ruamel.yaml==0.16.12 blessed

That means they are installed in ~/.local/lib, where they are only available to user pi and, more importantly, may not be available to a script that relies on their presence if that script is invoked by sudo.

If you do not understand why, run these two commands and study the results:

$ bash -c 'echo -e "HOME=$HOME\nPATH=$PATH"'
$ sudo bash -c 'echo -e "HOME=$HOME\nPATH=$PATH"'

The reason why the two commands produce different answers is because of the way sudo works. It means "become root, and then run the command". Becoming root changes a lot of things, including the home directory and path.

Because of this problem, I think it is preferable for dependencies to have global scope. You can find out how the dependencies were installed on your system by:

$ pip3 show ruamel.yaml blessed | egrep "^Location:"

If the answer is that either/both were installed in ~/.local/lib, and if you agree that a global scope would be preferable, you can fix it like this:

$ pip3 uninstall ruamel.yaml blessed
$ sudo pip3 install -U ruamel.yaml==0.16.12 blessed
$ pip3 show ruamel.yaml blessed | egrep "^Location:"

The result will show the dependencies installed in /usr/local/lib.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment