Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Openshift 3.11 in WSL2

Running Openshift 3.11 inside WSL2

This is not running yet, but nearly almost

Install WSL2 and oc client

First install a WSL2 with Ubuntu 20.04 as described at Microsoft

Enter wsl shell

wsl

And download the Openshift oc binary

curl -L https://github.com/openshift/origin/releases/download/v3.11.0/openshift-origin-client-tools-v3.11.0-0cbc58b-linux-64bit.tar.gz | tar xvz --directory /tmp
mv -v /tmp/openshift-origin-client-tools-v3.11.0-0cbc58b-linux-64bit/oc /usr/local/bin/oc
chmod +x /usr/local/bin/oc
rm -rf /tmp/openshift-origin-client-tools-v3.11.0-0cbc58b-linux-64bit

and socat

apt-get update && apt-get install -y socat

WSL2 and Docker Desktop for Windows

Docker Desktop with WSL2 and Openshift did NOT work for me.
But here is the documentation, what I tried:

I installed Docker for Windows with WSL Backend

I went to my wsl commandline and tried oc cluster up

oc cluster up --base-dir="/openshift" --public-hostname="127.0.0.1.nip.io" --forward-ports=true --loglevel=5 --server-loglevel=5

The main problem is, that oc cluster up starts some containers with --net=host (see source-code).
The Openshift Api is started on https://127.0.0.1:8443 and oc cluster waits for it to become ready.
But --net=host is not working yet with Docker Desktop, see Github issue.
The port is opened in the docker-desktop WSL distro (run wsl --list -v to see it), because that is the host linux for Docker Desktop Containers in WSL2.

So oc cluster up waits for 127.0.0.1:8443 to become ready, but it can't reach the port.

I played around with creating a tcp proxy with socat, but didn't find a way to go.

Another problem were the volume mounts. oc creates many containers mounting some volumes and I'm not sure, if they were correctly mapped between the distros.

Install Docker inside WSL2 (no Docker Desktop)

Openshift startup works with this, but I currently do not get access from Windows to Openshift.

So first I shutdown Docker Desktop and install Docker inside my WSL

curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

create a /etc/docker/daemon.json with some insecure registries for Openshift

{
  "insecure-registries": [
    "127.0.0.1/16",
    "docker-registry-default.127.0.0.1.nip.io:80"
  ],
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

then (re)start docker

sudo service docker restart

When starting a container I got the following error (see Github issue)

cgroups: cannot find cgroup mount destination: unknown

The following "fix" did help

sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd
sudo service docker restart
docker run --rm -it hello-world

OK, now I'm ready to run oc cluster up again

oc cluster up --base-dir="/openshift" --public-hostname="127.0.0.1.nip.io" --loglevel=5

Openshift started successfully

But now I don't get access from my Windows machine, but this is an issue of WSL2 itself, I think

Notes

  • oc cluster up with --server-loglevel=5 could be useful for debugging, but maybe bad for performance
  • oc cluster up with --forward-ports=true did not work for me (Api server not reachable)
  • Maybe an access from your Windows to https://127.0.0.1.nip.io:8443 does not work.
    This could be a WSL2 port forwarding issue.
    Stopping WSL with wsl --shutdown and then restarting it could help.
@KamranShahid

This comment has been minimized.

Copy link

@KamranShahid KamranShahid commented Apr 19, 2021

Getting following

I0419 18:16:47.968650 16263 run_self_hosted.go:181] Waiting for the kube-apiserver to be ready ...
I0419 18:16:47.969837 16263 run_self_hosted.go:557] Server isn't healthy yet. Waiting a little while. Get https://127.0.0.1:8443/healthz?timeout=32s: dial tcp 127.0.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment