Workaround for minikube docker-env bind mount

minikube-docker-bind-mount.md

Workaround for minikube docker-env bind mount

To run docker-engine on macos (Big Sur 11.6, Intel, in this case) without the use of Docker Desktop (Desktop.app) is not as straightfoward as you might think. You'll need a Linux VM.

One way to do this is to install and run hyperkit and minikube. I used this: https://arnon.me/2021/09/replace-docker-with-minikube/ and this: https://itnext.io/goodbye-docker-desktop-hello-minikube-3649f2a1c469

Problem

❯ docker run --rm --mount type=bind,src="$(pwd)"/config,target=/config -it python:3.9 bash
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /Users/dennis.jonez/src/docker-minikube/config.
See 'docker run --help'.

As you can see, the simple bind mount of ./config to /config in the container, when using minikube, needs some love.

Solution!

My k8s diapers are showing :/ Spoiled by Linux.

Unlike docker on Linux or Docker Desktop app for Mac, a local folder needs to be mounted to the VM so docker can then mount it. This is why the docker ... --mount -type=bind ... complains it can't find the directory.

The first step of the solution is to mount the local folder into k8s:

minikube mount /Users/dennis.jonez/src/docker-minikube/config:/config

But, I still get the same error as above :(

The Workaround

It turns out, for whatever reason on my "I don't want to think, I just want it to work" minikube setup, the cluster's subnet was:

❯ minikube ip
172.16.21.3

BUT:

❯ minikube mount /Users/dennis.jonez:/Users/dennis.jonez
📁  Mounting host path /Users/dennis.mahle into VM as /Users/dennis.jonez ...
    ▪ Mount type:
    ▪ User ID:      docker
    ▪ Group ID:     docker
    ▪ Version:      9p2000.L
    ▪ Message Size: 262144
    ▪ Permissions:  755 (-rwxr-xr-x)
    ▪ Options:      map[]
    ▪ Bind Address: 192.168.64.1:63568
🚀  Userspace file server: ufs starting
🛑  Userspace file server is shutdown

I discovered a minikube mount bug because if you look carefully (even though I didn't):

• Bind Address: 192.168.64.1:63568

The mount command assumes I'm on 192.168.64.1 when I'm actually on 172.16.21.3 (subnet 172.16.21.0)

I found this back-channel page after much searching and headscratching: https://githubmemory.com/repo/kubernetes/minikube/issues/12729

This^ is where the magic happens:

❯ minikube mount /Users/dennis.jonez/src:/Users/dennis.jonez/src --ip 172.16.21.1
📁  Mounting host path /Users/dennis.jonez/src into VM as /Users/dennis.jonez/src ...
    ▪ Mount type:
    ▪ User ID:      docker
    ▪ Group ID:     docker
    ▪ Version:      9p2000.L
    ▪ Message Size: 262144
    ▪ Permissions:  755 (-rwxr-xr-x)
    ▪ Options:      map[]
    ▪ Bind Address: 172.16.21.1:63762
🚀  Userspace file server: ufs starting
✅  Successfully mounted /Users/dennis.jonez/src to /Users/dennis.jonez/src

📌  NOTE: This process must stay alive for the mount to be accessible ...

❯ docker run --rm --mount type=bind,src="$(pwd)"/config,target=/config -it python:3.9 bash
root@a5564de7b95f:/# ls -ld /config
drwxr-xr-x 1 1000 1000 224 Nov 11 20:11 /config
root@a5564de7b95f:/#

Yesssss!!!

Conclusion

Hopefully you get the gist:

• get your networking right
  • minikube ip
  • use the /24, host .1 to feed to the mount command
  • 172.16.21.1 in my case
  • minikube mount /source/path:/container/path --ip 172.16.21.1

This gist was created for myself but hopefully you found it quickly and didn't end up pulling your hair out too much 8)

Robot Bait

Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64 x86_64

Ahhh..finally explains the mystery with my mac. Thx so much! 👍

Glad you found this and that it helped! 🥳

Thank you 👍