Skip to content

Instantly share code, notes, and snippets.

@protosam
Last active March 8, 2022 22:50
Show Gist options
  • Save protosam/11800faea25a3f89af9ece4f11c72f1d to your computer and use it in GitHub Desktop.
Save protosam/11800faea25a3f89af9ece4f11c72f1d to your computer and use it in GitHub Desktop.
Notes for switching from Docker Desktop to Minikube

With Docker Desktop becoming more restricted, I've decided to move on to just using minikube. In doing so, I've consolidated my notes as follows.

Installation

Use brew to install the docker cli and minikube.

$ brew install minikube docker kubectl hyperkit

Running Minikube

The first time you start minikube, you should specify any settings you desire.

$ minikube start --addons=registry --cni=calico --driver=hyperkit --cpus=8 --memory=8g

When you start with --cni=calico it might take some extra time for the CNI to become active.

$ kubectl -n kube-system wait --for=condition=ready --all pods --timeout=10m

You can check the status with this command if you're unsure if minikube is running or not.

$ minikube status

Anytime you need to start minikube back up, you don't need to set additional flags anymore

$ minikube start

If you need to stop minikube (to free up RAM or CPU), run this command.

$ minikube stop

Resetting Minikube

It doesn't matter if minikube is running or not, you can issue the delete command.

$ minikube delete
🔥  Deleting "minikube" in hyperkit ...
💀  Removed all traces of the "minikube" cluster.

Then just do the start command as mentioned in the Running Minikube section.

Using Docker CLI

Minikube needs to be running and you need to use envirnment variables so that the docker cli tool can contact minikube.

This command does the environment setup for you, after which docker commands should just work.

eval $(minikube docker-env)

If you add this to your ~/.bashrc or ~/.bash_profile, it will automatically setup the docker environment variables for you when minikube is running, in new terminal sessions.

minikube status > /dev/null && eval $(minikube docker-env)

Something important to know, when using --volume and --port it will be applied to the Hyperkit VM, not localhost like with Docker Desktop.

You can get the minikube ip with:

$ minikube ip

As a pro-tip if you need to just curl something, you can do this:

$ docker run -d -p 8000:80 httpd
$ curl http://$(minikube ip):8000

Accessing the Minikube VM

When using the hyperkit driver, you can access the VM if necessary.

The simple way.

$ minikube ssh

The verbose way, in case you need to login manually or use sftp or something.

$ ssh -i ~/.minikube/machines/minikube/id_rsa docker@$(minikube ip)

Cluster Switching

The drop down menu in Docker Desktop is a very convenient tool for switching between clusters. The same thing can be done with kubectl directly though.

$ kubectl config get-contexts
$ kubectl config use-context CLUSTERNAME

Issues After Switching

Homebrew Cleanup Error

When I went to install something, homebrew attempted to do brew cleanup after 30 days. It presented an error.

[protosam@localhost]$ brew install openshift-cli
...
==> `brew cleanup` has not been run in the last 30 days, running now...
Removing: /Users/pilot/Library/Caches/Homebrew/bash--5.1.8... (3.0MB)
Removing: /Users/pilot/Library/Caches/Homebrew/bdw-gc--8.0.4_2... (513.9KB)
Removing: /usr/local/Cellar/c-ares/1.17.1... (85 files, 672.5KB)
Removing: /Users/pilot/Library/Caches/Homebrew/devspace--5.14.4... (14.7MB)
...
Removing: /Users/pilot/Library/Logs/Homebrew/lzo... (64B)
Removing: /Users/pilot/Library/Logs/Homebrew/python@3.9... (2 files, 4.3KB)
Removing: /Users/pilot/Library/Logs/Homebrew/gnutls... (64B)
Error: Permission denied @ apply2files - /usr/local/lib/docker/cli-plugins

The problem was that it couldn't reach a symlinked directory. The directory /usr/local/lib/docker/cli-plugins was pointing to /Applications/Docker.app/Contents/Resources/cli-plugins.

Deleting /usr/local/lib/docker solved the issue.

$ sudo rm -rf /usr/local/lib/docker
@protosam
Copy link
Author

@mteodori you are correct.

Didn't notice that since I type it in everytime. lol

Updated the gist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment