Skip to content

Instantly share code, notes, and snippets.

@kevin-smets
Last active January 6, 2024 22:04
Show Gist options
  • Save kevin-smets/b91a34cea662d0c523968472a81788f7 to your computer and use it in GitHub Desktop.
Save kevin-smets/b91a34cea662d0c523968472a81788f7 to your computer and use it in GitHub Desktop.
Local Kubernetes setup on macOS with minikube on VirtualBox and local Docker registry

Requirements

Minikube requires that VT-x/AMD-v virtualization is enabled in BIOS. To check that this is enabled on OSX / macOS run:

sysctl -a | grep machdep.cpu.features | grep VMX

If there's output, you're good!

Prerequisites

  • kubectl
  • docker (for Mac)
  • minikube
  • virtualbox
brew update && brew install kubectl && brew install --cask docker virtualbox && brew install minikube

Verify

docker --version                # Docker version 17.09.0-ce, build afdb6d4
docker-compose --version        # docker-compose version 1.16.1, build 6d1ac21
docker-machine --version        # docker-machine version 0.12.2, build 9371605
minikube version                # minikube version: v0.22.3
kubectl version --client        # Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.1", GitCommit:"f38e43b221d08850172a9a4ea785a86a3ffa3b3a", GitTreeState:"clean", BuildDate:"2017-10-12T00:45:05Z", GoVersion:"go1.9.1", Compiler:"gc", Platform:"darwin/amd64"}      

Start

minikube start

This can take a while, expected output:

Starting local Kubernetes cluster...
Kubectl is now configured to use the cluster.

Great! You now have a running Kubernetes cluster locally. Minikube started a virtual machine for you, and a Kubernetes cluster is now running in that VM.

Check k8s

kubectl get nodes

Should output something like:

NAME       STATUS    ROLES     AGE       VERSION
minikube   Ready     <none>    40s       v1.7.5

Use minikube's built-in docker daemon:

eval $(minikube docker-env)

Add this line to .bash_profile or .zshrc or ... if you want to use minikube's daemon by default (or if you do not want to set this every time you open a new terminal).

You can revert back to the host docker daemon by running:

eval $(docker-machine env -u)

If you now run docker ps, it should now output something like:

CONTAINER ID        IMAGE                                         COMMAND                 CREATED             STATUS              PORTS               NAMES
e97128790bf9        gcr.io/google-containers/kube-addon-manager   "/opt/kube-addons.sh"   22 seconds ago      Up 22 seconds                           k8s_kube-addon-manager_kube-addon-manager-minikube_kube-system_c654b2f084cf26941c334a2c3d6db53d_0
69707e54d1d0        gcr.io/google_containers/pause-amd64:3.0      "/pause"                33 seconds ago      Up 33 seconds                           k8s_POD_kube-addon-manager-minikube_kube-system_c654b2f084cf26941c334a2c3d6db53d_0

Build, deploy and run an image on your local k8s setup

First setup a local registry, so Kubernetes can pull the image(s) from there:

docker run -d -p 5000:5000 --restart=always --name registry registry:2

Build

First of, store all files (Dockerfile, my-app.yml, index.html) in this gist locally in some new (empty) directory.

You can build the Dockerfile below locally if you want to follow this guide to the letter. Store the Dockerfile locally, preferably in an empty directory and run:

docker build . --tag my-app

You should now have an image named 'my-app' locally, check by using docker images (or your own image of course). You can then publish it to your local docker registry:

docker tag my-app localhost:5000/my-app:0.1.0

Running docker images should now output the following:

REPOSITORY                                             TAG                 IMAGE ID            CREATED             SIZE
my-app                                                 latest              cc949ad8c8d3        44 seconds ago      89.3MB
localhost:5000/my-app                                  0.1.0               cc949ad8c8d3        44 seconds ago      89.3MB
httpd                                                  2.4-alpine          fe26194c0b94        7 days ago          89.3MB

Deploy and run

Store the file below my-app.yml on your system and run the following:

kubectl create -f my-app.yml

You should now see your pod and your service:

kubectl get all

The configuration exposes my-app outside of the cluster, you can get the address to access it by running:

minikube service my-app --url

This should give an output like http://192.168.99.100:30304 (the port will most likely differ). Go there with your favorite browser, you should see "Hello world!". You just accessed your application from outside of your local Kubernetes cluster!

Kubernetes GUI

minikube dashboard

Delete deployment of my-app

kubectl delete deploy my-app
kubectl delete service my-app

You're now good to go and deploy other images!

Reset everything

minikube stop;
minikube delete;
rm -rf ~/.minikube ~/.kube;
brew uninstall kubectl;
brew cask uninstall docker virtualbox minikube;

Version

Last tested on 2017 October 20th macOS Sierra 10.12.6

# Just for demo purposes obviously
FROM httpd:2.4-alpine
COPY ./index.html /usr/local/apache2/htdocs/
Hello world!
# APP DEPLOYMENT
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
run: my-app
name: my-app
spec:
replicas: 1
selector:
matchLabels:
run: my-app-exposed
template:
metadata:
labels:
run: my-app-exposed
spec:
containers:
- image: localhost:5000/my-app:0.1.0
name: my-app
ports:
- containerPort: 80
protocol: TCP
---
# APP SERVICE
apiVersion: v1
kind: Service
metadata:
labels:
run: my-app
name: my-app
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: my-app-exposed
type: NodePort
@vishwac09
Copy link

Good one thanks

@michaelahlers
Copy link

If you're using this and minikube hangs at Starting cluster components..., you might be running into kubernetes/minikube#2765 (see also: kubernetes/minikube#2791). A workaround is to use the localkube bootstrapper by starting with minikube start --bootstrapper=localkube.

@nicroto
Copy link

nicroto commented Jun 1, 2018

@kevin-smets Such a nice gist! Thanks a lot.

@inevity
Copy link

inevity commented Jun 13, 2018

@michaelahlers I use the updatedest minikube , when use proxy,no longer hang with Starting cluster components.
no_proxy=192.168.99.100 https_proxy='http://:8080' minikube start --docker-env 'HTTP_PROXY=http://:8123' --docker-env 'HTTPS_PROXY=http://:8123'
First no any proxy,hang,then only https_proxy='http://:8080' minikube start,hang, then as above ,it work.

@capitalterefe
Copy link

Runeval "$(docker-machine env -u)"to go back to your host docker daemon

@17twenty
Copy link

Yeah - i spent a while without realising the eval $(minikube docker-env) needs to be run in any shell you're planning on using to interact with docker.

I was publishing to Docker Hub from the wrong console window and obviously my registry was running in minikube under different export values.

@kevin-smets
Copy link
Author

@17twenty good tip, I added a line about making this permanent if you want to do prevent from running into this.

@renepardon
Copy link

You should also add docker-compose to the install command because this is not installed automatically with docker.

@pyaesone17
Copy link

You are my hero

@onpaws
Copy link

onpaws commented Oct 8, 2018

According to this SO answer you may not necessarily need to start a local registry with newer versions of the software...

@johntellsall
Copy link

typo. In the "Reset Everything" section, it should be:

rm -rf ~/.minikube ~/.kube;

@kevin-smets
Copy link
Author

Nice catch @johntellsall - fixed it!

@anhtv08
Copy link

anhtv08 commented Dec 12, 2018

Hi,

Below are my configuration version:

docker-compose version 1.22.0, build f46880f
docker-machine version 0.15.0, build b48dc28d
docker-machine version 0.15.0, build b48dc28d
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.0", GitCommit:"ddf47ac13c1a9483ea035a79cd7c10005ff21a6d", GitTreeState:"clean", BuildDate:"2018-12-04T07:48:45Z", GoVersion:"go1.11.2", Compiler:"gc", `Platform:"darwin/amd64"}

I got following error when I start minikube.

E1212 08:21:57.797601 24610 start.go:281] Error restarting cluster: running cmd:
sudo kubeadm alpha phase certs all --config /var/lib/kubeadm.yaml &&
sudo /usr/bin/kubeadm alpha phase kubeconfig all --config /var/lib/kubeadm.yaml &&
sudo /usr/bin/kubeadm alpha phase controlplane all --config /var/lib/kubeadm.yaml &&
sudo /usr/bin/kubeadm alpha phase etcd local --config /var/lib/kubeadm.yaml
: Process exited with status 1

Copy link

ghost commented Jan 11, 2019

minikube stop;
minikube delete;
rm -rf ~/.minikube ~/.kube;

In some cases, with VBox manager installed, doing this can leave you with a VM in VirtualBox with no associated disk.
Attempting to restart minikube after will result in a vm failure.

[Workspace] $ minikube start
Starting local Kubernetes v1.12.4 cluster...
Starting VM...
E0111 09:54:45.302821   74151 start.go:187] Error starting host: Error starting stopped host: Unable to start the VM: /usr/local/bin/VBoxManage startvm minikube --type headless failed:
VBoxManage: error: UUID {4f939cca-1d1f-467a-9ad9-8c13b3847c47} of the medium '.../.minikube/machines/minikube/disk.vmdk' does not match the value {d4075459-35be-4715-a606-2c2fdc9d38ec} stored in the media registry ('/Users/.../Library/VirtualBox/VirtualBox.xml')
VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component MediumWrap, interface IMedium

Details: 00:00:00.052189 Power up failed (vrc=VINF_SUCCESS, rc=NS_ERROR_FAILURE (0X80004005)).

You can remove the orphans from the the Vbox Manager GUI - or use VBoxManage unregistervm <vmname or guid> to clean these up if you have already run the stop/rm commands.

Make sure all your components are up to date. Running minikube delete should cleanly remove all components.

@aesteve
Copy link

aesteve commented Jan 25, 2019

Isn't it missing a docker push after docker tag ?

Maybe I'm mistaking, but after tagging the image as localhost:5000, I need to push it to the localhost:5000 registry for it to be available in minikube :)

Thanks a lot for your gist ! Full of useful info. Great job.

@viggy28
Copy link

viggy28 commented Feb 25, 2019

Thanks so much.

@peterstouw
Copy link

Thank you, nice manual.
Works also on macOS High Sierra 10.13.6

@MihaiLupoiu
Copy link

Hi!
I just realise that you don't actually push the image to the registry. It works because you are still in the docker environment inside minikube.

If you do the docker push: docker push localhost:5000/my-app:0.1.0 and after that you do: curl http://localhost:5000/v2/_catalog you should see the image:
{"repositories":["my-app"]}

Or if you execute curl http://localhost:5000/v2/my-app/tags/list should show:
{"name":"my-app","tags":["0.1.0"]}

This is just a clarification. Thanks again for the gist! Very useful.

@vitor-diego-s
Copy link

Nice guide!
Thank you!

@bravoecho
Copy link

bravoecho commented Jul 12, 2019

Isn't it missing a docker push after docker tag ?

Maybe I'm mistaking, but after tagging the image as localhost:5000, I need to push it to the localhost:5000 registry for it to be available in minikube :)

@aesteve This is what I'm finding too. I have to push with docker push localhost:5000/my-app:0.1.0 or I will still get ErrImagePull.

@MihaiLupoiu just to make sure that I understand your comment: does it work for you without pushing?

@gaggle
Copy link

gaggle commented Aug 5, 2019

I got errors ala invalid object doesn't have additional properties when applying kubectl create -f my-app.yml. I got it resolved it by following this answer: https://stackoverflow.com/questions/55417410/kubernetes-create-deployment-unexpected-schemaerror/55737973#55737973

Thanks for this gist.

@dennis-eichardt-spring
Copy link

Wow, you rock! You maybe saved me a very massive amount of time! There are sooooo many bad tutorials out there! :(

@ckumar87
Copy link

Thank you, You have made my day! 🙏

@maheshavadhani
Copy link

Hi,

Few small changes required in .yml file for latest vesion of kubernetes (1.16)

apiVersion: apps/v1
kind: Deployment

@naviat
Copy link

naviat commented Mar 18, 2020

Update: Minikube is not delivered with cask right now. Change command brew cask install minikube to brew install minikube

@kevin-smets
Copy link
Author

Done, thanks for the heads up!

@cijalba
Copy link

cijalba commented Apr 20, 2021

Another update: brew cask has been deprecated. so the cmd should be something similar to:

brew update && brew install kubectl && brew install docker virtualbox && brew install minikube

or with docker-compose and machine if it's also needed:

brew update && brew install kubectl && brew install docker docker-compose docker-machine virtualbox && brew install minikube

@ajaychinthapalli
Copy link

Thanks for this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment