Instantly share code, notes, and snippets.

Embed
What would you like to do?
Local Kubernetes setup on macOS with minikube on VirtualBox and local Docker registry

Requirements

Minikube requires that VT-x/AMD-v virtualization is enabled in BIOS. To check that this is enabled on OSX / macOS run:

sysctl -a | grep machdep.cpu.features | grep VMX

If there's output, you're good!

Prerequisites

  • kubectl
  • docker (for Mac)
  • minikube
  • virtualbox
brew update && brew install kubectl && brew cask install docker minikube virtualbox

Verify

docker --version                # Docker version 17.09.0-ce, build afdb6d4
docker-compose --version        # docker-compose version 1.16.1, build 6d1ac21
docker-machine --version        # docker-machine version 0.12.2, build 9371605
minikube version                # minikube version: v0.22.3
kubectl version --client        # Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.1", GitCommit:"f38e43b221d08850172a9a4ea785a86a3ffa3b3a", GitTreeState:"clean", BuildDate:"2017-10-12T00:45:05Z", GoVersion:"go1.9.1", Compiler:"gc", Platform:"darwin/amd64"}      

Start

minikube start

This can take a while, expected output:

Starting local Kubernetes cluster...
Kubectl is now configured to use the cluster.

Great! You now have a running Kubernetes cluster locally. Minikube started a virtual machine for you, and a Kubernetes cluster is now running in that VM.

Check k8s

kubectl get nodes

Should output something like:

NAME       STATUS    ROLES     AGE       VERSION
minikube   Ready     <none>    40s       v1.7.5

Use minikube's built-in docker daemon:

eval $(minikube docker-env)

Add this line to .bash_profile or .zshrc or ... if you want to use minikube's daemon by default (or if you do not want to set this every time you open a new terminal).

You can revert back to the host docker daemon by running:

eval $(docker-machine env -u)

If you now run docker ps, it should now output something like:

CONTAINER ID        IMAGE                                         COMMAND                 CREATED             STATUS              PORTS               NAMES
e97128790bf9        gcr.io/google-containers/kube-addon-manager   "/opt/kube-addons.sh"   22 seconds ago      Up 22 seconds                           k8s_kube-addon-manager_kube-addon-manager-minikube_kube-system_c654b2f084cf26941c334a2c3d6db53d_0
69707e54d1d0        gcr.io/google_containers/pause-amd64:3.0      "/pause"                33 seconds ago      Up 33 seconds                           k8s_POD_kube-addon-manager-minikube_kube-system_c654b2f084cf26941c334a2c3d6db53d_0

Build, deploy and run an image on your local k8s setup

First setup a local registry, so Kubernetes can pull the image(s) from there:

docker run -d -p 5000:5000 --restart=always --name registry registry:2

Build

First of, store all files (Dockerfile, my-app.yml, index.html) in this gist locally in some new (empty) directory.

You can build the Dockerfile below locally if you want to follow this guide to the letter. Store the Dockerfile locally, preferably in an empty directory and run:

docker build . --tag my-app

You should now have an image named 'my-app' locally, check by using docker images (or your own image of course). You can then publish it to your local docker registry:

docker tag my-app localhost:5000/my-app:0.1.0

Running docker images should now output the following:

REPOSITORY                                             TAG                 IMAGE ID            CREATED             SIZE
my-app                                                 latest              cc949ad8c8d3        44 seconds ago      89.3MB
localhost:5000/my-app                                  0.1.0               cc949ad8c8d3        44 seconds ago      89.3MB
httpd                                                  2.4-alpine          fe26194c0b94        7 days ago          89.3MB

Deploy and run

Store the file below my-app.yml on your system and run the following:

kubectl create -f my-app.yml

You should now see your pod and your service:

kubectl get all

The configuration exposes my-app outside of the cluster, you can get the address to access it by running:

minikube service my-app --url

This should give an output like http://192.168.99.100:30304 (the port will most likely differ). Go there with your favorite browser, you should see "Hello world!". You just accessed your application from outside of your local Kubernetes cluster!

Kubernetes GUI

minikube dashboard

Delete deployment of my-app

kubectl delete deploy my-app
kubectl delete service my-app

You're now good to go and deploy other images!

Reset everything

minikube stop;
minikube delete;
rm -rf ~/.minikube ~/.kube;
brew uninstall kubectl;
brew cask uninstall docker virtualbox minikube;

Version

Last tested on 2017 October 20th macOS Sierra 10.12.6

# Just for demo purposes obviously
FROM httpd:2.4-alpine
COPY ./index.html /usr/local/apache2/htdocs/
Hello world!
# APP DEPLOYMENT
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
run: my-app
name: my-app
spec:
replicas: 1
selector:
matchLabels:
run: my-app-exposed
template:
metadata:
labels:
run: my-app-exposed
spec:
containers:
- image: localhost:5000/my-app:0.1.0
name: my-app
ports:
- containerPort: 80
protocol: TCP
---
# APP SERVICE
apiVersion: v1
kind: Service
metadata:
labels:
run: my-app
name: my-app
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: my-app-exposed
type: NodePort
@udev

This comment has been minimized.

udev commented Jul 12, 2017

This is a life saver. Thank you!

@Paxa

This comment has been minimized.

Paxa commented Jul 14, 2017

minikube dashboard

Open kubernetes UI in default browser

@kevin-smets

This comment has been minimized.

Owner

kevin-smets commented Aug 10, 2017

@Paxa thanks! Gave this gist an overhaul to reflect the proper setup for the latest software versions.

@icarus0

This comment has been minimized.

icarus0 commented Aug 11, 2017

thank you for this guide 🏊 🥇

@altfatterz

This comment has been minimized.

altfatterz commented Sep 16, 2017

The deployment is not created:

➜  study-kubernetes kubectl create -f my-app.yml
service "my-app" created
The Deployment "my-app" is invalid: spec.template.metadata.labels: Invalid value: map[string]string{"run":"my-app-exposed"}: `selector` does not match template `labels`

versions:

➜  study-kubernetes minikube version
minikube version: v0.22.1
➜  study-kubernetes kubectl version
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.5", GitCommit:"17d7182a7ccbb167074be7a87f0a68bd00d58d97", GitTreeState:"clean", BuildDate:"2017-08-31T19:32:26Z", GoVersion:"go1.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.5", GitCommit:"17d7182a7ccbb167074be7a87f0a68bd00d58d97", GitTreeState:"clean", BuildDate:"2017-09-11T21:52:19Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
➜  study-kubernetes docker version
Client:
 Version:      17.06.2-ce
 API version:  1.23
 Go version:   go1.8.3
 Git commit:   cec0b72
 Built:        Tue Sep  5 20:12:06 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      1.12.6
 API version:  1.24 (minimum version )
 Go version:   go1.6.4
 Git commit:   78d1802
 Built:        Wed Jan 11 00:23:16 2017
 OS/Arch:      linux/amd64
 Experimental: false
@riteshn-niara

This comment has been minimized.

riteshn-niara commented Oct 4, 2017

I get:

error validating data: unknown object type schema.GroupVersionKind{Group:"extensions", Version:"v1beta1", Kind:"Deployment"}; if you choose to ignore these errors, turn validation off with --validate=false

versions:

cobra-~/src/golang/src/event-gateway git:(master) ✗: minikube version
minikube version: v0.22.2
cobra-~/src/golang/src/event-gateway git:(master) git:(master) ✗: kubectl version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0", GitCommit:"0b9efaeb34a2fc51ff8e4d34ad9bc6375459c4a4", GitTreeState:"clean", BuildDate:"2017-09-29T05:56:06Z", GoVersion:"go1.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.5", GitCommit:"17d7182a7ccbb167074be7a87f0a68bd00d58d97", GitTreeState:"clean", BuildDate:"2017-09-18T20:30:29Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
cobra-~/src/golang/src/event-gateway git:(master) ✗: docker version
Client:
 Version:      17.06.2-ce
 API version:  1.23
 Go version:   go1.8.3
 Git commit:   cec0b72
 Built:        Tue Sep  5 20:12:06 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      1.12.6
 API version:  1.24 (minimum version )
 Go version:   go1.6.4
 Git commit:   78d1802
 Built:        Wed Jan 11 00:23:16 2017
 OS/Arch:      linux/amd64
 Experimental: false
@rahulyennawar21

This comment has been minimized.

rahulyennawar21 commented Oct 8, 2017

I get:

$ kubectl create -f my-app.yml
service "my-app" created
The Deployment "my-app" is invalid: spec.template.metadata.labels: Invalid value: map[string]string{"run":"my-app-exposed"}: selector does not match template labels
$

@k-oguma

This comment has been minimized.

k-oguma commented Oct 12, 2017

@altfatterz @rahulyennawar21

I encountered the same error.

% kubectl create -f my-app.yml
service "my-app" created
The Deployment "my-app" is invalid: spec.template.metadata.labels: Invalid value: map[string]string{"run":"my-app-exposed"}: `selector` does not match template `labels`

solution

I edited.

% git diff --staged my-app.yml
diff --git a/my-app.yml b/my-app.yml
index 4dff9ab..1c1157c 100644
--- a/my-app.yml
+++ b/my-app.yml
@@ -4,13 +4,13 @@ apiVersion: extensions/v1beta1
 kind: Deployment
 metadata:
   labels:
-    run: my-app
+    run: my-app-exposed
   name: my-app
 spec:
   replicas: 1
   selector:
     matchLabels:
-      run: my-app
+      run: my-app-exposed
   template:
     metadata:
       labels:
@@ -31,7 +31,7 @@ apiVersion: v1
 kind: Service
 metadata:
   labels:
-    run: my-app
+    run: my-app-exposed
   name: my-app
 spec:
   ports:
@@ -40,4 +40,4 @@ spec:
     targetPort: 80
   selector:
     run: my-app-exposed # must match the label of the pod, otherwise it will not be exposed
-  type: NodePort
\ No newline at end of file
+  type: NodePort

retry

% kubectl delete deploy my-app
kubectl delete service my-app
deployment "my-app" deleted
service "my-app" deleted

% kubectl create -f my-app.yml
deployment "my-app" created
service "my-app" created

% kubectl get deployment
Successfully built 45e0639bec5c
NAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
my-app    1         1         1            1           3m

% kubectl get service
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP        2h
my-app       NodePort    10.0.0.187   <none>        80:31596/TCP   3m
@chrisalexander55

This comment has been minimized.

chrisalexander55 commented Oct 17, 2017

Thank you!

@kevin-smets

This comment has been minimized.

Owner

kevin-smets commented Oct 20, 2017

Sorry, the yaml was indeed not properly formatted. Went through it again end to end and now it should work again as advertised :). Also added a "Hello world!" example to properly showcase the external access.

@hannes-angst

This comment has been minimized.

hannes-angst commented Oct 22, 2017

$kubectl create -f my-app.yml 
deployment "my-app" created
service "my-app" created
$ kubectl get all  
NAME            DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/my-app   1         1         1            0           4s

NAME                   DESIRED   CURRENT   READY     AGE
rs/my-app-1143846683   1         1         0         4s
 
NAME            DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/my-app   1         1         1            0           4s

NAME                         READY     STATUS         RESTARTS   AGE
po/my-app-1143846683-1157m   0/1       ErrImagePull   0          4s

From the pod logs:

Failed to pull image "localhost:5000/my-app:0.0.1": rpc error: code = 2 desc = Error: image my-app:0.0.1 not found Error syncing pod
@hannes-angst

This comment has been minimized.

hannes-angst commented Oct 22, 2017

Please change the version in the yml file to 0.1.0 to make it work ;)

spec:
  containers:
  - image: localhost:5000/my-app:0.1.0
@kevin-smets

This comment has been minimized.

Owner

kevin-smets commented Oct 23, 2017

@hannes-angst done, thanks for the heads up! Damn you copy paste :).

@ryan-blunden

This comment has been minimized.

ryan-blunden commented Jan 15, 2018

Thanks for this! It's awesome.

Heads up that xhyve support has been deprecated in minikube officially so you can remove that from your TODO.

@kevin-smets

This comment has been minimized.

Owner

kevin-smets commented Jan 22, 2018

Done, thanks for the heads up @ryan-blunden

@maz100

This comment has been minimized.

maz100 commented Feb 8, 2018

Thanks, this helped me get minikube working first time.

@lon-io

This comment has been minimized.

lon-io commented Mar 6, 2018

Don't forget to run:
eval "$(docker-machine env -u)"
to undo https://gist.github.com/kevin-smets/b91a34cea662d0c523968472a81788f7#use-minikubes-built-in-docker-daemon and
return to your default Docker Daemon

@dchangtech

This comment has been minimized.

dchangtech commented Mar 15, 2018

Good job! thank you very much

@vishwac09

This comment has been minimized.

vishwac09 commented Apr 27, 2018

Good one thanks

@michaelahlers

This comment has been minimized.

michaelahlers commented May 30, 2018

If you're using this and minikube hangs at Starting cluster components..., you might be running into kubernetes/minikube#2765 (see also: kubernetes/minikube#2791). A workaround is to use the localkube bootstrapper by starting with minikube start --bootstrapper=localkube.

@nicroto

This comment has been minimized.

nicroto commented Jun 1, 2018

@kevin-smets Such a nice gist! Thanks a lot.

@inevity

This comment has been minimized.

inevity commented Jun 13, 2018

@michaelahlers I use the updatedest minikube , when use proxy,no longer hang with Starting cluster components.
no_proxy=192.168.99.100 https_proxy='http://:8080' minikube start --docker-env 'HTTP_PROXY=http://:8123' --docker-env 'HTTPS_PROXY=http://:8123'
First no any proxy,hang,then only https_proxy='http://:8080' minikube start,hang, then as above ,it work.

@capitalterefe

This comment has been minimized.

capitalterefe commented Jun 16, 2018

Runeval "$(docker-machine env -u)"to go back to your host docker daemon

@17twenty

This comment has been minimized.

17twenty commented Jun 18, 2018

Yeah - i spent a while without realising the eval $(minikube docker-env) needs to be run in any shell you're planning on using to interact with docker.

I was publishing to Docker Hub from the wrong console window and obviously my registry was running in minikube under different export values.

@kevin-smets

This comment has been minimized.

Owner

kevin-smets commented Jun 21, 2018

@17twenty good tip, I added a line about making this permanent if you want to do prevent from running into this.

@renepardon

This comment has been minimized.

renepardon commented Jul 2, 2018

You should also add docker-compose to the install command because this is not installed automatically with docker.

@pyaesone17

This comment has been minimized.

pyaesone17 commented Oct 4, 2018

You are my hero

@onpaws

This comment has been minimized.

onpaws commented Oct 8, 2018

According to this SO answer you may not necessarily need to start a local registry with newer versions of the software...

@johntellsall

This comment has been minimized.

johntellsall commented Nov 21, 2018

typo. In the "Reset Everything" section, it should be:

rm -rf ~/.minikube ~/.kube;

@kevin-smets

This comment has been minimized.

Owner

kevin-smets commented Nov 27, 2018

Nice catch @johntellsall - fixed it!

@anhtv08

This comment has been minimized.

anhtv08 commented Dec 12, 2018

Hi,

Below are my configuration version:

docker-compose version 1.22.0, build f46880f
docker-machine version 0.15.0, build b48dc28d
docker-machine version 0.15.0, build b48dc28d
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.0", GitCommit:"ddf47ac13c1a9483ea035a79cd7c10005ff21a6d", GitTreeState:"clean", BuildDate:"2018-12-04T07:48:45Z", GoVersion:"go1.11.2", Compiler:"gc", `Platform:"darwin/amd64"}

I got following error when I start minikube.

E1212 08:21:57.797601 24610 start.go:281] Error restarting cluster: running cmd:
sudo kubeadm alpha phase certs all --config /var/lib/kubeadm.yaml &&
sudo /usr/bin/kubeadm alpha phase kubeconfig all --config /var/lib/kubeadm.yaml &&
sudo /usr/bin/kubeadm alpha phase controlplane all --config /var/lib/kubeadm.yaml &&
sudo /usr/bin/kubeadm alpha phase etcd local --config /var/lib/kubeadm.yaml
: Process exited with status 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment