Skip to content

Instantly share code, notes, and snippets.

@thcipriani

thcipriani/helmcinotes.org Secret

Last active April 5, 2018 17:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save thcipriani/dc79855487ae36040c70c28d97a288f0 to your computer and use it in GitHub Desktop.
Save thcipriani/dc79855487ae36040c70c28d97a288f0 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
helmcinotes.org

Helm

Minikube

Get your sudoers straight

Could probably just add jenkins-deploy to docker group? Dunno, this may be safer

jenkins-deploy ALL=(ALL) NOPASSWD: SETENV: /usr/bin/minikube*, /usr/bin/docker build*

Instructions

start minikube, init helm/tiller

sudo apt-get install -y helm minikube kubernetes-client socat

export MINIKUBE_WANTUPDATENOTIFICATION=false
export MINIKUBE_WANTREPORTERRORPROMPT=false
export MINIKUBE_HOME=$HOME
export CHANGE_MINIKUBE_NONE_USER=true
mkdir $HOME/.kube || true
touch $HOME/.kube/config
export KUBECONFIG=$HOME/.kube/config

sudo -E minikube start --vm-driver none --bootstrapper=localkube
helm init --tiller-image=gcr.io/kubernetes-helm/tiller:v2.8.1

get/build sources

Build a mathoid docker image, download deploy charts

git clone https://gerrit.wikimedia.org/r/operations/deployment-charts
git clone https://gerrit.wikimedia.org/r/mediawiki/services/mathoid

cd mathoid
blubber dist/pipeline/blubber.yaml production | sudo docker build -t mathoid -f - .
helm init --tiller-image=gcr.io/kubernetes-helm/tiller:v2.8.1

created override.yaml

to override deployment chart stuff

namespace: default                                                                                                                                    
docker:
  registry: 'test'
  pull_policy: Never
main_app:
  image: mathoid

Tag existing images

Since you’re overriding the docker registry name to =’test’= in override.yaml, you need to retag some things…

docker tag mathoid test/mathoid
docker pull docker-registry.wikimedia.org/prometheus-statsd-exporter:latest
docker tag docker-registry.wikimedia.org/prometheus-statsd-exporter:latest test/prometheus-statsd-exporter:latest

Working

helm install -f override.yaml deployment-charts/charts/mathoid
jenkins-deploy@integration-slave-k8s-10:~$ kubectl get svc                                                                                          
NAME                     CLUSTER-IP    EXTERNAL-IP   PORT(S)           AGE                                                                            
kubernetes               10.96.0.1     <none>        443/TCP           2d
mathoid-youthful-zebra   10.96.203.8   <nodes>       10044:30001/TCP   9m
jenkins-deploy@integration-slave-k8s-1015:~$ curl -d'q=e=mc^2' $(sudo minikube service mathoid-youthful-zebra --url)/svg                              
<svg xmlns:xlink="http://www.w3.org/1999/xlink" width="8.283ex" height="2.676ex" style="vertical-align: -0.338ex;" viewBox="0 -1006.6 3566.5 1152.1" role="img" focusable="false" xmlns="http://www.w3.org/2000/svg" aria-labelledby="MathJax-SVG-1-Title">
<title id="MathJax-SVG-1-Title">e=mc^2</title>
<defs aria-hidden="true">
<path stroke-width="1" id="E1-MJMATHI-65" d="M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z"></path>
<path stroke-width="1" id="E1-MJMAIN-3D" d="M56 347Q56 360 70 367H707Q722 359 722 347Q722 336 708 328L390 327H72Q56 332 56 347ZM56 153Q56 168 72 173H708Q722 163 722 153Q722 140 707 133H70Q56 140 56 153Z"></path>
<path stroke-width="1" id="E1-MJMATHI-6D" d="M21 287Q22 293 24 303T36 341T56 388T88 425T132 442T175 435T205 417T221 395T229 376L231 369Q231 367 232 367L243 378Q303 442 384 442Q401 442 415 440T441 433T460 423T475 411T485 398T493 385T497 373T500 364T502 357L510 367Q573 442 659 442Q713 442 746 415T780 336Q780 285 742 178T704 50Q705 36 709 31T724 26Q752 26 776 56T815 138Q818 149 821 151T837 153Q857 153 857 145Q857 144 853 130Q845 101 831 73T785 17T716 -10Q669 -10 648 17T627 73Q627 92 663 193T700 345Q700 404 656 404H651Q565 404 506 303L499 291L466 157Q433 26 428 16Q415 -11 385 -11Q372 -11 364 -4T353 8T350 18Q350 29 384 161L420 307Q423 322 423 345Q423 404 379 404H374Q288 404 229 303L222 291L189 157Q156 26 151 16Q138 -11 108 -11Q95 -11 87 -5T76 7T74 17Q74 30 112 181Q151 335 151 342Q154 357 154 369Q154 405 129 405Q107 405 92 377T69 316T57 280Q55 278 41 278H27Q21 284 21 287Z"></path>
<path stroke-width="1" id="E1-MJMATHI-63" d="M34 159Q34 268 120 355T306 442Q362 442 394 418T427 355Q427 326 408 306T360 285Q341 285 330 295T319 325T330 359T352 380T366 386H367Q367 388 361 392T340 400T306 404Q276 404 249 390Q228 381 206 359Q162 315 142 235T121 119Q121 73 147 50Q169 26 205 26H209Q321 26 394 111Q403 121 406 121Q410 121 419 112T429 98T420 83T391 55T346 25T282 0T202 -11Q127 -11 81 37T34 159Z"></path>
<path stroke-width="1" id="E1-MJMAIN-32" d="M109 429Q82 429 66 447T50 491Q50 562 103 614T235 666Q326 666 387 610T449 465Q449 422 429 383T381 315T301 241Q265 210 201 149L142 93L218 92Q375 92 385 97Q392 99 409 186V189H449V186Q448 183 436 95T421 3V0H50V19V31Q50 38 56 46T86 81Q115 113 136 137Q145 147 170 174T204 211T233 244T261 278T284 308T305 340T320 369T333 401T340 431T343 464Q343 527 309 573T212 619Q179 619 154 602T119 569T109 550Q109 549 114 549Q132 549 151 535T170 489Q170 464 154 447T109 429Z"></path>
</defs>
<g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)" aria-hidden="true">
 <use xlink:href="#E1-MJMATHI-65" x="0" y="0"></use>
 <use xlink:href="#E1-MJMAIN-3D" x="744" y="0"></use>
 <use xlink:href="#E1-MJMATHI-6D" x="1800" y="0"></use>
<g transform="translate(2679,0)">
 <use xlink:href="#E1-MJMATHI-63" x="0" y="0"></use>
 <use transform="scale(0.707)" xlink:href="#E1-MJMAIN-32" x="613" y="583"></use>
</g>
</g>
</svg>

Helm test

First we have to get the service-checker image and since we overwrote the registry when we did helm install we have to retag it:

sudo docker pull docker-registry.wikimedia.org/service-checker:latest
sudo docker tag docker-registry.wikimedia.org/service-checker:latest test/service-checker:latest

Then we can test the helm deployment

jenkins-deploy@integration-slave-k8s-1015:~$ helm list                                                                                                
NAME            REVISION        UPDATED                         STATUS          CHART           NAMESPACE                                             
youthful-zebra  1               Thu Apr  5 17:30:22 2018        DEPLOYED        mathoid-0.0.2   default  
jenkins-deploy@integration-slave-k8s-1015:~$ helm test youthful-zebra
RUNNING: mathoid-youthful-zebra-service-checker
PASSED: mathoid-youthful-zebra-service-checker
jenkins-deploy@integration-slave-k8s-1015:~$ echo $?
0

Random Minikube/Helm/Tiller notes

Deleting tiller deploy (if necessary)

Tiller is in another namespace, so to stop it

kubectl get namespaces
kubectl describe deployment tiller-deploy --namespace=kube-system
kubectl delete deployment tiller-deploy --namespace=kube-system

Setup

You have minikube installed somehow.

Check minikube’s status:

minikube status

If that gives you some noise like:

There is a newer version of minikube available (v0.22.2). Download it here: https://github.com/kubernetes/minikube/releases/tag/v0.22.2

To disable this notification, run the following: minikube config set WantUpdateNotification false minikubeVM: Stopped localkube: N/A

maybe just run start…or update-check then if there is a newer version: Update it via curl…because fucking of course.

Make a docker container to run on minikube

Ensure that you are using the minikube docker context:

$ eval $(minikube docker-env)

Now you can build an image via docker and it will build in the minikube context:

$ docker build -t thing -f Dockerfile.thing .

Deploy newly built image on minikube

Create deployment

$ kubectl run --image-pull-policy=Never --image=thing thing-deploy --port=8080

The --image-pull-policy=never is important, otherwise it will try (and fail) to pull the thing image and you’ll see ErrImagePull when you do kubectl get pods -l run=thing

Check deployment

$ kubectl get deployments
NAME           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE                                                                                                               
thing-deploy   1         1         1            1           26m

Check deployment pods

$ kubectl get pods -l run=thing-deploy
NAME                            READY     STATUS    RESTARTS   AGE                                                                                                            
thing-deploy-57549c74bc-l5wk9   1/1       Running   0          27m

Expose a deployment via a service

$ kubectl expose deployment thing-deploy --type=LoadBalancer
service "thing" exposed
$ kubectl get services
NAME         CLUSTER-IP       EXTERNAL-IP   PORT(S)           AGE
kubernetes   10.96.0.1        <none>        443/TCP           16h
mathoid      10.102.202.109   <pending>     10044:31364/TCP   11s

Now the service is available through

$ minikube service thing --url

Logs are available for each pod through

$ kubectl get pods -l run=thing
NAME                       READY     STATUS    RESTARTS   AGE                                                                                                            
thing-57549c74bc-l5wk9   1/1       Running   0          27m 
$ kubectl log -f thing-57549c74bc-l5wk9

Helm

helm init
helm init --upgrade
kubeadm join --token ea3308.741df36f1a40842f 159.203.118.0:6443 --discovery-token-ca-cert-hash sha256:4c9d2061e640f2b3447f16cde88efb79fdb74caa314ad7d2e604abe3b3e21467
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment