Skip to content

Instantly share code, notes, and snippets.

@danielkec
Last active March 5, 2021 15:57
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save danielkec/c1bb405108a537c908c66978573d66c8 to your computer and use it in GitHub Desktop.
Save danielkec/c1bb405108a537c908c66978573d66c8 to your computer and use it in GitHub Desktop.
verrazzano_microk8s.md

microk8s 1.18

alias kubectl='microk8s.kubectl'

sudo snap install microk8s --classic --channel=1.18/stable
microk8s start

Enable Use ServiceAccountToken volumes linkerd/linkerd2#3260 (comment) kubeflow/manifests#974

sudo echo -e '\n--service-account-issuer=api
--service-account-signing-key-file=${SNAP_DATA}/certs/serviceaccount.key
--service-account-api-audiences=api' >> /var/snap/microk8s/current/args/kube-apiserver
microk8s enable storage metallb
# For use by MetalLB, assign a range of IP addresses at the end of the network’s subnet CIDR range.

microk8s status # check addons are started
microk8s inspect # check api daemon is started

kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"

Verrazzano

https://verrazzano.io/docs/setup/install/installation/#install-the-verrazzano-platform-operator

kubectl apply -f https://github.com/verrazzano/verrazzano/releases/latest/download/operator.yaml
kubectl -n verrazzano-install rollout status deployment/verrazzano-platform-operator
kubectl apply -f - <<EOF
apiVersion: install.verrazzano.io/v1alpha1
kind: Verrazzano
metadata:
  name: example-verrazzano
spec:
  profile: dev
EOF

Check installation logs

kubectl logs -f \
    $( \
      kubectl get pod  \
          -l job-name=verrazzano-install-example-verrazzano \
          -o jsonpath="{.items[0].metadata.name}" \
    )

Deploy example Helidon

kubectl create namespace hello-helidon
kubectl label namespace hello-helidon verrazzano-managed=true
kubectl apply -f https://raw.githubusercontent.com/verrazzano/verrazzano/master/examples/hello-helidon/hello-helidon-comp.yaml
kubectl apply -f https://raw.githubusercontent.com/verrazzano/verrazzano/master/examples/hello-helidon/hello-helidon-app.yaml

Save GW host

HOST=$(kubectl get gateway hello-helidon-hello-helidon-appconf-gw -n hello-helidon -o jsonpath='{.spec.servers[0].hosts[0]}')

Ping helidon

curl -X GET "https://${HOST}/greet"

Clean up k8s

microk8s reset snap remove --purge microk8s

@danielkec
Copy link
Author

danielkec commented Mar 5, 2021

Showstopper:
When using Verrazzano installed DNS istiocoredns its not possible to proceed with installation due an error when installing cert-manager:

root@romulus:/home/kec# kubectl -n cert-manager logs cert-manager-6b9f46d864-f7ww2
panic: runtime error: slice bounds out of range [:9] with length 8

goroutine 1 [running]:
github.com/miekg/dns.ClientConfigFromFile(0x1ce47f6, 0x10, 0x0, 0x0, 0x0)
	external/com_github_miekg_dns/clientconfig.go:86 +0xa02
github.com/jetstack/cert-manager/pkg/issuer/acme/dns/util.getNameservers(0x1ce47f6, 0x10, 0x2ea6800, 0x2, 0x2, 0x206da00, 0x0, 0x25)
	pkg/issuer/acme/dns/util/wait.go:51 +0x4d
github.com/jetstack/cert-manager/pkg/issuer/acme/dns/util.init()
	pkg/issuer/acme/dns/util/wait.go:44 +0x7b

When installing coredns ourselfs, we are able to finish installation, cert-manager is announced as installed successfully:

[2021-03-05 11:14:33 UTC] customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created
[2021-03-05 11:14:33 UTC] Release "cert-manager" does not exist. Installing it now.
[2021-03-05 11:14:44 UTC] NAME: cert-manager
[2021-03-05 11:14:44 UTC] LAST DEPLOYED: Fri Mar  5 11:14:35 2021
[2021-03-05 11:14:44 UTC] NAMESPACE: cert-manager
[2021-03-05 11:14:44 UTC] STATUS: deployed
[2021-03-05 11:14:44 UTC] REVISION: 1
[2021-03-05 11:14:44 UTC] TEST SUITE: None
[2021-03-05 11:14:44 UTC] NOTES:
[2021-03-05 11:14:44 UTC] cert-manager has been deployed successfully!
[2021-03-05 11:14:44 UTC] 
[2021-03-05 11:14:44 UTC] In order to begin issuing certificates, you will need to set up a ClusterIssuer
[2021-03-05 11:14:44 UTC] or Issuer resource (for example, by creating a 'letsencrypt-staging' issuer).

Installation seems to finish successfully then:

[2021-03-05 11:24:01 UTC] Installing MySQL                                                             [  OK  ]
[2021-03-05 11:24:01 UTC] Installing Keycloak                                                          [ .... ]
[2021-03-05 11:24:01 UTC] NAME         TYPE     DATA   AGE
[2021-03-05 11:24:01 UTC] verrazzano   Opaque   4      6m7s
[2021-03-05 11:24:01 UTC] secret/keycloak-realm-cacert created
[2021-03-05 11:24:02 UTC] secret/keycloak-http created
[2021-03-05 11:24:02 UTC] NAME    TYPE     DATA   AGE
[2021-03-05 11:24:02 UTC] mysql   Opaque   2      53s
[2021-03-05 11:24:02 UTC] Release "keycloak" does not exist. Installing it now.
[2021-03-05 11:29:02 UTC] Error: timed out waiting for the condition
[2021-03-05 11:29:02 UTC] error: unable to upgrade connection: container not found ("keycloak")
[2021-03-05 11:29:02 UTC] certificate.cert-manager.io/default-secret condition met
[2021-03-05 11:29:02 UTC] Installing Keycloak                                                          [  OK  ]
[2021-03-05 11:29:02 UTC] 
[2021-03-05 11:29:02 UTC] Installation Complete.
[2021-03-05 11:29:02 UTC] 
[2021-03-05 11:29:02 UTC] Verrazzano provides various user interfaces.
[2021-03-05 11:29:02 UTC] 
[2021-03-05 11:29:02 UTC] Grafana - https://grafana.vmi.system.default.10.1.56.211.xip.io
[2021-03-05 11:29:02 UTC] Prometheus - https://prometheus.vmi.system.default.10.1.56.211.xip.io
[2021-03-05 11:29:02 UTC] Kibana - https://kibana.vmi.system.default.10.1.56.211.xip.io
[2021-03-05 11:29:02 UTC] Elasticsearch - https://elasticsearch.vmi.system.default.10.1.56.211.xip.io
[2021-03-05 11:29:02 UTC] Verrazzano Console - https://verrazzano.default.10.1.56.211.xip.io
[2021-03-05 11:29:02 UTC] 
[2021-03-05 11:29:02 UTC] You will need the credentials to access the preceding user interfaces.  They are all accessed by the same username/password.
[2021-03-05 11:29:02 UTC] User: verrazzano
[2021-03-05 11:29:02 UTC] Password: kubectl get secret --namespace verrazzano-system verrazzano -o jsonpath={.data.password} | base64 --decode; echo
[2021-03-05 11:29:02 UTC] 
[2021-03-05 11:29:02 UTC] Rancher - https://rancher.default.10.1.56.211.xip.io
[2021-03-05 11:29:02 UTC] User: admin
[2021-03-05 11:29:02 UTC] Password: kubectl get secret --namespace cattle-system rancher-admin-secret -o jsonpath={.data.password} | base64 --decode; echo
[2021-03-05 11:29:02 UTC] 
[2021-03-05 11:29:02 UTC] Keycloak - https://keycloak.default.10.1.56.211.xip.io
[2021-03-05 11:29:02 UTC] User: keycloakadmin
[2021-03-05 11:29:02 UTC] Password: kubectl get secret --namespace keycloak keycloak-http -o jsonpath={.data.password} | base64 --decode; echo

But Keycloak is stuck in CrashLoopBackOff because of:

Caused by: java.net.UnknownHostException: mysql: Name or service not known
	at java.base/java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
	at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929)
	at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1515)
	at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848)
	at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
	at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1364)
	at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
	at com.mysql.jdbc@8.0.19//com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:132)
	at com.mysql.jdbc@8.0.19//com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:65)
	... 64 more

Installation of Helidon example is successful then but it is not accessible over the hello-helidon-hello-helidon-appconf-gw but only directly.

@danielkec
Copy link
Author

danielkec commented Mar 5, 2021

curl -v $(kubectl -n hello-helidon get endpoints -o jsonpath='{$..subsets..ip}:{$..subsets..port}/greet')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment