tectonic installer notes

tectonic-installer-notes.txt

instances start with rhcos image + ignition configs (per role) generated by "tectonic init"
for aws: user_data
for libvirt: qemu_fw_cfg

== master ===
kubelet.service starts
runs bootstrap control plane runs as static pods on the bootstrap master
bootkube.service starts
runs kube-core-renderer container
https://github.com/coreos-inc/tectonic-operators/blob/master/renderer/Documentation/README.md
reads kco-config.yaml (kind: KubeCoreOperatorConfig) and populates /opt/tectonic ???
runs tectonic-node-controller-operator container
https://github.com/coreos-inc/tectonic-operators/blob/master/operator/node-controller/README.md
reads tnco-config.yaml renders output to /opt/tectonic/tnc-bootstrap
mkdir -p /etc/kubernetes/manifests/
cp "$(pwd)/tnc-bootstrap/tectonic-node-controller-pod.yaml" /etc/kubernetes/manifests/
cp "$(pwd)/tnc-bootstrap/tectonic-node-controller-config.yaml" /etc/kubernetes/tnc-config
runs kube-etcd-signer-server container
https://github.com/coreos/kubecsr/blob/master/Documentation/etcd-certificate-generator.md
reads /opt/tectonic/tls/{etcd-client*,apiserver*}, runs in host network namespace, and waits on signing request from etcd members

=== etcd ===
etcd-member-tls.service starts
runs kube-client-agent container twice, once for server CSR and once for peer CSR
https://github.com/coreos/kubecsr/blob/master/Documentation/etcd-certificate-generator.md
issues CSR to kube-etcd-signer-server running on the master and stores peer and server certs in /etc/ssl/etcd
etc-member.service starts with newly signed certs
runs etcd container
this unblocks bootkube.sh on the master

=== master ===
kills kube-etcd-signer-server container
rm /etc/kubernetes/manifests/tectonic-node-controller-pod.yaml
cp -r "$(pwd)/bootstrap-configs" /etc/kubernetes/bootstrap-configs
runs bootkube container (hostpath equivalent command "bootkube start --asset-dir=/opt/tectonic")
https://github.com/kubernetes-incubator/bootkube
bootkube basically just creates all the resources in /opt/tectonic/manifests, waits for the new self-hosted kube-apiserver (started by the kube-apiserver DS) to be running, and then deletes the bootstrap control plane i.e. removes the pod manifests from /etc/kubernetes/manifestes

from there, operators start the rest of the initial applications


files generated by `tectonic init` and injected into the bootstrap with ignition

/opt/tectonic/manifests/cluster-config.yaml
/opt/tectonic/tectonic/cluster-config.yaml
/opt/tectonic/tnco-config.yaml
/opt/tectonic/kco-config.yaml
/etc/kubernetes/kubeconfig
/var/lib/kubelet/kubeconfig
/etc/ssl/etcd/root-ca.crt
/etc/ssl/etcd/ca.crt
/etc/ssl/certs/root_ca.pem
/etc/ssl/certs/ingress_ca.pem
/etc/ssl/certs/etcd_ca.pem
/opt/tectonic/bootkube.sh
/opt/tectonic/auth/kubeconfig
/opt/tectonic/auth/kubeconfig-kubelet
/opt/tectonic/manifests/01-tectonic-namespace.yaml
/opt/tectonic/manifests/02-ingress-namespace.yaml
/opt/tectonic/manifests/03-openshift-web-console-namespace.yaml
/opt/tectonic/manifests/app-version-kind.yaml
/opt/tectonic/manifests/app-version-tectonic-network.yaml
/opt/tectonic/manifests/app-version-tnc.yaml
/opt/tectonic/manifests/kube-apiserver-secret.yaml
/opt/tectonic/manifests/kube-cloud-config.yaml
/opt/tectonic/manifests/kube-controller-manager-secret.yaml
/opt/tectonic/manifests/node-config-kind.yaml
/opt/tectonic/manifests/openshift-apiserver-secret.yaml
/opt/tectonic/manifests/cluster-apiserver-secret.yaml
/opt/tectonic/manifests/pull.json
/opt/tectonic/manifests/tectonic-network-operator.yaml
/opt/tectonic/manifests/tectonic-node-controller-operator.yaml
/opt/tectonic/manifests/tnc-tls-secret.yaml
/opt/tectonic/tectonic.sh
/opt/tectonic/tectonic-wrapper.sh
/opt/tectonic/tectonic/ingress/cluster-config.yaml
/opt/tectonic/tectonic/ingress/pull.json
/opt/tectonic/tectonic/ingress/README.md
/opt/tectonic/tectonic/ingress/svc-account.yaml
/opt/tectonic/tectonic/rbac/binding-admin.yaml
/opt/tectonic/tectonic/rbac/binding-discovery.yaml
/opt/tectonic/tectonic/rbac/role-admin.yaml
/opt/tectonic/tectonic/rbac/role-user.yaml
/opt/tectonic/tectonic/secrets/ca-cert.yaml
/opt/tectonic/tectonic/secrets/ingress-tls.yaml
/opt/tectonic/tectonic/secrets/license.json
/opt/tectonic/tectonic/secrets/pull.json
/opt/tectonic/tectonic/security/priviledged-scc-tectonic.yaml
/opt/tectonic/tectonic/updater/app-version-kind.yaml
/opt/tectonic/tectonic/updater/app_versions/app-version-kube-core.yaml
/opt/tectonic/tectonic/updater/app_versions/app-version-kube-addon.yaml
/opt/tectonic/tectonic/updater/app_versions/app-version-tectonic-alm.yaml
/opt/tectonic/tectonic/updater/app_versions/app-version-tectonic-cluster.yaml
/opt/tectonic/tectonic/updater/app_versions/app-version-tectonic-ingress.yaml
/opt/tectonic/tectonic/updater/app_versions/app-version-tectonic-utility.yaml
/opt/tectonic/tectonic/updater/migration-status-kind.yaml
/opt/tectonic/tectonic/updater/operators/kube-core-operator.yaml
/opt/tectonic/tectonic/updater/operators/kube-addon-operator.yaml
/opt/tectonic/tectonic/updater/operators/tectonic-alm-operator.yaml
/opt/tectonic/tectonic/updater/operators/tectonic-channel-operator.yaml
/opt/tectonic/tectonic/updater/operators/tectonic-ingress-controller-operator.yaml
/opt/tectonic/tectonic/updater/operators/tectonic-utility-operator.yaml
/opt/tectonic/tectonic/updater/tectonic-channel-operator-config.yaml
/opt/tectonic/tectonic/updater/tectonic-channel-operator-kind.yaml
/opt/tectonic/tls/root-ca.crt
/opt/tectonic/tls/kube-ca.key
/opt/tectonic/tls/kube-ca.crt
/opt/tectonic/tls/aggregator-ca.key
/opt/tectonic/tls/aggregator-ca.crt
/opt/tectonic/tls/service-serving-ca.key
/opt/tectonic/tls/service-serving-ca.crt
/opt/tectonic/tls/etcd-client-ca.key
/opt/tectonic/tls/etcd-client-ca.crt
/opt/tectonic/tls/cluster-apiserver-ca.key
/opt/tectonic/tls/cluster-apiserver-ca.crt
/opt/tectonic/tls/etcd-client.crt
/opt/tectonic/tls/etcd-client.key
/opt/tectonic/tls/apiserver.key
/opt/tectonic/tls/apiserver.crt
/opt/tectonic/tls/openshift-apiserver.key
/opt/tectonic/tls/openshift-apiserver.crt
/opt/tectonic/tls/apiserver-proxy.key
/opt/tectonic/tls/apiserver-proxy.crt
/opt/tectonic/tls/admin.key
/opt/tectonic/tls/admin.crt
/opt/tectonic/tls/kubelet.key
/opt/tectonic/tls/kubelet.crt
/opt/tectonic/tls/tnc.key
/opt/tectonic/tls/tnc.crt
/opt/tectonic/tls/service-account.key
/opt/tectonic/tls/service-account.pub

ingition inject files on the master from the TNC

/etc/hosts
/etc/ssl/etcd/ca.crt
/etc/kubernetes/kubeconfig
/etc/sysctl.d/10-max-user-watches.conf
/etc/docker/certs.d/docker-registry.default.svc:5000/ca.crt
/etc/ssl/etcd/root-ca.crt
/etc/coreos/update.conf
/opt/tectonic/node-annotations.json
/etc/systemd/system/docker.service.d/10-dockeropts.conf
/etc/systemd/system/docker.service
/etc/systemd/system/etcd-member-tls.service
/etc/systemd/system/etcd-member.service
/etc/systemd/system/kubelet-workaround.service
/etc/systemd/system/kubelet.service
/etc/systemd/system/locksmith.service

ingition inject files on the worker from the TNC

/etc/hosts
/etc/kubernetes/kubeconfig
/etc/sysctl.d/10-max-user-watches.conf
/etc/docker/certs.d/docker-registry.default.svc:5000/ca.crt
/etc/coreos/update.conf
/opt/tectonic/node-annotations.json
/etc/systemd/system/docker.service.d/10-dockeropts.conf
/etc/systemd/system/docker.service
/etc/systemd/system/kubelet-workaround.service
/etc/systemd/system/kubelet.service
/etc/systemd/system/locksmith.service