Skip to content

Instantly share code, notes, and snippets.

@yifan-gu
Last active October 31, 2017 20:27
Show Gist options
  • Save yifan-gu/8392d7d2f2a9b92a3f9c126d035d34ad to your computer and use it in GitHub Desktop.
Save yifan-gu/8392d7d2f2a9b92a3f9c126d035d34ad to your computer and use it in GitHub Desktop.
checkpointer flake
I1031 20:06:45.955006 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:06:45.955056 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:06:48.975356 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:06:48.992369 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-0000" already exists. Skipping
I1031 20:06:49.005260 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
I1031 20:06:49.010436 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-network-checkpointer-2npz7" already exists. Skipping
I1031 20:06:49.023779 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:06:49.023799 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-0000
I1031 20:06:49.023805 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-network-checkpointer-2npz7
I1031 20:06:49.023874 1 main.go:232] API GC: skipping inactive checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:06:49.023898 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:06:49.023938 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:06:52.065797 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:06:52.083896 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-0000" already exists. Skipping
I1031 20:06:52.097394 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
I1031 20:06:52.101875 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-network-checkpointer-2npz7" already exists. Skipping
I1031 20:06:52.116724 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:06:52.116750 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-0000
I1031 20:06:52.116757 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-network-checkpointer-2npz7
I1031 20:06:52.116763 1 main.go:232] API GC: skipping inactive checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:06:52.116769 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:06:52.116828 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:06:55.138449 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
I1031 20:06:55.142792 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-network-checkpointer-2npz7" already exists. Skipping
I1031 20:06:55.147235 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:06:55.164544 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-0000" already exists. Skipping
I1031 20:06:55.177914 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:06:55.177932 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-0000
I1031 20:06:55.177939 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-network-checkpointer-2npz7
I1031 20:06:55.177945 1 main.go:232] API GC: skipping inactive checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:06:55.177951 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:06:55.177990 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:06:58.191313 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-network-checkpointer-2npz7" already exists. Skipping
I1031 20:06:58.195531 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:06:58.221055 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-0000" already exists. Skipping
I1031 20:06:58.234567 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
I1031 20:06:58.247964 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-network-checkpointer-2npz7
I1031 20:06:58.247988 1 main.go:232] API GC: skipping inactive checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:06:58.247995 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:06:58.248000 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-0000
I1031 20:06:58.248005 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:06:58.248038 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:01.261644 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-network-checkpointer-2npz7" already exists. Skipping
I1031 20:07:01.277207 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
I1031 20:07:01.301509 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-0000" already exists. Skipping
I1031 20:07:01.306722 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:01.322840 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:07:01.323001 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-0000
I1031 20:07:01.323028 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-network-checkpointer-2npz7
I1031 20:07:01.323114 1 main.go:232] API GC: skipping inactive checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:01.323144 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:01.323262 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:04.349413 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
I1031 20:07:04.353764 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-network-checkpointer-2npz7" already exists. Skipping
I1031 20:07:04.358296 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:04.375732 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-0000" already exists. Skipping
I1031 20:07:04.389156 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:07:04.389178 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-0000
I1031 20:07:04.389186 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-network-checkpointer-2npz7
I1031 20:07:04.389192 1 main.go:232] API GC: skipping inactive checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:04.389198 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:04.389234 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:07.401227 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:07.418890 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-0000" already exists. Skipping
I1031 20:07:07.432303 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
I1031 20:07:07.436647 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-network-checkpointer-2npz7" already exists. Skipping
I1031 20:07:07.450506 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:07:07.450529 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-0000
I1031 20:07:07.450536 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-network-checkpointer-2npz7
I1031 20:07:07.450542 1 main.go:232] API GC: skipping inactive checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:07.450551 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:07.450592 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:10.462472 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:10.480949 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-0000" already exists. Skipping
I1031 20:07:10.493859 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
I1031 20:07:10.498285 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-network-checkpointer-2npz7" already exists. Skipping
I1031 20:07:10.513918 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:07:10.513942 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-0000
I1031 20:07:10.513949 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-network-checkpointer-2npz7
I1031 20:07:10.513955 1 main.go:232] API GC: skipping inactive checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:10.513961 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:10.514016 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
E1031 20:07:23.523508 1 main.go:341] Failed to checkpoint secrets for pod kube-system/pod-checkpointer-wqbq7: failed to checkpoint secret for pod kube-system/pod-checkpointer-wqbq7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: net/http: TLS handshake timeout
E1031 20:07:25.032958 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-0000: failed to checkpoint secret for pod kube-system/kube-etcd-0000: failed to retrieve secret kube-system/etcd-peer-tls: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/etcd-peer-tls: EOF
E1031 20:07:25.034817 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-apiserver-qtmd7: failed to checkpoint secret for pod kube-system/kube-apiserver-qtmd7: failed to retrieve secret kube-system/kube-apiserver: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/kube-apiserver: EOF
E1031 20:07:25.037972 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to checkpoint secret for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
W1031 20:07:25.039318 1 main.go:480] Unable to contact APIServer, skipping garbage collection: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-0-9-167.us-west-1.compute.internal: EOF
I1031 20:07:25.040163 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:25.040183 1 main.go:290] Should start checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:07:25.040212 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:25.040241 1 main.go:388] Writing manifest for "kube-system/kube-apiserver-qtmd7" to "/etc/kubernetes/manifests/kube-system-kube-apiserver-qtmd7.json"
E1031 20:07:28.063885 1 main.go:341] Failed to checkpoint secrets for pod kube-system/pod-checkpointer-wqbq7: failed to checkpoint secret for pod kube-system/pod-checkpointer-wqbq7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
E1031 20:07:28.067072 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-0000: failed to checkpoint secret for pod kube-system/kube-etcd-0000: failed to retrieve secret kube-system/etcd-peer-tls: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/etcd-peer-tls: EOF
E1031 20:07:28.068547 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-apiserver-qtmd7: failed to checkpoint secret for pod kube-system/kube-apiserver-qtmd7: failed to retrieve secret kube-system/kube-apiserver: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/kube-apiserver: EOF
E1031 20:07:28.071960 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to checkpoint secret for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
W1031 20:07:28.074160 1 main.go:480] Unable to contact APIServer, skipping garbage collection: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-0-9-167.us-west-1.compute.internal: EOF
I1031 20:07:28.076163 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:28.076179 1 main.go:290] Should start checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:07:28.076211 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:28.076240 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
E1031 20:07:31.095252 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-apiserver-qtmd7: failed to checkpoint secret for pod kube-system/kube-apiserver-qtmd7: failed to retrieve secret kube-system/kube-apiserver: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/kube-apiserver: EOF
E1031 20:07:31.099580 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to checkpoint secret for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
E1031 20:07:31.101039 1 main.go:341] Failed to checkpoint secrets for pod kube-system/pod-checkpointer-wqbq7: failed to checkpoint secret for pod kube-system/pod-checkpointer-wqbq7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
E1031 20:07:31.104338 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-0000: failed to checkpoint secret for pod kube-system/kube-etcd-0000: failed to retrieve secret kube-system/etcd-peer-tls: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/etcd-peer-tls: EOF
W1031 20:07:31.106931 1 main.go:480] Unable to contact APIServer, skipping garbage collection: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-0-9-167.us-west-1.compute.internal: EOF
I1031 20:07:31.108033 1 main.go:290] Should start checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:07:31.108086 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:31.108163 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
I1031 20:07:31.108215 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
E1031 20:07:34.126146 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-apiserver-qtmd7: failed to checkpoint secret for pod kube-system/kube-apiserver-qtmd7: failed to retrieve secret kube-system/kube-apiserver: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/kube-apiserver: EOF
E1031 20:07:34.129696 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to checkpoint secret for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
E1031 20:07:34.131394 1 main.go:341] Failed to checkpoint secrets for pod kube-system/pod-checkpointer-wqbq7: failed to checkpoint secret for pod kube-system/pod-checkpointer-wqbq7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
E1031 20:07:34.134509 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-0000: failed to checkpoint secret for pod kube-system/kube-etcd-0000: failed to retrieve secret kube-system/etcd-peer-tls: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/etcd-peer-tls: EOF
W1031 20:07:34.135878 1 main.go:480] Unable to contact APIServer, skipping garbage collection: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-0-9-167.us-west-1.compute.internal: EOF
I1031 20:07:34.136887 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:34.136902 1 main.go:290] Should start checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:07:34.136928 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:34.136955 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
I1031 20:07:37.165006 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:37.181716 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-0000" already exists. Skipping
I1031 20:07:37.194819 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
I1031 20:07:37.199154 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-network-checkpointer-2npz7" already exists. Skipping
I1031 20:07:37.214674 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:07:37.214701 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-0000
I1031 20:07:37.214708 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-network-checkpointer-2npz7
I1031 20:07:37.214714 1 main.go:232] API GC: skipping inactive checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:37.214718 1 main.go:290] Should start checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:07:37.214723 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:37.214752 1 main.go:385] Checkpoint manifest for "kube-system/kube-apiserver-qtmd7" already exists. Skipping
I1031 20:07:37.214800 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:40.226077 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-network-checkpointer-2npz7" already exists. Skipping
I1031 20:07:40.230187 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:40.248693 1 main.go:385] Checkpoint manifest for "kube-system/kube-etcd-0000" already exists. Skipping
I1031 20:07:40.262189 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-0000
I1031 20:07:40.262210 1 main.go:232] API GC: skipping inactive checkpoint kube-system/kube-etcd-network-checkpointer-2npz7
I1031 20:07:40.262217 1 main.go:232] API GC: skipping inactive checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:40.262224 1 main.go:241] API GC: should remove inactive checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:07:40.262235 1 main.go:257] API GC: should remove active checkpoint kube-system/kube-apiserver-qtmd7
I1031 20:07:40.262241 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:40.262273 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
I1031 20:07:40.262284 1 main.go:617] Removing checkpoint of: kube-system/kube-apiserver-qtmd7
E1031 20:07:43.272495 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-0000: failed to checkpoint secret for pod kube-system/kube-etcd-0000: failed to retrieve secret kube-system/etcd-peer-tls: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/etcd-peer-tls: EOF
E1031 20:07:50.034525 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to checkpoint secret for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
E1031 20:07:50.036149 1 main.go:341] Failed to checkpoint secrets for pod kube-system/pod-checkpointer-wqbq7: failed to checkpoint secret for pod kube-system/pod-checkpointer-wqbq7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
W1031 20:07:50.039083 1 main.go:480] Unable to contact APIServer, skipping garbage collection: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-0-9-167.us-west-1.compute.internal: EOF
I1031 20:07:50.039735 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:50.039769 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
E1031 20:07:53.049777 1 main.go:341] Failed to checkpoint secrets for pod kube-system/pod-checkpointer-wqbq7: failed to checkpoint secret for pod kube-system/pod-checkpointer-wqbq7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
E1031 20:07:53.051248 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to checkpoint secret for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
E1031 20:07:53.054388 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-0000: failed to checkpoint secret for pod kube-system/kube-etcd-0000: failed to retrieve secret kube-system/etcd-peer-tls: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/etcd-peer-tls: EOF
W1031 20:07:53.055652 1 main.go:480] Unable to contact APIServer, skipping garbage collection: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-0-9-167.us-west-1.compute.internal: EOF
I1031 20:07:53.056366 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:53.056410 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
E1031 20:07:56.066028 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-0000: failed to checkpoint secret for pod kube-system/kube-etcd-0000: failed to retrieve secret kube-system/etcd-peer-tls: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/etcd-peer-tls: EOF
E1031 20:07:56.069141 1 main.go:341] Failed to checkpoint secrets for pod kube-system/pod-checkpointer-wqbq7: failed to checkpoint secret for pod kube-system/pod-checkpointer-wqbq7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
E1031 20:07:56.070456 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to checkpoint secret for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
W1031 20:07:56.073531 1 main.go:480] Unable to contact APIServer, skipping garbage collection: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-0-9-167.us-west-1.compute.internal: EOF
I1031 20:07:56.074148 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:56.074196 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
E1031 20:07:59.098551 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to checkpoint secret for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
E1031 20:07:59.100082 1 main.go:341] Failed to checkpoint secrets for pod kube-system/pod-checkpointer-wqbq7: failed to checkpoint secret for pod kube-system/pod-checkpointer-wqbq7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
E1031 20:07:59.103167 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-0000: failed to checkpoint secret for pod kube-system/kube-etcd-0000: failed to retrieve secret kube-system/etcd-peer-tls: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/etcd-peer-tls: EOF
W1031 20:07:59.104434 1 main.go:480] Unable to contact APIServer, skipping garbage collection: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-0-9-167.us-west-1.compute.internal: EOF
I1031 20:07:59.105113 1 main.go:290] Should start checkpoint kube-system/pod-checkpointer-wqbq7
I1031 20:07:59.105150 1 main.go:385] Checkpoint manifest for "kube-system/pod-checkpointer-wqbq7" already exists. Skipping
E1031 20:08:02.120029 1 main.go:341] Failed to checkpoint secrets for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to checkpoint secret for pod kube-system/kube-etcd-network-checkpointer-2npz7: failed to retrieve secret kube-system/default-token-z91t3: Get https://my-test-cluster-api.tectonic.dev.coreos.systems:443/api/v1/namespaces/kube-system/secrets/default-token-z91t3: EOF
I1031 20:06:41.059947 1 kube_version_operator.go:663] all container images cached
I1031 20:06:42.076052 1 kube_version_operator.go:577] Updating component kube-apiserver
W1031 20:07:12.161071 1 reflector.go:326] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:200: watch of *v1.Node ended with: very short watch: github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:200: Unexpected watch close - watch lasted less than a second and no items received
W1031 20:07:12.161193 1 reflector.go:326] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:205: watch of *v1.Pod ended with: very short watch: github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:205: Unexpected watch close - watch lasted less than a second and no items received
W1031 20:07:12.161244 1 reflector.go:326] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:201: watch of *v1beta1.DaemonSet ended with: very short watch: github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:201: Unexpected watch close - watch lasted less than a second and no items received
W1031 20:07:12.161336 1 reflector.go:326] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:203: watch of *v1beta1.DaemonSet ended with: very short watch: github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:203: Unexpected watch close - watch lasted less than a second and no items received
W1031 20:07:12.161385 1 reflector.go:326] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:202: watch of *v1beta1.Deployment ended with: very short watch: github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:202: Unexpected watch close - watch lasted less than a second and no items received
W1031 20:07:12.161432 1 reflector.go:326] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:207: watch of *v1.Service ended with: very short watch: github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:207: Unexpected watch close - watch lasted less than a second and no items received
W1031 20:07:12.161518 1 reflector.go:326] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:206: watch of *v1.Service ended with: very short watch: github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:206: Unexpected watch close - watch lasted less than a second and no items received
W1031 20:07:12.161565 1 reflector.go:326] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:204: watch of *v1beta1.Deployment ended with: very short watch: github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:204: Unexpected watch close - watch lasted less than a second and no items received
E1031 20:07:13.107296 1 daemonsets.go:195] error getting DaemonSet kube-apiserver during rollout: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/kube-system/daemonsets/kube-apiserver: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:13.162048 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:201: Failed to list *v1beta1.DaemonSet: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/kube-system/daemonsets?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:13.162220 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:205: Failed to list *v1.Pod: Get https://10.3.0.1:443/api/v1/namespaces/kube-system/pods?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:13.162260 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:200: Failed to list *v1.Node: Get https://10.3.0.1:443/api/v1/nodes?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:13.162290 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:202: Failed to list *v1beta1.Deployment: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/kube-system/deployments?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:13.162320 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:204: Failed to list *v1beta1.Deployment: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/tectonic-system/deployments?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:13.162608 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:203: Failed to list *v1beta1.DaemonSet: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/tectonic-system/daemonsets?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:13.162614 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:207: Failed to list *v1.Service: Get https://10.3.0.1:443/api/v1/namespaces/tectonic-system/services?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:13.162783 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:206: Failed to list *v1.Service: Get https://10.3.0.1:443/api/v1/namespaces/kube-system/services?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:14.107335 1 daemonsets.go:195] error getting DaemonSet kube-apiserver during rollout: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/kube-system/daemonsets/kube-apiserver: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:14.162802 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:201: Failed to list *v1beta1.DaemonSet: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/kube-system/daemonsets?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:14.163866 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:205: Failed to list *v1.Pod: Get https://10.3.0.1:443/api/v1/namespaces/kube-system/pods?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:14.164892 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:200: Failed to list *v1.Node: Get https://10.3.0.1:443/api/v1/nodes?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:14.166067 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:202: Failed to list *v1beta1.Deployment: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/kube-system/deployments?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:14.167270 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:204: Failed to list *v1beta1.Deployment: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/tectonic-system/deployments?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:14.168424 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:203: Failed to list *v1beta1.DaemonSet: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/tectonic-system/daemonsets?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:14.169502 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:207: Failed to list *v1.Service: Get https://10.3.0.1:443/api/v1/namespaces/tectonic-system/services?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:14.170691 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:206: Failed to list *v1.Service: Get https://10.3.0.1:443/api/v1/namespaces/kube-system/services?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:15.107497 1 daemonsets.go:195] error getting DaemonSet kube-apiserver during rollout: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/kube-system/daemonsets/kube-apiserver: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:15.163617 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:201: Failed to list *v1beta1.DaemonSet: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/kube-system/daemonsets?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:15.164575 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:205: Failed to list *v1.Pod: Get https://10.3.0.1:443/api/v1/namespaces/kube-system/pods?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:15.165376 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:200: Failed to list *v1.Node: Get https://10.3.0.1:443/api/v1/nodes?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:15.166597 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:202: Failed to list *v1beta1.Deployment: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/kube-system/deployments?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:15.167813 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:204: Failed to list *v1beta1.Deployment: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/tectonic-system/deployments?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:15.168959 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:203: Failed to list *v1beta1.DaemonSet: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/tectonic-system/daemonsets?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:15.170027 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:207: Failed to list *v1.Service: Get https://10.3.0.1:443/api/v1/namespaces/tectonic-system/services?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:15.171253 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:206: Failed to list *v1.Service: Get https://10.3.0.1:443/api/v1/namespaces/kube-system/services?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:16.107584 1 daemonsets.go:195] error getting DaemonSet kube-apiserver during rollout: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/kube-system/daemonsets/kube-apiserver: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:16.164436 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:201: Failed to list *v1beta1.DaemonSet: Get https://10.3.0.1:443/apis/extensions/v1beta1/namespaces/kube-system/daemonsets?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:16.165603 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:205: Failed to list *v1.Pod: Get https://10.3.0.1:443/api/v1/namespaces/kube-system/pods?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
E1031 20:07:16.166621 1 reflector.go:199] github.com/coreos-inc/kube-version-operator/pkg/cluster/watcher.go:200: Failed to list *v1.Node: Get https://10.3.0.1:443/api/v1/nodes?resourceVersion=0: dial tcp 10.3.0.1:443: getsockopt: connection refused
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment