Skip to content

Instantly share code, notes, and snippets.

@mohag

mohag/cluster-manifest.yaml

Last active June 27, 2019 14:58
Show Gist options
  • Save mohag/2244eb229b15e776fd65b84a35a8cfb0 to your computer and use it in GitHub Desktop.
Save mohag/2244eb229b15e776fd65b84a35a8cfb0 to your computer and use it in GitHub Desktop.
Download ZIP
Cluster manifest for KOPS issue
Raw
cluster-manifest.yaml
kind: Cluster
metadata:
creationTimestamp: 2019-06-27T13:58:33Z
name: dev-qa.k8s.domain.com
spec:
additionalPolicies:
node: |
[
{
"Effect": "Allow",
"Action": [
"route53:ListHostedZones",
"route53:ListResourceRecordSets"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"route53:ChangeResourceRecordSets"
],
"Resource": [
"arn:aws:route53:::hostedzone/*"
]
},
{
"Effect": "Allow",
"Action": [
"ec2:AttachVolume",
"ec2:DetachVolume"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeTags",
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"codecommit:BatchGet*",
"codecommit:Get*",
"codecommit:Describe*",
"codecommit:List*",
"codecommit:GitPull"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"sqs:*",
"sns:*",
"ecr:*",
"s3:*"
],
"Resource": "*"
}
]
api:
loadBalancer:
type: Public
authorization:
rbac: {}
channel: stable
cloudProvider: aws
configBase: s3://my-state-store/dev-qa.k8s.domain.com
dnsZone: k8s.domain.com
etcdClusters:
- cpuRequest: 200m
etcdMembers:
- encryptedVolume: true
instanceGroup: master-eu-central-1a
name: a
- encryptedVolume: true
instanceGroup: master-eu-central-1b
name: b
- encryptedVolume: true
instanceGroup: master-eu-central-1c
name: c
memoryRequest: 100Mi
name: main
version: 3.2.24
- cpuRequest: 100m
etcdMembers:
- encryptedVolume: true
instanceGroup: master-eu-central-1a
name: a
- encryptedVolume: true
instanceGroup: master-eu-central-1b
name: b
- encryptedVolume: true
instanceGroup: master-eu-central-1c
name: c
memoryRequest: 100Mi
name: events
version: 3.2.24
iam:
allowContainerRegistry: true
legacy: false
kubelet:
anonymousAuth: false
kubernetesApiAccess:
- 0.0.0.0/0
kubernetesVersion: 1.12.9
masterPublicName: api.dev-qa.k8s.domain.com
networkCIDR: 172.31.239.0/24
networking:
calico:
majorVersion: v3
nonMasqueradeCIDR: 100.64.0.0/10
sshAccess:
- 0.0.0.0/0
subnets:
- cidr: 172.31.239.32/27
name: eu-central-1a
type: Private
zone: eu-central-1a
- cidr: 172.31.239.64/27
name: eu-central-1b
type: Private
zone: eu-central-1b
- cidr: 172.31.239.96/27
name: eu-central-1c
type: Private
zone: eu-central-1c
- cidr: 172.31.239.0/30
name: utility-eu-central-1a
type: Utility
zone: eu-central-1a
- cidr: 172.31.239.4/30
name: utility-eu-central-1b
type: Utility
zone: eu-central-1b
- cidr: 172.31.239.8/30
name: utility-eu-central-1c
type: Utility
zone: eu-central-1c
topology:
bastion:
bastionPublicName: bastion.dev-qa.k8s.domain.com
dns:
type: Public
masters: private
nodes: private
Raw
kops-create-cluster.log
I0627 16:48:40.034676 22451 create_cluster.go:1486] Using SSH public key: /home/gertvdb/.ssh/k8s-dev-id_rsa.pub
I0627 16:48:40.035331 22451 factory.go:68] state store s3://my-state-store
I0627 16:48:40.036186 22451 s3context.go:325] unable to read /sys/devices/virtual/dmi/id/product_uuid, assuming not running on EC2: open /sys/devices/virtual/dmi/id/product_uuid: no such file or directory
I0627 16:48:40.036705 22451 s3context.go:170] defaulting region to "us-east-1"
I0627 16:48:41.509793 22451 s3context.go:210] found bucket in region "eu-central-1"
I0627 16:48:41.510760 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/config"
I0627 16:48:42.596790 22451 channel.go:97] resolving "stable" against default channel location "https://raw.githubusercontent.com/kubernetes/kops/master/channels/"
I0627 16:48:42.597508 22451 channel.go:102] Loading channel from "https://raw.githubusercontent.com/kubernetes/kops/master/channels/stable"
I0627 16:48:42.605505 22451 context.go:159] Performing HTTP request: GET https://raw.githubusercontent.com/kubernetes/kops/master/channels/stable
I0627 16:48:43.312579 22451 channel.go:111] Channel contents: spec:
images:
# We put the "legacy" version first, for kops versions that don't support versions ( < 1.5.0 )
- name: kope.io/k8s-1.4-debian-jessie-amd64-hvm-ebs-2017-07-28
providerID: aws
kubernetesVersion: ">=1.4.0 <1.5.0"
- name: kope.io/k8s-1.5-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.5.0 <1.6.0"
- name: kope.io/k8s-1.6-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.6.0 <1.7.0"
- name: kope.io/k8s-1.7-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.7.0 <1.8.0"
- name: kope.io/k8s-1.8-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.8.0 <1.9.0"
- name: kope.io/k8s-1.9-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.9.0 <1.10.0"
- name: kope.io/k8s-1.10-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.10.0 <1.11.0"
# Stretch is the default for 1.11 (for nvme)
- name: kope.io/k8s-1.11-debian-stretch-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.11.0 <1.12.0"
- name: kope.io/k8s-1.12-debian-stretch-amd64-hvm-ebs-2019-05-13
providerID: aws
kubernetesVersion: ">=1.12.0"
- providerID: gce
name: "cos-cloud/cos-stable-65-10323-99-0"
cluster:
kubernetesVersion: v1.5.8
networking:
kubenet: {}
kubernetesVersions:
- range: ">=1.14.0"
recommendedVersion: 1.14.1
requiredVersion: 1.14.0
- range: ">=1.13.0"
recommendedVersion: 1.13.5
requiredVersion: 1.13.0
- range: ">=1.12.0"
recommendedVersion: 1.12.8
requiredVersion: 1.12.0
- range: ">=1.11.0"
recommendedVersion: 1.11.10
requiredVersion: 1.11.0
- range: ">=1.10.0"
recommendedVersion: 1.10.13
requiredVersion: 1.10.0
- range: ">=1.9.0"
recommendedVersion: 1.9.11
requiredVersion: 1.9.0
- range: ">=1.8.0"
recommendedVersion: 1.8.15
requiredVersion: 1.8.0
- range: ">=1.7.0"
recommendedVersion: 1.7.16
requiredVersion: 1.7.0
- range: ">=1.6.0"
recommendedVersion: 1.6.13
requiredVersion: 1.6.0
- range: ">=1.5.0"
recommendedVersion: 1.5.8
requiredVersion: 1.5.1
- range: "<1.5.0"
recommendedVersion: 1.4.12
requiredVersion: 1.4.2
kopsVersions:
- range: ">=1.14.0-alpha.1"
#recommendedVersion: "1.14.0"
#requiredVersion: 1.14.0
kubernetesVersion: 1.14.1
- range: ">=1.13.0-alpha.1"
#recommendedVersion: "1.13.0"
#requiredVersion: 1.13.0
kubernetesVersion: 1.13.5
- range: ">=1.12.0-alpha.1"
recommendedVersion: "1.12.1"
#requiredVersion: 1.12.0
kubernetesVersion: 1.12.8
- range: ">=1.11.0-alpha.1"
#recommendedVersion: "1.11.0"
#requiredVersion: 1.11.0
kubernetesVersion: 1.11.10
- range: ">=1.10.0-alpha.1"
recommendedVersion: "1.10.0"
#requiredVersion: 1.10.0
kubernetesVersion: 1.10.13
- range: ">=1.9.0-alpha.1"
recommendedVersion: 1.9.2
#requiredVersion: 1.9.0
kubernetesVersion: 1.9.11
- range: ">=1.8.0-alpha.1"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.8.15
- range: ">=1.7.0-alpha.1"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.7.16
- range: ">=1.6.0-alpha.1"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.6.13
- range: ">=1.5.0-alpha1"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.5.8
- range: "<1.5.0"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.4.12
I0627 16:48:43.340910 22451 channel.go:239] version range ">=1.14.0-alpha.1" does not apply to version "1.12.1"; skipping
I0627 16:48:43.341584 22451 channel.go:239] version range ">=1.13.0-alpha.1" does not apply to version "1.12.1"; skipping
I0627 16:48:43.342335 22451 create_cluster.go:1031] networking mode=calico => {"calico":{"majorVersion":"v3"}}
I0627 16:48:43.344462 22451 aws_cloud.go:1212] Querying EC2 for all valid zones in region "eu-central-1"
I0627 16:48:43.344874 22451 request_logger.go:45] AWS request: ec2/DescribeAvailabilityZones
I0627 16:48:44.640801 22451 subnets.go:184] Assigned CIDR 172.31.239.32/27 to subnet eu-central-1a
I0627 16:48:44.641518 22451 subnets.go:184] Assigned CIDR 172.31.239.64/27 to subnet eu-central-1b
I0627 16:48:44.641916 22451 subnets.go:184] Assigned CIDR 172.31.239.96/27 to subnet eu-central-1c
I0627 16:48:44.642277 22451 subnets.go:198] Assigned CIDR 172.31.239.0/30 to subnet utility-eu-central-1a
I0627 16:48:44.642667 22451 subnets.go:198] Assigned CIDR 172.31.239.4/30 to subnet utility-eu-central-1b
I0627 16:48:44.643210 22451 subnets.go:198] Assigned CIDR 172.31.239.8/30 to subnet utility-eu-central-1c
I0627 16:48:44.643547 22451 defaults.go:213] Not setting up Proxy Excludes
I0627 16:48:44.651778 22451 aws.go:99] machineType m5a.2xlarge requires an image based on stretch to operate. Trying to check compatibility
I0627 16:48:44.655477 22451 populate_cluster_spec.go:371] Defaulted KubeControllerManager.ClusterCIDR to 100.96.0.0/11
I0627 16:48:44.656044 22451 populate_cluster_spec.go:378] Defaulted ServiceClusterIPRange to 100.64.0.0/13
I0627 16:48:44.656795 22451 aws_cloud.go:1212] Querying EC2 for all valid zones in region "eu-central-1"
I0627 16:48:44.657312 22451 request_logger.go:45] AWS request: ec2/DescribeAvailabilityZones
I0627 16:48:44.863443 22451 subnets.go:49] All subnets have CIDRs; skipping assignment logic
I0627 16:48:44.864177 22451 defaults.go:213] Not setting up Proxy Excludes
I0627 16:48:44.864571 22451 aws_cloud.go:1212] Querying EC2 for all valid zones in region "eu-central-1"
I0627 16:48:44.865153 22451 request_logger.go:45] AWS request: ec2/DescribeAvailabilityZones
I0627 16:48:45.076451 22451 tagbuilder.go:95] tags: [_aws _k8s_1_6]
I0627 16:48:45.077262 22451 options_loader.go:130] executing builder *components.DefaultsOptionsBuilder
I0627 16:48:45.077577 22451 options_loader.go:130] executing builder *components.EtcdOptionsBuilder
I0627 16:48:45.077905 22451 options_loader.go:130] executing builder *etcdmanager.EtcdManagerOptionsBuilder
I0627 16:48:45.078209 22451 options_loader.go:130] executing builder *nodeauthorizer.OptionsBuilder
I0627 16:48:45.078496 22451 options_loader.go:130] executing builder *components.KubeAPIServerOptionsBuilder
I0627 16:48:45.078822 22451 options_loader.go:130] executing builder *components.DockerOptionsBuilder
I0627 16:48:45.079134 22451 options_loader.go:130] executing builder *components.NetworkingOptionsBuilder
I0627 16:48:45.079425 22451 options_loader.go:130] executing builder *components.KubeDnsOptionsBuilder
I0627 16:48:45.079707 22451 options_loader.go:130] executing builder *components.KubeletOptionsBuilder
I0627 16:48:45.080308 22451 kubelet.go:171] Cloud Provider: aws
I0627 16:48:45.080546 22451 options_loader.go:130] executing builder *components.KubeControllerManagerOptionsBuilder
I0627 16:48:45.080798 22451 kubecontrollermanager.go:74] Kubernetes version "1.12.9" supports AttachDetachReconcileSyncPeriod; will configure
I0627 16:48:45.081065 22451 kubecontrollermanager.go:79] AttachDetachReconcileSyncPeriod is not set; will set to default 1m0s
I0627 16:48:45.081279 22451 options_loader.go:130] executing builder *components.KubeSchedulerOptionsBuilder
I0627 16:48:45.081549 22451 options_loader.go:130] executing builder *components.KubeProxyOptionsBuilder
I0627 16:48:45.082195 22451 options_loader.go:130] executing builder *components.DefaultsOptionsBuilder
I0627 16:48:45.082392 22451 options_loader.go:130] executing builder *components.EtcdOptionsBuilder
I0627 16:48:45.082662 22451 options_loader.go:130] executing builder *etcdmanager.EtcdManagerOptionsBuilder
I0627 16:48:45.082895 22451 options_loader.go:130] executing builder *nodeauthorizer.OptionsBuilder
I0627 16:48:45.083141 22451 options_loader.go:130] executing builder *components.KubeAPIServerOptionsBuilder
I0627 16:48:45.083415 22451 options_loader.go:130] executing builder *components.DockerOptionsBuilder
I0627 16:48:45.083613 22451 options_loader.go:130] executing builder *components.NetworkingOptionsBuilder
I0627 16:48:45.083836 22451 options_loader.go:130] executing builder *components.KubeDnsOptionsBuilder
I0627 16:48:45.084029 22451 options_loader.go:130] executing builder *components.KubeletOptionsBuilder
I0627 16:48:45.084274 22451 kubelet.go:171] Cloud Provider: aws
I0627 16:48:45.084480 22451 options_loader.go:130] executing builder *components.KubeControllerManagerOptionsBuilder
I0627 16:48:45.084685 22451 kubecontrollermanager.go:74] Kubernetes version "1.12.9" supports AttachDetachReconcileSyncPeriod; will configure
I0627 16:48:45.084953 22451 options_loader.go:130] executing builder *components.KubeSchedulerOptionsBuilder
I0627 16:48:45.085153 22451 options_loader.go:130] executing builder *components.KubeProxyOptionsBuilder
I0627 16:48:45.089036 22451 spec_builder.go:49] options: {
"channel": "stable",
"configBase": "s3://my-state-store/dev-qa.k8s.domain.com",
"cloudProvider": "aws",
"kubernetesVersion": "1.12.9",
"subnets": [
{
"name": "eu-central-1a",
"cidr": "172.31.239.32/27",
"zone": "eu-central-1a",
"type": "Private"
},
{
"name": "eu-central-1b",
"cidr": "172.31.239.64/27",
"zone": "eu-central-1b",
"type": "Private"
},
{
"name": "eu-central-1c",
"cidr": "172.31.239.96/27",
"zone": "eu-central-1c",
"type": "Private"
},
{
"name": "utility-eu-central-1a",
"cidr": "172.31.239.0/30",
"zone": "eu-central-1a",
"type": "Utility"
},
{
"name": "utility-eu-central-1b",
"cidr": "172.31.239.4/30",
"zone": "eu-central-1b",
"type": "Utility"
},
{
"name": "utility-eu-central-1c",
"cidr": "172.31.239.8/30",
"zone": "eu-central-1c",
"type": "Utility"
}
],
"masterPublicName": "api.dev-qa.k8s.domain.com",
"masterInternalName": "api.internal.dev-qa.k8s.domain.com",
"networkCIDR": "172.31.239.0/24",
"topology": {
"masters": "private",
"nodes": "private",
"bastion": {
"bastionPublicName": "bastion.dev-qa.k8s.domain.com"
},
"dns": {
"type": "Public"
}
},
"secretStore": "s3://my-state-store/dev-qa.k8s.domain.com/secrets",
"keyStore": "s3://my-state-store/dev-qa.k8s.domain.com/pki",
"configStore": "s3://my-state-store/dev-qa.k8s.domain.com",
"dnsZone": "k8s.domain.com",
"clusterDNSDomain": "cluster.local",
"serviceClusterIPRange": "100.64.0.0/13",
"nonMasqueradeCIDR": "100.64.0.0/10",
"sshAccess": [
"0.0.0.0/0"
],
"kubernetesApiAccess": [
"0.0.0.0/0"
],
"etcdClusters": [
{
"name": "main",
"provider": "Manager",
"etcdMembers": [
{
"name": "a",
"instanceGroup": "master-eu-central-1a",
"encryptedVolume": true
},
{
"name": "b",
"instanceGroup": "master-eu-central-1b",
"encryptedVolume": true
},
{
"name": "c",
"instanceGroup": "master-eu-central-1c",
"encryptedVolume": true
}
],
"enableEtcdTLS": true,
"enableTLSAuth": true,
"version": "3.2.24",
"backups": {
"backupStore": "s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main"
},
"manager": {},
"memoryRequest": "100Mi",
"cpuRequest": "200m"
},
{
"name": "events",
"provider": "Manager",
"etcdMembers": [
{
"name": "a",
"instanceGroup": "master-eu-central-1a",
"encryptedVolume": true
},
{
"name": "b",
"instanceGroup": "master-eu-central-1b",
"encryptedVolume": true
},
{
"name": "c",
"instanceGroup": "master-eu-central-1c",
"encryptedVolume": true
}
],
"enableEtcdTLS": true,
"enableTLSAuth": true,
"version": "3.2.24",
"backups": {
"backupStore": "s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events"
},
"manager": {},
"memoryRequest": "100Mi",
"cpuRequest": "100m"
}
],
"docker": {
"ipMasq": false,
"ipTables": false,
"logDriver": "json-file",
"logLevel": "warn",
"logOpt": [
"max-size=10m",
"max-file=5"
],
"storage": "overlay2,overlay,aufs",
"version": "18.06.3"
},
"kubeDNS": {
"cacheMaxSize": 1000,
"cacheMaxConcurrent": 150,
"domain": "cluster.local",
"replicas": 2,
"serverIP": "100.64.0.10",
"memoryRequest": "70Mi",
"cpuRequest": "100m",
"memoryLimit": "170Mi"
},
"kubeAPIServer": {
"image": "k8s.gcr.io/kube-apiserver:v1.12.9",
"logLevel": 2,
"cloudProvider": "aws",
"securePort": 443,
"insecurePort": 8080,
"bindAddress": "0.0.0.0",
"insecureBindAddress": "127.0.0.1",
"enableAdmissionPlugins": [
"NamespaceLifecycle",
"LimitRanger",
"ServiceAccount",
"PersistentVolumeLabel",
"DefaultStorageClass",
"DefaultTolerationSeconds",
"MutatingAdmissionWebhook",
"ValidatingAdmissionWebhook",
"NodeRestriction",
"ResourceQuota"
],
"serviceClusterIPRange": "100.64.0.0/13",
"etcdServers": [
"http://127.0.0.1:4001"
],
"etcdServersOverrides": [
"/events#http://127.0.0.1:4002"
],
"allowPrivileged": true,
"apiServerCount": 3,
"anonymousAuth": false,
"kubeletPreferredAddressTypes": [
"InternalIP",
"Hostname",
"ExternalIP"
],
"storageBackend": "etcd3",
"authorizationMode": "RBAC",
"requestheaderUsernameHeaders": [
"X-Remote-User"
],
"requestheaderGroupHeaders": [
"X-Remote-Group"
],
"requestheaderExtraHeaderPrefixes": [
"X-Remote-Extra-"
],
"requestheaderAllowedNames": [
"aggregator"
]
},
"kubeControllerManager": {
"logLevel": 2,
"image": "k8s.gcr.io/kube-controller-manager:v1.12.9",
"cloudProvider": "aws",
"clusterName": "dev-qa.k8s.domain.com",
"clusterCIDR": "100.96.0.0/11",
"allocateNodeCIDRs": true,
"configureCloudRoutes": false,
"leaderElection": {
"leaderElect": true
},
"attachDetachReconcileSyncPeriod": "1m0s",
"useServiceAccountCredentials": true
},
"kubeScheduler": {
"logLevel": 2,
"image": "k8s.gcr.io/kube-scheduler:v1.12.9",
"leaderElection": {
"leaderElect": true
}
},
"kubeProxy": {
"image": "k8s.gcr.io/kube-proxy:v1.12.9",
"cpuRequest": "100m",
"logLevel": 2,
"clusterCIDR": "100.96.0.0/11",
"hostnameOverride": "@aws"
},
"kubelet": {
"anonymousAuth": false,
"kubeconfigPath": "/var/lib/kubelet/kubeconfig",
"logLevel": 2,
"podManifestPath": "/etc/kubernetes/manifests",
"hostnameOverride": "@aws",
"podInfraContainerImage": "k8s.gcr.io/pause-amd64:3.0",
"allowPrivileged": true,
"enableDebuggingHandlers": true,
"clusterDomain": "cluster.local",
"clusterDNS": "100.64.0.10",
"networkPluginName": "cni",
"cloudProvider": "aws",
"cgroupRoot": "/",
"nonMasqueradeCIDR": "100.64.0.0/10",
"evictionHard": "memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%",
"featureGates": {
"ExperimentalCriticalPodAnnotation": "true"
}
},
"masterKubelet": {
"anonymousAuth": false,
"kubeconfigPath": "/var/lib/kubelet/kubeconfig",
"logLevel": 2,
"podManifestPath": "/etc/kubernetes/manifests",
"hostnameOverride": "@aws",
"podInfraContainerImage": "k8s.gcr.io/pause-amd64:3.0",
"allowPrivileged": true,
"enableDebuggingHandlers": true,
"clusterDomain": "cluster.local",
"clusterDNS": "100.64.0.10",
"networkPluginName": "cni",
"cloudProvider": "aws",
"cgroupRoot": "/",
"registerSchedulable": false,
"nonMasqueradeCIDR": "100.64.0.0/10",
"evictionHard": "memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%",
"featureGates": {
"ExperimentalCriticalPodAnnotation": "true"
}
},
"networking": {
"calico": {
"majorVersion": "v3"
}
},
"api": {
"loadBalancer": {
"type": "Public"
}
},
"authorization": {
"rbac": {}
},
"iam": {
"legacy": false,
"allowContainerRegistry": true
}
}
I0627 16:48:45.149283 22451 aws_cloud.go:1212] Querying EC2 for all valid zones in region "eu-central-1"
I0627 16:48:45.150220 22451 request_logger.go:45] AWS request: ec2/DescribeAvailabilityZones
I0627 16:48:45.384938 22451 aws_cloud.go:1390] checking if instance type "t2.micro" is supported in region "eu-central-1"
I0627 16:48:45.385876 22451 request_logger.go:45] AWS request: ec2/DescribeReservedInstancesOfferings
I0627 16:48:45.869614 22451 aws.go:99] machineType m5a.2xlarge requires an image based on stretch to operate. Trying to check compatibility
I0627 16:48:45.874647 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/config"
I0627 16:48:46.069471 22451 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/config"
I0627 16:48:46.070336 22451 s3context.go:238] Checking default bucket encryption for "my-state-store"
I0627 16:48:46.070969 22451 s3context.go:243] Calling S3 GetBucketEncryption Bucket="my-state-store"
I0627 16:48:46.253794 22451 s3context.go:250] Unable to read bucket encryption policy for "my-state-store": will encrypt using AES256
I0627 16:48:46.254428 22451 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/config" SSE="AES256" ACL=""
I0627 16:48:46.509676 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1a"
I0627 16:48:46.692914 22451 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1a"
I0627 16:48:46.693646 22451 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/master-eu-central-1a" SSE="AES256" ACL=""
I0627 16:48:46.900407 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1b"
I0627 16:48:47.090535 22451 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1b"
I0627 16:48:47.090948 22451 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/master-eu-central-1b" SSE="AES256" ACL=""
I0627 16:48:47.293995 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1c"
I0627 16:48:47.478728 22451 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1c"
I0627 16:48:47.479413 22451 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/master-eu-central-1c" SSE="AES256" ACL=""
I0627 16:48:47.702646 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/nodes"
I0627 16:48:47.889663 22451 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/nodes"
I0627 16:48:47.890475 22451 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/nodes" SSE="AES256" ACL=""
I0627 16:48:48.097145 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/bastions"
I0627 16:48:48.298959 22451 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/bastions"
I0627 16:48:48.299519 22451 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/bastions" SSE="AES256" ACL=""
I0627 16:48:48.553423 22451 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/cluster.spec"
I0627 16:48:48.554314 22451 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/cluster.spec" SSE="AES256" ACL=""
I0627 16:48:48.785398 22451 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/ssh/public/admin/e4d52e2ca78fef469afe377a974f975b"
I0627 16:48:48.786112 22451 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/ssh/public/admin/e4d52e2ca78fef469afe377a974f975b" SSE="AES256" ACL=""
Previewing changes that will be made:
I0627 16:48:48.983458 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/config"
I0627 16:48:49.175778 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/instancegroup/"
I0627 16:48:49.364733 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/instancegroup: [s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/bastions s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1a s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1b s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1c s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/nodes]
I0627 16:48:49.366172 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/bastions"
I0627 16:48:49.571405 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1a"
I0627 16:48:49.758850 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1b"
I0627 16:48:49.960958 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1c"
I0627 16:48:50.168190 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/nodes"
I0627 16:48:50.368053 22451 channel.go:97] resolving "stable" against default channel location "https://raw.githubusercontent.com/kubernetes/kops/master/channels/"
I0627 16:48:50.369246 22451 channel.go:102] Loading channel from "https://raw.githubusercontent.com/kubernetes/kops/master/channels/stable"
I0627 16:48:50.369702 22451 context.go:159] Performing HTTP request: GET https://raw.githubusercontent.com/kubernetes/kops/master/channels/stable
I0627 16:48:50.376360 22451 channel.go:111] Channel contents: spec:
images:
# We put the "legacy" version first, for kops versions that don't support versions ( < 1.5.0 )
- name: kope.io/k8s-1.4-debian-jessie-amd64-hvm-ebs-2017-07-28
providerID: aws
kubernetesVersion: ">=1.4.0 <1.5.0"
- name: kope.io/k8s-1.5-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.5.0 <1.6.0"
- name: kope.io/k8s-1.6-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.6.0 <1.7.0"
- name: kope.io/k8s-1.7-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.7.0 <1.8.0"
- name: kope.io/k8s-1.8-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.8.0 <1.9.0"
- name: kope.io/k8s-1.9-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.9.0 <1.10.0"
- name: kope.io/k8s-1.10-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.10.0 <1.11.0"
# Stretch is the default for 1.11 (for nvme)
- name: kope.io/k8s-1.11-debian-stretch-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.11.0 <1.12.0"
- name: kope.io/k8s-1.12-debian-stretch-amd64-hvm-ebs-2019-05-13
providerID: aws
kubernetesVersion: ">=1.12.0"
- providerID: gce
name: "cos-cloud/cos-stable-65-10323-99-0"
cluster:
kubernetesVersion: v1.5.8
networking:
kubenet: {}
kubernetesVersions:
- range: ">=1.14.0"
recommendedVersion: 1.14.1
requiredVersion: 1.14.0
- range: ">=1.13.0"
recommendedVersion: 1.13.5
requiredVersion: 1.13.0
- range: ">=1.12.0"
recommendedVersion: 1.12.8
requiredVersion: 1.12.0
- range: ">=1.11.0"
recommendedVersion: 1.11.10
requiredVersion: 1.11.0
- range: ">=1.10.0"
recommendedVersion: 1.10.13
requiredVersion: 1.10.0
- range: ">=1.9.0"
recommendedVersion: 1.9.11
requiredVersion: 1.9.0
- range: ">=1.8.0"
recommendedVersion: 1.8.15
requiredVersion: 1.8.0
- range: ">=1.7.0"
recommendedVersion: 1.7.16
requiredVersion: 1.7.0
- range: ">=1.6.0"
recommendedVersion: 1.6.13
requiredVersion: 1.6.0
- range: ">=1.5.0"
recommendedVersion: 1.5.8
requiredVersion: 1.5.1
- range: "<1.5.0"
recommendedVersion: 1.4.12
requiredVersion: 1.4.2
kopsVersions:
- range: ">=1.14.0-alpha.1"
#recommendedVersion: "1.14.0"
#requiredVersion: 1.14.0
kubernetesVersion: 1.14.1
- range: ">=1.13.0-alpha.1"
#recommendedVersion: "1.13.0"
#requiredVersion: 1.13.0
kubernetesVersion: 1.13.5
- range: ">=1.12.0-alpha.1"
recommendedVersion: "1.12.1"
#requiredVersion: 1.12.0
kubernetesVersion: 1.12.8
- range: ">=1.11.0-alpha.1"
#recommendedVersion: "1.11.0"
#requiredVersion: 1.11.0
kubernetesVersion: 1.11.10
- range: ">=1.10.0-alpha.1"
recommendedVersion: "1.10.0"
#requiredVersion: 1.10.0
kubernetesVersion: 1.10.13
- range: ">=1.9.0-alpha.1"
recommendedVersion: 1.9.2
#requiredVersion: 1.9.0
kubernetesVersion: 1.9.11
- range: ">=1.8.0-alpha.1"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.8.15
- range: ">=1.7.0-alpha.1"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.7.16
- range: ">=1.6.0-alpha.1"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.6.13
- range: ">=1.5.0-alpha1"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.5.8
- range: "<1.5.0"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.4.12
I0627 16:48:50.399236 22451 populate_cluster_spec.go:371] Defaulted KubeControllerManager.ClusterCIDR to 100.96.0.0/11
I0627 16:48:50.399428 22451 populate_cluster_spec.go:378] Defaulted ServiceClusterIPRange to 100.64.0.0/13
I0627 16:48:50.399688 22451 aws_cloud.go:1212] Querying EC2 for all valid zones in region "eu-central-1"
I0627 16:48:50.404681 22451 request_logger.go:45] AWS request: ec2/DescribeAvailabilityZones
I0627 16:48:50.619700 22451 subnets.go:49] All subnets have CIDRs; skipping assignment logic
I0627 16:48:50.620511 22451 defaults.go:213] Not setting up Proxy Excludes
I0627 16:48:50.620879 22451 aws_cloud.go:1212] Querying EC2 for all valid zones in region "eu-central-1"
I0627 16:48:50.621422 22451 request_logger.go:45] AWS request: ec2/DescribeAvailabilityZones
I0627 16:48:50.838979 22451 tagbuilder.go:95] tags: [_aws _k8s_1_6]
I0627 16:48:50.840226 22451 options_loader.go:130] executing builder *components.DefaultsOptionsBuilder
I0627 16:48:50.840489 22451 options_loader.go:130] executing builder *components.EtcdOptionsBuilder
I0627 16:48:50.840733 22451 options_loader.go:130] executing builder *etcdmanager.EtcdManagerOptionsBuilder
I0627 16:48:50.840940 22451 options_loader.go:130] executing builder *nodeauthorizer.OptionsBuilder
I0627 16:48:50.841157 22451 options_loader.go:130] executing builder *components.KubeAPIServerOptionsBuilder
I0627 16:48:50.841430 22451 options_loader.go:130] executing builder *components.DockerOptionsBuilder
I0627 16:48:50.841668 22451 options_loader.go:130] executing builder *components.NetworkingOptionsBuilder
I0627 16:48:50.841918 22451 options_loader.go:130] executing builder *components.KubeDnsOptionsBuilder
I0627 16:48:50.842117 22451 options_loader.go:130] executing builder *components.KubeletOptionsBuilder
I0627 16:48:50.842385 22451 kubelet.go:171] Cloud Provider: aws
I0627 16:48:50.842624 22451 options_loader.go:130] executing builder *components.KubeControllerManagerOptionsBuilder
I0627 16:48:50.842894 22451 kubecontrollermanager.go:74] Kubernetes version "1.12.9" supports AttachDetachReconcileSyncPeriod; will configure
I0627 16:48:50.843148 22451 kubecontrollermanager.go:79] AttachDetachReconcileSyncPeriod is not set; will set to default 1m0s
I0627 16:48:50.843406 22451 options_loader.go:130] executing builder *components.KubeSchedulerOptionsBuilder
I0627 16:48:50.843660 22451 options_loader.go:130] executing builder *components.KubeProxyOptionsBuilder
I0627 16:48:50.844279 22451 options_loader.go:130] executing builder *components.DefaultsOptionsBuilder
I0627 16:48:50.844578 22451 options_loader.go:130] executing builder *components.EtcdOptionsBuilder
I0627 16:48:50.844808 22451 options_loader.go:130] executing builder *etcdmanager.EtcdManagerOptionsBuilder
I0627 16:48:50.845091 22451 options_loader.go:130] executing builder *nodeauthorizer.OptionsBuilder
I0627 16:48:50.845317 22451 options_loader.go:130] executing builder *components.KubeAPIServerOptionsBuilder
I0627 16:48:50.845532 22451 options_loader.go:130] executing builder *components.DockerOptionsBuilder
I0627 16:48:50.845769 22451 options_loader.go:130] executing builder *components.NetworkingOptionsBuilder
I0627 16:48:50.845943 22451 options_loader.go:130] executing builder *components.KubeDnsOptionsBuilder
I0627 16:48:50.846112 22451 options_loader.go:130] executing builder *components.KubeletOptionsBuilder
I0627 16:48:50.846369 22451 kubelet.go:171] Cloud Provider: aws
I0627 16:48:50.846518 22451 options_loader.go:130] executing builder *components.KubeControllerManagerOptionsBuilder
I0627 16:48:50.846696 22451 kubecontrollermanager.go:74] Kubernetes version "1.12.9" supports AttachDetachReconcileSyncPeriod; will configure
I0627 16:48:50.846971 22451 options_loader.go:130] executing builder *components.KubeSchedulerOptionsBuilder
I0627 16:48:50.847155 22451 options_loader.go:130] executing builder *components.KubeProxyOptionsBuilder
I0627 16:48:50.847781 22451 spec_builder.go:49] options: {
"channel": "stable",
"configBase": "s3://my-state-store/dev-qa.k8s.domain.com",
"cloudProvider": "aws",
"kubernetesVersion": "1.12.9",
"subnets": [
{
"name": "eu-central-1a",
"cidr": "172.31.239.32/27",
"zone": "eu-central-1a",
"type": "Private"
},
{
"name": "eu-central-1b",
"cidr": "172.31.239.64/27",
"zone": "eu-central-1b",
"type": "Private"
},
{
"name": "eu-central-1c",
"cidr": "172.31.239.96/27",
"zone": "eu-central-1c",
"type": "Private"
},
{
"name": "utility-eu-central-1a",
"cidr": "172.31.239.0/30",
"zone": "eu-central-1a",
"type": "Utility"
},
{
"name": "utility-eu-central-1b",
"cidr": "172.31.239.4/30",
"zone": "eu-central-1b",
"type": "Utility"
},
{
"name": "utility-eu-central-1c",
"cidr": "172.31.239.8/30",
"zone": "eu-central-1c",
"type": "Utility"
}
],
"masterPublicName": "api.dev-qa.k8s.domain.com",
"masterInternalName": "api.internal.dev-qa.k8s.domain.com",
"networkCIDR": "172.31.239.0/24",
"topology": {
"masters": "private",
"nodes": "private",
"bastion": {
"bastionPublicName": "bastion.dev-qa.k8s.domain.com"
},
"dns": {
"type": "Public"
}
},
"secretStore": "s3://my-state-store/dev-qa.k8s.domain.com/secrets",
"keyStore": "s3://my-state-store/dev-qa.k8s.domain.com/pki",
"configStore": "s3://my-state-store/dev-qa.k8s.domain.com",
"dnsZone": "k8s.domain.com",
"clusterDNSDomain": "cluster.local",
"serviceClusterIPRange": "100.64.0.0/13",
"nonMasqueradeCIDR": "100.64.0.0/10",
"sshAccess": [
"0.0.0.0/0"
],
"kubernetesApiAccess": [
"0.0.0.0/0"
],
"etcdClusters": [
{
"name": "main",
"provider": "Manager",
"etcdMembers": [
{
"name": "a",
"instanceGroup": "master-eu-central-1a",
"encryptedVolume": true
},
{
"name": "b",
"instanceGroup": "master-eu-central-1b",
"encryptedVolume": true
},
{
"name": "c",
"instanceGroup": "master-eu-central-1c",
"encryptedVolume": true
}
],
"enableEtcdTLS": true,
"enableTLSAuth": true,
"version": "3.2.24",
"backups": {
"backupStore": "s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main"
},
"manager": {},
"memoryRequest": "100Mi",
"cpuRequest": "200m"
},
{
"name": "events",
"provider": "Manager",
"etcdMembers": [
{
"name": "a",
"instanceGroup": "master-eu-central-1a",
"encryptedVolume": true
},
{
"name": "b",
"instanceGroup": "master-eu-central-1b",
"encryptedVolume": true
},
{
"name": "c",
"instanceGroup": "master-eu-central-1c",
"encryptedVolume": true
}
],
"enableEtcdTLS": true,
"enableTLSAuth": true,
"version": "3.2.24",
"backups": {
"backupStore": "s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events"
},
"manager": {},
"memoryRequest": "100Mi",
"cpuRequest": "100m"
}
],
"docker": {
"ipMasq": false,
"ipTables": false,
"logDriver": "json-file",
"logLevel": "warn",
"logOpt": [
"max-size=10m",
"max-file=5"
],
"storage": "overlay2,overlay,aufs",
"version": "18.06.3"
},
"kubeDNS": {
"cacheMaxSize": 1000,
"cacheMaxConcurrent": 150,
"domain": "cluster.local",
"replicas": 2,
"serverIP": "100.64.0.10",
"memoryRequest": "70Mi",
"cpuRequest": "100m",
"memoryLimit": "170Mi"
},
"kubeAPIServer": {
"image": "k8s.gcr.io/kube-apiserver:v1.12.9",
"logLevel": 2,
"cloudProvider": "aws",
"securePort": 443,
"insecurePort": 8080,
"bindAddress": "0.0.0.0",
"insecureBindAddress": "127.0.0.1",
"enableAdmissionPlugins": [
"NamespaceLifecycle",
"LimitRanger",
"ServiceAccount",
"PersistentVolumeLabel",
"DefaultStorageClass",
"DefaultTolerationSeconds",
"MutatingAdmissionWebhook",
"ValidatingAdmissionWebhook",
"NodeRestriction",
"ResourceQuota"
],
"serviceClusterIPRange": "100.64.0.0/13",
"etcdServers": [
"http://127.0.0.1:4001"
],
"etcdServersOverrides": [
"/events#http://127.0.0.1:4002"
],
"allowPrivileged": true,
"apiServerCount": 3,
"anonymousAuth": false,
"kubeletPreferredAddressTypes": [
"InternalIP",
"Hostname",
"ExternalIP"
],
"storageBackend": "etcd3",
"authorizationMode": "RBAC",
"requestheaderUsernameHeaders": [
"X-Remote-User"
],
"requestheaderGroupHeaders": [
"X-Remote-Group"
],
"requestheaderExtraHeaderPrefixes": [
"X-Remote-Extra-"
],
"requestheaderAllowedNames": [
"aggregator"
]
},
"kubeControllerManager": {
"logLevel": 2,
"image": "k8s.gcr.io/kube-controller-manager:v1.12.9",
"cloudProvider": "aws",
"clusterName": "dev-qa.k8s.domain.com",
"clusterCIDR": "100.96.0.0/11",
"allocateNodeCIDRs": true,
"configureCloudRoutes": false,
"leaderElection": {
"leaderElect": true
},
"attachDetachReconcileSyncPeriod": "1m0s",
"useServiceAccountCredentials": true
},
"kubeScheduler": {
"logLevel": 2,
"image": "k8s.gcr.io/kube-scheduler:v1.12.9",
"leaderElection": {
"leaderElect": true
}
},
"kubeProxy": {
"image": "k8s.gcr.io/kube-proxy:v1.12.9",
"cpuRequest": "100m",
"logLevel": 2,
"clusterCIDR": "100.96.0.0/11",
"hostnameOverride": "@aws"
},
"kubelet": {
"anonymousAuth": false,
"kubeconfigPath": "/var/lib/kubelet/kubeconfig",
"logLevel": 2,
"podManifestPath": "/etc/kubernetes/manifests",
"hostnameOverride": "@aws",
"podInfraContainerImage": "k8s.gcr.io/pause-amd64:3.0",
"allowPrivileged": true,
"enableDebuggingHandlers": true,
"clusterDomain": "cluster.local",
"clusterDNS": "100.64.0.10",
"networkPluginName": "cni",
"cloudProvider": "aws",
"cgroupRoot": "/",
"nonMasqueradeCIDR": "100.64.0.0/10",
"evictionHard": "memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%",
"featureGates": {
"ExperimentalCriticalPodAnnotation": "true"
}
},
"masterKubelet": {
"anonymousAuth": false,
"kubeconfigPath": "/var/lib/kubelet/kubeconfig",
"logLevel": 2,
"podManifestPath": "/etc/kubernetes/manifests",
"hostnameOverride": "@aws",
"podInfraContainerImage": "k8s.gcr.io/pause-amd64:3.0",
"allowPrivileged": true,
"enableDebuggingHandlers": true,
"clusterDomain": "cluster.local",
"clusterDNS": "100.64.0.10",
"networkPluginName": "cni",
"cloudProvider": "aws",
"cgroupRoot": "/",
"registerSchedulable": false,
"nonMasqueradeCIDR": "100.64.0.0/10",
"evictionHard": "memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%",
"featureGates": {
"ExperimentalCriticalPodAnnotation": "true"
}
},
"networking": {
"calico": {
"majorVersion": "v3"
}
},
"api": {
"loadBalancer": {
"type": "Public"
}
},
"authorization": {
"rbac": {}
},
"iam": {
"legacy": false,
"allowContainerRegistry": true
}
}
I0627 16:48:50.910225 22451 channel.go:239] version range ">=1.14.0-alpha.1" does not apply to version "1.12.1"; skipping
I0627 16:48:50.910443 22451 channel.go:239] version range ">=1.13.0-alpha.1" does not apply to version "1.12.1"; skipping
I0627 16:48:50.910720 22451 channel.go:162] RecommendedVersion="1.12.1", Have="1.12.1". No upgrade needed.
I0627 16:48:50.910947 22451 channel.go:190] VersionRecommendationSpec does not specify RequiredVersion
I0627 16:48:50.911141 22451 channel.go:218] version range ">=1.14.0" does not apply to version "1.12.9"; skipping
I0627 16:48:50.911476 22451 channel.go:218] version range ">=1.13.0" does not apply to version "1.12.9"; skipping
I0627 16:48:50.911702 22451 channel.go:142] RecommendedVersion="1.12.8", Have="1.12.9". No upgrade needed.
I0627 16:48:50.911994 22451 channel.go:182] RequiredVersion="1.12.0", Have="1.12.9". No upgrade needed.
I0627 16:48:50.912258 22451 aws.go:99] machineType m5a.2xlarge requires an image based on stretch to operate. Trying to check compatibility
I0627 16:48:50.915492 22451 context.go:159] Performing HTTP request: GET https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubelet.sha1
I0627 16:48:51.473328 22451 builder.go:301] Found hash "e914b17532c411cb7c0cc472131b61935fb66b31" for "https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubelet"
I0627 16:48:51.473889 22451 builder.go:234] adding file: &{FileURL:https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubelet CanonicalFileURL:<nil> SHAValue:e914b17532c411cb7c0cc472131b61935fb66b31}
I0627 16:48:51.482178 22451 context.go:159] Performing HTTP request: GET https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubectl.sha1
I0627 16:48:51.498978 22451 builder.go:301] Found hash "aa3e93897a6999d6c7dedbc41793c90d41eeb000" for "https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubectl"
I0627 16:48:51.499409 22451 builder.go:234] adding file: &{FileURL:https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubectl CanonicalFileURL:<nil> SHAValue:aa3e93897a6999d6c7dedbc41793c90d41eeb000}
I0627 16:48:51.499909 22451 networking.go:168] Adding default CNI asset for k8s >= 1.11: https://storage.googleapis.com/kubernetes-release/network-plugins/cni-plugins-amd64-v0.6.0.tgz
I0627 16:48:51.500342 22451 urls.go:81] Using default base url: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/"
I0627 16:48:51.500703 22451 context.go:159] Performing HTTP request: GET https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/utils.tar.gz.sha1
I0627 16:48:53.170355 22451 builder.go:301] Found hash "ad4085c05ff02c241a38ff0033d76c699316f298" for "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/utils.tar.gz"
I0627 16:48:53.171361 22451 builder.go:234] adding file: &{FileURL:https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/utils.tar.gz CanonicalFileURL:<nil> SHAValue:ad4085c05ff02c241a38ff0033d76c699316f298}
I0627 16:48:53.171954 22451 urls.go:73] Using cached kopsBaseUrl url: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/"
I0627 16:48:53.172364 22451 context.go:159] Performing HTTP request: GET https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/nodeup.sha1
I0627 16:48:53.420046 22451 builder.go:301] Found hash "8ff9c81e668969bfaf8fefb5c61f0b9fddc8ab44" for "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/nodeup"
I0627 16:48:53.420757 22451 builder.go:234] adding file: &{FileURL:https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/nodeup CanonicalFileURL:<nil> SHAValue:8ff9c81e668969bfaf8fefb5c61f0b9fddc8ab44}
I0627 16:48:53.421298 22451 urls.go:135] Using default nodeup location: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/nodeup"
I0627 16:48:53.421612 22451 urls.go:73] Using cached kopsBaseUrl url: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/"
I0627 16:48:53.422055 22451 context.go:159] Performing HTTP request: GET https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz.sha1
I0627 16:48:53.665505 22451 builder.go:301] Found hash "175d2d9df2b9e317a40749a6d735577546a3c38d" for "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz"
I0627 16:48:53.666309 22451 builder.go:234] adding file: &{FileURL:https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz CanonicalFileURL:<nil> SHAValue:175d2d9df2b9e317a40749a6d735577546a3c38d}
I0627 16:48:53.666776 22451 urls.go:172] Using default protokube location: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz"
I0627 16:48:53.667224 22451 aws_cloud.go:1212] Querying EC2 for all valid zones in region "eu-central-1"
I0627 16:48:53.667845 22451 request_logger.go:45] AWS request: ec2/DescribeAvailabilityZones
I0627 16:48:53.875898 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/ssh/public/admin/"
I0627 16:48:54.072130 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/ssh/public/admin: [s3://my-state-store/dev-qa.k8s.domain.com/pki/ssh/public/admin/e4d52e2ca78fef469afe377a974f975b]
I0627 16:48:54.072721 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/ssh/public/admin/e4d52e2ca78fef469afe377a974f975b"
I0627 16:48:54.266041 22451 route53.go:53] AWS request: route53 ListHostedZones
I0627 16:48:55.998571 22451 dns.go:102] Doing DNS lookup to verify NS records for "k8s.domain.com"
I0627 16:48:56.347685 22451 dns.go:119] Found NS records for "k8s.domain.com": [ns-1227.awsdns-25.org. ns-1578.awsdns-05.co.uk. ns-298.awsdns-37.com. ns-608.awsdns-12.net.]
I0627 16:48:56.348477 22451 tagbuilder.go:95] tags: [_aws _k8s_1_6]
I0627 16:48:56.349820 22451 templates.go:80] loading (templated) resource "addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:48:56.350202 22451 templates.go:80] loading (templated) resource "addons/kube-dns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.350697 22451 templates.go:80] loading (templated) resource "addons/kube-dns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:48:56.351227 22451 templates.go:80] loading (templated) resource "addons/networking.cilium.io/k8s-1.12.yaml"
I0627 16:48:56.351767 22451 templates.go:80] loading (templated) resource "addons/networking.cilium.io/k8s-1.7.yaml"
I0627 16:48:56.352301 22451 templates.go:80] loading (templated) resource "addons/spotinst-kubernetes-cluster-controller.addons.k8s.io/v1.9.0.yaml"
I0627 16:48:56.352784 22451 templates.go:80] loading (templated) resource "addons/spotinst-kubernetes-cluster-controller.addons.k8s.io/v1.8.0.yaml"
I0627 16:48:56.353600 22451 templates.go:88] loading resource "addons/storage-gce.addons.k8s.io/v1.6.0.yaml"
I0627 16:48:56.353927 22451 templates.go:88] loading resource "addons/storage-gce.addons.k8s.io/v1.7.0.yaml"
I0627 16:48:56.354254 22451 templates.go:88] loading resource "addons/limit-range.addons.k8s.io/addon.yaml"
I0627 16:48:56.354551 22451 templates.go:88] loading resource "addons/limit-range.addons.k8s.io/v1.5.0.yaml"
I0627 16:48:56.355093 22451 templates.go:80] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.7.yaml"
I0627 16:48:56.355750 22451 templates.go:80] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.8.yaml"
I0627 16:48:56.356549 22451 templates.go:80] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.10.yaml"
I0627 16:48:56.359900 22451 templates.go:80] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.12.yaml"
I0627 16:48:56.360437 22451 templates.go:80] loading (templated) resource "addons/digitalocean-cloud-controller.addons.k8s.io/k8s-1.8.yaml"
I0627 16:48:56.361122 22451 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.12.yaml"
I0627 16:48:56.361695 22451 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.6.yaml"
I0627 16:48:56.362247 22451 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.7-v3.yaml"
I0627 16:48:56.362830 22451 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.7.yaml"
I0627 16:48:56.363200 22451 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org/pre-k8s-1.6.yaml"
I0627 16:48:56.363591 22451 templates.go:80] loading (templated) resource "addons/node-authorizer.addons.k8s.io/k8s-1.10.yaml"
I0627 16:48:56.363931 22451 templates.go:80] loading (templated) resource "addons/node-authorizer.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.364248 22451 templates.go:80] loading (templated) resource "addons/networking.romana/k8s-1.12.yaml"
I0627 16:48:56.364552 22451 templates.go:80] loading (templated) resource "addons/networking.romana/k8s-1.7.yaml"
I0627 16:48:56.364856 22451 templates.go:80] loading (templated) resource "addons/podsecuritypolicy.addons.k8s.io/k8s-1.10.yaml"
I0627 16:48:56.365143 22451 templates.go:80] loading (templated) resource "addons/podsecuritypolicy.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.365478 22451 templates.go:80] loading (templated) resource "addons/podsecuritypolicy.addons.k8s.io/k8s-1.9.yaml"
I0627 16:48:56.365735 22451 templates.go:88] loading resource "addons/scheduler.addons.k8s.io/v1.7.0.yaml"
I0627 16:48:56.366026 22451 templates.go:80] loading (templated) resource "addons/dns-controller.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.366266 22451 templates.go:80] loading (templated) resource "addons/dns-controller.addons.k8s.io/k8s-1.6.yaml"
I0627 16:48:56.366553 22451 templates.go:80] loading (templated) resource "addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:48:56.366895 22451 templates.go:88] loading resource "addons/external-dns.addons.k8s.io/README.md"
I0627 16:48:56.367325 22451 templates.go:80] loading (templated) resource "addons/external-dns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.370845 22451 templates.go:80] loading (templated) resource "addons/external-dns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:48:56.371386 22451 templates.go:80] loading (templated) resource "addons/external-dns.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:48:56.371841 22451 templates.go:80] loading (templated) resource "addons/networking.weave/k8s-1.7.yaml"
I0627 16:48:56.372178 22451 templates.go:80] loading (templated) resource "addons/networking.weave/k8s-1.8.yaml"
I0627 16:48:56.372591 22451 templates.go:80] loading (templated) resource "addons/networking.weave/pre-k8s-1.6.yaml"
I0627 16:48:56.373009 22451 templates.go:80] loading (templated) resource "addons/networking.weave/k8s-1.12.yaml"
I0627 16:48:56.373372 22451 templates.go:80] loading (templated) resource "addons/networking.weave/k8s-1.6.yaml"
I0627 16:48:56.373678 22451 templates.go:88] loading resource "addons/rbac.addons.k8s.io/k8s-1.8.yaml"
I0627 16:48:56.374046 22451 templates.go:80] loading (templated) resource "addons/authentication.aws/k8s-1.10.yaml"
I0627 16:48:56.374304 22451 templates.go:80] loading (templated) resource "addons/authentication.aws/k8s-1.12.yaml"
I0627 16:48:56.374658 22451 templates.go:88] loading resource "addons/core.addons.k8s.io/addon.yaml"
I0627 16:48:56.374913 22451 templates.go:80] loading (templated) resource "addons/core.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.375243 22451 templates.go:80] loading (templated) resource "addons/core.addons.k8s.io/k8s-1.7.yaml"
I0627 16:48:56.375484 22451 templates.go:88] loading resource "addons/core.addons.k8s.io/v1.4.0.yaml"
I0627 16:48:56.375784 22451 templates.go:88] loading resource "addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml"
I0627 16:48:56.376059 22451 templates.go:80] loading (templated) resource "addons/openstack.addons.k8s.io/k8s-1.11.yaml"
I0627 16:48:56.376344 22451 templates.go:88] loading resource "addons/storage-aws.addons.k8s.io/v1.6.0.yaml"
I0627 16:48:56.376604 22451 templates.go:88] loading resource "addons/storage-aws.addons.k8s.io/v1.7.0.yaml"
I0627 16:48:56.376909 22451 templates.go:80] loading (templated) resource "addons/coredns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.377195 22451 templates.go:80] loading (templated) resource "addons/coredns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:48:56.377515 22451 templates.go:80] loading (templated) resource "addons/networking.flannel/k8s-1.12.yaml"
I0627 16:48:56.377797 22451 templates.go:80] loading (templated) resource "addons/networking.flannel/k8s-1.6.yaml"
I0627 16:48:56.378105 22451 templates.go:80] loading (templated) resource "addons/networking.flannel/pre-k8s-1.6.yaml"
I0627 16:48:56.378371 22451 templates.go:88] loading resource "addons/networking.kope.io/k8s-1.6.yaml"
I0627 16:48:56.378653 22451 templates.go:88] loading resource "addons/networking.kope.io/pre-k8s-1.6.yaml"
I0627 16:48:56.380731 22451 templates.go:88] loading resource "addons/networking.kope.io/k8s-1.12.yaml"
I0627 16:48:56.381138 22451 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.9.yaml"
I0627 16:48:56.381506 22451 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org.canal/pre-k8s-1.6.yaml"
I0627 16:48:56.382043 22451 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.12.yaml"
I0627 16:48:56.382529 22451 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.6.yaml"
I0627 16:48:56.383027 22451 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.8.yaml"
I0627 16:48:56.383415 22451 templates.go:88] loading resource "addons/authentication.kope.io/k8s-1.12.yaml"
I0627 16:48:56.383792 22451 templates.go:88] loading resource "addons/authentication.kope.io/k8s-1.8.yaml"
I0627 16:48:56.384122 22451 templates.go:80] loading (templated) resource "addons/networking.kuberouter/k8s-1.12.yaml"
I0627 16:48:56.384515 22451 templates.go:80] loading (templated) resource "addons/networking.kuberouter/k8s-1.6.yaml"
I0627 16:48:56.384858 22451 tree_walker.go:98] visit "cloudup/resources"
I0627 16:48:56.385270 22451 tree_walker.go:98] visit "cloudup/resources/addons"
I0627 16:48:56.385652 22451 tree_walker.go:98] visit "cloudup/resources/addons/limit-range.addons.k8s.io"
I0627 16:48:56.385966 22451 tree_walker.go:98] visit "cloudup/resources/addons/limit-range.addons.k8s.io/v1.5.0.yaml"
I0627 16:48:56.386275 22451 loader.go:362] loading resource "addons/limit-range.addons.k8s.io/v1.5.0.yaml"
I0627 16:48:56.386618 22451 tree_walker.go:98] visit "cloudup/resources/addons/limit-range.addons.k8s.io/addon.yaml"
I0627 16:48:56.387012 22451 loader.go:362] loading resource "addons/limit-range.addons.k8s.io/addon.yaml"
I0627 16:48:56.387247 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.amazon-vpc-routed-eni"
I0627 16:48:56.387511 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.amazon-vpc-routed-eni/k8s-1.10.yaml.template"
I0627 16:48:56.387873 22451 loader.go:354] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.10.yaml"
I0627 16:48:56.388124 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.amazon-vpc-routed-eni/k8s-1.12.yaml.template"
I0627 16:48:56.388531 22451 loader.go:354] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.12.yaml"
I0627 16:48:56.389263 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.amazon-vpc-routed-eni/k8s-1.7.yaml.template"
I0627 16:48:56.392193 22451 loader.go:354] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.7.yaml"
I0627 16:48:56.392551 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.amazon-vpc-routed-eni/k8s-1.8.yaml.template"
I0627 16:48:56.392975 22451 loader.go:354] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.8.yaml"
I0627 16:48:56.393291 22451 tree_walker.go:98] visit "cloudup/resources/addons/digitalocean-cloud-controller.addons.k8s.io"
I0627 16:48:56.393653 22451 tree_walker.go:98] visit "cloudup/resources/addons/digitalocean-cloud-controller.addons.k8s.io/k8s-1.8.yaml.template"
I0627 16:48:56.394004 22451 loader.go:354] loading (templated) resource "addons/digitalocean-cloud-controller.addons.k8s.io/k8s-1.8.yaml"
I0627 16:48:56.394230 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org"
I0627 16:48:56.394508 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org/k8s-1.12.yaml.template"
I0627 16:48:56.394932 22451 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.12.yaml"
I0627 16:48:56.395167 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org/k8s-1.6.yaml.template"
I0627 16:48:56.395615 22451 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.6.yaml"
I0627 16:48:56.395861 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org/k8s-1.7-v3.yaml.template"
I0627 16:48:56.396285 22451 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.7-v3.yaml"
I0627 16:48:56.396554 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org/k8s-1.7.yaml.template"
I0627 16:48:56.396952 22451 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.7.yaml"
I0627 16:48:56.397192 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org/pre-k8s-1.6.yaml.template"
I0627 16:48:56.397521 22451 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org/pre-k8s-1.6.yaml"
I0627 16:48:56.397757 22451 tree_walker.go:98] visit "cloudup/resources/addons/node-authorizer.addons.k8s.io"
I0627 16:48:56.397999 22451 tree_walker.go:98] visit "cloudup/resources/addons/node-authorizer.addons.k8s.io/k8s-1.10.yaml.template"
I0627 16:48:56.398269 22451 loader.go:354] loading (templated) resource "addons/node-authorizer.addons.k8s.io/k8s-1.10.yaml"
I0627 16:48:56.398555 22451 tree_walker.go:98] visit "cloudup/resources/addons/node-authorizer.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:48:56.398865 22451 loader.go:354] loading (templated) resource "addons/node-authorizer.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.399101 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.romana"
I0627 16:48:56.399266 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.romana/k8s-1.12.yaml.template"
I0627 16:48:56.399763 22451 loader.go:354] loading (templated) resource "addons/networking.romana/k8s-1.12.yaml"
I0627 16:48:56.400210 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.romana/k8s-1.7.yaml.template"
I0627 16:48:56.403695 22451 loader.go:354] loading (templated) resource "addons/networking.romana/k8s-1.7.yaml"
I0627 16:48:56.404021 22451 tree_walker.go:98] visit "cloudup/resources/addons/podsecuritypolicy.addons.k8s.io"
I0627 16:48:56.404267 22451 tree_walker.go:98] visit "cloudup/resources/addons/podsecuritypolicy.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:48:56.404636 22451 loader.go:354] loading (templated) resource "addons/podsecuritypolicy.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.404912 22451 tree_walker.go:98] visit "cloudup/resources/addons/podsecuritypolicy.addons.k8s.io/k8s-1.9.yaml.template"
I0627 16:48:56.405224 22451 loader.go:354] loading (templated) resource "addons/podsecuritypolicy.addons.k8s.io/k8s-1.9.yaml"
I0627 16:48:56.405490 22451 tree_walker.go:98] visit "cloudup/resources/addons/podsecuritypolicy.addons.k8s.io/k8s-1.10.yaml.template"
I0627 16:48:56.405714 22451 loader.go:354] loading (templated) resource "addons/podsecuritypolicy.addons.k8s.io/k8s-1.10.yaml"
I0627 16:48:56.405897 22451 tree_walker.go:98] visit "cloudup/resources/addons/scheduler.addons.k8s.io"
I0627 16:48:56.406077 22451 tree_walker.go:98] visit "cloudup/resources/addons/scheduler.addons.k8s.io/v1.7.0.yaml"
I0627 16:48:56.406308 22451 loader.go:362] loading resource "addons/scheduler.addons.k8s.io/v1.7.0.yaml"
I0627 16:48:56.406534 22451 tree_walker.go:98] visit "cloudup/resources/addons/dns-controller.addons.k8s.io"
I0627 16:48:56.406718 22451 tree_walker.go:98] visit "cloudup/resources/addons/dns-controller.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:48:56.406950 22451 loader.go:354] loading (templated) resource "addons/dns-controller.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.407130 22451 tree_walker.go:98] visit "cloudup/resources/addons/dns-controller.addons.k8s.io/k8s-1.6.yaml.template"
I0627 16:48:56.407516 22451 loader.go:354] loading (templated) resource "addons/dns-controller.addons.k8s.io/k8s-1.6.yaml"
I0627 16:48:56.407698 22451 tree_walker.go:98] visit "cloudup/resources/addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml.template"
I0627 16:48:56.407918 22451 loader.go:354] loading (templated) resource "addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:48:56.408099 22451 tree_walker.go:98] visit "cloudup/resources/addons/external-dns.addons.k8s.io"
I0627 16:48:56.408279 22451 tree_walker.go:98] visit "cloudup/resources/addons/external-dns.addons.k8s.io/README.md"
I0627 16:48:56.408536 22451 loader.go:362] loading resource "addons/external-dns.addons.k8s.io/README.md"
I0627 16:48:56.408709 22451 tree_walker.go:98] visit "cloudup/resources/addons/external-dns.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:48:56.408938 22451 loader.go:354] loading (templated) resource "addons/external-dns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.409123 22451 tree_walker.go:98] visit "cloudup/resources/addons/external-dns.addons.k8s.io/k8s-1.6.yaml.template"
I0627 16:48:56.409397 22451 loader.go:354] loading (templated) resource "addons/external-dns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:48:56.409613 22451 tree_walker.go:98] visit "cloudup/resources/addons/external-dns.addons.k8s.io/pre-k8s-1.6.yaml.template"
I0627 16:48:56.409878 22451 loader.go:354] loading (templated) resource "addons/external-dns.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:48:56.410067 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.weave"
I0627 16:48:56.410235 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.weave/pre-k8s-1.6.yaml.template"
I0627 16:48:56.410474 22451 loader.go:354] loading (templated) resource "addons/networking.weave/pre-k8s-1.6.yaml"
I0627 16:48:56.410658 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.weave/k8s-1.12.yaml.template"
I0627 16:48:56.410897 22451 loader.go:354] loading (templated) resource "addons/networking.weave/k8s-1.12.yaml"
I0627 16:48:56.411070 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.weave/k8s-1.6.yaml.template"
I0627 16:48:56.411348 22451 loader.go:354] loading (templated) resource "addons/networking.weave/k8s-1.6.yaml"
I0627 16:48:56.411592 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.weave/k8s-1.7.yaml.template"
I0627 16:48:56.411891 22451 loader.go:354] loading (templated) resource "addons/networking.weave/k8s-1.7.yaml"
I0627 16:48:56.412116 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.weave/k8s-1.8.yaml.template"
I0627 16:48:56.412461 22451 loader.go:354] loading (templated) resource "addons/networking.weave/k8s-1.8.yaml"
I0627 16:48:56.415011 22451 tree_walker.go:98] visit "cloudup/resources/addons/rbac.addons.k8s.io"
I0627 16:48:56.415239 22451 tree_walker.go:98] visit "cloudup/resources/addons/rbac.addons.k8s.io/k8s-1.8.yaml"
I0627 16:48:56.415814 22451 loader.go:362] loading resource "addons/rbac.addons.k8s.io/k8s-1.8.yaml"
I0627 16:48:56.416211 22451 tree_walker.go:98] visit "cloudup/resources/addons/authentication.aws"
I0627 16:48:56.416611 22451 tree_walker.go:98] visit "cloudup/resources/addons/authentication.aws/k8s-1.10.yaml.template"
I0627 16:48:56.417020 22451 loader.go:354] loading (templated) resource "addons/authentication.aws/k8s-1.10.yaml"
I0627 16:48:56.417259 22451 tree_walker.go:98] visit "cloudup/resources/addons/authentication.aws/k8s-1.12.yaml.template"
I0627 16:48:56.417623 22451 loader.go:354] loading (templated) resource "addons/authentication.aws/k8s-1.12.yaml"
I0627 16:48:56.417924 22451 tree_walker.go:98] visit "cloudup/resources/addons/core.addons.k8s.io"
I0627 16:48:56.418221 22451 tree_walker.go:98] visit "cloudup/resources/addons/core.addons.k8s.io/addon.yaml"
I0627 16:48:56.418629 22451 loader.go:362] loading resource "addons/core.addons.k8s.io/addon.yaml"
I0627 16:48:56.418834 22451 tree_walker.go:98] visit "cloudup/resources/addons/core.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:48:56.419148 22451 loader.go:354] loading (templated) resource "addons/core.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.419454 22451 tree_walker.go:98] visit "cloudup/resources/addons/core.addons.k8s.io/k8s-1.7.yaml.template"
I0627 16:48:56.419764 22451 loader.go:354] loading (templated) resource "addons/core.addons.k8s.io/k8s-1.7.yaml"
I0627 16:48:56.420121 22451 tree_walker.go:98] visit "cloudup/resources/addons/core.addons.k8s.io/v1.4.0.yaml"
I0627 16:48:56.420522 22451 loader.go:362] loading resource "addons/core.addons.k8s.io/v1.4.0.yaml"
I0627 16:48:56.420847 22451 tree_walker.go:98] visit "cloudup/resources/addons/kubelet-api.rbac.addons.k8s.io"
I0627 16:48:56.421178 22451 tree_walker.go:98] visit "cloudup/resources/addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml"
I0627 16:48:56.421612 22451 loader.go:362] loading resource "addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml"
I0627 16:48:56.421857 22451 tree_walker.go:98] visit "cloudup/resources/addons/openstack.addons.k8s.io"
I0627 16:48:56.422104 22451 tree_walker.go:98] visit "cloudup/resources/addons/openstack.addons.k8s.io/k8s-1.11.yaml.template"
I0627 16:48:56.422391 22451 loader.go:354] loading (templated) resource "addons/openstack.addons.k8s.io/k8s-1.11.yaml"
I0627 16:48:56.422572 22451 tree_walker.go:98] visit "cloudup/resources/addons/storage-aws.addons.k8s.io"
I0627 16:48:56.422754 22451 tree_walker.go:98] visit "cloudup/resources/addons/storage-aws.addons.k8s.io/v1.6.0.yaml"
I0627 16:48:56.422967 22451 loader.go:362] loading resource "addons/storage-aws.addons.k8s.io/v1.6.0.yaml"
I0627 16:48:56.423139 22451 tree_walker.go:98] visit "cloudup/resources/addons/storage-aws.addons.k8s.io/v1.7.0.yaml"
I0627 16:48:56.423389 22451 loader.go:362] loading resource "addons/storage-aws.addons.k8s.io/v1.7.0.yaml"
I0627 16:48:56.425644 22451 tree_walker.go:98] visit "cloudup/resources/addons/coredns.addons.k8s.io"
I0627 16:48:56.425835 22451 tree_walker.go:98] visit "cloudup/resources/addons/coredns.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:48:56.426088 22451 loader.go:354] loading (templated) resource "addons/coredns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.426280 22451 tree_walker.go:98] visit "cloudup/resources/addons/coredns.addons.k8s.io/k8s-1.6.yaml.template"
I0627 16:48:56.426609 22451 loader.go:354] loading (templated) resource "addons/coredns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:48:56.426797 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.flannel"
I0627 16:48:56.426966 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.flannel/k8s-1.6.yaml.template"
I0627 16:48:56.427212 22451 loader.go:354] loading (templated) resource "addons/networking.flannel/k8s-1.6.yaml"
I0627 16:48:56.427453 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.flannel/pre-k8s-1.6.yaml.template"
I0627 16:48:56.427784 22451 loader.go:354] loading (templated) resource "addons/networking.flannel/pre-k8s-1.6.yaml"
I0627 16:48:56.427967 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.flannel/k8s-1.12.yaml.template"
I0627 16:48:56.428202 22451 loader.go:354] loading (templated) resource "addons/networking.flannel/k8s-1.12.yaml"
I0627 16:48:56.428424 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.kope.io"
I0627 16:48:56.428594 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.kope.io/k8s-1.12.yaml"
I0627 16:48:56.428820 22451 loader.go:362] loading resource "addons/networking.kope.io/k8s-1.12.yaml"
I0627 16:48:56.428990 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.kope.io/k8s-1.6.yaml"
I0627 16:48:56.429225 22451 loader.go:362] loading resource "addons/networking.kope.io/k8s-1.6.yaml"
I0627 16:48:56.429449 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.kope.io/pre-k8s-1.6.yaml"
I0627 16:48:56.429669 22451 loader.go:362] loading resource "addons/networking.kope.io/pre-k8s-1.6.yaml"
I0627 16:48:56.429839 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org.canal"
I0627 16:48:56.430030 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.12.yaml.template"
I0627 16:48:56.430477 22451 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.12.yaml"
I0627 16:48:56.430661 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.6.yaml.template"
I0627 16:48:56.430988 22451 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.6.yaml"
I0627 16:48:56.431190 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.8.yaml.template"
I0627 16:48:56.431894 22451 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.8.yaml"
I0627 16:48:56.432339 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.9.yaml.template"
I0627 16:48:56.432866 22451 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.9.yaml"
I0627 16:48:56.433074 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org.canal/pre-k8s-1.6.yaml.template"
I0627 16:48:56.433409 22451 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org.canal/pre-k8s-1.6.yaml"
I0627 16:48:56.433653 22451 tree_walker.go:98] visit "cloudup/resources/addons/authentication.kope.io"
I0627 16:48:56.433862 22451 tree_walker.go:98] visit "cloudup/resources/addons/authentication.kope.io/k8s-1.12.yaml"
I0627 16:48:56.434221 22451 loader.go:362] loading resource "addons/authentication.kope.io/k8s-1.12.yaml"
I0627 16:48:56.437092 22451 tree_walker.go:98] visit "cloudup/resources/addons/authentication.kope.io/k8s-1.8.yaml"
I0627 16:48:56.437565 22451 loader.go:362] loading resource "addons/authentication.kope.io/k8s-1.8.yaml"
I0627 16:48:56.437788 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.kuberouter"
I0627 16:48:56.438010 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.kuberouter/k8s-1.12.yaml.template"
I0627 16:48:56.438379 22451 loader.go:354] loading (templated) resource "addons/networking.kuberouter/k8s-1.12.yaml"
I0627 16:48:56.438751 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.kuberouter/k8s-1.6.yaml.template"
I0627 16:48:56.439114 22451 loader.go:354] loading (templated) resource "addons/networking.kuberouter/k8s-1.6.yaml"
I0627 16:48:56.439338 22451 tree_walker.go:98] visit "cloudup/resources/addons/kube-dns.addons.k8s.io"
I0627 16:48:56.439613 22451 tree_walker.go:98] visit "cloudup/resources/addons/kube-dns.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:48:56.440017 22451 loader.go:354] loading (templated) resource "addons/kube-dns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.440244 22451 tree_walker.go:98] visit "cloudup/resources/addons/kube-dns.addons.k8s.io/k8s-1.6.yaml.template"
I0627 16:48:56.440628 22451 loader.go:354] loading (templated) resource "addons/kube-dns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:48:56.440884 22451 tree_walker.go:98] visit "cloudup/resources/addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml.template"
I0627 16:48:56.441232 22451 loader.go:354] loading (templated) resource "addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:48:56.441497 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.cilium.io"
I0627 16:48:56.441677 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.cilium.io/k8s-1.12.yaml.template"
I0627 16:48:56.442010 22451 loader.go:354] loading (templated) resource "addons/networking.cilium.io/k8s-1.12.yaml"
I0627 16:48:56.442189 22451 tree_walker.go:98] visit "cloudup/resources/addons/networking.cilium.io/k8s-1.7.yaml.template"
I0627 16:48:56.442589 22451 loader.go:354] loading (templated) resource "addons/networking.cilium.io/k8s-1.7.yaml"
I0627 16:48:56.442790 22451 tree_walker.go:98] visit "cloudup/resources/addons/spotinst-kubernetes-cluster-controller.addons.k8s.io"
I0627 16:48:56.442984 22451 tree_walker.go:98] visit "cloudup/resources/addons/spotinst-kubernetes-cluster-controller.addons.k8s.io/v1.9.0.yaml.template"
I0627 16:48:56.443324 22451 loader.go:354] loading (templated) resource "addons/spotinst-kubernetes-cluster-controller.addons.k8s.io/v1.9.0.yaml"
I0627 16:48:56.443546 22451 tree_walker.go:98] visit "cloudup/resources/addons/spotinst-kubernetes-cluster-controller.addons.k8s.io/v1.8.0.yaml.template"
I0627 16:48:56.443815 22451 loader.go:354] loading (templated) resource "addons/spotinst-kubernetes-cluster-controller.addons.k8s.io/v1.8.0.yaml"
I0627 16:48:56.444013 22451 tree_walker.go:98] visit "cloudup/resources/addons/storage-gce.addons.k8s.io"
I0627 16:48:56.444192 22451 tree_walker.go:98] visit "cloudup/resources/addons/storage-gce.addons.k8s.io/v1.7.0.yaml"
I0627 16:48:56.444463 22451 loader.go:362] loading resource "addons/storage-gce.addons.k8s.io/v1.7.0.yaml"
I0627 16:48:56.444635 22451 tree_walker.go:98] visit "cloudup/resources/addons/storage-gce.addons.k8s.io/v1.6.0.yaml"
I0627 16:48:56.444857 22451 loader.go:362] loading resource "addons/storage-gce.addons.k8s.io/v1.6.0.yaml"
I0627 16:48:56.448206 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.4.0"
I0627 16:48:56.448983 22451 visitor.go:40] float64 value at spec.strategy.rollingUpdate.maxUnavailable: 0.000000
I0627 16:48:56.449211 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13"
I0627 16:48:56.449400 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.timeoutSeconds: 5.000000
I0627 16:48:56.449591 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.failureThreshold: 5.000000
I0627 16:48:56.449842 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.httpGet.port: 10054.000000
I0627 16:48:56.450235 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:48:56.450634 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.successThreshold: 1.000000
I0627 16:48:56.451088 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[0].containerPort: 10053.000000
I0627 16:48:56.451395 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[1].containerPort: 10053.000000
I0627 16:48:56.451692 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[2].containerPort: 10055.000000
I0627 16:48:56.452131 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.httpGet.port: 8081.000000
I0627 16:48:56.452454 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.initialDelaySeconds: 3.000000
I0627 16:48:56.452802 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.timeoutSeconds: 5.000000
I0627 16:48:56.453075 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.13"
I0627 16:48:56.453263 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.failureThreshold: 5.000000
I0627 16:48:56.453449 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.httpGet.port: 10054.000000
I0627 16:48:56.453634 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:48:56.453874 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.successThreshold: 1.000000
I0627 16:48:56.454059 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.timeoutSeconds: 5.000000
I0627 16:48:56.454246 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].ports.[0].containerPort: 53.000000
I0627 16:48:56.454429 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].ports.[1].containerPort: 53.000000
I0627 16:48:56.454626 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.13"
I0627 16:48:56.454851 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.failureThreshold: 5.000000
I0627 16:48:56.455038 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.httpGet.port: 10054.000000
I0627 16:48:56.455222 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:48:56.455408 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.successThreshold: 1.000000
I0627 16:48:56.455592 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.timeoutSeconds: 5.000000
I0627 16:48:56.455806 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].ports.[0].containerPort: 10054.000000
I0627 16:48:56.458011 22451 visitor.go:35] string value at spec.template.spec.volumes.[0].configMap.optional: true
I0627 16:48:56.459260 22451 visitor.go:40] float64 value at spec.ports.[0].port: 53.000000
I0627 16:48:56.459425 22451 visitor.go:40] float64 value at spec.ports.[1].port: 53.000000
I0627 16:48:56.460745 22451 template_functions.go:190] watch-ingress=false set on dns-controller
I0627 16:48:56.461524 22451 visitor.go:40] float64 value at spec.replicas: 1.000000
I0627 16:48:56.461830 22451 images.go:59] Consider image for re-mapping: "kope/dns-controller:1.12.0"
I0627 16:48:56.462158 22451 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:48:56.474143 22451 images.go:59] Consider image for re-mapping: "quay.io/calico/node:v3.4.0"
I0627 16:48:56.474558 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.initialDelaySeconds: 10.000000
I0627 16:48:56.474964 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.periodSeconds: 10.000000
I0627 16:48:56.475351 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.failureThreshold: 6.000000
I0627 16:48:56.475713 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.httpGet.port: 9099.000000
I0627 16:48:56.476102 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.periodSeconds: 10.000000
I0627 16:48:56.476503 22451 visitor.go:35] string value at spec.template.spec.containers.[0].securityContext.privileged: true
I0627 16:48:56.476869 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[0].readOnly: true
I0627 16:48:56.478162 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[1].readOnly: false
I0627 16:48:56.478523 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[2].readOnly: false
I0627 16:48:56.478907 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[3].readOnly: true
I0627 16:48:56.479393 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[4].readOnly: true
I0627 16:48:56.479611 22451 images.go:59] Consider image for re-mapping: "quay.io/calico/cni:v3.4.0"
I0627 16:48:56.479932 22451 visitor.go:35] string value at spec.template.spec.containers.[1].volumeMounts.[2].readOnly: true
I0627 16:48:56.480200 22451 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:48:56.480436 22451 images.go:59] Consider image for re-mapping: "calico/upgrade:v1.0.5"
I0627 16:48:56.480766 22451 visitor.go:35] string value at spec.template.spec.initContainers.[0].volumeMounts.[0].readOnly: true
I0627 16:48:56.481529 22451 visitor.go:35] string value at spec.template.spec.initContainers.[0].volumeMounts.[1].readOnly: true
I0627 16:48:56.481861 22451 visitor.go:40] float64 value at spec.template.spec.terminationGracePeriodSeconds: 0.000000
I0627 16:48:56.482211 22451 visitor.go:40] float64 value at spec.updateStrategy.rollingUpdate.maxUnavailable: 1.000000
I0627 16:48:56.484245 22451 visitor.go:40] float64 value at spec.replicas: 1.000000
I0627 16:48:56.484778 22451 images.go:59] Consider image for re-mapping: "quay.io/calico/kube-controllers:v3.4.0"
I0627 16:48:56.485201 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[0].readOnly: true
I0627 16:48:56.485532 22451 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:48:56.485788 22451 images.go:59] Consider image for re-mapping: "calico/upgrade:v1.0.5"
I0627 16:48:56.486120 22451 visitor.go:35] string value at spec.template.spec.initContainers.[0].volumeMounts.[0].readOnly: true
I0627 16:48:56.486456 22451 visitor.go:35] string value at spec.template.spec.initContainers.[0].volumeMounts.[1].readOnly: true
I0627 16:48:56.488405 22451 images.go:59] Consider image for re-mapping: "calico/upgrade:v1.0.5"
I0627 16:48:56.489611 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[0].readOnly: true
I0627 16:48:56.490018 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[1].readOnly: true
I0627 16:48:56.490355 22451 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:48:56.495903 22451 visitor.go:40] float64 value at spec.replicas: 0.000000
I0627 16:48:56.496252 22451 images.go:59] Consider image for re-mapping: "quay.io/calico/kube-controllers:v3.4.0"
I0627 16:48:56.496569 22451 images.go:59] Consider image for re-mapping: "calico/upgrade:v1.0.5"
I0627 16:48:56.497094 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.periodSeconds: 10.000000
I0627 16:48:56.497405 22451 visitor.go:35] string value at spec.template.spec.containers.[0].securityContext.privileged: true
I0627 16:48:56.497680 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[0].readOnly: true
I0627 16:48:56.499092 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[1].readOnly: false
I0627 16:48:56.499416 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[2].readOnly: false
I0627 16:48:56.499693 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[3].readOnly: false
I0627 16:48:56.500038 22451 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[4].readOnly: true
I0627 16:48:56.500352 22451 images.go:59] Consider image for re-mapping: "quay.io/calico/node:v3.4.0"
I0627 16:48:56.500750 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.failureThreshold: 6.000000
I0627 16:48:56.501081 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.httpGet.port: 9099.000000
I0627 16:48:56.501407 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.initialDelaySeconds: 10.000000
I0627 16:48:56.501683 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.periodSeconds: 10.000000
I0627 16:48:56.501943 22451 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:48:56.502211 22451 images.go:59] Consider image for re-mapping: "quay.io/calico/cni:v3.4.0"
I0627 16:48:56.502452 22451 visitor.go:40] float64 value at spec.template.spec.terminationGracePeriodSeconds: 0.000000
I0627 16:48:56.502669 22451 visitor.go:40] float64 value at spec.updateStrategy.rollingUpdate.maxUnavailable: 1.000000
I0627 16:48:56.505295 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.0.0"
I0627 16:48:56.505903 22451 visitor.go:40] float64 value at spec.strategy.rollingUpdate.maxUnavailable: 0.000000
I0627 16:48:56.506157 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[0].containerPort: 10053.000000
I0627 16:48:56.506405 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[1].containerPort: 10053.000000
I0627 16:48:56.506663 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[2].containerPort: 10055.000000
I0627 16:48:56.507110 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.httpGet.port: 8081.000000
I0627 16:48:56.507414 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.initialDelaySeconds: 3.000000
I0627 16:48:56.509254 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.timeoutSeconds: 5.000000
I0627 16:48:56.509961 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/kubedns-amd64:1.9"
I0627 16:48:56.510838 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.failureThreshold: 5.000000
I0627 16:48:56.511330 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.httpGet.port: 8080.000000
I0627 16:48:56.511655 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:48:56.511955 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.successThreshold: 1.000000
I0627 16:48:56.512231 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.timeoutSeconds: 5.000000
I0627 16:48:56.512644 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-dnsmasq-amd64:1.14.10"
I0627 16:48:56.512953 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.httpGet.port: 8080.000000
I0627 16:48:56.513224 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:48:56.513555 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.successThreshold: 1.000000
I0627 16:48:56.513820 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.timeoutSeconds: 5.000000
I0627 16:48:56.514063 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.failureThreshold: 5.000000
I0627 16:48:56.514274 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].ports.[0].containerPort: 53.000000
I0627 16:48:56.514510 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].ports.[1].containerPort: 53.000000
I0627 16:48:56.514696 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/dnsmasq-metrics-amd64:1.0"
I0627 16:48:56.514871 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.failureThreshold: 5.000000
I0627 16:48:56.515055 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.httpGet.port: 10054.000000
I0627 16:48:56.515237 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:48:56.515466 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.successThreshold: 1.000000
I0627 16:48:56.515647 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.timeoutSeconds: 5.000000
I0627 16:48:56.515829 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].ports.[0].containerPort: 10054.000000
I0627 16:48:56.516011 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/exechealthz-amd64:1.2"
I0627 16:48:56.516185 22451 visitor.go:40] float64 value at spec.template.spec.containers.[3].ports.[0].containerPort: 8080.000000
I0627 16:48:56.517574 22451 visitor.go:40] float64 value at spec.ports.[0].port: 53.000000
I0627 16:48:56.520169 22451 visitor.go:40] float64 value at spec.ports.[1].port: 53.000000
I0627 16:48:56.522257 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.1.2-r2"
I0627 16:48:56.522985 22451 visitor.go:35] string value at spec.template.spec.volumes.[0].configMap.optional: true
I0627 16:48:56.523287 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.10"
I0627 16:48:56.523985 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.failureThreshold: 5.000000
I0627 16:48:56.524589 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.httpGet.port: 10054.000000
I0627 16:48:56.525320 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:48:56.526246 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.successThreshold: 1.000000
I0627 16:48:56.526906 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.timeoutSeconds: 5.000000
I0627 16:48:56.527214 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[0].containerPort: 10053.000000
I0627 16:48:56.527513 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[1].containerPort: 10053.000000
I0627 16:48:56.527812 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[2].containerPort: 10055.000000
I0627 16:48:56.528103 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.httpGet.port: 8081.000000
I0627 16:48:56.528446 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.initialDelaySeconds: 3.000000
I0627 16:48:56.528638 22451 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.timeoutSeconds: 5.000000
I0627 16:48:56.528837 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.10"
I0627 16:48:56.529023 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.successThreshold: 1.000000
I0627 16:48:56.529211 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.timeoutSeconds: 5.000000
I0627 16:48:56.529402 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.failureThreshold: 5.000000
I0627 16:48:56.529590 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.httpGet.port: 10054.000000
I0627 16:48:56.529777 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:48:56.529966 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].ports.[0].containerPort: 53.000000
I0627 16:48:56.530153 22451 visitor.go:40] float64 value at spec.template.spec.containers.[1].ports.[1].containerPort: 53.000000
I0627 16:48:56.532103 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].ports.[0].containerPort: 10054.000000
I0627 16:48:56.532299 22451 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.10"
I0627 16:48:56.532648 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.successThreshold: 1.000000
I0627 16:48:56.532835 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.timeoutSeconds: 5.000000
I0627 16:48:56.533019 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.failureThreshold: 5.000000
I0627 16:48:56.533203 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.httpGet.port: 10054.000000
I0627 16:48:56.533432 22451 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:48:56.533913 22451 visitor.go:40] float64 value at spec.strategy.rollingUpdate.maxUnavailable: 0.000000
I0627 16:48:56.535297 22451 visitor.go:40] float64 value at spec.ports.[0].port: 53.000000
I0627 16:48:56.535556 22451 visitor.go:40] float64 value at spec.ports.[1].port: 53.000000
I0627 16:48:56.536490 22451 template_functions.go:190] watch-ingress=false set on dns-controller
I0627 16:48:56.537051 22451 visitor.go:40] float64 value at spec.replicas: 1.000000
I0627 16:48:56.537428 22451 images.go:59] Consider image for re-mapping: "kope/dns-controller:1.12.0"
I0627 16:48:56.537632 22451 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:48:56.538370 22451 template_functions.go:190] watch-ingress=false set on dns-controller
I0627 16:48:56.539025 22451 visitor.go:40] float64 value at spec.replicas: 1.000000
I0627 16:48:56.539186 22451 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:48:56.539405 22451 images.go:59] Consider image for re-mapping: "kope/dns-controller:1.12.0"
I0627 16:48:56.540278 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.542085 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.542234 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.542426 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.542573 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.542721 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.542867 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.543011 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.543274 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.543466 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.543609 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.543757 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.544007 22451 task.go:103] testing task "Secret"
I0627 16:48:56.544226 22451 task.go:103] testing task "Secret"
I0627 16:48:56.544436 22451 task.go:103] testing task "Secret"
I0627 16:48:56.544587 22451 task.go:103] testing task "Secret"
I0627 16:48:56.544730 22451 task.go:103] testing task "Secret"
I0627 16:48:56.544872 22451 task.go:103] testing task "Secret"
I0627 16:48:56.545020 22451 task.go:103] testing task "Secret"
I0627 16:48:56.545166 22451 task.go:103] testing task "Secret"
I0627 16:48:56.545311 22451 task.go:103] testing task "Secret"
I0627 16:48:56.545707 22451 task.go:103] testing task "MirrorSecrets"
I0627 16:48:56.546025 22451 task.go:103] testing task "MirrorKeystore"
I0627 16:48:56.546212 22451 decoder.go:224] decoding stream as YAML
I0627 16:48:56.549931 22451 build_flags.go:50] ignoring non-field:
I0627 16:48:56.550095 22451 build_flags.go:50] ignoring non-field:
I0627 16:48:56.558181 22451 task.go:103] testing task "ManagedFile"
I0627 16:48:56.558524 22451 task.go:103] testing task "ManagedFile"
I0627 16:48:56.558750 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.558980 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.559203 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.559435 22451 decoder.go:224] decoding stream as YAML
I0627 16:48:56.559919 22451 build_flags.go:50] ignoring non-field:
I0627 16:48:56.560185 22451 build_flags.go:50] ignoring non-field:
I0627 16:48:56.560964 22451 task.go:103] testing task "ManagedFile"
I0627 16:48:56.561172 22451 task.go:103] testing task "ManagedFile"
I0627 16:48:56.561371 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.561569 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.561781 22451 task.go:103] testing task "Keypair"
I0627 16:48:56.563186 22451 task.go:75] EnsureTask ignoring identical
I0627 16:48:56.563410 22451 task.go:103] testing task "EBSVolume"
I0627 16:48:56.563564 22451 task.go:103] testing task "EBSVolume"
I0627 16:48:56.563715 22451 task.go:103] testing task "EBSVolume"
I0627 16:48:56.563891 22451 task.go:103] testing task "EBSVolume"
I0627 16:48:56.564061 22451 task.go:103] testing task "EBSVolume"
I0627 16:48:56.564210 22451 task.go:103] testing task "EBSVolume"
I0627 16:48:56.574294 22451 task.go:103] testing task "LoadBalancer"
I0627 16:48:56.574765 22451 task.go:103] testing task "SecurityGroup"
I0627 16:48:56.575066 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.575406 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.575767 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.576044 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.576374 22451 task.go:103] testing task "LoadBalancerAttachment"
I0627 16:48:56.576687 22451 task.go:103] testing task "LoadBalancerAttachment"
I0627 16:48:56.576957 22451 task.go:103] testing task "LoadBalancerAttachment"
I0627 16:48:56.577285 22451 task.go:103] testing task "SecurityGroup"
I0627 16:48:56.577565 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.577778 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.577983 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.578190 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.578402 22451 task.go:103] testing task "SecurityGroup"
I0627 16:48:56.578671 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.578851 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.579106 22451 task.go:103] testing task "LoadBalancer"
I0627 16:48:56.579311 22451 task.go:103] testing task "LoadBalancerAttachment"
I0627 16:48:56.579463 22451 task.go:103] testing task "DNSName"
I0627 16:48:56.579614 22451 task.go:103] testing task "DNSZone"
I0627 16:48:56.579761 22451 task.go:103] testing task "DNSZone"
I0627 16:48:56.579953 22451 task.go:75] EnsureTask ignoring identical
I0627 16:48:56.580117 22451 task.go:103] testing task "DNSName"
I0627 16:48:56.580261 22451 task.go:103] testing task "DNSZone"
I0627 16:48:56.580412 22451 task.go:75] EnsureTask ignoring identical
I0627 16:48:56.580569 22451 external_access.go:60] bastion is in use; won't configure SSH access to master / node instances
I0627 16:48:56.580756 22451 task.go:103] testing task "SecurityGroup"
I0627 16:48:56.580924 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.581089 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.581241 22451 task.go:103] testing task "SecurityGroup"
I0627 16:48:56.581388 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.581536 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.581684 22451 task.go:103] testing task "SecurityGroupRule"
W0627 16:48:56.581838 22451 firewall.go:250] Opening etcd port on masters for access from the nodes, for calico. This is unsafe in untrusted environments.
I0627 16:48:56.589622 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.590036 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.590445 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.590654 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.590847 22451 task.go:103] testing task "SecurityGroupRule"
I0627 16:48:56.591156 22451 task.go:103] testing task "SSHKey"
I0627 16:48:56.591368 22451 task.go:103] testing task "VPC"
I0627 16:48:56.591572 22451 task.go:103] testing task "DHCPOptions"
I0627 16:48:56.591765 22451 task.go:103] testing task "VPCDHCPOptionsAssociation"
I0627 16:48:56.592012 22451 task.go:103] testing task "InternetGateway"
I0627 16:48:56.592222 22451 task.go:103] testing task "RouteTable"
I0627 16:48:56.592415 22451 task.go:103] testing task "Route"
I0627 16:48:56.592611 22451 network.go:202] applying subnet tags
I0627 16:48:56.592802 22451 task.go:103] testing task "Subnet"
I0627 16:48:56.592987 22451 task.go:103] testing task "RouteTableAssociation"
I0627 16:48:56.593189 22451 network.go:202] applying subnet tags
I0627 16:48:56.593394 22451 task.go:103] testing task "Subnet"
I0627 16:48:56.593581 22451 task.go:103] testing task "RouteTableAssociation"
I0627 16:48:56.593780 22451 network.go:202] applying subnet tags
I0627 16:48:56.593974 22451 task.go:103] testing task "Subnet"
I0627 16:48:56.594161 22451 task.go:103] testing task "RouteTableAssociation"
I0627 16:48:56.594373 22451 network.go:202] applying subnet tags
I0627 16:48:56.594582 22451 task.go:103] testing task "Subnet"
I0627 16:48:56.594785 22451 task.go:103] testing task "RouteTableAssociation"
I0627 16:48:56.594981 22451 network.go:202] applying subnet tags
I0627 16:48:56.595190 22451 task.go:103] testing task "Subnet"
I0627 16:48:56.595392 22451 task.go:103] testing task "RouteTableAssociation"
I0627 16:48:56.595599 22451 network.go:202] applying subnet tags
I0627 16:48:56.595774 22451 task.go:103] testing task "Subnet"
I0627 16:48:56.595965 22451 task.go:103] testing task "RouteTableAssociation"
I0627 16:48:56.598238 22451 task.go:103] testing task "ElasticIP"
I0627 16:48:56.598448 22451 task.go:103] testing task "NatGateway"
I0627 16:48:56.598628 22451 task.go:103] testing task "RouteTable"
I0627 16:48:56.598776 22451 task.go:103] testing task "Route"
I0627 16:48:56.598977 22451 task.go:103] testing task "ElasticIP"
I0627 16:48:56.599130 22451 task.go:103] testing task "NatGateway"
I0627 16:48:56.599280 22451 task.go:103] testing task "RouteTable"
I0627 16:48:56.599431 22451 task.go:103] testing task "Route"
I0627 16:48:56.599579 22451 task.go:103] testing task "ElasticIP"
I0627 16:48:56.599730 22451 task.go:103] testing task "NatGateway"
I0627 16:48:56.599880 22451 task.go:103] testing task "RouteTable"
I0627 16:48:56.600079 22451 task.go:103] testing task "Route"
I0627 16:48:56.600282 22451 task.go:103] testing task "IAMRole"
I0627 16:48:56.600520 22451 task.go:103] testing task "IAMRolePolicy"
I0627 16:48:56.600698 22451 task.go:103] testing task "IAMInstanceProfile"
I0627 16:48:56.600883 22451 task.go:103] testing task "IAMInstanceProfileRole"
I0627 16:48:56.601247 22451 task.go:103] testing task "IAMRolePolicy"
I0627 16:48:56.601764 22451 task.go:103] testing task "IAMRole"
I0627 16:48:56.601998 22451 task.go:103] testing task "IAMRolePolicy"
I0627 16:48:56.602212 22451 task.go:103] testing task "IAMInstanceProfile"
I0627 16:48:56.602366 22451 task.go:103] testing task "IAMInstanceProfileRole"
I0627 16:48:56.602523 22451 task.go:103] testing task "IAMRolePolicy"
I0627 16:48:56.602708 22451 task.go:103] testing task "IAMRole"
I0627 16:48:56.603008 22451 task.go:103] testing task "IAMRolePolicy"
I0627 16:48:56.603216 22451 task.go:103] testing task "IAMInstanceProfile"
I0627 16:48:56.603379 22451 task.go:103] testing task "IAMInstanceProfileRole"
I0627 16:48:56.603577 22451 task.go:103] testing task "IAMRolePolicy"
I0627 16:48:56.603785 22451 task.go:103] testing task "LaunchConfiguration"
I0627 16:48:56.604027 22451 task.go:103] testing task "AutoscalingGroup"
I0627 16:48:56.604298 22451 task.go:103] testing task "LaunchConfiguration"
I0627 16:48:56.604489 22451 task.go:103] testing task "AutoscalingGroup"
I0627 16:48:56.604687 22451 task.go:103] testing task "LaunchConfiguration"
I0627 16:48:56.604887 22451 task.go:103] testing task "AutoscalingGroup"
I0627 16:48:56.605235 22451 task.go:103] testing task "LaunchConfiguration"
I0627 16:48:56.605398 22451 task.go:103] testing task "AutoscalingGroup"
I0627 16:48:56.605634 22451 task.go:103] testing task "LaunchConfiguration"
I0627 16:48:56.605808 22451 task.go:103] testing task "AutoscalingGroup"
I0627 16:48:56.607934 22451 topological_sort.go:64] Dependencies:
I0627 16:48:56.609055 22451 topological_sort.go:66] IAMRolePolicy/additional.bastions.dev-qa.k8s.domain.com: [IAMRole/bastions.dev-qa.k8s.domain.com]
I0627 16:48:56.609515 22451 topological_sort.go:66] RouteTableAssociation/private-eu-central-1c.dev-qa.k8s.domain.com: [RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com Subnet/eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:48:56.609879 22451 topological_sort.go:66] Keypair/etcd-peers-ca-events: []
I0627 16:48:56.610218 22451 topological_sort.go:66] MirrorSecrets/mirror-secrets: [Secret/kubelet Secret/system:logging Secret/system:scheduler Secret/system:dns Secret/kube Secret/system:controller_manager Secret/admin Secret/kube-proxy Secret/system:monitoring]
I0627 16:48:56.610524 22451 topological_sort.go:66] SecurityGroupRule/node-to-master-tcp-2382-4001: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:48:56.610748 22451 topological_sort.go:66] IAMRolePolicy/bastions.dev-qa.k8s.domain.com: [IAMRole/bastions.dev-qa.k8s.domain.com DNSZone/k8s.domain.com]
I0627 16:48:56.610960 22451 topological_sort.go:66] LoadBalancerAttachment/api-master-eu-central-1a: [LoadBalancer/api.dev-qa.k8s.domain.com AutoscalingGroup/master-eu-central-1a.masters.dev-qa.k8s.domain.com]
I0627 16:48:56.611263 22451 topological_sort.go:66] LaunchConfiguration/master-eu-central-1c.masters.dev-qa.k8s.domain.com: [IAMInstanceProfile/masters.dev-qa.k8s.domain.com SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b SecurityGroup/masters.dev-qa.k8s.domain.com]
I0627 16:48:56.611622 22451 topological_sort.go:66] DNSName/api.dev-qa.k8s.domain.com: [DNSZone/k8s.domain.com LoadBalancer/api.dev-qa.k8s.domain.com]
I0627 16:48:56.611885 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.7.0: []
I0627 16:48:56.612218 22451 topological_sort.go:66] IAMRolePolicy/masters.dev-qa.k8s.domain.com: [IAMRole/masters.dev-qa.k8s.domain.com DNSZone/k8s.domain.com]
I0627 16:48:56.612560 22451 topological_sort.go:66] EBSVolume/b.etcd-main.dev-qa.k8s.domain.com: []
I0627 16:48:56.612837 22451 topological_sort.go:66] NatGateway/eu-central-1a.dev-qa.k8s.domain.com: [ElasticIP/eu-central-1a.dev-qa.k8s.domain.com Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com]
I0627 16:48:56.613246 22451 topological_sort.go:66] Route/private-eu-central-1c-0.0.0.0/0: [RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com NatGateway/eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:48:56.613579 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.12: []
I0627 16:48:56.613924 22451 topological_sort.go:66] Keypair/apiserver-aggregator: [Keypair/apiserver-aggregator-ca]
I0627 16:48:56.614201 22451 topological_sort.go:66] SecurityGroup/nodes.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.614418 22451 topological_sort.go:66] RouteTableAssociation/utility-eu-central-1b.dev-qa.k8s.domain.com: [RouteTable/dev-qa.k8s.domain.com Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:48:56.614684 22451 topological_sort.go:66] SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b: []
I0627 16:48:56.614917 22451 topological_sort.go:66] EBSVolume/b.etcd-events.dev-qa.k8s.domain.com: []
I0627 16:48:56.615111 22451 topological_sort.go:66] SecurityGroupRule/node-to-master-protocol-ipip: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:48:56.615355 22451 topological_sort.go:66] Keypair/apiserver-aggregator-ca: []
I0627 16:48:56.615537 22451 topological_sort.go:66] SecurityGroupRule/node-to-master-udp-1-65535: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:48:56.615765 22451 topological_sort.go:66] RouteTableAssociation/utility-eu-central-1a.dev-qa.k8s.domain.com: [RouteTable/dev-qa.k8s.domain.com Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com]
I0627 16:48:56.616018 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-pre-k8s-1.6: []
I0627 16:48:56.616260 22451 topological_sort.go:66] VPCDHCPOptionsAssociation/dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com DHCPOptions/dev-qa.k8s.domain.com]
I0627 16:48:56.616533 22451 topological_sort.go:66] Secret/system:logging: []
I0627 16:48:56.616729 22451 topological_sort.go:66] EBSVolume/a.etcd-events.dev-qa.k8s.domain.com: []
I0627 16:48:56.616938 22451 topological_sort.go:66] IAMInstanceProfileRole/bastions.dev-qa.k8s.domain.com: [IAMInstanceProfile/bastions.dev-qa.k8s.domain.com IAMRole/bastions.dev-qa.k8s.domain.com]
I0627 16:48:56.620450 22451 topological_sort.go:66] SecurityGroupRule/https-elb-to-master: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/api-elb.dev-qa.k8s.domain.com]
I0627 16:48:56.620702 22451 topological_sort.go:66] IAMInstanceProfile/nodes.dev-qa.k8s.domain.com: []
I0627 16:48:56.620889 22451 topological_sort.go:66] SecurityGroupRule/node-to-master-tcp-1-2379: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:48:56.621115 22451 topological_sort.go:66] SecurityGroupRule/https-api-elb-0.0.0.0/0: [SecurityGroup/api-elb.dev-qa.k8s.domain.com]
I0627 16:48:56.621371 22451 topological_sort.go:66] Keypair/etcd-clients-ca: []
I0627 16:48:56.621545 22451 topological_sort.go:66] SecurityGroupRule/bastion-to-node-ssh: [SecurityGroup/nodes.dev-qa.k8s.domain.com SecurityGroup/bastion.dev-qa.k8s.domain.com]
I0627 16:48:56.621769 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-bootstrap: []
I0627 16:48:56.621949 22451 topological_sort.go:66] ElasticIP/eu-central-1c.dev-qa.k8s.domain.com: [RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:48:56.622195 22451 topological_sort.go:66] VPC/dev-qa.k8s.domain.com: []
I0627 16:48:56.622388 22451 topological_sort.go:66] Route/private-eu-central-1b-0.0.0.0/0: [RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com NatGateway/eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:48:56.622645 22451 topological_sort.go:66] Subnet/eu-central-1a.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.622844 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-limit-range.addons.k8s.io: []
I0627 16:48:56.623048 22451 topological_sort.go:66] Keypair/kube-proxy: [Keypair/ca]
I0627 16:48:56.623269 22451 topological_sort.go:66] Secret/system:monitoring: []
I0627 16:48:56.623456 22451 topological_sort.go:66] IAMRole/masters.dev-qa.k8s.domain.com: []
I0627 16:48:56.623635 22451 topological_sort.go:66] EBSVolume/a.etcd-main.dev-qa.k8s.domain.com: []
I0627 16:48:56.623819 22451 topological_sort.go:66] Keypair/kube-controller-manager: [Keypair/ca]
I0627 16:48:56.624002 22451 topological_sort.go:66] ManagedFile/manifests-etcdmanager-events: []
I0627 16:48:56.624223 22451 topological_sort.go:66] SecurityGroupRule/bastion-egress: [SecurityGroup/bastion.dev-qa.k8s.domain.com]
I0627 16:48:56.624421 22451 topological_sort.go:66] SecurityGroupRule/all-master-to-master: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/masters.dev-qa.k8s.domain.com]
I0627 16:48:56.624643 22451 topological_sort.go:66] Keypair/apiserver-proxy-client: [Keypair/ca]
I0627 16:48:56.624824 22451 topological_sort.go:66] Secret/kube: []
I0627 16:48:56.624990 22451 topological_sort.go:66] IAMRole/nodes.dev-qa.k8s.domain.com: []
I0627 16:48:56.625210 22451 topological_sort.go:66] SecurityGroupRule/ssh-external-to-bastion-elb-0.0.0.0/0: [SecurityGroup/bastion-elb.dev-qa.k8s.domain.com]
I0627 16:48:56.625439 22451 topological_sort.go:66] DNSZone/k8s.domain.com: []
I0627 16:48:56.625611 22451 topological_sort.go:66] Keypair/master: [Keypair/ca]
I0627 16:48:56.625787 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.6: []
I0627 16:48:56.625995 22451 topological_sort.go:66] ElasticIP/eu-central-1a.dev-qa.k8s.domain.com: [RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com]
I0627 16:48:56.626246 22451 topological_sort.go:66] EBSVolume/c.etcd-main.dev-qa.k8s.domain.com: []
I0627 16:48:56.626428 22451 topological_sort.go:66] SecurityGroupRule/node-to-master-tcp-4003-65535: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:48:56.626648 22451 topological_sort.go:66] DHCPOptions/dev-qa.k8s.domain.com: []
I0627 16:48:56.626825 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.12: []
I0627 16:48:56.627018 22451 topological_sort.go:66] SecurityGroupRule/master-egress: [SecurityGroup/masters.dev-qa.k8s.domain.com]
I0627 16:48:56.627261 22451 topological_sort.go:66] Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.627471 22451 topological_sort.go:66] AutoscalingGroup/nodes.dev-qa.k8s.domain.com: [LaunchConfiguration/nodes.dev-qa.k8s.domain.com Subnet/eu-central-1a.dev-qa.k8s.domain.com Subnet/eu-central-1b.dev-qa.k8s.domain.com Subnet/eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:48:56.627758 22451 topological_sort.go:66] Keypair/kube-scheduler: [Keypair/ca]
I0627 16:48:56.628010 22451 topological_sort.go:66] Keypair/ca: []
I0627 16:48:56.628211 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.6.0: []
I0627 16:48:56.628414 22451 topological_sort.go:66] Keypair/etcd-manager-ca-events: []
I0627 16:48:56.628591 22451 topological_sort.go:66] AutoscalingGroup/master-eu-central-1b.masters.dev-qa.k8s.domain.com: [LaunchConfiguration/master-eu-central-1b.masters.dev-qa.k8s.domain.com Subnet/eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:48:56.628868 22451 topological_sort.go:66] SecurityGroupRule/ssh-elb-to-bastion: [SecurityGroup/bastion.dev-qa.k8s.domain.com SecurityGroup/bastion-elb.dev-qa.k8s.domain.com]
I0627 16:48:56.629088 22451 topological_sort.go:66] Secret/system:scheduler: []
I0627 16:48:56.632372 22451 topological_sort.go:66] SecurityGroupRule/all-master-to-node: [SecurityGroup/nodes.dev-qa.k8s.domain.com SecurityGroup/masters.dev-qa.k8s.domain.com]
I0627 16:48:56.632774 22451 topological_sort.go:66] ManagedFile/manifests-etcdmanager-main: []
I0627 16:48:56.633040 22451 topological_sort.go:66] SecurityGroupRule/node-egress: [SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:48:56.633416 22451 topological_sort.go:66] LoadBalancer/api.dev-qa.k8s.domain.com: [Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com SecurityGroup/api-elb.dev-qa.k8s.domain.com]
I0627 16:48:56.633918 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.12: []
I0627 16:48:56.634466 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.7-v3: []
I0627 16:48:56.634945 22451 topological_sort.go:66] NatGateway/eu-central-1b.dev-qa.k8s.domain.com: [ElasticIP/eu-central-1b.dev-qa.k8s.domain.com Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:48:56.635486 22451 topological_sort.go:66] IAMInstanceProfile/bastions.dev-qa.k8s.domain.com: []
I0627 16:48:56.635948 22451 topological_sort.go:66] IAMRolePolicy/nodes.dev-qa.k8s.domain.com: [IAMRole/nodes.dev-qa.k8s.domain.com DNSZone/k8s.domain.com]
I0627 16:48:56.636606 22451 topological_sort.go:66] LaunchConfiguration/master-eu-central-1b.masters.dev-qa.k8s.domain.com: [IAMInstanceProfile/masters.dev-qa.k8s.domain.com SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b SecurityGroup/masters.dev-qa.k8s.domain.com]
I0627 16:48:56.637158 22451 topological_sort.go:66] IAMRolePolicy/additional.masters.dev-qa.k8s.domain.com: [IAMRole/masters.dev-qa.k8s.domain.com]
I0627 16:48:56.637542 22451 topological_sort.go:66] AutoscalingGroup/master-eu-central-1c.masters.dev-qa.k8s.domain.com: [LaunchConfiguration/master-eu-central-1c.masters.dev-qa.k8s.domain.com Subnet/eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:48:56.637999 22451 topological_sort.go:66] LoadBalancerAttachment/bastion-elb-attachment: [LoadBalancer/bastion.dev-qa.k8s.domain.com AutoscalingGroup/bastions.dev-qa.k8s.domain.com]
I0627 16:48:56.638522 22451 topological_sort.go:66] LoadBalancerAttachment/api-master-eu-central-1b: [LoadBalancer/api.dev-qa.k8s.domain.com AutoscalingGroup/master-eu-central-1b.masters.dev-qa.k8s.domain.com]
I0627 16:48:56.639006 22451 topological_sort.go:66] SecurityGroup/bastion-elb.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.639722 22451 topological_sort.go:66] Keypair/kubelet-api: [Keypair/ca]
I0627 16:48:56.640652 22451 topological_sort.go:66] IAMInstanceProfile/masters.dev-qa.k8s.domain.com: []
I0627 16:48:56.640998 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-rbac.addons.k8s.io-k8s-1.8: []
I0627 16:48:56.643082 22451 topological_sort.go:66] RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.643508 22451 topological_sort.go:66] DNSName/bastion.dev-qa.k8s.domain.com: [DNSZone/k8s.domain.com LoadBalancer/bastion.dev-qa.k8s.domain.com]
I0627 16:48:56.643911 22451 topological_sort.go:66] SecurityGroupRule/icmp-pmtu-api-elb-0.0.0.0/0: [SecurityGroup/api-elb.dev-qa.k8s.domain.com]
I0627 16:48:56.644246 22451 topological_sort.go:66] Keypair/etcd-peers-ca-main: []
I0627 16:48:56.644595 22451 topological_sort.go:66] SecurityGroupRule/bastion-elb-egress: [SecurityGroup/bastion-elb.dev-qa.k8s.domain.com]
I0627 16:48:56.644943 22451 topological_sort.go:66] Route/private-eu-central-1a-0.0.0.0/0: [RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com NatGateway/eu-central-1a.dev-qa.k8s.domain.com]
I0627 16:48:56.645374 22451 topological_sort.go:66] Secret/system:controller_manager: []
I0627 16:48:56.645704 22451 topological_sort.go:66] Keypair/etcd-manager-ca-main: []
I0627 16:48:56.646017 22451 topological_sort.go:66] ElasticIP/eu-central-1b.dev-qa.k8s.domain.com: [RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:48:56.646391 22451 topological_sort.go:66] AutoscalingGroup/master-eu-central-1a.masters.dev-qa.k8s.domain.com: [LaunchConfiguration/master-eu-central-1a.masters.dev-qa.k8s.domain.com Subnet/eu-central-1a.dev-qa.k8s.domain.com]
I0627 16:48:56.646806 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-pre-k8s-1.6: []
I0627 16:48:56.647105 22451 topological_sort.go:66] LaunchConfiguration/nodes.dev-qa.k8s.domain.com: [IAMInstanceProfile/nodes.dev-qa.k8s.domain.com SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:48:56.647528 22451 topological_sort.go:66] Secret/kubelet: []
I0627 16:48:56.647781 22451 topological_sort.go:66] LaunchConfiguration/bastions.dev-qa.k8s.domain.com: [IAMInstanceProfile/bastions.dev-qa.k8s.domain.com SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b SecurityGroup/bastion.dev-qa.k8s.domain.com]
I0627 16:48:56.648108 22451 topological_sort.go:66] Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.648364 22451 topological_sort.go:66] Secret/admin: []
I0627 16:48:56.648610 22451 topological_sort.go:66] EBSVolume/c.etcd-events.dev-qa.k8s.domain.com: []
I0627 16:48:56.648869 22451 topological_sort.go:66] SecurityGroup/masters.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.649096 22451 topological_sort.go:66] RouteTableAssociation/utility-eu-central-1c.dev-qa.k8s.domain.com: [RouteTable/dev-qa.k8s.domain.com Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:48:56.650055 22451 topological_sort.go:66] LaunchConfiguration/master-eu-central-1a.masters.dev-qa.k8s.domain.com: [IAMInstanceProfile/masters.dev-qa.k8s.domain.com SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b SecurityGroup/masters.dev-qa.k8s.domain.com]
I0627 16:48:56.651094 22451 topological_sort.go:66] SecurityGroupRule/api-elb-egress: [SecurityGroup/api-elb.dev-qa.k8s.domain.com]
I0627 16:48:56.655325 22451 topological_sort.go:66] Keypair/kubelet: [Keypair/ca]
I0627 16:48:56.655604 22451 topological_sort.go:66] RouteTableAssociation/private-eu-central-1b.dev-qa.k8s.domain.com: [RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com Subnet/eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:48:56.655874 22451 topological_sort.go:66] Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.656077 22451 topological_sort.go:66] LoadBalancer/bastion.dev-qa.k8s.domain.com: [Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com SecurityGroup/bastion-elb.dev-qa.k8s.domain.com]
I0627 16:48:56.656388 22451 topological_sort.go:66] LoadBalancerAttachment/api-master-eu-central-1c: [LoadBalancer/api.dev-qa.k8s.domain.com AutoscalingGroup/master-eu-central-1c.masters.dev-qa.k8s.domain.com]
I0627 16:48:56.656650 22451 topological_sort.go:66] AutoscalingGroup/bastions.dev-qa.k8s.domain.com: [LaunchConfiguration/bastions.dev-qa.k8s.domain.com Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:48:56.656956 22451 topological_sort.go:66] Subnet/eu-central-1c.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.657180 22451 topological_sort.go:66] Secret/system:dns: []
I0627 16:48:56.657373 22451 topological_sort.go:66] IAMInstanceProfileRole/nodes.dev-qa.k8s.domain.com: [IAMInstanceProfile/nodes.dev-qa.k8s.domain.com IAMRole/nodes.dev-qa.k8s.domain.com]
I0627 16:48:56.657595 22451 topological_sort.go:66] IAMRolePolicy/additional.nodes.dev-qa.k8s.domain.com: [IAMRole/nodes.dev-qa.k8s.domain.com]
I0627 16:48:56.657797 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.6: []
I0627 16:48:56.657988 22451 topological_sort.go:66] RouteTable/dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.658188 22451 topological_sort.go:66] Secret/kube-proxy: []
I0627 16:48:56.658358 22451 topological_sort.go:66] ManagedFile/etcd-cluster-spec-events: []
I0627 16:48:56.658537 22451 topological_sort.go:66] IAMRole/bastions.dev-qa.k8s.domain.com: []
I0627 16:48:56.658715 22451 topological_sort.go:66] RouteTableAssociation/private-eu-central-1a.dev-qa.k8s.domain.com: [RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com Subnet/eu-central-1a.dev-qa.k8s.domain.com]
I0627 16:48:56.658966 22451 topological_sort.go:66] RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.659177 22451 topological_sort.go:66] Route/0.0.0.0/0: [RouteTable/dev-qa.k8s.domain.com InternetGateway/dev-qa.k8s.domain.com]
I0627 16:48:56.659379 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-kubelet-api.rbac.addons.k8s.io-k8s-1.9: []
I0627 16:48:56.659648 22451 topological_sort.go:66] SecurityGroupRule/bastion-to-master-ssh: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/bastion.dev-qa.k8s.domain.com]
I0627 16:48:56.659869 22451 topological_sort.go:66] ManagedFile/etcd-cluster-spec-main: []
I0627 16:48:56.660181 22451 topological_sort.go:66] Keypair/kubecfg: [Keypair/ca]
I0627 16:48:56.660389 22451 topological_sort.go:66] Subnet/eu-central-1b.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.660584 22451 topological_sort.go:66] RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.660866 22451 topological_sort.go:66] SecurityGroup/api-elb.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.661062 22451 topological_sort.go:66] SecurityGroup/bastion.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.661308 22451 topological_sort.go:66] SecurityGroupRule/all-node-to-node: [SecurityGroup/nodes.dev-qa.k8s.domain.com SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:48:56.661524 22451 topological_sort.go:66] Keypair/kops: [Keypair/ca]
I0627 16:48:56.661696 22451 topological_sort.go:66] NatGateway/eu-central-1c.dev-qa.k8s.domain.com: [ElasticIP/eu-central-1c.dev-qa.k8s.domain.com Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:48:56.661971 22451 topological_sort.go:66] dev-qa.k8s.domain.com-addons-core.addons.k8s.io: []
I0627 16:48:56.662193 22451 topological_sort.go:66] InternetGateway/dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:48:56.662403 22451 topological_sort.go:66] MirrorKeystore/mirror-keystore: [Secret/admin Secret/kube-proxy Secret/system:monitoring Secret/kubelet Secret/system:logging Secret/system:scheduler Secret/system:dns Secret/kube Secret/system:controller_manager]
I0627 16:48:56.662686 22451 topological_sort.go:66] IAMInstanceProfileRole/masters.dev-qa.k8s.domain.com: [IAMInstanceProfile/masters.dev-qa.k8s.domain.com IAMRole/masters.dev-qa.k8s.domain.com]
I0627 16:48:56.663089 22451 executor.go:103] Tasks: 0 done / 143 total; 51 can run
I0627 16:48:56.663924 22451 executor.go:178] Executing task "VPC/dev-qa.k8s.domain.com": *awstasks.VPC {"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:48:56.663950 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-bootstrap": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-bootstrap","Lifecycle":"Sync","Location":"addons/bootstrap-channel.yaml","Contents":{"Name":"","Resource":"kind: Addons\nmetadata:\n creationTimestamp: null\n name: bootstrap\nspec:\n addons:\n - manifest: core.addons.k8s.io/v1.4.0.yaml\n name: core.addons.k8s.io\n selector:\n k8s-addon: core.addons.k8s.io\n version: 1.4.0\n - id: pre-k8s-1.6\n kubernetesVersion: \u003c1.6.0\n manifest: kube-dns.addons.k8s.io/pre-k8s-1.6.yaml\n name: kube-dns.addons.k8s.io\n selector:\n k8s-addon: kube-dns.addons.k8s.io\n version: 1.14.13-kops.1\n - id: k8s-1.6\n kubernetesVersion: '\u003e=1.6.0 \u003c1.12.0'\n manifest: kube-dns.addons.k8s.io/k8s-1.6.yaml\n name: kube-dns.addons.k8s.io\n selector:\n k8s-addon: kube-dns.addons.k8s.io\n version: 1.14.13-kops.1\n - id: k8s-1.12\n kubernetesVersion: '\u003e=1.12.0'\n manifest: kube-dns.addons.k8s.io/k8s-1.12.yaml\n name: kube-dns.addons.k8s.io\n selector:\n k8s-addon: kube-dns.addons.k8s.io\n version: 1.14.13-kops.1\n - id: k8s-1.8\n kubernetesVersion: '\u003e=1.8.0'\n manifest: rbac.addons.k8s.io/k8s-1.8.yaml\n name: rbac.addons.k8s.io\n selector:\n k8s-addon: rbac.addons.k8s.io\n version: 1.8.0\n - id: k8s-1.9\n kubernetesVersion: '\u003e=1.9.0'\n manifest: kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml\n name: kubelet-api.rbac.addons.k8s.io\n selector:\n k8s-addon: kubelet-api.rbac.addons.k8s.io\n version: v0.0.1\n - manifest: limit-range.addons.k8s.io/v1.5.0.yaml\n name: limit-range.addons.k8s.io\n selector:\n k8s-addon: limit-range.addons.k8s.io\n version: 1.5.0\n - id: pre-k8s-1.6\n kubernetesVersion: \u003c1.6.0\n manifest: dns-controller.addons.k8s.io/pre-k8s-1.6.yaml\n name: dns-controller.addons.k8s.io\n selector:\n k8s-addon: dns-controller.addons.k8s.io\n version: 1.12.1\n - id: k8s-1.6\n kubernetesVersion: '\u003e=1.6.0 \u003c1.12.0'\n manifest: dns-controller.addons.k8s.io/k8s-1.6.yaml\n name: dns-controller.addons.k8s.io\n selector:\n k8s-addon: dns-controller.addons.k8s.io\n version: 1.12.1\n - id: k8s-1.12\n kubernetesVersion: '\u003e=1.12.0'\n manifest: dns-controller.addons.k8s.io/k8s-1.12.yaml\n name: dns-controller.addons.k8s.io\n selector:\n k8s-addon: dns-controller.addons.k8s.io\n version: 1.12.1\n - id: v1.7.0\n kubernetesVersion: '\u003e=1.7.0'\n manifest: storage-aws.addons.k8s.io/v1.7.0.yaml\n name: storage-aws.addons.k8s.io\n selector:\n k8s-addon: storage-aws.addons.k8s.io\n version: 1.7.0\n - id: v1.6.0\n kubernetesVersion: \u003c1.7.0\n manifest: storage-aws.addons.k8s.io/v1.6.0.yaml\n name: storage-aws.addons.k8s.io\n selector:\n k8s-addon: storage-aws.addons.k8s.io\n version: 1.7.0\n - id: k8s-1.12\n kubernetesVersion: '\u003e=1.12.0'\n manifest: networking.projectcalico.org/k8s-1.12.yaml\n name: networking.projectcalico.org\n selector:\n role.kubernetes.io/networking: \"1\"\n version: 3.4.0-kops.4\n - id: k8s-1.7-v3\n kubernetesVersion: '\u003e=1.7.0 \u003c1.12.0'\n manifest: networking.projectcalico.org/k8s-1.7-v3.yaml\n name: networking.projectcalico.org\n selector:\n role.kubernetes.io/networking: \"1\"\n version: 3.4.0-kops.3\n"}}
I0627 16:48:56.663795 22451 executor.go:178] Executing task "ManagedFile/etcd-cluster-spec-events": *fitasks.ManagedFile {"Name":"etcd-cluster-spec-events","Lifecycle":"Sync","Location":"backups/etcd/events/control/etcd-cluster-spec","Contents":{"Name":"","Resource":"{\n \"memberCount\": 3,\n \"etcdVersion\": \"3.2.24\"\n}"}}
I0627 16:48:56.672642 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events/control/etcd-cluster-spec"
I0627 16:48:56.663975 22451 executor.go:178] Executing task "SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b": *awstasks.SSHKey {"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":null}
I0627 16:48:56.663938 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-limit-range.addons.k8s.io": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-limit-range.addons.k8s.io","Lifecycle":"Sync","Location":"addons/limit-range.addons.k8s.io/v1.5.0.yaml","Contents":{"Name":"","Resource":"apiVersion: v1\nkind: LimitRange\nmetadata:\n name: limits\n namespace: default\nspec:\n limits:\n - defaultRequest:\n cpu: 100m\n type: Container\n"}}
I0627 16:48:56.663983 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.6": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.6","Lifecycle":"Sync","Location":"addons/dns-controller.addons.k8s.io/k8s-1.6.yaml","Contents":{"Name":"","Resource":"apiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n k8s-app: dns-controller\n version: v1.12.0\n name: dns-controller\n namespace: kube-system\nspec:\n replicas: 1\n selector:\n matchLabels:\n k8s-app: dns-controller\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n scheduler.alpha.kubernetes.io/tolerations: '[{\"key\": \"dedicated\", \"value\":\n \"master\"}]'\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n k8s-app: dns-controller\n version: v1.12.0\n spec:\n containers:\n - command:\n - /usr/bin/dns-controller\n - --watch-ingress=false\n - --dns=aws-route53\n - --zone=k8s.domain.com\n - --zone=*/*\n - -v=2\n image: kope/dns-controller:1.12.0\n name: dns-controller\n resources:\n requests:\n cpu: 50m\n memory: 50Mi\n dnsPolicy: Default\n hostNetwork: true\n nodeSelector:\n node-role.kubernetes.io/master: \"\"\n serviceAccount: dns-controller\n tolerations:\n - effect: NoSchedule\n key: node-role.kubernetes.io/master\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n name: dns-controller\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRole\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n name: kops:dns-controller\nrules:\n- apiGroups:\n - \"\"\n resources:\n - endpoints\n - services\n - pods\n - ingress\n - nodes\n verbs:\n - get\n - list\n - watch\n- apiGroups:\n - extensions\n resources:\n - ingresses\n verbs:\n - get\n - list\n - watch\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n name: kops:dns-controller\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: kops:dns-controller\nsubjects:\n- apiGroup: rbac.authorization.k8s.io\n kind: User\n name: system:serviceaccount:kube-system:dns-controller\n"}}
I0627 16:48:56.663949 22451 executor.go:178] Executing task "IAMRole/masters.dev-qa.k8s.domain.com": *awstasks.IAMRole {"ID":null,"Lifecycle":"Sync","Name":"masters.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"masters"}
I0627 16:48:56.664000 22451 executor.go:178] Executing task "IAMRole/bastions.dev-qa.k8s.domain.com": *awstasks.IAMRole {"ID":null,"Lifecycle":"Sync","Name":"bastions.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"bastions"}
I0627 16:48:56.663952 22451 executor.go:178] Executing task "DHCPOptions/dev-qa.k8s.domain.com": *awstasks.DHCPOptions {"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"DomainName":"eu-central-1.compute.internal","DomainNameServers":"AmazonProvidedDNS","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:48:56.664016 22451 executor.go:178] Executing task "ManagedFile/manifests-etcdmanager-main": *fitasks.ManagedFile {"Name":"manifests-etcdmanager-main","Lifecycle":"Sync","Location":"manifests/etcd/main.yaml","Contents":{"Name":"","Resource":"apiVersion: v1\nkind: Pod\nmetadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n creationTimestamp: null\n labels:\n k8s-app: etcd-manager-main\n name: etcd-manager-main\n namespace: kube-system\nspec:\n containers:\n - command:\n - /bin/sh\n - -c\n - mkfifo /tmp/pipe; (tee -a /var/log/etcd.log \u003c /tmp/pipe \u0026 ) ; exec /etcd-manager\n --backup-store=s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main --client-urls=https://__name__:4001\n --cluster-name=etcd --containerized=true --dns-suffix=.internal.dev-qa.k8s.domain.com\n --etcd-insecure=false --grpc-port=3996 --insecure=false --peer-urls=https://__name__:2380\n --quarantine-client-urls=https://__name__:3994 --v=6 --volume-name-tag=k8s.io/etcd/main\n --volume-provider=aws --volume-tag=k8s.io/etcd/main --volume-tag=k8s.io/role/master=1\n --volume-tag=kubernetes.io/cluster/dev-qa.k8s.domain.com=owned \u003e /tmp/pipe 2\u003e\u00261\n image: kopeio/etcd-manager:3.0.20190516\n name: etcd-manager\n resources:\n requests:\n cpu: 200m\n memory: 100Mi\n securityContext:\n privileged: true\n volumeMounts:\n - mountPath: /rootfs\n name: rootfs\n - mountPath: /etc/hosts\n name: hosts\n - mountPath: /etc/kubernetes/pki/etcd-manager\n name: pki\n - mountPath: /var/log/etcd.log\n name: varlogetcd\n hostNetwork: true\n hostPID: true\n tolerations:\n - key: CriticalAddonsOnly\n operator: Exists\n volumes:\n - hostPath:\n path: /\n type: Directory\n name: rootfs\n - hostPath:\n path: /etc/hosts\n type: File\n name: hosts\n - hostPath:\n path: /etc/kubernetes/pki/etcd-manager-main\n type: DirectoryOrCreate\n name: pki\n - hostPath:\n path: /var/log/etcd.log\n type: FileOrCreate\n name: varlogetcd\nstatus: {}\n"}}
I0627 16:48:56.663955 22451 executor.go:178] Executing task "DNSZone/k8s.domain.com": *awstasks.DNSZone {"Name":"k8s.domain.com","Lifecycle":"Sync","DNSName":"k8s.domain.com","ZoneID":null,"Private":null,"PrivateVPC":null}
I0627 16:48:56.663889 22451 executor.go:178] Executing task "IAMInstanceProfile/bastions.dev-qa.k8s.domain.com": *awstasks.IAMInstanceProfile {"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false}
I0627 16:48:56.664131 22451 executor.go:178] Executing task "IAMInstanceProfile/masters.dev-qa.k8s.domain.com": *awstasks.IAMInstanceProfile {"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false}
I0627 16:48:56.664171 22451 executor.go:178] Executing task "IAMInstanceProfile/nodes.dev-qa.k8s.domain.com": *awstasks.IAMInstanceProfile {"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false}
I0627 16:48:56.664164 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-kubelet-api.rbac.addons.k8s.io-k8s-1.9": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-kubelet-api.rbac.addons.k8s.io-k8s-1.9","Lifecycle":"Sync","Location":"addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml","Contents":{"Name":"","Resource":"apiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n name: kops:system:kubelet-api-admin\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: system:kubelet-api-admin\nsubjects:\n- apiGroup: rbac.authorization.k8s.io\n kind: User\n name: kubelet-api\n"}}
I0627 16:48:56.663792 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-core.addons.k8s.io": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-core.addons.k8s.io","Lifecycle":"Sync","Location":"addons/core.addons.k8s.io/v1.4.0.yaml","Contents":{"Name":"","Resource":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: kube-system\n"}}
I0627 16:48:56.664091 22451 executor.go:178] Executing task "ManagedFile/manifests-etcdmanager-events": *fitasks.ManagedFile {"Name":"manifests-etcdmanager-events","Lifecycle":"Sync","Location":"manifests/etcd/events.yaml","Contents":{"Name":"","Resource":"apiVersion: v1\nkind: Pod\nmetadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n creationTimestamp: null\n labels:\n k8s-app: etcd-manager-events\n name: etcd-manager-events\n namespace: kube-system\nspec:\n containers:\n - command:\n - /bin/sh\n - -c\n - mkfifo /tmp/pipe; (tee -a /var/log/etcd.log \u003c /tmp/pipe \u0026 ) ; exec /etcd-manager\n --backup-store=s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events --client-urls=https://__name__:4002\n --cluster-name=etcd-events --containerized=true --dns-suffix=.internal.dev-qa.k8s.domain.com\n --etcd-insecure=false --grpc-port=3997 --insecure=false --peer-urls=https://__name__:2381\n --quarantine-client-urls=https://__name__:3995 --v=6 --volume-name-tag=k8s.io/etcd/events\n --volume-provider=aws --volume-tag=k8s.io/etcd/events --volume-tag=k8s.io/role/master=1\n --volume-tag=kubernetes.io/cluster/dev-qa.k8s.domain.com=owned \u003e /tmp/pipe 2\u003e\u00261\n image: kopeio/etcd-manager:3.0.20190516\n name: etcd-manager\n resources:\n requests:\n cpu: 100m\n memory: 100Mi\n securityContext:\n privileged: true\n volumeMounts:\n - mountPath: /rootfs\n name: rootfs\n - mountPath: /etc/hosts\n name: hosts\n - mountPath: /etc/kubernetes/pki/etcd-manager\n name: pki\n - mountPath: /var/log/etcd.log\n name: varlogetcd\n hostNetwork: true\n hostPID: true\n tolerations:\n - key: CriticalAddonsOnly\n operator: Exists\n volumes:\n - hostPath:\n path: /\n type: Directory\n name: rootfs\n - hostPath:\n path: /etc/hosts\n type: File\n name: hosts\n - hostPath:\n path: /etc/kubernetes/pki/etcd-manager-events\n type: DirectoryOrCreate\n name: pki\n - hostPath:\n path: /var/log/etcd-events.log\n type: FileOrCreate\n name: varlogetcd\nstatus: {}\n"}}
I0627 16:48:56.663771 22451 executor.go:178] Executing task "Secret/admin": *fitasks.Secret {"Name":"admin","Lifecycle":"Sync"}
I0627 16:48:56.663771 22451 executor.go:178] Executing task "ManagedFile/etcd-cluster-spec-main": *fitasks.ManagedFile {"Name":"etcd-cluster-spec-main","Lifecycle":"Sync","Location":"backups/etcd/main/control/etcd-cluster-spec","Contents":{"Name":"","Resource":"{\n \"memberCount\": 3,\n \"etcdVersion\": \"3.2.24\"\n}"}}
I0627 16:48:56.664183 22451 executor.go:178] Executing task "Secret/kube": *fitasks.Secret {"Name":"kube","Lifecycle":"Sync"}
I0627 16:48:56.663909 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.7.0": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.7.0","Lifecycle":"Sync","Location":"addons/storage-aws.addons.k8s.io/v1.7.0.yaml","Contents":{"Name":"","Resource":"apiVersion: storage.k8s.io/v1\nkind: StorageClass\nmetadata:\n labels:\n k8s-addon: storage-aws.addons.k8s.io\n name: default\nparameters:\n type: gp2\nprovisioner: kubernetes.io/aws-ebs\n\n---\n\napiVersion: storage.k8s.io/v1\nkind: StorageClass\nmetadata:\n annotations:\n storageclass.beta.kubernetes.io/is-default-class: \"true\"\n labels:\n k8s-addon: storage-aws.addons.k8s.io\n name: gp2\nparameters:\n type: gp2\nprovisioner: kubernetes.io/aws-ebs\n"}}
I0627 16:48:56.664114 22451 executor.go:178] Executing task "Secret/kubelet": *fitasks.Secret {"Name":"kubelet","Lifecycle":"Sync"}
I0627 16:48:56.663968 22451 executor.go:178] Executing task "Secret/system:monitoring": *fitasks.Secret {"Name":"system:monitoring","Lifecycle":"Sync"}
I0627 16:48:56.663790 22451 executor.go:178] Executing task "Secret/kube-proxy": *fitasks.Secret {"Name":"kube-proxy","Lifecycle":"Sync"}
I0627 16:48:56.663774 22451 executor.go:178] Executing task "EBSVolume/c.etcd-events.dev-qa.k8s.domain.com": *awstasks.EBSVolume {"Name":"c.etcd-events.dev-qa.k8s.domain.com","Lifecycle":"Sync","AvailabilityZone":"eu-central-1c","Encrypted":true,"ID":null,"KmsKeyId":null,"SizeGB":20,"Tags":{"k8s.io/etcd/events":"c/a,b,c","k8s.io/role/master":"1","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"VolumeIops":null,"VolumeType":"gp2"}
I0627 16:48:56.664103 22451 executor.go:178] Executing task "Secret/system:controller_manager": *fitasks.Secret {"Name":"system:controller_manager","Lifecycle":"Sync"}
I0627 16:48:56.664248 22451 executor.go:178] Executing task "EBSVolume/a.etcd-main.dev-qa.k8s.domain.com": *awstasks.EBSVolume {"Name":"a.etcd-main.dev-qa.k8s.domain.com","Lifecycle":"Sync","AvailabilityZone":"eu-central-1a","Encrypted":true,"ID":null,"KmsKeyId":null,"SizeGB":20,"Tags":{"k8s.io/etcd/main":"a/a,b,c","k8s.io/role/master":"1","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"VolumeIops":null,"VolumeType":"gp2"}
I0627 16:48:56.664117 22451 executor.go:178] Executing task "Secret/system:logging": *fitasks.Secret {"Name":"system:logging","Lifecycle":"Sync"}
I0627 16:48:56.664129 22451 executor.go:178] Executing task "Secret/system:dns": *fitasks.Secret {"Name":"system:dns","Lifecycle":"Sync"}
I0627 16:48:56.664151 22451 executor.go:178] Executing task "Secret/system:scheduler": *fitasks.Secret {"Name":"system:scheduler","Lifecycle":"Sync"}
I0627 16:48:56.663893 22451 executor.go:178] Executing task "EBSVolume/c.etcd-main.dev-qa.k8s.domain.com": *awstasks.EBSVolume {"Name":"c.etcd-main.dev-qa.k8s.domain.com","Lifecycle":"Sync","AvailabilityZone":"eu-central-1c","Encrypted":true,"ID":null,"KmsKeyId":null,"SizeGB":20,"Tags":{"k8s.io/etcd/main":"c/a,b,c","k8s.io/role/master":"1","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"VolumeIops":null,"VolumeType":"gp2"}
I0627 16:48:56.663951 22451 executor.go:178] Executing task "EBSVolume/b.etcd-main.dev-qa.k8s.domain.com": *awstasks.EBSVolume {"Name":"b.etcd-main.dev-qa.k8s.domain.com","Lifecycle":"Sync","AvailabilityZone":"eu-central-1b","Encrypted":true,"ID":null,"KmsKeyId":null,"SizeGB":20,"Tags":{"k8s.io/etcd/main":"b/a,b,c","k8s.io/role/master":"1","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"VolumeIops":null,"VolumeType":"gp2"}
I0627 16:48:56.664126 22451 executor.go:178] Executing task "EBSVolume/a.etcd-events.dev-qa.k8s.domain.com": *awstasks.EBSVolume {"Name":"a.etcd-events.dev-qa.k8s.domain.com","Lifecycle":"Sync","AvailabilityZone":"eu-central-1a","Encrypted":true,"ID":null,"KmsKeyId":null,"SizeGB":20,"Tags":{"k8s.io/etcd/events":"a/a,b,c","k8s.io/role/master":"1","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"VolumeIops":null,"VolumeType":"gp2"}
I0627 16:48:56.664074 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.7-v3": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.7-v3","Lifecycle":"Sync","Location":"addons/networking.projectcalico.org/k8s-1.7-v3.yaml","Contents":{"Name":"","Resource":"apiVersion: v1\ndata:\n calico_backend: bird\n cni_network_config: |-\n {\n \"name\": \"k8s-pod-network\",\n \"cniVersion\": \"0.3.0\",\n \"plugins\": [\n {\n \"type\": \"calico\",\n \"etcd_endpoints\": \"__ETCD_ENDPOINTS__\",\n \"etcd_ca_cert_file\": \"/srv/kubernetes/calico/ca.pem\",\n \"etcd_cert_file\": \"/srv/kubernetes/calico/calico-client.pem\",\n \"etcd_key_file\": \"/srv/kubernetes/calico/calico-client-key.pem\",\n \"etcd_scheme\": \"https\",\n \"log_level\": \"info\",\n \"ipam\": {\n \"type\": \"calico-ipam\"\n },\n \"policy\": {\n \"type\": \"k8s\"\n },\n \"kubernetes\": {\n \"kubeconfig\": \"/etc/cni/net.d/__KUBECONFIG_FILENAME__\"\n }\n },\n {\n \"type\": \"portmap\",\n \"snat\": true,\n \"capabilities\": {\"portMappings\": true}\n }\n ]\n }\n etcd_endpoints: https://etcd-a.internal.dev-qa.k8s.domain.com:4001,https://etcd-b.internal.dev-qa.k8s.domain.com:4001,https://etcd-c.internal.dev-qa.k8s.domain.com:4001\nkind: ConfigMap\nmetadata:\n name: calico-config\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRole\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-node\nrules:\n- apiGroups:\n - \"\"\n resources:\n - pods\n - nodes\n - namespaces\n verbs:\n - get\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-node\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-node\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: calico-node\nsubjects:\n- kind: ServiceAccount\n name: calico-node\n namespace: kube-system\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRole\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\nrules:\n- apiGroups:\n - \"\"\n - extensions\n resources:\n - pods\n - namespaces\n - networkpolicies\n - nodes\n verbs:\n - watch\n - list\n- apiGroups:\n - networking.k8s.io\n resources:\n - networkpolicies\n verbs:\n - watch\n - list\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: calico-kube-controllers\nsubjects:\n- kind: ServiceAccount\n name: calico-kube-controllers\n namespace: kube-system\n\n---\n\napiVersion: extensions/v1beta1\nkind: DaemonSet\nmetadata:\n labels:\n k8s-app: calico-node\n role.kubernetes.io/networking: \"1\"\n name: calico-node\n namespace: kube-system\nspec:\n selector:\n matchLabels:\n k8s-app: calico-node\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n labels:\n k8s-app: calico-node\n role.kubernetes.io/networking: \"1\"\n spec:\n containers:\n - env:\n - name: ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n - name: CALICO_NETWORKING_BACKEND\n valueFrom:\n configMapKeyRef:\n key: calico_backend\n name: calico-config\n - name: CLUSTER_TYPE\n value: kops,bgp\n - name: CALICO_DISABLE_FILE_LOGGING\n value: \"true\"\n - name: CALICO_K8S_NODE_REF\n valueFrom:\n fieldRef:\n fieldPath: spec.nodeName\n - name: FELIX_DEFAULTENDPOINTTOHOSTACTION\n value: ACCEPT\n - name: CALICO_IPV4POOL_CIDR\n value: 100.96.0.0/11\n - name: CALICO_IPV4POOL_IPIP\n value: always\n - name: FELIX_IPV6SUPPORT\n value: \"false\"\n - name: FELIX_LOGSEVERITYSCREEN\n value: info\n - name: FELIX_PROMETHEUSMETRICSENABLED\n value: \"false\"\n - name: FELIX_PROMETHEUSMETRICSPORT\n value: \"9091\"\n - name: FELIX_PROMETHEUSGOMETRICSENABLED\n value: \"true\"\n - name: FELIX_PROMETHEUSPROCESSMETRICSENABLED\n value: \"true\"\n - name: IP\n value: autodetect\n - name: FELIX_HEALTHENABLED\n value: \"true\"\n image: quay.io/calico/node:v3.4.0\n livenessProbe:\n failureThreshold: 6\n httpGet:\n host: localhost\n path: /liveness\n port: 9099\n initialDelaySeconds: 10\n periodSeconds: 10\n name: calico-node\n readinessProbe:\n exec:\n command:\n - /bin/calico-node\n - -bird-ready\n - -felix-ready\n periodSeconds: 10\n resources:\n requests:\n cpu: 10m\n securityContext:\n privileged: true\n volumeMounts:\n - mountPath: /lib/modules\n name: lib-modules\n readOnly: true\n - mountPath: /var/run/calico\n name: var-run-calico\n readOnly: false\n - mountPath: /var/lib/calico\n name: var-lib-calico\n readOnly: false\n - mountPath: /etc/hosts\n name: etc-hosts\n readOnly: true\n - mountPath: /certs\n name: calico\n readOnly: true\n - command:\n - /install-cni.sh\n env:\n - name: CNI_CONF_NAME\n value: 10-calico.conflist\n - name: ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CNI_NETWORK_CONFIG\n valueFrom:\n configMapKeyRef:\n key: cni_network_config\n name: calico-config\n image: quay.io/calico/cni:v3.4.0\n name: install-cni\n resources:\n requests:\n cpu: 10m\n volumeMounts:\n - mountPath: /host/opt/cni/bin\n name: cni-bin-dir\n - mountPath: /host/etc/cni/net.d\n name: cni-net-dir\n - mountPath: /etc/hosts\n name: etc-hosts\n readOnly: true\n hostNetwork: true\n initContainers:\n - command:\n - /bin/sh\n - -c\n - /node-init-container.sh\n env:\n - name: CALICO_ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CALICO_APIV1_DATASTORE_TYPE\n value: etcdv2\n - name: CALICO_APIV1_ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CALICO_ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: CALICO_ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: CALICO_ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n - name: CALICO_APIV1_ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: CALICO_APIV1_ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: CALICO_APIV1_ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n image: calico/upgrade:v1.0.5\n name: migrate\n volumeMounts:\n - mountPath: /etc/hosts\n name: etc-hosts\n readOnly: true\n - mountPath: /certs\n name: calico\n readOnly: true\n serviceAccountName: calico-node\n terminationGracePeriodSeconds: 0\n tolerations:\n - effect: NoSchedule\n operator: Exists\n - key: CriticalAddonsOnly\n operator: Exists\n - effect: NoExecute\n operator: Exists\n volumes:\n - hostPath:\n path: /lib/modules\n name: lib-modules\n - hostPath:\n path: /var/run/calico\n name: var-run-calico\n - hostPath:\n path: /var/lib/calico\n name: var-lib-calico\n - hostPath:\n path: /opt/cni/bin\n name: cni-bin-dir\n - hostPath:\n path: /etc/cni/net.d\n name: cni-net-dir\n - hostPath:\n path: /etc/hosts\n name: etc-hosts\n - hostPath:\n path: /srv/kubernetes/calico\n name: calico\n updateStrategy:\n rollingUpdate:\n maxUnavailable: 1\n type: RollingUpdate\n\n---\n\napiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n labels:\n k8s-app: calico-kube-controllers\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\n namespace: kube-system\nspec:\n replicas: 1\n strategy:\n type: Recreate\n template:\n metadata:\n labels:\n k8s-app: calico-kube-controllers\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\n namespace: kube-system\n spec:\n containers:\n - env:\n - name: ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: ENABLED_CONTROLLERS\n value: policy,profile,workloadendpoint,node\n - name: ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n image: quay.io/calico/kube-controllers:v3.4.0\n name: calico-kube-controllers\n readinessProbe:\n exec:\n command:\n - /usr/bin/check-status\n - -r\n resources:\n requests:\n cpu: 10m\n volumeMounts:\n - mountPath: /certs\n name: calico\n readOnly: true\n hostNetwork: true\n initContainers:\n - command:\n - /bin/sh\n - -c\n - /controller-init.sh\n env:\n - name: CALICO_ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CALICO_APIV1_DATASTORE_TYPE\n value: etcdv2\n - name: CALICO_APIV1_ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CALICO_ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: CALICO_ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: CALICO_ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n - name: CALICO_APIV1_ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: CALICO_APIV1_ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: CALICO_APIV1_ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n image: calico/upgrade:v1.0.5\n name: migrate\n volumeMounts:\n - mountPath: /etc/hosts\n name: etc-hosts\n readOnly: true\n - mountPath: /certs\n name: calico\n readOnly: true\n serviceAccountName: calico-kube-controllers\n tolerations:\n - key: CriticalAddonsOnly\n operator: Exists\n - effect: NoSchedule\n key: node-role.kubernetes.io/master\n volumes:\n - hostPath:\n path: /etc/hosts\n name: etc-hosts\n - hostPath:\n path: /srv/kubernetes/calico\n name: calico\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-upgrade-job\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRole\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-upgrade-job\nrules:\n- apiGroups:\n - extensions\n resources:\n - daemonsets\n - daemonsets/status\n verbs:\n - get\n - list\n - watch\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-upgrade-job\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: calico-upgrade-job\nsubjects:\n- kind: ServiceAccount\n name: calico-upgrade-job\n namespace: kube-system\n\n---\n\napiVersion: batch/v1\nkind: Job\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-complete-upgrade-v331\n namespace: kube-system\nspec:\n template:\n metadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n spec:\n containers:\n - command:\n - /bin/sh\n - -c\n - /completion-job.sh\n env:\n - name: EXPECTED_NODE_IMAGE\n value: quay.io/calico/node:v3.4.0\n - name: CALICO_ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CALICO_APIV1_DATASTORE_TYPE\n value: etcdv2\n - name: CALICO_APIV1_ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CALICO_ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: CALICO_ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: CALICO_ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n - name: CALICO_APIV1_ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: CALICO_APIV1_ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: CALICO_APIV1_ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n image: calico/upgrade:v1.0.5\n name: migrate-completion\n volumeMounts:\n - mountPath: /etc/hosts\n name: etc-hosts\n readOnly: true\n - mountPath: /certs\n name: calico\n readOnly: true\n hostNetwork: true\n restartPolicy: OnFailure\n serviceAccountName: calico-upgrade-job\n volumes:\n - hostPath:\n path: /etc/hosts\n name: etc-hosts\n - hostPath:\n path: /srv/kubernetes/calico\n name: calico\n"}}
I0627 16:48:56.664314 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.6.0": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.6.0","Lifecycle":"Sync","Location":"addons/storage-aws.addons.k8s.io/v1.6.0.yaml","Contents":{"Name":"","Resource":"apiVersion: storage.k8s.io/v1beta1\nkind: StorageClass\nmetadata:\n labels:\n k8s-addon: storage-aws.addons.k8s.io\n name: default\nparameters:\n type: gp2\nprovisioner: kubernetes.io/aws-ebs\n\n---\n\napiVersion: storage.k8s.io/v1beta1\nkind: StorageClass\nmetadata:\n annotations:\n storageclass.beta.kubernetes.io/is-default-class: \"true\"\n labels:\n k8s-addon: storage-aws.addons.k8s.io\n name: gp2\nparameters:\n type: gp2\nprovisioner: kubernetes.io/aws-ebs\n"}}
I0627 16:48:56.664212 22451 executor.go:178] Executing task "EBSVolume/b.etcd-events.dev-qa.k8s.domain.com": *awstasks.EBSVolume {"Name":"b.etcd-events.dev-qa.k8s.domain.com","Lifecycle":"Sync","AvailabilityZone":"eu-central-1b","Encrypted":true,"ID":null,"KmsKeyId":null,"SizeGB":20,"Tags":{"k8s.io/etcd/events":"b/a,b,c","k8s.io/role/master":"1","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"VolumeIops":null,"VolumeType":"gp2"}
I0627 16:48:56.663775 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.12": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.12","Lifecycle":"Sync","Location":"addons/kube-dns.addons.k8s.io/k8s-1.12.yaml","Contents":{"Name":"","Resource":"null\n\n---\n\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns-autoscaler\n kubernetes.io/cluster-service: \"true\"\n name: kube-dns-autoscaler\n namespace: kube-system\nspec:\n selector:\n matchLabels:\n k8s-app: kube-dns-autoscaler\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n labels:\n k8s-app: kube-dns-autoscaler\n spec:\n containers:\n - command:\n - /cluster-proportional-autoscaler\n - --namespace=kube-system\n - --configmap=kube-dns-autoscaler\n - --target=Deployment/kube-dns\n - --default-params={\"linear\":{\"coresPerReplica\":256,\"nodesPerReplica\":16,\"preventSinglePointFailure\":true}}\n - --logtostderr=true\n - --v=2\n image: k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.4.0\n name: autoscaler\n resources:\n requests:\n cpu: 20m\n memory: 10Mi\n serviceAccountName: kube-dns-autoscaler\n tolerations:\n - key: CriticalAddonsOnly\n operator: Exists\n\n---\n\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns\n kubernetes.io/cluster-service: \"true\"\n name: kube-dns\n namespace: kube-system\nspec:\n selector:\n matchLabels:\n k8s-app: kube-dns\n strategy:\n rollingUpdate:\n maxSurge: 10%\n maxUnavailable: 0\n template:\n metadata:\n annotations:\n prometheus.io/port: \"10055\"\n prometheus.io/scrape: \"true\"\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n labels:\n k8s-app: kube-dns\n spec:\n containers:\n - args:\n - --config-dir=/kube-dns-config\n - --dns-port=10053\n - --domain=cluster.local.\n - --v=2\n env:\n - name: PROMETHEUS_PORT\n value: \"10055\"\n image: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /healthcheck/kubedns\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: kubedns\n ports:\n - containerPort: 10053\n name: dns-local\n protocol: UDP\n - containerPort: 10053\n name: dns-tcp-local\n protocol: TCP\n - containerPort: 10055\n name: metrics\n protocol: TCP\n readinessProbe:\n httpGet:\n path: /readiness\n port: 8081\n scheme: HTTP\n initialDelaySeconds: 3\n timeoutSeconds: 5\n resources:\n limits:\n memory: 170Mi\n requests:\n cpu: 100m\n memory: 70Mi\n volumeMounts:\n - mountPath: /kube-dns-config\n name: kube-dns-config\n - args:\n - -v=2\n - -logtostderr\n - -configDir=/etc/k8s/dns/dnsmasq-nanny\n - -restartDnsmasq=true\n - --\n - -k\n - --cache-size=1000\n - --dns-forward-max=150\n - --no-negcache\n - --log-facility=-\n - --server=/cluster.local/127.0.0.1#10053\n - --server=/in-addr.arpa/127.0.0.1#10053\n - --server=/in6.arpa/127.0.0.1#10053\n image: k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.13\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /healthcheck/dnsmasq\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: dnsmasq\n ports:\n - containerPort: 53\n name: dns\n protocol: UDP\n - containerPort: 53\n name: dns-tcp\n protocol: TCP\n resources:\n requests:\n cpu: 150m\n memory: 20Mi\n volumeMounts:\n - mountPath: /etc/k8s/dns/dnsmasq-nanny\n name: kube-dns-config\n - args:\n - --v=2\n - --logtostderr\n - --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local,5,A\n - --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local,5,A\n image: k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.13\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /metrics\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: sidecar\n ports:\n - containerPort: 10054\n name: metrics\n protocol: TCP\n resources:\n requests:\n cpu: 10m\n memory: 20Mi\n dnsPolicy: Default\n serviceAccountName: kube-dns\n volumes:\n - configMap:\n name: kube-dns\n optional: true\n name: kube-dns-config\n\n---\n\napiVersion: v1\nkind: Service\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns\n kubernetes.io/cluster-service: \"true\"\n kubernetes.io/name: KubeDNS\n name: kube-dns\n namespace: kube-system\nspec:\n clusterIP: 100.64.0.10\n ports:\n - name: dns\n port: 53\n protocol: UDP\n - name: dns-tcp\n port: 53\n protocol: TCP\n selector:\n k8s-app: kube-dns\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n name: kube-dns-autoscaler\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n name: kube-dns-autoscaler\nrules:\n- apiGroups:\n - \"\"\n resources:\n - nodes\n verbs:\n - list\n - watch\n- apiGroups:\n - \"\"\n resources:\n - replicationcontrollers/scale\n verbs:\n - get\n - update\n- apiGroups:\n - extensions\n - apps\n resources:\n - deployments/scale\n - replicasets/scale\n verbs:\n - get\n - update\n- apiGroups:\n - \"\"\n resources:\n - configmaps\n verbs:\n - get\n - create\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n name: kube-dns-autoscaler\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: kube-dns-autoscaler\nsubjects:\n- kind: ServiceAccount\n name: kube-dns-autoscaler\n namespace: kube-system\n"}}
I0627 16:48:56.664373 22451 executor.go:178] Executing task "IAMRole/nodes.dev-qa.k8s.domain.com": *awstasks.IAMRole {"ID":null,"Lifecycle":"Sync","Name":"nodes.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"nodes"}
I0627 16:48:56.664152 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.12": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.12","Lifecycle":"Sync","Location":"addons/networking.projectcalico.org/k8s-1.12.yaml","Contents":{"Name":"","Resource":"apiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: felixconfigurations.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: FelixConfiguration\n plural: felixconfigurations\n singular: felixconfiguration\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: bgppeers.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: BGPPeer\n plural: bgppeers\n singular: bgppeer\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: bgpconfigurations.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: BGPConfiguration\n plural: bgpconfigurations\n singular: bgpconfiguration\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: ippools.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: IPPool\n plural: ippools\n singular: ippool\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: hostendpoints.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: HostEndpoint\n plural: hostendpoints\n singular: hostendpoint\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: clusterinformations.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: ClusterInformation\n plural: clusterinformations\n singular: clusterinformation\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: globalnetworkpolicies.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: GlobalNetworkPolicy\n plural: globalnetworkpolicies\n singular: globalnetworkpolicy\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: globalnetworksets.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: GlobalNetworkSet\n plural: globalnetworksets\n singular: globalnetworkset\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: networkpolicies.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: NetworkPolicy\n plural: networkpolicies\n singular: networkpolicy\n scope: Namespaced\n version: v1\n\n---\n\napiVersion: v1\ndata:\n calico_backend: bird\n cni_network_config: |-\n {\n \"name\": \"k8s-pod-network\",\n \"cniVersion\": \"0.3.0\",\n \"plugins\": [\n {\n \"type\": \"calico\",\n \"log_level\": \"info\",\n \"datastore_type\": \"kubernetes\",\n \"nodename\": \"__KUBERNETES_NODE_NAME__\",\n \"mtu\": __CNI_MTU__,\n \"ipam\": {\n \"type\": \"host-local\",\n \"subnet\": \"usePodCidr\"\n },\n \"policy\": {\n \"type\": \"k8s\"\n },\n \"kubernetes\": {\n \"kubeconfig\": \"/etc/cni/net.d/__KUBECONFIG_FILENAME__\"\n }\n },\n {\n \"type\": \"portmap\",\n \"snat\": true,\n \"capabilities\": {\"portMappings\": true}\n }\n ]\n }\n typha_service_name: none\n veth_mtu: \"1440\"\nkind: ConfigMap\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-config\n namespace: kube-system\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-node\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-node\nrules:\n- apiGroups:\n - \"\"\n resources:\n - pods\n - nodes\n - namespaces\n verbs:\n - get\n- apiGroups:\n - \"\"\n resources:\n - endpoints\n - services\n verbs:\n - watch\n - list\n - get\n- apiGroups:\n - \"\"\n resources:\n - nodes/status\n verbs:\n - patch\n - update\n- apiGroups:\n - networking.k8s.io\n resources:\n - networkpolicies\n verbs:\n - watch\n - list\n- apiGroups:\n - \"\"\n resources:\n - pods\n - namespaces\n - serviceaccounts\n verbs:\n - list\n - watch\n- apiGroups:\n - \"\"\n resources:\n - pods/status\n verbs:\n - patch\n- apiGroups:\n - crd.projectcalico.org\n resources:\n - globalfelixconfigs\n - felixconfigurations\n - bgppeers\n - globalbgpconfigs\n - bgpconfigurations\n - ippools\n - globalnetworkpolicies\n - globalnetworksets\n - networkpolicies\n - clusterinformations\n - hostendpoints\n verbs:\n - get\n - list\n - watch\n- apiGroups:\n - crd.projectcalico.org\n resources:\n - ippools\n - felixconfigurations\n - clusterinformations\n verbs:\n - create\n - update\n- apiGroups:\n - \"\"\n resources:\n - nodes\n verbs:\n - get\n - list\n - watch\n- apiGroups:\n - crd.projectcalico.org\n resources:\n - bgpconfigurations\n - bgppeers\n verbs:\n - create\n - update\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-node\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: calico-node\nsubjects:\n- kind: ServiceAccount\n name: calico-node\n namespace: kube-system\n\n---\n\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n labels:\n k8s-app: calico-kube-controllers\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\n namespace: kube-system\nspec:\n replicas: 0\n selector:\n matchLabels:\n k8s-app: calico-kube-controllers\n strategy:\n type: Recreate\n template:\n metadata:\n labels:\n k8s-app: calico-kube-controllers\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\n namespace: kube-system\n spec:\n containers:\n - image: quay.io/calico/kube-controllers:v3.4.0\n name: calico-kube-controllers\n initContainers:\n - image: calico/upgrade:v1.0.5\n name: migrate\n\n---\n\napiVersion: apps/v1\nkind: DaemonSet\nmetadata:\n labels:\n k8s-app: calico-node\n role.kubernetes.io/networking: \"1\"\n name: calico-node\n namespace: kube-system\nspec:\n selector:\n matchLabels:\n k8s-app: calico-node\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n labels:\n k8s-app: calico-node\n role.kubernetes.io/networking: \"1\"\n spec:\n containers:\n - env:\n - name: DATASTORE_TYPE\n value: kubernetes\n - name: FELIX_TYPHAK8SSERVICENAME\n valueFrom:\n configMapKeyRef:\n key: typha_service_name\n name: calico-config\n - name: WAIT_FOR_DATASTORE\n value: \"true\"\n - name: NODENAME\n valueFrom:\n fieldRef:\n fieldPath: spec.nodeName\n - name: CALICO_NETWORKING_BACKEND\n valueFrom:\n configMapKeyRef:\n key: calico_backend\n name: calico-config\n - name: CLUSTER_TYPE\n value: kops,bgp\n - name: IP\n value: autodetect\n - name: CALICO_IPV4POOL_IPIP\n value: always\n - name: FELIX_IPINIPMTU\n valueFrom:\n configMapKeyRef:\n key: veth_mtu\n name: calico-config\n - name: CALICO_IPV4POOL_CIDR\n value: 100.96.0.0/11\n - name: CALICO_DISABLE_FILE_LOGGING\n value: \"true\"\n - name: FELIX_DEFAULTENDPOINTTOHOSTACTION\n value: ACCEPT\n - name: FELIX_IPV6SUPPORT\n value: \"false\"\n - name: FELIX_LOGSEVERITYSCREEN\n value: info\n - name: FELIX_HEALTHENABLED\n value: \"true\"\n - name: FELIX_PROMETHEUSMETRICSENABLED\n value: \"false\"\n - name: FELIX_PROMETHEUSMETRICSPORT\n value: \"9091\"\n - name: FELIX_PROMETHEUSGOMETRICSENABLED\n value: \"true\"\n - name: FELIX_PROMETHEUSPROCESSMETRICSENABLED\n value: \"true\"\n image: quay.io/calico/node:v3.4.0\n livenessProbe:\n failureThreshold: 6\n httpGet:\n host: localhost\n path: /liveness\n port: 9099\n initialDelaySeconds: 10\n periodSeconds: 10\n name: calico-node\n readinessProbe:\n exec:\n command:\n - /bin/calico-node\n - -bird-ready\n - -felix-ready\n periodSeconds: 10\n resources:\n requests:\n cpu: 10m\n securityContext:\n privileged: true\n volumeMounts:\n - mountPath: /lib/modules\n name: lib-modules\n readOnly: true\n - mountPath: /run/xtables.lock\n name: xtables-lock\n readOnly: false\n - mountPath: /var/run/calico\n name: var-run-calico\n readOnly: false\n - mountPath: /var/lib/calico\n name: var-lib-calico\n readOnly: false\n - mountPath: /etc/hosts\n name: etc-hosts\n readOnly: true\n hostNetwork: true\n initContainers:\n - command:\n - /install-cni.sh\n env:\n - name: CNI_CONF_NAME\n value: 10-calico.conflist\n - name: CNI_NETWORK_CONFIG\n valueFrom:\n configMapKeyRef:\n key: cni_network_config\n name: calico-config\n - name: KUBERNETES_NODE_NAME\n valueFrom:\n fieldRef:\n fieldPath: spec.nodeName\n - name: CNI_MTU\n valueFrom:\n configMapKeyRef:\n key: veth_mtu\n name: calico-config\n - name: SLEEP\n value: \"false\"\n image: quay.io/calico/cni:v3.4.0\n name: install-cni\n volumeMounts:\n - mountPath: /host/opt/cni/bin\n name: cni-bin-dir\n - mountPath: /host/etc/cni/net.d\n name: cni-net-dir\n serviceAccountName: calico-node\n terminationGracePeriodSeconds: 0\n tolerations:\n - effect: NoSchedule\n operator: Exists\n - key: CriticalAddonsOnly\n operator: Exists\n - effect: NoExecute\n operator: Exists\n volumes:\n - hostPath:\n path: /lib/modules\n name: lib-modules\n - hostPath:\n path: /var/run/calico\n name: var-run-calico\n - hostPath:\n path: /var/lib/calico\n name: var-lib-calico\n - hostPath:\n path: /run/xtables.lock\n type: FileOrCreate\n name: xtables-lock\n - hostPath:\n path: /opt/cni/bin\n name: cni-bin-dir\n - hostPath:\n path: /etc/cni/net.d\n name: cni-net-dir\n - hostPath:\n path: /etc/hosts\n name: etc-hosts\n updateStrategy:\n rollingUpdate:\n maxUnavailable: 1\n type: RollingUpdate\n\n---\n\nnull\n"}}
I0627 16:48:56.663984 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-pre-k8s-1.6": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-pre-k8s-1.6","Lifecycle":"Sync","Location":"addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml","Contents":{"Name":"","Resource":"apiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n k8s-app: dns-controller\n version: v1.12.0\n name: dns-controller\n namespace: kube-system\nspec:\n replicas: 1\n selector:\n matchLabels:\n k8s-app: dns-controller\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n scheduler.alpha.kubernetes.io/tolerations: '[{\"key\": \"dedicated\", \"value\":\n \"master\"}]'\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n k8s-app: dns-controller\n version: v1.12.0\n spec:\n containers:\n - command:\n - /usr/bin/dns-controller\n - --watch-ingress=false\n - --dns=aws-route53\n - --zone=k8s.domain.com\n - --zone=*/*\n - -v=2\n image: kope/dns-controller:1.12.0\n name: dns-controller\n resources:\n requests:\n cpu: 50m\n memory: 50Mi\n dnsPolicy: Default\n hostNetwork: true\n nodeSelector:\n kubernetes.io/role: master\n"}}
I0627 16:48:56.664374 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-rbac.addons.k8s.io-k8s-1.8": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-rbac.addons.k8s.io-k8s-1.8","Lifecycle":"Sync","Location":"addons/rbac.addons.k8s.io/k8s-1.8.yaml","Contents":{"Name":"","Resource":"apiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n addonmanager.kubernetes.io/mode: Reconcile\n k8s-addon: rbac.addons.k8s.io\n kubernetes.io/cluster-service: \"true\"\n name: kubelet-cluster-admin\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: system:node\nsubjects:\n- apiGroup: rbac.authorization.k8s.io\n kind: User\n name: kubelet\n"}}
I0627 16:48:56.663790 22451 executor.go:178] Executing task "Keypair/ca": *fitasks.Keypair {"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"}
I0627 16:48:56.663939 22451 executor.go:178] Executing task "Keypair/etcd-manager-ca-events": *fitasks.Keypair {"Name":"etcd-manager-ca-events","alternateNames":null,"alternateNameTasks":null,"Lifecycle":null,"Signer":null,"subject":"cn=etcd-manager-ca-events","type":"ca","format":"v1alpha2"}
I0627 16:48:56.663923 22451 executor.go:178] Executing task "Keypair/etcd-clients-ca": *fitasks.Keypair {"Name":"etcd-clients-ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":null,"Signer":null,"subject":"cn=etcd-clients-ca","type":"ca","format":"v1alpha2"}
I0627 16:48:56.664376 22451 executor.go:178] Executing task "Keypair/apiserver-aggregator-ca": *fitasks.Keypair {"Name":"apiserver-aggregator-ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=apiserver-aggregator-ca","type":"ca","format":"v1alpha2"}
I0627 16:48:56.664203 22451 executor.go:178] Executing task "Keypair/etcd-peers-ca-events": *fitasks.Keypair {"Name":"etcd-peers-ca-events","alternateNames":null,"alternateNameTasks":null,"Lifecycle":null,"Signer":null,"subject":"cn=etcd-peers-ca-events","type":"ca","format":"v1alpha2"}
I0627 16:48:56.663935 22451 executor.go:178] Executing task "Keypair/etcd-peers-ca-main": *fitasks.Keypair {"Name":"etcd-peers-ca-main","alternateNames":null,"alternateNameTasks":null,"Lifecycle":null,"Signer":null,"subject":"cn=etcd-peers-ca-main","type":"ca","format":"v1alpha2"}
I0627 16:48:56.663972 22451 executor.go:178] Executing task "Keypair/etcd-manager-ca-main": *fitasks.Keypair {"Name":"etcd-manager-ca-main","alternateNames":null,"alternateNameTasks":null,"Lifecycle":null,"Signer":null,"subject":"cn=etcd-manager-ca-main","type":"ca","format":"v1alpha2"}
I0627 16:48:56.664390 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.12": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.12","Lifecycle":"Sync","Location":"addons/dns-controller.addons.k8s.io/k8s-1.12.yaml","Contents":{"Name":"","Resource":"apiVersion: apps/v1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n k8s-app: dns-controller\n version: v1.12.0\n name: dns-controller\n namespace: kube-system\nspec:\n replicas: 1\n selector:\n matchLabels:\n k8s-app: dns-controller\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n k8s-app: dns-controller\n version: v1.12.0\n spec:\n containers:\n - command:\n - /usr/bin/dns-controller\n - --watch-ingress=false\n - --dns=aws-route53\n - --zone=k8s.domain.com\n - --zone=*/*\n - -v=2\n image: kope/dns-controller:1.12.0\n name: dns-controller\n resources:\n requests:\n cpu: 50m\n memory: 50Mi\n dnsPolicy: Default\n hostNetwork: true\n nodeSelector:\n node-role.kubernetes.io/master: \"\"\n serviceAccount: dns-controller\n tolerations:\n - effect: NoSchedule\n key: node-role.kubernetes.io/master\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n name: dns-controller\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n name: kops:dns-controller\nrules:\n- apiGroups:\n - \"\"\n resources:\n - endpoints\n - services\n - pods\n - ingress\n - nodes\n verbs:\n - get\n - list\n - watch\n- apiGroups:\n - extensions\n resources:\n - ingresses\n verbs:\n - get\n - list\n - watch\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n name: kops:dns-controller\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: kops:dns-controller\nsubjects:\n- apiGroup: rbac.authorization.k8s.io\n kind: User\n name: system:serviceaccount:kube-system:dns-controller\n"}}
I0627 16:48:56.663911 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.6": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.6","Lifecycle":"Sync","Location":"addons/kube-dns.addons.k8s.io/k8s-1.6.yaml","Contents":{"Name":"","Resource":"null\n\n---\n\napiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns-autoscaler\n kubernetes.io/cluster-service: \"true\"\n name: kube-dns-autoscaler\n namespace: kube-system\nspec:\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n scheduler.alpha.kubernetes.io/tolerations: '[{\"key\":\"CriticalAddonsOnly\",\n \"operator\":\"Exists\"}]'\n labels:\n k8s-app: kube-dns-autoscaler\n spec:\n containers:\n - command:\n - /cluster-proportional-autoscaler\n - --namespace=kube-system\n - --configmap=kube-dns-autoscaler\n - --target=Deployment/kube-dns\n - --default-params={\"linear\":{\"coresPerReplica\":256,\"nodesPerReplica\":16,\"preventSinglePointFailure\":true}}\n - --logtostderr=true\n - --v=2\n image: k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.1.2-r2\n name: autoscaler\n resources:\n requests:\n cpu: 20m\n memory: 10Mi\n serviceAccountName: kube-dns-autoscaler\n tolerations:\n - key: CriticalAddonsOnly\n operator: Exists\n\n---\n\napiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns\n kubernetes.io/cluster-service: \"true\"\n name: kube-dns\n namespace: kube-system\nspec:\n selector:\n matchLabels:\n k8s-app: kube-dns\n strategy:\n rollingUpdate:\n maxSurge: 10%\n maxUnavailable: 0\n template:\n metadata:\n annotations:\n prometheus.io/port: \"10055\"\n prometheus.io/scrape: \"true\"\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n scheduler.alpha.kubernetes.io/tolerations: '[{\"key\":\"CriticalAddonsOnly\",\n \"operator\":\"Exists\"}]'\n labels:\n k8s-app: kube-dns\n spec:\n containers:\n - args:\n - --config-dir=/kube-dns-config\n - --dns-port=10053\n - --domain=cluster.local.\n - --v=2\n env:\n - name: PROMETHEUS_PORT\n value: \"10055\"\n image: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.10\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /healthcheck/kubedns\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: kubedns\n ports:\n - containerPort: 10053\n name: dns-local\n protocol: UDP\n - containerPort: 10053\n name: dns-tcp-local\n protocol: TCP\n - containerPort: 10055\n name: metrics\n protocol: TCP\n readinessProbe:\n httpGet:\n path: /readiness\n port: 8081\n scheme: HTTP\n initialDelaySeconds: 3\n timeoutSeconds: 5\n resources:\n limits:\n memory: 170Mi\n requests:\n cpu: 100m\n memory: 70Mi\n volumeMounts:\n - mountPath: /kube-dns-config\n name: kube-dns-config\n - args:\n - -v=2\n - -logtostderr\n - -configDir=/etc/k8s/dns/dnsmasq-nanny\n - -restartDnsmasq=true\n - --\n - -k\n - --cache-size=1000\n - --dns-forward-max=150\n - --no-negcache\n - --log-facility=-\n - --server=/cluster.local/127.0.0.1#10053\n - --server=/in-addr.arpa/127.0.0.1#10053\n - --server=/in6.arpa/127.0.0.1#10053\n image: k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.10\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /healthcheck/dnsmasq\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: dnsmasq\n ports:\n - containerPort: 53\n name: dns\n protocol: UDP\n - containerPort: 53\n name: dns-tcp\n protocol: TCP\n resources:\n requests:\n cpu: 150m\n memory: 20Mi\n volumeMounts:\n - mountPath: /etc/k8s/dns/dnsmasq-nanny\n name: kube-dns-config\n - args:\n - --v=2\n - --logtostderr\n - --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local,5,A\n - --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local,5,A\n image: k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.10\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /metrics\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: sidecar\n ports:\n - containerPort: 10054\n name: metrics\n protocol: TCP\n resources:\n requests:\n cpu: 10m\n memory: 20Mi\n dnsPolicy: Default\n serviceAccountName: kube-dns\n volumes:\n - configMap:\n name: kube-dns\n optional: true\n name: kube-dns-config\n\n---\n\napiVersion: v1\nkind: Service\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns\n kubernetes.io/cluster-service: \"true\"\n kubernetes.io/name: KubeDNS\n name: kube-dns\n namespace: kube-system\nspec:\n clusterIP: 100.64.0.10\n ports:\n - name: dns\n port: 53\n protocol: UDP\n - name: dns-tcp\n port: 53\n protocol: TCP\n selector:\n k8s-app: kube-dns\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n name: kube-dns-autoscaler\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRole\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n name: kube-dns-autoscaler\nrules:\n- apiGroups:\n - \"\"\n resources:\n - nodes\n verbs:\n - list\n- apiGroups:\n - \"\"\n resources:\n - replicationcontrollers/scale\n verbs:\n - get\n - update\n- apiGroups:\n - extensions\n resources:\n - deployments/scale\n - replicasets/scale\n verbs:\n - get\n - update\n- apiGroups:\n - \"\"\n resources:\n - configmaps\n verbs:\n - get\n - create\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n name: kube-dns-autoscaler\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: kube-dns-autoscaler\nsubjects:\n- kind: ServiceAccount\n name: kube-dns-autoscaler\n namespace: kube-system\n"}}
I0627 16:48:56.664224 22451 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-pre-k8s-1.6": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-pre-k8s-1.6","Lifecycle":"Sync","Location":"addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml","Contents":{"Name":"","Resource":"apiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns-autoscaler\n kubernetes.io/cluster-service: \"true\"\n name: kube-dns-autoscaler\n namespace: kube-system\nspec:\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n scheduler.alpha.kubernetes.io/tolerations: '[{\"key\":\"CriticalAddonsOnly\",\n \"operator\":\"Exists\"}]'\n labels:\n k8s-app: kube-dns-autoscaler\n spec:\n containers:\n - command:\n - /cluster-proportional-autoscaler\n - --namespace=kube-system\n - --configmap=kube-dns-autoscaler\n - --mode=linear\n - --target=Deployment/kube-dns\n - --default-params={\"linear\":{\"coresPerReplica\":256,\"nodesPerReplica\":16,\"min\":2}}\n - --logtostderr=true\n - --v=2\n image: k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.0.0\n name: autoscaler\n resources:\n requests:\n cpu: 20m\n memory: 10Mi\n\n---\n\napiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns\n kubernetes.io/cluster-service: \"true\"\n name: kube-dns\n namespace: kube-system\nspec:\n selector:\n matchLabels:\n k8s-app: kube-dns\n strategy:\n rollingUpdate:\n maxSurge: 10%\n maxUnavailable: 0\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n scheduler.alpha.kubernetes.io/tolerations: '[{\"key\":\"CriticalAddonsOnly\",\n \"operator\":\"Exists\"}]'\n labels:\n k8s-app: kube-dns\n spec:\n containers:\n - args:\n - --domain=cluster.local.\n - --dns-port=10053\n - --config-map=kube-dns\n - --v=2\n env:\n - name: PROMETHEUS_PORT\n value: \"10055\"\n image: k8s.gcr.io/kubedns-amd64:1.9\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /healthz-kubedns\n port: 8080\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: kubedns\n ports:\n - containerPort: 10053\n name: dns-local\n protocol: UDP\n - containerPort: 10053\n name: dns-tcp-local\n protocol: TCP\n - containerPort: 10055\n name: metrics\n protocol: TCP\n readinessProbe:\n httpGet:\n path: /readiness\n port: 8081\n scheme: HTTP\n initialDelaySeconds: 3\n timeoutSeconds: 5\n resources:\n limits:\n memory: 170Mi\n requests:\n cpu: 100m\n memory: 70Mi\n - args:\n - --cache-size=1000\n - --dns-forward-max=150\n - --no-resolv\n - --server=127.0.0.1#10053\n - --log-facility=-\n image: k8s.gcr.io/k8s-dns-dnsmasq-amd64:1.14.10\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /healthz-dnsmasq\n port: 8080\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: dnsmasq\n ports:\n - containerPort: 53\n name: dns\n protocol: UDP\n - containerPort: 53\n name: dns-tcp\n protocol: TCP\n resources:\n requests:\n cpu: 150m\n memory: 10Mi\n - args:\n - --v=2\n - --logtostderr\n image: k8s.gcr.io/dnsmasq-metrics-amd64:1.0\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /metrics\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: dnsmasq-metrics\n ports:\n - containerPort: 10054\n name: metrics\n protocol: TCP\n resources:\n requests:\n memory: 10Mi\n - args:\n - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1 \u003e/dev/null\n - --url=/healthz-dnsmasq\n - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1:10053 \u003e/dev/null\n - --url=/healthz-kubedns\n - --port=8080\n - --quiet\n image: k8s.gcr.io/exechealthz-amd64:1.2\n name: healthz\n ports:\n - containerPort: 8080\n protocol: TCP\n resources:\n limits:\n memory: 50Mi\n requests:\n cpu: 10m\n memory: 50Mi\n dnsPolicy: Default\n\n---\n\napiVersion: v1\nkind: Service\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns\n kubernetes.io/cluster-service: \"true\"\n kubernetes.io/name: KubeDNS\n name: kube-dns\n namespace: kube-system\nspec:\n clusterIP: 100.64.0.10\n ports:\n - name: dns\n port: 53\n protocol: UDP\n - name: dns-tcp\n port: 53\n protocol: TCP\n selector:\n k8s-app: kube-dns\n"}}
I0627 16:48:56.668224 22451 request_logger.go:45] AWS request: ec2/DescribeVpcs
I0627 16:48:56.671122 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/bootstrap-channel.yaml"
I0627 16:48:56.674321 22451 sshkey.go:109] Computed SSH key fingerprint as "c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"
I0627 16:48:56.674897 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/limit-range.addons.k8s.io/v1.5.0.yaml"
I0627 16:48:56.680171 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/dns-controller.addons.k8s.io/k8s-1.6.yaml"
I0627 16:48:56.681830 22451 request_logger.go:45] AWS request: iam/GetRole
I0627 16:48:56.682831 22451 request_logger.go:45] AWS request: iam/GetRole
I0627 16:48:56.684276 22451 request_logger.go:45] AWS request: ec2/DescribeDhcpOptions
I0627 16:48:56.686258 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/manifests/etcd/main.yaml"
I0627 16:48:56.687086 22451 request_logger.go:45] AWS request: route53/ListHostedZonesByName
I0627 16:48:56.687444 22451 request_logger.go:45] AWS request: iam/GetInstanceProfile
I0627 16:48:56.687963 22451 request_logger.go:45] AWS request: iam/GetInstanceProfile
I0627 16:48:56.690064 22451 request_logger.go:45] AWS request: iam/GetInstanceProfile
I0627 16:48:56.690673 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml"
I0627 16:48:56.691083 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/core.addons.k8s.io/v1.4.0.yaml"
I0627 16:48:56.692778 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/manifests/etcd/events.yaml"
I0627 16:48:56.693140 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/admin"
I0627 16:48:56.693593 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main/control/etcd-cluster-spec"
I0627 16:48:56.693934 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kube"
I0627 16:48:56.694656 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/storage-aws.addons.k8s.io/v1.7.0.yaml"
I0627 16:48:56.694976 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kubelet"
I0627 16:48:56.695574 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:monitoring"
I0627 16:48:56.696246 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kube-proxy"
I0627 16:48:56.697689 22451 request_logger.go:45] AWS request: ec2/DescribeVolumes
I0627 16:48:56.697922 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:controller_manager"
I0627 16:48:56.698553 22451 request_logger.go:45] AWS request: ec2/DescribeVolumes
I0627 16:48:56.704632 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:logging"
I0627 16:48:56.705078 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:dns"
I0627 16:48:56.705438 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:scheduler"
I0627 16:48:56.706113 22451 request_logger.go:45] AWS request: ec2/DescribeVolumes
I0627 16:48:56.706730 22451 request_logger.go:45] AWS request: ec2/DescribeVolumes
I0627 16:48:56.707441 22451 request_logger.go:45] AWS request: ec2/DescribeVolumes
I0627 16:48:56.723167 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/networking.projectcalico.org/k8s-1.7-v3.yaml"
I0627 16:48:56.724008 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/storage-aws.addons.k8s.io/v1.6.0.yaml"
I0627 16:48:56.724798 22451 request_logger.go:45] AWS request: ec2/DescribeVolumes
I0627 16:48:56.729211 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/kube-dns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.729736 22451 request_logger.go:45] AWS request: iam/GetRole
I0627 16:48:56.742643 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/networking.projectcalico.org/k8s-1.12.yaml"
I0627 16:48:56.746204 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:48:56.746892 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/rbac.addons.k8s.io/k8s-1.8.yaml"
I0627 16:48:56.747326 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml"
I0627 16:48:56.747704 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/keyset.yaml"
I0627 16:48:56.748133 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/keyset.yaml"
I0627 16:48:56.748570 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/keyset.yaml"
I0627 16:48:56.748993 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/keyset.yaml"
I0627 16:48:56.749394 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/keyset.yaml"
I0627 16:48:56.749791 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/keyset.yaml"
I0627 16:48:56.755072 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/dns-controller.addons.k8s.io/k8s-1.12.yaml"
I0627 16:48:56.759726 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/kube-dns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:48:56.763266 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:48:56.769387 22451 request_logger.go:45] AWS request: ec2/DescribeKeyPairs
I0627 16:48:57.022379 22451 request_logger.go:45] AWS request: route53/GetHostedZone
I0627 16:48:57.897892 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml", falling back to directory-list method
I0627 16:48:57.898536 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/ca/"
I0627 16:48:57.942983 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/keyset.yaml", falling back to directory-list method
I0627 16:48:57.943734 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/"
I0627 16:48:57.961039 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/keyset.yaml", falling back to directory-list method
I0627 16:48:57.962129 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/"
I0627 16:48:57.971885 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/keyset.yaml", falling back to directory-list method
I0627 16:48:57.972417 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/"
I0627 16:48:57.995242 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/keyset.yaml", falling back to directory-list method
I0627 16:48:57.995808 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/"
I0627 16:48:58.000275 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/keyset.yaml", falling back to directory-list method
I0627 16:48:58.000803 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/"
I0627 16:48:58.021409 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/keyset.yaml", falling back to directory-list method
I0627 16:48:58.022065 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/"
I0627 16:48:58.109622 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca: []
I0627 16:48:58.110411 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml"
I0627 16:48:58.127993 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main: []
I0627 16:48:58.128419 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/keyset.yaml"
I0627 16:48:58.147468 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca: []
I0627 16:48:58.148110 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/keyset.yaml"
I0627 16:48:58.179997 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events: []
I0627 16:48:58.180900 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/keyset.yaml"
I0627 16:48:58.191471 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca: []
I0627 16:48:58.191983 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/keyset.yaml"
I0627 16:48:58.192000 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events: []
I0627 16:48:58.193201 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/keyset.yaml"
I0627 16:48:58.207824 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main: []
I0627 16:48:58.208426 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/keyset.yaml"
I0627 16:48:58.292995 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml", falling back to directory-list method
I0627 16:48:58.293580 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/ca/"
I0627 16:48:58.331417 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/keyset.yaml", falling back to directory-list method
I0627 16:48:58.332128 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/"
I0627 16:48:58.331427 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/keyset.yaml", falling back to directory-list method
I0627 16:48:58.333150 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/"
I0627 16:48:58.362363 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/keyset.yaml", falling back to directory-list method
I0627 16:48:58.363242 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/"
I0627 16:48:58.379683 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/keyset.yaml", falling back to directory-list method
I0627 16:48:58.380217 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/"
I0627 16:48:58.387586 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/keyset.yaml", falling back to directory-list method
I0627 16:48:58.388087 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/"
I0627 16:48:58.395742 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/keyset.yaml", falling back to directory-list method
I0627 16:48:58.396439 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/"
I0627 16:48:58.483210 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca: []
I0627 16:48:58.518430 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main: []
I0627 16:48:58.522401 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-clients-ca: []
I0627 16:48:58.561361 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events: []
I0627 16:48:58.570247 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca: []
I0627 16:48:58.582264 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main: []
I0627 16:48:58.586314 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events: []
I0627 16:48:58.587860 22451 executor.go:103] Tasks: 51 done / 143 total; 38 can run
I0627 16:48:58.589997 22451 executor.go:178] Executing task "Keypair/master": *fitasks.Keypair {"Name":"master","alternateNames":["kubernetes","kubernetes.default","kubernetes.default.svc","kubernetes.default.svc.cluster.local","api.dev-qa.k8s.domain.com","api.internal.dev-qa.k8s.domain.com","100.64.0.1","127.0.0.1"],"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"cn=kubernetes-master","type":"server","format":"v1alpha2"}
I0627 16:48:58.591037 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master/keyset.yaml"
I0627 16:48:58.590018 22451 executor.go:178] Executing task "Keypair/kubecfg": *fitasks.Keypair {"Name":"kubecfg","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"o=system:masters,cn=kubecfg","type":"client","format":"v1alpha2"}
I0627 16:48:58.592553 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg/keyset.yaml"
I0627 16:48:58.590203 22451 executor.go:178] Executing task "IAMRolePolicy/nodes.dev-qa.k8s.domain.com": *awstasks.IAMRolePolicy {"ID":null,"Lifecycle":"Sync","Name":"nodes.dev-qa.k8s.domain.com","Role":{"ID":null,"Lifecycle":"Sync","Name":"nodes.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"nodes"},"PolicyDocument":{"Builder":{"Cluster":{"metadata":{"name":"dev-qa.k8s.domain.com","creationTimestamp":"2019-06-27T14:48:45Z"},"spec":{"channel":"stable","configBase":"s3://my-state-store/dev-qa.k8s.domain.com","cloudProvider":"aws","kubernetesVersion":"1.12.9","subnets":[{"name":"eu-central-1a","cidr":"172.31.239.32/27","zone":"eu-central-1a","type":"Private"},{"name":"eu-central-1b","cidr":"172.31.239.64/27","zone":"eu-central-1b","type":"Private"},{"name":"eu-central-1c","cidr":"172.31.239.96/27","zone":"eu-central-1c","type":"Private"},{"name":"utility-eu-central-1a","cidr":"172.31.239.0/30","zone":"eu-central-1a","type":"Utility"},{"name":"utility-eu-central-1b","cidr":"172.31.239.4/30","zone":"eu-central-1b","type":"Utility"},{"name":"utility-eu-central-1c","cidr":"172.31.239.8/30","zone":"eu-central-1c","type":"Utility"}],"masterPublicName":"api.dev-qa.k8s.domain.com","masterInternalName":"api.internal.dev-qa.k8s.domain.com","networkCIDR":"172.31.239.0/24","topology":{"masters":"private","nodes":"private","bastion":{"bastionPublicName":"bastion.dev-qa.k8s.domain.com"},"dns":{"type":"Public"}},"secretStore":"s3://my-state-store/dev-qa.k8s.domain.com/secrets","keyStore":"s3://my-state-store/dev-qa.k8s.domain.com/pki","configStore":"s3://my-state-store/dev-qa.k8s.domain.com","dnsZone":"k8s.domain.com","clusterDNSDomain":"cluster.local","serviceClusterIPRange":"100.64.0.0/13","nonMasqueradeCIDR":"100.64.0.0/10","sshAccess":["0.0.0.0/0"],"kubernetesApiAccess":["0.0.0.0/0"],"etcdClusters":[{"name":"main","provider":"Manager","etcdMembers":[{"name":"a","instanceGroup":"master-eu-central-1a","encryptedVolume":true},{"name":"b","instanceGroup":"master-eu-central-1b","encryptedVolume":true},{"name":"c","instanceGroup":"master-eu-central-1c","encryptedVolume":true}],"enableEtcdTLS":true,"enableTLSAuth":true,"version":"3.2.24","backups":{"backupStore":"s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main"},"manager":{},"memoryRequest":"100Mi","cpuRequest":"200m"},{"name":"events","provider":"Manager","etcdMembers":[{"name":"a","instanceGroup":"master-eu-central-1a","encryptedVolume":true},{"name":"b","instanceGroup":"master-eu-central-1b","encryptedVolume":true},{"name":"c","instanceGroup":"master-eu-central-1c","encryptedVolume":true}],"enableEtcdTLS":true,"enableTLSAuth":true,"version":"3.2.24","backups":{"backupStore":"s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events"},"manager":{},"memoryRequest":"100Mi","cpuRequest":"100m"}],"docker":{"ipMasq":false,"ipTables":false,"logDriver":"json-file","logLevel":"warn","logOpt":["max-size=10m","max-file=5"],"storage":"overlay2,overlay,aufs","version":"18.06.3"},"kubeDNS":{"cacheMaxSize":1000,"cacheMaxConcurrent":150,"domain":"cluster.local","replicas":2,"serverIP":"100.64.0.10","memoryRequest":"70Mi","cpuRequest":"100m","memoryLimit":"170Mi"},"kubeAPIServer":{"image":"k8s.gcr.io/kube-apiserver:v1.12.9","logLevel":2,"cloudProvider":"aws","securePort":443,"insecurePort":8080,"bindAddress":"0.0.0.0","insecureBindAddress":"127.0.0.1","enableAdmissionPlugins":["NamespaceLifecycle","LimitRanger","ServiceAccount","PersistentVolumeLabel","DefaultStorageClass","DefaultTolerationSeconds","MutatingAdmissionWebhook","ValidatingAdmissionWebhook","NodeRestriction","ResourceQuota"],"serviceClusterIPRange":"100.64.0.0/13","etcdServers":["http://127.0.0.1:4001"],"etcdServersOverrides":["/events#http://127.0.0.1:4002"],"allowPrivileged":true,"apiServerCount":3,"anonymousAuth":false,"kubeletPreferredAddressTypes":["InternalIP","Hostname","ExternalIP"],"storageBackend":"etcd3","authorizationMode":"RBAC","requestheaderUsernameHeaders":["X-Remote-User"],"requestheaderGroupHeaders":["X-Remote-Group"],"requestheaderExtraHeaderPrefixes":["X-Remote-Extra-"],"requestheaderAllowedNames":["aggregator"]},"kubeControllerManager":{"logLevel":2,"image":"k8s.gcr.io/kube-controller-manager:v1.12.9","cloudProvider":"aws","clusterName":"dev-qa.k8s.domain.com","clusterCIDR":"100.96.0.0/11","allocateNodeCIDRs":true,"configureCloudRoutes":false,"leaderElection":{"leaderElect":true},"attachDetachReconcileSyncPeriod":"1m0s","useServiceAccountCredentials":true},"kubeScheduler":{"logLevel":2,"image":"k8s.gcr.io/kube-scheduler:v1.12.9","leaderElection":{"leaderElect":true}},"kubeProxy":{"image":"k8s.gcr.io/kube-proxy:v1.12.9","cpuRequest":"100m","logLevel":2,"clusterCIDR":"100.96.0.0/11","hostnameOverride":"@aws"},"kubelet":{"anonymousAuth":false,"kubeconfigPath":"/var/lib/kubelet/kubeconfig","logLevel":2,"podManifestPath":"/etc/kubernetes/manifests","hostnameOverride":"@aws","podInfraContainerImage":"k8s.gcr.io/pause-amd64:3.0","allowPrivileged":true,"enableDebuggingHandlers":true,"clusterDomain":"cluster.local","clusterDNS":"100.64.0.10","networkPluginName":"cni","cloudProvider":"aws","cgroupRoot":"/","nonMasqueradeCIDR":"100.64.0.0/10","evictionHard":"memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%","featureGates":{"ExperimentalCriticalPodAnnotation":"true"}},"masterKubelet":{"anonymousAuth":false,"kubeconfigPath":"/var/lib/kubelet/kubeconfig","logLevel":2,"podManifestPath":"/etc/kubernetes/manifests","hostnameOverride":"@aws","podInfraContainerImage":"k8s.gcr.io/pause-amd64:3.0","allowPrivileged":true,"enableDebuggingHandlers":true,"clusterDomain":"cluster.local","clusterDNS":"100.64.0.10","networkPluginName":"cni","cloudProvider":"aws","cgroupRoot":"/","registerSchedulable":false,"nonMasqueradeCIDR":"100.64.0.0/10","evictionHard":"memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%","featureGates":{"ExperimentalCriticalPodAnnotation":"true"}},"networking":{"calico":{"majorVersion":"v3"}},"api":{"loadBalancer":{"type":"Public"}},"authorization":{"rbac":{}},"iam":{"legacy":false,"allowContainerRegistry":true}}},"HostedZoneID":"","KMSKeys":null,"Region":"eu-central-1","ResourceARN":null,"Role":"Node"},"DNSZone":{"Name":"k8s.domain.com","Lifecycle":"Sync","DNSName":"k8s.domain.com","ZoneID":"Z34N9ZS9SNSUL7","Private":null,"PrivateVPC":null}}}
I0627 16:48:58.590152 22451 executor.go:178] Executing task "Keypair/kops": *fitasks.Keypair {"Name":"kops","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"o=system:masters,cn=kops","type":"client","format":"v1alpha2"}
I0627 16:48:58.598117 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops/keyset.yaml"
I0627 16:48:58.590154 22451 executor.go:178] Executing task "Keypair/apiserver-aggregator": *fitasks.Keypair {"Name":"apiserver-aggregator","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"apiserver-aggregator-ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=apiserver-aggregator-ca","type":"ca","format":"v1alpha2"},"subject":"cn=aggregator","type":"client","format":"v1alpha2"}
I0627 16:48:58.590048 22451 executor.go:178] Executing task "Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:48:58.590122 22451 executor.go:178] Executing task "RouteTable/dev-qa.k8s.domain.com": *awstasks.RouteTable {"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"public"}}
I0627 16:48:58.590184 22451 executor.go:178] Executing task "Keypair/kube-proxy": *fitasks.Keypair {"Name":"kube-proxy","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"cn=system:kube-proxy","type":"client","format":"v1alpha2"}
I0627 16:48:58.590170 22451 executor.go:178] Executing task "Subnet/eu-central-1b.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"eu-central-1b.dev-qa.k8s.domain.com","ShortName":"eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.64/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}
I0627 16:48:58.590209 22451 executor.go:178] Executing task "Keypair/apiserver-proxy-client": *fitasks.Keypair {"Name":"apiserver-proxy-client","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"cn=apiserver-proxy-client","type":"client","format":"v1alpha2"}
I0627 16:48:58.590111 22451 executor.go:178] Executing task "Subnet/eu-central-1a.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"eu-central-1a.dev-qa.k8s.domain.com","ShortName":"eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.32/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}
I0627 16:48:58.590128 22451 executor.go:178] Executing task "RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com": *awstasks.RouteTable {"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1a"}}
I0627 16:48:58.590224 22451 executor.go:178] Executing task "Keypair/kubelet": *fitasks.Keypair {"Name":"kubelet","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"o=system:nodes,cn=kubelet","type":"client","format":"v1alpha2"}
I0627 16:48:58.590229 22451 executor.go:178] Executing task "Keypair/kube-scheduler": *fitasks.Keypair {"Name":"kube-scheduler","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"cn=system:kube-scheduler","type":"client","format":"v1alpha2"}
I0627 16:48:58.590239 22451 executor.go:178] Executing task "Keypair/kube-controller-manager": *fitasks.Keypair {"Name":"kube-controller-manager","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"cn=system:kube-controller-manager","type":"client","format":"v1alpha2"}
I0627 16:48:58.590242 22451 executor.go:178] Executing task "RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com": *awstasks.RouteTable {"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1c"}}
I0627 16:48:58.590102 22451 executor.go:178] Executing task "SecurityGroup/bastion-elb.dev-qa.k8s.domain.com": *awstasks.SecurityGroup {"Name":"bastion-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:48:58.590243 22451 executor.go:178] Executing task "IAMInstanceProfileRole/bastions.dev-qa.k8s.domain.com": *awstasks.IAMInstanceProfileRole {"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","InstanceProfile":{"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"Role":{"ID":null,"Lifecycle":"Sync","Name":"bastions.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"bastions"}}
I0627 16:48:58.590108 22451 executor.go:178] Executing task "SecurityGroup/bastion.dev-qa.k8s.domain.com": *awstasks.SecurityGroup {"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:48:58.590267 22451 executor.go:178] Executing task "SecurityGroup/masters.dev-qa.k8s.domain.com": *awstasks.SecurityGroup {"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:48:58.590088 22451 executor.go:178] Executing task "Subnet/eu-central-1c.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"eu-central-1c.dev-qa.k8s.domain.com","ShortName":"eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.96/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}
I0627 16:48:58.590282 22451 executor.go:178] Executing task "IAMInstanceProfileRole/nodes.dev-qa.k8s.domain.com": *awstasks.IAMInstanceProfileRole {"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","InstanceProfile":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"Role":{"ID":null,"Lifecycle":"Sync","Name":"nodes.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"nodes"}}
I0627 16:48:58.590140 22451 executor.go:178] Executing task "SecurityGroup/api-elb.dev-qa.k8s.domain.com": *awstasks.SecurityGroup {"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:48:58.590102 22451 executor.go:178] Executing task "InternetGateway/dev-qa.k8s.domain.com": *awstasks.InternetGateway {"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:48:58.590255 22451 executor.go:178] Executing task "IAMInstanceProfileRole/masters.dev-qa.k8s.domain.com": *awstasks.IAMInstanceProfileRole {"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","InstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"Role":{"ID":null,"Lifecycle":"Sync","Name":"masters.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"masters"}}
I0627 16:48:58.590333 22451 executor.go:178] Executing task "Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:48:58.590323 22451 executor.go:178] Executing task "RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com": *awstasks.RouteTable {"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1b"}}
I0627 16:48:58.590157 22451 executor.go:178] Executing task "SecurityGroup/nodes.dev-qa.k8s.domain.com": *awstasks.SecurityGroup {"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:48:58.590218 22451 executor.go:178] Executing task "MirrorKeystore/mirror-keystore": *fitasks.MirrorKeystore {"Name":"mirror-keystore","Lifecycle":null,"MirrorPath":{}}
I0627 16:48:58.590365 22451 executor.go:178] Executing task "Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:48:58.590278 22451 executor.go:178] Executing task "MirrorSecrets/mirror-secrets": *fitasks.MirrorSecrets {"Name":"mirror-secrets","Lifecycle":null,"MirrorPath":{}}
I0627 16:48:58.590088 22451 executor.go:178] Executing task "IAMRolePolicy/additional.bastions.dev-qa.k8s.domain.com": *awstasks.IAMRolePolicy {"ID":null,"Lifecycle":"Sync","Name":"additional.bastions.dev-qa.k8s.domain.com","Role":{"ID":null,"Lifecycle":"Sync","Name":"bastions.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"bastions"},"PolicyDocument":{"Name":"","Resource":""}}
I0627 16:48:58.590091 22451 executor.go:178] Executing task "IAMRolePolicy/additional.masters.dev-qa.k8s.domain.com": *awstasks.IAMRolePolicy {"ID":null,"Lifecycle":"Sync","Name":"additional.masters.dev-qa.k8s.domain.com","Role":{"ID":null,"Lifecycle":"Sync","Name":"masters.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"masters"},"PolicyDocument":{"Name":"","Resource":""}}
I0627 16:48:58.590126 22451 executor.go:178] Executing task "IAMRolePolicy/additional.nodes.dev-qa.k8s.domain.com": *awstasks.IAMRolePolicy {"ID":null,"Lifecycle":"Sync","Name":"additional.nodes.dev-qa.k8s.domain.com","Role":{"ID":null,"Lifecycle":"Sync","Name":"nodes.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"nodes"},"PolicyDocument":{"Name":"","Resource":""}}
I0627 16:48:58.590204 22451 executor.go:178] Executing task "VPCDHCPOptionsAssociation/dev-qa.k8s.domain.com": *awstasks.VPCDHCPOptionsAssociation {"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"DHCPOptions":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"DomainName":"eu-central-1.compute.internal","DomainNameServers":"AmazonProvidedDNS","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}}
I0627 16:48:58.589992 22451 executor.go:178] Executing task "IAMRolePolicy/masters.dev-qa.k8s.domain.com": *awstasks.IAMRolePolicy {"ID":null,"Lifecycle":"Sync","Name":"masters.dev-qa.k8s.domain.com","Role":{"ID":null,"Lifecycle":"Sync","Name":"masters.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"masters"},"PolicyDocument":{"Builder":{"Cluster":{"metadata":{"name":"dev-qa.k8s.domain.com","creationTimestamp":"2019-06-27T14:48:45Z"},"spec":{"channel":"stable","configBase":"s3://my-state-store/dev-qa.k8s.domain.com","cloudProvider":"aws","kubernetesVersion":"1.12.9","subnets":[{"name":"eu-central-1a","cidr":"172.31.239.32/27","zone":"eu-central-1a","type":"Private"},{"name":"eu-central-1b","cidr":"172.31.239.64/27","zone":"eu-central-1b","type":"Private"},{"name":"eu-central-1c","cidr":"172.31.239.96/27","zone":"eu-central-1c","type":"Private"},{"name":"utility-eu-central-1a","cidr":"172.31.239.0/30","zone":"eu-central-1a","type":"Utility"},{"name":"utility-eu-central-1b","cidr":"172.31.239.4/30","zone":"eu-central-1b","type":"Utility"},{"name":"utility-eu-central-1c","cidr":"172.31.239.8/30","zone":"eu-central-1c","type":"Utility"}],"masterPublicName":"api.dev-qa.k8s.domain.com","masterInternalName":"api.internal.dev-qa.k8s.domain.com","networkCIDR":"172.31.239.0/24","topology":{"masters":"private","nodes":"private","bastion":{"bastionPublicName":"bastion.dev-qa.k8s.domain.com"},"dns":{"type":"Public"}},"secretStore":"s3://my-state-store/dev-qa.k8s.domain.com/secrets","keyStore":"s3://my-state-store/dev-qa.k8s.domain.com/pki","configStore":"s3://my-state-store/dev-qa.k8s.domain.com","dnsZone":"k8s.domain.com","clusterDNSDomain":"cluster.local","serviceClusterIPRange":"100.64.0.0/13","nonMasqueradeCIDR":"100.64.0.0/10","sshAccess":["0.0.0.0/0"],"kubernetesApiAccess":["0.0.0.0/0"],"etcdClusters":[{"name":"main","provider":"Manager","etcdMembers":[{"name":"a","instanceGroup":"master-eu-central-1a","encryptedVolume":true},{"name":"b","instanceGroup":"master-eu-central-1b","encryptedVolume":true},{"name":"c","instanceGroup":"master-eu-central-1c","encryptedVolume":true}],"enableEtcdTLS":true,"enableTLSAuth":true,"version":"3.2.24","backups":{"backupStore":"s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main"},"manager":{},"memoryRequest":"100Mi","cpuRequest":"200m"},{"name":"events","provider":"Manager","etcdMembers":[{"name":"a","instanceGroup":"master-eu-central-1a","encryptedVolume":true},{"name":"b","instanceGroup":"master-eu-central-1b","encryptedVolume":true},{"name":"c","instanceGroup":"master-eu-central-1c","encryptedVolume":true}],"enableEtcdTLS":true,"enableTLSAuth":true,"version":"3.2.24","backups":{"backupStore":"s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events"},"manager":{},"memoryRequest":"100Mi","cpuRequest":"100m"}],"docker":{"ipMasq":false,"ipTables":false,"logDriver":"json-file","logLevel":"warn","logOpt":["max-size=10m","max-file=5"],"storage":"overlay2,overlay,aufs","version":"18.06.3"},"kubeDNS":{"cacheMaxSize":1000,"cacheMaxConcurrent":150,"domain":"cluster.local","replicas":2,"serverIP":"100.64.0.10","memoryRequest":"70Mi","cpuRequest":"100m","memoryLimit":"170Mi"},"kubeAPIServer":{"image":"k8s.gcr.io/kube-apiserver:v1.12.9","logLevel":2,"cloudProvider":"aws","securePort":443,"insecurePort":8080,"bindAddress":"0.0.0.0","insecureBindAddress":"127.0.0.1","enableAdmissionPlugins":["NamespaceLifecycle","LimitRanger","ServiceAccount","PersistentVolumeLabel","DefaultStorageClass","DefaultTolerationSeconds","MutatingAdmissionWebhook","ValidatingAdmissionWebhook","NodeRestriction","ResourceQuota"],"serviceClusterIPRange":"100.64.0.0/13","etcdServers":["http://127.0.0.1:4001"],"etcdServersOverrides":["/events#http://127.0.0.1:4002"],"allowPrivileged":true,"apiServerCount":3,"anonymousAuth":false,"kubeletPreferredAddressTypes":["InternalIP","Hostname","ExternalIP"],"storageBackend":"etcd3","authorizationMode":"RBAC","requestheaderUsernameHeaders":["X-Remote-User"],"requestheaderGroupHeaders":["X-Remote-Group"],"requestheaderExtraHeaderPrefixes":["X-Remote-Extra-"],"requestheaderAllowedNames":["aggregator"]},"kubeControllerManager":{"logLevel":2,"image":"k8s.gcr.io/kube-controller-manager:v1.12.9","cloudProvider":"aws","clusterName":"dev-qa.k8s.domain.com","clusterCIDR":"100.96.0.0/11","allocateNodeCIDRs":true,"configureCloudRoutes":false,"leaderElection":{"leaderElect":true},"attachDetachReconcileSyncPeriod":"1m0s","useServiceAccountCredentials":true},"kubeScheduler":{"logLevel":2,"image":"k8s.gcr.io/kube-scheduler:v1.12.9","leaderElection":{"leaderElect":true}},"kubeProxy":{"image":"k8s.gcr.io/kube-proxy:v1.12.9","cpuRequest":"100m","logLevel":2,"clusterCIDR":"100.96.0.0/11","hostnameOverride":"@aws"},"kubelet":{"anonymousAuth":false,"kubeconfigPath":"/var/lib/kubelet/kubeconfig","logLevel":2,"podManifestPath":"/etc/kubernetes/manifests","hostnameOverride":"@aws","podInfraContainerImage":"k8s.gcr.io/pause-amd64:3.0","allowPrivileged":true,"enableDebuggingHandlers":true,"clusterDomain":"cluster.local","clusterDNS":"100.64.0.10","networkPluginName":"cni","cloudProvider":"aws","cgroupRoot":"/","nonMasqueradeCIDR":"100.64.0.0/10","evictionHard":"memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%","featureGates":{"ExperimentalCriticalPodAnnotation":"true"}},"masterKubelet":{"anonymousAuth":false,"kubeconfigPath":"/var/lib/kubelet/kubeconfig","logLevel":2,"podManifestPath":"/etc/kubernetes/manifests","hostnameOverride":"@aws","podInfraContainerImage":"k8s.gcr.io/pause-amd64:3.0","allowPrivileged":true,"enableDebuggingHandlers":true,"clusterDomain":"cluster.local","clusterDNS":"100.64.0.10","networkPluginName":"cni","cloudProvider":"aws","cgroupRoot":"/","registerSchedulable":false,"nonMasqueradeCIDR":"100.64.0.0/10","evictionHard":"memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%","featureGates":{"ExperimentalCriticalPodAnnotation":"true"}},"networking":{"calico":{"majorVersion":"v3"}},"api":{"loadBalancer":{"type":"Public"}},"authorization":{"rbac":{}},"iam":{"legacy":false,"allowContainerRegistry":true}}},"HostedZoneID":"","KMSKeys":null,"Region":"eu-central-1","ResourceARN":null,"Role":"Master"},"DNSZone":{"Name":"k8s.domain.com","Lifecycle":"Sync","DNSName":"k8s.domain.com","ZoneID":"Z34N9ZS9SNSUL7","Private":null,"PrivateVPC":null}}}
I0627 16:48:58.590070 22451 executor.go:178] Executing task "IAMRolePolicy/bastions.dev-qa.k8s.domain.com": *awstasks.IAMRolePolicy {"ID":null,"Lifecycle":"Sync","Name":"bastions.dev-qa.k8s.domain.com","Role":{"ID":null,"Lifecycle":"Sync","Name":"bastions.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"bastions"},"PolicyDocument":{"Builder":{"Cluster":{"metadata":{"name":"dev-qa.k8s.domain.com","creationTimestamp":"2019-06-27T14:48:45Z"},"spec":{"channel":"stable","configBase":"s3://my-state-store/dev-qa.k8s.domain.com","cloudProvider":"aws","kubernetesVersion":"1.12.9","subnets":[{"name":"eu-central-1a","cidr":"172.31.239.32/27","zone":"eu-central-1a","type":"Private"},{"name":"eu-central-1b","cidr":"172.31.239.64/27","zone":"eu-central-1b","type":"Private"},{"name":"eu-central-1c","cidr":"172.31.239.96/27","zone":"eu-central-1c","type":"Private"},{"name":"utility-eu-central-1a","cidr":"172.31.239.0/30","zone":"eu-central-1a","type":"Utility"},{"name":"utility-eu-central-1b","cidr":"172.31.239.4/30","zone":"eu-central-1b","type":"Utility"},{"name":"utility-eu-central-1c","cidr":"172.31.239.8/30","zone":"eu-central-1c","type":"Utility"}],"masterPublicName":"api.dev-qa.k8s.domain.com","masterInternalName":"api.internal.dev-qa.k8s.domain.com","networkCIDR":"172.31.239.0/24","topology":{"masters":"private","nodes":"private","bastion":{"bastionPublicName":"bastion.dev-qa.k8s.domain.com"},"dns":{"type":"Public"}},"secretStore":"s3://my-state-store/dev-qa.k8s.domain.com/secrets","keyStore":"s3://my-state-store/dev-qa.k8s.domain.com/pki","configStore":"s3://my-state-store/dev-qa.k8s.domain.com","dnsZone":"k8s.domain.com","clusterDNSDomain":"cluster.local","serviceClusterIPRange":"100.64.0.0/13","nonMasqueradeCIDR":"100.64.0.0/10","sshAccess":["0.0.0.0/0"],"kubernetesApiAccess":["0.0.0.0/0"],"etcdClusters":[{"name":"main","provider":"Manager","etcdMembers":[{"name":"a","instanceGroup":"master-eu-central-1a","encryptedVolume":true},{"name":"b","instanceGroup":"master-eu-central-1b","encryptedVolume":true},{"name":"c","instanceGroup":"master-eu-central-1c","encryptedVolume":true}],"enableEtcdTLS":true,"enableTLSAuth":true,"version":"3.2.24","backups":{"backupStore":"s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main"},"manager":{},"memoryRequest":"100Mi","cpuRequest":"200m"},{"name":"events","provider":"Manager","etcdMembers":[{"name":"a","instanceGroup":"master-eu-central-1a","encryptedVolume":true},{"name":"b","instanceGroup":"master-eu-central-1b","encryptedVolume":true},{"name":"c","instanceGroup":"master-eu-central-1c","encryptedVolume":true}],"enableEtcdTLS":true,"enableTLSAuth":true,"version":"3.2.24","backups":{"backupStore":"s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events"},"manager":{},"memoryRequest":"100Mi","cpuRequest":"100m"}],"docker":{"ipMasq":false,"ipTables":false,"logDriver":"json-file","logLevel":"warn","logOpt":["max-size=10m","max-file=5"],"storage":"overlay2,overlay,aufs","version":"18.06.3"},"kubeDNS":{"cacheMaxSize":1000,"cacheMaxConcurrent":150,"domain":"cluster.local","replicas":2,"serverIP":"100.64.0.10","memoryRequest":"70Mi","cpuRequest":"100m","memoryLimit":"170Mi"},"kubeAPIServer":{"image":"k8s.gcr.io/kube-apiserver:v1.12.9","logLevel":2,"cloudProvider":"aws","securePort":443,"insecurePort":8080,"bindAddress":"0.0.0.0","insecureBindAddress":"127.0.0.1","enableAdmissionPlugins":["NamespaceLifecycle","LimitRanger","ServiceAccount","PersistentVolumeLabel","DefaultStorageClass","DefaultTolerationSeconds","MutatingAdmissionWebhook","ValidatingAdmissionWebhook","NodeRestriction","ResourceQuota"],"serviceClusterIPRange":"100.64.0.0/13","etcdServers":["http://127.0.0.1:4001"],"etcdServersOverrides":["/events#http://127.0.0.1:4002"],"allowPrivileged":true,"apiServerCount":3,"anonymousAuth":false,"kubeletPreferredAddressTypes":["InternalIP","Hostname","ExternalIP"],"storageBackend":"etcd3","authorizationMode":"RBAC","requestheaderUsernameHeaders":["X-Remote-User"],"requestheaderGroupHeaders":["X-Remote-Group"],"requestheaderExtraHeaderPrefixes":["X-Remote-Extra-"],"requestheaderAllowedNames":["aggregator"]},"kubeControllerManager":{"logLevel":2,"image":"k8s.gcr.io/kube-controller-manager:v1.12.9","cloudProvider":"aws","clusterName":"dev-qa.k8s.domain.com","clusterCIDR":"100.96.0.0/11","allocateNodeCIDRs":true,"configureCloudRoutes":false,"leaderElection":{"leaderElect":true},"attachDetachReconcileSyncPeriod":"1m0s","useServiceAccountCredentials":true},"kubeScheduler":{"logLevel":2,"image":"k8s.gcr.io/kube-scheduler:v1.12.9","leaderElection":{"leaderElect":true}},"kubeProxy":{"image":"k8s.gcr.io/kube-proxy:v1.12.9","cpuRequest":"100m","logLevel":2,"clusterCIDR":"100.96.0.0/11","hostnameOverride":"@aws"},"kubelet":{"anonymousAuth":false,"kubeconfigPath":"/var/lib/kubelet/kubeconfig","logLevel":2,"podManifestPath":"/etc/kubernetes/manifests","hostnameOverride":"@aws","podInfraContainerImage":"k8s.gcr.io/pause-amd64:3.0","allowPrivileged":true,"enableDebuggingHandlers":true,"clusterDomain":"cluster.local","clusterDNS":"100.64.0.10","networkPluginName":"cni","cloudProvider":"aws","cgroupRoot":"/","nonMasqueradeCIDR":"100.64.0.0/10","evictionHard":"memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%","featureGates":{"ExperimentalCriticalPodAnnotation":"true"}},"masterKubelet":{"anonymousAuth":false,"kubeconfigPath":"/var/lib/kubelet/kubeconfig","logLevel":2,"podManifestPath":"/etc/kubernetes/manifests","hostnameOverride":"@aws","podInfraContainerImage":"k8s.gcr.io/pause-amd64:3.0","allowPrivileged":true,"enableDebuggingHandlers":true,"clusterDomain":"cluster.local","clusterDNS":"100.64.0.10","networkPluginName":"cni","cloudProvider":"aws","cgroupRoot":"/","registerSchedulable":false,"nonMasqueradeCIDR":"100.64.0.0/10","evictionHard":"memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%","featureGates":{"ExperimentalCriticalPodAnnotation":"true"}},"networking":{"calico":{"majorVersion":"v3"}},"api":{"loadBalancer":{"type":"Public"}},"authorization":{"rbac":{}},"iam":{"legacy":false,"allowContainerRegistry":true}}},"HostedZoneID":"","KMSKeys":null,"Region":"eu-central-1","ResourceARN":null,"Role":"Bastion"},"DNSZone":{"Name":"k8s.domain.com","Lifecycle":"Sync","DNSName":"k8s.domain.com","ZoneID":"Z34N9ZS9SNSUL7","Private":null,"PrivateVPC":null}}}
I0627 16:48:58.590160 22451 executor.go:178] Executing task "Keypair/kubelet-api": *fitasks.Keypair {"Name":"kubelet-api","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"cn=kubelet-api","type":"client","format":"v1alpha2"}
I0627 16:48:58.597642 22451 request_logger.go:45] AWS request: iam/GetRolePolicy
I0627 16:48:58.605057 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/keyset.yaml"
I0627 16:48:58.607205 22451 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:48:58.608176 22451 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:48:58.608688 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy/keyset.yaml"
I0627 16:48:58.609476 22451 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:48:58.610093 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/keyset.yaml"
I0627 16:48:58.610983 22451 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:48:58.611739 22451 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:48:58.612354 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet/keyset.yaml"
I0627 16:48:58.615209 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler/keyset.yaml"
I0627 16:48:58.615764 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/keyset.yaml"
I0627 16:48:58.616601 22451 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:48:58.620417 22451 iaminstanceprofilerole.go:45] Role/RoleID not set
I0627 16:48:58.623064 22451 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:48:58.623539 22451 iaminstanceprofilerole.go:45] Role/RoleID not set
I0627 16:48:58.627451 22451 request_logger.go:45] AWS request: ec2/DescribeInternetGateways
I0627 16:48:58.628115 22451 iaminstanceprofilerole.go:45] Role/RoleID not set
I0627 16:48:58.629041 22451 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:48:58.629771 22451 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:48:58.632605 22451 request_logger.go:45] AWS request: iam/GetRolePolicy
I0627 16:48:58.632730 22451 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:48:58.633195 22451 request_logger.go:45] AWS request: iam/GetRolePolicy
I0627 16:48:58.634101 22451 request_logger.go:45] AWS request: iam/GetRolePolicy
I0627 16:48:58.642275 22451 request_logger.go:45] AWS request: iam/GetRolePolicy
I0627 16:48:58.646627 22451 request_logger.go:45] AWS request: iam/GetRolePolicy
I0627 16:48:58.654499 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api/keyset.yaml"
I0627 16:48:58.771894 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master/keyset.yaml", falling back to directory-list method
I0627 16:48:58.772682 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/master/"
I0627 16:48:58.780934 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg/keyset.yaml", falling back to directory-list method
I0627 16:48:58.781508 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kubecfg/"
I0627 16:48:58.897861 22451 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:48:58.907471 22451 iam_builder.go:300] Ignoring location "s3://my-state-store/dev-qa.k8s.domain.com/pki/" because found parent "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:48:58.908353 22451 iam_builder.go:300] Ignoring location "s3://my-state-store/dev-qa.k8s.domain.com/secrets/" because found parent "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:48:58.908691 22451 iam_builder.go:305] Found root location "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:48:58.951119 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops/keyset.yaml", falling back to directory-list method
I0627 16:48:58.951856 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kops/"
I0627 16:48:58.972102 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/keyset.yaml", falling back to directory-list method
I0627 16:48:58.972854 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/"
I0627 16:48:59.135772 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy/keyset.yaml", falling back to directory-list method
I0627 16:48:59.136488 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kube-proxy/"
I0627 16:48:59.154417 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/keyset.yaml", falling back to directory-list method
I0627 16:48:59.155000 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/"
I0627 16:48:59.315810 22451 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:48:59.328457 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet/keyset.yaml", falling back to directory-list method
I0627 16:48:59.328951 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kubelet/"
I0627 16:48:59.337362 22451 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:48:59.338863 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler/keyset.yaml", falling back to directory-list method
I0627 16:48:59.339285 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kube-scheduler/"
I0627 16:48:59.420186 22451 iam_builder.go:300] Ignoring location "s3://my-state-store/dev-qa.k8s.domain.com/pki/" because found parent "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:48:59.420778 22451 iam_builder.go:300] Ignoring location "s3://my-state-store/dev-qa.k8s.domain.com/secrets/" because found parent "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:48:59.421095 22451 iam_builder.go:305] Found root location "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:48:59.511589 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/keyset.yaml", falling back to directory-list method
I0627 16:48:59.512267 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/"
I0627 16:48:59.537918 22451 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api/keyset.yaml", falling back to directory-list method
I0627 16:48:59.538587 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kubelet-api/"
I0627 16:48:59.681745 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master: []
I0627 16:48:59.682365 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/master/keyset.yaml"
I0627 16:48:59.684058 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg: []
I0627 16:48:59.684451 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubecfg/keyset.yaml"
I0627 16:48:59.684058 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops: []
I0627 16:48:59.685336 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kops/keyset.yaml"
I0627 16:48:59.685428 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator: []
I0627 16:48:59.686328 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/keyset.yaml"
I0627 16:48:59.684296 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy: []
I0627 16:48:59.687291 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy/keyset.yaml"
I0627 16:48:59.735666 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client: []
I0627 16:48:59.736352 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/keyset.yaml"
I0627 16:48:59.735784 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager: []
I0627 16:48:59.737350 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-controller-manager/keyset.yaml"
I0627 16:48:59.735784 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet: []
I0627 16:48:59.738340 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet/keyset.yaml"
I0627 16:48:59.739951 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api: []
I0627 16:48:59.740288 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet-api/keyset.yaml"
I0627 16:48:59.748839 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler: []
I0627 16:48:59.749366 22451 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-scheduler/keyset.yaml"
I0627 16:48:59.771502 22451 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:48:59.863460 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/master/keyset.yaml", falling back to directory-list method
I0627 16:48:59.864137 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/master/"
I0627 16:48:59.867035 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kops/keyset.yaml", falling back to directory-list method
I0627 16:48:59.867676 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kops/"
I0627 16:48:59.867077 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubecfg/keyset.yaml", falling back to directory-list method
I0627 16:48:59.869074 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kubecfg/"
I0627 16:48:59.867126 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/keyset.yaml", falling back to directory-list method
I0627 16:48:59.869993 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/"
I0627 16:48:59.915564 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/keyset.yaml", falling back to directory-list method
I0627 16:48:59.916364 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/"
I0627 16:48:59.919322 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy/keyset.yaml", falling back to directory-list method
I0627 16:48:59.919839 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kube-proxy/"
I0627 16:48:59.919393 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet/keyset.yaml", falling back to directory-list method
I0627 16:48:59.920804 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kubelet/"
I0627 16:48:59.919431 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-controller-manager/keyset.yaml", falling back to directory-list method
I0627 16:48:59.922440 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kube-controller-manager/"
I0627 16:48:59.926929 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet-api/keyset.yaml", falling back to directory-list method
I0627 16:48:59.927364 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kubelet-api/"
I0627 16:49:00.042269 22451 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-scheduler/keyset.yaml", falling back to directory-list method
I0627 16:49:00.042930 22451 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kube-scheduler/"
I0627 16:49:00.053057 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/master: []
I0627 16:49:00.053146 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kops: []
I0627 16:49:00.053145 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubecfg: []
I0627 16:49:00.093389 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator: []
I0627 16:49:00.099709 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client: []
I0627 16:49:00.104814 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet: []
I0627 16:49:00.104829 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy: []
I0627 16:49:00.118044 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet-api: []
I0627 16:49:00.124626 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-controller-manager: []
I0627 16:49:00.237430 22451 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-scheduler: []
I0627 16:49:00.238521 22451 executor.go:103] Tasks: 89 done / 143 total; 37 can run
I0627 16:49:00.239065 22451 executor.go:178] Executing task "SecurityGroupRule/https-api-elb-0.0.0.0/0": *awstasks.SecurityGroupRule {"Name":"https-api-elb-0.0.0.0/0","Lifecycle":"Sync","SecurityGroup":{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":"tcp","FromPort":443,"ToPort":443,"SourceGroup":null,"Egress":null}
I0627 16:49:00.239077 22451 executor.go:178] Executing task "SecurityGroupRule/ssh-external-to-bastion-elb-0.0.0.0/0": *awstasks.SecurityGroupRule {"Name":"ssh-external-to-bastion-elb-0.0.0.0/0","Lifecycle":"Sync","SecurityGroup":{"Name":"bastion-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":"tcp","FromPort":22,"ToPort":22,"SourceGroup":null,"Egress":null}
I0627 16:49:00.239044 22451 executor.go:178] Executing task "ElasticIP/eu-central-1b.dev-qa.k8s.domain.com": *awstasks.ElasticIP {"Name":"eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"PublicIP":null,"TagOnSubnet":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedNatGatewayRouteTable":{"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1b"}}}
I0627 16:49:00.239108 22451 executor.go:178] Executing task "SecurityGroupRule/node-egress": *awstasks.SecurityGroupRule {"Name":"node-egress","Lifecycle":"Sync","SecurityGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":null,"Egress":true}
I0627 16:49:00.238987 22451 executor.go:178] Executing task "SecurityGroupRule/icmp-pmtu-api-elb-0.0.0.0/0": *awstasks.SecurityGroupRule {"Name":"icmp-pmtu-api-elb-0.0.0.0/0","Lifecycle":"Sync","SecurityGroup":{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":"icmp","FromPort":3,"ToPort":4,"SourceGroup":null,"Egress":null}
I0627 16:49:00.239159 22451 executor.go:178] Executing task "SecurityGroupRule/node-to-master-tcp-1-2379": *awstasks.SecurityGroupRule {"Name":"node-to-master-tcp-1-2379","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":1,"ToPort":2379,"SourceGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:49:00.239186 22451 executor.go:178] Executing task "SecurityGroupRule/all-node-to-node": *awstasks.SecurityGroupRule {"Name":"all-node-to-node","Lifecycle":"Sync","SecurityGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:49:00.239023 22451 executor.go:178] Executing task "LaunchConfiguration/bastions.dev-qa.k8s.domain.com": *awstasks.LaunchConfiguration {"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":true,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t2.micro","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":32,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":null}
I0627 16:49:00.255105 22451 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:49:00.239240 22451 executor.go:178] Executing task "ElasticIP/eu-central-1c.dev-qa.k8s.domain.com": *awstasks.ElasticIP {"Name":"eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"PublicIP":null,"TagOnSubnet":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedNatGatewayRouteTable":{"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1c"}}}
I0627 16:49:00.239284 22451 executor.go:178] Executing task "SecurityGroupRule/node-to-master-tcp-4003-65535": *awstasks.SecurityGroupRule {"Name":"node-to-master-tcp-4003-65535","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":4003,"ToPort":65535,"SourceGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:49:00.239082 22451 executor.go:178] Executing task "RouteTableAssociation/private-eu-central-1c.dev-qa.k8s.domain.com": *awstasks.RouteTableAssociation {"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"RouteTable":{"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1c"}},"Subnet":{"Name":"eu-central-1c.dev-qa.k8s.domain.com","ShortName":"eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.96/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}}
I0627 16:49:00.239445 22451 executor.go:178] Executing task "SecurityGroupRule/node-to-master-udp-1-65535": *awstasks.SecurityGroupRule {"Name":"node-to-master-udp-1-65535","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"udp","FromPort":1,"ToPort":65535,"SourceGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:49:00.239472 22451 executor.go:178] Executing task "LaunchConfiguration/master-eu-central-1a.masters.dev-qa.k8s.domain.com": *awstasks.LaunchConfiguration {"Name":"master-eu-central-1a.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}}
I0627 16:49:00.239255 22451 executor.go:178] Executing task "RouteTableAssociation/private-eu-central-1a.dev-qa.k8s.domain.com": *awstasks.RouteTableAssociation {"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"RouteTable":{"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1a"}},"Subnet":{"Name":"eu-central-1a.dev-qa.k8s.domain.com","ShortName":"eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.32/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}}
I0627 16:49:00.239604 22451 executor.go:178] Executing task "RouteTableAssociation/utility-eu-central-1a.dev-qa.k8s.domain.com": *awstasks.RouteTableAssociation {"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"RouteTable":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"public"}},"Subnet":{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}}
I0627 16:49:00.239624 22451 executor.go:178] Executing task "SecurityGroupRule/https-elb-to-master": *awstasks.SecurityGroupRule {"Name":"https-elb-to-master","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":443,"ToPort":443,"SourceGroup":{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:49:00.239641 22451 executor.go:178] Executing task "LaunchConfiguration/nodes.dev-qa.k8s.domain.com": *awstasks.LaunchConfiguration {"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"m5a.2xlarge","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":300,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}}
I0627 16:49:00.239271 22451 executor.go:178] Executing task "LaunchConfiguration/master-eu-central-1b.masters.dev-qa.k8s.domain.com": *awstasks.LaunchConfiguration {"Name":"master-eu-central-1b.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}}
I0627 16:49:00.239708 22451 executor.go:178] Executing task "SecurityGroupRule/bastion-elb-egress": *awstasks.SecurityGroupRule {"Name":"bastion-elb-egress","Lifecycle":"Sync","SecurityGroup":{"Name":"bastion-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":null,"Egress":true}
I0627 16:49:00.239016 22451 executor.go:178] Executing task "SecurityGroupRule/node-to-master-protocol-ipip": *awstasks.SecurityGroupRule {"Name":"node-to-master-protocol-ipip","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"4","FromPort":null,"ToPort":null,"SourceGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:49:00.239160 22451 executor.go:178] Executing task "RouteTableAssociation/utility-eu-central-1b.dev-qa.k8s.domain.com": *awstasks.RouteTableAssociation {"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"RouteTable":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"public"}},"Subnet":{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}}
I0627 16:49:00.239734 22451 executor.go:178] Executing task "SecurityGroupRule/bastion-egress": *awstasks.SecurityGroupRule {"Name":"bastion-egress","Lifecycle":"Sync","SecurityGroup":{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":null,"Egress":true}
I0627 16:49:00.239175 22451 executor.go:178] Executing task "RouteTableAssociation/utility-eu-central-1c.dev-qa.k8s.domain.com": *awstasks.RouteTableAssociation {"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"RouteTable":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"public"}},"Subnet":{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}}
I0627 16:49:00.239792 22451 executor.go:178] Executing task "LaunchConfiguration/master-eu-central-1c.masters.dev-qa.k8s.domain.com": *awstasks.LaunchConfiguration {"Name":"master-eu-central-1c.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}}
I0627 16:49:00.239817 22451 executor.go:178] Executing task "SecurityGroupRule/bastion-to-node-ssh": *awstasks.SecurityGroupRule {"Name":"bastion-to-node-ssh","Lifecycle":"Sync","SecurityGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":22,"ToPort":22,"SourceGroup":{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:49:00.239865 22451 executor.go:178] Executing task "SecurityGroupRule/ssh-elb-to-bastion": *awstasks.SecurityGroupRule {"Name":"ssh-elb-to-bastion","Lifecycle":"Sync","SecurityGroup":{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":22,"ToPort":22,"SourceGroup":{"Name":"bastion-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:49:00.239896 22451 executor.go:178] Executing task "ElasticIP/eu-central-1a.dev-qa.k8s.domain.com": *awstasks.ElasticIP {"Name":"eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"PublicIP":null,"TagOnSubnet":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedNatGatewayRouteTable":{"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1a"}}}
I0627 16:49:00.239902 22451 executor.go:178] Executing task "SecurityGroupRule/bastion-to-master-ssh": *awstasks.SecurityGroupRule {"Name":"bastion-to-master-ssh","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":22,"ToPort":22,"SourceGroup":{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:49:00.239770 22451 executor.go:178] Executing task "SecurityGroupRule/node-to-master-tcp-2382-4001": *awstasks.SecurityGroupRule {"Name":"node-to-master-tcp-2382-4001","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":2382,"ToPort":4001,"SourceGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:49:00.239049 22451 executor.go:178] Executing task "SecurityGroupRule/master-egress": *awstasks.SecurityGroupRule {"Name":"master-egress","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":null,"Egress":true}
I0627 16:49:00.239000 22451 executor.go:178] Executing task "SecurityGroupRule/api-elb-egress": *awstasks.SecurityGroupRule {"Name":"api-elb-egress","Lifecycle":"Sync","SecurityGroup":{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":null,"Egress":true}
I0627 16:49:00.239413 22451 executor.go:178] Executing task "RouteTableAssociation/private-eu-central-1b.dev-qa.k8s.domain.com": *awstasks.RouteTableAssociation {"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"RouteTable":{"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1b"}},"Subnet":{"Name":"eu-central-1b.dev-qa.k8s.domain.com","ShortName":"eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.64/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}}
I0627 16:49:00.239917 22451 executor.go:178] Executing task "SecurityGroupRule/all-master-to-master": *awstasks.SecurityGroupRule {"Name":"all-master-to-master","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:49:00.239030 22451 executor.go:178] Executing task "SecurityGroupRule/all-master-to-node": *awstasks.SecurityGroupRule {"Name":"all-master-to-node","Lifecycle":"Sync","SecurityGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:49:00.238982 22451 executor.go:178] Executing task "Route/0.0.0.0/0": *awstasks.Route {"Name":"0.0.0.0/0","Lifecycle":"Sync","RouteTable":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"public"}},"Instance":null,"CIDR":"0.0.0.0/0","InternetGateway":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"NatGateway":null}
I0627 16:49:00.244119 22451 elastic_ip.go:90] AssociatedNatGatewayRouteTable not found
I0627 16:49:00.239065 22451 executor.go:178] Executing task "LoadBalancer/bastion.dev-qa.k8s.domain.com": *awstasks.LoadBalancer {"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","LoadBalancerName":"bastion-dev-qa-k8s-iotnxt-0chdmv","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"bastion-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"Listeners":{"22":{"InstancePort":22,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"TCP:22","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":null,"ConnectionDraining":null,"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":null,"SSLCertificateID":"","Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:49:00.239567 22451 executor.go:178] Executing task "LoadBalancer/api.dev-qa.k8s.domain.com": *awstasks.LoadBalancer {"Name":"api.dev-qa.k8s.domain.com","Lifecycle":"Sync","LoadBalancerName":"api-dev-qa-k8s-iotnxt-io-9s1if5","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":null,"ConnectionDraining":null,"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":null,"SSLCertificateID":"","Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:49:00.258020 22451 elastic_ip.go:90] AssociatedNatGatewayRouteTable not found
I0627 16:49:00.270420 22451 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:49:00.280872 22451 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:49:00.282641 22451 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:49:00.295701 22451 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:49:00.303171 22451 elastic_ip.go:90] AssociatedNatGatewayRouteTable not found
I0627 16:49:00.320880 22451 load_balancer.go:183] Listing all ELBs for findLoadBalancerByNameTag
I0627 16:49:00.326937 22451 load_balancer.go:183] Listing all ELBs for findLoadBalancerByNameTag
I0627 16:49:00.329420 22451 request_logger.go:45] AWS request: elasticloadbalancing/DescribeLoadBalancers
I0627 16:49:00.329709 22451 request_logger.go:45] AWS request: elasticloadbalancing/DescribeLoadBalancers
I0627 16:49:01.512421 22451 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:49:01.525421 22451 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:49:01.531270 22451 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:49:01.535270 22451 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:49:01.538900 22451 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:49:01.833646 22451 load_balancer.go:268] Querying ELB tags for [a761943728e8611e9b99e065ee05bf9e]
I0627 16:49:01.834510 22451 request_logger.go:45] AWS request: elasticloadbalancing/DescribeTags
I0627 16:49:01.857705 22451 load_balancer.go:268] Querying ELB tags for [a761943728e8611e9b99e065ee05bf9e]
I0627 16:49:01.858328 22451 request_logger.go:45] AWS request: elasticloadbalancing/DescribeTags
I0627 16:49:02.059902 22451 executor.go:103] Tasks: 126 done / 143 total; 10 can run
I0627 16:49:02.060559 22451 executor.go:178] Executing task "NatGateway/eu-central-1a.dev-qa.k8s.domain.com": *awstasks.NatGateway {"Name":"eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ElasticIP":{"Name":"eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"PublicIP":null,"TagOnSubnet":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedNatGatewayRouteTable":{"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1a"}}},"Subnet":{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},"ID":null,"EgressId":null,"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedRouteTable":{"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1a"}}}
I0627 16:49:02.062742 22451 natgateway.go:151] Unable to find subnet, bypassing Find() for NatGateway
I0627 16:49:02.060517 22451 executor.go:178] Executing task "NatGateway/eu-central-1b.dev-qa.k8s.domain.com": *awstasks.NatGateway {"Name":"eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ElasticIP":{"Name":"eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"PublicIP":null,"TagOnSubnet":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedNatGatewayRouteTable":{"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1b"}}},"Subnet":{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},"ID":null,"EgressId":null,"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedRouteTable":{"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1b"}}}
I0627 16:49:02.065214 22451 natgateway.go:151] Unable to find subnet, bypassing Find() for NatGateway
I0627 16:49:02.060600 22451 executor.go:178] Executing task "NatGateway/eu-central-1c.dev-qa.k8s.domain.com": *awstasks.NatGateway {"Name":"eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ElasticIP":{"Name":"eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"PublicIP":null,"TagOnSubnet":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedNatGatewayRouteTable":{"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1c"}}},"Subnet":{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},"ID":null,"EgressId":null,"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedRouteTable":{"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1c"}}}
I0627 16:49:02.060579 22451 executor.go:178] Executing task "AutoscalingGroup/master-eu-central-1c.masters.dev-qa.k8s.domain.com": *awstasks.AutoscalingGroup {"Name":"master-eu-central-1c.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"master-eu-central-1c.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"eu-central-1c.dev-qa.k8s.domain.com","ShortName":"eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.96/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}],"SuspendProcesses":[],"Tags":{"k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"master-eu-central-1c","k8s.io/role/master":"1"}}
I0627 16:49:02.060637 22451 executor.go:178] Executing task "AutoscalingGroup/nodes.dev-qa.k8s.domain.com": *awstasks.AutoscalingGroup {"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"m5a.2xlarge","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":300,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"eu-central-1a.dev-qa.k8s.domain.com","ShortName":"eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.32/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}},{"Name":"eu-central-1b.dev-qa.k8s.domain.com","ShortName":"eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.64/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}},{"Name":"eu-central-1c.dev-qa.k8s.domain.com","ShortName":"eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.96/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}],"SuspendProcesses":[],"Tags":{"k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"nodes","k8s.io/role/node":"1"}}
I0627 16:49:02.060648 22451 executor.go:178] Executing task "DNSName/bastion.dev-qa.k8s.domain.com": *awstasks.DNSName {"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Zone":{"Name":"k8s.domain.com","Lifecycle":"Sync","DNSName":"k8s.domain.com","ZoneID":"Z34N9ZS9SNSUL7","Private":null,"PrivateVPC":null},"ResourceType":"A","TargetLoadBalancer":{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","LoadBalancerName":"bastion-dev-qa-k8s-iotnxt-0chdmv","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"bastion-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"Listeners":{"22":{"InstancePort":22,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"TCP:22","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":null,"ConnectionDraining":null,"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":null,"SSLCertificateID":"","Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}}
I0627 16:49:02.060618 22451 executor.go:178] Executing task "AutoscalingGroup/master-eu-central-1b.masters.dev-qa.k8s.domain.com": *awstasks.AutoscalingGroup {"Name":"master-eu-central-1b.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"master-eu-central-1b.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"eu-central-1b.dev-qa.k8s.domain.com","ShortName":"eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.64/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}],"SuspendProcesses":[],"Tags":{"k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"master-eu-central-1b","k8s.io/role/master":"1"}}
I0627 16:49:02.060648 22451 executor.go:178] Executing task "AutoscalingGroup/master-eu-central-1a.masters.dev-qa.k8s.domain.com": *awstasks.AutoscalingGroup {"Name":"master-eu-central-1a.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"master-eu-central-1a.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"eu-central-1a.dev-qa.k8s.domain.com","ShortName":"eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.32/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}],"SuspendProcesses":[],"Tags":{"k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"master-eu-central-1a","k8s.io/role/master":"1"}}
I0627 16:49:02.060652 22451 executor.go:178] Executing task "AutoscalingGroup/bastions.dev-qa.k8s.domain.com": *awstasks.AutoscalingGroup {"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":true,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t2.micro","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":32,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":null},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}],"SuspendProcesses":[],"Tags":{"k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"bastions","k8s.io/role/bastion":"1"}}
I0627 16:49:02.060689 22451 executor.go:178] Executing task "DNSName/api.dev-qa.k8s.domain.com": *awstasks.DNSName {"Name":"api.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Zone":{"Name":"k8s.domain.com","Lifecycle":"Sync","DNSName":"k8s.domain.com","ZoneID":"Z34N9ZS9SNSUL7","Private":null,"PrivateVPC":null},"ResourceType":"A","TargetLoadBalancer":{"Name":"api.dev-qa.k8s.domain.com","Lifecycle":"Sync","LoadBalancerName":"api-dev-qa-k8s-iotnxt-io-9s1if5","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":null,"ConnectionDraining":null,"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":null,"SSLCertificateID":"","Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}}
I0627 16:49:02.068177 22451 natgateway.go:151] Unable to find subnet, bypassing Find() for NatGateway
I0627 16:49:02.074933 22451 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:49:02.079124 22451 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:49:02.082065 22451 request_logger.go:45] AWS request: route53/ListResourceRecordSets
I0627 16:49:02.089850 22451 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:49:02.092522 22451 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:49:02.095779 22451 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:49:02.102785 22451 request_logger.go:45] AWS request: route53/ListResourceRecordSets
I0627 16:49:02.381706 22451 dnsname.go:76] Found DNS resource "NS" "k8s.domain.com."
I0627 16:49:02.382266 22451 dnsname.go:76] Found DNS resource "SOA" "k8s.domain.com."
I0627 16:49:02.638884 22451 dnsname.go:76] Found DNS resource "NS" "k8s.domain.com."
I0627 16:49:02.639657 22451 dnsname.go:76] Found DNS resource "SOA" "k8s.domain.com."
I0627 16:49:02.700403 22451 executor.go:103] Tasks: 136 done / 143 total; 7 can run
I0627 16:49:02.701050 22451 executor.go:178] Executing task "Route/private-eu-central-1b-0.0.0.0/0": *awstasks.Route {"Name":"private-eu-central-1b-0.0.0.0/0","Lifecycle":"Sync","RouteTable":{"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1b"}},"Instance":null,"CIDR":"0.0.0.0/0","InternetGateway":null,"NatGateway":{"Name":"eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ElasticIP":{"Name":"eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"PublicIP":null,"TagOnSubnet":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedNatGatewayRouteTable":{"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1b"}}},"Subnet":{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},"ID":null,"EgressId":null,"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedRouteTable":{"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1b"}}}}
I0627 16:49:02.701053 22451 executor.go:178] Executing task "Route/private-eu-central-1a-0.0.0.0/0": *awstasks.Route {"Name":"private-eu-central-1a-0.0.0.0/0","Lifecycle":"Sync","RouteTable":{"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1a"}},"Instance":null,"CIDR":"0.0.0.0/0","InternetGateway":null,"NatGateway":{"Name":"eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ElasticIP":{"Name":"eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"PublicIP":null,"TagOnSubnet":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedNatGatewayRouteTable":{"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1a"}}},"Subnet":{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},"ID":null,"EgressId":null,"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedRouteTable":{"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1a"}}}}
I0627 16:49:02.701139 22451 executor.go:178] Executing task "Route/private-eu-central-1c-0.0.0.0/0": *awstasks.Route {"Name":"private-eu-central-1c-0.0.0.0/0","Lifecycle":"Sync","RouteTable":{"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1c"}},"Instance":null,"CIDR":"0.0.0.0/0","InternetGateway":null,"NatGateway":{"Name":"eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ElasticIP":{"Name":"eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"PublicIP":null,"TagOnSubnet":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedNatGatewayRouteTable":{"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1c"}}},"Subnet":{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},"ID":null,"EgressId":null,"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedRouteTable":{"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1c"}}}}
I0627 16:49:02.700989 22451 executor.go:178] Executing task "LoadBalancerAttachment/bastion-elb-attachment": *awstasks.LoadBalancerAttachment {"Name":"bastion-elb-attachment","Lifecycle":"Sync","LoadBalancer":{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","LoadBalancerName":"bastion-dev-qa-k8s-iotnxt-0chdmv","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"bastion-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"Listeners":{"22":{"InstancePort":22,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"TCP:22","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":null,"ConnectionDraining":null,"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":null,"SSLCertificateID":"","Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AutoscalingGroup":{"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":true,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t2.micro","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":32,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":null},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}],"SuspendProcesses":[],"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastions.dev-qa.k8s.domain.com","k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"bastions","k8s.io/role/bastion":"1"}},"Subnet":null,"Instance":null}
I0627 16:49:02.701113 22451 executor.go:178] Executing task "LoadBalancerAttachment/api-master-eu-central-1b": *awstasks.LoadBalancerAttachment {"Name":"api-master-eu-central-1b","Lifecycle":"Sync","LoadBalancer":{"Name":"api.dev-qa.k8s.domain.com","Lifecycle":"Sync","LoadBalancerName":"api-dev-qa-k8s-iotnxt-io-9s1if5","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":null,"ConnectionDraining":null,"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":null,"SSLCertificateID":"","Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AutoscalingGroup":{"Name":"master-eu-central-1b.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"master-eu-central-1b.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"eu-central-1b.dev-qa.k8s.domain.com","ShortName":"eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.64/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}],"SuspendProcesses":[],"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"master-eu-central-1b.masters.dev-qa.k8s.domain.com","k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"master-eu-central-1b","k8s.io/role/master":"1"}},"Subnet":null,"Instance":null}
I0627 16:49:02.726378 22451 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:49:02.701099 22451 executor.go:178] Executing task "LoadBalancerAttachment/api-master-eu-central-1c": *awstasks.LoadBalancerAttachment {"Name":"api-master-eu-central-1c","Lifecycle":"Sync","LoadBalancer":{"Name":"api.dev-qa.k8s.domain.com","Lifecycle":"Sync","LoadBalancerName":"api-dev-qa-k8s-iotnxt-io-9s1if5","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":null,"ConnectionDraining":null,"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":null,"SSLCertificateID":"","Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AutoscalingGroup":{"Name":"master-eu-central-1c.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"master-eu-central-1c.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"eu-central-1c.dev-qa.k8s.domain.com","ShortName":"eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.96/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}],"SuspendProcesses":[],"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"master-eu-central-1c.masters.dev-qa.k8s.domain.com","k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"master-eu-central-1c","k8s.io/role/master":"1"}},"Subnet":null,"Instance":null}
I0627 16:49:02.701036 22451 executor.go:178] Executing task "LoadBalancerAttachment/api-master-eu-central-1a": *awstasks.LoadBalancerAttachment {"Name":"api-master-eu-central-1a","Lifecycle":"Sync","LoadBalancer":{"Name":"api.dev-qa.k8s.domain.com","Lifecycle":"Sync","LoadBalancerName":"api-dev-qa-k8s-iotnxt-io-9s1if5","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}},{"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":null,"ConnectionDraining":null,"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":null,"SSLCertificateID":"","Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AutoscalingGroup":{"Name":"master-eu-central-1a.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"master-eu-central-1a.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"eu-central-1a.dev-qa.k8s.domain.com","ShortName":"eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.32/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}],"SuspendProcesses":[],"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"master-eu-central-1a.masters.dev-qa.k8s.domain.com","k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"master-eu-central-1a","k8s.io/role/master":"1"}},"Subnet":null,"Instance":null}
I0627 16:49:02.721402 22451 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:49:02.733754 22451 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:49:02.743205 22451 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:49:03.168811 22451 executor.go:103] Tasks: 143 done / 143 total; 0 can run
I0627 16:49:03.170694 22451 print.go:104] Unhandled kind in asString for "": awstasks.LoadBalancerListener
I0627 16:49:03.171148 22451 print.go:104] Unhandled kind in asString for "": awstasks.LoadBalancerHealthCheck
I0627 16:49:03.171446 22451 print.go:104] Unhandled kind in asString for "": awstasks.LoadBalancerConnectionSettings
I0627 16:49:03.171821 22451 print.go:104] Unhandled kind in asString for "": awstasks.LoadBalancerListener
I0627 16:49:03.172229 22451 print.go:104] Unhandled kind in asString for "": awstasks.LoadBalancerHealthCheck
I0627 16:49:03.172589 22451 print.go:104] Unhandled kind in asString for "": awstasks.LoadBalancerConnectionSettings
I0627 16:49:03.173595 22451 dryrun_target.go:255] ContainerAssets:
I0627 16:49:03.173921 22451 dryrun_target.go:257] k8s.gcr.io/kube-apiserver:v1.12.9
I0627 16:49:03.174172 22451 dryrun_target.go:257] k8s.gcr.io/pause-amd64:3.0
I0627 16:49:03.174478 22451 dryrun_target.go:257] k8s.gcr.io/kube-controller-manager:v1.12.9
I0627 16:49:03.174728 22451 dryrun_target.go:257] k8s.gcr.io/kube-scheduler:v1.12.9
I0627 16:49:03.175038 22451 dryrun_target.go:257] k8s.gcr.io/kube-proxy:v1.12.9
I0627 16:49:03.175279 22451 dryrun_target.go:257] k8s.gcr.io/kube-apiserver:v1.12.9
I0627 16:49:03.175597 22451 dryrun_target.go:257] k8s.gcr.io/pause-amd64:3.0
I0627 16:49:03.175839 22451 dryrun_target.go:257] k8s.gcr.io/kube-controller-manager:v1.12.9
I0627 16:49:03.176116 22451 dryrun_target.go:257] k8s.gcr.io/kube-proxy:v1.12.9
I0627 16:49:03.176315 22451 dryrun_target.go:257] k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.4.0
I0627 16:49:03.176522 22451 dryrun_target.go:257] k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13
I0627 16:49:03.176742 22451 dryrun_target.go:257] k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.13
I0627 16:49:03.177148 22451 dryrun_target.go:257] k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.13
I0627 16:49:03.177367 22451 dryrun_target.go:257] kope/dns-controller:1.12.0
I0627 16:49:03.177542 22451 dryrun_target.go:257] quay.io/calico/node:v3.4.0
I0627 16:49:03.179574 22451 dryrun_target.go:257] quay.io/calico/cni:v3.4.0
I0627 16:49:03.179788 22451 dryrun_target.go:257] calico/upgrade:v1.0.5
I0627 16:49:03.179938 22451 dryrun_target.go:257] quay.io/calico/kube-controllers:v3.4.0
I0627 16:49:03.180175 22451 dryrun_target.go:257] calico/upgrade:v1.0.5
I0627 16:49:03.180378 22451 dryrun_target.go:257] calico/upgrade:v1.0.5
I0627 16:49:03.180570 22451 dryrun_target.go:257] quay.io/calico/kube-controllers:v3.4.0
I0627 16:49:03.180797 22451 dryrun_target.go:257] calico/upgrade:v1.0.5
I0627 16:49:03.180976 22451 dryrun_target.go:257] quay.io/calico/node:v3.4.0
I0627 16:49:03.181284 22451 dryrun_target.go:257] quay.io/calico/cni:v3.4.0
I0627 16:49:03.181838 22451 dryrun_target.go:257] k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.0.0
I0627 16:49:03.182106 22451 dryrun_target.go:257] k8s.gcr.io/kubedns-amd64:1.9
I0627 16:49:03.182524 22451 dryrun_target.go:257] k8s.gcr.io/k8s-dns-dnsmasq-amd64:1.14.10
I0627 16:49:03.182824 22451 dryrun_target.go:257] k8s.gcr.io/dnsmasq-metrics-amd64:1.0
I0627 16:49:03.183052 22451 dryrun_target.go:257] k8s.gcr.io/exechealthz-amd64:1.2
I0627 16:49:03.183267 22451 dryrun_target.go:257] k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.1.2-r2
I0627 16:49:03.183577 22451 dryrun_target.go:257] k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.10
I0627 16:49:03.183791 22451 dryrun_target.go:257] k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.10
I0627 16:49:03.184059 22451 dryrun_target.go:257] k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.10
I0627 16:49:03.184268 22451 dryrun_target.go:257] kope/dns-controller:1.12.0
I0627 16:49:03.184423 22451 dryrun_target.go:257] kope/dns-controller:1.12.0
I0627 16:49:03.184702 22451 dryrun_target.go:257] kopeio/etcd-manager:3.0.20190516
I0627 16:49:03.184903 22451 dryrun_target.go:257] kopeio/etcd-manager:3.0.20190516
I0627 16:49:03.185091 22451 dryrun_target.go:262] FileAssets:
I0627 16:49:03.185361 22451 dryrun_target.go:267] https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubelet
I0627 16:49:03.185582 22451 dryrun_target.go:267] https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubectl
I0627 16:49:03.185853 22451 dryrun_target.go:267] https://storage.googleapis.com/kubernetes-release/network-plugins/cni-plugins-amd64-v0.7.5.tgz
I0627 16:49:03.186063 22451 dryrun_target.go:267] https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/utils.tar.gz
I0627 16:49:03.186322 22451 dryrun_target.go:267] https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/nodeup
I0627 16:49:03.186553 22451 dryrun_target.go:267] https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz
Will create resources:
AutoscalingGroup/bastions.dev-qa.k8s.domain.com
Granularity 1Minute
LaunchConfiguration name:bastions.dev-qa.k8s.domain.com
MaxSize 1
Metrics [GroupDesiredCapacity, GroupInServiceInstances, GroupMaxSize, GroupMinSize, GroupPendingInstances, GroupStandbyInstances, GroupTerminatingInstances, GroupTotalInstances]
MinSize 1
Subnets [name:utility-eu-central-1a.dev-qa.k8s.domain.com, name:utility-eu-central-1b.dev-qa.k8s.domain.com, name:utility-eu-central-1c.dev-qa.k8s.domain.com]
SuspendProcesses []
Tags {KubernetesCluster: dev-qa.k8s.domain.com, k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup: bastions, k8s.io/role/bastion: 1, Name: bastions.dev-qa.k8s.domain.com}
AutoscalingGroup/master-eu-central-1a.masters.dev-qa.k8s.domain.com
Granularity 1Minute
LaunchConfiguration name:master-eu-central-1a.masters.dev-qa.k8s.domain.com
MaxSize 1
Metrics [GroupDesiredCapacity, GroupInServiceInstances, GroupMaxSize, GroupMinSize, GroupPendingInstances, GroupStandbyInstances, GroupTerminatingInstances, GroupTotalInstances]
MinSize 1
Subnets [name:eu-central-1a.dev-qa.k8s.domain.com]
SuspendProcesses []
Tags {KubernetesCluster: dev-qa.k8s.domain.com, k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup: master-eu-central-1a, k8s.io/role/master: 1, Name: master-eu-central-1a.masters.dev-qa.k8s.domain.com}
AutoscalingGroup/master-eu-central-1b.masters.dev-qa.k8s.domain.com
Granularity 1Minute
LaunchConfiguration name:master-eu-central-1b.masters.dev-qa.k8s.domain.com
MaxSize 1
Metrics [GroupDesiredCapacity, GroupInServiceInstances, GroupMaxSize, GroupMinSize, GroupPendingInstances, GroupStandbyInstances, GroupTerminatingInstances, GroupTotalInstances]
MinSize 1
Subnets [name:eu-central-1b.dev-qa.k8s.domain.com]
SuspendProcesses []
Tags {k8s.io/role/master: 1, Name: master-eu-central-1b.masters.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup: master-eu-central-1b}
AutoscalingGroup/master-eu-central-1c.masters.dev-qa.k8s.domain.com
Granularity 1Minute
LaunchConfiguration name:master-eu-central-1c.masters.dev-qa.k8s.domain.com
MaxSize 1
Metrics [GroupDesiredCapacity, GroupInServiceInstances, GroupMaxSize, GroupMinSize, GroupPendingInstances, GroupStandbyInstances, GroupTerminatingInstances, GroupTotalInstances]
MinSize 1
Subnets [name:eu-central-1c.dev-qa.k8s.domain.com]
SuspendProcesses []
Tags {Name: master-eu-central-1c.masters.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup: master-eu-central-1c, k8s.io/role/master: 1}
AutoscalingGroup/nodes.dev-qa.k8s.domain.com
Granularity 1Minute
LaunchConfiguration name:nodes.dev-qa.k8s.domain.com
MaxSize 1
Metrics [GroupDesiredCapacity, GroupInServiceInstances, GroupMaxSize, GroupMinSize, GroupPendingInstances, GroupStandbyInstances, GroupTerminatingInstances, GroupTotalInstances]
MinSize 1
Subnets [name:eu-central-1a.dev-qa.k8s.domain.com, name:eu-central-1b.dev-qa.k8s.domain.com, name:eu-central-1c.dev-qa.k8s.domain.com]
SuspendProcesses []
Tags {k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup: nodes, k8s.io/role/node: 1, Name: nodes.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com}
DHCPOptions/dev-qa.k8s.domain.com
DomainName eu-central-1.compute.internal
DomainNameServers AmazonProvidedDNS
Shared false
Tags {Name: dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned}
DNSName/api.dev-qa.k8s.domain.com
Zone name:k8s.domain.com id:k8s.domain.com
ResourceType A
TargetLoadBalancer name:api.dev-qa.k8s.domain.com id:api.dev-qa.k8s.domain.com
DNSName/bastion.dev-qa.k8s.domain.com
Zone name:k8s.domain.com id:k8s.domain.com
ResourceType A
TargetLoadBalancer name:bastion.dev-qa.k8s.domain.com id:bastion.dev-qa.k8s.domain.com
EBSVolume/a.etcd-events.dev-qa.k8s.domain.com
AvailabilityZone eu-central-1a
Encrypted true
SizeGB 20
Tags {k8s.io/etcd/events: a/a,b,c, k8s.io/role/master: 1, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, Name: a.etcd-events.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com}
VolumeType gp2
EBSVolume/a.etcd-main.dev-qa.k8s.domain.com
AvailabilityZone eu-central-1a
Encrypted true
SizeGB 20
Tags {k8s.io/etcd/main: a/a,b,c, k8s.io/role/master: 1, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, Name: a.etcd-main.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com}
VolumeType gp2
EBSVolume/b.etcd-events.dev-qa.k8s.domain.com
AvailabilityZone eu-central-1b
Encrypted true
SizeGB 20
Tags {k8s.io/etcd/events: b/a,b,c, k8s.io/role/master: 1, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, Name: b.etcd-events.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com}
VolumeType gp2
EBSVolume/b.etcd-main.dev-qa.k8s.domain.com
AvailabilityZone eu-central-1b
Encrypted true
SizeGB 20
Tags {KubernetesCluster: dev-qa.k8s.domain.com, k8s.io/etcd/main: b/a,b,c, k8s.io/role/master: 1, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, Name: b.etcd-main.dev-qa.k8s.domain.com}
VolumeType gp2
EBSVolume/c.etcd-events.dev-qa.k8s.domain.com
AvailabilityZone eu-central-1c
Encrypted true
SizeGB 20
Tags {k8s.io/etcd/events: c/a,b,c, k8s.io/role/master: 1, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, Name: c.etcd-events.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com}
VolumeType gp2
EBSVolume/c.etcd-main.dev-qa.k8s.domain.com
AvailabilityZone eu-central-1c
Encrypted true
SizeGB 20
Tags {k8s.io/etcd/main: c/a,b,c, k8s.io/role/master: 1, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, Name: c.etcd-main.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com}
VolumeType gp2
ElasticIP/eu-central-1a.dev-qa.k8s.domain.com
Tags {kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, Name: eu-central-1a.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com}
AssociatedNatGatewayRouteTable name:private-eu-central-1a.dev-qa.k8s.domain.com
ElasticIP/eu-central-1b.dev-qa.k8s.domain.com
Tags {Name: eu-central-1b.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned}
AssociatedNatGatewayRouteTable name:private-eu-central-1b.dev-qa.k8s.domain.com
ElasticIP/eu-central-1c.dev-qa.k8s.domain.com
Tags {KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, Name: eu-central-1c.dev-qa.k8s.domain.com}
AssociatedNatGatewayRouteTable name:private-eu-central-1c.dev-qa.k8s.domain.com
IAMInstanceProfile/bastions.dev-qa.k8s.domain.com
Shared false
IAMInstanceProfile/masters.dev-qa.k8s.domain.com
Shared false
IAMInstanceProfile/nodes.dev-qa.k8s.domain.com
Shared false
IAMInstanceProfileRole/bastions.dev-qa.k8s.domain.com
InstanceProfile name:bastions.dev-qa.k8s.domain.com id:bastions.dev-qa.k8s.domain.com
Role name:bastions.dev-qa.k8s.domain.com
IAMInstanceProfileRole/masters.dev-qa.k8s.domain.com
InstanceProfile name:masters.dev-qa.k8s.domain.com id:masters.dev-qa.k8s.domain.com
Role name:masters.dev-qa.k8s.domain.com
IAMInstanceProfileRole/nodes.dev-qa.k8s.domain.com
InstanceProfile name:nodes.dev-qa.k8s.domain.com id:nodes.dev-qa.k8s.domain.com
Role name:nodes.dev-qa.k8s.domain.com
IAMRole/bastions.dev-qa.k8s.domain.com
ExportWithID bastions
IAMRole/masters.dev-qa.k8s.domain.com
ExportWithID masters
IAMRole/nodes.dev-qa.k8s.domain.com
ExportWithID nodes
IAMRolePolicy/bastions.dev-qa.k8s.domain.com
Role name:bastions.dev-qa.k8s.domain.com
IAMRolePolicy/masters.dev-qa.k8s.domain.com
Role name:masters.dev-qa.k8s.domain.com
IAMRolePolicy/nodes.dev-qa.k8s.domain.com
Role name:nodes.dev-qa.k8s.domain.com
InternetGateway/dev-qa.k8s.domain.com
VPC name:dev-qa.k8s.domain.com
Shared false
Tags {Name: dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned}
Keypair/apiserver-aggregator
Signer name:apiserver-aggregator-ca id:cn=apiserver-aggregator-ca
Subject cn=aggregator
Type client
Format v1alpha2
Keypair/apiserver-aggregator-ca
Subject cn=apiserver-aggregator-ca
Type ca
Format v1alpha2
Keypair/apiserver-proxy-client
Signer name:ca id:cn=kubernetes
Subject cn=apiserver-proxy-client
Type client
Format v1alpha2
Keypair/ca
Subject cn=kubernetes
Type ca
Format v1alpha2
Keypair/etcd-clients-ca
Subject cn=etcd-clients-ca
Type ca
Format v1alpha2
Keypair/etcd-manager-ca-events
Subject cn=etcd-manager-ca-events
Type ca
Format v1alpha2
Keypair/etcd-manager-ca-main
Subject cn=etcd-manager-ca-main
Type ca
Format v1alpha2
Keypair/etcd-peers-ca-events
Subject cn=etcd-peers-ca-events
Type ca
Format v1alpha2
Keypair/etcd-peers-ca-main
Subject cn=etcd-peers-ca-main
Type ca
Format v1alpha2
Keypair/kops
Signer name:ca id:cn=kubernetes
Subject o=system:masters,cn=kops
Type client
Format v1alpha2
Keypair/kube-controller-manager
Signer name:ca id:cn=kubernetes
Subject cn=system:kube-controller-manager
Type client
Format v1alpha2
Keypair/kube-proxy
Signer name:ca id:cn=kubernetes
Subject cn=system:kube-proxy
Type client
Format v1alpha2
Keypair/kube-scheduler
Signer name:ca id:cn=kubernetes
Subject cn=system:kube-scheduler
Type client
Format v1alpha2
Keypair/kubecfg
Signer name:ca id:cn=kubernetes
Subject o=system:masters,cn=kubecfg
Type client
Format v1alpha2
Keypair/kubelet
Signer name:ca id:cn=kubernetes
Subject o=system:nodes,cn=kubelet
Type client
Format v1alpha2
Keypair/kubelet-api
Signer name:ca id:cn=kubernetes
Subject cn=kubelet-api
Type client
Format v1alpha2
Keypair/master
AlternateNames [100.64.0.1, 127.0.0.1, api.dev-qa.k8s.domain.com, api.internal.dev-qa.k8s.domain.com, kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster.local]
Signer name:ca id:cn=kubernetes
Subject cn=kubernetes-master
Type server
Format v1alpha2
LaunchConfiguration/bastions.dev-qa.k8s.domain.com
AssociatePublicIP true
IAMInstanceProfile name:bastions.dev-qa.k8s.domain.com id:bastions.dev-qa.k8s.domain.com
ImageID 595879546273/CoreOS-stable-2135.4.0-hvm
InstanceType t2.micro
RootVolumeSize 32
RootVolumeType gp2
SSHKey name:kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b id:kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b
SecurityGroups [name:bastion.dev-qa.k8s.domain.com]
SpotPrice
LaunchConfiguration/master-eu-central-1a.masters.dev-qa.k8s.domain.com
AssociatePublicIP false
IAMInstanceProfile name:masters.dev-qa.k8s.domain.com id:masters.dev-qa.k8s.domain.com
ImageID 595879546273/CoreOS-stable-2135.4.0-hvm
InstanceType t3.large
RootVolumeSize 100
RootVolumeType gp2
SSHKey name:kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b id:kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b
SecurityGroups [name:masters.dev-qa.k8s.domain.com]
SpotPrice
LaunchConfiguration/master-eu-central-1b.masters.dev-qa.k8s.domain.com
AssociatePublicIP false
IAMInstanceProfile name:masters.dev-qa.k8s.domain.com id:masters.dev-qa.k8s.domain.com
ImageID 595879546273/CoreOS-stable-2135.4.0-hvm
InstanceType t3.large
RootVolumeSize 100
RootVolumeType gp2
SSHKey name:kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b id:kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b
SecurityGroups [name:masters.dev-qa.k8s.domain.com]
SpotPrice
LaunchConfiguration/master-eu-central-1c.masters.dev-qa.k8s.domain.com
AssociatePublicIP false
IAMInstanceProfile name:masters.dev-qa.k8s.domain.com id:masters.dev-qa.k8s.domain.com
ImageID 595879546273/CoreOS-stable-2135.4.0-hvm
InstanceType t3.large
RootVolumeSize 100
RootVolumeType gp2
SSHKey name:kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b id:kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b
SecurityGroups [name:masters.dev-qa.k8s.domain.com]
SpotPrice
LaunchConfiguration/nodes.dev-qa.k8s.domain.com
AssociatePublicIP false
IAMInstanceProfile name:nodes.dev-qa.k8s.domain.com id:nodes.dev-qa.k8s.domain.com
ImageID 595879546273/CoreOS-stable-2135.4.0-hvm
InstanceType m5a.2xlarge
RootVolumeSize 300
RootVolumeType gp2
SSHKey name:kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b id:kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b
SecurityGroups [name:nodes.dev-qa.k8s.domain.com]
SpotPrice
LoadBalancer/api.dev-qa.k8s.domain.com
LoadBalancerName api-dev-qa-k8s-iotnxt-io-9s1if5
Subnets [name:utility-eu-central-1a.dev-qa.k8s.domain.com, name:utility-eu-central-1b.dev-qa.k8s.domain.com, name:utility-eu-central-1c.dev-qa.k8s.domain.com]
SecurityGroups [name:api-elb.dev-qa.k8s.domain.com]
Listeners {443: {"InstancePort":443,"SSLCertificateID":""}}
HealthCheck {"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5}
ConnectionSettings {"IdleTimeout":300}
SSLCertificateID
Tags {Name: api.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned}
LoadBalancer/bastion.dev-qa.k8s.domain.com
LoadBalancerName bastion-dev-qa-k8s-iotnxt-0chdmv
Subnets [name:utility-eu-central-1a.dev-qa.k8s.domain.com, name:utility-eu-central-1b.dev-qa.k8s.domain.com, name:utility-eu-central-1c.dev-qa.k8s.domain.com]
SecurityGroups [name:bastion-elb.dev-qa.k8s.domain.com]
Listeners {22: {"InstancePort":22,"SSLCertificateID":""}}
HealthCheck {"Target":"TCP:22","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5}
ConnectionSettings {"IdleTimeout":300}
SSLCertificateID
Tags {Name: bastion.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned}
LoadBalancerAttachment/api-master-eu-central-1a
LoadBalancer name:api.dev-qa.k8s.domain.com id:api.dev-qa.k8s.domain.com
AutoscalingGroup name:master-eu-central-1a.masters.dev-qa.k8s.domain.com id:master-eu-central-1a.masters.dev-qa.k8s.domain.com
LoadBalancerAttachment/api-master-eu-central-1b
LoadBalancer name:api.dev-qa.k8s.domain.com id:api.dev-qa.k8s.domain.com
AutoscalingGroup name:master-eu-central-1b.masters.dev-qa.k8s.domain.com id:master-eu-central-1b.masters.dev-qa.k8s.domain.com
LoadBalancerAttachment/api-master-eu-central-1c
LoadBalancer name:api.dev-qa.k8s.domain.com id:api.dev-qa.k8s.domain.com
AutoscalingGroup name:master-eu-central-1c.masters.dev-qa.k8s.domain.com id:master-eu-central-1c.masters.dev-qa.k8s.domain.com
LoadBalancerAttachment/bastion-elb-attachment
LoadBalancer name:bastion.dev-qa.k8s.domain.com id:bastion.dev-qa.k8s.domain.com
AutoscalingGroup name:bastions.dev-qa.k8s.domain.com id:bastions.dev-qa.k8s.domain.com
ManagedFile/dev-qa.k8s.domain.com-addons-bootstrap
Location addons/bootstrap-channel.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-core.addons.k8s.io
Location addons/core.addons.k8s.io/v1.4.0.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.12
Location addons/dns-controller.addons.k8s.io/k8s-1.12.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.6
Location addons/dns-controller.addons.k8s.io/k8s-1.6.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-pre-k8s-1.6
Location addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.12
Location addons/kube-dns.addons.k8s.io/k8s-1.12.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.6
Location addons/kube-dns.addons.k8s.io/k8s-1.6.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-pre-k8s-1.6
Location addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-kubelet-api.rbac.addons.k8s.io-k8s-1.9
Location addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-limit-range.addons.k8s.io
Location addons/limit-range.addons.k8s.io/v1.5.0.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.12
Location addons/networking.projectcalico.org/k8s-1.12.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.7-v3
Location addons/networking.projectcalico.org/k8s-1.7-v3.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-rbac.addons.k8s.io-k8s-1.8
Location addons/rbac.addons.k8s.io/k8s-1.8.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.6.0
Location addons/storage-aws.addons.k8s.io/v1.6.0.yaml
ManagedFile/dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.7.0
Location addons/storage-aws.addons.k8s.io/v1.7.0.yaml
ManagedFile/etcd-cluster-spec-events
Location backups/etcd/events/control/etcd-cluster-spec
ManagedFile/etcd-cluster-spec-main
Location backups/etcd/main/control/etcd-cluster-spec
ManagedFile/manifests-etcdmanager-events
Location manifests/etcd/events.yaml
ManagedFile/manifests-etcdmanager-main
Location manifests/etcd/main.yaml
NatGateway/eu-central-1a.dev-qa.k8s.domain.com
ElasticIP name:eu-central-1a.dev-qa.k8s.domain.com
Subnet name:utility-eu-central-1a.dev-qa.k8s.domain.com
Tags {Name: eu-central-1a.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned}
AssociatedRouteTable name:private-eu-central-1a.dev-qa.k8s.domain.com
NatGateway/eu-central-1b.dev-qa.k8s.domain.com
ElasticIP name:eu-central-1b.dev-qa.k8s.domain.com
Subnet name:utility-eu-central-1b.dev-qa.k8s.domain.com
Tags {Name: eu-central-1b.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned}
AssociatedRouteTable name:private-eu-central-1b.dev-qa.k8s.domain.com
NatGateway/eu-central-1c.dev-qa.k8s.domain.com
ElasticIP name:eu-central-1c.dev-qa.k8s.domain.com
Subnet name:utility-eu-central-1c.dev-qa.k8s.domain.com
Tags {Name: eu-central-1c.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned}
AssociatedRouteTable name:private-eu-central-1c.dev-qa.k8s.domain.com
Route/0.0.0.0/0
RouteTable name:dev-qa.k8s.domain.com
CIDR 0.0.0.0/0
InternetGateway name:dev-qa.k8s.domain.com
Route/private-eu-central-1a-0.0.0.0/0
RouteTable name:private-eu-central-1a.dev-qa.k8s.domain.com
CIDR 0.0.0.0/0
NatGateway name:eu-central-1a.dev-qa.k8s.domain.com
Route/private-eu-central-1b-0.0.0.0/0
RouteTable name:private-eu-central-1b.dev-qa.k8s.domain.com
CIDR 0.0.0.0/0
NatGateway name:eu-central-1b.dev-qa.k8s.domain.com
Route/private-eu-central-1c-0.0.0.0/0
RouteTable name:private-eu-central-1c.dev-qa.k8s.domain.com
CIDR 0.0.0.0/0
NatGateway name:eu-central-1c.dev-qa.k8s.domain.com
RouteTable/dev-qa.k8s.domain.com
VPC name:dev-qa.k8s.domain.com
Shared false
Tags {Name: dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, kubernetes.io/kops/role: public}
RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com
VPC name:dev-qa.k8s.domain.com
Shared false
Tags {Name: private-eu-central-1a.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, kubernetes.io/kops/role: private-eu-central-1a}
RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com
VPC name:dev-qa.k8s.domain.com
Shared false
Tags {KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, kubernetes.io/kops/role: private-eu-central-1b, Name: private-eu-central-1b.dev-qa.k8s.domain.com}
RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com
VPC name:dev-qa.k8s.domain.com
Shared false
Tags {KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, kubernetes.io/kops/role: private-eu-central-1c, Name: private-eu-central-1c.dev-qa.k8s.domain.com}
RouteTableAssociation/private-eu-central-1a.dev-qa.k8s.domain.com
RouteTable name:private-eu-central-1a.dev-qa.k8s.domain.com
Subnet name:eu-central-1a.dev-qa.k8s.domain.com
RouteTableAssociation/private-eu-central-1b.dev-qa.k8s.domain.com
RouteTable name:private-eu-central-1b.dev-qa.k8s.domain.com
Subnet name:eu-central-1b.dev-qa.k8s.domain.com
RouteTableAssociation/private-eu-central-1c.dev-qa.k8s.domain.com
RouteTable name:private-eu-central-1c.dev-qa.k8s.domain.com
Subnet name:eu-central-1c.dev-qa.k8s.domain.com
RouteTableAssociation/utility-eu-central-1a.dev-qa.k8s.domain.com
RouteTable name:dev-qa.k8s.domain.com
Subnet name:utility-eu-central-1a.dev-qa.k8s.domain.com
RouteTableAssociation/utility-eu-central-1b.dev-qa.k8s.domain.com
RouteTable name:dev-qa.k8s.domain.com
Subnet name:utility-eu-central-1b.dev-qa.k8s.domain.com
RouteTableAssociation/utility-eu-central-1c.dev-qa.k8s.domain.com
RouteTable name:dev-qa.k8s.domain.com
Subnet name:utility-eu-central-1c.dev-qa.k8s.domain.com
SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b
KeyFingerprint c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8
Secret/admin
Secret/kube
Secret/kube-proxy
Secret/kubelet
Secret/system:controller_manager
Secret/system:dns
Secret/system:logging
Secret/system:monitoring
Secret/system:scheduler
SecurityGroup/api-elb.dev-qa.k8s.domain.com
Description Security group for api ELB
VPC name:dev-qa.k8s.domain.com
RemoveExtraRules [port=443]
Tags {Name: api-elb.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned}
SecurityGroup/bastion-elb.dev-qa.k8s.domain.com
Description Security group for bastion ELB
VPC name:dev-qa.k8s.domain.com
RemoveExtraRules [port=22]
Tags {kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, Name: bastion-elb.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com}
SecurityGroup/bastion.dev-qa.k8s.domain.com
Description Security group for bastion
VPC name:dev-qa.k8s.domain.com
RemoveExtraRules [port=22]
Tags {KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, Name: bastion.dev-qa.k8s.domain.com}
SecurityGroup/masters.dev-qa.k8s.domain.com
Description Security group for masters
VPC name:dev-qa.k8s.domain.com
RemoveExtraRules [port=22, port=443, port=2380, port=2381, port=4001, port=4002, port=4789, port=179]
Tags {Name: masters.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned}
SecurityGroup/nodes.dev-qa.k8s.domain.com
Description Security group for nodes
VPC name:dev-qa.k8s.domain.com
RemoveExtraRules [port=22]
Tags {Name: nodes.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned}
SecurityGroupRule/all-master-to-master
SecurityGroup name:masters.dev-qa.k8s.domain.com
SourceGroup name:masters.dev-qa.k8s.domain.com
SecurityGroupRule/all-master-to-node
SecurityGroup name:nodes.dev-qa.k8s.domain.com
SourceGroup name:masters.dev-qa.k8s.domain.com
SecurityGroupRule/all-node-to-node
SecurityGroup name:nodes.dev-qa.k8s.domain.com
SourceGroup name:nodes.dev-qa.k8s.domain.com
SecurityGroupRule/api-elb-egress
SecurityGroup name:api-elb.dev-qa.k8s.domain.com
CIDR 0.0.0.0/0
Egress true
SecurityGroupRule/bastion-egress
SecurityGroup name:bastion.dev-qa.k8s.domain.com
CIDR 0.0.0.0/0
Egress true
SecurityGroupRule/bastion-elb-egress
SecurityGroup name:bastion-elb.dev-qa.k8s.domain.com
CIDR 0.0.0.0/0
Egress true
SecurityGroupRule/bastion-to-master-ssh
SecurityGroup name:masters.dev-qa.k8s.domain.com
Protocol tcp
FromPort 22
ToPort 22
SourceGroup name:bastion.dev-qa.k8s.domain.com
SecurityGroupRule/bastion-to-node-ssh
SecurityGroup name:nodes.dev-qa.k8s.domain.com
Protocol tcp
FromPort 22
ToPort 22
SourceGroup name:bastion.dev-qa.k8s.domain.com
SecurityGroupRule/https-api-elb-0.0.0.0/0
SecurityGroup name:api-elb.dev-qa.k8s.domain.com
CIDR 0.0.0.0/0
Protocol tcp
FromPort 443
ToPort 443
SecurityGroupRule/https-elb-to-master
SecurityGroup name:masters.dev-qa.k8s.domain.com
Protocol tcp
FromPort 443
ToPort 443
SourceGroup name:api-elb.dev-qa.k8s.domain.com
SecurityGroupRule/icmp-pmtu-api-elb-0.0.0.0/0
SecurityGroup name:api-elb.dev-qa.k8s.domain.com
CIDR 0.0.0.0/0
Protocol icmp
FromPort 3
ToPort 4
SecurityGroupRule/master-egress
SecurityGroup name:masters.dev-qa.k8s.domain.com
CIDR 0.0.0.0/0
Egress true
SecurityGroupRule/node-egress
SecurityGroup name:nodes.dev-qa.k8s.domain.com
CIDR 0.0.0.0/0
Egress true
SecurityGroupRule/node-to-master-protocol-ipip
SecurityGroup name:masters.dev-qa.k8s.domain.com
Protocol 4
SourceGroup name:nodes.dev-qa.k8s.domain.com
SecurityGroupRule/node-to-master-tcp-1-2379
SecurityGroup name:masters.dev-qa.k8s.domain.com
Protocol tcp
FromPort 1
ToPort 2379
SourceGroup name:nodes.dev-qa.k8s.domain.com
SecurityGroupRule/node-to-master-tcp-2382-4001
SecurityGroup name:masters.dev-qa.k8s.domain.com
Protocol tcp
FromPort 2382
ToPort 4001
SourceGroup name:nodes.dev-qa.k8s.domain.com
SecurityGroupRule/node-to-master-tcp-4003-65535
SecurityGroup name:masters.dev-qa.k8s.domain.com
Protocol tcp
FromPort 4003
ToPort 65535
SourceGroup name:nodes.dev-qa.k8s.domain.com
SecurityGroupRule/node-to-master-udp-1-65535
SecurityGroup name:masters.dev-qa.k8s.domain.com
Protocol udp
FromPort 1
ToPort 65535
SourceGroup name:nodes.dev-qa.k8s.domain.com
SecurityGroupRule/ssh-elb-to-bastion
SecurityGroup name:bastion.dev-qa.k8s.domain.com
Protocol tcp
FromPort 22
ToPort 22
SourceGroup name:bastion-elb.dev-qa.k8s.domain.com
SecurityGroupRule/ssh-external-to-bastion-elb-0.0.0.0/0
SecurityGroup name:bastion-elb.dev-qa.k8s.domain.com
CIDR 0.0.0.0/0
Protocol tcp
FromPort 22
ToPort 22
Subnet/eu-central-1a.dev-qa.k8s.domain.com
ShortName eu-central-1a
VPC name:dev-qa.k8s.domain.com
AvailabilityZone eu-central-1a
CIDR 172.31.239.32/27
Shared false
Tags {Name: eu-central-1a.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, SubnetType: Private, kubernetes.io/role/internal-elb: 1}
Subnet/eu-central-1b.dev-qa.k8s.domain.com
ShortName eu-central-1b
VPC name:dev-qa.k8s.domain.com
AvailabilityZone eu-central-1b
CIDR 172.31.239.64/27
Shared false
Tags {SubnetType: Private, kubernetes.io/role/internal-elb: 1, Name: eu-central-1b.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned}
Subnet/eu-central-1c.dev-qa.k8s.domain.com
ShortName eu-central-1c
VPC name:dev-qa.k8s.domain.com
AvailabilityZone eu-central-1c
CIDR 172.31.239.96/27
Shared false
Tags {kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, SubnetType: Private, kubernetes.io/role/internal-elb: 1, Name: eu-central-1c.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com}
Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com
ShortName utility-eu-central-1a
VPC name:dev-qa.k8s.domain.com
AvailabilityZone eu-central-1a
CIDR 172.31.239.0/30
Shared false
Tags {kubernetes.io/role/elb: 1, Name: utility-eu-central-1a.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, SubnetType: Utility}
Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com
ShortName utility-eu-central-1b
VPC name:dev-qa.k8s.domain.com
AvailabilityZone eu-central-1b
CIDR 172.31.239.4/30
Shared false
Tags {Name: utility-eu-central-1b.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, SubnetType: Utility, kubernetes.io/role/elb: 1}
Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com
ShortName utility-eu-central-1c
VPC name:dev-qa.k8s.domain.com
AvailabilityZone eu-central-1c
CIDR 172.31.239.8/30
Shared false
Tags {Name: utility-eu-central-1c.dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com, kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, SubnetType: Utility, kubernetes.io/role/elb: 1}
VPC/dev-qa.k8s.domain.com
CIDR 172.31.239.0/24
EnableDNSHostnames true
EnableDNSSupport true
Shared false
Tags {kubernetes.io/cluster/dev-qa.k8s.domain.com: owned, Name: dev-qa.k8s.domain.com, KubernetesCluster: dev-qa.k8s.domain.com}
VPCDHCPOptionsAssociation/dev-qa.k8s.domain.com
VPC name:dev-qa.k8s.domain.com
DHCPOptions name:dev-qa.k8s.domain.com
I0627 16:49:03.372791 22451 context.go:91] deleting temp dir: "/tmp/deploy905510386"
Must specify --yes to apply changes
Cluster configuration has been created.
Suggestions:
* list clusters with: kops get cluster
* edit this cluster with: kops edit cluster dev-qa.k8s.domain.com
* edit your node instance group: kops edit ig --name=dev-qa.k8s.domain.com nodes
* edit your master instance group: kops edit ig --name=dev-qa.k8s.domain.com master-eu-central-1a
Finally configure your cluster with: kops update cluster --name dev-qa.k8s.domain.com --yes
Raw
kops-update-cluster.log
This file has been truncated, but you can view the full file.
I0627 16:53:46.868344 22505 factory.go:68] state store s3://my-state-store
I0627 16:53:46.868925 22505 s3context.go:325] unable to read /sys/devices/virtual/dmi/id/product_uuid, assuming not running on EC2: open /sys/devices/virtual/dmi/id/product_uuid: no such file or directory
I0627 16:53:46.869237 22505 s3context.go:170] defaulting region to "us-east-1"
I0627 16:53:48.471231 22505 s3context.go:210] found bucket in region "eu-central-1"
I0627 16:53:48.472295 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/config"
I0627 16:53:49.557334 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/instancegroup/"
I0627 16:53:49.766304 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/instancegroup: [s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/bastions s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1a s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1b s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1c s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/nodes]
I0627 16:53:49.767199 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/bastions"
I0627 16:53:49.956671 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1a"
I0627 16:53:50.141920 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1b"
I0627 16:53:50.333209 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1c"
I0627 16:53:50.549761 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/nodes"
I0627 16:53:50.774332 22505 channel.go:97] resolving "stable" against default channel location "https://raw.githubusercontent.com/kubernetes/kops/master/channels/"
I0627 16:53:50.775571 22505 channel.go:102] Loading channel from "https://raw.githubusercontent.com/kubernetes/kops/master/channels/stable"
I0627 16:53:50.775933 22505 context.go:159] Performing HTTP request: GET https://raw.githubusercontent.com/kubernetes/kops/master/channels/stable
I0627 16:53:51.427475 22505 channel.go:111] Channel contents: spec:
images:
# We put the "legacy" version first, for kops versions that don't support versions ( < 1.5.0 )
- name: kope.io/k8s-1.4-debian-jessie-amd64-hvm-ebs-2017-07-28
providerID: aws
kubernetesVersion: ">=1.4.0 <1.5.0"
- name: kope.io/k8s-1.5-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.5.0 <1.6.0"
- name: kope.io/k8s-1.6-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.6.0 <1.7.0"
- name: kope.io/k8s-1.7-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.7.0 <1.8.0"
- name: kope.io/k8s-1.8-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.8.0 <1.9.0"
- name: kope.io/k8s-1.9-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.9.0 <1.10.0"
- name: kope.io/k8s-1.10-debian-jessie-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.10.0 <1.11.0"
# Stretch is the default for 1.11 (for nvme)
- name: kope.io/k8s-1.11-debian-stretch-amd64-hvm-ebs-2018-08-17
providerID: aws
kubernetesVersion: ">=1.11.0 <1.12.0"
- name: kope.io/k8s-1.12-debian-stretch-amd64-hvm-ebs-2019-05-13
providerID: aws
kubernetesVersion: ">=1.12.0"
- providerID: gce
name: "cos-cloud/cos-stable-65-10323-99-0"
cluster:
kubernetesVersion: v1.5.8
networking:
kubenet: {}
kubernetesVersions:
- range: ">=1.14.0"
recommendedVersion: 1.14.1
requiredVersion: 1.14.0
- range: ">=1.13.0"
recommendedVersion: 1.13.5
requiredVersion: 1.13.0
- range: ">=1.12.0"
recommendedVersion: 1.12.8
requiredVersion: 1.12.0
- range: ">=1.11.0"
recommendedVersion: 1.11.10
requiredVersion: 1.11.0
- range: ">=1.10.0"
recommendedVersion: 1.10.13
requiredVersion: 1.10.0
- range: ">=1.9.0"
recommendedVersion: 1.9.11
requiredVersion: 1.9.0
- range: ">=1.8.0"
recommendedVersion: 1.8.15
requiredVersion: 1.8.0
- range: ">=1.7.0"
recommendedVersion: 1.7.16
requiredVersion: 1.7.0
- range: ">=1.6.0"
recommendedVersion: 1.6.13
requiredVersion: 1.6.0
- range: ">=1.5.0"
recommendedVersion: 1.5.8
requiredVersion: 1.5.1
- range: "<1.5.0"
recommendedVersion: 1.4.12
requiredVersion: 1.4.2
kopsVersions:
- range: ">=1.14.0-alpha.1"
#recommendedVersion: "1.14.0"
#requiredVersion: 1.14.0
kubernetesVersion: 1.14.1
- range: ">=1.13.0-alpha.1"
#recommendedVersion: "1.13.0"
#requiredVersion: 1.13.0
kubernetesVersion: 1.13.5
- range: ">=1.12.0-alpha.1"
recommendedVersion: "1.12.1"
#requiredVersion: 1.12.0
kubernetesVersion: 1.12.8
- range: ">=1.11.0-alpha.1"
#recommendedVersion: "1.11.0"
#requiredVersion: 1.11.0
kubernetesVersion: 1.11.10
- range: ">=1.10.0-alpha.1"
recommendedVersion: "1.10.0"
#requiredVersion: 1.10.0
kubernetesVersion: 1.10.13
- range: ">=1.9.0-alpha.1"
recommendedVersion: 1.9.2
#requiredVersion: 1.9.0
kubernetesVersion: 1.9.11
- range: ">=1.8.0-alpha.1"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.8.15
- range: ">=1.7.0-alpha.1"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.7.16
- range: ">=1.6.0-alpha.1"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.6.13
- range: ">=1.5.0-alpha1"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.5.8
- range: "<1.5.0"
recommendedVersion: 1.8.1
requiredVersion: 1.7.1
kubernetesVersion: 1.4.12
I0627 16:53:51.455810 22505 populate_cluster_spec.go:371] Defaulted KubeControllerManager.ClusterCIDR to 100.96.0.0/11
I0627 16:53:51.456160 22505 populate_cluster_spec.go:378] Defaulted ServiceClusterIPRange to 100.64.0.0/13
I0627 16:53:51.457879 22505 aws_cloud.go:1212] Querying EC2 for all valid zones in region "eu-central-1"
I0627 16:53:51.458298 22505 request_logger.go:45] AWS request: ec2/DescribeAvailabilityZones
I0627 16:53:52.559982 22505 subnets.go:49] All subnets have CIDRs; skipping assignment logic
I0627 16:53:52.560628 22505 defaults.go:213] Not setting up Proxy Excludes
I0627 16:53:52.561049 22505 aws_cloud.go:1212] Querying EC2 for all valid zones in region "eu-central-1"
I0627 16:53:52.561383 22505 request_logger.go:45] AWS request: ec2/DescribeAvailabilityZones
I0627 16:53:52.769732 22505 tagbuilder.go:95] tags: [_aws _k8s_1_6]
I0627 16:53:52.770459 22505 options_loader.go:130] executing builder *components.DefaultsOptionsBuilder
I0627 16:53:52.770818 22505 options_loader.go:130] executing builder *components.EtcdOptionsBuilder
I0627 16:53:52.771160 22505 options_loader.go:130] executing builder *etcdmanager.EtcdManagerOptionsBuilder
I0627 16:53:52.771528 22505 options_loader.go:130] executing builder *nodeauthorizer.OptionsBuilder
I0627 16:53:52.771956 22505 options_loader.go:130] executing builder *components.KubeAPIServerOptionsBuilder
I0627 16:53:52.772303 22505 options_loader.go:130] executing builder *components.DockerOptionsBuilder
I0627 16:53:52.772674 22505 options_loader.go:130] executing builder *components.NetworkingOptionsBuilder
I0627 16:53:52.772958 22505 options_loader.go:130] executing builder *components.KubeDnsOptionsBuilder
I0627 16:53:52.773287 22505 options_loader.go:130] executing builder *components.KubeletOptionsBuilder
I0627 16:53:52.773566 22505 kubelet.go:171] Cloud Provider: aws
I0627 16:53:52.773869 22505 options_loader.go:130] executing builder *components.KubeControllerManagerOptionsBuilder
I0627 16:53:52.774088 22505 kubecontrollermanager.go:74] Kubernetes version "1.12.9" supports AttachDetachReconcileSyncPeriod; will configure
I0627 16:53:52.774444 22505 kubecontrollermanager.go:79] AttachDetachReconcileSyncPeriod is not set; will set to default 1m0s
I0627 16:53:52.774691 22505 options_loader.go:130] executing builder *components.KubeSchedulerOptionsBuilder
I0627 16:53:52.774890 22505 options_loader.go:130] executing builder *components.KubeProxyOptionsBuilder
I0627 16:53:52.775519 22505 options_loader.go:130] executing builder *components.DefaultsOptionsBuilder
I0627 16:53:52.775771 22505 options_loader.go:130] executing builder *components.EtcdOptionsBuilder
I0627 16:53:52.776000 22505 options_loader.go:130] executing builder *etcdmanager.EtcdManagerOptionsBuilder
I0627 16:53:52.776249 22505 options_loader.go:130] executing builder *nodeauthorizer.OptionsBuilder
I0627 16:53:52.776508 22505 options_loader.go:130] executing builder *components.KubeAPIServerOptionsBuilder
I0627 16:53:52.776746 22505 options_loader.go:130] executing builder *components.DockerOptionsBuilder
I0627 16:53:52.776992 22505 options_loader.go:130] executing builder *components.NetworkingOptionsBuilder
I0627 16:53:52.777218 22505 options_loader.go:130] executing builder *components.KubeDnsOptionsBuilder
I0627 16:53:52.777482 22505 options_loader.go:130] executing builder *components.KubeletOptionsBuilder
I0627 16:53:52.777740 22505 kubelet.go:171] Cloud Provider: aws
I0627 16:53:52.778043 22505 options_loader.go:130] executing builder *components.KubeControllerManagerOptionsBuilder
I0627 16:53:52.778402 22505 kubecontrollermanager.go:74] Kubernetes version "1.12.9" supports AttachDetachReconcileSyncPeriod; will configure
I0627 16:53:52.781266 22505 options_loader.go:130] executing builder *components.KubeSchedulerOptionsBuilder
I0627 16:53:52.781633 22505 options_loader.go:130] executing builder *components.KubeProxyOptionsBuilder
I0627 16:53:52.782449 22505 spec_builder.go:49] options: {
"channel": "stable",
"configBase": "s3://my-state-store/dev-qa.k8s.domain.com",
"cloudProvider": "aws",
"kubernetesVersion": "1.12.9",
"subnets": [
{
"name": "eu-central-1a",
"cidr": "172.31.239.32/27",
"zone": "eu-central-1a",
"type": "Private"
},
{
"name": "eu-central-1b",
"cidr": "172.31.239.64/27",
"zone": "eu-central-1b",
"type": "Private"
},
{
"name": "eu-central-1c",
"cidr": "172.31.239.96/27",
"zone": "eu-central-1c",
"type": "Private"
},
{
"name": "utility-eu-central-1a",
"cidr": "172.31.239.0/30",
"zone": "eu-central-1a",
"type": "Utility"
},
{
"name": "utility-eu-central-1b",
"cidr": "172.31.239.4/30",
"zone": "eu-central-1b",
"type": "Utility"
},
{
"name": "utility-eu-central-1c",
"cidr": "172.31.239.8/30",
"zone": "eu-central-1c",
"type": "Utility"
}
],
"masterPublicName": "api.dev-qa.k8s.domain.com",
"masterInternalName": "api.internal.dev-qa.k8s.domain.com",
"networkCIDR": "172.31.239.0/24",
"topology": {
"masters": "private",
"nodes": "private",
"bastion": {
"bastionPublicName": "bastion.dev-qa.k8s.domain.com"
},
"dns": {
"type": "Public"
}
},
"secretStore": "s3://my-state-store/dev-qa.k8s.domain.com/secrets",
"keyStore": "s3://my-state-store/dev-qa.k8s.domain.com/pki",
"configStore": "s3://my-state-store/dev-qa.k8s.domain.com",
"dnsZone": "k8s.domain.com",
"clusterDNSDomain": "cluster.local",
"serviceClusterIPRange": "100.64.0.0/13",
"nonMasqueradeCIDR": "100.64.0.0/10",
"sshAccess": [
"0.0.0.0/0"
],
"kubernetesApiAccess": [
"0.0.0.0/0"
],
"etcdClusters": [
{
"name": "main",
"provider": "Manager",
"etcdMembers": [
{
"name": "a",
"instanceGroup": "master-eu-central-1a",
"encryptedVolume": true
},
{
"name": "b",
"instanceGroup": "master-eu-central-1b",
"encryptedVolume": true
},
{
"name": "c",
"instanceGroup": "master-eu-central-1c",
"encryptedVolume": true
}
],
"enableEtcdTLS": true,
"enableTLSAuth": true,
"version": "3.2.24",
"backups": {
"backupStore": "s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main"
},
"manager": {},
"memoryRequest": "100Mi",
"cpuRequest": "200m"
},
{
"name": "events",
"provider": "Manager",
"etcdMembers": [
{
"name": "a",
"instanceGroup": "master-eu-central-1a",
"encryptedVolume": true
},
{
"name": "b",
"instanceGroup": "master-eu-central-1b",
"encryptedVolume": true
},
{
"name": "c",
"instanceGroup": "master-eu-central-1c",
"encryptedVolume": true
}
],
"enableEtcdTLS": true,
"enableTLSAuth": true,
"version": "3.2.24",
"backups": {
"backupStore": "s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events"
},
"manager": {},
"memoryRequest": "100Mi",
"cpuRequest": "100m"
}
],
"docker": {
"ipMasq": false,
"ipTables": false,
"logDriver": "json-file",
"logLevel": "warn",
"logOpt": [
"max-size=10m",
"max-file=5"
],
"storage": "overlay2,overlay,aufs",
"version": "18.06.3"
},
"kubeDNS": {
"cacheMaxSize": 1000,
"cacheMaxConcurrent": 150,
"domain": "cluster.local",
"replicas": 2,
"serverIP": "100.64.0.10",
"memoryRequest": "70Mi",
"cpuRequest": "100m",
"memoryLimit": "170Mi"
},
"kubeAPIServer": {
"image": "k8s.gcr.io/kube-apiserver:v1.12.9",
"logLevel": 2,
"cloudProvider": "aws",
"securePort": 443,
"insecurePort": 8080,
"bindAddress": "0.0.0.0",
"insecureBindAddress": "127.0.0.1",
"enableAdmissionPlugins": [
"NamespaceLifecycle",
"LimitRanger",
"ServiceAccount",
"PersistentVolumeLabel",
"DefaultStorageClass",
"DefaultTolerationSeconds",
"MutatingAdmissionWebhook",
"ValidatingAdmissionWebhook",
"NodeRestriction",
"ResourceQuota"
],
"serviceClusterIPRange": "100.64.0.0/13",
"etcdServers": [
"http://127.0.0.1:4001"
],
"etcdServersOverrides": [
"/events#http://127.0.0.1:4002"
],
"allowPrivileged": true,
"apiServerCount": 3,
"anonymousAuth": false,
"kubeletPreferredAddressTypes": [
"InternalIP",
"Hostname",
"ExternalIP"
],
"storageBackend": "etcd3",
"authorizationMode": "RBAC",
"requestheaderUsernameHeaders": [
"X-Remote-User"
],
"requestheaderGroupHeaders": [
"X-Remote-Group"
],
"requestheaderExtraHeaderPrefixes": [
"X-Remote-Extra-"
],
"requestheaderAllowedNames": [
"aggregator"
]
},
"kubeControllerManager": {
"logLevel": 2,
"image": "k8s.gcr.io/kube-controller-manager:v1.12.9",
"cloudProvider": "aws",
"clusterName": "dev-qa.k8s.domain.com",
"clusterCIDR": "100.96.0.0/11",
"allocateNodeCIDRs": true,
"configureCloudRoutes": false,
"leaderElection": {
"leaderElect": true
},
"attachDetachReconcileSyncPeriod": "1m0s",
"useServiceAccountCredentials": true
},
"kubeScheduler": {
"logLevel": 2,
"image": "k8s.gcr.io/kube-scheduler:v1.12.9",
"leaderElection": {
"leaderElect": true
}
},
"kubeProxy": {
"image": "k8s.gcr.io/kube-proxy:v1.12.9",
"cpuRequest": "100m",
"logLevel": 2,
"clusterCIDR": "100.96.0.0/11",
"hostnameOverride": "@aws"
},
"kubelet": {
"anonymousAuth": false,
"kubeconfigPath": "/var/lib/kubelet/kubeconfig",
"logLevel": 2,
"podManifestPath": "/etc/kubernetes/manifests",
"hostnameOverride": "@aws",
"podInfraContainerImage": "k8s.gcr.io/pause-amd64:3.0",
"allowPrivileged": true,
"enableDebuggingHandlers": true,
"clusterDomain": "cluster.local",
"clusterDNS": "100.64.0.10",
"networkPluginName": "cni",
"cloudProvider": "aws",
"cgroupRoot": "/",
"nonMasqueradeCIDR": "100.64.0.0/10",
"evictionHard": "memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%",
"featureGates": {
"ExperimentalCriticalPodAnnotation": "true"
}
},
"masterKubelet": {
"anonymousAuth": false,
"kubeconfigPath": "/var/lib/kubelet/kubeconfig",
"logLevel": 2,
"podManifestPath": "/etc/kubernetes/manifests",
"hostnameOverride": "@aws",
"podInfraContainerImage": "k8s.gcr.io/pause-amd64:3.0",
"allowPrivileged": true,
"enableDebuggingHandlers": true,
"clusterDomain": "cluster.local",
"clusterDNS": "100.64.0.10",
"networkPluginName": "cni",
"cloudProvider": "aws",
"cgroupRoot": "/",
"registerSchedulable": false,
"nonMasqueradeCIDR": "100.64.0.0/10",
"evictionHard": "memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%",
"featureGates": {
"ExperimentalCriticalPodAnnotation": "true"
}
},
"networking": {
"calico": {
"majorVersion": "v3"
}
},
"api": {
"loadBalancer": {
"type": "Public"
}
},
"authorization": {
"rbac": {}
},
"iam": {
"legacy": false,
"allowContainerRegistry": true
}
}
I0627 16:53:52.844337 22505 channel.go:239] version range ">=1.14.0-alpha.1" does not apply to version "1.12.1"; skipping
I0627 16:53:52.844741 22505 channel.go:239] version range ">=1.13.0-alpha.1" does not apply to version "1.12.1"; skipping
I0627 16:53:52.845071 22505 channel.go:162] RecommendedVersion="1.12.1", Have="1.12.1". No upgrade needed.
I0627 16:53:52.845490 22505 channel.go:190] VersionRecommendationSpec does not specify RequiredVersion
I0627 16:53:52.845818 22505 channel.go:218] version range ">=1.14.0" does not apply to version "1.12.9"; skipping
I0627 16:53:52.846134 22505 channel.go:218] version range ">=1.13.0" does not apply to version "1.12.9"; skipping
I0627 16:53:52.846466 22505 channel.go:142] RecommendedVersion="1.12.8", Have="1.12.9". No upgrade needed.
I0627 16:53:52.846685 22505 channel.go:182] RequiredVersion="1.12.0", Have="1.12.9". No upgrade needed.
I0627 16:53:52.847074 22505 aws.go:99] machineType m5a.2xlarge requires an image based on stretch to operate. Trying to check compatibility
I0627 16:53:52.847330 22505 context.go:159] Performing HTTP request: GET https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubelet.sha1
I0627 16:53:53.413250 22505 builder.go:301] Found hash "e914b17532c411cb7c0cc472131b61935fb66b31" for "https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubelet"
I0627 16:53:53.413948 22505 builder.go:234] adding file: &{FileURL:https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubelet CanonicalFileURL:<nil> SHAValue:e914b17532c411cb7c0cc472131b61935fb66b31}
I0627 16:53:53.414408 22505 context.go:159] Performing HTTP request: GET https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubectl.sha1
I0627 16:53:53.418799 22505 builder.go:301] Found hash "aa3e93897a6999d6c7dedbc41793c90d41eeb000" for "https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubectl"
I0627 16:53:53.419217 22505 builder.go:234] adding file: &{FileURL:https://storage.googleapis.com/kubernetes-release/release/v1.12.9/bin/linux/amd64/kubectl CanonicalFileURL:<nil> SHAValue:aa3e93897a6999d6c7dedbc41793c90d41eeb000}
I0627 16:53:53.419703 22505 networking.go:168] Adding default CNI asset for k8s >= 1.11: https://storage.googleapis.com/kubernetes-release/network-plugins/cni-plugins-amd64-v0.6.0.tgz
I0627 16:53:53.420162 22505 urls.go:81] Using default base url: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/"
I0627 16:53:53.420512 22505 context.go:159] Performing HTTP request: GET https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/utils.tar.gz.sha1
I0627 16:53:54.917070 22505 builder.go:301] Found hash "ad4085c05ff02c241a38ff0033d76c699316f298" for "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/utils.tar.gz"
I0627 16:53:54.917961 22505 builder.go:234] adding file: &{FileURL:https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/utils.tar.gz CanonicalFileURL:<nil> SHAValue:ad4085c05ff02c241a38ff0033d76c699316f298}
I0627 16:53:54.918442 22505 urls.go:73] Using cached kopsBaseUrl url: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/"
I0627 16:53:54.918770 22505 context.go:159] Performing HTTP request: GET https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/nodeup.sha1
I0627 16:53:55.158845 22505 builder.go:301] Found hash "8ff9c81e668969bfaf8fefb5c61f0b9fddc8ab44" for "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/nodeup"
I0627 16:53:55.159575 22505 builder.go:234] adding file: &{FileURL:https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/nodeup CanonicalFileURL:<nil> SHAValue:8ff9c81e668969bfaf8fefb5c61f0b9fddc8ab44}
I0627 16:53:55.160026 22505 urls.go:135] Using default nodeup location: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/linux/amd64/nodeup"
I0627 16:53:55.160435 22505 urls.go:73] Using cached kopsBaseUrl url: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/"
I0627 16:53:55.160932 22505 context.go:159] Performing HTTP request: GET https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz.sha1
I0627 16:53:55.436799 22505 builder.go:301] Found hash "175d2d9df2b9e317a40749a6d735577546a3c38d" for "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz"
I0627 16:53:55.437491 22505 builder.go:234] adding file: &{FileURL:https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz CanonicalFileURL:<nil> SHAValue:175d2d9df2b9e317a40749a6d735577546a3c38d}
I0627 16:53:55.438011 22505 urls.go:172] Using default protokube location: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz"
I0627 16:53:55.438472 22505 aws_cloud.go:1212] Querying EC2 for all valid zones in region "eu-central-1"
I0627 16:53:55.438890 22505 request_logger.go:45] AWS request: ec2/DescribeAvailabilityZones
I0627 16:53:55.644787 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/ssh/public/admin/"
I0627 16:53:55.835784 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/ssh/public/admin: [s3://my-state-store/dev-qa.k8s.domain.com/pki/ssh/public/admin/e4d52e2ca78fef469afe377a974f975b]
I0627 16:53:55.836435 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/ssh/public/admin/e4d52e2ca78fef469afe377a974f975b"
I0627 16:53:56.034283 22505 route53.go:53] AWS request: route53 ListHostedZones
I0627 16:53:57.348876 22505 dns.go:102] Doing DNS lookup to verify NS records for "k8s.domain.com"
I0627 16:53:57.701856 22505 dns.go:119] Found NS records for "k8s.domain.com": [ns-298.awsdns-37.com. ns-608.awsdns-12.net. ns-1227.awsdns-25.org. ns-1578.awsdns-05.co.uk.]
I0627 16:53:57.702294 22505 tagbuilder.go:95] tags: [_aws _k8s_1_6]
I0627 16:53:57.702785 22505 templates.go:80] loading (templated) resource "addons/dns-controller.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.703197 22505 templates.go:80] loading (templated) resource "addons/dns-controller.addons.k8s.io/k8s-1.6.yaml"
I0627 16:53:57.703560 22505 templates.go:80] loading (templated) resource "addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:53:57.703903 22505 templates.go:80] loading (templated) resource "addons/external-dns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.704259 22505 templates.go:80] loading (templated) resource "addons/external-dns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:53:57.704559 22505 templates.go:80] loading (templated) resource "addons/external-dns.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:53:57.705007 22505 templates.go:88] loading resource "addons/external-dns.addons.k8s.io/README.md"
I0627 16:53:57.705391 22505 templates.go:80] loading (templated) resource "addons/networking.romana/k8s-1.12.yaml"
I0627 16:53:57.705663 22505 templates.go:80] loading (templated) resource "addons/networking.romana/k8s-1.7.yaml"
I0627 16:53:57.705945 22505 templates.go:88] loading resource "addons/scheduler.addons.k8s.io/v1.7.0.yaml"
I0627 16:53:57.706194 22505 templates.go:80] loading (templated) resource "addons/spotinst-kubernetes-cluster-controller.addons.k8s.io/v1.8.0.yaml"
I0627 16:53:57.706451 22505 templates.go:80] loading (templated) resource "addons/spotinst-kubernetes-cluster-controller.addons.k8s.io/v1.9.0.yaml"
I0627 16:53:57.706700 22505 templates.go:80] loading (templated) resource "addons/digitalocean-cloud-controller.addons.k8s.io/k8s-1.8.yaml"
I0627 16:53:57.706955 22505 templates.go:80] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.10.yaml"
I0627 16:53:57.707199 22505 templates.go:80] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.12.yaml"
I0627 16:53:57.707458 22505 templates.go:80] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.7.yaml"
I0627 16:53:57.707704 22505 templates.go:80] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.8.yaml"
I0627 16:53:57.707916 22505 templates.go:88] loading resource "addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml"
I0627 16:53:57.708165 22505 templates.go:80] loading (templated) resource "addons/node-authorizer.addons.k8s.io/k8s-1.10.yaml"
I0627 16:53:57.708411 22505 templates.go:80] loading (templated) resource "addons/node-authorizer.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.708655 22505 templates.go:80] loading (templated) resource "addons/openstack.addons.k8s.io/k8s-1.11.yaml"
I0627 16:53:57.708862 22505 templates.go:88] loading resource "addons/storage-aws.addons.k8s.io/v1.6.0.yaml"
I0627 16:53:57.709077 22505 templates.go:88] loading resource "addons/storage-aws.addons.k8s.io/v1.7.0.yaml"
I0627 16:53:57.709277 22505 templates.go:88] loading resource "addons/storage-gce.addons.k8s.io/v1.6.0.yaml"
I0627 16:53:57.709534 22505 templates.go:88] loading resource "addons/storage-gce.addons.k8s.io/v1.7.0.yaml"
I0627 16:53:57.709766 22505 templates.go:88] loading resource "addons/authentication.kope.io/k8s-1.12.yaml"
I0627 16:53:57.710040 22505 templates.go:88] loading resource "addons/authentication.kope.io/k8s-1.8.yaml"
I0627 16:53:57.710439 22505 templates.go:88] loading resource "addons/networking.kope.io/k8s-1.12.yaml"
I0627 16:53:57.714010 22505 templates.go:88] loading resource "addons/networking.kope.io/k8s-1.6.yaml"
I0627 16:53:57.714388 22505 templates.go:88] loading resource "addons/networking.kope.io/pre-k8s-1.6.yaml"
I0627 16:53:57.714653 22505 templates.go:88] loading resource "addons/rbac.addons.k8s.io/k8s-1.8.yaml"
I0627 16:53:57.715062 22505 templates.go:80] loading (templated) resource "addons/networking.cilium.io/k8s-1.12.yaml"
I0627 16:53:57.715428 22505 templates.go:80] loading (templated) resource "addons/networking.cilium.io/k8s-1.7.yaml"
I0627 16:53:57.715707 22505 templates.go:80] loading (templated) resource "addons/networking.flannel/k8s-1.12.yaml"
I0627 16:53:57.715983 22505 templates.go:80] loading (templated) resource "addons/networking.flannel/k8s-1.6.yaml"
I0627 16:53:57.716254 22505 templates.go:80] loading (templated) resource "addons/networking.flannel/pre-k8s-1.6.yaml"
I0627 16:53:57.716721 22505 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.7.yaml"
I0627 16:53:57.717081 22505 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org/pre-k8s-1.6.yaml"
I0627 16:53:57.717527 22505 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.12.yaml"
I0627 16:53:57.717929 22505 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.6.yaml"
I0627 16:53:57.718306 22505 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.7-v3.yaml"
I0627 16:53:57.718693 22505 templates.go:80] loading (templated) resource "addons/coredns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.719033 22505 templates.go:80] loading (templated) resource "addons/coredns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:53:57.719300 22505 templates.go:88] loading resource "addons/limit-range.addons.k8s.io/addon.yaml"
I0627 16:53:57.719612 22505 templates.go:88] loading resource "addons/limit-range.addons.k8s.io/v1.5.0.yaml"
I0627 16:53:57.719901 22505 templates.go:80] loading (templated) resource "addons/networking.kuberouter/k8s-1.12.yaml"
I0627 16:53:57.720242 22505 templates.go:80] loading (templated) resource "addons/networking.kuberouter/k8s-1.6.yaml"
I0627 16:53:57.720699 22505 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.12.yaml"
I0627 16:53:57.721278 22505 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.6.yaml"
I0627 16:53:57.721747 22505 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.8.yaml"
I0627 16:53:57.722247 22505 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.9.yaml"
I0627 16:53:57.722631 22505 templates.go:80] loading (templated) resource "addons/networking.projectcalico.org.canal/pre-k8s-1.6.yaml"
I0627 16:53:57.724710 22505 templates.go:80] loading (templated) resource "addons/podsecuritypolicy.addons.k8s.io/k8s-1.10.yaml"
I0627 16:53:57.725071 22505 templates.go:80] loading (templated) resource "addons/podsecuritypolicy.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.725360 22505 templates.go:80] loading (templated) resource "addons/podsecuritypolicy.addons.k8s.io/k8s-1.9.yaml"
I0627 16:53:57.725721 22505 templates.go:80] loading (templated) resource "addons/authentication.aws/k8s-1.12.yaml"
I0627 16:53:57.725968 22505 templates.go:80] loading (templated) resource "addons/authentication.aws/k8s-1.10.yaml"
I0627 16:53:57.726306 22505 templates.go:88] loading resource "addons/core.addons.k8s.io/addon.yaml"
I0627 16:53:57.726573 22505 templates.go:80] loading (templated) resource "addons/core.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.726982 22505 templates.go:80] loading (templated) resource "addons/core.addons.k8s.io/k8s-1.7.yaml"
I0627 16:53:57.727321 22505 templates.go:88] loading resource "addons/core.addons.k8s.io/v1.4.0.yaml"
I0627 16:53:57.727804 22505 templates.go:80] loading (templated) resource "addons/kube-dns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.728275 22505 templates.go:80] loading (templated) resource "addons/kube-dns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:53:57.728784 22505 templates.go:80] loading (templated) resource "addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:53:57.729340 22505 templates.go:80] loading (templated) resource "addons/networking.weave/k8s-1.7.yaml"
I0627 16:53:57.730411 22505 templates.go:80] loading (templated) resource "addons/networking.weave/k8s-1.8.yaml"
I0627 16:53:57.731397 22505 templates.go:80] loading (templated) resource "addons/networking.weave/pre-k8s-1.6.yaml"
I0627 16:53:57.732032 22505 templates.go:80] loading (templated) resource "addons/networking.weave/k8s-1.12.yaml"
I0627 16:53:57.732421 22505 templates.go:80] loading (templated) resource "addons/networking.weave/k8s-1.6.yaml"
I0627 16:53:57.732816 22505 tree_walker.go:98] visit "cloudup/resources"
I0627 16:53:57.733037 22505 tree_walker.go:98] visit "cloudup/resources/addons"
I0627 16:53:57.733262 22505 tree_walker.go:98] visit "cloudup/resources/addons/authentication.aws"
I0627 16:53:57.735142 22505 tree_walker.go:98] visit "cloudup/resources/addons/authentication.aws/k8s-1.10.yaml.template"
I0627 16:53:57.735469 22505 loader.go:354] loading (templated) resource "addons/authentication.aws/k8s-1.10.yaml"
I0627 16:53:57.735694 22505 tree_walker.go:98] visit "cloudup/resources/addons/authentication.aws/k8s-1.12.yaml.template"
I0627 16:53:57.735958 22505 loader.go:354] loading (templated) resource "addons/authentication.aws/k8s-1.12.yaml"
I0627 16:53:57.736176 22505 tree_walker.go:98] visit "cloudup/resources/addons/core.addons.k8s.io"
I0627 16:53:57.736620 22505 tree_walker.go:98] visit "cloudup/resources/addons/core.addons.k8s.io/addon.yaml"
I0627 16:53:57.737187 22505 loader.go:362] loading resource "addons/core.addons.k8s.io/addon.yaml"
I0627 16:53:57.737524 22505 tree_walker.go:98] visit "cloudup/resources/addons/core.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:53:57.737938 22505 loader.go:354] loading (templated) resource "addons/core.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.738171 22505 tree_walker.go:98] visit "cloudup/resources/addons/core.addons.k8s.io/k8s-1.7.yaml.template"
I0627 16:53:57.738485 22505 loader.go:354] loading (templated) resource "addons/core.addons.k8s.io/k8s-1.7.yaml"
I0627 16:53:57.738715 22505 tree_walker.go:98] visit "cloudup/resources/addons/core.addons.k8s.io/v1.4.0.yaml"
I0627 16:53:57.738964 22505 loader.go:362] loading resource "addons/core.addons.k8s.io/v1.4.0.yaml"
I0627 16:53:57.739184 22505 tree_walker.go:98] visit "cloudup/resources/addons/kube-dns.addons.k8s.io"
I0627 16:53:57.739517 22505 tree_walker.go:98] visit "cloudup/resources/addons/kube-dns.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:53:57.739894 22505 loader.go:354] loading (templated) resource "addons/kube-dns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.740132 22505 tree_walker.go:98] visit "cloudup/resources/addons/kube-dns.addons.k8s.io/k8s-1.6.yaml.template"
I0627 16:53:57.740532 22505 loader.go:354] loading (templated) resource "addons/kube-dns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:53:57.740779 22505 tree_walker.go:98] visit "cloudup/resources/addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml.template"
I0627 16:53:57.741133 22505 loader.go:354] loading (templated) resource "addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:53:57.741509 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.weave"
I0627 16:53:57.742191 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.weave/k8s-1.12.yaml.template"
I0627 16:53:57.742712 22505 loader.go:354] loading (templated) resource "addons/networking.weave/k8s-1.12.yaml"
I0627 16:53:57.742908 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.weave/k8s-1.6.yaml.template"
I0627 16:53:57.743168 22505 loader.go:354] loading (templated) resource "addons/networking.weave/k8s-1.6.yaml"
I0627 16:53:57.749545 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.weave/k8s-1.7.yaml.template"
I0627 16:53:57.750002 22505 loader.go:354] loading (templated) resource "addons/networking.weave/k8s-1.7.yaml"
I0627 16:53:57.750283 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.weave/k8s-1.8.yaml.template"
I0627 16:53:57.750702 22505 loader.go:354] loading (templated) resource "addons/networking.weave/k8s-1.8.yaml"
I0627 16:53:57.751005 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.weave/pre-k8s-1.6.yaml.template"
I0627 16:53:57.751367 22505 loader.go:354] loading (templated) resource "addons/networking.weave/pre-k8s-1.6.yaml"
I0627 16:53:57.751784 22505 tree_walker.go:98] visit "cloudup/resources/addons/dns-controller.addons.k8s.io"
I0627 16:53:57.752174 22505 tree_walker.go:98] visit "cloudup/resources/addons/dns-controller.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:53:57.752576 22505 loader.go:354] loading (templated) resource "addons/dns-controller.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.752891 22505 tree_walker.go:98] visit "cloudup/resources/addons/dns-controller.addons.k8s.io/k8s-1.6.yaml.template"
I0627 16:53:57.753272 22505 loader.go:354] loading (templated) resource "addons/dns-controller.addons.k8s.io/k8s-1.6.yaml"
I0627 16:53:57.753612 22505 tree_walker.go:98] visit "cloudup/resources/addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml.template"
I0627 16:53:57.754411 22505 loader.go:354] loading (templated) resource "addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:53:57.754843 22505 tree_walker.go:98] visit "cloudup/resources/addons/external-dns.addons.k8s.io"
I0627 16:53:57.755198 22505 tree_walker.go:98] visit "cloudup/resources/addons/external-dns.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:53:57.755606 22505 loader.go:354] loading (templated) resource "addons/external-dns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.755844 22505 tree_walker.go:98] visit "cloudup/resources/addons/external-dns.addons.k8s.io/k8s-1.6.yaml.template"
I0627 16:53:57.756150 22505 loader.go:354] loading (templated) resource "addons/external-dns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:53:57.756405 22505 tree_walker.go:98] visit "cloudup/resources/addons/external-dns.addons.k8s.io/pre-k8s-1.6.yaml.template"
I0627 16:53:57.756631 22505 loader.go:354] loading (templated) resource "addons/external-dns.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:53:57.756813 22505 tree_walker.go:98] visit "cloudup/resources/addons/external-dns.addons.k8s.io/README.md"
I0627 16:53:57.757061 22505 loader.go:362] loading resource "addons/external-dns.addons.k8s.io/README.md"
I0627 16:53:57.757233 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.romana"
I0627 16:53:57.757455 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.romana/k8s-1.12.yaml.template"
I0627 16:53:57.757750 22505 loader.go:354] loading (templated) resource "addons/networking.romana/k8s-1.12.yaml"
I0627 16:53:57.757940 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.romana/k8s-1.7.yaml.template"
I0627 16:53:57.758210 22505 loader.go:354] loading (templated) resource "addons/networking.romana/k8s-1.7.yaml"
I0627 16:53:57.761612 22505 tree_walker.go:98] visit "cloudup/resources/addons/scheduler.addons.k8s.io"
I0627 16:53:57.762098 22505 tree_walker.go:98] visit "cloudup/resources/addons/scheduler.addons.k8s.io/v1.7.0.yaml"
I0627 16:53:57.762572 22505 loader.go:362] loading resource "addons/scheduler.addons.k8s.io/v1.7.0.yaml"
I0627 16:53:57.763159 22505 tree_walker.go:98] visit "cloudup/resources/addons/spotinst-kubernetes-cluster-controller.addons.k8s.io"
I0627 16:53:57.763765 22505 tree_walker.go:98] visit "cloudup/resources/addons/spotinst-kubernetes-cluster-controller.addons.k8s.io/v1.8.0.yaml.template"
I0627 16:53:57.764330 22505 loader.go:354] loading (templated) resource "addons/spotinst-kubernetes-cluster-controller.addons.k8s.io/v1.8.0.yaml"
I0627 16:53:57.764755 22505 tree_walker.go:98] visit "cloudup/resources/addons/spotinst-kubernetes-cluster-controller.addons.k8s.io/v1.9.0.yaml.template"
I0627 16:53:57.765252 22505 loader.go:354] loading (templated) resource "addons/spotinst-kubernetes-cluster-controller.addons.k8s.io/v1.9.0.yaml"
I0627 16:53:57.765708 22505 tree_walker.go:98] visit "cloudup/resources/addons/digitalocean-cloud-controller.addons.k8s.io"
I0627 16:53:57.766076 22505 tree_walker.go:98] visit "cloudup/resources/addons/digitalocean-cloud-controller.addons.k8s.io/k8s-1.8.yaml.template"
I0627 16:53:57.766509 22505 loader.go:354] loading (templated) resource "addons/digitalocean-cloud-controller.addons.k8s.io/k8s-1.8.yaml"
I0627 16:53:57.766853 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.amazon-vpc-routed-eni"
I0627 16:53:57.767169 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.amazon-vpc-routed-eni/k8s-1.8.yaml.template"
I0627 16:53:57.767561 22505 loader.go:354] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.8.yaml"
I0627 16:53:57.767867 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.amazon-vpc-routed-eni/k8s-1.10.yaml.template"
I0627 16:53:57.768250 22505 loader.go:354] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.10.yaml"
I0627 16:53:57.768550 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.amazon-vpc-routed-eni/k8s-1.12.yaml.template"
I0627 16:53:57.768837 22505 loader.go:354] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.12.yaml"
I0627 16:53:57.769066 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.amazon-vpc-routed-eni/k8s-1.7.yaml.template"
I0627 16:53:57.770809 22505 loader.go:354] loading (templated) resource "addons/networking.amazon-vpc-routed-eni/k8s-1.7.yaml"
I0627 16:53:57.771859 22505 tree_walker.go:98] visit "cloudup/resources/addons/kubelet-api.rbac.addons.k8s.io"
I0627 16:53:57.772283 22505 tree_walker.go:98] visit "cloudup/resources/addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml"
I0627 16:53:57.772759 22505 loader.go:362] loading resource "addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml"
I0627 16:53:57.773087 22505 tree_walker.go:98] visit "cloudup/resources/addons/node-authorizer.addons.k8s.io"
I0627 16:53:57.773431 22505 tree_walker.go:98] visit "cloudup/resources/addons/node-authorizer.addons.k8s.io/k8s-1.10.yaml.template"
I0627 16:53:57.773875 22505 loader.go:354] loading (templated) resource "addons/node-authorizer.addons.k8s.io/k8s-1.10.yaml"
I0627 16:53:57.774186 22505 tree_walker.go:98] visit "cloudup/resources/addons/node-authorizer.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:53:57.774559 22505 loader.go:354] loading (templated) resource "addons/node-authorizer.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.774874 22505 tree_walker.go:98] visit "cloudup/resources/addons/openstack.addons.k8s.io"
I0627 16:53:57.775157 22505 tree_walker.go:98] visit "cloudup/resources/addons/openstack.addons.k8s.io/k8s-1.11.yaml.template"
I0627 16:53:57.775492 22505 loader.go:354] loading (templated) resource "addons/openstack.addons.k8s.io/k8s-1.11.yaml"
I0627 16:53:57.775807 22505 tree_walker.go:98] visit "cloudup/resources/addons/storage-aws.addons.k8s.io"
I0627 16:53:57.776091 22505 tree_walker.go:98] visit "cloudup/resources/addons/storage-aws.addons.k8s.io/v1.6.0.yaml"
I0627 16:53:57.776348 22505 loader.go:362] loading resource "addons/storage-aws.addons.k8s.io/v1.6.0.yaml"
I0627 16:53:57.776647 22505 tree_walker.go:98] visit "cloudup/resources/addons/storage-aws.addons.k8s.io/v1.7.0.yaml"
I0627 16:53:57.777237 22505 loader.go:362] loading resource "addons/storage-aws.addons.k8s.io/v1.7.0.yaml"
I0627 16:53:57.777768 22505 tree_walker.go:98] visit "cloudup/resources/addons/storage-gce.addons.k8s.io"
I0627 16:53:57.778276 22505 tree_walker.go:98] visit "cloudup/resources/addons/storage-gce.addons.k8s.io/v1.7.0.yaml"
I0627 16:53:57.778665 22505 loader.go:362] loading resource "addons/storage-gce.addons.k8s.io/v1.7.0.yaml"
I0627 16:53:57.778988 22505 tree_walker.go:98] visit "cloudup/resources/addons/storage-gce.addons.k8s.io/v1.6.0.yaml"
I0627 16:53:57.779381 22505 loader.go:362] loading resource "addons/storage-gce.addons.k8s.io/v1.6.0.yaml"
I0627 16:53:57.779676 22505 tree_walker.go:98] visit "cloudup/resources/addons/authentication.kope.io"
I0627 16:53:57.783479 22505 tree_walker.go:98] visit "cloudup/resources/addons/authentication.kope.io/k8s-1.12.yaml"
I0627 16:53:57.783896 22505 loader.go:362] loading resource "addons/authentication.kope.io/k8s-1.12.yaml"
I0627 16:53:57.784268 22505 tree_walker.go:98] visit "cloudup/resources/addons/authentication.kope.io/k8s-1.8.yaml"
I0627 16:53:57.784760 22505 loader.go:362] loading resource "addons/authentication.kope.io/k8s-1.8.yaml"
I0627 16:53:57.785121 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.kope.io"
I0627 16:53:57.785499 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.kope.io/k8s-1.12.yaml"
I0627 16:53:57.786040 22505 loader.go:362] loading resource "addons/networking.kope.io/k8s-1.12.yaml"
I0627 16:53:57.786444 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.kope.io/k8s-1.6.yaml"
I0627 16:53:57.786827 22505 loader.go:362] loading resource "addons/networking.kope.io/k8s-1.6.yaml"
I0627 16:53:57.787141 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.kope.io/pre-k8s-1.6.yaml"
I0627 16:53:57.787495 22505 loader.go:362] loading resource "addons/networking.kope.io/pre-k8s-1.6.yaml"
I0627 16:53:57.787785 22505 tree_walker.go:98] visit "cloudup/resources/addons/rbac.addons.k8s.io"
I0627 16:53:57.788093 22505 tree_walker.go:98] visit "cloudup/resources/addons/rbac.addons.k8s.io/k8s-1.8.yaml"
I0627 16:53:57.788485 22505 loader.go:362] loading resource "addons/rbac.addons.k8s.io/k8s-1.8.yaml"
I0627 16:53:57.788712 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.cilium.io"
I0627 16:53:57.788977 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.cilium.io/k8s-1.12.yaml.template"
I0627 16:53:57.789397 22505 loader.go:354] loading (templated) resource "addons/networking.cilium.io/k8s-1.12.yaml"
I0627 16:53:57.789605 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.cilium.io/k8s-1.7.yaml.template"
I0627 16:53:57.790070 22505 loader.go:354] loading (templated) resource "addons/networking.cilium.io/k8s-1.7.yaml"
I0627 16:53:57.790392 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.flannel"
I0627 16:53:57.790721 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.flannel/pre-k8s-1.6.yaml.template"
I0627 16:53:57.791235 22505 loader.go:354] loading (templated) resource "addons/networking.flannel/pre-k8s-1.6.yaml"
I0627 16:53:57.791537 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.flannel/k8s-1.12.yaml.template"
I0627 16:53:57.794626 22505 loader.go:354] loading (templated) resource "addons/networking.flannel/k8s-1.12.yaml"
I0627 16:53:57.795120 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.flannel/k8s-1.6.yaml.template"
I0627 16:53:57.795828 22505 loader.go:354] loading (templated) resource "addons/networking.flannel/k8s-1.6.yaml"
I0627 16:53:57.796862 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org"
I0627 16:53:57.797507 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org/k8s-1.12.yaml.template"
I0627 16:53:57.799054 22505 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.12.yaml"
I0627 16:53:57.799802 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org/k8s-1.6.yaml.template"
I0627 16:53:57.800319 22505 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.6.yaml"
I0627 16:53:57.800734 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org/k8s-1.7-v3.yaml.template"
I0627 16:53:57.801838 22505 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.7-v3.yaml"
I0627 16:53:57.802332 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org/k8s-1.7.yaml.template"
I0627 16:53:57.803415 22505 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org/k8s-1.7.yaml"
I0627 16:53:57.805126 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org/pre-k8s-1.6.yaml.template"
I0627 16:53:57.805621 22505 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org/pre-k8s-1.6.yaml"
I0627 16:53:57.806226 22505 tree_walker.go:98] visit "cloudup/resources/addons/coredns.addons.k8s.io"
I0627 16:53:57.806827 22505 tree_walker.go:98] visit "cloudup/resources/addons/coredns.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:53:57.807554 22505 loader.go:354] loading (templated) resource "addons/coredns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.808112 22505 tree_walker.go:98] visit "cloudup/resources/addons/coredns.addons.k8s.io/k8s-1.6.yaml.template"
I0627 16:53:57.808526 22505 loader.go:354] loading (templated) resource "addons/coredns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:53:57.808985 22505 tree_walker.go:98] visit "cloudup/resources/addons/limit-range.addons.k8s.io"
I0627 16:53:57.809433 22505 tree_walker.go:98] visit "cloudup/resources/addons/limit-range.addons.k8s.io/addon.yaml"
I0627 16:53:57.810215 22505 loader.go:362] loading resource "addons/limit-range.addons.k8s.io/addon.yaml"
I0627 16:53:57.810767 22505 tree_walker.go:98] visit "cloudup/resources/addons/limit-range.addons.k8s.io/v1.5.0.yaml"
I0627 16:53:57.811153 22505 loader.go:362] loading resource "addons/limit-range.addons.k8s.io/v1.5.0.yaml"
I0627 16:53:57.811416 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.kuberouter"
I0627 16:53:57.811795 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.kuberouter/k8s-1.12.yaml.template"
I0627 16:53:57.812331 22505 loader.go:354] loading (templated) resource "addons/networking.kuberouter/k8s-1.12.yaml"
I0627 16:53:57.813296 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.kuberouter/k8s-1.6.yaml.template"
I0627 16:53:57.813880 22505 loader.go:354] loading (templated) resource "addons/networking.kuberouter/k8s-1.6.yaml"
I0627 16:53:57.816518 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org.canal"
I0627 16:53:57.816861 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.12.yaml.template"
I0627 16:53:57.817504 22505 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.12.yaml"
I0627 16:53:57.817810 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.6.yaml.template"
I0627 16:53:57.818316 22505 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.6.yaml"
I0627 16:53:57.818689 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.8.yaml.template"
I0627 16:53:57.819235 22505 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.8.yaml"
I0627 16:53:57.819611 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org.canal/k8s-1.9.yaml.template"
I0627 16:53:57.820170 22505 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org.canal/k8s-1.9.yaml"
I0627 16:53:57.820600 22505 tree_walker.go:98] visit "cloudup/resources/addons/networking.projectcalico.org.canal/pre-k8s-1.6.yaml.template"
I0627 16:53:57.821207 22505 loader.go:354] loading (templated) resource "addons/networking.projectcalico.org.canal/pre-k8s-1.6.yaml"
I0627 16:53:57.821583 22505 tree_walker.go:98] visit "cloudup/resources/addons/podsecuritypolicy.addons.k8s.io"
I0627 16:53:57.822248 22505 tree_walker.go:98] visit "cloudup/resources/addons/podsecuritypolicy.addons.k8s.io/k8s-1.12.yaml.template"
I0627 16:53:57.823155 22505 loader.go:354] loading (templated) resource "addons/podsecuritypolicy.addons.k8s.io/k8s-1.12.yaml"
I0627 16:53:57.823550 22505 tree_walker.go:98] visit "cloudup/resources/addons/podsecuritypolicy.addons.k8s.io/k8s-1.9.yaml.template"
I0627 16:53:57.824140 22505 loader.go:354] loading (templated) resource "addons/podsecuritypolicy.addons.k8s.io/k8s-1.9.yaml"
I0627 16:53:57.824571 22505 tree_walker.go:98] visit "cloudup/resources/addons/podsecuritypolicy.addons.k8s.io/k8s-1.10.yaml.template"
I0627 16:53:57.833610 22505 loader.go:354] loading (templated) resource "addons/podsecuritypolicy.addons.k8s.io/k8s-1.10.yaml"
I0627 16:53:57.836947 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.1.2-r2"
I0627 16:53:57.838387 22505 visitor.go:40] float64 value at spec.strategy.rollingUpdate.maxUnavailable: 0.000000
I0627 16:53:57.838896 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:53:57.839237 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.successThreshold: 1.000000
I0627 16:53:57.839603 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.timeoutSeconds: 5.000000
I0627 16:53:57.839880 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.failureThreshold: 5.000000
I0627 16:53:57.840221 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.httpGet.port: 10054.000000
I0627 16:53:57.840511 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[0].containerPort: 10053.000000
I0627 16:53:57.840759 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[1].containerPort: 10053.000000
I0627 16:53:57.841020 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[2].containerPort: 10055.000000
I0627 16:53:57.841290 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.10"
I0627 16:53:57.841567 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.httpGet.port: 8081.000000
I0627 16:53:57.843573 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.initialDelaySeconds: 3.000000
I0627 16:53:57.844070 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.timeoutSeconds: 5.000000
I0627 16:53:57.844531 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].ports.[0].containerPort: 53.000000
I0627 16:53:57.844950 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].ports.[1].containerPort: 53.000000
I0627 16:53:57.845246 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.10"
I0627 16:53:57.845718 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.failureThreshold: 5.000000
I0627 16:53:57.846088 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.httpGet.port: 10054.000000
I0627 16:53:57.846659 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:53:57.847088 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.successThreshold: 1.000000
I0627 16:53:57.847468 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.timeoutSeconds: 5.000000
I0627 16:53:57.848176 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].ports.[0].containerPort: 10054.000000
I0627 16:53:57.848696 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.10"
I0627 16:53:57.849733 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.failureThreshold: 5.000000
I0627 16:53:57.850252 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.httpGet.port: 10054.000000
I0627 16:53:57.850577 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:53:57.850911 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.successThreshold: 1.000000
I0627 16:53:57.851558 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.timeoutSeconds: 5.000000
I0627 16:53:57.854171 22505 visitor.go:35] string value at spec.template.spec.volumes.[0].configMap.optional: true
I0627 16:53:57.856677 22505 visitor.go:40] float64 value at spec.ports.[0].port: 53.000000
I0627 16:53:57.857045 22505 visitor.go:40] float64 value at spec.ports.[1].port: 53.000000
I0627 16:53:57.858242 22505 template_functions.go:190] watch-ingress=false set on dns-controller
I0627 16:53:57.858743 22505 visitor.go:40] float64 value at spec.replicas: 1.000000
I0627 16:53:57.859033 22505 images.go:59] Consider image for re-mapping: "kope/dns-controller:1.12.0"
I0627 16:53:57.859401 22505 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:53:57.865511 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.periodSeconds: 10.000000
I0627 16:53:57.866070 22505 visitor.go:35] string value at spec.template.spec.containers.[0].securityContext.privileged: true
I0627 16:53:57.866462 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[0].readOnly: true
I0627 16:53:57.866878 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[1].readOnly: false
I0627 16:53:57.867157 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[2].readOnly: false
I0627 16:53:57.867683 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[3].readOnly: true
I0627 16:53:57.867989 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[4].readOnly: true
I0627 16:53:57.868256 22505 images.go:59] Consider image for re-mapping: "quay.io/calico/node:v3.4.0"
I0627 16:53:57.868467 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.failureThreshold: 6.000000
I0627 16:53:57.868696 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.httpGet.port: 9099.000000
I0627 16:53:57.868962 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.initialDelaySeconds: 10.000000
I0627 16:53:57.869291 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.periodSeconds: 10.000000
I0627 16:53:57.869626 22505 images.go:59] Consider image for re-mapping: "quay.io/calico/cni:v3.4.0"
I0627 16:53:57.869829 22505 visitor.go:35] string value at spec.template.spec.containers.[1].volumeMounts.[2].readOnly: true
I0627 16:53:57.870054 22505 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:53:57.870237 22505 visitor.go:35] string value at spec.template.spec.initContainers.[0].volumeMounts.[0].readOnly: true
I0627 16:53:57.870484 22505 visitor.go:35] string value at spec.template.spec.initContainers.[0].volumeMounts.[1].readOnly: true
I0627 16:53:57.870707 22505 images.go:59] Consider image for re-mapping: "calico/upgrade:v1.0.5"
I0627 16:53:57.870982 22505 visitor.go:40] float64 value at spec.template.spec.terminationGracePeriodSeconds: 0.000000
I0627 16:53:57.871190 22505 visitor.go:40] float64 value at spec.updateStrategy.rollingUpdate.maxUnavailable: 1.000000
I0627 16:53:57.873619 22505 visitor.go:40] float64 value at spec.replicas: 1.000000
I0627 16:53:57.876069 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[0].readOnly: true
I0627 16:53:57.876421 22505 images.go:59] Consider image for re-mapping: "quay.io/calico/kube-controllers:v3.4.0"
I0627 16:53:57.876686 22505 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:53:57.876920 22505 images.go:59] Consider image for re-mapping: "calico/upgrade:v1.0.5"
I0627 16:53:57.877228 22505 visitor.go:35] string value at spec.template.spec.initContainers.[0].volumeMounts.[0].readOnly: true
I0627 16:53:57.877577 22505 visitor.go:35] string value at spec.template.spec.initContainers.[0].volumeMounts.[1].readOnly: true
I0627 16:53:57.879852 22505 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:53:57.880341 22505 images.go:59] Consider image for re-mapping: "calico/upgrade:v1.0.5"
I0627 16:53:57.880710 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[0].readOnly: true
I0627 16:53:57.881083 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[1].readOnly: true
I0627 16:53:57.884128 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.0.0"
I0627 16:53:57.887396 22505 visitor.go:40] float64 value at spec.strategy.rollingUpdate.maxUnavailable: 0.000000
I0627 16:53:57.887674 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.httpGet.port: 8081.000000
I0627 16:53:57.887931 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.initialDelaySeconds: 3.000000
I0627 16:53:57.888149 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.timeoutSeconds: 5.000000
I0627 16:53:57.888409 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/kubedns-amd64:1.9"
I0627 16:53:57.888612 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.successThreshold: 1.000000
I0627 16:53:57.888851 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.timeoutSeconds: 5.000000
I0627 16:53:57.889076 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.failureThreshold: 5.000000
I0627 16:53:57.889308 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.httpGet.port: 8080.000000
I0627 16:53:57.889575 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:53:57.889788 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[0].containerPort: 10053.000000
I0627 16:53:57.890010 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[1].containerPort: 10053.000000
I0627 16:53:57.890217 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[2].containerPort: 10055.000000
I0627 16:53:57.890427 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-dnsmasq-amd64:1.14.10"
I0627 16:53:57.890639 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.failureThreshold: 5.000000
I0627 16:53:57.890864 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.httpGet.port: 8080.000000
I0627 16:53:57.891076 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:53:57.891341 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.successThreshold: 1.000000
I0627 16:53:57.891540 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.timeoutSeconds: 5.000000
I0627 16:53:57.891765 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].ports.[0].containerPort: 53.000000
I0627 16:53:57.891975 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].ports.[1].containerPort: 53.000000
I0627 16:53:57.892197 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/dnsmasq-metrics-amd64:1.0"
I0627 16:53:57.892425 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.successThreshold: 1.000000
I0627 16:53:57.892634 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.timeoutSeconds: 5.000000
I0627 16:53:57.892953 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.failureThreshold: 5.000000
I0627 16:53:57.893153 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.httpGet.port: 10054.000000
I0627 16:53:57.893435 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:53:57.893677 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].ports.[0].containerPort: 10054.000000
I0627 16:53:57.893925 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/exechealthz-amd64:1.2"
I0627 16:53:57.894286 22505 visitor.go:40] float64 value at spec.template.spec.containers.[3].ports.[0].containerPort: 8080.000000
I0627 16:53:57.896101 22505 visitor.go:40] float64 value at spec.ports.[0].port: 53.000000
I0627 16:53:57.898685 22505 visitor.go:40] float64 value at spec.ports.[1].port: 53.000000
I0627 16:53:57.901119 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.4.0"
I0627 16:53:57.901689 22505 visitor.go:40] float64 value at spec.strategy.rollingUpdate.maxUnavailable: 0.000000
I0627 16:53:57.902109 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[0].containerPort: 10053.000000
I0627 16:53:57.902503 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[1].containerPort: 10053.000000
I0627 16:53:57.902914 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].ports.[2].containerPort: 10055.000000
I0627 16:53:57.903256 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.httpGet.port: 8081.000000
I0627 16:53:57.903681 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.initialDelaySeconds: 3.000000
I0627 16:53:57.903885 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.timeoutSeconds: 5.000000
I0627 16:53:57.904155 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13"
I0627 16:53:57.904430 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:53:57.904780 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.successThreshold: 1.000000
I0627 16:53:57.905128 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.timeoutSeconds: 5.000000
I0627 16:53:57.905369 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.failureThreshold: 5.000000
I0627 16:53:57.905645 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.httpGet.port: 10054.000000
I0627 16:53:57.905893 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.13"
I0627 16:53:57.906214 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.failureThreshold: 5.000000
I0627 16:53:57.906435 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.httpGet.port: 10054.000000
I0627 16:53:57.906633 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:53:57.906830 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.successThreshold: 1.000000
I0627 16:53:57.908673 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].livenessProbe.timeoutSeconds: 5.000000
I0627 16:53:57.909038 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].ports.[0].containerPort: 53.000000
I0627 16:53:57.909238 22505 visitor.go:40] float64 value at spec.template.spec.containers.[1].ports.[1].containerPort: 53.000000
I0627 16:53:57.909432 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.failureThreshold: 5.000000
I0627 16:53:57.909625 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.httpGet.port: 10054.000000
I0627 16:53:57.909821 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.initialDelaySeconds: 60.000000
I0627 16:53:57.910054 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.successThreshold: 1.000000
I0627 16:53:57.910376 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].livenessProbe.timeoutSeconds: 5.000000
I0627 16:53:57.910593 22505 visitor.go:40] float64 value at spec.template.spec.containers.[2].ports.[0].containerPort: 10054.000000
I0627 16:53:57.910828 22505 images.go:59] Consider image for re-mapping: "k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.13"
I0627 16:53:57.911167 22505 visitor.go:35] string value at spec.template.spec.volumes.[0].configMap.optional: true
I0627 16:53:57.912570 22505 visitor.go:40] float64 value at spec.ports.[0].port: 53.000000
I0627 16:53:57.912781 22505 visitor.go:40] float64 value at spec.ports.[1].port: 53.000000
I0627 16:53:57.913890 22505 template_functions.go:190] watch-ingress=false set on dns-controller
I0627 16:53:57.914716 22505 visitor.go:40] float64 value at spec.replicas: 1.000000
I0627 16:53:57.914888 22505 images.go:59] Consider image for re-mapping: "kope/dns-controller:1.12.0"
I0627 16:53:57.915079 22505 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:53:57.917460 22505 template_functions.go:190] watch-ingress=false set on dns-controller
I0627 16:53:57.920215 22505 visitor.go:40] float64 value at spec.replicas: 1.000000
I0627 16:53:57.920929 22505 images.go:59] Consider image for re-mapping: "kope/dns-controller:1.12.0"
I0627 16:53:57.921333 22505 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:53:57.925756 22505 visitor.go:40] float64 value at spec.replicas: 0.000000
I0627 16:53:57.925930 22505 images.go:59] Consider image for re-mapping: "quay.io/calico/kube-controllers:v3.4.0"
I0627 16:53:57.926111 22505 images.go:59] Consider image for re-mapping: "calico/upgrade:v1.0.5"
I0627 16:53:57.926516 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[0].readOnly: true
I0627 16:53:57.926970 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[1].readOnly: false
I0627 16:53:57.927267 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[2].readOnly: false
I0627 16:53:57.927589 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[3].readOnly: false
I0627 16:53:57.928009 22505 visitor.go:35] string value at spec.template.spec.containers.[0].volumeMounts.[4].readOnly: true
I0627 16:53:57.928270 22505 images.go:59] Consider image for re-mapping: "quay.io/calico/node:v3.4.0"
I0627 16:53:57.930141 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.failureThreshold: 6.000000
I0627 16:53:57.930597 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.httpGet.port: 9099.000000
I0627 16:53:57.931514 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.initialDelaySeconds: 10.000000
I0627 16:53:57.931788 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].livenessProbe.periodSeconds: 10.000000
I0627 16:53:57.932153 22505 visitor.go:40] float64 value at spec.template.spec.containers.[0].readinessProbe.periodSeconds: 10.000000
I0627 16:53:57.932337 22505 visitor.go:35] string value at spec.template.spec.containers.[0].securityContext.privileged: true
I0627 16:53:57.932572 22505 visitor.go:35] string value at spec.template.spec.hostNetwork: true
I0627 16:53:57.932740 22505 images.go:59] Consider image for re-mapping: "quay.io/calico/cni:v3.4.0"
I0627 16:53:57.932924 22505 visitor.go:40] float64 value at spec.template.spec.terminationGracePeriodSeconds: 0.000000
I0627 16:53:57.933116 22505 visitor.go:40] float64 value at spec.updateStrategy.rollingUpdate.maxUnavailable: 1.000000
I0627 16:53:57.934398 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.934627 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.934772 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.934916 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.935060 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.935201 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.935347 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.935522 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.935663 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.935805 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.935945 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.936094 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.936246 22505 task.go:103] testing task "Secret"
I0627 16:53:57.936393 22505 task.go:103] testing task "Secret"
I0627 16:53:57.936584 22505 task.go:103] testing task "Secret"
I0627 16:53:57.936727 22505 task.go:103] testing task "Secret"
I0627 16:53:57.936867 22505 task.go:103] testing task "Secret"
I0627 16:53:57.937008 22505 task.go:103] testing task "Secret"
I0627 16:53:57.937153 22505 task.go:103] testing task "Secret"
I0627 16:53:57.937298 22505 task.go:103] testing task "Secret"
I0627 16:53:57.937472 22505 task.go:103] testing task "Secret"
I0627 16:53:57.937680 22505 task.go:103] testing task "MirrorSecrets"
I0627 16:53:57.937834 22505 task.go:103] testing task "MirrorKeystore"
I0627 16:53:57.937994 22505 decoder.go:224] decoding stream as YAML
I0627 16:53:57.941677 22505 build_flags.go:50] ignoring non-field:
I0627 16:53:57.942082 22505 build_flags.go:50] ignoring non-field:
I0627 16:53:57.946802 22505 task.go:103] testing task "ManagedFile"
I0627 16:53:57.947221 22505 task.go:103] testing task "ManagedFile"
I0627 16:53:57.947403 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.947592 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.947743 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.947896 22505 decoder.go:224] decoding stream as YAML
I0627 16:53:57.948294 22505 build_flags.go:50] ignoring non-field:
I0627 16:53:57.948455 22505 build_flags.go:50] ignoring non-field:
I0627 16:53:57.949014 22505 task.go:103] testing task "ManagedFile"
I0627 16:53:57.949170 22505 task.go:103] testing task "ManagedFile"
I0627 16:53:57.949322 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.949481 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.949627 22505 task.go:103] testing task "Keypair"
I0627 16:53:57.949781 22505 task.go:75] EnsureTask ignoring identical
I0627 16:53:57.949944 22505 task.go:103] testing task "EBSVolume"
I0627 16:53:57.950098 22505 task.go:103] testing task "EBSVolume"
I0627 16:53:57.950248 22505 task.go:103] testing task "EBSVolume"
I0627 16:53:57.950398 22505 task.go:103] testing task "EBSVolume"
I0627 16:53:57.952301 22505 task.go:103] testing task "EBSVolume"
I0627 16:53:57.952478 22505 task.go:103] testing task "EBSVolume"
I0627 16:53:57.952688 22505 task.go:103] testing task "LoadBalancer"
I0627 16:53:57.952881 22505 task.go:103] testing task "SecurityGroup"
I0627 16:53:57.953047 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.953222 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.953402 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.953574 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.953730 22505 task.go:103] testing task "LoadBalancerAttachment"
I0627 16:53:57.953893 22505 task.go:103] testing task "LoadBalancerAttachment"
I0627 16:53:57.954049 22505 task.go:103] testing task "LoadBalancerAttachment"
I0627 16:53:57.954237 22505 task.go:103] testing task "SecurityGroup"
I0627 16:53:57.954440 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.954598 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.954753 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.954905 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.955062 22505 task.go:103] testing task "SecurityGroup"
I0627 16:53:57.955214 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.955369 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.955547 22505 task.go:103] testing task "LoadBalancer"
I0627 16:53:57.955695 22505 task.go:103] testing task "LoadBalancerAttachment"
I0627 16:53:57.955852 22505 task.go:103] testing task "DNSName"
I0627 16:53:57.956006 22505 task.go:103] testing task "DNSZone"
I0627 16:53:57.956158 22505 task.go:103] testing task "DNSZone"
I0627 16:53:57.956314 22505 task.go:75] EnsureTask ignoring identical
I0627 16:53:57.956469 22505 task.go:103] testing task "DNSName"
I0627 16:53:57.956621 22505 task.go:103] testing task "DNSZone"
I0627 16:53:57.956774 22505 task.go:75] EnsureTask ignoring identical
I0627 16:53:57.956936 22505 external_access.go:60] bastion is in use; won't configure SSH access to master / node instances
I0627 16:53:57.957132 22505 task.go:103] testing task "SecurityGroup"
I0627 16:53:57.957284 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.957446 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.957604 22505 task.go:103] testing task "SecurityGroup"
I0627 16:53:57.957755 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.957907 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.958059 22505 task.go:103] testing task "SecurityGroupRule"
W0627 16:53:57.958215 22505 firewall.go:250] Opening etcd port on masters for access from the nodes, for calico. This is unsafe in untrusted environments.
I0627 16:53:57.959092 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.959245 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.959397 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.959553 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.959703 22505 task.go:103] testing task "SecurityGroupRule"
I0627 16:53:57.959896 22505 task.go:103] testing task "SSHKey"
I0627 16:53:57.960081 22505 task.go:103] testing task "VPC"
I0627 16:53:57.960243 22505 task.go:103] testing task "DHCPOptions"
I0627 16:53:57.960398 22505 task.go:103] testing task "VPCDHCPOptionsAssociation"
I0627 16:53:57.964477 22505 task.go:103] testing task "InternetGateway"
I0627 16:53:57.964671 22505 task.go:103] testing task "RouteTable"
I0627 16:53:57.964825 22505 task.go:103] testing task "Route"
I0627 16:53:57.964985 22505 network.go:202] applying subnet tags
I0627 16:53:57.965138 22505 task.go:103] testing task "Subnet"
I0627 16:53:57.965289 22505 task.go:103] testing task "RouteTableAssociation"
I0627 16:53:57.965467 22505 network.go:202] applying subnet tags
I0627 16:53:57.965643 22505 task.go:103] testing task "Subnet"
I0627 16:53:57.965792 22505 task.go:103] testing task "RouteTableAssociation"
I0627 16:53:57.965951 22505 network.go:202] applying subnet tags
I0627 16:53:57.966101 22505 task.go:103] testing task "Subnet"
I0627 16:53:57.966248 22505 task.go:103] testing task "RouteTableAssociation"
I0627 16:53:57.966403 22505 network.go:202] applying subnet tags
I0627 16:53:57.966657 22505 task.go:103] testing task "Subnet"
I0627 16:53:57.966985 22505 task.go:103] testing task "RouteTableAssociation"
I0627 16:53:57.967154 22505 network.go:202] applying subnet tags
I0627 16:53:57.967305 22505 task.go:103] testing task "Subnet"
I0627 16:53:57.967460 22505 task.go:103] testing task "RouteTableAssociation"
I0627 16:53:57.967624 22505 network.go:202] applying subnet tags
I0627 16:53:57.967775 22505 task.go:103] testing task "Subnet"
I0627 16:53:57.967947 22505 task.go:103] testing task "RouteTableAssociation"
I0627 16:53:57.968106 22505 task.go:103] testing task "ElasticIP"
I0627 16:53:57.968263 22505 task.go:103] testing task "NatGateway"
I0627 16:53:57.968426 22505 task.go:103] testing task "RouteTable"
I0627 16:53:57.968585 22505 task.go:103] testing task "Route"
I0627 16:53:57.968822 22505 task.go:103] testing task "ElasticIP"
I0627 16:53:57.968981 22505 task.go:103] testing task "NatGateway"
I0627 16:53:57.969160 22505 task.go:103] testing task "RouteTable"
I0627 16:53:57.969314 22505 task.go:103] testing task "Route"
I0627 16:53:57.969473 22505 task.go:103] testing task "ElasticIP"
I0627 16:53:57.969625 22505 task.go:103] testing task "NatGateway"
I0627 16:53:57.969776 22505 task.go:103] testing task "RouteTable"
I0627 16:53:57.969924 22505 task.go:103] testing task "Route"
I0627 16:53:57.970112 22505 task.go:103] testing task "IAMRole"
I0627 16:53:57.970295 22505 task.go:103] testing task "IAMRolePolicy"
I0627 16:53:57.970450 22505 task.go:103] testing task "IAMInstanceProfile"
I0627 16:53:57.970608 22505 task.go:103] testing task "IAMInstanceProfileRole"
I0627 16:53:57.970764 22505 task.go:103] testing task "IAMRolePolicy"
I0627 16:53:57.970934 22505 task.go:103] testing task "IAMRole"
I0627 16:53:57.971125 22505 task.go:103] testing task "IAMRolePolicy"
I0627 16:53:57.971274 22505 task.go:103] testing task "IAMInstanceProfile"
I0627 16:53:57.971424 22505 task.go:103] testing task "IAMInstanceProfileRole"
I0627 16:53:57.971576 22505 task.go:103] testing task "IAMRolePolicy"
I0627 16:53:57.971745 22505 task.go:103] testing task "IAMRole"
I0627 16:53:57.971892 22505 task.go:103] testing task "IAMRolePolicy"
I0627 16:53:57.972065 22505 task.go:103] testing task "IAMInstanceProfile"
I0627 16:53:57.972219 22505 task.go:103] testing task "IAMInstanceProfileRole"
I0627 16:53:57.972373 22505 task.go:103] testing task "IAMRolePolicy"
I0627 16:53:57.972555 22505 task.go:103] testing task "LaunchConfiguration"
I0627 16:53:57.972715 22505 task.go:103] testing task "AutoscalingGroup"
I0627 16:53:57.972955 22505 task.go:103] testing task "LaunchConfiguration"
I0627 16:53:57.976196 22505 task.go:103] testing task "AutoscalingGroup"
I0627 16:53:57.976503 22505 task.go:103] testing task "LaunchConfiguration"
I0627 16:53:57.976839 22505 task.go:103] testing task "AutoscalingGroup"
I0627 16:53:57.977457 22505 task.go:103] testing task "LaunchConfiguration"
I0627 16:53:57.978434 22505 task.go:103] testing task "AutoscalingGroup"
I0627 16:53:57.978858 22505 task.go:103] testing task "LaunchConfiguration"
I0627 16:53:57.979126 22505 task.go:103] testing task "AutoscalingGroup"
I0627 16:53:57.981505 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/cluster.spec"
I0627 16:53:57.981702 22505 s3context.go:238] Checking default bucket encryption for "my-state-store"
I0627 16:53:57.981868 22505 s3context.go:243] Calling S3 GetBucketEncryption Bucket="my-state-store"
I0627 16:53:58.159701 22505 s3context.go:250] Unable to read bucket encryption policy for "my-state-store": will encrypt using AES256
I0627 16:53:58.160340 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/cluster.spec" SSE="AES256" ACL=""
I0627 16:53:58.373444 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/bastions"
I0627 16:53:58.558919 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/bastions"
I0627 16:53:58.559530 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/bastions" SSE="AES256" ACL=""
I0627 16:53:58.756565 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/bastions"
I0627 16:53:58.757204 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/bastions" SSE="AES256" ACL=""
I0627 16:53:58.989074 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1a"
I0627 16:53:59.206497 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1a"
I0627 16:53:59.207141 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/master-eu-central-1a" SSE="AES256" ACL=""
I0627 16:53:59.442048 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1a"
I0627 16:53:59.442883 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/master-eu-central-1a" SSE="AES256" ACL=""
I0627 16:53:59.654645 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1b"
I0627 16:53:59.840043 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1b"
I0627 16:53:59.840948 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/master-eu-central-1b" SSE="AES256" ACL=""
I0627 16:54:00.090894 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1b"
I0627 16:54:00.091528 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/master-eu-central-1b" SSE="AES256" ACL=""
I0627 16:54:00.312522 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1c"
I0627 16:54:00.499493 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1c"
I0627 16:54:00.500219 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/master-eu-central-1c" SSE="AES256" ACL=""
I0627 16:54:00.701835 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/master-eu-central-1c"
I0627 16:54:00.702454 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/master-eu-central-1c" SSE="AES256" ACL=""
I0627 16:54:00.901694 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/nodes"
I0627 16:54:01.091582 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/nodes"
I0627 16:54:01.092278 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/nodes" SSE="AES256" ACL=""
I0627 16:54:01.310861 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/instancegroup/nodes"
I0627 16:54:01.311678 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/instancegroup/nodes" SSE="AES256" ACL=""
I0627 16:54:01.506123 22505 topological_sort.go:64] Dependencies:
I0627 16:54:01.506712 22505 topological_sort.go:66] SecurityGroupRule/https-api-elb-0.0.0.0/0: [SecurityGroup/api-elb.dev-qa.k8s.domain.com]
I0627 16:54:01.507085 22505 topological_sort.go:66] RouteTableAssociation/utility-eu-central-1a.dev-qa.k8s.domain.com: [RouteTable/dev-qa.k8s.domain.com Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com]
I0627 16:54:01.507546 22505 topological_sort.go:66] SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b: []
I0627 16:54:01.507995 22505 topological_sort.go:66] SecurityGroupRule/ssh-elb-to-bastion: [SecurityGroup/bastion.dev-qa.k8s.domain.com SecurityGroup/bastion-elb.dev-qa.k8s.domain.com]
I0627 16:54:01.508448 22505 topological_sort.go:66] SecurityGroupRule/bastion-to-master-ssh: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/bastion.dev-qa.k8s.domain.com]
I0627 16:54:01.508921 22505 topological_sort.go:66] IAMRolePolicy/bastions.dev-qa.k8s.domain.com: [IAMRole/bastions.dev-qa.k8s.domain.com DNSZone/k8s.domain.com]
I0627 16:54:01.509794 22505 topological_sort.go:66] Keypair/etcd-clients-ca: []
I0627 16:54:01.510528 22505 topological_sort.go:66] Keypair/etcd-peers-ca-main: []
I0627 16:54:01.511100 22505 topological_sort.go:66] RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.511718 22505 topological_sort.go:66] Secret/system:dns: []
I0627 16:54:01.512173 22505 topological_sort.go:66] Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.512527 22505 topological_sort.go:66] Keypair/etcd-manager-ca-events: []
I0627 16:54:01.512878 22505 topological_sort.go:66] LoadBalancerAttachment/api-master-eu-central-1b: [LoadBalancer/api.dev-qa.k8s.domain.com AutoscalingGroup/master-eu-central-1b.masters.dev-qa.k8s.domain.com]
I0627 16:54:01.513310 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-bootstrap: []
I0627 16:54:01.513674 22505 topological_sort.go:66] SecurityGroupRule/bastion-egress: [SecurityGroup/bastion.dev-qa.k8s.domain.com]
I0627 16:54:01.514071 22505 topological_sort.go:66] SecurityGroupRule/node-to-master-protocol-ipip: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:54:01.514524 22505 topological_sort.go:66] Secret/system:scheduler: []
I0627 16:54:01.514888 22505 topological_sort.go:66] ElasticIP/eu-central-1a.dev-qa.k8s.domain.com: [RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com]
I0627 16:54:01.517109 22505 topological_sort.go:66] NatGateway/eu-central-1a.dev-qa.k8s.domain.com: [ElasticIP/eu-central-1a.dev-qa.k8s.domain.com Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com]
I0627 16:54:01.517682 22505 topological_sort.go:66] Keypair/kubelet: [Keypair/ca]
I0627 16:54:01.518060 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-rbac.addons.k8s.io-k8s-1.8: []
I0627 16:54:01.518369 22505 topological_sort.go:66] SecurityGroupRule/node-to-master-tcp-4003-65535: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:54:01.518762 22505 topological_sort.go:66] RouteTableAssociation/utility-eu-central-1b.dev-qa.k8s.domain.com: [RouteTable/dev-qa.k8s.domain.com Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:54:01.519209 22505 topological_sort.go:66] SecurityGroupRule/node-to-master-tcp-2382-4001: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:54:01.519540 22505 topological_sort.go:66] SecurityGroupRule/all-node-to-node: [SecurityGroup/nodes.dev-qa.k8s.domain.com SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:54:01.519921 22505 topological_sort.go:66] SecurityGroupRule/icmp-pmtu-api-elb-0.0.0.0/0: [SecurityGroup/api-elb.dev-qa.k8s.domain.com]
I0627 16:54:01.520268 22505 topological_sort.go:66] SecurityGroupRule/node-to-master-tcp-1-2379: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:54:01.520590 22505 topological_sort.go:66] ElasticIP/eu-central-1b.dev-qa.k8s.domain.com: [RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:54:01.520982 22505 topological_sort.go:66] LoadBalancer/bastion.dev-qa.k8s.domain.com: [Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com SecurityGroup/bastion-elb.dev-qa.k8s.domain.com]
I0627 16:54:01.521388 22505 topological_sort.go:66] ManagedFile/manifests-etcdmanager-events: []
I0627 16:54:01.521655 22505 topological_sort.go:66] ManagedFile/etcd-cluster-spec-main: []
I0627 16:54:01.521856 22505 topological_sort.go:66] EBSVolume/a.etcd-events.dev-qa.k8s.domain.com: []
I0627 16:54:01.522126 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.7-v3: []
I0627 16:54:01.522398 22505 topological_sort.go:66] SecurityGroupRule/all-master-to-master: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/masters.dev-qa.k8s.domain.com]
I0627 16:54:01.522709 22505 topological_sort.go:66] SecurityGroup/nodes.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.522961 22505 topological_sort.go:66] AutoscalingGroup/master-eu-central-1c.masters.dev-qa.k8s.domain.com: [LaunchConfiguration/master-eu-central-1c.masters.dev-qa.k8s.domain.com Subnet/eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:54:01.523316 22505 topological_sort.go:66] Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.523536 22505 topological_sort.go:66] IAMInstanceProfile/bastions.dev-qa.k8s.domain.com: []
I0627 16:54:01.523805 22505 topological_sort.go:66] RouteTable/dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.524043 22505 topological_sort.go:66] AutoscalingGroup/master-eu-central-1a.masters.dev-qa.k8s.domain.com: [LaunchConfiguration/master-eu-central-1a.masters.dev-qa.k8s.domain.com Subnet/eu-central-1a.dev-qa.k8s.domain.com]
I0627 16:54:01.524376 22505 topological_sort.go:66] IAMInstanceProfileRole/nodes.dev-qa.k8s.domain.com: [IAMInstanceProfile/nodes.dev-qa.k8s.domain.com IAMRole/nodes.dev-qa.k8s.domain.com]
I0627 16:54:01.524655 22505 topological_sort.go:66] LaunchConfiguration/master-eu-central-1a.masters.dev-qa.k8s.domain.com: [IAMInstanceProfile/masters.dev-qa.k8s.domain.com SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b SecurityGroup/masters.dev-qa.k8s.domain.com]
I0627 16:54:01.525043 22505 topological_sort.go:66] IAMRole/masters.dev-qa.k8s.domain.com: []
I0627 16:54:01.528239 22505 topological_sort.go:66] ManagedFile/manifests-etcdmanager-main: []
I0627 16:54:01.528509 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.7.0: []
I0627 16:54:01.528808 22505 topological_sort.go:66] LaunchConfiguration/master-eu-central-1c.masters.dev-qa.k8s.domain.com: [IAMInstanceProfile/masters.dev-qa.k8s.domain.com SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b SecurityGroup/masters.dev-qa.k8s.domain.com]
I0627 16:54:01.529363 22505 topological_sort.go:66] SecurityGroup/bastion.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.530239 22505 topological_sort.go:66] RouteTableAssociation/private-eu-central-1a.dev-qa.k8s.domain.com: [RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com Subnet/eu-central-1a.dev-qa.k8s.domain.com]
I0627 16:54:01.530992 22505 topological_sort.go:66] RouteTableAssociation/private-eu-central-1c.dev-qa.k8s.domain.com: [RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com Subnet/eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:54:01.531489 22505 topological_sort.go:66] DNSName/api.dev-qa.k8s.domain.com: [DNSZone/k8s.domain.com LoadBalancer/api.dev-qa.k8s.domain.com]
I0627 16:54:01.531914 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-core.addons.k8s.io: []
I0627 16:54:01.532262 22505 topological_sort.go:66] ElasticIP/eu-central-1c.dev-qa.k8s.domain.com: [RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:54:01.532571 22505 topological_sort.go:66] LoadBalancerAttachment/api-master-eu-central-1c: [LoadBalancer/api.dev-qa.k8s.domain.com AutoscalingGroup/master-eu-central-1c.masters.dev-qa.k8s.domain.com]
I0627 16:54:01.532994 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-kubelet-api.rbac.addons.k8s.io-k8s-1.9: []
I0627 16:54:01.533410 22505 topological_sort.go:66] AutoscalingGroup/bastions.dev-qa.k8s.domain.com: [LaunchConfiguration/bastions.dev-qa.k8s.domain.com Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:54:01.533814 22505 topological_sort.go:66] Secret/system:controller_manager: []
I0627 16:54:01.534152 22505 topological_sort.go:66] IAMRolePolicy/nodes.dev-qa.k8s.domain.com: [IAMRole/nodes.dev-qa.k8s.domain.com DNSZone/k8s.domain.com]
I0627 16:54:01.534474 22505 topological_sort.go:66] SecurityGroupRule/node-to-master-udp-1-65535: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:54:01.534787 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.12: []
I0627 16:54:01.535198 22505 topological_sort.go:66] InternetGateway/dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.535491 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.12: []
I0627 16:54:01.535854 22505 topological_sort.go:66] IAMRole/bastions.dev-qa.k8s.domain.com: []
I0627 16:54:01.536163 22505 topological_sort.go:66] Keypair/kube-proxy: [Keypair/ca]
I0627 16:54:01.536457 22505 topological_sort.go:66] Keypair/etcd-peers-ca-events: []
I0627 16:54:01.536744 22505 topological_sort.go:66] LoadBalancer/api.dev-qa.k8s.domain.com: [Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com SecurityGroup/api-elb.dev-qa.k8s.domain.com]
I0627 16:54:01.539437 22505 topological_sort.go:66] DNSZone/k8s.domain.com: []
I0627 16:54:01.540197 22505 topological_sort.go:66] IAMInstanceProfileRole/bastions.dev-qa.k8s.domain.com: [IAMInstanceProfile/bastions.dev-qa.k8s.domain.com IAMRole/bastions.dev-qa.k8s.domain.com]
I0627 16:54:01.540898 22505 topological_sort.go:66] Subnet/eu-central-1b.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.541225 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-limit-range.addons.k8s.io: []
I0627 16:54:01.541495 22505 topological_sort.go:66] VPCDHCPOptionsAssociation/dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com DHCPOptions/dev-qa.k8s.domain.com]
I0627 16:54:01.541986 22505 topological_sort.go:66] DNSName/bastion.dev-qa.k8s.domain.com: [DNSZone/k8s.domain.com LoadBalancer/bastion.dev-qa.k8s.domain.com]
I0627 16:54:01.542306 22505 topological_sort.go:66] LaunchConfiguration/bastions.dev-qa.k8s.domain.com: [IAMInstanceProfile/bastions.dev-qa.k8s.domain.com SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b SecurityGroup/bastion.dev-qa.k8s.domain.com]
I0627 16:54:01.542895 22505 topological_sort.go:66] Keypair/etcd-manager-ca-main: []
I0627 16:54:01.543310 22505 topological_sort.go:66] Route/private-eu-central-1c-0.0.0.0/0: [RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com NatGateway/eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:54:01.543834 22505 topological_sort.go:66] MirrorKeystore/mirror-keystore: [Secret/kube-proxy Secret/system:dns Secret/system:logging Secret/system:controller_manager Secret/admin Secret/system:scheduler Secret/kubelet Secret/system:monitoring Secret/kube]
I0627 16:54:01.544255 22505 topological_sort.go:66] RouteTableAssociation/private-eu-central-1b.dev-qa.k8s.domain.com: [RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com Subnet/eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:54:01.544643 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-pre-k8s-1.6: []
I0627 16:54:01.544902 22505 topological_sort.go:66] EBSVolume/b.etcd-events.dev-qa.k8s.domain.com: []
I0627 16:54:01.545142 22505 topological_sort.go:66] Subnet/eu-central-1a.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.545412 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.12: []
I0627 16:54:01.545730 22505 topological_sort.go:66] LoadBalancerAttachment/api-master-eu-central-1a: [LoadBalancer/api.dev-qa.k8s.domain.com AutoscalingGroup/master-eu-central-1a.masters.dev-qa.k8s.domain.com]
I0627 16:54:01.546028 22505 topological_sort.go:66] Route/private-eu-central-1b-0.0.0.0/0: [RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com NatGateway/eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:54:01.546324 22505 topological_sort.go:66] Keypair/master: [Keypair/ca]
I0627 16:54:01.546530 22505 topological_sort.go:66] DHCPOptions/dev-qa.k8s.domain.com: []
I0627 16:54:01.546778 22505 topological_sort.go:66] SecurityGroupRule/bastion-elb-egress: [SecurityGroup/bastion-elb.dev-qa.k8s.domain.com]
I0627 16:54:01.547038 22505 topological_sort.go:66] ManagedFile/etcd-cluster-spec-events: []
I0627 16:54:01.547277 22505 topological_sort.go:66] IAMInstanceProfile/masters.dev-qa.k8s.domain.com: []
I0627 16:54:01.547460 22505 topological_sort.go:66] Route/private-eu-central-1a-0.0.0.0/0: [RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com NatGateway/eu-central-1a.dev-qa.k8s.domain.com]
I0627 16:54:01.549949 22505 topological_sort.go:66] EBSVolume/b.etcd-main.dev-qa.k8s.domain.com: []
I0627 16:54:01.550278 22505 topological_sort.go:66] SecurityGroupRule/all-master-to-node: [SecurityGroup/nodes.dev-qa.k8s.domain.com SecurityGroup/masters.dev-qa.k8s.domain.com]
I0627 16:54:01.550635 22505 topological_sort.go:66] RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.550966 22505 topological_sort.go:66] Keypair/apiserver-aggregator: [Keypair/apiserver-aggregator-ca]
I0627 16:54:01.551205 22505 topological_sort.go:66] Secret/system:monitoring: []
I0627 16:54:01.551380 22505 topological_sort.go:66] Keypair/apiserver-aggregator-ca: []
I0627 16:54:01.551583 22505 topological_sort.go:66] Keypair/kube-scheduler: [Keypair/ca]
I0627 16:54:01.551818 22505 topological_sort.go:66] RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.552072 22505 topological_sort.go:66] Secret/kubelet: []
I0627 16:54:01.552239 22505 topological_sort.go:66] Secret/system:logging: []
I0627 16:54:01.552410 22505 topological_sort.go:66] Keypair/ca: []
I0627 16:54:01.552611 22505 topological_sort.go:66] SecurityGroupRule/ssh-external-to-bastion-elb-0.0.0.0/0: [SecurityGroup/bastion-elb.dev-qa.k8s.domain.com]
I0627 16:54:01.552842 22505 topological_sort.go:66] VPC/dev-qa.k8s.domain.com: []
I0627 16:54:01.553014 22505 topological_sort.go:66] AutoscalingGroup/master-eu-central-1b.masters.dev-qa.k8s.domain.com: [LaunchConfiguration/master-eu-central-1b.masters.dev-qa.k8s.domain.com Subnet/eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:54:01.553292 22505 topological_sort.go:66] Secret/kube-proxy: []
I0627 16:54:01.553460 22505 topological_sort.go:66] SecurityGroup/bastion-elb.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.553710 22505 topological_sort.go:66] Route/0.0.0.0/0: [RouteTable/dev-qa.k8s.domain.com InternetGateway/dev-qa.k8s.domain.com]
I0627 16:54:01.553919 22505 topological_sort.go:66] IAMRolePolicy/additional.masters.dev-qa.k8s.domain.com: [IAMRole/masters.dev-qa.k8s.domain.com]
I0627 16:54:01.554127 22505 topological_sort.go:66] IAMInstanceProfile/nodes.dev-qa.k8s.domain.com: []
I0627 16:54:01.554314 22505 topological_sort.go:66] SecurityGroupRule/api-elb-egress: [SecurityGroup/api-elb.dev-qa.k8s.domain.com]
I0627 16:54:01.554517 22505 topological_sort.go:66] LaunchConfiguration/nodes.dev-qa.k8s.domain.com: [IAMInstanceProfile/nodes.dev-qa.k8s.domain.com SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:54:01.555485 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.6: []
I0627 16:54:01.555963 22505 topological_sort.go:66] IAMRole/nodes.dev-qa.k8s.domain.com: []
I0627 16:54:01.556170 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.6: []
I0627 16:54:01.556387 22505 topological_sort.go:66] SecurityGroup/api-elb.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.556680 22505 topological_sort.go:66] LoadBalancerAttachment/bastion-elb-attachment: [LoadBalancer/bastion.dev-qa.k8s.domain.com AutoscalingGroup/bastions.dev-qa.k8s.domain.com]
I0627 16:54:01.556912 22505 topological_sort.go:66] EBSVolume/c.etcd-events.dev-qa.k8s.domain.com: []
I0627 16:54:01.557095 22505 topological_sort.go:66] IAMRolePolicy/masters.dev-qa.k8s.domain.com: [IAMRole/masters.dev-qa.k8s.domain.com DNSZone/k8s.domain.com]
I0627 16:54:01.557311 22505 topological_sort.go:66] RouteTableAssociation/utility-eu-central-1c.dev-qa.k8s.domain.com: [RouteTable/dev-qa.k8s.domain.com Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:54:01.557596 22505 topological_sort.go:66] SecurityGroupRule/https-elb-to-master: [SecurityGroup/masters.dev-qa.k8s.domain.com SecurityGroup/api-elb.dev-qa.k8s.domain.com]
I0627 16:54:01.557838 22505 topological_sort.go:66] EBSVolume/c.etcd-main.dev-qa.k8s.domain.com: []
I0627 16:54:01.558043 22505 topological_sort.go:66] SecurityGroupRule/master-egress: [SecurityGroup/masters.dev-qa.k8s.domain.com]
I0627 16:54:01.558255 22505 topological_sort.go:66] IAMRolePolicy/additional.bastions.dev-qa.k8s.domain.com: [IAMRole/bastions.dev-qa.k8s.domain.com]
I0627 16:54:01.558458 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.6.0: []
I0627 16:54:01.563292 22505 topological_sort.go:66] NatGateway/eu-central-1b.dev-qa.k8s.domain.com: [ElasticIP/eu-central-1b.dev-qa.k8s.domain.com Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:54:01.563939 22505 topological_sort.go:66] Subnet/eu-central-1c.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.564334 22505 topological_sort.go:66] dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-pre-k8s-1.6: []
I0627 16:54:01.564857 22505 topological_sort.go:66] SecurityGroup/masters.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.565297 22505 topological_sort.go:66] IAMRolePolicy/additional.nodes.dev-qa.k8s.domain.com: [IAMRole/nodes.dev-qa.k8s.domain.com]
I0627 16:54:01.565746 22505 topological_sort.go:66] SecurityGroupRule/node-egress: [SecurityGroup/nodes.dev-qa.k8s.domain.com]
I0627 16:54:01.566148 22505 topological_sort.go:66] LaunchConfiguration/master-eu-central-1b.masters.dev-qa.k8s.domain.com: [IAMInstanceProfile/masters.dev-qa.k8s.domain.com SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b SecurityGroup/masters.dev-qa.k8s.domain.com]
I0627 16:54:01.566619 22505 topological_sort.go:66] Keypair/apiserver-proxy-client: [Keypair/ca]
I0627 16:54:01.567040 22505 topological_sort.go:66] Keypair/kubelet-api: [Keypair/ca]
I0627 16:54:01.567381 22505 topological_sort.go:66] SecurityGroupRule/bastion-to-node-ssh: [SecurityGroup/nodes.dev-qa.k8s.domain.com SecurityGroup/bastion.dev-qa.k8s.domain.com]
I0627 16:54:01.567683 22505 topological_sort.go:66] EBSVolume/a.etcd-main.dev-qa.k8s.domain.com: []
I0627 16:54:01.567916 22505 topological_sort.go:66] Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com: [VPC/dev-qa.k8s.domain.com]
I0627 16:54:01.568177 22505 topological_sort.go:66] Keypair/kube-controller-manager: [Keypair/ca]
I0627 16:54:01.568421 22505 topological_sort.go:66] AutoscalingGroup/nodes.dev-qa.k8s.domain.com: [LaunchConfiguration/nodes.dev-qa.k8s.domain.com Subnet/eu-central-1a.dev-qa.k8s.domain.com Subnet/eu-central-1b.dev-qa.k8s.domain.com Subnet/eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:54:01.568779 22505 topological_sort.go:66] MirrorSecrets/mirror-secrets: [Secret/kube-proxy Secret/system:dns Secret/system:logging Secret/system:controller_manager Secret/admin Secret/system:scheduler Secret/kubelet Secret/system:monitoring Secret/kube]
I0627 16:54:01.569128 22505 topological_sort.go:66] Keypair/kops: [Keypair/ca]
I0627 16:54:01.569353 22505 topological_sort.go:66] Secret/kube: []
I0627 16:54:01.569611 22505 topological_sort.go:66] Keypair/kubecfg: [Keypair/ca]
I0627 16:54:01.570299 22505 topological_sort.go:66] NatGateway/eu-central-1c.dev-qa.k8s.domain.com: [ElasticIP/eu-central-1c.dev-qa.k8s.domain.com Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com]
I0627 16:54:01.571227 22505 topological_sort.go:66] IAMInstanceProfileRole/masters.dev-qa.k8s.domain.com: [IAMInstanceProfile/masters.dev-qa.k8s.domain.com IAMRole/masters.dev-qa.k8s.domain.com]
I0627 16:54:01.571605 22505 topological_sort.go:66] Secret/admin: []
I0627 16:54:01.574781 22505 executor.go:103] Tasks: 0 done / 143 total; 51 can run
I0627 16:54:01.575238 22505 executor.go:178] Executing task "Secret/kube": *fitasks.Secret {"Name":"kube","Lifecycle":"Sync"}
I0627 16:54:01.575812 22505 executor.go:178] Executing task "VPC/dev-qa.k8s.domain.com": *awstasks.VPC {"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:54:01.576100 22505 executor.go:178] Executing task "IAMInstanceProfile/bastions.dev-qa.k8s.domain.com": *awstasks.IAMInstanceProfile {"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false}
I0627 16:54:01.579027 22505 request_logger.go:45] AWS request: iam/GetInstanceProfile
I0627 16:54:01.575884 22505 executor.go:178] Executing task "Keypair/apiserver-aggregator-ca": *fitasks.Keypair {"Name":"apiserver-aggregator-ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=apiserver-aggregator-ca","type":"ca","format":"v1alpha2"}
I0627 16:54:01.577425 22505 request_logger.go:45] AWS request: ec2/DescribeVpcs
I0627 16:54:01.575315 22505 executor.go:178] Executing task "IAMInstanceProfile/nodes.dev-qa.k8s.domain.com": *awstasks.IAMInstanceProfile {"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false}
I0627 16:54:01.575377 22505 executor.go:178] Executing task "Secret/admin": *fitasks.Secret {"Name":"admin","Lifecycle":"Sync"}
I0627 16:54:01.575300 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.12": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.12","Lifecycle":"Sync","Location":"addons/dns-controller.addons.k8s.io/k8s-1.12.yaml","Contents":{"Name":"","Resource":"apiVersion: apps/v1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n k8s-app: dns-controller\n version: v1.12.0\n name: dns-controller\n namespace: kube-system\nspec:\n replicas: 1\n selector:\n matchLabels:\n k8s-app: dns-controller\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n k8s-app: dns-controller\n version: v1.12.0\n spec:\n containers:\n - command:\n - /usr/bin/dns-controller\n - --watch-ingress=false\n - --dns=aws-route53\n - --zone=k8s.domain.com\n - --zone=*/*\n - -v=2\n image: kope/dns-controller:1.12.0\n name: dns-controller\n resources:\n requests:\n cpu: 50m\n memory: 50Mi\n dnsPolicy: Default\n hostNetwork: true\n nodeSelector:\n node-role.kubernetes.io/master: \"\"\n serviceAccount: dns-controller\n tolerations:\n - effect: NoSchedule\n key: node-role.kubernetes.io/master\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n name: dns-controller\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n name: kops:dns-controller\nrules:\n- apiGroups:\n - \"\"\n resources:\n - endpoints\n - services\n - pods\n - ingress\n - nodes\n verbs:\n - get\n - list\n - watch\n- apiGroups:\n - extensions\n resources:\n - ingresses\n verbs:\n - get\n - list\n - watch\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n name: kops:dns-controller\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: kops:dns-controller\nsubjects:\n- apiGroup: rbac.authorization.k8s.io\n kind: User\n name: system:serviceaccount:kube-system:dns-controller\n"}}
I0627 16:54:01.575283 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.6.0": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.6.0","Lifecycle":"Sync","Location":"addons/storage-aws.addons.k8s.io/v1.6.0.yaml","Contents":{"Name":"","Resource":"apiVersion: storage.k8s.io/v1beta1\nkind: StorageClass\nmetadata:\n labels:\n k8s-addon: storage-aws.addons.k8s.io\n name: default\nparameters:\n type: gp2\nprovisioner: kubernetes.io/aws-ebs\n\n---\n\napiVersion: storage.k8s.io/v1beta1\nkind: StorageClass\nmetadata:\n annotations:\n storageclass.beta.kubernetes.io/is-default-class: \"true\"\n labels:\n k8s-addon: storage-aws.addons.k8s.io\n name: gp2\nparameters:\n type: gp2\nprovisioner: kubernetes.io/aws-ebs\n"}}
I0627 16:54:01.575313 22505 executor.go:178] Executing task "EBSVolume/c.etcd-main.dev-qa.k8s.domain.com": *awstasks.EBSVolume {"Name":"c.etcd-main.dev-qa.k8s.domain.com","Lifecycle":"Sync","AvailabilityZone":"eu-central-1c","Encrypted":true,"ID":null,"KmsKeyId":null,"SizeGB":20,"Tags":{"k8s.io/etcd/main":"c/a,b,c","k8s.io/role/master":"1","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"VolumeIops":null,"VolumeType":"gp2"}
I0627 16:54:01.575429 22505 executor.go:178] Executing task "Secret/system:logging": *fitasks.Secret {"Name":"system:logging","Lifecycle":"Sync"}
I0627 16:54:01.575367 22505 executor.go:178] Executing task "EBSVolume/a.etcd-events.dev-qa.k8s.domain.com": *awstasks.EBSVolume {"Name":"a.etcd-events.dev-qa.k8s.domain.com","Lifecycle":"Sync","AvailabilityZone":"eu-central-1a","Encrypted":true,"ID":null,"KmsKeyId":null,"SizeGB":20,"Tags":{"k8s.io/etcd/events":"a/a,b,c","k8s.io/role/master":"1","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"VolumeIops":null,"VolumeType":"gp2"}
I0627 16:54:01.575352 22505 executor.go:178] Executing task "EBSVolume/a.etcd-main.dev-qa.k8s.domain.com": *awstasks.EBSVolume {"Name":"a.etcd-main.dev-qa.k8s.domain.com","Lifecycle":"Sync","AvailabilityZone":"eu-central-1a","Encrypted":true,"ID":null,"KmsKeyId":null,"SizeGB":20,"Tags":{"k8s.io/etcd/main":"a/a,b,c","k8s.io/role/master":"1","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"VolumeIops":null,"VolumeType":"gp2"}
I0627 16:54:01.575333 22505 executor.go:178] Executing task "EBSVolume/b.etcd-events.dev-qa.k8s.domain.com": *awstasks.EBSVolume {"Name":"b.etcd-events.dev-qa.k8s.domain.com","Lifecycle":"Sync","AvailabilityZone":"eu-central-1b","Encrypted":true,"ID":null,"KmsKeyId":null,"SizeGB":20,"Tags":{"k8s.io/etcd/events":"b/a,b,c","k8s.io/role/master":"1","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"VolumeIops":null,"VolumeType":"gp2"}
I0627 16:54:01.575447 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-limit-range.addons.k8s.io": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-limit-range.addons.k8s.io","Lifecycle":"Sync","Location":"addons/limit-range.addons.k8s.io/v1.5.0.yaml","Contents":{"Name":"","Resource":"apiVersion: v1\nkind: LimitRange\nmetadata:\n name: limits\n namespace: default\nspec:\n limits:\n - defaultRequest:\n cpu: 100m\n type: Container\n"}}
I0627 16:54:01.575382 22505 executor.go:178] Executing task "Keypair/etcd-peers-ca-main": *fitasks.Keypair {"Name":"etcd-peers-ca-main","alternateNames":null,"alternateNameTasks":null,"Lifecycle":null,"Signer":null,"subject":"cn=etcd-peers-ca-main","type":"ca","format":"v1alpha2"}
I0627 16:54:01.575202 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-kubelet-api.rbac.addons.k8s.io-k8s-1.9": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-kubelet-api.rbac.addons.k8s.io-k8s-1.9","Lifecycle":"Sync","Location":"addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml","Contents":{"Name":"","Resource":"apiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n name: kops:system:kubelet-api-admin\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: system:kubelet-api-admin\nsubjects:\n- apiGroup: rbac.authorization.k8s.io\n kind: User\n name: kubelet-api\n"}}
I0627 16:54:01.575612 22505 executor.go:178] Executing task "EBSVolume/b.etcd-main.dev-qa.k8s.domain.com": *awstasks.EBSVolume {"Name":"b.etcd-main.dev-qa.k8s.domain.com","Lifecycle":"Sync","AvailabilityZone":"eu-central-1b","Encrypted":true,"ID":null,"KmsKeyId":null,"SizeGB":20,"Tags":{"k8s.io/etcd/main":"b/a,b,c","k8s.io/role/master":"1","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"VolumeIops":null,"VolumeType":"gp2"}
I0627 16:54:01.575527 22505 executor.go:178] Executing task "DHCPOptions/dev-qa.k8s.domain.com": *awstasks.DHCPOptions {"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"DomainName":"eu-central-1.compute.internal","DomainNameServers":"AmazonProvidedDNS","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:54:01.575504 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.7.0": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-storage-aws.addons.k8s.io-v1.7.0","Lifecycle":"Sync","Location":"addons/storage-aws.addons.k8s.io/v1.7.0.yaml","Contents":{"Name":"","Resource":"apiVersion: storage.k8s.io/v1\nkind: StorageClass\nmetadata:\n labels:\n k8s-addon: storage-aws.addons.k8s.io\n name: default\nparameters:\n type: gp2\nprovisioner: kubernetes.io/aws-ebs\n\n---\n\napiVersion: storage.k8s.io/v1\nkind: StorageClass\nmetadata:\n annotations:\n storageclass.beta.kubernetes.io/is-default-class: \"true\"\n labels:\n k8s-addon: storage-aws.addons.k8s.io\n name: gp2\nparameters:\n type: gp2\nprovisioner: kubernetes.io/aws-ebs\n"}}
I0627 16:54:01.575483 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-bootstrap": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-bootstrap","Lifecycle":"Sync","Location":"addons/bootstrap-channel.yaml","Contents":{"Name":"","Resource":"kind: Addons\nmetadata:\n creationTimestamp: null\n name: bootstrap\nspec:\n addons:\n - manifest: core.addons.k8s.io/v1.4.0.yaml\n name: core.addons.k8s.io\n selector:\n k8s-addon: core.addons.k8s.io\n version: 1.4.0\n - id: pre-k8s-1.6\n kubernetesVersion: \u003c1.6.0\n manifest: kube-dns.addons.k8s.io/pre-k8s-1.6.yaml\n name: kube-dns.addons.k8s.io\n selector:\n k8s-addon: kube-dns.addons.k8s.io\n version: 1.14.13-kops.1\n - id: k8s-1.6\n kubernetesVersion: '\u003e=1.6.0 \u003c1.12.0'\n manifest: kube-dns.addons.k8s.io/k8s-1.6.yaml\n name: kube-dns.addons.k8s.io\n selector:\n k8s-addon: kube-dns.addons.k8s.io\n version: 1.14.13-kops.1\n - id: k8s-1.12\n kubernetesVersion: '\u003e=1.12.0'\n manifest: kube-dns.addons.k8s.io/k8s-1.12.yaml\n name: kube-dns.addons.k8s.io\n selector:\n k8s-addon: kube-dns.addons.k8s.io\n version: 1.14.13-kops.1\n - id: k8s-1.8\n kubernetesVersion: '\u003e=1.8.0'\n manifest: rbac.addons.k8s.io/k8s-1.8.yaml\n name: rbac.addons.k8s.io\n selector:\n k8s-addon: rbac.addons.k8s.io\n version: 1.8.0\n - id: k8s-1.9\n kubernetesVersion: '\u003e=1.9.0'\n manifest: kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml\n name: kubelet-api.rbac.addons.k8s.io\n selector:\n k8s-addon: kubelet-api.rbac.addons.k8s.io\n version: v0.0.1\n - manifest: limit-range.addons.k8s.io/v1.5.0.yaml\n name: limit-range.addons.k8s.io\n selector:\n k8s-addon: limit-range.addons.k8s.io\n version: 1.5.0\n - id: pre-k8s-1.6\n kubernetesVersion: \u003c1.6.0\n manifest: dns-controller.addons.k8s.io/pre-k8s-1.6.yaml\n name: dns-controller.addons.k8s.io\n selector:\n k8s-addon: dns-controller.addons.k8s.io\n version: 1.12.1\n - id: k8s-1.6\n kubernetesVersion: '\u003e=1.6.0 \u003c1.12.0'\n manifest: dns-controller.addons.k8s.io/k8s-1.6.yaml\n name: dns-controller.addons.k8s.io\n selector:\n k8s-addon: dns-controller.addons.k8s.io\n version: 1.12.1\n - id: k8s-1.12\n kubernetesVersion: '\u003e=1.12.0'\n manifest: dns-controller.addons.k8s.io/k8s-1.12.yaml\n name: dns-controller.addons.k8s.io\n selector:\n k8s-addon: dns-controller.addons.k8s.io\n version: 1.12.1\n - id: v1.7.0\n kubernetesVersion: '\u003e=1.7.0'\n manifest: storage-aws.addons.k8s.io/v1.7.0.yaml\n name: storage-aws.addons.k8s.io\n selector:\n k8s-addon: storage-aws.addons.k8s.io\n version: 1.7.0\n - id: v1.6.0\n kubernetesVersion: \u003c1.7.0\n manifest: storage-aws.addons.k8s.io/v1.6.0.yaml\n name: storage-aws.addons.k8s.io\n selector:\n k8s-addon: storage-aws.addons.k8s.io\n version: 1.7.0\n - id: k8s-1.12\n kubernetesVersion: '\u003e=1.12.0'\n manifest: networking.projectcalico.org/k8s-1.12.yaml\n name: networking.projectcalico.org\n selector:\n role.kubernetes.io/networking: \"1\"\n version: 3.4.0-kops.4\n - id: k8s-1.7-v3\n kubernetesVersion: '\u003e=1.7.0 \u003c1.12.0'\n manifest: networking.projectcalico.org/k8s-1.7-v3.yaml\n name: networking.projectcalico.org\n selector:\n role.kubernetes.io/networking: \"1\"\n version: 3.4.0-kops.3\n"}}
I0627 16:54:01.575912 22505 executor.go:178] Executing task "Keypair/ca": *fitasks.Keypair {"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"}
I0627 16:54:01.575401 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.6": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.6","Lifecycle":"Sync","Location":"addons/kube-dns.addons.k8s.io/k8s-1.6.yaml","Contents":{"Name":"","Resource":"null\n\n---\n\napiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns-autoscaler\n kubernetes.io/cluster-service: \"true\"\n name: kube-dns-autoscaler\n namespace: kube-system\nspec:\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n scheduler.alpha.kubernetes.io/tolerations: '[{\"key\":\"CriticalAddonsOnly\",\n \"operator\":\"Exists\"}]'\n labels:\n k8s-app: kube-dns-autoscaler\n spec:\n containers:\n - command:\n - /cluster-proportional-autoscaler\n - --namespace=kube-system\n - --configmap=kube-dns-autoscaler\n - --target=Deployment/kube-dns\n - --default-params={\"linear\":{\"coresPerReplica\":256,\"nodesPerReplica\":16,\"preventSinglePointFailure\":true}}\n - --logtostderr=true\n - --v=2\n image: k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.1.2-r2\n name: autoscaler\n resources:\n requests:\n cpu: 20m\n memory: 10Mi\n serviceAccountName: kube-dns-autoscaler\n tolerations:\n - key: CriticalAddonsOnly\n operator: Exists\n\n---\n\napiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns\n kubernetes.io/cluster-service: \"true\"\n name: kube-dns\n namespace: kube-system\nspec:\n selector:\n matchLabels:\n k8s-app: kube-dns\n strategy:\n rollingUpdate:\n maxSurge: 10%\n maxUnavailable: 0\n template:\n metadata:\n annotations:\n prometheus.io/port: \"10055\"\n prometheus.io/scrape: \"true\"\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n scheduler.alpha.kubernetes.io/tolerations: '[{\"key\":\"CriticalAddonsOnly\",\n \"operator\":\"Exists\"}]'\n labels:\n k8s-app: kube-dns\n spec:\n containers:\n - args:\n - --config-dir=/kube-dns-config\n - --dns-port=10053\n - --domain=cluster.local.\n - --v=2\n env:\n - name: PROMETHEUS_PORT\n value: \"10055\"\n image: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.10\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /healthcheck/kubedns\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: kubedns\n ports:\n - containerPort: 10053\n name: dns-local\n protocol: UDP\n - containerPort: 10053\n name: dns-tcp-local\n protocol: TCP\n - containerPort: 10055\n name: metrics\n protocol: TCP\n readinessProbe:\n httpGet:\n path: /readiness\n port: 8081\n scheme: HTTP\n initialDelaySeconds: 3\n timeoutSeconds: 5\n resources:\n limits:\n memory: 170Mi\n requests:\n cpu: 100m\n memory: 70Mi\n volumeMounts:\n - mountPath: /kube-dns-config\n name: kube-dns-config\n - args:\n - -v=2\n - -logtostderr\n - -configDir=/etc/k8s/dns/dnsmasq-nanny\n - -restartDnsmasq=true\n - --\n - -k\n - --cache-size=1000\n - --dns-forward-max=150\n - --no-negcache\n - --log-facility=-\n - --server=/cluster.local/127.0.0.1#10053\n - --server=/in-addr.arpa/127.0.0.1#10053\n - --server=/in6.arpa/127.0.0.1#10053\n image: k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.10\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /healthcheck/dnsmasq\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: dnsmasq\n ports:\n - containerPort: 53\n name: dns\n protocol: UDP\n - containerPort: 53\n name: dns-tcp\n protocol: TCP\n resources:\n requests:\n cpu: 150m\n memory: 20Mi\n volumeMounts:\n - mountPath: /etc/k8s/dns/dnsmasq-nanny\n name: kube-dns-config\n - args:\n - --v=2\n - --logtostderr\n - --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local,5,A\n - --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local,5,A\n image: k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.10\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /metrics\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: sidecar\n ports:\n - containerPort: 10054\n name: metrics\n protocol: TCP\n resources:\n requests:\n cpu: 10m\n memory: 20Mi\n dnsPolicy: Default\n serviceAccountName: kube-dns\n volumes:\n - configMap:\n name: kube-dns\n optional: true\n name: kube-dns-config\n\n---\n\napiVersion: v1\nkind: Service\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns\n kubernetes.io/cluster-service: \"true\"\n kubernetes.io/name: KubeDNS\n name: kube-dns\n namespace: kube-system\nspec:\n clusterIP: 100.64.0.10\n ports:\n - name: dns\n port: 53\n protocol: UDP\n - name: dns-tcp\n port: 53\n protocol: TCP\n selector:\n k8s-app: kube-dns\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n name: kube-dns-autoscaler\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRole\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n name: kube-dns-autoscaler\nrules:\n- apiGroups:\n - \"\"\n resources:\n - nodes\n verbs:\n - list\n- apiGroups:\n - \"\"\n resources:\n - replicationcontrollers/scale\n verbs:\n - get\n - update\n- apiGroups:\n - extensions\n resources:\n - deployments/scale\n - replicasets/scale\n verbs:\n - get\n - update\n- apiGroups:\n - \"\"\n resources:\n - configmaps\n verbs:\n - get\n - create\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n name: kube-dns-autoscaler\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: kube-dns-autoscaler\nsubjects:\n- kind: ServiceAccount\n name: kube-dns-autoscaler\n namespace: kube-system\n"}}
I0627 16:54:01.575939 22505 executor.go:178] Executing task "Keypair/etcd-manager-ca-main": *fitasks.Keypair {"Name":"etcd-manager-ca-main","alternateNames":null,"alternateNameTasks":null,"Lifecycle":null,"Signer":null,"subject":"cn=etcd-manager-ca-main","type":"ca","format":"v1alpha2"}
I0627 16:54:01.575957 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kube"
I0627 16:54:01.575345 22505 executor.go:178] Executing task "Secret/system:scheduler": *fitasks.Secret {"Name":"system:scheduler","Lifecycle":"Sync"}
I0627 16:54:01.575957 22505 executor.go:178] Executing task "Keypair/etcd-peers-ca-events": *fitasks.Keypair {"Name":"etcd-peers-ca-events","alternateNames":null,"alternateNameTasks":null,"Lifecycle":null,"Signer":null,"subject":"cn=etcd-peers-ca-events","type":"ca","format":"v1alpha2"}
I0627 16:54:01.575976 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-pre-k8s-1.6": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-pre-k8s-1.6","Lifecycle":"Sync","Location":"addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml","Contents":{"Name":"","Resource":"apiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n k8s-app: dns-controller\n version: v1.12.0\n name: dns-controller\n namespace: kube-system\nspec:\n replicas: 1\n selector:\n matchLabels:\n k8s-app: dns-controller\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n scheduler.alpha.kubernetes.io/tolerations: '[{\"key\": \"dedicated\", \"value\":\n \"master\"}]'\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n k8s-app: dns-controller\n version: v1.12.0\n spec:\n containers:\n - command:\n - /usr/bin/dns-controller\n - --watch-ingress=false\n - --dns=aws-route53\n - --zone=k8s.domain.com\n - --zone=*/*\n - -v=2\n image: kope/dns-controller:1.12.0\n name: dns-controller\n resources:\n requests:\n cpu: 50m\n memory: 50Mi\n dnsPolicy: Default\n hostNetwork: true\n nodeSelector:\n kubernetes.io/role: master\n"}}
I0627 16:54:01.575965 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.6": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-dns-controller.addons.k8s.io-k8s-1.6","Lifecycle":"Sync","Location":"addons/dns-controller.addons.k8s.io/k8s-1.6.yaml","Contents":{"Name":"","Resource":"apiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n k8s-app: dns-controller\n version: v1.12.0\n name: dns-controller\n namespace: kube-system\nspec:\n replicas: 1\n selector:\n matchLabels:\n k8s-app: dns-controller\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n scheduler.alpha.kubernetes.io/tolerations: '[{\"key\": \"dedicated\", \"value\":\n \"master\"}]'\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n k8s-app: dns-controller\n version: v1.12.0\n spec:\n containers:\n - command:\n - /usr/bin/dns-controller\n - --watch-ingress=false\n - --dns=aws-route53\n - --zone=k8s.domain.com\n - --zone=*/*\n - -v=2\n image: kope/dns-controller:1.12.0\n name: dns-controller\n resources:\n requests:\n cpu: 50m\n memory: 50Mi\n dnsPolicy: Default\n hostNetwork: true\n nodeSelector:\n node-role.kubernetes.io/master: \"\"\n serviceAccount: dns-controller\n tolerations:\n - effect: NoSchedule\n key: node-role.kubernetes.io/master\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n name: dns-controller\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRole\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n name: kops:dns-controller\nrules:\n- apiGroups:\n - \"\"\n resources:\n - endpoints\n - services\n - pods\n - ingress\n - nodes\n verbs:\n - get\n - list\n - watch\n- apiGroups:\n - extensions\n resources:\n - ingresses\n verbs:\n - get\n - list\n - watch\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n k8s-addon: dns-controller.addons.k8s.io\n name: kops:dns-controller\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: kops:dns-controller\nsubjects:\n- apiGroup: rbac.authorization.k8s.io\n kind: User\n name: system:serviceaccount:kube-system:dns-controller\n"}}
I0627 16:54:01.576054 22505 executor.go:178] Executing task "Secret/system:monitoring": *fitasks.Secret {"Name":"system:monitoring","Lifecycle":"Sync"}
I0627 16:54:01.575992 22505 executor.go:178] Executing task "ManagedFile/manifests-etcdmanager-main": *fitasks.ManagedFile {"Name":"manifests-etcdmanager-main","Lifecycle":"Sync","Location":"manifests/etcd/main.yaml","Contents":{"Name":"","Resource":"apiVersion: v1\nkind: Pod\nmetadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n creationTimestamp: null\n labels:\n k8s-app: etcd-manager-main\n name: etcd-manager-main\n namespace: kube-system\nspec:\n containers:\n - command:\n - /bin/sh\n - -c\n - mkfifo /tmp/pipe; (tee -a /var/log/etcd.log \u003c /tmp/pipe \u0026 ) ; exec /etcd-manager\n --backup-store=s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main --client-urls=https://__name__:4001\n --cluster-name=etcd --containerized=true --dns-suffix=.internal.dev-qa.k8s.domain.com\n --etcd-insecure=false --grpc-port=3996 --insecure=false --peer-urls=https://__name__:2380\n --quarantine-client-urls=https://__name__:3994 --v=6 --volume-name-tag=k8s.io/etcd/main\n --volume-provider=aws --volume-tag=k8s.io/etcd/main --volume-tag=k8s.io/role/master=1\n --volume-tag=kubernetes.io/cluster/dev-qa.k8s.domain.com=owned \u003e /tmp/pipe 2\u003e\u00261\n image: kopeio/etcd-manager:3.0.20190516\n name: etcd-manager\n resources:\n requests:\n cpu: 200m\n memory: 100Mi\n securityContext:\n privileged: true\n volumeMounts:\n - mountPath: /rootfs\n name: rootfs\n - mountPath: /etc/hosts\n name: hosts\n - mountPath: /etc/kubernetes/pki/etcd-manager\n name: pki\n - mountPath: /var/log/etcd.log\n name: varlogetcd\n hostNetwork: true\n hostPID: true\n tolerations:\n - key: CriticalAddonsOnly\n operator: Exists\n volumes:\n - hostPath:\n path: /\n type: Directory\n name: rootfs\n - hostPath:\n path: /etc/hosts\n type: File\n name: hosts\n - hostPath:\n path: /etc/kubernetes/pki/etcd-manager-main\n type: DirectoryOrCreate\n name: pki\n - hostPath:\n path: /var/log/etcd.log\n type: FileOrCreate\n name: varlogetcd\nstatus: {}\n"}}
I0627 16:54:01.576075 22505 executor.go:178] Executing task "Secret/system:dns": *fitasks.Secret {"Name":"system:dns","Lifecycle":"Sync"}
I0627 16:54:01.576062 22505 executor.go:178] Executing task "ManagedFile/etcd-cluster-spec-main": *fitasks.ManagedFile {"Name":"etcd-cluster-spec-main","Lifecycle":"Sync","Location":"backups/etcd/main/control/etcd-cluster-spec","Contents":{"Name":"","Resource":"{\n \"memberCount\": 3,\n \"etcdVersion\": \"3.2.24\"\n}"}}
I0627 16:54:01.576125 22505 executor.go:178] Executing task "Secret/kube-proxy": *fitasks.Secret {"Name":"kube-proxy","Lifecycle":"Sync"}
I0627 16:54:01.575307 22505 executor.go:178] Executing task "IAMInstanceProfile/masters.dev-qa.k8s.domain.com": *awstasks.IAMInstanceProfile {"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Shared":false}
I0627 16:54:01.576090 22505 executor.go:178] Executing task "SSHKey/kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b": *awstasks.SSHKey {"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":null}
I0627 16:54:01.576184 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-core.addons.k8s.io": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-core.addons.k8s.io","Lifecycle":"Sync","Location":"addons/core.addons.k8s.io/v1.4.0.yaml","Contents":{"Name":"","Resource":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: kube-system\n"}}
I0627 16:54:01.576268 22505 executor.go:178] Executing task "Secret/system:controller_manager": *fitasks.Secret {"Name":"system:controller_manager","Lifecycle":"Sync"}
I0627 16:54:01.576242 22505 executor.go:178] Executing task "Keypair/etcd-clients-ca": *fitasks.Keypair {"Name":"etcd-clients-ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":null,"Signer":null,"subject":"cn=etcd-clients-ca","type":"ca","format":"v1alpha2"}
I0627 16:54:01.576731 22505 executor.go:178] Executing task "Keypair/etcd-manager-ca-events": *fitasks.Keypair {"Name":"etcd-manager-ca-events","alternateNames":null,"alternateNameTasks":null,"Lifecycle":null,"Signer":null,"subject":"cn=etcd-manager-ca-events","type":"ca","format":"v1alpha2"}
I0627 16:54:01.576085 22505 executor.go:178] Executing task "EBSVolume/c.etcd-events.dev-qa.k8s.domain.com": *awstasks.EBSVolume {"Name":"c.etcd-events.dev-qa.k8s.domain.com","Lifecycle":"Sync","AvailabilityZone":"eu-central-1c","Encrypted":true,"ID":null,"KmsKeyId":null,"SizeGB":20,"Tags":{"k8s.io/etcd/events":"c/a,b,c","k8s.io/role/master":"1","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"VolumeIops":null,"VolumeType":"gp2"}
I0627 16:54:01.575503 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.12": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-k8s-1.12","Lifecycle":"Sync","Location":"addons/kube-dns.addons.k8s.io/k8s-1.12.yaml","Contents":{"Name":"","Resource":"null\n\n---\n\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns-autoscaler\n kubernetes.io/cluster-service: \"true\"\n name: kube-dns-autoscaler\n namespace: kube-system\nspec:\n selector:\n matchLabels:\n k8s-app: kube-dns-autoscaler\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n labels:\n k8s-app: kube-dns-autoscaler\n spec:\n containers:\n - command:\n - /cluster-proportional-autoscaler\n - --namespace=kube-system\n - --configmap=kube-dns-autoscaler\n - --target=Deployment/kube-dns\n - --default-params={\"linear\":{\"coresPerReplica\":256,\"nodesPerReplica\":16,\"preventSinglePointFailure\":true}}\n - --logtostderr=true\n - --v=2\n image: k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.4.0\n name: autoscaler\n resources:\n requests:\n cpu: 20m\n memory: 10Mi\n serviceAccountName: kube-dns-autoscaler\n tolerations:\n - key: CriticalAddonsOnly\n operator: Exists\n\n---\n\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns\n kubernetes.io/cluster-service: \"true\"\n name: kube-dns\n namespace: kube-system\nspec:\n selector:\n matchLabels:\n k8s-app: kube-dns\n strategy:\n rollingUpdate:\n maxSurge: 10%\n maxUnavailable: 0\n template:\n metadata:\n annotations:\n prometheus.io/port: \"10055\"\n prometheus.io/scrape: \"true\"\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n labels:\n k8s-app: kube-dns\n spec:\n containers:\n - args:\n - --config-dir=/kube-dns-config\n - --dns-port=10053\n - --domain=cluster.local.\n - --v=2\n env:\n - name: PROMETHEUS_PORT\n value: \"10055\"\n image: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /healthcheck/kubedns\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: kubedns\n ports:\n - containerPort: 10053\n name: dns-local\n protocol: UDP\n - containerPort: 10053\n name: dns-tcp-local\n protocol: TCP\n - containerPort: 10055\n name: metrics\n protocol: TCP\n readinessProbe:\n httpGet:\n path: /readiness\n port: 8081\n scheme: HTTP\n initialDelaySeconds: 3\n timeoutSeconds: 5\n resources:\n limits:\n memory: 170Mi\n requests:\n cpu: 100m\n memory: 70Mi\n volumeMounts:\n - mountPath: /kube-dns-config\n name: kube-dns-config\n - args:\n - -v=2\n - -logtostderr\n - -configDir=/etc/k8s/dns/dnsmasq-nanny\n - -restartDnsmasq=true\n - --\n - -k\n - --cache-size=1000\n - --dns-forward-max=150\n - --no-negcache\n - --log-facility=-\n - --server=/cluster.local/127.0.0.1#10053\n - --server=/in-addr.arpa/127.0.0.1#10053\n - --server=/in6.arpa/127.0.0.1#10053\n image: k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.13\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /healthcheck/dnsmasq\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: dnsmasq\n ports:\n - containerPort: 53\n name: dns\n protocol: UDP\n - containerPort: 53\n name: dns-tcp\n protocol: TCP\n resources:\n requests:\n cpu: 150m\n memory: 20Mi\n volumeMounts:\n - mountPath: /etc/k8s/dns/dnsmasq-nanny\n name: kube-dns-config\n - args:\n - --v=2\n - --logtostderr\n - --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local,5,A\n - --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local,5,A\n image: k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.13\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /metrics\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: sidecar\n ports:\n - containerPort: 10054\n name: metrics\n protocol: TCP\n resources:\n requests:\n cpu: 10m\n memory: 20Mi\n dnsPolicy: Default\n serviceAccountName: kube-dns\n volumes:\n - configMap:\n name: kube-dns\n optional: true\n name: kube-dns-config\n\n---\n\napiVersion: v1\nkind: Service\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns\n kubernetes.io/cluster-service: \"true\"\n kubernetes.io/name: KubeDNS\n name: kube-dns\n namespace: kube-system\nspec:\n clusterIP: 100.64.0.10\n ports:\n - name: dns\n port: 53\n protocol: UDP\n - name: dns-tcp\n port: 53\n protocol: TCP\n selector:\n k8s-app: kube-dns\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n name: kube-dns-autoscaler\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n name: kube-dns-autoscaler\nrules:\n- apiGroups:\n - \"\"\n resources:\n - nodes\n verbs:\n - list\n - watch\n- apiGroups:\n - \"\"\n resources:\n - replicationcontrollers/scale\n verbs:\n - get\n - update\n- apiGroups:\n - extensions\n - apps\n resources:\n - deployments/scale\n - replicasets/scale\n verbs:\n - get\n - update\n- apiGroups:\n - \"\"\n resources:\n - configmaps\n verbs:\n - get\n - create\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n name: kube-dns-autoscaler\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: kube-dns-autoscaler\nsubjects:\n- kind: ServiceAccount\n name: kube-dns-autoscaler\n namespace: kube-system\n"}}
I0627 16:54:01.576244 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-pre-k8s-1.6": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-kube-dns.addons.k8s.io-pre-k8s-1.6","Lifecycle":"Sync","Location":"addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml","Contents":{"Name":"","Resource":"apiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns-autoscaler\n kubernetes.io/cluster-service: \"true\"\n name: kube-dns-autoscaler\n namespace: kube-system\nspec:\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n scheduler.alpha.kubernetes.io/tolerations: '[{\"key\":\"CriticalAddonsOnly\",\n \"operator\":\"Exists\"}]'\n labels:\n k8s-app: kube-dns-autoscaler\n spec:\n containers:\n - command:\n - /cluster-proportional-autoscaler\n - --namespace=kube-system\n - --configmap=kube-dns-autoscaler\n - --mode=linear\n - --target=Deployment/kube-dns\n - --default-params={\"linear\":{\"coresPerReplica\":256,\"nodesPerReplica\":16,\"min\":2}}\n - --logtostderr=true\n - --v=2\n image: k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.0.0\n name: autoscaler\n resources:\n requests:\n cpu: 20m\n memory: 10Mi\n\n---\n\napiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns\n kubernetes.io/cluster-service: \"true\"\n name: kube-dns\n namespace: kube-system\nspec:\n selector:\n matchLabels:\n k8s-app: kube-dns\n strategy:\n rollingUpdate:\n maxSurge: 10%\n maxUnavailable: 0\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n scheduler.alpha.kubernetes.io/tolerations: '[{\"key\":\"CriticalAddonsOnly\",\n \"operator\":\"Exists\"}]'\n labels:\n k8s-app: kube-dns\n spec:\n containers:\n - args:\n - --domain=cluster.local.\n - --dns-port=10053\n - --config-map=kube-dns\n - --v=2\n env:\n - name: PROMETHEUS_PORT\n value: \"10055\"\n image: k8s.gcr.io/kubedns-amd64:1.9\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /healthz-kubedns\n port: 8080\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: kubedns\n ports:\n - containerPort: 10053\n name: dns-local\n protocol: UDP\n - containerPort: 10053\n name: dns-tcp-local\n protocol: TCP\n - containerPort: 10055\n name: metrics\n protocol: TCP\n readinessProbe:\n httpGet:\n path: /readiness\n port: 8081\n scheme: HTTP\n initialDelaySeconds: 3\n timeoutSeconds: 5\n resources:\n limits:\n memory: 170Mi\n requests:\n cpu: 100m\n memory: 70Mi\n - args:\n - --cache-size=1000\n - --dns-forward-max=150\n - --no-resolv\n - --server=127.0.0.1#10053\n - --log-facility=-\n image: k8s.gcr.io/k8s-dns-dnsmasq-amd64:1.14.10\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /healthz-dnsmasq\n port: 8080\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: dnsmasq\n ports:\n - containerPort: 53\n name: dns\n protocol: UDP\n - containerPort: 53\n name: dns-tcp\n protocol: TCP\n resources:\n requests:\n cpu: 150m\n memory: 10Mi\n - args:\n - --v=2\n - --logtostderr\n image: k8s.gcr.io/dnsmasq-metrics-amd64:1.0\n livenessProbe:\n failureThreshold: 5\n httpGet:\n path: /metrics\n port: 10054\n scheme: HTTP\n initialDelaySeconds: 60\n successThreshold: 1\n timeoutSeconds: 5\n name: dnsmasq-metrics\n ports:\n - containerPort: 10054\n name: metrics\n protocol: TCP\n resources:\n requests:\n memory: 10Mi\n - args:\n - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1 \u003e/dev/null\n - --url=/healthz-dnsmasq\n - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1:10053 \u003e/dev/null\n - --url=/healthz-kubedns\n - --port=8080\n - --quiet\n image: k8s.gcr.io/exechealthz-amd64:1.2\n name: healthz\n ports:\n - containerPort: 8080\n protocol: TCP\n resources:\n limits:\n memory: 50Mi\n requests:\n cpu: 10m\n memory: 50Mi\n dnsPolicy: Default\n\n---\n\napiVersion: v1\nkind: Service\nmetadata:\n labels:\n k8s-addon: kube-dns.addons.k8s.io\n k8s-app: kube-dns\n kubernetes.io/cluster-service: \"true\"\n kubernetes.io/name: KubeDNS\n name: kube-dns\n namespace: kube-system\nspec:\n clusterIP: 100.64.0.10\n ports:\n - name: dns\n port: 53\n protocol: UDP\n - name: dns-tcp\n port: 53\n protocol: TCP\n selector:\n k8s-app: kube-dns\n"}}
I0627 16:54:01.575281 22505 executor.go:178] Executing task "ManagedFile/manifests-etcdmanager-events": *fitasks.ManagedFile {"Name":"manifests-etcdmanager-events","Lifecycle":"Sync","Location":"manifests/etcd/events.yaml","Contents":{"Name":"","Resource":"apiVersion: v1\nkind: Pod\nmetadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n creationTimestamp: null\n labels:\n k8s-app: etcd-manager-events\n name: etcd-manager-events\n namespace: kube-system\nspec:\n containers:\n - command:\n - /bin/sh\n - -c\n - mkfifo /tmp/pipe; (tee -a /var/log/etcd.log \u003c /tmp/pipe \u0026 ) ; exec /etcd-manager\n --backup-store=s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events --client-urls=https://__name__:4002\n --cluster-name=etcd-events --containerized=true --dns-suffix=.internal.dev-qa.k8s.domain.com\n --etcd-insecure=false --grpc-port=3997 --insecure=false --peer-urls=https://__name__:2381\n --quarantine-client-urls=https://__name__:3995 --v=6 --volume-name-tag=k8s.io/etcd/events\n --volume-provider=aws --volume-tag=k8s.io/etcd/events --volume-tag=k8s.io/role/master=1\n --volume-tag=kubernetes.io/cluster/dev-qa.k8s.domain.com=owned \u003e /tmp/pipe 2\u003e\u00261\n image: kopeio/etcd-manager:3.0.20190516\n name: etcd-manager\n resources:\n requests:\n cpu: 100m\n memory: 100Mi\n securityContext:\n privileged: true\n volumeMounts:\n - mountPath: /rootfs\n name: rootfs\n - mountPath: /etc/hosts\n name: hosts\n - mountPath: /etc/kubernetes/pki/etcd-manager\n name: pki\n - mountPath: /var/log/etcd.log\n name: varlogetcd\n hostNetwork: true\n hostPID: true\n tolerations:\n - key: CriticalAddonsOnly\n operator: Exists\n volumes:\n - hostPath:\n path: /\n type: Directory\n name: rootfs\n - hostPath:\n path: /etc/hosts\n type: File\n name: hosts\n - hostPath:\n path: /etc/kubernetes/pki/etcd-manager-events\n type: DirectoryOrCreate\n name: pki\n - hostPath:\n path: /var/log/etcd-events.log\n type: FileOrCreate\n name: varlogetcd\nstatus: {}\n"}}
I0627 16:54:01.575288 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-rbac.addons.k8s.io-k8s-1.8": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-rbac.addons.k8s.io-k8s-1.8","Lifecycle":"Sync","Location":"addons/rbac.addons.k8s.io/k8s-1.8.yaml","Contents":{"Name":"","Resource":"apiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n addonmanager.kubernetes.io/mode: Reconcile\n k8s-addon: rbac.addons.k8s.io\n kubernetes.io/cluster-service: \"true\"\n name: kubelet-cluster-admin\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: system:node\nsubjects:\n- apiGroup: rbac.authorization.k8s.io\n kind: User\n name: kubelet\n"}}
I0627 16:54:01.577732 22505 executor.go:178] Executing task "DNSZone/k8s.domain.com": *awstasks.DNSZone {"Name":"k8s.domain.com","Lifecycle":"Sync","DNSName":"k8s.domain.com","ZoneID":null,"Private":null,"PrivateVPC":null}
I0627 16:54:01.575263 22505 executor.go:178] Executing task "ManagedFile/etcd-cluster-spec-events": *fitasks.ManagedFile {"Name":"etcd-cluster-spec-events","Lifecycle":"Sync","Location":"backups/etcd/events/control/etcd-cluster-spec","Contents":{"Name":"","Resource":"{\n \"memberCount\": 3,\n \"etcdVersion\": \"3.2.24\"\n}"}}
I0627 16:54:01.575205 22505 executor.go:178] Executing task "IAMRole/masters.dev-qa.k8s.domain.com": *awstasks.IAMRole {"ID":null,"Lifecycle":"Sync","Name":"masters.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"masters"}
I0627 16:54:01.575443 22505 executor.go:178] Executing task "IAMRole/bastions.dev-qa.k8s.domain.com": *awstasks.IAMRole {"ID":null,"Lifecycle":"Sync","Name":"bastions.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"bastions"}
I0627 16:54:01.575316 22505 executor.go:178] Executing task "IAMRole/nodes.dev-qa.k8s.domain.com": *awstasks.IAMRole {"ID":null,"Lifecycle":"Sync","Name":"nodes.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"nodes"}
I0627 16:54:01.578059 22505 executor.go:178] Executing task "Secret/kubelet": *fitasks.Secret {"Name":"kubelet","Lifecycle":"Sync"}
I0627 16:54:01.575660 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.12": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.12","Lifecycle":"Sync","Location":"addons/networking.projectcalico.org/k8s-1.12.yaml","Contents":{"Name":"","Resource":"apiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: felixconfigurations.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: FelixConfiguration\n plural: felixconfigurations\n singular: felixconfiguration\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: bgppeers.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: BGPPeer\n plural: bgppeers\n singular: bgppeer\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: bgpconfigurations.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: BGPConfiguration\n plural: bgpconfigurations\n singular: bgpconfiguration\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: ippools.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: IPPool\n plural: ippools\n singular: ippool\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: hostendpoints.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: HostEndpoint\n plural: hostendpoints\n singular: hostendpoint\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: clusterinformations.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: ClusterInformation\n plural: clusterinformations\n singular: clusterinformation\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: globalnetworkpolicies.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: GlobalNetworkPolicy\n plural: globalnetworkpolicies\n singular: globalnetworkpolicy\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: globalnetworksets.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: GlobalNetworkSet\n plural: globalnetworksets\n singular: globalnetworkset\n scope: Cluster\n version: v1\n\n---\n\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: networkpolicies.crd.projectcalico.org\nspec:\n group: crd.projectcalico.org\n names:\n kind: NetworkPolicy\n plural: networkpolicies\n singular: networkpolicy\n scope: Namespaced\n version: v1\n\n---\n\napiVersion: v1\ndata:\n calico_backend: bird\n cni_network_config: |-\n {\n \"name\": \"k8s-pod-network\",\n \"cniVersion\": \"0.3.0\",\n \"plugins\": [\n {\n \"type\": \"calico\",\n \"log_level\": \"info\",\n \"datastore_type\": \"kubernetes\",\n \"nodename\": \"__KUBERNETES_NODE_NAME__\",\n \"mtu\": __CNI_MTU__,\n \"ipam\": {\n \"type\": \"host-local\",\n \"subnet\": \"usePodCidr\"\n },\n \"policy\": {\n \"type\": \"k8s\"\n },\n \"kubernetes\": {\n \"kubeconfig\": \"/etc/cni/net.d/__KUBECONFIG_FILENAME__\"\n }\n },\n {\n \"type\": \"portmap\",\n \"snat\": true,\n \"capabilities\": {\"portMappings\": true}\n }\n ]\n }\n typha_service_name: none\n veth_mtu: \"1440\"\nkind: ConfigMap\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-config\n namespace: kube-system\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-node\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-node\nrules:\n- apiGroups:\n - \"\"\n resources:\n - pods\n - nodes\n - namespaces\n verbs:\n - get\n- apiGroups:\n - \"\"\n resources:\n - endpoints\n - services\n verbs:\n - watch\n - list\n - get\n- apiGroups:\n - \"\"\n resources:\n - nodes/status\n verbs:\n - patch\n - update\n- apiGroups:\n - networking.k8s.io\n resources:\n - networkpolicies\n verbs:\n - watch\n - list\n- apiGroups:\n - \"\"\n resources:\n - pods\n - namespaces\n - serviceaccounts\n verbs:\n - list\n - watch\n- apiGroups:\n - \"\"\n resources:\n - pods/status\n verbs:\n - patch\n- apiGroups:\n - crd.projectcalico.org\n resources:\n - globalfelixconfigs\n - felixconfigurations\n - bgppeers\n - globalbgpconfigs\n - bgpconfigurations\n - ippools\n - globalnetworkpolicies\n - globalnetworksets\n - networkpolicies\n - clusterinformations\n - hostendpoints\n verbs:\n - get\n - list\n - watch\n- apiGroups:\n - crd.projectcalico.org\n resources:\n - ippools\n - felixconfigurations\n - clusterinformations\n verbs:\n - create\n - update\n- apiGroups:\n - \"\"\n resources:\n - nodes\n verbs:\n - get\n - list\n - watch\n- apiGroups:\n - crd.projectcalico.org\n resources:\n - bgpconfigurations\n - bgppeers\n verbs:\n - create\n - update\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-node\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: calico-node\nsubjects:\n- kind: ServiceAccount\n name: calico-node\n namespace: kube-system\n\n---\n\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n labels:\n k8s-app: calico-kube-controllers\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\n namespace: kube-system\nspec:\n replicas: 0\n selector:\n matchLabels:\n k8s-app: calico-kube-controllers\n strategy:\n type: Recreate\n template:\n metadata:\n labels:\n k8s-app: calico-kube-controllers\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\n namespace: kube-system\n spec:\n containers:\n - image: quay.io/calico/kube-controllers:v3.4.0\n name: calico-kube-controllers\n initContainers:\n - image: calico/upgrade:v1.0.5\n name: migrate\n\n---\n\napiVersion: apps/v1\nkind: DaemonSet\nmetadata:\n labels:\n k8s-app: calico-node\n role.kubernetes.io/networking: \"1\"\n name: calico-node\n namespace: kube-system\nspec:\n selector:\n matchLabels:\n k8s-app: calico-node\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n labels:\n k8s-app: calico-node\n role.kubernetes.io/networking: \"1\"\n spec:\n containers:\n - env:\n - name: DATASTORE_TYPE\n value: kubernetes\n - name: FELIX_TYPHAK8SSERVICENAME\n valueFrom:\n configMapKeyRef:\n key: typha_service_name\n name: calico-config\n - name: WAIT_FOR_DATASTORE\n value: \"true\"\n - name: NODENAME\n valueFrom:\n fieldRef:\n fieldPath: spec.nodeName\n - name: CALICO_NETWORKING_BACKEND\n valueFrom:\n configMapKeyRef:\n key: calico_backend\n name: calico-config\n - name: CLUSTER_TYPE\n value: kops,bgp\n - name: IP\n value: autodetect\n - name: CALICO_IPV4POOL_IPIP\n value: always\n - name: FELIX_IPINIPMTU\n valueFrom:\n configMapKeyRef:\n key: veth_mtu\n name: calico-config\n - name: CALICO_IPV4POOL_CIDR\n value: 100.96.0.0/11\n - name: CALICO_DISABLE_FILE_LOGGING\n value: \"true\"\n - name: FELIX_DEFAULTENDPOINTTOHOSTACTION\n value: ACCEPT\n - name: FELIX_IPV6SUPPORT\n value: \"false\"\n - name: FELIX_LOGSEVERITYSCREEN\n value: info\n - name: FELIX_HEALTHENABLED\n value: \"true\"\n - name: FELIX_PROMETHEUSMETRICSENABLED\n value: \"false\"\n - name: FELIX_PROMETHEUSMETRICSPORT\n value: \"9091\"\n - name: FELIX_PROMETHEUSGOMETRICSENABLED\n value: \"true\"\n - name: FELIX_PROMETHEUSPROCESSMETRICSENABLED\n value: \"true\"\n image: quay.io/calico/node:v3.4.0\n livenessProbe:\n failureThreshold: 6\n httpGet:\n host: localhost\n path: /liveness\n port: 9099\n initialDelaySeconds: 10\n periodSeconds: 10\n name: calico-node\n readinessProbe:\n exec:\n command:\n - /bin/calico-node\n - -bird-ready\n - -felix-ready\n periodSeconds: 10\n resources:\n requests:\n cpu: 10m\n securityContext:\n privileged: true\n volumeMounts:\n - mountPath: /lib/modules\n name: lib-modules\n readOnly: true\n - mountPath: /run/xtables.lock\n name: xtables-lock\n readOnly: false\n - mountPath: /var/run/calico\n name: var-run-calico\n readOnly: false\n - mountPath: /var/lib/calico\n name: var-lib-calico\n readOnly: false\n - mountPath: /etc/hosts\n name: etc-hosts\n readOnly: true\n hostNetwork: true\n initContainers:\n - command:\n - /install-cni.sh\n env:\n - name: CNI_CONF_NAME\n value: 10-calico.conflist\n - name: CNI_NETWORK_CONFIG\n valueFrom:\n configMapKeyRef:\n key: cni_network_config\n name: calico-config\n - name: KUBERNETES_NODE_NAME\n valueFrom:\n fieldRef:\n fieldPath: spec.nodeName\n - name: CNI_MTU\n valueFrom:\n configMapKeyRef:\n key: veth_mtu\n name: calico-config\n - name: SLEEP\n value: \"false\"\n image: quay.io/calico/cni:v3.4.0\n name: install-cni\n volumeMounts:\n - mountPath: /host/opt/cni/bin\n name: cni-bin-dir\n - mountPath: /host/etc/cni/net.d\n name: cni-net-dir\n serviceAccountName: calico-node\n terminationGracePeriodSeconds: 0\n tolerations:\n - effect: NoSchedule\n operator: Exists\n - key: CriticalAddonsOnly\n operator: Exists\n - effect: NoExecute\n operator: Exists\n volumes:\n - hostPath:\n path: /lib/modules\n name: lib-modules\n - hostPath:\n path: /var/run/calico\n name: var-run-calico\n - hostPath:\n path: /var/lib/calico\n name: var-lib-calico\n - hostPath:\n path: /run/xtables.lock\n type: FileOrCreate\n name: xtables-lock\n - hostPath:\n path: /opt/cni/bin\n name: cni-bin-dir\n - hostPath:\n path: /etc/cni/net.d\n name: cni-net-dir\n - hostPath:\n path: /etc/hosts\n name: etc-hosts\n updateStrategy:\n rollingUpdate:\n maxUnavailable: 1\n type: RollingUpdate\n\n---\n\nnull\n"}}
I0627 16:54:01.575300 22505 executor.go:178] Executing task "dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.7-v3": *fitasks.ManagedFile {"Name":"dev-qa.k8s.domain.com-addons-networking.projectcalico.org-k8s-1.7-v3","Lifecycle":"Sync","Location":"addons/networking.projectcalico.org/k8s-1.7-v3.yaml","Contents":{"Name":"","Resource":"apiVersion: v1\ndata:\n calico_backend: bird\n cni_network_config: |-\n {\n \"name\": \"k8s-pod-network\",\n \"cniVersion\": \"0.3.0\",\n \"plugins\": [\n {\n \"type\": \"calico\",\n \"etcd_endpoints\": \"__ETCD_ENDPOINTS__\",\n \"etcd_ca_cert_file\": \"/srv/kubernetes/calico/ca.pem\",\n \"etcd_cert_file\": \"/srv/kubernetes/calico/calico-client.pem\",\n \"etcd_key_file\": \"/srv/kubernetes/calico/calico-client-key.pem\",\n \"etcd_scheme\": \"https\",\n \"log_level\": \"info\",\n \"ipam\": {\n \"type\": \"calico-ipam\"\n },\n \"policy\": {\n \"type\": \"k8s\"\n },\n \"kubernetes\": {\n \"kubeconfig\": \"/etc/cni/net.d/__KUBECONFIG_FILENAME__\"\n }\n },\n {\n \"type\": \"portmap\",\n \"snat\": true,\n \"capabilities\": {\"portMappings\": true}\n }\n ]\n }\n etcd_endpoints: https://etcd-a.internal.dev-qa.k8s.domain.com:4001,https://etcd-b.internal.dev-qa.k8s.domain.com:4001,https://etcd-c.internal.dev-qa.k8s.domain.com:4001\nkind: ConfigMap\nmetadata:\n name: calico-config\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRole\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-node\nrules:\n- apiGroups:\n - \"\"\n resources:\n - pods\n - nodes\n - namespaces\n verbs:\n - get\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-node\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-node\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: calico-node\nsubjects:\n- kind: ServiceAccount\n name: calico-node\n namespace: kube-system\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRole\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\nrules:\n- apiGroups:\n - \"\"\n - extensions\n resources:\n - pods\n - namespaces\n - networkpolicies\n - nodes\n verbs:\n - watch\n - list\n- apiGroups:\n - networking.k8s.io\n resources:\n - networkpolicies\n verbs:\n - watch\n - list\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: calico-kube-controllers\nsubjects:\n- kind: ServiceAccount\n name: calico-kube-controllers\n namespace: kube-system\n\n---\n\napiVersion: extensions/v1beta1\nkind: DaemonSet\nmetadata:\n labels:\n k8s-app: calico-node\n role.kubernetes.io/networking: \"1\"\n name: calico-node\n namespace: kube-system\nspec:\n selector:\n matchLabels:\n k8s-app: calico-node\n template:\n metadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n labels:\n k8s-app: calico-node\n role.kubernetes.io/networking: \"1\"\n spec:\n containers:\n - env:\n - name: ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n - name: CALICO_NETWORKING_BACKEND\n valueFrom:\n configMapKeyRef:\n key: calico_backend\n name: calico-config\n - name: CLUSTER_TYPE\n value: kops,bgp\n - name: CALICO_DISABLE_FILE_LOGGING\n value: \"true\"\n - name: CALICO_K8S_NODE_REF\n valueFrom:\n fieldRef:\n fieldPath: spec.nodeName\n - name: FELIX_DEFAULTENDPOINTTOHOSTACTION\n value: ACCEPT\n - name: CALICO_IPV4POOL_CIDR\n value: 100.96.0.0/11\n - name: CALICO_IPV4POOL_IPIP\n value: always\n - name: FELIX_IPV6SUPPORT\n value: \"false\"\n - name: FELIX_LOGSEVERITYSCREEN\n value: info\n - name: FELIX_PROMETHEUSMETRICSENABLED\n value: \"false\"\n - name: FELIX_PROMETHEUSMETRICSPORT\n value: \"9091\"\n - name: FELIX_PROMETHEUSGOMETRICSENABLED\n value: \"true\"\n - name: FELIX_PROMETHEUSPROCESSMETRICSENABLED\n value: \"true\"\n - name: IP\n value: autodetect\n - name: FELIX_HEALTHENABLED\n value: \"true\"\n image: quay.io/calico/node:v3.4.0\n livenessProbe:\n failureThreshold: 6\n httpGet:\n host: localhost\n path: /liveness\n port: 9099\n initialDelaySeconds: 10\n periodSeconds: 10\n name: calico-node\n readinessProbe:\n exec:\n command:\n - /bin/calico-node\n - -bird-ready\n - -felix-ready\n periodSeconds: 10\n resources:\n requests:\n cpu: 10m\n securityContext:\n privileged: true\n volumeMounts:\n - mountPath: /lib/modules\n name: lib-modules\n readOnly: true\n - mountPath: /var/run/calico\n name: var-run-calico\n readOnly: false\n - mountPath: /var/lib/calico\n name: var-lib-calico\n readOnly: false\n - mountPath: /etc/hosts\n name: etc-hosts\n readOnly: true\n - mountPath: /certs\n name: calico\n readOnly: true\n - command:\n - /install-cni.sh\n env:\n - name: CNI_CONF_NAME\n value: 10-calico.conflist\n - name: ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CNI_NETWORK_CONFIG\n valueFrom:\n configMapKeyRef:\n key: cni_network_config\n name: calico-config\n image: quay.io/calico/cni:v3.4.0\n name: install-cni\n resources:\n requests:\n cpu: 10m\n volumeMounts:\n - mountPath: /host/opt/cni/bin\n name: cni-bin-dir\n - mountPath: /host/etc/cni/net.d\n name: cni-net-dir\n - mountPath: /etc/hosts\n name: etc-hosts\n readOnly: true\n hostNetwork: true\n initContainers:\n - command:\n - /bin/sh\n - -c\n - /node-init-container.sh\n env:\n - name: CALICO_ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CALICO_APIV1_DATASTORE_TYPE\n value: etcdv2\n - name: CALICO_APIV1_ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CALICO_ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: CALICO_ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: CALICO_ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n - name: CALICO_APIV1_ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: CALICO_APIV1_ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: CALICO_APIV1_ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n image: calico/upgrade:v1.0.5\n name: migrate\n volumeMounts:\n - mountPath: /etc/hosts\n name: etc-hosts\n readOnly: true\n - mountPath: /certs\n name: calico\n readOnly: true\n serviceAccountName: calico-node\n terminationGracePeriodSeconds: 0\n tolerations:\n - effect: NoSchedule\n operator: Exists\n - key: CriticalAddonsOnly\n operator: Exists\n - effect: NoExecute\n operator: Exists\n volumes:\n - hostPath:\n path: /lib/modules\n name: lib-modules\n - hostPath:\n path: /var/run/calico\n name: var-run-calico\n - hostPath:\n path: /var/lib/calico\n name: var-lib-calico\n - hostPath:\n path: /opt/cni/bin\n name: cni-bin-dir\n - hostPath:\n path: /etc/cni/net.d\n name: cni-net-dir\n - hostPath:\n path: /etc/hosts\n name: etc-hosts\n - hostPath:\n path: /srv/kubernetes/calico\n name: calico\n updateStrategy:\n rollingUpdate:\n maxUnavailable: 1\n type: RollingUpdate\n\n---\n\napiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n annotations:\n scheduler.alpha.kubernetes.io/critical-pod: \"\"\n labels:\n k8s-app: calico-kube-controllers\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\n namespace: kube-system\nspec:\n replicas: 1\n strategy:\n type: Recreate\n template:\n metadata:\n labels:\n k8s-app: calico-kube-controllers\n role.kubernetes.io/networking: \"1\"\n name: calico-kube-controllers\n namespace: kube-system\n spec:\n containers:\n - env:\n - name: ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: ENABLED_CONTROLLERS\n value: policy,profile,workloadendpoint,node\n - name: ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n image: quay.io/calico/kube-controllers:v3.4.0\n name: calico-kube-controllers\n readinessProbe:\n exec:\n command:\n - /usr/bin/check-status\n - -r\n resources:\n requests:\n cpu: 10m\n volumeMounts:\n - mountPath: /certs\n name: calico\n readOnly: true\n hostNetwork: true\n initContainers:\n - command:\n - /bin/sh\n - -c\n - /controller-init.sh\n env:\n - name: CALICO_ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CALICO_APIV1_DATASTORE_TYPE\n value: etcdv2\n - name: CALICO_APIV1_ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CALICO_ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: CALICO_ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: CALICO_ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n - name: CALICO_APIV1_ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: CALICO_APIV1_ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: CALICO_APIV1_ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n image: calico/upgrade:v1.0.5\n name: migrate\n volumeMounts:\n - mountPath: /etc/hosts\n name: etc-hosts\n readOnly: true\n - mountPath: /certs\n name: calico\n readOnly: true\n serviceAccountName: calico-kube-controllers\n tolerations:\n - key: CriticalAddonsOnly\n operator: Exists\n - effect: NoSchedule\n key: node-role.kubernetes.io/master\n volumes:\n - hostPath:\n path: /etc/hosts\n name: etc-hosts\n - hostPath:\n path: /srv/kubernetes/calico\n name: calico\n\n---\n\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-upgrade-job\n namespace: kube-system\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRole\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-upgrade-job\nrules:\n- apiGroups:\n - extensions\n resources:\n - daemonsets\n - daemonsets/status\n verbs:\n - get\n - list\n - watch\n\n---\n\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRoleBinding\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-upgrade-job\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: calico-upgrade-job\nsubjects:\n- kind: ServiceAccount\n name: calico-upgrade-job\n namespace: kube-system\n\n---\n\napiVersion: batch/v1\nkind: Job\nmetadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n name: calico-complete-upgrade-v331\n namespace: kube-system\nspec:\n template:\n metadata:\n labels:\n role.kubernetes.io/networking: \"1\"\n spec:\n containers:\n - command:\n - /bin/sh\n - -c\n - /completion-job.sh\n env:\n - name: EXPECTED_NODE_IMAGE\n value: quay.io/calico/node:v3.4.0\n - name: CALICO_ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CALICO_APIV1_DATASTORE_TYPE\n value: etcdv2\n - name: CALICO_APIV1_ETCD_ENDPOINTS\n valueFrom:\n configMapKeyRef:\n key: etcd_endpoints\n name: calico-config\n - name: CALICO_ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: CALICO_ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: CALICO_ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n - name: CALICO_APIV1_ETCD_CERT_FILE\n value: /certs/calico-client.pem\n - name: CALICO_APIV1_ETCD_KEY_FILE\n value: /certs/calico-client-key.pem\n - name: CALICO_APIV1_ETCD_CA_CERT_FILE\n value: /certs/ca.pem\n image: calico/upgrade:v1.0.5\n name: migrate-completion\n volumeMounts:\n - mountPath: /etc/hosts\n name: etc-hosts\n readOnly: true\n - mountPath: /certs\n name: calico\n readOnly: true\n hostNetwork: true\n restartPolicy: OnFailure\n serviceAccountName: calico-upgrade-job\n volumes:\n - hostPath:\n path: /etc/hosts\n name: etc-hosts\n - hostPath:\n path: /srv/kubernetes/calico\n name: calico\n"}}
I0627 16:54:01.580543 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/keyset.yaml"
I0627 16:54:01.581296 22505 request_logger.go:45] AWS request: iam/GetInstanceProfile
I0627 16:54:01.581638 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/admin"
I0627 16:54:01.585937 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/dns-controller.addons.k8s.io/k8s-1.12.yaml"
I0627 16:54:01.587299 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/storage-aws.addons.k8s.io/v1.6.0.yaml"
I0627 16:54:01.588151 22505 request_logger.go:45] AWS request: ec2/DescribeVolumes
I0627 16:54:01.588469 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:logging"
I0627 16:54:01.589387 22505 request_logger.go:45] AWS request: ec2/DescribeVolumes
I0627 16:54:01.590246 22505 request_logger.go:45] AWS request: ec2/DescribeVolumes
I0627 16:54:01.591650 22505 request_logger.go:45] AWS request: ec2/DescribeVolumes
I0627 16:54:01.592169 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/limit-range.addons.k8s.io/v1.5.0.yaml"
I0627 16:54:01.592826 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/keyset.yaml"
I0627 16:54:01.593554 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml"
I0627 16:54:01.594483 22505 request_logger.go:45] AWS request: ec2/DescribeVolumes
I0627 16:54:01.597580 22505 request_logger.go:45] AWS request: ec2/DescribeDhcpOptions
I0627 16:54:01.598297 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/storage-aws.addons.k8s.io/v1.7.0.yaml"
I0627 16:54:01.600787 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/bootstrap-channel.yaml"
I0627 16:54:01.601213 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml"
I0627 16:54:01.611498 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/kube-dns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:54:01.612049 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/keyset.yaml"
I0627 16:54:01.612970 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:scheduler"
I0627 16:54:01.613443 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/keyset.yaml"
I0627 16:54:01.614534 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:54:01.616628 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/dns-controller.addons.k8s.io/k8s-1.6.yaml"
I0627 16:54:01.617009 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:monitoring"
I0627 16:54:01.618571 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/manifests/etcd/main.yaml"
I0627 16:54:01.618960 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:dns"
I0627 16:54:01.619435 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main/control/etcd-cluster-spec"
I0627 16:54:01.622609 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kube-proxy"
I0627 16:54:01.623820 22505 request_logger.go:45] AWS request: iam/GetInstanceProfile
I0627 16:54:01.624748 22505 sshkey.go:109] Computed SSH key fingerprint as "c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"
I0627 16:54:01.625248 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/core.addons.k8s.io/v1.4.0.yaml"
I0627 16:54:01.625737 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:controller_manager"
I0627 16:54:01.626356 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/keyset.yaml"
I0627 16:54:01.626905 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/keyset.yaml"
I0627 16:54:01.627603 22505 request_logger.go:45] AWS request: ec2/DescribeVolumes
I0627 16:54:01.635821 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/kube-dns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:54:01.639346 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:54:01.640876 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/manifests/etcd/events.yaml"
I0627 16:54:01.641534 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/rbac.addons.k8s.io/k8s-1.8.yaml"
I0627 16:54:01.641934 22505 request_logger.go:45] AWS request: route53/ListHostedZonesByName
I0627 16:54:01.642370 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events/control/etcd-cluster-spec"
I0627 16:54:01.642840 22505 request_logger.go:45] AWS request: iam/GetRole
I0627 16:54:01.643758 22505 request_logger.go:45] AWS request: iam/GetRole
I0627 16:54:01.644279 22505 request_logger.go:45] AWS request: iam/GetRole
I0627 16:54:01.647822 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kubelet"
I0627 16:54:01.664724 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/networking.projectcalico.org/k8s-1.12.yaml"
I0627 16:54:01.681727 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/addons/networking.projectcalico.org/k8s-1.7-v3.yaml"
I0627 16:54:01.698770 22505 request_logger.go:45] AWS request: ec2/DescribeKeyPairs
I0627 16:54:01.792657 22505 vpc.go:155] Creating VPC with CIDR: "172.31.239.0/24"
I0627 16:54:01.793342 22505 request_logger.go:45] AWS request: ec2/CreateVpc
I0627 16:54:01.800277 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kube"
I0627 16:54:01.951251 22505 request_logger.go:45] AWS request: route53/GetHostedZone
I0627 16:54:02.000079 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/keyset.yaml", falling back to directory-list method
I0627 16:54:02.001190 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/"
I0627 16:54:02.015805 22505 ebsvolume.go:140] Creating PersistentVolume with Name:"c.etcd-main.dev-qa.k8s.domain.com"
I0627 16:54:02.016696 22505 request_logger.go:45] AWS request: ec2/CreateVolume
I0627 16:54:02.182229 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/admin"
I0627 16:54:02.216761 22505 ebsvolume.go:140] Creating PersistentVolume with Name:"a.etcd-events.dev-qa.k8s.domain.com"
I0627 16:54:02.217538 22505 request_logger.go:45] AWS request: ec2/CreateVolume
I0627 16:54:02.365988 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/dns-controller.addons.k8s.io/k8s-1.12.yaml"
I0627 16:54:02.366686 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/dns-controller.addons.k8s.io/k8s-1.12.yaml" SSE="AES256" ACL=""
I0627 16:54:02.548961 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/storage-aws.addons.k8s.io/v1.6.0.yaml"
I0627 16:54:02.549826 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/storage-aws.addons.k8s.io/v1.6.0.yaml" SSE="AES256" ACL=""
I0627 16:54:02.729691 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:logging"
I0627 16:54:02.753879 22505 ebsvolume.go:140] Creating PersistentVolume with Name:"a.etcd-main.dev-qa.k8s.domain.com"
I0627 16:54:02.754672 22505 request_logger.go:45] AWS request: ec2/CreateVolume
I0627 16:54:02.790785 22505 ebsvolume.go:140] Creating PersistentVolume with Name:"b.etcd-events.dev-qa.k8s.domain.com"
I0627 16:54:02.791577 22505 request_logger.go:45] AWS request: ec2/CreateVolume
I0627 16:54:02.792749 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/keyset.yaml", falling back to directory-list method
I0627 16:54:02.793546 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/"
I0627 16:54:02.792857 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/keyset.yaml", falling back to directory-list method
I0627 16:54:02.792888 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/limit-range.addons.k8s.io/v1.5.0.yaml"
I0627 16:54:02.792924 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/core.addons.k8s.io/v1.4.0.yaml"
I0627 16:54:02.792969 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main/control/etcd-cluster-spec"
I0627 16:54:02.792974 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/rbac.addons.k8s.io/k8s-1.8.yaml"
I0627 16:54:02.794423 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:dns"
I0627 16:54:02.794548 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/keyset.yaml", falling back to directory-list method
I0627 16:54:02.794928 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:monitoring"
I0627 16:54:02.795756 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:54:02.795889 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:controller_manager"
I0627 16:54:02.796372 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/"
I0627 16:54:02.797241 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/limit-range.addons.k8s.io/v1.5.0.yaml" SSE="AES256" ACL=""
I0627 16:54:02.797784 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/core.addons.k8s.io/v1.4.0.yaml" SSE="AES256" ACL=""
I0627 16:54:02.798202 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/backups/etcd/main/control/etcd-cluster-spec" SSE="AES256" ACL=""
I0627 16:54:02.798531 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/rbac.addons.k8s.io/k8s-1.8.yaml" SSE="AES256" ACL=""
I0627 16:54:02.799455 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/"
I0627 16:54:02.801833 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/kube-dns.addons.k8s.io/pre-k8s-1.6.yaml" SSE="AES256" ACL=""
I0627 16:54:02.804953 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml", falling back to directory-list method
I0627 16:54:02.806944 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/ca/"
I0627 16:54:02.810004 22505 ebsvolume.go:140] Creating PersistentVolume with Name:"b.etcd-main.dev-qa.k8s.domain.com"
I0627 16:54:02.812463 22505 request_logger.go:45] AWS request: ec2/CreateVolume
I0627 16:54:02.812832 22505 sshkey.go:135] Creating SSHKey with Name:"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b"
I0627 16:54:02.810137 22505 dhcp_options.go:142] Creating DHCPOptions with Name:"dev-qa.k8s.domain.com"
I0627 16:54:02.813295 22505 request_logger.go:45] AWS request: ec2/ImportKeyPair
I0627 16:54:02.813739 22505 request_logger.go:45] AWS request: ec2/CreateDhcpOptions
I0627 16:54:02.818048 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/keyset.yaml", falling back to directory-list method
I0627 16:54:02.818507 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/"
I0627 16:54:02.824798 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kube-proxy"
I0627 16:54:02.824867 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/storage-aws.addons.k8s.io/v1.7.0.yaml"
I0627 16:54:02.825525 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/storage-aws.addons.k8s.io/v1.7.0.yaml" SSE="AES256" ACL=""
I0627 16:54:02.825140 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/kube-dns.addons.k8s.io/k8s-1.6.yaml"
I0627 16:54:02.826505 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/kube-dns.addons.k8s.io/k8s-1.6.yaml" SSE="AES256" ACL=""
I0627 16:54:02.829302 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/bootstrap-channel.yaml"
I0627 16:54:02.829873 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/bootstrap-channel.yaml" SSE="AES256" ACL=""
I0627 16:54:02.829942 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml"
I0627 16:54:02.830756 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/kubelet-api.rbac.addons.k8s.io/k8s-1.9.yaml" SSE="AES256" ACL=""
I0627 16:54:02.829658 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/manifests/etcd/main.yaml"
I0627 16:54:02.829391 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/dns-controller.addons.k8s.io/k8s-1.6.yaml"
I0627 16:54:02.833685 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/manifests/etcd/main.yaml" SSE="AES256" ACL=""
I0627 16:54:02.834082 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/dns-controller.addons.k8s.io/k8s-1.6.yaml" SSE="AES256" ACL=""
I0627 16:54:02.835043 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml"
I0627 16:54:02.835347 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/dns-controller.addons.k8s.io/pre-k8s-1.6.yaml" SSE="AES256" ACL=""
I0627 16:54:02.838419 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/keyset.yaml", falling back to directory-list method
I0627 16:54:02.839023 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/"
I0627 16:54:02.838459 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/kube-dns.addons.k8s.io/k8s-1.12.yaml"
I0627 16:54:02.840330 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/kube-dns.addons.k8s.io/k8s-1.12.yaml" SSE="AES256" ACL=""
I0627 16:54:02.842363 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events/control/etcd-cluster-spec"
I0627 16:54:02.842415 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kubelet"
I0627 16:54:02.842420 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:scheduler"
I0627 16:54:02.844780 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/backups/etcd/events/control/etcd-cluster-spec" SSE="AES256" ACL=""
I0627 16:54:02.849569 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kube"
I0627 16:54:02.849593 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/networking.projectcalico.org/k8s-1.7-v3.yaml"
I0627 16:54:02.850460 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/networking.projectcalico.org/k8s-1.7-v3.yaml" SSE="AES256" ACL=""
I0627 16:54:02.849593 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/addons/networking.projectcalico.org/k8s-1.12.yaml"
I0627 16:54:02.851630 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/addons/networking.projectcalico.org/k8s-1.12.yaml" SSE="AES256" ACL=""
I0627 16:54:02.856002 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/manifests/etcd/events.yaml"
I0627 16:54:02.856396 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/manifests/etcd/events.yaml" SSE="AES256" ACL=""
I0627 16:54:02.856976 22505 ebsvolume.go:140] Creating PersistentVolume with Name:"c.etcd-events.dev-qa.k8s.domain.com"
I0627 16:54:02.857484 22505 request_logger.go:45] AWS request: ec2/CreateVolume
I0627 16:54:02.893883 22505 request_logger.go:45] AWS request: ec2/ModifyVpcAttribute
I0627 16:54:02.914599 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca: []
I0627 16:54:02.915483 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/keyset.yaml"
I0627 16:54:02.914698 22505 iamrole.go:144] Creating IAMRole with Name:"nodes.dev-qa.k8s.domain.com"
I0627 16:54:02.916565 22505 request_logger.go:45] AWS request: iam/CreateRole
I0627 16:54:02.920924 22505 iamrole.go:144] Creating IAMRole with Name:"masters.dev-qa.k8s.domain.com"
I0627 16:54:02.921443 22505 request_logger.go:45] AWS request: iam/CreateRole
I0627 16:54:02.962803 22505 iamrole.go:144] Creating IAMRole with Name:"bastions.dev-qa.k8s.domain.com"
I0627 16:54:02.963557 22505 request_logger.go:45] AWS request: iam/CreateRole
I0627 16:54:02.973625 22505 iaminstanceprofile.go:114] Creating IAMInstanceProfile with Name:"bastions.dev-qa.k8s.domain.com"
I0627 16:54:02.974108 22505 request_logger.go:45] AWS request: iam/CreateInstanceProfile
I0627 16:54:02.979834 22505 iaminstanceprofile.go:114] Creating IAMInstanceProfile with Name:"masters.dev-qa.k8s.domain.com"
I0627 16:54:02.980019 22505 iaminstanceprofile.go:114] Creating IAMInstanceProfile with Name:"nodes.dev-qa.k8s.domain.com"
I0627 16:54:02.982277 22505 request_logger.go:45] AWS request: iam/CreateInstanceProfile
I0627 16:54:02.980878 22505 request_logger.go:45] AWS request: iam/CreateInstanceProfile
I0627 16:54:02.995518 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events: []
I0627 16:54:02.996292 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/keyset.yaml"
I0627 16:54:03.008307 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main: []
I0627 16:54:03.009136 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/keyset.yaml"
I0627 16:54:03.016321 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events: []
I0627 16:54:03.016909 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/keyset.yaml"
I0627 16:54:03.023141 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca: []
I0627 16:54:03.023619 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml"
I0627 16:54:03.026964 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca: []
I0627 16:54:03.027516 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/keyset.yaml"
I0627 16:54:03.047953 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main: []
I0627 16:54:03.048593 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/keyset.yaml"
I0627 16:54:03.048046 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:03.103013 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:03.111530 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kube"
I0627 16:54:03.113339 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/secrets/kube" SSE="AES256" ACL=""
I0627 16:54:03.178627 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:03.200550 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/keyset.yaml", falling back to directory-list method
I0627 16:54:03.201100 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/"
I0627 16:54:03.202303 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/keyset.yaml", falling back to directory-list method
I0627 16:54:03.202574 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/"
I0627 16:54:03.206545 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/keyset.yaml", falling back to directory-list method
I0627 16:54:03.207106 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/"
I0627 16:54:03.206702 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:03.206718 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml", falling back to directory-list method
I0627 16:54:03.208108 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/ca/"
I0627 16:54:03.210881 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/keyset.yaml", falling back to directory-list method
I0627 16:54:03.211370 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/"
I0627 16:54:03.211132 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:03.220945 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/keyset.yaml", falling back to directory-list method
I0627 16:54:03.221371 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/"
I0627 16:54:03.227054 22505 request_logger.go:45] AWS request: iam/GetInstanceProfile
I0627 16:54:03.235675 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/keyset.yaml", falling back to directory-list method
I0627 16:54:03.236096 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/"
I0627 16:54:03.261010 22505 request_logger.go:45] AWS request: iam/GetInstanceProfile
I0627 16:54:03.274833 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:03.286203 22505 request_logger.go:45] AWS request: iam/GetInstanceProfile
I0627 16:54:03.314515 22505 aws_cloud.go:868] adding tags to "dopt-0552eab376b7ad010": map[Name:dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned]
I0627 16:54:03.315497 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:03.325103 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:03.355228 22505 request_logger.go:45] AWS request: ec2/ModifyVpcAttribute
I0627 16:54:03.408231 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main: []
I0627 16:54:03.408844 22505 keypair.go:181] creating brand new certificate
I0627 16:54:03.409992 22505 keypair.go:201] Creating PKI keypair "etcd-peers-ca-main"
I0627 16:54:03.410335 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/keyset.yaml"
I0627 16:54:03.408248 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca: []
I0627 16:54:03.410506 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events: []
I0627 16:54:03.411984 22505 keypair.go:181] creating brand new certificate
I0627 16:54:03.410573 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events: []
I0627 16:54:03.410594 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca: []
I0627 16:54:03.410611 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-clients-ca: []
I0627 16:54:03.411488 22505 keypair.go:181] creating brand new certificate
I0627 16:54:03.412318 22505 keypair.go:201] Creating PKI keypair "etcd-peers-ca-events"
I0627 16:54:03.412757 22505 keypair.go:181] creating brand new certificate
I0627 16:54:03.413161 22505 keypair.go:181] creating brand new certificate
I0627 16:54:03.413618 22505 keypair.go:181] creating brand new certificate
I0627 16:54:03.413951 22505 keypair.go:201] Creating PKI keypair "apiserver-aggregator-ca"
I0627 16:54:03.414048 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kube"
I0627 16:54:03.414059 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/admin"
I0627 16:54:03.414261 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/keyset.yaml"
I0627 16:54:03.414667 22505 keypair.go:201] Creating PKI keypair "etcd-manager-ca-events"
I0627 16:54:03.414967 22505 keypair.go:201] Creating PKI keypair "ca"
I0627 16:54:03.415253 22505 keypair.go:201] Creating PKI keypair "etcd-clients-ca"
I0627 16:54:03.415537 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/keyset.yaml"
I0627 16:54:03.419767 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/keyset.yaml"
I0627 16:54:03.420029 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml"
I0627 16:54:03.420267 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/keyset.yaml"
I0627 16:54:03.427028 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main: []
I0627 16:54:03.427636 22505 keypair.go:181] creating brand new certificate
I0627 16:54:03.427903 22505 keypair.go:201] Creating PKI keypair "etcd-manager-ca-main"
I0627 16:54:03.429833 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/keyset.yaml"
I0627 16:54:03.512410 22505 iaminstanceprofile.go:144] Found IAM instance profile "nodes.dev-qa.k8s.domain.com"
I0627 16:54:03.555441 22505 iaminstanceprofile.go:144] Found IAM instance profile "masters.dev-qa.k8s.domain.com"
I0627 16:54:03.569223 22505 iaminstanceprofile.go:144] Found IAM instance profile "bastions.dev-qa.k8s.domain.com"
I0627 16:54:03.612467 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/keyset.yaml", falling back to directory-list method
I0627 16:54:03.613332 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/"
I0627 16:54:03.613337 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/admin"
I0627 16:54:03.614285 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/secrets/admin" SSE="AES256" ACL=""
I0627 16:54:03.612935 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/keyset.yaml", falling back to directory-list method
I0627 16:54:03.615156 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/"
I0627 16:54:03.648602 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:03.796306 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml", falling back to directory-list method
I0627 16:54:03.796976 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/ca/"
I0627 16:54:03.796332 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/keyset.yaml", falling back to directory-list method
I0627 16:54:03.797726 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/"
I0627 16:54:03.796387 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/keyset.yaml", falling back to directory-list method
I0627 16:54:03.798477 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/"
I0627 16:54:03.826501 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/keyset.yaml", falling back to directory-list method
I0627 16:54:03.827067 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/"
I0627 16:54:03.864714 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/keyset.yaml", falling back to directory-list method
I0627 16:54:03.865373 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/"
I0627 16:54:03.873666 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main: []
I0627 16:54:03.875655 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/keyset.yaml"
I0627 16:54:03.885496 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/admin"
I0627 16:54:03.885539 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:logging"
I0627 16:54:03.895724 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events: []
I0627 16:54:03.896475 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/keyset.yaml"
I0627 16:54:03.900733 22505 aws_cloud.go:868] adding tags to "vpc-0f71f85232d256375": map[Name:dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned]
I0627 16:54:03.901263 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:03.987819 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events: []
I0627 16:54:03.988683 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/keyset.yaml"
I0627 16:54:03.987837 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca: []
I0627 16:54:03.989752 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml"
I0627 16:54:04.022520 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca: []
I0627 16:54:04.023333 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/keyset.yaml"
I0627 16:54:04.057091 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca: []
I0627 16:54:04.057938 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/keyset.yaml"
I0627 16:54:04.059803 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main: []
I0627 16:54:04.060799 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/keyset.yaml"
I0627 16:54:04.067801 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/keyset.yaml", falling back to directory-list method
I0627 16:54:04.068461 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/"
I0627 16:54:04.169009 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/keyset.yaml", falling back to directory-list method
I0627 16:54:04.169743 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/"
I0627 16:54:04.175214 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:logging"
I0627 16:54:04.175703 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/secrets/system:logging" SSE="AES256" ACL=""
I0627 16:54:04.242159 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/keyset.yaml", falling back to directory-list method
I0627 16:54:04.242841 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/"
I0627 16:54:04.273523 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/keyset.yaml", falling back to directory-list method
I0627 16:54:04.274775 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/"
I0627 16:54:04.273539 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml", falling back to directory-list method
I0627 16:54:04.273573 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/keyset.yaml", falling back to directory-list method
I0627 16:54:04.275811 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/ca/"
I0627 16:54:04.276337 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/"
I0627 16:54:04.335274 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/keyset.yaml", falling back to directory-list method
I0627 16:54:04.336104 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/"
I0627 16:54:04.453501 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events: []
I0627 16:54:04.454114 22505 keypair.go:212] Creating privateKey "etcd-peers-ca-events"
I0627 16:54:04.470886 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:logging"
I0627 16:54:04.471181 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:dns"
I0627 16:54:04.481939 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events: []
I0627 16:54:04.482897 22505 keypair.go:212] Creating privateKey "etcd-manager-ca-events"
I0627 16:54:04.485719 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main: []
I0627 16:54:04.486220 22505 keypair.go:212] Creating privateKey "etcd-peers-ca-main"
I0627 16:54:04.485752 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca: []
I0627 16:54:04.487011 22505 keypair.go:212] Creating privateKey "ca"
I0627 16:54:04.493349 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca: []
I0627 16:54:04.493738 22505 keypair.go:212] Creating privateKey "apiserver-aggregator-ca"
I0627 16:54:04.556730 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main: []
I0627 16:54:04.557360 22505 keypair.go:212] Creating privateKey "etcd-manager-ca-main"
I0627 16:54:04.556748 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-clients-ca: []
I0627 16:54:04.558000 22505 keypair.go:212] Creating privateKey "etcd-clients-ca"
I0627 16:54:04.670978 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:dns"
I0627 16:54:04.690548 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/secrets/system:dns" SSE="AES256" ACL=""
I0627 16:54:04.712206 22505 vfs_castore.go:729] Issuing new certificate: "apiserver-aggregator-ca"
I0627 16:54:04.734121 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/"
I0627 16:54:04.787224 22505 vfs_castore.go:729] Issuing new certificate: "etcd-clients-ca"
I0627 16:54:04.791852 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/"
I0627 16:54:04.797431 22505 vfs_castore.go:729] Issuing new certificate: "etcd-peers-ca-events"
I0627 16:54:04.801647 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/"
I0627 16:54:04.819586 22505 vfs_castore.go:729] Issuing new certificate: "etcd-peers-ca-main"
I0627 16:54:04.824134 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/"
I0627 16:54:04.845373 22505 vfs_castore.go:729] Issuing new certificate: "etcd-manager-ca-events"
I0627 16:54:04.848430 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/"
I0627 16:54:04.893868 22505 vfs_castore.go:729] Issuing new certificate: "etcd-manager-ca-main"
I0627 16:54:04.897027 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/"
I0627 16:54:04.916219 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:dns"
I0627 16:54:04.916250 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:controller_manager"
I0627 16:54:04.928211 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca: []
I0627 16:54:04.929404 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/keyset.yaml"
I0627 16:54:04.930064 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:04.982972 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-clients-ca: []
I0627 16:54:04.984076 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/keyset.yaml"
I0627 16:54:04.984464 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:04.999360 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events: []
I0627 16:54:05.000217 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/keyset.yaml"
I0627 16:54:05.000614 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:05.046269 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events: []
I0627 16:54:05.046361 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main: []
I0627 16:54:05.047601 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/keyset.yaml"
I0627 16:54:05.047900 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:05.047203 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/keyset.yaml"
I0627 16:54:05.048876 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:05.091861 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main: []
I0627 16:54:05.092863 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/keyset.yaml"
I0627 16:54:05.093194 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:05.101621 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:controller_manager"
I0627 16:54:05.102435 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/secrets/system:controller_manager" SSE="AES256" ACL=""
I0627 16:54:05.147279 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/6707223843927303564786986934.key"
I0627 16:54:05.148109 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/6707223843927303564786986934.key" SSE="AES256" ACL=""
I0627 16:54:05.192215 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/6707223844249545512212272389.key"
I0627 16:54:05.192811 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/etcd-clients-ca/6707223844249545512212272389.key" SSE="AES256" ACL=""
I0627 16:54:05.195580 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/6707223844293389830689665949.key"
I0627 16:54:05.196181 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-events/6707223844293389830689665949.key" SSE="AES256" ACL=""
I0627 16:54:05.256939 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/6707223844388546548809205220.key"
I0627 16:54:05.257566 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/etcd-peers-ca-main/6707223844388546548809205220.key" SSE="AES256" ACL=""
I0627 16:54:05.284372 22505 vfs_castore.go:729] Issuing new certificate: "ca"
I0627 16:54:05.287365 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/6707223844499263503708881691.key"
I0627 16:54:05.287769 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-events/6707223844499263503708881691.key" SSE="AES256" ACL=""
I0627 16:54:05.287548 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/ca/"
I0627 16:54:05.308638 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/6707223844707572851071139931.key"
I0627 16:54:05.309753 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/etcd-manager-ca-main/6707223844707572851071139931.key" SSE="AES256" ACL=""
I0627 16:54:05.319467 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:controller_manager"
I0627 16:54:05.319538 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:monitoring"
I0627 16:54:05.374128 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/"
I0627 16:54:05.402253 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/"
I0627 16:54:05.402292 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/"
I0627 16:54:05.466766 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/"
I0627 16:54:05.476272 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/ca: []
I0627 16:54:05.479702 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/ca/keyset.yaml"
I0627 16:54:05.480154 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/ca/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:05.511079 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/"
I0627 16:54:05.519809 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/"
I0627 16:54:05.556817 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:monitoring"
I0627 16:54:05.557465 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/secrets/system:monitoring" SSE="AES256" ACL=""
I0627 16:54:05.590831 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events: []
I0627 16:54:05.591681 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/keyset.yaml"
I0627 16:54:05.592029 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:05.591229 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca: []
I0627 16:54:05.593703 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/keyset.yaml"
I0627 16:54:05.612468 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:05.664865 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main: []
I0627 16:54:05.666178 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/keyset.yaml"
I0627 16:54:05.664920 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca: []
I0627 16:54:05.666637 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:05.667546 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/keyset.yaml"
I0627 16:54:05.668425 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:05.700064 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events: []
I0627 16:54:05.701941 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/keyset.yaml"
I0627 16:54:05.702458 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:05.710795 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/ca/6707223846384776908801404048.key"
I0627 16:54:05.712055 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/ca/6707223846384776908801404048.key" SSE="AES256" ACL=""
I0627 16:54:05.710796 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main: []
I0627 16:54:05.713869 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/keyset.yaml"
I0627 16:54:05.714474 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:05.770379 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:monitoring"
I0627 16:54:05.770854 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kube-proxy"
I0627 16:54:05.812568 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/6707223844293389830689665949.crt"
I0627 16:54:05.813437 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/6707223844293389830689665949.crt" SSE="AES256" ACL=""
I0627 16:54:05.845871 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/6707223843927303564786986934.crt"
I0627 16:54:05.846587 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/6707223843927303564786986934.crt" SSE="AES256" ACL=""
I0627 16:54:05.867411 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/6707223844249545512212272389.crt"
I0627 16:54:05.868141 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/6707223844249545512212272389.crt" SSE="AES256" ACL=""
I0627 16:54:05.927544 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/ca/"
I0627 16:54:05.927551 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/6707223844388546548809205220.crt"
I0627 16:54:05.928697 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/6707223844388546548809205220.crt" SSE="AES256" ACL=""
I0627 16:54:05.927631 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/6707223844499263503708881691.crt"
I0627 16:54:05.929744 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/6707223844499263503708881691.crt" SSE="AES256" ACL=""
I0627 16:54:05.949108 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/6707223844707572851071139931.crt"
I0627 16:54:05.950263 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/6707223844707572851071139931.crt" SSE="AES256" ACL=""
I0627 16:54:05.965041 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kube-proxy"
I0627 16:54:05.965386 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/secrets/kube-proxy" SSE="AES256" ACL=""
I0627 16:54:06.008873 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-events/6707223844293389830689665949.crt"
I0627 16:54:06.052922 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/6707223843927303564786986934.crt"
I0627 16:54:06.077959 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-clients-ca/6707223844249545512212272389.crt"
I0627 16:54:06.119302 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca: []
I0627 16:54:06.120090 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml"
I0627 16:54:06.120482 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:06.126825 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-peers-ca-main/6707223844388546548809205220.crt"
I0627 16:54:06.155673 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-events/6707223844499263503708881691.crt"
I0627 16:54:06.171101 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/etcd-manager-ca-main/6707223844707572851071139931.crt"
I0627 16:54:06.176488 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kube-proxy"
I0627 16:54:06.176992 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kubelet"
I0627 16:54:06.194358 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:06.194843 22505 keypair.go:230] created certificate with cn=etcd-peers-ca-events
I0627 16:54:06.237852 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:06.238630 22505 keypair.go:230] created certificate with cn=apiserver-aggregator-ca
I0627 16:54:06.273961 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:06.274814 22505 keypair.go:230] created certificate with cn=etcd-clients-ca
I0627 16:54:06.316006 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:06.316688 22505 keypair.go:230] created certificate with cn=etcd-peers-ca-main
I0627 16:54:06.325124 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/6707223846384776908801404048.crt"
I0627 16:54:06.325856 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/ca/6707223846384776908801404048.crt" SSE="AES256" ACL=""
I0627 16:54:06.434620 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:06.435505 22505 keypair.go:230] created certificate with cn=etcd-manager-ca-main
I0627 16:54:06.434824 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:06.436288 22505 keypair.go:230] created certificate with cn=etcd-manager-ca-events
I0627 16:54:06.434875 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kubelet"
I0627 16:54:06.437041 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/secrets/kubelet" SSE="AES256" ACL=""
I0627 16:54:06.562816 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/6707223846384776908801404048.crt"
I0627 16:54:06.660376 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/kubelet"
I0627 16:54:06.660861 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:scheduler"
I0627 16:54:06.749955 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:06.750674 22505 keypair.go:230] created certificate with cn=kubernetes
I0627 16:54:06.849084 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:scheduler"
I0627 16:54:06.849950 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/secrets/system:scheduler" SSE="AES256" ACL=""
I0627 16:54:07.049430 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/secrets/system:scheduler"
I0627 16:54:07.235879 22505 executor.go:103] Tasks: 51 done / 143 total; 38 can run
I0627 16:54:07.236912 22505 executor.go:178] Executing task "Keypair/apiserver-aggregator": *fitasks.Keypair {"Name":"apiserver-aggregator","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"apiserver-aggregator-ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=apiserver-aggregator-ca","type":"ca","format":"v1alpha2"},"subject":"cn=aggregator","type":"client","format":"v1alpha2"}
I0627 16:54:07.237780 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/keyset.yaml"
I0627 16:54:07.236933 22505 executor.go:178] Executing task "Keypair/kube-proxy": *fitasks.Keypair {"Name":"kube-proxy","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"cn=system:kube-proxy","type":"client","format":"v1alpha2"}
I0627 16:54:07.239121 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy/keyset.yaml"
I0627 16:54:07.236892 22505 executor.go:178] Executing task "RouteTable/private-eu-central-1b.dev-qa.k8s.domain.com": *awstasks.RouteTable {"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1b"}}
I0627 16:54:07.236971 22505 executor.go:178] Executing task "Keypair/kube-controller-manager": *fitasks.Keypair {"Name":"kube-controller-manager","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"cn=system:kube-controller-manager","type":"client","format":"v1alpha2"}
I0627 16:54:07.236987 22505 executor.go:178] Executing task "Keypair/kops": *fitasks.Keypair {"Name":"kops","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"o=system:masters,cn=kops","type":"client","format":"v1alpha2"}
I0627 16:54:07.236980 22505 executor.go:178] Executing task "Keypair/master": *fitasks.Keypair {"Name":"master","alternateNames":["kubernetes","kubernetes.default","kubernetes.default.svc","kubernetes.default.svc.cluster.local","api.dev-qa.k8s.domain.com","api.internal.dev-qa.k8s.domain.com","100.64.0.1","127.0.0.1"],"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"cn=kubernetes-master","type":"server","format":"v1alpha2"}
I0627 16:54:07.236987 22505 executor.go:178] Executing task "RouteTable/dev-qa.k8s.domain.com": *awstasks.RouteTable {"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"public"}}
I0627 16:54:07.237034 22505 executor.go:178] Executing task "IAMRolePolicy/additional.nodes.dev-qa.k8s.domain.com": *awstasks.IAMRolePolicy {"ID":null,"Lifecycle":"Sync","Name":"additional.nodes.dev-qa.k8s.domain.com","Role":{"ID":"AROA4CSJTTD5LM3QF3H7I","Lifecycle":"Sync","Name":"nodes.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"nodes"},"PolicyDocument":{"Name":"","Resource":""}}
I0627 16:54:07.237011 22505 executor.go:178] Executing task "VPCDHCPOptionsAssociation/dev-qa.k8s.domain.com": *awstasks.VPCDHCPOptionsAssociation {"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"DHCPOptions":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"dopt-0552eab376b7ad010","DomainName":"eu-central-1.compute.internal","DomainNameServers":"AmazonProvidedDNS","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}}
I0627 16:54:07.237070 22505 executor.go:178] Executing task "Keypair/kubelet": *fitasks.Keypair {"Name":"kubelet","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"o=system:nodes,cn=kubelet","type":"client","format":"v1alpha2"}
I0627 16:54:07.237016 22505 executor.go:178] Executing task "IAMInstanceProfileRole/masters.dev-qa.k8s.domain.com": *awstasks.IAMInstanceProfileRole {"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","InstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"AIPA4CSJTTD5DOX6RY3MZ","Shared":false},"Role":{"ID":"AROA4CSJTTD5PMSBB3BOF","Lifecycle":"Sync","Name":"masters.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"masters"}}
I0627 16:54:07.237124 22505 executor.go:178] Executing task "Keypair/kubecfg": *fitasks.Keypair {"Name":"kubecfg","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"o=system:masters,cn=kubecfg","type":"client","format":"v1alpha2"}
I0627 16:54:07.237113 22505 executor.go:178] Executing task "IAMInstanceProfileRole/nodes.dev-qa.k8s.domain.com": *awstasks.IAMInstanceProfileRole {"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","InstanceProfile":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"AIPA4CSJTTD5IYDEL47MM","Shared":false},"Role":{"ID":"AROA4CSJTTD5LM3QF3H7I","Lifecycle":"Sync","Name":"nodes.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"nodes"}}
I0627 16:54:07.237069 22505 executor.go:178] Executing task "InternetGateway/dev-qa.k8s.domain.com": *awstasks.InternetGateway {"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:54:07.236843 22505 executor.go:178] Executing task "IAMRolePolicy/bastions.dev-qa.k8s.domain.com": *awstasks.IAMRolePolicy {"ID":null,"Lifecycle":"Sync","Name":"bastions.dev-qa.k8s.domain.com","Role":{"ID":"AROA4CSJTTD5HAPI63HG7","Lifecycle":"Sync","Name":"bastions.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"bastions"},"PolicyDocument":{"Builder":{"Cluster":{"metadata":{"name":"dev-qa.k8s.domain.com","creationTimestamp":"2019-06-27T14:48:45Z"},"spec":{"channel":"stable","configBase":"s3://my-state-store/dev-qa.k8s.domain.com","cloudProvider":"aws","kubernetesVersion":"1.12.9","subnets":[{"name":"eu-central-1a","cidr":"172.31.239.32/27","zone":"eu-central-1a","type":"Private"},{"name":"eu-central-1b","cidr":"172.31.239.64/27","zone":"eu-central-1b","type":"Private"},{"name":"eu-central-1c","cidr":"172.31.239.96/27","zone":"eu-central-1c","type":"Private"},{"name":"utility-eu-central-1a","cidr":"172.31.239.0/30","zone":"eu-central-1a","type":"Utility"},{"name":"utility-eu-central-1b","cidr":"172.31.239.4/30","zone":"eu-central-1b","type":"Utility"},{"name":"utility-eu-central-1c","cidr":"172.31.239.8/30","zone":"eu-central-1c","type":"Utility"}],"masterPublicName":"api.dev-qa.k8s.domain.com","masterInternalName":"api.internal.dev-qa.k8s.domain.com","networkCIDR":"172.31.239.0/24","topology":{"masters":"private","nodes":"private","bastion":{"bastionPublicName":"bastion.dev-qa.k8s.domain.com"},"dns":{"type":"Public"}},"secretStore":"s3://my-state-store/dev-qa.k8s.domain.com/secrets","keyStore":"s3://my-state-store/dev-qa.k8s.domain.com/pki","configStore":"s3://my-state-store/dev-qa.k8s.domain.com","dnsZone":"k8s.domain.com","clusterDNSDomain":"cluster.local","serviceClusterIPRange":"100.64.0.0/13","nonMasqueradeCIDR":"100.64.0.0/10","sshAccess":["0.0.0.0/0"],"kubernetesApiAccess":["0.0.0.0/0"],"etcdClusters":[{"name":"main","provider":"Manager","etcdMembers":[{"name":"a","instanceGroup":"master-eu-central-1a","encryptedVolume":true},{"name":"b","instanceGroup":"master-eu-central-1b","encryptedVolume":true},{"name":"c","instanceGroup":"master-eu-central-1c","encryptedVolume":true}],"enableEtcdTLS":true,"enableTLSAuth":true,"version":"3.2.24","backups":{"backupStore":"s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main"},"manager":{},"memoryRequest":"100Mi","cpuRequest":"200m"},{"name":"events","provider":"Manager","etcdMembers":[{"name":"a","instanceGroup":"master-eu-central-1a","encryptedVolume":true},{"name":"b","instanceGroup":"master-eu-central-1b","encryptedVolume":true},{"name":"c","instanceGroup":"master-eu-central-1c","encryptedVolume":true}],"enableEtcdTLS":true,"enableTLSAuth":true,"version":"3.2.24","backups":{"backupStore":"s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events"},"manager":{},"memoryRequest":"100Mi","cpuRequest":"100m"}],"docker":{"ipMasq":false,"ipTables":false,"logDriver":"json-file","logLevel":"warn","logOpt":["max-size=10m","max-file=5"],"storage":"overlay2,overlay,aufs","version":"18.06.3"},"kubeDNS":{"cacheMaxSize":1000,"cacheMaxConcurrent":150,"domain":"cluster.local","replicas":2,"serverIP":"100.64.0.10","memoryRequest":"70Mi","cpuRequest":"100m","memoryLimit":"170Mi"},"kubeAPIServer":{"image":"k8s.gcr.io/kube-apiserver:v1.12.9","logLevel":2,"cloudProvider":"aws","securePort":443,"insecurePort":8080,"bindAddress":"0.0.0.0","insecureBindAddress":"127.0.0.1","enableAdmissionPlugins":["NamespaceLifecycle","LimitRanger","ServiceAccount","PersistentVolumeLabel","DefaultStorageClass","DefaultTolerationSeconds","MutatingAdmissionWebhook","ValidatingAdmissionWebhook","NodeRestriction","ResourceQuota"],"serviceClusterIPRange":"100.64.0.0/13","etcdServers":["http://127.0.0.1:4001"],"etcdServersOverrides":["/events#http://127.0.0.1:4002"],"allowPrivileged":true,"apiServerCount":3,"anonymousAuth":false,"kubeletPreferredAddressTypes":["InternalIP","Hostname","ExternalIP"],"storageBackend":"etcd3","authorizationMode":"RBAC","requestheaderUsernameHeaders":["X-Remote-User"],"requestheaderGroupHeaders":["X-Remote-Group"],"requestheaderExtraHeaderPrefixes":["X-Remote-Extra-"],"requestheaderAllowedNames":["aggregator"]},"kubeControllerManager":{"logLevel":2,"image":"k8s.gcr.io/kube-controller-manager:v1.12.9","cloudProvider":"aws","clusterName":"dev-qa.k8s.domain.com","clusterCIDR":"100.96.0.0/11","allocateNodeCIDRs":true,"configureCloudRoutes":false,"leaderElection":{"leaderElect":true},"attachDetachReconcileSyncPeriod":"1m0s","useServiceAccountCredentials":true},"kubeScheduler":{"logLevel":2,"image":"k8s.gcr.io/kube-scheduler:v1.12.9","leaderElection":{"leaderElect":true}},"kubeProxy":{"image":"k8s.gcr.io/kube-proxy:v1.12.9","cpuRequest":"100m","logLevel":2,"clusterCIDR":"100.96.0.0/11","hostnameOverride":"@aws"},"kubelet":{"anonymousAuth":false,"kubeconfigPath":"/var/lib/kubelet/kubeconfig","logLevel":2,"podManifestPath":"/etc/kubernetes/manifests","hostnameOverride":"@aws","podInfraContainerImage":"k8s.gcr.io/pause-amd64:3.0","allowPrivileged":true,"enableDebuggingHandlers":true,"clusterDomain":"cluster.local","clusterDNS":"100.64.0.10","networkPluginName":"cni","cloudProvider":"aws","cgroupRoot":"/","nonMasqueradeCIDR":"100.64.0.0/10","evictionHard":"memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%","featureGates":{"ExperimentalCriticalPodAnnotation":"true"}},"masterKubelet":{"anonymousAuth":false,"kubeconfigPath":"/var/lib/kubelet/kubeconfig","logLevel":2,"podManifestPath":"/etc/kubernetes/manifests","hostnameOverride":"@aws","podInfraContainerImage":"k8s.gcr.io/pause-amd64:3.0","allowPrivileged":true,"enableDebuggingHandlers":true,"clusterDomain":"cluster.local","clusterDNS":"100.64.0.10","networkPluginName":"cni","cloudProvider":"aws","cgroupRoot":"/","registerSchedulable":false,"nonMasqueradeCIDR":"100.64.0.0/10","evictionHard":"memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%","featureGates":{"ExperimentalCriticalPodAnnotation":"true"}},"networking":{"calico":{"majorVersion":"v3"}},"api":{"loadBalancer":{"type":"Public"}},"authorization":{"rbac":{}},"iam":{"legacy":false,"allowContainerRegistry":true}}},"HostedZoneID":"","KMSKeys":null,"Region":"eu-central-1","ResourceARN":null,"Role":"Bastion"},"DNSZone":{"Name":"k8s.domain.com","Lifecycle":"Sync","DNSName":"k8s.domain.com","ZoneID":"Z34N9ZS9SNSUL7","Private":null,"PrivateVPC":null}}}
I0627 16:54:07.237152 22505 executor.go:178] Executing task "Keypair/apiserver-proxy-client": *fitasks.Keypair {"Name":"apiserver-proxy-client","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"cn=apiserver-proxy-client","type":"client","format":"v1alpha2"}
I0627 16:54:07.237124 22505 executor.go:178] Executing task "RouteTable/private-eu-central-1c.dev-qa.k8s.domain.com": *awstasks.RouteTable {"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1c"}}
I0627 16:54:07.237177 22505 executor.go:178] Executing task "Keypair/kubelet-api": *fitasks.Keypair {"Name":"kubelet-api","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"cn=kubelet-api","type":"client","format":"v1alpha2"}
I0627 16:54:07.237174 22505 executor.go:178] Executing task "IAMRolePolicy/additional.bastions.dev-qa.k8s.domain.com": *awstasks.IAMRolePolicy {"ID":null,"Lifecycle":"Sync","Name":"additional.bastions.dev-qa.k8s.domain.com","Role":{"ID":"AROA4CSJTTD5HAPI63HG7","Lifecycle":"Sync","Name":"bastions.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"bastions"},"PolicyDocument":{"Name":"","Resource":""}}
I0627 16:54:07.237200 22505 executor.go:178] Executing task "Keypair/kube-scheduler": *fitasks.Keypair {"Name":"kube-scheduler","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":{"Name":"ca","alternateNames":null,"alternateNameTasks":null,"Lifecycle":"Sync","Signer":null,"subject":"cn=kubernetes","type":"ca","format":"v1alpha2"},"subject":"cn=system:kube-scheduler","type":"client","format":"v1alpha2"}
I0627 16:54:07.236843 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:07.237188 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:07.237224 22505 executor.go:178] Executing task "RouteTable/private-eu-central-1a.dev-qa.k8s.domain.com": *awstasks.RouteTable {"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1a"}}
I0627 16:54:07.236886 22505 executor.go:178] Executing task "SecurityGroup/nodes.dev-qa.k8s.domain.com": *awstasks.SecurityGroup {"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:54:07.236949 22505 executor.go:178] Executing task "IAMRolePolicy/nodes.dev-qa.k8s.domain.com": *awstasks.IAMRolePolicy {"ID":null,"Lifecycle":"Sync","Name":"nodes.dev-qa.k8s.domain.com","Role":{"ID":"AROA4CSJTTD5LM3QF3H7I","Lifecycle":"Sync","Name":"nodes.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"nodes"},"PolicyDocument":{"Builder":{"Cluster":{"metadata":{"name":"dev-qa.k8s.domain.com","creationTimestamp":"2019-06-27T14:48:45Z"},"spec":{"channel":"stable","configBase":"s3://my-state-store/dev-qa.k8s.domain.com","cloudProvider":"aws","kubernetesVersion":"1.12.9","subnets":[{"name":"eu-central-1a","cidr":"172.31.239.32/27","zone":"eu-central-1a","type":"Private"},{"name":"eu-central-1b","cidr":"172.31.239.64/27","zone":"eu-central-1b","type":"Private"},{"name":"eu-central-1c","cidr":"172.31.239.96/27","zone":"eu-central-1c","type":"Private"},{"name":"utility-eu-central-1a","cidr":"172.31.239.0/30","zone":"eu-central-1a","type":"Utility"},{"name":"utility-eu-central-1b","cidr":"172.31.239.4/30","zone":"eu-central-1b","type":"Utility"},{"name":"utility-eu-central-1c","cidr":"172.31.239.8/30","zone":"eu-central-1c","type":"Utility"}],"masterPublicName":"api.dev-qa.k8s.domain.com","masterInternalName":"api.internal.dev-qa.k8s.domain.com","networkCIDR":"172.31.239.0/24","topology":{"masters":"private","nodes":"private","bastion":{"bastionPublicName":"bastion.dev-qa.k8s.domain.com"},"dns":{"type":"Public"}},"secretStore":"s3://my-state-store/dev-qa.k8s.domain.com/secrets","keyStore":"s3://my-state-store/dev-qa.k8s.domain.com/pki","configStore":"s3://my-state-store/dev-qa.k8s.domain.com","dnsZone":"k8s.domain.com","clusterDNSDomain":"cluster.local","serviceClusterIPRange":"100.64.0.0/13","nonMasqueradeCIDR":"100.64.0.0/10","sshAccess":["0.0.0.0/0"],"kubernetesApiAccess":["0.0.0.0/0"],"etcdClusters":[{"name":"main","provider":"Manager","etcdMembers":[{"name":"a","instanceGroup":"master-eu-central-1a","encryptedVolume":true},{"name":"b","instanceGroup":"master-eu-central-1b","encryptedVolume":true},{"name":"c","instanceGroup":"master-eu-central-1c","encryptedVolume":true}],"enableEtcdTLS":true,"enableTLSAuth":true,"version":"3.2.24","backups":{"backupStore":"s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main"},"manager":{},"memoryRequest":"100Mi","cpuRequest":"200m"},{"name":"events","provider":"Manager","etcdMembers":[{"name":"a","instanceGroup":"master-eu-central-1a","encryptedVolume":true},{"name":"b","instanceGroup":"master-eu-central-1b","encryptedVolume":true},{"name":"c","instanceGroup":"master-eu-central-1c","encryptedVolume":true}],"enableEtcdTLS":true,"enableTLSAuth":true,"version":"3.2.24","backups":{"backupStore":"s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events"},"manager":{},"memoryRequest":"100Mi","cpuRequest":"100m"}],"docker":{"ipMasq":false,"ipTables":false,"logDriver":"json-file","logLevel":"warn","logOpt":["max-size=10m","max-file=5"],"storage":"overlay2,overlay,aufs","version":"18.06.3"},"kubeDNS":{"cacheMaxSize":1000,"cacheMaxConcurrent":150,"domain":"cluster.local","replicas":2,"serverIP":"100.64.0.10","memoryRequest":"70Mi","cpuRequest":"100m","memoryLimit":"170Mi"},"kubeAPIServer":{"image":"k8s.gcr.io/kube-apiserver:v1.12.9","logLevel":2,"cloudProvider":"aws","securePort":443,"insecurePort":8080,"bindAddress":"0.0.0.0","insecureBindAddress":"127.0.0.1","enableAdmissionPlugins":["NamespaceLifecycle","LimitRanger","ServiceAccount","PersistentVolumeLabel","DefaultStorageClass","DefaultTolerationSeconds","MutatingAdmissionWebhook","ValidatingAdmissionWebhook","NodeRestriction","ResourceQuota"],"serviceClusterIPRange":"100.64.0.0/13","etcdServers":["http://127.0.0.1:4001"],"etcdServersOverrides":["/events#http://127.0.0.1:4002"],"allowPrivileged":true,"apiServerCount":3,"anonymousAuth":false,"kubeletPreferredAddressTypes":["InternalIP","Hostname","ExternalIP"],"storageBackend":"etcd3","authorizationMode":"RBAC","requestheaderUsernameHeaders":["X-Remote-User"],"requestheaderGroupHeaders":["X-Remote-Group"],"requestheaderExtraHeaderPrefixes":["X-Remote-Extra-"],"requestheaderAllowedNames":["aggregator"]},"kubeControllerManager":{"logLevel":2,"image":"k8s.gcr.io/kube-controller-manager:v1.12.9","cloudProvider":"aws","clusterName":"dev-qa.k8s.domain.com","clusterCIDR":"100.96.0.0/11","allocateNodeCIDRs":true,"configureCloudRoutes":false,"leaderElection":{"leaderElect":true},"attachDetachReconcileSyncPeriod":"1m0s","useServiceAccountCredentials":true},"kubeScheduler":{"logLevel":2,"image":"k8s.gcr.io/kube-scheduler:v1.12.9","leaderElection":{"leaderElect":true}},"kubeProxy":{"image":"k8s.gcr.io/kube-proxy:v1.12.9","cpuRequest":"100m","logLevel":2,"clusterCIDR":"100.96.0.0/11","hostnameOverride":"@aws"},"kubelet":{"anonymousAuth":false,"kubeconfigPath":"/var/lib/kubelet/kubeconfig","logLevel":2,"podManifestPath":"/etc/kubernetes/manifests","hostnameOverride":"@aws","podInfraContainerImage":"k8s.gcr.io/pause-amd64:3.0","allowPrivileged":true,"enableDebuggingHandlers":true,"clusterDomain":"cluster.local","clusterDNS":"100.64.0.10","networkPluginName":"cni","cloudProvider":"aws","cgroupRoot":"/","nonMasqueradeCIDR":"100.64.0.0/10","evictionHard":"memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%","featureGates":{"ExperimentalCriticalPodAnnotation":"true"}},"masterKubelet":{"anonymousAuth":false,"kubeconfigPath":"/var/lib/kubelet/kubeconfig","logLevel":2,"podManifestPath":"/etc/kubernetes/manifests","hostnameOverride":"@aws","podInfraContainerImage":"k8s.gcr.io/pause-amd64:3.0","allowPrivileged":true,"enableDebuggingHandlers":true,"clusterDomain":"cluster.local","clusterDNS":"100.64.0.10","networkPluginName":"cni","cloudProvider":"aws","cgroupRoot":"/","registerSchedulable":false,"nonMasqueradeCIDR":"100.64.0.0/10","evictionHard":"memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%","featureGates":{"ExperimentalCriticalPodAnnotation":"true"}},"networking":{"calico":{"majorVersion":"v3"}},"api":{"loadBalancer":{"type":"Public"}},"authorization":{"rbac":{}},"iam":{"legacy":false,"allowContainerRegistry":true}}},"HostedZoneID":"","KMSKeys":null,"Region":"eu-central-1","ResourceARN":null,"Role":"Node"},"DNSZone":{"Name":"k8s.domain.com","Lifecycle":"Sync","DNSName":"k8s.domain.com","ZoneID":"Z34N9ZS9SNSUL7","Private":null,"PrivateVPC":null}}}
I0627 16:54:07.236921 22505 executor.go:178] Executing task "IAMRolePolicy/additional.masters.dev-qa.k8s.domain.com": *awstasks.IAMRolePolicy {"ID":null,"Lifecycle":"Sync","Name":"additional.masters.dev-qa.k8s.domain.com","Role":{"ID":"AROA4CSJTTD5PMSBB3BOF","Lifecycle":"Sync","Name":"masters.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"masters"},"PolicyDocument":{"Name":"","Resource":""}}
I0627 16:54:07.237081 22505 executor.go:178] Executing task "MirrorKeystore/mirror-keystore": *fitasks.MirrorKeystore {"Name":"mirror-keystore","Lifecycle":null,"MirrorPath":{}}
I0627 16:54:07.237044 22505 executor.go:178] Executing task "Subnet/eu-central-1c.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"eu-central-1c.dev-qa.k8s.domain.com","ShortName":"eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.96/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}
I0627 16:54:07.237220 22505 executor.go:178] Executing task "SecurityGroup/bastion-elb.dev-qa.k8s.domain.com": *awstasks.SecurityGroup {"Name":"bastion-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:54:07.236999 22505 executor.go:178] Executing task "SecurityGroup/bastion.dev-qa.k8s.domain.com": *awstasks.SecurityGroup {"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:54:07.237057 22505 executor.go:178] Executing task "Subnet/eu-central-1b.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"eu-central-1b.dev-qa.k8s.domain.com","ShortName":"eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.64/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}
I0627 16:54:07.237099 22505 executor.go:178] Executing task "SecurityGroup/masters.dev-qa.k8s.domain.com": *awstasks.SecurityGroup {"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:54:07.237031 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:07.237148 22505 executor.go:178] Executing task "MirrorSecrets/mirror-secrets": *fitasks.MirrorSecrets {"Name":"mirror-secrets","Lifecycle":null,"MirrorPath":{}}
I0627 16:54:07.237293 22505 executor.go:178] Executing task "IAMInstanceProfileRole/bastions.dev-qa.k8s.domain.com": *awstasks.IAMInstanceProfileRole {"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","InstanceProfile":{"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"AIPA4CSJTTD5PJWJTK6ZP","Shared":false},"Role":{"ID":"AROA4CSJTTD5HAPI63HG7","Lifecycle":"Sync","Name":"bastions.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"bastions"}}
I0627 16:54:07.237068 22505 executor.go:178] Executing task "Subnet/eu-central-1a.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"eu-central-1a.dev-qa.k8s.domain.com","ShortName":"eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.32/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}
I0627 16:54:07.237208 22505 executor.go:178] Executing task "SecurityGroup/api-elb.dev-qa.k8s.domain.com": *awstasks.SecurityGroup {"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}
I0627 16:54:07.236866 22505 executor.go:178] Executing task "IAMRolePolicy/masters.dev-qa.k8s.domain.com": *awstasks.IAMRolePolicy {"ID":null,"Lifecycle":"Sync","Name":"masters.dev-qa.k8s.domain.com","Role":{"ID":"AROA4CSJTTD5PMSBB3BOF","Lifecycle":"Sync","Name":"masters.dev-qa.k8s.domain.com","RolePolicyDocument":{"Name":"","Resource":{}},"ExportWithID":"masters"},"PolicyDocument":{"Builder":{"Cluster":{"metadata":{"name":"dev-qa.k8s.domain.com","creationTimestamp":"2019-06-27T14:48:45Z"},"spec":{"channel":"stable","configBase":"s3://my-state-store/dev-qa.k8s.domain.com","cloudProvider":"aws","kubernetesVersion":"1.12.9","subnets":[{"name":"eu-central-1a","cidr":"172.31.239.32/27","zone":"eu-central-1a","type":"Private"},{"name":"eu-central-1b","cidr":"172.31.239.64/27","zone":"eu-central-1b","type":"Private"},{"name":"eu-central-1c","cidr":"172.31.239.96/27","zone":"eu-central-1c","type":"Private"},{"name":"utility-eu-central-1a","cidr":"172.31.239.0/30","zone":"eu-central-1a","type":"Utility"},{"name":"utility-eu-central-1b","cidr":"172.31.239.4/30","zone":"eu-central-1b","type":"Utility"},{"name":"utility-eu-central-1c","cidr":"172.31.239.8/30","zone":"eu-central-1c","type":"Utility"}],"masterPublicName":"api.dev-qa.k8s.domain.com","masterInternalName":"api.internal.dev-qa.k8s.domain.com","networkCIDR":"172.31.239.0/24","topology":{"masters":"private","nodes":"private","bastion":{"bastionPublicName":"bastion.dev-qa.k8s.domain.com"},"dns":{"type":"Public"}},"secretStore":"s3://my-state-store/dev-qa.k8s.domain.com/secrets","keyStore":"s3://my-state-store/dev-qa.k8s.domain.com/pki","configStore":"s3://my-state-store/dev-qa.k8s.domain.com","dnsZone":"k8s.domain.com","clusterDNSDomain":"cluster.local","serviceClusterIPRange":"100.64.0.0/13","nonMasqueradeCIDR":"100.64.0.0/10","sshAccess":["0.0.0.0/0"],"kubernetesApiAccess":["0.0.0.0/0"],"etcdClusters":[{"name":"main","provider":"Manager","etcdMembers":[{"name":"a","instanceGroup":"master-eu-central-1a","encryptedVolume":true},{"name":"b","instanceGroup":"master-eu-central-1b","encryptedVolume":true},{"name":"c","instanceGroup":"master-eu-central-1c","encryptedVolume":true}],"enableEtcdTLS":true,"enableTLSAuth":true,"version":"3.2.24","backups":{"backupStore":"s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/main"},"manager":{},"memoryRequest":"100Mi","cpuRequest":"200m"},{"name":"events","provider":"Manager","etcdMembers":[{"name":"a","instanceGroup":"master-eu-central-1a","encryptedVolume":true},{"name":"b","instanceGroup":"master-eu-central-1b","encryptedVolume":true},{"name":"c","instanceGroup":"master-eu-central-1c","encryptedVolume":true}],"enableEtcdTLS":true,"enableTLSAuth":true,"version":"3.2.24","backups":{"backupStore":"s3://my-state-store/dev-qa.k8s.domain.com/backups/etcd/events"},"manager":{},"memoryRequest":"100Mi","cpuRequest":"100m"}],"docker":{"ipMasq":false,"ipTables":false,"logDriver":"json-file","logLevel":"warn","logOpt":["max-size=10m","max-file=5"],"storage":"overlay2,overlay,aufs","version":"18.06.3"},"kubeDNS":{"cacheMaxSize":1000,"cacheMaxConcurrent":150,"domain":"cluster.local","replicas":2,"serverIP":"100.64.0.10","memoryRequest":"70Mi","cpuRequest":"100m","memoryLimit":"170Mi"},"kubeAPIServer":{"image":"k8s.gcr.io/kube-apiserver:v1.12.9","logLevel":2,"cloudProvider":"aws","securePort":443,"insecurePort":8080,"bindAddress":"0.0.0.0","insecureBindAddress":"127.0.0.1","enableAdmissionPlugins":["NamespaceLifecycle","LimitRanger","ServiceAccount","PersistentVolumeLabel","DefaultStorageClass","DefaultTolerationSeconds","MutatingAdmissionWebhook","ValidatingAdmissionWebhook","NodeRestriction","ResourceQuota"],"serviceClusterIPRange":"100.64.0.0/13","etcdServers":["http://127.0.0.1:4001"],"etcdServersOverrides":["/events#http://127.0.0.1:4002"],"allowPrivileged":true,"apiServerCount":3,"anonymousAuth":false,"kubeletPreferredAddressTypes":["InternalIP","Hostname","ExternalIP"],"storageBackend":"etcd3","authorizationMode":"RBAC","requestheaderUsernameHeaders":["X-Remote-User"],"requestheaderGroupHeaders":["X-Remote-Group"],"requestheaderExtraHeaderPrefixes":["X-Remote-Extra-"],"requestheaderAllowedNames":["aggregator"]},"kubeControllerManager":{"logLevel":2,"image":"k8s.gcr.io/kube-controller-manager:v1.12.9","cloudProvider":"aws","clusterName":"dev-qa.k8s.domain.com","clusterCIDR":"100.96.0.0/11","allocateNodeCIDRs":true,"configureCloudRoutes":false,"leaderElection":{"leaderElect":true},"attachDetachReconcileSyncPeriod":"1m0s","useServiceAccountCredentials":true},"kubeScheduler":{"logLevel":2,"image":"k8s.gcr.io/kube-scheduler:v1.12.9","leaderElection":{"leaderElect":true}},"kubeProxy":{"image":"k8s.gcr.io/kube-proxy:v1.12.9","cpuRequest":"100m","logLevel":2,"clusterCIDR":"100.96.0.0/11","hostnameOverride":"@aws"},"kubelet":{"anonymousAuth":false,"kubeconfigPath":"/var/lib/kubelet/kubeconfig","logLevel":2,"podManifestPath":"/etc/kubernetes/manifests","hostnameOverride":"@aws","podInfraContainerImage":"k8s.gcr.io/pause-amd64:3.0","allowPrivileged":true,"enableDebuggingHandlers":true,"clusterDomain":"cluster.local","clusterDNS":"100.64.0.10","networkPluginName":"cni","cloudProvider":"aws","cgroupRoot":"/","nonMasqueradeCIDR":"100.64.0.0/10","evictionHard":"memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%","featureGates":{"ExperimentalCriticalPodAnnotation":"true"}},"masterKubelet":{"anonymousAuth":false,"kubeconfigPath":"/var/lib/kubelet/kubeconfig","logLevel":2,"podManifestPath":"/etc/kubernetes/manifests","hostnameOverride":"@aws","podInfraContainerImage":"k8s.gcr.io/pause-amd64:3.0","allowPrivileged":true,"enableDebuggingHandlers":true,"clusterDomain":"cluster.local","clusterDNS":"100.64.0.10","networkPluginName":"cni","cloudProvider":"aws","cgroupRoot":"/","registerSchedulable":false,"nonMasqueradeCIDR":"100.64.0.0/10","evictionHard":"memory.available\u003c100Mi,nodefs.available\u003c10%,nodefs.inodesFree\u003c5%,imagefs.available\u003c10%,imagefs.inodesFree\u003c5%","featureGates":{"ExperimentalCriticalPodAnnotation":"true"}},"networking":{"calico":{"majorVersion":"v3"}},"api":{"loadBalancer":{"type":"Public"}},"authorization":{"rbac":{}},"iam":{"legacy":false,"allowContainerRegistry":true}}},"HostedZoneID":"","KMSKeys":null,"Region":"eu-central-1","ResourceARN":null,"Role":"Master"},"DNSZone":{"Name":"k8s.domain.com","Lifecycle":"Sync","DNSName":"k8s.domain.com","ZoneID":"Z34N9ZS9SNSUL7","Private":null,"PrivateVPC":null}}}
I0627 16:54:07.240660 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:07.241441 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/keyset.yaml"
I0627 16:54:07.242024 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops/keyset.yaml"
I0627 16:54:07.242912 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master/keyset.yaml"
I0627 16:54:07.245216 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:07.247868 22505 request_logger.go:45] AWS request: iam/GetRolePolicy
I0627 16:54:07.248799 22505 aws_cloud.go:1124] Calling DescribeVPC for VPC "vpc-0f71f85232d256375"
I0627 16:54:07.249430 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet/keyset.yaml"
I0627 16:54:07.250129 22505 request_logger.go:45] AWS request: iam/GetInstanceProfile
I0627 16:54:07.250747 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg/keyset.yaml"
I0627 16:54:07.251715 22505 request_logger.go:45] AWS request: iam/GetInstanceProfile
I0627 16:54:07.252537 22505 request_logger.go:45] AWS request: ec2/DescribeInternetGateways
I0627 16:54:07.261990 22505 request_logger.go:45] AWS request: iam/GetRolePolicy
I0627 16:54:07.262609 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/keyset.yaml"
I0627 16:54:07.263350 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:07.263891 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api/keyset.yaml"
I0627 16:54:07.264610 22505 request_logger.go:45] AWS request: iam/GetRolePolicy
I0627 16:54:07.265147 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler/keyset.yaml"
I0627 16:54:07.266331 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:07.267079 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:07.267795 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:07.268659 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:07.278935 22505 request_logger.go:45] AWS request: iam/GetRolePolicy
I0627 16:54:07.279410 22505 request_logger.go:45] AWS request: iam/GetRolePolicy
I0627 16:54:07.280565 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:07.281273 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:07.282043 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:07.283627 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:07.284414 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:07.285318 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:07.286216 22505 request_logger.go:45] AWS request: iam/GetInstanceProfile
I0627 16:54:07.290581 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:07.292986 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:07.297844 22505 request_logger.go:45] AWS request: iam/GetRolePolicy
I0627 16:54:07.305268 22505 request_logger.go:45] AWS request: ec2/DescribeVpcs
I0627 16:54:07.422581 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/keyset.yaml", falling back to directory-list method
I0627 16:54:07.423329 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/"
I0627 16:54:07.422581 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy/keyset.yaml", falling back to directory-list method
I0627 16:54:07.424308 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kube-proxy/"
I0627 16:54:07.521436 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:07.522501 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:07.552678 22505 request_logger.go:45] AWS request: iam/AddRoleToInstanceProfile
I0627 16:54:07.607923 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops/keyset.yaml", falling back to directory-list method
I0627 16:54:07.608568 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kops/"
I0627 16:54:07.607964 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/keyset.yaml", falling back to directory-list method
I0627 16:54:07.609480 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/"
I0627 16:54:07.727360 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:07.745638 22505 internetgateway.go:150] Creating InternetGateway
I0627 16:54:07.746095 22505 request_logger.go:45] AWS request: ec2/CreateInternetGateway
I0627 16:54:07.788562 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master/keyset.yaml", falling back to directory-list method
I0627 16:54:07.789294 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/master/"
I0627 16:54:07.792349 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet/keyset.yaml", falling back to directory-list method
I0627 16:54:07.793229 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kubelet/"
I0627 16:54:07.809755 22505 request_logger.go:45] AWS request: iam/AddRoleToInstanceProfile
I0627 16:54:07.810037 22505 iamrolepolicy.go:147] Creating IAMRolePolicy
I0627 16:54:07.811383 22505 iamrolepolicy.go:175] PutRolePolicy RoleName=bastions.dev-qa.k8s.domain.com PolicyName=bastions.dev-qa.k8s.domain.com: {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeRegions"
],
"Resource": [
"*"
]
}
]
}
I0627 16:54:07.814898 22505 request_logger.go:45] AWS request: iam/PutRolePolicy
I0627 16:54:07.967097 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg/keyset.yaml", falling back to directory-list method
I0627 16:54:07.967877 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kubecfg/"
I0627 16:54:07.976404 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/keyset.yaml", falling back to directory-list method
I0627 16:54:07.978371 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/"
I0627 16:54:08.016697 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.0/30"
I0627 16:54:08.017418 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:08.030104 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.4/30"
I0627 16:54:08.030426 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:08.091933 22505 iam_builder.go:300] Ignoring location "s3://my-state-store/dev-qa.k8s.domain.com/pki/" because found parent "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:54:08.092772 22505 iam_builder.go:300] Ignoring location "s3://my-state-store/dev-qa.k8s.domain.com/secrets/" because found parent "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:54:08.093209 22505 iam_builder.go:305] Found root location "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:54:08.093699 22505 iam_builder.go:300] Ignoring location "s3://my-state-store/dev-qa.k8s.domain.com/pki/" because found parent "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:54:08.094055 22505 iam_builder.go:300] Ignoring location "s3://my-state-store/dev-qa.k8s.domain.com/secrets/" because found parent "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:54:08.094449 22505 iam_builder.go:305] Found root location "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:54:08.094930 22505 iamrolepolicy.go:147] Creating IAMRolePolicy
I0627 16:54:08.095249 22505 iamrolepolicy.go:175] PutRolePolicy RoleName=nodes.dev-qa.k8s.domain.com PolicyName=nodes.dev-qa.k8s.domain.com: {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeRegions"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:GetEncryptionConfiguration",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::my-state-store"
]
},
{
"Effect": "Allow",
"Action": [
"s3:Get*"
],
"Resource": [
"arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/addons/*",
"arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/cluster.spec",
"arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/config",
"arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/instancegroup/*",
"arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/pki/issued/*",
"arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy/*",
"arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet/*",
"arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/pki/ssh/*",
"arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/secrets/dockerconfig"
]
},
{
"Effect": "Allow",
"Action": [
"s3:Get*"
],
"Resource": "arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/pki/private/calico-client/*"
},
{
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:GetRepositoryPolicy",
"ecr:DescribeRepositories",
"ecr:ListImages",
"ecr:BatchGetImage"
],
"Resource": [
"*"
]
}
]
}
I0627 16:54:08.108960 22505 request_logger.go:45] AWS request: iam/PutRolePolicy
I0627 16:54:08.147729 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api/keyset.yaml", falling back to directory-list method
I0627 16:54:08.148567 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kubelet-api/"
I0627 16:54:08.158242 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler/keyset.yaml", falling back to directory-list method
I0627 16:54:08.158730 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kube-scheduler/"
I0627 16:54:08.230687 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:08.262897 22505 securitygroup.go:166] Creating SecurityGroup with Name:"nodes.dev-qa.k8s.domain.com" VPC:"vpc-0f71f85232d256375"
I0627 16:54:08.263529 22505 request_logger.go:45] AWS request: ec2/CreateSecurityGroup
I0627 16:54:08.337941 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator: []
I0627 16:54:08.338563 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/keyset.yaml"
I0627 16:54:08.338524 22505 request_logger.go:45] AWS request: iam/AddRoleToInstanceProfile
I0627 16:54:08.353069 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy: []
I0627 16:54:08.353903 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy/keyset.yaml"
I0627 16:54:08.377768 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master: []
I0627 16:54:08.378437 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/master/keyset.yaml"
I0627 16:54:08.381781 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager: []
I0627 16:54:08.382266 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-controller-manager/keyset.yaml"
I0627 16:54:08.381797 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops: []
I0627 16:54:08.383964 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kops/keyset.yaml"
I0627 16:54:08.382708 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet: []
I0627 16:54:08.385348 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet/keyset.yaml"
I0627 16:54:08.423534 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client: []
I0627 16:54:08.424362 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/keyset.yaml"
I0627 16:54:08.423533 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg: []
I0627 16:54:08.425477 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubecfg/keyset.yaml"
I0627 16:54:08.430597 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler: []
I0627 16:54:08.431455 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-scheduler/keyset.yaml"
I0627 16:54:08.431523 22505 securitygroup.go:166] Creating SecurityGroup with Name:"bastion-elb.dev-qa.k8s.domain.com" VPC:"vpc-0f71f85232d256375"
I0627 16:54:08.433742 22505 request_logger.go:45] AWS request: ec2/CreateSecurityGroup
I0627 16:54:08.433881 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.8/30"
I0627 16:54:08.434533 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:08.436854 22505 securitygroup.go:166] Creating SecurityGroup with Name:"api-elb.dev-qa.k8s.domain.com" VPC:"vpc-0f71f85232d256375"
I0627 16:54:08.437294 22505 request_logger.go:45] AWS request: ec2/CreateSecurityGroup
I0627 16:54:08.436877 22505 securitygroup.go:166] Creating SecurityGroup with Name:"bastion.dev-qa.k8s.domain.com" VPC:"vpc-0f71f85232d256375"
I0627 16:54:08.438115 22505 request_logger.go:45] AWS request: ec2/CreateSecurityGroup
I0627 16:54:08.442345 22505 securitygroup.go:166] Creating SecurityGroup with Name:"masters.dev-qa.k8s.domain.com" VPC:"vpc-0f71f85232d256375"
I0627 16:54:08.444097 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.96/27"
I0627 16:54:08.444122 22505 request_logger.go:45] AWS request: ec2/CreateSecurityGroup
I0627 16:54:08.444499 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:08.455442 22505 routetable.go:174] Creating RouteTable with VPC: "vpc-0f71f85232d256375"
I0627 16:54:08.456034 22505 request_logger.go:45] AWS request: ec2/CreateRouteTable
I0627 16:54:08.465938 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api: []
I0627 16:54:08.466493 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet-api/keyset.yaml"
I0627 16:54:08.470468 22505 routetable.go:174] Creating RouteTable with VPC: "vpc-0f71f85232d256375"
I0627 16:54:08.471696 22505 request_logger.go:45] AWS request: ec2/CreateRouteTable
I0627 16:54:08.470468 22505 routetable.go:174] Creating RouteTable with VPC: "vpc-0f71f85232d256375"
I0627 16:54:08.472362 22505 request_logger.go:45] AWS request: ec2/CreateRouteTable
I0627 16:54:08.479747 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.64/27"
I0627 16:54:08.479826 22505 routetable.go:174] Creating RouteTable with VPC: "vpc-0f71f85232d256375"
I0627 16:54:08.483700 22505 request_logger.go:45] AWS request: ec2/CreateRouteTable
I0627 16:54:08.483230 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:08.484640 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.32/27"
I0627 16:54:08.485570 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:08.492592 22505 changes.go:81] Field changed "DHCPOptions" actual="{<nil> <nil> 0xc4201098e8 <nil> <nil> <nil> map[]}" expected="{0xc4213e1030 0xc421021720 0xc4212fae08 0xc4213e1050 0xc4213e1040 0xc420f445cd map[KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned Name:dev-qa.k8s.domain.com]}"
I0627 16:54:08.494856 22505 vpc_dhcpoptions_association.go:89] calling EC2 AssociateDhcpOptions
I0627 16:54:08.495529 22505 request_logger.go:45] AWS request: ec2/AssociateDhcpOptions
I0627 16:54:08.508723 22505 internetgateway.go:163] Creating InternetGatewayAttachment
I0627 16:54:08.509379 22505 request_logger.go:45] AWS request: ec2/AttachInternetGateway
I0627 16:54:08.508789 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.536416 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy/keyset.yaml", falling back to directory-list method
I0627 16:54:08.537087 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kube-proxy/"
I0627 16:54:08.536429 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/keyset.yaml", falling back to directory-list method
I0627 16:54:08.537968 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/"
I0627 16:54:08.561948 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/master/keyset.yaml", falling back to directory-list method
I0627 16:54:08.562714 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/master/"
I0627 16:54:08.570118 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kops/keyset.yaml", falling back to directory-list method
I0627 16:54:08.570514 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kops/"
I0627 16:54:08.570120 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-controller-manager/keyset.yaml", falling back to directory-list method
I0627 16:54:08.572998 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kube-controller-manager/"
I0627 16:54:08.587981 22505 iam_builder.go:300] Ignoring location "s3://my-state-store/dev-qa.k8s.domain.com/pki/" because found parent "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:54:08.588526 22505 iam_builder.go:300] Ignoring location "s3://my-state-store/dev-qa.k8s.domain.com/secrets/" because found parent "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:54:08.588926 22505 iam_builder.go:305] Found root location "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:54:08.589354 22505 iam_builder.go:300] Ignoring location "s3://my-state-store/dev-qa.k8s.domain.com/pki/" because found parent "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:54:08.589592 22505 iam_builder.go:300] Ignoring location "s3://my-state-store/dev-qa.k8s.domain.com/secrets/" because found parent "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:54:08.589810 22505 iam_builder.go:305] Found root location "s3://my-state-store/dev-qa.k8s.domain.com/"
I0627 16:54:08.591481 22505 iamrolepolicy.go:147] Creating IAMRolePolicy
I0627 16:54:08.591872 22505 iamrolepolicy.go:175] PutRolePolicy RoleName=masters.dev-qa.k8s.domain.com PolicyName=masters.dev-qa.k8s.domain.com: {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeRegions",
"ec2:DescribeRouteTables",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeVolumes"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateSecurityGroup",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DescribeVolumesModifications",
"ec2:ModifyInstanceAttribute",
"ec2:ModifyVolume"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"ec2:AttachVolume",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CreateRoute",
"ec2:DeleteRoute",
"ec2:DeleteSecurityGroup",
"ec2:DeleteVolume",
"ec2:DetachVolume",
"ec2:RevokeSecurityGroupIngress"
],
"Resource": [
"*"
],
"Condition": {
"StringEquals": {
"ec2:ResourceTag/KubernetesCluster": "dev-qa.k8s.domain.com"
}
}
},
{
"Effect": "Allow",
"Action": [
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeTags",
"ec2:DescribeLaunchTemplateVersions"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup",
"autoscaling:UpdateAutoScalingGroup"
],
"Resource": [
"*"
],
"Condition": {
"StringEquals": {
"autoscaling:ResourceTag/KubernetesCluster": "dev-qa.k8s.domain.com"
}
}
},
{
"Effect": "Allow",
"Action": [
"elasticloadbalancing:AddTags",
"elasticloadbalancing:AttachLoadBalancerToSubnets",
"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",
"elasticloadbalancing:CreateLoadBalancer",
"elasticloadbalancing:CreateLoadBalancerPolicy",
"elasticloadbalancing:CreateLoadBalancerListeners",
"elasticloadbalancing:ConfigureHealthCheck",
"elasticloadbalancing:DeleteLoadBalancer",
"elasticloadbalancing:DeleteLoadBalancerListeners",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"elasticloadbalancing:DetachLoadBalancerFromSubnets",
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
"elasticloadbalancing:ModifyLoadBalancerAttributes",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"ec2:DescribeVpcs",
"elasticloadbalancing:AddTags",
"elasticloadbalancing:CreateListener",
"elasticloadbalancing:CreateTargetGroup",
"elasticloadbalancing:DeleteListener",
"elasticloadbalancing:DeleteTargetGroup",
"elasticloadbalancing:DeregisterTargets",
"elasticloadbalancing:DescribeListeners",
"elasticloadbalancing:DescribeLoadBalancerPolicies",
"elasticloadbalancing:DescribeTargetGroups",
"elasticloadbalancing:DescribeTargetHealth",
"elasticloadbalancing:ModifyListener",
"elasticloadbalancing:ModifyTargetGroup",
"elasticloadbalancing:RegisterTargets",
"elasticloadbalancing:SetLoadBalancerPoliciesOfListener"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"iam:ListServerCertificates",
"iam:GetServerCertificate"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:GetEncryptionConfiguration",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::my-state-store"
]
},
{
"Effect": "Allow",
"Action": [
"s3:Get*"
],
"Resource": "arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:DeleteObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/backups/etcd/main/*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:DeleteObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::my-state-store/dev-qa.k8s.domain.com/backups/etcd/events/*"
},
{
"Effect": "Allow",
"Action": [
"route53:ChangeResourceRecordSets",
"route53:ListResourceRecordSets",
"route53:GetHostedZone"
],
"Resource": [
"arn:aws:route53:::hostedzone/Z34N9ZS9SNSUL7"
]
},
{
"Effect": "Allow",
"Action": [
"route53:GetChange"
],
"Resource": [
"arn:aws:route53:::change/*"
]
},
{
"Effect": "Allow",
"Action": [
"route53:ListHostedZones"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:GetRepositoryPolicy",
"ecr:DescribeRepositories",
"ecr:ListImages",
"ecr:BatchGetImage"
],
"Resource": [
"*"
]
}
]
}
I0627 16:54:08.607951 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet/keyset.yaml", falling back to directory-list method
I0627 16:54:08.642222 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kubelet/"
I0627 16:54:08.609836 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/keyset.yaml", falling back to directory-list method
I0627 16:54:08.611123 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubecfg/keyset.yaml", falling back to directory-list method
I0627 16:54:08.641887 22505 request_logger.go:45] AWS request: iam/PutRolePolicy
I0627 16:54:08.643804 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/"
I0627 16:54:08.644208 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kubecfg/"
I0627 16:54:08.714955 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-scheduler/keyset.yaml", falling back to directory-list method
I0627 16:54:08.715846 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kube-scheduler/"
I0627 16:54:08.761893 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.771890 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet-api/keyset.yaml", falling back to directory-list method
I0627 16:54:08.772365 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kubelet-api/"
I0627 16:54:08.772182 22505 aws_cloud.go:868] adding tags to "sg-0cdcf150c8ee17b57": map[Name:nodes.dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned]
I0627 16:54:08.772191 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.772198 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.772288 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.776490 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:08.779409 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy: []
I0627 16:54:08.779744 22505 keypair.go:181] creating brand new certificate
I0627 16:54:08.779986 22505 keypair.go:201] Creating PKI keypair "kube-proxy"
I0627 16:54:08.780200 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy/keyset.yaml"
I0627 16:54:08.787529 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator: []
I0627 16:54:08.787996 22505 keypair.go:181] creating brand new certificate
I0627 16:54:08.788221 22505 keypair.go:201] Creating PKI keypair "apiserver-aggregator"
I0627 16:54:08.788427 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/keyset.yaml"
I0627 16:54:08.787604 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/master: []
I0627 16:54:08.789224 22505 keypair.go:181] creating brand new certificate
I0627 16:54:08.789515 22505 keypair.go:201] Creating PKI keypair "master"
I0627 16:54:08.787740 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.789044 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kops: []
I0627 16:54:08.789847 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master/keyset.yaml"
I0627 16:54:08.790613 22505 keypair.go:181] creating brand new certificate
I0627 16:54:08.791608 22505 keypair.go:201] Creating PKI keypair "kops"
I0627 16:54:08.791810 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops/keyset.yaml"
I0627 16:54:08.815730 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-controller-manager: []
I0627 16:54:08.816608 22505 keypair.go:181] creating brand new certificate
I0627 16:54:08.816917 22505 keypair.go:201] Creating PKI keypair "kube-controller-manager"
I0627 16:54:08.817238 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/keyset.yaml"
I0627 16:54:08.844648 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.844675 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.849525 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.864216 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet: []
I0627 16:54:08.864934 22505 keypair.go:181] creating brand new certificate
I0627 16:54:08.865359 22505 keypair.go:201] Creating PKI keypair "kubelet"
I0627 16:54:08.865765 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet/keyset.yaml"
I0627 16:54:08.868896 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client: []
I0627 16:54:08.869621 22505 keypair.go:181] creating brand new certificate
I0627 16:54:08.869933 22505 keypair.go:201] Creating PKI keypair "apiserver-proxy-client"
I0627 16:54:08.870266 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/keyset.yaml"
I0627 16:54:08.873073 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.873150 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.914088 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.937464 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubecfg: []
I0627 16:54:08.938293 22505 keypair.go:181] creating brand new certificate
I0627 16:54:08.938638 22505 keypair.go:201] Creating PKI keypair "kubecfg"
I0627 16:54:08.938995 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg/keyset.yaml"
I0627 16:54:08.943160 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:08.978615 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet-api: []
I0627 16:54:08.979409 22505 keypair.go:181] creating brand new certificate
I0627 16:54:08.978628 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/keyset.yaml", falling back to directory-list method
I0627 16:54:08.978675 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy/keyset.yaml", falling back to directory-list method
I0627 16:54:08.978690 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master/keyset.yaml", falling back to directory-list method
I0627 16:54:08.979722 22505 keypair.go:201] Creating PKI keypair "kubelet-api"
I0627 16:54:08.980178 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/"
I0627 16:54:08.980615 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kube-proxy/"
I0627 16:54:08.981117 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/master/"
I0627 16:54:08.981386 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api/keyset.yaml"
I0627 16:54:08.993244 22505 aws_cloud.go:868] adding tags to "rtb-0f0f3517420890e60": map[Name:dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned kubernetes.io/kops/role:public]
I0627 16:54:08.993871 22505 aws_cloud.go:868] adding tags to "rtb-0964a27e551a895ab": map[Name:private-eu-central-1a.dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned kubernetes.io/kops/role:private-eu-central-1a]
I0627 16:54:08.994683 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:08.993951 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:08.994780 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-scheduler: []
I0627 16:54:08.996060 22505 keypair.go:181] creating brand new certificate
I0627 16:54:08.996382 22505 keypair.go:201] Creating PKI keypair "kube-scheduler"
I0627 16:54:08.996695 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler/keyset.yaml"
I0627 16:54:09.005681 22505 aws_cloud.go:868] adding tags to "rtb-08dbb33ad0858207f": map[Name:private-eu-central-1c.dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned kubernetes.io/kops/role:private-eu-central-1c]
I0627 16:54:09.005871 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops/keyset.yaml", falling back to directory-list method
I0627 16:54:09.006843 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kops/"
I0627 16:54:09.006525 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:09.012906 22505 aws_cloud.go:868] adding tags to "rtb-0296e08527ccea7de": map[KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned kubernetes.io/kops/role:private-eu-central-1b Name:private-eu-central-1b.dev-qa.k8s.domain.com]
I0627 16:54:09.013640 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:09.046219 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/keyset.yaml", falling back to directory-list method
I0627 16:54:09.046938 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/"
I0627 16:54:09.049211 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet/keyset.yaml", falling back to directory-list method
I0627 16:54:09.049721 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kubelet/"
I0627 16:54:09.062908 22505 aws_cloud.go:868] adding tags to "sg-024318132f830658e": map[KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned Name:masters.dev-qa.k8s.domain.com]
I0627 16:54:09.063558 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:09.062925 22505 aws_cloud.go:868] adding tags to "subnet-012648da9a3f30689": map[Name:eu-central-1c.dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned SubnetType:Private kubernetes.io/role/internal-elb:1]
I0627 16:54:09.064496 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:09.092444 22505 aws_cloud.go:868] adding tags to "sg-06daeabdcc934f845": map[KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned Name:bastion-elb.dev-qa.k8s.domain.com]
I0627 16:54:09.094254 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:09.092444 22505 aws_cloud.go:868] adding tags to "igw-0be945eadfd017fcb": map[kubernetes.io/cluster/dev-qa.k8s.domain.com:owned Name:dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com]
I0627 16:54:09.094977 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:09.114295 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:09.125987 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/keyset.yaml", falling back to directory-list method
I0627 16:54:09.126541 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/"
I0627 16:54:09.143520 22505 aws_cloud.go:868] adding tags to "subnet-00d0d5b9df3c16e84": map[kubernetes.io/role/internal-elb:1 Name:eu-central-1b.dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned SubnetType:Private]
I0627 16:54:09.146236 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:09.158447 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg/keyset.yaml", falling back to directory-list method
I0627 16:54:09.158956 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kubecfg/"
I0627 16:54:09.173148 22505 aws_cloud.go:868] adding tags to "subnet-03dccf8dee7a3f038": map[Name:eu-central-1a.dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned SubnetType:Private kubernetes.io/role/internal-elb:1]
I0627 16:54:09.173839 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:09.174898 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy: []
I0627 16:54:09.175609 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy/keyset.yaml"
I0627 16:54:09.187305 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator: []
I0627 16:54:09.187771 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/keyset.yaml"
I0627 16:54:09.189106 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master: []
I0627 16:54:09.189519 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/master/keyset.yaml"
I0627 16:54:09.191185 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api/keyset.yaml", falling back to directory-list method
I0627 16:54:09.191687 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kubelet-api/"
I0627 16:54:09.210072 22505 aws_cloud.go:868] adding tags to "sg-00d5498015189de9e": map[Name:bastion.dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned]
I0627 16:54:09.211039 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:09.215890 22505 aws_cloud.go:868] adding tags to "sg-085f05df237ba6a4d": map[Name:api-elb.dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned]
I0627 16:54:09.216627 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:09.228767 22505 vfs_castore.go:377] no certificate bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler/keyset.yaml", falling back to directory-list method
I0627 16:54:09.229518 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kube-scheduler/"
I0627 16:54:09.259067 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops: []
I0627 16:54:09.260302 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kops/keyset.yaml"
I0627 16:54:09.336235 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager: []
I0627 16:54:09.337809 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-controller-manager/keyset.yaml"
I0627 16:54:09.350148 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:09.362488 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client: []
I0627 16:54:09.363327 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/keyset.yaml"
I0627 16:54:09.368931 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy/keyset.yaml", falling back to directory-list method
I0627 16:54:09.369378 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kube-proxy/"
I0627 16:54:09.372616 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/keyset.yaml", falling back to directory-list method
I0627 16:54:09.373024 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/"
I0627 16:54:09.372755 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet: []
I0627 16:54:09.375479 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet/keyset.yaml"
I0627 16:54:09.374143 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg: []
I0627 16:54:09.376364 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubecfg/keyset.yaml"
I0627 16:54:09.377948 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:09.418799 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/master/keyset.yaml", falling back to directory-list method
I0627 16:54:09.419502 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/master/"
I0627 16:54:09.447803 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api: []
I0627 16:54:09.448460 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet-api/keyset.yaml"
I0627 16:54:09.476382 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:09.476382 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:09.590835 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler: []
I0627 16:54:09.591804 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-scheduler/keyset.yaml"
I0627 16:54:09.591208 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/keyset.yaml", falling back to directory-list method
I0627 16:54:09.592808 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/"
I0627 16:54:09.592812 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator: []
I0627 16:54:09.593829 22505 keypair.go:212] Creating privateKey "apiserver-aggregator"
I0627 16:54:09.591302 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-controller-manager/keyset.yaml", falling back to directory-list method
I0627 16:54:09.592929 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy: []
I0627 16:54:09.591237 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kops/keyset.yaml", falling back to directory-list method
I0627 16:54:09.592992 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet/keyset.yaml", falling back to directory-list method
I0627 16:54:09.594954 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kube-controller-manager/"
I0627 16:54:09.595369 22505 keypair.go:212] Creating privateKey "kube-proxy"
I0627 16:54:09.595684 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kops/"
I0627 16:54:09.596025 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kubelet/"
I0627 16:54:09.605708 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubecfg/keyset.yaml", falling back to directory-list method
I0627 16:54:09.606238 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kubecfg/"
I0627 16:54:09.636674 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/master: []
I0627 16:54:09.637133 22505 keypair.go:212] Creating privateKey "master"
I0627 16:54:09.775638 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-scheduler/keyset.yaml", falling back to directory-list method
I0627 16:54:09.776428 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kube-scheduler/"
I0627 16:54:09.780720 22505 vfs_castore.go:831] no private key bundle "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet-api/keyset.yaml", falling back to directory-list method
I0627 16:54:09.780974 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kubelet-api/"
I0627 16:54:09.789298 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-controller-manager: []
I0627 16:54:09.789716 22505 keypair.go:212] Creating privateKey "kube-controller-manager"
I0627 16:54:09.789307 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client: []
I0627 16:54:09.790403 22505 keypair.go:212] Creating privateKey "apiserver-proxy-client"
I0627 16:54:09.800736 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kops: []
I0627 16:54:09.801060 22505 keypair.go:212] Creating privateKey "kops"
I0627 16:54:09.808947 22505 vfs_castore.go:729] Issuing new certificate: "kube-proxy"
I0627 16:54:09.809260 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/ca/keyset.yaml"
I0627 16:54:09.862875 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet: []
I0627 16:54:09.863396 22505 keypair.go:212] Creating privateKey "kubelet"
I0627 16:54:09.874437 22505 vfs_castore.go:729] Issuing new certificate: "apiserver-aggregator"
I0627 16:54:09.895771 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubecfg: []
I0627 16:54:09.896272 22505 keypair.go:212] Creating privateKey "kubecfg"
I0627 16:54:09.962887 22505 vfs_castore.go:729] Issuing new certificate: "kops"
I0627 16:54:09.974569 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet-api: []
I0627 16:54:09.975028 22505 keypair.go:212] Creating privateKey "kubelet-api"
I0627 16:54:10.004127 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-scheduler: []
I0627 16:54:10.004658 22505 keypair.go:212] Creating privateKey "kube-scheduler"
I0627 16:54:10.023976 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:10.051508 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/ca/keyset.yaml"
I0627 16:54:10.049916 22505 vfs_castore.go:729] Issuing new certificate: "kubelet"
I0627 16:54:10.097510 22505 vfs_castore.go:729] Issuing new certificate: "apiserver-proxy-client"
I0627 16:54:10.124369 22505 vfs_castore.go:729] Issuing new certificate: "kubecfg"
I0627 16:54:10.245387 22505 vfs_castore.go:729] Issuing new certificate: "kubelet-api"
I0627 16:54:10.253460 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:10.253812 22505 privatekey.go:176] Parsing pem block: "RSA PRIVATE KEY"
I0627 16:54:10.254633 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator-ca/keyset.yaml"
I0627 16:54:10.259687 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kube-proxy/"
I0627 16:54:10.337200 22505 vfs_castore.go:729] Issuing new certificate: "kube-scheduler"
I0627 16:54:10.471254 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:10.471726 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator-ca/keyset.yaml"
I0627 16:54:10.483537 22505 vfs_castore.go:729] Issuing new certificate: "master"
I0627 16:54:10.484274 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy: []
I0627 16:54:10.484966 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy/keyset.yaml"
I0627 16:54:10.485205 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kube-proxy/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:10.686566 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-proxy/6707223865817682856550501614.key"
I0627 16:54:10.687181 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kube-proxy/6707223865817682856550501614.key" SSE="AES256" ACL=""
I0627 16:54:10.700399 22505 vfs_castore.go:729] Issuing new certificate: "kube-controller-manager"
I0627 16:54:10.833143 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:10.833900 22505 privatekey.go:176] Parsing pem block: "RSA PRIVATE KEY"
I0627 16:54:10.838320 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/"
I0627 16:54:10.839006 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kops/"
I0627 16:54:10.839128 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kubelet/"
I0627 16:54:10.839780 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/"
I0627 16:54:10.841384 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kubelet-api/"
I0627 16:54:10.842386 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kubecfg/"
I0627 16:54:10.843289 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/master/"
I0627 16:54:10.843771 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kube-scheduler/"
I0627 16:54:10.843917 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/private/kube-controller-manager/"
I0627 16:54:10.937459 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kube-proxy/"
I0627 16:54:11.026783 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator: []
I0627 16:54:11.027705 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/keyset.yaml"
I0627 16:54:11.028053 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:11.026803 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kops: []
I0627 16:54:11.029118 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kops/keyset.yaml"
I0627 16:54:11.029344 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kops/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:11.124778 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet: []
I0627 16:54:11.126004 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet/keyset.yaml"
I0627 16:54:11.126330 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kubelet/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:11.215522 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client: []
I0627 16:54:11.216608 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/keyset.yaml"
I0627 16:54:11.217032 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:11.217039 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet-api: []
I0627 16:54:11.218472 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet-api/keyset.yaml"
I0627 16:54:11.218838 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kubelet-api/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:11.318471 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubecfg: []
I0627 16:54:11.319699 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubecfg/keyset.yaml"
I0627 16:54:11.320010 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kubecfg/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:11.401911 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/master: []
I0627 16:54:11.403163 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/master/keyset.yaml"
I0627 16:54:11.403457 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/master/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:11.410028 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-scheduler: []
I0627 16:54:11.410891 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-scheduler/keyset.yaml"
I0627 16:54:11.411902 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kube-scheduler/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:11.511388 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-controller-manager: []
I0627 16:54:11.512526 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-controller-manager/keyset.yaml"
I0627 16:54:11.512774 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kube-controller-manager/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:11.598118 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy: []
I0627 16:54:11.599045 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy/keyset.yaml"
I0627 16:54:11.599483 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kube-proxy/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:11.643600 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/6707223866098923759050229564.key"
I0627 16:54:11.644312 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/apiserver-aggregator/6707223866098923759050229564.key" SSE="AES256" ACL=""
I0627 16:54:11.725104 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kops/6707223866478860001935658280.key"
I0627 16:54:11.725884 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kops/6707223866478860001935658280.key" SSE="AES256" ACL=""
I0627 16:54:11.803296 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet/6707223866852632538202887245.key"
I0627 16:54:11.803963 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kubelet/6707223866852632538202887245.key" SSE="AES256" ACL=""
I0627 16:54:11.848831 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/6707223867057033895039868977.key"
I0627 16:54:11.849390 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/apiserver-proxy-client/6707223867057033895039868977.key" SSE="AES256" ACL=""
I0627 16:54:11.939663 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubelet-api/6707223867692184395712832722.key"
I0627 16:54:11.941420 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kubelet-api/6707223867692184395712832722.key" SSE="AES256" ACL=""
I0627 16:54:11.939894 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/master/6707223868715036011724290911.key"
I0627 16:54:11.940668 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/"
I0627 16:54:11.942232 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/master/6707223868715036011724290911.key" SSE="AES256" ACL=""
I0627 16:54:11.944783 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy/6707223865817682856550501614.crt"
I0627 16:54:11.945276 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kube-proxy/6707223865817682856550501614.crt" SSE="AES256" ACL=""
I0627 16:54:11.944801 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kubecfg/6707223867172401443404623053.key"
I0627 16:54:11.946286 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kubecfg/6707223867172401443404623053.key" SSE="AES256" ACL=""
I0627 16:54:11.944843 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-scheduler/6707223868086522953879788107.key"
I0627 16:54:11.947181 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kube-scheduler/6707223868086522953879788107.key" SSE="AES256" ACL=""
I0627 16:54:11.956794 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kops/"
I0627 16:54:11.965604 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/private/kube-controller-manager/6707223869646437038107642996.key"
I0627 16:54:11.965944 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/private/kube-controller-manager/6707223869646437038107642996.key" SSE="AES256" ACL=""
I0627 16:54:12.032812 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kubelet/"
I0627 16:54:12.088865 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/"
I0627 16:54:12.134964 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator: []
I0627 16:54:12.136793 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/keyset.yaml"
I0627 16:54:12.137147 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:12.142103 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kubelet-api/"
I0627 16:54:12.142206 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/master/"
I0627 16:54:12.146318 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-proxy/6707223865817682856550501614.crt"
I0627 16:54:12.146399 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kubecfg/"
I0627 16:54:12.184160 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kube-scheduler/"
I0627 16:54:12.184258 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops: []
I0627 16:54:12.185935 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops/keyset.yaml"
I0627 16:54:12.186301 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kops/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:12.250769 22505 s3fs.go:257] Listing objects in S3 bucket "my-state-store" with prefix "dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/"
I0627 16:54:12.282310 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet: []
I0627 16:54:12.283185 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet/keyset.yaml"
I0627 16:54:12.283620 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kubelet/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:12.314491 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client: []
I0627 16:54:12.315506 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/keyset.yaml"
I0627 16:54:12.315899 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:12.330425 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api: []
I0627 16:54:12.330789 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master: []
I0627 16:54:12.331108 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api/keyset.yaml"
I0627 16:54:12.331356 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kubelet-api/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:12.331369 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master/keyset.yaml"
I0627 16:54:12.332460 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/master/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:12.330959 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:12.334535 22505 keypair.go:230] created certificate with cn=system:kube-proxy
I0627 16:54:12.335011 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg: []
I0627 16:54:12.335623 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg/keyset.yaml"
I0627 16:54:12.335990 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kubecfg/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:12.357898 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/6707223866098923759050229564.crt"
I0627 16:54:12.359432 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/6707223866098923759050229564.crt" SSE="AES256" ACL=""
I0627 16:54:12.374688 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler: []
I0627 16:54:12.375354 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler/keyset.yaml"
I0627 16:54:12.375751 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kube-scheduler/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:12.401365 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops/6707223866478860001935658280.crt"
I0627 16:54:12.401969 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kops/6707223866478860001935658280.crt" SSE="AES256" ACL=""
I0627 16:54:12.440183 22505 s3fs.go:285] Listed files in s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager: []
I0627 16:54:12.441506 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/keyset.yaml"
I0627 16:54:12.442037 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/keyset.yaml" SSE="AES256" ACL=""
I0627 16:54:12.480253 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet/6707223866852632538202887245.crt"
I0627 16:54:12.480729 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kubelet/6707223866852632538202887245.crt" SSE="AES256" ACL=""
I0627 16:54:12.519434 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/6707223867057033895039868977.crt"
I0627 16:54:12.520093 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/6707223867057033895039868977.crt" SSE="AES256" ACL=""
I0627 16:54:12.531384 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api/6707223867692184395712832722.crt"
I0627 16:54:12.531785 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kubelet-api/6707223867692184395712832722.crt" SSE="AES256" ACL=""
I0627 16:54:12.532468 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master/6707223868715036011724290911.crt"
I0627 16:54:12.531417 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg/6707223867172401443404623053.crt"
I0627 16:54:12.532904 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/master/6707223868715036011724290911.crt" SSE="AES256" ACL=""
I0627 16:54:12.533406 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kubecfg/6707223867172401443404623053.crt" SSE="AES256" ACL=""
I0627 16:54:12.575491 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-aggregator/6707223866098923759050229564.crt"
I0627 16:54:12.575509 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler/6707223868086522953879788107.crt"
I0627 16:54:12.576194 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kube-scheduler/6707223868086522953879788107.crt" SSE="AES256" ACL=""
I0627 16:54:12.594099 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kops/6707223866478860001935658280.crt"
I0627 16:54:12.657368 22505 s3fs.go:128] Writing file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/6707223869646437038107642996.crt"
I0627 16:54:12.658279 22505 s3fs.go:166] Calling S3 PutObject Bucket="my-state-store" Key="dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/6707223869646437038107642996.crt" SSE="AES256" ACL=""
I0627 16:54:12.729634 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet/6707223866852632538202887245.crt"
I0627 16:54:12.729943 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/master/6707223868715036011724290911.crt"
I0627 16:54:12.750711 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubelet-api/6707223867692184395712832722.crt"
I0627 16:54:12.753150 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/apiserver-proxy-client/6707223867057033895039868977.crt"
I0627 16:54:12.774394 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kubecfg/6707223867172401443404623053.crt"
I0627 16:54:12.781843 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:12.782360 22505 keypair.go:230] created certificate with cn=aggregator
I0627 16:54:12.799874 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-scheduler/6707223868086522953879788107.crt"
I0627 16:54:12.879046 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:12.879719 22505 keypair.go:230] created certificate with cn=kops
I0627 16:54:12.915043 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:12.916181 22505 keypair.go:230] created certificate with cn=kubelet
I0627 16:54:12.932665 22505 s3fs.go:220] Reading file "s3://my-state-store/dev-qa.k8s.domain.com/pki/issued/kube-controller-manager/6707223869646437038107642996.crt"
I0627 16:54:12.936706 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:12.937154 22505 keypair.go:230] created certificate with cn=kubernetes-master
I0627 16:54:12.940380 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:12.940781 22505 keypair.go:230] created certificate with cn=kubelet-api
I0627 16:54:12.958975 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:12.959749 22505 keypair.go:230] created certificate with cn=apiserver-proxy-client
I0627 16:54:12.984482 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:12.985185 22505 keypair.go:230] created certificate with cn=kubecfg
I0627 16:54:12.984500 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:12.985640 22505 keypair.go:230] created certificate with cn=system:kube-scheduler
I0627 16:54:13.179815 22505 certificate.go:103] Parsing pem block: "CERTIFICATE"
I0627 16:54:13.180546 22505 keypair.go:230] created certificate with cn=system:kube-controller-manager
W0627 16:54:13.181206 22505 executor.go:130] error running task "Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com" (9m54s remaining to succeed): error creating subnet: InvalidSubnet.Range: The CIDR '172.31.239.0/30' is invalid.
status code: 400, request id: 1f18b651-4856-4e53-ba95-0202273c336c
W0627 16:54:13.183003 22505 executor.go:130] error running task "Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com" (9m54s remaining to succeed): error creating subnet: InvalidSubnet.Range: The CIDR '172.31.239.8/30' is invalid.
status code: 400, request id: 52d04ca2-43d6-4a62-83e6-5d50b44d0693
W0627 16:54:13.184009 22505 executor.go:130] error running task "Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com" (9m54s remaining to succeed): error creating subnet: InvalidSubnet.Range: The CIDR '172.31.239.4/30' is invalid.
status code: 400, request id: a2158a38-1a74-4526-8ce8-8b554f8c5a65
I0627 16:54:13.184812 22505 executor.go:103] Tasks: 86 done / 143 total; 35 can run
I0627 16:54:13.185150 22505 executor.go:178] Executing task "SecurityGroupRule/api-elb-egress": *awstasks.SecurityGroupRule {"Name":"api-elb-egress","Lifecycle":"Sync","SecurityGroup":{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-085f05df237ba6a4d","Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":null,"Egress":true}
I0627 16:54:13.186436 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.185216 22505 executor.go:178] Executing task "SecurityGroupRule/icmp-pmtu-api-elb-0.0.0.0/0": *awstasks.SecurityGroupRule {"Name":"icmp-pmtu-api-elb-0.0.0.0/0","Lifecycle":"Sync","SecurityGroup":{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-085f05df237ba6a4d","Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":"icmp","FromPort":3,"ToPort":4,"SourceGroup":null,"Egress":null}
I0627 16:54:13.187816 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.185216 22505 executor.go:178] Executing task "RouteTableAssociation/private-eu-central-1a.dev-qa.k8s.domain.com": *awstasks.RouteTableAssociation {"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"RouteTable":{"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"rtb-0964a27e551a895ab","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1a"}},"Subnet":{"Name":"eu-central-1a.dev-qa.k8s.domain.com","ShortName":"eu-central-1a","Lifecycle":"Sync","ID":"subnet-03dccf8dee7a3f038","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.32/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}}
I0627 16:54:13.185250 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:13.185288 22505 executor.go:178] Executing task "SecurityGroupRule/all-master-to-node": *awstasks.SecurityGroupRule {"Name":"all-master-to-node","Lifecycle":"Sync","SecurityGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-0cdcf150c8ee17b57","Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:54:13.185278 22505 executor.go:178] Executing task "SecurityGroupRule/all-master-to-master": *awstasks.SecurityGroupRule {"Name":"all-master-to-master","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:54:13.185291 22505 executor.go:178] Executing task "ElasticIP/eu-central-1c.dev-qa.k8s.domain.com": *awstasks.ElasticIP {"Name":"eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"PublicIP":null,"TagOnSubnet":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedNatGatewayRouteTable":{"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"rtb-08dbb33ad0858207f","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1c"}}}
I0627 16:54:13.185356 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:13.185359 22505 executor.go:178] Executing task "SecurityGroupRule/bastion-elb-egress": *awstasks.SecurityGroupRule {"Name":"bastion-elb-egress","Lifecycle":"Sync","SecurityGroup":{"Name":"bastion-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-06daeabdcc934f845","Description":"Security group for bastion ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":null,"Egress":true}
I0627 16:54:13.185322 22505 executor.go:178] Executing task "SecurityGroupRule/all-node-to-node": *awstasks.SecurityGroupRule {"Name":"all-node-to-node","Lifecycle":"Sync","SecurityGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-0cdcf150c8ee17b57","Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-0cdcf150c8ee17b57","Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:54:13.185367 22505 executor.go:178] Executing task "SecurityGroupRule/bastion-to-master-ssh": *awstasks.SecurityGroupRule {"Name":"bastion-to-master-ssh","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":22,"ToPort":22,"SourceGroup":{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-00d5498015189de9e","Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:54:13.185353 22505 executor.go:178] Executing task "ElasticIP/eu-central-1a.dev-qa.k8s.domain.com": *awstasks.ElasticIP {"Name":"eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"PublicIP":null,"TagOnSubnet":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedNatGatewayRouteTable":{"Name":"private-eu-central-1a.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"rtb-0964a27e551a895ab","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1a.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1a"}}}
I0627 16:54:13.185401 22505 executor.go:178] Executing task "SecurityGroupRule/node-to-master-tcp-2382-4001": *awstasks.SecurityGroupRule {"Name":"node-to-master-tcp-2382-4001","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":2382,"ToPort":4001,"SourceGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-0cdcf150c8ee17b57","Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:54:13.185410 22505 executor.go:178] Executing task "SecurityGroupRule/ssh-external-to-bastion-elb-0.0.0.0/0": *awstasks.SecurityGroupRule {"Name":"ssh-external-to-bastion-elb-0.0.0.0/0","Lifecycle":"Sync","SecurityGroup":{"Name":"bastion-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-06daeabdcc934f845","Description":"Security group for bastion ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":"tcp","FromPort":22,"ToPort":22,"SourceGroup":null,"Egress":null}
I0627 16:54:13.185425 22505 executor.go:178] Executing task "SecurityGroupRule/bastion-egress": *awstasks.SecurityGroupRule {"Name":"bastion-egress","Lifecycle":"Sync","SecurityGroup":{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-00d5498015189de9e","Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":null,"Egress":true}
I0627 16:54:13.185423 22505 executor.go:178] Executing task "SecurityGroupRule/bastion-to-node-ssh": *awstasks.SecurityGroupRule {"Name":"bastion-to-node-ssh","Lifecycle":"Sync","SecurityGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-0cdcf150c8ee17b57","Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":22,"ToPort":22,"SourceGroup":{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-00d5498015189de9e","Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:54:13.185437 22505 executor.go:178] Executing task "SecurityGroupRule/ssh-elb-to-bastion": *awstasks.SecurityGroupRule {"Name":"ssh-elb-to-bastion","Lifecycle":"Sync","SecurityGroup":{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-00d5498015189de9e","Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":22,"ToPort":22,"SourceGroup":{"Name":"bastion-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-06daeabdcc934f845","Description":"Security group for bastion ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:54:13.185462 22505 executor.go:178] Executing task "SecurityGroupRule/node-to-master-protocol-ipip": *awstasks.SecurityGroupRule {"Name":"node-to-master-protocol-ipip","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"4","FromPort":null,"ToPort":null,"SourceGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-0cdcf150c8ee17b57","Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:54:13.185470 22505 executor.go:178] Executing task "ElasticIP/eu-central-1b.dev-qa.k8s.domain.com": *awstasks.ElasticIP {"Name":"eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"PublicIP":null,"TagOnSubnet":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"},"AssociatedNatGatewayRouteTable":{"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"rtb-0296e08527ccea7de","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1b"}}}
I0627 16:54:13.185467 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:13.185479 22505 executor.go:178] Executing task "RouteTableAssociation/private-eu-central-1b.dev-qa.k8s.domain.com": *awstasks.RouteTableAssociation {"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"RouteTable":{"Name":"private-eu-central-1b.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"rtb-0296e08527ccea7de","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1b.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1b"}},"Subnet":{"Name":"eu-central-1b.dev-qa.k8s.domain.com","ShortName":"eu-central-1b","Lifecycle":"Sync","ID":"subnet-00d0d5b9df3c16e84","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.64/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}}
I0627 16:54:13.185496 22505 executor.go:178] Executing task "SecurityGroupRule/node-to-master-tcp-4003-65535": *awstasks.SecurityGroupRule {"Name":"node-to-master-tcp-4003-65535","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":4003,"ToPort":65535,"SourceGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-0cdcf150c8ee17b57","Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:54:13.185514 22505 executor.go:178] Executing task "RouteTableAssociation/private-eu-central-1c.dev-qa.k8s.domain.com": *awstasks.RouteTableAssociation {"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":null,"RouteTable":{"Name":"private-eu-central-1c.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"rtb-08dbb33ad0858207f","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"private-eu-central-1c.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"private-eu-central-1c"}},"Subnet":{"Name":"eu-central-1c.dev-qa.k8s.domain.com","ShortName":"eu-central-1c","Lifecycle":"Sync","ID":"subnet-012648da9a3f30689","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.96/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}}
I0627 16:54:13.185189 22505 executor.go:178] Executing task "SecurityGroupRule/node-egress": *awstasks.SecurityGroupRule {"Name":"node-egress","Lifecycle":"Sync","SecurityGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-0cdcf150c8ee17b57","Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":null,"Egress":true}
I0627 16:54:13.185514 22505 executor.go:178] Executing task "SecurityGroupRule/node-to-master-tcp-1-2379": *awstasks.SecurityGroupRule {"Name":"node-to-master-tcp-1-2379","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":1,"ToPort":2379,"SourceGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-0cdcf150c8ee17b57","Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:54:13.185531 22505 executor.go:178] Executing task "SecurityGroupRule/node-to-master-udp-1-65535": *awstasks.SecurityGroupRule {"Name":"node-to-master-udp-1-65535","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"udp","FromPort":1,"ToPort":65535,"SourceGroup":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-0cdcf150c8ee17b57","Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:54:13.185206 22505 executor.go:178] Executing task "SecurityGroupRule/master-egress": *awstasks.SecurityGroupRule {"Name":"master-egress","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":null,"FromPort":null,"ToPort":null,"SourceGroup":null,"Egress":true}
I0627 16:54:13.185561 22505 executor.go:178] Executing task "SecurityGroupRule/https-api-elb-0.0.0.0/0": *awstasks.SecurityGroupRule {"Name":"https-api-elb-0.0.0.0/0","Lifecycle":"Sync","SecurityGroup":{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-085f05df237ba6a4d","Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":"0.0.0.0/0","Protocol":"tcp","FromPort":443,"ToPort":443,"SourceGroup":null,"Egress":null}
I0627 16:54:13.185565 22505 executor.go:178] Executing task "SecurityGroupRule/https-elb-to-master": *awstasks.SecurityGroupRule {"Name":"https-elb-to-master","Lifecycle":"Sync","SecurityGroup":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"CIDR":null,"Protocol":"tcp","FromPort":443,"ToPort":443,"SourceGroup":{"Name":"api-elb.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-085f05df237ba6a4d","Description":"Security group for api ELB","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"api-elb.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Egress":null}
I0627 16:54:13.185440 22505 executor.go:178] Executing task "Route/0.0.0.0/0": *awstasks.Route {"Name":"0.0.0.0/0","Lifecycle":"Sync","RouteTable":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"rtb-0f0f3517420890e60","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/kops/role":"public"}},"Instance":null,"CIDR":"0.0.0.0/0","InternetGateway":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"igw-0be945eadfd017fcb","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"NatGateway":null}
I0627 16:54:13.185228 22505 executor.go:178] Executing task "LaunchConfiguration/bastions.dev-qa.k8s.domain.com": *awstasks.LaunchConfiguration {"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":true,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"bastions.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"AIPA4CSJTTD5PJWJTK6ZP","Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t2.micro","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":32,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"bastion.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-00d5498015189de9e","Description":"Security group for bastion","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"bastion.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":null}
I0627 16:54:13.185386 22505 executor.go:178] Executing task "LaunchConfiguration/master-eu-central-1c.masters.dev-qa.k8s.domain.com": *awstasks.LaunchConfiguration {"Name":"master-eu-central-1c.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"AIPA4CSJTTD5DOX6RY3MZ","Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}}
I0627 16:54:13.185411 22505 executor.go:178] Executing task "LaunchConfiguration/nodes.dev-qa.k8s.domain.com": *awstasks.LaunchConfiguration {"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"AIPA4CSJTTD5IYDEL47MM","Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"m5a.2xlarge","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":300,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-0cdcf150c8ee17b57","Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}}
I0627 16:54:13.185424 22505 executor.go:178] Executing task "LaunchConfiguration/master-eu-central-1a.masters.dev-qa.k8s.domain.com": *awstasks.LaunchConfiguration {"Name":"master-eu-central-1a.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"AIPA4CSJTTD5DOX6RY3MZ","Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}}
I0627 16:54:13.185337 22505 executor.go:178] Executing task "LaunchConfiguration/master-eu-central-1b.masters.dev-qa.k8s.domain.com": *awstasks.LaunchConfiguration {"Name":"master-eu-central-1b.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"AIPA4CSJTTD5DOX6RY3MZ","Shared":false},"ID":null,"ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}}
I0627 16:54:13.192215 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:13.193127 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:13.194683 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.196792 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.197975 22505 natgateway.go:205] trying to match NatGateway via RouteTable rtb-08dbb33ad0858207f
I0627 16:54:13.200371 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:13.203865 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.205270 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.206739 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.207854 22505 natgateway.go:205] trying to match NatGateway via RouteTable rtb-0964a27e551a895ab
I0627 16:54:13.209635 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.211160 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.212106 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.216273 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.217553 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.219464 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.221208 22505 natgateway.go:205] trying to match NatGateway via RouteTable rtb-0296e08527ccea7de
I0627 16:54:13.222199 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:13.228035 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:13.229342 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.230524 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:13.231396 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.232802 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.234170 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.235131 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.235959 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.241168 22505 request_logger.go:45] AWS request: ec2/DescribeSecurityGroups
I0627 16:54:13.244532 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:13.246655 22505 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:54:13.251086 22505 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:54:13.254008 22505 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:54:13.255833 22505 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:54:13.259103 22505 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:54:13.265529 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:13.267292 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:13.269983 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:13.442526 22505 securitygrouprule.go:278] icmp-pmtu-api-elb-0.0.0.0/0: Calling EC2 AuthorizeSecurityGroupIngress (protocol=icmp fromPort=3 toPort=4 cidr=0.0.0.0/0)
I0627 16:54:13.443582 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:13.673271 22505 routetableassociation.go:149] Checking for existing RouteTableAssociation to subnet
I0627 16:54:13.674102 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:13.695048 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.4/30"
I0627 16:54:13.697315 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:13.889936 22505 securitygrouprule.go:278] all-master-to-node: Calling EC2 AuthorizeSecurityGroupIngress (sourceGroup=sg-024318132f830658e)
I0627 16:54:13.890545 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.057661 22505 securitygrouprule.go:278] all-master-to-master: Calling EC2 AuthorizeSecurityGroupIngress (sourceGroup=sg-024318132f830658e)
I0627 16:54:14.058409 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.094758 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.0/30"
I0627 16:54:14.095443 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:14.313074 22505 securitygrouprule.go:278] all-node-to-node: Calling EC2 AuthorizeSecurityGroupIngress (sourceGroup=sg-0cdcf150c8ee17b57)
I0627 16:54:14.314001 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.370850 22505 natgateway.go:220] no NatGateway found in route table rtb-0964a27e551a895ab
I0627 16:54:14.371508 22505 elastic_ip.go:90] AssociatedNatGatewayRouteTable not found
I0627 16:54:14.371951 22505 elastic_ip.go:237] Creating ElasticIP for VPC
I0627 16:54:14.372395 22505 request_logger.go:45] AWS request: ec2/AllocateAddress
I0627 16:54:14.375027 22505 securitygrouprule.go:278] ssh-external-to-bastion-elb-0.0.0.0/0: Calling EC2 AuthorizeSecurityGroupIngress (protocol=tcp fromPort=22 toPort=22 cidr=0.0.0.0/0)
I0627 16:54:14.375661 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.375823 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.8/30"
I0627 16:54:14.376652 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:14.377816 22505 securitygrouprule.go:278] bastion-to-node-ssh: Calling EC2 AuthorizeSecurityGroupIngress (protocol=tcp fromPort=22 toPort=22 sourceGroup=sg-00d5498015189de9e)
I0627 16:54:14.378214 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.384191 22505 securitygrouprule.go:278] bastion-to-master-ssh: Calling EC2 AuthorizeSecurityGroupIngress (protocol=tcp fromPort=22 toPort=22 sourceGroup=sg-00d5498015189de9e)
I0627 16:54:14.385014 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.385780 22505 route.go:174] Creating Route with RouteTable:"rtb-0f0f3517420890e60" CIDR:"0.0.0.0/0"
I0627 16:54:14.386688 22505 request_logger.go:45] AWS request: ec2/CreateRoute
I0627 16:54:14.384294 22505 securitygrouprule.go:278] node-to-master-tcp-2382-4001: Calling EC2 AuthorizeSecurityGroupIngress (protocol=tcp fromPort=2382 toPort=4001 sourceGroup=sg-0cdcf150c8ee17b57)
I0627 16:54:14.385886 22505 routetableassociation.go:149] Checking for existing RouteTableAssociation to subnet
I0627 16:54:14.388022 22505 securitygrouprule.go:278] https-api-elb-0.0.0.0/0: Calling EC2 AuthorizeSecurityGroupIngress (protocol=tcp fromPort=443 toPort=443 cidr=0.0.0.0/0)
I0627 16:54:14.388999 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.388099 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.388513 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:14.411667 22505 natgateway.go:220] no NatGateway found in route table rtb-08dbb33ad0858207f
I0627 16:54:14.412349 22505 elastic_ip.go:90] AssociatedNatGatewayRouteTable not found
I0627 16:54:14.412723 22505 elastic_ip.go:237] Creating ElasticIP for VPC
I0627 16:54:14.413200 22505 request_logger.go:45] AWS request: ec2/AllocateAddress
I0627 16:54:14.443496 22505 launchconfiguration.go:269] Creating AutoscalingLaunchConfiguration with Name:"bastions.dev-qa.k8s.domain.com-20190627145414"
I0627 16:54:14.444172 22505 aws_cloud.go:1155] Calling DescribeImages to resolve name "595879546273/CoreOS-stable-2135.4.0-hvm"
I0627 16:54:14.444858 22505 request_logger.go:45] AWS request: ec2/DescribeImages
I0627 16:54:14.444951 22505 launchconfiguration.go:269] Creating AutoscalingLaunchConfiguration with Name:"master-eu-central-1c.masters.dev-qa.k8s.domain.com-20190627145414"
I0627 16:54:14.446692 22505 aws_cloud.go:1155] Calling DescribeImages to resolve name "595879546273/CoreOS-stable-2135.4.0-hvm"
I0627 16:54:14.444964 22505 natgateway.go:220] no NatGateway found in route table rtb-0296e08527ccea7de
I0627 16:54:14.447570 22505 request_logger.go:45] AWS request: ec2/DescribeImages
I0627 16:54:14.448152 22505 elastic_ip.go:90] AssociatedNatGatewayRouteTable not found
I0627 16:54:14.448854 22505 elastic_ip.go:237] Creating ElasticIP for VPC
I0627 16:54:14.449251 22505 request_logger.go:45] AWS request: ec2/AllocateAddress
I0627 16:54:14.479943 22505 securitygrouprule.go:278] https-elb-to-master: Calling EC2 AuthorizeSecurityGroupIngress (protocol=tcp fromPort=443 toPort=443 sourceGroup=sg-085f05df237ba6a4d)
I0627 16:54:14.479969 22505 securitygrouprule.go:278] ssh-elb-to-bastion: Calling EC2 AuthorizeSecurityGroupIngress (protocol=tcp fromPort=22 toPort=22 sourceGroup=sg-06daeabdcc934f845)
I0627 16:54:14.481095 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.480731 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.515494 22505 securitygrouprule.go:278] node-to-master-udp-1-65535: Calling EC2 AuthorizeSecurityGroupIngress (protocol=udp fromPort=1 toPort=65535 sourceGroup=sg-0cdcf150c8ee17b57)
I0627 16:54:14.516147 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.515494 22505 securitygrouprule.go:278] node-to-master-tcp-4003-65535: Calling EC2 AuthorizeSecurityGroupIngress (protocol=tcp fromPort=4003 toPort=65535 sourceGroup=sg-0cdcf150c8ee17b57)
I0627 16:54:14.517017 22505 securitygrouprule.go:278] node-to-master-tcp-1-2379: Calling EC2 AuthorizeSecurityGroupIngress (protocol=tcp fromPort=1 toPort=2379 sourceGroup=sg-0cdcf150c8ee17b57)
I0627 16:54:14.517501 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.517060 22505 launchconfiguration.go:269] Creating AutoscalingLaunchConfiguration with Name:"master-eu-central-1b.masters.dev-qa.k8s.domain.com-20190627145414"
I0627 16:54:14.518228 22505 aws_cloud.go:1155] Calling DescribeImages to resolve name "595879546273/CoreOS-stable-2135.4.0-hvm"
I0627 16:54:14.515753 22505 launchconfiguration.go:269] Creating AutoscalingLaunchConfiguration with Name:"nodes.dev-qa.k8s.domain.com-20190627145414"
I0627 16:54:14.515766 22505 launchconfiguration.go:269] Creating AutoscalingLaunchConfiguration with Name:"master-eu-central-1a.masters.dev-qa.k8s.domain.com-20190627145414"
I0627 16:54:14.516886 22505 routetableassociation.go:172] Creating RouteTableAssociation
I0627 16:54:14.517078 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.515497 22505 securitygrouprule.go:278] node-to-master-protocol-ipip: Calling EC2 AuthorizeSecurityGroupIngress (protocol=4 sourceGroup=sg-0cdcf150c8ee17b57)
I0627 16:54:14.517239 22505 routetableassociation.go:149] Checking for existing RouteTableAssociation to subnet
I0627 16:54:14.518763 22505 request_logger.go:45] AWS request: ec2/DescribeImages
I0627 16:54:14.519013 22505 aws_cloud.go:1155] Calling DescribeImages to resolve name "595879546273/CoreOS-stable-2135.4.0-hvm"
I0627 16:54:14.519347 22505 aws_cloud.go:1155] Calling DescribeImages to resolve name "595879546273/CoreOS-stable-2135.4.0-hvm"
I0627 16:54:14.519686 22505 request_logger.go:45] AWS request: ec2/AssociateRouteTable
I0627 16:54:14.520477 22505 request_logger.go:45] AWS request: ec2/AuthorizeSecurityGroupIngress
I0627 16:54:14.520717 22505 request_logger.go:45] AWS request: ec2/DescribeRouteTables
I0627 16:54:14.521358 22505 request_logger.go:45] AWS request: ec2/DescribeImages
I0627 16:54:14.521728 22505 request_logger.go:45] AWS request: ec2/DescribeImages
I0627 16:54:14.673852 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:14.793974 22505 routetableassociation.go:172] Creating RouteTableAssociation
I0627 16:54:14.794574 22505 request_logger.go:45] AWS request: ec2/AssociateRouteTable
I0627 16:54:14.810424 22505 aws_cloud.go:1207] Resolved image "ami-0eb0d9bb7ad1bd1e9"
I0627 16:54:14.810788 22505 aws_cloud.go:1155] Calling DescribeImages to resolve name "595879546273/CoreOS-stable-2135.4.0-hvm"
I0627 16:54:14.811326 22505 request_logger.go:45] AWS request: ec2/DescribeImages
I0627 16:54:14.812413 22505 aws_cloud.go:1207] Resolved image "ami-0eb0d9bb7ad1bd1e9"
I0627 16:54:14.813328 22505 aws_cloud.go:1155] Calling DescribeImages to resolve name "595879546273/CoreOS-stable-2135.4.0-hvm"
I0627 16:54:14.813892 22505 request_logger.go:45] AWS request: ec2/DescribeImages
I0627 16:54:14.875305 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:14.875448 22505 aws_cloud.go:1207] Resolved image "ami-0eb0d9bb7ad1bd1e9"
I0627 16:54:14.876294 22505 aws_cloud.go:1155] Calling DescribeImages to resolve name "595879546273/CoreOS-stable-2135.4.0-hvm"
I0627 16:54:14.876635 22505 request_logger.go:45] AWS request: ec2/DescribeImages
I0627 16:54:14.889665 22505 aws_cloud.go:868] adding tags to "eipalloc-0f881078ab382d5a3": map[Name:eu-central-1a.dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned]
I0627 16:54:14.890091 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:14.894265 22505 aws_cloud.go:1207] Resolved image "ami-0eb0d9bb7ad1bd1e9"
I0627 16:54:14.894581 22505 aws_cloud.go:1155] Calling DescribeImages to resolve name "595879546273/CoreOS-stable-2135.4.0-hvm"
I0627 16:54:14.895798 22505 request_logger.go:45] AWS request: ec2/DescribeImages
I0627 16:54:14.896863 22505 routetableassociation.go:172] Creating RouteTableAssociation
I0627 16:54:14.897280 22505 request_logger.go:45] AWS request: ec2/AssociateRouteTable
I0627 16:54:14.897045 22505 aws_cloud.go:1207] Resolved image "ami-0eb0d9bb7ad1bd1e9"
I0627 16:54:14.897862 22505 aws_cloud.go:1155] Calling DescribeImages to resolve name "595879546273/CoreOS-stable-2135.4.0-hvm"
I0627 16:54:14.898236 22505 request_logger.go:45] AWS request: ec2/DescribeImages
I0627 16:54:14.924885 22505 request_logger.go:45] AWS request: ec2/DescribeTags
I0627 16:54:15.042945 22505 aws_cloud.go:1207] Resolved image "ami-0eb0d9bb7ad1bd1e9"
I0627 16:54:15.043591 22505 launchconfiguration.go:351] AWS CreateLaunchConfiguration bastions.dev-qa.k8s.domain.com-20190627145414
I0627 16:54:15.044155 22505 request_logger.go:45] AWS request: autoscaling/CreateLaunchConfiguration
I0627 16:54:15.047885 22505 aws_cloud.go:1207] Resolved image "ami-0eb0d9bb7ad1bd1e9"
I0627 16:54:15.049274 22505 tagbuilder.go:95] tags: [_aws _k8s_1_6]
I0627 16:54:15.049617 22505 urls.go:162] Using cached protokube location: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz"
I0627 16:54:15.050395 22505 launchconfiguration.go:351] AWS CreateLaunchConfiguration master-eu-central-1c.masters.dev-qa.k8s.domain.com-20190627145414
I0627 16:54:15.051216 22505 request_logger.go:45] AWS request: autoscaling/CreateLaunchConfiguration
I0627 16:54:15.087986 22505 aws_cloud.go:868] adding tags to "eipalloc-0506460cf0a77c11d": map[kubernetes.io/cluster/dev-qa.k8s.domain.com:owned Name:eu-central-1b.dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com]
I0627 16:54:15.088894 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:15.100624 22505 aws_cloud.go:1207] Resolved image "ami-0eb0d9bb7ad1bd1e9"
I0627 16:54:15.102153 22505 tagbuilder.go:95] tags: [_aws _k8s_1_6]
I0627 16:54:15.102516 22505 urls.go:162] Using cached protokube location: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz"
I0627 16:54:15.103021 22505 launchconfiguration.go:351] AWS CreateLaunchConfiguration master-eu-central-1b.masters.dev-qa.k8s.domain.com-20190627145414
I0627 16:54:15.103738 22505 request_logger.go:45] AWS request: autoscaling/CreateLaunchConfiguration
I0627 16:54:15.122781 22505 aws_cloud.go:1207] Resolved image "ami-0eb0d9bb7ad1bd1e9"
I0627 16:54:15.124111 22505 tagbuilder.go:95] tags: [_aws _k8s_1_6]
I0627 16:54:15.124481 22505 urls.go:162] Using cached protokube location: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz"
I0627 16:54:15.125735 22505 launchconfiguration.go:351] AWS CreateLaunchConfiguration nodes.dev-qa.k8s.domain.com-20190627145414
I0627 16:54:15.128403 22505 request_logger.go:45] AWS request: autoscaling/CreateLaunchConfiguration
I0627 16:54:15.137717 22505 aws_cloud.go:1207] Resolved image "ami-0eb0d9bb7ad1bd1e9"
I0627 16:54:15.138827 22505 tagbuilder.go:95] tags: [_aws _k8s_1_6]
I0627 16:54:15.139079 22505 urls.go:162] Using cached protokube location: "https://kubeupv2.s3.amazonaws.com/kops/1.12.1/images/protokube.tar.gz"
I0627 16:54:15.139731 22505 launchconfiguration.go:351] AWS CreateLaunchConfiguration master-eu-central-1a.masters.dev-qa.k8s.domain.com-20190627145414
I0627 16:54:15.140489 22505 request_logger.go:45] AWS request: autoscaling/CreateLaunchConfiguration
I0627 16:54:15.144307 22505 aws_cloud.go:868] adding tags to "eipalloc-0820d0b823c373599": map[Name:eu-central-1c.dev-qa.k8s.domain.com KubernetesCluster:dev-qa.k8s.domain.com kubernetes.io/cluster/dev-qa.k8s.domain.com:owned]
I0627 16:54:15.144876 22505 request_logger.go:45] AWS request: ec2/CreateTags
I0627 16:54:15.152541 22505 elastic_ip.go:275] ElasticIP "35.159.8.196" not tagged on subnet; risk of leaking
I0627 16:54:15.349531 22505 elastic_ip.go:275] ElasticIP "52.57.237.97" not tagged on subnet; risk of leaking
I0627 16:54:15.391317 22505 elastic_ip.go:275] ElasticIP "35.157.248.173" not tagged on subnet; risk of leaking
I0627 16:54:15.429357 22505 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:54:15.433739 22505 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:54:15.757585 22505 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:54:15.767568 22505 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
I0627 16:54:16.105805 22505 request_logger.go:45] AWS request: autoscaling/DescribeLaunchConfigurations
W0627 16:54:16.360811 22505 executor.go:130] error running task "Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com" (9m50s remaining to succeed): error creating subnet: InvalidSubnet.Range: The CIDR '172.31.239.4/30' is invalid.
status code: 400, request id: 91a14c04-0449-4582-884f-0f652b45a7c2
W0627 16:54:16.362043 22505 executor.go:130] error running task "Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com" (9m50s remaining to succeed): error creating subnet: InvalidSubnet.Range: The CIDR '172.31.239.0/30' is invalid.
status code: 400, request id: cf38c826-b99a-4360-8beb-4be16a092290
W0627 16:54:16.362890 22505 executor.go:130] error running task "Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com" (9m50s remaining to succeed): error creating subnet: InvalidSubnet.Range: The CIDR '172.31.239.8/30' is invalid.
status code: 400, request id: 3b5cab27-2401-4203-ac7f-cc519b665d09
I0627 16:54:16.363746 22505 executor.go:103] Tasks: 118 done / 143 total; 7 can run
I0627 16:54:16.364099 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:16.365157 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:16.364166 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:16.366464 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:16.364195 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:16.367849 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:16.364208 22505 executor.go:178] Executing task "AutoscalingGroup/master-eu-central-1b.masters.dev-qa.k8s.domain.com": *awstasks.AutoscalingGroup {"Name":"master-eu-central-1b.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"master-eu-central-1b.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"AIPA4CSJTTD5DOX6RY3MZ","Shared":false},"ID":"master-eu-central-1b.masters.dev-qa.k8s.domain.com-20190627145414","ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"eu-central-1b.dev-qa.k8s.domain.com","ShortName":"eu-central-1b","Lifecycle":"Sync","ID":"subnet-00d0d5b9df3c16e84","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.64/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}],"SuspendProcesses":[],"Tags":{"k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"master-eu-central-1b","k8s.io/role/master":"1"}}
I0627 16:54:16.364231 22505 executor.go:178] Executing task "AutoscalingGroup/master-eu-central-1a.masters.dev-qa.k8s.domain.com": *awstasks.AutoscalingGroup {"Name":"master-eu-central-1a.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"master-eu-central-1a.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"AIPA4CSJTTD5DOX6RY3MZ","Shared":false},"ID":"master-eu-central-1a.masters.dev-qa.k8s.domain.com-20190627145414","ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"eu-central-1a.dev-qa.k8s.domain.com","ShortName":"eu-central-1a","Lifecycle":"Sync","ID":"subnet-03dccf8dee7a3f038","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.32/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}],"SuspendProcesses":[],"Tags":{"k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"master-eu-central-1a","k8s.io/role/master":"1"}}
I0627 16:54:16.364213 22505 executor.go:178] Executing task "AutoscalingGroup/nodes.dev-qa.k8s.domain.com": *awstasks.AutoscalingGroup {"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"AIPA4CSJTTD5IYDEL47MM","Shared":false},"ID":"nodes.dev-qa.k8s.domain.com-20190627145414","ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"m5a.2xlarge","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":300,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"nodes.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-0cdcf150c8ee17b57","Description":"Security group for nodes","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"nodes.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"eu-central-1a.dev-qa.k8s.domain.com","ShortName":"eu-central-1a","Lifecycle":"Sync","ID":"subnet-03dccf8dee7a3f038","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.32/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}},{"Name":"eu-central-1b.dev-qa.k8s.domain.com","ShortName":"eu-central-1b","Lifecycle":"Sync","ID":"subnet-00d0d5b9df3c16e84","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.64/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}},{"Name":"eu-central-1c.dev-qa.k8s.domain.com","ShortName":"eu-central-1c","Lifecycle":"Sync","ID":"subnet-012648da9a3f30689","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.96/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}],"SuspendProcesses":[],"Tags":{"k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"nodes","k8s.io/role/node":"1"}}
I0627 16:54:16.364228 22505 executor.go:178] Executing task "AutoscalingGroup/master-eu-central-1c.masters.dev-qa.k8s.domain.com": *awstasks.AutoscalingGroup {"Name":"master-eu-central-1c.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","Granularity":"1Minute","LaunchConfiguration":{"Name":"master-eu-central-1c.masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","AssociatePublicIP":false,"BlockDeviceMappings":null,"IAMInstanceProfile":{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"AIPA4CSJTTD5DOX6RY3MZ","Shared":false},"ID":"master-eu-central-1c.masters.dev-qa.k8s.domain.com-20190627145414","ImageID":"595879546273/CoreOS-stable-2135.4.0-hvm","InstanceMonitoring":null,"InstanceType":"t3.large","RootVolumeIops":null,"RootVolumeOptimization":null,"RootVolumeSize":100,"RootVolumeType":"gp2","SSHKey":{"Name":"kubernetes.dev-qa.k8s.domain.com-e4:d5:2e:2c:a7:8f:ef:46:9a:fe:37:7a:97:4f:97:5b","Lifecycle":"Sync","PublicKey":{"Name":"","Resource":"ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEAy+YtD9VgdGnDgkYZI7WunUhcKozB15sOPnxpTEcLZGsOvOpJBu8FDmmo3nTct8YdiKnIs+lSVjs71V+Cgu373fOV3F9ugg5890Hfj+KX32vd/bRHInY0h8gICPmZi1PaqPMPJDy5lWCQ3ZDAnDxVi0DIp90kpciEFdPuVxqpVpRr9cK1ePeix+s3y/XnKEh+ohggBmuVwxvk65fzn1s+Lxk+ZXSz4a/Vy7Xww9E3BfpwOhQMhP84/tkDYZeHZQu5az7E1HH6ZwiqrUDZ730co5OqKDkibzeRfqY4VZAZ6sVRkdloyrm+y95JG/KZC7uqUDlJi+i1jxCfJMYoKv4a+98DUYBYx5dGtuq8BgKQRzZ/1mgnQwcWh6O/bAOQjZR8DIgU9npcfXEf9pBDNdmhXCgp2hkoHDJrpcM/rAa+LtoyWe85Z+uGtbtBEtvziXubx7aloYfXhM5dscrP6SUHao9bJgoe7EF2qvQYZXsuNFvqaAF5FMBez3RjWpqCrJl5sZwdPO5lYubgqqzxXZYcXIpTCZQ2feBqfC0fc2jHvqRAx3YGh0A/dZHJXHbKOlVzP5DoLTC50Wwan2xd0ENcFHwWrPfQrwL+GmgB5ph3oysXW0KjV3Vi8kUXt1FllXPr96cV+7sBExTYZKy+lDlPqYH8UyxOKGekhk3BL0MDL3s= k8s-dev-cluster-key\n"},"KeyFingerprint":"c4:3f:f0:10:fc:41:ca:86:95:8f:73:82:99:f5:ec:a8"},"SecurityGroups":[{"Name":"masters.dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"sg-024318132f830658e","Description":"Security group for masters","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"RemoveExtraRules":["port=22","port=443","port=2380","port=2381","port=4001","port=4002","port=4789","port=179"],"Shared":null,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"masters.dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}}],"SpotPrice":"","Tenancy":null,"UserData":{"Name":"","Resource":{}}},"LaunchTemplate":null,"MaxSize":1,"Metrics":["GroupDesiredCapacity","GroupInServiceInstances","GroupMaxSize","GroupMinSize","GroupPendingInstances","GroupStandbyInstances","GroupTerminatingInstances","GroupTotalInstances"],"MinSize":1,"MixedInstanceOverrides":null,"MixedOnDemandAllocationStrategy":null,"MixedOnDemandBase":null,"MixedOnDemandAboveBase":null,"MixedSpotAllocationStrategy":null,"MixedSpotInstancePools":null,"Subnets":[{"Name":"eu-central-1c.dev-qa.k8s.domain.com","ShortName":"eu-central-1c","Lifecycle":"Sync","ID":"subnet-012648da9a3f30689","VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.96/27","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Private","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/internal-elb":"1"}}],"SuspendProcesses":[],"Tags":{"k8s.io/cluster-autoscaler/node-template/label/kops.k8s.io/instancegroup":"master-eu-central-1c","k8s.io/role/master":"1"}}
I0627 16:54:16.374917 22505 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:54:16.377646 22505 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:54:16.381537 22505 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:54:16.391240 22505 request_logger.go:45] AWS request: autoscaling/DescribeAutoScalingGroups
I0627 16:54:16.582991 22505 autoscalinggroup.go:245] Creating autoscaling group with name: master-eu-central-1a.masters.dev-qa.k8s.domain.com
I0627 16:54:16.583712 22505 request_logger.go:45] AWS request: autoscaling/CreateAutoScalingGroup
I0627 16:54:16.587213 22505 autoscalinggroup.go:245] Creating autoscaling group with name: master-eu-central-1b.masters.dev-qa.k8s.domain.com
I0627 16:54:16.587754 22505 request_logger.go:45] AWS request: autoscaling/CreateAutoScalingGroup
I0627 16:54:16.601876 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.4/30"
I0627 16:54:16.603057 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:16.601905 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.0/30"
I0627 16:54:16.603790 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:16.775456 22505 autoscalinggroup.go:245] Creating autoscaling group with name: nodes.dev-qa.k8s.domain.com
I0627 16:54:16.776268 22505 request_logger.go:45] AWS request: autoscaling/CreateAutoScalingGroup
I0627 16:54:16.787334 22505 autoscalinggroup.go:245] Creating autoscaling group with name: master-eu-central-1c.masters.dev-qa.k8s.domain.com
I0627 16:54:16.787857 22505 request_logger.go:45] AWS request: autoscaling/CreateAutoScalingGroup
I0627 16:54:16.804854 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.8/30"
I0627 16:54:16.805581 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:17.157059 22505 request_logger.go:45] AWS request: autoscaling/EnableMetricsCollection
I0627 16:54:17.212662 22505 request_logger.go:45] AWS request: autoscaling/EnableMetricsCollection
I0627 16:54:17.261585 22505 request_logger.go:45] AWS request: autoscaling/EnableMetricsCollection
I0627 16:54:17.282423 22505 request_logger.go:45] AWS request: autoscaling/EnableMetricsCollection
W0627 16:54:17.642852 22505 executor.go:130] error running task "Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com" (9m49s remaining to succeed): error creating subnet: InvalidSubnet.Range: The CIDR '172.31.239.0/30' is invalid.
status code: 400, request id: eba6f0fd-178b-4303-8f27-31e949eaad5d
W0627 16:54:17.644090 22505 executor.go:130] error running task "Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com" (9m49s remaining to succeed): error creating subnet: InvalidSubnet.Range: The CIDR '172.31.239.8/30' is invalid.
status code: 400, request id: 5ad82257-5529-442e-8aec-3a214f974023
W0627 16:54:17.644726 22505 executor.go:130] error running task "Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com" (9m49s remaining to succeed): error creating subnet: InvalidSubnet.Range: The CIDR '172.31.239.4/30' is invalid.
status code: 400, request id: db1dbb8b-dbe1-4d86-b698-fb40290c3861
I0627 16:54:17.645409 22505 executor.go:103] Tasks: 122 done / 143 total; 3 can run
I0627 16:54:17.645730 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:17.645783 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:17.647550 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:17.645806 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1a","CIDR":"172.31.239.0/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1a.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:17.648848 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:17.646745 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:17.862301 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.0/30"
I0627 16:54:17.863156 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:17.862301 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.4/30"
I0627 16:54:17.864002 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
I0627 16:54:18.073743 22505 subnet.go:181] Creating Subnet with CIDR: "172.31.239.8/30"
I0627 16:54:18.074433 22505 request_logger.go:45] AWS request: ec2/CreateSubnet
W0627 16:54:18.772087 22505 executor.go:130] error running task "Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com" (9m48s remaining to succeed): error creating subnet: InvalidSubnet.Range: The CIDR '172.31.239.4/30' is invalid.
status code: 400, request id: 6cce29ba-93c8-4181-9024-87da8e073e81
W0627 16:54:18.773163 22505 executor.go:130] error running task "Subnet/utility-eu-central-1a.dev-qa.k8s.domain.com" (9m48s remaining to succeed): error creating subnet: InvalidSubnet.Range: The CIDR '172.31.239.0/30' is invalid.
status code: 400, request id: ea3bd467-86f3-456f-983a-d87cdbc8f6ba
W0627 16:54:18.773923 22505 executor.go:130] error running task "Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com" (9m48s remaining to succeed): error creating subnet: InvalidSubnet.Range: The CIDR '172.31.239.8/30' is invalid.
status code: 400, request id: 42ee3944-a548-473d-a6df-de9b76e9ef4b
I0627 16:54:18.774641 22505 executor.go:145] No progress made, sleeping before retrying 3 failed task(s)
I0627 16:54:28.775689 22505 executor.go:103] Tasks: 122 done / 143 total; 3 can run
I0627 16:54:28.776548 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1c.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1c","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1c","CIDR":"172.31.239.8/30","Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"utility-eu-central-1c.dev-qa.k8s.domain.com","SubnetType":"Utility","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned","kubernetes.io/role/elb":"1"}}
I0627 16:54:28.777608 22505 request_logger.go:45] AWS request: ec2/DescribeSubnets
I0627 16:54:28.776587 22505 executor.go:178] Executing task "Subnet/utility-eu-central-1b.dev-qa.k8s.domain.com": *awstasks.Subnet {"Name":"utility-eu-central-1b.dev-qa.k8s.domain.com","ShortName":"utility-eu-central-1b","Lifecycle":"Sync","ID":null,"VPC":{"Name":"dev-qa.k8s.domain.com","Lifecycle":"Sync","ID":"vpc-0f71f85232d256375","CIDR":"172.31.239.0/24","EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"dev-qa.k8s.domain.com","Name":"dev-qa.k8s.domain.com","kubernetes.io/cluster/dev-qa.k8s.domain.com":"owned"}},"AvailabilityZone":"eu-central-1b","CIDR":"172.31.239.4/30","Shared":false,"Tags":{"KubernetesCluster":"dev-
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment