Skip to content

Instantly share code, notes, and snippets.

@joshm1
Created November 23, 2015 15:47
Show Gist options
  • Save joshm1/2b87310ed336abd927fd to your computer and use it in GitHub Desktop.
Save joshm1/2b87310ed336abd927fd to your computer and use it in GitHub Desktop.
kubernetes-master salt failing to update kubelet manifests
root@ip-172-20-0-9:/srv/salt# sudo salt --force-color '*' state.highstate
ip-172-20-0-9.ec2.internal:
----------
ID: pkg-core
Function: pkg.installed
Name: glusterfs-client
Result: True
Comment: Package glusterfs-client is already installed.
Started: 15:42:01.494523
Duration: 1920.381 ms
Changes:
----------
ID: unattended-upgrades
Function: pkg.installed
Result: True
Comment: Package unattended-upgrades is already installed.
Started: 15:42:03.415042
Duration: 0.431 ms
Changes:
----------
ID: /etc/apt/apt.conf.d/20auto-upgrades
Function: file.managed
Result: True
Comment: File /etc/apt/apt.conf.d/20auto-upgrades is in the correct state
Started: 15:42:03.431366
Duration: 27.426 ms
Changes:
----------
ID: pkg-core
Function: pkg.installed
Name: nfs-common
Result: True
Comment: Package nfs-common is already installed.
Started: 15:42:03.458873
Duration: 0.399 ms
Changes:
----------
ID: /opt/kubernetes/helpers
Function: file.directory
Result: True
Comment: Directory /opt/kubernetes/helpers is in the correct state
Started: 15:42:03.459367
Duration: 0.513 ms
Changes:
----------
ID: pkg-core
Function: pkg.installed
Name: apt-transport-https
Result: True
Comment: Package apt-transport-https is already installed.
Started: 15:42:03.459948
Duration: 0.325 ms
Changes:
----------
ID: /opt/kubernetes/helpers/services
Function: file.managed
Result: True
Comment: File /opt/kubernetes/helpers/services is in the correct state
Started: 15:42:03.460370
Duration: 14.983 ms
Changes:
----------
ID: pkg-core
Function: pkg.installed
Name: socat
Result: True
Comment: Package socat is already installed.
Started: 15:42:03.475429
Duration: 0.348 ms
Changes:
----------
ID: pkg-core
Function: pkg.installed
Name: curl
Result: True
Comment: Package curl is already installed.
Started: 15:42:03.475863
Duration: 0.292 ms
Changes:
----------
ID: ntp
Function: pkg.installed
Result: True
Comment: Package ntp is already installed.
Started: 15:42:03.476219
Duration: 0.316 ms
Changes:
----------
ID: pkg-core
Function: pkg.installed
Name: python-apt
Result: True
Comment: Package python-apt is already installed.
Started: 15:42:03.476612
Duration: 0.292 ms
Changes:
----------
ID: ntp-service
Function: service.running
Name: ntp
Result: True
Comment: The service ntp is already running
Started: 15:42:03.477443
Duration: 86.698 ms
Changes:
----------
ID: kube-cert
Function: group.present
Result: True
Comment: Group kube-cert is present and up to date
Started: 15:42:03.564286
Duration: 0.422 ms
Changes:
----------
ID: kubernetes-cert
Function: cmd.script
Result: True
Comment: unless execution succeeded
Started: 15:42:03.565637
Duration: 995.604 ms
Changes:
----------
ID: delete_etc_etcd_dir
Function: file.absent
Name: /etc/etcd
Result: True
Comment: File /etc/etcd is not present
Started: 15:42:04.561389
Duration: 0.345 ms
Changes:
----------
ID: delete_etcd_conf
Function: file.absent
Name: /etc/etcd/etcd.conf
Result: True
Comment: File /etc/etcd/etcd.conf is not present
Started: 15:42:04.561801
Duration: 0.202 ms
Changes:
----------
ID: delete_etcd_default
Function: file.absent
Name: /etc/default/etcd
Result: True
Comment: File /etc/default/etcd is not present
Started: 15:42:04.562066
Duration: 0.195 ms
Changes:
----------
ID: delete_etcd_service_file
Function: file.absent
Name: /lib/systemd/system/etcd.service
Result: True
Comment: File /lib/systemd/system/etcd.service is not present
Started: 15:42:04.562320
Duration: 0.21 ms
Changes:
----------
ID: delete_etcd_initd
Function: file.absent
Name: /etc/init.d/etcd
Result: True
Comment: File /etc/init.d/etcd is not present
Started: 15:42:04.562591
Duration: 0.207 ms
Changes:
----------
ID: stop_etcd-service
Function: service.dead
Name: etcd
Result: True
Comment: The named service etcd is not available
Started: 15:42:04.562872
Duration: 116.052 ms
Changes:
----------
ID: touch /var/log/etcd.log
Function: cmd.run
Result: True
Comment: /var/log/etcd.log exists
Started: 15:42:04.679029
Duration: 0.412 ms
Changes:
----------
ID: touch /var/log/etcd-events.log
Function: cmd.run
Result: True
Comment: /var/log/etcd-events.log exists
Started: 15:42:04.679520
Duration: 0.264 ms
Changes:
----------
ID: /var/etcd
Function: file.directory
Result: True
Comment: Directory /var/etcd is in the correct state
Started: 15:42:04.679849
Duration: 179.338 ms
Changes:
----------
ID: /etc/kubernetes/manifests/etcd.manifest
Function: file.managed
Result: True
Comment: File /etc/kubernetes/manifests/etcd.manifest is in the correct state
Started: 15:42:04.859298
Duration: 148.167 ms
Changes:
----------
ID: /etc/kubernetes/manifests/etcd-events.manifest
Function: file.managed
Result: True
Comment: File /etc/kubernetes/manifests/etcd-events.manifest is in the correct state
Started: 15:42:05.007596
Duration: 34.71 ms
Changes:
----------
ID: /srv/kubernetes/known_tokens.csv
Function: file.managed
Result: True
Comment: File /srv/kubernetes/known_tokens.csv is in the correct state
Started: 15:42:05.042398
Duration: 50.806 ms
Changes:
----------
ID: /srv/kubernetes/basic_auth.csv
Function: file.managed
Result: True
Comment: File /srv/kubernetes/basic_auth.csv is in the correct state
Started: 15:42:05.093286
Duration: 1.389 ms
Changes:
----------
ID: /var/log/kube-apiserver.log
Function: file.managed
Result: True
Comment: File /var/log/kube-apiserver.log exists with proper permissions. No changes made.
Started: 15:42:05.094778
Duration: 0.831 ms
Changes:
----------
ID: docker-upgrade
Function: pkg.installed
Result: True
Comment: All specified packages are already installed.
Started: 15:42:05.227766
Duration: 0.718 ms
Changes:
----------
ID: /etc/sysconfig/docker
Function: file.managed
Result: True
Comment: File /etc/sysconfig/docker is in the correct state
Started: 15:42:05.228568
Duration: 265.274 ms
Changes:
----------
ID: /lib/systemd/system/docker.service
Function: file.managed
Result: True
Comment: File /lib/systemd/system/docker.service is in the correct state
Started: 15:42:05.563369
Duration: 3.269 ms
Changes:
----------
ID: docker
Function: service.running
Result: True
Comment: Service docker is already disabled, and is running
Started: 15:42:05.610229
Duration: 1845.011 ms
Changes:
----------
docker:
True
----------
ID: /usr/local/bin/kubelet
Function: file.managed
Result: True
Comment: File /usr/local/bin/kubelet is in the correct state
Started: 15:42:07.488411
Duration: 1435.454 ms
Changes:
----------
ID: /lib/systemd/system/kubelet.service
Function: file.managed
Result: True
Comment: File /lib/systemd/system/kubelet.service is in the correct state
Started: 15:42:08.923979
Duration: 1.983 ms
Changes:
----------
ID: /etc/sysconfig/kubelet
Function: file.managed
Result: True
Comment: File /etc/sysconfig/kubelet is in the correct state
Started: 15:42:08.926038
Duration: 178.772 ms
Changes:
----------
ID: /var/lib/kubelet/kubeconfig
Function: file.managed
Result: True
Comment: File /var/lib/kubelet/kubeconfig is in the correct state
Started: 15:42:09.104905
Duration: 14.375 ms
Changes:
----------
ID: kubelet
Function: service.running
Result: False
Comment: The named service kubelet is not available
Started: 15:42:09.121382
Duration: 16.52 ms
Changes:
----------
ID: /etc/kubernetes/manifests/kube-apiserver.manifest
Function: file.managed
Result: False
Comment: One or more requisite failed
Started:
Duration:
Changes:
----------
ID: stop_kube-apiserver
Function: service.dead
Name: kube-apiserver
Result: True
Comment: The named service kube-apiserver is not available
Started: 15:42:09.138924
Duration: 26.012 ms
Changes:
----------
ID: /etc/kubernetes/manifests/kube-controller-manager.manifest
Function: file.managed
Result: False
Comment: One or more requisite failed
Started:
Duration:
Changes:
----------
ID: /var/log/kube-controller-manager.log
Function: file.managed
Result: True
Comment: File /var/log/kube-controller-manager.log exists with proper permissions. No changes made.
Started: 15:42:09.165933
Duration: 27.303 ms
Changes:
----------
ID: stop-legacy-kube_controller_manager
Function: service.dead
Name: kube-controller-manager
Result: True
Comment: The named service kube-controller-manager is not available
Started: 15:42:09.193347
Duration: 74.155 ms
Changes:
----------
ID: /etc/kubernetes/manifests/kube-scheduler.manifest
Function: file.managed
Result: False
Comment: One or more requisite failed
Started:
Duration:
Changes:
----------
ID: /var/log/kube-scheduler.log
Function: file.managed
Result: True
Comment: File /var/log/kube-scheduler.log exists with proper permissions. No changes made.
Started: 15:42:09.311449
Duration: 1.202 ms
Changes:
----------
ID: stop_kube-scheduler
Function: service.dead
Name: kube-scheduler
Result: True
Comment: The named service kube-scheduler is not available
Started: 15:42:09.312722
Duration: 70.01 ms
Changes:
----------
ID: delete_cadvisor_manifest
Function: file.absent
Name: /etc/kubernetes/manifests/cadvisor.manifest
Result: True
Comment: File /etc/kubernetes/manifests/cadvisor.manifest is not present
Started: 15:42:09.382831
Duration: 0.32 ms
Changes:
----------
ID: /usr/local/bin/kubectl
Function: file.managed
Result: True
Comment: File /usr/local/bin/kubectl is in the correct state
Started: 15:42:09.383222
Duration: 1157.458 ms
Changes:
----------
ID: /etc/kubernetes/kube-master-addons.sh
Function: file.managed
Result: True
Comment: File /etc/kubernetes/kube-master-addons.sh is in the correct state
Started: 15:42:10.540790
Duration: 14.531 ms
Changes:
----------
ID: master-docker-image-tags
Function: file.touch
Name: /srv/pillar/docker-images.sls
Result: True
Comment: Updated times on file /srv/pillar/docker-images.sls
Started: 15:42:10.555413
Duration: 0.327 ms
Changes:
----------
touched:
/srv/pillar/docker-images.sls
----------
ID: /lib/systemd/system/kube-master-addons.service
Function: file.managed
Result: True
Comment: File /lib/systemd/system/kube-master-addons.service is in the correct state
Started: 15:42:10.555803
Duration: 1.517 ms
Changes:
----------
ID: /lib/systemd/system/kube-master-addons.service
Function: cmd.wait
Name: /opt/kubernetes/helpers/services bounce kube-master-addons
Result: True
Comment: Command "/opt/kubernetes/helpers/services bounce kube-master-addons" run
Started: 15:42:10.568087
Duration: 1361.711 ms
Changes:
----------
pid:
24525
retcode:
0
stderr:
stdout:
----------
ID: /etc/kubernetes/admission-controls/limit-range
Function: file.recurse
Result: True
Comment: The directory /etc/kubernetes/admission-controls/limit-range is in the correct state
Started: 15:42:11.930020
Duration: 86.264 ms
Changes:
----------
ID: /etc/kubernetes/manifests/fluentd-es.yaml
Function: file.managed
Result: True
Comment: File /etc/kubernetes/manifests/fluentd-es.yaml is in the correct state
Started: 15:42:12.016429
Duration: 1.709 ms
Changes:
----------
ID: logrotate
Function: pkg.installed
Result: True
Comment: Package logrotate is already installed.
Started: 15:42:12.018227
Duration: 57.761 ms
Changes:
----------
ID: /etc/logrotate.d/kube-scheduler
Function: file.managed
Result: True
Comment: File /etc/logrotate.d/kube-scheduler is in the correct state
Started: 15:42:12.076109
Duration: 12.099 ms
Changes:
----------
ID: /etc/logrotate.d/kube-proxy
Function: file.managed
Result: True
Comment: File /etc/logrotate.d/kube-proxy is in the correct state
Started: 15:42:12.088348
Duration: 2.809 ms
Changes:
----------
ID: /etc/logrotate.d/kubelet
Function: file.managed
Result: True
Comment: File /etc/logrotate.d/kubelet is in the correct state
Started: 15:42:12.091257
Duration: 102.643 ms
Changes:
----------
ID: /etc/logrotate.d/kube-apiserver
Function: file.managed
Result: True
Comment: File /etc/logrotate.d/kube-apiserver is in the correct state
Started: 15:42:12.194018
Duration: 27.854 ms
Changes:
----------
ID: /etc/logrotate.d/kube-controller-manager
Function: file.managed
Result: True
Comment: File /etc/logrotate.d/kube-controller-manager is in the correct state
Started: 15:42:12.221979
Duration: 3.055 ms
Changes:
----------
ID: /etc/logrotate.d/kube-addons
Function: file.managed
Result: True
Comment: File /etc/logrotate.d/kube-addons is in the correct state
Started: 15:42:12.225114
Duration: 118.183 ms
Changes:
----------
ID: /etc/logrotate.d/docker
Function: file.managed
Result: True
Comment: File /etc/logrotate.d/docker is in the correct state
Started: 15:42:12.441058
Duration: 148.171 ms
Changes:
----------
ID: /etc/logrotate.d/docker-containers
Function: file.managed
Result: True
Comment: File /etc/logrotate.d/docker-containers is in the correct state
Started: 15:42:12.589352
Duration: 17.037 ms
Changes:
----------
ID: /etc/cron.hourly/logrotate
Function: file.managed
Result: True
Comment: File /etc/cron.hourly/logrotate is in the correct state
Started: 15:42:12.606484
Duration: 116.665 ms
Changes:
----------
ID: addon-dir-delete
Function: file.absent
Name: /etc/kubernetes/addons
Result: True
Comment: Removed directory /etc/kubernetes/addons
Started: 15:42:12.723244
Duration: 0.828 ms
Changes:
----------
removed:
/etc/kubernetes/addons
----------
ID: addon-dir-create
Function: file.directory
Name: /etc/kubernetes/addons
Result: True
Comment: Directory /etc/kubernetes/addons updated
Started: 15:42:12.740315
Duration: 9.656 ms
Changes:
----------
/etc/kubernetes/addons:
New Dir
----------
ID: /etc/kubernetes/addons/namespace.yaml
Function: file.managed
Result: True
Comment: File /etc/kubernetes/addons/namespace.yaml updated
Started: 15:42:12.750050
Duration: 61.545 ms
Changes:
----------
diff:
New file
mode:
0644
----------
ID: /etc/kubernetes/addons/cluster-monitoring/influxdb
Function: file.recurse
Result: True
Comment: Recursively updated /etc/kubernetes/addons/cluster-monitoring/influxdb
Started: 15:42:12.811695
Duration: 4061.456 ms
Changes:
----------
/etc/kubernetes/addons/cluster-monitoring/influxdb/grafana-service.yaml:
----------
diff:
New file
mode:
0644
/etc/kubernetes/addons/cluster-monitoring/influxdb/heapster-controller.yaml:
----------
diff:
New file
mode:
0644
/etc/kubernetes/addons/cluster-monitoring/influxdb/heapster-service.yaml:
----------
diff:
New file
mode:
0644
/etc/kubernetes/addons/cluster-monitoring/influxdb/influxdb-grafana-controller.yaml:
----------
diff:
New file
mode:
0644
/etc/kubernetes/addons/cluster-monitoring/influxdb/influxdb-service.yaml:
----------
diff:
New file
mode:
0644
----------
ID: /etc/kubernetes/addons/dns/skydns-svc.yaml
Function: file.managed
Result: True
Comment: File /etc/kubernetes/addons/dns/skydns-svc.yaml updated
Started: 15:42:16.873291
Duration: 16.929 ms
Changes:
----------
diff:
New file
mode:
0644
----------
ID: /etc/kubernetes/addons/dns/skydns-rc.yaml
Function: file.managed
Result: True
Comment: File /etc/kubernetes/addons/dns/skydns-rc.yaml updated
Started: 15:42:16.890326
Duration: 75.218 ms
Changes:
----------
diff:
New file
mode:
0644
----------
ID: /etc/kubernetes/addons/fluentd-elasticsearch
Function: file.recurse
Result: True
Comment: Recursively updated /etc/kubernetes/addons/fluentd-elasticsearch
Started: 15:42:16.965627
Duration: 3554.104 ms
Changes:
----------
/etc/kubernetes/addons/fluentd-elasticsearch/es-controller.yaml:
----------
diff:
New file
mode:
0644
/etc/kubernetes/addons/fluentd-elasticsearch/es-service.yaml:
----------
diff:
New file
mode:
0644
/etc/kubernetes/addons/fluentd-elasticsearch/kibana-controller.yaml:
----------
diff:
New file
mode:
0644
/etc/kubernetes/addons/fluentd-elasticsearch/kibana-service.yaml:
----------
diff:
New file
mode:
0644
----------
ID: /etc/kubernetes/kube-addons.sh
Function: file.managed
Result: True
Comment: File /etc/kubernetes/kube-addons.sh is in the correct state
Started: 15:42:20.519899
Duration: 2.029 ms
Changes:
----------
ID: /etc/kubernetes/kube-addon-update.sh
Function: file.managed
Result: True
Comment: File /etc/kubernetes/kube-addon-update.sh is in the correct state
Started: 15:42:20.522002
Duration: 2.819 ms
Changes:
----------
ID: /lib/systemd/system/kube-addons.service
Function: file.managed
Result: True
Comment: File /lib/systemd/system/kube-addons.service is in the correct state
Started: 15:42:20.524914
Duration: 23.836 ms
Changes:
----------
ID: /lib/systemd/system/kube-addons.service
Function: cmd.wait
Name: /opt/kubernetes/helpers/services bounce kube-addons
Result: True
Comment:
Started: 15:42:20.549553
Duration: 0.22 ms
Changes:
----------
ID: service-kube-addon-stop
Function: service.dead
Name: kube-addons
Result: True
Comment: The named service kube-addons is not available
Started: 15:42:20.549863
Duration: 105.279 ms
Changes:
----------
ID: kube-addons
Function: service.running
Result: False
Comment: The named service kube-addons is not available
Started: 15:42:20.656326
Duration: 120.152 ms
Changes:
----------
ID: bridge-utils
Function: pkg.installed
Result: True
Comment: Package bridge-utils is already installed.
Started: 15:42:20.776596
Duration: 0.408 ms
Changes:
----------
ID: /etc/sysctl.d/99-salt.conf
Function: file.touch
Result: True
Comment: Updated times on file /etc/sysctl.d/99-salt.conf
Started: 15:42:20.777085
Duration: 0.308 ms
Changes:
----------
touched:
/etc/sysctl.d/99-salt.conf
----------
ID: net.ipv4.ip_forward
Function: sysctl.present
Result: True
Comment: Sysctl value net.ipv4.ip_forward = 1 is already set
Started: 15:42:20.777474
Duration: 854.534 ms
Changes:
----------
ID: purge-old-docker-package
Function: pkg.removed
Result: True
Comment: None of the targeted packages are installed
Started: 15:42:21.632131
Duration: 1.846 ms
Changes:
----------
ID: /var/cache/docker-install/lxc-docker-1.7.1_1.7.1_amd64.deb
Function: file.managed
Result: True
Comment: File /var/cache/docker-install/lxc-docker-1.7.1_1.7.1_amd64.deb is in the correct state
Started: 15:42:21.634077
Duration: 113.63 ms
Changes:
----------
ID: /usr/share/doc/docker/apache.txt
Function: file.managed
Result: True
Comment: File /usr/share/doc/docker/apache.txt is in the correct state
Started: 15:42:21.747824
Duration: 14.285 ms
Changes:
----------
ID: fix-service-docker
Function: cmd.wait
Name: /opt/kubernetes/helpers/services bounce docker
Result: False
Comment: The following requisites were not found:
require:
pkg: lxc-docker-1.7.1-1.7.1
Started:
Duration:
Changes:
----------
ID: fix-service-kubelet
Function: cmd.wait
Name: /opt/kubernetes/helpers/services bounce kubelet
Result: True
Comment:
Started: 15:42:21.775494
Duration: 0.298 ms
Changes:
Summary
-------------
Succeeded: 78 (changed=11)
Failed: 6
-------------
Total states run: 84
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment