Skip to content

Instantly share code, notes, and snippets.

@rterbush
Last active October 21, 2015 19:53
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rterbush/a8f0caccc4b9ab1f6d0f to your computer and use it in GitHub Desktop.
Save rterbush/a8f0caccc4b9ab1f6d0f to your computer and use it in GitHub Desktop.
Failed deploy of salt-minion from git tag
running install_egg_info
Removing /usr/lib/python2.7/site-packages/salt-2015.8.1-py2.7.egg-info
Writing /usr/lib/python2.7/site-packages/salt-2015.8.1-py2.7.egg-info
* INFO: Running install_centos_git_post()
* INFO: Running install_centos_check_services()
* INFO: Running install_centos_restart_daemons()
* INFO: Running daemons_running()
* ERROR: salt-minion was not found running
* ERROR: Failed to run daemons_running()!!!
* ERROR: salt-minion was not found running. Pass '-D' to bootstrap-salt.sh when bootstrapping for additional debugging information...
[DEBUG ] -> /usr/share/man/man7
running install_egg_info
Removing /usr/lib/python2.7/site-packages/salt-2015.8.1-py2.7.egg-info
Writing /usr/lib/python2.7/site-packages/salt-2015.8.1-py2.7.egg-info
* INFO: Running install_centos_git_post()
* INFO: Running install_centos_check_services()
* INFO: Running install_centos_restart_daemons()
* INFO: Running daemons_running()
* ERROR: salt-minion was not found running
* ERROR: Failed to run daemons_running()!!!
* ERROR: salt-minion was not found running. Pass '-D' to bootstrap-salt.sh when bootstrapping for additional debugging information...
[ERROR ] Failed to deploy 'api.0.domainname'. Error: Command 'ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -oPasswordAuthentication=no -oChallengeResponseAuthentication=no -oPubkeyAuthentication=yes -oKbdInteractiveAuthentication=no -i /srv/etc/.ssh/mykey.pem -p 22 centos@10.1.2.103 \'sudo /tmp/.saltcloud-43149ec4-d09f-481f-a1ea-e13df079636f/deploy.sh -c \'"\'"\'/tmp/.saltcloud-43149ec4-d09f-481f-a1ea-e13df079636f\'"\'"\' git v2015.8.1\'' failed. Exit code: 1
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/salt/cloud/__init__.py", line 2109, in run_map
profile, local_master=local_master
File "/usr/lib/python2.7/site-packages/salt/cloud/__init__.py", line 1249, in create
output = self.clouds[func](vm_)
File "/usr/lib/python2.7/site-packages/salt/cloud/clouds/ec2.py", line 2398, in create
for key, value in six.iteritems(salt.utils.cloud.bootstrap(vm_, __opts__)):
File "/usr/lib/python2.7/site-packages/salt/utils/cloud.py", line 501, in bootstrap
deployed = deploy_script(**deploy_kwargs)
File "/usr/lib/python2.7/site-packages/salt/utils/cloud.py", line 1479, in deploy_script
if root_cmd(deploy_command, tty, sudo, **ssh_kwargs) != 0:
File "/usr/lib/python2.7/site-packages/salt/utils/cloud.py", line 2101, in root_cmd
retcode = _exec_ssh_cmd(cmd, allow_failure=allow_failure, **kwargs)
File "/usr/lib/python2.7/site-packages/salt/utils/cloud.py", line 1749, in _exec_ssh_cmd
cmd, proc.exitstatus
SaltCloudSystemExit: Command 'ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -oPasswordAuthentication=no -oChallengeResponseAuthentication=no -oPubkeyAuthentication=yes -oKbdInteractiveAuthentication=no -i /srv/etc/.ssh/mykey.pem -p 22 centos@10.1.2.103 \'sudo /tmp/.saltcloud-43149ec4-d09f-481f-a1ea-e13df079636f/deploy.sh -c \'"\'"\'/tmp/.saltcloud-43149ec4-d09f-481f-a1ea-e13df079636f\'"\'"\' git v2015.8.1\'' failed. Exit code: 1
[INFO ] Complete
[DEBUG ] LazyLoaded nested.output
api.0.domain:
----------
Error:
Command 'ssh -t -t -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oControlPath=none -oPasswordAuthentication=no -oChallengeResponseAuthentication=no -oPubkeyAuthentication=yes -oKbdInteractiveAuthentication=no -i /srv/etc/.ssh/mykey.pem -p 22 centos@10.1.2.103 \'sudo /tmp/.saltcloud-43149ec4-d09f-481f-a1ea-e13df079636f/deploy.sh -c \'"\'"\'/tmp/.saltcloud-43149ec4-d09f-481f-a1ea-e13df079636f\'"\'"\' git v2015.8.1\'' failed. Exit code: 1
@rterbush
Copy link
Author

I am attempting to deploy into ec2 using script_args: tag to deploy from the latest release tag, v2015.8.1

on the minion, if I attempt to start the minion by hand, it complains that the master is refusing to accept the minion key.

if I delete the key as instructed from the master, salt-minion starts up fine and appears to work as expected.

so it seems that somewhere in the process, a key is getting created that blocks the startup of the new minion.

if I use salt-cloud -d to destroy the working minion, it cleans up the key as expected so I know the key does not exist before kicking off a new deploy.

this worked yesterday when deploying using the distro repo for the minion code, but is producing other issues because the distro repos are deploying older minion code that is not playing nicely with 2015.8

@rterbush
Copy link
Author

Another note:

salt-key seems to have an accepted key and denied key of the same name.

[root@manage pki]# salt-key
Accepted Keys:
api.0.domainname
master-minion.domainname
nat.0.domainname
Denied Keys:
api.0.domainname
Unaccepted Keys:
Rejected Keys:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment