Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Deis upgrade issue
INFO [firmware]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware]: adding 30s on to the original 120s timeout to account for the initial delay specified in the liveness / readiness probe
INFO [firmware]: This deployments overall timeout is 150s - batch timout is 150s and there are 1 batches to deploy with a total of 3 pods
INFO [firmware]: waited 10s and 3 pods are in service
INFO [firmware]: waited 20s and 3 pods are in service
INFO [firmware]: waited 30s and 3 pods are in service
INFO [firmware-api]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api]: adding 50s on to the original 120s timeout to account for the initial delay specified in the liveness / readiness probe
INFO [firmware-api]: This deployments overall timeout is 170s - batch timout is 170s and there are 1 batches to deploy with a total of 1 pods
INFO [firmware-api-staging]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api-staging]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api-staging]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api-staging]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api-staging]: buildpack type detected. Defaulting to $PORT 5000
ERROR [firmware-api-staging]: There was a problem deploying v20. Rolling back process types to release v19.
INFO [firmware-api-staging]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api-staging]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api-staging]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api-staging]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api-staging]: buildpack type detected. Defaulting to $PORT 5000
INFO [firmware-api-staging]: adding 50s on to the original 120s timeout to account for the initial delay specified in the liveness / readiness probe
INFO [firmware-api-staging]: This deployments overall timeout is 170s - batch timout is 170s and there are 1 batches to deploy with a total of 1 pods
ERROR [firmware-api-staging]: (app::deploy): There was a problem deploying v20. Rolling back process types to release v19.
('failed to failed to create Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 409 Conflict secrets "objectstorage-keyfile" already exists', 'objectstorage-keyfile', 'firmware-api-staging')
Traceback (most recent call last):
File "/app/scheduler/__init__.py", line 253, in deploy
deployment = self.deployment.get(namespace, name).json()
File "/app/scheduler/resources/deployment.py", line 29, in get
raise KubeHTTPException(response, message, *args)
scheduler.exceptions.KubeHTTPException: ('failed to get Deployment "firmware-api-staging-rake" in Namespace "firmware-api-staging": 404 Not Found deployments.extensions "firmware-api-staging-rake" not found', 'firmware-api-staging-rake', 'firmware-api-staging')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/scheduler/resources/pod.py", line 138, in manifest
self.secret.get(namespace, 'objectstorage-keyfile')
File "/app/scheduler/resources/secret.py", line 25, in get
raise KubeHTTPException(response, message, *args)
scheduler.exceptions.KubeHTTPException: ('failed to get Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 404 Not Found secrets "objectstorage-keyfile" not found', 'objectstorage-keyfile', 'firmware-api-staging')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/api/models/app.py", line 487, in deploy
async_run(tasks)
File "/app/api/utils.py", line 169, in async_run
raise error
File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
result = coro.throw(exc)
File "/app/api/utils.py", line 182, in async_task
yield from loop.run_in_executor(None, params)
File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
yield self # This tells Task to wait for completion.
File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
future.result()
File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
result = self.fn(*self.args, **self.kwargs)
File "/app/scheduler/__init__.py", line 260, in deploy
namespace, name, image, entrypoint, command, **kwargs
File "/app/scheduler/resources/deployment.py", line 111, in create
entrypoint, command, **kwargs)
File "/app/scheduler/resources/deployment.py", line 105, in manifest
manifest['spec']['template'] = self.pod.manifest(namespace, name, image, **kwargs)
File "/app/scheduler/resources/pod.py", line 141, in manifest
self.secret.create(namespace, 'objectstorage-keyfile', secret['data'])
File "/app/scheduler/resources/secret.py", line 88, in create
'failed to create Secret "{}" in Namespace "{}"', name, namespace
scheduler.exceptions.KubeHTTPException: ('failed to failed to create Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 409 Conflict secrets "objectstorage-keyfile" already exists', 'objectstorage-keyfile', 'firmware-api-staging')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/api/models/app.py", line 496, in deploy
raise DeisException('{}\n{}'.format(err, str(e))) from e
api.exceptions.DeisException: There was a problem deploying v20. Rolling back process types to release v19.
('failed to failed to create Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 409 Conflict secrets "objectstorage-keyfile" already exists', 'objectstorage-keyfile', 'firmware-api-staging')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/manage.py", line 12, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.5/dist-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.5/dist-packages/django/core/management/__init__.py", line 359, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 294, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 345, in execute
output = self.handle(*args, **options)
File "/app/api/management/commands/load_db_state_to_k8s.py", line 34, in handle
entrypoint, command, **kwargs)
File "/app/scheduler/resources/deployment.py", line 105, in manifest
manifest['spec']['template'] = self.pod.manifest(namespace, name, image, **kwargs)
File "/app/scheduler/resources/pod.py", line 141, in manifest
self.secret.create(namespace, 'objectstorage-keyfile', secret['data'])
File "/app/scheduler/resources/secret.py", line 88, in create
'failed to create Secret "{}" in Namespace "{}"', name, namespace
scheduler.exceptions.KubeHTTPException: ('failed to failed to create Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 409 Conflict secrets "objectstorage-keyfile" already exists', 'objectstorage-keyfile', 'firmware-api-staging')
Task exception was never retrieved
future: <Task finished coro=<async_task() done, defined at /app/api/utils.py:174> exception=KubeHTTPException('failed to failed to create Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 409 Conflict secrets "objectstorage-keyfile" already exists', 'objectstorage-keyfile', 'firmware-api-staging')>
Traceback (most recent call last):
File "/app/scheduler/__init__.py", line 253, in deploy
deployment = self.deployment.get(namespace, name).json()
File "/app/scheduler/resources/deployment.py", line 29, in get
raise KubeHTTPException(response, message, *args)
scheduler.exceptions.KubeHTTPException: ('failed to get Deployment "firmware-api-staging-worker" in Namespace "firmware-api-staging": 404 Not Found deployments.extensions "firmware-api-staging-worker" not found', 'firmware-api-staging-worker', 'firmware-api-staging')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/scheduler/resources/pod.py", line 138, in manifest
self.secret.get(namespace, 'objectstorage-keyfile')
File "/app/scheduler/resources/secret.py", line 25, in get
raise KubeHTTPException(response, message, *args)
scheduler.exceptions.KubeHTTPException: ('failed to get Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 404 Not Found secrets "objectstorage-keyfile" not found', 'objectstorage-keyfile', 'firmware-api-staging')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
result = coro.throw(exc)
File "/app/api/utils.py", line 182, in async_task
yield from loop.run_in_executor(None, params)
File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
yield self # This tells Task to wait for completion.
File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
future.result()
File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
result = self.fn(*self.args, **self.kwargs)
File "/app/scheduler/__init__.py", line 260, in deploy
namespace, name, image, entrypoint, command, **kwargs
File "/app/scheduler/resources/deployment.py", line 111, in create
entrypoint, command, **kwargs)
File "/app/scheduler/resources/deployment.py", line 105, in manifest
manifest['spec']['template'] = self.pod.manifest(namespace, name, image, **kwargs)
File "/app/scheduler/resources/pod.py", line 141, in manifest
self.secret.create(namespace, 'objectstorage-keyfile', secret['data'])
File "/app/scheduler/resources/secret.py", line 88, in create
'failed to create Secret "{}" in Namespace "{}"', name, namespace
scheduler.exceptions.KubeHTTPException: ('failed to failed to create Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 409 Conflict secrets "objectstorage-keyfile" already exists', 'objectstorage-keyfile', 'firmware-api-staging')
entrypoint, command, **kwargs)
File "/app/scheduler/resources/deployment.py", line 105, in manifest
manifest['spec']['template'] = self.pod.manifest(namespace, name, image, **kwargs)
File "/app/scheduler/resources/pod.py", line 141, in manifest
self.secret.create(namespace, 'objectstorage-keyfile', secret['data'])
File "/app/scheduler/resources/secret.py", line 88, in create
'failed to create Secret "{}" in Namespace "{}"', name, namespace
scheduler.exceptions.KubeHTTPException: ('failed to failed to create Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 409 Conflict secrets "objectstorage-keyfile" already exists', 'objectstorage-keyfile', 'firmware-api-staging')
Task exception was never retrieved
future: <Task finished coro=<async_task() done, defined at /app/api/utils.py:174> exception=KubeHTTPException('failed to failed to create Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 409 Conflict secrets "objectstorage-keyfile" already exists', 'objectstorage-keyfile', 'firmware-api-staging')>
Traceback (most recent call last):
File "/app/scheduler/__init__.py", line 253, in deploy
deployment = self.deployment.get(namespace, name).json()
File "/app/scheduler/resources/deployment.py", line 29, in get
raise KubeHTTPException(response, message, *args)
scheduler.exceptions.KubeHTTPException: ('failed to get Deployment "firmware-api-staging-worker" in Namespace "firmware-api-staging": 404 Not Found deployments.extensions "firmware-api-staging-worker" not found', 'firmware-api-staging-worker', 'firmware-api-staging')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/scheduler/resources/pod.py", line 138, in manifest
self.secret.get(namespace, 'objectstorage-keyfile')
File "/app/scheduler/resources/secret.py", line 25, in get
entrypoint, command, **kwargs)
File "/app/scheduler/resources/deployment.py", line 105, in manifest
manifest['spec']['template'] = self.pod.manifest(namespace, name, image, **kwargs)
File "/app/scheduler/resources/pod.py", line 141, in manifest
self.secret.create(namespace, 'objectstorage-keyfile', secret['data'])
File "/app/scheduler/resources/secret.py", line 88, in create
'failed to create Secret "{}" in Namespace "{}"', name, namespace
scheduler.exceptions.KubeHTTPException: ('failed to failed to create Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 409 Conflict secrets "objectstorage-keyfile" already exists', 'objectstorage-keyfile', 'firmware-api-staging')
Task exception was never retrieved
future: <Task finished coro=<async_task() done, defined at /app/api/utils.py:174> exception=KubeHTTPException('failed to failed to create Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 409 Conflict secrets "objectstorage-keyfile" already exists', 'objectstorage-keyfile', 'firmware-api-staging')>
Traceback (most recent call last):
File "/app/scheduler/__init__.py", line 253, in deploy
deployment = self.deployment.get(namespace, name).json()
File "/app/scheduler/resources/deployment.py", line 29, in get
raise KubeHTTPException(response, message, *args)
scheduler.exceptions.KubeHTTPException: ('failed to get Deployment "firmware-api-staging-worker" in Namespace "firmware-api-staging": 404 Not Found deployments.extensions "firmware-api-staging-worker" not found', 'firmware-api-staging-worker', 'firmware-api-staging')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/scheduler/resources/pod.py", line 138, in manifest
self.secret.get(namespace, 'objectstorage-keyfile')
File "/app/scheduler/resources/secret.py", line 25, in get
raise KubeHTTPException(response, message, *args)
scheduler.exceptions.KubeHTTPException: ('failed to get Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 404 Not Found secrets "objectstorage-keyfile" not found', 'objectstorage-keyfile', 'firmware-api-staging')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
result = coro.throw(exc)
File "/app/api/utils.py", line 182, in async_task
yield from loop.run_in_executor(None, params)
File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
yield self # This tells Task to wait for completion.
File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
future.result()
File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
result = self.fn(*self.args, **self.kwargs)
File "/app/scheduler/__init__.py", line 260, in deploy
namespace, name, image, entrypoint, command, **kwargs
File "/app/scheduler/resources/deployment.py", line 111, in create
entrypoint, command, **kwargs)
File "/app/scheduler/resources/deployment.py", line 105, in manifest
manifest['spec']['template'] = self.pod.manifest(namespace, name, image, **kwargs)
File "/app/scheduler/resources/pod.py", line 141, in manifest
self.secret.create(namespace, 'objectstorage-keyfile', secret['data'])
File "/app/scheduler/resources/secret.py", line 88, in create
'failed to create Secret "{}" in Namespace "{}"', name, namespace
scheduler.exceptions.KubeHTTPException: ('failed to failed to create Secret "objectstorage-keyfile" in Namespace "firmware-api-staging": 409 Conflict secrets "objectstorage-keyfile" already exists', 'objectstorage-keyfile', 'firmware-api-staging')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment