Skip to content

Instantly share code, notes, and snippets.

@djhaskin987
Created June 28, 2018 22:09
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save djhaskin987/a8124a3bd2efe9ba2e81c8a967af973e to your computer and use it in GitHub Desktop.
Save djhaskin987/a8124a3bd2efe9ba2e81c8a967af973e to your computer and use it in GitHub Desktop.
mistral_hangs
2018-06-28 21:38:37.101 29017 INFO mistral.notifiers.notification_server [req-22840e07-dbe4-4213-9ab7-a2b6ae0fe840 - - - - -] Received RPC request 'notify'[ex_id=fca02cf9-e359-4f15-8399-f87fed8f5b51, event=WORKFLOW_FAILED, timestamp=2018-06-28T21:38:36.000000, data={u'workflow_name': u'examples.mistral-basic', u'updated_at': u'2018-06-28 21:38:36', u'workflow_id': u'39a0a055-68f9-439c-8475-7ecde31af63f', u'id': u'fca02cf9-e359-4f15-8399-f87fed8f5b51', u'state_info': u"Failure caused by error in tasks: task1\n\n task1 [task_ex_id=a9cfeda0-d8ee-4f00-bddf-d807a12ab25d] -> {status: 404, cookies: {}, encoding: None, url: http://127.0.0.1/api/v1/executions, elapsed: 0.006918, content: {u'faultstring': u'The resource could not be found.'}, headers: {u'Content-Length': u'57', u'Access-Control-Allow-Headers': u'Content-Type,Authorization,X-Auth-Token,St2-Api-Key,X-Request-ID', u'Access-Control-Expose-Headers': u'Content-Type,X-Limit,X-Total-Count,X-Request-ID', u'X-Request-ID': u'015a26a0-512d-4570-9706-e85736050037', u'Server': u'nginx/1.14.0', u'Connection': u'keep-alive', u'Access-Control-Allow-Credentials': u'true', u'Date': u'Thu, 28 Jun 2018 21:38:36 GMT', u'Access-Control-Allow-Origin': u'http://127.0.0.1:3000', u'Access-Control-Allow-Methods': u'GET,POST,PUT,DELETE,OPTIONS', u'Content-Type': u'application/json'}, reason: Not Found, history: []}\n [action_ex_id=63a580d4-6f6a-423a-9362-3d7de2e55853, idx=0]: {u'status': 404, u'cookies': {}, u'encoding': None, u'url': u'http://127.0.0.1/api/v1/executions', u'elapsed': 0.006918, u'content': {u'faultstring': u'The resource could not be found.'}, u'headers': {u'Content-Length': u'57', u'Access-Control-Allow-Methods': u'GET,POST,PUT,DELETE,OPTIONS', u'Access-Control-Expose-Headers': u'Content-Type,X-Limit,X-Total-Count,X-Request-ID', u'X-Request-ID': u'015a26a0-512d-4570-9706-e85736050037', u'Server': u'nginx/1.14.0', u'Connection': u'keep-alive', u'Access-Control-Allow-Credentials': u'true', u'Date': u'Thu, 28 Jun 2018 21:38:36 GMT', u'Access-Control-Allow-Origin': u'http://127.0.0.1:3000', u'Access-Control-Allow-Headers': u'Content-Type,Authorization,X-Auth-Token,St2-Api-Key,X-Request-ID', u'Content-Type': u'application/json'}, u'reason': u'Not Found', u'history': []}\n", u'runtime_context': {u'index': 0}, u'state': u'ERROR', u'params': {u'namespace': u'', u'notify': [{u'type': u'st2'}], u'env': {u'st2_liveaction_id': u'5b35555b05b985704b542cf4', u'st2_execution_id': u'5b35555b05b985704b542cf5', u'__actions': {u'st2.action': {u'st2_context': {u'endpoint': u'http://127.0.0.1/api/v1/actionexecutions', u'api_url': u'http://127.0.0.1/api/v1', u'parent': {u'parent': {u'user': u'st2admin', u'execution_id': u'5b35555b05b9857326463351', u'pack': u'examples'}, u'pack': u'examples', u'chain': {u'name': u'mistral-basic', u'parameters': {u'cmd': u'hostname'}, u'publish': None, u'params': None, u'notify': None, u'on_failure': None, u'ref': u'examples.mistral-basic', u'on_success': u'mistral-workbook-complex'}, u'execution_id': u'5b35555b05b985704b542cf5', u'user': u'st2admin'}, u'auth_token': u'2f7d292bf44c4bbca18f7858739e91ea', u'skip_notify_tasks': [], u'notify': {}}}}, u'st2_action_api_url': u'http://127.0.0.1/api/v1'}}, u'workflow_namespace': u'', u'input': {u'cmd': u'hostname', u'timeout': 60}, u'project_id': u'<default-project>', u'spec': {u'tasks': {u'task1': {u'name': u'task1', u'publish': {u'stderr': u'<% task(task1).result.stderr %>', u'stdout': u'<% task(task1).result.stdout %>'}, u'version': u'2.0', u'action': u'st2.action', u'input': {u'ref': u'core.local', u'parameters': {u'cmd': u'<% $.cmd %>', u'timeout': u'<% $.timeout %>'}}, u'type': u'direct'}}, u'description': u'A basic workflow that runs an arbitrary linux command.', u'version': u'2.0', u'input': [u'cmd', u'timeout'], u'output': {u'stdout': u'<% $.stdout %>'}, u'type': u'direct', u'name': u'examples.mistral-basic'}, u'root_execution_id': None, u'description': u'', u'tags': None, u'task_execution_id': None, u'scope': u'private', u'accepted': True, u'name': u'examples.mistral-basic', u'created_at': u'2018-06-28 21:38:36', u'context': {u'__env': {u'st2_liveaction_id': u'5b35555b05b985704b542cf4', u'st2_execution_id': u'5b35555b05b985704b542cf5', u'__actions': {u'st2.action': {u'st2_context': {u'endpoint': u'http://127.0.0.1/api/v1/actionexecutions', u'api_url': u'http://127.0.0.1/api/v1', u'parent': {u'parent': {u'user': u'st2admin', u'execution_id': u'5b35555b05b9857326463351', u'pack': u'examples'}, u'pack': u'examples', u'chain': {u'name': u'mistral-basic', u'parameters': {u'cmd': u'hostname'}, u'publish': None, u'params': None, u'notify': None, u'on_failure': None, u'ref': u'examples.mistral-basic', u'on_success': u'mistral-workbook-complex'}, u'execution_id': u'5b35555b05b985704b542cf5', u'user': u'st2admin'}, u'auth_token': u'2f7d292bf44c4bbca18f7858739e91ea', u'notify': {}, u'skip_notify_tasks': []}}}, u'st2_action_api_url': u'http://127.0.0.1/api/v1'}, u'__execution': {u'id': u'fca02cf9-e359-4f15-8399-f87fed8f5b51'}}, u'output': {u'result': u"Failure caused by error in tasks: task1\n\n task1 [task_ex_id=a9cfeda0-d8ee-4f00-bddf-d807a12ab25d] -> {status: 404, cookies: {}, encoding: None, url: http://127.0.0.1/api/v1/executions, elapsed: 0.006918, content: {u'faultstring': u'The resource could not be found.'}, headers: {u'Content-Length': u'57', u'Access-Control-Allow-Headers': u'Content-Type,Authorization,X-Auth-Token,St2-Api-Key,X-Request-ID', u'Access-Control-Expose-Headers': u'Content-Type,X-Limit,X-Total-Count,X-Request-ID', u'X-Request-ID': u'015a26a0-512d-4570-9706-e85736050037', u'Server': u'nginx/1.14.0', u'Connection': u'keep-alive', u'Access-Control-Allow-Credentials': u'true', u'Date': u'Thu, 28 Jun 2018 21:38:36 GMT', u'Access-Control-Allow-Origin': u'http://127.0.0.1:3000', u'Access-Control-Allow-Methods': u'GET,POST,PUT,DELETE,OPTIONS', u'Content-Type': u'application/json'}, reason: Not Found, history: []}\n [action_ex_id=63a580d4-6f6a-423a-9362-3d7de2e55853, idx=0]: {u'status': 404, u'cookies': {}, u'encoding': None, u'url': u'http://127.0.0.1/api/v1/executions', u'elapsed': 0.006918, u'content': {u'faultstring': u'The resource could not be found.'}, u'headers': {u'Content-Length': u'57', u'Access-Control-Allow-Methods': u'GET,POST,PUT,DELETE,OPTIONS', u'Access-Control-Expose-Headers': u'Content-Type,X-Limit,X-Total-Count,X-Request-ID', u'X-Request-ID': u'015a26a0-512d-4570-9706-e85736050037', u'Server': u'nginx/1.14.0', u'Connection': u'keep-alive', u'Access-Control-Allow-Credentials': u'true', u'Date': u'Thu, 28 Jun 2018 21:38:36 GMT', u'Access-Control-Allow-Origin': u'http://127.0.0.1:3000', u'Access-Control-Allow-Headers': u'Content-Type,Authorization,X-Auth-Token,St2-Api-Key,X-Request-ID', u'Content-Type': u'application/json'}, u'reason': u'Not Found', u'history': []}\n"}}, publishers=[{u'type': u'st2'}]]
2018-06-28 21:38:37.127 29017 INFO st2mistral.notifiers.stackstorm_notifier [req-22840e07-dbe4-4213-9ab7-a2b6ae0fe840 - - - - -] [fca02cf9-e359-4f15-8399-f87fed8f5b51] The workflow event WORKFLOW_FAILED for fca02cf9-e359-4f15-8399-f87fed8f5b51 will be published to st2.
2018-06-28 21:38:37.136 29017 ERROR mistral.notifiers.default_notifier [req-22840e07-dbe4-4213-9ab7-a2b6ae0fe840 - - - - -] Unable to process event for publisher "st2".: Exception: [fca02cf9-e359-4f15-8399-f87fed8f5b51] Unable to publish event because st2 returned status code 404. {
"faultstring": "The resource could not be found."
}
2018-06-28 21:38:37.136 29017 ERROR mistral.notifiers.default_notifier Traceback (most recent call last):
2018-06-28 21:38:37.136 29017 ERROR mistral.notifiers.default_notifier File "/opt/stackstorm/mistral/lib/python2.7/site-packages/mistral/notifiers/default_notifier.py", line 39, in notify
2018-06-28 21:38:37.136 29017 ERROR mistral.notifiers.default_notifier publisher.publish(ex_id, data, event, timestamp, **params)
2018-06-28 21:38:37.136 29017 ERROR mistral.notifiers.default_notifier File "/opt/stackstorm/mistral/lib/python2.7/site-packages/st2mistral/notifiers/stackstorm_notifier.py", line 297, in publish
2018-06-28 21:38:37.136 29017 ERROR mistral.notifiers.default_notifier func(ex_id, data, event, timestamp, **kwargs)
2018-06-28 21:38:37.136 29017 ERROR mistral.notifiers.default_notifier File "/opt/stackstorm/mistral/lib/python2.7/site-packages/st2mistral/notifiers/stackstorm_notifier.py", line 145, in on_workflow_status_update
2018-06-28 21:38:37.136 29017 ERROR mistral.notifiers.default_notifier 'status code %s. %s' % (root_id, resp.status_code, resp.text)
2018-06-28 21:38:37.136 29017 ERROR mistral.notifiers.default_notifier Exception: [fca02cf9-e359-4f15-8399-f87fed8f5b51] Unable to publish event because st2 returned status code 404. {
2018-06-28 21:38:37.136 29017 ERROR mistral.notifiers.default_notifier "faultstring": "The resource could not be found."
2018-06-28 21:38:37.136 29017 ERROR mistral.notifiers.default_notifier }
2018-06-28 21:38:37.136 29017 ERROR mistral.notifiers.default_notifier
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment