gistfile1.txt

[2021-07-29 23:23:37,320] {taskinstance.py:1302} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=test_airflow_retry
AIRFLOW_CTX_TASK_ID=fail_every_other
AIRFLOW_CTX_EXECUTION_DATE=2019-02-01T01:01:01+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2019-02-01T01:01:01+00:00
[2021-07-29 23:23:37,324] {base_component_launcher.py:195} INFO - Running driver for fail_every_other
[2021-07-29 23:23:37,327] {metadata_store.py:94} INFO - MetadataStore with DB connection initialized
[2021-07-29 23:23:37,338] {taskinstance.py:1501} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/home/jupyter/env/lib/python3.7/site-packages/tfx/orchestration/metadata.py", line 698, in register_execution
    contexts=contexts + [component_run_context])
  File "/home/jupyter/env/lib/python3.7/site-packages/ml_metadata/metadata_store/metadata_store.py", line 577, in put_execution
    self._call('PutExecution', request, response)
  File "/home/jupyter/env/lib/python3.7/site-packages/ml_metadata/metadata_store/metadata_store.py", line 174, in _call
    return self._call_method(method_name, request, response)
  File "/home/jupyter/env/lib/python3.7/site-packages/ml_metadata/metadata_store/metadata_store.py", line 195, in _call_method
    self._pywrap_cc_call(cc_method, request, response)
  File "/home/jupyter/env/lib/python3.7/site-packages/ml_metadata/metadata_store/metadata_store.py", line 226, in _pywrap_cc_call
    raise _make_exception(error_message.decode('utf-8'), status_code)
ml_metadata.errors.AlreadyExistsError: Given node already exists: type_id: 4
name: "test_airflow_retry.manual__2019-02-01T01:01:01+00:00.fail_every_other"
properties {
  key: "component_id"
  value {
    string_value: "fail_every_other"
  }
}
properties {
  key: "pipeline_name"
  value {
    string_value: "test_airflow_retry"
  }
}
properties {
  key: "run_id"
  value {
    string_value: "manual__2019-02-01T01:01:01+00:00"
  }
}
Internal: Error when executing query: UNIQUE constraint failed: Context.type_id, Context.name query:  INSERT INTO `Context`(    `type_id`, `name`,    `create_time_since_epoch`, `last_update_time_since_epoch` ) VALUES(4, 'test_airflow_retry.manual__2019-02-01T01:01:01+00:00.fail_every_other', 1627601017337, 1627601017337);
	Cannot create node for type_id: 4 name: "test_airflow_retry.manual__2019-02-01T01:01:01+00:00.fail_every_other" properties { key: "component_id" value { string_value: "fail_every_other" } } properties { key: "pipeline_name" value { string_value: "test_airflow_retry" } } properties { key: "run_id" value { string_value: "manual__2019-02-01T01:01:01+00:00" } }

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jupyter/env/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1157, in _run_raw_task
    self._prepare_and_execute_task_with_callbacks(context, task)
  File "/home/jupyter/env/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1331, in _prepare_and_execute_task_with_callbacks
    result = self._execute_task(context, task_copy)
  File "/home/jupyter/env/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1361, in _execute_task
    result = task_copy.execute(context=context)
  File "/home/jupyter/env/lib/python3.7/site-packages/airflow/operators/python.py", line 150, in execute
    return_value = self.execute_callable()
  File "/home/jupyter/env/lib/python3.7/site-packages/airflow/operators/python.py", line 161, in execute_callable
    return self.python_callable(*self.op_args, **self.op_kwargs)
  File "/home/jupyter/env/lib/python3.7/site-packages/tfx/orchestration/airflow/airflow_component.py", line 77, in _airflow_component_launcher
    launcher.launch()
  File "/home/jupyter/env/lib/python3.7/site-packages/tfx/orchestration/launcher/base_component_launcher.py", line 197, in launch
    self._exec_properties)
  File "/home/jupyter/env/lib/python3.7/site-packages/tfx/orchestration/launcher/base_component_launcher.py", line 166, in _run_driver
    component_info=self._component_info)
  File "/home/jupyter/env/lib/python3.7/site-packages/tfx/dsl/components/base/base_driver.py", line 277, in pre_execution
    contexts=contexts)
  File "/home/jupyter/env/lib/python3.7/site-packages/tfx/orchestration/metadata.py", line 712, in register_execution
    contexts=contexts + [component_run_context])
  File "/home/jupyter/env/lib/python3.7/site-packages/ml_metadata/metadata_store/metadata_store.py", line 577, in put_execution
    self._call('PutExecution', request, response)
  File "/home/jupyter/env/lib/python3.7/site-packages/ml_metadata/metadata_store/metadata_store.py", line 174, in _call
    return self._call_method(method_name, request, response)
  File "/home/jupyter/env/lib/python3.7/site-packages/ml_metadata/metadata_store/metadata_store.py", line 195, in _call_method
    self._pywrap_cc_call(cc_method, request, response)
  File "/home/jupyter/env/lib/python3.7/site-packages/ml_metadata/metadata_store/metadata_store.py", line 226, in _pywrap_cc_call
    raise _make_exception(error_message.decode('utf-8'), status_code)
ml_metadata.errors.InvalidArgumentError: Given type_id 6 is different from the one known before: 3
[2021-07-29 23:23:37,341] {taskinstance.py:1551} INFO - Marking task as UP_FOR_RETRY. dag_id=test_airflow_retry, task_id=fail_every_other, execution_date=20190201T010101, start_date=20210729T232337, end_date=20210729T232337
[2021-07-29 23:23:37,394] {local_task_job.py:149} INFO - Task exited with return code 1