airflow-dagster-failed-task-log.txt

*** Reading local file: /usr/local/home/airflow/logs/programs_pipeline/intellect_program_full_pipeline_solid.program_table_name/2020-01-24T14:47:12.884249+00:00/1.log
[2020-01-24 14:51:02,651] {{base_task_runner.py:133}} INFO - Running: ['airflow', 'run', 'programs_pipeline', 'intellect_program_full_pipeline_solid.program_table_name', '2020-01-24T14:47:12.884249+00:00', '--job_id', '138', '--pool', 'default_pool', '--raw', '-sd', 'DAGS_FOLDER/programs_pipeline.py', '--cfg_path', '/tmp/tmp12w6ki2u']
[2020-01-24 14:51:03,087] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name /usr/lib/python3.7/site-packages/airflow/utils/helpers.py:36: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
[2020-01-24 14:51:03,088] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   from collections import Iterable
[2020-01-24 14:51:03,190] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name [2020-01-24 14:51:03,190] {{settings.py:252}} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=33245
[2020-01-24 14:51:03,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name [2020-01-24 14:51:03,903] {{__init__.py:51}} INFO - Using executor LocalExecutor
[2020-01-24 14:51:03,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name [2020-01-24 14:51:03,904] {{dagbag.py:92}} INFO - Filling up the DagBag from /usr/local/home/airflow/dags/programs_pipeline.py
[2020-01-24 14:51:04,146] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name /usr/local/home/airflow/dags/programs_pipeline.py:22: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
[2020-01-24 14:51:04,146] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   storage_env = yaml.load(STORAGE_ENVIRONMENT)
[2020-01-24 14:51:04,147] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name /usr/local/home/airflow/dags/programs_pipeline.py:34: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
[2020-01-24 14:51:04,147] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   pipeline_env = yaml.load(environment_file)
[2020-01-24 14:51:04,171] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name /usr/lib/python3.7/site-packages/dagster/core/serdes/__init__.py:199: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
[2020-01-24 14:51:04,172] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   config_dict = yaml.load(self.config_yaml)
[2020-01-24 14:51:04,258] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name /usr/lib/python3.7/site-packages/dagster/core/serdes/__init__.py:199: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
[2020-01-24 14:51:04,258] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   config_dict = yaml.load(self.config_yaml)
[2020-01-24 14:51:04,451] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name /usr/lib/python3.7/site-packages/dagster/utils/yaml_utils.py:36: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
[2020-01-24 14:51:04,451] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   return yaml.load(ff)
[2020-01-24 14:51:04,856] {{python_operator.py:99}} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_ID=programs_pipeline
AIRFLOW_CTX_TASK_ID=intellect_program_full_pipeline_solid.program_table_name
AIRFLOW_CTX_EXECUTION_DATE=2020-01-24T14:47:12.884249+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2020-01-24T14:47:12.884249+00:00
[2020-01-24 14:51:04,902] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name Traceback (most recent call last):
[2020-01-24 14:51:04,902] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
[2020-01-24 14:51:04,902] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     cursor, statement, parameters, context
[2020-01-24 14:51:04,902] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 581, in do_execute
[2020-01-24 14:51:04,902] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     cursor.execute(statement, parameters)
[2020-01-24 14:51:04,902] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name sqlite3.IntegrityError: UNIQUE constraint failed: runs.run_id
[2020-01-24 14:51:04,902] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name 
[2020-01-24 14:51:04,902] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name The above exception was the direct cause of the following exception:
[2020-01-24 14:51:04,902] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name 
[2020-01-24 14:51:04,902] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name Traceback (most recent call last):
[2020-01-24 14:51:04,903] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/bin/airflow", line 37, in <module>
[2020-01-24 14:51:04,903] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     args.func(args)
[2020-01-24 14:51:04,903] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/airflow/utils/cli.py", line 74, in wrapper
[2020-01-24 14:51:04,903] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     return f(*args, **kwargs)
[2020-01-24 14:51:04,903] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/airflow/bin/cli.py", line 551, in run
[2020-01-24 14:51:04,903] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     _run(args, dag, ti)
[2020-01-24 14:51:04,903] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/airflow/bin/cli.py", line 469, in _run
[2020-01-24 14:51:04,903] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     pool=args.pool,
[2020-01-24 14:51:04,903] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper
[2020-01-24 14:51:04,903] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     return func(*args, **kwargs)
[2020-01-24 14:51:04,903] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 930, in _run_raw_task
[2020-01-24 14:51:04,903] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     result = task_copy.execute(context=context)
[2020-01-24 14:51:04,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/dagster_airflow/vendor/python_operator.py", line 108, in execute
[2020-01-24 14:51:04,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     return_value = self.execute_callable()
[2020-01-24 14:51:04,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/dagster_airflow/vendor/python_operator.py", line 113, in execute_callable
[2020-01-24 14:51:04,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     return self.python_callable(*self.op_args, **self.op_kwargs)
[2020-01-24 14:51:04,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/dagster_airflow/operators/python_operator.py", line 71, in python_callable
[2020-01-24 14:51:04,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     status=PipelineRunStatus.MANAGED,
[2020-01-24 14:51:04,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/dagster/core/instance/__init__.py", line 283, in get_or_create_run
[2020-01-24 14:51:04,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     return self.create_run(pipeline_run)
[2020-01-24 14:51:04,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/dagster/core/instance/__init__.py", line 275, in create_run
[2020-01-24 14:51:04,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     run = self._run_storage.add_run(pipeline_run)
[2020-01-24 14:51:04,904] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/dagster/core/storage/runs/sql_run_storage.py", line 39, in add_run
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     conn.execute(runs_insert)
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 982, in execute
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     return meth(self, multiparams, params)
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     return connection._execute_clauseelement(self, multiparams, params)
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1101, in _execute_clauseelement
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     distilled_params,
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1250, in _execute_context
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     e, statement, parameters, cursor, context
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     util.raise_from_cause(sqlalchemy_exception, exc_info)
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
[2020-01-24 14:51:04,905] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     reraise(type(exception), exception, tb=exc_tb, cause=cause)
[2020-01-24 14:51:04,906] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
[2020-01-24 14:51:04,906] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     raise value.with_traceback(tb)
[2020-01-24 14:51:04,906] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
[2020-01-24 14:51:04,906] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     cursor, statement, parameters, context
[2020-01-24 14:51:04,906] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name   File "/usr/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 581, in do_execute
[2020-01-24 14:51:04,906] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name     cursor.execute(statement, parameters)
[2020-01-24 14:51:04,906] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: runs.run_id
[2020-01-24 14:51:04,906] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name [SQL: INSERT INTO runs (run_id, pipeline_name, status, run_body) VALUES (?, ?, ?, ?)]
[2020-01-24 14:51:04,906] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name [parameters: ('manual__2020-01-24T14:47:12.884249+00:00', 'programs_pipeline', 'MANAGED', '{"__class__": "PipelineRun", "environment_dict": {"resources": {"db_info": {"config": {"postgres_db_name": "data_resource_dev", "postgres_hostname":  ... (4239 characters truncated) ... ", "name": "programs_pipeline", "solid_subset": null}, "status": {"__enum__": "PipelineRunStatus.MANAGED"}, "step_keys_to_execute": null, "tags": {}}')]
[2020-01-24 14:51:04,906] {{base_task_runner.py:115}} INFO - Job 138: Subtask intellect_program_full_pipeline_solid.program_table_name (Background on this error at: http://sqlalche.me/e/gkpj)