Last active
August 6, 2023 01:05
-
-
Save potiuk/6bd2d855a01ed3e8877a30cefcec01b3 to your computer and use it in GitHub Desktop.
Examples of deadlocks caused by
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| See https://github.com/apache/airflow/issues/32778 | |
| 2023-08-05 21:33:46.760 UTC [1045] DETAIL: Process 1045 waits for ShareLock on transaction 15501; blocked by process 1124. | |
| Process 1124 waits for ShareLock on transaction 15500; blocked by process 1045. | |
| Process 1045: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T21:33:42.780404+00:00'::timestamptz WHERE dag_run.id = 1 | |
| Process 1124: UPDATE task_instance SET start_date='2023-08-05T21:33:45.762562+00:00'::timestamptz, end_date='2023-08-05T21:33:45.762562+00:00'::timestamptz, duration=0.0, state='skipped', updated_at='2023-08-05T21:33:45.765508+00:00'::timestamptz WHERE task_instance.dag_id = 'example_bash_operator' AND task_instance.task_id = 'run_this_last' AND task_instance.run_id = 'backfill__2017-01-01T00:00:00+00:00' AND task_instance.map_index = -1 | |
| 2023-08-05 21:33:46.760 UTC [1045] HINT: See server log for query details. | |
| 2023-08-05 21:33:46.760 UTC [1045] CONTEXT: while updating tuple (0,228) in relation "dag_run" | |
| 2023-08-05 21:33:46.760 UTC [1045] STATEMENT: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T21:33:42.780404+00:00'::timestamptz WHERE dag_run.id = 1 | |
| 2023-08-05 22:33:14.472 UTC [6341] ERROR: deadlock detected | |
| 2023-08-05 22:33:14.472 UTC [6341] DETAIL: Process 6341 waits for ShareLock on transaction 22384; blocked by process 6441. | |
| Process 6441 waits for ShareLock on transaction 22383; blocked by process 6341. | |
| Process 6341: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T22:33:09.303686+00:00'::timestamptz WHERE dag_run.id = 1 | |
| Process 6441: UPDATE task_instance SET state='scheduled', updated_at='2023-08-05T22:33:13.473951+00:00'::timestamptz WHERE task_instance.dag_id = 'example_bash_operator' AND task_instance.run_id = 'backfill__2017-01-01T00:00:00+00:00' AND (task_instance.task_id, task_instance.map_index) IN (('run_after_loop', -1)) | |
| 2023-08-05 22:33:14.472 UTC [6341] HINT: See server log for query details. | |
| 2023-08-05 22:33:14.472 UTC [6341] CONTEXT: while updating tuple (1,41) in relation "dag_run" | |
| 2023-08-05 22:33:14.472 UTC [6341] STATEMENT: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T22:33:09.303686+00:00'::timestamptz WHERE dag_run.id = 1 | |
| 2023-08-05 22:33:49.519 UTC [6601] ERROR: deadlock detected | |
| 2023-08-05 22:33:49.519 UTC [6601] DETAIL: Process 6601 waits for ShareLock on transaction 27423; blocked by process 6698. | |
| Process 6698 waits for ShareLock on transaction 27422; blocked by process 6601. | |
| Process 6601: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T22:33:48.518752+00:00'::timestamptz WHERE dag_run.id = 1 | |
| Process 6698: UPDATE task_instance SET state='scheduled', updated_at='2023-08-05T22:33:48.524549+00:00'::timestamptz WHERE task_instance.dag_id = 'example_bash_operator' AND task_instance.run_id = 'backfill__2017-01-01T00:00:00+00:00' AND (task_instance.task_id, task_instance.map_index) IN (('run_after_loop', -1)) | |
| 2023-08-05 22:33:49.519 UTC [6601] HINT: See server log for query details. | |
| 2023-08-05 22:33:49.519 UTC [6601] CONTEXT: while updating tuple (1,45) in relation "dag_run" | |
| 2023-08-05 22:33:49.519 UTC [6601] STATEMENT: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T22:33:48.518752+00:00'::timestamptz WHERE dag_run.id = 1 | |
| 2023-08-05 22:34:56.394 UTC [7131] ERROR: deadlock detected | |
| 2023-08-05 22:34:56.394 UTC [7131] DETAIL: Process 7131 waits for ShareLock on transaction 37533; blocked by process 7228. | |
| Process 7228 waits for ShareLock on transaction 37532; blocked by process 7131. | |
| Process 7131: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T22:34:51.212388+00:00'::timestamptz WHERE dag_run.id = 1 | |
| Process 7228: UPDATE task_instance SET state='scheduled', updated_at='2023-08-05T22:34:55.396872+00:00'::timestamptz WHERE task_instance.dag_id = 'example_bash_operator' AND task_instance.run_id = 'backfill__2017-01-01T00:00:00+00:00' AND (task_instance.task_id, task_instance.map_index) IN (('run_after_loop', -1)) | |
| 2023-08-05 22:34:56.394 UTC [7131] HINT: See server log for query details. | |
| 2023-08-05 22:34:56.394 UTC [7131] CONTEXT: while updating tuple (1,37) in relation "dag_run" | |
| 2023-08-05 22:34:56.394 UTC [7131] STATEMENT: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T22:34:51.212388+00:00'::timestamptz WHERE dag_run.id = 1 | |
| 2023-08-05 22:35:12.628 UTC [7366] ERROR: deadlock detected | |
| 2023-08-05 22:35:12.628 UTC [7366] DETAIL: Process 7366 waits for ShareLock on transaction 40059; blocked by process 7365. | |
| Process 7365 waits for ShareLock on transaction 40060; blocked by process 7265. | |
| Process 7265 waits for ExclusiveLock on tuple (1,39) of relation 24793 of database 16384; blocked by process 7366. | |
| Process 7366: SELECT dag_run.state AS dag_run_state, dag_run.id AS dag_run_id, dag_run.dag_id AS dag_run_dag_id, dag_run.queued_at AS dag_run_queued_at, dag_run.execution_date AS dag_run_execution_date, dag_run.start_date AS dag_run_start_date, dag_run.end_date AS dag_run_end_date, dag_run.run_id AS dag_run_run_id, dag_run.creating_job_id AS dag_run_creating_job_id, dag_run.external_trigger AS dag_run_external_trigger, dag_run.run_type AS dag_run_run_type, dag_run.conf AS dag_run_conf, dag_run.data_interval_start AS dag_run_data_interval_start, dag_run.data_interval_end AS dag_run_data_interval_end, dag_run.last_scheduling_decision AS dag_run_last_scheduling_decision, dag_run.dag_hash AS dag_run_dag_hash, dag_run.log_template_id AS dag_run_log_template_id, dag_run.updated_at AS dag_run_updated_at | |
| FROM dag_run | |
| WHERE dag_run.dag_id = 'example_bash_operator' AND dag_run.run_id = 'backfill__2017-01-01T00:00:00+00:00' FOR UPDATE | |
| Process 7365: UPDATE task_instance SET state='scheduled', updated_at='2023-08-05T22:35:11.632033+00:00'::timestamptz WHERE task_instance.dag_id = 'example_bash_operator' AND task_instance.run_id = 'backfill__2017-01-01T00:00:00+00:00' AND (task_instance.task_id, task_instance.map_index) IN (('run_after_loop', -1)) | |
| Process 7265: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T22:35:07.480161+00:00'::timestamptz WHERE dag_run.id = 1 | |
| 2023-08-05 22:35:12.628 UTC [7366] HINT: See server log for query details. | |
| 2023-08-05 22:35:12.628 UTC [7366] CONTEXT: while locking tuple (1,39) in relation "dag_run" | |
| 2023-08-05 22:35:12.628 UTC [7366] STATEMENT: SELECT dag_run.state AS dag_run_state, dag_run.id AS dag_run_id, dag_run.dag_id AS dag_run_dag_id, dag_run.queued_at AS dag_run_queued_at, dag_run.execution_date AS dag_run_execution_date, dag_run.start_date AS dag_run_start_date, dag_run.end_date AS dag_run_end_date, dag_run.run_id AS dag_run_run_id, dag_run.creating_job_id AS dag_run_creating_job_id, dag_run.external_trigger AS dag_run_external_trigger, dag_run.run_type AS dag_run_run_type, dag_run.conf AS dag_run_conf, dag_run.data_interval_start AS dag_run_data_interval_start, dag_run.data_interval_end AS dag_run_data_interval_end, dag_run.last_scheduling_decision AS dag_run_last_scheduling_decision, dag_run.dag_hash AS dag_run_dag_hash, dag_run.log_template_id AS dag_run_log_template_id, dag_run.updated_at AS dag_run_updated_at | |
| FROM dag_run | |
| WHERE dag_run.dag_id = 'example_bash_operator' AND dag_run.run_id = 'backfill__2017-01-01T00:00:00+00:00' FOR UPDATE | |
| 2023-08-05 22:35:12.648 UTC [7365] ERROR: deadlock detected | |
| 2023-08-05 22:35:12.648 UTC [7365] DETAIL: Process 7365 waits for ShareLock on transaction 40060; blocked by process 7265. | |
| Process 7265 waits for ShareLock on transaction 40059; blocked by process 7365. | |
| Process 7365: UPDATE task_instance SET state='scheduled', updated_at='2023-08-05T22:35:11.632033+00:00'::timestamptz WHERE task_instance.dag_id = 'example_bash_operator' AND task_instance.run_id = 'backfill__2017-01-01T00:00:00+00:00' AND (task_instance.task_id, task_instance.map_index) IN (('run_after_loop', -1)) | |
| Process 7265: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T22:35:07.480161+00:00'::timestamptz WHERE dag_run.id = 1 | |
| 2023-08-05 22:35:12.648 UTC [7365] HINT: See server log for query details. | |
| 2023-08-05 22:35:12.648 UTC [7365] CONTEXT: while updating tuple (0,2) in relation "task_instance" | |
| 2023-08-05 22:35:12.648 UTC [7365] STATEMENT: UPDATE task_instance SET state='scheduled', updated_at='2023-08-05T22:35:11.632033+00:00'::timestamptz WHERE task_instance.dag_id = 'example_bash_operator' AND task_instance.run_id = 'backfill__2017-01-01T00:00:00+00:00' AND (task_instance.task_id, task_instance.map_index) IN (('run_after_loop', -1)) | |
| 2023-08-05 22:36:17.074 UTC [7782] ERROR: deadlock detected | |
| 2023-08-05 22:36:17.074 UTC [7782] DETAIL: Process 7782 waits for ShareLock on transaction 50158; blocked by process 7883. | |
| Process 7883 waits for ShareLock on transaction 50159; blocked by process 7782. | |
| Process 7782: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T22:36:11.906239+00:00'::timestamptz WHERE dag_run.id = 1 | |
| Process 7883: UPDATE task_instance SET state='scheduled', updated_at='2023-08-05T22:36:16.073758+00:00'::timestamptz WHERE task_instance.dag_id = 'example_bash_operator' AND task_instance.run_id = 'backfill__2017-01-01T00:00:00+00:00' AND (task_instance.task_id, task_instance.map_index) IN (('run_after_loop', -1)) | |
| 2023-08-05 22:36:17.074 UTC [7782] HINT: See server log for query details. | |
| 2023-08-05 22:36:17.074 UTC [7782] CONTEXT: while updating tuple (1,44) in relation "dag_run" | |
| 2023-08-05 22:36:17.074 UTC [7782] STATEMENT: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T22:36:11.906239+00:00'::timestamptz WHERE dag_run.id = 1 | |
| 2023-08-05 22:37:37.294 UTC [8439] ERROR: deadlock detected | |
| 2023-08-05 22:37:37.294 UTC [8439] DETAIL: Process 8439 waits for ShareLock on transaction 62711; blocked by process 8539. | |
| Process 8539 waits for ShareLock on transaction 62712; blocked by process 8439. | |
| Process 8439: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T22:37:36.293859+00:00'::timestamptz WHERE dag_run.id = 1 | |
| Process 8539: UPDATE task_instance SET state='scheduled', updated_at='2023-08-05T22:37:36.296801+00:00'::timestamptz WHERE task_instance.dag_id = 'example_bash_operator' AND task_instance.run_id = 'backfill__2017-01-01T00:00:00+00:00' AND (task_instance.task_id, task_instance.map_index) IN (('run_after_loop', -1)) | |
| 2023-08-05 22:37:37.294 UTC [8439] HINT: See server log for query details. | |
| 2023-08-05 22:37:37.294 UTC [8439] CONTEXT: while updating tuple (0,49) in relation "dag_run" | |
| 2023-08-05 22:37:37.294 UTC [8439] STATEMENT: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T22:37:36.293859+00:00'::timestamptz WHERE dag_run.id = 1 | |
| 2023-08-05 23:07:43.245 UTC [11337] ERROR: deadlock detected | |
| 2023-08-05 23:07:43.245 UTC [11337] DETAIL: Process 11337 waits for ShareLock on transaction 70152; blocked by process 11417. | |
| Process 11417 waits for ShareLock on transaction 70151; blocked by process 11337. | |
| Process 11337: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T23:07:39.040007+00:00'::timestamptz WHERE dag_run.id = 1 | |
| Process 11417: UPDATE task_instance SET start_date='2023-08-05T23:07:42.249709+00:00'::timestamptz, end_date='2023-08-05T23:07:42.249709+00:00'::timestamptz, duration=0.0, state='skipped', updated_at='2023-08-05T23:07:42.253235+00:00'::timestamptz WHERE task_instance.dag_id = 'example_bash_operator' AND task_instance.task_id = 'run_this_last' AND task_instance.run_id = 'backfill__2017-01-01T00:00:00+00:00' AND task_instance.map_index = -1 | |
| 2023-08-05 23:07:43.245 UTC [11337] HINT: See server log for query details. | |
| 2023-08-05 23:07:43.245 UTC [11337] CONTEXT: while updating tuple (0,214) in relation "dag_run" | |
| 2023-08-05 23:07:43.245 UTC [11337] STATEMENT: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T23:07:39.040007+00:00'::timestamptz WHERE dag_run.id = 1 | |
| 2023-08-05 23:28:19.255 UTC [12151] ERROR: deadlock detected | |
| 2023-08-05 23:28:19.255 UTC [12151] DETAIL: Process 12151 waits for ShareLock on transaction 74230; blocked by process 12217. | |
| Process 12217 waits for ShareLock on transaction 74232; blocked by process 12151. | |
| Process 12151: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T23:28:18.254981+00:00'::timestamptz WHERE dag_run.id = 1 | |
| Process 12217: UPDATE task_instance SET start_date='2023-08-05T23:28:18.253428+00:00'::timestamptz, end_date='2023-08-05T23:28:18.253428+00:00'::timestamptz, duration=0.0, state='skipped', updated_at='2023-08-05T23:28:18.256404+00:00'::timestamptz WHERE task_instance.dag_id = 'example_bash_operator' AND task_instance.task_id = 'run_this_last' AND task_instance.run_id = 'backfill__2017-01-01T00:00:00+00:00' AND task_instance.map_index = -1 | |
| 2023-08-05 23:28:19.255 UTC [12151] HINT: See server log for query details. | |
| 2023-08-05 23:28:19.255 UTC [12151] CONTEXT: while updating tuple (0,217) in relation "dag_run" | |
| 2023-08-05 23:28:19.255 UTC [12151] STATEMENT: UPDATE dag_run SET last_scheduling_decision=NULL, updated_at='2023-08-05T23:28:18.254981+00:00'::timestamptz WHERE dag_run.id = 1 | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment