Skip to content

Instantly share code, notes, and snippets.

@pclewis

pclewis/deadlock.log

Created April 28, 2019 13:35
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save pclewis/7e555fcc953fd32d3228b0af82f9b18c to your computer and use it in GitHub Desktop.
Save pclewis/7e555fcc953fd32d3228b0af82f9b18c to your computer and use it in GitHub Desktop.
Download ZIP
Raw
deadlock.log
Apr 28 02:00:03 nixos systemd[1]: Starting ZFS pools scrubbing...
Apr 28 02:04:00 nixos kernel: INFO: task txg_sync:1559 blocked for more than 120 seconds.
Apr 28 02:04:00 nixos kernel: Tainted: P O 4.19.36 #1-NixOS
Apr 28 02:04:00 nixos kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 28 02:04:00 nixos kernel: txg_sync D 0 1559 2 0x80000000
Apr 28 02:04:00 nixos kernel: Call Trace:
Apr 28 02:04:00 nixos kernel: ? __schedule+0x1d3/0x6f0
Apr 28 02:04:00 nixos kernel: ? _cond_resched+0x15/0x40
Apr 28 02:04:00 nixos kernel: schedule+0x28/0x80
Apr 28 02:04:00 nixos kernel: cv_wait_common+0x118/0x120 [spl]
Apr 28 02:04:00 nixos kernel: ? wait_woken+0x80/0x80
Apr 28 02:04:00 nixos kernel: spa_config_enter+0xed/0x100 [zfs]
Apr 28 02:04:00 nixos kernel: spa_sync+0x194/0xf80 [zfs]
Apr 28 02:04:00 nixos kernel: ? _cond_resched+0x15/0x40
Apr 28 02:04:00 nixos kernel: ? spa_txg_history_set.part.6+0xc1/0xf0 [zfs]
Apr 28 02:04:00 nixos kernel: ? spa_txg_history_init_io+0xde/0x110 [zfs]
Apr 28 02:04:00 nixos kernel: txg_sync_thread+0x288/0x450 [zfs]
Apr 28 02:04:00 nixos kernel: ? txg_wait_open+0xd0/0xd0 [zfs]
Apr 28 02:04:00 nixos kernel: ? __thread_exit+0x20/0x20 [spl]
Apr 28 02:04:00 nixos kernel: thread_generic_wrapper+0x6f/0x80 [spl]
Apr 28 02:04:00 nixos kernel: kthread+0x113/0x130
Apr 28 02:04:00 nixos kernel: ? kthread_create_worker_on_cpu+0x70/0x70
Apr 28 02:04:00 nixos kernel: ret_from_fork+0x35/0x40
Apr 28 02:04:00 nixos kernel: INFO: task zpool:8247 blocked for more than 120 seconds.
Apr 28 02:04:00 nixos kernel: Tainted: P O 4.19.36 #1-NixOS
Apr 28 02:04:00 nixos kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 28 02:04:00 nixos kernel: zpool D 0 8247 8245 0x00000000
Apr 28 02:04:00 nixos kernel: Call Trace:
Apr 28 02:04:00 nixos kernel: ? __schedule+0x1d3/0x6f0
Apr 28 02:04:00 nixos kernel: ? vdev_open_children+0x160/0x160 [zfs]
Apr 28 02:04:00 nixos kernel: schedule+0x28/0x80
Apr 28 02:04:00 nixos kernel: taskq_wait+0x7a/0xc0 [spl]
Apr 28 02:04:00 nixos kernel: ? wait_woken+0x80/0x80
Apr 28 02:04:00 nixos kernel: taskq_destroy+0x40/0x160 [spl]
Apr 28 02:04:00 nixos kernel: vdev_open_children+0x10a/0x160 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_root_open+0x37/0x130 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open+0xb1/0x720 [zfs]
Apr 28 02:04:00 nixos kernel: ? vdev_cache_purge+0x6e/0x80 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_reopen+0x2e/0xc0 [zfs]
Apr 28 02:04:00 nixos kernel: dsl_scan+0x36/0x120 [zfs]
Apr 28 02:04:00 nixos kernel: zfs_ioc_pool_scan+0x5a/0xd0 [zfs]
Apr 28 02:04:00 nixos kernel: zfsdev_ioctl+0x49d/0x8c0 [zfs]
Apr 28 02:04:00 nixos kernel: ? __handle_mm_fault+0xb67/0xec0
Apr 28 02:04:00 nixos kernel: do_vfs_ioctl+0xa4/0x630
Apr 28 02:04:00 nixos kernel: ? handle_mm_fault+0xdc/0x210
Apr 28 02:04:00 nixos kernel: ksys_ioctl+0x70/0x80
Apr 28 02:04:00 nixos kernel: __x64_sys_ioctl+0x16/0x20
Apr 28 02:04:00 nixos kernel: do_syscall_64+0x4e/0x100
Apr 28 02:04:00 nixos kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Apr 28 02:04:00 nixos kernel: RIP: 0033:0x7f34933efb57
Apr 28 02:04:00 nixos kernel: Code: Bad RIP value.
Apr 28 02:04:00 nixos kernel: RSP: 002b:00007fff6a9a3c08 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Apr 28 02:04:00 nixos kernel: RAX: ffffffffffffffda RBX: 00007fff6a9a4040 RCX: 00007f34933efb57
Apr 28 02:04:00 nixos kernel: RDX: 00007fff6a9a4040 RSI: 0000000000005a07 RDI: 0000000000000003
Apr 28 02:04:00 nixos kernel: RBP: 00007fff6a9a7620 R08: 0000000000000008 R09: 0000000000010011
Apr 28 02:04:00 nixos kernel: R10: 0000000000000001 R11: 0000000000000246 R12: 00000000020335d0
Apr 28 02:04:00 nixos kernel: R13: 0000000000000001 R14: 0000000000000000 R15: 0000000002031430
Apr 28 02:04:00 nixos kernel: INFO: task vdev_open:8248 blocked for more than 120 seconds.
Apr 28 02:04:00 nixos kernel: Tainted: P O 4.19.36 #1-NixOS
Apr 28 02:04:00 nixos kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 28 02:04:00 nixos kernel: vdev_open D 0 8248 2 0x80000000
Apr 28 02:04:00 nixos kernel: Call Trace:
Apr 28 02:04:00 nixos kernel: ? __schedule+0x1d3/0x6f0
Apr 28 02:04:00 nixos kernel: ? vdev_open_children+0x160/0x160 [zfs]
Apr 28 02:04:00 nixos kernel: schedule+0x28/0x80
Apr 28 02:04:00 nixos kernel: taskq_wait+0x7a/0xc0 [spl]
Apr 28 02:04:00 nixos kernel: ? wait_woken+0x80/0x80
Apr 28 02:04:00 nixos kernel: taskq_destroy+0x40/0x160 [spl]
Apr 28 02:04:00 nixos kernel: vdev_open_children+0x10a/0x160 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_raidz_open+0x40/0x170 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open+0xb1/0x720 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open_child+0x1e/0x40 [zfs]
Apr 28 02:04:00 nixos kernel: taskq_thread+0x295/0x4a0 [spl]
Apr 28 02:04:00 nixos kernel: ? __switch_to_asm+0x34/0x70
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: ? taskq_thread_should_stop+0x70/0x70 [spl]
Apr 28 02:04:00 nixos kernel: kthread+0x113/0x130
Apr 28 02:04:00 nixos kernel: ? kthread_create_worker_on_cpu+0x70/0x70
Apr 28 02:04:00 nixos kernel: ret_from_fork+0x35/0x40
Apr 28 02:04:00 nixos kernel: INFO: task vdev_open:8249 blocked for more than 120 seconds.
Apr 28 02:04:00 nixos kernel: Tainted: P O 4.19.36 #1-NixOS
Apr 28 02:04:00 nixos kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 28 02:04:00 nixos kernel: vdev_open D 0 8249 2 0x80000000
Apr 28 02:04:00 nixos kernel: Call Trace:
Apr 28 02:04:00 nixos kernel: ? __schedule+0x1d3/0x6f0
Apr 28 02:04:00 nixos kernel: ? vdev_open_children+0x160/0x160 [zfs]
Apr 28 02:04:00 nixos kernel: schedule+0x28/0x80
Apr 28 02:04:00 nixos kernel: taskq_wait+0x7a/0xc0 [spl]
Apr 28 02:04:00 nixos kernel: ? wait_woken+0x80/0x80
Apr 28 02:04:00 nixos kernel: taskq_destroy+0x40/0x160 [spl]
Apr 28 02:04:00 nixos kernel: vdev_open_children+0x10a/0x160 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_raidz_open+0x40/0x170 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open+0xb1/0x720 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open_child+0x1e/0x40 [zfs]
Apr 28 02:04:00 nixos kernel: taskq_thread+0x295/0x4a0 [spl]
Apr 28 02:04:00 nixos kernel: ? __switch_to_asm+0x34/0x70
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: ? taskq_thread_should_stop+0x70/0x70 [spl]
Apr 28 02:04:00 nixos kernel: kthread+0x113/0x130
Apr 28 02:04:00 nixos kernel: ? kthread_create_worker_on_cpu+0x70/0x70
Apr 28 02:04:00 nixos kernel: ret_from_fork+0x35/0x40
Apr 28 02:04:00 nixos kernel: INFO: task vdev_open:8250 blocked for more than 120 seconds.
Apr 28 02:04:00 nixos kernel: Tainted: P O 4.19.36 #1-NixOS
Apr 28 02:04:00 nixos kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 28 02:04:00 nixos kernel: vdev_open D 0 8250 2 0x80000000
Apr 28 02:04:00 nixos kernel: Call Trace:
Apr 28 02:04:00 nixos kernel: ? __schedule+0x1d3/0x6f0
Apr 28 02:04:00 nixos kernel: schedule+0x28/0x80
Apr 28 02:04:00 nixos kernel: schedule_timeout+0x1ee/0x380
Apr 28 02:04:00 nixos kernel: ? try_to_wake_up+0x55/0x440
Apr 28 02:04:00 nixos kernel: wait_for_completion+0xc0/0x150
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: call_usermodehelper_exec+0x13a/0x170
Apr 28 02:04:00 nixos kernel: vdev_elevator_switch+0x13b/0x1a0 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_disk_open+0x294/0x460 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open+0xb1/0x720 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open_child+0x1e/0x40 [zfs]
Apr 28 02:04:00 nixos kernel: taskq_thread+0x295/0x4a0 [spl]
Apr 28 02:04:00 nixos kernel: ? __switch_to_asm+0x34/0x70
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: ? taskq_thread_should_stop+0x70/0x70 [spl]
Apr 28 02:04:00 nixos kernel: kthread+0x113/0x130
Apr 28 02:04:00 nixos kernel: ? kthread_create_worker_on_cpu+0x70/0x70
Apr 28 02:04:00 nixos kernel: ret_from_fork+0x35/0x40
Apr 28 02:04:00 nixos kernel: INFO: task vdev_open:8251 blocked for more than 120 seconds.
Apr 28 02:04:00 nixos kernel: Tainted: P O 4.19.36 #1-NixOS
Apr 28 02:04:00 nixos kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 28 02:04:00 nixos kernel: vdev_open D 0 8251 2 0x80000000
Apr 28 02:04:00 nixos kernel: Call Trace:
Apr 28 02:04:00 nixos kernel: ? __schedule+0x1d3/0x6f0
Apr 28 02:04:00 nixos kernel: schedule+0x28/0x80
Apr 28 02:04:00 nixos kernel: schedule_timeout+0x1ee/0x380
Apr 28 02:04:00 nixos kernel: wait_for_completion+0xc0/0x150
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: call_usermodehelper_exec+0x13a/0x170
Apr 28 02:04:00 nixos kernel: vdev_elevator_switch+0x13b/0x1a0 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_disk_open+0x294/0x460 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open+0xb1/0x720 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open_child+0x1e/0x40 [zfs]
Apr 28 02:04:00 nixos kernel: taskq_thread+0x295/0x4a0 [spl]
Apr 28 02:04:00 nixos kernel: ? __switch_to_asm+0x34/0x70
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: ? taskq_thread_should_stop+0x70/0x70 [spl]
Apr 28 02:04:00 nixos kernel: kthread+0x113/0x130
Apr 28 02:04:00 nixos kernel: ? kthread_create_worker_on_cpu+0x70/0x70
Apr 28 02:04:00 nixos kernel: ret_from_fork+0x35/0x40
Apr 28 02:04:00 nixos kernel: INFO: task vdev_open:8252 blocked for more than 120 seconds.
Apr 28 02:04:00 nixos kernel: Tainted: P O 4.19.36 #1-NixOS
Apr 28 02:04:00 nixos kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 28 02:04:00 nixos kernel: vdev_open D 0 8252 2 0x80000000
Apr 28 02:04:00 nixos kernel: Call Trace:
Apr 28 02:04:00 nixos kernel: ? __schedule+0x1d3/0x6f0
Apr 28 02:04:00 nixos kernel: schedule+0x28/0x80
Apr 28 02:04:00 nixos kernel: schedule_timeout+0x1ee/0x380
Apr 28 02:04:00 nixos kernel: ? try_to_wake_up+0x55/0x440
Apr 28 02:04:00 nixos kernel: wait_for_completion+0xc0/0x150
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: call_usermodehelper_exec+0x13a/0x170
Apr 28 02:04:00 nixos kernel: vdev_elevator_switch+0x13b/0x1a0 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_disk_open+0x294/0x460 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open+0xb1/0x720 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open_child+0x1e/0x40 [zfs]
Apr 28 02:04:00 nixos kernel: taskq_thread+0x295/0x4a0 [spl]
Apr 28 02:04:00 nixos kernel: ? __switch_to_asm+0x34/0x70
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: ? taskq_thread_should_stop+0x70/0x70 [spl]
Apr 28 02:04:00 nixos kernel: kthread+0x113/0x130
Apr 28 02:04:00 nixos kernel: ? kthread_create_worker_on_cpu+0x70/0x70
Apr 28 02:04:00 nixos kernel: ret_from_fork+0x35/0x40
Apr 28 02:04:00 nixos kernel: INFO: task vdev_open:8253 blocked for more than 120 seconds.
Apr 28 02:04:00 nixos kernel: Tainted: P O 4.19.36 #1-NixOS
Apr 28 02:04:00 nixos kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 28 02:04:00 nixos kernel: vdev_open D 0 8253 2 0x80000000
Apr 28 02:04:00 nixos kernel: Call Trace:
Apr 28 02:04:00 nixos kernel: ? __schedule+0x1d3/0x6f0
Apr 28 02:04:00 nixos kernel: ? scsi_block_when_processing_errors+0x22/0x110 [scsi_mod]
Apr 28 02:04:00 nixos kernel: schedule+0x28/0x80
Apr 28 02:04:00 nixos kernel: schedule_timeout+0x1ee/0x380
Apr 28 02:04:00 nixos kernel: wait_for_completion+0xc0/0x150
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: call_usermodehelper_exec+0x13a/0x170
Apr 28 02:04:00 nixos kernel: vdev_elevator_switch+0x13b/0x1a0 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_disk_open+0x294/0x460 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open+0xb1/0x720 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open_child+0x1e/0x40 [zfs]
Apr 28 02:04:00 nixos kernel: taskq_thread+0x295/0x4a0 [spl]
Apr 28 02:04:00 nixos kernel: ? __switch_to_asm+0x34/0x70
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: ? taskq_thread_should_stop+0x70/0x70 [spl]
Apr 28 02:04:00 nixos kernel: kthread+0x113/0x130
Apr 28 02:04:00 nixos kernel: ? kthread_create_worker_on_cpu+0x70/0x70
Apr 28 02:04:00 nixos kernel: ret_from_fork+0x35/0x40
Apr 28 02:04:00 nixos kernel: INFO: task vdev_open:8254 blocked for more than 120 seconds.
Apr 28 02:04:00 nixos kernel: Tainted: P O 4.19.36 #1-NixOS
Apr 28 02:04:00 nixos kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 28 02:04:00 nixos kernel: vdev_open D 0 8254 2 0x80000000
Apr 28 02:04:00 nixos kernel: Call Trace:
Apr 28 02:04:00 nixos kernel: ? __schedule+0x1d3/0x6f0
Apr 28 02:04:00 nixos kernel: ? scsi_block_when_processing_errors+0x22/0x110 [scsi_mod]
Apr 28 02:04:00 nixos kernel: schedule+0x28/0x80
Apr 28 02:04:00 nixos kernel: schedule_timeout+0x1ee/0x380
Apr 28 02:04:00 nixos kernel: ? try_to_wake_up+0x55/0x440
Apr 28 02:04:00 nixos kernel: wait_for_completion+0xc0/0x150
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: call_usermodehelper_exec+0x13a/0x170
Apr 28 02:04:00 nixos kernel: vdev_elevator_switch+0x13b/0x1a0 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_disk_open+0x294/0x460 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open+0xb1/0x720 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open_child+0x1e/0x40 [zfs]
Apr 28 02:04:00 nixos kernel: taskq_thread+0x295/0x4a0 [spl]
Apr 28 02:04:00 nixos kernel: ? __switch_to_asm+0x34/0x70
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: ? taskq_thread_should_stop+0x70/0x70 [spl]
Apr 28 02:04:00 nixos kernel: kthread+0x113/0x130
Apr 28 02:04:00 nixos kernel: ? kthread_create_worker_on_cpu+0x70/0x70
Apr 28 02:04:00 nixos kernel: ret_from_fork+0x35/0x40
Apr 28 02:04:00 nixos kernel: INFO: task vdev_open:8255 blocked for more than 120 seconds.
Apr 28 02:04:00 nixos kernel: Tainted: P O 4.19.36 #1-NixOS
Apr 28 02:04:00 nixos kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 28 02:04:00 nixos kernel: vdev_open D 0 8255 2 0x80000000
Apr 28 02:04:00 nixos kernel: Call Trace:
Apr 28 02:04:00 nixos kernel: ? __schedule+0x1d3/0x6f0
Apr 28 02:04:00 nixos kernel: ? scsi_block_when_processing_errors+0x22/0x110 [scsi_mod]
Apr 28 02:04:00 nixos kernel: schedule+0x28/0x80
Apr 28 02:04:00 nixos kernel: schedule_timeout+0x1ee/0x380
Apr 28 02:04:00 nixos kernel: wait_for_completion+0xc0/0x150
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: call_usermodehelper_exec+0x13a/0x170
Apr 28 02:04:00 nixos kernel: vdev_elevator_switch+0x13b/0x1a0 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_disk_open+0x294/0x460 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open+0xb1/0x720 [zfs]
Apr 28 02:04:00 nixos kernel: vdev_open_child+0x1e/0x40 [zfs]
Apr 28 02:04:00 nixos kernel: taskq_thread+0x295/0x4a0 [spl]
Apr 28 02:04:00 nixos kernel: ? __switch_to_asm+0x34/0x70
Apr 28 02:04:00 nixos kernel: ? wake_up_q+0x70/0x70
Apr 28 02:04:00 nixos kernel: ? taskq_thread_should_stop+0x70/0x70 [spl]
Apr 28 02:04:00 nixos kernel: kthread+0x113/0x130
Apr 28 02:04:00 nixos kernel: ? kthread_create_worker_on_cpu+0x70/0x70
Apr 28 02:04:00 nixos kernel: ret_from_fork+0x35/0x40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment