Skip to content

Instantly share code, notes, and snippets.

@uroni

uroni/gist:5709555

Created June 4, 2013 20:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save uroni/5709555 to your computer and use it in GitHub Desktop.
Save uroni/5709555 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
Jun 4 22:17:02 atomsrv /USR/SBIN/CRON[3308]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jun 4 23:00:20 atomsrv kernel: [ 5161.108064] INFO: task kswapd0:33 blocked for more than 120 seconds.
Jun 4 23:00:20 atomsrv kernel: [ 5161.108087] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 4 23:00:20 atomsrv kernel: [ 5161.108106] kswapd0 D ffff88007f513780 0 33 2 0x00000000
Jun 4 23:00:20 atomsrv kernel: [ 5161.108118] ffff88007c1dc930 0000000000000046 0000000000000000 ffff88007c043610
Jun 4 23:00:20 atomsrv kernel: [ 5161.108130] 0000000000013780 ffff88007988dfd8 ffff88007988dfd8 ffff88007c1dc930
Jun 4 23:00:20 atomsrv kernel: [ 5161.108141] 0000000000000202 000000018134d9a4 ffff880077d08b28 ffff880077d08b20
Jun 4 23:00:20 atomsrv kernel: [ 5161.108152] Call Trace:
Jun 4 23:00:20 atomsrv kernel: [ 5161.108182] [<ffffffffa0326643>] ? cv_wait_common+0xdc/0x171 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.108194] [<ffffffff81037d81>] ? __wake_up+0x35/0x46
Jun 4 23:00:20 atomsrv kernel: [ 5161.108204] [<ffffffff8105f973>] ? add_wait_queue+0x3c/0x3c
Jun 4 23:00:20 atomsrv kernel: [ 5161.108285] [<ffffffffa03de3e5>] ? txg_wait_open+0xbe/0xf4 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.108294] [<ffffffff81036428>] ? should_resched+0x5/0x23
Jun 4 23:00:20 atomsrv kernel: [ 5161.108356] [<ffffffffa03aeaf6>] ? dmu_tx_assign+0xf9/0x345 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.108421] [<ffffffffa03b85c7>] ? dsl_dataset_block_freeable+0x31/0x3f [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.108508] [<ffffffffa0405ae0>] ? zfs_inactive+0xa7/0x13b [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.108518] [<ffffffff8110c748>] ? evict+0x96/0x148
Jun 4 23:00:20 atomsrv kernel: [ 5161.108526] [<ffffffff8110c9b2>] ? dispose_list+0x27/0x31
Jun 4 23:00:20 atomsrv kernel: [ 5161.108534] [<ffffffff8110d4f2>] ? prune_icache_sb+0x24f/0x25e
Jun 4 23:00:20 atomsrv kernel: [ 5161.108544] [<ffffffff810fc222>] ? prune_super+0xd5/0x13f
Jun 4 23:00:20 atomsrv kernel: [ 5161.108553] [<ffffffff810c1ad9>] ? shrink_slab+0x18f/0x24d
Jun 4 23:00:20 atomsrv kernel: [ 5161.108562] [<ffffffff810c44a7>] ? balance_pgdat+0x283/0x4b7
Jun 4 23:00:20 atomsrv kernel: [ 5161.108571] [<ffffffff810c49ae>] ? kswapd+0x2d3/0x303
Jun 4 23:00:20 atomsrv kernel: [ 5161.108580] [<ffffffff8105f973>] ? add_wait_queue+0x3c/0x3c
Jun 4 23:00:20 atomsrv kernel: [ 5161.108588] [<ffffffff810c46db>] ? balance_pgdat+0x4b7/0x4b7
Jun 4 23:00:20 atomsrv kernel: [ 5161.108597] [<ffffffff8105f321>] ? kthread+0x76/0x7e
Jun 4 23:00:20 atomsrv kernel: [ 5161.108606] [<ffffffff81354ab4>] ? kernel_thread_helper+0x4/0x10
Jun 4 23:00:20 atomsrv kernel: [ 5161.108616] [<ffffffff8105f2ab>] ? kthread_worker_fn+0x139/0x139
Jun 4 23:00:20 atomsrv kernel: [ 5161.108624] [<ffffffff81354ab0>] ? gs_change+0x13/0x13
Jun 4 23:00:20 atomsrv kernel: [ 5161.108640] INFO: task spl_kmem_cache/:724 blocked for more than 120 seconds.
Jun 4 23:00:20 atomsrv kernel: [ 5161.108656] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 4 23:00:20 atomsrv kernel: [ 5161.108674] spl_kmem_cache/ D ffff88007f413780 0 724 2 0x00000000
Jun 4 23:00:20 atomsrv kernel: [ 5161.108683] ffff88007aa8c830 0000000000000046 ffff8800000007fe ffff880077eeb1a0
Jun 4 23:00:20 atomsrv kernel: [ 5161.108694] 0000000000013780 ffff88007b12dfd8 ffff88007b12dfd8 ffff88007aa8c830
Jun 4 23:00:20 atomsrv kernel: [ 5161.108705] 0000000000000202 ffffffff8134d9a4 ffff880077d08b28 ffff880077d08b20
Jun 4 23:00:20 atomsrv kernel: [ 5161.108715] Call Trace:
Jun 4 23:00:20 atomsrv kernel: [ 5161.108725] [<ffffffff8134d9a4>] ? _raw_spin_lock_irqsave+0x9/0x25
Jun 4 23:00:20 atomsrv kernel: [ 5161.108743] [<ffffffffa0326643>] ? cv_wait_common+0xdc/0x171 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.108752] [<ffffffff81037d81>] ? __wake_up+0x35/0x46
Jun 4 23:00:20 atomsrv kernel: [ 5161.108760] [<ffffffff8105f973>] ? add_wait_queue+0x3c/0x3c
Jun 4 23:00:20 atomsrv kernel: [ 5161.108838] [<ffffffffa03de3e5>] ? txg_wait_open+0xbe/0xf4 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.108846] [<ffffffff81036428>] ? should_resched+0x5/0x23
Jun 4 23:00:20 atomsrv kernel: [ 5161.108908] [<ffffffffa03aeaf6>] ? dmu_tx_assign+0xf9/0x345 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.108973] [<ffffffffa03b85c7>] ? dsl_dataset_block_freeable+0x31/0x3f [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109059] [<ffffffffa0405ae0>] ? zfs_inactive+0xa7/0x13b [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109068] [<ffffffff8110c748>] ? evict+0x96/0x148
Jun 4 23:00:20 atomsrv kernel: [ 5161.109076] [<ffffffff8110c9b2>] ? dispose_list+0x27/0x31
Jun 4 23:00:20 atomsrv kernel: [ 5161.109084] [<ffffffff8110d4f2>] ? prune_icache_sb+0x24f/0x25e
Jun 4 23:00:20 atomsrv kernel: [ 5161.109093] [<ffffffff810fc222>] ? prune_super+0xd5/0x13f
Jun 4 23:00:20 atomsrv kernel: [ 5161.109101] [<ffffffff810c1ad9>] ? shrink_slab+0x18f/0x24d
Jun 4 23:00:20 atomsrv kernel: [ 5161.109110] [<ffffffff810c3d14>] ? do_try_to_free_pages+0x2a7/0x41c
Jun 4 23:00:20 atomsrv kernel: [ 5161.109119] [<ffffffff810c40f4>] ? try_to_free_pages+0xa9/0xe9
Jun 4 23:00:20 atomsrv kernel: [ 5161.109129] [<ffffffff810bb0d4>] ? __alloc_pages_nodemask+0x4ed/0x7aa
Jun 4 23:00:20 atomsrv kernel: [ 5161.109140] [<ffffffff810e4706>] ? alloc_pages_current+0xc7/0xe4
Jun 4 23:00:20 atomsrv kernel: [ 5161.109148] [<ffffffff810b7e02>] ? __get_free_pages+0x6/0x42
Jun 4 23:00:20 atomsrv kernel: [ 5161.109157] [<ffffffff810cef78>] ? __pte_alloc_kernel+0x12/0xa3
Jun 4 23:00:20 atomsrv kernel: [ 5161.109166] [<ffffffff810da40e>] ? vmap_page_range_noflush+0x166/0x274
Jun 4 23:00:20 atomsrv kernel: [ 5161.109175] [<ffffffff810da541>] ? map_vm_area+0x25/0x36
Jun 4 23:00:20 atomsrv kernel: [ 5161.109183] [<ffffffff810dbda1>] ? __vmalloc_node_range+0x190/0x1fa
Jun 4 23:00:20 atomsrv kernel: [ 5161.109199] [<ffffffffa031e51c>] ? kv_alloc.isra.8+0x39/0x3b [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109208] [<ffffffff810dbe37>] ? __vmalloc_node+0x2c/0x31
Jun 4 23:00:20 atomsrv kernel: [ 5161.109224] [<ffffffffa031e51c>] ? kv_alloc.isra.8+0x39/0x3b [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109239] [<ffffffffa031e51c>] ? kv_alloc.isra.8+0x39/0x3b [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109254] [<ffffffffa031e552>] ? spl_cache_grow_work+0x34/0x388 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109263] [<ffffffff8134c634>] ? __schedule+0x5ac/0x5c3
Jun 4 23:00:20 atomsrv kernel: [ 5161.109272] [<ffffffff81070c4d>] ? arch_local_irq_save+0x11/0x17
Jun 4 23:00:20 atomsrv kernel: [ 5161.109289] [<ffffffffa03217e3>] ? taskq_thread+0x290/0x3e6 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109297] [<ffffffff8103f44e>] ? try_to_wake_up+0x197/0x197
Jun 4 23:00:20 atomsrv kernel: [ 5161.109314] [<ffffffffa0321553>] ? task_done+0x110/0x110 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109323] [<ffffffff8105f321>] ? kthread+0x76/0x7e
Jun 4 23:00:20 atomsrv kernel: [ 5161.109331] [<ffffffff81354ab4>] ? kernel_thread_helper+0x4/0x10
Jun 4 23:00:20 atomsrv kernel: [ 5161.109340] [<ffffffff8105f2ab>] ? kthread_worker_fn+0x139/0x139
Jun 4 23:00:20 atomsrv kernel: [ 5161.109348] [<ffffffff81354ab0>] ? gs_change+0x13/0x13
Jun 4 23:00:20 atomsrv kernel: [ 5161.109378] INFO: task urbackup_srv:3226 blocked for more than 120 seconds.
Jun 4 23:00:20 atomsrv kernel: [ 5161.109393] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 4 23:00:20 atomsrv kernel: [ 5161.109411] urbackup_srv D ffff88007f413780 0 3226 1 0x00000000
Jun 4 23:00:20 atomsrv kernel: [ 5161.109420] ffff8800707d5710 0000000000000082 0000000000000000 ffffffff8160d020
Jun 4 23:00:20 atomsrv kernel: [ 5161.109431] 0000000000013780 ffff880070a1dfd8 ffff880070a1dfd8 ffff8800707d5710
Jun 4 23:00:20 atomsrv kernel: [ 5161.109441] 0000000000000202 000000018134d9a4 ffff880077d08b28 ffff880077d08b20
Jun 4 23:00:20 atomsrv kernel: [ 5161.109452] Call Trace:
Jun 4 23:00:20 atomsrv kernel: [ 5161.109470] [<ffffffffa0326643>] ? cv_wait_common+0xdc/0x171 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109479] [<ffffffff81037d81>] ? __wake_up+0x35/0x46
Jun 4 23:00:20 atomsrv kernel: [ 5161.109486] [<ffffffff81036428>] ? should_resched+0x5/0x23
Jun 4 23:00:20 atomsrv kernel: [ 5161.109495] [<ffffffff8105f973>] ? add_wait_queue+0x3c/0x3c
Jun 4 23:00:20 atomsrv kernel: [ 5161.109572] [<ffffffffa03de3e5>] ? txg_wait_open+0xbe/0xf4 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109659] [<ffffffffa040b3c6>] ? zfs_write+0x4e5/0x965 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109668] [<ffffffff81036428>] ? should_resched+0x5/0x23
Jun 4 23:00:20 atomsrv kernel: [ 5161.109676] [<ffffffff8134c68f>] ? _cond_resched+0x7/0x1c
Jun 4 23:00:20 atomsrv kernel: [ 5161.109686] [<ffffffff81108350>] ? core_sys_select+0x1d7/0x21a
Jun 4 23:00:20 atomsrv kernel: [ 5161.109695] [<ffffffff810fb0f3>] ? fget_light+0x5f/0x73
Jun 4 23:00:20 atomsrv kernel: [ 5161.109703] [<ffffffff810fb0f3>] ? fget_light+0x5f/0x73
Jun 4 23:00:20 atomsrv kernel: [ 5161.109792] [<ffffffffa04193d3>] ? zpl_write_common+0x51/0x69 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109881] [<ffffffffa0419444>] ? zpl_write+0x59/0x7c [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.109889] [<ffffffff810f9e6d>] ? vfs_write+0xa2/0xe9
Jun 4 23:00:20 atomsrv kernel: [ 5161.109897] [<ffffffff810fa04a>] ? sys_write+0x45/0x6b
Jun 4 23:00:20 atomsrv kernel: [ 5161.109907] [<ffffffff81352952>] ? system_call_fastpath+0x16/0x1b
Jun 4 23:00:20 atomsrv kernel: [ 5161.109915] INFO: task urbackup_srv:3233 blocked for more than 120 seconds.
Jun 4 23:00:20 atomsrv kernel: [ 5161.109931] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 4 23:00:20 atomsrv kernel: [ 5161.109948] urbackup_srv D ffff88007f413780 0 3233 1 0x00000000
Jun 4 23:00:20 atomsrv kernel: [ 5161.109958] ffff8800377683c0 0000000000000082 0000000000000003 ffff880077eeb1a0
Jun 4 23:00:20 atomsrv kernel: [ 5161.109968] 0000000000013780 ffff88006d915fd8 ffff88006d915fd8 ffff8800377683c0
Jun 4 23:00:20 atomsrv kernel: [ 5161.109979] 0000000000000202 ffffffff8134d9a4 ffff880077d08b28 ffff880077d08b20
Jun 4 23:00:20 atomsrv kernel: [ 5161.109989] Call Trace:
Jun 4 23:00:20 atomsrv kernel: [ 5161.109997] [<ffffffff8134d9a4>] ? _raw_spin_lock_irqsave+0x9/0x25
Jun 4 23:00:20 atomsrv kernel: [ 5161.110016] [<ffffffffa0326643>] ? cv_wait_common+0xdc/0x171 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.110024] [<ffffffff81037d81>] ? __wake_up+0x35/0x46
Jun 4 23:00:20 atomsrv kernel: [ 5161.110031] [<ffffffff81036428>] ? should_resched+0x5/0x23
Jun 4 23:00:20 atomsrv kernel: [ 5161.110040] [<ffffffff8105f973>] ? add_wait_queue+0x3c/0x3c
Jun 4 23:00:20 atomsrv kernel: [ 5161.110117] [<ffffffffa03de3e5>] ? txg_wait_open+0xbe/0xf4 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.110204] [<ffffffffa040607b>] ? zfs_link+0x2b6/0x367 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.110213] [<ffffffff81104563>] ? kern_path_create+0x70/0xf3
Jun 4 23:00:20 atomsrv kernel: [ 5161.110302] [<ffffffffa0419a15>] ? zpl_link+0x65/0x96 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.110311] [<ffffffff81103b89>] ? vfs_link+0xbf/0x10a
Jun 4 23:00:20 atomsrv kernel: [ 5161.110320] [<ffffffff81105a8d>] ? sys_linkat+0x17b/0x1c7
Jun 4 23:00:20 atomsrv kernel: [ 5161.110328] [<ffffffff811065f6>] ? sys_fcntl+0x491/0x4a1
Jun 4 23:00:20 atomsrv kernel: [ 5161.110337] [<ffffffff81352952>] ? system_call_fastpath+0x16/0x1b
Jun 4 23:00:20 atomsrv kernel: [ 5161.110347] INFO: task urbackup_srv:3250 blocked for more than 120 seconds.
Jun 4 23:00:20 atomsrv kernel: [ 5161.110362] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 4 23:00:20 atomsrv kernel: [ 5161.110379] urbackup_srv D ffff88007f493780 0 3250 1 0x00000000
Jun 4 23:00:20 atomsrv kernel: [ 5161.110389] ffff8800706623c0 0000000000000082 0000000000000000 ffff88007c0020c0
Jun 4 23:00:20 atomsrv kernel: [ 5161.110399] 0000000000013780 ffff88005ef47fd8 ffff88005ef47fd8 ffff8800706623c0
Jun 4 23:00:20 atomsrv kernel: [ 5161.110410] 0000000000000202 000000018134d9a4 ffff880077d08b28 ffff880077d08b20
Jun 4 23:00:20 atomsrv kernel: [ 5161.110420] Call Trace:
Jun 4 23:00:20 atomsrv kernel: [ 5161.110439] [<ffffffffa0326643>] ? cv_wait_common+0xdc/0x171 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.110447] [<ffffffff81037d81>] ? __wake_up+0x35/0x46
Jun 4 23:00:20 atomsrv kernel: [ 5161.110454] [<ffffffff81036428>] ? should_resched+0x5/0x23
Jun 4 23:00:20 atomsrv kernel: [ 5161.110463] [<ffffffff8105f973>] ? add_wait_queue+0x3c/0x3c
Jun 4 23:00:20 atomsrv kernel: [ 5161.110541] [<ffffffffa03de3e5>] ? txg_wait_open+0xbe/0xf4 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.110627] [<ffffffffa040b3c6>] ? zfs_write+0x4e5/0x965 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.110637] [<ffffffff8106dc26>] ? futex_wait+0xeb/0x236
Jun 4 23:00:20 atomsrv kernel: [ 5161.110647] [<ffffffff81013a17>] ? native_sched_clock+0x27/0x2f
Jun 4 23:00:20 atomsrv kernel: [ 5161.110655] [<ffffffff81013a24>] ? sched_clock+0x5/0x8
Jun 4 23:00:20 atomsrv kernel: [ 5161.110664] [<ffffffff81063adc>] ? sched_clock_local+0xd/0x6f
Jun 4 23:00:20 atomsrv kernel: [ 5161.110754] [<ffffffffa04193d3>] ? zpl_write_common+0x51/0x69 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.110842] [<ffffffffa0419444>] ? zpl_write+0x59/0x7c [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.110851] [<ffffffff810f9e6d>] ? vfs_write+0xa2/0xe9
Jun 4 23:00:20 atomsrv kernel: [ 5161.110859] [<ffffffff810fa04a>] ? sys_write+0x45/0x6b
Jun 4 23:00:20 atomsrv kernel: [ 5161.110868] [<ffffffff81352952>] ? system_call_fastpath+0x16/0x1b
Jun 4 23:00:20 atomsrv kernel: [ 5161.110881] INFO: task z_wr_iss/0:2416 blocked for more than 120 seconds.
Jun 4 23:00:20 atomsrv kernel: [ 5161.110896] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 4 23:00:20 atomsrv kernel: [ 5161.110913] z_wr_iss/0 D ffff88007f413780 0 2416 2 0x00000000
Jun 4 23:00:20 atomsrv kernel: [ 5161.110923] ffff88007688a180 0000000000000046 ffff88007688a180 ffff8800768b7750
Jun 4 23:00:20 atomsrv kernel: [ 5161.110934] 0000000000013780 ffff880076893fd8 ffff880076893fd8 ffff88007688a180
Jun 4 23:00:20 atomsrv kernel: [ 5161.110944] ffff88007688a180 0000003000000000 ffffffff81036428 ffff880077d6b718
Jun 4 23:00:20 atomsrv kernel: [ 5161.110955] Call Trace:
Jun 4 23:00:20 atomsrv kernel: [ 5161.110963] [<ffffffff81036428>] ? should_resched+0x5/0x23
Jun 4 23:00:20 atomsrv kernel: [ 5161.110971] [<ffffffff8134cf8f>] ? __mutex_lock_common.isra.5+0xff/0x164
Jun 4 23:00:20 atomsrv kernel: [ 5161.110980] [<ffffffff8134ce7d>] ? mutex_lock+0x1a/0x2d
Jun 4 23:00:20 atomsrv kernel: [ 5161.111058] [<ffffffffa03e7cd8>] ? vdev_queue_io+0x5b/0xd7 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.111146] [<ffffffffa04153c7>] ? zio_vdev_io_start+0x1cf/0x220 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.111234] [<ffffffffa0415637>] ? zio_nowait+0xe8/0x10a [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.111312] [<ffffffffa03e7280>] ? vdev_mirror_io_start+0x2ec/0x305 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.111390] [<ffffffffa03e6c58>] ? vdev_config_sync+0x13f/0x13f [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.111478] [<ffffffffa041683b>] ? zio_ready+0x249/0x2e1 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.111566] [<ffffffffa0413594>] ? zio_execute+0xc5/0xe7 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.111583] [<ffffffffa03217e3>] ? taskq_thread+0x290/0x3e6 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.111591] [<ffffffff8103f44e>] ? try_to_wake_up+0x197/0x197
Jun 4 23:00:20 atomsrv kernel: [ 5161.111608] [<ffffffffa0321553>] ? task_done+0x110/0x110 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.111616] [<ffffffff8105f321>] ? kthread+0x76/0x7e
Jun 4 23:00:20 atomsrv kernel: [ 5161.111625] [<ffffffff81354ab4>] ? kernel_thread_helper+0x4/0x10
Jun 4 23:00:20 atomsrv kernel: [ 5161.111634] [<ffffffff8105f2ab>] ? kthread_worker_fn+0x139/0x139
Jun 4 23:00:20 atomsrv kernel: [ 5161.111642] [<ffffffff81354ab0>] ? gs_change+0x13/0x13
Jun 4 23:00:20 atomsrv kernel: [ 5161.111649] INFO: task z_wr_iss/1:2417 blocked for more than 120 seconds.
Jun 4 23:00:20 atomsrv kernel: [ 5161.111664] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 4 23:00:20 atomsrv kernel: [ 5161.111692] z_wr_iss/1 D ffff88007f493780 0 2417 2 0x00000000
Jun 4 23:00:20 atomsrv kernel: [ 5161.111701] ffff880076895690 0000000000000046 ffff880000000000 ffff88007c0020c0
Jun 4 23:00:20 atomsrv kernel: [ 5161.111712] 0000000000013780 ffff880076897fd8 ffff880076897fd8 ffff880076895690
Jun 4 23:00:20 atomsrv kernel: [ 5161.111723] 0000000000008000 0000000181041e20 ffffffff81036428 ffff880077d6b718
Jun 4 23:00:20 atomsrv kernel: [ 5161.111733] Call Trace:
Jun 4 23:00:20 atomsrv kernel: [ 5161.111741] [<ffffffff81036428>] ? should_resched+0x5/0x23
Jun 4 23:00:20 atomsrv kernel: [ 5161.111749] [<ffffffff8134cf8f>] ? __mutex_lock_common.isra.5+0xff/0x164
Jun 4 23:00:20 atomsrv kernel: [ 5161.111758] [<ffffffff8134ce7d>] ? mutex_lock+0x1a/0x2d
Jun 4 23:00:20 atomsrv kernel: [ 5161.111836] [<ffffffffa03e7cd8>] ? vdev_queue_io+0x5b/0xd7 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.111924] [<ffffffffa04153c7>] ? zio_vdev_io_start+0x1cf/0x220 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.112012] [<ffffffffa0415637>] ? zio_nowait+0xe8/0x10a [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.112142] [<ffffffffa03e7280>] ? vdev_mirror_io_start+0x2ec/0x305 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.112234] [<ffffffffa03e6c58>] ? vdev_config_sync+0x13f/0x13f [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.112334] [<ffffffffa0413594>] ? zio_execute+0xc5/0xe7 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.112364] [<ffffffffa03217e3>] ? taskq_thread+0x290/0x3e6 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.112384] [<ffffffff8103f44e>] ? try_to_wake_up+0x197/0x197
Jun 4 23:00:20 atomsrv kernel: [ 5161.112411] [<ffffffffa0321553>] ? task_done+0x110/0x110 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.112432] [<ffffffff8105f321>] ? kthread+0x76/0x7e
Jun 4 23:00:20 atomsrv kernel: [ 5161.112451] [<ffffffff81354ab4>] ? kernel_thread_helper+0x4/0x10
Jun 4 23:00:20 atomsrv kernel: [ 5161.112472] [<ffffffff8105f2ab>] ? kthread_worker_fn+0x139/0x139
Jun 4 23:00:20 atomsrv kernel: [ 5161.112492] [<ffffffff81354ab0>] ? gs_change+0x13/0x13
Jun 4 23:00:20 atomsrv kernel: [ 5161.112508] INFO: task z_wr_iss/2:2418 blocked for more than 120 seconds.
Jun 4 23:00:20 atomsrv kernel: [ 5161.112546] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 4 23:00:20 atomsrv kernel: [ 5161.112596] z_wr_iss/2 D ffff88007f513780 0 2418 2 0x00000000
Jun 4 23:00:20 atomsrv kernel: [ 5161.112621] ffff880076894fa0 0000000000000046 ffff880000000000 ffff88007c043610
Jun 4 23:00:20 atomsrv kernel: [ 5161.112652] 0000000000013780 ffff88007689bfd8 ffff88007689bfd8 ffff880076894fa0
Jun 4 23:00:20 atomsrv kernel: [ 5161.112684] ffff880076894fa0 0000000100000000 ffffffff81036428 ffff880077d6b718
Jun 4 23:00:20 atomsrv kernel: [ 5161.112714] Call Trace:
Jun 4 23:00:20 atomsrv kernel: [ 5161.112730] [<ffffffff81036428>] ? should_resched+0x5/0x23
Jun 4 23:00:20 atomsrv kernel: [ 5161.112750] [<ffffffff8134cf8f>] ? __mutex_lock_common.isra.5+0xff/0x164
Jun 4 23:00:20 atomsrv kernel: [ 5161.112771] [<ffffffff8134ce7d>] ? mutex_lock+0x1a/0x2d
Jun 4 23:00:20 atomsrv kernel: [ 5161.112860] [<ffffffffa03e7cd8>] ? vdev_queue_io+0x5b/0xd7 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.112959] [<ffffffffa04153c7>] ? zio_vdev_io_start+0x1cf/0x220 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.113060] [<ffffffffa0415637>] ? zio_nowait+0xe8/0x10a [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.113150] [<ffffffffa03e7280>] ? vdev_mirror_io_start+0x2ec/0x305 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.113241] [<ffffffffa03e6c58>] ? vdev_config_sync+0x13f/0x13f [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.113341] [<ffffffffa0413594>] ? zio_execute+0xc5/0xe7 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.113369] [<ffffffffa03217e3>] ? taskq_thread+0x290/0x3e6 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.113390] [<ffffffff8103f44e>] ? try_to_wake_up+0x197/0x197
Jun 4 23:00:20 atomsrv kernel: [ 5161.113417] [<ffffffffa0321553>] ? task_done+0x110/0x110 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.113437] [<ffffffff8105f321>] ? kthread+0x76/0x7e
Jun 4 23:00:20 atomsrv kernel: [ 5161.113457] [<ffffffff81354ab4>] ? kernel_thread_helper+0x4/0x10
Jun 4 23:00:20 atomsrv kernel: [ 5161.113478] [<ffffffff8105f2ab>] ? kthread_worker_fn+0x139/0x139
Jun 4 23:00:20 atomsrv kernel: [ 5161.113497] [<ffffffff81354ab0>] ? gs_change+0x13/0x13
Jun 4 23:00:20 atomsrv kernel: [ 5161.113515] INFO: task z_wr_int/0:2425 blocked for more than 120 seconds.
Jun 4 23:00:20 atomsrv kernel: [ 5161.113552] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 4 23:00:20 atomsrv kernel: [ 5161.113602] z_wr_int/0 D ffff88007f413780 0 2425 2 0x00000000
Jun 4 23:00:20 atomsrv kernel: [ 5161.113626] ffff8800768ad710 0000000000000046 ffff8800768ad710 ffff880079c9cf20
Jun 4 23:00:20 atomsrv kernel: [ 5161.113659] 0000000000013780 ffff8800768affd8 ffff8800768affd8 ffff8800768ad710
Jun 4 23:00:20 atomsrv kernel: [ 5161.113689] ffff8800768ad710 ffff880079b1f040 0000000000000000 ffff880077d6b718
Jun 4 23:00:20 atomsrv kernel: [ 5161.113720] Call Trace:
Jun 4 23:00:20 atomsrv kernel: [ 5161.113736] [<ffffffff8134cf8f>] ? __mutex_lock_common.isra.5+0xff/0x164
Jun 4 23:00:20 atomsrv kernel: [ 5161.113758] [<ffffffff8134ce7d>] ? mutex_lock+0x1a/0x2d
Jun 4 23:00:20 atomsrv kernel: [ 5161.113848] [<ffffffffa03e7d7c>] ? vdev_queue_io_done+0x28/0xa0 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.113947] [<ffffffffa0412f8d>] ? zio_vdev_io_done+0x65/0x13f [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.114047] [<ffffffffa0413594>] ? zio_execute+0xc5/0xe7 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.114076] [<ffffffffa03217e3>] ? taskq_thread+0x290/0x3e6 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.114096] [<ffffffff8103f44e>] ? try_to_wake_up+0x197/0x197
Jun 4 23:00:20 atomsrv kernel: [ 5161.114123] [<ffffffffa0321553>] ? task_done+0x110/0x110 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.114144] [<ffffffff8105f321>] ? kthread+0x76/0x7e
Jun 4 23:00:20 atomsrv kernel: [ 5161.114163] [<ffffffff81354ab4>] ? kernel_thread_helper+0x4/0x10
Jun 4 23:00:20 atomsrv kernel: [ 5161.114184] [<ffffffff8105f2ab>] ? kthread_worker_fn+0x139/0x139
Jun 4 23:00:20 atomsrv kernel: [ 5161.114203] [<ffffffff81354ab0>] ? gs_change+0x13/0x13
Jun 4 23:00:20 atomsrv kernel: [ 5161.114219] INFO: task z_wr_int/1:2426 blocked for more than 120 seconds.
Jun 4 23:00:20 atomsrv kernel: [ 5161.114256] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 4 23:00:20 atomsrv kernel: [ 5161.114306] z_wr_int/1 D ffff88007f493780 0 2426 2 0x00000000
Jun 4 23:00:20 atomsrv kernel: [ 5161.114331] ffff8800768ad020 0000000000000046 0000000000000001 ffff88007998d850
Jun 4 23:00:20 atomsrv kernel: [ 5161.114363] 0000000000013780 ffff8800768b1fd8 ffff8800768b1fd8 ffff8800768ad020
Jun 4 23:00:20 atomsrv kernel: [ 5161.114394] 0000000000000246 ffffffffa031ccd5 ffffffff81036428 ffff880077d6b718
Jun 4 23:00:20 atomsrv kernel: [ 5161.114425] Call Trace:
Jun 4 23:00:20 atomsrv kernel: [ 5161.114448] [<ffffffffa031ccd5>] ? spl_kmem_cache_free+0x2c8/0x318 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.114469] [<ffffffff81036428>] ? should_resched+0x5/0x23
Jun 4 23:00:20 atomsrv kernel: [ 5161.114489] [<ffffffff8134cf8f>] ? __mutex_lock_common.isra.5+0xff/0x164
Jun 4 23:00:20 atomsrv kernel: [ 5161.114510] [<ffffffff8134ce7d>] ? mutex_lock+0x1a/0x2d
Jun 4 23:00:20 atomsrv kernel: [ 5161.114598] [<ffffffffa03e7d7c>] ? vdev_queue_io_done+0x28/0xa0 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.114688] [<ffffffffa0412f8d>] ? zio_vdev_io_done+0x65/0x13f [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.114791] [<ffffffffa0413594>] ? zio_execute+0xc5/0xe7 [zfs]
Jun 4 23:00:20 atomsrv kernel: [ 5161.114818] [<ffffffffa03217e3>] ? taskq_thread+0x290/0x3e6 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.114831] [<ffffffff8103f44e>] ? try_to_wake_up+0x197/0x197
Jun 4 23:00:20 atomsrv kernel: [ 5161.114854] [<ffffffffa0321553>] ? task_done+0x110/0x110 [spl]
Jun 4 23:00:20 atomsrv kernel: [ 5161.114869] [<ffffffff8105f321>] ? kthread+0x76/0x7e
Jun 4 23:00:20 atomsrv kernel: [ 5161.114880] [<ffffffff81354ab4>] ? kernel_thread_helper+0x4/0x10
Jun 4 23:00:20 atomsrv kernel: [ 5161.114891] [<ffffffff8105f2ab>] ? kthread_worker_fn+0x139/0x139
Jun 4 23:00:20 atomsrv kernel: [ 5161.114901] [<ffffffff81354ab0>] ? gs_change+0x13/0x13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment