Skip to content

Instantly share code, notes, and snippets.

@deajan

deajan/zfs-kswapd

Created Apr 5, 2015
Embed
What would you like to do?
Apr 5 05:49:22 backupmaster kernel: INFO: task kswapd0:46 blocked for more than 120 seconds.
Apr 5 05:49:22 backupmaster kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 5 05:49:22 backupmaster kernel: kswapd0 D ffff88021fc13680 0 46 2 0x00000000
Apr 5 05:49:22 backupmaster kernel: ffff880210fab778 0000000000000046 ffff880210fabfd8 0000000000013680
Apr 5 05:49:22 backupmaster kernel: ffff880210fabfd8 0000000000013680 ffff880213d64440 ffff880213d64440
Apr 5 05:49:22 backupmaster kernel: ffff8800d22ee570 fffffffeffffffff ffff8800d22ee578 0000000000000001
Apr 5 05:49:22 backupmaster kernel: Call Trace:
Apr 5 05:49:22 backupmaster kernel: [<ffffffff81609e29>] schedule+0x29/0x70
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8160b925>] rwsem_down_read_failed+0xf5/0x165
Apr 5 05:49:22 backupmaster kernel: [<ffffffff812ede0d>] ? list_del+0xd/0x30
Apr 5 05:49:22 backupmaster kernel: [<ffffffff812e31e4>] call_rwsem_down_read_failed+0x14/0x30
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa02b59d2>] ? spl_kmem_free+0x32/0x50 [spl]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff816091f0>] ? down_read+0x20/0x30
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03effeb>] dmu_zfetch_find+0x4b/0x830 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff812ede0d>] ? list_del+0xd/0x30
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03f0add>] dmu_zfetch+0xcd/0x930 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03d7249>] dbuf_read+0xa99/0xbd0 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03f2548>] dnode_hold_impl+0x198/0x610 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811ae07d>] ? __kmalloc_node+0x13d/0x280
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03f29d9>] dnode_hold+0x19/0x20 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03ed732>] dmu_tx_hold_object_impl.isra.1+0x42/0x190 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03ed895>] dmu_tx_hold_bonus+0x15/0x30 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03ef01c>] dmu_tx_hold_sa+0x3c/0x180 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa047302b>] zfs_inactive+0x15b/0x290 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa048b783>] zpl_evict_inode+0x43/0x60 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811e2237>] evict+0xa7/0x170
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811e233e>] dispose_list+0x3e/0x50
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811e3213>] prune_icache_sb+0x163/0x320
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811ca496>] prune_super+0xd6/0x1a0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811691a5>] shrink_slab+0x165/0x300
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811c0751>] ? vmpressure+0x21/0x90
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8116cdf1>] balance_pgdat+0x4b1/0x5e0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8116d093>] kswapd+0x173/0x450
Apr 5 05:49:22 backupmaster kernel: [<ffffffff81098340>] ? wake_up_bit+0x30/0x30
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8116cf20>] ? balance_pgdat+0x5e0/0x5e0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8109738f>] kthread+0xcf/0xe0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8161497c>] ret_from_fork+0x7c/0xb0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:49:22 backupmaster kernel: INFO: task khugepaged:48 blocked for more than 120 seconds.
Apr 5 05:49:22 backupmaster kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 5 05:49:22 backupmaster kernel: khugepaged D ffff88021fd93680 0 48 2 0x00000000
Apr 5 05:49:22 backupmaster kernel: ffff880210fef578 0000000000000046 ffff880210feffd8 0000000000013680
Apr 5 05:49:22 backupmaster kernel: ffff880210feffd8 0000000000013680 ffff880213d62d80 ffff880213d62d80
Apr 5 05:49:22 backupmaster kernel: ffff8800d22ee570 ffffffffffffffff ffff8800d22ee578 0000000000000001
Apr 5 05:49:22 backupmaster kernel: Call Trace:
Apr 5 05:49:22 backupmaster kernel: [<ffffffff81609e29>] schedule+0x29/0x70
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8160b925>] rwsem_down_read_failed+0xf5/0x165
Apr 5 05:49:22 backupmaster kernel: [<ffffffff812e31e4>] call_rwsem_down_read_failed+0x14/0x30
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa02b59d2>] ? spl_kmem_free+0x32/0x50 [spl]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff816091f0>] ? down_read+0x20/0x30
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03effeb>] dmu_zfetch_find+0x4b/0x830 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03c75c1>] ? remove_reference.isra.11+0xe1/0x130 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff816081b2>] ? mutex_lock+0x12/0x2f
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03f0add>] dmu_zfetch+0xcd/0x930 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03d7249>] dbuf_read+0xa99/0xbd0 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03f2548>] dnode_hold_impl+0x198/0x610 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811ae07d>] ? __kmalloc_node+0x13d/0x280
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03f29d9>] dnode_hold+0x19/0x20 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03ed732>] dmu_tx_hold_object_impl.isra.1+0x42/0x190 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03ed895>] dmu_tx_hold_bonus+0x15/0x30 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03ef01c>] dmu_tx_hold_sa+0x3c/0x180 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa047302b>] zfs_inactive+0x15b/0x290 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa048b783>] zpl_evict_inode+0x43/0x60 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811e2237>] evict+0xa7/0x170
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811e233e>] dispose_list+0x3e/0x50
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811e3213>] prune_icache_sb+0x163/0x320
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811ca496>] prune_super+0xd6/0x1a0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811691a5>] shrink_slab+0x165/0x300
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8117ad32>] ? compact_zone+0x32/0x400
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8116c2f2>] do_try_to_free_pages+0x3c2/0x4e0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8116c50c>] try_to_free_pages+0xfc/0x180
Apr 5 05:49:22 backupmaster kernel: [<ffffffff81160bad>] __alloc_pages_nodemask+0x7fd/0xb90
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811b43d9>] khugepaged_scan_mm_slot+0x3c9/0xca0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff81607cdd>] ? schedule_timeout+0x17d/0x2d0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811b4f0f>] khugepaged+0x25f/0x4a0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff81098340>] ? wake_up_bit+0x30/0x30
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811b4cb0>] ? khugepaged_scan_mm_slot+0xca0/0xca0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8109738f>] kthread+0xcf/0xe0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8161497c>] ret_from_fork+0x7c/0xb0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:49:22 backupmaster kernel: INFO: task rsync:27795 blocked for more than 120 seconds.
Apr 5 05:49:22 backupmaster kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 5 05:49:22 backupmaster kernel: rsync D ffff88021fd93680 0 27795 27789 0x00000080
Apr 5 05:49:22 backupmaster kernel: ffff8802076eeb68 0000000000000086 ffff8802076effd8 0000000000013680
Apr 5 05:49:22 backupmaster kernel: ffff8802076effd8 0000000000013680 ffff8800bfbc2d80 ffff8800bfbc2d80
Apr 5 05:49:22 backupmaster kernel: ffff8800d22ee570 ffffffffffffffff ffff8800d22ee578 0000000000000001
Apr 5 05:49:22 backupmaster kernel: Call Trace:
Apr 5 05:49:22 backupmaster kernel: [<ffffffff81609e29>] schedule+0x29/0x70
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8160b925>] rwsem_down_read_failed+0xf5/0x165
Apr 5 05:49:22 backupmaster kernel: [<ffffffff812e31e4>] call_rwsem_down_read_failed+0x14/0x30
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa02b59d2>] ? spl_kmem_free+0x32/0x50 [spl]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff816091f0>] ? down_read+0x20/0x30
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03effeb>] dmu_zfetch_find+0x4b/0x830 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03f0add>] dmu_zfetch+0xcd/0x930 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03d7249>] dbuf_read+0xa99/0xbd0 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03f2548>] dnode_hold_impl+0x198/0x610 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811ae07d>] ? __kmalloc_node+0x13d/0x280
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03f29d9>] dnode_hold+0x19/0x20 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03ed732>] dmu_tx_hold_object_impl.isra.1+0x42/0x190 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03ed895>] dmu_tx_hold_bonus+0x15/0x30 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03ef01c>] dmu_tx_hold_sa+0x3c/0x180 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa047302b>] zfs_inactive+0x15b/0x290 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa048b783>] zpl_evict_inode+0x43/0x60 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811e2237>] evict+0xa7/0x170
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811e233e>] dispose_list+0x3e/0x50
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811e3213>] prune_icache_sb+0x163/0x320
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811ca496>] prune_super+0xd6/0x1a0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811691a5>] shrink_slab+0x165/0x300
Apr 5 05:49:22 backupmaster kernel: [<ffffffff816081b2>] ? mutex_lock+0x12/0x2f
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811c0791>] ? vmpressure+0x61/0x90
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8116c2f2>] do_try_to_free_pages+0x3c2/0x4e0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8116c50c>] try_to_free_pages+0xfc/0x180
Apr 5 05:49:22 backupmaster kernel: [<ffffffff81160bad>] __alloc_pages_nodemask+0x7fd/0xb90
Apr 5 05:49:22 backupmaster kernel: [<ffffffff8119f4e9>] alloc_pages_current+0xa9/0x170
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811a9595>] new_slab+0x275/0x300
Apr 5 05:49:22 backupmaster kernel: [<ffffffff81601a3b>] __slab_alloc+0x315/0x48f
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa02b6d0a>] ? spl_kmem_cache_alloc+0xaa/0x180 [spl]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811abbd3>] kmem_cache_alloc+0x193/0x1d0
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa02b6d0a>] ? spl_kmem_cache_alloc+0xaa/0x180 [spl]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa02b6d0a>] spl_kmem_cache_alloc+0xaa/0x180 [spl]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa047fa93>] zio_buf_alloc+0x23/0x30 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03c9d62>] arc_get_data_buf.isra.19+0x2b2/0x4a0 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03ccf7f>] arc_buf_alloc+0x10f/0x140 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03cd96c>] arc_read+0x1cc/0xb60 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03d6ff2>] ? dbuf_read+0x842/0xbd0 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03d75d3>] ? __dbuf_hold_impl+0x253/0x5b0 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03d7bc3>] dbuf_prefetch+0x1f3/0x340 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03eff34>] dmu_zfetch_dofetch.isra.4+0x114/0x180 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03f10b2>] dmu_zfetch+0x6a2/0x930 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03d6bef>] dbuf_read+0x43f/0xbd0 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03f2548>] dnode_hold_impl+0x198/0x610 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03f29d9>] dnode_hold+0x19/0x20 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03e0b68>] dmu_bonus_hold+0x38/0x300 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa0419a7e>] sa_buf_hold+0xe/0x10 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa0479039>] zfs_zget+0x89/0x460 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa0455dd5>] zfs_dirent_lock+0x4d5/0x620 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa0455fb0>] zfs_dirlook+0x90/0x380 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa03c75c1>] ? remove_reference.isra.11+0xe1/0x130 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa046c7c6>] zfs_lookup+0x306/0x360 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffffa048ae03>] zpl_lookup+0x93/0x100 [zfs]
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811d0b4d>] lookup_real+0x1d/0x50
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811d1492>] __lookup_hash+0x42/0x60
Apr 5 05:49:22 backupmaster kernel: [<ffffffff81601e12>] lookup_slow+0x42/0xa7
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811d55b3>] path_lookupat+0x773/0x7a0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811abbd3>] ? kmem_cache_alloc+0x193/0x1d0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811d716f>] ? getname_flags+0x4f/0x190
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811d560b>] filename_lookup+0x2b/0xc0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811d7fe7>] user_path_at_empty+0x67/0xc0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811d8051>] user_path_at+0x11/0x20
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811cbb23>] vfs_fstatat+0x63/0xc0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff81076182>] ? current_fs_time+0x12/0x60
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811cc0f1>] SYSC_newlstat+0x31/0x60
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811da44c>] ? vfs_readdir+0x8c/0xe0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff810faeb6>] ? __audit_syscall_exit+0x1f6/0x2a0
Apr 5 05:49:22 backupmaster kernel: [<ffffffff811cc37e>] SyS_newlstat+0xe/0x10
Apr 5 05:49:22 backupmaster kernel: [<ffffffff81614a29>] system_call_fastpath+0x16/0x1b
Apr 5 05:51:01 backupmaster systemd: Starting Session 368 of user root.
Apr 5 05:51:01 backupmaster systemd: Started Session 368 of user root.
Apr 5 05:51:01 backupmaster systemd: Failed to reset devices.list on /machine.slice: Invalid argument
Apr 5 05:51:22 backupmaster kernel: INFO: task kswapd0:46 blocked for more than 120 seconds.
Apr 5 05:51:22 backupmaster kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 5 05:51:22 backupmaster kernel: kswapd0 D ffff88021fc13680 0 46 2 0x00000000
Apr 5 05:51:22 backupmaster kernel: ffff880210fab778 0000000000000046 ffff880210fabfd8 0000000000013680
Apr 5 05:51:22 backupmaster kernel: ffff880210fabfd8 0000000000013680 ffff880213d64440 ffff880213d64440
Apr 5 05:51:22 backupmaster kernel: ffff8800d22ee570 fffffffeffffffff ffff8800d22ee578 0000000000000001
Apr 5 05:51:22 backupmaster kernel: Call Trace:
Apr 5 05:51:22 backupmaster kernel: [<ffffffff81609e29>] schedule+0x29/0x70
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8160b925>] rwsem_down_read_failed+0xf5/0x165
Apr 5 05:51:22 backupmaster kernel: [<ffffffff812ede0d>] ? list_del+0xd/0x30
Apr 5 05:51:22 backupmaster kernel: [<ffffffff812e31e4>] call_rwsem_down_read_failed+0x14/0x30
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa02b59d2>] ? spl_kmem_free+0x32/0x50 [spl]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff816091f0>] ? down_read+0x20/0x30
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03effeb>] dmu_zfetch_find+0x4b/0x830 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff812ede0d>] ? list_del+0xd/0x30
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03f0add>] dmu_zfetch+0xcd/0x930 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03d7249>] dbuf_read+0xa99/0xbd0 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03f2548>] dnode_hold_impl+0x198/0x610 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811ae07d>] ? __kmalloc_node+0x13d/0x280
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03f29d9>] dnode_hold+0x19/0x20 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03ed732>] dmu_tx_hold_object_impl.isra.1+0x42/0x190 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03ed895>] dmu_tx_hold_bonus+0x15/0x30 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03ef01c>] dmu_tx_hold_sa+0x3c/0x180 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa047302b>] zfs_inactive+0x15b/0x290 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa048b783>] zpl_evict_inode+0x43/0x60 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811e2237>] evict+0xa7/0x170
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811e233e>] dispose_list+0x3e/0x50
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811e3213>] prune_icache_sb+0x163/0x320
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811ca496>] prune_super+0xd6/0x1a0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811691a5>] shrink_slab+0x165/0x300
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811c0751>] ? vmpressure+0x21/0x90
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8116cdf1>] balance_pgdat+0x4b1/0x5e0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8116d093>] kswapd+0x173/0x450
Apr 5 05:51:22 backupmaster kernel: [<ffffffff81098340>] ? wake_up_bit+0x30/0x30
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8116cf20>] ? balance_pgdat+0x5e0/0x5e0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8109738f>] kthread+0xcf/0xe0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8161497c>] ret_from_fork+0x7c/0xb0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:51:22 backupmaster kernel: INFO: task khugepaged:48 blocked for more than 120 seconds.
Apr 5 05:51:22 backupmaster kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 5 05:51:22 backupmaster kernel: khugepaged D ffff88021fd93680 0 48 2 0x00000000
Apr 5 05:51:22 backupmaster kernel: ffff880210fef578 0000000000000046 ffff880210feffd8 0000000000013680
Apr 5 05:51:22 backupmaster kernel: ffff880210feffd8 0000000000013680 ffff880213d62d80 ffff880213d62d80
Apr 5 05:51:22 backupmaster kernel: ffff8800d22ee570 ffffffffffffffff ffff8800d22ee578 0000000000000001
Apr 5 05:51:22 backupmaster kernel: Call Trace:
Apr 5 05:51:22 backupmaster kernel: [<ffffffff81609e29>] schedule+0x29/0x70
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8160b925>] rwsem_down_read_failed+0xf5/0x165
Apr 5 05:51:22 backupmaster kernel: [<ffffffff812e31e4>] call_rwsem_down_read_failed+0x14/0x30
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa02b59d2>] ? spl_kmem_free+0x32/0x50 [spl]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff816091f0>] ? down_read+0x20/0x30
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03effeb>] dmu_zfetch_find+0x4b/0x830 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03c75c1>] ? remove_reference.isra.11+0xe1/0x130 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff816081b2>] ? mutex_lock+0x12/0x2f
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03f0add>] dmu_zfetch+0xcd/0x930 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03d7249>] dbuf_read+0xa99/0xbd0 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03f2548>] dnode_hold_impl+0x198/0x610 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811ae07d>] ? __kmalloc_node+0x13d/0x280
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03f29d9>] dnode_hold+0x19/0x20 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03ed732>] dmu_tx_hold_object_impl.isra.1+0x42/0x190 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03ed895>] dmu_tx_hold_bonus+0x15/0x30 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03ef01c>] dmu_tx_hold_sa+0x3c/0x180 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa047302b>] zfs_inactive+0x15b/0x290 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa048b783>] zpl_evict_inode+0x43/0x60 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811e2237>] evict+0xa7/0x170
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811e233e>] dispose_list+0x3e/0x50
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811e3213>] prune_icache_sb+0x163/0x320
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811ca496>] prune_super+0xd6/0x1a0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811691a5>] shrink_slab+0x165/0x300
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8117ad32>] ? compact_zone+0x32/0x400
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8116c2f2>] do_try_to_free_pages+0x3c2/0x4e0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8116c50c>] try_to_free_pages+0xfc/0x180
Apr 5 05:51:22 backupmaster kernel: [<ffffffff81160bad>] __alloc_pages_nodemask+0x7fd/0xb90
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811b43d9>] khugepaged_scan_mm_slot+0x3c9/0xca0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff81607cdd>] ? schedule_timeout+0x17d/0x2d0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811b4f0f>] khugepaged+0x25f/0x4a0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff81098340>] ? wake_up_bit+0x30/0x30
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811b4cb0>] ? khugepaged_scan_mm_slot+0xca0/0xca0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8109738f>] kthread+0xcf/0xe0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8161497c>] ret_from_fork+0x7c/0xb0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:51:22 backupmaster kernel: INFO: task rsync:27795 blocked for more than 120 seconds.
Apr 5 05:51:22 backupmaster kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 5 05:51:22 backupmaster kernel: rsync D ffff88021fd93680 0 27795 27789 0x00000080
Apr 5 05:51:22 backupmaster kernel: ffff8802076eeb68 0000000000000086 ffff8802076effd8 0000000000013680
Apr 5 05:51:22 backupmaster kernel: ffff8802076effd8 0000000000013680 ffff8800bfbc2d80 ffff8800bfbc2d80
Apr 5 05:51:22 backupmaster kernel: ffff8800d22ee570 ffffffffffffffff ffff8800d22ee578 0000000000000001
Apr 5 05:51:22 backupmaster kernel: Call Trace:
Apr 5 05:51:22 backupmaster kernel: [<ffffffff81609e29>] schedule+0x29/0x70
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8160b925>] rwsem_down_read_failed+0xf5/0x165
Apr 5 05:51:22 backupmaster kernel: [<ffffffff812e31e4>] call_rwsem_down_read_failed+0x14/0x30
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa02b59d2>] ? spl_kmem_free+0x32/0x50 [spl]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff816091f0>] ? down_read+0x20/0x30
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03effeb>] dmu_zfetch_find+0x4b/0x830 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03f0add>] dmu_zfetch+0xcd/0x930 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03d7249>] dbuf_read+0xa99/0xbd0 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03f2548>] dnode_hold_impl+0x198/0x610 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811ae07d>] ? __kmalloc_node+0x13d/0x280
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03f29d9>] dnode_hold+0x19/0x20 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03ed732>] dmu_tx_hold_object_impl.isra.1+0x42/0x190 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03ed895>] dmu_tx_hold_bonus+0x15/0x30 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03ef01c>] dmu_tx_hold_sa+0x3c/0x180 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa047302b>] zfs_inactive+0x15b/0x290 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa048b783>] zpl_evict_inode+0x43/0x60 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811e2237>] evict+0xa7/0x170
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811e233e>] dispose_list+0x3e/0x50
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811e3213>] prune_icache_sb+0x163/0x320
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811ca496>] prune_super+0xd6/0x1a0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811691a5>] shrink_slab+0x165/0x300
Apr 5 05:51:22 backupmaster kernel: [<ffffffff816081b2>] ? mutex_lock+0x12/0x2f
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811c0791>] ? vmpressure+0x61/0x90
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8116c2f2>] do_try_to_free_pages+0x3c2/0x4e0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8116c50c>] try_to_free_pages+0xfc/0x180
Apr 5 05:51:22 backupmaster kernel: [<ffffffff81160bad>] __alloc_pages_nodemask+0x7fd/0xb90
Apr 5 05:51:22 backupmaster kernel: [<ffffffff8119f4e9>] alloc_pages_current+0xa9/0x170
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811a9595>] new_slab+0x275/0x300
Apr 5 05:51:22 backupmaster kernel: [<ffffffff81601a3b>] __slab_alloc+0x315/0x48f
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa02b6d0a>] ? spl_kmem_cache_alloc+0xaa/0x180 [spl]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811abbd3>] kmem_cache_alloc+0x193/0x1d0
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa02b6d0a>] ? spl_kmem_cache_alloc+0xaa/0x180 [spl]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa02b6d0a>] spl_kmem_cache_alloc+0xaa/0x180 [spl]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa047fa93>] zio_buf_alloc+0x23/0x30 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03c9d62>] arc_get_data_buf.isra.19+0x2b2/0x4a0 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03ccf7f>] arc_buf_alloc+0x10f/0x140 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03cd96c>] arc_read+0x1cc/0xb60 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03d6ff2>] ? dbuf_read+0x842/0xbd0 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03d75d3>] ? __dbuf_hold_impl+0x253/0x5b0 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03d7bc3>] dbuf_prefetch+0x1f3/0x340 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03eff34>] dmu_zfetch_dofetch.isra.4+0x114/0x180 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03f10b2>] dmu_zfetch+0x6a2/0x930 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03d6bef>] dbuf_read+0x43f/0xbd0 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03f2548>] dnode_hold_impl+0x198/0x610 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03f29d9>] dnode_hold+0x19/0x20 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03e0b68>] dmu_bonus_hold+0x38/0x300 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa0419a7e>] sa_buf_hold+0xe/0x10 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa0479039>] zfs_zget+0x89/0x460 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa0455dd5>] zfs_dirent_lock+0x4d5/0x620 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa0455fb0>] zfs_dirlook+0x90/0x380 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa03c75c1>] ? remove_reference.isra.11+0xe1/0x130 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa046c7c6>] zfs_lookup+0x306/0x360 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffffa048ae03>] zpl_lookup+0x93/0x100 [zfs]
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811d0b4d>] lookup_real+0x1d/0x50
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811d1492>] __lookup_hash+0x42/0x60
Apr 5 05:51:22 backupmaster kernel: [<ffffffff81601e12>] lookup_slow+0x42/0xa7
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811d55b3>] path_lookupat+0x773/0x7a0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811abbd3>] ? kmem_cache_alloc+0x193/0x1d0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811d716f>] ? getname_flags+0x4f/0x190
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811d560b>] filename_lookup+0x2b/0xc0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811d7fe7>] user_path_at_empty+0x67/0xc0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811d8051>] user_path_at+0x11/0x20
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811cbb23>] vfs_fstatat+0x63/0xc0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff81076182>] ? current_fs_time+0x12/0x60
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811cc0f1>] SYSC_newlstat+0x31/0x60
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811da44c>] ? vfs_readdir+0x8c/0xe0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff810faeb6>] ? __audit_syscall_exit+0x1f6/0x2a0
Apr 5 05:51:22 backupmaster kernel: [<ffffffff811cc37e>] SyS_newlstat+0xe/0x10
Apr 5 05:51:22 backupmaster kernel: [<ffffffff81614a29>] system_call_fastpath+0x16/0x1b
Apr 5 05:53:22 backupmaster kernel: INFO: task kswapd0:46 blocked for more than 120 seconds.
Apr 5 05:53:22 backupmaster kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 5 05:53:22 backupmaster kernel: kswapd0 D ffff88021fc13680 0 46 2 0x00000000
Apr 5 05:53:22 backupmaster kernel: ffff880210fab778 0000000000000046 ffff880210fabfd8 0000000000013680
Apr 5 05:53:22 backupmaster kernel: ffff880210fabfd8 0000000000013680 ffff880213d64440 ffff880213d64440
Apr 5 05:53:22 backupmaster kernel: ffff8800d22ee570 fffffffeffffffff ffff8800d22ee578 0000000000000001
Apr 5 05:53:22 backupmaster kernel: Call Trace:
Apr 5 05:53:22 backupmaster kernel: [<ffffffff81609e29>] schedule+0x29/0x70
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8160b925>] rwsem_down_read_failed+0xf5/0x165
Apr 5 05:53:22 backupmaster kernel: [<ffffffff812ede0d>] ? list_del+0xd/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffff812e31e4>] call_rwsem_down_read_failed+0x14/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa02b59d2>] ? spl_kmem_free+0x32/0x50 [spl]
Apr 5 05:53:22 backupmaster kernel: [<ffffffff816091f0>] ? down_read+0x20/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03effeb>] dmu_zfetch_find+0x4b/0x830 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffff812ede0d>] ? list_del+0xd/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03f0add>] dmu_zfetch+0xcd/0x930 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03d7249>] dbuf_read+0xa99/0xbd0 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03f2548>] dnode_hold_impl+0x198/0x610 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811ae07d>] ? __kmalloc_node+0x13d/0x280
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03f29d9>] dnode_hold+0x19/0x20 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03ed732>] dmu_tx_hold_object_impl.isra.1+0x42/0x190 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03ed895>] dmu_tx_hold_bonus+0x15/0x30 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03ef01c>] dmu_tx_hold_sa+0x3c/0x180 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa047302b>] zfs_inactive+0x15b/0x290 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa048b783>] zpl_evict_inode+0x43/0x60 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811e2237>] evict+0xa7/0x170
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811e233e>] dispose_list+0x3e/0x50
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811e3213>] prune_icache_sb+0x163/0x320
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811ca496>] prune_super+0xd6/0x1a0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811691a5>] shrink_slab+0x165/0x300
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811c0751>] ? vmpressure+0x21/0x90
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8116cdf1>] balance_pgdat+0x4b1/0x5e0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8116d093>] kswapd+0x173/0x450
Apr 5 05:53:22 backupmaster kernel: [<ffffffff81098340>] ? wake_up_bit+0x30/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8116cf20>] ? balance_pgdat+0x5e0/0x5e0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8109738f>] kthread+0xcf/0xe0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8161497c>] ret_from_fork+0x7c/0xb0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:53:22 backupmaster kernel: INFO: task ksmd:47 blocked for more than 120 seconds.
Apr 5 05:53:22 backupmaster kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 5 05:53:22 backupmaster kernel: ksmd D ffff88021fd93680 0 47 2 0x00000000
Apr 5 05:53:22 backupmaster kernel: ffff880210fafcf8 0000000000000046 ffff880210faffd8 0000000000013680
Apr 5 05:53:22 backupmaster kernel: ffff880210faffd8 0000000000013680 ffff880213d64fa0 ffff880213d64fa0
Apr 5 05:53:22 backupmaster kernel: ffff8802108570f8 ffffffffffffffff ffff880210857100 0000000000000000
Apr 5 05:53:22 backupmaster kernel: Call Trace:
Apr 5 05:53:22 backupmaster kernel: [<ffffffff81609e29>] schedule+0x29/0x70
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8160b925>] rwsem_down_read_failed+0xf5/0x165
Apr 5 05:53:22 backupmaster kernel: [<ffffffff812e31e4>] call_rwsem_down_read_failed+0x14/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffff816091f0>] ? down_read+0x20/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811a63dd>] ksm_do_scan+0x9d/0xd90
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811a715f>] ksm_scan_thread+0x8f/0x240
Apr 5 05:53:22 backupmaster kernel: [<ffffffff81098340>] ? wake_up_bit+0x30/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811a70d0>] ? ksm_do_scan+0xd90/0xd90
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8109738f>] kthread+0xcf/0xe0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8161497c>] ret_from_fork+0x7c/0xb0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:53:22 backupmaster kernel: INFO: task khugepaged:48 blocked for more than 120 seconds.
Apr 5 05:53:22 backupmaster kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 5 05:53:22 backupmaster kernel: khugepaged D ffff88021fd93680 0 48 2 0x00000000
Apr 5 05:53:22 backupmaster kernel: ffff880210fef578 0000000000000046 ffff880210feffd8 0000000000013680
Apr 5 05:53:22 backupmaster kernel: ffff880210feffd8 0000000000013680 ffff880213d62d80 ffff880213d62d80
Apr 5 05:53:22 backupmaster kernel: ffff8800d22ee570 ffffffffffffffff ffff8800d22ee578 0000000000000001
Apr 5 05:53:22 backupmaster kernel: Call Trace:
Apr 5 05:53:22 backupmaster kernel: [<ffffffff81609e29>] schedule+0x29/0x70
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8160b925>] rwsem_down_read_failed+0xf5/0x165
Apr 5 05:53:22 backupmaster kernel: [<ffffffff812e31e4>] call_rwsem_down_read_failed+0x14/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa02b59d2>] ? spl_kmem_free+0x32/0x50 [spl]
Apr 5 05:53:22 backupmaster kernel: [<ffffffff816091f0>] ? down_read+0x20/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03effeb>] dmu_zfetch_find+0x4b/0x830 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03c75c1>] ? remove_reference.isra.11+0xe1/0x130 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffff816081b2>] ? mutex_lock+0x12/0x2f
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03f0add>] dmu_zfetch+0xcd/0x930 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03d7249>] dbuf_read+0xa99/0xbd0 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03f2548>] dnode_hold_impl+0x198/0x610 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811ae07d>] ? __kmalloc_node+0x13d/0x280
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03f29d9>] dnode_hold+0x19/0x20 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03ed732>] dmu_tx_hold_object_impl.isra.1+0x42/0x190 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03ed895>] dmu_tx_hold_bonus+0x15/0x30 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa03ef01c>] dmu_tx_hold_sa+0x3c/0x180 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa047302b>] zfs_inactive+0x15b/0x290 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa048b783>] zpl_evict_inode+0x43/0x60 [zfs]
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811e2237>] evict+0xa7/0x170
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811e233e>] dispose_list+0x3e/0x50
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811e3213>] prune_icache_sb+0x163/0x320
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811ca496>] prune_super+0xd6/0x1a0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811691a5>] shrink_slab+0x165/0x300
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8117ad32>] ? compact_zone+0x32/0x400
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8116c2f2>] do_try_to_free_pages+0x3c2/0x4e0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8116c50c>] try_to_free_pages+0xfc/0x180
Apr 5 05:53:22 backupmaster kernel: [<ffffffff81160bad>] __alloc_pages_nodemask+0x7fd/0xb90
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811b43d9>] khugepaged_scan_mm_slot+0x3c9/0xca0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff81607cdd>] ? schedule_timeout+0x17d/0x2d0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811b4f0f>] khugepaged+0x25f/0x4a0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff81098340>] ? wake_up_bit+0x30/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811b4cb0>] ? khugepaged_scan_mm_slot+0xca0/0xca0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8109738f>] kthread+0xcf/0xe0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8161497c>] ret_from_fork+0x7c/0xb0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff810972c0>] ? kthread_create_on_node+0x140/0x140
Apr 5 05:53:22 backupmaster kernel: INFO: task qemu-kvm:3407 blocked for more than 120 seconds.
Apr 5 05:53:22 backupmaster kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 5 05:53:22 backupmaster kernel: qemu-kvm D ffffffffa071343d 0 3407 1 0x00000080
Apr 5 05:53:22 backupmaster kernel: ffff8800ccb8fa98 0000000000000082 ffff8800ccb8ffd8 0000000000013680
Apr 5 05:53:22 backupmaster kernel: ffff8800ccb8ffd8 0000000000013680 ffff8800cdfcad80 ffff8800cdfcad80
Apr 5 05:53:22 backupmaster kernel: ffff8802108570f8 ffffffffffffffff ffff880210857100 0000000000000002
Apr 5 05:53:22 backupmaster kernel: Call Trace:
Apr 5 05:53:22 backupmaster kernel: [<ffffffff81609e29>] schedule+0x29/0x70
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8160b925>] rwsem_down_read_failed+0xf5/0x165
Apr 5 05:53:22 backupmaster kernel: [<ffffffff812e31e4>] call_rwsem_down_read_failed+0x14/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8101cb3e>] ? arch_dup_task_struct+0x4e/0x100
Apr 5 05:53:22 backupmaster kernel: [<ffffffff816091f0>] ? down_read+0x20/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa071827a>] kvm_host_page_size+0x4a/0x90 [kvm]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa073267e>] mapping_level.isra.97+0x1e/0xa0 [kvm]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa0738d0d>] tdp_page_fault+0xad/0x220 [kvm]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa073e7b6>] ? writeback_registers+0x56/0x70 [kvm]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa0732e04>] kvm_mmu_page_fault+0x24/0x100 [kvm]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa085c2a4>] handle_ept_violation+0x94/0x160 [kvm_intel]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa0862235>] vmx_handle_exit+0xd5/0xa30 [kvm_intel]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa072796a>] vcpu_enter_guest+0x6ba/0xd70 [kvm]
Apr 5 05:53:22 backupmaster kernel: [<ffffffff810981f6>] ? finish_wait+0x56/0x70
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa07139db>] ? kvm_vcpu_block+0xab/0xd0 [kvm]
Apr 5 05:53:22 backupmaster kernel: [<ffffffff81098340>] ? wake_up_bit+0x30/0x30
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa072c1f8>] kvm_arch_vcpu_ioctl_run+0x208/0x480 [kvm]
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa071632a>] kvm_vcpu_ioctl+0x2aa/0x580 [kvm]
Apr 5 05:53:22 backupmaster kernel: [<ffffffff813ae304>] ? io_serial_in+0x14/0x20
Apr 5 05:53:22 backupmaster kernel: [<ffffffffa0712180>] ? kvm_vcpu_mmap+0x20/0x20 [kvm]
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811da125>] do_vfs_ioctl+0x2e5/0x4c0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff8126f7ae>] ? file_has_perm+0xae/0xc0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff811da3a1>] SyS_ioctl+0xa1/0xc0
Apr 5 05:53:22 backupmaster kernel: [<ffffffff81614a29>] system_call_fastpath+0x16/0x1b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment