Skip to content

Instantly share code, notes, and snippets.

@twslankard

twslankard/syslog

Created August 29, 2013 16:03
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save twslankard/6380033 to your computer and use it in GitHub Desktop.
Save twslankard/6380033 to your computer and use it in GitHub Desktop.
Download ZIP
soft lockups on t1.micro
Raw
syslog
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450207] BUG: soft lockup - CPU#0 stuck for 40s! [ps:17019]
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450221] Modules linked in: isofs acpiphp
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450226] CPU 0
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450227] Modules linked in: isofs acpiphp
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450230]
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450233] Pid: 17019, comm: ps Not tainted 3.2.0-52-virtual #78-Ubuntu
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450236] RIP: e030:[<ffffffff8100122a>] [<ffffffff8100122a>] hypercall_page+0x22a/0x1000
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450248] RSP: e02b:ffff880023c8bdc8 EFLAGS: 00000246
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450249] RAX: 0000000000030004 RBX: ffff880023c8bf58 RCX: ffffffff8100122a
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450251] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450253] RBP: ffff880023c8bde0 R08: 0000000000000007 R09: 0000000000000000
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450255] R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000028
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450256] R13: 0000000000000000 R14: ffff880023472068 R15: ffff880023472000
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450262] FS: 00007f3b0165e700(0000) GS:ffff880026400000(0000) knlGS:0000000000000000
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450265] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450266] CR2: 00007f3b01663000 CR3: 00000000243e6000 CR4: 0000000000002660
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450269] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450271] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450273] Process ps (pid: 17019, threadinfo ffff880023c8a000, task ffff8800028a8000)
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450275] Stack:
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450278] 0000000000000246 00000000ffffffff ffffffff8100a53d ffff880023c8bf48
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450282] ffffffff8100ad02 0000000000000246 0000000000000001 0000000000000000
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450285] 0000000000000007 ffff880023c8bf58 0000000000000004 0000000000000000
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450288] Call Trace:
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450297] [<ffffffff8100a53d>] ? xen_force_evtchn_callback+0xd/0x10
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450300] [<ffffffff8100ad02>] check_events+0x12/0x20
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450303] [<ffffffff8100aca9>] ? xen_irq_enable_direct_reloc+0x4/0x4
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450310] [<ffffffff8165d0a7>] ? do_page_fault+0x1e7/0x550
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450316] [<ffffffff81142d28>] ? do_mmap_pgoff+0x348/0x360
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450322] [<ffffffff8120ff7c>] ? ext4_llseek+0x2c/0x50
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450325] [<ffffffff81659c75>] page_fault+0x25/0x30
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450327] Code: cc 51 41 53 b8 10 00 00 00 0f 05 41 5b 59 c3 cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc 51 4
1 53 b8 11 00 00 00 0f 05 <41> 5b 59 c3 cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450384] Call Trace:
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450387] [<ffffffff8100a53d>] ? xen_force_evtchn_callback+0xd/0x10
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450389] [<ffffffff8100ad02>] check_events+0x12/0x20
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450392] [<ffffffff8100aca9>] ? xen_irq_enable_direct_reloc+0x4/0x4
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450395] [<ffffffff8165d0a7>] ? do_page_fault+0x1e7/0x550
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450397] [<ffffffff81142d28>] ? do_mmap_pgoff+0x348/0x360
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450400] [<ffffffff8120ff7c>] ? ext4_llseek+0x2c/0x50
Aug 29 04:28:45 ip-10-245-130-205 kernel: [11693.450402] [<ffffffff81659c75>] page_fault+0x25/0x30
Aug 29 04:35:28 ip-10-245-130-205 kernel: [12107.159130] BUG: soft lockup - CPU#0 stuck for 29s! [free:17142]
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159149] Modules linked in: isofs acpiphp
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159155] CPU 0
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159156] Modules linked in: isofs acpiphp
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159160]
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159164] Pid: 17142, comm: free Not tainted 3.2.0-52-virtual #78-Ubuntu
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159168] RIP: e030:[<ffffffff8100122a>] [<ffffffff8100122a>] hypercall_page+0x22a/0x1000
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159184] RSP: e02b:ffff880023b799a8 EFLAGS: 00000246
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159186] RAX: 0000000000030004 RBX: ffff880023bb8000 RCX: ffffffff8100122a
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159188] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159189] RBP: ffff880023b799c0 R08: 0000000000000000 R09: 0000000000016470
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159191] R10: 0000000000007ff0 R11: 0000000000000246 R12: 00000001a85fe067
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159193] R13: 0000000000000201 R14: ffff8800238d5564 R15: ffff880023b22000
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159203] FS: 00007fbd0edcc700(0000) GS:ffff880026400000(0000) knlGS:0000000000000000
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159205] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159207] CR2: 0000000000602078 CR3: 0000000023b22000 CR4: 0000000000002660
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159209] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159211] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159214] Process free (pid: 17142, threadinfo ffff880023b78000, task ffff880023f40000)
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159216] Stack:
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159219] 0000000000000000 0000000000000000 ffffffff8100a53d ffff880023b79a78
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159223] ffffffff8100ad02 0000000000000000 0000000000007ff0 0000000000016470
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159226] 0000000000000000 0000000000000201 0000000080000001 0000000000000000
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159229] Call Trace:
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159240] [<ffffffff8100a53d>] ? xen_force_evtchn_callback+0xd/0x10
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159244] [<ffffffff8100ad02>] check_events+0x12/0x20
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159247] [<ffffffff8100acef>] ? xen_restore_fl_direct_reloc+0x4/0x4 [0/139]
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159250] [<ffffffff81007dce>] ? xen_set_pud_hyper+0x7e/0x140
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159253] [<ffffffff81005a88>] ? pte_pfn_to_mfn+0x88/0xa0
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159256] [<ffffffff81007f37>] xen_set_pud+0xa7/0xb0
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159261] [<ffffffff8113cc0d>] __pmd_alloc+0xbd/0xe0
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159264] [<ffffffff8113dd0d>] handle_mm_fault+0x32d/0x370
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159274] [<ffffffff8165d044>] do_page_fault+0x184/0x550
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159279] [<ffffffff81142723>] ? mmap_region+0x233/0x4f0
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159282] [<ffffffff81659c75>] page_fault+0x25/0x30
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159288] [<ffffffff81317e54>] ? __clear_user+0x24/0x50
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159290] [<ffffffff81317ff6>] clear_user+0x36/0x40
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159297] [<ffffffff811c7ccd>] padzero+0x2d/0x40
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159300] [<ffffffff811c987b>] load_elf_binary+0xa4b/0xe80
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159308] [<ffffffff8117e114>] search_binary_handler+0x124/0x330
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159310] [<ffffffff811c8e30>] ? do_mmap+0x40/0x40
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159313] [<ffffffff8117f4dd>] do_execve_common.isra.26+0x2ad/0x340
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159316] [<ffffffff8117f58b>] do_execve+0x1b/0x20
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159321] [<ffffffff8101d597>] sys_execve+0x47/0x70
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159325] [<ffffffff81661f9c>] stub_execve+0x6c/0xc0
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.159327] Code: cc 51 41 53 b8 10 00 00 00 0f 05 41 5b 59 c3 cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc 51 4
1 53 b8 11 00 00 00 0f 05 <41> 5b 59 c3 cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] Call Trace:
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff8100a53d>] ? xen_force_evtchn_callback+0xd/0x10
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff8100ad02>] check_events+0x12/0x20
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff8100acef>] ? xen_restore_fl_direct_reloc+0x4/0x4
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff81007dce>] ? xen_set_pud_hyper+0x7e/0x140
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff81005a88>] ? pte_pfn_to_mfn+0x88/0xa0
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff81007f37>] xen_set_pud+0xa7/0xb0
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff8113cc0d>] __pmd_alloc+0xbd/0xe0
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff8113dd0d>] handle_mm_fault+0x32d/0x370
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff8165d044>] do_page_fault+0x184/0x550
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff81142723>] ? mmap_region+0x233/0x4f0
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff81659c75>] page_fault+0x25/0x30
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff81317e54>] ? __clear_user+0x24/0x50
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff81317ff6>] clear_user+0x36/0x40
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff811c7ccd>] padzero+0x2d/0x40
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff811c987b>] load_elf_binary+0xa4b/0xe80
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff8117e114>] search_binary_handler+0x124/0x330
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff811c8e30>] ? do_mmap+0x40/0x40
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff8117f4dd>] do_execve_common.isra.26+0x2ad/0x340
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff8117f58b>] do_execve+0x1b/0x20
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff8101d597>] sys_execve+0x47/0x70
Aug 29 04:35:29 ip-10-245-130-205 kernel: [12107.163071] [<ffffffff81661f9c>] stub_execve+0x6c/0xc0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment