Skip to content

Instantly share code, notes, and snippets.

@bassu
Created November 15, 2013 15:59
Show Gist options
  • Save bassu/7486695 to your computer and use it in GitHub Desktop.
Save bassu/7486695 to your computer and use it in GitHub Desktop.
ZOL Out of RAM
Out of memory: Kill process 2813 (sshd) score 1 or sacrifice child
Killed process 2815 (bash) total-vm:108572kB, anon-rss:0kB, file-rss:716kB
init invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
init cpuset=/ mems_allowed=0
Pid: 1, comm: init Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff8115a126>] alloc_pages_vma+0xc6/0x1a0
[<ffffffff81113f90>] ? sleep_on_page+0x20/0x20
[<ffffffff8114d7d2>] read_swap_cache_async+0xf2/0x170
[<ffffffff8114d8de>] swapin_readahead+0x8e/0xd0
[<ffffffff8113d415>] do_swap_page+0x115/0x630
[<ffffffff8113ce5e>] ? do_wp_page+0x38e/0x830
[<ffffffff8100556d>] ? pte_mfn_to_pfn+0x7d/0x100
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e07a>] handle_pte_fault+0x1aa/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff81009e6d>] ? xen_force_evtchn_callback+0xd/0x10
[<ffffffff8100a602>] ? check_events+0x12/0x20
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff81056898>] ? sys_waitid+0xa8/0x1f0
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:0 isolated_file:0
unevictable:645 dirty:0 writeback:0 unstable:0
free:26434 slab_reclaimable:1588 slab_unreclaimable:315909
mapped:509 shmem:0 pagetables:403 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:77832kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:48kB slab_unreclaimable:502196kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:7264 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:12012kB min:16488kB low:20608kB high:24732kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:2580kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2580kB dirty:0kB writeback:0kB mapped:2032kB shmem:0kB slab_reclaimable:6304kB slab_unreclaimable:761440kB kernel_stack:1840kB pagetables:1612kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 20*4kB 34*8kB 58*16kB 19*32kB 19*64kB 11*128kB 10*256kB 5*512kB 7*1024kB 12*2048kB 9*4096kB = 78240kB
Node 0 Normal: 1100*4kB 319*8kB 9*16kB 9*32kB 6*64kB 6*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 12120kB
544 total pagecache pages
0 pages in swap cache
Swap cache stats: add 39398, delete 39398, find 11929/16275
Free swap = 15610296kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
2125 pages shared
4919668 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 73 0 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 155 3 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 0 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 150 1 0 0 python
Out of memory: Kill process 2813 (sshd) score 1 or sacrifice child
Killed process 2813 (sshd) total-vm:98120kB, anon-rss:0kB, file-rss:804kB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment