Skip to content

Instantly share code, notes, and snippets.

/gist:5008615

Created February 21, 2013 21:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save anonymous/5008615 to your computer and use it in GitHub Desktop.
Save anonymous/5008615 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
Feb 19 22:20:02 xxxxx kernel: crond invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0, oom_score_adj=0
Feb 19 22:20:02 xxxxx kernel: crond cpuset=/ mems_allowed=0
Feb 19 22:20:02 xxxxx kernel: Pid: 1972, comm: crond Not tainted 2.6.32-279.14.1.el6.x86_64 #1
Feb 19 22:20:02 xxxxx kernel: Call Trace:
Feb 19 22:20:02 xxxxx kernel: [<ffffffff810c4cd1>] ? cpuset_print_task_mems_allowed+0x91/0xb0
Feb 19 22:20:02 xxxxx kernel: [<ffffffff81117440>] ? dump_header+0x90/0x1b0
Feb 19 22:20:02 xxxxx kernel: [<ffffffff81214c8c>] ? security_real_capable_noaudit+0x3c/0x70
Feb 19 22:20:02 xxxxx kernel: [<ffffffff811178c2>] ? oom_kill_process+0x82/0x2a0
Feb 19 22:20:02 xxxxx kernel: [<ffffffff811177be>] ? select_bad_process+0x9e/0x120
Feb 19 22:20:02 xxxxx kernel: [<ffffffff81117d00>] ? out_of_memory+0x220/0x3c0
Feb 19 22:20:02 xxxxx kernel: [<ffffffff81127a1e>] ? __alloc_pages_nodemask+0x89e/0x940
Feb 19 22:20:02 xxxxx kernel: [<ffffffff8115c7ea>] ? alloc_pages_current+0xaa/0x110
Feb 19 22:20:02 xxxxx kernel: [<ffffffff8112572e>] ? __get_free_pages+0xe/0x50
Feb 19 22:20:02 xxxxx kernel: [<ffffffff810694d4>] ? copy_process+0xe4/0x13c0
Feb 19 22:20:02 xxxxx kernel: [<ffffffff8104457c>] ? __do_page_fault+0x1ec/0x480
Feb 19 22:20:02 xxxxx kernel: [<ffffffff8106a844>] ? do_fork+0x94/0x460
Feb 19 22:20:02 xxxxx kernel: [<ffffffff810d6d42>] ? audit_syscall_entry+0x272/0x2a0
Feb 19 22:20:02 xxxxx kernel: [<ffffffff81009598>] ? sys_clone+0x28/0x30
Feb 19 22:20:02 xxxxx kernel: [<ffffffff8100b413>] ? stub_clone+0x13/0x20
Feb 19 22:20:02 xxxxx kernel: [<ffffffff8100b0f2>] ? system_call_fastpath+0x16/0x1b
Feb 19 22:20:02 xxxxx kernel: Mem-Info:
Feb 19 22:20:02 xxxxx kernel: Node 0 DMA per-cpu:
Feb 19 22:20:02 xxxxx kernel: CPU 0: hi: 0, btch: 1 usd: 0
Feb 19 22:20:02 xxxxx kernel: CPU 1: hi: 0, btch: 1 usd: 0
Feb 19 22:20:02 xxxxx kernel: Node 0 DMA32 per-cpu:
Feb 19 22:20:02 xxxxx kernel: CPU 0: hi: 186, btch: 31 usd: 0
Feb 19 22:20:02 xxxxx kernel: CPU 1: hi: 186, btch: 31 usd: 0
Feb 19 22:20:02 xxxxx kernel: Node 0 Normal per-cpu:
Feb 19 22:20:02 xxxxx kernel: CPU 0: hi: 186, btch: 31 usd: 0
Feb 19 22:20:02 xxxxx kernel: CPU 1: hi: 186, btch: 31 usd: 0
Feb 19 22:20:02 xxxxx kernel: active_anon:699971 inactive_anon:180479 isolated_anon:0
Feb 19 22:20:02 xxxxx kernel: active_file:384 inactive_file:7062 isolated_file:0
Feb 19 22:20:02 xxxxx kernel: unevictable:0 dirty:7079 writeback:0 unstable:0
Feb 19 22:20:02 xxxxx kernel: free:71653 slab_reclaimable:17411 slab_unreclaimable:15982
Feb 19 22:20:02 xxxxx kernel: mapped:401 shmem:8130 pagetables:3739 bounce:0
Feb 19 22:20:02 xxxxx kernel: Node 0 DMA free:15744kB min:248kB low:308kB high:372kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15352kB mlocked:0kB d
irty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Feb 19 22:20:02 xxxxx kernel: lowmem_reserve[]: 0 3511 4016 4016
Feb 19 22:20:02 xxxxx kernel: Node 0 DMA32 free:260164kB min:58864kB low:73580kB high:88296kB active_anon:2598216kB inactive_anon:520116kB active_file:1536kB inactive_file:28248kB unevictable:0kB isolated(anon):0kB isolated(file):0kB pr
esent:3596064kB mlocked:0kB dirty:28316kB writeback:0kB mapped:1600kB shmem:148kB slab_reclaimable:56964kB slab_unreclaimable:36900kB kernel_stack:1040kB pagetables:9800kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unr
eclaimable? no
Feb 19 22:20:02 xxxxx kernel: lowmem_reserve[]: 0 0 505 505
Feb 19 22:20:02 xxxxx kernel: Node 0 Normal free:10704kB min:8464kB low:10580kB high:12696kB active_anon:201668kB inactive_anon:201800kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:517
120kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:32372kB slab_reclaimable:12680kB slab_unreclaimable:27028kB kernel_stack:1160kB pagetables:5156kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Feb 19 22:20:02 xxxxx kernel: lowmem_reserve[]: 0 0 0 0
Feb 19 22:20:02 xxxxx kernel: Node 0 DMA: 2*4kB 1*8kB 1*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15744kB
Feb 19 22:20:02 xxxxx kernel: Node 0 DMA32: 61677*4kB 1685*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 260252kB
Feb 19 22:20:02 xxxxx kernel: Node 0 Normal: 2160*4kB 2*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 10704kB
Feb 19 22:20:02 xxxxx kernel: 15606 total pagecache pages
Feb 19 22:20:02 xxxxx kernel: 0 pages in swap cache
Feb 19 22:20:02 xxxxx kernel: Swap cache stats: add 0, delete 0, find 0/0
Feb 19 22:20:02 xxxxx kernel: Free swap = 0kB
Feb 19 22:20:02 xxxxx kernel: Total swap = 0kB
Feb 19 22:20:02 xxxxx kernel: 1048575 pages RAM
Feb 19 22:20:02 xxxxx kernel: 34843 pages reserved
Feb 19 22:20:02 xxxxx kernel: 8683 pages shared
Feb 19 22:20:02 xxxxx kernel: 930639 pages non-shared
Feb 19 22:20:02 xxxxx kernel: [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Feb 19 22:20:02 xxxxx kernel: [ 836] 0 836 2695 133 1 -17 -1000 udevd
Feb 19 22:20:02 xxxxx kernel: [ 1493] 0 1493 2535 124 1 0 0 dhclient
Feb 19 22:20:02 xxxxx kernel: [ 1537] 0 1537 23548 71 0 -17 -1000 auditd
Feb 19 22:20:02 xxxxx kernel: [ 1562] 0 1562 62526 370 0 0 0 rsyslogd
Feb 19 22:20:02 xxxxx kernel: [ 1574] 0 1574 2286 80 0 0 0 irqbalance
Feb 19 22:20:02 xxxxx kernel: [ 1588] 32 1588 4743 65 0 0 0 rpcbind
Feb 19 22:20:02 xxxxx kernel: [ 1606] 29 1606 5836 113 0 0 0 rpc.statd
Feb 19 22:20:02 xxxxx kernel: [ 1618] 0 1618 1171 38 0 0 0 mdadm
Feb 19 22:20:02 xxxxx kernel: [ 1638] 0 1638 6290 57 0 0 0 rpc.idmapd
Feb 19 22:20:02 xxxxx kernel: [ 1685] 81 1685 5350 75 1 0 0 dbus-daemon
Feb 19 22:20:02 xxxxx kernel: [ 1696] 0 1696 47289 219 0 0 0 cupsd
Feb 19 22:20:02 xxxxx kernel: [ 1708] 0 1708 1019 28 1 0 0 acpid
Feb 19 22:20:02 xxxxx kernel: [ 1717] 68 1717 6326 243 0 0 0 hald
Feb 19 22:20:02 xxxxx kernel: [ 1718] 0 1718 4526 52 0 0 0 hald-runner
Feb 19 22:20:02 xxxxx kernel: [ 1749] 0 1749 5062 44 1 0 0 hald-addon-inpu
Feb 19 22:20:02 xxxxx kernel: [ 1772] 68 1772 4451 37 1 0 0 hald-addon-acpi
Feb 19 22:20:02 xxxxx kernel: [ 1798] 0 1798 113068 639 1 0 0 automount
Feb 19 22:20:02 xxxxx kernel: [ 1830] 0 1830 16018 166 1 -17 -1000 sshd
Feb 19 22:20:02 xxxxx kernel: [ 1867] 0 1867 426264 34387 1 0 0 glusterfsd
Feb 19 22:20:02 xxxxx kernel: [ 1932] 0 1932 19671 215 1 0 0 master
Feb 19 22:20:02 xxxxx kernel: [ 1954] 0 1954 27567 46 0 0 0 abrtd
Feb 19 22:20:02 xxxxx kernel: [ 1962] 0 1962 27016 43 0 0 0 abrt-dump-oops
Feb 19 22:20:02 xxxxx kernel: [ 1971] 89 1971 19734 220 1 0 0 qmgr
Feb 19 22:20:02 xxxxx kernel: [ 1972] 0 1972 29302 189 1 0 0 crond
Feb 19 22:20:02 xxxxx kernel: [ 1983] 0 1983 5618 46 0 0 0 atd
Feb 19 22:20:02 xxxxx kernel: [ 2033] 0 2033 6312 157 1 0 0 smartd
Feb 19 22:20:02 xxxxx kernel: [ 2041] 0 2041 1015 18 0 0 0 mingetty
Feb 19 22:20:02 xxxxx kernel: [ 2043] 0 2043 1015 19 0 0 0 mingetty
Feb 19 22:20:02 xxxxx kernel: [ 2045] 0 2045 1015 19 0 0 0 mingetty
Feb 19 22:20:02 xxxxx kernel: [ 2047] 0 2047 1015 18 1 0 0 mingetty
Feb 19 22:20:02 xxxxx kernel: [ 2049] 0 2049 1015 19 0 0 0 mingetty
Feb 19 22:20:02 xxxxx kernel: [ 2051] 0 2051 1015 18 1 0 0 mingetty
Feb 19 22:20:02 xxxxx kernel: [ 5942] 0 5942 39539 2012 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 5950] 0 5950 39539 2012 1 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 5958] 0 5958 41806 3394 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6009] 0 6009 39539 2012 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6017] 0 6017 39539 2012 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6025] 0 6025 41806 3359 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6068] 0 6068 39539 2012 1 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6076] 0 6076 39539 2013 1 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6084] 0 6084 41806 3359 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6146] 0 6146 39539 2012 1 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6154] 0 6154 39539 2013 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6162] 0 6162 41806 2847 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6186] 0 6186 39539 2012 1 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6194] 0 6194 39539 2012 1 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6202] 0 6202 41806 2847 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6242] 0 6242 39539 2012 1 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6250] 0 6250 39539 2013 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6258] 0 6258 41806 2848 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6282] 0 6282 39539 2013 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6290] 0 6290 39539 2012 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6298] 0 6298 41806 2847 0 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [ 6406] 0 6406 59604 11721 1 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [30271] 0 30271 51363 2029 0 0 0 snmpd
Feb 19 22:20:02 xxxxx kernel: [30366] 0 30366 2694 132 1 -17 -1000 udevd
Feb 19 22:20:02 xxxxx kernel: [30367] 0 30367 2694 132 1 -17 -1000 udevd
Feb 19 22:20:02 xxxxx kernel: [19693] 0 19693 779924 715783 1 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [19714] 0 19714 153478 32996 0 0 0 glusterd
Feb 19 22:20:02 xxxxx kernel: [19741] 0 19741 92787 23184 1 0 0 glusterfs
Feb 19 22:20:02 xxxxx kernel: [11551] 89 11551 19691 210 0 0 0 pickup
Feb 19 22:20:02 xxxxx kernel: Out of memory: Kill process 19693 (glusterfs) score 677 or sacrifice child
Feb 19 22:20:02 xxxxx kernel: Killed process 19693, UID 0, (glusterfs) total-vm:3119696kB, anon-rss:2862148kB, file-rss:984kB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment