Skip to content

Instantly share code, notes, and snippets.

@TvL2386

TvL2386/gist:6dd00e62bf6a1c7afc373df9fc03c226 Secret

Created March 8, 2017 14:21
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save TvL2386/6dd00e62bf6a1c7afc373df9fc03c226 to your computer and use it in GitHub Desktop.
Save TvL2386/6dd00e62bf6a1c7afc373df9fc03c226 to your computer and use it in GitHub Desktop.
Download ZIP
oom-killer kills glusterfsd :'(
Raw
gistfile1.txt
[4823047.872481] mkinitramfs invoked oom-killer: gfp_mask=0x26000c0, order=2, oom_score_adj=0
[4823047.872484] mkinitramfs cpuset=/ mems_allowed=0
[4823047.872488] CPU: 3 PID: 29999 Comm: mkinitramfs Not tainted 4.4.0-59-generic #80-Ubuntu
[4823047.872490] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
[4823047.872491] 0000000000000286 000000004c3580ff ffff88002f22baf0 ffffffff813f7583
[4823047.872493] ffff88002f22bcc8 ffff8800b8a41c00 ffff88002f22bb60 ffffffff8120ad5e
[4823047.872495] ffff88013fd9a870 ffff88013fd9a860 ffffea00045cc340 0000000100000001
[4823047.872497] Call Trace:
[4823047.872503] [<ffffffff813f7583>] dump_stack+0x63/0x90
[4823047.872506] [<ffffffff8120ad5e>] dump_header+0x5a/0x1c5
[4823047.872510] [<ffffffff81192722>] oom_kill_process+0x202/0x3c0
[4823047.872511] [<ffffffff81192b49>] out_of_memory+0x219/0x460
[4823047.872514] [<ffffffff81198abd>] __alloc_pages_slowpath.constprop.88+0x8fd/0xa70
[4823047.872516] [<ffffffff81198eb6>] __alloc_pages_nodemask+0x286/0x2a0
[4823047.872518] [<ffffffff81198f6b>] alloc_kmem_pages_node+0x4b/0xc0
[4823047.872522] [<ffffffff8107ea5e>] copy_process+0x1be/0x1b70
[4823047.872525] [<ffffffff811c1670>] ? handle_mm_fault+0xce0/0x1820
[4823047.872527] [<ffffffff810805a0>] _do_fork+0x80/0x360
[4823047.872529] [<ffffffff81080929>] SyS_clone+0x19/0x20
[4823047.872532] [<ffffffff818384f2>] entry_SYSCALL_64_fastpath+0x16/0x71
[4823047.872534] Mem-Info:
[4823047.872537] active_anon:34308 inactive_anon:71558 isolated_anon:0
active_file:331279 inactive_file:238343 isolated_file:0
unevictable:914 dirty:19170 writeback:0 unstable:0
slab_reclaimable:185084 slab_unreclaimable:42456
mapped:19748 shmem:8974 pagetables:1414 bounce:0
free:35853 free_pcp:0 free_cma:0
[4823047.872540] Node 0 DMA free:15740kB min:268kB low:332kB high:400kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[4823047.872544] lowmem_reserve[]: 0 2938 3896 3896 3896
[4823047.872546] Node 0 DMA32 free:99200kB min:50752kB low:63440kB high:76128kB active_anon:100108kB inactive_anon:206956kB active_file:1068552kB inactive_file:704504kB unevictable:3544kB isolated(anon):0kB isolated(file):0kB present:3129280kB managed:3048660kB mlocked:3544kB dirty:41812kB writeback:0kB mapped:63680kB shmem:31588kB slab_reclaimable:546908kB slab_unreclaimable:117688kB kernel_stack:3024kB pagetables:4424kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[4823047.872550] lowmem_reserve[]: 0 0 958 958 958
[4823047.872552] Node 0 Normal free:28472kB min:16560kB low:20700kB high:24840kB active_anon:37124kB inactive_anon:79276kB active_file:256564kB inactive_file:248868kB unevictable:112kB isolated(anon):0kB isolated(file):0kB present:1048576kB managed:981744kB mlocked:112kB dirty:34868kB writeback:0kB mapped:15312kB shmem:4308kB slab_reclaimable:193428kB slab_unreclaimable:52128kB kernel_stack:1632kB pagetables:1232kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:12 all_unreclaimable? no
[4823047.872555] lowmem_reserve[]: 0 0 0 0 0
[4823047.872557] Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 1*32kB (U) 1*64kB (U) 0*128kB 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15740kB
[4823047.872564] Node 0 DMA32: 17623*4kB (UME) 3624*8kB (UME) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 99484kB
[4823047.872569] Node 0 Normal: 5998*4kB (UME) 585*8kB (UME) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 28672kB
[4823047.872575] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[4823047.872576] 581499 total pagecache pages
[4823047.872577] 2223 pages in swap cache
[4823047.872578] Swap cache stats: add 84703, delete 82480, find 4164/6427
[4823047.872579] Free swap = 1989428kB
[4823047.872579] Total swap = 2097148kB
[4823047.872580] 1048462 pages RAM
[4823047.872581] 0 pages HighMem/MovableOnly
[4823047.872581] 36884 pages reserved
[4823047.872582] 0 pages cma reserved
[4823047.872583] 0 pages hwpoisoned
[4823047.872583] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
[4823047.872588] [ 432] 0 432 10189 2809 24 3 0 0 systemd-journal
[4823047.872590] [ 456] 0 456 25742 341 18 3 0 0 lvmetad
[4823047.872592] [ 488] 0 488 11368 1110 25 3 38 -1000 systemd-udevd
[4823047.872594] [ 1124] 0 1124 86966 1467 32 5 254 0 snapd
[4823047.872595] [ 1130] 0 1130 7252 668 20 3 0 0 cron
[4823047.872596] [ 1132] 0 1132 6511 522 18 3 0 0 atd
[4823047.872598] [ 1134] 0 1134 31420 3532 52 3 1546 0 vmtoolsd
[4823047.872599] [ 1136] 106 1136 10725 899 26 3 0 -900 dbus-daemon
[4823047.872601] [ 1144] 104 1144 64099 908 27 3 3 0 rsyslogd
[4823047.872602] [ 1149] 0 1149 69111 1402 38 3 599 0 accounts-daemon
[4823047.872603] [ 1155] 0 1155 7165 752 19 3 3 0 systemd-logind
[4823047.872605] [ 1158] 0 1158 206507 1035 34 4 168 0 lxcfs
[4823047.872606] [ 1162] 0 1162 1100 306 8 3 0 0 acpid
[4823047.872607] [ 1208] 112 1208 1884 269 9 3 0 0 vnstatd
[4823047.872609] [ 1215] 0 1215 16380 1172 37 3 44 -1000 sshd
[4823047.872610] [ 1228] 0 1228 3344 37 11 3 0 0 mdadm
[4823047.872611] [ 1239] 0 1239 1306 30 8 3 0 0 iscsid
[4823047.872612] [ 1240] 0 1240 1431 884 8 3 0 -17 iscsid
[4823047.872614] [ 1254] 0 1254 69272 1334 38 3 1 0 polkitd
[4823047.872615] [ 1335] 0 1335 3985 393 13 3 0 0 agetty
[4823047.872616] [ 1346] 0 1346 4906 477 15 3 7 0 irqbalance
[4823047.872618] [ 1349] 114 1349 10067 893 22 3 2 0 ntpd
[4823047.872619] [ 1351] 0 1351 180182 4937 73 3 3681 0 glusterd
[4823047.872620] [ 1359] 111 1359 14875 1608 31 3 232 0 snmpd
[4823047.872622] [ 1753] 0 1753 226666 14978 109 4 6056 0 glusterfsd
[4823047.872623] [ 8846] 1000 8846 11342 919 27 3 155 0 systemd
[4823047.872625] [ 8851] 1000 8851 15389 472 33 3 97 0 (sd-pam)
[4823047.872626] [ 8999] 0 8999 444433 27990 154 5 4918 0 glusterfsd
[4823047.872628] [ 9221] 0 9221 174175 9088 63 3 923 0 glusterfs
[4823047.872629] [ 9308] 0 9308 288904 18707 117 4 6624 0 glusterfs
[4823047.872630] [ 9322] 0 9322 6833 524 16 3 121 0 screen
[4823047.872631] [ 9323] 0 9323 5330 875 15 3 105 0 bash
[4823047.872633] [27998] 0 27998 1127 430 8 3 0 0 apt.systemd.dai
[4823047.872635] [28346] 0 28346 40679 24875 84 3 0 0 unattended-upgr
[4823047.872636] [28351] 0 28351 40670 19302 81 3 0 0 unattended-upgr
[4823047.872637] [28442] 0 28442 5050 1113 16 3 0 0 dpkg
[4823047.872639] [28443] 0 28443 6176 1562 17 3 0 0 linux-image-4.4
[4823047.872640] [28470] 0 28470 1092 167 8 4 0 0 run-parts
[4823047.872641] [28522] 0 28522 1127 189 8 3 0 0 initramfs-tools
[4823047.872643] [28524] 0 28524 1127 175 8 3 0 0 update-initramf
[4823047.872644] [28526] 0 28526 1157 480 8 3 0 0 mkinitramfs
[4823047.872645] [29999] 0 29999 1157 321 7 3 0 0 mkinitramfs
[4823047.872647] Out of memory: Kill process 8999 (glusterfsd) score 20 or sacrifice child
[4823047.872719] Killed process 8999 (glusterfsd) total-vm:1777732kB, anon-rss:108108kB, file-rss:3852kB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment