Skip to content

Instantly share code, notes, and snippets.

@mattstauffer

mattstauffer/syslog.log Secret

Last active August 29, 2015 14:16
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mattstauffer/59e74d0a6b57c7999689 to your computer and use it in GitHub Desktop.
Save mattstauffer/59e74d0a6b57c7999689 to your computer and use it in GitHub Desktop.
Download ZIP
Java OOM ElasticSearch
Raw
syslog.log
kernel: [65905.924782] java invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
kernel: [65905.924810] java cpuset=/ mems_allowed=0
kernel: [65905.924822] CPU: 0 PID: 1429 Comm: java Not tainted 3.13.0-43-generic #72-Ubuntu
kernel: [65905.924824] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
kernel: [65905.924828] 0000000000000000 ffff88001f897968 ffffffff81720bf6 ffff88001bc2c800
kernel: [65905.924840] ffff88001f8979f0 ffffffff8171b4b1 ffffffff81067986 ffff88001f8979c8
kernel: [65905.924842] ffffffff810c794c 0000000000000002 ffff88001fffae28 0000000000000000
kernel: [65905.924844] Call Trace:
kernel: [65905.924878] [<ffffffff81720bf6>] dump_stack+0x45/0x56
kernel: [65905.924884] [<ffffffff8171b4b1>] dump_header+0x7f/0x1f1
kernel: [65905.924902] [<ffffffff81067986>] ? put_online_cpus+0x56/0x80
kernel: [65905.924912] [<ffffffff810c794c>] ? rcu_oom_notify+0xcc/0xf0
kernel: [65905.924922] [<ffffffff811526de>] oom_kill_process+0x1ce/0x330
kernel: [65905.924935] [<ffffffff812d6ce5>] ? security_capable_noaudit+0x15/0x20
kernel: [65905.924937] [<ffffffff81152e14>] out_of_memory+0x414/0x450
kernel: [65905.924940] [<ffffffff81159180>] __alloc_pages_nodemask+0xa60/0xb80
kernel: [65905.924949] [<ffffffff811977a3>] alloc_pages_current+0xa3/0x160
kernel: [65905.924954] [<ffffffff8114f297>] __page_cache_alloc+0x97/0xc0
kernel: [65905.924957] [<ffffffff81150ca5>] filemap_fault+0x185/0x410
kernel: [65905.924962] [<ffffffff81175b4f>] __do_fault+0x6f/0x530
kernel: [65905.924965] [<ffffffff81179d12>] handle_mm_fault+0x482/0xf00
kernel: [65905.924972] [<ffffffff810d7e28>] ? get_futex_key+0x1d8/0x2c0
kernel: [65905.924977] [<ffffffff8172cc14>] __do_page_fault+0x184/0x560
kernel: [65905.924980] [<ffffffff810db15a>] ? do_futex+0x10a/0x760
kernel: [65905.924988] [<ffffffff810a0255>] ? set_next_entity+0x95/0xb0
kernel: [65905.924997] [<ffffffff81012609>] ? __switch_to+0x169/0x4c0
kernel: [65905.925000] [<ffffffff8172d00a>] do_page_fault+0x1a/0x70
kernel: [65905.925002] [<ffffffff8172c689>] do_async_page_fault+0x29/0xe0
kernel: [65905.925004] [<ffffffff81729498>] async_page_fault+0x28/0x30
kernel: [65905.925006] Mem-Info:
kernel: [65905.925010] Node 0 DMA per-cpu:
kernel: [65905.925015] CPU 0: hi: 0, btch: 1 usd: 0
kernel: [65905.925016] Node 0 DMA32 per-cpu:
kernel: [65905.925018] CPU 0: hi: 186, btch: 31 usd: 131
kernel: [65905.925021] active_anon:99489 inactive_anon:15072 isolated_anon:0
kernel: [65905.925021] active_file:66 inactive_file:77 isolated_file:0
kernel: [65905.925021] unevictable:0 dirty:0 writeback:0 unstable:0
kernel: [65905.925021] free:1238 slab_reclaimable:2140 slab_unreclaimable:2401
kernel: [65905.925021] mapped:14906 shmem:15164 pagetables:1616 bounce:0
kernel: [65905.925021] free_cma:0
kernel: [65905.925024] Node 0 DMA free:1972kB min:88kB low:108kB high:132kB active_anon:11376kB inactive_anon:1688kB active_file:4kB inactive_file:4kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:1684kB shmem:1720kB slab_reclaimable:128kB slab_unreclaimable:264kB kernel_stack:64kB pagetables:208kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:12 all_unreclaimable? yes
kernel: [65905.925031] lowmem_reserve[]: 0 471 471 471
kernel: [65905.925035] Node 0 DMA32 free:2980kB min:2732kB low:3412kB high:4096kB active_anon:386580kB inactive_anon:58600kB active_file:260kB inactive_file:304kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:507896kB managed:485892kB mlocked:0kB dirty:0kB writeback:0kB mapped:57940kB shmem:58936kB slab_reclaimable:8432kB slab_unreclaimable:9340kB kernel_stack:1208kB pagetables:6256kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:958 all_unreclaimable? yes
kernel: [65905.925038] lowmem_reserve[]: 0 0 0 0
kernel: [65905.925040] Node 0 DMA: 3*4kB (MR) 1*8kB (R) 0*16kB 1*32kB (R) 0*64kB 1*128kB (R) 1*256kB (R) 1*512kB (R) 1*1024kB (R) 0*2048kB 0*4096kB = 1972kB
kernel: [65905.925060] Node 0 DMA32: 281*4kB (UEM) 24*8kB (EM) 4*16kB (M) 0*32kB 1*64kB (R) 2*128kB (R) 1*256kB (R) 0*512kB 1*1024kB (R) 0*2048kB 0*4096kB = 2980kB
kernel: [65905.925074] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
kernel: [65905.925075] 15310 total pagecache pages
kernel: [65905.925081] 0 pages in swap cache
kernel: [65905.925082] Swap cache stats: add 0, delete 0, find 0/0
kernel: [65905.925083] Free swap = 0kB
kernel: [65905.925084] Total swap = 0kB
kernel: [65905.925085] 130972 pages RAM
kernel: [65905.925085] 0 pages HighMem/MovableOnly
kernel: [65905.925086] 5501 pages reserved
kernel: [65905.925087] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name
kernel: [65905.925091] [ 309] 0 309 4869 47 13 0 0 upstart-udev-br
kernel: [65905.925093] [ 322] 0 322 12839 171 27 0 -1000 systemd-udevd
kernel: [65905.925095] [ 559] 0 559 3815 55 12 0 0 upstart-socket-
kernel: [65905.925097] [ 837] 102 837 9807 96 23 0 0 dbus-daemon
kernel: [65905.925099] [ 888] 0 888 10863 90 26 0 0 systemd-logind
kernel: [65905.925100] [ 893] 101 893 63961 328 28 0 0 rsyslogd
kernel: [65905.925102] [ 918] 0 918 3819 65 12 0 0 upstart-file-br
kernel: [65905.925104] [ 994] 0 994 3955 41 13 0 0 getty
kernel: [65905.925105] [ 997] 0 997 3955 42 13 0 0 getty
kernel: [65905.925107] [ 1009] 0 1009 3955 39 13 0 0 getty
kernel: [65905.925108] [ 1010] 0 1010 3955 39 13 0 0 getty
kernel: [65905.925110] [ 1013] 0 1013 3955 39 13 0 0 getty
kernel: [65905.925115] [ 1041] 0 1041 15341 171 34 0 -1000 sshd
kernel: [65905.925116] [ 1048] 0 1048 1092 37 8 0 0 acpid
kernel: [65905.925118] [ 1050] 0 1050 5914 64 17 0 0 cron
kernel: [65905.925119] [ 1051] 0 1051 4785 41 12 0 0 atd
kernel: [65905.925121] [ 1053] 108 1053 8797 238 13 0 0 redis-server
kernel: [65905.925123] [ 1086] 0 1086 85750 2725 120 0 0 php5-fpm
kernel: [65905.925125] [ 1093] 106 1093 139595 9125 64 0 0 mysqld
kernel: [65905.925126] [ 1116] 1000 1116 104631 13388 141 0 0 php5-fpm
kernel: [65905.925128] [ 1117] 1000 1117 105356 12483 140 0 0 php5-fpm
kernel: [65905.925130] [ 1141] 107 1141 62650 3019 59 0 -900 postgres
kernel: [65905.925132] [ 1144] 107 1144 62650 360 49 0 0 postgres
kernel: [65905.925133] [ 1145] 107 1145 62650 611 51 0 0 postgres
kernel: [65905.925135] [ 1146] 107 1146 62650 360 49 0 0 postgres
kernel: [65905.925136] [ 1147] 107 1147 62751 436 52 0 0 postgres
kernel: [65905.925138] [ 1148] 107 1148 26436 362 47 0 0 postgres
kernel: [65905.925140] [ 1292] 110 1292 1118 23 8 0 0 beanstalkd
kernel: [65905.925141] [ 1315] 109 1315 81349 119 24 0 0 memcached
kernel: [65905.925143] [ 1330] 0 1330 22602 321 42 0 0 nginx
kernel: [65905.925144] [ 1332] 1000 1332 22757 476 42 0 0 nginx
kernel: [65905.925146] [ 1344] 0 1344 15130 2552 32 0 0 supervisord
kernel: [65905.925148] [ 1410] 111 1410 464567 53090 154 0 0 java
kernel: [65905.925150] [ 1451] 0 1451 66389 1775 32 0 0 fail2ban-server
kernel: [65905.925158] [ 1504] 0 1504 3955 40 13 0 0 getty
kernel: [65905.925160] [ 6770] 0 6770 14910 113 34 0 0 cron
kernel: [65905.925162] [ 6771] 0 6771 1111 25 7 0 0 sh
kernel: [65905.925163] [ 6772] 0 6772 1084 28 8 0 0 run-parts
kernel: [65905.925165] [ 6776] 0 6776 1111 37 6 0 0 apt
kernel: [65905.925166] [ 6900] 0 6900 38309 24331 79 0 0 update-apt-xapi
kernel: [65905.925168] [ 6901] 107 6901 62754 435 53 0 0 postgres
kernel: [65905.925171] Out of memory: Kill process 1410 (java) score 424 or sacrifice child
kernel: [65905.925354] Killed process 1410 (java) total-vm:1858268kB, anon-rss:212360kB, file-rss:0kB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment