Skip to content

Instantly share code, notes, and snippets.

@Jerry-Fix
Created December 17, 2013 14:20
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Jerry-Fix/8005600 to your computer and use it in GitHub Desktop.
Save Jerry-Fix/8005600 to your computer and use it in GitHub Desktop.
nginx dmesg
[32799839.603312] swapper/14: page allocation failure: order:4, mode:0x4020
[32799839.603317] Pid: 0, comm: swapper/14 Tainted: G W 3.2.0-34-generic #53-Ubuntu
[32799839.603320] Call Trace:
[32799839.603322] <IRQ> [<ffffffff8111ceb6>] warn_alloc_failed+0xf6/0x150
[32799839.603336] [<ffffffff81120c57>] __alloc_pages_nodemask+0x6d7/0x8f0
[32799839.603341] [<ffffffff8157646c>] ? inet_getpeer+0x17c/0x2b0
[32799839.603346] [<ffffffff8153c74b>] ? secure_ip_id+0x3b/0x40
[32799839.603351] [<ffffffff816474de>] kmalloc_large_node+0x57/0x85
[32799839.603357] [<ffffffff81165ea5>] __kmalloc_node_track_caller+0x195/0x1e0
[32799839.603360] [<ffffffff81534d8b>] ? __alloc_skb+0x4b/0x240
[32799839.603363] [<ffffffff815355a5>] ? skb_copy+0x45/0xb0
[32799839.603365] [<ffffffff81534db8>] __alloc_skb+0x78/0x240
[32799839.603367] [<ffffffff815355a5>] skb_copy+0x45/0xb0
[32799839.603378] [<ffffffffa002ab05>] tigon3_dma_hwbug_workaround+0x205/0x270 [tg3]
[32799839.603383] [<ffffffff81327919>] ? swiotlb_unmap_page+0x9/0x10
[32799839.603387] [<ffffffffa002ba1a>] tg3_start_xmit+0x42a/0x930 [tg3]
[32799839.603391] [<ffffffff815448e6>] dev_hard_start_xmit+0x256/0x540
[32799839.603395] [<ffffffff815615ce>] sch_direct_xmit+0xfe/0x1d0
[32799839.603398] [<ffffffff81544d07>] dev_queue_xmit+0x137/0x420
[32799839.603401] [<ffffffff8157b6bb>] ip_finish_output+0x16b/0x2f0
[32799839.603404] [<ffffffff8157c228>] ip_output+0x98/0xa0
[32799839.603407] [<ffffffff8157b8e4>] ? __ip_local_out+0xa4/0xb0
[32799839.603410] [<ffffffff8157b919>] ip_local_out+0x29/0x30
[32799839.603413] [<ffffffff8157ba7c>] ip_queue_xmit+0x15c/0x410
[32799839.603418] [<ffffffff81593bf9>] tcp_transmit_skb+0x359/0x580
[32799839.603422] [<ffffffff815946e7>] tcp_write_xmit+0x1d7/0x420
[32799839.603427] [<ffffffff81594996>] __tcp_push_pending_frames+0x26/0x90
[32799839.603432] [<ffffffff81590e9d>] tcp_rcv_established+0x10d/0x7e0
[32799839.603436] [<ffffffff815993ff>] tcp_v4_do_rcv+0xff/0x1d0
[32799839.603440] [<ffffffff8159b021>] tcp_v4_rcv+0x581/0x820
[32799839.603444] [<ffffffff81576980>] ? ip_rcv_finish+0x370/0x370
[32799839.603448] [<ffffffff81576980>] ? ip_rcv_finish+0x370/0x370
[32799839.603452] [<ffffffff81576a5d>] ip_local_deliver_finish+0xdd/0x280
[32799839.603456] [<ffffffff81576dc8>] ip_local_deliver+0x88/0x90
[32799839.603460] [<ffffffff8157671d>] ip_rcv_finish+0x10d/0x370
[32799839.603464] [<ffffffff81577005>] ip_rcv+0x235/0x300
[32799839.603469] [<ffffffff815423c6>] __netif_receive_skb+0x4d6/0x550
[32799839.603473] [<ffffffff815431b0>] netif_receive_skb+0x80/0x90
[32799839.603477] [<ffffffff815435b9>] ? dev_gro_receive+0x1b9/0x2c0
[32799839.603481] [<ffffffff81543300>] napi_skb_finish+0x50/0x70
[32799839.603484] [<ffffffff81543905>] napi_gro_receive+0xf5/0x140
[32799839.603491] [<ffffffffa0028519>] tg3_rx+0x349/0x5c0 [tg3]
[32799839.603494] [<ffffffff815357ed>] ? consume_skb+0x3d/0xb0
[32799839.603498] [<ffffffffa00282ca>] ? tg3_rx+0xfa/0x5c0 [tg3]
[32799839.603502] [<ffffffffa0028809>] tg3_poll_work+0x79/0x190 [tg3]
[32799839.603506] [<ffffffffa0028960>] tg3_poll_msix+0x40/0xd0 [tg3]
[32799839.603509] [<ffffffff81543b54>] net_rx_action+0x134/0x290
[32799839.603513] [<ffffffff810561d0>] ? finish_task_switch+0x50/0xf0
[32799839.603518] [<ffffffff8106e6e8>] __do_softirq+0xa8/0x210
[32799839.603522] [<ffffffff8165bdce>] ? _raw_spin_lock+0xe/0x20
[32799839.603527] [<ffffffff8166662c>] call_softirq+0x1c/0x30
[32799839.603533] [<ffffffff81015305>] do_softirq+0x65/0xa0
[32799839.603536] [<ffffffff8106eace>] irq_exit+0x8e/0xb0
[32799839.603540] [<ffffffff81666ee3>] do_IRQ+0x63/0xe0
[32799839.603543] [<ffffffff8165c26e>] common_interrupt+0x6e/0x6e
[32799839.603544] <EOI> [<ffffffff810561d0>] ? finish_task_switch+0x50/0xf0
[32799839.603549] [<ffffffff810561ca>] ? finish_task_switch+0x4a/0xf0
[32799839.603554] [<ffffffff8165986c>] __schedule+0x3cc/0x6f0
[32799839.603558] [<ffffffff81659ebf>] schedule+0x3f/0x60
[32799839.603560] [<ffffffff81012268>] cpu_idle+0x108/0x120
[32799839.603564] [<ffffffff81639882>] start_secondary+0xd9/0xdb
[32799839.603566] Mem-Info:
[32799839.603567] Node 0 DMA per-cpu:
[32799839.603569] CPU 0: hi: 0, btch: 1 usd: 0
[32799839.603571] CPU 1: hi: 0, btch: 1 usd: 0
[32799839.603573] CPU 2: hi: 0, btch: 1 usd: 0
[32799839.603574] CPU 3: hi: 0, btch: 1 usd: 0
[32799839.603576] CPU 4: hi: 0, btch: 1 usd: 0
[32799839.603577] CPU 5: hi: 0, btch: 1 usd: 0
[32799839.603579] CPU 6: hi: 0, btch: 1 usd: 0
[32799839.603581] CPU 7: hi: 0, btch: 1 usd: 0
[32799839.603582] CPU 8: hi: 0, btch: 1 usd: 0
[32799839.603584] CPU 9: hi: 0, btch: 1 usd: 0
[32799839.603585] CPU 10: hi: 0, btch: 1 usd: 0
[32799839.603587] CPU 11: hi: 0, btch: 1 usd: 0
[32799839.603588] CPU 12: hi: 0, btch: 1 usd: 0
[32799839.603590] CPU 13: hi: 0, btch: 1 usd: 0
[32799839.603592] CPU 14: hi: 0, btch: 1 usd: 0
[32799839.603593] CPU 15: hi: 0, btch: 1 usd: 0
[32799839.603595] CPU 16: hi: 0, btch: 1 usd: 0
[32799839.603596] CPU 17: hi: 0, btch: 1 usd: 0
[32799839.603598] CPU 18: hi: 0, btch: 1 usd: 0
[32799839.603599] CPU 19: hi: 0, btch: 1 usd: 0
[32799839.603601] CPU 20: hi: 0, btch: 1 usd: 0
[32799839.603602] CPU 21: hi: 0, btch: 1 usd: 0
[32799839.603604] CPU 22: hi: 0, btch: 1 usd: 0
[32799839.603605] CPU 23: hi: 0, btch: 1 usd: 0
[32799839.603607] Node 0 DMA32 per-cpu:
[32799839.603609] CPU 0: hi: 186, btch: 31 usd: 35
[32799839.603611] CPU 1: hi: 186, btch: 31 usd: 180
[32799839.603612] CPU 2: hi: 186, btch: 31 usd: 18
[32799839.603615] CPU 3: hi: 186, btch: 31 usd: 158
[32799839.603617] CPU 4: hi: 186, btch: 31 usd: 165
[32799839.603619] CPU 5: hi: 186, btch: 31 usd: 166
[32799839.603621] CPU 6: hi: 186, btch: 31 usd: 15
[32799839.603623] CPU 7: hi: 186, btch: 31 usd: 163
[32799839.603626] CPU 8: hi: 186, btch: 31 usd: 45
[32799839.603628] CPU 9: hi: 186, btch: 31 usd: 184
[32799839.603631] CPU 10: hi: 186, btch: 31 usd: 49
[32799839.603633] CPU 11: hi: 186, btch: 31 usd: 184
[32799839.603636] CPU 12: hi: 186, btch: 31 usd: 86
[32799839.603638] CPU 13: hi: 186, btch: 31 usd: 9
[32799839.603640] CPU 14: hi: 186, btch: 31 usd: 35
[32799839.603642] CPU 15: hi: 186, btch: 31 usd: 2
[32799839.603644] CPU 16: hi: 186, btch: 31 usd: 30
[32799839.603646] CPU 17: hi: 186, btch: 31 usd: 2
[32799839.603648] CPU 18: hi: 186, btch: 31 usd: 36
[32799839.603650] CPU 19: hi: 186, btch: 31 usd: 4
[32799839.603652] CPU 20: hi: 186, btch: 31 usd: 72
[32799839.603654] CPU 21: hi: 186, btch: 31 usd: 12
[32799839.603657] CPU 22: hi: 186, btch: 31 usd: 29
[32799839.603659] CPU 23: hi: 186, btch: 31 usd: 4
[32799839.603661] Node 0 Normal per-cpu:
[32799839.603663] CPU 0: hi: 186, btch: 31 usd: 46
[32799839.603664] CPU 1: hi: 186, btch: 31 usd: 181
[32799839.603666] CPU 2: hi: 186, btch: 31 usd: 91
[32799839.603668] CPU 3: hi: 186, btch: 31 usd: 170
[32799839.603669] CPU 4: hi: 186, btch: 31 usd: 175
[32799839.603671] CPU 5: hi: 186, btch: 31 usd: 185
[32799839.603672] CPU 6: hi: 186, btch: 31 usd: 80
[32799839.603674] CPU 7: hi: 186, btch: 31 usd: 159
[32799839.603675] CPU 8: hi: 186, btch: 31 usd: 105
[32799839.603677] CPU 9: hi: 186, btch: 31 usd: 156
[32799839.603679] CPU 10: hi: 186, btch: 31 usd: 86
[32799839.603680] CPU 11: hi: 186, btch: 31 usd: 170
[32799839.603682] CPU 12: hi: 186, btch: 31 usd: 94
[32799839.603683] CPU 13: hi: 186, btch: 31 usd: 3
[32799839.603685] CPU 14: hi: 186, btch: 31 usd: 78
[32799839.603687] CPU 15: hi: 186, btch: 31 usd: 0
[32799839.603688] CPU 16: hi: 186, btch: 31 usd: 67
[32799839.603690] CPU 17: hi: 186, btch: 31 usd: 1
[32799839.603691] CPU 18: hi: 186, btch: 31 usd: 91
[32799839.603693] CPU 19: hi: 186, btch: 31 usd: 5
[32799839.603694] CPU 20: hi: 186, btch: 31 usd: 37
[32799839.603696] CPU 21: hi: 186, btch: 31 usd: 1
[32799839.603697] CPU 22: hi: 186, btch: 31 usd: 22
[32799839.603699] CPU 23: hi: 186, btch: 31 usd: 0
[32799839.603701] Node 1 Normal per-cpu:
[32799839.603702] CPU 0: hi: 186, btch: 31 usd: 173
[32799839.603704] CPU 1: hi: 186, btch: 31 usd: 155
[32799839.603706] CPU 2: hi: 186, btch: 31 usd: 160
[32799839.603707] CPU 3: hi: 186, btch: 31 usd: 154
[32799839.603709] CPU 4: hi: 186, btch: 31 usd: 181
[32799839.603710] CPU 5: hi: 186, btch: 31 usd: 156
[32799839.603712] CPU 6: hi: 186, btch: 31 usd: 171
[32799839.603713] CPU 7: hi: 186, btch: 31 usd: 151
[32799839.603715] CPU 8: hi: 186, btch: 31 usd: 172
[32799839.603717] CPU 9: hi: 186, btch: 31 usd: 150
[32799839.603718] CPU 10: hi: 186, btch: 31 usd: 182
[32799839.603720] CPU 11: hi: 186, btch: 31 usd: 159
[32799839.603721] CPU 12: hi: 186, btch: 31 usd: 170
[32799839.603723] CPU 13: hi: 186, btch: 31 usd: 147
[32799839.603725] CPU 14: hi: 186, btch: 31 usd: 179
[32799839.603727] CPU 15: hi: 186, btch: 31 usd: 170
[32799839.603729] CPU 16: hi: 186, btch: 31 usd: 159
[32799839.603730] CPU 17: hi: 186, btch: 31 usd: 162
[32799839.603732] CPU 18: hi: 186, btch: 31 usd: 170
[32799839.603734] CPU 19: hi: 186, btch: 31 usd: 144
[32799839.603735] CPU 20: hi: 186, btch: 31 usd: 184
[32799839.603737] CPU 21: hi: 186, btch: 31 usd: 152
[32799839.603738] CPU 22: hi: 186, btch: 31 usd: 172
[32799839.603740] CPU 23: hi: 186, btch: 31 usd: 151
[32799839.603744] active_anon:1054334 inactive_anon:85394 isolated_anon:0
[32799839.603745] active_file:1070595 inactive_file:4449563 isolated_file:0
[32799839.603746] unevictable:0 dirty:428 writeback:0 unstable:0
[32799839.603747] free:641785 slab_reclaimable:213029 slab_unreclaimable:508854
[32799839.603747] mapped:2785 shmem:378 pagetables:4244 bounce:0
[32799839.603749] Node 0 DMA free:15900kB min:40kB low:48kB high:60kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15644kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[32799839.603757] lowmem_reserve[]: 0 3171 16023 16023
[32799839.603761] Node 0 DMA32 free:214156kB min:8880kB low:11100kB high:13320kB active_anon:330172kB inactive_anon:61040kB active_file:202784kB inactive_file:1335192kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3247232kB mlocked:0kB dirty:80kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:139112kB slab_unreclaimable:882724kB kernel_stack:216kB pagetables:2676kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[32799839.603769] lowmem_reserve[]: 0 0 12852 12852
[32799839.603772] Node 0 Normal free:44708kB min:36004kB low:45004kB high:54004kB active_anon:2021776kB inactive_anon:141100kB active_file:1089696kB inactive_file:8431668kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:13160448kB mlocked:0kB dirty:292kB writeback:0kB mapped:736kB shmem:32kB slab_reclaimable:239996kB slab_unreclaimable:850120kB kernel_stack:1112kB pagetables:5424kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[32799839.603780] lowmem_reserve[]: 0 0 0 0
[32799839.603783] Node 1 Normal free:2292376kB min:45180kB low:56472kB high:67768kB active_anon:1865388kB inactive_anon:139436kB active_file:2989900kB inactive_file:8031392kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:16515072kB mlocked:0kB dirty:1340kB writeback:0kB mapped:10400kB shmem:1480kB slab_reclaimable:473008kB slab_unreclaimable:302572kB kernel_stack:664kB pagetables:8876kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no
[32799839.603791] lowmem_reserve[]: 0 0 0 0
[32799839.603793] Node 0 DMA: 1*4kB 1*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15900kB
[32799839.603800] Node 0 DMA32: 11018*4kB 10048*8kB 5096*16kB 245*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 213832kB
[32799839.603808] Node 0 Normal: 10350*4kB 0*8kB 0*16kB 0*32kB 2*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 45112kB
[32799839.603817] Node 1 Normal: 135037*4kB 218606*8kB 8*16kB 1*32kB 0*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 2292996kB
[32799839.603828] 5524434 total pagecache pages
[32799839.603830] 4076 pages in swap cache
[32799839.603832] Swap cache stats: add 5220230, delete 5216154, find 808330747/808800864
[32799839.603834] Free swap = 15590308kB
[32799839.603835] Total swap = 15624188kB
[32799839.722542] 8388592 pages RAM
[32799839.722544] 171382 pages reserved
[32799839.722545] 2282468 pages shared
[32799839.722546] 5286962 pages non-shared
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment