Skip to content

Instantly share code, notes, and snippets.

@bassu
Last active December 28, 2015 10:49
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bassu/7489357 to your computer and use it in GitHub Desktop.
Save bassu/7489357 to your computer and use it in GitHub Desktop.
ZOL OOM Log for Behlendorf. The spl/slab and arcstat concatenation is at end of this file
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 3.4.61-9.el6.centos.alt.x86_64 (mockbuild@bn3.alt.bsys.dev.centos.org) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Wed Sep 11 15:34:17 UTC 2013
Command line: ro root=UUID=94bf0589-cc13-4e45-a8e7-3f7e3b383e23 rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_MD_UUID=57d163f2:904115e4:f08b19c3:347d9c81 SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_NO_DM rd_NO_LVM rd_MD_UUID=89ca9bed:6f8d56ea:df10f673:9f7edfb7 rhgb quiet pcie_aspm=off
Freeing 9a-100 pfn range: 102 pages freed
1-1 mapping on 9a->100
Freeing bf780-100000 pfn range: 264320 pages freed
1-1 mapping on bf780->100000
Released 264422 pages of unused memory
Set 264422 page(s) to 1-1 mapping
Populating 580000-5c08e6 pfn range: 264422 pages added
BIOS-provided physical RAM map:
Xen: 0000000000000000 - 000000000009a000 (usable)
Xen: 000000000009ac00 - 0000000000100000 (reserved)
Xen: 0000000000100000 - 00000000bf780000 (usable)
Xen: 00000000bf78e000 - 00000000bf790000 type 9
Xen: 00000000bf790000 - 00000000bf79e000 (ACPI data)
Xen: 00000000bf79e000 - 00000000bf7d0000 (ACPI NVS)
Xen: 00000000bf7d0000 - 00000000bf7e0000 (reserved)
Xen: 00000000bf7ec000 - 00000000c0000000 (reserved)
Xen: 00000000e0000000 - 00000000f0000000 (reserved)
Xen: 00000000fec00000 - 00000000fec01000 (reserved)
Xen: 00000000fec8a000 - 00000000fec8b000 (reserved)
Xen: 00000000fee00000 - 00000000fee01000 (reserved)
Xen: 00000000ffc00000 - 0000000100000000 (reserved)
Xen: 0000000100000000 - 00000005c08e6000 (usable)
Xen: 00000005c08e6000 - 0000001840000000 (unusable)
NX (Execute Disable) protection: active
SMBIOS 2.6 present.
DMI: Supermicro X8DTU/X8DTU, BIOS 2.1b 12/30/2011
e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
No AGP bridge found
last_pfn = 0x5c08e6 max_arch_pfn = 0x400000000
last_pfn = 0xbf780 max_arch_pfn = 0x400000000
found SMP MP-table at [ffff8800000ff780] ff780
initial memory mapped : 0 - 07033000
Base memory trampoline at [ffff880000095000] 95000 size 20480
init_memory_mapping: 0000000000000000-00000000bf780000
0000000000 - 00bf780000 page 4k
kernel direct mapping tables up to 0xbf77ffff @ [mem 0x0251d000-0x02b1cfff]
xen: setting RW the range 2ac7000 - 2b1d000
init_memory_mapping: 0000000100000000-00000005c08e6000
0100000000 - 05c08e6000 page 4k
kernel direct mapping tables up to 0x5c08e5fff @ [mem 0xbd166000-0xbf77ffff]
xen: setting RW the range bf77f000 - bf780000
RAMDISK: 02b1d000 - 07033000
crashkernel: memory value expected
ACPI: RSDP 00000000000face0 00024 (v02 ACPIAM)
ACPI: XSDT 00000000bf790100 0008C (v01 SMCI 20111230 MSFT 00000097)
ACPI: FACP 00000000bf790290 000F4 (v04 123011 FACP1954 20111230 MSFT 00000097)
ACPI: DSDT 00000000bf7906a0 06580 (v02 10600 10600000 00000000 INTL 20051117)
ACPI: FACS 00000000bf79e000 00040
ACPI: APIC 00000000bf790390 0011E (v02 123011 APIC1954 20111230 MSFT 00000097)
ACPI: MCFG 00000000bf7904b0 0003C (v01 123011 OEMMCFG 20111230 MSFT 00000097)
ACPI: SLIT 00000000bf7904f0 00030 (v01 123011 OEMSLIT 20111230 MSFT 00000097)
ACPI: OEMB 00000000bf79e040 00086 (v01 123011 OEMB1954 20111230 MSFT 00000097)
ACPI: SRAT 00000000bf79a6a0 001D0 (v02 123011 OEMSRAT 00000001 INTL 00000001)
ACPI: HPET 00000000bf79a870 00038 (v01 123011 OEMHPET 20111230 MSFT 00000097)
ACPI: XMAR 00000000bf79e0d0 00130 (v01 AMI OEMDMAR 00000001 MSFT 00000097)
ACPI: SSDT 00000000bf7a1b30 00363 (v01 DpgPmm CpuPm 00000012 INTL 20051117)
ACPI: EINJ 00000000bf79a8b0 00130 (v01 AMIER AMI_EINJ 20111230 MSFT 00000097)
ACPI: BERT 00000000bf79aa40 00030 (v01 AMIER AMI_BERT 20111230 MSFT 00000097)
ACPI: ERST 00000000bf79aa70 001B0 (v01 AMIER AMI_ERST 20111230 MSFT 00000097)
ACPI: HEST 00000000bf79ac20 000A8 (v01 AMIER ABC_HEST 20111230 MSFT 00000097)
ACPI: Local APIC address 0xfee00000
NUMA turned off
Faking a node at 0000000000000000-00000005c08e6000
Initmem setup node 0 0000000000000000-00000005c08e6000
NODE_DATA [mem 0x5c08c0000-0x5c08e5fff]
Zone PFN ranges:
DMA 0x00000010 -> 0x00001000
DMA32 0x00001000 -> 0x00100000
Normal 0x00100000 -> 0x005c08e6
Movable zone start PFN for each node
Early memory PFN ranges
0: 0x00000010 -> 0x0000009a
0: 0x00000100 -> 0x000bf780
0: 0x00100000 -> 0x005c08e6
On node 0 totalpages: 5767152
DMA zone: 56 pages used for memmap
DMA zone: 5 pages reserved
DMA zone: 3917 pages, LIFO batch:0
DMA32 zone: 14280 pages used for memmap
DMA32 zone: 765880 pages, LIFO batch:31
Normal zone: 68128 pages used for memmap
Normal zone: 4914886 pages, LIFO batch:31
ACPI: PM-Timer IO Port: 0x808
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x12] enabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x14] enabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x20] enabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x22] enabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x32] enabled)
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x34] enabled)
ACPI: LAPIC (acpi_id[0x09] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x03] enabled)
ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x13] enabled)
ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x15] enabled)
ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x21] enabled)
ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x23] enabled)
ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x33] enabled)
ACPI: LAPIC (acpi_id[0x10] lapic_id[0x35] enabled)
ACPI: LAPIC (acpi_id[0x11] lapic_id[0x90] disabled)
ACPI: LAPIC (acpi_id[0x12] lapic_id[0x91] disabled)
ACPI: LAPIC (acpi_id[0x13] lapic_id[0x92] disabled)
ACPI: LAPIC (acpi_id[0x14] lapic_id[0x93] disabled)
ACPI: LAPIC (acpi_id[0x15] lapic_id[0x94] disabled)
ACPI: LAPIC (acpi_id[0x16] lapic_id[0x95] disabled)
ACPI: LAPIC (acpi_id[0x17] lapic_id[0x96] disabled)
ACPI: LAPIC (acpi_id[0x18] lapic_id[0x97] disabled)
ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
ACPI: IOAPIC (id[0x06] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 6, version 253, address 0xfec00000, GSI 0-253
ACPI: IOAPIC (id[0x07] address[0xfec8a000] gsi_base[24])
IOAPIC[1]: apic_id 7, version 253, address 0xfec8a000, GSI 24-277
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
ACPI: HPET id: 0x8086a301 base: 0xfed00000
SMP: Allowing 24 CPUs, 8 hotplug CPUs
nr_irqs_gsi: 294
PM: Registered nosave memory: 000000000009a000 - 000000000009b000
PM: Registered nosave memory: 000000000009b000 - 0000000000100000
PM: Registered nosave memory: 00000000bf780000 - 00000000bf78e000
PM: Registered nosave memory: 00000000bf78e000 - 00000000bf790000
PM: Registered nosave memory: 00000000bf790000 - 00000000bf79e000
PM: Registered nosave memory: 00000000bf79e000 - 00000000bf7d0000
PM: Registered nosave memory: 00000000bf7d0000 - 00000000bf7e0000
PM: Registered nosave memory: 00000000bf7e0000 - 00000000bf7ec000
PM: Registered nosave memory: 00000000bf7ec000 - 00000000c0000000
PM: Registered nosave memory: 00000000c0000000 - 00000000e0000000
PM: Registered nosave memory: 00000000e0000000 - 00000000f0000000
PM: Registered nosave memory: 00000000f0000000 - 00000000fec00000
PM: Registered nosave memory: 00000000fec00000 - 00000000fec01000
PM: Registered nosave memory: 00000000fec01000 - 00000000fec8a000
PM: Registered nosave memory: 00000000fec8a000 - 00000000fec8b000
PM: Registered nosave memory: 00000000fec8b000 - 00000000fee00000
PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000
PM: Registered nosave memory: 00000000fee01000 - 00000000ffc00000
PM: Registered nosave memory: 00000000ffc00000 - 0000000100000000
Allocating PCI resources starting at c0000000 (gap: c0000000:20000000)
Booting paravirtualized kernel on Xen
Xen version: 4.2.3-23.el6 (preserve-AD)
setup_percpu: NR_CPUS:4096 nr_cpumask_bits:24 nr_cpu_ids:24 nr_node_ids:1
PERCPU: Embedded 28 pages/cpu @ffff8805bfc00000 s82432 r8192 d24064 u131072
pcpu-alloc: s82432 r8192 d24064 u131072 alloc=1*2097152
pcpu-alloc: [0] 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15
pcpu-alloc: [0] 16 17 18 19 20 21 22 23 -- -- -- -- -- -- -- --
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 5684683
Policy zone: Normal
Kernel command line: ro root=UUID=94bf0589-cc13-4e45-a8e7-3f7e3b383e23 rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_MD_UUID=57d163f2:904115e4:f08b19c3:347d9c81 SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_NO_DM rd_NO_LVM rd_MD_UUID=89ca9bed:6f8d56ea:df10f673:9f7edfb7 rhgb quiet pcie_aspm=off
PCIe ASPM is disabled
PID hash table entries: 4096 (order: 3, 32768 bytes)
Placing 64MB software IO TLB between ffff8805a5e00000 - ffff8805a9e00000
software IO TLB at phys 0x5a5e00000 - 0x5a9e00000
Memory: 22502988k/24126360k available (5659k kernel code, 1057752k absent, 565620k reserved, 5627k data, 1468k init)
Hierarchical RCU implementation.
NR_IRQS:262400 nr_irqs:1024 16
xen: sci override: global_irq=9 trigger=0 polarity=0
xen: registering gsi 9 triggering 0 polarity 0
xen: --> pirq=9 -> irq=9 (gsi=9)
xen: acpi sci 9
xen: --> pirq=1 -> irq=1 (gsi=1)
xen: --> pirq=2 -> irq=2 (gsi=2)
xen: --> pirq=3 -> irq=3 (gsi=3)
xen: --> pirq=4 -> irq=4 (gsi=4)
xen: --> pirq=5 -> irq=5 (gsi=5)
xen: --> pirq=6 -> irq=6 (gsi=6)
xen: --> pirq=7 -> irq=7 (gsi=7)
xen: --> pirq=8 -> irq=8 (gsi=8)
xen: --> pirq=10 -> irq=10 (gsi=10)
xen: --> pirq=11 -> irq=11 (gsi=11)
xen: --> pirq=12 -> irq=12 (gsi=12)
xen: --> pirq=13 -> irq=13 (gsi=13)
xen: --> pirq=14 -> irq=14 (gsi=14)
xen: --> pirq=15 -> irq=15 (gsi=15)
Console: colour VGA+ 80x25
console [tty0] enabled
allocated 92798976 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
Xen: using vcpuop timer interface
installing Xen timer for CPU 0
Detected 2400.174 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency.. 4800.34 BogoMIPS (lpj=2400174)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux: Initializing.
SELinux: Starting in permissive mode
Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
Mount-cache hash table entries: 256
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys blkio
Initializing cgroup subsys perf_event
ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
ACPI: Core revision 20120320
ftrace: allocating 22518 entries in 88 pages
Performance Events: unsupported p6 CPU model 44 no PMU driver, software events only.
NMI watchdog: disabled (cpu0): hardware events not enabled
installing Xen timer for CPU 1
NMI watchdog: disabled (cpu1): hardware events not enabled
installing Xen timer for CPU 2
NMI watchdog: disabled (cpu2): hardware events not enabled
installing Xen timer for CPU 3
NMI watchdog: disabled (cpu3): hardware events not enabled
Brought up 4 CPUs
devtmpfs: initialized
PM: Registering ACPI NVS region [mem 0xbf79e000-0xbf7cffff] (204800 bytes)
Grant tables using version 2 layout.
Grant table initialized
dummy:
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in E820
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: Added _OSI(Module Device)
ACPI: Added _OSI(Processor Device)
ACPI: Added _OSI(3.0 _SCP Extensions)
ACPI: Added _OSI(Processor Aggregator Device)
ACPI: EC: Look up EC in DSDT
ACPI: Executed 1 blocks of module-level executable AML code
ACPI: SSDT 00000000bf79e200 02FB4 (v01 DpgPmm P001Ist 00000011 INTL 20051117)
ACPI: Dynamic OEM Table Load:
ACPI: SSDT (null) 02FB4 (v01 DpgPmm P001Ist 00000011 INTL 20051117)
ACPI: SSDT 00000000bf7a11c0 00961 (v01 PmRef P001Cst 00003001 INTL 20051117)
ACPI: Dynamic OEM Table Load:
ACPI: SSDT (null) 00961 (v01 PmRef P001Cst 00003001 INTL 20051117)
ACPI: Interpreter enabled
ACPI: (supports S0 S1 S4 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI: No dock devices found.
HEST: Table parsing has been initialized.
PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
pci_root PNP0A08:00: host bridge window [io 0x0000-0x03af]
pci_root PNP0A08:00: host bridge window [io 0x03e0-0x0cf7]
pci_root PNP0A08:00: host bridge window [io 0x03b0-0x03bb]
pci_root PNP0A08:00: host bridge window [io 0x03c0-0x03df]
pci_root PNP0A08:00: host bridge window [io 0x0d00-0xefff]
pci_root PNP0A08:00: host bridge window [io 0xf000-0xffff]
pci_root PNP0A08:00: host bridge window [mem 0x000a0000-0x000bffff]
pci_root PNP0A08:00: host bridge window [mem 0x000d0000-0x000dffff]
pci_root PNP0A08:00: host bridge window [mem 0xc0000000-0xdfffffff]
pci_root PNP0A08:00: host bridge window [mem 0xf0000000-0xfed8ffff]
pci_root PNP0A08:00: host bridge window [mem 0xfed40000-0xfed44fff]
pci_root PNP0A08:00: host bridge window expanded to [mem 0xf0000000-0xfed8ffff]; [mem 0xfed40000-0xfed44fff] ignored
pci_root PNP0A08:00: ignoring host bridge window [mem 0x000d0000-0x000dffff] (conflicts with Adapter ROM [mem 0x000ce000-0x000d3bff])
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io 0x0000-0x03af]
pci_bus 0000:00: root bus resource [io 0x03e0-0x0cf7]
pci_bus 0000:00: root bus resource [io 0x03b0-0x03bb]
pci_bus 0000:00: root bus resource [io 0x03c0-0x03df]
pci_bus 0000:00: root bus resource [io 0x0d00-0xefff]
pci_bus 0000:00: root bus resource [io 0xf000-0xffff]
pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: root bus resource [mem 0xc0000000-0xdfffffff]
pci_bus 0000:00: root bus resource [mem 0xf0000000-0xfed8ffff]
pci 0000:00:00.0: [8086:3406] type 00 class 0x060000
pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
pci 0000:00:01.0: [8086:3408] type 01 class 0x060400
pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
pci 0000:00:03.0: [8086:340a] type 01 class 0x060400
pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
pci 0000:00:05.0: [8086:340c] type 01 class 0x060400
pci 0000:00:05.0: PME# supported from D0 D3hot D3cold
pci 0000:00:07.0: [8086:340e] type 01 class 0x060400
pci 0000:00:07.0: PME# supported from D0 D3hot D3cold
pci 0000:00:09.0: [8086:3410] type 01 class 0x060400
pci 0000:00:09.0: PME# supported from D0 D3hot D3cold
pci 0000:00:13.0: [8086:342d] type 00 class 0x080020
pci 0000:00:13.0: reg 10: [mem 0xfec8a000-0xfec8afff]
pci 0000:00:13.0: PME# supported from D0 D3hot D3cold
pci 0000:00:14.0: [8086:342e] type 00 class 0x080000
pci 0000:00:14.1: [8086:3422] type 00 class 0x080000
pci 0000:00:14.2: [8086:3423] type 00 class 0x080000
pci 0000:00:14.3: [8086:3438] type 00 class 0x080000
pci 0000:00:16.0: [8086:3430] type 00 class 0x088000
pci 0000:00:16.0: reg 10: [mem 0xfbef8000-0xfbefbfff 64bit]
pci 0000:00:16.1: [8086:3431] type 00 class 0x088000
pci 0000:00:16.1: reg 10: [mem 0xfbef4000-0xfbef7fff 64bit]
pci 0000:00:16.2: [8086:3432] type 00 class 0x088000
pci 0000:00:16.2: reg 10: [mem 0xfbef0000-0xfbef3fff 64bit]
pci 0000:00:16.3: [8086:3433] type 00 class 0x088000
pci 0000:00:16.3: reg 10: [mem 0xfbeec000-0xfbeeffff 64bit]
pci 0000:00:16.4: [8086:3429] type 00 class 0x088000
pci 0000:00:16.4: reg 10: [mem 0xfbee8000-0xfbeebfff 64bit]
pci 0000:00:16.5: [8086:342a] type 00 class 0x088000
pci 0000:00:16.5: reg 10: [mem 0xfbee4000-0xfbee7fff 64bit]
pci 0000:00:16.6: [8086:342b] type 00 class 0x088000
pci 0000:00:16.6: reg 10: [mem 0xfbee0000-0xfbee3fff 64bit]
pci 0000:00:16.7: [8086:342c] type 00 class 0x088000
pci 0000:00:16.7: reg 10: [mem 0xfbedc000-0xfbedffff 64bit]
pci 0000:00:1a.0: [8086:3a37] type 00 class 0x0c0300
pci 0000:00:1a.0: reg 20: [io 0xbc00-0xbc1f]
pci 0000:00:1a.1: [8086:3a38] type 00 class 0x0c0300
pci 0000:00:1a.1: reg 20: [io 0xb880-0xb89f]
pci 0000:00:1a.2: [8086:3a39] type 00 class 0x0c0300
pci 0000:00:1a.2: reg 20: [io 0xb800-0xb81f]
pci 0000:00:1a.7: [8086:3a3c] type 00 class 0x0c0320
pci 0000:00:1a.7: reg 10: [mem 0xfbeda000-0xfbeda3ff]
pci 0000:00:1a.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1d.0: [8086:3a34] type 00 class 0x0c0300
pci 0000:00:1d.0: reg 20: [io 0xb480-0xb49f]
pci 0000:00:1d.1: [8086:3a35] type 00 class 0x0c0300
pci 0000:00:1d.1: reg 20: [io 0xb400-0xb41f]
pci 0000:00:1d.2: [8086:3a36] type 00 class 0x0c0300
pci 0000:00:1d.2: reg 20: [io 0xb080-0xb09f]
pci 0000:00:1d.7: [8086:3a3a] type 00 class 0x0c0320
pci 0000:00:1d.7: reg 10: [mem 0xfbed8000-0xfbed83ff]
pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1e.0: [8086:244e] type 01 class 0x060401
pci 0000:00:1f.0: [8086:3a16] type 00 class 0x060100
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 1 PIO at 0a00 (mask 00ff)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 2 PIO at 4700 (mask 00ff)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 3 PIO at 1640 (mask 000f)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 4 PIO at 0ca0 (mask 000f)
pci 0000:00:1f.2: [8086:3a22] type 00 class 0x010601
pci 0000:00:1f.2: reg 10: [io 0xa480-0xa487]
pci 0000:00:1f.2: reg 14: [io 0xb000-0xb003]
pci 0000:00:1f.2: reg 18: [io 0xac00-0xac07]
pci 0000:00:1f.2: reg 1c: [io 0xa880-0xa883]
pci 0000:00:1f.2: reg 20: [io 0xa800-0xa81f]
pci 0000:00:1f.2: reg 24: [mem 0xfbed6000-0xfbed67ff]
pci 0000:00:1f.2: PME# supported from D3hot
pci 0000:00:1f.3: [8086:3a30] type 00 class 0x0c0500
pci 0000:00:1f.3: reg 10: [mem 0xfbed4000-0xfbed40ff 64bit]
pci 0000:00:1f.3: reg 20: [io 0x0400-0x041f]
pci 0000:01:00.0: [8086:10c9] type 00 class 0x020000
pci 0000:01:00.0: reg 10: [mem 0xfbbe0000-0xfbbfffff]
pci 0000:01:00.0: reg 14: [mem 0xfbbc0000-0xfbbdffff]
pci 0000:01:00.0: reg 18: [io 0xcc00-0xcc1f]
pci 0000:01:00.0: reg 1c: [mem 0xfbb9c000-0xfbb9ffff]
pci 0000:01:00.0: reg 30: [mem 0xfbba0000-0xfbbbffff pref]
pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
pci 0000:01:00.1: [8086:10c9] type 00 class 0x020000
pci 0000:01:00.1: reg 10: [mem 0xfbb60000-0xfbb7ffff]
pci 0000:01:00.1: reg 14: [mem 0xfbb40000-0xfbb5ffff]
pci 0000:01:00.1: reg 18: [io 0xc880-0xc89f]
pci 0000:01:00.1: reg 1c: [mem 0xfbb1c000-0xfbb1ffff]
pci 0000:01:00.1: reg 30: [mem 0xfbb20000-0xfbb3ffff pref]
pci 0000:01:00.1: PME# supported from D0 D3hot D3cold
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0: bridge window [io 0xc000-0xcfff]
pci 0000:00:01.0: bridge window [mem 0xfbb00000-0xfbbfffff]
pci 0000:00:03.0: PCI bridge to [bus 02-02]
pci 0000:00:05.0: PCI bridge to [bus 03-03]
pci 0000:04:00.0: [1000:0070] type 00 class 0x010700
pci 0000:04:00.0: reg 10: [io 0xd000-0xd0ff]
pci 0000:04:00.0: reg 14: [mem 0xfbc3c000-0xfbc3ffff 64bit]
pci 0000:04:00.0: reg 1c: [mem 0xfbc40000-0xfbc7ffff 64bit]
pci 0000:04:00.0: reg 30: [mem 0xfbc80000-0xfbcfffff pref]
pci 0000:04:00.0: supports D1 D2
pci 0000:04:00.0: reg 174: [mem 0x00000000-0x00003fff 64bit]
pci 0000:04:00.0: reg 17c: [mem 0x00000000-0x0003ffff 64bit]
pci 0000:00:07.0: PCI bridge to [bus 04-04]
pci 0000:00:07.0: bridge window [io 0xd000-0xdfff]
pci 0000:00:07.0: bridge window [mem 0xfbc00000-0xfbcfffff]
pci 0000:05:00.0: [8086:10d3] type 00 class 0x020000
pci 0000:05:00.0: reg 10: [mem 0xfbd20000-0xfbd3ffff]
pci 0000:05:00.0: reg 14: [mem 0xfbd80000-0xfbdfffff]
pci 0000:05:00.0: reg 18: [io 0xec00-0xec1f]
pci 0000:05:00.0: reg 1c: [mem 0xfbd1c000-0xfbd1ffff]
pci 0000:05:00.0: reg 30: [mem 0xfbd40000-0xfbd7ffff pref]
pci 0000:05:00.0: PME# supported from D0 D3hot D3cold
pci 0000:00:09.0: PCI bridge to [bus 05-05]
pci 0000:00:09.0: bridge window [io 0xe000-0xefff]
pci 0000:00:09.0: bridge window [mem 0xfbd00000-0xfbdfffff]
pci 0000:06:01.0: [102b:0532] type 00 class 0x030000
pci 0000:06:01.0: reg 10: [mem 0xf9000000-0xf9ffffff pref]
pci 0000:06:01.0: reg 14: [mem 0xfaffc000-0xfaffffff]
pci 0000:06:01.0: reg 18: [mem 0xfb000000-0xfb7fffff]
pci 0000:00:1e.0: PCI bridge to [bus 06-06] (subtractive decode)
pci 0000:00:1e.0: bridge window [mem 0xfaf00000-0xfb7fffff]
pci 0000:00:1e.0: bridge window [mem 0xf9000000-0xf9ffffff 64bit pref]
pci 0000:00:1e.0: bridge window [io 0x0000-0x03af] (subtractive decode)
pci 0000:00:1e.0: bridge window [io 0x03e0-0x0cf7] (subtractive decode)
pci 0000:00:1e.0: bridge window [io 0x03b0-0x03bb] (subtractive decode)
pci 0000:00:1e.0: bridge window [io 0x03c0-0x03df] (subtractive decode)
pci 0000:00:1e.0: bridge window [io 0x0d00-0xefff] (subtractive decode)
pci 0000:00:1e.0: bridge window [io 0xf000-0xffff] (subtractive decode)
pci 0000:00:1e.0: bridge window [mem 0x000a0000-0x000bffff] (subtractive decode)
pci 0000:00:1e.0: bridge window [mem 0xc0000000-0xdfffffff] (subtractive decode)
pci 0000:00:1e.0: bridge window [mem 0xf0000000-0xfed8ffff] (subtractive decode)
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE3._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE5._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE7._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE9._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
pci0000:00: Unable to request _OSC control (_OSC support mask: 0x19)
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 6 7 10 11 12 14 *15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 6 7 10 11 12 *14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 6 7 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 6 7 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 6 *7 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 *6 7 10 11 12 14 15)
xen/balloon: Initialising balloon driver.
xen-balloon: Initialising balloon driver.
vgaarb: device added: PCI:0000:06:01.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
vgaarb: bridge control possible 0000:06:01.0
SCSI subsystem initialized
libata version 3.00 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
PCI: Discovered peer bus fe
PCI: root bus fe: using default resources
PCI host bridge to bus 0000:fe
pci_bus 0000:fe: root bus resource [io 0x0000-0xffff]
pci_bus 0000:fe: root bus resource [mem 0x00000000-0xffffffffff]
pci 0000:fe:00.0: [8086:2c70] type 00 class 0x060000
pci 0000:fe:00.1: [8086:2d81] type 00 class 0x060000
pci 0000:fe:02.0: [8086:2d90] type 00 class 0x060000
pci 0000:fe:02.1: [8086:2d91] type 00 class 0x060000
pci 0000:fe:02.2: [8086:2d92] type 00 class 0x060000
pci 0000:fe:02.3: [8086:2d93] type 00 class 0x060000
pci 0000:fe:02.4: [8086:2d94] type 00 class 0x060000
pci 0000:fe:02.5: [8086:2d95] type 00 class 0x060000
pci 0000:fe:03.0: [8086:2d98] type 00 class 0x060000
pci 0000:fe:03.1: [8086:2d99] type 00 class 0x060000
pci 0000:fe:03.2: [8086:2d9a] type 00 class 0x060000
pci 0000:fe:03.4: [8086:2d9c] type 00 class 0x060000
pci 0000:fe:04.0: [8086:2da0] type 00 class 0x060000
pci 0000:fe:04.1: [8086:2da1] type 00 class 0x060000
pci 0000:fe:04.2: [8086:2da2] type 00 class 0x060000
pci 0000:fe:04.3: [8086:2da3] type 00 class 0x060000
pci 0000:fe:05.0: [8086:2da8] type 00 class 0x060000
pci 0000:fe:05.1: [8086:2da9] type 00 class 0x060000
pci 0000:fe:05.2: [8086:2daa] type 00 class 0x060000
pci 0000:fe:05.3: [8086:2dab] type 00 class 0x060000
pci 0000:fe:06.0: [8086:2db0] type 00 class 0x060000
pci 0000:fe:06.1: [8086:2db1] type 00 class 0x060000
pci 0000:fe:06.2: [8086:2db2] type 00 class 0x060000
pci 0000:fe:06.3: [8086:2db3] type 00 class 0x060000
PCI: Discovered peer bus ff
PCI: root bus ff: using default resources
PCI host bridge to bus 0000:ff
pci_bus 0000:ff: root bus resource [io 0x0000-0xffff]
pci_bus 0000:ff: root bus resource [mem 0x00000000-0xffffffffff]
pci 0000:ff:00.0: [8086:2c70] type 00 class 0x060000
pci 0000:ff:00.1: [8086:2d81] type 00 class 0x060000
pci 0000:ff:02.0: [8086:2d90] type 00 class 0x060000
pci 0000:ff:02.1: [8086:2d91] type 00 class 0x060000
pci 0000:ff:02.2: [8086:2d92] type 00 class 0x060000
pci 0000:ff:02.3: [8086:2d93] type 00 class 0x060000
pci 0000:ff:02.4: [8086:2d94] type 00 class 0x060000
pci 0000:ff:02.5: [8086:2d95] type 00 class 0x060000
pci 0000:ff:03.0: [8086:2d98] type 00 class 0x060000
pci 0000:ff:03.1: [8086:2d99] type 00 class 0x060000
pci 0000:ff:03.2: [8086:2d9a] type 00 class 0x060000
pci 0000:ff:03.4: [8086:2d9c] type 00 class 0x060000
pci 0000:ff:04.0: [8086:2da0] type 00 class 0x060000
pci 0000:ff:04.1: [8086:2da1] type 00 class 0x060000
pci 0000:ff:04.2: [8086:2da2] type 00 class 0x060000
pci 0000:ff:04.3: [8086:2da3] type 00 class 0x060000
pci 0000:ff:05.0: [8086:2da8] type 00 class 0x060000
pci 0000:ff:05.1: [8086:2da9] type 00 class 0x060000
pci 0000:ff:05.2: [8086:2daa] type 00 class 0x060000
pci 0000:ff:05.3: [8086:2dab] type 00 class 0x060000
pci 0000:ff:06.0: [8086:2db0] type 00 class 0x060000
pci 0000:ff:06.1: [8086:2db1] type 00 class 0x060000
pci 0000:ff:06.2: [8086:2db2] type 00 class 0x060000
pci 0000:ff:06.3: [8086:2db3] type 00 class 0x060000
PCI: pci_cache_line_size set to 64 bytes
reserve RAM buffer: 000000000009a000 - 000000000009ffff
reserve RAM buffer: 00000000bf780000 - 00000000bfffffff
reserve RAM buffer: 00000005c08e6000 - 00000005c3ffffff
NetLabel: Initializing
NetLabel: domain hash size = 128
NetLabel: protocols = UNLABELED CIPSOv4
NetLabel: unlabeled traffic allowed by default
nfc: nfc_init: NFC Core ver 0.1
NET: Registered protocol family 39
Switching to clocksource xen
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp 00:00: [bus 00-ff]
pnp 00:00: [io 0x0cf8-0x0cff]
pnp 00:00: [io 0x0000-0x03af window]
pnp 00:00: [io 0x03e0-0x0cf7 window]
pnp 00:00: [io 0x03b0-0x03bb window]
pnp 00:00: [io 0x03c0-0x03df window]
pnp 00:00: [io 0x0d00-0xefff window]
pnp 00:00: [io 0xf000-0xffff window]
pnp 00:00: [mem 0x000a0000-0x000bffff window]
pnp 00:00: [mem 0x000d0000-0x000dffff window]
pnp 00:00: [mem 0xc0000000-0xdfffffff window]
pnp 00:00: [mem 0xf0000000-0xfed8ffff window]
pnp 00:00: [mem 0xfed40000-0xfed44fff]
pnp 00:00: Plug and Play ACPI device, IDs PNP0a08 PNP0a03 (active)
pnp 00:01: [mem 0xfed1c000-0xfed1ffff]
system 00:01: [mem 0xfed1c000-0xfed1ffff] has been reserved
system 00:01: Plug and Play ACPI device, IDs PNP0c01 (active)
pnp 00:02: [dma 4]
pnp 00:02: [io 0x0000-0x000f]
pnp 00:02: [io 0x0081-0x0083]
pnp 00:02: [io 0x0087]
pnp 00:02: [io 0x0089-0x008b]
pnp 00:02: [io 0x008f]
pnp 00:02: [io 0x00c0-0x00df]
pnp 00:02: Plug and Play ACPI device, IDs PNP0200 (active)
pnp 00:03: [io 0x0070-0x0071]
xen: registering gsi 8 triggering 1 polarity 0
pnp 00:03: [irq 8]
pnp 00:03: Plug and Play ACPI device, IDs PNP0b00 (active)
pnp 00:04: [io 0x0061]
pnp 00:04: Plug and Play ACPI device, IDs PNP0800 (active)
pnp 00:05: [io 0x00f0-0x00ff]
xen: registering gsi 13 triggering 1 polarity 0
pnp 00:05: [irq 13]
pnp 00:05: Plug and Play ACPI device, IDs PNP0c04 (active)
pnp 00:06: [io 0x0000-0xffffffffffffffff disabled]
pnp 00:06: [io 0x0000-0xffffffffffffffff disabled]
pnp 00:06: [io 0x0a10-0x0a1f]
system 00:06: [io 0x0a10-0x0a1f] has been reserved
system 00:06: Plug and Play ACPI device, IDs PNP0c02 (active)
pnp 00:07: [io 0x03f8-0x03ff]
xen: registering gsi 4 triggering 1 polarity 0
pnp 00:07: [irq 4]
pnp 00:07: [dma 0 disabled]
pnp 00:07: Plug and Play ACPI device, IDs PNP0501 (active)
pnp 00:08: [io 0x02f8-0x02ff]
xen: registering gsi 3 triggering 1 polarity 0
pnp 00:08: [irq 3]
pnp 00:08: [dma 0 disabled]
pnp 00:08: Plug and Play ACPI device, IDs PNP0501 (active)
pnp 00:09: [io 0x0010-0x001f]
pnp 00:09: [io 0x0022-0x003f]
pnp 00:09: [io 0x0044-0x004d]
pnp 00:09: [io 0x0050-0x005f]
pnp 00:09: [io 0x0062-0x0063]
pnp 00:09: [io 0x0065-0x006f]
pnp 00:09: [io 0x0072-0x007f]
pnp 00:09: [io 0x0080]
pnp 00:09: [io 0x0084-0x0086]
pnp 00:09: [io 0x0088]
pnp 00:09: [io 0x008c-0x008e]
pnp 00:09: [io 0x0090-0x009f]
pnp 00:09: [io 0x00a2-0x00bf]
pnp 00:09: [io 0x00e0-0x00ef]
pnp 00:09: [io 0x0ca2-0x0ca3]
pnp 00:09: [io 0x0cf8-0x0cff]
pnp 00:09: [io 0x04d0-0x04d1]
pnp 00:09: [mem 0x00000400-0x000004ff]
pnp 00:09: [io 0x0800-0x087f]
pnp 00:09: [io 0x0000-0xffffffffffffffff disabled]
pnp 00:09: [io 0x0500-0x057f]
pnp 00:09: [mem 0xfed1c000-0xfed1ffff]
pnp 00:09: [mem 0xfed20000-0xfed3ffff]
pnp 00:09: [mem 0xfed40000-0xfed8ffff]
pnp 00:09: disabling [mem 0x00000400-0x000004ff] because it overlaps 0000:04:00.0 BAR 7 [mem 0x00000000-0x0003ffff 64bit]
pnp 00:09: disabling [mem 0x00000400-0x000004ff disabled] because it overlaps 0000:04:00.0 BAR 9 [mem 0x00000000-0x003fffff 64bit]
system 00:09: [io 0x0ca2-0x0ca3] has been reserved
system 00:09: [io 0x0cf8-0x0cff] could not be reserved
system 00:09: [io 0x04d0-0x04d1] has been reserved
system 00:09: [io 0x0800-0x087f] has been reserved
system 00:09: [io 0x0500-0x057f] has been reserved
system 00:09: [mem 0xfed1c000-0xfed1ffff] has been reserved
system 00:09: [mem 0xfed20000-0xfed3ffff] has been reserved
system 00:09: [mem 0xfed40000-0xfed8ffff] has been reserved
system 00:09: Plug and Play ACPI device, IDs PNP0c02 (active)
pnp 00:0a: [mem 0xfed00000-0xfed003ff]
pnp 00:0a: Plug and Play ACPI device, IDs PNP0103 (active)
pnp 00:0b: [io 0x0060]
pnp 00:0b: [io 0x0064]
pnp 00:0b: [mem 0xfec00000-0xfec00fff]
pnp 00:0b: [mem 0xfee00000-0xfee00fff]
system 00:0b: [mem 0xfec00000-0xfec00fff] could not be reserved
system 00:0b: [mem 0xfee00000-0xfee00fff] has been reserved
system 00:0b: Plug and Play ACPI device, IDs PNP0c02 (active)
pnp 00:0c: [mem 0xe0000000-0xefffffff]
system 00:0c: [mem 0xe0000000-0xefffffff] has been reserved
system 00:0c: Plug and Play ACPI device, IDs PNP0c02 (active)
pnp 00:0d: [mem 0x000c0000-0x000cffff]
pnp 00:0d: [mem 0x000e0000-0x000fffff]
pnp 00:0d: [mem 0xfed90000-0xffffffff]
pnp 00:0d: disabling [mem 0x000c0000-0x000cffff] because it overlaps 0000:04:00.0 BAR 9 [mem 0x00000000-0x003fffff 64bit]
pnp 00:0d: disabling [mem 0x000e0000-0x000fffff] because it overlaps 0000:04:00.0 BAR 9 [mem 0x00000000-0x003fffff 64bit]
system 00:0d: [mem 0xfed90000-0xffffffff] could not be reserved
system 00:0d: Plug and Play ACPI device, IDs PNP0c01 (active)
pnp: PnP ACPI: found 14 devices
ACPI: ACPI bus type pnp unregistered
PM-Timer failed consistency check (0x0xffffff) - aborting.
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0: bridge window [io 0xc000-0xcfff]
pci 0000:00:01.0: bridge window [mem 0xfbb00000-0xfbbfffff]
pci 0000:00:03.0: PCI bridge to [bus 02-02]
pci 0000:00:05.0: PCI bridge to [bus 03-03]
pci 0000:04:00.0: reg 174: [mem 0x00000000-0x00003fff 64bit]
pci 0000:04:00.0: reg 17c: [mem 0x00000000-0x0003ffff 64bit]
pci 0000:04:00.0: reg 174: [mem 0x00000000-0x00003fff 64bit]
pci 0000:04:00.0: reg 17c: [mem 0x00000000-0x0003ffff 64bit]
pci 0000:04:00.0: BAR 9: can't assign mem (size 0x400000)
pci 0000:04:00.0: reg 174: [mem 0x00000000-0x00003fff 64bit]
pci 0000:04:00.0: BAR 7: can't assign mem (size 0x40000)
pci 0000:00:07.0: PCI bridge to [bus 04-04]
pci 0000:00:07.0: bridge window [io 0xd000-0xdfff]
pci 0000:00:07.0: bridge window [mem 0xfbc00000-0xfbcfffff]
pci 0000:00:09.0: PCI bridge to [bus 05-05]
pci 0000:00:09.0: bridge window [io 0xe000-0xefff]
pci 0000:00:09.0: bridge window [mem 0xfbd00000-0xfbdfffff]
pci 0000:00:1e.0: PCI bridge to [bus 06-06]
pci 0000:00:1e.0: bridge window [mem 0xfaf00000-0xfb7fffff]
pci 0000:00:1e.0: bridge window [mem 0xf9000000-0xf9ffffff 64bit pref]
Some PCI device resources are unassigned, try booting with pci=realloc
pci 0000:00:1e.0: setting latency timer to 64
pci_bus 0000:00: resource 4 [io 0x0000-0x03af]
pci_bus 0000:00: resource 5 [io 0x03e0-0x0cf7]
pci_bus 0000:00: resource 6 [io 0x03b0-0x03bb]
pci_bus 0000:00: resource 7 [io 0x03c0-0x03df]
pci_bus 0000:00: resource 8 [io 0x0d00-0xefff]
pci_bus 0000:00: resource 9 [io 0xf000-0xffff]
pci_bus 0000:00: resource 10 [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: resource 11 [mem 0xc0000000-0xdfffffff]
pci_bus 0000:00: resource 12 [mem 0xf0000000-0xfed8ffff]
pci_bus 0000:01: resource 0 [io 0xc000-0xcfff]
pci_bus 0000:01: resource 1 [mem 0xfbb00000-0xfbbfffff]
pci_bus 0000:04: resource 0 [io 0xd000-0xdfff]
pci_bus 0000:04: resource 1 [mem 0xfbc00000-0xfbcfffff]
pci_bus 0000:05: resource 0 [io 0xe000-0xefff]
pci_bus 0000:05: resource 1 [mem 0xfbd00000-0xfbdfffff]
pci_bus 0000:06: resource 1 [mem 0xfaf00000-0xfb7fffff]
pci_bus 0000:06: resource 2 [mem 0xf9000000-0xf9ffffff 64bit pref]
pci_bus 0000:06: resource 4 [io 0x0000-0x03af]
pci_bus 0000:06: resource 5 [io 0x03e0-0x0cf7]
pci_bus 0000:06: resource 6 [io 0x03b0-0x03bb]
pci_bus 0000:06: resource 7 [io 0x03c0-0x03df]
pci_bus 0000:06: resource 8 [io 0x0d00-0xefff]
pci_bus 0000:06: resource 9 [io 0xf000-0xffff]
pci_bus 0000:06: resource 10 [mem 0x000a0000-0x000bffff]
pci_bus 0000:06: resource 11 [mem 0xc0000000-0xdfffffff]
pci_bus 0000:06: resource 12 [mem 0xf0000000-0xfed8ffff]
pci_bus 0000:fe: resource 4 [io 0x0000-0xffff]
pci_bus 0000:fe: resource 5 [mem 0x00000000-0xffffffffff]
pci_bus 0000:ff: resource 4 [io 0x0000-0xffff]
pci_bus 0000:ff: resource 5 [mem 0x00000000-0xffffffffff]
NET: Registered protocol family 2
IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
TCP: reno registered
UDP hash table entries: 16384 (order: 7, 524288 bytes)
UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)
NET: Registered protocol family 1
xen: registering gsi 16 triggering 0 polarity 1
xen: --> pirq=16 -> irq=16 (gsi=16)
xen: registering gsi 21 triggering 0 polarity 1
xen: --> pirq=21 -> irq=21 (gsi=21)
xen: registering gsi 19 triggering 0 polarity 1
xen: --> pirq=19 -> irq=19 (gsi=19)
xen: registering gsi 18 triggering 0 polarity 1
xen: --> pirq=18 -> irq=18 (gsi=18)
xen: registering gsi 23 triggering 0 polarity 1
xen: --> pirq=23 -> irq=23 (gsi=23)
xen: registering gsi 19 triggering 0 polarity 1
Already setup the GSI :19
xen: registering gsi 18 triggering 0 polarity 1
Already setup the GSI :18
xen: registering gsi 23 triggering 0 polarity 1
Already setup the GSI :23
pci 0000:06:01.0: Boot video device
PCI: CLS 256 bytes, default 64
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 70744k freed
audit: initializing netlink socket (disabled)
type=2000 audit(1384488225.688:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 32768
SELinux: Registering netfilter hooks
alg: No test for stdrng (krng)
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
ioapic: probe of 0000:00:13.0 failed with error -22
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
intel_idle: does not run on family 6 model 44
input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
ACPI: Power Button [PWRB]
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
ACPI: Power Button [PWRF]
ACPI: Requesting acpi_cpufreq
ERST: Failed to get Error Log Address Range.
[Firmware Warn]: GHES: Poll interval is 0 for generic hardware error source: 1, disabled.
GHES: Failed to enable APEI firmware first mode.
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:07: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:08: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
hpet_acpi_add: no address or irqs in _CRS
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
[drm] Initialized drm 1.1.0 20060810
brd: module loaded
loop: module loaded
Fixed MDIO Bus: probed
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
xen: registering gsi 18 triggering 0 polarity 1
Already setup the GSI :18
ehci_hcd 0000:00:1a.7: setting latency timer to 64
ehci_hcd 0000:00:1a.7: EHCI Host Controller
ehci_hcd 0000:00:1a.7: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:1a.7: debug port 1
ehci_hcd 0000:00:1a.7: cache line size of 256 is not supported
ehci_hcd 0000:00:1a.7: irq 18, io mem 0xfbeda000
ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 3.4.61-9.el6.centos.alt.x86_64 ehci_hcd
usb usb1: SerialNumber: 0000:00:1a.7
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 6 ports detected
xen: registering gsi 23 triggering 0 polarity 1
Already setup the GSI :23
ehci_hcd 0000:00:1d.7: setting latency timer to 64
ehci_hcd 0000:00:1d.7: EHCI Host Controller
ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 2
ehci_hcd 0000:00:1d.7: debug port 1
ehci_hcd 0000:00:1d.7: cache line size of 256 is not supported
ehci_hcd 0000:00:1d.7: irq 23, io mem 0xfbed8000
ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: EHCI Host Controller
usb usb2: Manufacturer: Linux 3.4.61-9.el6.centos.alt.x86_64 ehci_hcd
usb usb2: SerialNumber: 0000:00:1d.7
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 6 ports detected
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
xen: registering gsi 16 triggering 0 polarity 1
Already setup the GSI :16
uhci_hcd 0000:00:1a.0: setting latency timer to 64
uhci_hcd 0000:00:1a.0: UHCI Host Controller
uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 3
uhci_hcd 0000:00:1a.0: irq 16, io base 0x0000bc00
usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb3: Product: UHCI Host Controller
usb usb3: Manufacturer: Linux 3.4.61-9.el6.centos.alt.x86_64 uhci_hcd
usb usb3: SerialNumber: 0000:00:1a.0
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
xen: registering gsi 21 triggering 0 polarity 1
Already setup the GSI :21
uhci_hcd 0000:00:1a.1: setting latency timer to 64
uhci_hcd 0000:00:1a.1: UHCI Host Controller
uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus number 4
uhci_hcd 0000:00:1a.1: irq 21, io base 0x0000b880
usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb4: Product: UHCI Host Controller
usb usb4: Manufacturer: Linux 3.4.61-9.el6.centos.alt.x86_64 uhci_hcd
usb usb4: SerialNumber: 0000:00:1a.1
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 2 ports detected
xen: registering gsi 19 triggering 0 polarity 1
Already setup the GSI :19
uhci_hcd 0000:00:1a.2: setting latency timer to 64
uhci_hcd 0000:00:1a.2: UHCI Host Controller
uhci_hcd 0000:00:1a.2: new USB bus registered, assigned bus number 5
uhci_hcd 0000:00:1a.2: irq 19, io base 0x0000b800
usb usb5: New USB device found, idVendor=1d6b, idProduct=0001
usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb5: Product: UHCI Host Controller
usb usb5: Manufacturer: Linux 3.4.61-9.el6.centos.alt.x86_64 uhci_hcd
usb usb5: SerialNumber: 0000:00:1a.2
hub 5-0:1.0: USB hub found
hub 5-0:1.0: 2 ports detected
xen: registering gsi 23 triggering 0 polarity 1
Already setup the GSI :23
uhci_hcd 0000:00:1d.0: setting latency timer to 64
uhci_hcd 0000:00:1d.0: UHCI Host Controller
uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 6
uhci_hcd 0000:00:1d.0: irq 23, io base 0x0000b480
usb usb6: New USB device found, idVendor=1d6b, idProduct=0001
usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb6: Product: UHCI Host Controller
usb usb6: Manufacturer: Linux 3.4.61-9.el6.centos.alt.x86_64 uhci_hcd
usb usb6: SerialNumber: 0000:00:1d.0
hub 6-0:1.0: USB hub found
hub 6-0:1.0: 2 ports detected
xen: registering gsi 19 triggering 0 polarity 1
Already setup the GSI :19
uhci_hcd 0000:00:1d.1: setting latency timer to 64
uhci_hcd 0000:00:1d.1: UHCI Host Controller
uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 7
uhci_hcd 0000:00:1d.1: irq 19, io base 0x0000b400
usb usb7: New USB device found, idVendor=1d6b, idProduct=0001
usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb7: Product: UHCI Host Controller
usb usb7: Manufacturer: Linux 3.4.61-9.el6.centos.alt.x86_64 uhci_hcd
usb usb7: SerialNumber: 0000:00:1d.1
hub 7-0:1.0: USB hub found
hub 7-0:1.0: 2 ports detected
xen: registering gsi 18 triggering 0 polarity 1
Already setup the GSI :18
uhci_hcd 0000:00:1d.2: setting latency timer to 64
uhci_hcd 0000:00:1d.2: UHCI Host Controller
uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 8
uhci_hcd 0000:00:1d.2: irq 18, io base 0x0000b080
usb usb8: New USB device found, idVendor=1d6b, idProduct=0001
usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb8: Product: UHCI Host Controller
usb usb8: Manufacturer: Linux 3.4.61-9.el6.centos.alt.x86_64 uhci_hcd
usb usb8: SerialNumber: 0000:00:1d.2
hub 8-0:1.0: USB hub found
hub 8-0:1.0: 2 ports detected
usbcore: registered new interface driver libusual
i8042: PNP: No PS/2 controller found. Probing ports directly.
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mousedev: PS/2 mouse device common for all mice
rtc_cmos 00:03: RTC can wake from S4
rtc_cmos 00:03: rtc core: registered rtc_cmos as rtc0
rtc0: alarms up to one month, y3k, 114 bytes nvram
EFI Variables Facility v0.08 2004-May-17
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
TCP: cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
registered taskstats version 1
IMA: No TPM chip found, activating TPM-bypass!
rtc_cmos 00:03: setting system clock to 2013-11-15 04:03:45 UTC (1384488225)
Initializing network drop monitor service
Freeing unused kernel memory: 1468k freed
Write protecting the kernel read-only data: 10240k
Freeing unused kernel memory: 464k freed
Freeing unused kernel memory: 1424k freed
dracut: dracut-004-303.el6
dracut: rd_NO_LUKS: removing cryptoluks activation
dracut: rd_NO_LVM: removing LVM activation
Loading iSCSI transport class v2.0-870.
iscsi: registered transport (qla4xxx)
QLogic iSCSI HBA Driver
libcxgbi:libcxgbi_init_module: tag itt 0x1fff, 13 bits, age 0xf, 4 bits.
libcxgbi:ddp_setup_host_page_size: system PAGE 4096, ddp idx 0.
Chelsio T3 iSCSI Driver cxgb3i v2.0.0 (Jun. 2010)
iscsi: registered transport (cxgb3i)
Chelsio T4 iSCSI Driver cxgb4i v0.9.1 (Aug. 2010)
iscsi: registered transport (cxgb4i)
NET: Registered protocol family 10
cnic: Broadcom NetXtreme II CNIC Driver cnic v2.5.10 (March 21, 2012)
Broadcom NetXtreme II iSCSI Driver bnx2i v2.7.0.3 (Jun 15, 2011)
iscsi: registered transport (bnx2i)
iscsi: registered transport (be2iscsi)
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.22.0-ioctl (2011-10-19) initialised: dm-devel@redhat.com
udev: starting version 147
udevd (110): /proc/110/oom_adj is deprecated, please use /proc/110/oom_score_adj instead.
dracut: Starting plymouth daemon
dca service started, version 1.12.1
igb: Intel(R) Gigabit Ethernet Network Driver - version 3.2.10-k
igb: Copyright (c) 2007-2012 Intel Corporation.
xen: registering gsi 28 triggering 0 polarity 1
xen: --> pirq=28 -> irq=28 (gsi=28)
igb 0000:01:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:01:00.0: eth0: (PCIe:2.5Gb/s:Width x4) 00:25:90:6a:90:da
igb 0000:01:00.0: eth0: PBA No: FFFFFF-0FF
igb 0000:01:00.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
xen: registering gsi 40 triggering 0 polarity 1
xen: --> pirq=40 -> irq=40 (gsi=40)
usb 4-1: new full-speed USB device number 2 using uhci_hcd
igb 0000:01:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0000:01:00.1: eth1: (PCIe:2.5Gb/s:Width x4) 00:25:90:6a:90:db
igb 0000:01:00.1: eth1: PBA No: FFFFFF-0FF
igb 0000:01:00.1: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
mpt2sas version 12.100.00.00 loaded
scsi0 : Fusion MPT SAS Host
xen: registering gsi 30 triggering 0 polarity 1
xen: --> pirq=30 -> irq=30 (gsi=30)
mpt2sas0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (22577088 kB)
mpt2sas0-msix0: PCI-MSI-X enabled: IRQ 334
mpt2sas0: iomem(0x00000000fbc3c000), mapped(0xffffc90013f48000), size(16384)
mpt2sas0: ioport(0x000000000000d000), size(256)
usb 4-1: New USB device found, idVendor=046b, idProduct=ff10
usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 4-1: Product: Virtual Keyboard and Mouse
usb 4-1: Manufacturer: American Megatrends Inc.
usb 4-1: SerialNumber: serial
mpt2sas0: sending diag reset !!
input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/pci0000:00/0000:00:1a.1/usb4/4-1/4-1:1.0/input/input2
generic-usb 0003:046B:FF10.0001: input,hidraw0: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.1-1/input0
input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/pci0000:00/0000:00:1a.1/usb4/4-1/4-1:1.1/input/input3
generic-usb 0003:046B:FF10.0002: input,hidraw1: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1a.1-1/input1
mpt2sas0: diag reset: SUCCESS
mpt2sas0: Allocated physical memory: size(3379 kB)
mpt2sas0: Current Controller Queue Depth(1483), Max Controller Queue Depth(1720)
mpt2sas0: Scatter Gather Elements per IO(128)
mpt2sas0: LSISAS2004: FWVersion(15.00.00.00), ChipRevision(0x03), BiosVersion(07.29.00.00)
mpt2sas0: Protocol=(Initiator), Capabilities=(Raid,TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)
mpt2sas0: sending port enable !!
e1000e: Intel(R) PRO/1000 Network Driver - 1.9.5-k
e1000e: Copyright(c) 1999 - 2012 Intel Corporation.
e1000e 0000:05:00.0: Disabling ASPM L0s L1
xen: registering gsi 32 triggering 0 polarity 1
xen: --> pirq=32 -> irq=32 (gsi=32)
e1000e 0000:05:00.0: (unregistered net_device): Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
e1000e 0000:05:00.0: eth2: (PCI Express:2.5GT/s:Width x1) 68:05:ca:0c:57:6b
e1000e 0000:05:00.0: eth2: Intel(R) PRO/1000 Network Connection
e1000e 0000:05:00.0: eth2: MAC: 3, PHY: 8, PBA No: E46981-007
ahci 0000:00:1f.2: version 3.0
xen: registering gsi 19 triggering 0 polarity 1
Already setup the GSI :19
ahci: SSS flag set, parallel bus scan disabled
ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports 3 Gbps 0x3f impl SATA mode
ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led clo pio slum part ccc ems sxs
ahci 0000:00:1f.2: setting latency timer to 64
scsi1 : ahci
scsi2 : ahci
scsi3 : ahci
scsi4 : ahci
scsi5 : ahci
scsi6 : ahci
ata1: SATA max UDMA/133 abar m2048@0xfbed6000 port 0xfbed6100 irq 338
ata2: SATA max UDMA/133 abar m2048@0xfbed6000 port 0xfbed6180 irq 338
ata3: SATA max UDMA/133 abar m2048@0xfbed6000 port 0xfbed6200 irq 338
ata4: SATA max UDMA/133 abar m2048@0xfbed6000 port 0xfbed6280 irq 338
ata5: SATA max UDMA/133 abar m2048@0xfbed6000 port 0xfbed6300 irq 338
ata6: SATA max UDMA/133 abar m2048@0xfbed6000 port 0xfbed6380 irq 338
ata1: SATA link down (SStatus 0 SControl 300)
ata2: SATA link down (SStatus 0 SControl 300)
ata3: SATA link down (SStatus 0 SControl 300)
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata4.00: ATA-9: Samsung SSD 840 PRO Series, DXM05B0Q, max UDMA/133
ata4.00: 250069680 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
ata4.00: configured for UDMA/133
scsi: waiting for bus probes to complete ...
mpt2sas0: host_add: handle(0x0001), sas_addr(0x500062b0002b4948), phys(8)
mpt2sas0: port enable: SUCCESS
scsi 0:0:0:0: Direct-Access ATA WDC WD3000F9YZ-0 1A01 PQ: 0 ANSI: 6
scsi 0:0:0:0: SATA: handle(0x000c), sas_addr(0x4433221103000000), phy(3), device_name(0x50014ee3001eb3a4)
scsi 0:0:0:0: SATA: enclosure_logical_id(0x500062b0002b4948), slot(28)
scsi 0:0:0:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
scsi 0:0:0:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
scsi 0:0:1:0: Direct-Access ATA WDC WD3000F9YZ-0 1A01 PQ: 0 ANSI: 6
scsi 0:0:1:0: SATA: handle(0x0009), sas_addr(0x4433221100000000), phy(0), device_name(0x50014ee058e6348e)
scsi 0:0:1:0: SATA: enclosure_logical_id(0x500062b0002b4948), slot(31)
scsi 0:0:1:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
scsi 0:0:1:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
scsi 0:0:2:0: Direct-Access ATA WDC WD3000F9YZ-0 1A01 PQ: 0 ANSI: 6
scsi 0:0:2:0: SATA: handle(0x000a), sas_addr(0x4433221101000000), phy(1), device_name(0x50014ee0038cd9a4)
scsi 0:0:2:0: SATA: enclosure_logical_id(0x500062b0002b4948), slot(30)
scsi 0:0:2:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
scsi 0:0:2:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
scsi 0:0:3:0: Direct-Access ATA WDC WD3000F9YZ-0 1A01 PQ: 0 ANSI: 6
scsi 0:0:3:0: SATA: handle(0x000b), sas_addr(0x4433221102000000), phy(2), device_name(0x50014ee3556d4990)
scsi 0:0:3:0: SATA: enclosure_logical_id(0x500062b0002b4948), slot(29)
scsi 0:0:3:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
scsi 0:0:3:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
scsi 4:0:0:0: Direct-Access ATA Samsung SSD 840 DXM0 PQ: 0 ANSI: 5
ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata5.00: ATA-9: Samsung SSD 840 PRO Series, DXM05B0Q, max UDMA/133
ata5.00: 250069680 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
ata5.00: configured for UDMA/133
scsi 5:0:0:0: Direct-Access ATA Samsung SSD 840 DXM0 PQ: 0 ANSI: 5
ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata6.00: ATAPI: DV-28S-W, 1.2B, max UDMA/100
ata6.00: configured for UDMA/100
scsi 6:0:0:0: CD-ROM TEAC DV-28S-W 1.2B PQ: 0 ANSI: 5
sd 4:0:0:0: [sde] 250069680 512-byte logical blocks: (128 GB/119 GiB)
sd 5:0:0:0: [sdf] 250069680 512-byte logical blocks: (128 GB/119 GiB)
sd 4:0:0:0: [sde] Write Protect is off
sd 4:0:0:0: [sde] Mode Sense: 00 3a 00 00
sd 5:0:0:0: [sdf] Write Protect is off
sd 4:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 5:0:0:0: [sdf] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB)
sd 0:0:0:0: [sda] 4096-byte physical blocks
sd 5:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 0:0:1:0: [sdb] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB)
sd 0:0:1:0: [sdb] 4096-byte physical blocks
sd 0:0:2:0: [sdc] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB)
sd 0:0:2:0: [sdc] 4096-byte physical blocks
sd 0:0:3:0: [sdd] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB)
sd 0:0:3:0: [sdd] 4096-byte physical blocks
sde: sde1 sde2
sdf: sdf1 sdf2
sd 4:0:0:0: [sde] Attached SCSI disk
sd 5:0:0:0: [sdf] Attached SCSI disk
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 7f 00 10 08
sd 0:0:1:0: [sdb] Write Protect is off
sd 0:0:1:0: [sdb] Mode Sense: 7f 00 10 08
sd 0:0:3:0: [sdd] Write Protect is off
sd 0:0:3:0: [sdd] Mode Sense: 7f 00 10 08
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
sd 0:0:2:0: [sdc] Write Protect is off
sd 0:0:2:0: [sdc] Mode Sense: 7f 00 10 08
sd 0:0:3:0: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA
sd 0:0:1:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
sd 0:0:2:0: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA
sda: sda1 sda2 sda3 sda4
sdd: sdd1 sdd2 sdd3 sdd4
sdc: sdc1 sdc2 sdc3 sdc4
sdb: sdb1 sdb2 sdb3 sdb4
sd 0:0:0:0: [sda] Attached SCSI disk
sd 0:0:3:0: [sdd] Attached SCSI disk
sd 0:0:2:0: [sdc] Attached SCSI disk
sd 0:0:1:0: [sdb] Attached SCSI disk
sr0: scsi3-mmc drive: 24x/24x cd/rw xa/form2 cdda tray
cdrom: Uniform CD-ROM driver Revision: 3.20
sr 6:0:0:0: Attached scsi CD-ROM sr0
md: bind<sdc3>
md: bind<sdc2>
md: bind<sdb3>
md: bind<sdb2>
md: bind<sdd2>
md: bind<sda3>
md: bind<sda2>
md: raid10 personality registered for level 10
bio: create slab <bio-1> at 1
md/raid10:md1: active with 4 out of 4 devices
created bitmap (1 pages) for device md1
md1: bitmap initialized from disk: read 1/1 pages, set 0 of 617 bits
md1: detected capacity change from 0 to 41396731904
md1: unknown partition table
md: bind<sdd3>
md/raid10:md2: active with 4 out of 4 devices
md2: detected capacity change from 0 to 15998124032
md2: unknown partition table
EXT4-fs (md1): mounted filesystem with ordered data mode. Opts: (null)
dracut: Mounted root filesystem /dev/md1
SELinux: Disabled at runtime.
SELinux: Unregistering netfilter hooks
type=1404 audit(1384488235.609:2): selinux=0 auid=4294967295 ses=4294967295
dracut:
dracut: Switching root
udev: starting version 147
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 0:0:1:0: Attached scsi generic sg1 type 0
sd 0:0:2:0: Attached scsi generic sg2 type 0
sd 0:0:3:0: Attached scsi generic sg3 type 0
sd 4:0:0:0: Attached scsi generic sg4 type 0
sd 5:0:0:0: Attached scsi generic sg5 type 0
sr 6:0:0:0: Attached scsi generic sg6 type 5
EDAC MC: Ver: 2.1.0
EDAC MC1: Giving out device to 'i7core_edac.c' 'i7 core #1': DEV 0000:fe:03.0
EDAC PCI0: Giving out device to module 'i7core_edac' controller 'EDAC PCI controller': DEV '0000:fe:03.0' (POLLED)
EDAC MC0: Giving out device to 'i7core_edac.c' 'i7 core #0': DEV 0000:ff:03.0
EDAC PCI1: Giving out device to module 'i7core_edac' controller 'EDAC PCI controller': DEV '0000:ff:03.0' (POLLED)
EDAC i7core: Driver loaded, 2 memory controller(s) found.
ioatdma: Intel(R) QuickData Technology Driver 4.00
xen: registering gsi 43 triggering 0 polarity 1
xen: --> pirq=43 -> irq=43 (gsi=43)
xen: registering gsi 44 triggering 0 polarity 1
xen: --> pirq=44 -> irq=44 (gsi=44)
xen: registering gsi 45 triggering 0 polarity 1
xen: --> pirq=45 -> irq=45 (gsi=45)
xen: registering gsi 46 triggering 0 polarity 1
xen: --> pirq=46 -> irq=46 (gsi=46)
xen: registering gsi 43 triggering 0 polarity 1
Already setup the GSI :43
xen: registering gsi 44 triggering 0 polarity 1
Already setup the GSI :44
xen: registering gsi 45 triggering 0 polarity 1
Already setup the GSI :45
xen: registering gsi 46 triggering 0 polarity 1
Already setup the GSI :46
iTCO_vendor_support: vendor-support=0
iTCO_wdt: Intel TCO WatchDog Timer Driver v1.07
iTCO_wdt: Found a ICH10R TCO device (Version=2, TCOBASE=0x0860)
md: bind<sdd1>
iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)
md: bind<sda1>
md: bind<sdc1>
xen: registering gsi 18 triggering 0 polarity 1
Already setup the GSI :18
md: bind<sdb1>
md: raid1 personality registered for level 1
md/raid1:md0: active with 4 out of 4 mirrors
md0: detected capacity change from 0 to 298831872
md0: unknown partition table
input: PC Speaker as /devices/platform/pcspkr/input/input4
microcode: CPU0 sig=0x206c2, pf=0x1, revision=0x15
microcode: CPU1 sig=0x206c2, pf=0x1, revision=0x15
microcode: CPU2 sig=0x206c2, pf=0x1, revision=0x15
microcode: CPU3 sig=0x206c2, pf=0x1, revision=0x15
microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
SPL: Loaded module v0.6.2-1
zunicode: module license 'CDDL' taints kernel.
Disabling lock debugging due to kernel taint
ZFS: Loaded module v0.6.2-1, ZFS pool version 5000, ZFS filesystem version 5
zd0: p1 p2
zd16: p1 p2
zd32: p1 p2
zd48: p1 p2
SPL: using hostid 0x00000000
Event-channel device installed.
blktap_device_init: blktap device major 252
blktap_ring_init: blktap ring major: 249
EXT4-fs (md1): re-mounted. Opts: (null)
EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: (null)
Adding 15623164k swap on /dev/md2. Priority:-1 extents:1 across:15623164k
ip_tables: (C) 2000-2006 Netfilter Core Team
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
Bridge firewalling registered
ADDRCONF(NETDEV_UP): eth0: link is not ready
device eth0 entered promiscuous mode
ADDRCONF(NETDEV_UP): eth1: link is not ready
device eth1 entered promiscuous mode
ADDRCONF(NETDEV_UP): br0: link is not ready
igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
br0: topology change detected, propagating
br0: port 1(eth0) entered forwarding state
br0: port 1(eth0) entered forwarding state
ADDRCONF(NETDEV_CHANGE): br0: link becomes ready
igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
br1: topology change detected, propagating
br1: port 1(eth1) entered forwarding state
br1: port 1(eth1) entered forwarding state
xen-pciback: backend is vpci
br0: no IPv6 routers present
br1: no IPv6 routers present
eth0: no IPv6 routers present
eth1: no IPv6 routers present
device vif1.0 entered promiscuous mode
ADDRCONF(NETDEV_UP): vif1.0: link is not ready
xen-blkback:ring-ref 8, event-channel 13, protocol 1 (x86_64-abi)
ADDRCONF(NETDEV_CHANGE): vif1.0: link becomes ready
br0: topology change detected, propagating
br0: port 2(vif1.0) entered forwarding state
br0: port 2(vif1.0) entered forwarding state
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
device vif2.0 entered promiscuous mode
ADDRCONF(NETDEV_UP): vif2.0: link is not ready
device vif2.1 entered promiscuous mode
ADDRCONF(NETDEV_UP): vif2.1: link is not ready
device vif2.0-emu entered promiscuous mode
br0: topology change detected, propagating
br0: port 4(vif2.0-emu) entered forwarding state
br0: port 4(vif2.0-emu) entered forwarding state
device vif2.1-emu entered promiscuous mode
br1: topology change detected, propagating
br1: port 3(vif2.1-emu) entered forwarding state
br1: port 3(vif2.1-emu) entered forwarding state
vif1.0: no IPv6 routers present
vif2.1-emu: no IPv6 routers present
vif2.0-emu: no IPv6 routers present
br1: port 3(vif2.1-emu) entered disabled state
br1: port 3(vif2.1-emu) entered disabled state
device vif2.1-emu left promiscuous mode
br1: port 3(vif2.1-emu) entered disabled state
br0: port 4(vif2.0-emu) entered disabled state
device vif2.0-emu left promiscuous mode
br0: port 4(vif2.0-emu) entered disabled state
xen-blkback:ring-ref 16383, event-channel 8, protocol 1 (unspecified, assuming native)
ADDRCONF(NETDEV_CHANGE): vif2.0: link becomes ready
br0: topology change detected, propagating
br0: port 3(vif2.0) entered forwarding state
br0: port 3(vif2.0) entered forwarding state
vif2.0: no IPv6 routers present
block tda: sector-size: 512/512+0 capacity: 6464 discard: 0+0 flush: 0x0
device vif3.0 entered promiscuous mode
ADDRCONF(NETDEV_UP): vif3.0: link is not ready
device vif3.0-emu entered promiscuous mode
br0: topology change detected, propagating
br0: port 5(vif3.0-emu) entered forwarding state
br0: port 5(vif3.0-emu) entered forwarding state
vif3.0-emu: no IPv6 routers present
br0: port 5(vif3.0-emu) entered disabled state
device vif3.0-emu left promiscuous mode
br0: port 5(vif3.0-emu) entered disabled state
xen-blkback:ring-ref 16383, event-channel 8, protocol 1 (unspecified, assuming native)
xen-blkback:ring-ref 16382, event-channel 9, protocol 1 (unspecified, assuming native)
ADDRCONF(NETDEV_CHANGE): vif3.0: link becomes ready
br0: topology change detected, propagating
br0: port 4(vif3.0) entered forwarding state
br0: port 4(vif3.0) entered forwarding state
vif3.0: no IPv6 routers present
INFO: task spl_system_task:1243 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
spl_system_task D ffff8805bfc73580 0 1243 2 0x00000000
ffff880598cdd020 0000000000000246 ffff880598cddfd8 0000000000013580
ffff880598cdc010 0000000000013580 0000000000013580 0000000000013580
ffff880598cddfd8 0000000000013580 ffff88059f5e0ac0 ffff880597a49580
Call Trace:
[<ffffffff815798c9>] schedule+0x29/0x70
[<ffffffffa0369285>] cv_wait_common+0x105/0x1c0 [spl]
[<ffffffff810838c3>] ? finish_task_switch+0x53/0xe0
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffffa0369395>] __cv_wait+0x15/0x20 [spl]
[<ffffffffa043f3a5>] traverse_prefetcher+0xa5/0x150 [zfs]
[<ffffffffa043f739>] traverse_visitbp+0x2e9/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa0424fb0>] ? arc_buf_evict+0x2d0/0x2d0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fa63>] traverse_visitbp+0x613/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa04278b7>] ? arc_get_data_buf+0x307/0x480 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fb21>] traverse_visitbp+0x6d1/0x830 [zfs]
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffffa0440183>] traverse_prefetch_thread+0x83/0xc0 [zfs]
[<ffffffffa043f300>] ? prefetch_dnode_metadata+0xb0/0xb0 [zfs]
[<ffffffffa03624ed>] taskq_thread+0x22d/0x4f0 [spl]
[<ffffffff81087230>] ? try_to_wake_up+0x2c0/0x2c0
[<ffffffffa03622c0>] ? task_expire+0x120/0x120 [spl]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
INFO: task spl_system_task:1240 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
spl_system_task D ffff8805bfc13580 0 1240 2 0x00000000
ffff880598cd7020 0000000000000246 ffff880598cd7fd8 0000000000013580
ffff880598cd6010 0000000000013580 0000000000013580 0000000000013580
ffff880598cd7fd8 0000000000013580 ffffffff81a13020 ffff880598e9a040
Call Trace:
[<ffffffff815798c9>] schedule+0x29/0x70
[<ffffffffa0369285>] cv_wait_common+0x105/0x1c0 [spl]
[<ffffffff810838c3>] ? finish_task_switch+0x53/0xe0
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffffa0369395>] __cv_wait+0x15/0x20 [spl]
[<ffffffffa043f3a5>] traverse_prefetcher+0xa5/0x150 [zfs]
[<ffffffffa043f739>] traverse_visitbp+0x2e9/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffff81583d4b>] ? xen_hypervisor_callback+0x1b/0x20
[<ffffffff8157acca>] ? error_exit+0x2a/0x60
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fa63>] traverse_visitbp+0x613/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa04278b7>] ? arc_get_data_buf+0x307/0x480 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fb21>] traverse_visitbp+0x6d1/0x830 [zfs]
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffffa0440183>] traverse_prefetch_thread+0x83/0xc0 [zfs]
[<ffffffffa043f300>] ? prefetch_dnode_metadata+0xb0/0xb0 [zfs]
[<ffffffffa03624ed>] taskq_thread+0x22d/0x4f0 [spl]
[<ffffffff81087230>] ? try_to_wake_up+0x2c0/0x2c0
[<ffffffffa03622c0>] ? task_expire+0x120/0x120 [spl]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
INFO: task spl_system_task:1241 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
spl_system_task D ffff8805bfc33580 0 1241 2 0x00000000
ffff880598cd9020 0000000000000246 ffff880598cd9fd8 0000000000013580
ffff880598cd8010 0000000000013580 0000000000013580 0000000000013580
ffff880598cd9fd8 0000000000013580 ffff88059f57a040 ffff880599c34b40
Call Trace:
[<ffffffff815798c9>] schedule+0x29/0x70
[<ffffffffa0369285>] cv_wait_common+0x105/0x1c0 [spl]
[<ffffffff810838c3>] ? finish_task_switch+0x53/0xe0
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffffa0369395>] __cv_wait+0x15/0x20 [spl]
[<ffffffffa043f3a5>] traverse_prefetcher+0xa5/0x150 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa043f739>] traverse_visitbp+0x2e9/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa0424fb0>] ? arc_buf_evict+0x2d0/0x2d0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fa63>] traverse_visitbp+0x613/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa04278b7>] ? arc_get_data_buf+0x307/0x480 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fb21>] traverse_visitbp+0x6d1/0x830 [zfs]
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffffa0440183>] traverse_prefetch_thread+0x83/0xc0 [zfs]
[<ffffffffa043f300>] ? prefetch_dnode_metadata+0xb0/0xb0 [zfs]
[<ffffffffa03624ed>] taskq_thread+0x22d/0x4f0 [spl]
[<ffffffff81087230>] ? try_to_wake_up+0x2c0/0x2c0
[<ffffffffa03622c0>] ? task_expire+0x120/0x120 [spl]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
INFO: task spl_system_task:1242 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
spl_system_task D ffff8805bfc53580 0 1242 2 0x00000000
ffff880598cdb1e0 0000000000000246 ffff880598cdbfd8 0000000000013580
ffff880598cda010 0000000000013580 0000000000013580 0000000000013580
ffff880598cdbfd8 0000000000013580 ffff88059f5e1500 ffff880598b89500
Call Trace:
[<ffffffff815798c9>] schedule+0x29/0x70
[<ffffffffa0369285>] cv_wait_common+0x105/0x1c0 [spl]
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffffa0369395>] __cv_wait+0x15/0x20 [spl]
[<ffffffffa043f3a5>] traverse_prefetcher+0xa5/0x150 [zfs]
[<ffffffffa043f739>] traverse_visitbp+0x2e9/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fa63>] traverse_visitbp+0x613/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa04278b7>] ? arc_get_data_buf+0x307/0x480 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fb21>] traverse_visitbp+0x6d1/0x830 [zfs]
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffffa0440183>] traverse_prefetch_thread+0x83/0xc0 [zfs]
[<ffffffffa043f300>] ? prefetch_dnode_metadata+0xb0/0xb0 [zfs]
[<ffffffffa03624ed>] taskq_thread+0x22d/0x4f0 [spl]
[<ffffffff81087230>] ? try_to_wake_up+0x2c0/0x2c0
[<ffffffffa03622c0>] ? task_expire+0x120/0x120 [spl]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
INFO: task spl_system_task:1243 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
spl_system_task D ffff8805bfc73580 0 1243 2 0x00000000
ffff880598cdd020 0000000000000246 ffff880598cddfd8 0000000000013580
ffff880598cdc010 0000000000013580 0000000000013580 0000000000013580
ffff880598cddfd8 0000000000013580 ffff8805988e54c0 ffff880597a49580
Call Trace:
[<ffffffff815798c9>] schedule+0x29/0x70
[<ffffffffa0369285>] cv_wait_common+0x105/0x1c0 [spl]
[<ffffffff810838c3>] ? finish_task_switch+0x53/0xe0
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffff815781ae>] ? mutex_lock+0x1e/0x50
[<ffffffffa0369395>] __cv_wait+0x15/0x20 [spl]
[<ffffffffa043f3a5>] traverse_prefetcher+0xa5/0x150 [zfs]
[<ffffffff815781ae>] ? mutex_lock+0x1e/0x50
[<ffffffffa043f739>] traverse_visitbp+0x2e9/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa0424fb0>] ? arc_buf_evict+0x2d0/0x2d0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fa63>] traverse_visitbp+0x613/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa04278b7>] ? arc_get_data_buf+0x307/0x480 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fb21>] traverse_visitbp+0x6d1/0x830 [zfs]
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffffa0440183>] traverse_prefetch_thread+0x83/0xc0 [zfs]
[<ffffffffa043f300>] ? prefetch_dnode_metadata+0xb0/0xb0 [zfs]
[<ffffffffa03624ed>] taskq_thread+0x22d/0x4f0 [spl]
[<ffffffff81087230>] ? try_to_wake_up+0x2c0/0x2c0
[<ffffffffa03622c0>] ? task_expire+0x120/0x120 [spl]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
device vif4.1 entered promiscuous mode
device vif4.0 entered promiscuous mode
ADDRCONF(NETDEV_UP): vif4.1: link is not ready
ADDRCONF(NETDEV_UP): vif4.0: link is not ready
xen-blkback:ring-ref 8, event-channel 13, protocol 1 (x86_64-abi)
ADDRCONF(NETDEV_CHANGE): vif4.0: link becomes ready
br0: topology change detected, propagating
br0: port 5(vif4.0) entered forwarding state
br0: port 5(vif4.0) entered forwarding state
ADDRCONF(NETDEV_CHANGE): vif4.1: link becomes ready
br1: topology change detected, propagating
br1: port 3(vif4.1) entered forwarding state
br1: port 3(vif4.1) entered forwarding state
vif4.1: no IPv6 routers present
vif4.0: no IPv6 routers present
INFO: task spl_system_task:1241 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
spl_system_task D ffff8805bfc33580 0 1241 2 0x00000000
ffff880598cd9100 0000000000000246 ffff880598cd9fd8 0000000000013580
ffff880598cd8010 0000000000013580 0000000000013580 0000000000013580
ffff880598cd9fd8 0000000000013580 ffff88059267e080 ffff880599c34b40
Call Trace:
[<ffffffff815798c9>] schedule+0x29/0x70
[<ffffffffa0369285>] cv_wait_common+0x105/0x1c0 [spl]
[<ffffffff810838c3>] ? finish_task_switch+0x53/0xe0
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffffa0369395>] __cv_wait+0x15/0x20 [spl]
[<ffffffffa043f3a5>] traverse_prefetcher+0xa5/0x150 [zfs]
[<ffffffffa043f739>] traverse_visitbp+0x2e9/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa0424fb0>] ? arc_buf_evict+0x2d0/0x2d0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fa63>] traverse_visitbp+0x613/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa04278b7>] ? arc_get_data_buf+0x307/0x480 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fb21>] traverse_visitbp+0x6d1/0x830 [zfs]
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffffa0440183>] traverse_prefetch_thread+0x83/0xc0 [zfs]
[<ffffffffa043f300>] ? prefetch_dnode_metadata+0xb0/0xb0 [zfs]
[<ffffffffa03624ed>] taskq_thread+0x22d/0x4f0 [spl]
[<ffffffff81087230>] ? try_to_wake_up+0x2c0/0x2c0
[<ffffffffa03622c0>] ? task_expire+0x120/0x120 [spl]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
INFO: task spl_system_task:1241 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
spl_system_task D ffff8805bfc33580 0 1241 2 0x00000000
ffff880598cd9100 0000000000000246 ffff880598cd9fd8 0000000000013580
ffff880598cd8010 0000000000013580 0000000000013580 0000000000013580
ffff880598cd9fd8 0000000000013580 ffff88059267e080 ffff880599c34b40
Call Trace:
[<ffffffff815798c9>] schedule+0x29/0x70
[<ffffffffa0369285>] cv_wait_common+0x105/0x1c0 [spl]
[<ffffffff810838c3>] ? finish_task_switch+0x53/0xe0
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffffa0369395>] __cv_wait+0x15/0x20 [spl]
[<ffffffffa043f3a5>] traverse_prefetcher+0xa5/0x150 [zfs]
[<ffffffffa043f739>] traverse_visitbp+0x2e9/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa0424fb0>] ? arc_buf_evict+0x2d0/0x2d0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fa63>] traverse_visitbp+0x613/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa04278b7>] ? arc_get_data_buf+0x307/0x480 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fb21>] traverse_visitbp+0x6d1/0x830 [zfs]
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffffa0440183>] traverse_prefetch_thread+0x83/0xc0 [zfs]
[<ffffffffa043f300>] ? prefetch_dnode_metadata+0xb0/0xb0 [zfs]
[<ffffffffa03624ed>] taskq_thread+0x22d/0x4f0 [spl]
[<ffffffff81087230>] ? try_to_wake_up+0x2c0/0x2c0
[<ffffffffa03622c0>] ? task_expire+0x120/0x120 [spl]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
INFO: task spl_system_task:1241 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
spl_system_task D ffff8805bfc33580 0 1241 2 0x00000000
ffff880598cd9100 0000000000000246 ffff880598cd9fd8 0000000000013580
ffff880598cd8010 0000000000013580 0000000000013580 0000000000013580
ffff880598cd9fd8 0000000000013580 ffff88059267e080 ffff880599c34b40
Call Trace:
[<ffffffff815798c9>] schedule+0x29/0x70
[<ffffffffa0369285>] cv_wait_common+0x105/0x1c0 [spl]
[<ffffffff810838c3>] ? finish_task_switch+0x53/0xe0
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffffa0369395>] __cv_wait+0x15/0x20 [spl]
[<ffffffffa043f3a5>] traverse_prefetcher+0xa5/0x150 [zfs]
[<ffffffffa043f739>] traverse_visitbp+0x2e9/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa0424fb0>] ? arc_buf_evict+0x2d0/0x2d0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fa63>] traverse_visitbp+0x613/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa04c7a53>] ? zio_buf_alloc+0x23/0x30 [zfs]
[<ffffffffa04278b7>] ? arc_get_data_buf+0x307/0x480 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fb21>] traverse_visitbp+0x6d1/0x830 [zfs]
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffffa0440183>] traverse_prefetch_thread+0x83/0xc0 [zfs]
[<ffffffffa043f300>] ? prefetch_dnode_metadata+0xb0/0xb0 [zfs]
[<ffffffffa03624ed>] taskq_thread+0x22d/0x4f0 [spl]
[<ffffffff81087230>] ? try_to_wake_up+0x2c0/0x2c0
[<ffffffffa03622c0>] ? task_expire+0x120/0x120 [spl]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
INFO: task spl_system_task:1240 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
spl_system_task D ffff8805bfc13580 0 1240 2 0x00000000
ffff880598cd7020 0000000000000246 ffff880598cd7fd8 0000000000013580
ffff880598cd6010 0000000000013580 0000000000013580 0000000000013580
ffff880598cd7fd8 0000000000013580 ffffffff81a13020 ffff880598e9a040
Call Trace:
[<ffffffff815798c9>] schedule+0x29/0x70
[<ffffffffa0369285>] cv_wait_common+0x105/0x1c0 [spl]
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffffa0369395>] __cv_wait+0x15/0x20 [spl]
[<ffffffffa043f3a5>] traverse_prefetcher+0xa5/0x150 [zfs]
[<ffffffff815781ae>] ? mutex_lock+0x1e/0x50
[<ffffffffa043f739>] traverse_visitbp+0x2e9/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa0424fb0>] ? arc_buf_evict+0x2d0/0x2d0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffff81583d4b>] ? xen_hypervisor_callback+0x1b/0x20
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffff815781ae>] ? mutex_lock+0x1e/0x50
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fa63>] traverse_visitbp+0x613/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0427a13>] ? arc_get_data_buf+0x463/0x480 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fb21>] traverse_visitbp+0x6d1/0x830 [zfs]
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffffa0440183>] traverse_prefetch_thread+0x83/0xc0 [zfs]
[<ffffffffa043f300>] ? prefetch_dnode_metadata+0xb0/0xb0 [zfs]
[<ffffffffa03624ed>] taskq_thread+0x22d/0x4f0 [spl]
[<ffffffff81087230>] ? try_to_wake_up+0x2c0/0x2c0
[<ffffffffa03622c0>] ? task_expire+0x120/0x120 [spl]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
INFO: task spl_system_task:1240 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
spl_system_task D ffff8805bfc13580 0 1240 2 0x00000000
ffff880598cd7020 0000000000000246 ffff880598cd7fd8 0000000000013580
ffff880598cd6010 0000000000013580 0000000000013580 0000000000013580
ffff880598cd7fd8 0000000000013580 ffffffff81a13020 ffff880598e9a040
Call Trace:
[<ffffffff815798c9>] schedule+0x29/0x70
[<ffffffffa0369285>] cv_wait_common+0x105/0x1c0 [spl]
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffffa0369395>] __cv_wait+0x15/0x20 [spl]
[<ffffffffa043f3a5>] traverse_prefetcher+0xa5/0x150 [zfs]
[<ffffffff815781ae>] ? mutex_lock+0x1e/0x50
[<ffffffffa043f739>] traverse_visitbp+0x2e9/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa0424fb0>] ? arc_buf_evict+0x2d0/0x2d0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffff81583d4b>] ? xen_hypervisor_callback+0x1b/0x20
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffff815781ae>] ? mutex_lock+0x1e/0x50
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fa63>] traverse_visitbp+0x613/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0428989>] ? arc_read+0x6c9/0x8c0 [zfs]
[<ffffffffa043f8dd>] traverse_visitbp+0x48d/0x830 [zfs]
[<ffffffffa0427a13>] ? arc_get_data_buf+0x463/0x480 [zfs]
[<ffffffffa0440074>] traverse_dnode+0x74/0x100 [zfs]
[<ffffffffa043fb21>] traverse_visitbp+0x6d1/0x830 [zfs]
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffffa0440183>] traverse_prefetch_thread+0x83/0xc0 [zfs]
[<ffffffffa043f300>] ? prefetch_dnode_metadata+0xb0/0xb0 [zfs]
[<ffffffffa03624ed>] taskq_thread+0x22d/0x4f0 [spl]
[<ffffffff81087230>] ? try_to_wake_up+0x2c0/0x2c0
[<ffffffffa03622c0>] ? task_expire+0x120/0x120 [spl]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
qemu-dm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
qemu-dm cpuset=/ mems_allowed=0
Pid: 2553, comm: qemu-dm Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff81159156>] alloc_pages_current+0xb6/0x120
[<ffffffff811143a7>] __page_cache_alloc+0xb7/0xd0
[<ffffffff811153ef>] filemap_fault+0x1af/0x4b0
[<ffffffff81189a40>] ? __pollwait+0xf0/0xf0
[<ffffffff8113d9a2>] __do_fault+0x72/0x5a0
[<ffffffff8113dfb7>] handle_pte_fault+0xe7/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff8118a5d1>] ? poll_select_copy_remaining+0x101/0x150
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:121 inactive_anon:18 isolated_anon:0
active_file:7 inactive_file:27 isolated_file:0
unevictable:651 dirty:0 writeback:0 unstable:0
free:26313 slab_reclaimable:2609 slab_unreclaimable:295551
mapped:510 shmem:1 pagetables:733 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:78948kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:64kB active_file:0kB inactive_file:8kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:1360kB slab_unreclaimable:486984kB kernel_stack:40kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:152 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:10412kB min:16488kB low:20608kB high:24732kB active_anon:484kB inactive_anon:8kB active_file:28kB inactive_file:100kB unevictable:2604kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2604kB dirty:0kB writeback:0kB mapped:2036kB shmem:4kB slab_reclaimable:9076kB slab_unreclaimable:695220kB kernel_stack:2040kB pagetables:2932kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:989 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 95*4kB 73*8kB 71*16kB 34*32kB 30*64kB 27*128kB 17*256kB 9*512kB 10*1024kB 15*2048kB 5*4096kB = 78964kB
Node 0 Normal: 2109*4kB 104*8kB 10*16kB 4*32kB 2*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 10580kB
645 total pagecache pages
91 pages in swap cache
Swap cache stats: add 8330, delete 8239, find 681/934
Free swap = 15599184kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
4235 pages shared
4969800 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 57 2 -17 -1000 auditd
[ 2199] 0 2199 60745 120 0 0 0 rsyslogd
[ 2213] 0 2213 1183 93 1 0 0 mdadm
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2279] 0 2279 2688 114 0 0 0 xenstored
[ 2285] 0 2285 21644 130 0 0 0 xenconsoled
[ 2290] 0 2290 52930 110 3 0 0 qemu-system-i38
[ 2308] 0 2308 21142 90 2 0 0 sendmail
[ 2316] 51 2316 20073 91 2 0 0 sendmail
[ 2325] 0 2325 29321 131 0 0 0 crond
[ 2505] 0 2505 25510 45 1 0 0 xl
[ 2553] 0 2553 57939 159 0 0 0 qemu-dm
[ 2792] 0 2792 25469 43 2 0 0 xl
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 0 0 0 tapdisk
[ 2954] 0 2954 62805 159 3 0 0 qemu-dm
[ 3075] 0 3075 25487 36 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 45 2 0 0 xl
[ 3497] 0 3497 24493 198 3 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 170 2 0 0 python
Out of memory: Kill process 2199 (rsyslogd) score 1 or sacrifice child
Killed process 2199 (rsyslogd) total-vm:242980kB, anon-rss:0kB, file-rss:480kB
netback/2 invoked oom-killer: gfp_mask=0x1d0, order=0, oom_adj=0, oom_score_adj=0
netback/2 cpuset=/ mems_allowed=0
Pid: 1580, comm: netback/2 Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff81159156>] alloc_pages_current+0xb6/0x120
[<ffffffffa05a07dc>] xen_netbk_tx_build_gops+0x49c/0x920 [xen_netback]
[<ffffffff8116210d>] ? kfree+0x12d/0x320
[<ffffffff81004c60>] ? xen_mc_flush+0xb0/0x1b0
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffff810838c3>] ? finish_task_switch+0x53/0xe0
[<ffffffff8157a307>] ? _raw_spin_unlock_irqrestore+0x17/0x20
[<ffffffffa05a0d0c>] xen_netbk_kthread+0xac/0x1d0 [xen_netback]
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffffa05a0c60>] ? xen_netbk_tx_build_gops+0x920/0x920 [xen_netback]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:86 inactive_anon:0 isolated_anon:0
active_file:31 inactive_file:91 isolated_file:4
unevictable:651 dirty:0 writeback:0 unstable:0
free:27542 slab_reclaimable:1606 slab_unreclaimable:314485
mapped:660 shmem:1 pagetables:714 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:79280kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:128kB inactive_file:228kB unevictable:0kB isolated(anon):0kB isolated(file):16kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:256kB shmem:0kB slab_reclaimable:52kB slab_unreclaimable:503048kB kernel_stack:40kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:562 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:14996kB min:16488kB low:20608kB high:24732kB active_anon:456kB inactive_anon:56kB active_file:0kB inactive_file:136kB unevictable:2604kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2604kB dirty:0kB writeback:0kB mapped:2384kB shmem:4kB slab_reclaimable:6372kB slab_unreclaimable:754892kB kernel_stack:2008kB pagetables:2856kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 31*4kB 44*8kB 42*16kB 26*32kB 32*64kB 15*128kB 10*256kB 7*512kB 8*1024kB 11*2048kB 9*4096kB = 79676kB
Node 0 Normal: 1362*4kB 441*8kB 66*16kB 15*32kB 7*64kB 5*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 15184kB
649 total pagecache pages
79 pages in swap cache
Swap cache stats: add 26424, delete 26345, find 7481/10072
Free swap = 15599588kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
4108 pages shared
4934298 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 77 3 -17 -1000 auditd
[ 2213] 0 2213 1183 93 1 0 0 mdadm
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2279] 0 2279 2688 114 0 0 0 xenstored
[ 2285] 0 2285 21644 130 0 0 0 xenconsoled
[ 2290] 0 2290 52930 91 2 0 0 qemu-system-i38
[ 2308] 0 2308 21142 94 2 0 0 sendmail
[ 2316] 51 2316 20073 69 3 0 0 sendmail
[ 2325] 0 2325 29321 131 1 0 0 crond
[ 2505] 0 2505 25510 45 1 0 0 xl
[ 2553] 0 2553 57939 159 3 0 0 qemu-dm
[ 2792] 0 2792 25469 43 2 0 0 xl
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 139 1 0 0 qemu-dm
[ 3075] 0 3075 25487 36 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 45 2 0 0 xl
[ 3497] 0 3497 24530 187 3 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 166 2 0 0 python
Out of memory: Kill process 2213 (mdadm) score 1 or sacrifice child
Killed process 2213 (mdadm) total-vm:4732kB, anon-rss:0kB, file-rss:372kB
qemu-dm invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
qemu-dm cpuset=/ mems_allowed=0
Pid: 2954, comm: qemu-dm Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff81113f90>] ? sleep_on_page+0x20/0x20
[<ffffffff8115a126>] alloc_pages_vma+0xc6/0x1a0
[<ffffffff8114d7d2>] read_swap_cache_async+0xf2/0x170
[<ffffffff81074b20>] ? autoremove_wake_function+0x40/0x40
[<ffffffff8114d8de>] swapin_readahead+0x8e/0xd0
[<ffffffff8113d415>] do_swap_page+0x115/0x630
[<ffffffff8111405a>] ? unlock_page+0x2a/0x40
[<ffffffff8113dda9>] ? __do_fault+0x479/0x5a0
[<ffffffff8113e07a>] handle_pte_fault+0x1aa/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff8117698a>] ? do_sync_write+0xea/0x130
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:50 inactive_anon:1 isolated_anon:0
active_file:29 inactive_file:0 isolated_file:0
unevictable:651 dirty:0 writeback:0 unstable:0
free:25517 slab_reclaimable:1606 slab_unreclaimable:315405
mapped:533 shmem:1 pagetables:706 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:77972kB min:2568kB low:3208kB high:3852kB active_anon:24kB inactive_anon:32kB active_file:8kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:52kB slab_unreclaimable:503728kB kernel_stack:40kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1364 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:8204kB min:16488kB low:20608kB high:24732kB active_anon:176kB inactive_anon:0kB active_file:108kB inactive_file:0kB unevictable:2604kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2604kB dirty:0kB writeback:0kB mapped:2128kB shmem:4kB slab_reclaimable:6372kB slab_unreclaimable:757892kB kernel_stack:2000kB pagetables:2824kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1094 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 19*4kB 18*8kB 22*16kB 24*32kB 22*64kB 13*128kB 10*256kB 7*512kB 8*1024kB 11*2048kB 9*4096kB = 78140kB
Node 0 Normal: 1037*4kB 1*8kB 2*16kB 4*32kB 4*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 8540kB
684 total pagecache pages
0 pages in swap cache
Swap cache stats: add 27701, delete 27701, find 7929/10687
Free swap = 15599776kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
4007 pages shared
4934120 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 77 3 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2279] 0 2279 2688 114 0 0 0 xenstored
[ 2285] 0 2285 21644 130 0 0 0 xenconsoled
[ 2290] 0 2290 52930 92 2 0 0 qemu-system-i38
[ 2308] 0 2308 21142 94 2 0 0 sendmail
[ 2316] 51 2316 20073 69 3 0 0 sendmail
[ 2325] 0 2325 29321 131 1 0 0 crond
[ 2505] 0 2505 25510 45 1 0 0 xl
[ 2553] 0 2553 57939 161 3 0 0 qemu-dm
[ 2792] 0 2792 25469 43 2 0 0 xl
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 160 2 0 0 qemu-dm
[ 3075] 0 3075 25487 36 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 45 2 0 0 xl
[ 3497] 0 3497 24530 190 3 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 170 3 0 0 python
Out of memory: Kill process 2279 (xenstored) score 1 or sacrifice child
Killed process 2279 (xenstored) total-vm:10752kB, anon-rss:0kB, file-rss:456kB
qemu-system-i38 invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
qemu-system-i38 cpuset=/ mems_allowed=0
Pid: 2292, comm: qemu-system-i38 Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff81159156>] alloc_pages_current+0xb6/0x120
[<ffffffff811143a7>] __page_cache_alloc+0xb7/0xd0
[<ffffffff811153ef>] filemap_fault+0x1af/0x4b0
[<ffffffff8113d9a2>] __do_fault+0x72/0x5a0
[<ffffffff8113dfb7>] handle_pte_fault+0xe7/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff81176aba>] ? do_sync_read+0xea/0x130
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff8120897b>] ? security_file_permission+0x8b/0x90
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:50 inactive_anon:1 isolated_anon:0
active_file:0 inactive_file:0 isolated_file:0
unevictable:651 dirty:0 writeback:0 unstable:0
free:27268 slab_reclaimable:1606 slab_unreclaimable:315405
mapped:533 shmem:1 pagetables:706 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:79128kB min:2568kB low:3208kB high:3852kB active_anon:24kB inactive_anon:32kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:52kB slab_unreclaimable:503728kB kernel_stack:40kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1630 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:14052kB min:16488kB low:20608kB high:24732kB active_anon:176kB inactive_anon:0kB active_file:84kB inactive_file:0kB unevictable:2604kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2604kB dirty:0kB writeback:0kB mapped:2128kB shmem:4kB slab_reclaimable:6372kB slab_unreclaimable:757892kB kernel_stack:2000kB pagetables:2824kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1955 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 28*4kB 83*8kB 48*16kB 28*32kB 23*64kB 13*128kB 10*256kB 7*512kB 8*1024kB 11*2048kB 9*4096kB = 79304kB
Node 0 Normal: 1480*4kB 103*8kB 44*16kB 31*32kB 16*64kB 6*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 14072kB
684 total pagecache pages
0 pages in swap cache
Swap cache stats: add 27709, delete 27709, find 7930/10691
Free swap = 15600224kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
3906 pages shared
4931965 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 77 3 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2285] 0 2285 21644 130 0 0 0 xenconsoled
[ 2290] 0 2290 52930 92 2 0 0 qemu-system-i38
[ 2308] 0 2308 21142 94 2 0 0 sendmail
[ 2316] 51 2316 20073 69 3 0 0 sendmail
[ 2325] 0 2325 29321 131 1 0 0 crond
[ 2505] 0 2505 25510 44 1 0 0 xl
[ 2553] 0 2553 57939 160 3 0 0 qemu-dm
[ 2792] 0 2792 25469 43 2 0 0 xl
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 159 3 0 0 qemu-dm
[ 3075] 0 3075 25487 35 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 45 2 0 0 xl
[ 3497] 0 3497 24530 190 3 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 170 3 0 0 python
Out of memory: Kill process 2285 (xenconsoled) score 1 or sacrifice child
Killed process 2285 (xenconsoled) total-vm:86576kB, anon-rss:0kB, file-rss:520kB
xl invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
xl cpuset=/ mems_allowed=0
Pid: 2793, comm: xl Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff8115a126>] alloc_pages_vma+0xc6/0x1a0
[<ffffffff81113f90>] ? sleep_on_page+0x20/0x20
[<ffffffff8114d7d2>] read_swap_cache_async+0xf2/0x170
[<ffffffff8114d8de>] swapin_readahead+0x8e/0xd0
[<ffffffff8113d415>] do_swap_page+0x115/0x630
[<ffffffff8111405a>] ? unlock_page+0x2a/0x40
[<ffffffff8113dda9>] ? __do_fault+0x479/0x5a0
[<ffffffff8113e07a>] handle_pte_fault+0x1aa/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff81176aba>] ? do_sync_read+0xea/0x130
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff8120897b>] ? security_file_permission+0x8b/0x90
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:50 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:41 isolated_file:0
unevictable:651 dirty:0 writeback:0 unstable:0
free:25888 slab_reclaimable:1606 slab_unreclaimable:315570
mapped:533 shmem:1 pagetables:706 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:78216kB min:2568kB low:3208kB high:3852kB active_anon:24kB inactive_anon:28kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:52kB slab_unreclaimable:503728kB kernel_stack:40kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1473 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:9444kB min:16488kB low:20608kB high:24732kB active_anon:176kB inactive_anon:0kB active_file:0kB inactive_file:184kB unevictable:2604kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2604kB dirty:0kB writeback:0kB mapped:2128kB shmem:4kB slab_reclaimable:6372kB slab_unreclaimable:758552kB kernel_stack:2000kB pagetables:2824kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2116 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 22*4kB 15*8kB 18*16kB 29*32kB 23*64kB 14*128kB 10*256kB 7*512kB 8*1024kB 11*2048kB 9*4096kB = 78416kB
Node 0 Normal: 1053*4kB 12*8kB 10*16kB 17*32kB 10*64kB 2*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 9748kB
629 total pagecache pages
0 pages in swap cache
Swap cache stats: add 28026, delete 28026, find 7983/10786
Free swap = 15600412kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
3759 pages shared
4934290 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 77 3 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2290] 0 2290 52930 87 3 0 0 qemu-system-i38
[ 2308] 0 2308 21142 94 2 0 0 sendmail
[ 2316] 51 2316 20073 69 3 0 0 sendmail
[ 2325] 0 2325 29321 131 1 0 0 crond
[ 2505] 0 2505 25510 40 1 0 0 xl
[ 2553] 0 2553 57939 156 3 0 0 qemu-dm
[ 2792] 0 2792 25469 38 2 0 0 xl
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 153 0 0 0 qemu-dm
[ 3075] 0 3075 25487 30 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 40 2 0 0 xl
[ 3497] 0 3497 24530 190 3 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 170 0 0 0 python
Out of memory: Kill process 2290 (qemu-system-i38) score 1 or sacrifice child
Killed process 2290 (qemu-system-i38) total-vm:211720kB, anon-rss:0kB, file-rss:348kB
xl invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
xl cpuset=/ mems_allowed=0
Pid: 2506, comm: xl Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff81159156>] alloc_pages_current+0xb6/0x120
[<ffffffff811143a7>] __page_cache_alloc+0xb7/0xd0
[<ffffffff811153ef>] filemap_fault+0x1af/0x4b0
[<ffffffff8111405a>] ? unlock_page+0x2a/0x40
[<ffffffff8113d9a2>] __do_fault+0x72/0x5a0
[<ffffffff8113dfb7>] handle_pte_fault+0xe7/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff81176aba>] ? do_sync_read+0xea/0x130
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff8120897b>] ? security_file_permission+0x8b/0x90
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 30
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 60
active_anon:50 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:0 isolated_file:0
unevictable:651 dirty:0 writeback:0 unstable:0
free:26997 slab_reclaimable:1606 slab_unreclaimable:315625
mapped:533 shmem:1 pagetables:706 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:78528kB min:2568kB low:3208kB high:3852kB active_anon:24kB inactive_anon:28kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:52kB slab_unreclaimable:503728kB kernel_stack:40kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1448 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:13568kB min:16488kB low:20608kB high:24732kB active_anon:176kB inactive_anon:0kB active_file:4kB inactive_file:0kB unevictable:2604kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2604kB dirty:0kB writeback:0kB mapped:2128kB shmem:4kB slab_reclaimable:6372kB slab_unreclaimable:758772kB kernel_stack:2000kB pagetables:2824kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:356 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 26*4kB 17*8kB 20*16kB 31*32kB 25*64kB 14*128kB 10*256kB 7*512kB 8*1024kB 11*2048kB 9*4096kB = 78672kB
Node 0 Normal: 1020*4kB 15*8kB 27*16kB 41*32kB 29*64kB 14*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 13432kB
629 total pagecache pages
1 pages in swap cache
Swap cache stats: add 28042, delete 28041, find 7991/10795
Free swap = 15600412kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
3759 pages shared
4933018 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 77 3 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2292] 0 2290 52930 108 0 0 0 qemu-system-i38
[ 2308] 0 2308 21142 94 2 0 0 sendmail
[ 2316] 51 2316 20073 69 3 0 0 sendmail
[ 2325] 0 2325 29321 131 1 0 0 crond
[ 2505] 0 2505 25510 40 1 0 0 xl
[ 2553] 0 2553 57939 156 3 0 0 qemu-dm
[ 2792] 0 2792 25469 38 2 0 0 xl
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 153 0 0 0 qemu-dm
[ 3075] 0 3075 25487 30 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 40 2 0 0 xl
[ 3497] 0 3497 24530 190 3 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 170 0 0 0 python
Out of memory: Kill process 2292 (qemu-system-i38) score 1 or sacrifice child
Killed process 2292 (qemu-system-i38) total-vm:211720kB, anon-rss:0kB, file-rss:432kB
qemu-dm invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0
qemu-dm cpuset=/ mems_allowed=0
Pid: 2954, comm: qemu-dm Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff81159156>] alloc_pages_current+0xb6/0x120
[<ffffffff8111aaee>] __get_free_pages+0xe/0x50
[<ffffffffa0587796>] evtchn_write+0x36/0x140 [xen_evtchn]
[<ffffffff81176f38>] vfs_write+0xc8/0x190
[<ffffffff81177101>] sys_write+0x51/0x90
[<ffffffff81582929>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 29
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:53 isolated_file:0
unevictable:650 dirty:0 writeback:0 unstable:0
free:26004 slab_reclaimable:1602 slab_unreclaimable:314939
mapped:427 shmem:0 pagetables:614 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:78580kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:28kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:52kB slab_unreclaimable:503236kB kernel_stack:40kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:422 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:9544kB min:16488kB low:20608kB high:24732kB active_anon:0kB inactive_anon:60kB active_file:0kB inactive_file:184kB unevictable:2600kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2600kB dirty:0kB writeback:0kB mapped:1704kB shmem:0kB slab_reclaimable:6356kB slab_unreclaimable:756520kB kernel_stack:1888kB pagetables:2456kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:368 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 60*4kB 25*8kB 20*16kB 23*32kB 21*64kB 16*128kB 10*256kB 7*512kB 8*1024kB 11*2048kB 9*4096kB = 78616kB
Node 0 Normal: 1275*4kB 59*8kB 1*16kB 4*32kB 4*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 9428kB
630 total pagecache pages
0 pages in swap cache
Swap cache stats: add 30796, delete 30796, find 8860/12043
Free swap = 15602032kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
3659 pages shared
4932438 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 77 3 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2308] 0 2308 21142 94 2 0 0 sendmail
[ 2316] 51 2316 20073 69 3 0 0 sendmail
[ 2325] 0 2325 29321 131 1 0 0 crond
[ 2505] 0 2505 25510 51 1 0 0 xl
[ 2553] 0 2553 57939 164 2 0 0 qemu-dm
[ 2792] 0 2792 25469 49 2 0 0 xl
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 148 3 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 200 1 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 161 0 0 0 python
Out of memory: Kill process 2308 (sendmail) score 1 or sacrifice child
Killed process 2308 (sendmail) total-vm:84568kB, anon-rss:0kB, file-rss:376kB
python invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
python cpuset=/ mems_allowed=0
Pid: 5698, comm: python Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff81159156>] alloc_pages_current+0xb6/0x120
[<ffffffff811143a7>] __page_cache_alloc+0xb7/0xd0
[<ffffffff811153ef>] filemap_fault+0x1af/0x4b0
[<ffffffff8113d9a2>] __do_fault+0x72/0x5a0
[<ffffffff8107d2e9>] ? __wake_up_common+0x59/0x90
[<ffffffff8113dfb7>] handle_pte_fault+0xe7/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff8118a5d1>] ? poll_select_copy_remaining+0x101/0x150
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:105 inactive_anon:18 isolated_anon:0
active_file:2 inactive_file:35 isolated_file:0
unevictable:650 dirty:0 writeback:0 unstable:0
free:26096 slab_reclaimable:1597 slab_unreclaimable:318911
mapped:540 shmem:1 pagetables:572 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:78292kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:40kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:52kB slab_unreclaimable:503444kB kernel_stack:40kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1455 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:10200kB min:16488kB low:20608kB high:24732kB active_anon:420kB inactive_anon:72kB active_file:0kB inactive_file:124kB unevictable:2600kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2600kB dirty:0kB writeback:0kB mapped:2156kB shmem:4kB slab_reclaimable:6336kB slab_unreclaimable:772200kB kernel_stack:1880kB pagetables:2288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1005 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 19*4kB 15*8kB 18*16kB 35*32kB 21*64kB 13*128kB 10*256kB 7*512kB 8*1024kB 11*2048kB 9*4096kB = 78340kB
Node 0 Normal: 1038*4kB 138*8kB 12*16kB 29*32kB 3*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 10408kB
612 total pagecache pages
0 pages in swap cache
Swap cache stats: add 35241, delete 35241, find 10485/14258
Free swap = 15603796kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
3561 pages shared
4926264 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 85 2 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2316] 51 2316 20073 69 3 0 0 sendmail
[ 2325] 0 2325 29321 131 1 0 0 crond
[ 2505] 0 2505 25510 51 1 0 0 xl
[ 2553] 0 2553 57939 158 3 0 0 qemu-dm
[ 2792] 0 2792 25469 49 2 0 0 xl
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 168 2 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 2 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 170 1 0 0 python
Out of memory: Kill process 2316 (sendmail) score 1 or sacrifice child
Killed process 2316 (sendmail) total-vm:80292kB, anon-rss:0kB, file-rss:276kB
python invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
python cpuset=/ mems_allowed=0
Pid: 5698, comm: python Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff81159156>] alloc_pages_current+0xb6/0x120
[<ffffffff811143a7>] __page_cache_alloc+0xb7/0xd0
[<ffffffff811153ef>] filemap_fault+0x1af/0x4b0
[<ffffffff8113d9a2>] __do_fault+0x72/0x5a0
[<ffffffff8107d2e9>] ? __wake_up_common+0x59/0x90
[<ffffffff8113dfb7>] handle_pte_fault+0xe7/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff8118a5d1>] ? poll_select_copy_remaining+0x101/0x150
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 29
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:105 inactive_anon:18 isolated_anon:0
active_file:0 inactive_file:0 isolated_file:0
unevictable:650 dirty:0 writeback:0 unstable:0
free:25724 slab_reclaimable:1597 slab_unreclaimable:318911
mapped:540 shmem:1 pagetables:572 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:78292kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:40kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:52kB slab_unreclaimable:503444kB kernel_stack:40kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1456 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:8712kB min:16488kB low:20608kB high:24732kB active_anon:420kB inactive_anon:72kB active_file:0kB inactive_file:0kB unevictable:2600kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2600kB dirty:0kB writeback:0kB mapped:2156kB shmem:4kB slab_reclaimable:6336kB slab_unreclaimable:772200kB kernel_stack:1880kB pagetables:2288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:759 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 19*4kB 15*8kB 18*16kB 35*32kB 21*64kB 13*128kB 10*256kB 7*512kB 8*1024kB 11*2048kB 9*4096kB = 78340kB
Node 0 Normal: 1117*4kB 45*8kB 0*16kB 3*32kB 0*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 8764kB
612 total pagecache pages
0 pages in swap cache
Swap cache stats: add 35246, delete 35246, find 10485/14260
Free swap = 15605200kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
3470 pages shared
4926396 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 85 2 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2325] 0 2325 29321 131 1 0 0 crond
[ 2505] 0 2505 25510 51 1 0 0 xl
[ 2553] 0 2553 57939 158 0 0 0 qemu-dm
[ 2792] 0 2792 25469 49 2 0 0 xl
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 168 2 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 2 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 170 1 0 0 python
Out of memory: Kill process 2325 (crond) score 1 or sacrifice child
Killed process 2325 (crond) total-vm:117284kB, anon-rss:0kB, file-rss:524kB
qemu-dm invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
qemu-dm cpuset=/ mems_allowed=0
Pid: 2553, comm: qemu-dm Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff8115a126>] alloc_pages_vma+0xc6/0x1a0
[<ffffffff8114d7d2>] read_swap_cache_async+0xf2/0x170
[<ffffffff8114d8de>] swapin_readahead+0x8e/0xd0
[<ffffffff8113d415>] do_swap_page+0x115/0x630
[<ffffffff81189a40>] ? __pollwait+0xf0/0xf0
[<ffffffff8113e07a>] handle_pte_fault+0x1aa/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff81058237>] ? current_fs_time+0x27/0x30
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff81180cb9>] ? pipe_read+0x289/0x500
[<ffffffff8118ad63>] ? core_sys_select+0x1c3/0x2f0
[<ffffffff8157aa65>] page_fault+0x25/0x30
[<ffffffff81274ffd>] ? copy_user_generic_string+0x2d/0x40
[<ffffffff8118a5d1>] ? poll_select_copy_remaining+0x101/0x150
[<ffffffff8118b0f9>] sys_select+0x59/0x110
[<ffffffff81582929>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:105 inactive_anon:18 isolated_anon:0
active_file:0 inactive_file:27 isolated_file:0
unevictable:650 dirty:0 writeback:0 unstable:0
free:26750 slab_reclaimable:1597 slab_unreclaimable:318966
mapped:540 shmem:1 pagetables:572 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:78204kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:40kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:52kB slab_unreclaimable:503444kB kernel_stack:40kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1608 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:12904kB min:16488kB low:20608kB high:24732kB active_anon:420kB inactive_anon:72kB active_file:0kB inactive_file:92kB unevictable:2600kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2600kB dirty:0kB writeback:0kB mapped:2156kB shmem:4kB slab_reclaimable:6336kB slab_unreclaimable:772420kB kernel_stack:1880kB pagetables:2288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:989 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 19*4kB 21*8kB 18*16kB 37*32kB 21*64kB 13*128kB 10*256kB 7*512kB 8*1024kB 11*2048kB 9*4096kB = 78452kB
Node 0 Normal: 1182*4kB 204*8kB 21*16kB 32*32kB 9*64kB 6*128kB 2*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 13160kB
612 total pagecache pages
0 pages in swap cache
Swap cache stats: add 35252, delete 35252, find 10486/14262
Free swap = 15605852kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
3340 pages shared
4925577 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 85 2 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2505] 0 2505 25510 51 1 0 0 xl
[ 2553] 0 2553 57939 158 0 0 0 qemu-dm
[ 2792] 0 2792 25469 49 2 0 0 xl
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 168 3 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 2 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 170 1 0 0 python
Out of memory: Kill process 2505 (xl) score 1 or sacrifice child
Killed process 2505 (xl) total-vm:102040kB, anon-rss:0kB, file-rss:204kB
init invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
init cpuset=/ mems_allowed=0
Pid: 1, comm: init Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8100a602>] ? check_events+0x12/0x20
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff8100a5a9>] ? xen_irq_enable_direct_reloc+0x4/0x4
[<ffffffff810838c3>] ? finish_task_switch+0x53/0xe0
[<ffffffff8115a126>] alloc_pages_vma+0xc6/0x1a0
[<ffffffff8114d7d2>] read_swap_cache_async+0xf2/0x170
[<ffffffff8157a307>] ? _raw_spin_unlock_irqrestore+0x17/0x20
[<ffffffff8114d8de>] swapin_readahead+0x8e/0xd0
[<ffffffff8113d415>] do_swap_page+0x115/0x630
[<ffffffff8113e07a>] handle_pte_fault+0x1aa/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff81189a40>] ? __pollwait+0xf0/0xf0
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff81189a40>] ? __pollwait+0xf0/0xf0
[<ffffffff8116d677>] ? mem_cgroup_uncharge_swap+0x27/0x90
[<ffffffff8100556d>] ? pte_mfn_to_pfn+0x7d/0x100
[<ffffffff8100543d>] ? pte_pfn_to_mfn+0x8d/0xa0
[<ffffffff81009c3f>] ? xen_set_pte_at+0x8f/0x230
[<ffffffff8111405a>] ? unlock_page+0x2a/0x40
[<ffffffff8157aa65>] page_fault+0x25/0x30
[<ffffffff81274ffd>] ? copy_user_generic_string+0x2d/0x40
[<ffffffff8101efe8>] ? save_i387_xstate+0x1f8/0x260
[<ffffffff810145f0>] __setup_rt_frame+0x320/0x360
[<ffffffff81014799>] handle_signal+0x169/0x210
[<ffffffff81014911>] do_signal+0xd1/0x140
[<ffffffff81040f38>] ? pvclock_clocksource_read+0x58/0xd0
[<ffffffff810149e5>] do_notify_resume+0x65/0x80
[<ffffffff81582be2>] int_signal+0x12/0x17
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:9 inactive_file:14 isolated_file:0
unevictable:650 dirty:0 writeback:0 unstable:0
free:26953 slab_reclaimable:1597 slab_unreclaimable:319159
mapped:509 shmem:0 pagetables:518 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:78452kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:52kB slab_unreclaimable:503604kB kernel_stack:40kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:140 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:13468kB min:16488kB low:20608kB high:24732kB active_anon:0kB inactive_anon:0kB active_file:36kB inactive_file:0kB unevictable:2600kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2600kB dirty:0kB writeback:0kB mapped:2032kB shmem:0kB slab_reclaimable:6336kB slab_unreclaimable:773032kB kernel_stack:1880kB pagetables:2072kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:184 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 20*4kB 21*8kB 18*16kB 39*32kB 21*64kB 13*128kB 10*256kB 7*512kB 8*1024kB 11*2048kB 9*4096kB = 78520kB
Node 0 Normal: 1199*4kB 194*8kB 21*16kB 45*32kB 11*64kB 6*128kB 2*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 13692kB
544 total pagecache pages
0 pages in swap cache
Swap cache stats: add 35264, delete 35264, find 10486/14264
Free swap = 15606384kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
3268 pages shared
4925869 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 85 2 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2553] 0 2553 57939 158 2 0 0 qemu-dm
[ 2792] 0 2792 25469 49 2 0 0 xl
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 168 3 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 2 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 170 1 0 0 python
Out of memory: Kill process 2553 (qemu-dm) score 1 or sacrifice child
Killed process 2553 (qemu-dm) total-vm:231756kB, anon-rss:0kB, file-rss:632kB
qemu-dm invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0
qemu-dm cpuset=/ mems_allowed=0
Pid: 2954, comm: qemu-dm Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff81159156>] alloc_pages_current+0xb6/0x120
[<ffffffff8111aaee>] __get_free_pages+0xe/0x50
[<ffffffffa0587796>] evtchn_write+0x36/0x140 [xen_evtchn]
[<ffffffff81176f38>] vfs_write+0xc8/0x190
[<ffffffff81177101>] sys_write+0x51/0x90
[<ffffffff81582929>] system_call_fastpath+0x16/0x1b
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:12 inactive_file:0 isolated_file:0
unevictable:650 dirty:0 writeback:0 unstable:0
free:27027 slab_reclaimable:1597 slab_unreclaimable:319159
mapped:509 shmem:0 pagetables:436 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:78452kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:148kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:52kB slab_unreclaimable:503604kB kernel_stack:40kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:234 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:13764kB min:16488kB low:20608kB high:24732kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:2600kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2600kB dirty:0kB writeback:0kB mapped:2032kB shmem:0kB slab_reclaimable:6336kB slab_unreclaimable:773032kB kernel_stack:1880kB pagetables:1744kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:161 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 23*4kB 24*8kB 18*16kB 44*32kB 21*64kB 13*128kB 10*256kB 7*512kB 8*1024kB 11*2048kB 9*4096kB = 78716kB
Node 0 Normal: 1145*4kB 133*8kB 21*16kB 76*32kB 12*64kB 6*128kB 2*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 14044kB
544 total pagecache pages
0 pages in swap cache
Swap cache stats: add 35283, delete 35283, find 10489/14268
Free swap = 15608792kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
3102 pages shared
4925675 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 85 2 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2792] 0 2792 25469 49 2 0 0 xl
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 168 0 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 2 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 170 1 0 0 python
Out of memory: Kill process 2792 (xl) score 1 or sacrifice child
Killed process 2792 (xl) total-vm:101876kB, anon-rss:0kB, file-rss:196kB
netback/0 invoked oom-killer: gfp_mask=0x1d0, order=0, oom_adj=0, oom_score_adj=0
netback/0 cpuset=/ mems_allowed=0
Pid: 1578, comm: netback/0 Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff81159156>] alloc_pages_current+0xb6/0x120
[<ffffffffa05a07dc>] xen_netbk_tx_build_gops+0x49c/0x920 [xen_netback]
[<ffffffff81004c60>] ? xen_mc_flush+0xb0/0x1b0
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffff8108c94d>] ? idle_balance+0xed/0x140
[<ffffffff810838c3>] ? finish_task_switch+0x53/0xe0
[<ffffffff8157a307>] ? _raw_spin_unlock_irqrestore+0x17/0x20
[<ffffffffa05a0d0c>] xen_netbk_kthread+0xac/0x1d0 [xen_netback]
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffffa05a0c60>] ? xen_netbk_tx_build_gops+0x920/0x920 [xen_netback]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 1
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 74
CPU 3: hi: 186, btch: 31 usd: 116
active_anon:27 inactive_anon:0 isolated_anon:127
active_file:12 inactive_file:7 isolated_file:0
unevictable:645 dirty:0 writeback:0 unstable:0
free:25592 slab_reclaimable:1588 slab_unreclaimable:315716
mapped:548 shmem:1 pagetables:419 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:77840kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:60kB active_file:28kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:48kB slab_unreclaimable:502184kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:11616 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:8636kB min:16488kB low:20608kB high:24732kB active_anon:108kB inactive_anon:0kB active_file:20kB inactive_file:0kB unevictable:2580kB isolated(anon):508kB isolated(file):0kB present:19659544kB mlocked:2580kB dirty:0kB writeback:0kB mapped:2188kB shmem:4kB slab_reclaimable:6304kB slab_unreclaimable:760680kB kernel_stack:1840kB pagetables:1676kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:201 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 18*4kB 14*8kB 55*16kB 19*32kB 20*64kB 11*128kB 10*256kB 5*512kB 7*1024kB 12*2048kB 9*4096kB = 78088kB
Node 0 Normal: 1133*4kB 85*8kB 0*16kB 0*32kB 0*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 8924kB
679 total pagecache pages
13 pages in swap cache
Swap cache stats: add 39241, delete 39228, find 11890/16205
Free swap = 15609164kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
3042 pages shared
4917796 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 73 0 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2801] 0 2801 1027 121 3 0 0 mingetty
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 167 1 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 0 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 154 3 0 0 python
Out of memory: Kill process 2801 (mingetty) score 1 or sacrifice child
Killed process 2801 (mingetty) total-vm:4108kB, anon-rss:0kB, file-rss:484kB
netback/0 invoked oom-killer: gfp_mask=0x1d0, order=0, oom_adj=0, oom_score_adj=0
netback/0 cpuset=/ mems_allowed=0
Pid: 1578, comm: netback/0 Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff81159156>] alloc_pages_current+0xb6/0x120
[<ffffffffa05a07dc>] xen_netbk_tx_build_gops+0x49c/0x920 [xen_netback]
[<ffffffff81004c60>] ? xen_mc_flush+0xb0/0x1b0
[<ffffffff8100386e>] ? xen_end_context_switch+0x1e/0x30
[<ffffffff8101377c>] ? __switch_to+0x15c/0x440
[<ffffffff8108c94d>] ? idle_balance+0xed/0x140
[<ffffffff810838c3>] ? finish_task_switch+0x53/0xe0
[<ffffffff8157a307>] ? _raw_spin_unlock_irqrestore+0x17/0x20
[<ffffffffa05a0d0c>] xen_netbk_kthread+0xac/0x1d0 [xen_netback]
[<ffffffff81074ae0>] ? wake_up_bit+0x40/0x40
[<ffffffffa05a0c60>] ? xen_netbk_tx_build_gops+0x920/0x920 [xen_netback]
[<ffffffff8107442e>] kthread+0x9e/0xb0
[<ffffffff81583c24>] kernel_thread_helper+0x4/0x10
[<ffffffff8157a7f8>] ? retint_restore_args+0x5/0x6
[<ffffffff81583c20>] ? gs_change+0x13/0x13
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 1
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 83
CPU 3: hi: 186, btch: 31 usd: 163
active_anon:27 inactive_anon:0 isolated_anon:127
active_file:12 inactive_file:7 isolated_file:0
unevictable:645 dirty:0 writeback:0 unstable:0
free:25592 slab_reclaimable:1588 slab_unreclaimable:315716
mapped:548 shmem:1 pagetables:419 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:77840kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:60kB active_file:28kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:48kB slab_unreclaimable:502184kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:11644 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:8636kB min:16488kB low:20608kB high:24732kB active_anon:108kB inactive_anon:0kB active_file:20kB inactive_file:0kB unevictable:2580kB isolated(anon):508kB isolated(file):0kB present:19659544kB mlocked:2580kB dirty:0kB writeback:0kB mapped:2188kB shmem:4kB slab_reclaimable:6304kB slab_unreclaimable:760680kB kernel_stack:1840kB pagetables:1676kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:214 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 18*4kB 14*8kB 55*16kB 19*32kB 20*64kB 11*128kB 10*256kB 5*512kB 7*1024kB 12*2048kB 9*4096kB = 78088kB
Node 0 Normal: 1134*4kB 69*8kB 0*16kB 0*32kB 0*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 8800kB
679 total pagecache pages
4 pages in swap cache
Swap cache stats: add 39241, delete 39237, find 11890/16206
Free swap = 15609240kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
2912 pages shared
4917780 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 73 0 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2803] 0 2803 1027 121 1 0 0 mingetty
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 167 1 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 0 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 154 3 0 0 python
Out of memory: Kill process 2803 (mingetty) score 1 or sacrifice child
Killed process 2803 (mingetty) total-vm:4108kB, anon-rss:0kB, file-rss:484kB
qemu-dm invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
qemu-dm cpuset=/ mems_allowed=0
Pid: 2954, comm: qemu-dm Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff8115a126>] alloc_pages_vma+0xc6/0x1a0
[<ffffffff81113f90>] ? sleep_on_page+0x20/0x20
[<ffffffff8114d7d2>] read_swap_cache_async+0xf2/0x170
[<ffffffff8114d8de>] swapin_readahead+0x8e/0xd0
[<ffffffff8113d415>] do_swap_page+0x115/0x630
[<ffffffff8113ce5e>] ? do_wp_page+0x38e/0x830
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e07a>] handle_pte_fault+0x1aa/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff81004c60>] ? xen_mc_flush+0xb0/0x1b0
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff810cdf06>] ? __audit_syscall_exit+0x416/0x470
[<ffffffff81014b3d>] ? restore_sigcontext+0x11d/0x160
[<ffffffff81020c15>] ? syscall_trace_leave+0xa5/0x100
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:25 inactive_file:0 isolated_file:0
unevictable:645 dirty:0 writeback:0 unstable:0
free:26742 slab_reclaimable:1588 slab_unreclaimable:315909
mapped:509 shmem:0 pagetables:403 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:77832kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:48kB slab_unreclaimable:502196kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1356 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:13244kB min:16488kB low:20608kB high:24732kB active_anon:0kB inactive_anon:0kB active_file:120kB inactive_file:0kB unevictable:2580kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2580kB dirty:0kB writeback:0kB mapped:2032kB shmem:0kB slab_reclaimable:6304kB slab_unreclaimable:761440kB kernel_stack:1840kB pagetables:1612kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:207 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 20*4kB 34*8kB 58*16kB 19*32kB 19*64kB 11*128kB 10*256kB 5*512kB 7*1024kB 12*2048kB 9*4096kB = 78240kB
Node 0 Normal: 1084*4kB 62*8kB 46*16kB 28*32kB 19*64kB 13*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 12928kB
544 total pagecache pages
0 pages in swap cache
Swap cache stats: add 39398, delete 39398, find 11929/16274
Free swap = 15609324kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
2787 pages shared
4919223 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 73 0 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2805] 0 2805 1027 121 3 0 0 mingetty
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 155 2 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 0 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 150 1 0 0 python
Out of memory: Kill process 2805 (mingetty) score 1 or sacrifice child
Killed process 2805 (mingetty) total-vm:4108kB, anon-rss:0kB, file-rss:484kB
init invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
init cpuset=/ mems_allowed=0
Pid: 1, comm: init Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff8115a126>] alloc_pages_vma+0xc6/0x1a0
[<ffffffff81113f90>] ? sleep_on_page+0x20/0x20
[<ffffffff8114d7d2>] read_swap_cache_async+0xf2/0x170
[<ffffffff8114d8de>] swapin_readahead+0x8e/0xd0
[<ffffffff8113d415>] do_swap_page+0x115/0x630
[<ffffffff8113ce5e>] ? do_wp_page+0x38e/0x830
[<ffffffff8100556d>] ? pte_mfn_to_pfn+0x7d/0x100
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e07a>] handle_pte_fault+0x1aa/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff81009e6d>] ? xen_force_evtchn_callback+0xd/0x10
[<ffffffff8100a602>] ? check_events+0x12/0x20
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff81056898>] ? sys_waitid+0xa8/0x1f0
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 30
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:31 inactive_file:10 isolated_file:0
unevictable:645 dirty:0 writeback:0 unstable:0
free:26987 slab_reclaimable:1588 slab_unreclaimable:315909
mapped:509 shmem:0 pagetables:403 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:77832kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:48kB slab_unreclaimable:502196kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4128 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:14224kB min:16488kB low:20608kB high:24732kB active_anon:0kB inactive_anon:0kB active_file:144kB inactive_file:0kB unevictable:2580kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2580kB dirty:0kB writeback:0kB mapped:2032kB shmem:0kB slab_reclaimable:6304kB slab_unreclaimable:761440kB kernel_stack:1840kB pagetables:1612kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:230 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 20*4kB 34*8kB 58*16kB 19*32kB 19*64kB 11*128kB 10*256kB 5*512kB 7*1024kB 12*2048kB 9*4096kB = 78240kB
Node 0 Normal: 1085*4kB 54*8kB 56*16kB 38*32kB 29*64kB 15*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 14244kB
544 total pagecache pages
0 pages in swap cache
Swap cache stats: add 39398, delete 39398, find 11929/16274
Free swap = 15609408kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
2666 pages shared
4920463 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 73 0 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2807] 0 2807 1027 121 1 0 0 mingetty
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 155 2 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 0 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 150 1 0 0 python
Out of memory: Kill process 2807 (mingetty) score 1 or sacrifice child
Killed process 2807 (mingetty) total-vm:4108kB, anon-rss:0kB, file-rss:484kB
init invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
init cpuset=/ mems_allowed=0
Pid: 1, comm: init Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff8115a126>] alloc_pages_vma+0xc6/0x1a0
[<ffffffff81113f90>] ? sleep_on_page+0x20/0x20
[<ffffffff8114d7d2>] read_swap_cache_async+0xf2/0x170
[<ffffffff8114d8de>] swapin_readahead+0x8e/0xd0
[<ffffffff8113d415>] do_swap_page+0x115/0x630
[<ffffffff8113ce5e>] ? do_wp_page+0x38e/0x830
[<ffffffff8100556d>] ? pte_mfn_to_pfn+0x7d/0x100
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e07a>] handle_pte_fault+0x1aa/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff81009e6d>] ? xen_force_evtchn_callback+0xd/0x10
[<ffffffff8100a602>] ? check_events+0x12/0x20
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff81056898>] ? sys_waitid+0xa8/0x1f0
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 60
CPU 1: hi: 186, btch: 31 usd: 44
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:0 isolated_file:0
unevictable:645 dirty:0 writeback:0 unstable:0
free:25473 slab_reclaimable:1588 slab_unreclaimable:315909
mapped:509 shmem:0 pagetables:403 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:77832kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:48kB slab_unreclaimable:502196kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:5416 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:8168kB min:16488kB low:20608kB high:24732kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:2580kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2580kB dirty:0kB writeback:0kB mapped:2032kB shmem:0kB slab_reclaimable:6304kB slab_unreclaimable:761440kB kernel_stack:1840kB pagetables:1612kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 20*4kB 34*8kB 58*16kB 19*32kB 19*64kB 11*128kB 10*256kB 5*512kB 7*1024kB 12*2048kB 9*4096kB = 78240kB
Node 0 Normal: 1085*4kB 13*8kB 1*16kB 1*32kB 0*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 8204kB
544 total pagecache pages
0 pages in swap cache
Swap cache stats: add 39398, delete 39398, find 11929/16274
Free swap = 15609488kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
2545 pages shared
4920671 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 73 0 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2809] 0 2809 1027 121 0 0 0 mingetty
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 155 2 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 0 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 150 1 0 0 python
Out of memory: Kill process 2809 (mingetty) score 1 or sacrifice child
Killed process 2809 (mingetty) total-vm:4108kB, anon-rss:0kB, file-rss:484kB
python invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
python cpuset=/ mems_allowed=0
Pid: 5698, comm: python Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff8115a126>] alloc_pages_vma+0xc6/0x1a0
[<ffffffff81113f90>] ? sleep_on_page+0x20/0x20
[<ffffffff8114d7d2>] read_swap_cache_async+0xf2/0x170
[<ffffffff8114d8de>] swapin_readahead+0x8e/0xd0
[<ffffffff8113d415>] do_swap_page+0x115/0x630
[<ffffffff8111405a>] ? unlock_page+0x2a/0x40
[<ffffffff8113dda9>] ? __do_fault+0x479/0x5a0
[<ffffffff8113e07a>] handle_pte_fault+0x1aa/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff81274ffd>] ? copy_user_generic_string+0x2d/0x40
[<ffffffff8118a5d1>] ? poll_select_copy_remaining+0x101/0x150
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 76
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:0 isolated_file:0
unevictable:645 dirty:0 writeback:0 unstable:0
free:25461 slab_reclaimable:1588 slab_unreclaimable:315909
mapped:509 shmem:0 pagetables:403 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:77832kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:48kB slab_unreclaimable:502196kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:6228 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:8120kB min:16488kB low:20608kB high:24732kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:2580kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2580kB dirty:0kB writeback:0kB mapped:2032kB shmem:0kB slab_reclaimable:6304kB slab_unreclaimable:761440kB kernel_stack:1840kB pagetables:1612kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 20*4kB 34*8kB 58*16kB 19*32kB 19*64kB 11*128kB 10*256kB 5*512kB 7*1024kB 12*2048kB 9*4096kB = 78240kB
Node 0 Normal: 1085*4kB 12*8kB 0*16kB 0*32kB 1*64kB 2*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 8340kB
544 total pagecache pages
0 pages in swap cache
Swap cache stats: add 39398, delete 39398, find 11929/16274
Free swap = 15609572kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
2424 pages shared
4920735 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 73 0 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2811] 0 2811 1027 121 1 0 0 mingetty
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 155 2 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 0 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 150 1 0 0 python
Out of memory: Kill process 2811 (mingetty) score 1 or sacrifice child
Killed process 2811 (mingetty) total-vm:4108kB, anon-rss:0kB, file-rss:484kB
qemu-dm invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
qemu-dm cpuset=/ mems_allowed=0
Pid: 2954, comm: qemu-dm Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff8115a126>] alloc_pages_vma+0xc6/0x1a0
[<ffffffff81113f90>] ? sleep_on_page+0x20/0x20
[<ffffffff8114d7d2>] read_swap_cache_async+0xf2/0x170
[<ffffffff8114d8de>] swapin_readahead+0x8e/0xd0
[<ffffffff8113d415>] do_swap_page+0x115/0x630
[<ffffffff8113ce5e>] ? do_wp_page+0x38e/0x830
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e07a>] handle_pte_fault+0x1aa/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff81004c60>] ? xen_mc_flush+0xb0/0x1b0
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff810cdf06>] ? __audit_syscall_exit+0x416/0x470
[<ffffffff81014b3d>] ? restore_sigcontext+0x11d/0x160
[<ffffffff81020c15>] ? syscall_trace_leave+0xa5/0x100
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 32
CPU 1: hi: 186, btch: 31 usd: 2
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:10 isolated_file:0
unevictable:645 dirty:0 writeback:0 unstable:0
free:25437 slab_reclaimable:1588 slab_unreclaimable:315909
mapped:509 shmem:0 pagetables:403 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:77832kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:48kB slab_unreclaimable:502196kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:6340 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:8024kB min:16488kB low:20608kB high:24732kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:2580kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2580kB dirty:0kB writeback:0kB mapped:2032kB shmem:0kB slab_reclaimable:6304kB slab_unreclaimable:761440kB kernel_stack:1840kB pagetables:1612kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:23 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 20*4kB 34*8kB 58*16kB 19*32kB 19*64kB 11*128kB 10*256kB 5*512kB 7*1024kB 12*2048kB 9*4096kB = 78240kB
Node 0 Normal: 1094*4kB 14*8kB 0*16kB 0*32kB 0*64kB 2*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 8328kB
544 total pagecache pages
0 pages in swap cache
Swap cache stats: add 39398, delete 39398, find 11929/16274
Free swap = 15609656kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
2303 pages shared
4920744 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 73 0 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2815] 0 2815 27143 179 1 0 0 bash
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 155 3 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 0 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 150 1 0 0 python
Out of memory: Kill process 2813 (sshd) score 1 or sacrifice child
Killed process 2815 (bash) total-vm:108572kB, anon-rss:0kB, file-rss:716kB
init invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
init cpuset=/ mems_allowed=0
Pid: 1, comm: init Tainted: P O 3.4.61-9.el6.centos.alt.x86_64 #1
Call Trace:
[<ffffffff8111785b>] dump_header+0x8b/0x1d0
[<ffffffff8111732f>] ? find_lock_task_mm+0x2f/0x80
[<ffffffff81206e75>] ? security_capable_noaudit+0x15/0x20
[<ffffffff8126f270>] ? ___ratelimit+0xa0/0x120
[<ffffffff81117f69>] oom_kill_process+0x289/0x2c0
[<ffffffff811180a0>] out_of_memory+0x100/0x200
[<ffffffff8111d8b1>] __alloc_pages_slowpath+0x6a1/0x710
[<ffffffff8111db4a>] __alloc_pages_nodemask+0x22a/0x240
[<ffffffff8115a126>] alloc_pages_vma+0xc6/0x1a0
[<ffffffff81113f90>] ? sleep_on_page+0x20/0x20
[<ffffffff8114d7d2>] read_swap_cache_async+0xf2/0x170
[<ffffffff8114d8de>] swapin_readahead+0x8e/0xd0
[<ffffffff8113d415>] do_swap_page+0x115/0x630
[<ffffffff8113ce5e>] ? do_wp_page+0x38e/0x830
[<ffffffff8100556d>] ? pte_mfn_to_pfn+0x7d/0x100
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e07a>] handle_pte_fault+0x1aa/0x210
[<ffffffff8100564e>] ? xen_pmd_val+0xe/0x10
[<ffffffff810051a9>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[<ffffffff8113e351>] handle_mm_fault+0x271/0x380
[<ffffffff81009e6d>] ? xen_force_evtchn_callback+0xd/0x10
[<ffffffff8100a602>] ? check_events+0x12/0x20
[<ffffffff8157e01e>] do_page_fault+0x13e/0x490
[<ffffffff81056898>] ? sys_waitid+0xa8/0x1f0
[<ffffffff8157aa65>] page_fault+0x25/0x30
Mem-Info:
Node 0 DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
CPU 2: hi: 0, btch: 1 usd: 0
CPU 3: hi: 0, btch: 1 usd: 0
Node 0 DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
Node 0 Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 0
CPU 1: hi: 186, btch: 31 usd: 0
CPU 2: hi: 186, btch: 31 usd: 0
CPU 3: hi: 186, btch: 31 usd: 0
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:0 inactive_file:0 isolated_file:0
unevictable:645 dirty:0 writeback:0 unstable:0
free:26434 slab_reclaimable:1588 slab_unreclaimable:315909
mapped:509 shmem:0 pagetables:403 bounce:0
Node 0 DMA free:15892kB min:12kB low:12kB high:16kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15668kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2991 22190 22190
Node 0 DMA32 free:77832kB min:2568kB low:3208kB high:3852kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3063520kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:48kB slab_unreclaimable:502196kB kernel_stack:32kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:7264 all_unreclaimable? yes
lowmem_reserve[]: 0 0 19198 19198
Node 0 Normal free:12012kB min:16488kB low:20608kB high:24732kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:2580kB isolated(anon):0kB isolated(file):0kB present:19659544kB mlocked:2580kB dirty:0kB writeback:0kB mapped:2032kB shmem:0kB slab_reclaimable:6304kB slab_unreclaimable:761440kB kernel_stack:1840kB pagetables:1612kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 1*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15892kB
Node 0 DMA32: 20*4kB 34*8kB 58*16kB 19*32kB 19*64kB 11*128kB 10*256kB 5*512kB 7*1024kB 12*2048kB 9*4096kB = 78240kB
Node 0 Normal: 1100*4kB 319*8kB 9*16kB 9*32kB 6*64kB 6*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 12120kB
544 total pagecache pages
0 pages in swap cache
Swap cache stats: add 39398, delete 39398, find 11929/16275
Free swap = 15610296kB
Total swap = 15623164kB
5769430 pages RAM
125863 pages reserved
2125 pages shared
4919668 pages non-shared
[ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
[ 673] 0 673 2800 61 1 -17 -1000 udevd
[ 2183] 0 2183 6409 73 0 -17 -1000 auditd
[ 2249] 0 2249 16575 71 1 -17 -1000 sshd
[ 2813] 0 2813 24530 201 0 0 0 sshd
[ 2907] 0 2907 3036 642 1 0 0 tapdisk
[ 2954] 0 2954 62805 155 3 0 0 qemu-dm
[ 3075] 0 3075 25487 42 2 0 0 xl
[ 3080] 0 3080 2799 62 0 -17 -1000 udevd
[ 3416] 0 3416 2799 34 2 -17 -1000 udevd
[ 3456] 0 3456 25511 51 2 0 0 xl
[ 3497] 0 3497 24530 188 0 0 0 sshd
[ 3500] 0 3500 27143 179 1 0 0 bash
[ 5698] 0 5698 36020 150 1 0 0 python
Out of memory: Kill process 2813 (sshd) score 1 or sacrifice child
Killed process 2813 (sshd) total-vm:98120kB, anon-rss:0kB, file-rss:804kB
--------------------- cache ------------------------------------------------------- ----- slab ------ ---- object ----- --- emergency ---
name flags size alloc slabsize objsize total alloc max total alloc max dlock alloc max
spl_vn_cache 0x00020 12288 5544 4096 88 3 3 3 63 63 63 0 0 0
spl_vn_file_cache 0x00020 12288 6048 4096 96 3 3 3 63 63 63 0 0 0
spl_zlib_workspace_cache 0x00140 0 0 8388608 268104 0 0 0 0 0 0 0 0 0
space_seg_cache 0x00020 8744960 746752 4096 64 2135 2135 7505 81130 11668 285190 0 0 0
zio_cache 0x00020 458752 224640 32768 1080 14 14 5462 406 208 158398 0 0 0
zio_link_cache 0x00020 69632 26592 4096 48 17 17 3531 765 554 158895 0 0 0
zio_vdev_cache 0x00040 29360128 23071488 4194304 131088 7 7 9 210 176 248 0 0 0
zio_buf_512 0x00020 1146880 505856 32768 512 35 35 407 1085 988 12617 0 0 0
zio_data_buf_512 0x00020 65536 31744 32768 512 2 2 2 62 62 62 0 0 0
zio_buf_1024 0x00040 524288 278528 32768 1024 16 16 24 336 272 504 0 0 0
zio_data_buf_1024 0x00040 32768 21504 32768 1024 1 1 1 21 21 21 0 0 0
zio_buf_1536 0x00040 655360 374784 65536 1536 10 10 15 310 244 465 0 0 0
zio_data_buf_1536 0x00040 327680 238080 65536 1536 5 5 5 155 155 155 0 0 0
zio_buf_2048 0x00040 851968 516096 65536 2048 13 13 111 325 252 2775 0 0 0
zio_data_buf_2048 0x00040 196608 153600 65536 2048 3 3 3 75 75 75 0 0 0
zio_buf_2560 0x00040 1114112 693760 65536 2560 17 17 20 357 271 420 0 0 0
zio_data_buf_2560 0x00040 196608 161280 65536 2560 3 3 3 63 63 63 0 0 0
zio_buf_3072 0x00040 1310720 878592 65536 3072 20 20 23 360 286 414 0 0 0
zio_data_buf_3072 0x00040 196608 165888 65536 3072 3 3 3 54 54 54 0 0 0
zio_buf_3584 0x00040 1703936 756224 131072 3584 13 13 16 403 211 496 0 0 0
zio_data_buf_3584 0x00040 393216 333312 131072 3584 3 3 3 93 93 93 0 0 0
zio_buf_4096 0x00040 1310720 634880 262144 4096 5 5 1283 155 155 39773 5 0 2
zio_data_buf_4096 0x00040 5505024 2125824 262144 4096 21 21 25 651 519 775 0 0 0
zio_buf_5120 0x00040 1310720 747520 131072 5120 10 10 21 210 146 441 0 0 0
zio_data_buf_5120 0x00040 262144 215040 131072 5120 2 2 2 42 42 42 0 0 0
zio_buf_6144 0x00040 1703936 958464 131072 6144 13 13 18 234 156 324 0 0 0
zio_data_buf_6144 0x00040 262144 221184 131072 6144 2 2 2 36 36 36 0 0 0
zio_buf_7168 0x00040 1572864 637952 262144 7168 6 6 10 186 89 310 0 0 0
zio_data_buf_7168 0x00040 524288 444416 262144 7168 2 2 2 62 62 62 0 0 0
zio_buf_8192 0x00040 2097152 1261568 262144 8192 8 8 1857 168 154 38997 1 0 0
zio_data_buf_8192 0x00040 14933032960 6482534400 262144 8192 56965 56965 62473 1196265 791325 1311933 0 0 0
zio_buf_10240 0x00040 1048576 860160 262144 10240 4 4 4 84 84 84 0 0 0
zio_data_buf_10240 0x00040 1048576 860160 262144 10240 4 4 4 84 84 84 0 0 0
zio_buf_12288 0x00040 3145728 1499136 524288 12288 6 6 7 186 122 217 0 0 0
zio_data_buf_12288 0x00040 2097152 1130496 524288 12288 4 4 4 124 92 124 0 0 0
zio_buf_14336 0x00040 2097152 1777664 524288 14336 4 4 4 124 124 124 0 0 0
zio_data_buf_14336 0x00040 2097152 1777664 524288 14336 4 4 4 124 124 124 0 0 0
zio_buf_16384 0x00040 271581184 193134592 524288 16384 518 518 845 12950 11788 21125 0 0 0
zio_data_buf_16384 0x00040 16777216 10436608 524288 16384 32 32 51 800 637 1275 0 0 0
zio_buf_20480 0x00040 3145728 2580480 524288 20480 6 6 6 126 126 126 0 0 0
zio_data_buf_20480 0x00040 524288 430080 524288 20480 1 1 1 21 21 21 0 0 0
zio_buf_24576 0x00040 3670016 3096576 524288 24576 7 7 7 126 126 126 0 0 0
zio_data_buf_24576 0x00040 0 0 524288 24576 0 0 0 0 0 0 0 0 0
zio_buf_28672 0x00040 4194304 3555328 1048576 28672 4 4 4 124 124 124 0 0 0
zio_data_buf_28672 0x00040 0 0 1048576 28672 0 0 0 0 0 0 0 0 0
zio_buf_32768 0x00040 4194304 3670016 1048576 32768 4 4 4 112 112 112 0 0 0
zio_data_buf_32768 0x00040 0 0 1048576 32768 0 0 0 0 0 0 0 0 0
zio_buf_36864 0x00040 4194304 3686400 1048576 36864 4 4 4 100 100 100 0 0 0
zio_data_buf_36864 0x00040 0 0 1048576 36864 0 0 0 0 0 0 0 0 0
zio_buf_40960 0x00040 4194304 3768320 1048576 40960 4 4 4 92 92 92 0 0 0
zio_data_buf_40960 0x00040 0 0 1048576 40960 0 0 0 0 0 0 0 0 0
zio_buf_45056 0x00040 4194304 3784704 1048576 45056 4 4 4 84 84 84 0 0 0
zio_data_buf_45056 0x00040 0 0 1048576 45056 0 0 0 0 0 0 0 0 0
zio_buf_49152 0x00040 8388608 5898240 1048576 49152 8 8 10 152 120 190 0 0 0
zio_data_buf_49152 0x00040 7340032 6537216 1048576 49152 7 7 7 133 133 133 0 0 0
zio_buf_53248 0x00040 0 0 1048576 53248 0 0 0 0 0 0 0 0 0
zio_data_buf_53248 0x00040 0 0 1048576 53248 0 0 0 0 0 0 0 0 0
zio_buf_57344 0x00040 0 0 1048576 57344 0 0 0 0 0 0 0 0 0
zio_data_buf_57344 0x00040 0 0 1048576 57344 0 0 0 0 0 0 0 0 0
zio_buf_61440 0x00040 0 0 2097152 61440 0 0 0 0 0 0 0 0 0
zio_data_buf_61440 0x00040 0 0 2097152 61440 0 0 0 0 0 0 0 0 0
zio_buf_65536 0x00040 0 0 2097152 65536 0 0 0 0 0 0 0 0 0
zio_data_buf_65536 0x00040 0 0 2097152 65536 0 0 0 0 0 0 0 0 0
zio_buf_69632 0x00040 0 0 2097152 69632 0 0 0 0 0 0 0 0 0
zio_data_buf_69632 0x00040 0 0 2097152 69632 0 0 0 0 0 0 0 0 0
zio_buf_73728 0x00040 0 0 2097152 73728 0 0 0 0 0 0 0 0 0
zio_data_buf_73728 0x00040 0 0 2097152 73728 0 0 0 0 0 0 0 0 0
zio_buf_77824 0x00040 0 0 2097152 77824 0 0 0 0 0 0 0 0 0
zio_data_buf_77824 0x00040 0 0 2097152 77824 0 0 0 0 0 0 0 0 0
zio_buf_81920 0x00040 0 0 2097152 81920 0 0 0 0 0 0 0 0 0
zio_data_buf_81920 0x00040 0 0 2097152 81920 0 0 0 0 0 0 0 0 0
zio_buf_86016 0x00040 0 0 2097152 86016 0 0 0 0 0 0 0 0 0
zio_data_buf_86016 0x00040 0 0 2097152 86016 0 0 0 0 0 0 0 0 0
zio_buf_90112 0x00040 0 0 2097152 90112 0 0 0 0 0 0 0 0 0
zio_data_buf_90112 0x00040 0 0 2097152 90112 0 0 0 0 0 0 0 0 0
zio_buf_94208 0x00040 0 0 2097152 94208 0 0 0 0 0 0 0 0 0
zio_data_buf_94208 0x00040 0 0 2097152 94208 0 0 0 0 0 0 0 0 0
zio_buf_98304 0x00040 0 0 2097152 98304 0 0 0 0 0 0 0 0 0
zio_data_buf_98304 0x00040 0 0 2097152 98304 0 0 0 0 0 0 0 0 0
zio_buf_102400 0x00040 0 0 2097152 102400 0 0 0 0 0 0 0 0 0
zio_data_buf_102400 0x00040 0 0 2097152 102400 0 0 0 0 0 0 0 0 0
zio_buf_106496 0x00040 0 0 2097152 106496 0 0 0 0 0 0 0 0 0
zio_data_buf_106496 0x00040 0 0 2097152 106496 0 0 0 0 0 0 0 0 0
zio_buf_110592 0x00040 0 0 2097152 110592 0 0 0 0 0 0 0 0 0
zio_data_buf_110592 0x00040 0 0 2097152 110592 0 0 0 0 0 0 0 0 0
zio_buf_114688 0x00040 6291456 5849088 2097152 114688 3 3 3 51 51 51 0 0 0
zio_data_buf_114688 0x00040 0 0 2097152 114688 0 0 0 0 0 0 0 0 0
zio_buf_118784 0x00040 0 0 2097152 118784 0 0 0 0 0 0 0 0 0
zio_data_buf_118784 0x00040 0 0 2097152 118784 0 0 0 0 0 0 0 0 0
zio_buf_122880 0x00040 0 0 2097152 122880 0 0 0 0 0 0 0 0 0
zio_data_buf_122880 0x00040 0 0 2097152 122880 0 0 0 0 0 0 0 0 0
zio_buf_126976 0x00040 0 0 4194304 126976 0 0 0 0 0 0 0 0 0
zio_data_buf_126976 0x00040 0 0 4194304 126976 0 0 0 0 0 0 0 0 0
zio_buf_131072 0x00040 37748736 32505856 4194304 131072 9 9 46 279 248 1411 0 0 0
zio_data_buf_131072 0x00040 20971520 12976128 4194304 131072 5 5 5 155 99 155 0 0 0
lz4_cache 0x00040 2097152 2031616 524288 16384 4 4 4 124 124 124 0 0 0
sa_cache 0x00020 4096 2640 4096 80 1 1 1 33 33 33 0 0 0
spill_cache 0x00040 0 0 4194304 131072 0 0 0 0 0 0 0 0 0
dnode_t 0x00020 1032192 935424 16384 928 63 63 780 1008 1008 12480 0 0 0
dmu_buf_impl_t 0x00020 212541440 174695360 8192 280 25945 25945 50735 648625 623912 1268375 0 0 0
arc_buf_hdr_t 0x00020 5452996608 4569014736 8192 264 665649 665649 665649 17306874 17306874 17306874 0 0 0
arc_buf_t 0x00020 74231808 46603872 4096 72 18123 18123 35879 652428 647276 1291644 0 0 0
zil_lwb_cache 0x00020 16384 12800 4096 200 4 4 4 64 64 64 0 0 0
zfs_znode_cache 0x00020 32768 31000 32768 1000 1 1 1 31 31 31 0 0 0
4 1 0x01 84 4032 12880081404 44608662862609
name type data
hits 4 458323003
misses 4 113083008
demand_data_hits 4 434763918
demand_data_misses 4 52921073
demand_metadata_hits 4 18953339
demand_metadata_misses 4 4575941
prefetch_data_hits 4 3861525
prefetch_data_misses 4 54870668
prefetch_metadata_hits 4 744221
prefetch_metadata_misses 4 715326
mru_hits 4 146455622
mru_ghost_hits 4 11757365
mfu_hits 4 308088259
mfu_ghost_hits 4 12405148
deleted 4 144141562
recycle_miss 4 9012426
mutex_miss 4 7875590
evict_skip 4 1986757794
evict_l2_cached 4 451309950976
evict_l2_eligible 4 666233502208
evict_l2_ineligible 4 273431949312
hash_elements 4 17313412
hash_elements_max 4 17313427
hash_collisions 4 219321309
hash_chains 4 524288
hash_chain_max 4 66
p 4 3422016070
c 4 10737418240
c_min 4 1444933632
c_max 4 10737418240
size 4 10737280536
hdr_size 4 358332984
data_size 4 5418797056
other_size 4 174367208
anon_size 4 29237248
anon_evict_data 4 0
anon_evict_metadata 4 0
mru_size 4 3479287808
mru_evict_data 4 3445882880
mru_evict_metadata 4 0
mru_ghost_size 4 7258003968
mru_ghost_evict_data 4 2143903744
mru_ghost_evict_metadata 4 5114100224
mfu_size 4 1910272000
mfu_evict_data 4 1750065152
mfu_evict_metadata 4 0
mfu_ghost_size 4 825333760
mfu_ghost_evict_data 4 758628352
mfu_ghost_evict_metadata 4 66705408
l2_hits 4 22579313
l2_misses 4 90503672
l2_feeds 4 48243
l2_rw_clash 4 33894
l2_read_bytes 4 17490233856
l2_write_bytes 4 97442256384
l2_writes_sent 4 35396
l2_writes_done 4 35396
l2_writes_error 4 0
l2_writes_hdr_miss 4 57619
l2_evict_lock_retry 4 301
l2_evict_reading 4 0
l2_free_on_write 4 437613
l2_abort_lowmem 4 160
l2_cksum_bad 4 0
l2_io_error 4 0
l2_size 4 137442087936
l2_asize 4 14034853376
l2_hdr_size 4 4785783288
l2_compress_successes 4 39532419
l2_compress_zeros 4 0
l2_compress_failures 4 8782763
memory_throttle_count 4 0
duplicate_buffers 4 0
duplicate_buffers_size 4 0
duplicate_reads 4 800
memory_direct_count 4 6446
memory_indirect_count 4 44381
arc_no_grow 4 0
arc_tempreserve 4 0
arc_loaned_bytes 4 0
arc_prune 4 1836218
arc_meta_used 4 5541209624
arc_meta_limit 4 2889867264
arc_meta_max 4 5541544864
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment