2008-02-04 16:11:32

by Sachin Sant

[permalink] [raw]
Subject: [mm] Crashkernel memory reservation fails with 2.6.24-rc8-mm1

Linux version 2.6.24-rc8-mm1 (root@xxxxxxxxxx) (gcc version 4.1.1 20070105 (Red Hat 4.1.1-52)) #2 SMP Mon Feb 4 20:24:24 IST 2008
Command line: root=/dev/sdd1 console=tty0 console=ttyS0,38400 crashkernel=128M@32M selinux=0
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009dc00 (usable)
BIOS-e820: 000000000009dc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 00000000c7fcae00 (usable)
BIOS-e820: 00000000c7fcae00 - 00000000c7fcf800 (ACPI data)
BIOS-e820: 00000000c7fcf800 - 00000000c8000000 (reserved)
BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000238000000 (usable)
Entering add_active_range(0, 0, 157) 0 entries of 3200 used
Entering add_active_range(0, 256, 819146) 1 entries of 3200 used
Entering add_active_range(0, 1048576, 2326528) 2 entries of 3200 used
end_pfn_map = 2326528
early_ioremap(000000000000040e, 00000002) => -000002105539570
early_iounmap(ffffffff8280040e, 00000002)
early_ioremap(000000000009dc00, 00000002) => -000002104894464
early_iounmap(ffffffff8289dc00, 00000002)
early_ioremap(0000000000008000, 00001000) => -000002105507840
early_ioremap(0000000000009000, 00001000) => -000002103406592
early_iounmap(ffffffff82a09000, 00001000)
early_ioremap(000000000000a000, 00001000) => -000002103402496
early_iounmap(ffffffff82a0a000, 00001000)
early_ioremap(000000000000b000, 00001000) => -000002103398400
early_iounmap(ffffffff82a0b000, 00001000)
early_ioremap(000000000000c000, 00001000) => -000002103394304
early_iounmap(ffffffff82a0c000, 00001000)
early_ioremap(000000000000d000, 00001000) => -000002103390208
early_iounmap(ffffffff82a0d000, 00001000)
early_ioremap(000000000000e000, 00001000) => -000002103386112
early_iounmap(ffffffff82a0e000, 00001000)
early_ioremap(000000000000f000, 00001000) => -000002103382016
early_iounmap(ffffffff82a0f000, 00001000)
early_ioremap(0000000000010000, 00001000) => -000002103377920
early_iounmap(ffffffff82a10000, 00001000)
early_ioremap(0000000000011000, 00001000) => -000002103373824
early_iounmap(ffffffff82a11000, 00001000)
early_iounmap(ffffffff82808000, 00001000)
early_ioremap(00000000000f0000, 00010000) => -000002104557568
DMI 2.3 present.
early_ioremap(00000000000f83eb, 000009e1) => -000002102426645
early_iounmap(ffffffff82af83eb, 000009e1)
early_iounmap(ffff81000000040e, 00000002)
early_iounmap(ffff81000009dc00, 00000400)
early_iounmap(ffff8100000e0000, 00020000)
ACPI: RSDP 000FDFB0, 0024 (r2 IBM )
early_iounmap(ffff8100000fdfb0, 00000024)
early_iounmap(ffff8100c7fcf700, 00000024)
early_iounmap(ffff8100c7fcf700, 0000003c)
ACPI: XSDT C7FCF700, 003C (r1 IBM SERONYXP 1001 IBM 45444F43)
early_iounmap(ffff8100c7fcf700, 00000024)
early_iounmap(ffff8100c7fcf700, 0000003c)
ACPI: FACP C7FCF640, 0084 (r2 IBM SERONYXP 1001 IBM 45444F43)
early_iounmap(ffff8100c7fcf640, 00000024)
early_iounmap(ffff8100c7fcf640, 00000084)
ACPI: DSDT C7FCAE00, 4168 (r2 IBM SERGEODE 1000 INTL 20041203)
early_iounmap(ffff8100c7fcae00, 00000024)
ACPI: FACS C7FCF500, 0040
early_iounmap(ffff8100c7fcf500, 00000024)
ACPI: APIC C7FCF580, 009C (r1 IBM SERONYXP 1001 IBM 45444F43)
early_iounmap(ffff8100c7fcf580, 00000024)
ACPI: MCFG C7FCF540, 003C (r1 IBM SERONYXP 1001 IBM 45444F43)
early_iounmap(ffff8100c7fcf540, 00000024)
early_iounmap(ffff8100c7fcae00, 00000024)
early_iounmap(ffff8100c7fcae00, 00000024)
early_iounmap(ffff8100c7fcae00, 00000024)
early_iounmap(ffff8100c7fcae00, 00000024)
No NUMA configuration found
Faking a node at 0000000000000000-0000000238000000
Entering add_active_range(0, 0, 157) 0 entries of 3200 used
Entering add_active_range(0, 256, 819146) 1 entries of 3200 used
Entering add_active_range(0, 1048576, 2326528) 2 entries of 3200 used
Bootmem setup node 0 0000000000000000-0000000238000000
early_ioremap(0000000000000000, 00000400) => -000002103443456
early_iounmap(ffffffff82a00000, 00000400)
early_ioremap(000000000009fc00, 00000400) => -000002102789120
early_iounmap(ffffffff82a9fc00, 00000400)
early_ioremap(00000000000f0000, 00010000) => -000002102460416
early_iounmap(ffffffff82af0000, 00010000)
early_ioremap(000000000000040e, 00000002) => -000002103442418
early_iounmap(ffffffff82a0040e, 00000002)
early_ioremap(000000000009dc00, 00001000) => -000002102797312
early_iounmap(ffffffff82a9dc00, 00001000)
DEBUG crash_base = 2000000 crash_size = 8000000
hm, page 02a9d000 reserved twice.
crashkernel reservation failed - memory is in use
[ffffe20000000000-ffffe200001fffff] PMD ->ffff810001200000 on node 0
[ffffe20000200000-ffffe200003fffff] PMD ->ffff810001600000 on node 0
[ffffe20000400000-ffffe200005fffff] PMD ->ffff810001a00000 on node 0
[ffffe20000600000-ffffe200007fffff] PMD ->ffff810001e00000 on node 0
[ffffe20000800000-ffffe200009fffff] PMD ->ffff810002200000 on node 0
[ffffe20000a00000-ffffe20000bfffff] PMD ->ffff810002600000 on node 0
[ffffe20000c00000-ffffe20000dfffff] PMD ->ffff810002e00000 on node 0
[ffffe20000e00000-ffffe20000ffffff] PMD ->ffff810003200000 on node 0
[ffffe20001000000-ffffe200011fffff] PMD ->ffff810003600000 on node 0
[ffffe20001200000-ffffe200013fffff] PMD ->ffff810003a00000 on node 0
[ffffe20001400000-ffffe200015fffff] PMD ->ffff810003e00000 on node 0
[ffffe20001600000-ffffe200017fffff] PMD ->ffff810004200000 on node 0
[ffffe20001800000-ffffe200019fffff] PMD ->ffff810004600000 on node 0
[ffffe20001a00000-ffffe20001bfffff] PMD ->ffff810004a00000 on node 0
[ffffe20001c00000-ffffe20001dfffff] PMD ->ffff810004e00000 on node 0
[ffffe20001e00000-ffffe20001ffffff] PMD ->ffff810005200000 on node 0
[ffffe20002000000-ffffe200021fffff] PMD ->ffff810005600000 on node 0
[ffffe20002200000-ffffe200023fffff] PMD ->ffff810005a00000 on node 0
[ffffe20002400000-ffffe200025fffff] PMD ->ffff810005e00000 on node 0
[ffffe20002600000-ffffe200027fffff] PMD ->ffff810006200000 on node 0
[ffffe20002800000-ffffe200029fffff] PMD ->ffff810006600000 on node 0
[ffffe20002a00000-ffffe20002bfffff] PMD ->ffff810006a00000 on node 0
[ffffe20003800000-ffffe200039fffff] PMD ->ffff810006e00000 on node 0
[ffffe20003a00000-ffffe20003bfffff] PMD ->ffff810007200000 on node 0
[ffffe20003c00000-ffffe20003dfffff] PMD ->ffff810007600000 on node 0
[ffffe20003e00000-ffffe20003ffffff] PMD ->ffff810007a00000 on node 0
[ffffe20004000000-ffffe200041fffff] PMD ->ffff810007e00000 on node 0
[ffffe20004200000-ffffe200043fffff] PMD ->ffff810008200000 on node 0
[ffffe20004400000-ffffe200045fffff] PMD ->ffff810008600000 on node 0
[ffffe20004600000-ffffe200047fffff] PMD ->ffff810008a00000 on node 0
[ffffe20004800000-ffffe200049fffff] PMD ->ffff810008e00000 on node 0
[ffffe20004a00000-ffffe20004bfffff] PMD ->ffff810009200000 on node 0
[ffffe20004c00000-ffffe20004dfffff] PMD ->ffff810009600000 on node 0
[ffffe20004e00000-ffffe20004ffffff] PMD ->ffff810009a00000 on node 0
[ffffe20005000000-ffffe200051fffff] PMD ->ffff810009e00000 on node 0
[ffffe20005200000-ffffe200053fffff] PMD ->ffff81000a200000 on node 0
[ffffe20005400000-ffffe200055fffff] PMD ->ffff81000a600000 on node 0
[ffffe20005600000-ffffe200057fffff] PMD ->ffff81000aa00000 on node 0
[ffffe20005800000-ffffe200059fffff] PMD ->ffff81000ae00000 on node 0
[ffffe20005a00000-ffffe20005bfffff] PMD ->ffff81000b200000 on node 0
[ffffe20005c00000-ffffe20005dfffff] PMD ->ffff81000b600000 on node 0
[ffffe20005e00000-ffffe20005ffffff] PMD ->ffff81000ba00000 on node 0
[ffffe20006000000-ffffe200061fffff] PMD ->ffff81000be00000 on node 0
[ffffe20006200000-ffffe200063fffff] PMD ->ffff81000c200000 on node 0
[ffffe20006400000-ffffe200065fffff] PMD ->ffff81000c600000 on node 0
[ffffe20006600000-ffffe200067fffff] PMD ->ffff81000ca00000 on node 0
[ffffe20006800000-ffffe200069fffff] PMD ->ffff81000ce00000 on node 0
[ffffe20006a00000-ffffe20006bfffff] PMD ->ffff81000d200000 on node 0
[ffffe20006c00000-ffffe20006dfffff] PMD ->ffff81000d600000 on node 0
[ffffe20006e00000-ffffe20006ffffff] PMD ->ffff81000da00000 on node 0
[ffffe20007000000-ffffe200071fffff] PMD ->ffff81000de00000 on node 0
[ffffe20007200000-ffffe200073fffff] PMD ->ffff81000e200000 on node 0
[ffffe20007400000-ffffe200075fffff] PMD ->ffff81000e600000 on node 0
[ffffe20007600000-ffffe200077fffff] PMD ->ffff81000ea00000 on node 0
[ffffe20007800000-ffffe200079fffff] PMD ->ffff81000ee00000 on node 0
[ffffe20007a00000-ffffe20007bfffff] PMD ->ffff81000f200000 on node 0
[ffffe20007c00000-ffffe20007dfffff] PMD ->ffff81000f600000 on node 0
sizeof(struct page) = 56
Zone PFN ranges:
DMA 0 -> 4096
DMA32 4096 -> 1048576
Normal 1048576 -> 2326528
Movable zone start PFN for each node
early_node_map[3] active PFN ranges
0: 0 -> 157
0: 256 -> 819146
0: 1048576 -> 2326528
On node 0 totalpages: 2096999
DMA zone: 56 pages used for memmap
DMA zone: 1959 pages reserved
DMA zone: 1982 pages, LIFO batch:0
DMA32 zone: 14280 pages used for memmap
DMA32 zone: 800770 pages, LIFO batch:31
Normal zone: 17472 pages used for memmap
Normal zone: 1260480 pages, LIFO batch:31
Movable zone: 0 pages used for memmap
ACPI: PM-Timer IO Port: 0x588
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 (Bootup-CPU)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x06] enabled)
Processor #6
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Processor #1
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x07] enabled)
Processor #7
ACPI: LAPIC_NMI (acpi_id[0x00] dfl dfl lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] dfl dfl lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x02] dfl dfl lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x03] dfl dfl lint[0x1])
ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 14, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x0d] address[0xfec80000] gsi_base[24])
IOAPIC[1]: apic_id 13, address 0xfec80000, GSI 24-47
ACPI: IOAPIC (id[0x0c] address[0xfec80400] gsi_base[48])
IOAPIC[2]: apic_id 12, address 0xfec80400, GSI 48-71
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Setting APIC routing to flat
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at cc000000 (gap: c8000000:36c00000)
SMP: Allowing 4 CPUs, 0 hotplug CPUs
PERCPU: Allocating 66848 bytes of per cpu data
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 2063232
Policy zone: Normal
Kernel command line: root=/dev/sdd1 console=tty0 console=ttyS0,38400 crashkernel=128M@32M selinux=0
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
TSC calibrated against PM_TIMER
Marking TSC unstable due to TSCs unsynchronized

<< Rest of the log messages were removed >>


Attachments:
dmesg-log-2624rc8mm1 (10.60 kB)

2008-02-04 18:38:51

by Vivek Goyal

[permalink] [raw]
Subject: Re: [mm] Crashkernel memory reservation fails with 2.6.24-rc8-mm1

On Mon, Feb 04, 2008 at 09:41:11PM +0530, Sachin P. Sant wrote:
> While trying to configure kdump with 2.6.24-rc8-mm1 [ on a x86-64 box ]
> i ran into this problem. Here is the snippet from dmesg during the
> failure. [ dmesg log attached ]
>
> early_ioremap(000000000000040e, 00000002) => -000002103442418
> early_iounmap(ffffffff82a0040e, 00000002)
> early_ioremap(000000000009dc00, 00001000) => -000002102797312
> early_iounmap(ffffffff82a9dc00, 00001000)
> crashkernel reservation failed - memory is in use
> [ffffe20000000000-ffffe200001fffff] PMD ->ffff810001200000 on node 0
> [ffffe20000200000-ffffe200003fffff] PMD ->ffff810001600000 on node 0
> [ffffe20000400000-ffffe200005fffff] PMD ->ffff810001a00000 on node 0
> [ffffe20000600000-ffffe200007fffff] PMD ->ffff810001e00000 on node 0
>
> I am using 128M@32M value for crashkernel parameter.
>

Hmm..., So somebody has already reserved the memory we are trying to
reserve. Bernhard's BOOTMEM_EXCLUSIVE flag is atleast giving the warning
otherwise it would have silently created an overlapping reserved memory
area.

Sachin, can you please put some debugging statements before the calls
to reserve_bootmem() in arch/x86 to find out who all are reserving what
memory area and then we can nail down the guy who is reserving some
memory between 32M and 160M

Wished we had some debugging printk statements for reserve_bootmem too
(the way we have for early_ioremap and early_iounmap).

Bernahard, any idea who is the competitor here?

Thanks
Vivek

2008-02-04 19:35:35

by Bernhard Walle

[permalink] [raw]
Subject: Re: [mm] Crashkernel memory reservation fails with 2.6.24-rc8-mm1

* Vivek Goyal <[email protected]> [2008-02-04 19:38]:
>
> Bernahard, any idea who is the competitor here?

Hm ..., can you boot the kernel without crashkernel= and provide the
/proc/iomem?


Bernhard

2008-02-05 05:34:06

by Sachin Sant

[permalink] [raw]
Subject: Re: [mm] Crashkernel memory reservation fails with 2.6.24-rc8-mm1

00000000-0009dbff : System RAM
0009dc00-0009ffff : reserved
000e0000-000fffff : reserved
00100000-c7fcadff : System RAM
00200000-004780d6 : Kernel code
004780d7-0059e867 : Kernel data
008a2000-009969ab : Kernel bss
c7fcae00-c7fcf7ff : ACPI Tables
c7fcf800-c7ffffff : reserved
c8000000-c9ffffff : PCI Bus #07
c9ff0000-c9ffffff : 0000:07:00.0
c9ff0000-c9ffffff : tg3
ca000000-cbbfffff : PCI Bus #06
cbbf0000-cbbfffff : 0000:06:00.0
cbbf0000-cbbfffff : tg3
cbc00000-cdafffff : PCI Bus #02
cbc00000-cbffffff : PCI Bus #04
cc000000-cdafffff : PCI Bus #03
cdb00000-cfffffff : PCI Bus #02
cdb00000-cdefffff : PCI Bus #04
cdfff000-cdffffff : 0000:02:00.2
cdfff000-cdfff027 : shpchp
ce000000-cfffffff : PCI Bus #03
cfffc000-cfffdfff : 0000:03:07.1
cfffc000-cfffcfff : aic79xx
cfffe000-cfffffff : 0000:03:07.0
cfffe000-cfffefff : aic79xx
d0000000-ddffffff : PCI Bus #01
d0000000-d7ffffff : 0000:01:06.0
de000000-dfffffff : PCI Bus #01
dfff0000-dfffffff : 0000:01:06.0
e0000000-efffffff : PCI MMCONFIG 0
f0000000-f00003ff : 0000:00:1d.7
f0000000-f00003ff : ehci_hcd
f0000400-f00007ff : 0000:00:1f.1
fec00000-fec00fff : IOAPIC 0
fec80000-fec80fff : IOAPIC 1
fed20800-fed208ff : pnp 00:0e
fed30800-fed308ff : pnp 00:0e
fee00000-fee00fff : Local APIC
ff000000-ff000fff : 0000:00:00.0
ff000000-ff000fff : pnp 00:0e
fff00000-ffffffff : pnp 00:0e
100000000-237ffffff : System RAM


Attachments:
iomem_with_crashkernel (1.39 kB)
iomem_without_crashkernel (1.39 kB)
Download all attachments

2008-02-05 06:39:46

by Sachin Sant

[permalink] [raw]
Subject: Re: [mm] Crashkernel memory reservation fails with 2.6.24-rc8-mm1

Sachin P. Sant wrote:
> Bernhard Walle wrote:
>> * Vivek Goyal <[email protected]> [2008-02-04 19:38]:
>>> Bernahard, any idea who is the competitor here?
>> Hm ..., can you boot the kernel without crashkernel= and provide the
>> /proc/iomem?
> Attached is the /proc/iomem output with and without crashkernel
> parameter.
>
> Adding debug gives this extra information.
>
> hm, page 02a9d000 reserved twice.
> crashkernel reservation failed - memory is in use
>
> Will try to add more debug statements.
Hm . no problem with 2.6.24-mm1.

early res: 4 [9dc00-a0bff] EBDA
early res: 5 [8000-11fff] PGTABLE
Reserving 128MB of memory at 32MB for crashkernel (System RAM: 9088MB)
[ffffe20000000000-ffffe200001fffff] PMD ->ffff810001200000 on node 0

# cat /proc/iomem
......
008b8000-0099dc8b : Kernel bss
02000000-09ffffff : Crash kernel
c7fcae00-c7fcf7ff : ACPI Tables
.....

Thanks
-Sachin

2008-02-05 08:09:18

by Bernhard Walle

[permalink] [raw]
Subject: Re: [mm] Crashkernel memory reservation fails with 2.6.24-rc8-mm1

* Sachin P. Sant <[email protected]> [2008-02-05 06:33]:
> Bernhard Walle wrote:
>> * Vivek Goyal <[email protected]> [2008-02-04 19:38]:
>>
>>> Bernahard, any idea who is the competitor here?
>>>
>> Hm ..., can you boot the kernel without crashkernel= and provide the
>> /proc/iomem?
>>
> Attached is the /proc/iomem output with and without crashkernel
> parameter.

Looks ok. In my case when I added the warning message the kernel BSS
was too large. But that doesn't seem to be the case here, BSS ends
with 9M and you use 16M as start address for the crashkernel.


Bernhard


Attachments:
(No filename) (597.00 B)
(No filename) (194.00 B)
Download all attachments