LinuxLists.cc - Commit for mm/page_alloc.c breaks boot process on my machine

[permalink] [raw]

Subject: Re: Commit for mm/page_alloc.c breaks boot process on my machine

On (01/02/08 19:42), Gerhard Pircher didst pronounce:
>
> I'm trying to get the 2.6.24 Linux kernel booting on my AmigaOne machine
> (it's a desktop PowerPC G4 machine with 1.5GB RAM). I have written the
> device tree and platform setup files for arch/powerpc, which seem to be
> working so far.
> Kernel version 2.6.24-rc2 was booting fine with my patches until the
> kernel executes the INIT command. But kernel release v2.6.24 dies early
> during the boot process (even before the radeonfb driver initializes and
> outputs kernel messages). Bisecting revealed that this commit causes my
> machine to fail booting:
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=44048d700bcbfaf4bcca6e2e0a73d89d01ec0878
> (Revert "Bias the placement of kernel pages at lower PFNs")
>
> The machine boots fine, if I revert the patch. But I wonder what side
> effects this patch may have? Can somebody shed some light on this?
>

With this patch, early boot would use pages from lower PFNs. Without the
patch, it would use memory from higher PFNs. That is the only real
difference.

1. Is there any chance that all of your memory is not being properly
initialised?
2. Any chance of seeing a dmesg log?
3. If you boot with the patch reverted and then do something to consume
all memory like build loads of kernel trees simultaneously, do you see
any problems?

--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab

2008-02-01 20:05:29

[permalink] [raw]

Subject: Re: Commit for mm/page_alloc.c breaks boot process on my machine

-------- Original-Nachricht --------
> Datum: Fri, 1 Feb 2008 19:11:19 +0000
> Von: Mel Gorman <[email protected]>
> An: Gerhard Pircher <[email protected]>
> CC: [email protected]
> Betreff: Re: Commit for mm/page_alloc.c breaks boot process on my machine

> With this patch, early boot would use pages from lower PFNs. Without the
> patch, it would use memory from higher PFNs. That is the only real
> difference.
>
> 1. Is there any chance that all of your memory is not being properly
> initialised?
Do you mean uninitialized hardware? That shouldn't be a problem, since
older kernels (e.g. 2.6.19 with platform patches for arch/ppc) are running
fine.

> 2. Any chance of seeing a dmesg log?
That's a little bit of a problem. The kernel log in memory doesn't show
any kernel oops, but is also fragmented (small fragments seem to have been
overwritten with 0x0). I could extract it by using the U-boot firmware,
but that takes some time, as the serial console is not yet working.
I tried to boot the kernel with 512MB and 1.5GB RAM. The amount of memory
doesn't have any effect and the amount of memory is reported correctly
in the kernel log.

> 3. If you boot with the patch reverted and then do something to consume
> all memory like build loads of kernel trees simultaneously, do you see
> any problems?
Well, I can't answer this question. The kernel currently locks up when
loading the INIT program. But that is another problem (I still have to
bisect it) and doesn't seem to be related to this problem.

Thanks for your help!

Gerhard
--
GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS.
Alle Infos und kostenlose Anmeldung: http://www.gmx.net/de/go/freemail

2008-02-01 20:25:28

[permalink] [raw]

Subject: Re: Commit for mm/page_alloc.c breaks boot process on my machine

On (01/02/08 21:05), Gerhard Pircher didst pronounce:
>
> -------- Original-Nachricht --------
> > Datum: Fri, 1 Feb 2008 19:11:19 +0000
> > Von: Mel Gorman <[email protected]>
> > An: Gerhard Pircher <[email protected]>
> > CC: [email protected]
> > Betreff: Re: Commit for mm/page_alloc.c breaks boot process on my machine
>
> > With this patch, early boot would use pages from lower PFNs. Without the
> > patch, it would use memory from higher PFNs. That is the only real
> > difference.
> >
> > 1. Is there any chance that all of your memory is not being properly
> > initialised?
> Do you mean uninitialized hardware? That shouldn't be a problem, since
> older kernels (e.g. 2.6.19 with platform patches for arch/ppc) are running
> fine.
>

I meant uninitialised memory but I also wonder could something like this
happen if you are trying to use memory that doesn't exist. i.e. you are
trying to access more memory than you really have but you indicate later
that this is not the case.

> > 2. Any chance of seeing a dmesg log?
> That's a little bit of a problem. The kernel log in memory doesn't show
> any kernel oops, but is also fragmented (small fragments seem to have been
> overwritten with 0x0).

err, that doesn't sound very healthy.

> I could extract it by using the U-boot firmware,
> but that takes some time, as the serial console is not yet working.
> I tried to boot the kernel with 512MB and 1.5GB RAM. The amount of memory
> doesn't have any effect and the amount of memory is reported correctly
> in the kernel log.
>
> > 3. If you boot with the patch reverted and then do something to consume
> > all memory like build loads of kernel trees simultaneously, do you see
> > any problems?
> Well, I can't answer this question. The kernel currently locks up when
> loading the INIT program. But that is another problem (I still have to
> bisect it) and doesn't seem to be related to this problem.

INIT would be the first MOVABLE allocation so it would be using memory
at the end of the physical adddress range. i.e. the crash happens when
memory towards the end and the only difference between the patch applied
and reverted is when it happens.

Could you try booting with 16MB less memory using mem=?

--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab

2008-02-01 21:07:10

[permalink] [raw]

Subject: Re: Commit for mm/page_alloc.c breaks boot process on my machine

-------- Original-Nachricht --------
> Datum: Fri, 1 Feb 2008 20:25:18 +0000
> Von: Mel Gorman <[email protected]>
> An: Gerhard Pircher <[email protected]>
> CC: [email protected]
> Betreff: Re: Commit for mm/page_alloc.c breaks boot process on my machine

> I meant uninitialised memory but I also wonder could something like this
> happen if you are trying to use memory that doesn't exist. i.e. you are
> trying to access more memory than you really have but you indicate later
> that this is not the case.
Good question. The memory is in the physical address range from 0x00000000
to 0x60000000 (1536MB).

> > > 2. Any chance of seeing a dmesg log?
> > That's a little bit of a problem. The kernel log in memory doesn't show
> > any kernel oops, but is also fragmented (small fragments seem to have
> > been overwritten with 0x0).
>
> err, that doesn't sound very healthy.
Yeah, I know. But the platform code hasn't changed much when porting it
from arch/ppc to arch/powerpc. That's why I'm a little bit lost in this
case. :-)

> > Well, I can't answer this question. The kernel currently locks up when
> > loading the INIT program. But that is another problem (I still have to
> > bisect it) and doesn't seem to be related to this problem.
>
> INIT would be the first MOVABLE allocation so it would be using memory
> at the end of the physical adddress range. i.e. the crash happens when
> memory towards the end and the only difference between the patch applied
> and reverted is when it happens.
Oh, that sounds interesting!

> Could you try booting with 16MB less memory using mem=?
I started the kernel with 512MB RAM (mem=496) and 1.5GB (mem=1520). The
kernel oopes in both cases with a "Unable to handle kernel paging request
for data address 0xbffff000", followed by a "Oops: kernel access of bad
area, sig 11" message. The end of the stack trace shows the start_here()
function.
I'm not a PowerPC expert, but if 0xbffff000 is a virtual address, then
it would be in the user program address space, right? If it is a physical
address, then it is somewhere in the unallocated PCI address space.

Gerhard

--
Ist Ihr Browser Vista-kompatibel? Jetzt die neuesten
Browser-Versionen downloaden: http://www.gmx.net/de/go/browser

2008-02-04 10:42:44

[permalink] [raw]

Subject: Re: Commit for mm/page_alloc.c breaks boot process on my machine

On (01/02/08 22:06), Gerhard Pircher didst pronounce:
>
> -------- Original-Nachricht --------
> > Datum: Fri, 1 Feb 2008 20:25:18 +0000
> > Von: Mel Gorman <[email protected]>
> > An: Gerhard Pircher <[email protected]>
> > CC: [email protected]
> > Betreff: Re: Commit for mm/page_alloc.c breaks boot process on my machine
>
> > I meant uninitialised memory but I also wonder could something like this
> > happen if you are trying to use memory that doesn't exist. i.e. you are
> > trying to access more memory than you really have but you indicate later
> > that this is not the case.
>
> Good question. The memory is in the physical address range from 0x00000000
> to 0x60000000 (1536MB).
>
> > > > 2. Any chance of seeing a dmesg log?
> > > That's a little bit of a problem. The kernel log in memory doesn't show
> > > any kernel oops, but is also fragmented (small fragments seem to have
> > > been overwritten with 0x0).
> >
> > err, that doesn't sound very healthy.
>
> Yeah, I know. But the platform code hasn't changed much when porting it
> from arch/ppc to arch/powerpc. That's why I'm a little bit lost in this
> case. :-)
>

I'm at a bit of a loss to guess what might have changed in powerpc code
that would explain this. I've added the linuxppc-dev mailing list in
case they can make a guess.

I think you are also going to need to start bisecting searching
specifically for the patch that causes these overwrites.

> > > Well, I can't answer this question. The kernel currently locks up when
> > > loading the INIT program. But that is another problem (I still have to
> > > bisect it) and doesn't seem to be related to this problem.
> >
> > INIT would be the first MOVABLE allocation so it would be using memory
> > at the end of the physical adddress range. i.e. the crash happens when
> > memory towards the end and the only difference between the patch applied
> > and reverted is when it happens.
> Oh, that sounds interesting!
>
> > Could you try booting with 16MB less memory using mem=?
> I started the kernel with 512MB RAM (mem=496) and 1.5GB (mem=1520). The
> kernel oopes in both cases with a "Unable to handle kernel paging request
> for data address 0xbffff000", followed by a "Oops: kernel access of bad
> area, sig 11" message. The end of the stack trace shows the start_here()
> function.
> I'm not a PowerPC expert, but if 0xbffff000 is a virtual address, then
> it would be in the user program address space, right? If it is a physical
> address, then it is somewhere in the unallocated PCI address space.
>

It's a virtual address so it depends on the value of CONFIG_KERNEL_START
as to whether this is a user program address or not.

--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab

2008-02-04 22:20:58

[permalink] [raw]

Subject: Re: Commit for mm/page_alloc.c breaks boot process on my machine

-------- Original-Nachricht --------
> Datum: Mon, 4 Feb 2008 10:42:32 +0000
> Von: Mel Gorman <[email protected]>
> An: Gerhard Pircher <[email protected]>
> CC: [email protected], [email protected]
> Betreff: Re: Commit for mm/page_alloc.c breaks boot process on my machine

> > > > > 2. Any chance of seeing a dmesg log?
> > > > That's a little bit of a problem. The kernel log in memory doesn't
> > > > show any kernel oops, but is also fragmented (small fragments seem
> > > > to have been overwritten with 0x0).
> > >
> > > err, that doesn't sound very healthy.
> >
> > Yeah, I know. But the platform code hasn't changed much when porting it
> > from arch/ppc to arch/powerpc. That's why I'm a little bit lost in this
> > case. :-)
> >
>
> I'm at a bit of a loss to guess what might have changed in powerpc code
> that would explain this. I've added the linuxppc-dev mailing list in
> case they can make a guess.
Yes, I'll try to get some comments on the linuxppc-dev mailing list.

> I think you are also going to need to start bisecting searching
> specifically for the patch that causes these overwrites.
I think I had a similar problem with kernel v2.6.23, too and therefore
waited for kernel 2.6.24. So the problem may exist since a long time.

> It's a virtual address so it depends on the value of CONFIG_KERNEL_START
> as to whether this is a user program address or not.
AFAIK start_kernel() is called very early in the boot process, were no
user or kernel thread is active. I also wonder why the kernel crashes when
the mem boot option is used. Let's see what the linuxppc-dev people say.

Thanks for your help!

Gerhard

--
Psssst! Schon vom neuen GMX MultiMessenger geh?rt?
Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger

2008-02-04 23:30:28

by Michael Ellerman

[permalink] [raw]

Subject: Re: Commit for mm/page_alloc.c breaks boot process on my machine

On Mon, 2008-02-04 at 23:20 +0100, Gerhard Pircher wrote:
> -------- Original-Nachricht --------
> > Datum: Mon, 4 Feb 2008 10:42:32 +0000
> > Von: Mel Gorman <[email protected]>
> > An: Gerhard Pircher <[email protected]>
> > CC: [email protected], [email protected]
> > Betreff: Re: Commit for mm/page_alloc.c breaks boot process on my machine
>
> > > > > > 2. Any chance of seeing a dmesg log?
> > > > > That's a little bit of a problem. The kernel log in memory doesn't
> > > > > show any kernel oops, but is also fragmented (small fragments seem
> > > > > to have been overwritten with 0x0).
> > > >
> > > > err, that doesn't sound very healthy.
> > >
> > > Yeah, I know. But the platform code hasn't changed much when porting it
> > > from arch/ppc to arch/powerpc. That's why I'm a little bit lost in this
> > > case. :-)
> > >
> >
> > I'm at a bit of a loss to guess what might have changed in powerpc code
> > that would explain this. I've added the linuxppc-dev mailing list in
> > case they can make a guess.
> Yes, I'll try to get some comments on the linuxppc-dev mailing list.
>
> > I think you are also going to need to start bisecting searching
> > specifically for the patch that causes these overwrites.
> I think I had a similar problem with kernel v2.6.23, too and therefore
> waited for kernel 2.6.24. So the problem may exist since a long time.
>
> > It's a virtual address so it depends on the value of CONFIG_KERNEL_START
> > as to whether this is a user program address or not.
> AFAIK start_kernel() is called very early in the boot process, were no
> user or kernel thread is active. I also wonder why the kernel crashes when
> the mem boot option is used. Let's see what the linuxppc-dev people say.

Can you post a fresh bug report? There's lots of details missing from
the email you forwarded.

cheers

--
Michael Ellerman
OzLabs, IBM Australia Development Lab

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person

Attachments:

signature.asc (189.00 B)
This is a digitally signed message part

2008-02-05 00:02:26

[permalink] [raw]

Subject: Re: Commit for mm/page_alloc.c breaks boot process on my machine

>
> It's a virtual address so it depends on the value of CONFIG_KERNEL_START
> as to whether this is a user program address or not.

Shouldn't it be PAGE_OFFSET ? I mean, CONFIG_KERNEL_START should work
on powerpc, but still...

Ben.

2008-02-05 00:07:13

[permalink] [raw]

Subject: Re: Commit for mm/page_alloc.c breaks boot process on my machine

On Fri, 2008-02-01 at 21:05 +0100, Gerhard Pircher wrote:
> -------- Original-Nachricht --------
> > Datum: Fri, 1 Feb 2008 19:11:19 +0000
> > Von: Mel Gorman <[email protected]>
> > An: Gerhard Pircher <[email protected]>
> > CC: [email protected]
> > Betreff: Re: Commit for mm/page_alloc.c breaks boot process on my machine
>
> > With this patch, early boot would use pages from lower PFNs. Without the
> > patch, it would use memory from higher PFNs. That is the only real
> > difference.
> >
> > 1. Is there any chance that all of your memory is not being properly
> > initialised?
> Do you mean uninitialized hardware? That shouldn't be a problem, since
> older kernels (e.g. 2.6.19 with platform patches for arch/ppc) are running
> fine.

No, it looks more to me like you aren't properly defining the available
memory somewhere, or along those lines. Are you using CONFIG_HIGHMEM ? I
think there have been some issues until recent patches from Kumar, if
you try to reserve memory in the highmem region (though that may not be
your problem).

Or maybe you are colliding with some of the non-coherent hacks ?

> > 2. Any chance of seeing a dmesg log?
> That's a little bit of a problem. The kernel log in memory doesn't show
> any kernel oops, but is also fragmented (small fragments seem to have been
> overwritten with 0x0). I could extract it by using the U-boot firmware,
> but that takes some time, as the serial console is not yet working.
> I tried to boot the kernel with 512MB and 1.5GB RAM. The amount of memory
> doesn't have any effect and the amount of memory is reported correctly
> in the kernel log.
>
> > 3. If you boot with the patch reverted and then do something to consume
> > all memory like build loads of kernel trees simultaneously, do you see
> > any problems?
> Well, I can't answer this question. The kernel currently locks up when
> loading the INIT program. But that is another problem (I still have to
> bisect it) and doesn't seem to be related to this problem.

What happens without CONFIG_HIGHMEM ?

Ben.

2008-02-05 00:08:21

[permalink] [raw]

Subject: Re: Commit for mm/page_alloc.c breaks boot process on my machine

On Fri, 2008-02-01 at 22:06 +0100, Gerhard Pircher wrote:
> > Could you try booting with 16MB less memory using mem=?
> I started the kernel with 512MB RAM (mem=496) and 1.5GB (mem=1520).
> The
> kernel oopes in both cases with a "Unable to handle kernel paging
> request
> for data address 0xbffff000", followed by a "Oops: kernel access of
> bad
> area, sig 11" message. The end of the stack trace shows the
> start_here()
> function.
> I'm not a PowerPC expert, but if 0xbffff000 is a virtual address, then
> it would be in the user program address space, right? If it is a
> physical
> address, then it is somewhere in the unallocated PCI address space

Have you tried DEBUG_PAGEALLOC ? It might fault earlier which might give
us better informations. Also, a stactrace from the oops might be useful,
along with a copy of your device-tree.

Ben.

2008-02-05 10:23:59

[permalink] [raw]

Subject: Re: Commit for mm/page_alloc.c breaks boot process on my machine

On (05/02/08 11:01), Benjamin Herrenschmidt didst pronounce:
>
> >
> > It's a virtual address so it depends on the value of CONFIG_KERNEL_START
> > as to whether this is a user program address or not.
>
> Shouldn't it be PAGE_OFFSET ? I mean, CONFIG_KERNEL_START should work
> on powerpc, but still...
>

mel@arnold:~/git/linux-2.6$ grep PAGE_OFFSET include/asm-ppc/page.h
#define PAGE_OFFSET CONFIG_KERNEL_START

--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab

2008-02-05 20:50:56

[permalink] [raw]

Subject: Re: Commit for mm/page_alloc.c breaks boot process on my machine

On Tue, 2008-02-05 at 10:23 +0000, Mel Gorman wrote:
> On (05/02/08 11:01), Benjamin Herrenschmidt didst pronounce:
> >
> > >
> > > It's a virtual address so it depends on the value of CONFIG_KERNEL_START
> > > as to whether this is a user program address or not.
> >
> > Shouldn't it be PAGE_OFFSET ? I mean, CONFIG_KERNEL_START should work
> > on powerpc, but still...
> >
>
> mel@arnold:~/git/linux-2.6$ grep PAGE_OFFSET include/asm-ppc/page.h
> #define PAGE_OFFSET CONFIG_KERNEL_START

On powerpc sure.... but it's still dodgy I think.

Ben.

2008-02-05 23:17:20