We're seeing programs die with "bus error" (SIGBUS) randomly on a dual
processor Opteron machine. I've run memtest86+ and cpuburn stress tests
with no failure. gdb on a core file seems uninteresting. Is there some
way to trace the kernel to try to get more insight?
Thanks!
--
Orion Poplawski
System Administrator 303-415-9701 x222
NWRA/CoRA Division FAX: 303-415-9702
3380 Mitchell Lane [email protected]
Boulder, CO 80301 http://www.cora.nwra.com
On Fri, 22 Sep 2006, Orion Poplawski wrote:
>
> We're seeing programs die with "bus error" (SIGBUS) randomly on a dual
> processor Opteron machine. I've run memtest86+ and cpuburn stress tests with
> no failure. gdb on a core file seems uninteresting. Is there some way to
> trace the kernel to try to get more insight?
Which kernel?
Opterons in SMP environments had a TLB flushing bug if the tlb flush
filter was enabled.
We've had that fixed for a long time, but if you have a kernel older that
2.6.14-rc2 or so (but still new enough to use the four-level page tables:
it's not a big window), you can hit the bug. We fixed it pretty much
exactly a year go ("Sat Sep 17, 2005").
Other than that, does it tend to happen to the same particular program? It
might just be a user space bug that needs a specific set of things to
happen. Some of those bugs go away when you disable address space
randomization etc, since they can depend on just pure luck (or rather,
lack there-of).
If it happens to lots of different programs, and you sometimes see
SIGSEGV's and occasionally perhaps data corruption, that does tend to
indicate memory problems (or wild kernel pointers, but on the whole memory
problems tend to be more common).
Linus
Linus Torvalds wrote:
>
> On Fri, 22 Sep 2006, Orion Poplawski wrote:
>> We're seeing programs die with "bus error" (SIGBUS) randomly on a dual
>> processor Opteron machine. I've run memtest86+ and cpuburn stress tests with
>> no failure. gdb on a core file seems uninteresting. Is there some way to
>> trace the kernel to try to get more insight?
>
> Which kernel?
>
Sorry, 2.6.17-1.2142_FC4smp, so fairly recent.
> Other than that, does it tend to happen to the same particular program? It
> might just be a user space bug that needs a specific set of things to
> happen. Some of those bugs go away when you disable address space
> randomization etc, since they can depend on just pure luck (or rather,
> lack there-of).
So far just one program. Forgot about address space randomization, may
need to look into turning that off to help debugging.
Thanks!
--
Orion Poplawski
System Administrator 303-415-9701 x222
NWRA/CoRA Division FAX: 303-415-9702
3380 Mitchell Lane [email protected]
Boulder, CO 80301 http://www.cora.nwra.com