2004-09-09 20:15:06

by H. J. Lu

[permalink] [raw]
Subject: Hanging process on SMP machines?

I notice that a process may hang on SMP machines at random:

http://bugzilla.kernel.org/show_bug.cgi?id=3332

I can reliably trigger it within 15 minutes under SMP kernel on P4 HT
and 4-way ia64 machines. Has anyone else seen it?


H.J.


2004-09-10 07:48:46

by Andrew Morton

[permalink] [raw]
Subject: Re: Hanging process on SMP machines?

"H. J. Lu" <[email protected]> wrote:
>
> I notice that a process may hang on SMP machines at random:
>
> http://bugzilla.kernel.org/show_bug.cgi?id=3332
>
> I can reliably trigger it within 15 minutes under SMP kernel on P4 HT
> and 4-way ia64 machines. Has anyone else seen it?

It's easy to reproduce on 2-way x86, however it doesn't look like a kernel
bug.


akpm 2503 0.0 0.2 3892 640 pts/0 S 00:34 0:00 | \_ make
akpm 2504 0.0 0.0 1368 192 pts/0 S 00:34 0:00 | \_ time expect test.exp
akpm 2505 0.0 0.5 4776 1276 pts/0 S 00:34 0:00 | \_ expect test.exp
akpm 4726 0.0 0.0 0 0 ? Z 00:35 0:00 | \_ [true] <defunct>

process 4726 is sleeping at the end of do_exit():


schedule(); <<- here
BUG();
/* Avoid "noreturn function does return". */
for (;;) ;
}

So it has completely exitted and is waiting for someone to reap its exit
code and stack slot via wait4(). So what is its parent up to?


(gdb) thread 69
[Switching to thread 69 (Thread 2505)]#0 0xc01653e9 in do_select (n=5, fds=0xcc8b5fa4,
timeout=0xcc8b5fa0) at fs/select.c:257
257 __timeout = schedule_timeout(__timeout);
(gdb) bt
#0 0xc01653e9 in do_select (n=5, fds=0xcc8b5fa4, timeout=0xcc8b5fa0) at fs/select.c:257
#1 0xc016579c in sys_select (n=5, inp=0x804b444, outp=0x804b4c4, exp=0x804b544, tvp=0xbfffe1d0)
at fs/select.c:354
#2 0xc0105e39 in sysenter_past_esp () at arch/i386/kernel/semaphore.c:177

The parent is sleeping in select() rather than wait()ing for children.


(gdb) f 0
#0 0xc01653e9 in do_select (n=5, fds=0xcc8b5fa4, timeout=0xcc8b5fa0) at fs/select.c:257
257 __timeout = schedule_timeout(__timeout);
(gdb) p __timeout
$1 = 3000000

For 3,000 seconds.


I changed your 3000 to 30 and lo, the script hangs for 30 seconds every now
and then, and then resumes.

2004-09-10 16:46:42

by H. J. Lu

[permalink] [raw]
Subject: Re: Hanging process on SMP machines?

On Fri, Sep 10, 2004 at 12:45:49AM -0700, Andrew Morton wrote:
> "H. J. Lu" <[email protected]> wrote:
> >
> > I notice that a process may hang on SMP machines at random:
> >
> > http://bugzilla.kernel.org/show_bug.cgi?id=3332
> >
> > I can reliably trigger it within 15 minutes under SMP kernel on P4 HT
> > and 4-way ia64 machines. Has anyone else seen it?
>
> It's easy to reproduce on 2-way x86, however it doesn't look like a kernel
> bug.
>
>
> akpm 2503 0.0 0.2 3892 640 pts/0 S 00:34 0:00 | \_ make
> akpm 2504 0.0 0.0 1368 192 pts/0 S 00:34 0:00 | \_ time expect test.exp
> akpm 2505 0.0 0.5 4776 1276 pts/0 S 00:34 0:00 | \_ expect test.exp
> akpm 4726 0.0 0.0 0 0 ? Z 00:35 0:00 | \_ [true] <defunct>
>
> process 4726 is sleeping at the end of do_exit():
>
>
> schedule(); <<- here
> BUG();
> /* Avoid "noreturn function does return". */
> for (;;) ;
> }
>
> So it has completely exitted and is waiting for someone to reap its exit
> code and stack slot via wait4(). So what is its parent up to?
>
>
> (gdb) thread 69
> [Switching to thread 69 (Thread 2505)]#0 0xc01653e9 in do_select (n=5, fds=0xcc8b5fa4,
> timeout=0xcc8b5fa0) at fs/select.c:257
> 257 __timeout = schedule_timeout(__timeout);
> (gdb) bt
> #0 0xc01653e9 in do_select (n=5, fds=0xcc8b5fa4, timeout=0xcc8b5fa0) at fs/select.c:257
> #1 0xc016579c in sys_select (n=5, inp=0x804b444, outp=0x804b4c4, exp=0x804b544, tvp=0xbfffe1d0)
> at fs/select.c:354
> #2 0xc0105e39 in sysenter_past_esp () at arch/i386/kernel/semaphore.c:177
>
> The parent is sleeping in select() rather than wait()ing for children.
>

I don't think SIGCHLD is blocked. Shouldn't SIGCHLD interrupt select?
UP kernel doesn't have this problem.


H.J.