2006-12-18 16:36:40

by kyle

[permalink] [raw]
Subject: schedule_timeout: wrong timeout value

Hi,

Recently my mysql servershows something like:
Dec 18 18:24:05 sql kernel: schedule_timeout: wrong timeout value ffffffff
from c0284efd
Dec 18 18:24:36 sql last message repeated 19939 times
Dec 18 18:25:37 sql last message repeated 33392 times

from syslog every 1 or 2 days. Whenever the messages show, mysql server stop
accept new connections from the same network, and I need to restart the
mysql service and then it will keep running well for 1-2 days until the
messages show up again.

The server has been running over 1 year without any problem, the problem
started show up around 2 weeks ago. It's running kernel 2.6.12, and mysql
server, nothing else. Hardware is Pentium 4 2.8GHz with hyperthreading
enabled.

What does the kernel message mean and why it make mysql stop accept new
connections? Is it hardware problem or try upgrade the kernel may help?
Please CC me if possible. Thank you

Kyle




2006-12-19 02:35:29

by Robert Hancock

[permalink] [raw]
Subject: Re: schedule_timeout: wrong timeout value

kyle wrote:
> Hi,
>
> Recently my mysql servershows something like:
> Dec 18 18:24:05 sql kernel: schedule_timeout: wrong timeout value
> ffffffff from c0284efd
> Dec 18 18:24:36 sql last message repeated 19939 times
> Dec 18 18:25:37 sql last message repeated 33392 times
>
> from syslog every 1 or 2 days. Whenever the messages show, mysql server
> stop accept new connections from the same network, and I need to restart
> the mysql service and then it will keep running well for 1-2 days until
> the messages show up again.
>
> The server has been running over 1 year without any problem, the problem
> started show up around 2 weeks ago. It's running kernel 2.6.12, and
> mysql server, nothing else. Hardware is Pentium 4 2.8GHz with
> hyperthreading enabled.
>
> What does the kernel message mean and why it make mysql stop accept new
> connections? Is it hardware problem or try upgrade the kernel may help?
> Please CC me if possible. Thank you

The message means some code in the kernel or in some module passed a
negative value to schedule_timeout which it shouldn't have. The c0284efd
value is the address of the function that made the call - you may be
able to look that up in your /proc/ksyms or the System.map file and
figure out what function that is..

--
Robert Hancock Saskatoon, SK, Canada
To email, remove "nospam" from [email protected]
Home Page: http://www.roberthancock.com/

2006-12-20 01:59:54

by Andrew Morton

[permalink] [raw]
Subject: Re: schedule_timeout: wrong timeout value

On Mon, 18 Dec 2006 20:34:43 -0600
Robert Hancock <[email protected]> wrote:

> kyle wrote:
> > Hi,
> >
> > Recently my mysql servershows something like:
> > Dec 18 18:24:05 sql kernel: schedule_timeout: wrong timeout value
> > ffffffff from c0284efd
> > Dec 18 18:24:36 sql last message repeated 19939 times
> > Dec 18 18:25:37 sql last message repeated 33392 times
> >
> > from syslog every 1 or 2 days. Whenever the messages show, mysql server
> > stop accept new connections from the same network, and I need to restart
> > the mysql service and then it will keep running well for 1-2 days until
> > the messages show up again.
> >
> > The server has been running over 1 year without any problem, the problem
> > started show up around 2 weeks ago. It's running kernel 2.6.12, and
> > mysql server, nothing else. Hardware is Pentium 4 2.8GHz with
> > hyperthreading enabled.
> >
> > What does the kernel message mean and why it make mysql stop accept new
> > connections? Is it hardware problem or try upgrade the kernel may help?
> > Please CC me if possible. Thank you
>
> The message means some code in the kernel or in some module passed a
> negative value to schedule_timeout which it shouldn't have. The c0284efd
> value is the address of the function that made the call - you may be
> able to look that up in your /proc/ksyms or the System.map file and
> figure out what function that is..
>

I queued this:


From: Andrew Morton <[email protected]>

Kyle is hitting this warning, and we don't have a clue what it's caused by.
Add the obligatory dump_stack().

Cc: kyle <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
---

kernel/timer.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)

diff -puN kernel/timer.c~schedule_timeout-improve-warning-message kernel/timer.c
--- a/kernel/timer.c~schedule_timeout-improve-warning-message
+++ a/kernel/timer.c
@@ -1344,11 +1344,10 @@ fastcall signed long __sched schedule_ti
* should never happens anyway). You just have the printk()
* that will tell you if something is gone wrong and where.
*/
- if (timeout < 0)
- {
+ if (timeout < 0) {
printk(KERN_ERR "schedule_timeout: wrong timeout "
- "value %lx from %p\n", timeout,
- __builtin_return_address(0));
+ "value %lx\n", timeout);
+ dump_stack();
current->state = TASK_RUNNING;
goto out;
}
_

2006-12-20 03:29:50

by kyle

[permalink] [raw]
Subject: Re: schedule_timeout: wrong timeout value


----- Original Message -----
From: "Robert Hancock" <[email protected]>
To: "kyle" <[email protected]>
Cc: <[email protected]>
Sent: Tuesday, December 19, 2006 10:34 AM
Subject: Re: schedule_timeout: wrong timeout value


> kyle wrote:
>> Hi,
>>
>> Recently my mysql servershows something like:
>> Dec 18 18:24:05 sql kernel: schedule_timeout: wrong timeout value
>> ffffffff from c0284efd
>> Dec 18 18:24:36 sql last message repeated 19939 times
>> Dec 18 18:25:37 sql last message repeated 33392 times
>>

> The message means some code in the kernel or in some module passed a
> negative value to schedule_timeout which it shouldn't have. The c0284efd
> value is the address of the function that made the call - you may be able
> to look that up in your /proc/ksyms or the System.map file and figure out
> what function that is..

There was no module loaded, and unfortunlately, I cannot find the System.map
or /proc/ksyms file for the affected kernel!
Anyway thank you for your explanation. I have upgraded the kernel to
2.6.17.14 and wish that it can fix the problem. Thank you

Kyle