2008-10-01 14:18:35

by Gregory Haskins

Subject: [PATCH] sched: add a stacktrace on enqueue_pushable error

Hi Chirag,
Please apply this patch to your 26.5-rt9 tree for ppc64, enable
CONFIG_PROVE_LOCKING (which enables CONFIG_STACKTRACE) and give it a whirl.
If you get an oops, please post the console output.

Thanks!

-Greg

------------------
sched: add a stacktrace on enqueue_pushable error

NOT FOR INCLUSION!

This is to help debug an issue discovered by Chirag Jog in the thread

http://lkml.org/lkml/2008/9/25/189

Signed-off-by: Gregory Haskins <[email protected]>
---

include/linux/sched.h | 6 ++++++
kernel/sched_rt.c | 20 +++++++++++++++++++-
2 files changed, 25 insertions(+), 1 deletions(-)

diff --git a/include/linux/sched.h b/include/linux/sched.h
index 67da014..53e8a6a 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -87,6 +87,7 @@ struct sched_param {
#include <linux/task_io_accounting.h>
#include <linux/kobject.h>
#include <linux/latencytop.h>
+#include <linux/stacktrace.h>

#include <asm/processor.h>

@@ -1170,6 +1171,11 @@ struct task_struct {

struct list_head tasks;
struct plist_node pushable_tasks;
+ struct {
+ unsigned long data[15];
+ struct stack_trace trace;
+ pid_t pid;
+ } pushable_stack;

/*
* ptrace_list/ptrace_children forms the list of my children
diff --git a/kernel/sched_rt.c b/kernel/sched_rt.c
index 57a0c0d..0e6a88c 100644
--- a/kernel/sched_rt.c
+++ b/kernel/sched_rt.c
@@ -55,7 +55,25 @@ static void update_rt_migration(struct rq *rq)

static void enqueue_pushable_task(struct rq *rq, struct task_struct *p)
{
- plist_del(&p->pushable_tasks, &rq->rt.pushable_tasks);
+ if (plist_node_empty(&p->pushable_tasks)) {
+ struct stack_trace *trace = &p->pushable_stack.trace;
+
+ trace->nr_entries = 0;
+ trace->max_entries = sizeof(p->pushable_stack.data)/sizeof(p->pushable_stack.data[0]);
+ trace->entries = &p->pushable_stack.data[0];
+ trace->skip = 0;
+
+ save_stack_trace(trace);
+
+ p->pushable_stack.pid = current->pid;
+ } else {
+ printk(KERN_CRIT "redundant enqueue by %d detected\n",
+ p->pushable_stack.pid);
+ print_stack_trace(&p->pushable_stack.trace, 5);
+
+ BUG_ON(!plist_node_empty(&p->pushable_tasks));
+ }
+
plist_node_init(&p->pushable_tasks, p->prio);
plist_add(&p->pushable_tasks, &rq->rt.pushable_tasks);
}

2008-10-02 09:42:31

by Gilles Carry

Subject: Re: [PATCH] sched: add a stacktrace on enqueue_pushable error

Hi,

I could reproduce the bug on an intel architecture.
I found where the problem is.
I'll post the patch in a few hours.

Gilles.

Gregory Haskins wrote:
> Hi Chirag,
> Please apply this patch to your 26.5-rt9 tree for ppc64, enable
> CONFIG_PROVE_LOCKING (which enables CONFIG_STACKTRACE) and give it a whirl.
> If you get an oops, please post the console output.
>
> Thanks!
>
> -Greg
>
> ------------------
> sched: add a stacktrace on enqueue_pushable error
>
> NOT FOR INCLUSION!
>
> This is to help debug an issue discovered by Chirag Jog in the thread
>
> http://lkml.org/lkml/2008/9/25/189
>
> Signed-off-by: Gregory Haskins <[email protected]>
> ---
>
> include/linux/sched.h | 6 ++++++
> kernel/sched_rt.c | 20 +++++++++++++++++++-
> 2 files changed, 25 insertions(+), 1 deletions(-)
>
> diff --git a/include/linux/sched.h b/include/linux/sched.h
> index 67da014..53e8a6a 100644
> --- a/include/linux/sched.h
> +++ b/include/linux/sched.h
> @@ -87,6 +87,7 @@ struct sched_param {
> #include <linux/task_io_accounting.h>
> #include <linux/kobject.h>
> #include <linux/latencytop.h>
> +#include <linux/stacktrace.h>
>
> #include <asm/processor.h>
>
> @@ -1170,6 +1171,11 @@ struct task_struct {
>
> struct list_head tasks;
> struct plist_node pushable_tasks;
> + struct {
> + unsigned long data[15];
> + struct stack_trace trace;
> + pid_t pid;
> + } pushable_stack;
>
> /*
> * ptrace_list/ptrace_children forms the list of my children
> diff --git a/kernel/sched_rt.c b/kernel/sched_rt.c
> index 57a0c0d..0e6a88c 100644
> --- a/kernel/sched_rt.c
> +++ b/kernel/sched_rt.c
> @@ -55,7 +55,25 @@ static void update_rt_migration(struct rq *rq)
>
> static void enqueue_pushable_task(struct rq *rq, struct task_struct *p)
> {
> - plist_del(&p->pushable_tasks, &rq->rt.pushable_tasks);
> + if (plist_node_empty(&p->pushable_tasks)) {
> + struct stack_trace *trace = &p->pushable_stack.trace;
> +
> + trace->nr_entries = 0;
> + trace->max_entries = sizeof(p->pushable_stack.data)/sizeof(p->pushable_stack.data[0]);
> + trace->entries = &p->pushable_stack.data[0];
> + trace->skip = 0;
> +
> + save_stack_trace(trace);
> +
> + p->pushable_stack.pid = current->pid;
> + } else {
> + printk(KERN_CRIT "redundant enqueue by %d detected\n",
> + p->pushable_stack.pid);
> + print_stack_trace(&p->pushable_stack.trace, 5);
> +
> + BUG_ON(!plist_node_empty(&p->pushable_tasks));
> + }
> +
> plist_node_init(&p->pushable_tasks, p->prio);
> plist_add(&p->pushable_tasks, &rq->rt.pushable_tasks);
> }
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Gilles.Carry
Linux Project team
mailto: [email protected]
Phone: +33 (0)4 76 29 74 27
Addr.: BULL S.A. 1 rue de Provence, B.P. 208 38432 Echirolles Cedex
http://www.bull.com
http://www.bullopensource.org/
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~