2009-03-24 16:28:21

by Mathieu Desnoyers

[permalink] [raw]
Subject: [patch 3/9] LTTng instrumentation tasklets

tasklet entry and exit events.

Signed-off-by: Mathieu Desnoyers <[email protected]>
CC: 'Ingo Molnar' <[email protected]>
CC: Frederic Weisbecker <[email protected]>
CC: Jason Baron <[email protected]>
CC: 'Peter Zijlstra' <[email protected]>
CC: Thomas Gleixner <[email protected]>
CC: Russell King <[email protected]>
CC: Masami Hiramatsu <[email protected]>
CC: "Frank Ch. Eigler" <[email protected]>
CC: 'Hideo AOKI' <[email protected]>
CC: Takashi Nishiie <[email protected]>
CC: 'Steven Rostedt' <[email protected]>
CC: Eduard - Gabriel Munteanu <[email protected]>
---
include/trace/irq.h | 13 +++++++++++++
kernel/softirq.c | 10 ++++++++++
2 files changed, 23 insertions(+)

Index: linux-2.6-lttng/kernel/softirq.c
===================================================================
--- linux-2.6-lttng.orig/kernel/softirq.c 2009-03-24 11:46:28.000000000 -0400
+++ linux-2.6-lttng/kernel/softirq.c 2009-03-24 11:48:36.000000000 -0400
@@ -23,6 +23,7 @@
#include <linux/rcupdate.h>
#include <linux/smp.h>
#include <linux/tick.h>
+#include <trace/irq.h>

#include <asm/irq.h>
/*
@@ -52,6 +53,11 @@ static struct softirq_action softirq_vec

static DEFINE_PER_CPU(struct task_struct *, ksoftirqd);

+DEFINE_TRACE(irq_tasklet_high_entry);
+DEFINE_TRACE(irq_tasklet_high_exit);
+DEFINE_TRACE(irq_tasklet_low_entry);
+DEFINE_TRACE(irq_tasklet_low_exit);
+
/*
* we cannot loop indefinitely here to avoid userspace starvation,
* but we also don't want to introduce a worst case 1/HZ latency
@@ -380,7 +386,9 @@ static void tasklet_action(struct softir
if (!atomic_read(&t->count)) {
if (!test_and_clear_bit(TASKLET_STATE_SCHED, &t->state))
BUG();
+ trace_irq_tasklet_low_entry(t);
t->func(t->data);
+ trace_irq_tasklet_low_exit(t);
tasklet_unlock(t);
continue;
}
@@ -415,7 +423,9 @@ static void tasklet_hi_action(struct sof
if (!atomic_read(&t->count)) {
if (!test_and_clear_bit(TASKLET_STATE_SCHED, &t->state))
BUG();
+ trace_irq_tasklet_high_entry(t);
t->func(t->data);
+ trace_irq_tasklet_high_exit(t);
tasklet_unlock(t);
continue;
}
Index: linux-2.6-lttng/include/trace/irq.h
===================================================================
--- linux-2.6-lttng.orig/include/trace/irq.h 2009-03-24 11:46:29.000000000 -0400
+++ linux-2.6-lttng/include/trace/irq.h 2009-03-24 11:48:36.000000000 -0400
@@ -19,4 +19,17 @@ DECLARE_TRACE(irq_next_handler,
TPPROTO(unsigned int id, struct irqaction *action, irqreturn_t ret),
TPARGS(id, action, ret));

+DECLARE_TRACE(irq_tasklet_low_entry,
+ TPPROTO(struct tasklet_struct *t),
+ TPARGS(t));
+DECLARE_TRACE(irq_tasklet_low_exit,
+ TPPROTO(struct tasklet_struct *t),
+ TPARGS(t));
+DECLARE_TRACE(irq_tasklet_high_entry,
+ TPPROTO(struct tasklet_struct *t),
+ TPARGS(t));
+DECLARE_TRACE(irq_tasklet_high_exit,
+ TPPROTO(struct tasklet_struct *t),
+ TPARGS(t));
+
#endif

--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68


2009-03-24 17:58:38

by Ingo Molnar

[permalink] [raw]
Subject: Re: [patch 3/9] LTTng instrumentation tasklets


* Mathieu Desnoyers <[email protected]> wrote:

> tasklet entry and exit events.

> +DEFINE_TRACE(irq_tasklet_high_entry);
> +DEFINE_TRACE(irq_tasklet_high_exit);
> +DEFINE_TRACE(irq_tasklet_low_entry);
> +DEFINE_TRACE(irq_tasklet_low_exit);

Dunno - tasklets are a legacy mechanism, not sure we want to
instrument them. That some tasklet activity happens will be obvious
from the softirq tracepoints already.

Anyway ... i guess we could do it if it's done via the more complete
TRACE_EVENT() mechanism in the tracing tree.

Ingo

2009-03-25 13:52:52

by Chetan.Loke

[permalink] [raw]
Subject: RE: [patch 3/9] LTTng instrumentation tasklets



> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Ingo Molnar
> Sent: Tuesday, March 24, 2009 1:56 PM
> To: Mathieu Desnoyers
> Cc: [email protected]; [email protected];
> [email protected]; Frederic Weisbecker; Jason
> Baron; Peter Zijlstra; Thomas Gleixner; Russell King; Masami
> Hiramatsu; Frank Ch. Eigler; Hideo AOKI; Takashi Nishiie;
> Steven Rostedt; Eduard - Gabriel Munteanu
> Subject: Re: [patch 3/9] LTTng instrumentation tasklets
>
>
> * Mathieu Desnoyers <[email protected]> wrote:
>
> > tasklet entry and exit events.
>
> > +DEFINE_TRACE(irq_tasklet_high_entry);
> > +DEFINE_TRACE(irq_tasklet_high_exit);
> > +DEFINE_TRACE(irq_tasklet_low_entry);
> > +DEFINE_TRACE(irq_tasklet_low_exit);
>
> Dunno - tasklets are a legacy mechanism, not sure we want to
> instrument them.


Quick question. I understand this is unrelated to this patch. So I apologize in advance.
Ingo - you mentioned "tasklets are a legacy mechanism". Is there a plan to phase them out ? Let me draw a small picture as to what's bothering me.

With the SR-IOV support if there are 'N' virtual functions then there will be 'N' driver instances(actually N+1, 1 for the PF). If that driver drains the responses in the interrupt context then all such VF-instances could virtually block everyone else(because SR-IOV guys might also have MSI-X enabled).
So now all such drivers should alter their Rx path.Driver's can queue tasklets and can also get the performance they want.

Any suggestions?

thanks
Chetan-

2009-03-25 14:22:31

by Peter Zijlstra

[permalink] [raw]
Subject: RE: [patch 3/9] LTTng instrumentation tasklets

On Wed, 2009-03-25 at 06:52 -0700, [email protected] wrote:
> Quick question. I understand this is unrelated to this patch. So I
> apologize in advance.
> Ingo - you mentioned "tasklets are a legacy mechanism". Is there a
> plan to phase them out ? Let me draw a small picture as to what's
> bothering me.
>
> With the SR-IOV support if there are 'N' virtual functions then there
> will be 'N' driver instances(actually N+1, 1 for the PF). If that
> driver drains the responses in the interrupt context then all such
> VF-instances could virtually block everyone else(because SR-IOV guys
> might also have MSI-X enabled).
> So now all such drivers should alter their Rx path.Driver's can queue
> tasklets and can also get the performance they want.
>
> Any suggestions?

Threaded interrupts?

2009-03-25 17:37:53

by Chetan.Loke

[permalink] [raw]
Subject: RE: [patch 3/9] LTTng instrumentation tasklets



> -----Original Message-----
> From: Peter Zijlstra [mailto:[email protected]]
> Sent: Wednesday, March 25, 2009 10:18 AM
> To: Loke,Chetan
> Cc: [email protected]; [email protected];
> [email protected]; [email protected];
> [email protected]; [email protected];
> [email protected]; [email protected];
> [email protected]; [email protected];
> [email protected]; [email protected];
> [email protected]; [email protected];
> [email protected]
> Subject: RE: [patch 3/9] LTTng instrumentation tasklets
>
> On Wed, 2009-03-25 at 06:52 -0700, [email protected] wrote:
> > Quick question. I understand this is unrelated to this patch. So I
> > apologize in advance.
> > Ingo - you mentioned "tasklets are a legacy mechanism". Is there a
> > plan to phase them out ? Let me draw a small picture as to what's
> > bothering me.
> >
> > With the SR-IOV support if there are 'N' virtual functions
> then there
> > will be 'N' driver instances(actually N+1, 1 for the PF). If that
> > driver drains the responses in the interrupt context then all such
> > VF-instances could virtually block everyone else(because
> SR-IOV guys
> > might also have MSI-X enabled).
> > So now all such drivers should alter their Rx path.Driver's
> can queue
> > tasklets and can also get the performance they want.
> >
> > Any suggestions?
>
> Threaded interrupts?
>

If we truly need to address performance and scalability for the SCSI-subsystem then we need something lightweight.-

2009-03-25 17:52:46

by Steven Rostedt

[permalink] [raw]
Subject: RE: [patch 3/9] LTTng instrumentation tasklets


On Wed, 25 Mar 2009, [email protected] wrote:

>
>
> > -----Original Message-----
> > From: Peter Zijlstra [mailto:[email protected]]
> > Sent: Wednesday, March 25, 2009 10:18 AM
> > To: Loke,Chetan
> > Cc: [email protected]; [email protected];
> > [email protected]; [email protected];
> > [email protected]; [email protected];
> > [email protected]; [email protected];
> > [email protected]; [email protected];
> > [email protected]; [email protected];
> > [email protected]; [email protected];
> > [email protected]
> > Subject: RE: [patch 3/9] LTTng instrumentation tasklets
> >
> > On Wed, 2009-03-25 at 06:52 -0700, [email protected] wrote:
> > > Quick question. I understand this is unrelated to this patch. So I
> > > apologize in advance.
> > > Ingo - you mentioned "tasklets are a legacy mechanism". Is there a
> > > plan to phase them out ? Let me draw a small picture as to what's
> > > bothering me.
> > >
> > > With the SR-IOV support if there are 'N' virtual functions
> > then there
> > > will be 'N' driver instances(actually N+1, 1 for the PF). If that
> > > driver drains the responses in the interrupt context then all such
> > > VF-instances could virtually block everyone else(because
> > SR-IOV guys
> > > might also have MSI-X enabled).
> > > So now all such drivers should alter their Rx path.Driver's
> > can queue
> > > tasklets and can also get the performance they want.
> > >
> > > Any suggestions?
> >
> > Threaded interrupts?
> >
>
> If we truly need to address performance and scalability for the SCSI-subsystem then we need something lightweight.

Threaded interrupts are actually quite light.

-- Steve