2008-12-02 20:02:01

by Mathieu Desnoyers

[permalink] [raw]
Subject: Re: [ltt-dev] [PATCH] add tracepoints to track change in napi states on network receive

* Neil Horman ([email protected]) wrote:
> Hey there-
> I thought it would be handy to track the napi receive state of different
> network devices using ltt. This patch adds trace points to indicate when a napi
> instance has been scheduled, when its serviced and when it completes.
>

Sounds great, see comments below,

> Regards
> Neil
>
> Signed-off-by: Neil Horman <[email protected]>
>
>
> include/linux/netdevice.h | 4 ++++
> include/trace/netdevice.h | 36 ++++++++++++++++++++++++++++++++++++
> ltt/probes/net-trace.c | 22 ++++++++++++++++++++++
> net/core/dev.c | 6 +++++-
> 4 files changed, 67 insertions(+), 1 deletion(-)
>
>
> diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> index 488c56e..9f35f84 100644
> --- a/include/linux/netdevice.h
> +++ b/include/linux/netdevice.h
> @@ -43,6 +43,8 @@
>
> #include <net/net_namespace.h>
>
> +#include <trace/netdevice.h>
> +
> struct vlan_group;
> struct ethtool_ops;
> struct netpoll_info;
> @@ -386,6 +388,7 @@ static inline void napi_complete(struct napi_struct *n)
> local_irq_save(flags);
> __napi_complete(n);
> local_irq_restore(flags);
> + trace_napi_complete(n);
> }
>
> /**
> @@ -1726,6 +1729,7 @@ static inline int skb_bond_should_drop(struct sk_buff *skb)
> return 0;
> }
>
> +
> #endif /* __KERNEL__ */
>
> #endif /* _LINUX_DEV_H */
> diff --git a/include/trace/netdevice.h b/include/trace/netdevice.h
> new file mode 100644
> index 0000000..c75030b
> --- /dev/null
> +++ b/include/trace/netdevice.h
> @@ -0,0 +1,36 @@
> +#ifndef _INCLUDE_NETDEVICE_H_
> +#define _INCLUDE_NETDEVICE_H_
> +
> +#include <linux/timer.h>
> +#include <linux/delay.h>
> +#include <asm/atomic.h>
> +#include <asm/cache.h>
> +#include <asm/byteorder.h>
> +
> +#include <linux/device.h>
> +#include <linux/percpu.h>
> +#include <linux/dmaengine.h>
> +#include <linux/workqueue.h>
> +
> +#include <net/net_namespace.h>
> +

Hrm ? Do we need all these includes ?

> +/* Tracepoints */
> +
> +/*
> + * Note this is actually the trace for __netif_rx_schedule
> + * since I wanted to trace every call which had us set the RX_SCHED bit

Could you put this comment in the 3rd person ? Also, I wonder if that
mean there might be other cause for schedule we would not trace ?


> + */
> +DEFINE_TRACE(napi_schedule,
> + TPPROTO(struct napi_struct *n),
> + TPARGS(n));
> +
> +DEFINE_TRACE(napi_poll,
> + TPPROTO(struct napi_struct *n),
> + TPARGS(n));
> +
> +DEFINE_TRACE(napi_complete,
> + TPPROTO(struct napi_struct *n),
> + TPARGS(n));
> +
> +#endif
> +
> diff --git a/ltt/probes/net-trace.c b/ltt/probes/net-trace.c
> index bac2b21..3de1ff8 100644
> --- a/ltt/probes/net-trace.c
> +++ b/ltt/probes/net-trace.c
> @@ -9,6 +9,7 @@
> #include <trace/ipv4.h>
> #include <trace/ipv6.h>
> #include <trace/socket.h>
> +#include <trace/netdevice.h>
>
> void probe_net_dev_xmit(struct sk_buff *skb)
> {
> @@ -104,6 +105,27 @@ void probe_socket_call(int call, unsigned long a0)
> "call %d a0 %lu", call, a0);
> }
>
> +void probe_napi_schedule(struct napi_struct *n)
> +{
> + trace_mark_tp(net_napi_schedule, napi_schedule, probe_napi_schedule,
> + "schedule napi instance %p, dev %s",


Field names should have no space, e.g. :

"napi_struct %p name %s"

Mathieu

> + n, n->dev->name);
> +}
> +
> +void probe_napi_poll(struct napi_struct *n)
> +{
> + trace_mark_tp(net_napi_poll, napi_poll, probe_napi_poll,
> + "service napi instance %p, dev %s",
> + n, n->dev->name);
> +}
> +
> +void probe_napi_complete(struct napi_struct *n)
> +{
> + trace_mark_tp(net_napi_complete, napi_complete, probe_napi_complete,
> + "complete napi instance %p, dev %s",
> + n, n->dev->name);
> +}
> +
> MODULE_LICENSE("GPL");
> MODULE_AUTHOR("Mathieu Desnoyers");
> MODULE_DESCRIPTION("Net Tracepoint Probes");
> diff --git a/net/core/dev.c b/net/core/dev.c
> index 836fe6e..6021010 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -2329,6 +2329,8 @@ void __napi_schedule(struct napi_struct *n)
> {
> unsigned long flags;
>
> + trace_napi_poll(n);
> +
> local_irq_save(flags);
> list_add_tail(&n->poll_list, &__get_cpu_var(softnet_data).poll_list);
> __raise_softirq_irqoff(NET_RX_SOFTIRQ);
> @@ -2380,8 +2382,10 @@ static void net_rx_action(struct softirq_action *h)
> * accidently calling ->poll() when NAPI is not scheduled.
> */
> work = 0;
> - if (test_bit(NAPI_STATE_SCHED, &n->state))
> + if (test_bit(NAPI_STATE_SCHED, &n->state)) {
> + trace_napi_poll(n);
> work = n->poll(n, weight);
> + }
>
> WARN_ON_ONCE(work > weight);
>
> --
> /****************************************************
> * Neil Horman <[email protected]>
> * Software Engineer, Red Hat
> ****************************************************/
>
> _______________________________________________
> ltt-dev mailing list
> [email protected]
> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>

--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68


2008-12-02 20:38:36

by Neil Horman

[permalink] [raw]
Subject: Re: [ltt-dev] [PATCH] add tracepoints to track change in napi states on network receive

On Tue, Dec 02, 2008 at 03:01:50PM -0500, Mathieu Desnoyers wrote:
> * Neil Horman ([email protected]) wrote:
> > Hey there-
> > I thought it would be handy to track the napi receive state of different
> > network devices using ltt. This patch adds trace points to indicate when a napi
> > instance has been scheduled, when its serviced and when it completes.
> >
>
> Sounds great, see comments below,
>
That all looks good. I'll revise and repost in the morning.

Thanks & Regards
Neil



--
/****************************************************
* Neil Horman <[email protected]>
* Software Engineer, Red Hat
****************************************************/

2008-12-03 16:04:07

by Neil Horman

[permalink] [raw]
Subject: Re: [ltt-dev] [PATCH] add tracepoints to track change in napi states on network receive

On Tue, Dec 02, 2008 at 03:01:50PM -0500, Mathieu Desnoyers wrote:
> * Neil Horman ([email protected]) wrote:
> > Hey there-
> > I thought it would be handy to track the napi receive state of different
> > network devices using ltt. This patch adds trace points to indicate when a napi
> > instance has been scheduled, when its serviced and when it completes.
> >
>
> Sounds great, see comments below,
<snip>

Hey,
New patch, with previous comments taken into account.

Patch to track the napi receive stat of different network devices using ltt.
Adds trace points to indicate when a napi instance has been scheduled, when its
serviced and when it completes

Regards
Neil

Signed-off-by: Neil Horman <[email protected]>


include/linux/netdevice.h | 5 +++++
include/trace/netdevice.h | 30 ++++++++++++++++++++++++++++++
ltt/probes/net-trace.c | 22 ++++++++++++++++++++++
net/core/dev.c | 7 ++++++-
4 files changed, 63 insertions(+), 1 deletion(-)


diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 488c56e..7da71eb 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -43,6 +43,9 @@

#include <net/net_namespace.h>

+struct napi_struct;
+#include <trace/netdevice.h>
+
struct vlan_group;
struct ethtool_ops;
struct netpoll_info;
@@ -386,6 +389,7 @@ static inline void napi_complete(struct napi_struct *n)
local_irq_save(flags);
__napi_complete(n);
local_irq_restore(flags);
+ trace_napi_complete(n);
}

/**
@@ -1726,6 +1730,7 @@ static inline int skb_bond_should_drop(struct sk_buff *skb)
return 0;
}

+
#endif /* __KERNEL__ */

#endif /* _LINUX_DEV_H */
diff --git a/include/trace/netdevice.h b/include/trace/netdevice.h
new file mode 100644
index 0000000..ba48f6c
--- /dev/null
+++ b/include/trace/netdevice.h
@@ -0,0 +1,30 @@
+#ifndef _INCLUDE_NETDEVICE_H_
+#define _INCLUDE_NETDEVICE_H_
+
+#include <linux/netdevice.h>
+
+/* Tracepoints */
+
+/*
+ * Note these first 2 traces are actually in __napi_schedule and net_rx_action
+ * respectively. The former is in __napi_schedule because it uses at-most-once
+ * logic and placing it in the calling routine (napi_schedule) would produce
+ * countless trace events that were effectively no-ops. napi_poll is
+ * implemented in net_rx_action, because thats where we do our polling on
+ * devices. The last trace point is in napi_complete, right where you would
+ * think it would be.
+ */
+DEFINE_TRACE(napi_schedule,
+ TPPROTO(struct napi_struct *n),
+ TPARGS(n));
+
+DEFINE_TRACE(napi_poll,
+ TPPROTO(struct napi_struct *n),
+ TPARGS(n));
+
+DEFINE_TRACE(napi_complete,
+ TPPROTO(struct napi_struct *n),
+ TPARGS(n));
+
+#endif
+
diff --git a/ltt/probes/net-trace.c b/ltt/probes/net-trace.c
index bac2b21..bd4e621 100644
--- a/ltt/probes/net-trace.c
+++ b/ltt/probes/net-trace.c
@@ -9,6 +9,7 @@
#include <trace/ipv4.h>
#include <trace/ipv6.h>
#include <trace/socket.h>
+#include <trace/netdevice.h>

void probe_net_dev_xmit(struct sk_buff *skb)
{
@@ -104,6 +105,27 @@ void probe_socket_call(int call, unsigned long a0)
"call %d a0 %lu", call, a0);
}

+void probe_napi_schedule(struct napi_struct *n)
+{
+ trace_mark_tp(net_napi_schedule, napi_schedule, probe_napi_schedule,
+ "schedule napi_struct %p name %s",
+ n, n->dev->name);
+}
+
+void probe_napi_poll(struct napi_struct *n)
+{
+ trace_mark_tp(net_napi_poll, napi_poll, probe_napi_poll,
+ "service napi_struct %p name %s",
+ n, n->dev->name);
+}
+
+void probe_napi_complete(struct napi_struct *n)
+{
+ trace_mark_tp(net_napi_complete, napi_complete, probe_napi_complete,
+ "complete napi_struct %p name %s",
+ n, n->dev->name);
+}
+
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Mathieu Desnoyers");
MODULE_DESCRIPTION("Net Tracepoint Probes");
diff --git a/net/core/dev.c b/net/core/dev.c
index 836fe6e..0c55c0f 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -128,6 +128,7 @@
#include <linux/jhash.h>
#include <linux/random.h>
#include <trace/net.h>
+#include <trace/netdevice.h>

#include "net-sysfs.h"

@@ -2329,6 +2330,8 @@ void __napi_schedule(struct napi_struct *n)
{
unsigned long flags;

+ trace_napi_poll(n);
+
local_irq_save(flags);
list_add_tail(&n->poll_list, &__get_cpu_var(softnet_data).poll_list);
__raise_softirq_irqoff(NET_RX_SOFTIRQ);
@@ -2380,8 +2383,10 @@ static void net_rx_action(struct softirq_action *h)
* accidently calling ->poll() when NAPI is not scheduled.
*/
work = 0;
- if (test_bit(NAPI_STATE_SCHED, &n->state))
+ if (test_bit(NAPI_STATE_SCHED, &n->state)) {
+ trace_napi_poll(n);
work = n->poll(n, weight);
+ }

WARN_ON_ONCE(work > weight);

--
/****************************************************
* Neil Horman <[email protected]>
* Software Engineer, Red Hat
****************************************************/

2008-12-03 17:25:25

by Mathieu Desnoyers

[permalink] [raw]
Subject: Re: [ltt-dev] [PATCH] add tracepoints to track change in napi states on network receive

* Neil Horman ([email protected]) wrote:
> On Tue, Dec 02, 2008 at 03:01:50PM -0500, Mathieu Desnoyers wrote:
> > * Neil Horman ([email protected]) wrote:
> > > Hey there-
> > > I thought it would be handy to track the napi receive state of different
> > > network devices using ltt. This patch adds trace points to indicate when a napi
> > > instance has been scheduled, when its serviced and when it completes.
> > >
> >
> > Sounds great, see comments below,
> <snip>
>
> Hey,
> New patch, with previous comments taken into account.
>
> Patch to track the napi receive stat of different network devices using ltt.
> Adds trace points to indicate when a napi instance has been scheduled, when its
> serviced and when it completes
>
> Regards
> Neil
>
> Signed-off-by: Neil Horman <[email protected]>
>
>
> include/linux/netdevice.h | 5 +++++
> include/trace/netdevice.h | 30 ++++++++++++++++++++++++++++++
> ltt/probes/net-trace.c | 22 ++++++++++++++++++++++
> net/core/dev.c | 7 ++++++-
> 4 files changed, 63 insertions(+), 1 deletion(-)
>
>
> diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> index 488c56e..7da71eb 100644
> --- a/include/linux/netdevice.h
> +++ b/include/linux/netdevice.h
> @@ -43,6 +43,9 @@
>
> #include <net/net_namespace.h>
>
> +struct napi_struct;
> +#include <trace/netdevice.h>
> +
> struct vlan_group;
> struct ethtool_ops;
> struct netpoll_info;
> @@ -386,6 +389,7 @@ static inline void napi_complete(struct napi_struct *n)
> local_irq_save(flags);
> __napi_complete(n);
> local_irq_restore(flags);
> + trace_napi_complete(n);
> }
>
> /**
> @@ -1726,6 +1730,7 @@ static inline int skb_bond_should_drop(struct sk_buff *skb)
> return 0;
> }
>
> +
> #endif /* __KERNEL__ */
>
> #endif /* _LINUX_DEV_H */
> diff --git a/include/trace/netdevice.h b/include/trace/netdevice.h
> new file mode 100644
> index 0000000..ba48f6c
> --- /dev/null
> +++ b/include/trace/netdevice.h
> @@ -0,0 +1,30 @@
> +#ifndef _INCLUDE_NETDEVICE_H_
> +#define _INCLUDE_NETDEVICE_H_
> +
> +#include <linux/netdevice.h>
> +
> +/* Tracepoints */
> +
> +/*
> + * Note these first 2 traces are actually in __napi_schedule and net_rx_action
> + * respectively. The former is in __napi_schedule because it uses at-most-once
> + * logic and placing it in the calling routine (napi_schedule) would produce
> + * countless trace events that were effectively no-ops. napi_poll is
> + * implemented in net_rx_action, because thats where we do our polling on
> + * devices. The last trace point is in napi_complete, right where you would
> + * think it would be.
> + */
> +DEFINE_TRACE(napi_schedule,
> + TPPROTO(struct napi_struct *n),
> + TPARGS(n));
> +
> +DEFINE_TRACE(napi_poll,
> + TPPROTO(struct napi_struct *n),
> + TPARGS(n));
> +
> +DEFINE_TRACE(napi_complete,
> + TPPROTO(struct napi_struct *n),
> + TPARGS(n));
> +
> +#endif
> +
> diff --git a/ltt/probes/net-trace.c b/ltt/probes/net-trace.c
> index bac2b21..bd4e621 100644
> --- a/ltt/probes/net-trace.c
> +++ b/ltt/probes/net-trace.c
> @@ -9,6 +9,7 @@
> #include <trace/ipv4.h>
> #include <trace/ipv6.h>
> #include <trace/socket.h>
> +#include <trace/netdevice.h>
>
> void probe_net_dev_xmit(struct sk_buff *skb)
> {
> @@ -104,6 +105,27 @@ void probe_socket_call(int call, unsigned long a0)
> "call %d a0 %lu", call, a0);
> }
>
> +void probe_napi_schedule(struct napi_struct *n)
> +{
> + trace_mark_tp(net_napi_schedule, napi_schedule, probe_napi_schedule,
> + "schedule napi_struct %p name %s",

Almost there :)

There is a extra whitespace above,

and also "schedule napi_struct" should probably be changed into
napi_struct only, this removes the whitespace and duplicated information
(I guess we know that given we are in event net_napi_schedule, this will
have something to do with schedule...). Same applies to trace_mark_tp
below.

Thanks,

Mathieu

> + n, n->dev->name);
> +}
> +
> +void probe_napi_poll(struct napi_struct *n)
> +{
> + trace_mark_tp(net_napi_poll, napi_poll, probe_napi_poll,
> + "service napi_struct %p name %s",
> + n, n->dev->name);
> +}
> +
> +void probe_napi_complete(struct napi_struct *n)
> +{
> + trace_mark_tp(net_napi_complete, napi_complete, probe_napi_complete,
> + "complete napi_struct %p name %s",
> + n, n->dev->name);
> +}
> +
> MODULE_LICENSE("GPL");
> MODULE_AUTHOR("Mathieu Desnoyers");
> MODULE_DESCRIPTION("Net Tracepoint Probes");
> diff --git a/net/core/dev.c b/net/core/dev.c
> index 836fe6e..0c55c0f 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -128,6 +128,7 @@
> #include <linux/jhash.h>
> #include <linux/random.h>
> #include <trace/net.h>
> +#include <trace/netdevice.h>
>
> #include "net-sysfs.h"
>
> @@ -2329,6 +2330,8 @@ void __napi_schedule(struct napi_struct *n)
> {
> unsigned long flags;
>
> + trace_napi_poll(n);
> +
> local_irq_save(flags);
> list_add_tail(&n->poll_list, &__get_cpu_var(softnet_data).poll_list);
> __raise_softirq_irqoff(NET_RX_SOFTIRQ);
> @@ -2380,8 +2383,10 @@ static void net_rx_action(struct softirq_action *h)
> * accidently calling ->poll() when NAPI is not scheduled.
> */
> work = 0;
> - if (test_bit(NAPI_STATE_SCHED, &n->state))
> + if (test_bit(NAPI_STATE_SCHED, &n->state)) {
> + trace_napi_poll(n);
> work = n->poll(n, weight);
> + }
>
> WARN_ON_ONCE(work > weight);
>
> --
> /****************************************************
> * Neil Horman <[email protected]>
> * Software Engineer, Red Hat
> ****************************************************/
>

--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68

2008-12-03 17:47:23

by Neil Horman

[permalink] [raw]
Subject: Re: [ltt-dev] [PATCH] add tracepoints to track change in napi states on network receive

On Wed, Dec 03, 2008 at 12:25:07PM -0500, Mathieu Desnoyers wrote:
> Almost there :)
>
> There is a extra whitespace above,
>
> and also "schedule napi_struct" should probably be changed into
> napi_struct only, this removes the whitespace and duplicated information
> (I guess we know that given we are in event net_napi_schedule, this will
> have something to do with schedule...). Same applies to trace_mark_tp
> below.
>
> Thanks,
>
> Mathieu


Sure, here you go, new patch

Patch to trace napi states for various net devices and their napi instances.
Traces scheduling, servicing and completion of a napi poll request.

Regards
Neil

Signed-off-by: Neil Horman <[email protected]>


include/linux/netdevice.h | 5 +++++
include/trace/netdevice.h | 30 ++++++++++++++++++++++++++++++
ltt/probes/net-trace.c | 22 ++++++++++++++++++++++
net/core/dev.c | 7 ++++++-
4 files changed, 63 insertions(+), 1 deletion(-)


diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 488c56e..7da71eb 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -43,6 +43,9 @@

#include <net/net_namespace.h>

+struct napi_struct;
+#include <trace/netdevice.h>
+
struct vlan_group;
struct ethtool_ops;
struct netpoll_info;
@@ -386,6 +389,7 @@ static inline void napi_complete(struct napi_struct *n)
local_irq_save(flags);
__napi_complete(n);
local_irq_restore(flags);
+ trace_napi_complete(n);
}

/**
@@ -1726,6 +1730,7 @@ static inline int skb_bond_should_drop(struct sk_buff *skb)
return 0;
}

+
#endif /* __KERNEL__ */

#endif /* _LINUX_DEV_H */
diff --git a/include/trace/netdevice.h b/include/trace/netdevice.h
new file mode 100644
index 0000000..ba48f6c
--- /dev/null
+++ b/include/trace/netdevice.h
@@ -0,0 +1,30 @@
+#ifndef _INCLUDE_NETDEVICE_H_
+#define _INCLUDE_NETDEVICE_H_
+
+#include <linux/netdevice.h>
+
+/* Tracepoints */
+
+/*
+ * Note these first 2 traces are actually in __napi_schedule and net_rx_action
+ * respectively. The former is in __napi_schedule because it uses at-most-once
+ * logic and placing it in the calling routine (napi_schedule) would produce
+ * countless trace events that were effectively no-ops. napi_poll is
+ * implemented in net_rx_action, because thats where we do our polling on
+ * devices. The last trace point is in napi_complete, right where you would
+ * think it would be.
+ */
+DEFINE_TRACE(napi_schedule,
+ TPPROTO(struct napi_struct *n),
+ TPARGS(n));
+
+DEFINE_TRACE(napi_poll,
+ TPPROTO(struct napi_struct *n),
+ TPARGS(n));
+
+DEFINE_TRACE(napi_complete,
+ TPPROTO(struct napi_struct *n),
+ TPARGS(n));
+
+#endif
+
diff --git a/ltt/probes/net-trace.c b/ltt/probes/net-trace.c
index bac2b21..e887e70 100644
--- a/ltt/probes/net-trace.c
+++ b/ltt/probes/net-trace.c
@@ -9,6 +9,7 @@
#include <trace/ipv4.h>
#include <trace/ipv6.h>
#include <trace/socket.h>
+#include <trace/netdevice.h>

void probe_net_dev_xmit(struct sk_buff *skb)
{
@@ -104,6 +105,27 @@ void probe_socket_call(int call, unsigned long a0)
"call %d a0 %lu", call, a0);
}

+void probe_napi_schedule(struct napi_struct *n)
+{
+ trace_mark_tp(net_napi_schedule, napi_schedule, probe_napi_schedule,
+ "napi_struct %p name %s",
+ n, n->dev->name);
+}
+
+void probe_napi_poll(struct napi_struct *n)
+{
+ trace_mark_tp(net_napi_poll, napi_poll, probe_napi_poll,
+ "napi_struct %p name %s",
+ n, n->dev->name);
+}
+
+void probe_napi_complete(struct napi_struct *n)
+{
+ trace_mark_tp(net_napi_complete, napi_complete, probe_napi_complete,
+ "napi_struct %p name %s",
+ n, n->dev->name);
+}
+
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Mathieu Desnoyers");
MODULE_DESCRIPTION("Net Tracepoint Probes");
diff --git a/net/core/dev.c b/net/core/dev.c
index 836fe6e..0c55c0f 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -128,6 +128,7 @@
#include <linux/jhash.h>
#include <linux/random.h>
#include <trace/net.h>
+#include <trace/netdevice.h>

#include "net-sysfs.h"

@@ -2329,6 +2330,8 @@ void __napi_schedule(struct napi_struct *n)
{
unsigned long flags;

+ trace_napi_poll(n);
+
local_irq_save(flags);
list_add_tail(&n->poll_list, &__get_cpu_var(softnet_data).poll_list);
__raise_softirq_irqoff(NET_RX_SOFTIRQ);
@@ -2380,8 +2383,10 @@ static void net_rx_action(struct softirq_action *h)
* accidently calling ->poll() when NAPI is not scheduled.
*/
work = 0;
- if (test_bit(NAPI_STATE_SCHED, &n->state))
+ if (test_bit(NAPI_STATE_SCHED, &n->state)) {
+ trace_napi_poll(n);
work = n->poll(n, weight);
+ }

WARN_ON_ONCE(work > weight);

--
/****************************************************
* Neil Horman <[email protected]>
* Software Engineer, Red Hat
****************************************************/

2008-12-03 18:09:11

by Mathieu Desnoyers

[permalink] [raw]
Subject: Re: [ltt-dev] [PATCH] add tracepoints to track change in napi states on network receive

* Neil Horman ([email protected]) wrote:
> On Wed, Dec 03, 2008 at 12:25:07PM -0500, Mathieu Desnoyers wrote:
> > Almost there :)
> >
> > There is a extra whitespace above,
> >
> > and also "schedule napi_struct" should probably be changed into
> > napi_struct only, this removes the whitespace and duplicated information
> > (I guess we know that given we are in event net_napi_schedule, this will
> > have something to do with schedule...). Same applies to trace_mark_tp
> > below.
> >
> > Thanks,
> >
> > Mathieu
>
>
> Sure, here you go, new patch
>
> Patch to trace napi states for various net devices and their napi instances.
> Traces scheduling, servicing and completion of a napi poll request.
>
> Regards
> Neil
>

Merged in the LTTng tree, I simply removed one whitespace.

Thanks !

Mathieu

> Signed-off-by: Neil Horman <[email protected]>
>
>
> include/linux/netdevice.h | 5 +++++
> include/trace/netdevice.h | 30 ++++++++++++++++++++++++++++++
> ltt/probes/net-trace.c | 22 ++++++++++++++++++++++
> net/core/dev.c | 7 ++++++-
> 4 files changed, 63 insertions(+), 1 deletion(-)
>
>
> diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> index 488c56e..7da71eb 100644
> --- a/include/linux/netdevice.h
> +++ b/include/linux/netdevice.h
> @@ -43,6 +43,9 @@
>
> #include <net/net_namespace.h>
>
> +struct napi_struct;
> +#include <trace/netdevice.h>
> +
> struct vlan_group;
> struct ethtool_ops;
> struct netpoll_info;
> @@ -386,6 +389,7 @@ static inline void napi_complete(struct napi_struct *n)
> local_irq_save(flags);
> __napi_complete(n);
> local_irq_restore(flags);
> + trace_napi_complete(n);
> }
>
> /**
> @@ -1726,6 +1730,7 @@ static inline int skb_bond_should_drop(struct sk_buff *skb)
> return 0;
> }
>
> +
> #endif /* __KERNEL__ */
>
> #endif /* _LINUX_DEV_H */
> diff --git a/include/trace/netdevice.h b/include/trace/netdevice.h
> new file mode 100644
> index 0000000..ba48f6c
> --- /dev/null
> +++ b/include/trace/netdevice.h
> @@ -0,0 +1,30 @@
> +#ifndef _INCLUDE_NETDEVICE_H_
> +#define _INCLUDE_NETDEVICE_H_
> +
> +#include <linux/netdevice.h>
> +
> +/* Tracepoints */
> +
> +/*
> + * Note these first 2 traces are actually in __napi_schedule and net_rx_action
> + * respectively. The former is in __napi_schedule because it uses at-most-once
> + * logic and placing it in the calling routine (napi_schedule) would produce
> + * countless trace events that were effectively no-ops. napi_poll is
> + * implemented in net_rx_action, because thats where we do our polling on
> + * devices. The last trace point is in napi_complete, right where you would
> + * think it would be.
> + */
> +DEFINE_TRACE(napi_schedule,
> + TPPROTO(struct napi_struct *n),
> + TPARGS(n));
> +
> +DEFINE_TRACE(napi_poll,
> + TPPROTO(struct napi_struct *n),
> + TPARGS(n));
> +
> +DEFINE_TRACE(napi_complete,
> + TPPROTO(struct napi_struct *n),
> + TPARGS(n));
> +
> +#endif
> +
> diff --git a/ltt/probes/net-trace.c b/ltt/probes/net-trace.c
> index bac2b21..e887e70 100644
> --- a/ltt/probes/net-trace.c
> +++ b/ltt/probes/net-trace.c
> @@ -9,6 +9,7 @@
> #include <trace/ipv4.h>
> #include <trace/ipv6.h>
> #include <trace/socket.h>
> +#include <trace/netdevice.h>
>
> void probe_net_dev_xmit(struct sk_buff *skb)
> {
> @@ -104,6 +105,27 @@ void probe_socket_call(int call, unsigned long a0)
> "call %d a0 %lu", call, a0);
> }
>
> +void probe_napi_schedule(struct napi_struct *n)
> +{
> + trace_mark_tp(net_napi_schedule, napi_schedule, probe_napi_schedule,
> + "napi_struct %p name %s",
> + n, n->dev->name);
> +}
> +
> +void probe_napi_poll(struct napi_struct *n)
> +{
> + trace_mark_tp(net_napi_poll, napi_poll, probe_napi_poll,
> + "napi_struct %p name %s",
> + n, n->dev->name);
> +}
> +
> +void probe_napi_complete(struct napi_struct *n)
> +{
> + trace_mark_tp(net_napi_complete, napi_complete, probe_napi_complete,
> + "napi_struct %p name %s",
> + n, n->dev->name);
> +}
> +
> MODULE_LICENSE("GPL");
> MODULE_AUTHOR("Mathieu Desnoyers");
> MODULE_DESCRIPTION("Net Tracepoint Probes");
> diff --git a/net/core/dev.c b/net/core/dev.c
> index 836fe6e..0c55c0f 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -128,6 +128,7 @@
> #include <linux/jhash.h>
> #include <linux/random.h>
> #include <trace/net.h>
> +#include <trace/netdevice.h>
>
> #include "net-sysfs.h"
>
> @@ -2329,6 +2330,8 @@ void __napi_schedule(struct napi_struct *n)
> {
> unsigned long flags;
>
> + trace_napi_poll(n);
> +
> local_irq_save(flags);
> list_add_tail(&n->poll_list, &__get_cpu_var(softnet_data).poll_list);
> __raise_softirq_irqoff(NET_RX_SOFTIRQ);
> @@ -2380,8 +2383,10 @@ static void net_rx_action(struct softirq_action *h)
> * accidently calling ->poll() when NAPI is not scheduled.
> */
> work = 0;
> - if (test_bit(NAPI_STATE_SCHED, &n->state))
> + if (test_bit(NAPI_STATE_SCHED, &n->state)) {
> + trace_napi_poll(n);
> work = n->poll(n, weight);
> + }
>
> WARN_ON_ONCE(work > weight);
>
> --
> /****************************************************
> * Neil Horman <[email protected]>
> * Software Engineer, Red Hat
> ****************************************************/
>

--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68

2008-12-03 18:11:18

by Mathieu Desnoyers

[permalink] [raw]
Subject: Re: [ltt-dev] [PATCH] add tracepoints to track change in napi states on network receive

* Neil Horman ([email protected]) wrote:
> On Wed, Dec 03, 2008 at 12:25:07PM -0500, Mathieu Desnoyers wrote:
> > Almost there :)
> >
> > There is a extra whitespace above,
> >
> > and also "schedule napi_struct" should probably be changed into
> > napi_struct only, this removes the whitespace and duplicated information
> > (I guess we know that given we are in event net_napi_schedule, this will
> > have something to do with schedule...). Same applies to trace_mark_tp
> > below.
> >
> > Thanks,
> >
> > Mathieu
>
>
> Sure, here you go, new patch
>
> Patch to trace napi states for various net devices and their napi instances.
> Traces scheduling, servicing and completion of a napi poll request.
>
> Regards
> Neil
>
> Signed-off-by: Neil Horman <[email protected]>
>

Actually I notice that you used the old "DEFINE_TRACE" tracepoint API.
I'll update your patch to the new DECLARE/DEFINE_TRACE tracepoint API.

Mathieu

>
> include/linux/netdevice.h | 5 +++++
> include/trace/netdevice.h | 30 ++++++++++++++++++++++++++++++
> ltt/probes/net-trace.c | 22 ++++++++++++++++++++++
> net/core/dev.c | 7 ++++++-
> 4 files changed, 63 insertions(+), 1 deletion(-)
>
>
> diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> index 488c56e..7da71eb 100644
> --- a/include/linux/netdevice.h
> +++ b/include/linux/netdevice.h
> @@ -43,6 +43,9 @@
>
> #include <net/net_namespace.h>
>
> +struct napi_struct;
> +#include <trace/netdevice.h>
> +
> struct vlan_group;
> struct ethtool_ops;
> struct netpoll_info;
> @@ -386,6 +389,7 @@ static inline void napi_complete(struct napi_struct *n)
> local_irq_save(flags);
> __napi_complete(n);
> local_irq_restore(flags);
> + trace_napi_complete(n);
> }
>
> /**
> @@ -1726,6 +1730,7 @@ static inline int skb_bond_should_drop(struct sk_buff *skb)
> return 0;
> }
>
> +
> #endif /* __KERNEL__ */
>
> #endif /* _LINUX_DEV_H */
> diff --git a/include/trace/netdevice.h b/include/trace/netdevice.h
> new file mode 100644
> index 0000000..ba48f6c
> --- /dev/null
> +++ b/include/trace/netdevice.h
> @@ -0,0 +1,30 @@
> +#ifndef _INCLUDE_NETDEVICE_H_
> +#define _INCLUDE_NETDEVICE_H_
> +
> +#include <linux/netdevice.h>
> +
> +/* Tracepoints */
> +
> +/*
> + * Note these first 2 traces are actually in __napi_schedule and net_rx_action
> + * respectively. The former is in __napi_schedule because it uses at-most-once
> + * logic and placing it in the calling routine (napi_schedule) would produce
> + * countless trace events that were effectively no-ops. napi_poll is
> + * implemented in net_rx_action, because thats where we do our polling on
> + * devices. The last trace point is in napi_complete, right where you would
> + * think it would be.
> + */
> +DEFINE_TRACE(napi_schedule,
> + TPPROTO(struct napi_struct *n),
> + TPARGS(n));
> +
> +DEFINE_TRACE(napi_poll,
> + TPPROTO(struct napi_struct *n),
> + TPARGS(n));
> +
> +DEFINE_TRACE(napi_complete,
> + TPPROTO(struct napi_struct *n),
> + TPARGS(n));
> +
> +#endif
> +
> diff --git a/ltt/probes/net-trace.c b/ltt/probes/net-trace.c
> index bac2b21..e887e70 100644
> --- a/ltt/probes/net-trace.c
> +++ b/ltt/probes/net-trace.c
> @@ -9,6 +9,7 @@
> #include <trace/ipv4.h>
> #include <trace/ipv6.h>
> #include <trace/socket.h>
> +#include <trace/netdevice.h>
>
> void probe_net_dev_xmit(struct sk_buff *skb)
> {
> @@ -104,6 +105,27 @@ void probe_socket_call(int call, unsigned long a0)
> "call %d a0 %lu", call, a0);
> }
>
> +void probe_napi_schedule(struct napi_struct *n)
> +{
> + trace_mark_tp(net_napi_schedule, napi_schedule, probe_napi_schedule,
> + "napi_struct %p name %s",
> + n, n->dev->name);
> +}
> +
> +void probe_napi_poll(struct napi_struct *n)
> +{
> + trace_mark_tp(net_napi_poll, napi_poll, probe_napi_poll,
> + "napi_struct %p name %s",
> + n, n->dev->name);
> +}
> +
> +void probe_napi_complete(struct napi_struct *n)
> +{
> + trace_mark_tp(net_napi_complete, napi_complete, probe_napi_complete,
> + "napi_struct %p name %s",
> + n, n->dev->name);
> +}
> +
> MODULE_LICENSE("GPL");
> MODULE_AUTHOR("Mathieu Desnoyers");
> MODULE_DESCRIPTION("Net Tracepoint Probes");
> diff --git a/net/core/dev.c b/net/core/dev.c
> index 836fe6e..0c55c0f 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -128,6 +128,7 @@
> #include <linux/jhash.h>
> #include <linux/random.h>
> #include <trace/net.h>
> +#include <trace/netdevice.h>
>
> #include "net-sysfs.h"
>
> @@ -2329,6 +2330,8 @@ void __napi_schedule(struct napi_struct *n)
> {
> unsigned long flags;
>
> + trace_napi_poll(n);
> +
> local_irq_save(flags);
> list_add_tail(&n->poll_list, &__get_cpu_var(softnet_data).poll_list);
> __raise_softirq_irqoff(NET_RX_SOFTIRQ);
> @@ -2380,8 +2383,10 @@ static void net_rx_action(struct softirq_action *h)
> * accidently calling ->poll() when NAPI is not scheduled.
> */
> work = 0;
> - if (test_bit(NAPI_STATE_SCHED, &n->state))
> + if (test_bit(NAPI_STATE_SCHED, &n->state)) {
> + trace_napi_poll(n);
> work = n->poll(n, weight);
> + }
>
> WARN_ON_ONCE(work > weight);
>
> --
> /****************************************************
> * Neil Horman <[email protected]>
> * Software Engineer, Red Hat
> ****************************************************/
>

--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68

2008-12-03 18:21:44

by Mathieu Desnoyers

[permalink] [raw]
Subject: Re: [ltt-dev] [PATCH] add tracepoints to track change in napi states on network receive

* Mathieu Desnoyers ([email protected]) wrote:
> * Neil Horman ([email protected]) wrote:
> > On Wed, Dec 03, 2008 at 12:25:07PM -0500, Mathieu Desnoyers wrote:
> > > Almost there :)
> > >
> > > There is a extra whitespace above,
> > >
> > > and also "schedule napi_struct" should probably be changed into
> > > napi_struct only, this removes the whitespace and duplicated information
> > > (I guess we know that given we are in event net_napi_schedule, this will
> > > have something to do with schedule...). Same applies to trace_mark_tp
> > > below.
> > >
> > > Thanks,
> > >
> > > Mathieu
> >
> >
> > Sure, here you go, new patch
> >
> > Patch to trace napi states for various net devices and their napi instances.
> > Traces scheduling, servicing and completion of a napi poll request.
> >
> > Regards
> > Neil
> >
> > Signed-off-by: Neil Horman <[email protected]>
> >
>
> Actually I notice that you used the old "DEFINE_TRACE" tracepoint API.

The patch has been edited and folded into
lttng-instrumentation-net.patch
lttng-instrumentation-net-tracepoints-probes.patch

Mathieu

> I'll update your patch to the new DECLARE/DEFINE_TRACE tracepoint API.
>
> Mathieu
>
> >
> > include/linux/netdevice.h | 5 +++++
> > include/trace/netdevice.h | 30 ++++++++++++++++++++++++++++++
> > ltt/probes/net-trace.c | 22 ++++++++++++++++++++++
> > net/core/dev.c | 7 ++++++-
> > 4 files changed, 63 insertions(+), 1 deletion(-)
> >
> >
> > diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> > index 488c56e..7da71eb 100644
> > --- a/include/linux/netdevice.h
> > +++ b/include/linux/netdevice.h
> > @@ -43,6 +43,9 @@
> >
> > #include <net/net_namespace.h>
> >
> > +struct napi_struct;
> > +#include <trace/netdevice.h>
> > +
> > struct vlan_group;
> > struct ethtool_ops;
> > struct netpoll_info;
> > @@ -386,6 +389,7 @@ static inline void napi_complete(struct napi_struct *n)
> > local_irq_save(flags);
> > __napi_complete(n);
> > local_irq_restore(flags);
> > + trace_napi_complete(n);
> > }
> >
> > /**
> > @@ -1726,6 +1730,7 @@ static inline int skb_bond_should_drop(struct sk_buff *skb)
> > return 0;
> > }
> >
> > +
> > #endif /* __KERNEL__ */
> >
> > #endif /* _LINUX_DEV_H */
> > diff --git a/include/trace/netdevice.h b/include/trace/netdevice.h
> > new file mode 100644
> > index 0000000..ba48f6c
> > --- /dev/null
> > +++ b/include/trace/netdevice.h
> > @@ -0,0 +1,30 @@
> > +#ifndef _INCLUDE_NETDEVICE_H_
> > +#define _INCLUDE_NETDEVICE_H_
> > +
> > +#include <linux/netdevice.h>
> > +
> > +/* Tracepoints */
> > +
> > +/*
> > + * Note these first 2 traces are actually in __napi_schedule and net_rx_action
> > + * respectively. The former is in __napi_schedule because it uses at-most-once
> > + * logic and placing it in the calling routine (napi_schedule) would produce
> > + * countless trace events that were effectively no-ops. napi_poll is
> > + * implemented in net_rx_action, because thats where we do our polling on
> > + * devices. The last trace point is in napi_complete, right where you would
> > + * think it would be.
> > + */
> > +DEFINE_TRACE(napi_schedule,
> > + TPPROTO(struct napi_struct *n),
> > + TPARGS(n));
> > +
> > +DEFINE_TRACE(napi_poll,
> > + TPPROTO(struct napi_struct *n),
> > + TPARGS(n));
> > +
> > +DEFINE_TRACE(napi_complete,
> > + TPPROTO(struct napi_struct *n),
> > + TPARGS(n));
> > +
> > +#endif
> > +
> > diff --git a/ltt/probes/net-trace.c b/ltt/probes/net-trace.c
> > index bac2b21..e887e70 100644
> > --- a/ltt/probes/net-trace.c
> > +++ b/ltt/probes/net-trace.c
> > @@ -9,6 +9,7 @@
> > #include <trace/ipv4.h>
> > #include <trace/ipv6.h>
> > #include <trace/socket.h>
> > +#include <trace/netdevice.h>
> >
> > void probe_net_dev_xmit(struct sk_buff *skb)
> > {
> > @@ -104,6 +105,27 @@ void probe_socket_call(int call, unsigned long a0)
> > "call %d a0 %lu", call, a0);
> > }
> >
> > +void probe_napi_schedule(struct napi_struct *n)
> > +{
> > + trace_mark_tp(net_napi_schedule, napi_schedule, probe_napi_schedule,
> > + "napi_struct %p name %s",
> > + n, n->dev->name);
> > +}
> > +
> > +void probe_napi_poll(struct napi_struct *n)
> > +{
> > + trace_mark_tp(net_napi_poll, napi_poll, probe_napi_poll,
> > + "napi_struct %p name %s",
> > + n, n->dev->name);
> > +}
> > +
> > +void probe_napi_complete(struct napi_struct *n)
> > +{
> > + trace_mark_tp(net_napi_complete, napi_complete, probe_napi_complete,
> > + "napi_struct %p name %s",
> > + n, n->dev->name);
> > +}
> > +
> > MODULE_LICENSE("GPL");
> > MODULE_AUTHOR("Mathieu Desnoyers");
> > MODULE_DESCRIPTION("Net Tracepoint Probes");
> > diff --git a/net/core/dev.c b/net/core/dev.c
> > index 836fe6e..0c55c0f 100644
> > --- a/net/core/dev.c
> > +++ b/net/core/dev.c
> > @@ -128,6 +128,7 @@
> > #include <linux/jhash.h>
> > #include <linux/random.h>
> > #include <trace/net.h>
> > +#include <trace/netdevice.h>
> >
> > #include "net-sysfs.h"
> >
> > @@ -2329,6 +2330,8 @@ void __napi_schedule(struct napi_struct *n)
> > {
> > unsigned long flags;
> >
> > + trace_napi_poll(n);
> > +
> > local_irq_save(flags);
> > list_add_tail(&n->poll_list, &__get_cpu_var(softnet_data).poll_list);
> > __raise_softirq_irqoff(NET_RX_SOFTIRQ);
> > @@ -2380,8 +2383,10 @@ static void net_rx_action(struct softirq_action *h)
> > * accidently calling ->poll() when NAPI is not scheduled.
> > */
> > work = 0;
> > - if (test_bit(NAPI_STATE_SCHED, &n->state))
> > + if (test_bit(NAPI_STATE_SCHED, &n->state)) {
> > + trace_napi_poll(n);
> > work = n->poll(n, weight);
> > + }
> >
> > WARN_ON_ONCE(work > weight);
> >
> > --
> > /****************************************************
> > * Neil Horman <[email protected]>
> > * Software Engineer, Red Hat
> > ****************************************************/
> >
>
> --
> Mathieu Desnoyers
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
>
> _______________________________________________
> ltt-dev mailing list
> [email protected]
> http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>

--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68

2008-12-03 19:28:23

by Neil Horman

[permalink] [raw]
Subject: Re: [ltt-dev] [PATCH] add tracepoints to track change in napi states on network receive

On Wed, Dec 03, 2008 at 01:21:33PM -0500, Mathieu Desnoyers wrote:
> * Mathieu Desnoyers ([email protected]) wrote:
> > * Neil Horman ([email protected]) wrote:
> > > On Wed, Dec 03, 2008 at 12:25:07PM -0500, Mathieu Desnoyers wrote:
> > > > Almost there :)
> > > >
> > > > There is a extra whitespace above,
> > > >
> > > > and also "schedule napi_struct" should probably be changed into
> > > > napi_struct only, this removes the whitespace and duplicated information
> > > > (I guess we know that given we are in event net_napi_schedule, this will
> > > > have something to do with schedule...). Same applies to trace_mark_tp
> > > > below.
> > > >
> > > > Thanks,
> > > >
> > > > Mathieu
> > >
> > >
> > > Sure, here you go, new patch
> > >
> > > Patch to trace napi states for various net devices and their napi instances.
> > > Traces scheduling, servicing and completion of a napi poll request.
> > >
> > > Regards
> > > Neil
> > >
> > > Signed-off-by: Neil Horman <[email protected]>
> > >
> >
> > Actually I notice that you used the old "DEFINE_TRACE" tracepoint API.
>
> The patch has been edited and folded into
> lttng-instrumentation-net.patch
> lttng-instrumentation-net-tracepoints-probes.patch
>
Thanks!
Neil


--
/****************************************************
* Neil Horman <[email protected]>
* Software Engineer, Red Hat
****************************************************/