On Mon, Dec 03, 2007 at 02:49:12PM +0300, Alexey Kuznetsov wrote:
> On Mon, Dec 03, 2007 at 10:39:35PM +1100, Herbert Xu wrote:
> > So we need to fix this, and whatever the fix is will probably render
> > the BH/USER distinction obsolete.
>
> Hmm, I would think opposite. USER (or generic) is expensive variant,
> BH is lite. No?
Yes that would certainly be the obvious fix. In other words, just use
smp_processor_id instead of the raw version. I suppose the only issue
would be that disabling/enabling preemption isn't exactly cost-free and
we're going to be doing that for every single increment.
Hmm, wasn't someone else talking about a non-atomic version of atomic
ops lately (i.e., atomic with respect to the local CPU only)? Perhaps
this is the killer app for it :)
Thanks,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
On Mon, Dec 03, 2007 at 10:54:35PM +1100, Herbert Xu wrote:
>
> Hmm, wasn't someone else talking about a non-atomic version of atomic
> ops lately (i.e., atomic with respect to the local CPU only)? Perhaps
> this is the killer app for it :)
Never mind, we already have that in local_t and as Alexey correctly
points out, USER is still going to be the expensive variant with the
preempt_disable (well until BH gets threaded). So how about this patch?
Thanks,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
diff --git a/include/net/snmp.h b/include/net/snmp.h
index ea206bf..9444b54 100644
--- a/include/net/snmp.h
+++ b/include/net/snmp.h
@@ -23,6 +23,7 @@
#include <linux/cache.h>
#include <linux/snmp.h>
+#include <linux/smp.h>
/*
* Mibs are stored in array of unsigned long.
@@ -137,7 +138,10 @@ struct linux_mib {
#define SNMP_INC_STATS_OFFSET_BH(mib, field, offset) \
(per_cpu_ptr(mib[0], raw_smp_processor_id())->mibs[field + (offset)]++)
#define SNMP_INC_STATS_USER(mib, field) \
- (per_cpu_ptr(mib[1], raw_smp_processor_id())->mibs[field]++)
+ do { \
+ per_cpu_ptr(mib[1], get_cpu())->mibs[field]++; \
+ put_cpu(); \
+ } while (0)
#define SNMP_INC_STATS(mib, field) \
(per_cpu_ptr(mib[!in_softirq()], raw_smp_processor_id())->mibs[field]++)
#define SNMP_DEC_STATS(mib, field) \
@@ -145,6 +149,9 @@ struct linux_mib {
#define SNMP_ADD_STATS_BH(mib, field, addend) \
(per_cpu_ptr(mib[0], raw_smp_processor_id())->mibs[field] += addend)
#define SNMP_ADD_STATS_USER(mib, field, addend) \
- (per_cpu_ptr(mib[1], raw_smp_processor_id())->mibs[field] += addend)
+ do { \
+ per_cpu_ptr(mib[1], get_cpu())->mibs[field] += addend; \
+ put_cpu(); \
+ } while (0)
#endif
Herbert Xu said the following on 2007-12-3 21:17:
> On Mon, Dec 03, 2007 at 10:54:35PM +1100, Herbert Xu wrote:
> diff --git a/include/net/snmp.h b/include/net/snmp.h
> index ea206bf..9444b54 100644
> --- a/include/net/snmp.h
> +++ b/include/net/snmp.h
> @@ -23,6 +23,7 @@
>
> #include <linux/cache.h>
> #include <linux/snmp.h>
> +#include <linux/smp.h>
It's no need to include smp.h?
--
WCN
On Tue, Dec 04, 2007 at 11:50:55AM +0800, Wang Chen wrote:
>
> > #include <linux/cache.h>
> > #include <linux/snmp.h>
> > +#include <linux/smp.h>
>
> It's no need to include smp.h?
We need it for smp_processor_id. Well we needed it before too but
there's probably some implicit inclusion which has made it work.
It's better to declare these inclusions explicitly as otherwise
they may break on less common architectures later.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Ob Tue, Dec 04, 2007 at 12:17:23AM +1100, Herbert Xu wrote:
>
> Never mind, we already have that in local_t and as Alexey correctly
> points out, USER is still going to be the expensive variant with the
> preempt_disable (well until BH gets threaded). So how about this patch?
I didn't hear any objections so here is the patch again.
[SNMP]: Fix SNMP counters with PREEMPT
The SNMP macros use raw_smp_processor_id() in process context
which is illegal because the process may be preempted and then
migrated to another CPU.
This patch makes it use get_cpu/put_cpu to disable preemption.
Signed-off-by: Herbert Xu <[email protected]>
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
diff --git a/include/net/snmp.h b/include/net/snmp.h
index ea206bf..9444b54 100644
--- a/include/net/snmp.h
+++ b/include/net/snmp.h
@@ -23,6 +23,7 @@
#include <linux/cache.h>
#include <linux/snmp.h>
+#include <linux/smp.h>
/*
* Mibs are stored in array of unsigned long.
@@ -137,7 +138,10 @@ struct linux_mib {
#define SNMP_INC_STATS_OFFSET_BH(mib, field, offset) \
(per_cpu_ptr(mib[0], raw_smp_processor_id())->mibs[field + (offset)]++)
#define SNMP_INC_STATS_USER(mib, field) \
- (per_cpu_ptr(mib[1], raw_smp_processor_id())->mibs[field]++)
+ do { \
+ per_cpu_ptr(mib[1], get_cpu())->mibs[field]++; \
+ put_cpu(); \
+ } while (0)
#define SNMP_INC_STATS(mib, field) \
(per_cpu_ptr(mib[!in_softirq()], raw_smp_processor_id())->mibs[field]++)
#define SNMP_DEC_STATS(mib, field) \
@@ -145,6 +149,9 @@ struct linux_mib {
#define SNMP_ADD_STATS_BH(mib, field, addend) \
(per_cpu_ptr(mib[0], raw_smp_processor_id())->mibs[field] += addend)
#define SNMP_ADD_STATS_USER(mib, field, addend) \
- (per_cpu_ptr(mib[1], raw_smp_processor_id())->mibs[field] += addend)
+ do { \
+ per_cpu_ptr(mib[1], get_cpu())->mibs[field] += addend; \
+ put_cpu(); \
+ } while (0)
#endif
Herbert Xu a ?crit :
> Ob Tue, Dec 04, 2007 at 12:17:23AM +1100, Herbert Xu wrote:
>> Never mind, we already have that in local_t and as Alexey correctly
>> points out, USER is still going to be the expensive variant with the
>> preempt_disable (well until BH gets threaded). So how about this patch?
>
> I didn't hear any objections so here is the patch again.
>
> [SNMP]: Fix SNMP counters with PREEMPT
>
> The SNMP macros use raw_smp_processor_id() in process context
> which is illegal because the process may be preempted and then
> migrated to another CPU.
>
> This patch makes it use get_cpu/put_cpu to disable preemption.
>
> Signed-off-by: Herbert Xu <[email protected]>
>
> Cheers,
#define SNMP_INC_STATS_USER(mib, field) \
- (per_cpu_ptr(mib[1], raw_smp_processor_id())->mibs[field]++)
+ do { \
+ per_cpu_ptr(mib[1], get_cpu())->mibs[field]++; \
+ put_cpu(); \
+ } while (0)
#define SNMP_INC_STATS(mib, field) \
(per_cpu_ptr(mib[!in_softirq()], raw_smp_processor_id())->mibs[field]++)
How come you change SNMP_INC_STATS_USER() but not SNMP_INC_STATS() ?
* Herbert Xu <[email protected]> wrote:
> Ob Tue, Dec 04, 2007 at 12:17:23AM +1100, Herbert Xu wrote:
> >
> > Never mind, we already have that in local_t and as Alexey correctly
> > points out, USER is still going to be the expensive variant with the
> > preempt_disable (well until BH gets threaded). So how about this patch?
>
> I didn't hear any objections so here is the patch again.
>
> [SNMP]: Fix SNMP counters with PREEMPT
>
> The SNMP macros use raw_smp_processor_id() in process context which is
> illegal because the process may be preempted and then migrated to
> another CPU.
nit: please use 'invalid' instead of 'illegal'.
> This patch makes it use get_cpu/put_cpu to disable preemption.
>
> Signed-off-by: Herbert Xu <[email protected]>
> - (per_cpu_ptr(mib[1], raw_smp_processor_id())->mibs[field]++)
> + do { \
> + per_cpu_ptr(mib[1], get_cpu())->mibs[field]++; \
> + put_cpu(); \
> + } while (0)
> - (per_cpu_ptr(mib[1], raw_smp_processor_id())->mibs[field] += addend)
> + do { \
> + per_cpu_ptr(mib[1], get_cpu())->mibs[field] += addend; \
> + put_cpu(); \
> + } while (0)
we could perhaps introduce stat_smp_processor_id(), which signals that
the CPU id is used for statistical purposes and does not have to be
exact? In any case, your patch looks good too.
Ingo
On Sat, Dec 15, 2007 at 06:03:19PM +0100, Eric Dumazet wrote:
>
> How come you change SNMP_INC_STATS_USER() but not SNMP_INC_STATS() ?
Heh, my brain must have blocked me from seeing it because it's
too hard :)
Let's fix it the stupid way first and I'll do a local_t conversion
later.
[SNMP]: Fix SNMP counters with PREEMPT
The SNMP macros use raw_smp_processor_id() in process context
which is illegal because the process may be preempted and then
migrated to another CPU.
This patch makes it use get_cpu/put_cpu to disable preemption.
Signed-off-by: Herbert Xu <[email protected]>
Thanks,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
diff --git a/include/net/snmp.h b/include/net/snmp.h
index 9c5793d..fbb6666 100644
--- a/include/net/snmp.h
+++ b/include/net/snmp.h
@@ -23,6 +23,7 @@
#include <linux/cache.h>
#include <linux/snmp.h>
+#include <linux/smp.h>
/*
* Mibs are stored in array of unsigned long.
@@ -135,14 +136,26 @@ struct linux_mib {
#define SNMP_INC_STATS_BH(mib, field) \
(per_cpu_ptr(mib[0], raw_smp_processor_id())->mibs[field]++)
#define SNMP_INC_STATS_USER(mib, field) \
- (per_cpu_ptr(mib[1], raw_smp_processor_id())->mibs[field]++)
+ do { \
+ per_cpu_ptr(mib[1], get_cpu())->mibs[field]++; \
+ put_cpu(); \
+ } while (0)
#define SNMP_INC_STATS(mib, field) \
- (per_cpu_ptr(mib[!in_softirq()], raw_smp_processor_id())->mibs[field]++)
+ do { \
+ per_cpu_ptr(mib[!in_softirq()], get_cpu())->mibs[field]++; \
+ put_cpu(); \
+ } while (0)
#define SNMP_DEC_STATS(mib, field) \
- (per_cpu_ptr(mib[!in_softirq()], raw_smp_processor_id())->mibs[field]--)
+ do { \
+ per_cpu_ptr(mib[!in_softirq()], get_cpu())->mibs[field]--; \
+ put_cpu(); \
+ } while (0)
#define SNMP_ADD_STATS_BH(mib, field, addend) \
(per_cpu_ptr(mib[0], raw_smp_processor_id())->mibs[field] += addend)
#define SNMP_ADD_STATS_USER(mib, field, addend) \
- (per_cpu_ptr(mib[1], raw_smp_processor_id())->mibs[field] += addend)
+ do { \
+ per_cpu_ptr(mib[1], get_cpu())->mibs[field] += addend; \
+ put_cpu(); \
+ } while (0)
#endif
On Sat, Dec 15, 2007 at 07:43:28PM +0100, Ingo Molnar wrote:
>
> we could perhaps introduce stat_smp_processor_id(), which signals that
> the CPU id is used for statistical purposes and does not have to be
> exact? In any case, your patch looks good too.
Unfortunately that doesn't work because we can then have two
CPUs trying to update the same counter which may corrupt it.
However, an optimisation is indeed possible with some work.
If we can get the address of the per-cpu counter against
some sort of a per-cpu base pointer, e.g., %gs on x86, then
we can do
incq %gs:(%rax)
where %rax would be the offset with %gs as the base. This would
obviate the need for the CPU ID and therefore avoid disabling
preemption.
Hmm, wasn't Christoph working on something like that?
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
* Herbert Xu <[email protected]> wrote:
> On Sat, Dec 15, 2007 at 07:43:28PM +0100, Ingo Molnar wrote:
> >
> > we could perhaps introduce stat_smp_processor_id(), which signals that
> > the CPU id is used for statistical purposes and does not have to be
> > exact? In any case, your patch looks good too.
>
> Unfortunately that doesn't work because we can then have two CPUs
> trying to update the same counter which may corrupt it.
ah, indeed. I missed that correctness aspect of your patch. Good catch!
Ingo
The cpu alloc patches also fix this issue one way (disabling preempt) or
the other (atomic instruction that does not need disabling of
preeemption).
On Sun, 16 Dec 2007, Herbert Xu wrote:
> If we can get the address of the per-cpu counter against
> some sort of a per-cpu base pointer, e.g., %gs on x86, then
> we can do
>
> incq %gs:(%rax)
>
> where %rax would be the offset with %gs as the base. This would
> obviate the need for the CPU ID and therefore avoid disabling
> preemption.
>
> Hmm, wasn't Christoph working on something like that?
Yes that is what the cpu alloc patchset implements.
From: Herbert Xu <[email protected]>
Date: Sat, 15 Dec 2007 21:58:52 +0800
> [SNMP]: Fix SNMP counters with PREEMPT
>
> The SNMP macros use raw_smp_processor_id() in process context
> which is illegal because the process may be preempted and then
> migrated to another CPU.
>
> This patch makes it use get_cpu/put_cpu to disable preemption.
>
> Signed-off-by: Herbert Xu <[email protected]>
Applied to net-2.6.25, thanks.
From: Herbert Xu <[email protected]>
Date: Sun, 16 Dec 2007 10:30:25 +0800
> On Sat, Dec 15, 2007 at 06:03:19PM +0100, Eric Dumazet wrote:
> >
> > How come you change SNMP_INC_STATS_USER() but not SNMP_INC_STATS() ?
>
> Heh, my brain must have blocked me from seeing it because it's
> too hard :)
>
> Let's fix it the stupid way first and I'll do a local_t conversion
> later.
>
> [SNMP]: Fix SNMP counters with PREEMPT
>
> The SNMP macros use raw_smp_processor_id() in process context
> which is illegal because the process may be preempted and then
> migrated to another CPU.
>
> This patch makes it use get_cpu/put_cpu to disable preemption.
>
> Signed-off-by: Herbert Xu <[email protected]>
I just noticed this and replaced the other SNMP fix patch
with this one.