2011-02-01 05:27:36

by Santosh Shilimkar

[permalink] [raw]
Subject: RE: [PATCH v2] oprofile: add SMP barriers for hrtimer hotplug code

> -----Original Message-----
> From: Will Deacon [mailto:[email protected]]
> Sent: Monday, January 31, 2011 11:23 PM
> To: [email protected]; [email protected]
> Cc: Will Deacon; Santosh Shilimkar; Robert Richter
> Subject: [PATCH v2] oprofile: add SMP barriers for hrtimer hotplug
> code
>
> OProfile uses a CPU notifier to start and stop any hrtimers when
> CPUs change
> between ONLINE and DEAD. A static int ctr_running is used to keep
> track of
> the counter state.
>
> This can lead to problems where writes to the state variable are re-
> ordered
> with repect to reads of the variable occurring on other CPUs,
> meaning that
> __oprofile_hrtimer_start may read ctr_running as 0 and not
> initialise the
> hrtimer. Potential deadlock can occur in __oprofile_hrtimer_stop
> because
> lock_hrtimer_base will poll until timer->base != NULL, which will
> never
> happen.
>
> This patch adds an smp_mb() before initialising the hrtimers to
> ensure that
> ctr_running mirrors the correct counter state.
>
> Cc: Santosh Shilimkar <[email protected]>
> Cc: Robert Richter <[email protected]>
> Signed-off-by: Will Deacon <[email protected]>
> ---
>
Acked-by: Santosh Shilimkar <[email protected]>

> Changes since v1: Removed redundant barrier from stop() code.
>
> drivers/oprofile/timer_int.c | 1 +
> 1 files changed, 1 insertions(+), 0 deletions(-)
>
> diff --git a/drivers/oprofile/timer_int.c
> b/drivers/oprofile/timer_int.c
> index 0107251..a3a94ea 100644
> --- a/drivers/oprofile/timer_int.c
> +++ b/drivers/oprofile/timer_int.c
> @@ -48,6 +48,7 @@ static int oprofile_hrtimer_start(void)
> {
> get_online_cpus();
> ctr_running = 1;
> + smp_mb();
> on_each_cpu(__oprofile_hrtimer_start, NULL, 1);
> put_online_cpus();
> return 0;
> --
> 1.7.0.4
>


2011-02-01 08:03:56

by Ming Lei

[permalink] [raw]
Subject: Re: [PATCH v2] oprofile: add SMP barriers for hrtimer hotplug code

Hi,

2011/2/1 Santosh Shilimkar <[email protected]>:
>> -----Original Message-----
>> From: Will Deacon [mailto:[email protected]]
>> diff --git a/drivers/oprofile/timer_int.c
>> b/drivers/oprofile/timer_int.c
>> index 0107251..a3a94ea 100644
>> --- a/drivers/oprofile/timer_int.c
>> +++ b/drivers/oprofile/timer_int.c
>> @@ -48,6 +48,7 @@ static int oprofile_hrtimer_start(void)
>> ?{
>> ? ? ? get_online_cpus();
>> ? ? ? ctr_running = 1;
>> + ? ? smp_mb();
>> ? ? ? on_each_cpu(__oprofile_hrtimer_start, NULL, 1);

Just a little question:

Before calling __oprofile_hrtimer_start in other CPUs,
there have been some smp_mb executed already, such as called by
smp_call_function_many in current CPU, and called by
generic_smp_call_function_interrupt in the func-calling CPU,
so are these smp_mb enough for correct order of access
for 'ctr_running' ?

>> ? ? ? put_online_cpus();
>> ? ? ? return 0;

thanks,
--
Lei Ming

2011-02-01 17:08:37

by Will Deacon

[permalink] [raw]
Subject: Re: [PATCH v2] oprofile: add SMP barriers for hrtimer hotplug code

On Tue, 2011-02-01 at 08:03 +0000, Ming Lei wrote:
> Hi,
>
Hello Lei Ming,

> 2011/2/1 Santosh Shilimkar <[email protected]>:
> >> -----Original Message-----
> >> From: Will Deacon [mailto:[email protected]]
> >> diff --git a/drivers/oprofile/timer_int.c
> >> b/drivers/oprofile/timer_int.c
> >> index 0107251..a3a94ea 100644
> >> --- a/drivers/oprofile/timer_int.c
> >> +++ b/drivers/oprofile/timer_int.c
> >> @@ -48,6 +48,7 @@ static int oprofile_hrtimer_start(void)
> >> {
> >> get_online_cpus();
> >> ctr_running = 1;
> >> + smp_mb();
> >> on_each_cpu(__oprofile_hrtimer_start, NULL, 1);
>
> Just a little question:
>
> Before calling __oprofile_hrtimer_start in other CPUs,
> there have been some smp_mb executed already, such as called by
> smp_call_function_many in current CPU, and called by
> generic_smp_call_function_interrupt in the func-calling CPU,
> so are these smp_mb enough for correct order of access
> for 'ctr_running' ?

I didn't dive into the guts of the cross-calling implementation, but it
appears that you're correct. There's also the csd_{un}lock functions
which have smp_mb() in them too, so I think we have plenty of barrier
goodness as it is.

I'll trash the patch.

Cheers,

Will