x86: Don't send RESCHEDULE_VECTOR to offlined cpus.
From: Gautham R Shenoy <[email protected]>
In the x86 native_smp_send_reschedule_function(), don't send the IPI if the
cpu has gone offline already. Warn nevertheless!!
Signed-off-by: Gautham R Shenoy <[email protected]>
---
arch/x86/kernel/smp_32.c | 5 ++++-
1 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/arch/x86/kernel/smp_32.c b/arch/x86/kernel/smp_32.c
index dc0cde9..4df9042 100644
--- a/arch/x86/kernel/smp_32.c
+++ b/arch/x86/kernel/smp_32.c
@@ -472,7 +472,10 @@ void flush_tlb_all(void)
*/
static void native_smp_send_reschedule(int cpu)
{
- WARN_ON(cpu_is_offline(cpu));
+ if (unlikely(cpu_is_offline(cpu))) {
+ WARN_ON(1);
+ return;
+ }
send_IPI_mask(cpumask_of_cpu(cpu), RESCHEDULE_VECTOR);
}
--
Thanks and Regards
gautham
* Gautham R Shenoy <[email protected]> wrote:
> x86: Don't send RESCHEDULE_VECTOR to offlined cpus.
> From: Gautham R Shenoy <[email protected]>
>
> In the x86 native_smp_send_reschedule_function(), don't send the IPI
> if the cpu has gone offline already. Warn nevertheless!!
have you seen this happen?
Ingo
On Mon, Mar 10, 2008 at 01:24:18PM +0100, Ingo Molnar wrote:
>
> * Gautham R Shenoy <[email protected]> wrote:
>
> > x86: Don't send RESCHEDULE_VECTOR to offlined cpus.
> > From: Gautham R Shenoy <[email protected]>
> >
> > In the x86 native_smp_send_reschedule_function(), don't send the IPI
> > if the cpu has gone offline already. Warn nevertheless!!
>
> have you seen this happen?
Yup, this afternoon while running
cpu-hotplug stress tests (http://lkml.org/lkml/2008/3/3/11)
in parallel with kern-bench, I got this on the serial console.
------------[ cut here ]------------
WARNING: at arch/x86/kernel/smp_32.c:475 native_smp_send_reschedule+0x22/0x3f()
Modules linked in: dock
Pid: 4722, comm: kstopmachine Not tainted 2.6.25-rc3 #46
[<c011f424>] warn_on_slowpath+0x41/0x51
[<c013ac74>] ? __lock_acquire+0xaae/0xaf6
[<c011a446>] ? resched_cpu+0x2c/0x6f
[<c0111691>] native_smp_send_reschedule+0x22/0x3f
[<c01186c7>] __resched_task+0x5f/0x63
[<c011a479>] resched_cpu+0x5f/0x6f
[<c011bb6d>] scheduler_tick+0x214/0x28f
[<c012737b>] update_process_times+0x3d/0x49
[<c013751d>] tick_sched_timer+0x6e/0xa6
[<c01374af>] ? tick_sched_timer+0x0/0xa6
[<c0131e8c>] __run_hrtimer+0x39/0x70
[<c0132668>] hrtimer_interrupt+0xeb/0x154
[<c01127bc>] smp_apic_timer_interrupt+0x6c/0x80
[<c0143670>] ? stopmachine+0x0/0x98
[<c0105553>] apic_timer_interrupt+0x33/0x38
[<c0143670>] ? stopmachine+0x0/0x98
[<c013007b>] ? sample_to_timespec+0x16/0x35
[<c0143701>] ? stopmachine+0x91/0x98
[<c01056eb>] kernel_thread_helper+0x7/0x10
=======================
---[ end trace 1016b68a0b60b50b ]---
------------[ cut here ]------------
>
> Ingo
--
Thanks and Regards
gautham
Gautham R Shenoy wrote:
> On Mon, Mar 10, 2008 at 01:24:18PM +0100, Ingo Molnar wrote:
>> * Gautham R Shenoy <[email protected]> wrote:
>>
>>> x86: Don't send RESCHEDULE_VECTOR to offlined cpus.
>>> From: Gautham R Shenoy <[email protected]>
>>>
>>> In the x86 native_smp_send_reschedule_function(), don't send the IPI
>>> if the cpu has gone offline already. Warn nevertheless!!
>> have you seen this happen?
>
> Yup, this afternoon while running
> cpu-hotplug stress tests (http://lkml.org/lkml/2008/3/3/11)
> in parallel with kern-bench, I got this on the serial console.
from arch/m32r/kernel/smp.c:134
void smp_send_reschedule(int cpu_id)
{
WARN_ON(cpu_is_offline(cpu_id));
send_IPI_mask(cpumask_of_cpu(cpu_id), RESCHEDULE_IPI, 1);
}
should this be changed as well?
Gautham R Shenoy wrote:
> On Mon, Mar 10, 2008 at 01:24:18PM +0100, Ingo Molnar wrote:
>> * Gautham R Shenoy <[email protected]> wrote:
>>
>>> x86: Don't send RESCHEDULE_VECTOR to offlined cpus.
>>> From: Gautham R Shenoy <[email protected]>
>>>
>>> In the x86 native_smp_send_reschedule_function(), don't send the IPI
>>> if the cpu has gone offline already. Warn nevertheless!!
>> have you seen this happen?
>
> Yup, this afternoon while running
> cpu-hotplug stress tests (http://lkml.org/lkml/2008/3/3/11)
> in parallel with kern-bench, I got this on the serial console.
Note that this may be a side-effect of the bug we are chasing with the
rd->online problem.
E.g. if the rd->online issue is causing us to route tasks to dead CPUs,
this phenomenon would probably go away once rd->online is fixed (which I
believe it is with the patches that were submitted).
Regards,
-Greg
* Gregory Haskins <[email protected]> wrote:
>> Yup, this afternoon while running cpu-hotplug stress tests
>> (http://lkml.org/lkml/2008/3/3/11) in parallel with kern-bench, I got
>> this on the serial console.
>
> Note that this may be a side-effect of the bug we are chasing with the
> rd->online problem.
>
> E.g. if the rd->online issue is causing us to route tasks to dead
> CPUs, this phenomenon would probably go away once rd->online is fixed
> (which I believe it is with the patches that were submitted).
yeah, that makes sense. I've applied your patch nevertheless because it
is the better failure mode for that code.
Ingo