2018-09-06 06:42:04

by Olaf Hering

[permalink] [raw]
Subject: [PATCH v2] xen: avoid crash in disable_hotplug_cpu

The command 'xl vcpu-set 0 0', issued in dom0, will crash dom0:

BUG: unable to handle kernel NULL pointer dereference at 00000000000002d8
PGD 0 P4D 0
Oops: 0000 [#1] PREEMPT SMP NOPTI
CPU: 7 PID: 65 Comm: xenwatch Not tainted 4.19.0-rc2-1.ga9462db-default #1 openSUSE Tumbleweed (unreleased)
Hardware name: Intel Corporation S5520UR/S5520UR, BIOS S5500.86B.01.00.0050.050620101605 05/06/2010
RIP: e030:device_offline+0x9/0xb0
Code: 77 24 00 e9 ce fe ff ff 48 8b 13 e9 68 ff ff ff 48 8b 13 e9 29 ff ff ff 48 8b 13 e9 ea fe ff ff 90 66 66 66 66 90 41 54 55 53 <f6> 87 d8 02 00 00 01 0f 85 88 00 00 00 48 c7 c2 20 09 60 81 31 f6
RSP: e02b:ffffc90040f27e80 EFLAGS: 00010203
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: ffff8801f3800000 RSI: ffffc90040f27e70 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffffffff820e47b3 R09: 0000000000000000
R10: 0000000000007ff0 R11: 0000000000000000 R12: ffffffff822e6d30
R13: dead000000000200 R14: dead000000000100 R15: ffffffff8158b4e0
FS: 00007ffa595158c0(0000) GS:ffff8801f39c0000(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000000002d8 CR3: 00000001d9602000 CR4: 0000000000002660
Call Trace:
handle_vcpu_hotplug_event+0xb5/0xc0
xenwatch_thread+0x80/0x140
? wait_woken+0x80/0x80
kthread+0x112/0x130
? kthread_create_worker_on_cpu+0x40/0x40
ret_from_fork+0x3a/0x50

This happens because handle_vcpu_hotplug_event is called twice. In the
first iteration cpu_present is still true, in the second iteration
cpu_present is false which causes get_cpu_device to return NULL.
In case of cpu#0, cpu_online is apparently always true.

Fix this crash by checking if the cpu can be hotplugged, which is false
for a cpu that was just removed.

Signed-off-by: Olaf Hering <[email protected]>
---
drivers/xen/cpu_hotplug.c | 2 ++
1 file changed, 2 insertions(+)

diff --git a/drivers/xen/cpu_hotplug.c b/drivers/xen/cpu_hotplug.c
index d4265c8ebb22..bf1e41ed9d41 100644
--- a/drivers/xen/cpu_hotplug.c
+++ b/drivers/xen/cpu_hotplug.c
@@ -19,6 +19,8 @@ static void enable_hotplug_cpu(int cpu)

static void disable_hotplug_cpu(int cpu)
{
+ if (!cpu_is_hotpluggable(cpu))
+ return;
if (cpu_online(cpu)) {
lock_device_hotplug();
device_offline(get_cpu_device(cpu));


2018-09-06 21:15:07

by Boris Ostrovsky

[permalink] [raw]
Subject: Re: [PATCH v2] xen: avoid crash in disable_hotplug_cpu

On 09/06/2018 02:37 AM, Olaf Hering wrote:
> The command 'xl vcpu-set 0 0', issued in dom0, will crash dom0:
>
> BUG: unable to handle kernel NULL pointer dereference at 00000000000002d8
> PGD 0 P4D 0
> Oops: 0000 [#1] PREEMPT SMP NOPTI
> CPU: 7 PID: 65 Comm: xenwatch Not tainted 4.19.0-rc2-1.ga9462db-default #1 openSUSE Tumbleweed (unreleased)
> Hardware name: Intel Corporation S5520UR/S5520UR, BIOS S5500.86B.01.00.0050.050620101605 05/06/2010
> RIP: e030:device_offline+0x9/0xb0
> Code: 77 24 00 e9 ce fe ff ff 48 8b 13 e9 68 ff ff ff 48 8b 13 e9 29 ff ff ff 48 8b 13 e9 ea fe ff ff 90 66 66 66 66 90 41 54 55 53 <f6> 87 d8 02 00 00 01 0f 85 88 00 00 00 48 c7 c2 20 09 60 81 31 f6
> RSP: e02b:ffffc90040f27e80 EFLAGS: 00010203
> RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
> RDX: ffff8801f3800000 RSI: ffffc90040f27e70 RDI: 0000000000000000
> RBP: 0000000000000000 R08: ffffffff820e47b3 R09: 0000000000000000
> R10: 0000000000007ff0 R11: 0000000000000000 R12: ffffffff822e6d30
> R13: dead000000000200 R14: dead000000000100 R15: ffffffff8158b4e0
> FS: 00007ffa595158c0(0000) GS:ffff8801f39c0000(0000) knlGS:0000000000000000
> CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00000000000002d8 CR3: 00000001d9602000 CR4: 0000000000002660
> Call Trace:
> handle_vcpu_hotplug_event+0xb5/0xc0
> xenwatch_thread+0x80/0x140
> ? wait_woken+0x80/0x80
> kthread+0x112/0x130
> ? kthread_create_worker_on_cpu+0x40/0x40
> ret_from_fork+0x3a/0x50
>
> This happens because handle_vcpu_hotplug_event is called twice. In the
> first iteration cpu_present is still true, in the second iteration
> cpu_present is false which causes get_cpu_device to return NULL.
> In case of cpu#0, cpu_online is apparently always true.
>
> Fix this crash by checking if the cpu can be hotplugged, which is false
> for a cpu that was just removed.
>
> Signed-off-by: Olaf Hering <[email protected]>
> ---
> drivers/xen/cpu_hotplug.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/drivers/xen/cpu_hotplug.c b/drivers/xen/cpu_hotplug.c
> index d4265c8ebb22..bf1e41ed9d41 100644
> --- a/drivers/xen/cpu_hotplug.c
> +++ b/drivers/xen/cpu_hotplug.c
> @@ -19,6 +19,8 @@ static void enable_hotplug_cpu(int cpu)
>
> static void disable_hotplug_cpu(int cpu)
> {
> + if (!cpu_is_hotpluggable(cpu))


I think we should check both this and num_online_cpus() != 0.

Even though I don't believe cpu0_hotpluggable currently works, at some
point it might.

-boris


> + return;
> if (cpu_online(cpu)) {
> lock_device_hotplug();
> device_offline(get_cpu_device(cpu));


2018-09-06 21:27:51

by Olaf Hering

[permalink] [raw]
Subject: Re: [PATCH v2] xen: avoid crash in disable_hotplug_cpu

Am Thu, 6 Sep 2018 14:45:57 -0400
schrieb Boris Ostrovsky <[email protected]>:

> On 09/06/2018 02:37 AM, Olaf Hering wrote:
> > The command 'xl vcpu-set 0 0', issued in dom0, will crash dom0:

> > This happens because handle_vcpu_hotplug_event is called twice. In the
> > first iteration cpu_present is still true, in the second iteration
> > cpu_present is false which causes get_cpu_device to return NULL.
> > In case of cpu#0, cpu_online is apparently always true.

> I think we should check both this and num_online_cpus() != 0.

This can not possibly help. cpu#0 is the first one that goes offline.
IF cpu0_hotpluggable is broken, then only "if (!cpu) return;" can help.

Olaf


Attachments:
(No filename) (201.00 B)
Digitale Signatur von OpenPGP

2018-09-06 22:53:51

by Boris Ostrovsky

[permalink] [raw]
Subject: Re: [PATCH v2] xen: avoid crash in disable_hotplug_cpu

On 09/06/2018 04:31 PM, Olaf Hering wrote:
> Am Thu, 6 Sep 2018 14:45:57 -0400
> schrieb Boris Ostrovsky <[email protected]>:
>
>> On 09/06/2018 02:37 AM, Olaf Hering wrote:
>>> The command 'xl vcpu-set 0 0', issued in dom0, will crash dom0:
>>> This happens because handle_vcpu_hotplug_event is called twice. In the
>>> first iteration cpu_present is still true, in the second iteration
>>> cpu_present is false which causes get_cpu_device to return NULL.
>>> In case of cpu#0, cpu_online is apparently always true.
>> I think we should check both this and num_online_cpus() != 0.
> This can not possibly help. cpu#0 is the first one that goes offline.
> IF cpu0_hotpluggable is broken, then only "if (!cpu) return;" can help.


And maybe that needs to be part of the check, in addition to
cpu_is_hotpluggable() test.

Offlining CPU0 is problematic. For example, look at xen_pv_cpu_disable().



-boris


Attachments:
signature.asc (849.00 B)
OpenPGP digital signature

2018-09-07 05:19:22

by Juergen Gross

[permalink] [raw]
Subject: Re: [PATCH v2] xen: avoid crash in disable_hotplug_cpu

On 06/09/18 22:31, Olaf Hering wrote:
> Am Thu, 6 Sep 2018 14:45:57 -0400
> schrieb Boris Ostrovsky <[email protected]>:
>
>> On 09/06/2018 02:37 AM, Olaf Hering wrote:
>>> The command 'xl vcpu-set 0 0', issued in dom0, will crash dom0:
>
>>> This happens because handle_vcpu_hotplug_event is called twice. In the
>>> first iteration cpu_present is still true, in the second iteration
>>> cpu_present is false which causes get_cpu_device to return NULL.
>>> In case of cpu#0, cpu_online is apparently always true.
>
>> I think we should check both this and num_online_cpus() != 0.
>
> This can not possibly help. cpu#0 is the first one that goes offline.
> IF cpu0_hotpluggable is broken, then only "if (!cpu) return;" can help.

We should add the Xen PV guest test to arch_register_cpu() and switch
cpu0_hotpluggable off.


Juergen

2018-09-07 05:47:55

by Olaf Hering

[permalink] [raw]
Subject: Re: [PATCH v2] xen: avoid crash in disable_hotplug_cpu

Am Thu, 6 Sep 2018 22:31:45 +0200
schrieb Olaf Hering <[email protected]>:

> IF cpu0_hotpluggable is broken, then only "if (!cpu) return;" can help.

Another option is to do cpu_online() twice to check if device_offline did anything.
Not sure if the compiler would fold the two checks into a single check.

Olaf


Attachments:
(No filename) (201.00 B)
Digitale Signatur von OpenPGP