2020-09-14 12:29:40

by Wei Liu

Subject: [PATCH RFC v1 10/18] x86/hyperv: implement and use hv_smp_prepare_cpus

Microsoft Hypervisor requires the root partition to make a few
hypercalls to setup application processors before they can be used.

Signed-off-by: Lillian Grassin-Drake <[email protected]>
Signed-off-by: Sunil Muthuswamy <[email protected]>
Co-Developed-by: Lillian Grassin-Drake <[email protected]>
Co-Developed-by: Sunil Muthuswamy <[email protected]>
Signed-off-by: Wei Liu <[email protected]>
---
CPU hotplug and unplug is not yet supported in this setup, so those
paths remain untouched.
---
arch/x86/kernel/cpu/mshyperv.c | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)

diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 1bf57d310f78..7522cae02759 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -203,6 +203,31 @@ static void __init hv_smp_prepare_boot_cpu(void)
hv_init_spinlocks();
#endif
}
+
+static void __init hv_smp_prepare_cpus(unsigned int max_cpus)
+{
+#if defined(CONFIG_X86_64)
+ int i;
+ int vp_index = 1;
+ int ret;
+
+ native_smp_prepare_cpus(max_cpus);
+
+ for_each_present_cpu(i) {
+ if (i == 0)
+ continue;
+ ret = hv_call_add_logical_proc(numa_cpu_node(i), i, cpu_physical_id(i));
+ BUG_ON(ret);
+ }
+
+ for_each_present_cpu(i) {
+ if (i == 0)
+ continue;
+ ret = hv_call_create_vp(numa_cpu_node(i), hv_current_partition_id, vp_index++, i);
+ BUG_ON(ret);
+ }
+#endif
+}
#endif

static void __init ms_hyperv_init_platform(void)
@@ -359,6 +384,8 @@ static void __init ms_hyperv_init_platform(void)

# ifdef CONFIG_SMP
smp_ops.smp_prepare_boot_cpu = hv_smp_prepare_boot_cpu;
+ if (hv_root_partition)
+ smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus;
# endif

/*
--
2.20.1

2020-09-16 00:57:58

by Vitaly Kuznetsov

Subject: Re: [PATCH RFC v1 10/18] x86/hyperv: implement and use hv_smp_prepare_cpus

Wei Liu <[email protected]> writes:

> Microsoft Hypervisor requires the root partition to make a few
> hypercalls to setup application processors before they can be used.
>
> Signed-off-by: Lillian Grassin-Drake <[email protected]>
> Signed-off-by: Sunil Muthuswamy <[email protected]>
> Co-Developed-by: Lillian Grassin-Drake <[email protected]>
> Co-Developed-by: Sunil Muthuswamy <[email protected]>
> Signed-off-by: Wei Liu <[email protected]>
> ---
> CPU hotplug and unplug is not yet supported in this setup, so those
> paths remain untouched.
> ---
> arch/x86/kernel/cpu/mshyperv.c | 27 +++++++++++++++++++++++++++
> 1 file changed, 27 insertions(+)
>
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> index 1bf57d310f78..7522cae02759 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -203,6 +203,31 @@ static void __init hv_smp_prepare_boot_cpu(void)
> hv_init_spinlocks();
> #endif
> }
> +
> +static void __init hv_smp_prepare_cpus(unsigned int max_cpus)
> +{
> +#if defined(CONFIG_X86_64)

I think it makes little sense to try to make Linux work as Hyper-V root
partition when !CONFIG_X86_64. If we still care about Hyper-V enablement
for !CONFIG_X86_64 we can probably introduce something like
CONFIG_HYPERV_ROOT and enable it automatically, e.g.

config HYPERV_ROOT
def_bool HYPERV && X86_64

and use it instead.

> + int i;
> + int vp_index = 1;
> + int ret;
> +
> + native_smp_prepare_cpus(max_cpus);
> +
> + for_each_present_cpu(i) {
> + if (i == 0)
> + continue;
> + ret = hv_call_add_logical_proc(numa_cpu_node(i), i, cpu_physical_id(i));
> + BUG_ON(ret);
> + }
> +
> + for_each_present_cpu(i) {
> + if (i == 0)
> + continue;
> + ret = hv_call_create_vp(numa_cpu_node(i), hv_current_partition_id, vp_index++, i);

So vp_index variable is needed here to make sure there are no gaps? (or
we could've just used 'i')?

> + BUG_ON(ret);
> + }
> +#endif
> +}
> #endif
>
> static void __init ms_hyperv_init_platform(void)
> @@ -359,6 +384,8 @@ static void __init ms_hyperv_init_platform(void)
>
> # ifdef CONFIG_SMP
> smp_ops.smp_prepare_boot_cpu = hv_smp_prepare_boot_cpu;
> + if (hv_root_partition)
> + smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus;
> # endif
>
> /*

--
Vitaly

2020-10-27 14:57:27

by Wei Liu

Subject: Re: [PATCH RFC v1 10/18] x86/hyperv: implement and use hv_smp_prepare_cpus

On Tue, Oct 27, 2020 at 01:47:57PM +0000, Wei Liu wrote:
> On Tue, Sep 15, 2020 at 01:14:16PM +0200, Vitaly Kuznetsov wrote:
> > Wei Liu <[email protected]> writes:
> >
> > > Microsoft Hypervisor requires the root partition to make a few
> > > hypercalls to setup application processors before they can be used.
> > >
> > > Signed-off-by: Lillian Grassin-Drake <[email protected]>
> > > Signed-off-by: Sunil Muthuswamy <[email protected]>
> > > Co-Developed-by: Lillian Grassin-Drake <[email protected]>
> > > Co-Developed-by: Sunil Muthuswamy <[email protected]>
> > > Signed-off-by: Wei Liu <[email protected]>
> > > ---
> > > CPU hotplug and unplug is not yet supported in this setup, so those
> > > paths remain untouched.
> > > ---
> > > arch/x86/kernel/cpu/mshyperv.c | 27 +++++++++++++++++++++++++++
> > > 1 file changed, 27 insertions(+)
> > >
> > > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > > index 1bf57d310f78..7522cae02759 100644
> > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > @@ -203,6 +203,31 @@ static void __init hv_smp_prepare_boot_cpu(void)
> > > hv_init_spinlocks();
> > > #endif
> > > }
> > > +
> > > +static void __init hv_smp_prepare_cpus(unsigned int max_cpus)
> > > +{
> > > +#if defined(CONFIG_X86_64)
> >
> > I think it makes little sense to try to make Linux work as Hyper-V root
> > partition when !CONFIG_X86_64. If we still care about Hyper-V enablement
> > for !CONFIG_X86_64 we can probably introduce something like
> > CONFIG_HYPERV_ROOT and enable it automatically, e.g.
> >
> > config HYPERV_ROOT
> > def_bool HYPERV && X86_64
> >
> > and use it instead.
> >
>
> We have a patch for such a config option in the /dev/mshv patch set. But
> that's not yet included here so I will keep this as-is.
>
> > > + int i;
> > > + int vp_index = 1;
> > > + int ret;
> > > +
> > > + native_smp_prepare_cpus(max_cpus);
> > > +
> > > + for_each_present_cpu(i) {
> > > + if (i == 0)
> > > + continue;
> > > + ret = hv_call_add_logical_proc(numa_cpu_node(i), i, cpu_physical_id(i));
> > > + BUG_ON(ret);
> > > + }
> > > +
> > > + for_each_present_cpu(i) {
> > > + if (i == 0)
> > > + continue;
> > > + ret = hv_call_create_vp(numa_cpu_node(i), hv_current_partition_id, vp_index++, i);
> >
> > So vp_index variable is needed here to make sure there are no gaps? (or
> > we could've just used 'i')?
>
> Not sure. I didn't write the original code in this function. The last
> argument (i) is the logical processor index.
>
> I don't see a reason why vp_index and lp_index can't be the same. I will
> try dropping vp_index. If that works then great; if not, I will keep the
> code as-is.
>
> Sunil, if you have more insight, please chime in.
>

A quick test shows that replacing vp_index with i works just fine.

Wei.

2020-10-28 07:54:05

by Wei Liu

Subject: Re: [PATCH RFC v1 10/18] x86/hyperv: implement and use hv_smp_prepare_cpus

On Tue, Sep 15, 2020 at 01:14:16PM +0200, Vitaly Kuznetsov wrote:
> Wei Liu <[email protected]> writes:
>
> > Microsoft Hypervisor requires the root partition to make a few
> > hypercalls to setup application processors before they can be used.
> >
> > Signed-off-by: Lillian Grassin-Drake <[email protected]>
> > Signed-off-by: Sunil Muthuswamy <[email protected]>
> > Co-Developed-by: Lillian Grassin-Drake <[email protected]>
> > Co-Developed-by: Sunil Muthuswamy <[email protected]>
> > Signed-off-by: Wei Liu <[email protected]>
> > ---
> > CPU hotplug and unplug is not yet supported in this setup, so those
> > paths remain untouched.
> > ---
> > arch/x86/kernel/cpu/mshyperv.c | 27 +++++++++++++++++++++++++++
> > 1 file changed, 27 insertions(+)
> >
> > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > index 1bf57d310f78..7522cae02759 100644
> > --- a/arch/x86/kernel/cpu/mshyperv.c
> > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > @@ -203,6 +203,31 @@ static void __init hv_smp_prepare_boot_cpu(void)
> > hv_init_spinlocks();
> > #endif
> > }
> > +
> > +static void __init hv_smp_prepare_cpus(unsigned int max_cpus)
> > +{
> > +#if defined(CONFIG_X86_64)
>
> I think it makes little sense to try to make Linux work as Hyper-V root
> partition when !CONFIG_X86_64. If we still care about Hyper-V enablement
> for !CONFIG_X86_64 we can probably introduce something like
> CONFIG_HYPERV_ROOT and enable it automatically, e.g.
>
> config HYPERV_ROOT
> def_bool HYPERV && X86_64
>
> and use it instead.
>

We have a patch for such a config option in the /dev/mshv patch set. But
that's not yet included here so I will keep this as-is.

> > + int i;
> > + int vp_index = 1;
> > + int ret;
> > +
> > + native_smp_prepare_cpus(max_cpus);
> > +
> > + for_each_present_cpu(i) {
> > + if (i == 0)
> > + continue;
> > + ret = hv_call_add_logical_proc(numa_cpu_node(i), i, cpu_physical_id(i));
> > + BUG_ON(ret);
> > + }
> > +
> > + for_each_present_cpu(i) {
> > + if (i == 0)
> > + continue;
> > + ret = hv_call_create_vp(numa_cpu_node(i), hv_current_partition_id, vp_index++, i);
>
> So vp_index variable is needed here to make sure there are no gaps? (or
> we could've just used 'i')?

Not sure. I didn't write the original code in this function. The last
argument (i) is the logical processor index.

I don't see a reason why vp_index and lp_index can't be the same. I will
try dropping vp_index. If that works then great; if not, I will keep the
code as-is.

Sunil, if you have more insight, please chime in.

Wei.

>
> > + BUG_ON(ret);
> > + }
> > +#endif
> > +}
> > #endif
> >
> > static void __init ms_hyperv_init_platform(void)
> > @@ -359,6 +384,8 @@ static void __init ms_hyperv_init_platform(void)
> >
> > # ifdef CONFIG_SMP
> > smp_ops.smp_prepare_boot_cpu = hv_smp_prepare_boot_cpu;
> > + if (hv_root_partition)
> > + smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus;
> > # endif
> >
> > /*
>
> --
> Vitaly
>