2023-04-04 11:40:19

by Jinank Jain

Subject: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

In case of nested MSHV, retargeting interrupt hypercall should be sent
to L0 hypervisor instead of L1 hypervisor.

Signed-off-by: Jinank Jain <[email protected]>
---
drivers/pci/controller/pci-hyperv.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index f33370b75628..2123f632ecf7 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -704,8 +704,14 @@ static void hv_arch_irq_unmask(struct irq_data *data)
}
}

- res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT | (var_size << 17),
- params, NULL);
+ if (hv_nested)
+ res = hv_do_nested_hypercall(HVCALL_RETARGET_INTERRUPT |
+ (var_size << 17),
+ params, NULL);
+ else
+ res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT |
+ (var_size << 17),
+ params, NULL);

exit_unlock:
spin_unlock_irqrestore(&hbus->retarget_msi_interrupt_lock, flags);
--
2.34.1

2023-04-04 17:06:07

by Nuno Das Neves

Subject: Re: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

Reviewed-by: Nuno Das Neves <[email protected]>

On 4/4/2023 4:35 AM, Jinank Jain wrote:
> In case of nested MSHV, retargeting interrupt hypercall should be sent
> to L0 hypervisor instead of L1 hypervisor.
>
> Signed-off-by: Jinank Jain <[email protected]>
> ---
> drivers/pci/controller/pci-hyperv.c | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
> index f33370b75628..2123f632ecf7 100644
> --- a/drivers/pci/controller/pci-hyperv.c
> +++ b/drivers/pci/controller/pci-hyperv.c
> @@ -704,8 +704,14 @@ static void hv_arch_irq_unmask(struct irq_data *data)
> }
> }
>
> - res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT | (var_size << 17),
> - params, NULL);
> + if (hv_nested)
> + res = hv_do_nested_hypercall(HVCALL_RETARGET_INTERRUPT |
> + (var_size << 17),
> + params, NULL);
> + else
> + res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT |
> + (var_size << 17),
> + params, NULL);
>
> exit_unlock:
> spin_unlock_irqrestore(&hbus->retarget_msi_interrupt_lock, flags);

2023-04-05 23:46:00

by Wei Liu

Subject: Re: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

On Tue, Apr 04, 2023 at 11:35:46AM +0000, Jinank Jain wrote:
> In case of nested MSHV, retargeting interrupt hypercall should be sent
> to L0 hypervisor instead of L1 hypervisor.
>
> Signed-off-by: Jinank Jain <[email protected]>

While I think this is a sensible change -- how did you discover this?
Can you provide a bit more information?

> ---
> drivers/pci/controller/pci-hyperv.c | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
> index f33370b75628..2123f632ecf7 100644
> --- a/drivers/pci/controller/pci-hyperv.c
> +++ b/drivers/pci/controller/pci-hyperv.c
> @@ -704,8 +704,14 @@ static void hv_arch_irq_unmask(struct irq_data *data)
> }
> }
>
> - res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT | (var_size << 17),
> - params, NULL);
> + if (hv_nested)
> + res = hv_do_nested_hypercall(HVCALL_RETARGET_INTERRUPT |
> + (var_size << 17),
> + params, NULL);
> + else
> + res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT |
> + (var_size << 17),
> + params, NULL);
>
> exit_unlock:
> spin_unlock_irqrestore(&hbus->retarget_msi_interrupt_lock, flags);
> --
> 2.34.1
>

2023-04-06 06:12:58

by Jinank Jain

Subject: Re: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

This was observed while testing pass-through PCI devices on the nested
MSHV setup.

Regards,

Jinank

On 4/6/2023 4:47 AM, Wei Liu wrote:
> On Tue, Apr 04, 2023 at 11:35:46AM +0000, Jinank Jain wrote:
>> In case of nested MSHV, retargeting interrupt hypercall should be sent
>> to L0 hypervisor instead of L1 hypervisor.
>>
>> Signed-off-by: Jinank Jain <[email protected]>
> While I think this is a sensible change -- how did you discover this?
> Can you provide a bit more information?
>
>> ---
>> drivers/pci/controller/pci-hyperv.c | 10 ++++++++--
>> 1 file changed, 8 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
>> index f33370b75628..2123f632ecf7 100644
>> --- a/drivers/pci/controller/pci-hyperv.c
>> +++ b/drivers/pci/controller/pci-hyperv.c
>> @@ -704,8 +704,14 @@ static void hv_arch_irq_unmask(struct irq_data *data)
>> }
>> }
>>
>> - res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT | (var_size << 17),
>> - params, NULL);
>> + if (hv_nested)
>> + res = hv_do_nested_hypercall(HVCALL_RETARGET_INTERRUPT |
>> + (var_size << 17),
>> + params, NULL);
>> + else
>> + res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT |
>> + (var_size << 17),
>> + params, NULL);
>>
>> exit_unlock:
>> spin_unlock_irqrestore(&hbus->retarget_msi_interrupt_lock, flags);
>> --
>> 2.34.1
>>

2023-04-13 01:24:10

by Wei Liu

Subject: Re: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

On Tue, Apr 04, 2023 at 11:35:46AM +0000, Jinank Jain wrote:
> In case of nested MSHV, retargeting interrupt hypercall should be sent
> to L0 hypervisor instead of L1 hypervisor.
>
> Signed-off-by: Jinank Jain <[email protected]>

Applied to hyperv-next. Thanks.

2023-04-13 03:16:36

by Michael Kelley (LINUX)

Subject: RE: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

From: Wei Liu <[email protected]> Sent: Wednesday, April 12, 2023 6:23 PM
>
> On Tue, Apr 04, 2023 at 11:35:46AM +0000, Jinank Jain wrote:
> > In case of nested MSHV, retargeting interrupt hypercall should be sent
> > to L0 hypervisor instead of L1 hypervisor.
> >
> > Signed-off-by: Jinank Jain <[email protected]>
>
> Applied to hyperv-next. Thanks.

I'd like to hold off on taking this change. Nuno and I are discussing
how best to handle nested hypercalls. In addition to the proposed
nested changes, we have hypercall changes coming as part of the
TDX and fully enlightened SNP patch sets. If possible, I'd like to
avoid adding logic at the hv_do_hypercall() call sites. It's not clear
whether avoiding such logic will really be feasible, but I'd like to
think about it for a bit before reaching that conclusion.

Michael

2023-04-13 15:38:51

by Wei Liu

Subject: Re: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

On Thu, Apr 13, 2023 at 03:05:09AM +0000, Michael Kelley (LINUX) wrote:
> From: Wei Liu <[email protected]> Sent: Wednesday, April 12, 2023 6:23 PM
> >
> > On Tue, Apr 04, 2023 at 11:35:46AM +0000, Jinank Jain wrote:
> > > In case of nested MSHV, retargeting interrupt hypercall should be sent
> > > to L0 hypervisor instead of L1 hypervisor.
> > >
> > > Signed-off-by: Jinank Jain <[email protected]>
> >
> > Applied to hyperv-next. Thanks.
>
> I'd like to hold off on taking this change. Nuno and I are discussing
> how best to handle nested hypercalls. In addition to the proposed
> nested changes, we have hypercall changes coming as part of the
> TDX and fully enlightened SNP patch sets. If possible, I'd like to
> avoid adding logic at the hv_do_hypercall() call sites. It's not clear
> whether avoiding such logic will really be feasible, but I'd like to
> think about it for a bit before reaching that conclusion.

I thought that discussion will go on for a while but this patch fixed a
real bug.

Holding off is fine too. I will remove this patch from hyperv-next.

Thanks,
Wei.

>
> Michael
>