Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Sun, 08 Aug 2021 11:19:09 +0100
Message-ID: <87o8a81boi.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Sunil Muthuswamy <sunilmut@microsoft.com>
Cc:     Robin Murphy <robin.murphy@arm.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "catalin.marinas@arm.com" <catalin.marinas@arm.com>,
        "will@kernel.org" <will@kernel.org>,
        Michael Kelley <mikelley@microsoft.com>,
        Boqun Feng <Boqun.Feng@microsoft.com>,
        KY Srinivasan <kys@microsoft.com>,
        Arnd Bergmann <arnd@arndb.de>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Subject: Re: [EXTERNAL] Re: [RFC 1/1] irqchip/gic-v3-its: Add irq domain and chip for Direct LPI without ITS
In-Reply-To: <MW4PR21MB2002ABD170CC9786CDF5D493C0F39@MW4PR21MB2002.namprd21.prod.outlook.com>
References: <MW4PR21MB2002BA20FA4A8DAA1C27F4C6C0199@MW4PR21MB2002.namprd21.prod.outlook.com>
        <87a6mt2jke.wl-maz@kernel.org>
        <MW4PR21MB2002E51429F7E13B61A34FFEC0E89@MW4PR21MB2002.namprd21.prod.outlook.com>
        <87tuka24kj.wl-maz@kernel.org>
        <MW4PR21MB20027EAC23E8053210364E2BC0F09@MW4PR21MB2002.namprd21.prod.outlook.com>
        <b2dea108-9166-dc9d-abd0-d22491f78568@arm.com>
        <87r1f9wooc.wl-maz@kernel.org>
        <MW4PR21MB2002860BAED850B2B52D6E2BC0F19@MW4PR21MB2002.namprd21.prod.outlook.com>
        <87wnp0b86y.wl-maz@kernel.org>
        <MW4PR21MB2002ABD170CC9786CDF5D493C0F39@MW4PR21MB2002.namprd21.prod.outlook.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1
 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk

On Fri, 06 Aug 2021 20:14:13 +0100,
Sunil Muthuswamy <sunilmut@microsoft.com> wrote:
> 
> On Thursday, August 5, 2021 1:35 AM,
> Marc Zyngier <maz@kernel.org> wrote:
> [...]
> > > Hey Mark
> > 
> > I assume this is for me?
> > 
> Yes, sorry.
> 
> > > would you be willing to consider a scoped down implementation of GIC
> > > Direct LPI with just an IRQ chip implementation and no Direct LPI
> > > PCI-MSI IRQ chip.
> > 
> > Could you please clarify? If you are not implementing MSIs, how can a
> > device signal LPIs? At the end of the day, something has to write into
> > the RD, and it isn't going to happen by sheer magic.
> > 
> > > This will allow a MSI provider (such as Hyper-V vPCI) to provide a
> > > PCI-MSI IRQ chip on top of the Direct LPI IRQ chip and enable
> > > PCI-MSI scenarios, and avoid building in assumptions in other cases
> > > (like PCI) where firmware specification is not available.
> > 
> > I really don't get what you are suggesting. Could you please describe
> > what you have in mind?
> > 
> The suggestion was to *not* implement the PCI-MSI IRQ chip in the
> Direct LPI GIC implementation, but let the endpoint/specific
> implementation provide for the MSI IRQ chip.
> 
> For example, the Hyper-V vPCI module does implement a PCI-MSI IRQ
> chip (refer to 'hv_pcie_init_irq_domain'). Microsoft Hypervisor
> provides/virtualizes the MSI address to be used for Hyper-V vPCI. The
> Hypervisor might be using the ITS in the background, but it expects
> the device to be programmed with the MSI address that it provides.
> And, the Hypervisor takes care of routing the interrupt to the guest.
> This is done by the Hyper-V vPCI MSI IRQ chip (refer to
> hv_msi_irq_chip) compose MSI message.
> 
> Today, for X64, Hyper-V vPCI PCI-MSI irq chip parents itself to the
> architectural 'x86_vector_domain'. The suggestion here was to see
> if we can support a similar setup for ARM, where the Hyper-V vPCI
> PCI-MSI irq chip can parent itself to the 'Direct LPI arch IRQ chip'
> (to be implemented).
> 
> The arch Direct LPI arch IRQ chip is needed to enable LPIs, invalidate
> the LPI configuration (GICR_INVLPIR et. al. ) and allocate LPI IRQs from
> the LPI interrupt namespace (i.e. 8192-<num LPIS>).
> 
> Happy to expand on this further, if the above is not clear.

[+Lorenzo, as he deals with both PCI and ACPI]

I think it is clear enough, but I don't see what this buys us other
than turning DirectLPI into something that is essentially private to
Hyper-V, just for the sake of sidestepping a firmware description.
Furthermore, the Hyper-V "single MSI address" model doesn't match the
way DirectLPI works (after all, this is one of the many reasons why we
have the ITS).

The architecture already gives you everything you need to make things
work in Hyper-V. You can easily implement the DirectLPI support (you
definitely need most of it anyway), and the PCI-MSI layer is
*free*. All you need is a firmware description. Which I don't believe
is the end of the world, given that DT is freely hackable and that
IORT is an ARM-private extension to ACPI. I'm sure you know who to
reach out to in order to start a draft (and if you don't, I can give
you names offline).

I am not interested in expanding the GICv3 architecture support if it
cannot be generally used in a way that is *compliant with the
architecture*. If you think DirectLPIs can make the hypervisor
simpler, I'm fine with it. But let's fully embrace the concept instead
of hiding it behind a layer of PV muck. It will even have a chance of
working on bare metal (such as on the machine that is humming behind
me, or even the Fast Model).

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.