Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Mon, 09 Aug 2021 10:15:15 +0100
Message-ID: <87pmunasik.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Sunil Muthuswamy <sunilmut@microsoft.com>
Cc:     Robin Murphy <robin.murphy@arm.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "catalin.marinas@arm.com" <catalin.marinas@arm.com>,
        "will@kernel.org" <will@kernel.org>,
        Michael Kelley <mikelley@microsoft.com>,
        Boqun Feng <Boqun.Feng@microsoft.com>,
        KY Srinivasan <kys@microsoft.com>,
        Arnd Bergmann <arnd@arndb.de>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Subject: Re: [EXTERNAL] Re: [RFC 1/1] irqchip/gic-v3-its: Add irq domain and chip for Direct LPI without ITS
In-Reply-To: <MW4PR21MB200286E2CF4DA229D7059598C0F69@MW4PR21MB2002.namprd21.prod.outlook.com>
References: <MW4PR21MB2002BA20FA4A8DAA1C27F4C6C0199@MW4PR21MB2002.namprd21.prod.outlook.com>
        <87a6mt2jke.wl-maz@kernel.org>
        <MW4PR21MB2002E51429F7E13B61A34FFEC0E89@MW4PR21MB2002.namprd21.prod.outlook.com>
        <87tuka24kj.wl-maz@kernel.org>
        <MW4PR21MB20027EAC23E8053210364E2BC0F09@MW4PR21MB2002.namprd21.prod.outlook.com>
        <b2dea108-9166-dc9d-abd0-d22491f78568@arm.com>
        <87r1f9wooc.wl-maz@kernel.org>
        <MW4PR21MB2002860BAED850B2B52D6E2BC0F19@MW4PR21MB2002.namprd21.prod.outlook.com>
        <87wnp0b86y.wl-maz@kernel.org>
        <MW4PR21MB2002ABD170CC9786CDF5D493C0F39@MW4PR21MB2002.namprd21.prod.outlook.com>
        <87o8a81boi.wl-maz@kernel.org>
        <MW4PR21MB200286E2CF4DA229D7059598C0F69@MW4PR21MB2002.namprd21.prod.outlook.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1
 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk

On Mon, 09 Aug 2021 03:35:05 +0100,
Sunil Muthuswamy <sunilmut@microsoft.com> wrote:
> 
> Sunday, August 8, 2021 3:19 AM,
> Marc Zyngier <maz@kernel.org> wrote:
> > 
> > [+Lorenzo, as he deals with both PCI and ACPI]
> > 
> > I think it is clear enough, but I don't see what this buys us other
> > than turning DirectLPI into something that is essentially private to
> > Hyper-V, just for the sake of sidestepping a firmware description.
> > Furthermore, the Hyper-V "single MSI address" model doesn't match the
> > way DirectLPI works (after all, this is one of the many reasons why we
> > have the ITS).
> > 
> > The architecture already gives you everything you need to make things
> > work in Hyper-V. You can easily implement the DirectLPI support (you
> > definitely need most of it anyway), and the PCI-MSI layer is
> > *free*. All you need is a firmware description. Which I don't believe
> > is the end of the world, given that DT is freely hackable and that
> > IORT is an ARM-private extension to ACPI. I'm sure you know who to
> > reach out to in order to start a draft (and if you don't, I can give
> > you names offline).
> > 
> That sounds reasonable. The DT extension here alone wont suffice for
> Hyper-V and we would need the ACPI IORT extension here as well. Our
> general experience with ACPI extension is that they can take time. But,
> I am curious to hear from Lorenzo on his thoughts here and if just an
> IORT extension means anything else in terms of timelines.
> 
> > I am not interested in expanding the GICv3 architecture support if it
> > cannot be generally used in a way that is *compliant with the
> > architecture*. If you think DirectLPIs can make the hypervisor
> > simpler, I'm fine with it. But let's fully embrace the concept instead
> > of hiding it behind a layer of PV muck. It will even have a chance of
> > working on bare metal (such as on the machine that is humming behind
> > me, or even the Fast Model).
> > 
> Appreciate your inputs. Since we are discussing options, there is one more
> option to enable Hyper-V virtual PCI for ARM64 that I do would like to
> discuss. That option is to implement the IRQ chip completely within the
> Hyper-V module itself. That IRQ chip will take care of allocating LPI vectors,
> enabling the interrupt (unmask, mask etc..). It won't be a Direct LPI based,
> but based on custom Hyper-V methods. That chip will parent itself to the
> GIC v3 IRQ domain. The only extra thing needed for this is for the Hyper-V
> IRQ chip to be able to discover the GIC v3 IRQ domain, for which it
> cannot use the 'irq_find_matching_fwnode' method as Hyper-V virtual
> PCI bus/devices are not firmware enumerated. I am not sure if there is any
> other way to discover the GIC (DOMAIN_BUS_WIRED) domain.
>
> What are your thoughts/feedback on the above? This will allow us to
> enable the scenario for the business timelines we are targeting for, while
> we wait for firmware spec updates. Long term we also want to use
> architectural methods here as that helps the Hypervisor as well, and I
> would be glad to pursue the firmware spec updates in parallel.

This feels like yet another layer of PV ugliness that has the side
effect of fragmenting the architectural support.

If you plug directly into the GICv3 layer, I'd rather you inject SPIs,
just like any other non-architectural MSI controller. You can directly
interface with the ACPI GSI layer for that, without any need to mess
with the GICv3 internals. The SPI space isn't very large, but still
much larger than the equivalent x86 space (close to 1000).

If time is of the essence, I suggest you go the SPI way. For anything
involving LPIs, I really want to see a firmware spec that works for
everyone as opposed to a localised Hyper-V hack.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.