Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Sat, 31 Jul 2021 10:52:28 +0100
Message-ID: <87tuka24kj.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Sunil Muthuswamy <sunilmut@microsoft.com>
Cc:     Thomas Gleixner <tglx@linutronix.de>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "catalin.marinas@arm.com" <catalin.marinas@arm.com>,
        "will@kernel.org" <will@kernel.org>,
        Michael Kelley <mikelley@microsoft.com>,
        Boqun Feng <Boqun.Feng@microsoft.com>,
        KY Srinivasan <kys@microsoft.com>,
        Arnd Bergmann <arnd@arndb.de>
Subject: Re: [EXTERNAL] Re: [RFC 1/1] irqchip/gic-v3-its: Add irq domain and chip for Direct LPI without ITS
In-Reply-To: <MW4PR21MB2002E51429F7E13B61A34FFEC0E89@MW4PR21MB2002.namprd21.prod.outlook.com>
References: <MW4PR21MB2002BA20FA4A8DAA1C27F4C6C0199@MW4PR21MB2002.namprd21.prod.outlook.com>
        <87a6mt2jke.wl-maz@kernel.org>
        <MW4PR21MB2002E51429F7E13B61A34FFEC0E89@MW4PR21MB2002.namprd21.prod.outlook.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1
 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk

On Mon, 26 Jul 2021 16:33:39 +0100,
Sunil Muthuswamy <sunilmut@microsoft.com> wrote:

[...]

> > I also want to understand *how* you are going to plumb this into both
> > ACPI and DT, given that neither understand how to link a PCI endpoint
> > to a set of RDs.
> > 
> > 	M.
> 
> One way to do this for NUMA-aware systems would be to use the NUMA
> related information that is available with PCI endpoints or root complex, to
> pick a Redistributor/CPU that is in the NUMA node, as specified by the PCI
> endpoint/root complex. In DT PCI devices can specify this using
> 'numa-node-id' and in ACPI using the '_PXM (Proximity)'. For systems that
> are not NUMA-aware, we can go with *any* Redistributor/CPU.

This makes zero sense. From the point of view of a device, all the RDs
should be reachable, and firmware has no say in it. Dealing with
interrupt affinity is the responsibility of the endpoint driver, and
NUMA affinity is only a performance optimisation.

> Is there any additional information we would be able to gather from ACPI
> or DT that's not there currently, that would be useful here?

You will need some firmware information describing that a given set of
devices must use the RDs for their MSIs. Just like we currently
describe it in IORT for the ITS. You cannot /assume/ things. At the
moment, there is nothing at all, because no-one (including Microsoft)
thought it would be a good idea not to have an ITS, which is also why
ACPI doesn't describe MBIs as a potential MSI provider.

> The other question is, what DT property can be used to instantiate the
> PCI-MSI IRQ domain for Direct LPI? As per the DT spec, there is only
> 'msi-controller' Sub-node for the ITS. 

Read again. We already a msi-controller property in the main GICv3
node for the purpose of MBIs. You can reuse this property, but you
will have to discriminate whether you want MBIs or DirectLPIs with
some extra property.

	M.

-- 
Without deviation from the norm, progress is not possible.