Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 5 Jan 2023 13:20:36 -0800
From:   Dan Williams <dan.j.williams@intel.com>
To:     Bjorn Helgaas <helgaas@kernel.org>,
        Dan Williams <dan.j.williams@intel.com>
CC:     "Luck, Tony" <tony.luck@intel.com>,
        "Liang, Kan" <kan.liang@linux.intel.com>,
        "bhelgaas@google.com" <bhelgaas@google.com>,
        "hdegoede@redhat.com" <hdegoede@redhat.com>,
        "kernelorg@undead.fr" <kernelorg@undead.fr>,
        "kjhambrick@gmail.com" <kjhambrick@gmail.com>,
        "2lprbe78@duck.com" <2lprbe78@duck.com>,
        "nicholas.johnson-opensource@outlook.com.au" 
        <nicholas.johnson-opensource@outlook.com.au>,
        "benoitg@coeus.ca" <benoitg@coeus.ca>,
        "mika.westerberg@linux.intel.com" <mika.westerberg@linux.intel.com>,
        "wse@tuxedocomputers.com" <wse@tuxedocomputers.com>,
        "mumblingdrunkard@protonmail.com" <mumblingdrunkard@protonmail.com>,
        "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Box, David E" <david.e.box@intel.com>,
        "Sun, Yunying" <yunying.sun@intel.com>
Subject: Re: Bug report: the extended PCI config space is missed with 6.2-rc2
Message-ID: <63b73f24c1a_517412943a@dwillia2-xfh.jf.intel.com.notmuch>
References: <63b7289cb7439_5174129445@dwillia2-xfh.jf.intel.com.notmuch>
 <20230105202324.GA1165237@bhelgaas>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230105202324.GA1165237@bhelgaas>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?3j9TgjWsa22NRzR0q6Sc2yvegtF39ObC0W/outJ8L4XvuwX853Tt2PwJNu44?=
 =?us-ascii?Q?kdk+ErnlK2zIB+mO8VJHCB5NbP1ijGLEGMZGSBx+TW8xddAih5Q//rfrqpFj?=
 =?us-ascii?Q?LKuwz76ijEdCQtiE9tCwvzedkHyqj8HexLgGovE8rTMrNVSwb/G8FcS8HNSG?=
 =?us-ascii?Q?aKym1tvDPPC5hbChqcR8Ur1pUMuiLi3QlU1dsf4qbSL0sQFfOxfht5EKtrbM?=
 =?us-ascii?Q?to5oj88o9Hn0aHtjqZP8yqR64gvMZLr9zpRyOdOm/WSrrr/uX4He2SxV5UNq?=
 =?us-ascii?Q?MzksIF5dXsWC+29OVcIZApU4Dngo/xP5QZ9qYEMyiqjAW37H6k4JZpcu7B9s?=
 =?us-ascii?Q?vu3ZZ0qBm1cI+s1DFmNcdmjQWyWupa7bNiwNhVmLCNGTZAyfKmPgtEEMNMSO?=
 =?us-ascii?Q?VuYneuesFXl5x5oKJ0XgZK3QuVonXIsrHZcDRdJl1UXthpsmrGEto9dThmaM?=
 =?us-ascii?Q?VuBrqJqBLhF24UPLOAtu+LfG23C5Bfn6sqEc3dfhXXFDI4u28U/Wb0bwbM6t?=
 =?us-ascii?Q?kx03HER5YI9svNYr8wPNWFQAHqwPtmCRyKAyPhfc7GkufgIFxxGQcaFL5D8z?=
 =?us-ascii?Q?XCG2rRhHmOPSz/Lwa27eHTmpklru8xGzdimtj+QWZlOvsL4zP71dAnhut1Qm?=
 =?us-ascii?Q?OMBtCfrmCNHT+p8ykDhuJsUe/MJIzOAw/O8VKuH6WNI/f34ql1rsr3yKX5vV?=
 =?us-ascii?Q?ycX7+/jPbbGEZJ3KFdrxJgmT9nFItq6E7s3yxISfFRSfdsnc6RGlK6nKtuL6?=
 =?us-ascii?Q?WVJqKv1pHZTNcIaGGgFQAhyXCw2iByFgzuw6G8QHErgFmhlGkWFFSVh00k/p?=
 =?us-ascii?Q?GctL82YKQKMp3H61rSYDhbeOoR6wy5UP9MOHGgSGPA6Bfl8ZBQfHtm/2FEBL?=
 =?us-ascii?Q?fvaDbZTGj6fxrfenxoCsgxopuBOc4JnhYJrJLoBYHmNrPF5s58489VoDXces?=
 =?us-ascii?Q?3tK1dSxEWy7Ii2DzhoGiKkra8IAp6i2rNJHuSgwmsS0gklDQteKtpIBUJSHu?=
 =?us-ascii?Q?Un2iN8UwXWQ1veDYeAtb6SVOCZom7DtLismPUEoa62EULfVHAZgDUUDLAlU4?=
 =?us-ascii?Q?NBDG2EvV9tbN10glaZzY/O69wPNeXWm79fG96UKblwYxKR2pFCnPPAvX2Q4i?=
 =?us-ascii?Q?y6EOvEhOTg0OGpYpitZJEt0X0Nnwznp/KRaBlSZ0jt/RmmJNEbWnZuRvc0b/?=
 =?us-ascii?Q?AJba4Laa8Pd7A58ft0xa5ILXfK/kmob4RZkQPhxWVf7NZ1g7J8dAvh1aJ4Uv?=
 =?us-ascii?Q?PVfUwyvJI8S4wo3Wg3Mj3dJCo7vhQmlx+v3+oiiZ+vcEZlMgOfonkBT8eglO?=
 =?us-ascii?Q?T4gvEWR6FtpArshMzeYLt60x2sNNTeuIDH7p6Z1tyrjHtHHjrs6Ic/qO8Hp7?=
 =?us-ascii?Q?J1aBN4ZygOKbbxyfcGhFCKOkegeb2RqKtQeBDftXSROvFAcaNyoSEnCjwZdN?=
 =?us-ascii?Q?6qkz9K5Zf74twZ9Y2IsUw7HddMPKc7wcki+eLY9YUJ5XA7oNHee7JJbt7jwz?=
 =?us-ascii?Q?UpD5gRHs18OM5J/2AiLb9KpFX8ZPI7GKR0G5tT/R25M99ubbgu206HE9O6BY?=
 =?us-ascii?Q?kiYijkFqRh0KvId4bOEEVALRpFF+I7geMDuaadA2QA8+4jK8xISDDmGA/vok?=
 =?us-ascii?Q?bQ=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: cceecb78-ea2e-4b60-c522-08daef62b176
X-MS-Exchange-CrossTenant-AuthSource: MWHPR1101MB2126.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jan 2023 21:20:39.1093
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: nwaAbkEnl4iALqm9g3UcBLJWhhVhCQ64fBsLg4VIeSbgklMP/T+Ddq5ngqnQIVjwfuTjc5va2q/EpuQWYYV+lYYq6GNl/JA9RHHIzaI1tvI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR11MB7533
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,
        SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Bjorn Helgaas wrote:
> On Thu, Jan 05, 2023 at 11:44:28AM -0800, Dan Williams wrote:
> > Bjorn Helgaas wrote:
> 
> > > Apparently the only mention of [mem 0x80000000-0x8fffffff] in the
> > > firmware/kernel interface is as an EfiMemoryMappedIO region.
> > > 
> > > I think this is a firmware bug, but obviously we're going to have to
> > > figure out a way around it.
> > 
> > Definitely an ambiguity / conflict, but not sure it is a bug when you
> > look at from the perspective of how would an EFI runtime service use
> > ECAM/MMCONFIG space? 
> 
> I think it's perfectly fine for firmware to advertise ECAM space as an
> EfiMemoryMappedIO region via EFI GetMemoryMap() because it certainly
> makes sense that EFI runtime services would use config space.
> 
> My understanding is that the OS should learn about device address
> space via ACPI _CRS, not GetMemoryMap().  The MCFG spec (PCI Firmware
> Spec, r3.3, sec 4.1.2) requires ECAM space to be reserved via a
> PNP0C02 motherboard device _CRS.
> 
> So what I think *is* a bug is that this firmware doesn't report the
> ECAM space via PNP0C02 _CRS.
> 
> If somebody thinks the lack of this reservation is not a bug, I would
> love to hear ideas about how Linux *should* be handling this.  There
> are many variations on how firmware does things like this, and it's
> been a nightmare trying to figure out something that works with all of
> them.

I am trying to get a statement from a BIOS person, but in the meantime I
am confused by this lead in sentence of Note 2 in "PCI Firmware Spec
v3.2 Table 4-2: MCFG Table to Support Enhanced Configuration Space
Access":

    If the operating system does not natively comprehend reserving the MMCFG
    region, the MMCFG region must be reserved by firmware. The address range
    reported in the MCFG table or by _CBA method (see Section 4.1.3) must be
    reserved by declaring a motherboard resource...

Which seems to say it is ok for the OS to treat MMCFG space as reserved
by default. It certainly fails the Robustness Principle for the BIOS to
*assume* that the OS can natively comprehend that reservation, but it
seems Linux is in its rights to make that assumption.

> 
> > Would it be enough to add this clarification in "EFI 2.9 Table 7-6
> > Memory Type Usage after ExitBootServices()"?
> > 
> > s/This memory is not used by the OS./This memory is not used by the OS,
> > unless ACPI declares it for another purpose./
> 
> I guess the idea is that MCFG is a form of "ACPI declaring it"?  I
> don't have an explicit citation for it, but I infer at [1] that ACPI
> static tables are second-class citizens and not intended as a way of
> reserving address space because that would lead to problems booting
> old OSes on firmware that provides new tables unknown to the OS.

Ah, true, certainly for new stuff, but what about MCFG specifically?
What harm is there an assuming that MMCONFIG intersecting with
EfiMemoryMappedIO shall be treated as reserved for MMCONFIG usage.

> 
> Bjorn
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/PCI/acpi-info.rst?id=v6.1#n32