Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Baowen Zheng <baowen.zheng@corigine.com>
To:     helgaas@kernel.org
Cc:     2lprbe78@duck.com, benoitg@coeus.ca, bhelgaas@google.com,
        hdegoede@redhat.com, kernelorg@undead.fr, kjhambrick@gmail.com,
        linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org,
        mika.westerberg@linux.intel.com, mumblingdrunkard@protonmail.com,
        nicholas.johnson-opensource@outlook.com.au,
        wse@tuxedocomputers.com, simon.horman@corigine.com,
        yinjun.zhang@corigine.com, tianyu.yuan@corigine.com
Subject: Re:[PATCH v2 1/4] efi/x86: Remove EfiMemoryMappedIO from E820 map
Date:   Fri, 13 Jan 2023 18:46:03 +0800
Message-Id: <1673606763-30159-1-git-send-email-baowen.zheng@corigine.com>
In-Reply-To: <20221208190341.1560157-2-helgaas@kernel.org>
References: <20221208190341.1560157-2-helgaas@kernel.org>
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?gJF1ZOwSOBnvLc149AavokwosTIYDJHxhmFjP/f22Ltvop2GMXKDZCldAi//?=
 =?us-ascii?Q?wNOe4Dwrz6eAbOkDdY5mL/+nsathifndJPQAWP7wYmiJe8ypa5wo9ztAQ1Rr?=
 =?us-ascii?Q?uxoxhMcva9jAbr2FsQ1AcQleBMcQrNSFFMPs6VkpnDTVzqMckcCqqHgX02e8?=
 =?us-ascii?Q?PrmI1SRckVuwQZ7ifxU1DjnNpEnZubOSXKhCpbHCRfyPBU7VFKX/yWrS04lQ?=
 =?us-ascii?Q?A/85wPEj1iqqQfqrNV2BjWPerGBo5ZncJQYOM9eKn0CMFKWvgOF8zn7qyiiw?=
 =?us-ascii?Q?1vQvOyhiA2ZhBixurPr/Qkv8LDIqQXrX6XKQW62W9obaGJjTyRkcTbXUak7+?=
 =?us-ascii?Q?ZAIerDB6iFWSNKcVuv7zZLzORdPzwNUwIRasz4ZLo+9a0l4Xg6HwAgQiIaRV?=
 =?us-ascii?Q?vfH3Yt9UhA172eMLlAVGyOyxqYpjcSeyFh0NUuYpexmSuyHDYQXlJEa/i9ny?=
 =?us-ascii?Q?gyRLJU0LRxXJxRrj7EqLRIscC257V+EK+VEYQ6lq1q+VLPT5r08Mqb1oVruY?=
 =?us-ascii?Q?Xqy7vg4aka5OPGkNQ3jWQMJm1TUquyJtf8gOPt9bFLvk2ZXE9DDYLWthYGaD?=
 =?us-ascii?Q?aF45MyLX741pl1vaNSbdDYFkEgZLnbjtfLakX2bNIrWvxIcVd8KmZmx8Tm6l?=
 =?us-ascii?Q?CFuaofq5S3sGlt1ZVJw9GgECTA52QYZh7r4dAJuC82FCalbDorH1TFMypfOv?=
 =?us-ascii?Q?zwnXqEJB0/zWm3VQuCBdCepVvtVeI47DNx9FIN9wPtsrYp/inDGwQnPXDqqF?=
 =?us-ascii?Q?iyYB7lTuMGetS3BsrlCL7ff+tKI7fwhItqQ0/69/5AUWVT4ONJ6o/d8dhZmE?=
 =?us-ascii?Q?gUUIFep7b9Z6DnWqM8Wmmk2ck5ZvoftsjGiwu2UpCQqeuCYjbb+BfmLthp5p?=
 =?us-ascii?Q?WfOCE8nuZfDEvwft6UdGjHVF4bLjHWpDYBrLEwIGaLDmQfwBvvtS36DcOnh4?=
 =?us-ascii?Q?6YxWoByquOZv/BJgYwMklX5S9AH1LRjb9VblxxaRLjQMb11twrzgyfUpDtgN?=
 =?us-ascii?Q?ZyLRjE4o8l8AvmzmKk9EF7PmIh3PxoaL5niXP5dtUaQLQA92L2XaYRVClz7g?=
 =?us-ascii?Q?CF+H+wPSoqyLHfsMbQ3rzUT7bXghYuS8oWrkZiAtg0EwjKZA/L0X5jwu0p5n?=
 =?us-ascii?Q?y3VzBfc40HAxDGd4pJNVY728WSe24vlBoCTsatFAuqzHj/6JmgKmxNqPjEvj?=
 =?us-ascii?Q?C7RdGEyXICJOaWENqzKxo49nlBKw+RyOd0X4wiVShG2J4fXfn0m98Gx/ppcO?=
 =?us-ascii?Q?+qX5y4WMZSmt3kesjUKYDneWlXosM8Pa8lXW8FIo2t3xaVeG3HFqVvFOE5XT?=
 =?us-ascii?Q?qIDtvV4Zs9ry1/vND+nwFBBHjKZxHORk1eyiEsqghjKFnuJbXF+0srLmG83u?=
 =?us-ascii?Q?Xgol6OeTXB6ca3FHefoVlgpE+CRfuOOuZ7QiUFbT3oDfQ8Ip7hwzzr5sRy1d?=
 =?us-ascii?Q?PYpczEUgQThku2frQ3NKdbEr32VGqmbeixTReZrcaZcHy1cr7mi5rdcDXEPG?=
 =?us-ascii?Q?xx5HI7IMxmfGZzTA9bSMOB/pmXp+h06i/9Ckkqqw8cddZg9rTosivbvtGFDy?=
 =?us-ascii?Q?KfZM57NFk6v1TkRAv4mlY/1xCtOGBgAJMvmE6j6KMAfwH+krdThdcX57iY/T?=
 =?us-ascii?Q?DA=3D=3D?=
X-OriginatorOrg: corigine.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 778fd6c6-2363-4111-cf81-08daf5536505
X-MS-Exchange-CrossTenant-AuthSource: DM5PR1301MB2172.namprd13.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jan 2023 10:46:15.6379
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kIb5wbDLxAkKlrBedWL5skg5ZCSewvLzFib5EKYOO/OxWhy/W6TG25fROnZlv7OiFNOnQwPlx6P8+mS7n9LNckMjJjM38l0oAnWVFC8o0OY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR13MB5324
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hello Bjorn Helgaas:

When we tested our NIC, we found we will fail to access pcie extent configuration space.
Actually we can only access the 256 Bytes of the pcie configuration space.
You can make check with command of "lspci -s XXXX -vv"
It seems relates to this change, could you please help to make a check?


On Thu, 8 Dec 2022 13:03:38 Bjorn Helgaas wrote:
>Firmware can use EfiMemoryMappedIO to request that MMIO regions be mapped
>by the OS so they can be accessed by EFI runtime services, but should have
>no other significance to the OS (UEFI r2.10, sec 7.2).  However, most
>bootloaders and EFI stubs convert EfiMemoryMappedIO regions to
>E820_TYPE_RESERVED entries, which prevent Linux from allocating space from
>them (see remove_e820_regions()).
>
>Some platforms use EfiMemoryMappedIO entries for PCI MMCONFIG space and PCI
>host bridge windows, which means Linux can't allocate BAR space for
>hot-added devices.
>
>Remove large EfiMemoryMappedIO regions from the E820 map to avoid this
>problem.
>

...

>
>Link: https://bugzilla.kernel.org/show_bug.cgi?id=216565     Clevo NL4XLU
>Link: https://bugzilla.kernel.org/show_bug.cgi?id=206459#c78 Clevo X170KM-G
>Link: https://bugzilla.redhat.com/show_bug.cgi?id=1868899    Ideapad Slim 3
>Link: https://bugzilla.redhat.com/show_bug.cgi?id=2029207    X1 Carbon
>Reported-by: Florent DELAHAYE <kernelorg@undead.fr>
>Tested-by: Konrad J Hambrick <kjhambrick@gmail.com>
>Tested-by: Matt Hansen <2lprbe78@duck.com>
>Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>Cc: Hans de Goede <hdegoede@redhat.com>
>---
> arch/x86/platform/efi/efi.c | 46 +++++++++++++++++++++++++++++++++++++
> 1 file changed, 46 insertions(+)
>
>diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
>index ebc98a68c400..dee1852e95cd 100644
>--- a/arch/x86/platform/efi/efi.c
>+++ b/arch/x86/platform/efi/efi.c
>@@ -303,6 +303,50 @@ static void __init efi_clean_memmap(void)
> 	}
> }
> 
>+/*
>+ * Firmware can use EfiMemoryMappedIO to request that MMIO regions be
>+ * mapped by the OS so they can be accessed by EFI runtime services, but
>+ * should have no other significance to the OS (UEFI r2.10, sec 7.2).
>+ * However, most bootloaders and EFI stubs convert EfiMemoryMappedIO
>+ * regions to E820_TYPE_RESERVED entries, which prevent Linux from
>+ * allocating space from them (see remove_e820_regions()).
>+ *
>+ * Some platforms use EfiMemoryMappedIO entries for PCI MMCONFIG space and
>+ * PCI host bridge windows, which means Linux can't allocate BAR space for
>+ * hot-added devices.
>+ *
>+ * Remove large EfiMemoryMappedIO regions from the E820 map to avoid this
>+ * problem.
>+ *
>+ * Retain small EfiMemoryMappedIO regions because on some platforms, these
>+ * describe non-window space that's included in host bridge _CRS.  If we
>+ * assign that space to PCI devices, they don't work.
>+ */
>+static void __init efi_remove_e820_mmio(void)
>+{
>+	efi_memory_desc_t *md;
>+	u64 size, start, end;
>+	int i = 0;
>+
>+	for_each_efi_memory_desc(md) {
>+		if (md->type == EFI_MEMORY_MAPPED_IO) {
>+			size = md->num_pages << EFI_PAGE_SHIFT;
>+			if (size >= 256*1024) {
>+				start = md->phys_addr;
>+				end = start + size - 1;
>+				pr_info("Remove mem%02u: MMIO range=[0x%08llx-0x%08llx] (%lluMB) from e820 map\n",
>+					i, start, end, size >> 20);
>+				e820__range_remove(start, size,
>+						   E820_TYPE_RESERVED, 1);
>+			} else {
>+				pr_info("Not removing mem%02u: MMIO range=[0x%08llx-0x%08llx] (%lluKB) from e820 map\n",
>+					i, start, end, size >> 10);
>+			}
>+		}
>+		i++;
>+	}
>+}
>+
> void __init efi_print_memmap(void)
> {
> 	efi_memory_desc_t *md;
>@@ -474,6 +518,8 @@ void __init efi_init(void)
> 	set_bit(EFI_RUNTIME_SERVICES, &efi.flags);
> 	efi_clean_memmap();
> 
>+	efi_remove_e820_mmio();
>+
> 	if (efi_enabled(EFI_DBG))
> 		efi_print_memmap();
> }