Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Xiaochun XC17 Li <lixc17@lenovo.com>
To:     Bjorn Helgaas <helgaas@kernel.org>
CC:     Xiaochun Lee <lixiaochun.2888@163.com>,
        "nirmal.patel@linux.intel.com" <nirmal.patel@linux.intel.com>,
        "jonathan.derrick@linux.dev" <jonathan.derrick@linux.dev>,
        "lpieralisi@kernel.org" <lpieralisi@kernel.org>,
        "robh@kernel.org" <robh@kernel.org>, "kw@linux.com" <kw@linux.com>,
        "bhelgaas@google.com" <bhelgaas@google.com>,
        "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [External] Re: [PATCH v1] PCI: Set no io resource for bridges
 that behind VMD controller
Thread-Topic: [External] Re: [PATCH v1] PCI: Set no io resource for bridges
 that behind VMD controller
Thread-Index: AQHY9ijyZogN/mU3ikCFvi4cdjkRD65A5kcwgAAOsACAAAGRsA==
Date:   Wed, 16 Nov 2022 04:26:02 +0000
Message-ID: <SEYPR03MB68773929A3008A3F5C7E0898BC079@SEYPR03MB6877.apcprd03.prod.outlook.com>
References: <SEYPR03MB6877FB2A677FBBCF80E6BD3ABC079@SEYPR03MB6877.apcprd03.prod.outlook.com>
 <20221116040520.GA1078511@bhelgaas>
In-Reply-To: <20221116040520.GA1078511@bhelgaas>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?qYJ8bN3XWaAG3h7A57iAzP5W4hAdgTwaAn44c4YCElBlaRtkfjSt6dDsQny0?=
 =?us-ascii?Q?3yJIp8rBMVYiP8zBvTXG5kYNdeWQ/tfGazV5gFxiA2OvVbMbmYy8Wr4fl7Fh?=
 =?us-ascii?Q?xiXu4PBh0G2pY/AbJKURNelZ86Bfs5JC4SaRlWnwOPaXW03eOMzlr96bGhuT?=
 =?us-ascii?Q?N7eYLP+eEAckqfoUbqUA0ilminWMDYmZF7aB2Rx/vE49tKGkWJmDncd8PJ2e?=
 =?us-ascii?Q?lDnwnznGmWQskrBlQK2t/FRVQxsQcfxu/zd84M94P9MKBH2xkmUUBznhr20X?=
 =?us-ascii?Q?BHcgm9BYx+Na0SdGy/D8jQkcWqNMuA3ds9mk3tHPim1cIVEAjl7RzvwHJTug?=
 =?us-ascii?Q?h/fwbRQyAmYxZ3YXwNzHSsH1ZeQVHtu3yu5x/RFjMv2UO4mxsApPZL3RQjO6?=
 =?us-ascii?Q?qhEBSEVVmn+KT6+3cfF0Y7NzEBEYEHxC0mR5z36eaj2hEo/DrShrlarOWFaR?=
 =?us-ascii?Q?Pns7LcVGneP++0QLrFwBzL99B/ouuavuPDoBJf0ptemfFyP10NL7mph/PYdD?=
 =?us-ascii?Q?MMBpAeaSotb3CnrBxttXO3oW0gEVLmBULKfn6o4oAU29wUmCJt0dJpvXHq+Q?=
 =?us-ascii?Q?D7TF4qoE/IfGjwxEMRahBgtyeaL1O1lOcagPkL0PTDfWCP8lWWuVKLX6Enfd?=
 =?us-ascii?Q?oN8MtQ4E8l4Ju3UnQ74PE0w9dUofIsHZpJ8WTExNqne1Ub8L2xfRYszz/zIk?=
 =?us-ascii?Q?oYPO4bzn+jAltCGAijIVjVwpDv6zWTE9ZQ6An0dV3wp9Uzc0k4S8knFrblKE?=
 =?us-ascii?Q?h3Yob2h3laS4nZVKGNFafxMhHB5HA8mXAKFKXpMu/jDuO8c9fBeC+4MfRKKI?=
 =?us-ascii?Q?fs6y09wWwWEnfbV/R0LAHtFdEmpIJFYj4l/0adU4Bden866aMAUJBjZfIPaG?=
 =?us-ascii?Q?98lPMkqE9axKHa1C7pab9AX1/7H1nyIMfvGdL8vsUX8Ly3DhytH2lhiubtQP?=
 =?us-ascii?Q?on6yoj6MV+GSPhBXGx+MXbZDqcFy2vTTcdWnQRL43H2mGSeFTDrGZsXY2wS5?=
 =?us-ascii?Q?2ziOscYSvRG+6FaA8omLCUqfGX4vbjZUuZk/VhhMTndaDzhonNJPGVLvTF2n?=
 =?us-ascii?Q?LuHIRhby8GZrmT516FoHU1+nFMgb5B99M7QXnpWPV2TuP0a/KA1ZecYXNcb9?=
 =?us-ascii?Q?d03vREdU77FsQzEazP8pQrom1lzJpwbBVnnQqJy952BTvGc5nRWlpOgeWGLI?=
 =?us-ascii?Q?KRlzEKB99WCsaVZdaqzQRSUCgViK2JAJDOovTVssEAU4K8rQ0L0ynpfGvYVR?=
 =?us-ascii?Q?op8x6ctgp9N/HbE9lwuZrvmwKfc/rauHF09fdFp7H/5IaH9Ta4MSCV745nus?=
 =?us-ascii?Q?Njl2GeDXvX8iatxnsQDS2DVoZdL03kU3cnILQFqmiEZzXB1F2BN1q0aNwAUJ?=
 =?us-ascii?Q?2gGq1GaMDMq/HAiXQFQvbVddAJJfgJEP5tqyoTWttPP8M1eb0QTCygFCriM2?=
 =?us-ascii?Q?lNWtXFkeR8JMJgTAeE/jTvKR8wrDUZ+Ts14EAY02NhM2rp3OVNAbVayuWXr3?=
 =?us-ascii?Q?dhZrcQWG1JtCiv5/A1q5PM1fWtJCcoXSkpIV4/avo3k96y12QdiJE79HIHJH?=
 =?us-ascii?Q?z4moZ+L9h/Ei+4IepIs=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: lenovo.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SEYPR03MB6877.apcprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7d949a9c-ee61-405a-7331-08dac78aabb2
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Nov 2022 04:26:02.6203
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5c7d0b28-bdf8-410c-aa93-4df372b16203
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: dgWsNHVQ7XYVMlhCLVyA3uhfwFB4gxZurXKGPi5Hv2A5T+DxGPKkyBO7VwWBMmGuCzjA1J0ZzjFgq5t7eJba4A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2PR03MB6732
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Nov 16, 2022 at 12:10 PM +0800, "Bjorn Helgaas" <helgaas@kernel.org=
> wrote:
> On Wed, Nov 16, 2022 at 03:34:34AM +0000, Xiaochun XC17 Li wrote:
> > On Sat, Nov 11, 2022 at 07:54:25, "Bjorn Helgaas" <helgaas@kernel.org>
> wrote:
> > > On Tue, Sep 13, 2022 at 09:24:45PM +0800, Xiaochun Lee wrote:
> > > > From: Xiaochun Lee <lixc17@lenovo.com>
> > > >
> > > > When enable VMDs on Intel CPUs, VMD controllers(8086:28c0) be
> > > > recognized by VMD driver and there are many failed messages of BAR
> > > > 13 when scan the bridges and assign IO resource behind it as
> > > > listed below, the bridge wants to get 0x6000 as its IO resource,
> > > > but there is no IO resources on the host bridge.
> > > >
> > > > VMD host bridge resources:
> > > > vmd 0000:64:00.5: PCI host bridge to bus 10000:80 pci_bus 10000:80:
> > > > root bus resource [bus 80-9f] pci_bus 10000:80: root bus resource
> > > > [mem 0xe0000000-0xe1ffffff] pci_bus 10000:80: root bus resource
> > > > [mem 0x24ffff02010-0x24fffffffff 64bit]
> > > >
> > > > Failed messages of BAR#13:
> > > > pci 10000:80:02.0: BAR 13: no space for [io  size 0x1000] pci
> > > > 10000:80:02.0: BAR 13: failed to assign [io  size 0x1000] pci
> > > > 10000:80:03.0: BAR 13: no space for [io  size 0x1000] pci
> > > > 10000:80:03.0: BAR 13: failed to assign [io  size 0x1000]
> > > >
> > > > VMD-enabled root ports use
> > > > Enhanced Configuration Access Mechanism (ECAM) access PCI
> Express
> > > > configuration space, and offer VMD_CFGBAR as base of PCI Express
> > > > configuration space for the bridges behind it. The configuration
> > > > space includes IO resources, but these IO resources are not
> > > > actually used on X86, especially the NVMes as device connected on
> > > > this hot plug bridges, and it can result in BAR#13 assign IO
> > > > resource failed. So we clear IO resources by setting an IO base
> > > > value greater than limit to these bridges. Hence, we can leverage
> > > > kernel parameter "pci=3Dhpiosize=3D0KB" to avoid this failed messag=
es
> show out.
> > > >
> > > > Signed-off-by: Xiaochun Lee <lixc17@lenovo.com>
> > >
> > > Some of the discussion here got lost because of email issues.  Lore
> > > has
> > > some:
> > >
> https://apc01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flore.
> > > kernel.org%2Fall%2F1663075485-20591-1-git-send-email-
> > >
> lixiaochun.2888%40163.com%2FT%2F%23u&amp;data=3D05%7C01%7Clixc17
> > > %40lenovo.com%7C9cd095ffdb584e492dec08dac440139b%7C5c7d0b28
> bdf
> > >
> 8410caa934df372b16203%7C0%7C0%7C638038076734438158%7CUnknown
> > > %7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1
> ha
> > >
> WwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=3DJzWYkIsaEfINofiqo
> > > XyjEh43VjXO3HZw2JLSsmhpUiQ%3D&amp;reserved=3D0,
> > > and patchwork has a v2 with a little more discussion:
> > >
> https://apc01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fpa
> > > tc
> > > hwork.kernel.org%2Fproject%2Flinux-pci%2Fpatch%2F1664288166-
> 7432-1-
> > > git-send-email-
> > >
> lixiaochun.2888%40163.com%2F&amp;data=3D05%7C01%7Clixc17%40lenovo.
> > >
> com%7C9cd095ffdb584e492dec08dac440139b%7C5c7d0b28bdf8410caa934
> > >
> df372b16203%7C0%7C0%7C638038076734438158%7CUnknown%7CTWFpb
> > >
> GZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI
> > >
> 6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=3Dy%2BtIsepTpyLeoHW6CrgkZz2
> > > tiMjY0TONfK7zNCKXQ90%3D&amp;reserved=3D0
> > >
> > > But the v2 patch doesn't seem to have made it to the mailing lists
> > > or to lore
> > >
> (https://apc01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fl
> > > or
> > >
> e.kernel.org%2Fall%2F%3Fq%3Df%253Alixc17&amp;data=3D05%7C01%7Clixc1
> > >
> 7%40lenovo.com%7C9cd095ffdb584e492dec08dac440139b%7C5c7d0b28bd
> > >
> f8410caa934df372b16203%7C0%7C0%7C638038076734438158%7CUnknow
> > >
> n%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h
> > >
> aWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=3DzxGz1hZOD2tvQP
> > > EsbxQTzjHwQvXvqeO%2FUd6I9S%2Fj314%3D&amp;reserved=3D0) and I
> don't
> > > apply things until they appear on the mailing list.
> > >
> > > I *would* like to get rid of those "no space" and "failed to assign"
> > > messages.  This is an issue for platforms other than VMD, too.  Just
> > > an FYI that you need to follow up on this if we want make progress.
> >
> > Thanks for your comments, so do you mean we'd better come up with a
> > solution to avoid messages like "no space" and "failed to assign"
> > using a common way, for both VMD and other platforms?
>=20
> I tried to say two separate things:
>=20
> 1) It doesn't seem like a VMD-specific thing, so it would be ideal if the
> solution were generic instead of being VMD-specific.

OK, I'll try to find out a generic approach.

>=20
> 2) Some of your previous patches didn't make it to the mailing list, so w=
e
> couldn't really do anything with them.
>=20
> Bjorn

Actually, the V2 patch didn't following the mailing of V1 patch,
I sent it out separately like the first time for V1 patch, I don't know if =
there
is a mistake in the way I sent the V2 patch. I was wondering if I was going=
 to
send out V2 patches based on the mailing list of V1 patch, if it is yes, I =
must
study how to summit the V2 patch, so would you please correct me?

Xiaochun