Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 11 Nov 2022 17:54:25 -0600
From:   Bjorn Helgaas <helgaas@kernel.org>
To:     Xiaochun Lee <lixiaochun.2888@163.com>
Cc:     nirmal.patel@linux.intel.com, jonathan.derrick@linux.dev,
        lpieralisi@kernel.org, robh@kernel.org, kw@linux.com,
        bhelgaas@google.com, linux-pci@vger.kernel.org,
        linux-kernel@vger.kernel.org, Xiaochun Lee <lixc17@lenovo.com>
Subject: Re: [PATCH v1] PCI: Set no io resource for bridges that behind VMD
 controller
Message-ID: <20221111235425.GA764316@bhelgaas>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1663075485-20591-1-git-send-email-lixiaochun.2888@163.com>
Precedence: bulk

On Tue, Sep 13, 2022 at 09:24:45PM +0800, Xiaochun Lee wrote:
> From: Xiaochun Lee <lixc17@lenovo.com>
> 
> When enable VMDs on Intel CPUs, VMD controllers(8086:28c0) be
> recognized by VMD driver and there are many failed messages of
> BAR 13 when scan the bridges and assign IO resource behind it
> as listed below, the bridge wants to get 0x6000 as its IO
> resource, but there is no IO resources on the host bridge.
> 
> VMD host bridge resources:
> vmd 0000:64:00.5: PCI host bridge to bus 10000:80
> pci_bus 10000:80: root bus resource [bus 80-9f]
> pci_bus 10000:80: root bus resource [mem 0xe0000000-0xe1ffffff]
> pci_bus 10000:80: root bus resource [mem 0x24ffff02010-0x24fffffffff 64bit]
> 
> Failed messages of BAR#13:
> pci 10000:80:02.0: BAR 13: no space for [io  size 0x1000]
> pci 10000:80:02.0: BAR 13: failed to assign [io  size 0x1000]
> pci 10000:80:03.0: BAR 13: no space for [io  size 0x1000]
> pci 10000:80:03.0: BAR 13: failed to assign [io  size 0x1000]
> 
> VMD-enabled root ports use
> Enhanced Configuration Access Mechanism (ECAM) access
> PCI Express configuration space, and offer VMD_CFGBAR as
> base of PCI Express configuration space for the bridges
> behind it. The configuration space includes IO resources,
> but these IO resources are not actually used on X86,
> especially the NVMes as device connected on this hot plug
> bridges, and it can result in BAR#13 assign IO resource
> failed. So we clear IO resources by setting an IO base value
> greater than limit to these bridges. Hence, we can leverage
> kernel parameter "pci=hpiosize=0KB" to avoid this failed
> messages show out.
> 
> Signed-off-by: Xiaochun Lee <lixc17@lenovo.com>

Some of the discussion here got lost because of email issues.  Lore
has some:
https://lore.kernel.org/all/1663075485-20591-1-git-send-email-lixiaochun.2888@163.com/T/#u,
and patchwork has a v2 with a little more discussion:
https://patchwork.kernel.org/project/linux-pci/patch/1664288166-7432-1-git-send-email-lixiaochun.2888@163.com/

But the v2 patch doesn't seem to have made it to the mailing lists or
to lore (https://lore.kernel.org/all/?q=f%3Alixc17) and I don't apply
things until they appear on the mailing list.

I *would* like to get rid of those "no space" and "failed to assign"
messages.  This is an issue for platforms other than VMD, too.  Just
an FYI that you need to follow up on this if we want make progress.

Bjorn