Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Sender: Guenter Roeck <groeck7@gmail.com>
Subject: Re: [PATCH v2] PCI: Coalesce contiguous regions for host bridges
To:     Kai-Heng Feng <kai.heng.feng@canonical.com>
Cc:     Bjorn Helgaas <bhelgaas@google.com>,
        "open list:PCI SUBSYSTEM" <linux-pci@vger.kernel.org>,
        open list <linux-kernel@vger.kernel.org>
References: <20210401131252.531935-1-kai.heng.feng@canonical.com>
 <20210709231529.GA3270116@roeck-us.net>
 <CAAd53p7s=k7pa_GdaetQGQYp9GbQR+jkQWLQoe6-c0oTjCQXxw@mail.gmail.com>
From:   Guenter Roeck <linux@roeck-us.net>
Message-ID: <f2239f52-8675-17fb-bf5f-80be0abe78d2@roeck-us.net>
Date:   Sun, 11 Jul 2021 21:50:26 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <CAAd53p7s=k7pa_GdaetQGQYp9GbQR+jkQWLQoe6-c0oTjCQXxw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 7/11/21 8:50 PM, Kai-Heng Feng wrote:
> Hi Guenter,
> 
> On Sat, Jul 10, 2021 at 7:15 AM Guenter Roeck <linux@roeck-us.net> wrote:
>>
>> Hi,
>>
>> On Thu, Apr 01, 2021 at 09:12:52PM +0800, Kai-Heng Feng wrote:
>>> Built-in graphics on HP EliteDesk 805 G6 doesn't work because graphics
>>> can't get the BAR it needs:
>>> [    0.611504] pci_bus 0000:00: root bus resource [mem 0x10020200000-0x100303fffff window]
>>> [    0.611505] pci_bus 0000:00: root bus resource [mem 0x10030400000-0x100401fffff window]
>>> ...
>>> [    0.638083] pci 0000:00:08.1:   bridge window [mem 0xd2000000-0xd23fffff]
>>> [    0.638086] pci 0000:00:08.1:   bridge window [mem 0x10030000000-0x100401fffff 64bit pref]
>>> [    0.962086] pci 0000:00:08.1: can't claim BAR 15 [mem 0x10030000000-0x100401fffff 64bit pref]: no compatible bridge window
>>> [    0.962086] pci 0000:00:08.1: [mem 0x10030000000-0x100401fffff 64bit pref] clipped to [mem 0x10030000000-0x100303fffff 64bit pref]
>>> [    0.962086] pci 0000:00:08.1:   bridge window [mem 0x10030000000-0x100303fffff 64bit pref]
>>> [    0.962086] pci 0000:07:00.0: can't claim BAR 0 [mem 0x10030000000-0x1003fffffff 64bit pref]: no compatible bridge window
>>> [    0.962086] pci 0000:07:00.0: can't claim BAR 2 [mem 0x10040000000-0x100401fffff 64bit pref]: no compatible bridge window
>>>
>>> However, the root bus has two contiguous regions that can contain the
>>> child resource requested.
>>>
>>> Bjorn Helgaas pointed out that we can simply coalesce contiguous regions
>>> for host bridges, since host bridge don't have _SRS. So do that
>>> accordingly to make child resource can be contained. This change makes
>>> the graphics works on the system in question.
>>>
>>> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=212013
>>> Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
>>> Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com>
>>
>> With this patch in place, I can no longer boot the ppc:sam460ex
>> qemu emulation from nvme. I see the following boot error:
>>
>> nvme nvme0: Device not ready; aborting initialisation, CSTS=0x0
>> nvme nvme0: Removing after probe failure status: -19
>>
>> A key difference seems to be swapped region addresses:
>>
>> ok:
>>
>> PCI host bridge to bus 0002:00^M
>> pci_bus 0002:00: root bus resource [io  0x0000-0xffff]
>> pci_bus 0002:00: root bus resource [mem 0xd80000000-0xdffffffff] (bus address [0x80000000-0xffffffff])
>> pci_bus 0002:00: root bus resource [mem 0xc0ee00000-0xc0eefffff] (bus address [0x00000000-0x000fffff])
>>
>> bad:
>>
>> PCI host bridge to bus 0002:00^M
>> pci_bus 0002:00: root bus resource [io  0x0000-0xffff]
>> pci_bus 0002:00: root bus resource [mem 0xc0ee00000-0xc0eefffff] (bus address [0x00000000-0x000fffff])
>> pci_bus 0002:00: root bus resource [mem 0xd80000000-0xdffffffff] (bus address [0x80000000-0xffffffff])
>>
>> and then bar address assignments are swapped/changed.
>>
>> ok:
>>
>> pci 0002:00:06.0: BAR 0: assigned [mem 0xd80000000-0xd83ffffff]^M
>> pci 0002:00:06.0: BAR 1: assigned [mem 0xd84000000-0xd841fffff]^M
>> pci 0002:00:02.0: BAR 0: assigned [mem 0xd84200000-0xd84203fff 64bit]^M
>> pci 0002:00:01.0: BAR 5: assigned [mem 0xd84204000-0xd842041ff]^M
>> pci 0002:00:03.0: BAR 0: assigned [io  0x1000-0x107f]^M
>> pci 0002:00:03.0: BAR 1: assigned [mem 0xd84204200-0xd8420427f]^M
>> pci 0002:00:01.0: BAR 4: assigned [io  0x1080-0x108f]^M
>> pci 0002:00:01.0: BAR 0: assigned [io  0x1090-0x1097]^M
>> pci 0002:00:01.0: BAR 2: assigned [io  0x1098-0x109f]^M
>> pci 0002:00:01.0: BAR 1: assigned [io  0x10a0-0x10a3]^M
>> pci 0002:00:01.0: BAR 3: assigned [io  0x10a4-0x10a7]^M
>> pci_bus 0002:00: resource 4 [io  0x0000-0xffff]^M
>> pci_bus 0002:00: resource 5 [mem 0xd80000000-0xdffffffff]^M
>> pci_bus 0002:00: resource 6 [mem 0xc0ee00000-0xc0eefffff]^M
>>
>> bad:
>>
>> pci 0002:00:06.0: BAR 0: assigned [mem 0xd80000000-0xd83ffffff]^M
>> pci 0002:00:06.0: BAR 1: assigned [mem 0xd84000000-0xd841fffff]^M
>> pci 0002:00:02.0: BAR 0: assigned [mem 0xc0ee00000-0xc0ee03fff 64bit]^M
>> pci 0002:00:01.0: BAR 5: assigned [mem 0xc0ee04000-0xc0ee041ff]^M
>> pci 0002:00:03.0: BAR 0: assigned [io  0x1000-0x107f]^M
>> pci 0002:00:03.0: BAR 1: assigned [mem 0xc0ee04200-0xc0ee0427f]^M
>> pci 0002:00:01.0: BAR 4: assigned [io  0x1080-0x108f]^M
>> pci 0002:00:01.0: BAR 0: assigned [io  0x1090-0x1097]^M
>> pci 0002:00:01.0: BAR 2: assigned [io  0x1098-0x109f]^M
>> pci 0002:00:01.0: BAR 1: assigned [io  0x10a0-0x10a3]^M
>> pci 0002:00:01.0: BAR 3: assigned [io  0x10a4-0x10a7]^M
>> pci_bus 0002:00: resource 4 [io  0x0000-0xffff]^M
>> pci_bus 0002:00: resource 5 [mem 0xc0ee00000-0xc0eefffff]^M
>> pci_bus 0002:00: resource 6 [mem 0xd80000000-0xdffffffff]^M
>>
>> Reverting this patch fixes the problem.
> 
> Can you please comment out the list_sort()? Seems like the precaution
> breaks your system...
> 

Yes, everything works if I re-apply your patch and comment out the
call to list_sort().

Guenter