Hi all,
I want to be able to override the bus resource from ACPI, but nocrs does
not do it. I am putting this out here to get a feel for the sentiment
for doing something like this.
What is my motivation for doing this?
I have a Gigabyte Z170X Designare motherboard which only gives resource
[bus 00-7e]. I want the full [bus 00-ff] because I am trying to add as
many Thunderbolt 3 ports with add-in cards as possible. Thunderbolt
consumes bus numbers quickly. An Intel Ice Lake implementation (ideal)
consumes 42 busses per port, but prior solutions consume 50 busses per
port and have additional busses required for the NHI and USB
controllers, as well as the bridges from the root port.
Why not change nocrs to do this? Why the new kernel parameter?
I imagine that on systems with multiple PCI root complexes, things will
get hairy if we do this, if they are not placed on separate segments /
domains by the firmware. I do not own such a beast, but from what I
understand, the firmware normally places them on the same segment /
domain with non-overlapping bus numbers. But we may still want to use
nocrs for other reasons. I need to use nocrs to allow Linux to allocate
vast amounts of MMIO and MMIO_PREF under the Thunderbolt root ports
without the BIOS support for Thunderbolt. Hence, they should be kept
separate.
Why do this in general?
The bus resource is still a resource which is specified from ACPI, just
like those overridden by nocrs. Even if we do not use pci=nocrs to
override it, it should be possible to override it, just as it is
possible to override _CRS.
Topics for discussion include, but are not limited to:
Is my code great, good, bad, ugly, or does it need work, etc?
Which way to skin the cat / achieve the end goal, in terms of
code?
Should this be done?
Is pci=nobbn a good name for the new parameter?
Should the documentation notice say to report a bug if you use
this, like the nocrs one does?
What are the potential risks and fallout if it is done, if any?
My stance on this is that it will be limited to people who use
the parameter, so it should be safe.
What would it take to convince you to support this?
In relation to arch/x86/pci/acpi.c in the function
pci_acpi_root_prepare_resources(): When using CRS, we remove the
resource that satisfies resource_is_pcicfg_ioport() without the
dev_printk(). But when doing nocrs, we remove it along with all
of the others and do the dev_printk() notice. Should it be
changed in another patch to skip printing the notice for this
resource when using nocrs?
Lastly (semi-unrelated), you may notice that the email linkage is
broken. It is because I made the mistake of using an @outlook email
address. It interferes with the encoding. If I send patches with git
send-email, the encoding is broken. If I use mutt -H, the encoding
works, but the linkage is broken. I have heard that Gmail also has
problems, but I tested it the other day with an old Gmail address I
have, and it appears to work. But I am open for suggestions on what my
new email domain for kernel development should be. I hope I can use a
consumer email provider, and not mess with hosting my own domain (it
sounds like a lot of work). I will switch over soon and send an email
from this account to confirm that the new account is actually me.
Thanks for any comments or insights.
Kind regards,
Nicholas Johnson
Nicholas Johnson (1):
PCI: Add pci=nobbn to ignore ACPI _BBN method to override host bridge
bus window
Documentation/admin-guide/kernel-parameters.txt | 2 ++
arch/x86/include/asm/pci_x86.h | 1 +
arch/x86/pci/acpi.c | 11 +++++++++++
arch/x86/pci/common.c | 3 +++
4 files changed, 17 insertions(+)
--
2.24.0
On Thu, Nov 21, 2019 at 9:02 PM Nicholas Johnson
<[email protected]> wrote:
>
> Hi all,
>
> I want to be able to override the bus resource from ACPI, but nocrs does
> not do it. I am putting this out here to get a feel for the sentiment
> for doing something like this.
This should be cc'd to linus-pci and linux-acpi (added). I only
noticed this message by accident. And I don't see the patch at all.
> What is my motivation for doing this?
>
> I have a Gigabyte Z170X Designare motherboard which only gives resource
> [bus 00-7e]. I want the full [bus 00-ff] because I am trying to add as
> many Thunderbolt 3 ports with add-in cards as possible. Thunderbolt
> consumes bus numbers quickly. An Intel Ice Lake implementation (ideal)
> consumes 42 busses per port, but prior solutions consume 50 busses per
> port and have additional busses required for the NHI and USB
> controllers, as well as the bridges from the root port.
>
> Why not change nocrs to do this? Why the new kernel parameter?
>
> I imagine that on systems with multiple PCI root complexes, things will
> get hairy if we do this, if they are not placed on separate segments /
> domains by the firmware. I do not own such a beast, but from what I
> understand, the firmware normally places them on the same segment /
> domain with non-overlapping bus numbers. But we may still want to use
> nocrs for other reasons. I need to use nocrs to allow Linux to allocate
> vast amounts of MMIO and MMIO_PREF under the Thunderbolt root ports
> without the BIOS support for Thunderbolt. Hence, they should be kept
> separate.
>
> Why do this in general?
>
> The bus resource is still a resource which is specified from ACPI, just
> like those overridden by nocrs. Even if we do not use pci=nocrs to
> override it, it should be possible to override it, just as it is
> possible to override _CRS.
pci=nocrs is for working around defects in firmware or Linux. The
firmware knows more about the platform than Linux, and in general we
have to trust it. We probably should taint the kernel when we use it.
Any parameter like this should work the same on all ACPI systems,
including ia64 and arm64, and should probably also taint the kernel.
I can't see the patch itself, but I'm a little confused because we
normally get the bus number range from _CRS in acpi_pci_root_add() and
your patch doesn't appear to touch that.
> Nicholas Johnson (1):
> PCI: Add pci=nobbn to ignore ACPI _BBN method to override host bridge
> bus window
>
> Documentation/admin-guide/kernel-parameters.txt | 2 ++
> arch/x86/include/asm/pci_x86.h | 1 +
> arch/x86/pci/acpi.c | 11 +++++++++++
> arch/x86/pci/common.c | 3 +++
> 4 files changed, 17 insertions(+)
>
> --
> 2.24.0
>
On Mon, Dec 09, 2019 at 10:10:55AM -0600, Bjorn Helgaas wrote:
> On Thu, Nov 21, 2019 at 9:02 PM Nicholas Johnson
> <[email protected]> wrote:
> >
> > Hi all,
> >
> > I want to be able to override the bus resource from ACPI, but nocrs does
> > not do it. I am putting this out here to get a feel for the sentiment
> > for doing something like this.
>
> This should be cc'd to linus-pci and linux-acpi (added). I only
> noticed this message by accident. And I don't see the patch at all.
I thought I had you cc'd into this - you were in the get_maintainer.pl
output.
Looks like I posted the cover letter twice instead of the patch. I will
re-post shortly.
>
> > What is my motivation for doing this?
> >
> > I have a Gigabyte Z170X Designare motherboard which only gives resource
> > [bus 00-7e]. I want the full [bus 00-ff] because I am trying to add as
> > many Thunderbolt 3 ports with add-in cards as possible. Thunderbolt
> > consumes bus numbers quickly. An Intel Ice Lake implementation (ideal)
> > consumes 42 busses per port, but prior solutions consume 50 busses per
> > port and have additional busses required for the NHI and USB
> > controllers, as well as the bridges from the root port.
> >
> > Why not change nocrs to do this? Why the new kernel parameter?
> >
> > I imagine that on systems with multiple PCI root complexes, things will
> > get hairy if we do this, if they are not placed on separate segments /
> > domains by the firmware. I do not own such a beast, but from what I
> > understand, the firmware normally places them on the same segment /
> > domain with non-overlapping bus numbers. But we may still want to use
> > nocrs for other reasons. I need to use nocrs to allow Linux to allocate
> > vast amounts of MMIO and MMIO_PREF under the Thunderbolt root ports
> > without the BIOS support for Thunderbolt. Hence, they should be kept
> > separate.
> >
> > Why do this in general?
> >
> > The bus resource is still a resource which is specified from ACPI, just
> > like those overridden by nocrs. Even if we do not use pci=nocrs to
> > override it, it should be possible to override it, just as it is
> > possible to override _CRS.
>
> pci=nocrs is for working around defects in firmware or Linux. The
> firmware knows more about the platform than Linux, and in general we
> have to trust it. We probably should taint the kernel when we use it.
>
> Any parameter like this should work the same on all ACPI systems,
> including ia64 and arm64, and should probably also taint the kernel.
>
> I can't see the patch itself, but I'm a little confused because we
> normally get the bus number range from _CRS in acpi_pci_root_add() and
> your patch doesn't appear to touch that.
Because pci=nocrs is x86-only, I was making pci=nobbn x86-only. Which
means I only touched the areas in arch/x86 where pci=nocrs is handled.
If you want, I can change nocrs to work on any arch, although I will
need somebody who owns ACPI systems of different architectures to test.
The only such system I know of (that is not server hardware) is the
Microsoft Surface Pro X, which is an ARM64 system with ACPI and at least
three PCIe root complexes on PCI domains/segments 0, 2, 3. Mainline
Linux does not run properly even on X86 Surface devices, so I imagine it
will be an absolute nightmare getting Linux to run on this. I also have
no intention of buying anything without Thunderbolt / USB4.
Kind regards,
Nicholas
>
> > Nicholas Johnson (1):
> > PCI: Add pci=nobbn to ignore ACPI _BBN method to override host bridge
> > bus window
> >
> > Documentation/admin-guide/kernel-parameters.txt | 2 ++
> > arch/x86/include/asm/pci_x86.h | 1 +
> > arch/x86/pci/acpi.c | 11 +++++++++++
> > arch/x86/pci/common.c | 3 +++
> > 4 files changed, 17 insertions(+)
> >
> > --
> > 2.24.0
> >