2014-02-14 19:11:37

by Yinghai Lu

[permalink] [raw]
Subject: Re: pci-3.14 resource alloc

On Fri, Feb 14, 2014 at 7:46 AM, Steven Newbury <[email protected]> wrote:
>>
>> Oh, never mind! I didn't notice pref_bar has been renamed to
>> assign_pref_bars. It's working now! :)
>
> There's no pci bridge/bus hotplug though. Docking doesn't reveal the
> pci-e->pci bridge or the (radeon) devices on the other side.

oh, no. could be other regression in linus tree or pci/next.

Can you check if linus tree could reveal pci-e->pci bridge?

or do you use iommu/dmar with the system?

Thanks

Yinghai


2014-02-18 10:08:35

by Steven Newbury

[permalink] [raw]
Subject: Re: pci-3.14 resource alloc

On Fri, 2014-02-14 at 11:11 -0800, Yinghai Lu wrote:
> On Fri, Feb 14, 2014 at 7:46 AM, Steven Newbury <[email protected]> wrote:
> >>
> >> Oh, never mind! I didn't notice pref_bar has been renamed to
> >> assign_pref_bars. It's working now! :)
> >
> > There's no pci bridge/bus hotplug though. Docking doesn't reveal the
> > pci-e->pci bridge or the (radeon) devices on the other side.
>
> oh, no. could be other regression in linus tree or pci/next.
>

Sorry about taking a while to get back to you.

Previously I needed the busn work to get it working, this was included
in the resource-alloc branch. It never worked on mainline, the bridge
used to show up but never get scanned. Now it's not showing up at all
on hotplug. It could be a dock driver regression.

> Can you check if linus tree could reveal pci-e->pci bridge?
>
I'll give it a go.

> or do you use iommu/dmar with the system?
>
No, the only unusual option is I use pci=nocrs since the bios is
completely ignorant of support for 64bit BARs.
> Thanks
>
> Yinghai


2014-02-18 14:00:16

by Bjorn Helgaas

[permalink] [raw]
Subject: Re: pci-3.14 resource alloc

On Tue, Feb 18, 2014 at 3:08 AM, Steven Newbury <[email protected]> wrote:
> On Fri, 2014-02-14 at 11:11 -0800, Yinghai Lu wrote:
>> On Fri, Feb 14, 2014 at 7:46 AM, Steven Newbury <[email protected]> wrote:
>> >>
>> >> Oh, never mind! I didn't notice pref_bar has been renamed to
>> >> assign_pref_bars. It's working now! :)
>> >
>> > There's no pci bridge/bus hotplug though. Docking doesn't reveal the
>> > pci-e->pci bridge or the (radeon) devices on the other side.
>>
>> oh, no. could be other regression in linus tree or pci/next.
>>
>
> Sorry about taking a while to get back to you.
>
> Previously I needed the busn work to get it working, this was included
> in the resource-alloc branch. It never worked on mainline, the bridge
> used to show up but never get scanned. Now it's not showing up at all
> on hotplug. It could be a dock driver regression.
>
>> Can you check if linus tree could reveal pci-e->pci bridge?
>>
> I'll give it a go.
>
>> or do you use iommu/dmar with the system?
>>
> No, the only unusual option is I use pci=nocrs since the bios is
> completely ignorant of support for 64bit BARs.

Hi Steven, this is a tangent, but can you collect the complete dmesg
log with "pci=nocrs" and let me know what happens when you *don't* use
"pci=nocrs" (a complete dmesg log there would be useful too, but I
don't know if you can boot without it)? I think Linux *should* be
able to handle 64bit BARs even if the BIOS doesn't.

Please post these as a separate thread so we don't muddle this
conversation with Yinghai.

Bjorn

2014-02-18 14:37:29

by Bjorn Helgaas

[permalink] [raw]
Subject: Re: pci-3.14 resource alloc

On Tue, Feb 18, 2014 at 6:59 AM, Bjorn Helgaas <[email protected]> wrote:
> On Tue, Feb 18, 2014 at 3:08 AM, Steven Newbury <[email protected]> wrote:

>> No, the only unusual option is I use pci=nocrs since the bios is
>> completely ignorant of support for 64bit BARs.
>
> Hi Steven, this is a tangent, but can you collect the complete dmesg
> log with "pci=nocrs" and let me know what happens when you *don't* use
> "pci=nocrs" (a complete dmesg log there would be useful too, but I
> don't know if you can boot without it)? I think Linux *should* be
> able to handle 64bit BARs even if the BIOS doesn't.
>
> Please post these as a separate thread so we don't muddle this
> conversation with Yinghai.

Oh, never mind. I think I remember now. I think the problem is not
so much that the BIOS doesn't handle 64bit *BARs*, but that your BIOS
doesn't report 64bit host bridge *apertures*, and you want to use
space above 4G for a graphics card or something.

That's not really anything we can fix in PCI because PCI doesn't
discover those apertures. Host bridge drivers like the generic ACPI
pci_root.c or the native amd_bus.c discover them. If those drivers
don't discover them, PCI can't just make them up.

It might be nice if we had cleaner kernel options to say "assume a
host bridge window here" and "put this device here," so you didn't
have to use the big hammer of "pci=nocrs". It's sort of a pain to
figure out how to attach info like that to the correct device (there
might be several host bridges, so how do you specify which one you
care about?), but maybe it could be done.

Bjorn

2014-02-18 17:31:24

by Steven Newbury

[permalink] [raw]
Subject: Re: pci-3.14 resource alloc

On Tue, 2014-02-18 at 07:37 -0700, Bjorn Helgaas wrote:
> On Tue, Feb 18, 2014 at 6:59 AM, Bjorn Helgaas <[email protected]> wrote:
> > On Tue, Feb 18, 2014 at 3:08 AM, Steven Newbury <[email protected]> wrote:
>
> >> No, the only unusual option is I use pci=nocrs since the bios is
> >> completely ignorant of support for 64bit BARs.
> >
> > Hi Steven, this is a tangent, but can you collect the complete dmesg
> > log with "pci=nocrs" and let me know what happens when you *don't* use
> > "pci=nocrs" (a complete dmesg log there would be useful too, but I
> > don't know if you can boot without it)? I think Linux *should* be
> > able to handle 64bit BARs even if the BIOS doesn't.
> >
> > Please post these as a separate thread so we don't muddle this
> > conversation with Yinghai.
>
> Oh, never mind. I think I remember now. I think the problem is not
> so much that the BIOS doesn't handle 64bit *BARs*, but that your BIOS
> doesn't report 64bit host bridge *apertures*, and you want to use
> space above 4G for a graphics card or something.
>
Yes, exactly. I should have stated it better. I need to push the
on-board i965 up above 4G to make room for a radeon which is accessed
through a 32 bit only bridge. The BIOS doesn't provide any
functionality like this. The intel devs seemed a little surprised that
the i965 actually worked stably in this configuration given known
errata, but I've actually had no problems with this side of things.

> That's not really anything we can fix in PCI because PCI doesn't
> discover those apertures. Host bridge drivers like the generic ACPI
> pci_root.c or the native amd_bus.c discover them. If those drivers
> don't discover them, PCI can't just make them up.

32 bit only bridges must be getting pretty rare by now, and I presume
any new hardware out there with a dockable PCI bus would provide 64 bit
resource allocation for host bridge apertures when booting to a 64 bit
OS? Maybe not?
>
> It might be nice if we had cleaner kernel options to say "assume a
> host bridge window here" and "put this device here," so you didn't
> have to use the big hammer of "pci=nocrs". It's sort of a pain to
> figure out how to attach info like that to the correct device (there
> might be several host bridges, so how do you specify which one you
> care about?), but maybe it could be done.
>
That would be much better, even if it's only some way to say "ignore crs
for *this* host bridge (and parents)", or "for host bridges to which
*this* device is attached".

2014-02-18 18:52:58

by Yinghai Lu

[permalink] [raw]
Subject: Re: pci-3.14 resource alloc

On Tue, Feb 18, 2014 at 2:08 AM, Steven Newbury <[email protected]> wrote:
>> >
>> > There's no pci bridge/bus hotplug though. Docking doesn't reveal the
>> > pci-e->pci bridge or the (radeon) devices on the other side.
>>
>> oh, no. could be other regression in linus tree or pci/next.
>>
>
> Previously I needed the busn work to get it working, this was included
> in the resource-alloc branch. It never worked on mainline, the bridge
> used to show up but never get scanned. Now it's not showing up at all
> on hotplug. It could be a dock driver regression.

I had the busn_res_alloc patches in the branch.

There is some changes about acpi dock driver from Rafael in recent
kernels. Maybe Rafael could suggest which commit could cause problem,
then you could try revert them on top of branch.

Thanks

Yinghai

2014-02-18 22:15:10

by Rafael J. Wysocki

[permalink] [raw]
Subject: Re: pci-3.14 resource alloc

On Tuesday, February 18, 2014 10:52:54 AM Yinghai Lu wrote:
> On Tue, Feb 18, 2014 at 2:08 AM, Steven Newbury <[email protected]> wrote:
> >> >
> >> > There's no pci bridge/bus hotplug though. Docking doesn't reveal the
> >> > pci-e->pci bridge or the (radeon) devices on the other side.
> >>
> >> oh, no. could be other regression in linus tree or pci/next.
> >>
> >
> > Previously I needed the busn work to get it working, this was included
> > in the resource-alloc branch. It never worked on mainline, the bridge
> > used to show up but never get scanned. Now it's not showing up at all
> > on hotplug. It could be a dock driver regression.
>
> I had the busn_res_alloc patches in the branch.
>
> There is some changes about acpi dock driver from Rafael in recent
> kernels. Maybe Rafael could suggest which commit could cause problem,
> then you could try revert them on top of branch.

I kind of suspect what might have caused them, but that particular thing
would not be easy to revert.

Steven, what was the last kernel in which the bridge showed up?

Did you test 3.14-rc3?

Rafael

2014-02-19 09:11:57

by Steven Newbury

[permalink] [raw]
Subject: Re: pci-3.14 resource alloc

On Tue, 2014-02-18 at 23:29 +0100, Rafael J. Wysocki wrote:
> On Tuesday, February 18, 2014 10:52:54 AM Yinghai Lu wrote:
> > On Tue, Feb 18, 2014 at 2:08 AM, Steven Newbury <[email protected]> wrote:
> > >> >
> > >> > There's no pci bridge/bus hotplug though. Docking doesn't reveal the
> > >> > pci-e->pci bridge or the (radeon) devices on the other side.
> > >>
> > >> oh, no. could be other regression in linus tree or pci/next.
> > >>
> > >
> > > Previously I needed the busn work to get it working, this was included
> > > in the resource-alloc branch. It never worked on mainline, the bridge
> > > used to show up but never get scanned. Now it's not showing up at all
> > > on hotplug. It could be a dock driver regression.
> >
> > I had the busn_res_alloc patches in the branch.
> >
> > There is some changes about acpi dock driver from Rafael in recent
> > kernels. Maybe Rafael could suggest which commit could cause problem,
> > then you could try revert them on top of branch.
>
> I kind of suspect what might have caused them, but that particular thing
> would not be easy to revert.
>
> Steven, what was the last kernel in which the bridge showed up?
>
> Did you test 3.14-rc3?
>
> Rafael
>

I'll try a few different kernels today and see when it last worked. I
hadn't updated the machine since some 3.12-rc + my local patches so I've
no idea at the moment when it stopped working...