LinuxLists.cc - [PATCH 1/1] amdgpu: use MMIO to init atombios if device is Thunderbolt / USB4 attached

2021-03-14 06:12:32

Subject: [PATCH 1/1] amdgpu: use MMIO to init atombios if device is Thunderbolt / USB4 attached

When using some Thunderbolt hosts using BIOS-assisted PCI enumeration
with IO BAR assigned, we get an atombios timeout, such as:

[drm:atom_op_jump [amdgpu]] *ERROR* atombios stuck in loop for more than 20secs aborting
[drm:amdgpu_atom_execute_table_locked [amdgpu]] *ERROR* atombios stuck executing B456 (len 304, WS 4, PS 0) @ 0xB51B
[drm:amdgpu_atom_execute_table_locked [amdgpu]] *ERROR* atombios stuck executing B104 (len 183, WS 0, PS 8) @ 0xB17E
amdgpu 0000:08:00.0: amdgpu: gpu post error!
amdgpu 0000:08:00.0: amdgpu: Fatal error during GPU init
amdgpu: probe of 0000:08:00.0 failed with error -22

A workaround is to use MMIO to access ATOMBIOS when device is
Thunderbolt / USB4 attached.
---
drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
index 86add0f4e..5d16ec10d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
@@ -1999,11 +1999,15 @@ int amdgpu_atombios_init(struct amdgpu_device *adev)
atom_card_info->reg_read = cail_reg_read;
atom_card_info->reg_write = cail_reg_write;
/* needed for iio ops */
- if (adev->rio_mem) {
+ if (adev->rio_mem && !pci_is_thunderbolt_attached(adev->pdev)) {
atom_card_info->ioreg_read = cail_ioreg_read;
atom_card_info->ioreg_write = cail_ioreg_write;
} else {
- DRM_DEBUG("PCI I/O BAR is not found. Using MMIO to access ATOM BIOS\n");
+ if (pci_is_thunderbolt_attached(adev->pdev))
+ DRM_DEBUG("Device is attached via Thunderbolt / USB4. Using MMIO to access ATOM BIOS\n");
+ else
+ DRM_DEBUG("PCI I/O BAR is not found. Using MMIO to access ATOM BIOS\n");
+
atom_card_info->ioreg_read = cail_reg_read;
atom_card_info->ioreg_write = cail_reg_write;
}
--
2.30.2

2021-03-15 15:09:36

by Alex Deucher

[permalink] [raw]

Subject: Re: [PATCH 1/1] amdgpu: use MMIO to init atombios if device is Thunderbolt / USB4 attached

On Mon, Mar 15, 2021 at 4:04 AM Nicholas Johnson
<[email protected]> wrote:
>
> When using some Thunderbolt hosts using BIOS-assisted PCI enumeration
> with IO BAR assigned, we get an atombios timeout, such as:
>
> [drm:atom_op_jump [amdgpu]] *ERROR* atombios stuck in loop for more than 20secs aborting
> [drm:amdgpu_atom_execute_table_locked [amdgpu]] *ERROR* atombios stuck executing B456 (len 304, WS 4, PS 0) @ 0xB51B
> [drm:amdgpu_atom_execute_table_locked [amdgpu]] *ERROR* atombios stuck executing B104 (len 183, WS 0, PS 8) @ 0xB17E
> amdgpu 0000:08:00.0: amdgpu: gpu post error!
> amdgpu 0000:08:00.0: amdgpu: Fatal error during GPU init
> amdgpu: probe of 0000:08:00.0 failed with error -22
>
> A workaround is to use MMIO to access ATOMBIOS when device is
> Thunderbolt / USB4 attached.

Missing your signed-off-by. Also, we can just remove the legacy IO
callbacks altogether. They are leftover from radeon and not required
at all on amdgpu.

Alex

> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
> index 86add0f4e..5d16ec10d 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
> @@ -1999,11 +1999,15 @@ int amdgpu_atombios_init(struct amdgpu_device *adev)
> atom_card_info->reg_read = cail_reg_read;
> atom_card_info->reg_write = cail_reg_write;
> /* needed for iio ops */
> - if (adev->rio_mem) {
> + if (adev->rio_mem && !pci_is_thunderbolt_attached(adev->pdev)) {
> atom_card_info->ioreg_read = cail_ioreg_read;
> atom_card_info->ioreg_write = cail_ioreg_write;
> } else {
> - DRM_DEBUG("PCI I/O BAR is not found. Using MMIO to access ATOM BIOS\n");
> + if (pci_is_thunderbolt_attached(adev->pdev))
> + DRM_DEBUG("Device is attached via Thunderbolt / USB4. Using MMIO to access ATOM BIOS\n");
> + else
> + DRM_DEBUG("PCI I/O BAR is not found. Using MMIO to access ATOM BIOS\n");
> +
> atom_card_info->ioreg_read = cail_reg_read;
> atom_card_info->ioreg_write = cail_reg_write;
> }
> --
> 2.30.2
>
> _______________________________________________
> amd-gfx mailing list
> [email protected]
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx

2021-03-16 11:26:23

by Nicholas Johnson

[permalink] [raw]

Subject: Re: [PATCH 1/1] amdgpu: use MMIO to init atombios if device is Thunderbolt / USB4 attached

On Mon, Mar 15, 2021 at 11:05:21AM -0400, Alex Deucher wrote:
> On Mon, Mar 15, 2021 at 4:04 AM Nicholas Johnson
> <[email protected]> wrote:
> >
> > When using some Thunderbolt hosts using BIOS-assisted PCI enumeration
> > with IO BAR assigned, we get an atombios timeout, such as:
> >
> > [drm:atom_op_jump [amdgpu]] *ERROR* atombios stuck in loop for more than 20secs aborting
> > [drm:amdgpu_atom_execute_table_locked [amdgpu]] *ERROR* atombios stuck executing B456 (len 304, WS 4, PS 0) @ 0xB51B
> > [drm:amdgpu_atom_execute_table_locked [amdgpu]] *ERROR* atombios stuck executing B104 (len 183, WS 0, PS 8) @ 0xB17E
> > amdgpu 0000:08:00.0: amdgpu: gpu post error!
> > amdgpu 0000:08:00.0: amdgpu: Fatal error during GPU init
> > amdgpu: probe of 0000:08:00.0 failed with error -22
> >
> > A workaround is to use MMIO to access ATOMBIOS when device is
> > Thunderbolt / USB4 attached.
>
> Missing your signed-off-by. Also, we can just remove the legacy IO
> callbacks altogether. They are leftover from radeon and not required
> at all on amdgpu.
Sorry, it's been a while; I forgot "-s". And I like your patch much
better. I look forward to the day when all new PCIe devices only have
64-bit MMIO_PREF BARs.

Thanks for looking at this! If you are still doing work on surprise
removal / driver unloading for Thunderbolt, then I am happy to do
testing for you. Removal of DRM devices in Linux is the main sore point
for me, and I would love to see it through.

Regards,
Nicholas Johnson
>
> Alex
>
>
> > ---
> > drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 8 ++++++--
> > 1 file changed, 6 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
> > index 86add0f4e..5d16ec10d 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
> > @@ -1999,11 +1999,15 @@ int amdgpu_atombios_init(struct amdgpu_device *adev)
> > atom_card_info->reg_read = cail_reg_read;
> > atom_card_info->reg_write = cail_reg_write;
> > /* needed for iio ops */
> > - if (adev->rio_mem) {
> > + if (adev->rio_mem && !pci_is_thunderbolt_attached(adev->pdev)) {
> > atom_card_info->ioreg_read = cail_ioreg_read;
> > atom_card_info->ioreg_write = cail_ioreg_write;
> > } else {
> > - DRM_DEBUG("PCI I/O BAR is not found. Using MMIO to access ATOM BIOS\n");
> > + if (pci_is_thunderbolt_attached(adev->pdev))
> > + DRM_DEBUG("Device is attached via Thunderbolt / USB4. Using MMIO to access ATOM BIOS\n");
> > + else
> > + DRM_DEBUG("PCI I/O BAR is not found. Using MMIO to access ATOM BIOS\n");
> > +
> > atom_card_info->ioreg_read = cail_reg_read;
> > atom_card_info->ioreg_write = cail_reg_write;
> > }
> > --
> > 2.30.2
> >
> > _______________________________________________
> > amd-gfx mailing list
> > [email protected]
> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx

2021-03-16 20:33:22

by Andrey Grodzovsky

[permalink] [raw]

Subject: Re: [PATCH 1/1] amdgpu: use MMIO to init atombios if device is Thunderbolt / USB4 attached

We actually do, while currently it's on hold since I switched to working
on surprise insertion
for some time it still would be helpful if you could give it a try.

https://cgit.freedesktop.org/~agrodzov/linux/log/?h=drm-misc-next

Andrey

On 2021-03-15 11:01 p.m., Nicholas Johnson wrote:
> On Mon, Mar 15, 2021 at 11:05:21AM -0400, Alex Deucher wrote:
>> On Mon, Mar 15, 2021 at 4:04 AM Nicholas Johnson
>> <[email protected]> wrote:
>>> When using some Thunderbolt hosts using BIOS-assisted PCI enumeration
>>> with IO BAR assigned, we get an atombios timeout, such as:
>>>
>>> [drm:atom_op_jump [amdgpu]] *ERROR* atombios stuck in loop for more than 20secs aborting
>>> [drm:amdgpu_atom_execute_table_locked [amdgpu]] *ERROR* atombios stuck executing B456 (len 304, WS 4, PS 0) @ 0xB51B
>>> [drm:amdgpu_atom_execute_table_locked [amdgpu]] *ERROR* atombios stuck executing B104 (len 183, WS 0, PS 8) @ 0xB17E
>>> amdgpu 0000:08:00.0: amdgpu: gpu post error!
>>> amdgpu 0000:08:00.0: amdgpu: Fatal error during GPU init
>>> amdgpu: probe of 0000:08:00.0 failed with error -22
>>>
>>> A workaround is to use MMIO to access ATOMBIOS when device is
>>> Thunderbolt / USB4 attached.
>> Missing your signed-off-by. Also, we can just remove the legacy IO
>> callbacks altogether. They are leftover from radeon and not required
>> at all on amdgpu.
> Sorry, it's been a while; I forgot "-s". And I like your patch much
> better. I look forward to the day when all new PCIe devices only have
> 64-bit MMIO_PREF BARs.
>
> Thanks for looking at this! If you are still doing work on surprise
> removal / driver unloading for Thunderbolt, then I am happy to do
> testing for you. Removal of DRM devices in Linux is the main sore point
> for me, and I would love to see it through.
>
> Regards,
> Nicholas Johnson
>> Alex
>>
>>
>>> ---
>>> drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 8 ++++++--
>>> 1 file changed, 6 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
>>> index 86add0f4e..5d16ec10d 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
>>> @@ -1999,11 +1999,15 @@ int amdgpu_atombios_init(struct amdgpu_device *adev)
>>> atom_card_info->reg_read = cail_reg_read;
>>> atom_card_info->reg_write = cail_reg_write;
>>> /* needed for iio ops */
>>> - if (adev->rio_mem) {
>>> + if (adev->rio_mem && !pci_is_thunderbolt_attached(adev->pdev)) {
>>> atom_card_info->ioreg_read = cail_ioreg_read;
>>> atom_card_info->ioreg_write = cail_ioreg_write;
>>> } else {
>>> - DRM_DEBUG("PCI I/O BAR is not found. Using MMIO to access ATOM BIOS\n");
>>> + if (pci_is_thunderbolt_attached(adev->pdev))
>>> + DRM_DEBUG("Device is attached via Thunderbolt / USB4. Using MMIO to access ATOM BIOS\n");
>>> + else
>>> + DRM_DEBUG("PCI I/O BAR is not found. Using MMIO to access ATOM BIOS\n");
>>> +
>>> atom_card_info->ioreg_read = cail_reg_read;
>>> atom_card_info->ioreg_write = cail_reg_write;
>>> }
>>> --
>>> 2.30.2
>>>
>>> _______________________________________________
>>> amd-gfx mailing list
>>> [email protected]
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&data=04%7C01%7Candrey.grodzovsky%40amd.com%7Cbbb3dd71d833433cc92b08d8e8537709%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637514792478220135%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=LBx9fok9E7hD8AKWm9kqFVjhsxsvnLKifDX18MR3eC4%3D&reserved=0
> _______________________________________________
> amd-gfx mailing list
> [email protected]
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&data=04%7C01%7Candrey.grodzovsky%40amd.com%7Cbbb3dd71d833433cc92b08d8e8537709%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637514792478230124%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=3uh0ATLmWUeuYF6b0GZ64b5sKvzyWk2p0v8LjXJf9Ww%3D&reserved=0