2020-06-02 09:23:11

by Piotr Stankiewicz

[permalink] [raw]
Subject: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where appropriate

Seeing as there is shorthand available to use when asking for any type
of interrupt, or any type of message signalled interrupt, leverage it.

Signed-off-by: Piotr Stankiewicz <[email protected]>
Reviewed-by: Andy Shevchenko <[email protected]>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
index 5ed4227f304b..6dbe173a9fd4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
@@ -251,11 +251,11 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
int nvec = pci_msix_vec_count(adev->pdev);
unsigned int flags;

- if (nvec <= 0) {
+ if (nvec > 0)
+ flags = PCI_IRQ_MSI_TYPES;
+ else
flags = PCI_IRQ_MSI;
- } else {
- flags = PCI_IRQ_MSI | PCI_IRQ_MSIX;
- }
+
/* we only need one vector */
nvec = pci_alloc_irq_vectors(adev->pdev, 1, 1, flags);
if (nvec > 0) {
--
2.17.2


2020-06-02 09:51:54

by Andy Shevchenko

[permalink] [raw]
Subject: Re: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where appropriate

On Tue, Jun 2, 2020 at 12:24 PM Piotr Stankiewicz
<[email protected]> wrote:
>
> Seeing as there is shorthand available to use when asking for any type
> of interrupt, or any type of message signalled interrupt, leverage it.
>
> Signed-off-by: Piotr Stankiewicz <[email protected]>
> Reviewed-by: Andy Shevchenko <[email protected]>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> index 5ed4227f304b..6dbe173a9fd4 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> @@ -251,11 +251,11 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
> int nvec = pci_msix_vec_count(adev->pdev);
> unsigned int flags;
>
> - if (nvec <= 0) {
> + if (nvec > 0)
> + flags = PCI_IRQ_MSI_TYPES;
> + else
> flags = PCI_IRQ_MSI;
> - } else {
> - flags = PCI_IRQ_MSI | PCI_IRQ_MSIX;
> - }
> +
> /* we only need one vector */
> nvec = pci_alloc_irq_vectors(adev->pdev, 1, 1, flags);

I'm not sure if you have seen my last comment internally about this patch.

I don't understand why we need these pci_msix_vec_count() followed by
conditional at all.
Perhaps we may simple drop all these and supply flag directly?

But OTOH, I don't know the initial motivation, so, the above patch is
non-intrusive and keeps original logic.

> if (nvec > 0) {
> --
> 2.17.2
>


--
With Best Regards,
Andy Shevchenko

2020-06-02 10:01:00

by Piotr Stankiewicz

[permalink] [raw]
Subject: RE: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where appropriate

> -----Original Message-----
> From: Andy Shevchenko <[email protected]>
> Sent: Tuesday, June 2, 2020 11:49 AM
> To: Stankiewicz, Piotr <[email protected]>
> Cc: Alex Deucher <[email protected]>; Christian König
> <[email protected]>; David Zhou <[email protected]>; David
> Airlie <[email protected]>; Daniel Vetter <[email protected]>; amd-
> [email protected]; dri-devel <[email protected]>; Linux
> Kernel Mailing List <[email protected]>
> Subject: Re: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where
> appropriate
>
> On Tue, Jun 2, 2020 at 12:24 PM Piotr Stankiewicz
> <[email protected]> wrote:
> >
> > Seeing as there is shorthand available to use when asking for any type
> > of interrupt, or any type of message signalled interrupt, leverage it.
> >
> > Signed-off-by: Piotr Stankiewicz <[email protected]>
> > Reviewed-by: Andy Shevchenko <[email protected]>
> > ---
> > drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 8 ++++----
> > 1 file changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> > index 5ed4227f304b..6dbe173a9fd4 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> > @@ -251,11 +251,11 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
> > int nvec = pci_msix_vec_count(adev->pdev);
> > unsigned int flags;
> >
> > - if (nvec <= 0) {
> > + if (nvec > 0)
> > + flags = PCI_IRQ_MSI_TYPES;
> > + else
> > flags = PCI_IRQ_MSI;
> > - } else {
> > - flags = PCI_IRQ_MSI | PCI_IRQ_MSIX;
> > - }
> > +
> > /* we only need one vector */
> > nvec = pci_alloc_irq_vectors(adev->pdev, 1, 1, flags);
>
> I'm not sure if you have seen my last comment internally about this patch.
>
> I don't understand why we need these pci_msix_vec_count() followed by
> conditional at all.
> Perhaps we may simple drop all these and supply flag directly?
>
> But OTOH, I don't know the initial motivation, so, the above patch is
> non-intrusive and keeps original logic.
>

Sorry, I must have misunderstood or missed that comment. I am happy
to do a V2 if dropping the conditional is preferable.

> > if (nvec > 0) {
> > --
> > 2.17.2
> >
>
>
> --
> With Best Regards,
> Andy Shevchenko

BR,
Piotr Stankiewicz

2020-06-02 10:02:21

by Andy Shevchenko

[permalink] [raw]
Subject: Re: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where appropriate

On Tue, Jun 2, 2020 at 12:58 PM Stankiewicz, Piotr
<[email protected]> wrote:
> > -----Original Message-----
> > From: Andy Shevchenko <[email protected]>
> > Sent: Tuesday, June 2, 2020 11:49 AM
> > To: Stankiewicz, Piotr <[email protected]>
> > Cc: Alex Deucher <[email protected]>; Christian König
> > <[email protected]>; David Zhou <[email protected]>; David
> > Airlie <[email protected]>; Daniel Vetter <[email protected]>; amd-
> > [email protected]; dri-devel <[email protected]>; Linux
> > Kernel Mailing List <[email protected]>
> > Subject: Re: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where
> > appropriate
> > On Tue, Jun 2, 2020 at 12:24 PM Piotr Stankiewicz
> > <[email protected]> wrote:

...

> > > int nvec = pci_msix_vec_count(adev->pdev);
> > > unsigned int flags;
> > >
> > > - if (nvec <= 0) {
> > > + if (nvec > 0)
> > > + flags = PCI_IRQ_MSI_TYPES;
> > > + else
> > > flags = PCI_IRQ_MSI;
> > > - } else {
> > > - flags = PCI_IRQ_MSI | PCI_IRQ_MSIX;
> > > - }
> > > +
> > > /* we only need one vector */
> > > nvec = pci_alloc_irq_vectors(adev->pdev, 1, 1, flags);
> >
> > I'm not sure if you have seen my last comment internally about this patch.
> >
> > I don't understand why we need these pci_msix_vec_count() followed by
> > conditional at all.
> > Perhaps we may simple drop all these and supply flag directly?
> >
> > But OTOH, I don't know the initial motivation, so, the above patch is
> > non-intrusive and keeps original logic.
> >
>
> Sorry, I must have misunderstood or missed that comment. I am happy
> to do a V2 if dropping the conditional is preferable.

Let's wait for AMD people to confirm either.

--
With Best Regards,
Andy Shevchenko

2020-06-02 13:40:43

by Michael J. Ruhl

[permalink] [raw]
Subject: RE: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where appropriate

>-----Original Message-----
>From: dri-devel <[email protected]> On Behalf Of
>Piotr Stankiewicz
>Sent: Tuesday, June 2, 2020 5:21 AM
>To: Alex Deucher <[email protected]>; Christian K?nig
><[email protected]>; David Zhou <[email protected]>; David
>Airlie <[email protected]>; Daniel Vetter <[email protected]>
>Cc: Stankiewicz, Piotr <[email protected]>; dri-
>[email protected]; [email protected]; linux-
>[email protected]
>Subject: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where
>appropriate
>
>Seeing as there is shorthand available to use when asking for any type
>of interrupt, or any type of message signalled interrupt, leverage it.
>
>Signed-off-by: Piotr Stankiewicz <[email protected]>
>Reviewed-by: Andy Shevchenko <[email protected]>
>---
> drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
>diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>index 5ed4227f304b..6dbe173a9fd4 100644
>--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>@@ -251,11 +251,11 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
> int nvec = pci_msix_vec_count(adev->pdev);
> unsigned int flags;
>
>- if (nvec <= 0) {
>+ if (nvec > 0)
>+ flags = PCI_IRQ_MSI_TYPES;
>+ else
> flags = PCI_IRQ_MSI;
>- } else {
>- flags = PCI_IRQ_MSI | PCI_IRQ_MSIX;
>- }

Minor nit:

Is it really necessary to set do this check? Can flags just
be set?

I.e.:
flags = PCI_IRQ_MSI_TYPES;

pci_alloc_irq_vector() tries stuff in order. If MSIX is not available,
it will try MSI.

M

>+
> /* we only need one vector */
> nvec = pci_alloc_irq_vectors(adev->pdev, 1, 1, flags);
> if (nvec > 0) {
>--
>2.17.2
>
>_______________________________________________
>dri-devel mailing list
>[email protected]
>https://lists.freedesktop.org/mailman/listinfo/dri-devel

2020-06-02 14:03:55

by Andy Shevchenko

[permalink] [raw]
Subject: Re: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where appropriate

On Tue, Jun 2, 2020 at 4:38 PM Ruhl, Michael J <[email protected]> wrote:
> >-----Original Message-----
> >From: dri-devel <[email protected]> On Behalf Of
> >Piotr Stankiewicz
> >Sent: Tuesday, June 2, 2020 5:21 AM
> >To: Alex Deucher <[email protected]>; Christian König
> ><[email protected]>; David Zhou <[email protected]>; David
> >Airlie <[email protected]>; Daniel Vetter <[email protected]>
> >Cc: Stankiewicz, Piotr <[email protected]>; dri-
> >[email protected]; [email protected]; linux-
> >[email protected]
> >Subject: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where
> >appropriate

...

> > int nvec = pci_msix_vec_count(adev->pdev);
> > unsigned int flags;
> >
> >- if (nvec <= 0) {
> >+ if (nvec > 0)
> >+ flags = PCI_IRQ_MSI_TYPES;
> >+ else
> > flags = PCI_IRQ_MSI;
> >- } else {
> >- flags = PCI_IRQ_MSI | PCI_IRQ_MSIX;
> >- }
>
> Minor nit:
>
> Is it really necessary to set do this check? Can flags just
> be set?
>
> I.e.:
> flags = PCI_IRQ_MSI_TYPES;
>
> pci_alloc_irq_vector() tries stuff in order. If MSIX is not available,
> it will try MSI.

That's also what I proposed earlier. But I suggested as well to wait
for AMD people to confirm that neither pci_msix_vec_count() nor flags
is needed and we can directly supply MSI_TYPES to the below call.

> > /* we only need one vector */
> > nvec = pci_alloc_irq_vectors(adev->pdev, 1, 1, flags);

--
With Best Regards,
Andy Shevchenko

2020-06-02 14:23:40

by Alex Deucher

[permalink] [raw]
Subject: Re: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where appropriate

On Tue, Jun 2, 2020 at 10:00 AM Andy Shevchenko
<[email protected]> wrote:
>
> On Tue, Jun 2, 2020 at 4:38 PM Ruhl, Michael J <[email protected]> wrote:
> > >-----Original Message-----
> > >From: dri-devel <[email protected]> On Behalf Of
> > >Piotr Stankiewicz
> > >Sent: Tuesday, June 2, 2020 5:21 AM
> > >To: Alex Deucher <[email protected]>; Christian König
> > ><[email protected]>; David Zhou <[email protected]>; David
> > >Airlie <[email protected]>; Daniel Vetter <[email protected]>
> > >Cc: Stankiewicz, Piotr <[email protected]>; dri-
> > >[email protected]; [email protected]; linux-
> > >[email protected]
> > >Subject: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where
> > >appropriate
>
> ...
>
> > > int nvec = pci_msix_vec_count(adev->pdev);
> > > unsigned int flags;
> > >
> > >- if (nvec <= 0) {
> > >+ if (nvec > 0)
> > >+ flags = PCI_IRQ_MSI_TYPES;
> > >+ else
> > > flags = PCI_IRQ_MSI;
> > >- } else {
> > >- flags = PCI_IRQ_MSI | PCI_IRQ_MSIX;
> > >- }
> >
> > Minor nit:
> >
> > Is it really necessary to set do this check? Can flags just
> > be set?
> >
> > I.e.:
> > flags = PCI_IRQ_MSI_TYPES;
> >
> > pci_alloc_irq_vector() tries stuff in order. If MSIX is not available,
> > it will try MSI.
>
> That's also what I proposed earlier. But I suggested as well to wait
> for AMD people to confirm that neither pci_msix_vec_count() nor flags
> is needed and we can directly supply MSI_TYPES to the below call.
>

I think it was leftover from debugging and just to be careful. We had
some issues when we originally enabled MSI-X on certain boards. The
fix was to just allocate a single vector (since that is all we use
anyway) and we were using the wrong irq (pdev->irq vs
pci_irq_vector(pdev, 0)). For reference, the original patch to add
MSI-X:

commit bd660f4f111161f60392dd02424c3a3d2240dc2f
Author: shaoyunl <[email protected]>
Date: Tue Oct 1 15:52:31 2019 -0400

drm/amdgpu : enable msix for amdgpu driver

We might used out of the msi resources in some cloud project
which have a lot gpu devices(including PF and VF), msix can
provide enough resources from system level view

Signed-off-by: shaoyunl <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

And the fix:

commit 8a745c7ff2ddb8511ef760b4d9cb4cf56a15fc8d
Author: Alex Deucher <[email protected]>
Date: Thu Oct 3 10:34:30 2019 -0500

drm/amdgpu: improve MSI-X handling (v3)

Check the number of supported vectors and fall back to MSI if
we return or error or 0 MSI-X vectors.

v2: only allocate one vector. We can't currently use more than
one anyway.

v3: install the irq on vector 0.

Tested-by: Tom St Denis <[email protected]>
Reviewed-by: Shaoyun liu <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

Alex

2020-06-02 14:37:38

by Andy Shevchenko

[permalink] [raw]
Subject: Re: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where appropriate

On Tue, Jun 2, 2020 at 5:21 PM Alex Deucher <[email protected]> wrote:
> On Tue, Jun 2, 2020 at 10:00 AM Andy Shevchenko
> <[email protected]> wrote:
> > On Tue, Jun 2, 2020 at 4:38 PM Ruhl, Michael J <[email protected]> wrote:
> > > >From: dri-devel <[email protected]> On Behalf Of
> > > >Piotr Stankiewicz

> > > > int nvec = pci_msix_vec_count(adev->pdev);
> > > > unsigned int flags;
> > > >
> > > >- if (nvec <= 0) {
> > > >+ if (nvec > 0)
> > > >+ flags = PCI_IRQ_MSI_TYPES;
> > > >+ else
> > > > flags = PCI_IRQ_MSI;
> > > >- } else {
> > > >- flags = PCI_IRQ_MSI | PCI_IRQ_MSIX;
> > > >- }
> > >
> > > Minor nit:
> > >
> > > Is it really necessary to set do this check? Can flags just
> > > be set?
> > >
> > > I.e.:
> > > flags = PCI_IRQ_MSI_TYPES;
> > >
> > > pci_alloc_irq_vector() tries stuff in order. If MSIX is not available,
> > > it will try MSI.
> >
> > That's also what I proposed earlier. But I suggested as well to wait
> > for AMD people to confirm that neither pci_msix_vec_count() nor flags
> > is needed and we can directly supply MSI_TYPES to the below call.
> >
>
> I think it was leftover from debugging and just to be careful. We had
> some issues when we originally enabled MSI-X on certain boards. The
> fix was to just allocate a single vector (since that is all we use
> anyway) and we were using the wrong irq (pdev->irq vs
> pci_irq_vector(pdev, 0)).

Do you agree that simple

nvec = pci_alloc_irq_vectors(adev->pdev, 1, 1, PCI_IRQ_MSI_TYPES);

will work and we can remove that leftover?

--
With Best Regards,
Andy Shevchenko

2020-06-02 14:44:50

by Alex Deucher

[permalink] [raw]
Subject: Re: [PATCH 07/15] drm/amdgpu: use PCI_IRQ_MSI_TYPES where appropriate

On Tue, Jun 2, 2020 at 10:35 AM Andy Shevchenko
<[email protected]> wrote:
>
> On Tue, Jun 2, 2020 at 5:21 PM Alex Deucher <[email protected]> wrote:
> > On Tue, Jun 2, 2020 at 10:00 AM Andy Shevchenko
> > <[email protected]> wrote:
> > > On Tue, Jun 2, 2020 at 4:38 PM Ruhl, Michael J <[email protected]> wrote:
> > > > >From: dri-devel <[email protected]> On Behalf Of
> > > > >Piotr Stankiewicz
>
> > > > > int nvec = pci_msix_vec_count(adev->pdev);
> > > > > unsigned int flags;
> > > > >
> > > > >- if (nvec <= 0) {
> > > > >+ if (nvec > 0)
> > > > >+ flags = PCI_IRQ_MSI_TYPES;
> > > > >+ else
> > > > > flags = PCI_IRQ_MSI;
> > > > >- } else {
> > > > >- flags = PCI_IRQ_MSI | PCI_IRQ_MSIX;
> > > > >- }
> > > >
> > > > Minor nit:
> > > >
> > > > Is it really necessary to set do this check? Can flags just
> > > > be set?
> > > >
> > > > I.e.:
> > > > flags = PCI_IRQ_MSI_TYPES;
> > > >
> > > > pci_alloc_irq_vector() tries stuff in order. If MSIX is not available,
> > > > it will try MSI.
> > >
> > > That's also what I proposed earlier. But I suggested as well to wait
> > > for AMD people to confirm that neither pci_msix_vec_count() nor flags
> > > is needed and we can directly supply MSI_TYPES to the below call.
> > >
> >
> > I think it was leftover from debugging and just to be careful. We had
> > some issues when we originally enabled MSI-X on certain boards. The
> > fix was to just allocate a single vector (since that is all we use
> > anyway) and we were using the wrong irq (pdev->irq vs
> > pci_irq_vector(pdev, 0)).
>
> Do you agree that simple
>
> nvec = pci_alloc_irq_vectors(adev->pdev, 1, 1, PCI_IRQ_MSI_TYPES);
>
> will work and we can remove that leftover?

Yes, I believe so. Tom, can you give this a quick spin on raven just
in case if you get a chance? Something like this:

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
index 0cc4c67f95f7..c59111b57cc2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
@@ -248,16 +248,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
adev->irq.msi_enabled = false;

if (amdgpu_msi_ok(adev)) {
- int nvec = pci_msix_vec_count(adev->pdev);
- unsigned int flags;
+ int nvec;

- if (nvec <= 0) {
- flags = PCI_IRQ_MSI;
- } else {
- flags = PCI_IRQ_MSI | PCI_IRQ_MSIX;
- }
/* we only need one vector */
- nvec = pci_alloc_irq_vectors(adev->pdev, 1, 1, flags);
+ nvec = pci_alloc_irq_vectors(adev->pdev, 1, 1,
PCI_IRQ_MSI | PCI_IRQ_MSIX);
if (nvec > 0) {
adev->irq.msi_enabled = true;
dev_dbg(adev->dev, "using MSI/MSI-X.\n");


Thanks,

Alex