2011-11-16 10:07:45

by Stanislaw Gruszka

[permalink] [raw]
Subject: [PATCH] rt2800pci: handle spurious interrupts

Some devices may generate spurious interrupts, we have to handle them
otherwise interrupt line will be disabled with below message and driver
will not work:

[ 2052.114334] irq 17: nobody cared (try booting with the "irqpoll" option)
[ 2052.114339] Pid: 0, comm: swapper Tainted: P 2.6.35.6-48.fc14.x86_64 #1
[ 2052.114341] Call Trace:
[ 2052.114342] <IRQ> [<ffffffff810a6e2b>] __report_bad_irq.clone.1+0x3d/0x8b
[ 2052.114349] [<ffffffff810a6f93>] note_interrupt+0x11a/0x17f
[ 2052.114352] [<ffffffff810a7a73>] handle_fasteoi_irq+0xa8/0xce
[ 2052.114355] [<ffffffff8100c2ea>] handle_irq+0x88/0x90
[ 2052.114357] [<ffffffff8146f034>] do_IRQ+0x5c/0xb4
[ 2052.114360] [<ffffffff81469593>] ret_from_intr+0x0/0x11
[ 2052.114361] <EOI> [<ffffffff8102b7f9>] ? native_safe_halt+0xb/0xd
[ 2052.114366] [<ffffffff81010f03>] ? need_resched+0x23/0x2d
[ 2052.114367] [<ffffffff8101102a>] default_idle+0x34/0x4f
[ 2052.114370] [<ffffffff81008325>] cpu_idle+0xaa/0xcc
[ 2052.114373] [<ffffffff81461f2a>] start_secondary+0x24d/0x28e
[ 2052.114374] handlers:
[ 2052.114375] [<ffffffff81332944>] (usb_hcd_irq+0x0/0x7c)
[ 2052.114378] [<ffffffffa00697da>] (rt2800pci_interrupt+0x0/0x18d [rt2800pci])
[ 2052.114384] Disabling IRQ #17

Resolve:
https://bugzilla.redhat.com/show_bug.cgi?id=658451

Reported-and-tested-by: Amir Hedayaty <[email protected]>
Cc: [email protected]
Signed-off-by: Stanislaw Gruszka <[email protected]>
---
I changed rt2800pci only because get bug report for that device, not
sure if similar changes are not needed for other rt2x00 PCI devices.

drivers/net/wireless/rt2x00/rt2800pci.c | 7 ++++++-
1 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/drivers/net/wireless/rt2x00/rt2800pci.c b/drivers/net/wireless/rt2x00/rt2800pci.c
index da48c8a..4dc2d0f 100644
--- a/drivers/net/wireless/rt2x00/rt2800pci.c
+++ b/drivers/net/wireless/rt2x00/rt2800pci.c
@@ -880,8 +880,13 @@ static irqreturn_t rt2800pci_interrupt(int irq, void *dev_instance)
rt2x00pci_register_read(rt2x00dev, INT_SOURCE_CSR, &reg);
rt2x00pci_register_write(rt2x00dev, INT_SOURCE_CSR, reg);

+ /*
+ * Some devices can generate interrupts with empty CSR register, we
+ * "handle" such irq's to prevent interrupt controller treat them as
+ * spurious interrupts and disable irq line.
+ */
if (!reg)
- return IRQ_NONE;
+ return IRQ_HANDLED;

if (!test_bit(DEVICE_STATE_ENABLED_RADIO, &rt2x00dev->flags))
return IRQ_HANDLED;
--
1.7.1



2011-11-16 10:46:48

by Helmut Schaa

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

On Wed, Nov 16, 2011 at 11:09 AM, Stanislaw Gruszka <[email protected]> wrote:
> + ? ? ? /*
> + ? ? ? ?* Some devices can generate interrupts with empty CSR register, we
> + ? ? ? ?* "handle" such irq's to prevent interrupt controller treat them as
> + ? ? ? ?* spurious interrupts and disable irq line.
> + ? ? ? ?*/
> ? ? ? ?if (!reg)
> - ? ? ? ? ? ? ? return IRQ_NONE;
> + ? ? ? ? ? ? ? return IRQ_HANDLED;

What if this interrupt belongs to a different device on a shared interrupt line?
In case reg is zero we can't be sure our device generated this interrupt, right?

Helmut

2011-11-23 14:45:55

by Luis Correia

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

Hi Stanislaw,

On Wed, Nov 23, 2011 at 14:40, Stanislaw Gruszka <[email protected]> wrote:
> Hi Luis,
>
> (adding CCes again, please keep them next time :-)

Sorry about the mess, I was in a hurry to leave and really wanted to
provide feedback.

>
> On Wed, Nov 23, 2011 at 10:54:03AM +0000, Luis Correia wrote:
>> this suggestion won't really help much, but you could try also booting
>> with the 'irqpoll' option.
>>
>> This slows down a bit all interrupt processing but can help to
>> investigate situations in where the IRQ lines are really shared.
>
> Thanks, this is a good hint. I think irqpoll option can be used
> as workaround for the bug.
>
> Stanislaw
>

The last time I used it, it really did avoid kernel panic because the
two drivers that were sharing the same IRQ line were misbehaving.

Luis Correia

2011-11-23 08:51:39

by Helmut Schaa

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

On Wed, Nov 23, 2011 at 9:05 AM, Stanislaw Gruszka <[email protected]> wrote:
> On Tue, Nov 22, 2011 at 04:49:40PM -0500, John W. Linville wrote:
>> On Wed, Nov 16, 2011 at 11:09:17AM +0100, Stanislaw Gruszka wrote:
>> Perhaps there is some way to convince the hardware not to generate
>> spurious interrupts?
>
> Ivo, Gertjan, Helmut, is there a possibility that I can get hardware
> documentation or any other support from Ralink, that could help to
> solve this (and possible other issues)?

There is sort of documentation for this hardware but not freely available :(

Are we 100% sure that the rt2800pci device is generating the interrupts?

Just a shot in the dark but instead of masking out interrupts we don't handle
just enable all device interrupts. Mind to try the below patch?

Helmut

---
diff --git a/drivers/net/wireless/rt2x00/rt2800pci.c
b/drivers/net/wireless/rt2x00/rt2800pci.c
index 4dc2d0f..c0badec 100644
--- a/drivers/net/wireless/rt2x00/rt2800pci.c
+++ b/drivers/net/wireless/rt2x00/rt2800pci.c
@@ -437,24 +437,24 @@ static void rt2800pci_toggle_irq(struct
rt2x00_dev *rt2x00dev,

spin_lock_irqsave(&rt2x00dev->irqmask_lock, flags);
rt2x00pci_register_read(rt2x00dev, INT_MASK_CSR, &reg);
- rt2x00_set_field32(&reg, INT_MASK_CSR_RXDELAYINT, 0);
- rt2x00_set_field32(&reg, INT_MASK_CSR_TXDELAYINT, 0);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_RXDELAYINT, mask);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_TXDELAYINT, mask);
rt2x00_set_field32(&reg, INT_MASK_CSR_RX_DONE, mask);
- rt2x00_set_field32(&reg, INT_MASK_CSR_AC0_DMA_DONE, 0);
- rt2x00_set_field32(&reg, INT_MASK_CSR_AC1_DMA_DONE, 0);
- rt2x00_set_field32(&reg, INT_MASK_CSR_AC2_DMA_DONE, 0);
- rt2x00_set_field32(&reg, INT_MASK_CSR_AC3_DMA_DONE, 0);
- rt2x00_set_field32(&reg, INT_MASK_CSR_HCCA_DMA_DONE, 0);
- rt2x00_set_field32(&reg, INT_MASK_CSR_MGMT_DMA_DONE, 0);
- rt2x00_set_field32(&reg, INT_MASK_CSR_MCU_COMMAND, 0);
- rt2x00_set_field32(&reg, INT_MASK_CSR_RXTX_COHERENT, 0);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_AC0_DMA_DONE, mask);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_AC1_DMA_DONE, mask);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_AC2_DMA_DONE, mask);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_AC3_DMA_DONE, mask);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_HCCA_DMA_DONE, mask);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_MGMT_DMA_DONE, mask);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_MCU_COMMAND, mask);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_RXTX_COHERENT, mask);
rt2x00_set_field32(&reg, INT_MASK_CSR_TBTT, mask);
rt2x00_set_field32(&reg, INT_MASK_CSR_PRE_TBTT, mask);
rt2x00_set_field32(&reg, INT_MASK_CSR_TX_FIFO_STATUS, mask);
rt2x00_set_field32(&reg, INT_MASK_CSR_AUTO_WAKEUP, mask);
- rt2x00_set_field32(&reg, INT_MASK_CSR_GPTIMER, 0);
- rt2x00_set_field32(&reg, INT_MASK_CSR_RX_COHERENT, 0);
- rt2x00_set_field32(&reg, INT_MASK_CSR_TX_COHERENT, 0);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_GPTIMER, mask);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_RX_COHERENT, mask);
+ rt2x00_set_field32(&reg, INT_MASK_CSR_TX_COHERENT, mask);
rt2x00pci_register_write(rt2x00dev, INT_MASK_CSR, reg);
spin_unlock_irqrestore(&rt2x00dev->irqmask_lock, flags);

2011-11-16 11:15:31

by Stanislaw Gruszka

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

On Wed, Nov 16, 2011 at 11:14:41AM +0100, Ivo Van Doorn wrote:
> > I changed rt2800pci only because get bug report for that device, not
> > sure if similar changes are not needed for other rt2x00 PCI devices.
>
> The ticket mentiones that the end-result is a slow connection,
> this is also a bug in older chipsets. So could you prepare a patch
> for the other drivers as well?
Yes, sure. Do you have handy links to bug reports that I could
reference in the changelog?

Stanislaw

2011-11-22 20:35:06

by Michael Basse

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Stanislaw,
first of all thank you for your patch.

> I changed rt2800pci only because get bug report for that device,
> not sure if similar changes are not needed for other rt2x00 PCI
> devices

Me (and many other) ubuntu-users are facing issues with different
Version of Kernel 3.x. As it seems all users are using "rt2800pci" for
there wifi.

Model: "RaLink RT2860"
Vendor: pci 0x1814 "RaLink"
Device: pci 0x0781 "RT2860"
SubVendor: pci 0x1814 "RaLink"
SubDevice: pci 0x2790
Driver: "rt2800pci"
Driver Modules: "rt2800pci"

See also

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/869502
http://thread.gmane.org/gmane.linux.kernel/1211290
http://rt2x00.serialmonkey.com/phpBB/viewtopic.php?f=5&t=6192

I have build the newest ubuntu-kernel with your patch

http://alpha-unix.de/~iceroot/ubuntu/

Hopefully your patch is fixing the issues we are facing.
Just wanted to say thank you to your work and i will give you feedback
about your patch.

Greetings from germany

Michael Basse
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJOzAcdAAoJEMIajL8dFT4GAXwIAJJJoUghmTUbw/354z09CDuB
eCpIaJw0is+zyp3RLssUJmmabyFtLkC0nFJ4iAGJfL/ozf7ISjtk1nLlQFbt6/iy
knV1uSGv6o9UuFCzJZ7boJ9cdNq35pAVul86PFqQ+fgDQYdlqfdLyNc71JRazvIE
b01jUudgjZDe09rKmrxiBvcr7FVKPhsSvrH5t6mSYpHJufDDm+TmQk+TJBT+KS8/
aDMPKXlCaAXNlXlm8tgLds8Flx5nnxGvvSXA1DDw37Z7kHOWKLOeHz8ILuctneVz
e8Fp5kgl02VVUVFK/K6+M4EbF/jmTo7FIW5Nlv60H9qsfx+peTkhKqx8o48FGAQ=
=D2BU
-----END PGP SIGNATURE-----


2011-11-23 10:48:55

by Stanislaw Gruszka

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

On Wed, Nov 23, 2011 at 09:51:38AM +0100, Helmut Schaa wrote:
> Are we 100% sure that the rt2800pci device is generating the interrupts?
No, good point. There is also usb_hcd_irq() registered on that line.

I wonder if there is method to control irq routing. I don't know
about any that could allow to control assignment of individual irq's.
There are nomsi, acpi=noirq and noacpi kernel parameters which influence
irq routing, so perhaps with a bit of luck they could cause exclusively
Ralink device will be assigned to irq line>

Amir, could you configure system to assure Ralink device have
assigned exclusive (non shared) irq line. Interrupts assignment
can be seen in /proc/interrupts. If above kernel parameters do not
achieve that, you can try disabling USB in BIOS and (if possible)
changing pci slot of Ralink device.

> Just a shot in the dark but instead of masking out interrupts we don't handle
> just enable all device interrupts. Mind to try the below patch?
Amir, the best way is to test it on top of compat wireless, if you will
be unable to do this, I'll prepare fedora koji build with patch.

Thanks
Stanislaw

> ---
> diff --git a/drivers/net/wireless/rt2x00/rt2800pci.c
> b/drivers/net/wireless/rt2x00/rt2800pci.c
> index 4dc2d0f..c0badec 100644
> --- a/drivers/net/wireless/rt2x00/rt2800pci.c
> +++ b/drivers/net/wireless/rt2x00/rt2800pci.c
> @@ -437,24 +437,24 @@ static void rt2800pci_toggle_irq(struct
> rt2x00_dev *rt2x00dev,
>
> spin_lock_irqsave(&rt2x00dev->irqmask_lock, flags);
> rt2x00pci_register_read(rt2x00dev, INT_MASK_CSR, &reg);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_RXDELAYINT, 0);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_TXDELAYINT, 0);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_RXDELAYINT, mask);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_TXDELAYINT, mask);
> rt2x00_set_field32(&reg, INT_MASK_CSR_RX_DONE, mask);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_AC0_DMA_DONE, 0);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_AC1_DMA_DONE, 0);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_AC2_DMA_DONE, 0);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_AC3_DMA_DONE, 0);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_HCCA_DMA_DONE, 0);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_MGMT_DMA_DONE, 0);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_MCU_COMMAND, 0);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_RXTX_COHERENT, 0);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_AC0_DMA_DONE, mask);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_AC1_DMA_DONE, mask);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_AC2_DMA_DONE, mask);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_AC3_DMA_DONE, mask);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_HCCA_DMA_DONE, mask);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_MGMT_DMA_DONE, mask);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_MCU_COMMAND, mask);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_RXTX_COHERENT, mask);
> rt2x00_set_field32(&reg, INT_MASK_CSR_TBTT, mask);
> rt2x00_set_field32(&reg, INT_MASK_CSR_PRE_TBTT, mask);
> rt2x00_set_field32(&reg, INT_MASK_CSR_TX_FIFO_STATUS, mask);
> rt2x00_set_field32(&reg, INT_MASK_CSR_AUTO_WAKEUP, mask);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_GPTIMER, 0);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_RX_COHERENT, 0);
> - rt2x00_set_field32(&reg, INT_MASK_CSR_TX_COHERENT, 0);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_GPTIMER, mask);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_RX_COHERENT, mask);
> + rt2x00_set_field32(&reg, INT_MASK_CSR_TX_COHERENT, mask);
> rt2x00pci_register_write(rt2x00dev, INT_MASK_CSR, reg);
> spin_unlock_irqrestore(&rt2x00dev->irqmask_lock, flags);

2011-11-16 11:01:59

by Helmut Schaa

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

On Wed, Nov 16, 2011 at 11:52 AM, Ivo Van Doorn <[email protected]> wrote:
> On Wed, Nov 16, 2011 at 11:46 AM, Helmut Schaa
> <[email protected]> wrote:
>> On Wed, Nov 16, 2011 at 11:09 AM, Stanislaw Gruszka <[email protected]> wrote:
>>> + ? ? ? /*
>>> + ? ? ? ?* Some devices can generate interrupts with empty CSR register, we
>>> + ? ? ? ?* "handle" such irq's to prevent interrupt controller treat them as
>>> + ? ? ? ?* spurious interrupts and disable irq line.
>>> + ? ? ? ?*/
>>> ? ? ? ?if (!reg)
>>> - ? ? ? ? ? ? ? return IRQ_NONE;
>>> + ? ? ? ? ? ? ? return IRQ_HANDLED;
>>
>> What if this interrupt belongs to a different device on a shared interrupt line?
>> In case reg is zero we can't be sure our device generated this interrupt, right?
>
> In the bugzilla report ?Stanislaw indicated that even when rt2800pci
> indicates IRQ_HANDLED,
> all other drivers on the same interruptline will still get the interrupt event.

Ah, ok. Should be fine with me then ...

Thanks.

2011-11-23 14:38:17

by Stanislaw Gruszka

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

Hi Luis,

(adding CCes again, please keep them next time :-)

On Wed, Nov 23, 2011 at 10:54:03AM +0000, Luis Correia wrote:
> this suggestion won't really help much, but you could try also booting
> with the 'irqpoll' option.
>
> This slows down a bit all interrupt processing but can help to
> investigate situations in where the IRQ lines are really shared.

Thanks, this is a good hint. I think irqpoll option can be used
as workaround for the bug.

Stanislaw

2011-11-23 08:50:48

by Ivo Van Doorn

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

Hi,

On Wed, Nov 23, 2011 at 9:05 AM, Stanislaw Gruszka <[email protected]> wrote:
> On Tue, Nov 22, 2011 at 04:49:40PM -0500, John W. Linville wrote:
>> On Wed, Nov 16, 2011 at 11:09:17AM +0100, Stanislaw Gruszka wrote:
>> > Some devices may generate spurious interrupts, we have to handle them
>> > otherwise interrupt line will be disabled with below message and driver
>> > will not work:
>> >
>> > [ 2052.114334] irq 17: nobody cared (try booting with the "irqpoll" option)
>> > [ 2052.114339] Pid: 0, comm: swapper Tainted: P ? ? ? ? ? 2.6.35.6-48.fc14.x86_64 #1
>> > [ 2052.114341] Call Trace:
>> > [ 2052.114342] ?<IRQ> ?[<ffffffff810a6e2b>] __report_bad_irq.clone.1+0x3d/0x8b
>> > [ 2052.114349] ?[<ffffffff810a6f93>] note_interrupt+0x11a/0x17f
>> > [ 2052.114352] ?[<ffffffff810a7a73>] handle_fasteoi_irq+0xa8/0xce
>> > [ 2052.114355] ?[<ffffffff8100c2ea>] handle_irq+0x88/0x90
>> > [ 2052.114357] ?[<ffffffff8146f034>] do_IRQ+0x5c/0xb4
>> > [ 2052.114360] ?[<ffffffff81469593>] ret_from_intr+0x0/0x11
>> > [ 2052.114361] ?<EOI> ?[<ffffffff8102b7f9>] ? native_safe_halt+0xb/0xd
>> > [ 2052.114366] ?[<ffffffff81010f03>] ? need_resched+0x23/0x2d
>> > [ 2052.114367] ?[<ffffffff8101102a>] default_idle+0x34/0x4f
>> > [ 2052.114370] ?[<ffffffff81008325>] cpu_idle+0xaa/0xcc
>> > [ 2052.114373] ?[<ffffffff81461f2a>] start_secondary+0x24d/0x28e
>> > [ 2052.114374] handlers:
>> > [ 2052.114375] [<ffffffff81332944>] (usb_hcd_irq+0x0/0x7c)
>> > [ 2052.114378] [<ffffffffa00697da>] (rt2800pci_interrupt+0x0/0x18d [rt2800pci])
>> > [ 2052.114384] Disabling IRQ #17
>> >
>> > Resolve:
>> > https://bugzilla.redhat.com/show_bug.cgi?id=658451
>> >
>> > Reported-and-tested-by: Amir Hedayaty <[email protected]>
>> > Cc: [email protected]
>> > Signed-off-by: Stanislaw Gruszka <[email protected]>
>>
>> I trust that you saw the comments in the other thread. ?I have reverted
>> this patch and the related rt2x00 one.
> I responded there.
>
>> Perhaps there is some way to convince the hardware not to generate
>> spurious interrupts?
>
> Ivo, Gertjan, Helmut, is there a possibility that I can get hardware
> documentation or any other support from Ralink, that could help to
> solve this (and possible other issues)?

In the rt2x00 project we do have the early draft of the Specsheet for
the first 2800pci devices. However this falls in the category of specsheets
which is not yet publically published by Ralink, and hence we can't
distribute it further.

I've added Eddy Tsai to the recipientlist for this mail, he is the Ralink
developer, and he might have a better idea on this issue.

Eddy, we have an issue with rt2860 devices which are generating
spurious interrupts,
however how can we determine in our IRQ handler if the rt2860 generated the
interrupt and not another device on the same IRQ line?
In the driver released by Ralink the CSR is checked to see if any interrupt bit
is set, and if nothing is set IRQ_HANDLED is returned. However this means
we can return IRQ_HANDLED when the interrupt was triggered by another device
as well.
Is there a way to prevent rt2860 from firing spurious interrupts, or
is there a better
way to detect if rt2860 fired the interrupt?

Thanks,

Ivo

2011-11-23 06:33:27

by Michael Basse

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

Am 22.11.2011 23:10, schrieb Helmut Schaa:
> Why do you think the panics are produced by rt2800pci? The stacktraces
> attached to https://bugs.launchpad.net/ubuntu/+source/linux/+bug/869502
> don't show rt2x00 related functions ...
>
> Helmut

Hi Helmut,
i am not sure if the panics are related to rt2800pci. Because of the
following facts i thought it was related to that module.

- all systems facing that kind of panics are using rt2800pci
- with the newest compat wifi-drivers that kind of panics are gone
- panics are not happening when wifi is disabled (using fn-key to
disable wifi)
- i saw the patch from Stanislaw and thought it will fix the issue
because its related to rt2800pci

My problem is that i dont know how to do a better debbuging for that
kind of problem, so i was just trying some things around and thought the
issues are coming from rt2800pci. But all i am doing at the moment is
just guessing, because i am lost with that kind of problem.


Greetings

Michael




2011-11-16 10:52:24

by Ivo Van Doorn

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

On Wed, Nov 16, 2011 at 11:46 AM, Helmut Schaa
<[email protected]> wrote:
> On Wed, Nov 16, 2011 at 11:09 AM, Stanislaw Gruszka <[email protected]> wrote:
>> + ? ? ? /*
>> + ? ? ? ?* Some devices can generate interrupts with empty CSR register, we
>> + ? ? ? ?* "handle" such irq's to prevent interrupt controller treat them as
>> + ? ? ? ?* spurious interrupts and disable irq line.
>> + ? ? ? ?*/
>> ? ? ? ?if (!reg)
>> - ? ? ? ? ? ? ? return IRQ_NONE;
>> + ? ? ? ? ? ? ? return IRQ_HANDLED;
>
> What if this interrupt belongs to a different device on a shared interrupt line?
> In case reg is zero we can't be sure our device generated this interrupt, right?

In the bugzilla report Stanislaw indicated that even when rt2800pci
indicates IRQ_HANDLED,
all other drivers on the same interruptline will still get the interrupt event.

Ivo

2011-11-22 22:00:30

by John W. Linville

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

On Wed, Nov 16, 2011 at 11:09:17AM +0100, Stanislaw Gruszka wrote:
> Some devices may generate spurious interrupts, we have to handle them
> otherwise interrupt line will be disabled with below message and driver
> will not work:
>
> [ 2052.114334] irq 17: nobody cared (try booting with the "irqpoll" option)
> [ 2052.114339] Pid: 0, comm: swapper Tainted: P 2.6.35.6-48.fc14.x86_64 #1
> [ 2052.114341] Call Trace:
> [ 2052.114342] <IRQ> [<ffffffff810a6e2b>] __report_bad_irq.clone.1+0x3d/0x8b
> [ 2052.114349] [<ffffffff810a6f93>] note_interrupt+0x11a/0x17f
> [ 2052.114352] [<ffffffff810a7a73>] handle_fasteoi_irq+0xa8/0xce
> [ 2052.114355] [<ffffffff8100c2ea>] handle_irq+0x88/0x90
> [ 2052.114357] [<ffffffff8146f034>] do_IRQ+0x5c/0xb4
> [ 2052.114360] [<ffffffff81469593>] ret_from_intr+0x0/0x11
> [ 2052.114361] <EOI> [<ffffffff8102b7f9>] ? native_safe_halt+0xb/0xd
> [ 2052.114366] [<ffffffff81010f03>] ? need_resched+0x23/0x2d
> [ 2052.114367] [<ffffffff8101102a>] default_idle+0x34/0x4f
> [ 2052.114370] [<ffffffff81008325>] cpu_idle+0xaa/0xcc
> [ 2052.114373] [<ffffffff81461f2a>] start_secondary+0x24d/0x28e
> [ 2052.114374] handlers:
> [ 2052.114375] [<ffffffff81332944>] (usb_hcd_irq+0x0/0x7c)
> [ 2052.114378] [<ffffffffa00697da>] (rt2800pci_interrupt+0x0/0x18d [rt2800pci])
> [ 2052.114384] Disabling IRQ #17
>
> Resolve:
> https://bugzilla.redhat.com/show_bug.cgi?id=658451
>
> Reported-and-tested-by: Amir Hedayaty <[email protected]>
> Cc: [email protected]
> Signed-off-by: Stanislaw Gruszka <[email protected]>

I trust that you saw the comments in the other thread. I have reverted
this patch and the related rt2x00 one.

Perhaps there is some way to convince the hardware not to generate
spurious interrupts?

John
--
John W. Linville Someday the world will need a hero, and you
[email protected] might be all we have. Be ready.

2011-11-16 11:45:07

by Ivo Van Doorn

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

On Wed, Nov 16, 2011 at 12:17 PM, Stanislaw Gruszka <[email protected]> wrote:
> On Wed, Nov 16, 2011 at 11:14:41AM +0100, Ivo Van Doorn wrote:
>> > I changed rt2800pci only because get bug report for that device, not
>> > sure if similar changes are not needed for other rt2x00 PCI devices.
>>
>> The ticket mentiones that the end-result is a slow connection,
>> this is also a bug in older chipsets. So could you prepare a patch
>> for the other drivers as well?
> Yes, sure. Do you have handy links to bug reports that I could
> reference in the changelog?

Not really, I think most complains were on the mailinglist. There
are probably some bugreports, but I am not sure if that was the kernel
bugzilla, or on one of the distributions bugzilla's.

Ivo

2011-11-23 15:51:29

by Michael Basse

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts



Hi

Am 23.11.2011 08:35, schrieb Stanislaw Gruszka:
> Hi
>
> I do not think my patch fix this problem, but perhaps Gertjan's
> patch do this:
>
> http://marc.info/?l=linux-wireless&m=132112188308082&w=2
>
> Note, if you are hitting the same issue, what is fixed by this patch,
> it's enough to do "iwconfig wlan0 power off" to workaround the problem.
> So you can avoid applying the patch, and just use above command and
> check if issue is still reproducible.
>
> Stanislaw

Thank you very much for that feedback. I have posted that informations
on the ubuntu-bugtracker. I will do some testing with that workaround
and hopefully it is fixing the issue.

Greetings

Michael


2011-11-22 22:10:12

by Helmut Schaa

[permalink] [raw]
Subject: Re: Re: [PATCH] rt2800pci: handle spurious interrupts

Am Dienstag, 22. November 2011, 21:33:34 schrieb Michael Basse:
> Hopefully your patch is fixing the issues we are facing.

Why do you think the panics are produced by rt2800pci? The stacktraces
attached to https://bugs.launchpad.net/ubuntu/+source/linux/+bug/869502
don't show rt2x00 related functions ...

Helmut

2011-11-16 10:14:42

by Ivo Van Doorn

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

Hi,

On Wed, Nov 16, 2011 at 11:09 AM, Stanislaw Gruszka <[email protected]> wrote:
> Some devices may generate spurious interrupts, we have to handle them
> otherwise interrupt line will be disabled with below message and driver
> will not work:
>
> [ 2052.114334] irq 17: nobody cared (try booting with the "irqpoll" option)
> [ 2052.114339] Pid: 0, comm: swapper Tainted: P ? ? ? ? ? 2.6.35.6-48.fc14.x86_64 #1
> [ 2052.114341] Call Trace:
> [ 2052.114342] ?<IRQ> ?[<ffffffff810a6e2b>] __report_bad_irq.clone.1+0x3d/0x8b
> [ 2052.114349] ?[<ffffffff810a6f93>] note_interrupt+0x11a/0x17f
> [ 2052.114352] ?[<ffffffff810a7a73>] handle_fasteoi_irq+0xa8/0xce
> [ 2052.114355] ?[<ffffffff8100c2ea>] handle_irq+0x88/0x90
> [ 2052.114357] ?[<ffffffff8146f034>] do_IRQ+0x5c/0xb4
> [ 2052.114360] ?[<ffffffff81469593>] ret_from_intr+0x0/0x11
> [ 2052.114361] ?<EOI> ?[<ffffffff8102b7f9>] ? native_safe_halt+0xb/0xd
> [ 2052.114366] ?[<ffffffff81010f03>] ? need_resched+0x23/0x2d
> [ 2052.114367] ?[<ffffffff8101102a>] default_idle+0x34/0x4f
> [ 2052.114370] ?[<ffffffff81008325>] cpu_idle+0xaa/0xcc
> [ 2052.114373] ?[<ffffffff81461f2a>] start_secondary+0x24d/0x28e
> [ 2052.114374] handlers:
> [ 2052.114375] [<ffffffff81332944>] (usb_hcd_irq+0x0/0x7c)
> [ 2052.114378] [<ffffffffa00697da>] (rt2800pci_interrupt+0x0/0x18d [rt2800pci])
> [ 2052.114384] Disabling IRQ #17
>
> Resolve:
> https://bugzilla.redhat.com/show_bug.cgi?id=658451
>
> Reported-and-tested-by: Amir Hedayaty <[email protected]>
> Cc: [email protected]
> Signed-off-by: Stanislaw Gruszka <[email protected]>

Acked-by: Ivo van Doorn <[email protected]>

> ---
> I changed rt2800pci only because get bug report for that device, not
> sure if similar changes are not needed for other rt2x00 PCI devices.

The ticket mentiones that the end-result is a slow connection,
this is also a bug in older chipsets. So could you prepare a patch
for the other drivers as well?

> ?drivers/net/wireless/rt2x00/rt2800pci.c | ? ?7 ++++++-
> ?1 files changed, 6 insertions(+), 1 deletions(-)
>
> diff --git a/drivers/net/wireless/rt2x00/rt2800pci.c b/drivers/net/wireless/rt2x00/rt2800pci.c
> index da48c8a..4dc2d0f 100644
> --- a/drivers/net/wireless/rt2x00/rt2800pci.c
> +++ b/drivers/net/wireless/rt2x00/rt2800pci.c
> @@ -880,8 +880,13 @@ static irqreturn_t rt2800pci_interrupt(int irq, void *dev_instance)
> ? ? ? ?rt2x00pci_register_read(rt2x00dev, INT_SOURCE_CSR, &reg);
> ? ? ? ?rt2x00pci_register_write(rt2x00dev, INT_SOURCE_CSR, reg);
>
> + ? ? ? /*
> + ? ? ? ?* Some devices can generate interrupts with empty CSR register, we
> + ? ? ? ?* "handle" such irq's to prevent interrupt controller treat them as
> + ? ? ? ?* spurious interrupts and disable irq line.
> + ? ? ? ?*/
> ? ? ? ?if (!reg)
> - ? ? ? ? ? ? ? return IRQ_NONE;
> + ? ? ? ? ? ? ? return IRQ_HANDLED;
>
> ? ? ? ?if (!test_bit(DEVICE_STATE_ENABLED_RADIO, &rt2x00dev->flags))
> ? ? ? ? ? ? ? ?return IRQ_HANDLED;
> --
> 1.7.1
>
>

2011-11-23 08:02:46

by Stanislaw Gruszka

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

On Tue, Nov 22, 2011 at 04:49:40PM -0500, John W. Linville wrote:
> On Wed, Nov 16, 2011 at 11:09:17AM +0100, Stanislaw Gruszka wrote:
> > Some devices may generate spurious interrupts, we have to handle them
> > otherwise interrupt line will be disabled with below message and driver
> > will not work:
> >
> > [ 2052.114334] irq 17: nobody cared (try booting with the "irqpoll" option)
> > [ 2052.114339] Pid: 0, comm: swapper Tainted: P 2.6.35.6-48.fc14.x86_64 #1
> > [ 2052.114341] Call Trace:
> > [ 2052.114342] <IRQ> [<ffffffff810a6e2b>] __report_bad_irq.clone.1+0x3d/0x8b
> > [ 2052.114349] [<ffffffff810a6f93>] note_interrupt+0x11a/0x17f
> > [ 2052.114352] [<ffffffff810a7a73>] handle_fasteoi_irq+0xa8/0xce
> > [ 2052.114355] [<ffffffff8100c2ea>] handle_irq+0x88/0x90
> > [ 2052.114357] [<ffffffff8146f034>] do_IRQ+0x5c/0xb4
> > [ 2052.114360] [<ffffffff81469593>] ret_from_intr+0x0/0x11
> > [ 2052.114361] <EOI> [<ffffffff8102b7f9>] ? native_safe_halt+0xb/0xd
> > [ 2052.114366] [<ffffffff81010f03>] ? need_resched+0x23/0x2d
> > [ 2052.114367] [<ffffffff8101102a>] default_idle+0x34/0x4f
> > [ 2052.114370] [<ffffffff81008325>] cpu_idle+0xaa/0xcc
> > [ 2052.114373] [<ffffffff81461f2a>] start_secondary+0x24d/0x28e
> > [ 2052.114374] handlers:
> > [ 2052.114375] [<ffffffff81332944>] (usb_hcd_irq+0x0/0x7c)
> > [ 2052.114378] [<ffffffffa00697da>] (rt2800pci_interrupt+0x0/0x18d [rt2800pci])
> > [ 2052.114384] Disabling IRQ #17
> >
> > Resolve:
> > https://bugzilla.redhat.com/show_bug.cgi?id=658451
> >
> > Reported-and-tested-by: Amir Hedayaty <[email protected]>
> > Cc: [email protected]
> > Signed-off-by: Stanislaw Gruszka <[email protected]>
>
> I trust that you saw the comments in the other thread. I have reverted
> this patch and the related rt2x00 one.
I responded there.

> Perhaps there is some way to convince the hardware not to generate
> spurious interrupts?

Ivo, Gertjan, Helmut, is there a possibility that I can get hardware
documentation or any other support from Ralink, that could help to
solve this (and possible other issues)?

Stanislaw

2011-11-23 20:49:07

by David Miller

[permalink] [raw]
Subject: Re: [PATCH] rt2800pci: handle spurious interrupts

From: Stanislaw Gruszka <[email protected]>
Date: Wed, 23 Nov 2011 11:51:07 +0100

> On Wed, Nov 23, 2011 at 09:51:38AM +0100, Helmut Schaa wrote:
>> Are we 100% sure that the rt2800pci device is generating the interrupts?
> No, good point. There is also usb_hcd_irq() registered on that line.
>
> I wonder if there is method to control irq routing.

The IRQ routing is likely determined by the physical line traces on
your motherboard and therefore cannot be changed.