Subject: [BUG][mvebu] mvneta: cannot request irq 25 on openblocks-ax3

Hi,

Here I've hit a bug on the recent kernel. As far as I know, this bug
exists on 3.9-rc1 too.

When I tried the latest mvebu for-next tree
(git://git.infradead.org/users/jcooper/linux.git mvebu/for-next),
I got below warning at bootup time and mvneta didn't work (link was never up).
I ensured that "ifconfig ethX up" always caused that.

Does anyone succeed to boot openblocks-ax3 recently or hit same
trouble?

------------[ cut here ]------------
WARNING: at /ssd/ksrc/linux-3/kernel/irq/manage.c:1417
request_threaded_irq+0x64/0x124()
Modules linked in:
Backtrace:
[<c00119a4>] (dump_backtrace+0x0/0x110) from [<c04b2bac>] (dump_stack+0x18/0x1c)
r6:c05ad11a r5:00000589 r4:00000000 r3:ed80e000
[<c04b2b94>] (dump_stack+0x0/0x1c) from [<c0021484>]
(warn_slowpath_common+0x54/0x70)
[<c0021430>] (warn_slowpath_common+0x0/0x70) from [<c00214c4>]
(warn_slowpath_null+0x24/0x2c)
r8:00000000 r7:ef0a4600 r6:ee9c4d40 r5:00000000 r4:c037378c
r3:00000009
[<c00214a0>] (warn_slowpath_null+0x0/0x2c) from [<c0084bb8>]
(request_threaded_irq+0x64/0x124)
[<c0084b54>] (request_threaded_irq+0x0/0x124) from [<c037418c>]
(mvneta_open+0x80/0x15c)
[<c037410c>] (mvneta_open+0x0/0x15c) from [<c03c9e3c>] (__dev_open+0xa0/0xf4)
r6:00001002 r5:c050bb24 r4:ee9c4800
[<c03c9d9c>] (__dev_open+0x0/0xf4) from [<c03ca070>] (__dev_change_flags+0x94/0x118)
r7:00000001 r6:00001002 r5:00001003 r4:ee9c4800
[<c03c9fdc>] (__dev_change_flags+0x0/0x118) from [<c03ca170>]
(dev_change_flags+0x18/0x4c)
r7:00000000 r6:edd0d380 r5:00001002 r4:ee9c4800
[<c03ca158>] (dev_change_flags+0x0/0x4c) from [<c03d6168>] (do_setlink+0x2a4/0x7a0)
r6:edd0d380 r5:ed80fb4c r4:ee9c4800 r3:00001002
[<c03d5ec4>] (do_setlink+0x0/0x7a0) from [<c03d72d8>] (rtnl_newlink+0x248/0x45c)
[<c03d7090>] (rtnl_newlink+0x0/0x45c) from [<c03d7060>]
(rtnetlink_rcv_msg+0x248/0x278)
[<c03d6e18>] (rtnetlink_rcv_msg+0x0/0x278) from [<c03ead78>]
(netlink_rcv_skb+0x58/0xb4)
[<c03ead20>] (netlink_rcv_skb+0x0/0xb4) from [<c03d5d28>] (rtnetlink_rcv+0x20/0x2c)
r6:eea75c00 r5:eea87240 r4:eea87240 r3:c03d5d08
[<c03d5d08>] (rtnetlink_rcv+0x0/0x2c) from [<c03ea780>]
(netlink_unicast+0x15c/0x214)
r4:ef0a3000 r3:c03d5d08
[<c03ea624>] (netlink_unicast+0x0/0x214) from [<c03eab94>]
(netlink_sendmsg+0x2b8/0x348)
[<c03ea8dc>] (netlink_sendmsg+0x0/0x348) from [<c03b2dbc>] (sock_sendmsg+0x88/0xa4)
[<c03b2d34>] (sock_sendmsg+0x0/0xa4) from [<c03b3330>] (__sys_sendmsg+0x1c8/0x25c)
r7:00000000 r6:00000020 r5:eefccc80 r4:ed80ff64
[<c03b3168>] (__sys_sendmsg+0x0/0x25c) from [<c03b4f30>] (sys_sendmsg+0x44/0x68)
[<c03b4eec>] (sys_sendmsg+0x0/0x68) from [<c000daa0>] (ret_fast_syscall+0x0/0x30)
r6:be9f367c r5:00000000 r4:5146f675
---[ end trace a8ceb54233e388b5 ]---
mvneta d0070000.ethernet eth0: cannot request irq 25

Thank you,
--
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: [email protected]



2013-03-19 13:33:30

by Jason Cooper

[permalink] [raw]
Subject: Re: [BUG][mvebu] mvneta: cannot request irq 25 on openblocks-ax3

On Tue, Mar 19, 2013 at 10:12:37PM +0900, Masami Hiramatsu wrote:
> Hi,
>
> Here I've hit a bug on the recent kernel. As far as I know, this bug
> exists on 3.9-rc1 too.
>
> When I tried the latest mvebu for-next tree
> (git://git.infradead.org/users/jcooper/linux.git mvebu/for-next),

FYI: that branch isn't stable, it's used as a merge-test of
arm-soc/for-next (also not stable) and any branches I am trying to push
upstream that day.

Gregory has a patch in the works for this. Hopefully he'll submit
it by the end of the week.

thx,

Jason.

2013-03-19 13:39:31

by Ezequiel Garcia

[permalink] [raw]
Subject: Re: [BUG][mvebu] mvneta: cannot request irq 25 on openblocks-ax3

Hi Masami,

On Tue, Mar 19, 2013 at 10:12:37PM +0900, Masami Hiramatsu wrote:
>
> Here I've hit a bug on the recent kernel. As far as I know, this bug
> exists on 3.9-rc1 too.
>
> When I tried the latest mvebu for-next tree
> (git://git.infradead.org/users/jcooper/linux.git mvebu/for-next),
> I got below warning at bootup time and mvneta didn't work (link was never up).
> I ensured that "ifconfig ethX up" always caused that.
>
> Does anyone succeed to boot openblocks-ax3 recently or hit same
> trouble?

This is a known bug. Gregory Clement already has a fix and he
will submit it soon. In case you need this fixed ASAP, I'm attaching
you a patch with a fix.

Please note the attached patch is not ready for mainline inclusion,
as I said Gregory will submit a cleaner version soon.

--
Ezequiel García, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com


Attachments:
(No filename) (901.00 B)
0001-net-mvneta-convert-to-percpu-interrupt.patch (1.76 kB)
Download all attachments
Subject: Re: Re: [BUG][mvebu] mvneta: cannot request irq 25 on openblocks-ax3

Hi Jason,

(2013/03/19 22:33), Jason Cooper wrote:
> On Tue, Mar 19, 2013 at 10:12:37PM +0900, Masami Hiramatsu wrote:
>> Hi,
>>
>> Here I've hit a bug on the recent kernel. As far as I know, this bug
>> exists on 3.9-rc1 too.
>>
>> When I tried the latest mvebu for-next tree
>> (git://git.infradead.org/users/jcooper/linux.git mvebu/for-next),
>
> FYI: that branch isn't stable, it's used as a merge-test of
> arm-soc/for-next (also not stable) and any branches I am trying to push
> upstream that day.

Thanks! could you tell me which branch is stable?
(however, I'd like to try new fixes/features on my device too :))

> Gregory has a patch in the works for this. Hopefully he'll submit
> it by the end of the week.

Thank you,

--
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: [email protected]

Subject: Re: Re: [BUG][mvebu] mvneta: cannot request irq 25 on openblocks-ax3

Hi Ezequiel,

(2013/03/19 22:39), Ezequiel Garcia wrote:
> Hi Masami,
>
> On Tue, Mar 19, 2013 at 10:12:37PM +0900, Masami Hiramatsu wrote:
>>
>> Here I've hit a bug on the recent kernel. As far as I know, this bug
>> exists on 3.9-rc1 too.
>>
>> When I tried the latest mvebu for-next tree
>> (git://git.infradead.org/users/jcooper/linux.git mvebu/for-next),
>> I got below warning at bootup time and mvneta didn't work (link was never up).
>> I ensured that "ifconfig ethX up" always caused that.
>>
>> Does anyone succeed to boot openblocks-ax3 recently or hit same
>> trouble?
>
> This is a known bug. Gregory Clement already has a fix and he
> will submit it soon. In case you need this fixed ASAP, I'm attaching
> you a patch with a fix.

Thanks! I'll try that.

> Please note the attached patch is not ready for mainline inclusion,
> as I said Gregory will submit a cleaner version soon.

Yeah, I look forward to it :)

Thank you,

--
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: [email protected]

2013-03-19 14:33:10

by Jason Cooper

[permalink] [raw]
Subject: Re: Re: [BUG][mvebu] mvneta: cannot request irq 25 on openblocks-ax3

On Tue, Mar 19, 2013 at 11:19:05PM +0900, Masami Hiramatsu wrote:
> Hi Jason,
>
> (2013/03/19 22:33), Jason Cooper wrote:
> > On Tue, Mar 19, 2013 at 10:12:37PM +0900, Masami Hiramatsu wrote:
> >> Hi,
> >>
> >> Here I've hit a bug on the recent kernel. As far as I know, this bug
> >> exists on 3.9-rc1 too.
> >>
> >> When I tried the latest mvebu for-next tree
> >> (git://git.infradead.org/users/jcooper/linux.git mvebu/for-next),
> >
> > FYI: that branch isn't stable, it's used as a merge-test of
> > arm-soc/for-next (also not stable) and any branches I am trying to push
> > upstream that day.
>
> Thanks! could you tell me which branch is stable?
> (however, I'd like to try new fixes/features on my device too :))

Generally, we advise using one of Linus' tags (eg v3.9-rc3). Only if
that doesn't work, seek out a branch containing the fix/feature. When
in doubt, I'll do:

$ ./scripts/get_maintainer.pl -f drivers/net/ethernet/marvell/mvneta.c

and ask those listed which branch might contain a fix or feature.

In this case, it'll probably go through David Miller and the netdev
mailinglist, but it isn't there yet. ;-)

hth,

Jason.

2013-03-19 16:51:31

by Gregory CLEMENT

[permalink] [raw]
Subject: Re: [BUG][mvebu] mvneta: cannot request irq 25 on openblocks-ax3

On 03/19/2013 03:19 PM, Masami Hiramatsu wrote:
> Hi Ezequiel,
>
> (2013/03/19 22:39), Ezequiel Garcia wrote:
>> Hi Masami,
>>
>> On Tue, Mar 19, 2013 at 10:12:37PM +0900, Masami Hiramatsu wrote:
>>>
>>> Here I've hit a bug on the recent kernel. As far as I know, this bug
>>> exists on 3.9-rc1 too.
>>>
>>> When I tried the latest mvebu for-next tree
>>> (git://git.infradead.org/users/jcooper/linux.git mvebu/for-next),
>>> I got below warning at bootup time and mvneta didn't work (link was never up).
>>> I ensured that "ifconfig ethX up" always caused that.
>>>
>>> Does anyone succeed to boot openblocks-ax3 recently or hit same
>>> trouble?
>>
>> This is a known bug. Gregory Clement already has a fix and he
>> will submit it soon. In case you need this fixed ASAP, I'm attaching
>> you a patch with a fix.
>
> Thanks! I'll try that.
>
>> Please note the attached patch is not ready for mainline inclusion,
>> as I said Gregory will submit a cleaner version soon.
>
> Yeah, I look forward to it :)

Hi Masami,

You can try this patch if you want.
I don't have the hardware today so I didn't test it.
If you (and also Florian and Ezequiel) can test it and if it fixed
the bug, then I will be able send a proper email for it,

Thanks,
--
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com


Attachments:
0001-net-mvneta-convert-to-local-interrupt.patch (3.07 kB)

2013-03-19 19:49:11

by Florian Fainelli

[permalink] [raw]
Subject: Re: [BUG][mvebu] mvneta: cannot request irq 25 on openblocks-ax3

On Tuesday 19 March 2013 17:43:49 Gregory CLEMENT wrote:
>
> Hi Masami,
>
> You can try this patch if you want.
> I don't have the hardware today so I didn't test it.
> If you (and also Florian and Ezequiel) can test it and if it fixed
> the bug, then I will be able send a proper email for it,

I just tested your patch on RD-A370-A1 and DB-MV784MP-GP and it works fine,
thanks! Feel free to add my Tested-by: Florian Fainelli <[email protected]>
to your next submission.

Thanks!
--
Florian

2013-03-20 02:11:53

by Ryan Press

[permalink] [raw]
Subject: Re: [BUG][mvebu] mvneta: cannot request irq 25 on openblocks-ax3

Hi Gregory,

>On 03/19/2013 05:43 PM, Gregory CLEMENT wrote:
>> On Tue, Mar 19, 2013 at 10:12:37PM +0900, Masami Hiramatsu wrote:
>>>
>>> Here I've hit a bug on the recent kernel. As far as I know, this bug
>>> exists on 3.9-rc1 too.
>>>
>>> When I tried the latest mvebu for-next tree
>>> (git://git.infradead.org/users/jcooper/linux.git mvebu/for-next),
>>> I got below warning at bootup time and mvneta didn't work (link was never up).
>>> I ensured that "ifconfig ethX up" always caused that.
>>>
>>> Does anyone succeed to boot openblocks-ax3 recently or hit same
>>> trouble?
>
>Hi Masami,
>
>You can try this patch if you want.
>I don't have the hardware today so I didn't test it.
>If you (and also Florian and Ezequiel) can test it and if it fixed
>the bug, then I will be able send a proper email for it,

Thanks for this! I've been trying to build 3.9-rc1 on my Mirabox, and
I had the same problem as Masami. The patch allows mvneta to be
built-in to the kernel, but built as a module it gives the linker
error below:

ERROR: "request_percpu_irq" [drivers/net/ethernet/marvell/mvneta.ko] undefined!
ERROR: "free_percpu_irq" [drivers/net/ethernet/marvell/mvneta.ko] undefined!

Linking it into the kernel and installing allows the network
interfaces to come up, and they can receive traffic, but oddly enough
even though tcpdump shows that traffic is transmitting and the link
lights blink, the other side does not see any traffic. I've tested
this with different computers and different link speeds to no avail.
The fact that the link lights blink when transmitting makes me think
it's not a firewall problem (and that this rootfs works with another
arm box), but I'm not sure where to look next. Any idea?

Thanks,
Ryan

2013-03-20 15:09:10

by Gregory CLEMENT

[permalink] [raw]
Subject: Re: [BUG][mvebu] mvneta: cannot request irq 25 on openblocks-ax3

On 03/19/2013 08:48 PM, Florian Fainelli wrote:
> On Tuesday 19 March 2013 17:43:49 Gregory CLEMENT wrote:
>>
>> Hi Masami,
>>
>> You can try this patch if you want.
>> I don't have the hardware today so I didn't test it.
>> If you (and also Florian and Ezequiel) can test it and if it fixed
>> the bug, then I will be able send a proper email for it,
>
> I just tested your patch on RD-A370-A1 and DB-MV784MP-GP and it works fine,
> thanks! Feel free to add my Tested-by: Florian Fainelli <[email protected]>
> to your next submission.
>
> Thanks!
>
Thanks for having tested it, but with this patch there is still an issue when
we try to release the network interface. In this case the the interrupt line
is not properly freed.
Unfortunately the state of the percpu_irq API prevents the driver to use it.

Indeed the interrupts have to be freed when the .stop() function is
called. As the free_percpu_irq() function don't disable the interrupt
line, we have to do it on each CPU before calling this. The function
disable_percpu_irq() only disable the percpu on the current CPU and
there is no function which allows to disable a percpu irq on a given
CPU. Waiting for the extension of the percpu_irq API, this fix allows
to use again the mvneta driver.

I have just wrote and tested a new patch to fix this issue by just go back
on the old way we managed the irq for the mvenat as global irq and not
as local ones.

I am going to send the fix on the LAKML in a few minutes.

Thanks,

Gregory

Subject: Re: Re: [BUG][mvebu] mvneta: cannot request irq 25 on openblocks-ax3

(2013/03/20 1:43), Gregory CLEMENT wrote:
> Hi Masami,
>
> You can try this patch if you want.
> I don't have the hardware today so I didn't test it.
> If you (and also Florian and Ezequiel) can test it and if it fixed
> the bug, then I will be able send a proper email for it,

Hi Gregory,

Thank you for giving me the latest patch.
I've tested it on OpenBlocks-AX4 and that fixed the problem!

Tested-by: Masami Hiramatsu <[email protected]>

Thanks,
--
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: [email protected]