2019-05-23 09:12:13

by Pavel Machek

[permalink] [raw]
Subject: 5.2-rc1 on droid4: spi crash

Hi!

This was greeting me overnight... I don't yet know how reproducible it
is, it happened once so far.

Best regards,
Pavel

root@devuan:/sys/class/leds#
Message from syslogd@devuan at May 23 00:11:31 ...
kernel:[14889.641143] Internal error: Oops: 5 [#1] SMP ARM

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.638275] Process spi0 (pid: 100, stack limit =
0x97305d31)

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.650238] Stack: (0xedee7ed0 to 0xedee8000)

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.652069] 7ec0:
00000000 00000000 eba35c14 eba35c50

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.662841] 7ee0: edd93000 edd93360 ede3da50 c052683c
ffffffff edadcc10 edadcc9c 00000004

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.671051] 7f00: 00000000 ede3d800 600f0013 edd93000
eba35c14 eba35c50 edadcc10 edadcc10

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.679260] 7f20: 00000002 00000001 edd9322c c0526cb4
edd932a4 edb34600 edd932a0 edd932a4

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.687499] 7f40: 00000000 edd932d0 edd932a0 edd932a4
00000000 00000001 c0e82d14 edd932b4

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.695709] 7f60: c0e82d14 c0148f40 00000000 edca5580
00000000 ede39400 edca55a8 edd932a0

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.703918] 7f80: ed88dd48 c0148e74 00000000 c01491ec
ede39400 c01490dc 00000000 00000000

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.708251] 7fa0: 00000000 00000000 00000000 c01010e8
00000000 00000000 00000000 00000000

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.720367] 7fc0: 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.724334] 7fe0: 00000000 00000000 00000000 00000000
00000013 00000000 00000000 00000000

Message from syslogd@devuan at May 23 00:11:51 ...
kernel:[14909.807312] Code: e3a08c02 e5954034 e1a01005 e1a00007
(e5943008)
Write failed: Broken pipe
pavel@duo:/data/l/k$

--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html


Attachments:
(No filename) (2.34 kB)
signature.asc (188.00 B)
Digital signature
Download all attachments

2019-05-23 09:35:29

by Sebastian Reichel

[permalink] [raw]
Subject: Re: 5.2-rc1 on droid4: spi crash

Hi,

On Thu, May 23, 2019 at 11:09:26AM +0200, Pavel Machek wrote:
> This was greeting me overnight... I don't yet know how reproducible it
> is, it happened once so far.

Please pipe the stacktrace into ./scripts/decode_stacktrace.sh
to get a readable stacktrace, otherwise this is pretty much useless.
FWIW the only SPI device in the Droid 4 is the PMIC.

-- Sebastian

> root@devuan:/sys/class/leds#
> Message from syslogd@devuan at May 23 00:11:31 ...
> kernel:[14889.641143] Internal error: Oops: 5 [#1] SMP ARM
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.638275] Process spi0 (pid: 100, stack limit =
> 0x97305d31)
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.650238] Stack: (0xedee7ed0 to 0xedee8000)
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.652069] 7ec0:
> 00000000 00000000 eba35c14 eba35c50
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.662841] 7ee0: edd93000 edd93360 ede3da50 c052683c
> ffffffff edadcc10 edadcc9c 00000004
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.671051] 7f00: 00000000 ede3d800 600f0013 edd93000
> eba35c14 eba35c50 edadcc10 edadcc10
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.679260] 7f20: 00000002 00000001 edd9322c c0526cb4
> edd932a4 edb34600 edd932a0 edd932a4
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.687499] 7f40: 00000000 edd932d0 edd932a0 edd932a4
> 00000000 00000001 c0e82d14 edd932b4
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.695709] 7f60: c0e82d14 c0148f40 00000000 edca5580
> 00000000 ede39400 edca55a8 edd932a0
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.703918] 7f80: ed88dd48 c0148e74 00000000 c01491ec
> ede39400 c01490dc 00000000 00000000
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.708251] 7fa0: 00000000 00000000 00000000 c01010e8
> 00000000 00000000 00000000 00000000
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.720367] 7fc0: 00000000 00000000 00000000 00000000
> 00000000 00000000 00000000 00000000
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.724334] 7fe0: 00000000 00000000 00000000 00000000
> 00000013 00000000 00000000 00000000
>
> Message from syslogd@devuan at May 23 00:11:51 ...
> kernel:[14909.807312] Code: e3a08c02 e5954034 e1a01005 e1a00007
> (e5943008)
> Write failed: Broken pipe
> pavel@duo:/data/l/k$
>
> --
> (english) http://www.livejournal.com/~pavelmachek
> (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html



Attachments:
(No filename) (2.70 kB)
signature.asc (849.00 B)
Download all attachments

2019-05-27 06:08:39

by Tony Lindgren

[permalink] [raw]
Subject: Re: 5.2-rc1 on droid4: spi crash

Hi,

* Sebastian Reichel <[email protected]> [190523 09:33]:
> Hi,
>
> On Thu, May 23, 2019 at 11:09:26AM +0200, Pavel Machek wrote:
> > This was greeting me overnight... I don't yet know how reproducible it
> > is, it happened once so far.
>
> Please pipe the stacktrace into ./scripts/decode_stacktrace.sh
> to get a readable stacktrace, otherwise this is pretty much useless.
> FWIW the only SPI device in the Droid 4 is the PMIC.

I've seen this too, and looks like reverting commit c9ba7a16d0f1
("spi: Release spi_res after finalizing message") fixes it based
several days of testing.

Noralf and Mark, any ideas what needs to be fixed here?

Below is the stacktrace I see without c9ba7a16d0f1 reverted,
not sure how to reproduce but it seems to happen within about
one to two days of uptime.

Regards,

Tony

8< -----------------
Unable to handle kernel NULL pointer dereference at virtual address 00000008
pgd = 829f0a5b
[00000008] *pgd=00000000
Internal error: Oops: 80000005 [#1] SMP ARM
...
CPU: 0 PID: 71 Comm: spi0 Tainted: G W 5.2.0-rc1+ #5983
Hardware name: Generic OMAP4 (Flattened Device Tree)
PC is at 0x8
LR is at spi_res_release+0x54/0x80
pc : [<00000008>] lr : [<c06faa48>] psr: 20000113
sp : ed6e3e88 ip : ed6e3eb0 fp : ed6e3eac
r10: c0b9eca8 r9 : 00000100 r8 : 00000200
r7 : ed65bc00 r6 : ed6e5d3c r5 : ed6e5d0c r4 : c0d05254
r3 : 00000008 r2 : c0d05264 r1 : ed6e5d0c r0 : ed65bc00
Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
Control: 10c5387d Table: abf3c04a DAC: 00000051
Process spi0 (pid: 71, stack limit = 0x0ef66f65)
Stack: (0xed6e3e88 to 0xed6e4000)
3e80: ed6e5cd0 ed6e5d0c ed65bc00 c0daf080 00000000 ed510410
3ea0: ed6e3eec ed6e3eb0 c06fd8c4 c06faa00 ed65b800 00000000 ed65ba20 ed65bee0
3ec0: ed6e3eec ed65bc00 ed6e5cd0 ed6e5d0c 00000000 ed510410 ed510410 00000001
3ee0: ed6e3f2c ed6e3ef0 c06fdcd4 c06fd560 00000004 c0170948 ed6e3f20 ed65bdfc
3f00: ffffe000 ed65be68 ed65be44 ffffe000 c0dc7734 ed65be48 c0166f88 00000000
3f20: ed6e3f3c ed6e3f30 c06fe10c c06fd9a4 ed6e3f74 ed6e3f40 c0166f54 c06fe0f8
3f40: ed6e3f74 6eb8f9ff c0166780 00000000 ed3bccc0 ed659c00 ed6e2000 ed65be44
3f60: c0166eac ed115c44 ed6e3fac ed6e3f78 c0166e58 c0166eb8 ed3bccdc ed3bccdc
3f80: ed6e3fac ed659c00 c0166cf8 00000000 00000000 00000000 00000000 00000000
3fa0: 00000000 ed6e3fb0 c01010e8 c0166d04 00000000 00000000 00000000 00000000
3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
Backtrace:
[<c06fa9f4>] (spi_res_release) from [<c06fd8c4>] (spi_transfer_one_message+0x370/0x444)
r9:ed510410 r8:00000000 r7:c0daf080 r6:ed65bc00 r5:ed6e5d0c r4:ed6e5cd0
[<c06fd554>] (spi_transfer_one_message) from [<c06fdcd4>] (__spi_pump_messages+0x33c/0x754)
r10:00000001 r9:ed510410 r8:ed510410 r7:00000000 r6:ed6e5d0c r5:ed6e5cd0
r4:ed65bc00
[<c06fd998>] (__spi_pump_messages) from [<c06fe10c>] (spi_pump_messages+0x20/0x24)
r10:00000000 r9:c0166f88 r8:ed65be48 r7:c0dc7734 r6:ffffe000 r5:ed65be44
r4:ed65be68
[<c06fe0ec>] (spi_pump_messages) from [<c0166f54>] (kthread_worker_fn+0xa8/0x268)
[<c0166eac>] (kthread_worker_fn) from [<c0166e58>] (kthread+0x160/0x178)
r10:ed115c44 r9:c0166eac r8:ed65be44 r7:ed6e2000 r6:ed659c00 r5:ed3bccc0
r4:00000000
[<c0166cf8>] (kthread) from [<c01010e8>] (ret_from_fork+0x14/0x2c)
Exception stack(0xed6e3fb0 to 0xed6e3ff8)
3fa0: 00000000 00000000 00000000 00000000
3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c0166cf8
r4:ed659c00
Code: bad PC value
---[ end trace a8011e9722dfda5e ]---

2019-05-27 13:25:56

by Noralf Trønnes

[permalink] [raw]
Subject: Re: 5.2-rc1 on droid4: spi crash



Den 2019-05-27 07:53, skrev Tony Lindgren:
> Hi,
>
> * Sebastian Reichel <[email protected]> [190523 09:33]:
>> Hi,
>>
>> On Thu, May 23, 2019 at 11:09:26AM +0200, Pavel Machek wrote:
>> > This was greeting me overnight... I don't yet know how reproducible it
>> > is, it happened once so far.
>>
>> Please pipe the stacktrace into ./scripts/decode_stacktrace.sh
>> to get a readable stacktrace, otherwise this is pretty much useless.
>> FWIW the only SPI device in the Droid 4 is the PMIC.
>
> I've seen this too, and looks like reverting commit c9ba7a16d0f1
> ("spi: Release spi_res after finalizing message") fixes it based
> several days of testing.
>
> Noralf and Mark, any ideas what needs to be fixed here?

Mark has a revert in his for-5.2 branch:
spi: Fix Raspberry Pi breakage
https://git.kernel.org/pub/scm/linux/kernel/git/broonie/spi.git/commit/?h=for-5.2&id=0ed56252c9567351344cb7b5cff6140e1bcec943

I don't know when or if he has sent a pull request.
Sorry about the breakage.

Noralf.

>
> Below is the stacktrace I see without c9ba7a16d0f1 reverted,
> not sure how to reproduce but it seems to happen within about
> one to two days of uptime.
>
> Regards,
>
> Tony
>
> 8< -----------------
> Unable to handle kernel NULL pointer dereference at virtual address
> 00000008
> pgd = 829f0a5b
> [00000008] *pgd=00000000
> Internal error: Oops: 80000005 [#1] SMP ARM
> ...
> CPU: 0 PID: 71 Comm: spi0 Tainted: G W 5.2.0-rc1+ #5983
> Hardware name: Generic OMAP4 (Flattened Device Tree)
> PC is at 0x8
> LR is at spi_res_release+0x54/0x80
> pc : [<00000008>] lr : [<c06faa48>] psr: 20000113
> sp : ed6e3e88 ip : ed6e3eb0 fp : ed6e3eac
> r10: c0b9eca8 r9 : 00000100 r8 : 00000200
> r7 : ed65bc00 r6 : ed6e5d3c r5 : ed6e5d0c r4 : c0d05254
> r3 : 00000008 r2 : c0d05264 r1 : ed6e5d0c r0 : ed65bc00
> Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
> Control: 10c5387d Table: abf3c04a DAC: 00000051
> Process spi0 (pid: 71, stack limit = 0x0ef66f65)
> Stack: (0xed6e3e88 to 0xed6e4000)
> 3e80: ed6e5cd0 ed6e5d0c ed65bc00 c0daf080 00000000
> ed510410
> 3ea0: ed6e3eec ed6e3eb0 c06fd8c4 c06faa00 ed65b800 00000000 ed65ba20
> ed65bee0
> 3ec0: ed6e3eec ed65bc00 ed6e5cd0 ed6e5d0c 00000000 ed510410 ed510410
> 00000001
> 3ee0: ed6e3f2c ed6e3ef0 c06fdcd4 c06fd560 00000004 c0170948 ed6e3f20
> ed65bdfc
> 3f00: ffffe000 ed65be68 ed65be44 ffffe000 c0dc7734 ed65be48 c0166f88
> 00000000
> 3f20: ed6e3f3c ed6e3f30 c06fe10c c06fd9a4 ed6e3f74 ed6e3f40 c0166f54
> c06fe0f8
> 3f40: ed6e3f74 6eb8f9ff c0166780 00000000 ed3bccc0 ed659c00 ed6e2000
> ed65be44
> 3f60: c0166eac ed115c44 ed6e3fac ed6e3f78 c0166e58 c0166eb8 ed3bccdc
> ed3bccdc
> 3f80: ed6e3fac ed659c00 c0166cf8 00000000 00000000 00000000 00000000
> 00000000
> 3fa0: 00000000 ed6e3fb0 c01010e8 c0166d04 00000000 00000000 00000000
> 00000000
> 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000
> 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000
> 00000000
> Backtrace:
> [<c06fa9f4>] (spi_res_release) from [<c06fd8c4>]
> (spi_transfer_one_message+0x370/0x444)
> r9:ed510410 r8:00000000 r7:c0daf080 r6:ed65bc00 r5:ed6e5d0c
> r4:ed6e5cd0
> [<c06fd554>] (spi_transfer_one_message) from [<c06fdcd4>]
> (__spi_pump_messages+0x33c/0x754)
> r10:00000001 r9:ed510410 r8:ed510410 r7:00000000 r6:ed6e5d0c
> r5:ed6e5cd0
> r4:ed65bc00
> [<c06fd998>] (__spi_pump_messages) from [<c06fe10c>]
> (spi_pump_messages+0x20/0x24)
> r10:00000000 r9:c0166f88 r8:ed65be48 r7:c0dc7734 r6:ffffe000
> r5:ed65be44
> r4:ed65be68
> [<c06fe0ec>] (spi_pump_messages) from [<c0166f54>]
> (kthread_worker_fn+0xa8/0x268)
> [<c0166eac>] (kthread_worker_fn) from [<c0166e58>]
> (kthread+0x160/0x178)
> r10:ed115c44 r9:c0166eac r8:ed65be44 r7:ed6e2000 r6:ed659c00
> r5:ed3bccc0
> r4:00000000
> [<c0166cf8>] (kthread) from [<c01010e8>] (ret_from_fork+0x14/0x2c)
> Exception stack(0xed6e3fb0 to 0xed6e3ff8)
> 3fa0: 00000000 00000000 00000000
> 00000000
> 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000
> 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
> r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000
> r5:c0166cf8
> r4:ed659c00
> Code: bad PC value
> ---[ end trace a8011e9722dfda5e ]---

2019-05-27 13:32:45

by Tony Lindgren

[permalink] [raw]
Subject: Re: 5.2-rc1 on droid4: spi crash

* Noralf Trønnes <[email protected]> [190527 13:16]:
>
>
> Den 2019-05-27 07:53, skrev Tony Lindgren:
> > Hi,
> >
> > * Sebastian Reichel <[email protected]> [190523 09:33]:
> > > Hi,
> > >
> > > On Thu, May 23, 2019 at 11:09:26AM +0200, Pavel Machek wrote:
> > > > This was greeting me overnight... I don't yet know how reproducible it
> > > > is, it happened once so far.
> > >
> > > Please pipe the stacktrace into ./scripts/decode_stacktrace.sh
> > > to get a readable stacktrace, otherwise this is pretty much useless.
> > > FWIW the only SPI device in the Droid 4 is the PMIC.
> >
> > I've seen this too, and looks like reverting commit c9ba7a16d0f1
> > ("spi: Release spi_res after finalizing message") fixes it based
> > several days of testing.
> >
> > Noralf and Mark, any ideas what needs to be fixed here?
>
> Mark has a revert in his for-5.2 branch:
> spi: Fix Raspberry Pi breakage
> https://git.kernel.org/pub/scm/linux/kernel/git/broonie/spi.git/commit/?h=for-5.2&id=0ed56252c9567351344cb7b5cff6140e1bcec943

Oh OK good to hear.

> I don't know when or if he has sent a pull request.
> Sorry about the breakage.

No worries, thanks for the update.

Regards,

Tony