2017-08-30 20:15:57

by Pasi Kärkkäinen

[permalink] [raw]
Subject: Status of reverted Linux patch "tty: Fix ldisc crash on reopened tty", Linux 4.9 kernel frequent crashes

Hello everyone,

Recently Nathan March reported on centos-virt list he's getting frequent Linux kernel crashes with Linux 4.9 LTS kernel because of the missing patch "tty: Fix ldisc crash on reopened tty".

The patch was already merged upstream here:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=71472fa9c52b1da27663c275d416d8654b905f05

but then reverted here:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=896d81fefe5d1919537db2c2150ab6384e4a6610

Nathan confirmed if he applies the patch from 71472fa9c52b1da27663c275d416d8654b905f05 to his Linux 4.9 LTS kernel the bug/problem goes away, so the patch (or similar fix) is still needed, at least for 4.9 LTS kernel.


Mikulas reported he's able to trigger the same crash on Linux 4.10:
https://www.spinics.net/lists/kernel/msg2440637.html
https://lists.gt.net/linux/kernel/2664604?search_string=ldisc%20reopened;#2664604

Michael Neuling reported he's able to trigger the bug on PowerPC:
https://lkml.org/lkml/2017/3/10/1582


So now the question is.. is anyone currently working on getting this patch fixed and applied upstream? I think one of the problems earlier was being able to reliable reproduce the crash.. Nathan says he's able to reproduce it many times per week on his environment on x86_64.


Thanks a lot,

-- Pasi


2017-08-31 04:35:59

by Greg KH

[permalink] [raw]
Subject: Re: Status of reverted Linux patch "tty: Fix ldisc crash on reopened tty", Linux 4.9 kernel frequent crashes

On Wed, Aug 30, 2017 at 11:10:14PM +0300, Pasi K?rkk?inen wrote:
> Hello everyone,
>
> Recently Nathan March reported on centos-virt list he's getting frequent Linux kernel crashes with Linux 4.9 LTS kernel because of the missing patch "tty: Fix ldisc crash on reopened tty".

Crashes with "normal" operation, or crashes when running a fuzzer or
other type of program?

> The patch was already merged upstream here:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=71472fa9c52b1da27663c275d416d8654b905f05
>
> but then reverted here:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=896d81fefe5d1919537db2c2150ab6384e4a6610
>
> Nathan confirmed if he applies the patch from 71472fa9c52b1da27663c275d416d8654b905f05 to his Linux 4.9 LTS kernel the bug/problem goes away, so the patch (or similar fix) is still needed, at least for 4.9 LTS kernel.
>
>
> Mikulas reported he's able to trigger the same crash on Linux 4.10:
> https://www.spinics.net/lists/kernel/msg2440637.html
> https://lists.gt.net/linux/kernel/2664604?search_string=ldisc%20reopened;#2664604
>
> Michael Neuling reported he's able to trigger the bug on PowerPC:
> https://lkml.org/lkml/2017/3/10/1582
>
>
> So now the question is.. is anyone currently working on getting this patch fixed and applied upstream? I think one of the problems earlier was being able to reliable reproduce the crash.. Nathan says he's able to reproduce it many times per week on his environment on x86_64.

I don't know of anyone working on it, want to do it yourself?

thanks,

greg k-h

2017-08-31 05:22:08

by Michael Neuling

[permalink] [raw]
Subject: Re: Status of reverted Linux patch "tty: Fix ldisc crash on reopened tty", Linux 4.9 kernel frequent crashes

On Thu, 2017-08-31 at 06:36 +0200, Greg Kroah-Hartman wrote:
> On Wed, Aug 30, 2017 at 11:10:14PM +0300, Pasi Kärkkäinen wrote:
> > Hello everyone,
> >
> > Recently Nathan March reported on centos-virt list he's getting frequent
> > Linux kernel crashes with Linux 4.9 LTS kernel because of the missing patch
> > "tty: Fix ldisc crash on reopened tty".
>
> Crashes with "normal" operation, or crashes when running a fuzzer or
> other type of program?

For me it crashed on boot.

>
> > The patch was already merged upstream here:
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i
> > d=71472fa9c52b1da27663c275d416d8654b905f05
> >
> > but then reverted here:
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i
> > d=896d81fefe5d1919537db2c2150ab6384e4a6610
> >
> > Nathan confirmed if he applies the patch from
> > 71472fa9c52b1da27663c275d416d8654b905f05 to his Linux 4.9 LTS kernel the
> > bug/problem goes away, so the patch (or similar fix) is still needed, at
> > least for 4.9 LTS kernel.
> >
> >
> > Mikulas reported he's able to trigger the same crash on Linux 4.10:
> > https://www.spinics.net/lists/kernel/msg2440637.html
> > https://lists.gt.net/linux/kernel/2664604?search_string=ldisc%20reopened;#26
> > 64604
> >
> > Michael Neuling reported he's able to trigger the bug on PowerPC:
> > https://lkml.org/lkml/2017/3/10/1582
> >
> >
> > So now the question is.. is anyone currently working on getting this patch
> > fixed and applied upstream? I think one of the problems earlier was being
> > able to reliable reproduce the crash.. Nathan says he's able to reproduce it
> > many times per week on his environment on x86_64.
>
> I don't know of anyone working on it, want to do it yourself?

I'm not anymore. We found it was only triggered on a bogus CONFIG option
combination. Once we removed that, it no longer happened.

The underlying bug was still there though.

Mikey

2017-09-01 13:00:02

by Pasi Kärkkäinen

[permalink] [raw]
Subject: Re: Status of reverted Linux patch "tty: Fix ldisc crash on reopened tty", Linux 4.9 kernel frequent crashes

On Thu, Aug 31, 2017 at 03:22:05PM +1000, Michael Neuling wrote:
> On Thu, 2017-08-31 at 06:36 +0200, Greg Kroah-Hartman wrote:
> > On Wed, Aug 30, 2017 at 11:10:14PM +0300, Pasi K?rkk?inen wrote:
> > > Hello everyone,
> > >
> > > Recently Nathan March reported on centos-virt list he's getting frequent
> > > Linux kernel crashes with Linux 4.9 LTS kernel because of the missing patch
> > > "tty: Fix ldisc crash on reopened tty".
> >
> > Crashes with "normal" operation, or crashes when running a fuzzer or
> > other type of program?
>
> For me it crashed on boot.
>

Nathan said he's getting the crashes at runtime, randomly, but often.


> >
> > > The patch was already merged upstream here:
> > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i
> > > d=71472fa9c52b1da27663c275d416d8654b905f05
> > >
> > > but then reverted here:
> > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i
> > > d=896d81fefe5d1919537db2c2150ab6384e4a6610
> > >
> > > Nathan confirmed if he applies the patch from
> > > 71472fa9c52b1da27663c275d416d8654b905f05 to his Linux 4.9 LTS kernel the
> > > bug/problem goes away, so the patch (or similar fix) is still needed, at
> > > least for 4.9 LTS kernel.
> > >
> > >
> > > Mikulas reported he's able to trigger the same crash on Linux 4.10:
> > > https://www.spinics.net/lists/kernel/msg2440637.html
> > > https://lists.gt.net/linux/kernel/2664604?search_string=ldisc%20reopened;#26
> > > 64604
> > >
> > > Michael Neuling reported he's able to trigger the bug on PowerPC:
> > > https://lkml.org/lkml/2017/3/10/1582
> > >
> > >
> > > So now the question is.. is anyone currently working on getting this patch
> > > fixed and applied upstream? I think one of the problems earlier was being
> > > able to reliable reproduce the crash.. Nathan says he's able to reproduce it
> > > many times per week on his environment on x86_64.
> >
> > I don't know of anyone working on it, want to do it yourself?
>
> I'm not anymore. We found it was only triggered on a bogus CONFIG option
> combination. Once we removed that, it no longer happened.
>
> The underlying bug was still there though.
>


Yep.. and the bug seems to trigger at runtime.



> Mikey


-- Pasi

2017-09-01 20:48:29

by Mikulas Patocka

[permalink] [raw]
Subject: Re: Status of reverted Linux patch "tty: Fix ldisc crash on reopened tty", Linux 4.9 kernel frequent crashes



On Thu, 31 Aug 2017, Greg Kroah-Hartman wrote:

> On Wed, Aug 30, 2017 at 11:10:14PM +0300, Pasi K?rkk?inen wrote:
> > Hello everyone,
> >
> > Recently Nathan March reported on centos-virt list he's getting frequent Linux kernel crashes with Linux 4.9 LTS kernel because of the missing patch "tty: Fix ldisc crash on reopened tty".
>
> Crashes with "normal" operation, or crashes when running a fuzzer or
> other type of program?

I can crash it reliably (in a few tries), if I use an old Debian 5
userspace on PA-RISC. The crash happens when I connect to the machine with
ssh and type something to the terminal before the prompt appears.

Mikulas

> > The patch was already merged upstream here:
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=71472fa9c52b1da27663c275d416d8654b905f05
> >
> > but then reverted here:
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=896d81fefe5d1919537db2c2150ab6384e4a6610
> >
> > Nathan confirmed if he applies the patch from 71472fa9c52b1da27663c275d416d8654b905f05 to his Linux 4.9 LTS kernel the bug/problem goes away, so the patch (or similar fix) is still needed, at least for 4.9 LTS kernel.
> >
> >
> > Mikulas reported he's able to trigger the same crash on Linux 4.10:
> > https://www.spinics.net/lists/kernel/msg2440637.html
> > https://lists.gt.net/linux/kernel/2664604?search_string=ldisc%20reopened;#2664604
> >
> > Michael Neuling reported he's able to trigger the bug on PowerPC:
> > https://lkml.org/lkml/2017/3/10/1582
> >
> >
> > So now the question is.. is anyone currently working on getting this patch fixed and applied upstream? I think one of the problems earlier was being able to reliable reproduce the crash.. Nathan says he's able to reproduce it many times per week on his environment on x86_64.
>
> I don't know of anyone working on it, want to do it yourself?
>
> thanks,
>
> greg k-h
>