#regzbot introduced: v6.1.70..v6.1.71
After kernel upgrade 6.1.70 to 6.1.71, the computer hangs during shutdown.
The problem is related to NFS service. Stopping NFS service hangs:
# /etc/init.d/nfs stop
* Caching service dependencies ... [ ok ]
* Stopping NFS mountd ... [ ok ]
* Stopping NFS daemon ... [ ok ]
then it hangs
Shutdown does not hang when NFS service is removed.
# rc-update del nfs
I had this kernel error in the log:
Jan 10 17:32:25 [rpc.mountd] Caught signal 15, un-registering and exiting.
Jan 10 17:32:25 [kernel] [ 2005.560991] ------------[ cut here ]------------
Jan 10 17:32:25 [kernel] [ 2005.560996] kernel BUG at net/sunrpc/svc.c:576!
Jan 10 17:32:25 [kernel] [ 2005.561004] invalid opcode: 0000 [#1]
PREEMPT SMP PTI
Jan 10 17:32:25 [kernel] [ 2005.561012] CPU: 0 PID: 8079 Comm: nfsd
Tainted: P O 6.1.71-gentoo #1
Jan 10 17:32:25 [kernel] [ 2005.561017] Hardware name: Gigabyte
Technology Co., Ltd. X58A-UD3R/X58A-UD3R, BIOS FB 08/24/2010
Jan 10 17:32:25 [kernel] [ 2005.561020] RIP: 0010:svc_destroy+0x1f/0x56
Jan 10 17:32:25 [kernel] [ 2005.561033] Code: 5b 5d 41 5c 41 5d c3 cc cc
cc cc 55 48 8d 6f ec 53 48 89 fb 48 83 c7 44 e8 b6 6c 5f ff 48 8b 53 1c
48 8d 43 1c 48 39 c2 74 02 <0f> 0b 48 8b 53 2c 48 8d 43 2c 48 39 c2 74
02 0f 0b 48 89 ef e8 6b
Jan 10 17:32:25 [kernel] [ 2005.561038] RSP: 0018:ffffc90001edbee8
EFLAGS: 00010287
Jan 10 17:32:25 [kernel] [ 2005.561043] RAX: ffff88816c1a1c30 RBX:
ffff88816c1a1c14 RCX: 0000000000000000
Jan 10 17:32:25 [kernel] [ 2005.561047] RDX: ffff88813dfcc018 RSI:
0000000000000286 RDI: ffff88890bc9b9c0
Jan 10 17:32:25 [kernel] [ 2005.561051] RBP: ffff88816c1a1c00 R08:
ffff88810310b600 R09: 0000000000000000
Jan 10 17:32:25 [kernel] [ 2005.561055] R10: ffff8881bc3b4000 R11:
ffff8881bc3b4000 R12: ffffffff82e061c0
Jan 10 17:32:25 [kernel] [ 2005.561058] R13: ffff8881bc0e8000 R14:
ffff88810310b600 R15: ffffc90002323c80
Jan 10 17:32:25 [kernel] [ 2005.561062] FS: 0000000000000000(0000)
GS:ffff88890bc00000(0000) knlGS:0000000000000000
Jan 10 17:32:25 [kernel] [ 2005.561067] CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033
Jan 10 17:32:25 [kernel] [ 2005.561071] CR2: 000055e971e376f8 CR3:
000000000260a000 CR4: 00000000000006f0
Jan 10 17:32:25 [kernel] [ 2005.561075] Call Trace:
Jan 10 17:32:25 [kernel] [ 2005.561079] <TASK>
Jan 10 17:32:25 [kernel] [ 2005.561082] ? __die_body+0x15/0x57
Jan 10 17:32:25 [kernel] [ 2005.561091] ? die+0x2b/0x44
Jan 10 17:32:25 [kernel] [ 2005.561097] ? do_trap+0x76/0xf9
Jan 10 17:32:25 [kernel] [ 2005.561102] ? svc_destroy+0x1f/0x56
Jan 10 17:32:25 [kernel] [ 2005.561108] ? svc_destroy+0x1f/0x56
Jan 10 17:32:25 [kernel] [ 2005.561114] ? do_error_trap+0x69/0x93
Jan 10 17:32:25 [kernel] [ 2005.561119] ? svc_destroy+0x1f/0x56
Jan 10 17:32:25 [kernel] [ 2005.561126] ? exc_invalid_op+0x49/0x5d
Jan 10 17:32:25 [kernel] [ 2005.561133] ? svc_destroy+0x1f/0x56
Jan 10 17:32:25 [kernel] [ 2005.561139] ? asm_exc_invalid_op+0x16/0x20
Jan 10 17:32:25 [kernel] [ 2005.561148] ? svc_destroy+0x1f/0x56
Jan 10 17:32:25 [kernel] [ 2005.561155] ? svc_destroy+0x12/0x56
Jan 10 17:32:25 [kernel] [ 2005.561161] nfsd+0x13d/0x162
Jan 10 17:32:25 [kernel] [ 2005.561170] ? svc_put+0x2f/0x2f
Jan 10 17:32:25 [kernel] [ 2005.561176] kthread+0xd0/0xd8
Jan 10 17:32:25 [kernel] [ 2005.561183] ?
kthread_complete_and_exit+0x16/0x16
Jan 10 17:32:25 [kernel] [ 2005.561189] ret_from_fork+0x22/0x30
Jan 10 17:32:25 [kernel] [ 2005.561196] </TASK>
Jan 10 17:32:25 [kernel] [ 2005.561198] Modules linked in: snd_seq_dummy
snd_seq snd_seq_device nvidia_uvm(PO) bluetooth ecdh_generic ecc bridge
stp llc ipv6 crc_ccitt ch341 usbserial nvidia_drm(PO) nvidia_modeset(PO)
nvidia(PO) tda10048 tda8290 iTCO_wdt iTCO_vendor_support tda18271 it87
hwmon_vid dm_crypt coretemp dm_multipath dm_mod kvm_intel dax video
snd_hda_codec_realtek snd_hda_codec_generic kvm drm_kms_helper irqbypass
ledtrig_audio i2c_i801 pcspkr serio_raw drm i2c_smbus snd_hda_intel
i2c_core snd_intel_dspcfg fb_sys_fops syscopyarea snd_hda_codec
sysfillrect lpc_ich snd_hda_core sysimgblt mfd_core snd_hwdep uhci_hcd
rtc_cmos wmi
Jan 10 17:32:25 [kernel] [ 2005.561274] ---[ end trace 0000000000000000 ]---
Jan 10 17:32:25 [kernel] [ 2005.561277] RIP: 0010:svc_destroy+0x1f/0x56
Jan 10 17:32:25 [kernel] [ 2005.561284] Code: 5b 5d 41 5c 41 5d c3 cc cc
cc cc 55 48 8d 6f ec 53 48 89 fb 48 83 c7 44 e8 b6 6c 5f ff 48 8b 53 1c
48 8d 43 1c 48 39 c2 74 02 <0f> 0b 48 8b 53 2c 48 8d 43 2c 48 39 c2 74
02 0f 0b 48 89 ef e8 6b
The problem is always reproducible. Steps to reproduce:
1. Update to kernel 6.1.71
2. Start nfs service
3. Try to shutdown
4. The computer hangs
Reverting the following 3 commits fixed the problem in kernel 6.1.71:
f9a01938e07910224d4a2fd00583725d686c3f38
bb4f791cb2de1140d0fbcedfe9e791ff364021d7
03d68ffc48b94cc1e15bbf3b4f16f1e1e4fa286a
On Thu, Jan 11, 2024 at 07:20:02PM +1100, email200202 wrote:
> Reverting the following 3 commits fixed the problem in kernel 6.1.71:
>
> f9a01938e07910224d4a2fd00583725d686c3f38
> bb4f791cb2de1140d0fbcedfe9e791ff364021d7
> 03d68ffc48b94cc1e15bbf3b4f16f1e1e4fa286a
When sending us git ids, please show the full context so we have a hint
as to what they are. For this, it should be:
f9a01938e079 ("NFSD: fix possible oops when nfsd/pool_stats is closed.")
bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()")
03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
Do you also have these issues in the latest 6.6.y release? 6.7?
thanks,
greg k-h
Hi Greg
I'm sorry. This my first kernel report.
I didn't test 6.6.x and 6.7.x. I use only 6.1.x.
Best Regards
John G
On 11/1/24 20:30, Greg KH wrote:
> On Thu, Jan 11, 2024 at 07:20:02PM +1100, email200202 wrote:
>> Reverting the following 3 commits fixed the problem in kernel 6.1.71:
>>
>> f9a01938e07910224d4a2fd00583725d686c3f38
>> bb4f791cb2de1140d0fbcedfe9e791ff364021d7
>> 03d68ffc48b94cc1e15bbf3b4f16f1e1e4fa286a
> When sending us git ids, please show the full context so we have a hint
> as to what they are. For this, it should be:
>
> f9a01938e079 ("NFSD: fix possible oops when nfsd/pool_stats is closed.")
> bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()")
> 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
>
> Do you also have these issues in the latest 6.6.y release? 6.7?
>
> thanks,
>
> greg k-h
On Thu, Jan 11, 2024 at 09:10:39PM +1100, email200202 wrote:
> Hi Greg
>
> I'm sorry. This my first kernel report.
>
> I didn't test 6.6.x and 6.7.x.? I use only 6.1.x.
Can you do so?
thanks,
greg k-h
Hi Greg
Here is the test results for the latest versions of 6.6.x and 6.7.x
available in Gentoo portage
1- Stopping NFS service failed but, unlike 6.1.x, it did NOT hang.
# uname -r
6.6.11-gentoo
# /etc/init.d/nfs stop
* Stopping NFS mountd ... [ ok ]
* Stopping NFS daemon ...
* start-stop-daemon: 8 process(es) refused to stop [ !! ]
* Unexporting NFS directories ... [ ok ]
* ERROR: nfs failed to stop
# /etc/init.d/nfs start
* WARNING: nfs has already been started
# uname -r
6.7.0-gentoo
# /etc/init.d/nfs stop
* Stopping NFS mountd ... [ ok ]
* Stopping NFS daemon ...
* start-stop-daemon: 8 process(es) refused to stop [ !! ]
* Unexporting NFS directories ... [ ok ]
* ERROR: nfs failed to stop
# /etc/init.d/nfs start
* WARNING: nfs has already been started
2- Shutdown didn't hang in both of them
Best regards
John G
On 11/1/24 21:21, Greg KH wrote:
> On Thu, Jan 11, 2024 at 09:10:39PM +1100, email200202 wrote:
>> Hi Greg
>>
>> I'm sorry. This my first kernel report.
>>
>> I didn't test 6.6.x and 6.7.x. I use only 6.1.x.
> Can you do so?
>
> thanks,
>
> greg k-h
> On Jan 11, 2024, at 8:58 AM, email200202 <[email protected]> wrote:
>
> Hi Greg
>
> Here is the test results for the latest versions of 6.6.x and 6.7.x available in Gentoo portage
>
> 1- Stopping NFS service failed but, unlike 6.1.x, it did NOT hang.
>
> # uname -r
> 6.6.11-gentoo
> # /etc/init.d/nfs stop
> * Stopping NFS mountd ... [ ok ]
> * Stopping NFS daemon ...
> * start-stop-daemon: 8 process(es) refused to stop [ !! ]
> * Unexporting NFS directories ... [ ok ]
> * ERROR: nfs failed to stop
> # /etc/init.d/nfs start
> * WARNING: nfs has already been started
>
>
> # uname -r
> 6.7.0-gentoo
> # /etc/init.d/nfs stop
> * Stopping NFS mountd ... [ ok ]
> * Stopping NFS daemon ...
> * start-stop-daemon: 8 process(es) refused to stop [ !! ]
> * Unexporting NFS directories ... [ ok ]
> * ERROR: nfs failed to stop
> # /etc/init.d/nfs start
> * WARNING: nfs has already been started
>
> 2- Shutdown didn't hang in both of them
>
> Best regards
> John G
>
> On 11/1/24 21:21, Greg KH wrote:
>> On Thu, Jan 11, 2024 at 09:10:39PM +1100, email200202 wrote:
>>> Hi Greg
>>>
>>> I'm sorry. This my first kernel report.
>>>
>>> I didn't test 6.6.x and 6.7.x. I use only 6.1.x.
>> Can you do so?
>>
>> thanks,
>>
>> greg k-h
I haven't seen emails before this one. But here are the only
three new NFSD commits in v6.1.71:
f9a01938e079 NFSD: fix possible oops when nfsd/pool_stats is closed.
bb4f791cb2de nfsd: call nfsd_last_thread() before final nfsd_put()
03d68ffc48b9 nfsd: separate nfsd_last_thread() from nfsd_put()
And the latter two are directly related to NFSD shut down.
--
Chuck Lever
On Thu, 2024-01-11 at 10:30 +0100, Greg KH wrote:
> On Thu, Jan 11, 2024 at 07:20:02PM +1100, email200202 wrote:
> > Reverting the following 3 commits fixed the problem in kernel 6.1.71:
> >
> > f9a01938e07910224d4a2fd00583725d686c3f38
> > bb4f791cb2de1140d0fbcedfe9e791ff364021d7
> > 03d68ffc48b94cc1e15bbf3b4f16f1e1e4fa286a
>
> When sending us git ids, please show the full context so we have a hint
> as to what they are. For this, it should be:
>
> f9a01938e079 ("NFSD: fix possible oops when nfsd/pool_stats is closed.")
> bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()")
> 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
>
> Do you also have these issues in the latest 6.6.y release? 6.7?
>
>
I wasn't able to reproduce the exact same bug as the reporter, but I did
see a different panic due to the above patches. Some of those fixes
aren't appropriate without pulling in earlier commits, but I think it's
probably best to just drop these two:
bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()")
03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
Greg, does that sound OK?
Thanks,
--
Jeff Layton <[email protected]>
On Thu, Jan 11, 2024 at 12:26:39PM -0500, Jeff Layton wrote:
> On Thu, 2024-01-11 at 10:30 +0100, Greg KH wrote:
> > On Thu, Jan 11, 2024 at 07:20:02PM +1100, email200202 wrote:
> > > Reverting the following 3 commits fixed the problem in kernel 6.1.71:
> > >
> > > f9a01938e07910224d4a2fd00583725d686c3f38
> > > bb4f791cb2de1140d0fbcedfe9e791ff364021d7
> > > 03d68ffc48b94cc1e15bbf3b4f16f1e1e4fa286a
> >
> > When sending us git ids, please show the full context so we have a hint
> > as to what they are. For this, it should be:
> >
> > f9a01938e079 ("NFSD: fix possible oops when nfsd/pool_stats is closed.")
> > bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()")
> > 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
> >
> > Do you also have these issues in the latest 6.6.y release? 6.7?
> >
> >
>
> I wasn't able to reproduce the exact same bug as the reporter, but I did
> see a different panic due to the above patches. Some of those fixes
> aren't appropriate without pulling in earlier commits, but I think it's
> probably best to just drop these two:
>
> bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()")
> 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
>
> Greg, does that sound OK?
Sure, can you send reverts for them, or do you need me to make them?
thanks,
greg k-h
On Sat, Jan 13, 2024 at 10:35:09AM +0100, Greg KH wrote:
> On Thu, Jan 11, 2024 at 12:26:39PM -0500, Jeff Layton wrote:
> > On Thu, 2024-01-11 at 10:30 +0100, Greg KH wrote:
> > > On Thu, Jan 11, 2024 at 07:20:02PM +1100, email200202 wrote:
> > > > Reverting the following 3 commits fixed the problem in kernel 6.1.71:
> > > >
> > > > f9a01938e07910224d4a2fd00583725d686c3f38
> > > > bb4f791cb2de1140d0fbcedfe9e791ff364021d7
> > > > 03d68ffc48b94cc1e15bbf3b4f16f1e1e4fa286a
> > >
> > > When sending us git ids, please show the full context so we have a hint
> > > as to what they are. For this, it should be:
> > >
> > > f9a01938e079 ("NFSD: fix possible oops when nfsd/pool_stats is closed.")
> > > bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()")
> > > 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
> > >
> > > Do you also have these issues in the latest 6.6.y release? 6.7?
> > >
> > >
> >
> > I wasn't able to reproduce the exact same bug as the reporter, but I did
> > see a different panic due to the above patches. Some of those fixes
> > aren't appropriate without pulling in earlier commits, but I think it's
> > probably best to just drop these two:
> >
> > bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()")
> > 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
> >
> > Greg, does that sound OK?
>
> Sure, can you send reverts for them, or do you need me to make them?
Nevermind, I made them, thanks.
greg k-h
On 11.01.24 09:20, email200202 wrote:
>
> #regzbot introduced: v6.1.70..v6.1.71
>
>
> After kernel upgrade 6.1.70 to 6.1.71, the computer hangs during shutdown.
>
> The problem is related to NFS service. Stopping NFS service hangs:
>
> # /etc/init.d/nfs stop
> * Caching service dependencies ... [ ok ]
> * Stopping NFS mountd ... [ ok ]
> * Stopping NFS daemon ... [ ok ]
> [...]
#regzbot fix: b2c545c39877408a2fe2
#regzbot ignore-activity
Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
--
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
That page also explains what to do if mails like this annoy you.