2023-03-01 06:26:08

by François Valenduc

[permalink] [raw]
Subject: Hangs with NetworkManager and rtw8723_de with kernel 6.1

Good morning,

systemrescue recently switched from kernel 5.15 to kernel 6.1 and
NetworkManager started to hangs at startup. This seems to be linked to
my wifi card (RTL8723DE). I am also able to reproduce the problem under
gentoo with the same versions for NetworkManager and linux-firmware than
those used in systemrescue (1.42.2 and 20230210). Under gentoo, the
problem appears with kernel 6.1.14 and 6.2.1. I am able to use
NetworkManager and systemrescue without problem on another computer
which has another wifi card.

Does anybody have a solution to this problem ?

Thanks in advance for your help,

Best regards,

François Valenduc



2023-03-01 16:40:15

by Larry Finger

[permalink] [raw]
Subject: Re: Hangs with NetworkManager and rtw8723_de with kernel 6.1

On 3/1/23 00:26, François Valenduc wrote:
> systemrescue recently switched from kernel 5.15 to kernel 6.1 and NetworkManager
> started to hangs at startup. This seems to be linked to my wifi card
> (RTL8723DE). I am also able to reproduce the problem under gentoo with the same
> versions for NetworkManager and linux-firmware than those used in systemrescue
> (1.42.2 and 20230210). Under gentoo, the problem appears with kernel 6.1.14 and
> 6.2.1. I am able to use NetworkManager and systemrescue without problem on
> another computer which has another wifi card.
>
> Does anybody have a solution to this problem ?

François,

Without any log information regarding the specific hang, it will be difficult to
debug this issue.

Have you installed the drivers from the rtw88 repo at GitHub.com? If so, they
may be interfering with the drivers in the kernel. You can test to see if this
is the problem by doing an edit of the command line while booting. When GRUB is
showing the kernel you wish to boot, press "e". This should bring up a minimal
editor. Scroll down until you find the stanza pertaining to the kernel you are
booting. It will start with "menuentry 'xxxxx'", where the x's are replaced with
the description of the kernel you want.

Once you find that entry, scroll down to the line that starts with "linuxefi" or
"linux", then move to the end of that line, and add
"rd.driver.blacklist=rtw_8723de" to blacklist the driver from GitHub, or
"rd.driver.blacklist=rtw88_8723de" to blacklist the kernel version. Once you
have added the new info, press the F10 key yo continue the boot.

If one of those lets the computer boot, add that driver to a blacklist file in
/usr/lib/modprobe.d/, or wherever your distro keeps the modprobe.d directory.

If neither work, then report back, and we will try to get a dump of the logs
when the hand happens.

Larry


2023-03-01 19:15:42

by François Valenduc

[permalink] [raw]
Subject: Re: Hangs with NetworkManager and rtw8723_de with kernel 6.1


Le 1/03/23 à 17:40, Larry Finger a écrit :
> On 3/1/23 00:26, François Valenduc wrote:
>> systemrescue recently switched from kernel 5.15 to kernel 6.1 and
>> NetworkManager started to hangs at startup. This seems to be linked
>> to my wifi card (RTL8723DE). I am also able to reproduce the problem
>> under gentoo with the same versions for NetworkManager and
>> linux-firmware than those used in systemrescue (1.42.2 and 20230210).
>> Under gentoo, the problem appears with kernel 6.1.14 and 6.2.1. I am
>> able to use NetworkManager and systemrescue without problem on
>> another computer which has another wifi card.
>>
>> Does anybody have a solution to this problem ?
>
> François,
>
> Without any log information regarding the specific hang, it will be
> difficult to debug this issue.
>
> Have you installed the drivers from the rtw88 repo at GitHub.com? If
> so, they may be interfering with the drivers in the kernel. You can
> test to see if this is the problem by doing an edit of the command
> line while booting. When GRUB is showing the kernel you wish to boot,
> press "e". This should bring up a minimal editor. Scroll down until
> you find the stanza pertaining to the kernel you are booting. It will
> start with "menuentry 'xxxxx'", where the x's are replaced with the
> description of the kernel you want.
>
> Once you find that entry, scroll down to the line that starts with
> "linuxefi" or "linux", then move to the end of that line, and add
> "rd.driver.blacklist=rtw_8723de" to blacklist the driver from GitHub, or
> "rd.driver.blacklist=rtw88_8723de" to blacklist the kernel version.
> Once you have added the new info, press the F10 key yo continue the boot.
>
> If one of those lets the computer boot, add that driver to a blacklist
> file in /usr/lib/modprobe.d/, or wherever your distro keeps the
> modprobe.d directory.
>
> If neither work, then report back, and we will try to get a dump of
> the logs when the hand happens.
>
> Larry
>
Good evening,

For sure, I am using the kernel driver in gentoo or in systemrescue. Can
you explain how I could obtains the logs when the crash occurs ?

François


2023-03-01 21:09:55

by Larry Finger

[permalink] [raw]
Subject: Re: Hangs with NetworkManager and rtw8723_de with kernel 6.1

On 3/1/23 13:15, François Valenduc wrote:
> Good evening,
>
> For sure, I am using the kernel driver in gentoo or in systemrescue. Can you
> explain how I could obtains the logs when the crash occurs ?

François,

The best way is to use netconsole, which requires another Linux computer
(target) connected to your network, and a wired connection on the computer under
test. The documentation is at
https://www.kernel.org/doc/html/latest/networking/netconsole.html.

That writeup is not easy to follow, but you need to do the following:
1. Install netconsole or nc depending on your distro on your target computer.
2. Run the command 'nc -u -l -p 6665' or 'netconsole -u -l -p 6665' depending on
your distro.
3. Learn the IP address of the target computer using 'ip address'.
4. Know the network name of your wired connection on the test computer.
5. Create an options file in /usr/lib/modprobe.d/50-netconsole.conf. This line
should contain the following line:
options netconsole netconsole="@/<name from step4>,@<address from step 3>/"
My computer contains options netconsole netconsole="@/enp0s25,@192.168.1.51/"
6. Force module netconsole to be loaded at boot. How to do that is distro
dependent. You will need to explore that on your own.

Reboot the test computer in the configuration that fails. If setup correctly,
the console dump will appear on the target computer.

Good luck.

Larry



2023-03-02 01:28:28

by Ping-Ke Shih

[permalink] [raw]
Subject: RE: Hangs with NetworkManager and rtw8723_de with kernel 6.1



> -----Original Message-----
> From: François Valenduc <[email protected]>
> Sent: Wednesday, March 1, 2023 2:26 PM
> To: [email protected]
> Subject: Hangs with NetworkManager and rtw8723_de with kernel 6.1
>
> Good morning,
>
> systemrescue recently switched from kernel 5.15 to kernel 6.1 and
> NetworkManager started to hangs at startup. This seems to be linked to
> my wifi card (RTL8723DE). I am also able to reproduce the problem under
> gentoo with the same versions for NetworkManager and linux-firmware than
> those used in systemrescue (1.42.2 and 20230210). Under gentoo, the
> problem appears with kernel 6.1.14 and 6.2.1. I am able to use
> NetworkManager and systemrescue without problem on another computer
> which has another wifi card.
>
> Does anybody have a solution to this problem ?
>

I think this problem is the same as
https://bugzilla.kernel.org/show_bug.cgi?id=217016

Ping-Ke


2023-03-02 06:24:35

by François Valenduc

[permalink] [raw]
Subject: Re: Hangs with NetworkManager and rtw8723_de with kernel 6.1


Le 2/03/23 à 02:27, Ping-Ke Shih a écrit :
>
>> -----Original Message-----
>> From: François Valenduc <[email protected]>
>> Sent: Wednesday, March 1, 2023 2:26 PM
>> To: [email protected]
>> Subject: Hangs with NetworkManager and rtw8723_de with kernel 6.1
>>
>> Good morning,
>>
>> systemrescue recently switched from kernel 5.15 to kernel 6.1 and
>> NetworkManager started to hangs at startup. This seems to be linked to
>> my wifi card (RTL8723DE). I am also able to reproduce the problem under
>> gentoo with the same versions for NetworkManager and linux-firmware than
>> those used in systemrescue (1.42.2 and 20230210). Under gentoo, the
>> problem appears with kernel 6.1.14 and 6.2.1. I am able to use
>> NetworkManager and systemrescue without problem on another computer
>> which has another wifi card.
>>
>> Does anybody have a solution to this problem ?
>>
> I think this problem is the same as
> https://bugzilla.kernel.org/show_bug.cgi?id=217016
>
> Ping-Ke
>
>
This is indeed the same bug. I applied this patch on top of kernel
6.1.14 and the hang doesn't occur anymore, neither in Gentoo or in
SystemRescue. I see this patch has been merged in mainline and marked
for stable. However, it is not included in the current review for
6.1.15. I hope it will make it to stable kernels soon.

Thanks for your help,

François Valenduc


2023-03-02 17:36:36

by Larry Finger

[permalink] [raw]
Subject: Re: Hangs with NetworkManager and rtw8723_de with kernel 6.1

On 3/2/23 00:24, François Valenduc wrote:
>
> Le 2/03/23 à 02:27, Ping-Ke Shih a écrit :
>>
>>> -----Original Message-----
>>> From: François Valenduc <[email protected]>
>>> Sent: Wednesday, March 1, 2023 2:26 PM
>>> To: [email protected]
>>> Subject: Hangs with NetworkManager and rtw8723_de with kernel 6.1
>>>
>>> Good morning,
>>>
>>> systemrescue recently switched from kernel 5.15 to kernel 6.1 and
>>> NetworkManager started to hangs at startup. This seems to be linked to
>>> my wifi card (RTL8723DE). I am also able to reproduce the problem under
>>> gentoo with the same versions for NetworkManager and linux-firmware than
>>> those used in systemrescue (1.42.2 and 20230210). Under gentoo, the
>>> problem appears with kernel 6.1.14 and 6.2.1. I am able to use
>>> NetworkManager and systemrescue without problem on another computer
>>> which has another wifi card.
>>>
>>> Does anybody have a solution to this problem ?
>>>
>> I think this problem is the same as
>> https://bugzilla.kernel.org/show_bug.cgi?id=217016
>>
>> Ping-Ke
>>
>>
> This is indeed the same bug. I applied this patch on top of kernel 6.1.14 and
> the hang doesn't occur anymore, neither in Gentoo or in SystemRescue. I see this
> patch has been merged in mainline and marked for stable. However, it is not
> included in the current review for 6.1.15. I hope it will make it to stable
> kernels soon.

Thanks Ping-Ke for the help here. I am glad the problem is solved.

For your information, that patch is included in the rtw88 repo at GitHub.com.

Larry