Received-SPF: pass (google.com: domain of linux-kernel+bounces-196745-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <69d0c5f0b237990fd2c7cd88768aa2a70a5ee83a.camel@sapience.com>
Subject: Re: Hung tasks due to a AB-BA deadlock between the leds_list_lock
 rwsem and the rtnl mutex
From: Genes Lists <lists@sapience.com>
To: Hans de Goede <hdegoede@redhat.com>, Linux regressions mailing list
	 <regressions@lists.linux.dev>, Pavel Machek <pavel@ucw.cz>, Lee Jones
	 <lee@kernel.org>, Linux LEDs <linux-leds@vger.kernel.org>, Heiner Kallweit
	 <hkallweit1@gmail.com>
Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, andrew@lunn.ch, 
 davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
 pabeni@redhat.com,  johanneswueller@gmail.com, "Russell King (Oracle)"
 <linux@armlinux.org.uk>
Date: Fri, 31 May 2024 07:55:25 -0400
In-Reply-To: <618601d8-f82a-402f-bf7f-831671d3d83f@redhat.com>
References: <9d189ec329cfe68ed68699f314e191a10d4b5eda.camel@sapience.com>
	 <15a0bbd24cd01bd0b60b7047958a2e3ab556ea6f.camel@sapience.com>
	 <ZliHhebSGQYZ/0S0@shell.armlinux.org.uk>
	 <42d498fc-c95b-4441-b81a-aee4237d1c0d@leemhuis.info>
	 <618601d8-f82a-402f-bf7f-831671d3d83f@redhat.com>
Autocrypt: addr=lists@sapience.com; prefer-encrypt=mutual;
 keydata=mDMEXSY9GRYJKwYBBAHaRw8BAQdAwzFfmp+m0ldl2vgmbtPC/XN7/k5vscpADq3BmRy5R
 7y0LU1haWwgTGlzdHMgKEwwIDIwMTkwNzEwKSA8bGlzdHNAc2FwaWVuY2UuY29tPoiWBBMWCAA+Ah
 sBBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAFiEE5YMoUxcbEgQOvOMKc+dlCv6PxQAFAmPJfooFCRl
 vRHEACgkQc+dlCv6PxQAc/wEA/Dbmg91DOGXll0OW1GKaZQGQDl7fHibMOKRGC6X/emoA+wQR5FIz
 BnV/PrXbao8LS/h0tSkeXgPsYxrzvfZInIAC
Content-Type: multipart/signed; micalg="pgp-sha384";
	protocol="application/pgp-signature"; boundary="=-Y2OqDd5dYDgI7spdeu7K"
User-Agent: Evolution 3.52.2 
Precedence: bulk
MIME-Version: 1.0


--=-Y2OqDd5dYDgI7spdeu7K
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, 2024-05-31 at 11:50 +0200, Hans de Goede wrote:
> Hi,
>=20
> ...

> I actually have been looking at a ledtrig-netdev lockdep warning
> yesterday
> which I believe is the same thing. I'll include the lockdep trace
> below.
>=20
> According to lockdep there indeed is a ABBA (ish) cyclic deadlock
> with
> the rtnl mutex vs led-triggers related locks. I believe that this
> problem
> may be a pre-existing problem but this now actually gets hit in
> kernels >=3D
> 6.9 because of commit 66601a29bb23 ("leds: class: If no default
> trigger is
> given, make hw_control trigger the default trigger"). Before that
> commit
> the "netdev" trigger would not be bound / set as phy LEDs trigger by
> default.
>=20
> +Cc Heiner Kallweit who authored that commit.
>=20
> The netdev trigger typically is not needed because the PHY LEDs are
> typically
> under hw-control and the netdev trigger even tries to leave things
> that way
> so setting it as the active trigger for the LED class device is
> basically
> a no-op. I guess the goal of that commit is correctly have the
> triggers
> file content reflect that the LED is controlled by a netdev and to
> allow
> changing the hw-control mode without the user first needing to set
> netdev
> as trigger before being able to change the mode.
>=20
> But there is a price to this, besides the locking problem this also
> causes the ledtrig-netdev module to load on pretty much everyones
> systems (when build as a module) even though 99.999% of our users
> likely does not need this at all...
>=20
> Given this price and the troubles this is causing I think it might be
> best
> to revert 66601a29bb23. There might still be a locking issue when
> setting
> the trigger to netdev manually (I'll check and follow up) but this
> should
> fix the regression users are hitting since typically users do not set
> the trigger manually.
>=20
> Gene, as the original reporter of this can you do "modinfo
> ledtrig_netdev"
> and if this shows that ledtrig_netdev is a module for you try
> blacklisting
> ledtrig_netdev ?=C2=A0 And if it is not a module can you try building a
> 6.9
> kernel with commit 66601a29bb23 reverted and see if that helps ?

Thank you - I've blacklisted ledtrig_netdev and will report back if
anything interesting happens.

best

gene

>=20
>=20
> Regards,
>=20
> Hans

--=20
Gene


--=-Y2OqDd5dYDgI7spdeu7K
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iHUEABYJAB0WIQRByXNdQO2KDRJ2iXo5BdB0L6Ze2wUCZlm6rQAKCRA5BdB0L6Ze
2ztQAP9qMQRR3zWotstYwUJgtGTnC8aBJ+98ER37vLWOQ8tUfwD/YXQxaCPd9Swg
ncFhq3xPmLoOpSikgJb3evYfVsGEzQU=
=tg5T
-----END PGP SIGNATURE-----

--=-Y2OqDd5dYDgI7spdeu7K--