Received-SPF: pass (google.com: domain of linux-kernel+bounces-195650-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Message-ID: <73d37ff63ee7cd88772fc0767f4474317b56a0a8.camel@sapience.com>
Subject: Re: 6.9.3 Hung tasks
From: Genes Lists <lists@sapience.com>
To: "Russell King (Oracle)" <linux@armlinux.org.uk>
Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, andrew@lunn.ch, 
 hkallweit1@gmail.com, davem@davemloft.net, edumazet@google.com,
 kuba@kernel.org,  pabeni@redhat.com, johanneswueller@gmail.com, Thorsten
 Leemhuis <linux@leemhuis.info>
Date: Thu, 30 May 2024 12:23:48 -0400
In-Reply-To: <ZliHhebSGQYZ/0S0@shell.armlinux.org.uk>
References: <9d189ec329cfe68ed68699f314e191a10d4b5eda.camel@sapience.com>
	 <15a0bbd24cd01bd0b60b7047958a2e3ab556ea6f.camel@sapience.com>
	 <ZliHhebSGQYZ/0S0@shell.armlinux.org.uk>
Autocrypt: addr=lists@sapience.com; prefer-encrypt=mutual;
 keydata=mDMEXSY9GRYJKwYBBAHaRw8BAQdAwzFfmp+m0ldl2vgmbtPC/XN7/k5vscpADq3BmRy5R
 7y0LU1haWwgTGlzdHMgKEwwIDIwMTkwNzEwKSA8bGlzdHNAc2FwaWVuY2UuY29tPoiWBBMWCAA+Ah
 sBBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAFiEE5YMoUxcbEgQOvOMKc+dlCv6PxQAFAmPJfooFCRl
 vRHEACgkQc+dlCv6PxQAc/wEA/Dbmg91DOGXll0OW1GKaZQGQDl7fHibMOKRGC6X/emoA+wQR5FIz
 BnV/PrXbao8LS/h0tSkeXgPsYxrzvfZInIAC
Content-Type: multipart/signed; micalg="pgp-sha384";
	protocol="application/pgp-signature"; boundary="=-+QEfLFhrShG4Mf1bhoXy"
User-Agent: Evolution 3.52.2 
Precedence: bulk
MIME-Version: 1.0


--=-+QEfLFhrShG4Mf1bhoXy
Content-Type: multipart/alternative; boundary="=-2wjINsonf24vl+/OcDje"

--=-2wjINsonf24vl+/OcDje
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, 2024-05-30 at 15:04 +0100, Russell King (Oracle) wrote:
> ...
> And then we get to pid 858. This is in set_device_name(), which
> was called from led_trigger_set() and led_trigger_register().
> We know from pid 663 that led_trigger_register() can take a read
> on leds_list_lock, and indeed it does and then calls
> led_match_default_trigger(), which then goes on to call
> led_trigger_set(). Bingo, this is why pid 666 is blocked, which
> then blocks pid 663. pid 663 takes the rtnl lock, which blocks
> everything else _and_ also blocks pid 858 in set_device_name().
>=20
> Lockdep would've found this... this is a classic AB-BA deadlock
> between the leds_list_lock rwsem and the rtnl mutex.
>=20
> I haven't checked to see how that deadlock got introduced, that's
> for someone else to do.
>=20


Thank you for the analysis - hopefully someone can track down the
culprit.

cc: =C2=A0thorsten

--=20
Gene


--=-2wjINsonf24vl+/OcDje
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

<html><head><style>pre,code,address {
  margin: 0px;
}
h1,h2,h3,h4,h5,h6 {
  margin-top: 0.2em;
  margin-bottom: 0.2em;
}
ol,ul {
  margin-top: 0em;
  margin-bottom: 0em;
}
blockquote {
  margin-top: 0em;
  margin-bottom: 0em;
}
</style></head><body><div>On Thu, 2024-05-30 at 15:04 +0100, Russell King (=
Oracle) wrote:</div><blockquote type=3D"cite" style=3D"margin:0 0 0 .8ex; b=
order-left:2px #729fcf solid;padding-left:1ex"><div>...</div><div>And then =
we get to pid 858. This is in set_device_name(), which<br></div><div>was ca=
lled from led_trigger_set() and led_trigger_register().<br></div><div>We kn=
ow from pid 663 that led_trigger_register() can take a read<br></div><div>o=
n leds_list_lock, and indeed it does and then calls<br></div><div>led_match=
_default_trigger(), which then goes on to call<br></div><div>led_trigger_se=
t(). Bingo, this is why pid 666 is blocked, which<br></div><div>then blocks=
 pid 663. pid 663 takes the rtnl lock, which blocks<br></div><div>everythin=
g else _and_ also blocks pid 858 in set_device_name().<br></div><div><br></=
div><div>Lockdep would've found this... this is a classic AB-BA deadlock<br=
></div><div>between the leds_list_lock rwsem and the rtnl mutex.<br></div><=
div><br></div><div>I haven't checked to see how that deadlock got introduce=
d, that's<br></div><div>for someone else to do.<br></div><div><br></div></b=
lockquote><div><br></div><blockquote type=3D"cite" style=3D"margin:0 0 0 .8=
ex; border-left:2px #729fcf solid;padding-left:1ex"></blockquote><div>Thank=
 you for the analysis - hopefully someone can track down the culprit.</div>=
<div><br></div><div>cc: &nbsp;thorsten</div><div><br></div><div><span><pre>=
-- <br></pre><div><span style=3D"background-color: inherit;">Gene</span></d=
iv><div><br></div></span></div></body></html>

--=-2wjINsonf24vl+/OcDje--

--=-+QEfLFhrShG4Mf1bhoXy
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iHUEABYJAB0WIQRByXNdQO2KDRJ2iXo5BdB0L6Ze2wUCZlioFAAKCRA5BdB0L6Ze
2ztUAP9jIfYUH867epX9kme9kL3SIae0lnrtM3DSeYE1axiCawD7BDiTCWFUP3i/
i0ulDQ52DV3I490ratrq6pxYUlXj1w0=
=Jxt6
-----END PGP SIGNATURE-----

--=-+QEfLFhrShG4Mf1bhoXy--