Received-SPF: pass (google.com: domain of linux-kernel+bounces-95936-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <49ca7920-d429-434a-aede-1a200e8d5ce8@linux.dev>
Date: Thu, 7 Mar 2024 12:26:00 -0500
Precedence: bulk
MIME-Version: 1.0
Subject: Re: [RESEND2 PATCH net v4 2/2] soc: fsl: qbman: Use raw spinlock for
 cgr_lock
Content-Language: en-US
To: Christophe Leroy <christophe.leroy@csgroup.eu>,
 "David S . Miller" <davem@davemloft.net>, Eric Dumazet
 <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>,
 Paolo Abeni <pabeni@redhat.com>,
 "netdev@vger.kernel.org" <netdev@vger.kernel.org>
Cc: Vladimir Oltean <vladimir.oltean@nxp.com>, Roy Pledge
 <roy.pledge@nxp.com>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "stable@vger.kernel.org" <stable@vger.kernel.org>,
 Li Yang <leoyang.li@nxp.com>, Scott Wood <oss@buserror.net>,
 Claudiu Manoil <claudiu.manoil@nxp.com>,
 Camelia Groza <camelia.groza@nxp.com>,
 Steffen Trumtrar <s.trumtrar@pengutronix.de>,
 "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>,
 "linux-arm-kernel@lists.infradead.org" <linux-arm-kernel@lists.infradead.org>
References: <20240222170749.2607485-1-sean.anderson@linux.dev>
 <20240222170749.2607485-2-sean.anderson@linux.dev>
 <53b401d7-934c-4937-ab83-6732af47668d@csgroup.eu>
 <34da1e7b-029e-410b-8735-a10d6d267e2b@linux.dev>
 <6764b9c5-b61a-4f20-a41a-125d5015a3e6@linux.dev>
 <63ab7b62-853c-4996-a493-465283252d5a@csgroup.eu>
From: Sean Anderson <sean.anderson@linux.dev>
In-Reply-To: <63ab7b62-853c-4996-a493-465283252d5a@csgroup.eu>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

On 3/5/24 17:18, Christophe Leroy wrote:
> 
> 
> Le 05/03/2024 à 19:14, Sean Anderson a écrit :
>> [Vous ne recevez pas souvent de courriers de sean.anderson@linux.dev. Découvrez pourquoi ceci est important à https://aka.ms/LearnAboutSenderIdentification ]
>> 
>> Hi,
>> 
>> On 2/23/24 11:02, Sean Anderson wrote:
>>> On 2/23/24 00:38, Christophe Leroy wrote:
>>>> Le 22/02/2024 à 18:07, Sean Anderson a écrit :
>>>>> [Vous ne recevez pas souvent de courriers de sean.anderson@linux.dev. Découvrez pourquoi ceci est important à https://aka.ms/LearnAboutSenderIdentification ]
>>>>>
>>>>> cgr_lock may be locked with interrupts already disabled by
>>>>> smp_call_function_single. As such, we must use a raw spinlock to avoid
>>>>> problems on PREEMPT_RT kernels. Although this bug has existed for a
>>>>> while, it was not apparent until commit ef2a8d5478b9 ("net: dpaa: Adjust
>>>>> queue depth on rate change") which invokes smp_call_function_single via
>>>>> qman_update_cgr_safe every time a link goes up or down.
>>>>
>>>> Why a raw spinlock to avoid problems on PREEMPT_RT, can you elaborate ?
>>>
>>> smp_call_function always runs its callback in hard IRQ context, even on
>>> PREEMPT_RT, where spinlocks can sleep. So we need to use raw spinlocks
>>> to ensure we aren't waiting on a sleeping task. See the first bug report
>>> for more discussion.
>>>
>>> In the longer term it would be better to switch to some other
>>> abstraction.
>> 
>> Does this make sense to you?
> 
> Yes that fine, thanks for the clarification. Maybe you can explain that 
> in the patch description in case you send a v5.

Hm, I thought I put this description in the commit message already.
Maybe something like

| smp_call_function always runs its callback in hard IRQ context, even on
| PREEMPT_RT, where spinlocks can sleep. So we need to use a raw spinlock
| for cgr_lock to ensure we aren't waiting on a sleeping task.
| 
| Although this bug has existed for a while, it was not apparent until
| commit ef2a8d5478b9 ("net: dpaa: Adjust queue depth on rate change")
| which invokes smp_call_function_single via qman_update_cgr_safe every
| time a link goes up or down.

would be clearer.

--Sean