Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3;
References: <20231107215742.363031-1-ankur.a.arora@oracle.com>
 <20231107215742.363031-41-ankur.a.arora@oracle.com>
 <20231108091659.GK8262@noisy.programming.kicks-ass.net>
User-agent: mu4e 1.4.10; emacs 27.2
From:   Ankur Arora <ankur.a.arora@oracle.com>
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     Ankur Arora <ankur.a.arora@oracle.com>,
        linux-kernel@vger.kernel.org, tglx@linutronix.de,
        torvalds@linux-foundation.org, paulmck@kernel.org,
        linux-mm@kvack.org, x86@kernel.org, akpm@linux-foundation.org,
        luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com,
        hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com,
        vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de,
        jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com,
        boris.ostrovsky@oracle.com, konrad.wilk@oracle.com,
        jgross@suse.com, andrew.cooper3@citrix.com, mingo@kernel.org,
        bristot@kernel.org, mathieu.desnoyers@efficios.com,
        geert@linux-m68k.org, glaubitz@physik.fu-berlin.de,
        anton.ivanov@cambridgegreys.com, mattst88@gmail.com,
        krypton@ulrich-teichert.org, rostedt@goodmis.org,
        David.Laight@aculab.com, richard@nod.at, mjguzik@gmail.com
Subject: Re: [RFC PATCH 40/86] context_tracking: add ct_state_cpu()
In-reply-to: <20231108091659.GK8262@noisy.programming.kicks-ass.net>
Date:   Mon, 20 Nov 2023 22:32:16 -0800
Message-ID: <87sf4zy5u7.fsf@oracle.com>
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Rm1g4U7iUdhkWasvnjqEyLzlJmjwaYpPcXcKgJjtii1cFfz1AGHn3LW0jAr0?=
 =?us-ascii?Q?pbWjGzsruah7HuQDQdHOO13Kq+GRtsW5nAsM7M8FsOMnwSabGq5u7jtYDovA?=
 =?us-ascii?Q?p3c9sbeSTya9QMhLLxZ7SCXm1O/Xps9NV8fDq0fXbEQJzygT0+2RRuuKKwRP?=
 =?us-ascii?Q?vSE4O+DVDJHKVpUGYQ1KOLspug6sfNIZiW+AAmh3b12/kGbGNhz6EjgPgchI?=
 =?us-ascii?Q?AKMjDfqW0F5hIsuu0jM3NizwIVeRCE+XaIOxqsl3cFHD9wucchuSbO+rijnG?=
 =?us-ascii?Q?vRphsQUwKDjvkKbICwAOktDLa43nBggozk5CgUXcDX/X6Fsi4ouXbmfrDeI1?=
 =?us-ascii?Q?nVRHrxUbAgbAtG4LcDZGaT4xxFxrRwLirZQN21oNTTuj23x0CFJW+rkwsSEg?=
 =?us-ascii?Q?x+3Vq+jTgt0s2JbfXuAUkDzIjKlYALhdc3uYvrT+IYnNPVRe2Wncc/z51XwD?=
 =?us-ascii?Q?rS7vestX47/oc3zJwyk0zWWvDx3/c+t1VTi/eYqSv1QZ3Jmo4kpr/tA4FJtA?=
 =?us-ascii?Q?pNERGyGs4gTx6HRf6DMjzKdHyQB8qBssO8XclG0zaEwN43G3ZXQ3H0qq/9cg?=
 =?us-ascii?Q?P9RFQY43DFBudxZ0fS1pHOcsq1+zd4eeSNdRY1Fjjx+AESLENmUuB+UfINM/?=
 =?us-ascii?Q?RpNQ0iWopk6Obomp80LVZjCWI8z+nSFxuaBIuZS3EmDuYl2WFsk4eNp7G5Er?=
 =?us-ascii?Q?6hm4UZD234Njb/Ap1Cimq8O5Y9XBdsK5Mm9R9OSC534MD6N+a0tIhiRBRYIk?=
 =?us-ascii?Q?LkvpvYO0XSd601KyuOx4efDWiPtuXVUSzTZrSTsh0MqJrAymAYWK4GJHao8m?=
 =?us-ascii?Q?1XOB6srLVFncXduVkrPZmnmrruW3WM4SRuU5yVTf/uf4FM1jdQ1GY09f3ytW?=
 =?us-ascii?Q?18AX1ZRhMp+zkiyJj+I+XxEGD+DwimoGpx/e/xI/x6uF2P/dYYRFDdVLcq9K?=
 =?us-ascii?Q?YCImq+l0BCvEJV7kvZOHFgRIt26cjFplWPLeAW2Ch29AXYZ/uY2DQ8Qinj63?=
 =?us-ascii?Q?AFbaWmL/dP8peNKMz6ijNCM7n2HgAMXwGevelAOOfB33CtvTbbaM1neCC4GN?=
 =?us-ascii?Q?wgZR1artyMh/meYxWBuyuEkyfSz9HGAoD8/2PdnL293a02Hcsh+YmoxDTONw?=
 =?us-ascii?Q?yXB9Qzs/mbxTRLZQZ6iGty/YnOoVsxph20OuIkpoT/M6JbYD3tdb/9Eauuhu?=
 =?us-ascii?Q?Vq6GqkxpR9GCVldQyJT2Vl73faGO4WB76eHahqWEoIRJK/ErdnSNBqomYDBP?=
 =?us-ascii?Q?M4y/4odFm7Ii3zHiEdGpU5qT6k7lNCvgzcU7nCh8rESwKesxg0P6A+S+XWqh?=
 =?us-ascii?Q?A+li6BU5ASoObqcO66iczVOB+lYVWqpYprtP/7rx9UNWhTM8SwzBWVSxz3fq?=
 =?us-ascii?Q?YXqu2RzqsaVtI0/S7XFiep1wzCjTcGyz4fcPgwtDImbKomPaQrf1MbYy+ktX?=
 =?us-ascii?Q?ca2t0jTwLPfg43HMjOQT1OeXOjYke+IH/bFyRwO8yGuuOAXv+vYWvLhMKYhg?=
 =?us-ascii?Q?8V3ohfsN7ZY4MLbfiAxsOrJRwQsgT44tssfgcI/NGckuwQs+HOBtjLQjxAE0?=
 =?us-ascii?Q?cyMwnQseiVVIpvl/u04sMoFgE+CL/GLGrUxEKtTJLEFQab2YhfexDhd8BW56?=
 =?us-ascii?Q?GQ=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?us-ascii?Q?rs8j+yAD6nx5L8zyMr4AjuXJshoVYFhSXg/bYoCf5iLJwZ5i9d6beRzWnrMJ?=
 =?us-ascii?Q?Dn52xo4iBu1Jj/I4M8r8SqWI3wucBcP7BFdpjPMLC2jZB4xzax7AZojKoBMy?=
 =?us-ascii?Q?AVrhgJUDI8+rJ5qNH8Z4ZYezEbiDcu+C24TAFkAcj8iZ9q1HZe98cC6Q2IIV?=
 =?us-ascii?Q?NjrZmWuAyskDwRh8BP8CQWdXjEcnzHVHCnYY3u3YIj7Y9bbui/LC310sqUI/?=
 =?us-ascii?Q?BwKpsORiwgEwiTbhQ3Yd/YMJJVPfWDQ89RlTNuFeF/ltW5cXwRgx8pmG9Bkh?=
 =?us-ascii?Q?fZVNiVD0TbrLIlS/FbnO6Eg3LdkJpsnbS8poVrh/CZwQoBmwsjnFDD49WbQ/?=
 =?us-ascii?Q?0qww5+VK3PuyZyqb78+VEYqVakuh+stDBAV5mRWJ6beW29DJ3ffq+EO3zpd8?=
 =?us-ascii?Q?sIPG6/j6wzTYaZ8bfXdUYmHipAtYGZRNjbzeja/C33evVptBzO0SVvN9YSav?=
 =?us-ascii?Q?6/+37VbR6Nm6tf9NZ9ic2wrRzcfXaeV3j5Skqbi6OyxPYzV3clPzJ4Q6UMms?=
 =?us-ascii?Q?1tHs+/Bn3QZcxMCz9c4QNPP590nLELDME+jrX6Xh7c3WDCWxHGe+fQzJa5LP?=
 =?us-ascii?Q?H94p7E9nYNLc8XNgAAE3eVj0Dk8OKGOrQqVHFYsYj50qvVummwo/qwRbUIDW?=
 =?us-ascii?Q?VRT8a1WW/y+aXWtp1z68VBP4FBX9cB+GN/OONYrD7LCkuPQS7vcx5j9Hag+R?=
 =?us-ascii?Q?HpiSSzGHP/mKG0LrWMzPLJdk1EOfwUwZXiyLLzUMCeZ1As5q17m2oFw8r3tJ?=
 =?us-ascii?Q?D6wpsUAh293DrRYz6FjIAO4xRiLwF2fWDy4NyXUk2+N5czlh3Ndq4BwmY21+?=
 =?us-ascii?Q?pN+RI8R13ZXjP/TjDxdm0T/VW2pIjSmZQlL4wU068ou6hMP/30oXiZ5OsTgF?=
 =?us-ascii?Q?R+TB5qHHDW+hECvn/ztBe9nY1107mzbKkhlknbAacWybH7UdM0npVlD5RMtB?=
 =?us-ascii?Q?THjcFTuJPwlrKLdcnGhYNjsqZFSAszYCOwejRzF9Gor8tdqXq2YTExTAhxap?=
 =?us-ascii?Q?DIOso3x0mJR2G2fpFpBkD+kwBpI7oMqFyVJZ52a8RiyJBrexijmzrMWsj40p?=
 =?us-ascii?Q?T73b+JlbKgkrrP7nQjzHv3kGzDkYnyUhOblThcb3+uX4KvdFjx3ofbmXKS3u?=
 =?us-ascii?Q?eemSAOrAv24gNQcNnAOGb+450GAuAmmxuaXpvkd7VRvjCfjXSTgS7BYIfz/1?=
 =?us-ascii?Q?FQ8Ld1IbsWR7wHo3Yck4ydCg9M1XqdpTY7fuwgOPu+yn6sNSZJ8SgLV/q/PJ?=
 =?us-ascii?Q?vXdnk62ynVnh6WZZ6npw6IRyU07CxniRHUwcXSyYh/CX5heYpKt0keJox8IH?=
 =?us-ascii?Q?ny1JtjVmp6QKrhE4LJwyghIhr8QnrTVycNcoNWF1/aeHgKvXkYLCBXluGLrE?=
 =?us-ascii?Q?WX4DpvmjMYhUjy+XpwxpQcDMgfDL8d1rFPfSpouHgsXEqxA1tbn4x5TBqomx?=
 =?us-ascii?Q?HXFhhgRJE73wI/pq7IAQPbgeI6kRH8aSi2b5E7LYZ3fMBpTcKeWwTDeIL3Sf?=
 =?us-ascii?Q?GRbIfzUAkEwZxsuyu1LyXQGut2wPpkP4jB9udn6RgtZRPYAw7j6mFTDaEw?=
 =?us-ascii?Q?=3D=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e28f719d-bafc-4d63-146c-08dbea5b9c8b
X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Nov 2023 06:32:19.4253
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: tTvjSDAnzOwwU9mr9ebPBBWbKUgdXFSF4OsAe3WLA5BYd00TalT/ayjx1hrLsBLNfwr1sLHzP3BnOyJ5TlIcS3cl+GT6ZUDkp3O0y7KC0iE=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR10MB6733
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.987,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2023-11-21_03,2023-11-20_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 bulkscore=0
 malwarescore=0 mlxlogscore=999 adultscore=0 phishscore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000
 definitions=main-2311210049
X-Proofpoint-GUID: INORxKXbiNpDB28ErgchHM-WHnmop0ja
X-Proofpoint-ORIG-GUID: INORxKXbiNpDB28ErgchHM-WHnmop0ja
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 20 Nov 2023 22:33:46 -0800 (PST)


Peter Zijlstra <peterz@infradead.org> writes:

> On Tue, Nov 07, 2023 at 01:57:26PM -0800, Ankur Arora wrote:
>> While making up its mind about whether to reschedule a target
>> runqueue eagerly or lazily, resched_curr() needs to know if the
>> target is executing in the kernel or in userspace.
>>
>> Add ct_state_cpu().
>>
>> Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
>>
>> ---
>> Using context-tracking for this seems like overkill. Is there a better
>> way to achieve this? One problem with depending on user_enter() is that
>> it happens much too late for our purposes. From the scheduler's
>> point-of-view the exit state has effectively transitioned once the
>> task exits the exit_to_user_loop() so we will see stale state
>> while the task is done with exit_to_user_loop() but has not yet
>> executed user_enter().
>>
>> ---
>>  include/linux/context_tracking_state.h | 21 +++++++++++++++++++++
>>  kernel/Kconfig.preempt                 |  1 +
>>  2 files changed, 22 insertions(+)
>>
>> diff --git a/include/linux/context_tracking_state.h b/include/linux/context_tracking_state.h
>> index bbff5f7f8803..6a8f1c7ba105 100644
>> --- a/include/linux/context_tracking_state.h
>> +++ b/include/linux/context_tracking_state.h
>> @@ -53,6 +53,13 @@ static __always_inline int __ct_state(void)
>>  {
>>  	return raw_atomic_read(this_cpu_ptr(&context_tracking.state)) & CT_STATE_MASK;
>>  }
>> +
>> +static __always_inline int __ct_state_cpu(int cpu)
>> +{
>> +	struct context_tracking *ct = per_cpu_ptr(&context_tracking, cpu);
>> +
>> +	return atomic_read(&ct->state) & CT_STATE_MASK;
>> +}
>>  #endif
>>
>>  #ifdef CONFIG_CONTEXT_TRACKING_IDLE
>> @@ -139,6 +146,20 @@ static __always_inline int ct_state(void)
>>  	return ret;
>>  }
>>
>> +static __always_inline int ct_state_cpu(int cpu)
>> +{
>> +	int ret;
>> +
>> +	if (!context_tracking_enabled_cpu(cpu))
>> +		return CONTEXT_DISABLED;
>> +
>> +	preempt_disable();
>> +	ret = __ct_state_cpu(cpu);
>> +	preempt_enable();
>> +
>> +	return ret;
>> +}
>
> Those preempt_disable/enable are pointless.
>
> But this patch is problematic, you do *NOT* want to rely on context
> tracking. Context tracking adds atomics to the entry path, this is slow
> and even with CONFIG_CONTEXT_TRACKING it is disabled until you configure
> the NOHZ_FULL nonsense.

Yeah, I had missed the fact that even though the ct->state was updated
for both ct->active, !ct->active but the static branch was only enabled
with NOHZ_FULL.

Will drop.

--
ankur