Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 26 Jun 2023 20:53:35 +0800
From:   Aaron Lu <aaron.lu@intel.com>
To:     Chen Yu <yu.c.chen@intel.com>
CC:     Peter Zijlstra <peterz@infradead.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Deng Pan <pan.deng@intel.com>, <tim.c.chen@intel.com>,
        <linux-kernel@vger.kernel.org>, <tianyou.li@intel.com>,
        <yu.ma@intel.com>, <lipeng.zhu@intel.com>,
        Tim Chen <tim.c.chen@linux.intel.com>
Subject: Re: [PATCH v2] sched/task_group: Re-layout structure to reduce false
 sharing
Message-ID: <20230626125335.GA508448@ziqianlu-dell>
References: <20230621081425.420607-1-pan.deng@intel.com>
 <20230626054756.GA435374@ziqianlu-dell>
 <ZJlDsbyNmcKtweg4@chenyu5-mobl2.ccr.corp.intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <ZJlDsbyNmcKtweg4@chenyu5-mobl2.ccr.corp.intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?jK6zmTLoxYttoniBJdjnnPqcfMq6/s0tv/lsBzaO5ZTcwfhj6ihw43X5YDVL?=
 =?us-ascii?Q?r5Wtha40ZiHbs7rYt1X0vCneLvpwLMlcMfeexH0HU/MzSzdgQjzdN4gWrWSv?=
 =?us-ascii?Q?hzTVpxifpSbZ6Lx5vjJGNIvrBvT3p5UcFJu3veuI72PsD6wK+eNn40T2GEut?=
 =?us-ascii?Q?uBf6tYLKqI0yeBnJoqQTDJOqOFMG5XPLj9N+Iuo8Y/xOG9clSx8f5ziKcY3w?=
 =?us-ascii?Q?UaRwABOz+1VkkNXw+qMuD2AyXAvAy2iEqoH87CKBbikvyJSI/XbOONwtV7/+?=
 =?us-ascii?Q?6stTOxlLj6FV3sqeDHGmZXkW28ZAnoHfGY11jb9k0xB+oOiQldT+rzrqEgT2?=
 =?us-ascii?Q?2sVYe/AofIxGywYEUdADYdRJUI8JXW1pFSoJFVQNRKQPcikKnVhTkl0U4XfT?=
 =?us-ascii?Q?EVN1PdKVZbcobItdoooUxAlVpMWuktMV9mXstdlB+cTaknbruXzRFJDfVp7T?=
 =?us-ascii?Q?Lgy/A4AZEkjkpBKKEF85NgyxobFwv0Z3M7Ah0k8frLDqkRznK536RZka6+5n?=
 =?us-ascii?Q?BZU7PkMflsXQkM4GXja4eKgY2iHQlYoG3+H+b1Ti9d7yna9WT4lRolFecxCG?=
 =?us-ascii?Q?hKOE9thApUdIThIfZAHlXpIWrRuqea6EAtMuI8F9ut7aigATFIImjQ2DK82Y?=
 =?us-ascii?Q?tur1FWt6rhhA4Zxys8fAXFUdaH9iqi9OLNFLgO/1hcoHgbQROenMRVAcy7DD?=
 =?us-ascii?Q?8OncIVSvAqkzvkBDYLvehyX04KZ1gEymfutbzw9g35RdRitIZ6RgmOyXKn44?=
 =?us-ascii?Q?/06d4qRnvcux1uXi8kgXUXsnHYYR8xcvLebjJFeUfIQxYrNmQSgSc36/WpDG?=
 =?us-ascii?Q?DzodGLmMvQIIWUr6DRN/KMIOUHGeHuOQyhNh7PoZHP4kh/zlbNu1YIa9uvnO?=
 =?us-ascii?Q?umKL/UkcmKnj72qBTrY/YiyeoUOaWCCVBp0tPfv+ivh8gco9O9hyYo+mBgTa?=
 =?us-ascii?Q?+62vqygLPQZxZSG4VwTc8WXPuzsDQZ87AOAbHoYiLejfHTYZPcfxCj3oas6q?=
 =?us-ascii?Q?uV0FBUiOiTXqmJzp2/lFu+k6QptrBdfDxSfUGk2jxRdQXv+gpItROwAcRcox?=
 =?us-ascii?Q?L7PfwWtiKfeCKKmU4aLVVZMYKN3plZWTFNZNeGJgQatYVuHSCYpV8F+MZpVi?=
 =?us-ascii?Q?ebKmNEsyEaTHBnCHfOSMwC+L4tSI0HkEkBk/1qmd4Ec5oWTmLpw9VqxtGaNj?=
 =?us-ascii?Q?Xk1LMpD24aa8aFb0zmi5B/4Bedy+55Y3/zw4b8/J1x8Exrfq2jPzg28z+LEL?=
 =?us-ascii?Q?ZNcOFA1zM8OBL/SXlwUpnseshfZNVbY3DYXLxqDDiCSHQWUVLdfMWHx3/4yl?=
 =?us-ascii?Q?lOifRjabRI0vef5+RAS7ozCY8DXSGT/jhBwK+pgqaA6pE8RviCok1Glh07Jd?=
 =?us-ascii?Q?HK1EBl6RuX/kH5gO7s9fPeVePMmq7H9M6oI0j1yZJn0Kul4ithg/ENORxAi9?=
 =?us-ascii?Q?CIn6aHANIjAUJe0Zow9kjbnfoliKMk2fiY2/fx6AXcRUxMKnGIV66u1w43EP?=
 =?us-ascii?Q?dMa21+BkPBrO7Jx8CMJNkhbHqe2msFR2bmlLAvjWtOd1uVlzfv8Ipv94CvtU?=
 =?us-ascii?Q?/xMXT3zy48bAPQF0VSYRpDRuAo2dqCAMj1EId6UE?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3647131b-269f-48a3-a95e-08db764461df
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3062.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jun 2023 12:53:47.6790
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fdk9ZxmnR9IESbRiB6iR4FP9dGPxhZD6GbcwJmfv+erQ7qHrHquQiN2nw24WzKuFyY4UMwSxCXTjHYLoph7zMQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR11MB4556
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,
        SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, Jun 26, 2023 at 03:52:17PM +0800, Chen Yu wrote:
> Besides the cache line alignment, if the task is not a rt one,
> why do we have to touch that, I wonder if the following change can avoid that:
> 
> diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
> index ec7b3e0a2b20..067f1310bad2 100644
> --- a/kernel/sched/sched.h
> +++ b/kernel/sched/sched.h
> @@ -1958,8 +1958,10 @@ static inline void set_task_rq(struct task_struct *p, unsigned int cpu)
>  #endif
>  
>  #ifdef CONFIG_RT_GROUP_SCHED
> -	p->rt.rt_rq  = tg->rt_rq[cpu];
> -	p->rt.parent = tg->rt_se[cpu];
> +	if (p->sched_class = &rt_sched_class) {
                           ==  :-)

> +		p->rt.rt_rq  = tg->rt_rq[cpu];
> +		p->rt.parent = tg->rt_se[cpu];
> +	}
>  #endif
>  }

If a task starts life as a SCHED_NORMAL one and then after some time
it's changed to a RT one, then during its next ttwu(), if it didn't
migrate, then set_task_rq() will not be called and p->rt.rt_rq will
keep as NULL which will cause problem when this task gets enqueued as
a rt one.

The follow diff seems to cure this issue:

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index c7db597e8175..8c57148e668c 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7801,6 +7801,20 @@ static int __sched_setscheduler(struct task_struct *p,
 	}
 	__setscheduler_uclamp(p, attr);
 
+#ifdef CONFIG_RT_GROUP_SCHED
+	/*
+	 * Make sure when this task becomes a rt one,
+	 * its rt fields have valid value.
+	 */
+	if (rt_prio(newprio)) {
+		struct task_group *tg = task_group(p);
+		int cpu = cpu_of(rq);
+
+		p->rt.rt_rq = tg->rt_rq[cpu];
+		p->rt.parent = tg->rt_se[cpu];
+	}
+#endif
+
 	if (queued) {
 		/*
 		 * We enqueue to tail when the priority of a task is

But I'm not sure if it's worth the trouble.

Thanks,
Aaron