Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3;
Date:   Sat, 7 Oct 2023 08:37:02 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     Peter Zijlstra <peterz@infradead.org>
CC:     Mike Galbraith <efault@gmx.de>,
        Marek Szyprowski <m.szyprowski@samsung.com>,
        Biju Das <biju.das.jz@bp.renesas.com>,
        "bsegall@google.com" <bsegall@google.com>,
        "bristot@redhat.com" <bristot@redhat.com>,
        "chris.hyser@oracle.com" <chris.hyser@oracle.com>,
        "corbet@lwn.net" <corbet@lwn.net>,
        "dietmar.eggemann@arm.com" <dietmar.eggemann@arm.com>,
        "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
        "joel@joelfernandes.org" <joel@joelfernandes.org>,
        "joshdon@google.com" <joshdon@google.com>,
        "juri.lelli@redhat.com" <juri.lelli@redhat.com>,
        "kprateek.nayak@amd.com" <kprateek.nayak@amd.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "mingo@kernel.org" <mingo@kernel.org>,
        "patrick.bellasi@matbug.net" <patrick.bellasi@matbug.net>,
        Pavel Machek <pavel@ucw.cz>, "pjt@google.com" <pjt@google.com>,
        "qperret@google.com" <qperret@google.com>,
        "qyousef@layalina.io" <qyousef@layalina.io>,
        "rostedt@goodmis.org" <rostedt@goodmis.org>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "tim.c.chen@linux.intel.com" <tim.c.chen@linux.intel.com>,
        "timj@gnu.org" <timj@gnu.org>,
        "vincent.guittot@linaro.org" <vincent.guittot@linaro.org>,
        "youssefesmat@chromium.org" <youssefesmat@chromium.org>,
        "mgorman@suse.de" <mgorman@suse.de>,
        "linux-renesas-soc@vger.kernel.org" 
        <linux-renesas-soc@vger.kernel.org>
Subject: Re: [PATCH] sched/fair: fix pick_eevdf to always find the correct se
Message-ID: <ZSCoLllO/cYnnIMi@chenyu5-mobl2.ccr.corp.intel.com>
References: <OS0PR01MB59220AF3959BDC5FEFC0340F86CAA@OS0PR01MB5922.jpnprd01.prod.outlook.com>
 <20231005150258.GA36277@noisy.programming.kicks-ass.net>
 <CGME20231005150845eucas1p1ed3aae6b90c411b5c26a5dfadf7e0733@eucas1p1.samsung.com>
 <OS0PR01MB592295E06AD01CAEFBA83BF386CAA@OS0PR01MB5922.jpnprd01.prod.outlook.com>
 <553e2ee4-ab3a-4635-a74f-0ba4cc03f3f9@samsung.com>
 <867f5121d7d010cacf938c293f862b0cea560ec2.camel@gmx.de>
 <20231006140042.GG36277@noisy.programming.kicks-ass.net>
 <e2645a659b6fc1c592e2f2dc45273c361a2d2f47.camel@gmx.de>
 <20231006155501.GH36277@noisy.programming.kicks-ass.net>
 <20231006192445.GE743@noisy.programming.kicks-ass.net>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20231006192445.GE743@noisy.programming.kicks-ass.net>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?DgBfgKgYofyR2ngcbv91SJJU5tlPqa0zGFgof2UdILO82FFQJXmOJhIoKAPA?=
 =?us-ascii?Q?ToIK/7LhUCY3EdHN2x+e8/mDkf4UMx1OZnXP8Q+LepaMgaglcUYqyloSKNug?=
 =?us-ascii?Q?hTwgEca+pFhZeURY9U7q8hNrlvTlhWNq7YuwPrhTeOXzqoaJUpCzRR5eT9ro?=
 =?us-ascii?Q?TpJtSXYbo7doQYHQZ/bd9riwKImzGw2zR4TVLuIQsVRlq5J1LZfbd598mLI7?=
 =?us-ascii?Q?tjJfLuBefHB/jdCndq3fGPHU0H8sO3digYx48SDGenJvgYwUnVM7eFIRpMEu?=
 =?us-ascii?Q?NMECW6gd4T2i9ZjBa0Fpib8ag5diTTuIpxyKZRfK0Uf89d/dwSNZCUHTEB9L?=
 =?us-ascii?Q?RBCpdLJA+m6ELy56FbS9fFEOpe9CZ7Z/W+ZsiOIF3mR8MGirkhZMiy9rbDsT?=
 =?us-ascii?Q?WFxfz4/fVVHHWNqQKA3kct8AHCUcGoSj3cZPfGmPvgjY3NPnhKaq2Am1AiCV?=
 =?us-ascii?Q?SpwmeHvV7PEQDDR4vNk641VX2YokuHztzh42/PqGp4irWunKZy3gVe3iXBAc?=
 =?us-ascii?Q?shw+S3b0Fu0GyrKxQuPKO+286IbtQBfPLWUI5a5aZzlCX0KO5UbWrB/EDgi8?=
 =?us-ascii?Q?b/vYE87IMS5SwpsENsTB7QhqSt/1L5QDegUR3XaItYZYinYIc0nn74OY2dIA?=
 =?us-ascii?Q?W90LihJFW7DIl4yNjldxt29ui8mMQfqGzFHzcrPrkXAXflBYoF5ARkhwbYqA?=
 =?us-ascii?Q?6bR85Jrv/5pAw2lc5ocolobWGjHA/UcYS0HMxJPuv96h2omQ90Wa8GdBV75J?=
 =?us-ascii?Q?PcL6W19ifnB+839kw6PNeHIsAm5DT1OvafZ4ibaaiPndllBA4NN7a6J6siRU?=
 =?us-ascii?Q?bsoar9uSV6ZQ09nmQt0W52CxY+Jb3sD7ihFDQF7Ub62K9Phd9oXnygyuCfgz?=
 =?us-ascii?Q?Wdi4gaATtj8y7TQCay+8XrgjXAxNVBv+XAYBvJu/q5mBB3E/W2Y8nz7IzEWq?=
 =?us-ascii?Q?PjlKnrn3HLIUdg28WJq8vC/8BNG7HoyqM7p97Jl051e0wROxngn0tkqV3JJL?=
 =?us-ascii?Q?QoaL1hxq4d07kCdHwQPkmuweRj+UXiHp2hvAW1L0vvXfKNIageD23xt3PjqJ?=
 =?us-ascii?Q?jK6UMkK0AQHZQW4TR5ShIbh1pv1RjqeOBOq+gHhc9IQoraw29dw/80wm4MLa?=
 =?us-ascii?Q?r6iINIgjAc4+Zx727Z3zGVzKIDP7o2smvC8vOlxwlRwsQpb8Z2uggNUjpZg0?=
 =?us-ascii?Q?O6x0IwRL3uQWapsxEeZKNFumfgV3F0ycm2ObfpQmu+JaXROGMc3vMCSyBjA7?=
 =?us-ascii?Q?k/yLub3w/JBZUupzoJ/nJGmnjQLOFpF5Hn7ju+EZhpZka7GLD6pYAcgFzFZ/?=
 =?us-ascii?Q?eMHhMxg7zJNacPPB1guNtS4/CE9dVH2EAoU9OYrrSnhoqKiJU02KfRAmJM6H?=
 =?us-ascii?Q?fA2vWM5mtvFZtUuHxbvBNRGy1lUfYwhoWpvGcJd1mgmtQIUnAR01e6FfxHel?=
 =?us-ascii?Q?RES73umo8DTyUY5WmDELhmmOrweRJXrWKPNuyUxSm5cJ7PD8voU9JoMOEClC?=
 =?us-ascii?Q?h4sNMAl0zXK/kaKeAxyr5AWX6F7unqoBtl+Y7m87q8bOAiq5KkUIeu4aLeIH?=
 =?us-ascii?Q?DypVkQZ42+C1wLJpS3ifvhD8/mXRrq3j0aZ1euKj?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 2303b3ca-8793-4f04-7937-08dbc6cd9424
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2023 00:37:26.0508
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: saBgpBGKjrQKNFVdITA0G3vLnydw3swnkO5iY0e915FNrEpxjLrrarMEaWxlg31XPKuvzs7s/GQq06N3nyQ8aA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR11MB6162
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=2.7 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Fri, 06 Oct 2023 17:38:08 -0700 (PDT)
X-Spam-Level: **

On 2023-10-06 at 21:24:45 +0200, Peter Zijlstra wrote:
> On Fri, Oct 06, 2023 at 05:55:01PM +0200, Peter Zijlstra wrote:
> 
> > And yeah, min_deadline is hosed somehow.
> > 
> > migration/28-185     [028] d..2.    70.264274: validate_cfs_rq: --- /
> > migration/28-185     [028] d..2.    70.264277: __print_se: ffff88845cf48080 w: 1024 ve: -58857638 lag: 870381 vd: -55861854 vmd: -66302085 E (11372/tr)
> > migration/28-185     [028] d..2.    70.264280: __print_se:   ffff88810d165800 w: 25 ve: -80323686 lag: 22336429 vd: -41496434 vmd: -66302085 E (-1//autogroup-31)
> > migration/28-185     [028] d..2.    70.264282: __print_se:   ffff888108379000 w: 25 ve: 0 lag: -57987257 vd: 114632828 vmd: 114632828 N (-1//autogroup-33)
> > migration/28-185     [028] d..2.    70.264283: validate_cfs_rq: min_deadline: -55861854 avg_vruntime: -62278313462 / 1074 = -57987256
> > 
> > I need to go make dinner (kids hungry), but I'll see if I can figure out
> > how this happens...
> 
> *sigh*, does the below help?
> 
> ---
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 04fbcbda97d5..6a670f119efa 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -3632,6 +3747,7 @@ static void reweight_entity(struct cfs_rq *cfs_rq, struct sched_entity *se,
>  		 */
>  		deadline = div_s64(deadline * old_weight, weight);
>  		se->deadline = se->vruntime + deadline;
> +		min_deadline_cb_propagate(&se->run_node, NULL);
>  	}
>  
>  #ifdef CONFIG_SMP

Is it because without this patch, the se->deadline is not always synced with se->min_deadline,
then in pick_eevdf() the following condition could not be met, thus we get a NULL candidate
and has to pick the leftmost one?
	if (se->deadline == se->min_deadline)

Regarding the circular locking warning triggered by printk, does it mean we should not get a
NULL candidate from __pick_eevdf() in theory? And besides, we should not use printk with rq lock
hold?

thanks,
Chenyu