Received-SPF: pass (google.com: domain of linux-kernel+bounces-188869-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Date: Fri, 24 May 2024 23:30:12 +0800
From: Chen Yu <yu.c.chen@intel.com>
To: Chunxin Zang <spring.cxz@gmail.com>
CC: <mingo@redhat.com>, <peterz@infradead.org>, <juri.lelli@redhat.com>,
	<vincent.guittot@linaro.org>, <dietmar.eggemann@arm.com>,
	<rostedt@goodmis.org>, <bsegall@google.com>, <mgorman@suse.de>,
	<bristot@redhat.com>, <vschneid@redhat.com>, <linux-kernel@vger.kernel.org>,
	<yangchen11@lixiang.com>, <zhouchunhua@lixiang.com>,
	<zangchunxin@lixiang.com>, <kprateek.nayak@amd.com>
Subject: Re: [PATCH] sched/fair: Reschedule the cfs_rq when current is
 ineligible
Message-ID: <ZlCyhDspcZQhxlNk@chenyu5-mobl2>
References: <20240524134011.270861-1-spring.cxz@gmail.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240524134011.270861-1-spring.cxz@gmail.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?SkZI9xJOU3MH07Ak6MIdP9K7F2bQCHFnVT3SM4sAEbKkkSzxx/PQMJ9OOEXM?=
 =?us-ascii?Q?TWtGX0jTaVDEnsafdKlyhkITPuc8soaqvseoBQP0eCsc7c4xUz00w1i4Syic?=
 =?us-ascii?Q?FNfka1AjEhf1PHBvaT65t4Oo2GHer98ve2GDb4H3LnEB9kRFHssS26oNsiFe?=
 =?us-ascii?Q?9dvPNGnvB4POfa7PxR73Y4nzRckH7fT3yc7r66ADSB2OHG3IocKmdYPcYW4N?=
 =?us-ascii?Q?JOb+Z7lr+dzLTZSu3bMRjCbS4Kl1EOhtssvMaNBHm5YbMzZgg+TuGWyZeqKE?=
 =?us-ascii?Q?MhKF5NotIAi67AuFurzgovdGemqwmi6Y1QJO3FKTUAd1y1nLHdVB6RrUbTPr?=
 =?us-ascii?Q?Vc4Bas3rGqEEadIF+x/f9nZgye+doa5AsZC/i2F1oyLDwmknWow1/dAv0zWW?=
 =?us-ascii?Q?RIaSPhoIpTDf1ETjgmMPQpRCisIKRVlnhcsGiANQT7NWLMxq252kCzCpJbS7?=
 =?us-ascii?Q?bniGlAeYoZYw3fqZDHN3L/oUsprTEn7/M01WXTJF84nOzc/1JwVrLikQREpx?=
 =?us-ascii?Q?mRH3+SDtvZAq9ZCIDNRNvZYPjimz08PilyJOgyfObzHI44Ta/9wWVwMjvKNi?=
 =?us-ascii?Q?si0naVSEZU18P9tQT4uBe5LbW0rapZs3aPp+lI9VHNqCH7E+jlRAhZY61rGm?=
 =?us-ascii?Q?FUtZETvUdQ3haF3j3X+XGOcrbcGXJOCqkDiXCjlBdkHtXhgKsvmwB0lfm9ED?=
 =?us-ascii?Q?xci8OuLAcHQ2o7gLrZfwrjOrQVV9uRIxVPjr784KB1VgFqt51LHdvPv2pfg9?=
 =?us-ascii?Q?N6CgsNb3Umlo+VzZBpd6wk1UEybYzD2Q/BAlkYXGqCpfcH+ytT8wOmSfK7pz?=
 =?us-ascii?Q?pEHLda3ZvUFsrytc5COrnqtRanbDwBV18dtT10rdc7OwaJhKYvW4JyJzWkWd?=
 =?us-ascii?Q?3hu99zLZ0WBSl4kfYbjI9qhTq3dkLgFvlqYXnD2csomkkBZSOZ5aZL6iykl9?=
 =?us-ascii?Q?oNdACfuVV/FDwSDE/hvdMqWh/H9gZGYwcgxU4k9A9ZZM59/zWohDNGXJMhYw?=
 =?us-ascii?Q?BNnqsaOvFFfQhVYx1zl4GMQTMAkVUIuub2OXQrvnsu82GEp1CyUFwB4ClyVP?=
 =?us-ascii?Q?AIWopTfUwkYkU7SUrpi85gAaOM9nD3monhI6Wix0E6lWY4lB1IeBmtWmtUJ2?=
 =?us-ascii?Q?eRX+wptm1jdsUCOMu+/UoObait6atnultQnS4hRNS/GTjhFQVtsiyl71mVjN?=
 =?us-ascii?Q?3sxmU6frY6NBQlRd5zU5iGrPGuexNMU8WjMPIRfJZ8GKOk+FcrphD7uillCr?=
 =?us-ascii?Q?wJj/1Xq7k17LM2QrsMqcU+zugAckSaTNv+l9ejR7/kUvGo700nmgVQARw4Mw?=
 =?us-ascii?Q?5zFxOt3a9Eybr0WJIQHA4uFw9kKqCid1m5ezoGOP94w4KMkNM2f8Tp+jjmNj?=
 =?us-ascii?Q?Zh7WxEmkYlFd4shm5Jr/2Y/U0HEy6zK2Yn6b22OUtJjEfhKlNsxqythhWwgZ?=
 =?us-ascii?Q?7dw8cfpu+UXQQVaI9AWvj7USl5UC13woMqKx3YwLhwpTLRJD2L5aZ7CwGKeC?=
 =?us-ascii?Q?iFPHiWeUtuhdX5BXEgZReNg5ijA9Nfe8TmD37Il+NJ7PW+J4mX9ReTyHPFa5?=
 =?us-ascii?Q?oPXHIPBIe0+v0vWMs0oZoM/yw6oZB2zabf5Ggf+m?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3f905488-e51a-4f4f-69a9-08dc7c066ef4
X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6020.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 15:30:25.5131
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oxXhp8rthQTS1zwRBMmrtmkHEC913Xs510AVR3kwPa7YFnw7tRahiR58UGvEmfyTz4+48N1f6BqJxAE5OS8Z9g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB7420
X-OriginatorOrg: intel.com

On 2024-05-24 at 21:40:11 +0800, Chunxin Zang wrote:
> I found that some tasks have been running for a long enough time and
> have become illegal, but they are still not releasing the CPU. This
> will increase the scheduling delay of other processes. Therefore, I
> tried checking the current process in wakeup_preempt and entity_tick,
> and if it is illegal, reschedule that cfs queue.
>
> The modification can reduce the scheduling delay by about 30% when
> RUN_TO_PARITY is enabled.
> So far, it has been running well in my test environment, and I have
> pasted some test results below.
> 

Interesting, besides hackbench, I assume that you have workload in
real production environment that is sensitive to wakeup latency?

>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 03be0d1330a6..a0005d240db5 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -5523,6 +5523,9 @@ entity_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr, int queued)
>  			hrtimer_active(&rq_of(cfs_rq)->hrtick_timer))
>  		return;
>  #endif
> +
> +	if (!entity_eligible(cfs_rq, curr))
> +		resched_curr(rq_of(cfs_rq));
>  }
>

entity_tick() -> update_curr() -> update_deadline():
se->vruntime >= se->deadline ? resched_curr()
only current has expired its slice will it be scheduled out.

So here you want to schedule current out if its lag becomes 0.

In lastest sched/eevdf branch, it is controlled by two sched features:
RESPECT_SLICE: Inhibit preemption until the current task has exhausted it's slice.
RUN_TO_PARITY: Relax RESPECT_SLICE and only protect current until 0-lag.
https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/commit/?h=sched/eevdf&id=e04f5454d68590a239092a700e9bbaf84270397c

Maybe something like this can achieve your goal
	if (sched_feat(RUN_TOPARITY) && !entity_eligible(cfs_rq, curr))
		resched_curr

>  
> @@ -8325,6 +8328,9 @@ static void check_preempt_wakeup_fair(struct rq *rq, struct task_struct *p, int
>  	if (unlikely(p->policy != SCHED_NORMAL) || !sched_feat(WAKEUP_PREEMPTION))
>  		return;
>  
> +	if (!entity_eligible(cfs_rq, se))
> +		goto preempt;
> +

Not sure if this is applicable, later in this function, pick_eevdf() checks
if the current is eligible, !entity_eligible(cfs_rq, curr), if not, curr will
be evicted. And this change does not consider the cgroup hierarchy.

Besides, the check of current eligiblity can get false negative result,
if the enqueued entity has a positive lag. Prateek proposed to
remove the check of current's eligibility in pick_eevdf():
https://lore.kernel.org/lkml/20240325060226.1540-2-kprateek.nayak@amd.com/

If I understand your requirement correctly, you want to reduce the wakeup
latency. There are some codes under developed by Peter, which could
customized task's wakeup latency via setting its slice:
https://lore.kernel.org/lkml/20240405110010.934104715@infradead.org/

thanks,
Chenyu