Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Sat, 3 Dec 2022 15:55:19 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     Chengming Zhou <zhouchengming@bytedance.com>
CC:     <mingo@redhat.com>, <peterz@infradead.org>, <vschneid@redhat.com>,
        <juri.lelli@redhat.com>, <vincent.guittot@linaro.org>,
        <dietmar.eggemann@arm.com>, <rostedt@goodmis.org>,
        <bsegall@google.com>, <mgorman@suse.de>, <bristot@redhat.com>,
        <linux-kernel@vger.kernel.org>, <bagasdotme@gmail.com>
Subject: Re: [PATCH v4] sched/core: Minor optimize ttwu_runnable()
Message-ID: <Y4sA5xDe+sBfQUnG@chenyu5-mobl1>
References: <20221202080644.76999-1-zhouchengming@bytedance.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20221202080644.76999-1-zhouchengming@bytedance.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?VHWWVrXEgoWlP7wVBrVpDcHoWSFoCts+k3cRazUPUrBUtpFZKBko6v1atbou?=
 =?us-ascii?Q?nF92h9cOZxC4jZwROhVqmq09L69/Zv16IbStd1Yv0DMfTU2z7JmITmT5NXo7?=
 =?us-ascii?Q?ZCEK0i9la+tlcd6L3dH44BgC6lDbhTTq/lg4xP6HaZOF5q82PtTWhplj5PSJ?=
 =?us-ascii?Q?ZdrZa8zyzrn28UYhRCeCbi8eDmr0GMPwUI+I01DBPdrL2W4pWWNn3UuaVXZ/?=
 =?us-ascii?Q?tEGu0GwC7c/iUivEIIwWDbZIM3uW2LxKgJn/dnKJHkFHiTh3Q+6cyMLXUG8D?=
 =?us-ascii?Q?FV/CnXX+66SIdkKg6njXwZAoprvjTSaCAZg+osgSM0Gvra4IqrVNokJkrMb9?=
 =?us-ascii?Q?0QGKQRwHI0Hwf2RrDRm3zmla8zhU9eXsfZc5/2SZTWg9SJc5CzqbplBaMgyW?=
 =?us-ascii?Q?lDeL++htoP0neXDb1BUI74MALn00nH1U6gXdH3iL/kCg/yOX13K/nRa7pByS?=
 =?us-ascii?Q?b5Mrf5/qgSzHtuRk8kTnFvkonsg+/nCI4eTr7Eqlb13Pybb3i1cH5SA3WITe?=
 =?us-ascii?Q?DC6wXNifoM42V4jkr9PNG7KCV95TRUPXyVfaUz0PoapU3okAxqtnjnAx7cFx?=
 =?us-ascii?Q?pkz7q3Kf6nSHTkvO4wSo7X60lPdhqjE0FaHKiJOMLyHamNWRYKT4oGy9M5M3?=
 =?us-ascii?Q?7M5O1mD90/rgQ1n6eaxZL4/ri6fuFDsOolzvsQ+3ZFqGAsAhBcIvfhn1CRrP?=
 =?us-ascii?Q?5WdEHNHeW9a6SZPAix5/xjw6OvtS3mifjLj8SUP4qRN7gyC68UWApLGVPzuy?=
 =?us-ascii?Q?8AKFpOvYcfUXTC3I+K2QIeVqiFg9+lm3/yIerxPHIWTMv+91wuMeUcy0Pn0m?=
 =?us-ascii?Q?FW/BWWUDKjXmmBLvGw0TNNqgTYiGs3y0DhcZAseewnt5WtmNLzNrpq0sJDFc?=
 =?us-ascii?Q?KZpxfmohh5gikwCkY+1uMiRR39bBXyRHk4aVCQKs+0XWx7vcPv3LSNEDoLnS?=
 =?us-ascii?Q?WcSzJ9+M6C5j/rVftdaTbdJmSAaSneFqtBLpsL3ZV6rGtupd7gh+ZX6/hEiv?=
 =?us-ascii?Q?rYzt+BFkLar3H/vvZm5j5UkEc6iD457lYp7Fc5n/G5hREbCtb6SuaR8He2ei?=
 =?us-ascii?Q?1vUdqQdjRFw5ZC5kueWBxFgD+bvIFP6jsrxf9orbLEKdUo4W8CxMBsTXIDNg?=
 =?us-ascii?Q?BLaMrEb8FG+Pf4r17aqE2EmxIWVa0iYECpZQGjtqKUCdK/GhGfYA8fHquF2C?=
 =?us-ascii?Q?zBpKejOuX9TsjfFNYbuhFkSeQ7qZyrkeaB9ek0+3dyWo+tDuYqdBOkM+RYKk?=
 =?us-ascii?Q?zsGctkLXQ2YXIoAicjJ9dt3PpnptxMySkvmQAkqNL4HTUwUt0mel5szGTTRd?=
 =?us-ascii?Q?VqowY3l+BwgyLnNAW6M8s9j9U48qAXZ4gRoAdM1toIyzOJBVY3EN4yKMyxQA?=
 =?us-ascii?Q?njCdTkMolNr2qgsu9B6Z1i8dEsQbRdQSKP7t/Su8M5RgJfGOvC6zsj2ZNPYJ?=
 =?us-ascii?Q?QXclTcVOswCVdssI8w3seyPbmoE4QViL+mG5QON+1G+d5pBc/CnSDuzAo/7f?=
 =?us-ascii?Q?h9VdVN1H5IJQ9BJMFbQR82yHO875AUVy99SBlHTTgvUYERg06QUA6UVnaBs0?=
 =?us-ascii?Q?iQT1aUM1/Y4UkhqxDAXW/xIfne70fz89K8uJV9p7?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e348da0a-1f88-40c0-8ac4-08dad503c257
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Dec 2022 07:55:35.0668
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VRjQL8zgIvCQY2QMfZ0XQtAcchSFoKDJbbfZNlRPxB4iJvrdV+ZgvxOAMWhM0TfieGX6Yzp1a8Gqn4GUK8OW6A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR11MB5227
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 2022-12-02 at 16:06:44 +0800, Chengming Zhou wrote:
> ttwu_runnable() is used as a fast wakeup path when the wakee task
> is running on CPU or runnable on RQ, in both cases we can just
> set its state to TASK_RUNNING to prevent a sleep.
> 
> If the wakee task is on_cpu running, we don't need to update_rq_clock()
> or check_preempt_curr().
> 
> But if the wakee task is on_rq && !on_cpu (e.g. an IRQ hit before
> the task got to schedule() and the task been preempted), we should
> check_preempt_curr() to see if it can preempt the current running.
> 
> Reorganize ttwu_do_wakeup() and ttwu_do_activate() to make
> ttwu_do_wakeup() only mark the task runnable, so it can be used
> in ttwu_runnable() and try_to_wake_up() fast paths.
> 
> This also removes the class->task_woken() callback from ttwu_runnable(),
> which wasn't required per the RT/DL implementations: any required push
> operation would have been queued during class->set_next_task() when p
> got preempted.
> 
> ttwu_runnable() also loses the update to rq->idle_stamp, as by definition
> the rq cannot be idle in this scenario.
> 
> Signed-off-by: Chengming Zhou <zhouchengming@bytedance.com>
> Suggested-by: Valentin Schneider <vschneid@redhat.com>
> Suggested-by: Peter Zijlstra <peterz@infradead.org>
> Reviewed-by: Valentin Schneider <vschneid@redhat.com>
> ---
> v4:
>  - s/This patch reorg/Reorganize/ per Bagas Sanjaya. Thanks!
> 
> v3:
>  - Improve the changelog per Valentin Schneider. Thanks!
> 
> v2:
>  - keep check_preempt_curr() for on_rq && !on_cpu case in ttwu_runnable(),
>    per Valentin Schneider.
>  - reorg ttwu_do_wakeup() and ttwu_do_activate() code, so ttwu_do_wakeup()
>    can be reused in ttwu_runnable(), per Peter Zijlstra.
>  - reuse ttwu_do_wakeup() in try_to_wake_up() (p == current) fast path too,
>    so ttwu_do_wakeup() become the only place we mark task runnable.
> ---
>  kernel/sched/core.c | 73 ++++++++++++++++++++++++---------------------
>  1 file changed, 39 insertions(+), 34 deletions(-)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 314c2c0219d9..d8216485b0ad 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3623,14 +3623,39 @@ ttwu_stat(struct task_struct *p, int cpu, int wake_flags)
>  }
>  
>  /*
> - * Mark the task runnable and perform wakeup-preemption.
> + * Mark the task runnable.
>   */
> -static void ttwu_do_wakeup(struct rq *rq, struct task_struct *p, int wake_flags,
> -			   struct rq_flags *rf)
> +static inline void ttwu_do_wakeup(struct task_struct *p)
>  {
> -	check_preempt_curr(rq, p, wake_flags);
>  	WRITE_ONCE(p->__state, TASK_RUNNING);
>  	trace_sched_wakeup(p);
> +}
> +
> +static void
> +ttwu_do_activate(struct rq *rq, struct task_struct *p, int wake_flags,
> +		 struct rq_flags *rf)
> +{
> +	int en_flags = ENQUEUE_WAKEUP | ENQUEUE_NOCLOCK;
> +
> +	lockdep_assert_rq_held(rq);
> +
> +	if (p->sched_contributes_to_load)
> +		rq->nr_uninterruptible--;
> +
> +#ifdef CONFIG_SMP
> +	if (wake_flags & WF_MIGRATED)
> +		en_flags |= ENQUEUE_MIGRATED;
> +	else
> +#endif
> +	if (p->in_iowait) {
> +		delayacct_blkio_end(p);
> +		atomic_dec(&task_rq(p)->nr_iowait);
> +	}
> +
> +	activate_task(rq, p, en_flags);
> +	check_preempt_curr(rq, p, wake_flags);
> +
> +	ttwu_do_wakeup(p);
>  
>  #ifdef CONFIG_SMP
>  	if (p->sched_class->task_woken) {
> @@ -3660,31 +3685,6 @@ static void ttwu_do_wakeup(struct rq *rq, struct task_struct *p, int wake_flags,
>  #endif
>  }
>  
> -static void
> -ttwu_do_activate(struct rq *rq, struct task_struct *p, int wake_flags,
> -		 struct rq_flags *rf)
> -{
> -	int en_flags = ENQUEUE_WAKEUP | ENQUEUE_NOCLOCK;
> -
> -	lockdep_assert_rq_held(rq);
> -
> -	if (p->sched_contributes_to_load)
> -		rq->nr_uninterruptible--;
> -
> -#ifdef CONFIG_SMP
> -	if (wake_flags & WF_MIGRATED)
> -		en_flags |= ENQUEUE_MIGRATED;
> -	else
> -#endif
> -	if (p->in_iowait) {
> -		delayacct_blkio_end(p);
> -		atomic_dec(&task_rq(p)->nr_iowait);
> -	}
> -
> -	activate_task(rq, p, en_flags);
> -	ttwu_do_wakeup(rq, p, wake_flags, rf);
> -}
> -
>  /*
>   * Consider @p being inside a wait loop:
>   *
> @@ -3718,9 +3718,15 @@ static int ttwu_runnable(struct task_struct *p, int wake_flags)
>  
>  	rq = __task_rq_lock(p, &rf);
>  	if (task_on_rq_queued(p)) {
> -		/* check_preempt_curr() may use rq clock */
> -		update_rq_clock(rq);
> -		ttwu_do_wakeup(rq, p, wake_flags, &rf);
> +		if (!task_on_cpu(rq, p)) {
> +			/*
> +			 * When on_rq && !on_cpu the task is preempted, see if
> +			 * it should preempt whatever is current there now.
> +			 */
> +			update_rq_clock(rq);
> +			check_preempt_curr(rq, p, wake_flags);
> +		}
> +		ttwu_do_wakeup(p);
>  		ret = 1;
>  	}
>  	__task_rq_unlock(rq, &rf);
> @@ -4086,8 +4092,7 @@ try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags)
>  			goto out;
>  
>  		trace_sched_waking(p);
> -		WRITE_ONCE(p->__state, TASK_RUNNING);
> -		trace_sched_wakeup(p);
> +		ttwu_do_wakeup(p);
>  		goto out;
>  	}
>
Just wonder if we could split the reorganization and optimization into two patches,
so we can track the code change a little easier in the future?

thanks,
Chenyu