Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 1 Nov 2022 21:51:25 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     Peter Zijlstra <peterz@infradead.org>
CC:     Tianchen Ding <dtcccc@linux.alibaba.com>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>,
        "Mel Gorman" <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] sched: Clear ttwu_pending after enqueue_task
Message-ID: <Y2EkXYqZ15/Kjl6H@chenyu5-mobl1>
References: <20221101073630.2797-1-dtcccc@linux.alibaba.com>
 <Y2D2HIZuGP81w25+@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <Y2D2HIZuGP81w25+@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?T5ygWFFtZEMLN4ulilpCwqID6d/O4ni0c8BySXccjSzLbO33n+9R8dciDRJN?=
 =?us-ascii?Q?UstDl/NPGC8dVE2Drsf7+7IXSPtRa6AvW8wwji31sjZXZJ2hX9p115p2xUVW?=
 =?us-ascii?Q?8SJ1LBvrqjBLiHsoqf7DmyPU8psOXWbo8cDmYr7Lt3m9O7KKmTj08b748xtn?=
 =?us-ascii?Q?xksXm9B5/h3SsIP3Ak6nySiYna8HQbs3jF79lN5dY4/SvdDJHu103jBlDdDa?=
 =?us-ascii?Q?MX+Vu+TSZs1ytC3xi5SRkWdn2raauNiaMENHTDCKrCtMRpdnBpSYWSv5r3/0?=
 =?us-ascii?Q?jrT/ugulzT87b6FUI2m05zoGPQMLeuvYjdqsJqOzFN1SaI7EeAW9WZOPlGEh?=
 =?us-ascii?Q?7MXbeZX4AvqG2qW2kyHQ+Z8LOZU6dwB7F1Qt62S7+bDpbm2fXvwhgt+bv5Cv?=
 =?us-ascii?Q?d9cNdhWk/IErJuTeLk29FoANmcizqeBeSiiWIK8zqyOkK/+adQ2WxfVcyHPB?=
 =?us-ascii?Q?GyNdwbyg8aZ0PO17Jz5wVIh4g8TDdAAw29D9ggaf/Zy+PbQBLSJSgoDi5u6n?=
 =?us-ascii?Q?pzbTGplfEE3z4Xz+JSWo1EId3txGcCtH7G+hjFPx1C5sp0VfhAIxkpTnv09i?=
 =?us-ascii?Q?9B93nySIsS3TBnH4iOxvz/+wRH2fzTCVeG/GnmwHJF9UPfFIj1hTU5k6Uq0U?=
 =?us-ascii?Q?CINlk452g8Ko3IFvB2diAKCLUXf4QsCOcP42Z5Qi8Diqag8pc5yEhXa/0U3t?=
 =?us-ascii?Q?gas5TI+jWW07d+4sAyxZoD5QMHpxortY+3/S9ode2Yu6VzdoHBR2bih8AdCi?=
 =?us-ascii?Q?bQtNflhbs4SRyC5R+hgIDepJkWhT6GpllZuwZ2Jtu/8ZL3yxkDn3phwll0MW?=
 =?us-ascii?Q?1dNX4Lm0Qdl8TFp8yOW3WrKHtkIg8BRQIW/Kum6mtZlRNJ93by657Z9P5uqB?=
 =?us-ascii?Q?c4paW7kfWtpwefd39w0d3Uh8hVokaRGDw29tEIigu4Znr8ODbPs2tWz9TRA2?=
 =?us-ascii?Q?6LOadn7jZEn7xlCzXBPKO3hlzX14fcGpWu5Mc9j/3m3EbaWCzx5byrteKOTV?=
 =?us-ascii?Q?H9soRv0h2C2FzYehnXnXLev10W3OdPuYCau8RCL1vfW/J90aSE3q5j16uVVd?=
 =?us-ascii?Q?h0/bYBcXYZbA3QCafdvucDc0vKtwNpC9R70+eEwswhFehh9XnqwsQ0RHbfCi?=
 =?us-ascii?Q?oI2H7YFTMBrb1EjWCXivjJQVMwH+c2KHsRUMUW9XAxxgKsA1jA2R0DTUbDec?=
 =?us-ascii?Q?jYzeKnKNlAzJRSITkQF2LeWi+QNoPdFI4iOHj8F+iF7EzJIAKQvVpQRpVEIA?=
 =?us-ascii?Q?FCxHWDMl+woHu1XQzo6SVyRCtsbDlftxy8yL/YDLWKMQBESAXc0j6xvaI2DO?=
 =?us-ascii?Q?rUG2e0qGZQlCyu1k+/+RsMmavoTxTACEHtN27d2+2obohV32RF1IdFBJC7fk?=
 =?us-ascii?Q?sQ5GJeYlMlE/SWYG75t8IQCLVmQd06JBIeDMhCyfT5WHz3FDCHoOKbVFDDs0?=
 =?us-ascii?Q?lBSAlNRaU3IlEqcWgaqLO2jEJXdrq08i4yLOgNz4mzbeddW61Rnmt05s4Kvr?=
 =?us-ascii?Q?ze+XDJNmF15HSNrSRjeq+0jzuqfAkv3NE+s4IlDeCdFOCwkkj4oZwOm+jYHE?=
 =?us-ascii?Q?PS/FKoJ8A9vLK5Ec+jjZjtzr8ijv3hdRq3pvWgZ3?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 1323a473-0f87-4b6e-52db-08dabc103528
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Nov 2022 13:51:42.5867
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 137zTMHpCSqprknwtPofVNWJRzlHGccR37xByj/JMiWvqQn7miTwhePgi/CBR5wwtHU/yZd5M9jMZuuFIm7Iqg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB7039
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-5.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 2022-11-01 at 11:34:04 +0100, Peter Zijlstra wrote:
> On Tue, Nov 01, 2022 at 03:36:30PM +0800, Tianchen Ding wrote:
> > We found a long tail latency in schbench whem m*t is close to nr_cpus.
> > (e.g., "schbench -m 2 -t 16" on a machine with 32 cpus.)
> > 
> > This is because when the wakee cpu is idle, rq->ttwu_pending is cleared
> > too early, and idle_cpu() will return true until the wakee task enqueued.
> > This will mislead the waker when selecting idle cpu, and wake multiple
> > worker threads on the same wakee cpu. This situation is enlarged by
> > commit f3dd3f674555 ("sched: Remove the limitation of WF_ON_CPU on
> > wakelist if wakee cpu is idle") because it tends to use wakelist.
> > 
> > Here is the result of "schbench -m 2 -t 16" on a VM with 32vcpu
> > (Intel(R) Xeon(R) Platinum 8369B).
> > 
> > Latency percentiles (usec):
> >                 base      base+revert_f3dd3f674555   base+this_patch
> > 50.0000th:         9                            13                 9
> > 75.0000th:        12                            19                12
> > 90.0000th:        15                            22                15
> > 95.0000th:        18                            24                17
> > *99.0000th:       27                            31                24
> > 99.5000th:      3364                            33                27
> > 99.9000th:     12560                            36                30
> 
> Nice; but have you also ran other benchmarks and confirmed it doesn't
> negatively affect those?
> 
> If so; mentioning that is very helpful. If not; best go do so :-)
> 
> > Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
> > ---
> >  kernel/sched/core.c | 8 +-------
> >  1 file changed, 1 insertion(+), 7 deletions(-)
> > 
> > diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> > index 87c9cdf37a26..b07de1753be5 100644
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -3739,13 +3739,6 @@ void sched_ttwu_pending(void *arg)
> >  	if (!llist)
> >  		return;
> >  
> > -	/*
> > -	 * rq::ttwu_pending racy indication of out-standing wakeups.
> > -	 * Races such that false-negatives are possible, since they
> > -	 * are shorter lived that false-positives would be.
> > -	 */
> > -	WRITE_ONCE(rq->ttwu_pending, 0);
> > -
> >  	rq_lock_irqsave(rq, &rf);
> >  	update_rq_clock(rq);
> >  
> 
> Could you try the below instead? Also note the comment; since you did
> the work to figure out why -- best record that for posterity.
> 
> @@ -3737,6 +3730,13 @@ void sched_ttwu_pending(void *arg)
>  			set_task_cpu(p, cpu_of(rq));
>  
>  		ttwu_do_activate(rq, p, p->sched_remote_wakeup ? WF_MIGRATED : 0, &rf);
> +		/*
> +		 * Must be after enqueueing at least once task such that
> +		 * idle_cpu() does not observe a false-negative -- if it does,
> +		 * it is possible for select_idle_siblings() to stack a number
> +		 * of tasks on this CPU during that window.
> +		 */
> +		WRITE_ONCE(rq->ttwu_pending, 0);
Just curious why do we put above code inside llist_for_each_entry_safe loop?
My understanding is that once 1 task is queued, select_idle_cpu() would not
treat this rq as idle anymore because nr_running is not 0. But would this bring
overhead to write the rq->ttwu_pending multiple times, do I miss something?

thanks,
Chenyu
>  	}
>  
>  	rq_unlock_irqrestore(rq, &rf);