Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 19 Jan 2023 22:29:01 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     Peter Zijlstra <peterz@infradead.org>
CC:     Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Tim Chen <tim.c.chen@intel.com>,
        Mel Gorman <mgorman@techsingularity.net>,
        Juri Lelli <juri.lelli@redhat.com>,
        "Rik van Riel" <riel@surriel.com>, Aaron Lu <aaron.lu@intel.com>,
        Abel Wu <wuyun.abel@bytedance.com>,
        K Prateek Nayak <kprateek.nayak@amd.com>,
        "Yicong Yang" <yangyicong@hisilicon.com>,
        "Gautham R . Shenoy" <gautham.shenoy@amd.com>,
        Ingo Molnar <mingo@redhat.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>,
        "Daniel Bristot de Oliveira" <bristot@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Hillf Danton <hdanton@sina.com>,
        Honglei Wang <wanghonglei@didichuxing.com>,
        Len Brown <len.brown@intel.com>,
        Chen Yu <yu.chen.surf@gmail.com>,
        "Tianchen Ding" <dtcccc@linux.alibaba.com>,
        Joel Fernandes <joel@joelfernandes.org>,
        Josh Don <joshdon@google.com>, <linux-kernel@vger.kernel.org>
Subject: Re: [RFC PATCH v4 1/2] sched/fair: Introduce short duration task
 check
Message-ID: <Y8lTrW/WrFPWCHFo@chenyu5-mobl1>
References: <cover.1671158588.git.yu.c.chen@intel.com>
 <ec049fd9b635f76a9e1d1ad380fd9184ebeeca53.1671158588.git.yu.c.chen@intel.com>
 <82689dd6-9db1-dd00-069b-73a637a21126@arm.com>
 <Y8Un9nmJauxpuSVE@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <Y8Un9nmJauxpuSVE@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?SrTGYpLdtEbzpF/ZtW/4LkhiCmsJMUjC87oChuSX9CH3jJjGdJspnp//yX57?=
 =?us-ascii?Q?eCQjPeKFXl8DKii8PJTvrrDQrIeieb26gOPXuoIQ8FRGp1VsZkD1aEgR9GXh?=
 =?us-ascii?Q?Gcb1yWIr5+yIu7eaunf4ZfwoHF/Zw1KKWNqhZ8a557530D59XT2sf+chPZ70?=
 =?us-ascii?Q?fqS+vDKTbxyrsJglzhNdvN2q7DxsO1C6NxXOhYgvWjH+FJeI9d7dlAFnf1up?=
 =?us-ascii?Q?6rNanM9RfbgGFWw1oDTueq33Ja8D9yucS6huzdo/BQzweU9Hbx0N8EHErwTj?=
 =?us-ascii?Q?qMOVf800DD9wL2DqU2RQiMLgAM4pVxX8DzGim0XIJGXEy3GUl9BQcV1VrOAH?=
 =?us-ascii?Q?RyUdPXIPkuS1YdxlIvgBx7Xf8DJRHkreWoo8J+m0UJuc5poFbv8av72ZinB4?=
 =?us-ascii?Q?W8WP6YvzbAJwwCNUcwJBjMeK5kbt3AogdaN3JGxmkVrITHPg8y7eNTdqt4x8?=
 =?us-ascii?Q?3xJvHXmsdn9mwXLr4kZ8Cg6dMFyTChnvguSF/kj6BinSBVWXCinzQZc3ZUyn?=
 =?us-ascii?Q?s7XboMPFo7AWlhx8LwskKE/CwUi0I4tI22RDTvuIMw64oop+UWITXTo66C6f?=
 =?us-ascii?Q?g2CmfhZTug0OAM5+EMU1zPbxVT3O5tHBU/gFTPESRXw3dNbe37dhVav45QwS?=
 =?us-ascii?Q?ojSr2Yf685F8F2BNgioIvKZ0a/tb4vRNqLPVedV6TErDBAooHGqpiU5lBkrt?=
 =?us-ascii?Q?IThix6AkrF38RvOoP402aUgwaeXQpKGCwi1QLwd9icPl1pV8582ehsP3CecF?=
 =?us-ascii?Q?ui5PxydWHbl6KBseGP64ahDwFrRK30I4999y4V7ftcrytwYxxmGLxl6VVAw3?=
 =?us-ascii?Q?dcRlGPpbvOJrBH8q52S0M/OBeKheWbYD1DQ4/FH2EqB8VUqDLiM1A95j2CVQ?=
 =?us-ascii?Q?Jxe6rg9zT6TOyI4T993oR7D8CQE/1OVCKNn56f/M3YCFe2dnfSEcQD6CLvxR?=
 =?us-ascii?Q?E1ZR0y2394aPI6aRfMBZLxzre6IYtxn6GACthcynFu3Z67zld8W22Kgl09BI?=
 =?us-ascii?Q?8Jc8OipIikVPMQ8bRu78BS7D7c/NfCT86UpSc8eqmei6WTMfqhYTSEYwXC6w?=
 =?us-ascii?Q?U/KKKEse80v4vtvxmoodTunoA18cfn5rrUu15eOzyOBWdt0PK1GaxpIdrlP8?=
 =?us-ascii?Q?LK9tqoFWypyBmQh8KgO4vxDc5YdadZyK9Xf5a9wNbETzTqvNnwXh41Ph2RxY?=
 =?us-ascii?Q?YdDWkarBbwy4Z3JxpVmATYwr3Dosp3051oN+GvvdL38Y2Th1osKLn7bcfLSX?=
 =?us-ascii?Q?QqElejvp+O/TfgCxwwetLIY3bfm7cIqu12xw+4k84ECQRW7JlTBlJ3gpOKBY?=
 =?us-ascii?Q?vdHusus6QhbdlW98iCwpMBMMCSRH6ViDpcsKig1Ojn6gmlu7hLNkzpeQKPHv?=
 =?us-ascii?Q?7bJ5zNsJq3TPlLgQP1J0Eu1J22w8O4tdmFgPsSgnjR0cwjvIE07DaCPmUOxr?=
 =?us-ascii?Q?R0+AeyzlyDwim7hOlrr4C4Ofslo+En1M89vcN9PRONprCekyfEENxdz0i/ON?=
 =?us-ascii?Q?f79pINrN9jprO7LyWrmGcgTyoTvo77KNy3ihQRsbZdfxquuIN+hJFd9NaXbt?=
 =?us-ascii?Q?qiiHTa0NJv1aAJ3hGDN4tHb+EGSr/mkUrUGvHBrU?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 2b6a2f38-2da0-4f64-2e0f-08dafa298fd4
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jan 2023 14:29:24.3121
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: E5gFeQRg/tUaUxBp1CI3eOsbQJHhT3C1LqCZ9WLPxuPF6/xtvh2CpDWRiLtOBUq8D1HXvOSGSaZbWu0y2g0gNA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR11MB7453
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Peter,
On 2023-01-16 at 11:33:26 +0100, Peter Zijlstra wrote:
> On Thu, Jan 05, 2023 at 12:33:16PM +0100, Dietmar Eggemann wrote:
> > On 16/12/2022 07:11, Chen Yu wrote:
> > 
> > [...]
> > 
> > > @@ -5995,6 +6005,18 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
> > >  
> > >  static void set_next_buddy(struct sched_entity *se);
> > >  
> > > +static inline void dur_avg_update(struct task_struct *p, bool task_sleep)
> > > +{
> > > +	u64 dur;
> > > +
> > > +	if (!task_sleep)
> > > +		return;
> > > +
> > > +	dur = p->se.sum_exec_runtime - p->se.prev_sum_exec_runtime_vol;
> > > +	p->se.prev_sum_exec_runtime_vol = p->se.sum_exec_runtime;
> > 
> > Shouldn't se->prev_sum_exec_runtime_vol be set in enqueue_task_fair()
> > and not in dequeue_task_fair()->dur_avg_update()? Otherwise `dur` will
> > contain sleep time.
> > 
> > Like we do for se->prev_sum_exec_runtime in set_next_entity() but for
> > one `set_next_entity()-put_prev_entity()` run section.
> > 
> > AFAICS, you want to measure the exec_runtime sum over all run sections
> > between enqueue and dequeue.
> 
> You were thinking of the dynamic PELT window size thread? (which is what
> I had to think of when I looked at this).
> 
> I think we can still do that with this prev_sum_exec_runtime_vol (can't
> say I love the name though).
I agree that this name is not accurate, maybe prev_sleep_sum_exec_runtime?
I'm open to any other name for this : )

Currently I'm checking Prateek's data on Zen3 and Yicong's data on Arm64,
and their data suggested that: inhibiting the spreading of short wakee is not
always a good idea on a system with small LLC. Meanwhile, according to my
test on a system with large number of CPUs in 1 LLC, short duration wakee become
a trouble maker if spreading them on different CPUs, which could trigger unexpected
race condition. I'm thinking of taking nr_llc_cpu into consideration when defining
a short duration task, and do some experiment on this.

thanks,
Chenyu