Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 21 Jul 2023 14:42:17 +0800
From:   Aaron Lu <aaron.lu@intel.com>
To:     Dietmar Eggemann <dietmar.eggemann@arm.com>
CC:     Vincent Guittot <vincent.guittot@linaro.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Daniel Jordan <daniel.m.jordan@oracle.com>,
        "Steven Rostedt" <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        "Valentin Schneider" <vschneid@redhat.com>,
        Tim Chen <tim.c.chen@intel.com>,
        "Nitin Tekchandani" <nitin.tekchandani@intel.com>,
        Yu Chen <yu.c.chen@intel.com>,
        Waiman Long <longman@redhat.com>,
        <linux-kernel@vger.kernel.org>
Subject: Re: [RFC PATCH 3/4] sched/fair: delay update_tg_load_avg() for
 cfs_rq's removed load
Message-ID: <20230721064217.GA225030@ziqianlu-dell>
References: <20230718134120.81199-4-aaron.lu@intel.com>
 <CAKfTPtAu33AN6=X82T=yOgm40S8OXi+sPcF0QyD-bYRPV=xPEg@mail.gmail.com>
 <20230719051826.GB84059@ziqianlu-dell>
 <20230719080105.GA90441@ziqianlu-dell>
 <CAKfTPtCnvOehfrz1OSW+rmkKW+28bdGw8fU+AvVrZTxkHibL_g@mail.gmail.com>
 <20230719132914.GA91858@ziqianlu-dell>
 <CAKfTPtAqpAo8Y9BdWZ-fmnyYgA8PEtFbObqWJxc-hs2Ktqkt3Q@mail.gmail.com>
 <20230720144233.GA185317@ziqianlu-dell>
 <CAKfTPtANqtEQjv2UThb5s0TPCS2adhC_14Gfv5ayAOva9Mib8g@mail.gmail.com>
 <96efae1a-37f6-c8da-5cdb-07b460307d6b@arm.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <96efae1a-37f6-c8da-5cdb-07b460307d6b@arm.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Q8PlgsaWHEk5ZTol3+3EJ3R+6bxfFTdX3MsG5m+efBPIUGbIsmlQQE9EOwyK?=
 =?us-ascii?Q?fOrygFM/PUgLpQlFthQ5wAmnN0XbVXsPMPEsJ7hyYvz+1Ju3Ju9IQCokTVSc?=
 =?us-ascii?Q?9od/Ef6bxtBdmlVYz37te1E+s4ohju9F3BXyzG4tZ1pJIzD/VDcJXHtshzRZ?=
 =?us-ascii?Q?avmHGq2emydyppnqjInmNsOpewAn8GxpkRP4vIK25Cd3Vd0MRvwD9CgVtbNJ?=
 =?us-ascii?Q?z2Vl5OY3vSsDt0UIvE2iZFNNW7RfYTOnNfGvaKKDqlmUeYbUabAGxdLA2MJ2?=
 =?us-ascii?Q?QAOIfghZtR3UgIaNynXqdIpnWUzEv6xlPLhV8y9pj9eRcbSYwUevrs6P6ctO?=
 =?us-ascii?Q?GxZRCwamvAWXtuhcZEpoXAtcQ6bQADs2G2d2Kn5rYCGsLLAXusKf8al2nOTs?=
 =?us-ascii?Q?m8MUndGNspOsHBU00J3vByX36X0YLdMhLZt5WKClc3ENzXChyAqNi3kmXchX?=
 =?us-ascii?Q?9bIA5sjCQviIWhi+jnyM3im3KN+WJtEjGxvAIiff4tre070U0Z9PjnrkTKCI?=
 =?us-ascii?Q?/O5IARMZduBkXQQBhGn7mldR/sXq7QGzcxqyUEgyCqZ1kM5bKuQt0T5Qhwi3?=
 =?us-ascii?Q?NYvA5o8219PFnoMXQi+8FYmwsWS3XhTnr1rSB8mN/CyJ4SAFlrySVIagfLhu?=
 =?us-ascii?Q?BaPEC4j9K0BfhZn3BUKIMlxsglPViN9Lqxilj0GpNsiqjvi4cvrpBmmjwe8+?=
 =?us-ascii?Q?BC9ZIyTLsEGQR9dliGneShVfkJYFRBqHK6h8ra+YR2MeS3Wzav42egudc0/3?=
 =?us-ascii?Q?7GZ9jcqWWD62g1PsJtJ8JQVrFRRtIQ8DXeWBWtRS8ZdRgc7p0WYCVWY5wl7U?=
 =?us-ascii?Q?bHFNoM5hMh1XYatiepqsH8kuFib4VTtLsgqIakCGpiQ8MchJRStY9dtUuUGj?=
 =?us-ascii?Q?5Bt1GSepPV2BKhR/DDiDEcsG5e/kdwfRK/5nOcbUDIgupegSN/o6usj4/0Lr?=
 =?us-ascii?Q?Yab4W1JdpzK3qs5qAiFOjLIa5q4rdXityhFmwu6RAfo2fLDYyfxtsphmLyXM?=
 =?us-ascii?Q?jrGX6fTO9TN8AcvCJeMe8LUwgk7Y60UZT7vzspW1YGw2a/lgRs+ccQ198Akn?=
 =?us-ascii?Q?EtZ6YKPCx9+rKlxWM5trKIWdUrAI/WOFGfFAJOJ7vBC8teQ9XXqu+Mx/pF5m?=
 =?us-ascii?Q?ntTnOCI7ba6PQu9fUXCXXBYLt6heEfFUjBvsFqIztR4hyEgnQyi4nQ0szRrs?=
 =?us-ascii?Q?xOxRIVvQAZ5/S7sM8JUyharWwRBDYPl91vvst6NSAzTByKj8RBO4RkjOHbl1?=
 =?us-ascii?Q?72OuXU74Zk/y0qUjZpkW1zyG4PVJYHyh/TuuB+DbIMmQZq4JUjFC8H2kY31C?=
 =?us-ascii?Q?hwemXE/mridh/XJ2rpgTN0EdDPaga7e7xqLqRigCWDiOSFaGxFcNrS48vwRG?=
 =?us-ascii?Q?U6s4sheVi1lonisc/Pu4HNdXMFl8UvxIjH1EJVqVvy/OzHlrkSyybekXhi+b?=
 =?us-ascii?Q?3be5B5VIYoFw9h84E8y8i3jFvguTx/TqgH3k0PVMFVjhFLEfMcnfZelUoac9?=
 =?us-ascii?Q?SJ4vltY1P4Huj2Zc0opmFC5QwXG6guHzn7L945qZHy1wO4YObLMPS1X+Pnrf?=
 =?us-ascii?Q?okwatF8YBR6HNXB/9jvGsGWzy3N5K8whSYxNBp+6?=
X-MS-Exchange-CrossTenant-Network-Message-Id: cf9cf1fb-7262-466e-45a8-08db89b5a73e
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3062.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2023 06:42:30.0452
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BRSsiAGvq3YdjTQYSFzpdfigPo9296lr9jfos1GunuW+Eku+3VQ7itT8wRGUxbZVuBeWHcIhHUO9M3Co5U7+zw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR11MB8587
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,
        RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Jul 20, 2023 at 05:22:26PM +0200, Dietmar Eggemann wrote:
> On 20/07/2023 17:02, Vincent Guittot wrote:
> > On Thu, 20 Jul 2023 at 16:42, Aaron Lu <aaron.lu@intel.com> wrote:
> >>
> >> On Thu, Jul 20, 2023 at 03:10:30PM +0200, Vincent Guittot wrote:
> >>> On Wed, 19 Jul 2023 at 15:29, Aaron Lu <aaron.lu@intel.com> wrote:
> >>>>
> >>>> On Wed, Jul 19, 2023 at 11:47:06AM +0200, Vincent Guittot wrote:
> >>>>> On Wed, 19 Jul 2023 at 10:01, Aaron Lu <aaron.lu@intel.com> wrote:
> >>>>>>
> >>>>>> On Wed, Jul 19, 2023 at 01:18:26PM +0800, Aaron Lu wrote:
> 
> [...]
> 
> > What was wrong with your proposal to limit the update inside
> > update_tg_load_avg()  ? except maybe s/1000000/NSEC_PER_MSEC/ and
> > computing delta after testing the time since last update
> > 
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index a80a73909dc2..e48fd0e6982d 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -3665,6 +3665,7 @@ static inline bool cfs_rq_is_decayed(struct
> > cfs_rq *cfs_rq)
> >  static inline void update_tg_load_avg(struct cfs_rq *cfs_rq)
> >  {
> >         long delta = cfs_rq->avg.load_avg - cfs_rq->tg_load_avg_contrib;
> > +       u64 now = cfs_rq_clock_pelt(cfs_rq);
> 
> Could this be `u64 now = sched_clock_cpu()` like in
> migrate_se_pelt_lag() or newidle_balance() to avoid the time morphing
> due to PELT's frequency and uArch invariance?

Yes, will use sched_clock_cpu() instead of cfs_rq_clock_pelt().
Thanks.

> > 
> >         /*
> >          * No need to update load_avg for root_task_group as it is not used.
> 
> [...]
>