Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <87541edf7b46c1475f73cf464a9edca932f65da5.camel@linux.intel.com>
Subject: Re: [PATCH v2][RFC] sched/fair: Change SIS_PROP to search idle CPU
 based on sum of util_avg
From:   Tim Chen <tim.c.chen@linux.intel.com>
To:     Chen Yu <yu.c.chen@intel.com>, Abel Wu <wuyun.abel@bytedance.com>
Cc:     linux-kernel@vger.kernel.org, Tim Chen <tim.c.chen@intel.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Mel Gorman <mgorman@suse.de>,
        Viresh Kumar <viresh.kumar@linaro.org>,
        Barry Song <21cnbao@gmail.com>,
        Barry Song <song.bao.hua@hisilicon.com>,
        Yicong Yang <yangyicong@hisilicon.com>,
        Srikar Dronamraju <srikar@linux.vnet.ibm.com>,
        Len Brown <len.brown@intel.com>,
        Ben Segall <bsegall@google.com>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Aubrey Li <aubrey.li@intel.com>,
        K Prateek Nayak <kprateek.nayak@amd.com>
Date:   Mon, 14 Mar 2022 10:34:30 -0700
In-Reply-To: <20220314125657.GA30418@chenyu5-mobl1>
References: <20220310005228.11737-1-yu.c.chen@intel.com>
         <444bfebb-ac1c-42b9-58f5-332780e749f7@bytedance.com>
         <20220314125657.GA30418@chenyu5-mobl1>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.34.4 (3.34.4-1.fc31) 
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Mon, 2022-03-14 at 20:56 +0800, Chen Yu wrote:
> 
> > 
> > So nr_scan will probably be updated at llc-domain-lb-interval, which
> > is llc_size milliseconds. Since load can be varied a lot during such
> > a period, would this brought accuracy issues?
> > 
> I agree there might be delay in reflecting the latest utilization.
> The sum_util calculated by periodic load balance after 112ms would be
> decay to about 0.5 * 0.5 * 0.5 * 0.7 = 8.75%.
> But consider that this is a server platform, I have an impression that
> the CPU utilization jitter during a small period of time is not a regular
> scenario? It seems to be a trade-off. Checking the util_avg in newidle
> load balance path would be more frequent, but it also brings overhead -
> multiple CPUs write/read the per-LLC shared variable and introduces cache
> false sharing. But to make this more robust, maybe we can add time interval
> control in newidle load balance too.
> 
> 

Also the idea is we allow ourselves to be non-optimal in terms of
scheduling for the short term variations.  But we want to make sure that if
there's a long term trend in the load behavior, the scheduler should
adjust for that.  I think if you see high utilization and CPUs are
all close to fully busy for quite a while, that is a long term trend 
that overwhelms any short load jitters.

Tim