Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Message-ID: <ea2f9038f00d3b4c0008235079e1868145b47621.camel@gmx.de>
Subject: Re: [PATCH 2/2] sched/fair: Scale wakeup granularity relative to
 nr_running
From:   Mike Galbraith <efault@gmx.de>
To:     Mel Gorman <mgorman@techsingularity.net>
Cc:     Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@kernel.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Valentin Schneider <valentin.schneider@arm.com>,
        Aubrey Li <aubrey.li@linux.intel.com>,
        Barry Song <song.bao.hua@hisilicon.com>,
        Srikar Dronamraju <srikar@linux.vnet.ibm.com>,
        LKML <linux-kernel@vger.kernel.org>
Date:   Wed, 22 Sep 2021 07:22:20 +0200
In-Reply-To: <20210921103621.GM3959@techsingularity.net>
References: <20210920142614.4891-1-mgorman@techsingularity.net>
         <20210920142614.4891-3-mgorman@techsingularity.net>
         <22e7133d674b82853a5ee64d3f5fc6b35a8e18d6.camel@gmx.de>
         <20210921103621.GM3959@techsingularity.net>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.42.0 
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, 2021-09-21 at 11:36 +0100, Mel Gorman wrote:
> On Tue, Sep 21, 2021 at 05:52:32AM +0200, Mike Galbraith wrote:
>
>
> > Preemption does rapidly run into diminishing return as load climbs for
> > a lot of loads, but as you know, it's a rather sticky wicket because
> > even when over-committed, preventing light control threads from slicin=
g
> > through (what can be a load's own work crew of) hogs can seriously
> > injure performance.
> >
>
> Turning this into a classic Rob Peter To Pay Paul problem. We don't know
> if there is a light control thread that needs to run or not that affects
> overall performance. It all depends on whether that control thread needs
> to make progress for the overall workload or whether there are a mix of
> workloads resulting in overloading.

WRT overload, and our good buddies Peter and Paul :) I added...
	if (gran >=3D sysctl_sched_latency >> 1)
		trace_printk("runnable:%d preempt disabled\n",cfs_rq->nr_running);
...to watch, and met the below when I.. logged in.

homer:..debug/tracing # tail -20 trace
               X-2229    [002] d..5.    60.690322: wakeup_gran: runnable:9=
 preempt disabled
               X-2229    [002] d..5.    60.690325: wakeup_gran: runnable:1=
0 preempt disabled
               X-2229    [002] d..5.    60.690330: wakeup_gran: runnable:1=
1 preempt disabled
               X-2229    [002] d..5.    60.690363: wakeup_gran: runnable:1=
3 preempt disabled
               X-2229    [002] d..5.    60.690377: wakeup_gran: runnable:1=
4 preempt disabled
               X-2229    [002] d..5.    60.690390: wakeup_gran: runnable:1=
5 preempt disabled
               X-2229    [002] d..5.    60.690404: wakeup_gran: runnable:1=
6 preempt disabled
               X-2229    [002] d..5.    60.690425: wakeup_gran: runnable:9=
 preempt disabled
       ksmserver-2694    [003] d..3.    60.690432: wakeup_gran: runnable:6=
 preempt disabled
       ksmserver-2694    [003] d..3.    60.690436: wakeup_gran: runnable:7=
 preempt disabled
               X-2229    [002] d..5.    60.690451: wakeup_gran: runnable:6=
 preempt disabled
               X-2229    [002] d..5.    60.690465: wakeup_gran: runnable:7=
 preempt disabled
            kmix-2736    [000] d..3.    60.690491: wakeup_gran: runnable:6=
 preempt disabled
               X-2229    [004] d..5.    92.889635: wakeup_gran: runnable:6=
 preempt disabled
               X-2229    [004] d..5.    92.889675: wakeup_gran: runnable:6=
 preempt disabled
               X-2229    [004] d..5.    92.889863: wakeup_gran: runnable:6=
 preempt disabled
               X-2229    [004] d..5.    92.889944: wakeup_gran: runnable:6=
 preempt disabled
               X-2229    [004] d..5.    92.889957: wakeup_gran: runnable:7=
 preempt disabled
               X-2229    [004] d..5.    92.889968: wakeup_gran: runnable:8=
 preempt disabled
  QXcbEventQueue-2740    [000] d..4.    92.890025: wakeup_gran: runnable:6=
 preempt disabled
homer:..debug/tracing

Watching 'while sleep 1; do clear;tail trace; done' with nothing but a
kbuild running is like watching top.  There's enough stacking during
routine use of my desktop box that it runs into the tick granularity
wall pretty much continuously, so 'overload' may want redefining.

	-Mike