Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <795a6d9475ecb444d219a9d36dc93b48a69e960e.camel@gmx.de>
Subject: Re: [RFC PATCH] sched/fair: Introduce SIS_PAIR to wakeup task on
 local idle core first
From:   Mike Galbraith <efault@gmx.de>
To:     Chen Yu <yu.c.chen@intel.com>
Cc:     Peter Zijlstra <peterz@infradead.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Mel Gorman <mgorman@techsingularity.net>,
        Tim Chen <tim.c.chen@intel.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        K Prateek Nayak <kprateek.nayak@amd.com>,
        Abel Wu <wuyun.abel@bytedance.com>,
        Yicong Yang <yangyicong@hisilicon.com>,
        "Gautham R . Shenoy" <gautham.shenoy@amd.com>,
        Len Brown <len.brown@intel.com>,
        Chen Yu <yu.chen.surf@gmail.com>,
        Arjan Van De Ven <arjan.van.de.ven@intel.com>,
        Aaron Lu <aaron.lu@intel.com>, Barry Song <baohua@kernel.org>,
        linux-kernel@vger.kernel.org
Date:   Tue, 16 May 2023 13:51:26 +0200
In-Reply-To: <ZGNBt7vWJ3fDs5Sc@chenyu5-mobl1>
References: <20230516011159.4552-1-yu.c.chen@intel.com>
         <19664c68f77f5b23a86e5636a17ad2cbfa073f78.camel@gmx.de>
         <ZGNBt7vWJ3fDs5Sc@chenyu5-mobl1>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.42.4 
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
UI-OutboundReport: notjunk:1;M01:P0:3dq5S01HEgM=;m/rM7B8GH12glxRej1DTakfAed5
 1NVC75YOgTdxMXEZavWeAUoKZm/LZWlV+f97RLtakzjLKDZ1XYO8JRRL5p4fer0PI7JDiHygN
 bWHx4ioziKJLKDgVMWKDWESOJDxU2/88iadyNy9FF4R/MapFQXJnJrJlidAcdoItKH2Yp5Mxu
 +J0vwFQ/j4mvDzMWmv0fe6XM+trVC/B94Xq70bJyg6FDeF8ZNVd+p8eVGotjidW4860dVnl2T
 wQeH+ocbfLs+cFryunYhqzUVM5ZUVPzmXeC/bh5+MrROexFICKM52sNRDc/Q5yo5mJSrU90a5
 r1oHi/At8mXtM6DcWy6UUcWOhFKODH6IjmBC3rcZfH3yR3nwaAiSmTrAYK1r49eVCQtYTcVQe
 L5Ja4FORQDYNXoteg5Xgpl1Gjju06zwHIo4IolKNrhZ3djjBLMcDEheR6FaZX93L3l5qhQCLP
 wkS2iCTDzwxiPFc4pVK1aeW/G1zj2RJB0XpBs6vGK6k001zfsw9fRBhJdXGEEHO6CjuFKsW4d
 U0i65clA2kM7ow3cjaqXODF5a9zwkjFgVv5h3mTILocwKOwlMB9NHvAGK9rfHhQBpbfplM0aO
 6+ET52dFCcuslPBID2OI0qvwb9mkxhlkvQNWvnFTu6dIhDWG8FdbR1ED70Qa6FB2K8zXabphL
 APB2T+ayWa56TuozE7jqYQFhD0JcSsC8VVhk93jMLz4z9djQuvuvbor3giQCCqQBnx3RlILIG
 Xr0oIXDU+S1sZyvWmtsP+XooymRcMo/ihMPTlcWypDpjyHAoeiGgRp/NbAI0TNjtYDpWOpJxU
 N2aEwlJs6vF6NZnrWZqkDIMunEqnD7POotlhsUbrWOq+JHlABKPGb/LCFXIxaD4bCYcCJwYgT
 r9GbLZGC1m/YIO0MkGpeFhHHO53Yk/XOYVvaB6XF3XrCReIS4va+Y5CApgcClNeajfH1uwUAk
 TIvk3hAQuaHlKxsr8fObYDY8iGw=
Precedence: bulk

On Tue, 2023-05-16 at 16:41 +0800, Chen Yu wrote:
> >
> IMO this issue could be generic, the cost of C2C is O(sqrt (n)), in theo=
ry on
> a system with a large number of LLC and with SMT enabled, the issue is e=
asy to
> be detected.
>
> As an example, I did not choose a super big system,
> but a desktop i9-10980XE, launches 2 pairs of ping-ping tasks.
>
> Each pair of tasks are bound to 1 dedicated core:
> ./context_switch1_processes -s 30 -t 2
> average:956883
>
> No CPU affinity for the tasks:
> ./context_switch1_processes -s 30 -t 2 -n
> average:849209
>
> We can see that, waking up the wakee on local core brings benefits on th=
is platform.

Sure, tiny ping-pong balls fly really fast in a shared L2 siblings. The
players being truly synchronous, there's not a whole lot of room for
resource competition, and they do about as close to nothing but
schedule as you can get.

That's not a workload, much less one worthy of special consideration.
It is a useful tool though, exposed a big socket issue, good job tool.
Changing task placement strategy in response to that issue makes zero
sense to me.  There are many ways to schedule and wake other tasks at
high frequency, all use the same paths.  Reduce the pain that big box
sees when playing high speed ping-pong, and real workloads will profit
in boxen large and small.  More in large than small, but nobody loses,
everybody wins.

	-Mike
>