Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38;
Date:   Mon, 6 Nov 2023 13:52:44 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     K Prateek Nayak <kprateek.nayak@amd.com>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC:     Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Peter Zijlstra <peterz@infradead.org>,
        <linux-kernel@vger.kernel.org>, Ingo Molnar <mingo@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        "Vincent Guittot" <vincent.guittot@linaro.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Swapnil Sapkal <Swapnil.Sapkal@amd.com>,
        Aaron Lu <aaron.lu@intel.com>,
        "Tim Chen" <tim.c.chen@intel.com>,
        "Gautham R . Shenoy" <gautham.shenoy@amd.com>, <x86@kernel.org>
Subject: Re: [RFC PATCH v2 0/2] sched/fair migration reduction features
Message-ID: <ZUh/LK4iy3ukVaCn@chenyu5-mobl2.ccr.corp.intel.com>
References: <20231019160523.1582101-1-mathieu.desnoyers@efficios.com>
 <85b710a9-5b26-b0df-8c21-c2768a21e182@amd.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <85b710a9-5b26-b0df-8c21-c2768a21e182@amd.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?fGnLARxo0Q6L5gEKloCWmQUla18wiRwmsqu6THmEb31qX5q0PYsOZYzlXbjp?=
 =?us-ascii?Q?WvBuqCV50RPw+VMIv10W62B7SxdlnC5m9wjMipE4TU0DrqCZEgZlIraBkHDO?=
 =?us-ascii?Q?UEMQFdjWuFk/mEE9NGXzH5O3HkgvzqcCCcvWS70xMiXCelLQ4BlaP/Tr2lJ7?=
 =?us-ascii?Q?4F+oyHIYwqxaA/3UVTvWFRexi9vOkYY3nNR8Jjix2a8Sr+3nyrSXxzSXf5xI?=
 =?us-ascii?Q?FZwT5GyL1OU5J9DX+kVhj+3PDwI7xD7Xprj5puZVkHBYYA2W+W9WLXqxCglF?=
 =?us-ascii?Q?Y43L7wWMpmMdIM4kSKjKwa/1IXdn3NYzZw6aNWzKqhTzhMFCuAQ3a1kYihC3?=
 =?us-ascii?Q?e13Pb39ZjQ460IRb/2iyRC3r8xFBGG98uL1gns0WUS8034n19HNiChL3Z9L0?=
 =?us-ascii?Q?ipi32U7oNyhiqRnaVrKNn8zuL9Kdchkctcitw3bzatRlB6h3TFayQXV8y3UV?=
 =?us-ascii?Q?hK+lvnTnFcwJNE81rm7oUNhidbHlfr3AKQCx1WMxRvgFB5nbLwvLjsAnSA5x?=
 =?us-ascii?Q?WvZ0Ile1+hML4yL0212nK1OAFlTrN6JZZ5jp+56mKJme+HvZCRDfvJiS9nrp?=
 =?us-ascii?Q?NNk9QlYOgS3lkcOfGHy2GYd9MDxJ7KeEHcF49pjGd4r7Lv5e2Th26iGHU8CT?=
 =?us-ascii?Q?G+T5VoCT2SL63yMP6MkI47zhF1LhOrRupZLDZVsY9X/nDTnK7H98gKrFpszL?=
 =?us-ascii?Q?huMkHE/+a3TSsdJgOdikl5ZV2izkA/vPhFXKjTveMacFTSCILLlEVKMNC07x?=
 =?us-ascii?Q?zS+aTaX2xaSGBRQOfCROOy0kA+tHv56iovrHsGwRibdZNUmrnFTKg6KW8Psf?=
 =?us-ascii?Q?/un3jaxh2AXtJIWwdoLDn2uEPRong+uw6UE8pMmuyZMjstNn+fIw/qa55Hf8?=
 =?us-ascii?Q?cDhzVfmr2147uoIf0m0kMixoa54MqmRjVvRm2BP8SPcWeIwdE/CRfsAIwOmM?=
 =?us-ascii?Q?rOwGMaXcT7l/OzG8q5dgV8MJuRzDYcL1y8rdlPh77KQdV2g8INcUeGdpOQUz?=
 =?us-ascii?Q?bouN+Rls8OEW2gu8ycpW6awOZgR9vmLaYQQj/s5SL1oyK2AKcQ57huh4bolF?=
 =?us-ascii?Q?EG6lvKBaAmQ9x90kX4xxz2gBLni0JhVBGkSGFYHv34SRDYdqjCPCgis3JaWu?=
 =?us-ascii?Q?GJB/1L/ZFUSgf/wTeV671hPoyQhEs15S+0P2XDTwBoGSbyyZsVmHXHkx00lP?=
 =?us-ascii?Q?3l8LTuHzzKI1656ylvuNIBzy3UqeUiDhuXfF1XR1cgYyBaiMwMyFMAGvVB6D?=
 =?us-ascii?Q?2ntoKvH2ifOUkQ9+sHFxp+KpCGeOyqJP+nE1jIDA2iSh+MoLwIHRHH2B80Fz?=
 =?us-ascii?Q?d4Hp+gi7ahAmYKB9pK/dnkBcwfniQqzGs8GMzw2OeZH0B3d8N7KPzg/OD6mv?=
 =?us-ascii?Q?c/7vLfwgf2Qylcs+dNAtRDPMj+71bJ5QJzG3Fyf9kxaI/Tq0jI5H6Qs/vlue?=
 =?us-ascii?Q?NQ7jh0M0kFoAN681PdKmWehGxA8xTlKza8BjFN2uoQgnuiP5vxlznSDJzE4V?=
 =?us-ascii?Q?d6OiYizvPZ5UHSi77bq7+0dchvRDelZ7Wai4z16xa8J56I8g70kJQbKtS+2/?=
 =?us-ascii?Q?Mswoz3HwpNvJQOLZvpaSG7SXyxLoiD9GiC9SN9Vd?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 20508bee-64f3-4933-773c-08dbde8ca44b
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Nov 2023 05:53:03.9829
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Rh5VZmh4Grm4r2eXno6tnvwdBn74KgI7LiyNWvjH0TDJZzTraHSu4p/X1RAa+qDLnXs3ie5O5xzZwkxMYTEhfQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB7044
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-1.4 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Sun, 05 Nov 2023 21:53:23 -0800 (PST)

On 2023-10-27 at 08:57:00 +0530, K Prateek Nayak wrote:
> Hello Mathieu,
> 
> On 10/19/2023 9:35 PM, Mathieu Desnoyers wrote:
> > Hi,
> > 
> > This series introduces two new scheduler features: UTIL_FITS_CAPACITY
> > and SELECT_BIAS_PREV. When used together, they achieve a 41% speedup of
> > a hackbench workload which leaves some idle CPU time on a 192-core AMD
> > EPYC.
> > 
> > The main metrics which are significantly improved are:
> > 
> > - cpu-migrations are reduced by 80%,
> > - CPU utilization is increased by 17%.
> > 
> > Feedback is welcome. I am especially interested to learn whether this
> > series has positive or detrimental effects on performance of other
> > workloads.
> 
> I got a chance to test this series on a dual socket 3rd Generation EPYC
> System (2 x 64C/128T). Following is a quick summary:
> 
> - stream and ycsb-mongodb don't see any changes.
> 
> - hackbench and DeathStarBench see a major improvement. Both are high
>   utilization workloads with CPUs being overloaded most of the time.
>   DeathStarBench is known to benefit from lower migration count. It was
>   discussed by Gautham at OSPM '23.
> 
> - tbench, netperf, and sch bench regresses. The former two when the
>   system is near fully loaded, and the latter for most cases.

Does it mean hackbench gets benefits when the system is overloaded, while
tbench/netperf do not get benefit when the system is underloaded?

>   All these benchmarks are client-server / messenger-worker oriented and is
>   known to perform better when client-server / messenger-worker are on
>   same CCX (LLC domain).

I thought hackbench should also be of client-server mode, because hackbench has
socket/pipe mode and exchanges datas between sender/receiver.

This reminds me of your proposal to provide user hint to the scheduler
to whether do task consolidation vs task spreading, and could this also
be applied to Mathieu's case? For task or task group with "consolidate"
flag set, tasks prefer to be woken up on target/previous CPU if the wakee
fits into that CPU. In this way we could bring benefit and not introduce
regress.

thanks,
Chenyu