Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33;
Date:   Sat, 7 Oct 2023 10:10:28 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     David Vernet <void@manifault.com>
CC:     K Prateek Nayak <kprateek.nayak@amd.com>,
        <linux-kernel@vger.kernel.org>, <peterz@infradead.org>,
        <mingo@redhat.com>, <juri.lelli@redhat.com>,
        <vincent.guittot@linaro.org>, <dietmar.eggemann@arm.com>,
        <rostedt@goodmis.org>, <bsegall@google.com>, <mgorman@suse.de>,
        <bristot@redhat.com>, <vschneid@redhat.com>, <tj@kernel.org>,
        <roman.gushchin@linux.dev>, <gautham.shenoy@amd.com>,
        <aaron.lu@intel.com>, <wuyun.abel@bytedance.com>,
        <kernel-team@meta.com>
Subject: Re: [RFC PATCH 3/3] sched/fair: Add a per-shard overload flag
Message-ID: <ZSC+FLLbia6YpSLt@chenyu5-mobl2.ccr.corp.intel.com>
References: <31aeb639-1d66-2d12-1673-c19fed0ab33a@amd.com>
 <20230831104508.7619-1-kprateek.nayak@amd.com>
 <20230831104508.7619-4-kprateek.nayak@amd.com>
 <20230831191103.GC531917@maniforge>
 <258c9412-445a-04f5-dd0d-1f6699a257b6@amd.com>
 <ZRPS0cQo0/XcbkOj@chenyu5-mobl2.ccr.corp.intel.com>
 <20231003210511.GB30978@maniforge>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20231003210511.GB30978@maniforge>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?HQCjGD2F1YbIRtZ+kA8XKtI83LFIOZ3BQJdUKjxqvYqWM/kYJeclgZq3FKFL?=
 =?us-ascii?Q?cK6j3NFLRiifEdH1dfIaYguG32nIuuE5fHOq8EA1VFvPVN5AqOvIvF+MABYx?=
 =?us-ascii?Q?FPsQFTVg3SU7+hm07lcRCuQkMqV32Jtc0mBnJ5+aM5tJ84nxIOesiIj6kmNC?=
 =?us-ascii?Q?x6jYK4F3kKQr49sve658rDj+ZsoQH9/28J3F7gQTQvXDOYB2gGeqVpvM3sP4?=
 =?us-ascii?Q?LMpIh2krR3rHsmJ4gRfTfQsm2Q+iYIr72R/xDyf02Xgkd1iLtjBOrIGEImF2?=
 =?us-ascii?Q?gSxaCR8m+cQvfBl9Og6oA0dC6kUMxanJP2QAljYfVBUT2YxY/SbCy6VMuELu?=
 =?us-ascii?Q?4bok+2yLW/m77on+vekCqMc9lI/sxzH2pdeOs9OBg9XyXy7fKrewXw4ZVD8t?=
 =?us-ascii?Q?0etwsWnJFFj5uDyK1IAkPXiHG5bZw4GZAGJTvDQF8XQcABfLtncHGlIrISKq?=
 =?us-ascii?Q?S/MsMci6hR286C0mp9pq7h5mUd1vBCOm0ACoySLeBqf+AyfCGvvXEU2B30XI?=
 =?us-ascii?Q?KGx7XWQFKwPbqcfd3lEiuke2qnOdvTfSHFHEp/nHia9nH5FGrGzPNdjloPDT?=
 =?us-ascii?Q?b5mdwvGxiYsYzYgAG+eyrn7z5wx1Kr7aMA7JWEN6+Llq4D6IQb6TIUVIvp2c?=
 =?us-ascii?Q?FYZChyJqueohy0lN0qI97Pnsjl/ahF4bMK0MloNHoYffNRG/FscyyCiQus2g?=
 =?us-ascii?Q?RqUMb45ewmwm3NwmXKol5OVaW7NH7FpdJvdQ1pa9IlkwHK5Mq+3f9py1QE13?=
 =?us-ascii?Q?W8Ltm6zPnRvKxbf9h6tSFCK+KibAPKEwUbOvMKUQu+SHFL9oOoJuQoCnrPim?=
 =?us-ascii?Q?750eh+iafCpO7zcbtZJ9FfRUmASMDDCApG1vAh6iKGRP5STv5vs4QdF1KSuf?=
 =?us-ascii?Q?PaKldYkcIuC8JDGCHP/CJRyo1JlZUG2QGE9L6GqtMmwTCTwJ6vGK0zHUJtah?=
 =?us-ascii?Q?m94DFKVSGfXZXPtIwOcMLGpn9yAEO9WYadd2mXsOJAi+e0lRRHHrYqQ249ba?=
 =?us-ascii?Q?IHKOdNXfHVgMA9DGsmOa63IBcjCxDQgBeJCQ2OVQ5xTgFTYCaZVG0qlmnOvr?=
 =?us-ascii?Q?Qi3LUVzBh2qmrpc488sWCTMV72BCGSQkk7RtSNIo8OfrqqWRiCmqIegLMykJ?=
 =?us-ascii?Q?Zqb7S7KZiXPFcjW0cP1fNsdnqj/O+f4nRVNv0MK76inXkhSP2sKDsQ8R0yvB?=
 =?us-ascii?Q?H1YIYIylBofQ6siqztWhjAn/HY0lDp8vkcjkp+P0seJJWstsLdBjOUlqvmTT?=
 =?us-ascii?Q?KLxSMqFCIQ9K1UKUTsGXHCmXH6TEDqptEStAsYl/Y35jgHtzvoNUcNldjU00?=
 =?us-ascii?Q?7OBUNW1EiPyr/yHkVpRkWQrqeNve+Aeqj8956Aqg2a9Lt/vAE6OtNNVjHM//?=
 =?us-ascii?Q?N2MMUDaZi0/M/fhn7b0BQu2/jquq21Pg0OqNbzqOwSEoP7f2ztQ3xuo1YV4O?=
 =?us-ascii?Q?gKSdkXGAme8hXyIMvbmUjMzyEN4j9vKc6tvk84eLxfAtT2Fh/q43N6q8l/YR?=
 =?us-ascii?Q?iaBV9DRdFpe5XhgQJijC8OapuO4swGH3kdnoGnOrBAogwqIrawgSNXwgd1sd?=
 =?us-ascii?Q?mklcHFaG1CXWeA8v70nB3TwxGHC6e0vjlQ/5sYhs?=
X-MS-Exchange-CrossTenant-Network-Message-Id: dcb732b0-e5cc-4ab6-e788-08dbc6da9cd1
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2023 02:10:44.5162
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: UIHe+9pjkhxThvWyinnBtswqId0+ssPdmmqshx42cZVySh2xmGixvJGqx858C5cSK2HKKV2rilRb1dJXZlL4Rw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB5918
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=2.7 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Fri, 06 Oct 2023 19:11:04 -0700 (PDT)
X-Spam-Level: **

Hi David,

On 2023-10-03 at 16:05:11 -0500, David Vernet wrote:
> On Wed, Sep 27, 2023 at 02:59:29PM +0800, Chen Yu wrote:
> > Hi Prateek,
> 
> Hi Chenyu,
> 
> > On 2023-09-27 at 09:53:13 +0530, K Prateek Nayak wrote:
> > > Hello David,
> > > 
> > > Some more test results (although this might be slightly irrelevant with
> > > next version around the corner)
> > > 
> > > On 9/1/2023 12:41 AM, David Vernet wrote:
> > > > On Thu, Aug 31, 2023 at 04:15:08PM +0530, K Prateek Nayak wrote:
> > > > 
> > > -> With EEVDF
> > > 
> > > o tl;dr
> > > 
> > > - Same as what was observed without EEVDF  but shared_runq shows
> > >   serious regression with multiple more variants of tbench and
> > >   netperf now.
> > > 
> > > o Kernels
> > > 
> > > eevdf			: tip:sched/core at commit b41bbb33cf75 ("Merge branch 'sched/eevdf' into sched/core")
> > > shared_runq		: eevdf + correct time accounting with v3 of the series without any other changes
> > > shared_runq_idle_check	: shared_runq + move the rq->avg_idle check before peeking into the shared_runq
> > > 			  (the rd->overload check still remains below the shared_runq access)
> > >
> > 
> > I did not see any obvious regression on a Sapphire Rapids server and it seems that
> > the result on your platform suggests that C/S workload could be impacted
> > by shared_runq. Meanwhile some individual workloads like HHVM in David's environment
> > (no shared resource between tasks if I understand correctly) could benefit from
> 
> Correct, hhvmworkers are largely independent, though they do sometimes
> synchronize, and they also sometimes rely on I/O happening in other
> tasks.
> 
> > shared_runq a lot. This makes me wonder if we can let shared_runq skip the C/S tasks.
> 
> I'm also open to this possibility, but I worry that we'd be going down
> the same rabbit hole as what fair.c does already, which is use
> heuristics to determine when something should or shouldn't be migrated,
> etc. I really do feel that there's value in SHARED_RUNQ providing
> consistent and predictable work conservation behavior.
> 
> On the other hand, it's clear that there are things we can do to improve
> performance for some of these client/server workloads that hammer the
> runqueue on larger CCXs / sockets. If we can avoid those regressions
> while still having reasonably high confidence that work conservation
> won't disproportionately suffer, I'm open to us making some tradeoffs
> and/or adding a bit of complexity to avoid some of this unnecessary
> contention.
> 

Since I did not observe any regression(although I did not test hackbench
yet) on the latest version you sent to me, I'm OK with postponing the
client/server optimization to make the patchset simple, and Prateek
has other proposal to deal with the regression.

> I think it's probably about time for v4 to be sent out. What do you
> folks think about including:
>

It's OK for me and I can launch the test once the latest version is released.

thanks,
Chenyu