Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34;
Date:   Wed, 27 Sep 2023 14:59:29 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     K Prateek Nayak <kprateek.nayak@amd.com>
CC:     David Vernet <void@manifault.com>, <linux-kernel@vger.kernel.org>,
        <peterz@infradead.org>, <mingo@redhat.com>,
        <juri.lelli@redhat.com>, <vincent.guittot@linaro.org>,
        <dietmar.eggemann@arm.com>, <rostedt@goodmis.org>,
        <bsegall@google.com>, <mgorman@suse.de>, <bristot@redhat.com>,
        <vschneid@redhat.com>, <tj@kernel.org>, <roman.gushchin@linux.dev>,
        <gautham.shenoy@amd.com>, <aaron.lu@intel.com>,
        <wuyun.abel@bytedance.com>, <kernel-team@meta.com>
Subject: Re: [RFC PATCH 3/3] sched/fair: Add a per-shard overload flag
Message-ID: <ZRPS0cQo0/XcbkOj@chenyu5-mobl2.ccr.corp.intel.com>
References: <31aeb639-1d66-2d12-1673-c19fed0ab33a@amd.com>
 <20230831104508.7619-1-kprateek.nayak@amd.com>
 <20230831104508.7619-4-kprateek.nayak@amd.com>
 <20230831191103.GC531917@maniforge>
 <258c9412-445a-04f5-dd0d-1f6699a257b6@amd.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <258c9412-445a-04f5-dd0d-1f6699a257b6@amd.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ZBpQ0AnFdtwGkTvN70UOkDXdJxRVL4MBzTEzL1NZOzUE7GRLoK7l/86zYynA?=
 =?us-ascii?Q?5jUyMA1Eatq8VNwxgI4cO/pD7xLWDid/7S5SI4Or1hs8aXCMnNYxMoLaXwd0?=
 =?us-ascii?Q?UWEFSQk8RTPI+61tTPOxhAHWoe3BZTUcmx5MC8JU0SSBc6FPN3MMR0Bc1PU4?=
 =?us-ascii?Q?5COIaYbPlMnv6m3zA7e6wgLHN7nmTY/6HRkd5gVR6OWtbgcs/EYQDAjNnbsV?=
 =?us-ascii?Q?blLpvFxdANB8X5QNYF3Mg77RucKAWkD4G0fjKHnMcz6CAeop3aA7ZeISZ5uH?=
 =?us-ascii?Q?PbYtUhRbkK8x554Q7AeBmK2uuyWRPfsxt6Q2U32o5o8wJExAPI0JgAmWMGZ7?=
 =?us-ascii?Q?n7i6rlNC/anJM9SBxbCReCgkcDc1iUOvPEu+JwmXnYOvSXWnmwMtMXyOAfmx?=
 =?us-ascii?Q?nI3lNePNV/trZb8t5Yc7sYiBR+yCiA02TTX2Rl36oQJOVLdgM6BVhMxSbSis?=
 =?us-ascii?Q?1KyhSLqN+LWOu76Kft6TxVPvpcGULcngAisbxB2wnIuUyy3mcATc1wBYFLUS?=
 =?us-ascii?Q?m653y37mHAimfYrC7Uj/K86C6D4zCXb7nxplf4nJPMZBLGVIoc0tafGCSgCs?=
 =?us-ascii?Q?A0d7D2Of10lxxWmo4QIEjRyr7ykU+i0+rJ8WU+G5al37qtm7azo/VUCRj5XJ?=
 =?us-ascii?Q?ZIqlk4njAqxvWfnpPsCsxZg8XTvJ2FIsd/R5RLo4G32h2pGDp25pBkRs0bNc?=
 =?us-ascii?Q?x3G11klBj5bc0h6M6wLZJl62UFLe/OP8XO0rU2RWOjS0qOUDitcypTU69xVJ?=
 =?us-ascii?Q?odmwgX6COpWGQVFkFOs3YzlO93ND641hDIXgWs4hb6GNe576Wrfhw4QbicSe?=
 =?us-ascii?Q?UWNI7sz11/rbBqr0/KpjOb8sD5wa/JXH11XHlZI2UN0FP0B/4uCGNWGMi7Aj?=
 =?us-ascii?Q?3LgsbRWofhUrUFzLxSiKAwMbBnRzS1mXWQNQTRUrWRAjMnS9f+pE5ZKH2lNo?=
 =?us-ascii?Q?xOTE1d6f4dLDfPugwHQDs9+DbIA/nvXK5W8eNi9N1FYOUNy0F7k8TiJf4Ngy?=
 =?us-ascii?Q?vgMwrEaVNrt4iVVtg4DQCU1+XQ5uiQBwySHXbxxGLTXSb74ObgTLpiaz+6Mo?=
 =?us-ascii?Q?5oKuhq3cE5m8BP7mB88uggKEH2f5+aJUhYTKtIH7D1MjHQl06Qy7xMQLg3gD?=
 =?us-ascii?Q?4Bt8fTWU4hhip0iGxrX1IxXobUfWUGPGpqxsV2MHIF5PdV/FZQnzX7HLBcnl?=
 =?us-ascii?Q?zAzY7q0yZpjCzhpg5smZdo0ngKH6hPaxLl2IH4TJw1bbL57KnK+X/EAxlevT?=
 =?us-ascii?Q?7OePKbb36PMUnYhWdmDYRNLbfswH+huQqB4ZQgGDd7o2aQjqSnNsYvW0a6pU?=
 =?us-ascii?Q?mjyb7FuzmoXU8KuD/HLMWcRJH2wan9O9FL+nad21trdhleZ8BfGnCbGLiPQT?=
 =?us-ascii?Q?3zngh/nbgiiP5WhTOD0YIHQJFYZ7BALZd+XhqdYRQMMqBsQpEXZM8oqtQc3m?=
 =?us-ascii?Q?kWwLqOl043KbiKa+rTop+vjv6RknhX2lWaGm060tnXw4N5GKIotvD5DvBKwa?=
 =?us-ascii?Q?YXjM7Mp+t30Eiq892nkFEuiU4Vp9MdrQrbjgl+4AVcqkND/feMlRs3boIGqH?=
 =?us-ascii?Q?HMZNH+pvivOb8x+fSPCntuPQZhePesSUqgkg5aqG?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d1272379-51dc-4539-2640-08dbbf2756ae
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2023 06:59:48.4231
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 79PPUQC1BKvkglvJ1qXprJfpldN2zxkacrmDl/S5zPJXchJfmKijJrPQaIrnUL+du6freqwGQEbkDoip5+VLJA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB8454
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,
        RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Wed, 27 Sep 2023 00:00:37 -0700 (PDT)

Hi Prateek,

On 2023-09-27 at 09:53:13 +0530, K Prateek Nayak wrote:
> Hello David,
> 
> Some more test results (although this might be slightly irrelevant with
> next version around the corner)
> 
> On 9/1/2023 12:41 AM, David Vernet wrote:
> > On Thu, Aug 31, 2023 at 04:15:08PM +0530, K Prateek Nayak wrote:
> > 
> -> With EEVDF
> 
> o tl;dr
> 
> - Same as what was observed without EEVDF  but shared_runq shows
>   serious regression with multiple more variants of tbench and
>   netperf now.
> 
> o Kernels
> 
> eevdf			: tip:sched/core at commit b41bbb33cf75 ("Merge branch 'sched/eevdf' into sched/core")
> shared_runq		: eevdf + correct time accounting with v3 of the series without any other changes
> shared_runq_idle_check	: shared_runq + move the rq->avg_idle check before peeking into the shared_runq
> 			  (the rd->overload check still remains below the shared_runq access)
>

I did not see any obvious regression on a Sapphire Rapids server and it seems that
the result on your platform suggests that C/S workload could be impacted
by shared_runq. Meanwhile some individual workloads like HHVM in David's environment
(no shared resource between tasks if I understand correctly) could benefit from
shared_runq a lot. This makes me wonder if we can let shared_runq skip the C/S tasks.
The question would be how to define C/S tasks. At first thought:
A only wakes up B, and B only wakes up A, then they could be regarded as a pair
of C/S
 (A->last_wakee == B && B->last_wakee == A &&
  A->wakee_flips <= 1 && B->wakee_flips <= 1)
But for netperf/tbench, this does not apply, because netperf client leverages kernel
thread(workqueue) to wake up the netserver, that is A wakes up kthread T, then T
wakes up B. Unless we have a chain, we can not detect this wakeup behavior.

thanks,
Chenyu