Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 19 Jan 2023 22:58:49 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     K Prateek Nayak <kprateek.nayak@amd.com>
CC:     Peter Zijlstra <peterz@infradead.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Tim Chen <tim.c.chen@intel.com>,
        Mel Gorman <mgorman@techsingularity.net>,
        Juri Lelli <juri.lelli@redhat.com>,
        "Rik van Riel" <riel@surriel.com>, Aaron Lu <aaron.lu@intel.com>,
        Abel Wu <wuyun.abel@bytedance.com>,
        Yicong Yang <yangyicong@hisilicon.com>,
        "Gautham R . Shenoy" <gautham.shenoy@amd.com>,
        Ingo Molnar <mingo@redhat.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>,
        "Daniel Bristot de Oliveira" <bristot@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Hillf Danton <hdanton@sina.com>,
        Honglei Wang <wanghonglei@didichuxing.com>,
        Len Brown <len.brown@intel.com>,
        Chen Yu <yu.chen.surf@gmail.com>,
        "Tianchen Ding" <dtcccc@linux.alibaba.com>,
        Joel Fernandes <joel@joelfernandes.org>,
        Josh Don <joshdon@google.com>, <linux-kernel@vger.kernel.org>
Subject: Re: [RFC PATCH v4 0/2] sched/fair: Choose the CPU where short task
 is running during wake up
Message-ID: <Y8laqT+7OX0I3pCu@chenyu5-mobl1>
References: <cover.1671158588.git.yu.c.chen@intel.com>
 <105eede3-2125-5a1e-836c-954fc67f575b@amd.com>
 <Y65RNzj522d7Q3OI@chenyu5-mobl1>
 <45b9c3e5-7ab7-0256-790e-cee92c37762d@amd.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <45b9c3e5-7ab7-0256-790e-cee92c37762d@amd.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?b9bRXnAGXumCub+xa1EcsCqntQgyBREhtiu7Arcq0rPzDDmNI69OQjbJA6pL?=
 =?us-ascii?Q?8Aiv4YcZC1yjqh4seNOTxdDzc5DfCYeNKC22FQANVKQb+stzYQjOK3mKPS4c?=
 =?us-ascii?Q?BPwKFW01zeJjK42Gl+Zwf+rC7vPzUwrB3SCvyVft5YH9G8G6syLUx/Oyk/go?=
 =?us-ascii?Q?GOnWARKN1i9EieXkyqFORNPKv5FCwkQ0WLSHRxBGdI6JNz5L2WAglX1+unwo?=
 =?us-ascii?Q?Z9T57TZBUmGHPgv73azbG335by5Nf57REEEj+J0X5XvwrK5q/o3U0tlBdiX6?=
 =?us-ascii?Q?vLmUYRsK2MU1CgsN0SV8NdAQ70H4w5Yot7ug5yXx49YYjmpXvrBxYa6N8LDh?=
 =?us-ascii?Q?7jNirqH7w1C0cFcsA4AenL/6wZNVyY0E20tRmkEegJmDVZRGnmFalR+ihSQz?=
 =?us-ascii?Q?jepkk/VXHly8OWIetDwgox/r+JMaz8MT/hmfCG9N/yg04RmVr9qAvXmTisgF?=
 =?us-ascii?Q?GU8HrF0IK/Yggq130fCvo/pd74y9heln1YotuKkb3uWmz0hpgXMbYY25xpxv?=
 =?us-ascii?Q?O/cenO9Jk2zjjaw9K1Ti+nVvzINysdwxnCisa/J0TTPEYrqtPsPlQvens9ls?=
 =?us-ascii?Q?IBdROab+YcvqZFTzgKvV6S/1kS+2Cds0JoUMJVuva4flF1INboNWm/aoqKvQ?=
 =?us-ascii?Q?qyjVOdwDD8mUOsuYgHn0/lcvfbeFgQl2RII8a+k2PjR1cDKnfXn7/NXY7a0p?=
 =?us-ascii?Q?7IZ/S0WJvUTu2LHXdeERgaZdtAGvTw04Yd8Mnv6z5wgfE1UQ/KS3j3Msn4Zk?=
 =?us-ascii?Q?JKmiX16YKCDZiH5Ffbl/wyi3BBvLx9MV3U70pI8RGGm9EZZLyRKJmla661os?=
 =?us-ascii?Q?yKhA9ScqeQzSLLcQf7reHDz4+ToS9zT3ch/bqrYVUirpbHa7PYRnOUOmYBUr?=
 =?us-ascii?Q?L0qpCbjksDY0u7rxUCguEJSO/sCF6AAEpaNHVrymwYZJJ8zQ5YCSm9mJPyF1?=
 =?us-ascii?Q?ryRtrgw7P2nIzZXm0rW5tRmwPINK1HDqh/qLSfREoAMiRk+LWtDDknXLZEih?=
 =?us-ascii?Q?x6DQKMfXswzzC9igGcre4kAd6PaSC8lUPY+KL3Jwktl2WUMkun8umPQ29C44?=
 =?us-ascii?Q?Nve1/0Ubt2ikYErZsBXXQZGwXfP5X0vqs/hR4uLW9GODyRSB2zjyPYXEKzt6?=
 =?us-ascii?Q?v2gYCs02xcK18EEJbz4eFj+uTDE17Qw9GT9LkxcDb0a0KP8TqKRR9Sw0rh1f?=
 =?us-ascii?Q?HEtSTNzwLuZj/KtBwyIttfx1uet69k7f32tfkCjt6IHNavC62MbBzDWkhEuf?=
 =?us-ascii?Q?0HhXS9bwXkbyFoS6fUKWfq0OUX9BQ3xsSQdUM4hggvQZdQ2lfoU7jvyoGakN?=
 =?us-ascii?Q?ocTurtjQXOrt7xh/LJ+qXvfnffAG1ydIMOVB08nA89CB1LH1P+cb6EQ2XInv?=
 =?us-ascii?Q?mg5BaL3RwYH82UCKZYoHSdfmgJ5Qf15DTI9pjUWZqW80pnE0Ik9i/XSD0rQ2?=
 =?us-ascii?Q?T6fEE8qDsAemrGcWM62kaZq9JTWQPzhWlaaNt4vTK+sAKqaE/6Js8sWWFqQE?=
 =?us-ascii?Q?+9TaalpYkFEaK9q3Pq7CtYDZFlPXHTRNhhLgu4TIoYRucpYp3amukXWvFlhB?=
 =?us-ascii?Q?WsbaiO97C5iDSWjtAr6AALIa59wkDGvMIMyri1CC?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 62ccedfd-a46b-45a1-c72d-08dafa2dba4e
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jan 2023 14:59:13.5511
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1J/iks0ytg1/5wmHkT9wH1mmaEhOgsYiT9cAcNJkwUOxwf57JpR6t0XFme06mB6Tz5+aCM5i3LyRwAfmRyyPIA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR11MB5233
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 2023-01-16 at 16:23:13 +0530, K Prateek Nayak wrote:
> Hello Chenyu,
> 
> On 12/30/2022 8:17 AM, Chen Yu wrote:
> > On 2022-12-29 at 12:46:59 +0530, K Prateek Nayak wrote:
> >> Hello Chenyu,
> >>
> >> Including the detailed results from testing below.
> >>
> >> tl;dr
> >>
> >> o There seems to be 3 noticeable regressions:
> >>   - tbench for lower number of clients. The schedstat data shows
> >>     an increase in wait time.
> >>   - SpecJBB MultiJVM performance drops as the workload prefers
> >>     an idle CPU over a busy one.
> >>   - Unixbench-pipe benchmark performance drops.
> >>
> >> o Most benchmark numbers remain same.
> >>
> >> o Small gains seen for ycsb-mongodb and unixbench-syscall.
> >>
> 
> Please ignore the last test results. The tests did not use
> exactly same config for tip and sis_short kernel which led
> to more overhead in the network stack for sis_short kernel
> and the longer wait time seen in sched_stat data for tbench
> was a result of each loop taking longer to finish.
> 
> I reran the benchmarks on the latest tip making sure the
> configs are identical this time and only notice one
> regression in Spec-JBB Critical-jOPS.
>
> tl;dr
> 
> o tbench sees good improvement in the throughput when
>   the machine is fully loaded and beyond.
> o Some unixbench test cases show improvement as well as
>   ycsb-mongodb in NPS2 and NPS4 mode.
> o Most benchmark results are same.
> o SpecJBB Critical-jOPS are still down. I'll share full
>   schedstat dump for tasks separately with you.
> 
Thanks Prateek! I checked the task duration for these workloads,
they fall into the short duration task range, so SIS_SHORT takes
effect.
[snip]
> 
> SpecJBB Critical-jOPS performance is known to suffer when tasks
> queue behind each other. I'll share the data separately. I do see
> the average wait_sum go up 1.3%. The Max-jOPS throughput, however,
> is identical on both kernels which means sis_short does not affect
> the overall throughput but only for the critical jobs, do we see
> the regression due to possible queuing of tasks.
>
If I understand correctly, most workloads on Zen3 prefers to be spreaded
on idle CPUs in the same LLC, except for the scenario when the system is
extremly busy(and SIS_UTIL handles that). As Zen3 has 8C/16T per LLC,
it is unlikely to trigger the race condition I described in PATCH 2/2.
I'm preparing for a patch to also take nr_llc into considerdation.

thanks,
Chenyu
> --
> Thanks and Regards,
> Prateek