Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Deng, Pan" <pan.deng@intel.com>
To:     "Chen, Yu C" <yu.c.chen@intel.com>,
        Tim Chen <tim.c.chen@linux.intel.com>
CC:     "Lu, Aaron" <aaron.lu@intel.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        "Chen, Tim C" <tim.c.chen@intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Li, Tianyou" <tianyou.li@intel.com>, "Ma, Yu" <yu.ma@intel.com>,
        "Zhu, Lipeng" <lipeng.zhu@intel.com>
Subject: RE: [PATCH v2] sched/task_group: Re-layout structure to reduce false
 sharing
Thread-Topic: [PATCH v2] sched/task_group: Re-layout structure to reduce false
 sharing
Thread-Index: AQHZpBf8y9gtMyr/606YJQEIlMf8Fa+cm1MAgAAivoCAAJuXAIAA+yqAgABvq/A=
Date:   Tue, 27 Jun 2023 14:52:50 +0000
Message-ID: <DM6PR11MB3260FC52E13ADB32AB745DFC9627A@DM6PR11MB3260.namprd11.prod.outlook.com>
References: <20230621081425.420607-1-pan.deng@intel.com>
 <20230626054756.GA435374@ziqianlu-dell>
 <ZJlDsbyNmcKtweg4@chenyu5-mobl2.ccr.corp.intel.com>
 <0a611696cbc63a3a9b99a7966b8fe4f7e20a014b.camel@linux.intel.com>
 <ZJqY5xnCv7BNkWuj@chenyu5-mobl2.ccr.corp.intel.com>
In-Reply-To: <ZJqY5xnCv7BNkWuj@chenyu5-mobl2.ccr.corp.intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?jA+5xJJ8IslKxEaLAhf+/SxFp5C2p6YsAQ57uQtT2KDcZ82R2r9cFfY1iKJE?=
 =?us-ascii?Q?4wKRRP4pp7F1DB7/xW207TRtxrlZOlnaXMaLWcVHM4nWRy0mFnZ7vAXRsbJ1?=
 =?us-ascii?Q?q+ziR45iGXAfEZJwWl7ofySUKTCGx6CrwuM0lVKuU1TV7A+yGQggO+O4HnRF?=
 =?us-ascii?Q?Ua80aUkv92aU4gSDBpgst7lP2pEKtVI91RPlB5LiEW7MxUGJvRqQtWtmrDDk?=
 =?us-ascii?Q?jSUYr7ENSaTVXNT2YrR65WjImpKEF6FHF/IIBjv9swU5YxLOeBFKJD34RYvX?=
 =?us-ascii?Q?l3Q5ylu4kpARBP/QCkjvSgkpwC68BdUDdYgWmbmTM9TOHbGFDY3s7V8Bs5FH?=
 =?us-ascii?Q?hR+ooUgatxr+FhaVj9qyebiKtnP1vEQ7m0K0H+HUaUN/WuQ+r8JVdh46dHEa?=
 =?us-ascii?Q?zz852WAREWzLm0S3QFvxVSkTeEBJTc5hxiI7U0WjSfuN47WF9LFVWS0e1XuM?=
 =?us-ascii?Q?ukYWlPSRVwlMV7jeoVrXnAy0NWc2w/HOz3A1doZshhT4Z+aBn0NeWKF7MZ2j?=
 =?us-ascii?Q?pxT/f7m0mjqPkA9XXRuZX/XLC6abbAsFSQQ1jIX1f6+Zy7OKfQG7qZQnKZKl?=
 =?us-ascii?Q?GX0pU+dtG1VGdkPTEOHZswse4UrB/xGsH5x4bjz63Y3VDH1rxU0AeXahJWt9?=
 =?us-ascii?Q?GDemXvWCibtzRlbLVDs/M26fAWQD3gF0umhI3Hv4SwCNASaWfUrjqpjPeh2+?=
 =?us-ascii?Q?vUuDa0M5iMeR3oz0VLdifHRufznw2refIkrxKa6rnZDUr9t6sXaZu3La1aM5?=
 =?us-ascii?Q?MYvh+6vylW3IF2mkMv0Ur5MdMkRSFJ8+XlNFFfqTJuA0/3p+KggR0fEclEga?=
 =?us-ascii?Q?g02Ek0QmwO43SMyMGf4hk4MBg2a5AMGJ5HsIX+iRYrIM6+OwbwusdGHuhqEF?=
 =?us-ascii?Q?GryGevswnlXnfpqGVtpnn1uXCuHF7+cHcbUC28bWbL24rHaPzillINP57OOU?=
 =?us-ascii?Q?hA3J54qTv+JhV8t4CwbLJmoxU1Tl4oYWQIy1wmeFigtfUlJJx55J5q3S0sNV?=
 =?us-ascii?Q?xXY2AE+fzyTLBu7XADLYge/UqTVPbqRDoFQv9J6Naeq+pZzbt+cYgQbUsVpt?=
 =?us-ascii?Q?ZjurhJBfOx2rJu7aR0/hsyxtdgpnBrSCyWwjgA/Bgz3xq1rIkrS61B9kIjlz?=
 =?us-ascii?Q?aIb7m5F6qPme03iCkeBXOrqplC9BVDrKammmlGnAaALdW2JmpxyOInBN+AdI?=
 =?us-ascii?Q?ve8Px4clXLrJjY4V5FJ9wuByNxyZKlVBWUZsf5JiUTKDc1hlqihUJCsWybPs?=
 =?us-ascii?Q?YMfWCpKuxq/UKrTGvESN6h2KL7C1VE5I4ZPIVORgzW9UwnEKLWtGxVLkSULH?=
 =?us-ascii?Q?507cID9IZpMwlwoC29p3vmsuHQbR+Uzhyfvqr6luL8HqA6NT0juVHamAhhKk?=
 =?us-ascii?Q?CkIiMbcDpEnL6g5QRZNcQ7RBx6wUhDr/M8yaJhD5mboL0y36WqdsTdvmlKuM?=
 =?us-ascii?Q?X9zGhzM23gqYx2mZGDXXkkybBXePyEFNnTi175XZk2GQaoQxW7K3hdgGfhuH?=
 =?us-ascii?Q?6RZYD1hWR656bHunVkiGeztoYIN/pJ0qATTawsuwaQdtVb4pw19VkISBCsz4?=
 =?us-ascii?Q?GzcYbft43hy1smwCkco=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB3260.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 99356a5f-ae0a-40de-2d49-08db771e2dc2
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Jun 2023 14:52:50.3314
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Mw/6kHEM3880pDDF5Qd+qFNvGQP3/6zPV3iuxzhelPSIiWRBZZ9XGCaaghcijw4bFRLjGPTqg+CfHigyaObOeA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB6919
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> -----Original Message-----
> From: Chen, Yu C <yu.c.chen@intel.com>
> Sent: Tuesday, June 27, 2023 4:08 PM
> To: Tim Chen <tim.c.chen@linux.intel.com>
> Cc: Lu, Aaron <aaron.lu@intel.com>; Peter Zijlstra <peterz@infradead.org>=
;
> Vincent Guittot <vincent.guittot@linaro.org>; Ingo Molnar
> <mingo@redhat.com>; Juri Lelli <juri.lelli@redhat.com>; Deng, Pan
> <pan.deng@intel.com>; Chen, Tim C <tim.c.chen@intel.com>; linux-
> kernel@vger.kernel.org; Li, Tianyou <tianyou.li@intel.com>; Ma, Yu
> <yu.ma@intel.com>; Zhu, Lipeng <lipeng.zhu@intel.com>
> Subject: Re: [PATCH v2] sched/task_group: Re-layout structure to reduce
> false sharing
>=20
> On 2023-06-26 at 10:09:10 -0700, Tim Chen wrote:
> > On Mon, 2023-06-26 at 15:52 +0800, Chen Yu wrote:
> > >
> > >
> > > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index
> > > ec7b3e0a2b20..067f1310bad2 100644
> > > --- a/kernel/sched/sched.h
> > > +++ b/kernel/sched/sched.h
> > > @@ -1958,8 +1958,10 @@ static inline void set_task_rq(struct
> > > task_struct *p, unsigned int cpu)  #endif
> > >
> > >  #ifdef CONFIG_RT_GROUP_SCHED
> > > -	p->rt.rt_rq  =3D tg->rt_rq[cpu];
> > > -	p->rt.parent =3D tg->rt_se[cpu];
> > > +	if (p->sched_class =3D &rt_sched_class) {
> > > +		p->rt.rt_rq  =3D tg->rt_rq[cpu];
> > > +		p->rt.parent =3D tg->rt_se[cpu];
> > > +	}
> >
> > If we avoid the assignment for non-rt task, do you see it further
> > improves c2c bounce?
> >
> I did not see noticeable extra throughput improvement from netperf, with
> above change + Aaron's fix + Pan's cacheline alignment adjustment.

Echo Yu.
Cacheline alignment adjustment resolved set_task_rq c2c bounce with load_av=
g,
The remained cycles of "assignment for non-rt task" is little according to =
perf record
data, the should be the reason of no extra throughput improvement.

Thanks
Pan=20
>=20
> thanks,
> Chenyu