Received-SPF: pass (google.com: domain of linux-kernel+bounces-101339-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Date: Wed, 13 Mar 2024 11:23:02 +0100
From: Maciej Wieczor-Retman <maciej.wieczor-retman@intel.com>
To: Ilpo =?utf-8?B?SsOkcnZpbmVu?= <ilpo.jarvinen@linux.intel.com>
CC: Fenghua Yu <fenghua.yu@intel.com>, Reinette Chatre
	<reinette.chatre@intel.com>, Shuah Khan <shuah@kernel.org>,
	<tony.luck@intel.com>, "Shaopeng Tan (Fujitsu)" <tan.shaopeng@fujitsu.com>,
	LKML <linux-kernel@vger.kernel.org>, <linux-kselftest@vger.kernel.org>
Subject: Re: [PATCH 2/4] selftests/resctrl: SNC support for CMT
Message-ID: <wsdxrn7hiovyzs7a7hom6yiwzucsdu2a4lpheblmhblscbkwjn@heyryn3cfiw6>
References: <cover.1709721159.git.maciej.wieczor-retman@intel.com>
 <75849cb145429798b21c23b6be4abd7ece9df57b.1709721159.git.maciej.wieczor-retman@intel.com>
 <6181937a-54b4-8b17-0c24-734764a205d1@linux.intel.com>
 <5983d1aa-6e1e-fb49-b95a-429a9decfe7f@linux.intel.com>
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <5983d1aa-6e1e-fb49-b95a-429a9decfe7f@linux.intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?SzSnCFijkm3eAy+bSEIpOO8bXxh7X8xmdo3McQloGGEpPr6b8J24y2ZR4X?=
 =?iso-8859-1?Q?3ZNEtg0U+96S1k5zqiR0068g06nolQ9cXN0nLICmemoYTXfyjtCyDeuw2t?=
 =?iso-8859-1?Q?kObV9B7G4ckBZfnZTOtB5pG2WHLXcGP7iAhMxKfFGj1KnZlmZcSDo6jjqU?=
 =?iso-8859-1?Q?kvHgAP+WLUqOUPIDvzg+0KbObJ5W3wvQ/1sLHAyp12pZvGPbo7PM0eN6EL?=
 =?iso-8859-1?Q?6oUjOmV7nsE/iSLxBwpJyZH5+ktZvoCVnOz1KHq4r3H5S1FSo6EFpArKcP?=
 =?iso-8859-1?Q?efa9Xvp3SHB2/Q9jRhFXZJ9DISwEMHLRucEgYatnVDZQwytXW6vV0YyuOZ?=
 =?iso-8859-1?Q?fA6GfM9s/aeECaMdyuIzsLclRoOzmlFSAOEhbDkUnNGTRssK4JAfF799GC?=
 =?iso-8859-1?Q?sVqW3OZ/MzjB90vTk1BNe8caoME9vzk8YoWAp4LCreRAfHcE1eGUJosX5j?=
 =?iso-8859-1?Q?d5EXUJBlaGwgWOBEbK9WJKqR1UV56ho3AcaS8++urw7Kd1dN2dTlw3wQBu?=
 =?iso-8859-1?Q?+3wzmuNK0VGtfahYA6yMnVzikNhuL7IJvYLtq+k4ostqlUF7CFxqYbQKLv?=
 =?iso-8859-1?Q?GdCUvs61b6sH2DidMId6vVkI3E+403t7TpbBHZVEd24dlHrHu0Vu0ZYa/6?=
 =?iso-8859-1?Q?4LEvxrcuPMCRLT41xLsvwfMGdER+mWxc32EWubRWKwdwV0pbtCkwT5eD/I?=
 =?iso-8859-1?Q?lKeJdyb9aOPkf6HQkK0vJK8K5Cb/En3ACq8C185FLvrVC+XmmHlH2ODAoy?=
 =?iso-8859-1?Q?EJ2Zu/owwLRgRlzaXc2Pd2GKWoq7HYWsUbsIJQ8UPfNYNRcgSbfg6Z7UWI?=
 =?iso-8859-1?Q?AXSJQNyXBMl6ogM1zKwlLCIb7Zsswt03nQJLN9VBupcy0pGXdTEWQHayeF?=
 =?iso-8859-1?Q?5WEBKJ9sJF4Fj1vEY6ECjpe+I57bhGS/I7QuCJCRtAT1bk8+nD2vS+KMUS?=
 =?iso-8859-1?Q?ZhNhPNpeSAB1YGAvtqg1T51HqBuc4suTxLmSbM+7okXy25ZeIW5Gl3MRmQ?=
 =?iso-8859-1?Q?VMdj5Kl7qjwVqcajoCI1yS4wrHiLKxvQL0jnnbEpZNIvoA1ydAKKMl6EHu?=
 =?iso-8859-1?Q?sIPVjpimpkJ6RR99JagNzIniwOk+k/DOO87GjL3bp8w4LSQEQPJ1UtQW1E?=
 =?iso-8859-1?Q?L2FOQ6WXspeUHJtMPRALZ//3E6E37/YSO/XzyowAgve/GO7P8tzm0JPts7?=
 =?iso-8859-1?Q?eMmfthqr5oITAfTjR5+dpfHqRgo1/JVR30Sx7kIrFfHQkUNjl3tadnMl8K?=
 =?iso-8859-1?Q?+hMqstNN6L2tKnCW5kCtHDLiZu9nic6ncG1f3Y1TeVumiYAzplBW8UO53Q?=
 =?iso-8859-1?Q?KKffeKi4YeYb12uaSBmKXFGUv1efP74T5QQYq7aCtf1WGNwX6vYMXOQV5T?=
 =?iso-8859-1?Q?kRpXa+fN7vkJihXZuAxHsLZv/HJHGz1sqmXEPMXjvXiFw5nlQcIaakEoGO?=
 =?iso-8859-1?Q?rOFYxlobWe8aIEuMa7WtPha8Y3eywwMrU+DFUKl2cTU3NxsBIscHT7mHyD?=
 =?iso-8859-1?Q?/mDmXaGDr1Dlq1tFuxBx104VlmO+VT/44lyZGbZKga4Yc7RUW7yQymwrYt?=
 =?iso-8859-1?Q?bxvMntJMdkxWrPtUO/3ae4UcjsdE6va3nmsgGRIpeSxgw3MGzaz0InMYmb?=
 =?iso-8859-1?Q?CJR+L9c4d10VZIjR3qQ1eNzL/yyfNmJj0yjp66YabJi0gYatnYzr5iyto4?=
 =?iso-8859-1?Q?NJI5Fcv2jZ3bpznhXXA=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 618d87eb-b5ee-4f90-af30-08dc43479695
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6231.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Mar 2024 10:23:12.9978
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6lOmZqliH5LtutJfEFo4ownYuXvEJ1n/ZCoLltqV7httDxXpA9ZSSYtTKKPvqOPBxdD4+z5hcL3q5y/agSAUkwT+JtduI1g5wHPCMT+Ckxc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6286
X-OriginatorOrg: intel.com

On 2024-03-08 at 15:59:02 +0200, Ilpo J?rvinen wrote:
>On Fri, 8 Mar 2024, Ilpo J?rvinen wrote:
>
>> On Wed, 6 Mar 2024, Maciej Wieczor-Retman wrote:
>> 
>> > Cache Monitoring Technology (CMT) works by measuring how much data in L3
>> > cache is occupied by a given process identified by its Resource
>> > Monitoring ID (RMID).
>> > 
>> > On systems with Sub-Numa Clusters (SNC) enabled, a process can occupy
>> > not only the cache that belongs to its own NUMA node but also pieces of
>> > other NUMA nodes' caches that lie on the same socket.
>> > 
>> > A simple correction to make the CMT selftest NUMA-aware is to sum values
>> > reported by all nodes on the same socket for a given RMID.
>> > 
>> > Reported-by: "Shaopeng Tan (Fujitsu)" <tan.shaopeng@fujitsu.com>
>> > Closes: https://lore.kernel.org/all/TYAPR01MB6330B9B17686EF426D2C3F308B25A@TYAPR01MB6330.jpnprd01.prod.outlook.com/
>> > Signed-off-by: Maciej Wieczor-Retman <maciej.wieczor-retman@intel.com>
>> > ---
>
>> > @@ -828,6 +828,8 @@ int resctrl_val(const struct resctrl_test *test,
>> >  	sleep(1);
>> >  
>> >  	/* Test runs until the callback setup() tells the test to stop. */
>> > +	get_domain_id("L3", uparams->cpu, &res_id);
>> 
>> Hardcoding L3 here limits the genericness of this function. You don't even 
>> need to do it, get_domain_id() does "MB" -> "L3" transformation implicitly 
>> for you so you can just pass test->resource instead.
>> 
>> Also, I don't understand why you now again make the naming inconsistent 
>> with "res_id".
>> 
>> If you based this on top of the patches I just posted, resctl_val() 
>> already the domain_id variable.
>
>Ah, I retract what I said. I see you actually want it only from L3.
>
>> > +     res_id *= snc_ways();
>
>I don't understand what this is trying to achieve and how.

We exchanged some private messages on this but I'll post the explanation here
too if anyone else was looking for it.

get_domain_id("L3"...) essentially gives us the number of the socket (on
platforms that have one L3 cache per socket - I still have too look into other
ones). The problem here is that to get an accurate reading with SNC enabled we
need to collect values from all nodes on a single socket that has the CPU our
test is running on. So we need to find the first node on that socket so then we
can loop through all the nodes on that socket. To do that we multiply res_id by
the amount of SNC nodes per socket (res_id *= snc_ways()) and that's it.

I'll add some helper with an explaining comment on what it does in the next
version.

>
>-- 
> i.


-- 
Kind regards
Maciej Wiecz?r-Retman