Received-SPF: pass (google.com: domain of linux-kernel+bounces-178838-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <2efcae46-736a-4809-8530-7dde3977f3ce@intel.com>
Date: Tue, 14 May 2024 08:08:19 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v17 7/9] x86/resctrl: Add new monitor files for Sub-NUMA
 cluster (SNC) monitoring
To: Tony Luck <tony.luck@intel.com>
CC: Fenghua Yu <fenghua.yu@intel.com>, Maciej Wieczor-Retman
	<maciej.wieczor-retman@intel.com>, Peter Newman <peternewman@google.com>,
	James Morse <james.morse@arm.com>, Babu Moger <babu.moger@amd.com>, "Drew
 Fustini" <dfustini@baylibre.com>, Dave Martin <Dave.Martin@arm.com>,
	<x86@kernel.org>, <linux-kernel@vger.kernel.org>, <patches@lists.linux.dev>
References: <20240503203325.21512-1-tony.luck@intel.com>
 <20240503203325.21512-8-tony.luck@intel.com>
 <0178e84e-d55f-47bf-b8b0-58e05fcaa108@intel.com>
 <ZkJIZdU2knEUJN7Q@agluck-desk3>
 <f49931b0-d9bf-45f0-ab35-93b1a78f6b97@intel.com>
 <ZkKupOKRu5S7Rkgx@agluck-desk3>
Content-Language: en-US
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <ZkKupOKRu5S7Rkgx@agluck-desk3>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SzkvQ29mTW9ZdkZVaUhZN0lwamZmR2U3cG1leHBOakFrczFHQyswZVZuVXF4?=
 =?utf-8?B?RGhnQkUyRHBQWTBsc214VC9XeXNKcWthQ2RDSm5pRzVPZEZ4cnh3NFRaMWNC?=
 =?utf-8?B?Q2hha3BzdDBZMFAyNlp3WDh2dXY0bEZDOTl5Q1ZzWmQxN0ZVZkVrUzFzWWMy?=
 =?utf-8?B?TU5wWjBTSldwY1lCN1loV3BKRHVuc0o2cXBOTjdNQ2pmOWwrMHowSHUyOXpz?=
 =?utf-8?B?eUk0ZVRoT3cxaGs4VHQ4ZzF0RnpYSlFTcDdGdnlXRE9XMXNrcDRrNUhFTzhn?=
 =?utf-8?B?U3ovcGttTVdQQVY1S1k0ZE1tY0I3VnpTazdQLzcwUlFkd0ZpSmxIdE1PMUt5?=
 =?utf-8?B?V2RvK0ROY3dFbzlGeC85M1NEaDV3bXN6WEVZTDRWYmlLYWRxcW5sSXFCd1Zz?=
 =?utf-8?B?N1JUWUhiOElscGZXL0NXaHM5bjVkZTZudWZzdWRXbUFwMEZlUzE1dVN1NTEx?=
 =?utf-8?B?K0lZSzlzS2NDM1BjZmZ2WUwzM0pwWTB1Qk9Qd24vRzN1azMyTVZOdDU1Y1FR?=
 =?utf-8?B?QVA5RjNmUHhibjVKYndmSDcxYmpyNnZkVVovMzRyaEY5NDRKZFFWRmRGK1lN?=
 =?utf-8?B?QlZEY2VRZjdIWCsxcVpYZlcxQUZCK05FN0V3VEpNQndaRlltR1prT3lwMk1z?=
 =?utf-8?B?OE5scUIwL2ZDMXBoaHB5ZXhVeCtFR2xMTEV0ekJGSXdSYlFOa1ZjQVdyWG1i?=
 =?utf-8?B?dkxlV3hNS0I4UzNsQ0o1THZheGlTTXZkakx6S1BHT2JvbFYxY1NSOEIrK25Q?=
 =?utf-8?B?Y001c0RoL1RYRkNBTDZ4L1ZPa0FiNyt1ZC96OTd5SUxpbU9lTkdGMzJ1SDMv?=
 =?utf-8?B?aUg2Tll0SFFhSjlMMzBQeUpCb1c5bFNnZ0V5WEp5VFpSaUI1cEMrWVFoc2w2?=
 =?utf-8?B?Y2pZQzRzK0hqYUYxeTJaMGVXeTl0TGwwcWI0TWlKK3V2R1J4UjBQb1puaE1l?=
 =?utf-8?B?U3JHNmtuZmVZZkZrZmdPcHVMV21YMlFIeDlrZ2NJU1JJa1ZLTXUwQ1dNcUlO?=
 =?utf-8?B?aUxBTlZ4TVhJY2NQWnNzM2FxUWdKN0hPbG40U1dJQTNGalRmd1pzYldOZ0Q5?=
 =?utf-8?B?SUdGL0hEaElPcTlDZGpFaUgyYVFYZWhPS3h3Z0ExcWo4eDN6M0NqeXdCZDA3?=
 =?utf-8?B?ZWJBM0t0MzdtalQ1TFQzVHdqdHZYUmg3YlZRUU1INlhXNFVjcmxGSkRFUjFL?=
 =?utf-8?B?VXMyM1NBbHFSOEJQMTcxQTJEbFVBbGV3SnhzVEJISTJwQzNKTmxiUERSU1Ay?=
 =?utf-8?B?SjFaY0hYWUJ6OUFWbU93NWxqN2pPeVd6UkN4MUNyaWQ1Y1hNRG9lVnZvV0hM?=
 =?utf-8?B?UEZrM2ZwQ1NycHBIMGFIbTZQMlVvMnVzOEp5VDdlZE80Tkt2UlRvS1l0SXZQ?=
 =?utf-8?B?UzA1UTZBay9NRWJ6b3ZWL1ZkOWxubFlLWXFiTHdPSUkvOS91d3pQdmdJMGZn?=
 =?utf-8?B?bFk3KzZrbS9FbEx2MGdFbTA4K1p6YndqVDBCcHY2SUVOeld4Y2NUMm9mRzdk?=
 =?utf-8?B?cmJZZXZKcy9CeWlLWGkxcDNEdnNxY1E1UnZpZFZFaklMMFFGdm4yRjd6L3NU?=
 =?utf-8?B?L2RsUDBBTWxZdWcrQlhTc0J0MlFNWHV1MzZEbDhkbzRyU2w0NGJ3VG1HdWR5?=
 =?utf-8?B?SVV3VkU4YXQ5WnpBRU1tTDRBWVJHNWJpY1VGUEpYdko4aFhkTmdjS2UycFhC?=
 =?utf-8?B?c0x4TW1hYkRZcFRJYVBsOXo5LzNDNVREZVlWamFOSzhLSXlsbHdSNnBZQTdQ?=
 =?utf-8?B?TURUM1VVYXBMZmpUQ2VaUTE1V2JPeDRJWXphTkNrRnZ4RlQ3cmlGRXVMa200?=
 =?utf-8?B?TWNUc0ZmQmhGa1BxM1plV0hVUEVISENoVWg0TUdXZUFiUnNjWTZCMEpaVVZC?=
 =?utf-8?B?aE5wdWdNT1RGaHZWTnZDVlRPdytPWGl2R09laDlod1p6SkJ1MVU4L0RNWVJ0?=
 =?utf-8?B?WWUzUThEN0VaOE1xVFZjVUo1OUtPUlRzMk9Iczc2K0VRamFobGZoNE5US0RU?=
 =?utf-8?B?Z3dZQlBaWk8wd1dHTVUwRmhxbXR6ejRINHVXYVhZY2JpVDVaV3UxNWVrdjNL?=
 =?utf-8?B?UTZKTm92UnBzUmpjWnEwa0dxbTg5MlV6WGZ6R1loSUoyUk1yKzlOUVRLNmI3?=
 =?utf-8?B?RE5Wc0JydXJPb2dlNU1ONjhvZHdVM1piRmtsMmRyNXQ2eVNRZ2szWEJWTmFo?=
 =?utf-8?B?aHc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 535f3ebc-34ca-4da6-81e9-08dc7427b174
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2024 15:08:21.2096
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: a0Sqtr0X32KBSJUHVqyb0ZgL6EIKU1LlGg+RInN1MUnWHgqC9QrqCJfNtWa81K7qFKxMh4rMO+DmLXDE32qa7yOLks8bqAJ4M/k0dfUSrK0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR11MB6865
X-OriginatorOrg: intel.com

Hi Tony,

On 5/13/2024 5:21 PM, Tony Luck wrote:
> On Mon, May 13, 2024 at 11:53:17AM -0700, Reinette Chatre wrote:
>> On 5/13/2024 10:05 AM, Tony Luck wrote:
>>> On Fri, May 10, 2024 at 02:24:13PM -0700, Reinette Chatre wrote:
>>> Thanks for the review. Detailed comments below. But overall I'm
>>> going to split patch 7 into a bunch of smaller changes, each with
>>> a better commit message.
>>>
>>>> On 5/3/2024 1:33 PM, Tony Luck wrote:
>>>>
>>>> (Could you please start the changelog with some context?)
>>>>
>>>>> Add a field to the rdt_resource structure to track whether monitoring
>>>>> resources are tracked by hardware at a different scope (NODE) from
>>>>> the legacy L3 scope.
>>>>
>>>> This seems to describe @mon_scope that was introduced in patch #3?
>>>
>>> Not really. Patch #3 made the change so that control an monitor
>>> functions can have different scope. That's still needed as with SNC
>>> enabled the underlying data collection is at the node level for
>>> monitoring, while control stays at the L3 cache scope.
>>>
>>> This new field describes the legacy scope of monitoring, so that
>>> resctrl can provide correctly scoped monitor files for legacy
>>> applications that aren't aware of SNC. So I'm using this both
>>> to indicate when SNC is enabled (with mon_scope != mon_display_scope)
>>> or disabled (when they are the same).
>>
>> This seems to enforce the idea that these new additions aim to be
>> generic on the surface but the only goal is to support SNC.
> 
> If you have some more ideas on how to make this more generic and
> less SNC specific I'm all ears.

It may not end up being totally generic. It should not pretend to be
when it is not. It makes the flows difficult to follow when there are
these unexpected checks/quirks in what claims to be core code.

>>>>>  	}
>>>>> +
>>>>> +	return 0;
>>>>> +}
>>>>> +
>>>>> +static int mkdir_mondata_subdir(struct kernfs_node *parent_kn,
>>>>> +				struct rdt_mon_domain *d,
>>>>> +				struct rdt_resource *r, struct rdtgroup *prgrp)
>>>>> +{
>>>>> +	struct kernfs_node *kn, *ckn;
>>>>> +	char name[32];
>>>>> +	bool do_sum;
>>>>> +	int ret;
>>>>> +
>>>>> +	do_sum = r->mon_scope != r->mon_display_scope;
>>>>> +	sprintf(name, "mon_%s_%02d", r->name, d->display_id);
>>>>> +	kn = kernfs_find_and_get_ns(parent_kn, name, NULL);
>>>>> +	if (!kn) {
>>>>> +		/* create the directory */
>>>>> +		kn = kernfs_create_dir(parent_kn, name, parent_kn->mode, prgrp);
>>>>> +		if (IS_ERR(kn))
>>>>> +			return PTR_ERR(kn);
>>>>> +
>>>>> +		ret = rdtgroup_kn_set_ugid(kn);
>>>>> +		if (ret)
>>>>> +			goto out_destroy;
>>>>> +		ret = mon_add_all_files(kn, d, r, prgrp, do_sum);
>>>>
>>>> This does not look right. If I understand correctly the private data
>>>> of these event files will have whichever mon domain came up first as
>>>> its domain id. That seems completely arbitrary and does not reflect
>>>> accurate state for this file. Since "do_sum" is essentially a "flag"
>>>> on how this file can be treated, can its "dom_id" not rather be
>>>> the "monitor scope domain id"? Could that not help to eliminate 
>>>> that per-domain "display_id"?
>>>
>>> You are correct that this should be the "monitor scope domain id" rather
>>> than the first SNC domain that appears. I'll change to use that. I don't
>>> think it helps in removing the per-domain display_id.
>>
>> Wouldn't the file metadata then be the "display_id"?
> 
> Yes. The metadata is the display_id for files that need to sum across
> SNC nodes, but the domain id for ones where no summation is needed.

Right ... and there is a "sum" flag to tell which is which?

Reinette