Received-SPF: pass (google.com: domain of linux-kernel+bounces-187996-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <013c9532-cacf-4604-b1d9-e90fdc80e7dd@intel.com>
Date: Thu, 23 May 2024 13:56:17 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v18 06/17] x86/resctrl: Introduce snc_nodes_per_l3_cache
To: "Luck, Tony" <tony.luck@intel.com>, "Yu, Fenghua" <fenghua.yu@intel.com>,
	"Wieczor-Retman, Maciej" <maciej.wieczor-retman@intel.com>, Peter Newman
	<peternewman@google.com>, James Morse <james.morse@arm.com>, Babu Moger
	<babu.moger@amd.com>, Drew Fustini <dfustini@baylibre.com>, Dave Martin
	<Dave.Martin@arm.com>
CC: "x86@kernel.org" <x86@kernel.org>, "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>, "patches@lists.linux.dev"
	<patches@lists.linux.dev>
References: <20240515222326.74166-1-tony.luck@intel.com>
 <20240515222326.74166-7-tony.luck@intel.com>
 <beee3369-0075-462e-8449-88fee807463e@intel.com>
 <SJ1PR11MB6083117715CD53FA4360C56CFCF42@SJ1PR11MB6083.namprd11.prod.outlook.com>
From: Reinette Chatre <reinette.chatre@intel.com>
Content-Language: en-US
In-Reply-To: <SJ1PR11MB6083117715CD53FA4360C56CFCF42@SJ1PR11MB6083.namprd11.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WU5oRjZRSVFLTjdYUERKVDRqeWowYmZ1OXJXQW1ZRkVqUDVKbUhqTnBTOGNV?=
 =?utf-8?B?WlBERjhNc1J0NG54ZmNGV21PR1JIenNVQ2w5WW9wMG9PQVoxMFdYUHpScHBM?=
 =?utf-8?B?YWlGaVcvZnhSbWF2Uit4QTFENWQ5N0hmU054YURPT3cvQXJvbUFvelZabnA4?=
 =?utf-8?B?a1R2VjIxQ1RnWDgzeGZtU09iTzlFZ1VCWCtZQ2dvc29MWFhhNThDWkNrYVhK?=
 =?utf-8?B?QjBMcnJZVkJiNjZZNS9yczVENEtPb0p6QTFEMDNZeXE1TVZwMExteE45Nlhj?=
 =?utf-8?B?ZTFoYlh3ZEx3Q0VRbm83SVhpa2JJWlNScVVZdzREcFdLc1hpT05CeVBpKytU?=
 =?utf-8?B?ckpYUTJnNG8yNEU3Q0U5cU10VTAzTkRqd3IycEI1WnhzKzE2a1NmUXRzMWtX?=
 =?utf-8?B?VUlJRTlBOG1TRjZCelFBUXNkMnFnMC9Xck9IWHR0c3JaTVZ3T3Y2YjMvQVlP?=
 =?utf-8?B?QllDRUFuOVFJS2U3QjN3N1IvRUpEOFF4NTByOFMxMjZhT2dYNnYyMU5wcGwy?=
 =?utf-8?B?Z2FKZW1GblJSWTNjc2RlV0dRWXN3TzlITkl2b1dTR0JnQ1BLSWNFQnVvcnI2?=
 =?utf-8?B?ZHVidVZ5eU56MExiZ1NUY3dhSHdmT1lqWTVDN3MwaUo1c3VqdVBTRHc3OTdq?=
 =?utf-8?B?U3VOSVoyanhWWjhmbjNMQ3JJMWZCb2laTktzbEJKQTZVZWdJUHdrS2VWd3p2?=
 =?utf-8?B?Mk5vMVprMnh0MW8yajZPY2lISkdLZDExSlV1UmE2eVQzRG1ZMGVyL1hld1JZ?=
 =?utf-8?B?a1VMUDFWcGlWZ2dSbFgxcVRYckdqZ05xRmFkNTFlMi90SjNlYU4yeEx4YnBL?=
 =?utf-8?B?UnV1cFFXMkxKMUlNa0JoTFFrNmlEdCtGS2ZtT3pzbXVCdHVVY1NFR2dvUnZG?=
 =?utf-8?B?T3hOVWgzQ3QrUUZkdnIxQUI3MHIvRHRPNFhXQkhDOUxaSmFxdlYvR281RGdr?=
 =?utf-8?B?MDdGQUgrTEVDOU1Vd213Qi9WcUZ1MVNnSEZraUo0NVdyYkhqVjBBMzNRd1d3?=
 =?utf-8?B?dDBIN2NEOWVvWG1uSEorMkUwZEJwekZnQ3FoSm5XR1NHb1dWVnA1S1c0SWhJ?=
 =?utf-8?B?VmloUXFGR1BmeHJZY2hsc2kzRzYrY0VQZXhNRjZTTktXKzZCVmEvelNERmlW?=
 =?utf-8?B?cUdOdytLNWxIQ09rUFNQRnRYeG9YWmREU3VsNlB4bEsycko5di9JV3hBTmhK?=
 =?utf-8?B?azlVRSs5TkRURDM3Vk96Q1ltVVRiT1IwWHNad0RMOXdyOXM4U2s2TnBnQmpa?=
 =?utf-8?B?Z1d5MWFJZS80YjVNUGZKV0NVc0ZmTVlFVGZhTWVCbkJFUnVHUGtLYVNMd1VV?=
 =?utf-8?B?YXFwSTRTS2x2ZktCVzZsWmNTeXhQVHFQbTZaNmcxZGxEMnlyRDUrdWNrUlhM?=
 =?utf-8?B?Q0tRblVxUk5PRUtmYnIrdkY4TTJtNVF0UjBNVG9XM2RIT0tzVTRZK2dWeWFi?=
 =?utf-8?B?YlpUVmlkcFJWNjdocUxiRjZXYnloVVA3ZUR6K3JKRHRyNWIwb0hZdFpTZVFm?=
 =?utf-8?B?M0VRNUxsLzFKNlBKSHllVTM4dUkzcE52eWM5OFRrQ2x6dTFqMnc0OG9zN29U?=
 =?utf-8?B?Zk1FbzZQV1pTU0RCdnF5bkQ4U1pVZlYvZjdBKzJwU09nRUJrb01yUW1ySUdo?=
 =?utf-8?B?bkEyaG1LYmV2dlcrR0h1SUI1WW5aK29EWHArdGx0ZU1iTndDYnRNOUFsWWhB?=
 =?utf-8?B?VURISjE4d1BqMjlMTzRHcVQwNE1QWU84NnVjbE8zWHoxRTgrS3F1Q3pjcEpm?=
 =?utf-8?B?ZzlybDFyY0VQbVBmeWsvbk5JTHlqTEc5RElBSFZ5RVoyb0NENHZCSzRqWnBh?=
 =?utf-8?B?cU9ubWZQWWp0MnFlellFNGVVdC9BaXNRckhQeCtCeCt4ZWEzZmNpRTR0a1h3?=
 =?utf-8?B?TTlzeGVML2JoSFROSlRiU2NGMXp3OFErRVNLMVptSzFuTm8vUEkrMFR5bTdG?=
 =?utf-8?B?UDloMzdLVytMRXQ3aUYxVUxtYnR5bTlQWnRVcEErWXVmS3hxTENhSXJmbktF?=
 =?utf-8?B?UkltSnJvR2VqWVoxL0lqT21tTGVmTkVLMlpqS1Z5ZXRmVGdvU3grYUdkQnZR?=
 =?utf-8?B?MUVtdUFzcFNYdFhZd1psT2k5Qy9EMGcvZmJCcG8xQTRyak9Yb0RBL2FUZTl3?=
 =?utf-8?B?b2d2MkNKL3Z2dC9ZdEI4eDZFTkhKOVFhekg3dEsvYmVGOEJGUmIwVlVsMzlM?=
 =?utf-8?B?M3c9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 93658272-b172-4903-64d3-08dc7b6acc3c
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 20:56:20.4110
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: G9xkvoPCG13gyRE4qYTftHP8ixWkHznyryZAndt+N/LxpEr/WavRlbunmAV+HMP0+QXWJ88D8DTrfbtJSZM1n7Rh0KyPz93FVnxOTMqEPXY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR11MB7696
X-OriginatorOrg: intel.com

Hi Tony,

On 5/23/24 12:04 PM, Luck, Tony wrote:
>>>      return (is_mbm_local_enabled() &&
>>> -           r->alloc_capable && is_mba_linear());
>>> +           r->alloc_capable && is_mba_linear() &&
>>> +           snc_nodes_per_l3_cache == 1);
>>>   }
>>>
>>>   /*
>>
>> Since the software controller is a filesystem feature the above
>> now requires that snc_nodes_per_l3_cache becomes part of the resctrl
>> filesystem code and every architecture will need to set snc_nodes_per_l3_cache.
>> Every architecture will thus need to interpret what "SNC" means for them
>> using the term introduced here. That may be ok ... but the term "SNC"
>> will then surely not identify an Intel feature and Intel needs to be ok
>> that any architecture calls their "similar to SNC but not quite identical"
>> "SNC".
>>
>> I assume now that as part of the fs/arch split there needs to be
>> a new helper that allows different architectures to set this
>> filesystem variable?
> 
> I can change this check to better reflect the underlying reason to
> disable the software controller. Which is that the MBM monitor scope
> does not match the MBA control scope. This seems like an architecture
> neutral expression.
> 
> So code would look like this:
> 
> 	struct rdt_resource *rmbm = &rdt_resources_all[RDT_RESOURCE_L3].r_rescrl;
> 	struct rdt_resource *rmba = &rdt_resources_all[RDT_RESOURCE_MBA].r_rescrl;
> 
> 	...
> 
> 	 return (is_mbm_local_enabled() &&
>                  r->alloc_capable && is_mba_linear() &&
>                  rmbm->mon_scope == rmba->ctrl_scope);
> 
> I'm also contemplating dropping snc_nodes_per_l3_cache from being a
> global variable and making it a field in "struct rdt_resource" (only needed
> for the RDT_RESOURCE_L3 resource). N.B. Babu had suggested it
> shouldn't be global many patch versions ago.
> 
> Perhaps name it .domains_per_l3_cache or .subdomains_per_l3_cache?
> 
> Bad idea? Good idea (but you have a better name for the field)?

With the check in supports_mba_mbps() changed I do not see
snc_nodes_per_l3_cache needed by anything but arch specific code.
I thus do not see any reason for the resctrl filesystem (or, for
example, Arm) to know that this value even exists.

While struct rdt_hw_resource is a place to put architecture
specific information it does not seem appropriate to force every
resource to carry what is essentially a system wide (not specific to
resctrl) L3 specific property. To me this really seems like an
architecture specific global setting but I'd also like to hear the
motivations why it should not be.

Reinette