Received-SPF: pass (google.com: domain of linux-kernel+bounces-136192-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <9d339257-97c3-4d9f-aa02-c43713fd33df@intel.com>
Date: Mon, 8 Apr 2024 20:19:40 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v1 13/31] x86/resctrl: Stop using the
 for_each_*_rdt_resource() walkers
To: James Morse <james.morse@arm.com>, <x86@kernel.org>,
	<linux-kernel@vger.kernel.org>
CC: Fenghua Yu <fenghua.yu@intel.com>, Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>, H Peter Anvin
	<hpa@zytor.com>, Babu Moger <Babu.Moger@amd.com>,
	<shameerali.kolothum.thodi@huawei.com>, D Scott Phillips OS
	<scott@os.amperecomputing.com>, <carl@os.amperecomputing.com>,
	<lcherian@marvell.com>, <bobo.shaobowang@huawei.com>,
	<tan.shaopeng@fujitsu.com>, <baolin.wang@linux.alibaba.com>, Jamie Iles
	<quic_jiles@quicinc.com>, Xin Hao <xhao@linux.alibaba.com>,
	<peternewman@google.com>, <dfustini@baylibre.com>, <amitsinght@marvell.com>,
	David Hildenbrand <david@redhat.com>, Rex Nie <rex.nie@jaguarmicro.com>, Dave
 Martin <dave.martin@arm.com>
References: <20240321165106.31602-1-james.morse@arm.com>
 <20240321165106.31602-14-james.morse@arm.com>
Content-Language: en-US
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <20240321165106.31602-14-james.morse@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TVl2a2tzZ0lwVThGZmMzVW1adHZjazRKbW82c1crbkZ2akNBaHhTNjFrR0J1?=
 =?utf-8?B?VUNLMkhCNjFWOW51c3ZUTUVEOEI0UnlreUNnV1pvMER5SkdqcFB0TnlZbjQv?=
 =?utf-8?B?bzdHTTNuNHlCNVJUSm9pNDRycURINkZlb3NCTlk0Q2IxaGJ6NmNhaUpwVHk0?=
 =?utf-8?B?NXBNYXZxRjVYbG13eEw5dytqRmlnZmk2K01wbXJQbk5OcHNPTWtXeWd3OEEr?=
 =?utf-8?B?Vk9ISVJhZWZSMWl3UU14RzRiaHVZSTMycVFaQ3YrNEhKL1VnM1pwaDlJZmhS?=
 =?utf-8?B?TUVKajRqVXFRd0p2aG5aU0ZmelhqL2JTWjNXUnVpVzc1ZWhheHNvMVBhWURQ?=
 =?utf-8?B?d1BuaGx3VkVVVHpwVXJvOXBJL2hOcWg4NDZpeVRSTHdScG5CQkRjTGYrSjRo?=
 =?utf-8?B?b2NPeWNleEZpVmozYng0NHRkdStITjZSakxpd2RCVWx0MGpaMElMWDBWZEI5?=
 =?utf-8?B?OUxLSkJpSUFua25QM24xVjREcXdmamoyd2RsM0ZYSTVEUDNmbnlkNTdHeG13?=
 =?utf-8?B?TEI4QzN6b3hXa1VLN0hMQ0Y2YWxyMldNbzB3Sm1uUGQvcHYrM2lITFl1SFh6?=
 =?utf-8?B?L2JLdUp2bXBZN3VPb3ZyTVdnSjR0Si8wYkVOSkd2T2tWNkFIY2pSZzBaSEJF?=
 =?utf-8?B?aUhERk5OV2ZDTkNIaURtYzdJbndMV2owd0h0dDVaakRtVys0NHYxV3RrRlpQ?=
 =?utf-8?B?bnNqSGpYUFc4UTEvL2w2WFRnK0tJK3FGa3krRjZkbnZicnRVY3hzUk1oMXQv?=
 =?utf-8?B?bkQ5VDhKS3N6SWtnQWlUd24yTzIxdEpVN3ZhMm15dVZSOUhuYis5ZDVUcTlx?=
 =?utf-8?B?VjJkRWVUOENuR2NIK3B1dEJ4UDlXb1lnakVNMFFQMGJBTmdCNFBSVFhCODVB?=
 =?utf-8?B?Z0liTFBKSG5GRTlYYmtJVmdoOVBiaVdjMHBobEJ4Wlk2Y3pnL3ozR1p4N2l3?=
 =?utf-8?B?YXFLaENhNktPMzhQa2ZOUGltMjZjMDYrRzA4VksvZXZrY3R4elkzYmo5MmhV?=
 =?utf-8?B?V2dRQ1pGcVZlLzI4dXBwWjc3SFFtVWU1aEx0a2MxMFhSbDNnS21DQUJCOU9L?=
 =?utf-8?B?MGZ1U1hCSEh1RWMrU2JiTVFiSXlDaVFJdDE4MXRmVFBadDJtaDlBclA4QzBN?=
 =?utf-8?B?SW5ucjFSbnB5SHc3YmlFbFhjUmxocWRvV29TZDJabk80QmdmeU1BeFFyUlNm?=
 =?utf-8?B?MFVtQ2JicEFoN1NHM05jck1zVDBmUXR2eUVHbFROQjNGamk2bWxkSXhKdUtR?=
 =?utf-8?B?Vkh5Nnl6RXZJSjYySFhXWDg1aGlQUkNXU0dIS1QwMzV4V3REbThKcWN6dUJp?=
 =?utf-8?B?NWRNakN5Ti92LzlReDRXYy9ESU45bXZLYlY2SEdtRjlvWC9Ga0ZkNk5JYnB4?=
 =?utf-8?B?Y1EzdTFnSWRHeDhUa3Y1OHpEUFF5bDBzUHVScVFXRjZadzdmRnJSSWZ5NW95?=
 =?utf-8?B?eUdETWx5ZFByTEpudVpBTVZXRlRhTjlHRThTdExGYlp4NWY5a1dWUEszSS9E?=
 =?utf-8?B?VkF3cTQwbGdtQWVCVXpVbGpqcUNXZW5Ea1BaelBPd2RwaHpzVUZLTU51ZEky?=
 =?utf-8?B?VWhHYUVJSm1UQ1dXWmtEODd1OTZMSG1kbEpPcDZ2MVU1Rjgzempnc0RTTFlU?=
 =?utf-8?B?cUM4R0NFL1UzRk9yZHNxQVN5RkFSOXl1YTJQOC9pd1BRS3AxdHo1bTRULytn?=
 =?utf-8?B?OHRXVmsxcENqSzR2czJNdi9Cc1ZHZ2o1b2RYR3hDRHZza1lHbjZ3c21FZ1VG?=
 =?utf-8?B?ZE0vdzNtSjdYY3FGVTAvaDlWUS9lc0picU80NGEwc25qQ1ZzNzdnNHB1ZjVY?=
 =?utf-8?B?U0d5YkJ5SzFQNjlvTTkxRnJ2VFo3UDNIRzNUOFBwemVZWG1VWmppWW10bm5L?=
 =?utf-8?B?U2tYbnh6UTZrNUdJeUd5Nll5Vi9NOCtkZW1OaytzZUFOU0tvdTBGQTJOaEQ2?=
 =?utf-8?B?VVNxcCtiWXE3TmFYVm1CUlhpOFU4UkNXeGM1ODNRZk9lN0p2NjBJLzZvNXVv?=
 =?utf-8?B?RXdVRGljczlIQjVvYWZnTzAzc0Yxc010SWo3aERjbnRpNkhJcGFZQktsMVZE?=
 =?utf-8?B?QUxYNlc4WWpjek03WWk2WGp1WDlsb2ZzZVZKeDVuOFo2T3dHV1Y0SGdGbENh?=
 =?utf-8?B?cTVnZTE1NExKMFhYTUticXFJZzhCS3BlZ0FKRDZqQXBlWElUWGttYzRaK01o?=
 =?utf-8?B?Ymc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: a4d8ff20-0b5c-450c-00e9-08dc5843e675
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Apr 2024 03:19:43.3434
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9/OvORPlmlXnUsg8jPDVKqath0wHGXjdwqhnaNR2KeVjN2b/EXFMsZu7fE6tkCEX4l0P+4yH+eRpz3zw08uCL5dpEeJCKuHfplDKqBdPhiQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6357
X-OriginatorOrg: intel.com

Hi James,

On 3/21/2024 9:50 AM, James Morse wrote:
> The for_each_*_rdt_resource() helpers walk the architectures array

architecture's ?

> of structures, using the resctrl visible part as an iterator. These
> became over-complex when the structures were split into a
> filesystem and architecture-specific struct. This approach avoided
> the need to touch every call site.
> 
> Once the filesystem parts of resctrl are moved to /fs/, both the
> architecture's resource array, and the definition of those structures
> is no longer accessible. To support resctrl, each architecture would
> have to provide equally complex macros.
> 
> Change the resctrl code that uses these to walk through the resource_level
> enum and check the mon/alloc capable flags instead. Instances in core.c,
> and resctrl_arch_reset_resources() remain part of x86's architecture
> specific code.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/x86/kernel/cpu/resctrl/pseudo_lock.c |  7 +++++-
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 30 +++++++++++++++++++----
>  2 files changed, 31 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
> index 884b88e25141..f2315a50ea4f 100644
> --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
> +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
> @@ -840,6 +840,7 @@ bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_domain *d, unsigned long cbm
>  bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
>  {
>  	cpumask_var_t cpu_with_psl;
> +	enum resctrl_res_level i;
>  	struct rdt_resource *r;
>  	struct rdt_domain *d_i;
>  	bool ret = false;
> @@ -854,7 +855,11 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)
>  	 * First determine which cpus have pseudo-locked regions
>  	 * associated with them.
>  	 */
> -	for_each_alloc_capable_rdt_resource(r) {
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->alloc_capable)
> +			continue;
> +
>  		list_for_each_entry(d_i, &r->domains, list) {
>  			if (d_i->plr)
>  				cpumask_or(cpu_with_psl, cpu_with_psl,
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index e736e4d20f63..3f16e7854411 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -98,12 +98,17 @@ void rdt_last_cmd_printf(const char *fmt, ...)
>  
>  void rdt_staged_configs_clear(void)
>  {
> +	enum resctrl_res_level i;
>  	struct rdt_resource *r;
>  	struct rdt_domain *dom;
>  
>  	lockdep_assert_held(&rdtgroup_mutex);
>  
> -	for_each_alloc_capable_rdt_resource(r) {
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->alloc_capable)
> +			continue;
> +
>  		list_for_each_entry(dom, &r->domains, list)
>  			memset(dom->staged_config, 0, sizeof(dom->staged_config));
>  	}
> @@ -2181,6 +2186,7 @@ static int rdtgroup_mkdir_info_resdir(void *priv, char *name,
>  
>  static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
>  {
> +	enum resctrl_res_level i;
>  	struct resctrl_schema *s;
>  	struct rdt_resource *r;
>  	unsigned long fflags;
> @@ -2205,8 +2211,12 @@ static int rdtgroup_create_info_dir(struct kernfs_node *parent_kn)
>  			goto out_destroy;
>  	}
>  
> -	for_each_mon_capable_rdt_resource(r) {
> -		fflags = r->fflags | RFTYPE_MON_INFO;
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->mon_capable)
> +			continue;
> +
> +		fflags =  r->fflags | RFTYPE_MON_INFO;

Please check spacing.

>  		sprintf(name, "%s_MON", r->name);
>  		ret = rdtgroup_mkdir_info_resdir(r, name, fflags);
>  		if (ret)
> @@ -2615,10 +2625,15 @@ static int schemata_list_add(struct rdt_resource *r, enum resctrl_conf_type type
>  
>  static int schemata_list_create(void)
>  {
> +	enum resctrl_res_level i;
>  	struct rdt_resource *r;
>  	int ret = 0;
>  
> -	for_each_alloc_capable_rdt_resource(r) {
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->alloc_capable)
> +			continue;
> +
>  		if (resctrl_arch_get_cdp_enabled(r->rid)) {
>  			ret = schemata_list_add(r, CDP_CODE);
>  			if (ret)
> @@ -3166,6 +3181,7 @@ static int mkdir_mondata_all(struct kernfs_node *parent_kn,
>  			     struct rdtgroup *prgrp,
>  			     struct kernfs_node **dest_kn)
>  {
> +	enum resctrl_res_level i;
>  	struct rdt_resource *r;
>  	struct kernfs_node *kn;
>  	int ret;
> @@ -3184,7 +3200,11 @@ static int mkdir_mondata_all(struct kernfs_node *parent_kn,
>  	 * Create the subdirectories for each domain. Note that all events
>  	 * in a domain like L3 are grouped into a resource whose domain is L3
>  	 */
> -	for_each_mon_capable_rdt_resource(r) {
> +	for (i = 0; i < RDT_NUM_RESOURCES; i++) {
> +		r = resctrl_arch_get_resource(i);
> +		if (!r->mon_capable)
> +			continue;
> +
>  		ret = mkdir_mondata_subdir_alldom(kn, r, prgrp);
>  		if (ret)
>  			goto out_destroy;

Reinette