Received-SPF: pass (google.com: domain of linux-kernel+bounces-186730-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <7f72f98e-65ee-466d-a931-35bd744296f2@intel.com>
Date: Wed, 22 May 2024 14:20:40 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v18 13/17] x86/resctrl: Handle removing directories in
 Sub-NUMA (SNC) mode
To: Tony Luck <tony.luck@intel.com>, Fenghua Yu <fenghua.yu@intel.com>, Maciej
 Wieczor-Retman <maciej.wieczor-retman@intel.com>, Peter Newman
	<peternewman@google.com>, James Morse <james.morse@arm.com>, Babu Moger
	<babu.moger@amd.com>, Drew Fustini <dfustini@baylibre.com>, Dave Martin
	<Dave.Martin@arm.com>
CC: <x86@kernel.org>, <linux-kernel@vger.kernel.org>,
	<patches@lists.linux.dev>
References: <20240515222326.74166-1-tony.luck@intel.com>
 <20240515222326.74166-14-tony.luck@intel.com>
Content-Language: en-US
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <20240515222326.74166-14-tony.luck@intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TWlQSDZHMUNaOEpFUGVnQlRwRkp6cEJudTk0VG9mWngxTm9IWHVXaVppOEpn?=
 =?utf-8?B?djdrYjhoandHVVdITSsyQStiQ2s4eXZ6UElmZnZVQWNEcW1GeVc3MVlyWnBp?=
 =?utf-8?B?T3ZBYkJlR1JMVm1IM1ljd2lQWmh4SVFWdFJNOHhjQTB5bzM5ZTk0UjFCdXdx?=
 =?utf-8?B?SGZEbHhzY1FkTXJIaU01dWVnZ2M3Uk13T0hWcm9IdUZhVXk4SGRmQkUyM1NM?=
 =?utf-8?B?RWlLMFdoUGxVcXpQOUVnNHRHUm9Xd0F6Z1FsOTJQNjJxa3dxa3BSS01wRnlV?=
 =?utf-8?B?QXhtdDZkOGdCYkt4ZmJCcWJ3OWlMN2NmNGhKSUZoeVI1TC96VmtXbnVEaDRj?=
 =?utf-8?B?YjU4TGJCSkxmVmsyTHpaYVRhN1RqdVhkc2tYWmlDUFBFVGdyQStHNWRES2VF?=
 =?utf-8?B?RWkzTnpGdm0wL3RlZ2ZlbHRacmNLUWt3UWxNSTl1QWMrbGZoVHlEWjJCbUxt?=
 =?utf-8?B?eTBBYmNQODlYZFpNQnJsK3ZaUnh3YzVVRWp2ZFNPS0xSWHM2OGl5dWhZRUFJ?=
 =?utf-8?B?bUVlenZVa3VCZ29aMGF6RmtXNGNjQ1NCVFRrU2VVcUp5cUJ3Nkt3OEl3OExo?=
 =?utf-8?B?bzFXMDZyNmpOM0pSWU9ieVRZa1hyeGxvdzBGa3BFb3RWNStIeDFZWWZDLy8x?=
 =?utf-8?B?OEIrY1hvUEZYUXdTU2NES0hoRWs1enhNbFk1NzVoaUhLczZKOFRiQ09ET0xU?=
 =?utf-8?B?SXg5dnBBQ083dGVjTDU5Z0xaR3FrVmtjQ2x6ZnRJL3kvVXhvdXpCcTNWWnJi?=
 =?utf-8?B?Nm82cjFHTUJmZzdWOXFaVVVrcklPZ044RXBzbmFHM3ZWckg2V25RUThLSEZ5?=
 =?utf-8?B?UmlnNjQranQybE5FNTEwcXptalJvR05rcm1sN3pleHluYi9WejZMYW9YNHdo?=
 =?utf-8?B?WW8wcGs4THdEbjFyOXJ4WVBuZUJaSktWZHlqV1A3T3ZPSXM3amxHN3htYlE4?=
 =?utf-8?B?WHFFL0g2SzFDcitiVWRCVStBVVRuNjV4YjYzODB2Z09rMU1ncXkrQkUzaEhO?=
 =?utf-8?B?eGlxQWl1NTRLckFxRHJqeUtVZ2Rla3pDQzFRZFhRZHJMUHZ5TzBjOFBVcEc5?=
 =?utf-8?B?TkxqcGZsVHByQWhESW5BbzBlS1hFK0hNenU0eUk4Z28yOTU3YnlHUEtTQXph?=
 =?utf-8?B?QXRZVDZtbkNRZVlHWE1Hc21PVld1emt1K25SdzZZTzhTUDVxMm5PbytFZG9F?=
 =?utf-8?B?cStHS3NHRmRZQVlkblF2d1cwZmJSNEYrUWhCTGxubHYzc0FubW85ZXJJd2xh?=
 =?utf-8?B?aVo0WGNYZnRsL004UVZUNGtydGxjZkdtZFpzTVpnTS9nb3NiL3N3Ni9Rb2Fy?=
 =?utf-8?B?aHpMbHNDaFgra0hqd2tja2FZRXFSaTJjYTdPd1hXNStkYlZFVVJ1ZEs3SHRC?=
 =?utf-8?B?N2RNRUFSSTMxVi9pZ0JpWU03eTJVVi8zNFM5VEtSZnFBZTNzS0VFaHp1bSt2?=
 =?utf-8?B?bTlKUVhUSy9QZlRJa2dKQnc4M203bTVXaEhBdmdOV1pKblFWNEcxRGNXRzBH?=
 =?utf-8?B?VnhaaENNaWQyMnppeC8wMzdnRm44U2NlZ2swclByQlZQYmgrNVZVVXBXNU1U?=
 =?utf-8?B?UFhOTnFnS0hBM0MwbTFWSzlyMUNNQ2gwVzdLOWV1T3BBT21OTnNqOU5IWmRZ?=
 =?utf-8?B?VVFieUxRc0hNNEFESC9SQnppejEzSlZsVXVFRklFN1dLU3lBeEZaK3BEZU9y?=
 =?utf-8?B?clBOdml4QlRQZjhrY0hCYUM5QVJmd3l3Wm9hYkxOSXl3S2RkdmtISWNqK0cw?=
 =?utf-8?B?bnQySjlxTkpNT1BIUWhKMjZMOUdrSklGbE42c1l6N29pdlpzVVpvczJXOURt?=
 =?utf-8?B?RDZSMnUwejNPdjJYMS9ycGZodVBBd2l4RWhMVEU3N1p0Nm1lWHpicjZJdnlB?=
 =?utf-8?B?aEs1UC9tK1QrRG8vTy9LNk9vNDJGVjNheDBuaWxsY2RGU1NSQndpbVB5ZzFU?=
 =?utf-8?B?SWpVVEkrdzNRM25rUzFFazBJL1BEQkk2RkR3T2Z3bFpPVy9iV3RDb2pCaFBV?=
 =?utf-8?B?T0k2SkhLdmZJZGZkdDRnTGkxRVZsRjdpV01GUjVSTXNsM0Z6NmJ1eXNMcFND?=
 =?utf-8?B?d29LNFZRUXpDSVZsTFh4OUYzQnZEK1N5Rk9Wby9MbEtTSUNVVXRTWVpodzd0?=
 =?utf-8?B?Q0NUNjJhVUdlbEdleWJEYUN3TTlNeG5ZSWIrTHhUNTRMMUVPWE5NbzhDWnlw?=
 =?utf-8?B?VVE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: cc8cdd93-b9dc-4e18-860b-08dc7aa50927
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 21:20:42.2858
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4K/XCarTjaU7i8tvJ49trH6LfoZEQBRPhzLa6ux5CS20nlzwv4BXTFvMoRSjCyLjXsl+c5eAQtTApRDuYblziXOdmGMw28c7b12ouKH55XA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB7422
X-OriginatorOrg: intel.com

Hi Tony,

On 5/15/2024 3:23 PM, Tony Luck wrote:
> In SNC mode there are multiple subdirectories in each L3 level monitor
> directory (one for each SNC node). If all the CPUs in an SNC node are
> taken offline, then just that SNC node directory must be removed. In

imperative tone needed

> non-SNC mode, or when the last SNC node directory is removed, also
> remove the L3 monitor directory.

There is a disconnect between changelog and code. The code tries to be
generic while the changelog is as specific to SNC as possible. This makes
it hard to go from changelog (ignoring that changelog does not follow x86
customs to begin with) to patch.

> 
> Signed-off-by: Tony Luck <tony.luck@intel.com>
> ---
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 43 +++++++++++++++++++++-----
>  1 file changed, 36 insertions(+), 7 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index f0f468babdea..cac32ddd3afd 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -3011,17 +3011,46 @@ static int mon_addfile(struct kernfs_node *parent_kn, const char *name,
>   * and monitor groups with given domain id.
>   */
>  static void rmdir_mondata_subdir_allrdtgrp(struct rdt_resource *r,
> -					   unsigned int dom_id)
> +					   struct rdt_mon_domain *d)
>  {
>  	struct rdtgroup *prgrp, *crgrp;
> +	struct rdt_mon_domain *dom;
> +	bool remove_all = true;
> +	struct kernfs_node *kn;
> +	char subname[32];
>  	char name[32];
>  
> +	sprintf(name, "mon_%s_%02d", r->name, d->display_id);
> +	if (r->mon_scope != r->mon_display_scope) {
> +		int count = 0;
> +
> +		list_for_each_entry(dom, &r->mon_domains, hdr.list)
> +			if (d->display_id == dom->display_id)
> +				count++;
> +		if (count > 1) {
> +			remove_all = false;
> +			sprintf(subname, "mon_sub_%s_%02d", r->name, d->hdr.id);
> +		}
> +	}

This continues to look suspect to me. When I took a closer look during previous
version I thought this information can only be accessed via inode. Seeing this
code again made me look more closely and it seems there is no problem to just
query how many subdirectories a directory has. See for example, kobject_has_children().
Doing something like that seems more intuitive than this quirky way to set and
use a flag.

> +
>  	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
> -		sprintf(name, "mon_%s_%02d", r->name, dom_id);
> -		kernfs_remove_by_name(prgrp->mon.mon_data_kn, name);
> +		if (remove_all) {
> +			kernfs_remove_by_name(prgrp->mon.mon_data_kn, name);
> +		} else {
> +			kn = kernfs_find_and_get_ns(prgrp->mon.mon_data_kn, name, NULL);

kernfs_find_and_get()?

> +			if (kn)
> +				kernfs_remove_by_name(kn, subname);
> +		}
>  
> -		list_for_each_entry(crgrp, &prgrp->mon.crdtgrp_list, mon.crdtgrp_list)
> -			kernfs_remove_by_name(crgrp->mon.mon_data_kn, name);
> +		list_for_each_entry(crgrp, &prgrp->mon.crdtgrp_list, mon.crdtgrp_list) {
> +			if (remove_all) {
> +				kernfs_remove_by_name(crgrp->mon.mon_data_kn, name);
> +			} else {
> +				kn = kernfs_find_and_get_ns(prgrp->mon.mon_data_kn, name, NULL);

copy&paste ?

> +				if (kn)
> +					kernfs_remove_by_name(kn, subname);
> +			}
> +		}
>  	}
>  }
>  
> @@ -3111,8 +3140,8 @@ static int mkdir_mondata_subdir(struct kernfs_node *parent_kn,
>  static void mkdir_mondata_subdir_allrdtgrp(struct rdt_resource *r,
>  					   struct rdt_mon_domain *d)
>  {
> -	struct kernfs_node *parent_kn;
>  	struct rdtgroup *prgrp, *crgrp;
> +	struct kernfs_node *parent_kn;
>  	struct list_head *head;
>  

Stray snippet?

>  	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
> @@ -3984,7 +4013,7 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_mon_domain *d
>  	 * per domain monitor data directories.
>  	 */
>  	if (resctrl_mounted && resctrl_arch_mon_capable())
> -		rmdir_mondata_subdir_allrdtgrp(r, d->hdr.id);
> +		rmdir_mondata_subdir_allrdtgrp(r, d);
>  
>  	if (is_mbm_enabled())
>  		cancel_delayed_work(&d->mbm_over);

Reinette