Received-SPF: pass (google.com: domain of linux-kernel+bounces-214197-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <b722c5da-812c-4ccb-ae7b-e7bc9c9de50f@intel.com>
Date: Thu, 13 Jun 2024 18:50:39 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 16/19] x86/resctrl: Enable ABMC by default on resctrl
 mount
To: Babu Moger <babu.moger@amd.com>, <corbet@lwn.net>, <fenghua.yu@intel.com>,
	<tglx@linutronix.de>, <mingo@redhat.com>, <bp@alien8.de>,
	<dave.hansen@linux.intel.com>
CC: <x86@kernel.org>, <hpa@zytor.com>, <paulmck@kernel.org>,
	<rdunlap@infradead.org>, <tj@kernel.org>, <peterz@infradead.org>,
	<yanjiewtw@gmail.com>, <kim.phillips@amd.com>, <lukas.bulwahn@gmail.com>,
	<seanjc@google.com>, <jmattson@google.com>, <leitao@debian.org>,
	<jpoimboe@kernel.org>, <rick.p.edgecombe@intel.com>,
	<kirill.shutemov@linux.intel.com>, <jithu.joseph@intel.com>,
	<kai.huang@intel.com>, <kan.liang@linux.intel.com>,
	<daniel.sneddon@linux.intel.com>, <pbonzini@redhat.com>,
	<sandipan.das@amd.com>, <ilpo.jarvinen@linux.intel.com>,
	<peternewman@google.com>, <maciej.wieczor-retman@intel.com>,
	<linux-doc@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<eranian@google.com>, <james.morse@arm.com>
References: <cover.1716552602.git.babu.moger@amd.com>
 <74f014062885d707701873527c1b12268603a525.1716552602.git.babu.moger@amd.com>
Content-Language: en-US
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <74f014062885d707701873527c1b12268603a525.1716552602.git.babu.moger@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cU9mR1RVeUcrRHJ3OG9samdLb25TT1RRS0ltYUJob2hIeHpsNHBmUzN5ekVL?=
 =?utf-8?B?eEI1aHpGWUhuNjhrWHFNd2lQaElVY2lvN2xLb1I4RVJBQlhwYUJWMW5BZUdZ?=
 =?utf-8?B?S0oydmRURXJmYm1QeVplSUF0U054RlptbDFPQ3JDSklCVGhZVStGUVByMUxJ?=
 =?utf-8?B?NVp1T1RkT2IrcE9xbnpPVDIzVE9VSWFlUkJJZGozTGpENWpxdThMQ3J0dEEv?=
 =?utf-8?B?Wnd6aGorekZQblRXeWpIYVMzTE02VFBhRU84VzhuaFlxS2sxY3p4MU4vMUR1?=
 =?utf-8?B?VXYya1E3eURrTVBvT29jT3M5ajV6VXQvVGlhSTRURFVXNm01WmFISTYvNXc2?=
 =?utf-8?B?ZlRBTmxvRHUxZE9rcUZUK1pCOG1NczlhMTNXRUlGTklqU2k0R3NPc1pXNmZw?=
 =?utf-8?B?c09XZ3dKMUM4c3Eycng2NGtpS1dxL1VLdW1Idjc1OWFUdkJJOWViVXYyNm5T?=
 =?utf-8?B?cEZUWjIyT3JodUM1cUdLQU8wbGhndmk0bXA5ZmtxZnhQZ0RoNGo5V05sYlhv?=
 =?utf-8?B?NWsyTUlJRlJacXdxQUlFTW9PRnFPc0ppTm1lKzlMRHVFOXRYSEZLcUZEb0tM?=
 =?utf-8?B?QW1zbDBrbW0rbyttV2czdGNZTlQ4SGlZYmxhck9BMSs0UjhWeHNlQ3IzQXdS?=
 =?utf-8?B?MUpySWNBMVhlQlRJOFB5N0VuaXB3b0RRRk5pY2IvaVN1Sng2S3NGbEV6WW9H?=
 =?utf-8?B?aTBNZVBLcE03ekl5L0IvdmFZdGFZbjdjdm5rOFRIQmlzeGYrYVFtclhTN002?=
 =?utf-8?B?SklzKzRodERSRWlpODNEUDhoREJtanl6VHBrMDM3a1RFTHplK2NTS0hzM0VN?=
 =?utf-8?B?TkM2SGdpUFRTT2N4OEdQU3VwT2szK3lGak9YNkxudmxMWTcwWEluL3cxSnV1?=
 =?utf-8?B?c3owTndVcTN4MHNiYzV4WHdtNXk1R0xtRDQyZlBGZjZER0IzcFhkcUNSREJH?=
 =?utf-8?B?eHlsNWpOTDEzNTdzV2hzdXVTTTY2ZGJyTGdCMFFpU3ZwRHRjSjlxR2ljaVFk?=
 =?utf-8?B?NHp2d0dlREhOb2VMSVFnRFdkSkFPSVRNK2hoaURUL2VZMzc5VkpSOW5aTVh4?=
 =?utf-8?B?VFRteWpqMm5IOTJEaEIxUk9kRVBmcEhVYkJ0d28rSHNvUm91NGJvREY5MENJ?=
 =?utf-8?B?SnpLSm1ScGc4NkR6WXR2TXdJOWhPeDNjTERUN01ibXZRWTVSWmhxSWNFejNl?=
 =?utf-8?B?clFpRUhpd0w3VG1vL2V6SXpTelE3Rk9pak1NYlp3aGovUnpXZ2pEM0NrbURU?=
 =?utf-8?B?b0RpWk92MTg4ZmEweThPenVCdy80akZ2emRtZU42Qm5HOTFKYitXTlNzZXpp?=
 =?utf-8?B?c2ppK3dmbUwyR0N5anNQdldpVERNbGNrSlVxNXc4aXlZMmJNR3p6dVNpSWls?=
 =?utf-8?B?a3AxUTl1TUpGS0MvazFsbWRpNnY1VzZ2ak9vK3l3RElLazk2M21rN2ljeVdj?=
 =?utf-8?B?NEUxNVRJckhHZ1BnY24vZFVBd1R4dHdLTnFGb3RPaEVLdDRmYUluKzkvSWcz?=
 =?utf-8?B?NllsSlpDN1NlWWJtSFNGNG50YTJhSDR3OTRxK3hDZVp1YytKNnl4MGl4endp?=
 =?utf-8?B?b3lCYzA2TGpTMHk3SGJHL1NoVDE5VTJsT3ZWanYycU9QUU4xdGRGQ2tWdDJO?=
 =?utf-8?B?UWRsKzZvOFJSUjV2TzUvK1pXcUZlb2ZQa0N4Sm9SVExCcGFCbHFpRWppV2Yz?=
 =?utf-8?B?TEFsaFFvV3BFNlZWNGc2UnBhVnVVa1FDTW5OdU15ZFhxRmJ1YzM5MFgzV2hP?=
 =?utf-8?B?aWtYQmM4QXgxM2tXbXZxZnVHVmtPeDNlVSt2bHU2dGlHMnpZTUJma3U4NnRj?=
 =?utf-8?B?bE81QmZmbk9GTmVmdkMxZVFwcWs1eUNCa2ZCVmhTNDMzVGZKTkIrQTRURGYz?=
 =?utf-8?B?RHhoaGZ5T3RRVDQySUVhakxWNmQ2bFB2ZVdXcUQxRzUya1puT2ZWUy9ERVBS?=
 =?utf-8?B?c3MzeW1jcER0UFl5QUtWUFJpR1YzdEhNUnFFRzBHbVNtaUl3RGVCSXVveW8v?=
 =?utf-8?B?ZmJjOHJJU0RNY3dibnppK0Vpd3VIdjA0RUluOXJxZVY2ZnArOTBIS3UrVjRx?=
 =?utf-8?B?Z2crUFR3R1VRQlVvbk9iTmQzWUVHRlVQZWNOeEFPYXRQdHZWRy8xSGs4ZjFW?=
 =?utf-8?B?MWFXZFJwTm84TitZcHhIalMzVjBwamRQc3hsMjBONjVqZFBlSm9wNU5BUHBx?=
 =?utf-8?B?eFE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d2a2a129-9b93-4983-ed4f-08dc8c146948
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2024 01:50:47.4615
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AxcJltPVKbpEfRcOgPh5ZW8ts7LKuIYAE5zZywJaKGQj3nGAE/iEVpqHzVcTTSfCpk2euLb/Vjc9eOxtxpmZ83GYMJDWQuJ3TkOrOeekfgc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR11MB6203
X-OriginatorOrg: intel.com

Hi Babu,

On 5/24/24 5:23 AM, Babu Moger wrote:
> Enable ABMC by default when feature is supported.

Why enable/disable it on every mount/unmount cycle though? Why not just
enable it at the beginning and only change if requested by user?

> 
> Counters are assigned automatically on group creation. If the counters
> are exhausted, report the warnings and continue. It is not required to
> fail group creation for assignment failures. Users will have the option
> to modify the assignments.
> 
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---
> v4: Few name changes based on the upstream discussion.
>      Commit message update.
> 
> v3: This is a new patch. Patch addresses the upstream comment to enable
>      ABMC feature by default if the feature is available.
> ---
>   arch/x86/kernel/cpu/resctrl/rdtgroup.c | 69 ++++++++++++++++++++++++++
>   1 file changed, 69 insertions(+)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index 5ea1e58c7201..f452b6d9bb99 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -2756,6 +2756,7 @@ static void rdt_disable_ctx(void)
>   {
>   	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L3, false);
>   	resctrl_arch_set_cdp_enabled(RDT_RESOURCE_L2, false);
> +	resctrl_abmc_disable(RDT_RESOURCE_L3);
>   	set_mba_sc(false);
>   
>   	resctrl_debug = false;
> @@ -2786,6 +2787,8 @@ static int rdt_enable_ctx(struct rdt_fs_context *ctx)
>   	if (ctx->enable_debug)
>   		resctrl_debug = true;
>   
> +	resctrl_abmc_enable(RDT_RESOURCE_L3);
> +
>   	return 0;
>   
>   out_cdpl3:
> @@ -2882,6 +2885,41 @@ static void schemata_list_destroy(void)
>   	}
>   }
>   
> +static int resctrl_assign_events(struct rdtgroup *rdtgrp)
> +{
> +	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
> +	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
> +	int ret = 0;
> +
> +	if (!hw_res->abmc_enabled)
> +		return 0;

resctrl fs should not be peeking in arch structure

> +
> +	if (is_mbm_total_enabled())
> +		ret = resctrl_grp_assign(rdtgrp, QOS_L3_MBM_TOTAL_EVENT_ID);
> +
> +	if (!ret && is_mbm_local_enabled())
> +		ret = resctrl_grp_assign(rdtgrp, QOS_L3_MBM_LOCAL_EVENT_ID);
> +

This function needs a comment about why it is ok to return with error and
partial changes.

> +	return ret;
> +}
> +
> +static int resctrl_unassign_events(struct rdtgroup *rdtgrp)
> +{
> +	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
> +	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
> +	int ret = 0;
> +
> +	if (!hw_res->abmc_enabled)
> +		return 0;
> +
> +	if (is_mbm_total_enabled())
> +		ret = resctrl_grp_unassign(rdtgrp, QOS_L3_MBM_TOTAL_EVENT_ID);
> +
> +	if (!ret && is_mbm_local_enabled())
> +		ret = resctrl_grp_unassign(rdtgrp, QOS_L3_MBM_LOCAL_EVENT_ID);
> +	return ret;
> +}
> +
>   static int rdt_get_tree(struct fs_context *fc)
>   {
>   	struct rdt_fs_context *ctx = rdt_fc2context(fc);
> @@ -2941,6 +2979,14 @@ static int rdt_get_tree(struct fs_context *fc)
>   		if (ret < 0)
>   			goto out_mongrp;
>   		rdtgroup_default.mon.mon_data_kn = kn_mondata;
> +
> +		/*
> +		 * Assign the monitor counters if it is available. If it fails,
> +		 * report the warnings and continue. It is not nessaccery to
> +		 * fail here.
> +		 */

nessaccery -> necessary

Please elaborate why it is not necessary to fail.


> +		if (resctrl_assign_events(&rdtgroup_default) < 0)
> +			rdt_last_cmd_puts("Monitor assignment failed\n");

How will user know that there is a warning? This does not return an error
so nothing will prompt user to check the status file. Perhaps add a comment
in the docs to help users at least know that this exists. Another helpful
thing will be to have the counter return "Unassigned" if it has not been assigned
since users familiar with old interface may see "Unavailable" without
knowing that user action is required to get a working event.

>   	}
>   
>   	ret = rdt_pseudo_lock_init();
> @@ -3214,6 +3260,8 @@ static void rdt_kill_sb(struct super_block *sb)
>   	cpus_read_lock();
>   	mutex_lock(&rdtgroup_mutex);
>   
> +	resctrl_unassign_events(&rdtgroup_default);
> +
>   	rdt_disable_ctx();
>   
>   	/*Put everything back to default values. */
> @@ -3752,6 +3800,14 @@ static int rdtgroup_mkdir_mon(struct kernfs_node *parent_kn,
>   		goto out_unlock;
>   	}
>   
> +	/*
> +	 * Assign the monitor counters if it is available. If it fails,
> +	 * report the warnings and continue. It is not nessaccery to
> +	 * fail here.
> +	 */

Fix copy&paste here and below.

> +	if (resctrl_assign_events(rdtgrp) < 0)
> +		rdt_last_cmd_puts("Monitor assignment failed\n");
> +
>   	kernfs_activate(rdtgrp->kn);
>   
>   	/*
> @@ -3796,6 +3852,14 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn,
>   	if (ret)
>   		goto out_closid_free;
>   
> +	/*
> +	 * Assign the monitor counters if it is available. If it fails,
> +	 * report the warnings and continue. It is not nessaccery to
> +	 * fail here.
> +	 */
> +	if (resctrl_assign_events(rdtgrp) < 0)
> +		rdt_last_cmd_puts("Monitor assignment failed\n");
> +
>   	kernfs_activate(rdtgrp->kn);
>   
>   	ret = rdtgroup_init_alloc(rdtgrp);
> @@ -3891,6 +3955,9 @@ static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
>   	update_closid_rmid(tmpmask, NULL);
>   
>   	rdtgrp->flags = RDT_DELETED;
> +
> +	resctrl_unassign_events(rdtgrp);
> +
>   	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
>   
>   	/*
> @@ -3937,6 +4004,8 @@ static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)
>   	cpumask_or(tmpmask, tmpmask, &rdtgrp->cpu_mask);
>   	update_closid_rmid(tmpmask, NULL);
>   
> +	resctrl_unassign_events(rdtgrp);
> +
>   	free_rmid(rdtgrp->closid, rdtgrp->mon.rmid);
>   	closid_free(rdtgrp->closid);
>   

Reinette