Received-SPF: pass (google.com: domain of linux-kernel+bounces-214188-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <48e0d78c-5ed8-4b93-8f12-3ce3fd74116c@intel.com>
Date: Thu, 13 Jun 2024 18:43:28 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 11/19] x86/resctrl: Introduce mbm_total_cfg and
 mbm_local_cfg
To: Babu Moger <babu.moger@amd.com>, <corbet@lwn.net>, <fenghua.yu@intel.com>,
	<tglx@linutronix.de>, <mingo@redhat.com>, <bp@alien8.de>,
	<dave.hansen@linux.intel.com>
CC: <x86@kernel.org>, <hpa@zytor.com>, <paulmck@kernel.org>,
	<rdunlap@infradead.org>, <tj@kernel.org>, <peterz@infradead.org>,
	<yanjiewtw@gmail.com>, <kim.phillips@amd.com>, <lukas.bulwahn@gmail.com>,
	<seanjc@google.com>, <jmattson@google.com>, <leitao@debian.org>,
	<jpoimboe@kernel.org>, <rick.p.edgecombe@intel.com>,
	<kirill.shutemov@linux.intel.com>, <jithu.joseph@intel.com>,
	<kai.huang@intel.com>, <kan.liang@linux.intel.com>,
	<daniel.sneddon@linux.intel.com>, <pbonzini@redhat.com>,
	<sandipan.das@amd.com>, <ilpo.jarvinen@linux.intel.com>,
	<peternewman@google.com>, <maciej.wieczor-retman@intel.com>,
	<linux-doc@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<eranian@google.com>, <james.morse@arm.com>
References: <cover.1716552602.git.babu.moger@amd.com>
 <a02dd2b9fa06f360eabe923c5c6d17fa4036aa9a.1716552602.git.babu.moger@amd.com>
Content-Language: en-US
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <a02dd2b9fa06f360eabe923c5c6d17fa4036aa9a.1716552602.git.babu.moger@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?N2p1WFN4bjYxZkRheVc2Nnk1cHMveXJ4UnEvT0h1YnJSYXFPWnpuWktkYW9I?=
 =?utf-8?B?ZlRBVXl1NTZnNWV5Q3V1d2pLQ0JNUWxRNXJRd2g5N01nZGRMTFpBQ1hTczJ5?=
 =?utf-8?B?WWp1SDYvaFY4QUtFSTQ4RFNzUmRUcmhBa09ha09VaTJJZUN0WXAyT085Tzl5?=
 =?utf-8?B?VjhGLzYrMklLU3RuaTJGSkJ6ZzhkSzhQd2llUGNFNnhodFo2dW13aFpoWTRz?=
 =?utf-8?B?UmdFSlhvekJpZFZBZW9Hak81VG4vYU11TDBlbkhzSGtjZjExWHVLOWIxN28y?=
 =?utf-8?B?NUp0bVkwYVI0QnVsTjJZZytHRTA3WlpyYzdiZ3BxWkQwSGZPNVBCZjZuNlhW?=
 =?utf-8?B?SEQ2d3BkL0VLNExoM0p1dHh6b2ZlYWZCcitRYjRJQitHZWtxaVMzY1hZQlcx?=
 =?utf-8?B?VVZDYTNVcFJrZXF5bGtwOThvTEFPZ1ZRLzV1aDhFSDd5dFlnSUx2MzZtZHF6?=
 =?utf-8?B?ZlVRUDVjbklYbERackthaUJjYzJZa0xhbHdDYjQ5SkJlNHNBSDV0MTRSMm9k?=
 =?utf-8?B?VFhwb2hHY2dlZEszYjZvYTB1K01vQzlRWU9XOXA4YzVQWVU3YlRIVmREMk5Z?=
 =?utf-8?B?K1Z2ejkvTjBzTEpkWk9ZbVh6SnFwUkdQZmorS2FNcy9kTHdQOWVNNzZFSkV5?=
 =?utf-8?B?azIrRW1NQ2IvU1ovQnFSc2lwUUgya3JtMXJmUitsZ1QxN1NqMi9KZmJOMlAy?=
 =?utf-8?B?U2UrWHdNLzhSMFFhQkRjQ2RzOXVBUVdvSW0xbEtDdk1sa2VxTWdPeDYzUVhO?=
 =?utf-8?B?dG9wVUwvNnNud3YwQ0huZEN4Z1hMMXAxMnpoTWh2ckFSNDZnV08waWRWOWdY?=
 =?utf-8?B?MlZqL3BVUkx1UkpWbG9yVnZiUG5hdjdYaWw5NVdhbnd4VTJHNlpuOUYvTTRO?=
 =?utf-8?B?T1VPTWdHbWREcjdScGtNU1U1cGhHbTA1SERUOCtLTGZzYm43MVpRNTR6eW5O?=
 =?utf-8?B?czNxTlJQUlhrS3pUUThiQk83TnFSZ2h0R1BXdEJKL2hWcE1QUXFxY2syUzkx?=
 =?utf-8?B?MldSYTQ4N21xUzhHV2pObEdCUEozMnpoMDVDcitnZitOQW5BQzI0T2N0bXFp?=
 =?utf-8?B?N0pZZTN2RzEzMHdPTzYyTThoTkEwS0lKR25xTTBWNWNaVzlBYk8wNlZRVUxQ?=
 =?utf-8?B?MG00d2w1eFp0MHBxRm9aOE54ZFZwQmxJZTlxY3EzTkRBd203Rzc1emZrL0Ra?=
 =?utf-8?B?OVViMmJYN2lscnVwUUFzWjJjWnhDNHJwWkV0aUEwd2NzWmFBQ3VMVFlPaDlL?=
 =?utf-8?B?WVFiMWZodlZwdkRHOCtCMlArMEMxb1BUR2R0MkVOM08zMWFFTnllV1BuMGZH?=
 =?utf-8?B?SE1rMkZ0TjlKdjFMU29acVpDMll3a2pTcGtFYXJBTUd1TSs3cGU5OUtBWEYz?=
 =?utf-8?B?ZC9mNVMveW1YdGtHZHlKN0dQQ21zUHAzMnp6VFFva3VVTHF5S0U0SU5OOXp5?=
 =?utf-8?B?YkdZbTBlM2tFUTNIa0FoY2pIUnlyU3dHN3hZdHZ0RkFZcHNhYWdmZWl4M2tr?=
 =?utf-8?B?WmFNUUFuY0hUTjJOTkJydmJGRHlwQ3dEVlJudVVkUmxQenZZMmJPNXZZWTJs?=
 =?utf-8?B?VldTWmNqN1lkTGRsbXVrVzQwbjBBS0pPb3RwYW1Vcjdzb284OXM1bTJFbXJW?=
 =?utf-8?B?ZHdpVDNXdlNvcFVwOWtidmVpQjJLQkZEdmIzOWZ3cmlLcGZ1RzRKdHhud3pV?=
 =?utf-8?B?MEtxMHk4SHN1VWhkMHBqWG5FNEZibkRkT0JQcmU3N2ltbGR6MkNwaHQ0WXhZ?=
 =?utf-8?B?V2Z4WjlyQWxOSlpTdWZZOE5xck9CYXRDbmlSK2pYS3hGZzNsOXk1YnYxcTE5?=
 =?utf-8?B?YlhxREhjQitxZStSU3pHc1JVWkxmVWNtM1NiaktOVWd4V05kZzlYa2NrMFhh?=
 =?utf-8?B?OFJ4cjl5M2RSN3RYZzlhWG5reGU1VkNHdEs4YVVVUUJQRExod3RHajNpZm9X?=
 =?utf-8?B?WThGOXJJb0lVWFVoaTgzbnZhTzhKZ2p3dE96Uzc1aVhtVEoyRVAyUTBYaks2?=
 =?utf-8?B?eDJ2UUJrM0p6dExQL1NvNUgvWWtnZ2thdTI0aDRzbGxnRi9nTmltejBWVGZt?=
 =?utf-8?B?OFZ1UXF3cldHRFgrQWIrbjVBclVaeGFuaG5MbHN1NVo2RFBqU1F2akkvY0pR?=
 =?utf-8?B?SDVWdmRhek16akp1WDlTOXppL2JjbXd2ODNrV0c1N3Q3ZXVEUDgzMjlGM2RE?=
 =?utf-8?B?L1E9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 18122751-b530-4830-61c0-08dc8c13676c
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2024 01:43:34.8329
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: C/IaqsU1nOodcc1vRP9TP+2PTLmtQBPSchtUCFwpYrEtiuhKdkr5GueLQbZEjHI+Na6ZOdO5/QD3K0Eysg5fRfrV/l12UZUUeMG0eNtpVnI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR11MB6203
X-OriginatorOrg: intel.com

Hi Babu,

On 5/24/24 5:23 AM, Babu Moger wrote:
> If the BMEC (Bandwidth Monitoring Event Configuration) feature is
> supported, the bandwidth events can be configured to track specific
> events. The event configuration is domain specific. ABMC (Assignable
> Bandwidth Monitoring Counters) feature needs event configuration
> information to assign hardware counter to an RMID. Event configurations
> are not stored in resctrl but instead always read from or written to
> hardware directly when prompted by user space.
> 
> Read the event configuration from the hardware during the domain
> initialization. Save the configuration information in the rdt_hw_domain,
> so it can be used for counter assignment.
> 
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---
> v4: Read the configuration information from the hardware to initialize.
>      Added few commit messages.
>      Fixed the tab spaces.
> 
> v3: Minor changes related to rebase in mbm_config_write_domain.
> 
> v2: No changes.
> ---
>   arch/x86/kernel/cpu/resctrl/core.c     |  2 ++
>   arch/x86/kernel/cpu/resctrl/internal.h |  5 +++++
>   arch/x86/kernel/cpu/resctrl/monitor.c  | 21 +++++++++++++++++++++
>   3 files changed, 28 insertions(+)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
> index ec93f6a50308..856c46d12177 100644
> --- a/arch/x86/kernel/cpu/resctrl/core.c
> +++ b/arch/x86/kernel/cpu/resctrl/core.c
> @@ -542,6 +542,8 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)
>   		return;
>   	}
>   
> +	arch_domain_mbm_evt_config(hw_dom);
> +
>   	list_add_tail_rcu(&d->list, add_pos);
>   
>   	err = resctrl_online_domain(r, d);
> diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
> index 5e7e76cd512f..60a1ca0a11a7 100644
> --- a/arch/x86/kernel/cpu/resctrl/internal.h
> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
> @@ -373,6 +373,8 @@ struct arch_mbm_state {
>    * @ctrl_val:	array of cache or mem ctrl values (indexed by CLOSID)
>    * @arch_mbm_total:	arch private state for MBM total bandwidth
>    * @arch_mbm_local:	arch private state for MBM local bandwidth
> + * @mbm_total_cfg:	MBM total bandwidth configuration
> + * @mbm_local_cfg:	MBM local bandwidth configuration
>    *
>    * Members of this structure are accessed via helpers that provide abstraction.
>    */
> @@ -381,6 +383,8 @@ struct rdt_hw_domain {
>   	u32				*ctrl_val;
>   	struct arch_mbm_state		*arch_mbm_total;
>   	struct arch_mbm_state		*arch_mbm_local;
> +	u32				mbm_total_cfg;
> +	u32				mbm_local_cfg;
>   };

Similar to the abmc_enabled member of rdt_hw_resource, these new
members of rdt_hw_domain are architecture specific and should never be
touched directly by resctrl fs code, for example, from mbm_config_show().

>   
>   static inline struct rdt_hw_domain *resctrl_to_arch_dom(struct rdt_domain *r)
> @@ -622,6 +626,7 @@ void __check_limbo(struct rdt_domain *d, bool force_free);
>   void rdt_domain_reconfigure_cdp(struct rdt_resource *r);
>   void __init resctrl_file_fflags_init(const char *config,
>   				     unsigned long fflags);
> +void arch_domain_mbm_evt_config(struct rdt_hw_domain *hw_dom);
>   void rdt_staged_configs_clear(void);
>   bool closid_allocated(unsigned int closid);
>   int resctrl_find_cleanest_closid(void);
> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
> index b1d002e5e93d..ab0f4bb49bd9 100644
> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
> @@ -1093,6 +1093,27 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
>   	return 0;
>   }
>   
> +void arch_domain_mbm_evt_config(struct rdt_hw_domain *hw_dom)
> +{
> +	u64 msrval;
> +
> +	/*
> +	 * Read the configuration registers QOS_EVT_CFG_n, where <n> is
> +	 * the BMEC event number (EvtID).
> +	 * 0 for evtid == QOS_L3_MBM_TOTAL_EVENT_ID
> +	 * 1 for evtid == QOS_L3_MBM_LOCAL_EVENT_ID

This is what mon_event_config_index_get() is for, no?

> +	 */
> +	if (mbm_total_event.configurable) {
> +		rdmsrl(MSR_IA32_EVT_CFG_BASE, msrval);
> +		hw_dom->mbm_total_cfg = msrval & MAX_EVT_CONFIG_BITS;
> +	}
> +
> +	if (mbm_local_event.configurable) {
> +		rdmsrl(MSR_IA32_EVT_CFG_BASE + 1, msrval);
> +		hw_dom->mbm_local_cfg = msrval & MAX_EVT_CONFIG_BITS;
> +	}
> +}
> +
>   void __exit rdt_put_mon_l3_config(void)
>   {
>   	dom_data_exit();

Reinette