Received-SPF: pass (google.com: domain of linux-kernel+bounces-173517-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <7fd1ec49-fd10-4749-acb6-264cd9f4c300@amd.com>
Date: Wed, 8 May 2024 10:57:33 -0500
User-Agent: Mozilla Thunderbird
Reply-To: babu.moger@amd.com
Subject: Re: [RFC PATCH v3 11/17] x86/resctrl: Introduce mbm_total_cfg and
 mbm_local_cfg
To: Reinette Chatre <reinette.chatre@intel.com>, corbet@lwn.net,
 fenghua.yu@intel.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
 dave.hansen@linux.intel.com
Cc: x86@kernel.org, hpa@zytor.com, paulmck@kernel.org, rdunlap@infradead.org,
 tj@kernel.org, peterz@infradead.org, yanjiewtw@gmail.com,
 kim.phillips@amd.com, lukas.bulwahn@gmail.com, seanjc@google.com,
 jmattson@google.com, leitao@debian.org, jpoimboe@kernel.org,
 rick.p.edgecombe@intel.com, kirill.shutemov@linux.intel.com,
 jithu.joseph@intel.com, kai.huang@intel.com, kan.liang@linux.intel.com,
 daniel.sneddon@linux.intel.com, pbonzini@redhat.com, sandipan.das@amd.com,
 ilpo.jarvinen@linux.intel.com, peternewman@google.com,
 maciej.wieczor-retman@intel.com, linux-doc@vger.kernel.org,
 linux-kernel@vger.kernel.org, eranian@google.com, james.morse@arm.com
References: <cover.1711674410.git.babu.moger@amd.com>
 <46f83accddba3c25a428d1a30f6091913a4f7c73.1711674410.git.babu.moger@amd.com>
 <5c612f5c-7709-49db-b998-a676b61245e1@intel.com>
Content-Language: en-US
From: "Moger, Babu" <babu.moger@amd.com>
In-Reply-To: <5c612f5c-7709-49db-b998-a676b61245e1@intel.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bDVrUC9EWENHMmdRb2h3dlhyV0F1ZnRqRTR5YUJCdlIxbEtwekJHNE9Hb1RH?=
 =?utf-8?B?dWhUM3ExaDUvVWlXNEhlc01sWjhzd2hMSHVaZkdSa2ZBVG9uZ1YydjdxQ3Jr?=
 =?utf-8?B?VkxNa2Q4UzBET2FJRTFGcWtWelBDMU80RHY5R2dJWk9tc3ZRc0VwNWJaNUQw?=
 =?utf-8?B?cGVuSlJsVnJaWXdwQnl0WU82cGtBYy9mbFgyWjZsdEVGdGJzNXhxRWRyNjN2?=
 =?utf-8?B?Unp4RXNwOGtWVmh2V3hKRWhQUklsbGQraUxVUW5hQ1VVdGhQa3lRbFpqc2VG?=
 =?utf-8?B?TzVyOWpJOGRMTEdwcFNEZnBlK0JGR0dETGJVem1TS2pLMTF4QnJ5V2dIcVg3?=
 =?utf-8?B?YlJlNldyemM4a0pjVGtGa0xQL2FBQ0xQRGVHblhDVjBzSU82UG9nTXNqblJW?=
 =?utf-8?B?VHp1MUJSMmJ0d0VtQk1hT1BnTkhjTFZiVWZXbGp1enZTSkJmZi82RmUzaGVT?=
 =?utf-8?B?TThJOE5mbUpybVRCeThsOERtWUZHS1U0d2dGbTNrbllHY0ZCUi8xUGc0clJa?=
 =?utf-8?B?cmlPMktVZXU2SmtoTHRJdktWSnpzano0ajgvUWhYK2w4bDlLNVBYN3gyZ2hF?=
 =?utf-8?B?UUY2Wkttc2pwbXdDQkF3b0Nmb2NmOFYwUGZuTUtuKzRla3FOcXZocVlqRUMx?=
 =?utf-8?B?OWlWekZ4c29VQS9nZkM2QmhLZGkrakN4RzExZEtEcG4zYWxLNkR4MEZYOGVk?=
 =?utf-8?B?NXdSVkF5ak1QdGNjVFhXOUV2K1lpcjl6SHlZcTRFa1J2clo1K2JxR2tPQjlV?=
 =?utf-8?B?djdzTVJrSm0rQ0dzTnlGRGhpa0pCYWJHUTlqSHZwWTF0NG1UdThMR0k0N3Ri?=
 =?utf-8?B?N3JuenFjeFlwdWRld24zS1ZyNzVTdFVhUWc0V0lzazlhQndIOHFEd3VLMFRw?=
 =?utf-8?B?L1hYUDU1YTJLM3l6aVNOQXVzblZySTl2aWROY0tpNWo0OVBsNktSalJDM3hm?=
 =?utf-8?B?OWVpU0MrN0w1MG80SjQ5YWo0MFhkNi9PdjBLVGExamgwdGxha2psZTJXczN6?=
 =?utf-8?B?YU4rVTAzdTg2QkJiY1krOERSUURpNVlPNGluYUFPM1hFVDFMNDFUZ09tZFdk?=
 =?utf-8?B?ZVBKMGRwQXBPdmtyalh5c0VRK0hLS1JBd2JoWDVWd1V4ZE55UEl0V1JWOGhX?=
 =?utf-8?B?MDZHbmIvRjI5RWZjcDU4QUh3bFdzZ1JuNVNkN0RQRVp6WisxWHhpRHNCVUZj?=
 =?utf-8?B?V0M3VWVheGRGZWRveWhuek9GVE0xc3hqcjVBRG8zSHhMOWpoMHFGRmVCMW5q?=
 =?utf-8?B?WnJyUEhYMGFuMzl3dkF2QXF0bVV3Z0ZNaU1NWUJqVFh0cHF4WERnZnBtcXcy?=
 =?utf-8?B?YndWTndJMm5FZEx0bWxUazh2eTJSRmRsWGZXVmVzVDNZQW8rRmd1ZEtLeTA5?=
 =?utf-8?B?T0lmL1pINFVMUzg1TnAxWm8yaEowa3U2YnFsTUxMandCMTRGdFZHanRHQTRW?=
 =?utf-8?B?bzYwbFAzaTFaZjB4MHpjRzg4cjQzUHNDaGxSVGxGaElLRm1CZWRCcjBac1Jm?=
 =?utf-8?B?N0NleHRob1Z0SmxiVWJWdm9LViszT0pRQzNiUXcvWC9YZHd1ZUZmWVd4d3l4?=
 =?utf-8?B?OUo5blJSUzJQdmhHeVNpSVRvMXU5V0pCNWVpN2hkMmF6QW0zbktnRno5VDBv?=
 =?utf-8?B?QkhEYkZ4SUZBS0k3NmZMNkdhMElMMFFvV3F5TGd2N2VzQlhJUFF1MFM0eGgw?=
 =?utf-8?B?M1M0NE92bkdXSGRxZkN2KzFrdTdTWGpNcmVabk92YWh5N051NVNVZUEvRDIy?=
 =?utf-8?B?Q2NjeWhHcSs1NkpHOXptblhDSnZSTVl0ZXVLeTJKRXpXU2tFOG12aXNLTnJk?=
 =?utf-8?B?Rm94N2ZVbVoyZDJET2U4YzNIWnI2MWdzeTEyZEh0Nk42SUIwenBKN1Ixak1k?=
 =?utf-8?B?U3NFOHVEYktTZW1OM3NNQVZYRFJSNFFVYUZKdzFCa0lxOHJlZFdFdzRWcmNS?=
 =?utf-8?B?NlpxK1pxNm9GdVlqYWhKSUEzZkJBUGNmaGUrTlZFTE9RMGlSYkpEMWs0eFVY?=
 =?utf-8?B?amVJcEN0dmhyYUVqaFJpNTlpL2dWQlpZcVAyalRxaWlMMTN1VG9HY21lOFZw?=
 =?utf-8?B?bUxRT3VkOU9TbnV2bkg2RFROK3NZWFNTWXMvS1hLbTUxdmVxbWhNeVhKWTBV?=
 =?utf-8?Q?VoX8=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c25bcea3-902d-45f5-1a08-08dc6f7794dc
X-MS-Exchange-CrossTenant-AuthSource: MW3PR12MB4553.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 15:57:37.0979
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CxhGZomiTutDD1lnTnWi9FNnES1fJaBOv3kQnn/Y4zum7UZugzf1pX/pvvrfPspk
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4337

Hi Reinette,

On 5/3/24 18:33, Reinette Chatre wrote:
> Hi Babu,
> 
> On 3/28/2024 6:06 PM, Babu Moger wrote:
>> If the BMEC (Bandwidth Monitoring Event Configuration) feature is
>> supported, the bandwidth events can be configured to track specific events.
>> The event configuration is domain specific. ABMC (Assignable Bandwidth
>> Monitoring Counters) feature needs event configuration information to
>> assign RMID to the hardware counter. Currently, this information is not
>> available.
> 
> hmmm ... "Currently, this information is not available." does not sound
> accurate. Perhaps it can be made more specific with something like:
> "Event configurations are not stored in resctrl but instead always
> read from or written to hardware directly when prompted by user space."
> (feel free to improve)

ok. Sure

> 
>>
>> Save the event configuration information in the rdt_hw_domain, so it can
>> be used while for RMID assignment.
> 
> "be used while for RMID assignment" -> "be used for RMID assignment"?

Sure.

> 
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>>
>> ---
>> v3: Minor changes related to rebase in mbm_config_write_domain.
>>
>> v2: No changes.
>> ---
>>  arch/x86/kernel/cpu/resctrl/core.c     |  2 ++
>>  arch/x86/kernel/cpu/resctrl/internal.h |  3 +++
>>  arch/x86/kernel/cpu/resctrl/monitor.c  | 11 +++++++++++
>>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 16 +++++++++++++++-
>>  4 files changed, 31 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
>> index 50e9ec5e547b..ed4f6d49d737 100644
>> --- a/arch/x86/kernel/cpu/resctrl/core.c
>> +++ b/arch/x86/kernel/cpu/resctrl/core.c
>> @@ -555,6 +555,8 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)
>>  		return;
>>  	}
>>  
>> +	arch_domain_mbm_evt_config(hw_dom);
>> +
>>  	list_add_tail_rcu(&d->list, add_pos);
>>  
>>  	err = resctrl_online_domain(r, d);
>> diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
>> index 41b06d46ea74..88453c86474b 100644
>> --- a/arch/x86/kernel/cpu/resctrl/internal.h
>> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
>> @@ -385,6 +385,8 @@ struct rdt_hw_domain {
>>  	u32				*ctrl_val;
>>  	struct arch_mbm_state		*arch_mbm_total;
>>  	struct arch_mbm_state		*arch_mbm_local;
>> +	u32				mbm_total_cfg;
>> +	u32                             mbm_local_cfg;
> 
> (please fix tabs/spaces)

Ack.
> 
>>  };
>>  
>>  static inline struct rdt_hw_domain *resctrl_to_arch_dom(struct rdt_domain *r)
>> @@ -648,6 +650,7 @@ void __check_limbo(struct rdt_domain *d, bool force_free);
>>  void rdt_domain_reconfigure_cdp(struct rdt_resource *r);
>>  void __init resctrl_file_fflags_init(const char *config,
>>  				     unsigned long fflags);
>> +void arch_domain_mbm_evt_config(struct rdt_hw_domain *hw_dom);
>>  void rdt_staged_configs_clear(void);
>>  bool closid_allocated(unsigned int closid);
>>  int resctrl_find_cleanest_closid(void);
>> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
>> index 56dc49021540..8677dbf6de43 100644
>> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
>> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
>> @@ -1090,3 +1090,14 @@ void __init intel_rdt_mbm_apply_quirk(void)
>>  	mbm_cf_rmidthreshold = mbm_cf_table[cf_index].rmidthreshold;
>>  	mbm_cf = mbm_cf_table[cf_index].cf;
>>  }
>> +
>> +void arch_domain_mbm_evt_config(struct rdt_hw_domain *hw_dom)
>> +{
>> +	if (mbm_total_event.configurable)
>> +		hw_dom->mbm_total_cfg = MAX_EVT_CONFIG_BITS;
>> +
>> +	if (mbm_local_event.configurable)
>> +		hw_dom->mbm_local_cfg = READS_TO_LOCAL_MEM |
>> +					NON_TEMP_WRITE_TO_LOCAL_MEM |
>> +					READS_TO_LOCAL_S_MEM;
>> +}
> 
> Shouldn't the defaults be discovered from hardware?

Yes. That is correct. We can read from the hardware.

-- 
Thanks
Babu Moger