Received-SPF: pass (google.com: domain of linux-kernel+bounces-175177-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <686af2dc-40dd-9a95-d967-2d8fdaaff197@amd.com>
Date: Thu, 9 May 2024 19:28:42 -0500
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Reply-To: babu.moger@amd.com
Subject: Re: [RFC PATCH v3 10/17] x86/resctrl: Add data structures for ABMC
 assignment
Content-Language: en-US
To: Reinette Chatre <reinette.chatre@intel.com>, babu.moger@amd.com,
 corbet@lwn.net, fenghua.yu@intel.com, tglx@linutronix.de, mingo@redhat.com,
 bp@alien8.de, dave.hansen@linux.intel.com
Cc: x86@kernel.org, hpa@zytor.com, paulmck@kernel.org, rdunlap@infradead.org,
 tj@kernel.org, peterz@infradead.org, yanjiewtw@gmail.com,
 kim.phillips@amd.com, lukas.bulwahn@gmail.com, seanjc@google.com,
 jmattson@google.com, leitao@debian.org, jpoimboe@kernel.org,
 rick.p.edgecombe@intel.com, kirill.shutemov@linux.intel.com,
 jithu.joseph@intel.com, kai.huang@intel.com, kan.liang@linux.intel.com,
 daniel.sneddon@linux.intel.com, pbonzini@redhat.com, sandipan.das@amd.com,
 ilpo.jarvinen@linux.intel.com, peternewman@google.com,
 maciej.wieczor-retman@intel.com, linux-doc@vger.kernel.org,
 linux-kernel@vger.kernel.org, eranian@google.com, james.morse@arm.com
References: <cover.1711674410.git.babu.moger@amd.com>
 <c3103dd30bc07070380a01922f66208759afe837.1711674410.git.babu.moger@amd.com>
 <3149b62a-b9a4-49c6-a757-5a0e36df62df@intel.com>
 <b367a6ba-3ff1-4173-bf71-cd7a04bf9882@amd.com>
 <89d9996e-3414-43f9-82f6-0260a4f49285@intel.com>
From: "Moger, Babu" <bmoger@amd.com>
In-Reply-To: <89d9996e-3414-43f9-82f6-0260a4f49285@intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bGhRdlRjZ0VDbHVkR3c0Zy9zd0RmUSsvSU5lTVpvL1dVM0hHT3NHdDczOHhL?=
 =?utf-8?B?R2YwSEdJd0k0dG9xU0hjWFZ5anhXcU9NZWZKUlM1NUVHWStacVA3LzJoVExo?=
 =?utf-8?B?dDE0aUJYTmtWbVJia3NmMFhDajFGNEFRckVuMkpGV0FPTks4SVExeVlLV20z?=
 =?utf-8?B?dTg4NEhxSWZUazRJa3FBMDhwRUk4WWdWYTNiZ0Fla3k3aVRIUk1XTkdjVnA3?=
 =?utf-8?B?ckpRTlhIK3k0RUhEYnNoUVhTN0hxNjZONHFMcUdUaFRWUXpBekhVSDg2enJF?=
 =?utf-8?B?VzQ5ZnNnR2xpVjRGNUZlZnB2U2x2QU40djdXTC85SVp5eDdrSmtja3d1dUpN?=
 =?utf-8?B?VUhLVG16UlFuN1J1NUdCcXJ2OTAvTWhadVppSWgzWDA0UDcyNDhoSmhBelNz?=
 =?utf-8?B?NDI5ell5cEx2eDVjdkFzZkdWeVJFbWVlQTVBQm81VkROKzNPZzBqdWswNTh6?=
 =?utf-8?B?SnFvN0NIMzVLdGd5R0ZHdEpDS1BYR3BWK1liYmEvZkFYbktkZmpWTWlKQlJU?=
 =?utf-8?B?ZHVjSWhQaURiM2FOQlNaampQZCswdkJWdlZBVUpZSEw2c1pLRktYSTVOcitk?=
 =?utf-8?B?dEQyb25OaUZjcmtQaElTR0xCOEprSklibGhVTjRwTktMY09CcUV5d0drbWtT?=
 =?utf-8?B?NE5xeUhrU2RyVlNiVVhXQW9Ea0NZbFRCWDZadisxeVBZTkRJN0VxWDhzYmVn?=
 =?utf-8?B?V3NrNFZmZy9acFhma3BQOTZnazR4SmlScUt0VWRwdzVnWFM3UE1CRnVXQnl2?=
 =?utf-8?B?dkxKYmFxSGtlWUd5QmhiSjZmVXRWVkZjNlJIcDBtR2VOR2hTYmx3SSs0bkg1?=
 =?utf-8?B?aWJPNGx1NDhhdFpKR3hWdTIxVDN5K3VPWncrZzU5VlM0cWlvK3N4Q2lhYk8x?=
 =?utf-8?B?S1VzSU9pOWkrU01nUG1UWjRUWFdZWmRROUxVdGVoUnFqUFVrK054Qk1ONTg2?=
 =?utf-8?B?aWxZeDdCaFdKWVhJaGVEVHBGSUJwMXZuc29ocHJwUk45MjdZdXZJVnVLN1dt?=
 =?utf-8?B?L1FlYjVGOWpmN2hpOTJjcDI4cjR5MFlwQ05MOXhxcy9oU0U2bkdTandEUTd6?=
 =?utf-8?B?VzJpTXp3ZXhIdVJZb2FpeU5vVUJUTmhkNElNQWxLaXpwdlBwYndLMmVmbWRB?=
 =?utf-8?B?UFlsODhOcVM1OCtYV2VTUEhHU2Naa1ZNMmJiYmN0aHUrQ1lJbDBMVlhrWTZk?=
 =?utf-8?B?MnlYS2dhWXFzTnp1ckczL0xDZjFuWUZyWGFrVC9hd1lOWmVXeEw2bkRvejhq?=
 =?utf-8?B?U1FvbDRNVXFXSFdXVmxiZndER0hWaGxFaFk4MW9VaGVabTVtaGlFLzJob1dp?=
 =?utf-8?B?TGx5ckhxdUxpWUNXVGhadkdrSWd2SVg0d1BUWms2SjltWTFIY3ZhcVp5TzVn?=
 =?utf-8?B?THZLUFd0ZWJYek9BdzJBYUFZSEZOZS93MWdhdVl0YmoyK2Jla0c5TGNhbWtZ?=
 =?utf-8?B?UmtIZlJHbnh4RUc5N3h5RGhiWEttcHdWaUlOellWM3F4V3JYL0xmNzZGRjRL?=
 =?utf-8?B?a0swZ244SVZvMHZEdFU5N0RMSzBvUGpMVVRpNFgxYTJ3K29SNmtxZVpXM0Zw?=
 =?utf-8?B?eSszcDBrVWpuRzVDeFlScmxmZjJvUlFRVExSbnYrdkk3WU5yVGYwMXNuWDhL?=
 =?utf-8?B?MXl3WGlOSXNadWgxMWU4dm4zYW1BdThrN2JjREgweWgxR1hZNndZT3ZobEVB?=
 =?utf-8?B?UnErV3RCSTR5b0hoNEppL2ViYjM3SUxYdjAzN2NCUnBSWFhFU01UWjhOcjVt?=
 =?utf-8?B?ZUM4UkczWTllWUQ5dHY1VU82UFNhamNvdi9SZmI2cGpBYmRUWkZhOC9Ma1RF?=
 =?utf-8?B?OGRhNE9oT3NFbkNuWHBORzFMZ09pdlhzZlNBcmhVaXJmK2tIUlVqVGpQS1Ro?=
 =?utf-8?B?WUQ4bFNKWUw2dWZqbnpNL01SMkFpSUJTV2pZbUt3NjZIcmpYakZSUGo0Mmht?=
 =?utf-8?B?UFJkMHlVZmU4RVQzVjB1SmI4bEVUdXdpUFhtdGFMSzJVWXdOcjFqeUZoNSty?=
 =?utf-8?B?TkM5T2xaS1JxSHdZQ1ZucXJ6NEhBTjBjRUx0b2lpV1JtelJwUHg3YnNYcGVu?=
 =?utf-8?B?Z0xER25rVXJ6VmtONFVlaE1jM09yeWJUMUdSZnV4K2l6b1pHMmVPcmYyUTF6?=
 =?utf-8?Q?5WvQ=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9a660f64-7fd2-48e7-31ec-08dc708828bb
X-MS-Exchange-CrossTenant-AuthSource: MW3PR12MB4553.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2024 00:28:48.2507
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 83HJZTwxadigbc4Gmmpo7Zyg/L7PqVbRJqlaOeukHjLrbeUM3T5+BtmB4jvWxM5q
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6210

Hi Reinette,

On 5/7/2024 6:06 PM, Reinette Chatre wrote:
> Hi Babu,
> 
> On 5/7/2024 1:40 PM, Moger, Babu wrote:
>> On 5/3/24 18:32, Reinette Chatre wrote:
>>> On 3/28/2024 6:06 PM, Babu Moger wrote:
>>>> ABMC (Bandwidth Monitoring Event Configuration) counters can be configured
>>>> by writing to L3_QOS_ABMC_CFG MSR. When ABMC is enabled, the user can
>>>> configure a counter by writing to L3_QOS_ABMC_CFG setting the CfgEn field
>>>> while specifying the Bandwidth Source, Bandwidth Types, and Counter
>>>> Identifier. Add the MSR definition and individual field definitions.
>>>>
>>>> MSR L3_QOS_ABMC_CFG (C000_03FDh) definitions.
>>>>
>>>> ==========================================================================
>>>> Bits 	Mnemonic	Description		Access Type   Reset Value
>>>> ==========================================================================
>>>> 63 	CfgEn 		Configuration Enable 	R/W 		0
>>>>
>>>> 62 	CtrEn 		Counter Enable 		R/W 		0
>>>>
>>>> 61:53 	– 		Reserved 		MBZ 		0
>>>>
>>>> 52:48 	CtrID 		Counter Identifier	R/W		0
>>>>
>>>> 47 	IsCOS		BwSrc field is a COS	R/W		0
>>>> 			(not an RMID)
>>>>
>>>> 46:44 	–		Reserved		MBZ		0
>>>>
>>>> 43:32	BwSrc		Bandwidth Source	R/W		0
>>>> 			(RMID or COS)
>>>>
>>>> 31:0	BwType		Bandwidth types to	R/W		0
>>>> 			track for this counter
>>>> ==========================================================================
>>>>
>>>> The feature details are documentd in the APM listed below [1].
> 
> documentd -> documented

Sure.
> 
>>>> [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming
>>>> Publication # 24593 Revision 3.41 section 19.3.3.3 Assignable Bandwidth
>>>> Monitoring (ABMC).
>>>
>>> This changelog is purely a summary of the hardware architecture. I have not come
>>> across a clear explanation on how this architecture is intended to be supported
>>> by resctrl. When would resctrl need/want to set particular fields? What is
>>> the mapping to resctrl?
>>
>> Something like this in the changelog?
>>
>> ABMC feature provides an option to assign(or pin) an RMID to the
>> hardware counter and monitor the bandwidth for a longer duration.
> 
> Regarding "a longer duration":
> https://lore.kernel.org/lkml/b5e68d85-4bf2-4e55-a9c1-b39cb7d94db6@intel.com/

Sure. Will take care of this.
> 
>>
>> Hardware counters can be configured by writing to L3_QOS_ABMC_CFG MSR.
>> Configuration is done by setting the CfgEn field while specifying the
>> Bandwidth Source(RMID or CLOS), Bandwidth Types, and Counter Identifier.
>>
>> Add the configuration register definition.
> 
> This still looks to me like a cryptic description of the hardware architecture.
> Could you please spell out to me how the above answers each of my questions?

Will try to elaborate this in next revision.
Thanks
- Babu Moger