Received-SPF: pass (google.com: domain of linux-kernel+bounces-168403-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <46fe7620-24c3-4f0c-8877-a529626c5025@intel.com>
Date: Fri, 3 May 2024 16:25:00 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC PATCH v3 01/17] x86/resctrl: Add support for Assignable
 Bandwidth Monitoring Counters (ABMC)
To: Babu Moger <babu.moger@amd.com>, <corbet@lwn.net>, <fenghua.yu@intel.com>,
	<tglx@linutronix.de>, <mingo@redhat.com>, <bp@alien8.de>,
	<dave.hansen@linux.intel.com>
CC: <x86@kernel.org>, <hpa@zytor.com>, <paulmck@kernel.org>,
	<rdunlap@infradead.org>, <tj@kernel.org>, <peterz@infradead.org>,
	<yanjiewtw@gmail.com>, <kim.phillips@amd.com>, <lukas.bulwahn@gmail.com>,
	<seanjc@google.com>, <jmattson@google.com>, <leitao@debian.org>,
	<jpoimboe@kernel.org>, <rick.p.edgecombe@intel.com>,
	<kirill.shutemov@linux.intel.com>, <jithu.joseph@intel.com>,
	<kai.huang@intel.com>, <kan.liang@linux.intel.com>,
	<daniel.sneddon@linux.intel.com>, <pbonzini@redhat.com>,
	<sandipan.das@amd.com>, <ilpo.jarvinen@linux.intel.com>,
	<peternewman@google.com>, <maciej.wieczor-retman@intel.com>,
	<linux-doc@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<eranian@google.com>, <james.morse@arm.com>
References: <cover.1711674410.git.babu.moger@amd.com>
 <1194d7c66607648368fe34460709649b76ab85e3.1711674410.git.babu.moger@amd.com>
Content-Language: en-US
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <1194d7c66607648368fe34460709649b76ab85e3.1711674410.git.babu.moger@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?b1R1MjRTL3J4NEduaEhQWXUxN3N5WmVtaTAxT09adDJzbFJRSUZFSWtXRllU?=
 =?utf-8?B?bm92VU9JOVVwaElKTThkdTJENzVjV0plb0JZUVRaeWpvbFBjVmVzbkFySzZY?=
 =?utf-8?B?dUduWXp6NXU1eVk4b243WXlXQzBXNXhvd20vWjdTOWhCakFHSkNXWXdubnF4?=
 =?utf-8?B?QkltQlR6QkhBeE1sRXRHU3d4eU04eG9jVk4xR2dNdkMzaFpHdDBIaWtVUExy?=
 =?utf-8?B?anRsajlEL1dkK29qemwxY1JaTkN0eDUydVdyZE9BZ2h3ODNVMU52SUlzYTZx?=
 =?utf-8?B?blJEVXFDMWpBQjA4ZHJBSE1McExaTnZQZStPek5ocXZHRTRFT1pkQzhrdU1F?=
 =?utf-8?B?TlhaWWVEcGRwVWlMU3BaY1Ryc3A3TVBTY1pXTXUyYUcxbHM3Z2FqRFNLbVdk?=
 =?utf-8?B?bmRvbGFqc0tkYURwV2dmaCswTENSUytVVEdZSmdBZmEwYkNKRVBCSXJBQ1hW?=
 =?utf-8?B?Y0lzRWRNRDZnQjcxWnhxQjk5MGhPcngrV1o3cEJZQXRyMmhJcndsdmdNQVpz?=
 =?utf-8?B?ZVNhYkhiOGMxcFZIQ25DL2V0K3BqSTRtUXo5TThnUk9ONURyb3ljVGZ3K29m?=
 =?utf-8?B?Y2liK2ZUdENWY0RpbmFqQkwvQStHM2F3aWI1bjNucDBPYy8vSXIvQ29HSW5y?=
 =?utf-8?B?bmNHblZXQ2FOeFRaR1JxQU1FY0F5ZHNnVWNIRVBUTUtYZm9rdHljK0Y3bE56?=
 =?utf-8?B?VHBFMDlKRXZNejNEeURueGZVSWYzVC8yYURDR2J5cU94aS9RbWNTMkxGcSty?=
 =?utf-8?B?eDh3RHhaN0taOHJvcGdYSnA4bDMwdmJXUWdFNFkzQjR2VnVqTVlCdHFSVjNh?=
 =?utf-8?B?Y2R2R0hiRGlRN1k2dXJ1SFN1dG5GeE1Yemt5RnQwRjFIcmd5S3NvcWpXRVU0?=
 =?utf-8?B?cktwR1hOZkt6aFdPbjIzQXRZT3RNdXB5cXhzSUFnRWFobUZmMWxtRHVPemU3?=
 =?utf-8?B?UVNRUGU3SnJybE1SaGdLM0s2NWMxcTZ3M3hUYVpKZ09rQlc2cWxaWEtRWGln?=
 =?utf-8?B?MmVTQm5OeU10WmJudDFleEhRcUdaYU9xMmhXaUJJV2tGeVQ3MmRoZWVoRmhy?=
 =?utf-8?B?QVZTckZOeSthMTljbm1lMElKVWxHd1JCMDJzMkczTlhXMlZORk16blVBczIr?=
 =?utf-8?B?RDR4aGNka1FuQ21KeHBDQmlMWTdEU2hxUlNXVloyY2FHZXU2NkthQVRCU0VW?=
 =?utf-8?B?a2R4bmY3OG5jNGIrOU9mZk1lRjFDdWlOcUNHeUN3RkozNytPei9kSVQvUW9I?=
 =?utf-8?B?YTVWWTZaaTg1KzdKWklQM0pyMU5IWkxnVmFyRmlmNzFsbWp5NFlwU2dGekdu?=
 =?utf-8?B?ZmlpTDJzQmtjMXc1MklERFQxL3pOQ0wrY25seHNPblBCR0dHdWlmREZCNjNJ?=
 =?utf-8?B?UjlOd2xJNmNPY3RTSEd3dGdjSzRmbWRSSXhIai8rUysrT3VIWkIvZDVmSGxp?=
 =?utf-8?B?NWRpaXBhMnFSck1Wek04UHVjOEwyNzcwbER1L1g1NXBWVUNTellnRUN2cW8w?=
 =?utf-8?B?NGd2b0tNRjljYnUvdXdDSzlVZkhxL09RUlhoVFlKRXpHSVhCVUExQWtUcjZP?=
 =?utf-8?B?anhLS0tlL2dPcHcwQW5YdW91T3cxZzIzdEhMY0Z3QS9CbEd2aytxVFZMeCtK?=
 =?utf-8?B?eDdxbXNVMnA5OWxHRFdmeTNqd3hRTW5ST0xzN0JYTENmNTFMOWRhVFZCU09w?=
 =?utf-8?B?dmZLQk5hbnNRSG1ZbDJFREIrUXUySGRxM3c5dFJOUG1QWE9zTVdqTkpPSmRU?=
 =?utf-8?B?NlBkeDA5OW05Qks3dEVVcVBsODRPc2Y2d3VGWVoyVVJUTzhuVGJwNDVVODR5?=
 =?utf-8?B?UUZDSmkzaGZ4enRjQnlWTkhLTFVBeEhVZ1VOWkpBL3B4N2loajZ5N2h1NFFu?=
 =?utf-8?B?WDhSQ2dOYS83VEVZM3h2SWF3dFo1U1VtamYvckFBUnFCWXJUL3J1R0M3bVQv?=
 =?utf-8?B?TVIrZFJyNlhDVUlmM0N2QUZRYTB6ZnJLaVFnaDRSQTVOK0xuTTQ5bktteXNi?=
 =?utf-8?B?aDBDUU1YWXVVUDB0b053VjIrWHMxekhxSEZVOG5JeHppRXhYVVA3WFkxaVBt?=
 =?utf-8?B?MTdQN0llMHpjdTg2NDE2V2UzaEdIL1d1bG96TUNqMkgrR0hodzgvL0lJcXhG?=
 =?utf-8?B?T0RjcDFxL011ZTkzTGN3RURvQ1BQWHVSMVBoOGhhZWpwZ2UycTN1WWg0c25C?=
 =?utf-8?B?VGc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 6757f06d-dcc5-4d0e-1d84-08dc6bc842b7
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 May 2024 23:25:03.8073
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8ag5w1I3LKkRSaN1BsTaMojli0SPNYi/oa86a4MVoMueuTE5E0E6xJJR4BZ2N1bEOAew7zKYTD2OpHgKUnLvOJ8N8DVJmb2W3hF/48ZhP3U=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR11MB8607
X-OriginatorOrg: intel.com

Hi Babu,

On 3/28/2024 6:06 PM, Babu Moger wrote:
> AMD hardware can support 256 or more RMIDs. However, bandwidth monitoring
> feature only guarantees that RMIDs currently assigned to a processor will
> be tracked by hardware. The counters of any other RMIDs which are no longer
> being tracked will be reset to zero. The MBM event counters return
> "Unavailable" for the RMIDs that are not active.

I think it will be helpful to use consistent terms. For example, above uses
"tracked by hardware" as well as "active". "tracked by hardware" seems more
specific to me and I think it would help to understand this work if this is
used consistently.

> 
> Users can create 256 or more monitor groups. But there can be only limited

I think you write "Users can create 256 or more monitor groups." to match
with earlier "AMD hardware can support 256 or more RMIDs.". Can this be made
specific with "Users can create as many monitor groups as RMIDs supported."?
(please feel free to improve)

> number of groups that can give guaranteed monitoring numbers.  With ever
> changing configurations there is no way to definitely know which of these
> groups will be active for certain point of time. Users do not have the
> option to monitor a group or set of groups for certain period of time
> without worrying about RMID being reset in between.
> 
> The ABMC feature provides an option to the user to assign an RMID to the
> hardware counter and monitor the bandwidth for a longer duration.
> The assigned RMID will be active until the user unassigns it manually.
> There is no need to worry about counters being reset during this period.
> Additionally, the user can specify a bitmask identifying the specific
> bandwidth types from the given source to track with the counter.
> 
> Linux resctrl subsystem provides the interface to count maximum of two
> memory bandwidth events per group, from a combination of available total
> and local events. Keeping the current interface, users can assign a maximum
> of 2 ABMC counters per group. User will also have the option to assign only
> one counter to the group. If the system runs out of assignable ABMC
> counters, kernel will display an error. Users need to unassign an already
> assigned counter to make space for new assignments.
> 
> AMD hardware provides total of 32 ABMC counters when supported.

I am not sure if you want to mention this. As written this sounds like
a hardcoded value but it clear from later patches the number of counters
is learned from hardware.

> 
> The feature can be detected via CPUID_Fn80000020_EBX_x00 bit 5.
> Bits Description
> 5    ABMC (Assignable Bandwidth Monitoring Counters)
> 
> The feature details are documented in APM listed below [1].
> [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming
> Publication # 24593 Revision 3.41 section 19.3.3.3 Assignable Bandwidth
> Monitoring (ABMC).
> 
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537

Reinette