Received-SPF: pass (google.com: domain of linux-kernel+bounces-170260-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <23514112-3125-4cb1-8178-1a8d582e5a5a@amd.com>
Date: Mon, 6 May 2024 12:57:14 -0500
User-Agent: Mozilla Thunderbird
Reply-To: babu.moger@amd.com
Subject: Re: [RFC PATCH v3 01/17] x86/resctrl: Add support for Assignable
 Bandwidth Monitoring Counters (ABMC)
To: Reinette Chatre <reinette.chatre@intel.com>, corbet@lwn.net,
 fenghua.yu@intel.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
 dave.hansen@linux.intel.com
Cc: x86@kernel.org, hpa@zytor.com, paulmck@kernel.org, rdunlap@infradead.org,
 tj@kernel.org, peterz@infradead.org, yanjiewtw@gmail.com,
 kim.phillips@amd.com, lukas.bulwahn@gmail.com, seanjc@google.com,
 jmattson@google.com, leitao@debian.org, jpoimboe@kernel.org,
 rick.p.edgecombe@intel.com, kirill.shutemov@linux.intel.com,
 jithu.joseph@intel.com, kai.huang@intel.com, kan.liang@linux.intel.com,
 daniel.sneddon@linux.intel.com, pbonzini@redhat.com, sandipan.das@amd.com,
 ilpo.jarvinen@linux.intel.com, peternewman@google.com,
 maciej.wieczor-retman@intel.com, linux-doc@vger.kernel.org,
 linux-kernel@vger.kernel.org, eranian@google.com, james.morse@arm.com
References: <cover.1711674410.git.babu.moger@amd.com>
 <1194d7c66607648368fe34460709649b76ab85e3.1711674410.git.babu.moger@amd.com>
 <46fe7620-24c3-4f0c-8877-a529626c5025@intel.com>
Content-Language: en-US
From: "Moger, Babu" <babu.moger@amd.com>
In-Reply-To: <46fe7620-24c3-4f0c-8877-a529626c5025@intel.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?L2NucVlPTDlCTGdSVzJ3ai9vOUNIV3A4aWlRS1pIVnhULzlsZWh1WHNPVmNI?=
 =?utf-8?B?NzN6RzZGQytXVFNwSTFNNDJzT05jUjdjTXNQTjFVZmtnWGhTT09LUnhGb2NV?=
 =?utf-8?B?Z3UrSjhKSjIrejNXS1JuQUR1cWhkc1FJUzZPWkxYb2JwT08zL0Z1cjdwSU5k?=
 =?utf-8?B?SjVQcDZMM1lBdGdGNis4WDZYdHBMaXVYYWR1OTFYd1lDd0oxbHNXUzd3MlBs?=
 =?utf-8?B?MnJReVU1VEhScGEvZjJwZXpFOStwSVlVTXJ5bmJsNmhHMU9RQUlNeU9iWEFi?=
 =?utf-8?B?R2tIRHFia1RQTVdPMGRqZDVWOER6Z0Z2eXZ2VUxmalNmRlVVSU4vWkRhTHh6?=
 =?utf-8?B?WG5TOCttL2txM0Y5MlBONXV5MTZpaTRIdGNOVTdML0RRQ1pOK2dPVjJiV0Fp?=
 =?utf-8?B?Z0Y1NVN3RjArZHByV0dmSkdWZFZVeC9zU1pORENhaGNWeGt6bkoxbmlZWGFt?=
 =?utf-8?B?YlRwaXlsSUFVT0ZFTytmVjgvTDNVZVdFdG9sZ0xYMXpNZmt3TnJWWG9qSS93?=
 =?utf-8?B?bjRoNDdzTGpSdG5ZZ0RpbC9ZbmxwSkVZWGE4SjJBa1RtMUp5OGhtcUVoWklO?=
 =?utf-8?B?YmdCK0UweDBZaS9Bd0RFcEF3R0dBdllyZ1hrSW1tTldzRWl1RTVXaXhtb241?=
 =?utf-8?B?Ukp1TDQrVmpjTG1xeXRua2lUdHRRUmliUlFTUEluZThwRDIxYi9nVjc3SXlB?=
 =?utf-8?B?UG1vQjNSWTlLYWNLeGN0djBhd0tOc1h3MHMwSThRV0FEZWFmRWZ0USt2ZjBw?=
 =?utf-8?B?S0MvR3JEVkU2QjlUM0xqL0ZYSjVGd0xxK3pxZmtNck9RUnFOOTNTM0x5SGJ2?=
 =?utf-8?B?WmRIY2hsTmllME5LRDBPT1o2S0RMZzljdVdUUDBsRytQbE1HbklQOFVIVnFR?=
 =?utf-8?B?bEZuZyt2d3dzU0VtcDNrSlIwU3AyUU1lL0czQVlmTC9lWXNPRFdXQy9Xd05s?=
 =?utf-8?B?dlU4UTlvTXppcGUzcGxCdGlDL1ZVYzRCRFBmQlZlZG4wRU5lL1pHQVdGd0VL?=
 =?utf-8?B?WVFPcXFqYTJTKzlrb2hkL0J6SDR3a05YR1M3VllQWWlLWDVENkZ5c0NoajVX?=
 =?utf-8?B?cWlmM3VhQzIvaEJMYjc0WlQrT3JKRmY1b3J3TW9CUWpGeHNjUVdlWURMUU5k?=
 =?utf-8?B?eHVmMll1L281RW9IRXZDbHZJOU1GOWl1NU9SWTdWYVpZWUhWUDcycXhoRTV4?=
 =?utf-8?B?UzR2VWN4QU5sRGRNaUZwaTBMV3hBUVJNWFNoYU1ac0xHdFk5VHoyMXllZ29h?=
 =?utf-8?B?RUFIS2pFRDNyN01WeXYwT0RHWnVOWmkvVWV2cTdUM0dLVzNleHBSTXdkbHpq?=
 =?utf-8?B?ZWo4eUlTUjVMU28xTjhBUmlnR1FSdEFQTFR2TkV6bm8rSHc2ckl6YUV6N2Rp?=
 =?utf-8?B?OEhvWE5VWE9oK3pqQXR5cm9PLzlwUk1BU2xWNDl1QzlnTWw5WGJkRXBHZEIv?=
 =?utf-8?B?d0pSeExsdzU0UFU0aTlsdys1eFIySHhnQVBwN1p2L2ZSYUxEeEFkK0FjWVdi?=
 =?utf-8?B?NmpFWlJqMnVUTnlKYXBHR2J1OWZWQUNWS1VoWW1WZVBKRHdpZGtQenplK3dx?=
 =?utf-8?B?NXpuajcweXE4cXRlTGxPcnp0aGg1eGorNmlBMk52ZlMyWVp5MUJ5eThBd1Fw?=
 =?utf-8?B?NktDYk9PdGRaaHEwRi8rZjBaRU1kVTFYS09MV1NObkxZTTN4Y2h6SVpEUlpC?=
 =?utf-8?B?QlZzZWRKRU92VStMTHNDYkNVQXpTODl4R2ZSdkJjZmJ1YUxOWDRHdTBtSGg5?=
 =?utf-8?B?WlpGOXdGdTFLc0RoMmhmZWQ2T0RDZDlkR2JNNGJka215T1VDSVJhK2JSbGJn?=
 =?utf-8?B?OGU3VEVuMzcwWHI3ak52M0s0bTJGb09uRGoxcS9VaytaZ2RPS3Y2U09aREJS?=
 =?utf-8?B?aFZPU3Z0YWhWVDdxd2tXVjk5RlJuNFA4OUtoNVB4bWZZeUkvQVZVcjBVVHov?=
 =?utf-8?B?SHNXOGpuOCs5ZC9Rb2pLbUtSUWc5UDJER1J5c2RjYTdsWTQrTFIvVDQxV29i?=
 =?utf-8?B?MkNqdy9JckNoRTNpbnEzUlJGUTdCMW1TR3d1OUZpREZ5dEcxSDBEdjQwS0Nk?=
 =?utf-8?B?b1BQMGhlUzBycHovN3ZIc1pncEZ2cVRQcWFUTGxoSmt1Z1crRVRUUmdYNnJh?=
 =?utf-8?Q?twns=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7fd28e36-3b5b-4740-ae1e-08dc6df5f896
X-MS-Exchange-CrossTenant-AuthSource: MW3PR12MB4553.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2024 17:57:18.6872
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EpJeS3wjt/JmA4nqZl5tlzJpR54wmTabfDT05fFtOlM4yUdOrybIBJs5YKJeNeVs
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6802

Hi Reinette,

On 5/3/24 18:25, Reinette Chatre wrote:
> Hi Babu,
> 
> On 3/28/2024 6:06 PM, Babu Moger wrote:
>> AMD hardware can support 256 or more RMIDs. However, bandwidth monitoring
>> feature only guarantees that RMIDs currently assigned to a processor will
>> be tracked by hardware. The counters of any other RMIDs which are no longer
>> being tracked will be reset to zero. The MBM event counters return
>> "Unavailable" for the RMIDs that are not active.
> 
> I think it will be helpful to use consistent terms. For example, above uses
> "tracked by hardware" as well as "active". "tracked by hardware" seems more
> specific to me and I think it would help to understand this work if this is
> used consistently.

Sure.

> 
>>
>> Users can create 256 or more monitor groups. But there can be only limited
> 
> I think you write "Users can create 256 or more monitor groups." to match
> with earlier "AMD hardware can support 256 or more RMIDs.". Can this be made
> specific with "Users can create as many monitor groups as RMIDs supported."?

Sure.

> (please feel free to improve)
> 
>> number of groups that can give guaranteed monitoring numbers.  With ever
>> changing configurations there is no way to definitely know which of these
>> groups will be active for certain point of time. Users do not have the
>> option to monitor a group or set of groups for certain period of time
>> without worrying about RMID being reset in between.
>>
>> The ABMC feature provides an option to the user to assign an RMID to the
>> hardware counter and monitor the bandwidth for a longer duration.
>> The assigned RMID will be active until the user unassigns it manually.
>> There is no need to worry about counters being reset during this period.
>> Additionally, the user can specify a bitmask identifying the specific
>> bandwidth types from the given source to track with the counter.
>>
>> Linux resctrl subsystem provides the interface to count maximum of two
>> memory bandwidth events per group, from a combination of available total
>> and local events. Keeping the current interface, users can assign a maximum
>> of 2 ABMC counters per group. User will also have the option to assign only
>> one counter to the group. If the system runs out of assignable ABMC
>> counters, kernel will display an error. Users need to unassign an already
>> assigned counter to make space for new assignments.
>>
>> AMD hardware provides total of 32 ABMC counters when supported.
> 
> I am not sure if you want to mention this. As written this sounds like
> a hardcoded value but it clear from later patches the number of counters
> is learned from hardware.

ok. Will remove it.

> 
>>
>> The feature can be detected via CPUID_Fn80000020_EBX_x00 bit 5.
>> Bits Description
>> 5    ABMC (Assignable Bandwidth Monitoring Counters)
>>
>> The feature details are documented in APM listed below [1].
>> [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming
>> Publication # 24593 Revision 3.41 section 19.3.3.3 Assignable Bandwidth
>> Monitoring (ABMC).
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537
> 
> Reinette
> 

-- 
Thanks
Babu Moger