Received-SPF: pass (google.com: domain of linux-kernel+bounces-83940-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Message-ID: <db9a87b8-40e5-419c-b36e-400f380892a0@amd.com>
Date: Tue, 27 Feb 2024 13:37:26 -0600
User-Agent: Mozilla Thunderbird
Reply-To: babu.moger@amd.com
Subject: Re: [PATCH v2 00/17] x86/resctrl : Support AMD Assignable Bandwidth
 Monitoring Counters (ABMC)
Content-Language: en-US
To: Peter Newman <peternewman@google.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>,
 James Morse <james.morse@arm.com>, corbet@lwn.net, fenghua.yu@intel.com,
 tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
 dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com,
 paulmck@kernel.org, rdunlap@infradead.org, tj@kernel.org,
 peterz@infradead.org, yanjiewtw@gmail.com, kim.phillips@amd.com,
 lukas.bulwahn@gmail.com, seanjc@google.com, jmattson@google.com,
 leitao@debian.org, jpoimboe@kernel.org, rick.p.edgecombe@intel.com,
 kirill.shutemov@linux.intel.com, jithu.joseph@intel.com,
 kai.huang@intel.com, kan.liang@linux.intel.com,
 daniel.sneddon@linux.intel.com, pbonzini@redhat.com, sandipan.das@amd.com,
 ilpo.jarvinen@linux.intel.com, maciej.wieczor-retman@intel.com,
 linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, eranian@google.com
References: <20231201005720.235639-1-babu.moger@amd.com>
 <cover.1705688538.git.babu.moger@amd.com>
 <2f373abf-f0c0-4f5d-9e22-1039a40a57f0@arm.com>
 <474ebe02-2d24-4ce3-b26a-46c520efd453@amd.com>
 <b6bb6a59-67c2-47bc-b8d3-04cf8fd21219@intel.com>
 <3fe3f235-d8a6-453b-b69d-6b7f81c07ae1@amd.com>
 <9b94b97e-4a8c-415e-af7a-d3f832592cf9@intel.com>
 <1ae73c9a-cec4-4496-86c6-3ffcef7940d6@amd.com>
 <32a588e2-7b09-4257-b838-4268583a724d@intel.com>
 <088878bd-7533-492d-838c-6b39a93aad4d@amd.com>
 <CALPaoCgxSAWPYGcmnZZS7M31M+gMJQ-vWd+Q5Zn1Y548bxi2Kw@mail.gmail.com>
From: "Moger, Babu" <babu.moger@amd.com>
In-Reply-To: <CALPaoCgxSAWPYGcmnZZS7M31M+gMJQ-vWd+Q5Zn1Y548bxi2Kw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?QzVjNVpYSHR6WFRlL3FmcjNkTjg0eGxtcThEZStzRk1pTmRpRnFucmtnamN3?=
 =?utf-8?B?SUtaUTdvN1VmNUZsR1FOazJwUlE2VkpjZktiMFl1bDI0MkNLeklLbk5RK01o?=
 =?utf-8?B?NmUrZUVJaTJlZ2xtSXFoNkMvOHhHWHVSYXJvbURXaEdDQldMZzk1aFhGZS90?=
 =?utf-8?B?djQyVWYvNGRDSXg0ejRiRUk2YmErbm5rWnhpQ1BJKzQyZ2g2K1BPa1ZxRjd4?=
 =?utf-8?B?K2FWTmt5TFAyWXhnUmFXSjZWTWJRM2Q0SEJ4WWVBTW9oMVVLWCszM2J1K254?=
 =?utf-8?B?aXIyVGdnSld4eG9pYi9DenFVUk8xaXJ3c3VDUXllalhKa25FSXJzNCtadW5u?=
 =?utf-8?B?dnlibUh0c3REaTV0MzdXS2tOQWprbEtEaTAxNThjSm1sa05JQTZsSzlQUFdT?=
 =?utf-8?B?T1d3SHVPd093aWJjN1N4ZkU1NU1iV2xLazVPNDN1ejAwelBISUlCclh6ZGZZ?=
 =?utf-8?B?L2ppaVo3VXVpWENhSU14TVlvbDlOVllVWWpMYURyYzlIeURBNENNWEpMQ05q?=
 =?utf-8?B?RDRSYVhpWHlPTFgwUGt4V1RNRGNETHd2aU5xQ1JqVDlmWWtuNDlza1k1VUNP?=
 =?utf-8?B?b3REL3JYMGYwYUxSMzRweFluT05HYjZjM01SSUdwRHBPYUxUQXBBTTNNYVlT?=
 =?utf-8?B?UElXWDJOL0RBUEluSm9vSXlZUVdwRUdQK3FPVDArcnFWM2V6QWFTZUtWZlVW?=
 =?utf-8?B?OFF5d2pEMGdqS1ZnQ0FKaWRqRnZQZmJ3eGpEWitJMjNMbGF6eEtnVmhMSUpn?=
 =?utf-8?B?Z0EzOVZKc3hVRGo3Y3BWQ1J4dUpVTWZvWmVLOHNsaGpqMDNDZXRpN3g2dVd1?=
 =?utf-8?B?QmpzQ1JwOVZndjZMTTc5N3hIbm5qYnVYWTQ5TUpoZU4weitqMzlqY004V0hy?=
 =?utf-8?B?cmhEZWg1QkRvYm5TTlA0Q0VSQ1FxdlMzMGlROXBSb0xvRk5Oa2ZLcFpVbmRS?=
 =?utf-8?B?UmZzWElLNTBsSUVOZFFqUnFCaFpzdEJyaUVtTHlxY0E5SEFPR3d5TXFJcDYz?=
 =?utf-8?B?azJDYWVVMUFFQmYwcmJrYWhIYlREL1BndVlxY3JXTk5NczkySzA0UjB6TnYw?=
 =?utf-8?B?ZjVrek9RQkZkVEhpL09tMVdyYjJkc1hBek1WQjNQczlaYVZsMmpIYXB6ZTho?=
 =?utf-8?B?QVZYYWNPN0xJcURXNXhJZXZJV0w4NU5OSm0zNEV6VWNRUi8xU2F0amxiSkdV?=
 =?utf-8?B?VjJwSU00SWRLNkFiZmh2ZGl3bk1IRXJEelQ3S052V3dKZDBpSVUydWljdmJW?=
 =?utf-8?B?ZDQvbHJxcC9QL2FhOHpiaUZ5RHJTUVR6RmxodG1yWDFlQnFiZ1ZFbFhRazhY?=
 =?utf-8?B?ZjBxdzEwQ1pGbWYyL29NU21COUZBSUE1SXR6dm8xek9rOEN0VFAyWW1oS2Fm?=
 =?utf-8?B?QWc5eWdNMUErUmlMTllZMi8rcTh2bjJaZ0VaYjVJSk5sbG0rWitwTS9Ncnlt?=
 =?utf-8?B?b2JpTWFXL3pEK1dQVURpNnA5QTA1Sm5vUG96VEpzSDgvYWVxQ1hKVjV2anZk?=
 =?utf-8?B?TFFGa042dlBsRFNIYVB2K2hwWFVJRmNoR0REL0lUS1FjS1prUFVXR1oyQ0F6?=
 =?utf-8?B?UERNSUxGcG1QQk1BaWcrSzNFVk1adVZaV3VXREFqQ3VMa25jWFRhbzhMcGhM?=
 =?utf-8?B?OWU3SGNxKzdoTjVwNVQ1NE1md1dNdCtBNUkwcEp4NGVmTDd3eFpQVElETGl1?=
 =?utf-8?B?cFdFZE5mQWFGNGtsWjBjWHVKcDJlUVMzOU5QbkNheXlyblZ5NTRGdzNUVXZ6?=
 =?utf-8?B?T3g1STQwQ3Y1eU1kY3ZXMitjTzU0Z3pwcXk3ZTBubGVBQnRJZ2lxTzJIMGFm?=
 =?utf-8?B?Q1RuVlBPUFI2ejcwM29Td1BjeUdBV21qQWFQRzVwb240alZrYzRwVENabWNl?=
 =?utf-8?B?YVhIS3Jpd2ZESjhmWUthM3JBSFFNbTMwMW9iZ0ZNRlAzZzVVaVQ4WU9sdy9S?=
 =?utf-8?B?WXdiVm92MXpiaEd0aTJEWmw1QkowcEVibFZLNHZnVndLbHZWalp6SlNsYjJl?=
 =?utf-8?B?U3N2Mjg2OHRRbjh3ZENtL2NoVVBQS2JDTXhHM1luTFc2d20xN0taMTkwYkRI?=
 =?utf-8?B?VlBBRlJldTg0cTFGRXdnR3lDSnpkM3c4cFppOVlXbElzRjBRa05tME9kaVJW?=
 =?utf-8?Q?S8g4=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1d4e09c6-8c7f-4b3a-a737-08dc37cb89c5
X-MS-Exchange-CrossTenant-AuthSource: MW3PR12MB4553.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2024 19:37:31.1735
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: la7Nl3YS9MfTM+/h24g8uosrMFHjIlxuK5TpT0qCPWcTcYDZIvTXf+CJaovM+ss5
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB8772

Hi Peter,

On 2/27/24 12:26, Peter Newman wrote:
> Hi Babu,
> 
> On Tue, Feb 27, 2024 at 10:12 AM Moger, Babu <babu.moger@amd.com> wrote:
>>
>> On 2/26/24 15:20, Reinette Chatre wrote:
>>>
>>> For example, if I understand correctly, theoretically, when ABMC is enabled then
>>> "num_rmids" can be U32_MAX (after a quick look it is not clear to me why r->num_rmid
>>> is not unsigned, tbd if number of directories may also be limited by kernfs).
>>> User space could theoretically create more monitor groups than the number of
>>> rmids that a resource claims to support using current upstream enumeration.
>>
>> CPU or task association still uses PQR_ASSOC(MSR C8Fh). There are only 11
>> bits(depends on specific h/w) to represent RMIDs. So, we cannot create
>> more than this limit(r->num_rmid).
>>
>> In case of ABMC, h/w uses another counter(mbm_assignable_counters) with
>> RMID to assign the monitoring. So, assignment limit is
>> mbm_assignable_counters. The number of mon groups limit is still r->num_rmid.
> 
> That is not entirely true. As long as you don't need to maintain
> bandwidth counts for unassigned monitoring groups, there's no need to
> allocate a HW RMID to a monitoring group.

We don't need to allocate a h/w counter for unassigned group.
My proposal is to allocate h/w counter only if user requests a assignment.
 The limit for assigned events at time is mbm_assignable_counters(32 right
now).

> 
> In my soft-ABMC prototype, where a limited number of HW RMIDs are
> allocated to assigned monitoring groups, I was forced to replace the
> HW RMID value stored in the task_struct to a pointer to the struct
> mongroup, since the RMID value assigned to the mongroup would
> frequently change, resulting in excessive walks down the tasklist to
> find all of the tasks using the previous value.
> 
> However, the number of hardware monitor group identifiers supported
> (i.e., RMID, PARTID:PMG) is usually high enough that I don't think
> there's much motivation to support unlimited monitoring groups. In
> both soft-RMID and soft-ABMC, I didn't bother supporting more groups
> than num_rmids, because the number was large enough.

What is soft-ABMC?

-- 
Thanks
Babu Moger