Received-SPF: pass (google.com: domain of linux-kernel+bounces-172123-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <b247fb17-7234-4b7d-9906-f23689dd5b50@amd.com>
Date: Tue, 7 May 2024 15:03:26 -0500
User-Agent: Mozilla Thunderbird
Reply-To: babu.moger@amd.com
Subject: Re: [RFC PATCH v3 08/17] x86/resctrl: Initialize assignable counters
 bitmap
To: Reinette Chatre <reinette.chatre@intel.com>, corbet@lwn.net,
 fenghua.yu@intel.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
 dave.hansen@linux.intel.com
Cc: x86@kernel.org, hpa@zytor.com, paulmck@kernel.org, rdunlap@infradead.org,
 tj@kernel.org, peterz@infradead.org, yanjiewtw@gmail.com,
 kim.phillips@amd.com, lukas.bulwahn@gmail.com, seanjc@google.com,
 jmattson@google.com, leitao@debian.org, jpoimboe@kernel.org,
 rick.p.edgecombe@intel.com, kirill.shutemov@linux.intel.com,
 jithu.joseph@intel.com, kai.huang@intel.com, kan.liang@linux.intel.com,
 daniel.sneddon@linux.intel.com, pbonzini@redhat.com, sandipan.das@amd.com,
 ilpo.jarvinen@linux.intel.com, peternewman@google.com,
 maciej.wieczor-retman@intel.com, linux-doc@vger.kernel.org,
 linux-kernel@vger.kernel.org, eranian@google.com, james.morse@arm.com
References: <cover.1711674410.git.babu.moger@amd.com>
 <d9fe6638d901791dfdec26caed034179c636e17b.1711674410.git.babu.moger@amd.com>
 <aec3fdfa-ad57-402b-9c21-555b5ddc85f9@intel.com>
Content-Language: en-US
From: "Moger, Babu" <babu.moger@amd.com>
In-Reply-To: <aec3fdfa-ad57-402b-9c21-555b5ddc85f9@intel.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aXRLa0pBYlhJZ0VoNjJFWk9JQlRIQzdGU05JVkZoenJONHVTbjF6UUlNaysr?=
 =?utf-8?B?bGNpemJEd2d4RHh6N3dHcmVTU1BsbG1XNnFlSGNEa0xKMWlXMjg5RDREdmlM?=
 =?utf-8?B?bTRRVms3eHNLUHlTbzZSMnZDejhMNWZZUXVjem94N2M5QURkbm1wREZQWE5a?=
 =?utf-8?B?R29JQTFmdzZLRXVQUW9sa3hUYXRFSTRsUVJIdnNkaXJqcXAvdTlIdFMzTk9Y?=
 =?utf-8?B?WFc4TE12MkROU1djOURkTStHUzdDYUpxWjNCVkRCczd3c05ET2Q1UWRIWFRZ?=
 =?utf-8?B?SDBHeXN5Wk95ZCs1bzNpZ2dDZHlGc3BNcUlmUXRhbDVCUDArNmZyaTRPNEZN?=
 =?utf-8?B?bVBpUEhzaHFXVHJFaGErZWpWUFJlTnllMTgrSllubEdLeFNCTUF6OFArK083?=
 =?utf-8?B?UXV3enNKUHlVUTY3RkVQRkQwWnF5cThXT05jeEVtVUlSN3BON05INndZNzVQ?=
 =?utf-8?B?cE9xaE1hM0VPU2dFNTYraXBVV0hmUmNoNzBDenJSQlNvenFNR0xLS2RSbThu?=
 =?utf-8?B?cDF0VnFPQTJ1UTFIU2crRkNyQUl4NEhWaitqazZzUWRIbkNyWUFEM0sxVEE0?=
 =?utf-8?B?dEZLenRIZFcvNnFENmRDVDBDMGVKaElCNnlCN0l3WS8vNENOWEVmb2MrclEw?=
 =?utf-8?B?eEZ5eVlWVW15ZXh3R0ZxblBCT2xaMVoyN21JTUxqYWNrNEVwTWtUTEo2Tm5u?=
 =?utf-8?B?Zm5lQm1lTzhlODJzaG5FOXp1aStVaHY4eUdnNG9TZWptd3kvQkx0TjZxTDg5?=
 =?utf-8?B?UnNiNkdkaWlNeTlZY0RDaExjQ1hNMEZRYnNRaEdpQXZ0OURUMjdja0JoZzN6?=
 =?utf-8?B?UGhWM3gyRk9GL2hRVTVrSzBVaUNMcTU1bURPOVdXN1VsdHpYT0UvMWFFY0Vi?=
 =?utf-8?B?WFhBRnBnalM2OGJCdGJXdlRUNU5LYi8yVjVGbFZUMm1KMGdibnRWbFljUnpv?=
 =?utf-8?B?TDkrSEtUemF0N012S2Q3ZmI3MUdYaFk0VlZBT28zbVQ2QnFkWnViYTRwTVdC?=
 =?utf-8?B?WWpGaDVsNjFKNE9PYjhoQWNWb2ZmV2hzQjE0eVlJeW0rN2dSOEFwejRnZStE?=
 =?utf-8?B?azgwZ2pmZGt6azE3bjZDbW5DWnBjL1ZpZnEyYmNwRG1Vd3ZCT2VpUGZBMmZ2?=
 =?utf-8?B?Rk02Ykc4TmR4RGVtRTIyWnAxWUMrQ3JuU0hNOGoya1Q5TlM1cURVK3pTRnBY?=
 =?utf-8?B?b2FxbGxCclhNdlp6V3ViQnUvOGJ0Ulk0RVkyZkVwOU83UVdjODVYRFV0aXhF?=
 =?utf-8?B?MEkrZnVpckluWTZ1TkZKZElQSW5HSkhmczVlNXJsYVAyUUUzWnpOTURoRG55?=
 =?utf-8?B?aXFnRVJIalQ1b1V2Q2FkOW9tOFBidXoxeDFjY0ZVSWpmUHpDcm9qcmpQdFQw?=
 =?utf-8?B?aTZUS1FUdmJmRG9TSzBtOTRHUlI1VU1NZzg0Mm5hNll1MUFRemU3U2dkTmVa?=
 =?utf-8?B?NVlzODRUWWU3a2xBVHBYZVlOYlcwaVJLVktZb0tBNGtnbTZBQVJaWDVPbnBO?=
 =?utf-8?B?cER0a1JyQmdWbzJZQnlSZ3ZaZjNLaFA1dzhNRzBacC81bDFGZGFrTHYzYWg0?=
 =?utf-8?B?RWN0Mjc2cFlrTXhsYzRIUit6TzRya2U4amZGVU4vY2JpY1pmdkZwNEo0c2Ir?=
 =?utf-8?B?dG5mUDZqbXVnMDBJRjJsdVFnb01tU1Q0Um1jRGt1cDNJVTVsQk1FbnlUc1Vn?=
 =?utf-8?B?c0JrMktad1Z1WTg0MFBjaXBJZ25Wb3dUSmJ6cU1nNU44azQzZWRWU0FlSXVa?=
 =?utf-8?B?OXlkV2c3NTB6U3ZmT2NHUDU4NGRyNnR3RDhFNXdiT2t2N1M0dFdKM3hYUGda?=
 =?utf-8?B?NzJ1SW5icjQrV3A0cWJRU3ltSXF0VTE2UWFpeFZmSm9Beis4N1FJUGc5VEFl?=
 =?utf-8?B?MTdUWGlQUE90djBKeE54RXRmWkZjbkd6VjJNVEJrY3pxYmw5KzVPSk9VM3Nm?=
 =?utf-8?B?Z04xRTN5SFg2WDNqb0R6ak1pV0s2dmlNaWo0UTZ2dVNpRHIvdXEzbVMrY0Y2?=
 =?utf-8?B?bXRXY0tuOVFBSTFIVmxzUy9rY1UzTnAzckU5NzNOVWgwNmtkZFR3OFZ2Mnk3?=
 =?utf-8?B?WUt4eld1UDh5ZisrNEpYYUFmSWFLU2NxYnJkdTRGOU9GWUgvTmJWZ1RMckV1?=
 =?utf-8?Q?85vY=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4739163e-bd27-4e3e-96b8-08dc6ed0c46e
X-MS-Exchange-CrossTenant-AuthSource: MW3PR12MB4553.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2024 20:03:30.8991
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MXEqs1AgKBRzoSnw1syowPcGsJqISlh6Q/+4xTcxAnUWrAI6KPP7Vl8pqwIsrtSK
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7801

Hi Reinette,

On 5/3/24 18:31, Reinette Chatre wrote:
> Hi Babu,
> 
> On 3/28/2024 6:06 PM, Babu Moger wrote:
>> AMD Hardware provides a set of counters when the ABMC feature is supported.
>> These counters are used for assigning events to the resctrl group.
>>
>> Introduce the bitmap assign_cntrs_free_map to allocate and free the
>> counters.
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>>
>> ---
>> v3: Changed the bitmap name to assign_cntrs_free_map. Removed abmc
>>     from the name.
>>
>> v2: Changed the bitmap name to assignable_counter_free_map from
>>     abmc_counter_free_map.
>> ---
>>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 +++++++++++++++++++
>>  1 file changed, 19 insertions(+)
>>
>> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> index f49073c86884..2c7583e7b541 100644
>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> @@ -186,6 +186,22 @@ bool closid_allocated(unsigned int closid)
>>  	return !test_bit(closid, &closid_free_map);
>>  }
>>  
>> +static u64 assign_cntrs_free_map;
>> +static u32 assign_cntrs_free_map_len;
> 
> Please provide summary in comments about what these globals are and how they
> are used.
> 

Sure.

>> +
>> +static void assign_cntrs_init(void)
>> +{
>> +	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
>> +
>> +	if (r->mbm_assign_cntrs > 64) {
>> +		r->mbm_assign_cntrs = 64;
>> +		WARN(1, "Cannot support more than 64 Assignable counters\n");
> 
> I am a bit confused here. The configuration registers are introduced in patch #10
> and if I counted right there are 5 bits for the counter id. It thus seems to me
> as though there needs to be some checking during enumeration time to ensure
> that all counters enumerated can be configured.

Ok. I will remove this check here and move it to enumeration time in
rdt_get_mon_l3_config.

> 
>> +	}
>> +
>> +	assign_cntrs_free_map = BIT_MASK(r->mbm_assign_cntrs) - 1;
> 
> Please use bitmap API. For example, bitmap_fill()

Ok. Sure.

> 
>> +	assign_cntrs_free_map_len = r->mbm_assign_cntrs;
>> +}
>> +
>>  /**
>>   * rdtgroup_mode_by_closid - Return mode of resource group with closid
>>   * @closid: closid if the resource group
>> @@ -2459,6 +2475,9 @@ static int resctrl_abmc_setup(enum resctrl_res_level l, bool enable)
>>  	struct rdt_resource *r = &rdt_resources_all[l].r_resctrl;
>>  	struct rdt_domain *d;
>>  
>> +	/* Reset the counters bitmap */
>> +	assign_cntrs_init();
>> +
> 
> (At this point it is unclear when resctrl_abmc_setup() is called to understand
> if reset of bitmap may be appropriate. Please do expand all changelogs to help
> readers along with how this implementation is intended to work.)

resctrl_abmc_setup() is called either while enabling or disabling the ABMC
feature. In both these cases the bitmap assign_cntrs_free_map needs to be
reset.

Will add comments here,


> 
>>  	/* Update QOS_CFG MSR on all the CPUs in cpu_mask */
>>  	list_for_each_entry(d, &r->domains, list) {
>>  		on_each_cpu_mask(&d->cpu_mask, resctrl_abmc_msrwrite, &enable, 1);
> 
> Reinette
> 

-- 
Thanks
Babu Moger