Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3;
Date:   Thu, 28 Sep 2023 09:06:28 +0200
From:   Maciej =?utf-8?Q?Wiecz=C3=B3r-Retman?= 
        <maciej.wieczor-retman@intel.com>
To:     <babu.moger@amd.com>
CC:     Fenghua Yu <fenghua.yu@intel.com>,
        Reinette Chatre <reinette.chatre@intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        "Ingo Molnar" <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, <x86@kernel.org>,
        "H. Peter Anvin" <hpa@zytor.com>, <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2 1/4] x86/resctrl: Enable non-contiguous bits in Intel
 CAT
Message-ID: <4sxjxe4cgvookcobwbrmsoeszjordi3bpjg22zxut3lanjc46j@xpqt6nblx6uc>
References: <cover.1695371055.git.maciej.wieczor-retman@intel.com>
 <918e147601697b1d4b8f8589f5751e05d6ceccdf.1695371055.git.maciej.wieczor-retman@intel.com>
 <9751cbaf-8653-d32a-6058-13c251c59b3d@amd.com>
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <9751cbaf-8653-d32a-6058-13c251c59b3d@amd.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?ichJWf5aI0GJJlSUCW5eQvR5M+IZNdFKWYJLv/pJkAuYrWurF03s+IF7FD?=
 =?iso-8859-1?Q?d03zAaDOZB+ET0PiBX2D8f/4Hn3yBjgJ268pzEYlwqaZOXvGagCxtZgGNL?=
 =?iso-8859-1?Q?06gmiksmxfEzje3ZLth/17nmpBx6ppO4cGFYsjUEBJEcQTsLZwQ8+KjcyY?=
 =?iso-8859-1?Q?wzvBU5IbQKocI896h1E8VueKzdes50YyWSY61GjweRmteY9JwrZAeXCdQS?=
 =?iso-8859-1?Q?8oowqdwftb6KNGwzlXikPOSPvLfvW2OlE4GQqmcDMBIk9FI64pHh31IeBS?=
 =?iso-8859-1?Q?HDG49HgmiLrsA2lw39Z1RVewYTAqad2gNDLihfsN32jOifYAFbEtwou9rU?=
 =?iso-8859-1?Q?ilWNFRaNrZ3+97dK9ip3a1kAoHluEMVA7D1XVpeBZiwnp/o0jVWwaImpcL?=
 =?iso-8859-1?Q?YoyelLndoFK3lCh3XlHwwOMMppLoPaEYmE4ddN5MC2K/Qlv6BbYkmyUkrh?=
 =?iso-8859-1?Q?DJXSZQghEJEOJbpP78DIoY4PuH3ryA/x3oukPLbayDmQIpR3LA6IMGWAqC?=
 =?iso-8859-1?Q?S1a0kAaHo36vdfrpJew+65jM+qWfPO6yPsSDkPOncuFP7xNotUN07SlDh+?=
 =?iso-8859-1?Q?/vTUb5qYUs1uq77cz7hqZar40zQcuIwbI8SH7Z/Fg3lXxg2rR/upCIDNva?=
 =?iso-8859-1?Q?Cau6TY6D5yMj09ED+o50KzF0uQDRfZDP0GwZpld/sWLXmworekNZr+zv8o?=
 =?iso-8859-1?Q?7dBwaInkZ9PuLycma6evkmcWWE+JlgMVn7iwL0xStUgEkEu0gs4VTvIZfN?=
 =?iso-8859-1?Q?gbbRCGqKypOJ+7AYuvs+s0vqzvODt2LfUa14C0XulYNmcyqBQV+D3cwa0w?=
 =?iso-8859-1?Q?4w09hLnmcNc6LYqGrxUxax11lbtaCqGsR+JJ506O0rmfGQ77C1Ch6er3Yl?=
 =?iso-8859-1?Q?zK7a5FVxxL7OBi/czDskA7l3fUIO4o69aw4sj0ZjYzyQhe9PFtSax1V/QL?=
 =?iso-8859-1?Q?ZM9MiUHc6mV5MK0WwkOIIruvL0Y2KCdBV5iSPPgjrsRagC06fkvaVRnCBt?=
 =?iso-8859-1?Q?ajQ1YDRESUFTPGrh2OeFwzeFRk0QY2MVt5wRwpV3nhL8OBfW+w/qmgRwnU?=
 =?iso-8859-1?Q?Mar4/bjrRIvbG789jsDJ33WdEl5Qn5xxlBtaiu3V96tAZoS2LuJY/tvMtx?=
 =?iso-8859-1?Q?VIj05c1hkDstb3IXPnPQjL2qhf0cvbQTrrtFdBwegcUb1aK6z8pQ7ggJ5M?=
 =?iso-8859-1?Q?KGxG+N2UVNsGo1k1P/gWah2dbsh2W7whFLI9HRkcn5zFwM7qTgRA0SP8kG?=
 =?iso-8859-1?Q?FSSvHb6G8YX3PxlWN1W5Sf5zsRiXnRDaYSNPVY5aeNFVBAdwCQZmdLmg0n?=
 =?iso-8859-1?Q?V+jYcTmgCRrChlZ0gKk6b8KuCpMqv9itGJWeDXR1HbV6QXbTic5PskcnVB?=
 =?iso-8859-1?Q?s34EQgLvOSfF9gOx4gthOQfPgAk49BMv8nQo53AaGd6iKftrrN1e5+dbyv?=
 =?iso-8859-1?Q?MebaIPA6VnfqsdgKIjIja338lArGGmsFugmPScjN7+UnJplBCOEFyRFkqM?=
 =?iso-8859-1?Q?WrcVN2v7zBAcPfU+x1DNsYV/3xBMp2KNW3SRvksdr5/tyXcXwN4R9c5JCd?=
 =?iso-8859-1?Q?6gIsJ5kOqwWZjE1fv87J4ZvoJzAjR9ubLlzhX06KcIWPVN99fmKNoJysQ4?=
 =?iso-8859-1?Q?tJTeFAMgBA0MaAYgDLAWKCbnCNY9rTu5Ml8kYeo3xnIjP/AXpXsIaRP045?=
 =?iso-8859-1?Q?Sq0ZAl0KuZwbTEuAIYM=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 8030c7e5-2c04-48c4-5b89-08dbbff172d8
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6231.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2023 07:06:34.0581
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DQ0SUJKc6BTPSXSeyR9IQT254uc5YBUutoT8rdR5eqG9opohvAUOk5DjgYUTi4yUbKXu7gviqnep3e/3oBC+kkd6DbSwbp9C7uB/6+RvLho=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB4770
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Thu, 28 Sep 2023 00:07:00 -0700 (PDT)

Hi, thanks for reviewing the series,

On 2023-09-27 at 17:34:27 -0500, Moger, Babu wrote:
>Hi Maciej,
>
>How about this subject line?
>
>x86/resctrl: Enable non-contiguous CBMs on Intel CAT

Changing "bits" to "CBMs" does indeed seem sensible so I'll do that if
there are no objections.

But I'm not sure the preposition collocation change from "in" to "on"
would be grammatical (at least from what I've read in docs about Intel
CAT so far).

>On 9/22/2023 3:48 AM, Maciej Wieczor-Retman wrote:
>> The setting for non-contiguous 1s support in Intel CAT is
>
>> hardcoded to false. On these systems, writing non-contiguous
>> 1s into the schemata file will fail before resctrl passes
>> the value to the hardware.
>> 
>> In Intel CAT CPUID.0x10.1:ECX[3] and CPUID.0x10.2:ECX[3] stopped
>> being reserved and now carry information about non-contiguous 1s
>> value support for L3 and L2 cache respectively. The CAT
>> capacity bitmask (CBM) supports a non-contiguous 1s value if
>> the bit is set.
>> 
>> Replace the hardcoded non-contiguous support value with
>> the support learned from the hardware. Add hardcoded non-contiguous
>> support value to Haswell probe since it can't make use of CPUID for
>> Cache allocation.
>> 
>> Originally-by: Fenghua Yu <fenghua.yu@intel.com>
>> Signed-off-by: Maciej Wieczor-Retman <maciej.wieczor-retman@intel.com>
>> ---
>> Changelog v2:
>> - Rewrite part of a comment concerning Haswell. (Reinette)
>> 
>>   arch/x86/kernel/cpu/resctrl/core.c        |  9 ++++++---
>>   arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 10 ++++++----
>>   arch/x86/kernel/cpu/resctrl/internal.h    |  9 +++++++++
>>   3 files changed, 21 insertions(+), 7 deletions(-)
>> 
>> diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
>> index 030d3b409768..c783a873147c 100644
>> --- a/arch/x86/kernel/cpu/resctrl/core.c
>> +++ b/arch/x86/kernel/cpu/resctrl/core.c
>> @@ -152,6 +152,7 @@ static inline void cache_alloc_hsw_probe(void)
>>   	r->cache.cbm_len = 20;
>>   	r->cache.shareable_bits = 0xc0000;
>>   	r->cache.min_cbm_bits = 2;
>> +	r->cache.arch_has_sparse_bitmaps = false;
>
>Is this change required?
>
>This is always set to false in rdt_init_res_defs_intel().

The logic behind moving this variable initialization from
rdt_init_res_defs_intel() into both cache_alloc_hsw_probe() and
rdt_get_cache_alloc_cfg() is that the variable doesn't really have a
default value anymore. It used to when the CPUID.0x10.1:ECX[3] and
CPUID.0x10.2:ECX[3] bits were reserved.

Now for the general case the variable is dependent on CPUID output.
And only for Haswell case it needs to be hardcoded to "false", so the
assignment makes more sense in Haswell probe rather than in the default
section.

>>   	r->alloc_capable = true;
>>   	rdt_alloc_capable = true;
>> @@ -267,15 +268,18 @@ static void rdt_get_cache_alloc_cfg(int idx, struct rdt_resource *r)
>>   {
>>   	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
>>   	union cpuid_0x10_1_eax eax;
>> +	union cpuid_0x10_x_ecx ecx;
>>   	union cpuid_0x10_x_edx edx;
>> -	u32 ebx, ecx;
>> +	u32 ebx;
>> -	cpuid_count(0x00000010, idx, &eax.full, &ebx, &ecx, &edx.full);
>> +	cpuid_count(0x00000010, idx, &eax.full, &ebx, &ecx.full, &edx.full);
>>   	hw_res->num_closid = edx.split.cos_max + 1;
>>   	r->cache.cbm_len = eax.split.cbm_len + 1;
>>   	r->default_ctrl = BIT_MASK(eax.split.cbm_len + 1) - 1;
>>   	r->cache.shareable_bits = ebx & r->default_ctrl;
>>   	r->data_width = (r->cache.cbm_len + 3) / 4;
>> +	if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL)
>> +		r->cache.arch_has_sparse_bitmaps = ecx.split.noncont;
>>   	r->alloc_capable = true;
>>   }
>> @@ -872,7 +876,6 @@ static __init void rdt_init_res_defs_intel(void)
>>   		if (r->rid == RDT_RESOURCE_L3 ||
>>   		    r->rid == RDT_RESOURCE_L2) {
>> -			r->cache.arch_has_sparse_bitmaps = false;
>
>Why do you have to remove this one here??? This seems like a right place to
>initialize.

Look at the previous comment.

-- 
Kind regards
Maciej Wiecz?r-Retman