Received-SPF: pass (google.com: domain of linux-kernel+bounces-196053-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <992cd5db-1cc8-4238-879b-8ef0406edc33@intel.com>
Date: Thu, 30 May 2024 16:51:33 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/2] selftests/resctrl: Adjust effective L3 cache size
 with SNC enabled
To: "Luck, Tony" <tony.luck@intel.com>, "Wieczor-Retman, Maciej"
	<maciej.wieczor-retman@intel.com>, "Yu, Fenghua" <fenghua.yu@intel.com>,
	"shuah@kernel.org" <shuah@kernel.org>
CC: "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"ilpo.jarvinen@linux.intel.com" <ilpo.jarvinen@linux.intel.com>
References: <cover.1715769576.git.maciej.wieczor-retman@intel.com>
 <fe9295c6be677d187b1607185e23993dbfe74761.1715769576.git.maciej.wieczor-retman@intel.com>
 <9fa47acf-86b1-4602-8790-39ed80fd775a@intel.com>
 <SJ1PR11MB60833CF8BD209555A5370299FCF32@SJ1PR11MB6083.namprd11.prod.outlook.com>
Content-Language: en-US
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <SJ1PR11MB60833CF8BD209555A5370299FCF32@SJ1PR11MB6083.namprd11.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZmI4ZThSZkNsRitsODdselFlNXIwK3dTK0xvNEVyU2xqMU12c2J2L2Q3OUl2?=
 =?utf-8?B?dnVxdnZxYjFJZFdvaGY3aFlXbzk1eXl6S05lMHBpaVZYM1YwTDIzQkFwM1Bx?=
 =?utf-8?B?NWUyUTQrK1R3YThrMzdadkZwMmFJT0c1VXRrWUMyQVY2amZ5UzRXUlZ1b3pY?=
 =?utf-8?B?YjlmQjBmY2RuTnBpUGs3R1orYzFYQnNrRkswZVNDSjVaUHVnTytyUk5rU1ph?=
 =?utf-8?B?RWtPVUIxaG1YQ0xmY1BQQjBaRXlzb3JmbzhCd1RtT1hBdUFtYWNTVG9pVWI0?=
 =?utf-8?B?U3o2cEFqNzVPTEYwb2o5elhMcGlFdmdab2d4Q1plcTFOOGV0L2cxem9aZ0Zu?=
 =?utf-8?B?MGNTVmNhLzgvVjYyMWN6T2RhYUtzREVQdkMrbEpFVUpIdXozVjN2OWExOW9B?=
 =?utf-8?B?eDJTeG4yTThCaGQ5WVhmRElNTzlTV29FMjFQNzNGU0tDOXRqRVVWMVJwTTNP?=
 =?utf-8?B?RndHZW8vaENPSEdzMExKR2pnc0N0KzVXRXhSalQrZUQ0YzlvNDlSdU0zSGJj?=
 =?utf-8?B?OE5rb2IrWk1XTzY3WlBXZFUvcnR6UjB4cDN6QXBzSFlYNmhQSGdWdjRDREFH?=
 =?utf-8?B?WC91TXpSdzJFMDV5OFJ0WWNnaFE3TExDb2V3bmJtdWJUb2tCcit2aUdwZy9G?=
 =?utf-8?B?OU5xYWdlVi9wV0k0aXB5OFpHRFhnU1BRdzNlQUdCTHYvY0JDMEVLa3ZoNDlu?=
 =?utf-8?B?alN3aHp3SUMwdGJueURGbS9KenRNa3U2Z1h5L3lWbFF2UFVsWmdPM3BGeS9z?=
 =?utf-8?B?SHNpT2dVVDZWNGk0clJhNUlDQkNZcUErT281VkE3cmJpcnNPQ3Eva3FBWlRW?=
 =?utf-8?B?bVZRVGtVZHpldEhrTzkzM0psUXBvTCtpK2lxMzEyTGdnVzhIb3g4Qm9ZalB0?=
 =?utf-8?B?RjVkZGgrTkJHTENlMEZoZlV4MnY2OVZOTEJ6dlZKYzFVYjVFTE5nMEJFVDNs?=
 =?utf-8?B?ZjVQUFZ6OHVRc0JXaHluVzY4UUtQN0RkTjZmWE5IUERWTkJJRlM1UHVRTUVY?=
 =?utf-8?B?MVZNR09rM1hraW9lL1pIN0EwZDZncjVMSXRXR1VoZ1RpUi9sZ2pCSGRKNW4w?=
 =?utf-8?B?UUJXelJJT0IrZC9jaGRqSTJoMkpkWlVydTVjRjY1RG9GRWMvTi9XOFFxTXlG?=
 =?utf-8?B?QmozRUZyUjd2ZmIzWmhvV29NU0NkcE0zQ0F1ZmQzZTdBcGtvbmNIRVYxVE05?=
 =?utf-8?B?RHdQZjFCYWF2bWppaTNZMkMrU2szeVdNeWx3bWFSdEdmN3N4NXp1ajZESUZY?=
 =?utf-8?B?RVR3ZlJoRGdNcFNiVVhXbk95TFc0UExRc0w3S0gvaWgvRzB0KzFGbHJxcWE4?=
 =?utf-8?B?aVFicXdPSVVzT2c0cjhtV2lxMFd6Tm9zb3NGNjl6TGVLRHdzenZvVTM0cm53?=
 =?utf-8?B?Y1lMc2FQQm5UVFR5UlhhUlZpSkE5NGtsTCt3T0NSaHFLRE1ZYk1JQTU5Kzhr?=
 =?utf-8?B?R20rVnZJL0tTRHYrRGtQRGZobmR3aGdFQnJiZEVxRFM5ZmVjQXB2YnR6Z2pa?=
 =?utf-8?B?UlhyWkJ0VlJmVktpU2dqeTlYYVFkM3BiaHo1SHlHYTlIZnRidFpGQU53ZGJm?=
 =?utf-8?B?STJkeDFKTXQ2b280QSthOUd4bTJkNUJhMThraTRRUEZQYVdqRUt1Wkw5Q2Js?=
 =?utf-8?B?UTZwNGxqSnd3ejVSckM0RGRSWmZ0SGhXYTJJZGNEMHRjUjl5SXZGdnRwbUZa?=
 =?utf-8?B?Qk5hbUR1VmJPZ1lld3gzcWZYbnlqRXErcGVjcHk5d1ozMzdaTDBBRjFjdDd2?=
 =?utf-8?B?dmx5MEdIZjhhcjZORVFxTFFNSFNORGlwTTI2M3F4TmJkSlJXcWI1SVNwTGU1?=
 =?utf-8?B?c1ZaNWZPRjVFcEltOXhQaHBTN043MlBjd05sWEx3MVl1NkxnQ3h5UkRCdnBi?=
 =?utf-8?B?M0pjaWF4WXAweTFuU25waDNrTnlwaFdhcjRVVlZMUE1pbS9qOUJIY01jZHZy?=
 =?utf-8?B?RHY1dm1ubzJkUldIV3hYbmd6M2ozcUU4c3dRNkZ5dzFTdW1UdGZmdGhoL05S?=
 =?utf-8?B?a3RlZ0dZajAyOGFVREtLUWF6OU9OeHA1Y0Q3RnVYUFJmYSthS3NvSXVmYjZF?=
 =?utf-8?B?VmNrL0RiY2d4QjdOY0NNTDF5VkJIV1cxanFoNmtpc1JpTFJzaWFOOXFJL012?=
 =?utf-8?B?UVJjT20wRkFKZkdmaUdvbG8vZE5xUzA1aDVZN01vSFRlOHZOSW1wUzA1emZ1?=
 =?utf-8?B?SFE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 74e4629d-8ed7-45a5-bef3-08dc81037056
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2024 23:51:35.0895
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1SebOemQjMg3VG5XTeI8eLzYJog0Sm+rUgLFY1ON/bfLeNACJoPUhlezwo4hGG6cRWK11N5FlzVjva59EoiFpcY+PdV1zt0yHWp577uYEyA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR11MB8498
X-OriginatorOrg: intel.com

Hi Tony,

On 5/30/24 4:46 PM, Luck, Tony wrote:
>>> When SNC mode is enabled the effective amount of L3 cache available
>>> for allocation is divided by the number of nodes per L3.
>>
>> This was a mistake in original implementation and no longer done.
> 
> My original kernel code adjusted value reported in the "size" file in resctrl.
> That's no longer done because the effective size depends on how applications
> are allocating and using memory. Since the kernel can't know that, it
> seemed best to just report the total size of the cache.
> 
> But I think the resctrl tests still need to take this into account when running
> llc_occupancy tests.
> 
> E.g. on a 2-way SNC system with a 100MB L3 cache a test that allocates
> memory from its local SNC node (default behavior without using libnuma)
> will only see 50 MB llc_occupancy with a fully populated L3 mask in the
> schemata file.

This seems to contradict the "Cache and memory bandwidth allocation features
continue to operate at the scope of the L3 cache." statement from [1]?

Reinette

[1] https://lore.kernel.org/lkml/20240528222006.58283-1-tony.luck@intel.com/