Received-SPF: pass (google.com: domain of linux-kernel+bounces-215291-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <f57e9cf2-35b6-401d-afc2-8d11b22836c2@intel.com>
Date: Fri, 14 Jun 2024 09:46:25 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v20 00/18] Add support for Sub-NUMA cluster (SNC) systems
To: <babu.moger@amd.com>, Tony Luck <tony.luck@intel.com>, Fenghua Yu
	<fenghua.yu@intel.com>, Maciej Wieczor-Retman
	<maciej.wieczor-retman@intel.com>, Peter Newman <peternewman@google.com>,
	James Morse <james.morse@arm.com>, Drew Fustini <dfustini@baylibre.com>,
	"Dave Martin" <Dave.Martin@arm.com>
CC: <x86@kernel.org>, <linux-kernel@vger.kernel.org>,
	<patches@lists.linux.dev>
References: <20240610183528.349198-1-tony.luck@intel.com>
 <658d9869-ef22-48a7-876a-5bbba4f134ff@amd.com>
 <bc52a051-4296-48ac-9a83-29139855553f@intel.com>
 <e17d3a79-9c07-44cd-b0ea-2d0545680342@amd.com>
From: Reinette Chatre <reinette.chatre@intel.com>
Content-Language: en-US
In-Reply-To: <e17d3a79-9c07-44cd-b0ea-2d0545680342@amd.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bEhkMmF5bDN4WWpEdjZ3Z3NWQlFBQWY3YWgzTURxRWg1d01IUjJvWmFJdDBH?=
 =?utf-8?B?L252b1Bma28rc2FYT1dGREVjTEF0MjlFU1AwUTVDdkMwcUdVdDlXY1NNTE9n?=
 =?utf-8?B?UE5jSFg0bTlKTmhtM1V4Nnl3ODk0cDZWV1I4U0wrSE82M2w5bGxyUDRIeWRI?=
 =?utf-8?B?djd3RlN2L05IMjJWalozZ1pRdGlYK3M1dG15dXFOVTg3Y0o4UnZKa0JvMFhr?=
 =?utf-8?B?VEJqUU1ERExFL24wN1ZNSzZsVnBsZjBwRDVDelN0QXQ5MDZPMU5RK0dWcjcw?=
 =?utf-8?B?RjlDeVR5SG13NXhpOVRzVkdPa0lWbVM4aGRjcTJSZEtGbnZlZlBIVkExUCt6?=
 =?utf-8?B?MVBTcDQzT0NDcUxGSFBtb0NTTGJHSzNMdWpsVnBwMkdIZzlWU0JtMnV2aVVu?=
 =?utf-8?B?Zk5DWndTOTJjQjVyczByMUVGVXc1VmxEZWF5Ykp3bllHQUZsZTREcm1EOWxC?=
 =?utf-8?B?aW1uWUJORmxVdW93c0JDUS9QMXZpZU04ZFpjSERjSFF1R3VHRE9nem1uNWlC?=
 =?utf-8?B?TmVWM2ZUWE4vMGoxSWVDWVBUR0I1ZUxrRkZlMDhkMldZMU8rK2hYQ2VQRmd4?=
 =?utf-8?B?ZFlPWExUcXV1NkVoRHU1ZTg5bUFoMmE5K093dW1rSUxUNWtUK0dkdWUxR2ZZ?=
 =?utf-8?B?cjVEbm9SckF5eDNtYkhZTm9WMlMwNGU5SEVjV2RDRi81TVR0ekZoT1BINmox?=
 =?utf-8?B?STBIQVVCZHYzamlBQlRIQlVyQTJEWDVWWkR2enlQT2hTVDE1V012SDJlUUdT?=
 =?utf-8?B?dy8vWnVibGFITDAyVkllVENjKzlGN2cwVjRYVEMyYWxMbW9DSFpEQnhEZi9m?=
 =?utf-8?B?SlA0RG5jb3pFMGZtWG9GcDdXcW1haXgrYjdEY2NnSGJrZWRCQkN1dlF5SU4y?=
 =?utf-8?B?ck5hdUZOaGw5NGhob2dSbitZMXRxMzdaTGhHc2Q3a3RSRzUxeXpNWTJEOUE3?=
 =?utf-8?B?ak5YN05VN0NidUx0eW9XSGp4YVFqRXpuNG5WM0VjZmZicnptQm16MDl3dXVK?=
 =?utf-8?B?bFhOc3c2dzZVL3JHd2tjM05DMjM5VGhENy9hTStPN3R5eXp1Q3NMUUlVTHlL?=
 =?utf-8?B?TU9Od2tzMHhNWEZQdDk0SW8vYUpucHF5RnN6Uy9hdXRLY0pjNXpIZ2k4QTJi?=
 =?utf-8?B?M214cnRISkVBS0hsRkMyTmhsZTFuWkpCYkc1a3VWR2pjckJqa1NqWENrOEQ5?=
 =?utf-8?B?M2xJZ2NiYzN0R0NkcEZmaHVjSVBZdlU1SWl1eVZIMXVEVXA1ZDh5VTBhV0Nh?=
 =?utf-8?B?WmtST2l5YjJDakdvSVdmRFhJcXJRNEVkUFAydFlkRGxpUlNUVkxoU2hmVmpo?=
 =?utf-8?B?WnJpNXFLaFJrRldZR1NNbkEweXBTeWp1QmVkSjJaQVlhV1l0SzUxeVA2V3BQ?=
 =?utf-8?B?cEtNaTVSNk1QV3RnNTMvU0gwc0o1L2kxV0UxZExqdXozbnp2VFJad2VTcVNQ?=
 =?utf-8?B?cTZjQmZLTnU0ZldHZnYzdzdrbE14RndFVmdJMURzbTdjcDZTVDNaTDc0TzVN?=
 =?utf-8?B?U1AxdjNIV2pibkxRcFRUUlB4SlYvb3ZTNTZIajJrZXVydmVMUFRIbkJ1eTFa?=
 =?utf-8?B?LzByUFF3SEt4ZDFqMm43dWVLb2VNd1E5RG5rYkk3VExaaGNqMFBKYlhOTy9M?=
 =?utf-8?B?aTVBU3hwQ2ozS3ZNbHNsUXk5SlJ1WmhNZUpscm4yS09DYkhDdnB1OFpNbVBU?=
 =?utf-8?B?ZjZCK09vVzAzc0pvRzd1Tkk4R0k5cWloSDF2NXBweFB1WTIxMXRNejBYRWVh?=
 =?utf-8?B?bVZ2eU5IQzdqWE5JdGRwY28yMVI2WWhEMTcvNEl0ZVJRWFQ3a2Zwd1g3ejEy?=
 =?utf-8?B?a1FBUXd2b2lmckNnWTBlcnlyaUVSQ2lPYW1WalhoR3U2UVBPaUk5cjNwQ1Fx?=
 =?utf-8?B?VnNOT3RJVGZGWjNTL3hQTXJYL1BwaGw4ZStVTU5MbUFmQmdjTkVMN1VIMTJr?=
 =?utf-8?B?c0d1MEd3VVlmVUhMV3lSSTdocHNaUDRGcWtoMm9lRTJzcXBtcXRnMEw5NVZZ?=
 =?utf-8?B?dHk4REUybHcweFR3dndnT3M5NzBkRW5SMm1PNk5rMnUxeUkzbW1KbC9BT1hG?=
 =?utf-8?B?YjdQVmd1M1B5T2RpclpycVNiQ2p5S0J0TTVtc1lIOEdxeHE5T0xFZGRna0RO?=
 =?utf-8?B?c2FVSDdic1FPWWMrbFRpQm5qVXc1R0dEQkZvS3NMY2hPQzBib0JmajJBbldh?=
 =?utf-8?B?NHc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 6e8ccf84-ef90-4132-0a6c-08dc8c9188c3
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2024 16:46:27.3683
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 0HipWmpYcNEmgvjmfRAF9dn54HCofeL1OeKVqojs1j378nGq1iejOX9VyGh7+k+/UMpsY/DImzW7Sec1ARmW8eMPumlRFJvVCbj9ffKNFVc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR11MB8437
X-OriginatorOrg: intel.com

Hi Babu,

On 6/14/24 9:27 AM, Moger, Babu wrote:
> Hi Reinette,
> 
> On 6/13/24 15:32, Reinette Chatre wrote:
>> Hi Babu,
>>
>> On 6/13/24 12:17 PM, Moger, Babu wrote:
>>> I may be little bit out of sync here. Also, sorry to come back late in the
>>> series.
>>>
>>> Looking at the series again, I see this approach adds lots of code.
>>> Look at this structure.
>>>
>>>
>>> @@ -187,10 +196,12 @@ struct rdt_resource {
>>>        bool            alloc_capable;
>>>        bool            mon_capable;
>>>        int            num_rmid;
>>> -    enum resctrl_scope    scope;
>>> +    enum resctrl_scope    ctrl_scope;
>>> +    enum resctrl_scope    mon_scope;
>>>        struct resctrl_cache    cache;
>>>        struct resctrl_membw    membw;
>>> -    struct list_head    domains;
>>> +    struct list_head    ctrl_domains;
>>> +    struct list_head    mon_domains;
>>>        char            *name;
>>>        int            data_width;
>>>        u32            default_ctrl;
>>>
>>> There are two scope fields.
>>> There are two domains fields.
>>>
>>> These are very confusing and very hard to maintain. Also, I am not sure if
>>> these fields are useful for anything other than SNC feature. This approach
>>> adds quite a bit of code for no specific advantage.
>>>
>>> Why don't we just split the RDT_RESOURCE_L3 resource
>>> into separate resources, one for control, one for monitoring.
>>> We already have "control" only resources (MBA, SMBA, L2). Lets create new
>>> "monitor" only resource. I feel it will be much cleaner approach.
>>>
>>> Tony has already tried that approach and showed that it is much simpler.
>>>
>>> v15-RFC :
>>> https://lore.kernel.org/lkml/20240130222034.37181-1-tony.luck@intel.com/
>>>
>>> What do you think?
>>>
>>
>> Some highlights of my thoughts in response to that series, but the whole
>> thread
>> may be of interest to you:
>> https://lore.kernel.org/lkml/78c88c6d-2e8d-42d1-a6f2-1c5ac38fb258@intel.com/
>> https://lore.kernel.org/lkml/59944211-d34a-4ba3-a1de-095822c0b3f0@intel.com/
>>
> 
> Went through the thread, in summary:
> 
> The main concerns are related to duplication of code and data structures.
> 
> The solutions are
> 
> a) Split the domains.
> This is what this series is doing now. This creates members like
> ctrl_scope, mon_scope, ctrl_domains etc.. These fields are added to all
> the resources (MBA, SMBA and L2). Then there is additional domain header.
> 
> 
> b) Split the resource.
>    Split RDT_RESOURCE_L3 into two, one for "monitor" and one for "control".
>    There will be one domain structure for "monitor" and  one for "control"
> 
> Both these approaches have code and data duplication. So, there is no
> difference that way.

Could you please elaborate where code and data duplication of (a) is?

> 
> But complexity wise, approach (a) adds quite bit of complexity. Doesn't it?

"complex" is a subjective term. Could you please elaborate what is complex
about this? Is your concern about the size of the patch? To me that is
not a concern when considering the end result of how the resctrl structures
are organized.

> 
> For me, solution (b) looks simple and easy. Eventually, we may have to
> restructure these data structures anyways. I feel it is the right direction.
> 

I understand that it is tempting to look for smallest patch possible but we
really need to ensure that any work integrates well into resctrl. Doing
so may end up with larger patches but in the end it makes the data structures
and code easier to understand. I specifically find the duplication of structures
troublesome since that requires developers to always be on high alert of
what code is being worked on and what flows the particular code participates in
since the duplication results in members of structures be invalid based on which
code flow is used. To me this is an unnecessary burden on developers and against
your original goal of making resctrl easier to maintain.

Reinette