Received-SPF: pass (google.com: domain of linux-kernel+bounces-215243-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Message-ID: <e17d3a79-9c07-44cd-b0ea-2d0545680342@amd.com>
Date: Fri, 14 Jun 2024 11:27:48 -0500
User-Agent: Mozilla Thunderbird
Reply-To: babu.moger@amd.com
Subject: Re: [PATCH v20 00/18] Add support for Sub-NUMA cluster (SNC) systems
To: Reinette Chatre <reinette.chatre@intel.com>,
 Tony Luck <tony.luck@intel.com>, Fenghua Yu <fenghua.yu@intel.com>,
 Maciej Wieczor-Retman <maciej.wieczor-retman@intel.com>,
 Peter Newman <peternewman@google.com>, James Morse <james.morse@arm.com>,
 Drew Fustini <dfustini@baylibre.com>, Dave Martin <Dave.Martin@arm.com>
Cc: x86@kernel.org, linux-kernel@vger.kernel.org, patches@lists.linux.dev
References: <20240610183528.349198-1-tony.luck@intel.com>
 <658d9869-ef22-48a7-876a-5bbba4f134ff@amd.com>
 <bc52a051-4296-48ac-9a83-29139855553f@intel.com>
Content-Language: en-US
From: "Moger, Babu" <babu.moger@amd.com>
In-Reply-To: <bc52a051-4296-48ac-9a83-29139855553f@intel.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?enZCNHFLMkdCbUNVN0ZmOExld3FCaVIrODRhcmx3dGwrbUdXNnA0WFpGdVdy?=
 =?utf-8?B?YVlJeVZkc2piWWhEb3hoRDlGdFRFWHJKaGI1YS9peVJwa2hkSHNjMmNZL2xW?=
 =?utf-8?B?VHBJRjBZdHBJL0Ezckp6MzVKczlDdzhTZGhMTEVJM2cyS2VBbDdHajBmZzNv?=
 =?utf-8?B?M0k1MVB1VCtmN2w1cVJYVmFwbHN6VzdMS0FPZS9nMEE5akZMbnR3M3ZSNW1P?=
 =?utf-8?B?WkpqL0hOaGZjZTNCZzBWUVdCdFlwQ3o1Nk5tWXh2QmtSclBsRkxEK3dKMTA5?=
 =?utf-8?B?M1VhTjF0KzZHT1M1TDNmeVdDWDF5UXFLUVd2SFZmcGx1TFl2V3lJbUVhMDhO?=
 =?utf-8?B?ei85dzJUUHdvdkpQVmlEcE5lVFgwY2kzYVJkSzhNdEkxZVRzeS9qbFhyOG55?=
 =?utf-8?B?YmFTblFWNnllcGpaYXcyV29Lc1F2R3RaWTdKZ0Y3RzN0MFZrdG9jZlVDcnBx?=
 =?utf-8?B?OE5zTzRhZUxUWTZMUFBXcnFUZjcyOCt6OTE4L0RUNHRjSDVkbDNXR0g3OVFp?=
 =?utf-8?B?OWQ0VityME1uQmRTTjd5QXB1YTF4Mmo0SXZwN2NRUnl4TERWTzBpeHZBT2E4?=
 =?utf-8?B?WFlMN0NweUFZUXFuemRrV2ZuRlY5RUxtTXZBNTkxZTZtbXllajAzcG9pMk1W?=
 =?utf-8?B?QTNMQUsyVkN2RDRKQ2NibDZzTGR4TFBXRk1tejZORlplUTdKNE5IZzM5dEpQ?=
 =?utf-8?B?YUo0WVhyNHo2U250ZVoxTnNPSzV5UHArS3hZZnB3RzE1Y2xlN2JGcXoxL0gy?=
 =?utf-8?B?TVRTWUJ6U2tjMzBNS084ZnFJSnRaR25nWkpLRk8xazhCdWRHUmdkTCtlNkIx?=
 =?utf-8?B?Y0FhbHB0T3ZjNWdraFZ5dGJkR1dYRjNCTGpMUklPS0lKcFJjb2VWVWRNUEgr?=
 =?utf-8?B?TGpKM0FYY1dJd2pqd3c2ZFBoT1hDeXlWa25tNHJLU1R1TC9IYlUvRzlTZHpC?=
 =?utf-8?B?N1pIWUEzaDNKb3pJTUszOWdnSnVhUE80RmtybHBZWU5majJ3ZE11S1JXcmlE?=
 =?utf-8?B?aHdaNFR3Nm9mTHdPNlh2WmxPdlVtYThZdmlXOHJQQWJZeTB6UStJbkVXVUU2?=
 =?utf-8?B?NGU1T3ByaVZ2eGZVYlhnMldNRzdNOGxBRUl1QjNWa21GS2l4QU5PS2pTamdF?=
 =?utf-8?B?amMyeXFkWUhzWEpKem91cDNPOFhlWlFKZ1NGYkFNdEZSWGJhNldlVElaQVJ1?=
 =?utf-8?B?Y1pvSi9XV29XaDNKWHVKS1NXUkpIYy82Z3Z2MjY2bkRITGE4TGdQdVdET0RT?=
 =?utf-8?B?WFNvejV6TEVOU2VLdG5EZmlJc0VEU2JVTGR5d0NJQng2ZVlwTVdHNFpIYmVx?=
 =?utf-8?B?M3VEbHA5R3E4SHRvZEhsRUxaS1dWVkRuUVZyVkFZVjBvOVZ6RE8wK0ZYV1BD?=
 =?utf-8?B?d01RVmdxcXRkcEdLQk9Ubzl6d0xFL25IUkhYcXpaZGlpSWo4alQ4R1BIOUxM?=
 =?utf-8?B?ZlUwMXQ4b3hMUHJRZk13MFpqR0Z3T0QzMm5vRlZLZlFkNWVwYkM5Y2l5eTNu?=
 =?utf-8?B?bXN0Y1JkYzd1UzhwRHo2bGpoUTllUVVLakRIcWlFOEticWFCcjJyYlVKSmFi?=
 =?utf-8?B?YXZYS0RJSVJXNFlaK2RzbXBGVEZUbm53OVZVMTNmSllHbHo5ajZGRnRMSUNr?=
 =?utf-8?B?T0YwbzEwQ3NJOXc5V1NvR3FZQ1g2b2NzOFlZaXF3OHI4N1ZWUTZGMy8wazA2?=
 =?utf-8?B?dHBxcWx0dm9NakhIWnlHeWRpN0tZdXlKeDB4bENsU05yaHBGM0hYckx4Ukh4?=
 =?utf-8?B?NzZvR05yR1R0N0ZKNkdwdCsrc2JReUlaQVIvTWFDc2lRcjVpRTZYR3ZaOE1P?=
 =?utf-8?B?bmJVWkREZGt1SUFQZitpYzlTOWx0Vm5MQzgzUWVPQzBiMXVrSWgyT2hlWGs3?=
 =?utf-8?B?NTdPbzdRaGxQMXRQY0hTTlNrVkNnZkhwSmFmNlB6cHF3NW1Lb1V4M1c1OVd4?=
 =?utf-8?B?NS9NNlAzREJ2b3ZTYVlDY0MzNnVXSFdIbDAybVhWeU9vbVhjUkF4TzhTVmZM?=
 =?utf-8?B?c2JxeWdVdUVUOFN5Uy8zSStGZXdNNlo0K2FYWTJWbU0wY2U0NHcwVmFXTFk2?=
 =?utf-8?B?ZWFWYlY5Z3ZHZ2xkVG8wRnVaaXYzeW42dUtFeCtPOXNhTHhtZlUyaXZSU0x0?=
 =?utf-8?Q?D1S4=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3d82d7cb-3085-4dcf-586b-08dc8c8eef7c
X-MS-Exchange-CrossTenant-AuthSource: MW3PR12MB4553.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2024 16:27:51.3227
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ODb3W6d4Iw5xkgGvZMMpLAPoTkSwy1Xz4bcUnVPESRsMf5hFPsx2c5P1so2Xy+1s
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8136

Hi Reinette,

On 6/13/24 15:32, Reinette Chatre wrote:
> Hi Babu,
> 
> On 6/13/24 12:17 PM, Moger, Babu wrote:
>> I may be little bit out of sync here. Also, sorry to come back late in the
>> series.
>>
>> Looking at the series again, I see this approach adds lots of code.
>> Look at this structure.
>>
>>
>> @@ -187,10 +196,12 @@ struct rdt_resource {
>>       bool            alloc_capable;
>>       bool            mon_capable;
>>       int            num_rmid;
>> -    enum resctrl_scope    scope;
>> +    enum resctrl_scope    ctrl_scope;
>> +    enum resctrl_scope    mon_scope;
>>       struct resctrl_cache    cache;
>>       struct resctrl_membw    membw;
>> -    struct list_head    domains;
>> +    struct list_head    ctrl_domains;
>> +    struct list_head    mon_domains;
>>       char            *name;
>>       int            data_width;
>>       u32            default_ctrl;
>>
>> There are two scope fields.
>> There are two domains fields.
>>
>> These are very confusing and very hard to maintain. Also, I am not sure if
>> these fields are useful for anything other than SNC feature. This approach
>> adds quite a bit of code for no specific advantage.
>>
>> Why don't we just split the RDT_RESOURCE_L3 resource
>> into separate resources, one for control, one for monitoring.
>> We already have "control" only resources (MBA, SMBA, L2). Lets create new
>> "monitor" only resource. I feel it will be much cleaner approach.
>>
>> Tony has already tried that approach and showed that it is much simpler.
>>
>> v15-RFC :
>> https://lore.kernel.org/lkml/20240130222034.37181-1-tony.luck@intel.com/
>>
>> What do you think?
>>
> 
> Some highlights of my thoughts in response to that series, but the whole
> thread
> may be of interest to you:
> https://lore.kernel.org/lkml/78c88c6d-2e8d-42d1-a6f2-1c5ac38fb258@intel.com/
> https://lore.kernel.org/lkml/59944211-d34a-4ba3-a1de-095822c0b3f0@intel.com/
> 

Went through the thread, in summary:

The main concerns are related to duplication of code and data structures.

The solutions are

a) Split the domains.
This is what this series is doing now. This creates members like
ctrl_scope, mon_scope, ctrl_domains etc.. These fields are added to all
the resources (MBA, SMBA and L2). Then there is additional domain header.


b) Split the resource.
  Split RDT_RESOURCE_L3 into two, one for "monitor" and one for "control".
  There will be one domain structure for "monitor" and  one for "control"

Both these approaches have code and data duplication. So, there is no
difference that way.

But complexity wise, approach (a) adds quite bit of complexity. Doesn't it?

For me, solution (b) looks simple and easy. Eventually, we may have to
restructure these data structures anyways. I feel it is the right direction.

-- 
Thanks
Babu Moger