Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37;
Message-ID: <8c762203-db0f-d5f4-fc81-9cc548ee92af@amd.com>
Date:   Mon, 4 Dec 2023 15:20:18 -0600
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Reply-To: babu.moger@amd.com
Subject: Re: [PATCH v13 0/8] Add support for Sub-NUMA cluster (SNC) systems
Content-Language: en-US
To:     Tony Luck <tony.luck@intel.com>, Fenghua Yu <fenghua.yu@intel.com>,
        Reinette Chatre <reinette.chatre@intel.com>,
        Peter Newman <peternewman@google.com>,
        Jonathan Corbet <corbet@lwn.net>,
        Shuah Khan <skhan@linuxfoundation.org>, x86@kernel.org
Cc:     Shaopeng Tan <tan.shaopeng@fujitsu.com>,
        James Morse <james.morse@arm.com>,
        Jamie Iles <quic_jiles@quicinc.com>,
        Babu Moger <babu.moger@amd.com>,
        Randy Dunlap <rdunlap@infradead.org>,
        linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org,
        patches@lists.linux.dev
References: <20231130003418.89964-1-tony.luck@intel.com>
 <20231204185357.120501-1-tony.luck@intel.com>
From:   "Moger, Babu" <bmoger@amd.com>
In-Reply-To: <20231204185357.120501-1-tony.luck@intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MG5hZTRobHY1UTRqbUE5cGJlWlJ5cmR5SHVYRmtETTRnNWdWZlUveEFjdFlD?=
 =?utf-8?B?Q0pqbWpVMnFkRjdlSTRodHhDQlRDLys0OWNCOHhRMjBSOWZ6c0lZVmkxZ0py?=
 =?utf-8?B?WUFGSncxaUI2WVF3ZHJJV1hnS3huelJBNzFyODlmRVI1ekVaZUdscVN4VWR1?=
 =?utf-8?B?RGxNVS92M0VqcUdCQm9yMnNncmwxenVkM0R5aVMvdEJEL0hVVmc3QWIydGF5?=
 =?utf-8?B?bmNPKzA4Z3l5Z2VVa2c0Nm5YNmpvVlRBZ2hGcDd1ZllaejdFRHNZYWdSWmcw?=
 =?utf-8?B?QWsyZW8zV2RqeHcxL1hMdS81T3VlNDBIOGhUZkU3cnphUThSOTh1ZCtGeGxm?=
 =?utf-8?B?bGZHR3ZjczFRbk1LdGExY2pGa0F4azRkOW9hdndjdlA3bHdxVzZ3QmdQZHBw?=
 =?utf-8?B?RkdwRTFXTVd4YlFmVWpHMlhueitlZ2hjeHNNNGhNWE52d0R1ekVIYmJjWE11?=
 =?utf-8?B?Z2dXZUxMRWs2Mml6WXNjYjBpaXpSTTJmYWdxOW1uNUtqVythU1VqUlN4YlVP?=
 =?utf-8?B?MlZndUxnZkkxTzRjM2ZzMmFvY1RnTlR5OStPZVYyWE1uT1NRbzlFaEkzQXRW?=
 =?utf-8?B?cXluZEp6M0hjTnhHM3BYdm1seGs2d2dLNlNkcktRZnB3Vjd3VmtVY0o2ZTEx?=
 =?utf-8?B?cGE5cmVjcDJpVThLak5VakNiUkp6K2VXdDNtZjk2RjYxY3VBNlR4OEdBRy9y?=
 =?utf-8?B?QnA0QWZNUVYwL1V2amgyVlhXaTZMa1lHOU8vZW9lQzY2TTcrUDlseEgvNHBo?=
 =?utf-8?B?dTM4SGdsbjN0bHhMM0RJRndJL1FMTHRDU0xObXNTVGRMY1pqSkdqS2JqUXVZ?=
 =?utf-8?B?Mm9pcVJDYzRNbU5XWW1UaTg3b2pVWGZlc01sN0ZwYUFzaUlmbWFnRUh5UFJy?=
 =?utf-8?B?RnhrTmZHMlJKZ2VndTRXSjdLaC85S3lPRjFnOHhOTFAzSVNvQUJIajdaSC9z?=
 =?utf-8?B?TWVONktrWlpNUFArQkMwYU1ZN1NBZmdtdzYrWXkzdnA3RnpxMm5ROTNVTVlY?=
 =?utf-8?B?MUdLS3Z5eXRaVFpUNjc3NU1tdFhIV0JPbHlqaTBqSkxjUmdSV2diT054TWVx?=
 =?utf-8?B?eDRVVnI4L0JuSkdwWVMrekMzSkxjRGxFWHdkV2pXR3hDYk4wMGc4ZUF5R2E4?=
 =?utf-8?B?YlFTeHErWnJFMGZzN2hRLy9IbFhXbitSZm9wMGk4eHdmUDRrSHBlU1QvSnlI?=
 =?utf-8?B?TEVOV0V4LzdQRno4M0VyS0Q1REZ4WTY2aW5LRXlOSjMvUEJydHcxNjk2Q3l4?=
 =?utf-8?B?SWx4WERNRW85dVlHT2NBZ3Z4U2cxcGxDbzFQYzV4VDZyZnVHMzEvK1Q3Z1N3?=
 =?utf-8?B?ajduZ2FyYmhjbW9TdDJuQ01wZy9qU2NVUC9RbzJ6RWd6ZW5yMG14QVpaM3Qv?=
 =?utf-8?B?dnZBYWtnb29XN1dCT21QMzBZOElYNHlwWXBrS1UxeXJhdTZzb3l4WlE1dm5B?=
 =?utf-8?B?aFJPYXBKcHdnUDQrOUZRZCtDMWVNV0lrRlhDTE02RFFvRHJvQ2R5bkR4Nk43?=
 =?utf-8?B?eUtJV1VIckNvUVZKZ1hEem9XRmU4ZFhHY0paRVQxZHJkekFKRE1MZ24vUU9L?=
 =?utf-8?B?RmtzVnJJL1VkVFo2NllYZkVlQVI5Y09mdUpiNmt6cFJVZXlGNFlFKzJvZDNk?=
 =?utf-8?B?TlVONVBtRHpiRDEwYmNWYkszeENhVnpOaHR0dnNnS0hvUGtsd0V4bzRsRm1G?=
 =?utf-8?B?Y3Z4d0ZRa1hpSG9lanNIb1pBTHE5bWFJT1VZZS9halJQWHFkWUJYY2xrZTBQ?=
 =?utf-8?B?STBERFB3ejI0K0lxdGNTSzFwczdTWDBwaEZjdUFQd3FjcG5qKzNEaWNlK2dW?=
 =?utf-8?B?eHZaRi9XQUcyYW5iUURUeFlEQ0FPT0VDc2RLaUpXZTJ6ZzZyRTBsbHBITXdx?=
 =?utf-8?B?VHhNV0hGd2NIUDJLNzVvRVRyMEN0ZWJRRW5iMHFiU1BpejNhc3FKNG9qOUtD?=
 =?utf-8?B?SHFJZXpwc3A4UE5KNVJCQlAyTXh0NERUMjQ1OERFZ093bWNMcDZqbitHeStP?=
 =?utf-8?B?T2EveUZkSElidUZBTEVGcEoyQmUwUWh2MjRjVWxQSlZ5K2ZSdWZUUkw5VW50?=
 =?utf-8?B?K3ZJZHdzLzZpS2pXRmNyRS9ZdE4xV3RHQTE4NzcwYTRFd3J1QUtHOC95Y3hD?=
 =?utf-8?Q?leL6Nox2ird2h4m1mz2v3HgQO?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 88e991c4-5ef8-47b9-149f-08dbf50ed2aa
X-MS-Exchange-CrossTenant-AuthSource: MW3PR12MB4553.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Dec 2023 21:20:21.7077
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +VkRxNxOun652P/4tuF/g6f12j/tWVP7zAKrJEQ3BVCT2zZ+/et2g802yqXjey97
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4218
X-Spam-Status: No, score=-5.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        NICE_REPLY_A,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,
        SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 04 Dec 2023 13:20:38 -0800 (PST)

Hi Tony,

Tested the series on AMD system. Just ran few basic tests. Everything 
looking good.

Thanks

Babu

On 12/4/2023 12:53 PM, Tony Luck wrote:
> The Sub-NUMA cluster feature on some Intel processors partitions the CPUs
> that share an L3 cache into two or more sets. This plays havoc with the
> Resource Director Technology (RDT) monitoring features.  Prior to this
> patch Intel has advised that SNC and RDT are incompatible.
>
> Some of these CPU support an MSR that can partition the RMID counters in
> the same way. This allows monitoring features to be used. With the caveat
> that users must be aware that Linux may migrate tasks more frequently
> between SNC nodes than between "regular" NUMA nodes, so reading counters
> from all SNC nodes may be needed to get a complete picture of activity
> for tasks.
>
> Cache and memory bandwidth allocation features continue to operate at
> the scope of the L3 cache.
>
> Signed-off-by: Tony Luck <tony.luck@intel.com>
>
> Changes since v12:
>
> All:
> 	Reinette - put commit tags in right order for TIP (Tested-by before
> 	Reviewed-by)
>
> Patch 7:
> 	Fam Zheng - Check for -1 return from get_cpu_cacheinfo_id() and
> 	increase size of bitmap tracking # of L3 instances.
> 	Reinette - Add extra sanity checks. Note that this patch has
> 	some additional tweaks beyond the e-mail discussion.
> 		1) "3" is a valid return in addition to 1, 2, 4
> 		2) Added a warning if the sanity checks fail that
> 		prints number of CPU nodes and number of L3 cache
> 		instances that were found.
>
> Patch 8:
> 	Babu - Fix grammar with an additional comma.
>
>
> Tony Luck (8):
>    x86/resctrl: Prepare for new domain scope
>    x86/resctrl: Prepare to split rdt_domain structure
>    x86/resctrl: Prepare for different scope for control/monitor
>      operations
>    x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
>    x86/resctrl: Add node-scope to the options for feature scope
>    x86/resctrl: Introduce snc_nodes_per_l3_cache
>    x86/resctrl: Sub NUMA Cluster detection and enable
>    x86/resctrl: Update documentation with Sub-NUMA cluster changes
>
>   Documentation/arch/x86/resctrl.rst        |  25 +-
>   include/linux/resctrl.h                   |  85 +++--
>   arch/x86/include/asm/msr-index.h          |   1 +
>   arch/x86/kernel/cpu/resctrl/internal.h    |  66 ++--
>   arch/x86/kernel/cpu/resctrl/core.c        | 433 +++++++++++++++++-----
>   arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  58 +--
>   arch/x86/kernel/cpu/resctrl/monitor.c     |  68 ++--
>   arch/x86/kernel/cpu/resctrl/pseudo_lock.c |  26 +-
>   arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 149 ++++----
>   9 files changed, 629 insertions(+), 282 deletions(-)
>
>
> base-commit: 2cc14f52aeb78ce3f29677c2de1f06c0e91471ab