Received-SPF: pass (google.com: domain of linux-kernel+bounces-178834-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Tue, 14 May 2024 17:02:12 +0200
From: Maciej Wieczor-Retman <maciej.wieczor-retman@intel.com>
To: Tony Luck <tony.luck@intel.com>
CC: Fenghua Yu <fenghua.yu@intel.com>, Reinette Chatre
	<reinette.chatre@intel.com>, Peter Newman <peternewman@google.com>, "James
 Morse" <james.morse@arm.com>, Babu Moger <babu.moger@amd.com>, Drew Fustini
	<dfustini@baylibre.com>, Dave Martin <Dave.Martin@arm.com>, <x86@kernel.org>,
	<linux-kernel@vger.kernel.org>, <patches@lists.linux.dev>
Subject: Re: [PATCH v17 0/9] Add support for Sub-NUMA cluster (SNC) systems
Message-ID: <j6y653o4dy7bizn2yflp6ve3qhfmco2sqeeyyny5crqit5lmuf@ylw4bcjgm5y7>
References: <20240503203325.21512-1-tony.luck@intel.com>
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20240503203325.21512-1-tony.luck@intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?A75jEq6C5PVTpoSkIDhmYzXXNXGehsNAWYlM67yJ8ljFG8fmrPsAPyav84?=
 =?iso-8859-1?Q?yP3d+uswjIkaqu+WuhcjENDxg8iG1BvRwqhb8gSx1ZLxG51aZt396tuT+H?=
 =?iso-8859-1?Q?TX2xFGCp5TG5j/DYUMFVqBDSYcBWYcLcw6t6jqbBK0d4qynNq0MkVnNUZG?=
 =?iso-8859-1?Q?y0BW6LFsd1TUfTwFKfx3ODodh2KPKPygkxlR7vsLh1Ylu5f18LEnF56fMo?=
 =?iso-8859-1?Q?oLBG6p/Ynl10UsFVnHx91afbHGcFfMePrVMlaRp3LLaPOrFsGs26h0Auld?=
 =?iso-8859-1?Q?AS9pCy6/NGwVpL7rP5rwKfu/XCjcbb80HD6WUCR21EUSwHiA0XBRb4n1n+?=
 =?iso-8859-1?Q?x6uKUbLAvnQGqwp+bdK1/6Ykx7g2Mbxy8sG+fV88uJU91otSFPSaWgy5MP?=
 =?iso-8859-1?Q?F8kkZ/rgKRl80twMNg5MxjQN8Azx5aumUGcGPCsMoMUkUJtU86jVwn8Xfc?=
 =?iso-8859-1?Q?+H1bkqHFg94nPeI0BCnFINTTerboeZhsTyMx9bI/rZ+we3YuSAAjh9BA1d?=
 =?iso-8859-1?Q?ZQeuVg6w7dawuRqA5TVGluzrC3deDGbfrgKaOh0oXPt9o2zlV9ubAWiE5B?=
 =?iso-8859-1?Q?KsXV9x+PtlPGfuCaaly420sOcB8fR3jmbpgWPK4JKqEIIn6JnMAHmlZdB/?=
 =?iso-8859-1?Q?iI7dcZMEKVVn2wM+jpJL15tUC9H9pcp2srgkDivFGnagiLHqjToxofTkMv?=
 =?iso-8859-1?Q?nX6B7v8iGiQtUfSkKFTssJGWy1+kJpSTynMI2BuvBMZu0k0aQk/g1OMkiU?=
 =?iso-8859-1?Q?CPiItNz8xLpkJZaz8EOpyc/vC8N9DjAQ8eIaREwguA3CLfyRsx7yDoX2+e?=
 =?iso-8859-1?Q?lYrcCMtIJvZhU6yq1a8X0U9krmH7iECgekumFEsmLXoFvO1MWfk8rfgbjG?=
 =?iso-8859-1?Q?ipkORf98z+uBn/TXf5Wu7T9+jKYVPha8LoxSRVztOylOBNFa3fnVJNa851?=
 =?iso-8859-1?Q?vnEQkqOYsI4yYaDU6JU58OQ7CpyZwPZ8CRLw9jDZah2VyjIZOMHXk3IvFy?=
 =?iso-8859-1?Q?6fJo6jUlZryWrTbQudTnrEbA/l7gK5K2HxBeQ2XPxqMK96mA1x1uwcBz9x?=
 =?iso-8859-1?Q?aU9YVeiPFOVvMM+bN7a4txwHT9jeA4BYJc1viCJHDeLDJH0yd4ES2KAXTi?=
 =?iso-8859-1?Q?bVfg3fUavCGo3pE2rOF65nZVbQEG9pAbw7JwmUWRM1H8DSSgOLH4HVtmxU?=
 =?iso-8859-1?Q?8jla7GEdv0zGRmnXC03a8H3w/SPLqSH4m9N06isazum+Tn7Dj7RC4zr3tN?=
 =?iso-8859-1?Q?zwUJYwRoP0GsS1yj5w4coA704lLdWlTCserPGgzIeJu+SfER47HV21r7Rx?=
 =?iso-8859-1?Q?Fags54xXnEvKi6TY0wnx0Pwsb0OfpdvQaZ9ZdhLzmHtgnt4ZhoI/Qkmb6r?=
 =?iso-8859-1?Q?UqlQZmDluc2snN2euvoY4sClWGCa8iOZzHZ2BzWGmfYlqlx86qoGDsqGqV?=
 =?iso-8859-1?Q?/sK0Oie6ZuQJ7wmXXce846fKbteLFMn5jqt7Hp1mI+PYGMVsuyuvCm65Jg?=
 =?iso-8859-1?Q?my6Eb5s7WgBtdiPQGmToSPFWt+IHjNb0Lh61IM+tBmOgh/tLwTCeXWj3PB?=
 =?iso-8859-1?Q?9GxX34ieM0BgMCKj9rSx+SqOITKhoOGH5PmyXGyVDj6dVAEb5JxJkywpL1?=
 =?iso-8859-1?Q?A/DGSkd/IGDvuGxbHkU5ma4HeWqf4DQOQGK0InZfomv/Sgbfgp3oIJbtSI?=
 =?iso-8859-1?Q?B0edCi4vtKnEyLN7n1s=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 278b8f5c-a2b5-4b69-e03c-08dc7426de3f
X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6239.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 May 2024 15:02:26.8716
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gM0bArfxrmxw2GGpi28yIjDIDrPeoj43VccB5lNDyXpcsb13mjl/dp7DGxPnvFJ36levVBoWyiH08n91f6ZHCLPEJvaZFeeN/t6+77orCDI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB4791
X-OriginatorOrg: intel.com

Hi,
thanks for adding the 7th patch, I tested the series on a Sapphire Rapids with
resctrl selftests and the summation works well on SNC-2.

Now I can also drop a few patches from my series on SNC in resctrl selftests and
only keep the part about discovering kernel support.

On 2024-05-03 at 13:33:16 -0700, Tony Luck wrote:
>Note: Jump straight to patch 7 for the new stuff. Just minor tweaks in
>other patches.
>
>This series based on top of TIP x86/cache branch:
> 931be446c6cb ("x86/resctrl: Add tracepoint for llc_occupancy tracking")
>
>The Sub-NUMA cluster feature on some Intel processors partitions the CPUs
>that share an L3 cache into two or more sets. This plays havoc with the
>Resource Director Technology (RDT) monitoring features.  Prior to this
>patch Intel has advised that SNC and RDT are incompatible.
>
>Some of these CPUs support an MSR that can partition the RMID counters
>in the same way. This allows monitoring features to be used. Legacy
>monitoring files provide the sum of counters from each SNC node for
>backwards compatibility. Additional  files per SNC node provide details
>per node.
>
>Cache and memory bandwidth allocation features continue to operate at
>the scope of the L3 cache.
>
>Signed-off-by: Tony Luck <tony.luck@intel.com>
>
>---
>Changes since v16: https://lore.kernel.org/all/20240312214247.91772-1-tony.luck@intel.com/
>
>Patch 1: Reinette pointed out that rdt_find_domain() no longer returns ERR_PTR()
>but one of the callers was still checking return with IS_ERR().
>
>Patch 2: Tip tree added a tracing patch. That needed s/d->id/d->hdr.id/
>
>Patch 3: Reinette: Keep the "RCU" in the kerneldoc description of the
>domain list fields after the split into separate ctrl/mon lists.
>
>Patch 4: No change
>
>Patch 5: No change
>
>Patch 6: Drop the change that divided output of the resctrl "size" file
>by the number of SNC domains per L3 cache. Now that this series
>preserves the contents of the legacy llc_occupancy files this isn't
>useful.
>
>Patch 7: NEW in this series. Add per-SNC domain monitor files while
>making the original files sum across SNC nodes.
>
>Patch 8: (formerly 7) No change
>
>Patch 9: (formerly 8) Add documentation for new per-SNC directories and files
>
>Tony Luck (9):
>  x86/resctrl: Prepare for new domain scope
>  x86/resctrl: Prepare to split rdt_domain structure
>  x86/resctrl: Prepare for different scope for control/monitor
>    operations
>  x86/resctrl: Split the rdt_domain and rdt_hw_domain structures
>  x86/resctrl: Add node-scope to the options for feature scope
>  x86/resctrl: Introduce snc_nodes_per_l3_cache
>  x86/resctrl: Add new monitor files for Sub-NUMA cluster (SNC)
>    monitoring
>  x86/resctrl: Sub NUMA Cluster detection and enable
>  x86/resctrl: Update documentation with Sub-NUMA cluster changes
>
> Documentation/arch/x86/resctrl.rst        |  17 +
> include/linux/resctrl.h                   |  89 +++--
> arch/x86/include/asm/msr-index.h          |   1 +
> arch/x86/kernel/cpu/resctrl/internal.h    |  72 ++--
> arch/x86/kernel/cpu/resctrl/core.c        | 430 ++++++++++++++++++----
> arch/x86/kernel/cpu/resctrl/ctrlmondata.c |  57 +--
> arch/x86/kernel/cpu/resctrl/monitor.c     |  98 +++--
> arch/x86/kernel/cpu/resctrl/pseudo_lock.c |  26 +-
> arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 263 ++++++++-----
> 9 files changed, 759 insertions(+), 294 deletions(-)
>
>
>base-commit: 931be446c6cbc15691dd499957e961f4e1d56afb
>-- 
>2.44.0
>

Tested-by: Maciej Wieczor-Retman <maciej.wieczor-retman@intel.com>

-- 
Kind regards
Maciej Wiecz?r-Retman