Received-SPF: pass (google.com: domain of linux-kernel+bounces-188124-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <12e0979c-fec1-40bb-954f-2233c6b4a003@intel.com>
Date: Thu, 23 May 2024 16:48:32 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v18 06/17] x86/resctrl: Introduce snc_nodes_per_l3_cache
To: "Luck, Tony" <tony.luck@intel.com>, "Yu, Fenghua" <fenghua.yu@intel.com>,
	"Wieczor-Retman, Maciej" <maciej.wieczor-retman@intel.com>, Peter Newman
	<peternewman@google.com>, James Morse <james.morse@arm.com>, Babu Moger
	<babu.moger@amd.com>, Drew Fustini <dfustini@baylibre.com>, Dave Martin
	<Dave.Martin@arm.com>
CC: "x86@kernel.org" <x86@kernel.org>, "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>, "patches@lists.linux.dev"
	<patches@lists.linux.dev>
References: <20240515222326.74166-1-tony.luck@intel.com>
 <20240515222326.74166-7-tony.luck@intel.com>
 <beee3369-0075-462e-8449-88fee807463e@intel.com>
 <SJ1PR11MB6083117715CD53FA4360C56CFCF42@SJ1PR11MB6083.namprd11.prod.outlook.com>
 <013c9532-cacf-4604-b1d9-e90fdc80e7dd@intel.com>
 <SJ1PR11MB6083C6A9DFFF58FE305A33F3FCF42@SJ1PR11MB6083.namprd11.prod.outlook.com>
 <f326387d-9636-4377-b82f-10e8d335c6ff@intel.com>
 <SJ1PR11MB60832095AC783D064541F0E3FCF42@SJ1PR11MB6083.namprd11.prod.outlook.com>
Content-Language: en-US
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <SJ1PR11MB60832095AC783D064541F0E3FCF42@SJ1PR11MB6083.namprd11.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Yk8ySzBGZnAzQWUyV09NSU9uWXJGMUM0YVFEaXVtRmErR0VJdEJ5VmFOcUtH?=
 =?utf-8?B?MWN0M0JBZlFUckdNSXhjdUloUFd6SzhaNGZNUWpQZGFHVXNUSUR2VFJUd2s4?=
 =?utf-8?B?bVVvbVVpd29UbjcrMjJKQlpjQTVBZVZ3QUJTRDBramM0ODQydTkzNmZlU05M?=
 =?utf-8?B?S0NHY1hIa0M2UWhmRnc0cTBQK2toMmh0TURWcHJmTGdXT1o0OWdRSWdiUXQy?=
 =?utf-8?B?bmZmb2tkZjV2RGx0cXNDVUIrNHBCRUdsckdUaVRWVFY4OXVsM2t2S1l1RTFV?=
 =?utf-8?B?RVJ3ZXg3OUE0Z2dyeGtwMGZrSFJQRG5XeExFWXk2T3N6Vml4N2NFbTNRMXVO?=
 =?utf-8?B?bUZNa2VyR2sxSzRncVRQQm8xODNtVldJWW5jZlJtbHpBazB4MngzR0Y4Tzd6?=
 =?utf-8?B?aW4vY3JYY0Z5eEx6Zkp1MDd1MGZucjJaNTVkOTVBcUZPVVhaOU53LzQ1TXBQ?=
 =?utf-8?B?OStOS2d6UWV6dS9ydzZiQ3YwcWpmTU9RSS93d0FTZFJWbkN1MHRrM0pEUXlB?=
 =?utf-8?B?TGk0SUVxNkIwSkNxVzJOc290ZXdnbzdlNlRubVBYTUJld1ZUUjFsaXo4MCth?=
 =?utf-8?B?enZhMko2L05YWlowZEZkaDFKRnNucTVWdkMxSDM1Z1ljZ3lMbG9LOGVwelhn?=
 =?utf-8?B?bW9DVElnNVJwM0V1UlM4UmV5ejRCaGtqYjJWY1BFQ0w2aHljSlZtZHpuUm93?=
 =?utf-8?B?V1pHQ09oMmw5L2FwNG41cEFDa3FwY1M2bW82NEZsalE1SEFEbjhCS1paeldl?=
 =?utf-8?B?WDB5dXd0WUZ0aWhlL085M21Ld2pvZ3ZsSzRUT1kwbjFYYXRFaDgwaTFScmVu?=
 =?utf-8?B?MzI4dVdqMWJqV0dubnJLTlFzZmFtdlpka0ZTQnMvN1VFLzZoRm56SWdTcFRQ?=
 =?utf-8?B?UDNhSjhjT1VZWXRwL1ZEZG1jTXZxTG40VCtaN3ljRHE2eHhCYm5wY09DYnpy?=
 =?utf-8?B?UERmR1B2d2UrNUl4dDRGWE1HSzVTMS9veEFIcVZ3WFJrU1NrdlNNbGcvV3pV?=
 =?utf-8?B?R01EanNnSGVkWERtR1FSNlhrcXFtZVRBZ3Zad0oyMGFDdGhEMHlkblFzblRM?=
 =?utf-8?B?SFl4S3ZxVC9Od3pBU1k5Q2tuVVBxMDAzRVgva3ZtbFpTeklWbGtqQWx3L01o?=
 =?utf-8?B?cWVGM3VTVEJ2dUplZFB2MDdxTEc0QmROM1RmQThRUmxQY0p4VUFpc2h2dVZ1?=
 =?utf-8?B?N3p3VDNtYnRYc2RBcUJDcUpFYzhJTlQ4aFhQb2haajA3S01vVkZPMHFtOVF4?=
 =?utf-8?B?MmkxZElYTTBtZlhEdnhHZjFhRVlvV0ErTWswZndnam5Va2Y5bG9EcHM5YVVx?=
 =?utf-8?B?YmVlMlh5OUdlek1sV3hMUFNYdldQZ0VHdTczRFNZRjhURUt1VEZRYWNqcFd2?=
 =?utf-8?B?cXVNL3RWTDErZ2h3ZUpGQWM3YlUxQU5xU1VWOGFwb0h3Y0xhN2x5N1lXd29H?=
 =?utf-8?B?YUVTdmZnRE4xdUhKNExZKytQN2gwNkg1bTVKWjMwazJHT3Y2ZTRLdkIyOFl6?=
 =?utf-8?B?NkpQR2FZaU8zcDdQSFRyRlVpM09UUjFNdlZMZnYxdWpINmh3TjVBbTV1Qk91?=
 =?utf-8?B?dnM3T2xHNXRIT25zMHBMeFdFYW53d1FGWXhaZDZ4K1lkRXZBYStodmt3cmgw?=
 =?utf-8?B?VnVrclU4akxYYmRrRTVmRzE2YVQ1RG04elJXSWsvcTFqdmRna0VEVDZydTdN?=
 =?utf-8?B?OFZBendRZlpUdk5tVm1GdTA0WDg1Ykg1V2UrUXZFcTR5T1djMEdseFJlbEtY?=
 =?utf-8?B?cEdjNlF2Y1MyMk96bERIZ2JibHF1L1JwVTVEby9HNTRlanFRdTB3NlB4UGVE?=
 =?utf-8?B?Y2cxMGVENFg0QmkrUjB2S0RQajY5SDNRdkJQZmJmdVV6WXhQWE9KWnErMllX?=
 =?utf-8?B?T3YrNzZxTkcwclhhSGNkVVJ0Sit0QnhEdnhvTEhzc1prSDlJbUxaQlQ4bDlp?=
 =?utf-8?B?alJYOUVSZU4yR2EvcnJLM2c5aWZ0bUgvRkM3bTZKZGFlcEJXUStkckNiUWZv?=
 =?utf-8?B?Mko0dzR3NmdmRFBscU9NbEIrOUVWaHBhUnNWaW5KN2M4NWpGR25hL0NrVE5z?=
 =?utf-8?B?WGUyR2tjVm9kM0RBSzZ2N3N6NWs1cE9JVjZYTVhUdHJYZ3dabGt1SENKY0VH?=
 =?utf-8?B?L20rRmpGTXR5TWI2S2E0VXZOblltdnRkdmFJVGJXOFh3bW93UTJoSG90R0VY?=
 =?utf-8?B?OEE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 11981c6e-1c09-48ac-6e09-08dc7b82dbab
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2024 23:48:34.2210
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ieu+KjePMCYiNygu8X+wdXDA2FZWtEJO2iQOxffGCNbBw3hAOcxCAkP0mNrnjQMEXsu5m9qpcWnnDtsWigi3jgZPV30xTT+dUT3Q8V0wnG8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB6329
X-OriginatorOrg: intel.com

Hi Tony,

On 5/23/24 4:18 PM, Luck, Tony wrote:
>>> So (in arch/x86/kernel/cpu/resctrl/monitor.c)
>>>
>>> static int snc_nodes_per_l3_cache = 1;
>>>
>>> Set and use only in this (arch specific) file.
>>
>> Since this series initializes this value in
>> arch/x86/kernel/cpu/resctrl/core.c it is not clear to
>> me from just this one line how you envision the changes.
> 
> v18 did the initialization in core.c. But since SNC is all about monitor
> features it looks more logical to do this here:
> 
> resctrl_late_init()
>      get_rdt_resources()
> 	get_rdt_mon_resources()
> 	    rdt_get_mon_l3_config()  <<<< Do SNC enumeration here
> 	

ok.

> 
>> Just to be clear ... when I refer to arch specific and
>> filesystem code I am considering how this series integrates with [1]
>> since that is the direction resctrl is headed in.
>> Being "arch specific" thus does not require that it be moved into
>> monitor.c - it could be added to arch/x86/kernel/cpu/resctrl/internal.h
>> where it can remain after the fs/arch split.
> 
> The logical place to convert from logical RMID to physical RMID looks
> to be in __rmid_read(). Just need to pass in the domain pointer (from
> both resctrl_arch_reset_rmid() and resctrl_arch_rmid_read().

This is not obvious to me. Do you need domain pointer to figure out
which node the domain belongs to? It looks to me as though these
calls are already running on a CPU belonging to the domain so perhaps
smp_processor_id() is sufficient?

>> It will be very helpful if you view your series while taking
>> [1] into account. For example, when looking at [1] you will see that
>> mon_event_count() and __mon_event_count() are resctrl filesystem
>> functions. When you consider that it should be clear that adding
>> an arch specific get_node_rmid() between these functions will make
>> the arch/fs split more difficult.
> 
> I'll try to keep that in mind as I rework my series. In v18 my "sum" code
> went into __mon_event_count().  I'll see if I can push that down another
> level.

Thank you

Reinette