Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <8c1efa87-ddbd-1d88-3964-eddfbffe8bf6@intel.com>
Date:   Thu, 15 Jun 2023 15:10:32 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Firefox/102.0 Thunderbird/102.11.2
Subject: Re: [PATCH v4 13/24] x86/resctrl: Queue mon_event_read() instead of
 sending an IPI
Content-Language: en-US
To:     James Morse <james.morse@arm.com>, <x86@kernel.org>,
        <linux-kernel@vger.kernel.org>
CC:     Fenghua Yu <fenghua.yu@intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        H Peter Anvin <hpa@zytor.com>,
        Babu Moger <Babu.Moger@amd.com>,
        <shameerali.kolothum.thodi@huawei.com>,
        D Scott Phillips OS <scott@os.amperecomputing.com>,
        <carl@os.amperecomputing.com>, <lcherian@marvell.com>,
        <bobo.shaobowang@huawei.com>, <tan.shaopeng@fujitsu.com>,
        <xingxin.hx@openanolis.org>, <baolin.wang@linux.alibaba.com>,
        Jamie Iles <quic_jiles@quicinc.com>,
        Xin Hao <xhao@linux.alibaba.com>, <peternewman@google.com>,
        <dfustini@baylibre.com>
References: <20230525180209.19497-1-james.morse@arm.com>
 <20230525180209.19497-14-james.morse@arm.com>
From:   Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <20230525180209.19497-14-james.morse@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cjlOdXZINEJzV09XalJtZ1RESEJ4MC80ZmZ6RkNTa3Z4ckduRHpCYWFNRGt1?=
 =?utf-8?B?THhtdnY2OWJNTHlYK0o1S3lLY08xUXhrSDBUekFEVlBVMS9pVXMwOFR0WGZZ?=
 =?utf-8?B?SWxjbVJNMDBGVGNGQmRXbnJrWXFFajFJZkN4R3V5RzhIak5TdEYvQ3N3TytY?=
 =?utf-8?B?Zm1xKzcra1NvbzFFREdNR0wvc3paMHY4cTl4N0dZUzZ3RmM5RWlHVC8xR0wr?=
 =?utf-8?B?VlNNL2lRVUltVm9LOUYvTG1yM3ZMekZMaEdHNzkzWWo4Nk5RNXJPTzIvT0xG?=
 =?utf-8?B?SGFWLzdoYU9pK1Q4Yy9PTWE4QmIvcGp2NEdUSGJjOFhmakRNQnZ6V21DQ1hn?=
 =?utf-8?B?V0x0TUk5SEw0YWJZUnZzcGlXRTdrUlpMaWg3dVNUbzliRUdEdWFNSmpEK254?=
 =?utf-8?B?NFlCODE1c0ZQc1NXOEtOZVQ2U09XMUMraC9ZN3IwSUJQU3JxZUNRajFzbThz?=
 =?utf-8?B?NG5NU1NUR0NOVzdqQzAvdHMvd3ZrS1gvcmk1cnFPam1zcXpvSjhrUXNreDYz?=
 =?utf-8?B?NHNPMnZ1b2RxZEpWZEREaDF3TmIxakYrUHBnOEM4aCtjdnI0MXd5Sk1vRC8x?=
 =?utf-8?B?dTBqNlUvcy9Kbk5oV2hEU2tJWDJ3YVZzVVdmY1ZWRnBzdmlaemo5TjhIN0Nv?=
 =?utf-8?B?akdvb1BLWlIyak02Qm1FV0paeXNMeENKb09DVXlycW1YeG0yYUE5Y1VleG9x?=
 =?utf-8?B?U0RteDk4enlsZGtRQ1pXcXdCMU5mOUN3eWp5QUYvbE8rZCtmZmhYUUlxRlpB?=
 =?utf-8?B?YVFpeXJ2ekJQTzlaUGlKSTVXSDlZbkJBbnBKQ2ZaK1ZMTWdvVnRVVmJ5aGwy?=
 =?utf-8?B?ZElIWG9rckFxbVpOMEZLcmlHckZILzlud0NPYVBHMVpBejNBRWpBRUcvOWMr?=
 =?utf-8?B?b1pUNWZRMStOY0pZeGlUWU9DVENmUlkyYXpYMnZkS3lYcVc2RVo1UzF3SVJS?=
 =?utf-8?B?dHZKeE0zdEQ2T2Q4VUJtYm1GNEdmWU94d3hKS1VBRXpLM1dhQm5HSmMyN3pz?=
 =?utf-8?B?YVpMTm4yMS91RGt6dlg5UHRvcHpLM3RJdERMUnJTamFibTNNWmNZUlZuUG9E?=
 =?utf-8?B?Nyt4WVJTQStXVnlDMUN3MW4rVzFiOGdyRHZqd3drcm1ScjJhY1pXaktST2R2?=
 =?utf-8?B?SDVWMkFGYVJiRCtRZ2FLejI5WFZFd3hzV1AwMG4rQVJJM3dIL2NCdkExWlFJ?=
 =?utf-8?B?MTM2RERaRUZFeHVFTWVtc1luNXRjeDRLNTNPbjcxdndJL3QvMHBPb0tCNXA3?=
 =?utf-8?B?WWdaMkR2NHRFeHRBNGxyK0JjbHUvTDE1Mnp6ZXVSQ1FGZUswL2lwbGtyRXNZ?=
 =?utf-8?B?dURHbnR3QzNIL3dNN0hPNVlOTzUrbjA3b3dqRlVvOGd1VGxPZE5ta3BjeUdk?=
 =?utf-8?B?Yy9MMTkzYngxNXZJQllFZWo0NGgwQzBCdy80am1UVTVrVExGWmRhb2NsUUtr?=
 =?utf-8?B?a0czVkNsTW5LOHFKbnlEakxnNWFHcDhjK2VNN3VkTzl4VmVnRGxIMURCV1dS?=
 =?utf-8?B?Vit3R0JYQWpoU3J1aHJTZVJ0SC9WT216QUlmelpRenBrNU40RktyVXBvSUI1?=
 =?utf-8?B?UFU3U2xzdnpJYUZNU3dYV1prU2svSWRMUkpYT3Y3STc3eDhFR3VPUEdCWHJh?=
 =?utf-8?B?YisxS3ByTmhaYmhjSVU2SytiL0JuUnlZSEF6eVVLOWJ2QzJLWWhOZzZXUFQx?=
 =?utf-8?B?a0phb0FicnVHOUZneWtYejdMTFpmZmJpbVNDVXZGUUdjLzI3MDMxOE9FUG5J?=
 =?utf-8?B?VGpxQk50WmU2U0VScFEyck1ZNTc0WGlscWRwaWFFRHBzNUpjUFZlOXpoWE9k?=
 =?utf-8?B?aGxJcktxSVdvSDdqVmFsUUpIWEdpdGRYTExPVlQzQ2NDaWYxUUJWalNzRUxV?=
 =?utf-8?B?cWVxSHFOYUZZMDR4VlI3Ukdsa3hEMFBuVVBBNlJ1NlNKVFM5a1NiaFdSQ3Zl?=
 =?utf-8?B?VzNMZVA4VGdDeG9TQ1hlMXhJMGprRHpZSHlRTjZRKytzMFNtOWdQYVVLMGll?=
 =?utf-8?B?VExxQ3E0M3REeXVxdFU1UENuMUFKdnhtTC9ONXlXWlRtN01oYjNvRXR5RHNi?=
 =?utf-8?B?SFlQYVlERHFxQytKYXRqRzlvR2h4YjNsZFN2RlBMWDhPRVNIZWZ1ZnNZd0lQ?=
 =?utf-8?B?RU05bGRGeHR0REEwamNvMnRMdWpCNSs4TStNRHFsWHVpRFhmNDA0Y01MWWVD?=
 =?utf-8?B?Rmc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5b9861d2-d649-43f8-932c-08db6ded73ae
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jun 2023 22:11:21.9960
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: SEnS1EeHl1xnIdQQoWYrsIckbH9vrUY6pJUYh1pEu8tIaP7ref9UXduLppnPp73I77+kTb9wf3Un80G1CPdMadYeTn0lvR0ltjBFzbCAHRc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB8204
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi James,

On 5/25/2023 11:01 AM, James Morse wrote:

...

> @@ -520,12 +521,24 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of,
>  	return ret;
>  }
>  
> +static int smp_mon_event_count(void *arg)
> +{
> +	mon_event_count(arg);
> +
> +	return 0;
> +}
> +
>  void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
>  		    struct rdt_domain *d, struct rdtgroup *rdtgrp,
>  		    int evtid, int first)
>  {
> +	int cpu;
> +
> +	/* When picking a CPU from cpu_mask, ensure it can't race with cpuhp */
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
>  	/*
> -	 * setup the parameters to send to the IPI to read the data.
> +	 * setup the parameters to pass to mon_event_count() to read the data.
>  	 */

I understand that you are just copying existing comment but could you
please change it so that the sentence starts with a capital letter?

>  	rr->rgrp = rdtgrp;
>  	rr->evtid = evtid;
> @@ -534,7 +547,18 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
>  	rr->val = 0;
>  	rr->first = first;
>  
> -	smp_call_function_any(&d->cpu_mask, mon_event_count, rr, 1);
> +	cpu = cpumask_any_housekeeping(&d->cpu_mask);
> +
> +	/*
> +	 * cpumask_any_housekeeping() prefers housekeeping CPUs, but
> +	 * are all the CPUs nohz_full? If yes, pick a CPU to IPI.
> +	 * MPAM's resctrl_arch_rmid_read() is unable to read the
> +	 * counters on some platforms if its called in irq context.
> +	 */
> +	if (tick_nohz_full_cpu(cpu))
> +		smp_call_function_any(&d->cpu_mask, mon_event_count, rr, 1);
> +	else
> +		smp_call_on_cpu(cpu, smp_mon_event_count, rr, false);
>  }
>  

Reinette