Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <d1998eb4-296f-f1ac-8deb-1b6d4fbdc1fa@intel.com>
Date:   Mon, 25 Apr 2022 16:17:46 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Firefox/91.0 Thunderbird/91.8.0
Subject: Re: [PATCH v3 6/9] x86: resctrl: pseudo_lock: Fix to restore to
 original value when re-enabling hardware prefetch register
Content-Language: en-US
To:     Kohei Tarumizu <tarumizu.kohei@fujitsu.com>,
        <catalin.marinas@arm.com>, <will@kernel.org>, <tglx@linutronix.de>,
        <mingo@redhat.com>, <bp@alien8.de>, <dave.hansen@linux.intel.com>,
        <x86@kernel.org>, <hpa@zytor.com>,
        <linux-arm-kernel@lists.infradead.org>,
        <linux-kernel@vger.kernel.org>, <fenghua.yu@intel.com>
References: <20220420030223.689259-1-tarumizu.kohei@fujitsu.com>
 <20220420030223.689259-7-tarumizu.kohei@fujitsu.com>
From:   Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <20220420030223.689259-7-tarumizu.kohei@fujitsu.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZlZQWlp6WVhpWHhUcDlEV010T0Zob3JSRVVKL2ttTW9xcHZkYytvSG82bE13?=
 =?utf-8?B?aitXY2c2eC9FVGc0R3R5bFlhNDV2QTdRdzg4aVN0V2RKZGhhUndwQXFCc3dN?=
 =?utf-8?B?WG52SmxIMnpDUWI4TkpRNFpEMnhGb2kyQWgyenl0Qnh6S2lmYVVqWExnSDFw?=
 =?utf-8?B?d24zYkw0SVZobHJnUjdrcEgxbHF6S29qbm1NNk9YdllQQ0RsMXEycTVUd1U2?=
 =?utf-8?B?OHI2bmw4T3E5amFxQ2l6NVEwMjdqbnBpY0NDNldBdVRkdkp4eExqYkNsVk0z?=
 =?utf-8?B?T3ZnWERVTHdycDA0L1NWNFlyNUw4czlZbU1teUhpc29SOVZhVW5WUXZYR1l2?=
 =?utf-8?B?bUYvMkxLM282eHRmMm9PYnpmbFpsY3k5bjFrZmxEU1U4V0FHT0hNSmZkLzVr?=
 =?utf-8?B?VTRaM2E4MGxPZ3MvRW9UYyt4aTJXSnpCQmpFQ3pPaC93b0JMNkUwVW5PN1cv?=
 =?utf-8?B?WTZUakNtdUJBZEE1QWxBZWM4SWhIVUpjMk82VXBRb1lDZlplc3hrOTFyajNm?=
 =?utf-8?B?TnJwTGRIYk95UnpYemp3ejlMZStNRDlwR0JEL2dCSDRZODd5RXR3cVlOQUdP?=
 =?utf-8?B?RmtqUG1oWFVVc3RSWHFuZW9NWFZWOEtxcVE2dGV6QUtqSUxuVXZrcGh0NFJ4?=
 =?utf-8?B?b2dycDI0Zk9RSDVSaTB4UWNDR3ovUmljZzA3NUZIditxL3VoaGhTeXM2OVh1?=
 =?utf-8?B?azllSHNjQUQyZFMrRGM1UUR3VXlseTdwN2V5Nk1GbU1RMUxTSEdxQ0c1TDQ1?=
 =?utf-8?B?QzMxMUtuZU1lWmlYMXlsNlJkbXhNcGNiYjR3azc4ajAybEYzRWVjeFB0SHk5?=
 =?utf-8?B?djNmUGdkaDhXMGM4Si9Kb2U1RkRqcWN3OS9HcHdwSXAvN3lwMUxEWjNWUytZ?=
 =?utf-8?B?RmJsa0ZlV3F1MnFzYlRTTitKNWJLU2srRGFJR0VyL0phVXA5SXhlSFVLSEhl?=
 =?utf-8?B?Y1BqMjRUdzZqZFgwV2NDT2pOVzJoM3Z2eXd4Mk9RdmVZVmhaMkkxV1VjeWlE?=
 =?utf-8?B?VVpFdXNDRUwvdlZ0cmpXRFNFclNHdGRFNkRXYVAwK0J4OWErQ2NvbW42c3h2?=
 =?utf-8?B?anU1c2FENjRxNXJCckIxT2hvTjdSZXJXUDVnczRvVDFxOXpYL0NhWHFKMmhU?=
 =?utf-8?B?MHcrTEhnUW8xYlM4bUsxS1dyb0ZoSVM4alRqYWZrUUdYWlkxZTdZS1hzeUR5?=
 =?utf-8?B?WEY5UG5lLzZVU3JpQWlOblZVU3JBWVN0UldXR3g4MXVCT2xSSUd5Z3U0cTZM?=
 =?utf-8?B?b04ycXY1cUVoVEdyTGVNdk1ZWTgwOGJxeG1aT0V0amFHcWdvOWR2OVZ1WWVM?=
 =?utf-8?B?R0w1WGtXS2ZveWZxRmx3QkNNZ2YvOXU1VmVtK214MlFvcW8vZVpRcU5nRUR5?=
 =?utf-8?B?emdDNTB2NndpRGlwR0NIaG5VZW1RSzlPVlRvVTlieENuQ2tZZ0o0UzdPSkR5?=
 =?utf-8?B?YlppRjRvVm44YW9OU04zREw2a2N0a2M2RUoyVWpvVTN4QWVLbVRsNWVvUDZY?=
 =?utf-8?B?eUFEUytRYnBiNGZ1ZTlmWUZ3QU52ODlYTGplVi9WTldiQ1FReTlBRHMzVmpw?=
 =?utf-8?B?VVdpUmNGUVAzcTNwRlg2M09TQjJTYWF5OUJacTc1TmNKdDJyQTgrNmxjT0pr?=
 =?utf-8?B?ZTFrYzlndW5lcnY2MkZ0NndzTG50cGFtN2ZoSjg5OTRPMVI1RlJmZVg4VE1w?=
 =?utf-8?B?ZGprb1pUa0JBYkt1ZmVuQ3c1WFVoanFqU09TWmtKamVEMFhZV1ZUVlNXZXhD?=
 =?utf-8?B?dXdWTjFIbnRXbHY2K2VjMTBuekNaRnhFd3h2UTBxa2p1YW1HeHlFSWdJbGNq?=
 =?utf-8?B?L2JwTjNJQ2ZzaU5aeHNxeHpud3BVU2J4emllMDlyUlo2RlZYbnNwaGlOSEd5?=
 =?utf-8?B?MnJsTXlrcW52MWNPdUV5TnF5ck1BenhvbnVUcE9VY3Vqa1p2YTROSFc2emt1?=
 =?utf-8?B?c1ZPWGkwWWY0M2g2N2p0bXo4V0QvN29xSEErVmZxSDRQenBFSWFwWEhIa0VD?=
 =?utf-8?B?K2QvQXRHZEh6VVlsb1RvanFTZVZTUDdnSmpCTXpJaW5ZeHpBOE1rRXlRcjFv?=
 =?utf-8?B?dStXV1dsbThpTmVFekxDYUI0T2trVjVaSEVoU0s5TkRvN1RUc2QwWHhUVlpI?=
 =?utf-8?B?clF2NjhOUmM3TmxCRmthQmRZcy93cWNlY2pTMUIwblZzQjlEL0FUUkwwS25l?=
 =?utf-8?B?Y1JVbTJxdDMyZ2VLQ016ZEtsWDh6bnU0YVgwaHhRVDV4blFSRFMySkZuWHZP?=
 =?utf-8?B?aStFM2diQmdBQkEwcVFDelQ4VGlNN09jd3FvbWt3WTNPcnZlRnU2QTBudE5L?=
 =?utf-8?B?V1oxYUQwb3FjOVlZZnlaRllZK3F3WFRLQmpoVXdlL1BEMkZJUUs1NXZVNWky?=
 =?utf-8?Q?CL+HyVSorJ0SWmnM=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7fa38883-b90a-48c1-7637-08da2711d118
X-MS-Exchange-CrossTenant-AuthSource: BN0PR11MB5744.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 23:17:50.4970
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: nEjIIwXDd9eL/Oia5+8QB8GYRIDvEklWbFqH377ep3B4S1TOO+xgbHI+19NrUEqX+UBA953ELDK2B1S/Qd+4C4ECnt4CUgQrnrGwNDDqVCY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB2565
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Kohei,

Thank you very much for catching this issue. This fix is not specific to
or required by the driver you are creating in this series so you could also
extract this patch and submit it separately as a fix to resctrl.

When you do resubmit there are a few style related points that I highlight here,
the fix itself looks good.

For the subject, please use "x86/resctrl:" prefix in the subject. 

On 4/19/2022 8:02 PM, Kohei Tarumizu wrote:
> The current pseudo_lock.c code overwrittes the value of the

overwrittes -> overwrites

> MSR_MISC_FEATURE_CONTROL to 0 even if the original value is not 0.
> Therefore, modify it to save and restore the original values.
> 

This needs a Fixes tag. A few patches are impacted by this fix:
 
Fixes: 018961ae5579 ("x86/intel_rdt: Pseudo-lock region creation/removal core")
Fixes: 443810fe6160 ("x86/intel_rdt: Create debugfs files for pseudo-locking testing")
Fixes: 8a2fc0e1bc0c ("x86/intel_rdt: More precise L2 hit/miss measurements")

> Signed-off-by: Kohei Tarumizu <tarumizu.kohei@fujitsu.com>
> ---
>  arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
> index db813f819ad6..2d713c20f55f 100644
> --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
> +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c
> @@ -420,6 +420,7 @@ static int pseudo_lock_fn(void *_rdtgrp)
>  	struct pseudo_lock_region *plr = rdtgrp->plr;
>  	u32 rmid_p, closid_p;
>  	unsigned long i;
> +	u64 saved_msr;
>  #ifdef CONFIG_KASAN
>  	/*
>  	 * The registers used for local register variables are also used
> @@ -463,6 +464,7 @@ static int pseudo_lock_fn(void *_rdtgrp)
>  	 * the buffer and evict pseudo-locked memory read earlier from the
>  	 * cache.
>  	 */
> +	saved_msr = __rdmsr(MSR_MISC_FEATURE_CONTROL);
>  	__wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0);
>  	closid_p = this_cpu_read(pqr_state.cur_closid);
>  	rmid_p = this_cpu_read(pqr_state.cur_rmid);
> @@ -514,7 +516,7 @@ static int pseudo_lock_fn(void *_rdtgrp)
>  	__wrmsr(IA32_PQR_ASSOC, rmid_p, closid_p);
>  
>  	/* Re-enable the hardware prefetcher(s) */
> -	wrmsr(MSR_MISC_FEATURE_CONTROL, 0x0, 0x0);
> +	wrmsrl(MSR_MISC_FEATURE_CONTROL, saved_msr);
>  	local_irq_enable();
>  
>  	plr->thread_done = 1;
> @@ -873,12 +875,14 @@ static int measure_cycles_lat_fn(void *_plr)
>  	struct pseudo_lock_region *plr = _plr;
>  	unsigned long i;
>  	u64 start, end;
> +	u32 saved_low, saved_high;
>  	void *mem_r;

Please do follow the current style of using "reverse fir tree order".
More information in:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/maintainer-tip.rst#n587

>  
>  	local_irq_disable();
>  	/*
>  	 * Disable hardware prefetchers.
>  	 */
> +	rdmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high);
>  	wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0);
>  	mem_r = READ_ONCE(plr->kmem);
>  	/*
> @@ -895,7 +899,7 @@ static int measure_cycles_lat_fn(void *_plr)
>  		end = rdtsc_ordered();
>  		trace_pseudo_lock_mem_latency((u32)(end - start));
>  	}
> -	wrmsr(MSR_MISC_FEATURE_CONTROL, 0x0, 0x0);
> +	wrmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high);
>  	local_irq_enable();
>  	plr->thread_done = 1;
>  	wake_up_interruptible(&plr->lock_thread_wq);
> @@ -945,6 +949,7 @@ static int measure_residency_fn(struct perf_event_attr *miss_attr,
>  	unsigned long i;
>  	void *mem_r;
>  	u64 tmp;
> +	u32 saved_low, saved_high;

Same as above.

>  
>  	miss_event = perf_event_create_kernel_counter(miss_attr, plr->cpu,
>  						      NULL, NULL, NULL);
> @@ -973,6 +978,7 @@ static int measure_residency_fn(struct perf_event_attr *miss_attr,
>  	/*
>  	 * Disable hardware prefetchers.
>  	 */
> +	rdmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high);
>  	wrmsr(MSR_MISC_FEATURE_CONTROL, prefetch_disable_bits, 0x0);
>  
>  	/* Initialize rest of local variables */
> @@ -1031,7 +1037,7 @@ static int measure_residency_fn(struct perf_event_attr *miss_attr,
>  	 */
>  	rmb();
>  	/* Re-enable hardware prefetchers */
> -	wrmsr(MSR_MISC_FEATURE_CONTROL, 0x0, 0x0);
> +	wrmsr(MSR_MISC_FEATURE_CONTROL, saved_low, saved_high);
>  	local_irq_enable();
>  out_hit:
>  	perf_event_release_kernel(hit_event);

Thank you very much.

Reinette