Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <ff44b0ff-6adb-3bae-d17e-4c341c09df5d@intel.com>
Date:   Thu, 20 Oct 2022 09:35:59 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Firefox/102.0 Thunderbird/102.3.3
Subject: Re: [PATCH v2] x86/resctrl: Clear the stale staged config after the
 configuration is completed
Content-Language: en-US
To:     Shawn Wang <shawnwang@linux.alibaba.com>, <fenghua.yu@intel.com>
CC:     <tglx@linutronix.de>, <mingo@redhat.com>, <bp@alien8.de>,
        <dave.hansen@linux.intel.com>, <x86@kernel.org>, <hpa@zytor.com>,
        James Morse <james.morse@arm.com>, <jamie@nuviainc.com>,
        <linux-kernel@vger.kernel.org>
References: <1665304608-120466-1-git-send-email-shawnwang@linux.alibaba.com>
 <7fa6ed4e-abae-85fb-4e95-8c73755a4263@intel.com>
 <ad08eee6-cfea-3858-0def-e2e3fef315fb@linux.alibaba.com>
From:   Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <ad08eee6-cfea-3858-0def-e2e3fef315fb@linux.alibaba.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?a0c2L0hINk9lTy8vM0plakExMFc2ajEvR1NYd2NaZ3UxdXdnczJQcHgyT2Nq?=
 =?utf-8?B?Qks2MS8rQzNPdzBpMWJhcXZYdjVxcjgxMFpva2FveE95b3dObFBPK081czQw?=
 =?utf-8?B?M05PenJncTh3Q3A1S2h4L09uR0tMOVBaM3p4ZENUZDFabk1Tai9RV0lPUVFt?=
 =?utf-8?B?djc2WWpLQi81SlFPUklsSHdEWGVQY1RaL1J0ZHB1MUUzK204c3lsWnBTNHZx?=
 =?utf-8?B?ZHNwNmU0TmhIVW85OU5meTh2ZUNQc1dYU2VJSlBZbXJhRDlVVkt4TmdLN2tD?=
 =?utf-8?B?bTd5TCswRTJCdStzMlgzV3lDQUtrSWxiT2Z0MnZVVkFJNWRGNVVzaG9CR0dL?=
 =?utf-8?B?bk5wb3lhTUQ5M3FDNUc5dXZWcDBNYjNxdmNBaUVsU3d3R0pDK3BCeXVYSTcv?=
 =?utf-8?B?bFBmRzQ3WFlib2JDQzA5cnQ0U3NXM1VlU2tGaXQ1VVZUZDBHRXcvQlRGUlBn?=
 =?utf-8?B?RW9JM2huanVPSS9DanNUbkg0VU0rT0hQQ2RobUk0c3k4WkZBTjNUTnN3alFv?=
 =?utf-8?B?bUpjYWU4My9URnBsK0F5NE5OSisvZDAzMS84cWk0ditQTW5DZUk0ZlZPTzlk?=
 =?utf-8?B?WE5QSG9tenpuMHZtUnpsVklHL2JsdUVmNVpxK21YbGYwSGV0Tm1vaFVRcHhw?=
 =?utf-8?B?ODV0NGludXZHdnM5aEk5QzZPeHBDM0prcGQ2L1I5cXNKN3RiT21tVUNqQ0l2?=
 =?utf-8?B?VDhjVkpTTWZZMm56RXNhT2hIZlRrOU55d1NncEQxait3bjI0QW40ekJhRFYx?=
 =?utf-8?B?ekg5allQVWoxajd6bXJMRFFNK0Z2QVEycEtMTGFFbzZkd0Z1MVVEVGI0NFZT?=
 =?utf-8?B?VVQ3cVZPR283Y01Ob0pmOUJPQVlFOXVMRlkvNm8wWER4d2duK2JxOU1EYXdU?=
 =?utf-8?B?Yy9NUytrMjduRGVXQlNVeDV2QWsrVzAzSC9tYWI5NnorYldORWlaUUgwNXp6?=
 =?utf-8?B?RCtmUjg0Zi9nVUREVUZVOVp6dVZMN0hQVFhUYUdJc0wwUGFFbS9jV0JxdFo0?=
 =?utf-8?B?VWtMOHZkajFmNlJnQlJOMVN6VjZIMUVPcW15SkRUdkNGcGE3M2pIUGRjeXln?=
 =?utf-8?B?ZXdhUXR0MGdOa3JrcTJuVUhPTUxmNGZGanVuZ2p3bm5IclVXcXZ2RXBDTk9p?=
 =?utf-8?B?d1dUZFRMYndwTm5EV28wQU1xbm5sUDIvSk5sbTVhaHpSWUFGNnJ0UXk3Zjdy?=
 =?utf-8?B?dTI2M3J6V08xNEJEUEtPOXdHNTlpOXF6dTc3dWZJVHlNSDhZM1NvR3BkclVy?=
 =?utf-8?B?blZKbEMyT295WGZCNXdYdi9kdE00cCsyckRVS1hOZjF1Y2hnWVIxNm1wR1Iy?=
 =?utf-8?B?RURiNktoZEtlVkFvSTFuZlpMMjh4L0lIMWJhMTZYcVRxckZPZUlZcDdTMnVZ?=
 =?utf-8?B?Q1FSWHhxSy8rNUw4eEVNNndFbzZlcGR6c0pSMFB1UzVhL3BiUTBaWStHcDdJ?=
 =?utf-8?B?TXRlZFZDYWNsM1YvcVJ2OHpYOUwrbEJXS3FCTVc0bXNFdzkrZ2d3RkREY3N6?=
 =?utf-8?B?OHVNMGlpTVlNMUZvYmpzVDBDRHRpTWF2aFRyaHlEM0JZRG1aVGRqbytRenRM?=
 =?utf-8?B?bTdtQVRCd0lrTndVb2kxb1A1M3ZjR2ptWGttbktNTTlwYjROemh3TzFkV2I5?=
 =?utf-8?B?TDM3ZHNYekRUWGlEb3BPU1NtL1dMRFNBZWcwOXloNEJWYkFHQXgxRFhxNVlH?=
 =?utf-8?B?blI5S3FtOXJBUXp4azdMc2poYUxhSXZ1NGpXYmpkVE1la0txWUVNYXl6cSs0?=
 =?utf-8?B?ZjZ0TG1tdmROdTdMaUVkd3ZUa085TGlDeTN1QkpMZGgxdkVSeHNQRWR2YkZU?=
 =?utf-8?B?eDZBa0hTNlZLb2FvS2lqZUl4bGVwNkpjNlBxcFN1K1VJNlNJMEpndEIyTVNu?=
 =?utf-8?B?ekEwSUZreGNYV3FBRzFNSjNYdkF2NDBPT2NucVpaZ2lKR2FQNGdyNUJvZjJt?=
 =?utf-8?B?OWx2RjJBMGJ1czNWV1dmU1BuVHJ6dHhFcDVGdEJETlcyS2tyYUt2aFJ1UStN?=
 =?utf-8?B?Qm5aQWpIZkhLMHpoREtUL3NzVCtHMXN4R3g2bGRQaDQ2bnJqR09jOGRZS1FJ?=
 =?utf-8?B?dDcwTEFkTFQ1d0lSV1hySGZyYm5CVG5LdWFtOFlsaTlFdmNza3BOR0FQOS9a?=
 =?utf-8?B?anRSZFNXTGRRMEhncHRnSHd0NENxMFkxNTJwS1RTWSt5YmdqN3I4Ny85ZHpN?=
 =?utf-8?B?U3c9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 212e877e-3d6f-46da-2724-08dab2b92ca7
X-MS-Exchange-CrossTenant-AuthSource: CY4PR11MB1862.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Oct 2022 16:36:01.6067
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OsFr9/1pdZf4BmJTb5EW1sJbrh8Qp+s3qau0Z1+oFjIPSUcw1tmS7F8JLj0riPxE+iuzWL0unE+yahnXlfN9QjJDiKbdi7uC97MM7vx76AM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR11MB6186
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_NONE,URIBL_BLOCKED autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Shawn,

On 10/19/2022 10:55 PM, Shawn Wang wrote:
> Hi Reinette,
> 
> Sorry for replying now due to other things.

No problem.

> 
> On 10/12/2022 7:48 AM, Reinette Chatre wrote:
>> Hi Shawn,
>>
>> Thank you very much for working on getting this fixed.
>>
>> On 10/9/2022 1:36 AM, Shawn Wang wrote:

...

>>> diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
>>> index 1dafbdc5ac31..2c719da5544f 100644
>>> --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
>>> +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
>>> @@ -338,6 +338,8 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)
>>>                   msr_param.high = max(msr_param.high, idx + 1);
>>>               }
>>>           }
>>> +        /* Clear the stale staged config */
>>> +        memset(d->staged_config, 0, sizeof(d->staged_config));
>>>       }
>>>         if (cpumask_empty(cpu_mask))
>>
>> Please also ensure that the temporary storage is cleared if there is an
>> early exist because of failure. Please do not duplicate the memset() code
>> but instead move it to a common exit location.
>>
> 
> There are two different resctrl_arch_update_domains() function call paths:
> 
> 1.rdtgroup_mkdir()->rdtgroup_mkdir_ctrl_mon()->rdtgroup_init_alloc()->resctrl_arch_update_domains()
> 2.rdtgroup_schemata_write()->resctrl_arch_update_domains()
> 
> Perhaps there is no common exit location if we want to clear staged_config[] after every call of resctrl_arch_update_domains().

I was referring to a common exit out of resctrl_arch_update_domains().

Look at how resctrl_arch_update_domains() behaves with this change:

resctrl_arch_update_domains()
{
	...

	if (!zalloc_cpumask_var(&cpu_mask, GFP_KERNEL))
		return -ENOMEM;

	...
	list_for_each_entry(d, &r->domains, list) {
		...
		memset(d->staged_config, 0, sizeof(d->staged_config));
	}


	...
done:
	free_cpumask_var(cpu_mask);
	
	return 0;
}


The goal of this fix is to ensure that staged_config[] is cleared on
return from resctrl_arch_update_domains() so that there is no stale
data in staged_config[] when resctrl_arch_update_domains() is called
again.

Considering this, I can see two scenarios in the above solution where
staged_config[] is not cleared on exit from resctrl_arch_update_domains():
1) If the zalloc_cpumask_var() fails then it returns -ENOMEM right away
   without clearing staged_config[].
2) If there are no domains associated with the resource (although, this
   seems not possible because that would imply no CPUs are online ...
   but let's make this code robust against strange scenarios).

What I referred to with a "common exit location" was something like this:

resctrl_arch_update_domains()
{
	...
	int ret = -EINVAL;
	...

	if (!zalloc_cpumask_var(&cpu_mask, GFP_KERNEL)) {
		ret = -ENOMEM;
		goto done;
	}

	...
	list_for_each_entry(d, &r->domains, list) {
		...
	}


	...
	ret = 0;
done:
	free_cpumask_var(cpu_mask);
	memset(d->staged_config, 0, sizeof(d->staged_config));	
	return ret;
}

Something like the above will ensure that staged_config[] is
always cleared on exit from resctrl_arch_update_domains().

Reinette