Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36;
Message-ID: <68e2a732-3016-a8a6-3f16-3c9ea1a0f9f6@intel.com>
Date:   Thu, 14 Sep 2023 10:29:54 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Firefox/102.0 Thunderbird/102.15.0
Subject: Re: [PATCH 1/5] selftests/resctrl: Extend signal handler coverage to
 unmount on receiving signal
Content-Language: en-US
To:     =?UTF-8?Q?Ilpo_J=c3=a4rvinen?= <ilpo.jarvinen@linux.intel.com>
CC:     Shuah Khan <shuah@kernel.org>,
        Shuah Khan <skhan@linuxfoundation.org>,
        <linux-kselftest@vger.kernel.org>,
        =?UTF-8?Q?Maciej_Wiecz=c3=b3r-Retman?= 
        <maciej.wieczor-retman@intel.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>,
        <stable@vger.kernel.org>
References: <20230911111930.16088-1-ilpo.jarvinen@linux.intel.com>
 <20230911111930.16088-2-ilpo.jarvinen@linux.intel.com>
 <4176a620-4cec-5d57-42a3-a15c0fe3eb73@intel.com>
 <aab4b1cc-6eb5-c324-e97e-c6699e2d165@linux.intel.com>
 <d6cc3829-9b7d-6a0b-c734-c33f6a66959c@intel.com>
 <9dfe761f-6779-44c6-c227-5175d53ac25@linux.intel.com>
 <b32f4bd1-9e99-3daa-9d39-8f241b41170c@intel.com>
 <52478b0-15e6-9072-df34-612ec0ebfa@linux.intel.com>
From:   Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <52478b0-15e6-9072-df34-612ec0ebfa@linux.intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bHg3amNrUUlsRFcxV1dGMStoeENBUlVnMDBVSFl3cDFEZC96bzBmclFSYWdl?=
 =?utf-8?B?WWl4dXFZeUlWWWFreEZIR1BjbnJCN291c21Kb1NHMERvTmVRbnVGK3kwUUdo?=
 =?utf-8?B?SmlaTVFIRmtkeUdVenh2emc5UXlBVW41OXN2MngvWGVPSHExa0dWbXQzRnpS?=
 =?utf-8?B?eVg1K01wdWhkUlprSGdrQVlZNDBiWGNaaVc1UUMwR3lpcjNqRTN1WGxKRXg4?=
 =?utf-8?B?TUdUY0hQOXZIWm81VU5mUlh2cEI0cURyMXlKam9UY0hCNGdXWHVqU3ZmaHE0?=
 =?utf-8?B?LzhUeDRMV3l1Y0ZaUUhaM1FFMll0TjFtejM5OVEvczZDNnJFLzB0bFBCSFZY?=
 =?utf-8?B?UjZlVm5jcDlZVUFYRFNLZGN5dGxXNDlxNzNIazJUWnBndWRPTDJjVU84UnlV?=
 =?utf-8?B?N2d4SVlUSVl6ajdMRytHZWdTdEFWNkJTMHRwNXNLTDh6QjNXSERZU2w0YTFG?=
 =?utf-8?B?UlRJa2FEa280YWNudXZHODhjVDBraGdYdkxHUDEyRW9ram1OV3VmUGJ5Qk9B?=
 =?utf-8?B?SXpQZC9MbHR1Yjd2MC9jNnJwNldxUFlEcXFBeThTakdTdThnSHN1VHFvejZw?=
 =?utf-8?B?UGZNcjZyNURvVVppSHZQZW0vd2Z0L0Jub0t5d1YvZmFQaEY5VU1pQ1J3UUxr?=
 =?utf-8?B?TldlRXlyNlBFN3BkaDlEbmNKcS9BaUF0eDNvZnAvNlVuUW9hd2dhRGljQjFH?=
 =?utf-8?B?bjU5ejR3OStTVGRoWDFKMXJHUzhiL2E5a0hudFYvM2JKQ2ZEaGpUS0hvV0M4?=
 =?utf-8?B?ME8rQWdabHh4ZWNiZWVMUlgyL3kyY0hWZWFMSE1pZGZnQlliRC85ZzZhR0Ey?=
 =?utf-8?B?VllIWk14NmplVURyRGZ2RDZXVDBnSTM4SlY2Yk1OQ3poNXlWTjhMc0FwOUlX?=
 =?utf-8?B?TW4zaFdqS3RtaEszd2RsSHVKS0dhTVhMZ2dTSitjWDdMcWg2cnF2NnNqMGZ0?=
 =?utf-8?B?enpUQmJJOEY5TzNFVGdUbFhFRmh4U0hiZ0RVbit0QmMvTzVqQVAwNmRrdFVO?=
 =?utf-8?B?WDZxSnA2ZkhXM3BTOWJPSUlob2hQaHdFM0p2OXIyVVpjdWR5dDdEMFpoVmxz?=
 =?utf-8?B?TFdGUE1hVXR0RHN2N0E2M1cwOWU1SVQ4Vk1qNEFpUkFQMElEQkFmak5kc0VQ?=
 =?utf-8?B?aWxaeStUZHBFOUFoVDRJZktnTjVibHY2bTFTWVd6dHFCU3V2bG4rT3NVd2Fo?=
 =?utf-8?B?MW1nSWNaYTZTWVR0QzJaM3NlZWM1WXE4bHc1MGw5c3h2M3BFclB0NGZ5UzRr?=
 =?utf-8?B?U3NxTzIxaXBtblgwakxOZGpLMlNkS21LVnpYZCtPQkJUYkVSd3hzNWZDQmxl?=
 =?utf-8?B?a2J3S1N3OHExSm9YdGhYQ1VTczNWR1d2YVR4QUxqd2pEejc3UWVqMWxYN3ZO?=
 =?utf-8?B?T0xvWlFPVS9TUndJdXVQUWh0eFhJYXAwM3czb1ozRk42SlYvSjRQOWhZaldG?=
 =?utf-8?B?NUdIWFBHa0hrdFpUVy9XMlJDSTBCalUxRktLdTFTdGFaRnlSK2FVaGxpdUZD?=
 =?utf-8?B?NkQ4dE1yU2N1NGJmNStTTmtkazR3KzRjSFZ0Ym1FaEZFK09Sb21MbmhHZmJH?=
 =?utf-8?B?Tjl4NDd2Q2tUcDdvVGxmQTlCbUx3K3BrTnc0YzdpN2VMS1ZYNlFSWldxN09x?=
 =?utf-8?B?QmFUVUg1RUFTcHBHck1YWlBTWnQ0YUVyUmJGRldnaTY3MGVaelhVZkwra2hu?=
 =?utf-8?B?UkkvOHE0OGZUTVdNa09LMkt4S3pYTml3V3AxaklSZFBnNVh4VSs4aXJpdWRi?=
 =?utf-8?B?cnp4L0Y5UkNlYm1ndE9tSHBjTDRCODRXa0tWWmRodnpGU2VyNXNMMTBXazhm?=
 =?utf-8?B?Wk84ZnFXTjJqeDRxMCtVazhVRjNpOGwwQXgwUVRrZDA1clU0Tk5pd1NqbjBE?=
 =?utf-8?B?dkVodW16alQ5Z3h2T0RTNStKRW1iVGc5TGZxL2ZYRTJ2Yy9Id0gxY2cyWGQ2?=
 =?utf-8?B?WEpUenp6cFRUaytTL2JyTFAxN0ROckZXeElnejJTVVZJWXJ2T3duYk5QZ1hv?=
 =?utf-8?B?TzdBNmgrUFFzWFFBbHBDZTZTbjBUZ3VSQ2ord01kRUxuWEVmRzVWMWxMWUxI?=
 =?utf-8?B?SUwwNjVJOFdTTHIvQXhhRnRFVU9ISXdyeUtzN2NpclZVb1pTOWZ6djVyYytP?=
 =?utf-8?B?ZUNKMmRrRHhwZzRZYW93dnpKKy9EUk1oSGZNN3lLTDA3K0VPYjdOb0VrNEZG?=
 =?utf-8?B?MlE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 2c47fb68-749e-4e5e-dc00-08dbb54837ab
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2023 17:29:58.0593
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: aJK9xu6iXQzdh55rs2/gk2+NeCE+twBu76N7p70t8xhvpzlcajdd/bfo5bOfkicg82zVYwwu8lSkblZBmlRC0NhsaRBo5/lslDb/9KZg7z4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR11MB7950
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Thu, 14 Sep 2023 10:30:32 -0700 (PDT)
X-Spam-Status: No, score=-2.3 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email

Hi Ilpo,

On 9/14/2023 10:05 AM, Ilpo Järvinen wrote:
> On Thu, 14 Sep 2023, Reinette Chatre wrote:
>> On 9/14/2023 3:16 AM, Ilpo Järvinen wrote:
>>> On Wed, 13 Sep 2023, Reinette Chatre wrote:
>>>> On 9/13/2023 3:01 AM, Ilpo Järvinen wrote:
>>>>> On Tue, 12 Sep 2023, Reinette Chatre wrote:
>>>>>> On 9/11/2023 4:19 AM, Ilpo Järvinen wrote:
>>>>>>> Unmounting resctrl FS has been moved into the per test functions in
>>>>>>> resctrl_tests.c by commit caddc0fbe495 ("selftests/resctrl: Move
>>>>>>> resctrl FS mount/umount to higher level"). In case a signal (SIGINT,
>>>>>>> SIGTERM, or SIGHUP) is received, the running selftest is aborted by
>>>>>>> ctrlc_handler() which then unmounts resctrl fs before exiting. The
>>>>>>> current section between signal_handler_register() and
>>>>>>> signal_handler_unregister(), however, does not cover the entire
>>>>>>> duration when resctrl FS is mounted.
>>>>>>>
>>>>>>> Move signal_handler_register() and signal_handler_unregister() call
>>>>>>> into the test functions in resctrl_tests.c to properly unmount resctrl
>>>>>>> fs. Adjust child process kill() call in ctrlc_handler() to only be
>>>>>>> invoked if the child was already forked.
>>>>>>
>>>>>> Thank you for catching this.
>>>>>>
>>>>>>>
>>>>>>> Fixes: caddc0fbe495 ("selftests/resctrl: Move resctrl FS mount/umount to higher level")
>>>>>>> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
>>>>>>> Cc: <stable@vger.kernel.org>
>>>>>>> ---
>>>>>>>  tools/testing/selftests/resctrl/cat_test.c    |  8 -------
>>>>>>>  .../testing/selftests/resctrl/resctrl_tests.c | 24 +++++++++++++++++++
>>>>>>>  tools/testing/selftests/resctrl/resctrl_val.c | 22 ++++++++---------
>>>>>>>  3 files changed, 34 insertions(+), 20 deletions(-)
>>>>>>>
>>>>>>> diff --git a/tools/testing/selftests/resctrl/cat_test.c b/tools/testing/selftests/resctrl/cat_test.c
>>>>>>> index 97b87285ab2a..224ba8544d8a 100644
>>>>>>> --- a/tools/testing/selftests/resctrl/cat_test.c
>>>>>>> +++ b/tools/testing/selftests/resctrl/cat_test.c
>>>>>>> @@ -167,12 +167,6 @@ int cat_perf_miss_val(int cpu_no, int n, char *cache_type)
>>>>>>>  		strcpy(param.filename, RESULT_FILE_NAME1);
>>>>>>>  		param.num_of_runs = 0;
>>>>>>>  		param.cpu_no = sibling_cpu_no;
>>>>>>> -	} else {
>>>>>>> -		ret = signal_handler_register();
>>>>>>> -		if (ret) {
>>>>>>> -			kill(bm_pid, SIGKILL);
>>>>>>> -			goto out;
>>>>>>> -		}
>>>>>>>  	}
>>>>>>>  
>>>>>>>  	remove(param.filename);
>>>>>>> @@ -209,10 +203,8 @@ int cat_perf_miss_val(int cpu_no, int n, char *cache_type)
>>>>>>>  		}
>>>>>>>  		close(pipefd[0]);
>>>>>>>  		kill(bm_pid, SIGKILL);
>>>>>>> -		signal_handler_unregister();
>>>>>>>  	}
>>>>>>>  
>>>>>>> -out:
>>>>>>>  	cat_test_cleanup();
>>>>>>>  
>>>>>>>  	return ret;
>>>>>>> diff --git a/tools/testing/selftests/resctrl/resctrl_tests.c b/tools/testing/selftests/resctrl/resctrl_tests.c
>>>>>>> index 823672a20a43..3d66fbdc2df3 100644
>>>>>>> --- a/tools/testing/selftests/resctrl/resctrl_tests.c
>>>>>>> +++ b/tools/testing/selftests/resctrl/resctrl_tests.c
>>>>>>> @@ -73,8 +73,13 @@ static void run_mbm_test(const char * const *benchmark_cmd, int cpu_no)
>>>>>>>  
>>>>>>>  	ksft_print_msg("Starting MBM BW change ...\n");
>>>>>>>  
>>>>>>> +	res = signal_handler_register();
>>>>>>> +	if (res)
>>>>>>> +		return;
>>>>>>> +
>>>>>>>  	res = mount_resctrlfs();
>>>>>>>  	if (res) {
>>>>>>> +		signal_handler_unregister();
>>>>>>>  		ksft_exit_fail_msg("Failed to mount resctrl FS\n");
>>>>>>>  		return;
>>>>>>>  	}
>>>>>>> @@ -91,6 +96,7 @@ static void run_mbm_test(const char * const *benchmark_cmd, int cpu_no)
>>>>>>>  
>>>>>>>  umount:
>>>>>>>  	umount_resctrlfs();
>>>>>>> +	signal_handler_unregister();
>>>>>>>  }
>>>>>>>  
>>>>>>>  static void run_mba_test(const char * const *benchmark_cmd, int cpu_no)
>>>>>>> @@ -99,8 +105,13 @@ static void run_mba_test(const char * const *benchmark_cmd, int cpu_no)
>>>>>>>  
>>>>>>>  	ksft_print_msg("Starting MBA Schemata change ...\n");
>>>>>>>  
>>>>>>> +	res = signal_handler_register();
>>>>>>> +	if (res)
>>>>>>> +		return;
>>>>>>> +
>>>>>>>  	res = mount_resctrlfs();
>>>>>>>  	if (res) {
>>>>>>> +		signal_handler_unregister();
>>>>>>>  		ksft_exit_fail_msg("Failed to mount resctrl FS\n");
>>>>>>>  		return;
>>>>>>>  	}
>>>>>>> @@ -115,6 +126,7 @@ static void run_mba_test(const char * const *benchmark_cmd, int cpu_no)
>>>>>>>  
>>>>>>>  umount:
>>>>>>>  	umount_resctrlfs();
>>>>>>> +	signal_handler_unregister();
>>>>>>>  }
>>>>>>>  
>>>>>>
>>>>>> This adds more duplicated code for every test. Have you considered a
>>>>>> single test setup function that can be used to mount resctrl FS and setup
>>>>>> the signal handler paired with a single test teardown function?
>>>>>
>>>>> Yes. Consolidating all these is among my not-yet submitted patches.
>>>>> I just had to do a backport-friendly Fixes patch first for this.
>>>>>
>>>>
>>>> Could you please help me understand how the duplicate calls are more
>>>> backport friendly?
>>>
>>> Hi,
>>>
>>> It's simply because the refactoring that has to be done to be able to 
>>> introduce the generalized test framework is much more invasive and far 
>>> reaching than this patch. Essentially, all the call signatures of the test 
>>> functions need to match and the feature checks need to be done in new per 
>>> test functions too. This is the diffstat of those changes alone:
>>>
>>>  tools/testing/selftests/resctrl/cat_test.c      |  21 +++--
>>>  tools/testing/selftests/resctrl/cmt_test.c      |  26 +++--
>>>  tools/testing/selftests/resctrl/mba_test.c      |  20 +++-
>>>  tools/testing/selftests/resctrl/mbm_test.c      |  20 +++-
>>>  tools/testing/selftests/resctrl/resctrl.h       |  43 ++++++++-
>>>  tools/testing/selftests/resctrl/resctrl_tests.c | 220 +++++++++++++++----------------------------
>>>  tools/testing/selftests/resctrl/resctrlfs.c     |   5 +
>>>
>>> (tools/testing/selftests/resctrl/resctrl_tests.c --- part would 
>>> be slightly less if I'd reorder this patch but that only 24 lines off as 
>>> per diffstat of this patch).
>>>
>>> But that's not all.... To be able to push the generalized test framework 
>>> to stable, you need to also count in the benchmark cmd changes which 
>>> worked towards making the call signatures identical. So here's the 
>>> diffstat for that series for quick reference:
>>>
>>>  tools/testing/selftests/resctrl/cache.c       |   5 +-
>>>  tools/testing/selftests/resctrl/cat_test.c    |  13 +--
>>>  tools/testing/selftests/resctrl/cmt_test.c    |  34 ++++--
>>>  tools/testing/selftests/resctrl/mba_test.c    |   4 +-
>>>  tools/testing/selftests/resctrl/mbm_test.c    |   7 +-
>>>  tools/testing/selftests/resctrl/resctrl.h     |  16 +--
>>>  .../testing/selftests/resctrl/resctrl_tests.c | 100 ++++++++----------
>>>  tools/testing/selftests/resctrl/resctrl_val.c |  10 +-
>>>
>>> That's ~500 lines changed vs ~50 so it's a magnitude worse and much less 
>>> localized.
>>>
>>> And rest assured, I did not like introducing the duplicated calls any more 
>>> than you do (I did not write the generalized test framework for nothing, 
>>> after all) but the way taken in this patch seemed the most reasonable 
>>> option under these circumstances.
>>>
>>
>> hmmm ... I did not expect that a total refactoring would be needed.
>>
>> I was thinking about a change from this:
>>
>>
>> 	testX(...) 
>> 	{
>> 	
>> 		res = signal_handler_register();
>> 		/* error handling */
>> 		res = mount_resctrlfs();
>> 		/* error handling */
>> 		
>> 		/* test */
>>
>> 		unmount_resctrlfs();
>> 		signal_handler_register();
>>
>> 	}
>>
>>
>> to this:
>>
>>
>> 	int test_setup(...)
>> 	{
>> 		res = signal_handler_register();
>> 		/* error handling */
>> 		res = mount_resctrlfs();
>> 		/* error handling */
>> 	}
>>
>>
>> 	void test_cleanup(...)
>> 	{
>> 		unmount_resctrlfs();
>> 		signal_handler_register();
>> 	}
>>
>>
>> 	testX(...)
>> 	{
>>
>> 		res = test_setup(..);
>> 		/* error handling */
>>
>> 		/* test */
>>
>> 		test_cleanup();
>> 	}
>>
>> I expect this to also support the bigger refactoring.
> 
> Okay, I'll do so then.
> 
> However, having already written the generic run_single_test() function 
> that is part of the generic test framework, I definitely don't feel those 
> helpers would be that helpful for it. It more feels like they'd make the 
> flow less obvious by adding those two extra calls there but that's of 
> course matter of taste.

Sounds like there is some room for improvement here, perhaps open coding
the test_setup() and test_cleanup() helpers within run_single_test().
This is purely speculation on my part as I have not seen the code.

Reinette