Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
Message-ID: <0033ee44-5383-4e69-b85d-ed88592bee94@intel.com>
Date:   Wed, 4 Oct 2023 10:43:04 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/6] selftests/resctrl: Extend signal handler coverage
 to unmount on receiving signal
Content-Language: en-US
To:     "Shaopeng Tan (Fujitsu)" <tan.shaopeng@fujitsu.com>,
        =?UTF-8?Q?Ilpo_J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>,
        Shuah Khan <shuah@kernel.org>,
        Shuah Khan <skhan@linuxfoundation.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        =?UTF-8?Q?Maciej_Wiecz=C3=B3r-Retman?= 
        <maciej.wieczor-retman@intel.com>
CC:     LKML <linux-kernel@vger.kernel.org>,
        "stable@vger.kernel.org" <stable@vger.kernel.org>
References: <20230915154438.82931-1-ilpo.jarvinen@linux.intel.com>
 <20230915154438.82931-2-ilpo.jarvinen@linux.intel.com>
 <d2d94def-742d-7661-3632-b9f7e4996415@intel.com>
 <TYAPR01MB6330F248767F48A2A649CDE58BC1A@TYAPR01MB6330.jpnprd01.prod.outlook.com>
From:   Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <TYAPR01MB6330F248767F48A2A649CDE58BC1A@TYAPR01MB6330.jpnprd01.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZkxCRmJDUjZMRVVaNkJacXczcXJ0TEx0YUMyaVhvSDUwNFFPbWZVam55Y1JK?=
 =?utf-8?B?bFRZSW4zRnl1VTk4OU81bW9Sc0hYakFUeUdaaGM0V1V2SU0yMWxwbXFtdjJI?=
 =?utf-8?B?V0NSMWtha1BSV0o0R1lMWUtUYkU3b3EvVmtPNW5MQW8rT24waG5JdlREeWhs?=
 =?utf-8?B?a0pDQk5SUnh1aFAzMU9PWjUwRndiMXB3cnc1ZXdYM3Z4RG5Za2h4Sm9SYlUw?=
 =?utf-8?B?YTBIYTNhR1RFMDVRZlhMejA5OEozV0dsQkdEV0FZTS8vaVIvTUNQS0lQZXhz?=
 =?utf-8?B?amQycWNncWxROTRFYlo3eGlHOE1qL0FydUI1dkZmV0RsdTlaV3RXL3p1eU9t?=
 =?utf-8?B?ZkhvdWtHTFJZS1NEWkdxUm1ybzdIUThkckl5bzNMUWpSQ25qQUU0OGVGUUho?=
 =?utf-8?B?NGdWVHFYems0R1pOcFdiQlQ3TWlmNUNMemFvVUpMTFBLYnNGOEo1c2pDYTN2?=
 =?utf-8?B?dUtqVzNHcHVFQU9sTFZSNXFQaUpJM0pzcVJzaE0xZndYUVNIaFl0aXE4SFJy?=
 =?utf-8?B?ZzhKeWRDR3U2d2g4Mnk1WXVSbnF3ZXBXKzZKQkdtZHYxalZUTFdlUEloLzFE?=
 =?utf-8?B?V1FNaTNFbitpM3pUVTdHM2tmb1NuWFdFUlhudGEyRDdOdVpISDdPMmZHcVlH?=
 =?utf-8?B?M1hvTlh5S0pLdUdNZFhwN3Y2L2ZIRjBaeHRmRG9BR1N2Z3hBTGhHQmFPN2JD?=
 =?utf-8?B?eFBhWDh4RGRpRXFpMjAzUldaaDVOZnZBRFBjQWNzdE1YTVBqN1pReWZ4Mlhi?=
 =?utf-8?B?ZGhlTU8vR3Y0M25yTnBja1BsWi95YTNYUzJkVFhMMFAzOVluODNwdWROWTVx?=
 =?utf-8?B?enVxTllDaDE3clJFajV1SHRUSUN6U0s3Y0tNdEI4eUVjSVgvclZ5R0dBY2Fq?=
 =?utf-8?B?VXhmc2NwUExmSUVFLytMUXJpOTVMUUpGYjhOTUppNDRZbldveFBBNzJGUVJH?=
 =?utf-8?B?K3ZWTHlxOU0ydU5adTJsOEVHZkFBU3A3NVJrNFhmSWg0dW9IZnh0VS8waE5B?=
 =?utf-8?B?V3NZYTRCM0VvUlp1WHJESzVtT1Roa2ltZHVPbCtXVUs5RjZ5dDdCL0kvZWll?=
 =?utf-8?B?QjZ6VzBMWE1SUVlociswNVh1UFNKUW50YkNHb3VFVW9CMkZ3TzdjQlQzQWlY?=
 =?utf-8?B?aXZRV21XMXJLNTZkVVFFSkxFSW9reUpGVEJnREcvK2xQSHNYYjc0aVZWdkcv?=
 =?utf-8?B?b3F5OU9xUDVTdG9ZL0ZEZjZCUHNyOUk0emNsREhIRUtteWx4U2JVTndoQlo1?=
 =?utf-8?B?b0dpem1KZFIvNlMzejlYMGdhaTY1YjFIUnNqVUo1MTlLQ0pxMGhyb1kvTDBq?=
 =?utf-8?B?U0ZnZGlRUEhGY01WZHlYc3d0aVhjc21KaGVDSGUzeGNsTE9ZMDE4YmN2MTJI?=
 =?utf-8?B?WUpQTGhKNFErK2p3Mk0yTnR0UlJCNTVISm41bW02T0JWZ3N5NGh0Tm1ackZu?=
 =?utf-8?B?YUpJM1NYTy9FSk1WcEhXRkNJaXh6THk1QVdVcURkUVZPZkRTQWhDalU2dGg1?=
 =?utf-8?B?Uk5XV1VZQ0ZQdWhKeFJVQmQyRUttUVgzMWhINlptWE1NNGpxSkE5QTlpVkF4?=
 =?utf-8?B?QnJyYU9pWEJhMTcyL0hOYTB5djBqN3RQT3RZbmtrNmFSanV5SDJXYXAxMHY0?=
 =?utf-8?B?QUR2TXJpaTNkVFE4bXZLandFM3FyVUR3bFRuSUVlczM4cUpKYXMzQ0lPRnE5?=
 =?utf-8?B?YTI0MVoxTmozTHNaa2ZIUGRkemdjMDZGc0ZXSzNrMDZueDBIeU1KWG05eUhX?=
 =?utf-8?B?VCtUWDJTNzhFU1dmWmw5RU15RktMUnNFT1E2Q0wrS09vZjRLdGVLRnZiWHY5?=
 =?utf-8?B?cjBlZktieXhTaUNCVS9aMlZpSXc4T0tXSVpmaXczeW55ZGhsQUd6eTN5anIz?=
 =?utf-8?B?d2VCMFZvM1RGWUF3MklOSmtWMzQ2K1VXZlMwVlJGZzM5b3lqeEtoNFB0bEhF?=
 =?utf-8?B?RmdxQVBqYU1NR2piVTNxK1JDUUx5VGR0WlZIbVhYZFlKWDRXM2p3WncwZDV4?=
 =?utf-8?B?a3JrdTJrNlJzVU9NczhmYVc4b3g0TDZVUmxBK1pMSTFZa0dXZSs2OWZCK2tV?=
 =?utf-8?B?M2JhTzJrS3JqZGE1RDcvdjNqSElhWFY5TUc0NjlObFJ6Ujk2VUYyZ1ZPY0dT?=
 =?utf-8?B?bitVbTFCV2hBU3QwbEtsMHhjNEZXU3E4NDJCSU9oM1dmQUpzazRMc2JKcENV?=
 =?utf-8?B?enc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 24f0b0be-e640-4c68-b086-08dbc5015ed9
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Oct 2023 17:43:08.1885
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kf8eAsUsntbNOr90fnxZsWi0ZRp0DczfWg+zUltBUaZlxYKDBZiRxd4fUwpONY5Shkme0NH+UzsvhsALQilmrR2VVgGuiG3AMwMVea1Pfto=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB7254
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Wed, 04 Oct 2023 10:43:22 -0700 (PDT)

Hi Shaopeng,

On 9/28/2023 1:10 AM, Shaopeng Tan (Fujitsu) wrote:
>> On 9/15/2023 8:44 AM, Ilpo Järvinen wrote:

...

>>> +static void run_mbm_test(const char * const *benchmark_cmd, int
>>> +cpu_no) {
>>> +	int res;
>>> +
>>> +	ksft_print_msg("Starting MBM BW change ...\n");
>>> +
>>> +	if (test_prepare())
>>> +		return;
>>>
>>
>> I am not sure about this. With this exit the kselftest machinery is not aware of
>> the test passing or failing. I wonder if there should not rather be a "goto" here
>> that triggers ksft_test_result()? This needs some more thought though. First,
>> with this change test_prepare() officially gains responsibility to determine if a
>> failure is transient (just a single test
>> fails) or permanent (no use trying any other tests if this fails). For the former it
>> would then be up to the caller to call ksft_test_result() and for the latter
>> test_prepare() will call ksft_exit_fail_msg().
>> Second, that SNC warning may be an inconvenience with a new goto. Here it
>> may be ok to print that message before the test failure?
> 
> If a failure may be permanent, it may be best to detect it before running all tests, rather than in test_prepare().
> Now some detections are completed before running all tests. For example:
> 273         if (geteuid() != 0)
> 274                 return ksft_exit_skip("Not running as root. Skipping...\n");
> 275
> 276         if (!check_resctrlfs_support())
> 277                 return ksft_exit_skip("resctrl FS does not exist. Enable X86_CPU_RESCTRL config option.\n");
> 278
> 279         if (umount_resctrlfs())
> 280                 return ksft_exit_skip("resctrl FS unmount failed.\n");
> 

You are correct that the tests should aim to detect as early as possible if
no test has a chance of succeeding. This is covered in the checks you mention.
The purpose of test_prepare()/test_cleanup() pair is to perform actions that
should be done for every test. For example, resctrl is mounted before each
test and unmounted after each test. Since these actions are required to be done
for every test it cannot be a single call before all tests are run.

It may be possible to add a test_prepare() directly followed by a test_cleanup()
before any test is run to be more explicit about early detection but that
does not seem necessary considering the checks would be done anyway when the
first test is run. Even when doing so it would not eliminate the need for 
test_prepare()/test_cleanup() to form part of every test run and needing to exit
if, for example, a previous test triggered a fault preventing resctrl from
being mounted.

Reinette