Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8;
Message-ID: <56769c01-e21d-4775-94f0-ef9f863c1e92@intel.com>
Date:   Thu, 2 Nov 2023 10:52:24 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 18/24] selftests/resctrl: Introduce generalized test
 framework
Content-Language: en-US
To:     =?UTF-8?Q?Ilpo_J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>,
        <linux-kselftest@vger.kernel.org>, Shuah Khan <shuah@kernel.org>,
        "Shaopeng Tan" <tan.shaopeng@jp.fujitsu.com>,
        =?UTF-8?Q?Maciej_Wiecz=C3=B3r-Retman?= 
        <maciej.wieczor-retman@intel.com>,
        Fenghua Yu <fenghua.yu@intel.com>
CC:     <linux-kernel@vger.kernel.org>
References: <20231024092634.7122-1-ilpo.jarvinen@linux.intel.com>
 <20231024092634.7122-19-ilpo.jarvinen@linux.intel.com>
From:   Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <20231024092634.7122-19-ilpo.jarvinen@linux.intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Y3FsbVVDUlNsWWVWZitGcitGaFBETzNuWXhOanVqdkNRQnUxQURCcUNjSTVE?=
 =?utf-8?B?S0ZmZVZCdWlibTFJd29WSGEvY1BnN1R0SzR3UnFHczVvT2JYUUNobjBtOXVm?=
 =?utf-8?B?Ky9WeUREcjY4WDN1MHd1M1dDWDhIM3JZSmZVeGtLaHp6cFZGdzRQQzRIbjB4?=
 =?utf-8?B?Ky9uZ0pSdER6N0xIakhISmRLOGRGZE1JNHg2TTNzOEpKYXFtTEIwbldXQTI3?=
 =?utf-8?B?ck5PRUM0ZjdNUG1yZ3VzTUEvNzdOVmtPellHRFpnNXNnbnE4aXA5R0ZvR1lK?=
 =?utf-8?B?VHlRMUNjSS80Z0N4VFpJd0VXMmJGekRUeVRFa1l1UFNCcVBmL0d4VUJ6YlNz?=
 =?utf-8?B?TkJOU1BlUjVzUFA2TUc1cjQ3bDJvYWZWZjFBMTZ0L3haTmJKakM3NDRYZzEy?=
 =?utf-8?B?dGR4Z3N0WkFOajYvV2VvTklxOUtGV1g0b25xYWFGMGVpSHVzQ2trRzVOVzU4?=
 =?utf-8?B?aVdRd2lSR2YwYVlyVlhERmFsTjlseVpJMm1wYkMwaVp3VStsR2I3a2NlMk1O?=
 =?utf-8?B?RXk5UWk5VU5sVzkwckMzSmZ5NERFZmJhYmZsQWpMTHpqNGt0VmJwUVAzdE1m?=
 =?utf-8?B?cHhTay9XNi91N2YyQ1RpOXo5Q2VnUGhqSGZGQ3hLV0wwUGNIdzE3YWNJVWIx?=
 =?utf-8?B?V0NvWHpYckRZMEk5OFRHUU0vR29TSUxOMlBzSS9sWmNCaExtZkM1TGhudTdo?=
 =?utf-8?B?RUFmbUE1a1lxd0JDbHZUbVNoS2ZPVEZPZG5iS2lZdGJ6dFhxVEpPTFExQjZW?=
 =?utf-8?B?VGt4K3R1RDR4c1hrYlJWT3lqU2xNVVNFdk1KZXFxVVVLdnRBOFp5WmZjUkU0?=
 =?utf-8?B?L2tjcXZVOGEwbjZobkdLemtuakpQOXBrdW9PS3BLeVRTNGtEZVZpMGY1U0cv?=
 =?utf-8?B?Y1FGRkc0MjJLWHQzT2NrdjhrbTdaSzhHelN3a3FIRVJuS1Vpa2c1UlRhSlE5?=
 =?utf-8?B?Q3REKzZtVGhlT3RHTVJJRHhZR0haV1FvZE5sTEppOU10ZXdtdzU4ZkpNTmVn?=
 =?utf-8?B?MTNBTDRvOTQzTmo3dHUzamdOSml2dS90N1NsSGZjTGF3RDlLcFhOYmh6MjFz?=
 =?utf-8?B?QThqWnVFU2lFM3VJSXlaMld0dDlhTmRDTUREdDd0RmpsZXM3MHd1TXI3a1FB?=
 =?utf-8?B?QkY0eWdOcEhoTkR0SFM5d0pQdnBMbHovMTAxZ090Z2MvZmU0QjVzMmljQW5h?=
 =?utf-8?B?OVlrYktpd1lOSWRUUVhFZTE2QWtCODVESitVTzY5eWVVUC9ndzNlOW9BMW9D?=
 =?utf-8?B?MGZna3VPUUxFKzRLcmpUQWRlU29ZVVJVeXNVM1VxS3JsWEZGWkpEMFpzS0tK?=
 =?utf-8?B?bkgydUlBSC9XVXl1QmhQYklVOFJsMmhwb3BaVm5iUi9oaWcyODhHK0tXQXho?=
 =?utf-8?B?TGQ1RytIbTZBcVl2TjZXaUFaTVVsSFdsNUd4UzI4dStCaXNYZHJOcWpCanF3?=
 =?utf-8?B?V2kvYk8zU1lZeU1kKzYvbjY1MlBlZmxmN1F3d0JYVG42RWhtSU1LanpuU2gv?=
 =?utf-8?B?NnQ3WUh5MHlka2U5T0pUSllOazZRWk1ub1dCVWJUVVRDRnVRcnN0dk5qRWJY?=
 =?utf-8?B?WkRkczdCRGFSZzI2cVBBYVJoZ0QvZ2poNFZOMmJOMnNxdloxZTJXRXluaFRJ?=
 =?utf-8?B?S3Q4bXE3KzJzVGQ5czRvWm02c1lsV2UxUFdSQzd4ZnAzcHdwQW02WDV5UUJF?=
 =?utf-8?B?WVhmakVPd2J3N2xPUUpkR0pVZGYvVUd3MU9lNTJyMm5GNVJXMkg2UFZYUFJI?=
 =?utf-8?B?bmszMWl4Y0ZMOXdmSVRRaFJiRXNZTkZNd3NQVHJVU2cza3g4V1l4SW1xaUZq?=
 =?utf-8?B?Q0pzNUs2MlgyaFp6TkJLUXJMM3Zsa3BEYU5TMVh1RXRxZ20xU2JTY3J4TFAy?=
 =?utf-8?B?aThFL0x0MU9IUnJXMzdDWUd2NVoweTQ4ODkxUmRqTEYvand6WFpiVE00b0Z6?=
 =?utf-8?B?VDNFNzVXcm4zVWtoMzdBSXVKYksvbG5PQXZCdFNHNEczaXVCOXFlUmlxY0dH?=
 =?utf-8?B?WHdNOElmYlJsbFNCcW5LMzRzSU82RXpsRUgzNGpHangzb21ham1SODNOZGZ2?=
 =?utf-8?B?SWliWCtRSXpiS0RqbXNjUnRVekl4WkZQbngyZG1xT3hQdGVNK2JhakFNOEc5?=
 =?utf-8?B?enNVU0hxNFBSanhvZmRBK3RucnYxUG9qNHJXWWp3eWhnZkVBQjQzL3ppZVNa?=
 =?utf-8?B?d3c9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 60832524-c153-4a26-a234-08dbdbcc78fa
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Nov 2023 17:52:25.4099
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dUMyY0hpJbxDqW162I1qWB2Yv6HL7oTGniuE9l8fz1OB/5bNqEPUcvD8D2oHpkzdakDRS+mrCZ3ZMgmMM14Y8xzF0HLQmKl2V9AS08pjRX4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR11MB8361
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-1.3 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,
	URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Thu, 02 Nov 2023 10:53:17 -0700 (PDT)

Hi Ilpo,

On 10/24/2023 2:26 AM, Ilpo Järvinen wrote:
...
> diff --git a/tools/testing/selftests/resctrl/resctrl.h b/tools/testing/selftests/resctrl/resctrl.h
> index ec6efd36f60a..e017adf1390d 100644
> --- a/tools/testing/selftests/resctrl/resctrl.h
> +++ b/tools/testing/selftests/resctrl/resctrl.h
> @@ -37,6 +37,8 @@
>  
>  #define DEFAULT_SPAN		(250 * MB)
>  
> +#define ARRAY_SIZE(arr)		(sizeof(arr) / sizeof((arr)[0]))
> +

Shuah worked hard to remove all these copies within kselftest. Please
use the one in kselftest.h instead.

>  #define PARENT_EXIT(err_msg)			\
>  	do {					\
>  		perror(err_msg);		\


...

> @@ -233,25 +183,26 @@ int main(int argc, char **argv)
>  		case 't':
>  			token = strtok(optarg, ",");
>  
> -			mbm_test = false;
> -			mba_test = false;
> -			cmt_test = false;
> -			cat_test = false;
> +			if (!test_param_seen) {
> +				for (i = 0; i < ARRAY_SIZE(resctrl_tests); i++)
> +					resctrl_tests[i]->disabled = true;
> +				tests = 0;
> +				test_param_seen = true;
> +			}
>  			while (token) {
> -				if (!strncmp(token, MBM_STR, sizeof(MBM_STR))) {
> -					mbm_test = true;
> -					tests++;
> -				} else if (!strncmp(token, MBA_STR, sizeof(MBA_STR))) {
> -					mba_test = true;
> -					tests++;
> -				} else if (!strncmp(token, CMT_STR, sizeof(CMT_STR))) {
> -					cmt_test = true;
> -					tests++;
> -				} else if (!strncmp(token, CAT_STR, sizeof(CAT_STR))) {
> -					cat_test = true;
> -					tests++;
> -				} else {
> -					printf("invalid argument\n");
> +				bool found = false;
> +
> +				for (i = 0; i < ARRAY_SIZE(resctrl_tests); i++) {
> +					if (!strcasecmp(token, resctrl_tests[i]->name)) {
> +						if (resctrl_tests[i]->disabled)
> +							tests++;
> +						resctrl_tests[i]->disabled = false;
> +						found = true;
> +					}
> +				}

Could providing multiple "-t" result in the test count not
matching the number of tests run?

> +
> +				if (!found) {
> +					printf("invalid test: %s\n", token);
>  
>  					return -1;
>  				}

A great improvement, thanks.

Reinette