Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38;
Message-ID: <4a8bf2e1-02e6-408a-85ff-aeaf66e1d700@intel.com>
Date:   Tue, 28 Nov 2023 14:12:05 -0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 07/26] selftests/resctrl: Exclude shareable bits from
 schemata in CAT test
Content-Language: en-US
To:     =?UTF-8?Q?Ilpo_J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>,
        <linux-kselftest@vger.kernel.org>, Shuah Khan <shuah@kernel.org>,
        "Shaopeng Tan" <tan.shaopeng@jp.fujitsu.com>,
        =?UTF-8?Q?Maciej_Wiecz=C3=B3r-Retman?= 
        <maciej.wieczor-retman@intel.com>,
        Fenghua Yu <fenghua.yu@intel.com>
CC:     <linux-kernel@vger.kernel.org>
References: <20231120111340.7805-1-ilpo.jarvinen@linux.intel.com>
 <20231120111340.7805-8-ilpo.jarvinen@linux.intel.com>
From:   Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <20231120111340.7805-8-ilpo.jarvinen@linux.intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OHRLckVCU2t1N1VDTEFTTzFoaHZ3ZUg3RmJ0TjR1TWpzSWM4N25IWFliajkr?=
 =?utf-8?B?OWRwbmhKSlFPSFJ5d09kN2dTTEJiSnNzYnRuY3lwNXRPVkJMNjNXVForMDla?=
 =?utf-8?B?SVZONWtqQnJ2M3hTd2hDdXpzNnRvWHI3bktXSTlVWFJlekxFSmxaY291eUpn?=
 =?utf-8?B?NVMzTHdFdzIrQmx0bEozRHAvRUpIcTdQOVl5UlgxNnBzSDdockV3a1cxSDJl?=
 =?utf-8?B?TjBzU2JUOWJkMFhuTTJIbEtDZXkva2lveGZiTTJyNFlBWkU5eTVsZDNjdlhQ?=
 =?utf-8?B?a2NXZlNBUDRnN0gyVS80Zm1Vd1FOa3lUQzNrcGdzU2NZQ2hoWnZrVWwySGdH?=
 =?utf-8?B?RTl3eFQraXIrNlIwNGdoUitIOEVqUStHUCs2SVd6djdEVmtMa2p2MmgwS1Fs?=
 =?utf-8?B?Qm1RaWRmakVCa0N5bjJ0L2hzaDJpT05WSWY1Tk56YmI5MkRCQnBtTmJESklC?=
 =?utf-8?B?YlRwbEU3N1NBYlBvNUlMbk5sQ0xaT0gvak9xeG0zVWt2UmFadnVMYjRUNXRo?=
 =?utf-8?B?dzFiakZsaWJLL1JGUkFFenhlZElHWHlPNE56ZXJFUXZrckFhL3NvdjlhckRN?=
 =?utf-8?B?Y1Y1dFV5aHE2Y1JiKzdjbFhvdGwyZEZId0NQUXN0M1hGL08ya29mMUpNejY2?=
 =?utf-8?B?US90aGhOcWpDUlhqRzdWT0VsY0tmVzJmSEdaOU1xYVhRak5LZEQxNVdIMFRh?=
 =?utf-8?B?bmtKRjE4QWhBVVVOR0RIcmp3VXNVVWJpWkpxQ2I5eTVlSk5kMFZSSkl1djN0?=
 =?utf-8?B?MVo0QmMwbUNINTFFa0dPOFVTSk1GRGd3M29aQzk2a3Z3cjF0azQxL29GWFZp?=
 =?utf-8?B?b3o1OERHSEdoUzE4SHQrYXJZdWcvQi9DellDWkdabGdkd2NCWUR2UGtIRkdt?=
 =?utf-8?B?UkdTUWMxTVpqSUJHSCtwY1ZmQVNWS0tRWnRBM1p0ZXE3Z0tOQlJjWjNaSXRj?=
 =?utf-8?B?b1dnQWpGc2pXUklTR2xxSE4yeEVnRTcrUVVDUXVtckZoaVJIdGJFSXk0TFd1?=
 =?utf-8?B?RklrOHpzZ1hjVGcrYlAxN0hES21nR0tCWHBTbkZkaXVRejQ1K2h0blhwTDQ2?=
 =?utf-8?B?TDh5MFBBYjV5MkVIcnVvdHFaYkxXUU40dzB6MCtOVzNVcDFBWUhuYnhTaEdu?=
 =?utf-8?B?Wi85R3FDMTdIL3RmTno2OHdOYUEvVkxRaURZTXBRcVNzU0Rhb25vWWtkci91?=
 =?utf-8?B?bzFteDM3SEM5RWRDK0VZYjZJWEdIMHIzWGZKKzVaQm1iUFlGUTdwektHQTBv?=
 =?utf-8?B?UFVISzdZNWdIMTBVcEZaMXc1L1M5Tm1TUjEyRDNRZ3MxRS9xbWcvQmtHWVJR?=
 =?utf-8?B?VFlaR2hvSjB6bXhveEtaMnVzYVBIV2xnTnNwTHNrTEQvckxCYVVkUXJFSnNK?=
 =?utf-8?B?Z1dvMER3VUlBaEVsODd0Ym5PanhnbFRQcm0xMllOS3hBWGhTUzZtdnFrSENn?=
 =?utf-8?B?OFA2VkMvaVRlWTcrWVZpOCtkODlXZS80NEJLeC9kTnhIaytQSHB6eFFlUHg4?=
 =?utf-8?B?NW8xUkxSZHVGbTZRVTY3WTl1S0ZLTEd0aDZGditnd0VGTTFFTENNUnJuUG54?=
 =?utf-8?B?dWY2RFNqZmh2bk9pck1MemlBVDVkaVRSaXIrc1VtVFZ0L2trUENDNmx6b3Uz?=
 =?utf-8?B?blJleXZES3B2bzhvT3R1SGViTkFWTHVrNjNCdnpWK01hdkNYeVZkMXd1b3Bu?=
 =?utf-8?B?emh6RHJhRldLQzAreHpDd2dFRFQzVXBMK08zRXM3bVBwZUMwd2daTHphekUv?=
 =?utf-8?B?ampzWnRoVkNjWjVsd2NIaUZvMndtN3Azdm9rdWNaZzlZNFhDeDNnMVorWU5l?=
 =?utf-8?B?VjYvYmhqamxWRklBUWR0YlN0RWRSdDdZbzA4MmQwVlQraHVMZGk0dzBNeGF0?=
 =?utf-8?B?S25DaWVNK0l6ZVBYMW9QTmltejUyY3FUSk1tdGNELzcraDg4SzlvR014Kzh3?=
 =?utf-8?B?cnhXalhTcHYyWnJKSHNmN0RxMVQ5K2hHVjFJaCsvaXovVGZYQ2hmNndqTVpX?=
 =?utf-8?B?eXlIR3pvckduMDNwZ0p6ZitkNnM1QWhQRTlOclFWVXlrRXJETXVPY0JPYUQ1?=
 =?utf-8?B?TGZGZTd3S29kWTRLeU5qZ1dETVliVmw5YWp1ZjFOQ0ZkcXJjY2pFNWhiWGlJ?=
 =?utf-8?B?VGlibnRHcFJ6ZVZHRE5ra09kZFNCV2hWUnpYNDAyUFArUUw3VVR5NXVjdlUw?=
 =?utf-8?B?bVE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 1919510b-66af-42c6-e1f4-08dbf05f0e6b
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Nov 2023 22:12:05.8858
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +M0pFR9/sWnwRn1dsvN0OKDc56Tkl+v2tYREB229xRQldMp+zOrDYdVUz9TtSkS5tMd/Ep3I/Sm6U8gQKFJ6ykvg7MDWIb1qC90Ed6ZAfXQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB8284
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Tue, 28 Nov 2023 14:12:32 -0800 (PST)

Hi Ilpo,

On 11/20/2023 3:13 AM, Ilpo Järvinen wrote:
> CAT test doesn't take shareable bits into account, i.e., the test might
> be sharing cache with some devices (e.g., graphics).
> 
> Introduce get_mask_no_shareable() and use it to provision an
> environment for CAT test where the allocated LLC is isolated better.
> Excluding shareable_bits may create hole(s) into the cbm_mask, thus add
> a new helper count_contiguous_bits() to find the longest contiguous set
> of CBM bits.
> 
> create_bit_mask() is needed by an upcoming CAT test rewrite so make it
> available in resctrl.h right away.
> 
> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> ---
>  tools/testing/selftests/resctrl/cat_test.c  | 12 ++-
>  tools/testing/selftests/resctrl/resctrl.h   |  3 +
>  tools/testing/selftests/resctrl/resctrlfs.c | 89 +++++++++++++++++++++
>  3 files changed, 100 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/testing/selftests/resctrl/cat_test.c b/tools/testing/selftests/resctrl/cat_test.c
> index cfcdb7bd600f..2c8b37c0a179 100644
> --- a/tools/testing/selftests/resctrl/cat_test.c
> +++ b/tools/testing/selftests/resctrl/cat_test.c
> @@ -89,15 +89,19 @@ void cat_test_cleanup(void)
>  
>  int cat_perf_miss_val(int cpu_no, int n, char *cache_type)
>  {
> +	unsigned long full_cache_mask, long_mask;
>  	unsigned long l_mask, l_mask_1;
>  	int ret, pipefd[2], sibling_cpu_no;
>  	unsigned long cache_total_size = 0;
> -	unsigned long long_mask;
>  	int count_of_bits;
>  	char pipe_message;
>  	size_t span;
>  
> -	ret = get_full_cbm(cache_type, &long_mask);
> +	ret = get_full_cbm(cache_type, &full_cache_mask);
> +	if (ret)
> +		return ret;
> +	/* Get the exclusive portion of the cache */

How about:
	"Get largest contiguous exclusive portion of the cache" ?

> +	ret = get_mask_no_shareable(cache_type, &long_mask);
>  	if (ret)
>  		return ret;
>  
> @@ -136,7 +140,7 @@ int cat_perf_miss_val(int cpu_no, int n, char *cache_type)
>  	/* Set param values for parent thread which will be allocated bitmask
>  	 * with (max_bits - n) bits
>  	 */
> -	span = cache_portion_size(cache_total_size, l_mask, long_mask);
> +	span = cache_portion_size(cache_total_size, l_mask, full_cache_mask);
>  	strcpy(param.ctrlgrp, "c2");
>  	strcpy(param.mongrp, "m2");
>  	strcpy(param.filename, RESULT_FILE_NAME2);
> @@ -158,7 +162,7 @@ int cat_perf_miss_val(int cpu_no, int n, char *cache_type)
>  		param.mask = l_mask_1;
>  		strcpy(param.ctrlgrp, "c1");
>  		strcpy(param.mongrp, "m1");
> -		span = cache_portion_size(cache_total_size, l_mask_1, long_mask);
> +		span = cache_portion_size(cache_total_size, l_mask_1, full_cache_mask);
>  		strcpy(param.filename, RESULT_FILE_NAME1);
>  		param.num_of_runs = 0;
>  		param.cpu_no = sibling_cpu_no;
> diff --git a/tools/testing/selftests/resctrl/resctrl.h b/tools/testing/selftests/resctrl/resctrl.h
> index 57fe42f4eda2..a911b08fa595 100644
> --- a/tools/testing/selftests/resctrl/resctrl.h
> +++ b/tools/testing/selftests/resctrl/resctrl.h
> @@ -99,7 +99,10 @@ void tests_cleanup(void);
>  void mbm_test_cleanup(void);
>  int mba_schemata_change(int cpu_no, const char * const *benchmark_cmd);
>  void mba_test_cleanup(void);
> +unsigned long create_bit_mask(unsigned int start, unsigned int len);
>  int get_full_cbm(const char *cache_type, unsigned long *mask);
> +int get_shareable_mask(const char *cache_type, unsigned long *shareable_mask);

Does this need to be exported?

> +int get_mask_no_shareable(const char *cache_type, unsigned long *mask);
>  int get_cache_size(int cpu_no, const char *cache_type, unsigned long *cache_size);
>  void ctrlc_handler(int signum, siginfo_t *info, void *ptr);
>  int signal_handler_register(void);
> diff --git a/tools/testing/selftests/resctrl/resctrlfs.c b/tools/testing/selftests/resctrl/resctrlfs.c
> index 1ea610fa65b1..3a9ed918d657 100644
> --- a/tools/testing/selftests/resctrl/resctrlfs.c
> +++ b/tools/testing/selftests/resctrl/resctrlfs.c
> @@ -228,6 +228,44 @@ static int get_bit_mask(const char *filename, unsigned long *mask)
>  	return 0;
>  }
>  
> +/*
> + * create_bit_mask- Create bit mask from start,len pair

Please add a space after the comma.

> + * @start:	LSB of the mask
> + * @len		Number of bits in the mask
> + */
> +unsigned long create_bit_mask(unsigned int start, unsigned int len)
> +{
> +	return ((1UL << len) - 1UL) << start;
> +}
> +
> +/*
> + * count_contiguous_bits - Returns the longest train of bits in a bit mask
> + * @val		A bit mask
> + * @start	The location of the least-significant bit of the longest train
> + *
> + * Return:	The length of the contiguous bits in the longest train of bits
> + */
> +static unsigned int count_contiguous_bits(unsigned long val, unsigned int *start)
> +{
> +	unsigned long last_val;
> +	unsigned int count = 0;
> +
> +	while (val) {
> +		last_val = val;
> +		val &= (val >> 1);
> +		count++;
> +	}
> +
> +	if (start) {
> +		if (count)
> +			*start = ffsl(last_val) - 1;
> +		else
> +			*start = 0;
> +	}
> +
> +	return count;
> +}
> +
>  /*
>   * get_full_cbm - Get full Cache Bit Mask (CBM)
>   * @cache_type:	Cache type as "L2" or "L3"
> @@ -254,6 +292,57 @@ int get_full_cbm(const char *cache_type, unsigned long *mask)
>  	return 0;
>  }
>  
> +/*
> + * get_shareable_mask - Get shareable mask from shareable_bits
> + * @cache_type:		Cache type as "L2" or "L3"
> + * @shareable_mask:	Shareable mask returned as unsigned long
> + *
> + * Return: = 0 on success, < 0 on failure.
> + */
> +int get_shareable_mask(const char *cache_type, unsigned long *shareable_mask)

It looks like this can be static.

> +{
> +	char mask_path[PATH_MAX];
> +
> +	if (!cache_type)
> +		return -1;
> +
> +	snprintf(mask_path, sizeof(mask_path), "%s/%s/shareable_bits",
> +		 INFO_PATH, cache_type);
> +
> +	return get_bit_mask(mask_path, shareable_mask);
> +}
> +
> +/*
> + * get_mask_no_shareable - Get Cache Bit Mask (CBM) without shareable bits
> + * @cache_type:		Cache type as "L2" or "L3"
> + * @mask:		The largest exclusive portion of the cache out of the
> + *			full CBM, returned as unsigned long
> + *
> + * Parts of a cache may be shared with other devices such as GPU. This function
> + * calculates the largest exclusive portion of the cache where no other devices
> + * besides CPU have access to the cache portion.
> + *
> + * Return: = 0 on success, < 0 on failure.
> + */
> +int get_mask_no_shareable(const char *cache_type, unsigned long *mask)
> +{
> +	unsigned long full_mask, shareable_mask;
> +	unsigned int start, len;
> +
> +	if (get_full_cbm(cache_type, &full_mask) < 0)
> +		return -1;
> +	if (get_shareable_mask(cache_type, &shareable_mask) < 0)
> +		return -1;
> +
> +	len = count_contiguous_bits(full_mask & ~shareable_mask, &start);
> +	if (!len)
> +		return -1;
> +
> +	*mask = create_bit_mask(start, len);
> +
> +	return 0;
> +}
> +
>  /*
>   * get_core_sibling - Get sibling core id from the same socket for given CPU
>   * @cpu_no:	CPU number


Reinette