Received-SPF: pass (google.com: domain of linux-kernel+bounces-188131-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <04d0a5d6-82fa-4cc7-bd80-ee5cbd35f0c3@intel.com>
Date: Thu, 23 May 2024 17:10:41 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 02/16] selftests/resctrl: Calculate resctrl FS derived
 mem bw over sleep(1) only
To: =?UTF-8?Q?Ilpo_J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>,
	<linux-kselftest@vger.kernel.org>, Shuah Khan <shuah@kernel.org>, Babu Moger
	<babu.moger@amd.com>, =?UTF-8?Q?Maciej_Wiecz=C3=B3r-Retman?=
	<maciej.wieczor-retman@intel.com>
CC: <linux-kernel@vger.kernel.org>, Fenghua Yu <fenghua.yu@intel.com>, "Shuah
 Khan" <skhan@linuxfoundation.org>
References: <20240520123020.18938-1-ilpo.jarvinen@linux.intel.com>
 <20240520123020.18938-3-ilpo.jarvinen@linux.intel.com>
From: Reinette Chatre <reinette.chatre@intel.com>
Content-Language: en-US
In-Reply-To: <20240520123020.18938-3-ilpo.jarvinen@linux.intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MnBoQU8waGFVc3RWQ0U4akMrbXlTZzZ6R2JDN1ZQQUJtWmN2MUZYN0hrQy93?=
 =?utf-8?B?K2VSVmtGZW9oODdwZG1vUFZSdFpsV1UzWkpFK1kyVXVGTlZpUnlVT2RRVkJX?=
 =?utf-8?B?MWQxWnprenI0YWxiYWwvNWNBbXJFb09odFA5aTAxWG44ZDFnRnhKMDc5d2x5?=
 =?utf-8?B?TWZQelAwN2dKVFZ3MWN1U3N0d0lubE55N1pFSTZRWEhiSXhyUXNCbmtWWWVu?=
 =?utf-8?B?dENZMjZqczVUM1pCNHpKMzJZclJNQ2JNZ3l0azNvMzMxUUJNQ2pXSkFwd085?=
 =?utf-8?B?ejJFQW5Ed0NCUmNPSmVicWhGU0k2SjdKZmtjQkJoSHRYUCtPUU1VVG5ic0lz?=
 =?utf-8?B?VDMvOVd2eFNURW5IN2RKMHQzRUNzcDd2Yk1TM3daOGdBeGVoZDJJL1NLQXJx?=
 =?utf-8?B?L0NBbDljdlVkUTJhSktPQ3FDdHNwWTJiOVNDLzZ0OEZ1Wk1LaklWT2FZdFZC?=
 =?utf-8?B?NHhNai9LTVVIeFJGTmhHUW9wNGRrT2NOTC9rcGFZRTJHR0t5ZlByZlkvT2Ux?=
 =?utf-8?B?N0hDeFF0SGpXdlBJT0FjTlJTRGdTSFMyL1dyb1I3UDBtMk4zTGQxTDljaE9v?=
 =?utf-8?B?MFNURnpvK0lGNjBQeS8wOXBXQzdKZ1g5UVVMRFNzRURhRldmVXNoY2V6OFZX?=
 =?utf-8?B?VU1MWnFSaEJnRzNQMGtQTDhLWHRBaklURFF2UkQ5NngxcFh6Mmd4SUtSak9X?=
 =?utf-8?B?WnMrL3BEZjlsNmtjbUFxc0RnRXBPUmkvbVFmZlc5ZXptb24zcEJDK08xK2lH?=
 =?utf-8?B?YzRwbDJUZjFwbk1LN00vNktNSTE1cElXUld3OERsS1BrMEkxdEM2RmVCK28x?=
 =?utf-8?B?aWhHUldwYldQMHdGZXFUcy83cUdVMVMvVFlucUwwTGFPaXdEdnJjeDJIc0o2?=
 =?utf-8?B?UUthc3hXYVBpNzZ3N1hPWXpUbEdxTTlYWEdQekhOaVhyaVlsQlpmeGRMYjVo?=
 =?utf-8?B?QXBhTzZ4MUlKeGF6TTNnc0dGY0RYWnFWeGdGTGsyN005VnE0TEdJckZYY3BU?=
 =?utf-8?B?SjdIZ3AvK0hmV1VqQmhnczlYMnRPRnhkcEdaOXlINUR4ZElCYnhQYWx0bWZF?=
 =?utf-8?B?RlNHSUNVeUlUMG4zQytZRWpwWTQxWE95SVlMWXl2dXRkZDErL2xRbXVRYWVM?=
 =?utf-8?B?ZldNUVZhTjU3eUNYNEZscnpwWW1FUUJIRE5iOWNPa0VNYnQ0NFU5NFRVUUg1?=
 =?utf-8?B?VXhja0xpdzM1N2x1Z3pnWU0xZ2l6K2Vyc2cvV2wxZk94MnFWNjJrRDBHdnMx?=
 =?utf-8?B?SW1lQ1h4aEovRy9LbE45SkkrTGhjVk81ZG9FeUpDcjZzalZUK2JkaDdOakx1?=
 =?utf-8?B?UEdzdzZJaW1YdHZacTVzS3RSdFlZRnVDa3R3Rm5YekdTcGxpTHRrWFh4Umpm?=
 =?utf-8?B?RWxGeXV1TTNnQjVOYXd5ZmoyRVZKMXg2bVhZdE9TUEFINktKUHYxeDRiNTF2?=
 =?utf-8?B?VFV5R0ViL05oUjRKZUFyczRKYlB6ODVpd2V6NHE4Z25wRFpLTEFnbklNa0JB?=
 =?utf-8?B?OHlac2habW43dXY4dVdzd2t4Sm0rU0YydThJSlVKWWVXeDVNcFNaM1AzLzVY?=
 =?utf-8?B?MXN4Q05iZldCZk9FUi9pY1NmTkJTTjR0N0NKTDVXL3diQ3pQZTFLNUlocC9Y?=
 =?utf-8?B?dVJMdWgvNklZWXkrL3c1bHJ6amI0bnNMMHVhVWVNNWd2M0NFNlYvendyWFBm?=
 =?utf-8?B?S1V2UUdqeGVVNU9PdTdjTmNGUTF3SWZ5M1g2OVBhOHY1d3dxNm1KZ0d3MnR2?=
 =?utf-8?B?VkpzVkhGQTdRMEZSelZJbVVLR0Jsekgvd0JWekxCcnJHNlRWdlBzb1Uxb2t3?=
 =?utf-8?B?VWsvSDhxUDV0a0xraVFDT29EMnR2eU1zRHJHbFRRTWYrdXZtMWs0MXNHVElY?=
 =?utf-8?B?U2JUS2Rwd3lyYVdtaml4Ti8vSWpxWi9OYjFVN1hwYlBqRFk2QURVS2czREF4?=
 =?utf-8?B?ZW1US3MyWFNrMC9Ra3U4ZkkzRmdsYXNZU3ltRUNJSGR5Z0NJM1krbVhpVk9B?=
 =?utf-8?B?WlM2bmQxOGR0dHZPeHFCK2NML2xWNlFlcUM0SEJISS9mK1JENGx4QVhYVFIv?=
 =?utf-8?B?dmxIZUl4RTFzODhBYk8vL01wOVF0NWVPL3RsYWlQeDc0OU8yN2FVS3BDc3pY?=
 =?utf-8?B?SXRHb0txSmwvTzFLRGs3UWxtVXhlT3JOL0xZcFFkTjRMQkU3NFJqc0Zac3cy?=
 =?utf-8?B?ZWc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0e4a0724-64d5-4a60-1d10-08dc7b85f44f
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 00:10:44.0322
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ukE0jXfyelD+c4L78puG6wMqFrbheI2MshBCvss9Gx+rFVO3Vd48F4GWcVMhzPjMocRVn+s0WVzt8v73dBrxVbAqBI7hS0Z4GIHKQG4jg98=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR11MB4604
X-OriginatorOrg: intel.com

Hi Ilpo,

On 5/20/24 5:30 AM, Ilpo Järvinen wrote:
> For MBM/MBA tests, measure_vals() calls get_mem_bw_imc() that performs
> the measurement over a duration of sleep(1) call. The memory bandwidth
> numbers from IMC are derived over this duration. The resctrl FS derived
> memory bandwidth, however, is calculated inside measure_vals() and only
> takes delta between the previous value and the current one which
> besides the actual test, also samples inter-test noise.
> 
> Rework the logic in measure_vals() and get_mem_bw_imc() such that the
> resctrl FS memory bandwidth section covers much shorter duration
> closely matching that of the IMC perf counters to improve measurement
> accuracy. Open two the resctrl mem bw files twice to avoid opening
> after the test during measurement period (reading the same file twice
> returns the same value so two files are needed).

I think this is only because of how the current reading is done, resctrl
surely supports keeping a file open and reading from it multiple times.

There seems to be two things that prevent current code from doing this
correctly:
(a) the fscanf() code does not take into account that resctrl also
     prints a "\n" ... (this seems to be the part that may cause the same
     value to be returned).
     So:
	if (fscanf(fp, "%lu", mbm_total) <= 0) {
     should be:
	if (fscanf(fp, "%lu\n", mbm_total) <= 0) {
(b) the current reading does not reset the file position so a second
     read will attempt to read past the beginning. A "rewind(fp)"
     should help here.

A small program like below worked for me by showing different values
on every read:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

const char *mbm_total_path = "/sys/fs/resctrl/mon_data/mon_L3_00/mbm_total_bytes";

int main(void)
{
	unsigned long mbm_total;
	FILE *fp;
	int count;

	fp = fopen(mbm_total_path, "r");
	if (!fp) {
		perror("Opening data file\n");
		exit(1);
	}
	for (count = 0; count < 100; count++) {
		if (fscanf(fp, "%lu\n", &mbm_total) <= 0) {
			perror("Unable to read from data file\n");
			exit(1);
		}
		printf("Read %d: %lu\n",count ,mbm_total );
		sleep(1);
		rewind(fp);
	}
	fclose(fp);
	return 0;
}

Reinette