Received-SPF: pass (google.com: domain of linux-kernel+bounces-188834-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Message-ID: <d8063ee7-1744-45a2-b6b9-506e68106baf@intel.com>
Date: Fri, 24 May 2024 08:14:57 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4 02/16] selftests/resctrl: Calculate resctrl FS derived
 mem bw over sleep(1) only
To: =?UTF-8?Q?Ilpo_J=C3=A4rvinen?= <ilpo.jarvinen@linux.intel.com>
CC: <linux-kselftest@vger.kernel.org>, Shuah Khan <shuah@kernel.org>, "Babu
 Moger" <babu.moger@amd.com>, =?UTF-8?Q?Maciej_Wiecz=C3=B3r-Retman?=
	<maciej.wieczor-retman@intel.com>, LKML <linux-kernel@vger.kernel.org>,
	Fenghua Yu <fenghua.yu@intel.com>, Shuah Khan <skhan@linuxfoundation.org>
References: <20240520123020.18938-1-ilpo.jarvinen@linux.intel.com>
 <20240520123020.18938-3-ilpo.jarvinen@linux.intel.com>
 <04d0a5d6-82fa-4cc7-bd80-ee5cbd35f0c3@intel.com>
 <ea0c86b9-ae77-c2d9-b52b-239ae42603e8@linux.intel.com>
Content-Language: en-US
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <ea0c86b9-ae77-c2d9-b52b-239ae42603e8@linux.intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eWZXdmtWOFNSK0VNeGIxQm1GR1NHcnVBMVdYOXpEb2RaYzJzYVZtdVo0SjRT?=
 =?utf-8?B?UHphMDhSZmpvODB0bTRqajA3UzdNLzI2VXA3NEV6RzRidWtobllaT1FRa2g3?=
 =?utf-8?B?andQdDFSZ25BYkZDZ3dOY3cydU0vQUhIdEFFYlVrNmZuOERMdStpV1UyMUkv?=
 =?utf-8?B?NmhTdjJHcis0blRBbkZrQWY5bGg5dXM3eHhRb0U5Rm42U3NHRVRhVjZqZDNS?=
 =?utf-8?B?UjJJNTRvMEtkdStpVER0UmJYTitucWg3eXJaU0xUYlNtUnk0ei9TSTBpamtC?=
 =?utf-8?B?Qi8yY0I4YWFWazZOQnVzTXNpekZDK0kxWTBpanJIaTNadDZSeDJ6Qi9UMGwr?=
 =?utf-8?B?aWN5YWkvS1BFajFhdnhHN3d5K1NobjJ4Nk5nNDcxRmNKbm04QkZ1bVRuUGky?=
 =?utf-8?B?aWNVUGo2ZUV6WjFKenNQTWgwcVlDbndlc0t2ZlZPRUo4K0tiS1hxZFQ3ZFVR?=
 =?utf-8?B?alFRQjYwWUEwc2FDQkNLOVFtRTBNQmUyM2ZHMmt2M1RzS3VtY2NiRU9BUkRO?=
 =?utf-8?B?WWdNMGV4NjZxWjg5UFlLRFRnaUZtN3VlZTRvQ2FuOXlyZkZKRVRjdkUzYmJi?=
 =?utf-8?B?eTdyZHpLS1ZNYW15YXFmOTd5RTgybmY0Rmhta1p5VURIQXUxZTV0d0pvN1o2?=
 =?utf-8?B?c2FJY01yOXlobWRvVFJLOVUyMWlhOFVUOStzNmttek84dW5ncStHQnlFcVFQ?=
 =?utf-8?B?am9jaEtCRlQrenIyVU9CbVVCTTFuS3NIM1Q1eDVpSThMTEFTbkovRndWbUM3?=
 =?utf-8?B?ZG1JQjFqRC8wZ1ZzSWhGOGVLYnlacjBwMHdWODNSV2h2bkduMmV2cXFWWU1U?=
 =?utf-8?B?T2lySVZkTjVjZWxCelZIdlAxVGFiT1ZHa29hQWV5dit4TktJY3hBdmpHaFBs?=
 =?utf-8?B?RndxZm52NU9UeDluZk9MY3RTS3g5eHo3bzE3cmczcUVPL1FNcXE3ZnA2Ukl5?=
 =?utf-8?B?TkxwTmM4a3NMR0Y0dUM2VUpaOGFwaThDTjZEUThMcGY3dUMwbUZoR0I4Z2pj?=
 =?utf-8?B?T3lxWm54UjZRSkw4RmRhWHZZREtRc2daNmFTTlFxYVFaWHhwWTBBR2NOWXFk?=
 =?utf-8?B?ZlBndVYrNFc0eTM0RmpnUGQrS2JIMTNEMUlQeGtkRnlhK0RVMXF5d250SFM1?=
 =?utf-8?B?VzNRVm9ycG05cnRsZGtzdkV2b0R2dWVUQnF1RW9Jd0Y4TTByOGlPT2N0VWdN?=
 =?utf-8?B?bFordmZBV1NaMnNib2tYNlQ0b1d2VXczditIZ1NyK2hJN2NSQityb3RYbWdV?=
 =?utf-8?B?a1FqdnZzT3JOTmo0ZmlsYXpCR2ZHZlVhV3FnR1JVeER4YWJ4RSttbnBQbFRw?=
 =?utf-8?B?ZGhvZFhTQlZxdVhhMWtESFV1RjNjVGd1M3g2Q0cwaXF6N0t3L0RBQ1NBbGla?=
 =?utf-8?B?c0hDWVo5UkYrQVc2bThIelVUNFdseWVxdE01LzJaUzI4V1BNMkRCdCt0bkhV?=
 =?utf-8?B?cVlLU0tQNXB2dEtmZGFuUDVvM2NSQTNPdHdrWEVVN1JONldhY2w1dDIvOUZy?=
 =?utf-8?B?RVluczNzUWxmVndDZTJDOHl2N3NyMzd0YlNNMktKRERaUWoySkU4ell6OHN1?=
 =?utf-8?B?VjhITmNNN3FraFpoZWhPQVNSUHNsbTFWbTBoYjl4ajJtZEpEZ0E1d0gzanhy?=
 =?utf-8?B?SVVneGt1bVc0cFdPakN6aFU1VFVONm1RL1lHZitjU29sTzA1NnRwRFg4UDBT?=
 =?utf-8?B?YnM5aGR3Q1kyRkpqdjExd3Q3UWVPRHVvTTR6ajNyRllKU0ZYTFN6RkkvVFhz?=
 =?utf-8?B?cDVjZmM4Y1NSdHlpajlwUVd6R2ZCTDgzc25BT1d2OWJRSjJkZStkbjliQXQ2?=
 =?utf-8?B?SFZ0ZVc2VFc3T2JzczNKMWZ0R0VmQXF5UDhsQTd6SHNQMjNRRjIzaTllODcx?=
 =?utf-8?B?dkR2T21VMFBFV3I2MFkxbVBxQTVnRStKZWExaW9PQWxvK1BWK3BkRGdaK1JL?=
 =?utf-8?B?MHhlUVpBZ21nbGhZMTROc09SVkowM1hQNXAxTEd1Qi9ReHVPV2Vpa1NHMFpo?=
 =?utf-8?B?ZlQwN2s3WFVuYVh5ZGNqc0NaTnBzTVUvUFBQcmZ4SmgrRS90UTQ1dDN4SzhO?=
 =?utf-8?B?ZUNRUHdLYW5ZWmN6TWFwKzB0SmFncitNRkEwaU50UnJ4dG0yL1dKTHdPMlhl?=
 =?utf-8?B?eWdQcnVSdUcvWGM3eDdwOTFsU3ZVM0xHNkV0V0VQaVR6aE1tdHIxc2pJakM3?=
 =?utf-8?B?SEE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0ce4d343-450c-40a2-a1c9-08dc7c044776
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2024 15:15:00.1582
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4ckBRWQKjslABGdtUSyK40t1xAbQBVyuKeu24ALBsUzp9RXCtlUQoY6p32qn8nRhXWjALj82eNbrfjoQD8LSKQUdV4xuefXwOLHbx8BbwBo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB6737
X-OriginatorOrg: intel.com

Hi Ilpo,

On 5/24/24 12:57 AM, Ilpo Järvinen wrote:
> On Thu, 23 May 2024, Reinette Chatre wrote:
>> On 5/20/24 5:30 AM, Ilpo Järvinen wrote:
>>> For MBM/MBA tests, measure_vals() calls get_mem_bw_imc() that performs
>>> the measurement over a duration of sleep(1) call. The memory bandwidth
>>> numbers from IMC are derived over this duration. The resctrl FS derived
>>> memory bandwidth, however, is calculated inside measure_vals() and only
>>> takes delta between the previous value and the current one which
>>> besides the actual test, also samples inter-test noise.
>>>
>>> Rework the logic in measure_vals() and get_mem_bw_imc() such that the
>>> resctrl FS memory bandwidth section covers much shorter duration
>>> closely matching that of the IMC perf counters to improve measurement
>>> accuracy. Open two the resctrl mem bw files twice to avoid opening
>>> after the test during measurement period (reading the same file twice
>>> returns the same value so two files are needed).
>>
>> I think this is only because of how the current reading is done, resctrl
>> surely supports keeping a file open and reading from it multiple times.
>>
>> There seems to be two things that prevent current code from doing this
>> correctly:
>> (a) the fscanf() code does not take into account that resctrl also
>>      prints a "\n" ... (this seems to be the part that may cause the same
>>      value to be returned).
>>      So:
>> 	if (fscanf(fp, "%lu", mbm_total) <= 0) {
>>      should be:
>> 	if (fscanf(fp, "%lu\n", mbm_total) <= 0) {
>> (b) the current reading does not reset the file position so a second
>>      read will attempt to read past the beginning. A "rewind(fp)"
>>      should help here.
> 
> (b) cannot be the cause for returning the same value again. It would
> not be able to reread the number at all if file position is not moved.

I know. This was not intended to explain the duplicate answer but instead
describe another change required to use current code in a loop. I
specifically said in (a) that "(this seems to be the part that may cause
the same value to be returned)".

> I certainly tried with fseek() and it is when I got same value on the
> second read which is when I just went to two files solution.
> 
>> A small program like below worked for me by showing different values
>> on every read:
>>
>> #include <stdio.h>
>> #include <stdlib.h>
>> #include <unistd.h>
>>
>> const char *mbm_total_path =
>> "/sys/fs/resctrl/mon_data/mon_L3_00/mbm_total_bytes";
>>
>> int main(void)
>> {
>> 	unsigned long mbm_total;
>> 	FILE *fp;
>> 	int count;
>>
>> 	fp = fopen(mbm_total_path, "r");
>> 	if (!fp) {
>> 		perror("Opening data file\n");
>> 		exit(1);
>> 	}
>> 	for (count = 0; count < 100; count++) {
>> 		if (fscanf(fp, "%lu\n", &mbm_total) <= 0) {
>> 			perror("Unable to read from data file\n");
>> 			exit(1);
>> 		}
>> 		printf("Read %d: %lu\n",count ,mbm_total );
>> 		sleep(1);
>> 		rewind(fp);
>> 	}
>> 	fclose(fp);
>> 	return 0;
>> }
> 
> Okay, so perhaps it's your explanation (a) but can libc be trusted to not
> do buffering/caching for FILE *? So to be on the safe side, it would

Coding with expectation that libc cannot be trusted sounds strange to me.

> need to use syscalls directly to guarantee it's read the file twice.
> 
> If I convert it into fds, fscanf() cannot be used which would complicate
> the string processing by adding extra steps.
> 

It is not clear to me why you think that fscanf() cannot be used. Could
you please elaborate what the buffering issues are?

It is not necessary to open and close the file every time a value needs
to be read from it.

Reinette