Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Message-ID: <db556fff-90f5-a513-e823-db33b121f3e0@intel.com>
Date:   Fri, 21 Jan 2022 10:06:12 -0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Firefox/91.0 Thunderbird/91.5.0
Subject: Re: [PATCH v2 4/5] selftests/resctrl: Change default limited time to
 120 seconds for resctrl_tests
Content-Language: en-US
To:     "tan.shaopeng@fujitsu.com" <tan.shaopeng@fujitsu.com>,
        Fenghua Yu <fenghua.yu@intel.com>,
        Shuah Khan <shuah@kernel.org>
CC:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>
References: <20211213100154.180599-1-tan.shaopeng@jp.fujitsu.com>
 <20211213100154.180599-5-tan.shaopeng@jp.fujitsu.com>
 <c8fd3125-b61c-8182-d7af-0f7a85eba345@intel.com>
 <TYAPR01MB63301CABB489B1A764A9E8718B5B9@TYAPR01MB6330.jpnprd01.prod.outlook.com>
From:   Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <TYAPR01MB63301CABB489B1A764A9E8718B5B9@TYAPR01MB6330.jpnprd01.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VlAzOGI2d1BYUDBiMDBLQUpKbWdFYmN4Znh2ZlVRa1RsU2pWYkdYSUcvUFZW?=
 =?utf-8?B?ZGdyN1JvdC9iSkFYWnh1aUI3cVozVkY0Z0NlaFlBRk16YVVWMkMrTFZNR3RC?=
 =?utf-8?B?dWgvaG54WDZyTnJRSVZDRHE5M3hxaVp5b0ljVUhySWphTnkwZmxsUGRXRjdW?=
 =?utf-8?B?MDJKVFRUczY4RVVCUzNvYW1ySW00aE5CcE9FdlgrRnhISnJING1pY1V4ajhh?=
 =?utf-8?B?RytnSU9MOUQ5ZG02RTlrYk80bDFvQUlYUHFnbEZINlFDTDI3bFFyRUJRL0NU?=
 =?utf-8?B?N1AyemRUZUhNQ2ZtbEgzTTA0UDVVVmI1cTl5YkprbzRZYTZXeUMwNVB2RzJB?=
 =?utf-8?B?V3FuM25XYUdyUmF6UDMwWDIvWUwvNWJacUduS292UHJoMGkwMjhKT1plRHMx?=
 =?utf-8?B?ZUNXYnhpdTlZUytMazhrMHdrbHVpUHFVZ1dYL1JpZjNmR2owRUYwdEhTNnlv?=
 =?utf-8?B?Ky9PSzZMYzZMbTV4MHZKOUV5RUpqbnAwQWNBQk9IYWQwdXZTZkU0WDlZSjJw?=
 =?utf-8?B?OHdQU2V3NzF5MmdicWxmR0NnTjhJdnlucU96eU5iV2xPUnZwM01iRXJJS3JV?=
 =?utf-8?B?dU0xK041d3gzb0xGekErRmw5NEY2UzJUWGtIMWVXb1V3RGw3M1ZHdDhVQlRy?=
 =?utf-8?B?bC9oSkxoRXVPOEhlR0RFSjNPYXpMN2RpelRnTHJaMDhXTk5CdUFMbE0xNFdw?=
 =?utf-8?B?T3BIRGdmYVR3UVFmUDV4TzBhN0c0c0F0a1c2TUNRdjVBL1RGdDJ6RDVpVXFp?=
 =?utf-8?B?TFdoRnRPM0swN3B6RE1tY3d6WWs4eFBjc2VMQVhaa00rOHVLNnk1aEpvajcz?=
 =?utf-8?B?dThYRDY1T0Z4U05ZWDc1UGF0U201QVlBd0ZTRGwwQTdHQnZUWTY4YWxFSWxt?=
 =?utf-8?B?UjFVamFqQ2s5a0VqRDQ3Vk83cko3eWZubjc2akRkUUphYTJLdmtjUWc2Uzkz?=
 =?utf-8?B?a21qSGhaWE9TYUFISnpvc1krWk9FWnNSWUhRZlN0N2dnaElQaVZLbjRaODZB?=
 =?utf-8?B?SW0xQXQwSUU1bjhhMDVINjZVVUZYRjhta0M1NGtiUzZHQStUYnZReGF6cDcy?=
 =?utf-8?B?ZEU3czFYejNJKzRyTW9UOFFLYTdBdGFraFFWc2dmRVFIMTRCMkkrUkwvWUZH?=
 =?utf-8?B?b0JTYUVldW1GR1E2dnJnYmlSczMxcUs5VWN3Sm15cFZOaloxN1B3R2VXVThO?=
 =?utf-8?B?aU1BYlJaU1lIYmhueDZLb1g0Y2ZtVUQ1QWlwS0hDbDJVdXNsU1hQM2t3M09U?=
 =?utf-8?B?S0VYZWYxUEJUMzIwOXlad0RPMkpJdjBmOHV4dE5yTlRZZmw0NEEwcCtzNE1C?=
 =?utf-8?B?WVJteklXUE5JQTA2ZzdIRlJoZGIvT2JsVjJWblF1elN4bWtzK2hFMGRkVktZ?=
 =?utf-8?B?L01VcUozTStOZEhTOXp1STdTK1FiL2pza1l4MW92MldwK2ZyZWl1SjJLYkQ1?=
 =?utf-8?B?cFRDZXZQS0JQblNiK2RJR3c4VlVrNGI1ZkU0bm5BSGttQUF5eG5DSCsrQllS?=
 =?utf-8?B?SzFkcEQvV01hdzZnRjQ3NDJxQS9QSGR3UVBzR1MzYTFyTWl5VUNwM0QxSncy?=
 =?utf-8?B?a3VNOEFJQVJLMUpTUy9oSlhWOE9TNi9mT0hpVUZXMmZKcUcvVVdwVm1abklY?=
 =?utf-8?B?aG1hYmo3SzhEa1B3eGU5TC9NeEt3RVVMUSsvNDJ2dlpOQnJMbEtJL1RIZFBs?=
 =?utf-8?B?R0I1QnFoamdzbGlyTEF2WnFGYVdTOUtQeWFOWkprNVdqVWQ4Ris5VmkxWU01?=
 =?utf-8?B?WVVlSytmL3ppNU1yLzJDT0FjNFVGZk10TjFqbk43N0JGZ3ZwWVNPZ1pFNzVa?=
 =?utf-8?B?dFVLdWNoYklwR0IxT0ppdUJITGNDeWZmbnlidm1TaGIrTkZLc3hYS2ovWE1N?=
 =?utf-8?B?ZmZkN0FsMHFZWVZDaDM1TWx5SExFbGpDUWFyWnpTSDZ2VEFQK2NWcEVITkNy?=
 =?utf-8?B?OGFvQWVVWlBkcWJBazJRNlhMQmlnY3I2UUlGelFmdS9ERTBoSkJrRlRsKzcy?=
 =?utf-8?B?Sk1VVmlhWEhKejlsbGFWVUkvM2FFWjVWR1EwRXZSMGVUamdHTVVFZ2Nmdk5t?=
 =?utf-8?B?S2NFOHNwM2RNWGd0WTdUajY0SGlvWkt3WWZIRHBORjJub2NHM25yNEt2QkNZ?=
 =?utf-8?B?enJqeWh3bjJFOHViTE84ZS9QcHZCU2czdFZoS0sxYU1lZUVqWmhsM1hzcStu?=
 =?utf-8?B?QXNKZGhzM05Sdmxaanc3RWJFcEU4ZzBhbVhCNDNzcmFYNjF5K3dGWWozcGVa?=
 =?utf-8?B?RHZhek4xVDc0QVg2TmVCZkpSbHVRPT0=?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 14c04d83-101a-4178-666e-08d9dd08b795
X-MS-Exchange-CrossTenant-AuthSource: BN0PR11MB5744.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jan 2022 18:06:16.2271
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DBPdKvnuGigSjeIcEr3NnW9EXtyOgKWmzYqMajRqpCzIu1QHcuun1RX4pS/uDqX0sc//X2Dq1Rhbd+PdufFlF3xrGuYSmmwPXnwoQX7WssA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB4925
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Tan Shaopeng,

On 1/20/2022 11:59 PM, tan.shaopeng@fujitsu.com wrote:
> Hi Reinette,
> 
>> On 12/13/2021 2:01 AM, Shaopeng Tan wrote:
>>> This commit changed the default limited time(45s) to 120 seconds for
>>
>> (This commit)
>>
>>> resctrl_tests by adding "setting" file.  Since 45 seconds may not be
>>> enough on some machine environments.
>>
>> This sounds very imprecise when you say "may not be enough". From what I
>> understand you have already encountered systems where this is not enough so
>> you have established that 45 seconds is too short. The next question is, how
>> was 120 seconds determined and how do you know it will be enough?
> 
> It took about 68 seconds in my environment (Intel(R) Xeon(R) Gold 6254 CPU @ 3.10GHz). 
> This test accesses a fixed size(250M) memory by default settings 
> which is used when run in kselftest framework. 
> I think that the execution time does not change significantly in different environments.
> So, I roughly doubled the execution time and get 120s, which I set to the limited time. 
> I think it is enough for any environments.
> If 120s are also not enough, user can freely set the limited time by "setting" file.
> 

How about something like:

When testing on a Intel(R) Xeon(R) Gold 6254 CPU @ 3.10GHz the resctrl selftests fail
due to timeout after exceeding the default time limit of 45 seconds. On this system
the test takes about 68 seconds. Since the failing test by default accesses a fixed
size of memory the execution time should not vary significantly between different
environment. A new default of 120 seconds should be sufficient yet easy to customize
with the introduction of the "settings" file for reference. 

Reinette