From: "Tan, Shaopeng" <[email protected]>
When the Intel Sub-NUMA Clustering(SNC) feature is enabled,
the CMT and MBM counters may not be accurate.
In this case, skip MBM&CMT tests.
Signed-off-by: Shaopeng Tan <[email protected]>
---
Hello,
According to the Intel RDT reference Manual,
when the sub-numa clustering feature is enabled, the CMT and MBM counters may not be accurate.
When running CMT tests and MBM tests on Intel processor, the result is "not ok".
So, fix it to skip the CMT & MBM test When the Intel Sub-NUMA Clustering(SNC) feature is enabled.
Thanks,
tools/testing/selftests/resctrl/resctrl.h | 1 +
tools/testing/selftests/resctrl/resctrl_tests.c | 51 +++++++++++++++++++++++++
tools/testing/selftests/resctrl/resctrlfs.c | 26 +++++++++++++
3 files changed, 78 insertions(+)
diff --git a/tools/testing/selftests/resctrl/resctrl.h b/tools/testing/selftests/resctrl/resctrl.h
index 1ad10c4..8e82ce3 100644
--- a/tools/testing/selftests/resctrl/resctrl.h
+++ b/tools/testing/selftests/resctrl/resctrl.h
@@ -85,6 +85,7 @@ struct resctrl_val_param {
int validate_bw_report_request(char *bw_report);
bool validate_resctrl_feature_request(const char *resctrl_val);
char *fgrep(FILE *inf, const char *str);
+char *fgrep_last_match_line(FILE *inf, const char *str);
int taskset_benchmark(pid_t bm_pid, int cpu_no);
void run_benchmark(int signum, siginfo_t *info, void *ucontext);
int write_schemata(char *ctrlgrp, char *schemata, int cpu_no,
diff --git a/tools/testing/selftests/resctrl/resctrl_tests.c b/tools/testing/selftests/resctrl/resctrl_tests.c
index 973f09a..122aab6 100644
--- a/tools/testing/selftests/resctrl/resctrl_tests.c
+++ b/tools/testing/selftests/resctrl/resctrl_tests.c
@@ -8,12 +8,15 @@
* Sai Praneeth Prakhya <[email protected]>,
* Fenghua Yu <[email protected]>
*/
+#include <numa.h>
+#include <string.h>
#include "resctrl.h"
#define BENCHMARK_ARGS 64
#define BENCHMARK_ARG_SIZE 64
bool is_amd;
+bool sub_numa_cluster_enable;
void detect_amd(void)
{
@@ -34,6 +37,35 @@ void detect_amd(void)
fclose(inf);
}
+void check_sub_numa_cluster(void)
+{
+ FILE *inf = fopen("/proc/cpuinfo", "r");
+ char *res, *s;
+ int socket_num = 0;
+ int numa_nodes = 0;
+
+ if (!inf)
+ return;
+
+ res = fgrep_last_match_line(inf, "physical id");
+
+ if (res) {
+ s = strpbrk(res, "1234567890");
+ socket_num = atoi(s) + 1;
+ free(res);
+ }
+ fclose(inf);
+
+ numa_nodes = numa_max_node() + 1;
+
+ /*
+ * when the Sub-NUMA Clustering(SNC) feature is enabled,
+ * the number of numa nodes is twice the number of sockets.
+ */
+ if (numa_nodes == (2 * socket_num))
+ sub_numa_cluster_enable = true;
+}
+
static void cmd_help(void)
{
printf("usage: resctrl_tests [-h] [-b \"benchmark_cmd [options]\"] [-t test list] [-n no_of_bits]\n");
@@ -61,6 +93,13 @@ static void run_mbm_test(bool has_ben, char **benchmark_cmd, int span,
ksft_print_msg("Starting MBM BW change ...\n");
+ /* when the Sub-NUMA Clustering(SNC) feature is enabled,
+ * the CMT and MBM counters may not be accurate
+ */
+ if (sub_numa_cluster_enable) {
+ ksft_test_result_skip("Sub-NUMA Clustering(SNC) feature is enabled, the MBM counters may not be accurate.\n");
+ return;
+ }
if (!validate_resctrl_feature_request(MBM_STR)) {
ksft_test_result_skip("Hardware does not support MBM or MBM is disabled\n");
return;
@@ -97,6 +136,14 @@ static void run_cmt_test(bool has_ben, char **benchmark_cmd, int cpu_no)
int res;
ksft_print_msg("Starting CMT test ...\n");
+
+ /* when the Sub-NUMA Clustering(SNC) feature is enabled,
+ * the CMT and MBM counters may not be accurate
+ */
+ if (sub_numa_cluster_enable) {
+ ksft_test_result_skip("Sub-NUMA Clustering(SNC) feature is enabled, the CMT counters may not be accurate.\n");
+ return;
+ }
if (!validate_resctrl_feature_request(CMT_STR)) {
ksft_test_result_skip("Hardware does not support CMT or CMT is disabled\n");
return;
@@ -210,6 +257,10 @@ int main(int argc, char **argv)
/* Detect AMD vendor */
detect_amd();
+ /* check whether sub numa clustering is enable or not */
+ if (!is_amd)
+ check_sub_numa_cluster();
+
if (has_ben) {
/* Extract benchmark command from command line. */
for (i = ben_ind; i < argc; i++) {
diff --git a/tools/testing/selftests/resctrl/resctrlfs.c b/tools/testing/selftests/resctrl/resctrlfs.c
index 5f5a166..1908ecb 100644
--- a/tools/testing/selftests/resctrl/resctrlfs.c
+++ b/tools/testing/selftests/resctrl/resctrlfs.c
@@ -606,6 +606,32 @@ char *fgrep(FILE *inf, const char *str)
}
/*
+ * Find the last matched line.
+ * Return a pointer to the string of the matched line,
+ * else retuen NULL if no matched line
+ */
+char *fgrep_last_match_line(FILE *inf, const char *str)
+{
+ char line[256];
+ char result_line[256];
+ int slen = strlen(str);
+
+ while (!feof(inf)) {
+ if (!fgets(line, 256, inf))
+ break;
+ if (strncmp(line, str, slen))
+ continue;
+
+ strcpy(result_line, line);
+ }
+
+ if (strlen(result_line) >= slen)
+ return strdup(result_line);
+
+ return NULL;
+}
+
+/*
* validate_resctrl_feature_request - Check if requested feature is valid.
* @resctrl_val: Requested feature
*
--
1.8.3.1
Hi,
On 11/10/2021 12:27 AM, Shaopeng Tan wrote:
> From: "Tan, Shaopeng" <[email protected]>
>
> When the Intel Sub-NUMA Clustering(SNC) feature is enabled,
> the CMT and MBM counters may not be accurate.
> In this case, skip MBM&CMT tests.
>
> Signed-off-by: Shaopeng Tan <[email protected]>
> ---
> Hello,
>
> According to the Intel RDT reference Manual,
> when the sub-numa clustering feature is enabled, the CMT and MBM counters may not be accurate.
> When running CMT tests and MBM tests on Intel processor, the result is "not ok".
> So, fix it to skip the CMT & MBM test When the Intel Sub-NUMA Clustering(SNC) feature is enabled.
>
It is not clear to me which exact document you refer to but I did find a
RDT reference manual at the link below that describes the problem you
mention:
https://www.intel.com/content/dam/develop/external/us/en/documents/180115-intel-rdtcascadelake-serverreferencemanual-806717.pdf
What is not mentioned in your description is that this is a hardware
errata so the test is expected to fail on these systems and I find that
disabling the test for all systems based on this hardware errata is too
drastic.
Reinette
Hi Reinette,
> On 11/10/2021 12:27 AM, Shaopeng Tan wrote:
> > From: "Tan, Shaopeng" <[email protected]>
> >
> > When the Intel Sub-NUMA Clustering(SNC) feature is enabled,
> > the CMT and MBM counters may not be accurate.
> > In this case, skip MBM&CMT tests.
> >
> > Signed-off-by: Shaopeng Tan <[email protected]>
> > ---
> > Hello,
> >
> > According to the Intel RDT reference Manual,
> > when the sub-numa clustering feature is enabled, the CMT and MBM
> counters may not be accurate.
> > When running CMT tests and MBM tests on Intel processor, the result is "not
> ok".
> > So, fix it to skip the CMT & MBM test When the Intel Sub-NUMA
> Clustering(SNC) feature is enabled.
> >
>
> It is not clear to me which exact document you refer to but I did find a
> RDT reference manual at the link below that describes the problem you
> mention:
> https://www.intel.com/content/dam/develop/external/us/en/documents/18
> 0115-intel-rdtcascadelake-serverreferencemanual-806717.pdf
Yes, I referred this manual.
> What is not mentioned in your description is that this is a hardware
> errata so the test is expected to fail on these systems and I find that
> disabling the test for all systems based on this hardware errata is too
> drastic.
Understood. It is not reasonable to disable the test for all systems
based on this hardware errata.
When I run restrl_test on Intel(R) Xeon(R) Gold 6254 CPU,
the result of CMT & MBM is "not ok", and I took some time to debug it.
In order to other people can do the test smoothly, I'd like to update the
patch to disable the test only on 2nd Generation Intel Xeon scalable processors.
Regards,
Shaopeng Tan
Hi Shaopeng Tan,
On 11/14/2021 11:11 PM, [email protected] wrote:
> Hi Reinette,
>
>> On 11/10/2021 12:27 AM, Shaopeng Tan wrote:
>>> From: "Tan, Shaopeng" <[email protected]>
>>>
>>> When the Intel Sub-NUMA Clustering(SNC) feature is enabled,
>>> the CMT and MBM counters may not be accurate.
>>> In this case, skip MBM&CMT tests.
>>>
>>> Signed-off-by: Shaopeng Tan <[email protected]>
>>> ---
>>> Hello,
>>>
>>> According to the Intel RDT reference Manual,
>>> when the sub-numa clustering feature is enabled, the CMT and MBM
>> counters may not be accurate.
>>> When running CMT tests and MBM tests on Intel processor, the result is "not
>> ok".
>>> So, fix it to skip the CMT & MBM test When the Intel Sub-NUMA
>> Clustering(SNC) feature is enabled.
>>>
>>
>> It is not clear to me which exact document you refer to but I did find a
>> RDT reference manual at the link below that describes the problem you
>> mention:
>> https://www.intel.com/content/dam/develop/external/us/en/documents/18
>> 0115-intel-rdtcascadelake-serverreferencemanual-806717.pdf
>
> Yes, I referred this manual.
>
>> What is not mentioned in your description is that this is a hardware
>> errata so the test is expected to fail on these systems and I find that
>> disabling the test for all systems based on this hardware errata is too
>> drastic.
>
> Understood. It is not reasonable to disable the test for all systems
> based on this hardware errata.
> When I run restrl_test on Intel(R) Xeon(R) Gold 6254 CPU,
> the result of CMT & MBM is "not ok", and I took some time to debug it.
> In order to other people can do the test smoothly, I'd like to update the
> patch to disable the test only on 2nd Generation Intel Xeon scalable processors.
I've been thinking about this some more and I do not think that the test
should be disabled. There is a clear incompatibility between SNC and RDT
on these systems and I do not think the test should hide that, indeed it
is helpful to highlight that there is an issue. Even so, spending time
to debug a known issue is not a good use of time. Instead of skipping
the test on these systems could the test perhaps be improved to provide
more information on failure to help user decide if they really need SNC
enabled? The test would show that RDT cannot be used on their system
with the SNC configuration, hiding that information by skipping the test
may create false idea that RDT is working with that configuration.
Reinette