2020-09-30 09:22:52

by Chen, Rong A

[permalink] [raw]
Subject: [perf report] 977f739b71: perf-test.skid_test.round2.MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_without_NMI_fixed_period.fail

Greeting,

FYI, we noticed the following commit (built with gcc-9):

commit: 977f739b7126bf98b5202e243f60cbc0a1ec2c3b ("perf report: Disable ordered_events for raw dump")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master


in testcase: perf-test
version: perf-test-x86_64-git-1_20200717
with following parameters:

type: lkp
ucode: 0xd6

test-description: The internal Perf Test suite.


on test machine: 4 threads Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz with 32G memory

caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):




If you fix the issue, kindly add following tag
Reported-by: kernel test robot <[email protected]>


2020-09-28 17:02:40 ./skid_test/run.sh
Mon Sep 28 17:02:58 UTC 2020
3 test cases pass for skid_test test. 3 test cases fail for skid_test test.
Test Case BR_INST_RETIRED.ALL_BRANCHES_PEBS_triad_loop_with_NMI PASS!
Test Case MEM_LOAD_RETIRED.L2_HIT_dtlb_with_NMI PASS!
Test Case MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_with_NMI FAILED!
Test Case BR_INST_RETIRED.ALL_BRANCHES_PEBS_triad_loop_without_NMI_fixed_period PASS!
Test Case MEM_LOAD_RETIRED.L2_HIT_dtlb_without_NMI_fixed_period FAILED!
Test Case MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_without_NMI_fixed_period FAILED!



To reproduce:

git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml



Thanks,
Rong Chen


Attachments:
(No filename) (1.51 kB)
config-5.9.0-rc1-00126-g977f739b7126b (172.86 kB)
job-script (5.31 kB)
kmsg.xz (43.27 kB)
perf-test (150.02 kB)
job.yaml (4.43 kB)
reproduce (684.00 B)
Download all attachments

2020-09-30 12:42:51

by Michael Petlan

[permalink] [raw]
Subject: Re: [perf report] 977f739b71: perf-test.skid_test.round2.MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_withou t_NMI_fixed_period.fail

Hello. To me, it looks rather a random failure. Is it actually reproducible
and proven that the patch has caused it? The patch doesn't seem to cause
the fails below.
Cheers,
Michael

On Wed, 30 Sep 2020, kernel test robot wrote:
> Greeting,
>
> FYI, we noticed the following commit (built with gcc-9):
>
> commit: 977f739b7126bf98b5202e243f60cbc0a1ec2c3b ("perf report: Disable ordered_events for raw dump")
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>
>
> in testcase: perf-test
> version: perf-test-x86_64-git-1_20200717
> with following parameters:
>
> type: lkp
> ucode: 0xd6
>
> test-description: The internal Perf Test suite.
>
>
> on test machine: 4 threads Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz with 32G memory
>
> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
>
>
>
>
> If you fix the issue, kindly add following tag
> Reported-by: kernel test robot <[email protected]>
>
>
> 2020-09-28 17:02:40 ./skid_test/run.sh
> Mon Sep 28 17:02:58 UTC 2020
> 3 test cases pass for skid_test test. 3 test cases fail for skid_test test.
> Test Case BR_INST_RETIRED.ALL_BRANCHES_PEBS_triad_loop_with_NMI PASS!
> Test Case MEM_LOAD_RETIRED.L2_HIT_dtlb_with_NMI PASS!
> Test Case MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_with_NMI FAILED!
> Test Case BR_INST_RETIRED.ALL_BRANCHES_PEBS_triad_loop_without_NMI_fixed_period PASS!
> Test Case MEM_LOAD_RETIRED.L2_HIT_dtlb_without_NMI_fixed_period FAILED!
> Test Case MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_without_NMI_fixed_period FAILED!
>
>
>
> To reproduce:
>
> git clone https://github.com/intel/lkp-tests.git
> cd lkp-tests
> bin/lkp install job.yaml # job file is attached in this email
> bin/lkp run job.yaml
>
>
>
> Thanks,
> Rong Chen
>
>

2020-10-02 05:50:18

by Philip Li

[permalink] [raw]
Subject: Re: [perf report] 977f739b71: perf-test.skid_test.round2.MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_withou t_NMI_fixed_period.fail

On Wed, Sep 30, 2020 at 02:41:07PM +0200, Michael Petlan wrote:
> Hello. To me, it looks rather a random failure. Is it actually reproducible
> and proven that the patch has caused it? The patch doesn't seem to cause
thanks for the input, we will look into this and provide update. Now since
we are in Chinese national holiday (until Oct 9), please allow some time
before we get back to work on this.

Thanks

> the fails below.
> Cheers,
> Michael
>
> On Wed, 30 Sep 2020, kernel test robot wrote:
> > Greeting,
> >
> > FYI, we noticed the following commit (built with gcc-9):
> >
> > commit: 977f739b7126bf98b5202e243f60cbc0a1ec2c3b ("perf report: Disable ordered_events for raw dump")
> > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
> >
> >
> > in testcase: perf-test
> > version: perf-test-x86_64-git-1_20200717
> > with following parameters:
> >
> > type: lkp
> > ucode: 0xd6
> >
> > test-description: The internal Perf Test suite.
> >
> >
> > on test machine: 4 threads Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz with 32G memory
> >
> > caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
> >
> >
> >
> >
> > If you fix the issue, kindly add following tag
> > Reported-by: kernel test robot <[email protected]>
> >
> >
> > 2020-09-28 17:02:40 ./skid_test/run.sh
> > Mon Sep 28 17:02:58 UTC 2020
> > 3 test cases pass for skid_test test. 3 test cases fail for skid_test test.
> > Test Case BR_INST_RETIRED.ALL_BRANCHES_PEBS_triad_loop_with_NMI PASS!
> > Test Case MEM_LOAD_RETIRED.L2_HIT_dtlb_with_NMI PASS!
> > Test Case MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_with_NMI FAILED!
> > Test Case BR_INST_RETIRED.ALL_BRANCHES_PEBS_triad_loop_without_NMI_fixed_period PASS!
> > Test Case MEM_LOAD_RETIRED.L2_HIT_dtlb_without_NMI_fixed_period FAILED!
> > Test Case MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_without_NMI_fixed_period FAILED!
> >
> >
> >
> > To reproduce:
> >
> > git clone https://github.com/intel/lkp-tests.git
> > cd lkp-tests
> > bin/lkp install job.yaml # job file is attached in this email
> > bin/lkp run job.yaml
> >
> >
> >
> > Thanks,
> > Rong Chen
> >
> >
>

2020-10-02 08:25:16

by Chen, Rong A

[permalink] [raw]
Subject: Re: [perf report] 977f739b71: perf-test.skid_test.round2.MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_withou t_NMI_fixed_period.fail



On 9/30/2020 8:41 PM, Michael Petlan wrote:
> Hello. To me, it looks rather a random failure. Is it actually reproducible
> and proven that the patch has caused it? The patch doesn't seem to cause
> the fails below.

Hi Michael,

It's not always reproducible, but we can reproduce it in most cases:

39c0a53b114d0317 977f739b7126bf98b5202e243f6
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:8 75% 6:8
perf-test.skid_test.round2.MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_with_NMI.fail
:8 88% 7:8
perf-test.skid_test.round2.MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_without_NMI_fixed_period.fail
:8 25% 2:8
perf-test.skid_test.round2.MEM_LOAD_RETIRED.L2_HIT_dtlb_with_NMI.fail
:8 62% 5:8
perf-test.skid_test.round2.MEM_LOAD_RETIRED.L2_HIT_dtlb_without_NMI_fixed_period.fail

Best Regards,
Rong Chen


> Cheers,
> Michael
>
> On Wed, 30 Sep 2020, kernel test robot wrote:
>> Greeting,
>>
>> FYI, we noticed the following commit (built with gcc-9):
>>
>> commit: 977f739b7126bf98b5202e243f60cbc0a1ec2c3b ("perf report: Disable ordered_events for raw dump")
>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>>
>>
>> in testcase: perf-test
>> version: perf-test-x86_64-git-1_20200717
>> with following parameters:
>>
>> type: lkp
>> ucode: 0xd6
>>
>> test-description: The internal Perf Test suite.
>>
>>
>> on test machine: 4 threads Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz with 32G memory
>>
>> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
>>
>>
>>
>>
>> If you fix the issue, kindly add following tag
>> Reported-by: kernel test robot <[email protected]>
>>
>>
>> 2020-09-28 17:02:40 ./skid_test/run.sh
>> Mon Sep 28 17:02:58 UTC 2020
>> 3 test cases pass for skid_test test. 3 test cases fail for skid_test test.
>> Test Case BR_INST_RETIRED.ALL_BRANCHES_PEBS_triad_loop_with_NMI PASS!
>> Test Case MEM_LOAD_RETIRED.L2_HIT_dtlb_with_NMI PASS!
>> Test Case MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_with_NMI FAILED!
>> Test Case BR_INST_RETIRED.ALL_BRANCHES_PEBS_triad_loop_without_NMI_fixed_period PASS!
>> Test Case MEM_LOAD_RETIRED.L2_HIT_dtlb_without_NMI_fixed_period FAILED!
>> Test Case MEM_LOAD_L3_MISS_RETIRED.LOCAL_DRAM_dtlb_without_NMI_fixed_period FAILED!
>>
>>
>>
>> To reproduce:
>>
>> git clone https://github.com/intel/lkp-tests.git
>> cd lkp-tests
>> bin/lkp install job.yaml # job file is attached in this email
>> bin/lkp run job.yaml
>>
>>
>>
>> Thanks,
>> Rong Chen
>>
>>
>