2022-01-18 03:03:16

by Paul Menzel

[permalink] [raw]
Subject: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

Dear Linux folks,


On an IBM S822LC SL822 (ppc64le) with Ubuntu 21.04, I tried to torture
the RCU. allmodconfig(2) failed though.

```
$ git describe
v5.16-10660-g0c947b893d69
$ tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
--- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
--- Results directory: 2022.01.17-18.44.36-torture
--- allmodconfig: Start Mon Jan 17 06:44:36 PM CET 2022
--- rcutorture: Start Mon Jan 17 06:45:02 PM CET 2022
--- rcutorture-kasan: Start Mon Jan 17 07:07:41 PM CET 2022
--- locktorture: Start Mon Jan 17 07:33:38 PM CET 2022
--- locktorture-kasan: Start Mon Jan 17 07:37:05 PM CET 2022
--- scftorture: Start Mon Jan 17 07:40:54 PM CET 2022
--- scftorture-kasan: Start Mon Jan 17 07:43:31 PM CET 2022
--- refscale-rcu: Start Mon Jan 17 07:46:34 PM CET 2022
--- refscale-rcu-kasan: Start Mon Jan 17 07:49:08 PM CET 2022
--- refscale-srcu: Start Mon Jan 17 07:52:05 PM CET 2022
--- refscale-srcu-kasan: Start Mon Jan 17 07:54:39 PM CET 2022
--- refscale-rcu-tasks: Start Mon Jan 17 07:57:35 PM CET 2022
--- refscale-rcu-tasks-kasan: Start Mon Jan 17 08:00:10 PM CET 2022
--- refscale-rcu-trace: Start Mon Jan 17 08:03:06 PM CET 2022
--- refscale-rcu-trace-kasan: Start Mon Jan 17 08:05:40 PM CET 2022
--- refscale-refcnt: Start Mon Jan 17 08:08:40 PM CET 2022
--- refscale-refcnt-kasan: Start Mon Jan 17 08:11:15 PM CET 2022
--- refscale-rwlock: Start Mon Jan 17 08:14:14 PM CET 2022
--- refscale-rwlock-kasan: Start Mon Jan 17 08:16:48 PM CET 2022
--- refscale-rwsem: Start Mon Jan 17 08:19:45 PM CET 2022
--- refscale-rwsem-kasan: Start Mon Jan 17 08:22:18 PM CET 2022
--- refscale-lock: Start Mon Jan 17 08:25:17 PM CET 2022
--- refscale-lock-kasan: Start Mon Jan 17 08:27:53 PM CET 2022
--- refscale-lock-irq: Start Mon Jan 17 08:30:58 PM CET 2022
--- refscale-lock-irq-kasan: Start Mon Jan 17 08:33:42 PM CET 2022
--- refscale-acqrel: Start Mon Jan 17 08:36:58 PM CET 2022
--- refscale-acqrel-kasan: Start Mon Jan 17 08:39:33 PM CET 2022
--- refscale-clock: Start Mon Jan 17 08:42:33 PM CET 2022
--- refscale-clock-kasan: Start Mon Jan 17 08:45:08 PM CET 2022
--- rcuscale-rcu: Start Mon Jan 17 08:48:06 PM CET 2022
--- rcuscale-rcu-kasan: Start Mon Jan 17 08:49:25 PM CET 2022
--- rcuscale-srcu: Start Mon Jan 17 08:50:56 PM CET 2022
--- rcuscale-srcu-kasan: Start Mon Jan 17 08:52:14 PM CET 2022
--- rcuscale-srcud: Start Mon Jan 17 08:53:43 PM CET 2022
--- rcuscale-srcud-kasan: Start Mon Jan 17 08:55:01 PM CET 2022
--- rcuscale-tasks: Start Mon Jan 17 08:56:31 PM CET 2022
--- rcuscale-tasks-kasan: Start Mon Jan 17 08:57:48 PM CET 2022
--- rcuscale-tasks-tracing: Start Mon Jan 17 08:59:17 PM CET 2022
--- rcuscale-tasks-tracing-kasan: Start Mon Jan 17 09:00:35 PM CET 2022
--- rcuscale-kvfree: Start Mon Jan 17 09:02:05 PM CET 2022
--- rcuscale-kvfree-kasan: Start Mon Jan 17 09:03:23 PM CET 2022
--- clocksourcewd-1: Start Mon Jan 17 09:04:52 PM CET 2022
--- clocksourcewd-1-kasan: Start Mon Jan 17 09:06:11 PM CET 2022
--- clocksourcewd-2: Start Mon Jan 17 09:07:42 PM CET 2022
--- clocksourcewd-2-kasan: Start Mon Jan 17 09:08:59 PM CET 2022
--- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
--- Done Mon Jan 17 09:10:29 PM CET 2022
SUCCESSES:
rcutorture(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture
rcutorture-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture-kasan
locktorture(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-locktorture
locktorture-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-locktorture-kasan
scftorture(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-scftorture
scftorture-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-scftorture-kasan
refscale-rcu(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu
refscale-rcu-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-kasan
refscale-srcu(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-srcu
refscale-srcu-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-srcu-kasan
refscale-rcu-tasks(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-tasks
refscale-rcu-tasks-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-tasks-kasan
refscale-rcu-trace(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-trace
refscale-rcu-trace-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-trace-kasan
refscale-refcnt(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-refcnt
refscale-refcnt-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-refcnt-kasan
refscale-rwlock(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rwlock
refscale-rwlock-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rwlock-kasan
refscale-rwsem(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rwsem
refscale-rwsem-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rwsem-kasan
refscale-lock(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-lock
refscale-lock-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-lock-kasan
refscale-lock-irq(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-lock-irq
refscale-lock-irq-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-lock-irq-kasan
refscale-acqrel(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-acqrel
refscale-acqrel-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-acqrel-kasan
refscale-clock(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-clock
refscale-clock-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-clock-kasan
rcuscale-rcu(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-rcu
rcuscale-rcu-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-rcu-kasan
rcuscale-srcu(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-srcu
rcuscale-srcu-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-srcu-kasan
rcuscale-srcud(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-srcud
rcuscale-srcud-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-srcud-kasan
rcuscale-tasks(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-tasks
rcuscale-tasks-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-tasks-kasan
rcuscale-tasks-tracing(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-tasks-tracing
rcuscale-tasks-tracing-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-tasks-tracing-kasan
rcuscale-kvfree(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-kvfree
rcuscale-kvfree-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-kvfree-kasan
clocksourcewd-1(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-clocksourcewd-1
clocksourcewd-1-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-clocksourcewd-1-kasan
clocksourcewd-2(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-clocksourcewd-2
clocksourcewd-2-kasan(0)
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-clocksourcewd-2-kasan
FAILURES:
allmodconfig(2)
tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/allmodconfig
Started at Mon Jan 17 06:44:36 PM CET 2022, ended at Mon Jan 17 09:10:29
PM CET 2022, duration 2:25:53.
Summary: Successes: 44 Failures: 1.
Size before compressing 0 files: 86M Mon Jan 17 09:10:29 PM CET 2022
Waiting for final batch 1 of 22 compressions Mon Jan 17 09:10:29 PM CET 2022
Size after compressing 0 files: 86M Mon Jan 17 09:10:29 PM CET 2022
Total duration 2:25:53.
```

Here is an excerpt from `Make.out` [1]:

```
CC [M] sound/isa/ad1848/ad1848.o
CC security/keys/encrypted-keys/ecryptfs_format.o
In file included from <command-line>:
././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No
such file or directory
5 | #include <generated/autoconf.h>
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
CC [M] fs/reiserfs/inode.o
In file included from <command-line>:
././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No
such file or directory
5 | #include <generated/autoconf.h>
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from <command-line>:
././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No
such file or directory
5 | #include <generated/autoconf.h>
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [scripts/Makefile.build:288:
security/keys/encrypted-keys/ecryptfs_format.o] Error 1
make[3]: *** Waiting for unfinished jobs....
In file included from <command-line>:
././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No
such file or directory
5 | #include <generated/autoconf.h>
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
CC [M] sound/i2c/i2c.o
In file included from <command-line>:
././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No
such file or directory
5 | #include <generated/autoconf.h>
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
CC [M] fs/ext4/block_validity.o
make[3]: *** [scripts/Makefile.build:288: sound/isa/ad1848/ad1848.o] Error 1
make[2]: *** [scripts/Makefile.build:550: sound/isa/ad1848] Error 2
make[1]: *** [scripts/Makefile.build:550: sound/isa] Error 2
make[1]: *** Waiting for unfinished jobs....
```

I wonder why other builds seem to have worked. But checking the other
results, despite the output saying the runs were a success the Linux
kernel builds seem to have failed. For example [2]:

```
$ more
tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture/log
tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 7
--configs CFLIST --trust-make --bootargs
rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot
rcupdate.rcu_task_stall_timeout=30000 --datestamp
2022.01.17-18.44.36-torture/results-rcutorture
----Start batch 1: Mon Jan 17 06:45:03 PM CET 2022
TREE03 16: Starting build. Mon Jan 17 06:45:03 PM CET 2022
[…]
TREE09 1: Build complete. Mon Jan 17 07:07:40 PM CET 2022
---- No kernel runs. Mon Jan 17 07:07:40 PM CET 2022
---- TREE03 16: Build/run results:
--- Mon Jan 17 06:45:03 PM CET 2022: Starting build, PID 3530842
:CONFIG_HYPERVISOR_GUEST=y: improperly set
:CONFIG_PARAVIRT=y: improperly set
Kernel build error
make[2]: *** No rule to make target
'arch/powerpc/kvm//dev/shm/linux/tools/testing/selftests/rcutorture/kvm_main.o',
needed by 'arch/powerpc/kvm/kvm.o'. Stop.
make[1]: *** [scripts/Makefile.build:550: arch/powerpc/kvm] Error 2
make: *** [Makefile:1840: arch/powerpc] Error 2
Run aborted.
Build failed, not running KVM, see
/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture/TREE03.
[…]
ESC[031mBUG: ESC[mBuild failed

/dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture/TREE09
--- Done at Mon Jan 17 07:07:41 PM CET 2022 (22:39.0) exitcode 0
```


Kind regards,

Paul


[1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
[2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt


2022-01-19 20:45:19

by Zhouyi Zhou

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

Hi,
I have studied the rcu torture test recently. I am also interested in
this topic.
But I can't open
[1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
[2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt

From your email, It seems that include/generated/autoconf.h is not
generated in the early stage of kernel build.

Could you try following one by one if you have time ;-)
after invoking each of the following commands, we can examine if
include/generated/autoconf.h is created in another terminal, and if it
is
created, we can try the next command.
tools/testing/selftests/rcutorture/bin/kvm.sh --buildonly --configs "TREE01"
tools/testing/selftests/rcutorture/bin/kvm.sh --buildonly --configs "TREE02"
tools/testing/selftests/rcutorture/bin/kvm.sh --buildonly --configs "TREE03"
tools/testing/selftests/rcutorture/bin/kvm.sh --buildonly --configs "TREE04"
tools/testing/selftests/rcutorture/bin/kvm.sh --buildonly --configs "TREE05"
tools/testing/selftests/rcutorture/bin/kvm.sh --buildonly --configs "TREE06"
tools/testing/selftests/rcutorture/bin/kvm.sh --buildonly --configs "TREE07"
tools/testing/selftests/rcutorture/bin/kvm.sh --buildonly --configs "TREE08"
tools/testing/selftests/rcutorture/bin/kvm.sh --buildonly --configs "TREE09"
tools/testing/selftests/rcutorture/bin/kvm.sh --buildonly --configs "TREE10"

I am a beginner at learning RCU ;-) I may not be right. Apologize in
advance if I am not right.

Thanks
Zhouyi

On Tue, Jan 18, 2022 at 11:03 AM Paul Menzel <[email protected]> wrote:
>
> Dear Linux folks,
>
>
> On an IBM S822LC SL822 (ppc64le) with Ubuntu 21.04, I tried to torture
> the RCU. allmodconfig(2) failed though.
>
> ```
> $ git describe
> v5.16-10660-g0c947b893d69
> $ tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> --- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> --- Results directory: 2022.01.17-18.44.36-torture
> --- allmodconfig: Start Mon Jan 17 06:44:36 PM CET 2022
> --- rcutorture: Start Mon Jan 17 06:45:02 PM CET 2022
> --- rcutorture-kasan: Start Mon Jan 17 07:07:41 PM CET 2022
> --- locktorture: Start Mon Jan 17 07:33:38 PM CET 2022
> --- locktorture-kasan: Start Mon Jan 17 07:37:05 PM CET 2022
> --- scftorture: Start Mon Jan 17 07:40:54 PM CET 2022
> --- scftorture-kasan: Start Mon Jan 17 07:43:31 PM CET 2022
> --- refscale-rcu: Start Mon Jan 17 07:46:34 PM CET 2022
> --- refscale-rcu-kasan: Start Mon Jan 17 07:49:08 PM CET 2022
> --- refscale-srcu: Start Mon Jan 17 07:52:05 PM CET 2022
> --- refscale-srcu-kasan: Start Mon Jan 17 07:54:39 PM CET 2022
> --- refscale-rcu-tasks: Start Mon Jan 17 07:57:35 PM CET 2022
> --- refscale-rcu-tasks-kasan: Start Mon Jan 17 08:00:10 PM CET 2022
> --- refscale-rcu-trace: Start Mon Jan 17 08:03:06 PM CET 2022
> --- refscale-rcu-trace-kasan: Start Mon Jan 17 08:05:40 PM CET 2022
> --- refscale-refcnt: Start Mon Jan 17 08:08:40 PM CET 2022
> --- refscale-refcnt-kasan: Start Mon Jan 17 08:11:15 PM CET 2022
> --- refscale-rwlock: Start Mon Jan 17 08:14:14 PM CET 2022
> --- refscale-rwlock-kasan: Start Mon Jan 17 08:16:48 PM CET 2022
> --- refscale-rwsem: Start Mon Jan 17 08:19:45 PM CET 2022
> --- refscale-rwsem-kasan: Start Mon Jan 17 08:22:18 PM CET 2022
> --- refscale-lock: Start Mon Jan 17 08:25:17 PM CET 2022
> --- refscale-lock-kasan: Start Mon Jan 17 08:27:53 PM CET 2022
> --- refscale-lock-irq: Start Mon Jan 17 08:30:58 PM CET 2022
> --- refscale-lock-irq-kasan: Start Mon Jan 17 08:33:42 PM CET 2022
> --- refscale-acqrel: Start Mon Jan 17 08:36:58 PM CET 2022
> --- refscale-acqrel-kasan: Start Mon Jan 17 08:39:33 PM CET 2022
> --- refscale-clock: Start Mon Jan 17 08:42:33 PM CET 2022
> --- refscale-clock-kasan: Start Mon Jan 17 08:45:08 PM CET 2022
> --- rcuscale-rcu: Start Mon Jan 17 08:48:06 PM CET 2022
> --- rcuscale-rcu-kasan: Start Mon Jan 17 08:49:25 PM CET 2022
> --- rcuscale-srcu: Start Mon Jan 17 08:50:56 PM CET 2022
> --- rcuscale-srcu-kasan: Start Mon Jan 17 08:52:14 PM CET 2022
> --- rcuscale-srcud: Start Mon Jan 17 08:53:43 PM CET 2022
> --- rcuscale-srcud-kasan: Start Mon Jan 17 08:55:01 PM CET 2022
> --- rcuscale-tasks: Start Mon Jan 17 08:56:31 PM CET 2022
> --- rcuscale-tasks-kasan: Start Mon Jan 17 08:57:48 PM CET 2022
> --- rcuscale-tasks-tracing: Start Mon Jan 17 08:59:17 PM CET 2022
> --- rcuscale-tasks-tracing-kasan: Start Mon Jan 17 09:00:35 PM CET 2022
> --- rcuscale-kvfree: Start Mon Jan 17 09:02:05 PM CET 2022
> --- rcuscale-kvfree-kasan: Start Mon Jan 17 09:03:23 PM CET 2022
> --- clocksourcewd-1: Start Mon Jan 17 09:04:52 PM CET 2022
> --- clocksourcewd-1-kasan: Start Mon Jan 17 09:06:11 PM CET 2022
> --- clocksourcewd-2: Start Mon Jan 17 09:07:42 PM CET 2022
> --- clocksourcewd-2-kasan: Start Mon Jan 17 09:08:59 PM CET 2022
> --- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> --- Done Mon Jan 17 09:10:29 PM CET 2022
> SUCCESSES:
> rcutorture(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture
> rcutorture-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture-kasan
> locktorture(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-locktorture
> locktorture-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-locktorture-kasan
> scftorture(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-scftorture
> scftorture-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-scftorture-kasan
> refscale-rcu(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu
> refscale-rcu-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-kasan
> refscale-srcu(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-srcu
> refscale-srcu-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-srcu-kasan
> refscale-rcu-tasks(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-tasks
> refscale-rcu-tasks-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-tasks-kasan
> refscale-rcu-trace(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-trace
> refscale-rcu-trace-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-trace-kasan
> refscale-refcnt(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-refcnt
> refscale-refcnt-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-refcnt-kasan
> refscale-rwlock(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rwlock
> refscale-rwlock-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rwlock-kasan
> refscale-rwsem(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rwsem
> refscale-rwsem-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rwsem-kasan
> refscale-lock(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-lock
> refscale-lock-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-lock-kasan
> refscale-lock-irq(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-lock-irq
> refscale-lock-irq-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-lock-irq-kasan
> refscale-acqrel(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-acqrel
> refscale-acqrel-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-acqrel-kasan
> refscale-clock(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-clock
> refscale-clock-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-clock-kasan
> rcuscale-rcu(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-rcu
> rcuscale-rcu-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-rcu-kasan
> rcuscale-srcu(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-srcu
> rcuscale-srcu-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-srcu-kasan
> rcuscale-srcud(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-srcud
> rcuscale-srcud-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-srcud-kasan
> rcuscale-tasks(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-tasks
> rcuscale-tasks-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-tasks-kasan
> rcuscale-tasks-tracing(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-tasks-tracing
> rcuscale-tasks-tracing-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-tasks-tracing-kasan
> rcuscale-kvfree(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-kvfree
> rcuscale-kvfree-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-kvfree-kasan
> clocksourcewd-1(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-clocksourcewd-1
> clocksourcewd-1-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-clocksourcewd-1-kasan
> clocksourcewd-2(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-clocksourcewd-2
> clocksourcewd-2-kasan(0)
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-clocksourcewd-2-kasan
> FAILURES:
> allmodconfig(2)
> tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/allmodconfig
> Started at Mon Jan 17 06:44:36 PM CET 2022, ended at Mon Jan 17 09:10:29
> PM CET 2022, duration 2:25:53.
> Summary: Successes: 44 Failures: 1.
> Size before compressing 0 files: 86M Mon Jan 17 09:10:29 PM CET 2022
> Waiting for final batch 1 of 22 compressions Mon Jan 17 09:10:29 PM CET 2022
> Size after compressing 0 files: 86M Mon Jan 17 09:10:29 PM CET 2022
> Total duration 2:25:53.
> ```
>
> Here is an excerpt from `Make.out` [1]:
>
> ```
> CC [M] sound/isa/ad1848/ad1848.o
> CC security/keys/encrypted-keys/ecryptfs_format.o
> In file included from <command-line>:
> ././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No
> such file or directory
> 5 | #include <generated/autoconf.h>
> | ^~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
> CC [M] fs/reiserfs/inode.o
> In file included from <command-line>:
> ././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No
> such file or directory
> 5 | #include <generated/autoconf.h>
> | ^~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
> In file included from <command-line>:
> ././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No
> such file or directory
> 5 | #include <generated/autoconf.h>
> | ^~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
> make[3]: *** [scripts/Makefile.build:288:
> security/keys/encrypted-keys/ecryptfs_format.o] Error 1
> make[3]: *** Waiting for unfinished jobs....
> In file included from <command-line>:
> ././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No
> such file or directory
> 5 | #include <generated/autoconf.h>
> | ^~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
> CC [M] sound/i2c/i2c.o
> In file included from <command-line>:
> ././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No
> such file or directory
> 5 | #include <generated/autoconf.h>
> | ^~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
> CC [M] fs/ext4/block_validity.o
> make[3]: *** [scripts/Makefile.build:288: sound/isa/ad1848/ad1848.o] Error 1
> make[2]: *** [scripts/Makefile.build:550: sound/isa/ad1848] Error 2
> make[1]: *** [scripts/Makefile.build:550: sound/isa] Error 2
> make[1]: *** Waiting for unfinished jobs....
> ```
>
> I wonder why other builds seem to have worked. But checking the other
> results, despite the output saying the runs were a success the Linux
> kernel builds seem to have failed. For example [2]:
>
> ```
> $ more
> tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture/log
> tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 7
> --configs CFLIST --trust-make --bootargs
> rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot
> rcupdate.rcu_task_stall_timeout=30000 --datestamp
> 2022.01.17-18.44.36-torture/results-rcutorture
> ----Start batch 1: Mon Jan 17 06:45:03 PM CET 2022
> TREE03 16: Starting build. Mon Jan 17 06:45:03 PM CET 2022
> […]
> TREE09 1: Build complete. Mon Jan 17 07:07:40 PM CET 2022
> ---- No kernel runs. Mon Jan 17 07:07:40 PM CET 2022
> ---- TREE03 16: Build/run results:
> --- Mon Jan 17 06:45:03 PM CET 2022: Starting build, PID 3530842
> :CONFIG_HYPERVISOR_GUEST=y: improperly set
> :CONFIG_PARAVIRT=y: improperly set
> Kernel build error
> make[2]: *** No rule to make target
> 'arch/powerpc/kvm//dev/shm/linux/tools/testing/selftests/rcutorture/kvm_main.o',
> needed by 'arch/powerpc/kvm/kvm.o'. Stop.
> make[1]: *** [scripts/Makefile.build:550: arch/powerpc/kvm] Error 2
> make: *** [Makefile:1840: arch/powerpc] Error 2
> Run aborted.
> Build failed, not running KVM, see
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture/TREE03.
> […]
> ESC[031mBUG: ESC[mBuild failed
>
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture/TREE09
> --- Done at Mon Jan 17 07:07:41 PM CET 2022 (22:39.0) exitcode 0
> ```
>
>
> Kind regards,
>
> Paul
>
>
> [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt

2022-01-19 22:05:03

by Paul Menzel

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

Dear Zhouyi,


Thank you for your quick response.


Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:

> I have studied the rcu torture test recently. I am also interested in
> this topic.
> But I can't open
> [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt

Sorry, about that. I should have checked those. I had put them into a
directory:

[1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
[2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt

I am going to try to test your suggestions at the end of the day.


Kind regards,

Paul

2022-01-19 22:05:51

by Paul Menzel

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

[Correct references]


Am 17.01.22 um 23:00 schrieb Paul Menzel:
> Dear Linux folks,
>
>
> On an IBM S822LC SL822 (ppc64le) with Ubuntu 21.04, I tried to torture
> the RCU. allmodconfig(2) failed though.
>
> ```
> $ git describe
> v5.16-10660-g0c947b893d69
> $ tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
>  ---  tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
>  --- Results directory:  2022.01.17-18.44.36-torture
>  --- allmodconfig: Start Mon Jan 17 06:44:36 PM CET 2022
>  --- rcutorture: Start Mon Jan 17 06:45:02 PM CET 2022
>  --- rcutorture-kasan: Start Mon Jan 17 07:07:41 PM CET 2022
>  --- locktorture: Start Mon Jan 17 07:33:38 PM CET 2022
>  --- locktorture-kasan: Start Mon Jan 17 07:37:05 PM CET 2022
>  --- scftorture: Start Mon Jan 17 07:40:54 PM CET 2022
>  --- scftorture-kasan: Start Mon Jan 17 07:43:31 PM CET 2022
>  --- refscale-rcu: Start Mon Jan 17 07:46:34 PM CET 2022
>  --- refscale-rcu-kasan: Start Mon Jan 17 07:49:08 PM CET 2022
>  --- refscale-srcu: Start Mon Jan 17 07:52:05 PM CET 2022
>  --- refscale-srcu-kasan: Start Mon Jan 17 07:54:39 PM CET 2022
>  --- refscale-rcu-tasks: Start Mon Jan 17 07:57:35 PM CET 2022
>  --- refscale-rcu-tasks-kasan: Start Mon Jan 17 08:00:10 PM CET 2022
>  --- refscale-rcu-trace: Start Mon Jan 17 08:03:06 PM CET 2022
>  --- refscale-rcu-trace-kasan: Start Mon Jan 17 08:05:40 PM CET 2022
>  --- refscale-refcnt: Start Mon Jan 17 08:08:40 PM CET 2022
>  --- refscale-refcnt-kasan: Start Mon Jan 17 08:11:15 PM CET 2022
>  --- refscale-rwlock: Start Mon Jan 17 08:14:14 PM CET 2022
>  --- refscale-rwlock-kasan: Start Mon Jan 17 08:16:48 PM CET 2022
>  --- refscale-rwsem: Start Mon Jan 17 08:19:45 PM CET 2022
>  --- refscale-rwsem-kasan: Start Mon Jan 17 08:22:18 PM CET 2022
>  --- refscale-lock: Start Mon Jan 17 08:25:17 PM CET 2022
>  --- refscale-lock-kasan: Start Mon Jan 17 08:27:53 PM CET 2022
>  --- refscale-lock-irq: Start Mon Jan 17 08:30:58 PM CET 2022
>  --- refscale-lock-irq-kasan: Start Mon Jan 17 08:33:42 PM CET 2022
>  --- refscale-acqrel: Start Mon Jan 17 08:36:58 PM CET 2022
>  --- refscale-acqrel-kasan: Start Mon Jan 17 08:39:33 PM CET 2022
>  --- refscale-clock: Start Mon Jan 17 08:42:33 PM CET 2022
>  --- refscale-clock-kasan: Start Mon Jan 17 08:45:08 PM CET 2022
>  --- rcuscale-rcu: Start Mon Jan 17 08:48:06 PM CET 2022
>  --- rcuscale-rcu-kasan: Start Mon Jan 17 08:49:25 PM CET 2022
>  --- rcuscale-srcu: Start Mon Jan 17 08:50:56 PM CET 2022
>  --- rcuscale-srcu-kasan: Start Mon Jan 17 08:52:14 PM CET 2022
>  --- rcuscale-srcud: Start Mon Jan 17 08:53:43 PM CET 2022
>  --- rcuscale-srcud-kasan: Start Mon Jan 17 08:55:01 PM CET 2022
>  --- rcuscale-tasks: Start Mon Jan 17 08:56:31 PM CET 2022
>  --- rcuscale-tasks-kasan: Start Mon Jan 17 08:57:48 PM CET 2022
>  --- rcuscale-tasks-tracing: Start Mon Jan 17 08:59:17 PM CET 2022
>  --- rcuscale-tasks-tracing-kasan: Start Mon Jan 17 09:00:35 PM CET 2022
>  --- rcuscale-kvfree: Start Mon Jan 17 09:02:05 PM CET 2022
>  --- rcuscale-kvfree-kasan: Start Mon Jan 17 09:03:23 PM CET 2022
>  --- clocksourcewd-1: Start Mon Jan 17 09:04:52 PM CET 2022
>  --- clocksourcewd-1-kasan: Start Mon Jan 17 09:06:11 PM CET 2022
>  --- clocksourcewd-2: Start Mon Jan 17 09:07:42 PM CET 2022
>  --- clocksourcewd-2-kasan: Start Mon Jan 17 09:08:59 PM CET 2022
>  ---  tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
>  ---  Done Mon Jan 17 09:10:29 PM CET 2022
> SUCCESSES:
> rcutorture(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture
> rcutorture-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture-kasan
> locktorture(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-locktorture
> locktorture-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-locktorture-kasan
> scftorture(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-scftorture
> scftorture-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-scftorture-kasan
> refscale-rcu(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu
> refscale-rcu-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-kasan
> refscale-srcu(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-srcu
> refscale-srcu-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-srcu-kasan
> refscale-rcu-tasks(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-tasks
> refscale-rcu-tasks-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-tasks-kasan
> refscale-rcu-trace(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-trace
> refscale-rcu-trace-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rcu-trace-kasan
> refscale-refcnt(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-refcnt
> refscale-refcnt-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-refcnt-kasan
> refscale-rwlock(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rwlock
> refscale-rwlock-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rwlock-kasan
> refscale-rwsem(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rwsem
> refscale-rwsem-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-rwsem-kasan
> refscale-lock(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-lock
> refscale-lock-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-lock-kasan
> refscale-lock-irq(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-lock-irq
> refscale-lock-irq-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-lock-irq-kasan
> refscale-acqrel(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-acqrel
> refscale-acqrel-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-acqrel-kasan
> refscale-clock(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-clock
> refscale-clock-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-refscale-clock-kasan
> rcuscale-rcu(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-rcu
> rcuscale-rcu-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-rcu-kasan
> rcuscale-srcu(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-srcu
> rcuscale-srcu-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-srcu-kasan
> rcuscale-srcud(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-srcud
> rcuscale-srcud-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-srcud-kasan
> rcuscale-tasks(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-tasks
> rcuscale-tasks-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-tasks-kasan
> rcuscale-tasks-tracing(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-tasks-tracing
> rcuscale-tasks-tracing-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-tasks-tracing-kasan
> rcuscale-kvfree(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-kvfree
> rcuscale-kvfree-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcuscale-kvfree-kasan
> clocksourcewd-1(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-clocksourcewd-1
> clocksourcewd-1-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-clocksourcewd-1-kasan
> clocksourcewd-2(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-clocksourcewd-2
> clocksourcewd-2-kasan(0) /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-clocksourcewd-2-kasan
>
> FAILURES:
> allmodconfig(2) tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/allmodconfig
> Started at Mon Jan 17 06:44:36 PM CET 2022, ended at Mon Jan 17 09:10:29
> PM CET 2022, duration 2:25:53.
> Summary: Successes: 44 Failures: 1.
> Size before compressing 0 files: 86M Mon Jan 17 09:10:29 PM CET 2022
> Waiting for final batch 1 of 22 compressions Mon Jan 17 09:10:29 PM CET 2022
> Size after compressing 0 files: 86M Mon Jan 17 09:10:29 PM CET 2022
> Total duration 2:25:53.
> ```
>
> Here is an excerpt from `Make.out` [1]:
>
> ```
>   CC [M]  sound/isa/ad1848/ad1848.o
>   CC      security/keys/encrypted-keys/ecryptfs_format.o
> In file included from <command-line>:
> ././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No such file or directory
>     5 | #include <generated/autoconf.h>
>       |          ^~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
>   CC [M]  fs/reiserfs/inode.o
> In file included from <command-line>:
> ././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No such file or directory
>     5 | #include <generated/autoconf.h>
>       |          ^~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
> In file included from <command-line>:
> ././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No such file or directory
>     5 | #include <generated/autoconf.h>
>       |          ^~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
> make[3]: *** [scripts/Makefile.build:288:
> security/keys/encrypted-keys/ecryptfs_format.o] Error 1
> make[3]: *** Waiting for unfinished jobs....
> In file included from <command-line>:
> ././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No such file or directory
>     5 | #include <generated/autoconf.h>
>       |          ^~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
>   CC [M]  sound/i2c/i2c.o
> In file included from <command-line>:
> ././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No such file or directory
>     5 | #include <generated/autoconf.h>
>       |          ^~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
>   CC [M]  fs/ext4/block_validity.o
> make[3]: *** [scripts/Makefile.build:288: sound/isa/ad1848/ad1848.o]
> Error 1
> make[2]: *** [scripts/Makefile.build:550: sound/isa/ad1848] Error 2
> make[1]: *** [scripts/Makefile.build:550: sound/isa] Error 2
> make[1]: *** Waiting for unfinished jobs....
> ```
>
> I wonder why other builds seem to have worked. But checking the other
> results, despite the output saying the runs were a success the Linux
> kernel builds seem to have failed. For example [2]:
>
> ```
> $ more
> tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture/log
>
> tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 7
> --configs CFLIST --trust-make --bootargs rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 --datestamp 2022.01.17-18.44.36-torture/results-rcutorture
> ----Start batch 1: Mon Jan 17 06:45:03 PM CET 2022
> TREE03 16: Starting build. Mon Jan 17 06:45:03 PM CET 2022
> […]
> TREE09 1: Build complete. Mon Jan 17 07:07:40 PM CET 2022
> ---- No kernel runs. Mon Jan 17 07:07:40 PM CET 2022
> ---- TREE03 16: Build/run results:
>  --- Mon Jan 17 06:45:03 PM CET 2022: Starting build, PID 3530842
> :CONFIG_HYPERVISOR_GUEST=y: improperly set
> :CONFIG_PARAVIRT=y: improperly set
> Kernel build error
> make[2]: *** No rule to make target
> 'arch/powerpc/kvm//dev/shm/linux/tools/testing/selftests/rcutorture/kvm_main.o', needed by 'arch/powerpc/kvm/kvm.o'.  Stop.
> make[1]: *** [scripts/Makefile.build:550: arch/powerpc/kvm] Error 2
> make: *** [Makefile:1840: arch/powerpc] Error 2
> Run aborted.
> Build failed, not running KVM, see
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture/TREE03.
>
> […]
> ESC[031mBUG: ESC[mBuild failed
>
> /dev/shm/linux/tools/testing/selftests/rcutorture/res/2022.01.17-18.44.36-torture/results-rcutorture/TREE09
>
>  --- Done at Mon Jan 17 07:07:41 PM CET 2022 (22:39.0) exitcode 0
> ```
>
>
> Kind regards,
>
> Paul
>
>
> [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt

I botched those. The correct once are:

[1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
[2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt

2022-01-20 07:06:58

by Zhouyi Zhou

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

Dear Paul

On Tue, Jan 18, 2022 at 3:56 PM Paul Menzel <[email protected]> wrote:
>
> Dear Zhouyi,
>
>
> Thank you for your quick response.
You are very welcome ;-)
>
>
> Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
>
> > I have studied the rcu torture test recently. I am also interested in
> > this topic.
> > But I can't open
> > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
>
> Sorry, about that. I should have checked those. I had put them into a
> directory:
>
> [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
I can open these URLs now, thank you, I will try my best to analyze
the logs in the coming days.
>
> I am going to try to test your suggestions at the end of the day.
Thanks a lot, looking forward to your test results.

>
>
> Kind regards,
>
> Paul
Cheers
Zhouyi

2022-01-20 19:19:06

by Paul E. McKenney

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

On Tue, Jan 18, 2022 at 08:56:24AM +0100, Paul Menzel wrote:
> Dear Zhouyi,
>
>
> Thank you for your quick response.
>
>
> Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
>
> > I have studied the rcu torture test recently. I am also interested in
> > this topic.
> > But I can't open
> > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
>
> Sorry, about that. I should have checked those. I had put them into a
> directory:
>
> [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
>
> I am going to try to test your suggestions at the end of the day.

On x86 rcutorture builds successfully. However, allmodconfig
on semi-recent -next got me "Can't open perl script
"./usr/include/headers_check.pl": No such file or directory".
Which might well be a local problem or might well be fixed by now.

Either way, it looks like I need to upgrade the torture.sh script's
checks for failed builds. Thank you for reporting this!

Thanx, Paul

2022-01-21 11:52:41

by Paul E. McKenney

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

On Tue, Jan 18, 2022 at 09:29:04AM -0800, Paul E. McKenney wrote:
> On Tue, Jan 18, 2022 at 08:56:24AM +0100, Paul Menzel wrote:
> > Dear Zhouyi,
> >
> >
> > Thank you for your quick response.
> >
> >
> > Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
> >
> > > I have studied the rcu torture test recently. I am also interested in
> > > this topic.
> > > But I can't open
> > > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
> >
> > Sorry, about that. I should have checked those. I had put them into a
> > directory:
> >
> > [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
> >
> > I am going to try to test your suggestions at the end of the day.
>
> On x86 rcutorture builds successfully. However, allmodconfig
> on semi-recent -next got me "Can't open perl script
> "./usr/include/headers_check.pl": No such file or directory".
> Which might well be a local problem or might well be fixed by now.

Not fixed as of next-20220118. Chasing it down... ;-)

> Either way, it looks like I need to upgrade the torture.sh script's
> checks for failed builds. Thank you for reporting this!

Does this make torture.sh more reliably report build failures?

Thanx, Paul

------------------------------------------------------------------------

commit 0d302830515307ceb58e89d5fb91e81b6d22e0bf
Author: Paul E. McKenney <[email protected]>
Date: Tue Jan 18 15:40:49 2022 -0800

torture: Make kvm-find-errors.sh notice missing vmlinux file

Currently, an obtuse compiler diagnostic can fool kvm-find-errors.sh
into believing that the build was successful. This commit therefore
adds a check for a missing vmlinux file.

Link: https://lore.kernel.org/lkml/[email protected]/
Reported-by: Paul Menzel <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>

diff --git a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
index 2e9e9e2eedb69..7d3e11a6b8290 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
@@ -30,10 +30,15 @@ editor=${EDITOR-vi}
files=
for i in ${rundir}/*/Make.out
do
+ scenariodir="`dirname $i`"
if egrep -q "error:|warning:|^ld: .*undefined reference to" < $i
then
egrep "error:|warning:|^ld: .*undefined reference to" < $i > $i.diags
files="$files $i.diags $i"
+ elif ! test -f ${scenariodir}/vmlinux
+ then
+ echo No ${scenariodir}/vmlinux file > $i.diags
+ files="$files $i.diags $i"
fi
done
if test -n "$files"

2022-01-21 13:59:35

by Zhouyi Zhou

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

Thanks Paul for looking into this

On Wed, Jan 19, 2022 at 7:46 AM Paul E. McKenney <[email protected]> wrote:
>
> On Tue, Jan 18, 2022 at 09:29:04AM -0800, Paul E. McKenney wrote:
> > On Tue, Jan 18, 2022 at 08:56:24AM +0100, Paul Menzel wrote:
> > > Dear Zhouyi,
> > >
> > >
> > > Thank you for your quick response.
> > >
> > >
> > > Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
> > >
> > > > I have studied the rcu torture test recently. I am also interested in
> > > > this topic.
> > > > But I can't open
> > > > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
> > >
> > > Sorry, about that. I should have checked those. I had put them into a
> > > directory:
> > >
> > > [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
> > >
> > > I am going to try to test your suggestions at the end of the day.
> >
> > On x86 rcutorture builds successfully. However, allmodconfig
> > on semi-recent -next got me "Can't open perl script
> > "./usr/include/headers_check.pl": No such file or directory".
> > Which might well be a local problem or might well be fixed by now.
>
> Not fixed as of next-20220118. Chasing it down... ;-)
I can do allmodconfig on -next,
$git describe
next-20220118
$tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
--- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
--- Results directory: 2022.01.19-09.14.39-torture
$ ps -aux|grep qemu-system
zzy 470309 773 0.3 1876544 153936 pts/0 Sl+ 09:55 31:27
qemu-system-x86_64 -enable-kvm -nographic -smp 16 -net none -machine
q35,accel=kvm -cpu kvm64 -serial
file:/tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/console.log
-m 512 -kernel /tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/bzImage
-append debug_boot_weak_hash panic=-1 selinux=0 initcall_debug debug
console=ttyS0 rcupdate.rcu_cpu_stall_suppress_at_boot=1
torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000
rcutorture.onoff_interval=200 rcutorture.onoff_holdoff=30
rcutree.gp_preinit_delay=12 rcutree.gp_init_delay=3
rcutree.gp_cleanup_delay=3 rcutree.kthread_prio=2 threadirqs
tree.use_softirq=0 rcutorture.n_barrier_cbs=4
rcutorture.stat_interval=15 rcutorture.shutdown_secs=420
rcutorture.test_no_idle_hz=1 rcutorture.verbose=1
zzy 755865 0.0 0.0 17676 2876 pts/2 S+ 09:59 0:00
grep --color=auto qemu-system
$ ls -l vmlinux
-rwxrwxr-x 1 zzy zzy 69349872 1月 19 09:55 vmlinux

Could you please try the following command ?
linux-next$ perl ./usr/include/headers_check.pl usr/include x86
usr/include/rdma/hfi/hfi1_user.h
linux-next$ echo $?
0
The headers_check.pl in linux-next
(https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/usr/include/headers_check.pl)
is used to check the validity of head files in ./usr/include
>
> > Either way, it looks like I need to upgrade the torture.sh script's
> > checks for failed builds. Thank you for reporting this!
>
> Does this make torture.sh more reliably report build failures?
I studied this commit line by line several times and benefited a lot. Thank you!
>
> Thanx, Paul
>
> ------------------------------------------------------------------------
>
> commit 0d302830515307ceb58e89d5fb91e81b6d22e0bf
> Author: Paul E. McKenney <[email protected]>
> Date: Tue Jan 18 15:40:49 2022 -0800
>
> torture: Make kvm-find-errors.sh notice missing vmlinux file
>
> Currently, an obtuse compiler diagnostic can fool kvm-find-errors.sh
> into believing that the build was successful. This commit therefore
> adds a check for a missing vmlinux file.
>
> Link: https://lore.kernel.org/lkml/[email protected]/
> Reported-by: Paul Menzel <[email protected]>
> Signed-off-by: Paul E. McKenney <[email protected]>
>
> diff --git a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> index 2e9e9e2eedb69..7d3e11a6b8290 100755
> --- a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> +++ b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> @@ -30,10 +30,15 @@ editor=${EDITOR-vi}
> files=
> for i in ${rundir}/*/Make.out
> do
> + scenariodir="`dirname $i`"
> if egrep -q "error:|warning:|^ld: .*undefined reference to" < $i
> then
> egrep "error:|warning:|^ld: .*undefined reference to" < $i > $i.diags
> files="$files $i.diags $i"
> + elif ! test -f ${scenariodir}/vmlinux
> + then
> + echo No ${scenariodir}/vmlinux file > $i.diags
> + files="$files $i.diags $i"
> fi
> done
> if test -n "$files"
Thanks
Zhouyi

2022-01-21 16:32:44

by Paul E. McKenney

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

On Wed, Jan 19, 2022 at 10:07:42AM +0800, Zhouyi Zhou wrote:
> Thanks Paul for looking into this
>
> On Wed, Jan 19, 2022 at 7:46 AM Paul E. McKenney <[email protected]> wrote:
> >
> > On Tue, Jan 18, 2022 at 09:29:04AM -0800, Paul E. McKenney wrote:
> > > On Tue, Jan 18, 2022 at 08:56:24AM +0100, Paul Menzel wrote:
> > > > Dear Zhouyi,
> > > >
> > > >
> > > > Thank you for your quick response.
> > > >
> > > >
> > > > Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
> > > >
> > > > > I have studied the rcu torture test recently. I am also interested in
> > > > > this topic.
> > > > > But I can't open
> > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
> > > >
> > > > Sorry, about that. I should have checked those. I had put them into a
> > > > directory:
> > > >
> > > > [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
> > > >
> > > > I am going to try to test your suggestions at the end of the day.
> > >
> > > On x86 rcutorture builds successfully. However, allmodconfig
> > > on semi-recent -next got me "Can't open perl script
> > > "./usr/include/headers_check.pl": No such file or directory".
> > > Which might well be a local problem or might well be fixed by now.
> >
> > Not fixed as of next-20220118. Chasing it down... ;-)
> I can do allmodconfig on -next,
> $git describe
> next-20220118
> $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> --- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> --- Results directory: 2022.01.19-09.14.39-torture
> $ ps -aux|grep qemu-system
> zzy 470309 773 0.3 1876544 153936 pts/0 Sl+ 09:55 31:27
> qemu-system-x86_64 -enable-kvm -nographic -smp 16 -net none -machine
> q35,accel=kvm -cpu kvm64 -serial
> file:/tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/console.log
> -m 512 -kernel /tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/bzImage
> -append debug_boot_weak_hash panic=-1 selinux=0 initcall_debug debug
> console=ttyS0 rcupdate.rcu_cpu_stall_suppress_at_boot=1
> torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000
> rcutorture.onoff_interval=200 rcutorture.onoff_holdoff=30
> rcutree.gp_preinit_delay=12 rcutree.gp_init_delay=3
> rcutree.gp_cleanup_delay=3 rcutree.kthread_prio=2 threadirqs
> tree.use_softirq=0 rcutorture.n_barrier_cbs=4
> rcutorture.stat_interval=15 rcutorture.shutdown_secs=420
> rcutorture.test_no_idle_hz=1 rcutorture.verbose=1
> zzy 755865 0.0 0.0 17676 2876 pts/2 S+ 09:59 0:00
> grep --color=auto qemu-system
> $ ls -l vmlinux
> -rwxrwxr-x 1 zzy zzy 69349872 1月 19 09:55 vmlinux
>
> Could you please try the following command ?
> linux-next$ perl ./usr/include/headers_check.pl usr/include x86
> usr/include/rdma/hfi/hfi1_user.h
> linux-next$ echo $?
> 0
> The headers_check.pl in linux-next
> (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/usr/include/headers_check.pl)
> is used to check the validity of head files in ./usr/include

I am currently bisecting, but once that finishes I will give this a
try, thank you!

Thanx, Paul

> > > Either way, it looks like I need to upgrade the torture.sh script's
> > > checks for failed builds. Thank you for reporting this!
> >
> > Does this make torture.sh more reliably report build failures?
> I studied this commit line by line several times and benefited a lot. Thank you!
> >
> > Thanx, Paul
> >
> > ------------------------------------------------------------------------
> >
> > commit 0d302830515307ceb58e89d5fb91e81b6d22e0bf
> > Author: Paul E. McKenney <[email protected]>
> > Date: Tue Jan 18 15:40:49 2022 -0800
> >
> > torture: Make kvm-find-errors.sh notice missing vmlinux file
> >
> > Currently, an obtuse compiler diagnostic can fool kvm-find-errors.sh
> > into believing that the build was successful. This commit therefore
> > adds a check for a missing vmlinux file.
> >
> > Link: https://lore.kernel.org/lkml/[email protected]/
> > Reported-by: Paul Menzel <[email protected]>
> > Signed-off-by: Paul E. McKenney <[email protected]>
> >
> > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > index 2e9e9e2eedb69..7d3e11a6b8290 100755
> > --- a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > +++ b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > @@ -30,10 +30,15 @@ editor=${EDITOR-vi}
> > files=
> > for i in ${rundir}/*/Make.out
> > do
> > + scenariodir="`dirname $i`"
> > if egrep -q "error:|warning:|^ld: .*undefined reference to" < $i
> > then
> > egrep "error:|warning:|^ld: .*undefined reference to" < $i > $i.diags
> > files="$files $i.diags $i"
> > + elif ! test -f ${scenariodir}/vmlinux
> > + then
> > + echo No ${scenariodir}/vmlinux file > $i.diags
> > + files="$files $i.diags $i"
> > fi
> > done
> > if test -n "$files"
> Thanks
> Zhouyi

2022-01-21 19:55:22

by Paul E. McKenney

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

On Tue, Jan 18, 2022 at 08:46:49PM -0800, Paul E. McKenney wrote:
> On Wed, Jan 19, 2022 at 10:07:42AM +0800, Zhouyi Zhou wrote:
> > Thanks Paul for looking into this
> >
> > On Wed, Jan 19, 2022 at 7:46 AM Paul E. McKenney <[email protected]> wrote:
> > >
> > > On Tue, Jan 18, 2022 at 09:29:04AM -0800, Paul E. McKenney wrote:
> > > > On Tue, Jan 18, 2022 at 08:56:24AM +0100, Paul Menzel wrote:
> > > > > Dear Zhouyi,
> > > > >
> > > > >
> > > > > Thank you for your quick response.
> > > > >
> > > > >
> > > > > Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
> > > > >
> > > > > > I have studied the rcu torture test recently. I am also interested in
> > > > > > this topic.
> > > > > > But I can't open
> > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
> > > > >
> > > > > Sorry, about that. I should have checked those. I had put them into a
> > > > > directory:
> > > > >
> > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
> > > > >
> > > > > I am going to try to test your suggestions at the end of the day.
> > > >
> > > > On x86 rcutorture builds successfully. However, allmodconfig
> > > > on semi-recent -next got me "Can't open perl script
> > > > "./usr/include/headers_check.pl": No such file or directory".
> > > > Which might well be a local problem or might well be fixed by now.
> > >
> > > Not fixed as of next-20220118. Chasing it down... ;-)
> > I can do allmodconfig on -next,
> > $git describe
> > next-20220118
> > $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > --- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > --- Results directory: 2022.01.19-09.14.39-torture
> > $ ps -aux|grep qemu-system
> > zzy 470309 773 0.3 1876544 153936 pts/0 Sl+ 09:55 31:27
> > qemu-system-x86_64 -enable-kvm -nographic -smp 16 -net none -machine
> > q35,accel=kvm -cpu kvm64 -serial
> > file:/tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/console.log
> > -m 512 -kernel /tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/bzImage
> > -append debug_boot_weak_hash panic=-1 selinux=0 initcall_debug debug
> > console=ttyS0 rcupdate.rcu_cpu_stall_suppress_at_boot=1
> > torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000
> > rcutorture.onoff_interval=200 rcutorture.onoff_holdoff=30
> > rcutree.gp_preinit_delay=12 rcutree.gp_init_delay=3
> > rcutree.gp_cleanup_delay=3 rcutree.kthread_prio=2 threadirqs
> > tree.use_softirq=0 rcutorture.n_barrier_cbs=4
> > rcutorture.stat_interval=15 rcutorture.shutdown_secs=420
> > rcutorture.test_no_idle_hz=1 rcutorture.verbose=1
> > zzy 755865 0.0 0.0 17676 2876 pts/2 S+ 09:59 0:00
> > grep --color=auto qemu-system
> > $ ls -l vmlinux
> > -rwxrwxr-x 1 zzy zzy 69349872 1月 19 09:55 vmlinux
> >
> > Could you please try the following command ?
> > linux-next$ perl ./usr/include/headers_check.pl usr/include x86
> > usr/include/rdma/hfi/hfi1_user.h
> > linux-next$ echo $?
> > 0
> > The headers_check.pl in linux-next
> > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/usr/include/headers_check.pl)
> > is used to check the validity of head files in ./usr/include
>
> I am currently bisecting, but once that finishes I will give this a
> try, thank you!

And the bisection converged badly due to there being more than one build
failure.

So I ran the command above and then the build succeeded.

Though it would be good if the build worked from "make distclean"...

Thanx, Paul

> > > > Either way, it looks like I need to upgrade the torture.sh script's
> > > > checks for failed builds. Thank you for reporting this!
> > >
> > > Does this make torture.sh more reliably report build failures?
> > I studied this commit line by line several times and benefited a lot. Thank you!
> > >
> > > Thanx, Paul
> > >
> > > ------------------------------------------------------------------------
> > >
> > > commit 0d302830515307ceb58e89d5fb91e81b6d22e0bf
> > > Author: Paul E. McKenney <[email protected]>
> > > Date: Tue Jan 18 15:40:49 2022 -0800
> > >
> > > torture: Make kvm-find-errors.sh notice missing vmlinux file
> > >
> > > Currently, an obtuse compiler diagnostic can fool kvm-find-errors.sh
> > > into believing that the build was successful. This commit therefore
> > > adds a check for a missing vmlinux file.
> > >
> > > Link: https://lore.kernel.org/lkml/[email protected]/
> > > Reported-by: Paul Menzel <[email protected]>
> > > Signed-off-by: Paul E. McKenney <[email protected]>
> > >
> > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > index 2e9e9e2eedb69..7d3e11a6b8290 100755
> > > --- a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > +++ b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > @@ -30,10 +30,15 @@ editor=${EDITOR-vi}
> > > files=
> > > for i in ${rundir}/*/Make.out
> > > do
> > > + scenariodir="`dirname $i`"
> > > if egrep -q "error:|warning:|^ld: .*undefined reference to" < $i
> > > then
> > > egrep "error:|warning:|^ld: .*undefined reference to" < $i > $i.diags
> > > files="$files $i.diags $i"
> > > + elif ! test -f ${scenariodir}/vmlinux
> > > + then
> > > + echo No ${scenariodir}/vmlinux file > $i.diags
> > > + files="$files $i.diags $i"
> > > fi
> > > done
> > > if test -n "$files"
> > Thanks
> > Zhouyi

2022-01-21 19:59:32

by Zhouyi Zhou

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

On Thu, Jan 20, 2022 at 2:21 AM Paul E. McKenney <[email protected]> wrote:
>
> On Tue, Jan 18, 2022 at 08:46:49PM -0800, Paul E. McKenney wrote:
> > On Wed, Jan 19, 2022 at 10:07:42AM +0800, Zhouyi Zhou wrote:
> > > Thanks Paul for looking into this
> > >
> > > On Wed, Jan 19, 2022 at 7:46 AM Paul E. McKenney <[email protected]> wrote:
> > > >
> > > > On Tue, Jan 18, 2022 at 09:29:04AM -0800, Paul E. McKenney wrote:
> > > > > On Tue, Jan 18, 2022 at 08:56:24AM +0100, Paul Menzel wrote:
> > > > > > Dear Zhouyi,
> > > > > >
> > > > > >
> > > > > > Thank you for your quick response.
> > > > > >
> > > > > >
> > > > > > Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
> > > > > >
> > > > > > > I have studied the rcu torture test recently. I am also interested in
> > > > > > > this topic.
> > > > > > > But I can't open
> > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
> > > > > >
> > > > > > Sorry, about that. I should have checked those. I had put them into a
> > > > > > directory:
> > > > > >
> > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
> > > > > >
> > > > > > I am going to try to test your suggestions at the end of the day.
> > > > >
> > > > > On x86 rcutorture builds successfully. However, allmodconfig
> > > > > on semi-recent -next got me "Can't open perl script
> > > > > "./usr/include/headers_check.pl": No such file or directory".
> > > > > Which might well be a local problem or might well be fixed by now.
> > > >
> > > > Not fixed as of next-20220118. Chasing it down... ;-)
> > > I can do allmodconfig on -next,
> > > $git describe
> > > next-20220118
> > > $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > --- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > --- Results directory: 2022.01.19-09.14.39-torture
> > > $ ps -aux|grep qemu-system
> > > zzy 470309 773 0.3 1876544 153936 pts/0 Sl+ 09:55 31:27
> > > qemu-system-x86_64 -enable-kvm -nographic -smp 16 -net none -machine
> > > q35,accel=kvm -cpu kvm64 -serial
> > > file:/tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/console.log
> > > -m 512 -kernel /tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/bzImage
> > > -append debug_boot_weak_hash panic=-1 selinux=0 initcall_debug debug
> > > console=ttyS0 rcupdate.rcu_cpu_stall_suppress_at_boot=1
> > > torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000
> > > rcutorture.onoff_interval=200 rcutorture.onoff_holdoff=30
> > > rcutree.gp_preinit_delay=12 rcutree.gp_init_delay=3
> > > rcutree.gp_cleanup_delay=3 rcutree.kthread_prio=2 threadirqs
> > > tree.use_softirq=0 rcutorture.n_barrier_cbs=4
> > > rcutorture.stat_interval=15 rcutorture.shutdown_secs=420
> > > rcutorture.test_no_idle_hz=1 rcutorture.verbose=1
> > > zzy 755865 0.0 0.0 17676 2876 pts/2 S+ 09:59 0:00
> > > grep --color=auto qemu-system
> > > $ ls -l vmlinux
> > > -rwxrwxr-x 1 zzy zzy 69349872 1月 19 09:55 vmlinux
> > >
> > > Could you please try the following command ?
> > > linux-next$ perl ./usr/include/headers_check.pl usr/include x86
> > > usr/include/rdma/hfi/hfi1_user.h
> > > linux-next$ echo $?
> > > 0
> > > The headers_check.pl in linux-next
> > > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/usr/include/headers_check.pl)
> > > is used to check the validity of head files in ./usr/include
> >
> > I am currently bisecting, but once that finishes I will give this a
> > try, thank you!
>
> And the bisection converged badly due to there being more than one build
> failure.
Sorry to hear the bisection is not successful.
>
> So I ran the command above and then the build succeeded.
I'm very happy that the command works.
>
> Though it would be good if the build worked from "make distclean"...
I would be very honored if I could join the efforts to make the -next better.

Many thanks
Zhouyi
>
> Thanx, Paul
>
> > > > > Either way, it looks like I need to upgrade the torture.sh script's
> > > > > checks for failed builds. Thank you for reporting this!
> > > >
> > > > Does this make torture.sh more reliably report build failures?
> > > I studied this commit line by line several times and benefited a lot. Thank you!
> > > >
> > > > Thanx, Paul
> > > >
> > > > ------------------------------------------------------------------------
> > > >
> > > > commit 0d302830515307ceb58e89d5fb91e81b6d22e0bf
> > > > Author: Paul E. McKenney <[email protected]>
> > > > Date: Tue Jan 18 15:40:49 2022 -0800
> > > >
> > > > torture: Make kvm-find-errors.sh notice missing vmlinux file
> > > >
> > > > Currently, an obtuse compiler diagnostic can fool kvm-find-errors.sh
> > > > into believing that the build was successful. This commit therefore
> > > > adds a check for a missing vmlinux file.
> > > >
> > > > Link: https://lore.kernel.org/lkml/[email protected]/
> > > > Reported-by: Paul Menzel <[email protected]>
> > > > Signed-off-by: Paul E. McKenney <[email protected]>
> > > >
> > > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > index 2e9e9e2eedb69..7d3e11a6b8290 100755
> > > > --- a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > +++ b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > @@ -30,10 +30,15 @@ editor=${EDITOR-vi}
> > > > files=
> > > > for i in ${rundir}/*/Make.out
> > > > do
> > > > + scenariodir="`dirname $i`"
> > > > if egrep -q "error:|warning:|^ld: .*undefined reference to" < $i
> > > > then
> > > > egrep "error:|warning:|^ld: .*undefined reference to" < $i > $i.diags
> > > > files="$files $i.diags $i"
> > > > + elif ! test -f ${scenariodir}/vmlinux
> > > > + then
> > > > + echo No ${scenariodir}/vmlinux file > $i.diags
> > > > + files="$files $i.diags $i"
> > > > fi
> > > > done
> > > > if test -n "$files"
> > > Thanks
> > > Zhouyi

2022-01-21 22:41:11

by Paul E. McKenney

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

On Thu, Jan 20, 2022 at 03:30:54AM +0800, Zhouyi Zhou wrote:
> On Thu, Jan 20, 2022 at 2:21 AM Paul E. McKenney <[email protected]> wrote:
> >
> > On Tue, Jan 18, 2022 at 08:46:49PM -0800, Paul E. McKenney wrote:
> > > On Wed, Jan 19, 2022 at 10:07:42AM +0800, Zhouyi Zhou wrote:
> > > > Thanks Paul for looking into this
> > > >
> > > > On Wed, Jan 19, 2022 at 7:46 AM Paul E. McKenney <[email protected]> wrote:
> > > > >
> > > > > On Tue, Jan 18, 2022 at 09:29:04AM -0800, Paul E. McKenney wrote:
> > > > > > On Tue, Jan 18, 2022 at 08:56:24AM +0100, Paul Menzel wrote:
> > > > > > > Dear Zhouyi,
> > > > > > >
> > > > > > >
> > > > > > > Thank you for your quick response.
> > > > > > >
> > > > > > >
> > > > > > > Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
> > > > > > >
> > > > > > > > I have studied the rcu torture test recently. I am also interested in
> > > > > > > > this topic.
> > > > > > > > But I can't open
> > > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
> > > > > > >
> > > > > > > Sorry, about that. I should have checked those. I had put them into a
> > > > > > > directory:
> > > > > > >
> > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
> > > > > > >
> > > > > > > I am going to try to test your suggestions at the end of the day.
> > > > > >
> > > > > > On x86 rcutorture builds successfully. However, allmodconfig
> > > > > > on semi-recent -next got me "Can't open perl script
> > > > > > "./usr/include/headers_check.pl": No such file or directory".
> > > > > > Which might well be a local problem or might well be fixed by now.
> > > > >
> > > > > Not fixed as of next-20220118. Chasing it down... ;-)
> > > > I can do allmodconfig on -next,
> > > > $git describe
> > > > next-20220118
> > > > $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > > --- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > > --- Results directory: 2022.01.19-09.14.39-torture
> > > > $ ps -aux|grep qemu-system
> > > > zzy 470309 773 0.3 1876544 153936 pts/0 Sl+ 09:55 31:27
> > > > qemu-system-x86_64 -enable-kvm -nographic -smp 16 -net none -machine
> > > > q35,accel=kvm -cpu kvm64 -serial
> > > > file:/tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/console.log
> > > > -m 512 -kernel /tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/bzImage
> > > > -append debug_boot_weak_hash panic=-1 selinux=0 initcall_debug debug
> > > > console=ttyS0 rcupdate.rcu_cpu_stall_suppress_at_boot=1
> > > > torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000
> > > > rcutorture.onoff_interval=200 rcutorture.onoff_holdoff=30
> > > > rcutree.gp_preinit_delay=12 rcutree.gp_init_delay=3
> > > > rcutree.gp_cleanup_delay=3 rcutree.kthread_prio=2 threadirqs
> > > > tree.use_softirq=0 rcutorture.n_barrier_cbs=4
> > > > rcutorture.stat_interval=15 rcutorture.shutdown_secs=420
> > > > rcutorture.test_no_idle_hz=1 rcutorture.verbose=1
> > > > zzy 755865 0.0 0.0 17676 2876 pts/2 S+ 09:59 0:00
> > > > grep --color=auto qemu-system
> > > > $ ls -l vmlinux
> > > > -rwxrwxr-x 1 zzy zzy 69349872 1月 19 09:55 vmlinux
> > > >
> > > > Could you please try the following command ?
> > > > linux-next$ perl ./usr/include/headers_check.pl usr/include x86
> > > > usr/include/rdma/hfi/hfi1_user.h
> > > > linux-next$ echo $?
> > > > 0
> > > > The headers_check.pl in linux-next
> > > > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/usr/include/headers_check.pl)
> > > > is used to check the validity of head files in ./usr/include
> > >
> > > I am currently bisecting, but once that finishes I will give this a
> > > try, thank you!
> >
> > And the bisection converged badly due to there being more than one build
> > failure.
> Sorry to hear the bisection is not successful.
> >
> > So I ran the command above and then the build succeeded.
> I'm very happy that the command works.
> >
> > Though it would be good if the build worked from "make distclean"...
> I would be very honored if I could join the efforts to make the -next better.

One approach would be to download the -next kernels as they appear and
run tests on them. Reporting bugs is good, and of course fixing them
even better.

You can clone -next from here:

git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git

Individual -next releases are tagged, for example, "next-20220118".

Me, I run torture.sh on them from time to time, depending on how much
other testing I am doing. So maybe once or twice a week. ;-)

Thanx, Paul

> Many thanks
> Zhouyi
> >
> > Thanx, Paul
> >
> > > > > > Either way, it looks like I need to upgrade the torture.sh script's
> > > > > > checks for failed builds. Thank you for reporting this!
> > > > >
> > > > > Does this make torture.sh more reliably report build failures?
> > > > I studied this commit line by line several times and benefited a lot. Thank you!
> > > > >
> > > > > Thanx, Paul
> > > > >
> > > > > ------------------------------------------------------------------------
> > > > >
> > > > > commit 0d302830515307ceb58e89d5fb91e81b6d22e0bf
> > > > > Author: Paul E. McKenney <[email protected]>
> > > > > Date: Tue Jan 18 15:40:49 2022 -0800
> > > > >
> > > > > torture: Make kvm-find-errors.sh notice missing vmlinux file
> > > > >
> > > > > Currently, an obtuse compiler diagnostic can fool kvm-find-errors.sh
> > > > > into believing that the build was successful. This commit therefore
> > > > > adds a check for a missing vmlinux file.
> > > > >
> > > > > Link: https://lore.kernel.org/lkml/[email protected]/
> > > > > Reported-by: Paul Menzel <[email protected]>
> > > > > Signed-off-by: Paul E. McKenney <[email protected]>
> > > > >
> > > > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > index 2e9e9e2eedb69..7d3e11a6b8290 100755
> > > > > --- a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > +++ b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > @@ -30,10 +30,15 @@ editor=${EDITOR-vi}
> > > > > files=
> > > > > for i in ${rundir}/*/Make.out
> > > > > do
> > > > > + scenariodir="`dirname $i`"
> > > > > if egrep -q "error:|warning:|^ld: .*undefined reference to" < $i
> > > > > then
> > > > > egrep "error:|warning:|^ld: .*undefined reference to" < $i > $i.diags
> > > > > files="$files $i.diags $i"
> > > > > + elif ! test -f ${scenariodir}/vmlinux
> > > > > + then
> > > > > + echo No ${scenariodir}/vmlinux file > $i.diags
> > > > > + files="$files $i.diags $i"
> > > > > fi
> > > > > done
> > > > > if test -n "$files"
> > > > Thanks
> > > > Zhouyi

2022-01-22 00:28:01

by Zhouyi Zhou

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

Thank Paul for your guidance!

On Fri, Jan 21, 2022 at 5:06 AM Paul E. McKenney <[email protected]> wrote:
>
> On Thu, Jan 20, 2022 at 03:30:54AM +0800, Zhouyi Zhou wrote:
> > On Thu, Jan 20, 2022 at 2:21 AM Paul E. McKenney <[email protected]> wrote:
> > >
> > > On Tue, Jan 18, 2022 at 08:46:49PM -0800, Paul E. McKenney wrote:
> > > > On Wed, Jan 19, 2022 at 10:07:42AM +0800, Zhouyi Zhou wrote:
> > > > > Thanks Paul for looking into this
> > > > >
> > > > > On Wed, Jan 19, 2022 at 7:46 AM Paul E. McKenney <[email protected]> wrote:
> > > > > >
> > > > > > On Tue, Jan 18, 2022 at 09:29:04AM -0800, Paul E. McKenney wrote:
> > > > > > > On Tue, Jan 18, 2022 at 08:56:24AM +0100, Paul Menzel wrote:
> > > > > > > > Dear Zhouyi,
> > > > > > > >
> > > > > > > >
> > > > > > > > Thank you for your quick response.
> > > > > > > >
> > > > > > > >
> > > > > > > > Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
> > > > > > > >
> > > > > > > > > I have studied the rcu torture test recently. I am also interested in
> > > > > > > > > this topic.
> > > > > > > > > But I can't open
> > > > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > > > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
> > > > > > > >
> > > > > > > > Sorry, about that. I should have checked those. I had put them into a
> > > > > > > > directory:
> > > > > > > >
> > > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> > > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
> > > > > > > >
> > > > > > > > I am going to try to test your suggestions at the end of the day.
> > > > > > >
> > > > > > > On x86 rcutorture builds successfully. However, allmodconfig
> > > > > > > on semi-recent -next got me "Can't open perl script
> > > > > > > "./usr/include/headers_check.pl": No such file or directory".
> > > > > > > Which might well be a local problem or might well be fixed by now.
> > > > > >
> > > > > > Not fixed as of next-20220118. Chasing it down... ;-)
> > > > > I can do allmodconfig on -next,
> > > > > $git describe
> > > > > next-20220118
> > > > > $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > > > --- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > > > --- Results directory: 2022.01.19-09.14.39-torture
> > > > > $ ps -aux|grep qemu-system
> > > > > zzy 470309 773 0.3 1876544 153936 pts/0 Sl+ 09:55 31:27
> > > > > qemu-system-x86_64 -enable-kvm -nographic -smp 16 -net none -machine
> > > > > q35,accel=kvm -cpu kvm64 -serial
> > > > > file:/tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/console.log
> > > > > -m 512 -kernel /tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/bzImage
> > > > > -append debug_boot_weak_hash panic=-1 selinux=0 initcall_debug debug
> > > > > console=ttyS0 rcupdate.rcu_cpu_stall_suppress_at_boot=1
> > > > > torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000
> > > > > rcutorture.onoff_interval=200 rcutorture.onoff_holdoff=30
> > > > > rcutree.gp_preinit_delay=12 rcutree.gp_init_delay=3
> > > > > rcutree.gp_cleanup_delay=3 rcutree.kthread_prio=2 threadirqs
> > > > > tree.use_softirq=0 rcutorture.n_barrier_cbs=4
> > > > > rcutorture.stat_interval=15 rcutorture.shutdown_secs=420
> > > > > rcutorture.test_no_idle_hz=1 rcutorture.verbose=1
> > > > > zzy 755865 0.0 0.0 17676 2876 pts/2 S+ 09:59 0:00
> > > > > grep --color=auto qemu-system
> > > > > $ ls -l vmlinux
> > > > > -rwxrwxr-x 1 zzy zzy 69349872 1月 19 09:55 vmlinux
> > > > >
> > > > > Could you please try the following command ?
> > > > > linux-next$ perl ./usr/include/headers_check.pl usr/include x86
> > > > > usr/include/rdma/hfi/hfi1_user.h
> > > > > linux-next$ echo $?
> > > > > 0
> > > > > The headers_check.pl in linux-next
> > > > > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/usr/include/headers_check.pl)
> > > > > is used to check the validity of head files in ./usr/include
> > > >
> > > > I am currently bisecting, but once that finishes I will give this a
> > > > try, thank you!
> > >
> > > And the bisection converged badly due to there being more than one build
> > > failure.
> > Sorry to hear the bisection is not successful.
> > >
> > > So I ran the command above and then the build succeeded.
> > I'm very happy that the command works.
> > >
> > > Though it would be good if the build worked from "make distclean"...
> > I would be very honored if I could join the efforts to make the -next better.
>
> One approach would be to download the -next kernels as they appear and
> run tests on them. Reporting bugs is good, and of course fixing them
> even better.
This provides a nice way of participating the efforts to make -next
better, and provides me a good way to learn new knowledge in -next!

>
> You can clone -next from here:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
>
> Individual -next releases are tagged, for example, "next-20220118".
I will clone -next from the tree, and will download the -next releases from
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
>
> Me, I run torture.sh on them from time to time, depending on how much
> other testing I am doing. So maybe once or twice a week. ;-)
You are the example worth learning in my life, to be honest , I will
do the same way as you do!

Sincerely
Zhouyi
>
> Thanx, Paul
>
> > Many thanks
> > Zhouyi
> > >
> > > Thanx, Paul
> > >
> > > > > > > Either way, it looks like I need to upgrade the torture.sh script's
> > > > > > > checks for failed builds. Thank you for reporting this!
> > > > > >
> > > > > > Does this make torture.sh more reliably report build failures?
> > > > > I studied this commit line by line several times and benefited a lot. Thank you!
> > > > > >
> > > > > > Thanx, Paul
> > > > > >
> > > > > > ------------------------------------------------------------------------
> > > > > >
> > > > > > commit 0d302830515307ceb58e89d5fb91e81b6d22e0bf
> > > > > > Author: Paul E. McKenney <[email protected]>
> > > > > > Date: Tue Jan 18 15:40:49 2022 -0800
> > > > > >
> > > > > > torture: Make kvm-find-errors.sh notice missing vmlinux file
> > > > > >
> > > > > > Currently, an obtuse compiler diagnostic can fool kvm-find-errors.sh
> > > > > > into believing that the build was successful. This commit therefore
> > > > > > adds a check for a missing vmlinux file.
> > > > > >
> > > > > > Link: https://lore.kernel.org/lkml/[email protected]/
> > > > > > Reported-by: Paul Menzel <[email protected]>
> > > > > > Signed-off-by: Paul E. McKenney <[email protected]>
> > > > > >
> > > > > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > index 2e9e9e2eedb69..7d3e11a6b8290 100755
> > > > > > --- a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > +++ b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > @@ -30,10 +30,15 @@ editor=${EDITOR-vi}
> > > > > > files=
> > > > > > for i in ${rundir}/*/Make.out
> > > > > > do
> > > > > > + scenariodir="`dirname $i`"
> > > > > > if egrep -q "error:|warning:|^ld: .*undefined reference to" < $i
> > > > > > then
> > > > > > egrep "error:|warning:|^ld: .*undefined reference to" < $i > $i.diags
> > > > > > files="$files $i.diags $i"
> > > > > > + elif ! test -f ${scenariodir}/vmlinux
> > > > > > + then
> > > > > > + echo No ${scenariodir}/vmlinux file > $i.diags
> > > > > > + files="$files $i.diags $i"
> > > > > > fi
> > > > > > done
> > > > > > if test -n "$files"
> > > > > Thanks
> > > > > Zhouyi

2022-01-25 23:02:36

by Zhouyi Zhou

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

Hi Paul

On Fri, Jan 21, 2022 at 8:44 AM Zhouyi Zhou <[email protected]> wrote:
>
> Thank Paul for your guidance!
>
> On Fri, Jan 21, 2022 at 5:06 AM Paul E. McKenney <[email protected]> wrote:
> >
> > On Thu, Jan 20, 2022 at 03:30:54AM +0800, Zhouyi Zhou wrote:
> > > On Thu, Jan 20, 2022 at 2:21 AM Paul E. McKenney <[email protected]> wrote:
> > > >
> > > > On Tue, Jan 18, 2022 at 08:46:49PM -0800, Paul E. McKenney wrote:
> > > > > On Wed, Jan 19, 2022 at 10:07:42AM +0800, Zhouyi Zhou wrote:
> > > > > > Thanks Paul for looking into this
> > > > > >
> > > > > > On Wed, Jan 19, 2022 at 7:46 AM Paul E. McKenney <[email protected]> wrote:
> > > > > > >
> > > > > > > On Tue, Jan 18, 2022 at 09:29:04AM -0800, Paul E. McKenney wrote:
> > > > > > > > On Tue, Jan 18, 2022 at 08:56:24AM +0100, Paul Menzel wrote:
> > > > > > > > > Dear Zhouyi,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thank you for your quick response.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
> > > > > > > > >
> > > > > > > > > > I have studied the rcu torture test recently. I am also interested in
> > > > > > > > > > this topic.
> > > > > > > > > > But I can't open
> > > > > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > > > > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
> > > > > > > > >
> > > > > > > > > Sorry, about that. I should have checked those. I had put them into a
> > > > > > > > > directory:
> > > > > > > > >
> > > > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> > > > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
> > > > > > > > >
> > > > > > > > > I am going to try to test your suggestions at the end of the day.
> > > > > > > >
> > > > > > > > On x86 rcutorture builds successfully. However, allmodconfig
> > > > > > > > on semi-recent -next got me "Can't open perl script
> > > > > > > > "./usr/include/headers_check.pl": No such file or directory".
> > > > > > > > Which might well be a local problem or might well be fixed by now.
> > > > > > >
> > > > > > > Not fixed as of next-20220118. Chasing it down... ;-)
> > > > > > I can do allmodconfig on -next,
> > > > > > $git describe
> > > > > > next-20220118
> > > > > > $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > > > > --- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > > > > --- Results directory: 2022.01.19-09.14.39-torture
> > > > > > $ ps -aux|grep qemu-system
> > > > > > zzy 470309 773 0.3 1876544 153936 pts/0 Sl+ 09:55 31:27
> > > > > > qemu-system-x86_64 -enable-kvm -nographic -smp 16 -net none -machine
> > > > > > q35,accel=kvm -cpu kvm64 -serial
> > > > > > file:/tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/console.log
> > > > > > -m 512 -kernel /tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/bzImage
> > > > > > -append debug_boot_weak_hash panic=-1 selinux=0 initcall_debug debug
> > > > > > console=ttyS0 rcupdate.rcu_cpu_stall_suppress_at_boot=1
> > > > > > torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000
> > > > > > rcutorture.onoff_interval=200 rcutorture.onoff_holdoff=30
> > > > > > rcutree.gp_preinit_delay=12 rcutree.gp_init_delay=3
> > > > > > rcutree.gp_cleanup_delay=3 rcutree.kthread_prio=2 threadirqs
> > > > > > tree.use_softirq=0 rcutorture.n_barrier_cbs=4
> > > > > > rcutorture.stat_interval=15 rcutorture.shutdown_secs=420
> > > > > > rcutorture.test_no_idle_hz=1 rcutorture.verbose=1
> > > > > > zzy 755865 0.0 0.0 17676 2876 pts/2 S+ 09:59 0:00
> > > > > > grep --color=auto qemu-system
> > > > > > $ ls -l vmlinux
> > > > > > -rwxrwxr-x 1 zzy zzy 69349872 1月 19 09:55 vmlinux
> > > > > >
> > > > > > Could you please try the following command ?
> > > > > > linux-next$ perl ./usr/include/headers_check.pl usr/include x86
> > > > > > usr/include/rdma/hfi/hfi1_user.h
> > > > > > linux-next$ echo $?
> > > > > > 0
> > > > > > The headers_check.pl in linux-next
> > > > > > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/usr/include/headers_check.pl)
> > > > > > is used to check the validity of head files in ./usr/include
> > > > >
> > > > > I am currently bisecting, but once that finishes I will give this a
> > > > > try, thank you!
> > > >
> > > > And the bisection converged badly due to there being more than one build
> > > > failure.
> > > Sorry to hear the bisection is not successful.
> > > >
> > > > So I ran the command above and then the build succeeded.
> > > I'm very happy that the command works.
> > > >
> > > > Though it would be good if the build worked from "make distclean"...
> > > I would be very honored if I could join the efforts to make the -next better.linux-next/tools/testing/selftests/rcutorture/res/2022.01.25-23.10.36-torture/allmodconfig
> >
> > One approach would be to download the -next kernels as they appear and
> > run tests on them. Reporting bugs is good, and of course fixing them
> > even better.
> This provides a nice way of participating the efforts to make -next
> better, and provides me a good way to learn new knowledge in -next!
>
> >
> > You can clone -next from here:
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> >
> > Individual -next releases are tagged, for example, "next-20220118".
> I will clone -next from the tree, and will download the -next releases from
> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
I cloned -next, and try to test rcu:
$git clone https://kernel.source.codeaurora.cn/pub/scm/linux/kernel/git/next/linux-next.git
(kernel.source.codeaurora.cn is Chinese mirror of git.kernel.org)
$cd linux-next
$git describe
next-20220125
$tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
$cd linux-next/tools/testing/selftests/rcutorture/res/2022.01.25-23.10.36-torture/allmodconfig
$cat Make.exitcode
2
$grep Stop Make.out
make[2]: *** No rule to make target
'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',
needed by 'arch/x86/kvm/kvm.o'. Stop.
I tried the above two times in two different machines today, they all failed.


Meanwhile I can config and build the kernel successfully by invoking
the build commands by hand:
$git clone https://kernel.source.codeaurora.cn/pub/scm/linux/kernel/git/next/linux-next.git
$cd linux-next
$git describe
next-20220125
$make allmodconfig
$make -j 16
Kernel: arch/x86/boot/bzImage is ready (#2)
GEN Module.symvers
$echo $?
0

I am very interested in this problem and I am eager to try to dig out
what happened and fix it before this Sunday. And it is more delightful
to me that other developers can solve this problem because they are
more mature than me ;-)

Many thanks
Zhouyi
> >
> > Me, I run torture.sh on them from time to time, depending on how much
> > other testing I am doing. So maybe once or twice a week. ;-)
> You are the example worth learning in my life, to be honest , I will
> do the same way as you do!
>
> Sincerely
> Zhouyi
> >
> > Thanx, Paul
> >
> > > Many thanks
> > > Zhouyi
> > > >
> > > > Thanx, Paul
> > > >
> > > > > > > > Either way, it looks like I need to upgrade the torture.sh script's
> > > > > > > > checks for failed builds. Thank you for reporting this!
> > > > > > >
> > > > > > > Does this make torture.sh more reliably report build failures?
> > > > > > I studied this commit line by line several times and benefited a lot. Thank you!
> > > > > > >
> > > > > > > Thanx, Paul
> > > > > > >
> > > > > > > ------------------------------------------------------------------------
> > > > > > >
> > > > > > > commit 0d302830515307ceb58e89d5fb91e81b6d22e0bf
> > > > > > > Author: Paul E. McKenney <[email protected]>
> > > > > > > Date: Tue Jan 18 15:40:49 2022 -0800
> > > > > > >
> > > > > > > torture: Make kvm-find-errors.sh notice missing vmlinux file
> > > > > > >
> > > > > > > Currently, an obtuse compiler diagnostic can fool kvm-find-errors.sh
> > > > > > > into believing that the build was successful. This commit therefore
> > > > > > > adds a check for a missing vmlinux file.
> > > > > > >
> > > > > > > Link: https://lore.kernel.org/lkml/[email protected]/
> > > > > > > Reported-by: Paul Menzel <[email protected]>
> > > > > > > Signed-off-by: Paul E. McKenney <[email protected]>
> > > > > > >
> > > > > > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > > index 2e9e9e2eedb69..7d3e11a6b8290 100755
> > > > > > > --- a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > > +++ b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > > @@ -30,10 +30,15 @@ editor=${EDITOR-vi}
> > > > > > > files=
> > > > > > > for i in ${rundir}/*/Make.out
> > > > > > > do
> > > > > > > + scenariodir="`dirname $i`"
> > > > > > > if egrep -q "error:|warning:|^ld: .*undefined reference to" < $i
> > > > > > > then
> > > > > > > egrep "error:|warning:|^ld: .*undefined reference to" < $i > $i.diags
> > > > > > > files="$files $i.diags $i"
> > > > > > > + elif ! test -f ${scenariodir}/vmlinux
> > > > > > > + then
> > > > > > > + echo No ${scenariodir}/vmlinux file > $i.diags
> > > > > > > + files="$files $i.diags $i"
> > > > > > > fi
> > > > > > > done
> > > > > > > if test -n "$files"
> > > > > > Thanks
> > > > > > Zhouyi

2022-01-26 03:18:00

by Paul E. McKenney

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

On Wed, Jan 26, 2022 at 12:19:58AM +0800, Zhouyi Zhou wrote:
> Hi Paul
>
> On Fri, Jan 21, 2022 at 8:44 AM Zhouyi Zhou <[email protected]> wrote:
> >
> > Thank Paul for your guidance!
> >
> > On Fri, Jan 21, 2022 at 5:06 AM Paul E. McKenney <[email protected]> wrote:
> > >
> > > On Thu, Jan 20, 2022 at 03:30:54AM +0800, Zhouyi Zhou wrote:
> > > > On Thu, Jan 20, 2022 at 2:21 AM Paul E. McKenney <[email protected]> wrote:
> > > > >
> > > > > On Tue, Jan 18, 2022 at 08:46:49PM -0800, Paul E. McKenney wrote:
> > > > > > On Wed, Jan 19, 2022 at 10:07:42AM +0800, Zhouyi Zhou wrote:
> > > > > > > Thanks Paul for looking into this
> > > > > > >
> > > > > > > On Wed, Jan 19, 2022 at 7:46 AM Paul E. McKenney <[email protected]> wrote:
> > > > > > > >
> > > > > > > > On Tue, Jan 18, 2022 at 09:29:04AM -0800, Paul E. McKenney wrote:
> > > > > > > > > On Tue, Jan 18, 2022 at 08:56:24AM +0100, Paul Menzel wrote:
> > > > > > > > > > Dear Zhouyi,
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Thank you for your quick response.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
> > > > > > > > > >
> > > > > > > > > > > I have studied the rcu torture test recently. I am also interested in
> > > > > > > > > > > this topic.
> > > > > > > > > > > But I can't open
> > > > > > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > > > > > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
> > > > > > > > > >
> > > > > > > > > > Sorry, about that. I should have checked those. I had put them into a
> > > > > > > > > > directory:
> > > > > > > > > >
> > > > > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> > > > > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
> > > > > > > > > >
> > > > > > > > > > I am going to try to test your suggestions at the end of the day.
> > > > > > > > >
> > > > > > > > > On x86 rcutorture builds successfully. However, allmodconfig
> > > > > > > > > on semi-recent -next got me "Can't open perl script
> > > > > > > > > "./usr/include/headers_check.pl": No such file or directory".
> > > > > > > > > Which might well be a local problem or might well be fixed by now.
> > > > > > > >
> > > > > > > > Not fixed as of next-20220118. Chasing it down... ;-)
> > > > > > > I can do allmodconfig on -next,
> > > > > > > $git describe
> > > > > > > next-20220118
> > > > > > > $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > > > > > --- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > > > > > --- Results directory: 2022.01.19-09.14.39-torture
> > > > > > > $ ps -aux|grep qemu-system
> > > > > > > zzy 470309 773 0.3 1876544 153936 pts/0 Sl+ 09:55 31:27
> > > > > > > qemu-system-x86_64 -enable-kvm -nographic -smp 16 -net none -machine
> > > > > > > q35,accel=kvm -cpu kvm64 -serial
> > > > > > > file:/tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/console.log
> > > > > > > -m 512 -kernel /tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/bzImage
> > > > > > > -append debug_boot_weak_hash panic=-1 selinux=0 initcall_debug debug
> > > > > > > console=ttyS0 rcupdate.rcu_cpu_stall_suppress_at_boot=1
> > > > > > > torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000
> > > > > > > rcutorture.onoff_interval=200 rcutorture.onoff_holdoff=30
> > > > > > > rcutree.gp_preinit_delay=12 rcutree.gp_init_delay=3
> > > > > > > rcutree.gp_cleanup_delay=3 rcutree.kthread_prio=2 threadirqs
> > > > > > > tree.use_softirq=0 rcutorture.n_barrier_cbs=4
> > > > > > > rcutorture.stat_interval=15 rcutorture.shutdown_secs=420
> > > > > > > rcutorture.test_no_idle_hz=1 rcutorture.verbose=1
> > > > > > > zzy 755865 0.0 0.0 17676 2876 pts/2 S+ 09:59 0:00
> > > > > > > grep --color=auto qemu-system
> > > > > > > $ ls -l vmlinux
> > > > > > > -rwxrwxr-x 1 zzy zzy 69349872 1月 19 09:55 vmlinux
> > > > > > >
> > > > > > > Could you please try the following command ?
> > > > > > > linux-next$ perl ./usr/include/headers_check.pl usr/include x86
> > > > > > > usr/include/rdma/hfi/hfi1_user.h
> > > > > > > linux-next$ echo $?
> > > > > > > 0
> > > > > > > The headers_check.pl in linux-next
> > > > > > > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/usr/include/headers_check.pl)
> > > > > > > is used to check the validity of head files in ./usr/include
> > > > > >
> > > > > > I am currently bisecting, but once that finishes I will give this a
> > > > > > try, thank you!
> > > > >
> > > > > And the bisection converged badly due to there being more than one build
> > > > > failure.
> > > > Sorry to hear the bisection is not successful.
> > > > >
> > > > > So I ran the command above and then the build succeeded.
> > > > I'm very happy that the command works.
> > > > >
> > > > > Though it would be good if the build worked from "make distclean"...
> > > > I would be very honored if I could join the efforts to make the -next better.linux-next/tools/testing/selftests/rcutorture/res/2022.01.25-23.10.36-torture/allmodconfig
> > >
> > > One approach would be to download the -next kernels as they appear and
> > > run tests on them. Reporting bugs is good, and of course fixing them
> > > even better.
> > This provides a nice way of participating the efforts to make -next
> > better, and provides me a good way to learn new knowledge in -next!
> >
> > >
> > > You can clone -next from here:
> > >
> > > git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> > >
> > > Individual -next releases are tagged, for example, "next-20220118".
> > I will clone -next from the tree, and will download the -next releases from
> > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> I cloned -next, and try to test rcu:
> $git clone https://kernel.source.codeaurora.cn/pub/scm/linux/kernel/git/next/linux-next.git
> (kernel.source.codeaurora.cn is Chinese mirror of git.kernel.org)
> $cd linux-next
> $git describe
> next-20220125
> $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> $cd linux-next/tools/testing/selftests/rcutorture/res/2022.01.25-23.10.36-torture/allmodconfig
> $cat Make.exitcode
> 2
> $grep Stop Make.out
> make[2]: *** No rule to make target
> 'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',
> needed by 'arch/x86/kvm/kvm.o'. Stop.
> I tried the above two times in two different machines today, they all failed.
>
>
> Meanwhile I can config and build the kernel successfully by invoking
> the build commands by hand:
> $git clone https://kernel.source.codeaurora.cn/pub/scm/linux/kernel/git/next/linux-next.git
> $cd linux-next
> $git describe
> next-20220125

Huh. What happens if you do a "make clean" betweentimes?

Alternatively, what happens if you change these two lines from torture.sh:

echo " --- make clean" > "$amcdir/Make.out" 2>&1
make -j$MAKE_ALLOTED_CPUS clean >> "$amcdir/Make.out" 2>&1

To this:

echo " --- make distclean" > "$amcdir/Make.out" 2>&1
make -j$MAKE_ALLOTED_CPUS distclean >> "$amcdir/Make.out" 2>&1

And then run torture.sh in the same way that got you the error?

Thanx, Paul

> $make allmodconfig
> $make -j 16
> Kernel: arch/x86/boot/bzImage is ready (#2)
> GEN Module.symvers
> $echo $?
> 0
>
> I am very interested in this problem and I am eager to try to dig out
> what happened and fix it before this Sunday. And it is more delightful
> to me that other developers can solve this problem because they are
> more mature than me ;-)
>
> Many thanks
> Zhouyi
> > >
> > > Me, I run torture.sh on them from time to time, depending on how much
> > > other testing I am doing. So maybe once or twice a week. ;-)
> > You are the example worth learning in my life, to be honest , I will
> > do the same way as you do!
> >
> > Sincerely
> > Zhouyi
> > >
> > > Thanx, Paul
> > >
> > > > Many thanks
> > > > Zhouyi
> > > > >
> > > > > Thanx, Paul
> > > > >
> > > > > > > > > Either way, it looks like I need to upgrade the torture.sh script's
> > > > > > > > > checks for failed builds. Thank you for reporting this!
> > > > > > > >
> > > > > > > > Does this make torture.sh more reliably report build failures?
> > > > > > > I studied this commit line by line several times and benefited a lot. Thank you!
> > > > > > > >
> > > > > > > > Thanx, Paul
> > > > > > > >
> > > > > > > > ------------------------------------------------------------------------
> > > > > > > >
> > > > > > > > commit 0d302830515307ceb58e89d5fb91e81b6d22e0bf
> > > > > > > > Author: Paul E. McKenney <[email protected]>
> > > > > > > > Date: Tue Jan 18 15:40:49 2022 -0800
> > > > > > > >
> > > > > > > > torture: Make kvm-find-errors.sh notice missing vmlinux file
> > > > > > > >
> > > > > > > > Currently, an obtuse compiler diagnostic can fool kvm-find-errors.sh
> > > > > > > > into believing that the build was successful. This commit therefore
> > > > > > > > adds a check for a missing vmlinux file.
> > > > > > > >
> > > > > > > > Link: https://lore.kernel.org/lkml/[email protected]/
> > > > > > > > Reported-by: Paul Menzel <[email protected]>
> > > > > > > > Signed-off-by: Paul E. McKenney <[email protected]>
> > > > > > > >
> > > > > > > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > > > index 2e9e9e2eedb69..7d3e11a6b8290 100755
> > > > > > > > --- a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > > > +++ b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > > > @@ -30,10 +30,15 @@ editor=${EDITOR-vi}
> > > > > > > > files=
> > > > > > > > for i in ${rundir}/*/Make.out
> > > > > > > > do
> > > > > > > > + scenariodir="`dirname $i`"
> > > > > > > > if egrep -q "error:|warning:|^ld: .*undefined reference to" < $i
> > > > > > > > then
> > > > > > > > egrep "error:|warning:|^ld: .*undefined reference to" < $i > $i.diags
> > > > > > > > files="$files $i.diags $i"
> > > > > > > > + elif ! test -f ${scenariodir}/vmlinux
> > > > > > > > + then
> > > > > > > > + echo No ${scenariodir}/vmlinux file > $i.diags
> > > > > > > > + files="$files $i.diags $i"
> > > > > > > > fi
> > > > > > > > done
> > > > > > > > if test -n "$files"
> > > > > > > Thanks
> > > > > > > Zhouyi

2022-01-26 15:22:04

by Zhouyi Zhou

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

Thank Paul for your attention

On Wed, Jan 26, 2022 at 2:09 AM Paul E. McKenney <[email protected]> wrote:
>
> On Wed, Jan 26, 2022 at 12:19:58AM +0800, Zhouyi Zhou wrote:
> > Hi Paul
> >
> > On Fri, Jan 21, 2022 at 8:44 AM Zhouyi Zhou <[email protected]> wrote:
> > >
> > > Thank Paul for your guidance!
> > >
> > > On Fri, Jan 21, 2022 at 5:06 AM Paul E. McKenney <[email protected]> wrote:
> > > >
> > > > On Thu, Jan 20, 2022 at 03:30:54AM +0800, Zhouyi Zhou wrote:
> > > > > On Thu, Jan 20, 2022 at 2:21 AM Paul E. McKenney <[email protected]> wrote:
> > > > > >
> > > > > > On Tue, Jan 18, 2022 at 08:46:49PM -0800, Paul E. McKenney wrote:
> > > > > > > On Wed, Jan 19, 2022 at 10:07:42AM +0800, Zhouyi Zhou wrote:
> > > > > > > > Thanks Paul for looking into this
> > > > > > > >
> > > > > > > > On Wed, Jan 19, 2022 at 7:46 AM Paul E. McKenney <[email protected]> wrote:
> > > > > > > > >
> > > > > > > > > On Tue, Jan 18, 2022 at 09:29:04AM -0800, Paul E. McKenney wrote:
> > > > > > > > > > On Tue, Jan 18, 2022 at 08:56:24AM +0100, Paul Menzel wrote:
> > > > > > > > > > > Dear Zhouyi,
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Thank you for your quick response.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
> > > > > > > > > > >
> > > > > > > > > > > > I have studied the rcu torture test recently. I am also interested in
> > > > > > > > > > > > this topic.
> > > > > > > > > > > > But I can't open
> > > > > > > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > > > > > > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
> > > > > > > > > > >
> > > > > > > > > > > Sorry, about that. I should have checked those. I had put them into a
> > > > > > > > > > > directory:
> > > > > > > > > > >
> > > > > > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> > > > > > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
> > > > > > > > > > >
> > > > > > > > > > > I am going to try to test your suggestions at the end of the day.
> > > > > > > > > >
> > > > > > > > > > On x86 rcutorture builds successfully. However, allmodconfig
> > > > > > > > > > on semi-recent -next got me "Can't open perl script
> > > > > > > > > > "./usr/include/headers_check.pl": No such file or directory".
> > > > > > > > > > Which might well be a local problem or might well be fixed by now.
> > > > > > > > >
> > > > > > > > > Not fixed as of next-20220118. Chasing it down... ;-)
> > > > > > > > I can do allmodconfig on -next,
> > > > > > > > $git describe
> > > > > > > > next-20220118
> > > > > > > > $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > > > > > > --- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > > > > > > --- Results directory: 2022.01.19-09.14.39-torture
> > > > > > > > $ ps -aux|grep qemu-system
> > > > > > > > zzy 470309 773 0.3 1876544 153936 pts/0 Sl+ 09:55 31:27
> > > > > > > > qemu-system-x86_64 -enable-kvm -nographic -smp 16 -net none -machine
> > > > > > > > q35,accel=kvm -cpu kvm64 -serial
> > > > > > > > file:/tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/console.log
> > > > > > > > -m 512 -kernel /tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/bzImage
> > > > > > > > -append debug_boot_weak_hash panic=-1 selinux=0 initcall_debug debug
> > > > > > > > console=ttyS0 rcupdate.rcu_cpu_stall_suppress_at_boot=1
> > > > > > > > torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000
> > > > > > > > rcutorture.onoff_interval=200 rcutorture.onoff_holdoff=30
> > > > > > > > rcutree.gp_preinit_delay=12 rcutree.gp_init_delay=3
> > > > > > > > rcutree.gp_cleanup_delay=3 rcutree.kthread_prio=2 threadirqs
> > > > > > > > tree.use_softirq=0 rcutorture.n_barrier_cbs=4
> > > > > > > > rcutorture.stat_interval=15 rcutorture.shutdown_secs=420
> > > > > > > > rcutorture.test_no_idle_hz=1 rcutorture.verbose=1
> > > > > > > > zzy 755865 0.0 0.0 17676 2876 pts/2 S+ 09:59 0:00
> > > > > > > > grep --color=auto qemu-system
> > > > > > > > $ ls -l vmlinux
> > > > > > > > -rwxrwxr-x 1 zzy zzy 69349872 1月 19 09:55 vmlinux
> > > > > > > >
> > > > > > > > Could you please try the following command ?
> > > > > > > > linux-next$ perl ./usr/include/headers_check.pl usr/include x86
> > > > > > > > usr/include/rdma/hfi/hfi1_user.h
> > > > > > > > linux-next$ echo $?
> > > > > > > > 0
> > > > > > > > The headers_check.pl in linux-next
> > > > > > > > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/usr/include/headers_check.pl)
> > > > > > > > is used to check the validity of head files in ./usr/include
> > > > > > >
> > > > > > > I am currently bisecting, but once that finishes I will give this a
> > > > > > > try, thank you!
> > > > > >
> > > > > > And the bisection converged badly due to there being more than one build
> > > > > > failure.
> > > > > Sorry to hear the bisection is not successful.
> > > > > >
> > > > > > So I ran the command above and then the build succeeded.
> > > > > I'm very happy that the command works.
> > > > > >
> > > > > > Though it would be good if the build worked from "make distclean"...
> > > > > I would be very honored if I could join the efforts to make the -next better.linux-next/tools/testing/selftests/rcutorture/res/2022.01.25-23.10.36-torture/allmodconfig
> > > >
> > > > One approach would be to download the -next kernels as they appear and
> > > > run tests on them. Reporting bugs is good, and of course fixing them
> > > > even better.
> > > This provides a nice way of participating the efforts to make -next
> > > better, and provides me a good way to learn new knowledge in -next!
> > >
> > > >
> > > > You can clone -next from here:
> > > >
> > > > git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> > > >
> > > > Individual -next releases are tagged, for example, "next-20220118".
> > > I will clone -next from the tree, and will download the -next releases from
> > > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> > I cloned -next, and try to test rcu:
> > $git clone https://kernel.source.codeaurora.cn/pub/scm/linux/kernel/git/next/linux-next.git
> > (kernel.source.codeaurora.cn is Chinese mirror of git.kernel.org)
> > $cd linux-next
> > $git describe
> > next-20220125
> > $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > $cd linux-next/tools/testing/selftests/rcutorture/res/2022.01.25-23.10.36-torture/allmodconfig
> > $cat Make.exitcode
> > 2
> > $grep Stop Make.out
> > make[2]: *** No rule to make target
> > 'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',
> > needed by 'arch/x86/kvm/kvm.o'. Stop.
> > I tried the above two times in two different machines today, they all failed.
> >
> >
> > Meanwhile I can config and build the kernel successfully by invoking
> > the build commands by hand:
> > $git clone https://kernel.source.codeaurora.cn/pub/scm/linux/kernel/git/next/linux-next.git
> > $cd linux-next
> > $git describe
> > next-20220125
>
> Huh. What happens if you do a "make clean" betweentimes?
do a "make clean" between times also fails, sorry
>
> Alternatively, what happens if you change these two lines from torture.sh:
>
> echo " --- make clean" > "$amcdir/Make.out" 2>&1
> make -j$MAKE_ALLOTED_CPUS clean >> "$amcdir/Make.out" 2>&1
>
> To this:
>
> echo " --- make distclean" > "$amcdir/Make.out" 2>&1
> make -j$MAKE_ALLOTED_CPUS distclean >> "$amcdir/Make.out" 2>&1
>
> And then run torture.sh in the same way that got you the error?
I do above, and also fails:
$git diff
diff --git a/tools/testing/selftests/rcutorture/bin/torture.sh
b/tools/testing/selftests/rcutorture/bin/torture.sh
index d1cb60085d8f..8ece2f0a51c7 100755
--- a/tools/testing/selftests/rcutorture/bin/torture.sh
+++ b/tools/testing/selftests/rcutorture/bin/torture.sh
@@ -319,8 +319,8 @@ then
echo " --- allmodconfig:" Start `date` | tee -a $T/log
amcdir="tools/testing/selftests/rcutorture/res/$ds/allmodconfig"
mkdir -p "$amcdir"
- echo " --- make clean" > "$amcdir/Make.out" 2>&1
- make -j$MAKE_ALLOTED_CPUS clean >> "$amcdir/Make.out" 2>&1
+ echo " --- make distclean" > "$amcdir/Make.out" 2>&1
+ make -j$MAKE_ALLOTED_CPUS distclean >> "$amcdir/Make.out" 2>&1
echo " --- make allmodconfig" >> "$amcdir/Make.out" 2>&1
make -j$MAKE_ALLOTED_CPUS allmodconfig >> "$amcdir/Make.out" 2>&1
echo " --- make " >> "$amcdir/Make.out" 2>&1
$tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
$grep kvm.o ./tools/testing/selftests/rcutorture/res/2022.01.26-09.08.53-torture/allmodconfig/Make.out
make[2]: *** No rule to make target
'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',
needed by 'arch/x86/kvm/kvm.o'. Stop.

I am very interested in this problem, and am willing to track this
problem with other developers in the coming days ;-)

Thanks
Zhouyi
>
> Thanx, Paul
>
> > $make allmodconfig
> > $make -j 16
> > Kernel: arch/x86/boot/bzImage is ready (#2)
> > GEN Module.symvers
> > $echo $?
> > 0
> >
> > I am very interested in this problem and I am eager to try to dig out
> > what happened and fix it before this Sunday. And it is more delightful
> > to me that other developers can solve this problem because they are
> > more mature than me ;-)
> >
> > Many thanks
> > Zhouyi
> > > >
> > > > Me, I run torture.sh on them from time to time, depending on how much
> > > > other testing I am doing. So maybe once or twice a week. ;-)
> > > You are the example worth learning in my life, to be honest , I will
> > > do the same way as you do!
> > >
> > > Sincerely
> > > Zhouyi
> > > >
> > > > Thanx, Paul
> > > >
> > > > > Many thanks
> > > > > Zhouyi
> > > > > >
> > > > > > Thanx, Paul
> > > > > >
> > > > > > > > > > Either way, it looks like I need to upgrade the torture.sh script's
> > > > > > > > > > checks for failed builds. Thank you for reporting this!
> > > > > > > > >
> > > > > > > > > Does this make torture.sh more reliably report build failures?
> > > > > > > > I studied this commit line by line several times and benefited a lot. Thank you!
> > > > > > > > >
> > > > > > > > > Thanx, Paul
> > > > > > > > >
> > > > > > > > > ------------------------------------------------------------------------
> > > > > > > > >
> > > > > > > > > commit 0d302830515307ceb58e89d5fb91e81b6d22e0bf
> > > > > > > > > Author: Paul E. McKenney <[email protected]>
> > > > > > > > > Date: Tue Jan 18 15:40:49 2022 -0800
> > > > > > > > >
> > > > > > > > > torture: Make kvm-find-errors.sh notice missing vmlinux file
> > > > > > > > >
> > > > > > > > > Currently, an obtuse compiler diagnostic can fool kvm-find-errors.sh
> > > > > > > > > into believing that the build was successful. This commit therefore
> > > > > > > > > adds a check for a missing vmlinux file.
> > > > > > > > >
> > > > > > > > > Link: https://lore.kernel.org/lkml/[email protected]/
> > > > > > > > > Reported-by: Paul Menzel <[email protected]>
> > > > > > > > > Signed-off-by: Paul E. McKenney <[email protected]>
> > > > > > > > >
> > > > > > > > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > > > > index 2e9e9e2eedb69..7d3e11a6b8290 100755
> > > > > > > > > --- a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > > > > +++ b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > > > > @@ -30,10 +30,15 @@ editor=${EDITOR-vi}
> > > > > > > > > files=
> > > > > > > > > for i in ${rundir}/*/Make.out
> > > > > > > > > do
> > > > > > > > > + scenariodir="`dirname $i`"
> > > > > > > > > if egrep -q "error:|warning:|^ld: .*undefined reference to" < $i
> > > > > > > > > then
> > > > > > > > > egrep "error:|warning:|^ld: .*undefined reference to" < $i > $i.diags
> > > > > > > > > files="$files $i.diags $i"
> > > > > > > > > + elif ! test -f ${scenariodir}/vmlinux
> > > > > > > > > + then
> > > > > > > > > + echo No ${scenariodir}/vmlinux file > $i.diags
> > > > > > > > > + files="$files $i.diags $i"
> > > > > > > > > fi
> > > > > > > > > done
> > > > > > > > > if test -n "$files"
> > > > > > > > Thanks
> > > > > > > > Zhouyi

2022-01-26 18:03:02

by Zhouyi Zhou

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

Great Discovery !!
Hi Paul,
I track the log
make[2]: *** No rule to make target
'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',
needed by 'arch/x86/kvm/kvm.o
into the file virt/kvm/Makefile.kvm:
1# SPDX-License-Identifier: GPL-2.0
2#
3# Makefile for Kernel-based Virtual Machine module
4#
5
6KVM ?= ../../../virt/kvm
7
8kvm-y := $(KVM)/kvm_main.o $(KVM)/eventfd.o $(KVM)/binary_stats.o
9kvm-$(CONFIG_KVM_VFIO) += $(KVM)/vfio.o
10kvm-$(CONFIG_KVM_MMIO) += $(KVM)/coalesced_mmio.o
11kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o
12kvm-$(CONFIG_HAVE_KVM_IRQ_ROUTING) += $(KVM)/irqchip.o
13kvm-$(CONFIG_HAVE_KVM_DIRTY_RING) += $(KVM)/dirty_ring.o
14kvm-$(CONFIG_HAVE_KVM_PFNCACHE) += $(KVM)/pfncache.o
line 6's ?= indicates to set the KDIR variable only if it's not
set/doesn't have a value.

I guess we have set the variable KVM somewhere before make, I will
continue tracking it.

Thanks
Zhouyi






On Wed, Jan 26, 2022 at 9:59 AM Zhouyi Zhou <[email protected]> wrote:
>
> Thank Paul for your attention
>
> On Wed, Jan 26, 2022 at 2:09 AM Paul E. McKenney <[email protected]> wrote:
> >
> > On Wed, Jan 26, 2022 at 12:19:58AM +0800, Zhouyi Zhou wrote:
> > > Hi Paul
> > >
> > > On Fri, Jan 21, 2022 at 8:44 AM Zhouyi Zhou <[email protected]> wrote:
> > > >
> > > > Thank Paul for your guidance!
> > > >
> > > > On Fri, Jan 21, 2022 at 5:06 AM Paul E. McKenney <[email protected]> wrote:
> > > > >
> > > > > On Thu, Jan 20, 2022 at 03:30:54AM +0800, Zhouyi Zhou wrote:
> > > > > > On Thu, Jan 20, 2022 at 2:21 AM Paul E. McKenney <[email protected]> wrote:
> > > > > > >
> > > > > > > On Tue, Jan 18, 2022 at 08:46:49PM -0800, Paul E. McKenney wrote:
> > > > > > > > On Wed, Jan 19, 2022 at 10:07:42AM +0800, Zhouyi Zhou wrote:
> > > > > > > > > Thanks Paul for looking into this
> > > > > > > > >
> > > > > > > > > On Wed, Jan 19, 2022 at 7:46 AM Paul E. McKenney <[email protected]> wrote:
> > > > > > > > > >
> > > > > > > > > > On Tue, Jan 18, 2022 at 09:29:04AM -0800, Paul E. McKenney wrote:
> > > > > > > > > > > On Tue, Jan 18, 2022 at 08:56:24AM +0100, Paul Menzel wrote:
> > > > > > > > > > > > Dear Zhouyi,
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Thank you for your quick response.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Am 18.01.22 um 08:34 schrieb Zhouyi Zhou:
> > > > > > > > > > > >
> > > > > > > > > > > > > I have studied the rcu torture test recently. I am also interested in
> > > > > > > > > > > > > this topic.
> > > > > > > > > > > > > But I can't open
> > > > > > > > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/allmodconf-Make.out.txt
> > > > > > > > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture-log.txt
> > > > > > > > > > > >
> > > > > > > > > > > > Sorry, about that. I should have checked those. I had put them into a
> > > > > > > > > > > > directory:
> > > > > > > > > > > >
> > > > > > > > > > > > [1]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/allmodconf-Make.out.txt
> > > > > > > > > > > > [2]: https://owww.molgen.mpg.de/~pmenzel/rcutorture/rcutorture-log.txt
> > > > > > > > > > > >
> > > > > > > > > > > > I am going to try to test your suggestions at the end of the day.
> > > > > > > > > > >
> > > > > > > > > > > On x86 rcutorture builds successfully. However, allmodconfig
> > > > > > > > > > > on semi-recent -next got me "Can't open perl script
> > > > > > > > > > > "./usr/include/headers_check.pl": No such file or directory".
> > > > > > > > > > > Which might well be a local problem or might well be fixed by now.
> > > > > > > > > >
> > > > > > > > > > Not fixed as of next-20220118. Chasing it down... ;-)
> > > > > > > > > I can do allmodconfig on -next,
> > > > > > > > > $git describe
> > > > > > > > > next-20220118
> > > > > > > > > $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > > > > > > > --- tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > > > > > > > --- Results directory: 2022.01.19-09.14.39-torture
> > > > > > > > > $ ps -aux|grep qemu-system
> > > > > > > > > zzy 470309 773 0.3 1876544 153936 pts/0 Sl+ 09:55 31:27
> > > > > > > > > qemu-system-x86_64 -enable-kvm -nographic -smp 16 -net none -machine
> > > > > > > > > q35,accel=kvm -cpu kvm64 -serial
> > > > > > > > > file:/tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/console.log
> > > > > > > > > -m 512 -kernel /tmp/linux-next/tools/testing/selftests/rcutorture/res/2022.01.19-09.14.39-torture/results-rcutorture/TREE03/bzImage
> > > > > > > > > -append debug_boot_weak_hash panic=-1 selinux=0 initcall_debug debug
> > > > > > > > > console=ttyS0 rcupdate.rcu_cpu_stall_suppress_at_boot=1
> > > > > > > > > torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000
> > > > > > > > > rcutorture.onoff_interval=200 rcutorture.onoff_holdoff=30
> > > > > > > > > rcutree.gp_preinit_delay=12 rcutree.gp_init_delay=3
> > > > > > > > > rcutree.gp_cleanup_delay=3 rcutree.kthread_prio=2 threadirqs
> > > > > > > > > tree.use_softirq=0 rcutorture.n_barrier_cbs=4
> > > > > > > > > rcutorture.stat_interval=15 rcutorture.shutdown_secs=420
> > > > > > > > > rcutorture.test_no_idle_hz=1 rcutorture.verbose=1
> > > > > > > > > zzy 755865 0.0 0.0 17676 2876 pts/2 S+ 09:59 0:00
> > > > > > > > > grep --color=auto qemu-system
> > > > > > > > > $ ls -l vmlinux
> > > > > > > > > -rwxrwxr-x 1 zzy zzy 69349872 1月 19 09:55 vmlinux
> > > > > > > > >
> > > > > > > > > Could you please try the following command ?
> > > > > > > > > linux-next$ perl ./usr/include/headers_check.pl usr/include x86
> > > > > > > > > usr/include/rdma/hfi/hfi1_user.h
> > > > > > > > > linux-next$ echo $?
> > > > > > > > > 0
> > > > > > > > > The headers_check.pl in linux-next
> > > > > > > > > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/usr/include/headers_check.pl)
> > > > > > > > > is used to check the validity of head files in ./usr/include
> > > > > > > >
> > > > > > > > I am currently bisecting, but once that finishes I will give this a
> > > > > > > > try, thank you!
> > > > > > >
> > > > > > > And the bisection converged badly due to there being more than one build
> > > > > > > failure.
> > > > > > Sorry to hear the bisection is not successful.
> > > > > > >
> > > > > > > So I ran the command above and then the build succeeded.
> > > > > > I'm very happy that the command works.
> > > > > > >
> > > > > > > Though it would be good if the build worked from "make distclean"...
> > > > > > I would be very honored if I could join the efforts to make the -next better.linux-next/tools/testing/selftests/rcutorture/res/2022.01.25-23.10.36-torture/allmodconfig
> > > > >
> > > > > One approach would be to download the -next kernels as they appear and
> > > > > run tests on them. Reporting bugs is good, and of course fixing them
> > > > > even better.
> > > > This provides a nice way of participating the efforts to make -next
> > > > better, and provides me a good way to learn new knowledge in -next!
> > > >
> > > > >
> > > > > You can clone -next from here:
> > > > >
> > > > > git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> > > > >
> > > > > Individual -next releases are tagged, for example, "next-20220118".
> > > > I will clone -next from the tree, and will download the -next releases from
> > > > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> > > I cloned -next, and try to test rcu:
> > > $git clone https://kernel.source.codeaurora.cn/pub/scm/linux/kernel/git/next/linux-next.git
> > > (kernel.source.codeaurora.cn is Chinese mirror of git.kernel.org)
> > > $cd linux-next
> > > $git describe
> > > next-20220125
> > > $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> > > $cd linux-next/tools/testing/selftests/rcutorture/res/2022.01.25-23.10.36-torture/allmodconfig
> > > $cat Make.exitcode
> > > 2
> > > $grep Stop Make.out
> > > make[2]: *** No rule to make target
> > > 'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',
> > > needed by 'arch/x86/kvm/kvm.o'. Stop.
> > > I tried the above two times in two different machines today, they all failed.
> > >
> > >
> > > Meanwhile I can config and build the kernel successfully by invoking
> > > the build commands by hand:
> > > $git clone https://kernel.source.codeaurora.cn/pub/scm/linux/kernel/git/next/linux-next.git
> > > $cd linux-next
> > > $git describe
> > > next-20220125
> >
> > Huh. What happens if you do a "make clean" betweentimes?
> do a "make clean" between times also fails, sorry
> >
> > Alternatively, what happens if you change these two lines from torture.sh:
> >
> > echo " --- make clean" > "$amcdir/Make.out" 2>&1
> > make -j$MAKE_ALLOTED_CPUS clean >> "$amcdir/Make.out" 2>&1
> >
> > To this:
> >
> > echo " --- make distclean" > "$amcdir/Make.out" 2>&1
> > make -j$MAKE_ALLOTED_CPUS distclean >> "$amcdir/Make.out" 2>&1
> >
> > And then run torture.sh in the same way that got you the error?
> I do above, and also fails:
> $git diff
> diff --git a/tools/testing/selftests/rcutorture/bin/torture.sh
> b/tools/testing/selftests/rcutorture/bin/torture.sh
> index d1cb60085d8f..8ece2f0a51c7 100755
> --- a/tools/testing/selftests/rcutorture/bin/torture.sh
> +++ b/tools/testing/selftests/rcutorture/bin/torture.sh
> @@ -319,8 +319,8 @@ then
> echo " --- allmodconfig:" Start `date` | tee -a $T/log
> amcdir="tools/testing/selftests/rcutorture/res/$ds/allmodconfig"
> mkdir -p "$amcdir"
> - echo " --- make clean" > "$amcdir/Make.out" 2>&1
> - make -j$MAKE_ALLOTED_CPUS clean >> "$amcdir/Make.out" 2>&1
> + echo " --- make distclean" > "$amcdir/Make.out" 2>&1
> + make -j$MAKE_ALLOTED_CPUS distclean >> "$amcdir/Make.out" 2>&1
> echo " --- make allmodconfig" >> "$amcdir/Make.out" 2>&1
> make -j$MAKE_ALLOTED_CPUS allmodconfig >> "$amcdir/Make.out" 2>&1
> echo " --- make " >> "$amcdir/Make.out" 2>&1
> $tools/testing/selftests/rcutorture/bin/torture.sh --duration 10
> $grep kvm.o ./tools/testing/selftests/rcutorture/res/2022.01.26-09.08.53-torture/allmodconfig/Make.out
> make[2]: *** No rule to make target
> 'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',
> needed by 'arch/x86/kvm/kvm.o'. Stop.
>
> I am very interested in this problem, and am willing to track this
> problem with other developers in the coming days ;-)
>
> Thanks
> Zhouyi
> >
> > Thanx, Paul
> >
> > > $make allmodconfig
> > > $make -j 16
> > > Kernel: arch/x86/boot/bzImage is ready (#2)
> > > GEN Module.symvers
> > > $echo $?
> > > 0
> > >
> > > I am very interested in this problem and I am eager to try to dig out
> > > what happened and fix it before this Sunday. And it is more delightful
> > > to me that other developers can solve this problem because they are
> > > more mature than me ;-)
> > >
> > > Many thanks
> > > Zhouyi
> > > > >
> > > > > Me, I run torture.sh on them from time to time, depending on how much
> > > > > other testing I am doing. So maybe once or twice a week. ;-)
> > > > You are the example worth learning in my life, to be honest , I will
> > > > do the same way as you do!
> > > >
> > > > Sincerely
> > > > Zhouyi
> > > > >
> > > > > Thanx, Paul
> > > > >
> > > > > > Many thanks
> > > > > > Zhouyi
> > > > > > >
> > > > > > > Thanx, Paul
> > > > > > >
> > > > > > > > > > > Either way, it looks like I need to upgrade the torture.sh script's
> > > > > > > > > > > checks for failed builds. Thank you for reporting this!
> > > > > > > > > >
> > > > > > > > > > Does this make torture.sh more reliably report build failures?
> > > > > > > > > I studied this commit line by line several times and benefited a lot. Thank you!
> > > > > > > > > >
> > > > > > > > > > Thanx, Paul
> > > > > > > > > >
> > > > > > > > > > ------------------------------------------------------------------------
> > > > > > > > > >
> > > > > > > > > > commit 0d302830515307ceb58e89d5fb91e81b6d22e0bf
> > > > > > > > > > Author: Paul E. McKenney <[email protected]>
> > > > > > > > > > Date: Tue Jan 18 15:40:49 2022 -0800
> > > > > > > > > >
> > > > > > > > > > torture: Make kvm-find-errors.sh notice missing vmlinux file
> > > > > > > > > >
> > > > > > > > > > Currently, an obtuse compiler diagnostic can fool kvm-find-errors.sh
> > > > > > > > > > into believing that the build was successful. This commit therefore
> > > > > > > > > > adds a check for a missing vmlinux file.
> > > > > > > > > >
> > > > > > > > > > Link: https://lore.kernel.org/lkml/[email protected]/
> > > > > > > > > > Reported-by: Paul Menzel <[email protected]>
> > > > > > > > > > Signed-off-by: Paul E. McKenney <[email protected]>
> > > > > > > > > >
> > > > > > > > > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > > > > > index 2e9e9e2eedb69..7d3e11a6b8290 100755
> > > > > > > > > > --- a/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > > > > > +++ b/tools/testing/selftests/rcutorture/bin/kvm-find-errors.sh
> > > > > > > > > > @@ -30,10 +30,15 @@ editor=${EDITOR-vi}
> > > > > > > > > > files=
> > > > > > > > > > for i in ${rundir}/*/Make.out
> > > > > > > > > > do
> > > > > > > > > > + scenariodir="`dirname $i`"
> > > > > > > > > > if egrep -q "error:|warning:|^ld: .*undefined reference to" < $i
> > > > > > > > > > then
> > > > > > > > > > egrep "error:|warning:|^ld: .*undefined reference to" < $i > $i.diags
> > > > > > > > > > files="$files $i.diags $i"
> > > > > > > > > > + elif ! test -f ${scenariodir}/vmlinux
> > > > > > > > > > + then
> > > > > > > > > > + echo No ${scenariodir}/vmlinux file > $i.diags
> > > > > > > > > > + files="$files $i.diags $i"
> > > > > > > > > > fi
> > > > > > > > > > done
> > > > > > > > > > if test -n "$files"
> > > > > > > > > Thanks
> > > > > > > > > Zhouyi

2022-01-26 18:37:08

by Paul E. McKenney

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

On Wed, Jan 26, 2022 at 12:09:12PM +0800, Zhouyi Zhou wrote:
> Great Discovery !!
> Hi Paul,
> I track the log
> make[2]: *** No rule to make target
> 'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',

That does look like a very odd pathname, now that you mention it.

> needed by 'arch/x86/kvm/kvm.o
> into the file virt/kvm/Makefile.kvm:
> 1# SPDX-License-Identifier: GPL-2.0
> 2#
> 3# Makefile for Kernel-based Virtual Machine module
> 4#
> 5
> 6KVM ?= ../../../virt/kvm
> 7
> 8kvm-y := $(KVM)/kvm_main.o $(KVM)/eventfd.o $(KVM)/binary_stats.o
> 9kvm-$(CONFIG_KVM_VFIO) += $(KVM)/vfio.o
> 10kvm-$(CONFIG_KVM_MMIO) += $(KVM)/coalesced_mmio.o
> 11kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o
> 12kvm-$(CONFIG_HAVE_KVM_IRQ_ROUTING) += $(KVM)/irqchip.o
> 13kvm-$(CONFIG_HAVE_KVM_DIRTY_RING) += $(KVM)/dirty_ring.o
> 14kvm-$(CONFIG_HAVE_KVM_PFNCACHE) += $(KVM)/pfncache.o
> line 6's ?= indicates to set the KDIR variable only if it's not
> set/doesn't have a value.
>
> I guess we have set the variable KVM somewhere before make, I will
> continue tracking it.

But if you build an allmodconfig kernel by hand afterwards, it works?

I wonder if I need to use a different environment variable in my scripts.
For example, from tools/testing/selftests/rcutorture/bin/kvm.sh:

KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
PATH=${KVM}/bin:$PATH; export PATH

Maybe like the (untested) patch shown below.

Thanx, Paul

------------------------------------------------------------------------

diff --git a/tools/testing/selftests/rcutorture/bin/kvm-again.sh b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
index 5a0023d183dac..0941f1ddab658 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-again.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
@@ -47,8 +47,8 @@ else
exit 1
fi

-KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
-PATH=${KVM}/bin:$PATH; export PATH
+RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
. functions.sh

dryrun=
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh b/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
index 370406bbfeedc..f17000a2ccf1f 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
@@ -49,8 +49,8 @@ fi
mkdir $resdir/$ds
echo Results directory: $resdir/$ds

-KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
-PATH=${KVM}/bin:$PATH; export PATH
+RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
. functions.sh
echo Using all `identify_qemu_vcpus` CPUs.

diff --git a/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
index e4a00779b8c69..ee886b40a5d2c 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
@@ -22,8 +22,8 @@ T=${TMPDIR-/tmp}/kvm-end-run-stats.sh.$$
trap 'rm -rf $T' 0
mkdir $T

-KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
-PATH=${KVM}/bin:$PATH; export PATH
+RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
. functions.sh
default_starttime="`get_starttime`"
starttime="${2-default_starttime}"
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
index 29b068a55b466..8c4c1e4792d02 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
@@ -19,8 +19,8 @@ then
exit 1
fi

-KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
-PATH=${KVM}/bin:$PATH; export PATH
+RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
. functions.sh

starttime="`get_starttime`"
@@ -108,8 +108,8 @@ else
cat $T/kvm-again.sh.out | tee -a "$oldrun/remote-log"
exit 2
fi
- cp -a "$rundir" "$KVM/res/"
- oldrun="$KVM/res/$ds"
+ cp -a "$rundir" "$RCUTORTURE/res/"
+ oldrun="$RCUTORTURE/res/$ds"
fi
echo | tee -a "$oldrun/remote-log"
echo " ----" kvm-again.sh output: "(`date`)" | tee -a "$oldrun/remote-log"
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index 348ad177a5acc..55b2c15332827 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -25,15 +25,15 @@ LANG=en_US.UTF-8; export LANG

dur=$((30*60))
dryrun=""
-KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
-PATH=${KVM}/bin:$PATH; export PATH
+RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
. functions.sh

TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"
TORTURE_DEFCONFIG=defconfig
TORTURE_BOOT_IMAGE=""
TORTURE_BUILDONLY=
-TORTURE_INITRD="$KVM/initrd"; export TORTURE_INITRD
+TORTURE_INITRD="$RCUTORTURE/initrd"; export TORTURE_INITRD
TORTURE_KCONFIG_ARG=""
TORTURE_KCONFIG_GDB_ARG=""
TORTURE_BOOT_GDB_ARG=""
@@ -262,7 +262,7 @@ else
exit 1
fi

-CONFIGFRAG=${KVM}/configs/${TORTURE_SUITE}; export CONFIGFRAG
+CONFIGFRAG=${RCUTORTURE}/configs/${TORTURE_SUITE}; export CONFIGFRAG

defaultconfigs="`tr '\012' ' ' < $CONFIGFRAG/CFLIST`"
if test -z "$configs"
@@ -272,7 +272,7 @@ fi

if test -z "$resdir"
then
- resdir=$KVM/res
+ resdir=$RCUTORTURE/res
fi

# Create a file of test-name/#cpus pairs, sorted by decreasing #cpus.
@@ -386,7 +386,7 @@ END {
# Generate a script to execute the tests in appropriate batches.
cat << ___EOF___ > $T/script
CONFIGFRAG="$CONFIGFRAG"; export CONFIGFRAG
-KVM="$KVM"; export KVM
+RCUTORTURE="$RCUTORTURE"; export RCUTORTURE
PATH="$PATH"; export PATH
TORTURE_ALLOTED_CPUS="$TORTURE_ALLOTED_CPUS"; export TORTURE_ALLOTED_CPUS
TORTURE_BOOT_IMAGE="$TORTURE_BOOT_IMAGE"; export TORTURE_BOOT_IMAGE
@@ -569,7 +569,7 @@ ___EOF___
awk < $T/cfgcpu.pack \
-v TORTURE_BUILDONLY="$TORTURE_BUILDONLY" \
-v CONFIGDIR="$CONFIGFRAG/" \
- -v KVM="$KVM" \
+ -v RCUTORTURE="$RCUTORTURE" \
-v ncpus=$cpus \
-v jitter="$jitter" \
-v rd=$resdir/$ds/ \
diff --git a/tools/testing/selftests/rcutorture/bin/torture.sh b/tools/testing/selftests/rcutorture/bin/torture.sh
index d1cb60085d8f2..e00e60efb2310 100755
--- a/tools/testing/selftests/rcutorture/bin/torture.sh
+++ b/tools/testing/selftests/rcutorture/bin/torture.sh
@@ -13,8 +13,8 @@
scriptname=$0
args="$*"

-KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
-PATH=${KVM}/bin:$PATH; export PATH
+RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
. functions.sh

TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"

2022-01-26 18:39:00

by Paul E. McKenney

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

On Tue, Jan 25, 2022 at 08:50:22PM -0800, Paul E. McKenney wrote:
> On Wed, Jan 26, 2022 at 12:09:12PM +0800, Zhouyi Zhou wrote:
> > Great Discovery !!
> > Hi Paul,
> > I track the log
> > make[2]: *** No rule to make target
> > 'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',
>
> That does look like a very odd pathname, now that you mention it.
>
> > needed by 'arch/x86/kvm/kvm.o
> > into the file virt/kvm/Makefile.kvm:
> > 1# SPDX-License-Identifier: GPL-2.0
> > 2#
> > 3# Makefile for Kernel-based Virtual Machine module
> > 4#
> > 5
> > 6KVM ?= ../../../virt/kvm
> > 7
> > 8kvm-y := $(KVM)/kvm_main.o $(KVM)/eventfd.o $(KVM)/binary_stats.o
> > 9kvm-$(CONFIG_KVM_VFIO) += $(KVM)/vfio.o
> > 10kvm-$(CONFIG_KVM_MMIO) += $(KVM)/coalesced_mmio.o
> > 11kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o
> > 12kvm-$(CONFIG_HAVE_KVM_IRQ_ROUTING) += $(KVM)/irqchip.o
> > 13kvm-$(CONFIG_HAVE_KVM_DIRTY_RING) += $(KVM)/dirty_ring.o
> > 14kvm-$(CONFIG_HAVE_KVM_PFNCACHE) += $(KVM)/pfncache.o
> > line 6's ?= indicates to set the KDIR variable only if it's not
> > set/doesn't have a value.
> >
> > I guess we have set the variable KVM somewhere before make, I will
> > continue tracking it.
>
> But if you build an allmodconfig kernel by hand afterwards, it works?
>
> I wonder if I need to use a different environment variable in my scripts.
> For example, from tools/testing/selftests/rcutorture/bin/kvm.sh:
>
> KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> PATH=${KVM}/bin:$PATH; export PATH
>
> Maybe like the (untested) patch shown below.

A somewhat more official version below. Thank you for following up
on this!

Thanx, Paul

------------------------------------------------------------------------

commit 6b5cdbe80b01ce604fdf0f3ef6717a6466aca041
Author: Paul E. McKenney <[email protected]>
Date: Tue Jan 25 21:08:55 2022 -0800

torture: Change KVM environment variable to RCUTORTURE

The torture-test scripting's long-standing use of KVM as the environment
variable tracking the pathname of the rcutorture directory now conflicts
with allmodconfig builds due to the virt/kvm/Makefile.kvm file's use
of this as a makefile variable. This commit therefore changes the
torture-test scripting from KVM to RCUTORTURE, avoiding the name conflict.

Reported-by: Zhouyi Zhou <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>

diff --git a/tools/testing/selftests/rcutorture/bin/kvm-again.sh b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
index 5a0023d183dac..0941f1ddab658 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-again.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
@@ -47,8 +47,8 @@ else
exit 1
fi

-KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
-PATH=${KVM}/bin:$PATH; export PATH
+RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
. functions.sh

dryrun=
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh b/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
index 370406bbfeedc..f17000a2ccf1f 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
@@ -49,8 +49,8 @@ fi
mkdir $resdir/$ds
echo Results directory: $resdir/$ds

-KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
-PATH=${KVM}/bin:$PATH; export PATH
+RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
. functions.sh
echo Using all `identify_qemu_vcpus` CPUs.

diff --git a/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
index e4a00779b8c69..ee886b40a5d2c 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
@@ -22,8 +22,8 @@ T=${TMPDIR-/tmp}/kvm-end-run-stats.sh.$$
trap 'rm -rf $T' 0
mkdir $T

-KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
-PATH=${KVM}/bin:$PATH; export PATH
+RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
. functions.sh
default_starttime="`get_starttime`"
starttime="${2-default_starttime}"
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
index 29b068a55b466..8c4c1e4792d02 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
@@ -19,8 +19,8 @@ then
exit 1
fi

-KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
-PATH=${KVM}/bin:$PATH; export PATH
+RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
. functions.sh

starttime="`get_starttime`"
@@ -108,8 +108,8 @@ else
cat $T/kvm-again.sh.out | tee -a "$oldrun/remote-log"
exit 2
fi
- cp -a "$rundir" "$KVM/res/"
- oldrun="$KVM/res/$ds"
+ cp -a "$rundir" "$RCUTORTURE/res/"
+ oldrun="$RCUTORTURE/res/$ds"
fi
echo | tee -a "$oldrun/remote-log"
echo " ----" kvm-again.sh output: "(`date`)" | tee -a "$oldrun/remote-log"
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index 348ad177a5acc..55b2c15332827 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -25,15 +25,15 @@ LANG=en_US.UTF-8; export LANG

dur=$((30*60))
dryrun=""
-KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
-PATH=${KVM}/bin:$PATH; export PATH
+RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
. functions.sh

TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"
TORTURE_DEFCONFIG=defconfig
TORTURE_BOOT_IMAGE=""
TORTURE_BUILDONLY=
-TORTURE_INITRD="$KVM/initrd"; export TORTURE_INITRD
+TORTURE_INITRD="$RCUTORTURE/initrd"; export TORTURE_INITRD
TORTURE_KCONFIG_ARG=""
TORTURE_KCONFIG_GDB_ARG=""
TORTURE_BOOT_GDB_ARG=""
@@ -262,7 +262,7 @@ else
exit 1
fi

-CONFIGFRAG=${KVM}/configs/${TORTURE_SUITE}; export CONFIGFRAG
+CONFIGFRAG=${RCUTORTURE}/configs/${TORTURE_SUITE}; export CONFIGFRAG

defaultconfigs="`tr '\012' ' ' < $CONFIGFRAG/CFLIST`"
if test -z "$configs"
@@ -272,7 +272,7 @@ fi

if test -z "$resdir"
then
- resdir=$KVM/res
+ resdir=$RCUTORTURE/res
fi

# Create a file of test-name/#cpus pairs, sorted by decreasing #cpus.
@@ -386,7 +386,7 @@ END {
# Generate a script to execute the tests in appropriate batches.
cat << ___EOF___ > $T/script
CONFIGFRAG="$CONFIGFRAG"; export CONFIGFRAG
-KVM="$KVM"; export KVM
+RCUTORTURE="$RCUTORTURE"; export RCUTORTURE
PATH="$PATH"; export PATH
TORTURE_ALLOTED_CPUS="$TORTURE_ALLOTED_CPUS"; export TORTURE_ALLOTED_CPUS
TORTURE_BOOT_IMAGE="$TORTURE_BOOT_IMAGE"; export TORTURE_BOOT_IMAGE
@@ -569,7 +569,7 @@ ___EOF___
awk < $T/cfgcpu.pack \
-v TORTURE_BUILDONLY="$TORTURE_BUILDONLY" \
-v CONFIGDIR="$CONFIGFRAG/" \
- -v KVM="$KVM" \
+ -v RCUTORTURE="$RCUTORTURE" \
-v ncpus=$cpus \
-v jitter="$jitter" \
-v rd=$resdir/$ds/ \
diff --git a/tools/testing/selftests/rcutorture/bin/torture.sh b/tools/testing/selftests/rcutorture/bin/torture.sh
index d1cb60085d8f2..e00e60efb2310 100755
--- a/tools/testing/selftests/rcutorture/bin/torture.sh
+++ b/tools/testing/selftests/rcutorture/bin/torture.sh
@@ -13,8 +13,8 @@
scriptname=$0
args="$*"

-KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
-PATH=${KVM}/bin:$PATH; export PATH
+RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
. functions.sh

TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"

2022-01-26 18:42:52

by Zhouyi Zhou

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

Thanks Paul for fixing it

I will perform the tests now, since
"tools/testing/selftests/rcutorture/bin/torture.sh --duration 10" may
last for some hours, I guess I will send the test results late at
night (Pacific Time) ;-)

Thanks again
Cheers
Zhouyi


On Wed, Jan 26, 2022 at 1:14 PM Paul E. McKenney <[email protected]> wrote:
>
> On Tue, Jan 25, 2022 at 08:50:22PM -0800, Paul E. McKenney wrote:
> > On Wed, Jan 26, 2022 at 12:09:12PM +0800, Zhouyi Zhou wrote:
> > > Great Discovery !!
> > > Hi Paul,
> > > I track the log
> > > make[2]: *** No rule to make target
> > > 'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',
> >
> > That does look like a very odd pathname, now that you mention it.
> >
> > > needed by 'arch/x86/kvm/kvm.o
> > > into the file virt/kvm/Makefile.kvm:
> > > 1# SPDX-License-Identifier: GPL-2.0
> > > 2#
> > > 3# Makefile for Kernel-based Virtual Machine module
> > > 4#
> > > 5
> > > 6KVM ?= ../../../virt/kvm
> > > 7
> > > 8kvm-y := $(KVM)/kvm_main.o $(KVM)/eventfd.o $(KVM)/binary_stats.o
> > > 9kvm-$(CONFIG_KVM_VFIO) += $(KVM)/vfio.o
> > > 10kvm-$(CONFIG_KVM_MMIO) += $(KVM)/coalesced_mmio.o
> > > 11kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o
> > > 12kvm-$(CONFIG_HAVE_KVM_IRQ_ROUTING) += $(KVM)/irqchip.o
> > > 13kvm-$(CONFIG_HAVE_KVM_DIRTY_RING) += $(KVM)/dirty_ring.o
> > > 14kvm-$(CONFIG_HAVE_KVM_PFNCACHE) += $(KVM)/pfncache.o
> > > line 6's ?= indicates to set the KDIR variable only if it's not
> > > set/doesn't have a value.
> > >
> > > I guess we have set the variable KVM somewhere before make, I will
> > > continue tracking it.
> >
> > But if you build an allmodconfig kernel by hand afterwards, it works?
> >
> > I wonder if I need to use a different environment variable in my scripts.
> > For example, from tools/testing/selftests/rcutorture/bin/kvm.sh:
> >
> > KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > PATH=${KVM}/bin:$PATH; export PATH
> >
> > Maybe like the (untested) patch shown below.
>
> A somewhat more official version below. Thank you for following up
> on this!
>
> Thanx, Paul
>
> ------------------------------------------------------------------------
>
> commit 6b5cdbe80b01ce604fdf0f3ef6717a6466aca041
> Author: Paul E. McKenney <[email protected]>
> Date: Tue Jan 25 21:08:55 2022 -0800
>
> torture: Change KVM environment variable to RCUTORTURE
>
> The torture-test scripting's long-standing use of KVM as the environment
> variable tracking the pathname of the rcutorture directory now conflicts
> with allmodconfig builds due to the virt/kvm/Makefile.kvm file's use
> of this as a makefile variable. This commit therefore changes the
> torture-test scripting from KVM to RCUTORTURE, avoiding the name conflict.
>
> Reported-by: Zhouyi Zhou <[email protected]>
> Signed-off-by: Paul E. McKenney <[email protected]>
>
> diff --git a/tools/testing/selftests/rcutorture/bin/kvm-again.sh b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
> index 5a0023d183dac..0941f1ddab658 100755
> --- a/tools/testing/selftests/rcutorture/bin/kvm-again.sh
> +++ b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
> @@ -47,8 +47,8 @@ else
> exit 1
> fi
>
> -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> -PATH=${KVM}/bin:$PATH; export PATH
> +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> . functions.sh
>
> dryrun=
> diff --git a/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh b/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
> index 370406bbfeedc..f17000a2ccf1f 100755
> --- a/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
> +++ b/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
> @@ -49,8 +49,8 @@ fi
> mkdir $resdir/$ds
> echo Results directory: $resdir/$ds
>
> -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> -PATH=${KVM}/bin:$PATH; export PATH
> +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> . functions.sh
> echo Using all `identify_qemu_vcpus` CPUs.
>
> diff --git a/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
> index e4a00779b8c69..ee886b40a5d2c 100755
> --- a/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
> +++ b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
> @@ -22,8 +22,8 @@ T=${TMPDIR-/tmp}/kvm-end-run-stats.sh.$$
> trap 'rm -rf $T' 0
> mkdir $T
>
> -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> -PATH=${KVM}/bin:$PATH; export PATH
> +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> . functions.sh
> default_starttime="`get_starttime`"
> starttime="${2-default_starttime}"
> diff --git a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
> index 29b068a55b466..8c4c1e4792d02 100755
> --- a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
> +++ b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
> @@ -19,8 +19,8 @@ then
> exit 1
> fi
>
> -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> -PATH=${KVM}/bin:$PATH; export PATH
> +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> . functions.sh
>
> starttime="`get_starttime`"
> @@ -108,8 +108,8 @@ else
> cat $T/kvm-again.sh.out | tee -a "$oldrun/remote-log"
> exit 2
> fi
> - cp -a "$rundir" "$KVM/res/"
> - oldrun="$KVM/res/$ds"
> + cp -a "$rundir" "$RCUTORTURE/res/"
> + oldrun="$RCUTORTURE/res/$ds"
> fi
> echo | tee -a "$oldrun/remote-log"
> echo " ----" kvm-again.sh output: "(`date`)" | tee -a "$oldrun/remote-log"
> diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
> index 348ad177a5acc..55b2c15332827 100755
> --- a/tools/testing/selftests/rcutorture/bin/kvm.sh
> +++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
> @@ -25,15 +25,15 @@ LANG=en_US.UTF-8; export LANG
>
> dur=$((30*60))
> dryrun=""
> -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> -PATH=${KVM}/bin:$PATH; export PATH
> +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> . functions.sh
>
> TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"
> TORTURE_DEFCONFIG=defconfig
> TORTURE_BOOT_IMAGE=""
> TORTURE_BUILDONLY=
> -TORTURE_INITRD="$KVM/initrd"; export TORTURE_INITRD
> +TORTURE_INITRD="$RCUTORTURE/initrd"; export TORTURE_INITRD
> TORTURE_KCONFIG_ARG=""
> TORTURE_KCONFIG_GDB_ARG=""
> TORTURE_BOOT_GDB_ARG=""
> @@ -262,7 +262,7 @@ else
> exit 1
> fi
>
> -CONFIGFRAG=${KVM}/configs/${TORTURE_SUITE}; export CONFIGFRAG
> +CONFIGFRAG=${RCUTORTURE}/configs/${TORTURE_SUITE}; export CONFIGFRAG
>
> defaultconfigs="`tr '\012' ' ' < $CONFIGFRAG/CFLIST`"
> if test -z "$configs"
> @@ -272,7 +272,7 @@ fi
>
> if test -z "$resdir"
> then
> - resdir=$KVM/res
> + resdir=$RCUTORTURE/res
> fi
>
> # Create a file of test-name/#cpus pairs, sorted by decreasing #cpus.
> @@ -386,7 +386,7 @@ END {
> # Generate a script to execute the tests in appropriate batches.
> cat << ___EOF___ > $T/script
> CONFIGFRAG="$CONFIGFRAG"; export CONFIGFRAG
> -KVM="$KVM"; export KVM
> +RCUTORTURE="$RCUTORTURE"; export RCUTORTURE
> PATH="$PATH"; export PATH
> TORTURE_ALLOTED_CPUS="$TORTURE_ALLOTED_CPUS"; export TORTURE_ALLOTED_CPUS
> TORTURE_BOOT_IMAGE="$TORTURE_BOOT_IMAGE"; export TORTURE_BOOT_IMAGE
> @@ -569,7 +569,7 @@ ___EOF___
> awk < $T/cfgcpu.pack \
> -v TORTURE_BUILDONLY="$TORTURE_BUILDONLY" \
> -v CONFIGDIR="$CONFIGFRAG/" \
> - -v KVM="$KVM" \
> + -v RCUTORTURE="$RCUTORTURE" \
> -v ncpus=$cpus \
> -v jitter="$jitter" \
> -v rd=$resdir/$ds/ \
> diff --git a/tools/testing/selftests/rcutorture/bin/torture.sh b/tools/testing/selftests/rcutorture/bin/torture.sh
> index d1cb60085d8f2..e00e60efb2310 100755
> --- a/tools/testing/selftests/rcutorture/bin/torture.sh
> +++ b/tools/testing/selftests/rcutorture/bin/torture.sh
> @@ -13,8 +13,8 @@
> scriptname=$0
> args="$*"
>
> -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> -PATH=${KVM}/bin:$PATH; export PATH
> +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> . functions.sh
>
> TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"

2022-01-27 08:15:46

by Zhouyi Zhou

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

Hi Paul

Thank you for your perfect patch!

After apply your patch, I invoke
"tools/testing/selftests/rcutorture/bin/torture.sh --duration 10" on
next-20220125. we succeed in allmodconfig!

Following are the test logs (154.223.142.244 is my own internet cloud
server maintained by myself, I believe it is safe to access the text
content via web, but not so sure)

1. torture log
http://154.223.142.244/logs/20220127/2022.01.26-19.43.04.log (there
are many Chinese words "Thursday" in it because I forget to set
terminal locale to English, I am sorry), I excerpt some content here:

SUCCESSES:
allmodconfig(0)
tools/testing/selftests/rcutorture/res/2022.01.26-19.43.04-torture/allmodconfig
FAILURES:
rcutorture(2) /home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/res/2022.01.26-19.43.04-torture/results-rcutorture
Summary: Warnings: 1 Call Traces: 1
Summary: Warnings: 1 Call Traces: 1
Summary: Warnings: 1 Call Traces: 1
Summary: Warnings: 1 Call Traces: 1
Summary: Warnings: 1 Call Traces: 1

2. Make.exitcode for allmodeconfig
http://154.223.142.244/logs/20220127/allmodconfig/Make.exitcode

3. Make.out for allmodeconfig
http://154.223.142.244/logs/20220127/allmodconfig/Make.out

4. result of invoking: find 2022.01.26-19.43.04-torture -name
console.log.diags|xargs -I{} echo "echo {}; cat {}"|xargs -I{} bash -c
"{}":
http://154.223.142.244/logs/20220127/consoles.logs.diags
All of the console.log.diags complain about one thing:
2022.01.26-19.43.04-torture/results-refscale-clock/NOPREEMPT/console.log.diags
[ 0.017721] WARNING: CPU: 0 PID: 0 at
kernel/trace/trace_events.c:417 trace_event_raw_init+0xea/0x580
[ 0.017754] Call Trace:
2022.01.26-19.43.04-torture/results-rcuscale-kvfree-kasan/TREE/console.log.diags
[ 0.102911] WARNING: CPU: 0 PID: 0 at
kernel/trace/trace_events.c:417 trace_event_raw_init+0x113/0x6e0
[ 0.102992] Call Trace:
2022.01.26-19.43.04-torture/results-rcuscale-tasks-tracing-kasan/TREE/console.log.diags
[ 0.060924] WARNING: CPU: 0 PID: 0 at
kernel/trace/trace_events.c:417 trace_event_raw_init+0x113/0x6e0
[ 0.061006] Call Trace:
I am glad to study these warnings further but they don't seem to be
related to this patch.


Thanks again
Sincerely
Zhouyi

On Wed, Jan 26, 2022 at 1:14 PM Paul E. McKenney <[email protected]> wrote:
>
> On Tue, Jan 25, 2022 at 08:50:22PM -0800, Paul E. McKenney wrote:
> > On Wed, Jan 26, 2022 at 12:09:12PM +0800, Zhouyi Zhou wrote:
> > > Great Discovery !!
> > > Hi Paul,
> > > I track the log
> > > make[2]: *** No rule to make target
> > > 'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',
> >
> > That does look like a very odd pathname, now that you mention it.
> >
> > > needed by 'arch/x86/kvm/kvm.o
> > > into the file virt/kvm/Makefile.kvm:
> > > 1# SPDX-License-Identifier: GPL-2.0
> > > 2#
> > > 3# Makefile for Kernel-based Virtual Machine module
> > > 4#
> > > 5
> > > 6KVM ?= ../../../virt/kvm
> > > 7
> > > 8kvm-y := $(KVM)/kvm_main.o $(KVM)/eventfd.o $(KVM)/binary_stats.o
> > > 9kvm-$(CONFIG_KVM_VFIO) += $(KVM)/vfio.o
> > > 10kvm-$(CONFIG_KVM_MMIO) += $(KVM)/coalesced_mmio.o
> > > 11kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o
> > > 12kvm-$(CONFIG_HAVE_KVM_IRQ_ROUTING) += $(KVM)/irqchip.o
> > > 13kvm-$(CONFIG_HAVE_KVM_DIRTY_RING) += $(KVM)/dirty_ring.o
> > > 14kvm-$(CONFIG_HAVE_KVM_PFNCACHE) += $(KVM)/pfncache.o
> > > line 6's ?= indicates to set the KDIR variable only if it's not
> > > set/doesn't have a value.
> > >
> > > I guess we have set the variable KVM somewhere before make, I will
> > > continue tracking it.
> >
> > But if you build an allmodconfig kernel by hand afterwards, it works?
> >
> > I wonder if I need to use a different environment variable in my scripts.
> > For example, from tools/testing/selftests/rcutorture/bin/kvm.sh:
> >
> > KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > PATH=${KVM}/bin:$PATH; export PATH
> >
> > Maybe like the (untested) patch shown below.
>
> A somewhat more official version below. Thank you for following up
> on this!
>
> Thanx, Paul
>
> ------------------------------------------------------------------------
>
> commit 6b5cdbe80b01ce604fdf0f3ef6717a6466aca041
> Author: Paul E. McKenney <[email protected]>
> Date: Tue Jan 25 21:08:55 2022 -0800
>
> torture: Change KVM environment variable to RCUTORTURE
>
> The torture-test scripting's long-standing use of KVM as the environment
> variable tracking the pathname of the rcutorture directory now conflicts
> with allmodconfig builds due to the virt/kvm/Makefile.kvm file's use
> of this as a makefile variable. This commit therefore changes the
> torture-test scripting from KVM to RCUTORTURE, avoiding the name conflict.
>
> Reported-by: Zhouyi Zhou <[email protected]>
> Signed-off-by: Paul E. McKenney <[email protected]>
>
> diff --git a/tools/testing/selftests/rcutorture/bin/kvm-again.sh b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
> index 5a0023d183dac..0941f1ddab658 100755
> --- a/tools/testing/selftests/rcutorture/bin/kvm-again.sh
> +++ b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
> @@ -47,8 +47,8 @@ else
> exit 1
> fi
>
> -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> -PATH=${KVM}/bin:$PATH; export PATH
> +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> . functions.sh
>
> dryrun=
> diff --git a/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh b/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
> index 370406bbfeedc..f17000a2ccf1f 100755
> --- a/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
> +++ b/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
> @@ -49,8 +49,8 @@ fi
> mkdir $resdir/$ds
> echo Results directory: $resdir/$ds
>
> -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> -PATH=${KVM}/bin:$PATH; export PATH
> +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> . functions.sh
> echo Using all `identify_qemu_vcpus` CPUs.
>
> diff --git a/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
> index e4a00779b8c69..ee886b40a5d2c 100755
> --- a/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
> +++ b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
> @@ -22,8 +22,8 @@ T=${TMPDIR-/tmp}/kvm-end-run-stats.sh.$$
> trap 'rm -rf $T' 0
> mkdir $T
>
> -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> -PATH=${KVM}/bin:$PATH; export PATH
> +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> . functions.sh
> default_starttime="`get_starttime`"
> starttime="${2-default_starttime}"
> diff --git a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
> index 29b068a55b466..8c4c1e4792d02 100755
> --- a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
> +++ b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
> @@ -19,8 +19,8 @@ then
> exit 1
> fi
>
> -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> -PATH=${KVM}/bin:$PATH; export PATH
> +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> . functions.sh
>
> starttime="`get_starttime`"
> @@ -108,8 +108,8 @@ else
> cat $T/kvm-again.sh.out | tee -a "$oldrun/remote-log"
> exit 2
> fi
> - cp -a "$rundir" "$KVM/res/"
> - oldrun="$KVM/res/$ds"
> + cp -a "$rundir" "$RCUTORTURE/res/"
> + oldrun="$RCUTORTURE/res/$ds"
> fi
> echo | tee -a "$oldrun/remote-log"
> echo " ----" kvm-again.sh output: "(`date`)" | tee -a "$oldrun/remote-log"
> diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
> index 348ad177a5acc..55b2c15332827 100755
> --- a/tools/testing/selftests/rcutorture/bin/kvm.sh
> +++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
> @@ -25,15 +25,15 @@ LANG=en_US.UTF-8; export LANG
>
> dur=$((30*60))
> dryrun=""
> -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> -PATH=${KVM}/bin:$PATH; export PATH
> +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> . functions.sh
>
> TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"
> TORTURE_DEFCONFIG=defconfig
> TORTURE_BOOT_IMAGE=""
> TORTURE_BUILDONLY=
> -TORTURE_INITRD="$KVM/initrd"; export TORTURE_INITRD
> +TORTURE_INITRD="$RCUTORTURE/initrd"; export TORTURE_INITRD
> TORTURE_KCONFIG_ARG=""
> TORTURE_KCONFIG_GDB_ARG=""
> TORTURE_BOOT_GDB_ARG=""
> @@ -262,7 +262,7 @@ else
> exit 1
> fi
>
> -CONFIGFRAG=${KVM}/configs/${TORTURE_SUITE}; export CONFIGFRAG
> +CONFIGFRAG=${RCUTORTURE}/configs/${TORTURE_SUITE}; export CONFIGFRAG
>
> defaultconfigs="`tr '\012' ' ' < $CONFIGFRAG/CFLIST`"
> if test -z "$configs"
> @@ -272,7 +272,7 @@ fi
>
> if test -z "$resdir"
> then
> - resdir=$KVM/res
> + resdir=$RCUTORTURE/res
> fi
>
> # Create a file of test-name/#cpus pairs, sorted by decreasing #cpus.
> @@ -386,7 +386,7 @@ END {
> # Generate a script to execute the tests in appropriate batches.
> cat << ___EOF___ > $T/script
> CONFIGFRAG="$CONFIGFRAG"; export CONFIGFRAG
> -KVM="$KVM"; export KVM
> +RCUTORTURE="$RCUTORTURE"; export RCUTORTURE
> PATH="$PATH"; export PATH
> TORTURE_ALLOTED_CPUS="$TORTURE_ALLOTED_CPUS"; export TORTURE_ALLOTED_CPUS
> TORTURE_BOOT_IMAGE="$TORTURE_BOOT_IMAGE"; export TORTURE_BOOT_IMAGE
> @@ -569,7 +569,7 @@ ___EOF___
> awk < $T/cfgcpu.pack \
> -v TORTURE_BUILDONLY="$TORTURE_BUILDONLY" \
> -v CONFIGDIR="$CONFIGFRAG/" \
> - -v KVM="$KVM" \
> + -v RCUTORTURE="$RCUTORTURE" \
> -v ncpus=$cpus \
> -v jitter="$jitter" \
> -v rd=$resdir/$ds/ \
> diff --git a/tools/testing/selftests/rcutorture/bin/torture.sh b/tools/testing/selftests/rcutorture/bin/torture.sh
> index d1cb60085d8f2..e00e60efb2310 100755
> --- a/tools/testing/selftests/rcutorture/bin/torture.sh
> +++ b/tools/testing/selftests/rcutorture/bin/torture.sh
> @@ -13,8 +13,8 @@
> scriptname=$0
> args="$*"
>
> -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> -PATH=${KVM}/bin:$PATH; export PATH
> +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> . functions.sh
>
> TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"

2022-01-27 08:57:48

by Paul E. McKenney

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

On Thu, Jan 27, 2022 at 08:57:02AM +0800, Zhouyi Zhou wrote:
> Hi Paul
>
> Thank you for your perfect patch!
>
> After apply your patch, I invoke
> "tools/testing/selftests/rcutorture/bin/torture.sh --duration 10" on
> next-20220125. we succeed in allmodconfig!

Very good!

> Following are the test logs (154.223.142.244 is my own internet cloud
> server maintained by myself, I believe it is safe to access the text
> content via web, but not so sure)
>
> 1. torture log
> http://154.223.142.244/logs/20220127/2022.01.26-19.43.04.log (there
> are many Chinese words "Thursday" in it because I forget to set
> terminal locale to English, I am sorry), I excerpt some content here:

Not a problem, Google Translate decided that this was the corresponding
English:

Thursday January 27th, 2022

I really do need to get my browser to do the Chinese characters. My
vim displays them just fine. :-/

> SUCCESSES:
> allmodconfig(0)
> tools/testing/selftests/rcutorture/res/2022.01.26-19.43.04-torture/allmodconfig
> FAILURES:
> rcutorture(2) /home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/res/2022.01.26-19.43.04-torture/results-rcutorture
> Summary: Warnings: 1 Call Traces: 1
> Summary: Warnings: 1 Call Traces: 1
> Summary: Warnings: 1 Call Traces: 1
> Summary: Warnings: 1 Call Traces: 1
> Summary: Warnings: 1 Call Traces: 1
>
> 2. Make.exitcode for allmodeconfig
> http://154.223.142.244/logs/20220127/allmodconfig/Make.exitcode

Zero, which is good. ;-)

> 3. Make.out for allmodeconfig
> http://154.223.142.244/logs/20220127/allmodconfig/Make.out
>
> 4. result of invoking: find 2022.01.26-19.43.04-torture -name
> console.log.diags|xargs -I{} echo "echo {}; cat {}"|xargs -I{} bash -c
> "{}":
> http://154.223.142.244/logs/20220127/consoles.logs.diags
> All of the console.log.diags complain about one thing:
> 2022.01.26-19.43.04-torture/results-refscale-clock/NOPREEMPT/console.log.diags
> [ 0.017721] WARNING: CPU: 0 PID: 0 at
> kernel/trace/trace_events.c:417 trace_event_raw_init+0xea/0x580
> [ 0.017754] Call Trace:
> 2022.01.26-19.43.04-torture/results-rcuscale-kvfree-kasan/TREE/console.log.diags
> [ 0.102911] WARNING: CPU: 0 PID: 0 at
> kernel/trace/trace_events.c:417 trace_event_raw_init+0x113/0x6e0
> [ 0.102992] Call Trace:
> 2022.01.26-19.43.04-torture/results-rcuscale-tasks-tracing-kasan/TREE/console.log.diags
> [ 0.060924] WARNING: CPU: 0 PID: 0 at
> kernel/trace/trace_events.c:417 trace_event_raw_init+0x113/0x6e0
> [ 0.061006] Call Trace:
> I am glad to study these warnings further but they don't seem to be
> related to this patch.

There was at least one RCU bug in this -next, fixed by this commit:

da123016ca8c ("rcu-tasks: Fix computation of CPU-to-list shift counts")

But this warning looks like something else. For one thing, you would
have had to be running on a system with exactly three CPUs to trigger
the RCU bug given the usual rcutorture scenarios.

May I add your Tested-by to this commit?

Thanx, Paul

> Thanks again
> Sincerely
> Zhouyi
>
> On Wed, Jan 26, 2022 at 1:14 PM Paul E. McKenney <[email protected]> wrote:
> >
> > On Tue, Jan 25, 2022 at 08:50:22PM -0800, Paul E. McKenney wrote:
> > > On Wed, Jan 26, 2022 at 12:09:12PM +0800, Zhouyi Zhou wrote:
> > > > Great Discovery !!
> > > > Hi Paul,
> > > > I track the log
> > > > make[2]: *** No rule to make target
> > > > 'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',
> > >
> > > That does look like a very odd pathname, now that you mention it.
> > >
> > > > needed by 'arch/x86/kvm/kvm.o
> > > > into the file virt/kvm/Makefile.kvm:
> > > > 1# SPDX-License-Identifier: GPL-2.0
> > > > 2#
> > > > 3# Makefile for Kernel-based Virtual Machine module
> > > > 4#
> > > > 5
> > > > 6KVM ?= ../../../virt/kvm
> > > > 7
> > > > 8kvm-y := $(KVM)/kvm_main.o $(KVM)/eventfd.o $(KVM)/binary_stats.o
> > > > 9kvm-$(CONFIG_KVM_VFIO) += $(KVM)/vfio.o
> > > > 10kvm-$(CONFIG_KVM_MMIO) += $(KVM)/coalesced_mmio.o
> > > > 11kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o
> > > > 12kvm-$(CONFIG_HAVE_KVM_IRQ_ROUTING) += $(KVM)/irqchip.o
> > > > 13kvm-$(CONFIG_HAVE_KVM_DIRTY_RING) += $(KVM)/dirty_ring.o
> > > > 14kvm-$(CONFIG_HAVE_KVM_PFNCACHE) += $(KVM)/pfncache.o
> > > > line 6's ?= indicates to set the KDIR variable only if it's not
> > > > set/doesn't have a value.
> > > >
> > > > I guess we have set the variable KVM somewhere before make, I will
> > > > continue tracking it.
> > >
> > > But if you build an allmodconfig kernel by hand afterwards, it works?
> > >
> > > I wonder if I need to use a different environment variable in my scripts.
> > > For example, from tools/testing/selftests/rcutorture/bin/kvm.sh:
> > >
> > > KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > > PATH=${KVM}/bin:$PATH; export PATH
> > >
> > > Maybe like the (untested) patch shown below.
> >
> > A somewhat more official version below. Thank you for following up
> > on this!
> >
> > Thanx, Paul
> >
> > ------------------------------------------------------------------------
> >
> > commit 6b5cdbe80b01ce604fdf0f3ef6717a6466aca041
> > Author: Paul E. McKenney <[email protected]>
> > Date: Tue Jan 25 21:08:55 2022 -0800
> >
> > torture: Change KVM environment variable to RCUTORTURE
> >
> > The torture-test scripting's long-standing use of KVM as the environment
> > variable tracking the pathname of the rcutorture directory now conflicts
> > with allmodconfig builds due to the virt/kvm/Makefile.kvm file's use
> > of this as a makefile variable. This commit therefore changes the
> > torture-test scripting from KVM to RCUTORTURE, avoiding the name conflict.
> >
> > Reported-by: Zhouyi Zhou <[email protected]>
> > Signed-off-by: Paul E. McKenney <[email protected]>
> >
> > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-again.sh b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
> > index 5a0023d183dac..0941f1ddab658 100755
> > --- a/tools/testing/selftests/rcutorture/bin/kvm-again.sh
> > +++ b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
> > @@ -47,8 +47,8 @@ else
> > exit 1
> > fi
> >
> > -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > -PATH=${KVM}/bin:$PATH; export PATH
> > +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> > +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> > . functions.sh
> >
> > dryrun=
> > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh b/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
> > index 370406bbfeedc..f17000a2ccf1f 100755
> > --- a/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
> > +++ b/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
> > @@ -49,8 +49,8 @@ fi
> > mkdir $resdir/$ds
> > echo Results directory: $resdir/$ds
> >
> > -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > -PATH=${KVM}/bin:$PATH; export PATH
> > +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> > +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> > . functions.sh
> > echo Using all `identify_qemu_vcpus` CPUs.
> >
> > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
> > index e4a00779b8c69..ee886b40a5d2c 100755
> > --- a/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
> > +++ b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
> > @@ -22,8 +22,8 @@ T=${TMPDIR-/tmp}/kvm-end-run-stats.sh.$$
> > trap 'rm -rf $T' 0
> > mkdir $T
> >
> > -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > -PATH=${KVM}/bin:$PATH; export PATH
> > +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> > +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> > . functions.sh
> > default_starttime="`get_starttime`"
> > starttime="${2-default_starttime}"
> > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
> > index 29b068a55b466..8c4c1e4792d02 100755
> > --- a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
> > +++ b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
> > @@ -19,8 +19,8 @@ then
> > exit 1
> > fi
> >
> > -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > -PATH=${KVM}/bin:$PATH; export PATH
> > +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> > +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> > . functions.sh
> >
> > starttime="`get_starttime`"
> > @@ -108,8 +108,8 @@ else
> > cat $T/kvm-again.sh.out | tee -a "$oldrun/remote-log"
> > exit 2
> > fi
> > - cp -a "$rundir" "$KVM/res/"
> > - oldrun="$KVM/res/$ds"
> > + cp -a "$rundir" "$RCUTORTURE/res/"
> > + oldrun="$RCUTORTURE/res/$ds"
> > fi
> > echo | tee -a "$oldrun/remote-log"
> > echo " ----" kvm-again.sh output: "(`date`)" | tee -a "$oldrun/remote-log"
> > diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
> > index 348ad177a5acc..55b2c15332827 100755
> > --- a/tools/testing/selftests/rcutorture/bin/kvm.sh
> > +++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
> > @@ -25,15 +25,15 @@ LANG=en_US.UTF-8; export LANG
> >
> > dur=$((30*60))
> > dryrun=""
> > -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > -PATH=${KVM}/bin:$PATH; export PATH
> > +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> > +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> > . functions.sh
> >
> > TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"
> > TORTURE_DEFCONFIG=defconfig
> > TORTURE_BOOT_IMAGE=""
> > TORTURE_BUILDONLY=
> > -TORTURE_INITRD="$KVM/initrd"; export TORTURE_INITRD
> > +TORTURE_INITRD="$RCUTORTURE/initrd"; export TORTURE_INITRD
> > TORTURE_KCONFIG_ARG=""
> > TORTURE_KCONFIG_GDB_ARG=""
> > TORTURE_BOOT_GDB_ARG=""
> > @@ -262,7 +262,7 @@ else
> > exit 1
> > fi
> >
> > -CONFIGFRAG=${KVM}/configs/${TORTURE_SUITE}; export CONFIGFRAG
> > +CONFIGFRAG=${RCUTORTURE}/configs/${TORTURE_SUITE}; export CONFIGFRAG
> >
> > defaultconfigs="`tr '\012' ' ' < $CONFIGFRAG/CFLIST`"
> > if test -z "$configs"
> > @@ -272,7 +272,7 @@ fi
> >
> > if test -z "$resdir"
> > then
> > - resdir=$KVM/res
> > + resdir=$RCUTORTURE/res
> > fi
> >
> > # Create a file of test-name/#cpus pairs, sorted by decreasing #cpus.
> > @@ -386,7 +386,7 @@ END {
> > # Generate a script to execute the tests in appropriate batches.
> > cat << ___EOF___ > $T/script
> > CONFIGFRAG="$CONFIGFRAG"; export CONFIGFRAG
> > -KVM="$KVM"; export KVM
> > +RCUTORTURE="$RCUTORTURE"; export RCUTORTURE
> > PATH="$PATH"; export PATH
> > TORTURE_ALLOTED_CPUS="$TORTURE_ALLOTED_CPUS"; export TORTURE_ALLOTED_CPUS
> > TORTURE_BOOT_IMAGE="$TORTURE_BOOT_IMAGE"; export TORTURE_BOOT_IMAGE
> > @@ -569,7 +569,7 @@ ___EOF___
> > awk < $T/cfgcpu.pack \
> > -v TORTURE_BUILDONLY="$TORTURE_BUILDONLY" \
> > -v CONFIGDIR="$CONFIGFRAG/" \
> > - -v KVM="$KVM" \
> > + -v RCUTORTURE="$RCUTORTURE" \
> > -v ncpus=$cpus \
> > -v jitter="$jitter" \
> > -v rd=$resdir/$ds/ \
> > diff --git a/tools/testing/selftests/rcutorture/bin/torture.sh b/tools/testing/selftests/rcutorture/bin/torture.sh
> > index d1cb60085d8f2..e00e60efb2310 100755
> > --- a/tools/testing/selftests/rcutorture/bin/torture.sh
> > +++ b/tools/testing/selftests/rcutorture/bin/torture.sh
> > @@ -13,8 +13,8 @@
> > scriptname=$0
> > args="$*"
> >
> > -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > -PATH=${KVM}/bin:$PATH; export PATH
> > +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> > +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> > . functions.sh
> >
> > TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"

2022-01-27 09:03:32

by Zhouyi Zhou

[permalink] [raw]
Subject: Re: Problems with rcutorture on ppc64le: allmodconfig(2) and other failures

On Thu, Jan 27, 2022 at 9:20 AM Paul E. McKenney <[email protected]> wrote:
>
> On Thu, Jan 27, 2022 at 08:57:02AM +0800, Zhouyi Zhou wrote:
> > Hi Paul
> >
> > Thank you for your perfect patch!
> >
> > After apply your patch, I invoke
> > "tools/testing/selftests/rcutorture/bin/torture.sh --duration 10" on
> > next-20220125. we succeed in allmodconfig!
>
> Very good!
>
> > Following are the test logs (154.223.142.244 is my own internet cloud
> > server maintained by myself, I believe it is safe to access the text
> > content via web, but not so sure)
> >
> > 1. torture log
> > http://154.223.142.244/logs/20220127/2022.01.26-19.43.04.log (there
> > are many Chinese words "Thursday" in it because I forget to set
> > terminal locale to English, I am sorry), I excerpt some content here:
>
> Not a problem, Google Translate decided that this was the corresponding
> English:
>
> Thursday January 27th, 2022
>
> I really do need to get my browser to do the Chinese characters. My
> vim displays them just fine. :-/
Thank Paul for your interest in Chinese culture, to be honest ;-)
>
> > SUCCESSES:
> > allmodconfig(0)
> > tools/testing/selftests/rcutorture/res/2022.01.26-19.43.04-torture/allmodconfig
> > FAILURES:
> > rcutorture(2) /home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/res/2022.01.26-19.43.04-torture/results-rcutorture
> > Summary: Warnings: 1 Call Traces: 1
> > Summary: Warnings: 1 Call Traces: 1
> > Summary: Warnings: 1 Call Traces: 1
> > Summary: Warnings: 1 Call Traces: 1
> > Summary: Warnings: 1 Call Traces: 1
> >
> > 2. Make.exitcode for allmodeconfig
> > http://154.223.142.244/logs/20220127/allmodconfig/Make.exitcode
>
> Zero, which is good. ;-)
glad you like it ;-)
>
> > 3. Make.out for allmodeconfig
> > http://154.223.142.244/logs/20220127/allmodconfig/Make.out
> >
> > 4. result of invoking: find 2022.01.26-19.43.04-torture -name
> > console.log.diags|xargs -I{} echo "echo {}; cat {}"|xargs -I{} bash -c
> > "{}":
> > http://154.223.142.244/logs/20220127/consoles.logs.diags
> > All of the console.log.diags complain about one thing:
> > 2022.01.26-19.43.04-torture/results-refscale-clock/NOPREEMPT/console.log.diags
> > [ 0.017721] WARNING: CPU: 0 PID: 0 at
> > kernel/trace/trace_events.c:417 trace_event_raw_init+0xea/0x580
> > [ 0.017754] Call Trace:
> > 2022.01.26-19.43.04-torture/results-rcuscale-kvfree-kasan/TREE/console.log.diags
> > [ 0.102911] WARNING: CPU: 0 PID: 0 at
> > kernel/trace/trace_events.c:417 trace_event_raw_init+0x113/0x6e0
> > [ 0.102992] Call Trace:
> > 2022.01.26-19.43.04-torture/results-rcuscale-tasks-tracing-kasan/TREE/console.log.diags
> > [ 0.060924] WARNING: CPU: 0 PID: 0 at
> > kernel/trace/trace_events.c:417 trace_event_raw_init+0x113/0x6e0
> > [ 0.061006] Call Trace:
> > I am glad to study these warnings further but they don't seem to be
> > related to this patch.
>
> There was at least one RCU bug in this -next, fixed by this commit:
>
> da123016ca8c ("rcu-tasks: Fix computation of CPU-to-list shift counts")
>
> But this warning looks like something else. For one thing, you would
> have had to be running on a system with exactly three CPUs to trigger
> the RCU bug given the usual rcutorture scenarios.
This give me new orientation to studying!
>
> May I add your Tested-by to this commit?
I feel greatly honored and encouraged!
>
> Thanx, Paul
Thanks again
Sincerely
Zhouyi
>
> > Thanks again
> > Sincerely
> > Zhouyi
> >
> > On Wed, Jan 26, 2022 at 1:14 PM Paul E. McKenney <[email protected]> wrote:
> > >
> > > On Tue, Jan 25, 2022 at 08:50:22PM -0800, Paul E. McKenney wrote:
> > > > On Wed, Jan 26, 2022 at 12:09:12PM +0800, Zhouyi Zhou wrote:
> > > > > Great Discovery !!
> > > > > Hi Paul,
> > > > > I track the log
> > > > > make[2]: *** No rule to make target
> > > > > 'arch/x86/kvm//home/zzy/Program/linux-next/linux-next/tools/testing/selftests/rcutorture/kvm_main.o',
> > > >
> > > > That does look like a very odd pathname, now that you mention it.
> > > >
> > > > > needed by 'arch/x86/kvm/kvm.o
> > > > > into the file virt/kvm/Makefile.kvm:
> > > > > 1# SPDX-License-Identifier: GPL-2.0
> > > > > 2#
> > > > > 3# Makefile for Kernel-based Virtual Machine module
> > > > > 4#
> > > > > 5
> > > > > 6KVM ?= ../../../virt/kvm
> > > > > 7
> > > > > 8kvm-y := $(KVM)/kvm_main.o $(KVM)/eventfd.o $(KVM)/binary_stats.o
> > > > > 9kvm-$(CONFIG_KVM_VFIO) += $(KVM)/vfio.o
> > > > > 10kvm-$(CONFIG_KVM_MMIO) += $(KVM)/coalesced_mmio.o
> > > > > 11kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o
> > > > > 12kvm-$(CONFIG_HAVE_KVM_IRQ_ROUTING) += $(KVM)/irqchip.o
> > > > > 13kvm-$(CONFIG_HAVE_KVM_DIRTY_RING) += $(KVM)/dirty_ring.o
> > > > > 14kvm-$(CONFIG_HAVE_KVM_PFNCACHE) += $(KVM)/pfncache.o
> > > > > line 6's ?= indicates to set the KDIR variable only if it's not
> > > > > set/doesn't have a value.
> > > > >
> > > > > I guess we have set the variable KVM somewhere before make, I will
> > > > > continue tracking it.
> > > >
> > > > But if you build an allmodconfig kernel by hand afterwards, it works?
> > > >
> > > > I wonder if I need to use a different environment variable in my scripts.
> > > > For example, from tools/testing/selftests/rcutorture/bin/kvm.sh:
> > > >
> > > > KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > > > PATH=${KVM}/bin:$PATH; export PATH
> > > >
> > > > Maybe like the (untested) patch shown below.
> > >
> > > A somewhat more official version below. Thank you for following up
> > > on this!
> > >
> > > Thanx, Paul
> > >
> > > ------------------------------------------------------------------------
> > >
> > > commit 6b5cdbe80b01ce604fdf0f3ef6717a6466aca041
> > > Author: Paul E. McKenney <[email protected]>
> > > Date: Tue Jan 25 21:08:55 2022 -0800
> > >
> > > torture: Change KVM environment variable to RCUTORTURE
> > >
> > > The torture-test scripting's long-standing use of KVM as the environment
> > > variable tracking the pathname of the rcutorture directory now conflicts
> > > with allmodconfig builds due to the virt/kvm/Makefile.kvm file's use
> > > of this as a makefile variable. This commit therefore changes the
> > > torture-test scripting from KVM to RCUTORTURE, avoiding the name conflict.
> > >
> > > Reported-by: Zhouyi Zhou <[email protected]>
> > > Signed-off-by: Paul E. McKenney <[email protected]>
> > >
> > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-again.sh b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
> > > index 5a0023d183dac..0941f1ddab658 100755
> > > --- a/tools/testing/selftests/rcutorture/bin/kvm-again.sh
> > > +++ b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
> > > @@ -47,8 +47,8 @@ else
> > > exit 1
> > > fi
> > >
> > > -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > > -PATH=${KVM}/bin:$PATH; export PATH
> > > +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> > > +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> > > . functions.sh
> > >
> > > dryrun=
> > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh b/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
> > > index 370406bbfeedc..f17000a2ccf1f 100755
> > > --- a/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
> > > +++ b/tools/testing/selftests/rcutorture/bin/kvm-check-branches.sh
> > > @@ -49,8 +49,8 @@ fi
> > > mkdir $resdir/$ds
> > > echo Results directory: $resdir/$ds
> > >
> > > -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > > -PATH=${KVM}/bin:$PATH; export PATH
> > > +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> > > +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> > > . functions.sh
> > > echo Using all `identify_qemu_vcpus` CPUs.
> > >
> > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
> > > index e4a00779b8c69..ee886b40a5d2c 100755
> > > --- a/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
> > > +++ b/tools/testing/selftests/rcutorture/bin/kvm-end-run-stats.sh
> > > @@ -22,8 +22,8 @@ T=${TMPDIR-/tmp}/kvm-end-run-stats.sh.$$
> > > trap 'rm -rf $T' 0
> > > mkdir $T
> > >
> > > -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > > -PATH=${KVM}/bin:$PATH; export PATH
> > > +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> > > +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> > > . functions.sh
> > > default_starttime="`get_starttime`"
> > > starttime="${2-default_starttime}"
> > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
> > > index 29b068a55b466..8c4c1e4792d02 100755
> > > --- a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
> > > +++ b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
> > > @@ -19,8 +19,8 @@ then
> > > exit 1
> > > fi
> > >
> > > -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > > -PATH=${KVM}/bin:$PATH; export PATH
> > > +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> > > +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> > > . functions.sh
> > >
> > > starttime="`get_starttime`"
> > > @@ -108,8 +108,8 @@ else
> > > cat $T/kvm-again.sh.out | tee -a "$oldrun/remote-log"
> > > exit 2
> > > fi
> > > - cp -a "$rundir" "$KVM/res/"
> > > - oldrun="$KVM/res/$ds"
> > > + cp -a "$rundir" "$RCUTORTURE/res/"
> > > + oldrun="$RCUTORTURE/res/$ds"
> > > fi
> > > echo | tee -a "$oldrun/remote-log"
> > > echo " ----" kvm-again.sh output: "(`date`)" | tee -a "$oldrun/remote-log"
> > > diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
> > > index 348ad177a5acc..55b2c15332827 100755
> > > --- a/tools/testing/selftests/rcutorture/bin/kvm.sh
> > > +++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
> > > @@ -25,15 +25,15 @@ LANG=en_US.UTF-8; export LANG
> > >
> > > dur=$((30*60))
> > > dryrun=""
> > > -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > > -PATH=${KVM}/bin:$PATH; export PATH
> > > +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> > > +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> > > . functions.sh
> > >
> > > TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"
> > > TORTURE_DEFCONFIG=defconfig
> > > TORTURE_BOOT_IMAGE=""
> > > TORTURE_BUILDONLY=
> > > -TORTURE_INITRD="$KVM/initrd"; export TORTURE_INITRD
> > > +TORTURE_INITRD="$RCUTORTURE/initrd"; export TORTURE_INITRD
> > > TORTURE_KCONFIG_ARG=""
> > > TORTURE_KCONFIG_GDB_ARG=""
> > > TORTURE_BOOT_GDB_ARG=""
> > > @@ -262,7 +262,7 @@ else
> > > exit 1
> > > fi
> > >
> > > -CONFIGFRAG=${KVM}/configs/${TORTURE_SUITE}; export CONFIGFRAG
> > > +CONFIGFRAG=${RCUTORTURE}/configs/${TORTURE_SUITE}; export CONFIGFRAG
> > >
> > > defaultconfigs="`tr '\012' ' ' < $CONFIGFRAG/CFLIST`"
> > > if test -z "$configs"
> > > @@ -272,7 +272,7 @@ fi
> > >
> > > if test -z "$resdir"
> > > then
> > > - resdir=$KVM/res
> > > + resdir=$RCUTORTURE/res
> > > fi
> > >
> > > # Create a file of test-name/#cpus pairs, sorted by decreasing #cpus.
> > > @@ -386,7 +386,7 @@ END {
> > > # Generate a script to execute the tests in appropriate batches.
> > > cat << ___EOF___ > $T/script
> > > CONFIGFRAG="$CONFIGFRAG"; export CONFIGFRAG
> > > -KVM="$KVM"; export KVM
> > > +RCUTORTURE="$RCUTORTURE"; export RCUTORTURE
> > > PATH="$PATH"; export PATH
> > > TORTURE_ALLOTED_CPUS="$TORTURE_ALLOTED_CPUS"; export TORTURE_ALLOTED_CPUS
> > > TORTURE_BOOT_IMAGE="$TORTURE_BOOT_IMAGE"; export TORTURE_BOOT_IMAGE
> > > @@ -569,7 +569,7 @@ ___EOF___
> > > awk < $T/cfgcpu.pack \
> > > -v TORTURE_BUILDONLY="$TORTURE_BUILDONLY" \
> > > -v CONFIGDIR="$CONFIGFRAG/" \
> > > - -v KVM="$KVM" \
> > > + -v RCUTORTURE="$RCUTORTURE" \
> > > -v ncpus=$cpus \
> > > -v jitter="$jitter" \
> > > -v rd=$resdir/$ds/ \
> > > diff --git a/tools/testing/selftests/rcutorture/bin/torture.sh b/tools/testing/selftests/rcutorture/bin/torture.sh
> > > index d1cb60085d8f2..e00e60efb2310 100755
> > > --- a/tools/testing/selftests/rcutorture/bin/torture.sh
> > > +++ b/tools/testing/selftests/rcutorture/bin/torture.sh
> > > @@ -13,8 +13,8 @@
> > > scriptname=$0
> > > args="$*"
> > >
> > > -KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
> > > -PATH=${KVM}/bin:$PATH; export PATH
> > > +RCUTORTURE="`pwd`/tools/testing/selftests/rcutorture"; export RCUTORTURE
> > > +PATH=${RCUTORTURE}/bin:$PATH; export PATH
> > > . functions.sh
> > >
> > > TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"