2019-09-18 00:29:18

by Shuah Khan

[permalink] [raw]
Subject: [GIT PULL] Kselftest update for Linux 5.4-rc1

Hi Linus,

Please pull the following Kselftest update for Linux 5.4-rc1.

This Kselftest update for Linux 5.4-rc1 consists of several fixes to
existing tests and adds KUnit, a lightweight unit testing and mocking
framework for the Linux kernel from Brendan Higgins.

KUnit is not an end-to-end testing framework. It is currently supported
on UML and sub-systems can write unit tests and run them in UML env.
KUnit documentation is included in this update.

In the future KUnit will be linked to Kselftest framework to provide
a way to trigger KUnit tests from user-space.

diff is attached.

Please note that there is conflict between

tools/testing/selftests/tpm2/Makefile

between commit:

3fb2179b0f3553a ("selftests/tpm2: Add the missing TEST_FILES assignment")

from the tpmdd tree and commit:

d04e26067d13f01 ("selftests: tpm2: install python files")

Please take the fix from kselftest tree which is the correct
version for this change.

d04e26067d13f01 ("selftests: tpm2: install python files")

Please let me know if you have any questions and/or concerns.

thanks,
-- Shuah

----------------------------------------------------------------
The following changes since commit d1abaeb3be7b5fa6d7a1fbbd2e14e3310005c4c1:

Linux 5.3-rc5 (2019-08-18 14:31:08 -0700)

are available in the Git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
tags/linux-kselftest-5.4-rc1

for you to fetch changes up to ce54eab71e210f36eebe291e632ed85913c5617e:

kunit: fix failure to build without printk (2019-09-06 09:56:08 -0600)

----------------------------------------------------------------
linux-kselftest-5.4-rc1

This Kselftest update for Linux 5.4-rc1 consists of several fixes to
existing tests and adds KUnit, a lightweight unit testing and mocking
framework for the Linux kernel from Brendan Higgins.

KUnit is not an end-to-end testing framework. It is currently supported
on UML and sub-systems can write unit tests and run them in UML env.
KUnit documentation is included in this update.

In the future KUnit will be linked to Kselftest framework to provide
a way to trigger KUnit tests from user-space.

----------------------------------------------------------------
Anders Roxell (2):
selftests: livepatch: add missing fragments to config
selftests: tpm2: install python files

Avinash Kondareddy (1):
kunit: test: add tests for KUnit managed resources

Brendan Higgins (16):
kunit: test: add KUnit test runner core
kunit: test: add test resource management API
kunit: test: add string_stream a std::stream like string builder
kunit: test: add assertion printing library
kunit: test: add the concept of expectations
kbuild: enable building KUnit
kunit: test: add initial tests
objtool: add kunit_try_catch_throw to the noreturn list
kunit: test: add support for test abort
kunit: test: add tests for kunit test abort
kunit: test: add the concept of assertions
kunit: defconfig: add defconfigs for building KUnit tests
Documentation: kunit: add documentation for KUnit
MAINTAINERS: add entry for KUnit the unit testing framework
MAINTAINERS: add proc sysctl KUnit test to PROC SYSCTL section
kunit: fix failure to build without printk

Felix Guo (1):
kunit: tool: add Python wrappers for running KUnit tests

George G. Davis (2):
selftests: watchdog: Add optional file argument
selftests: watchdog: cleanup whitespace in usage options

Ilya Leoshkevich (1):
selftests: use "$(MAKE)" instead of "make"

Iurii Zaikin (1):
kernel/sysctl-test: Add null pointer test for
sysctl.c:proc_dointvec()

Masanari Iida (1):
selftest/ftrace: Fix typo in trigger-snapshot.tc

Tycho Andersen (1):
selftests/seccomp: fix build on older kernels

Documentation/dev-tools/index.rst | 1 +
Documentation/dev-tools/kunit/api/index.rst | 16 +
Documentation/dev-tools/kunit/api/test.rst | 11 +
Documentation/dev-tools/kunit/faq.rst | 62 +
Documentation/dev-tools/kunit/index.rst | 79 ++
Documentation/dev-tools/kunit/start.rst | 180 +++
Documentation/dev-tools/kunit/usage.rst | 576 ++++++++
Kconfig | 2 +
MAINTAINERS | 13 +
Makefile | 2 +
arch/um/configs/kunit_defconfig | 3 +
include/kunit/assert.h | 356 +++++
include/kunit/string-stream.h | 51 +
include/kunit/test.h | 1490
++++++++++++++++++++
include/kunit/try-catch.h | 75 +
kernel/Makefile | 2 +
kernel/sysctl-test.c | 392 +++++
kunit/Kconfig | 38 +
kunit/Makefile | 9 +
kunit/assert.c | 141 ++
kunit/example-test.c | 88 ++
kunit/string-stream-test.c | 52 +
kunit/string-stream.c | 217 +++
kunit/test-test.c | 331 +++++
kunit/test.c | 478 +++++++
kunit/try-catch.c | 118 ++
lib/Kconfig.debug | 11 +
tools/objtool/check.c | 1 +
tools/testing/kunit/.gitignore | 3 +
tools/testing/kunit/configs/all_tests.config | 3 +
tools/testing/kunit/kunit.py | 136 ++
tools/testing/kunit/kunit_config.py | 66 +
tools/testing/kunit/kunit_kernel.py | 149 ++
tools/testing/kunit/kunit_parser.py | 310 ++++
tools/testing/kunit/kunit_tool_test.py | 206 +++
.../test_data/test_is_test_passed-all_passed.log | 32 +
.../kunit/test_data/test_is_test_passed-crash.log | 69 +
.../test_data/test_is_test_passed-failure.log | 36 +
.../test_data/test_is_test_passed-no_tests_run.log | 75 +
.../test_data/test_output_isolated_correctly.log | 106 ++
.../kunit/test_data/test_read_from_file.kconfig | 17 +
tools/testing/selftests/Makefile | 22 +-
.../ftrace/test.d/trigger/trigger-snapshot.tc | 2 +-
tools/testing/selftests/livepatch/config | 2 +
tools/testing/selftests/seccomp/seccomp_bpf.c | 5 +
tools/testing/selftests/tpm2/Makefile | 1 +
tools/testing/selftests/watchdog/watchdog-test.c | 41 +-
47 files changed, 6051 insertions(+), 25 deletions(-)
create mode 100644 Documentation/dev-tools/kunit/api/index.rst
create mode 100644 Documentation/dev-tools/kunit/api/test.rst
create mode 100644 Documentation/dev-tools/kunit/faq.rst
create mode 100644 Documentation/dev-tools/kunit/index.rst
create mode 100644 Documentation/dev-tools/kunit/start.rst
create mode 100644 Documentation/dev-tools/kunit/usage.rst
create mode 100644 arch/um/configs/kunit_defconfig
create mode 100644 include/kunit/assert.h
create mode 100644 include/kunit/string-stream.h
create mode 100644 include/kunit/test.h
create mode 100644 include/kunit/try-catch.h
create mode 100644 kernel/sysctl-test.c
create mode 100644 kunit/Kconfig
create mode 100644 kunit/Makefile
create mode 100644 kunit/assert.c
create mode 100644 kunit/example-test.c
create mode 100644 kunit/string-stream-test.c
create mode 100644 kunit/string-stream.c
create mode 100644 kunit/test-test.c
create mode 100644 kunit/test.c
create mode 100644 kunit/try-catch.c
create mode 100644 tools/testing/kunit/.gitignore
create mode 100644 tools/testing/kunit/configs/all_tests.config
create mode 100755 tools/testing/kunit/kunit.py
create mode 100644 tools/testing/kunit/kunit_config.py
create mode 100644 tools/testing/kunit/kunit_kernel.py
create mode 100644 tools/testing/kunit/kunit_parser.py
create mode 100755 tools/testing/kunit/kunit_tool_test.py
create mode 100644
tools/testing/kunit/test_data/test_is_test_passed-all_passed.log
create mode 100644
tools/testing/kunit/test_data/test_is_test_passed-crash.log
create mode 100644
tools/testing/kunit/test_data/test_is_test_passed-failure.log
create mode 100644
tools/testing/kunit/test_data/test_is_test_passed-no_tests_run.log
create mode 100644
tools/testing/kunit/test_data/test_output_isolated_correctly.log
create mode 100644
tools/testing/kunit/test_data/test_read_from_file.kconfig
----------------------------------------------------------------


Attachments:
linux-kselftest-5.4-rc1.diff (218.58 kB)

2019-09-22 18:53:30

by Linus Torvalds

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On Tue, Sep 17, 2019 at 12:26 PM Shuah Khan <[email protected]> wrote:
>
> This Kselftest update for Linux 5.4-rc1 consists of several fixes to
> existing tests and adds KUnit, a lightweight unit testing and mocking
> framework for the Linux kernel from Brendan Higgins.

So I pulled this, but then I almost immediately unpulled it.

My reason for doing that may be odd, but it's because of the top-level
'kunit' directory. This shouldn't be on the top level.

The reason I react so strongly is that it actually breaks my finger
memory. I don't type out filenames - I auto-compete them. So "kernel/"
is "k<tab>", "drivers/" is "d<tab>" etc.

It already doesn't work for everything ("mm/" is actually "mm<tab>"
not because we have files in the git tree, but because the build
creates various "module" files), but this breaks a common pattern for
me.

> In the future KUnit will be linked to Kselftest framework to provide
> a way to trigger KUnit tests from user-space.

Can the kernel parts please move to lib/kunit/ or something like that?

Linus

2019-09-22 18:56:43

by Linus Torvalds

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On Fri, Sep 20, 2019 at 9:35 AM Brendan Higgins
<[email protected]> wrote:
>
> Sorry about that. I am surprised that none of the other reviewers
> brought this up.

I think I'm "special".

There was some other similar change a few years ago, which I
absolutely hated because of how it broke autocomplete for me. Very few
other people seemed to react to it.

Part of it may be that the kernel is almost the _only_ project I work
with, so unlike a lot of other developers, I end up having muscle
memory for kernel-specific issues.

Auto-completion was also one of the (many) reasons why I hated CVS -
having that annoying "CVS" directory there just always annoyed me.
There's a reason why git uses a dot-file.

So I just have issues that perhaps other people don't react to as
much. And aggressive tab-completion happens to be a thing for me.

Linus

2019-09-22 18:58:10

by Brendan Higgins

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On Fri, Sep 20, 2019 at 9:27 AM Shuah Khan <[email protected]> wrote:
>
> Hi Linus,
>
> On Fri, Sep 20, 2019, 10:18 AM Linus Torvalds <[email protected]> wrote:
>>
>> On Tue, Sep 17, 2019 at 12:26 PM Shuah Khan <[email protected]> wrote:
>> >
>> > This Kselftest update for Linux 5.4-rc1 consists of several fixes to
>> > existing tests and adds KUnit, a lightweight unit testing and mocking
>> > framework for the Linux kernel from Brendan Higgins.
>>
>> So I pulled this, but then I almost immediately unpulled it.
>>
>> My reason for doing that may be odd, but it's because of the top-level
>> 'kunit' directory. This shouldn't be on the top level.
>>
>> The reason I react so strongly is that it actually breaks my finger
>> memory. I don't type out filenames - I auto-compete them. So "kernel/"
>> is "k<tab>", "drivers/" is "d<tab>" etc.
>>
>> It already doesn't work for everything ("mm/" is actually "mm<tab>"
>> not because we have files in the git tree, but because the build
>> creates various "module" files), but this breaks a common pattern for
>> me.

Sorry about that. I am surprised that none of the other reviewers
brought this up.

> On hindsight, I probably should have run this by you to get your feedback.
>
>> > In the future KUnit will be linked to Kselftest framework to provide
>> > a way to trigger KUnit tests from user-space.
>>
>> Can the kernel parts please move to lib/kunit/ or something like that.

I'm fine with lib/kunit/.

> I will work with Brendan and come up with a plan and send another request early next week.

Cheers

2019-09-22 19:00:01

by Shuah Khan

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On 9/20/19 10:51 AM, Linus Torvalds wrote:
> On Fri, Sep 20, 2019 at 9:35 AM Brendan Higgins
> <[email protected]> wrote:
>>
>> Sorry about that. I am surprised that none of the other reviewers
>> brought this up.
>
> I think I'm "special".
>
> There was some other similar change a few years ago, which I
> absolutely hated because of how it broke autocomplete for me. Very few
> other people seemed to react to it.
>
> Part of it may be that the kernel is almost the _only_ project I work
> with, so unlike a lot of other developers, I end up having muscle
> memory for kernel-specific issues.
>
> Auto-completion was also one of the (many) reasons why I hated CVS -
> having that annoying "CVS" directory there just always annoyed me.
> There's a reason why git uses a dot-file.
>
> So I just have issues that perhaps other people don't react to as
> much. And aggressive tab-completion happens to be a thing for me.
>

Thanks for explaining. Brendan and I will get this sorted out.

Looks like my previous response didn't make it to the kselftest
and kernel lists.

thanks,
-- Shuah

2019-09-23 06:45:24

by Randy Dunlap

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On 9/20/19 9:17 AM, Linus Torvalds wrote:
> On Tue, Sep 17, 2019 at 12:26 PM Shuah Khan <[email protected]> wrote:
>>
>> This Kselftest update for Linux 5.4-rc1 consists of several fixes to
>> existing tests and adds KUnit, a lightweight unit testing and mocking
>> framework for the Linux kernel from Brendan Higgins.
>
> So I pulled this, but then I almost immediately unpulled it.
>
> My reason for doing that may be odd, but it's because of the top-level
> 'kunit' directory. This shouldn't be on the top level.
>
> The reason I react so strongly is that it actually breaks my finger
> memory. I don't type out filenames - I auto-compete them. So "kernel/"
> is "k<tab>", "drivers/" is "d<tab>" etc.
>
> It already doesn't work for everything ("mm/" is actually "mm<tab>"
> not because we have files in the git tree, but because the build
> creates various "module" files), but this breaks a common pattern for
> me.
>
>> In the future KUnit will be linked to Kselftest framework to provide
>> a way to trigger KUnit tests from user-space.
>
> Can the kernel parts please move to lib/kunit/ or something like that?

Please also move the top-level Kconfig menu item "KUnit support" to
somewhere that is not top-level. Maybe also in the lib/ menu. Maybe
in lib/Kconfig.debug.


--
~Randy

2019-09-23 08:28:39

by Brendan Higgins

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On Fri, Sep 20, 2019 at 9:51 AM Linus Torvalds
<[email protected]> wrote:
>
> On Fri, Sep 20, 2019 at 9:35 AM Brendan Higgins
> <[email protected]> wrote:
> >
> > Sorry about that. I am surprised that none of the other reviewers
> > brought this up.
>
> I think I'm "special".

Heh.

> There was some other similar change a few years ago, which I
> absolutely hated because of how it broke autocomplete for me. Very few
> other people seemed to react to it.

Well, it's good to know I'm not the first. :-)

> Part of it may be that the kernel is almost the _only_ project I work
> with, so unlike a lot of other developers, I end up having muscle
> memory for kernel-specific issues.
>
> Auto-completion was also one of the (many) reasons why I hated CVS -
> having that annoying "CVS" directory there just always annoyed me.
> There's a reason why git uses a dot-file.

Yuck. I have never used CVS myself, but the dot-file approach seems
much more natural to me. Then again, I have been using git pretty much
since I first started programming, so it's hard to say that I am not
biased.

> So I just have issues that perhaps other people don't react to as
> much. And aggressive tab-completion happens to be a thing for me.

Fair enough. On that note, are you okay with the `include/kunit/`
directory, or do you want me to move it to `include/linux/kunit`?

2019-09-23 08:29:02

by Brendan Higgins

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On Fri, Sep 20, 2019 at 11:15 AM Linus Torvalds
<[email protected]> wrote:
>
> On Fri, Sep 20, 2019 at 11:03 AM Brendan Higgins
> <[email protected]> wrote:
> >
> > Fair enough. On that note, are you okay with the `include/kunit/`
> > directory, or do you want me to move it to `include/linux/kunit`?
>
> "include/kunit" should work just fine for me. At least I didn't react
> to it immediately when I had done my test-pull, and it doesn't change
> any auto-completion patterns for me either.

Sounds good to me.

Thanks!

2019-09-23 09:50:09

by Linus Torvalds

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On Fri, Sep 20, 2019 at 11:03 AM Brendan Higgins
<[email protected]> wrote:
>
> Fair enough. On that note, are you okay with the `include/kunit/`
> directory, or do you want me to move it to `include/linux/kunit`?

"include/kunit" should work just fine for me. At least I didn't react
to it immediately when I had done my test-pull, and it doesn't change
any auto-completion patterns for me either.

[ We already have two 'k' names under include, but even if that wasn't
true, I don't type those names anyway so I wouldn't have had
muscle-memory for those two directories in the first place.

Under include, it's "linux" (and to a smaller extent "asm-generic")
that I autocomplete. ]

Linus

2019-09-23 19:30:36

by Ingo Molnar

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1


* Linus Torvalds <[email protected]> wrote:

> On Fri, Sep 20, 2019 at 9:35 AM Brendan Higgins
> <[email protected]> wrote:
> >
> > Sorry about that. I am surprised that none of the other reviewers
> > brought this up.
>
> I think I'm "special".
>
> There was some other similar change a few years ago, which I
> absolutely hated because of how it broke autocomplete for me. Very few
> other people seemed to react to it.

FWIW, I am obsessively sensitive to autocomplete and overall source code
file hieararchy and nomenclature details as well, so it's not just you.

Beyond the muscle memory aspect, nonsensical naming and inanely flat file
hierarchies annoy kernel developers and makes it harder for newbies to
understand the kernel source as well.

The less clutter, the more organization, the better - and there's very
few valid technical reasons to add any new files or directories to the
top level directory - we should probably *remove* quite a few.

For example 'firmware/' was recently moved to drivers/firmware/, and in a
similar fashion about a third of the remaining 22 directories should
probably be moved too:

drwxr-xr-x arch
drwxr-xr-x block
drwxr-xr-x certs # move to build/certs/ dir
drwxr-xr-x crypto # move to kernel/crypto/ or security/crypto/
drwxr-xr-x Documentation
drwxr-xr-x drivers
drwxr-xr-x fs
drwxr-xr-x include
drwxr-xr-x init
drwxr-xr-x ipc # move to kernel/ipc/
drwxr-xr-x kernel
drwxr-xr-x lib
drwxr-xr-x LICENSES
drwxr-xr-x mm
drwxr-xr-x net
drwxr-xr-x samples # move to Documentation/samples/
drwxr-xr-x scripts # move to build/scripts/
drwxr-xr-x security
drwxr-xr-x sound # move to drivers/sound/
drwxr-xr-x tools
drwxr-xr-x usr # move to build/usr/
drwxr-xr-x virt # move to the already existing drivers/virt/

-rw-r--r-- COPYING
-rw-r--r-- CREDITS
-rw-r--r-- Kbuild
-rw-r--r-- Kconfig
-rw-r--r-- MAINTAINERS
-rw-r--r-- Makefile
-rw-r--r-- README

There's a few borderline ones:

- 'block' could in principle move to drivers/block/core/ but it's fine
at the top level too I think.

- 'init' could in principle be moved to kernel/init/ - but it's not
wrong at the top level either.

The remaining top level hierarchy would look pretty sweet and short:

drwxr-xr-x arch
drwxr-xr-x block
drwxr-xr-x build # new
drwxr-xr-x Documentation
drwxr-xr-x drivers
drwxr-xr-x fs
drwxr-xr-x include
drwxr-xr-x init
drwxr-xr-x kernel
drwxr-xr-x lib
drwxr-xr-x LICENSES
drwxr-xr-x mm
drwxr-xr-x net
drwxr-xr-x security
drwxr-xr-x tools

-rw-r--r-- COPYING
-rw-r--r-- CREDITS
-rw-r--r-- Kbuild
-rw-r--r-- Kconfig
-rw-r--r-- MAINTAINERS
-rw-r--r-- Makefile
-rw-r--r-- README

I'm volunteering to do this (in a scripted, repeatable, reviewable,
tweakable and "easy to execute in a quiet moment" fashion), although
I also expect you to balk at the churn. :-)

Thanks,

Ingo

2019-09-23 19:49:08

by Alexey Dobriyan

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

> Beyond the muscle memory aspect, nonsensical naming and inanely flat file
> hierarchies annoy kernel developers

Flat(ish) file hierarchies are good -- less typing.
If you're copy-pasting then it doesn't matter much (it still matters a
little because long filename occupy more space on the screen and in logs).

> makes it harder for newbies to understand the kernel source as well.

That's fine too.

> drwxr-xr-x crypto # move to kernel/crypto/ or security/crypto/

No, crypto/ is fine.

If everything arch independent should live at kernel/ then why should kernel/
exist at all? It should be trimmed and everything moved to the top level
directory (OK, I'm not really suggesting that).

> drwxr-xr-x ipc # move to kernel/ipc/

No, same reason. It was there since time immemorial.

> drwxr-xr-x samples # move to Documentation/samples/

Just delete it. Best API usage samples are in modern parts of main tree,
actively maintained/updated.

> drwxr-xr-x scripts # move to build/scripts/

eh

> drwxr-xr-x sound # move to drivers/sound/

NO! it has hw independent part and pretty big one.

> drwxr-xr-x tools

If tools/ people could somewhow stop duplicating large parts of include/linux
and arch/x86/include/asm it will be very much appreciated.

> - 'block' could in principle move to drivers/block/core/ but it's fine
> at the top level too I think.

It is fine indeed. Short name, top level dir, arch and hw independent
code -- it is perfect.

> I'm volunteering to do this (in a scripted, repeatable, reviewable,
> tweakable and "easy to execute in a quiet moment" fashion), although
> I also expect you to balk at the churn. :-)

Can I pay you $100 to not do this ever?

In Russia we say "what has grown has grown" and it is not like Linux is
perfect example of intelligent design.

2019-09-23 20:15:05

by Greg Kroah-Hartman

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On Sun, Sep 22, 2019 at 01:25:55PM +0200, Ingo Molnar wrote:
>
> * Linus Torvalds <[email protected]> wrote:
>
> > On Fri, Sep 20, 2019 at 9:35 AM Brendan Higgins
> > <[email protected]> wrote:
> > >
> > > Sorry about that. I am surprised that none of the other reviewers
> > > brought this up.
> >
> > I think I'm "special".
> >
> > There was some other similar change a few years ago, which I
> > absolutely hated because of how it broke autocomplete for me. Very few
> > other people seemed to react to it.
>
> FWIW, I am obsessively sensitive to autocomplete and overall source code
> file hieararchy and nomenclature details as well, so it's not just you.
>
> Beyond the muscle memory aspect, nonsensical naming and inanely flat file
> hierarchies annoy kernel developers and makes it harder for newbies to
> understand the kernel source as well.
>
> The less clutter, the more organization, the better - and there's very
> few valid technical reasons to add any new files or directories to the
> top level directory - we should probably *remove* quite a few.
>
> For example 'firmware/' was recently moved to drivers/firmware/, and in a
> similar fashion about a third of the remaining 22 directories should
> probably be moved too:
>
> drwxr-xr-x arch
> drwxr-xr-x block
> drwxr-xr-x certs # move to build/certs/ dir
> drwxr-xr-x crypto # move to kernel/crypto/ or security/crypto/
> drwxr-xr-x Documentation
> drwxr-xr-x drivers
> drwxr-xr-x fs
> drwxr-xr-x include
> drwxr-xr-x init
> drwxr-xr-x ipc # move to kernel/ipc/
> drwxr-xr-x kernel
> drwxr-xr-x lib
> drwxr-xr-x LICENSES
> drwxr-xr-x mm
> drwxr-xr-x net
> drwxr-xr-x samples # move to Documentation/samples/
> drwxr-xr-x scripts # move to build/scripts/
> drwxr-xr-x security
> drwxr-xr-x sound # move to drivers/sound/
> drwxr-xr-x tools
> drwxr-xr-x usr # move to build/usr/
> drwxr-xr-x virt # move to the already existing drivers/virt/
>
> -rw-r--r-- COPYING
> -rw-r--r-- CREDITS
> -rw-r--r-- Kbuild
> -rw-r--r-- Kconfig
> -rw-r--r-- MAINTAINERS
> -rw-r--r-- Makefile
> -rw-r--r-- README
>
> There's a few borderline ones:
>
> - 'block' could in principle move to drivers/block/core/ but it's fine
> at the top level too I think.
>
> - 'init' could in principle be moved to kernel/init/ - but it's not
> wrong at the top level either.
>
> The remaining top level hierarchy would look pretty sweet and short:
>
> drwxr-xr-x arch
> drwxr-xr-x block
> drwxr-xr-x build # new
> drwxr-xr-x Documentation
> drwxr-xr-x drivers
> drwxr-xr-x fs
> drwxr-xr-x include
> drwxr-xr-x init
> drwxr-xr-x kernel
> drwxr-xr-x lib
> drwxr-xr-x LICENSES
> drwxr-xr-x mm
> drwxr-xr-x net
> drwxr-xr-x security
> drwxr-xr-x tools
>
> -rw-r--r-- COPYING
> -rw-r--r-- CREDITS
> -rw-r--r-- Kbuild
> -rw-r--r-- Kconfig
> -rw-r--r-- MAINTAINERS
> -rw-r--r-- Makefile
> -rw-r--r-- README
>
> I'm volunteering to do this (in a scripted, repeatable, reviewable,
> tweakable and "easy to execute in a quiet moment" fashion), although
> I also expect you to balk at the churn. :-)

I for one would love the above changes. And I'm the one that has to
deal with all of the backporting issues that arise with stable backports :)

thanks,

greg k-h

2019-09-25 14:50:12

by Shuah Khan

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On 9/22/19 5:52 AM, Greg KH wrote:
> On Sun, Sep 22, 2019 at 01:25:55PM +0200, Ingo Molnar wrote:
>>
>> * Linus Torvalds <[email protected]> wrote:
>>
>>> On Fri, Sep 20, 2019 at 9:35 AM Brendan Higgins
>>> <[email protected]> wrote:
>>>>
>>>> Sorry about that. I am surprised that none of the other reviewers
>>>> brought this up.
>>>
>>> I think I'm "special".
>>>
>>> There was some other similar change a few years ago, which I
>>> absolutely hated because of how it broke autocomplete for me. Very few
>>> other people seemed to react to it.
>>
>> FWIW, I am obsessively sensitive to autocomplete and overall source code
>> file hieararchy and nomenclature details as well, so it's not just you.
>>
>> Beyond the muscle memory aspect, nonsensical naming and inanely flat file
>> hierarchies annoy kernel developers and makes it harder for newbies to
>> understand the kernel source as well.
>>
>> The less clutter, the more organization, the better - and there's very
>> few valid technical reasons to add any new files or directories to the
>> top level directory - we should probably *remove* quite a few.
>>
>> For example 'firmware/' was recently moved to drivers/firmware/, and in a
>> similar fashion about a third of the remaining 22 directories should
>> probably be moved too:
>>
>> drwxr-xr-x arch
>> drwxr-xr-x block
>> drwxr-xr-x certs # move to build/certs/ dir
>> drwxr-xr-x crypto # move to kernel/crypto/ or security/crypto/
>> drwxr-xr-x Documentation
>> drwxr-xr-x drivers
>> drwxr-xr-x fs
>> drwxr-xr-x include
>> drwxr-xr-x init
>> drwxr-xr-x ipc # move to kernel/ipc/
>> drwxr-xr-x kernel
>> drwxr-xr-x lib
>> drwxr-xr-x LICENSES
>> drwxr-xr-x mm
>> drwxr-xr-x net
>> drwxr-xr-x samples # move to Documentation/samples/
>> drwxr-xr-x scripts # move to build/scripts/
>> drwxr-xr-x security
>> drwxr-xr-x sound # move to drivers/sound/
>> drwxr-xr-x tools
>> drwxr-xr-x usr # move to build/usr/
>> drwxr-xr-x virt # move to the already existing drivers/virt/
>>
>> -rw-r--r-- COPYING
>> -rw-r--r-- CREDITS
>> -rw-r--r-- Kbuild
>> -rw-r--r-- Kconfig
>> -rw-r--r-- MAINTAINERS
>> -rw-r--r-- Makefile
>> -rw-r--r-- README
>>
>> There's a few borderline ones:
>>
>> - 'block' could in principle move to drivers/block/core/ but it's fine
>> at the top level too I think.
>>
>> - 'init' could in principle be moved to kernel/init/ - but it's not
>> wrong at the top level either.
>>
>> The remaining top level hierarchy would look pretty sweet and short:
>>
>> drwxr-xr-x arch
>> drwxr-xr-x block
>> drwxr-xr-x build # new
>> drwxr-xr-x Documentation
>> drwxr-xr-x drivers
>> drwxr-xr-x fs
>> drwxr-xr-x include
>> drwxr-xr-x init
>> drwxr-xr-x kernel
>> drwxr-xr-x lib
>> drwxr-xr-x LICENSES
>> drwxr-xr-x mm
>> drwxr-xr-x net
>> drwxr-xr-x security
>> drwxr-xr-x tools
>>
>> -rw-r--r-- COPYING
>> -rw-r--r-- CREDITS
>> -rw-r--r-- Kbuild
>> -rw-r--r-- Kconfig
>> -rw-r--r-- MAINTAINERS
>> -rw-r--r-- Makefile
>> -rw-r--r-- README
>>
>> I'm volunteering to do this (in a scripted, repeatable, reviewable,
>> tweakable and "easy to execute in a quiet moment" fashion), although
>> I also expect you to balk at the churn. :-)
>
> I for one would love the above changes. And I'm the one that has to
> deal with all of the backporting issues that arise with stable backports :)
>

I am exploring the possibility to move selftests to a better location
or add a git alias so it can be found easily. With the addition of KUnit
and future work that is planned to connect kselftest and KUnit, it would
make sense have selftests to be in a location that is better suited than
where it currently resides.

I have been getting feedback from some developers that they would like
to see selftests more visible and easier to find.

There are some dependencies (unintended, shouldn't exist) between some
tests and content under tools that might pose some logistical problems,
in addition to the churn of backporting.

I haven't explored "git alias" yet though. Since this topic of moving
came up, I would liek to get feedback on selftests location in general
and where would be a good place for it.

thanks,
-- Shuah

2019-09-25 23:21:11

by Ingo Molnar

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1


* Shuah Khan <[email protected]> wrote:

> I am exploring the possibility to move selftests to a better location
> or add a git alias so it can be found easily. With the addition of
> KUnit and future work that is planned to connect kselftest and KUnit,
> it would make sense have selftests to be in a location that is better
> suited than where it currently resides.
>
> I have been getting feedback from some developers that they would like
> to see selftests more visible and easier to find.
>
> There are some dependencies (unintended, shouldn't exist) between some
> tests and content under tools that might pose some logistical problems,
> in addition to the churn of backporting.
>
> I haven't explored "git alias" yet though. Since this topic of moving
> came up, I would liek to get feedback on selftests location in general
> and where would be a good place for it.

I'm not sure about the Git alias thing - but I do agree that
tools/testing/selftests is a pretty obscure location given the importance
of kernel unit tests - and I think it could be moved one level higher, to
tools/selftests? The "selftest" name already implies the "test" aspect
after all.

Thanks,

Ingo

2019-09-25 23:26:56

by Randy Dunlap

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On 9/23/19 12:43 PM, Ingo Molnar wrote:
>
> * Shuah Khan <[email protected]> wrote:
>
>> I am exploring the possibility to move selftests to a better location
>> or add a git alias so it can be found easily. With the addition of
>> KUnit and future work that is planned to connect kselftest and KUnit,
>> it would make sense have selftests to be in a location that is better
>> suited than where it currently resides.
>>
>> I have been getting feedback from some developers that they would like
>> to see selftests more visible and easier to find.
>>
>> There are some dependencies (unintended, shouldn't exist) between some
>> tests and content under tools that might pose some logistical problems,
>> in addition to the churn of backporting.
>>
>> I haven't explored "git alias" yet though. Since this topic of moving
>> came up, I would liek to get feedback on selftests location in general
>> and where would be a good place for it.
>
> I'm not sure about the Git alias thing - but I do agree that
> tools/testing/selftests is a pretty obscure location given the importance
> of kernel unit tests - and I think it could be moved one level higher, to
> tools/selftests? The "selftest" name already implies the "test" aspect
> after all.


Without trying to use too much paint, I would move testing/ to a top-level
dir, outside of tools/, and leave selftest/ under testing/.

--
~Randy

2019-09-25 23:44:37

by Shuah Khan

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On 9/23/19 1:52 PM, Randy Dunlap wrote:
> On 9/23/19 12:43 PM, Ingo Molnar wrote:
>>
>> * Shuah Khan <[email protected]> wrote:
>>
>>> I am exploring the possibility to move selftests to a better location
>>> or add a git alias so it can be found easily. With the addition of
>>> KUnit and future work that is planned to connect kselftest and KUnit,
>>> it would make sense have selftests to be in a location that is better
>>> suited than where it currently resides.
>>>
>>> I have been getting feedback from some developers that they would like
>>> to see selftests more visible and easier to find.
>>>
>>> There are some dependencies (unintended, shouldn't exist) between some
>>> tests and content under tools that might pose some logistical problems,
>>> in addition to the churn of backporting.
>>>
>>> I haven't explored "git alias" yet though. Since this topic of moving
>>> came up, I would liek to get feedback on selftests location in general
>>> and where would be a good place for it.
>>
>> I'm not sure about the Git alias thing - but I do agree that
>> tools/testing/selftests is a pretty obscure location given the importance
>> of kernel unit tests - and I think it could be moved one level higher, to
>> tools/selftests? The "selftest" name already implies the "test" aspect
>> after all.
>
Right. Obscure location given the importance is the problem.

>
> Without trying to use too much paint, I would move testing/ to a top-level
> dir, outside of tools/, and leave selftest/ under testing/.
>

Right. What you suggesting is very similar to and more complete than
what I have been thinking about and proposed at the KS kselftest track.


i.e move tools/testing/selftests to kselftest at the root level. I like
your idea of moving tools/testing up to root and keep selftests under
it.

If we are good with this kind of change, I would like to get this done
sooner than later. There is some back-porting churn to worry about.

thanks,
-- Shuah

2019-09-25 23:52:56

by Shuah Khan

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On 9/23/19 2:53 PM, Ingo Molnar wrote:
>
> * Shuah Khan <[email protected]> wrote:
>
>> Right. What you suggesting is very similar to and more complete than
>> what I have been thinking about and proposed at the KS kselftest track.
>>
>> i.e move tools/testing/selftests to kselftest at the root level. I like
>> your idea of moving tools/testing up to root and keep selftests under
>> it.
>>
>> If we are good with this kind of change, I would like to get this done
>> sooner than later. There is some back-porting churn to worry about.
>
> I think the movement I suggested would be sufficient:
>
> tools/testing/selftests/ => tools/selftests/
>
> I.e. let's not clutter up the top level directory.
>

Yeah good point.

-- Shuah

2019-09-26 00:37:37

by Bird, Tim

[permalink] [raw]
Subject: RE: [GIT PULL] Kselftest update for Linux 5.4-rc1

> -----Original Message-----
> From: Ingo Molnar on Sunday, September 22, 2019 1:26 AM
>
> * Linus Torvalds <[email protected]> wrote:
>
> > On Fri, Sep 20, 2019 at 9:35 AM Brendan Higgins
> > <[email protected]> wrote:
> > >
> > > Sorry about that. I am surprised that none of the other reviewers
> > > brought this up.
> >
> > I think I'm "special".
> >
> > There was some other similar change a few years ago, which I
> > absolutely hated because of how it broke autocomplete for me. Very few
> > other people seemed to react to it.
>
> FWIW, I am obsessively sensitive to autocomplete and overall source code
> file hieararchy and nomenclature details as well, so it's not just you.
>
> Beyond the muscle memory aspect, nonsensical naming and inanely flat file
> hierarchies annoy kernel developers and makes it harder for newbies to
> understand the kernel source as well.
>
> The less clutter, the more organization, the better - and there's very
> few valid technical reasons to add any new files or directories to the
> top level directory - we should probably *remove* quite a few.
>
> For example 'firmware/' was recently moved to drivers/firmware/, and in a
> similar fashion about a third of the remaining 22 directories should
> probably be moved too:
>
> drwxr-xr-x arch
> drwxr-xr-x block
> drwxr-xr-x certs # move to build/certs/ dir
> drwxr-xr-x crypto # move to kernel/crypto/ or security/crypto/
> drwxr-xr-x Documentation
> drwxr-xr-x drivers
> drwxr-xr-x fs
> drwxr-xr-x include
> drwxr-xr-x init
> drwxr-xr-x ipc # move to kernel/ipc/
> drwxr-xr-x kernel
> drwxr-xr-x lib
> drwxr-xr-x LICENSES
> drwxr-xr-x mm
> drwxr-xr-x net
> drwxr-xr-x samples # move to Documentation/samples/
> drwxr-xr-x scripts # move to build/scripts/

This one seems like it would break a lot of workflows, and contributor
muscle memory and scripts. get_maintainer.pl and checkpatch.pl
are probably in quite a few people's scripts.

Also, I'm not sure '/build' is the right destination for this. There
are a lot more things in there than just build scripts. If you really
want to remove the top level 'scripts', it might be best to put
the scripts from top-level '/scripts' into '/tools/scripts', which is
mostly empty now.
-- Tim

... rest snipped ...

2019-09-26 00:51:28

by Ingo Molnar

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1


* [email protected] <[email protected]> wrote:

> > -----Original Message-----
> > From: Ingo Molnar on Sunday, September 22, 2019 1:26 AM
> >
> > * Linus Torvalds <[email protected]> wrote:
> >
> > > On Fri, Sep 20, 2019 at 9:35 AM Brendan Higgins
> > > <[email protected]> wrote:
> > > >
> > > > Sorry about that. I am surprised that none of the other reviewers
> > > > brought this up.
> > >
> > > I think I'm "special".
> > >
> > > There was some other similar change a few years ago, which I
> > > absolutely hated because of how it broke autocomplete for me. Very few
> > > other people seemed to react to it.
> >
> > FWIW, I am obsessively sensitive to autocomplete and overall source code
> > file hieararchy and nomenclature details as well, so it's not just you.
> >
> > Beyond the muscle memory aspect, nonsensical naming and inanely flat file
> > hierarchies annoy kernel developers and makes it harder for newbies to
> > understand the kernel source as well.
> >
> > The less clutter, the more organization, the better - and there's very
> > few valid technical reasons to add any new files or directories to the
> > top level directory - we should probably *remove* quite a few.
> >
> > For example 'firmware/' was recently moved to drivers/firmware/, and in a
> > similar fashion about a third of the remaining 22 directories should
> > probably be moved too:
> >
> > drwxr-xr-x arch
> > drwxr-xr-x block
> > drwxr-xr-x certs # move to build/certs/ dir
> > drwxr-xr-x crypto # move to kernel/crypto/ or security/crypto/
> > drwxr-xr-x Documentation
> > drwxr-xr-x drivers
> > drwxr-xr-x fs
> > drwxr-xr-x include
> > drwxr-xr-x init
> > drwxr-xr-x ipc # move to kernel/ipc/
> > drwxr-xr-x kernel
> > drwxr-xr-x lib
> > drwxr-xr-x LICENSES
> > drwxr-xr-x mm
> > drwxr-xr-x net
> > drwxr-xr-x samples # move to Documentation/samples/
> > drwxr-xr-x scripts # move to build/scripts/
>
> This one seems like it would break a lot of workflows, and contributor
> muscle memory and scripts. get_maintainer.pl and checkpatch.pl
> are probably in quite a few people's scripts.
>
> Also, I'm not sure '/build' is the right destination for this. There
> are a lot more things in there than just build scripts. If you really
> want to remove the top level 'scripts', it might be best to put
> the scripts from top-level '/scripts' into '/tools/scripts', which is
> mostly empty now.

Agreed - I'll leave it alone for now, because you are right that it's
widely used.

Thanks,

Ingo

2019-09-26 02:45:03

by Ingo Molnar

[permalink] [raw]
Subject: [RFC, Tree] De-clutter the top level directory, move ipc/ => kernel/ipc/, samples/ => Documentation/samples/ and sound/ => drivers/sound/


* Greg KH <[email protected]> wrote:

> On Sun, Sep 22, 2019 at 01:25:55PM +0200, Ingo Molnar wrote:
> >
> > * Linus Torvalds <[email protected]> wrote:
> >
> > > On Fri, Sep 20, 2019 at 9:35 AM Brendan Higgins
> > > <[email protected]> wrote:
> > > >
> > > > Sorry about that. I am surprised that none of the other reviewers
> > > > brought this up.
> > >
> > > I think I'm "special".
> > >
> > > There was some other similar change a few years ago, which I
> > > absolutely hated because of how it broke autocomplete for me. Very few
> > > other people seemed to react to it.
> >
> > FWIW, I am obsessively sensitive to autocomplete and overall source code
> > file hieararchy and nomenclature details as well, so it's not just you.
> >
> > Beyond the muscle memory aspect, nonsensical naming and inanely flat file
> > hierarchies annoy kernel developers and makes it harder for newbies to
> > understand the kernel source as well.
> >
> > The less clutter, the more organization, the better - and there's very
> > few valid technical reasons to add any new files or directories to the
> > top level directory - we should probably *remove* quite a few.
> >
> > For example 'firmware/' was recently moved to drivers/firmware/, and in a
> > similar fashion about a third of the remaining 22 directories should
> > probably be moved too:
> >
> > drwxr-xr-x arch
> > drwxr-xr-x block
> > drwxr-xr-x certs # move to build/certs/ dir
> > drwxr-xr-x crypto # move to kernel/crypto/ or security/crypto/
> > drwxr-xr-x Documentation
> > drwxr-xr-x drivers
> > drwxr-xr-x fs
> > drwxr-xr-x include
> > drwxr-xr-x init
> > drwxr-xr-x ipc # move to kernel/ipc/
> > drwxr-xr-x kernel
> > drwxr-xr-x lib
> > drwxr-xr-x LICENSES
> > drwxr-xr-x mm
> > drwxr-xr-x net
> > drwxr-xr-x samples # move to Documentation/samples/
> > drwxr-xr-x scripts # move to build/scripts/
> > drwxr-xr-x security
> > drwxr-xr-x sound # move to drivers/sound/
> > drwxr-xr-x tools
> > drwxr-xr-x usr # move to build/usr/
> > drwxr-xr-x virt # move to the already existing drivers/virt/
> >
> > -rw-r--r-- COPYING
> > -rw-r--r-- CREDITS
> > -rw-r--r-- Kbuild
> > -rw-r--r-- Kconfig
> > -rw-r--r-- MAINTAINERS
> > -rw-r--r-- Makefile
> > -rw-r--r-- README
> >
> > There's a few borderline ones:
> >
> > - 'block' could in principle move to drivers/block/core/ but it's fine
> > at the top level too I think.
> >
> > - 'init' could in principle be moved to kernel/init/ - but it's not
> > wrong at the top level either.
> >
> > The remaining top level hierarchy would look pretty sweet and short:
> >
> > drwxr-xr-x arch
> > drwxr-xr-x block
> > drwxr-xr-x build # new
> > drwxr-xr-x Documentation
> > drwxr-xr-x drivers
> > drwxr-xr-x fs
> > drwxr-xr-x include
> > drwxr-xr-x init
> > drwxr-xr-x kernel
> > drwxr-xr-x lib
> > drwxr-xr-x LICENSES
> > drwxr-xr-x mm
> > drwxr-xr-x net
> > drwxr-xr-x security
> > drwxr-xr-x tools
> >
> > -rw-r--r-- COPYING
> > -rw-r--r-- CREDITS
> > -rw-r--r-- Kbuild
> > -rw-r--r-- Kconfig
> > -rw-r--r-- MAINTAINERS
> > -rw-r--r-- Makefile
> > -rw-r--r-- README
> >
> > I'm volunteering to do this (in a scripted, repeatable, reviewable,
> > tweakable and "easy to execute in a quiet moment" fashion), although
> > I also expect you to balk at the churn. :-)
>
> I for one would love the above changes. And I'm the one that has to
> deal with all of the backporting issues that arise with stable backports :)

Oh, that's a pleasant surprise, I didn't expect _100%_ support! :-)

So I started working on this today and whipped up three of these
movements, in a 100% scripted fashion.

You can have a sneak preview at the result in this tree:

git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git WIP.core/toplevel

...

2515 files changed, 42476 insertions(+), 42476 deletions(-)

I decided not to post these first patches to lkml, because many of the
patches are unreviewably large even with smart rename detection. Even the
diffstat was 5,000+ lines so I abbreviated it ...

The tree has 6 commits for now:

6576382877c0: toplevel: Move ipc/ to kernel/ipc/
8f7ace98971a: toplevel: Fix up kernel/ipc/ movement effects
cc703bc54b00: toplevel: Move sound/ to drivers/sound/
a63bdbd797da: toplevel: Fix up drivers/sound/ movement effects
28adaae5ae8f: toplevel: Move samples/ to Documentation/samples/
4e9f67e3bc48: toplevel: Fix up samples/ movement effects

Due to the scripting all the commits are auto-generated for the time
being - without SOB lines and proper changelogs, etc.

Note that each movement action consists of two commits:

cc703bc54b00: toplevel: Move sound/ to drivers/sound/
a63bdbd797da: toplevel: Fix up drivers/sound/ movement effects

The first one is just the result of the 'git mv', which gets most of the
noise.

The second one fixes up the build system, Kconfig system and any
references to the old file names. Forward and backward porting of patches
across the rename boundary is in most cases a simple matter of fixing up
the file names in the patches - I'll put both forward and backward
patch-consersion one-liner scripts for that into the changelog itself.

I believe this is easier to review, but it has the disadvantage that
bisection will trivially break if it hits any of the base commits. Can
combine the commits as well, or can add a suggestion to use
"git bisect next" to the changelog itself. Either approach is fine to me.

Anyway, I will fix the aforementioned details, but wanted to show the
gist of these changes and give you an opportunity to request changes,
before I go down this path too much.

The tree is functional here after some light testing - will put it
through more rigorous testing before posting future iterations.

The advantage of the scripting is that:

- I can re-run this on any base tree and with any granularity, and my
scripts also have some sanity checks to see whether any old filename
patterns escaped the conversion, etc.

- The order of the movement is discretionary as well, i.e. if you prefer
the sound/ movement in a single tree and at a specific time, that all
can be phased arbitrarily.

- The scripts can be re-run with less than a minute of runtime, so it
can all be timed for maximum convenience to you, Linus and affected
maintainers.

Is this close to what you had in mind?

Thanks,

Ingo

------------------>
Ingo Molnar (6):
toplevel: Move ipc/ to kernel/ipc/
toplevel: Fix up kernel/ipc/ movement effects
toplevel: Move sound/ to drivers/sound/
toplevel: Fix up drivers/sound/ movement effects
toplevel: Move samples/ to Documentation/samples/
toplevel: Fix up samples/ movement effects

2515 files changed, 42476 insertions(+), 42476 deletions(-)

2019-09-26 04:59:09

by Ingo Molnar

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1


* Shuah Khan <[email protected]> wrote:

> Right. What you suggesting is very similar to and more complete than
> what I have been thinking about and proposed at the KS kselftest track.
>
> i.e move tools/testing/selftests to kselftest at the root level. I like
> your idea of moving tools/testing up to root and keep selftests under
> it.
>
> If we are good with this kind of change, I would like to get this done
> sooner than later. There is some back-porting churn to worry about.

I think the movement I suggested would be sufficient:

tools/testing/selftests/ => tools/selftests/

I.e. let's not clutter up the top level directory.

Thanks,

Ingo

2019-09-26 12:54:24

by David Sterba

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On Sun, Sep 22, 2019 at 01:25:55PM +0200, Ingo Molnar wrote:
>
> * Linus Torvalds <[email protected]> wrote:
>
> > On Fri, Sep 20, 2019 at 9:35 AM Brendan Higgins
> > <[email protected]> wrote:
> > >
> > > Sorry about that. I am surprised that none of the other reviewers
> > > brought this up.
> >
> > I think I'm "special".
> >
> > There was some other similar change a few years ago, which I
> > absolutely hated because of how it broke autocomplete for me. Very few
> > other people seemed to react to it.
>
> FWIW, I am obsessively sensitive to autocomplete and overall source code
> file hieararchy and nomenclature details as well, so it's not just you.
>
> Beyond the muscle memory aspect, nonsensical naming and inanely flat file
> hierarchies annoy kernel developers and makes it harder for newbies to
> understand the kernel source as well.
>
> The less clutter, the more organization, the better - and there's very
> few valid technical reasons to add any new files or directories to the
> top level directory - we should probably *remove* quite a few.
>
> For example 'firmware/' was recently moved to drivers/firmware/, and in a
> similar fashion about a third of the remaining 22 directories should
> probably be moved too:
>
> drwxr-xr-x arch
> drwxr-xr-x block
> drwxr-xr-x certs # move to build/certs/ dir
> drwxr-xr-x crypto # move to kernel/crypto/ or security/crypto/

For code with lots of history and active development, moving is quite
counterproductive as it makes tracking a change tedious. Git can follow
the path changes, but that's exactly the step I'd like not to do. That's
similar to pure whitespace cleanup patches that are noise.

The decision for move should be IMO up to the maintainers of the code,
that apparently worked for firmware/ -> drivers/firmware that has been
mentioned. That's fine.

The muscle memory argument sounds quite weak to me, each of us has some
habits, editor settings and coding style preferences, we will never
agree. That's fine too.

The reason I'd find valid for moving is to reduce confusion when working
with the files, not to promote a "formally correct classification" and
hierarchy of directories that will stand in the way in the daily work.

Though I'm not directly affected by most of the proposed changes, I feel
I should speak up before the file maneuvers reach code I care about.

> - 'block' could in principle move to drivers/block/core/ but it's fine
> at the top level too I think.

Following that principle, we can move mm/ -> drivers/char/memory/ right? :)

2019-09-27 13:53:55

by Pavel Machek

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

Hi!

> > I think I'm "special".
> >
> > There was some other similar change a few years ago, which I
> > absolutely hated because of how it broke autocomplete for me. Very few
> > other people seemed to react to it.
>
> FWIW, I am obsessively sensitive to autocomplete and overall source code
> file hieararchy and nomenclature details as well, so it's not just you.
>
> Beyond the muscle memory aspect, nonsensical naming and inanely flat file
> hierarchies annoy kernel developers and makes it harder for newbies to
> understand the kernel source as well.
>
> The less clutter, the more organization, the better - and there's very
> few valid technical reasons to add any new files or directories to the
> top level directory - we should probably *remove* quite a few.
>
> For example 'firmware/' was recently moved to drivers/firmware/, and in a
> similar fashion about a third of the remaining 22 directories should
> probably be moved too:
>
> drwxr-xr-x arch
> drwxr-xr-x block
> drwxr-xr-x certs # move to build/certs/ dir
> drwxr-xr-x crypto # move to kernel/crypto/ or security/crypto/
> drwxr-xr-x Documentation
> drwxr-xr-x drivers
> drwxr-xr-x fs
> drwxr-xr-x include
> drwxr-xr-x init
> drwxr-xr-x ipc # move to kernel/ipc/
> drwxr-xr-x kernel
> drwxr-xr-x lib
> drwxr-xr-x LICENSES
> drwxr-xr-x mm
> drwxr-xr-x net
> drwxr-xr-x samples # move to Documentation/samples/
> drwxr-xr-x scripts # move to build/scripts/
> drwxr-xr-x security
> drwxr-xr-x sound # move to drivers/sound/

Heh, I was always surprised that sound/ made it into top level... and no, I'd not
mind it being moved away.

> There's a few borderline ones:
>
> - 'block' could in principle move to drivers/block/core/ but it's fine
> at the top level too I think.
>
> - 'init' could in principle be moved to kernel/init/ - but it's not
> wrong at the top level either.

net would also make sense as drivers/net/core... That is what inspired sound/ afaict.

> I'm volunteering to do this (in a scripted, repeatable, reviewable,
> tweakable and "easy to execute in a quiet moment" fashion), although
> I also expect you to balk at the churn. :-)

I'd like to see that happen...

Best regards,
Pavel

--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

2019-10-03 09:13:42

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [GIT PULL] Kselftest update for Linux 5.4-rc1

On Sun, Sep 22, 2019 at 8:26 PM Ingo Molnar <[email protected]> wrote:
>
>
> * Linus Torvalds <[email protected]> wrote:
>
> > On Fri, Sep 20, 2019 at 9:35 AM Brendan Higgins
> > <[email protected]> wrote:
> > >
> > > Sorry about that. I am surprised that none of the other reviewers
> > > brought this up.
> >
> > I think I'm "special".
> >
> > There was some other similar change a few years ago, which I
> > absolutely hated because of how it broke autocomplete for me. Very few
> > other people seemed to react to it.
>
> FWIW, I am obsessively sensitive to autocomplete and overall source code
> file hieararchy and nomenclature details as well, so it's not just you.
>
> Beyond the muscle memory aspect, nonsensical naming and inanely flat file
> hierarchies annoy kernel developers and makes it harder for newbies to
> understand the kernel source as well.
>
> The less clutter, the more organization, the better - and there's very
> few valid technical reasons to add any new files or directories to the
> top level directory - we should probably *remove* quite a few.
>
> For example 'firmware/' was recently moved to drivers/firmware/, and in a
> similar fashion about a third of the remaining 22 directories should
> probably be moved too:
>
> drwxr-xr-x arch
> drwxr-xr-x block
> drwxr-xr-x certs # move to build/certs/ dir
> drwxr-xr-x crypto # move to kernel/crypto/ or security/crypto/
> drwxr-xr-x Documentation
> drwxr-xr-x drivers
> drwxr-xr-x fs
> drwxr-xr-x include
> drwxr-xr-x init
> drwxr-xr-x ipc # move to kernel/ipc/
> drwxr-xr-x kernel
> drwxr-xr-x lib
> drwxr-xr-x LICENSES
> drwxr-xr-x mm
> drwxr-xr-x net
> drwxr-xr-x samples # move to Documentation/samples/
> drwxr-xr-x scripts # move to build/scripts/
> drwxr-xr-x security
> drwxr-xr-x sound # move to drivers/sound/
> drwxr-xr-x tools
> drwxr-xr-x usr # move to build/usr/
> drwxr-xr-x virt # move to the already existing drivers/virt/
>
> -rw-r--r-- COPYING
> -rw-r--r-- CREDITS
> -rw-r--r-- Kbuild
> -rw-r--r-- Kconfig
> -rw-r--r-- MAINTAINERS
> -rw-r--r-- Makefile
> -rw-r--r-- README
>
> There's a few borderline ones:
>
> - 'block' could in principle move to drivers/block/core/ but it's fine
> at the top level too I think.
>
> - 'init' could in principle be moved to kernel/init/ - but it's not
> wrong at the top level either.
>
> The remaining top level hierarchy would look pretty sweet and short:
>
> drwxr-xr-x arch
> drwxr-xr-x block
> drwxr-xr-x build # new

I am opposed to the "build".

The build tools will go too deep, like build/scripts/kconfig ?

I often use checkpatch.pl, get_maintainer.pl etc.

Do I have to type build/scripts/checkpatch.pl ?






> drwxr-xr-x Documentation
> drwxr-xr-x drivers
> drwxr-xr-x fs
> drwxr-xr-x include
> drwxr-xr-x init
> drwxr-xr-x kernel
> drwxr-xr-x lib
> drwxr-xr-x LICENSES
> drwxr-xr-x mm
> drwxr-xr-x net
> drwxr-xr-x security
> drwxr-xr-x tools
>
> -rw-r--r-- COPYING
> -rw-r--r-- CREDITS
> -rw-r--r-- Kbuild
> -rw-r--r-- Kconfig
> -rw-r--r-- MAINTAINERS
> -rw-r--r-- Makefile
> -rw-r--r-- README
>
> I'm volunteering to do this (in a scripted, repeatable, reviewable,
> tweakable and "easy to execute in a quiet moment" fashion), although
> I also expect you to balk at the churn. :-)
>
> Thanks,
>
> Ingo



--
Best Regards
Masahiro Yamada