2022-12-29 08:07:35

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH v3 1/2] .gitignore: update the command to check tracked files being ignored

Recent git versions do not accept the noted command.

$ git ls-files -i --exclude-standard
fatal: ls-files -i must be used with either -o or -c

The -c was implied before, but we need to make it explicit since
git commit b338e9f66873 ("ls-files: error out on -i unless -o or -c
are specified").

Also, replace --exclude-standard with --exclude-per-directory=.gitignore
so that everyone will get consistent results.

git-ls-files(1) says:

--exclude-standard
Add the standard Git exclusions: .git/info/exclude, .gitignore in
each directory, and the user's global exclusion file.

We cannot predict what is locally added to .git/info/exclude or the
user's global exclusion file.

We can only manage .gitignore files committed to the repository.

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Miguel Ojeda <[email protected]>
---

Changes in v3:
- Fix a typo. Update commit description per Miguel

.gitignore | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.gitignore b/.gitignore
index 3ec73ead6757..2e2e3d1eeaee 100644
--- a/.gitignore
+++ b/.gitignore
@@ -4,7 +4,7 @@
# subdirectories here. Add them in the ".gitignore" file
# in that subdirectory instead.
#
-# NOTE! Please use 'git ls-files -i --exclude-standard'
+# NOTE! Please use 'git ls-files -i -c --exclude-per-directory=.gitignore'
# command after changing this file, to see if there are
# any tracked files which get ignored after the change.
#
--
2.34.1


2022-12-29 08:23:06

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

The top .gitignore comments about how to detect files breaking
.gitignore rules, but people rarely care about it.

Add a new W=1 warning to detect files that are tracked but ignored by
git. If git is not installed or the source tree is not tracked by git
at all, this script does not print anything.

Running it on v6.2-rc1 detected the following:

$ make W=1 misc-check
Documentation/devicetree/bindings/.yamllint: warning: ignored by one of the .gitignore files
drivers/clk/.kunitconfig: warning: ignored by one of the .gitignore files
drivers/gpu/drm/tests/.kunitconfig: warning: ignored by one of the .gitignore files
drivers/hid/.kunitconfig: warning: ignored by one of the .gitignore files
fs/ext4/.kunitconfig: warning: ignored by one of the .gitignore files
fs/fat/.kunitconfig: warning: ignored by one of the .gitignore files
kernel/kcsan/.kunitconfig: warning: ignored by one of the .gitignore files
lib/kunit/.kunitconfig: warning: ignored by one of the .gitignore files
mm/kfence/.kunitconfig: warning: ignored by one of the .gitignore files
tools/testing/selftests/arm64/tags/.gitignore: warning: ignored by one of the .gitignore files
tools/testing/selftests/arm64/tags/Makefile: warning: ignored by one of the .gitignore files
tools/testing/selftests/arm64/tags/run_tags_test.sh: warning: ignored by one of the .gitignore files
tools/testing/selftests/arm64/tags/tags_test.c: warning: ignored by one of the .gitignore files

These are ignored by the '.*' or 'tags' in the top .gitignore, but
there is no rule to negate it.

You might be tempted to do 'git add -f' but I want to have the real
issue fixed (by fixing a .gitignore, or by renaming files, etc.).

Signed-off-by: Masahiro Yamada <[email protected]>
Reviewed-by: Nathan Chancellor <[email protected]>
Reviewed-by: Nicolas Schier <[email protected]>
---

Changes in v3:
- change working directory to srctree (Nicolas)

Changes in v2:
- Add $(srctree)/ to make it work with O=

Makefile | 6 ++++++
scripts/misc-check | 19 +++++++++++++++++++
2 files changed, 25 insertions(+)
create mode 100755 scripts/misc-check

diff --git a/Makefile b/Makefile
index acce5ec514f6..c0d7c75d8f14 100644
--- a/Makefile
+++ b/Makefile
@@ -1848,6 +1848,12 @@ rust-analyzer:
# Misc
# ---------------------------------------------------------------------------

+PHONY += misc-check
+misc-check:
+ $(Q)$(srctree)/scripts/misc-check
+
+all: misc-check
+
PHONY += scripts_gdb
scripts_gdb: prepare0
$(Q)$(MAKE) $(build)=scripts/gdb
diff --git a/scripts/misc-check b/scripts/misc-check
new file mode 100755
index 000000000000..d40d5484e0c5
--- /dev/null
+++ b/scripts/misc-check
@@ -0,0 +1,19 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-only
+
+set -e
+
+# Detect files that are tracked but ignored by git. This is checked only when
+# ${KBUILD_EXTRA_WARN} contains 1, git is installed, and the source tree is
+# tracked by git.
+check_tracked_ignored_files () {
+ case "${KBUILD_EXTRA_WARN}" in
+ *1*) ;;
+ *) return;;
+ esac
+
+ git -C ${srctree:-.} ls-files -i -c --exclude-per-directory=.gitignore 2>/dev/null |
+ sed 's/$/: warning: ignored by one of the .gitignore files/' >&2
+}
+
+check_tracked_ignored_files
--
2.34.1

2022-12-29 14:51:23

by Nicolas Schier

[permalink] [raw]
Subject: Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

On Thu, Dec 29, 2022 at 04:43:10PM +0900 Masahiro Yamada wrote:
> The top .gitignore comments about how to detect files breaking
> .gitignore rules, but people rarely care about it.
>
> Add a new W=1 warning to detect files that are tracked but ignored by
> git. If git is not installed or the source tree is not tracked by git
> at all, this script does not print anything.
>
> Running it on v6.2-rc1 detected the following:
>
> $ make W=1 misc-check
> Documentation/devicetree/bindings/.yamllint: warning: ignored by one of the .gitignore files
> drivers/clk/.kunitconfig: warning: ignored by one of the .gitignore files
> drivers/gpu/drm/tests/.kunitconfig: warning: ignored by one of the .gitignore files
> drivers/hid/.kunitconfig: warning: ignored by one of the .gitignore files
> fs/ext4/.kunitconfig: warning: ignored by one of the .gitignore files
> fs/fat/.kunitconfig: warning: ignored by one of the .gitignore files
> kernel/kcsan/.kunitconfig: warning: ignored by one of the .gitignore files
> lib/kunit/.kunitconfig: warning: ignored by one of the .gitignore files
> mm/kfence/.kunitconfig: warning: ignored by one of the .gitignore files
> tools/testing/selftests/arm64/tags/.gitignore: warning: ignored by one of the .gitignore files
> tools/testing/selftests/arm64/tags/Makefile: warning: ignored by one of the .gitignore files
> tools/testing/selftests/arm64/tags/run_tags_test.sh: warning: ignored by one of the .gitignore files
> tools/testing/selftests/arm64/tags/tags_test.c: warning: ignored by one of the .gitignore files
>
> These are ignored by the '.*' or 'tags' in the top .gitignore, but
> there is no rule to negate it.
>
> You might be tempted to do 'git add -f' but I want to have the real
> issue fixed (by fixing a .gitignore, or by renaming files, etc.).
>
> Signed-off-by: Masahiro Yamada <[email protected]>
> Reviewed-by: Nathan Chancellor <[email protected]>
> Reviewed-by: Nicolas Schier <[email protected]>
> ---
>
> Changes in v3:
> - change working directory to srctree (Nicolas)
>
> Changes in v2:
> - Add $(srctree)/ to make it work with O=
>
> Makefile | 6 ++++++
> scripts/misc-check | 19 +++++++++++++++++++
> 2 files changed, 25 insertions(+)
> create mode 100755 scripts/misc-check

Tested-by: Nicolas Schier <[email protected]>

out of curiosity: do you plan to implement more checks in the misc-checks
target?

Kind regards,
Nicolas


Attachments:
(No filename) (2.44 kB)
signature.asc (849.00 B)
Download all attachments

2022-12-30 03:47:39

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

On Thu, Dec 29, 2022 at 11:23 PM Nicolas Schier <[email protected]> wrote:
>
> On Thu, Dec 29, 2022 at 04:43:10PM +0900 Masahiro Yamada wrote:
> > The top .gitignore comments about how to detect files breaking
> > .gitignore rules, but people rarely care about it.
> >
> > Add a new W=1 warning to detect files that are tracked but ignored by
> > git. If git is not installed or the source tree is not tracked by git
> > at all, this script does not print anything.
> >
> > Running it on v6.2-rc1 detected the following:
> >
> > $ make W=1 misc-check
> > Documentation/devicetree/bindings/.yamllint: warning: ignored by one of the .gitignore files
> > drivers/clk/.kunitconfig: warning: ignored by one of the .gitignore files
> > drivers/gpu/drm/tests/.kunitconfig: warning: ignored by one of the .gitignore files
> > drivers/hid/.kunitconfig: warning: ignored by one of the .gitignore files
> > fs/ext4/.kunitconfig: warning: ignored by one of the .gitignore files
> > fs/fat/.kunitconfig: warning: ignored by one of the .gitignore files
> > kernel/kcsan/.kunitconfig: warning: ignored by one of the .gitignore files
> > lib/kunit/.kunitconfig: warning: ignored by one of the .gitignore files
> > mm/kfence/.kunitconfig: warning: ignored by one of the .gitignore files
> > tools/testing/selftests/arm64/tags/.gitignore: warning: ignored by one of the .gitignore files
> > tools/testing/selftests/arm64/tags/Makefile: warning: ignored by one of the .gitignore files
> > tools/testing/selftests/arm64/tags/run_tags_test.sh: warning: ignored by one of the .gitignore files
> > tools/testing/selftests/arm64/tags/tags_test.c: warning: ignored by one of the .gitignore files
> >
> > These are ignored by the '.*' or 'tags' in the top .gitignore, but
> > there is no rule to negate it.
> >
> > You might be tempted to do 'git add -f' but I want to have the real
> > issue fixed (by fixing a .gitignore, or by renaming files, etc.).
> >
> > Signed-off-by: Masahiro Yamada <[email protected]>
> > Reviewed-by: Nathan Chancellor <[email protected]>
> > Reviewed-by: Nicolas Schier <[email protected]>
> > ---
> >
> > Changes in v3:
> > - change working directory to srctree (Nicolas)
> >
> > Changes in v2:
> > - Add $(srctree)/ to make it work with O=
> >
> > Makefile | 6 ++++++
> > scripts/misc-check | 19 +++++++++++++++++++
> > 2 files changed, 25 insertions(+)
> > create mode 100755 scripts/misc-check
>
> Tested-by: Nicolas Schier <[email protected]>
>
> out of curiosity: do you plan to implement more checks in the misc-checks
> target?


I chose a generic name so I can put more in this script, but
honestly I have nothing else in my mind for now.




>
> Kind regards,
> Nicolas
>


--
Best Regards
Masahiro Yamada

2023-01-27 13:25:32

by Andy Shevchenko

[permalink] [raw]
Subject: Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

On Thu, Dec 29, 2022 at 04:43:10PM +0900, Masahiro Yamada wrote:
> The top .gitignore comments about how to detect files breaking
> .gitignore rules, but people rarely care about it.
>
> Add a new W=1 warning to detect files that are tracked but ignored by
> git. If git is not installed or the source tree is not tracked by git
> at all, this script does not print anything.
>
> Running it on v6.2-rc1 detected the following:

Since patch was published there is no sign it was ever meet Linux Next.
What's the plan?

--
With Best Regards,
Andy Shevchenko



2023-01-27 14:31:53

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

On Fri, Jan 27, 2023 at 10:25 PM Andy Shevchenko
<[email protected]> wrote:
>
> On Thu, Dec 29, 2022 at 04:43:10PM +0900, Masahiro Yamada wrote:
> > The top .gitignore comments about how to detect files breaking
> > .gitignore rules, but people rarely care about it.
> >
> > Add a new W=1 warning to detect files that are tracked but ignored by
> > git. If git is not installed or the source tree is not tracked by git
> > at all, this script does not print anything.
> >
> > Running it on v6.2-rc1 detected the following:
>
> Since patch was published there is no sign it was ever meet Linux Next.
> What's the plan?


Oh?
I can see this patch in linux-next.


$ git log next-20230127 -- scripts/misc-check


--
Best Regards
Masahiro Yamada

2023-01-27 14:41:49

by Andy Shevchenko

[permalink] [raw]
Subject: Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

On Fri, Jan 27, 2023 at 11:31:07PM +0900, Masahiro Yamada wrote:
> On Fri, Jan 27, 2023 at 10:25 PM Andy Shevchenko
> <[email protected]> wrote:
> >
> > On Thu, Dec 29, 2022 at 04:43:10PM +0900, Masahiro Yamada wrote:
> > > The top .gitignore comments about how to detect files breaking
> > > .gitignore rules, but people rarely care about it.
> > >
> > > Add a new W=1 warning to detect files that are tracked but ignored by
> > > git. If git is not installed or the source tree is not tracked by git
> > > at all, this script does not print anything.
> > >
> > > Running it on v6.2-rc1 detected the following:
> >
> > Since patch was published there is no sign it was ever meet Linux Next.
> > What's the plan?
>
> Oh?

Sorry, my mistake. I need to understand why these patches do not fix
the issue I have.

> I can see this patch in linux-next.
>
> $ git log next-20230127 -- scripts/misc-check

--
With Best Regards,
Andy Shevchenko



2023-01-27 14:50:47

by Andy Shevchenko

[permalink] [raw]
Subject: Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

On Fri, Jan 27, 2023 at 04:41:37PM +0200, Andy Shevchenko wrote:
> On Fri, Jan 27, 2023 at 11:31:07PM +0900, Masahiro Yamada wrote:
> > On Fri, Jan 27, 2023 at 10:25 PM Andy Shevchenko
> > <[email protected]> wrote:
> > >
> > > On Thu, Dec 29, 2022 at 04:43:10PM +0900, Masahiro Yamada wrote:
> > > > The top .gitignore comments about how to detect files breaking
> > > > .gitignore rules, but people rarely care about it.
> > > >
> > > > Add a new W=1 warning to detect files that are tracked but ignored by
> > > > git. If git is not installed or the source tree is not tracked by git
> > > > at all, this script does not print anything.
> > > >
> > > > Running it on v6.2-rc1 detected the following:
> > >
> > > Since patch was published there is no sign it was ever meet Linux Next.
> > > What's the plan?
> >
> > Oh?
>
> Sorry, my mistake. I need to understand why these patches do not fix
> the issue I have.

OK, after carefully reading the commit message it's actually the culprit of
the warnings I have.

So, it seems we need to wait maintainers / developers of the respective code
to go and fix this. Is it your intention?

--
With Best Regards,
Andy Shevchenko



2023-01-27 14:53:36

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v3 2/2] kbuild: make W=1 warn files that are tracked but ignored by git

On Fri, Jan 27, 2023 at 11:50 PM Andy Shevchenko
<[email protected]> wrote:
>
> On Fri, Jan 27, 2023 at 04:41:37PM +0200, Andy Shevchenko wrote:
> > On Fri, Jan 27, 2023 at 11:31:07PM +0900, Masahiro Yamada wrote:
> > > On Fri, Jan 27, 2023 at 10:25 PM Andy Shevchenko
> > > <[email protected]> wrote:
> > > >
> > > > On Thu, Dec 29, 2022 at 04:43:10PM +0900, Masahiro Yamada wrote:
> > > > > The top .gitignore comments about how to detect files breaking
> > > > > .gitignore rules, but people rarely care about it.
> > > > >
> > > > > Add a new W=1 warning to detect files that are tracked but ignored by
> > > > > git. If git is not installed or the source tree is not tracked by git
> > > > > at all, this script does not print anything.
> > > > >
> > > > > Running it on v6.2-rc1 detected the following:
> > > >
> > > > Since patch was published there is no sign it was ever meet Linux Next.
> > > > What's the plan?
> > >
> > > Oh?
> >
> > Sorry, my mistake. I need to understand why these patches do not fix
> > the issue I have.
>
> OK, after carefully reading the commit message it's actually the culprit of
> the warnings I have.
>
> So, it seems we need to wait maintainers / developers of the respective code
> to go and fix this. Is it your intention?


Yes.
I expect the 0day bot will block new breakages,
but actually the mainline got more warnings
under kselftest.




--
Best Regards
Masahiro Yamada