2023-03-25 14:25:20

by Masahiro Yamada

[permalink] [raw]
Subject: [PATCH] kbuild: fix package build error due to broken symlinks

'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in
a dirty source tree. Handle symlinks properly, and also, keep the
executable permission.

Fixes: 05e96e96a315 ("kbuild: use git-archive for source package creation")
Signed-off-by: Masahiro Yamada <[email protected]>
---

scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++---------
1 file changed, 27 insertions(+), 9 deletions(-)

diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
index f842ab50a780..23551de92e1b 100755
--- a/scripts/package/gen-diff-patch
+++ b/scripts/package/gen-diff-patch
@@ -23,16 +23,34 @@ fi
git -C ${srctree} status --porcelain --untracked-files=all |
while read stat path
do
- if [ "${stat}" = '??' ]; then
-
- if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
- ! head -n1 .tmp_diff | grep -q "Binary files"; then
- {
- echo "--- /dev/null"
- echo "+++ linux/$path"
- cat .tmp_diff | tail -n +3
- } >> ${untracked_patch}
+ if [ "${stat}" != '??' ]; then
+ continue
+ fi
+
+ if [ -L "${path}" ]; then
+ {
+ echo "diff --git a/${path} b/${path}"
+ echo "new file mode 120000"
+ echo "--- /dev/null"
+ echo "+++ b/$path"
+ echo "@@ -0,0 +1 @@"
+ printf "+"; readlink ${path}
+ echo '\ No newline at end of file'
+ } >> ${untracked_patch}
+ elif ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
+ ! head -n1 .tmp_diff | grep -q "Binary files"; then
+ if [ -x ${path} ]; then
+ mode=100755
+ else
+ mode=100644
fi
+ {
+ echo "diff --git a/${path} b/${path}"
+ echo "new file mode ${mode}"
+ echo "--- /dev/null"
+ echo "+++ b/$path"
+ cat .tmp_diff | tail -n +3
+ } >> ${untracked_patch}
fi
done

--
2.34.1


2023-03-29 20:20:56

by Nicolas Schier

[permalink] [raw]
Subject: Re: [PATCH] kbuild: fix package build error due to broken symlinks

On Sat 25 Mar 2023 23:19:09 GMT, Masahiro Yamada wrote:
> 'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in
> a dirty source tree. Handle symlinks properly, and also, keep the
> executable permission.
>
> Fixes: 05e96e96a315 ("kbuild: use git-archive for source package creation")
> Signed-off-by: Masahiro Yamada <[email protected]>
> ---
>
> scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++---------
> 1 file changed, 27 insertions(+), 9 deletions(-)
>
> diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
> index f842ab50a780..23551de92e1b 100755
> --- a/scripts/package/gen-diff-patch
> +++ b/scripts/package/gen-diff-patch
> @@ -23,16 +23,34 @@ fi
> git -C ${srctree} status --porcelain --untracked-files=all |
> while read stat path
> do
> - if [ "${stat}" = '??' ]; then
> -
> - if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
> - ! head -n1 .tmp_diff | grep -q "Binary files"; then
> - {
> - echo "--- /dev/null"
> - echo "+++ linux/$path"
> - cat .tmp_diff | tail -n +3
> - } >> ${untracked_patch}
> + if [ "${stat}" != '??' ]; then
> + continue
> + fi
> +
> + if [ -L "${path}" ]; then
> + {
> + echo "diff --git a/${path} b/${path}"
> + echo "new file mode 120000"
> + echo "--- /dev/null"
> + echo "+++ b/$path"
> + echo "@@ -0,0 +1 @@"
> + printf "+"; readlink ${path}

Better quote "${path}"?

> + echo '\ No newline at end of file'
> + } >> ${untracked_patch}

Here quoting should not be necessary as mkdebian and mkspec give only
save filenames, but for consistency I'd quote ${untracked_patch} as
well.

> + elif ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
> + ! head -n1 .tmp_diff | grep -q "Binary files"; then
> + if [ -x ${path} ]; then

Same here.

> + mode=100755
> + else
> + mode=100644
> fi
> + {
> + echo "diff --git a/${path} b/${path}"
> + echo "new file mode ${mode}"
> + echo "--- /dev/null"
> + echo "+++ b/$path"
> + cat .tmp_diff | tail -n +3
> + } >> ${untracked_patch}

And possibly here?

> fi
> done
>
> --
> 2.34.1

Reviewed-by: Nicolas Schier <[email protected]>


Attachments:
(No filename) (2.19 kB)
signature.asc (849.00 B)
Download all attachments

2023-04-01 14:59:38

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH] kbuild: fix package build error due to broken symlinks

On Thu, Mar 30, 2023 at 5:02 AM Nicolas Schier <[email protected]> wrote:
>
> On Sat 25 Mar 2023 23:19:09 GMT, Masahiro Yamada wrote:
> > 'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in
> > a dirty source tree. Handle symlinks properly, and also, keep the
> > executable permission.
> >
> > Fixes: 05e96e96a315 ("kbuild: use git-archive for source package creation")
> > Signed-off-by: Masahiro Yamada <[email protected]>
> > ---
> >
> > scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++---------
> > 1 file changed, 27 insertions(+), 9 deletions(-)
> >
> > diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
> > index f842ab50a780..23551de92e1b 100755
> > --- a/scripts/package/gen-diff-patch
> > +++ b/scripts/package/gen-diff-patch
> > @@ -23,16 +23,34 @@ fi
> > git -C ${srctree} status --porcelain --untracked-files=all |
> > while read stat path
> > do
> > - if [ "${stat}" = '??' ]; then
> > -
> > - if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
> > - ! head -n1 .tmp_diff | grep -q "Binary files"; then
> > - {
> > - echo "--- /dev/null"
> > - echo "+++ linux/$path"
> > - cat .tmp_diff | tail -n +3
> > - } >> ${untracked_patch}
> > + if [ "${stat}" != '??' ]; then
> > + continue
> > + fi
> > +
> > + if [ -L "${path}" ]; then
> > + {
> > + echo "diff --git a/${path} b/${path}"
> > + echo "new file mode 120000"
> > + echo "--- /dev/null"
> > + echo "+++ b/$path"
> > + echo "@@ -0,0 +1 @@"
> > + printf "+"; readlink ${path}
>
> Better quote "${path}"?


Thanks for the suggestion.

Quoting variables are correct in most cases.
But, that is not enough to generate a valid
patch when a file path contains spaces.



'git format-patch' produces a patch that
is accepted by GNU patch and also by dpkg-source.

I learned a trick from GIT source code.


If you are interested, what GIT does [1].


[1] https://github.com/git/git/commit/1a9eb3b9d50367bee8fe85022684d812816fe531


I will send v2 later, where I made some more efforts
to fix several corner cases even if that is not perfect.








--
Best Regards
Masahiro Yamada

2023-04-01 18:01:37

by Nicolas Schier

[permalink] [raw]
Subject: Re: [PATCH] kbuild: fix package build error due to broken symlinks

On Sat, Apr 01, 2023 at 11:55:44PM +0900 Masahiro Yamada wrote:
> On Thu, Mar 30, 2023 at 5:02 AM Nicolas Schier <[email protected]> wrote:
> >
> > On Sat 25 Mar 2023 23:19:09 GMT, Masahiro Yamada wrote:
> > > 'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in
> > > a dirty source tree. Handle symlinks properly, and also, keep the
> > > executable permission.
> > >
> > > Fixes: 05e96e96a315 ("kbuild: use git-archive for source package creation")
> > > Signed-off-by: Masahiro Yamada <[email protected]>
> > > ---
> > >
> > > scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++---------
> > > 1 file changed, 27 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
> > > index f842ab50a780..23551de92e1b 100755
> > > --- a/scripts/package/gen-diff-patch
> > > +++ b/scripts/package/gen-diff-patch
> > > @@ -23,16 +23,34 @@ fi
> > > git -C ${srctree} status --porcelain --untracked-files=all |
> > > while read stat path
> > > do
> > > - if [ "${stat}" = '??' ]; then
> > > -
> > > - if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
> > > - ! head -n1 .tmp_diff | grep -q "Binary files"; then
> > > - {
> > > - echo "--- /dev/null"
> > > - echo "+++ linux/$path"
> > > - cat .tmp_diff | tail -n +3
> > > - } >> ${untracked_patch}
> > > + if [ "${stat}" != '??' ]; then
> > > + continue
> > > + fi
> > > +
> > > + if [ -L "${path}" ]; then
> > > + {
> > > + echo "diff --git a/${path} b/${path}"
> > > + echo "new file mode 120000"
> > > + echo "--- /dev/null"
> > > + echo "+++ b/$path"
> > > + echo "@@ -0,0 +1 @@"
> > > + printf "+"; readlink ${path}
> >
> > Better quote "${path}"?
>
>
> Thanks for the suggestion.
>
> Quoting variables are correct in most cases.
> But, that is not enough to generate a valid
> patch when a file path contains spaces.
>
>
>
> 'git format-patch' produces a patch that
> is accepted by GNU patch and also by dpkg-source.
>
> I learned a trick from GIT source code.
>
>
> If you are interested, what GIT does [1].
>
>
> [1] https://github.com/git/git/commit/1a9eb3b9d50367bee8fe85022684d812816fe531

thanks for the pointer, that is really interesting!

Kind regards,
Nicolas


> I will send v2 later, where I made some more efforts
> to fix several corner cases even if that is not perfect.
>
>
>
>
>
>
>
>
> --
> Best Regards
> Masahiro Yamada


Attachments:
(No filename) (2.75 kB)
signature.asc (849.00 B)
Download all attachments

2023-04-02 02:29:38

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH] kbuild: fix package build error due to broken symlinks

On Sun, Apr 2, 2023 at 2:53 AM Nicolas Schier <[email protected]> wrote:
>
> On Sat, Apr 01, 2023 at 11:55:44PM +0900 Masahiro Yamada wrote:
> > On Thu, Mar 30, 2023 at 5:02 AM Nicolas Schier <[email protected]> wrote:
> > >
> > > On Sat 25 Mar 2023 23:19:09 GMT, Masahiro Yamada wrote:
> > > > 'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in
> > > > a dirty source tree. Handle symlinks properly, and also, keep the
> > > > executable permission.
> > > >
> > > > Fixes: 05e96e96a315 ("kbuild: use git-archive for source package creation")
> > > > Signed-off-by: Masahiro Yamada <[email protected]>
> > > > ---
> > > >
> > > > scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++---------
> > > > 1 file changed, 27 insertions(+), 9 deletions(-)
> > > >
> > > > diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
> > > > index f842ab50a780..23551de92e1b 100755
> > > > --- a/scripts/package/gen-diff-patch
> > > > +++ b/scripts/package/gen-diff-patch
> > > > @@ -23,16 +23,34 @@ fi
> > > > git -C ${srctree} status --porcelain --untracked-files=all |
> > > > while read stat path
> > > > do
> > > > - if [ "${stat}" = '??' ]; then
> > > > -
> > > > - if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
> > > > - ! head -n1 .tmp_diff | grep -q "Binary files"; then
> > > > - {
> > > > - echo "--- /dev/null"
> > > > - echo "+++ linux/$path"
> > > > - cat .tmp_diff | tail -n +3
> > > > - } >> ${untracked_patch}
> > > > + if [ "${stat}" != '??' ]; then
> > > > + continue
> > > > + fi
> > > > +
> > > > + if [ -L "${path}" ]; then
> > > > + {
> > > > + echo "diff --git a/${path} b/${path}"
> > > > + echo "new file mode 120000"
> > > > + echo "--- /dev/null"
> > > > + echo "+++ b/$path"
> > > > + echo "@@ -0,0 +1 @@"
> > > > + printf "+"; readlink ${path}
> > >
> > > Better quote "${path}"?
> >
> >
> > Thanks for the suggestion.
> >
> > Quoting variables are correct in most cases.
> > But, that is not enough to generate a valid
> > patch when a file path contains spaces.
> >
> >
> >
> > 'git format-patch' produces a patch that
> > is accepted by GNU patch and also by dpkg-source.
> >
> > I learned a trick from GIT source code.
> >
> >
> > If you are interested, what GIT does [1].
> >
> >
> > [1] https://github.com/git/git/commit/1a9eb3b9d50367bee8fe85022684d812816fe531
>
> thanks for the pointer, that is really interesting!
>
> Kind regards,
> Nicolas
>
>
> > I will send v2 later, where I made some more efforts
> > to fix several corner cases even if that is not perfect.


I wrote a patch (attached to this email) but I am
still thinking if this is worth the complicated scripting...

Another approach would be to stop caring untracked files.


--
Best Regards
Masahiro Yamada


Attachments:
0001-kbuild-make-package-builds-more-robust.patch (3.47 kB)