2013-06-06 17:06:07

by Kyle McMartin

[permalink] [raw]
Subject: [PATCH] Kbuild: pass headers to headers_install.sh on stdin

While using make V=1 to test some things, I noticed on our builders that
headers_install was failing because the argument list to /bin/sh was too
long. Working around it is slightly kludgy...

First, passing the args list to headers_install.sh via stdin instead of
as arguments.
Secondly, filtering $srctree out of $input-files to reduce the length of
the args list to the subshell.
Lastly, re-add $srctree when we loop over $input-files and pass the file
list back through to headers_install.sh.

You can reproduce the issue locally by putting your build dir in a
long path.

make[2]: execvp: /bin/sh: Argument list too long
make[2]: ***
[/home/kyle/A(lots of times)/linux/usr/include/linux/.install]
Error 127
make[1]: *** [linux] Error 2
make: *** [headers_install] Error 2

Signed-off-by: Kyle McMartin <[email protected]>

--- a/scripts/Makefile.headersinst
+++ b/scripts/Makefile.headersinst
@@ -59,6 +59,8 @@ input-files := $(foreach hdr, $(header-y), \
$(wildcard $(gendir)/$(hdr)), \
$(error Missing generated UAPI file $(gendir)/$(hdr)) \
))
+input-files := $(foreach hdr, $(input-files), \
+ $(subst $(srctree)/,,$(hdr)))

# Work out what needs to be removed
oldheaders := $(patsubst $(installdir)/%,%,$(wildcard $(installdir)/*.h))
@@ -72,7 +74,9 @@ printdir = $(patsubst $(INSTALL_HDR_PATH)/%/,%,$(dir $@))
quiet_cmd_install = INSTALL $(printdir) ($(words $(all-files))\
file$(if $(word 2, $(all-files)),s))
cmd_install = \
- $(CONFIG_SHELL) $< $(installdir) $(input-files); \
+ for f in $(input-files); do \
+ echo "$(srctree)/$$f"; \
+ done | $(CONFIG_SHELL) $< $(installdir); \
for F in $(wrapper-files); do \
echo "\#include <asm-generic/$$F>" > $(installdir)/$$F; \
done; \
diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
index 643764f..4e53688 100644
--- a/scripts/headers_install.sh
+++ b/scripts/headers_install.sh
@@ -2,7 +2,7 @@

if [ $# -lt 1 ]
then
- echo "Usage: headers_install.sh OUTDIR [FILES...]
+ echo "Usage: FILES | headers_install.sh OUTDIR
echo
echo "Prepares kernel header files for use by user space, by removing"
echo "all compiler.h definitions and #includes, removing any"
@@ -10,7 +10,7 @@ then
echo "asm/inline/volatile keywords."
echo
echo "OUTDIR: directory to write each userspace header FILE to."
- echo "FILES: list of header files to operate on."
+ echo "FILES: list of header files to operate on, passed stdin."

exit 1
fi
@@ -18,13 +18,12 @@ fi
# Grab arguments

OUTDIR="$1"
-shift

# Iterate through files listed on command line

FILE=
trap 'rm -f "$OUTDIR/$FILE" "$OUTDIR/$FILE.sed"' EXIT
-for i in "$@"
+while read i
do
FILE="$(basename "$i")"
sed -r \


2013-06-06 17:42:11

by Michal Marek

[permalink] [raw]
Subject: Re: [PATCH] Kbuild: pass headers to headers_install.sh on stdin

Dne 6.6.2013 19:05, Kyle McMartin napsal(a):
> While using make V=1 to test some things, I noticed on our builders that
> headers_install was failing because the argument list to /bin/sh was too
> long. Working around it is slightly kludgy...

This is already fixed with


commit c0ff68f1611d6855a06d672989ad5cfea160a4eb
Author: Nicolas Dichtel <[email protected]>
Date: Mon Apr 29 14:15:51 2013 +0200

kbuild: fix make headers_install when path is too long


in linux-next.

Michal

2013-06-06 17:55:38

by Kyle McMartin

[permalink] [raw]
Subject: Re: [PATCH] Kbuild: pass headers to headers_install.sh on stdin

On Thu, Jun 06, 2013 at 07:41:57PM +0200, Michal Marek wrote:
> Dne 6.6.2013 19:05, Kyle McMartin napsal(a):
> > While using make V=1 to test some things, I noticed on our builders that
> > headers_install was failing because the argument list to /bin/sh was too
> > long. Working around it is slightly kludgy...
>
> This is already fixed with
>
>
> commit c0ff68f1611d6855a06d672989ad5cfea160a4eb
> Author: Nicolas Dichtel <[email protected]>
> Date: Mon Apr 29 14:15:51 2013 +0200
>
> kbuild: fix make headers_install when path is too long
>
>
> in linux-next.
>

OK, I guess.

--Kyle

2013-06-06 17:59:24

by Josh Boyer

[permalink] [raw]
Subject: Re: [PATCH] Kbuild: pass headers to headers_install.sh on stdin

On Thu, Jun 6, 2013 at 1:55 PM, Kyle McMartin <[email protected]> wrote:
> On Thu, Jun 06, 2013 at 07:41:57PM +0200, Michal Marek wrote:
>> Dne 6.6.2013 19:05, Kyle McMartin napsal(a):
>> > While using make V=1 to test some things, I noticed on our builders that
>> > headers_install was failing because the argument list to /bin/sh was too
>> > long. Working around it is slightly kludgy...
>>
>> This is already fixed with
>>
>>
>> commit c0ff68f1611d6855a06d672989ad5cfea160a4eb
>> Author: Nicolas Dichtel <[email protected]>
>> Date: Mon Apr 29 14:15:51 2013 +0200
>>
>> kbuild: fix make headers_install when path is too long
>>
>>
>> in linux-next.
>>
>
> OK, I guess.

Wait... not OK. This is broken now. People are clearly having to fix
it again and wasting their time because a fix is already available.
Why is a fix for it sitting in linux-next? Please get that fix into
Linus' tree.

josh

2013-06-06 18:43:10

by Michal Marek

[permalink] [raw]
Subject: Re: [PATCH] Kbuild: pass headers to headers_install.sh on stdin

Dne 6.6.2013 19:59, Josh Boyer napsal(a):
> On Thu, Jun 6, 2013 at 1:55 PM, Kyle McMartin <[email protected]> wrote:
>> On Thu, Jun 06, 2013 at 07:41:57PM +0200, Michal Marek wrote:
>>> Dne 6.6.2013 19:05, Kyle McMartin napsal(a):
>>>> While using make V=1 to test some things, I noticed on our builders that
>>>> headers_install was failing because the argument list to /bin/sh was too
>>>> long. Working around it is slightly kludgy...
>>>
>>> This is already fixed with
>>>
>>>
>>> commit c0ff68f1611d6855a06d672989ad5cfea160a4eb
>>> Author: Nicolas Dichtel <[email protected]>
>>> Date: Mon Apr 29 14:15:51 2013 +0200
>>>
>>> kbuild: fix make headers_install when path is too long
>>>
>>>
>>> in linux-next.
>>>
>>
>> OK, I guess.
>
> Wait... not OK. This is broken now. People are clearly having to fix
> it again and wasting their time because a fix is already available.
> Why is a fix for it sitting in linux-next? Please get that fix into
> Linus' tree.

This bug has been there for about five years (as far as I can tell, it
was introduced by 7712401). It is a coincidence that two people report
it now within a couple of weeks, but IMO this does not make the bug so
urgent that it can't wait for the next -rc1.

Michal