2016-03-03 22:58:00

by Michal Marek

[permalink] [raw]
Subject: Re: [PATCH v4 5/8] kbuild: add fine grained build dependencies for exported symbols

Dne 29.2.2016 v 04:48 Nicolas Pitre napsal(a):
> +# Filter out exported kernel symbol names advertised as warning pragmas
> +# by the preprocessor and write them to $(1). We must consider continuation
> +# lines as well: they start with a blank, or the preceeding line ends with
> +# a ':'. Anything else is passed through as is.
> +# See also __KSYM_DEP() in include/linux/export.h.
> +ksym_dep_filter = sed -n \
> + -e '1 {x; $$!d}' \
> + -e '/^ / {H; $$!d}' \
> + -e 'x; /:$$/ {x; H; $$!d; s/^/ /; x}' \
> + -e ':filter; /^.*KBUILD_AUTOKSYM_DEP: /! {p; b next}' \
> + -e 's//KSYM_/; s/\n.*//; w $(1)' \
> + -e ':next; $$!d' \
> + -e '1 q; s/^/ /; x; /^ /! b filter'

This is unreadable and it does not work with my gcc version. I get
dependencies like

$(wildcard include/config/ksym/simple/strtoull [enabled by default].h) \

Please use some other way, which does not require parsing the compiler
diagnostic messages. A straightforward solution is to do something
similar to genksyms: A separate preprocessor pass with -Dsomething that
leaves the EXPORT_SYMBOL statements alone and just collect their occurences.

Thanks,
Michal


2016-03-04 02:46:42

by Nicolas Pitre

[permalink] [raw]
Subject: Re: [PATCH v4 5/8] kbuild: add fine grained build dependencies for exported symbols

On Thu, 3 Mar 2016, Michal Marek wrote:

> Dne 29.2.2016 v 04:48 Nicolas Pitre napsal(a):
> > +# Filter out exported kernel symbol names advertised as warning pragmas
> > +# by the preprocessor and write them to $(1). We must consider continuation
> > +# lines as well: they start with a blank, or the preceeding line ends with
> > +# a ':'. Anything else is passed through as is.
> > +# See also __KSYM_DEP() in include/linux/export.h.
> > +ksym_dep_filter = sed -n \
> > + -e '1 {x; $$!d}' \
> > + -e '/^ / {H; $$!d}' \
> > + -e 'x; /:$$/ {x; H; $$!d; s/^/ /; x}' \
> > + -e ':filter; /^.*KBUILD_AUTOKSYM_DEP: /! {p; b next}' \
> > + -e 's//KSYM_/; s/\n.*//; w $(1)' \
> > + -e ':next; $$!d' \
> > + -e '1 q; s/^/ /; x; /^ /! b filter'
>
> This is unreadable and it does not work with my gcc version. I get
> dependencies like
>
> $(wildcard include/config/ksym/simple/strtoull [enabled by default].h) \

That's too bad. This had almost zero overhead as the preprocessor
didn't have to be executed twice.

Probably adding s/ .*//; before w $(1) would fix that. But I agree this
is fragile. What gcc version is this?

> Please use some other way, which does not require parsing the compiler
> diagnostic messages. A straightforward solution is to do something
> similar to genksyms: A separate preprocessor pass with -Dsomething that
> leaves the EXPORT_SYMBOL statements alone and just collect their occurences.

I went that route at some point. But here I need to extract those
EXPORT_SYMBOL() instances for all if_changed_dep uses. That includes
cc_o_c, cc_s_c, cc_i_c, cc_lst_c, as_s_S, as_o_S, etc. etc. I need the
corresponding preprocessor flags without the usual target flags. Let's
see what I can do that is more readable than a sed script.

Oh and who said that a sed script had to, or could be readable? ;-)


Nicolas

2016-03-04 10:11:07

by Michal Marek

[permalink] [raw]
Subject: Re: [PATCH v4 5/8] kbuild: add fine grained build dependencies for exported symbols

On 2016-03-04 03:46, Nicolas Pitre wrote:
> On Thu, 3 Mar 2016, Michal Marek wrote:
>
>> Dne 29.2.2016 v 04:48 Nicolas Pitre napsal(a):
>>> +# Filter out exported kernel symbol names advertised as warning pragmas
>>> +# by the preprocessor and write them to $(1). We must consider continuation
>>> +# lines as well: they start with a blank, or the preceeding line ends with
>>> +# a ':'. Anything else is passed through as is.
>>> +# See also __KSYM_DEP() in include/linux/export.h.
>>> +ksym_dep_filter = sed -n \
>>> + -e '1 {x; $$!d}' \
>>> + -e '/^ / {H; $$!d}' \
>>> + -e 'x; /:$$/ {x; H; $$!d; s/^/ /; x}' \
>>> + -e ':filter; /^.*KBUILD_AUTOKSYM_DEP: /! {p; b next}' \
>>> + -e 's//KSYM_/; s/\n.*//; w $(1)' \
>>> + -e ':next; $$!d' \
>>> + -e '1 q; s/^/ /; x; /^ /! b filter'
>>
>> This is unreadable and it does not work with my gcc version. I get
>> dependencies like
>>
>> $(wildcard include/config/ksym/simple/strtoull [enabled by default].h) \
>
> That's too bad. This had almost zero overhead as the preprocessor
> didn't have to be executed twice.

I know you are trying to collect the exports in a single pass, but
parsing the compiler warnings is really fragile.


> Probably adding s/ .*//; before w $(1) would fix that. But I agree this
> is fragile. What gcc version is this?

$ gcc --version
gcc (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch revision 212064]
...

The output looks like this:
$ echo '_Pragma("GCC warning \"KBUILD_AUTOKSYM_DEP: test\"")' | gcc
-Wall -c -xc -
<stdin>:1:13: warning: KBUILD_AUTOKSYM_DEP: test [enabled by default]

Clang output is similar, except that the text appears twice in the output:
$ echo '_Pragma("GCC warning \"KBUILD_AUTOKSYM_DEP: test\"")' | clang
-Wall -c -xc -
<stdin>:1:1: warning: KBUILD_AUTOKSYM_DEP: test [-W#pragma-messages]
_Pragma("GCC warning \"KBUILD_AUTOKSYM_DEP: test\"")
^
<scratch space>:2:6: note: expanded from here
GCC warning "KBUILD_AUTOKSYM_DEP: test"
^

What is worse, older gccs do not support the GCC warning pragma:
$ gcc --version
gcc (GCC) 4.1.2 20070115 (SUSE Linux)
...
$ echo '_Pragma("GCC warning \"KBUILD_AUTOKSYM_DEP: test\"")' | gcc
-Wall -c -xc -
<stdin>:1: warning: ignoring #pragma GCC warning


> Oh and who said that a sed script had to, or could be readable? ;-)

I guess I could live with a non-redable sed script embedded in a
Makefile, if I knew that it won't need updating for each new compiler
release :).

Michal