2020-05-03 00:14:11

by Changbin Du

[permalink] [raw]
Subject: [PATCH v4] streamline_config.pl: add LMC_KEEP to preserve some kconfigs

Sometimes it is useful to preserve batches of configs when making
localmodconfig. For example, I usually don't want any usb and fs
modules to be disabled. Now we can do it by:

$ make LMC_KEEP="drivers/usb;fs" localmodconfig

Signed-off-by: Changbin Du <[email protected]>

---
v4: fix typo.
v3: rename LOCALMODCONFIG_PRESERVE to shorter LMC_KEEP.
v2: fix typo in documentation. (Randy Dunlap)
---
Documentation/admin-guide/README.rst | 8 +++++++-
scripts/kconfig/Makefile | 1 +
scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
3 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/README.rst b/Documentation/admin-guide/README.rst
index cc6151fc0845..1371deab8bc7 100644
--- a/Documentation/admin-guide/README.rst
+++ b/Documentation/admin-guide/README.rst
@@ -209,10 +209,16 @@ Configuring the kernel
store the lsmod of that machine into a file
and pass it in as a LSMOD parameter.

+ Also, you can preserve modules in certain folders
+ or kconfig files by specifying their paths in
+ parameter LMC_KEEP.
+
target$ lsmod > /tmp/mylsmod
target$ scp /tmp/mylsmod host:/tmp

- host$ make LSMOD=/tmp/mylsmod localmodconfig
+ host$ make LSMOD=/tmp/mylsmod \
+ LMC_KEEP="drivers/usb;drivers/gpu;fs" \
+ localmodconfig

The above also works when cross compiling.

diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index c9d0a4a8efb3..e0abbf5805f5 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -123,6 +123,7 @@ help:
@echo ' gconfig - Update current config utilising a GTK+ based front-end'
@echo ' oldconfig - Update current config utilising a provided .config as base'
@echo ' localmodconfig - Update current config disabling modules not loaded'
+ @echo ' except those preserved by LMC_KEEP environment variable'
@echo ' localyesconfig - Update current config converting local mods to core'
@echo ' defconfig - New config with default from ARCH supplied defconfig'
@echo ' savedefconfig - Save current config as ./defconfig (minimal config)'
diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index e2f8504f5a2d..d26543a807c9 100755
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -143,6 +143,7 @@ my %depends;
my %selects;
my %prompts;
my %objects;
+my %config2kfile;
my $var;
my $iflevel = 0;
my @ifdeps;
@@ -201,6 +202,7 @@ sub read_kconfig {
if (/^\s*(menu)?config\s+(\S+)\s*$/) {
$state = "NEW";
$config = $2;
+ $config2kfile{"CONFIG_$config"} = $kconfig;

# Add depends for 'if' nesting
for (my $i = 0; $i < $iflevel; $i++) {
@@ -592,6 +594,22 @@ while ($repeat) {

my %setconfigs;

+my @preserved_kconfigs;
+@preserved_kconfigs = split(/;/,$ENV{LMC_KEEP}) if (defined($ENV{LMC_KEEP}));
+
+sub in_preserved_kconfigs {
+ my $kconfig = $config2kfile{$_[0]};
+ if (!defined($kconfig)) {
+ return 0;
+ }
+ foreach my $excl (@preserved_kconfigs) {
+ if($kconfig =~ /^$excl/) {
+ return 1;
+ }
+ }
+ return 0;
+}
+
# Finally, read the .config file and turn off any module enabled that
# we could not find a reason to keep enabled.
foreach my $line (@config_file) {
@@ -644,6 +662,11 @@ foreach my $line (@config_file) {
}

if (/^(CONFIG.*)=(m|y)/) {
+ if (in_preserved_kconfigs($1)) {
+ dprint "Preserve config $1";
+ print;
+ next;
+ }
if (defined($configs{$1})) {
if ($localyesconfig) {
$setconfigs{$1} = 'y';
--
2.25.1


2020-05-07 03:30:06

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v4] streamline_config.pl: add LMC_KEEP to preserve some kconfigs

On Sun, May 3, 2020 at 9:11 AM Changbin Du <[email protected]> wrote:
>
> Sometimes it is useful to preserve batches of configs when making
> localmodconfig. For example, I usually don't want any usb and fs
> modules to be disabled. Now we can do it by:
>
> $ make LMC_KEEP="drivers/usb;fs" localmodconfig
>
> Signed-off-by: Changbin Du <[email protected]>
>
> ---
> v4: fix typo.
> v3: rename LOCALMODCONFIG_PRESERVE to shorter LMC_KEEP.
> v2: fix typo in documentation. (Randy Dunlap)
> ---


Personally, I do not mind the long LOCALMODCONFIG_PRESERVE,
but this tends to be bike-sheding.
I do not have a strong opinion.


> Documentation/admin-guide/README.rst | 8 +++++++-
> scripts/kconfig/Makefile | 1 +
> scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
> 3 files changed, 31 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/README.rst b/Documentation/admin-guide/README.rst
> index cc6151fc0845..1371deab8bc7 100644
> --- a/Documentation/admin-guide/README.rst
> +++ b/Documentation/admin-guide/README.rst
> @@ -209,10 +209,16 @@ Configuring the kernel
> store the lsmod of that machine into a file
> and pass it in as a LSMOD parameter.
>
> + Also, you can preserve modules in certain folders
> + or kconfig files by specifying their paths in
> + parameter LMC_KEEP.
> +
> target$ lsmod > /tmp/mylsmod
> target$ scp /tmp/mylsmod host:/tmp
>
> - host$ make LSMOD=/tmp/mylsmod localmodconfig
> + host$ make LSMOD=/tmp/mylsmod \
> + LMC_KEEP="drivers/usb;drivers/gpu;fs" \


This might be another bike-sheding item, but
can you use a space for the delimiter?


LMC_KEEP="drivers/usb drivers/gpu fs"

If you pass multiple directories,
you will need to surround them with double-quotes.




> + localmodconfig
>
> The above also works when cross compiling.
>
> diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
> index c9d0a4a8efb3..e0abbf5805f5 100644
> --- a/scripts/kconfig/Makefile
> +++ b/scripts/kconfig/Makefile
> @@ -123,6 +123,7 @@ help:
> @echo ' gconfig - Update current config utilising a GTK+ based front-end'
> @echo ' oldconfig - Update current config utilising a provided .config as base'
> @echo ' localmodconfig - Update current config disabling modules not loaded'
> + @echo ' except those preserved by LMC_KEEP environment variable'
> @echo ' localyesconfig - Update current config converting local mods to core'
> @echo ' defconfig - New config with default from ARCH supplied defconfig'
> @echo ' savedefconfig - Save current config as ./defconfig (minimal config)'
> diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
> index e2f8504f5a2d..d26543a807c9 100755
> --- a/scripts/kconfig/streamline_config.pl
> +++ b/scripts/kconfig/streamline_config.pl
> @@ -143,6 +143,7 @@ my %depends;
> my %selects;
> my %prompts;
> my %objects;
> +my %config2kfile;
> my $var;
> my $iflevel = 0;
> my @ifdeps;
> @@ -201,6 +202,7 @@ sub read_kconfig {
> if (/^\s*(menu)?config\s+(\S+)\s*$/) {
> $state = "NEW";
> $config = $2;
> + $config2kfile{"CONFIG_$config"} = $kconfig;
>
> # Add depends for 'if' nesting
> for (my $i = 0; $i < $iflevel; $i++) {
> @@ -592,6 +594,22 @@ while ($repeat) {
>
> my %setconfigs;
>
> +my @preserved_kconfigs;
> +@preserved_kconfigs = split(/;/,$ENV{LMC_KEEP}) if (defined($ENV{LMC_KEEP}));

Maybe, you can do 'my' declaration and the assignment
in a single line?

Can you drop the 'if ...' conditional?


Does this work for you?

my @preserved_kconfigs = split(/;/,$ENV{LMC_KEEP});







> +
> +sub in_preserved_kconfigs {
> + my $kconfig = $config2kfile{$_[0]};
> + if (!defined($kconfig)) {
> + return 0;
> + }
> + foreach my $excl (@preserved_kconfigs) {
> + if($kconfig =~ /^$excl/) {
> + return 1;
> + }
> + }
> + return 0;
> +}
> +
> # Finally, read the .config file and turn off any module enabled that
> # we could not find a reason to keep enabled.
> foreach my $line (@config_file) {
> @@ -644,6 +662,11 @@ foreach my $line (@config_file) {
> }
>
> if (/^(CONFIG.*)=(m|y)/) {
> + if (in_preserved_kconfigs($1)) {
> + dprint "Preserve config $1";
> + print;
> + next;
> + }
> if (defined($configs{$1})) {
> if ($localyesconfig) {
> $setconfigs{$1} = 'y';
> --
> 2.25.1
>


--
Best Regards
Masahiro Yamada

2020-05-07 13:23:35

by Steven Rostedt

[permalink] [raw]
Subject: Re: [PATCH v4] streamline_config.pl: add LMC_KEEP to preserve some kconfigs

On Thu, 7 May 2020 12:19:57 +0900
Masahiro Yamada <[email protected]> wrote:

> On Sun, May 3, 2020 at 9:11 AM Changbin Du <[email protected]> wrote:
> >
> > Sometimes it is useful to preserve batches of configs when making
> > localmodconfig. For example, I usually don't want any usb and fs
> > modules to be disabled. Now we can do it by:
> >
> > $ make LMC_KEEP="drivers/usb;fs" localmodconfig
> >
> > Signed-off-by: Changbin Du <[email protected]>
> >
> > ---
> > v4: fix typo.
> > v3: rename LOCALMODCONFIG_PRESERVE to shorter LMC_KEEP.
> > v2: fix typo in documentation. (Randy Dunlap)
> > ---
>
>
> Personally, I do not mind the long LOCALMODCONFIG_PRESERVE,

Perhasp we allow both? ;-)

And just have one set to the other.

> but this tends to be bike-sheding.
> I do not have a strong opinion.
>
>
> > Documentation/admin-guide/README.rst | 8 +++++++-
> > scripts/kconfig/Makefile | 1 +
> > scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
> > 3 files changed, 31 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/README.rst b/Documentation/admin-guide/README.rst
> > index cc6151fc0845..1371deab8bc7 100644
> > --- a/Documentation/admin-guide/README.rst
> > +++ b/Documentation/admin-guide/README.rst
> > @@ -209,10 +209,16 @@ Configuring the kernel
> > store the lsmod of that machine into a file
> > and pass it in as a LSMOD parameter.
> >
> > + Also, you can preserve modules in certain folders
> > + or kconfig files by specifying their paths in
> > + parameter LMC_KEEP.
> > +
> > target$ lsmod > /tmp/mylsmod
> > target$ scp /tmp/mylsmod host:/tmp
> >
> > - host$ make LSMOD=/tmp/mylsmod localmodconfig
> > + host$ make LSMOD=/tmp/mylsmod \
> > + LMC_KEEP="drivers/usb;drivers/gpu;fs" \
>
>
> This might be another bike-sheding item, but
> can you use a space for the delimiter?
>
>
> LMC_KEEP="drivers/usb drivers/gpu fs"
>
> If you pass multiple directories,
> you will need to surround them with double-quotes.

I agree that spaces look better.

>
>
>
>
> > + localmodconfig
> >
> > The above also works when cross compiling.
> >
> > diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
> > index c9d0a4a8efb3..e0abbf5805f5 100644
> > --- a/scripts/kconfig/Makefile
> > +++ b/scripts/kconfig/Makefile
> > @@ -123,6 +123,7 @@ help:
> > @echo ' gconfig - Update current config utilising a GTK+ based front-end'
> > @echo ' oldconfig - Update current config utilising a provided .config as base'
> > @echo ' localmodconfig - Update current config disabling modules not loaded'
> > + @echo ' except those preserved by LMC_KEEP environment variable'
> > @echo ' localyesconfig - Update current config converting local mods to core'
> > @echo ' defconfig - New config with default from ARCH supplied defconfig'
> > @echo ' savedefconfig - Save current config as ./defconfig (minimal config)'
> > diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
> > index e2f8504f5a2d..d26543a807c9 100755
> > --- a/scripts/kconfig/streamline_config.pl
> > +++ b/scripts/kconfig/streamline_config.pl
> > @@ -143,6 +143,7 @@ my %depends;
> > my %selects;
> > my %prompts;
> > my %objects;
> > +my %config2kfile;
> > my $var;
> > my $iflevel = 0;
> > my @ifdeps;
> > @@ -201,6 +202,7 @@ sub read_kconfig {
> > if (/^\s*(menu)?config\s+(\S+)\s*$/) {
> > $state = "NEW";
> > $config = $2;
> > + $config2kfile{"CONFIG_$config"} = $kconfig;
> >
> > # Add depends for 'if' nesting
> > for (my $i = 0; $i < $iflevel; $i++) {
> > @@ -592,6 +594,22 @@ while ($repeat) {
> >
> > my %setconfigs;
> >
> > +my @preserved_kconfigs;
> > +@preserved_kconfigs = split(/;/,$ENV{LMC_KEEP}) if (defined($ENV{LMC_KEEP}));
>
> Maybe, you can do 'my' declaration and the assignment
> in a single line?
>
> Can you drop the 'if ...' conditional?
>
>
> Does this work for you?
>
> my @preserved_kconfigs = split(/;/,$ENV{LMC_KEEP});
>
>

Will an undefined warning happen if LMC_KEEP isn't defined?

-- Steve

2020-05-08 14:04:34

by Changbin Du

[permalink] [raw]
Subject: Re: [PATCH v4] streamline_config.pl: add LMC_KEEP to preserve some kconfigs

On Thu, May 07, 2020 at 09:18:07AM -0400, Steven Rostedt wrote:
> On Thu, 7 May 2020 12:19:57 +0900
> Masahiro Yamada <[email protected]> wrote:
>
> > On Sun, May 3, 2020 at 9:11 AM Changbin Du <[email protected]> wrote:
> > >
> > > Sometimes it is useful to preserve batches of configs when making
> > > localmodconfig. For example, I usually don't want any usb and fs
> > > modules to be disabled. Now we can do it by:
> > >
> > > $ make LMC_KEEP="drivers/usb;fs" localmodconfig
> > >
> > > Signed-off-by: Changbin Du <[email protected]>
> > >
> > > ---
> > > v4: fix typo.
> > > v3: rename LOCALMODCONFIG_PRESERVE to shorter LMC_KEEP.
> > > v2: fix typo in documentation. (Randy Dunlap)
> > > ---
> >
> >
> > Personally, I do not mind the long LOCALMODCONFIG_PRESERVE,
>
> Perhasp we allow both? ;-)
>
> And just have one set to the other.
>
> > but this tends to be bike-sheding.
> > I do not have a strong opinion.
> >
> >
> > > Documentation/admin-guide/README.rst | 8 +++++++-
> > > scripts/kconfig/Makefile | 1 +
> > > scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
> > > 3 files changed, 31 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/Documentation/admin-guide/README.rst b/Documentation/admin-guide/README.rst
> > > index cc6151fc0845..1371deab8bc7 100644
> > > --- a/Documentation/admin-guide/README.rst
> > > +++ b/Documentation/admin-guide/README.rst
> > > @@ -209,10 +209,16 @@ Configuring the kernel
> > > store the lsmod of that machine into a file
> > > and pass it in as a LSMOD parameter.
> > >
> > > + Also, you can preserve modules in certain folders
> > > + or kconfig files by specifying their paths in
> > > + parameter LMC_KEEP.
> > > +
> > > target$ lsmod > /tmp/mylsmod
> > > target$ scp /tmp/mylsmod host:/tmp
> > >
> > > - host$ make LSMOD=/tmp/mylsmod localmodconfig
> > > + host$ make LSMOD=/tmp/mylsmod \
> > > + LMC_KEEP="drivers/usb;drivers/gpu;fs" \
> >
> >
> > This might be another bike-sheding item, but
> > can you use a space for the delimiter?
> >
> >
> > LMC_KEEP="drivers/usb drivers/gpu fs"
> >
> > If you pass multiple directories,
> > you will need to surround them with double-quotes.
>
> I agree that spaces look better.
>
I am okay with space. But what about ":" which is used by shell PATH and many
others?

> >
> >
> >
> >
> > > + localmodconfig
> > >
> > > The above also works when cross compiling.
> > >
> > > diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
> > > index c9d0a4a8efb3..e0abbf5805f5 100644
> > > --- a/scripts/kconfig/Makefile
> > > +++ b/scripts/kconfig/Makefile
> > > @@ -123,6 +123,7 @@ help:
> > > @echo ' gconfig - Update current config utilising a GTK+ based front-end'
> > > @echo ' oldconfig - Update current config utilising a provided .config as base'
> > > @echo ' localmodconfig - Update current config disabling modules not loaded'
> > > + @echo ' except those preserved by LMC_KEEP environment variable'
> > > @echo ' localyesconfig - Update current config converting local mods to core'
> > > @echo ' defconfig - New config with default from ARCH supplied defconfig'
> > > @echo ' savedefconfig - Save current config as ./defconfig (minimal config)'
> > > diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
> > > index e2f8504f5a2d..d26543a807c9 100755
> > > --- a/scripts/kconfig/streamline_config.pl
> > > +++ b/scripts/kconfig/streamline_config.pl
> > > @@ -143,6 +143,7 @@ my %depends;
> > > my %selects;
> > > my %prompts;
> > > my %objects;
> > > +my %config2kfile;
> > > my $var;
> > > my $iflevel = 0;
> > > my @ifdeps;
> > > @@ -201,6 +202,7 @@ sub read_kconfig {
> > > if (/^\s*(menu)?config\s+(\S+)\s*$/) {
> > > $state = "NEW";
> > > $config = $2;
> > > + $config2kfile{"CONFIG_$config"} = $kconfig;
> > >
> > > # Add depends for 'if' nesting
> > > for (my $i = 0; $i < $iflevel; $i++) {
> > > @@ -592,6 +594,22 @@ while ($repeat) {
> > >
> > > my %setconfigs;
> > >
> > > +my @preserved_kconfigs;
> > > +@preserved_kconfigs = split(/;/,$ENV{LMC_KEEP}) if (defined($ENV{LMC_KEEP}));
> >
> > Maybe, you can do 'my' declaration and the assignment
> > in a single line?
> >
> > Can you drop the 'if ...' conditional?
> >
> >
> > Does this work for you?
> >
> > my @preserved_kconfigs = split(/;/,$ENV{LMC_KEEP});
> >
> >
>
> Will an undefined warning happen if LMC_KEEP isn't defined?
>
I just verifed there is no warning raised.

> -- Steve

--
Cheers,
Changbin Du

2020-05-08 16:17:24

by Steven Rostedt

[permalink] [raw]
Subject: Re: [PATCH v4] streamline_config.pl: add LMC_KEEP to preserve some kconfigs

On Fri, 8 May 2020 22:00:47 +0800
Changbin Du <[email protected]> wrote:

> > I agree that spaces look better.
> >
> I am okay with space. But what about ":" which is used by shell PATH and many
> others?

I'm OK with a ':'

-- Steve

2020-05-09 11:11:08

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v4] streamline_config.pl: add LMC_KEEP to preserve some kconfigs

On Fri, May 8, 2020 at 11:00 PM Changbin Du <[email protected]> wrote:
>
> On Thu, May 07, 2020 at 09:18:07AM -0400, Steven Rostedt wrote:
> > On Thu, 7 May 2020 12:19:57 +0900
> > Masahiro Yamada <[email protected]> wrote:
> >
> > > On Sun, May 3, 2020 at 9:11 AM Changbin Du <[email protected]> wrote:
> > > >
> > > > Sometimes it is useful to preserve batches of configs when making
> > > > localmodconfig. For example, I usually don't want any usb and fs
> > > > modules to be disabled. Now we can do it by:
> > > >
> > > > $ make LMC_KEEP="drivers/usb;fs" localmodconfig
> > > >
> > > > Signed-off-by: Changbin Du <[email protected]>
> > > >
> > > > ---
> > > > v4: fix typo.
> > > > v3: rename LOCALMODCONFIG_PRESERVE to shorter LMC_KEEP.
> > > > v2: fix typo in documentation. (Randy Dunlap)
> > > > ---
> > >
> > >
> > > Personally, I do not mind the long LOCALMODCONFIG_PRESERVE,
> >
> > Perhasp we allow both? ;-)
> >
> > And just have one set to the other.
> >
> > > but this tends to be bike-sheding.
> > > I do not have a strong opinion.
> > >
> > >
> > > > Documentation/admin-guide/README.rst | 8 +++++++-
> > > > scripts/kconfig/Makefile | 1 +
> > > > scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
> > > > 3 files changed, 31 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/Documentation/admin-guide/README.rst b/Documentation/admin-guide/README.rst
> > > > index cc6151fc0845..1371deab8bc7 100644
> > > > --- a/Documentation/admin-guide/README.rst
> > > > +++ b/Documentation/admin-guide/README.rst
> > > > @@ -209,10 +209,16 @@ Configuring the kernel
> > > > store the lsmod of that machine into a file
> > > > and pass it in as a LSMOD parameter.
> > > >
> > > > + Also, you can preserve modules in certain folders
> > > > + or kconfig files by specifying their paths in
> > > > + parameter LMC_KEEP.
> > > > +
> > > > target$ lsmod > /tmp/mylsmod
> > > > target$ scp /tmp/mylsmod host:/tmp
> > > >
> > > > - host$ make LSMOD=/tmp/mylsmod localmodconfig
> > > > + host$ make LSMOD=/tmp/mylsmod \
> > > > + LMC_KEEP="drivers/usb;drivers/gpu;fs" \
> > >
> > >
> > > This might be another bike-sheding item, but
> > > can you use a space for the delimiter?
> > >
> > >
> > > LMC_KEEP="drivers/usb drivers/gpu fs"
> > >
> > > If you pass multiple directories,
> > > you will need to surround them with double-quotes.
> >
> > I agree that spaces look better.
> >
> I am okay with space. But what about ":" which is used by shell PATH and many
> others?


I am not sure 'many others',
but you can pick what you like.





--
Best Regards
Masahiro Yamada