2020-05-01 02:39:12

by Changbin Du

[permalink] [raw]
Subject: [PATCH] streamline_config.pl: add LOCALMODCONFIG_PRESERVE to preserve some kconfigs

Sometimes it is useful to preserve batches of configs when making
localmodconfig. For example, I usually don't want any usb and fs
modules to be disabled. Now we can do it by:

$ make LOCALMODCONFIG_PRESERVE="drivers/usb;fs" localmodconfig

Signed-off-by: Changbin Du <[email protected]>
---
Documentation/admin-guide/README.rst | 8 +++++++-
scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
2 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/README.rst b/Documentation/admin-guide/README.rst
index cc6151fc0845..6deff95362f8 100644
--- a/Documentation/admin-guide/README.rst
+++ b/Documentation/admin-guide/README.rst
@@ -209,10 +209,16 @@ Configuring the kernel
store the lsmod of that machine into a file
and pass it in as a LSMOD parameter.

+ Also, you can preserve modules in certen folders
+ or kconfig files by spcifying there paths in
+ parameter LOCALMODCONFIG_PRESERVE.
+
target$ lsmod > /tmp/mylsmod
target$ scp /tmp/mylsmod host:/tmp

- host$ make LSMOD=/tmp/mylsmod localmodconfig
+ host$ make LSMOD=/tmp/mylsmod \
+ LOCALMODCONFIG_PRESERVE="drivers/usb;drivers/gpu;fs" \
+ localmodconfig

The above also works when cross compiling.

diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index e2f8504f5a2d..ab5d1e10a5d0 100755
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -143,6 +143,7 @@ my %depends;
my %selects;
my %prompts;
my %objects;
+my %config2kfile;
my $var;
my $iflevel = 0;
my @ifdeps;
@@ -201,6 +202,7 @@ sub read_kconfig {
if (/^\s*(menu)?config\s+(\S+)\s*$/) {
$state = "NEW";
$config = $2;
+ $config2kfile{"CONFIG_$config"} = $kconfig;

# Add depends for 'if' nesting
for (my $i = 0; $i < $iflevel; $i++) {
@@ -592,6 +594,22 @@ while ($repeat) {

my %setconfigs;

+my @presevered_kconfigs;
+@presevered_kconfigs = split(/;/,$ENV{LOCALMODCONFIG_PRESERVE}) if (defined($ENV{LOCALMODCONFIG_PRESERVE}));
+
+sub in_presevered_kconfigs {
+ my $kconfig = $config2kfile{$_[0]};
+ if (!defined($kconfig)) {
+ return 0;
+ }
+ foreach my $excl (@presevered_kconfigs) {
+ if($kconfig =~ /^$excl/) {
+ return 1;
+ }
+ }
+ return 0;
+}
+
# Finally, read the .config file and turn off any module enabled that
# we could not find a reason to keep enabled.
foreach my $line (@config_file) {
@@ -644,6 +662,11 @@ foreach my $line (@config_file) {
}

if (/^(CONFIG.*)=(m|y)/) {
+ if (in_presevered_kconfigs($1)) {
+ dprint "Preserve config $1";
+ print;
+ next;
+ }
if (defined($configs{$1})) {
if ($localyesconfig) {
$setconfigs{$1} = 'y';
--
2.25.1


2020-05-01 16:53:09

by Randy Dunlap

[permalink] [raw]
Subject: Re: [PATCH] streamline_config.pl: add LOCALMODCONFIG_PRESERVE to preserve some kconfigs

On 4/30/20 7:37 PM, Changbin Du wrote:
> Sometimes it is useful to preserve batches of configs when making
> localmodconfig. For example, I usually don't want any usb and fs
> modules to be disabled. Now we can do it by:
>
> $ make LOCALMODCONFIG_PRESERVE="drivers/usb;fs" localmodconfig
>
> Signed-off-by: Changbin Du <[email protected]>
> ---
> Documentation/admin-guide/README.rst | 8 +++++++-
> scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
> 2 files changed, 30 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/README.rst b/Documentation/admin-guide/README.rst
> index cc6151fc0845..6deff95362f8 100644
> --- a/Documentation/admin-guide/README.rst
> +++ b/Documentation/admin-guide/README.rst
> @@ -209,10 +209,16 @@ Configuring the kernel
> store the lsmod of that machine into a file
> and pass it in as a LSMOD parameter.
>
> + Also, you can preserve modules in certen folders

typo: certain

> + or kconfig files by spcifying there paths in

again: specifying their

> + parameter LOCALMODCONFIG_PRESERVE.
> +
> target$ lsmod > /tmp/mylsmod
> target$ scp /tmp/mylsmod host:/tmp
>
> - host$ make LSMOD=/tmp/mylsmod localmodconfig
> + host$ make LSMOD=/tmp/mylsmod \
> + LOCALMODCONFIG_PRESERVE="drivers/usb;drivers/gpu;fs" \
> + localmodconfig
>
> The above also works when cross compiling.
>
> diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
> index e2f8504f5a2d..ab5d1e10a5d0 100755
> --- a/scripts/kconfig/streamline_config.pl
> +++ b/scripts/kconfig/streamline_config.pl
> @@ -143,6 +143,7 @@ my %depends;
> my %selects;
> my %prompts;
> my %objects;
> +my %config2kfile;
> my $var;
> my $iflevel = 0;
> my @ifdeps;
> @@ -201,6 +202,7 @@ sub read_kconfig {
> if (/^\s*(menu)?config\s+(\S+)\s*$/) {
> $state = "NEW";
> $config = $2;
> + $config2kfile{"CONFIG_$config"} = $kconfig;
>
> # Add depends for 'if' nesting
> for (my $i = 0; $i < $iflevel; $i++) {
> @@ -592,6 +594,22 @@ while ($repeat) {
>
> my %setconfigs;
>
> +my @presevered_kconfigs;
> +@presevered_kconfigs = split(/;/,$ENV{LOCALMODCONFIG_PRESERVE}) if (defined($ENV{LOCALMODCONFIG_PRESERVE}));
> +
> +sub in_presevered_kconfigs {
> + my $kconfig = $config2kfile{$_[0]};
> + if (!defined($kconfig)) {
> + return 0;
> + }
> + foreach my $excl (@presevered_kconfigs) {
> + if($kconfig =~ /^$excl/) {
> + return 1;
> + }
> + }
> + return 0;
> +}
> +
> # Finally, read the .config file and turn off any module enabled that
> # we could not find a reason to keep enabled.
> foreach my $line (@config_file) {
> @@ -644,6 +662,11 @@ foreach my $line (@config_file) {
> }
>
> if (/^(CONFIG.*)=(m|y)/) {
> + if (in_presevered_kconfigs($1)) {
> + dprint "Preserve config $1";
> + print;
> + next;
> + }
> if (defined($configs{$1})) {
> if ($localyesconfig) {
> $setconfigs{$1} = 'y';
>


--
~Randy
Reported-by: Randy Dunlap <[email protected]>

2020-05-01 17:09:23

by Steven Rostedt

[permalink] [raw]
Subject: Re: [PATCH] streamline_config.pl: add LOCALMODCONFIG_PRESERVE to preserve some kconfigs

On Fri, 1 May 2020 10:37:08 +0800
Changbin Du <[email protected]> wrote:

> Sometimes it is useful to preserve batches of configs when making
> localmodconfig. For example, I usually don't want any usb and fs
> modules to be disabled. Now we can do it by:
>
> $ make LOCALMODCONFIG_PRESERVE="drivers/usb;fs" localmodconfig

That's too much typing ;-) What about just "KEEP='drivers/usb;fs'"?


>
> Signed-off-by: Changbin Du <[email protected]>
> ---
> Documentation/admin-guide/README.rst | 8 +++++++-
> scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
> 2 files changed, 30 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/README.rst b/Documentation/admin-guide/README.rst
> index cc6151fc0845..6deff95362f8 100644
> --- a/Documentation/admin-guide/README.rst
> +++ b/Documentation/admin-guide/README.rst
> @@ -209,10 +209,16 @@ Configuring the kernel
> store the lsmod of that machine into a file
> and pass it in as a LSMOD parameter.
>
> + Also, you can preserve modules in certen folders
> + or kconfig files by spcifying there paths in
> + parameter LOCALMODCONFIG_PRESERVE.
> +
> target$ lsmod > /tmp/mylsmod
> target$ scp /tmp/mylsmod host:/tmp
>
> - host$ make LSMOD=/tmp/mylsmod localmodconfig
> + host$ make LSMOD=/tmp/mylsmod \
> + LOCALMODCONFIG_PRESERVE="drivers/usb;drivers/gpu;fs" \
> + localmodconfig
>
> The above also works when cross compiling.
>
> diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
> index e2f8504f5a2d..ab5d1e10a5d0 100755
> --- a/scripts/kconfig/streamline_config.pl
> +++ b/scripts/kconfig/streamline_config.pl
> @@ -143,6 +143,7 @@ my %depends;
> my %selects;
> my %prompts;
> my %objects;
> +my %config2kfile;
> my $var;
> my $iflevel = 0;
> my @ifdeps;
> @@ -201,6 +202,7 @@ sub read_kconfig {
> if (/^\s*(menu)?config\s+(\S+)\s*$/) {
> $state = "NEW";
> $config = $2;
> + $config2kfile{"CONFIG_$config"} = $kconfig;
>
> # Add depends for 'if' nesting
> for (my $i = 0; $i < $iflevel; $i++) {
> @@ -592,6 +594,22 @@ while ($repeat) {
>
> my %setconfigs;
>
> +my @presevered_kconfigs;
> +@presevered_kconfigs = split(/;/,$ENV{LOCALMODCONFIG_PRESERVE}) if (defined($ENV{LOCALMODCONFIG_PRESERVE}));
> +
> +sub in_presevered_kconfigs {
> + my $kconfig = $config2kfile{$_[0]};
> + if (!defined($kconfig)) {
> + return 0;
> + }
> + foreach my $excl (@presevered_kconfigs) {
> + if($kconfig =~ /^$excl/) {
> + return 1;
> + }
> + }
> + return 0;
> +}
> +
> # Finally, read the .config file and turn off any module enabled that
> # we could not find a reason to keep enabled.
> foreach my $line (@config_file) {
> @@ -644,6 +662,11 @@ foreach my $line (@config_file) {
> }
>
> if (/^(CONFIG.*)=(m|y)/) {
> + if (in_presevered_kconfigs($1)) {
> + dprint "Preserve config $1";
> + print;
> + next;
> + }
> if (defined($configs{$1})) {
> if ($localyesconfig) {
> $setconfigs{$1} = 'y';

I'll have to test it out, but I like the idea!

-- Steve

2020-05-02 13:28:19

by Changbin Du

[permalink] [raw]
Subject: Re: [PATCH] streamline_config.pl: add LOCALMODCONFIG_PRESERVE to preserve some kconfigs

On Fri, May 01, 2020 at 09:51:17AM -0700, Randy Dunlap wrote:
> On 4/30/20 7:37 PM, Changbin Du wrote:
> > Sometimes it is useful to preserve batches of configs when making
> > localmodconfig. For example, I usually don't want any usb and fs
> > modules to be disabled. Now we can do it by:
> >
> > $ make LOCALMODCONFIG_PRESERVE="drivers/usb;fs" localmodconfig
> >
> > Signed-off-by: Changbin Du <[email protected]>
> > ---
> > Documentation/admin-guide/README.rst | 8 +++++++-
> > scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
> > 2 files changed, 30 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/README.rst b/Documentation/admin-guide/README.rst
> > index cc6151fc0845..6deff95362f8 100644
> > --- a/Documentation/admin-guide/README.rst
> > +++ b/Documentation/admin-guide/README.rst
> > @@ -209,10 +209,16 @@ Configuring the kernel
> > store the lsmod of that machine into a file
> > and pass it in as a LSMOD parameter.
> >
> > + Also, you can preserve modules in certen folders
>
> typo: certain
>
> > + or kconfig files by spcifying there paths in
>
> again: specifying their
>
Thank you, I will fix them.

> > + parameter LOCALMODCONFIG_PRESERVE.
> > +
> > target$ lsmod > /tmp/mylsmod
> > target$ scp /tmp/mylsmod host:/tmp
> >
> > - host$ make LSMOD=/tmp/mylsmod localmodconfig
> > + host$ make LSMOD=/tmp/mylsmod \
> > + LOCALMODCONFIG_PRESERVE="drivers/usb;drivers/gpu;fs" \
> > + localmodconfig
> >
> > The above also works when cross compiling.
> >
> > diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
> > index e2f8504f5a2d..ab5d1e10a5d0 100755
> > --- a/scripts/kconfig/streamline_config.pl
> > +++ b/scripts/kconfig/streamline_config.pl
> > @@ -143,6 +143,7 @@ my %depends;
> > my %selects;
> > my %prompts;
> > my %objects;
> > +my %config2kfile;
> > my $var;
> > my $iflevel = 0;
> > my @ifdeps;
> > @@ -201,6 +202,7 @@ sub read_kconfig {
> > if (/^\s*(menu)?config\s+(\S+)\s*$/) {
> > $state = "NEW";
> > $config = $2;
> > + $config2kfile{"CONFIG_$config"} = $kconfig;
> >
> > # Add depends for 'if' nesting
> > for (my $i = 0; $i < $iflevel; $i++) {
> > @@ -592,6 +594,22 @@ while ($repeat) {
> >
> > my %setconfigs;
> >
> > +my @presevered_kconfigs;
> > +@presevered_kconfigs = split(/;/,$ENV{LOCALMODCONFIG_PRESERVE}) if (defined($ENV{LOCALMODCONFIG_PRESERVE}));
> > +
> > +sub in_presevered_kconfigs {
> > + my $kconfig = $config2kfile{$_[0]};
> > + if (!defined($kconfig)) {
> > + return 0;
> > + }
> > + foreach my $excl (@presevered_kconfigs) {
> > + if($kconfig =~ /^$excl/) {
> > + return 1;
> > + }
> > + }
> > + return 0;
> > +}
> > +
> > # Finally, read the .config file and turn off any module enabled that
> > # we could not find a reason to keep enabled.
> > foreach my $line (@config_file) {
> > @@ -644,6 +662,11 @@ foreach my $line (@config_file) {
> > }
> >
> > if (/^(CONFIG.*)=(m|y)/) {
> > + if (in_presevered_kconfigs($1)) {
> > + dprint "Preserve config $1";
> > + print;
> > + next;
> > + }
> > if (defined($configs{$1})) {
> > if ($localyesconfig) {
> > $setconfigs{$1} = 'y';
> >
>
>
> --
> ~Randy
> Reported-by: Randy Dunlap <[email protected]>

--
Cheers,
Changbin Du

2020-05-02 13:32:53

by Changbin Du

[permalink] [raw]
Subject: Re: [PATCH] streamline_config.pl: add LOCALMODCONFIG_PRESERVE to preserve some kconfigs

On Fri, May 01, 2020 at 01:07:29PM -0400, Steven Rostedt wrote:
> On Fri, 1 May 2020 10:37:08 +0800
> Changbin Du <[email protected]> wrote:
>
> > Sometimes it is useful to preserve batches of configs when making
> > localmodconfig. For example, I usually don't want any usb and fs
> > modules to be disabled. Now we can do it by:
> >
> > $ make LOCALMODCONFIG_PRESERVE="drivers/usb;fs" localmodconfig
>
> That's too much typing ;-) What about just "KEEP='drivers/usb;fs'"?
>
I think we'd better use a long name since it will be passed to the entire kbuild.
And we alreay have one named LOCALMODCONFIG_DEBUG. The prefix LOCALMODCONFIG_
can help to avoid namespace pollution.

>
> >
> > Signed-off-by: Changbin Du <[email protected]>
> > ---
> > Documentation/admin-guide/README.rst | 8 +++++++-
> > scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
> > 2 files changed, 30 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/README.rst b/Documentation/admin-guide/README.rst
> > index cc6151fc0845..6deff95362f8 100644
> > --- a/Documentation/admin-guide/README.rst
> > +++ b/Documentation/admin-guide/README.rst
> > @@ -209,10 +209,16 @@ Configuring the kernel
> > store the lsmod of that machine into a file
> > and pass it in as a LSMOD parameter.
> >
> > + Also, you can preserve modules in certen folders
> > + or kconfig files by spcifying there paths in
> > + parameter LOCALMODCONFIG_PRESERVE.
> > +
> > target$ lsmod > /tmp/mylsmod
> > target$ scp /tmp/mylsmod host:/tmp
> >
> > - host$ make LSMOD=/tmp/mylsmod localmodconfig
> > + host$ make LSMOD=/tmp/mylsmod \
> > + LOCALMODCONFIG_PRESERVE="drivers/usb;drivers/gpu;fs" \
> > + localmodconfig
> >
> > The above also works when cross compiling.
> >
> > diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
> > index e2f8504f5a2d..ab5d1e10a5d0 100755
> > --- a/scripts/kconfig/streamline_config.pl
> > +++ b/scripts/kconfig/streamline_config.pl
> > @@ -143,6 +143,7 @@ my %depends;
> > my %selects;
> > my %prompts;
> > my %objects;
> > +my %config2kfile;
> > my $var;
> > my $iflevel = 0;
> > my @ifdeps;
> > @@ -201,6 +202,7 @@ sub read_kconfig {
> > if (/^\s*(menu)?config\s+(\S+)\s*$/) {
> > $state = "NEW";
> > $config = $2;
> > + $config2kfile{"CONFIG_$config"} = $kconfig;
> >
> > # Add depends for 'if' nesting
> > for (my $i = 0; $i < $iflevel; $i++) {
> > @@ -592,6 +594,22 @@ while ($repeat) {
> >
> > my %setconfigs;
> >
> > +my @presevered_kconfigs;
> > +@presevered_kconfigs = split(/;/,$ENV{LOCALMODCONFIG_PRESERVE}) if (defined($ENV{LOCALMODCONFIG_PRESERVE}));
> > +
> > +sub in_presevered_kconfigs {
> > + my $kconfig = $config2kfile{$_[0]};
> > + if (!defined($kconfig)) {
> > + return 0;
> > + }
> > + foreach my $excl (@presevered_kconfigs) {
> > + if($kconfig =~ /^$excl/) {
> > + return 1;
> > + }
> > + }
> > + return 0;
> > +}
> > +
> > # Finally, read the .config file and turn off any module enabled that
> > # we could not find a reason to keep enabled.
> > foreach my $line (@config_file) {
> > @@ -644,6 +662,11 @@ foreach my $line (@config_file) {
> > }
> >
> > if (/^(CONFIG.*)=(m|y)/) {
> > + if (in_presevered_kconfigs($1)) {
> > + dprint "Preserve config $1";
> > + print;
> > + next;
> > + }
> > if (defined($configs{$1})) {
> > if ($localyesconfig) {
> > $setconfigs{$1} = 'y';
>
> I'll have to test it out, but I like the idea!
>
> -- Steve

--
Cheers,
Changbin Du

2020-05-02 13:42:08

by Steven Rostedt

[permalink] [raw]
Subject: Re: [PATCH] streamline_config.pl: add LOCALMODCONFIG_PRESERVE to preserve some kconfigs

On Sat, 2 May 2020 21:30:54 +0800
Changbin Du <[email protected]> wrote:

> > > Sometimes it is useful to preserve batches of configs when making
> > > localmodconfig. For example, I usually don't want any usb and fs
> > > modules to be disabled. Now we can do it by:
> > >
> > > $ make LOCALMODCONFIG_PRESERVE="drivers/usb;fs" localmodconfig
> >
> > That's too much typing ;-) What about just "KEEP='drivers/usb;fs'"?
> >
> I think we'd better use a long name since it will be passed to the entire kbuild.
> And we alreay have one named LOCALMODCONFIG_DEBUG. The prefix LOCALMODCONFIG_
> can help to avoid namespace pollution.

I politely disagree. Build options is not common. The
LOCALMODCONFIG_DEBUG is an environment variable, which I couldn't just
use DEBUG.

If you absolutely require a prefix, shorten it to LMC_ or something. I
already hate typing 'localmodconfig' once ;-)

-- Steve

2020-05-02 14:32:58

by Changbin Du

[permalink] [raw]
Subject: Re: [PATCH] streamline_config.pl: add LOCALMODCONFIG_PRESERVE to preserve some kconfigs

Hi, Steven,
On Sat, May 02, 2020 at 09:40:24AM -0400, Steven Rostedt wrote:
> On Sat, 2 May 2020 21:30:54 +0800
> Changbin Du <[email protected]> wrote:
>
> > > > Sometimes it is useful to preserve batches of configs when making
> > > > localmodconfig. For example, I usually don't want any usb and fs
> > > > modules to be disabled. Now we can do it by:
> > > >
> > > > $ make LOCALMODCONFIG_PRESERVE="drivers/usb;fs" localmodconfig
> > >
> > > That's too much typing ;-) What about just "KEEP='drivers/usb;fs'"?
> > >
> > I think we'd better use a long name since it will be passed to the entire kbuild.
> > And we alreay have one named LOCALMODCONFIG_DEBUG. The prefix LOCALMODCONFIG_
> > can help to avoid namespace pollution.
>
> I politely disagree. Build options is not common. The
> LOCALMODCONFIG_DEBUG is an environment variable, which I couldn't just
> use DEBUG.
>
This is what I am worried about since LOCALMODCONFIG_PRESERVE also is an
environment variable. A short name possibly overwrite internal kbuild
variables.

> If you absolutely require a prefix, shorten it to LMC_ or something. I
> already hate typing 'localmodconfig' once ;-)
>
I undstand your pain. This why I never type 'localmodconfig' but copy it
from 'make help' :). Maybe we can get more inputs from others.

> -- Steve

--
Cheers,
Changbin Du