2014-06-11 19:42:05

by Andy Lutomirski

[permalink] [raw]
Subject: [PATCH] x86,build: Fix make -jN modules_install install

Every few months, I forget why I type:

$ sudo make -j12 modules_install && sudo make -j12 install

instead of just:

$ sudo make -j12 modules_install install

I try the latter, it appears to work, and then my machine won't boot
because dracut got confused. This fixes it once and for all: if you
ask make to install modules and a kernel, you almost certainly want
the modules installed *first* so that your initramfs scripts can
find the modules.

Signed-off-by: Andy Lutomirski <[email protected]>
---
arch/x86/Makefile | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index 33f71b0..7280d28 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -240,6 +240,15 @@ PHONY += install
install:
$(Q)$(MAKE) $(build)=$(boot) $@

+# If installing modules and a kernel, it's very likely that some initramfs
+# script associated with installing the kernel will reference the modules,
+# so make sure that modules are installed first.
+ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
+ ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
+ install: modules_install
+ endif
+endif
+
PHONY += vdso_install
vdso_install:
$(Q)$(MAKE) $(build)=arch/x86/vdso $@
--
1.9.3


2014-06-11 19:44:58

by Sam Ravnborg

[permalink] [raw]
Subject: Re: [PATCH] x86,build: Fix make -jN modules_install install

On Wed, Jun 11, 2014 at 12:41:57PM -0700, Andy Lutomirski wrote:
> Every few months, I forget why I type:
>
> $ sudo make -j12 modules_install && sudo make -j12 install
>
> instead of just:
>
> $ sudo make -j12 modules_install install
>
> I try the latter, it appears to work, and then my machine won't boot
> because dracut got confused. This fixes it once and for all: if you
> ask make to install modules and a kernel, you almost certainly want
> the modules installed *first* so that your initramfs scripts can
> find the modules.

Is this problem x86 specific?

Sam

2014-06-11 19:46:46

by Andy Lutomirski

[permalink] [raw]
Subject: Re: [PATCH] x86,build: Fix make -jN modules_install install

On Wed, Jun 11, 2014 at 12:44 PM, Sam Ravnborg <[email protected]> wrote:
> On Wed, Jun 11, 2014 at 12:41:57PM -0700, Andy Lutomirski wrote:
>> Every few months, I forget why I type:
>>
>> $ sudo make -j12 modules_install && sudo make -j12 install
>>
>> instead of just:
>>
>> $ sudo make -j12 modules_install install
>>
>> I try the latter, it appears to work, and then my machine won't boot
>> because dracut got confused. This fixes it once and for all: if you
>> ask make to install modules and a kernel, you almost certainly want
>> the modules installed *first* so that your initramfs scripts can
>> find the modules.
>
> Is this problem x86 specific?

I don't know. But I also don't want to have 'make modules_install
install' on an arch without an install target appear to work, so I
don't know how to do it in the kbuild core. That is, I don't want to
define an install target; I just want to add a dependency if the
target is already there.

Also, this patch has a repeated ifneq. I'll fix it in v2.

>
> Sam



--
Andy Lutomirski
AMA Capital Management, LLC

2014-06-12 08:33:54

by Michal Marek

[permalink] [raw]
Subject: Re: [PATCH] x86,build: Fix make -jN modules_install install

Dne 11.6.2014 21:41, Andy Lutomirski napsal(a):
> Every few months, I forget why I type:
>
> $ sudo make -j12 modules_install && sudo make -j12 install
>
> instead of just:
>
> $ sudo make -j12 modules_install install
>
> I try the latter, it appears to work, and then my machine won't boot
> because dracut got confused. This fixes it once and for all: if you
> ask make to install modules and a kernel, you almost certainly want
> the modules installed *first* so that your initramfs scripts can
> find the modules.
>
> Signed-off-by: Andy Lutomirski <[email protected]>
> ---
> arch/x86/Makefile | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/arch/x86/Makefile b/arch/x86/Makefile
> index 33f71b0..7280d28 100644
> --- a/arch/x86/Makefile
> +++ b/arch/x86/Makefile
> @@ -240,6 +240,15 @@ PHONY += install
> install:
> $(Q)$(MAKE) $(build)=$(boot) $@
>
> +# If installing modules and a kernel, it's very likely that some initramfs
> +# script associated with installing the kernel will reference the modules,
> +# so make sure that modules are installed first.
> +ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
> + ifneq ($(filter modules_install,$(MAKECMDGOALS)),)

The two conditions are identical. Did you mean to check for "install" in
one of them?

Thanks,
Michal

2014-06-13 01:28:51

by Andy Lutomirski

[permalink] [raw]
Subject: Re: [PATCH] x86,build: Fix make -jN modules_install install

On Thu, Jun 12, 2014 at 1:33 AM, Michal Marek <[email protected]> wrote:
> Dne 11.6.2014 21:41, Andy Lutomirski napsal(a):
>> Every few months, I forget why I type:
>>
>> $ sudo make -j12 modules_install && sudo make -j12 install
>>
>> instead of just:
>>
>> $ sudo make -j12 modules_install install
>>
>> I try the latter, it appears to work, and then my machine won't boot
>> because dracut got confused. This fixes it once and for all: if you
>> ask make to install modules and a kernel, you almost certainly want
>> the modules installed *first* so that your initramfs scripts can
>> find the modules.
>>
>> Signed-off-by: Andy Lutomirski <[email protected]>
>> ---
>> arch/x86/Makefile | 9 +++++++++
>> 1 file changed, 9 insertions(+)
>>
>> diff --git a/arch/x86/Makefile b/arch/x86/Makefile
>> index 33f71b0..7280d28 100644
>> --- a/arch/x86/Makefile
>> +++ b/arch/x86/Makefile
>> @@ -240,6 +240,15 @@ PHONY += install
>> install:
>> $(Q)$(MAKE) $(build)=$(boot) $@
>>
>> +# If installing modules and a kernel, it's very likely that some initramfs
>> +# script associated with installing the kernel will reference the modules,
>> +# so make sure that modules are installed first.
>> +ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>> + ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>
> The two conditions are identical. Did you mean to check for "install" in
> one of them?

Yes, although I think that the check for "install" is actually unnecessary.

Is there some way to do this in the core kbuild?

--Andy

>
> Thanks,
> Michal



--
Andy Lutomirski
AMA Capital Management, LLC

2014-06-13 09:40:18

by Michal Marek

[permalink] [raw]
Subject: [PATCH] kbuild: Do not run modules_install and install in paralel

Based on a x86-only patch by Andy Lutomirski <[email protected]>

With modular kernels, 'make install' is going to need the installed
modules at some point to generate the initramfs.

Signed-off-by: Michal Marek <[email protected]>
---
Makefile | 6 ++++++
1 file changed, 6 insertions(+)

diff --git a/Makefile b/Makefile
index 7680d7c..7e5e483 100644
--- a/Makefile
+++ b/Makefile
@@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
endif
endif
endif
+# install and module_install need also be processed one by one
+ifneq ($(filter install,$(MAKECMDGOALS)),)
+ ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
+ mixed-targets := 1
+ endif
+endif

ifeq ($(mixed-targets),1)
# ===========================================================================
--
1.9.2

2014-06-13 09:45:20

by Michal Marek

[permalink] [raw]
Subject: Re: [PATCH] kbuild: Do not run modules_install and install in paralel

Dne 13.6.2014 11:39, Michal Marek napsal(a):
> Based on a x86-only patch by Andy Lutomirski <[email protected]>
>
> With modular kernels, 'make install' is going to need the installed
> modules at some point to generate the initramfs.
>
> Signed-off-by: Michal Marek <[email protected]>
> ---
> Makefile | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index 7680d7c..7e5e483 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
> endif
> endif
> endif
> +# install and module_install need also be processed one by one
> +ifneq ($(filter install,$(MAKECMDGOALS)),)
> + ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
> + mixed-targets := 1
> + endif
> +endif

Note that this version does not enforce the ordering, it just avoids the
interleaved execution. It can be added if desired.

Michal

2014-06-13 17:30:09

by Andy Lutomirski

[permalink] [raw]
Subject: Re: [PATCH] kbuild: Do not run modules_install and install in paralel

On Fri, Jun 13, 2014 at 2:45 AM, Michal Marek <[email protected]> wrote:
> Dne 13.6.2014 11:39, Michal Marek napsal(a):
>> Based on a x86-only patch by Andy Lutomirski <[email protected]>
>>
>> With modular kernels, 'make install' is going to need the installed
>> modules at some point to generate the initramfs.
>>
>> Signed-off-by: Michal Marek <[email protected]>
>> ---
>> Makefile | 6 ++++++
>> 1 file changed, 6 insertions(+)
>>
>> diff --git a/Makefile b/Makefile
>> index 7680d7c..7e5e483 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
>> endif
>> endif
>> endif
>> +# install and module_install need also be processed one by one
>> +ifneq ($(filter install,$(MAKECMDGOALS)),)
>> + ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>> + mixed-targets := 1
>> + endif
>> +endif
>
> Note that this version does not enforce the ordering, it just avoids the
> interleaved execution. It can be added if desired.

Hmm. This will fix 'make modules_install install' but will not fix
'make install modules_install'. I don't know how many people would
type the latter.

--Andy

2014-07-04 23:10:18

by Michal Marek

[permalink] [raw]
Subject: Re: [PATCH] kbuild: Do not run modules_install and install in paralel

Dne 13.6.2014 19:29, Andy Lutomirski napsal(a):
> On Fri, Jun 13, 2014 at 2:45 AM, Michal Marek <[email protected]> wrote:
>> Dne 13.6.2014 11:39, Michal Marek napsal(a):
>>> Based on a x86-only patch by Andy Lutomirski <[email protected]>
>>>
>>> With modular kernels, 'make install' is going to need the installed
>>> modules at some point to generate the initramfs.
>>>
>>> Signed-off-by: Michal Marek <[email protected]>
>>> ---
>>> Makefile | 6 ++++++
>>> 1 file changed, 6 insertions(+)
>>>
>>> diff --git a/Makefile b/Makefile
>>> index 7680d7c..7e5e483 100644
>>> --- a/Makefile
>>> +++ b/Makefile
>>> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
>>> endif
>>> endif
>>> endif
>>> +# install and module_install need also be processed one by one
>>> +ifneq ($(filter install,$(MAKECMDGOALS)),)
>>> + ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>>> + mixed-targets := 1
>>> + endif
>>> +endif
>>
>> Note that this version does not enforce the ordering, it just avoids the
>> interleaved execution. It can be added if desired.
>
> Hmm. This will fix 'make modules_install install' but will not fix
> 'make install modules_install'. I don't know how many people would
> type the latter.

Resuming a thread from last month -- As I said, it does not enforce the
ordering. So it's not as bullet-proof as your patch, but it works on all
architectures. I'm thinking about adding the patch to kbuild.git for
3.17. But if you want to fix the ordering as well, feel free to change
the __build_one_by_one rule to do this.

Thanks,
Michal

2014-07-04 23:46:18

by Andy Lutomirski

[permalink] [raw]
Subject: Re: [PATCH] kbuild: Do not run modules_install and install in paralel

On Fri, Jul 4, 2014 at 3:15 PM, Michal Marek <[email protected]> wrote:
> Dne 13.6.2014 19:29, Andy Lutomirski napsal(a):
>> On Fri, Jun 13, 2014 at 2:45 AM, Michal Marek <[email protected]> wrote:
>>> Dne 13.6.2014 11:39, Michal Marek napsal(a):
>>>> Based on a x86-only patch by Andy Lutomirski <[email protected]>
>>>>
>>>> With modular kernels, 'make install' is going to need the installed
>>>> modules at some point to generate the initramfs.
>>>>
>>>> Signed-off-by: Michal Marek <[email protected]>
>>>> ---
>>>> Makefile | 6 ++++++
>>>> 1 file changed, 6 insertions(+)
>>>>
>>>> diff --git a/Makefile b/Makefile
>>>> index 7680d7c..7e5e483 100644
>>>> --- a/Makefile
>>>> +++ b/Makefile
>>>> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
>>>> endif
>>>> endif
>>>> endif
>>>> +# install and module_install need also be processed one by one
>>>> +ifneq ($(filter install,$(MAKECMDGOALS)),)
>>>> + ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>>>> + mixed-targets := 1
>>>> + endif
>>>> +endif
>>>
>>> Note that this version does not enforce the ordering, it just avoids the
>>> interleaved execution. It can be added if desired.
>>
>> Hmm. This will fix 'make modules_install install' but will not fix
>> 'make install modules_install'. I don't know how many people would
>> type the latter.
>
> Resuming a thread from last month -- As I said, it does not enforce the
> ordering. So it's not as bullet-proof as your patch, but it works on all
> architectures. I'm thinking about adding the patch to kbuild.git for
> 3.17. But if you want to fix the ordering as well, feel free to change
> the __build_one_by_one rule to do this.

Sounds good to me.

Where's __build_one_by_one?

--Andy

>
> Thanks,
> Michal



--
Andy Lutomirski
AMA Capital Management, LLC

2014-07-07 11:03:45

by Michal Marek

[permalink] [raw]
Subject: Re: [PATCH] kbuild: Do not run modules_install and install in paralel

On 2014-07-05 01:45, Andy Lutomirski wrote:
> On Fri, Jul 4, 2014 at 3:15 PM, Michal Marek <[email protected]> wrote:
>> Dne 13.6.2014 19:29, Andy Lutomirski napsal(a):
>>> On Fri, Jun 13, 2014 at 2:45 AM, Michal Marek <[email protected]> wrote:
>>>> Dne 13.6.2014 11:39, Michal Marek napsal(a):
>>>>> Based on a x86-only patch by Andy Lutomirski <[email protected]>
>>>>>
>>>>> With modular kernels, 'make install' is going to need the installed
>>>>> modules at some point to generate the initramfs.
>>>>>
>>>>> Signed-off-by: Michal Marek <[email protected]>
>>>>> ---
>>>>> Makefile | 6 ++++++
>>>>> 1 file changed, 6 insertions(+)
>>>>>
>>>>> diff --git a/Makefile b/Makefile
>>>>> index 7680d7c..7e5e483 100644
>>>>> --- a/Makefile
>>>>> +++ b/Makefile
>>>>> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
>>>>> endif
>>>>> endif
>>>>> endif
>>>>> +# install and module_install need also be processed one by one
>>>>> +ifneq ($(filter install,$(MAKECMDGOALS)),)
>>>>> + ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>>>>> + mixed-targets := 1
>>>>> + endif
>>>>> +endif
>>>>
>>>> Note that this version does not enforce the ordering, it just avoids the
>>>> interleaved execution. It can be added if desired.
>>>
>>> Hmm. This will fix 'make modules_install install' but will not fix
>>> 'make install modules_install'. I don't know how many people would
>>> type the latter.
>>
>> Resuming a thread from last month -- As I said, it does not enforce the
>> ordering. So it's not as bullet-proof as your patch, but it works on all
>> architectures. I'm thinking about adding the patch to kbuild.git for
>> 3.17. But if you want to fix the ordering as well, feel free to change
>> the __build_one_by_one rule to do this.
>
> Sounds good to me.
>
> Where's __build_one_by_one?

A few lines below in the main Makefile:

513 ifeq ($(mixed-targets),1)
514 #
======================================================================
=====
515 # We're called with mixed targets (*config and build targets).
516 # Handle them one by one.
517
518 PHONY += $(MAKECMDGOALS) __build_one_by_one
519
520 $(filter-out __build_one_by_one, $(MAKECMDGOALS)):
__build_one_by_one
521 @:
522
523 __build_one_by_one:
524 $(Q)set -e; \
525 for i in $(MAKECMDGOALS); do \
526 $(MAKE) -f $(srctree)/Makefile $$i; \
527 done

Michal

2015-12-09 21:34:34

by Andy Lutomirski

[permalink] [raw]
Subject: Re: [PATCH] kbuild: Do not run modules_install and install in paralel

On Fri, Jun 13, 2014 at 2:39 AM, Michal Marek <[email protected]> wrote:
> Based on a x86-only patch by Andy Lutomirski <[email protected]>
>
> With modular kernels, 'make install' is going to need the installed
> modules at some point to generate the initramfs.
>
> Signed-off-by: Michal Marek <[email protected]>
> ---
> Makefile | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index 7680d7c..7e5e483 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
> endif
> endif
> endif
> +# install and module_install need also be processed one by one
> +ifneq ($(filter install,$(MAKECMDGOALS)),)
> + ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
> + mixed-targets := 1
> + endif
> +endif
>
> ifeq ($(mixed-targets),1)
> # ===========================================================================
> --
> 1.9.2
>

This patch seems to have gotten lost. Do you still like it? If so,
can someone apply it, please?

--Andy

2015-12-10 14:45:43

by Michal Marek

[permalink] [raw]
Subject: Re: [PATCH] kbuild: Do not run modules_install and install in paralel

Dne 9.12.2015 v 22:34 Andy Lutomirski napsal(a):
> On Fri, Jun 13, 2014 at 2:39 AM, Michal Marek <[email protected]> wrote:
>> Based on a x86-only patch by Andy Lutomirski <[email protected]>
>>
>> With modular kernels, 'make install' is going to need the installed
>> modules at some point to generate the initramfs.
>>
>> Signed-off-by: Michal Marek <[email protected]>
>> ---
>> Makefile | 6 ++++++
>> 1 file changed, 6 insertions(+)
>>
>> diff --git a/Makefile b/Makefile
>> index 7680d7c..7e5e483 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -503,6 +503,12 @@ ifeq ($(KBUILD_EXTMOD),)
>> endif
>> endif
>> endif
>> +# install and module_install need also be processed one by one
>> +ifneq ($(filter install,$(MAKECMDGOALS)),)
>> + ifneq ($(filter modules_install,$(MAKECMDGOALS)),)
>> + mixed-targets := 1
>> + endif
>> +endif
>>
>> ifeq ($(mixed-targets),1)
>> # ===========================================================================
>> --
>> 1.9.2
>>
>
> This patch seems to have gotten lost. Do you still like it? If so,
> can someone apply it, please?

Oops. I will rebase it and apply to the kbuild tree.

Michal