2018-01-22 21:47:16

by Jakub Kicinski

[permalink] [raw]
Subject: Mutli-directory module Makefiles

Hi!

in drivers/net/ethernet/netronome/nfp there is a module which is built
from C sources in 4 directories. What is the best way to handle that?

Currently we just add all the objects in one Makefile:

nfp-objs := \
nfpcore/nfp6000_pcie.o \
nfpcore/nfp_cppcore.o \
etc.

However, this makes it impossible to build a single object in subdirs:

$ make drivers/net/ethernet/netronome/nfp/nfpcore/nfp_cppcore.o
scripts/Makefile.build:45: drivers/net/ethernet/netronome/nfp/nfpcore/Makefile: No such file or directory
make[2]: *** No rule to make target 'drivers/net/ethernet/netronome/nfp/nfpcore/Makefile'. Stop.

Documentation/kbuild/modules.txt also contains an unclear remark that
this is not recommended (or is it about the use of $(src)?):

--- 4.3 Several Subdirectories
...
To build the module complex.ko, we then need the following
kbuild file:

--> filename: Kbuild
obj-m := complex.o
complex-y := src/complex_main.o
complex-y += src/hal/hardwareif.o

ccflags-y := -I$(src)/include
ccflags-y += -I$(src)/src/hal/include

As you can see, kbuild knows how to handle object files located
in other directories. The trick is to specify the directory
relative to the kbuild file's location. **That being said, this
is NOT recommended practice.**

Making the include optional would work:

diff --git a/scripts/Makefile.build b/scripts/Makefile.build
@@ -42,7 +42,7 @@ save-cflags := $(CFLAGS)
# The filename Kbuild has precedence over Makefile
kbuild-dir := $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
kbuild-file := $(if $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Kbuild,$(kbuild-dir)/Makefile)
-include $(kbuild-file)
+-include $(kbuild-file)

# If the save-* variables changed error out
ifeq ($(KBUILD_NOPEDANTIC),)

Or we could create empty Makefiles in subdirectories... Is there a
better way of handling this? Which is preferred?


2018-01-25 13:30:43

by Masahiro Yamada

[permalink] [raw]
Subject: Re: Mutli-directory module Makefiles

2018-01-23 6:46 GMT+09:00 Jakub Kicinski <[email protected]>:
> Hi!
>
> in drivers/net/ethernet/netronome/nfp there is a module which is built
> from C sources in 4 directories. What is the best way to handle that?
>
> Currently we just add all the objects in one Makefile:
>
> nfp-objs := \
> nfpcore/nfp6000_pcie.o \
> nfpcore/nfp_cppcore.o \
> etc.


I recommend you to refactor the Makefile,
adding Makefiles into nfpcore, nic, flower, bpf sub-directories.


Also, ifeq is ugly,
can you rewrite like follows?


nfp-$(CONFIG_NFP_APP_FLOWER) += flower/
nfp-$(CONFIG_BPF_SYSCALL) += bpf/
nfp-$(CONFIG_NFP_DEBUG) += nfp_net_debugfs.o





> However, this makes it impossible to build a single object in subdirs:
>
> $ make drivers/net/ethernet/netronome/nfp/nfpcore/nfp_cppcore.o
> scripts/Makefile.build:45: drivers/net/ethernet/netronome/nfp/nfpcore/Makefile: No such file or directory
> make[2]: *** No rule to make target 'drivers/net/ethernet/netronome/nfp/nfpcore/Makefile'. Stop.
>
> Documentation/kbuild/modules.txt also contains an unclear remark that
> this is not recommended (or is it about the use of $(src)?):
>
> --- 4.3 Several Subdirectories
> ...
> To build the module complex.ko, we then need the following
> kbuild file:
>
> --> filename: Kbuild
> obj-m := complex.o
> complex-y := src/complex_main.o
> complex-y += src/hal/hardwareif.o
>
> ccflags-y := -I$(src)/include
> ccflags-y += -I$(src)/src/hal/include
>
> As you can see, kbuild knows how to handle object files located
> in other directories. The trick is to specify the directory
> relative to the kbuild file's location. **That being said, this
> is NOT recommended practice.**
>
> Making the include optional would work:
>
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> @@ -42,7 +42,7 @@ save-cflags := $(CFLAGS)
> # The filename Kbuild has precedence over Makefile
> kbuild-dir := $(if $(filter /%,$(src)),$(src),$(srctree)/$(src))
> kbuild-file := $(if $(wildcard $(kbuild-dir)/Kbuild),$(kbuild-dir)/Kbuild,$(kbuild-dir)/Makefile)
> -include $(kbuild-file)
> +-include $(kbuild-file)
>
> # If the save-* variables changed error out
> ifeq ($(KBUILD_NOPEDANTIC),)
>
> Or we could create empty Makefiles in subdirectories... Is there a
> better way of handling this? Which is preferred?



--
Best Regards
Masahiro Yamada

2018-01-25 19:49:00

by Jakub Kicinski

[permalink] [raw]
Subject: Re: Mutli-directory module Makefiles

On Thu, 25 Jan 2018 22:08:34 +0900, Masahiro Yamada wrote:
> 2018-01-23 6:46 GMT+09:00 Jakub Kicinski <[email protected]>:
> > Hi!
> >
> > in drivers/net/ethernet/netronome/nfp there is a module which is built
> > from C sources in 4 directories. What is the best way to handle that?
> >
> > Currently we just add all the objects in one Makefile:
> >
> > nfp-objs := \
> > nfpcore/nfp6000_pcie.o \
> > nfpcore/nfp_cppcore.o \
> > etc.
>
>
> I recommend you to refactor the Makefile,
> adding Makefiles into nfpcore, nic, flower, bpf sub-directories.
>
>
> Also, ifeq is ugly,
> can you rewrite like follows?
>
>
> nfp-$(CONFIG_NFP_APP_FLOWER) += flower/
> nfp-$(CONFIG_BPF_SYSCALL) += bpf/
> nfp-$(CONFIG_NFP_DEBUG) += nfp_net_debugfs.o

I tried that, but what do I build inside the directories? I tried
adding the objects to nfp-objs, building a lib.a, or trying to build a
separate object, but it's either ignored or dependency is not obeyed
and nfp.o complains that the subdirectory object doesn't exist yet
when the build of subdirectory finishes later :(

Sorry for not spotting your response earlier! It went into Spam for
some reason :S


2018-01-25 20:45:25

by Sam Ravnborg

[permalink] [raw]
Subject: Re: Mutli-directory module Makefiles

Hi Jakub.

On Mon, Jan 22, 2018 at 01:46:07PM -0800, Jakub Kicinski wrote:
> Hi!
>
> in drivers/net/ethernet/netronome/nfp there is a module which is built
> from C sources in 4 directories. What is the best way to handle that?

Move all .c files to th same directory - like done for almost
all other drives.
In this case we have a directory: drivers/net/ethernet/netronome
which is just an indirection to nfp/
and nfp/ contains a structure with 25 .c files in subdirs.

With some careful renaming everything could reside in
one dir without loosing any structure.
And then the driver starts to look a bit
more like any other driver in the kernel.

And as an added benefit some of the other goodies like
building single files just works.

Sam

2018-01-27 22:30:06

by Masahiro Yamada

[permalink] [raw]
Subject: Re: Mutli-directory module Makefiles

2018-01-26 4:47 GMT+09:00 Jakub Kicinski <[email protected]>:
> On Thu, 25 Jan 2018 22:08:34 +0900, Masahiro Yamada wrote:
>> 2018-01-23 6:46 GMT+09:00 Jakub Kicinski <[email protected]>:
>> > Hi!
>> >
>> > in drivers/net/ethernet/netronome/nfp there is a module which is built
>> > from C sources in 4 directories. What is the best way to handle that?
>> >
>> > Currently we just add all the objects in one Makefile:
>> >
>> > nfp-objs := \
>> > nfpcore/nfp6000_pcie.o \
>> > nfpcore/nfp_cppcore.o \
>> > etc.
>>
>>
>> I recommend you to refactor the Makefile,
>> adding Makefiles into nfpcore, nic, flower, bpf sub-directories.
>>
>>
>> Also, ifeq is ugly,
>> can you rewrite like follows?
>>
>>
>> nfp-$(CONFIG_NFP_APP_FLOWER) += flower/
>> nfp-$(CONFIG_BPF_SYSCALL) += bpf/
>> nfp-$(CONFIG_NFP_DEBUG) += nfp_net_debugfs.o
>
> I tried that, but what do I build inside the directories? I tried
> adding the objects to nfp-objs, building a lib.a, or trying to build a
> separate object, but it's either ignored or dependency is not obeyed
> and nfp.o complains that the subdirectory object doesn't exist yet
> when the build of subdirectory finishes later :(
>
> Sorry for not spotting your response earlier! It went into Spam for
> some reason :S
>

Sorry, I was misunderstanding.

The notation I suggested
is not supported for composite objects (like nfp-objs, nfp-y).

The only solution is to move all files to the same directory,
as Sam pointed out.




--
Best Regards
Masahiro Yamada