2014-12-26 12:02:44

by Sedat Dilek

[permalink] [raw]
Subject: deb-pkg: Cleaning of debian/*tmp files when running 'make (dist)clean'

Hi,

I creating Debian/Ubuntu packages using builddeb script ('make deb-pkg').

While I was doing a 'make distclean' and grep-ed for some patterns I
saw that my debian/*tmp dirs were not deleted.

$ ls debian/*tmp
debian/fwtmp:
DEBIAN lib usr

debian/hdrtmp:
DEBIAN lib usr

debian/headertmp:
DEBIAN usr

debian/tmp:
boot DEBIAN etc lib usr

Any chance that this is also done on 'make (dist)clean' (not only
within the builddeb script)?

111:rm -rf "$tmpdir" "$fwdir" "$kernel_headers_dir"
"$libc_headers_dir" "$dbg_dir"

Any other make (PHONY) target I don't know?

Or introduce sth. like clean_tmpdirs() executed before 'exit 0' (or
simply invoke above rm-line).

BTW, can we have some more meaningful var-names for both headers-dirs,
something like...

[ scripts/package/builddeb ]

-kernel_headers_dir="$objtree/debian/hdrtmp"
-libc_headers_dir="$objtree/debian/headertmp"
+kernel_headers_dir="$objtree/debian/k_hdrtmp"
+libc_headers_dir="$objtree/debian/c_hdrtmp"

...it's a bit confusing when you look at filesystem-level (dirs, files).

Thanks.

Regards,
- Sedat -


2014-12-26 12:43:11

by Sedat Dilek

[permalink] [raw]
Subject: Re: deb-pkg: Cleaning of debian/*tmp files when running 'make (dist)clean'

On Fri, Dec 26, 2014 at 1:02 PM, Sedat Dilek <[email protected]> wrote:
[...]
>
> Any other make (PHONY) target I don't know?
>

I fell over "clean-dirs"...

scripts/package/Makefile:93:clean-dirs += $(objtree)/debian/

...but did not really understood how it works.
The main Makefile defines some clean-dirs PHONY#s.
I can only speculate, someone with more skillz in Makefile handling
should look at this.

Thanks.

- Sedat -

2014-12-27 18:52:16

by Ben Hutchings

[permalink] [raw]
Subject: Re: deb-pkg: Cleaning of debian/*tmp files when running 'make (dist)clean'

On Fri, 2014-12-26 at 13:43 +0100, Sedat Dilek wrote:
> On Fri, Dec 26, 2014 at 1:02 PM, Sedat Dilek <[email protected]> wrote:
> [...]
> >
> > Any other make (PHONY) target I don't know?
> >
>
> I fell over "clean-dirs"...
>
> scripts/package/Makefile:93:clean-dirs += $(objtree)/debian/
>
> ...but did not really understood how it works.
> The main Makefile defines some clean-dirs PHONY#s.
> I can only speculate, someone with more skillz in Makefile handling
> should look at this.

It looks like this has not worked for a long time. The scripts
directory is not included in the main recursive processes; it seems to
be special-cased for build and not for clean. This is partly deliberate
because clean is supposed to leave 'enough to build external modules'.
It's not obvious to include scripts/packages/ in clean.

Ben.

--
Ben Hutchings
Experience is directly proportional to the value of equipment destroyed.
- Carolyn Scheppner


Attachments:
signature.asc (811.00 B)
This is a digitally signed message part

2014-12-28 09:56:34

by Sedat Dilek

[permalink] [raw]
Subject: Re: deb-pkg: Cleaning of debian/*tmp files when running 'make (dist)clean'

On Fri, Dec 26, 2014 at 7:46 PM, Ben Hutchings <[email protected]> wrote:
> On Fri, 2014-12-26 at 13:43 +0100, Sedat Dilek wrote:
>> On Fri, Dec 26, 2014 at 1:02 PM, Sedat Dilek <[email protected]> wrote:
>> [...]
>> >
>> > Any other make (PHONY) target I don't know?
>> >
>>
>> I fell over "clean-dirs"...
>>
>> scripts/package/Makefile:93:clean-dirs += $(objtree)/debian/
>>
>> ...but did not really understood how it works.
>> The main Makefile defines some clean-dirs PHONY#s.
>> I can only speculate, someone with more skillz in Makefile handling
>> should look at this.
>
> It looks like this has not worked for a long time. The scripts
> directory is not included in the main recursive processes; it seems to
> be special-cased for build and not for clean. This is partly deliberate
> because clean is supposed to leave 'enough to build external modules'.
> It's not obvious to include scripts/packages/ in clean.
>

[ CCing some more people (submitters to scripts/Makefile.clean) ]

Thanks for your reply and confirmation.

So, how to fix that issue?

Cleanup debian/*tmp dirs within builddeb script for now?

Fix that clean-dirs mechanism (or remove it)?
These 4 Makefile#s use it...

Documentation/DocBook/Makefile
Makefile
scripts/Makefile.clean
scripts/package/Makefile

Any help appreciated.

- Sedat -

2014-12-31 13:56:11

by Michal Marek

[permalink] [raw]
Subject: Re: deb-pkg: Cleaning of debian/*tmp files when running 'make (dist)clean'

On 2014-12-26 19:46, Ben Hutchings wrote:
> On Fri, 2014-12-26 at 13:43 +0100, Sedat Dilek wrote:
>> On Fri, Dec 26, 2014 at 1:02 PM, Sedat Dilek <[email protected]> wrote:
>> [...]
>>>
>>> Any other make (PHONY) target I don't know?
>>>
>>
>> I fell over "clean-dirs"...
>>
>> scripts/package/Makefile:93:clean-dirs += $(objtree)/debian/
>>
>> ...but did not really understood how it works.
>> The main Makefile defines some clean-dirs PHONY#s.
>> I can only speculate, someone with more skillz in Makefile handling
>> should look at this.
>
> It looks like this has not worked for a long time. The scripts
> directory is not included in the main recursive processes; it seems to
> be special-cased for build and not for clean.

It is special-cased for mrproper and thus distclean:

mrproper-dirs := $(addprefix _mrproper_,Documentation/DocBook
scripts)

The problem is that

clean-dirs += $(objtree)/debian/

results in

clean-dirs += ./debian/

and scripts/Makefile.clean treats it as relative to scripts/package.

Michal

2014-12-31 14:22:18

by Sedat Dilek

[permalink] [raw]
Subject: Re: deb-pkg: Cleaning of debian/*tmp files when running 'make (dist)clean'

On Wed, Dec 31, 2014 at 2:56 PM, Michal Marek <[email protected]> wrote:
> On 2014-12-26 19:46, Ben Hutchings wrote:
>> On Fri, 2014-12-26 at 13:43 +0100, Sedat Dilek wrote:
>>> On Fri, Dec 26, 2014 at 1:02 PM, Sedat Dilek <[email protected]> wrote:
>>> [...]
>>>>
>>>> Any other make (PHONY) target I don't know?
>>>>
>>>
>>> I fell over "clean-dirs"...
>>>
>>> scripts/package/Makefile:93:clean-dirs += $(objtree)/debian/
>>>
>>> ...but did not really understood how it works.
>>> The main Makefile defines some clean-dirs PHONY#s.
>>> I can only speculate, someone with more skillz in Makefile handling
>>> should look at this.
>>
>> It looks like this has not worked for a long time. The scripts
>> directory is not included in the main recursive processes; it seems to
>> be special-cased for build and not for clean.
>
> It is special-cased for mrproper and thus distclean:
>
> mrproper-dirs := $(addprefix _mrproper_,Documentation/DocBook
> scripts)
>
> The problem is that
>
> clean-dirs += $(objtree)/debian/
>
> results in
>
> clean-dirs += ./debian/
>
> and scripts/Makefile.clean treats it as relative to scripts/package.
>

Thanks for your reply.

IIRC I looked through some commits touching this area.
>From your description, is the simplification ("kbuild: simplify build,
clean, modbuiltin shorthands") in [1] the cause for this?

[ I haven't tried to revert it, returned home today, still looking
through my inbox :-). ]

- Sedat -

[1] http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/scripts/Makefile.clean?id=5b2389b45d1a9c12b9f4b976e38c36264bfc42f5

2014-12-31 14:25:11

by Michal Marek

[permalink] [raw]
Subject: Re: deb-pkg: Cleaning of debian/*tmp files when running 'make (dist)clean'

On 2014-12-31 15:16, Sedat Dilek wrote:
> On Wed, Dec 31, 2014 at 2:56 PM, Michal Marek <[email protected]> wrote:
>> The problem is that
>>
>> clean-dirs += $(objtree)/debian/
>>
>> results in
>>
>> clean-dirs += ./debian/
>>
>> and scripts/Makefile.clean treats it as relative to scripts/package.
>>
>
> Thanks for your reply.
>
> IIRC I looked through some commits touching this area.
> From your description, is the simplification ("kbuild: simplify build,
> clean, modbuiltin shorthands") in [1] the cause for this?

No, it broke with 7e1c0477 (kbuild: Use relative path for $(objtree)).

I'll problably change the logic in Makefile.clean to check for paths
starting with './' AKA '$(objtree)/', unless I find a more elegant solution.

Michal

2014-12-31 14:26:22

by Sedat Dilek

[permalink] [raw]
Subject: Re: deb-pkg: Cleaning of debian/*tmp files when running 'make (dist)clean'

On Wed, Dec 31, 2014 at 3:25 PM, Michal Marek <[email protected]> wrote:
> On 2014-12-31 15:16, Sedat Dilek wrote:
>> On Wed, Dec 31, 2014 at 2:56 PM, Michal Marek <[email protected]> wrote:
>>> The problem is that
>>>
>>> clean-dirs += $(objtree)/debian/
>>>
>>> results in
>>>
>>> clean-dirs += ./debian/
>>>
>>> and scripts/Makefile.clean treats it as relative to scripts/package.
>>>
>>
>> Thanks for your reply.
>>
>> IIRC I looked through some commits touching this area.
>> From your description, is the simplification ("kbuild: simplify build,
>> clean, modbuiltin shorthands") in [1] the cause for this?
>
> No, it broke with 7e1c0477 (kbuild: Use relative path for $(objtree)).
>
> I'll problably change the logic in Makefile.clean to check for paths
> starting with './' AKA '$(objtree)/', unless I find a more elegant solution.
>

Thanks /o\.

- Sedat -

2014-12-31 15:35:31

by Michal Marek

[permalink] [raw]
Subject: [PATCH 1/2] kbuild: Fix removal of the debian/ directory

scripts/Makefile.clean treats absolute path specially, but
$(objtree)/debian is no longer an absolute path since 7e1c0477 (kbuild:
Use relative path for $(objtree). Work around this by checking if the
path starts with $(objtree)/.

Reported-by: Sedat Dilek <[email protected]>
Fixes: 7e1c0477 (kbuild: Use relative path for $(objtree)
Signed-off-by: Michal Marek <[email protected]>
---
scripts/Makefile.clean | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/scripts/Makefile.clean b/scripts/Makefile.clean
index 1bca180..627f8cb 100644
--- a/scripts/Makefile.clean
+++ b/scripts/Makefile.clean
@@ -42,19 +42,19 @@ __clean-files := $(extra-y) $(extra-m) $(extra-) \

__clean-files := $(filter-out $(no-clean-files), $(__clean-files))

-# as clean-files is given relative to the current directory, this adds
-# a $(obj) prefix, except for absolute paths
+# clean-files is given relative to the current directory, unless it
+# starts with $(objtree)/ (which means "./", so do not add "./" unless
+# you want to delete a file from the toplevel object directory).

__clean-files := $(wildcard \
- $(addprefix $(obj)/, $(filter-out /%, $(__clean-files))) \
- $(filter /%, $(__clean-files)))
+ $(addprefix $(obj)/, $(filter-out $(objtree)/%, $(__clean-files))) \
+ $(filter $(objtree)/%, $(__clean-files)))

-# as clean-dirs is given relative to the current directory, this adds
-# a $(obj) prefix, except for absolute paths
+# same as clean-files

__clean-dirs := $(wildcard \
- $(addprefix $(obj)/, $(filter-out /%, $(clean-dirs))) \
- $(filter /%, $(clean-dirs)))
+ $(addprefix $(obj)/, $(filter-out $(objtree)/%, $(clean-dirs))) \
+ $(filter $(objtree)/%, $(clean-dirs)))

# ==========================================================================

--
2.1.2

2014-12-31 15:35:37

by Michal Marek

[permalink] [raw]
Subject: [PATCH 2/2] kbuild: Drop support for clean-rule

clean-rule has not been used since 94869f86 (kbuild: Accept absolute
paths in clean-files and introduce clean-dirs) ten years ago.

Signed-off-by: Michal Marek <[email protected]>
---
scripts/Makefile.clean | 3 ---
1 file changed, 3 deletions(-)

diff --git a/scripts/Makefile.clean b/scripts/Makefile.clean
index 627f8cb..55c96cb 100644
--- a/scripts/Makefile.clean
+++ b/scripts/Makefile.clean
@@ -71,9 +71,6 @@ endif
ifneq ($(strip $(__clean-dirs)),)
+$(call cmd,cleandir)
endif
-ifneq ($(strip $(clean-rule)),)
- +$(clean-rule)
-endif
@:


--
2.1.2

2014-12-31 15:50:57

by Sedat Dilek

[permalink] [raw]
Subject: Re: [PATCH 1/2] kbuild: Fix removal of the debian/ directory

On Wed, Dec 31, 2014 at 4:35 PM, Michal Marek <[email protected]> wrote:
> scripts/Makefile.clean treats absolute path specially, but
> $(objtree)/debian is no longer an absolute path since 7e1c0477 (kbuild:
> Use relative path for $(objtree). Work around this by checking if the
> path starts with $(objtree)/.
>
> Reported-by: Sedat Dilek <[email protected]>
> Fixes: 7e1c0477 (kbuild: Use relative path for $(objtree)
> Signed-off-by: Michal Marek <[email protected]>

After applying your patchset, thing work as expected.

Feel free to add my Tested-by.

Hope this will get into v3.19-rc3?

- Sedat -

[1] https://patchwork.kernel.org/patch/5555911/
[2] https://patchwork.kernel.org/patch/5555931/

P.S.: Testing new patchset (especially 1/2).

$ make distclean
CLEAN .
CLEAN arch/x86/kernel
CLEAN arch/x86/realmode/rm
CLEAN arch/x86/vdso
CLEAN arch/x86/lib
CLEAN crypto/asymmetric_keys
CLEAN drivers/tty/vt
CLEAN kernel/debug/kdb
CLEAN kernel/time
CLEAN kernel
CLEAN lib/raid6
CLEAN lib
CLEAN security/apparmor
CLEAN security/selinux
CLEAN usr
CLEAN arch/x86/boot/compressed
CLEAN arch/x86/boot
CLEAN arch/x86/tools
CLEAN .tmp_versions
CLEAN scripts/basic
CLEAN scripts/genksyms
CLEAN scripts/kconfig
CLEAN scripts/mod
CLEAN scripts/selinux/genheaders
CLEAN scripts/selinux/mdp
CLEAN scripts
CLEAN include/config usr/include include/generated
arch/x86/include/generated
CLEAN .config .config.old .version
include/generated/uapi/linux/version.h Module.symvers signing_key.priv
signing_key.x509 x509.genkey

$ git am ../patches/kbuild-fixes-3.19-rc2/1*
Wende an: kbuild: Fix removal of the debian/ directory

$ git am ../patches/kbuild-fixes-3.19-rc2/2*
Wende an: kbuild: Drop support for clean-rule

$ ls debian/*tmp
debian/fwtmp:
DEBIAN lib usr

debian/hdrtmp:
DEBIAN lib usr

debian/headertmp:
DEBIAN usr

debian/tmp:
boot DEBIAN etc lib usr

$ make distclean
CLEAN ./debian/

$ LC_ALL=C ls debian/*tmp
ls: cannot access debian/*tmp: No such file or directory
- EOT .


> ---
> scripts/Makefile.clean | 16 ++++++++--------
> 1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/scripts/Makefile.clean b/scripts/Makefile.clean
> index 1bca180..627f8cb 100644
> --- a/scripts/Makefile.clean
> +++ b/scripts/Makefile.clean
> @@ -42,19 +42,19 @@ __clean-files := $(extra-y) $(extra-m) $(extra-) \
>
> __clean-files := $(filter-out $(no-clean-files), $(__clean-files))
>
> -# as clean-files is given relative to the current directory, this adds
> -# a $(obj) prefix, except for absolute paths
> +# clean-files is given relative to the current directory, unless it
> +# starts with $(objtree)/ (which means "./", so do not add "./" unless
> +# you want to delete a file from the toplevel object directory).
>
> __clean-files := $(wildcard \
> - $(addprefix $(obj)/, $(filter-out /%, $(__clean-files))) \
> - $(filter /%, $(__clean-files)))
> + $(addprefix $(obj)/, $(filter-out $(objtree)/%, $(__clean-files))) \
> + $(filter $(objtree)/%, $(__clean-files)))
>
> -# as clean-dirs is given relative to the current directory, this adds
> -# a $(obj) prefix, except for absolute paths
> +# same as clean-files
>
> __clean-dirs := $(wildcard \
> - $(addprefix $(obj)/, $(filter-out /%, $(clean-dirs))) \
> - $(filter /%, $(clean-dirs)))
> + $(addprefix $(obj)/, $(filter-out $(objtree)/%, $(clean-dirs))) \
> + $(filter $(objtree)/%, $(clean-dirs)))
>
> # ==========================================================================
>
> --
> 2.1.2
>

2014-12-31 16:05:39

by Sedat Dilek

[permalink] [raw]
Subject: Re: [PATCH 2/2] kbuild: Drop support for clean-rule

On Wed, Dec 31, 2014 at 4:35 PM, Michal Marek <[email protected]> wrote:
> clean-rule has not been used since 94869f86 (kbuild: Accept absolute
> paths in clean-files and introduce clean-dirs) ten years ago.
>
> Signed-off-by: Michal Marek <[email protected]>

Tested-by: Sedat Dilek <[email protected]>

- Sedat -

> ---
> scripts/Makefile.clean | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/scripts/Makefile.clean b/scripts/Makefile.clean
> index 627f8cb..55c96cb 100644
> --- a/scripts/Makefile.clean
> +++ b/scripts/Makefile.clean
> @@ -71,9 +71,6 @@ endif
> ifneq ($(strip $(__clean-dirs)),)
> +$(call cmd,cleandir)
> endif
> -ifneq ($(strip $(clean-rule)),)
> - +$(clean-rule)
> -endif
> @:
>
>
> --
> 2.1.2
>