Subject: [PATCH 1/4] Makefile: rules for printing kernel architecture and localversion

trivial rule to print out the kernel arch and localversion, so
external tools, like distro packagers, can easily get it.

Signed-off-by: Enrico Weigelt, metux IT consult <[email protected]>
---
Makefile | 6 ++++++
1 file changed, 6 insertions(+)

diff --git a/Makefile b/Makefile
index 3e4868a..5afc3de 100644
--- a/Makefile
+++ b/Makefile
@@ -1706,6 +1706,12 @@ kernelrelease:
kernelversion:
@echo $(KERNELVERSION)

+kernellocalversion:
+ @$(CONFIG_SHELL) $(srctree)/scripts/setlocalversion $(srctree) | sed -e 's~^\-~~'
+
+kernelarch:
+ @echo $(ARCH)
+
image_name:
@echo $(KBUILD_IMAGE)

--
1.9.1


Subject: [PATCH 4/4] debian: add generic rule file

Adding a generic debian rule file, so we can build the directly
via usual Debian package build tools (eg. git-buildpackage,
dck-buildpackage, etc). It expects the .config file already
placed in the source tree.

The rule file contains a rule for creating debian/control and
other metadata - this is done similar to the 'deb-pkg' make rule,
scripts/packaging/mkdebian.

Signed-off-by: Enrico Weigelt, metux IT consult <[email protected]>
---
.gitignore | 1 +
MAINTAINERS | 6 ++++++
debian/rules | 27 +++++++++++++++++++++++++++
3 files changed, 34 insertions(+)
create mode 100755 debian/rules

diff --git a/.gitignore b/.gitignore
index 7587ef56..01d742c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -69,6 +69,7 @@ modules.builtin
# Debian directory (make deb-pkg)
#
/debian/
+!/debian/rules

#
# Snap directory (make snap-pkg)
diff --git a/MAINTAINERS b/MAINTAINERS
index 558acf2..56e034c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4516,6 +4516,12 @@ F: include/uapi/linux/dccp.h
F: include/linux/tfrc.h
F: net/dccp/

+DEBIAN PACKAGING FILES
+M: Enrico Weigelt <[email protected]>
+L: [email protected]
+S: Maintained
+F: debian/
+
DECnet NETWORK LAYER
W: http://linux-decnet.sourceforge.net
L: [email protected]
diff --git a/debian/rules b/debian/rules
new file mode 100755
index 0000000..c2f0319
--- /dev/null
+++ b/debian/rules
@@ -0,0 +1,27 @@
+#!/usr/bin/make -f
+# SPDX-License-Identifier: GPL-2.0
+
+export MAKE
+export KERNELARCH = $(shell $(MAKE) kernelarch)
+export KERNELRELEASE = $(shell $(MAKE) kernelrelease)
+export KBUILD_DEBARCH = $(shell dpkg-architecture -qDEB_HOST_ARCH)
+export KBUILD_BUILD_VERSION = $(shell $(MAKE) kernellocalversion)
+export KDEB_RULES = debian/rules.auto
+export KDEB_SOURCENAME = linux-source
+export ARCH = $(KERNELARCH)
+
+debian/control debian/changelong debian/arch debian/copyright:
+debian/control:
+ ./scripts/package/mkdebian
+
+build: debian/control
+ $(MAKE) KERNELRELEASE=$(KERNELRELEASE) ARCH=$(KERNELARCH) KBUILD_BUILD_VERSION=$(KBUILD_BUILD_VERSION) KBUILD_SRC=
+
+binary-arch: debian/control
+ $(MAKE) KERNELRELEASE=$(KERNELRELEASE) ARCH=$(KERNELARCH) KBUILD_BUILD_VERSION=$(KBUILD_BUILD_VERSION) KBUILD_SRC= intdeb-pkg
+
+clean:
+ rm -rf debian/*tmp debian/files debian/changelog debian/control debian/copyright debian/rules.auto debian/arch
+ $(MAKE) clean
+
+binary: binary-arch
--
1.9.1

Subject: [PATCH 2/4] scripts: mkdebian: allow renaming generated debian/rules via env

Add new environment variable KDEB_RULES for controlling where the
generated debian rules are written to. By defaults, it's debian/rules,
but packagers might override it for providing their own rules file.

Signed-off-by: Enrico Weigelt, metux IT consult <[email protected]>
---
scripts/package/mkdebian | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
index 8351584..7a9ca4e 100755
--- a/scripts/package/mkdebian
+++ b/scripts/package/mkdebian
@@ -99,6 +99,9 @@ kernel_headers_packagename=linux-headers-$version
dbg_packagename=$packagename-dbg
debarch=
set_debarch
+if [ -z "$KDEB_RULES" ]; then
+ KDEB_RULES=debian/rules
+fi

if [ "$ARCH" = "um" ] ; then
packagename=user-mode-linux-$version
@@ -206,7 +209,7 @@ Description: Linux kernel debugging symbols for $version
all the necessary debug symbols for the kernel and its modules.
EOF

-cat <<EOF > debian/rules
+cat <<EOF > $KDEB_RULES
#!$(command -v $MAKE) -f

srctree ?= .
--
1.9.1

2019-07-09 14:56:40

by Randy Dunlap

[permalink] [raw]
Subject: Re: [PATCH 1/4] Makefile: rules for printing kernel architecture and localversion

On 7/9/19 2:32 AM, Enrico Weigelt, metux IT consult wrote:
> trivial rule to print out the kernel arch and localversion, so
> external tools, like distro packagers, can easily get it.
>
> Signed-off-by: Enrico Weigelt, metux IT consult <[email protected]>

Hi,
If accepted, these targets should be added to the top level Makefile's
"make help" output also.

Thanks.

> ---
> Makefile | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index 3e4868a..5afc3de 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1706,6 +1706,12 @@ kernelrelease:
> kernelversion:
> @echo $(KERNELVERSION)
>
> +kernellocalversion:
> + @$(CONFIG_SHELL) $(srctree)/scripts/setlocalversion $(srctree) | sed -e 's~^\-~~'
> +
> +kernelarch:
> + @echo $(ARCH)
> +
> image_name:
> @echo $(KBUILD_IMAGE)
>
>


--
~Randy

2019-07-15 12:31:26

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH 4/4] debian: add generic rule file

On Tue, Jul 9, 2019 at 6:33 PM Enrico Weigelt, metux IT consult
<[email protected]> wrote:
>
> Adding a generic debian rule file, so we can build the directly
> via usual Debian package build tools (eg. git-buildpackage,
> dck-buildpackage, etc). It expects the .config file already
> placed in the source tree.
>
> The rule file contains a rule for creating debian/control and
> other metadata - this is done similar to the 'deb-pkg' make rule,
> scripts/packaging/mkdebian.

I saw a similar patch submission before, and negative feedback about it.

Debian maintains its own debian/rules, and it is fine.
I do not like to check-in the one in the kernel tree.



> Signed-off-by: Enrico Weigelt, metux IT consult <[email protected]>
> ---
> .gitignore | 1 +
> MAINTAINERS | 6 ++++++
> debian/rules | 27 +++++++++++++++++++++++++++
> 3 files changed, 34 insertions(+)
> create mode 100755 debian/rules
>
> diff --git a/.gitignore b/.gitignore
> index 7587ef56..01d742c 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -69,6 +69,7 @@ modules.builtin
> # Debian directory (make deb-pkg)
> #
> /debian/
> +!/debian/rules
>
> #
> # Snap directory (make snap-pkg)
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 558acf2..56e034c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -4516,6 +4516,12 @@ F: include/uapi/linux/dccp.h
> F: include/linux/tfrc.h
> F: net/dccp/
>
> +DEBIAN PACKAGING FILES
> +M: Enrico Weigelt <[email protected]>
> +L: [email protected]
> +S: Maintained
> +F: debian/
> +
> DECnet NETWORK LAYER
> W: http://linux-decnet.sourceforge.net
> L: [email protected]
> diff --git a/debian/rules b/debian/rules
> new file mode 100755
> index 0000000..c2f0319
> --- /dev/null
> +++ b/debian/rules
> @@ -0,0 +1,27 @@
> +#!/usr/bin/make -f
> +# SPDX-License-Identifier: GPL-2.0
> +
> +export MAKE
> +export KERNELARCH = $(shell $(MAKE) kernelarch)
> +export KERNELRELEASE = $(shell $(MAKE) kernelrelease)
> +export KBUILD_DEBARCH = $(shell dpkg-architecture -qDEB_HOST_ARCH)
> +export KBUILD_BUILD_VERSION = $(shell $(MAKE) kernellocalversion)
> +export KDEB_RULES = debian/rules.auto
> +export KDEB_SOURCENAME = linux-source
> +export ARCH = $(KERNELARCH)
> +
> +debian/control debian/changelong debian/arch debian/copyright:
> +debian/control:
> + ./scripts/package/mkdebian
> +
> +build: debian/control
> + $(MAKE) KERNELRELEASE=$(KERNELRELEASE) ARCH=$(KERNELARCH) KBUILD_BUILD_VERSION=$(KBUILD_BUILD_VERSION) KBUILD_SRC=
> +
> +binary-arch: debian/control
> + $(MAKE) KERNELRELEASE=$(KERNELRELEASE) ARCH=$(KERNELARCH) KBUILD_BUILD_VERSION=$(KBUILD_BUILD_VERSION) KBUILD_SRC= intdeb-pkg
> +
> +clean:
> + rm -rf debian/*tmp debian/files debian/changelog debian/control debian/copyright debian/rules.auto debian/arch
> + $(MAKE) clean
> +
> +binary: binary-arch
> --
> 1.9.1
>


--
Best Regards
Masahiro Yamada

Subject: Re: [PATCH 4/4] debian: add generic rule file

On 15.07.19 14:28, Masahiro Yamada wrote:

>> The rule file contains a rule for creating debian/control and
>> other metadata - this is done similar to the 'deb-pkg' make rule,
>> scripts/packaging/mkdebian.
>
> I saw a similar patch submission before, and negative feedback about it.

Do you recall what negative feedback exactly ?

> Debian maintains its own debian/rules, and it is fine.

Not for me, I don't use it - given up trying to make anything useful
out of it. It's extremly complex, practically undebuggable and doesn't
even work w/o lots of external preparations.


--mtx

--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[email protected] -- +49-151-27565287

2019-07-15 19:13:56

by Theodore Ts'o

[permalink] [raw]
Subject: Re: [PATCH 4/4] debian: add generic rule file

On Mon, Jul 15, 2019 at 08:56:25PM +0200, Enrico Weigelt, metux IT consult wrote:
> On 15.07.19 14:28, Masahiro Yamada wrote:
>
> >> The rule file contains a rule for creating debian/control and
> >> other metadata - this is done similar to the 'deb-pkg' make rule,
> >> scripts/packaging/mkdebian.
> >
> > I saw a similar patch submission before, and negative feedback about it.
>
> Do you recall what negative feedback exactly ?

It's possible I'm not remembering some of the feedback, but the only
thing I recall was the comment I made that I'd really like this use
case:

make O=/build/linux-build bindeb-pkg

to not break. And as far as I can tell from the proposed patch series
(I haven't had a chance to experimentally verify it yet), I don't
think it should break anything --- I'm assuming that we will still
have a way of creating the debian/rules file in
/build/linux-build/debian/rules when doing a O= build, and that the
intdeb-pkg rule remains the same. At least, it appears to be the case
from my doing a quick look at the patches.

> > Debian maintains its own debian/rules, and it is fine.
>
> Not for me, I don't use it - given up trying to make anything useful
> out of it. It's extremly complex, practically undebuggable and doesn't
> even work w/o lots of external preparations.

Yeah, the official Debian debian/rules is optimized for doing a
distribution release, and in addition to the issues Enrico has raised,
last time I tried it, it was S-L-O-W since it was building a fully
generic kernel. It's not at all useable for general developer use.

It sounds like what Enrico is trying to do is to enable running
"dpkg-buildpackage -us -uc -b" from the the top-level kernel package
as being easier than running "make bindeb-pkg". I suspect this might
be because his goal is to integrate individual kernel builds from
using Debian's hermetic build / chroot systems (e.g., sbuild, pbuilder)?

- Ted

2019-07-16 09:00:23

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH 4/4] debian: add generic rule file

On Tue, Jul 16, 2019 at 4:13 AM Theodore Y. Ts'o <[email protected]> wrote:
>
> On Mon, Jul 15, 2019 at 08:56:25PM +0200, Enrico Weigelt, metux IT consult wrote:
> > On 15.07.19 14:28, Masahiro Yamada wrote:
> >
> > >> The rule file contains a rule for creating debian/control and
> > >> other metadata - this is done similar to the 'deb-pkg' make rule,
> > >> scripts/packaging/mkdebian.
> > >
> > > I saw a similar patch submission before, and negative feedback about it.
> >
> > Do you recall what negative feedback exactly ?

Sorry, my memory was broken.

I did not like this patch set from the beginning,
but missed to express my opinion strongly.

I want debian/ to be kept as a drop-in directory
for packagers, without replacing the upstream debian/rules.

If a check-in source file is modified in anyway,
scripts/setlocalversion would set -dirty flag,
which I want to avoid.



> It's possible I'm not remembering some of the feedback, but the only
> thing I recall was the comment I made that I'd really like this use
> case:
>
> make O=/build/linux-build bindeb-pkg
>
> to not break. And as far as I can tell from the proposed patch series
> (I haven't had a chance to experimentally verify it yet), I don't
> think it should break anything --- I'm assuming that we will still
> have a way of creating the debian/rules file in
> /build/linux-build/debian/rules when doing a O= build, and that the
> intdeb-pkg rule remains the same. At least, it appears to be the case
> from my doing a quick look at the patches.
>
> > > Debian maintains its own debian/rules, and it is fine.
> >
> > Not for me, I don't use it - given up trying to make anything useful
> > out of it. It's extremly complex, practically undebuggable and doesn't
> > even work w/o lots of external preparations.
>
> Yeah, the official Debian debian/rules is optimized for doing a
> distribution release, and in addition to the issues Enrico has raised,
> last time I tried it, it was S-L-O-W since it was building a fully
> generic kernel. It's not at all useable for general developer use.

It is OK if the package is targeting normal users instead of
kernel developers.


> It sounds like what Enrico is trying to do is to enable running
> "dpkg-buildpackage -us -uc -b" from the the top-level kernel package
> as being easier than running "make bindeb-pkg". I suspect this might
> be because his goal is to integrate individual kernel builds from
> using Debian's hermetic build / chroot systems (e.g., sbuild, pbuilder)?

I am OK with generating debian/rules with 'make bindeb-pkg', a shell scripts
or whatever, but I dislike to commit it in upstream git tree.

debian/rules is a hook for packagers to do their jobs in downstream.
"We kindly committed a generic one for you" sounds weird to me.

--
Best Regards
Masahiro Yamada

2019-07-16 12:36:02

by Theodore Ts'o

[permalink] [raw]
Subject: Re: [PATCH 4/4] debian: add generic rule file

On Tue, Jul 16, 2019 at 05:58:49PM +0900, Masahiro Yamada wrote:
> I want debian/ to be kept as a drop-in directory
> for packagers, without replacing the upstream debian/rules.
>
> If a check-in source file is modified in anyway,
> scripts/setlocalversion would set -dirty flag,
> which I want to avoid.

In practice, that's not going to be a problem for most distributions.
The traditional way Debian-derived systems have done builds is
completely outside of git. So there will be a linux_5.2.orig.tar.gz
and a linux_5.2-1.debian.tar.xz. dpkg_source -x will first unpackage
the orig.tar.gz, and then the debian.tar.xz, and if the second
overwrites the first, it's no big deal.

More modern Debian package maintainer workflows may be using git, but
in that case, all of the "Debianizations" are reflected in a separate
branch. So it's not going to set the -dirty flag.

There will be potential merge conflicts between Enrico's proposed
"upstream default debian/rules" file and the Debian/Ubuntu
debian/rules file on their distro branch. However, I don't think
that's a big issue, for two reasons.

First, once it's checked in, I expect changes to the default
debian/rules file will be relatively rare. Secondly, it's easy enough
to use gitattributes and defining a custom merge driver so that a
distribution can configure things so that they always use the version
of debian/rules from their branch, so the merge conflict resolution
can be set up to always do the right thing.

There are certainly other upstreams which ship their own debian/
directories. E2fsprogs is one such example, but in that case I'm
cheating because I'm both the Debian package maintainer as well as the
upstream maintainer. :-) However, it's never been an issue for Ubuntu
when they choose to ship their own customized debian/rules file.

> debian/rules is a hook for packagers to do their jobs in downstream.
> "We kindly committed a generic one for you" sounds weird to me.

It is weird, and it's not common for upstream packages (which are not
native Debian packages) to ship their own debian directory. But it
certainly does happen, and it won't cause any problems in actual
practice.

Regards,

- Ted

Subject: Re: [PATCH 4/4] debian: add generic rule file

On 15.07.19 21:12, Theodore Y. Ts'o wrote:

Hi,

> It's possible I'm not remembering some of the feedback, but the only> thing I recall was the comment I made that I'd really like this use>
case:> > make O=/build/linux-build bindeb-pkg
ah, I yet wanted to test that - thx for reminding me. > to not break.
And as far as I can tell from the proposed patch series> (I haven't had
a chance to experimentally verify it yet), I don't> think it should
break anything --- I'm assuming that we will still> have a way of
creating the debian/rules file in> /build/linux-build/debian/rules when
doing a O= build, and that the> intdeb-pkg rule remains the same. At
least, it appears to be the case> from my doing a quick look at the patches.
Yes (unless i've missed something), everything should remain as it is.
One thing that could happen (not checked yet) is that running good old
'make bindeb-pkg' without O=... could overwrite the now already existing
debian/rules file.

If that's really a problem, we could tweak the machinery to use a
different name for the rule file (for now, one the preceeding patch
just allows giving a different name for just *generating* the rules
file). Another idea could be rewriting the whole process so that no
rules file needs to be generated at all.

> Yeah, the official Debian debian/rules is optimized for doing a
> distribution release, and in addition to the issues Enrico has raised,
> last time I tried it, it was S-L-O-W since it was building a fully
> generic kernel. It's not at all useable for general developer use.

I'm a bit reluctant calling this 'optimized' :p

The strangest aspect (IMHO) is they're building several different trees
(w/ different huge patch queues) from only one source package. Instead
I'd rather:
* try to get as much as possible in one tree
* have separate source packages if there really need to be separate
patche queues (IMHO, these things, like RT stuff, just need proper
Kconfig's)
* do all the patching in git and skip the text-based patches entirely

Haven't found out, why they're actually doing it that complicated way
(didn't get any useful answers from debian kernel folks)

> It sounds like what Enrico is trying to do is to enable running
> "dpkg-buildpackage -us -uc -b" from the the top-level kernel package
> as being easier than running "make bindeb-pkg". I suspect this might
> be because his goal is to integrate individual kernel builds from
> using Debian's hermetic build / chroot systems (e.g., sbuild, pbuilder)?

Yes, I'm building all deb's by the same process / infrastructure.
In my case it's dck-buildpackage (*1) which runs the build in a docker
container (kinda pbuilder w/ docker). It always starts with a fresh
(minimal) base image, calls debian/rules to create debian/control
(if necessary) deploys the dependencies found in the control file
and finally fire's up dpkg-buildpackage - the output is collected
in an ready-to-use apt repo.

The goal of this is having a canonical build process for all deb
packages, not having to care of any special cases anymore. I also
have another tool ontop of that, which runs the whole show for dozens
of packages and targets (*2).

My first approach was trying to use Debian source packages with new
kernel trees, but had to give up after a few days. Then I've found out
that the kernel already has *almost* what I needed. The difference
between almost and fine is this patch queue (minus local .config files)


--mtx

--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[email protected] -- +49-151-27565287

Subject: Re: [PATCH 4/4] debian: add generic rule file

On 16.07.19 14:34, Theodore Y. Ts'o wrote:

Hi,

> In practice, that's not going to be a problem for most distributions.
> The traditional way Debian-derived systems have done builds is
> completely outside of git. So there will be a linux_5.2.orig.tar.gz
> and a linux_5.2-1.debian.tar.xz. dpkg_source -x will first unpackage
> the orig.tar.gz, and then the debian.tar.xz, and if the second
> overwrites the first, it's no big deal.

ACK. IIRC they already filter out debian/ directories when generating
upstream tarballs - other upstreams already provide their debian/
stuff, too.

> First, once it's checked in, I expect changes to the default
> debian/rules file will be relatively rare.

ACK. I currently don't see much reasons for future changes. If anybody
sees something missing, just let me know, and I'll take it up for the
next review round.

--mtx

--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[email protected] -- +49-151-27565287

2019-07-17 15:24:39

by Theodore Ts'o

[permalink] [raw]
Subject: Re: [PATCH 4/4] debian: add generic rule file

On Wed, Jul 17, 2019 at 04:16:39PM +0200, Enrico Weigelt, metux IT consult wrote:
>
> > In practice, that's not going to be a problem for most distributions.
> > The traditional way Debian-derived systems have done builds is
> > completely outside of git. So there will be a linux_5.2.orig.tar.gz
> > and a linux_5.2-1.debian.tar.xz. dpkg_source -x will first unpackage
> > the orig.tar.gz, and then the debian.tar.xz, and if the second
> > overwrites the first, it's no big deal.
>
> ACK. IIRC they already filter out debian/ directories when generating
> upstream tarballs - other upstreams already provide their debian/
> stuff, too.

Well, no, actually they don't. That's because as much as possible
they want the upstream tarball to be bit-for-bit identical to the one
published on the official upstream distribution site. That allows
them to include the detached PGP signature from the upstream
maintainer, if one is provided.

If there are files in the upstream debian/ directory that they don't
need, they can delete in the distro's debian/rules file. Ideally, so
we shouldn't include files in the Linux kernel's debian/ directory
willy-nilly. But the debian/rules file will *always* be present, and
so it will be overwritten by the <package>_<ver>.debian.tar.xz file,
and so it's no big deal.

- Ted