2017-12-01 16:52:36

by Ben Hutchings

[permalink] [raw]
Subject: Re: [PATCH] builddeb: introduce variables for control-file customization

On Fri, 2017-12-01 at 15:56 +0000, Henning Schild wrote:
> The debian packages coming out of "make *deb-pkg" lack some critical
> information in the control-files e.g. the "Depends:" field. If one
> tries to install a fresh system with such a "linux-image" debootstrap or
> multistrap might try to install the kernel before its deps and the
> package hooks will fail.

I assume you're talking about those hook scripts being run while the
packages they belong to are only unpacked? I hadn't thought about this
issue, but it seems to me that those hook scripts generally ought to be
fixed to handle this case properly. Most of the packages installing
hook scripts for kernel packages are not going to be dependencies of
linux-image packages, so it will never be safe for them to assume their
package has been fully installed.

> Different debian-based distros use different values for the missing
> fields. And the values differ between distro versions as well. So
> hardcoding of e.g. "Depends" is not possible.

The dependencies also depend on the kernel configuration. (And a
custom kernel built with 'make deb-pkg' often won't have any
dependencies outside of essential packages.)

> This patch introduces an option variable for every debian package built
> by builddeb. That allows advanced users to pass additional arguments to
> "dpkg-gencontrol" e.g. to set "Depends". All the new variables are
> optional.

This customisation mechanism seems too powerful to be maintainable.
There is a high risk that it would conflict with later improvements to
builddeb, either resulting in regressions or blocking those
improvements from being made.

> for example:
> make \
> KDEB_OPTS_IMAGE=\
> "-DDepends='initramfs-tools | linux-initramfs-tool, kmod, linux-base'" \
[...]

The maintainer scripts generated by builddeb currently don't run depmod
or any of the script in linux-base. So this seems like a bad example.
However, the dependency on initramfs-tools is an important one that
can't simply be inferred from the kernel configuration.

So I would support adding a means to append to the Depends field
specifically. Appending to the Breaks field may also be useful, as new
kernel versions may break specific utilities or user-space drivers.

Ben.

--
Ben Hutchings
When in doubt, use brute force. - Ken Thompson


Attachments:
signature.asc (833.00 B)
This is a digitally signed message part

2017-12-01 18:34:48

by Henning Schild

[permalink] [raw]
Subject: Re: [PATCH] builddeb: introduce variables for control-file customization

Am Fri, 1 Dec 2017 16:51:12 +0000
schrieb Ben Hutchings <[email protected]>:

> On Fri, 2017-12-01 at 15:56 +0000, Henning Schild wrote:
> > The debian packages coming out of "make *deb-pkg" lack some critical
> > information in the control-files e.g. the "Depends:" field. If one
> > tries to install a fresh system with such a "linux-image"
> > debootstrap or multistrap might try to install the kernel before
> > its deps and the package hooks will fail.
>
> I assume you're talking about those hook scripts being run while the
> packages they belong to are only unpacked? I hadn't thought about
> this issue, but it seems to me that those hook scripts generally
> ought to be fixed to handle this case properly. Most of the packages
> installing hook scripts for kernel packages are not going to be
> dependencies of linux-image packages, so it will never be safe for
> them to assume their package has been fully installed.

Yes these hook scripts fail when installing the kernel on another
system. Indeed we seem to have a case where packages installed on the
build-machine cause install-time deps for the package.
In my case the build-machine is pretty minimal but i still want some of
that i.e. initramfs.

> > Different debian-based distros use different values for the missing
> > fields. And the values differ between distro versions as well. So
> > hardcoding of e.g. "Depends" is not possible.
>
> The dependencies also depend on the kernel configuration. (And a
> custom kernel built with 'make deb-pkg' often won't have any
> dependencies outside of essential packages.)

In fact it does not have any at the moment, there is no essential. Or
maybe that is hidden in debian-magic.

> > This patch introduces an option variable for every debian package
> > built by builddeb. That allows advanced users to pass additional
> > arguments to "dpkg-gencontrol" e.g. to set "Depends". All the new
> > variables are optional.
>
> This customisation mechanism seems too powerful to be maintainable.
> There is a high risk that it would conflict with later improvements to
> builddeb, either resulting in regressions or blocking those
> improvements from being made.

Fair enough. But there needs to be a way to specifiy at least some
deps, inheriting them from the build-host would be wrong. I really just
care about deps but thought this powerful tool would be a good idea in
case anyone cares about suggest and recommend etc..

> > for example:
> > make \
> > KDEB_OPTS_IMAGE=\
> > "-DDepends='initramfs-tools | linux-initramfs-tool, kmod,
> > linux-base'" \
> [...]
>
> The maintainer scripts generated by builddeb currently don't run
> depmod or any of the script in linux-base. So this seems like a bad
> example. However, the dependency on initramfs-tools is an important
> one that can't simply be inferred from the kernel configuration.

Adding a depmod hook would probably be another patch. Let us keep that
in mind.

> So I would support adding a means to append to the Depends field
> specifically. Appending to the Breaks field may also be useful, as
> new kernel versions may break specific utilities or user-space
> drivers.

Ok, in that case i will come up with another patch introducing
KDEB_IMAGE_DEPENDS KDEB_HEADERS_DEPENDS etc. and maybe _BREAKS as well.

Those would be appended when the control-files are generated. i.e.

cat EOF...
Depends: foo bar $KDEB_IMAGE_DEPENDS
EOF

Henning

> Ben.
>

2017-12-01 18:49:00

by Ben Hutchings

[permalink] [raw]
Subject: Re: [PATCH] builddeb: introduce variables for control-file customization

On Fri, 2017-12-01 at 19:34 +0100, Henning Schild wrote:
> Am Fri, 1 Dec 2017 16:51:12 +0000
> schrieb Ben Hutchings <[email protected]>:
>
> > On Fri, 2017-12-01 at 15:56 +0000, Henning Schild wrote:
> > > The debian packages coming out of "make *deb-pkg" lack some critical
> > > information in the control-files e.g. the "Depends:" field. If one
> > > tries to install a fresh system with such a "linux-image"
> > > debootstrap or multistrap might try to install the kernel before
> > > its deps and the package hooks will fail.
> >
> > I assume you're talking about those hook scripts being run while the
> > packages they belong to are only unpacked? I hadn't thought about
> > this issue, but it seems to me that those hook scripts generally
> > ought to be fixed to handle this case properly. Most of the packages
> > installing hook scripts for kernel packages are not going to be
> > dependencies of linux-image packages, so it will never be safe for
> > them to assume their package has been fully installed.
>
> Yes these hook scripts fail when installing the kernel on another
> system. Indeed we seem to have a case where packages installed on the
> build-machine cause install-time deps for the package.

Can you give an example? I don't see how that would happen.

> In my case the build-machine is pretty minimal but i still want some of
> that i.e. initramfs.
>
> > > Different debian-based distros use different values for the missing
> > > fields. And the values differ between distro versions as well. So
> > > hardcoding of e.g. "Depends" is not possible.
> >
> > The dependencies also depend on the kernel configuration. (And a
> > custom kernel built with 'make deb-pkg' often won't have any
> > dependencies outside of essential packages.)
>
> In fact it does not have any at the moment, there is no essential. Or
> maybe that is hidden in debian-magic.
[...]

Essential packages are always installed, which means there is no need
to declare a dependency on them (in fact it is discouraged):
https://www.debian.org/doc/debian-policy/#dependencies

Ben.

--
Ben Hutchings
When in doubt, use brute force. - Ken Thompson


Attachments:
signature.asc (833.00 B)
This is a digitally signed message part

2017-12-04 09:02:14

by Henning Schild

[permalink] [raw]
Subject: Re: [PATCH] builddeb: introduce variables for control-file customization

Am Fri, 1 Dec 2017 18:47:38 +0000
schrieb Ben Hutchings <[email protected]>:

> On Fri, 2017-12-01 at 19:34 +0100, Henning Schild wrote:
> > Am Fri, 1 Dec 2017 16:51:12 +0000
> > schrieb Ben Hutchings <[email protected]>:
> >
> > > On Fri, 2017-12-01 at 15:56 +0000, Henning Schild wrote:
> > > > The debian packages coming out of "make *deb-pkg" lack some
> > > > critical information in the control-files e.g. the "Depends:"
> > > > field. If one tries to install a fresh system with such a
> > > > "linux-image" debootstrap or multistrap might try to install
> > > > the kernel before its deps and the package hooks will fail.
> > >
> > > I assume you're talking about those hook scripts being run while
> > > the packages they belong to are only unpacked? I hadn't thought
> > > about this issue, but it seems to me that those hook scripts
> > > generally ought to be fixed to handle this case properly. Most
> > > of the packages installing hook scripts for kernel packages are
> > > not going to be dependencies of linux-image packages, so it will
> > > never be safe for them to assume their package has been fully
> > > installed.
> >
> > Yes these hook scripts fail when installing the kernel on another
> > system. Indeed we seem to have a case where packages installed on
> > the build-machine cause install-time deps for the package.
>
> Can you give an example? I don't see how that would happen.

Scripts in /etc/kernel/ will end up as hooks for the kernel-package, if
you do not set KDEB_HOOKDIR. Looking at an example system that pulls in
things like "pm-utils, grub-pc .. initramfs". With this mechanism any
package placing a hook in /etc/kernel can influence the deps. I guess
in practice that is prevented with policies on what these scripts are
allowed to do.

> > In my case the build-machine is pretty minimal but i still want
> > some of that i.e. initramfs.
> >
> > > > Different debian-based distros use different values for the
> > > > missing fields. And the values differ between distro versions
> > > > as well. So hardcoding of e.g. "Depends" is not possible.
> > >
> > > The dependencies also depend on the kernel configuration. (And a
> > > custom kernel built with 'make deb-pkg' often won't have any
> > > dependencies outside of essential packages.)
> >
> > In fact it does not have any at the moment, there is no essential.
> > Or maybe that is hidden in debian-magic.
> [...]
>
> Essential packages are always installed, which means there is no need
> to declare a dependency on them (in fact it is discouraged):
> https://www.debian.org/doc/debian-policy/#dependencies

Ok, i will need to double-check how multistrap deals with those.

Henning

> Ben.
>

2017-12-04 13:15:31

by Riku Voipio

[permalink] [raw]
Subject: Re: [PATCH] builddeb: introduce variables for control-file customization

On 4 December 2017 at 11:01, Henning Schild <[email protected]> wrote:
> Am Fri, 1 Dec 2017 18:47:38 +0000
> schrieb Ben Hutchings <[email protected]>:
>
>> On Fri, 2017-12-01 at 19:34 +0100, Henning Schild wrote:
>> > Am Fri, 1 Dec 2017 16:51:12 +0000
>> > schrieb Ben Hutchings <[email protected]>:
>> >
>> > > On Fri, 2017-12-01 at 15:56 +0000, Henning Schild wrote:
>> > > > The debian packages coming out of "make *deb-pkg" lack some
>> > > > critical information in the control-files e.g. the "Depends:"
>> > > > field. If one tries to install a fresh system with such a
>> > > > "linux-image" debootstrap or multistrap might try to install
>> > > > the kernel before its deps and the package hooks will fail.

We don't usually install kernel during debootstrap.

>> > > I assume you're talking about those hook scripts being run while
>> > > the packages they belong to are only unpacked? I hadn't thought
>> > > about this issue, but it seems to me that those hook scripts
>> > > generally ought to be fixed to handle this case properly. Most
>> > > of the packages installing hook scripts for kernel packages are
>> > > not going to be dependencies of linux-image packages, so it will
>> > > never be safe for them to assume their package has been fully
>> > > installed.
>> >
>> > Yes these hook scripts fail when installing the kernel on another
>> > system. Indeed we seem to have a case where packages installed on
>> > the build-machine cause install-time deps for the package.
>>
>> Can you give an example? I don't see how that would happen.
>
> Scripts in /etc/kernel/ will end up as hooks for the kernel-package, if
> you do not set KDEB_HOOKDIR. Looking at an example system that pulls in
> things like "pm-utils, grub-pc .. initramfs". With this mechanism any
> package placing a hook in /etc/kernel can influence the deps. I guess
> in practice that is prevented with policies on what these scripts are
> allowed to do.

This doesn't seem to be the case on my system. And from the bits that
handle debhookdir in scripts/package/builddeb, I don't see how this
is happening.

2017-12-04 14:36:07

by Ben Hutchings

[permalink] [raw]
Subject: Re: [PATCH] builddeb: introduce variables for control-file customization

On Mon, 2017-12-04 at 10:01 +0100, Henning Schild wrote:
> Am Fri, 1 Dec 2017 18:47:38 +0000
> schrieb Ben Hutchings <[email protected]>:
>
> > On Fri, 2017-12-01 at 19:34 +0100, Henning Schild wrote:
> > > Am Fri, 1 Dec 2017 16:51:12 +0000
> > > schrieb Ben Hutchings <[email protected]>:
> > >
> > > > On Fri, 2017-12-01 at 15:56 +0000, Henning Schild wrote:
> > > > > The debian packages coming out of "make *deb-pkg" lack some
> > > > > critical information in the control-files e.g. the "Depends:"
> > > > > field. If one tries to install a fresh system with such a
> > > > > "linux-image" debootstrap or multistrap might try to install
> > > > > the kernel before its deps and the package hooks will fail.
> > > >
> > > > I assume you're talking about those hook scripts being run while
> > > > the packages they belong to are only unpacked? I hadn't thought
> > > > about this issue, but it seems to me that those hook scripts
> > > > generally ought to be fixed to handle this case properly. Most
> > > > of the packages installing hook scripts for kernel packages are
> > > > not going to be dependencies of linux-image packages, so it will
> > > > never be safe for them to assume their package has been fully
> > > > installed.
> > >
> > > Yes these hook scripts fail when installing the kernel on another
> > > system. Indeed we seem to have a case where packages installed on
> > > the build-machine cause install-time deps for the package.
> >
> > Can you give an example? I don't see how that would happen.
>
> Scripts in /etc/kernel/ will end up as hooks for the kernel-package, if
> you do not set KDEB_HOOKDIR. Looking at an example system that pulls in
> things like "pm-utils, grub-pc .. initramfs". With this mechanism any
> package placing a hook in /etc/kernel can influence the deps.
[...]

That is not a dependency. That is an extension mechanism.

Ben.

--
Ben Hutchings
Lowery's Law:
If it jams, force it. If it breaks, it needed replacing anyway.


Attachments:
signature.asc (833.00 B)
This is a digitally signed message part