Hi,
I noticed when trying to upgrade my e2fsprogs RPM set from a 1.41.12ish
release to a 1.42.3ish release only e2fsprogs and e2fsprogs-libs got
upgraded and libcom_err didn't get upgraded despite there being ABI
differences between 1.41.12 and 1.42.3. This results in the following
type of error:
# e2fsck
e2fsck: symbol lookup error: e2fsck: undefined symbol: set_com_err_gettext
# mke2fs
mke2fs: symbol lookup error: mke2fs: undefined symbol: set_com_err_gettext
This appears to be due to the addition of set_com_err_gettext() (in
4259052093da329907e255b11bf3e799872828c7) during the 1.42 development
cycle.
Since this addition introduces an ABI incompatibility in libcom_err,
isn't the process supposed to be to increase the SO's version so that
the tools find the proper version of the library and also so that it
gets upgraded accordingly by RPM with it's autoprovides/requires bits?
Cheers,
b.
On Wed, Aug 15, 2012 at 05:48:12PM -0400, Brian J. Murrell wrote:
> I noticed when trying to upgrade my e2fsprogs RPM set from a 1.41.12ish
> release to a 1.42.3ish release only e2fsprogs and e2fsprogs-libs got
> upgraded and libcom_err didn't get upgraded despite there being ABI
> differences between 1.41.12 and 1.42.3. This results in the following
> type of error:
>
> # e2fsck
> e2fsck: symbol lookup error: e2fsck: undefined symbol: set_com_err_gettext
....
>
> Since this addition introduces an ABI incompatibility in libcom_err,
> isn't the process supposed to be to increase the SO's version so that
> the tools find the proper version of the library and also so that it
> gets upgraded accordingly by RPM with it's autoprovides/requires bits?
There's a distinction which needs to be made between adding a new
symbol to a DSO, and making an incompatible chnage to the ABI of an
existing symbol of a DSO. The former doesn't break any existing
programs (although you need to make sure you have a sufficiently new
version of the library for executables which use the newer symbols).
The latter does break previously existing programs that were linked
against the DSO.
Normally, we only bump the SO version in the latter case.
It is indeed up to packaging systems to do the right thing vis-a-vis
having the correct dependencies in their packages. Debian/dpkg does
this automatically. Note the current set of dependencies for the
e2fsprogs package in Debian:
Pre-Depends: e2fslibs (= 1.42.5-1), libblkid1 (>= 2.17.2), libc6 (>= 2.11), libcomerr2 (>= 1.42~WIP-2011-10-05-1), libss2 (>= 1.34-1), libuuid1 (>= 2.16), util-linux (>= 2.15~rc1-1)
It does this by maintaining a list of symbols and the version when it
was first introduced. See the libcomerr2.symbols file, for which an
example can be found here:
http://git.kernel.org/?p=fs/ext2/e2fsprogs.git;a=blob;f=debian/libcomerr2.symbols;h=ce3b74a19d32ea9f43d67c976753e91ca113a894;hb=HEAD
Debian has packaging tools which will examine all of the symbols used
by the executables in a package, compare it against the .symbols file
for the library, and then determine the appropriate minimum version
needed for that library, and insert it into the package. This is how
the e2fsprogs package automatically included a versioned dependency
for libcomerr2 1.42~WIP-2011-10-05-1 (since this was the first Debian
package which provided version 1.41.99, as specified in the
libcomerr2.symbosl file). However, based on the symbols used from the
libss2 library, e2fsprogs only needs libss2 version of 1.34-1 or
newer.
If another package uses libcomerr2, but doesn't use the new
set_com_err_gettext() function, then the versioned dependency for that
package might be for an older version of libcomerr2. And all of this
is done automgically by the Debian dpkg tools. Cool!!
Unfortunately, RPM doesn't seem to do this automatically. It looks
like you have to explicitly specify these dependencies in the spec
file. Currently the e2fsprogs spec file for Fedora seems to mandate
that e2fslibs must match the version of e2fsprogs exactly, while any
version of libcomerr2 will do.
If we were to change the DSO version number, it would break older
executables that didn't use this new symbol, and given that the
dependency management in the RPM spec file is all horribly manual, it
wouldn't help you, since you'd have to manually adjust the dependency
in the spec file anyway. So you might as well manually adjust it to
require at least the 1.42 version of libcom_err.
Anyway, this is a packaging bug, and Debian provides a worked example
about how this can be done completely automatically, without needing
any special handling by the release engineer. Why RPM doesn't follow
Debian's lead by having an automated system is not something I've ever
understood (but there's a reason why I use Debian and not
Fedora.... and it's not just the abomination which is GNOME 3 / GNOME OS).
Best regards,
- Ted
On 12-08-15 09:47 PM, Theodore Ts'o wrote:
>
Hi Ted,
> There's a distinction which needs to be made between adding a new
> symbol to a DSO, and making an incompatible chnage to the ABI of an
> existing symbol of a DSO. The former doesn't break any existing
> programs (although you need to make sure you have a sufficiently new
> version of the library for executables which use the newer symbols).
> The latter does break previously existing programs that were linked
> against the DSO.
Ahhh. That's a fair point.
> It is indeed up to packaging systems to do the right thing vis-a-vis
> having the correct dependencies in their packages. Debian/dpkg does
> this automatically. Note the current set of dependencies for the
> e2fsprogs package in Debian:
>
> Pre-Depends: e2fslibs (= 1.42.5-1), libblkid1 (>= 2.17.2), libc6 (>= 2.11), libcomerr2 (>= 1.42~WIP-2011-10-05-1), libss2 (>= 1.34-1), libuuid1 (>= 2.16), util-linux (>= 2.15~rc1-1)
>
> It does this by maintaining a list of symbols and the version when it
> was first introduced.
I see. I am familiar with Debian's packaging but have not really delved
into it's shared symbol dependency handling. That is indeed
interesting. Funny enough, I was thinking about something along those
lines (albeit in respect of RPM dependencies) independently while I was
working through this issue -- that is tracking the symbols from release
to release automagically.
> Debian has packaging tools which will examine all of the symbols used
> by the executables in a package, compare it against the .symbols file
> for the library, and then determine the appropriate minimum version
> needed for that library, and insert it into the package.
Yeah. It is neat. On a bit of a tangent (which we don't have to get
into here. I am sure I can research it independently if I really feel a
need to know) given what's in the *.symbols files, I can't see how those
tools track an ABI change though. There's no function "signature" to
compare. Maybe they leave ABI changes up to the developer to recognize
and bump the DSO manually.
> If another package uses libcomerr2, but doesn't use the new
> set_com_err_gettext() function, then the versioned dependency for that
> package might be for an older version of libcomerr2. And all of this
> is done automgically by the Debian dpkg tools. Cool!!
It is cool, yes.
> Unfortunately, RPM doesn't seem to do this automatically.
At the same level of granularity, it seems you are correct. But RPM
does have automatic shared library dependency processing. RPM's
automatic shared library dependency information calculations are based
entirely off of DSO versions (i.e. e2fsprogs Depends on
libcom_err.so.2()(64bit) -- which is calculated automatically by RPM),
which is why I thought that bumping the DSO would solve the RPM
packaging problems. Well, it would solve them, but as you point out,
adding a symbol may not necessarily be enough reason to bump the DSO
version -- packaging problems aside.
> It looks
> like you have to explicitly specify these dependencies in the spec
> file. Currently the e2fsprogs spec file for Fedora seems to mandate
> that e2fslibs must match the version of e2fsprogs exactly, while any
> version of libcomerr2 will do.
Well, yes, that's what the spec mandates. But RPM's automatic shared
library dependency processor will insert an implicit "Depends:
libcom_err.so.2()(64bit)", so not really *any* libcommerr2, but yes, any
DSO version 2 libcommerr2 will do.
> If we were to change the DSO version number, it would break older
> executables that didn't use this new symbol,
Yeah. This is usually solved by allowing multiple versions of the
shared library to co-exist. i.e. libcomm_err.so.2 and libcomm_err.so.3,
etc. But that requires packaging libraries with their DSO version in
the name. i.e. package names libcommerr2, libcommerr3, etc. The
libcommerr2 package would install libcomm_err.so.2 and the libcommerr3
package would install libcomm_err.so.3 and both could co-exist.
> and given that the
> dependency management in the RPM spec file is all horribly manual,
Well, not _all_ manual. There is the above mentioned automatic DSO
dependency information that matches library RPMs to tool RPMs. That is,
*if* the DSO version were bumped, this would all just automatically work
with RPM even without having to specify manual dependencies. Again,
which is why simply bumping the DSO looked so inviting to me.
> it
> wouldn't help you, since you'd have to manually adjust the dependency
> in the spec file anyway.
Well, again, no, per the above.
But to be clear, I am not necessarily using RPM's weakness (as compared
to Debian's dpkg that is, in tracking shared library versions) here as a
case to bump the libcommerr2 DSO, but just trying to clear up the
[mis-]perceptions of RPM's ability with regard to automatically
providing shared library dependency tracking. It can do it. It just
depends on DSO version bumping, even for adding new symbols to the library.
> Anyway, this is a packaging bug, and Debian provides a worked example
> about how this can be done completely automatically, without needing
> any special handling by the release engineer.
Yeah.
> Why RPM doesn't follow
> Debian's lead by having an automated system
Well, it does. It's just not as granular as Debian's and seems to
expect that additions to a DSO will cause a DSO version bump. I wonder
what the RPM developers' perspective on this is, especially given that
Debian has provided a working example of how simply adding symbols and
not bumping the DSO version can still work with packaging.
> is not something I've ever
> understood (but there's a reason why I use Debian and not
> Fedora.... and it's not just the abomination which is GNOME 3 / GNOME OS).
Oh lord. Don't get me started on the abomination which is GNOME 3. :-)
Fortunately Cinnamon brings back some level of usability to GNOME 3
with it's new/replacement shell.
Cheers,
b.