Announcing Binary Compatibility/Testing
In talking to end users, distributions, OSS developers and
large scale ISV's one issue kept popping up. And that is
the fact that binaries keep breaking.
This is a real problem for large end users deploying Linux
in that they like to be able to run/roll forward the same version
of an application for 5 or so years. They can do this with their
legacy operating systems and we need to be able to do this
with Linux.
One of the big problems is that these ISV's release and test
on a cycle that is measured in calendar quarters and of course
the OSS cycle is measured in days. The idea is to move
testing of these binary applications upstream to match
the OSS development cycle. For this purpose I've started
a mailing list to discuss how to accomplish this. I've
got slides for anybody who is interested. (PDF.)
http://lists.osdl.org/mailman/listinfo/binary_sig
http://groups.osdl.org/sig (Follow binary testing for slides)
Let the flaming start. :-)
Tim
--
Timothy D. Witham - Chief Technology Officer - [email protected]
Open Source Development Lab Inc - A non-profit corporation
12725 SW Millikan Way - Suite 400 - Beaverton OR, 97005
(503)-626-2455 x11 (office) (503)-702-2871 (cell)
(503)-626-2436 (fax)
On Wed, 2004-10-13 at 15:16 -0700, Timothy D. Witham wrote:
> Announcing Binary Compatibility/Testing
>
> In talking to end users, distributions, OSS developers and
> large scale ISV's one issue kept popping up. And that is
> the fact that binaries keep breaking.
>
> This is a real problem for large end users deploying Linux
> in that they like to be able to run/roll forward the same version
> of an application for 5 or so years. They can do this with their
> legacy operating systems and we need to be able to do this
> with Linux.
>
> One of the big problems is that these ISV's release and test
> on a cycle that is measured in calendar quarters and of course
> the OSS cycle is measured in days. The idea is to move
> testing of these binary applications upstream to match
> the OSS development cycle. For this purpose I've started
> a mailing list to discuss how to accomplish this. I've
> got slides for anybody who is interested. (PDF.)
>
> http://lists.osdl.org/mailman/listinfo/binary_sig
>
> http://groups.osdl.org/sig (Follow binary testing for slides)
http://groups.osdl.org/sigs not sig - sorry
Tim
>
> Let the flaming start. :-)
>
> Tim
>
>
--
Timothy D. Witham - Chief Technology Officer - [email protected]
Open Source Development Lab Inc - A non-profit corporation
12725 SW Millikan Way - Suite 400 - Beaverton OR, 97005
(503)-626-2455 x11 (office) (503)-702-2871 (cell)
(503)-626-2436 (fax)
Timothy D. Witham wrote:
> Announcing Binary Compatibility/Testing
[...]
> Let the flaming start. :-)
Userland ABI compatibility has always been a strongly held value in
Linux, I don't think we would flame any efforts to support that...
Jeff
On Wed, 2004-10-13 at 18:39 -0400, Jeff Garzik wrote:
> Timothy D. Witham wrote:
> > Announcing Binary Compatibility/Testing
> [...]
> > Let the flaming start. :-)
>
>
> Userland ABI compatibility has always been a strongly held value in
> Linux, I don't think we would flame any efforts to support that...
>
And hence the smile.
> Jeff
>
--
Timothy D. Witham - Chief Technology Officer - [email protected]
Open Source Development Lab Inc - A non-profit corporation
12725 SW Millikan Way - Suite 400 - Beaverton OR, 97005
(503)-626-2455 x11 (office) (503)-702-2871 (cell)
(503)-626-2436 (fax)
On Wed, 2004-10-13 at 18:39 -0400, Jeff Garzik wrote:
> Userland ABI compatibility has always been a strongly held value in
> Linux, I don't think we would flame any efforts to support that...
Yah. With the exception of maybe changing something in /proc (which has
been rare, and hopefully will never happen with /sys) the kernel-to-user
ABI is really stable.
I'd venture, in fact, to say that this effort is very important but does
not affect the kernel at all. Current "fault" lies in things e.g. like
the C++ ABI, which is constantly fluctuating (rightly so, to fix bugs,
but still).
Any other incompatibility lies in libraries, but we have library
versioning. There is nothing wrong with newer libs breaking
compatibility so long as they have a different soname. Vendors just
need to ship compat libs and ISV's need to make sure they request the
right lib and don't touch internals.
Robert Love
On Wed, 2004-10-13 at 19:24 -0400, Robert Love wrote:
> On Wed, 2004-10-13 at 18:39 -0400, Jeff Garzik wrote:
>
> > Userland ABI compatibility has always been a strongly held value in
> > Linux, I don't think we would flame any efforts to support that...
>
> Yah. With the exception of maybe changing something in /proc (which has
> been rare, and hopefully will never happen with /sys) the kernel-to-user
> ABI is really stable.
>
I would tend to agree with that statement.
> I'd venture, in fact, to say that this effort is very important but does
> not affect the kernel at all. Current "fault" lies in things e.g. like
> the C++ ABI, which is constantly fluctuating (rightly so, to fix bugs,
> but still).
>
> Any other incompatibility lies in libraries, but we have library
> versioning. There is nothing wrong with newer libs breaking
> compatibility so long as they have a different soname. Vendors just
> need to ship compat libs and ISV's need to make sure they request the
> right lib and don't touch internals.
>
Part of the problem is knowing which things to request. I've
envisioned a database that has the matrix of tests and packages so that
people like ISV's and system integrators will be able to look
up what has been tested and passed. I think that this database
is the crucial portion of the new development.
I also expect that part of this process will be the finding that
an ISV used the API in a way that could of got them in trouble
and that the new version closes that hole. In this case it would
be a bug on the ISVs side but it would be known long before
it got deployed and the ISV could schedule the development and
testing of the patch to their software as part of their normal
deployment schedule.
> Robert Love
>
Tim
--
Timothy D. Witham - Chief Technology Officer - [email protected]
Open Source Development Lab Inc - A non-profit corporation
12725 SW Millikan Way - Suite 400 - Beaverton OR, 97005
(503)-626-2455 x11 (office) (503)-702-2871 (cell)
(503)-626-2436 (fax)
On Wed, 13 Oct 2004, Robert Love wrote:
>
> Any other incompatibility lies in libraries, but we have library
> versioning.
No we don't.
Yes, we "have the technology". But it's not actually used for libc (which
is most of the problematic stuff), so we do not actually have library
versioning.
Instead, glibc tries very hard to be binary compatible, and invariably
fails occasionally.
Oh, well.
Linus
El Thu, 14 Oct 2004 11:32:14 -0700 (PDT) Linus Torvalds <[email protected]> escribi?:
> No we don't.
>
> Yes, we "have the technology". But it's not actually used for libc (which
> is most of the problematic stuff), so we do not actually have library
> versioning.
is that a solvable problem?
quoting some dragonfly thread (whose aim is to implement something which
would solve that)
[...] via VFS 'environments', causes any particular package to see only
the dependancies that it depends on, and the proper version of said
dependancies as well. Multiple versions of third party apps that normally
conflict with each other could be installed simultaniously. The
packaging-system-controlled VFS environment would also hide everything a
package does not depend on, like other libraries in the system, in order
to guarentee that the dependancies listed in the packaging system are in
fact what the application depends on. There's no point in having a
packaging system that can't detect broken and incorrect dependancies or
we wind up with the same mess that we have with ports.
Diego Calleja (who wonders if this is feasible/worth of it)
Hi, Robert Love wrote:
> Any other incompatibility lies in libraries, but we have library
> versioning. There is nothing wrong with newer libs breaking
> compatibility so long as they have a different soname.
Glibc doesn't use library versioning. It uses *symbol* versioning.
Thus, for some library entry points you now see multiple symbols with the
same name but different versions (ancient vs. old vs. new "struct stat",
16- vs. 32-bit UID, whatever). As long as the header files you compile
against match the library you link to, it all works transparently.
For the library's user, that is. The author's job is harder...
--
Matthias Urlichs | {M:U} IT Design @ m-u-it.de | [email protected]
Hi Robert,
On Wed, Oct 13, 2004 at 07:24:15PM -0400, Robert Love wrote:
> I'd venture, in fact, to say that this effort is very important but does
> not affect the kernel at all. Current "fault" lies in things e.g. like
> the C++ ABI, which is constantly fluctuating (rightly so, to fix bugs,
> but still).
These days, it's mostly libstdc++ which is still developing; the
compiler side (name mangling, layout of virt tables, ...) is quite
stable these days; it's these occasional bugs which are found from
time to time in the most obscure scenarios, which are often overstated.
> Any other incompatibility lies in libraries, but we have library
> versioning. There is nothing wrong with newer libs breaking
> compatibility so long as they have a different soname. Vendors just
> need to ship compat libs and ISV's need to make sure they request the
> right lib and don't touch internals.
Except that we have deeps stacks of libraries these days and things
break apart if you need different versions of one library at the same
time ...
D needs libs C and B, but C needs A and B needs A'.
In reality, the picture is many more levels deep.
Thus breaking libs all the time brings nice challenges for integrators
and ISVs as well. Fortunately, most libs don't do that.
Regards,
--
Kurt Garloff, Director SUSE Labs, Novell Inc.