Hi all,
As has been noticed recently by a lot of different people, it seems like
we are breaking the userspace<->kernelspace interface a lot. Well, in
looking back over time, we always have been doing this, but no one seems
to notice (proc files changing format and location, netlink library
bindings, etc.)
Linux is a dynamic system, we add and change things all the time based
on the need of its developers and users. Because of this, we now run on
more platforms than any other operating system ever has, from the
world's top supercomputers, to the phone in your pocket. It is how we
have survived so far, and is how we will survive in the future.
In order to ensure that we can continue to be dynamic in the future, and
not get bogged down by interfaces that are half-baked, or just turn out
to be wrong once we implement them and find ways to break them (anyone
remember the sys_futex evolution?) we need to be able to handle the
changes in the userspace<->kernelspace ABI properly.
So, here's a first cut at how we can do this. Lots of other operating
systems explicity document what the interfaces to it are, and give a
"stability" rating of those interfaces (for one example, look at
http://opensolaris.org/os/community/onnv/devref_toc/devref_7/ ). I feel
that we too need to document this interface, in order to keep everyone
in the loop and not cause any unwanted surprises at times they do not
need them (like right before a company's deadline.)
I've sketched out a directory structure that starts in
Documentation/ABI/ and has five different states, "stable", "testing",
"unstable", "obsolete", and "private". The README file describes these
different states, and how things can move between them. I've also
seeded the directories with some well known examples of the different
interfaces that are already in these states.
So, any comments? Criticisms?
thanks,
greg k-h
p.s. I'd like to thank Kay Sievers for the main idea behind this
structure and need to document this.
-----------------------
From: Greg Kroah-Hartman <[email protected]>
Subject: Add kernel<->userspace ABI stability documentation
Signed-off-by: Kay Sievers <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
Documentation/ABI/README | 92 +++++++++++++++++++++++++++++++
Documentation/ABI/obsolete/devfs | 13 ++++
Documentation/ABI/private/alsa | 8 ++
Documentation/ABI/stable/syscalls | 10 +++
Documentation/ABI/stable/sysfs-module | 29 +++++++++
Documentation/ABI/testing/sysfs-class | 16 +++++
Documentation/ABI/unstable/sysfs-devices | 25 ++++++++
7 files changed, 193 insertions(+)
--- /dev/null
+++ gregkh-2.6/Documentation/ABI/README
@@ -0,0 +1,92 @@
+This directory attempts to document the ABI between the Linux kernel and
+userspace, and the relative stability of these interfaces. Due to the
+ever changing nature of Linux, and the differing maturity levels, these
+interfaces should be used by userspace programs in different ways.
+
+We have four different levels of ABI stability, as shown by the four
+different subdirectorys in this location. Interfaces may change levels
+of stability according to the rules described below.
+
+The different levels of stability are:
+
+ stable/
+ This directory documents the interfaces that have determined to
+ be stable. Userspace programs are free to use these interfaces
+ with no restrictions, and backward compatibility for them will
+ be guaranteed for at least 2 years. Most simple interfaces
+ (like syscalls) are expected to never change and always be
+ available.
+
+ testing/
+ This directory documents interfaces that are felt to be stable,
+ as the main development of this interface has been completed.
+ The interface can be changed to add new features, but the
+ current interface will not break by doing this.
+ Userspace programs can start to rely on these interfaces, but
+ they must be aware of changes that can occur before these
+ interfaces move to be marked stable. Programs that use these
+ interfaces are strongly encouraged to add their name to the
+ description of these interfaces, so that the kernel developers
+ can easily notify them if any changes occur (see the description
+ of the layout of the files below for details on how to do this.)
+
+ unstable/
+ This directory documents interfaces that are known to be
+ unstable, and not ready for widespread use by a lot of different
+ programs. That is not to say that they can not be used, but
+ developers of such programs should track their changes very
+ closely. Again, programs that uses these interfaces are
+ strongly encouraged to add their names to the description of the
+ interfaces so that they can be notified of changes.
+
+ obsolete/
+ This directory documents interfaces that are still remaining in
+ the kernel, but are marked to be removed at some later point in
+ time. The description of the interface will document the reason
+ why it is obsolete and when it can be expected to be removed.
+
+ private/
+ This interface is private between the kernel and a helper
+ userspace program or library. If you wish to use this interface
+ (like alsa or netlink) userspace must use the helper library and
+ not use the raw kernel interface directly.
+
+
+Every file in these directories will contain the following information:
+
+What: Short description of the interface
+Created: Date created
+Contact: Primary contact for this interface (may be a mailing list)
+Description: Long description of the interface and how to use it.
+Users: All users of this interface who wish to be notified for
+ when it changes. This is very important for interfaces
+ in the "testing" stage, and the "unstable" stage, so
+ that kernel developers can work with userspace
+ developers to ensure that things do not break in ways
+ that are unacceptable. It is also important to get
+ feedback for these interfaces to make sure they are
+ working in a proper way and do not need to be changed
+ further.
+
+
+How things move between states:
+
+Interfaces in stable may move to obsolete, as long as the proper
+notification is given.
+
+Interfaces may be removed from obsolete and the kernel as long as the
+documented amount of time has gone by.
+
+Interfaces may be moved from unstable to testing whenever the developers
+feel they are finished with the interface. Interfaces should not remain
+in unstable for very long periods of time without good reasons.
+
+Interfaces may be removed entirely from the kernel without notice if
+they are in the unstable state.
+
+Interfaces in the testing state can move to the stable state when the
+developers feel they are finished. They can not be removed from the
+kernel tree without going through the obsolete state first.
+
+It's up to the developer to place their interface in the category they
+wish for it to start out in.
--- /dev/null
+++ gregkh-2.6/Documentation/ABI/obsolete/devfs
@@ -0,0 +1,13 @@
+What: devfs
+Date: July 2005
+Contact: Greg Kroah-Hartman <[email protected]>
+Description:
+ devfs has been unmaintained for a number of years, has unfixable
+ races, contains a naming policy within the kernel that is
+ against the LSB, and can be replaced by using udev.
+ The files fs/devfs/*, include/linux/devfs_fs*.h will be removed,
+ along with the the assorted devfs function calls throughout the
+ kernel tree.
+
+Users:
+
--- /dev/null
+++ gregkh-2.6/Documentation/ABI/stable/syscalls
@@ -0,0 +1,10 @@
+What: The kernel syscall interface
+Description:
+ This interface matches much of the POSIX interface and is based
+ on it and other Unix based interfaces. It will only be added to
+ over time, and not have things removed from it.
+
+ Note that this interface is different for every architecture
+ that Linux supports. Please see the arch specific documentation
+ for details on the syscall numbers that are to be mapped to each
+ syscall.
--- /dev/null
+++ gregkh-2.6/Documentation/ABI/stable/sysfs-module
@@ -0,0 +1,29 @@
+What: /sys/module
+Description:
+ The /sys/module tree consists of the following structure:
+
+ /sys/module/MODULENAME
+ The name of the module that is in the kernel.
+ This module name will show up both if the module is built
+ directly into the kernel, or if it is loaded as a dyanmic
+ module.
+
+ /sys/module/MODULENAME/parameters
+ This directory contains individual files that are each
+ individual parameters into the module that are are able to be
+ changed at runtime. See the individual module documentation as
+ to the contents of these parameters and what they accomplish.
+
+ Note: The individual parameter names and values are not
+ considered stable, only the fact that they will be placed into
+ this location within sysfs. See the individual driver
+ documentation for details as to the stability of the different
+ parameters.
+
+ /sys/module/MODULENAME/refcnt
+ If the module is able to be unloaded from the kernel, this file
+ will contain the current reference count of the module.
+
+ Note: If the module is built into the kernel, or if the
+ CONFIG_MODULE_UNLOAD kernel configuration value is not enabled,
+ this file will not be present.
--- /dev/null
+++ gregkh-2.6/Documentation/ABI/testing/sysfs-class
@@ -0,0 +1,16 @@
+What: /sys/class/
+Date: Febuary 2006
+Contact: Greg Kroah-Hartman <[email protected]>
+Description:
+ The /sys/class directory will consist of a group of
+ subdirectories describing individual classes of devices
+ in the kernel. The individual directories will consist
+ of either subdirectories, or symlinks to other
+ directories.
+
+ All programs that use this directory tree must be able
+ to handle both subdirectories or symlinks in order to
+ work properly.
+
+Users:
+ udev <[email protected]>
--- /dev/null
+++ gregkh-2.6/Documentation/ABI/unstable/sysfs-devices
@@ -0,0 +1,25 @@
+What: /sys/devices
+Date: February 2006
+Contact: Greg Kroah-Hartman <[email protected]>
+Description:
+ The /sys/devices tree contains a snapshot of the
+ internal state of the kernel device tree. Devices will
+ be added and removed dynamically as the machine runs,
+ and between different kernel versions, the layout of the
+ devices within this tree will change.
+
+ Please do not rely on the format of this tree because of
+ this. If a program wishes to find different things in
+ the tree, please use the /sys/class structure and rely
+ on the symlinks there to point to the proper location
+ within the /sys/devices tree of the individual devices.
+ Or rely on the uevent messages to notify programs of
+ devices being added and removed from this tree to find
+ the location of those devices.
+
+ Note that sometimes not all devices along the directory
+ chain will have emitted uevent messages, so userspace
+ programs must be able to handle such occurances.
+
+Users:
+ udev <[email protected]>
--- /dev/null
+++ gregkh-2.6/Documentation/ABI/private/alsa
@@ -0,0 +1,8 @@
+What: Kernel Sound interface
+Date: Feburary 2006
+Who: Jaroslav Kysela <[email protected]>
+Description:
+ The use of the kernel sound interface must be done
+ through the ALSA library. For more details on this,
+ please see http://www.alsa-project.org/ and contact
+ <[email protected]>
On Mon, 2006-02-27 at 11:01 -0800, Greg KH wrote:
> Hi all,
>
> As has been noticed recently by a lot of different people, it seems like
> we are breaking the userspace<->kernelspace interface a lot. Well, in
> looking back over time, we always have been doing this, but no one seems
> to notice (proc files changing format and location, netlink library
> bindings, etc.)
2 remarks
1) it would make sense to keep track of "removed" interfaces as well
2) the per interface description needs a "depends on config option"
field; not all options are always there, but depend on a config option
to be set. It makes a lot of sense to mark these as such so that users
KNOW they have to deal with the interface not being there occasionally,
depending on the kernel.
On Mon, Feb 27, 2006 at 08:08:18PM +0100, Arjan van de Ven wrote:
> On Mon, 2006-02-27 at 11:01 -0800, Greg KH wrote:
> > Hi all,
> >
> > As has been noticed recently by a lot of different people, it seems like
> > we are breaking the userspace<->kernelspace interface a lot. Well, in
> > looking back over time, we always have been doing this, but no one seems
> > to notice (proc files changing format and location, netlink library
> > bindings, etc.)
>
>
> 2 remarks
>
> 1) it would make sense to keep track of "removed" interfaces as well
Good idea, removed/ should be where things go to after obsolete so
people can find things in the future.
> 2) the per interface description needs a "depends on config option"
> field; not all options are always there, but depend on a config option
> to be set. It makes a lot of sense to mark these as such so that users
> KNOW they have to deal with the interface not being there occasionally,
> depending on the kernel.
Hm, almost _everything_ is configurable these days, including sysfs. Do
we really want to keep the config value in sync with the kernel config
system too? I can add it, but it seems a bit unnecessary.
thanks for the quick feedback.
greg k-h
> > 2) the per interface description needs a "depends on config option"
> > field; not all options are always there, but depend on a config option
> > to be set. It makes a lot of sense to mark these as such so that users
> > KNOW they have to deal with the interface not being there occasionally,
> > depending on the kernel.
>
> Hm, almost _everything_ is configurable these days, including sysfs. Do
> we really want to keep the config value in sync with the kernel config
> system too? I can add it, but it seems a bit unnecessary.
well there is "configurable" and "configurable"... but yeah. I don't
mean to document which config options it depends on, but more
"applications need to deal with this not being there since it's optional
and not uncommon to be disabled" or something.
On Feb 27, 2006, at 1:01 PM, Greg KH wrote:
> Hi all,
>
> As has been noticed recently by a lot of different people, it seems
> like
> we are breaking the userspace<->kernelspace interface a lot. Well, in
> looking back over time, we always have been doing this, but no one
> seems
> to notice (proc files changing format and location, netlink library
> bindings, etc.)
>
> Linux is a dynamic system, we add and change things all the time based
> on the need of its developers and users. Because of this, we now
> run on
> more platforms than any other operating system ever has, from the
> world's top supercomputers, to the phone in your pocket. It is how we
> have survived so far, and is how we will survive in the future.
>
> In order to ensure that we can continue to be dynamic in the
> future, and
> not get bogged down by interfaces that are half-baked, or just turn
> out
> to be wrong once we implement them and find ways to break them (anyone
> remember the sys_futex evolution?) we need to be able to handle the
> changes in the userspace<->kernelspace ABI properly.
>
> So, here's a first cut at how we can do this. Lots of other operating
> systems explicity document what the interfaces to it are, and give a
> "stability" rating of those interfaces (for one example, look at
> http://opensolaris.org/os/community/onnv/devref_toc/devref_7/ ). I
> feel
> that we too need to document this interface, in order to keep everyone
> in the loop and not cause any unwanted surprises at times they do not
> need them (like right before a company's deadline.)
>
> I've sketched out a directory structure that starts in
> Documentation/ABI/ and has five different states, "stable", "testing",
> "unstable", "obsolete", and "private". The README file describes
> these
> different states, and how things can move between them. I've also
> seeded the directories with some well known examples of the different
> interfaces that are already in these states.
>
> So, any comments? Criticisms?
It would be nice if we can come up with some way for Linus to
document state changes in his release notes.
- kumar
On Mon, Feb 27, 2006 at 01:22:05PM -0600, Kumar Gala wrote:
> It would be nice if we can come up with some way for Linus to
> document state changes in his release notes.
The diffstat in the directories Documentation/ABI/ would give that for
whomever wants to read it.
thanks,
greg k-h
Greg KH <[email protected]> writes:
> Hi all,
>
> As has been noticed recently by a lot of different people, it seems like
> we are breaking the userspace<->kernelspace interface a lot. Well, in
> looking back over time, we always have been doing this, but no one seems
> to notice (proc files changing format and location, netlink library
> bindings, etc.)
Ok, but how do you plan to address the basic practical problem?
People cannot freely upgrade/downgrade kernels anymore since udev/hal
are used widely in distributions.
Does it imply you plan to change udev/hal to only use stable interfaces
for now? I would applaud such a move, but I guess it would come
at the cost of functionality.
If these applications are not changed then the documentation is likely useless
because it won't help anyways - things will still break, kernel
hackers and users will curse you all the time when they want
to test kernels etc.
> --- /dev/null
> +++ gregkh-2.6/Documentation/ABI/stable/syscalls
> @@ -0,0 +1,10 @@
> +What: The kernel syscall interface
> +Description:
> + This interface matches much of the POSIX interface and is based
> + on it and other Unix based interfaces. It will only be added to
> + over time, and not have things removed from it.
Some ioctls and socket options unfortunately don't follow this. I
guess you will need to document them separately.
Could be ugly to have hundreds of files for ioctls though.
Perhaps define core ioctls and then driver ioctls and define
all the driver ioctls unstable by default? But that also
would just mean the category stable would be useless
because people always would need to use unstable interfaces
too.
> --- /dev/null
> +++ gregkh-2.6/Documentation/ABI/testing/sysfs-class
> @@ -0,0 +1,16 @@
> +What: /sys/class/
> +Date: Febuary 2006
> +Contact: Greg Kroah-Hartman <[email protected]>
> +Description:
> + The /sys/class directory will consist of a group of
> + subdirectories describing individual classes of devices
> + in the kernel. The individual directories will consist
> + of either subdirectories, or symlinks to other
> + directories.
> +
> + All programs that use this directory tree must be able
> + to handle both subdirectories or symlinks in order to
> + work properly.
What good is it if you don't say anything about the stability of its contents?
Looks far too vague to me.
-Andi
El Mon, 27 Feb 2006 11:01:50 -0800,
Greg KH <[email protected]> escribi?:
> I've sketched out a directory structure that starts in
> Documentation/ABI/ and has five different states, "stable", "testing",
> "unstable", "obsolete", and "private". The README file describes these
With the current development model, does it have sense to have a "testing"
stage? Once the interfaces are released in the main kernel, people is going
to use them just like they were stable...
On Mon, Feb 27, 2006 at 11:01:50AM -0800, Greg KH wrote:
> --- /dev/null
> +++ gregkh-2.6/Documentation/ABI/private/alsa
> @@ -0,0 +1,8 @@
> +What: Kernel Sound interface
> +Date: Feburary 2006
> +Who: Jaroslav Kysela <[email protected]>
> +Description:
> + The use of the kernel sound interface must be done
> + through the ALSA library. For more details on this,
> + please see http://www.alsa-project.org/ and contact
> + <[email protected]>
How can something as widely used as sound not work from one kernel version
to the next, as seems to be implied with the "private" nature of the ABI?
This is a total cop-out and is IMHO very amateur of the developers. If
something like this is to be the case, at the very least the alsa libraries
need to provide a stable ABI and be shipped with the kernel.
-ben
On Mon, Feb 27, 2006 at 08:31:28PM +0100, Andi Kleen wrote:
> Greg KH <[email protected]> writes:
>
> > Hi all,
> >
> > As has been noticed recently by a lot of different people, it seems like
> > we are breaking the userspace<->kernelspace interface a lot. Well, in
> > looking back over time, we always have been doing this, but no one seems
> > to notice (proc files changing format and location, netlink library
> > bindings, etc.)
>
> Ok, but how do you plan to address the basic practical problem?
> People cannot freely upgrade/downgrade kernels anymore since udev/hal
> are used widely in distributions.
I can freely upgrade/downgrade kernels on some distros[1] if I wish to,
as they support such things. Just complain to your distro maker if you
have this issue :)
I'm worrying about documenting this stuff, and allowing a way for kernel
people to work with the userspace people that use these interfaces to
hopefully not break things in ways that are unexpected, and/or break
things at all.
> Does it imply you plan to change udev/hal to only use stable interfaces
> for now? I would applaud such a move, but I guess it would come
> at the cost of functionality.
It all depends on the type of interface, sysfs is still shaking itself
out as more and more people use it. udev today can work with the
changes we have in store for sysfs in the future (see Kay's old patches
on lkml for the kernel for examples of where we are going), so udev
looks to work just fine. But we want to know what other programs uses
these interfaces so they too can be prepared in case things happen.
And remember, HAL is there to handle all of the crap and flux in the
kernel api and hide it from all other userspace programs. It is _much_
easier to work with one program to ensure that we don't break things and
test stuff out to see how they will work better, than have to run around
all of Gnome and KDE to touch up all of the different programs. But
that's a topic that has nothing to do with this thread, sorry.
> If these applications are not changed then the documentation is likely
> useless because it won't help anyways - things will still break,
> kernel hackers and users will curse you all the time when they want to
> test kernels etc.
That's what the "Users:" lines are for, so that we can all work together
as one big happy team :)
> > --- /dev/null
> > +++ gregkh-2.6/Documentation/ABI/stable/syscalls
> > @@ -0,0 +1,10 @@
> > +What: The kernel syscall interface
> > +Description:
> > + This interface matches much of the POSIX interface and is based
> > + on it and other Unix based interfaces. It will only be added to
> > + over time, and not have things removed from it.
>
> Some ioctls and socket options unfortunately don't follow this. I
> guess you will need to document them separately.
Agreed.
> Could be ugly to have hundreds of files for ioctls though.
> Perhaps define core ioctls and then driver ioctls and define
> all the driver ioctls unstable by default? But that also
> would just mean the category stable would be useless
> because people always would need to use unstable interfaces
> too.
No, not many programs use ioctls. And a lot of them are stable and have
not changed in years (tty, block, etc.) and should be explicitly
documented here.
These examples were just examples of things that go into the different
directories. There is a lot more work ahead to document everything that
is currently implemented.
> > --- /dev/null
> > +++ gregkh-2.6/Documentation/ABI/testing/sysfs-class
> > @@ -0,0 +1,16 @@
> > +What: /sys/class/
> > +Date: Febuary 2006
> > +Contact: Greg Kroah-Hartman <[email protected]>
> > +Description:
> > + The /sys/class directory will consist of a group of
> > + subdirectories describing individual classes of devices
> > + in the kernel. The individual directories will consist
> > + of either subdirectories, or symlinks to other
> > + directories.
> > +
> > + All programs that use this directory tree must be able
> > + to handle both subdirectories or symlinks in order to
> > + work properly.
>
> What good is it if you don't say anything about the stability of its contents?
> Looks far too vague to me.
The "contents" are merely symlinks back to the device (well, some are
symlinks, others are directories, in the end all will be symlinks.)
And yes, the individual classes need to also be documented to be
through, again, this was a first cut to provide examples, not to be
complete. Patches are welcome to help flush it all out.
thanks,
greg k-h
[1] Gentoo for example works almost just fine[2] with udev with kernels
ranging from 2.6.9 to 2.6.16-rc4, I haven't tried older kernels than
that in a long time.
[2] cdrom symlinks don't get created on older kernels due to IDE
changes, but a simple rule change handles that, and could be added to
the main install if it becomes annoying.
On Mon, Feb 27, 2006 at 02:36:54PM -0500, Benjamin LaHaise wrote:
> On Mon, Feb 27, 2006 at 11:01:50AM -0800, Greg KH wrote:
> > --- /dev/null
> > +++ gregkh-2.6/Documentation/ABI/private/alsa
> > @@ -0,0 +1,8 @@
> > +What: Kernel Sound interface
> > +Date: Feburary 2006
> > +Who: Jaroslav Kysela <[email protected]>
> > +Description:
> > + The use of the kernel sound interface must be done
> > + through the ALSA library. For more details on this,
> > + please see http://www.alsa-project.org/ and contact
> > + <[email protected]>
>
> How can something as widely used as sound not work from one kernel version
> to the next, as seems to be implied with the "private" nature of the ABI?
> This is a total cop-out and is IMHO very amateur of the developers. If
> something like this is to be the case, at the very least the alsa libraries
> need to provide a stable ABI and be shipped with the kernel.
Then I suggest you work with the ALSA developers to come up with such a
"stable" api that never changes. They have been working at this for a
number of years, if it was a "simple" problem, it would have been done
already...
Anyway, netlink is in the same category, with a backing userspace
library tie :)
And, I have nothing against shipping userspace libraries with the kernel
like this, if people think that's the easiest way to do it. But even
then, the raw interface is still "private" and you need to use the
library to access it properly.
thanks,
greg k-h
On Mon, Feb 27, 2006 at 08:35:20PM +0100, Diego Calleja wrote:
> El Mon, 27 Feb 2006 11:01:50 -0800,
> Greg KH <[email protected]> escribi?:
>
>
> > I've sketched out a directory structure that starts in
> > Documentation/ABI/ and has five different states, "stable", "testing",
> > "unstable", "obsolete", and "private". The README file describes these
>
> With the current development model, does it have sense to have a "testing"
> stage? Once the interfaces are released in the main kernel, people is going
> to use them just like they were stable...
The whole point of this document is to state that they should be wary of
doing so, and be aware that things can change. And also that they need
to work _with_ the kernel developers if they are relying on things that
are "unstable" or in "testing" to be notified of future changes and just
to help make things move to "stable" quicker and more smoothly.
Also, if you look at other operating systems, they have this same kind
of "levels" of stability for their interfaces, so this is nothing new.
For us to say that our "first cut" implementation of some of these
interfaces should instantly be marked "stable" is just folley if we
think that we know-all about how stuff will work once it's being used by
lots of different people. I'm sure as hell not that smart to get
everything right the very first time, even if you might be :)
An explicit example of this is the evolution that sys_futex went
through, even after it was made a syscall...
thanks,
greg k-h
On Monday 27 February 2006 19:01, Greg KH wrote:
[snip]
> +
> +Interfaces in the testing state can move to the stable state when the
> +developers feel they are finished. They can not be removed from the
> +kernel tree without going through the obsolete state first.
> +
> +It's up to the developer to place their interface in the category they
> +wish for it to start out in.
> --- /dev/null
> +++ gregkh-2.6/Documentation/ABI/obsolete/devfs
> @@ -0,0 +1,13 @@
> +What: devfs
> +Date: July 2005
> +Contact: Greg Kroah-Hartman <[email protected]>
[snip]
July 2005? Either this date is wrong or the document is out of date.
--
Cheers,
Alistair.
'No sense being pessimistic, it probably wouldn't work anyway.'
Third year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.
On Mon, Feb 27, 2006 at 07:52:08PM +0000, Alistair John Strachan wrote:
> On Monday 27 February 2006 19:01, Greg KH wrote:
> [snip]
> > +
> > +Interfaces in the testing state can move to the stable state when the
> > +developers feel they are finished. They can not be removed from the
> > +kernel tree without going through the obsolete state first.
> > +
> > +It's up to the developer to place their interface in the category they
> > +wish for it to start out in.
> > --- /dev/null
> > +++ gregkh-2.6/Documentation/ABI/obsolete/devfs
> > @@ -0,0 +1,13 @@
> > +What: devfs
> > +Date: July 2005
> > +Contact: Greg Kroah-Hartman <[email protected]>
> [snip]
>
> July 2005? Either this date is wrong or the document is out of date.
Heh, I wish. Have you looked at
Documentation/feature-removal-schedule.txt lately?
Yeah, it's sad, but I keep trying...
thanks,
greg k-h
El Mon, 27 Feb 2006 11:49:41 -0800,
Greg KH <[email protected]> escribi?:
> An explicit example of this is the evolution that sys_futex went
> through, even after it was made a syscall...
I see...sorry for not explaining myself: Isn't "testing" just an
alias for "unstable"?
I understand that an interface can be in a "testing" stage and at
the same time not be "unstable" (even if "testing" also means it
can change). But shouldn't all the interfaces in "testing" stage
be also "unstable" by default? Or they're just different degrees
of "unstability"? It's a bit confusing IMO
On Mon, Feb 27, 2006 at 08:57:59PM +0100, Diego Calleja wrote:
> El Mon, 27 Feb 2006 11:49:41 -0800,
> Greg KH <[email protected]> escribi?:
>
>
> > An explicit example of this is the evolution that sys_futex went
> > through, even after it was made a syscall...
>
> I see...sorry for not explaining myself: Isn't "testing" just an
> alias for "unstable"?
No, look at the different descriptions that I gave them in the README
please. They are very different. If you think the wording there is not
precise enough, could you suggest some other wording?
thanks,
greg k-h
On 2/27/06, Greg KH <[email protected]> wrote:
> Hi all,
>
> As has been noticed recently by a lot of different people, it seems like
> we are breaking the userspace<->kernelspace interface a lot. Well, in
> looking back over time, we always have been doing this, but no one seems
> to notice (proc files changing format and location, netlink library
> bindings, etc.)
>
> Linux is a dynamic system, we add and change things all the time based
> on the need of its developers and users. Because of this, we now run on
> more platforms than any other operating system ever has, from the
> world's top supercomputers, to the phone in your pocket. It is how we
> have survived so far, and is how we will survive in the future.
>
> In order to ensure that we can continue to be dynamic in the future, and
> not get bogged down by interfaces that are half-baked, or just turn out
> to be wrong once we implement them and find ways to break them (anyone
> remember the sys_futex evolution?) we need to be able to handle the
> changes in the userspace<->kernelspace ABI properly.
>
> So, here's a first cut at how we can do this. Lots of other operating
> systems explicity document what the interfaces to it are, and give a
> "stability" rating of those interfaces (for one example, look at
> http://opensolaris.org/os/community/onnv/devref_toc/devref_7/ ). I feel
> that we too need to document this interface, in order to keep everyone
> in the loop and not cause any unwanted surprises at times they do not
> need them (like right before a company's deadline.)
>
> I've sketched out a directory structure that starts in
> Documentation/ABI/ and has five different states, "stable", "testing",
> "unstable", "obsolete", and "private". The README file describes these
> different states, and how things can move between them. I've also
> seeded the directories with some well known examples of the different
> interfaces that are already in these states.
>
> So, any comments? Criticisms?
>
Great initiative. Thanks.
> thanks,
>
> greg k-h
>
> p.s. I'd like to thank Kay Sievers for the main idea behind this
> structure and need to document this.
>
>
> -----------------------
>
> From: Greg Kroah-Hartman <[email protected]>
> Subject: Add kernel<->userspace ABI stability documentation
>
>
> Signed-off-by: Kay Sievers <[email protected]>
> Signed-off-by: Greg Kroah-Hartman <[email protected]>
>
> ---
> Documentation/ABI/README | 92 +++++++++++++++++++++++++++++++
> Documentation/ABI/obsolete/devfs | 13 ++++
> Documentation/ABI/private/alsa | 8 ++
> Documentation/ABI/stable/syscalls | 10 +++
> Documentation/ABI/stable/sysfs-module | 29 +++++++++
> Documentation/ABI/testing/sysfs-class | 16 +++++
> Documentation/ABI/unstable/sysfs-devices | 25 ++++++++
> 7 files changed, 193 insertions(+)
>
> --- /dev/null
> +++ gregkh-2.6/Documentation/ABI/README
> @@ -0,0 +1,92 @@
> +This directory attempts to document the ABI between the Linux kernel and
> +userspace, and the relative stability of these interfaces. Due to the
> +ever changing nature of Linux, and the differing maturity levels, these
> +interfaces should be used by userspace programs in different ways.
> +
> +We have four different levels of ABI stability, as shown by the four
> +different subdirectorys in this location. Interfaces may change levels
> +of stability according to the rules described below.
> +
> +The different levels of stability are:
> +
> + stable/
> + This directory documents the interfaces that have determined to
> + be stable. Userspace programs are free to use these interfaces
"have determined to be stable." ??
How about "have proven to be stable" or "we have determined to be
stable" or "we have defined as being stable and will not break"
instead?
> + with no restrictions, and backward compatibility for them will
> + be guaranteed for at least 2 years. Most simple interfaces
> + (like syscalls) are expected to never change and always be
> + available.
> +
> + testing/
> + This directory documents interfaces that are felt to be stable,
> + as the main development of this interface has been completed.
> + The interface can be changed to add new features, but the
> + current interface will not break by doing this.
> + Userspace programs can start to rely on these interfaces, but
> + they must be aware of changes that can occur before these
> + interfaces move to be marked stable. Programs that use these
> + interfaces are strongly encouraged to add their name to the
> + description of these interfaces, so that the kernel developers
> + can easily notify them if any changes occur (see the description
> + of the layout of the files below for details on how to do this.)
> +
Maybe a note here that "testing" interfaces may be change in
incompatible ways before moving to stable/ if grave errors or security
vulnerabilities are found in them?
> + unstable/
> + This directory documents interfaces that are known to be
> + unstable, and not ready for widespread use by a lot of different
> + programs. That is not to say that they can not be used, but
> + developers of such programs should track their changes very
> + closely. Again, programs that uses these interfaces are
> + strongly encouraged to add their names to the description of the
> + interfaces so that they can be notified of changes.
> +
> + obsolete/
> + This directory documents interfaces that are still remaining in
> + the kernel, but are marked to be removed at some later point in
> + time. The description of the interface will document the reason
> + why it is obsolete and when it can be expected to be removed.
> +
A note here that people should check
Documentation/feature-removal-schedule.txt perhaps?
> + private/
> + This interface is private between the kernel and a helper
> + userspace program or library. If you wish to use this interface
> + (like alsa or netlink) userspace must use the helper library and
> + not use the raw kernel interface directly.
> +
Perhaps add something along the lines of "Stability of the interface
is only guaranteed through the helper library, if you choose to
disregard this and use the raw kernel interface anyway, then your
program may break without warning in future kernel releases.".
> +
> +Every file in these directories will contain the following information:
> +
> +What: Short description of the interface
> +Created: Date created
Perhaps also a line here specifying the first mainline kernel release
(not -mm or -rc, but first stable kernel) that the interface is
available with. ??
> +Contact: Primary contact for this interface (may be a mailing list)
> +Description: Long description of the interface and how to use it.
> +Users: All users of this interface who wish to be notified for
> + when it changes. This is very important for interfaces
> + in the "testing" stage, and the "unstable" stage, so
> + that kernel developers can work with userspace
> + developers to ensure that things do not break in ways
> + that are unacceptable. It is also important to get
> + feedback for these interfaces to make sure they are
> + working in a proper way and do not need to be changed
> + further.
> +
> +
> +How things move between states:
> +
> +Interfaces in stable may move to obsolete, as long as the proper
> +notification is given.
> +
> +Interfaces may be removed from obsolete and the kernel as long as the
> +documented amount of time has gone by.
> +
> +Interfaces may be moved from unstable to testing whenever the developers
> +feel they are finished with the interface. Interfaces should not remain
> +in unstable for very long periods of time without good reasons.
> +
> +Interfaces may be removed entirely from the kernel without notice if
> +they are in the unstable state.
> +
> +Interfaces in the testing state can move to the stable state when the
> +developers feel they are finished. They can not be removed from the
> +kernel tree without going through the obsolete state first.
> +
> +It's up to the developer to place their interface in the category they
> +wish for it to start out in.
In my oppinion it should be a requirement that an interface spends at
least one kernel release in the "testing" state before moving to
"stable" and can't be added to "stable" straight away.
Just my 0.02 euro :-)
--
Jesper Juhl <[email protected]>
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please http://www.expita.com/nomime.html
On Monday 27 February 2006 19:57, Greg KH wrote:
> On Mon, Feb 27, 2006 at 07:52:08PM +0000, Alistair John Strachan wrote:
> > On Monday 27 February 2006 19:01, Greg KH wrote:
> > [snip]
> >
> > > +
> > > +Interfaces in the testing state can move to the stable state when the
> > > +developers feel they are finished. They can not be removed from the
> > > +kernel tree without going through the obsolete state first.
> > > +
> > > +It's up to the developer to place their interface in the category they
> > > +wish for it to start out in.
> > > --- /dev/null
> > > +++ gregkh-2.6/Documentation/ABI/obsolete/devfs
> > > @@ -0,0 +1,13 @@
> > > +What: devfs
> > > +Date: July 2005
> > > +Contact: Greg Kroah-Hartman <[email protected]>
> >
> > [snip]
> >
> > July 2005? Either this date is wrong or the document is out of date.
>
> Heh, I wish. Have you looked at
> Documentation/feature-removal-schedule.txt lately?
If you're going by the time (OBSOLETE) was added to the kernel help message,
this predates July 2005. If we're going by "no earlier than when
feature-removal-schedule was introduced", then I guess this is correct.
But even now, devfs is still in the kernel.
Thanks for the answer anyway, I guess this is a non-issue (who will try to use
code that can't be selected via config?).
--
Cheers,
Alistair.
'No sense being pessimistic, it probably wouldn't work anyway.'
Third year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.
On 27 Feb 2006 20:31:28 +0100, Andi Kleen <[email protected]> wrote:
> Greg KH <[email protected]> writes:
>
> > Hi all,
> >
> > As has been noticed recently by a lot of different people, it seems like
> > we are breaking the userspace<->kernelspace interface a lot. Well, in
> > looking back over time, we always have been doing this, but no one seems
> > to notice (proc files changing format and location, netlink library
> > bindings, etc.)
>
> Ok, but how do you plan to address the basic practical problem?
> People cannot freely upgrade/downgrade kernels anymore since udev/hal
> are used widely in distributions.
>
Just a datapoint: My distribution (Slackware) uses udev (but not hal)
and I move between lots of different 2.6.x kernels and even 2.4.x
kernels without problems.
--
Jesper Juhl <[email protected]>
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please http://www.expita.com/nomime.html
On Mon, Feb 27, 2006 at 11:46:23AM -0800, Greg KH wrote:
> Then I suggest you work with the ALSA developers to come up with such a
> "stable" api that never changes. They have been working at this for a
> number of years, if it was a "simple" problem, it would have been done
> already...
That depends on how it's being approached. Writing an ABI takes effort,
while it tends to be easier to simply write new code.
> Anyway, netlink is in the same category, with a backing userspace
> library tie :)
>
> And, I have nothing against shipping userspace libraries with the kernel
> like this, if people think that's the easiest way to do it. But even
> then, the raw interface is still "private" and you need to use the
> library to access it properly.
That's a lot easier if it gets installed with the kernel version as part of
the path. That might need some hacking in the dynamic linker. Before going
that far, it should really be a question of putting the ABI and necessary
extensions under a microscope to see how much stability in an ABI is
possible. Perhaps we've been too lax in reviewing extensions to the kernel's
ABI, resulting in things getting to the point where it now needs to be a
more explicit part of the review process.
Half the problem is that the bits that actually form an ABI tend to be
spread over random .c source files, include/asm and include/linux, so
catching a change is rather difficult even for experienced reviewers. It
might make sense to start splitting out the structure definitions into an
include/abi/ structure to make changes easier to spot. It'll be a lot of
work, but along the lines of the whole ioctl mess the end result will be
an easier system for users to cope with (which is the main concern in
maintaining an ABI -- making needless updates necessary for users and
software authors is something I feel we should try to avoid).
-ben
--
"Ladies and gentlemen, I'm sorry to interrupt, but the police are here
and they've asked us to stop the party." Don't Email: <[email protected]>.
> Then I suggest you work with the ALSA developers to come up with such a
> "stable" api that never changes. They have been working at this for a
> number of years, if it was a "simple" problem, it would have been done
> already...
>
> Anyway, netlink is in the same category, with a backing userspace
> library tie :)
>
> And, I have nothing against shipping userspace libraries with the kernel
> like this, if people think that's the easiest way to do it. But even
> then, the raw interface is still "private" and you need to use the
> library to access it properly.
if it's this volatile, the lib should come with the kernel.
On Mon, Feb 27, 2006 at 08:05:17PM +0000, Alistair John Strachan wrote:
> But even now, devfs is still in the kernel.
>
> Thanks for the answer anyway, I guess this is a non-issue (who will try to use
> code that can't be selected via config?).
Heh, true. Actually, devfs is working in the kernel, if you try to use
it you will notice that it does not work anymore. The fact that no one
has tried to use it and complain about it, makes me _know_ that no one
is using it anymore :)
thanks,
greg k-h
El Mon, 27 Feb 2006 12:00:41 -0800,
Greg KH <[email protected]> escribi?:
> No, look at the different descriptions that I gave them in the README
> please. They are very different. If you think the wording there is not
> precise enough, could you suggest some other wording?
Maybe "Developers should not release stable versions of userspace
applications which depend on "unstable" interfaces, they're only
good for development versions, if you need to use a "unstable"
interface for your program you should wait at least until it hits
"testing" (or even better, "stable")"; or something like that.
On Mon, Feb 27, 2006 at 03:01:07PM -0500, Benjamin LaHaise wrote:
> On Mon, Feb 27, 2006 at 11:46:23AM -0800, Greg KH wrote:
> > Then I suggest you work with the ALSA developers to come up with such a
> > "stable" api that never changes. They have been working at this for a
> > number of years, if it was a "simple" problem, it would have been done
> > already...
>
> That depends on how it's being approached. Writing an ABI takes effort,
> while it tends to be easier to simply write new code.
I agree.
> > Anyway, netlink is in the same category, with a backing userspace
> > library tie :)
> >
> > And, I have nothing against shipping userspace libraries with the kernel
> > like this, if people think that's the easiest way to do it. But even
> > then, the raw interface is still "private" and you need to use the
> > library to access it properly.
>
> That's a lot easier if it gets installed with the kernel version as part of
> the path. That might need some hacking in the dynamic linker. Before going
> that far, it should really be a question of putting the ABI and necessary
> extensions under a microscope to see how much stability in an ABI is
> possible. Perhaps we've been too lax in reviewing extensions to the kernel's
> ABI, resulting in things getting to the point where it now needs to be a
> more explicit part of the review process.
>
> Half the problem is that the bits that actually form an ABI tend to be
> spread over random .c source files, include/asm and include/linux, so
> catching a change is rather difficult even for experienced reviewers. It
> might make sense to start splitting out the structure definitions into an
> include/abi/ structure to make changes easier to spot. It'll be a lot of
> work, but along the lines of the whole ioctl mess the end result will be
> an easier system for users to cope with (which is the main concern in
> maintaining an ABI -- making needless updates necessary for users and
> software authors is something I feel we should try to avoid).
Again, I agree. People (including Linus) have said they will accept
something like include/abi/ (it was a different name last time that I
can't remember), but no one has done the work yet...
thanks,
greg k-h
On Mon, Feb 27, 2006 at 12:12:14PM -0800, Greg KH wrote:
> On Mon, Feb 27, 2006 at 08:05:17PM +0000, Alistair John Strachan wrote:
> > But even now, devfs is still in the kernel.
> >
> > Thanks for the answer anyway, I guess this is a non-issue (who will try to use
> > code that can't be selected via config?).
>
> Heh, true. Actually, devfs is working in the kernel,
Oops, ment to say "is not working"...
On Mon, 27 Feb 2006, Greg KH wrote:
> On Mon, Feb 27, 2006 at 02:36:54PM -0500, Benjamin LaHaise wrote:
> > On Mon, Feb 27, 2006 at 11:01:50AM -0800, Greg KH wrote:
> > > --- /dev/null
> > > +++ gregkh-2.6/Documentation/ABI/private/alsa
> > > @@ -0,0 +1,8 @@
> > > +What: Kernel Sound interface
> > > +Date: Feburary 2006
> > > +Who: Jaroslav Kysela <[email protected]>
> > > +Description:
> > > + The use of the kernel sound interface must be done
> > > + through the ALSA library. For more details on this,
> > > + please see http://www.alsa-project.org/ and contact
> > > + <[email protected]>
> >
> > How can something as widely used as sound not work from one kernel version
> > to the next, as seems to be implied with the "private" nature of the ABI?
> > This is a total cop-out and is IMHO very amateur of the developers. If
> > something like this is to be the case, at the very least the alsa libraries
> > need to provide a stable ABI and be shipped with the kernel.
>
> Then I suggest you work with the ALSA developers to come up with such a
> "stable" api that never changes. They have been working at this for a
> number of years, if it was a "simple" problem, it would have been done
> already...
I really don't much like the "private" and "unstable" subdirectories.
They seem to be just excuses for bad habits. And the notion of a "private"
interface is insane anyway, since it doesn't matter - the only thing that
matters is whether it breaks existing binaries or not, and being "private"
in no way makes any difference to that. If you need to compile or link
against a new library, it's broken - whether it was "private" or not makes
no difference.
The ALSA development model is in my opinion pretty broken (the development
seems to try to be pretty closed-up), but it's (a) gotten better and (b)
the alsa people do not seem to be breaking old binaries and libraries very
much. At least I don't remember seeing all that many problems lately.
So I just don't see any upsides to documenting anything private or
unstable. I see only downsides: it's an excuse to hide behind for
developers.
Linus
On Mon, Feb 27, 2006 at 12:13:23PM -0800, Greg KH wrote:
> Again, I agree. People (including Linus) have said they will accept
> something like include/abi/ (it was a different name last time that I
> can't remember), but no one has done the work yet...
Whether or not something is in include/abi/*.h should be a factor in
classifying it as "stable", "unstable", etc...?
John
--
John W. Linville
[email protected]
On Mon, Feb 27, 2006 at 12:20:49PM -0800, Linus Torvalds wrote:
> They seem to be just excuses for bad habits. And the notion of a "private"
> interface is insane anyway, since it doesn't matter - the only thing that
> matters is whether it breaks existing binaries or not, and being "private"
> in no way makes any difference to that. If you need to compile or link
> against a new library, it's broken - whether it was "private" or not makes
> no difference.
gregkh is being polite - s/private/but-we-are-special/ and it will make
more sense...
On Mon, Feb 27, 2006 at 12:15:18PM -0800, Greg KH wrote:
> On Mon, Feb 27, 2006 at 12:12:14PM -0800, Greg KH wrote:
> > On Mon, Feb 27, 2006 at 08:05:17PM +0000, Alistair John Strachan wrote:
> > > But even now, devfs is still in the kernel.
> > >
> > > Thanks for the answer anyway, I guess this is a non-issue (who will try to use
> > > code that can't be selected via config?).
> >
> > Heh, true. Actually, devfs is working in the kernel,
>
> Oops, ment to say "is not working"...
It works at least enough for the basic devices, the block devices and
ethernet to show up.
OG.
On Mon, Feb 27, 2006 at 09:10:50PM +0100, Arjan van de Ven wrote:
> if it's this volatile, the lib should come with the kernel.
The ALSA lib would never be accepted anywhere near the kernel.
OG.
On Mon, Feb 27, 2006 at 03:22:36PM -0500, John W. Linville wrote:
> On Mon, Feb 27, 2006 at 12:13:23PM -0800, Greg KH wrote:
>
> > Again, I agree. People (including Linus) have said they will accept
> > something like include/abi/ (it was a different name last time that I
> > can't remember), but no one has done the work yet...
>
> Whether or not something is in include/abi/*.h should be a factor in
> classifying it as "stable", "unstable", etc...?
Sure, but lots of our "ABI" is not encoded in header files (think
sysfs), so this will not work for everything.
thanks,
greg k-h
On Mon, 2006-02-27 at 21:04 +0000, Al Viro wrote:
> On Mon, Feb 27, 2006 at 12:20:49PM -0800, Linus Torvalds wrote:
> > They seem to be just excuses for bad habits. And the notion of a "private"
> > interface is insane anyway, since it doesn't matter - the only thing that
> > matters is whether it breaks existing binaries or not, and being "private"
> > in no way makes any difference to that. If you need to compile or link
> > against a new library, it's broken - whether it was "private" or not makes
> > no difference.
>
> gregkh is being polite - s/private/but-we-are-special/ and it will make
> more sense...
I agree with the criticism of the private interfaces, but unstable makes
sense in the new kernel development model world -- "testing" is "we're
pretty sure that this is done but it might not be and we might change it
in the future, so pay attention" while "unstable" is more of a "we think
the design is neat, but somebody is bound to find some way we've screwed
it up and it'll probably be changed in the future to fix it, so expect
your programs to break".
--
Nicholas Miell <[email protected]>
On Mon, Feb 27, 2006 at 12:20:49PM -0800, Linus Torvalds wrote:
>
>
> On Mon, 27 Feb 2006, Greg KH wrote:
>
> > On Mon, Feb 27, 2006 at 02:36:54PM -0500, Benjamin LaHaise wrote:
> > > On Mon, Feb 27, 2006 at 11:01:50AM -0800, Greg KH wrote:
> > > > --- /dev/null
> > > > +++ gregkh-2.6/Documentation/ABI/private/alsa
> > > > @@ -0,0 +1,8 @@
> > > > +What: Kernel Sound interface
> > > > +Date: Feburary 2006
> > > > +Who: Jaroslav Kysela <[email protected]>
> > > > +Description:
> > > > + The use of the kernel sound interface must be done
> > > > + through the ALSA library. For more details on this,
> > > > + please see http://www.alsa-project.org/ and contact
> > > > + <[email protected]>
> > >
> > > How can something as widely used as sound not work from one kernel version
> > > to the next, as seems to be implied with the "private" nature of the ABI?
> > > This is a total cop-out and is IMHO very amateur of the developers. If
> > > something like this is to be the case, at the very least the alsa libraries
> > > need to provide a stable ABI and be shipped with the kernel.
> >
> > Then I suggest you work with the ALSA developers to come up with such a
> > "stable" api that never changes. They have been working at this for a
> > number of years, if it was a "simple" problem, it would have been done
> > already...
>
> I really don't much like the "private" and "unstable" subdirectories.
>
> They seem to be just excuses for bad habits. And the notion of a "private"
> interface is insane anyway, since it doesn't matter - the only thing that
> matters is whether it breaks existing binaries or not, and being "private"
> in no way makes any difference to that. If you need to compile or link
> against a new library, it's broken - whether it was "private" or not makes
> no difference.
Ok, I don't mind the name change from something different than
"private". I was looking for something to call the interface between
the user and kernel that almost all userspace programs should be using
the library instead of the "raw" kernel interface. ALSA and netlink are
two examples of this, and I'm sure there's others.
We will probably have more of these in the future, and if they need to
move their "library" into the kernel tree, that's fine too.
> The ALSA development model is in my opinion pretty broken (the development
> seems to try to be pretty closed-up), but it's (a) gotten better and (b)
> the alsa people do not seem to be breaking old binaries and libraries very
> much. At least I don't remember seeing all that many problems lately.
Yes, ALSA has gotten a lot better. But what about the next project that
comes along that has the same kind of binding and it takes a year or so
to figure out the way that the interface should work as no one has ever
done that kind of interface before? For the majority of things that
people have been complaining about, this has never been done by any
operating system before, the others don't have these problems as they
don't even try :)
> So I just don't see any upsides to documenting anything private or
> unstable. I see only downsides: it's an excuse to hide behind for
> developers.
So should we just not even document anything we consider "unstable"?
The first trys at things are usually really wrong, and that only can be
detected after we've tried it out for a while and have a few serious
users. Should we brand anything new as "testing" if the developer feels
it is ready to go?
thanks,
greg k-h
On Mon, Feb 27, 2006 at 11:56:48PM +0100, Olivier Galibert wrote:
> On Mon, Feb 27, 2006 at 12:15:18PM -0800, Greg KH wrote:
> > On Mon, Feb 27, 2006 at 12:12:14PM -0800, Greg KH wrote:
> > > On Mon, Feb 27, 2006 at 08:05:17PM +0000, Alistair John Strachan wrote:
> > > > But even now, devfs is still in the kernel.
> > > >
> > > > Thanks for the answer anyway, I guess this is a non-issue (who will try to use
> > > > code that can't be selected via config?).
> > >
> > > Heh, true. Actually, devfs is working in the kernel,
> >
> > Oops, ment to say "is not working"...
>
> It works at least enough for the basic devices, the block devices and
> ethernet to show up.
ethernet and networking never had anything to do with devfs. And yes,
block devices do still work today.
thanks,
greg k-h
On Mon, Feb 27, 2006 at 09:13:15PM +0100, Diego Calleja wrote:
> El Mon, 27 Feb 2006 12:00:41 -0800,
> Greg KH <[email protected]> escribi?:
>
> > No, look at the different descriptions that I gave them in the README
> > please. They are very different. If you think the wording there is not
> > precise enough, could you suggest some other wording?
>
>
> Maybe "Developers should not release stable versions of userspace
> applications which depend on "unstable" interfaces, they're only
> good for development versions, if you need to use a "unstable"
> interface for your program you should wait at least until it hits
> "testing" (or even better, "stable")"; or something like that.
So, we will be forcing this scheme onto userspace programmers too? Yeah
right, this isn't Solaris :)
thanks,
greg k-h
[email protected] said:
> Ok, I don't mind the name change from something different than
> "private". I was looking for something to call the interface between
> the user and kernel that almost all userspace programs should be using
> the library instead of the "raw" kernel interface. ALSA and netlink are
> two examples of this, and I'm sure there's others.
well, what about calling it "raw" then?
Jason
On Mon, Feb 27, 2006 at 03:45:25PM -0800, Greg KH wrote:
> > So I just don't see any upsides to documenting anything private or
> > unstable. I see only downsides: it's an excuse to hide behind for
> > developers.
>
> So should we just not even document anything we consider "unstable"?
> The first trys at things are usually really wrong, and that only can be
> detected after we've tried it out for a while and have a few serious
> users. Should we brand anything new as "testing" if the developer feels
> it is ready to go?
How about "we don't let anything into mainline that we consider
'unstable' from an interface point of view"?
There seems to be a fetish going on today that everything possible
should be mindlessly pushed out into userspace regardless of whether
or not it makes sense. What folks don't seem to understand is that
there is a tradeoff between implementational complexity (in terms of
lines of code in /usr/src/linux) and interface complexity (see Rusty's
talk about designing good interfaces and how hard that can be).
If we're not sure we can get the interface right, then maybe it's a
sign it needs to stay in -mm longer, or maybe we were trying to push
the wrong thing out into userspace. If the interface isn't easy to
understand, and we aren't confident that we can promise to never
change it once we put it out there (although of course we can always
add additional interfaces as we add new features), then maybe the
mistake was in trying to create the interface in the first place.
Don't get me wrong; I'm a big fan of pushing policy out of the kernel;
but only if the interface that we use to expose the kernel
functionality has been very carefully designed.
Another alternative, as a few people including myself have noted, is to
shipping that part of the userspace with the kernel sources, so that
it is part of the kernel sources from a release management point of
view, even if it lives in userspace.
- Ted
On Tue, Feb 28, 2006 at 01:32:07AM -0500, Theodore Ts'o wrote:
> Another alternative, as a few people including myself have noted, is to
> shipping that part of the userspace with the kernel sources, so that
> it is part of the kernel sources from a release management point of
> view, even if it lives in userspace.
For system-critical bits of userspace I think this makes a lot
of sense for another reason: Code review. Userspace code doesn't
get anywhere near the scrutiny that kernel code gets.
We still allow the occasional crap into the kernel that probably
could have used another trip around the block before merging, but
the barrier to entry for new code is a lot higher than it used
to be a few years ago.
Moving stuff out to userspace is a 'get out of jail' card for a lot
of authors, but if we're all going to be relying on that code
doing the right thing, it should get an equal amount of code review.
Dave
Greg KH writes:
[...]
> +
> + stable/
> + This directory documents the interfaces that have determined to
> + be stable. Userspace programs are free to use these interfaces
> + with no restrictions, and backward compatibility for them will
> + be guaranteed for at least 2 years. Most simple interfaces
> + (like syscalls) are expected to never change and always be
> + available.
What about separating "stable" ("guaranteed for at least 2 years") and
"standard" (core unix interface is not going to change ever)?
Nikita.
On Mon, Feb 27, 2006 at 09:01:54PM +0100, Jesper Juhl wrote:
> On 2/27/06, Greg KH <[email protected]> wrote:
> > So, any comments? Criticisms?
> >
>
> Great initiative. Thanks.
Glad you like it.
> > + stable/
> > + This directory documents the interfaces that have determined to
> > + be stable. Userspace programs are free to use these interfaces
>
> "have determined to be stable." ??
> How about "have proven to be stable" or "we have determined to be
> stable" or "we have defined as being stable and will not break"
> instead?
Ok, I'll change this, it is ackward wording.
> > + testing/
> > + This directory documents interfaces that are felt to be stable,
> > + as the main development of this interface has been completed.
> > + The interface can be changed to add new features, but the
> > + current interface will not break by doing this.
> > + Userspace programs can start to rely on these interfaces, but
> > + they must be aware of changes that can occur before these
> > + interfaces move to be marked stable. Programs that use these
> > + interfaces are strongly encouraged to add their name to the
> > + description of these interfaces, so that the kernel developers
> > + can easily notify them if any changes occur (see the description
> > + of the layout of the files below for details on how to do this.)
> > +
>
> Maybe a note here that "testing" interfaces may be change in
> incompatible ways before moving to stable/ if grave errors or security
> vulnerabilities are found in them?
Good point, added.
> > + unstable/
> > + This directory documents interfaces that are known to be
> > + unstable, and not ready for widespread use by a lot of different
> > + programs. That is not to say that they can not be used, but
> > + developers of such programs should track their changes very
> > + closely. Again, programs that uses these interfaces are
> > + strongly encouraged to add their names to the description of the
> > + interfaces so that they can be notified of changes.
> > +
> > + obsolete/
> > + This directory documents interfaces that are still remaining in
> > + the kernel, but are marked to be removed at some later point in
> > + time. The description of the interface will document the reason
> > + why it is obsolete and when it can be expected to be removed.
> > +
>
> A note here that people should check
> Documentation/feature-removal-schedule.txt perhaps?
Yeah, I wonder if some of the things in that file can be moved to be
individual files in this directory too.
> > + private/
> > + This interface is private between the kernel and a helper
> > + userspace program or library. If you wish to use this interface
> > + (like alsa or netlink) userspace must use the helper library and
> > + not use the raw kernel interface directly.
> > +
>
> Perhaps add something along the lines of "Stability of the interface
> is only guaranteed through the helper library, if you choose to
> disregard this and use the raw kernel interface anyway, then your
> program may break without warning in future kernel releases.".
I'd like to just discourage their use without the helper at all, as that
is what the developers of that interface is saying already. I guess you
could want to implement your own alsa kernel interface if you really
want to :)
Anyway, I've added more wording here for that, thanks.
> > +Every file in these directories will contain the following information:
> > +
> > +What: Short description of the interface
> > +Created: Date created
>
> Perhaps also a line here specifying the first mainline kernel release
> (not -mm or -rc, but first stable kernel) that the interface is
> available with. ??
Good idea, added.
> > +It's up to the developer to place their interface in the category they
> > +wish for it to start out in.
>
> In my oppinion it should be a requirement that an interface spends at
> least one kernel release in the "testing" state before moving to
> "stable" and can't be added to "stable" straight away.
Hm, good idea, syscalls should even do this...
thanks again for your input, I'll repost an updated version based on
these comments and the others.
greg k-h
On Tue, Feb 28, 2006 at 02:39:29PM +0300, Nikita Danilov wrote:
> Greg KH writes:
>
> [...]
>
> > +
> > + stable/
> > + This directory documents the interfaces that have determined to
> > + be stable. Userspace programs are free to use these interfaces
> > + with no restrictions, and backward compatibility for them will
> > + be guaranteed for at least 2 years. Most simple interfaces
> > + (like syscalls) are expected to never change and always be
> > + available.
>
> What about separating "stable" ("guaranteed for at least 2 years") and
> "standard" (core unix interface is not going to change ever)?
Why? Would that mean that the POSIX-like syscalls would only be in
"standard"? What else would you think would be in that category?
thanks,
greg k-h
On Tue, Feb 28, 2006 at 01:32:07AM -0500, Theodore Ts'o wrote:
> On Mon, Feb 27, 2006 at 03:45:25PM -0800, Greg KH wrote:
> > > So I just don't see any upsides to documenting anything private or
> > > unstable. I see only downsides: it's an excuse to hide behind for
> > > developers.
> >
> > So should we just not even document anything we consider "unstable"?
> > The first trys at things are usually really wrong, and that only can be
> > detected after we've tried it out for a while and have a few serious
> > users. Should we brand anything new as "testing" if the developer feels
> > it is ready to go?
>
> How about "we don't let anything into mainline that we consider
> 'unstable' from an interface point of view"?
In a perfect world, where we are all kick-ass programmers and never get
anything wrong and can always anticipate exactly how people will use the
interfaces we create, sure we could say this.
But until then, there's no way this can happen :)
For example, look at all of the gyrations that the sys_futex call went
through. It took people really using the thing before the final version
of how it would work could be added.
And another example, /proc. How many times over the past 15 years have
we had to upgrade the procps package to handle the addition or change of
one thing or another? We evolve over time to handle the issues that
come up with different architectures and needs. That's what makes Linux
so great.
Now remember, this is only a very small subset of programs that fall
into this "changing interface" category. For 99% of the programs that
run out there, they only use the standard POSIX syscall api, and they
never change (Binaries built for Caldera 1.0 still run on 2.6 this day
as proof of that.)
But everyone here on this list feels the issues as we are doing the
development of this system. It is a real system, consisting of parts
that are outside of the kernel in order for everything to work properly.
And if we want to pull some of those pieces into the kernel tree proper,
in order to handle the change and churn better, that's fine with me too,
but is outside the scope of this classification document.
> There seems to be a fetish going on today that everything possible
> should be mindlessly pushed out into userspace regardless of whether
> or not it makes sense.
It's not a "new" fetish at all, look at the huge number of ioctls, proc
files, sysctls, mmap interfaces to devices, etc. It's just that the
list of these interfaces has never been fully documented or classified
(find any DRI/DRM interface docs in the kernel tree lately?)
Hopefully this proposal will help address the need for figuring out what
is changing in the kernel, and help userspace programmers to know what
they can and can not count on remaining stable in the future, both near
and far.
> If we're not sure we can get the interface right, then maybe it's a
> sign it needs to stay in -mm longer, or maybe we were trying to push
> the wrong thing out into userspace. If the interface isn't easy to
> understand, and we aren't confident that we can promise to never
> change it once we put it out there (although of course we can always
> add additional interfaces as we add new features), then maybe the
> mistake was in trying to create the interface in the first place.
The "stay in -mm" statement is nice, but unfortunately will not really
help. Look at how many people really run and test -mm. What is it, in
the low 100s at the most? And we have how many, 900+ individually
documented kernel contributors over the lifetime of the kernel? Even if
all of them run -mm, we would not have the spread or reach that we
really need to be able to classify interfaces as "stable" all the time.
All of those developers are working on their individual pieces and
rarely look up and see what else is going on around in the kernel.
thanks,
greg k-h
On Tue, 2006-02-28 at 16:34 -0800, Greg KH wrote:
> On Tue, Feb 28, 2006 at 01:32:07AM -0500, Theodore Ts'o wrote:
> > On Mon, Feb 27, 2006 at 03:45:25PM -0800, Greg KH wrote:
> > > > So I just don't see any upsides to documenting anything private or
> > > > unstable. I see only downsides: it's an excuse to hide behind for
> > > > developers.
> > >
> > > So should we just not even document anything we consider "unstable"?
> > > The first trys at things are usually really wrong, and that only can be
> > > detected after we've tried it out for a while and have a few serious
> > > users. Should we brand anything new as "testing" if the developer feels
> > > it is ready to go?
> >
> > How about "we don't let anything into mainline that we consider
> > 'unstable' from an interface point of view"?
>
> In a perfect world, where we are all kick-ass programmers and never get
> anything wrong and can always anticipate exactly how people will use the
> interfaces we create, sure we could say this.
>
> But until then, there's no way this can happen :)
>
> For example, look at all of the gyrations that the sys_futex call went
> through. It took people really using the thing before the final version
> of how it would work could be added.
>
> And another example, /proc. How many times over the past 15 years have
> we had to upgrade the procps package to handle the addition or change of
> one thing or another? We evolve over time to handle the issues that
> come up with different architectures and needs. That's what makes Linux
> so great.
This is a really bad example.
All the /proc related contortions are a direct result of the fact that
the multitudes of /proc "formats" are completely undocumented,
non-extensible, and largely unintended for programmatic usage[1]. (/sys
was supposed to solve some of these things, but it seems to be going the
same route, unfortunately.)
Honestly, despite what the ASCII fetish crowd[2] may say, Solaris got it
right by just exporting C structs. The parsing is certainly a hell of a
lot easier when you're dealing with actual C datatypes instead of
character strings and people hacking on /proc are probably less likely
to make ABI breaking changes when they're dealing with a struct instead
of a sprintf statement.
[1] Where's my EBNF?
[2] "But ASCII is easily manipulated by shell tools!!!!"
Well, then write a very small C program that spits out ASCII and stick
it in procps.
--
Nicholas Miell <[email protected]>
On Tue, 2006-02-28 at 16:23 -0800, Greg KH wrote:
> On Tue, Feb 28, 2006 at 02:39:29PM +0300, Nikita Danilov wrote:
> > Greg KH writes:
> >
> > [...]
> >
> > > +
> > > + stable/
> > > + This directory documents the interfaces that have determined to
> > > + be stable. Userspace programs are free to use these interfaces
> > > + with no restrictions, and backward compatibility for them will
> > > + be guaranteed for at least 2 years. Most simple interfaces
> > > + (like syscalls) are expected to never change and always be
> > > + available.
> >
> > What about separating "stable" ("guaranteed for at least 2 years") and
> > "standard" (core unix interface is not going to change ever)?
>
> Why? Would that mean that the POSIX-like syscalls would only be in
> "standard"? What else would you think would be in that category?
that sounds wrong. If you want posix behavior, use glibc. Not the kernel
directly. It's that simple. The kernel tends to follow posix mostly, to
allow glibc to do this job without too much hoops, but it's glibc that
provides the final posix API to the application. And it should be that
way.
On 2006-02-27T11:44:00, Greg KH <[email protected]> wrote:
> > Ok, but how do you plan to address the basic practical problem?
> > People cannot freely upgrade/downgrade kernels anymore since udev/hal
> > are used widely in distributions.
> I can freely upgrade/downgrade kernels on some distros[1] if I wish to,
> as they support such things. Just complain to your distro maker if you
> have this issue :)
This is a somewhat cheap excuse.
The fact is that now we have user-space and kernel space tied together
much more intimately than ever; udev & sysfs being the prime examples
these days, and then it's not that some figure in top is wrong, but
"oops my network no longer loads and the box is 400 miles away".
I don't have any issue with that per se, and the kernel can continue to
change as much as it wants, but for the users' sake, the user-land needs
to be at least _backwards compatible_ for a number of kernel releases.
ie, you can upgrade the kernel, and this forces you to upgrade the
user-space - annoying, but alright -, but at least said user-space will
support the older kernel(s) in case one has to go back (or accidentially
doesn't boot into the new kernel at all ;-).
So that a user could always be sure that if they are running udev-latest
(for example; a number of other projects have the same issue), they can
run almost any kernel up to that point.
This isn't a policy we can dictate to user-space, but an expectation it
should measure up to, at least for such critical components as required
for the system to get up to the point of being able to log in
(serial/console/ssh) again and fix anything.
Interfaces in the boot path are more critical than sound or even gfx
starting up or not, which is merely annoying ;-)
That said, the proposal is great, as are Ted's comments. We definetely
need to document which parts of the evolving interface are supposed to
be stable, otherwise we're confusing the users mightily and causing much
havoc. And users can also use this as evidence against us ("you said
the interface was _stable_, damn it!" and we'd have to at least go
"Oops, sorry, we had to." instead of "neener neener!" ;-) Please go
ahead with this!
Sincerely,
Lars Marowsky-Br?e
--
High Availability & Clustering
SUSE Labs, Research and Development
SUSE LINUX Products GmbH - A Novell Business -- Charles Darwin
"Ignorance more frequently begets confidence than does knowledge"
On Wed, Mar 01, 2006 at 02:53:56PM +0100, Lars Marowsky-Bree wrote:
> The fact is that now we have user-space and kernel space tied together
> much more intimately than ever; udev & sysfs being the prime examples
> these days, and then it's not that some figure in top is wrong, but
> "oops my network no longer loads and the box is 400 miles away".
IMHO this is not a good example as there is really no reason to install
udev on such a box at all. Remember: KISS. Having a static /dev and
/etc/modules filled in (or even better, a monolithic kernel) is far more
reliable to administer.
On a desktop machine when you are plugging in various USB/Firewire/etc.
devices all the time udev works great. On a remote server there is no
real need for udev.
Gabor
--
---------------------------------------------------------
MTA SZTAKI Computer and Automation Research Institute
Hungarian Academy of Sciences
---------------------------------------------------------
>>>>> "Gabor" == Gabor Gombas <[email protected]> writes:
Gabor> On Wed, Mar 01, 2006 at 02:53:56PM +0100, Lars Marowsky-Bree
Gabor> wrote:
>> The fact is that now we have user-space and kernel space tied
>> together much more intimately than ever; udev & sysfs being the
>> prime examples these days, and then it's not that some figure in
>> top is wrong, but "oops my network no longer loads and the box is
>> 400 miles away".
Gabor> IMHO this is not a good example as there is really no reason to
Gabor> install udev on such a box at all. Remember: KISS. Having a
Gabor> static /dev and /etc/modules filled in (or even better, a
Gabor> monolithic kernel) is far more reliable to administer.
Gabor> On a desktop machine when you are plugging in various
Gabor> USB/Firewire/etc. devices all the time udev works great. On a
Gabor> remote server there is no real need for udev.
Problem is that a lot of sites rely on 'enterprise' distros (SLES,
RHEL, etc.), so if those use udev, the users will end up with it as
well.
Cheers,
Jes
On 2006-03-01T15:10:31, Gabor Gombas <[email protected]> wrote:
> IMHO this is not a good example as there is really no reason to install
> udev on such a box at all. Remember: KISS. Having a static /dev and
> /etc/modules filled in (or even better, a monolithic kernel) is far more
> reliable to administer.
>
> On a desktop machine when you are plugging in various USB/Firewire/etc.
> devices all the time udev works great. On a remote server there is no
> real need for udev.
You can't get away that easily ;-)
First, even the enterprise distributions nowadays all use hotplug/udev
etc - for reducing the maintenance complexity, and also enterprise
systems do see a fair bit of hotplugging of new network cards, PCI
adapters, drives, or even USB attachments.
Second, sometimes my desktop is the machine I'm logging into from
remote while on the road.
The distinction isn't as clear as you think it is.
Sincerely,
Lars Marowsky-Br?e
--
High Availability & Clustering
SUSE Labs, Research and Development
SUSE LINUX Products GmbH - A Novell Business -- Charles Darwin
"Ignorance more frequently begets confidence than does knowledge"
On Wed, Mar 01, 2006 at 08:27:52AM +0100, Arjan van de Ven wrote:
> On Tue, 2006-02-28 at 16:23 -0800, Greg KH wrote:
> > On Tue, Feb 28, 2006 at 02:39:29PM +0300, Nikita Danilov wrote:
> > > Greg KH writes:
> > >
> > > [...]
> > >
> > > > +
> > > > + stable/
> > > > + This directory documents the interfaces that have determined to
> > > > + be stable. Userspace programs are free to use these interfaces
> > > > + with no restrictions, and backward compatibility for them will
> > > > + be guaranteed for at least 2 years. Most simple interfaces
> > > > + (like syscalls) are expected to never change and always be
> > > > + available.
> > >
> > > What about separating "stable" ("guaranteed for at least 2 years") and
> > > "standard" (core unix interface is not going to change ever)?
> >
> > Why? Would that mean that the POSIX-like syscalls would only be in
> > "standard"? What else would you think would be in that category?
>
> that sounds wrong. If you want posix behavior, use glibc. Not the kernel
> directly. It's that simple. The kernel tends to follow posix mostly, to
> allow glibc to do this job without too much hoops, but it's glibc that
> provides the final posix API to the application. And it should be that
> way.
Yes, sorry, that is much more correct.
greg k-h
On Tue, Feb 28, 2006 at 05:17:49PM -0800, Nicholas Miell wrote:
> On Tue, 2006-02-28 at 16:34 -0800, Greg KH wrote:
> > On Tue, Feb 28, 2006 at 01:32:07AM -0500, Theodore Ts'o wrote:
> > > On Mon, Feb 27, 2006 at 03:45:25PM -0800, Greg KH wrote:
> > > > > So I just don't see any upsides to documenting anything private or
> > > > > unstable. I see only downsides: it's an excuse to hide behind for
> > > > > developers.
> > > >
> > > > So should we just not even document anything we consider "unstable"?
> > > > The first trys at things are usually really wrong, and that only can be
> > > > detected after we've tried it out for a while and have a few serious
> > > > users. Should we brand anything new as "testing" if the developer feels
> > > > it is ready to go?
> > >
> > > How about "we don't let anything into mainline that we consider
> > > 'unstable' from an interface point of view"?
> >
> > In a perfect world, where we are all kick-ass programmers and never get
> > anything wrong and can always anticipate exactly how people will use the
> > interfaces we create, sure we could say this.
> >
> > But until then, there's no way this can happen :)
> >
> > For example, look at all of the gyrations that the sys_futex call went
> > through. It took people really using the thing before the final version
> > of how it would work could be added.
> >
> > And another example, /proc. How many times over the past 15 years have
> > we had to upgrade the procps package to handle the addition or change of
> > one thing or another? We evolve over time to handle the issues that
> > come up with different architectures and needs. That's what makes Linux
> > so great.
>
> This is a really bad example.
>
> All the /proc related contortions are a direct result of the fact that
> the multitudes of /proc "formats" are completely undocumented,
> non-extensible, and largely unintended for programmatic usage[1]. (/sys
> was supposed to solve some of these things, but it seems to be going the
> same route, unfortunately.)
sysfs is not going that same route at all. Sure there are a small
majority of files that are multi-line, but they are in the minority by
far.
> Honestly, despite what the ASCII fetish crowd[2] may say, Solaris got it
> right by just exporting C structs. The parsing is certainly a hell of a
> lot easier when you're dealing with actual C datatypes instead of
> character strings and people hacking on /proc are probably less likely
> to make ABI breaking changes when they're dealing with a struct instead
> of a sprintf statement.
Even Solaris documents the maturity level of its interfaces, that is all
I am trying to do here. I'm not trying to pass judgement on the quality
of any of these interfaces.
thanks,
greg k-h
Greg KH <[email protected]> writes:
> Even Solaris documents the maturity level of its interfaces, that is all
> I am trying to do here. I'm not trying to pass judgement on the quality
> of any of these interfaces.
So if we go down this path can we make this functional Documentation?
In particular can we have per process/per interface kinds of flags that
allow access to experimental subsystems?
If the developer has to jump through an extra hoop to use an interface
we have clearly documented this is unsupported and will change in the
future. Anything else can be easy to miss.
Eric
On Sun, Mar 05, 2006 at 09:17:59AM -0700, Eric W. Biederman wrote:
> So if we go down this path can we make this functional Documentation?
>
> In particular can we have per process/per interface kinds of flags that
> allow access to experimental subsystems?
Sounds fragile. It doesn't help the real problem that APIs need to be well
designed.
> If the developer has to jump through an extra hoop to use an interface
> we have clearly documented this is unsupported and will change in the
> future. Anything else can be easy to miss.
At this point if it ends up anywhere near the tools people need to use on
a regular basis, it will not have accomplished anything. APIs should be
reasonably stable, often extensible, by the time they hit the kernel. The
problem with any sort of 'experimental' API is that it probably means the
code is not ready to be in the kernel. Wrapping it up in flags doesn't
stop people from using it and tieing code to the interface; not shipping
code that isn't fully baked does.
-ben
--
"Time is of no importance, Mr. President, only life is important."
Don't Email: <[email protected]>.
Benjamin LaHaise <[email protected]> writes:
> On Sun, Mar 05, 2006 at 09:17:59AM -0700, Eric W. Biederman wrote:
>> So if we go down this path can we make this functional Documentation?
>>
>> In particular can we have per process/per interface kinds of flags that
>> allow access to experimental subsystems?
>
> Sounds fragile. It doesn't help the real problem that APIs need to be well
> designed.
Yes. But it does sure stick out like a sore thumb in a patch or when
you try to use it. Especially if you do something evil like require
user space to pass in a hash of the kernel code implementing the
interface. So even the smallest changes of the implementation break
user space.
>> If the developer has to jump through an extra hoop to use an interface
>> we have clearly documented this is unsupported and will change in the
>> future. Anything else can be easy to miss.
>
> At this point if it ends up anywhere near the tools people need to use on
> a regular basis, it will not have accomplished anything. APIs should be
> reasonably stable, often extensible, by the time they hit the kernel. The
> problem with any sort of 'experimental' API is that it probably means the
> code is not ready to be in the kernel. Wrapping it up in flags doesn't
> stop people from using it and tieing code to the interface; not shipping
> code that isn't fully baked does.
I don't know all of the answers. But if we are going to document something
is half backed, let have the code behave like the interface is half backed
and at least try to keep it out of the hands of most applications.
To a large extent I agree that we should have fully backed interfaces in
the stable kernel. Is that always possible? How often do things not show
up until they are being used in the real world? Is it possible to
find those things out in experimental branches?
Eric
On Sun, Mar 05, 2006 at 05:12:48PM -0700, Eric W. Biederman wrote:
> Yes. But it does sure stick out like a sore thumb in a patch or when
> you try to use it. Especially if you do something evil like require
> user space to pass in a hash of the kernel code implementing the
> interface. So even the smallest changes of the implementation break
> user space.
I thought the decree / consensus was that You Can't Do That. Yes, some
people want to, but that doesn't mean we should let them.
> I don't know all of the answers. But if we are going to document something
> is half backed, let have the code behave like the interface is half backed
> and at least try to keep it out of the hands of most applications.
Why should it be merged into base then? If it's in the mainstream kernel,
it needs to be reasonably solid (which has always been a precondition for
merging patches, aiui).
> To a large extent I agree that we should have fully backed interfaces in
> the stable kernel. Is that always possible? How often do things not show
> up until they are being used in the real world? Is it possible to
> find those things out in experimental branches?
There's a big difference between those interfaces that have had a decent
amount of thought put into them and those which have not. Take ext2,
which is a good example as it can still mount a filesystem made back in
the early '90s today, yet it has undergone a huge amount of change. What
was the secret? The superblock has a few flags for compatible and
incompatible features.
API design is not rocket science, it just requires effort. So long as we
keep beating the drums about how important this is, people will learn and
we will get better at catching these issues during review.
-ben
--
"Time is of no importance, Mr. President, only life is important."
Don't Email: <[email protected]>.
Benjamin LaHaise <[email protected]> writes:
> On Sun, Mar 05, 2006 at 05:12:48PM -0700, Eric W. Biederman wrote:
>> Yes. But it does sure stick out like a sore thumb in a patch or when
>> you try to use it. Especially if you do something evil like require
>> user space to pass in a hash of the kernel code implementing the
>> interface. So even the smallest changes of the implementation break
>> user space.
>
> I thought the decree / consensus was that You Can't Do That. Yes, some
> people want to, but that doesn't mean we should let them.
That is what I heard as well. However look at the topic of this
thread. Documenting various level of stability is the same as saying
something isn't stable. Which is the same as saying something is
going to break. So if we are going to break things on purpose we
should break them all of the time so dependencies don't build up. So
that it is almost impossible to use unstable interfaces.
>> I don't know all of the answers. But if we are going to document something
>> is half backed, let have the code behave like the interface is half backed
>> and at least try to keep it out of the hands of most applications.
>
> Why should it be merged into base then? If it's in the mainstream kernel,
> it needs to be reasonably solid (which has always been a precondition for
> merging patches, aiui).
I don't have a clue.
>> To a large extent I agree that we should have fully backed interfaces in
>> the stable kernel. Is that always possible? How often do things not show
>> up until they are being used in the real world? Is it possible to
>> find those things out in experimental branches?
>
> There's a big difference between those interfaces that have had a decent
> amount of thought put into them and those which have not. Take ext2,
> which is a good example as it can still mount a filesystem made back in
> the early '90s today, yet it has undergone a huge amount of change. What
> was the secret? The superblock has a few flags for compatible and
> incompatible features.
>
> API design is not rocket science, it just requires effort. So long as we
> keep beating the drums about how important this is, people will learn and
> we will get better at catching these issues during review.
Agreed.
However history does show that most people get API design wrong, at
least the first time.
If we were good at getting it right sys_ioctl would not be a problem,
and quite possibly would never have come into being.
So the question is can we get good enough at review that we can live
with the few mistakes that make it through?
Eric
On Sun, Mar 05, 2006 at 07:15:03PM -0700, Eric W. Biederman wrote:
>
> So the question is can we get good enough at review that we can live
> with the few mistakes that make it through?
Exactly.
Combined with the fact that we do not have many reviewers (it's a
thankless, grumpy job), makes it very hard to make it "good enough"...
thanks,
greg k-h
> > > API design is not rocket science, it just requires effort.
> >
> > Agreed.
> >
> > However history does show that most people get API design wrong, at
> > least the first time.
> >
> > So the question is can we get good enough at review that we can live
> > with the few mistakes that make it through?
>
> Exactly.
>
> Combined with the fact that we do not have many reviewers (it's a
> thankless, grumpy job), makes it very hard to make it "good enough"...
There is a secret in creating a successful API; it's called ABSTRACTION.
ABSTRACTION is a piece of cake, if supported by an intricate knowledge of the
subject involved.
--
Al
On 3/7/06, Al Boldi <[email protected]> wrote:
> > > > API design is not rocket science, it just requires effort.
> > >
> > > Agreed.
> > >
> > > However history does show that most people get API design wrong, at
> > > least the first time.
> > >
> > > So the question is can we get good enough at review that we can live
> > > with the few mistakes that make it through?
> >
> > Exactly.
> >
> > Combined with the fact that we do not have many reviewers (it's a
> > thankless, grumpy job), makes it very hard to make it "good enough"...
>
> There is a secret in creating a successful API; it's called ABSTRACTION.
This is about ABI not API. There is no API in the kernel. There _is_ and ABI.
josh