LinuxLists.cc - [ANNOUNCE] linux-libc-headers dead

[permalink] [raw]

Subject: Re: State of userland headers

On Thursday 23 March 2006 12:11 pm, Mariusz Mazur wrote:

> This means that initially the llh-ng project would need to
> start as a completely separate entity that would not require the
> original kernel headers for anything, and only later, after achieving
> some level of maturity and getting merged into the kernel, would come
> the time for removing some duplication. Of course that means that kernel
> hackers would (a) need to keep in mind that they have two places to
> update, (b) once llh-ng got merged, figure out where duplication can be
> avoided and do something about it and (c) even when that'd get done,
> still remember about double updates for places where duplication
> couldn't have been avoided.

I mentioned earlier that klibc seems like a good test case. Make userspace
headers that klibc can build against, and work up from there...

> I won't be loosing any sleep about it, though. Those mean kernel hackers
> never cared about us poor userland folks, so there's no reason we should
> feel sorry for them. Payback time.
>
> And now for some technical details on how I see it. I'd appreciate any
> input.
>
> First thing, is that we'd probably need to use a distributed rcs, since
> it's more flexible, and, well, distributed. Never used any, so git sounds
> as good an option, as anything else.
>
> Now, what I propose we do initially, is get a bunch of headers from current
> llh for the most popular archs and figure out how to build all the various
> *libcs against them. And of course update them to the newest kernel by the
> way.
>
> It'd probably be best if I did it. PLD has support for x86, x86_64, sparc,
> ppc and alpha, so that's what I'd take care of first. It might sound like a
> good idea to start with all available archs, but in reality we wouldn't
> even be able to test them most of the times, so it's best to stick with the
> archs we're directly interested in.

Here at timesys I've got access to arm, mips, and ppc. (Can't say I really
know what I'm doing on any of them, but I'm learning.)

> just because we're waiting for one of the more exotic archs. How does
> one version such a project? Not like the old llh (x.y.z.t, where x.y.z
> corresponds to the kernel version), since not all releases would be
> always fully up to date with a given kernel. So maybe just automatic
> snapshots every day (assuming a change has been made). But if so, who's
> tree gets to be treated as the main one? (Distributed rcs, remember).
> And should we choose such a person, what is the chance he'll end up just
> like I did (a lazy schmuck that is)?
>
> Ok, I'm getting too detailed here. One thing at a time.
>
> So, waiting for any insights, counterproposals and/or declarations who's
> willing to work on old llh, llh-ng or both. Shoot away.

I'm a little confused about the goals of the project.

In order to take advantage of new kernel features, like major/minor numbers
greater than 8 bits or DMA for cd burning, we need new kernel headers. That
sort of thing is why we didn't just build against 2.4 kernel headers, which
for some reason seemed to be usable from userspace just fine.

You also don't want to run a libc built with newer headers than the kernel
you're running on, or it'll try to use stuff that isn't there.

You're saying that the new kernel headers wouldn't be versioned using the
kernel's release numbers. How do we know what kernel version their feature
set matches then? (I'm confused. This happens easily...)

Rob
--
Never bet against the cheap plastic solution.

2006-03-24 18:52:17

[permalink] [raw]

Subject: Re: State of userland headers

On Mar 23, 2006, at 12:11:26, Mariusz Mazur wrote:
> There was a thread on lkml on this topic about a year ago. IIRC
> I've suggested, that the best option would be to get a new set of
> dirs somewhere inside the kernel, and gradually export the userland
> usable stuff from the kernel headers, so to (a) achieve full
> separation and (b) avoid duplication of definitions (meaning that
> kernel headers would simply include the userland ones where
> required). Linus said, that it would break stuff and so is
> unacceptable.

I seem to remember Linus saying that "breaking things is
unacceptable", not that the project was guaranteed to break things
(we would just need to be much more careful about it than most kernel
patches). What that seems to indicate to me is that an in-kernel
version would need to do the following for userspace-accessible
header files for a large number of kernel releases:

#ifndef _LINUX_HEADER_H
#define _LINUX_HEADER_H
#include <kabi/header.h>
/* Define or typedef a bunch of __kabi_ prefixes to the old
prefixes they used to have in the kernel header */
#ifndef __KERNEL__
# warning "The header file <linux/header.h> is deprecated for"
# warning "userspace, please use <kabi/header.h> instead."
#else
/* Kernel-only declarations/definitions */
#endif

If this were done carefully, all programs that compile against kernel
headers could be _fixed_ in the short term (they'd go from throwing
errors to giving a couple deprecation warnings). In the long term,
the extra ifdeffery could be removed and the <linux/*.h> headers for
which a <kabi/*.h> replacement had existed for a couple versions
could be removed. New ABIs (including IOCTLs, new syscalls, etc)
could be required to use <kabi/*.h> in the first place.

> Unfortunately I must agree with him -- I don't think it is possible
> to completely avoid duplication of definitions and all tries would
> lead to breakage of some obscure configurations -- kernel headers
> sometimes require various magic that should be avoided inside the
> userland headers at all cost. This means that initially the llh-ng
> project would need to start as a completely separate entity that
> would not require the original kernel headers for anything, and
> only later, after achieving some level of maturity and getting
> merged into the kernel, would come the time for removing some
> duplication.

I think that requiring any kind of duplication of effort on that
large a scale is virtually guaranteed not to work. It will break
down and be really painful for a long time.

> And here's where the first problem arises -- llh were so great
> initially, because they removed a lot of conflicts with glibc, by
> simply removing 'offending' linux headers and including the glibc
> counterparts (eg. linux/socket.h would do nothing else, than
> include sys/socket.h). Glibc's known for having lots of stuff
> simply 'hardcoded' into it's own set of headers, and more often
> than not, people do need to include headers from both places.

1: Ewww, bad glibc!
2: The symbols in kabi/*.h should probably all start with __kabi_

The kernel _internally_ would "#include <kabi/foo.h>" and then
promptly redefine or typedef all of those objects. If the kernel or
GLIBC want a struct renamed, they should #define
__kabi_name_of_struct name_of_struct just before including the header
file. We should also kind of frown upon non-libc userspace including
many <kabi/*.h> files, since if they get included before libc can
redefine the names to what it wants.

> I don't know about uclibc, but klibc afaik expects a lot more of
> the linux headers to be present, than glibc does.

Perhaps we should use klibc as our test-case for functional headers,
then, instead of glibc.

> Hell, if llh-ng is supposed to be a full set of apis, we can't
> expect any of the headers to come from other places, so if any
> other app (libcs included) has duplicates, that's too bad for the
> app, since it's in need of some patching.

The solution to that is to make sure that the new exported kernel
ABIs always have the __kabi_ prefix, so that we don't smash the
namespace of various programs.

Cheers,
Kyle Moffett

2006-03-24 21:24:37

[permalink] [raw]

Subject: Re: State of userland headers

On Friday 24 March 2006 1:51 pm, Kyle Moffett wrote:
> On Mar 23, 2006, at 12:11:26, Mariusz Mazur wrote:
> > There was a thread on lkml on this topic about a year ago. IIRC
> > I've suggested, that the best option would be to get a new set of
> > dirs somewhere inside the kernel, and gradually export the userland
> > usable stuff from the kernel headers, so to (a) achieve full
> > separation and (b) avoid duplication of definitions (meaning that
> > kernel headers would simply include the userland ones where
> > required). Linus said, that it would break stuff and so is
> > unacceptable.
>
> I seem to remember Linus saying that "breaking things is
> unacceptable", not that the project was guaranteed to break things
> (we would just need to be much more careful about it than most kernel
> patches).

The gentoo guys clean up their own headers, apparently. I'm told they do so
by moving around the #ifdef __KERNEL__ stuff to be in the correct places, and
that they're currently working on a 2.6.14 or 2.6.15 version of the headers:

http://www.gentoo.org/cgi-bin/viewcvs.cgi/src/patchsets/gentoo-headers/?root=gentoo

> What that seems to indicate to me is that an in-kernel
> version would need to do the following for userspace-accessible
> header files for a large number of kernel releases:
>
> #ifndef _LINUX_HEADER_H
> #define _LINUX_HEADER_H
> #include <kabi/header.h>
> /* Define or typedef a bunch of __kabi_ prefixes to the old
> prefixes they used to have in the kernel header */
> #ifndef __KERNEL__
> # warning "The header file <linux/header.h> is deprecated for"
> # warning "userspace, please use <kabi/header.h> instead."
> #else
> /* Kernel-only declarations/definitions */
> #endif

Changing the #include paths in all deployed software will basically never
happen. If this header package requires that, I'm not interested in it
because I can't build existing software against it, and I don't expect anyone
else to be.

I was thinking of possibly a parallel header set under linux-2.6.x/usr/include
which the linux-2.6.x/include/*.h could #include to clean out their #ifndef
__KERNEL__ stuff, and that eventually the usr/include stuff would contain
approximately what Mazur's headers had contained. Unfortunately, I'm under
the impression that's not a realistic approach.

> If this were done carefully, all programs that compile against kernel
> headers could be _fixed_ in the short term (they'd go from throwing
> errors to giving a couple deprecation warnings). In the long term,
> the extra ifdeffery could be removed and the <linux/*.h> headers for
> which a <kabi/*.h> replacement had existed for a couple versions
> could be removed. New ABIs (including IOCTLs, new syscalls, etc)
> could be required to use <kabi/*.h> in the first place.

A program that includes kabi/* instead of linux/* won't build against older C
libraries with older headers from older kernel versions (or older project's
like Mazur's headers).

> 1: Ewww, bad glibc!
> 2: The symbols in kabi/*.h should probably all start with __kabi_

Any grand new incompatible thing is something I will happily ignore for as
long as I am able to, and I'm not alone here. Your uptake will be zero.

Rob
--
Never bet against the cheap plastic solution.

2006-03-24 22:46:54

[permalink] [raw]

Subject: Re: State of userland headers

On Mar 24, 2006, at 16:48:47, Nix wrote:
> On 24 Mar 2006, Rob Landley suggested tentatively:
>> On Friday 24 March 2006 1:51 pm, Kyle Moffett wrote:
>>> 1: Ewww, bad glibc!
>>> 2: The symbols in kabi/*.h should probably all start with __kabi_
>>
>> Any grand new incompatible thing is something I will happily
>> ignore for as long as I am able to, and I'm not alone here. Your
>> uptake will be zero.
>
> I concur. The purpose of this thing is by definition to provide
> libcs with the kernel/user interface stuff they need in order for
> userspace programs to be compiled. There's no point defining a new
> interface because there is a massive quantity of *existing* code
> out there that we must work with. (Plus, it can be, uh, difficult
> to get changes of this nature into glibc in particular, and glibc
> is the 300-pound gorilla in this particular room. If the headers
> don't have working with it as a goal, they are pointless.)

Hmm, I didn't really explain my idea very well. Let me start with a
list of a facts. If anybody disagrees with any part of this, please
let me know.

1) The <linux/*.h> headers include a lot of information essential to
compiling userspace applications and libraries (libcs in
particular). That same information is also required while building
the kernel (IE: The ABI).
2) Those headers have a lot of declarations and definitions which
must *not* be present while compiling userspace applications, and is
basically kernel-only stuff.
3) Glibc is extremely large and complex 500-pound gorilla and
contains an ugly build process and a lot of static definitions in its
own header files that conflict with the definitions in the kernel
headers.
4) UML runs into a lot of problems when glibc's headers and the
native kernel headers headers conflict.

Here's some of my opinions about this:

1) Trying to create and maintain 2 separate versions of an ABI as
large and complex as the kernel<=>userspace ABI across new versions
and features would be extremely difficult and result in subtle bugs
and missing features, even over a short period of time.
2) Ideally there should be three distinct pieces, the kernel, the
ABI, and userspace. Compiling either the kernel or userspace
requires the ABI, but the ABI depends only on the compiler.
3) Breaking any compatibility is bad
4) Trying to continue to maintain the glibc custom-header-file
status-quo as more APIs and architectures get added to the kernel is
going to become an increasingly difficult and tedious task.

My proposal (which I'm working on sample patches for) would be to
divide up the kernel headers into 2 parts. The first part would be
<kabi/*.h>, and the second would be all the traditional kernel-only
headers. The kabi headers would *only* define things that begin with
the prefix __kabi_. This would mean that the kabi headers have no
risk of namespace contamination with anything else existing in the
kernel or userspace, and since they would depend only on the
compiler, they would be useable anywhere.

The second step would be to convert the traditional linux header to
include the corresponding kabi header, then redefine its own
structures and defines in terms of those in the kabi header. This
would provide complete backwards compatibility to all kernel code, as
well as to anything that currently compiles using the existing kernel
headers. The entire rest of the <linux/*.h> header file would be
wrapped in #ifdef __KERNEL__, as it should not be needed by anything
in userspace.

In the process of those two steps, we would relocate many of the
misplaced "#ifdef __KERNEL__" and "#endif /* __KERNEL__ */". The
kabi headers should not mention __KERNEL__ at all, and the linux/*
headers should be almost completely wrapped in __KERNEL__ ifdefs.
That should be enough to make klibc build correctly, although from
the description glibc needs significantly more work.

Once a significant portion of the kernel headers have been split that
way (preserving complete backwards compatibility), external projects
_may_ be converted to #include <kabi/*.h> instead of #include <linux/
*.h>, although this would require other changes to the source to
handle the __kabi_ prefix. Most of those should be straightforward,
however. Since the kabi/*.h headers would not be kernel-version-
specific, they could be copied to a system running an older kernel
and reused there without problems. Even though some of the syscalls
and ioctls referenced in the kabi headers might not be present on the
running kernel, portable programs are expected to be able to sanely
handle older kernels.

Once the kabi headers are available, it would be possible to begin
cleaning up many of the glibc headers without worrying about
differences between architectures. If all critical constants and
datatypes are already defined in <kabi/*.h> with __kabi_ or __KABI_
prefixes, it should be possible to import those definitions into
klibc and glibc without much effort.

UML has other issues with conflicts between the native kernel headers
and the GLIBC-provided stubs. It's been mentioned on the prior
threads about this topic that this sort of system would ease most of
the issues that UML runs into.

I'm working on some sample patches now which I'll try to post in a
few days if I get the time.

Cheers,
Kyle Moffett

2006-03-24 22:58:50

by Randy Dunlap

[permalink] [raw]

Subject: Re: State of userland headers

On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett wrote:

> On Mar 24, 2006, at 16:48:47, Nix wrote:
> > On 24 Mar 2006, Rob Landley suggested tentatively:
> >> On Friday 24 March 2006 1:51 pm, Kyle Moffett wrote:
> >>> 1: Ewww, bad glibc!
> >>> 2: The symbols in kabi/*.h should probably all start with __kabi_
> >>
> >> Any grand new incompatible thing is something I will happily
> >> ignore for as long as I am able to, and I'm not alone here. Your
> >> uptake will be zero.
> >
> > I concur. The purpose of this thing is by definition to provide
> > libcs with the kernel/user interface stuff they need in order for
> > userspace programs to be compiled. There's no point defining a new
> > interface because there is a massive quantity of *existing* code
> > out there that we must work with. (Plus, it can be, uh, difficult
> > to get changes of this nature into glibc in particular, and glibc
> > is the 300-pound gorilla in this particular room. If the headers
> > don't have working with it as a goal, they are pointless.)
>
> Hmm, I didn't really explain my idea very well. Let me start with a
> list of a facts. If anybody disagrees with any part of this, please
> let me know.
>
> 1) The <linux/*.h> headers include a lot of information essential to
> compiling userspace applications and libraries (libcs in
> particular). That same information is also required while building
> the kernel (IE: The ABI).
> 2) Those headers have a lot of declarations and definitions which
> must *not* be present while compiling userspace applications, and is
> basically kernel-only stuff.
> 3) Glibc is extremely large and complex 500-pound gorilla and
> contains an ugly build process and a lot of static definitions in its
> own header files that conflict with the definitions in the kernel
> headers.
> 4) UML runs into a lot of problems when glibc's headers and the
> native kernel headers headers conflict.
>
> Here's some of my opinions about this:
>
> 1) Trying to create and maintain 2 separate versions of an ABI as
> large and complex as the kernel<=>userspace ABI across new versions
> and features would be extremely difficult and result in subtle bugs
> and missing features, even over a short period of time.
> 2) Ideally there should be three distinct pieces, the kernel, the
> ABI, and userspace. Compiling either the kernel or userspace
> requires the ABI, but the ABI depends only on the compiler.
> 3) Breaking any compatibility is bad
> 4) Trying to continue to maintain the glibc custom-header-file
> status-quo as more APIs and architectures get added to the kernel is
> going to become an increasingly difficult and tedious task.
>
> My proposal (which I'm working on sample patches for) would be to
> divide up the kernel headers into 2 parts. The first part would be
> <kabi/*.h>, and the second would be all the traditional kernel-only
> headers. The kabi headers would *only* define things that begin with
> the prefix __kabi_. This would mean that the kabi headers have no
> risk of namespace contamination with anything else existing in the
> kernel or userspace, and since they would depend only on the
> compiler, they would be useable anywhere.
>
> The second step would be to convert the traditional linux header to
> include the corresponding kabi header, then redefine its own
> structures and defines in terms of those in the kabi header. This
> would provide complete backwards compatibility to all kernel code, as
> well as to anything that currently compiles using the existing kernel
> headers. The entire rest of the <linux/*.h> header file would be
> wrapped in #ifdef __KERNEL__, as it should not be needed by anything
> in userspace.
>
> In the process of those two steps, we would relocate many of the
> misplaced "#ifdef __KERNEL__" and "#endif /* __KERNEL__ */". The
> kabi headers should not mention __KERNEL__ at all, and the linux/*
> headers should be almost completely wrapped in __KERNEL__ ifdefs.
> That should be enough to make klibc build correctly, although from
> the description glibc needs significantly more work.
>
> Once a significant portion of the kernel headers have been split that
> way (preserving complete backwards compatibility), external projects
> _may_ be converted to #include <kabi/*.h> instead of #include <linux/
> *.h>, although this would require other changes to the source to
> handle the __kabi_ prefix. Most of those should be straightforward,
> however. Since the kabi/*.h headers would not be kernel-version-
> specific, they could be copied to a system running an older kernel
> and reused there without problems. Even though some of the syscalls
> and ioctls referenced in the kabi headers might not be present on the
> running kernel, portable programs are expected to be able to sanely
> handle older kernels.
>
> Once the kabi headers are available, it would be possible to begin
> cleaning up many of the glibc headers without worrying about
> differences between architectures. If all critical constants and
> datatypes are already defined in <kabi/*.h> with __kabi_ or __KABI_
> prefixes, it should be possible to import those definitions into
> klibc and glibc without much effort.
>
> UML has other issues with conflicts between the native kernel headers
> and the GLIBC-provided stubs. It's been mentioned on the prior
> threads about this topic that this sort of system would ease most of
> the issues that UML runs into.
>
> I'm working on some sample patches now which I'll try to post in a
> few days if I get the time.

Kyle,
Do you have (recorded) or recall any constraints or requirements
on this $subject from Linus or Andrew or others?
I mean just basic big items, like "thou shalt not mix foo and bar".

I'm just looking for the basic parameters of this task.

Thanks,
---
~Randy

2006-03-25 01:35:43

by Jeff Dike

[permalink] [raw]

Subject: Re: State of userland headers

On Fri, Mar 24, 2006 at 05:46:27PM -0500, Kyle Moffett wrote:
> 4) UML runs into a lot of problems when glibc's headers and the
> native kernel headers headers conflict.

> UML has other issues with conflicts between the native kernel headers
> and the GLIBC-provided stubs. It's been mentioned on the prior
> threads about this topic that this sort of system would ease most of
> the issues that UML runs into.

Actually, this isn't quite the same as what UML hits. I have an
amicable solution (with some warts) to the glibc/kernel header
conflicts - files build against either glibc headers or kernel
headers, but never both.

The warts are where I pass information between those two sets of files
that could be interpretted differently on either side, but aren't
because both sides are Linux. For example, I freely pass errno values
across that interface in the hope that the glibc headers and the
kernel headers agree on what they mean.

My problem with the kernel headers is that they are a mixture of
things that are usable in userspace and things that aren't. This is
closely related, but not identical to, things which are part of the
ABI and things which aren't.

For example, the kernel locks are quite usable in userspace, but you
would never make them part of the ABI.

So, a set of KABI headers would likely make UML's headers cleaner, by
avoiding copying arch headers and using various nasty tricks to
disable objectionable pieces of headers which I steal from the arch.

So what I really want is a superset of the KABI headers, but the KABI
will give me most of what I want.

Jeff

2006-03-25 03:20:32

[permalink] [raw]

Subject: Re: State of userland headers

On Friday 24 March 2006 5:46 pm, Kyle Moffett wrote:

> however. Since the kabi/*.h headers would not be kernel-version-
> specific, they could be copied to a system running an older kernel
> and reused there without problems.

Since when is the kernel ABI not kernel version specific? You can use an
older ABI on a newer kernel, but you can't use a newer ABI on an older
kernel.

> Even though some of the syscalls
> and ioctls referenced in the kabi headers might not be present on the
> running kernel, portable programs are expected to be able to sanely
> handle older kernels.

At the source level, maybe. At the binary level? Not really.

Rob
--
Never bet against the cheap plastic solution.

2006-03-25 06:27:38

[permalink] [raw]

Subject: Re: State of userland headers

On Mar 24, 2006, at 22:19:54, Rob Landley wrote:
> On Friday 24 March 2006 5:46 pm, Kyle Moffett wrote:
>> however. Since the kabi/*.h headers would not be kernel-version-
>> specific, they could be copied to a system running an older kernel
>> and reused there without problems.
>
> Since when is the kernel ABI not kernel version specific? You can
> use an older ABI on a newer kernel, but you can't use a newer ABI
> on an older kernel.

By the very definition of "ABI", it _must_ not be kernel version
specific. A program written for a newer ABI and compiled against it
_must_ be able to gracefully handle syscalls not existing when run on
an older kernel. The program may fail to work entirely because the
syscalls it needs are not present, but that's no different than a
"Kernel does not have feature CONFIG_FOO enabled" issue.

>> Even though some of the syscalls and ioctls referenced in the kabi
>> headers might not be present on the running kernel, portable
>> programs are expected to be able to sanely handle older kernels.
>
> At the source level, maybe. At the binary level? Not really.

The ABI must be compatible at both source and binary interfaces,
"Application _Binary_ Interface" is the name, after all. It's OK to
introduce new features so long as you can gracefully handle them not
being present. Look for example at what happens if I take V4L apps
and run them on a 2.4.0 kernel without V4L. They report that the
kernel is too old or missing features, but aside from V4L not being
present, everything just works.

As a result, I think that if we can clean up the headers in kernel,
we can reuse those same headers for compiling userspace everywhere,
even if the userspace is going to be run on an older kernel.

Cheers,
Kyle Moffett

2006-03-25 06:34:17

[permalink] [raw]

Subject: Re: State of userland headers

On Mar 24, 2006, at 20:36:15, Jeff Dike wrote:
> On Fri, Mar 24, 2006 at 05:46:27PM -0500, Kyle Moffett wrote:
>> 4) UML runs into a lot of problems when glibc's headers and the
>> native kernel headers headers conflict.
>>
>> UML has other issues with conflicts between the native kernel
>> headers and the GLIBC-provided stubs. It's been mentioned on the
>> prior threads about this topic that this sort of system would ease
>> most of the issues that UML runs into.
>
> Actually, this isn't quite the same as what UML hits. My problem
> with the kernel headers is that they are a mixture of things that
> are usable in userspace and things that aren't. This is closely
> related, but not identical to, things which are part of the ABI and
> things which aren't.
>
> For example, the kernel locks are quite usable in userspace, but
> you would never make them part of the ABI.
>
> So, a set of KABI headers would likely make UML's headers cleaner,
> by avoiding copying arch headers and using various nasty tricks to
> disable objectionable pieces of headers which I steal from the arch.
>
> So what I really want is a superset of the KABI headers, but the
> KABI will give me most of what I want.

So perhaps could we define an informal subset of the kernel code that
works in both userspace and kernel-space and put it in include/libk?
Stuff like linked lists, spinlocks (depends on arch, may not be
supported), etc could be in linux/libk and linux/include/libk or
similar, and then from there included into linux/include/linux/
list.h, etc, as well as into the UML files that need it. Since the
provider and user would both be the Linux kernel, I see no issues
with trying to provide a stable interface of any kind, especially if
we document it as "PRIVATE - FOR KERNEL USE ONLY!!!" with big warning
signs. As a nice bonus, this would make it possible to implement some
user-space unit tests of various pieces.

Cheers,
Kyle Moffett

2006-03-25 06:48:28

[permalink] [raw]

Subject: Re: State of userland headers

On Mar 24, 2006, at 18:01:00, Randy.Dunlap wrote:
> Kyle,
> Do you have (recorded) or recall any constraints or requirements on
> this $subject from Linus or Andrew or others? I mean just basic
> big items, like "thou shalt not mix foo and bar".
>
> I'm just looking for the basic parameters of this task.

Well, to a large extent he's actually wisely kind of stayed out of
the flamewars :-D I'm guessing he's hoping that we'll figure
something acceptable out on our own and send him patches without him
having to think about it much. He has said this, though:
> On the bigger question of what to do with kernel headers in
> general, let's just make one thing clear: the kernel headers are
> for the kernel, and big and painful re-organizations that don't
> help _existing_ user space are not going to happen.

He's also said this:
> On Tue, 30 Nov 2004, Alexandre Oliva wrote:
>> Then maybe this is the fundamental problem. As long as the kernel
>> doesn't recognize that an ABI is a contract, rather than an
>> imposition, kernel developers won't care.
>
> That's a silly analogy. Worse, it's a very flawed analogy.
>
> If you want to use a legal analogy, the ABI is not a contract, it's
> a public _license_.
>
> Why? A contract is something that you cannot change unilaterally.
> Clearly the kernel _can_ and will change the ABI without asking
> every existing program for permission.
>
> In a license, you can always just _expand_ the license eventually.
> Maybe you originally licensed for "fly-fishing for trout", and
> later you can expand that license to say "you can also catch
> crawfish" without impacting your existing licensees.
>
> And exactly as with an ABI, the only thing you can't do is _remove_
> rights without getting into trouble.
>
> So get your analogies straight. The kernel ABI is _not_ a contract.

My hope for this is that we can start doing _little_ and clearly
obvious changes that clean up header files. Basically a lot of the
__KERNEL__ definitions need janitorial work. Either they need to be
removed or they need to be put in the right places. Also it seems
like there's a lot of duplication between architectures; for one
example look at all the varieties of FD_SET code in the different
linux/include/asm-*/posix_types.h files. That's one of the areas I'm
trying to clean up into a single linux/fdset.h included from the
pertinent files. Notice how the file forcibly overrides GLIBC; I
think if we can define it as __KABI_FD* and only define the non-
prefixed version in __KERNEL__, then we can provide something that
GLIBC and klibc can get for free, without having to figure out their
own bitmaps. Likewise some of the user-accessible types are uniform
across architectures, and others are haphazardly arrayed, some to be
compatible with other OSes on that architecture, others just because
that's what the arch they copied from used. I'm hoping if I can get
enough small patches flowing, others will join in too and the process
will go easier.

Cheers,
Kyle Moffett

2006-03-25 16:02:13

by Jeff Dike

[permalink] [raw]

Subject: Re: State of userland headers

On Sat, Mar 25, 2006 at 01:33:55AM -0500, Kyle Moffett wrote:
> So perhaps could we define an informal subset of the kernel code that
> works in both userspace and kernel-space and put it in include/libk?
> Stuff like linked lists, spinlocks (depends on arch, may not be
> supported), etc could be in linux/libk and linux/include/libk or
> similar, and then from there included into linux/include/linux/
> list.h, etc, as well as into the UML files that need it. Since the
> provider and user would both be the Linux kernel, I see no issues
> with trying to provide a stable interface of any kind, especially if
> we document it as "PRIVATE - FOR KERNEL USE ONLY!!!" with big warning
> signs. As a nice bonus, this would make it possible to implement some
> user-space unit tests of various pieces.

This would be perfect.

Jeff

2006-03-26 11:52:29

[permalink] [raw]

Subject: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett <[email protected]> wrote:
> I'm working on some sample patches now which I'll try to post in a
> few days if I get the time.

Ok, here's a sample of the KABI conversion and cleanup patches that I'm
proposing. I have a few fundamental goals for these patches:
1) The Linux kernel compiles and works at every step along the way
2) Since most of the headers are currently quite broken with respect to
GLIBC and userspace, I won't spend much extra time preserving
compatibility with GLIBC, userspace, or non-GCC compilers.
3) Everything in include/kabi will have a __kabi_ or __KABI_ prefix.
4) Headers in include/linux that need the KABI interfaces will include
the corresponding <kabi/*.h> header and define or typedef the
necessary KABI definitions to the names the kernel wants.
5) The stuff in include/kabi/*.h should always be completely independent
of userspace/kernelspace and not require any includes outside of
<kabi/*>. This means that the only preprocessor symbols that we can
assume are present are those provided by the compiler itself.

Cheers,
Kyle Moffett

2006-03-26 11:54:31

[permalink] [raw]

Subject: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

Create initial kernel ABI header infrastructure

The model proposed for the new kernel ABI headers is that the headers
pertinent to both kernelspace and userspace would be reorganized and
cleaned out into include/kabi. The new headers should stay in a restricted
namespace such that they are useable anywhere. Just as include/asm is
symlinked to include/asm-$ARCH, include/kabi/arch will be symlinked to
include/kabi/arch-$ARCH to provide a straightforward way of selecting the
correct architecture ABI.

This creates the first architecture-specific header file, "stddef.h", which
implements a basic BITS_PER_LONG definition, and adds the corresponding
Makefile bits to create the include/kabi/arch symlink.

There's probably something else that needs to be done about getting the
correct include/kabi/* files in the tarball but I'm not aware of it at this
time, so I'll just pretend it doesn't exist :-D.

This appears to compile correctly with allmodconfig on powerpc (32-bit)

Signed-off-by: Kyle Moffett <[email protected]>

---
commit affadc51c2b4c96785809a352973a9f6a51a4c37
tree e0979fa864eab8902ccae459060b18f077ad1d4e
parent 36ddf5bbdea7ba4582abc62f106f0f0e9f0b6b91
author Kyle Moffett <[email protected]> Sun, 26 Mar 2006 06:18:06 -0500
committer Kyle Moffett <[email protected]> Sun, 26 Mar 2006 06:18:06 -0500

Makefile | 22 +++++++++++++++++++---
include/kabi/arch-alpha/stddef.h | 7 +++++++
include/kabi/arch-arm/stddef.h | 7 +++++++
include/kabi/arch-arm26/stddef.h | 7 +++++++
include/kabi/arch-cris/stddef.h | 7 +++++++
include/kabi/arch-frv/stddef.h | 7 +++++++
include/kabi/arch-h8300/stddef.h | 7 +++++++
include/kabi/arch-i386/stddef.h | 7 +++++++
include/kabi/arch-ia64/stddef.h | 7 +++++++
include/kabi/arch-m32r/stddef.h | 7 +++++++
include/kabi/arch-m68k/stddef.h | 7 +++++++
include/kabi/arch-m68knommu/stddef.h | 1 +
include/kabi/arch-mips/stddef.h | 12 ++++++++++++
include/kabi/arch-parisc/stddef.h | 12 ++++++++++++
include/kabi/arch-powerpc/stddef.h | 12 ++++++++++++
include/kabi/arch-ppc/stddef.h | 1 +
include/kabi/arch-s390/stddef.h | 12 ++++++++++++
include/kabi/arch-sh/stddef.h | 7 +++++++
include/kabi/arch-sh64/stddef.h | 7 +++++++
include/kabi/arch-sparc/stddef.h | 7 +++++++
include/kabi/arch-sparc64/stddef.h | 7 +++++++
include/kabi/arch-v850/stddef.h | 7 +++++++
include/kabi/arch-x86_64/stddef.h | 7 +++++++
include/kabi/arch-xtensa/stddef.h | 7 +++++++
24 files changed, 188 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index af6210d..8e9045a 100644
--- a/Makefile
+++ b/Makefile
@@ -787,13 +787,15 @@ ifneq ($(KBUILD_SRC),)
/bin/false; \
fi;
$(Q)if [ ! -d include2 ]; then mkdir -p include2; fi;
+ $(Q)if [ ! -d include2/kabi ]; then mkdir -p include2/kabi; fi;
$(Q)ln -fsn $(srctree)/include/asm-$(ARCH) include2/asm
+ $(Q)ln -fsn $(srctree)/include/kabi/arch-$(ARCH) include2/kabi/arch
endif

# prepare2 creates a makefile if using a separate output directory
prepare2: prepare3 outputmakefile

-prepare1: prepare2 include/linux/version.h include/asm \
+prepare1: prepare2 include/linux/version.h include/asm include/kabi/arch \
include/config/MARKER
ifneq ($(KBUILD_MODULES),)
$(Q)rm -rf $(MODVERDIR)
@@ -822,6 +824,12 @@ include/asm:
$(Q)if [ ! -d include ]; then mkdir -p include; fi;
@ln -fsn asm-$(ARCH) $@

+include/kabi/arch:
+ @echo ' SYMLINK $@ -> include/kabi/arch-$(ARCH)'
+ $(Q)if [ ! -d include ]; then mkdir -p include; fi;
+ $(Q)if [ ! -d include/kabi ]; then mkdir -p include/kabi; fi;
+ @ln -fsn arch-$(ARCH) $@
+
# Split autoconf.h into include/linux/config/*

include/config/MARKER: scripts/basic/split-include include/linux/autoconf.h
@@ -944,7 +952,8 @@ CLEAN_FILES += vmlinux System.map \
MRPROPER_DIRS += include/config include2
MRPROPER_FILES += .config .config.old include/asm .version .old_version \
include/linux/autoconf.h include/linux/version.h \
- .kernelrelease Module.symvers tags TAGS cscope*
+ .kernelrelease Module.symvers tags TAGS cscope* \
+ include/kabi/arch

# clean - Delete most, but leave enough to build external modules
#
@@ -1195,12 +1204,19 @@ define all-sources
find $(__srctree)security/selinux/include $(RCS_FIND_IGNORE) \
-name '*.[chS]' -print; \
find $(__srctree)include $(RCS_FIND_IGNORE) \
- $ -name config -o -name 'asm-*' $ -prune \
+ $ -name config -o -name 'asm-*' -o -name 'kabi' $ -prune \
+ -o -name '*.[chS]' -print; \
+ find $(__srctree)include/kabi $(RCS_FIND_IGNORE) \
+ $ -name 'arch-*' $ -prune \
-o -name '*.[chS]' -print; \
for ARCH in $(ALLINCLUDE_ARCHS) ; do \
find $(__srctree)include/asm-$${ARCH} $(RCS_FIND_IGNORE) \
-name '*.[chS]' -print; \
done ; \
+ for ARCH in $(ALLINCLUDE_ARCHS) ; do \
+ find $(__srctree)include/kabi/arch-$${ARCH} $(RCS_FIND_IGNORE) \
+ -name '*.[chS]' -print; \
+ done ; \
find $(__srctree)include/asm-generic $(RCS_FIND_IGNORE) \
-name '*.[chS]' -print )
endef
diff --git a/include/kabi/arch-alpha/stddef.h b/include/kabi/arch-alpha/stddef.h
new file mode 100644
index 0000000..dd0ea38
--- /dev/null
+++ b/include/kabi/arch-alpha/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_ALPHA_STDDEF_H
+# define __KABI_ARCH_ALPHA_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 64
+# define __KABI_BITS_PER_LONG_SHIFT 6
+
+#endif /* not __KABI_ARCH_ALPHA_STDDEF_H */
diff --git a/include/kabi/arch-arm/stddef.h b/include/kabi/arch-arm/stddef.h
new file mode 100644
index 0000000..556b13a
--- /dev/null
+++ b/include/kabi/arch-arm/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_ARM_STDDEF_H
+# define __KABI_ARCH_ARM_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_ARM_STDDEF_H */
diff --git a/include/kabi/arch-arm26/stddef.h b/include/kabi/arch-arm26/stddef.h
new file mode 100644
index 0000000..74a6959
--- /dev/null
+++ b/include/kabi/arch-arm26/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_ARM26_STDDEF_H
+# define __KABI_ARCH_ARM26_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_ARM26_STDDEF_H */
diff --git a/include/kabi/arch-cris/stddef.h b/include/kabi/arch-cris/stddef.h
new file mode 100644
index 0000000..ca3a965
--- /dev/null
+++ b/include/kabi/arch-cris/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_CRIS_STDDEF_H
+# define __KABI_ARCH_CRIS_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_CRIS_STDDEF_H */
diff --git a/include/kabi/arch-frv/stddef.h b/include/kabi/arch-frv/stddef.h
new file mode 100644
index 0000000..6d47d06
--- /dev/null
+++ b/include/kabi/arch-frv/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_FRV_STDDEF_H
+# define __KABI_ARCH_FRV_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_FRV_STDDEF_H */
diff --git a/include/kabi/arch-h8300/stddef.h b/include/kabi/arch-h8300/stddef.h
new file mode 100644
index 0000000..fc05a8e
--- /dev/null
+++ b/include/kabi/arch-h8300/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_H8300_STDDEF_H
+# define __KABI_ARCH_H8300_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_H8300_STDDEF_H */
diff --git a/include/kabi/arch-i386/stddef.h b/include/kabi/arch-i386/stddef.h
new file mode 100644
index 0000000..a2006b9
--- /dev/null
+++ b/include/kabi/arch-i386/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_I386_STDDEF_H
+# define __KABI_ARCH_I386_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_I386_STDDEF_H */
diff --git a/include/kabi/arch-ia64/stddef.h b/include/kabi/arch-ia64/stddef.h
new file mode 100644
index 0000000..6847a92
--- /dev/null
+++ b/include/kabi/arch-ia64/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_IA64_STDDEF_H
+# define __KABI_ARCH_IA64_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 64
+# define __KABI_BITS_PER_LONG_SHIFT 6
+
+#endif /* not __KABI_ARCH_IA64_STDDEF_H */
diff --git a/include/kabi/arch-m32r/stddef.h b/include/kabi/arch-m32r/stddef.h
new file mode 100644
index 0000000..fdb97ad
--- /dev/null
+++ b/include/kabi/arch-m32r/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_M32R_STDDEF_H
+# define __KABI_ARCH_M32R_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_M32R_STDDEF_H */
diff --git a/include/kabi/arch-m68k/stddef.h b/include/kabi/arch-m68k/stddef.h
new file mode 100644
index 0000000..b2c1490
--- /dev/null
+++ b/include/kabi/arch-m68k/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_M68K_STDDEF_H
+# define __KABI_ARCH_M68K_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_M68K_STDDEF_H */
diff --git a/include/kabi/arch-m68knommu/stddef.h b/include/kabi/arch-m68knommu/stddef.h
new file mode 100644
index 0000000..f01d529
--- /dev/null
+++ b/include/kabi/arch-m68knommu/stddef.h
@@ -0,0 +1 @@
+#include <kabi/arch-m68k/stddef.h>
diff --git a/include/kabi/arch-mips/stddef.h b/include/kabi/arch-mips/stddef.h
new file mode 100644
index 0000000..318a2d1
--- /dev/null
+++ b/include/kabi/arch-mips/stddef.h
@@ -0,0 +1,12 @@
+#ifndef __KABI_ARCH_MIPS_STDDEF_H
+# define __KABI_ARCH_MIPS_STDDEF_H 1
+
+# if _MIPS_SZLONG == 64
+# define __KABI_BITS_PER_LONG 64
+# define __KABI_BITS_PER_LONG_SHIFT 6
+# else
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+# endif
+
+#endif /* not __KABI_ARCH_MIPS_STDDEF_H */
diff --git a/include/kabi/arch-parisc/stddef.h b/include/kabi/arch-parisc/stddef.h
new file mode 100644
index 0000000..ae24ae9
--- /dev/null
+++ b/include/kabi/arch-parisc/stddef.h
@@ -0,0 +1,12 @@
+#ifndef __KABI_ARCH_PARISC_STDDEF_H
+# define __KABI_ARCH_PARISC_STDDEF_H 1
+
+# ifdef __LP64__
+# define __KABI_BITS_PER_LONG 64
+# define __KABI_BITS_PER_LONG_SHIFT 6
+# else
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+# endif
+
+#endif /* not __KABI_ARCH_PARISC_STDDEF_H */
diff --git a/include/kabi/arch-powerpc/stddef.h b/include/kabi/arch-powerpc/stddef.h
new file mode 100644
index 0000000..f9acee1
--- /dev/null
+++ b/include/kabi/arch-powerpc/stddef.h
@@ -0,0 +1,12 @@
+#ifndef __KABI_ARCH_POWERPC_STDDEF_H
+# define __KABI_ARCH_POWERPC_STDDEF_H 1
+
+# ifdef __powerpc64__
+# define __KABI_BITS_PER_LONG 64
+# define __KABI_BITS_PER_LONG_SHIFT 6
+# else
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+# endif
+
+#endif /* not __KABI_ARCH_POWERPC_STDDEF_H */
diff --git a/include/kabi/arch-ppc/stddef.h b/include/kabi/arch-ppc/stddef.h
new file mode 100644
index 0000000..a244f18
--- /dev/null
+++ b/include/kabi/arch-ppc/stddef.h
@@ -0,0 +1 @@
+#include <kabi/arch-powerpc/stddef.h>
diff --git a/include/kabi/arch-s390/stddef.h b/include/kabi/arch-s390/stddef.h
new file mode 100644
index 0000000..d1f26b0
--- /dev/null
+++ b/include/kabi/arch-s390/stddef.h
@@ -0,0 +1,12 @@
+#ifndef __KABI_ARCH_S390_STDDEF_H
+# define __KABI_ARCH_S390_STDDEF_H 1
+
+# ifdef __s390x__
+# define __KABI_BITS_PER_LONG 64
+# define __KABI_BITS_PER_LONG_SHIFT 6
+# else
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+# endif
+
+#endif /* not __KABI_ARCH_S390_STDDEF_H */
diff --git a/include/kabi/arch-sh/stddef.h b/include/kabi/arch-sh/stddef.h
new file mode 100644
index 0000000..d0f81dd
--- /dev/null
+++ b/include/kabi/arch-sh/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_SH_STDDEF_H
+# define __KABI_ARCH_SH_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_SH_STDDEF_H */
diff --git a/include/kabi/arch-sh64/stddef.h b/include/kabi/arch-sh64/stddef.h
new file mode 100644
index 0000000..5d00497
--- /dev/null
+++ b/include/kabi/arch-sh64/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_SH64_STDDEF_H
+# define __KABI_ARCH_SH64_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_SH64_STDDEF_H */
diff --git a/include/kabi/arch-sparc/stddef.h b/include/kabi/arch-sparc/stddef.h
new file mode 100644
index 0000000..e00c10c
--- /dev/null
+++ b/include/kabi/arch-sparc/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_SPARC_STDDEF_H
+# define __KABI_ARCH_SPARC_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_SPARC_STDDEF_H */
diff --git a/include/kabi/arch-sparc64/stddef.h b/include/kabi/arch-sparc64/stddef.h
new file mode 100644
index 0000000..5125f4b
--- /dev/null
+++ b/include/kabi/arch-sparc64/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_SPARC64_STDDEF_H
+# define __KABI_ARCH_SPARC64_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 64
+# define __KABI_BITS_PER_LONG_SHIFT 6
+
+#endif /* not __KABI_ARCH_SPARC64_STDDEF_H */
diff --git a/include/kabi/arch-v850/stddef.h b/include/kabi/arch-v850/stddef.h
new file mode 100644
index 0000000..84328e5
--- /dev/null
+++ b/include/kabi/arch-v850/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_V850_STDDEF_H
+# define __KABI_ARCH_V850_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_V850_STDDEF_H */
diff --git a/include/kabi/arch-x86_64/stddef.h b/include/kabi/arch-x86_64/stddef.h
new file mode 100644
index 0000000..e20e620
--- /dev/null
+++ b/include/kabi/arch-x86_64/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_X86_64_STDDEF_H
+# define __KABI_ARCH_X86_64_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 64
+# define __KABI_BITS_PER_LONG_SHIFT 6
+
+#endif /* not __KABI_ARCH_X86_64_STDDEF_H */
diff --git a/include/kabi/arch-xtensa/stddef.h b/include/kabi/arch-xtensa/stddef.h
new file mode 100644
index 0000000..672c19f
--- /dev/null
+++ b/include/kabi/arch-xtensa/stddef.h
@@ -0,0 +1,7 @@
+#ifndef __KABI_ARCH_XTENSA_STDDEF_H
+# define __KABI_ARCH_XTENSA_STDDEF_H 1
+
+# define __KABI_BITS_PER_LONG 32
+# define __KABI_BITS_PER_LONG_SHIFT 5
+
+#endif /* not __KABI_ARCH_XTENSA_STDDEF_H */

2006-03-26 11:55:39

[permalink] [raw]

Subject: [RFC][PATCH 2/2] Generalize fd_set handling across architectures

Generalize fd_set handling across architectures

The fd_set handling is identical across all architectures, but each has its
own unique set of ugly functions or macros to do it. This cleans up all of
the ugliness and merges the duplicated code into a single KABI header,
include/kabi/fdset.h, which is then included in the old places to preserve
existing semantics.

Appears to compile correctly with allmodconfig on powerpc (32-bit)

Signed-off-by: Kyle Moffett <[email protected]>

---
commit c00dc3cc75d218b5c23cefcb5324197f34d93de3
tree b4baa9126e0f7490c782279eb069ff184e59a73f
parent affadc51c2b4c96785809a352973a9f6a51a4c37
author Kyle Moffett <[email protected]> Sun, 26 Mar 2006 06:25:07 -0500
committer Kyle Moffett <[email protected]> Sun, 26 Mar 2006 06:25:07 -0500

include/asm-alpha/posix_types.h | 81 +------------------------------------
include/asm-arm/posix_types.h | 22 +---------
include/asm-arm26/posix_types.h | 22 +---------
include/asm-cris/posix_types.h | 23 -----------
include/asm-frv/posix_types.h | 18 +-------
include/asm-h8300/posix_types.h | 18 +-------
include/asm-i386/posix_types.h | 35 +---------------
include/asm-ia64/posix_types.h | 79 +-----------------------------------
include/asm-m32r/posix_types.h | 75 +---------------------------------
include/asm-m68k/posix_types.h | 18 +-------
include/asm-mips/posix_types.h | 73 ---------------------------------
include/asm-parisc/posix_types.h | 74 +---------------------------------
include/asm-powerpc/posix_types.h | 74 +---------------------------------
include/asm-s390/posix_types.h | 23 +----------
include/asm-sh/posix_types.h | 75 +---------------------------------
include/asm-sh64/posix_types.h | 75 +---------------------------------
include/asm-sparc/posix_types.h | 75 +---------------------------------
include/asm-sparc64/posix_types.h | 75 +---------------------------------
include/asm-v850/posix_types.h | 23 +----------
include/asm-x86_64/posix_types.h | 75 +---------------------------------
include/asm-xtensa/posix_types.h | 68 +------------------------------
include/kabi/fdset.h | 49 ++++++++++++++++++++++
include/linux/fdset.h | 15 +++++++
include/linux/posix_types.h | 35 ----------------
include/linux/time.h | 14 +++---
25 files changed, 112 insertions(+), 1102 deletions(-)

diff --git a/include/asm-alpha/posix_types.h b/include/asm-alpha/posix_types.h
index c78c04a..c303ade 100644
--- a/include/asm-alpha/posix_types.h
+++ b/include/asm-alpha/posix_types.h
@@ -1,6 +1,8 @@
#ifndef _ALPHA_POSIX_TYPES_H
#define _ALPHA_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -41,83 +43,4 @@ typedef __kernel_gid_t __kernel_gid32_t;

typedef unsigned int __kernel_old_dev_t;

-#ifdef __KERNEL__
-
-#ifndef __GNUC__
-
-#define __FD_SET(d, set) ((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-#define __FD_CLR(d, set) ((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-#define __FD_ISSET(d, set) (((set)->fds_bits[__FDELT(d)] & __FDMASK(d)) != 0)
-#define __FD_ZERO(set) \
- ((void) memset ((__ptr_t) (set), 0, sizeof (__kernel_fd_set)))
-
-#else /* __GNUC__ */
-
-/* With GNU C, use inline functions instead so args are evaluated only once: */
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, const __kernel_fd_set *p)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
- unsigned long *tmp = p->fds_bits;
- int i;
-
- if (__builtin_constant_p(__FDSET_LONGS)) {
- switch (__FDSET_LONGS) {
- case 16:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
- tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
- return;
-
- case 8:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- return;
-
- case 4:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- return;
- }
- }
- i = __FDSET_LONGS;
- while (i) {
- i--;
- *tmp = 0;
- tmp++;
- }
-}
-
-#endif /* __GNUC__ */
-
-#endif /* __KERNEL__ */
-
#endif /* _ALPHA_POSIX_TYPES_H */
diff --git a/include/asm-arm/posix_types.h b/include/asm-arm/posix_types.h
index e142a2a..918508b 100644
--- a/include/asm-arm/posix_types.h
+++ b/include/asm-arm/posix_types.h
@@ -13,6 +13,8 @@
#ifndef __ARCH_ARM_POSIX_TYPES_H
#define __ARCH_ARM_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -58,24 +60,4 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-#define __FD_SET(fd, fdsetp) \
- (((fd_set *)(fdsetp))->fds_bits[(fd) >> 5] |= (1<<((fd) & 31)))
-
-#undef __FD_CLR
-#define __FD_CLR(fd, fdsetp) \
- (((fd_set *)(fdsetp))->fds_bits[(fd) >> 5] &= ~(1<<((fd) & 31)))
-
-#undef __FD_ISSET
-#define __FD_ISSET(fd, fdsetp) \
- ((((fd_set *)(fdsetp))->fds_bits[(fd) >> 5] & (1<<((fd) & 31))) != 0)
-
-#undef __FD_ZERO
-#define __FD_ZERO(fdsetp) \
- (memset (fdsetp, 0, sizeof (*(fd_set *)(fdsetp))))
-
-#endif
-
#endif
diff --git a/include/asm-arm26/posix_types.h b/include/asm-arm26/posix_types.h
index f8d1eb4..918508b 100644
--- a/include/asm-arm26/posix_types.h
+++ b/include/asm-arm26/posix_types.h
@@ -13,6 +13,8 @@
#ifndef __ARCH_ARM_POSIX_TYPES_H
#define __ARCH_ARM_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -58,24 +60,4 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-#define __FD_SET(fd, fdsetp) \
- (((fd_set *)(fdsetp))->fds_bits[(fd) >> 5] |= (1<<((fd) & 31)))
-
-#undef __FD_CLR
-#define __FD_CLR(fd, fdsetp) \
- (((fd_set *)(fdsetp))->fds_bits[(fd) >> 5] &= ~(1<<((fd) & 31)))
-
-#undef __FD_ISSET
-#define __FD_ISSET(fd, fdsetp) \
- ((((fd_set *)(fdsetp))->fds_bits[(fd) >> 5] & (1<<((fd) & 31))) != 0)
-
-#undef __FD_ZERO
-#define __FD_ZERO(fdsetp) \
- (memset ((fdsetp), 0, sizeof (*(fd_set *)(fdsetp))))
-
-#endif
-
#endif
diff --git a/include/asm-cris/posix_types.h b/include/asm-cris/posix_types.h
index 6d26fee..fdf1e26 100644
--- a/include/asm-cris/posix_types.h
+++ b/include/asm-cris/posix_types.h
@@ -1,12 +1,9 @@
/* $Id: posix_types.h,v 1.1 2000/07/10 16:32:31 bjornw Exp $ */

-/* We cheat a bit and use our C-coded bitops functions from asm/bitops.h */
-/* I guess we should write these in assembler because they are used often. */
-
#ifndef __ARCH_CRIS_POSIX_TYPES_H
#define __ARCH_CRIS_POSIX_TYPES_H

-#include <asm/bitops.h>
+#include <linux/fdset.h>

/*
* This file is generally used by user-level software, so you need to
@@ -53,22 +50,4 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
} __kernel_fsid_t;

-/* should this ifdef be here ? */
-
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-#define __FD_SET(fd,fdsetp) set_bit(fd, (void *)(fdsetp))
-
-#undef __FD_CLR
-#define __FD_CLR(fd,fdsetp) clear_bit(fd, (void *)(fdsetp))
-
-#undef __FD_ISSET
-#define __FD_ISSET(fd,fdsetp) test_bit(fd, (void *)(fdsetp))
-
-#undef __FD_ZERO
-#define __FD_ZERO(fdsetp) memset((void *)(fdsetp), 0, __FDSET_LONGS << 2)
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif /* __ARCH_CRIS_POSIX_TYPES_H */
diff --git a/include/asm-frv/posix_types.h b/include/asm-frv/posix_types.h
index 73c2ba8..5b241f7 100644
--- a/include/asm-frv/posix_types.h
+++ b/include/asm-frv/posix_types.h
@@ -1,6 +1,8 @@
#ifndef _ASM_POSIX_TYPES_H
#define _ASM_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -46,21 +48,5 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-#define __FD_SET(d, set) ((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-
-#undef __FD_CLR
-#define __FD_CLR(d, set) ((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-
-#undef __FD_ISSET
-#define __FD_ISSET(d, set) (!!((set)->fds_bits[__FDELT(d)] & __FDMASK(d)))
-
-#undef __FD_ZERO
-#define __FD_ZERO(fdsetp) (memset (fdsetp, 0, sizeof(*(fd_set *)fdsetp)))
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif

diff --git a/include/asm-h8300/posix_types.h b/include/asm-h8300/posix_types.h
index 7de94b1..c65fbc9 100644
--- a/include/asm-h8300/posix_types.h
+++ b/include/asm-h8300/posix_types.h
@@ -1,6 +1,8 @@
#ifndef __ARCH_H8300_POSIX_TYPES_H
#define __ARCH_H8300_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -45,20 +47,4 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-#define __FD_SET(d, set) ((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-
-#undef __FD_CLR
-#define __FD_CLR(d, set) ((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-
-#undef __FD_ISSET
-#define __FD_ISSET(d, set) ((set)->fds_bits[__FDELT(d)] & __FDMASK(d))
-
-#undef __FD_ZERO
-#define __FD_ZERO(fdsetp) (memset (fdsetp, 0, sizeof(*(fd_set *)fdsetp)))
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif
diff --git a/include/asm-i386/posix_types.h b/include/asm-i386/posix_types.h
index 4e47ed0..7148932 100644
--- a/include/asm-i386/posix_types.h
+++ b/include/asm-i386/posix_types.h
@@ -1,6 +1,8 @@
#ifndef __ARCH_I386_POSIX_TYPES_H
#define __ARCH_I386_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -46,37 +48,4 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-#define __FD_SET(fd,fdsetp) \
- __asm__ __volatile__("btsl %1,%0": \
- "=m" (*(__kernel_fd_set *) (fdsetp)):"r" ((int) (fd)))
-
-#undef __FD_CLR
-#define __FD_CLR(fd,fdsetp) \
- __asm__ __volatile__("btrl %1,%0": \
- "=m" (*(__kernel_fd_set *) (fdsetp)):"r" ((int) (fd)))
-
-#undef __FD_ISSET
-#define __FD_ISSET(fd,fdsetp) (__extension__ ({ \
- unsigned char __result; \
- __asm__ __volatile__("btl %1,%2 ; setb %0" \
- :"=q" (__result) :"r" ((int) (fd)), \
- "m" (*(__kernel_fd_set *) (fdsetp))); \
- __result; }))
-
-#undef __FD_ZERO
-#define __FD_ZERO(fdsetp) \
-do { \
- int __d0, __d1; \
- __asm__ __volatile__("cld ; rep ; stosl" \
- :"=m" (*(__kernel_fd_set *) (fdsetp)), \
- "=&c" (__d0), "=&D" (__d1) \
- :"a" (0), "1" (__FDSET_LONGS), \
- "2" ((__kernel_fd_set *) (fdsetp)) : "memory"); \
-} while (0)
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif
diff --git a/include/asm-ia64/posix_types.h b/include/asm-ia64/posix_types.h
index adb6227..15a0ad7 100644
--- a/include/asm-ia64/posix_types.h
+++ b/include/asm-ia64/posix_types.h
@@ -1,6 +1,8 @@
#ifndef _ASM_IA64_POSIX_TYPES_H
#define _ASM_IA64_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -46,81 +48,4 @@ typedef __kernel_gid_t __kernel_gid32_t;

typedef unsigned int __kernel_old_dev_t;

-# ifdef __KERNEL__
-
-# ifndef __GNUC__
-
-#define __FD_SET(d, set) ((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-#define __FD_CLR(d, set) ((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-#define __FD_ISSET(d, set) (((set)->fds_bits[__FDELT(d)] & __FDMASK(d)) != 0)
-#define __FD_ZERO(set) \
- ((void) memset ((__ptr_t) (set), 0, sizeof (__kernel_fd_set)))
-
-# else /* !__GNUC__ */
-
-/* With GNU C, use inline functions instead so args are evaluated only once: */
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, const __kernel_fd_set *p)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
- unsigned long *tmp = p->fds_bits;
- int i;
-
- if (__builtin_constant_p(__FDSET_LONGS)) {
- switch (__FDSET_LONGS) {
- case 16:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
- tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
- return;
-
- case 8:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- return;
-
- case 4:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- return;
- }
- }
- i = __FDSET_LONGS;
- while (i) {
- i--;
- *tmp = 0;
- tmp++;
- }
-}
-
-# endif /* !__GNUC__ */
-# endif /* __KERNEL__ */
#endif /* _ASM_IA64_POSIX_TYPES_H */
diff --git a/include/asm-m32r/posix_types.h b/include/asm-m32r/posix_types.h
index 47e7e85..e79af7d 100644
--- a/include/asm-m32r/posix_types.h
+++ b/include/asm-m32r/posix_types.h
@@ -1,6 +1,8 @@
#ifndef _ASM_M32R_POSIX_TYPES_H
#define _ASM_M32R_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/* $Id$ */

/* orig : i386, sh 2.4.18 */
@@ -50,77 +52,4 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- __fdsetp->fds_bits[__tmp] |= (1UL<<__rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- __fdsetp->fds_bits[__tmp] &= ~(1UL<<__rem);
-}
-
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long __fd, const __kernel_fd_set *__p)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- return (__p->fds_bits[__tmp] & (1UL<<__rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *__p)
-{
- unsigned long *__tmp = __p->fds_bits;
- int __i;
-
- if (__builtin_constant_p(__FDSET_LONGS)) {
- switch (__FDSET_LONGS) {
- case 16:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- __tmp[ 4] = 0; __tmp[ 5] = 0;
- __tmp[ 6] = 0; __tmp[ 7] = 0;
- __tmp[ 8] = 0; __tmp[ 9] = 0;
- __tmp[10] = 0; __tmp[11] = 0;
- __tmp[12] = 0; __tmp[13] = 0;
- __tmp[14] = 0; __tmp[15] = 0;
- return;
-
- case 8:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- __tmp[ 4] = 0; __tmp[ 5] = 0;
- __tmp[ 6] = 0; __tmp[ 7] = 0;
- return;
-
- case 4:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- return;
- }
- }
- __i = __FDSET_LONGS;
- while (__i) {
- __i--;
- *__tmp = 0;
- __tmp++;
- }
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif /* _ASM_M32R_POSIX_TYPES_H */
diff --git a/include/asm-m68k/posix_types.h b/include/asm-m68k/posix_types.h
index fa166ee..39b1b77 100644
--- a/include/asm-m68k/posix_types.h
+++ b/include/asm-m68k/posix_types.h
@@ -1,6 +1,8 @@
#ifndef __ARCH_M68K_POSIX_TYPES_H
#define __ARCH_M68K_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -46,20 +48,4 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-#define __FD_SET(d, set) ((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-
-#undef __FD_CLR
-#define __FD_CLR(d, set) ((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-
-#undef __FD_ISSET
-#define __FD_ISSET(d, set) ((set)->fds_bits[__FDELT(d)] & __FDMASK(d))
-
-#undef __FD_ZERO
-#define __FD_ZERO(fdsetp) (memset (fdsetp, 0, sizeof(*(fd_set *)fdsetp)))
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif
diff --git a/include/asm-mips/posix_types.h b/include/asm-mips/posix_types.h
index c2e8a00..01ad0d7 100644
--- a/include/asm-mips/posix_types.h
+++ b/include/asm-mips/posix_types.h
@@ -10,6 +10,7 @@
#define _ASM_POSIX_TYPES_H

#include <asm/sgidefs.h>
+#include <linux/fdset.h>

/*
* This file is generally used by user-level software, so you need to
@@ -69,76 +70,4 @@ typedef struct {
#endif
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- __fdsetp->fds_bits[__tmp] |= (1UL<<__rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- __fdsetp->fds_bits[__tmp] &= ~(1UL<<__rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long __fd, const __kernel_fd_set *__p)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- return (__p->fds_bits[__tmp] & (1UL<<__rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *__p)
-{
- unsigned long *__tmp = __p->fds_bits;
- int __i;
-
- if (__builtin_constant_p(__FDSET_LONGS)) {
- switch (__FDSET_LONGS) {
- case 16:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- __tmp[ 4] = 0; __tmp[ 5] = 0;
- __tmp[ 6] = 0; __tmp[ 7] = 0;
- __tmp[ 8] = 0; __tmp[ 9] = 0;
- __tmp[10] = 0; __tmp[11] = 0;
- __tmp[12] = 0; __tmp[13] = 0;
- __tmp[14] = 0; __tmp[15] = 0;
- return;
-
- case 8:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- __tmp[ 4] = 0; __tmp[ 5] = 0;
- __tmp[ 6] = 0; __tmp[ 7] = 0;
- return;
-
- case 4:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- return;
- }
- }
- __i = __FDSET_LONGS;
- while (__i) {
- __i--;
- *__tmp = 0;
- __tmp++;
- }
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif /* _ASM_POSIX_TYPES_H */
diff --git a/include/asm-parisc/posix_types.h b/include/asm-parisc/posix_types.h
index 9b19970..c58df37 100644
--- a/include/asm-parisc/posix_types.h
+++ b/include/asm-parisc/posix_types.h
@@ -1,6 +1,8 @@
#ifndef __ARCH_PARISC_POSIX_TYPES_H
#define __ARCH_PARISC_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -58,76 +60,4 @@ typedef struct {
typedef __kernel_uid_t __kernel_old_uid_t;
typedef __kernel_gid_t __kernel_old_gid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- __fdsetp->fds_bits[__tmp] |= (1UL<<__rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- __fdsetp->fds_bits[__tmp] &= ~(1UL<<__rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long __fd, const __kernel_fd_set *__p)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- return (__p->fds_bits[__tmp] & (1UL<<__rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *__p)
-{
- unsigned long *__tmp = __p->fds_bits;
- int __i;
-
- if (__builtin_constant_p(__FDSET_LONGS)) {
- switch (__FDSET_LONGS) {
- case 16:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- __tmp[ 4] = 0; __tmp[ 5] = 0;
- __tmp[ 6] = 0; __tmp[ 7] = 0;
- __tmp[ 8] = 0; __tmp[ 9] = 0;
- __tmp[10] = 0; __tmp[11] = 0;
- __tmp[12] = 0; __tmp[13] = 0;
- __tmp[14] = 0; __tmp[15] = 0;
- return;
-
- case 8:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- __tmp[ 4] = 0; __tmp[ 5] = 0;
- __tmp[ 6] = 0; __tmp[ 7] = 0;
- return;
-
- case 4:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- return;
- }
- }
- __i = __FDSET_LONGS;
- while (__i) {
- __i--;
- *__tmp = 0;
- __tmp++;
- }
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif
diff --git a/include/asm-powerpc/posix_types.h b/include/asm-powerpc/posix_types.h
index c639107..7de6c18 100644
--- a/include/asm-powerpc/posix_types.h
+++ b/include/asm-powerpc/posix_types.h
@@ -1,6 +1,8 @@
#ifndef _ASM_POWERPC_POSIX_TYPES_H
#define _ASM_POWERPC_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -54,76 +56,4 @@ typedef struct {
int val[2];
} __kernel_fsid_t;

-#ifndef __GNUC__
-
-#define __FD_SET(d, set) ((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-#define __FD_CLR(d, set) ((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-#define __FD_ISSET(d, set) (((set)->fds_bits[__FDELT(d)] & __FDMASK(d)) != 0)
-#define __FD_ZERO(set) \
- ((void) memset ((__ptr_t) (set), 0, sizeof (__kernel_fd_set)))
-
-#else /* __GNUC__ */
-
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) \
- || (__GLIBC__ == 2 && __GLIBC_MINOR__ == 0)
-/* With GNU C, use inline functions instead so args are evaluated only once: */
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, __kernel_fd_set *p)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
- unsigned long *tmp = (unsigned long *)p->fds_bits;
- int i;
-
- if (__builtin_constant_p(__FDSET_LONGS)) {
- switch (__FDSET_LONGS) {
- case 16:
- tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
- tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
-
- case 8:
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
-
- case 4:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- return;
- }
- }
- i = __FDSET_LONGS;
- while (i) {
- i--;
- *tmp = 0;
- tmp++;
- }
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-#endif /* __GNUC__ */
#endif /* _ASM_POWERPC_POSIX_TYPES_H */
diff --git a/include/asm-s390/posix_types.h b/include/asm-s390/posix_types.h
index 61788de..6cf74f1 100644
--- a/include/asm-s390/posix_types.h
+++ b/include/asm-s390/posix_types.h
@@ -9,6 +9,8 @@
#ifndef __ARCH_S390_POSIX_TYPES_H
#define __ARCH_S390_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -75,25 +77,4 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL)*/
} __kernel_fsid_t;

-
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#ifndef _S390_BITOPS_H
-#include <asm/bitops.h>
-#endif
-
-#undef __FD_SET
-#define __FD_SET(fd,fdsetp) set_bit((fd),(fdsetp)->fds_bits)
-
-#undef __FD_CLR
-#define __FD_CLR(fd,fdsetp) clear_bit((fd),(fdsetp)->fds_bits)
-
-#undef __FD_ISSET
-#define __FD_ISSET(fd,fdsetp) test_bit((fd),(fdsetp)->fds_bits)
-
-#undef __FD_ZERO
-#define __FD_ZERO(fdsetp) (memset ((fdsetp), 0, sizeof(*(fd_set *)(fdsetp))))
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)*/
-
#endif
diff --git a/include/asm-sh/posix_types.h b/include/asm-sh/posix_types.h
index 0a3d2f5..4db22df 100644
--- a/include/asm-sh/posix_types.h
+++ b/include/asm-sh/posix_types.h
@@ -1,6 +1,8 @@
#ifndef __ASM_SH_POSIX_TYPES_H
#define __ASM_SH_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -46,77 +48,4 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- __fdsetp->fds_bits[__tmp] |= (1UL<<__rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- __fdsetp->fds_bits[__tmp] &= ~(1UL<<__rem);
-}
-
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long __fd, const __kernel_fd_set *__p)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- return (__p->fds_bits[__tmp] & (1UL<<__rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *__p)
-{
- unsigned long *__tmp = __p->fds_bits;
- int __i;
-
- if (__builtin_constant_p(__FDSET_LONGS)) {
- switch (__FDSET_LONGS) {
- case 16:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- __tmp[ 4] = 0; __tmp[ 5] = 0;
- __tmp[ 6] = 0; __tmp[ 7] = 0;
- __tmp[ 8] = 0; __tmp[ 9] = 0;
- __tmp[10] = 0; __tmp[11] = 0;
- __tmp[12] = 0; __tmp[13] = 0;
- __tmp[14] = 0; __tmp[15] = 0;
- return;
-
- case 8:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- __tmp[ 4] = 0; __tmp[ 5] = 0;
- __tmp[ 6] = 0; __tmp[ 7] = 0;
- return;
-
- case 4:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- return;
- }
- }
- __i = __FDSET_LONGS;
- while (__i) {
- __i--;
- *__tmp = 0;
- __tmp++;
- }
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif /* __ASM_SH_POSIX_TYPES_H */
diff --git a/include/asm-sh64/posix_types.h b/include/asm-sh64/posix_types.h
index 0620317..97448be 100644
--- a/include/asm-sh64/posix_types.h
+++ b/include/asm-sh64/posix_types.h
@@ -1,6 +1,8 @@
#ifndef __ASM_SH64_POSIX_TYPES_H
#define __ASM_SH64_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is subject to the terms and conditions of the GNU General Public
* License. See the file "COPYING" in the main directory of this archive
@@ -55,77 +57,4 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- __fdsetp->fds_bits[__tmp] |= (1UL<<__rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long __fd, __kernel_fd_set *__fdsetp)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- __fdsetp->fds_bits[__tmp] &= ~(1UL<<__rem);
-}
-
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long __fd, const __kernel_fd_set *__p)
-{
- unsigned long __tmp = __fd / __NFDBITS;
- unsigned long __rem = __fd % __NFDBITS;
- return (__p->fds_bits[__tmp] & (1UL<<__rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *__p)
-{
- unsigned long *__tmp = __p->fds_bits;
- int __i;
-
- if (__builtin_constant_p(__FDSET_LONGS)) {
- switch (__FDSET_LONGS) {
- case 16:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- __tmp[ 4] = 0; __tmp[ 5] = 0;
- __tmp[ 6] = 0; __tmp[ 7] = 0;
- __tmp[ 8] = 0; __tmp[ 9] = 0;
- __tmp[10] = 0; __tmp[11] = 0;
- __tmp[12] = 0; __tmp[13] = 0;
- __tmp[14] = 0; __tmp[15] = 0;
- return;
-
- case 8:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- __tmp[ 4] = 0; __tmp[ 5] = 0;
- __tmp[ 6] = 0; __tmp[ 7] = 0;
- return;
-
- case 4:
- __tmp[ 0] = 0; __tmp[ 1] = 0;
- __tmp[ 2] = 0; __tmp[ 3] = 0;
- return;
- }
- }
- __i = __FDSET_LONGS;
- while (__i) {
- __i--;
- *__tmp = 0;
- __tmp++;
- }
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif /* __ASM_SH64_POSIX_TYPES_H */
diff --git a/include/asm-sparc/posix_types.h b/include/asm-sparc/posix_types.h
index 9ef1b3d..6e6b39e 100644
--- a/include/asm-sparc/posix_types.h
+++ b/include/asm-sparc/posix_types.h
@@ -1,6 +1,8 @@
#ifndef __ARCH_SPARC_POSIX_TYPES_H
#define __ARCH_SPARC_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -46,77 +48,4 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, __const__ __kernel_fd_set *p)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant cases (8 or 32 longs,
- * for 256 and 1024-bit fd_sets respectively)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
- unsigned long *tmp = p->fds_bits;
- int i;
-
- if (__builtin_constant_p(__FDSET_LONGS)) {
- switch (__FDSET_LONGS) {
- case 32:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
- tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
- tmp[16] = 0; tmp[17] = 0; tmp[18] = 0; tmp[19] = 0;
- tmp[20] = 0; tmp[21] = 0; tmp[22] = 0; tmp[23] = 0;
- tmp[24] = 0; tmp[25] = 0; tmp[26] = 0; tmp[27] = 0;
- tmp[28] = 0; tmp[29] = 0; tmp[30] = 0; tmp[31] = 0;
- return;
- case 16:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
- tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
- return;
- case 8:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- return;
- case 4:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- return;
- }
- }
- i = __FDSET_LONGS;
- while (i) {
- i--;
- *tmp = 0;
- tmp++;
- }
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif /* !(__ARCH_SPARC_POSIX_TYPES_H) */
diff --git a/include/asm-sparc64/posix_types.h b/include/asm-sparc64/posix_types.h
index c86b945..3c99210 100644
--- a/include/asm-sparc64/posix_types.h
+++ b/include/asm-sparc64/posix_types.h
@@ -1,6 +1,8 @@
#ifndef __ARCH_SPARC64_POSIX_TYPES_H
#define __ARCH_SPARC64_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -50,77 +52,4 @@ typedef struct {
#endif /* !defined(__KERNEL__) && !defined(__USE_ALL) */
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, __const__ __kernel_fd_set *p)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant cases (8 or 32 longs,
- * for 256 and 1024-bit fd_sets respectively)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
- unsigned long *tmp = p->fds_bits;
- int i;
-
- if (__builtin_constant_p(__FDSET_LONGS)) {
- switch (__FDSET_LONGS) {
- case 32:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
- tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
- tmp[16] = 0; tmp[17] = 0; tmp[18] = 0; tmp[19] = 0;
- tmp[20] = 0; tmp[21] = 0; tmp[22] = 0; tmp[23] = 0;
- tmp[24] = 0; tmp[25] = 0; tmp[26] = 0; tmp[27] = 0;
- tmp[28] = 0; tmp[29] = 0; tmp[30] = 0; tmp[31] = 0;
- return;
- case 16:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
- tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
- return;
- case 8:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- return;
- case 4:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- return;
- }
- }
- i = __FDSET_LONGS;
- while (i) {
- i--;
- *tmp = 0;
- tmp++;
- }
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif /* !(__ARCH_SPARC64_POSIX_TYPES_H) */
diff --git a/include/asm-v850/posix_types.h b/include/asm-v850/posix_types.h
index ccb7297..1c6e0bb 100644
--- a/include/asm-v850/posix_types.h
+++ b/include/asm-v850/posix_types.h
@@ -14,6 +14,8 @@
#ifndef __V850_POSIX_TYPES_H__
#define __V850_POSIX_TYPES_H__

+#include <linux/fdset.h>
+
typedef unsigned long __kernel_ino_t;
typedef unsigned long long __kernel_ino64_t;
typedef unsigned int __kernel_mode_t;
@@ -52,25 +54,4 @@ typedef struct {
} __kernel_fsid_t;

-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2)
-
-/* We used to include <asm/bitops.h> here, which seems the right thing, but
- it caused nasty include-file definition order problems. Removing the
- include seems to work, so fingers crossed... */
-
-#undef __FD_SET
-#define __FD_SET(fd, fd_set) \
- __set_bit (fd, (void *)&((__kernel_fd_set *)fd_set)->fds_bits)
-#undef __FD_CLR
-#define __FD_CLR(fd, fd_set) \
- __clear_bit (fd, (void *)&((__kernel_fd_set *)fd_set)->fds_bits)
-#undef __FD_ISSET
-#define __FD_ISSET(fd, fd_set) \
- __test_bit (fd, (void *)&((__kernel_fd_set *)fd_set)->fds_bits)
-#undef __FD_ZERO
-#define __FD_ZERO(fd_set) \
- memset (fd_set, 0, sizeof (*(fd_set *)fd_set))
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-
#endif /* __V850_POSIX_TYPES_H__ */
diff --git a/include/asm-x86_64/posix_types.h b/include/asm-x86_64/posix_types.h
index 9926aa4..ce54799 100644
--- a/include/asm-x86_64/posix_types.h
+++ b/include/asm-x86_64/posix_types.h
@@ -1,6 +1,8 @@
#ifndef _ASM_X86_64_POSIX_TYPES_H
#define _ASM_X86_64_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -43,77 +45,4 @@ typedef __kernel_gid_t __kernel_gid32_t;

typedef unsigned long __kernel_old_dev_t;

-#ifdef __KERNEL__
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, __const__ __kernel_fd_set *p)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant cases (8 or 32 longs,
- * for 256 and 1024-bit fd_sets respectively)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
- unsigned long *tmp = p->fds_bits;
- int i;
-
- if (__builtin_constant_p(__FDSET_LONGS)) {
- switch (__FDSET_LONGS) {
- case 32:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
- tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
- tmp[16] = 0; tmp[17] = 0; tmp[18] = 0; tmp[19] = 0;
- tmp[20] = 0; tmp[21] = 0; tmp[22] = 0; tmp[23] = 0;
- tmp[24] = 0; tmp[25] = 0; tmp[26] = 0; tmp[27] = 0;
- tmp[28] = 0; tmp[29] = 0; tmp[30] = 0; tmp[31] = 0;
- return;
- case 16:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- tmp[ 8] = 0; tmp[ 9] = 0; tmp[10] = 0; tmp[11] = 0;
- tmp[12] = 0; tmp[13] = 0; tmp[14] = 0; tmp[15] = 0;
- return;
- case 8:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- tmp[ 4] = 0; tmp[ 5] = 0; tmp[ 6] = 0; tmp[ 7] = 0;
- return;
- case 4:
- tmp[ 0] = 0; tmp[ 1] = 0; tmp[ 2] = 0; tmp[ 3] = 0;
- return;
- }
- }
- i = __FDSET_LONGS;
- while (i) {
- i--;
- *tmp = 0;
- tmp++;
- }
-}
-
-#endif /* defined(__KERNEL__) */
-
#endif
diff --git a/include/asm-xtensa/posix_types.h b/include/asm-xtensa/posix_types.h
index 2c816b0..3d1309a 100644
--- a/include/asm-xtensa/posix_types.h
+++ b/include/asm-xtensa/posix_types.h
@@ -13,6 +13,8 @@
#ifndef _XTENSA_POSIX_TYPES_H
#define _XTENSA_POSIX_TYPES_H

+#include <linux/fdset.h>
+
/*
* This file is generally used by user-level software, so you need to
* be a little careful about namespace pollution etc. Also, we cannot
@@ -54,70 +56,4 @@ typedef struct {
int val[2];
} __kernel_fsid_t;

-#ifndef __GNUC__
-
-#define __FD_SET(d, set) ((set)->fds_bits[__FDELT(d)] |= __FDMASK(d))
-#define __FD_CLR(d, set) ((set)->fds_bits[__FDELT(d)] &= ~__FDMASK(d))
-#define __FD_ISSET(d, set) ((set)->fds_bits[__FDELT(d)] & __FDMASK(d))
-#define __FD_ZERO(set) \
- ((void) memset ((__ptr_t) (set), 0, sizeof (__kernel_fd_set)))
-
-#else /* __GNUC__ */
-
-#if defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) \
- || (__GLIBC__ == 2 && __GLIBC_MINOR__ == 0)
-/* With GNU C, use inline functions instead so args are evaluated only once: */
-
-#undef __FD_SET
-static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] |= (1UL<<_rem);
-}
-
-#undef __FD_CLR
-static __inline__ void __FD_CLR(unsigned long fd, __kernel_fd_set *fdsetp)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- fdsetp->fds_bits[_tmp] &= ~(1UL<<_rem);
-}
-
-#undef __FD_ISSET
-static __inline__ int __FD_ISSET(unsigned long fd, __kernel_fd_set *p)
-{
- unsigned long _tmp = fd / __NFDBITS;
- unsigned long _rem = fd % __NFDBITS;
- return (p->fds_bits[_tmp] & (1UL<<_rem)) != 0;
-}
-
-/*
- * This will unroll the loop for the normal constant case (8 ints,
- * for a 256-bit fd_set)
- */
-#undef __FD_ZERO
-static __inline__ void __FD_ZERO(__kernel_fd_set *p)
-{
- unsigned int *tmp = (unsigned int *)p->fds_bits;
- int i;
-
- if (__builtin_constant_p(__FDSET_LONGS)) {
- switch (__FDSET_LONGS) {
- case 8:
- tmp[0] = 0; tmp[1] = 0; tmp[2] = 0; tmp[3] = 0;
- tmp[4] = 0; tmp[5] = 0; tmp[6] = 0; tmp[7] = 0;
- return;
- }
- }
- i = __FDSET_LONGS;
- while (i) {
- i--;
- *tmp = 0;
- tmp++;
- }
-}
-
-#endif /* defined(__KERNEL__) || !defined(__GLIBC__) || (__GLIBC__ < 2) */
-#endif /* __GNUC__ */
#endif /* _XTENSA_POSIX_TYPES_H */
diff --git a/include/kabi/fdset.h b/include/kabi/fdset.h
new file mode 100644
index 0000000..57a9df9
--- /dev/null
+++ b/include/kabi/fdset.h
@@ -0,0 +1,49 @@
+#ifndef __KABI_FDSET_H
+# define __KABI_FDSET_H 1
+
+# include <kabi/arch/stddef.h>
+
+/*
+ * This allows for 1024 file descriptors: if NR_OPEN is ever grown beyond
+ * that you'll have to change this too. But 1024 fd's seem to be enough even
+ * for such "real" unices like OSF/1, so hopefully this is one limit that
+ * doesn't have to be changed [again].
+ *
+ * TODO: This should probably use the generic bitops in some form, since it's
+ * using unsigned longs (as do the generic bitops), and has an identical
+ * in-memory layout on all architectures.
+ */
+
+# define __KABI_NFDBITS __KABI_BITS_PER_LONG
+# define __KABI_FD_SETSIZE (1024UL)
+# define __KABI_FDSET_LONGS (__KABI_FD_SETSIZE/__KABI_NFDBITS)
+# define __KABI_FDELT(fd) ((fd)/__KABI_NFDBITS)
+# define __KABI_FDMASK(fd) (1UL << ((fd) % __KABI_NFDBITS))
+
+typedef struct {
+ unsigned long fds_bits [__KABI_FDSET_LONGS];
+} __kabi_fd_set;
+
+static __inline__ void __KABI_FD_SET(unsigned long fd, __kabi_fd_set *p)
+{
+ p->fds_bits[__KABI_FDELT(fd)] |= __KABI_FDMASK(fd);
+}
+
+static __inline__ void __KABI_FD_CLR(unsigned long fd, __kabi_fd_set *p)
+{
+ p->fds_bits[__KABI_FDELT(fd)] &= ~__KABI_FDMASK(fd);
+}
+
+static __inline__ int __KABI_FD_ISSET(unsigned long fd, __kabi_fd_set *p)
+{
+ return !!(p->fds_bits[__KABI_FDELT(fd)] & __KABI_FDMASK(fd));
+}
+
+static __inline__ void __KABI_FD_ZERO(__kabi_fd_set *p)
+{
+ unsigned long i;
+ for (i = 0; i < __KABI_FDSET_LONGS; i++)
+ p->fds_bits[i] = 0;
+}
+
+#endif /* not __KABI_FDSET_H */
diff --git a/include/linux/fdset.h b/include/linux/fdset.h
new file mode 100644
index 0000000..1e92ba4
--- /dev/null
+++ b/include/linux/fdset.h
@@ -0,0 +1,15 @@
+#ifndef __LINUX_FDSET_H
+# define __LINUX_FDSET_H 1
+
+# include <kabi/fdset.h>
+
+# define __NFDBITS __KABI_NFDBITS
+# define __FD_SETSIZE __KABI_FD_SETSIZE
+# define __FD_SET(fd,set) __KABI_FD_SET(fd,set)
+# define __FD_CLR(fd,set) __KABI_FD_CLR(fd,set)
+# define __FD_ISSET(fd,set) __KABI_FD_ISSET(fd,set)
+# define __FD_ZERO(set) __KABI_FD_ZERO(set)
+
+typedef __kabi_fd_set __kernel_fd_set;
+
+#endif /* not __LINUX_FDSET_H */
diff --git a/include/linux/posix_types.h b/include/linux/posix_types.h
index f04c98c..1b571da 100644
--- a/include/linux/posix_types.h
+++ b/include/linux/posix_types.h
@@ -2,40 +2,7 @@
#define _LINUX_POSIX_TYPES_H

#include <linux/stddef.h>
-
-/*
- * This allows for 1024 file descriptors: if NR_OPEN is ever grown
- * beyond that you'll have to change this too. But 1024 fd's seem to be
- * enough even for such "real" unices like OSF/1, so hopefully this is
- * one limit that doesn't have to be changed [again].
- *
- * Note that POSIX wants the FD_CLEAR(fd,fdsetp) defines to be in
- * <sys/time.h> (and thus <linux/time.h>) - but this is a more logical
- * place for them. Solved by having dummy defines in <sys/time.h>.
- */
-
-/*
- * Those macros may have been defined in <gnu/types.h>. But we always
- * use the ones here.
- */
-#undef __NFDBITS
-#define __NFDBITS (8 * sizeof(unsigned long))
-
-#undef __FD_SETSIZE
-#define __FD_SETSIZE 1024
-
-#undef __FDSET_LONGS
-#define __FDSET_LONGS (__FD_SETSIZE/__NFDBITS)
-
-#undef __FDELT
-#define __FDELT(d) ((d) / __NFDBITS)
-
-#undef __FDMASK
-#define __FDMASK(d) (1UL << ((d) % __NFDBITS))
-
-typedef struct {
- unsigned long fds_bits [__FDSET_LONGS];
-} __kernel_fd_set;
+#include <linux/fdset.h>

/* Type of a signal handler. */
typedef void (*__kernel_sighandler_t)(int);
diff --git a/include/linux/time.h b/include/linux/time.h
index bf0e785..e2dbfa9 100644
--- a/include/linux/time.h
+++ b/include/linux/time.h
@@ -150,13 +150,13 @@ extern struct timeval ns_to_timeval(cons

#endif /* __KERNEL__ */

-#define NFDBITS __NFDBITS
-
-#define FD_SETSIZE __FD_SETSIZE
-#define FD_SET(fd,fdsetp) __FD_SET(fd,fdsetp)
-#define FD_CLR(fd,fdsetp) __FD_CLR(fd,fdsetp)
-#define FD_ISSET(fd,fdsetp) __FD_ISSET(fd,fdsetp)
-#define FD_ZERO(fdsetp) __FD_ZERO(fdsetp)
+#include <kabi/fdset.h>
+#define NFDBITS __KABI_NFDBITS
+#define FD_SETSIZE __KABI_FD_SETSIZE
+#define FD_SET(fd,set) __KABI_FD_SET(fd,set)
+#define FD_CLR(fd,set) __KABI_FD_CLR(fd,set)
+#define FD_ISSET(fd,set) __KABI_FD_ISSET(fd,set)
+#define FD_ZERO(set) __KABI_FD_ZERO(set)

/*
* Names of the interval timers, and structure

2006-03-26 12:06:17

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Sun, 26 Mar 2006 06:52:05 -0500 Kyle Moffett <[email protected]> wrote:
> 2) Since most of the headers are currently quite broken with respect to
> GLIBC and userspace, I won't spend much extra time preserving
> compatibility with GLIBC, userspace, or non-GCC compilers.

That didn't come out right, but what I meant to say was this: Since the
headers in include/linux are quite broken with respect to GLIBC and
userspace, I won't let so-called "compatibility" code like this get in
the way:
#ifndef __GNUC__
#define DO_SOMETHING(foo) ICKY_MACRO
#else
static __inline__ void DO_SOMETHING(int foo)
{
sensible_inline_function();
}
#endif

You can see where I take that approach in the patches I sent.

One other thing I would like to point out: The fd_set code wants to use
__set_bit and __clear_bit from <linux/bitops.h>, but those really should
not be accessible to userspace directly. I would like to propose moving
that functionality into <__klib/*.h> from which it would be accessible
to both <linux/*.h> and <kabi/*.h>. I think this would also help with
the UML header issues by providing those kernel-internal APIs to the
kernel when run from userspace. (Please correct me if I'm wrong).

I appreciate your comments and corrections, thanks!

Cheers,
Kyle Moffett

2006-03-26 12:28:14

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Sun, 2006-03-26 at 06:52 -0500, Kyle Moffett wrote:
> On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett <[email protected]> wrote:
> > I'm working on some sample patches now which I'll try to post in a
> > few days if I get the time.
>
> Ok, here's a sample of the KABI conversion and cleanup patches that I'm
> proposing. I have a few fundamental goals for these patches:

is KABI the right name? I mean.. from the kernel pov it's the interface
to userspace ;)

2006-03-26 12:31:53

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Sun, 2006-03-26 at 14:26 +0200, Arjan van de Ven wrote:
> On Sun, 2006-03-26 at 06:52 -0500, Kyle Moffett wrote:
> > On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett <[email protected]> wrote:
> > > I'm working on some sample patches now which I'll try to post in a
> > > few days if I get the time.
> >
> > Ok, here's a sample of the KABI conversion and cleanup patches that I'm
> > proposing. I have a few fundamental goals for these patches:
>
> is KABI the right name? I mean.. from the kernel pov it's the interface
> to userspace ;)

ok proposal: just use "abi" without the K; it is THE abi, and the only
one the kernel has, so no need for prefixes or suffixes.

(BTW I'm 100% behind the idea of a clean set of seperate ABI headers)

2006-03-26 12:33:45

[permalink] [raw]

Subject: Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

On Sun, 2006-03-26 at 06:54 -0500, Kyle Moffett wrote:
> Create initial kernel ABI header infrastructure

it's nice that you picked this one;
for this you want an arch-generic/stddef32.h and stddef64.h

and have arch-foo just only include the proper generic one..

(and... why do you prefix these with _KABI? that's a mistake imo. Don't
both with that. Really. Either these need exporting to userspace, but
then either use __ as prefix or don't use a prefix. But KABI.. No.)

2006-03-26 12:34:36

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Mar 26, 2006, at 07:26:48, Arjan van de Ven wrote:
> On Sun, 2006-03-26 at 06:52 -0500, Kyle Moffett wrote:
>> On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett
>> <[email protected]> wrote:
>>> I'm working on some sample patches now which I'll try to post in
>>> a few days if I get the time.
>>
>> Ok, here's a sample of the KABI conversion and cleanup patches
>> that I'm proposing. I have a few fundamental goals for these
>> patches:
>
> is KABI the right name? I mean.. from the kernel pov it's the
> interface to userspace ;)

Well I guess you could call it UABI, but that might also imply that
it's _userspace_ that defines the interface, instead of the kernel.
Since the headers themselves are rather tightly coupled with the
kernel, I think I'll stick with the KABI name for now (unless
somebody can come up with a better one, of course :-D).

Cheers,
Kyle Moffett

2006-03-26 12:50:43

[permalink] [raw]

Subject: Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

On Mar 26, 2006, at 07:32:31, Arjan van de Ven wrote:
> On Sun, 2006-03-26 at 06:54 -0500, Kyle Moffett wrote:
>> Create initial kernel ABI header infrastructure
>
> it's nice that you picked this one;
> for this you want an arch-generic/stddef32.h and stddef64.h
>
> and have arch-foo just only include the proper generic one..

I plan to add a lot of other definitions to this file later on. For
example different architectures have different notions of what a
__kernel_ino_t is (unsigned int versus unsigned long). I may rename
this file as types.h, but from looking through the code I figure I'll
have enough general purpose declarations about "This architecture has
blah" that a separate stddef.h file will be worth it.

> (and... why do you prefix these with _KABI? that's a mistake imo.
> Don't bother with that. Really. Either these need exporting to
> userspace, but then either use __ as prefix or don't use a prefix.
> But KABI.. No.)

According to the various standards all symbols beginning with __ are
reserved for "The Implementation", including the compiler, the
standard library, the kernel, etc. In order to avoid clashing with
any/all of those, I picked the __KABI_ and __kabi_ prefixes for
uniqueness. In theory I could just use __, but there are problems
with that too. For example, note how the current compiler.h files
redefine __always_inline to mean something kinda different. The GCC
manual says we should be able to write this:

inline __attribute__((__always_inline)) int increment(int x)
{
return x+1;
}

Except when compiling the kernel headers turn that into this (which
obviously doesn't compile):
inline __attribute__((__attribute__((always_inline)))) int increment
(int x)
{
return x+1;
}

As a result, I kinda want to stay away from anything that remotely
looks like a conflicting namespace. Using such a unique namespace
means we can also safely do this if necessary (Since you can't
"typedef struct foo struct bar"):

kabi/foo.h:
struct __kabi_foo {
int x;
int y;
};

linux/foo.h:
#define __kabi_foo foo
#include <kabi/foo.h>

drivers/foo/foo.h:
#include <linux/foo.h>
void func()
{
struct foo = { .x = 1, .y = 2 };
}

Cheers,
Kyle Moffett

2006-03-26 12:59:37

by Martin Mares

[permalink] [raw]

Subject: Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

Hello!

> As a result, I kinda want to stay away from anything that remotely
> looks like a conflicting namespace. Using such a unique namespace
> means we can also safely do this if necessary (Since you can't
> "typedef struct foo struct bar"):
>
> kabi/foo.h:
> struct __kabi_foo {
> int x;
> int y;
> };
>
> linux/foo.h:
> #define __kabi_foo foo
> #include <kabi/foo.h>

This looks very fragile -- <kabi/foo.h> can be included earlier by another
header.

Have a nice fortnight
--
Martin `MJ' Mares <[email protected]> http://atrey.karlin.mff.cuni.cz/~mj/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
And God said: E = 1/2mv^2 - Ze^2/r ...and there *WAS* light!

2006-03-26 13:14:57

[permalink] [raw]

Subject: Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

On Mar 26, 2006, at 07:59:26, Martin Mares wrote:
> Hello!
>
>> As a result, I kinda want to stay away from anything that remotely
>> looks like a conflicting namespace. Using such a unique namespace
>> means we can also safely do this if necessary (Since you can't
>> "typedef struct foo struct bar"):
>>
>> kabi/foo.h:
>> struct __kabi_foo {
>> int x;
>> int y;
>> };
>>
>> linux/foo.h:
>> #define __kabi_foo foo
>> #include <kabi/foo.h>
>
> This looks very fragile -- <kabi/foo.h> can be included earlier by
> another header.

It _is_ fragile, but for a number of POSIX-defined structs that's
actually the only way to do it without duplicating the data structure
in entirety, unless the GCC people can implement a "typedef struct
foo struct bar;" Hopefully it would be a relatively rare occurrence
and carefully thought out in any case. We may end up with some
duplication regardless :-\.

Cheers,
Kyle Moffett

2006-03-26 13:15:08

by Mariusz Mazur

[permalink] [raw]

Subject: Re: State of userland headers

On Friday 24 March 2006 00:04, Rob Landley wrote:
> You also don't want to run a libc built with newer headers than the kernel
> you're running on, or it'll try to use stuff that isn't there.
>
> You're saying that the new kernel headers wouldn't be versioned using the
> kernel's release numbers. How do we know what kernel version their feature
> set matches then? (I'm confused. This happens easily...)

That's a tradeoff. You either version the headers just like I did, meaning
that a given version corresponds to a given kernel, but that means you can't
release before all of the archs are fully updated (and not relying on a
single person to do all of the updates is one of the points of the exercise;
and with more people, one can have delays) or you're forced to figure out
some other way to version the headers.

--
Judge others by their intentions and yourself by your results.
Guy Kawasaki
Education is an admirable thing, but it is well to remember from
time to time that nothing that is worth knowing can be taught.
Oscar Wilde

2006-03-26 13:24:38

by Giuseppe Bilotta

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Sun, 26 Mar 2006 07:34:12 -0500, Kyle Moffett wrote:

> On Mar 26, 2006, at 07:26:48, Arjan van de Ven wrote:
>> On Sun, 2006-03-26 at 06:52 -0500, Kyle Moffett wrote:
>>> On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett
>>> <[email protected]> wrote:
>>>> I'm working on some sample patches now which I'll try to post in
>>>> a few days if I get the time.
>>>
>>> Ok, here's a sample of the KABI conversion and cleanup patches
>>> that I'm proposing. I have a few fundamental goals for these
>>> patches:
>>
>> is KABI the right name? I mean.. from the kernel pov it's the
>> interface to userspace ;)
>
> Well I guess you could call it UABI, but that might also imply that
> it's _userspace_ that defines the interface, instead of the kernel.
> Since the headers themselves are rather tightly coupled with the
> kernel, I think I'll stick with the KABI name for now (unless
> somebody can come up with a better one, of course :-D).

Well, since I guess UAKDABI (User Accessible Kernel Defined) is out of
the question, what about LABI? (Linux ABI)

--
Giuseppe "Oblomov" Bilotta

Axiom I of the Giuseppe Bilotta
theory of IT:
Anything is better than MS

2006-03-26 13:29:55

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

Kyle Moffett wrote:
>
> Well I guess you could call it UABI, but that might also imply that
> it's _userspace_ that defines the interface, instead of the kernel.
> Since the headers themselves are rather tightly coupled with the
> kernel, I think I'll stick with the KABI name for now (unless somebody
> can come up with a better one, of course :-D).
>
How about __linux, or __linux_abi? There are ABIs for other components,
and other OSes. Linux is the name of the project after all.

--
error compiling committee.c: too many arguments to function

2006-03-26 13:44:11

by Nix

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Sun, 26 Mar 2006, Kyle Moffett said:
> On Sun, 26 Mar 2006 06:52:05 -0500 Kyle Moffett <[email protected]> wrote:
>> 2) Since most of the headers are currently quite broken with respect to
>> GLIBC and userspace, I won't spend much extra time preserving
>> compatibility with GLIBC, userspace, or non-GCC compilers.
>
> That didn't come out right, but what I meant to say was this: Since the
> headers in include/linux are quite broken with respect to GLIBC and
> userspace, I won't let so-called "compatibility" code like this get in
> the way:

As long as the eventual goal is something that *works* with glibc and
uClibc and the rest of them, that's good enough as far as I can tell.
Obviously compatibility crud *in the headers* which serves no obvious
purpose and which userspace doesn't actually need should die (and
there's certainly some like that lying around).

--
`Come now, you should know that whenever you plan the duration of your
unplanned downtime, you should add in padding for random management
freakouts.'

2006-03-26 13:48:13

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Mar 26, 2006, at 08:29:49, Avi Kivity wrote:
> Kyle Moffett wrote:
>> Well I guess you could call it UABI, but that might also imply
>> that it's _userspace_ that defines the interface, instead of the
>> kernel. Since the headers themselves are rather tightly coupled
>> with the kernel, I think I'll stick with the KABI name for now
>> (unless somebody can come up with a better one, of course :-D).
>
> How about __linux, or __linux_abi? There are ABIs for other
> components, and other OSes. Linux is the name of the project after
> all.

The other thing that I quickly noticed while writing up the patches
is that it's kind of tedious typing __kabi_ over and over again. I
actually did first try with __linux_abi_ but the typing effort and
finger cramps made me give up on that really quickly.

Cheers,
Kyle Moffett

2006-03-26 13:55:55

by Giuseppe Bilotta

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Sun, 26 Mar 2006 08:47:51 -0500, Kyle Moffett wrote:

> The other thing that I quickly noticed while writing up the patches
> is that it's kind of tedious typing __kabi_ over and over again. I
> actually did first try with __linux_abi_ but the typing effort and
> finger cramps made me give up on that really quickly.

What kind of barebone editor are you using that doesn't offer a way
(through macro or automatic completion or abbreviations or
substitution or plain copy and paste or whatever else) to shorten that
typing effort?

--
Giuseppe "Oblomov" Bilotta

[W]hat country can preserve its liberties, if its rulers are not
warned from time to time that [the] people preserve the spirit of
resistance? Let them take arms...The tree of liberty must be
refreshed from time to time, with the blood of patriots and
tyrants.
-- Thomas Jefferson, letter to Col. William S. Smith, 1787

2006-03-26 14:30:37

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Mar 26, 2006, at 08:53:05, Giuseppe Bilotta wrote:
> On Sun, 26 Mar 2006 08:47:51 -0500, Kyle Moffett wrote:
>> The other thing that I quickly noticed while writing up the
>> patches is that it's kind of tedious typing __kabi_ over and over
>> again. I actually did first try with __linux_abi_ but the typing
>> effort and finger cramps made me give up on that really quickly.
>
> What kind of barebone editor are you using that doesn't offer a way
> (through macro or automatic completion or abbreviations or
> substitution or plain copy and paste or whatever else) to shorten
> that typing effort?

Well, actually I was using vim, and although I know how to configure
shortcuts like that; do we really want to force people to use
shortcuts and abbreviations to be able to sanely edit the ABI code?
Isn't that counterproductive with the desire to get the maintainers
fixing up their own ABIs? Plus all the extra verbiage tends to make
the lines too long (80-column limit) and makes it harder to read.

Cheers,
Kyle Moffett

2006-03-26 14:32:31

by Éric Piel

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

26.03.2006 13:52, Kyle Moffett wrote/a ?crit:
> On Fri, 24 Mar 2006 17:46:27 -0500 Kyle Moffett <[email protected]> wrote:
>> I'm working on some sample patches now which I'll try to post in a
>> few days if I get the time.
>
> Ok, here's a sample of the KABI conversion and cleanup patches that I'm
> proposing. I have a few fundamental goals for these patches:
> 1) The Linux kernel compiles and works at every step along the way
> 2) Since most of the headers are currently quite broken with respect to
> GLIBC and userspace, I won't spend much extra time preserving
> compatibility with GLIBC, userspace, or non-GCC compilers.
> 3) Everything in include/kabi will have a __kabi_ or __KABI_ prefix.
> 4) Headers in include/linux that need the KABI interfaces will include
> the corresponding <kabi/*.h> header and define or typedef the
> necessary KABI definitions to the names the kernel wants.
> 5) The stuff in include/kabi/*.h should always be completely independent
> of userspace/kernelspace and not require any includes outside of
> <kabi/*>. This means that the only preprocessor symbols that we can
> assume are present are those provided by the compiler itself.
Hello,

I completely agree with rules 1, 2 and 5. However, IMHO rule 4 should
just be the inverse of rule 5: The stuff in include/linux should always
be independent from KABI (and userspace of course). Simply because the
way we _implement_ things in the kernel has to be different from the
things that we _specify_ in the kernel ABI. They just append to be both
written in C language, but it's not a reason to mix them. The kernel
developers has to be free of doing any kludge, clever things,
compatibility workarounds without affecting the userspace applications.
Otherwise, you'll end up with another include/linux after few months!
Separating the implementation and the binary specification has the
additional advantage that if some kernel hacker mistakenly change the
ABI, it's easy to say : "see, after your commit xxxxxxxx, the linux
header and the kabi header are semantically different. You did something
Wrong".

As for rule 3, if you have independent headers, this should be much less
necessary. Additionally, keeping all the names identical to what they
are already called will allow userspace to just use include/kabi/ as the
/usr/include/linux/ directory. Avoiding smelly things like:

linux/foo.h:
#define __kabi_foo foo
#include <kabi/foo.h>

That was my 2 cents :-)
Regards,
Eric

2006-03-26 14:40:47

[permalink] [raw]

Subject: Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

>
> According to the various standards all symbols beginning with __ are
> reserved for "The Implementation", including the compiler, the
> standard library, the kernel, etc. In order to avoid clashing with
> any/all of those, I picked the __KABI_ and __kabi_ prefixes for
> uniqueness. In theory I could just use __, but there are problems
> with that too. For example, note how the current compiler.h files
> redefine __always_inline to mean something kinda different.

well... the "problem" is there today, and... it's not much of a problem
if at all; there's just a few simple rules to keep it that way which
seem to wkr.

And your __alway_inline example.. that's something that really is kernel
internal and shouldn't be exposed to userland.

I think the "problem" really is not there if
1) we only use __ symbols like we do today for non-structs
2) avoid including kernel headers in kernel headers as far as possible.
This means, that if an application wants to use MTD struct
"struct mtd_foo" it will have to include the MTD header, but that
he otherwise never gets it. Eg all such symbols are in a "Yes I
really want it" header.
3) keep the userspace exposed stuff as small as reasonable. Your
__always_inline example doesn't make that cut. A struct used as
ioctl of a subsystem/driver in the header specially for that
subsystem/driver does.

2006-03-26 14:48:18

by Giuseppe Bilotta

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Sun, 26 Mar 2006 09:30:32 -0500, Kyle Moffett wrote:

> On Mar 26, 2006, at 08:53:05, Giuseppe Bilotta wrote:
>> On Sun, 26 Mar 2006 08:47:51 -0500, Kyle Moffett wrote:
>>> The other thing that I quickly noticed while writing up the
>>> patches is that it's kind of tedious typing __kabi_ over and over
>>> again. I actually did first try with __linux_abi_ but the typing
>>> effort and finger cramps made me give up on that really quickly.
>>
>> What kind of barebone editor are you using that doesn't offer a way
>> (through macro or automatic completion or abbreviations or
>> substitution or plain copy and paste or whatever else) to shorten
>> that typing effort?
>
> Well, actually I was using vim, and although I know how to configure
> shortcuts like that; do we really want to force people to use
> shortcuts and abbreviations to be able to sanely edit the ABI code?
> Isn't that counterproductive with the desire to get the maintainers
> fixing up their own ABIs? Plus all the extra verbiage tends to make
> the lines too long (80-column limit) and makes it harder to read.

Oh, I'm not exactly a fan of verbosity myself (essentially for the
lines too long and hard to read), which is why I suggested __labi_ as
a prefix --I wasn't arguing in favour of __linux_abi_.

However, as far as I know all programmer-friendly editors have some
form of easy to use code-completion feature (e.g. in vim ctrl+n/ctrl+p
are very handy without the need to define specific shortcuts, and the
new completion code in the future vim 7 is extremely promising ...)
and while I do agree that __linux_abi_ is too long, you remarked
yourself __[ukl]abi_ is still annoying, albeit shorter :) *Any* prefix
chosen will benefit from completion ;)

--
Giuseppe "Oblomov" Bilotta

"They that can give up essential liberty to obtain
a little temporary safety deserve neither liberty
nor safety." Benjamin Franklin

2006-03-26 15:23:28

[permalink] [raw]

Subject: Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

On Mar 26, 2006, at 09:39:28, Arjan van de Ven wrote:
>> According to the various standards all symbols beginning with __
>> are reserved for "The Implementation", including the compiler, the
>> standard library, the kernel, etc. In order to avoid clashing
>> with any/all of those, I picked the __KABI_ and __kabi_ prefixes
>> for uniqueness. In theory I could just use __, but there are
>> problems with that too. For example, note how the current
>> compiler.h files redefine __always_inline to mean something kinda
>> different.
>
> well... the "problem" is there today, and... it's not much of a
> problem if at all; there's just a few simple rules to keep it that
> way which seem to work.

The current rules work for things kept completely private to the
kernel. If we start exporting things into the guts of libc, we can
expect that those guys have _also_ been using all sorts of double-
underscore symbols in conflicting ways.

> And your __alway_inline example.. that's something that really is
> kernel internal and shouldn't be exposed to userland.

Yes, but it illustrates how a supposedly-innocuous double-underscore
symbol actually conflicts with a GCC builtin, and that's just GCC!
That's not including the massive "The Implementation" codebase that
is GLIBC.

> 2) avoid including kernel headers in kernel headers as far as
> possible. This means, that if an application wants to use MTD
> struct "struct mtd_foo" it will have to include the MTD header, but
> that he otherwise never gets it. Eg all such symbols are in a "Yes
> I really want it" header.

This is Hard(TM). Take a look at the interdependent mess which is
kernel.h, or sched.h. Feel free to submit patches :-D That's
actually part of the reason why I'm trying this kabi/*.h bit. I
think that if I can pull out userspace-required struct definitions
from the rest of the code, we'll notice that some of the header files
don't need as many dependencies anymore and can be cleaned out a bit.

I've also been looking at GCC's "precompiled header" stuff. I think
that a single kabi.h file that includes all other kabi headers,
precompiled to "kabi.h.gch", could potentially speed up a portion of
the linux kernel build by removing a lot of text parsing. I can't
tell for sure till I've cleaned up more crap and done benchmarks, but
I'm willing to give it a shot. :-D

Cheers,
Kyle Moffett

2006-03-26 15:39:05

by Martin Mares

[permalink] [raw]

Subject: Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

Hello!

> It _is_ fragile, but for a number of POSIX-defined structs that's
> actually the only way to do it without duplicating the data structure
> in entirety, unless the GCC people can implement a "typedef struct
> foo struct bar;"

Actually, something like that can be achieved using anonymous structure
members:

struct xxx {
struct yyy;
};

However, I'm not sure how old versions of GCC support this feature.

Have a nice fortnight
--
Martin `MJ' Mares <[email protected]> http://atrey.karlin.mff.cuni.cz/~mj/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
And God said: E = 1/2mv^2 - Ze^2/r ...and there *WAS* light!

2006-03-26 16:17:18

[permalink] [raw]

Subject: Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

On Mar 26, 2006, at 10:38:59, Martin Mares wrote:
>> It _is_ fragile, but for a number of POSIX-defined structs that's
>> actually the only way to do it without duplicating the data
>> structure in entirety, unless the GCC people can implement a
>> "typedef struct foo struct bar;"
>
> Actually, something like that can be achieved using anonymous
> structure members:
>
> struct xxx {
> struct yyy;
> };

Oh, if only that worked. Actually, what happens is the "struct yyy;"
declaration inside of struct xxx looks just like "struct yyy;" out in
the middle of some random header file. It predeclares the existence
of a struct yyy and does nothing else.

For instance, the following sample program:
struct foo {
int a;
int b;
};

struct bar {
struct foo;
};

int main()
{
struct foo myfoo = { .a = 1, .b = 2 };
struct bar mybar = { .a = 1, .b = 2 };
return 0;
}

Compiled like this:
gcc mytest.c -o mytest

Generates these errors:
mytest.c:7: warning: declaration does not declare anything
mytest.c: In function `main':
mytest.c:12: error: unknown field `a' specified in initializer
mytest.c:12: warning: excess elements in struct initializer
mytest.c:12: warning: (near initialization for `mybar')
mytest.c:12: error: unknown field `b' specified in initializer
mytest.c:12: warning: excess elements in struct initializer
mytest.c:12: warning: (near initialization for `mybar')

Cheers,
Kyle Moffett

2006-03-26 17:24:55

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

Kyle Moffett wrote:
> On Mar 26, 2006, at 08:29:49, Avi Kivity wrote:
>> Kyle Moffett wrote:
>>> Well I guess you could call it UABI, but that might also imply that
>>> it's _userspace_ that defines the interface, instead of the kernel.
>>> Since the headers themselves are rather tightly coupled with the
>>> kernel, I think I'll stick with the KABI name for now (unless
>>> somebody can come up with a better one, of course :-D).
>>
>> How about __linux, or __linux_abi? There are ABIs for other
>> components, and other OSes. Linux is the name of the project after all.
>
> The other thing that I quickly noticed while writing up the patches is
> that it's kind of tedious typing __kabi_ over and over again. I
> actually did first try with __linux_abi_ but the typing effort and
> finger cramps made me give up on that really quickly.
#define _LA(x) __linux_abi_##x

struct _LA(whatever) {
int foo;
int bar;
};

struct _LA(another) {
...
};

#undef _LA

?

--
error compiling committee.c: too many arguments to function

2006-03-26 17:31:13

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Sun, 2006-03-26 at 19:24 +0200, Avi Kivity wrote:
> Kyle Moffett wrote:
> > On Mar 26, 2006, at 08:29:49, Avi Kivity wrote:
> >> Kyle Moffett wrote:
> >>> Well I guess you could call it UABI, but that might also imply that
> >>> it's _userspace_ that defines the interface, instead of the kernel.
> >>> Since the headers themselves are rather tightly coupled with the
> >>> kernel, I think I'll stick with the KABI name for now (unless
> >>> somebody can come up with a better one, of course :-D).
> >>
> >> How about __linux, or __linux_abi? There are ABIs for other
> >> components, and other OSes. Linux is the name of the project after all.
> >
> > The other thing that I quickly noticed while writing up the patches is
> > that it's kind of tedious typing __kabi_ over and over again. I
> > actually did first try with __linux_abi_ but the typing effort and
> > finger cramps made me give up on that really quickly.
> #define _LA(x) __linux_abi_##x
>
> struct _LA(whatever) {
> int foo;
> int bar;
> };
>
> struct _LA(another) {
> ...
> };

this is a good sign that this is all very over designed :)

namespace pollution is perhaps evil, but we also should not overreact.
Especially for struct names. *IF* they are in a "narrow enough" header,
the user of the header knows what he is doing, and accepts these to be
in his namespace.

The problem is things like u64 etc that is VERY common in all headers,
but then again __u64 etc are just fine, history has proven that already.

2006-03-26 17:57:46

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

Arjan van de Ven wrote:
>> struct _LA(whatever) {
>> int foo;
>> int bar;
>> };
>>
>> struct _LA(another) {
>> ...
>> };
>>
>
> this is a good sign that this is all very over designed :)
>
>
It's an eyesore, isn't it? :)
> namespace pollution is perhaps evil, but we also should not overreact.
> Especially for struct names. *IF* they are in a "narrow enough" header,
> the user of the header knows what he is doing, and accepts these to be
> in his namespace.
>
This is true for a small enough application. But things grow, libraries
are added, and includes keep pulling other includes in. Sooner or later
you'll have a collision.
> The problem is things like u64 etc that is VERY common in all headers,
> but then again __u64 etc are just fine, history has proven that already.
>
Agree. But to be on the safe side one can use uint64_t and friends
(which the kernel can typedef to u64 and first degree relatives)

--
error compiling committee.c: too many arguments to function

2006-03-26 18:33:35

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

\
> This is true for a small enough application. But things grow, libraries
> are added, and includes keep pulling other includes in. Sooner or later
> you'll have a collision.
> > The problem is things like u64 etc that is VERY common in all headers,
> > but then again __u64 etc are just fine, history has proven that already.
> >
> Agree. But to be on the safe side one can use uint64_t and friends
> (which the kernel can typedef to u64 and first degree relatives)

actually uint64_t is a bigger namespace pollution than __u64 ;)

2006-03-26 20:06:43

by Sam Ravnborg

[permalink] [raw]

Subject: Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

On Sun, Mar 26, 2006 at 06:54:16AM -0500, Kyle Moffett wrote:
> Create initial kernel ABI header infrastructure
>
>
> diff --git a/Makefile b/Makefile
> index af6210d..8e9045a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -787,13 +787,15 @@ ifneq ($(KBUILD_SRC),)
> /bin/false; \
> fi;
> $(Q)if [ ! -d include2 ]; then mkdir -p include2; fi;
> + $(Q)if [ ! -d include2/kabi ]; then mkdir -p include2/kabi; fi;
> $(Q)ln -fsn $(srctree)/include/asm-$(ARCH) include2/asm
> + $(Q)ln -fsn $(srctree)/include/kabi/arch-$(ARCH) include2/kabi/arch

No - we do not want another symlink.
Create something like:
include/i386/kabi-asm/ <= i386 specific files
include/kabi/ <= general files

Then we can do:
LINUXINCLUDE += -Iinclude/kabi-$(ARCH)
And the following would work like expected:
#include <kabi/foo.h>
#include <kabi-asm/foo.h>

But this leaves all existing users in the dark cold.
So a more involved approach could be to tkae the opposite approach.
To dedicate an area for kernel only header files and make sure this
directory is searched _before_ include/

We could do something like
kinclude/linux/ <= generic kernel include headers
kinclude/$(arch)/asm/ <= arch specific include headers

Then adding to the top-level Makefile:
LINUXINCLUDE := -Ikinclude -Ikinclude/$(ARCH)
LINUXINCLUDE += ...

would actually cut it.
Then we would not hurt existing users since they continue to use
include/linux/* + include/asm/*
And we could migrate one by one to the kernel clean part.

In other words - a solution that keeps backwards compatibility.
A solution that distingush hard between what is the ABI and
what is kernel stuff.
And a namespace that is not in use today.

And we have so many users of include/linux today. They do not need
a _kabi_ prefix so let it go.

Sam

2006-03-26 20:39:55

[permalink] [raw]

Subject: Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

On Mar 26, 2006, at 15:05:37, Sam Ravnborg wrote:
> On Sun, Mar 26, 2006 at 06:54:16AM -0500, Kyle Moffett wrote:
>> Create initial kernel ABI header infrastructure
>>
>> diff --git a/Makefile b/Makefile
>> index af6210d..8e9045a 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -787,13 +787,15 @@ ifneq ($(KBUILD_SRC),)
>> /bin/false; \
>> fi;
>> $(Q)if [ ! -d include2 ]; then mkdir -p include2; fi;
>> + $(Q)if [ ! -d include2/kabi ]; then mkdir -p include2/kabi; fi;
>> $(Q)ln -fsn $(srctree)/include/asm-$(ARCH) include2/asm
>> + $(Q)ln -fsn $(srctree)/include/kabi/arch-$(ARCH) include2/kabi/arch
>
> No - we do not want another symlink.
> Create something like:
> include/i386/kabi-asm/ <= i386 specific files
> include/kabi/ <= general files
>
> Then we can do:
> LINUXINCLUDE += -Iinclude/kabi-$(ARCH)
> And the following would work like expected:
> #include <kabi/foo.h>
> #include <kabi-asm/foo.h>

Hmm, I see where you're going with this. Thanks for the tip!

> But this leaves all existing users in the dark cold. So a more
> involved approach could be to take the opposite approach. To
> dedicate an area for kernel only header files and make sure this
> directory is searched _before_ include/

I'm not entirely sure I understand this bit. The idea behind this
kabi stuff is precisely to split out portions of the headers so that
both userspace and kernelspace can get at them; to designate specific
items as "userspace clean" by putting them in kabi; everything else
need not care at all, and all those headers would remain in include/
linux where they are now. No sense moving _everything_ in include/
around, we just want the parts that userspace needs too.

> And we have so many users of include/linux today. They do not need
> a _kabi_ prefix so let it go.

_Exactly_ the point. I'd like to leave 99% of the kernel code
alone. Userspace doesn't work with the headers in the stock kernel
as-is, so I see no problem with working to provide a clean source ABI
even if it's partially incompatible with the old source ABI. The
binary ABI would stay the same, though, and I'd try to fix the old
source ABI where possible.

I think I'll try something like your original proposal:
include/kabi/ <= Userspace-clean generic ABI includes
include/$(ARCH)/ <= New arch-specific include directory
include/$(ARCH)/kabi-arch/ <= Arch-specific ABI includes

Then the following Makefile snippet:
LINUXINCLUDE += -Iinclude/$(ARCH)

And then in the header file:
#include <kabi-arch/stddef.h>
#include <kabi/types.h>

It would also be possible to move files from include/asm-$(ARCH) to
include/$(ARCH)/asm if desired, but that's a little offtopic for the
kernel-ABI headers project I'm working on right now.

Thanks for the advice!
Kyle Moffett

2006-03-26 20:59:45

[permalink] [raw]

Subject: Re: State of userland headers

On Sunday 26 March 2006 8:12 am, Mariusz Mazur wrote:
> On Friday 24 March 2006 00:04, Rob Landley wrote:
> > You also don't want to run a libc built with newer headers than the
> > kernel you're running on, or it'll try to use stuff that isn't there.
> >
> > You're saying that the new kernel headers wouldn't be versioned using the
> > kernel's release numbers. How do we know what kernel version their
> > feature set matches then? (I'm confused. This happens easily...)
>
> That's a tradeoff. You either version the headers just like I did, meaning
> that a given version corresponds to a given kernel, but that means you
> can't release before all of the archs are fully updated (and not relying on
> a single person to do all of the updates is one of the points of the
> exercise; and with more people, one can have delays) or you're forced to
> figure out some other way to version the headers.

If somebody #includes <linux/version.h> from the new headers, and can
determine what kernel version it goes with for their platform from that (even
if the different platforms are out of sync), and said information also goes
in a README or release notes or some such, I'll live.

Rob
--
Never bet against the cheap plastic solution.

2006-03-26 20:56:20

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Sunday 26 March 2006 7:34 am, Kyle Moffett wrote:
> Well I guess you could call it UABI, but that might also imply that
> it's _userspace_ that defines the interface, instead of the kernel.
> Since the headers themselves are rather tightly coupled with the
> kernel, I think I'll stick with the KABI name for now (unless
> somebody can come up with a better one, of course :-D).

Ok, question, using an example I'm familiar with. What's affected by the
prefix?

The busybox code for associating loopback devices with files was a problem for
_years_. The standard advice was not to include linux/loop.h, but instead
cut and paste the structure out of the header into your own code.
(Apparently, you're allowed to be uncomfortable if somebody suggests that
this is the right way to get portability between bsd/cygwin/linux, but
between 2.4 and 2.6 it's best practices. Ok.)

Unfortunately, the cut and paste approach turned out to be incredibly painful
because busybox needs to be portable between hardware platforms and between
kernel versions.

Problem 1: The actual type of __kernel_dev_t varies between
x86/arm/alpha/ppc/etc. Any attempt to define our own loop ioctl structure
meant we needed to either use the kernel-defined data type for this, or have
an #ifdef forest for every possible platform somebody might want to build
busybox on. (This was exploded into a mess instantly.)

But using the kernel kdev_t type sucked too, because between 2.4 and 2.6 it
got renamed to old_kdev_t so then we needed #ifdefs to check the kernel
version of the headers we'd imported to see which name it was using. (And it
still broke anyway in strange various corner cases depending on which distro
people were building under.)

We wandered through dozens of different approaches trying to avoid "#include
<linux/loop.h>". We had #ifdefs for glibc and #ifdefs for uclibc... It was
terrible. About half the suggestions got checked into the tree at various
points:
http://www.busybox.net/cgi-bin/viewcvs.cgi/trunk/busybox/libbb/loop.c?rev=14554&view=log

There were more that merely showed up on the list...

We eventually got it all untangled. Now we check if we're on 2.6 and if so
use the 64 bit API (which is stable on 2.6), and otherwise use a copy of the
32 bit structure that predates __kernel_dev_t being renamed. And we trust
the darn kernel headers to supply the types for us, because there's just no
alternative.

My question is: will your new approach work with our existing code, or are you
saying we need another #ifdef to check for yet another gratuitously
incompatible name, ala _kabi__kernel_dev_t? Because I _really_ don't want to
go there. Nobody will be able to build existing versions of busybox (at
least the mount/umount/losetup commands) against your new thing. New
versions that actually depended on _kabi_blah wouldn't build on any older
distros unless we add another #ifdef...

This is just an example. There is currently no existing program out in
userspace that is tries to talk to the kernel using names with _kabi_
prepended to them, that I am aware of. If you're adding this prefix, you're
writing a brand new interface which currently has exactly zero users. Is
this the intent?

> Cheers,
> Kyle Moffett

Rob
--
Never bet against the cheap plastic solution.

2006-03-26 21:09:26

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Sunday 26 March 2006 9:31 am, Eric Piel wrote:
> I completely agree with rules 1, 2 and 5. However, IMHO rule 4 should
> just be the inverse of rule 5: The stuff in include/linux should always
> be independent from KABI (and userspace of course). Simply because the
> way we _implement_ things in the kernel has to be different from the
> things that we _specify_ in the kernel ABI.

You know all the stuff that's marked __user? It's all kernel ABI. Having it
defined in two places invites version skew, and the kernel needs it too
because the kernel is parsing the stuff sent in by userspace and filling it
out to send back to userspace.

Lots of syscalls and ioctls and such pass a structure back and forth from
userspace to the kernel and back, right? Doing a stat() fills out a
structure, doing an losetup fills out a structure, and so on.

Userspace needs to know what's in this structure. It may be wrapped in a libc
function that fills out a different structure from the kernel structure, but
the data that goes back and forth between the program and the kernel has to
be defined in a header somewhere so the libc knows what the kernel's sending
and the kernel knows what the libc is sending. (And for those functions with
no libc wrapper, the user program needs to know the structure directly,
somehow.)

Having a data-marshalling ABI structure defined in two places invites version
skew. Userspace needs access to this (at least to build a libc), and the
kernel needs access to this, because it's a _communication_mechanism_. You
can't have a communication mechanism that's only defined at one end.

> As for rule 3, if you have independent headers, this should be much less
> necessary. Additionally, keeping all the names identical to what they
> are already called will allow userspace to just use include/kabi/ as the
> /usr/include/linux/ directory. Avoiding smelly things like:
>
> linux/foo.h:
> #define __kabi_foo foo
> #include <kabi/foo.h>

The #define is, indeed, smelly.

Rob
--
Never bet against the cheap plastic solution.

2006-03-26 21:18:56

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Sunday 26 March 2006 12:57 pm, Avi Kivity wrote:
> This is true for a small enough application. But things grow, libraries
> are added, and includes keep pulling other includes in. Sooner or later
> you'll have a collision.

And you'll fix it when it happens. Fact of life.

> > The problem is things like u64 etc that is VERY common in all headers,
> > but then again __u64 etc are just fine, history has proven that already.
>
> Agree. But to be on the safe side one can use uint64_t and friends
> (which the kernel can typedef to u64 and first degree relatives)

Now that the kernel no longer targets gcc before 3.2, c99 names are merely
ugly rather than an actual problem. :)

Rob
--
Never bet against the cheap plastic solution.

2006-03-26 21:26:47

by Sam Ravnborg

[permalink] [raw]

Subject: Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

On Sun, Mar 26, 2006 at 03:39:32PM -0500, Kyle Moffett wrote:
> On Mar 26, 2006, at 15:05:37, Sam Ravnborg wrote:
> >On Sun, Mar 26, 2006 at 06:54:16AM -0500, Kyle Moffett wrote:
> >>Create initial kernel ABI header infrastructure
> >>
> >>diff --git a/Makefile b/Makefile
> >>index af6210d..8e9045a 100644
> >>--- a/Makefile
> >>+++ b/Makefile
> >>@@ -787,13 +787,15 @@ ifneq ($(KBUILD_SRC),)
> >> /bin/false; \
> >> fi;
> >> $(Q)if [ ! -d include2 ]; then mkdir -p include2; fi;
> >>+ $(Q)if [ ! -d include2/kabi ]; then mkdir -p include2/kabi; fi;
> >> $(Q)ln -fsn $(srctree)/include/asm-$(ARCH) include2/asm
> >>+ $(Q)ln -fsn $(srctree)/include/kabi/arch-$(ARCH) include2/kabi/arch
> >
> >No - we do not want another symlink.
> >Create something like:
> >include/i386/kabi-asm/ <= i386 specific files
> >include/kabi/ <= general files
> >
> >Then we can do:
> >LINUXINCLUDE += -Iinclude/kabi-$(ARCH)
> >And the following would work like expected:
> >#include <kabi/foo.h>
> >#include <kabi-asm/foo.h>
>
> Hmm, I see where you're going with this. Thanks for the tip!
>
> >But this leaves all existing users in the dark cold. So a more
> >involved approach could be to take the opposite approach. To
> >dedicate an area for kernel only header files and make sure this
> >directory is searched _before_ include/
>
> I'm not entirely sure I understand this bit. The idea behind this
> kabi stuff is precisely to split out portions of the headers so that
> both userspace and kernelspace can get at them; to designate specific
> items as "userspace clean" by putting them in kabi; everything else
> need not care at all, and all those headers would remain in include/
> linux where they are now. No sense moving _everything_ in include/
> around, we just want the parts that userspace needs too.
>

There are today a great number of users of the existing kernel headers.
Introducing a new namespace for the userspace suers will leave them in a
dilemma where they have to support kernels before kabi, and kernels with
kabi. That alone will limit the acceptance of this.

So shifting to another approach where you take everything that we have
today and clean it up step by step will allow us in small steps to go
from current mess to a clean ernel ABI.
But we will not have this clean and virgin set of headers until a few
years ahead of us.

Keeping include/linux for the kernel ABI will allow us NOT to break
existing users and it will allow a stepwise apporach at the same time.
But what we make clean and virgin is the kernel specific stuff, not the
userspace specific stuff. In final result will be the same but shifting
focus to the kernel internal stuff allows us to do ti in incremental
steps that does NOT break userspace every and so often.
And we will only break external modules that does not use kbuild to
compile - and for hese I really do not care a whit.

Keeping the breakage minimal is the key point here.

Sam

2006-03-26 23:07:16

by Éric Piel

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

26.03.2006 23:09, Rob Landley wrote/a ?crit:
> On Sunday 26 March 2006 9:31 am, Eric Piel wrote:
>> I completely agree with rules 1, 2 and 5. However, IMHO rule 4 should
>> just be the inverse of rule 5: The stuff in include/linux should always
>> be independent from KABI (and userspace of course). Simply because the
>> way we _implement_ things in the kernel has to be different from the
>> things that we _specify_ in the kernel ABI.
>
> You know all the stuff that's marked __user? It's all kernel ABI. Having it
> defined in two places invites version skew, and the kernel needs it too
> because the kernel is parsing the stuff sent in by userspace and filling it
> out to send back to userspace.
>
> Lots of syscalls and ioctls and such pass a structure back and forth from
> userspace to the kernel and back, right? Doing a stat() fills out a
> structure, doing an losetup fills out a structure, and so on.
>
> Userspace needs to know what's in this structure. It may be wrapped in a libc
> function that fills out a different structure from the kernel structure, but
> the data that goes back and forth between the program and the kernel has to
> be defined in a header somewhere so the libc knows what the kernel's sending
> and the kernel knows what the libc is sending. (And for those functions with
> no libc wrapper, the user program needs to know the structure directly,
> somehow.)
>
> Having a data-marshalling ABI structure defined in two places invites version
> skew. Userspace needs access to this (at least to build a libc), and the
> kernel needs access to this, because it's a _communication_mechanism_. You
> can't have a communication mechanism that's only defined at one end.
Well, that's half true. Indeed, in general, having two separate
definitions invites version skew. However, in this particular case, it's
slightly different: because the principle of the ABI is to be stable, or
more exactly _compatible_. This means that if one definition was right
at some point in the time, it should always still be true ten years
later. At worse, the ABI can be extended, but never changed. If the
specification (KABI) and the implementation (Linux) are not compatible
it means the kernel developers screwed up, not that the KABI maintainers
haven't updated in time.

Of course, next to the theory, there is the reality. Some part of the
ABI has already be changed (broken) in the past (like the alsa ABI,
IIRC). In such case the KABI maintainers will have to handle the changes
as promptly as possible, but the responsibility will be held by the
kernel developer who has opted for breakage.

The real problem of sharing the same headers between kernel and KABI is
that it will end up by having to re-implement the "#ifdef __KERNEL__"'s.
Have a look at Kyle's second patch "Generalize fd_set handling across
architectures". Some headers had a different version of the __FD_*()
macros depending on the compiler. That's something you may want to have
in the implementation but definitely not in the specification. In this
situation, Kyle handled it nicely by writing versions compatible with
any compiler. Good looking solution inside the kernel will not always be
an option. IMHO, if KABI and the kernel share the same headers, it's
just a matter of time before someone introduces an "#ifdef
__KERNEL__"-alike mechanism, exactly what we've been trying to remove.

Eric

2006-03-27 00:13:15

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Mar 26, 2006, at 15:55:51, Rob Landley wrote:
> On Sunday 26 March 2006 7:34 am, Kyle Moffett wrote:
>> Well I guess you could call it UABI, but that might also imply
>> that it's _userspace_ that defines the interface, instead of the
>> kernel. Since the headers themselves are rather tightly coupled
>> with the kernel, I think I'll stick with the KABI name for now
>> (unless somebody can come up with a better one, of course :-D).
>
> Ok, question, using an example I'm familiar with. What's affected
> by the prefix?
>
> We eventually got it all untangled. Now we check if we're on 2.6
> and if so use the 64 bit API (which is stable on 2.6), and
> otherwise use a copy of the 32 bit structure that predates
> __kernel_dev_t being renamed. And we trust the darn kernel headers
> to supply the types for us, because there's just no alternative.
>
> My question is: will your new approach work with our existing code,
> or are you saying we need another #ifdef to check for yet another
> gratuitously incompatible name, ala _kabi__kernel_dev_t?

No, no need for a new #ifdef, at least not for a long time if you
don't want to. My plans for the immediate future are only to touch
files that are currently broken with respect to !__KERNEL__, and
linux/loop.h isn't. When I do get to it, my plan is to bring the
parts that userspace needs into kabi/loop.h, named as __kabi_*,
however linux/loop.h will define the new symbols to their old names
for compatibility with linux code and userspace code that happens to
be using those interfaces, so you still won't need to change
anything. Hopefully this would also mean that if the shipped linux/
loop.h is currently broken for your uses, I would be able to fix it
so you don't need a customized copy at all.

If you _did_ decide to switch busybox to use the __kabi_ symbols, you
would be able to take advantage of the fact that the same set of kabi
headers (which don't include kernelversion.h) would work unmodified
for kernels all the way back to 0.99, as long as you base your
decisions of what features are available on either a dynamically-
detected version or a pre-configured minimum version from the user.
I'm assuming here that it's possible to compile a busybox that works
on a 2.2 or 2.4 kernel even using linux-libc-headers 2.6.*, as long
as the user indicates such.

> This is just an example. There is currently no existing program
> out in userspace that is tries to talk to the kernel using names
> with _kabi_ prepended to them, that I am aware of. If you're
> adding this prefix, you're writing a brand new interface which
> currently has exactly zero users. Is this the intent?

I'd like this project to have a twofold purpose. First, in the short
term I want to fix up the general __KERNEL__ vs !__KERNEL__
brokenness in include/linux and clean up and document which APIs are
userspace ABI by moving them to include/kabi. If that first part
works, then in the very _long_ term, (on the order of a couple years
or so) I would like to encourage larger projects like klibc and GLIBC
to migrate to using *only* the include/kabi headers to avoid
namespace pollution and allow kernel devs to rearrange and adjust the
linux/* headers as they see fit without caring about breaking
external programs.

Once the libcs are using kabi/*, GLIBC can pick up a set of linux/*.h
"emulation" files based on kabi/*.h that provide all of the old
source-APIs for as long as anybody cares, and Linux can go back to
screwing around with its header files in any way that it pleases
(just as long as it keeps ABI stuff all in kabi). One of my major
goals for this rework is to ensure that the kabi/*.h header files
actually define the _ABI_ going back to the very early releases of
Linux. As Linus said, it's an ABI, which means that adding new
features is OK, but removing them is bad, so I'll try to make sure
it's _perfectly_ _ok_ to use a 2.6 kabi/ on a 2.0 kernel. It should
also be perfectly ok to use a 2.6 kabi/ on a 5-years-down-the-road
2.8 kernel as long as you restrict yourself to the subset of features
that were available in 2.6.

Cheers,
Kyle Moffett

2006-03-27 00:19:22

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Mar 26, 2006, at 16:18:29, Rob Landley wrote:
>>> The problem is things like u64 etc that is VERY common in all
>>> headers, but then again __u64 etc are just fine, history has
>>> proven that already.
>>
>> Agree. But to be on the safe side one can use uint64_t and friends
>> (which the kernel can typedef to u64 and first degree relatives)
>
> Now that the kernel no longer targets gcc before 3.2, c99 names are
> merely ugly rather than an actual problem. :)

No, they're still a problem in ABI headers because they have defined
visibility. A userspace program that does not include <stdint.h>
will not have those types put in its namespace. Including <sys/
time.h> _cannot_ bring <stdint.> in automatically, that breaks the
POSIX symbol visibility rules.

Cheers,
Kyle Moffett

2006-03-27 00:27:58

[permalink] [raw]

Subject: Re: [RFC][PATCH 1/2] Create initial kernel ABI header infrastructure

On Mar 26, 2006, at 16:26:01, Sam Ravnborg wrote:
> On Sun, Mar 26, 2006 at 03:39:32PM -0500, Kyle Moffett wrote:
>> I'm not entirely sure I understand this bit. The idea behind this
>> kabi stuff is precisely to split out portions of the headers so
>> that both userspace and kernelspace can get at them; to designate
>> specific items as "userspace clean" by putting them in kabi;
>> everything else need not care at all, and all those headers would
>> remain in include/ linux where they are now. No sense moving
>> _everything_ in include/ around, we just want the parts that
>> userspace needs too.
>
> There are today a great number of users of the existing kernel
> headers. Introducing a new namespace for the userspace suers will
> leave them in a dilemma where they have to support kernels before
> kabi, and kernels with kabi. That alone will limit the acceptance
> of this.

I tried to talk to this in my other email, but for emphasis; I'd like
to do two things:
1) Preserve compatibility with <linux/*.h>, so that old programs
still work fine
2) Cause <kabi/*.h> to work on any distro, and kernel, not just
kernel released after it is introduced. In practice this means that
a distro could just package up the kabi headers and install them in /
usr/include. A package that is altered to compile against only
__kabi_* would be able to use information about which syscalls and
structures are available in each version to either handle different
kernels at runtime or have a configure-time kernel version specified
by the user.

> Keeping include/linux for the kernel ABI will allow us NOT to break
> existing users and it will allow a stepwise apporach at the same time.

I completely agree. If you'll notice the patches I submitted were
quite careful to preserve the exact same semantics for the linux/*.h
headers, both in the interests of not breaking the kernel _and_ in
the interests of allowing userspace to still include <linux/
types.h>. IMHO, we should try to get the interface compatibility
crap _out_ of linux source tree and into a separate package which
depends only on the cleaned-up interface specification (IE: kabi/*).

Cheers,
Kyle Moffett

2006-03-27 00:40:34

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Mar 26, 2006, at 18:06:48, Eric Piel wrote:
> The real problem of sharing the same headers between kernel and
> KABI is that it will end up by having to re-implement the "#ifdef
> __KERNEL__"'s. Have a look at Kyle's second patch "Generalize
> fd_set handling across architectures". Some headers had a different
> version of the __FD_*() macros depending on the compiler. That's
> something you may want to have in the implementation but definitely
> not in the specification.

Actually, I think it's the other way around. The <kabi/*.h> files
should (eventually) be useable in basically any compilation
environment thrown at it. This means it should work from C and C++,
using GCC, ICC, or some custom barely-standards-compliant compiler.
I didn't bother with that part right now, since I still want to try
to reuse the generic bitops if possible, but it's something I plan to
address in future versions of the patchset (see below).

> In this situation, Kyle handled it nicely by writing versions
> compatible with any compiler.

Eh, not really. "__inline__" is GCC-specific and probably won't work
in other compilers (unless you did "#define __inline__", which would
bloat the code a lot).

This case highlights something else I'd like to do. A good chunk of
the functionality in the Linux kernel works both in userspace and
kernelspace, and some of those arch-specific primitives (like the
inline bitops) would be useful in defining the kabi headers.
According to Jeff Dike, UML would like access to some of that stuff
unrestricted by __KERNEL__ too. In all of those cases, it's not an
ABI and all the users are in-kernel so it could be changed at will.
I'd like to try to put some of that into a "klib" directory (though
with dependencies crossing between kabi and klib) so that it could be
used in kabi and UML without duplicating functionality. Naturally
much of that would be C-only and depend on GCC, but I would have to
be careful that the kabi portions used least-common-denominator
functionality.

That brings up one final point: Does anybody actually use any
compilers on Linux other than GCC?

Cheers,
Kyle Moffett

2006-03-27 03:12:23

by Jeff Dike

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Sun, Mar 26, 2006 at 07:40:14PM -0500, Kyle Moffett wrote:
> According to Jeff Dike, UML would like access to some of that stuff
> unrestricted by __KERNEL__ too.

Not quite true - I just want them physically separated from the
non-userspace-usable stuff, as in different files. I don't care
whether they are inside #ifdef __KERNEL__, because that's defined in
the UML build.

Jeff

2006-03-27 06:19:24

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

Rob Landley wrote:
> On Sunday 26 March 2006 12:57 pm, Avi Kivity wrote:
>
>> This is true for a small enough application. But things grow, libraries
>> are added, and includes keep pulling other includes in. Sooner or later
>> you'll have a collision.
>>
>
> And you'll fix it when it happens. Fact of life.
>
Fixing it will mean breaking either the ABI of the kernel or of the
large library you pulled in.

An ABI bug causes pain far beyond its size. Look at the trouble caused
when some interfaces uses unsigned instead of u64. In kernel APIs, you
just replace the type, but in the ABI, you add a new syscall or do some
other hack.

Much better to get it right the first time, even if it's ugly. It's an
ABI, not a beauty contest nominee.

--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

2006-03-27 19:49:24

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Monday 27 March 2006 1:19 am, Avi Kivity wrote:
> Rob Landley wrote:
> > On Sunday 26 March 2006 12:57 pm, Avi Kivity wrote:
> >> This is true for a small enough application. But things grow, libraries
> >> are added, and includes keep pulling other includes in. Sooner or later
> >> you'll have a collision.
> >
> > And you'll fix it when it happens. Fact of life.
>
> Fixing it will mean breaking either the ABI of the kernel or of the
> large library you pulled in.
>
> An ABI bug causes pain far beyond its size. Look at the trouble caused
> when some interfaces uses unsigned instead of u64. In kernel APIs, you
> just replace the type, but in the ABI, you add a new syscall or do some
> other hack.

One of busybox's users has a libc built with improperly sanitized kernel
headers that leak all the kernel's CONFIG_ symbols into the standard
namespace, and we found out because one of them clashes with a busybox CONFIG
symbol and it broke his build.

The easiest thing to do was rename our CONFIG symbol, which we did. (And
asked him to fix his system, which he didn't.) This was a while ago now...

> Much better to get it right the first time, even if it's ugly. It's an
> ABI, not a beauty contest nominee.

I've been cc'd on all this because I cared about Mazur's kernel headers enough
to email him several times. But I can't say I really care about this new
project that's taken over the discussion.

Either it doesn't break the existing interface, in which case it has no impact
on me and I can ignore it, or it requires all existing userspace programs
that care about the current state of things to change to care about your new
thing instead, in which case it's probably doomed. Unless the kernel picks
it up and imposes a flag day, at which point I'll find out about it from LWN
like everybody else.

Either way, it's not sounding like something I can grab and build uClibc
systems with any time soon, the way I could use Mazur's headers to build
uClibc. I'll probably wind up using the gentoo headers when the 2.6.14
version ships.

http://www.gentoo.org/cgi-bin/viewcvs.cgi/src/patchsets/gentoo-headers/?root=gentoo

*shrug*. Good luck with whatever it is you're trying to accomplish.

Rob
--
Never bet against the cheap plastic solution.

2006-03-28 14:20:24

by Jan Engelhardt

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

>
> Eh, not really. "__inline__" is GCC-specific and probably won't work in other
> compilers (unless you did "#define __inline__", which would bloat the code a
> lot).
>
But ___inline is a C99 keyword, isnot it?

Jan Engelhardt
--

2006-03-28 15:57:50

[permalink] [raw]

Subject: [OT] Non-GCC compilers used for linux userspace

On Mar 28, 2006, at 09:20:09, Jan Engelhardt wrote:
>> Eh, not really. "__inline__" is GCC-specific and probably won't
>> work in other compilers (unless you did "#define __inline__",
>> which would bloat the code a lot).
>
> But ___inline is a C99 keyword, is not it?

Not even GCC fully supports C99 (although I think it does support
that keyword when passed -std=c99 or -std=gnu99), and I suspect that
a majority of the other compilers for which we would want to add
support in the kernel headers would not support C99 or would do a
poor job of handling inline functions.

But my question still stands. Does anybody actually use any non-GCC
compiler for userspace in Linux?

Cheers,
Kyle Moffett

2006-03-28 16:13:50

by Éric Piel

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

03/28/2006 05:57 PM, Kyle Moffett wrote/a écrit:
>
> But my question still stands. Does anybody actually use any non-GCC
> compiler for userspace in Linux?
At least in the domain of HPC, I've seen people which were compiling
mostly *everything* with the intel compiler (x86 and ia64) for
performance reason. So... yes userspace is sometimes compiled with
non-GCC compiler :-)

Eric

2006-03-28 16:21:23

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On Mar 28, 2006, at 11:13:42, Eric Piel wrote:
> 03/28/2006 05:57 PM, Kyle Moffett wrote/a ?crit:
>> But my question still stands. Does anybody actually use any non-
>> GCC compiler for userspace in Linux?
>
> At least in the domain of HPC, I've seen people which were
> compiling mostly *everything* with the intel compiler (x86 and
> ia64) for performance reason. So... yes userspace is sometimes
> compiled with non-GCC compiler :-)

Ok, well, the Intel compiler actually ends up emulating GCC and
supports most of its extensions; supposedly it can even be used to
compile the kernel sources, as per include/linux/compiler-intel.h. (I
don't know how recently this has been tested, though) So does
anybody compile userspace under anything other than GCC or Intel
compilers? Do any such compilers even exist?

Cheers,
Kyle Moffett

2006-03-28 17:01:59

by Jason L Tibbitts III

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

>>>>> "KM" == Kyle Moffett <[email protected]> writes:

KM> So does anybody compile userspace under anything other than GCC or
KM> Intel compilers? Do any such compilers even exist?

PGI and PathScale are around. Lahey, too, although they seem to just
do Fortran now.

I doubt you'd want to worry about compiling the entire userland with
these compilers, however.

- J<

2006-03-28 17:08:51

by Catalin Marinas

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

Kyle Moffett <[email protected]> wrote:
> That brings up one final point: Does anybody actually use any
> compilers on Linux other than GCC?

Yes, some people use (or plan to use) the ARM Ltd toolchain
(http://www.arm.com/products/DevTools/RVCT.html) to cross-compile
applications for Linux (mainly for a better size/performance).

--
Catalin

2006-03-28 17:08:44

by Jan-Benedict Glaw

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On Tue, 2006-03-28 11:20:50 -0500, Kyle Moffett <[email protected]> wrote:
> don't know how recently this has been tested, though) So does
> anybody compile userspace under anything other than GCC or Intel
> compilers? Do any such compilers even exist?

tcc

MfG, JBG

--
Jan-Benedict Glaw [email protected] . +49-172-7608481 _ O _
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg _ _ O
für einen Freien Staat voll Freier Bürger" | im Internet! | im Irak! O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

Attachments:

(No filename) (602.00 B)
signature.asc (189.00 B)
Digital signature Download all attachments

2006-03-28 17:13:24

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On Mar 28, 2006, at 11:59:13, Jason L Tibbitts III wrote:
>>>>>> "KM" == Kyle Moffett <[email protected]> writes:
>> So does anybody compile userspace under anything other than GCC or
>> Intel compilers? Do any such compilers even exist?
>
> PGI and PathScale are around. Lahey, too, although they seem to
> just do Fortran now.
>
> I doubt you'd want to worry about compiling the entire userland
> with these compilers, however.

Mainly I want to know if I should even bother making the kabi headers
compile with anything other than GCC. Judging from the apparently
negligible number of users, it doesn't sound like something I should
spend much or any time on, at least for now.

Cheers,
Kyle Moffett

2006-03-28 17:18:14

by Ben Pfaff

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

Jan Engelhardt <[email protected]> writes:

>> Eh, not really. "__inline__" is GCC-specific and probably won't work in other
>> compilers (unless you did "#define __inline__", which would bloat the code a
>> lot).
>>
> But ___inline is a C99 keyword, isnot it?

___inline is not, but inline is.
--
"Platonically Evil Monkey has been symbolically representing the darkest
fears of humanity since the dawn of literature and religion, and I think
I speak for everyone when I give it a sidelong glance of uneasy recognition
this evening." --Scrymarch

2006-03-28 17:30:10

by Daniel Jacobowitz

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On Tue, Mar 28, 2006 at 12:13:15PM -0500, Kyle Moffett wrote:
> On Mar 28, 2006, at 11:59:13, Jason L Tibbitts III wrote:
> >>>>>>"KM" == Kyle Moffett <[email protected]> writes:
> >>So does anybody compile userspace under anything other than GCC or
> >>Intel compilers? Do any such compilers even exist?
> >
> >PGI and PathScale are around. Lahey, too, although they seem to
> >just do Fortran now.
> >
> >I doubt you'd want to worry about compiling the entire userland
> >with these compilers, however.
>
> Mainly I want to know if I should even bother making the kabi headers
> compile with anything other than GCC. Judging from the apparently
> negligible number of users, it doesn't sound like something I should
> spend much or any time on, at least for now.

I'm not sure how you got to that conclusion. People have already named
several non-GCC compilers that are used; and most of the users of
commercial compilers won't be reading this list.

If you want glibc to ever include these things, they had better be
portable C and work without GCC. Otherwise it's a non-starter.
Only GCC may be used to build glibc, but it deliberately supports any
conforming C compiler to build userspace code.

--
Daniel Jacobowitz
CodeSourcery

2006-03-28 17:41:20

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On Mar 28, 2006, at 12:28:47, Daniel Jacobowitz wrote:
> On Tue, Mar 28, 2006 at 12:13:15PM -0500, Kyle Moffett wrote:
>> On Mar 28, 2006, at 11:59:13, Jason L Tibbitts III wrote:
>>>>>>>> "KM" == Kyle Moffett <[email protected]> writes:
>>>> So does anybody compile userspace under anything other than GCC
>>>> or Intel compilers? Do any such compilers even exist?
>>>
>>> PGI and PathScale are around. Lahey, too, although they seem to
>>> just do Fortran now.
>>>
>>> I doubt you'd want to worry about compiling the entire userland
>>> with these compilers, however.
>>
>> Mainly I want to know if I should even bother making the kabi
>> headers compile with anything other than GCC. Judging from the
>> apparently negligible number of users, it doesn't sound like
>> something I should spend much or any time on, at least for now.
>
> I'm not sure how you got to that conclusion. People have already
> named several non-GCC compilers that are used; and most of the
> users of commercial compilers won't be reading this list.
>
> If you want glibc to ever include these things, they had better be
> portable C and work without GCC. Otherwise it's a non-starter.
> Only GCC may be used to build glibc, but it deliberately supports
> any conforming C compiler to build userspace code.

Ok, my email was a bit premature (I've gotten a couple other private
emails about other compilers too) and if people see this as an issue
then I'll try to make all the code C89-compliant from the start. I
just didn't want to go writing a whole bunch of compatibility macros
only to find out that they never got used.

Cheers,
Kyle Moffett

2006-03-28 17:56:29

by Jesper Juhl

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On 3/28/06, Kyle Moffett <[email protected]> wrote:
> On Mar 28, 2006, at 11:13:42, Eric Piel wrote:
> > 03/28/2006 05:57 PM, Kyle Moffett wrote/a ?crit:
> >> But my question still stands. Does anybody actually use any non-
> >> GCC compiler for userspace in Linux?
> >
> > At least in the domain of HPC, I've seen people which were
> > compiling mostly *everything* with the intel compiler (x86 and
> > ia64) for performance reason. So... yes userspace is sometimes
> > compiled with non-GCC compiler :-)
>
> Ok, well, the Intel compiler actually ends up emulating GCC and
> supports most of its extensions; supposedly it can even be used to
> compile the kernel sources, as per include/linux/compiler-intel.h. (I
> don't know how recently this has been tested, though) So does
> anybody compile userspace under anything other than GCC or Intel
> compilers? Do any such compilers even exist?
>

Other compilers do exist.

Over the years I've personally used a few to compile userspace apps
for different projects (though never for compiling the kernel).

Some of the compilers I have personally used for userspace apps on
Linux include ;

lcc - http://www.cs.princeton.edu/software/lcc/
tcc - http://fabrice.bellard.free.fr/tcc/

(plus gcc & icc ofcourse)

Others that I know of but have never used include :

sdcc - http://sdcc.sourceforge.net/
Compaq C for Linux - http://h30097.www3.hp.com/linux/compaq_c/index.html
Open Watcom - http://www.openwatcom.org/
vacpp - http://www-306.ibm.com/software/awdtools/vacpp/
XL C/C++ - http://www.absoft.com/Products/Compilers/C_C++/Linux/XLC/XLC.html

and I'm sure many more exist...

--
Jesper Juhl <[email protected]>
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please http://www.expita.com/nomime.html

2006-03-28 18:46:23

by Eric W. Biederman

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

Eric Piel <[email protected]> writes:

> 03/28/2006 05:57 PM, Kyle Moffett wrote/a ?crit:
>> But my question still stands. Does anybody actually use any non-GCC compiler
>> for userspace in Linux?
> At least in the domain of HPC, I've seen people which were compiling mostly
> *everything* with the intel compiler (x86 and ia64) for performance
> reason. So... yes userspace is sometimes compiled with non-GCC compiler :-)

The pathscale, and pgi compilers also get a reasonable amount of use.

pathscale has a gcc derived front-end so it isn't to much to worry about.

The pgi compiler as I recall is a fairly pedantic c90 compiler, that
doesn't try and support gcc extensions.

Eric

2006-03-28 20:06:41

by Mariusz Mazur

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Monday 27 March 2006 21:48, Rob Landley wrote:
> Either way, it's not sounding like something I can grab and build uClibc
> systems with any time soon, the way I could use Mazur's headers to build
> uClibc. I'll probably wind up using the gentoo headers when the 2.6.14
> version ships.

That's the trouble. While I have nothing against someone (as in -- not me :)
doing all that abi separation, it's not something I can use straight away.
Hell, I don't even get where in all of it I'd end up with something I could
use (granted, I haven't looked into the thread too closely). Not to mention,
that I suspect, that if there were enough people to do it, it would have
gotten done two years ago.

So unless anybody's got a better idea, I'll try releasing some initial version
of that llh-ng thingie rather soonish and see how that'll work out. Anybody
with me on that? :)

--
Judge others by their intentions and yourself by your results.
Guy Kawasaki
Education is an admirable thing, but it is well to remember from
time to time that nothing that is worth knowing can be taught.
Oscar Wilde

2006-03-28 20:14:28

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Mar 28, 2006, at 15:04:53, Mariusz Mazur wrote:
> On Monday 27 March 2006 21:48, Rob Landley wrote:
>> Either way, it's not sounding like something I can grab and build
>> uClibc systems with any time soon, the way I could use Mazur's
>> headers to build uClibc. I'll probably wind up using the gentoo
>> headers when the 2.6.14 version ships.
>
> That's the trouble. While I have nothing against someone (as in --
> not me :) doing all that abi separation, it's not something I can
> use straight away. Hell, I don't even get where in all of it I'd
> end up with something I could use (granted, I haven't looked into
> the thread too closely). Not to mention, that I suspect, that if
> there were enough people to do it, it would have gotten done two
> years ago.
>
> So unless anybody's got a better idea, I'll try releasing some
> initial version of that llh-ng thingie rather soonish and see how
> that'll work out. Anybody with me on that? :)

I agree with this approach. I don't think the llh-ng method will be
maintainable in the long-term, which is why I'm doing the KABI
patches, but I think that in the short term that's the fastest way to
get a working set of headers against which to build glibc.

Cheers,
Kyle Moffett

2006-03-28 20:18:40

by Jim Gifford

[permalink] [raw]

Subject: Re: State of userland headers

Just to through this out, I'm working on project right now where we want
to use the latest Kernel Headers, but since LLH is pretty much dead we
came up with a interim solution until something else gets worked out.
The script lists below only sanitizes the headers that are needed by
userspace, there may be some missing, but this has worked on the systems
I've built up to this point.

http://ftp.jg555.com/headers/headers2

--
----
Jim Gifford
[email protected]

2006-03-28 21:47:26

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On Tuesday 28 March 2006 11:20 am, Kyle Moffett wrote:
> On Mar 28, 2006, at 11:13:42, Eric Piel wrote:
> > 03/28/2006 05:57 PM, Kyle Moffett wrote/a ?crit:
> >> But my question still stands. Does anybody actually use any non-
> >> GCC compiler for userspace in Linux?
> >
> > At least in the domain of HPC, I've seen people which were
> > compiling mostly *everything* with the intel compiler (x86 and
> > ia64) for performance reason. So... yes userspace is sometimes
> > compiled with non-GCC compiler :-)
>
> Ok, well, the Intel compiler actually ends up emulating GCC and
> supports most of its extensions; supposedly it can even be used to
> compile the kernel sources, as per include/linux/compiler-intel.h. (I
> don't know how recently this has been tested, though) So does
> anybody compile userspace under anything other than GCC or Intel
> compilers? Do any such compilers even exist?
>
> Cheers,
> Kyle Moffett

I play with Fabrice Bellard's Tiny C Compiler (http://www.tinycc.org), and
hope to get a distro compiled with it someday, at least as a proof of
concept.

That aims for full c99 and is already implementing a lot of gcc stuff too.

Rob
--
Never bet against the cheap plastic solution.

2006-03-28 22:58:12

[permalink] [raw]

Subject: Re: [RFC][PATCH 0/2] KABI example conversion and cleanup

On Tuesday 28 March 2006 3:04 pm, Mariusz Mazur wrote:
> On Monday 27 March 2006 21:48, Rob Landley wrote:
> > Either way, it's not sounding like something I can grab and build uClibc
> > systems with any time soon, the way I could use Mazur's headers to build
> > uClibc. I'll probably wind up using the gentoo headers when the 2.6.14
> > version ships.
>
> That's the trouble. While I have nothing against someone (as in -- not me
> :) doing all that abi separation, it's not something I can use straight
> away. Hell, I don't even get where in all of it I'd end up with something I
> could use (granted, I haven't looked into the thread too closely). Not to
> mention, that I suspect, that if there were enough people to do it, it
> would have gotten done two years ago.
>
> So unless anybody's got a better idea, I'll try releasing some initial
> version of that llh-ng thingie rather soonish and see how that'll work out.
> Anybody with me on that? :)

I'm highly interested in seeing the result. :)

Rob
--
Never bet against the cheap plastic solution.

2006-03-29 04:27:48

by Peter Chubb

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

>>>>> "Kyle" == Kyle Moffett <[email protected]> writes:

Kyle> But my question still stands. Does anybody actually use any
Kyle> non-GCC compiler for userspace in Linux?

Yes. GCC produces much less than optimal code on Itanium, so the
Intel compiler often gets used.

--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au
http://www.ertos.nicta.com.au ERTOS within National ICT Australia

2006-03-29 13:27:04

by Mathieu Chouquet-Stringer

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

[email protected] (Kyle Moffett) writes:
> So does anybody compile userspace under anything other than GCC or Intel
> compilers? Do any such compilers even exist?

ccc on alpha
--
Mathieu Chouquet-Stringer
"Le disparu, si l'on v?n?re sa m?moire, est plus pr?sent et
plus puissant que le vivant".
-- Antoine de Saint-Exup?ry, Citadelle --

2006-03-29 21:24:34

by Nix

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On Tue, 28 Mar 2006, Rob Landley announced authoritatively:
> I play with Fabrice Bellard's Tiny C Compiler (http://www.tinycc.org), and
> hope to get a distro compiled with it someday, at least as a proof of
> concept.

It's a nifty idea, a sort of uClibc of C compilers. Alas it's useless on
almost all my systems because it's x86-only by design...

> That aims for full c99 and is already implementing a lot of gcc stuff too.

Good for it, as long as it doesn't go on to define __GNUC__ like icc did
at one point (even though it doesn't implement all GCC
extensions)... but Fabrice is sane so I doubt he'd do anything that
loopy.

--
`Come now, you should know that whenever you plan the duration of your
unplanned downtime, you should add in padding for random management
freakouts.'

2006-03-30 01:37:03

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On Wednesday 29 March 2006 4:23 pm, Nix wrote:
> On Tue, 28 Mar 2006, Rob Landley announced authoritatively:
> > I play with Fabrice Bellard's Tiny C Compiler (http://www.tinycc.org),
> > and hope to get a distro compiled with it someday, at least as a proof of
> > concept.
>
> It's a nifty idea, a sort of uClibc of C compilers. Alas it's useless on
> almost all my systems because it's x86-only by design...

Actually according to the changelog version 0.9.21 grew support for ARM, and I
believe it supports some other platforms too.

Notice that Fabrice Bellard (who wrote it) is also the guy behind qemu. My
vague understanding of how things went is that at some point after doing
tccboot, he was decoupled the C parser from the code generator in order to
retarget tcc to produce code for other platforms, and got the idea of
producing different front-ends too so it could input something other than C,
namely machine code.

The result was qemu, which sort of compiles machine code to machine code
dynamically, and which has taken up a large chunk of his time ever since.
(The speed of tcc development has tailed off noticeably since, but he still
spends a little time on it, and there are other developers...)

> > That aims for full c99 and is already implementing a lot of gcc stuff
> > too.
>
> Good for it, as long as it doesn't go on to define __GNUC__ like icc did
> at one point (even though it doesn't implement all GCC
> extensions)... but Fabrice is sane so I doubt he'd do anything that
> loopy.

That's more a header issue anyway. That's the uClibc developers problem. :)

Rob
--
Never bet against the cheap plastic solution.

2006-03-30 07:24:29

by Nix

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On Wed, 29 Mar 2006, Rob Landley whispered secretively:
> Actually according to the changelog version 0.9.21 grew support for ARM, and I
> believe it supports some other platforms too.

That's... impressive. Of course the more generality it grows the slower
it must necessarily become, although it'll be a while until its IR is as
bloated and thus as slow as GCC's... (GCC's only slow because of cache
effects, really).

> The result was qemu, which sort of compiles machine code to machine code
> dynamically, and which has taken up a large chunk of his time ever since.
> (The speed of tcc development has tailed off noticeably since, but he still
> spends a little time on it, and there are other developers...)

Well, I'd rather he spent time on qemu than tcc; there are other C compilers
but there's nothing quite like qemu (bochs doesn't work very well, valgrind
is similar in essence but very different in operation...)

>> > That aims for full c99 and is already implementing a lot of gcc stuff
>> > too.
>>
>> Good for it, as long as it doesn't go on to define __GNUC__ like icc did
>> at one point (even though it doesn't implement all GCC
>> extensions)... but Fabrice is sane so I doubt he'd do anything that
>> loopy.
>
> That's more a header issue anyway. That's the uClibc developers problem. :)

;)

--
`Come now, you should know that whenever you plan the duration of your
unplanned downtime, you should add in padding for random management
freakouts.'

2006-03-30 15:01:53

by Roger Heflin

[permalink] [raw]

Subject: RE: [OT] Non-GCC compilers used for linux userspace

> Not even GCC fully supports C99 (although I think it does
> support that keyword when passed -std=c99 or -std=gnu99), and
> I suspect that a majority of the other compilers for which we
> would want to add support in the kernel headers would not
> support C99 or would do a poor job of handling inline functions.
>
> But my question still stands. Does anybody actually use any
> non-GCC compiler for userspace in Linux?
>
> Cheers,
> Kyle Moffett

Sometimes.

I have customers that use Pathscale, Intel, and Portland
groups compilers.

Not all of them use the C compilers (Some just use the F90
compilers), but a number of them do use the C compiler, though
I would be that most of the applications are not going to care
about doing much against anything but standard ANSI C calls.

Roger

2006-03-30 20:26:44

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On Thursday 30 March 2006 2:24 am, Nix wrote:
> On Wed, 29 Mar 2006, Rob Landley whispered secretively:
> > Actually according to the changelog version 0.9.21 grew support for ARM,
> > and I believe it supports some other platforms too.
>
> That's... impressive. Of course the more generality it grows the slower
> it must necessarily become,

Not if compliation speed is the primary explicit design goal from day one, and
they regression test with that in mind.

Keep in mind that the main use of tcc these days is to turn c into a scripting
language. Just start your C file with

#!/usr/bin/tcc -run

And notice that #! is a preprocessor comment line as far as tcc is
concerned. :)

> > The result was qemu, which sort of compiles machine code to machine code
> > dynamically, and which has taken up a large chunk of his time ever since.
> > (The speed of tcc development has tailed off noticeably since, but he
> > still spends a little time on it, and there are other developers...)
>
> Well, I'd rather he spent time on qemu than tcc; there are other C
> compilers but there's nothing quite like qemu (bochs doesn't work very
> well, valgrind is similar in essence but very different in operation...)

They're still sort of related. The sad part is that tcc is -><- this close to
building an unmodified linux kernel, as tccboot demonstrates. But Fabrice
seems to have gone "ooh, shiny!" off in another direction, for entirely
understandable reasons... :)

I'm sure somebody will go take a whack at it sooner or later. (I haven't got
time either.)

Rob
--
Never bet against the cheap plastic solution.

2006-03-30 22:03:34

by Nix

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On Thu, 30 Mar 2006, Rob Landley suggested tentatively:
> On Thursday 30 March 2006 2:24 am, Nix wrote:
>> On Wed, 29 Mar 2006, Rob Landley whispered secretively:
>> > Actually according to the changelog version 0.9.21 grew support for ARM,
>> > and I believe it supports some other platforms too.
>>
>> That's... impressive. Of course the more generality it grows the slower
>> it must necessarily become,
>
> Not if compliation speed is the primary explicit design goal from day one, and
> they regression test with that in mind.

Aaah.

> Keep in mind that the main use of tcc these days is to turn c into a scripting
> language. Just start your C file with
>
> #!/usr/bin/tcc -run

I feel distinctly queasy. (I can't easily think of a less suitable language for
scripting than C, either: perhaps COBOL...)

> And notice that #! is a preprocessor comment line as far as tcc is
> concerned. :)

I noticed that, but I didn't think anyone actually *used* tcc for this.

--
`Come now, you should know that whenever you plan the duration of your
unplanned downtime, you should add in padding for random management
freakouts.'

2006-03-30 23:04:12

by Harald Arnesen

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

Nix <[email protected]> (Thu, 30 Mar 2006 23:02:59 +0100):

> > And notice that #! is a preprocessor comment line as far as tcc is
> > concerned. :)
>
> I noticed that, but I didn't think anyone actually *used* tcc for
> this.
>

I did...
--
Hilsen Harald.

2006-03-30 23:18:34

[permalink] [raw]

Subject: Re: [OT] Non-GCC compilers used for linux userspace

On Thursday 30 March 2006 5:02 pm, Nix wrote:
> > Keep in mind that the main use of tcc these days is to turn c into a
> > scripting language. Just start your C file with
> >
> > #!/usr/bin/tcc -run
>
> I feel distinctly queasy. (I can't easily think of a less suitable language
> for scripting than C, either: perhaps COBOL...)

No, the queasy bit is when you realize that you can stick library linkage
directives on that command line and thus you can dynamically compile and run
X apps.

They have examples of doing this. :)

> > And notice that #! is a preprocessor comment line as far as tcc is
> > concerned. :)
>
> I noticed that, but I didn't think anyone actually *used* tcc for this.

It's not exactly "what it's for", but it's up there.

Rob
--
Never bet against the cheap plastic solution.

2006-03-31 00:21:31