We've just release version 0.6 of Generalised Kernel Hooks Interface (GKHI)
see the IBM Linux Technology Centre's web page DProbes link:
http://oss.software.ibm.com/developerworks/opensource/linux
Some folks expressed an interest in this type of facility recently in
discussions concerning making call-backs from the kernel to kernel modules.
Here's the abstract for this facility. With this intend to modularise our
RAS offerings, in particular DProbes, so that they can be applied
dynamically without having to be carried as excess baggage.
Abstract:
Generalised Kernel Hooks Interface (GKHI) is a generalised facility for
placing hooks or exits in arbitrary kernel locations. It enables many
kernel enhancements, which are otherwise self-contained, to become
loadable kernel modules and retain a substantial degree of independence
from the kernel source. This affords advantages for maintenance and
co-existence with other kernel enhancements. The hook interface allows
multiple kernel modules to register their exits for a given hook, in order
to receive control at that hook location. Multiple hooks may be defined
within the kernel and a singe kernel module may register exits to use
multiple hooks. When hook exits register they may specify co-existence
criteria. Hooks may be placed in kernel modules as well as the kernel
itself with the proviso that the modules with hooks are loaded before the
gkhi hook interfacing module. A hook exit receives control as if called
from the code in which the hook is located. Parameters may be passed to a
hook exit and may be modified by an exit. For more information down-load
the tarball.
Note: GHKI is in late beta test - we currently do not support SMP, that
will occur soon. We also plan to support dynamic hook definition as little
later on so that kernel modules may dynamically register hooks for other
kernel modules to use.
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
Sounds great; unfortunately, the core group has spoken out against a
modular kernel.
Perhaps IBM should get together with SGI, HP and other interested
parties and start an Advanced Linux Kernel Project. Then they can run
off and make their scalable, modular, enterprise kernel and the Linus
Version can always merge back in features from it.
-M
[email protected] wrote:
>
> We've just release version 0.6 of Generalised Kernel Hooks Interface (GKHI)
> see the IBM Linux Technology Centre's web page DProbes link:
> http://oss.software.ibm.com/developerworks/opensource/linux
>
> Some folks expressed an interest in this type of facility recently in
> discussions concerning making call-backs from the kernel to kernel modules.
>
> Here's the abstract for this facility. With this intend to modularise our
> RAS offerings, in particular DProbes, so that they can be applied
> dynamically without having to be carried as excess baggage.
>
> Abstract:
> Generalised Kernel Hooks Interface (GKHI) is a generalised facility for
> placing hooks or exits in arbitrary kernel locations. It enables many
> kernel enhancements, which are otherwise self-contained, to become
> loadable kernel modules and retain a substantial degree of independence
> from the kernel source. This affords advantages for maintenance and
> co-existence with other kernel enhancements. The hook interface allows
> multiple kernel modules to register their exits for a given hook, in order
> to receive control at that hook location. Multiple hooks may be defined
> within the kernel and a singe kernel module may register exits to use
> multiple hooks. When hook exits register they may specify co-existence
> criteria. Hooks may be placed in kernel modules as well as the kernel
> itself with the proviso that the modules with hooks are loaded before the
> gkhi hook interfacing module. A hook exit receives control as if called
> from the code in which the hook is located. Parameters may be passed to a
> hook exit and may be modified by an exit. For more information down-load
> the tarball.
>
> Note: GHKI is in late beta test - we currently do not support SMP, that
> will occur soon. We also plan to support dynamic hook definition as little
> later on so that kernel modules may dynamically register hooks for other
> kernel modules to use.
>
> Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
>
> http://oss.software.ibm.com/developerworks/opensource/linux
> Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
> IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> Please read the FAQ at http://www.tux.org/lkml/
Hi Michael,
On Wed, 08 Nov 2000, Michael Rothwell wrote:
> Sounds great; unfortunately, the core group has spoken out against a
> modular kernel.
>
> Perhaps IBM should get together with SGI, HP and other interested
> parties and start an Advanced Linux Kernel Project. Then they can
> run off and make their scalable, modular, enterprise kernel and the
> Linus Version can always merge back in features from it.
*Are you crazy?* =:-0
Proposing proprietary kernel extensions to establish an enterprise
kernel? No thanks!
Greetings
Christoph
On Thu, Nov 09, 2000 at 08:44:11AM +0100, Christoph Rohland wrote:
> Hi Michael,
>
> On Wed, 08 Nov 2000, Michael Rothwell wrote:
> > Sounds great; unfortunately, the core group has spoken out against a
> > modular kernel.
> >
> > Perhaps IBM should get together with SGI, HP and other interested
> > parties and start an Advanced Linux Kernel Project. Then they can
> > run off and make their scalable, modular, enterprise kernel and the
> > Linus Version can always merge back in features from it.
>
> *Are you crazy?* =:-0
>
> Proposing proprietary kernel extensions to establish an enterprise
> kernel? No thanks!
Actually, I think this idea is a good one. I'm a big opponent of all the
big iron feature bloat getting into the kernel, and if SGI et al want to
go off and do their own thing, that's fine with me. As long as Linus
continues in his current role, I doubt much of anything that the big iron
boys do will really make it back into the generic kernel. Linus is really
smart about that stuff, are least it seems so to me; he seems to be well
aware that 99.9999% of the hardware in the world isn't big iron and never
will be, so something approximating 99% of the effort should be going towards
the common platforms, not the uncommon ones.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
Second or Third here!!!
TRG plans to create and publish a native RING 0 kernel and packages.
This may end up as a bolt on ./arch or something.
Not everyone in the world needs a SUPERCHARGED, FUEL-INJECTED, ALCOHOL,
FIRE-BREATHING kernel, but some do!
Andre Hedrick
CTO Timpanogas Research Group
EVP Linux Development, TRG
Linux ATA Development
On Wed, 8 Nov 2000, Larry McVoy wrote:
> On Thu, Nov 09, 2000 at 08:44:11AM +0100, Christoph Rohland wrote:
> > Hi Michael,
> >
> > On Wed, 08 Nov 2000, Michael Rothwell wrote:
> > > Sounds great; unfortunately, the core group has spoken out against a
> > > modular kernel.
> > >
> > > Perhaps IBM should get together with SGI, HP and other interested
> > > parties and start an Advanced Linux Kernel Project. Then they can
> > > run off and make their scalable, modular, enterprise kernel and the
> > > Linus Version can always merge back in features from it.
> >
> > *Are you crazy?* =:-0
> >
> > Proposing proprietary kernel extensions to establish an enterprise
> > kernel? No thanks!
>
> Actually, I think this idea is a good one. I'm a big opponent of all the
> big iron feature bloat getting into the kernel, and if SGI et al want to
> go off and do their own thing, that's fine with me. As long as Linus
> continues in his current role, I doubt much of anything that the big iron
> boys do will really make it back into the generic kernel. Linus is really
> smart about that stuff, are least it seems so to me; he seems to be well
> aware that 99.9999% of the hardware in the world isn't big iron and never
> will be, so something approximating 99% of the effort should be going towards
> the common platforms, not the uncommon ones.
> --
> ---
> Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> Please read the FAQ at http://www.tux.org/lkml/
>
Let be clear about one thing: the GKHI make no statement about enabling
proprietary extensions and that's a common misconception. GKHI is intended
to make optional facilities easier to co-install and change. We designed it
for DProbes, and when modularised will remain a GPL opensource offering.
The only motivation for providing GKHI is to make the kernel more
acceptable to the enterprise customer, but allowing, for example, RAS
capabilities to be brough in easily and dynmaically. This type of customer
will not readily succome to on-the-fly kernel rebuilds to diagnose problems
that occur only in complex production environments.
If anything opens the door to proprietary extensions it's the loadable
kernel modules capability or perhaps the loose wording of the GPL which
doesn't catch loadable kernel modules, or whatever... Bottom line GKHI
really has no bearing on this.
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
Christoph Rohland <[email protected]> on 09/11/2000 07:44:11
Please respond to Christoph Rohland <[email protected]>
To: Michael Rothwell <[email protected]>
cc: Richard J Moore/UK/IBM@IBMGB, [email protected]
Subject: Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)
Hi Michael,
On Wed, 08 Nov 2000, Michael Rothwell wrote:
> Sounds great; unfortunately, the core group has spoken out against a
> modular kernel.
>
> Perhaps IBM should get together with SGI, HP and other interested
> parties and start an Advanced Linux Kernel Project. Then they can
> run off and make their scalable, modular, enterprise kernel and the
> Linus Version can always merge back in features from it.
*Are you crazy?* =:-0
Proposing proprietary kernel extensions to establish an enterprise
kernel? No thanks!
Greetings
Christoph
Hi Larry,
On Wed, 8 Nov 2000, Larry McVoy wrote:
> On Thu, Nov 09, 2000 at 08:44:11AM +0100, Christoph Rohland wrote:
>> *Are you crazy?* =:-0
>>
>> Proposing proprietary kernel extensions to establish an enterprise
>> kernel? No thanks!
>
> Actually, I think this idea is a good one. I'm a big opponent of
> all the big iron feature bloat getting into the kernel, and if SGI
> et al want to go off and do their own thing, that's fine with me.
> As long as Linus continues in his current role, I doubt much of
> anything that the big iron boys do will really make it back into the
> generic kernel. Linus is really smart about that stuff, are least
> it seems so to me; he seems to be well aware that 99.9999% of the
> hardware in the world isn't big iron and never will be, so something
> approximating 99% of the effort should be going towards the common
> platforms, not the uncommon ones.
If we would not allow binary only modules I would not have such a big
problem with that...
I understand that the one size fits all approach has some limitations
if you want to run on PDAs up to big iron. But a framework to overload
core kernel functions with modules smells a lot of binary only, closed
source, vendor specific Linux on high end machines.
And then I don't see the value of Linux anymore.
Greetings
Christoph
Hi Richard,
On Thu, 9 Nov 2000, richardj moore wrote:
> Let be clear about one thing: the GKHI make no statement about
> enabling proprietary extensions and that's a common
> misconception. GKHI is intended to make optional facilities easier
> to co-install and change. We designed it for DProbes, and when
> modularised will remain a GPL opensource offering.
Yes, I understand that.
> The only motivation for providing GKHI is to make the kernel more
> acceptable to the enterprise customer, but allowing, for example,
> RAS capabilities to be brough in easily and dynmaically. This type
> of customer will not readily succome to on-the-fly kernel rebuilds
> to diagnose problems that occur only in complex production
> environments.
I know this problem pretty well.
> If anything opens the door to proprietary extensions it's the
> loadable kernel modules capability or perhaps the loose wording of
> the GPL which doesn't catch loadable kernel modules, or
> whatever... Bottom line GKHI really has no bearing on this.
Yes, and that's why I am opposing here: Technically you are right, but
proposing that enterprise Linux should go this way is inviting binary
only modules due to the lax handling of modules.
Please keep in mind: I did not react to your announcement but to the
proposal that the companies should jump on it to do a special
enterprise Linux. If we really need a special enterprise tree lets do
it without module tricks.
Greetings
Christoph
Christoph Rohland wrote:
> If we would not allow binary only modules I would not have such a big
> problem with that...
I'm not sure how you would do that.
> I understand that the one size fits all approach has some limitations
> if you want to run on PDAs up to big iron. But a framework to overload
> core kernel functions with modules smells a lot of binary only, closed
> source, vendor specific Linux on high end machines.
Since Linux is GPL, how would you stop this?
> And then I don't see the value of Linux anymore.
Same as before -- freedom and low cost. The primary advantae of Linux
over other OSes is the GPL.
I think and Advanced Linux Kernel PRoject would be a good idea for a
number of reasons. It would give "Enterprise" users their own special
kernel, just like the microcontroller and real-time guys have. It would
also provide a parallel development track for Linux that could provide
real competition and value to the Linus-version kernel. The "Enterprise"
machines that IBM, HP and SGI would target aren't all S/390s; there
would be significant overlap of their low end with Linus' high end, I
think. Like 8+-way SMP servers.
-M
Christoph Rohland wrote:
> If we really need a special enterprise tree lets do
> it without module tricks.
Why? I think the IBM GKHI code would be of tremendous value. It would
make the kernel much more flexible, and for users, much more friendly.
No more patch-and-recompile to add a filesystem or whatever. There's no
reason to hamstring their efforts because of the possibility of binary
modules. The GPL allows that, right? So any developer of binary-only
extensions using the GKHI would not be breaking the license agreement, I
don't think. There's lots of binary modules right now -- VMWare, Aureal
sound card drivers, etc.
I understand and agree with your desire for full source for everything,
but I disagree that we should artificially limit people's ability to use
Linux to solve their problems.
On 2000-11-09T07:25:52,
Michael Rothwell <[email protected]> said:
> Why? I think the IBM GKHI code would be of tremendous value. It would
> make the kernel much more flexible, and for users, much more friendly.
> No more patch-and-recompile to add a filesystem or whatever. There's no
> reason to hamstring their efforts because of the possibility of binary
> modules. The GPL allows that, right? So any developer of binary-only
> extensions using the GKHI would not be breaking the license agreement, I
> don't think. There's lots of binary modules right now -- VMWare, Aureal
> sound card drivers, etc.
And we already refuse to support those kernels - your point being?
Making this "commonplace" is a nightmare. Go away with that.
> I understand and agree with your desire for full source for everything,
> but I disagree that we should artificially limit people's ability to use
> Linux to solve their problems.
I want their solving of their problems not to create problems for me though.
Sincerely,
Lars Marowsky-Br?e <[email protected]>
Development HA
--
Perfection is our goal, excellence will be tolerated. -- J. Yahl
On 2000-11-09T07:20:27,
Michael Rothwell <[email protected]> said:
> > I understand that the one size fits all approach has some limitations
> > if you want to run on PDAs up to big iron. But a framework to overload
> > core kernel functions with modules smells a lot of binary only, closed
> > source, vendor specific Linux on high end machines.
>
> Since Linux is GPL, how would you stop this?
Christoph / SAP is in a rather good position to stop that being supported by
vendors...
> Same as before -- freedom and low cost. The primary advantae of Linux
> over other OSes is the GPL.
And that is why that has to govern the kernel and its modules as far as
possible.
Sincerely,
Lars Marowsky-Br?e <[email protected]>
Development HA
--
Perfection is our goal, excellence will be tolerated. -- J. Yahl
On Thu, 9 Nov 2000, Michael Rothwell wrote:
> Same as before -- freedom and low cost. The primary advantae of Linux
> over other OSes is the GPL.
Now, that's more than slightly insulting...
The problem with the hooks et.al. is very simple - they promote every
bloody implementation detail to exposed API. Sorry, but... See Figure 1.
It won't fly.
Lars Marowsky-Bree wrote:
> And we already refuse to support those kernels - your point being?
Who says you would support theirs? My point is, forks have been made in
the past and are useful for the people that use them, and prevent
"pollution" of the common kernel with hghly specialized modifications.
uCLinux and RTLinux are two examples.
> Making this "commonplace" is a nightmare. Go away with that.
It would be a third major fork (AFAIK), not a first, and not a
nightmare. Are RTLinux and uclinux nightmares? How much do they impact
your life?
> I want their solving of their problems not to create problems for me though.
How would it create problems for you? And as a separate question, why
should anyone care if it does? Part of the freedom guaranteed by the GPL
is the ability for anyone to fork a codebase for their own use. And
because it's all GPL code, with thoroughly divirsified copyright
assignment, they will be releasing any mods under GPL as well. The ones
the Linus likes, he can adapt and merge into the common kernel. If he
doesn't like them, he can ignore them.
Incidentally, I hear that a Utah company is going to position an older
Unix kernel with Linux compatibility added and possibly a GNU userspace
as "Enterprise Linux." Would you prefer that, or something actually
based on the Linux codebase?
-M
On Wed, 8 Nov 2000, Larry McVoy wrote:
> As long as Linus continues in his current role, I doubt much of
> anything that the big iron boys do will really make it back into the
> generic kernel.
That is great, thank you. At least I know now someone on this planet who
agrees with me! Everyone (regardless of whether he has seen commercial
UNIX source or not, but it pains most to hear from those who have) seems
to be sceptical when I say exactly the above. They think "how could that
possibly be -- the "big iron Unixen" had such a large investment of real
money to make them full of "enterprise-features", surely there must be
something useful for Linux to learn from". To which I reply -- "most
(probably 99-100%) Linux kernel hackers have access to the source code of
most (probably 40%-90% depending on their lack) flavours of commercial
UNIX and would be quite happy to "steal" code from it (as long as they,
like myself agree with RMS philosophy/morals) but the reality is -- there
is _nothing_ to steal from it". Commercial UNIXen are just useless -- we
_have_ to rewrite everything from scratch, not because we can't steal (or
think it immoral) but because there is nothing interesting to steal left.
The free stuff is actually better and technically superior to that which
is not free.
To contradict to myself (just a tiny bit!) I think it would still be
useful for the public in general (by "public in general" I meant those
clueless individuals who write the laws men have to obey) to recognize the
above fact and let the hackers be no longer shy about it, i.e. be able to
freely discuss any information they have (or had) access to, so that, in
the unlikely case that there is something useful to "steal" from it, they
can take it freely and put it into Linux.
Regards,
Tigran
PS. The very fact I can say the above and still (hope to) have a job
afterwards is a sign of much progress towards the better end :)
PPS. Better is the end of a thing than the beginning thereof: and the
patient in spirit is better than the proud in spirit. (Eccl 7:8)
On Thu, 9 Nov 2000, Michael Rothwell wrote:
> Why? I think the IBM GKHI code would be of tremendous value. It would
> make the kernel much more flexible, and for users, much more friendly.
> No more patch-and-recompile to add a filesystem or whatever. There's no
> reason to hamstring their efforts because of the possibility of binary
> modules. The GPL allows that, right?
no gpl definitely does not alow binary modules.
afaik linus allows binary modules in most cases.
> So any developer of binary-only
> extensions using the GKHI would not be breaking the license agreement, I
> don't think. There's lots of binary modules right now -- VMWare, Aureal
> sound card drivers, etc.
>
> I understand and agree with your desire for full source for everything,
> but I disagree that we should artificially limit people's ability to use
> Linux to solve their problems.
> -
--paulj
Paul Jakma wrote:
>
> On Thu, 9 Nov 2000, Michael Rothwell wrote:
>
> > Why? I think the IBM GKHI code would be of tremendous value. It would
> > make the kernel much more flexible, and for users, much more friendly.
> > No more patch-and-recompile to add a filesystem or whatever. There's no
> > reason to hamstring their efforts because of the possibility of binary
> > modules. The GPL allows that, right?
>
> no gpl definitely does not alow binary modules.
Well, then, problem solved.
> afaik linus allows binary modules in most cases.
And since an "Advanced Linux Kernel Project" wouldn't be a Linus kernel,
what then? Would they have the same discretion as Linus? Would Linus'
exception apply to them?
Alexander Viro wrote:
>
> On Thu, 9 Nov 2000, Michael Rothwell wrote:
>
> > Same as before -- freedom and low cost. The primary advantae of Linux
> > over other OSes is the GPL.
>
> Now, that's more than slightly insulting...
Well, it wasn't meant to be. I imagine RMS would make the same type of
statement -- Linux is libre, therefore superior. There's a number of
OSes that have advantages of Linux in some area; Solaris can use more
processors; QNX is real-time, smaller and still posix; Windows has
better application support (i.e., more of them); MacOS has better color
and font management. But, Linux is free. Let's say for a moment that
Linux was exactly the same as Solaris, technically. Linux would be
superior because it is licensed under the GPL, and is free; whereas
Solaris would not be.
> The problem with the hooks et.al. is very simple - they promote every
> bloody implementation detail to exposed API. Sorry, but... See Figure 1.
> It won't fly.
Figure 1?
-M
On Thu, 9 Nov 2000, Michael Rothwell wrote:
> Alexander Viro wrote:
> >
> > On Thu, 9 Nov 2000, Michael Rothwell wrote:
> >
> > > Same as before -- freedom and low cost. The primary advantae of Linux
> > > over other OSes is the GPL.
> >
> > Now, that's more than slightly insulting...
>
> Well, it wasn't meant to be. I imagine RMS would make the same type of
> statement -- Linux is libre, therefore superior. There's a number of
<shrug> RMS had repeatedly demonstrated what he's worth as a designer
and programmer. Way below zero. You may like or dislike his ideology,
but when it comes to technical stuff... Not funny.
> OSes that have advantages of Linux in some area; Solaris can use more
> processors; QNX is real-time, smaller and still posix; Windows has
> better application support (i.e., more of them); MacOS has better color
> and font management. But, Linux is free. Let's say for a moment that
> Linux was exactly the same as Solaris, technically. Linux would be
You mean, bloated tasteless parody on UNIX? Thanks, but no thanks.
> superior because it is licensed under the GPL, and is free; whereas
> Solaris would not be.
Small solace it would be.
> > The problem with the hooks et.al. is very simple - they promote every
> > bloody implementation detail to exposed API. Sorry, but... See Figure 1.
> > It won't fly.
>
> Figure 1?
Use search engine. On google "See Figure 1" brings the thing in the first
5 hits.
> reason to hamstring their efforts because of the possibility of binary
> modules. The GPL allows that, right? So any developer of binary-only
Its not clear the GPL does allow it.
> extensions using the GKHI would not be breaking the license agreement, I
> don't think. There's lots of binary modules right now -- VMWare, Aureal
> sound card drivers, etc.
All of which just cause large numbers of bugs to go in the bitbucket because
nobody can tell whose the problem is.
> > Making this "commonplace" is a nightmare. Go away with that.
>
> It would be a third major fork (AFAIK), not a first, and not a
> nightmare. Are RTLinux and uclinux nightmares? How much do they impact
> your life?
RTLinux is hardly a fork. UcLinux is a fork, it has its own mailing list, web
site and everything. Post 2.4 I'm still very interested in spending time merging
the 2.4 uc and the main tree. I think it can be done and they are doing it in
a way that leads logically to this.
To a lot of people the ucLinux is 2.0 and our MMU based boards run 2.2.18 and
this and that are different is a real pain
On Thu, 9 Nov 2000, Michael Rothwell wrote:
> Well, then, problem solved.
>
:)
> > afaik linus allows binary modules in most cases.
>
> And since an "Advanced Linux Kernel Project" wouldn't be a Linus kernel,
> what then? Would they have the same discretion as Linus? Would Linus'
> exception apply to them?
don't know. you'd have to ask him...
I actually think Linus has been too loose/vague on modules. The
official COPYING txt file in the tree contains an exception on linking
to the kernel using syscalls from linus and the GPL. nothing about
binary modules, and afaik the only statements he's ever made about
binary modules were off the cuff on l-k a long time (unless someone
knows a binary module whose vendor can show a written exception from
Linus et al).
The result of all this is that we've had plenty of vendors ignoring
the GPL as it applies to linux and release binary modules all because
linus said on a mailling list that he doesn't mind too much. not a
very strong affirmation of the conditions under which linux is
licensed.
be nice if the binary module thing could be clarified by the copyright
holders.
--paulj
Alexander Viro wrote:
> > Figure 1?
>
> Use search engine. On google "See Figure 1" brings the thing in the first
> 5 hits.
http://www.google.com/search?q=See+Figure+1&btnG=Google+Search
->
http://spiffy.cso.uiuc.edu/~kline/Stuff/see-figure-1.html
->
http://spiffy.cso.uiuc.edu/~kline/Stuff/f-you.gif
...
http://www.google.com/search?q=See+Figure+1&btnG=Google+Search
->
http://www.physics.ohio-state.edu/~bcd/humor/figure1.html
... How utterly typical of you, Viro.
-M
On Thu, 9 Nov 2000, Michael Rothwell wrote:
> Alexander Viro wrote:
> >
> > On Thu, 9 Nov 2000, Michael Rothwell wrote:
> >
> > > Same as before -- freedom and low cost. The primary advantae of Linux
> > > over other OSes is the GPL.
> >
> > Now, that's more than slightly insulting...
>
> Well, it wasn't meant to be. I imagine RMS would make the same type of
> statement -- Linux is libre, therefore superior. There's a number of
> OSes that have advantages of Linux in some area; Solaris can use more
> processors; QNX is real-time, smaller and still posix; Windows has
> better application support (i.e., more of them); MacOS has better color
> and font management. But, Linux is free. Let's say for a moment that
> Linux was exactly the same as Solaris, technically. Linux would be
> superior because it is licensed under the GPL, and is free; whereas
> Solaris would not be.
<lurker off>
Sorry, I couldn't resist.
1) "Solaris running on more processors"? Think of Beowulf clusters.
Strangely enough, people running them choose Linux as the kernel.
2) "QNX is real-time", true. Linux is not. Please don't compare apples with
oranges.
3) "Windows has more apps"? True. Is it a kernel property of any kind? No.
4) "MacOS has better color and font management" this is funny as it speaks
for itself. I'd also add "MacOS is usually housed in a better-looking
case". Luckily enough, under Linux color and font management is NOT a
kernel job. No more than the external design of the case, I mean.
And I really hope Linux will *never* be exactly the same as Solaris,
technically.
</lurker on>
> > The problem with the hooks et.al. is very simple - they promote every
> > bloody implementation detail to exposed API. Sorry, but... See Figure 1.
> > It won't fly.
>
> Figure 1?
>From "The design and implementation of the Perfect OS 1.0", yet to be
published. The good thing about this book is that there will never be
a second edition. Of course the only release of Perfect OS will be 1.0!
B-) B-) B-) B-)
>
> -M
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> Please read the FAQ at http://www.tux.org/lkml/
>
.TM.
--
____/ ____/ /
/ / / Marco Colombo
___/ ___ / / Technical Manager
/ / / ESI s.r.l.
_____/ _____/ _/ [email protected]
Alan Cox wrote:
> RTLinux is hardly a fork. UcLinux is a fork, it has its own mailing list, web
> site and everything. Post 2.4 I'm still very interested in spending time merging
> the 2.4 uc and the main tree. I think it can be done and they are doing it in
> a way that leads logically to this.
And how would a hypothetical Advanced Linux Kernel Project be different?
Set aside the GKHI and the issue of binary-only hook modules; how would
an "enterprise" fork be any different than RT or UC? It'll go off,
change and add some things, and then perhaps be merged back in later. In
the meantime, developers who want to add "enterpriseness" to Linux will
have an outlet and won't have to simply gripe on this list anymore. And
users who want an "enterprise" kernel can get one.
-M
Larry McVoy <[email protected]>:
> On Thu, Nov 09, 2000 at 08:44:11AM +0100, Christoph Rohland wrote:
> > Hi Michael,
> >
> > On Wed, 08 Nov 2000, Michael Rothwell wrote:
> > > Sounds great; unfortunately, the core group has spoken out against a
> > > modular kernel.
> > >
> > > Perhaps IBM should get together with SGI, HP and other interested
> > > parties and start an Advanced Linux Kernel Project. Then they can
> > > run off and make their scalable, modular, enterprise kernel and the
> > > Linus Version can always merge back in features from it.
> >
> > *Are you crazy?* =:-0
> >
> > Proposing proprietary kernel extensions to establish an enterprise
> > kernel? No thanks!
>
> Actually, I think this idea is a good one. I'm a big opponent of all the
> big iron feature bloat getting into the kernel, and if SGI et al want to
> go off and do their own thing, that's fine with me. As long as Linus
> continues in his current role, I doubt much of anything that the big iron
> boys do will really make it back into the generic kernel. Linus is really
> smart about that stuff, are least it seems so to me; he seems to be well
> aware that 99.9999% of the hardware in the world isn't big iron and never
> will be, so something approximating 99% of the effort should be going towards
> the common platforms, not the uncommon ones.
Not strictly true... Many of the capabilities in Linux now came from what
was "big iron" 10 years ago. At the rate that CPU speeds/multi-CPU systems
are becoming available, the current "big iron" will be on your desk, sharing
data among many other systems to solve even bigger problems.
I happen to be a proponent of big iron features... Just let ME choose which
ones I need, and which I don't.
I already need MLS and VPN support with encryption (IPSec + more :). When my
SGI workstation gets replaced (by a Linux PC) I'll want to be able to
administer (securely) the security structures supported by Cray UNICOS,
Trusted Solaris and Trusted IRIX.
Currently, I have to downgrade the security of the Cray systems just to
administer it (not good, but currently acceptable by the Powers That Be).
I want better. Better security, better authentication, better identification,
secure communication, and auditability. All of this is currently part of "big
iron", but is needed in the commercial world. IPSec is beginning to be called
for in business. MLS is already used there, and I think its use will only
expand. Anything that makes an improvement in this area is needed.
I believe that having the hooks might make it easier to implement higher
levels of security by option. It will certainly make it easer to test
new features. Include a module that attaches to hooks already present --
Other users wouldn't need the module. So let it be an option.
Linux already runs on big iron. Why not support it.
<joke>Anything that can put pressure on M$ ...</joke>
-------------------------------------------------------------------------
Jesse I Pollard, II
Email: [email protected]
Any opinions expressed are solely my own.
On Thu, 9 Nov 2000, Paul Jakma wrote:
> On Thu, 9 Nov 2000, Michael Rothwell wrote:
>
> > Well, then, problem solved.
> >
>
> :)
>
> > > afaik linus allows binary modules in most cases.
> >
> > And since an "Advanced Linux Kernel Project" wouldn't be a Linus kernel,
> > what then? Would they have the same discretion as Linus? Would Linus'
> > exception apply to them?
>
> don't know. you'd have to ask him...
>
> I actually think Linus has been too loose/vague on modules. The
> official COPYING txt file in the tree contains an exception on linking
> to the kernel using syscalls from linus and the GPL. nothing about
> binary modules, and afaik the only statements he's ever made about
> binary modules were off the cuff on l-k a long time (unless someone
> knows a binary module whose vendor can show a written exception from
> Linus et al).
>
> The result of all this is that we've had plenty of vendors ignoring
> the GPL as it applies to linux and release binary modules all because
> linus said on a mailling list that he doesn't mind too much. not a
> very strong affirmation of the conditions under which linux is
> licensed.
Well, HW vendors may provide a binary module as a timid attempt to
support Linux. A few have already understood that providing an Open Source
one is far a better attitude: they can *get* support for it from the
kernel community. They end up with a better driver, and they can even
learn something useful for their W98/NT/Sco/whatever drivers, too.
If they don't abuse of it (they are sicerely willing to "provide" something)
it's clearly a winning move.
Other vendors are just scared of the two words "Open Source" so they make
a little first step in releasing a binary only driver, which they are
more used to. I believe that sooner or later they'll realize the advantages
of the Open Source attitude, and they'll make the move.
A binary only file-system module is a completely different matter.
Legally, it may have the same "status" of a binary only driver.
Technically, it's just another module. But it seems to me a much clearer
violation of GPL. If you want to hide the internals of your software,
you're not GPL-compatible (a driver is slightly different in that
a HW company is probably worried about the internals of their HW).
>
> be nice if the binary module thing could be clarified by the copyright
> holders.
Of course.
>
> --paulj
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> Please read the FAQ at http://www.tux.org/lkml/
>
.TM.
--
____/ ____/ /
/ / / Marco Colombo
___/ ___ / / Technical Manager
/ / / ESI s.r.l.
_____/ _____/ _/ [email protected]
Date: Thu, 9 Nov 2000 13:39:04 +0000 (GMT)
From: Paul Jakma <[email protected]>
I actually think Linus has been too loose/vague on modules. The
official COPYING txt file in the tree contains an exception on linking
to the kernel using syscalls from linus and the GPL. nothing about
binary modules, and afaik the only statements he's ever made about
binary modules were off the cuff on l-k a long time (unless someone
knows a binary module whose vendor can show a written exception from
Linus et al).
Actually, he's been quite specific. It's ok to have binary modules as
long as they conform to the interface defined in /proc/ksyms.
That being said, the real problem with the GKHI is that as Al said, it
does expose internal kernel interfaces --- and the Linux kernel
development community as a whole refuses to be bound by such interfaces,
sometimes even during a stable kernel series.
So someone who releases only binary modules will likely be in a world of
hurt. And that's considered a feature. Certainly no one is going to
go out of their way to make life easier for binary module authors.
- Ted
Date: Thu, 09 Nov 2000 08:43:14 -0500
From: Michael Rothwell <[email protected]>
And how would a hypothetical Advanced Linux Kernel Project be different?
Set aside the GKHI and the issue of binary-only hook modules; how would
an "enterprise" fork be any different than RT or UC? It'll go off,
change and add some things, and then perhaps be merged back in later. In
the meantime, developers who want to add "enterpriseness" to Linux will
have an outlet and won't have to simply gripe on this list anymore. And
users who want an "enterprise" kernel can get one.
Well, there are three possibilities about how it can end up.
1) It will be loaded with so much crap that in fact it ends up being
less performant than the mainline Linux. No problem, we can ignore it.
2) It will be a thing of perfect beauty, in which every single change
is thoughtfully well-considered, and scales well to both the low end and
the high end. No problem, it's easy to merge stuff back.
3) It will be a mixture, of some stuff which is good, and some such
which is crap, and it will be very difficult hard to merge back some of
the good stuff, since it has dependencies on the crap. In the meantime,
mainline Linux will have continued moving forward, and it will be even
harder to merge the advanced features of Linux back into the forked
version of the kernel.
One of the big reasons why many of the big companies have been looking
at Linux is because Unix OS engineering is viewed as a cost center, not
as a profit center. The moment you fork and make an "Advanced Linux
Kernel Project", a lot of the costs involved with maintaining such a
beast come back. And sure, you can try to solve this problem by working
in a consoritum-like fashion with other Companies ---- just like OSF/1
and Monterrey tried to do.
So there are some real costs associated with forking. At the same time,
if these companies need to get product out the door quickly, short-term
forks can be good things. But nothing in life is free, and it's in
*everybody's* best interest to resolve forks as soon as possible.
Otherwise, you end up losing a lot of the advantages of the Open Source
development model.
- Ted
> Actually, he's been quite specific. It's ok to have binary modules as
> long as they conform to the interface defined in /proc/ksyms.
What is completely unclear is if he has the authority to say that given that
there is code from other people including the FSF merged into the tree.
I've taken to telling folks who ask about binary modules to talk to their legal
department. The whole question is simply to complicated for anyone else to
work on.
Alan
Date: Wed, 08 Nov 2000 16:35:33 -0500
From: Michael Rothwell <[email protected]>
Sounds great; unfortunately, the core group has spoken out against a
modular kernel.
This is true; that's because a modular kernel means that interfaces have
to be frozen in time, usually forever. This makes a lot of improvements
impossible, and over time there is more and more crud added to simply
act as "impedance matching" between incompatible/legacy interfaces.
However, that doesn't mean that the GKHI is *automatically* bad. It all
depends on how you use it. Just as with kernel modules, where it's
darned useful for distributions so they don't have to build custom
kernels for each installation (but source is included for all modules),
the GKHI could be used in a similar way.
The issue will be with what hooks are allowed to be exported via
the GKHI; this defines the interfaces. Also, it's important to note
which interfaces are and aren't guaranteed to be stable. If the
interfaces aren't guaranteed to be stable, then incompatibilities and
keeping up with the latest version are a problem for the provider of the
binary module, not of the Linux Kernel Development Community.
- Ted
Alan Cox wrote:
>
> > Actually, he's been quite specific. It's ok to have binary modules as
> > long as they conform to the interface defined in /proc/ksyms.
>
> What is completely unclear is if he has the authority to say that given that
> there is code from other people including the FSF merged into the tree.
I've always wondered if we shouldn't encourage people to assign the
copyright for their code to Linux International[1], or some other truly
neutral organization like the FSF[2]. gcc and related projects assign
their copyrights to the FSF to avoid messes like this...
At this point we couldn't force people to change their copyright, but we
can encourage, if LI or another entity exists to which copyrights can be
assigned.
Jeff
[1] After talking to LI lawyers and getting it all set up, etc.
[2] In this particular case, due to the modified kernel GPL, FSF is
probably a bad target for copyright assignments.
--
Jeff Garzik |
Building 1024 | Would you like a Twinkie?
MandrakeSoft |
On Wed, 8 Nov 2000, Larry McVoy wrote:
> smart about that stuff, are least it seems so to me; he seems to be
> well aware that 99.9999% of the hardware in the world isn't big iron
> and never will be, so something approximating 99% of the effort should
> be going towards the common platforms, not the uncommon ones.
yep, this is true. Still Linux appears to perform remarkably well on
so-called 'big iron'.
IMO it's a big misconception that 'big iron is different'. Yes, patches
and bad design can make source trees very different. But IMO big iron is
not more different from a normal PIII workstation than a PDA is different.
We doing a bad job if we cannot support them all - or at least we must
always be able to keep clean interfaces so keeping a forked sub-project
for a less known or less understood feature is easy and straightforward.
In fact i believe a PDA is much harder to do right than high-end systems.
Making something faster, given endless resources, is almost always easy.
But maximizing performance on a fundamentally and intentionally limited
platform is much less trivial and takes alot of clout.
in the 2.4 kernel we have all the features that is needed for 'enterprise
scalability', in fact i believe we have some scalability features (eg. big
reader spinlocks) that are not available on other systems.
Ingo
Alexander Viro <[email protected]> writes:
> <shrug> RMS had repeatedly demonstrated what he's worth as a designer
> and programmer. Way below zero. You may like or dislike his ideology,
> but when it comes to technical stuff... Not funny.
Huh?
<annoying valspeak tone>
*Hello*? GNU gcc? GNU emacs? Way below zero? *Hello*?!?
</annoying valspeak tone>
--
[O]ne of the features of the Internet [...] is that small groups of people can
greatly disturb large organizations. --Charles C. Mann
On 9 Nov 2000, Mike Coleman wrote:
> Alexander Viro <[email protected]> writes:
> > <shrug> RMS had repeatedly demonstrated what he's worth as a designer
> > and programmer. Way below zero. You may like or dislike his ideology,
> > but when it comes to technical stuff... Not funny.
>
> Huh?
>
> <annoying valspeak tone>
>
> *Hello*? GNU gcc? GNU emacs? Way below zero? *Hello*?!?
>
> </annoying valspeak tone>
Yes. GNU emacs. Bloated and ugly like hell. gcc. Codebase is anything but
pretty. Your point being? And no, it's not trolling. RMS really got no
taste - read his postings on _technical_ GNU lists and see yourself.
Crap can work. Given enough thrust pigs will fly, but it's not necessary a
good idea.
$DEITY witness, we've got enough ugly code, but compared to the average
level of GNU codebase kernel looks wonderful. Sorry. I think that I've
spent enough time dealing with both to be able to compare.
Date: Thu, 9 Nov 2000 14:26:33 +0000 (GMT)
From: Alan Cox <[email protected]>
> Actually, he's been quite specific. It's ok to have binary modules as
> long as they conform to the interface defined in /proc/ksyms.
What is completely unclear is if he has the authority to say that given that
there is code from other people including the FSF merged into the
tree.
He said it long enough ago that presumably people who merged code in
would have accepted it as an implicit agreement, if it had been
documented in the COPYING file. Unfortuantely, it wasn't so documented,
so I agree it's unclear.
I've taken to telling folks who ask about binary modules to talk to
their legal department. The whole question is simply to complicated
for anyone else to work on.
... and at that point we run intothe oft-debated (but never tested in a
court of law) question of into the whether or not Copyright can infect
across a link, especially since the link is done by the end-user. (Yes,
there are some questions about inline functions in the header files.)
The intent of what the FSF would like the case to be is clear, but it's
not clear at all that Copyright law works that way; intent doesn't
matter if the laws don't give you the right to sue on those grounds.
See a lawyer and get official legal advice indeed....
- Ted
Hi Ingo,
On Thu, 9 Nov 2000, Ingo Molnar wrote:
>
> On Wed, 8 Nov 2000, Larry McVoy wrote:
>
>> smart about that stuff, are least it seems so to me; he seems to be
>> well aware that 99.9999% of the hardware in the world isn't big
>> iron and never will be, so something approximating 99% of the
>> effort should be going towards the common platforms, not the
>> uncommon ones.
>
> yep, this is true. Still Linux appears to perform remarkably well on
> so-called 'big iron'.
Thanks Ingo, I agree to this and also agree that we should try to be
able to run (mostly) everything from the same code base and I think
that's Linux does a great job on this.
Having a seperated code base for 0.0001% of mission critical servers
will lead to very bad availability or very high development cost at
least. In the worst (and not so unprobable) case it will lead to a lot
of games with licenses and 'intellectual property'. This would mean
that Linux fails to deliver on its promises IMHO.
Greetings
Christoph
Hi Michael,
On Thu, 09 Nov 2000, Michael Rothwell wrote:
> Christoph Rohland wrote:
>> And then I don't see the value of Linux anymore.
>
> Same as before -- freedom and low cost. The primary advantae of
> Linux over other OSes is the GPL.
And you would loose exactly these two points for high end servers.
Greetings
Christoph
Alexander Viro wrote:
>
> On 9 Nov 2000, Mike Coleman wrote:
>
> > Alexander Viro <[email protected]> writes:
> > > <shrug> RMS had repeatedly demonstrated what he's worth as a designer
> > > and programmer. Way below zero. You may like or dislike his ideology,
> > > but when it comes to technical stuff... Not funny.
> >
> > Huh?
> >
> > <annoying valspeak tone>
> >
> > *Hello*? GNU gcc? GNU emacs? Way below zero? *Hello*?!?
> >
> > </annoying valspeak tone>
>
> Yes. GNU emacs. Bloated and ugly like hell. gcc. Codebase is anything but
> pretty. Your point being? And no, it's not trolling. RMS really got no
> taste - read his postings on _technical_ GNU lists and see yourself.
Actually emacs is complete crap int terms of both ergonomy and coding.
But the current gcc code base doesn't contain much from RMS anylonger.
The "beauty" of lisp coded in C it is...
>> Why? I think the IBM GKHI code would be of tremendous value. It would
....
>
> And we already refuse to support those kernels - your point being?
>
> Making this "commonplace" is a nightmare. Go away with that.
How is so?
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
> Yes, and that's why I am opposing here: Technically you are right, but
> proposing that enterprise Linux should go this way is inviting binary
> only modules due to the lax handling of modules.
Not so sure it does. If a kernel module wants to make use of GKHI then it
will have to
1) include a GKHI header file or copy some of the code in it,
2) Update kernel source in a minimal way to add the callbacks
Wouldn't 1) under GPL terms force the kernel module to be GPL?
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
>> extensions using the GKHI would not be breaking the license agreement, I
>> don't think. There's lots of binary modules right now -- VMWare, Aureal
> sound card drivers, etc.
>
>All of which just cause large numbers of bugs to go in the bitbucket
because
>nobody can tell whose the problem is.
I don't understand your point - are you saying that the existence of kernel
modules causes makes problems more difficult to solve. Why would that be?
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
> The problem with the hooks et.al. is very simple - they promote every
> bloody implementation detail to exposed API. ....
Surely not, having the kernel source does that. The alternative to the hook
is embed a patch in the kernel source. What proveds greater exposure to
internals: hooks of source?
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
> That being said, the real problem with the GKHI is that as Al said, it
> does expose internal kernel interfaces --- and the Linux kernel
> development community as a whole refuses to be bound by such interfaces,
> sometimes even during a stable kernel series.
I'm not sure that GKHI exposes any more interfaces than embedding a patch
directly into the kernel would.
It has the potential to to make patches easier to re-work for different
kernel versions, and to enable development maintence and fixing of the
patch to be done independently of a kernel build. And it also has the
potential of helping with co-existence. If for example the RAS community
could agree on a number of hooks (I'm thinking here of crash dump, trace,
dprobes and maybe KDB as well) then you'd probably find a good may on them
using then same hooks. The modifications to the kernel would be minimal and
the user would be left an easy means of installing a co-existing subset of
the offerings supported by hooks.
An example: DProbes is down to three hooks - that's three lines of code in
the kernel + three lines in ksyms.c
Patching DProbes onto any custom kernel is a doddle.
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
On Fri, 10 Nov 2000 [email protected] wrote:
>
>
>
> > The problem with the hooks et.al. is very simple - they promote every
> > bloody implementation detail to exposed API. ....
>
> Surely not, having the kernel source does that. The alternative to the hook
> is embed a patch in the kernel source. What proveds greater exposure to
> internals: hooks of source?
Sorry. You don't "embed" the patch. You either get it accepted or not.
Or you fork the tree and then it's officially None Of My Problems(tm).
If your private patch breaks because of the kernel changes - too fscking
bad, it's still None Of My Problems(tm). With hooks you have a published
API.
Alternative to hook is not a random patch. It's finding a way to use public
APIs (possibly at the cost of redesigning your code) or finding good arguments
for changing said APIs (ditto). And they'ld better be really good. "I don't
know how to do that without rethinking my code" doesn't cut it. Deal. There
is _no_ warranty that anything other than public APIs will not change at any
random moment. Period. You play with layering violations - you _will_ shoot
yourself in foot. It's not "if", it's "when".
Alexander "see figure 1" Viro wrote:
> Sorry. You don't "embed" the patch. You either get it accepted or not.
> Or you fork the tree and then it's officially None Of My Problems(tm).
Sounds like a good idea.
-M
how is that any different then a module? modules that are not included
with the kernel source are not guarenteed to work with any other kernel
version (including during the stable kernel series)
David Lang
On Fri, 10 Nov 2000, Alexander Viro wrote:
> On Fri, 10 Nov 2000 [email protected] wrote:
>
> > > The problem with the hooks et.al. is very simple - they promote every
> > > bloody implementation detail to exposed API. ....
> >
> > Surely not, having the kernel source does that. The alternative to the hook
> > is embed a patch in the kernel source. What proveds greater exposure to
> > internals: hooks of source?
>
> Sorry. You don't "embed" the patch. You either get it accepted or not.
> Or you fork the tree and then it's officially None Of My Problems(tm).
> If your private patch breaks because of the kernel changes - too fscking
> bad, it's still None Of My Problems(tm). With hooks you have a published
> API.
>
> Alternative to hook is not a random patch. It's finding a way to use public
> APIs (possibly at the cost of redesigning your code) or finding good arguments
> for changing said APIs (ditto). And they'ld better be really good. "I don't
> know how to do that without rethinking my code" doesn't cut it. Deal. There
> is _no_ warranty that anything other than public APIs will not change at any
> random moment. Period. You play with layering violations - you _will_ shoot
> yourself in foot. It's not "if", it's "when".
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> Please read the FAQ at http://www.tux.org/lkml/
>
On Fri, 10 Nov 2000, Michael Rothwell wrote:
> > Sorry. You don't "embed" the patch. You either get it accepted or not.
> > Or you fork the tree and then it's officially None Of My Problems(tm).
>
> Sounds like a good idea.
It's not a good idea, it's an obvious fact. Oh, you mean forking the tree?
Sure, go ahead - license explicitly allows that. You have to maintain the
thing afterwards, but that's kinda obvious.
I have been wondering what all of the furor has been about...
Initially I thought that it is "a way to load in a module which
defines its own syscalls, etc.." and/or "we want to sell binary
images which can activate some hooks" but having just read the
GKHI README, that thing is far away from its intentions.
(Well, it doesn't preclude those, but neither it mentions them
as objectives. And giving a license stating use of GNU GPL
also doesn't quite fit "proprietary binary hook" image..)
On Wed, Nov 08, 2000 at 04:35:33PM -0500, Michael Rothwell wrote:
> Sounds great; unfortunately, the core group has spoken out against a
> modular kernel.
Really ?
$ /sbin/lsmod
Module Size Used by
parport_pc 23184 1 (autoclean)
lp 5072 0 (autoclean) (unused)
parport 30048 1 (autoclean) [parport_pc lp]
8021q 10032 2
3c59x 24304 2 (autoclean)
ipv6 152816 -1 (autoclean)
autofs 11536 1 (autoclean)
usb-uhci 23408 0 (autoclean) (unused)
usbcore 49504 1 (autoclean) [usb-uhci]
es1371 29920 0
ac97_codec 7824 0 [es1371]
soundcore 4336 4 [es1371]
> -M
>
> [email protected] wrote:
> >
> > We've just release version 0.6 of Generalised Kernel Hooks Interface (GKHI)
> > see the IBM Linux Technology Centre's web page DProbes link:
> > http://oss.software.ibm.com/developerworks/opensource/linux
... (reordered, cut away..)
> > Here's the abstract for this facility. With this intend to modularise our
> > RAS offerings, in particular DProbes, so that they can be applied
> > dynamically without having to be carried as excess baggage.
Richard,
Please educate me, what does "our RAS offerings" mean here ?
(I didn't find "RAS" at your signature-URL site, but I didn't
poke around very much..)
I do know that when IBM suits speak with phrases like that,
they are selling me something which costs $$$.
Which definitely gives proprietary, binary only, hook image...
But GKHI, and DProbes are neither. Thus I am confused, but can
understand the furor...
> > Some folks expressed an interest in this type of facility recently in
> > discussions concerning making call-backs from the kernel to kernel modules.
Indeed, one such mechanism could be a way to register IOCTL
call chains, which now (for sockets) are quite ugly.
Lots and lots of subsystems do ioctl()s via /proc/ objects
just because other methods are way too messy.
[ ioctl's go via the protocol family of the control socket to
family-specific subset, but then the "fun" begins for things
which aren't quite of any specific protocol family -- see
DLCI support hooks at ipv4, and bridge ioctls at both ipv4
and at packet.
Grep the kernel source for "_hook", and you see a lot of
things.. Mostly varying mouses, and bridging, it seems.
Netfilter calls its managed coherent interface "hook", but
it is way better. ]
Also the bridging system is less than desirable looking with
its pervasive hooks, but that can be solved by making layer2
devices fully stackable. (Something for 2.5)
> > Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
> > http://oss.software.ibm.com/developerworks/opensource/linux
> > Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
> > IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
/Matti Aarnio
Matti Aarnio wrote:
> On Wed, Nov 08, 2000 at 04:35:33PM -0500, Michael Rothwell wrote:
> > Sounds great; unfortunately, the core group has spoken out against a
> > modular kernel.
>
> Really ?
>
> $ /sbin/lsmod
> Module Size Used by
> [...]
> soundcore 4336 4 [es1371]
Really. That the kernal has loadable modules does not mean that it is
modular.
-M
From: [email protected]
Date: Fri, 10 Nov 2000 11:41:09 +0000
It has the potential to to make patches easier to re-work for different
kernel versions, and to enable development maintence and fixing of the
patch to be done independently of a kernel build. And it also has the
potential of helping with co-existence. If for example the RAS community
could agree on a number of hooks (I'm thinking here of crash dump, trace,
dprobes and maybe KDB as well) then you'd probably find a good may on them
using then same hooks. The modifications to the kernel would be minimal and
the user would be left an easy means of installing a co-existing subset of
the offerings supported by hooks.
An example: DProbes is down to three hooks - that's three lines of code in
the kernel + three lines in ksyms.c
Patching DProbes onto any custom kernel is a doddle.
Right. So what you're saying is that GKHI is adding complexity to the
kernel to make it easier for peopel to put in non-standard patches which
exposes non-standard interfaces which will lead to kernels not supported
by the Linux Kernel Development Community. Right?
Alternatively, you can argue for specific hooks, and try to see if you
can convince the Kernel Developers (and Linus Torvalds, in
particular) that such hooks are good public interfaces, and that adding
such interfaces are a Good Thing. If such hooks are agreed to, then the
GKHI might be a good wya of implementing these hooks.
But if there are no standard hooks in the mainline kernel, it's going to
be hard to pursuade people that adding the GKHI would be a good thing.
So for the purposes of getting GKHI into the kernel, argueing for GKHI
in the abstract is putting the card before the horse. What I would
recommend is showing how certain hooks are good things to have in the
mainline kernel, and to try to pursuade people of that question first.
Only then try to argue for GKHI.
- Ted
P.S. There are some such RAS features which I wouldn't be surprised
there being interest in having integrated into the kernel directly
post-2.4, with no need to put in "kernel hooks" for that particular
feature. A good example of that would be kernel crash dumps. For all
Linux houses which are doing support of customers remotely, being able
to get a crash dump so that developers can investigate a problem
remotely instead of having to fly a developer out to the customer site
is invaluable. In fact, it might be considerd more valuable than the
kernel debugger....
Hi Theodore,
On Fri, 10 Nov 2000, Theodore Y. Ts'o wrote:
> P.S. There are some such RAS features which I wouldn't be surprised
> there being interest in having integrated into the kernel directly
> post-2.4, with no need to put in "kernel hooks" for that particular
> feature. A good example of that would be kernel crash dumps. For
> all Linux houses which are doing support of customers remotely,
> being able to get a crash dump so that developers can investigate a
> problem remotely instead of having to fly a developer out to the
> customer site is invaluable. In fact, it might be considerd more
> valuable than the kernel debugger....
*Yes* :-)
Greetings
Christoph
On Fri, Nov 10, 2000 at 11:24:28AM -0500, Theodore Y. Ts'o wrote:
> Right. So what you're saying is that GKHI is adding complexity to the
> kernel to make it easier for peopel to put in non-standard patches which
> exposes non-standard interfaces which will lead to kernels not supported
> by the Linux Kernel Development Community. Right?
My understanding is that GKHI does not change the kernel at all, except
for the three hooks needed for dprobes. All GKHI hooks are implemented
as dynamic probes, which are just like debugger breakpoints.
A dynamic probes breakpoint does not require any source
changes, but you have to check the assembly to find the right point for
them (at least in the current version, I don't know if IBM is planning
to support source level dprobes using the debugging information)
IMHO GKHI does not make mainteance of additional modules any easier, because
you always have to recheck the assembly if the dynamic probe still fits
(which may in some cases even be more work than reporting source patches,
it is also harder when you want to cover multiple architectures)
It will just help some people who have a unrational aversion against kernel
recompiles and believe in vendor blessed binaries.
-Andi
> Right. So what you're saying is that GKHI is adding complexity to the
> kernel to make it easier for peopel to put in non-standard patches which
> exposes non-standard interfaces which will lead to kernels not supported
> by the Linux Kernel Development Community. Right?
I don't think I mentioned complexity - in what was do you mean adding
complexity? If you mean addition funciton then yes. If you mean kernel
source modificaiton then no. If you mean the mechanism then it's nothing
special - just what a loader does when it fixes up an external reference.
> But if there are no standard hooks in the mainline kernel, it's going to
> be hard to pursuade people that adding the GKHI would be a good thing.
> So for the purposes of getting GKHI into the kernel, argueing for GKHI
> in the abstract is putting the card before the horse. What I would
> recommend is showing how certain hooks are good things to have in the
> mainline kernel, and to try to pursuade people of that question first.
> Only then try to argue for GKHI.
Quite agree, there are no standard hooks, no hooks at all in fact. I'm
neither seeking to get this accepted or not accepted into the kernel. What
it does do is give me an easier time both as a developer and an installer
when I want to include some rarefied code additions. GKHI was developed
primarily for DProbes.
> P.S. There are some such RAS features which I wouldn't be surprised
> there being interest in having integrated into the kernel directly
> post-2.4,
Great!
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
Christoph Rohland wrote:
>
> Hi Theodore,
>
> On Fri, 10 Nov 2000, Theodore Y. Ts'o wrote:
> > P.S. There are some such RAS features which I wouldn't be surprised
> > there being interest in having integrated into the kernel directly
> > post-2.4, with no need to put in "kernel hooks" for that particular
> > feature. A good example of that would be kernel crash dumps. For
> > all Linux houses which are doing support of customers remotely,
> > being able to get a crash dump so that developers can investigate a
> > problem remotely instead of having to fly a developer out to the
> > customer site is invaluable. In fact, it might be considerd more
> > valuable than the kernel debugger....
>
> *Yes* :-)
As soon as I finish writing raw write disk routines (not using kiobufs),
we can _maybe_ get LKCD accepted one of these days, especially now that we
don't have to build 'lcrash' against a kernel revision. I'm in the
middle of putting together raw IDE functions now -- see LKCD mailing
list for details if you're curious.
IMHO, GKHI is a good thing -- it would be great to see this used for
ASSERT() cases (something you can turn on by 'insmod assert.o', which
would then trigger assert conditionals throughout the kernel ...) I
realize it would mean some bloat, and I doubt Linus would accept it,
but it's a nifty concept for enterprise Linux servers (especially
those that want quick answers to system crashes).
--Matt
> Greetings
> Christoph
Matti,
> Please educate me, what does "our RAS offerings" mean here ?
> (I didn't find "RAS" at your signature-URL site, but I didn't
> poke around very much..)
RAS = Reliabilty, Availability & Serviceability = those things that are are
not mainline to an OS but add the qualities named in the acronym. That
includes self-healing, recoverability, diagnosis etc..
My specialism is in probelm diagnosis. When I say RAS I generally mean
debugging/diagnosis aids, but I also mean it not from a developement
standpoint but from a support standpoint. Which depending on how a system
is deployed can be very different things.
> I do know that when IBM suits speak with phrases like that,
> they are selling me something which costs $$$.
No always. All this stuff (Linux RAS) is free and given away under GPL. And
I'm not waring a suit either ;-) RAS in general is not sexy, it's difficult
to sell. So it's pad for from after sales service and other indirect means.
> Which definitely gives proprietary, binary only, hook image...
> But GKHI, and DProbes are neither. Thus I am confused, but can
> understand the furor...
Well I sort of can and can't as well. Here's a couple of circumstances
where I'd find GKHI useful:
I'm developing DProbes, I need the SGI KDB, a complex patch, as a debugging
aid. I also want to keep up with various kernel version. I've got limited
time so would like to spend it on DProbes on not re-working patches. So, I
know that DProbes only needs to get control in three places in kernel
processing:
1) Trap 1 handler
2) Trap 3 handler
3) Pagefault handler.
I reason that if I had a call inserted into the kernel source at these
points to respective entry points in my DProbes code I'd not have to spend
much time integrating SGI's KDB with DProbes.
The I realise if I leave the calls nop'd and dynamically patch them in
later I can build the kernel once and re-build DProbes (now a module) many
times over - and sometimes without re-booting.
So I create GKHI to mess around with NOPs converitng them into calls.
Second scenario:
I have a customer running Linux for a business purpose. They are not
developers and have no programming skill. Every now and again their system
crashes and they have to reboot. And down time costs them in real terms.
So they say to IBM, you supplied this ... system, you fix it. OK we say,
we'll need a dump. We'll send you a kernel with SGI's crash dump built in.
On no you won't they say, you're not sending us any more dodgy code until
you've fixed this problem. Anyway the server is in a secure remote branch
office. There's no technoical support on site and we cannot possibly have
developers messing with that system. Now suppose SGI or IBM have converted
SGI Kernel Crash Dump to a module and we supplly the system customised with
a few GKHI hooks in place. Then we say issue the following command: insmod
lkcd.o
We get a dump and discover that some cheesehead had overlaid a spinlock
causing re-entrancy and a crash. OK we say we know what happened by not who
did it. So, we need to trace all storage alterations to the spin-lock.
There are only a few valid user's of that spin-lock, which if we had a
trace, we could eliminate. By now DProbes and Linux Trace Toolkit are
working well together, and providing a dynamic trace capability. Also
DProbes is now offering the capability of probes on storage modifications.
So we say to the customer please issue three commands: insmod lkcd; insmod
dprobes; insmod ltt;
The system crashes, we get the trace. We duly find the address from which
the spin-lock was overwriten. We look in the dump and find it's a routine
in a device driver that's been passed invalid data, but actually, not
passed, but placed on a work queue. And furthermore the invalid data has a
particular look to it.
We explain to the customer that we now need just one more pice of
information. So finally we place a dprobe on the enqueuing routine, looking
at data enqueued until the invalid pattern occurs and make the probe
trigger another dump.
And finally we have it. The enqueuing routine was another driver .....
This scenario is not untypical of the sort of problem I earnt my living
solving for the past n years.
Now we could have supplied a system with Crash Dump, Dprobes, LTT, KDB, a
dozen other specialised RAS tools. The kernel would have 50% bigger, cost
us considerable time and effort whenever kernel maintenance was applied -
with obvious consequences. And in the end 99.9% of the time we don't need
these facilities, coz after all Linux is a pretty stable platform. So why
not allow them to be brought in dynamically.
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
No, misunderstood.
GKHI is not implemented using dynamic probes. GKHI places in the kernel
calls to APIs in the DProbes code. Since we'ed rather have Dprobes out of
the kernel then essentially it acts as a loader after the fact, i.e. it
fixes up the DProbes API calls when the DProbe module loads. Compare this
with the usual loading process where the fix-ups are done in the module
being loaded. Now, you might want to ask me why I want DProbes as a module?
They again you might not. Either way is fine by me ;-)
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
Andi Kleen <[email protected]> on 10/11/2000 16:54:05
Please respond to Andi Kleen <[email protected]>
To: "Theodore Y. Ts'o" <[email protected]>
cc: Richard J Moore/UK/IBM@IBMGB, Paul Jakma <[email protected]>, Michael
Rothwell <[email protected]>, Christoph Rohland
<[email protected]>, [email protected]
Subject: Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)
On Fri, Nov 10, 2000 at 11:24:28AM -0500, Theodore Y. Ts'o wrote:
> Right. So what you're saying is that GKHI is adding complexity to the
> kernel to make it easier for peopel to put in non-standard patches which
> exposes non-standard interfaces which will lead to kernels not supported
> by the Linux Kernel Development Community. Right?
My understanding is that GKHI does not change the kernel at all, except
for the three hooks needed for dprobes. All GKHI hooks are implemented
as dynamic probes, which are just like debugger breakpoints.
A dynamic probes breakpoint does not require any source
changes, but you have to check the assembly to find the right point for
them (at least in the current version, I don't know if IBM is planning
to support source level dprobes using the debugging information)
IMHO GKHI does not make mainteance of additional modules any easier,
because
you always have to recheck the assembly if the dynamic probe still fits
(which may in some cases even be more work than reporting source patches,
it is also harder when you want to cover multiple architectures)
It will just help some people who have a unrational aversion against kernel
recompiles and believe in vendor blessed binaries.
-Andi
Date: Fri, 10 Nov 2000 10:36:31 -0800
From: "Matt D. Robinson" <[email protected]>
As soon as I finish writing raw write disk routines (not using kiobufs),
we can _maybe_ get LKCD accepted one of these days, especially now that we
don't have to build 'lcrash' against a kernel revision. I'm in the
middle of putting together raw IDE functions now -- see LKCD mailing
list for details if you're curious.
Great! Are you thinking about putting the crash dumper and the raw
write disk routines in a separate text section, so they can be located
in pages which are write-protected from accidental modification in case
some kernel code goes wild? (Who me? Paranoid? :-)
- Ted
"Theodore Y. Ts'o" wrote:
>
> Date: Fri, 10 Nov 2000 10:36:31 -0800
> From: "Matt D. Robinson" <[email protected]>
>
> As soon as I finish writing raw write disk routines (not using kiobufs),
> we can _maybe_ get LKCD accepted one of these days, especially now that we
> don't have to build 'lcrash' against a kernel revision. I'm in the
> middle of putting together raw IDE functions now -- see LKCD mailing
> list for details if you're curious.
>
> Great! Are you thinking about putting the crash dumper and the raw
> write disk routines in a separate text section, so they can be located
> in pages which are write-protected from accidental modification in case
> some kernel code goes wild? (Who me? Paranoid? :-)
>
> - Ted
We're planning to isolate the write functions as much as possible.
In the past, we've been bitten by this whole concept of Linux "raw I/O".
When I was at SGI, we were able to write to a block device directly
through low-level driver functions that weren't inhibited by any
locking, and that was after shutting down all processors and any
other outstanding interrupts. For Linux, I had given up and stuck
with the raw I/O interpretation of kiobufs, which is just flat out
wrong to do for dumping purposes. Secondly, as Linus said to me a
few weeks ago, he doesn't trust the current disk drivers to be able
to reliably dump when a crash occurs. Don't even ask me to go into
all the reasons kiobufs are wrong for crash dumping. Just read
the code -- it'll be obvious enough.
So I guess after a few months/years, we're finally at a point where
we're saying, "Okay, forget all this, let's do this the right way,
so we don't have these problems anymore." We're removing lcrash from
the kernel, putting it into its own RPM, and adding patches to the
kernel for LKCD that build in crash dump functionality and make a new
"Kernsyms" file so that we can dynamically read the symbol table of
major parts of the kernel and give you memory dumps, stack traces,
and even dump out entire structures dynamically. Then we'll have
the right crash dump mechanism for everyone.
It's time to get RAS moving for Linux. GKHI and LKCD are the first
steps to get us there (IMHO).
--Matt
On Fri, 10 Nov 2000 19:29:26 -0800,
"Matt D. Robinson" <[email protected]> wrote:
>We're removing lcrash from
>the kernel, putting it into its own RPM, and adding patches to the
>kernel for LKCD that build in crash dump functionality and make a new
>"Kernsyms" file so that we can dynamically read the symbol table of
>major parts of the kernel and give you memory dumps, stack traces,
>and even dump out entire structures dynamically.
kallsyms goes a long way towards solving the symbol table problem for
debugging. It really only has three deficiencies, it does not detail
structure fields, it does not handle automatic variables and it does
not have source line numbers. All of those need the sort of detail
provided by gcc -g, but the amount of data that generates is
prohibitively large, 40+ megabytes is a bit much to load into kernel
space. I reluctantly decided that printing global addresses and
offsets was the best I could do, given the space constraints.
Instead of inventing your own kernsyms file, take a look at kallsyms.
It handles modules as well as the kernel. Let me know if you want any
additional data in kallsyms.
Date: Fri, 10 Nov 2000 19:29:26 -0800
From: "Matt D. Robinson" <[email protected]>
We're planning to isolate the write functions as much as possible.
In the past, we've been bitten by this whole concept of Linux "raw I/O".
When I was at SGI, we were able to write to a block device directly
through low-level driver functions that weren't inhibited by any
locking, and that was after shutting down all processors and any
other outstanding interrupts. For Linux, I had given up and stuck
with the raw I/O interpretation of kiobufs, which is just flat out
wrong to do for dumping purposes. Secondly, as Linus said to me a
few weeks ago, he doesn't trust the current disk drivers to be able
to reliably dump when a crash occurs. Don't even ask me to go into
all the reasons kiobufs are wrong for crash dumping. Just read
the code -- it'll be obvious enough.
Oh, yeah, I could have told you that from the beginning. kiobufs were
never intended to be crash-dump friendly. :-) My preference would be
that each block device that was going to be doing crash dumping would
use a special busy-looping driver that's guaranteed never to fail.
(Sort of like how the serial console driver is done; it's separate from
the rest of the driver, and does not depend on interrupts working.)
Hence my comment about putting that separate bit of code in a page which
is write-protected and segregated from everything else....
- Ted
On 2000-11-10T19:12:29,
"Theodore Y. Ts'o" <[email protected]> said:
> Great! Are you thinking about putting the crash dumper and the raw
> write disk routines in a separate text section, so they can be located
> in pages which are write-protected from accidental modification in case
> some kernel code goes wild? (Who me? Paranoid? :-)
I would also suggest to have a little checksum over the relevant pages, to
verify that the code is still correct and we are not going to crashdump all
over our valuable data...
And I am still very fond of the idea of crash dumping to a network server ;-)
Sincerely,
Lars Marowsky-Br?e <[email protected]>
Development HA
--
Perfection is our goal, excellence will be tolerated. -- J. Yahl
Lars Marowsky-Bree wrote:
> And I am still very fond of the idea of crash dumping to a network server ;-)
I second that. Serial can be slow, and has that pesky cable-length
limit...
Andi Kleen wrote:
> It will just help some people who have a unrational aversion against
kernel
>recompiles and believe in vendor blessed binaries.
An interesting remark Andi, especially in the light of your note to me
regarding your use of DProbes - i.e. you'd rather use DProbes to dump out
some info from the kernel than recompile it with printks.
I don't have an aversion to recompiling the kernel - it's great fun - I
love watching all the meeages go by, waiting with bated breath for a
compile error, which never seems to happen. Just like watching the National
Lottery, waiting for your own numbers to come up.
To be a little more serious, it's not recompilation that's a problem, its
re-working a set of (non-standard) patches together. I'm not that excited
by that - I'd rather develop new code than rework old. Anyway for a couple
of example scenarios see the response I made to Michael Rothwell. And by
the way, I absolutely agree with your approach to kernel problem solving -
but wouldn't it be a help if you didn't have to put a large or even
moderate effort into working the DProbes patch into some hot-off-the-press
version of the kernel?
Richard
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
Alexander Viro wrote:
> It's not a good idea, it's an obvious fact. Oh, you mean forking the
tree?
Again I find your terminology at odds with mine; what do you mean by
forking the tree? I get the impression that it's a very restrictive notion
where any functional ehancement applied as a patch on top of a standard
distribution kernel is considered by you as forking? Is that so? (And BTW
by patch I mean input to the patch command.)
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
[email protected] wrote:
>
> Date: Fri, 10 Nov 2000 19:29:26 -0800
> From: "Matt D. Robinson" <[email protected]>
>
> We're planning to isolate the write functions as much as possible.
> In the past, we've been bitten by this whole concept of Linux "raw I/O".
> When I was at SGI, we were able to write to a block device directly
> through low-level driver functions that weren't inhibited by any
> locking, and that was after shutting down all processors and any
> other outstanding interrupts. For Linux, I had given up and stuck
> with the raw I/O interpretation of kiobufs, which is just flat out
> wrong to do for dumping purposes. Secondly, as Linus said to me a
> few weeks ago, he doesn't trust the current disk drivers to be able
> to reliably dump when a crash occurs. Don't even ask me to go into
> all the reasons kiobufs are wrong for crash dumping. Just read
> the code -- it'll be obvious enough.
>
> Oh, yeah, I could have told you that from the beginning. kiobufs were
> never intended to be crash-dump friendly. :-) My preference would be
> that each block device that was going to be doing crash dumping would
> use a special busy-looping driver that's guaranteed never to fail.
> (Sort of like how the serial console driver is done; it's separate from
> the rest of the driver, and does not depend on interrupts working.)
> Hence my comment about putting that separate bit of code in a page which
> is write-protected and segregated from everything else....
This is also needed for single-machine source level kernel debugging.
--
Daniel
On Sun, Nov 12, 2000 at 11:27:26PM +0000, [email protected] wrote:
>
>
>
> Andi Kleen wrote:
> > It will just help some people who have a unrational aversion against
> kernel
> >recompiles and believe in vendor blessed binaries.
>
>
> An interesting remark Andi, especially in the light of your note to me
> regarding your use of DProbes - i.e. you'd rather use DProbes to dump out
> some info from the kernel than recompile it with printks.
When I wrote it I was still misunderstanding GKHI's nature (I was assuming
that it worked on top of dprobes, not under it -- I should have read the
source before commenting, my bad)
I think using dprobes for collecting information is ok, but when you want
to do actual actions with it (not only using it as a debugger) IMHO it
is better to patch and recompile the kernel.
>
> I don't have an aversion to recompiling the kernel - it's great fun - I
> love watching all the meeages go by, waiting with bated breath for a
> compile error, which never seems to happen. Just like watching the National
> Lottery, waiting for your own numbers to come up.
>
> To be a little more serious, it's not recompilation that's a problem, its
> re-working a set of (non-standard) patches together. I'm not that excited
> by that - I'd rather develop new code than rework old. Anyway for a couple
> of example scenarios see the response I made to Michael Rothwell. And by
> the way, I absolutely agree with your approach to kernel problem solving -
> but wouldn't it be a help if you didn't have to put a large or even
> moderate effort into working the DProbes patch into some hot-off-the-press
> version of the kernel?
As far as I can see GKHI is overkill for dprobes alone, the existing
notifier lists would be sufficient because dprobes does not hook into any
performance critical paths.
-Andi
This subject came up in the Generalized Kernel Hooks Interface thread, since it
is an area of interest to me I wanted to continue that conversation.
While I do not think it would be productive to enter a discussion whether there
is a need to fork the kernel to add features that would be beneficial to
mission/business critical applications, I am curious as to what are the features
that people consider important to have. By mission critical I mean systems that
if not functional bring a business to a halt, while by business critical I mean
systems that affect a division of a business.
Another problem is how people define Enterprise Systems. Many base it on the
definitions that go back to S390 systems, others in the context of the 24/7
nature of the internet. That would also be a healthy discussion to have.
At Oracle we are primarily interested on I/O subsystem features and memory
management. (For anyone that knows anything about Oracle this should not come
as a surprise, although I am pretty sure that any database vendor/developer
would be interested on those features as well.)
Regards,
--
=======================================================================
Josue Emmanuel Amaro [email protected]
Linux Products Manager
Intel and Linux Technologies Group
=======================================================================
On Mon, Nov 13, 2000 at 11:38:23AM +0100, Andi Kleen wrote:
> notifier lists would be sufficient because dprobes does not hook into any
> performance critical paths.
Current dprobes patch adds branches in the the main page fault handler,
device_not_available exception at least. Those are _very_ hot paths.
I had a look at gkhi and infact my guess is that the main reason of
implementing gkhi in first place is probably because they had to remove the
branches from the fast paths when dprobes is inactive by using self modifying
code to only have some additional nop in the exception fast paths during
production, and while doing that they probably choosed to generalize the
lightweight self modifying hook interface instead of doing a magic inside the
dprobes patch.
But (unless I'm missing something in the code :) gkhi has absolutely _nothing_
new with respect to binary modules except being able to introduce less costly
hooks in term of performance _only_when_ the hooks are unregistered (= when
machine is in production). It's only a matter of performance as far I can see.
To introduce a new hook using gkhi it's necessary to change the kernel sources,
recompile the kernel just like for every other module registration hook that we
just have in 2.4.0-test11-pre2. The modules using gkhi interface needs the hook
details (were the self modifying code is placed) exported via /proc/ksyms
infact, just like for any other regular hook registered via the init_module
without using self modifying code.
So binary only modules using gkhi _can't_ have _any_ "binary level" advantage
compared to all other regular binary only modules for 2.2.x and 2.4.x that
doesn't use gkhi.
And even if they did the other way around - so having gkhi on top of dprobes -
they would not only been dependent on symbols, but they would depend on almost
everything including a recompile of the kernel that doesn't affect the symbols
and they would be still add branches in the exception handlers as current
dprobe patch is doing. Not to tell that the dprobes hooks are very inefficient
and unsuitable for productions if they're recalled at high frequency in fast
paths since there are all generating CPU exceptions that have the same latency
of an enter/exit kernel every time an hook triggers plus there's the
interpreter that must execute the hook.
I think gkhi is good idea to be able to implement lightweight hooks that are
unregistered during production as it avoid the cost of dereferencing pointers
in a linked list to know if we have any hook registered or not. dprobes is
perfect user of the lightweight gkhi interface. However they're useful __only__
when the hook is unregistred all the time during production. For example
replacing the binfmt chain with gkhi would decrease performance because
gkhi force the hook handling to happen out of line. I don't think there's any
hook in a hot path and unregistered all the time in the current kernels so I
think only point of merging gkhi would be only to merge a more efficient
implementation of dprobes.
I think gkhi should be renamed to something like "Fast Unregistered Kernel
Hook Interface" to avoid confusion and wrong usage of it that would otherwise
leads to lower performance.
I've a few comments about the implementation:
__asm__ volatile (".global GKHook"h";
.global GKHook"h"_ret;
.global GKHook"h"_bp;
GKHook"h": jmp GKHook"h"_bp; /* nop; to activate */
^^^^^^^^^^^^^^^^
push $0;
nop;nop;nop;nop;nop; /* jmp GKHook"h"_dsp when active*/
GKHook"h"_ret: add $4,%%esp;
GKHook"h"_bp:;":::"%eax")
^^^^
1)
It would be better to have only 5 nop instructions (instead of more
instructions and a jmp over them) and patch them at runtime as soon as somebody
registers the hook to generate a `call hook_handler_1'. This will also save
icache.
For the atomicity of the operation in SMP (of the update of the 5 nops to call
the dispatcher) we should probably have a fixup entry in place for the GKHook
address, so that we can fixup the 5th byte after lefting an invalid opcode at
the first byte.
2)
eax needs to be saved internally to the hook. We must only add the 5 nop
penality in the unregistered case and primarly on x86 with so few regs we
really want to have the fast path not having to save/restore eax on the stack
across every nop gkhi hook.
3)
gkhi apparently doesn't yet know the word "SMP" 8).
4)
gkh_iflush should be done with flush_icache_range that is infact implemented
wrong for IA32 and it should be implemented as regs trashing cpuid (the fact
it's wrongly implemented means that in theory modules can break in IA32
on 2.4.x 2.2.x even on UP).
5)
Current dprobes v1.1.1 against 2.2.16 cames with a syscall collision:
sys_dprobes collides with ugetrlimit (not implemented in 2.2.x). That's fine
for internal use and to show the code, but make _sure_ not to ship any binary
to anybody implementing ugetrlimit as sys_dprobes 8).
Richard please ask Linus to reserve a syscall for dprobes. I recommend to
allocate the syscall out of the way (I mean using syscall 255 or even better
enlarging the syscall table from 256 to 512 and using number 511) so we make
sure not to waste precious dcachelines for debugging stuff.
BTW, for things like lkcd there's no reason to use gkhi to make it completly
modular since lkcd gets recalled in a completly slow path.
Andrea
Andi Kleen wrote:
>I think using dprobes for collecting information is ok, but when you want
>to do actual actions with it (not only using it as a debugger) IMHO it
>is better to patch and recompile the kernel.
I absolutely agree. The only time I ever used this capability was to modify
a proprietary binary, for which I did not have the source, so that I could
prove to the owner what needed fixing.
>As far as I can see GKHI is overkill for dprobes alone, the existing
>notifier lists would be sufficient because dprobes does not hook into any
>performance critical paths.
Again, I agree. My intent is that the RAS guys might club together - then
GKHI make much more sense.
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
On 2000-11-13T13:56:16,
Josue Emmanuel Amaro <[email protected]> said:
Good morning Josue,
I hope your certification matrix hasn't driven you mad yet ;-)
> While I do not think it would be productive to enter a discussion whether
> there is a need to fork the kernel to add features that would be beneficial
> to mission/business critical applications, I am curious as to what are the
> features that people consider important to have.
This is in fact the valuable subpart of the discussion.
Working for SuSE on High Availability, especially in the "enterprise" segment:
Here, referring to systems running databases (mostly Oracle, surprise),
ERP-Systems, but also providing services (NFS, Samba, firewalls) in such an
environment.
I personally need features which allow me to keep on running, shut down as
gracefully as possible if an error occurs, and if an error occured, diagnose
it out in the field.
This means: ECC memory, hotpluggable everything, proper error handling and
reporting in the kernel. Yes, christmas and easter do occur on the same day in
the real world, unfortunately.
This can best be summarised as "robustness".
If an error occured, I need to be able to fully diagnose it without having to
reproduce it - no, I do not wish to reproduce the error by crashing my
critical server on purpose, nor is "The error appears to have gone away, we
have no clue what it was" an acceptable answer. (kdb, LKCD, Oopsing to the
network etc: And they must be part of the default kernel as far as possible,
so they stay in sync and get widespread testing)
But also scalability: 2TB is a problem for me in some cases, 32bit just don't
cut it all the time - but I need to circumvent the storage problem even on a
32bit system. And adding disks to the system while running is desireable.
Cluster awareness, again mostly referring to storage: Yes, there is more than
one system accessing my SCSI bus, my FCAL RAID, and the error handling should
be architected in a way that they do not start reset wars.
The LVM should safeguard against multiple nodes changing the metadata. (Ok,
this can be solved in userspace too) LVM must be transactional, so a crash on
a node doesn't corrupt the data.
Basically, the talks in Miami (The Second Annual Linux Storage Management
Workshop) gave a great overview of everything I need.
And: I need all of this as Open Source. Period. No binary kernel modules do me
any good and I will pointedly ignore them.
Oh, and by the way - if any hot kernel hacker, not yet working on this full
time feels inspired to make this happen, contact me. Or any other Linux
company, as long as the job gets done. We'll be glad to make you a fulltime
kernel slave^Whacker! ;-)
> Another problem is how people define Enterprise Systems. Many base it on the
> definitions that go back to S390 systems, others in the context of the 24/7
> nature of the internet. That would also be a healthy discussion to have.
_
24/7 * 99.99% mission/business critical services with "medium to high" load.
Sincerely,
Lars Marowsky-Br?e <[email protected]>
Development HA
--
Perfection is our goal, excellence will be tolerated. -- J. Yahl
Josue Emmanuel Amaro <[email protected]>:
> This subject came up in the Generalized Kernel Hooks Interface thread, since it
> is an area of interest to me I wanted to continue that conversation.
>
> While I do not think it would be productive to enter a discussion whether there
> is a need to fork the kernel to add features that would be beneficial to
> mission/business critical applications, I am curious as to what are the features
> that people consider important to have. By mission critical I mean systems that
> if not functional bring a business to a halt, while by business critical I mean
> systems that affect a division of a business.
>
> Another problem is how people define Enterprise Systems. Many base it on the
> definitions that go back to S390 systems, others in the context of the 24/7
> nature of the internet. That would also be a healthy discussion to have.
>
> At Oracle we are primarily interested on I/O subsystem features and memory
> management. (For anyone that knows anything about Oracle this should not come
> as a surprise, although I am pretty sure that any database vendor/developer
> would be interested on those features as well.)
I reformatted/phrased your questions above to allow for separate answers:
Q1. How do you define Enterprise Systems? Many base it on the definitions that
go back to S390 systems, others in the context of the 24/7 nature of the
internet.
1. The system should be available 24 hours a day, 7 days a week, 52 weeks a
year :-), with time off for scheduled down time for maintenance and
upgrades.
2. It should be possible to take down a node of a cluster without affecting
the effectiveness of the other nodes. There is an expeced higher load on
the remaining nodes during the time the node is missing.
3. It should be possible to add nodes to a cluster without affecting the
effectiveness of the other nodes.
4. Unauthorized access to, modification to, or damage to the effectiveness of
the system should be possible (the ideal...). All security related events
should be audited and logged.
Q2. I am curious as to what are the features that people consider important to
have. By mission critical I mean systems that if not functional bring a
business to a halt, while by business critical I mean systems that affect
a division of a business.
1. Secure - Multi-level security (with compartmentalization) is needed to
be able to detect unauthorized attempts to modify the system. There should
be no all powerfull user. System updates should require three different
authorizations (security, administrator, and auditor) to take effect when
the system is on-line. All bets are off, of course, if the system is taken
offline for such modifications - at that point, the administrator would
be able to make any changes. The security administrator should validate
the system in some manner. The system should not be able to be brought
online without being validated.
IPSec to provide controled encryption between hosts. Inclusion of CIPSO
style extensions to allow for labeled network support. Virtical integration
to include user identification tags as well. I would like to be able to
identify the remote user, with confidence in that identity established
by the confidence in the host, which is in turn established by IPSec.
I (meaning me) would like the ability to audit every system call. (yes,
this is horrendous, if everything is logged, but I want to be able to
choose how much is logged at the source of the data, rather than at
the destination. That would reduce the total data flood to what is
manageable or desired.
I realize that this is extreme - but in some environments this degree of
control is necessary. It should be possible to downgrade this level of
control to the point that is required for other environments.
2. Allow for full accounting of user resources - memory, cpu, disk, IO.
3. It should not be possible for a user to exceed the resource quotas
established for the user. This control should be flexible enough to allow
for exceeding some quotas if additional resources are available, but
unused. (I'm considering memory resources and CPU time here. The user
should be able to exceed memory quota, but with the understanding that
the users processes will be trimmed down to the users quota limit if
needed for other users.
4. Batch jobs, using a more common definition of batch than that used by
the "batch" utility - job queues, with batch controled limits, job
checkpointing/restart, resource allocation controls... Batch jobs
should be able to migrate to other nodes/systems (as long as all other
required resources are available ... This is HARD to do :-).
5. Allow for multiple scheduling types, preferably concurrently, but changable
at runtime. Some real time, mostly batch and interactive.
-------------------------------------------------------------------------
Jesse I Pollard, II
Email: [email protected]
Any opinions expressed are solely my own.
On Tue, Nov 14, 2000 at 08:59:49AM -0600, Jesse Pollard wrote:
[snip]
> 4. Unauthorized access to, modification to, or damage to the
> effectiveness of the system should be possible (the ideal...).
> All security related events should be audited and logged.
Uhmmm, there should be some kind of negation above, shouldn't it?!
[snip]
/David Weinehall
_ _
// David Weinehall <[email protected]> /> Northern lights wander \\
// Project MCA Linux hacker // Dance across the winter sky //
\> http://www.acc.umu.se/~tao/ </ Full colour fire </
One historically significant "Enterprise" OS is Multics. It had nine
major goals. Perhaps we should think about how Linux measures up to
these 1965 goals for "Enterprise Computing."
1) Convenient remote terminal use.
Telnet, ssh, X windows, rsh, vnc, "screen," ethernet, serial, etc. I
think we have this one.
2) Continuous operation analogous to power & telephone services.
No way. Multics could have a whole bank of memory fail and keep running.
You could add CPUs, memory and IO devices while it was running without
interrupting users' work. Of course, a lot of this cannot be
accomplished due to the brain-dead hardware designs (especially PC)
prevalent today. However, Linux does not have any support for this type
of facility. I recently saw a patch to let Linux use partially bad
memory. This is a step in the right direction. The ability to scale the
system while on-line is extremely valuable.
3) A wide range of system configurations, changeable without system or
user program reorganization.
See comments in #2. Plus, the recent two-kernel-monte, linux-from-linux
type stuff would be especially excellent if it will allow the kernel to
be upgraded on the fly. If it could save state and have the new kernel
take over, that would rock. On a smaller scale, allowing this for
modules would be good. This would allow the upgrade of nic drivers, etc.
The GKHI would also be invaluable if it would allow the
replacement/augmentation of certain other subsystems on the fly -- such
as the scheduler, VFS, etc.
4) A high reliability internal file system.
Ext2 + bdflush + kupdated? Not likely. To quote the Be Filesystems
book, Ext2 throws safety to the wind to achieve speed. This also ties
into Linux' convoluted VM system, and is shot in the foot by NFS. We
would need minimally a journaled filesystem and a clean VM design,
probably with a unified cache (no separate buffer, directory entry and
page caches). The Tux2 Phase Trees look to be a good step in the
direction as well, in terms of FS reliability. The filesystem would have
to do checksums on every block. Some type of mirroring/clustering would
be good. And the ability to grow filesystems online, and replace disks
online, would be key. If your disks are getting old, you may want to
pre-emptively replace them with faster, newer, larger ones with more
MTBF left.
5) Support for selective information sharing.
Linux has a rather poor security model -- the Unix one. It needs ACLs no
only on filesystem objects, but on other OS features as well; such as
the ability to use network interfaces, packet types, display ACLs,
console ACLs, etc. If there's a function, it probably needs an ACL.
6) Hierarchical structures of information for system administration and
decentralization of user activities.
See #5. Linux really does not support delgation of authority well.
There's one user who can reconfigure/admin the system: root. Tools like
sudo only make you root for a moment, and don't inherently limit you to
that one activity. ACLs on most if not all system attributes and objects
would go a long way towards decentralized but safe administration.
7) Support for a wide range of applications.
Well... anything wih source or compiled for the Linux ABI. A
microkernel-type architecture with servers would provide a lot more of
this. See QNX, NT, Mach.
8) Support for multiple programming environments & human interfaces.
Many languages are supported by Linux. This is good. Linux has two
humnan interfaces: CLI and X Windows guis. I'm not sure what could be
added, except for voice input.
9) The ability to evolve the system with changes in technology and in
user aspirations.
Ties ties into #2 and #3and #5. Otherwise, rewrites of the Linux kernel
and userspace accomplish this. Unfortunately, that requires taking the
system, or minimally its applications, down.
Just some thoughts from 35 years ago. Please add your $0.02.
-M
Michael Rothwell wrote:
> Just some thoughts from 35 years ago. Please add your $0.02.
What's that $0.02 worth after 35 years of inflation?
=)
On Tue, 14 Nov 2000, Michael Rothwell wrote:
> One historically significant "Enterprise" OS is Multics. It had nine
> major goals. Perhaps we should think about how Linux measures up to
> these 1965 goals for "Enterprise Computing."
>
Multics??? No way. It was abandoned as unusable and part of the
kernel code, basically the boot loader, was modified to become
part of Unix.
You have way too many persons on this list who know the history of
Unix to try this BS.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.
Mike Dresser wrote:
> What's that $0.02 worth after 35 years of inflation?
About $6
"Richard B. Johnson" wrote:
> Multics??? [..] way too many persons on this list who know the history of
> Unix to try this BS.
So, you're saying their nine goals were bullshit? Multics had a lot of
problems. But it did a lot of ground-breaking. Perhaps you should reply
to the nine goals, or the general topic of "Enterpriseness," rather than
merely express your irrelevant hatred for Multics.
Michael Rothwell wrote:
>
> Mike Dresser wrote:
>
> > What's that $0.02 worth after 35 years of inflation?
>
> About $6
Sorry.. six times a much... not six dollars. Which means $0.02 circa
1965 is 'worth' $0.12 today, given an average annual devaluation of the
currency of 5.2% since 1965.
-M
> > Multics??? [..] way too many persons on this list who know the history of
> > Unix to try this BS.
>
> So, you're saying their nine goals were bullshit? Multics had a lot of
> problems. But it did a lot of ground-breaking. Perhaps you should reply
> to the nine goals, or the general topic of "Enterpriseness," rather than
> merely express your irrelevant hatred for Multics.
Linux is a good Unix. if adding "enterpriseness" means violating its
Unixness, then yes, the goals are bullshit. in particular, the kind
of extensive, kernel-based auditing and accounting some people talk about,
as well as the complete evisceration of the Unix design for security,
would make Linux no Unix at all. that would be very bad, and indeed Multics
is a fine example of the kind of history we should not repeat.
On Tue, 14 Nov 2000, Michael Rothwell wrote:
> "Richard B. Johnson" wrote:
> > Multics??? [..] way too many persons on this list who know the history of
> > Unix to try this BS.
>
> So, you're saying their nine goals were bullshit? Multics had a lot of
> problems. But it did a lot of ground-breaking. Perhaps you should reply
> to the nine goals, or the general topic of "Enterpriseness," rather than
> merely express your irrelevant hatred for Multics.
>
Relating some "nine goals of 'Enterprise Computing'" to Multics is
the bullshit. When Multics was being developed, the singular goal
was to make an operating system that worked on DEC Equipment without
having to use DEC software. The emphasis was on trying to make it
work period.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.
At 11:20 AM 11/14/00, Mike Dresser wrote:
>Michael Rothwell wrote:
>
> > Just some thoughts from 35 years ago. Please add your $0.02.
>
>What's that $0.02 worth after 35 years of inflation?
>
>=)
I'd say inflation has been easily 12x since then. So $0.02 is now worth
$0.25, i.e. the 2 cents of yesteryear is now 2 bits :-)
--------------------------------------------------------
David Relson Osage Software Systems, Inc.
[email protected] 514 W. Keech Ave.
http://www.osagesoftware.com Ann Arbor, MI 48103
voice: 734.821.8800 fax: 734.821.8800
At 11:41 AM 11/14/00, Richard B. Johnson wrote:
>On Tue, 14 Nov 2000, Michael Rothwell wrote:
>
> > "Richard B. Johnson" wrote:
> > > Multics??? [..] way too many persons on this list who know the
> history of
> > > Unix to try this BS.
> >
> > So, you're saying their nine goals were bullshit? Multics had a lot of
> > problems. But it did a lot of ground-breaking. Perhaps you should reply
> > to the nine goals, or the general topic of "Enterpriseness," rather than
> > merely express your irrelevant hatred for Multics.
> >
>
>Relating some "nine goals of 'Enterprise Computing'" to Multics is
>the bullshit. When Multics was being developed, the singular goal
>was to make an operating system that worked on DEC Equipment without
>having to use DEC software. The emphasis was on trying to make it
>work period.
DEC? Try GE, specifically the GE-645 (if my memory hasn't lost any bits).
Speaking of Multics, the last Multics system was just recently
decomissioned. I think 35 years is a very impressive lifetime for a
computer system. Linux, now at age 9, only has 26 years to go.
David
--------------------------------------------------------
David Relson Osage Software Systems, Inc.
[email protected] 514 W. Keech Ave.
http://www.osagesoftware.com Ann Arbor, MI 48103
voice: 734.821.8800 fax: 734.821.8800
On Tue, Nov 14, 2000 at 11:41:33AM -0500, Richard B. Johnson wrote:
> On Tue, 14 Nov 2000, Michael Rothwell wrote:
>
> > "Richard B. Johnson" wrote:
> > > Multics??? [..] way too many persons on this list who know the history of
> > > Unix to try this BS.
> >
> > So, you're saying their nine goals were bullshit? Multics had a lot of
> > problems. But it did a lot of ground-breaking. Perhaps you should reply
> > to the nine goals, or the general topic of "Enterpriseness," rather than
> > merely express your irrelevant hatred for Multics.
> >
>
> Relating some "nine goals of 'Enterprise Computing'" to Multics is
> the bullshit. When Multics was being developed, the singular goal
> was to make an operating system that worked on DEC Equipment without
> having to use DEC software. The emphasis was on trying to make it
> work period.
Ummm, the way I parse the above statement, you are saying that Multics was
developed to work on DEC equipment without having to use DEC software. Maybe
we are inhabiting parallel universes, but I'm pretty sure that in my universe,
Multics ran first on GE computers, and then on Honeywell computers when
Honeywell bought the division from GE. Note, DEC did bid for the Multics
contract but was turned down. Maybe you are thinking of Tenex or UNIX?
The original machine was a GE-645, which was a segmented, virtual memory system
using 36 bit words. The operating system and system software was written in
PL/1. Bell Labs had bought a GE-645 and was one of the three development
partners (along with GE and MIT) until they withdrew in April 1969. You might
want to browse:
http://www.multicians.org/
for more of the history of Multics.
--
Michael Meissner, Red Hat, Inc.
PMB 198, 174 Littleton Road #3, Westford, Massachusetts 01886, USA
Work: [email protected] phone: +1 978-486-9304
Non-work: [email protected] fax: +1 978-692-4482
On Tue, Nov 14, 2000 at 12:14:57PM -0500, Michael Rothwell wrote:
> Ext2 + bdflush + kupdated? Not likely. To quote the Be Filesystems
> book, Ext2 throws safety to the wind to achieve speed. This also ties
What safety problems bdflush/kupdated have? (if something they lacks in
performance since they works on a global dirty list while it should be per
queue dirty list to take the I/O pipeline full on all disks)
> [..] And the ability to grow filesystems online, [..]
This is provided in linux for ages by LVM+reiserfs also in 2.2.x.
Any filesystem with inode map and not dependent on disk offsets to find
stuff (exept the superblock of course) can do the grow almost trivially
and online, the shrink is some more complicated instead.
Andrea
"Richard B. Johnson" wrote:
> Relating some "nine goals of 'Enterprise Computing'" to Multics is
> the bullshit.
Funny, I got those off the "Multics FAQ" page.
-M
On Tue, Nov 14, 2000 at 08:59:49AM -0600, Jesse Pollard wrote:
> I (meaning me) would like the ability to audit every system call. (yes,
> this is horrendous, if everything is logged, but I want to be able to
> choose how much is logged at the source of the data, rather than at
> the destination. That would reduce the total data flood to what is
> manageable or desired.
Yes, you really need to control the logging at the source of the data. To do
that we need to use to use self modifying tricks as dprobes and GKHI does to
provide "fast unregistered hooks".
o with dprobes the hooks will be _absolutely_ zero cost if they're
unregistered but they're very costly (basically enter/exit kernel
for every hook) when they're registered (so they're ok if
your auditing doesn't happen all the time).
dprobe hooks also binds you to a certain binary image (not a generic
interface stable across different kernel binaries)
o GKHI provides fast unregistered hooks too, if implemented with
5 nops as suggested they will cost around 3 cycles when they're
unregistered, and they will provide good performance also when
they're registered (no enter/exit kernel as dprobes needs
to do) and you can control them via userspace without being dependent
on binary offsets (just like with every other hook we have in kernel
just now but with the difference that our current hooks aren't self
modifying so they adds branches also when they're unregistered so
they're bad for usages like syscall auditing). However bloating every
syscall with tons of GKHI hooks isn't possible either or it will become
a performance problem too eventually. It depends what you need exactly.
I'd say one GKHI hook per syscall shouldn't have measurable impact on
performance.
Andrea
At 01:10 PM 11/14/00 -0500, Michael Rothwell wrote:
>"Richard B. Johnson" wrote:
>
> > Relating some "nine goals of 'Enterprise Computing'" to Multics is
> > the bullshit.
>
>Funny, I got those off the "Multics FAQ" page.
It may be reasonable to question them as "goals of 'Enterprise Computing'".
I found, on http://www.multicians.org/general.html, a list of those same
nine goals, introduced by the sentence "As described in the 1965 paper
Introduction and Overview of the Multics System by Corbat? and Vyssotsky,
there were nine major goals for Multics:"
While those were the goals of Multics, it is not at all clear that Multics
would classify these days as a platform for "Enterprise Computing". I'll
note that the word "enterprise" does not appear in either the general FAQ
page I cited, nor in the linked article it cites.
Michael Rothwell writes:
> 4) A high reliability internal file system.
>
> Ext2 + bdflush + kupdated? Not likely. To quote the Be Filesystems
> book, Ext2 throws safety to the wind to achieve speed. This also ties
> into Linux' convoluted VM system, and is shot in the foot by NFS. We
> would need minimally a journaled filesystem and a clean VM design,
> probably with a unified cache (no separate buffer, directory entry and
> page caches). The Tux2 Phase Trees look to be a good step in the
> direction as well, in terms of FS reliability.
Ext3 is doing pretty well...
> The filesystem would have
> to do checksums on every block. Some type of mirroring/clustering would
> be good. And the ability to grow filesystems online, and replace disks
> online, would be key. If your disks are getting old, you may want to
> pre-emptively replace them with faster, newer, larger ones with more
> MTBF left.
You can always do this in the hardware - no reason to do it in software.
If you are using RAID 5, and you wanted to take the performance hit you
could always calculate the checksums for each stripe on read to detect
errors. You may even be able to add a second parity disk to do ECC on
the disk data.
As for online resizing, you can do this with ext2 for a long time with
my ext2online patches/tools. LVM lets you migrate between disks online.
You need hardware support (available) to do hot-swap disks - SCSI works,
but I don't think the IDE code is ready for hot-swap yet.
Cheers, Andreas
--
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert
On Tue, 14 Nov 2000, Michael Meissner wrote:
> On Tue, Nov 14, 2000 at 11:41:33AM -0500, Richard B. Johnson wrote:
> > On Tue, 14 Nov 2000, Michael Rothwell wrote:
> >
> > > "Richard B. Johnson" wrote:
> > > > Multics??? [..] way too many persons on this list who know the history of
> > > > Unix to try this BS.
> > >
> > > So, you're saying their nine goals were bullshit? Multics had a lot of
> > > problems. But it did a lot of ground-breaking. Perhaps you should reply
> > > to the nine goals, or the general topic of "Enterpriseness," rather than
> > > merely express your irrelevant hatred for Multics.
> > >
> >
> > Relating some "nine goals of 'Enterprise Computing'" to Multics is
> > the bullshit. When Multics was being developed, the singular goal
> > was to make an operating system that worked on DEC Equipment without
> > having to use DEC software. The emphasis was on trying to make it
> > work period.
>
> Ummm, the way I parse the above statement, you are saying that Multics was
> developed to work on DEC equipment without having to use DEC software. Maybe
> we are inhabiting parallel universes, but I'm pretty sure that in my universe,
> Multics ran first on GE computers, and then on Honeywell computers when
> Honeywell bought the division from GE. Note, DEC did bid for the Multics
> contract but was turned down. Maybe you are thinking of Tenex or UNIX?
>
> The original machine was a GE-645, which was a segmented, virtual memory system
> using 36 bit words. The operating system and system software was written in
> PL/1. Bell Labs had bought a GE-645 and was one of the three development
> partners (along with GE and MIT) until they withdrew in April 1969. You might
> want to browse:
>
No parallel universe. When Multics was being developed by AT&T,
it was found to be unusable on the DEC. It was a PDP-8, so the
story is told. General Electric got the first contract to make
a machine specifically designed for Multics and development
continued.
The original DEC was "given" to W. M. Ritchie and his staff in
"Department 58213". He wanted to use it for games. To do so, required
him to write some sort of OS, which became Unix.
As I said, when Multics was designed, the only criteria as to
get it to work on a DEC. It didn't. To use this development as
an example of "enterprise computing" is absurd and belies its
well documented history.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.
On Tue, 14 Nov 2000, Michael Rothwell wrote:
> "Richard B. Johnson" wrote:
>
> > Relating some "nine goals of 'Enterprise Computing'" to Multics is
> > the bullshit.
>
> Funny, I got those off the "Multics FAQ" page.
>
> -M
>
History is being rewritten. When Multics was being developed by AT&T,
it was found to be unusable on the DEC. It was a PDP-8, so the
story is told. General Electric got the first contract to make
a machine specifically designed for Multics and development
continued.
The original DEC was "given" to W. M. Ritchie and his staff in
"Department 58213". He wanted to use it for games. To do so, required
him to write some sort of OS, which became Unix.
As I said, when Multics was designed, the only criteria as to
get it to work on a DEC. It didn't. To use this development as
an example of "enterprise computing" is absurd and belies its
well documented history.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.
Sorry, wrong answer, but thanks for playing.
When Multics was developed, (early 60s,) DEC equipment wasn't even
interesting to much of an audience. The original equipment Multics ran on
was build by one of the "seven dwarf" computer companies, (GE) which was
soon to get out of the computer business altogether.
I would suggest Organick's book, if I could recall the title.
Marty
-----Original Message-----
From: Richard B. Johnson [mailto:[email protected]]
Sent: Tuesday, November 14, 2000 8:42 AM
To: Michael Rothwell
Cc: Linux kernel
Subject: Re: Advanced Linux Kernel/Enterprise Linux Kernel
On Tue, 14 Nov 2000, Michael Rothwell wrote:
> "Richard B. Johnson" wrote:
> > Multics??? [..] way too many persons on this list who know the history
of
> > Unix to try this BS.
>
> So, you're saying their nine goals were bullshit? Multics had a lot of
> problems. But it did a lot of ground-breaking. Perhaps you should reply
> to the nine goals, or the general topic of "Enterpriseness," rather than
> merely express your irrelevant hatred for Multics.
>
Relating some "nine goals of 'Enterprise Computing'" to Multics is
the bullshit. When Multics was being developed, the singular goal
was to make an operating system that worked on DEC Equipment without
having to use DEC software. The emphasis was on trying to make it
work period.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.
Sorry, wrong answer, but thanks for playing. Multics was not abandoned as
unusable, and was, in fact, widely used, sometimes in what would now be
called "mission critical" applications, for a long time. While Honeywell
finally stopped supporting Multics sometime in the 90s, I was surprised and
delighted to find that there are still Multics systems running.
There may be many people on this list who know the history of Unix, but from
this thread, I'm thinking that perhaps there is some confusion between the
history and the mythology.
Perhaps we could get AT&T, Lucent, or whomever owns the copyright these
days, to reprint the "Unix" issue of the Bell Systems Journal.
Marty
-----Original Message-----
From: Richard B. Johnson [mailto:[email protected]]
Sent: Tuesday, November 14, 2000 8:26 AM
To: Michael Rothwell
Cc: Linux kernel
Subject: Re: Advanced Linux Kernel/Enterprise Linux Kernel
On Tue, 14 Nov 2000, Michael Rothwell wrote:
> One historically significant "Enterprise" OS is Multics. It had nine
> major goals. Perhaps we should think about how Linux measures up to
> these 1965 goals for "Enterprise Computing."
>
Multics??? No way. It was abandoned as unusable and part of the
kernel code, basically the boot loader, was modified to become
part of Unix.
You have way too many persons on this list who know the history of
Unix to try this BS.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.
Actually, you have the sequence of events slightly out of order. AT&T,
specifically Bell Labs, was one of the participants in the program that
would develop Multics. AT&T opted out of the program, for various reasons,
but it continued apace. The PDP-8 of fame was one that, according to
Thompson, happened to be available and unused.
-----Original Message-----
From: Richard B. Johnson [mailto:[email protected]]
Sent: Tuesday, November 14, 2000 10:01 AM
To: Michael Rothwell
Cc: Linux kernel
Subject: Re: Advanced Linux Kernel/Enterprise Linux Kernel
On Tue, 14 Nov 2000, Michael Rothwell wrote:
> "Richard B. Johnson" wrote:
>
> > Relating some "nine goals of 'Enterprise Computing'" to Multics is
> > the bullshit.
>
> Funny, I got those off the "Multics FAQ" page.
>
> -M
>
History is being rewritten. When Multics was being developed by AT&T,
it was found to be unusable on the DEC. It was a PDP-8, so the
story is told. General Electric got the first contract to make
a machine specifically designed for Multics and development
continued.
The original DEC was "given" to W. M. Ritchie and his staff in
"Department 58213". He wanted to use it for games. To do so, required
him to write some sort of OS, which became Unix.
As I said, when Multics was designed, the only criteria as to
get it to work on a DEC. It didn't. To use this development as
an example of "enterprise computing" is absurd and belies its
well documented history.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.
On Tue, 14 Nov 2000, Richard B. Johnson wrote:
> On Tue, 14 Nov 2000, Michael Rothwell wrote:
>
> > One historically significant "Enterprise" OS is Multics. It had nine
> > major goals. Perhaps we should think about how Linux measures up to
> > these 1965 goals for "Enterprise Computing."
> >
>
> Multics??? No way. It was abandoned as unusable and part of the
> kernel code, basically the boot loader, was modified to become
> part of Unix.
"take a look at the goals Multics had" != "please reimplement Multics"
Flaming is ok, but you should really read the WHOLE email
before replying, otherwise you might end up with a flame
that isn't relevant at all to the email it supposedly is
a reply to...
Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000
http://www.conectiva.com/ http://www.surriel.com/
Dick Johnson wrote:
> The original DEC was "given" to W. M. Ritchie and his staff in
> "Department 58213". He wanted to use it for games. To do so, required
> him to write some sort of OS, which became Unix.
A typo, I assume. That's D(ennis) Ritchie.
> As I said, when Multics was designed, the only criteria as to
> get it to work on a DEC. It didn't. To use this development as
> an example of "enterprise computing" is absurd and belies its
> well documented history.
How odd then, that Corbato's '65 paper specifically describes it as a
research effort on a GE system, and both Ritchie and Thompson have written
to similar effect and Glasser et al wrote
In the late spring and early summer of 1964 it became obvious that greater
facility in the computing system was required if time-sharing techniques
were to move from the state of an interesting pilot experiment into that of
a useful prototype for remote access computer systems. Investigation proved
computers that were immediately available could not be adapted readily to
meet the difficult set of requirements time-sharing places on any machine.
However, there was one system that appeared to be extendible into what was
desired. This machine was the General Electric 635.
Multics grew out of research into the design of timesharing systems at MIT,
and is from the same family of systems as ITS. It had a long and
interesting history and was supported by Honeywell into the 90s.
There were several other interesting OSes developed in that time frame, such
as SDS's CP/V for the Sigma series, but most of them were not described in
the literature and so are long forgotten.
Marty
I would agree that Multics probably wouldn't qualify as a platform for
enterprise computing these days, but it is interesting to examine the 9
stated goals, and see how they relate to enterprise computing. They are
applicable goals, although they certainly don't qualify as the only set, and
could well be expanded given what has been learned in the 35 years since.
Marty
-----Original Message-----
From: Buddha Buck [mailto:[email protected]]
Sent: Tuesday, November 14, 2000 9:52 AM
To: Michael Rothwell; [email protected]
Cc: Linux kernel
Subject: Re: Advanced Linux Kernel/Enterprise Linux Kernel
At 01:10 PM 11/14/00 -0500, Michael Rothwell wrote:
>"Richard B. Johnson" wrote:
>
> > Relating some "nine goals of 'Enterprise Computing'" to Multics is
> > the bullshit.
>
>Funny, I got those off the "Multics FAQ" page.
It may be reasonable to question them as "goals of 'Enterprise Computing'".
I found, on http://www.multicians.org/general.html, a list of those same
nine goals, introduced by the sentence "As described in the 1965 paper
Introduction and Overview of the Multics System by Corbat? and Vyssotsky,
there were nine major goals for Multics:"
While those were the goals of Multics, it is not at all clear that Multics
would classify these days as a platform for "Enterprise Computing". I'll
note that the word "enterprise" does not appear in either the general FAQ
page I cited, nor in the linked article it cites.
if you look at the kstat structure under solaris, there's a lot of info
there that'd be good to be able to pull out of the linux kernel. that
would slow down the kernel a little, lead to some 'bloat' that linus
abhors and such, but its good to have that information for monitoring and
debugging problems. it'd also be nice to have hooks built in to monitor
errors in the disk subsystem and ideally warn of impending failures as
much as possible -- that might be better done in userspace. and pretty
much you want all error messages from different subsystems to be monitored
and appropriate action taken. ideally all error messages from the kernel
and device drivers should be standardized and HA software can then monitor
them and take appropriate actions when it sees one that indicates
failure. sun is currently in the process of documenting all the kernel
error messages from what i understand. those are the kinds of things that
give IT managers and sysadmins warm fuzzy feelings about solaris.
On Mon, 13 Nov 2000, Josue Emmanuel Amaro wrote:
> This subject came up in the Generalized Kernel Hooks Interface thread, since it
> is an area of interest to me I wanted to continue that conversation.
>
> While I do not think it would be productive to enter a discussion whether there
> is a need to fork the kernel to add features that would be beneficial to
> mission/business critical applications, I am curious as to what are the features
> that people consider important to have. By mission critical I mean systems that
> if not functional bring a business to a halt, while by business critical I mean
> systems that affect a division of a business.
>
> Another problem is how people define Enterprise Systems. Many base it on the
> definitions that go back to S390 systems, others in the context of the 24/7
> nature of the internet. That would also be a healthy discussion to have.
>
> At Oracle we are primarily interested on I/O subsystem features and memory
> management. (For anyone that knows anything about Oracle this should not come
> as a surprise, although I am pretty sure that any database vendor/developer
> would be interested on those features as well.)
>
> Regards,
>
> --
> =======================================================================
> Josue Emmanuel Amaro [email protected]
> Linux Products Manager
> Intel and Linux Technologies Group
> =======================================================================
>
>
>
In article <[email protected]> you wrote:
> 2) Continuous operation analogous to power & telephone services.
> No way. Multics could have a whole bank of memory fail and keep running.
> You could add CPUs, memory and IO devices while it was running without
> interrupting users' work. Of course, a lot of this cannot be
> accomplished due to the brain-dead hardware designs (especially PC)
> prevalent today. However, Linux does not have any support for this type
> of facility. I recently saw a patch to let Linux use partially bad
> memory. This is a step in the right direction. The ability to scale the
> system while on-line is extremely valuable.
Motorola has announced Linux Systems with a Hot-Plug CPU - but this seems
to be more a hardware then a software feature.
> 4) A high reliability internal file system.
> Ext2 + bdflush + kupdated? Not likely. To quote the Be Filesystems
> book, Ext2 throws safety to the wind to achieve speed.
No. Shure, you don't have all the nice features of log structured or
journaled filesystem, but ext2 is pretty reliable for a traditional fs.
(I'd like to see if the multics fs was better, do you have a pointer?)
> This also ties
> into Linux' convoluted VM system, and is shot in the foot by NFS. We
> would need minimally a journaled filesystem and a clean VM design,
> probably with a unified cache (no separate buffer, directory entry and
> page caches).
The dcache is not a disk cache. It caches directory lookups, it is
neither something VM related nor does it inpact reliability.
The buffercache seems pretty dead in the near future - filesystems
are going towards putting metadata in the page cache.
(See Al Viro's ext2 patches)
> The Tux2 Phase Trees look to be a good step in the
> direction as well, in terms of FS reliability.
> The filesystem would have to do checksums on every block.
The filesystem? This does not belong into the filesystem. An ecc
personality for md might be a much better idea ...
> Some type of mirroring/clustering would be good.
Mirroring does _not_ belong into the fs layer, it belongs into LVM, software -,
or if you want a really reliable system, hardware raid.
> And the ability to grow filesystems online, and replace disks
> online, would be key. If your disks are getting old, you may want to
> pre-emptively replace them with faster, newer, larger ones with more
> MTBF left.
Why don't you use LVM?
> 5) Support for selective information sharing.
> Linux has a rather poor security model -- the Unix one. It needs ACLs no
> only on filesystem objects, but on other OS features as well; such as
> the ability to use network interfaces, packet types, display ACLs,
> console ACLs, etc. If there's a function, it probably needs an ACL.
ACLs are not really interesting. They are like good-old file rights with
some nice new features and much more complicated. You want MACs and
Capabilities (the latter are implemented in Linux).
> 6) Hierarchical structures of information for system administration and
> decentralization of user activities.
> See #5. Linux really does not support delgation of authority well.
> There's one user who can reconfigure/admin the system: root.
No, there is not. There is a capability for each job (or a group of jobs).
The root user is just UNIX-Legacy. (ok, ok nearly every system has one -
but the Linux security model doesn't really need it).
> 7) Support for a wide range of applications.
> Well... anything wih source or compiled for the Linux ABI. A
> microkernel-type architecture with servers would provide a lot more of
> this. See QNX, NT, Mach.
Shure. NT supports win32, win16, dos and recompiled UNIX binaries.
QNX supports QNX and now Linux binaries.
Thanks to the personality architecture I can (and do) run UnixWare,
OpenServer, etc binaries under Linux. And I can use dosemu, wine, etc ...
Christoph
--
Always remember that you are unique. Just like everyone else.
On Tue, 14 Nov 2000, Richard B. Johnson wrote:
> On Tue, 14 Nov 2000, Michael Rothwell wrote:
>
> > One historically significant "Enterprise" OS is Multics. It had nine
> > major goals. Perhaps we should think about how Linux measures up to
> > these 1965 goals for "Enterprise Computing."
> >
> 2
> Multics??? No way. It was abandoned as unusable and part of the
> kernel code, basically the boot loader, was modified to become
> part of Unix.
>
> You have way too many persons on this list who know the history of
> Unix to try this BS.
Perhaps this is a call for forking off linux into other project, It is not
unreasonable, if goals of one do not match with the other. =)
However the guy said that goals are applicable to linux, he did not say
rewrite linux to be multics...
Pavel.
On Tue, 14 Nov 2000, Mark Hahn wrote:
> Linux is a good Unix. if adding "enterpriseness" means violating its
> Unixness, then yes, the goals are bullshit. in particular, the kind
> of extensive, kernel-based auditing and accounting some people talk about,
> as well as the complete evisceration of the Unix design for security,
> would make Linux no Unix at all. that would be very bad, and indeed Multics
> is a fine example of the kind of history we should not repeat.
Are you one of those people who say X11 sucks? And whole idea of a
neworked gui sucks? Like ones who threaten to rewrite entire gui without
planning, in OpenGL and use the rest of the energy tarishing X?
Well, if it is done, probably you won't have to compile those extensions
in anyway, so what's the whining about? Enterprise level features will
let linux run on Mainframes not in pitiful x86 emulation mode, with hot
swappable hardware. Just like raid. Would you say RAID sucks a** and
weighs down linux, because it is an enterprise feature? By the tone of
your voice it seems so.
I am for a concious featurism, add features as long as others are stable
and fixed(not in MS way)
pavel
Marty Fouts writes:
> Actually, you have the sequence of events slightly out of order. AT&T,
> specifically Bell Labs, was one of the participants in the program that
> would develop Multics. AT&T opted out of the program, for various reasons,
> but it continued apace. The PDP-8 of fame was one that, according to
> Thompson, happened to be available and unused.
The original system on which UNIX development started was not a PDP-8,
but a PDP-7. The earliest UNIX was also written in assembler. Thompson
and Ritchie developed C as a higher-level implementation language during
the process of porting UNIX from the PDP-7 to the PDP-11.
On Tue, 14 Nov 2000, Richard B. Johnson wrote:
> On Tue, 14 Nov 2000, Michael Rothwell wrote:
>
> > "Richard B. Johnson" wrote:
> > > Multics??? [..] way too many persons on this list who know the history of
> > > Unix to try this BS.
> >
> > So, you're saying their nine goals were bullshit? Multics had a lot of
> > problems. But it did a lot of ground-breaking. Perhaps you should reply
> > to the nine goals, or the general topic of "Enterpriseness," rather than
> > merely express your irrelevant hatred for Multics.
> >
>
> Relating some "nine goals of 'Enterprise Computing'" to Multics is
> the bullshit. When Multics was being developed, the singular goal
> was to make an operating system that worked on DEC Equipment without
> having to use DEC software. The emphasis was on trying to make it
> work period.
WTF are you smoking? Multics never ran on DEC hardware. Moreover, your
ideas about UNIX history ("modified Multics bootloader" bit) are, well,
interesting. Porting from GE to PDP-7... <shudder> Besides, the thing
had wildly different fs usage model - "everything is mmaped segment" vs.
UNIX "everything is stream of characters".
Michael Rothwell wrote:
> 2) Continuous operation analogous to power & telephone services.
>
> No way. Multics could have a whole bank of memory fail and keep running.
[...]
Considering that it's very cheap nowadays to have redundancy at the
box level, designs attempting to achieve robustness at the component
level may fail to benefit from changes in the hardware market in the
last few decades. Maybe we need a Beowulf for fault tolerance ...
> 3) A wide range of system configurations, changeable without system or
> user program reorganization.
I'd say we're largely there: /proc/sys and modules give you a lot of
freedom, if properly used.
> See comments in #2. Plus, the recent two-kernel-monte, linux-from-linux
> type stuff would be especially excellent if it will allow the kernel to
> be upgraded on the fly.
Difficult, because there's no reliable (= automated) means of tracking
data structures an the semantics of internal interfaces from kernel to
kernel. Feasible for selected subsystems and with coarse granularity,
though. (E.g. modules.)
> console ACLs, etc. If there's a function, it probably needs an ACL.
Empirical evidence with VMS suggests that overly sophisticated
security mechanisms can actually decrease overall security, because
they may lead people to micro-manage security and to neglect the
overall security concept.
> 7) Support for a wide range of applications.
>
> Well... anything wih source or compiled for the Linux ABI. A
> microkernel-type architecture with servers would provide a lot more of
> this. See QNX, NT, Mach.
Hmm, I don't think you want to say this :)
> 9) The ability to evolve the system with changes in technology and in
> user aspirations.
That's perhaps the most important point. The computing environment
has changed a lot. Take for example Google's PC farm and compare
this with the mainframe approach one would have chosen twenty or
thirty years ago. Mainframes are still the right answer for some
problems, but their role in the solution may be very different from
the days when they were the only solution, and all of the solution.
- Werner
--
_________________________________________________________________________
/ Werner Almesberger, ICA, EPFL, CH [email protected] /
/_IN_N_032__Tel_+41_21_693_6621__Fax_+41_21_693_6610_____________________/
On Tue, 14 Nov 2000, Richard B. Johnson wrote:
> On Tue, 14 Nov 2000, Michael Rothwell wrote:
>
> > "Richard B. Johnson" wrote:
> >
> > > Relating some "nine goals of 'Enterprise Computing'" to Multics is
> > > the bullshit.
> >
> > Funny, I got those off the "Multics FAQ" page.
> >
> > -M
> >
>
>
> History is being rewritten. When Multics was being developed by AT&T,
> it was found to be unusable on the DEC. It was a PDP-8, so the
> story is told. General Electric got the first contract to make
> a machine specifically designed for Multics and development
> continued.
>
> The original DEC was "given" to W. M. Ritchie and his staff in
> "Department 58213". He wanted to use it for games. To do so, required
> him to write some sort of OS, which became Unix.
>
> As I said, when Multics was designed, the only criteria as to
> get it to work on a DEC. It didn't. To use this development as
> an example of "enterprise computing" is absurd and belies its
> well documented history.
<SARCASM>
But .. but... but they said so on slashdot. That must make it true. ;)
</SARCASM>
--
Gerhard Mack
[email protected]
<>< As a computer I find your faith in technology amusing.
On Tuesday 14 November 2000 03:43 pm, Steve VanDevender wrote:
>Marty Fouts writes:
> > Actually, you have the sequence of events slightly out of order.
> > AT&T, specifically Bell Labs, was one of the participants in the
> > program that would develop Multics. AT&T opted out of the
> > program, for various reasons, but it continued apace. The PDP-8
> > of fame was one that, according to Thompson, happened to be
> > available and unused.
>
>The original system on which UNIX development started was not a
> PDP-8, but a PDP-7. The earliest UNIX was also written in
> assembler. Thompson and Ritchie developed C as a higher-level
> implementation language during the process of porting UNIX from the
> PDP-7 to the PDP-11.
I haven't seen anybody point at this URL
http://cm.bell-labs.com/cm/cs/who/dmr/hist.html
where you can read about early Unix story from the horse's mouth
himself. The paper is 20 years old, but still full of very tasty
tidbits. Browse around and you'll find many more interesting goodies.
In particular, if you want to know the story of how C came to be, as
told by its creator, try this URL
http://cm.bell-labs.com/cm/cs/who/dmr/chist.html
Take it from somebody who has been hacking all sorts of Unix and
Unix-like kernels for the last 25 years: Those who don't know Unix
history are doomed to reimplement it badly.
Er, um, yes. I stand corrected.
-----Original Message-----
From: Steve VanDevender [mailto:[email protected]]
Sent: Tuesday, November 14, 2000 11:44 AM
To: Marty Fouts
Cc: '[email protected]'; Michael Rothwell; Linux kernel
Subject: RE: Advanced Linux Kernel/Enterprise Linux Kernel
Marty Fouts writes:
> Actually, you have the sequence of events slightly out of order. AT&T,
> specifically Bell Labs, was one of the participants in the program that
> would develop Multics. AT&T opted out of the program, for various
reasons,
> but it continued apace. The PDP-8 of fame was one that, according to
> Thompson, happened to be available and unused.
The original system on which UNIX development started was not a PDP-8,
but a PDP-7. The earliest UNIX was also written in assembler. Thompson
and Ritchie developed C as a higher-level implementation language during
the process of porting UNIX from the PDP-7 to the PDP-11.
Michael Rothwell writes:
> 1) Convenient remote terminal use.
>
> Telnet, ssh, X windows, rsh, vnc, "screen," ethernet,
> serial, etc. I think we have this one.
Nope: /dev/audio, /dev/cdrom, /dev/floppy, fonts, etc.
Also one would want a local window manager for performance,
but this tends to interfere with starting apps on the other
system.
> 4) A high reliability internal file system.
Now we want it distributed, with version control, with
mirroring onto N of M machines and migration toward usage...
> 5) Support for selective information sharing.
>
> Linux has a rather poor security model -- the Unix one.
> It needs ACLs no only on filesystem objects, but on other
> OS features as well; such as the ability to use network
> interfaces, packet types, display ACLs, console ACLs, etc.
It would have been nice to have just put 2 entries right
in the inode years ago. With the KISS method, we'd be using
ACLs right now. Even just a list of UIDs that would share
permission bits with the file's GID would be very useful.
I just want to share a file with one other person!
Andrea,
I am very greatful for your detailed analysis. I have yet to digest
everything you commented but will get back to you on all points you raise
very soon. Here are my thoughts so far:
> I think gkhi should be renamed to something like "Fast Unregistered
Kernel
> Hook Interface" to avoid confusion and wrong usage of it that would
otherwise
> leads to lower performance.
A fair point.
On point 3):
> 3)
>
> gkhi apparently doesn't yet know the word "SMP" 8).
When I announced GKHI I did state that SMP support was to follow. The
updates are trivial but I didn't wan't to release the code until I had had
a chance to test it.
On point 4)
> 4)
>
> gkh_iflush should be done with flush_icache_range that is infact
implemented
> wrong for IA32 and it should be implemented as regs trashing cpuid (the
fact
> it's wrongly implemented means that in theory modules can break in IA32
> on 2.4.x 2.2.x even on UP).
Are you claiming that flush_icache_range has an error and should implement
the IA32 instruction flush as I did using CPUID? If this is the case has
this error been officially reported?
On point 5)
5)
> Current dprobes v1.1.1 against 2.2.16 cames with a syscall collision:
> sys_dprobes collides with ugetrlimit (not implemented in 2.2.x). That's
fine
> for internal use and to show the code, but make _sure_ not to ship any
binary
> to anybody implementing ugetrlimit as sys_dprobes 8).
>
> Richard please ask Linus to reserve a syscall for dprobes. I recommend to
> allocate the syscall out of the way (I mean using syscall 255 or even
better
> enlarging the syscall table from 256 to 512 and using number 511) so we
make
> sure not to waste precious dcachelines for debugging stuff.
Thanks for this information. Reserving a syscall will become irrelvant when
we release Dprobes as a module using gkhi since we will use ioctl() as the
application interface.
> BTW, for things like lkcd there's no reason to use gkhi to make it
completly
> modular since lkcd gets recalled in a completly slow path.
Well, not necessarily so while lkcd is not get accepted into the standard
kernel source. But also, even when lkcd becomes accepted, using gkhi with
lkcd will allow a crash dump capability to be actived dynamically. That
gives the user more fexibility. Even enterprise customers can sometimes
hedge their bets when it comes to RAS-like features.
Richard Moore - RAS Project Lead - Linux Technology Centre (PISC).
http://oss.software.ibm.com/developerworks/opensource/linux
Office: (+44) (0)1962-817072, Mobile: (+44) (0)7768-298183
IBM UK Ltd, MP135 Galileo Centre, Hursley Park, Winchester, SO21 2JN, UK
Please respond to Andrea Arcangeli <[email protected]>
To: Richard J Moore/UK/IBM@IBMGB
cc:
Subject: Re: [ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)
On Wed, Nov 15, 2000 at 05:14:57AM +0000, [email protected] wrote:
>
>
> Andrea,
>
> I am very greatful for your detailed analysis. I have yet to digest
> everything you commented but will get back to you on all points you raise
> very soon. Here are my thoughts so far:
I'm glad you appreciated my comments. I think dprobes gives an higher
level of flexibility for debugging purposes and I'd really like to
include it in the aa kernels until it will be included into the mainstream.
> When I announced GKHI I did state that SMP support was to follow. The
I probably overlooked that part of your announcement, sorry.
> updates are trivial but I didn't wan't to release the code until I had
had
> a chance to test it.
Very promising.
Note that SMP could introduce non trivial issues: the self modifying
changes
should be atomic with respect of the other CPUs executing the self
modifying
code and specs are often not very explicit about side effects of self
modifying
code in SMP, it's not only a matter of implementing the GKHI locks with SMP
locks).
> Are you claiming that flush_icache_range has an error and should
implement
> the IA32 instruction flush as I did using CPUID? If this is the case has
Exactly.
> this error been officially reported?
I hope I did that in my email :). Actually when I fixed the alpha port
some month ago (alpha needs an explicit imb() to flush the speculative
icache and it was really crashing in modules because of the missing
smp_imb) I
also noticed IA32 was buggy, since also IA32 execute out of order and specs
says we must do a cpuid as only way to serialize the istream. But I didn't
fixed IA32 because nobody ever got bitten by that race because of
timing/implementation reasons, but to be correcet we should do cpuid also
in
flush_icache_range.
Once flush_icache_range is fixed in IA32 you can use it inside GKHI too
(and then you'll get it right on all architectures).
> Thanks for this information. Reserving a syscall will become irrelvant
when
> we release Dprobes as a module using gkhi since we will use ioctl() as
the
> application interface.
Ok (still you need to reserve a blockdevice major minor number with Linus
though).
> Well, not necessarily so while lkcd is not get accepted into the standard
> kernel source. [..]
It won't until it uses a separate driver that doesn't depend on scsi or
ide layer.
Even ignoring the safety problem of scsi layer potentially corrupted by
memory corruption at crash time, the scsi layer doesn't work without being
interrupt driven. It will recurse on the stack badly if somebody ever tries
to
use it polled. Probably similar thing happens with IDE (but none IDE polled
hardware exists so we don't know). I documented all this in the `Linux
Kernel
Debugging' document on my ftp area in ftp.suse.com.
> [.] But also, even when lkcd becomes accepted, using gkhi with
> lkcd will allow a crash dump capability to be actived dynamically. [..]
We can control everything dynamically without self modifying code.
The _only_ point of self modifying code is performance. None other reason
to use it. lkcd is definitely called in an extremely slow path (infact
if all goes right it should never be recalled), so it doesn't give
any advantage to use self modifying code there.
> [..] That
> gives the user more fexibility. Even enterprise customers can sometimes
> hedge their bets when it comes to RAS-like features.
I agree that being able to enable/disable lkcd dynamically is fine
feature.
Andrea
[email protected] wrote:
> > Well, not necessarily so while lkcd is not get accepted into the standard
> > kernel source. [..]
>
> It won't until it uses a separate driver that doesn't depend on scsi or
> ide layer.
We're working on it ... can't quit my day job, you know ... :)
--Matt
In article <[email protected]> you wrote:
> But also scalability: 2TB is a problem for me in some cases, 32bit just don't
> cut it all the time - but I need to circumvent the storage problem even on a
> 32bit system. And adding disks to the system while running is desireable.
Why do you run 32bit Systems in the First Place? This can solve a lot of
flaky PC Hardware Issues, too... (also it does not help in Hotplug PCI and
CPU).
Greetings
Bernd
Michael Rothwell wrote:
> 4) A high reliability internal file system.
>
> Ext2 + bdflush + kupdated? Not likely. To quote the Be Filesystems
> book, Ext2 throws safety to the wind to achieve speed. This also ties
> into Linux' convoluted VM system, and is shot in the foot by NFS. We
> would need minimally a journaled filesystem and a clean VM design,
> probably with a unified cache (no separate buffer, directory entry and
> page caches). The Tux2 Phase Trees look to be a good step in the
> direction as well, in terms of FS reliability. The filesystem would have
> to do checksums on every block.
Actually, I was planning on doing on putting in a hack to do something
like that: calculate a checksum after every buffer data update and check
it after write completion, to make sure nothing scribbled in the buffer
in the interim. This would also pick up some bad memory problems.
> Some type of mirroring/clustering would
> be good. And the ability to grow filesystems online, and replace disks
> online, would be key. If your disks are getting old, you may want to
> pre-emptively replace them with faster, newer, larger ones with more
> MTBF left.
This is all coming, but as you say, it's not quite here yet.
--
Daniel
Daniel Phillips <[email protected]> writes:
> Actually, I was planning on doing on putting in a hack to do something
> like that: calculate a checksum after every buffer data update and check
> it after write completion, to make sure nothing scribbled in the buffer
> in the interim. This would also pick up some bad memory problems.
Be very careful that this just applies to metadata. For normal data
this is a valid case. Weird but valid.
Eric
"Eric W. Biederman" wrote:
>
> Daniel Phillips <[email protected]> writes:
>
> > Actually, I was planning on doing on putting in a hack to do something
> > like that: calculate a checksum after every buffer data update and check
> > it after write completion, to make sure nothing scribbled in the buffer
> > in the interim. This would also pick up some bad memory problems.
>
> Be very careful that this just applies to metadata. For normal data
> this is a valid case. Weird but valid.
I'm not sure what you mean. With the exception of mmap'd files, the
filesystem (or VFS) controls every transfer onto a buffer so... what
does that leave?
--
Daniel
Hi!
> > book, Ext2 throws safety to the wind to achieve speed. This also ties
> > into Linux' convoluted VM system, and is shot in the foot by NFS. We
> > would need minimally a journaled filesystem and a clean VM design,
> > probably with a unified cache (no separate buffer, directory entry and
> > page caches). The Tux2 Phase Trees look to be a good step in the
> > direction as well, in terms of FS reliability. The filesystem would have
> > to do checksums on every block.
>
> Actually, I was planning on doing on putting in a hack to do something
> like that: calculate a checksum after every buffer data update and check
> it after write completion, to make sure nothing scribbled in the buffer
> in the interim. This would also pick up some bad memory problems.
You might want to take look to a patch with crc loop option.
It does verify during read, not during write; but that's even better because
that way you pick up problems in IO subsystem, too.
--
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.
Pavel Machek wrote:
> > Actually, I was planning on doing on putting in a hack to do something
> > like that: calculate a checksum after every buffer data update and check
> > it after write completion, to make sure nothing scribbled in the buffer
> > in the interim. This would also pick up some bad memory problems.
>
> You might want to take look to a patch with crc loop option.
>
> It does verify during read, not during write; but that's even better because
> that way you pick up problems in IO subsystem, too.
You would have to store the checksums on the filesystem then, or use a
verify-after-write. What I was talking about is a
verify-the-buffer-didn't get scribbled. I'd then trust the hardware to
report a write failure. Note that if something scribbles on your buffer
between the time you put good data on it and when it gets transfered to
disk, you can verify perfectly and still have a hosed filesystem.
It was pointed out that you can't really do what I'm suggesting for
mmaped file data, and there's some truth to that - but certainly the
interval between when ->writepage gets called and when the actual buffer
write happens can be secured in this way. Doing this only for metadata
is also a good idea because then the overhead would be close to nil and
the basic fs integrity would be protected.
--
Daniel
Hi!
> > > Actually, I was planning on doing on putting in a hack to do something
> > > like that: calculate a checksum after every buffer data update and check
> > > it after write completion, to make sure nothing scribbled in the buffer
> > > in the interim. This would also pick up some bad memory problems.
> >
> > You might want to take look to a patch with crc loop option.
> >
> > It does verify during read, not during write; but that's even better because
> > that way you pick up problems in IO subsystem, too.
>
> You would have to store the checksums on the filesystem then, or use a
I store checksums in separate partition.
> verify-after-write. What I was talking about is a
> verify-the-buffer-didn't get scribbled. I'd then trust the hardware to
> report a write failure. Note that if something scribbles on your buffer
> between the time you put good data on it and when it gets transfered to
> disk, you can verify perfectly and still have a hosed filesystem.
Actually, I have 50% chance detecting that corruption. If it happens
between application and loop, I detect nothing. If it happens between
loop and disk, I catch it.
Pavel
--
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.
