hi,
as i love to put my oar in where it's unlikely that people
will listen, and as i have little to gain or lose by doing
so, i figured you can decide for yourselves whether to be
selectively deaf or not:
here's my take on where i believe the linux kernel needs to go,
in order to keep up.
what prompted me to send this message now was a recent report
where linus' no1 patcher is believed to be close to overload,
and in that report, i think it was andrew morton, it was
said that he believed the linux kernel development rate to be
slowing down, because it is nearing completion.
i think it safe to say that a project only nears completion
when it fulfils its requirements and, given that i believe that
there is going to be a critical shift in the requirements, it
logically follows that the linux kernel should not be believed
to be nearing completion.
with me so far? :)
okay, so what's the bit that's missing that mr really irritating,
oh-so-right and oh-so-killfile-ignorable luke kenneth casson
kipper lozenge has spotted that nobody else has, and what's
the fuss about?
well... to answer that, i need to outline a bit about processor
manufacturing: if you are familiar with processor design please
forgive me, this is for the benefit of those who might not be.
the basic premise: 90 nanometres is basically... well...
price/performance-wise, it's hit a brick wall at about 2.5Ghz, and
both intel and amd know it: they just haven't told anyone.
anyone (big) else has a _really_ hard time getting above 2Ghz,
because the amount of pipelining required is just... insane
(see recent ibm powerpc5 see slashdot - what speed does it do?
surprise: 2.1Ghz when everyone was hoping it would be 2.4-2.5ghz).
a _small_ chip design company (not an IBM, intel or amd)
will be lucky to see this side of 1Ghz, at 90nm.
also, the cost of mask charges for 90nm is insane: somewhere
around $2million and that's never going to go away.
the costs for 65nm are going to be far far greater than that,
and 45nm i don't even want to imagine what they're going to be.
plus, there's a problem of quantum mechanics, heat dissipation
and current drain that makes, with current manufacturing
techniques, the production of 65nm and 45nm chips really
problematic.
with present manufacturing techniques, the current drain and heat
dissipation associated with 45nm means that you have to cut the number
of gates down to ONE MILLION, otherwise the chip destroys itself.
(brighter readers might now have an inkling of where i'm going
with this - bear with me :)
compare that one million gates with the present number of gates in an
AMD or x86 chip - some oh, what, 20 million?
now you get it?
for the present insane uniprocessor architectures at least
(and certainly for the x86 design), 90nm is _it_ - and yet,
people demand ever more faster and faster amounts of processing,
and no amount of trying on the part of engineers can get round
the laws of physics.
so, what's the solution?
well.... it's to back to parallel processing techniques, of course.
and, surprise surprise, what do we have intel pushing?
apart from, of course, the performance per watt metric (which,
if you read a few paragraphs back, you realise why they have to
trick both their customers and their engineers into believing
that performance/watt is suddenly important, it's because they
have to carve a path for a while getting the current usage down
in order for the 65nm chips to become palatable - assuming they
can be made at all in a realistic yield - read price bracket)
well - intel is pushing "hyperthreading".
and surprise, surprise, what is amd pushing? dual-core chips.
and what is in the X-Box 360? a PowerPC _triple_ core, _dual_
hyper-threaded processor!!
i believe that the X-Box 360 processor is the way things
are going to be moving - quad-core quad-threaded processors;
16 and 32 core ultra-RISC processors: medium to massive parallel
processors, but this time single-chip unlike the past decade(s) where
multi-core was hip and cool and... expensive.
i believe the future to contain stacks of single-chip multiprocessing
designs in several forms - including intel's fun-and-games VLIW stuff.
remember: intel recently bought the company that has spent
15 years working on that DEC/Alpha just-in-time x86-to-alpha
assembly converter product (remember DEC/Alphas running NT 3.51,
anyone, and still being able to run x86 programs?)
and, what is the linux kernel?
it's a daft, monolithic design that is suitable and faster on
single-processor systems, and that design is going to look _really_
outdated, really soon.
fortunately, there is a research project that has already
done a significant amount of work in breaking away from the
monolithic design: the l4linux project.
last time i checked, a few months ago, they were keeping thoroughly
up-to-date and had either 2.6.11 or 2.6.12 ported, can't recall which.
the l4linux project puts the linux kernel on top of L4-compliant
microkernels (yes, just like the gnu hurd :) and there are several such
L4-compliant microkernels - named after nuts. pistachio, etc.
one of those l4-compliant microkernels is a parallel processor
based one - it's SMP compliant, it even has support for virtual
machining, whoopee, ain't that fun.
i remember now. university of south australia, and university
of karlsruhe. i probably spelled that wrong.
in short, basically, if you follow and agree with the logic, the
linux kernel - as maintained by linus - is far from complete.
i therefore invite you to consider the following strategy:
1) that the linux kernel should merge with the oskit project or that the
linux kernel should split into two projects - a) 30-40k lines of code
comprising the code in kernel/* and headers and ports and headers
b) device drivers i.e duh the oskit project.
2) that the linux kernel should merge and maintain the efforts
of the l4linux project as mainlined not sidelined.
3) that serious efforts be diverted into the l4 microkernels to make it
portable, work on parallel processor systems, hyperthreaded, SMP and
other (such as ACPI which has had to be #ifdef'd out even in XEN).
4) other.
yes, i know this flies in the face of linus' distaste for
message-based kernels, and it's because message-passing slows
things down... but they slow things down _only_ on uniprocessor
kernel designs, and uniprocessors are going to be blowing
goats / bubbles / insert-as-appropriate in the not-too-distant
future. there have _already_ been high-profile parallel
processor designs announced, released, and put into service
(e.g. dual-core AMD64, triple-core dual-hyperthreaded PowerPC in
the X-Box 360).
yes, i may have got things wrong.
yes, it is up to _you_ to point them out.
yes, it is up to _you_ to decide what to do, not me.
good luck.
l.
p.s. XEN is even getting lovely encouraging noises from intel
to support hyperthreading, isn't that nice boys and girls?
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
On Sun, 2 Oct 2005, Luke Kenneth Casson Leighton wrote:
> and, what is the linux kernel?
>
> it's a daft, monolithic design that is suitable and faster on
> single-processor systems, and that design is going to look _really_
> outdated, really soon.
Linux already has a number of scalable SMP synchronisation
mechanisms. The main scalability effort nowadays is about
the avoidance of so-called "cache line bouncing".
http://wiki.kernelnewbies.org/wiki/SMPSynchronisation
--
All Rights Reversed
Luke Kenneth Casson Leighton wrote:
> and, what is the linux kernel?
>
> it's a daft, monolithic design that is suitable and faster on
> single-processor systems, and that design is going to look _really_
> outdated, really soon.
Well, it sounds like it works pretty well on such things as 512 CPU
Altix systems, so it sounds like the suggestion that Linux is designed
solely for single-processor systems and isn't suitable for multicore,
hyperthreaded CPUs doesn't hold much water..
--
Robert Hancock Saskatoon, SK, Canada
To email, remove "nospam" from [email protected]
Home Page: http://www.roberthancock.com/
Let's hope these posts will stop when the UK starts to allow serving
drinks after 23:00. Post from half-drunk people that need to get a life
don't really help a lot.
On Sun, Oct 02, 2005 at 05:05:42PM -0400, Rik van Riel wrote:
> On Sun, 2 Oct 2005, Luke Kenneth Casson Leighton wrote:
>
> > and, what is the linux kernel?
> >
> > it's a daft, monolithic design that is suitable and faster on
> > single-processor systems, and that design is going to look _really_
> > outdated, really soon.
>
> Linux already has a number of scalable SMP synchronisation
> mechanisms.
... and you are tied in to the decisions made by the linux kernel
developers.
whereas, if you allow something like a message-passing design (such as
in the port of the linux kernel to l4), you have the option to try out
different underlying structures - _without_ having to totally redesign
the infrastructure.
several people involved with the l4linux project have already done
that: as i mentioned in the original post, there are about three or
four different and separate l4 microkernels available for download
(GPL) and one of them is ported to stacks of different architectures,
and one of them is SMP capable and even includes a virtual machine
environment.
and they're only approx 30-40,000 lines each, btw.
also, what about architectures that have features over-and-above SMP?
in the original design of SMP it was assumed that if you have
N processors that you have N-way access to memory.
what if, therefore, someone comes up with an architecture that is
better than or improves greatly upon SMP?
they will need to make _significant_ inroads into the linux kernel
code, whereas if, say, you oh i dunno provide hardware-accelerated
parallel support for a nanokernel (such as l4) which just _happens_
to be better than SMP then running anything which is l4 compliant gets
the benefit.
the reason i mention this is because arguments about saying "SMP is it,
SMP is great, SMP is everything, we're improving our SMP design" don't
entirely cut it, because SMP has limitations that don't scale properly
to say 64 or 128 processors: sooner or later someone's going to come up
with something better than SMP and all the efforts focussed on making
SMP better in the linux kernel are going to look lame.
l.
p.s. yes i do know of a company that has improved on SMP.
On Sun, Oct 02, 2005 at 11:49:57PM +0100, Christoph Hellwig wrote:
> Let's hope these posts will stop when the UK starts to allow serving
> drinks after 23:00. Post from half-drunk people that need to get a life
> don't really help a lot.
hi, christoph,
i assume that your global world-wide distribution of this message
was a mistake on your part. but, seeing as it _has_ gone out to
literally thousands of extremely busy people, i can only apologise
to them on your behalf for the mistake of wasting their valuable
time.
let's also hope that people who believe that comments such as the one
that you have made are useful and productive also think about the
consequences of doing so, bear in mind that internet archives are
forever, and also that they check whether the person that they are
criticising drinks at _all_.
personally, my average consumption of alcohol can be measured
as approx 1 bottle per decade. and i'm not talking meths.
if you don't like what i have to say, and don't want to listen,
even with a pinch of salt to me rambling, learn how to set
up a killfile, and use it. and think more before hitting the
reply-to-all button. key. whatever.
l.
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
On Mon, 3 Oct 2005, Luke Kenneth Casson Leighton wrote:
> On Sun, Oct 02, 2005 at 05:05:42PM -0400, Rik van Riel wrote:
> > Linux already has a number of scalable SMP synchronisation
> > mechanisms.
>
> ... and you are tied in to the decisions made by the linux kernel
> developers.
>
> whereas, if you allow something like a message-passing design (such as
> in the port of the linux kernel to l4), you have the option to try out
> different underlying structures - _without_ having to totally redesign
> the infrastructure.
Infrastructure is not what matters when it comes to SMP
scalability on modern systems, since lock contention is
not the primary SMP scalability problem.
Due to the large latency ratio between L1/L2 cache and
RAM, the biggest scalability problem is cache invalidation
and cache bounces.
Those are not solvable by using another underlying
infrastructure - they require a reorganization of the
datastructures on top, the data structures in Linux.
Note that message passing is by definition less efficient
than SMP synchronisation mechanisms that do not require
data to be exchanged between CPUs, eg. RCU or the use of
cpu-local data structures.
> p.s. yes i do know of a company that has improved on SMP.
SGI ? IBM ?
--
All Rights Reversed
On Sunday 02 October 2005 18:43, Robert Hancock wrote:
>Luke Kenneth Casson Leighton wrote:
>> and, what is the linux kernel?
>>
>> it's a daft, monolithic design that is suitable and faster on
>> single-processor systems, and that design is going to look _really_
>> outdated, really soon.
>
>Well, it sounds like it works pretty well on such things as 512 CPU
>Altix systems, so it sounds like the suggestion that Linux is designed
>solely for single-processor systems and isn't suitable for multicore,
>hyperthreaded CPUs doesn't hold much water..
Ahh, yes and no, Robert. The un-answered question, for that
512 processor Altix system, would be "but does it run things 512
times faster?" Methinks not, by a very wide margin. Yes, do a lot
of unrelated things fast maybe, but render a 30 megabyte page with
ghostscript in 10 milliseconds? Never happen IMO.
And Christoph in the next msg, calls him 1/2 drunk. He doesn't come
across to me as being more than 1 beer drunk. And he does make some
interesting points, so if they aren't valid, lets use proveable logic
to shoot them down, not name calling and pointless rhetoric.
--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.35% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
On Mon, 3 Oct 2005, Luke Kenneth Casson Leighton wrote:
> On Sun, Oct 02, 2005 at 05:05:42PM -0400, Rik van Riel wrote:
> > On Sun, 2 Oct 2005, Luke Kenneth Casson Leighton wrote:
> >
> > > and, what is the linux kernel?
> > >
> > > it's a daft, monolithic design that is suitable and faster on
> > > single-processor systems, and that design is going to look _really_
> > > outdated, really soon.
> >
> > Linux already has a number of scalable SMP synchronisation
> > mechanisms.
>
> ... and you are tied in to the decisions made by the linux kernel
> developers.
>
> whereas, if you allow something like a message-passing design (such as
> in the port of the linux kernel to l4), you have the option to try out
> different underlying structures - _without_ having to totally redesign
> the infrastructure.
>
> several people involved with the l4linux project have already done
> that: as i mentioned in the original post, there are about three or
> four different and separate l4 microkernels available for download
> (GPL) and one of them is ported to stacks of different architectures,
> and one of them is SMP capable and even includes a virtual machine
> environment.
>
> and they're only approx 30-40,000 lines each, btw.
>
>
> also, what about architectures that have features over-and-above SMP?
>
> in the original design of SMP it was assumed that if you have
> N processors that you have N-way access to memory.
>
> what if, therefore, someone comes up with an architecture that is
> better than or improves greatly upon SMP?
Like NUMA?
> they will need to make _significant_ inroads into the linux kernel
> code, whereas if, say, you oh i dunno provide hardware-accelerated
> parallel support for a nanokernel (such as l4) which just _happens_
> to be better than SMP then running anything which is l4 compliant gets
> the benefit.
>
>
> the reason i mention this is because arguments about saying "SMP is it,
> SMP is great, SMP is everything, we're improving our SMP design" don't
> entirely cut it, because SMP has limitations that don't scale properly
> to say 64 or 128 processors: sooner or later someone's going to come up
> with something better than SMP and all the efforts focussed on making
> SMP better in the linux kernel are going to look lame.
>
> l.
>
> p.s. yes i do know of a company that has improved on SMP.
>
> -
-Vadim Lobanov
On Sun, 2 Oct 2005, Gene Heskett wrote:
> On Sunday 02 October 2005 18:43, Robert Hancock wrote:
> >Luke Kenneth Casson Leighton wrote:
> >> and, what is the linux kernel?
> >>
> >> it's a daft, monolithic design that is suitable and faster on
> >> single-processor systems, and that design is going to look _really_
> >> outdated, really soon.
> >
> >Well, it sounds like it works pretty well on such things as 512 CPU
> >Altix systems, so it sounds like the suggestion that Linux is designed
> >solely for single-processor systems and isn't suitable for multicore,
> >hyperthreaded CPUs doesn't hold much water..
>
> Ahh, yes and no, Robert. The un-answered question, for that
> 512 processor Altix system, would be "but does it run things 512
> times faster?" Methinks not, by a very wide margin. Yes, do a lot
> of unrelated things fast maybe, but render a 30 megabyte page with
> ghostscript in 10 milliseconds? Never happen IMO.
This is only true for workloads that are parallelizable. I don't think
any kernel is quite good enough to divine what a single-threaded
userland application is doing and make its work parallel.
That is to say, if we are going to look at examples (and so we should),
then we need to pick an example that is actually expected to benefit
from many-processor machines.
> And Christoph in the next msg, calls him 1/2 drunk. He doesn't come
> across to me as being more than 1 beer drunk. And he does make some
> interesting points, so if they aren't valid, lets use proveable logic
> to shoot them down, not name calling and pointless rhetoric.
>
> --
> Cheers, Gene
> "There are four boxes to be used in defense of liberty:
> soap, ballot, jury, and ammo. Please use in that order."
> -Ed Howdershelt (Author)
> 99.35% setiathome rank, not too shabby for a WV hillbilly
> Yahoo.com and AOL/TW attorneys please note, additions to the above
> message by Gene Heskett are:
> Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
>
>
> -
-Vadim Lobanov
On Sun, 2 Oct 2005, Gene Heskett wrote:
> Ahh, yes and no, Robert. The un-answered question, for that
> 512 processor Altix system, would be "but does it run things 512
> times faster?" Methinks not, by a very wide margin. Yes, do a lot
> of unrelated things fast maybe, but render a 30 megabyte page with
> ghostscript in 10 milliseconds? Never happen IMO.
You haven't explained us why you think your proposal
would allow Linux to circumvent Amdahl's law...
--
All Rights Reversed
--Luke Kenneth Casson Leighton <[email protected]> wrote (on Monday, October 03, 2005 00:05:45 +0100):
> On Sun, Oct 02, 2005 at 05:05:42PM -0400, Rik van Riel wrote:
>> On Sun, 2 Oct 2005, Luke Kenneth Casson Leighton wrote:
>>
>> > and, what is the linux kernel?
>> >
>> > it's a daft, monolithic design that is suitable and faster on
>> > single-processor systems, and that design is going to look _really_
>> > outdated, really soon.
>>
>> Linux already has a number of scalable SMP synchronisation
>> mechanisms.
>
> ... and you are tied in to the decisions made by the linux kernel
> developers.
Yes. As are the rest of us. So if you want to implement something
different, that's your perogative. So feel free to go do it
somewhere else, and quit whining on this list.
We are not your implementation bitches. If you think it's such a great
idea, do it yourself.
M.
On Sun, 02 Oct 2005 17:04:51 -0700 Martin J. Bligh wrote:
> --Luke Kenneth Casson Leighton <[email protected]> wrote (on Monday, October 03, 2005 00:05:45 +0100):
>
> > On Sun, Oct 02, 2005 at 05:05:42PM -0400, Rik van Riel wrote:
> >> On Sun, 2 Oct 2005, Luke Kenneth Casson Leighton wrote:
> >>
> >> > and, what is the linux kernel?
> >> >
> >> > it's a daft, monolithic design that is suitable and faster on
> >> > single-processor systems, and that design is going to look _really_
> >> > outdated, really soon.
> >>
> >> Linux already has a number of scalable SMP synchronisation
> >> mechanisms.
> >
> > ... and you are tied in to the decisions made by the linux kernel
> > developers.
>
> Yes. As are the rest of us. So if you want to implement something
> different, that's your perogative. So feel free to go do it
> somewhere else, and quit whining on this list.
>
> We are not your implementation bitches. If you think it's such a great
> idea, do it yourself.
IOW, -ENOPATCH. where's your patch?
---
~Randy
You can't do anything without having to do something else first.
-- Belefant's Law
[...]
> with me so far? :)
Yes, and getting more annoyed at your condescending tone with each
paragraph.
> and, what is the linux kernel?
>
> it's a daft, monolithic design that is suitable and faster on
> single-processor systems, and that design is going to look _really_
> outdated, really soon.
Andrew Tannenbaum said the same thing in the early 1990s. That we're
here still having this discussion >10 years later is telling. Dr.
Tannenbaum might have been acadmeically and theoretically correct,
but, with a nod to OS X, the Linux kernel has proven itself by
implementation and has proven to be remarkably adaptable.
Check back in ten years. Or just come back when you've completed
implementing all the beauteous features you're selling.
> in short, basically, if you follow and agree with the logic, the
> linux kernel - as maintained by linus - is far from complete.
>
> i therefore invite you to consider the following strategy:
And the e.e. cummings affectation is even *more* annoying than your
condescension.
Kurt
--
Blood flows down one leg and up the other.
On Sun, Oct 02, 2005 at 11:49:57PM +0100, Christoph Hellwig took 8 lines to write:
> Let's hope these posts will stop when the UK starts to allow serving
> drinks after 23:00. Post from half-drunk people that need to get a life
> don't really help a lot.
As posts from fully drunk people will help, either, albeit they might be
more entertaining.
Kurt
--
Violence is the last refuge of the incompetent.
-- Salvor Hardin
> > it's a daft, monolithic design that is suitable and faster on
> > single-processor systems, and that design is going to look _really_
> > outdated, really soon.
>
> Andrew Tannenbaum said the same thing in the early 1990s. That we're
> here still having this discussion >10 years later is telling. Dr.
> Tannenbaum might have been acadmeically and theoretically correct,
> but, with a nod to OS X, the Linux kernel has proven itself by
> implementation and has proven to be remarkably adaptable.
Why are you nodding to OS X? It's not a real micokernel either. It
just happens to have all the foobage of a microkernel in a rather
monolithic design. The reason that the bsd personality is in the same
address space as the mach bits is because they didn't want to deal
with the overheads of the message passing from kernel to userspace.
The L4 people figured out how to get a lot of those inefficiencies to
disappear and L4Linux is quite "performant". In some cases, L4Linux
can be used to provide a device driver for other L4 threads that would
normally have to write their own [in user space and even with
respectable performance
http://www.ertos.nicta.com.au/Research/ULDD/Performance.pml]
That's an interesting re-use and combination of several philosophies
if you ask me.
There is a lot of "what's next for linux" going on behind the scenes
and the current path of linux is apparently good enough for
accomplishing it.
- Dave
On Sun, Oct 02, 2005 at 05:14:57PM -0700, Randy.Dunlap wrote:
> IOW, -ENOPATCH. where's your patch?
most of the relevant work has already been done (and not by
me): i invite you to consider searching with google for l4ka,
l4linux and oskit, or simply going to the web site l4linux.org
and l4ka.org.
the code for oskit has been available for some years, now,
and is regularly maintained. the l4linux people have had to
make some significant modifications to it (oskit), and also
to grub, and libstdc++, and pretty much everything else under
the sun and, it's all there, for the approx 100mb downloading.
l.
On Sun, Oct 02, 2005 at 04:37:52PM -0700, Vadim Lobanov wrote:
> > what if, therefore, someone comes up with an architecture that is
> > better than or improves greatly upon SMP?
>
> Like NUMA?
yes, like numa, and there is more.
i had the honour to work with someone who came up with a radical
enhancement even to _that_.
basically the company has implemented, in hardware (a
nanokernel), some operating system primitives, such as message
passing (based on a derivative by thompson of the "alice"
project from plessey, imperial and manchester university
in the mid-80s), hardware cache line lookups (which means
instead of linked list searching, the hardware does it for
you in a single cycle), stuff like that.
the message passing system is designed as a parallel message bus -
completely separate from the SMP and NUMA memory architecture, and as
such it is perfect for use in microkernel OSes.
(these sorts of things are unlikely to make it into the linux kernel, no
matter how much persuasion and how many patches they would write).
_however_, a much _better_ target would be to create an L4 microkernel
on top of their hardware kernel.
this company's hardware is kinda a bit difficult for most people to get
their heads round: it's basically parallelised hardware-acceleration for
operating systems, and very few people see the point in that.
however, as i pointed out, 90nm and approx-2Ghz is pretty much _it_,
and to get any faster you _have_ to go parallel.
and the drive for "faster", "better", "more sales" means more and more
parallelism.
it's _happening_ - and SMP ain't gonna cut it (which is why
these multi-core chips are coming out and why hyperthreading
is coming out).
so.
this is a heads-up.
what you choose to do with this analysis is up to you.
l.
On Sun, Oct 02, 2005 at 05:04:51PM -0700, Martin J. Bligh wrote:
> --Luke Kenneth Casson Leighton <[email protected]> wrote (on Monday, October 03, 2005 00:05:45 +0100):
>
> > On Sun, Oct 02, 2005 at 05:05:42PM -0400, Rik van Riel wrote:
> >> On Sun, 2 Oct 2005, Luke Kenneth Casson Leighton wrote:
> >>
> >> > and, what is the linux kernel?
> >> >
> >> > it's a daft, monolithic design that is suitable and faster on
> >> > single-processor systems, and that design is going to look _really_
> >> > outdated, really soon.
> >>
> >> Linux already has a number of scalable SMP synchronisation
> >> mechanisms.
> >
> > ... and you are tied in to the decisions made by the linux kernel
> > developers.
>
> Yes. As are the rest of us. So if you want to implement something
> different, that's your perogative. So feel free to go do it
> somewhere else, and quit whining on this list.
>
> We are not your implementation bitches. If you think it's such a great
> idea, do it yourself.
martin, i'm going to take a leaf from the great rusty russell's book,
because i was very impressed with the professional way in which he
dealt with someone who posted such immature and out-of-line comments:
he rewrote them in a much more non-hostile manner and then replied to
that.
so, here goes: i'm copying the above few [relevant] paragraphs
below, then rewriting them, here:
> >
> > ... and you are tied in to the decisions made by the linux kernel
> > developers.
>
> Yes, this is very true: we are all somewhat at the mercy of their
> decisions. However, fortunately, they had the foresight to work
> with free software, so any of us can try something different, if
> we wish.
>
> i am slightly confused by your message, however: forgive me for
> asking this but you are not expecting us to implement such a radical
> redesign, are you?
martin, hi, thank you for responding.
well... actually, as it turns out, the l4linux and l4ka people have
already done most of the work!!
i believe you may have missed part of my message (it was a bit long, i
admit) and i thank you for the opportunity, that your message presents,
to reiterate this: l4linux _exists_ - last time i checked (some months
ago) it had a port of 2.6.11 to the L4 microkernel.
so, in more ways than one, no i am of course not expecting people to
just take orders from someone as mad as myself :)
i really should reiterate this: i _invite_ people to _consider_ the
direction that processor designs - not just any "off-the-wall"
processor designs but _mainstream_ x86-compatible processor designs -
are likely to take. and they are becoming more and more parallel.
the kinds of questions that the experienced linux kernel
maintainers and developers really need to ask is: can the
present linux kernel design _cope_ with such parallelism?
is there an easier way?
that's mainly why i wished you "good luck" :)
l.
p.s. martin. _don't_ do that again. i don't care who you are:
internet archives are forever and your rudeness will be noted
by google-users and other search-users - long after you are dead.
On Mon, 3 Oct 2005, Luke Kenneth Casson Leighton wrote:
> well... actually, as it turns out, the l4linux and l4ka people have
> already done most of the work!!
And I am sure they have reasons for not submitting their
changes to the linux-kernel mailing list. They probably
know something we (including you) don't know.
Switching out the low level infrastructure does NOT help
with scalability. The only way to make the kernel more
parallelizable is by changing the high level code, ie.
Linux itself.
Adding a microkernel under Linux is not going to help
with anything you mentioned so far.
--
All Rights Reversed
On Mon, 3 Oct 2005, Luke Kenneth Casson Leighton wrote:
> On Sun, Oct 02, 2005 at 04:37:52PM -0700, Vadim Lobanov wrote:
>
> > > what if, therefore, someone comes up with an architecture that is
> > > better than or improves greatly upon SMP?
> >
> > Like NUMA?
>
> yes, like numa, and there is more.
The beauty of capitalization is that it makes it easier for others to
read what you have to say.
> i had the honour to work with someone who came up with a radical
> enhancement even to _that_.
>
> basically the company has implemented, in hardware (a
> nanokernel), some operating system primitives, such as message
> passing (based on a derivative by thompson of the "alice"
> project from plessey, imperial and manchester university
> in the mid-80s), hardware cache line lookups (which means
> instead of linked list searching, the hardware does it for
> you in a single cycle), stuff like that.
That sounds awesome, but I have something better -- a quantum computer.
And it's about as parallel as you're going to get anytime in the
foreseeable future!
...
Moral of the story: There are thousands of hardware doodads all around.
People only start to become interested when they have actual "metal"
freely available on the market, that they can play with and code to.
> the message passing system is designed as a parallel message bus -
> completely separate from the SMP and NUMA memory architecture, and as
> such it is perfect for use in microkernel OSes.
You're making an implicit assumption here that it will benefit _only_
microkernel designs. That is not at all immediate or obvious to me (or,
I suspect, others also) -- where's the proof?
> (these sorts of things are unlikely to make it into the linux kernel, no
> matter how much persuasion and how many patches they would write).
No, the kernel hackers are actually very sensible people. When they push
back, there's usually a darn good reason for it. See above point
regarding availability of hardware.
> _however_, a much _better_ target would be to create an L4 microkernel
> on top of their hardware kernel.
Perfect. You can do that, and benefit from the oodles of fame that
follow. Others might be less-than-convinced.
> this company's hardware is kinda a bit difficult for most people to get
> their heads round: it's basically parallelised hardware-acceleration for
> operating systems, and very few people see the point in that.
That just sounds condescending.
> however, as i pointed out, 90nm and approx-2Ghz is pretty much _it_,
> and to get any faster you _have_ to go parallel.
Sure, it's going to stop somewhere, but you have to be a heck of a
visionary to predict that it will stop _there_. People have been
surprised before on such matters, so don't go around yelling about the
impending doom quite yet.
> and the drive for "faster", "better", "more sales" means more and more
> parallelism.
>
> it's _happening_ - and SMP ain't gonna cut it (which is why
> these multi-core chips are coming out and why hyperthreading
> is coming out).
"Rah, rah, parallelism is great!" -- That's a great slogan, except...
Users, who also happen to be the target of those sales, care about
_userland_ applications. And the bitter truth is that the _vast_
majority of userland apps are single-threaded. Why? Two reasons --
first, it's harder to write a multithreaded application, and second,
some workloads simply can't be expressed "in parallel". Your kernel
might (might, not will) run like a speed-demon, but the userland stuff
will still be lackluster in comparison.
And that's when your slogan hits a wall, and the marketing hype dies.
The reality is that parallelism is something to be desired, but is not
always achievable.
> so.
>
> this is a heads-up.
>
> what you choose to do with this analysis is up to you.
I choose to wait for actual, concrete details and proofs of your design,
instead of the ambiguous "visionary" hand-waving so far. As has already
been said, -ENOPATCH.
> l.
>
-Vadim Lobanov
On Sun, Oct 02, 2005 at 07:26:21PM -0400, Rik van Riel wrote:
> On Mon, 3 Oct 2005, Luke Kenneth Casson Leighton wrote:
> > On Sun, Oct 02, 2005 at 05:05:42PM -0400, Rik van Riel wrote:
>
> > > Linux already has a number of scalable SMP synchronisation
> > > mechanisms.
> >
> > ... and you are tied in to the decisions made by the linux kernel
> > developers.
> >
> > whereas, if you allow something like a message-passing design (such as
> > in the port of the linux kernel to l4), you have the option to try out
> > different underlying structures - _without_ having to totally redesign
> > the infrastructure.
>
> Infrastructure is not what matters when it comes to SMP
> scalability on modern systems, since lock contention is
> not the primary SMP scalability problem.
>
> Due to the large latency ratio between L1/L2 cache and
> RAM, the biggest scalability problem is cache invalidation
> and cache bounces.
>
> Those are not solvable by using another underlying
> infrastructure - they require a reorganization of the
> datastructures on top, the data structures in Linux.
... ah, but what about in hardware? what if you had hardware support
for
_plus_ what if you had some other OS primitives implemented
in hardware, the use of which allowed you to avoid or minimise
cache invalidation problems?
not entirely, of course, but enough to make up for SMP's deficiencies.
> Note that message passing is by definition less efficient
> than SMP synchronisation mechanisms that do not require
> data to be exchanged between CPUs, eg. RCU or the use of
> cpu-local data structures.
how about message passing by reference - a la c++?
i.e. using an "out-of-band" parallel message bus, you pass
the address in a NUMA or SMP area of memory that is granted
to a specific processor, which says to another processor oh
something like "you now have access to this memory: by the time
you get this message i will have already cleared the cache so
you can get it immediately".
that sort of thing.
_and_ you use the parallel message bus to communicate memory
allocation, locking, etc.
_and_ you use the parallel message bus to implement semaphores and
mutexes.
_and_ if the message is small enough, you just pass the message across
without going via external memory.
... but i digress - but enough to demonstrate, i hope, that
this isn't some "pie-in-the-sky" thing, it's one hint at a
solution to the problem that a lot of hardware designers haven't
been able to solve, and up until now they haven't had to even
_consider_ it.
and they've avoided the problem by going "multi-core" and going
"hyperthreading".
but, at some point, hyperthreading isn't going to cut it, and at some
point multi-core isn't going to cut it.
and people are _still_ going to expect to see the monster
parallelism (32, 64, 128 parallel hardware threads) as
"one processor".
the question is - and i iterate it again: can the present
linux kernel design _cope_ with such monster parallelism?
answer, at present, as maintained as-it-is, not a chance.
question _that_ raises: do you _want_ to [make it cope with such
monster parallelism]?
and if the answer to that is "no, definitely not", then the
responsibility can be offloaded onto a microkernel, e.g. the L4
microkernel, and it _just_ so happens that the linux kernel has already
been ported to L4.
i raise this _one_ route - there are surely going to be others.
i invite you to consider discussing them.
LIKE FRIGGIN ADULTS, unlike the very spiteful comments i've
received indicate that some people would like to do (no i
don't count you in that number, rik, just in case you thought
i was because i'm replying direct to you!).
> > p.s. yes i do know of a company that has improved on SMP.
>
> SGI ? IBM ?
no, they're a startup.
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
I'd venture to say that Linux scalability is fantastic. This also sounds like
a repeat of a debate that happened ten years ago.
I too was intrigued by Andrew's comment about 'finishing the kernel', though
I'm guessing (albeit without ever having spoken to Andrew personally) that it
was partially in jest. What it does suggest, though, is a point that KDE
desktop developer Aaron Seigo has made recently about the focus moving up the
stack.
If we are admirably tackling the problems of hardware compatibility,
stability, scalability and we've implemented most of the important features
that belong in the kernel, then a lot of the development fire for a so-called
complete Linux system is going to have to move up the stack - into the
userland.
Indeed, adding 100 cores to my Pentium 4 isn't going to do me a damned bit of
good when Akregator goes to query some 40 RSS feeds and Kontact blocks,
refusing to process GUI events. It's also not going to make compiling a
single .c file any faster.
I have no doubt that the bright minds here on LKML will continue to find
places to improve Linux's scalability, but that certainly doesn't require
rebuilding the kernel - we're already doing remarkably well in the
scalability department.
The bottom line is that the application developers need to start being clever
with threads. I think I remember some interesting rumors about Perl 6, for
example, including 'autothreading' support - the idea that your optimizer
could be smart enough to identify certain work that can go parallel.
As dual cores and HT become more popular, the onus is going to be on the
applications, not the OS, to speed up.
Regards,
Chase Venters
On Sunday 02 October 2005 08:10 pm, Luke Kenneth Casson Leighton wrote:
> ... words ...
On Sun, Oct 02, 2005 at 06:20:38PM -0700, Vadim Lobanov wrote:
> I choose to wait for actual, concrete details and proofs of your design,
> instead of the ambiguous "visionary" hand-waving so far. As has already
> been said, -ENOPATCH.
Speaking of which, IIRC, somebody used to maintain a list of words, acronyms,
etc. useful to know if you want to read l-k. May I submit an addition to
that list?
visionary [n]: onanist with strong exhibitionist tendencies; from
"visions", the source of inspiration they refer to when it becomes
obvious that they have lost both sight and capacity for rational
thought.
On Mon, 3 Oct 2005, Al Viro wrote:
> On Sun, Oct 02, 2005 at 06:20:38PM -0700, Vadim Lobanov wrote:
> > I choose to wait for actual, concrete details and proofs of your design,
> > instead of the ambiguous "visionary" hand-waving so far. As has already
> > been said, -ENOPATCH.
>
> Speaking of which, IIRC, somebody used to maintain a list of words, acronyms,
> etc. useful to know if you want to read l-k. May I submit an addition to
> that list?
>
> visionary [n]: onanist with strong exhibitionist tendencies; from
> "visions", the source of inspiration they refer to when it becomes
> obvious that they have lost both sight and capacity for rational
> thought.
>
Nice. :-)
Just from idle curiosity, you wouldn't know where that list currently
resides, would you?
-Vadim Lobanov
On Sun, Oct 02, 2005 at 06:20:38PM -0700, Vadim Lobanov wrote:
> On Mon, 3 Oct 2005, Luke Kenneth Casson Leighton wrote:
>
> > On Sun, Oct 02, 2005 at 04:37:52PM -0700, Vadim Lobanov wrote:
> >
> > > > what if, therefore, someone comes up with an architecture that is
> > > > better than or improves greatly upon SMP?
> > >
> > > Like NUMA?
> >
> > yes, like numa, and there is more.
>
> The beauty of capitalization is that it makes it easier for others to
> read what you have to say.
sorry, vadim: haven't touched a shift key in over 20 years.
> > basically the company has implemented, in hardware (a
> > nanokernel), some operating system primitives, such as message
> > passing (based on a derivative by thompson of the "alice"
> > project from plessey, imperial and manchester university
> > in the mid-80s), hardware cache line lookups (which means
> > instead of linked list searching, the hardware does it for
> > you in a single cycle), stuff like that.
>
> That sounds awesome, but I have something better -- a quantum computer.
> And it's about as parallel as you're going to get anytime in the
> foreseeable future!
:)
*sigh* - i _so_ hope we don't need degrees in physics to program
them...
> > the message passing system is designed as a parallel message bus -
> > completely separate from the SMP and NUMA memory architecture, and as
> > such it is perfect for use in microkernel OSes.
>
> You're making an implicit assumption here that it will benefit _only_
> microkernel designs.
ah, i'm not: i just left out mentioning it :)
the message passing needs to be communicated down to manage
threads, and also to provide a means to manage semaphores and
mutexes: ultimately, support for such an architecture would
work its way down to libc.
and yes, if you _really_ didn't want a kernel in the way at all, you
could go embedded and just... do everything yourself.
or port reactos, the free software reimplementation of nt,
to it, or something :)
*shrug*.
> > this company's hardware is kinda a bit difficult for most people to get
> > their heads round: it's basically parallelised hardware-acceleration for
> > operating systems, and very few people see the point in that.
>
> That just sounds condescending.
i'm very sorry about that, it wasn't deliberate and ... re-reading
my comment, i should say that my comment isn't actually entirely true!
a correction/qualification: the people whom the startup company
contacted before they were put in touch with me had found that
everybody they had previously talked to just simply _did_ not
get it: this was presumably because of their choice of people
whom they were seeking funding from were not technically up
to the job of understanding the concept.
i didn't mean to imply that _everyone_ - or more specifically the
people reading this list - would not get it.
sorry.
> > however, as i pointed out, 90nm and approx-2Ghz is pretty much _it_,
> > and to get any faster you _have_ to go parallel.
>
> Sure, it's going to stop somewhere, but you have to be a heck of a
> visionary to predict that it will stop _there_.
okay, i admit it: you caught me out - i'm a mad visionary.
but seriously.
it won't stop - but the price of 90nm mask charges, at approx
$2m, is already far too high, and the number of large chips
being designed is plummetting like a stone as a result - from
like 15,000 per year a few years ago down to ... damn, can't remember -
less than a hundred (i think! don't quote me on that!)
when 90 nm was introduced, some mad fabs wanted to make 9
metre lenses, dude!!! until carl zeiss were called in and
managed to get it down to 3 metres.
and that lens is produced on a PER CHIP basis.
basically, it's about cost.
the costs of producing faster and faster uniprocessors is
getting out of control.
i'm not explaining things very well, but i'm trying. too many words,
not concise enough, too much to explain without people misunderstanding
or skipping things and getting the wrong end of the stick.
argh.
> > and the drive for "faster", "better", "more sales" means more and more
> > parallelism.
> >
> > it's _happening_ - and SMP ain't gonna cut it (which is why
> > these multi-core chips are coming out and why hyperthreading
> > is coming out).
>
> "Rah, rah, parallelism is great!" -- That's a great slogan, except...
>
> Users, who also happen to be the target of those sales, care about
> _userland_ applications. And the bitter truth is that the _vast_
> majority of userland apps are single-threaded. Why? Two reasons --
> first, it's harder to write a multithreaded application, and second,
> some workloads simply can't be expressed "in parallel". Your kernel
> might (might, not will) run like a speed-demon, but the userland stuff
> will still be lackluster in comparison.
>
> And that's when your slogan hits a wall, and the marketing hype dies.
> The reality is that parallelism is something to be desired, but is not
> always achievable.
okay: i will catch up on this bit, another time, because it is late
enough for me to be getting dizzy and appearing to be drunk.
this is one answer (and there are others i will write another time.
hint: automated code analysis tools, auto-parallelising tools, both
offline and realtime):
watch what intel and amd do: they will support _anything_ - clutch at
straws - to make parallelism palable, why? because in order to be
competitive - and realistically priced - they don't have any choice.
plus, i am expecting the chips to be thrown out there (like
the X-Box 360 which has SIX hardware threads remember) and
the software people to quite literally _have_ to deal with it.
i expect the hardware people to go: this is the limit, this is what we
can do, realistically price-performance-wise: lump it, deal with it.
when intel and amd start doing that, everyone _will_ lump it.
and deal with it.
... why do you think intel is hyping support for and backing
hyperthreads support in XEN/Linux so much?
l.
On Sun, Oct 02, 2005 at 06:50:46PM -0700, Vadim Lobanov wrote:
> > visionary [n]: onanist with strong exhibitionist tendencies; from
> > "visions", the source of inspiration they refer to when it becomes
> > obvious that they have lost both sight and capacity for rational
> > thought.
> >
>
> Nice. :-)
>
> Just from idle curiosity, you wouldn't know where that list currently
> resides, would you?
No idea... Probably somebody from kernelnewbies.org crowd would know
the current location...
On Mon, 3 Oct 2005, Luke Kenneth Casson Leighton wrote:
> how about message passing by reference - a la c++?
> _and_ you use the parallel message bus to communicate memory
> allocation, locking, etc.
Then you lose. It's the act of passing itself that causes
scalability problems and a loss of performance.
The best way to get SMP scalability is to avoid message
passing alltogether, using things like per-cpu data
structures and RCU.
Not having to pass a message is faster than any message
passing mechanism.
> ... but i digress - but enough to demonstrate, i hope, that
> this isn't some "pie-in-the-sky" thing,
You've made a lot of wild claims so far, most of which I'm
not ready to belief without some proof to back them up.
> it's one hint at a solution to the problem that a lot of hardware
> designers haven't been able to solve, and up until now they haven't
> had to even _consider_ it.
The main problem is that communication with bits of silicon
four inches away is a lot slower, or takes much more power,
than communication with bits of silicon half a millimeter away.
This makes cross-core communication, and even cross-thread
communication in SMT/HT, slower than not having to have such
communication at all.
> the question is - and i iterate it again: can the present
> linux kernel design _cope_ with such monster parallelism?
The SGI and IBM people seem fairly happy with current 128 CPU
performance, and appear to be making serious progress towards
512 CPUs and more.
> question _that_ raises: do you _want_ to [make it cope with such
> monster parallelism]?
>
> and if the answer to that is "no, definitely not", then the
> responsibility can be offloaded onto a microkernel,
No, that cannot be done, for all the reasons I mentioned
earlier in the thread.
Think about something like the directory entry cache (dcache),
all the CPUs need to see that cache consistently, and you cannot
avoid locking overhead by having the locking done by a microkernel.
The only way to avoid locking overhead is by changing the data
structure to something that doesn't need locking.
No matter how low your locking overhead - once you have 1024
CPUs it's probably too high.
--
All Rights Reversed
On Mon, Oct 03, 2005 at 02:53:00AM +0100, Al Viro wrote:
> On Sun, Oct 02, 2005 at 06:50:46PM -0700, Vadim Lobanov wrote:
> > > visionary [n]: onanist with strong exhibitionist tendencies; from
> > > "visions", the source of inspiration they refer to when it becomes
> > > obvious that they have lost both sight and capacity for rational
> > > thought.
oo, nice pretty flowers, wheeee :)
On Mon, 3 Oct 2005, Luke Kenneth Casson Leighton wrote:
> On Sun, Oct 02, 2005 at 06:20:38PM -0700, Vadim Lobanov wrote:
>
> > On Mon, 3 Oct 2005, Luke Kenneth Casson Leighton wrote:
> >
> > > On Sun, Oct 02, 2005 at 04:37:52PM -0700, Vadim Lobanov wrote:
> > >
> > > > > what if, therefore, someone comes up with an architecture that is
> > > > > better than or improves greatly upon SMP?
> > > >
> > > > Like NUMA?
> > >
> > > yes, like numa, and there is more.
> >
> > The beauty of capitalization is that it makes it easier for others to
> > read what you have to say.
>
> sorry, vadim: haven't touched a shift key in over 20 years.
It's not going to bite you. I promise.
> > > basically the company has implemented, in hardware (a
> > > nanokernel), some operating system primitives, such as message
> > > passing (based on a derivative by thompson of the "alice"
> > > project from plessey, imperial and manchester university
> > > in the mid-80s), hardware cache line lookups (which means
> > > instead of linked list searching, the hardware does it for
> > > you in a single cycle), stuff like that.
> >
> > That sounds awesome, but I have something better -- a quantum computer.
> > And it's about as parallel as you're going to get anytime in the
> > foreseeable future!
>
> :)
>
> *sigh* - i _so_ hope we don't need degrees in physics to program
> them...
>
> > > the message passing system is designed as a parallel message bus -
> > > completely separate from the SMP and NUMA memory architecture, and as
> > > such it is perfect for use in microkernel OSes.
> >
> > You're making an implicit assumption here that it will benefit _only_
> > microkernel designs.
>
> ah, i'm not: i just left out mentioning it :)
>
> the message passing needs to be communicated down to manage
> threads, and also to provide a means to manage semaphores and
> mutexes: ultimately, support for such an architecture would
> work its way down to libc.
>
>
> and yes, if you _really_ didn't want a kernel in the way at all, you
> could go embedded and just... do everything yourself.
>
> or port reactos, the free software reimplementation of nt,
> to it, or something :)
>
> *shrug*.
No, for reliability and performance reasons, I very much want a kernel
in the way. After all, kernel code is orders of magnitude better tuned
than almost all userland code.
The point I was making here is that, from what I can see, the current
Linux architecture is quite alright in anticipation of the hardware that
you're describing. It _could_ be better tuned for such hardware, sure,
but so far there is no need for such work at this particular moment.
> > > this company's hardware is kinda a bit difficult for most people to get
> > > their heads round: it's basically parallelised hardware-acceleration for
> > > operating systems, and very few people see the point in that.
> >
> > That just sounds condescending.
>
> i'm very sorry about that, it wasn't deliberate and ... re-reading
> my comment, i should say that my comment isn't actually entirely true!
>
> a correction/qualification: the people whom the startup company
> contacted before they were put in touch with me had found that
> everybody they had previously talked to just simply _did_ not
> get it: this was presumably because of their choice of people
> whom they were seeking funding from were not technically up
> to the job of understanding the concept.
>
> i didn't mean to imply that _everyone_ - or more specifically the
> people reading this list - would not get it.
>
> sorry.
>
> > > however, as i pointed out, 90nm and approx-2Ghz is pretty much _it_,
> > > and to get any faster you _have_ to go parallel.
> >
> > Sure, it's going to stop somewhere, but you have to be a heck of a
> > visionary to predict that it will stop _there_.
>
> okay, i admit it: you caught me out - i'm a mad visionary.
>
> but seriously.
>
> it won't stop - but the price of 90nm mask charges, at approx
> $2m, is already far too high, and the number of large chips
> being designed is plummetting like a stone as a result - from
> like 15,000 per year a few years ago down to ... damn, can't remember -
> less than a hundred (i think! don't quote me on that!)
>
> when 90 nm was introduced, some mad fabs wanted to make 9
> metre lenses, dude!!! until carl zeiss were called in and
> managed to get it down to 3 metres.
>
> and that lens is produced on a PER CHIP basis.
>
> basically, it's about cost.
I can guarantee one thing here -- the cost, as is, is absolutely
bearable. These companies make more money doing this than they spend in
doing it, otherwise they wouldn't be in business. From an economics
perspective, this industry is very much alive and well, proven by the
fact that these companies haven't bailed out of it yet.
> the costs of producing faster and faster uniprocessors is
> getting out of control.
>
> i'm not explaining things very well, but i'm trying. too many words,
> not concise enough, too much to explain without people misunderstanding
> or skipping things and getting the wrong end of the stick.
>
> argh.
>
>
> > > and the drive for "faster", "better", "more sales" means more and more
> > > parallelism.
> > >
> > > it's _happening_ - and SMP ain't gonna cut it (which is why
> > > these multi-core chips are coming out and why hyperthreading
> > > is coming out).
> >
> > "Rah, rah, parallelism is great!" -- That's a great slogan, except...
> >
> > Users, who also happen to be the target of those sales, care about
> > _userland_ applications. And the bitter truth is that the _vast_
> > majority of userland apps are single-threaded. Why? Two reasons --
> > first, it's harder to write a multithreaded application, and second,
> > some workloads simply can't be expressed "in parallel". Your kernel
> > might (might, not will) run like a speed-demon, but the userland stuff
> > will still be lackluster in comparison.
> >
> > And that's when your slogan hits a wall, and the marketing hype dies.
> > The reality is that parallelism is something to be desired, but is not
> > always achievable.
>
> okay: i will catch up on this bit, another time, because it is late
> enough for me to be getting dizzy and appearing to be drunk.
>
> this is one answer (and there are others i will write another time.
> hint: automated code analysis tools, auto-parallelising tools, both
> offline and realtime):
We don't need hints. We need actual performance statistics --
verifiable numbers that we can point to and say "Oh crap, we're losing."
or "Hah, we kick butt.", as the case may be.
> watch what intel and amd do: they will support _anything_ - clutch at
> straws - to make parallelism palable, why? because in order to be
> competitive - and realistically priced - they don't have any choice.
As stated earlier, I doubt they're in such dire straits as you predict.
Ultimately, the only reason why they need to advance their designs is to
be able to market it better. This means that truly innovative designs
may not be pursued because the up-front cost is too high.
There's a saying: "Let your competitor do your R&D for you."
> plus, i am expecting the chips to be thrown out there (like
> the X-Box 360 which has SIX hardware threads remember) and
> the software people to quite literally _have_ to deal with it.
>
> i expect the hardware people to go: this is the limit, this is what we
> can do, realistically price-performance-wise: lump it, deal with it.
>
> when intel and amd start doing that, everyone _will_ lump it.
> and deal with it.
Hardware without software is just as useless as software without
hardware. Any argument from any side that goes along the lines of "deal
with it" can be countered in kind.
What this boils down to is that hardware people try to make their
products appealing to program to, from _both_ a speed and a usability
perspective. That's how they get mindshare.
> ... why do you think intel is hyping support for and backing
> hyperthreads support in XEN/Linux so much?
At the risk of stepping on some toes, I believe that hyperthreading is
going out of style, in favor of multi-core processors.
> l.
>
In conclusion, you made claims that Linux is lagging behind. However,
such claims are rather useless without data and/or technical discussions
to back them up.
-Vadim Lobanov
On Mon, 03 Oct 2005 01:54:00 BST, Luke Kenneth Casson Leighton said:
> in the mid-80s), hardware cache line lookups (which means
> instead of linked list searching, the hardware does it for
> you in a single cycle), stuff like that.
OK.. I'll bite. How do you find the 5th or 6th entry in the linked list,
when only the first entry is in cache, in a single cycle, when a cache line
miss is more than a single cycle penalty, and you have several "These are not
the droids you're looking for" checks and go on to the next entry - and do it
in one clock cycle?
Now, it's really easy to imagine an execution unit that will execute this
as a single opcode, and stall until complete. Of course, this only really helps
if you have multiple execution units - which is what hyperthreading and
multi-core and all that is about. And guess what - it's not news...
The HP2114 and DEC KL10/20 were able to dereference a chain of indirect bits
back in the 70's (complete with warnings that hardware wedges could occur if
an indirect reference formed a loop or pointed at itself). Whoops. :)
And all the way back in 1964, IBM disk controllers were able to do some rather
sophisticated offloading of "channel control words" (amazing what you could do
with 'Search ID Equal', 'Transfer In-Channel' (really a misnamed branch
instruction), and self-modifying CCWs). But even then, they understood that
it was only a win if you could go do other stuff when you waited....
On Monday 03 October 2005 02:31, Vadim Lobanov wrote:
> On Mon, 3 Oct 2005, Luke Kenneth Casson Leighton wrote:
> > On Sun, Oct 02, 2005 at 06:20:38PM -0700, Vadim Lobanov wrote:
> > > On Mon, 3 Oct 2005, Luke Kenneth Casson Leighton wrote:
> > > > On Sun, Oct 02, 2005 at 04:37:52PM -0700, Vadim Lobanov wrote:
> > > > > > what if, therefore, someone comes up with an
> > > > > > architecture that is better than or improves greatly upon
> > > > > > SMP?
> > > > >
> > > > > Like NUMA?
> > > >
> > > > yes, like numa, and there is more.
> > >
> > > The beauty of capitalization is that it makes it easier for
> > > others to read what you have to say.
> >
> > sorry, vadim: haven't touched a shift key in over 20 years.
>
> It's not going to bite you. I promise.
You never know - someone might've rigged his keyboard to shock him
every time the shift key was pressed :)
<snip>
> > > > the message passing system is designed as a parallel message
> > > > bus - completely separate from the SMP and NUMA memory
> > > > architecture, and as such it is perfect for use in
> > > > microkernel OSes.
> > >
> > > You're making an implicit assumption here that it will benefit
> > > _only_ microkernel designs.
> >
> > ah, i'm not: i just left out mentioning it :)
> >
> > the message passing needs to be communicated down to manage
> > threads, and also to provide a means to manage semaphores and
> > mutexes: ultimately, support for such an architecture would
> > work its way down to libc.
> >
> >
> > and yes, if you _really_ didn't want a kernel in the way at all,
> > you could go embedded and just... do everything yourself.
> >
> > or port reactos, the free software reimplementation of nt,
> > to it, or something :)
> >
> > *shrug*.
>
> No, for reliability and performance reasons, I very much want a
> kernel in the way. After all, kernel code is orders of magnitude
> better tuned than almost all userland code.
>
> The point I was making here is that, from what I can see, the
> current Linux architecture is quite alright in anticipation of the
> hardware that you're describing. It _could_ be better tuned for
> such hardware, sure, but so far there is no need for such work at
> this particular moment.
Wholly agreed. The arguments over the benefits of running a
microkernel aren't ever really clear. Beyond that, I personally feel
that the whole micro vs. mono argument is a catfight between
academics. I'd rather have a system that works and is proven than a
system that is bleeding edge and never truly stable. To me this
means a monolithic kernel - microkernels are picky at best, and can
be highly insecure (and that means "unstable" in my book too).
<snip>
> > > > however, as i pointed out, 90nm and approx-2Ghz is pretty
> > > > much _it_, and to get any faster you _have_ to go parallel.
> > >
> > > Sure, it's going to stop somewhere, but you have to be a heck
> > > of a visionary to predict that it will stop _there_.
> >
> > okay, i admit it: you caught me out - i'm a mad visionary.
> >
> > but seriously.
> >
> > it won't stop - but the price of 90nm mask charges, at approx
> > $2m, is already far too high, and the number of large chips
> > being designed is plummetting like a stone as a result - from
> > like 15,000 per year a few years ago down to ... damn, can't
> > remember - less than a hundred (i think! don't quote me on
> > that!)
> >
> > when 90 nm was introduced, some mad fabs wanted to make 9
> > metre lenses, dude!!! until carl zeiss were called in and
> > managed to get it down to 3 metres.
> >
> > and that lens is produced on a PER CHIP basis.
> >
> > basically, it's about cost.
>
> I can guarantee one thing here -- the cost, as is, is absolutely
> bearable. These companies make more money doing this than they
> spend in doing it, otherwise they wouldn't be in business. From an
> economics perspective, this industry is very much alive and well,
> proven by the fact that these companies haven't bailed out of it
> yet.
I have to agree. And he is also completely ignoring the fact that both
Intel and AMD are either in the process of moving to (or have moved
to) a 65nm fab process - last news I saw about this said both
facilities were running into the multi-billion dollar cost range.
Companies worried about $2m for a mask charge wouldn't be investing
multiple billions of dollars in new plants and a new, smaller fab
process.
<snip>
> > > > and the drive for "faster", "better", "more sales" means
> > > > more and more parallelism.
> > > >
> > > > it's _happening_ - and SMP ain't gonna cut it (which is why
> > > > these multi-core chips are coming out and why hyperthreading
> > > > is coming out).
> > >
> > > "Rah, rah, parallelism is great!" -- That's a great slogan,
> > > except...
> > >
> > > Users, who also happen to be the target of those sales, care
> > > about _userland_ applications. And the bitter truth is that the
> > > _vast_ majority of userland apps are single-threaded. Why? Two
> > > reasons -- first, it's harder to write a multithreaded
> > > application, and second, some workloads simply can't be
> > > expressed "in parallel". Your kernel might (might, not will)
> > > run like a speed-demon, but the userland stuff will still be
> > > lackluster in comparison.
> > >
> > > And that's when your slogan hits a wall, and the marketing hype
> > > dies. The reality is that parallelism is something to be
> > > desired, but is not always achievable.
> >
> > okay: i will catch up on this bit, another time, because it is
> > late enough for me to be getting dizzy and appearing to be drunk.
> >
> > this is one answer (and there are others i will write another
> > time. hint: automated code analysis tools, auto-parallelising
> > tools, both offline and realtime):
>
> We don't need hints. We need actual performance statistics --
> verifiable numbers that we can point to and say "Oh crap, we're
> losing." or "Hah, we kick butt.", as the case may be.
Hear, hear! I'm still working my way through the source tree and
learning the general layout and functionality of the various bits,
but in just a pair of months of being on this list I can attest to
the fact that one thing all developers seem to ask for is statistics.
<snip>
> At the risk of stepping on some toes, I believe that hyperthreading
> is going out of style, in favor of multi-core processors.
Agreed. And multi-core processors aren't really new technology - there
have been multi-core designs out for a while, but those were usually
low production "research" chips.
DRH
On Sun, 2 Oct 2005, [email protected] wrote:
> OK.. I'll bite. How do you find the 5th or 6th entry in the linked
> list, when only the first entry is in cache, in a single cycle, when a
> cache line miss is more than a single cycle penalty, and you have
> several "These are not the droids you're looking for" checks and go on
> to the next entry - and do it in one clock cycle?
A nice saying from the last decade comes to mind:
"If you can do all that in one cycle, your cycles are too long."
--
All Rights Reversed
On Sunday 02 October 2005 19:48, Rik van Riel wrote:
>On Sun, 2 Oct 2005, Gene Heskett wrote:
>> Ahh, yes and no, Robert. The un-answered question, for that
>> 512 processor Altix system, would be "but does it run things 512
>> times faster?" Methinks not, by a very wide margin. Yes, do a lot
>> of unrelated things fast maybe, but render a 30 megabyte page with
>> ghostscript in 10 milliseconds? Never happen IMO.
>
>You haven't explained us why you think your proposal
>would allow Linux to circumvent Amdahl's law...
Amdahl's Law?
Thats a reference I don't believe I've been made aware of. Can you
elaborate?
Besides, it isn't my proposal, just a question in that I chose a
scenario (ghostscripts rendering of a page of text) that in fact only
runs maybe 10x faster on an XP-2800 Athlon with a gig of dram than it
did on my old 25 mhz 68040 equipt amiga with 64 megs of dram.
With 64 megs of dram, so it wasn't nearly as memory bound doing that
as most of the Amiga's were.
--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.35% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
On Mon, Oct 03, 2005 at 12:24:16AM +0100, Luke Kenneth Casson Leighton wrote:
> On Sun, Oct 02, 2005 at 11:49:57PM +0100, Christoph Hellwig wrote:
> > Let's hope these posts will stop when the UK starts to allow serving
> > drinks after 23:00. Post from half-drunk people that need to get a life
> > don't really help a lot.
>
> hi, christoph,
>
> i assume that your global world-wide distribution of this message
> was a mistake on your part. but, seeing as it _has_ gone out to
> literally thousands of extremely busy people, i can only apologise
> to them on your behalf for the mistake of wasting their valuable
> time.
If you think this, you don't know Christoph then. We all know him
for his "warm words" and his frankness. Sometimes he may be quite
a bit excessive, but here I tend to agree with him. He simply meant
that these long threads are often totally useless and consume a lot
of time to real developers. Generally speaking, calling developers
to tell them "hey, you work the wrong way" is not productive. If you
tell them that you can improve their work and show some code, it's
often more appreciated. Yes I know you provided some links to sites,
but without proposing one real guideline nor project. You'd better
have posted something like "announce: merging of l4linux into
mainline" and telling people that you will start a slow, non-intrusive
merging work of one of your favorite project. You'd have got lots
of opposition but this would have been more productive than just
speaking about philosophy.
> let's also hope that people who believe that comments such as the one
> that you have made are useful and productive also think about the
> consequences of doing so, bear in mind that internet archives are
> forever, and also that they check whether the person that they are
> criticising drinks at _all_.
[ cut all the uninteresting info about your drinking habits that you
sent to the whole world and which will be archived forever ]
Regards,
Willy
On Mon, Oct 03, 2005 at 01:54:00AM +0100, Luke Kenneth Casson Leighton wrote:
<snip>
> this company's hardware is kinda a bit difficult for most people to get
> their heads round: it's basically parallelised hardware-acceleration for
> operating systems, and very few people see the point in that.
Obviously, we are all clueless morons.
<snip>
> so.
>
> this is a heads-up.
>
> what you choose to do with this analysis is up to you.
>
> l.
Roll around on the floor while violently laughing for a while?
Allow me to apply Rusty's technique, if you will.
Luke Kenneth Casson Leighton wrote:
> Hi,
>
> Can all you great kernel hackers, who only know a little bit
> less than me and have only built a slightly less successful
> kernel than I have, stop what you are doing and do it my way
> instead?
>
Hi Luke,
Thanks for your concise and non-rambling letter that is actually
readable - a true rarity on lkml these days.
To answer your question: I think we would all be happy to examine
your ideas when you can provide some real numbers and comparisions
and actual technical arguments as to why they are better than the
current scheme we have in Linux.
Nick
PS. I am disappointed not to have seen any references to XML in
your proposal. May I suggest you adopt some kind of XML format
for your message protocol?
Send instant messages to your online friends http://au.messenger.yahoo.com
LKCL> the code for oskit has been available for some years, now,
LKCL> and is regularly maintained. the l4linux people have had to
My experience with oskit (trying to let students use it for OS course
homework) is quite ... underwhelming. It works as long as you try to use
it exactly like the developers did and breaks on a slightest sidestep
from that road. And there's not much documentation so it's hard to learn
where that road might be.
Switched to Linux/BSD code hacking with students, the code that actually
works.
YMMV.
--
Meelis Roos
On Mon, Oct 03, 2005 at 02:53:00AM +0100, Al Viro wrote:
> On Sun, Oct 02, 2005 at 06:50:46PM -0700, Vadim Lobanov wrote:
> > > visionary [n]: onanist with strong exhibitionist tendencies; from
> > > "visions", the source of inspiration they refer to when it becomes
> > > obvious that they have lost both sight and capacity for rational
> > > thought.
> > >
> >
> > Nice. :-)
> >
> > Just from idle curiosity, you wouldn't know where that list currently
> > resides, would you?
>
> No idea... Probably somebody from kernelnewbies.org crowd would know
> the current location...
There's not really a list of words on kernelnewbies.org, but a fortunes
file:
http://www.kernelnewbies.org/kernelnewbies-fortunes.tar.gz
(and I just updated it with your description of a visionary)
Erik
--
+-- Erik Mouw -- http://www.harddisk-recovery.nl -- 0800 220 20 20 --
| Eigen lab: Delftechpark 26, 2628 XH, Delft, Nederland
| Files foetsie, bestanden kwijt, alle data weg?!
| Blijf kalm en neem contact op met Harddisk-recovery.nl!
On 10/3/05, Gene Heskett <[email protected]> wrote:
> On Sunday 02 October 2005 19:48, Rik van Riel wrote:
> >On Sun, 2 Oct 2005, Gene Heskett wrote:
> >> Ahh, yes and no, Robert. The un-answered question, for that
> >> 512 processor Altix system, would be "but does it run things 512
> >> times faster?" Methinks not, by a very wide margin. Yes, do a lot
> >> of unrelated things fast maybe, but render a 30 megabyte page with
> >> ghostscript in 10 milliseconds? Never happen IMO.
> >
> >You haven't explained us why you think your proposal
> >would allow Linux to circumvent Amdahl's law...
>
> Amdahl's Law?
>
http://en.wikipedia.org/wiki/Amdahl's_law
http://home.wlu.edu/~whaleyt/classes/parallel/topics/amdahl.html
And google has even more. Wonderful thing those search engines...
--
Jesper Juhl <[email protected]>
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please http://www.expita.com/nomime.html
On Mon, 3 Oct 2005 02:53:02 +0100, Luke Kenneth Casson Leighton wrote:
> On Sun, Oct 02, 2005 at 06:20:38PM -0700, Vadim Lobanov wrote:
>> The beauty of capitalization is that it makes it easier for others to
>> read what you have to say.
>
> sorry, vadim: haven't touched a shift key in over 20 years.
>
[snip]
> *sigh* - i _so_ hope we don't need degrees in physics to program
> them...
[snip]
> ah, i'm not: i just left out mentioning it :)
I'd *love* a keyboard layout where * _ : ) are accesible without
shift! Can you send me yours?
--
Giuseppe "Oblomov" Bilotta
Hic manebimus optime
Luke Kenneth Casson Leighton <[email protected]> wrote:
> On Sun, Oct 02, 2005 at 04:37:52PM -0700, Vadim Lobanov wrote:
> > > what if, therefore, someone comes up with an architecture that is
> > > better than or improves greatly upon SMP?
> > Like NUMA?
> yes, like numa, and there is more.
>
> i had the honour to work with someone who came up with a radical
> enhancement even to _that_.
Any papers to look at?
> basically the company has implemented, in hardware (a nanokernel),
A nanokernel is a piece of software in my book?
> some
> operating system primitives, such as message passing (based on a
> derivative by thompson of the "alice" project from plessey, imperial and
> manchester university in the mid-80s), hardware cache line lookups
> (which means instead of linked list searching, the hardware does it for
> you in a single cycle), stuff like that.
Single CPU cycle for searching data in memory? Impossible.
> the message passing system is designed as a parallel message bus -
> completely separate from the SMP and NUMA memory architecture, and as
> such it is perfect for use in microkernel OSes.
Something must shuffle the data from "regular memory" into "message
memory", so I bet that soon becomes the bottleneck. And the duplicate data
paths add to the cost, money that could be spent on making memory access
faster, so...
> (these sorts of things are unlikely to make it into the linux kernel, no
> matter how much persuasion and how many patches they would write).
Your head would apin when looking at how fast this gets into Linux if there
were such machines around, and it is worth it.
> _however_, a much _better_ target would be to create an L4 microkernel
> on top of their hardware kernel.
Not yet another baroque CISC design, this time around with 1/3 of an OS in
it!
> this company's hardware is kinda a bit difficult for most people to get
> their heads round: it's basically parallelised hardware-acceleration for
> operating systems, and very few people see the point in that.
Perhaps most people that don't see the point do have a point?
> however, as i pointed out, 90nm and approx-2Ghz is pretty much _it_,
> and to get any faster you _have_ to go parallel.
Sorry, all this has been doomsayed (with different numbers) from 1965 or
so.
> and the drive for "faster", "better", "more sales" means more and more
> parallelism.
Right.
> it's _happening_ - and SMP ain't gonna cut it (which is why
> these multi-core chips are coming out and why hyperthreading
> is coming out).
Hyperthreading and multi-core /are/ SMP, just done a bit differently.
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
On 10/2/05, Luke Kenneth Casson Leighton <[email protected]> wrote:
> as i love to put my oar in where it's unlikely that people
> will listen, and as i have little to gain or lose by doing
> so, i figured you can decide for yourselves whether to be
> selectively deaf or not:
Hi Luke,
Haven't seen you since I believe you gave a somewhat interesting talk
on FUSE at an OxLUG a year or more back. I don't think anyone here is
selectively deaf, but some might just ignore you for such comments :-)
> what prompted me to send this message now was a recent report
> where linus' no1 patcher is believed to be close to overload,
> and in that report, i think it was andrew morton, it was
> said that he believed the linux kernel development rate to be
> slowing down, because it is nearing completion.
There was some general bollocks about Andrew being burned out, but
that wasn't what the point was as far as I saw it - more about how
things could be better streamlined than a sudden panic moment.
> i think it safe to say that a project only nears completion
> when it fulfils its requirements and, given that i believe that
> there is going to be a critical shift in the requirements, it
> logically follows that the linux kernel should not be believed
> to be nearing completion.
Whoever said it was?
> with me so far? :)
I don't think anyone with moderate grasp of the English language won't
have understood what you wrote above. They might not understand why
you said it, but that's another issue.
> the basic premise: 90 nanometres is basically... well...
> price/performance-wise, it's hit a brick wall at about 2.5Ghz, and
> both intel and amd know it: they just haven't told anyone.
But you /know/ this because you're a microprocessor designer as well
as a contributor to the FUSE project?
> anyone (big) else has a _really_ hard time getting above 2Ghz,
> because the amount of pipelining required is just... insane
> (see recent ibm powerpc5 see slashdot - what speed does it do?
> surprise: 2.1Ghz when everyone was hoping it would be 2.4-2.5ghz).
I think there are many possible reasons for that and I doubt slashdot
will reveal any of those reasons. The main issue (as I understand it)
is that SMT/SMP is taking off for many applications and manufacturers
want to cater for them while reducing heat output - so they care less
about MHz than about potential real world performance.
> so, what's the solution?
> well.... it's to back to parallel processing techniques, of course.
Yes. Wow! Of course! Whoda thunk it? I mean, parallel processing!
Let's get that right into the kern...oh wait, didn't Alan and a bunch
of others already do that years ago? Then again, we might have missed
all of the stuff which went into 2.2, 2.4 and then 2.6?
> well - intel is pushing "hyperthreading".
Wow! Really? I seem to have missed /all/ of those annoying ads. But
please tell me some more about it!
> and, what is the linux kernel?
> it's a daft, monolithic design that is suitable and faster on
> single-processor systems, and that design is going to look _really_
> outdated, really soon.
Why? I happen to think Microkernels are really sexy in a Computer
Science masturbatory kind of way, but Linux seems to do the job just
fine in real life. Do we need to have this whole
Microkernel/Monolithic conversation simply because you misunderstood
something about the kind of performance now possible in 2.6 kernels as
compared with adding a whole pointless level of message passing
underneath?
Jon.
> But you /know/ this because you're a microprocessor designer as well
> as a contributor to the FUSE project?
AFAIK Luke never contributed to the FUSE project. Hopefully that
answers your question.
FUSE and microkernels are sometimes mentioned together, but I believe
there's a very important philosophical difference:
FUSE was created to ease the development and use of a very _special_
group of filesystems. It was never meant to replace (and never will)
the fantastically efficient and flexible internal filesystem
interfaces in Linux and other monolithic kernels.
On the other hand, the microkernel approach is to restrict _all_
filesystems to the more secure, but less efficient and less flexible
interface. Which is stupid IMO.
Miklos
On Sun, 02 Oct 2005 22:12:38 EDT, Horst von Brand said:
> > some
> > operating system primitives, such as message passing (based on a
> > derivative by thompson of the "alice" project from plessey, imperial and
> > manchester university in the mid-80s), hardware cache line lookups
> > (which means instead of linked list searching, the hardware does it for
> > you in a single cycle), stuff like that.
>
> Single CPU cycle for searching data in memory? Impossible.
Well... if it was content-addressable RAM similar to what's already used for
the hardware TLB's and the like - just that it's one thing to make a 32 or 256
location content-addressable RAM, and totally another to have multiple megabytes
of the stuff. :)
Luke Kenneth Casson Leighton wrote:
> p.s. martin. _don't_ do that again. i don't care who you are:
> internet archives are forever and your rudeness will be noted
> by google-users and other search-users - long after you are dead.
and who are you, the thought police? Get off your high horse.
I'm sure he's well aware of the consequences of posting to this list, as
I'm sure we all are. Hell, even *I* know all my mails to this list are
going to be archived for eternity.
Look, if you want to be a productive member of our community, stop
bitching about the way things *should* be, and submit some patches like
everyone else. Code talks. Bullshit ... well, it doesn't do much but sit
around stinking the place up.
On Mon, Oct 03, 2005 at 10:50:00AM +0300, Meelis Roos wrote:
> LKCL> the code for oskit has been available for some years, now,
> LKCL> and is regularly maintained. the l4linux people have had to
>
> My experience with oskit (trying to let students use it for OS course
> homework) is quite ... underwhelming. It works as long as you try to use
> it exactly like the developers did and breaks on a slightest sidestep
> from that road. And there's not much documentation so it's hard to learn
> where that road might be.
>
> Switched to Linux/BSD code hacking with students, the code that actually
> works.
Can oskit be worse than nachos where the OS ran outside the memory space
and cpu with only applications being inside the emulated mips processor?
Made some things much too easy to do, and other things much to hard
(like converting an address from user space to kernel space an accessing
it, which should be easy, but was hard).
I suspect most 'simple' OS teaching tools are awful. Of course writing
a complete OS from scratch is a serious pain and makes debuging much
harder than if you can do your work on top of a working OS that can
print debug messages.
Len Sorensen
On Mon, Oct 03, 2005 at 02:53:02AM +0100, Luke Kenneth Casson Leighton wrote:
> sorry, vadim: haven't touched a shift key in over 20 years.
Except to type that ':' I suspect.
How about learning to use that shift key so that everyone else that
reads your typing don't have to spend so much time working it out when
proper syntax would have made it simpler. It may save you 1% of your
time typing it, but you cost thousands of people much more as a result.
net loss for the world as a whole.
> ah, i'm not: i just left out mentioning it :)
>
> the message passing needs to be communicated down to manage
> threads, and also to provide a means to manage semaphores and
> mutexes: ultimately, support for such an architecture would
> work its way down to libc.
>
> and yes, if you _really_ didn't want a kernel in the way at all, you
> could go embedded and just... do everything yourself.
>
> or port reactos, the free software reimplementation of nt,
> to it, or something :)
Microkernel, message passing, blah blah blah. Does GNU Hurd actually
run fast yet? I think it exists finally and works (at least mostly) but
how does the performance compare?
Most of your arguments seem like a repeat of more acedemic theories most
of which have not been used in a real system where performance running
average software was important, at least I never heard of them. Not
that that necesarily means much, other than they can't have gained much
popularity.
> it won't stop - but the price of 90nm mask charges, at approx
> $2m, is already far too high, and the number of large chips
> being designed is plummetting like a stone as a result - from
> like 15,000 per year a few years ago down to ... damn, can't remember -
> less than a hundred (i think! don't quote me on that!)
Hmm, so if we guess it might take 10 masks per processor type over it's
life time as they change features and such, that's still less than 1% of
the cost of the FAB in the first place. I agree with the person that
said intel/AMD/company probably don't care much, as long as their
engineers make really darn sure that the mask is correct when they go to
make one.
> okay: i will catch up on this bit, another time, because it is late
> enough for me to be getting dizzy and appearing to be drunk.
>
> this is one answer (and there are others i will write another time.
> hint: automated code analysis tools, auto-parallelising tools, both
> offline and realtime):
>
> watch what intel and amd do: they will support _anything_ - clutch at
> straws - to make parallelism palable, why? because in order to be
> competitive - and realistically priced - they don't have any choice.
>
> plus, i am expecting the chips to be thrown out there (like
> the X-Box 360 which has SIX hardware threads remember) and
> the software people to quite literally _have_ to deal with it.
Hey I like parallel processing. i think it's neat, and I have often
made some of my own tools multithreaded just because I found it could be
done for the task, and I often find multithreaded simpler to code (I
seem to be a bit of a weirdo that way) for certain tasks.
> i expect the hardware people to go: this is the limit, this is what we
> can do, realistically price-performance-wise: lump it, deal with it.
>
> when intel and amd start doing that, everyone _will_ lump it.
> and deal with it.
>
> ... why do you think intel is hyping support for and backing
> hyperthreads support in XEN/Linux so much?
Ehm, because intel has it and their P4 desperately needs help to gain
any performance it can until they get the Pentium-M based desktop chips
finished with multiple cores, and of course because AMD doesn't have it.
Seem like good reasons for intel to try and push it.
Len Sorensen
On Mon, 3 Oct 2005, Lennart Sorensen wrote:
> On Mon, Oct 03, 2005 at 10:50:00AM +0300, Meelis Roos wrote:
>> LKCL> the code for oskit has been available for some years, now,
>> LKCL> and is regularly maintained. the l4linux people have had to
>>
>> My experience with oskit (trying to let students use it for OS course
>> homework) is quite ... underwhelming. It works as long as you try to use
>> it exactly like the developers did and breaks on a slightest sidestep
>> from that road. And there's not much documentation so it's hard to learn
>> where that road might be.
>>
>> Switched to Linux/BSD code hacking with students, the code that actually
>> works.
>
> Can oskit be worse than nachos where the OS ran outside the memory space
> and cpu with only applications being inside the emulated mips processor?
> Made some things much too easy to do, and other things much to hard
> (like converting an address from user space to kernel space an accessing
> it, which should be easy, but was hard).
>
> I suspect most 'simple' OS teaching tools are awful. Of course writing
> a complete OS from scratch is a serious pain and makes debuging much
> harder than if you can do your work on top of a working OS that can
> print debug messages.
>
> Len Sorensen
> -
But the first thing you must do in a 'roll-your-own' OS is to make
provisions to write text to (sometimes a temporary) output device
and get some input from same. Writing such basic stuff is getting
harder because many embedded systems don't have UARTS, screen-cards,
keyboards, or any useful method of doing I/O. This is where an
existing OS (Like Linux) can help you get some I/O running, perhaps
through a USB bus. You debug and make it work as a Linux
Driver, then you link the working stuff into your headless CPU
board.
Cheers,
Dick Johnson
Penguin : Linux version 2.6.13 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.
****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.
Thank you.
On Sul, 2005-10-02 at 22:55 -0400, [email protected] wrote:
> The HP2114 and DEC KL10/20 were able to dereference a chain of indirect bits
> back in the 70's (complete with warnings that hardware wedges could occur if
> an indirect reference formed a loop or pointed at itself).
The KL10 has an 8 way limit. The PDP-6 didn't but then it also lacked an
MMU.
On Llu, 2005-10-03 at 01:54 +0100, Luke Kenneth Casson Leighton wrote:
> the message passing system is designed as a parallel message bus -
> completely separate from the SMP and NUMA memory architecture, and as
> such it is perfect for use in microkernel OSes.
I've got one of those. It has the memory attached. Makes a fantastic
message bus and has a really long queue. Also features shortcuts for
messages travelling between processors in short order cache to cache.
Made by AMD and Intel.
> however, as i pointed out, 90nm and approx-2Ghz is pretty much _it_,
> and to get any faster you _have_ to go parallel.
We do 512 processors passably now. Thats a lot of cores and more than
the commodity computing people can wire to memory subsystems at a price
people will pay.
Besides which you need to take it up with the desktop people really. Its
their apps that use most of the processor power and will benefit most
from parallelising and efficiency work.
Alan
On Mon, Oct 03, 2005 at 02:08:58PM -0400, Lennart Sorensen wrote:
> On Mon, Oct 03, 2005 at 10:50:00AM +0300, Meelis Roos wrote:
> > LKCL> the code for oskit has been available for some years, now,
> > LKCL> and is regularly maintained. the l4linux people have had to
> >
> > My experience with oskit (trying to let students use it for OS course
> > homework) is quite ... underwhelming. It works as long as you try to use
> > it exactly like the developers did and breaks on a slightest sidestep
> > from that road. And there's not much documentation so it's hard to learn
> > where that road might be.
analysis, verification, debugging and adoption of oskit by
the linux kernel maintainers would help enormously there,
i believe, which is why i invited the kernel maintainers to
give it some thought.
there are other reasons: not least is that oskit _is_ the
linux kernel source code - with the kernel/* bits removed and
the device drivers and support infrastructure remaining.
so the developers who split the linux source code out into oskit did
not, in your opinion and experience, meelis, do a very good job: so
educate them and tell them how to do it better.
l.
On Mon, Oct 03, 2005 at 12:32:35PM -0400, [email protected] wrote:
> On Sun, 02 Oct 2005 22:12:38 EDT, Horst von Brand said:
>
> > > some
> > > operating system primitives, such as message passing (based on a
> > > derivative by thompson of the "alice" project from plessey, imperial and
> > > manchester university in the mid-80s), hardware cache line lookups
> > > (which means instead of linked list searching, the hardware does it for
> > > you in a single cycle), stuff like that.
> >
> > Single CPU cycle for searching data in memory? Impossible.
>
> Well... if it was content-addressable RAM similar to what's already used for
> the hardware TLB's and the like - just that it's one thing to make a 32 or 256
> location content-addressable RAM, and totally another to have multiple megabytes
> of the stuff. :)
aspex microelectronics 4096 2-bit massively parallel SIMD
processor (does 1 terabit-ops / sec @ 250mhz which sounds a
lot until you try to do FPU emulation on it).
each 2-bit processor has 256 bits of content-addressable memory,
which can be 8-bit, 16-bit or 32-bit addressed (to make 4096 parallel
memory searches - in a single cycle).
absolutely friggin blindingly fast for certain issues (video
processing, certain kinds of audio processing - e.g. FFTs,
XML and HTTP parsing), and pissed all over for others such
as doing floating point arithmetic.
but anyway: that's a side issue. thanks for reminding me about CAM,
valdis.
l.
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
On Mon, Oct 03, 2005 at 06:00:56PM +0200, Miklos Szeredi wrote:
> > But you /know/ this because you're a microprocessor designer as well
> > as a contributor to the FUSE project?
>
> AFAIK Luke never contributed to the FUSE project. Hopefully that
> answers your question.
wrong.
i added xattr support to fuse, for use in selinux. it's a long story.
http://www.ussg.iu.edu/hypermail/linux/kernel/0409.2/1441.html
and yes, for the record, i am just as comfortable with hardware
designs as with software, having designed a massively parallel
encryption algorithm capable of doing up to about 16384-bit
block sizes with key sizes of up to around 8192-bits, (which
unfortunately wasn't very fast in software - you can't have
everything), came up with some significant improvements to
the plessey/imperial-uni/man-uni ALICE parallel transputer
network as a third year project, and also provided aspex,
the massively-parallel SIMD processor company, with enough
new material and ideas in four months for them to have to
register six new patents.
... why are you people bothering to attempt to go "oh, this
guy must not know anything therefore we'll waste the list's
time with our opinions on whether he cannot do anything",
such that i have to refute you, and look like a complete
egg-head jumped-up i'm-better-than-you horn-blowing tosser?
stop it!
everyone has their level and areas of expertise: instead of
turning this into a pissing contest, be glad and humbled for
an opportunity to learn from each other.
l.
> > > But you /know/ this because you're a microprocessor designer as well
> > > as a contributor to the FUSE project?
> >
> > AFAIK Luke never contributed to the FUSE project. Hopefully that
> > answers your question.
>
> wrong.
>
> i added xattr support to fuse, for use in selinux. it's a long story.
>
> http://www.ussg.iu.edu/hypermail/linux/kernel/0409.2/1441.html
>
Well, adding is a different thing from contribution.
Contribution is when you add something and also share it with the
maintainer of said software.
I can't remember you doing that, so that's why I said what I said.
Miklos
On 10/3/05, linux-os (Dick Johnson) <[email protected]> wrote:
>
> On Mon, 3 Oct 2005, Lennart Sorensen wrote:
> > I suspect most 'simple' OS teaching tools are awful. Of course writing
> > a complete OS from scratch is a serious pain and makes debuging much
> > harder than if you can do your work on top of a working OS that can
> > print debug messages.
> But the first thing you must do in a 'roll-your-own' OS is to make
> provisions to write text to (sometimes a temporary) output device
> and get some input from same.
Indeed. I started work on a microkernel for a final year University
project. Didn't get very far beyond minimal memory management and a
vague handy-wavy concept of a process in the end as it's easy to get
unstuck figuring out random blackbox hardware. Makes you respect some
of these people who really figured it out for real and got it working.
> Writing such basic stuff is getting harder because many embedded
> systems don't have UARTS, screen-cards, keyboards, or any useful
> method of doing I/O.
It's easier now that we have a growing number of cheaper ARM/PPC
boards on the market. But in order to do much of this, you really need
a hardware debugger. In my case, I tried to do this on an Apple
powerbook but once you've broken the BAT/page mapping for your
framebuffer you're rapidly running out of ways of debugging, e.g. a
VM. It's difficult enough even with a UART, or an LED, or whatever.
> This is where an existing OS (Like Linux) can help you get some I/O
> running, perhaps through a USB bus. You debug and make it work
> as a Linux Driver, then you link the working stuff into your headless
> CPU board.
A lot of people end up doing that - I've heard of some interesting
stories which I'm sure aren't widespread. One case, the guy had
basically bolted a small realtime module on to Linux (not really quite
like RTLinux) but had been able to do a lot of testing through
existing APIs. Another trick is to write as much as you can to sit
right atop the existing firmware - OpenFirmware, U-Boot, whatever and
perhaps even forgo trying to handle exceptions/VM for yourself in the
beginning.
Jon.
On Mon, Oct 03, 2005 at 03:20:46PM +0100, Jon Masters wrote:
> On 10/2/05, Luke Kenneth Casson Leighton <[email protected]> wrote:
>
> > as i love to put my oar in where it's unlikely that people
> > will listen, and as i have little to gain or lose by doing
> > so, i figured you can decide for yourselves whether to be
> > selectively deaf or not:
>
> Hi Luke,
hellooo jon.
> Haven't seen you since I believe you gave a somewhat interesting talk
> on FUSE at an OxLUG a year or more back.
good grief, that long ago?
> I don't think anyone here is
> selectively deaf, but some might just ignore you for such comments :-)
pardon? oh - yes, i'm counting on it, for a good signal/noise
ratio. sad to recount, the strategy ain't workin too well,
oh well. i've received about 10 hate mails so far, i _must_
be doing _something_ right.
> > i think it safe to say that a project only nears completion
> > when it fulfils its requirements and, given that i believe that
> > there is going to be a critical shift in the requirements, it
> > logically follows that the linux kernel should not be believed
> > to be nearing completion.
>
> Whoever said it was?
istrc it was in andrew morton's interview / comments :)
> > the basic premise: 90 nanometres is basically... well...
> > price/performance-wise, it's hit a brick wall at about 2.5Ghz, and
> > both intel and amd know it: they just haven't told anyone.
>
> But you /know/ this because you're a microprocessor designer as well
> as a contributor to the FUSE project?
i have been speaking on a regular basis with someone who
has been dealing for nearly twenty years now with processor
designs (from a business perspective, for assessing high-tech
companies for investment and recruitment purposes). i have
been fortunate enough to have the benefit of their experience
in assessing the viability of chip designs.
i haven't created silicon, but i've studied processor designs until
they were coming out of my ears.
... you are mistaken on one point, though: my work on fuse proved
unsuccessful because i was a) running out of time b) running out of
reasons to continue (the deal fell through) c) i ran into an error
message in selinux: the "please try later" one, which flummoxed me but
i now believe to be due to a crash in the userspace stuff. maybe.
urk. it's been a year.
anyway, we digress.
> > anyone (big) else has a _really_ hard time getting above 2Ghz,
> > because the amount of pipelining required is just... insane
> > (see recent ibm powerpc5 see slashdot - what speed does it do?
> > surprise: 2.1Ghz when everyone was hoping it would be 2.4-2.5ghz).
>
> I think there are many possible reasons for that and I doubt slashdot
> will reveal any of those reasons.
probably :)
> The main issue (as I understand it)
> is that SMT/SMP is taking off for many applications and manufacturers
> want to cater for them while reducing heat output - so they care less
> about MHz than about potential real world performance.
pipelining. pipelining. latency between blocks.
halving the microns should quadruple the speed: the distance is halved
so light has half the distance to travel and ... darn, can't remember
the other reason for the other factor-of-two.
if the latency between sub-blocks is large (or becomes
relevant), then it doesn't matter _what_ microns you attempt to
run in, your design will asymtote towards an upper speed limit.
if you're having to pipeline down to the level of 2-bit adders with 16
stages in order to do 32-bit adds at oh say 4ghz, you _know_ you're in
trouble.
> > so, what's the solution?
>
> > well.... it's to back to parallel processing techniques, of course.
>
> Yes. Wow! Of course! Whoda thunk it? I mean, parallel processing!
> Let's get that right into the kern...oh wait, didn't Alan and a bunch
> of others already do that years ago? Then again, we might have missed
> all of the stuff which went into 2.2, 2.4 and then 2.6?
jon, jon *sigh* :) i meant _hardware_ parallel processing - i wasn't
referring to anything led or initiated by the linux kernel, but instead
to the simple conclusion that if hardware is running out of steam in
uniprocessor (monster-pipelined; awful-prediction;
let's-put-five-separately-designed-algorithms-for-divide-into-the-chip-and-take-the-answer-of-the-first-unit-that-replies sort of design)
then chip designers are forced to parallelise.
> > well - intel is pushing "hyperthreading".
>
> Wow! Really? I seem to have missed /all/ of those annoying ads. But
> please tell me some more about it!
make me. nyer :)
> > and, what is the linux kernel?
>
> > it's a daft, monolithic design that is suitable and faster on
> > single-processor systems, and that design is going to look _really_
> > outdated, really soon.
>
> Why? I happen to think Microkernels are really sexy in a Computer
> Science masturbatory kind of way, but Linux seems to do the job just
> fine in real life. Do we need to have this whole
> Microkernel/Monolithic conversation simply because you misunderstood
> something about the kind of performance now possible in 2.6 kernels as
> compared with adding a whole pointless level of message passing
> underneath?
i'll answer rik's point later when i've thought about it some
more: in short, rik has concluded that because i advocated
message passing that somehow his SMP improvement work
(isolating data structures) is irrelevant - far from it: such
improvements would prove, i believe, to be a significant additional
augmentation, unburdening both a monolithic _and_ a microkernel'd linux
kernel from some of the cru-joze nastiness of SMP.
if there are better ways, _great_.
love to hear them.
l.
On Mon, Oct 03, 2005 at 12:36:20PM -0700, Joe Bob Spamtest wrote:
> The point being: If and when the industry switches its focus to highly
> parallel systems, Linux will shortly follow.
joe: hi, thanks for responding. i believe this to be a very
sound strategy, and given the technical expertise of the kernel
developers i have confidence in their abilities to pull that off.
personally i find that i like a bit of a run-up and/or advance notice
of major paradigm shifts. on the basis that other people might also
want to know, i initiated this discussion yesterday and it seems like
forever already! :)
l.
oh, and joe? my wife is the one with the high horse, not me.
she qualified for the national british dressage championships which
was last month, and came 17th in the country, at elementary
level, on her beautiful pony, blue. i am very proud of her.
http://www.bdchampionships.co.uk
On Mon, Oct 03, 2005 at 08:18:40PM +0100, Alan Cox wrote:
> On Llu, 2005-10-03 at 01:54 +0100, Luke Kenneth Casson Leighton wrote:
> > the message passing system is designed as a parallel message bus -
> > completely separate from the SMP and NUMA memory architecture, and as
> > such it is perfect for use in microkernel OSes.
>
> I've got one of those. It has the memory attached. Makes a fantastic
> message bus and has a really long queue. Also features shortcuts for
> messages travelling between processors in short order cache to cache.
> Made by AMD and Intel.
made? _cool_. actual hardware. new knowledge for me. do you know
of any online references, papers or stuff? [btw just to clarify:
you're saying you have a NUMA bus or you're saying you have an
augmented SMP+NUMA+separate-parallel-message-passing-bus er .. thing]
> > however, as i pointed out, 90nm and approx-2Ghz is pretty much _it_,
> > and to get any faster you _have_ to go parallel.
>
> We do 512 processors passably now.
wild.
> Thats a lot of cores and more than
> the commodity computing people can wire to memory subsystems at a price
> people will pay.
oops.
whereas, would you see it more reasonable for a commodity-level
chip to be something like 32- or even 64- ultra-RISC cores of
between 5000 and 10,000 gates each, resulting in a processor
of about 50% cache memory and 50% processing plus associated
parallel bus architecture at around 1 million gates?
running at oh say 1ghz or with careful design effort focussed on the
RISC cores maybe even 2ghz, resulting in 128 total GigaOps if you
go for 64 cpus @ 2ghz. that's a friggin lot of processing power
for a 1m gates processor!!
(hey, see, i can learn to use the shift key to highlight the keyword)
such a chip, in 90nm, would be approx $USD 20 in mass production.
small, good heat distribution, probably too many pins, probably
need some DRAM memory stamped upside down on top of the die,
instead of off-chip [putting DRAM and transistors on the same die
is a frequent and costly mistake: the yields are terrible].
putting DRAM upside down on top of a die and then hitting it
with a hammer (literally) is a frequently used technique to
avoid the problem.
> Besides which you need to take it up with the desktop people really.
i'm sort-of drafting a reply to rik's point in my head (no it's not
reiterations of things already said) and yes it involves legacy apps,
badly written apps, desktop focus etc.
server-side, yep, fine, 32-way, use it all, got it, heck,
even my apache server running on a P200 w/64mb RAM runs more
processes than that.
l.
On Mon, Oct 03, 2005 at 01:03:48AM -0400, Sonny Rao wrote:
> Roll around on the floor while violently laughing for a while?
_excellent_! can we watch? where's the mp4 :)
On Sun, Oct 02, 2005 at 10:55:49PM -0400, [email protected] wrote:
> On Mon, 03 Oct 2005 01:54:00 BST, Luke Kenneth Casson Leighton said:
>
> > in the mid-80s), hardware cache line lookups (which means
> > instead of linked list searching, the hardware does it for
> > you in a single cycle), stuff like that.
>
> OK.. I'll bite. How do you find the 5th or 6th entry in the linked list,
> when only the first entry is in cache, in a single cycle, when a cache line
> miss is more than a single cycle penalty, and you have several "These are not
> the droids you're looking for" checks and go on to the next entry - and do it
> in one clock cycle?
i was not privy to the design discussions: unfortunately i was only
given brief conclusions and hints by the designer.
my guess is that yes, as the later messages in this thread
hint at, CAM is probably the key: 256 blocks of 32-bit CAM,
something like that.
CAM is known to help dramatically decrease execution time
by orders of magnitude in linked list algorithms such as
searching and sorting, esp. where each CAM cell has built-in
processing, like the aspex.net massively-deep SIMD architecture has.
l.
On 3 Oct 2005, Giuseppe Bilotta prattled cheerily:
> I'd *love* a keyboard layout where * _ : ) are accesible without
> shift! Can you send me yours?
<http://www.maltron.co.uk/images/press/maltron-ergonomic-english-trackball-tq-hr1.jpg>
(downside: cost. Upside: feels lovely.)
--
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James Nicoll
On Llu, 2005-10-03 at 22:07 +0100, Luke Kenneth Casson Leighton wrote:
> made? _cool_. actual hardware. new knowledge for me. do you know
> of any online references, papers or stuff? [btw just to clarify:
> you're saying you have a NUMA bus or you're saying you have an
> augmented SMP+NUMA+separate-parallel-message-passing-bus er .. thing]
Its a standard current Intel feature. See "mwait" in the processor
manual. The CPUs are also smart enough to do cache to cache transfers.
No special hardware no magic.
And unless I want my messages to cause interrupts and wake events (in
which case the APIC does it nicely) then any locked operation on memory
will do the job just fine. I don't need funky hardware on a system. The
first point I need funky hardware is between boards and that isn't
consumer any more.
Alan
On Mon, Oct 03, 2005 at 09:22:39PM +0100, Luke Kenneth Casson Leighton wrote:
> On Mon, Oct 03, 2005 at 03:20:46PM +0100, Jon Masters wrote:
> > On 10/2/05, Luke Kenneth Casson Leighton <[email protected]> wrote:
> hellooo jon.
>
> > Haven't seen you since I believe you gave a somewhat interesting talk
> > on FUSE at an OxLUG a year or more back.
>
> good grief, that long ago?
Indeed. Time flies.
> > I don't think anyone here is
> > selectively deaf, but some might just ignore you for such comments :-)
>
> pardon? oh - yes, i'm counting on it, for a good signal/noise
> ratio. sad to recount, the strategy ain't workin too well,
> oh well. i've received about 10 hate mails so far, i _must_
> be doing _something_ right.
I hate to sound like "one of them" but nothing you've said is revolutionary -
really it's not - but you're saying it as if you just dreamed it up today and
/that's/ what got people annoyed. One of those pseudo-intellectual Starbucks
moments, if you will.
> > > the basic premise: 90 nanometres is basically... well...
> > > price/performance-wise, it's hit a brick wall at about 2.5Ghz, and
> > > both intel and amd know it: they just haven't told anyone.
> >
> > But you /know/ this because you're a microprocessor designer as well
> > as a contributor to the FUSE project?
>
> i have been speaking on a regular basis with someone who
> has been dealing for nearly twenty years now with processor
> designs (from a business perspective, for assessing high-tech
> companies for investment and recruitment purposes). i have
> been fortunate enough to have the benefit of their experience
> in assessing the viability of chip designs.
In other words, no. I'm not a processor designer either (very few people
are) but I do have a lot of experience with Xilinx FPGAs and I'll add
something of relevence to the end of this message. The point really is
that some people here know a great deal more than you do about this (and
I'm not saying I'm one of them), they're going to rightfully feel a
little annoyed if you start preeching to the choir.
> > The main issue (as I understand it)
> > is that SMT/SMP is taking off for many applications and manufacturers
> > want to cater for them while reducing heat output - so they care less
> > about MHz than about potential real world performance.
>
> pipelining. pipelining. latency between blocks.
Would you mind learning to use capitali[sz]ation in your mail? It's
really not very easy to read what you write. Was the above intended to
be an actual sentence (in which case I can't see any clauses in the
above) or just random words which - if said together - will summon some
mystical power upon us?
> > > so, what's the solution?
> >
> > > well.... it's to back to parallel processing techniques, of course.
> >
> > Yes. Wow! Of course! Whoda thunk it? I mean, parallel processing!
> > Let's get that right into the kern...oh wait, didn't Alan and a bunch
> > of others already do that years ago? Then again, we might have missed
> > all of the stuff which went into 2.2, 2.4 and then 2.6?
>
> jon, jon *sigh* :)
My point was that some very much more cleaver people have worked on this
for a very long time already. Alan did a lot of cool stuff in the
beginning, what Ingo does now just scares me, etc.
> i meant _hardware_ parallel processing
Old hat. It's been worked on for over a decade and some of it is working
out now in the form of concept processors that mix FPGAs with CPUs.
> - i wasn't referring to anything led or initiated by the linux kernel,
> but instead to the simple conclusion that if hardware is running out of
> steam in uniprocessor (monster-pipelined; awful-prediction;
> let's-put-five-separately-designed-algorithms-for-divide-into-the-chip-and-take-the-answer-of-the-first-unit-that-replies sort of design)
> then chip designers are forced to parallelise.
So simple in fact that it's already done in most common hardware. Why
else would we have any offload chips at all?
Please help me to understand the value of your original message. I've
read it a few times, but it continues to allude me.
Jon.
On Mon, Oct 03, 2005 at 10:12:26PM +0100, Luke Kenneth Casson Leighton wrote:
> On Mon, Oct 03, 2005 at 01:03:48AM -0400, Sonny Rao wrote:
> > Roll around on the floor while violently laughing for a while?
>
> _excellent_! can we watch? where's the mp4 :)
It'll be out there shortly... ;-P
But realistically, how can I take you seriously?
You come onto the Linux kernel mailing list, talk about hardware
implementations that are not generally known or available while
insulting the populace by claiming they would not understand
anyway, then insult the kernel developers by proclaiming their hard work
is "daft" and that the design is going to be "_really_ outdated,
really soon" without much explanation, go on to make wildly
generalized comments about processor frequency scaling with little
real insight or explanation of the issues, and apparently like making
long meandering posts even more difficult to read by not using capital
letters.
I also notice you suddenly decided to act morally superior to Martin
by taking "a leaf from the great rusty russell's book " and dealing
with "immature and out-of-line comments" in a "professional way."
Way to go...
Anyway, others have posted far more mature responses than my obvious
flamebait... apologies for the extra noise.
Sonny
Luke Kenneth Casson Leighton wrote:
> halving the microns should quadruple the speed: the distance is halved
> so light has half the distance to travel and ... darn, can't remember
> the other reason for the other factor-of-two.
2 dimensions?
--
Jason Stubbs
On Mon, 03 Oct 2005 22:07:22 BST, Luke Kenneth Casson Leighton said:
> whereas, would you see it more reasonable for a commodity-level
> chip to be something like 32- or even 64- ultra-RISC cores of
> between 5000 and 10,000 gates each, resulting in a processor
> of about 50% cache memory and 50% processing plus associated
> parallel bus architecture at around 1 million gates?
Read your history - especially IBM's 801 chipset, which became the RT,
and why they then replaced that with the Power architecture...
> running at oh say 1ghz or with careful design effort focussed on the
> RISC cores maybe even 2ghz, resulting in 128 total GigaOps if you
> go for 64 cpus @ 2ghz. that's a friggin lot of processing power
> for a 1m gates processor!!
Good. Were you planning to run the ucLinux branch on this, or include all
the pieces needed to support virtual memory? And do it inside that 10K gate
budget, too (hint - how many gates will you burn just doing a TLB big enough
to get the performance of mapping a virtual->real address to be good enough?)
You might want to read up on all the fun that IBM went through in designing
memory subsystems that can keep even the Power4 and Power5 chipsets fed too,
or the interesting stuff that SGI has to do to allow 64/128/512 processors
to beat up on a memory system - I'm sure there's some pretty high gate count
involved there..
If you're doing 64 10K-gate cores, you've blown 64% of your 1M gate budget.
You've got only 320K gates left to build cache *and* virtual memory support to
make that 1M gate budget. And yes, you need a cache, as IBM found out on
their RT processor.....
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Luke
> Kenneth Casson Leighton
> Sent: Monday, October 03, 2005 1:31 PM
> To: Joe Bob Spamtest
> Cc: linux-kernel
> Subject: Re: what's next for the linux kernel?
>
> On Mon, Oct 03, 2005 at 12:36:20PM -0700, Joe Bob Spamtest wrote:
> > The point being: If and when the industry switches its focus to
> > highly parallel systems, Linux will shortly follow.
>
>
> personally i find that i like a bit of a run-up and/or
> advance notice of major paradigm shifts. on the basis that
> other people might also want to know, i initiated this
> discussion yesterday and it seems like forever already! :)
There's no real need to hurry. The industry isn't going to shift its
focus to highly parallel systems in the near future. Highly parallel
systems have been around since the 60s, (when 'highly parallel' meant a
lot less than it does today, but was still parallel compared to typical
von neumann machines of the period.)
It's been fifteen years since I last played with them, and there's not
much chance that they'll get interesting in the near future.
They'll never go completely away, because there'll always be niches
where they make sense, but they'll never break out into the mainstream,
because those niches tend to get smaller, not larger, over time.
On Tue, Oct 04, 2005 at 10:33:16AM +0900, Jason Stubbs wrote:
> Luke Kenneth Casson Leighton wrote:
> > halving the microns should quadruple the speed: the distance is halved
> > so light has half the distance to travel and ... darn, can't remember
> > the other reason for the other factor-of-two.
>
> 2 dimensions?
Voltage-squared. capacitance. when you go down the microns, your
capacitance drops and the voltage squared goes down, too.
0.65nm is 1.2v
0.45 is aiming for 0.9 volts.
silicon germanium is going to hit a limit real soon.
you can't go below 0.8 volts, that's the gate "off" threshold.
l.
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
> Hmm, so if we guess it might take 10 masks per processor type over it's
> life time as they change features and such, that's still less than 1% of
> the cost of the FAB in the first place. I agree with the person that
> said intel/AMD/company probably don't care much, as long as their
> engineers make really darn sure that the mask is correct when they go to
> make one.
you elaborate, therefore, on my point.
anyone else, therefore, cannot hope to compete or even enter the
market, at 90nm.
which is why the first VIA eden processors maxed out at 800mhz (i'm
guessing they were a 0.13micron and therefore 2.5 volts)
> > ... why do you think intel is hyping support for and backing
> > hyperthreads support in XEN/Linux so much?
>
> Ehm, because intel has it and their P4 desperately needs help to gain
> any performance it can until they get the Pentium-M based desktop chips
> finished with multiple cores, and of course because AMD doesn't have it.
> Seem like good reasons for intel to try and push it.
you lend weight to my earlier points: the push is to
drive the engineers towards less gates on the excuse of
cart-before-horsing the market with their "performance / watt"
metrics, such that if 0.65nm comes off it's less painful
and not too much of a jump, and they aim for more parallel
processing (multiple cores).
current : 200 million gates with 90nm at 1.65 volt
estimated: 40 million gates with 65nm at 1.1 volt
estimated: 1 million gates with 45nm at 0.9 volt.
the "off" voltage of a silicon germanium transistor is 0.8 volts.
at 45nm the current leakage is so insane that the heat
dissipation, through the oxide layer which covers the chip,
ends up blowing the chip up.
trouble.
l.
On Sun, Oct 02, 2005 at 08:27:45PM -0500, Chase Venters wrote:
> The bottom line is that the application developers need to start being clever
> with threads.
yep! ah. but. see this:
http://lists.samba.org/archive/samba-technical/2004-December/038300.html
and think what would happen if glibc had hardware-support for
semaphores and mutexes.
> I think I remember some interesting rumors about Perl 6, for
> example, including 'autothreading' support - the idea that your optimizer
> could be smart enough to identify certain work that can go parallel.
http://www.ics.ele.tue.nl/~sander/publications.php
http://portal.acm.org/citation.cfm?id=582068
http://csdl.computer.org/comp/proceedings/acsd/2003/1887/00/18870237.pdf
to get the above references, put in "holland parallel code
analysis tools" into google.com.
put in "parallel code analysis tools" into google.com for a different
set.
l.
On Tue, 4 Oct 2005, Luke Kenneth Casson Leighton wrote:
>> Hmm, so if we guess it might take 10 masks per processor type over it's
>> life time as they change features and such, that's still less than 1% of
>> the cost of the FAB in the first place. I agree with the person that
>> said intel/AMD/company probably don't care much, as long as their
>> engineers make really darn sure that the mask is correct when they go to
>> make one.
>
> you elaborate, therefore, on my point.
>
> anyone else, therefore, cannot hope to compete or even enter the
> market, at 90nm.
>
> which is why the first VIA eden processors maxed out at 800mhz (i'm
> guessing they were a 0.13micron and therefore 2.5 volts)
>
>>> ... why do you think intel is hyping support for and backing
>>> hyperthreads support in XEN/Linux so much?
>>
>> Ehm, because intel has it and their P4 desperately needs help to gain
>> any performance it can until they get the Pentium-M based desktop chips
>> finished with multiple cores, and of course because AMD doesn't have it.
>> Seem like good reasons for intel to try and push it.
>
> you lend weight to my earlier points: the push is to
> drive the engineers towards less gates on the excuse of
> cart-before-horsing the market with their "performance / watt"
> metrics, such that if 0.65nm comes off it's less painful
> and not too much of a jump, and they aim for more parallel
> processing (multiple cores).
>
> current : 200 million gates with 90nm at 1.65 volt
> estimated: 40 million gates with 65nm at 1.1 volt
> estimated: 1 million gates with 45nm at 0.9 volt.
>
> the "off" voltage of a silicon germanium transistor is 0.8 volts.
>
> at 45nm the current leakage is so insane that the heat
> dissipation, through the oxide layer which covers the chip,
> ends up blowing the chip up.
>
> trouble.
>
> l.
Since the voltage goes down as the speed increases, an
industry joke has been that at 0 volts you can get the
speed that you want. Unfortunately, the required infinite
current is a bit hard to manage. This brings up the
configuration change that has been in the works for
some time, current-mode logic. Just like TTL gave way
to ECL to obtain two orders of magnitude increase in
random-logic speed, there will be a similar increase once
current rather than voltage is used for logic levels.
And, it's not absolute current, either but delta-current
which will define a logic state.
The problems with reducing capacity end up being exacerbated
when logic states are stored in the very capacity that is
being reduced. Eventually there is little difference between
"logic" and "noise". This brings up the new science of
"statistical logic" which has yet to make its way into
microprocessors, but soon will once the quantization noise
becomes a factor.
Cheers,
Dick Johnson
Penguin : Linux version 2.6.13 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.
****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.
Thank you.
On Tue, Oct 04, 2005 at 01:53:54PM +0100, Luke Kenneth Casson Leighton wrote:
> you elaborate, therefore, on my point.
>
> anyone else, therefore, cannot hope to compete or even enter the
> market, at 90nm.
>
> which is why the first VIA eden processors maxed out at 800mhz (i'm
> guessing they were a 0.13micron and therefore 2.5 volts)
Sticking with 0.13 also seems to make for a better embedded processor
since it seems when you go to 0.09 or smaller it becomes nearly
impossible to run at high and low temperatures which is what some
embedded applications want (like -40 to +85C). We had to change compact
flash suppliers when our supplier went to 90nm since they said their new
process couldn't work reliably at industrial temperature anymore. If
VIA wants the eden to run at a wide temperature range, it appears they
are better off sticking to a larger process and keeping the cpu speed
down to something reasonable. I imagine at some point someone will
manage to make a 90nm chip that does handle bigger temperature ranges
but I haven't seen one yet myself.
> you lend weight to my earlier points: the push is to
> drive the engineers towards less gates on the excuse of
> cart-before-horsing the market with their "performance / watt"
> metrics, such that if 0.65nm comes off it's less painful
> and not too much of a jump, and they aim for more parallel
> processing (multiple cores).
If that was the goal the x86 architecture should be dumped. It spends
too many gates converting x86 junk into something reasonable to execute.
People don't appear very eager to dump the x86 unfortunately. Something
to do with backwards compatibility and such.
> current : 200 million gates with 90nm at 1.65 volt
> estimated: 40 million gates with 65nm at 1.1 volt
> estimated: 1 million gates with 45nm at 0.9 volt.
A 1 million gate chip at 45nm would be rather tiny. The yield would
probably be amazingly good. Of course if it does give off a lot of heat
you still have the problem of how to get rid of the heat given it is
focused in a very very small space. Of course just reducing the size of
the cache on intel's chips to something sane would reduce the gate count
enourmously. But that won't happen until they make a more efficient
chip.
> the "off" voltage of a silicon germanium transistor is 0.8 volts.
>
> at 45nm the current leakage is so insane that the heat
> dissipation, through the oxide layer which covers the chip,
> ends up blowing the chip up.
Unless someone finds a way to reduce the leakage. It is worth a lot of
money to some companies to solve that problem after all.
Len Sorensen
Alan Cox <[email protected]> writes:
> On Llu, 2005-10-03 at 22:07 +0100, Luke Kenneth Casson Leighton wrote:
> > made? _cool_. actual hardware. new knowledge for me. do you know
> > of any online references, papers or stuff? [btw just to clarify:
> > you're saying you have a NUMA bus or you're saying you have an
> > augmented SMP+NUMA+separate-parallel-message-passing-bus er .. thing]
>
> Its a standard current Intel feature. See "mwait" in the processor
> manual. The CPUs are also smart enough to do cache to cache transfers.
> No special hardware no magic.
It's unfortunately useless for anything but kernels right now because
Intel has disabled it in ring 3 (and AMD doesn't support it yet)
And the only good use the kernel found for it so far is fast wakeup
from the idle loop.
> And unless I want my messages to cause interrupts and wake events (in
> which case the APIC does it nicely) then any locked operation on memory
> will do the job just fine. I don't need funky hardware on a system. The
> first point I need funky hardware is between boards and that isn't
> consumer any more.
Firewire + CLFLUSH should do the job.
-Andi
Hi all,
I am equally interested to see how all this will affect embedded world.
While processors will have to go parallel, I agree, there will be more
then one processor (may be uniprocessor) embedded in devices like
cellphone, MP3 player etc. against one on our desk and in server
rooms. Linux has to work on this devices too.
--
Regards,
Tushar
--------------------
It's not a problem, it's an opportunity for improvement. Lets improve.
Luke Kenneth Casson Leighton writes:
> On Sun, Oct 02, 2005 at 08:27:45PM -0500, Chase Venters wrote:
>
> > The bottom line is that the application developers need to start being clever
> > with threads.
>
> yep! ah. but. see this:
>
> http://lists.samba.org/archive/samba-technical/2004-December/038300.html
>
> and think what would happen if glibc had hardware-support for
> semaphores and mutexes.
Let me guess... nothing? Overhead of locking depends on data-structures
used by application/library and their access patterns: one thread has to
wait for another to finish with the shared resource. Implementing
locking in hardware is going to change nothing here (barring really
stupid implementations of locking primitives). Especially as we are
talking about blocking primitives, like pthread semaphore or mutex: an
entry into the scheduler will by far outweigh any advantages of
raw-metal synchronization.
>
> > I think I remember some interesting rumors about Perl 6, for
> > example, including 'autothreading' support - the idea that your optimizer
> > could be smart enough to identify certain work that can go parallel.
Fortran people automatically parallelize loops for a _long_ time.
>
> http://www.ics.ele.tue.nl/~sander/publications.php
> http://portal.acm.org/citation.cfm?id=582068
> http://csdl.computer.org/comp/proceedings/acsd/2003/1887/00/18870237.pdf
>
> to get the above references, put in "holland parallel code
> analysis tools" into google.com.
PS: I wonder why Luke Kenneth Casson Leighton, Esq., while failing to
spell the Grandeur of his Appellative with the full Capitalization in
not a single From header humble readers of this Thread have a rare Honor
to witness, insists on referring to his interlocutors in minuscule only?
Does this correlate with an abnormally frequent usage of word
"condescending" in this discussion?
Nikita.
On Tue, Oct 04, 2005 at 07:04:35PM +0400, Nikita Danilov wrote:
dude.
chill.
out.
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
On Tue, Oct 04, 2005 at 07:04:35PM +0400, Nikita Danilov wrote:
> Luke Kenneth Casson Leighton writes:
> > On Sun, Oct 02, 2005 at 08:27:45PM -0500, Chase Venters wrote:
> >
> > > The bottom line is that the application developers need to start being clever
> > > with threads.
> >
> > yep! ah. but. see this:
> >
> > http://lists.samba.org/archive/samba-technical/2004-December/038300.html
> >
> > and think what would happen if glibc had hardware-support for
> > semaphores and mutexes.
>
> Let me guess... nothing?
interesting.
> Overhead of locking depends on data-structures
> used by application/library and their access patterns: one thread has to
> wait for another to finish with the shared resource.
yes.
> locking in hardware is going to change nothing here (barring really
> stupid implementations of locking primitives). Especially as we are
> talking about blocking primitives, like pthread semaphore or mutex: an
> entry into the scheduler will by far outweigh any advantages of
> raw-metal synchronization.
so what would, in your opinion, be a good optimisation?
the references i found (just below) are to tool chains or research
projects for code or linker-level analysis and parallelisation tools.
what would, in your opinion, be a good way for hardware to assist
thread optimisation, at this level (glibc)?
assuming that you have an intelligent programmer (or some really good
and working parallelisation tools) who really knows his threads?
> > http://www.ics.ele.tue.nl/~sander/publications.php
> > http://portal.acm.org/citation.cfm?id=582068
> > http://csdl.computer.org/comp/proceedings/acsd/2003/1887/00/18870237.pdf
> >
> > to get the above references, put in "holland parallel code
> > analysis tools" into google.com.
>
> PS: I wonder why Luke Kenneth Casson Leighton, Esq., while failing to
can i invite you to consider, when replying to these lists, to consider
instead of treating it as a location where you can piss over anyone
that you do not believe to be in any way your equal or in fact the equal
of anyone, to instead consider the following template for your replies:
okay, right.
* i do/don't get what this guy is saying.
* i do/don't have an alternative idea (here it is / sorry)
* here's what's wrong / right with what he's saying.
* here's where it can/can't be done better.
the bits that are missing from your reply are:
* "you do/don't get where i'm going with this"
* you haven't specified an alternative idea
* you've outlined what's wrong but not what's right
* you haven't specified how it can be done better.
i therefore conclude that you are bully. a snob.
i _really_ detest bullying - and that's what you are doing.
intellectual bullying.
stop it.
so i sent some messages saying "i think the kernel developers could be
wrong in their design strategy" so FRIGGIN what?
prove me right or prove me wrong.
or shut up, or add my email address to your killfile.
_don't_ be an intellectual snob.
l.
On Tuesday 04 October 2005 08:53, Luke Kenneth Casson Leighton wrote:
[...]
> the "off" voltage of a silicon germanium transistor is 0.8
volts.
Isn't that a bit contradictory, Luke?. Or is there a new process that
mixes the two basic materials for making semiconductors from?
--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.35% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
Lennart Sorensen wrote:
> On Tue, Oct 04, 2005 at 01:53:54PM +0100, Luke Kenneth Casson Leighton wrote:
>
>> at 45nm the current leakage is so insane that the heat
>> dissipation, through the oxide layer which covers the chip,
>> ends up blowing the chip up.
>
>
> Unless someone finds a way to reduce the leakage. It is worth a lot of
> money to some companies to solve that problem after all.
That's one way, the other is to find a way to cool such a chip. I see
references to diamond substrate from time to time, good thermal
conductor. So are other carbon forms, Fullerines, etc.
Clearly reducing leakage is the optimal solution, "deal with the heat"
is the other.
Luke Kenneth Casson Leighton writes:
[...]
>
> assuming that you have an intelligent programmer (or some really good
> and working parallelisation tools) who really knows his threads?
Well, I'd like to have a hardware with CAS-n operation for one
thing. But what would this buy us? Having different kernel algorithms
for x86 and mythical cas-n-able hardware is not viable.
[...]
>
> can i invite you to consider, when replying to these lists, to consider
> instead of treating it as a location where you can piss over anyone
> that you do not believe to be in any way your equal or in fact the equal
> of anyone,
As this self-contradictory (unless equivalence is not reflective in your
world) description is not the way I treat "these lists", I'll --instead
of pointing out to errors in your proposal-- follow (with improvements)
your advice, and re-interpret it the way I want:
> * i do/don't get what this guy is saying.
Indeed. But maybe this is at least partially because "this guy" fails to
abide to the elementary rules of grammar?
One who really wants to communicate with other people, follows common
conventions that exist specifically on purpose of making communication
possible. If you want to engage into a discussion with other people, you
should put aside considerations of your own convenience or habit, and
stick to the common protocol. It's that simple.
[...]
>
> so i sent some messages saying "i think the kernel developers could be
> wrong in their design strategy" so FRIGGIN what?
Pray, do tell me you typed this without a shift key. I'd very much do
like to not destroy the very essence of 20 years of accumulated
experience.
>
> or shut up, or add my email address to your killfile.
Please do this with my address in exchange: suddenly, I am
overwhelmed with a glimmering presentiment of sharing this location with
a lot of worthy people.
Nikita.
On Tue, Oct 04, 2005 at 09:15:57PM +0400, Nikita Danilov wrote:
> Luke Kenneth Casson Leighton writes:
>
> [...]
>
> >
> > assuming that you have an intelligent programmer (or some really good
> > and working parallelisation tools) who really knows his threads?
>
> Well, I'd like to have a hardware with CAS-n operation for one
> thing.
CAS - compare and swap - by CAS-n i presume that you mean effectively a
SIMD CAS instruction?
> But what would this buy us?
you do not say :) i am genuinely interested to hear what it would buy.
> Having different kernel algorithms
> for x86 and mythical cas-n-able hardware is not viable.
if i can get an NPTL .deb package for glibc for x86 only it would tend
to imply that that isn't a valid conclusion: am i missing something?
cheers,
l.
On Tue, 4 Oct 2005, Luke Kenneth Casson Leighton wrote:
> okay, right.
> * i do/don't get what this guy is saying.
> * i do/don't have an alternative idea (here it is / sorry)
> * here's what's wrong / right with what he's saying.
> * here's where it can/can't be done better.
It would help if you added something from the third and
fourth bullet points to your posts, instead of sticking
with just the first two.
--
All Rights Reversed
Luke Kenneth Casson Leighton writes:
> On Tue, Oct 04, 2005 at 09:15:57PM +0400, Nikita Danilov wrote:
> > Luke Kenneth Casson Leighton writes:
> >
> > [...]
> >
> > >
> > > assuming that you have an intelligent programmer (or some really good
> > > and working parallelisation tools) who really knows his threads?
> >
> > Well, I'd like to have a hardware with CAS-n operation for one
> > thing.
>
> CAS - compare and swap - by CAS-n i presume that you mean effectively a
> SIMD CAS instruction?
An instruction that atomically compares and swaps n independent memory
locations with n given values. cas-1 (traditional compare-and-swap) is
enough to implement lock-less queue, cas-2 is enough to implement
double-linked lists, and was used by Synthesis lock-free kernel
(http://citeseer.ist.psu.edu/massalin91lockfree.html).
To be precise, cas-1 is theoretically enough to implement double-linked
lists too, but resulting algorithms are not pretty at all.
>
> > But what would this buy us?
>
> you do not say :) i am genuinely interested to hear what it would buy.
Nothing. That was an instance of "rhetorical question", sorry that I
made not this clear enough.
>
> > Having different kernel algorithms
> > for x86 and mythical cas-n-able hardware is not viable.
>
> if i can get an NPTL .deb package for glibc for x86 only it would tend
> to imply that that isn't a valid conclusion: am i missing something?
Yes: this is Linux _Kernel_ mailing list, and I was talking about kernel
code and kernel algorithms.
>
> cheers,
>
> l.
>
Nikita.
>
I think it's time for some innovative thinking and for people to step
outside the Linux box and look around at other operating systems for
some good ideas. I'll run through a few ideas here.
Reiser 4 - The idea of building a file system on top of a database is
the right way to go. Reiser is onto something here and this is a
technology that needs to be built upon. It's current condition is a
little on the week side - no ACLs for example - but the underlying
concept is ound.
Novell Netware type permissions. ACLs are a step in the right direction
but Linux isn't any where near where Novell was back in 1990. Linux lets
you - for example - to delete files that you have no read or write
access rights to. Netware on the other hand prevents you from deleting
files that you can't write to and if you have no right it is as if the
file isn't there. You can't even see it in the directory. Netware also
has inherited permissions like Windows and Samba has and this is doing
it right. File systems and individual directories should be able to be
flagged as casesensitive/insensitive. Permissions need to be fine
grained and easy to use. Netware is a good example to emulate.
The bootup sequence of Linux is pathetic. What an ungodly mess. The
FSTAB file needs to go and a smarter system needs to be developed. I
know this isn't entirely a kernel issue but it is somewhat related.
I think development needs to be done to make the kernel cleaner and
smarter rather than just bigger and faster. It's time to look at what
users need and try to make Linux somewhat more windows like in being
able to smartly recover from problems. Perhaps better error messages
that your traditional kernel panic or hex dump screen of death.
The big challenge for Linux is to be able to put it in the hands of
people who don't want to dedicate their entire life to understanding all
the little quirks that we have become used to. The slogan should be
"this just works" and is intuitive.
Anyhow - before I piss off too many people who are religiously attached
to Linux worshiping - I'll quit not. ;)
Marc Perkel
Linux Visionary
On Tue, Oct 04, 2005 at 12:47:25PM -0700, Marc Perkel wrote:
> The bootup sequence of Linux is pathetic. What an ungodly mess. The
> FSTAB file needs to go and a smarter system needs to be developed. I
> know this isn't entirely a kernel issue but it is somewhat related.
depinit. written by richard lightman. easily located with google.
on relatively inexpensive amd 2100 hardware, depinit results
in a startup time to console login of 5 seconds, and x-windows
in a further 3.
this is probably as good a time as any to mention this:
depinit on a 2.6 kernel has had to have a small script added which does
a sleep 3; kill -HUP <itself> - i.e. "kill -HUP 1".
if this is not done, then any child program that sends a signal to
process 1 is NOT SEEN.
richard believes the problem to be actually in the 2.6 kernel.
whilst /sbin/init only catches one signal, depinit catches quite
literally _all_ of them.
i'm relaying this from memory, so some of the above may be inaccurate.
> I think development needs to be done to make the kernel cleaner and
> smarter rather than just bigger and faster.
actually, on embedded systems the linux 2.6 kernel is bigger
and slower, which has prompted a large number of embedded systems
designers to stick with the [by now abandoned] 2.4 series.
> Marc Perkel
> Linux Visionary
^^^^^ ^^^^^^^^^
wha-heeeeey !
my main concern, btw, is that by the time linux kernel developers
"receive hardware to play with", it's already too late.
the hardware decisions have already been made.
you - worthy as you are and the work you are doing is -
are treated as second class citizens by the companies
manufacturing hardware.
time to put the horse before the cart.
l.
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
Marc Perkel <[email protected]> wrote:
[...]
> I'll run through a few ideas here.
> Novell Netware type permissions. ACLs are a step in the right direction
> but Linux isn't any where near where Novell was back in 1990. Linux lets
> you - for example - to delete files that you have no read or write
> access rights to.
It lets you unlink them. That's different from deleting, since the owner
may have his/her private link to that file.
Unlinking is changing the contents of a directory, and it's controlled by
the write permission of the containing directory.
> Netware on the other hand prevents you from deleting
> files that you can't write to and if you have no right it is as if the
> file isn't there.
Imagine a /tmp directory (writable by world) with user "a" creating a file
"foo", umask=077 and user "b" trying to do the same. User "b" will get
'file exists' if he tries to create it, and 'file does not exist' if he
tries to list it. He will go nuts.
BTW: YANI: That about a tmpfs where all-numerical entries can only be
created by the corresponding UID? This would provide a secure, private
tmp directory to each user without the possibility of races and denial-of-
service attacks. Maybe it should be controlled by a mount flag.
> You can't even see it in the directory. Netware also
> has inherited permissions like Windows and Samba has and this is doing
> it right.
You can't do that if you have hardlinks. However, I missed the feature of
overruling file permissions in some special directories, e.g. anything
put under /pub should ignore umask and be a+rX.
> File systems and individual directories should be able to be
> flagged as casesensitive/insensitive.
IMHO not needed.
> Permissions need to be fine
> grained and easy to use. Netware is a good example to emulate.
ACK.
> The bootup sequence of Linux is pathetic. What an ungodly mess.
Which one? The bsd-style, sysv-suse-style, the sysv-debian-style,
the sysv-gentoo-style, the supervise-style, ...?
> The
> FSTAB file needs to go and a smarter system needs to be developed.
Smarter than recognizing the partitions by GUID?
> I
> know this isn't entirely a kernel issue but it is somewhat related.
>
> I think development needs to be done to make the kernel cleaner and
> smarter rather than just bigger and faster. It's time to look at what
> users need and try to make Linux somewhat more windows like in being
> able to smartly recover from problems.
Using "windows" and "smartly recover from problems" is strange.
> Perhaps better error messages
And it becomes even more strange. Decent error messsages from windows are
as common as snowballs in hell.
> that your traditional kernel panic or hex dump screen of death.
?Some error occured. Press "OK".?
And if there is a help button, it won't.
> The big challenge for Linux is to be able to put it in the hands of
> people who don't want to dedicate their entire life to understanding all
> the little quirks that we have become used to. The slogan should be
> "this just works" and is intuitive.
"Just working" isn't easy if you have zillions of dependencies and even
more possibilities to choose from. You can e.g. make linux use a raid0
of a network block device and a loop-mounted file accessed over a ssh
session as it's root device (just in case if you you got bored of drilling
holes into your knees, pooring milk into them and raising fish in them.)
--
Ich danke GMX daf?r, die Verwendung meiner Adressen mittels per SPF
verbreiteten L?gen zu sabotieren.
On Tuesday 04 October 2005 02:47 pm, Marc Perkel wrote:
> The bootup sequence of Linux is pathetic. What an ungodly mess. The
> FSTAB file needs to go and a smarter system needs to be developed. I
> know this isn't entirely a kernel issue but it is somewhat related.
>
> I think development needs to be done to make the kernel cleaner and
> smarter rather than just bigger and faster. It's time to look at what
> users need and try to make Linux somewhat more windows like in being
> able to smartly recover from problems. Perhaps better error messages
> that your traditional kernel panic or hex dump screen of death.
>
> The big challenge for Linux is to be able to put it in the hands of
> people who don't want to dedicate their entire life to understanding all
> the little quirks that we have become used to. The slogan should be
> "this just works" and is intuitive.
I agree with the basic sentiment here.
fstab is pretty traditional - and you're right, it really isn't a kernel thing
per se. Let's not forget the value of simplicity though. fstab works and does
its job well. I do think it would be interesting for a distribution to
experiment with different approaches, though -- something with the emerging
hardware abstraction layer perhaps.
As for error messages... the equivalent of the Linux kernel panic is basically
the Windows BSOD. Neither one of them should appear in the day to day use of
the system as they indicate bugs. Linux is actually the clear winner here, I
think, because a Windows BSOD gives you a single hex code and no indication
of what happened, except for very vague codes like
"PAGE_FAULT_IN_NON_PAGED_AREA". I'd much rather have a backtrace :) In any
case, I'm watching the work on kdump with a keen interest.
Really, I think the whole issue of usability isn't tied directly to the
kernel. The kernel has been making leaps and bounds in making this easy for
userspace to deal with (where the approaches to solve the problem belong).
Sysfs is an obvious great example.
Work on dbus and HAL should give us good improvements in these areas. One
remaining challenge I see is system configuration - each daemon tends to
adopt its own syntax for configuration, which means that providing a GUI for
novice users to manage these systems means attacking each problem separately
and in full. Now I certainly wouldn't advocate a Windows-style registry,
because I think it's full of obvious problems. Nevertheless, it would be nice
to have some kind of configuration editor abstraction library that had some
sort of syntax definition database to allow for some interesting work on
GUIs.
In any case, I think pretty much all of this work lives outside the kernel.
There is one side note I'd make about booting - my own boot process has to
wait forever for my Adaptec SCSI controller to wake up. It would be
interesting if bootup initialization tasks could be organized into dependency
levels and run in parallel, though as I'm a beginner to the workings of the
kernel I'm not entirely sure how possible this would be.
Cheers,
Chase Venters
On Tuesday 04 October 2005 19:47, Marc Perkel wrote:
> I think it's time for some innovative thinking and for people to
> step outside the Linux box and look around at other operating
> systems for some good ideas. I'll run through a few ideas here.
>
> Reiser 4 - The idea of building a file system on top of a database
> is the right way to go. Reiser is onto something here and this is a
> technology that needs to be built upon. It's current condition is a
> little on the week side - no ACLs for example - but the underlying
> concept is ound.
A filesystem built on top of a database? Isn't that introducing
complexity into something that should be as simple as possible so
that the number of possible errors is reduced?
But then, I do not have experience with filesystem design and
implementation, so I cannot make a suggestion on this front.
> Novell Netware type permissions. ACLs are a step in the right
> direction but Linux isn't any where near where Novell was back in
> 1990. Linux lets you - for example - to delete files that you have
> no read or write access rights to.
As someone else pointed out, this is because unlinking is related to
your access permissions on the parent directory and not the file.
> Netware on the other hand
> prevents you from deleting files that you can't write to and if you
> have no right it is as if the file isn't there. You can't even see
> it in the directory.
This is just adding a layer of security through hiding data. I
personally don't like this idea for a number of reasons, not the
least of which is that it is the access permissions of the directory
that control whether or not you can see a file. This and the previous
comment about unlinking of files is, IIRC, actually part of the POSIX
standard.
> Netware also has inherited permissions like
> Windows and Samba has and this is doing it right.
This would only be a bonus with ACL's. It makes no real sense to
propogate a directories permissions down to the files in a directory
since an directory you can list the contents of has at least 1
execute bit set.
> File systems and
> individual directories should be able to be flagged as
> casesensitive/insensitive.
This would be a rarely used feature and would break many tools. Having
an extra bit would also require modifying the kernel and I doubt
anyone wants to tackle such a job, as it would also break all extant
filesystems.
> Permissions need to be fine grained and
> easy to use. Netware is a good example to emulate.
I do agree with this, but have to point out that this is already
allowed for under POSIX and a number of filesystems support this.
It's called "POSIX ACL's" in the kernel configuration system. Since I
don't use them on my home system (I see no need since it's just me
and whatever hacker has managed to penetrate the system (to date: 0))
I do use ACL's (POSIX and otherwise) on all the systems I maintain
for my various clients (providing the OS supports them)
> The bootup sequence of Linux is pathetic. What an ungodly mess. The
> FSTAB file needs to go and a smarter system needs to be developed.
> I know this isn't entirely a kernel issue but it is somewhat
> related.
I'll have to disagree about FSTAB - this is something that is at the
peak of it's usefulness and changing or removing it would require the
people that maintain the core utilities to rewrite mount(8) almost
entirely.
However, when it comes to the boot sequence as controlled by init(8) I
have to agree. I'm personally working on an entirely new set of
init-scripts for my system and have thought about seeing if anyone
has ever released an init(8) that is more functional than the basic
GNU/FSF version. If there was an init(8) that could run the scripts
in parallel I'd be using it as soon as I had a set of scripts lined
up that were designed to be run in parallel.
> I think development needs to be done to make the kernel cleaner and
> smarter rather than just bigger and faster. It's time to look at
> what users need and try to make Linux somewhat more windows like in
> being able to smartly recover from problems. Perhaps better error
> messages that your traditional kernel panic or hex dump screen of
> death.
lol. Nope. The kernel panic could be refined to contain even more
information and be even more user-friendly but it is definately
light-years ahead of a Windows BSOD. Now if you're talking about the
errors as seen by users of applications that's not a kernel issue and
is the purview of the application developers.
> The big challenge for Linux is to be able to put it in the hands of
> people who don't want to dedicate their entire life to
> understanding all the little quirks that we have become used to.
> The slogan should be "this just works" and is intuitive.
Yep. I do agree with that. However, until the rest of the big
companies catch up to the ones that already support Linux this will
never happen. Several of my non-business maintenance clients have
inquired abotu Linux and I've had to tell them to just stick with
Windows because they rely on a rather large number of Windows only
programs (that do _not_ run under Wine) and/or are not technically
enough inclined to be able to handle the learning curve involved in
moving to a non-MS operating system.
The fact that I have gotten those inquiries means that the news about
how stable Linux is is getting to the "mainstream" population. Only
problem is that MS and the home-PC boom has landed the PC and the
internet in the hands of too many people who are barely computer
literate enough to use a mouse. (I'm speaking from experience. A
large number of my private clients fit this description to a T.
Although they are good as a continuing source of income :) And with
Windows being as "User Friendly" as it is, and with at least 75% of
the major software firms still not supporting Linux, there is no way
Linux can make any real inroads into the desktop market.
OTOH several of my clients have inquired about Linux because of it's
security - it doesn't take a genius to see that the same reason
Firefox made such a big dent in MS's hold on the browser market could
work for Linux as well. And I have had more than one client ask if
there was an alternative to Windows than ran well - I've always
answered the same way: "Yes, but you would need to learn an all-new
way of doing things." Every last one of them dropped it on hearing
those words.
> Anyhow - before I piss off too many people who are religiously
> attached to Linux worshiping - I'll quit not. ;)
heh. keep it up. you've managed to turn a pointless argument of micro
versus mono into something productive. (even if you didn't mean to)
DRH
On Tue, 04 Oct 2005 18:40:33 CDT, Chase Venters said:
> Work on dbus and HAL should give us good improvements in these areas. One
> remaining challenge I see is system configuration - each daemon tends to
> adopt its own syntax for configuration, which means that providing a GUI for
> novice users to manage these systems means attacking each problem separately
> and in full. Now I certainly wouldn't advocate a Windows-style registry,
> because I think it's full of obvious problems. Nevertheless, it would be nice
> to have some kind of configuration editor abstraction library that had some
> sort of syntax definition database to allow for some interesting work on
> GUIs.
Anybody who tries to do this without at least understanding the design choices
made by AIX's SMIT tool deserves to re-invent it, poorly.
> In any case, I think pretty much all of this work lives outside the kernel.
Amen to that - although the whole hotplug/udev/sysfs aggregation has at least
made a semi-sane way to find out from userspace what the kernel thinks is going on...
Are there any drivers out there that don't play nice with sysfs? If so, should
a mention of them be added to http://kerneljanitors.org/TODO ?
D. Hazelton wrote:
>
>
>>Novell Netware type permissions. ACLs are a step in the right
>>direction but Linux isn't any where near where Novell was back in
>>1990. Linux lets you - for example - to delete files that you have
>>no read or write access rights to.
>>
>>
>
>As someone else pointed out, this is because unlinking is related to
>your access permissions on the parent directory and not the file.
>
>
>
Right - that's Unix "inside the box" thinking. The idea is to make the
operating system smarter so that the user doesn't have to deal with
what's computer friendly - but reather what makes sense to the user.
From a user's perspective if you have not rights to access a file then
why should you be allowed to delete it?
Now - the idea is to create choice. If you need to emulate Unix nehavior
for compatibility that's fine. But I would migrate away from that into a
permissions paradygme that worked like Netware.
I started with Netware and I'm spoiled. They had it right 15 years ago
and Linux isn't any where near what I was with Netware and DOS in 1990.
Once you've had this kind of permission power Linux is a real big step down.
So - the thread is about the future so I say - time to fix Unix.
On Tue, 04 Oct 2005 22:49:03 PDT, Marc Perkel said:
> From a user's perspective if you have not rights to access a file then
> why should you be allowed to delete it?
Because it's your directory, dammit, and nobody else should be allowed to
clutter it with files you can't even read. :)
What's so hard to understand about that viewpoint? Want to try to explain
the converse to a Windows/Netware user? "But it's *MY* folder, why can't I
get rid of this file I can't read and have no use for?"
Trying to make it "make sense to the user" without expecting the user to learn
at least a bit about what's going on is futile, as Alan Perlis understood:
When someone says "I want a programming language in which I need only
say what I wish done," give him a lollipop.
On Tue, 4 Oct 2005, Chase Venters wrote:
> As for error messages... the equivalent of the Linux kernel panic is basically
> the Windows BSOD. Neither one of them should appear in the day to day use of
> the system as they indicate bugs. Linux is actually the clear winner here, I
> think, because a Windows BSOD gives you a single hex code and no indication
> of what happened, except for very vague codes like
> "PAGE_FAULT_IN_NON_PAGED_AREA". I'd much rather have a backtrace :) In any
> case, I'm watching the work on kdump with a keen interest.
>
And what about kexec? To be able to boot into another kernel on a kernel
bug and still have access to all the memory and the system state of the
bug. That's pretty cool. It would be like Windows going straight to
Safe-Mode on a BSOFD without a reboot.
> In any case, I think pretty much all of this work lives outside the kernel.
> There is one side note I'd make about booting - my own boot process has to
> wait forever for my Adaptec SCSI controller to wake up. It would be
> interesting if bootup initialization tasks could be organized into dependency
> levels and run in parallel, though as I'm a beginner to the workings of the
> kernel I'm not entirely sure how possible this would be.
>
I've been thinking of at least trying to see what would happen if I
threaded the do_initcalls in main.c but lately I haven't had the time.
-- Steve
Marc Perkel writes:
[...]
> Right - that's Unix "inside the box" thinking. The idea is to make the
> operating system smarter so that the user doesn't have to deal with
> what's computer friendly - but reather what makes sense to the user.
> From a user's perspective if you have not rights to access a file then
> why should you be allowed to delete it?
Because in Unix a name is not an attribute of a file.
Files are objects that you read, write and truncate. They are
represented by inodes.
Separately from that, there is an indexing structure: directory
tree. Directories map symbolical names to inodes. Obviously, adding a
reference to an index, or removing it from one requires access
permission to the _index_ rather then to the object being referenced.
That two-level model of files and indexing on top of them is essential
to Unix due to the flexibility and conceptual economy it provides.
>
> Now - the idea is to create choice. If you need to emulate Unix nehavior
> for compatibility that's fine. But I would migrate away from that into a
> permissions paradygme that worked like Netware.
And there are people believing that ITS (or VMS, or <insert your first
passion here>...) set the standard to follow. :-)
[...]
>
> So - the thread is about the future so I say - time to fix Unix.
One thing is clear: it's too late to fix Netware. Why should Unix
emulate its lethal defects?
Nikita.
On Wed, Oct 05, 2005 at 01:24:12PM +0400, Nikita Danilov wrote:
> Marc Perkel writes:
>
> [...]
>
> > Right - that's Unix "inside the box" thinking. The idea is to make the
> > operating system smarter so that the user doesn't have to deal with
> > what's computer friendly - but reather what makes sense to the user.
> > From a user's perspective if you have not rights to access a file then
> > why should you be allowed to delete it?
>
> Because in Unix a name is not an attribute of a file.
there is no excuse.
selinux has already provided an alternative that is similar to NW
file permissions.
so there is no excuse.
think about this.
in what way is it possible for linux to fully support the NTFS
filesystem?
bearing in mind that you will need to communicate a little bit more
than, but the direct equivalent of unix uid gid and secondary groups,
from userspace into kernelspace - just like reading /etc/passwd but
instead reading from a userspace daemon.
l.
> > In any case, I think pretty much all of this work lives outside the kernel.
> > There is one side note I'd make about booting - my own boot process has to
> > wait forever for my Adaptec SCSI controller to wake up. It would be
> > interesting if bootup initialization tasks could be organized into dependency
> > levels and run in parallel, though as I'm a beginner to the workings of the
> > kernel I'm not entirely sure how possible this would be.
again - depinit.
the only thing that richard lightman didn't spend time on was the
splitting out of hotplug / hotplug scripts such that depinit could be
told "okay the ethernet's up now, all eth0 dependencies can now proceed"
or "okay the scsi controller is up now, all fstab entries depending
on this controller can now proceed".
richard's example scripts split out "var", "usr", "home", "boot" as
separate dependencies (actually they're symlinks to the "mount"
pseudo-dependency) on which things like pretty much all services
that need to do syslogging depend on the "var" dependency you get the
idea.
so as long as it's hotplug that gets the notifications, great.
if it's _not_ hotplug that's receiving the notifications, such that
device driver initialisation cannot be delayed because you're looking at
calling some boot-rom initialisation stuff, then, sorry, nope - you're
gonna have to just wait for that SCSI controller to get its act
together :)
but again, this is off-topic for the original discussion and is again
userspace not kernel oh well, in for a penny in for a pound.
l.
On Wed, Oct 05, 2005 at 01:35:57AM -0400, [email protected] wrote:
> On Tue, 04 Oct 2005 18:40:33 CDT, Chase Venters said:
> > Work on dbus and HAL should give us good improvements in these areas. One
HAL is great. dbus should have been shot at birth and the people who
initiated it without looking at freedce should have been fired.
l.
On Wed, Oct 05, 2005 at 01:22:33AM +0000, D. Hazelton wrote:
> On Tuesday 04 October 2005 19:47, Marc Perkel wrote:
> As someone else pointed out, this is because unlinking is related to
> your access permissions on the parent directory and not the file.
that's POSIX.
i trust that POSIX has not been hard-coded into the entire design of
the linux kernel filesystem architecture _just_ because it's ... POSIX.
l.
On Wed, 05 Oct 2005 11:09:42 BST, Luke Kenneth Casson Leighton said:
> On Wed, Oct 05, 2005 at 01:22:33AM +0000, D. Hazelton wrote:
> > On Tuesday 04 October 2005 19:47, Marc Perkel wrote:
>
> > As someone else pointed out, this is because unlinking is related to
> > your access permissions on the parent directory and not the file.
>
> that's POSIX.
>
> i trust that POSIX has not been hard-coded into the entire design of
> the linux kernel filesystem architecture _just_ because it's ... POSIX.
No, what got hard-coded were the concepts of inodes as the actual description
of filesystem objects, directories as lists of name-inode pairs, and the whole
user/group/other permission thing. "unlink depends on the directory
permissions not the object unlinked" has been the semantic that people depended
on ever since some code at Bell Labs started supporting tree-structured
directories and multiple hardlinks. POSIX merely codified existing behavior in
this case.
On Tue, Oct 04, 2005 at 06:40:33PM -0500, Chase Venters wrote:
> Work on dbus and HAL should give us good improvements in these areas. One
dbus. total waste of several man-years of effort that could have been
better spent in e.g. removing the dependency of posix draft 4 threads
from freedce (which i finally did last month) and e.g. adding an ultra-fast
shared-memory transport plugin.
hal. _excellent_. look forward to it being ported to win32 so that
it's useful for the kde-win32 port etc. etc.
> remaining challenge I see is system configuration - each daemon tends to
> adopt its own syntax for configuration, which means that providing a GUI for
> novice users to manage these systems means attacking each problem separately
> and in full.
that's quite straightforward to deal with but it _does_ mean using a
unified approach to writing APIs.
NT solved this problem by writing graphical tools in c and then
adopting dce/rpc as the means to administer the services both locally
_and_ remotely.
wholesale. utterly. everything. right from the simplest things like
rebooting the machine, through checking the MAC addresses of cards
(NetTransportEnum) all the way up to DNS administration - yes,
dnsadmin.exe will write out DNS zone files in proper bind format.
it's quite a brave choice to take.
> Now I certainly wouldn't advocate a Windows-style registry,
> because I think it's full of obvious problems.
such as? :)
they're not obvious to me. at the risk of in-for-penny, in-for-pound
_radically_ off-topic discussion encouragement here, and also for
completeness should someone come back to the archives in some years or
months and go "what obvious problems", could you kindly elaborate?
one i can think of is "eek, my system's broken, eek, i can't even use
vi to edit the configs".
and having described the problem, then.. .. well... actually...
it's simply dealt with:
http://www.bindview.com/Services/RAZOR/Utilities/Unix_Linux/ntreg_readme.cfm
todd sabin wrote a linux filesystem driver which is read-only, so at
least half the work's done.
(and the reactos people have written a complete implementation
of a registry, btw).
> Nevertheless, it would be nice
> to have some kind of configuration editor abstraction library that had some
> sort of syntax definition database to allow for some interesting work on
> GUIs.
i have to say this: it's almost too radical, dude :)
he he.
> In any case, I think pretty much all of this work lives outside the kernel.
ACK!
... well... not entirely.
a "registry" - god help us - would need to be stored on a filesystem.
and then, ideally, made accessible a la todd sabin's ntreg driver - via
a POSIX interface (because the linux kernel doesn't _do_ anything other
than POSIX filesystems *sigh*). and that makes it also convenient to
access from kernelspace, too.
hey, you know what? if linux got a registry, it would be possible for
the kernel to access - and store, and communicate - persistent
information.
conveniently.
hurrah.
Luke Kenneth Casson Leighton writes:
> On Wed, Oct 05, 2005 at 01:24:12PM +0400, Nikita Danilov wrote:
>
> > Marc Perkel writes:
> >
> > [...]
> >
> > > Right - that's Unix "inside the box" thinking. The idea is to make the
> > > operating system smarter so that the user doesn't have to deal with
> > > what's computer friendly - but reather what makes sense to the user.
> > > From a user's perspective if you have not rights to access a file then
> > > why should you be allowed to delete it?
> >
> > Because in Unix a name is not an attribute of a file.
>
> there is no excuse.
>
> selinux has already provided an alternative that is similar to NW
> file permissions.
That's exactly the point: Unix file system model is more flexible than
alternatives. So that one can emulate foreign semantics on top of
it. But not other way around.
Nikita.
On Wed, Oct 05, 2005 at 12:04:01AM +0200, Bodo Eggert wrote:
> > You can't even see it in the directory. Netware also
> > has inherited permissions like Windows and Samba has and this is doing
> > it right.
>
> You can't do that if you have hardlinks.
nt 5.0 added hardlinks to ntfs.
El Wed, 5 Oct 2005 11:26:50 +0100,
Luke Kenneth Casson Leighton <[email protected]> escribi?:
> > Now I certainly wouldn't advocate a Windows-style registry,
> > because I think it's full of obvious problems.
>
> such as? :)
The ugly implementation (inside the kernel and as a big file instead of doing it as a
userspace in top of NTFS files which would have helped them to avoid lots of problems
that they were forced to solve in XP/2003, it's clear from their docs that they didn't
expected that registry could grow _so_ much), the fact that they use it to store at
the same time userspace configuration and internal kernel structures. The "idea" is
nice but the way they've implemented and used it is horrible - just take a look
at how they're using XML configuration files for IIS now... (I've been said that ISS will
detect when you're editing those configuration files and will reload them to duplicate
the changes you just made in the registry ..... ugh)
> hey, you know what? if linux got a registry, it would be possible for
> the kernel to access - and store, and communicate - persistent
> information
right - why you would want to do such thing is another story
On Wed, Oct 05, 2005 at 06:23:09AM -0400, [email protected] wrote:
> > i trust that POSIX has not been hard-coded into the entire design of
> > the linux kernel filesystem architecture _just_ because it's ... POSIX.
>
> No, what got hard-coded were the concepts of inodes as the actual description
> of filesystem objects, directories as lists of name-inode pairs, and the whole
> user/group/other permission thing. "unlink depends on the directory
> permissions not the object unlinked" has been the semantic that people depended
> on
fortunately, selinux has begun the path away from that kind of implicit
ruling.
l.
On Wed, Oct 05, 2005 at 02:30:44PM +0400, Nikita Danilov wrote:
> Luke Kenneth Casson Leighton writes:
> > On Wed, Oct 05, 2005 at 01:24:12PM +0400, Nikita Danilov wrote:
> >
> > > Marc Perkel writes:
> > >
> > > [...]
> > >
> > > > Right - that's Unix "inside the box" thinking. The idea is to make the
> > > > operating system smarter so that the user doesn't have to deal with
> > > > what's computer friendly - but reather what makes sense to the user.
> > > > From a user's perspective if you have not rights to access a file then
> > > > why should you be allowed to delete it?
> > >
> > > Because in Unix a name is not an attribute of a file.
> >
> > there is no excuse.
> >
> > selinux has already provided an alternative that is similar to NW
> > file permissions.
>
> That's exactly the point: Unix file system model is more flexible than
> alternatives.
*grin*. sorry - i have to disagree with you (but see below).
i was called in to help a friend of mine at EDS to do a bastion sftp
server to write some selinux policy files because POSIX filepermissions
could not fulfil the requirements.
the requirements were two-fold:
1) for an authorised uploader to be able to copy files into
a subdirectory but to be banned from seeing other files and
banned from overwriting or deleting anything - existing or
uploaded. once the file's there, it's there.
2) for an authorised downloader to be able to copy files _out_
of a subdirectory and to be able to delete those files but
to be banned from overwriting files, or modifying files,
or adding files to the subdirectory.
it is not possible to fulfil those requirements with POSIX - not even
POSIX ACLs not even messing about creating groups with different rwx
permissions.
however, selinux was, because you have separate permissions for
directories to create and remove names, read and write to the
directory, and separate permissions _again_ for files.
basically, i think (without checking) that there is one
selinux permission per each inode operation and per each
dnode operation totalling somewhere around thirty separate
selinux filesystem permissions, which is _much_ better than
the pathetic POSIX rwx and implicit directory rules stuff.
POSIX permissions were designed to fit into what... 16 bits,
so they didn't have a lot to play with.
if i was to agree with you, it would be that the linux filesystem
API is more flexible than the alternatives, most definitely not the
Unix filesystem model (if you believe that to be POSIX).
but the linux filesystem API i do not believe to be _as_ flexible as
say the NTFS filesystem API (or the Netware one, although i don't know
much about that one).
l.
On Wed, Oct 05, 2005 at 01:24:12PM +0400, Nikita Danilov wrote:
> One thing is clear: it's too late to fix Netware. Why should Unix
> emulate its lethal defects?
the "lethal defects" of netware i believe could be more fairly
attributed to it being proprietary software and the consequences
thereof.
that doesn't mean that you can't learn from it - the good bits,
the bad bits and the ugly bits.
l.
On Tue, Oct 04, 2005 at 05:37:59PM -0400, Chris wrote:
> You write:
> | personally i find that i like a bit of a run-up and/or advance notice
> | of major paradigm shifts.
>
> I maintain that one of the problems is that we simply do not know how
> to efficiently (in all senses of the word) program multi-processing
> systems and applications. Various people have various theories about how
> it should be done, but they are not well-proven.
>
> We do know that the current approach is difficult and error-prone, but
> terribly attractive because of an at least superficial simplicity. So
> far none of the alternatives have been convincing enough to supplant it.
>
> Efficient changing of paradigms requires a target paradigm. In this
> environment, a good target paradigm does not appear to exist yet and
> may not be clear for years. Starting engineering work seems a little
> bit premature in that situation.
chris,
your words paraphrase nicely the issues faced - for software engineers.
however, if you were to speak to a hardware engineer - an embedded
systems designer of ASICs - they would have a completely different
take on it, because they are dealing with parallelism all the time,
_and_ they even have some tools to help them do that.
what those ASIC designers lack is an off-the-shelf affordable
multiprocessor chip that makes their parallel algorithms run
fast enough: they _have_ to go for custom ASIC.
which is where the pricing of 90nm - $250k mask charges making up most
of the $2m NREs - is so key. even .13micron you're looking at $200k
NREs [non-recoverable expenditure].
and if you were to do a chip at .13micron you would be
looking at 120 watts for something running at 800mhz with
only 1 million gates.
a pentium 3 800mhz in other words, and that won't exactly get you very
far.
what the software engineers you refer to above lack is the toolchain to
assist them in developing for anything other than uniprocessor targets.
such tools and techniques are being researched and developed (i sent
references to a coupl and also a google search criteria in another
earlier LKML email): however, yet again, it's chicken-and-egg.
until the chips start going multiprocessor, the tools are going to
remain in research labs. until the tools come out of research labs,
the chips are going to remain useless.
where it all goes a bit pearshaped with that chicken-and-egg vicious
cycle is if the bottom drops out of 65nm and 45nm processes, such that
_even_ the top uniprocessor mass-market chip manufacturers are forced
down a parallel processing line.
my point is: we're starting to see evidence of that happening
(small-scale, 2-cores, 2-hyperthreads, talk of 4-cores, etc.
even the X-Box 360 PPC 3x2)
_therefore_, i invite people who do linux kernel development
to think ahead - to take a _lead_ for once instead of waiting
for hardware to drop into their laps, at which point it is once
again too late, the hardware design decisions will have
already been made by someone else, and you will be treated
like second class citizens. again.
l.
Luke Kenneth Casson Leighton writes:
[...]
> > That's exactly the point: Unix file system model is more flexible than
> > alternatives.
>
> *grin*. sorry - i have to disagree with you (but see below).
>
> i was called in to help a friend of mine at EDS to do a bastion sftp
> server to write some selinux policy files because POSIX filepermissions
> could not fulfil the requirements.
First, I was talking about flexibility attained through the separation
of notions of file and index. You just claimed elsewhere that this is
the direction ntfs took (with the introduction of hard-links).
Then, every security model has its weakness and corner cases. Try to
express
rw-r-xrw- (0656)
POSIX bits with canonical NT ACLs (hint: in NT allow-ACEs are
accumulated).
[...]
>
> POSIX permissions were designed to fit into what... 16 bits,
> so they didn't have a lot to play with.
That very good property for a security model: simplicity is a virtue
here.
Nikita.
On Wed, Oct 05 2005, Luke Kenneth Casson Leighton wrote:
> _therefore_, i invite people who do linux kernel development
> to think ahead - to take a _lead_ for once instead of waiting
> for hardware to drop into their laps, at which point it is once
> again too late, the hardware design decisions will have
> already been made by someone else, and you will be treated
> like second class citizens. again.
You seem to be suffering the same fate as other 'pocket visionaries'
with grand visions on this mailing list. The combination of 'listen to
this awesome idea' (*) and someone who hasn't contributed anything
interesting is not very good. Add a pinch (or more) of 'I know better'
and you've got a great cocktail for a lengthy and tiresome thread, but
not for anything productive.
Why is that so hard to understand? Succesful contributions start at the
technical level, always have.
(*) Such an idea is often described in great text detail, but is often
greatly lacking in technical detail.
--
Jens Axboe
On Wed, Oct 05, 2005 at 04:17:35PM +0400, Nikita Danilov wrote:
> Luke Kenneth Casson Leighton writes:
>
> [...]
>
> > > That's exactly the point: Unix file system model is more flexible than
> > > alternatives.
> >
> > *grin*. sorry - i have to disagree with you (but see below).
> >
> > i was called in to help a friend of mine at EDS to do a bastion sftp
> > server to write some selinux policy files because POSIX filepermissions
> > could not fulfil the requirements.
>
> First, I was talking about flexibility attained through the separation
> of notions of file and index.
oh, right.
> You just claimed elsewhere that this is
> the direction ntfs took
with a leap of a few steps, possibly: certainly directly i don't
remember doing so.
> (with the introduction of hard-links).
> Then, every security model has its weakness and corner cases. Try to
> express
>
> rw-r-xrw- (0656)
>
> POSIX bits with canonical NT ACLs (hint: in NT allow-ACEs are
> accumulated).
they used not to be. accumulative inherited ACLs were introduced
in NT 5.0.
and is accumulated ACLs such a bad thing? it's certainly more
space-efficient and administrative-efficient.
l.
Nikita Danilov wrote:
>Marc Perkel writes:
>
>[...]
>
> > Right - that's Unix "inside the box" thinking. The idea is to make the
> > operating system smarter so that the user doesn't have to deal with
> > what's computer friendly - but reather what makes sense to the user.
> > From a user's perspective if you have not rights to access a file then
> > why should you be allowed to delete it?
>
>Because in Unix a name is not an attribute of a file.
>
>Files are objects that you read, write and truncate. They are
>represented by inodes.
>
>Separately from that, there is an indexing structure: directory
>tree. Directories map symbolical names to inodes. Obviously, adding a
>reference to an index, or removing it from one requires access
>permission to the _index_ rather then to the object being referenced.
>
>That two-level model of files and indexing on top of them is essential
>to Unix due to the flexibility and conceptual economy it provides.
>
>
Now of you think "outside" the Linux box" you can see where people in
the real world would expect that if you have no rights to a file to read
or write to it that you shouldn't be able to delete it. In the outside
world it's "duh - of course"! but for thouse that are in the "Unix Cult"
you can't think past inodes.
> >
> > Now - the idea is to create choice. If you need to emulate Unix nehavior
> > for compatibility that's fine. But I would migrate away from that into a
> > permissions paradygme that worked like Netware.
>
>And there are people believing that ITS (or VMS, or <insert your first
>passion here>...) set the standard to follow. :-)
>
>[...]
>
> >
> > So - the thread is about the future so I say - time to fix Unix.
>
>One thing is clear: it's too late to fix Netware. Why should Unix
>emulate its lethal defects?
>
>Nikita.
>
>
Once you'be had Netware permissions - even 1990 Netware permission - you
are spoiled and everything else isn't even close.
--
Marc Perkel - [email protected]
Spam Filter: http://www.junkemailfilter.com
My Blog: http://marc.perkel.com
On Wed, Oct 05, 2005 at 02:31:14PM +0200, Jens Axboe wrote:
> [i know better criticism and accusations deleted and not commented on]
> Why is that so hard to understand? Succesful contributions start at the
> technical level, always have.
then we will have to agree to disagree, because i believe that
successful contributions start with "what creative thing shall
we do now / what problem shall we tackle today in a creative
way?" and work their way down to the technical level, which,
as you rightly point out, requires successful _technical_
contributions.
l.
On Wed, Oct 05 2005, Luke Kenneth Casson Leighton wrote:
> On Wed, Oct 05, 2005 at 02:31:14PM +0200, Jens Axboe wrote:
>
> > [i know better criticism and accusations deleted and not commented on]
>
> > Why is that so hard to understand? Succesful contributions start at the
> > technical level, always have.
>
> then we will have to agree to disagree, because i believe that
> successful contributions start with "what creative thing shall
> we do now / what problem shall we tackle today in a creative
> way?" and work their way down to the technical level, which,
> as you rightly point out, requires successful _technical_
> contributions.
It's not my opinion, I'm stating historical fact. I honestly can't
remember when a succesfull contribution was made based on a long
non-technical thread. I can, however, remember oodles of cases where the
reverse is quite true.
--
Jens Axboe
Marc Perkel writes:
>
[...]
> Now of you think "outside" the Linux box" you can see where people in
> the real world would expect that if you have no rights to a file to read
> or write to it that you shouldn't be able to delete it. In the outside
> world it's "duh - of course"! but for thouse that are in the "Unix Cult"
> you can't think past inodes.
Deleting files without read/write access to them is exactly what happens
in the real world of classified information: people who physically burn
paper folders have no right to open them. :-)
Please understand one simple thing: unlink(2) system call does not
_remove_ file. It just removes one of possibly many references to this
file from an index. To erase (a part of) a file body, one uses
truncate(2) system call that --wonders!-- requires write access to the
file.
[...]
>
> Once you'be had Netware permissions - even 1990 Netware permission - you
> are spoiled and everything else isn't even close.
>
Repeating "sugar" doesn't make it sweet.
Nikita.
On 4 Oct 2005, Marc Perkel announced authoritatively:
> Reiser 4 - The idea of building a file system on top of a database is
> the right way to go. Reiser is onto something here and this is a
> technology that needs to be built upon. It's current condition is a
> little on the week side - no ACLs for example - but the underlying
> concept is ound.
Well, it's not a `technology' (whatever that means). It's an idea, and
an implementation: a unified namespace built atop a database. Parts of
it make sense (the unified namespace: look at /sys for a bunch of fairly
recent unification); much of it makes sense but is hard to implement
(e.g. plugins); much makes conceptual sense but breaks POSIX in horrible
ways (files-as-directories, possibly meta-directories with arbitrary
content, at least if you're allowed to do all POSIX-allowed things to
that content: what happens to stat() now if some bugger removes a file's
creation time or size, or moves it out to some parent directory? Does
that even make sense? Whether or not it's allowed, you've broken a good
few existing programs.)
> Novell Netware type permissions.
... are an administrative *nightmare*. One of the major advantages of
the perhaps-overly-simplistic Unix permissions model is that a simple
`ls -l' can show you the complete permissions of a lot of files in a few
screensful, in an easily-comprehensible manner.
Ever tried doing that with a directory with most files having ACLs?
Until we have better userspace tools, across-the-board ACLs are a
non-starter. Now admittedly the iprovements are minor (e.g. highlighting
things with ACLs differing from the norm), but the fact remains that
they are *not here yet*.
> ACLs are a step in the right
> direction but Linux isn't any where near where Novell was back in
> 1990. Linux lets you - for example - to delete files that you have no
> read or write access rights to. Netware on the other hand prevents you
> from deleting files that you can't write to and if you have no right
> it is as if the file isn't there. You can't even see it in the
> directory.
Oh, joy. So now I can't remove a writable directory if someone else has
created a file I can't write to in it, even if that directory is owned
by me. That sounds really unpleasant to me.
What next? Forbidding unlink() of running executables?
> Netware also has inherited permissions like Windows and
> Samba has and this is doing it right.
s/right/wrong/
Look at a NetWare permission on some file and you can't tell what the
effective permission is, because it depends on inherited ones as well.
Look at an effective permission and you can't tell which parts of it
will change if you change the inherited permission. Change an inherited
permission and some of the inheritees' permissions might suddenly make
no sense. The UI errs on the side of showing you effective permissions,
which makes most sense, but is still far from ideal.
Unix has inherited permissions in one sense: you can't get at a file
via some path if one of the directories in that path is unreadable.
Even *that* causes regular problems, such that sendmail (for instance)
needs special code to check for this case.
And you want to make this *more* complex?
Remember: complexity is the enemy of security.
> File systems and individual
> directories should be able to be flagged as casesensitive/insensitive.
Only if you're willing to change POSIX to include a call to check filenames
for identity, and rewrite every POSIX app to use such a call where filenames
are compared, *and* discard one or more of the following (I think you can't
avoid losing at least two, actually):
- user-settable locales via LC_* and LANG
- the property that directories may contain only one file of a given name
- the property that a rename from name A to name B and then back to name A
again yields a file with the same name it started with
- the property that open ("foo", ... | O_CREAT) yields a file named `foo'
for all `foo'
I doubt that losing any of these properties would be considered
acceptable, and as for rewriting every app on earth that does filename
comparison, no chance.
It would also require case-conversion and locale-handling code, probably
including UTF-8 canoncalization code, inside the kernel. This would
greatly increase kernel complexity for a very small reward, and lead to
Al Viro's early death from cerebral aneurysm combined with involuntary
projectile vomiting. This cannot be considered a good thing.
> Permissions need to be fine grained and
> easy to use.
This is an impossible combination, especially if you add `secure' to
that list. Fine-grained capabilities on executable files, now *that*
is a nice thing to mostly-replace the odious setuid bit with.
> The bootup sequence of Linux is pathetic. What an ungodly mess. The
`The' bootup sequence? There are a number of alternative init systems
under development, some in use by reasonably major distros.
> FSTAB file needs to go and a smarter system needs to be developed. I
What's wrong with it? Sure, it doesn't solve all problems
(e.g. automounting; that's why we have automounters), and some of its
fields are passing obsolete (e.g. the fs_freq field), but that doesn't
make it *bad*. Those problems it aims to solve, it solves well.
Now /etc/mtab, *that* is an abomination, and a small kernel improvement
(allowing arbitrary flag strings to be passed by mount into the kernel
solely for display in the appropriate /proc/mounts field) could
eliminate it and replace it entirely with /proc/mounts.
> I think development needs to be done to make the kernel cleaner and
This has long been a goal.
> smarter
`Smarter' is easy to *say*. Defining what it *means* is another matter.
Myself I consider the new pluggable I/O schedulers and pluggable
packet schedulers make the kernel `smarter'. Perhaps you do not.
> rather than just bigger
This has never been a *goal*. It's a side-effect, that's all.
> and faster.
Faster is important, especially on very small and very large hardware,
where unusual things can become bottlenecks.
> It's time to look at what
> users need
What? Not in the kernel, it isn't. Users never see the
kernel. Sysadmins, sure; glibc, sure; users, no. Not even userspace
programs, except inasmuch as their requirements are reflected by new
demands on the kernel.
> and try to make Linux somewhat more windows like in being
> able to smartly recover from problems.
In my experience Windows tries to smartly recover, fails, and implodes
in a heap, telling all and sundry to `contact your system
administrator'. When you *are* the system administrator, this is less
than helpful.
One thing that *would* be nice is an improvement on errno, which is
a terribly coarse-grained error handling system. Something like the
sa_siginfo field, so you can pass additional info back with errors...
but for now that can be kludged around by passing back errors by
filling up buffers passed in other parameters.
errno seems to be Good Enough, but it's... *primitive*.
(What did Plan 9 do? Error strings?)
> Perhaps better error messages
> that your traditional kernel panic or hex dump screen of death.
The `traditional kernel panic' includes a backtrace; if you have kernel
syslogging or the serial console turned on, this can even work if
process scheduling and disk I/O are both dead.
Linux doesn't do `hex dump screens of death'.
A number of userspace tools like smartd provide nice warnings of some
critical non-kernel system failures in whatever way you see fit (my
systems send me an email and a page and shut themselves down if they
overheat or suffer a disk failure; much more elaborate things are
possible). But, again, this is not a kernel thing.
> The big challenge for Linux is to be able to put it in the hands of
> people who don't want to dedicate their entire life to understanding
> all the little quirks that we have become used to. The slogan should
> be "this just works"
This is the distributors' job (those who want to do that). One
disadvantage of `just works' systems is that they tend to be fiercely
complex and thus, when they *do* fail, the failures are correspondingly
hard to diagnose. (This is not always true: udev just works and is much
*less* complex than the system it replaces, with almost entirely
non-opaque failure modes. This comes, I think, of designing it properly
from the start. Kudos to Greg K-H!)
There will always be openings for less elaborate systems, I think.
> and is intuitive.
`The only intuitive interface is the nipple.'
> Linux Visionary
*chuckle*
--
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James Nicoll
On 4 Oct 2005, Bodo Eggert stated:
> BTW: YANI: That about a tmpfs where all-numerical entries can only be
> created by the corresponding UID? This would provide a secure, private
> tmp directory to each user without the possibility of races and denial-of-
> service attacks. Maybe it should be controlled by a mount flag.
Wouldn't it be less kludgy to just use the existing private namespace
stuff to provide each user with its own /tmp? (Or each user's session,
rather, which is probably much easier, as that corresponds precisely to
one process tree).
--
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James Nicoll
Nix wrote:
>On 4 Oct 2005, Bodo Eggert stated:
>
>
>>BTW: YANI: That about a tmpfs where all-numerical entries can only be
>>created by the corresponding UID? This would provide a secure, private
>>tmp directory to each user without the possibility of races and denial-of-
>>service attacks. Maybe it should be controlled by a mount flag.
>>
>>
>
>Wouldn't it be less kludgy to just use the existing private namespace
>stuff to provide each user with its own /tmp? (Or each user's session,
>rather, which is probably much easier, as that corresponds precisely to
>one process tree).
>
>
>
If you were going to do it right here's what you would do:
People who had files in /tmp would have no rights at all to other users
/tmp files.
Listing the dirtectory would only display the files you had some access
to. If you have no rights you don't even see that the file is there.
The effect would be like giving people their own tmp directories.
--
Marc Perkel - [email protected]
Spam Filter: http://www.junkemailfilter.com
My Blog: http://marc.perkel.com
On Wed, Oct 05, 2005 at 07:41:21AM -0700, Marc Perkel wrote:
> If you were going to do it right here's what you would do:
>
> People who had files in /tmp would have no rights at all to other users
> /tmp files.
> Listing the dirtectory would only display the files you had some access
> to. If you have no rights you don't even see that the file is there.
> The effect would be like giving people their own tmp directories.
Except it still wouldn't be able to go: Does file xyz exist? If not,
create file xyz. If someone else had xyz that you didn't see, you would
still not be able to create it. So what is the point of NOT showing it
other than to make it much harder to avoid conflicting names?
if you want to not see files that you have no rights to, filter it in
your user space application when it matters, and let user space see the
files when they need to in order to avoid name conflicts.
It would be an incredibly idiotic system that auto hides files just
because you can't use them. We have ways to hide files in user space
for the convinience of users. It would be too inconvinient for
applications if the OS hid files on us.
Len Sorensen
Lennart Sorensen wrote:
>On Wed, Oct 05, 2005 at 07:41:21AM -0700, Marc Perkel wrote:
>
>
>>If you were going to do it right here's what you would do:
>>
>>People who had files in /tmp would have no rights at all to other users
>>/tmp files.
>>Listing the dirtectory would only display the files you had some access
>>to. If you have no rights you don't even see that the file is there.
>>The effect would be like giving people their own tmp directories.
>>
>>
>
>Except it still wouldn't be able to go: Does file xyz exist? If not,
>create file xyz. If someone else had xyz that you didn't see, you would
>still not be able to create it. So what is the point of NOT showing it
>other than to make it much harder to avoid conflicting names?
>
>if you want to not see files that you have no rights to, filter it in
>your user space application when it matters, and let user space see the
>files when they need to in order to avoid name conflicts.
>
>It would be an incredibly idiotic system that auto hides files just
>because you can't use them. We have ways to hide files in user space
>for the convinience of users. It would be too inconvinient for
>applications if the OS hid files on us.
>
>Len Sorensen
>
>
What is incredibly idiotic is a file system that allws you to delete
files that you have no write access to. That is stupid beyond belief and
only the Unix community doesn't get it.
--
Marc Perkel - [email protected]
Spam Filter: http://www.junkemailfilter.com
My Blog: http://marc.perkel.com
On Wed, 5 Oct 2005, Lennart Sorensen wrote:
> On Wed, Oct 05, 2005 at 07:41:21AM -0700, Marc Perkel wrote:
>> If you were going to do it right here's what you would do:
>>
>> People who had files in /tmp would have no rights at all to other users
>> /tmp files.
>> Listing the dirtectory would only display the files you had some access
>> to. If you have no rights you don't even see that the file is there.
>> The effect would be like giving people their own tmp directories.
>
> Except it still wouldn't be able to go: Does file xyz exist? If not,
> create file xyz. If someone else had xyz that you didn't see, you would
> still not be able to create it. So what is the point of NOT showing it
> other than to make it much harder to avoid conflicting names?
>
> if you want to not see files that you have no rights to, filter it in
> your user space application when it matters, and let user space see the
> files when they need to in order to avoid name conflicts.
>
> It would be an incredibly idiotic system that auto hides files just
> because you can't use them. We have ways to hide files in user space
> for the convinience of users. It would be too inconvinient for
> applications if the OS hid files on us.
>
> Len Sorensen
> -
Also it has nothing at all to do with the kernel. It's what `ls`
or some other directory-reading program provides for the user.
People often forget that PATH, `pwd`, etc., are just filter
components!
When you `cd` to somewhere, your location hasn't changed at
all!
Without involving the kernel, one can make any kind of filter
to cause any sort of display that you want.
Cheers,
Dick Johnson
Penguin : Linux version 2.6.13 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.
****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.
Thank you.
On 10/5/05, Nix <[email protected]> wrote:
> On 4 Oct 2005, Bodo Eggert stated:
> > BTW: YANI: That about a tmpfs where all-numerical entries can only be
> > created by the corresponding UID? This would provide a secure, private
> > tmp directory to each user without the possibility of races and denial-of-
> > service attacks. Maybe it should be controlled by a mount flag.
>
> Wouldn't it be less kludgy to just use the existing private namespace
> stuff to provide each user with its own /tmp? (Or each user's session,
> rather, which is probably much easier, as that corresponds precisely to
> one process tree).
>
It would if the rest of the system really enforced this "privacy". In
plan 9 /tmp is really a bind to /usr/$user/tmp. And if you launch
something like "ramfs" [a userland 9P server] it binds a ram disk
device over /tmp by default unless you tell it otherwise, then you
have a ram-backed directory only for the current process and its
children in /tmp. This is useful for pulling things out of the
encrypted storage like factotum keys [sort of like a keyring for all
factotum based authentication including 9P mounts and even ssh
connections that use no ssh-keys]. When your process goes away so
does the decrypted keyfile, pretty nice.
Back on topic...
The problem with private namespaces on Linux is that they really
aren't so much. mount will update /etc/mtab for all to see and even
/proc/<pid>/mounts is world readable [though it doesn't give useful
bind information anyway on linux... just the disk device it appears].
On one hand you've got very specific information in mtab about all the
binding that's been done and on the other hand you've got not so
useful information on a per-process basis in /proc.
I think private namespaces could actually be made more-so but the rest
of the system has to cooperate and I doubt that I have the energy to
do the evangelism and requisite proofs of concept for Linux. It's far
easier for me to just use Plan 9 and Inferno instead of trying to
assimilate Linux, even though I think I'd prefer Linux if it were more
like the former two.
Maybe it's time for another BSD fork? [*runs away from the disapproval*]
Dave
On Wed, Oct 05, 2005 at 07:48:12AM -0700, Marc Perkel wrote:
> What is incredibly idiotic is a file system that allws you to delete
> files that you have no write access to. That is stupid beyond belief and
> only the Unix community doesn't get it.
If I have a directory and I want to remove it, I can almost always do
that. The file only goes away if there are no other hardlinks to it.
If someone cares about the file, they should keep a hardlink to it in a
directory THEY own.
Directories within directories on the other hand can make things a pain
since if you don't own the subdir, you can't remove its contents, so you
can't remove it. You could however likely move the dir somewhere else
to get it out of your way.
My directory is MY file and I get to do whatever I want to it. Who
knows how someone else managed to get a file into it in the first place.
/tmp is of course different since it has the bit turned on that says
only the file owner can delete it. If you want that enabled on all
directories, go ahead. It is supported, although who knows what
applications that might break. unix supports both ways of directory
behaviour after all. It isn't one way or the other.
Len Sorensen
On Wed, Oct 05, 2005 at 10:52:59AM -0400, linux-os (Dick Johnson) wrote:
> Also it has nothing at all to do with the kernel. It's what `ls`
> or some other directory-reading program provides for the user.
> People often forget that PATH, `pwd`, etc., are just filter
> components!
>
> When you `cd` to somewhere, your location hasn't changed at
> all!
So what does bash do that makes the new location 'busy' when you cd to
it such that you can't unmount it?
> Without involving the kernel, one can make any kind of filter
> to cause any sort of display that you want.
An it certainly is something that should be done in user space.
Len Sorensen
Marc Perkel wrote:
>
> What is incredibly idiotic is a file system that allws you to delete
> files that you have no write access to. That is stupid beyond belief and
> only the Unix community doesn't get it.
No.
What's idiotic is leaving your property (files) on someone elses desk
(directory) and expecting them always to be there when you come back.
n
Lennart Sorensen wrote:
>On Wed, Oct 05, 2005 at 07:48:12AM -0700, Marc Perkel wrote:
>
>
>>What is incredibly idiotic is a file system that allws you to delete
>>files that you have no write access to. That is stupid beyond belief and
>>only the Unix community doesn't get it.
>>
>>
>
>If I have a directory and I want to remove it, I can almost always do
>that. The file only goes away if there are no other hardlinks to it.
>If someone cares about the file, they should keep a hardlink to it in a
>directory THEY own.
>
>Directories within directories on the other hand can make things a pain
>since if you don't own the subdir, you can't remove its contents, so you
>can't remove it. You could however likely move the dir somewhere else
>to get it out of your way.
>
>My directory is MY file and I get to do whatever I want to it. Who
>knows how someone else managed to get a file into it in the first place.
>
>/tmp is of course different since it has the bit turned on that says
>only the file owner can delete it. If you want that enabled on all
>directories, go ahead. It is supported, although who knows what
>applications that might break. unix supports both ways of directory
>behaviour after all. It isn't one way or the other.
>
>Len Sorensen
>
>
Agian - thinking outside the box.
If the permissions were don'e right in your own directories your
inherited rights would give your permissions automatically to your home
directory and all directories uner it. Netware has a concept called an
inherited rights mask - something Linux lacks. Windows also has rights
like this and Samba emulates it. So unless root put files in your
directory and specifically denied you rights to them, you would have
full rights to your own directory.
However - if you were browsing the /etc directory and there were files
there that you had no read or write access to - then you wouldn't even
be able to list them. If you went to the home directory and lets say
everyone had 700 permissions on all the directories withing home, you
would only see your own directory. You wouldn't even be able to know
what other directories existed there.
If you want to start thinking about DOING IT RIGHT you need to think
beyond the Unix model and start looking at Netware. Maybe in 5 years
Linux will evolve to where Netware was in 1990.
Unix permissions totally suck but it's old baggage that you're stuck
with somewhat. Are you going to be stuck forever and is Linux ever going
to grow up and move on to better things? Linux is crippled when it comes
to permissions. The Windows people are laughing at you and you don't
even get it why they are laughing.
--
Marc Perkel - [email protected]
Spam Filter: http://www.junkemailfilter.com
My Blog: http://marc.perkel.com
On Wed, Oct 05, 2005 at 07:41:21AM -0700, Marc Perkel wrote:
> If you were going to do it right here's what you would do:
>
> People who had files in /tmp would have no rights at all to other users
> /tmp files.
> Listing the dirtectory would only display the files you had some access
> to. If you have no rights you don't even see that the file is there.
> The effect would be like giving people their own tmp directories.
ahh, *sigh*, i remember the days.
in 1989 i looked in /tmp on our sunos 4.1.3 server at
imperial, which was running a bit slow, went "eek, that's
a lot of files in /tmp" and did am rm -fr /tmp.
a few minutes later the sysadmins quite literally stormed in.
apparently the printer queue temp files were stored in /tmp and 100
third year students were all trying to print out their course-work,
last minute.
oops.
yes, imperial college third year theory of computing students of
1987-1990, it was me.
l.
On Wed, Oct 05, 2005 at 08:08:26AM -0700, Marc Perkel wrote:
> Agian - thinking outside the box.
>
> If the permissions were don'e right in your own directories your
> inherited rights would give your permissions automatically to your home
> directory and all directories uner it. Netware has a concept called an
> inherited rights mask - something Linux lacks. Windows also has rights
> like this and Samba emulates it. So unless root put files in your
> directory and specifically denied you rights to them, you would have
> full rights to your own directory.
Well I could have setup my home dirs with ACL to have inherited rights
to all files created under it for the owner. Well I didn't but then
again I don't allow other people to write to my home dir so it hasn't
been a problem.
> However - if you were browsing the /etc directory and there were files
> there that you had no read or write access to - then you wouldn't even
> be able to list them. If you went to the home directory and lets say
> everyone had 700 permissions on all the directories withing home, you
> would only see your own directory. You wouldn't even be able to know
> what other directories existed there.
Well I don't have write access to /etc so who cares. I do have write
access to /tmp and there is matters that I can list files I have no
access to. Hence why /tmp IS readable by all. If I couldn't list files
there I would have to randomly try filenames until I found one that
wasn't already in use but invisible to me due to idiot magic.
What error does netware return if you try to create a file in a
directory that already exists but which you can't see? Any error would
be indirectly a way to see the file, so it should have just been visible
in the first place.
> If you want to start thinking about DOING IT RIGHT you need to think
> beyond the Unix model and start looking at Netware. Maybe in 5 years
> Linux will evolve to where Netware was in 1990.
NetWare was not the be all and end all of filesystems. Far from it. It
had some good points, but it certainly didn't solve everything.
> Unix permissions totally suck but it's old baggage that you're stuck
> with somewhat. Are you going to be stuck forever and is Linux ever going
> to grow up and move on to better things? Linux is crippled when it comes
> to permissions. The Windows people are laughing at you and you don't
> even get it why they are laughing.
I find unix permissions work great in general, and for complex things
posix ACL has done everything I wanted it to and works great with samba.
The behaviours you claim are so great in netware to me seem like things
I would very much NOT want to have to deal with. I like things simple
enough to understand them so I can make sure they are right.
Len Sorensen
On Wed, 5 Oct 2005, Lennart Sorensen wrote:
> On Wed, Oct 05, 2005 at 10:52:59AM -0400, linux-os (Dick Johnson) wrote:
>> Also it has nothing at all to do with the kernel. It's what `ls`
>> or some other directory-reading program provides for the user.
>> People often forget that PATH, `pwd`, etc., are just filter
>> components!
>>
>> When you `cd` to somewhere, your location hasn't changed at
>> all!
>
> So what does bash do that makes the new location 'busy' when you cd to
> it such that you can't unmount it?
Well it doesn't make a file-system location busy! It's only files
that are open-for-write that prevent a file-system from being un-mounted!
You can properly shut down a system with the following commands from
a dumb terminal (ctrl-ALT F1,F2, etc).
kill -TERM -1 # Kill everybody but me and 'init'
sleep 1 # Wait a bit
kill -9 -1 # Really kick the hangers-on
sleep 1 # Wait again
umount -a # Umount all file systems
After you execute `umount -a`, you can still read the file-system
because `umount` only made it R/O.
`>foo`
shows that the file-system is R/O, you can hit the reset or power
switch now.
Certain distros create a file in the top directory that is supposed
to tell startup that the system was not properly shut down, "/.autofsck",
if you deleted that before the above sequence, the machine can
be restarted with no informational error messages about the shutdown.
`cd` executes function-code 12 which makes all opens() start from the
input string "path" if it doesn't have a full path. It's just a filter.
Same for opendir() if a directory listing is to be obtained.
>
>> Without involving the kernel, one can make any kind of filter
>> to cause any sort of display that you want.
>
> An it certainly is something that should be done in user space.
>
Could be done from user-space but opening an ordinary file
would require that the full path be obtained from somewhere
because the kernel wouldn't "know" where to create it if the
full path wasn't part of the open. `cd` is a kernel-call that
conveniently stores the part of the path-name that you don't
want to have to repeat for every open.
> Len Sorensen
>
Cheers,
Dick Johnson
Penguin : Linux version 2.6.13 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.
****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.
Thank you.
On Wed, Oct 05, 2005 at 04:24:47PM +0100, Luke Kenneth Casson Leighton wrote:
> ahh, *sigh*, i remember the days.
>
> in 1989 i looked in /tmp on our sunos 4.1.3 server at
> imperial, which was running a bit slow, went "eek, that's
> a lot of files in /tmp" and did am rm -fr /tmp.
Why would /tmp allow you to delete files there you didn't own unless you
were root? Why would someone with root blindly delete things they
didn't know what were?
> a few minutes later the sysadmins quite literally stormed in.
And promptly removed root access from the person that wasn't qualified
to have it in the first place? :)
> apparently the printer queue temp files were stored in /tmp and 100
> third year students were all trying to print out their course-work,
> last minute.
>
And why would the printer queue use /tmp in the first place?
> oops.
>
> yes, imperial college third year theory of computing students of
> 1987-1990, it was me.
Did they ever let you have root again?
Len Sorensen
On Wed, Oct 05, 2005 at 03:40:43PM +0200, Jens Axboe wrote:
> On Wed, Oct 05 2005, Luke Kenneth Casson Leighton wrote:
> > On Wed, Oct 05, 2005 at 02:31:14PM +0200, Jens Axboe wrote:
> >
> > > [i know better criticism and accusations deleted and not commented on]
> >
> > > Why is that so hard to understand? Succesful contributions start at the
> > > technical level, always have.
> >
> > then we will have to agree to disagree, because i believe that
> > successful contributions start with "what creative thing shall
> > we do now / what problem shall we tackle today in a creative
> > way?" and work their way down to the technical level, which,
> > as you rightly point out, requires successful _technical_
> > contributions.
>
> It's not my opinion, I'm stating historical fact. I honestly can't
> remember when a succesfull contribution was made based on a long
> non-technical thread. I can, however, remember oodles of cases where the
> reverse is quite true.
ah.
who's hosting the linux-visionaries mailing list, then?
On Wed, Oct 05, 2005 at 11:30:06AM -0400, Lennart Sorensen wrote:
> On Wed, Oct 05, 2005 at 04:24:47PM +0100, Luke Kenneth Casson Leighton wrote:
> > ahh, *sigh*, i remember the days.
> >
> > in 1989 i looked in /tmp on our sunos 4.1.3 server at
> > imperial, which was running a bit slow, went "eek, that's
> > a lot of files in /tmp" and did am rm -fr /tmp.
>
> Why would /tmp allow you to delete files there you didn't own unless you
> were root?
i have no idea. as a user, i just did rm -fr /tmp/* (sorry - not
rm -fr /tmp) and it worked.
as a user.
not root.
> > a few minutes later the sysadmins quite literally stormed in.
>
> And promptly removed root access from the person that wasn't qualified
> to have it in the first place? :)
they weren't dumb enough to give it to me.
> > apparently the printer queue temp files were stored in /tmp and 100
> > third year students were all trying to print out their course-work,
> > last minute.
> >
> And why would the printer queue use /tmp in the first place?
ahh, that would answer the implicit question as to why they
jumped up and down at me rather than frog-marched me off campus.
> > oops.
> >
> > yes, imperial college third year theory of computing students of
> > 1987-1990, it was me.
>
> Did they ever let you have root again?
i was a student there. they didn't let _anyone_ like me have root.
someone got into trouble for even demonstrating a security
vulnerability.
l.
On Wed, Oct 05 2005, Luke Kenneth Casson Leighton wrote:
> On Wed, Oct 05, 2005 at 03:40:43PM +0200, Jens Axboe wrote:
> > On Wed, Oct 05 2005, Luke Kenneth Casson Leighton wrote:
> > > On Wed, Oct 05, 2005 at 02:31:14PM +0200, Jens Axboe wrote:
> > >
> > > > [i know better criticism and accusations deleted and not commented on]
> > >
> > > > Why is that so hard to understand? Succesful contributions start at the
> > > > technical level, always have.
> > >
> > > then we will have to agree to disagree, because i believe that
> > > successful contributions start with "what creative thing shall
> > > we do now / what problem shall we tackle today in a creative
> > > way?" and work their way down to the technical level, which,
> > > as you rightly point out, requires successful _technical_
> > > contributions.
> >
> > It's not my opinion, I'm stating historical fact. I honestly can't
> > remember when a succesfull contribution was made based on a long
> > non-technical thread. I can, however, remember oodles of cases where the
> > reverse is quite true.
>
> ah.
>
> who's hosting the linux-visionaries mailing list, then?
:-)
Don't think there is one, but if it will take the traffic of this list,
I'm sure davem can be quickly convinced to add one!
--
Jens Axboe
On Tue, 4 Oct 2005, Marc Perkel wrote:
> Marc Perkel
> Linux Visionary
The problem I have with most visionaries is that while
they see lots of stuff, they never show anything to the
rest of the world.
Unless you can explain to other people why your idea
makes sense, or unless you implement your idea, it won't
happen.
If the idea is good enough, either of explanation or
implementation should be enough.
Preaching, OTOH, does not convince people.
--
All Rights Reversed
On Wed, Oct 05, 2005 at 04:42:26PM +0100, Luke Kenneth Casson Leighton wrote:
> i have no idea. as a user, i just did rm -fr /tmp/* (sorry - not
> rm -fr /tmp) and it worked.
>
> as a user.
>
> not root.
Then some admin didn't qualify for root having apparently removed the t
bit from /tmp making it a world writeable dir. Ouch.
> they weren't dumb enough to give it to me.
But they made /tmp world writeable it seems. Impresive. :)
> ahh, that would answer the implicit question as to why they
> jumped up and down at me rather than frog-marched me off campus.
Yep. What you did should have been prevented by the system. So the
system was misconfigured.
> i was a student there. they didn't let _anyone_ like me have root.
>
> someone got into trouble for even demonstrating a security
> vulnerability.
Well this one sounds more liek a major misconfiguration than a security
problem. Well allowing people to mess with temp could be seen as a
security problem but only until the permissions were fixed back to what
they would have originally been when the system was installed.
Len Sorensen
Rik van Riel wrote:
>On Tue, 4 Oct 2005, Marc Perkel wrote:
>
>
>
>>Marc Perkel
>>Linux Visionary
>>
>>
>
>The problem I have with most visionaries is that while
>they see lots of stuff, they never show anything to the
>rest of the world.
>
>Unless you can explain to other people why your idea
>makes sense, or unless you implement your idea, it won't
>happen.
>
>If the idea is good enough, either of explanation or
>implementation should be enough.
>
>Preaching, OTOH, does not convince people.
>
>
>
This stuff I'm talking about is not theoretical. It's been in Novell
Netware since 1990 and it works great. Netware with DOS in 1990 is still
far superior to Linux today. Once you've had Netware - Linux is
laughable. All youhave to do is look ate Netware and copy it. Or the
mars-nwe netware emulator for that matter. The code to do this already
exists.
--
Marc Perkel - [email protected]
Spam Filter: http://www.junkemailfilter.com
My Blog: http://marc.perkel.com
Marc Perkel <[email protected]> wrote:
> I think it's time for some innovative thinking and for people to step
> outside the Linux box and look around at other operating systems for
> some good ideas. I'll run through a few ideas here.
Great! Where are your patches?
> Reiser 4 - The idea of building a file system on top of a database is
> the right way to go.
It's insanity. I'll give it to you in writing, if you want.
> Reiser is onto something here and this is a
> technology that needs to be built upon. It's current condition is a
> little on the week side - no ACLs for example - but the underlying
> concept is ound.
There are many who disagree. Even violently...
> Novell Netware type permissions. ACLs are a step in the right
> direction but Linux isn't any where near where Novell was back in
> 1990.
Details?
> Linux lets you - for example - to delete files that you have no
> read or write access rights to.
Yep. This is spelled U-N-I-X. You just (re)discovered the most "discovered"
(by newbies) "security bug" in Unix.
> Netware on the other hand prevents you
> from deleting files that you can't write to and if you have no right
> it is as if the file isn't there.
Sorry, but you don't "delete a file" here, you delete a /link/ to a file.
Yes, if it was the last one the file goes away, but that is just a special
case.
> You can't even see it in the
> directory. Netware also has inherited permissions like Windows
Inherited permissions are /evil/, as you need to check all the way up to /
to find out if something is allowed or not.
> and
> Samba has and this is doing it right. File systems and individual
> directories should be able to be flagged as
> casesensitive/insensitive.
Here file names aren't made up of "letters" that have "cases", they are
just random strings of (almost unrestricted) bytes. So you are free to
interpret them as japanese or arabic. Your choice.
> Permissions need to be fine grained and
> easy to use.
"Fine grained" is almost exactly the opposite to "easy to use".
> Netware is a good example to emulate.
Go ahead and write your filesystem then! It might even be of some use to be
able to read Netware volumes. I'm not trying to stop you.
> The bootup sequence of Linux is pathetic.
This has nothing to do with Linux (the kernel).
> What an ungodly mess. The
> FSTAB file needs to go and a smarter system needs to be developed. I
> know this isn't entirely a kernel issue but it is somewhat related.
In some 30 years nobody has been able to come up with something better. If
you have a better idea, I'm all ears. Note that the mechanism would have to
be just as simple (or hopefully simpler).
> I think development needs to be done to make the kernel cleaner
Sign up with the kernel janitors.
> and
> smarter
That's part of the idea around here.
> rather than just bigger
Nobody is advocating "bigger". It grows mostly to handle new pieces of
hardware, or new requirements. Again, if you have concrete proposals on how
to shrink the kernel, everybody here will jump of joy. Big is slow, and
thus bad.
> and faster.
What is bad about "faster"?
> It's time to look at what
> users need
The distribution people are looking into that, don't worry.
> and try to make Linux somewhat more windows like in being
> able to smartly recover from problems.
Please, everything /but/ "Windows style problem recovery"! I /much/ prefer
the system to crash than to (silently) paper over serious problems until it
is far too late to do anything about their effects.
> Perhaps better error messages
> that your traditional kernel panic or hex dump screen of death.
Patches are more than welcome.
> The big challenge for Linux is to be able to put it in the hands of
> people who don't want to dedicate their entire life to understanding
> all the little quirks that we have become used to. The slogan should
> be "this just works" and is intuitive.
It is almost there, in the form of the more user-friendly distributions.
Again, help is higly appreciated there.
> Anyhow - before I piss off too many people who are religiously
> attached to Linux worshiping - I'll quit not. ;)
Thanks.
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
On Wed, Oct 05, 2005 at 08:58:13AM -0700, Marc Perkel wrote:
> This stuff I'm talking about is not theoretical. It's been in Novell
> Netware since 1990 and it works great. Netware with DOS in 1990 is still
> far superior to Linux today. Once you've had Netware - Linux is
> laughable. All youhave to do is look ate Netware and copy it. Or the
> mars-nwe netware emulator for that matter. The code to do this already
> exists.
Novell will happily sell you Netware if you are so inclined, I suppose.
As long as its consentual, it's really your business - one of three is
not that bad, even if "sane" and "safe" are missing...
On Wed, 5 Oct 2005, Marc Perkel wrote:
> What is incredibly idiotic is a file system that allws you to delete
> files that you have no write access to. That is stupid beyond belief and
> only the Unix community doesn't get it.
1) Unlinking is not deleting, it may just trigger deleting. !Unix people
don't get it.
If you like non-owners to have no unlink permission, you have to set
the sticky bit. If you want other users not to be able to delete files
you put into a public directory, put a link into that directory and
keep one in a private directory.
2) If you're accounted for the space occupied by your directories, you
need the permission to remove the directory. Otherwise you could DoS
other users if you have write permission in one of his directories.
--
Funny quotes:
29. When someone asks you, "A penny for your thoughts" and you put your two
cents in . . . what happens to the other penny?
Al Viro wrote:
>On Wed, Oct 05, 2005 at 08:58:13AM -0700, Marc Perkel wrote:
>
>
>>This stuff I'm talking about is not theoretical. It's been in Novell
>>Netware since 1990 and it works great. Netware with DOS in 1990 is still
>>far superior to Linux today. Once you've had Netware - Linux is
>>laughable. All youhave to do is look ate Netware and copy it. Or the
>>mars-nwe netware emulator for that matter. The code to do this already
>>exists.
>>
>>
>
>Novell will happily sell you Netware if you are so inclined, I suppose.
>As long as its consentual, it's really your business - one of three is
>not that bad, even if "sane" and "safe" are missing...
>
>
That's not the point. The point is that Netware has a far superior
permission system and I am suggesting the the Linux community learn from
it and take advantage of seeing what better looks like and improving itself.
--
Marc Perkel - [email protected]
Spam Filter: http://www.junkemailfilter.com
My Blog: http://marc.perkel.com
On Wed, 5 Oct 2005, David Leimbach wrote:
[snip quotes]
> It would if the rest of the system really enforced this "privacy". In
> plan 9 /tmp is really a bind to /usr/$user/tmp. And if you launch
> something like "ramfs" [a userland 9P server] it binds a ram disk
> device over /tmp by default unless you tell it otherwise, then you
> have a ram-backed directory only for the current process and its
> children in /tmp.
[...]
> This is useful for pulling things out of the
> encrypted storage like factotum keys [sort of like a keyring for all
> factotum based authentication including 9P mounts and even ssh
> connections that use no ssh-keys]. When your process goes away so
> does the decrypted keyfile, pretty nice.
You'd usurally just create+open a file and erase it without closing it.
The only access to this file is by using the file descriptor (or, off
cause, /proc/pid/fd/num). If the last reference to this file, the running
process, is gone, so is the file.
> Back on topic...
>
> The problem with private namespaces on Linux is that they really
> aren't so much. mount will update /etc/mtab for all to see and even
Userspace problem.-)
> /proc/<pid>/mounts is world readable [though it doesn't give useful
> bind information anyway on linux... just the disk device it appears].
There was some proc privacy patch some time ago. It was argued about
because some sites want peer review on system usage. I lost track
if it was included.
> I think private namespaces could actually be made more-so but the rest
> of the system has to cooperate and I doubt that I have the energy to
> do the evangelism and requisite proofs of concept for Linux. It's far
> easier for me to just use Plan 9 and Inferno instead of trying to
> assimilate Linux, even though I think I'd prefer Linux if it were more
> like the former two.
The plan is:
1) make namespaces joinable
2) ???
3) profit
No, that's wrong. The plan is (should be?):
1) make namespaces joinable in a sane way
2) wait for the shared subtree patch
3) make pam join the per-user-namespace
4) make pam automount tmpfs on the private /tmp
--
Top 100 things you don't want the sysadmin to say:
44. System coming down in 0 min....
Nikita Danilov wrote:
> Marc Perkel writes:
>
> [...]
>
> > Right - that's Unix "inside the box" thinking. The idea is to make the
> > operating system smarter so that the user doesn't have to deal with
> > what's computer friendly - but reather what makes sense to the user.
> > From a user's perspective if you have not rights to access a file then
> > why should you be allowed to delete it?
>
> Because in Unix a name is not an attribute of a file.
>
> Files are objects that you read, write and truncate. They are
> represented by inodes.
>
> Separately from that, there is an indexing structure: directory
> tree. Directories map symbolical names to inodes. Obviously, adding a
> reference to an index, or removing it from one requires access
> permission to the _index_ rather then to the object being referenced.
>
> That two-level model of files and indexing on top of them is essential
> to Unix due to the flexibility and conceptual economy it provides.
>
We should print that on post-it notes for grandmothers
to read when they are interacting with Unix file systems.
> >
> > So - the thread is about the future so I say - time to fix Unix.
>
> One thing is clear: it's too late to fix Netware. Why should Unix
> emulate its lethal defects?
Like NetWare's defect of it being intuitive and easy to
administer file system rights? Hey, here's a thought. Maybe
the operating system could actually exist to SERVE the human
instead of vice versa.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Electronics
=============================
On Wed, 5 Oct 2005, Jens Axboe wrote:
> >
> > who's hosting the linux-visionaries mailing list, then?
>
> :-)
>
> Don't think there is one, but if it will take the traffic of this list,
> I'm sure davem can be quickly convinced to add one!
>
Amen!
(noise to content is very high lately)
And the sad part is that I keep reading this crap. It's like being sick
at home one day and getting stuck on a bad soap opera. You know the stuff
you're watching is garbage, but you just can't get away and stop watching
it.
At least if it moved to another list I'd be strong enough not to
subscribe. :-)
-- Steve
ss, is gone, so is the file.
>
> > Back on topic...
> >
> > The problem with private namespaces on Linux is that they really
> > aren't so much. mount will update /etc/mtab for all to see and even
>
> Userspace problem.-)
Sure is, which is why I think it's easier to fork a BSD to make it do
what someone wants than to roll my own linux distribution :-).
Perhaps that's a problem of perception and not a really good
measurement of the amount of energy involved in either alternative.
Sometimes it sure seems easier to keep the userspace stuff with the kernel.
> > I think private namespaces could actually be made more-so but the rest
> > of the system has to cooperate and I doubt that I have the energy to
> > do the evangelism and requisite proofs of concept for Linux. It's far
> > easier for me to just use Plan 9 and Inferno instead of trying to
> > assimilate Linux, even though I think I'd prefer Linux if it were more
> > like the former two.
>
> The plan is:
>
> 1) make namespaces joinable
> 2) ???
> 3) profit
>
> No, that's wrong. The plan is (should be?):
>
> 1) make namespaces joinable in a sane way
> 2) wait for the shared subtree patch
> 3) make pam join the per-user-namespace
> 4) make pam automount tmpfs on the private /tmp
I'm not sure what you mean by a joinable namespace. I also am not
sure I want them :-).
I think of namespaces as being fundamental to the process model and
that they are inherited from the parent and new ones are created in a
sort of COW fashion [or at least have similar behavior].
You might want a session namespace instead of a joinable per-process
namespace but I think that might be a slightly different point of
view.
Dave
On 10/5/05, Luke Kenneth Casson Leighton <[email protected]> wrote:
>
> where it all goes a bit pearshaped with that chicken-and-egg vicious
> cycle is if the bottom drops out of 65nm and 45nm processes, such that
> _even_ the top uniprocessor mass-market chip manufacturers are forced
> down a parallel processing line.
>
> my point is: we're starting to see evidence of that happening
> (small-scale, 2-cores, 2-hyperthreads, talk of 4-cores, etc.
> even the X-Box 360 PPC 3x2)
>
> _therefore_, i invite people who do linux kernel development
> to think ahead - to take a _lead_ for once instead of waiting
> for hardware to drop into their laps, at which point it is once
> again too late, the hardware design decisions will have
> already been made by someone else, and you will be treated
> like second class citizens. again.
With all due respect (and I do believe some is due); your comment
above makes no sense. Operating system designers design software to
operate on existing systems -- often doing as much as possible to
ensure that the design supports lots of different systems. However, _a
neccessary prerequisite_ for that activity is "hardware dropping into
their laps." You warn that "the hardware design decisions will have
already been made by someone else;" but that is the order in which it
_neccessarily_ works! What hardware designer out there spends his day
looking around for as-yet-unused (and hence untested) operating
systems that run on some hypothetical yet-to-be-designed hardware and
say to themselves, "ah, that looks nice -- I think I can design some
hardware that will run that software _really well_?"
Your argument defies logic, and it's that fact -- not your intentions
-- that makes this thread so tiresome.
Dave
On Wednesday 05 October 2005 05:49, Marc Perkel wrote:
> D. Hazelton wrote:
> >>Novell Netware type permissions. ACLs are a step in the right
> >>direction but Linux isn't any where near where Novell was back in
> >>1990. Linux lets you - for example - to delete files that you
> >> have no read or write access rights to.
> >
> >As someone else pointed out, this is because unlinking is related
> > to your access permissions on the parent directory and not the
> > file.
>
> Right - that's Unix "inside the box" thinking. The idea is to make
> the operating system smarter so that the user doesn't have to deal
> with what's computer friendly - but reather what makes sense to the
> user. From a user's perspective if you have not rights to access a
> file then why should you be allowed to delete it?
You're confusing concepts. In Unix unlinking a file is not the same as
deleting it. As has already been said, to remove content from a file,
you truncate it, which, no surprise, requires that you have write
access to a file. Even in DOS deleting a file, unless you use a
secure delete program, doesn't delete the file - it merely changes
the name slightly and marks the chain of FAT cluster entries as
usable.
I've had the displeasure of having to fix a netware system that had
been so fsked up by an admin that had been fired that it was easier
for me to remove the volume and restore it from a backup. The problem
was that he made a large number of files with the administrative
account removed from the ACL's... And the same problem plagues
(plagued? I haven't checked up on this in a while) NTFS. It is all to
possible to create a bunch of files with "Administrator" and all
other "Administrator" class users form the ACL's and then kill that
user.
> Now - the idea is to create choice. If you need to emulate Unix
> nehavior for compatibility that's fine. But I would migrate away
> from that into a permissions paradygme that worked like Netware.
So provide a filesystem and a set of tools for that filesystem. Nobody
is standing in your way and the Linux filesystem and block device
layers are open enough that this is an easy (though not simple) task.
> I started with Netware and I'm spoiled. They had it right 15 years
> ago and Linux isn't any where near what I was with Netware and DOS
> in 1990. Once you've had this kind of permission power Linux is a
> real big step down.
Oh, so that explains it. You got used to one paradigm and haven't been
able to adjust to another. Well, as I have previously said, go ahead
and provide us with the work.
> So - the thread is about the future so I say - time to fix Unix.
Time to fix Unix? I doubt something seriously borked would have
outlasted every other OS on the market. Unix was around before
Netware and, IMHO, will be around a long time after the last
adherents of NetWare are gone. (and with MS doing it's level best to
kill NetWare with it's own shared filesystems and built-in networking
this cannot be that far off. After all, Netware was developed to fill
a vacancy in the MS world.)
DRH
Nix <[email protected]> wrote:
> On 4 Oct 2005, Marc Perkel announced authoritatively:
>> Netware also has inherited permissions like Windows and
>> Samba has and this is doing it right.
>
> s/right/wrong/
>
> Look at a NetWare permission on some file and you can't tell what the
> effective permission is, because it depends on inherited ones as well.
> Look at an effective permission and you can't tell which parts of it
> will change if you change the inherited permission.
MS solved that part by not allowing partially defined permissions.
(At least AFAI can tell from the UI.)
> Unix has inherited permissions in one sense: you can't get at a file
> via some path if one of the directories in that path is unreadable.
NACK. ITYM non-accessable (-x), and it's only true if you can't reach it or
one of the other links from $PWD and the directories you can fchdir to
(which, admittedly, is the usural case).
> Even *that* causes regular problems,
ACK.-)
>> File systems and individual
>> directories should be able to be flagged as casesensitive/insensitive.
>
> Only if you're willing to change POSIX to include a call to check filenames
> for identity,
You'd "just" need a way to determine the canonialized form. Still an evil
masterplan.
[...]
> It would also require case-conversion and locale-handling code, probably
> including UTF-8 canoncalization code, inside the kernel. This would
> greatly increase kernel complexity for a very small reward, and lead to
> Al Viro's early death from cerebral aneurysm combined with involuntary
> projectile vomiting. This cannot be considered a good thing.
a) NLS is in the kernel, and if utf-8 filenames are supposed to be used,
an utf-8 checker rejecting non-canonialized strings will be desirable
to avoid binary trash in filenames or lookalike filenames. The
conversion to the canonialized form should happen outside the kernel.
(I know POSIX doesn't define a propper return value, but the return
value used for VFAT works and it's better than dealing with
$'M\x{0430}kefile' looking like Makefile)
b) ACK, I don't think caseless handling of filenames is a good thing,
it would needlessly bloat the kernel by opening a can of worms.
E.g. '?' would be converted to 'SS'[0] in German or 'B' in greek.
[0] or, if you like and bend the standard, to 'SZ' if the word with 'SS'
would clash with another word.
> Now /etc/mtab, *that* is an abomination, and a small kernel improvement
> (allowing arbitrary flag strings to be passed by mount into the kernel
> solely for display in the appropriate /proc/mounts field) could
> eliminate it and replace it entirely with /proc/mounts.
What about making all fs ignore the
-omount="$tool:foo:bar;$tool2=baz:barf..." parameter?
This is a cruel hack, but it will be backward compatible.
(If your hammer is big enough, the problem may turn out to be a nail.)
--
Ich danke GMX daf?r, die Verwendung meiner Adressen mittels per SPF
verbreiteten L?gen zu sabotieren.
Luke Kenneth Casson Leighton <[email protected]> wrote:
> On Wed, Oct 05, 2005 at 01:24:12PM +0400, Nikita Danilov wrote:
> > Marc Perkel writes:
> > [...]
> >
> > > Right - that's Unix "inside the box" thinking. The idea is to make the
> > > operating system smarter so that the user doesn't have to deal with
> > > what's computer friendly - but reather what makes sense to the user.
> > > From a user's perspective if you have not rights to access a file then
> > > why should you be allowed to delete it?
> > Because in Unix a name is not an attribute of a file.
> there is no excuse.
It's not an excuse, it's part of a coherent view of how things work. Just
as Netware used to have its, and DOS had its (sort of). As the world view
underneath Unix, it is darn hard to "fix".
[This discussion sounds quite a lot like it is /you/ who needs "fixing",
i.e., first wrap your head around Unix' ways...]
> selinux has already provided an alternative that is similar to NW
> file permissions.
Nope. SELinux provides MAC, i.e., mechanisms by which system-wide policy
(not the random owner of an object) ultimately decides what operations to
allow. That is not "file permissions". And yes, this is quite un-Unix-like.
[...]
> in what way is it possible for linux to fully support the NTFS
> filesystem?
If you ask me, preferably not at all, just let that unholy mess quietly go
the way of the dinosaurs. Sadly, interoperability is required at times,
so...
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
On Wed, 05 Oct 2005, Marc Perkel yowled:
> Agian - thinking outside the box.
I hate that phrase. There is no `box'.
> If the permissions were don'e right in your own directories your
> inherited rights would give your permissions automatically to your
> home directory and all directories uner it. Netware has a concept
> called an inherited rights mask - something Linux lacks. Windows also
> has rights like this and Samba emulates it. So unless root put files
> in your directory and specifically denied you rights to them, you
> would have full rights to your own directory.
So, um, what happens to these permissions when you copy a file and put
it somewhere else? Do the inherited rights go with it or not? In Unix
it's pretty intuitive. In this system there seem to be two right
answers, both of which seem... risky from a security perspective.
> However - if you were browsing the /etc directory and there were files
> there that you had no read or write access to - then you wouldn't even
> be able to list them.
/tmp is the problem here, and shows the futility and pointlessness of
this feature. If you have an unlistable file in /tmp, *its name is still
determinable*, because other users cannot create files with that
name. The concept adds *nothing* over some combination of dirs with the
execute bit cleared for some set of users and subdirectories which
cannot be read by some set of users. There's no need for this profoundly
non-Unixlike permission at all. (As usual, ACLs make managing this on
a fine-grained scale rather easier.)
> If you went to the home directory and lets say
> everyone had 700 permissions on all the directories withing home, you
> would only see your own directory. You wouldn't even be able to know
> what other directories existed there.
This is what per-process filesystems are for.
> If you want to start thinking about DOING IT RIGHT you need to think
> beyond the Unix model and start looking at Netware. Maybe in 5 years
> Linux will evolve to where Netware was in 1990.
I think Plan 9 is a better goal than Netware. At least it was designed
by people aiming for a better Unix rather than people trying to build a
better DOS, and so is more likely to have a compatible philosophy.
> Unix permissions totally suck but it's old baggage that you're stuck
> with somewhat. Are you going to be stuck forever and is Linux ever
> going to grow up and move on to better things? Linux is crippled when
> it comes to permissions.
Well, you can't change it drastically without violating POSIX. There's
no damned way Linux is going to do *that*.
> The Windows people are laughing at you and
> you don't even get it why they are laughing.
You *do* realise just how incapable the Windows permission-management
GUI is, don't you? Any OS where the command-line tools hide half
the permissions model and the GUI hides a slightly different half,
and where looking at a set of permissions and hitting cancel can
*change* those permissions drastically, is not sane.
(Disclaimer: the last time I bothered to verify the latter behaviour
was in NT4. Maybe they've partially fixed it.)
--
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James Nicoll
On Wed, 5 Oct 2005, David Leimbach said:
>> 1) make namespaces joinable in a sane way
>> 2) wait for the shared subtree patch
>> 3) make pam join the per-user-namespace
>> 4) make pam automount tmpfs on the private /tmp
>
> I'm not sure what you mean by a joinable namespace. I also am not
> sure I want them :-).
They are namespaces which processes can join. Right now you can do
it by chrooting into /proc/{pid}/root, but this is, as Bodo said,
not a very sane API.
Without this, a user starting two sessions gets two namespaces,
which is profoundly counterintuitive from the user's POV.
> I think of namespaces as being fundamental to the process model and
> that they are inherited from the parent and new ones are created in a
> sort of COW fashion [or at least have similar behavior].
Yes, but you can change them too (that's what e.g. mount() is for!)
> You might want a session namespace instead of a joinable per-process
> namespace but I think that might be a slightly different point of
> view.
I think that's the idea; a filesystem holding namespaces that you're
allowed to chroot() into.
--
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James Nicoll
On Wed, 05 Oct 2005, Bodo Eggert suggested tentatively:
> Nix <[email protected]> wrote:
>> On 4 Oct 2005, Marc Perkel announced authoritatively:
>
>> Look at a NetWare permission on some file and you can't tell what the
>> effective permission is, because it depends on inherited ones as well.
>> Look at an effective permission and you can't tell which parts of it
>> will change if you change the inherited permission.
>
> MS solved that part by not allowing partially defined permissions.
> (At least AFAI can tell from the UI.)
The UI lies grossly, but in this area I think you are right. (I'm still
unclear as to what happens to permissions when you copy something:
do the inherited parts change, or not?)
>> Unix has inherited permissions in one sense: you can't get at a file
>> via some path if one of the directories in that path is unreadable.
>
> NACK. ITYM non-accessable (-x), and it's only true if you can't reach it or
> one of the other links from $PWD and the directories you can fchdir to
> (which, admittedly, is the usural case).
Yeah, I was being sloppy. You got what I meant. (In similar wise, file
permissions don't stop you from reading a file if you can coerce someone
else into opening it and passing you its fd. Nonetheless, file
permissions almost always *do* work.)
>>> File systems and individual
>>> directories should be able to be flagged as casesensitive/insensitive.
>>
>> Only if you're willing to change POSIX to include a call to check filenames
>> for identity,
>
> You'd "just" need a way to determine the canonialized form. Still an evil
> masterplan.
It'd still require changing POSIX and rewriting a large part of the
universe.
> [...]
>> It would also require case-conversion and locale-handling code, probably
>> including UTF-8 canoncalization code, inside the kernel. This would
>> greatly increase kernel complexity for a very small reward, and lead to
>> Al Viro's early death from cerebral aneurysm combined with involuntary
>> projectile vomiting. This cannot be considered a good thing.
>
> a) NLS is in the kernel,
I don't think enough of it to do this is in there, at least not if you
aren't using something like NTFS.
> and if utf-8 filenames are supposed to be used,
> an utf-8 checker rejecting non-canonialized strings will be desirable
> to avoid binary trash in filenames or lookalike filenames. The
> conversion to the canonialized form should happen outside the kernel.
Yes. But where? libc? (I can just see Ulrich going for *this*!)
> b) ACK, I don't think caseless handling of filenames is a good thing,
> it would needlessly bloat the kernel by opening a can of worms.
> E.g. '?' would be converted to 'SS'[0] in German or 'B' in greek.
... which means that either you lose per-process locale-dependence
via LANG et all, or you get the possibility of directories containing
several files with the same name from some users' POV.
Neither seems good to me; even though we already have part of this with
NTFS, we should not inflict it on people needlessly.
>> Now /etc/mtab, *that* is an abomination, and a small kernel improvement
>> (allowing arbitrary flag strings to be passed by mount into the kernel
>> solely for display in the appropriate /proc/mounts field) could
>> eliminate it and replace it entirely with /proc/mounts.
>
> What about making all fs ignore the
> -omount="$tool:foo:bar;$tool2=baz:barf..." parameter?
Well, they'd have to dump it into /proc/mounts, too (I guess that would
happen magically). That would unbreak the few things this cares about.
> This is a cruel hack, but it will be backward compatible.
> (If your hammer is big enough, the problem may turn out to be a nail.)
Indeed. I'm fairly sure this problem is trivially solvable: it's far
easier than the *other* problems lying in wait down the per-user-
namespace path. :)
--
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James Nicoll
On Wed, Oct 05, 2005 at 09:23:56AM -0700, Marc Perkel wrote:
> That's not the point. The point is that Netware has a far superior
> permission system and I am suggesting the the Linux community learn from
> it and take advantage of seeing what better looks like and improving itself.
Linux is compatible with unix applications. Netware is not. Supporting
some useless netware feature at the expense of posix/unix compatibility
would be insane.
If you can't do it with unix permissions or unix permissions + ACL, you
don't need to do it at all most likely, and even more likely you
probably don't actually understand what you are trying to accomplish and
will probably end up making something insecure or broken instead.
Having dealt with netware, trying to administrate that mess was way to
painful and confusing. What a horrible interface. I can't imagine
anything I would want to borrow from netware.
Len Sorensen
Nix wrote:
>On Wed, 05 Oct 2005, Marc Perkel yowled:
>
>
>>Agian - thinking outside the box.
>>
>>
>
>I hate that phrase. There is no `box'.
>
>
>
That's what it looks like when you are inside it.
>>If the permissions were don'e right in your own directories your
>>inherited rights would give your permissions automatically to your
>>home directory and all directories uner it. Netware has a concept
>>called an inherited rights mask - something Linux lacks. Windows also
>>has rights like this and Samba emulates it. So unless root put files
>>in your directory and specifically denied you rights to them, you
>>would have full rights to your own directory.
>>
>>
>
>So, um, what happens to these permissions when you copy a file and put
>it somewhere else? Do the inherited rights go with it or not? In Unix
>it's pretty intuitive. In this system there seem to be two right
>answers, both of which seem... risky from a security perspective.
>
>
You inherit the rights of the new directory.
Also - under Netware not all permissions are stored with the file. The
rights are calculated from the file heirachy so you don't store a lot of
data with each file unless the file has permissions set that is
different than that of the directory it's in. So moving a file to
someone's home directory doesn't require any permissions to be set to
give the user rights to the file.
>
>
>>However - if you were browsing the /etc directory and there were files
>>there that you had no read or write access to - then you wouldn't even
>>be able to list them.
>>
>>
>
>/tmp is the problem here, and shows the futility and pointlessness of
>this feature. If you have an unlistable file in /tmp, *its name is still
>determinable*, because other users cannot create files with that
>name. The concept adds *nothing* over some combination of dirs with the
>execute bit cleared for some set of users and subdirectories which
>cannot be read by some set of users. There's no need for this profoundly
>non-Unixlike permission at all. (As usual, ACLs make managing this on
>a fine-grained scale rather easier.)
>
>
>
It doesn't really make sense to use the /tmp directory the way Unix uses
it. Why would you want just anyone to even know the names of the
temporary files you are using. Users should have their own temp
directory or create their own directory within /tmp
But - to address your question - if there were an invisible (to you)
file in a directory that you had create rights to then you would get a
file creation error.
>> If you went to the home directory and lets say
>>everyone had 700 permissions on all the directories withing home, you
>>would only see your own directory. You wouldn't even be able to know
>>what other directories existed there.
>>
>>
>
>This is what per-process filesystems are for.
>
>
>
>>If you want to start thinking about DOING IT RIGHT you need to think
>>beyond the Unix model and start looking at Netware. Maybe in 5 years
>>Linux will evolve to where Netware was in 1990.
>>
>>
>
>I think Plan 9 is a better goal than Netware. At least it was designed
>by people aiming for a better Unix rather than people trying to build a
>better DOS, and so is more likely to have a compatible philosophy.
>
>
>
I'm not familiar with Plan 9.
>>Unix permissions totally suck but it's old baggage that you're stuck
>>with somewhat. Are you going to be stuck forever and is Linux ever
>>going to grow up and move on to better things? Linux is crippled when
>>it comes to permissions.
>>
>>
>
>Well, you can't change it drastically without violating POSIX. There's
>no damned way Linux is going to do *that*.
>
>
>
>> The Windows people are laughing at you and
>>you don't even get it why they are laughing.
>>
>>
>
>You *do* realise just how incapable the Windows permission-management
>GUI is, don't you? Any OS where the command-line tools hide half
>the permissions model and the GUI hides a slightly different half,
>and where looking at a set of permissions and hitting cancel can
>*change* those permissions drastically, is not sane.
>
>
That's why I'm pushing netware as a model rather than windows. But
Windows file permissions are superior to Linux.
>(Disclaimer: the last time I bothered to verify the latter behaviour
>was in NT4. Maybe they've partially fixed it.)
>
>
>
One place where Windows wins over Linux is in the "easy to use"
category. Something the Linux world should look ast.
I am a Linux supporter and love it. I'm saying this to help make it better.
--
Marc Perkel - [email protected]
Spam Filter: http://www.junkemailfilter.com
My Blog: http://marc.perkel.com
On Wed, Oct 05, 2005 at 08:16:19PM +0100, Nix wrote:
> On Wed, 05 Oct 2005, Marc Perkel yowled:
> > If you want to start thinking about DOING IT RIGHT you need to think
> > beyond the Unix model and start looking at Netware. Maybe in 5 years
> > Linux will evolve to where Netware was in 1990.
Ugly as hell and thankfully about to go extinct outside of dark places
where its malignant presense still lingers amidst the Shoggoth droppings?
> I think Plan 9 is a better goal than Netware. At least it was designed
> by people aiming for a better Unix rather than people trying to build a
> better DOS, and so is more likely to have a compatible philosophy.
s/aiming.*/with taste that doesn't belong in R'Lyeh/
On Wed, 05 Oct 2005 07:48:12 -0700
Marc Perkel <[email protected]> wrote:
> What is incredibly idiotic is a file system that allws you to delete
> files that you have no write access to. That is stupid beyond belief and
> only the Unix community doesn't get it.
It stops being idiotic as soon as you realize that _deleting_ a
file doesn't involve _writing_ to it in any way. It's not about UNIX,
it's about common sense - try thinking outside of the Netware box for a
sec ;)
Florin
Florin Malita wrote:
>On Wed, 05 Oct 2005 07:48:12 -0700
>Marc Perkel <[email protected]> wrote:
>
>
>>What is incredibly idiotic is a file system that allws you to delete
>>files that you have no write access to. That is stupid beyond belief and
>>only the Unix community doesn't get it.
>>
>>
>
>It stops being idiotic as soon as you realize that _deleting_ a
>file doesn't involve _writing_ to it in any way. It's not about UNIX,
>it's about common sense - try thinking outside of the Netware box for a
>sec ;)
>
>Florin
>
>
What you don't get is that if you don't have rights to write to a file
then you shouldn't have the right to delete the file. Once you get past
the "inside the box" Unix thinking you'll see the logic in this. So what
if the process of deleting a file involves writing to it. That's not
relevant.
--
Marc Perkel - [email protected]
Spam Filter: http://www.junkemailfilter.com
My Blog: http://marc.perkel.com
Al Viro wrote:
>On Wed, Oct 05, 2005 at 08:16:19PM +0100, Nix wrote:
>
>
>>On Wed, 05 Oct 2005, Marc Perkel yowled:
>>
>>
>>>If you want to start thinking about DOING IT RIGHT you need to think
>>>beyond the Unix model and start looking at Netware. Maybe in 5 years
>>>Linux will evolve to where Netware was in 1990.
>>>
>>>
>
>Ugly as hell and thankfully about to go extinct outside of dark places
>where its malignant presense still lingers amidst the Shoggoth droppings?
>
>
>
The big boys are still running Netware because Linux doesn't do the
things they need netware to do. And Netware isn't going away until Linux
has the power to replace it. Linux permissions acient and awkward.
Netware is wonderful. Linux is crippled. Linux needs to modernize.
On Wed, Oct 05, 2005 at 12:44:25PM -0700, Marc Perkel wrote:
> What you don't get is that if you don't have rights to write to a file
> then you shouldn't have the right to delete the file. Once you get past
> the "inside the box" Unix thinking you'll see the logic in this. So what
> if the process of deleting a file involves writing to it. That's not
> relevant.
When a system supports hardlinks, it IS relevant.
So if I decide I want a link to a file like say /etc/group in my home
dir (let us pretend they are on the same partition) so I make a hardlink
to it in my home dir and end up with a file still owned by root (since I
shouldn't be able to add me as owner to the file just by linking to it
after all). Should I now have to go bother the admin about deleting the
file from my home dir if I decide that wasn't really what I wanted? If
I didn't have write permissions to the dir I wouldn't have been able to
make the link in the first place, so since I made it I should be able to
delete it, and I can with the unix way of doing things. I still can't
edit it anymore than I could in the original place since it linked with
the new link to the file having the excact same permissions as the
original. Only someone like root can go chance the owner of a hardlink
to someone else and start setting up some interesting file permissions
using multiple hardlinks to one file.
I suspect you can't do that on netware, so you would have to add
explicit permissions for each user to a single copy of the file instead,
and you would probably want them all to have read/write access but in
fact NOT have delete permissions.
Len Sorensen
On Wed, Oct 05, 2005 at 12:49:12PM -0700, Marc Perkel wrote:
> The big boys are still running Netware because Linux doesn't do the
> things they need netware to do. And Netware isn't going away until Linux
> has the power to replace it. Linux permissions acient and awkward.
> Netware is wonderful. Linux is crippled. Linux needs to modernize.
Netware is a fileserver for windows/dos/os2, linux is an operating
system for running applications. Being able to serve files to other
systems is just one potential application to run on linux but it has
plenty of other things it can do.
People that need netware are people that need a fileserver for their
windows systems. Linux is probably not what they need then.
Len Sorensen
On Wed, 2005-10-05 at 07:48 -0700, Marc Perkel wrote:
[...]
> What is incredibly idiotic is a file system that allws you to delete
> files that you have no write access to. That is stupid beyond belief and
One that allows you to remove read-only files.
> only the Unix community doesn't get it.
Bernd
--
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and Services
On Wed, 5 Oct 2005, Nix wrote:
> On Wed, 05 Oct 2005, Bodo Eggert suggested tentatively:
[...]
> > and if utf-8 filenames are supposed to be used,
> > an utf-8 checker rejecting non-canonialized strings will be desirable
> > to avoid binary trash in filenames or lookalike filenames. The
> > conversion to the canonialized form should happen outside the kernel.
>
> Yes. But where? libc? (I can just see Ulrich going for *this*!)
Either there or in a seperate library.
> > b) ACK, I don't think caseless handling of filenames is a good thing,
> > it would needlessly bloat the kernel by opening a can of worms.
> > E.g. '?' would be converted to 'SS'[0] in German or 'B' in greek.
>
> ... which means that either you lose per-process locale-dependence
> via LANG et all, or you get the possibility of directories containing
> several files with the same name from some users' POV.
>
> Neither seems good to me; even though we already have part of this with
> NTFS, we should not inflict it on people needlessly.
ACK, that's my point.
--
Field experience is something you don't get until just after you need it.
Lennart Sorensen wrote:
>On Wed, Oct 05, 2005 at 12:44:25PM -0700, Marc Perkel wrote:
>
>
>>What you don't get is that if you don't have rights to write to a file
>>then you shouldn't have the right to delete the file. Once you get past
>>the "inside the box" Unix thinking you'll see the logic in this. So what
>>if the process of deleting a file involves writing to it. That's not
>>relevant.
>>
>>
>
>When a system supports hardlinks, it IS relevant.
>
>So if I decide I want a link to a file like say /etc/group in my home
>dir (let us pretend they are on the same partition) so I make a hardlink
>to it in my home dir and end up with a file still owned by root (since I
>shouldn't be able to add me as owner to the file just by linking to it
>after all). Should I now have to go bother the admin about deleting the
>file from my home dir if I decide that wasn't really what I wanted? If
>I didn't have write permissions to the dir I wouldn't have been able to
>make the link in the first place, so since I made it I should be able to
>delete it, and I can with the unix way of doing things. I still can't
>edit it anymore than I could in the original place since it linked with
>the new link to the file having the excact same permissions as the
>original. Only someone like root can go chance the owner of a hardlink
>to someone else and start setting up some interesting file permissions
>using multiple hardlinks to one file.
>
>I suspect you can't do that on netware, so you would have to add
>explicit permissions for each user to a single copy of the file instead,
>and you would probably want them all to have read/write access but in
>fact NOT have delete permissions.
>
>Len Sorensen
>
>
What you don't understand is that Netware's permissions mechanish is
totally different that Linux. A hard link in Netware wouldn't inherit
rights the way Linux does. So the user would have rights to their hard
link to delete that link without having rights to unlink the file.
This is an important concept so pay attention. Linux stores all the
permission to a file with that file entry. Netware doesn't. Netware
calculates effective rights from the parent directories and it is all
inherited unless files or directoies are explicitly set differently. So
if files are added to other people folders then those people get rights
to it automatically without having to go to the second step of changing
the file's permissions.
--
Marc Perkel - [email protected]
Spam Filter: http://www.junkemailfilter.com
My Blog: http://marc.perkel.com
please can we move this discussion _to_ the linux visionaries
mailing list?
i for one would actually like a place where linux kernel developers can
say, instead of "get lost, xxxxhead, you're wasting our time" or
patronise people with IQs > 140 with "go read the kernel newbies faq":
"Hmm, sounds like you have a baad case of linux vision, son.
Go Preach it on LVML, and may you ever love the Penguin."
anyone willing to host such a list at a _real_ server please
speak up now or forever hold your peace.
l.
On Wed, Oct 05, 2005 at 11:54:03AM -0400, Rik van Riel wrote:
> On Tue, 4 Oct 2005, Marc Perkel wrote:
>
> > Marc Perkel
> > Linux Visionary
>
> The problem I have with most visionaries is that while
> they see lots of stuff, they never show anything to the
> rest of the world.
>
> Unless you can explain to other people why your idea
> makes sense, or unless you implement your idea, it won't
> happen.
>
> If the idea is good enough, either of explanation or
> implementation should be enough.
>
> Preaching, OTOH, does not convince people.
>
> --
> All Rights Reversed
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
On Wed, 05 Oct 2005 12:44:25 PDT, Marc Perkel said:
> What you don't get is that if you don't have rights to write to a file
> then you shouldn't have the right to delete the file. Once you get past
> the "inside the box" Unix thinking you'll see the logic in this. So what
> if the process of deleting a file involves writing to it. That's not
> relevant.
Oddly enough, killfiles are *also* based on the concept of removing something
without the need or ability to look at the contents....
*plonk*
(Real-world analogy - if you've *ever* thrown away a box of anything without
opening the box, you understand that you don't need to be able to modify the
box to discard it. The space enclosed by your domicile is basically read/write,
the box can be read-only and still end up in the dumpster. On the other hand,
discarding the back staircase is a lot harder, as the frame of the house is
basically read-only, and you need to make it read-write in order to remove the
attachment points of the staircase. Any Joe Sixpack carpenter understands this
to be entirely intuitive...)
On Wed, Oct 05, 2005 at 01:05:30PM -0700, Marc Perkel wrote:
> What you don't understand is that Netware's permissions mechanish is
> totally different that Linux. A hard link in Netware wouldn't inherit
> rights the way Linux does. So the user would have rights to their hard
> link to delete that link without having rights to unlink the file.
>
> This is an important concept so pay attention. Linux stores all the
> permission to a file with that file entry. Netware doesn't. Netware
> calculates effective rights from the parent directories and it is all
> inherited unless files or directoies are explicitly set differently. So
> if files are added to other people folders then those people get rights
> to it automatically without having to go to the second step of changing
> the file's permissions.
So if you were to moint a partition on /mnt, does the mounted thing now
have to inherit permissions from / and /mnt or from it's own root or
what?
If you chroot something, does it have access to checking permissions
past it's 'virtual' root?
Can users make hardlinks themselves on netware or does an admin have to
do it?
It seems rather inefficient to have to read 8 directories worth of
permissions and calculate them together if you have a file 8 directories
deep compared to doing a single read and compare against a 16bit value +
uid/gid check (ACL of course makes this more complex, but still only
involves reading permissions in one place since inherited permissions
are for create time, not access time).
Unix offers features netware doesn't and some of them require
permissions to work a certain way.
Len Sorensen
On Wed, 5 Oct 2005, Al Viro wrote:
> On Wed, Oct 05, 2005 at 08:16:19PM +0100, Nix wrote:
>> On Wed, 05 Oct 2005, Marc Perkel yowled:
>>> If you want to start thinking about DOING IT RIGHT you need to think
>>> beyond the Unix model and start looking at Netware. Maybe in 5 years
>>> Linux will evolve to where Netware was in 1990.
>
> Ugly as hell and thankfully about to go extinct outside of dark places
> where its malignant presense still lingers amidst the Shoggoth droppings?
>
>> I think Plan 9 is a better goal than Netware. At least it was designed
>> by people aiming for a better Unix rather than people trying to build a
>> better DOS, and so is more likely to have a compatible philosophy.
>
> s/aiming.*/with taste that doesn't belong in R'Lyeh/
>
Excuse me but wasn't Netware the DOS server that started as
a corrupted BSD Unix system, purposely obfuscated to prevent
prying eyes from discovering that it was an unauthorized Unix
variant?
Or was it really a variant of the defunct Xerox XNS that claimed
that they created SPX/IPX?
Cheers,
Dick Johnson
Penguin : Linux version 2.6.13 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.
****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.
Thank you.
On Wed, 05 Oct 2005, Marc Perkel stipulated:
> Nix wrote:
>>On Wed, 05 Oct 2005, Marc Perkel yowled:
>>So, um, what happens to these permissions when you copy a file and put
>>it somewhere else? Do the inherited rights go with it or not? In Unix
>>it's pretty intuitive. In this system there seem to be two right
>>answers, both of which seem... risky from a security perspective.
>
> You inherit the rights of the new directory.
That's a hugely counterintuitive potential security hole in waiting.
Because any number of permissions can be inherited (anything from none
of them, to all of them), this means that upon copying a file *you
cannot reliably tell what its permissions will become* without checking
it.
I don't think I need to point out the flaw in such a scheme.
> Also - under Netware not all permissions are stored with the file. The
> rights are calculated from the file heirachy so you don't store a lot
> of data with each file unless the file has permissions set that is
> different than that of the directory it's in. So moving a file to
> someone's home directory doesn't require any permissions to be set to
> give the user rights to the file.
I consider this a substantial *disadvantage*. Changing permissions should
be something that you have to explicitly do: it shouldn't happen quietly
behind your back merely on account of a rename().
>>/tmp is the problem here, and shows the futility and pointlessness of
>>this feature. If you have an unlistable file in /tmp, *its name is still
>>determinable*, because other users cannot create files with that
>>name. The concept adds *nothing* over some combination of dirs with the
>>execute bit cleared for some set of users and subdirectories which
>>cannot be read by some set of users. There's no need for this profoundly
>>non-Unixlike permission at all. (As usual, ACLs make managing this on
>>a fine-grained scale rather easier.)
>
> It doesn't really make sense to use the /tmp directory the way Unix
> uses it. Why would you want just anyone to even know the names of the
> temporary files you are using. Users should have their own temp
> directory or create their own directory within /tmp
This is what per-process namespaces are for.
> But - to address your question - if there were an invisible (to you)
> file in a directory that you had create rights to then you would get a
> file creation error.
i.e., the filename is *not* invisible, but trivially determinable by a
sufficiently determined attacker; sticking the file in a non-readable
directory is doable *now*, and *already* lacks this weakness.
i.e., the feature is useless.
>>I think Plan 9 is a better goal than Netware. At least it was designed
>>by people aiming for a better Unix rather than people trying to build a
>>better DOS, and so is more likely to have a compatible philosophy.
>>
> I'm not familiar with Plan 9.
Linux seems to be stealing all its good ideas bit by bit, so you'll
know sooner or later. :)
But per-process namespaces are *wonderful*. Goodbye, PATH, for instance:
just mount every binary this user should see in this user's /bin. He
gets his own /tmp, and /home contains *his home directory*.
(Special considerations have to be made for privileged processes: we
don't want the user to be able to fake them out by fooling with /etc,
say, or with their shared libraries. Perhaps setuid programs revert to a
standard namespace, and can see the user's one in
/proc/self/nonprivileged-root or something.)
--
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James Nicoll
Bodo Eggert wrote:
>Marc Perkel <[email protected]> wrote:
>
>[...]
>
>
>>I'll run through a few ideas here.
>>
>>
>
>
>
>>Novell Netware type permissions. ACLs are a step in the right direction
>>but Linux isn't any where near where Novell was back in 1990. Linux lets
>>you - for example - to delete files that you have no read or write
>>access rights to.
>>
>>
>
>It lets you unlink them. That's different from deleting, since the owner
>may have his/her private link to that file.
>
>Unlinking is changing the contents of a directory, and it's controlled by
>the write permission of the containing directory.
>
>
>
There would be different rights to eack link.
>>Netware on the other hand prevents you from deleting
>>files that you can't write to and if you have no right it is as if the
>>file isn't there.
>>
>>
>
>Imagine a /tmp directory (writable by world) with user "a" creating a file
>"foo", umask=077 and user "b" trying to do the same. User "b" will get
>'file exists' if he tries to create it, and 'file does not exist' if he
>tries to list it. He will go nuts.
>
>
Users should have private temp directory space. Two user trying to
create the same file in the same directory isn't going to work under any
operating system.
>BTW: YANI: That about a tmpfs where all-numerical entries can only be
>created by the corresponding UID? This would provide a secure, private
>tmp directory to each user without the possibility of races and denial-of-
>service attacks. Maybe it should be controlled by a mount flag.
>
>
>
>>You can't even see it in the directory. Netware also
>>has inherited permissions like Windows and Samba has and this is doing
>>it right.
>>
>>
>
>You can't do that if you have hardlinks. However, I missed the feature of
>overruling file permissions in some special directories, e.g. anything
>put under /pub should ignore umask and be a+rX.
>
>
You have to realize the Netware does things differently and that Linux
limitations don't apply to Netware.
>
>
>>File systems and individual directories should be able to be
>>flagged as casesensitive/insensitive.
>>
>>
>
>IMHO not needed.
>
>
But just because you don't need it doesn't mean other people don't. If
you are running Samba pretending to be a case insensitive file system
then this is a good feature.
On Wed, 05 Oct 2005, Marc Perkel announced authoritatively:
> If the permissions were don'e right in your own directories your
> inherited rights would give your permissions automatically to your
> home directory and all directories uner it. Netware has a concept
> called an inherited rights mask - something Linux lacks. Windows also
> has rights like this and Samba emulates it. So unless root put files
> in your directory and specifically denied you rights to them, you
> would have full rights to your own directory.
I quite forgot the most grotesque problem with this, which in my opinion
completely eliminates the possibility of inherited permissions on any
even vaguely-POSIX-like system.
What happens if you have a file with two links in directories with
different inheritable permissions set? What are its permissions now?
Surely it doesn't depend on where you happened to open it from!
--
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James Nicoll
On Wed, 05 Oct 2005 13:27:15 -0700
Marc Perkel <[email protected]> wrote:
> There would be different rights to eack link.
Well, color me confused.
You appear to be saying that the permission on a file differ depending
on which link you are accessing it by. Furthermore, your stance seems to
imply that linking to a file grants either write permission or ownership
on the new link.
So, under this permission model, I could link to /etc/passwd in my
home directory, edit the link to change my UID to zero, then relogin to
the system as an administrator.
Not that I would need to, of course, because any user who owns/could
write to a directory would be able to alter any file on the entire
system. I know they're called "permission" models, but that seems
*extremely* permissive...
--
-Julian Blake Kongslie
<[email protected]>
You can delete a directory entry to a file if you have proper
permission to the directory.
You cannot read or write the file if the file doesn't give you permission to.
A hard link makes an additional directory entry to a certain file. You
delete the directory entry to the file, not the file.
And permissions are the same for all instances of the file.
My 2 cents.
On 10/5/05, Julian Blake Kongslie <[email protected]> wrote:
> On Wed, 05 Oct 2005 13:27:15 -0700
> Marc Perkel <[email protected]> wrote:
> > There would be different rights to eack link.
>
> Well, color me confused.
>
> You appear to be saying that the permission on a file differ depending
> on which link you are accessing it by. Furthermore, your stance seems to
> imply that linking to a file grants either write permission or ownership
> on the new link.
>
> So, under this permission model, I could link to /etc/passwd in my
> home directory, edit the link to change my UID to zero, then relogin to
> the system as an administrator.
>
> Not that I would need to, of course, because any user who owns/could
> write to a directory would be able to alter any file on the entire
> system. I know they're called "permission" models, but that seems
> *extremely* permissive...
>
> --
> -Julian Blake Kongslie
> <[email protected]>
>
>
>
--
Bas Westerbaan
http://blog.w-nz.com/
GPG Public Keys: http://w-nz.com/keys/bas.westerbaan.asc
On Wed, 5 Oct 2005 22:51:32 +0200
Bas Westerbaan <[email protected]> wrote:
> You can delete a directory entry to a file if you have proper
> permission to the directory.
>
> You cannot read or write the file if the file doesn't give you
permission to.
>
> A hard link makes an additional directory entry to a certain file. You
> delete the directory entry to the file, not the file.
>
> And permissions are the same for all instances of the file.
>
> My 2 cents.
That is the UNIX model, yes. And I think it makes perfect sense. And as
a side effect, we can delete links to files which we do not own, and
cannot write to.
Does NetWare have an equivalent of hardlinks?
--
-Julian Blake Kongslie
<[email protected]>
On 10/5/05, Marc Perkel <[email protected]> wrote:
>
> What you don't get is that if you don't have rights to write to a file
> then you shouldn't have the right to delete the file. Once you get past
> the "inside the box" Unix thinking you'll see the logic in this. So what
> if the process of deleting a file involves writing to it. That's not
> relevant.
>
No. Listen to what other people are saying. If you have the right to
_create_ the file, you have the right to _delete_ the file. In one
case you are changing the state of where something is stored --
leaving your report on a desk. Changing the contents of the report is
different.
You may not like this paradigm, but it is the Unix/POSIX paradigm and
Linux is not going to suddenly ditch it because it doesn't fit the way
you think permissions should work. As others (and you yourself) have
pointed out, there are other mechanisms in Linux to build additional
policy layers above the basic FS semantics. If you care this much
about the issue, write your own filesystem w/ its own enhanced
permissions scheme, but stop asking the Linux development community to
morph its _Unix-like_ operating system into something else and
insisting everyone who doesn't share your subjective point of view is
ignorant.
Dave
On Wed, 5 Oct 2005, Marc Perkel wrote:
> What you don't get is that if you don't have rights to write to a file
> then you shouldn't have the right to delete the file.
In unix, nobody but the kernel has the right to *delete* a file. Therefore
nobody can delete a file without write permission.
Files are deleted if the last reference is gone. If you play a music file
and unlink it while it's playing, it won't be deleted untill the player
closes the file, since an open filehandle is a reference.
If you like, you can think of it as a kind of instant garbage collection:
Files are the objects referenced by these lists, and if you own
the object, you can change it. However, as long as there is a reference,
you can't destroy it, since this would invalidate all references.
Instead, you must remove all references.
Directories are lists of references, and these lists are independent from
the referenced file-objects. If you own the list, you can change it by
adding or removing files. You can even link files not owned or accessable
by you:
7eggert@be1:~/tmp > ls -l /tmp/foo/foo
---------- 1 7eggert_b users 0 2005-10-05 22:32 /tmp/foo/foo
7eggert@be1:~/tmp > ln /tmp/foo/foo .
7eggert@be1:~/tmp > ls -l
total 0
---------- 2 7eggert_b users 0 2005-10-05 22:32 foo
<snip>
Do you notice the link count in the second column?
Let's remove a link:
<snip>
*switch*
7eggert_b@be1:/tmp/foo> rm foo
rm: remove write-protected regular empty file `foo'? y
*switch*
7eggert@be1:~/tmp > ls -l
total 0
---------- 1 7eggert_b users 0 2005-10-05 22:32 foo
<snip>
As you can see, each directory-owner can independantly unlink the file.
BTW: The owner can change the permissions on the linked file anytime, so
even if you couldn't link non-accessable files, you could end up with
entries in your private directory you could neither access nor delete.
I can also open the file as 7eggert_b, delete it from another tty and
still access it's contents:
<snip>
7eggert_b@be1:/tmp/foo> cat > foo
as
*switch*
be1:/home/7eggert/tmp # rm /tmp/foo/foo
be1:/home/7eggert/tmp # ll /proc/4820/fd/1
l-wx------ 1 7eggert_b users 64 Oct 5 22:43 /proc/4820/fd/1
-> /tmp/foo/foo (deleted)
be1:/home/7eggert/tmp # cat /proc/4820/fd/1
as
*switch*
df
be1:/home/7eggert/tmp # cat /proc/4820/fd/1
as
df
<snip>
As you can see, the directory entry is deleted, but the file is still
there. However, making a hard link from a /proc/pid/fd entry is not (yet?)
possible: "ln: creating hard link `baz' to `/proc/4927/fd/1': Invalid
cross-device link".
--
Fun things to slip into your budget
Not in a budget, but in an annual report:
An employee stole 500,000+. They accounted for it on the annual report as
'involountary employee relations expense'
On Wed, Oct 05, 2005 at 05:51:39PM +0200, Jens Axboe wrote:
> On Wed, Oct 05 2005, Luke Kenneth Casson Leighton wrote:
> > On Wed, Oct 05, 2005 at 03:40:43PM +0200, Jens Axboe wrote:
> > > On Wed, Oct 05 2005, Luke Kenneth Casson Leighton wrote:
> > > > On Wed, Oct 05, 2005 at 02:31:14PM +0200, Jens Axboe wrote:
> > > >
> > > > > [i know better criticism and accusations deleted and not commented on]
> > > >
> > > > > Why is that so hard to understand? Succesful contributions start at the
> > > > > technical level, always have.
> > > >
> > > > then we will have to agree to disagree, because i believe that
> > > > successful contributions start with "what creative thing shall
> > > > we do now / what problem shall we tackle today in a creative
> > > > way?" and work their way down to the technical level, which,
> > > > as you rightly point out, requires successful _technical_
> > > > contributions.
> > >
> > > It's not my opinion, I'm stating historical fact. I honestly can't
> > > remember when a succesfull contribution was made based on a long
> > > non-technical thread. I can, however, remember oodles of cases where the
> > > reverse is quite true.
> >
> > ah.
> >
> > who's hosting the linux-visionaries mailing list, then?
>
> :-)
>
> Don't think there is one, but if it will take the traffic of this list,
> I'm sure davem can be quickly convinced to add one!
davem, can we have a LVML, pleeeease?
introductory text - capitalised of course:
"the LVML - linux visionaries mailing list - is a place where linux
kernel developers can gently but firmly push those people with
'A Vision for Linux' so that they can get on with ...errmm...
developing the linux kernel. it's the place where 'i would love it if'
can be frequently heard and revered, without fear of retribution
or requests for an accompanying kernel patch."
guidelines for posting on LVML:
the list should be full of discussions beginning:
"wouldn't it be great if...", "has anyone considered ...",
"i have an idea". "i am thinking of / considering ..."
with intermediate stages comprising:
"that's a good idea because X but the problems are Y" or
"that is unrealistic because A, but a better or existing way
to do it is B"
concluding in:
"okay, any takers for actually implementing this", "i've
implemented it", "i've started implementing it and really
now need to contact the LKML for a technical brain-picking
session: wish me luck".
the list should _not_ be full of "that's a rubbish idea; don't even
waste our time with it; prove it by implementing it and we won't speak
or listen to you until you _have_ implemented it".
you should really have a reasonable degree of expertise in at least one
of the following areas (please advise LVML if you believe there should
be any additions to this list. the criteria for such items on this
list is there to encourage people with a breadth or depth in
specialist areas of expertise to get invovled. for example, being a
babbling idiot requires a _significant_ amount of talent):
* understanding of the history of kernel development:
unix, NT, netware, beos, mach, l4, gnu/hurd, you name it,
you've dissected it, met the developers, _are_ the developer,
or broken it so badly that people ran away screaming.
* high degree of exposure to a _significant_ number of
different programming languages.
* knowledge, expertise or exposure to a significant number
of futuristic, experimental, modern, ancient, obscure or
archaic hardware platforms, of any form: valves,
relays, transistors, gates, optical, magnetic -
you name it, you should have messed with it and
preferably received an overdose of some kind from it.
* the ability to absorb the contents of several high traffic
free software mailing lists: typically over 1,000 messages
per day. people with adverse side-effects on brain function
of such a high degree of exposure to free software mailing
lists will be welcomed with open arms.
* the ability to converse happily with virtually any free
software developer, with or without beer, at conferences
and user groups, and to be able to make them think - even
if it's to think that you're a babbling idiot.
* talking complete and utter and very convincing nonsense.
* the ability to keep coming up with new ideas despite
clear and unequivocal contradictory statements from
virtually everyone around you that your ideas suck
goats.
you should also have a high degree of tolerance for incoherent babbling
idiots who at least show some some sparks of intelligence underneath.
a full understanding of the linux kernel tree and its history
is optional, except if you are a babbling idiot, under which
circumstances you are expected to have a photographic recall
of the entire linux source tree all the way back to its
derivation.
a liking for or tolerance of reactos, the gnu/hurd or any
other experimental free software operating system is preferred.
how about it?
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
On Wed, Oct 05, 2005 at 03:30:24PM -0400, Lennart Sorensen wrote:
> On Wed, Oct 05, 2005 at 09:23:56AM -0700, Marc Perkel wrote:
> > That's not the point. The point is that Netware has a far superior
> > permission system and I am suggesting the the Linux community learn from
> > it and take advantage of seeing what better looks like and improving itself.
>
> Linux is compatible with unix applications. Netware is not. Supporting
> some useless netware feature at the expense of posix/unix compatibility
> would be insane.
>
> If you can't do it with unix permissions or unix permissions + ACL, you
> don't need to do it at all most likely, and even more likely you
the bastion sftp example i gave which required selinux on top of a much
broader set of POSIX file permissions demonstrates the fallacy of your
statement.
try to achieve the same effect with POSIX - even POSIX ACLs
(uploader only has create and write, not read, not delete;
downloader has read and delete, not write, not create)
and you will fail, miserably, because under POSIX, write implies
create.
l.
On Wed, Oct 05, 2005 at 02:47:27PM -0400, Horst von Brand wrote:
> Luke Kenneth Casson Leighton <[email protected]> wrote:
> > On Wed, Oct 05, 2005 at 01:24:12PM +0400, Nikita Danilov wrote:
> > > Marc Perkel writes:
>
> > > [...]
> > >
> > > > Right - that's Unix "inside the box" thinking. The idea is to make the
> > > > operating system smarter so that the user doesn't have to deal with
> > > > what's computer friendly - but reather what makes sense to the user.
> > > > From a user's perspective if you have not rights to access a file then
> > > > why should you be allowed to delete it?
>
> > > Because in Unix a name is not an attribute of a file.
>
> > there is no excuse.
>
> It's not an excuse, it's part of a coherent view of how things work. Just
> as Netware used to have its, and DOS had its (sort of). As the world view
> underneath Unix, it is darn hard to "fix".
>
> [This discussion sounds quite a lot like it is /you/ who needs "fixing",
> i.e., first wrap your head around Unix' ways...]
asking "ordinary" people to do that is unrealistic: surely you know
that?
i just spent two hours helping a friend who wasn't familiar
with the concept of "give root password for maintenance or
press ctrl-d" they'd been pressing ctrl-d because it said so
and now i'm going to have a 5-hour round-trip journey and possibly
an overnight stay to sort out the mess.
> > selinux has already provided an alternative that is similar to NW
> > file permissions.
>
> Nope. SELinux provides MAC,
yes.
> i.e., mechanisms by which system-wide policy
> (not the random owner of an object) ultimately decides what operations to
> allow.
yes. the concept is not incompatible with what i said: the only bit
that is wrong with what you've said is the word "Nope".
> That is not "file permissions".
part of the coverage of the MAC is file and directory permissions, and
as i mentioned earlier, it so happens that each selinux permission
corresponds, i believe one-to-one, with a function in the dnode and
inode vfs higher-order-function tables in the linux kernel.
example permissions (from postfix.te, policy source version 18):
allow postfix_$1_t { sbin_t bin_t }:dir r_dir_perms;
allow postfix_$1_t { bin_t usr_t }:lnk_file { getattr read };
allow postfix_$1_t shell_exec_t:file rx_file_perms;
i am confident enough with selinux to say that those are file
and directory permissions.
(r_dir_perms is a macro that expands to directory read
permissions { read getattr lock search ioctl }, and
rx_file_perms is a macro that expands to { read getattr lock
execute ioctl })
what this is saying is that postfix_$whatever_t context is
allowed to read the contents of /sbin and /bin; it's also
allowed to know if symlinks in /bin and /usr actually exist,
and also allowed to follow those symlinks; and it's also allowed to
know if shell-scripts exist, and also to read and ultimately
execute them.
> And yes, this is quite un-Unix-like.
this is a good thing.
> [...]
>
> > in what way is it possible for linux to fully support the NTFS
> > filesystem?
>
> If you ask me, preferably not at all, just let that unholy mess quietly go
> the way of the dinosaurs. Sadly, interoperability is required at times,
> so...
*sigh*, tell me about it. well, when reactos gets its NTFS driver, i
will be sure to let you know. i promise :)
l.
> With all due respect (and I do believe some is due); your comment above
> makes no sense.
replying privately. l.
On 5 Oct 2005, Luke Kenneth Casson Leighton murmured woefully:
> On Wed, Oct 05, 2005 at 12:04:01AM +0200, Bodo Eggert wrote:
>> > You can't even see it in the directory. Netware also
>> > has inherited permissions like Windows and Samba has and this is doing
>> > it right.
>>
>> You can't do that if you have hardlinks.
>
> nt 5.0 added hardlinks to ntfs.
Actually they've been present from the start, but only accessible
through the POSIX subsystem and (IIRC) the Backup API (?!!)
--
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James Nicoll
This is the most entertaining thread I've seen for a longest time on
this list. Someone needs to write down the list of suggestions.
> > 1) make namespaces joinable in a sane way
> > 2) wait for the shared subtree patch
> > 3) make pam join the per-user-namespace
> > 4) make pam automount tmpfs on the private /tmp
per-user namespaces on /tmp would solve a lot of the problems faced by
selinux with respect to directory permissions on /tmp/.font-unix
and /tmp/.X11-unix.
there are a lot of legacy apps that no-one wants to modify to get them
to create/read /tmp/x-windows/.X11-unix.
oops.
On Wed, Oct 05, 2005 at 12:38:35PM -0400, Steven Rostedt wrote:
> At least if it moved to another list I'd be strong enough not to
> subscribe. :-)
and i'd feel less guilty about replying oh crap.
On Thu, Oct 06, 2005 at 12:12:49AM +0100, Nix wrote:
> On 5 Oct 2005, Luke Kenneth Casson Leighton murmured woefully:
> > On Wed, Oct 05, 2005 at 12:04:01AM +0200, Bodo Eggert wrote:
> >> > You can't even see it in the directory. Netware also
> >> > has inherited permissions like Windows and Samba has and this is doing
> >> > it right.
> >>
> >> You can't do that if you have hardlinks.
> >
> > nt 5.0 added hardlinks to ntfs.
>
> Actually they've been present from the start, but only accessible
> through the POSIX subsystem and (IIRC) the Backup API (?!!)
*hack* *cough* *choke* someone call 911. *thump* too late.
On Wednesday October 5, [email protected] wrote:
> This is the most entertaining thread I've seen for a longest time on
> this list. Someone needs to write down the list of suggestions.
>
But isn't that exactly what is wrong with this whole thread?
People saying "someone needs to" instead of "I will" or better yet, "I
have"??
NeilBrown
Neil Brown wrote:
>On Wednesday October 5, [email protected] wrote:
>
>
>>This is the most entertaining thread I've seen for a longest time on
>>this list. Someone needs to write down the list of suggestions.
>>
>>
>>
>
>But isn't that exactly what is wrong with this whole thread?
>People saying "someone needs to" instead of "I will" or better yet, "I
>have"??
>
>NeilBrown
>
>
>
Yep. AM should probably do it, then post the finalists and ask for a vote.
Jeff
On 10/5/05, Neil Brown <[email protected]> wrote:
> On Wednesday October 5, [email protected] wrote:
> > This is the most entertaining thread I've seen for a longest time on
> > this list. Someone needs to write down the list of suggestions.
> >
>
> But isn't that exactly what is wrong with this whole thread?
> People saying "someone needs to" instead of "I will" or better yet, "I
> have"??
Or perhaps they just need UTF-8-en_sarcasm so that we can better
detect it in his emails :-).
Dave
On Thu, 6 Oct 2005, Luke Kenneth Casson Leighton said:
> On Thu, Oct 06, 2005 at 12:12:49AM +0100, Nix wrote:
>> On 5 Oct 2005, Luke Kenneth Casson Leighton murmured woefully:
>> > nt 5.0 added hardlinks to ntfs.
>>
>> Actually they've been present from the start, but only accessible
>> through the POSIX subsystem and (IIRC) the Backup API (?!!)
>
> *hack* *cough* *choke* someone call 911. *thump* too late.
I agree, it's barmy. (Cygwin, running under the Win32 subsystem as it
does, manipulates hardlinks via the Backup API.)
I assume the `logic' was that even though hardlinks were only there
for the sake of the mostly-useless POSIX subsystem, you still
might want to back them up...
--
`Next: FEMA neglects to take into account the possibility of
fire in Old Balsawood Town (currently in its fifth year of drought
and home of the General Grant Home for Compulsive Arsonists).'
--- James Nicoll
On Wed, Oct 05, 2005 at 06:21:43AM -0700, Marc Perkel wrote:
>
> Now of you think "outside" the Linux box" you can see where people in
> the real world would expect that if you have no rights to a file to read
> or write to it that you shouldn't be able to delete it. In the outside
> world it's "duh - of course"! but for thouse that are in the "Unix Cult"
> you can't think past inodes.
>
On the contrary. There are read & write permissions, and there is a
separate delete permission. What is so hard to understand about that?
(In unix, "delete permission" is implemented as a write permission on
the directory that file resides in.)
Please get rid of your "real world" notion. There is no need to
assume that delete permission is tied to write permission, just
because _some_ os'es are like that. Some are not that way too,
just get over that.
And there is nothing strange about it, if we move outside the
"computer" box. I have various paper documents with other people's
signatures on them. I am free to shred them if I like (although
that might be bad for business.) But I have no kind of write permission,
altering a signed document is fakery - a criminal activity.
So - permission to delete read-only documents predates electronic
file systems, and is a concept most "real-world" people understand quite well.
Helge Hafting
okay, people: any chance of calming / ending / moving this rather
distracting thread?
davem - any chance of that LVML?
anyone know where dave works who is willing to kick him before
everyone goes nuts?
[everyone but rik & alan hit delete physically or mentally _now_ ...]
rik, alan, i replied privately seeking your input on a post i was
developing in reply to your earlier comments - the ones before this
thread went mad even by my standards.
in amongst the pigeons, there's one pigeon that goes "meow":
i figured you might have missed it: it's got the words
"PLEASE REVIEW" as part of the subject.
sorry to everyone else for having to use the list as a private
communications channel.
l.
From: Luke Kenneth Casson Leighton <[email protected]>
Date: Thu, 6 Oct 2005 01:03:40 +0100
> okay, people: any chance of calming / ending / moving this rather
> distracting thread?
>
> davem - any chance of that LVML?
>
> anyone know where dave works who is willing to kick him before
> everyone goes nuts?
I still don't see any good reason to make a new mailing list.
People argue here because they get a large audience. If we
create a new mailing list, the vast majority of folks won't
take these threads there because they'll get less of the attention
they're after.
Why don't we just kill off this silly thread instead? :-)
Howdy Luke...
Luke Kenneth Casson Leighton wrote:
> > If you can't do it with unix permissions or unix permissions + ACL,
> > you don't need to do it at all most likely, and even more likely
> > you
> the bastion sftp example i gave which required selinux on top of a
> much broader set of POSIX file permissions demonstrates the fallacy
> of your statement.
> try to achieve the same effect with POSIX - even POSIX ACLs (uploader
> only has create and write, not read, not delete; downloader has read
> and delete, not write, not create)
> and you will fail, miserably, because under POSIX, write implies
> create.
You're really muddying up the waters here. sftp is not POSIX. Like ftp,
it presents an abstraction of a generic filesystem, and that abstraction
lives at the application layer. As such, it is the application layer's
responsibility to define what features may or may not be implemented.
There are plenty of smart ftp servers that let you define precisely this
type of ACLs for the ftp users. The fact that it is both possible and
trivially easy to implement such an application on top of a POSIX
runtime environment tells me that there's nothing broken here.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/
David S. Miller wrote:
> From: Luke Kenneth Casson Leighton <[email protected]>
> Date: Thu, 6 Oct 2005 01:03:40 +0100
>
>
>>okay, people: any chance of calming / ending / moving this rather
>>distracting thread?
>>
>>davem - any chance of that LVML?
>>
>>anyone know where dave works who is willing to kick him before
>>everyone goes nuts?
>
>
> I still don't see any good reason to make a new mailing list.
>
> People argue here because they get a large audience. If we
> create a new mailing list, the vast majority of folks won't
> take these threads there because they'll get less of the attention
> they're after.
>
> Why don't we just kill off this silly thread instead? :-)
Hell no, I need something to read while I'm cogitating on work and
waiting for things to compile...any chance we can get all the major
participants into a room and televise it as a cage match?
*popcorn*
Luke Kenneth Casson Leighton <[email protected]> wrote:
> On Wed, Oct 05, 2005 at 02:47:27PM -0400, Horst von Brand wrote:
> > Luke Kenneth Casson Leighton <[email protected]> wrote:
> > > On Wed, Oct 05, 2005 at 01:24:12PM +0400, Nikita Danilov wrote:
> > > > Marc Perkel writes:
[...]
> > > > Because in Unix a name is not an attribute of a file.
> > > there is no excuse.
> > It's not an excuse, it's part of a coherent view of how things work. Just
> > as Netware used to have its, and DOS had its (sort of). As the world view
> > underneath Unix, it is darn hard to "fix".
> >
> > [This discussion sounds quite a lot like it is /you/ who needs "fixing",
> > i.e., first wrap your head around Unix' ways...]
> asking "ordinary" people to do that is unrealistic: surely you know
> that?
And asking ordinary people to understand the (much more complex and opaque)
idea of "inheriting permissions from directories (sometimes, unless...)" is
surely much easier...
> i just spent two hours helping a friend who wasn't familiar
> with the concept of "give root password for maintenance or
> press ctrl-d" they'd been pressing ctrl-d because it said so
> and now i'm going to have a 5-hour round-trip journey and possibly
> an overnight stay to sort out the mess.
Any suggestion for a better message?
[...]
> example permissions (from postfix.te, policy source version 18):
>
> allow postfix_$1_t { sbin_t bin_t }:dir r_dir_perms;
> allow postfix_$1_t { bin_t usr_t }:lnk_file { getattr read };
> allow postfix_$1_t shell_exec_t:file rx_file_perms;
>
> i am confident enough with selinux to say that those are file
> and directory permissions.
OK, now I know for sure you are just an elaborate troll. SELinux is
/harder/ to grasp than Unix permissions, and /requires/ you to grasp them
as foundation.
[...]
> > > in what way is it possible for linux to fully support the NTFS
> > > filesystem?
> >
> > If you ask me, preferably not at all, just let that unholy mess quietly go
> > the way of the dinosaurs. Sadly, interoperability is required at times,
> > so...
>
> *sigh*, tell me about it. well, when reactos gets its NTFS driver, i
> will be sure to let you know. i promise :)
Great. Just keep in mind that time wasted on LKML is time taken away from
NTFS for ReactOS.
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
Marc Perkel <[email protected]> wrote:
[...]
> What you don't understand is that Netware's permissions mechanish is
> totally different that Linux. A hard link in Netware wouldn't inherit
> rights the way Linux does. So the user would have rights to their hard
> link to delete that link without having rights to unlink the file.
OK, so a "hard link" isn't (because it has separate permissions than the
original). Sorry, watered-down symlinks don't cut it. Or just by linking
the file into my place I now have rights to modify it? The later idea makes
my skin try to crawl away...
> This is an important concept so pay attention. Linux stores all the
> permission to a file with that file entry.
You are completely right: This is an extremely central concept to
everything Unix.
> Netware doesn't. Netware
> calculates effective rights from the parent directories and it is all
> inherited unless files or directoies are explicitly set
> differently. So if files are added to other people folders then those
> people get rights to it automatically without having to go to the
> second step of changing the file's permissions.
Which is a very clear explanation of how broken it all is. No wonder
NetWare is no more. Files whose persmissions change depending on which way
you look at them is a nightmare. Sure, you /can/ manage that for small(ish)
setups by brute force, but it soon has to break down.
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
Hi All!
> Time to fix Unix? I doubt something seriously borked would have
> outlasted every other OS on the market. Unix was around before Netware
> and, IMHO, will be around a long time after the last adherents of
> NetWare are gone. (and with MS doing it's level best to kill NetWare
> with it's own shared filesystems and built-in networking this cannot
> be that far off. After all, Netware was developed to fill a vacancy in
> the MS world.)
>
>
>
Respect!
Nikolay N. Ivanov
Horst von Brand wrote:
>Marc Perkel <[email protected]> wrote:
>
>[...]
>
>
>
>>What you don't understand is that Netware's permissions mechanish is
>>totally different that Linux. A hard link in Netware wouldn't inherit
>>rights the way Linux does. So the user would have rights to their hard
>>link to delete that link without having rights to unlink the file.
>>
>>
>
>OK, so a "hard link" isn't (because it has separate permissions than the
>original). Sorry, watered-down symlinks don't cut it. Or just by linking
>the file into my place I now have rights to modify it? The later idea makes
>my skin try to crawl away...
>
>
>
>>This is an important concept so pay attention. Linux stores all the
>>permission to a file with that file entry.
>>
>>
>
>You are completely right: This is an extremely central concept to
>everything Unix.
>
>
>
>> Netware doesn't. Netware
>>calculates effective rights from the parent directories and it is all
>>inherited unless files or directoies are explicitly set
>>differently. So if files are added to other people folders then those
>>people get rights to it automatically without having to go to the
>>second step of changing the file's permissions.
>>
>>
>
>Which is a very clear explanation of how broken it all is. No wonder
>NetWare is no more. Files whose persmissions change depending on which way
>you look at them is a nightmare. Sure, you /can/ manage that for small(ish)
>setups by brute force, but it soon has to break down.
>
>
If you all think Netware is no more you are under an interesting
illusion. Linux being cheap has cut into the little server market - but
if you have thousands of servers all running off the same shared
permissions systems - you just aren't going to do that off of Linux.
--
Marc Perkel - [email protected]
Spam Filter: http://www.junkemailfilter.com
My Blog: http://marc.perkel.com
[Cc: list purged to save people time]
On Wed, Oct 05, 2005 at 08:50:54PM -0700, Marc Perkel wrote:
> If you all think Netware is no more you are under an interesting
> illusion. Linux being cheap has cut into the little server market - but
> if you have thousands of servers all running off the same shared
> permissions systems - you just aren't going to do that off of Linux.
Please will you move those boring threads to another mailing list or
even to usenet ? This is LKML, we're only the 6th of the month and
there are already 1250 messages, 200 of which come from this thread,
and many others coming from other long off-topic threads. 20% noise
is too high and disturbting. It becomes difficult to find someone
talking about subjects related to kernel development !
Thanks in advance
Willy
(NB: I'm not interested in your reply, so please don't Cc: me)
Willy Tarreau wrote:
>[Cc: list purged to save people time]
>
>On Wed, Oct 05, 2005 at 08:50:54PM -0700, Marc Perkel wrote:
>
>
>>If you all think Netware is no more you are under an interesting
>>illusion. Linux being cheap has cut into the little server market - but
>>if you have thousands of servers all running off the same shared
>>permissions systems - you just aren't going to do that off of Linux.
>>
>>
NetWhere? did you say. Oh yeah, that broken down piece of sh_t I rewrote
10 years ago?
The NetWhere-did-its-market-share-go operating system is dead.
>
>Please will you move those boring threads to another mailing list or
>even to usenet ? This is LKML, we're only the 6th of the month and
>there are already 1250 messages, 200 of which come from this thread,
>and many others coming from other long off-topic threads. 20% noise
>is too high and disturbting. It becomes difficult to find someone
>talking about subjects rouelated to kernel development !
>
>
>
You should move it to the "entertainment and trolling" section the Linux
Community. Just add the
word "merkey" somewhere in the body of the text and forward it to the
SCOX message board.
The thread will go on for days, and George Bush, Jimbo Wales, and even
Sollog will probably join
the discussion.
:-)
>Thanks in advance
>Willy
>
>(NB: I'm not interested in your reply, so please don't Cc: me)
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to [email protected]
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at http://www.tux.org/lkml/
>
>
>
On Wednesday 05 October 2005 05:26 am, Luke Kenneth Casson Leighton wrote:
> > Now I certainly wouldn't advocate a Windows-style registry,
> > because I think it's full of obvious problems.
>
> such as? :)
If such a thing were even going to be attempted on UNIX, it would have to be
so different than the NT approach that it would stop looking like a registry
altogether.
For one thing, you need the ability to chroot / have multiple namespaces. It's
totally possible that I as a user decide to install 78 different versions of
Apache for some wild reason, and I expect my configuration settings to stay
separate, damnit!
Also, it would need to be rock-solid. Losing blocks on the disk should not
mean losing the ability to boot the OS. Powering off in the middle of a write
should not be fatal.
What about applications attached to removable media? Should these applications
assume that it is correct to store state in my registry? What happens when I
then try to carry the media to another computer? If I wanted my settings to
migrate, the application would need to be smart enough not to use the
registry implementation, which makes it that much more worthless.
Why implement such cruft in the kernel? A user-space implementation is
perfectly reasonable. An example that comes to mind is gconf, which uses XML
files in a hierarchy to achieve something that looks sort of like a registry.
Indeed, the requirements I briefly touch on above (which are just examples of
the number of considerations I'm sure there really are) all point to an
implementation we already have - a filesystem. Why reinvent the wheel?
Moreover, I think the idea of a system-wide registry is a bad idea altogether.
In environments like GNOME or KDE it's generally OK because your applications
are designed for the environment. Not so in the general Linux environment -
we have tons and tons of applications we're capable of running that don't
have the first clue about registries or why they would want it. So we're
stuck either implementing it in all these applications (effectively forking
them from the versions that used to work on every other UNIX), or they don't
use the registry, in which case we've just added cruft that we don't even use
(at the expense of confusing our end-users, well, to no end), or we choose to
abandon these applications (not going to happen, ever).
KISS comes into play here, I think. A system registry is an interesting
attempt to solve the universal configuration problem, but I think it does not
adapt to the UNIX "way of thinking" well at all.
Regards,
Chase Venters
I haven't written a line of code for this yet on Linux, but I have kept
this design in my head and evolved it over the years in the back of my
head. Wiki's come close in terms of the ideas but that's only a shadow
of what was conceived so long ago. I have watched the industry
patiently for many years on this list, waiting, and I have seen some
elements of this scheme emerge, but have been disappointed since humans
just seem to feel more comfortable inside the box, and not go out of it.
I've worked extensively in the Linux VFS and I/O subsystems and created
companies that have sold for millions of dollars with specialized file
systems with extreme levels of performance. I believe I can finally
pull this off in Linux. This is a big project, and I cannot do it
alone. I also dislike the GPL because it concedes control of something
kernel.org could copyright as "the organization of kernel.org" and
control over the internet and allow healthier businesses to grow around
it. I am proposing not just a file system or clustering, but a unique
architecture and new communication model for the internet for
collaborative sharing at the kernel level with distributed data and
routing facilities and disconnectable operations. It does not resemble
anything that exists today. This FS will not be released under the GPL
but the BSD-mod license on Linux with a kicker that contributors can
retain copyrights. Since binary modules are allowed, any contributor
gets a share of the booty -- Just for fun.
Al Petrofsky and Pamela Jones managed to hornswaggle the Novell
Settlement agreement onto the internet, and it speaks for itself. At
this point, people I think know there are no IP claims related to any
"intallagible knowledge." So now it's time. I have my own NOS, but I
had always wanted to share, which for some reason you folks never get
that part. I am doing this "Just for Fun" at this point and for no
other reason. I dropped the lawsuits and flushed out the turkeys, and
its time to get back to work. This time without the "sword of damacles"
hanging over my head. It's pretty hard to get the whole internet to
hate you, but somehow I succeeded. Now time to pay the dues.
Website is:
en.wikigadugi.org
Post bugs for Linux here.
FTP is:
ftp.wikigadugi.org
Give me a couple of days, and I will be starting on my off time. I
work best alone, but we will see how it goes. I've had more than
enough of the wire brush of enlightenment in the last year, so I think
I'm well on my way to playing better in the sandbox.
White papers will be up in the next several weeks on the wiki. I need
to go over with folks who are interested in building a metropolitan area
network architecture in a collaborative setting the layout and
architecture and it needs to be flushed out. I am doing this for me
alone and just for the fun of building it. This is something Novell
doesn't have, Darl McSwine doesn't have, and M$ doesn't have. Let's get
there first. It will appear FIRST on the Linux Operating System.
Jeff
And yes. this will be open source. The designs will be released under
the GPL Documentation License, but the code will be under something
other than the GPL. I haven't decided just what yet
Jeff
jmerkey wrote:
>
> I haven't written a line of code for this yet on Linux, but I have
> kept this design in my head and evolved it over the years in the back
> of my head. Wiki's come close in terms of the ideas but that's only a
> shadow of what was conceived so long ago. I have watched the industry
> patiently for many years on this list, waiting, and I have seen some
> elements of this scheme emerge, but have been disappointed since
> humans just seem to feel more comfortable inside the box, and not go
> out of it.
>
> I've worked extensively in the Linux VFS and I/O subsystems and
> created companies that have sold for millions of dollars with
> specialized file systems with extreme levels of performance. I
> believe I can finally pull this off in Linux. This is a big project,
> and I cannot do it alone. I also dislike the GPL because it concedes
> control of something kernel.org could copyright as "the organization
> of kernel.org" and control over the internet and allow healthier
> businesses to grow around it. I am proposing not just a file system
> or clustering, but a unique architecture and new communication model
> for the internet for collaborative sharing at the kernel level with
> distributed data and routing facilities and disconnectable
> operations. It does not resemble anything that exists today. This FS
> will not be released under the GPL but the BSD-mod license on Linux
> with a kicker that contributors can retain copyrights. Since binary
> modules are allowed, any contributor gets a share of the booty -- Just
> for fun.
> Al Petrofsky and Pamela Jones managed to hornswaggle the Novell
> Settlement agreement onto the internet, and it speaks for itself. At
> this point, people I think know there are no IP claims related to any
> "intallagible knowledge." So now it's time. I have my own NOS, but
> I had always wanted to share, which for some reason you folks never
> get that part. I am doing this "Just for Fun" at this point and for
> no other reason. I dropped the lawsuits and flushed out the turkeys,
> and its time to get back to work. This time without the "sword of
> damacles" hanging over my head. It's pretty hard to get the whole
> internet to hate you, but somehow I succeeded. Now time to pay the dues.
> Website is:
>
> en.wikigadugi.org
>
> Post bugs for Linux here.
> FTP is:
>
> ftp.wikigadugi.org
>
> Give me a couple of days, and I will be starting on my off time. I
> work best alone, but we will see how it goes. I've had more than
> enough of the wire brush of enlightenment in the last year, so I think
> I'm well on my way to playing better in the sandbox.
> White papers will be up in the next several weeks on the wiki. I need
> to go over with folks who are interested in building a metropolitan
> area network architecture in a collaborative setting the layout and
> architecture and it needs to be flushed out. I am doing this for me
> alone and just for the fun of building it. This is something Novell
> doesn't have, Darl McSwine doesn't have, and M$ doesn't have. Let's
> get there first. It will appear FIRST on the Linux Operating System.
>
> Jeff
>
On Wed, 5 Oct 2005, Marc Perkel wrote:
> What you don't get is that if you don't have rights to write to a file
> then you shouldn't have the right to delete the file. Once you get past
> the "inside the box" Unix thinking you'll see the logic in this. So what
> if the process of deleting a file involves writing to it. That's not
> relevant.
Marc, Get off of it!
In 1987 I first started using Unix. Before that I worked mostly on DOS.
At first the delete w/o ownership shocked me, but then I changed my POV
thinking and found that unlinking from a directory only took write access
to the directory and it made sense to me. This was all before I began to
"think inside the Unix box". In fact, it was when I was "thinking inside
the DOS box".
God I was 19 at the time and I easily got a clue. You are stuck on your
own POV and you cant seem to see anything any other way. So stop trying
to convince us. We can make up our own minds and already have.
-- Steve
On Thu, 06 Oct 2005 00:03:09 BST, Luke Kenneth Casson Leighton said:
> On Wed, Oct 05, 2005 at 02:47:27PM -0400, Horst von Brand wrote:
> > Nope. SELinux provides MAC,
>
> yes.
>
> > i.e., mechanisms by which system-wide policy
> > (not the random owner of an object) ultimately decides what operations to
> > allow.
>
> yes. the concept is not incompatible with what i said: the only bit
> that is wrong with what you've said is the word "Nope".
>
> > That is not "file permissions".
>
> part of the coverage of the MAC is file and directory permissions, and
> as i mentioned earlier, it so happens that each selinux permission
> corresponds, i believe one-to-one, with a function in the dnode and
> inode vfs higher-order-function tables in the linux kernel.
>
> example permissions (from postfix.te, policy source version 18):
>
> allow postfix_$1_t { sbin_t bin_t }:dir r_dir_perms;
> allow postfix_$1_t { bin_t usr_t }:lnk_file { getattr read };
> allow postfix_$1_t shell_exec_t:file rx_file_perms;
>
> i am confident enough with selinux to say that those are file
> and directory permissions.
The part that you managed to miss is that this is MAC - *Mandatory*
Access Control. This means that the *sysadmin* gets to say "this user
can't look at that file" - and there's nothing(*) either the owner of the
file or the user can do about it. There's no chmod or chattr or chacl
command that the owner can issue to let somebody else read it - that's
the whole *point* of MAC.
(*) Well.. almost nothing. The owner *may* be able to copy the contents
of the file to another file that the other user is allowed to read. On the
other hand, the ability to do this would generally indicate a buggy policy....
On Wed, 05 Oct 2005 22:27:03 MDT, jmerkey said:
> anything that exists today. This FS will not be released under the GPL
> but the BSD-mod license on Linux with a kicker that contributors can
> retain copyrights.
Oh no. Not again. I thought we pounded a stake through the heart of this
"relicense the kernel" vampyre....
[email protected] wrote:
>The part that you managed to miss is that this is MAC - *Mandatory*
>Access Control. This means that the *sysadmin* gets to say "this user
>can't look at that file" - and there's nothing(*) either the owner of the
>file or the user can do about it. There's no chmod or chattr or chacl
>command that the owner can issue to let somebody else read it - that's
>the whole *point* of MAC.
>
>(*) Well.. almost nothing. The owner *may* be able to copy the contents
>of the file to another file that the other user is allowed to read. On the
>other hand, the ability to do this would generally indicate a buggy policy....
>
>
Seems to me there is no use taking away the owners ability to chmod,
precisely because the owner always can get around that. (Unless
the owner doesn't even have the right to read his own file.)
You can have a restricted "copy" program, but nothing prevents a
user from making his own copy program, if he can read the file.
Or the user can load the file into the intended app, and "save as"
to some other filename and directory. Or the user can do a screendump,
or even take a picture of the screen.
Company policy may of course forbid the user to bring a camera, just as it
might forbid the user to do "chmod o+r" on important files. I am not
sure that we need the OS to try to enforce such things.
Helge Hafting
El Thu, 6 Oct 2005 00:23:30 +0100,
Luke Kenneth Casson Leighton <[email protected]> escribi?:
> there are a lot of legacy apps that no-one wants to modify to get them
> to create/read /tmp/x-windows/.X11-unix.
What's the point of caring about security for a legacy app if nobody
is going to fix it if a security problema arises?
http://packages.debian.org/unstable/admin/libpam-tmpdir
is good enought IMO
On Thu, Oct 06, 2005 at 01:07:44AM +0100, Luke Kenneth Casson Leighton wrote:
[snip]
> in amongst the pigeons, there's one pigeon that goes "meow":
> i figured you might have missed it: it's got the words
> "PLEASE REVIEW" as part of the subject.
Ohhhh, I think your shift key suddenly started to work...
[snip]
Regards: David Weinehall
--
/) David Weinehall <[email protected]> /) Northern lights wander (\
// Maintainer of the v2.0 kernel // Dance across the winter sky //
\) http://www.acc.umu.se/~tao/ (/ Full colour fire (/
Luke Kenneth Casson Leighton writes:
[...]
> the bastion sftp example i gave which required selinux on top of a much
> broader set of POSIX file permissions demonstrates the fallacy of your
> statement.
Still waiting you to express 0656 with well-formed NT ACLs.
Nikita.
dear mr jmerkley,
your message would be most welcome on a linux visionaries mailing list,
if we can twist enough arms to get one created. my eye is beginning to
twitch at the number of emails with this subject line, knowing that
they're going to the LKML...
> >businesses to grow around it. I am proposing not just a file system
> >or clustering, but a unique architecture and new communication model
> >for the internet for collaborative sharing at the kernel level with
> >distributed data and routing facilities and disconnectable
> >operations.
On Thu, 6 Oct 2005 [email protected] wrote:
> El Thu, 6 Oct 2005 00:23:30 +0100,
> Luke Kenneth Casson Leighton <[email protected]> escribi?:
>
>> there are a lot of legacy apps that no-one wants to modify to get them
>> to create/read /tmp/x-windows/.X11-unix.
>
> What's the point of caring about security for a legacy app if nobody
> is going to fix it if a security problema arises?
>
>
> http://packages.debian.org/unstable/admin/libpam-tmpdir
>
> is good enought IMO
BTW. Also it will be good say something about storing unix sockets by
system programs (/tmp/x-windows/.X11-unix which is owned by root.root) in
/tmp.
http://www.pathname.com/fhs/pub/fhs-2.3.html#REQUIREMENTS14
says about /var/run:
"System programs that maintain transient UNIX-domain sockets must place
them in this directory."
kloczek
--
-----------------------------------------------------------
*Ludzie nie maj? problem?w, tylko sobie sami je stwarzaj?*
-----------------------------------------------------------
Tomasz K?oczko, sys adm @zie.pg.gda.pl|*e-mail: [email protected]*
On Wed, Oct 05, 2005 at 11:06:05PM -0400, Horst von Brand wrote:
> > i just spent two hours helping a friend who wasn't familiar
> > with the concept of "give root password for maintenance or
> > press ctrl-d" they'd been pressing ctrl-d because it said so
> > and now i'm going to have a 5-hour round-trip journey and possibly
> > an overnight stay to sort out the mess.
>
> Any suggestion for a better message?
warnings about "continuing to use your system without
repairing the filesystem will result in further filesystem
damage, ultimately making your system completely unusable.
DO NOT ignore this message and press ctrl-d in order to
continue using your system unless you REALLY know what you
are doing."
that sort of thing would have stopped my friend in their
tracks and got them to phone me.
l.
On Wed, Oct 05, 2005 at 05:14:55PM -0700, David S. Miller wrote:
> From: Luke Kenneth Casson Leighton <[email protected]>
> Date: Thu, 6 Oct 2005 01:03:40 +0100
>
> > okay, people: any chance of calming / ending / moving this rather
> > distracting thread?
> >
> > davem - any chance of that LVML?
> >
> > anyone know where dave works who is willing to kick him before
> > everyone goes nuts?
>
> I still don't see any good reason to make a new mailing list.
>
> People argue here because they get a large audience. If we
> create a new mailing list, the vast majority of folks won't
> take these threads there because they'll get less of the attention
> they're after.
oh go on: i promise i'll even subscribe to it. and lots of other
people will, too [won't we, folks YEERRRRS there you go, dave, see?]
we want the _right_ kind of attention, and _you_ need somewhere to
kindly but firmly push the people who are a bit brighter than kernel
newbies but who still talk 19-to-the-dozen without posting a patch.
i wouldn't even mind if there was an auto-redirect of certain
subject lines duly noted "as of right now, all messages with
this subject are being auto-forwarded to LVML".
> Why don't we just kill off this silly thread instead? :-)
nothing would give me more delight than to see that happen.
pleaaase, dave: you could even set up a redirect of this subject line
to forward the messages to LVML instead.
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
Helge Hafting <[email protected]> wrote:
> [email protected] wrote:
> >The part that you managed to miss is that this is MAC - *Mandatory*
> >Access Control. This means that the *sysadmin* gets to say "this user
> >can't look at that file" - and there's nothing(*) either the owner of the
> >file or the user can do about it. There's no chmod or chattr or chacl
> >command that the owner can issue to let somebody else read it - that's
> >the whole *point* of MAC.
> >
> >(*) Well.. almost nothing. The owner *may* be able to copy the contents
> >of the file to another file that the other user is allowed to read. On the
> >other hand, the ability to do this would generally indicate a buggy policy....
> Seems to me there is no use taking away the owners ability to chmod,
> precisely because the owner always can get around that. (Unless
> the owner doesn't even have the right to read his own file.)
No. The point is that a (correct, complete) policy will prevent the user
from copying the contents to a file with less protection, by any means. No,
I did emphatically /not/ try to imply this is easy to set up (or use).
[...]
> Company policy may of course forbid the user to bring a camera, just as it
> might forbid the user to do "chmod o+r" on important files. I am not
> sure that we need the OS to try to enforce such things.
If you don't trust your (typically fat-fingered) users and sysadmins...
Besides, the point behind the targeted policy in Red Hat/Fedora is to
forbid certain daemons to do nasty stuff. It is an additional protection
against misconfiguration or processes taken over by crackers.
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
Chase Venters wrote:
>On Wednesday 05 October 2005 05:26 am, Luke Kenneth Casson Leighton wrote:
>
>
>>>Now I certainly wouldn't advocate a Windows-style registry,
>>>because I think it's full of obvious problems.
>>>
>>>
>> such as? :)
>>
>>
>
>If such a thing were even going to be attempted on UNIX, it would have to be
>so different than the NT approach that it would stop looking like a registry
>altogether.
>
>
All good points, but perhaps the most compelling to me is that virtually
every successful windows virus out there does its real damage by
modifying the registry to replace key actions, associate bad actions
with good ones and just generally screw the system up...
One could argue that this is no different than a hapless victim running
as root getting his/her /etc/* files modified but:
a. the decentralization there makes it easy to distinguish those files
which have been touched and how to fix them
b. they are all ASCII
c. they are not modified often, most almost never after initial system
config
d. you don't have every app that runs mod'ing those files... (in fact
few are even allowed to)
e. what the !@#$ would I want to cache my most recently visited URLs in
the same place I decide where the entry hooks to my video driver live?
Anyone suggesting that Linux or Unix in general should inherit this,
what I consider, most fatal flaw of all the flaws of windows should be
dealt with harshly...
Sorry, could not resist responding - I cannot count the hours I have
spent searching and clearing registry entries in family and friend's
computers after they downloaded the latest cool virus...
/mike
On Thu, Oct 06, 2005 at 11:53:39AM +0200, [email protected] wrote:
> What's the point of caring about security for a legacy app if nobody
> is going to fix it if a security problema arises?
>
>
> http://packages.debian.org/unstable/admin/libpam-tmpdir
>
> is good enought IMO
That reminds me, gdm hangs (on Debian sid, anyway) when using the
pam_tmpdir.so module. I was meaning to file a bugreport... guess I'd
better get moving.
Agreed. Change the name to "Wiki File System" for the project. Wolf
Mountain might inflame some folks, and it's more like a
Wiki system anyway.
Jeff
Luke Kenneth Casson Leighton wrote:
>dear mr jmerkley,
>
>your message would be most welcome on a linux visionaries mailing list,
>if we can twist enough arms to get one created. my eye is beginning to
>twitch at the number of emails with this subject line, knowing that
>they're going to the LKML...
>
>
>
>>>businesses to grow around it. I am proposing not just a file system
>>>or clustering, but a unique architecture and new communication model
>>>for the internet for collaborative sharing at the kernel level with
>>>distributed data and routing facilities and disconnectable
>>>operations.
>>>
>>>
>
>
>
>
On Wed, Oct 05, 2005 at 11:55:16AM -0400, Lennart Sorensen wrote:
> On Wed, Oct 05, 2005 at 04:42:26PM +0100, Luke Kenneth Casson Leighton wrote:
> > i have no idea. as a user, i just did rm -fr /tmp/* (sorry - not
> > rm -fr /tmp) and it worked.
> >
> > as a user.
> >
> > not root.
>
> Then some admin didn't qualify for root having apparently removed the t
> bit from /tmp making it a world writeable dir. Ouch.
>
> > they weren't dumb enough to give it to me.
>
> But they made /tmp world writeable it seems. Impresive. :)
Silly accidents like that happen. A lazy tarballer in action:
# ls -ld foo
drwxr-xr-x 2 root root 48 Oct 6 17:34 foo
# cd foo
# tar cf ../foo.tar .
And too sleepy root who blindly untars to /tmp
# ls -ld tmp
drwxrwxrwt 2 root root 48 Oct 6 17:34 tmp
# tar xf foo.tar
# ls -ld tmp
drwxr-xr-x 2 root root 72 Oct 6 17:36 tmp
woops.
--
Ragnar Hojland - Project Manager
Linalco "Specialists in Linux and Free Software"
http://www.linalco.com Tel: +34-91-4561700
[email protected] wrote:
>On Wed, 05 Oct 2005 22:27:03 MDT, jmerkey said:
>
>
>
>>anything that exists today. This FS will not be released under the GPL
>>but the BSD-mod license on Linux with a kicker that contributors can
>>retain copyrights.
>>
>>
>
>Oh no. Not again. I thought we pounded a stake through the heart of this
>"relicense the kernel" vampyre....
>
>
>
>
Keep the Linux kernel, this is a just a freebie thrown over the fence.
Jeff
On Thu, Oct 06, 2005 at 12:10:50PM +0100, Luke Kenneth Casson Leighton wrote:
> oh go on: i promise i'll even subscribe to it. and lots of other
> people will, too [won't we, folks YEERRRRS there you go, dave, see?]
So why don't you run it on your own box?
> we want the _right_ kind of attention, and _you_ need somewhere to
> kindly but firmly push the people who are a bit brighter than kernel
> newbies but who still talk 19-to-the-dozen without posting a patch.
Stop insulting kernel newbies. There is a difference between "starting
to learn subject" and "picking buzzwords vaguely related to subject and
playing a Markov chain". And the latter does _not_ mean being brighter,
no matter how much new-age bullshit you spin around it.
On Thu, 6 Oct 2005, Luke Kenneth Casson Leighton wrote:
> rik, alan, i replied privately seeking your input on a post i was
> developing in reply to your earlier comments
> "PLEASE REVIEW" as part of the subject.
I've got better things to do than saving you from yourself.
I replied to your original email in an attempt to educate
some of the other readers on the linux-kernel mailing list.
Private email does not have this same benefit, so there
was no reason for me to reply.
--
All Rights Reversed
On Thu, 06 Oct 2005 11:31:59 +0200, Helge Hafting said:
> You can have a restricted "copy" program, but nothing prevents a
> user from making his own copy program, if he can read the file.
Right, but a properly functioning and sufficiently fascist MAC system won't let
the user create a file in other security contexts that can be read by people
outside that context. See the recent work on supporting MLS (multi-level
security) and MCS (multi-category) in the SELinux tree...
> Or the user can load the file into the intended app, and "save as"
Again, "save as" doesn't create a file the other person can read unless the
other person is authorized.
> to some other filename and directory. Or the user can do a screendump,
You *did* know that there's X hooks for Selinux, so that unauthorized applications
can't snarf the pixels of somebody else's window, right? ;)
> or even take a picture of the screen.
Not much we can do here. On the other hand, it certainly creates a very low
upper limit on how much info you can extract how quickly... ;)
> Company policy may of course forbid the user to bring a camera, just as it
> might forbid the user to do "chmod o+r" on important files. I am not
> sure that we need the OS to try to enforce such things.
No, that is indeed outside the kernel's area.
On Wed, Oct 05, 2005 at 01:04:10PM +0200, Diego Calleja wrote:
> El Wed, 5 Oct 2005 11:26:50 +0100,
> Luke Kenneth Casson Leighton <[email protected]> escribi?:
>
> > > Now I certainly wouldn't advocate a Windows-style registry,
> > > because I think it's full of obvious problems.
> >
> > such as? :)
>
>
> The ugly implementation (inside the kernel and as a big file instead of doing it as a
the nt 3.51 implementation got it right: userspace service (MSRPC
service) with LPC (this is NT, based on Mach, so they have LPC which is
message-passing - joy) communicating from userspace to kernelspace
where necessary.
nooo, it not okay to have registry in kernel. _access_ to it (via
ioctl's) yes. _in_ kernel, friggin'ell'no.
regarding the other points: yes, there's a per-user hive key, which is
"overlaid" onto parts of the sub-tree.
and yes, the previous poster is absolutely right: the benefits cannot
be felt unless evvverrryyy service under the sun is also using it.
... but heck - we do configuration of pretty much every major service
under the sun out of ldap, don't we?
and openldap itself just got the ability to read its own
config out of its own database, right?
it's not _that_ far off, not _that_ unachievable, s/ldap/registry.
there's just a few core services missing - initscripts is a good
example - which nobody's yet had the nerve to tackle (afaik) and dump
into LDAP.
in that example, mostly because there's not much point unless
you're also going to do something decent like put in proper
dependencies (see depinit).
anyway. how on _earth_ did we get here, and please could
someone advise me - and everyone else - of a more suitable
location to discuss these matters?
l.
On Thu, Oct 06, 2005 at 01:23:18PM -0400, Rik van Riel wrote:
> On Thu, 6 Oct 2005, Luke Kenneth Casson Leighton wrote:
>
> > rik, alan, i replied privately seeking your input on a post i was
> > developing in reply to your earlier comments
>
> > "PLEASE REVIEW" as part of the subject.
>
> I've got better things to do than saving you from yourself.
>
> I replied to your original email in an attempt to educate
> some of the other readers on the linux-kernel mailing list.
> Private email does not have this same benefit, so there
> was no reason for me to reply.
oh, right. okay.
i am disappointed by your response.
i will therefore, instead of soliciting your input _before_
sending to the LKLM, such that the quality of information sent
to the list is much higher, now need to go ahead without the
benefit of your advice, such that your input will be required
as a "fait accomplit" in order to educate other LKML readers,
with the unfortunate side-effect of an increased amount
of noise.
should you change your mind on this matter i would be honoured
to receive review comments prior to responding, which will
be in the next few days.
l.
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
On Thu, Oct 06, 2005 at 11:10:55AM -0400, Michael Concannon wrote:
> All good points, but perhaps the most compelling to me is that virtually
> every successful windows virus out there does its real damage by
> modifying the registry to replace key actions, associate bad actions
> with good ones and just generally screw the system up...
the damage is done because "admin" rights are forced out of the control
of the users and sysadmins and into the hands of the dumb-ass app
writers, for both the setup stage and then the actual day-to-day
usage of the app!
the registry on NT has ACLs - which are completely irrelevant as far as
users running as admin are concerned (because the dumb-ass app writers
force them to).
the nt registry - imagine it to be .... _like_ a filesystem, or _like_
an LDAP server.
except with proper ACLs and access controls [which everyone bypasses
because duh it's windows duh, not because it's impossible to do a decent
job with the API and its implementation].
l.
Luke Kenneth Casson Leighton wrote:
>On Thu, Oct 06, 2005 at 11:10:55AM -0400, Michael Concannon wrote:
>
>
>
>>All good points, but perhaps the most compelling to me is that virtually
>>every successful windows virus out there does its real damage by
>>modifying the registry to replace key actions, associate bad actions
>>with good ones and just generally screw the system up...
>>
>>
>
> the damage is done because "admin" rights are forced out of the control
> of the users and sysadmins and into the hands of the dumb-ass app
> writers, for both the setup stage and then the actual day-to-day
> usage of the app!
>
> the registry on NT has ACLs - which are completely irrelevant as far as
> users running as admin are concerned (because the dumb-ass app writers
> force them to).
>
> the nt registry - imagine it to be .... _like_ a filesystem, or _like_
> an LDAP server.
>
> except with proper ACLs and access controls [which everyone bypasses
> because duh it's windows duh, not because it's impossible to do a decent
> job with the API and its implementation].
>
>
No question that one could limit the damage with various tweaks to
permissions and access controls, but it is the very centralization of
information with such vastly disparate purposes (into a single file)
that is the flaw here...
You can view it, think about it and talk about it as a "file-system" and
that is fine.. much like /proc or sysfs, but when the system crashes:
1. It _is_ a file: registry.dat
2. It is a binary file at that...
3. That file has become a dumping ground for everything that every app
thinks is "important" and of course every app writer thinks everything
they write is the most important thing ever - I am sure a have never
done such a thing :-)
I guess you could argue that #3 is the fault of the app writers and not
the architecture, but clearly the current state is the result of those
app writers traveling the path of least resistance, so viewed as a whole
the architecture is to blame regardless... While it may be wrong for
people to steal money left on a table out in front of the bank, the bank
should have known that this would result and put the money inside...
#2 is an issue because of the complexity of the system which must be
function to perform the most basic functions of system recovery...
If you can boot a floppy/pendrive/cd and mount it with vi then it is a
disk in need of service...
If you cannot, it is a brick in need of re-installation...
I have booted linux a number of times with an NT drive as a slave and
recovered it. I have not ever done the inverse...
I hate vi with a passion, more often than not, I have to hit :q! a few
times before I remember what I have to type, but the fact is, it works
when nothing else does and it has saved a lot of systems for me...
#2 is also an issue of security because with very simple and reliable
tools, one can track and monitor changes to key files, one can impose
any level of security with any level of granularity (perhaps too many
grains with SELinux, but that is your choice). Before there was
tripwire, there were lots of people who wrote basically the same thing
in plain simple shell/perl scripts and it worked...
#2 is also an issue of backup and restoration... If it is a
file-system, it does not provide any useful methods of incremental
backup and restoration...
There is no equivalent of:
cd etc/xinet.d ; cvs update -A
/etc _is_ a filesystem with all the benefits that come with it...
/tmp is also a great file-system and a much better place to cache all
that "important" application specific temporary data... If they want to
save state, there is:
/etc/<appname>.conf for site-wide setups
~/.<appname> for user-specific state...
I was trying to stay out this thread - clearly I failed :-) No value
judgement intended for any of the comments made, this thread is a like a
car accident on a busy highway... everyone knows they are slowing
things down by looking, but they cannot look away...
/mike
> l.
>
>
>
>
> I have booted linux a number of times with an NT drive as a slave and
> recovered it. I have not ever done the inverse...
ok, that rambled...
I meant that I did not need to do that with linux... just used a
floppy/cd/usb drive and edited files...
With NT the only way I "recovered" was with a well timed backup of
registry.dat or a binary image of the whole system...
Nothing incremental about it...
/mike
On Thu, Oct 06, 2005 at 04:13:15PM -0400, Michael Concannon wrote:
> 1. It _is_ a file: registry.dat
> 2. It is a binary file at that...
so don't make the implementation a file.
or make the contents _of_ the file available via a file system interface.
quoted for a second time in this thread, this link:
http://www.bindview.com/Services/RAZOR/Utilities/Unix_Linux/ntreg_readme.cfm
todd sabin's linux filesystem device driver, which understands nt
registry fileformat.
NTREG
-----
This is a file system driver for linux, which
understands the NT registry file format. With it,
you can take registry files from NT, e.g., SAM,
SECURITY, etc., and mount them on linux. Currently,
it's read-only, though I may add read-write capability
in the future.
http://www.bindview.com/Resources/RAZOR/Files/ntreg.tar.gz
mentioned for a second time in this thread, the fact that reactos has
read-write capability into nt hive key (registry) format.
l.
On Thu, Oct 06, 2005 at 04:13:15PM -0400, Michael Concannon wrote:
> 1. It _is_ a file: registry.dat
> 2. It is a binary file at that...
> 3. That file has become a dumping ground for everything that every app
> thinks is "important" and of course every app writer thinks everything
> they write is the most important thing ever - I am sure a have never
> done such a thing :-)
s/"that file"/"openldap" and substitute "every app writer"
for "every major free software developer we respect greatly which
can store its data and/or configuration details in an LDAP database"
and your evident distaste for "that file" looks a little like religious
zealotry.
i say that with the greatest respect.
especially when "that file" is actually a database, just like
Berkeley DB (and we all know and _love_ Berkeley DB).
and especially in light it being possible to do a "decent" job, and
make "that file" available via a POSIX filesystem interface.
l.
> I guess you could argue that #3 is the fault of the app writers and not
> the architecture,
yes. i would say it's more to do with the dumb-ass nature of the app
writers, yes. typicall dumb-ass windows app writers give a shit about
security and care greatly about making money hand-over-fist.
whereas on linux it's far less likely for an app writer to
be able to get away with a) making money b) friggin up security. the
distros wouldn't allow an app writer to get away with either.
l.
Luke Kenneth Casson Leighton wrote:
>On Thu, Oct 06, 2005 at 04:13:15PM -0400, Michael Concannon wrote:
>
>
>
>>1. It _is_ a file: registry.dat
>>2. It is a binary file at that...
>>3. That file has become a dumping ground for everything that every app
>>thinks is "important" and of course every app writer thinks everything
>>they write is the most important thing ever - I am sure a have never
>>done such a thing :-)
>>
>>
>
> s/"that file"/"openldap" and substitute "every app writer"
> for "every major free software developer we respect greatly which
> can store its data and/or configuration details in an LDAP database"
> and your evident distaste for "that file" looks a little like religious
> zealotry.
>
>
I don't believe in religion :-) That is not to say I am atheist, just
don't see the distinction between one mythology and the next... but
that really is another thread...
As for your prior comment regarding mounting the registry as a
filesystem, I did see that and made a book mark for the next time I have
to recover an NT box... thanks :-)
However, I am now a little confused though (ok, I am always a little
confused - see comment above on religion - now I am really confused).
If you concede that it need not be a binary file and it need not be
centralized, whether by mounting a "filesystem" or not, then what are we
talking about? That is what we have with /etc /proc /sys?
Assuming you fix the issues of easy of editing it in "dead" filesystems,
binary corruption, permissions and programming style of dumping crap in
there, then I guess I could care less how that is implemented, proc is
already a "virtual filesystem"....
I think someone already mentioned that the issue is that the delta
between an idealized NT registry (which has a few notable hurdles - see
above) and what we have to day is simply a matter of "KISS". What do
you gain from complicating the system that cannot be gained with
visualization tools on top of what is there?
Someone wants XML in /proc? Well, that's just fabulous they can write a
virtual filesytem that accomplishes that on top what is there and leave
the rest of us out of it :-) If what they do becomes indespensible for
a critical mass, then even better, we mount "xmlprocfs" in our future
systems and are fat dumb and happy.
Back to your original point which seemed to be, at least to me, to try
to re-evaluate the portioning problem between
Hardware/Software/Drive/OS/User/Threads. I agree, that the system
appears to be strained and chaotic with all OSes chasing an ever
increasing and impossibly large array of hardware and all-the while the
future is even more complex as it seemingly must be a heavily
parallelized future to compensate for the "end of Moore's law". Given
the hardware in question, though, I am not sure I that I see that Linux
should go to micro-kernels to solve the problem...
<rant>
It seems to me that the driver for "correcting" this is actually closer
to the hardware side... I am flabbergasted as a hardware engineer that
at this point in time with the time elapsed between today and the first
PCs that things have evolved so little...
"drivers" should be the exception not the rule...
Few gadgets architecturally do anything different than anything else in
that class of gadget to really _require_ a driver that could not be
standardized. Gadgets need user-space applications, but with all the
well defined standards we have for talking to devices, there is no
excuse with the wealth of compute power and storage that we have that we
(the entire PC industry) are still shipping hacked, one-off drivers
wither every new gadget...
The same applies to the x86 instruction set - waded through that beast
(well all N volumes of it) recently? WTF? Its as if 199Million of
those 200M gate chips are devoted to obfuscating the user interface....
Most of those bits had a purpose at some point, but we don't seem to be
converging to simpler interface...
I would fire me if ever I even proposed such a horrible design... but
then I don't design x86 processors...
If the CPUs and associated hardware started providing a more pleasant
interface, then I am certain that Linux would respond by taking
advantage of it... We aren't there yet...
</rant>
/mike
> i say that with the greatest respect.
>
> especially when "that file" is actually a database, just like
> Berkeley DB (and we all know and _love_ Berkeley DB).
>
> and especially in light it being possible to do a "decent" job, and
> make "that file" available via a POSIX filesystem interface.
>
> l.
>
>
>
>>I guess you could argue that #3 is the fault of the app writers and not
>>the architecture,
>>
>>
>
> yes. i would say it's more to do with the dumb-ass nature of the app
> writers, yes. typicall dumb-ass windows app writers give a shit about
> security and care greatly about making money hand-over-fist.
>
> whereas on linux it's far less likely for an app writer to
> be able to get away with a) making money b) friggin up security. the
> distros wouldn't allow an app writer to get away with either.
>
> l.
>
>
>
>
On Thu, Oct 06, 2005 at 05:53:40PM -0400, Michael Concannon wrote:
> I think someone already mentioned that the issue is that the delta
> between an idealized NT registry (which has a few notable hurdles - see
> above) and what we have to day is simply a matter of "KISS". What do
> you gain from complicating the system that cannot be gained with
> visualization tools on top of what is there?
it is advocated that storing data in text format is easier
to recover with a minimum of tools.
if, as todd sabin's driver shows, you can present the binary data via a
text-editable filesystem driver, then even _that_ argument goes away.
> Someone wants XML in /proc? Well, that's just fabulous they can write a
> virtual filesytem that accomplishes that on top what is there and leave
> the rest of us out of it :-) If what they do becomes indespensible for
> a critical mass, then even better, we mount "xmlprocfs" in our future
> systems and are fat dumb and happy.
reiserfs already has had an XML plugin written for it, which resulted
in the person it was written for thanking the author for writing the
"best xml database they had ever seen".
think structured storage and streams (which NTFS used to support, and
which explorer.exe in nt 3.51 used to support before
bill/someone-on-high ordered it to be taken out) and now apply that
principle to XML instead.
> Back to your original point which seemed to be, at least to me, to try
> to re-evaluate the portioning problem between
> Hardware/Software/Drive/OS/User/Threads.
yes. seems like ages ago.
> I agree, that the system
> appears to be strained and chaotic with all OSes chasing an ever
> increasing and impossibly large array of hardware and all-the while the
> future is even more complex as it seemingly must be a heavily
> parallelized future to compensate for the "end of Moore's law".
what - the constantly revised one, or the original one?
> Given
> the hardware in question, though, I am not sure I that I see that Linux
> should go to micro-kernels to solve the problem...
i would be delighted to see #ifdef HAVE_L4_MICROKERNEL #endif in the
linux source code.
AS AN OPTION.
that people could choose "bugger that damn l4 trash" or "bugger that
monolithic trash" as they see fit.
and the l4linux source code _is_ there for the linux kernel developers
to pick up and incorporate.
that they have not chosen to do so - even though it is GPL code -
leaves me a little puzzled.
> <rant>
> It seems to me that the driver for "correcting" this is actually closer
> to the hardware side... I am flabbergasted as a hardware engineer that
> at this point in time with the time elapsed between today and the first
> PCs that things have evolved so little...
ah _ha_.
as a hardware engineer, what would _you_ like to see a
modern parallel processor design have - that linux for that
hardware would have an easy job of knocking the stuffing
out of anything remotely attempting to come close to it?
i.e. if you could have the proverbial cart before the
proverbial horse, and could make decisions about the DESIGN
of hardware - all of it - BEFORE it was dumped in your lap
and you were basically impicitly told "we're giving you
this for free and taking advantage of your willingness to go
'cool hardware! let's make it run linux".
we've already had one suggestion: a CAS-n instruction (a SIMD
compare-and-store) which can, according to the person who kindly
suggested it, be used for managing doubly-linked lists.
anyone got any more?
want to send them to me, and i will summarise to those people that
express an interest.
[i _so_ want this thread to die].
> The same applies to the x86 instruction set - waded through that beast
> (well all N volumes of it) recently? WTF?
even _decoding_ the x86 instruction set, due to the massive
number of exceptions / extensions, introduces significant
instruction latency.
a large amount of an x86 compatible processor is spent
translating that crap into a simpler _internal_ microcode set
of instructions.
unberfrigginlievable.
oh, these instructions are too slow. _let's_ go an' add multimedia
extensions. *noooooooo.*
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
>><rant>
>>It seems to me that the driver for "correcting" this is actually closer
>>to the hardware side... I am flabbergasted as a hardware engineer that
>>at this point in time with the time elapsed between today and the first
>>PCs that things have evolved so little...
>>
>>
>
> ah _ha_.
>
> as a hardware engineer, what would _you_ like to see a
> modern parallel processor design have - that linux for that
> hardware would have an easy job of knocking the stuffing
> out of anything remotely attempting to come close to it?
>
> i.e. if you could have the proverbial cart before the
> proverbial horse, and could make decisions about the DESIGN
> of hardware - all of it - BEFORE it was dumped in your lap
> and you were basically impicitly told "we're giving you
> this for free and taking advantage of your willingness to go
> 'cool hardware! let's make it run linux".
>
>
Actually, no... I'd like to see the OS equivalent of a GPU... We know
what the problems of an OS are now... they have not changed for 25
years... and in those 25 years we have gathered an enormous library of
lessons learned...
Let's learn from what we already know and take a step above push/pop/mov
(ok a leap) and stop working around what is now basically 30 years of
crap piled on the 8086 and create some hardware that is actually
"Specific to the Application" rather than the other way around...
/mike
ps. I am doing my part to kill this thread - "just say no" ;-)
>><rant>
>>It seems to me that the driver for "correcting" this is actually closer
>>to the hardware side... I am flabbergasted as a hardware engineer that
>>at this point in time with the time elapsed between today and the first
>>PCs that things have evolved so little...
>>
>>
>
> ah _ha_.
>
> as a hardware engineer, what would _you_ like to see a
> modern parallel processor design have - that linux for that
> hardware would have an easy job of knocking the stuffing
> out of anything remotely attempting to come close to it?
>
> i.e. if you could have the proverbial cart before the
> proverbial horse, and could make decisions about the DESIGN
> of hardware - all of it - BEFORE it was dumped in your lap
> and you were basically impicitly told "we're giving you
> this for free and taking advantage of your willingness to go
> 'cool hardware! let's make it run linux".
>
>
Actually, no... I'd like to see the OS equivalent of a GPU... We know
what the problems of an OS are now... they have not changed for 25
years... and in those 25 years we have gathered an enormous library of
lessons learned...
Let's learn from what we already know and take a step above push/pop/mov
(ok a leap) and stop working around what is now basically 30 years of
crap piled on the 8086 and create some hardware that is actual "Specific
to the Application" rather than the other way around...
/mike
ps. I am doing my part to kill this thread - "just say no" ;-)
Marc Perkel wrote:
> Right - that's Unix "inside the box" thinking. The idea is to make the
> operating system smarter so that the user doesn't have to deal with
> what's computer friendly - but reather what makes sense to the user.
> From a user's perspective if you have not rights to access a file then
> why should you be allowed to delete it?
then that file shouldn't be in a directory owned by $otheruser.
> Now - the idea is to create choice. If you need to emulate Unix nehavior
> for compatibility that's fine. But I would migrate away from that into a
> permissions paradygme that worked like Netware.
the word is 'paradigm.' anyhow, through posix acls and selinux you can
achieve the behaviour you so love.
> I started with Netware and I'm spoiled. They had it right 15 years ago
> and Linux isn't any where near what I was with Netware and DOS in 1990.
> Once you've had this kind of permission power Linux is a real big step
> down.
if you like netware so much, then fucking use it. nobody here will stop you.
> So - the thread is about the future so I say - time to fix Unix.
UNIX isn't broken. you're just not asking it to do the right things.
Marc Perkel wrote:
>> It would be an incredibly idiotic system that auto hides files just
>> because you can't use them. We have ways to hide files in user space
>> for the convinience of users. It would be too inconvinient for
>> applications if the OS hid files on us.
>>
>> Len Sorensen
>
> What is incredibly idiotic is a file system that allws you to delete
> files that you have no write access to. That is stupid beyond belief and
> only the Unix community doesn't get it.
A: You're stupid!
B: No, You're stupid!
A: My dad can beat up your dad!
B: No, my dad can beat up yours!
A: My penis is bigger than yours!
B: No, my penis is bigger!
A: (ad infinium)
Look, for those of you who love their beloved netware, go back and use
it. You don't like the way the unix filesystems work because you don't
UNDERSTAND how they work. You're talking about a bunch of special-case
stuff here -- POSIX ACLs and SELinux are your answer. Use them. But for
christ's sake, STOP BITCHING ABOUT NETWARE.
Marc Perkel wrote:
> This stuff I'm talking about is not theoretical. It's been in Novell
> Netware since 1990 and it works great. Netware with DOS in 1990 is still
> far superior to Linux today. Once you've had Netware - Linux is
> laughable. All youhave to do is look ate Netware and copy it. Or the
> mars-nwe netware emulator for that matter. The code to do this already
> exists.
If Linux is "laughable", and netware is "far superior", then get your
ass back on a netware box and stop complaining. If you like it so much,
use it. If you don't like the way linux works, don't use it. End of story.
http://www.eet.com/in_focus/embedded_systems/OEG20021213S0029
It was decided at the beginning that we would design
a system-on-chip (SoC) platform, which yields the
best unit price when manufactured in high volume. The
usual approach would be to license all the technology
from third party suppliers, [...] [but] we didn't
want to deal with huge NRE and royalty fees. Also,
we would not get the necessary know-how that is often
a determining factor when designing a new product in
today's ever decreasing time-to-market.
So, we decided to follow the Linux open source
philosophy and build our first platform on open
source technology. We took several open source IPs
from OpenCores [http://www.opencores.org] and integrate
them into an underlying hardware SoC platform optimized
for running Linux.
[...]
... As the main processor we chose OpenRISC 1200
[http://www.opencores.org/pnews.cgi/list/or1k?no_loop=yes],
a 32-bit RISC processor that comes with a stable GNU
Toolchain and a port of small footprint Microcontroller
Linux, the uClinux.
The next critical part of the whole project was to set up a
scheme on how to connect all the IPs in a modular way so that
we could configure the platform for different embedded
applications. [...]
We found out that a central configurable block
interconnecting the processor and peripheral IPs did
the trick. [...] we created a tool that automatically
generated this central block [...] and automatically
configures Linux kernel and device drivers for that
particular application.
As embedded developers often find out, it is difficult
to start writing and testing software, if hardware
designers are still designing the hardware. It is
necessary to parallel these two tasks in order to meet
^^^^^^^^^^^^^^^^^^^^^^^^
today's critical time-to-market schedules. In addition,
each group can provide some test cases to the other
as we found out.
these people designed the _entire_ embedded system - from free software
licensed components.
the processor design.
the software toolchain.
the kernel running on the free software licensed processor design.
it CAN be done. it HAS been done.
convincing yourselves that you "must have hardware before you will get
off your fat asses" is _so_ self-disempowering. STOP IT.
you - the linux kernel designers - are an extremely powerful
group who quite literally could hold the technical world to
ransom if you so chose (albeit for a very brief amount of time until
someone considered your actions to be the equivalent of a
"bus-running-over" event).
god help the world when you decide to actually say "thank you
for your hardware. next time, consult us on what should be
in it _before_ you finalise its design".
l.
Luke Kenneth Casson Leighton wrote:
>
> ... but heck - we do configuration of pretty much every major service
> under the sun out of ldap, don't we?
>
> and openldap itself just got the ability to read its own
> config out of its own database, right?
>
> it's not _that_ far off, not _that_ unachievable, s/ldap/registry.
I think both you and Michael have interesting points on this topic. Fyi,
OpenLDAP now has dynamic config (accessible/modifiable via LDAP) but the
backing store is still a bunch of flat files. There were two objectives
here - (1) make every knob tunable via LDAP, and (2) don't prevent an
admin from fixing things with vi if they have to. I've spent too many
times rescuing systems in single-user mode with only /bin/sh, to ever
commit to using a binary config database.
Yes, KISS is a good policy, you just have to understand what the 'It' is
that you're talking about in each instance. Putting a filesystem driver
on top of a registry.dat file seems to provide a simple user interface,
so it *looks* like you're adhering to KISS, but the innards are still
both complex and fragile. Hell, even the simplest filesystem driver you
can write is a couple hundred lines of code.
The LDAP-enabled config engine in OpenLDAP looks more structured / more
complex than the old flat slapd.conf file, but under the covers it's all
still plain text. In one case, you're taking something very complex and
putting a simple cover on it, in the other you have very simple building
blocks and put a complex / richer interface on top of it. Guess which
design is more likely to keep functioning in the face of a system failure.
The Unix programming philosophy is about taking small simple tools and
combining them to perform more complex tasks. You could say it's one of
the world's earliest object-oriented UIs. If you don't keep that in
mind, and try to build complexity in starting at your most basic
building blocks on the bottom (i.e., the kernel) then you're going to
have a nightmare trying to keep anything you build on top of it working.
One only need look at MS Windows to see how true this is.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/
Luke Kenneth Casson Leighton wrote:
> the bastion sftp example i gave which required selinux on top of a much
> broader set of POSIX file permissions demonstrates the fallacy of your
> statement.
>
> try to achieve the same effect with POSIX - even POSIX ACLs
> (uploader only has create and write, not read, not delete;
> downloader has read and delete, not write, not create)
>
> and you will fail, miserably, because under POSIX, write implies
> create.
you, however, seem to be missing the point that these are special
circumstances. in 99% of all cases, regular unix file permissions are
sufficient. when you start needing special silly permissions for things
like this, we have special silly tools to accommodate you. Use them.
Deal with it.
adding a permissions schema similar to that found in windows/netware
would only unneccessarily complicate things, and most likely end up
breaking everything.
bottom line: if you want to see support like this in linux, write a
filesystem with these capabilities built-in. If you don't want to/can't
write it, then stop complaining and continue to use netware (read: shit
or get off the pot).
On Fri, Oct 07, 2005 at 01:38:41AM +0100, Luke Kenneth Casson Leighton wrote:
> self-disempowering
*plonk*
On Thu, Oct 06, 2005 at 08:22:20PM +0100, Luke Kenneth Casson Leighton wrote:
> On Thu, Oct 06, 2005 at 01:23:18PM -0400, Rik van Riel wrote:
> > On Thu, 6 Oct 2005, Luke Kenneth Casson Leighton wrote:
> >
> > > rik, alan, i replied privately seeking your input on a post i was
> > > developing in reply to your earlier comments
> >
> > > "PLEASE REVIEW" as part of the subject.
> >
> > I've got better things to do than saving you from yourself.
> >
> > I replied to your original email in an attempt to educate
> > some of the other readers on the linux-kernel mailing list.
my first reaction to your message was one of disappointment: my initial
reply was therefore rather formal and curt, and i did not think much
about this bit of your message.
after some thought, i wanted to make it clear that i would
welcome such education just as much as anyone else would.
l.
On Wednesday 05 October 2005 22:30, Marc Perkel wrote:
> >So, um, what happens to these permissions when you copy a file and put
> >it somewhere else? Do the inherited rights go with it or not? In Unix
> >it's pretty intuitive. In this system there seem to be two right
> >answers, both of which seem... risky from a security perspective.
> >
> >
> You inherit the rights of the new directory.
>
> Also - under Netware not all permissions are stored with the file. The
> rights are calculated from the file heirachy so you don't store a lot of
> data with each file unless the file has permissions set that is
Since when 16 bits is a lot of data?
> different than that of the directory it's in. So moving a file to
> someone's home directory doesn't require any permissions to be set to
> give the user rights to the file.
>
> >/tmp is the problem here, and shows the futility and pointlessness of
> >this feature. If you have an unlistable file in /tmp, *its name is still
> >determinable*, because other users cannot create files with that
> >name. The concept adds *nothing* over some combination of dirs with the
> >execute bit cleared for some set of users and subdirectories which
> >cannot be read by some set of users. There's no need for this profoundly
> >non-Unixlike permission at all. (As usual, ACLs make managing this on
> >a fine-grained scale rather easier.)
> >
> It doesn't really make sense to use the /tmp directory the way Unix uses
> it. Why would you want just anyone to even know the names of the
> temporary files you are using. Users should have their own temp
> directory or create their own directory within /tmp
Then use ~/tmp in your homedir. What's the problem?
> >You *do* realise just how incapable the Windows permission-management
> >GUI is, don't you? Any OS where the command-line tools hide half
> >the permissions model and the GUI hides a slightly different half,
> >and where looking at a set of permissions and hitting cancel can
> >*change* those permissions drastically, is not sane.
> >
> >
> That's why I'm pushing netware as a model rather than windows. But
> Windows file permissions are superior to Linux.
>
> >(Disclaimer: the last time I bothered to verify the latter behaviour
> >was in NT4. Maybe they've partially fixed it.)
>
> One place where Windows wins over Linux is in the "easy to use"
> category. Something the Linux world should look ast.
"Easy to use" to whom?
Command line tools for changing permissions under Windows are anything
but "easy to use". They are also feature incomplete. Tools for changing
ownership do not exist at all. Tools for changing permissions/ownership
on registry do not exist also. Maybe third party ones exist, dunno.
Every one that I saw on the Inet, was either buggy, incomplete, or both.
For admins, this is a huge hole in usability.
> I am a Linux supporter and love it. I'm saying this to help make it better.
--
vda
On Thu, Oct 06, 2005 at 10:20:01PM +0100, Luke Kenneth Casson Leighton wrote:
> On Thu, Oct 06, 2005 at 04:13:15PM -0400, Michael Concannon wrote:
>
> > 1. It _is_ a file: registry.dat
> > 2. It is a binary file at that...
> > 3. That file has become a dumping ground for everything that every app
> > thinks is "important" and of course every app writer thinks everything
> > they write is the most important thing ever - I am sure a have never
> > done such a thing :-)
>
> s/"that file"/"openldap" and substitute "every app writer"
> for "every major free software developer we respect greatly which
> can store its data and/or configuration details in an LDAP database"
> and your evident distaste for "that file" looks a little like religious
> zealotry.
>
Well, there are many differences between the windows registry and
openldap on unix. First, openldap is voluntary. There are plenty
of linux systems not using ldap at all, and still using whatever apps
they want. People deploy ldap when they think the benefits outweighs
the added complexity. :-)
Helge Hafting
helge, yours is the kind of comment which is welcome on linux visionaries and
i wish, i wissshhh, for the thread on lkml with this subject to DIEEEE.
l.
On Sun, Oct 09, 2005 at 12:27:09AM +0200, Helge Hafting wrote:
> On Thu, Oct 06, 2005 at 10:20:01PM +0100, Luke Kenneth Casson Leighton wrote:
> > On Thu, Oct 06, 2005 at 04:13:15PM -0400, Michael Concannon wrote:
> >
> > > 1. It _is_ a file: registry.dat
> > > 2. It is a binary file at that...
> > > 3. That file has become a dumping ground for everything that every app
> > > thinks is "important" and of course every app writer thinks everything
> > > they write is the most important thing ever - I am sure a have never
> > > done such a thing :-)
> >
> > s/"that file"/"openldap" and substitute "every app writer"
> > for "every major free software developer we respect greatly which
> > can store its data and/or configuration details in an LDAP database"
> > and your evident distaste for "that file" looks a little like religious
> > zealotry.
> >
> Well, there are many differences between the windows registry and
> openldap on unix. First, openldap is voluntary. There are plenty
> of linux systems not using ldap at all, and still using whatever apps
> they want. People deploy ldap when they think the benefits outweighs
> the added complexity. :-)
>
> Helge Hafting
>
>
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--
Due to wiki based DOS attacks on this Linux project server, the wiki
server and ftp server are password protected at wolfmountaingroup.org.
Anyone interested in testing the code on Linux or helping with the Linux
design please email me and we will review your request and forward to
the project members for inclusion. Either myself or another admin can
grant you access. Please place in the subject of the email [WOLF
MOUNTAIN] so we know it's someone asking for access to this server.
Thanks,
Jeff
