2002-08-16 20:58:26

by Martin J. Bligh

[permalink] [raw]
Subject: IDE?

So did Linus get disk corruption or is something else afoot?

9 hours axboe 1.456.34.40 Delete 2.5 IDE core
9 hours axboe 1.456.34.41 Add 2.4 IDE core, based on late 2.4.19-pre-acX version


2002-08-16 21:24:30

by Linus Torvalds

[permalink] [raw]
Subject: Re: IDE?

In article <2444170000.1029531611@flay>,
Martin J. Bligh <[email protected]> wrote:
>So did Linus get disk corruption or is something else afoot?

Martin gave up the fight he had to do all the time, so..

Linus

2002-08-16 22:58:42

by Marc-Christian Petersen

[permalink] [raw]
Subject: Re: IDE?

Hi Linus,

> In article <2444170000.1029531611@flay>,
> Martin J. Bligh <[email protected]> wrote:
> > So did Linus get disk corruption or is something else afoot?
> Martin gave up the fight he had to do all the time, so..

I am beside my self with laughing, sorry :P

I really can imagine what are you dreaming of. Like:
"shit, f*ck, why the hell I kicked Andr? Hedrick in the ass
and why, for heaven's sake, I said jump in the lake to him?!!?"

Sorry, couldn't resist. ;)


--
Kind regards
Marc-Christian Petersen

http://sourceforge.net/projects/wolk

PGP/GnuPG Key: 1024D/569DE2E3DB441A16
Fingerprint: 3469 0CF8 CA7E 0042 7824 080A 569D E2E3 DB44 1A16
Key available at http://www.keyserver.net. Encrypted e-mail preferred.

2002-08-16 23:27:04

by Linus Torvalds

[permalink] [raw]
Subject: Re: IDE?


On Sat, 17 Aug 2002, Marc-Christian Petersen wrote:
>
> I am beside my self with laughing, sorry :P
>
> I really can imagine what are you dreaming of. Like:

Actually, you apparently can't.

I'm dreaming of an IDE maintainer that people (including, very much, me)
can work with. I don't know why, but IDE has pretty much since day one
been a fairly problematic area, and has caused a lot more maintainer
headache than the rest of the kernel put together..

There's been one fairly smooth IDE transition (the original transition
from hd.c to ide.c), and calling even that "smooth" is pretty much all
hindsight - at the time people thought it was horribly stupid to not allow
big controversial changes to hd.c, and the resulting code duplication was
considered a disaster.

Right now it looks like Alan is at least for the moment willing to work on
the IDE code, which is obviously great. I just wonder how long he'll stand
it (he's maintained various IDE buglists etc issues for years, so we can
hope).

Linus

2002-08-16 23:32:50

by Larry McVoy

[permalink] [raw]
Subject: Re: IDE?

On Fri, Aug 16, 2002 at 04:34:14PM -0700, Linus Torvalds wrote:
> I'm dreaming of an IDE maintainer that people (including, very much, me)
> can work with. I don't know why, but IDE has pretty much since day one
> been a fairly problematic area, and has caused a lot more maintainer
> headache than the rest of the kernel put together..

This may be politically incorrect, but could you (or anyone) provide a
history of the IDE maintainers to date and why they didn't work out
and what would need to change to make them work out? I'm sticking my
nose in where I know nothing but maybe one of them could rise up to
the necessary level if it were spelled out what that level was.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm

2002-08-16 23:55:18

by Linus Torvalds

[permalink] [raw]
Subject: Re: IDE?


On Fri, 16 Aug 2002, Larry McVoy wrote:
> On Fri, Aug 16, 2002 at 04:34:14PM -0700, Linus Torvalds wrote:
> > I'm dreaming of an IDE maintainer that people (including, very much, me)
> > can work with. I don't know why, but IDE has pretty much since day one
> > been a fairly problematic area, and has caused a lot more maintainer
> > headache than the rest of the kernel put together..
>
> This may be politically incorrect, but could you (or anyone) provide a
> history of the IDE maintainers to date and why they didn't work out
> and what would need to change to make them work out?

I actually don't think it's the people as much as it is the ridiculous
linkages inside ide.c and the hugely complicated rules. The code is messy.

The network drivers have various setups that share the same chipset, but
there they tend to be individual drivers that just share helper routines.
Each driver still does their own PCI driver registration etc. In contrast,
when it comes to IDE, you're supposed to be an IDE driver first, and a PCI
chipset driver second, and that putting o fthe cart before the horse
results in problems.

Even something as simple as a PIIX driver (which _should_ just register
itself as a driver for the piix chipsets) doesn't do that. Instead, we
have ide-pci.c, which has a list of all the chipsets it knows about, and
then does initialization and calls the init routines that it knows about.
That's just incestuous.

And we all know where incest leads. Hereditary insanity.

Linus

2002-08-16 23:57:13

by Anton Altaparmakov

[permalink] [raw]
Subject: Re: IDE?

On Fri, 16 Aug 2002, Linus Torvalds wrote:
> I'm dreaming of an IDE maintainer that people (including, very much, me)
> can work with. I don't know why, but IDE has pretty much since day one
> been a fairly problematic area, and has caused a lot more maintainer
> headache than the rest of the kernel put together..
>
> There's been one fairly smooth IDE transition (the original transition
> from hd.c to ide.c), and calling even that "smooth" is pretty much all
> hindsight - at the time people thought it was horribly stupid to not allow
> big controversial changes to hd.c, and the resulting code duplication was
> considered a disaster.
>
> Right now it looks like Alan is at least for the moment willing to work on
> the IDE code, which is obviously great. I just wonder how long he'll stand
> it (he's maintained various IDE buglists etc issues for years, so we can
> hope).

Linus,

Out of curiosity, who is going to be IDE 2.5 kernel maintainer now?

I am assuming you still maintain that working with Andre is difficult for
you...

Further, I am assuming that Alan is not going to be wanting to be IDE 2.5
maintainer. (I may be completely wrong of course... Alan?)

Jens perhaps? Again I assume he doesn't want the job. (Again, I may be
completely wrong... Jens?)

How about Bartlomiej Zolnierkiewicz as IDE 2.5 maintainer? He seems to be
happy to talk to Andre. And you are perhaps able to work with him well?
He has been submitting bug fixes to Martin and seems to generally know
what he is doing with IDE, certainly a lot better than many of us...

If you can work with him, then it would seem he would be well suited for
the job... Assuming he wants it... Bartlomiej?

Just my curious 2p...

Best regards,

Anton
--
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.openprojects.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

2002-08-16 23:59:16

by Linus Torvalds

[permalink] [raw]
Subject: Re: IDE?


On Fri, 16 Aug 2002, Linus Torvalds wrote:
>
> I actually don't think it's the people as much as it is the ridiculous
> linkages inside ide.c and the hugely complicated rules. The code is messy.

Note: it _is_ the people too, don't get me wrong. But in other areas we
have people like Al Viro, who can drive grown men to cry (and drink) with
his not-very-polite postings. And in those areas it hasn't been a huge
problem, even though some people probably take a valium or two before they
dare open emails from Al.

So the messiness and interconnectedness of the IDE layer just seems to
bring the people problem to a sharp and ugly point. The absolute lack of
communication skills wrt IDE among the people who have worked on it has
been stunning, and that probably _is_ because the code is so hard to even
talk about.

Linus

2002-08-17 00:05:15

by Larry McVoy

[permalink] [raw]
Subject: Re: IDE?

> I actually don't think it's the people as much as it is the ridiculous
> linkages inside ide.c and the hugely complicated rules. The code is messy.

So of all the people you know floating around in "active hacker" state, who
seems like the sort of person who could handle this mess? If there is no
person, is there a description which is more specific than "wanted: person
who wants thankless abusive non-payed job to clean up what is inherently a
mess"?

People are pretty amazing and I've found if you spell out exactly what
you are looking for you can sometimes get it. People tend to step
up to the plate more willingly if they know what is expected of them.
You are the master of getting people to do that, so this is sort of the
blind suggesting a path to the sighted, but this area seems harder than
normal so maybe it can benefit from a more defined description. Dunno.
Can't hurt. I hate to see more crap piled on Alan, he's a finite and
useful resource. Seems like spreading the load around would be good.

Another question: is there any operating system out there which handles
the IDE mess well? I suppose Windows is as good as it gets, right? Or
does Solaris/x86 do a good job? If so, maybe we can get them to donate
their work to Linux, since they seem to be looking for "we're good guys"
press coverage.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm

2002-08-17 00:03:41

by Linus Torvalds

[permalink] [raw]
Subject: Re: IDE?


On Sat, 17 Aug 2002, Anton Altaparmakov wrote:
>
> Out of curiosity, who is going to be IDE 2.5 kernel maintainer now?

Well, as I implied, Alan seems to be not completely unwilling to work on
it, and unlike me he _can_ interact with Andre most of the time. Possibly
Jens will do the 2.5.x side, of it (with Alan working on 2.4), but we've
not talked it through.

I'd like Vojtech to be a bit involved too, he seemed to do some
much-needed cleanups for PIIX4 IDE (now gone, since we couldn't save just
those parts..)

Linus

2002-08-17 00:03:28

by Anton Altaparmakov

[permalink] [raw]
Subject: Re: IDE?

[Apologies for the repost but I got Alan's address wrong first time
round... silly cut'n'paste error...]

On Fri, 16 Aug 2002, Linus Torvalds wrote:
> I'm dreaming of an IDE maintainer that people (including, very much, me)
> can work with. I don't know why, but IDE has pretty much since day one
> been a fairly problematic area, and has caused a lot more maintainer
> headache than the rest of the kernel put together..
>
> There's been one fairly smooth IDE transition (the original transition
> from hd.c to ide.c), and calling even that "smooth" is pretty much all
> hindsight - at the time people thought it was horribly stupid to not allow
> big controversial changes to hd.c, and the resulting code duplication was
> considered a disaster.
>
> Right now it looks like Alan is at least for the moment willing to work on
> the IDE code, which is obviously great. I just wonder how long he'll stand
> it (he's maintained various IDE buglists etc issues for years, so we can
> hope).

Linus,

Out of curiosity, who is going to be IDE 2.5 kernel maintainer now?

I am assuming you still maintain that working with Andre is difficult for
you...

Further, I am assuming that Alan is not going to be wanting to be IDE 2.5
maintainer. (I may be completely wrong of course... Alan?)

Jens perhaps? Again I assume he doesn't want the job. (Again, I may be
completely wrong... Jens?)

How about Bartlomiej Zolnierkiewicz as IDE 2.5 maintainer? He seems to be
happy to talk to Andre. And you are perhaps able to work with him well?
He has been submitting bug fixes to Martin and seems to generally know
what he is doing with IDE, certainly a lot better than many of us...

If you can work with him, then it would seem he would be well suited for
the job... Assuming he wants it... Bartlomiej?

Just my curious 2p...

Best regards,

Anton
--
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.openprojects.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/


2002-08-17 00:15:03

by Rik van Riel

[permalink] [raw]
Subject: Re: IDE?

On Sat, 17 Aug 2002, Marc-Christian Petersen wrote:
> > In article <2444170000.1029531611@flay>,
> > Martin J. Bligh <[email protected]> wrote:
> > > So did Linus get disk corruption or is something else afoot?
> > Martin gave up the fight he had to do all the time, so..
>
> I am beside my self with laughing, sorry :P

Having thrown away months and months of hard work, or
giving up on months of hard work is NOT FUN.

I'm thankful Martin tried to make the IDE layer better.

His method of removing things to "add a better implementation
later" may not have worked out in the end, but I'm thankful
he tried.

regards,

Rik
--
Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/ http://distro.conectiva.com/

2002-08-17 00:29:35

by Andries Brouwer

[permalink] [raw]
Subject: Re: IDE?

On Fri, Aug 16, 2002 at 04:36:42PM -0700, Larry McVoy wrote:

> This may be politically incorrect, but could you (or anyone) provide a
> history of the IDE maintainers to date and why they didn't work out
> and what would need to change to make them work out? I'm sticking my
> nose in where I know nothing but maybe one of them could rise up to
> the necessary level if it were spelled out what that level was.

Prehistory: Linus and others.

Since start of 1994: Mark Lord. Everybody was happy until mid 1998 (2.1.111),
when, after a discussion about problems a few people had with DMA
Linus patched linux/drivers/block/Config.in:

- bool ' Generic PCI bus-master DMA support' CONFIG_BLK_DEV_IDEDMA
+ # Either the DMA code is buggy or the DMA hardware is unreliable. FIXME.
+ #bool ' Generic PCI bus-master DMA support' CONFIG_BLK_DEV_IDEDMA

and Mark wrote: "I will not be updating the IDE driver again until the
linux/drivers/block/Config.in file is restored to its pre-111 state."

Since Fall 1998 (2.1.122): Andre Hedrick.

Recent history is known.

(Lots of other people also did useful work on IDE. I will not try to list
names since I would forget some.)

Of course, "IDE maintainer" implies work on the interface with the hardware
and work on the interface with the block I/O subsystem of the kernel.
Some people know all about the hardware, others know much less about
hardware but have good ideas about the driver interface.
There is no reason to force the "IDE maintainer" to be a single person.

Andries


2002-08-17 01:00:53

by Alexander Viro

[permalink] [raw]
Subject: Re: IDE?



On Fri, 16 Aug 2002, Linus Torvalds wrote:

>
> On Fri, 16 Aug 2002, Linus Torvalds wrote:
> >
> > I actually don't think it's the people as much as it is the ridiculous
> > linkages inside ide.c and the hugely complicated rules. The code is messy.
>
> Note: it _is_ the people too, don't get me wrong. But in other areas we
> have people like Al Viro, who can drive grown men to cry (and drink) with
> his not-very-polite postings. And in those areas it hasn't been a huge
> problem, even though some people probably take a valium or two before they
> dare open emails from Al.
>
> So the messiness and interconnectedness of the IDE layer just seems to
> bring the people problem to a sharp and ugly point. The absolute lack of
> communication skills wrt IDE among the people who have worked on it has
> been stunning, and that probably _is_ because the code is so hard to even
> talk about.

Sigh... What we need with IDE is
a) translator/bogon filter between hardware folks and the rest of
us. If Jens or Alan are willing to do that for a while - wonderful.
b) review of code structure in existing code. Doing that.
c) careful massage (as opposed to grand rewrite) of said structure
into something sane. With series of small provable equivalent transformations.
And whoever does that is in serious risk of burnout - current spaghetty in
there is a fscking mess. I'll try to help with that - I know how to do such
work, but I don't promise to get it all the way to sanity.

When we will have sane structure and sane interfaces, the life will get easier.
Until then full-time maintainership of drivers/ide/* is a one-way ticket to
Bedlam.

2002-08-17 01:14:49

by Linus Torvalds

[permalink] [raw]
Subject: Re: IDE?


On Sat, 17 Aug 2002, Andries Brouwer wrote:
>
> Of course, "IDE maintainer" implies work on the interface with the hardware
> and work on the interface with the block I/O subsystem of the kernel.
> Some people know all about the hardware, others know much less about
> hardware but have good ideas about the driver interface.
> There is no reason to force the "IDE maintainer" to be a single person.

There isn't in theory, but because a minor change to one part will make
other drivers fail subtly, one person has to be the one that holds the bag
in the end. Because somebody _will_ be the one that everybody looks at
when something breaks.

That is probably one of the largest reasons for the "IDE disease". The
symptoms of the disease is that people complain about the stuff not
working, and the maintainer eventually getting so fed up with the
complaints that he stops interacting with reality, and starts worrying
about compliance with the documentation instead, hoping that that will fix
everything.

Which would work fine, except a lot of the time the problems aren't due to
things in the documentation, but simply due to hardware that isn't really
in spec and needs to have workarounds etc. So once the "this is how it is
documented, and if it doesn't work your machine is broken" disease starts,
it's all downhill from there.

I will claim that this happens for a lot of other hardware too, but in
other hardware there often isn't quite as much baggage (people in the end
throw out the core and start on a new one without historical cruft), and
the inter-driver linkages do not exist to _nearly_ the same degree. When a
developer can work with just one chipset, it's still possible to believe
that you can keep up. But when you get blamed for all the different IDE
problems, you crawl into your shell and go away.

This is why I believe that the only sane result in the end is to have
independent drivers that probably end up having a lot of duplication (the
same way hd.c and ide.c started out with a lot of duplication), but where
there truly _can_ be multiple people in charge of their own drivers (and
also clear _whose_ problem it is when one IDE controller driver doesn't
work).

(Some of the infrastructure could be made truly generic, but the generic
part should _only_ be for stuff that is truly hardware-independent, and
simply _cannot_ be impacted by quirks and outright bugs in the hw
implementations. In short, only stuff that can be argued about on a
logical and clear level, and where the rules are made up by us, not by
quirky hardware).

Linus

2002-08-17 01:28:40

by Linus Torvalds

[permalink] [raw]
Subject: Re: IDE?


On Fri, 16 Aug 2002, Alexander Viro wrote:
>
> Sigh... What we need with IDE is
> a) translator/bogon filter between hardware folks and the rest of
> us. If Jens or Alan are willing to do that for a while - wonderful.

Agreed.

> b) review of code structure in existing code. Doing that.

Hey, my secret plan is to make sure those IDE people are kept in check, by
having AL flame them to smithereens if they do something stupid..

> c) careful massage (as opposed to grand rewrite) of said structure
> into something sane. With series of small provable equivalent transformations.
> And whoever does that is in serious risk of burnout - current spaghetty in
> there is a fscking mess. I'll try to help with that - I know how to do such
> work, but I don't promise to get it all the way to sanity.

Good luck, but I think those init rules etc are really horribly subtle.

I really would suggest an alternate (but not necessarily very different)
approach.

The approach I'd advocate is to

- move the major number registration out of the IDE code, and changing
all device numbers into indexes + a queue. This can be done without
actually changing any of the _IO_ the drivers do, and should be doable
with a transformation that doesn't change behaviur _at_all_.

This, btw, is an area that a certain Al is fairly intimate with anyway ;)

- once the IDE drivers don't even know about major and minor numbers
(right now it has 10 major numbers assigned to it, I think) and doesn't
register a block device with "register_blkdev()" at all, but instead
registers the controllers it finds one by one through some indirect
agent, we can now try phase 2.

- phase 2: IDE-TNG. Leave the current IDE code unchanged, and plan to
obsolete it. It's the "stable IDE", and by virtue of being stable,
nobody will mind work on new drivers that (by definition) cannot screw
up unless you use them.

IDE-TNG would:
- be controller-specific (ie one driver for one controller family)
- be able to say "screw it" for old or broken setups (which are left
fot the old IDE code)
- in particular, it would only bother with PCI (or better)
controllers, and with UDMA-only setups.

The point of IDE-TNG would be to only support the major controllers this
way, but let those major controllers have a driver that is meant for
_them_ and doesn't have to worry about historical baggage.

And then in five years, in Linux-3.2, we might finally just drop support
for the old IDE code with PIO etc. Inevitably some people will still use
it (the same way some people still use Linux-2.0 with hd.c), but it won't
have been in the way for making a cleaner driver in the meantime.

And yes, by now this all is obviously 2.7.x material.

Linus

2002-08-17 01:53:55

by Andrew Morton

[permalink] [raw]
Subject: Re: IDE?

Rik van Riel wrote:
>
> On Sat, 17 Aug 2002, Marc-Christian Petersen wrote:
> > > In article <2444170000.1029531611@flay>,
> > > Martin J. Bligh <[email protected]> wrote:
> > > > So did Linus get disk corruption or is something else afoot?
> > > Martin gave up the fight he had to do all the time, so..
> >
> > I am beside my self with laughing, sorry :P
>
> Having thrown away months and months of hard work, or
> giving up on months of hard work is NOT FUN.
>
> I'm thankful Martin tried to make the IDE layer better.
>
> His method of removing things to "add a better implementation
> later" may not have worked out in the end, but I'm thankful
> he tried.
>

Yes. Martin starkly demonstrated how much work is needed in
there, and how much cruft has accumulated. That is valuable.

And I take back my bugreport regarding failure to read the final
eight sectors of my 80 gig maxtors on HPT374. Linus' current
tree has the same failure.

(gdb) bt
#0 __lock_page (page=0xc108141c) at /usr/src/25/include/asm/bitops.h:136
#1 0xc012d313 in lock_page (page=0xc3ff48c0) at filemap.c:710
#2 0xc012e561 in read_cache_page (mapping=0xc2f6d3c4, index=7, filler=0xc0143dc0 <blkdev_readpage>, data=0x0)
at filemap.c:1765
#3 0xc016488e in read_dev_sector (bdev=0xc2f11f60, n=63, p=0xc3fdbe38) at check.c:458
#4 0xc0164a3b in parse_extended (state=0xc2f02000, bdev=0xc2f11f60, first_sector=63, first_size=48243132)
at msdos.c:106
#5 0xc0164d7f in msdos_partition (state=0xc2f02000, bdev=0xc2f11f60) at msdos.c:433
#6 0xc01646a6 in check_partition (hd=0xc2fb73c0, bdev=0xc2f11f60) at check.c:288
#7 0xc0164840 in grok_partitions (dev={value = 8448}, size=160086528) at check.c:448
#8 0xc01647b6 in register_disk (gdev=0xc2fb73c0, dev={value = 8448}, minors=64, ops=0xc0322660, size=160086528)
at check.c:422
#9 0xc0212193 in idedisk_attach (drive=0xc03895dc) at ide-disk.c:1492
#10 0xc020a751 in subdriver_match (channel=0xc03894b0, ops=0xc03230e0) at main.c:526
#11 0xc020aac6 in register_ata_driver (driver=0xc03230e0) at main.c:1090
#12 0xc033f3dd in idedisk_init () at ide-disk.c:1503
#13 0xc033d914 in ata_module_init () at main.c:1377
#14 0xc033da27 in init_ata () at main.c:1417


(And grep bio drivers/ide/ataraid.c || echo drat)

2002-08-17 02:28:13

by Alexander Viro

[permalink] [raw]
Subject: Re: IDE?



On Fri, 16 Aug 2002, Linus Torvalds wrote:

> Good luck, but I think those init rules etc are really horribly subtle.

They are, but there is a buttload of amazingly convoluted C on top of them.
And I'd like to shave _that_ off. After that we'll be left with real
complexity imposed by hardware - $DEITY witness, there's enough of it to
make the things nasty; no need to further complicate control flow...

Let me put it another way: I feel that a lot of things can be un-obfuscated
by pure equivalent transformations that would treat almost all driver as
block box - making sure that same functions are called with the same
arguments in the same order and not even thinking about possible
reordering/changes inside the "payload" part.

IOW, there is high-level logics that
(a) is sufficiently separate from the guts
(b) would be worth IOCCC submission if it passed the size limit
(c) manages to mask and obfuscate _real_ complexity present in there.

And yes, I'd like that to be gone. After that... at the very least we will
see what's really going on in there. I'm rather sceptical about IDE-TNG -
grand rewrites _might_ be necessary at some point, but right now the mess
in the interfaces and in the way top-level code is organized is the worst
problem. Any work with real guts of the driver is complicated by that and
IMO any decisions on what to do with the guts should wait until we _see_
said guts.

2002-08-17 08:22:08

by Jens Axboe

[permalink] [raw]
Subject: Re: IDE?

On Fri, Aug 16 2002, Linus Torvalds wrote:
>
> On Sat, 17 Aug 2002, Anton Altaparmakov wrote:
> >
> > Out of curiosity, who is going to be IDE 2.5 kernel maintainer now?
>
> Well, as I implied, Alan seems to be not completely unwilling to work on
> it, and unlike me he _can_ interact with Andre most of the time. Possibly
> Jens will do the 2.5.x side, of it (with Alan working on 2.4), but we've
> not talked it through.

I'd never want to be the maintainer, but I do not mind taking care of
2.5 ide in the sense that 2.4-ac (or whatever) gets adapted, split to
digestible pieces (Andre's stuff always appears in big globs) and pushed
upstream.

> I'd like Vojtech to be a bit involved too, he seemed to do some
> much-needed cleanups for PIIX4 IDE (now gone, since we couldn't save just
> those parts..)

Me too. And Bartlomiej too, for that matter.

--
Jens Axboe

2002-08-17 08:18:47

by Russell King

[permalink] [raw]
Subject: Re: IDE?

On Fri, Aug 16, 2002 at 06:35:29PM -0700, Linus Torvalds wrote:
> And then in five years, in Linux-3.2, we might finally just drop support
> for the old IDE code with PIO etc. Inevitably some people will still use
> it (the same way some people still use Linux-2.0 with hd.c), but it won't
> have been in the way for making a cleaner driver in the meantime.

I think you're being too ""mainstream" orientated" here. Let's look
at something called PCMCIA. PCMCIA CF cards. They're IDE devices
that only do PIO.

The majority of ARM platforms being actively produced today provide a
PCMCIA or CF (_not_ cardbus) socket. Neither PCI nor Cardbus makes any
sense in these machines. Why? Because they're not your average power
hungry desktop box that's always plugged into the mains supply.

I think we're going to have PIO mode IDE around for a fair (ten at
least?) number of years yet.

--
Russell King ([email protected]) The developer of ARM Linux
http://www.arm.linux.org.uk/personal/aboutme.html

2002-08-17 09:17:31

by Andries Brouwer

[permalink] [raw]
Subject: Re: IDE?

On Fri, Aug 16, 2002 at 10:32:10PM -0400, Alexander Viro wrote:

> Let me put it another way: I feel that a lot of things can be un-obfuscated
> by pure equivalent transformations that would treat almost all driver as
> block box

Yes.

(But things here are a bit more subtle than for example VFS,
since there are timing restrictions.)

2002-08-17 11:05:02

by Rogier Wolff

[permalink] [raw]
Subject: Re: IDE?

On Fri, Aug 16, 2002 at 06:35:29PM -0700, Linus Torvalds wrote:
> And yes, by now this all is obviously 2.7.x material.

Linus,

I sure hope that you're talking about "finishing" your project plan,
and not about "starting it". I would really prefer to have work on
the IDE-TNG started soonish, rather than defered another year or so.

There will be plenty of bugs to be found, which can benefit from a
longer testing-period by people daring enough to try the new and
experimental driver.....

Roger.

--
** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2600998 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots.
* There are also old, bald pilots.

2002-08-17 11:48:48

by Matthias Andree

[permalink] [raw]
Subject: Re: IDE?

On Fri, 16 Aug 2002, Linus Torvalds wrote:

> In article <2444170000.1029531611@flay>,
> Martin J. Bligh <[email protected]> wrote:
> >So did Linus get disk corruption or is something else afoot?
>
> Martin gave up the fight he had to do all the time, so..

Not having seen much of all the work, this sounds like it is a sad day,
for Martin who contributed a lot of his time to work on the issues,
while people seemed not to be too grateful, screaming from either end of
the road, this must wear people out.

Is there a way how the improvements that parts of the stuff have
received can be rescued somehow? Or at least the knowledge of which can
be used somewhat directly for a IDE-TNG driver? I'd find it really sad
to just let go of so much time that had been invested into the project
-- but keep in mind I have NO knowledge of the gory details of the last
2.5 IDE stuff.

--
Matthias Andree

2002-08-17 12:04:44

by Russell King

[permalink] [raw]
Subject: Re: IDE?

On Sat, Aug 17, 2002 at 01:52:43PM +0200, Matthias Andree wrote:
> Is there a way how the improvements that parts of the stuff have
> received can be rescued somehow?

There were a few bits I submitted that should probably go in - I'll
look at reviving them when I'm less busy and the direction of IDE
stuff has settled down a bit.

--
Russell King ([email protected]) The developer of ARM Linux
http://www.arm.linux.org.uk/personal/aboutme.html

2002-08-17 12:12:01

by Andre Hedrick

[permalink] [raw]
Subject: Re: IDE? IDE-TNG driver


I will hand it to you guys on a silver platter IDE-TNG.

Below yields modular chipsets and channel index registration.
Selectable IOPS for arch independent Taskfile Transport layers.
Now to finish the job with device class link lists to address fully
modular subdrivers. It also includes 1st generation of device open and
select calls of subdrivers.

You have ide-cd registered on a cdrw and you want to burn a cd?
open(/dev/hdX) transform_subdriver_scsi close(/dev/hdX)
open(/dev/sg) and burn baby burn.
close(/dev/sg) releases transform_subdriver_scsi
open(/dev/hdX) load native atapi transport.

Will do it for TAPE-FLOPPY-DVDCD/RW ...


Uniform Multi-Platform E-IDE driver Revision: 6.31
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
SiI680: IDE controller on PCI bus 00 dev 90
SiI680: chipset revision 1
SiI680: not 100% native mode: will probe irqs later
SiI680: BASE CLOCK == 133
ide0: MMIO-DMA at 0xe080df00-0xe080df07, BIOS settings: hda:pio, hdb:pio
ide1: MMIO-DMA at 0xe080df08-0xe080df0f, BIOS settings: hdc:pio, hdd:pio
hda: Maxtor 4G160J8, ATA DISK drive
hdb: Maxtor 4G160J8, ATA DISK drive
ide0 at 0xe080df80-0xe080df87,0xe080df8a on irq 9
hda: host protected area => 1
hda: 320173056 sectors (163929 MB) w/2048KiB Cache, CHS=19929/255/63, UDMA(133)
hdb: host protected area => 1
hdb: 320173056 sectors (163929 MB) w/2048KiB Cache, CHS=19929/255/63, UDMA(133)
hdc: Maxtor 4G160J8, ATA DISK drive
hdd: Maxtor 4G160J8, ATA DISK drive
ide1 at 0xe080dfc0-0xe080dfc7,0xe080dfca on irq 9
hdc: host protected area => 1
hdc: 320173056 sectors (163929 MB) w/2048KiB Cache, CHS=19929/255/63, UDMA(133)
hdd: host protected area => 1
hdd: 320173056 sectors (163929 MB) w/2048KiB Cache, CHS=19929/255/63, UDMA(133)
PIIX3: IDE controller on PCI bus 00 dev 39
PIIX3: chipset revision 0
PIIX3: not 100% native mode: will probe irqs later
ide2: BM-DMA at 0xffa0-0xffa7, BIOS settings: hde:DMA, hdf:DMA
ide3: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdg:pio, hdh:pio
hde: ATAPI 44X CDROM, ATAPI CD/DVD-ROM drive
hdf: CREATIVEDVD5240E-1, ATAPI CD/DVD-ROM drive
ide2 at 0x1f0-0x1f7,0x3f6 on irq 14
hde: ATAPI 40X CD-ROM drive, 128kB Cache, (U)DMA
Uniform CD-ROM driver Revision: 3.12
hdf: ATAPI 32X DVD-ROM drive, 512kB Cache, DMA
Partition check:
/dev/ide/host0/bus0/target0/lun0: p1
/dev/ide/host0/bus0/target1/lun0: p1
/dev/ide/host0/bus1/target0/lun0: p1
/dev/ide/host0/bus1/target1/lun0: p1


If this is what you want, this is what I have to put on the table.
If you do not I will delete the code.


Regards,

Andre Hedrick
LAD Storage Consulting Group

On Sat, 17 Aug 2002, Matthias Andree wrote:

> On Fri, 16 Aug 2002, Linus Torvalds wrote:
>
> > In article <2444170000.1029531611@flay>,
> > Martin J. Bligh <[email protected]> wrote:
> > >So did Linus get disk corruption or is something else afoot?
> >
> > Martin gave up the fight he had to do all the time, so..
>
> Not having seen much of all the work, this sounds like it is a sad day,
> for Martin who contributed a lot of his time to work on the issues,
> while people seemed not to be too grateful, screaming from either end of
> the road, this must wear people out.
>
> Is there a way how the improvements that parts of the stuff have
> received can be rescued somehow? Or at least the knowledge of which can
> be used somewhat directly for a IDE-TNG driver? I'd find it really sad
> to just let go of so much time that had been invested into the project
> -- but keep in mind I have NO knowledge of the gory details of the last
> 2.5 IDE stuff.
>
> --
> Matthias Andree
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

2002-08-17 13:00:52

by Adam J. Richter

[permalink] [raw]
Subject: Re: IDE?

I just looked at the patch to switch to "2.4 forward port"
version of drivers/ide. If I got my shell commands right, Martin's
tree is 8606 lines shorter than the 2.4 forward port.

2.4 forward port 49,205 lines
Martin's version 40,599 lines
------------
8,606 lines difference

It's often amazing how much cleaning up it takes to shrink
code a little bit. Shrinking the IDE tree this much is a lot of
work to throw away.

In comparison, I think Niklaus Wirth's Modula-2 compiler for
the Lilith machine was 5,000 lines.

Is the 2.5.31 IDE tree that buggy? I would hope that stamping
out bugs from Martin's tree would be less work than cleaning up
the 2.4 version to that point again.

>If you can work with him, then it would seem he would be well suited for
>the job... Assuming he wants it... Bartlomiej?

I'd be quite relieved if we could convince Bartlomiej to
adopt Martin's tree and continue with Martin's tree as at least
a configuration option.

Adam J. Richter __ ______________ 575 Oroville Road
[email protected] \ / Milpitas, California 95035
+1 408 309-6081 | g g d r a s i l United States of America
"Free Software For The Rest Of Us."

2002-08-17 13:18:03

by Alexander Kellett

[permalink] [raw]
Subject: Re: IDE?

[shortening cc:'s as they are probably not interested in me :) ]

On Sat, Aug 17, 2002 at 06:02:16AM -0700, Adam J. Richter wrote:
> I'd be quite relieved if we could convince Bartlomiej to
> adopt Martin's tree and continue with Martin's tree as at least
> a configuration option.

Couldn't agree more here. Although I usually hate <aol>-like
me too posts. I think that its really quite important for the
state of the ide layer that the work thats already gone into
it isn't lost.

IMO this is not something that you can see a positive, or as a
lesson learned. This to me just seems like a lot of hard work
going down the drain.

So, pleeease Bartlomiej/Alan/Jens, whoever. Someone step up
to get most/some of Marcin' cleanup patches into 2.5 again.

Alex

2002-08-17 13:43:29

by Skip Ford

[permalink] [raw]
Subject: Re: IDE? IDE-TNG driver

Andre Hedrick wrote:
>
> I will hand it to you guys on a silver platter IDE-TNG.
>
[snip]
>
> If this is what you want, this is what I have to put on the table.
> If you do not I will delete the code.

Can't you just create a patch and send it to the list? I for one would
like to try out your code. Just diff it and send it without the song
and dance please.

--
Skip

2002-08-17 14:36:46

by Mike Dresser

[permalink] [raw]
Subject: Re: IDE?

> IDE-TNG would:

As long as we don't have IDE-Voy, or IDE-DS9, this should be good.

Mike

2002-08-17 17:15:06

by Ruth Ivimey-Cook

[permalink] [raw]
Subject: Re: IDE? IDE-TNG driver

On Sat, 17 Aug 2002, Andre Hedrick wrote:

>
>I will hand it to you guys on a silver platter IDE-TNG.
>
>Below yields modular chipsets and channel index registration.
>Selectable IOPS for arch independent Taskfile Transport layers.
...
>You have ide-cd registered on a cdrw and you want to burn a cd?
>open(/dev/hdX) transform_subdriver_scsi close(/dev/hdX)
>open(/dev/sg) and burn baby burn.
>close(/dev/sg) releases transform_subdriver_scsi
>open(/dev/hdX) load native atapi transport.


Andre, I see the thought, but surely this is prine to races and other
difficulties.

Wouldn't it be better to provide an IDE ioctl() that enables the caller to use
set the SCSI transport on an open FD, so your sequence becomes:

open(/dev/hdX)
ioctl(transform_subdriver_scsi)
ioctl(scsi_ops)
write(data)
close(/dev/hdX)

Ruth

--
Ruth Ivimey-Cook
Software engineer and technical writer.

2002-08-17 17:43:11

by Andre Hedrick

[permalink] [raw]
Subject: Re: IDE?

On Sat, 17 Aug 2002, Alexander Kellett wrote:

> [shortening cc:'s as they are probably not interested in me :) ]
>
> IMO this is not something that you can see a positive, or as a
> lesson learned. This to me just seems like a lot of hard work
> going down the drain.

This one of my arguments against deleting Marcin work.
There are things he did which are good and real cleanups.

> So, pleeease Bartlomiej/Alan/Jens, whoever. Someone step up
> to get most/some of Marcin' cleanup patches into 2.5 again.

I have addressed most of his work which was sane and did not violate
transport protocols.

Regards,

Andre Hedrick
LAD Storage Consulting Group

2002-08-17 17:49:18

by Dax Kelson

[permalink] [raw]
Subject: Does Solaris really scale this well?


From:

http://www.itworld.com/Man/3828/020816mcnealy/

Scott McNealy:

"When you take a 99-way UltraSPARC III machine and add a 100th processor,
you get 94 percent linear scalability. You can't get 94 percent linear
scalability on your first Intel chip. It's very, very hard to do, and they
have not done it."



2002-08-17 18:12:27

by Jan-Benedict Glaw

[permalink] [raw]
Subject: Re: IDE?

On Fri, 2002-08-16 18:35:29 -0700, Linus Torvalds <[email protected]>
wrote in message <[email protected]>:
> On Fri, 16 Aug 2002, Alexander Viro wrote:

> - in particular, it would only bother with PCI (or better)
> controllers, and with UDMA-only setups.
[...]
> And then in five years, in Linux-3.2, we might finally just drop support
> for the old IDE code with PIO etc. Inevitably some people will still use

That's bad. Then, you're nailed to use old kernels without having
possibilities of recent kernels only because you're working with eg. old
Alphas, PCMCIA-IDE things or so? Bad, bad, badhorribly bad. Even it's
sloooow, there'll always some need for PIO-only controller support...

MfG, JBG

--
Jan-Benedict Glaw . [email protected] . +49-172-7608481
-- New APT-Proxy written in shell script --
http://lug-owl.de/~jbglaw/software/ap2/


Attachments:
(No filename) (921.00 B)
(No filename) (189.00 B)
Download all attachments

2002-08-17 18:08:59

by Andre Hedrick

[permalink] [raw]
Subject: Re: IDE? IDE-TNG driver



ide_ioctl(fd, HDIO_SET_IDE_SCSI, bool)

Where bool does the subdriver switch.
Just that ioctl's are being blasted and people using are frowned upon.

This was a feature Alan Cox poked me for to try and move away from how
modules are basically an all or nothing grab-all.

Regards,

Andre Hedrick
LAD Storage Consulting Group

On Sat, 17 Aug 2002, Ruth Ivimey-Cook wrote:

> On Sat, 17 Aug 2002, Andre Hedrick wrote:
>
> >
> >I will hand it to you guys on a silver platter IDE-TNG.
> >
> >Below yields modular chipsets and channel index registration.
> >Selectable IOPS for arch independent Taskfile Transport layers.
> ...
> >You have ide-cd registered on a cdrw and you want to burn a cd?
> >open(/dev/hdX) transform_subdriver_scsi close(/dev/hdX)
> >open(/dev/sg) and burn baby burn.
> >close(/dev/sg) releases transform_subdriver_scsi
> >open(/dev/hdX) load native atapi transport.
>
>
> Andre, I see the thought, but surely this is prine to races and other
> difficulties.
>
> Wouldn't it be better to provide an IDE ioctl() that enables the caller to use
> set the SCSI transport on an open FD, so your sequence becomes:
>
> open(/dev/hdX)
> ioctl(transform_subdriver_scsi)
> ioctl(scsi_ops)
> write(data)
> close(/dev/hdX)
>
> Ruth
>
> --
> Ruth Ivimey-Cook
> Software engineer and technical writer.
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

2002-08-17 18:25:19

by Andreas Dilger

[permalink] [raw]
Subject: Re: IDE?

On Aug 17, 2002 06:02 -0700, Adam J. Richter wrote:
> I just looked at the patch to switch to "2.4 forward port"
> version of drivers/ide. If I got my shell commands right, Martin's
> tree is 8606 lines shorter than the 2.4 forward port.
>
> 2.4 forward port 49,205 lines
> Martin's version 40,599 lines
> ------------
> 8,606 lines difference
>
> It's often amazing how much cleaning up it takes to shrink
> code a little bit. Shrinking the IDE tree this much is a lot of
> work to throw away.
>
> In comparison, I think Niklaus Wirth's Modula-2 compiler for
> the Lilith machine was 5,000 lines.
>
> Is the 2.5.31 IDE tree that buggy? I would hope that stamping
> out bugs from Martin's tree would be less work than cleaning up
> the 2.4 version to that point again.

Why don't we just start with the now-discarded 2.5 IDE code as IDE-TNG?
If people want to develop/hack then they can use that, and if they
want to hack on other things they use the old code. You just need to
make the two config options mutually exclusive until the drivers learn
to play well together (by being able to control separate drives/ctrlr).

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

2002-08-17 18:27:42

by Anton Altaparmakov

[permalink] [raw]
Subject: Re: IDE?

At 19:16 17/08/02, Jan-Benedict Glaw wrote:
>On Fri, 2002-08-16 18:35:29 -0700, Linus Torvalds <[email protected]>
>wrote in message <[email protected]>:
> > On Fri, 16 Aug 2002, Alexander Viro wrote:
>
> > - in particular, it would only bother with PCI (or better)
> > controllers, and with UDMA-only setups.
>[...]
> > And then in five years, in Linux-3.2, we might finally just drop support
> > for the old IDE code with PIO etc. Inevitably some people will still use
>
>That's bad. Then, you're nailed to use old kernels without having
>possibilities of recent kernels only because you're working with eg. old
>Alphas, PCMCIA-IDE things or so? Bad, bad, badhorribly bad. Even it's
>sloooow, there'll always some need for PIO-only controller support...

I don't think it is possible to have DMA only drivers. On DMA
failure/timeouts/whatever, the current DMA drivers always fall back to PIO
mode and this is a good thing. Otherwise many transfers would simply fail.
Dropping PIO mode fallback would mean a lot of IO errors. Any system put
under stress will at some point fall back to PIO mode (at least judgjing
from the limited number of systems I have) because it doesn't manage to do
the DMA transfers in time. That was very visible during the period when
Andre's new IDE core went into 2.5.something_early and it turned out that
PIO was broken at that point. For example my VIA box was running just fine
then in DMA mode but as soon as I put it under stress it blew up due to it
falling out of DMA and the then broken PIO mode... And VIA686b is
mainstream hardware...

Best regards,

Anton


--
"I've not lost my mind. It's backed up on tape somewhere." - Unknown
--
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS Maintainer / IRC: #ntfs on irc.openprojects.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

2002-08-17 18:24:25

by Andre Hedrick

[permalink] [raw]
Subject: Re: IDE?


One of the issues I am addressing is how to deal w/ VDMA or PIO over
PCI-DMA or FP-DMA issues. First Party DMA is how things will get done,
and regardless there will still be an need for PIO.

As long as ther transport layer requires it regardless of the wrapper or
pipe it is run down, it shall be around.

Regards,

Andre Hedrick
LAD Storage Consulting Group


On Sat, 17 Aug 2002, Jan-Benedict Glaw wrote:

> On Fri, 2002-08-16 18:35:29 -0700, Linus Torvalds <[email protected]>
> wrote in message <[email protected]>:
> > On Fri, 16 Aug 2002, Alexander Viro wrote:
>
> > - in particular, it would only bother with PCI (or better)
> > controllers, and with UDMA-only setups.
> [...]
> > And then in five years, in Linux-3.2, we might finally just drop support
> > for the old IDE code with PIO etc. Inevitably some people will still use
>
> That's bad. Then, you're nailed to use old kernels without having
> possibilities of recent kernels only because you're working with eg. old
> Alphas, PCMCIA-IDE things or so? Bad, bad, badhorribly bad. Even it's
> sloooow, there'll always some need for PIO-only controller support...
>
> MfG, JBG
>
> --
> Jan-Benedict Glaw . [email protected] . +49-172-7608481
> -- New APT-Proxy written in shell script --
> http://lug-owl.de/~jbglaw/software/ap2/
>

2002-08-17 18:23:16

by Matti Aarnio

[permalink] [raw]
Subject: Re: Does Solaris really scale this well?

On Sat, Aug 17, 2002 at 11:53:16AM -0600, Dax Kelson wrote:
> From:
> http://www.itworld.com/Man/3828/020816mcnealy/
>
> Scott McNealy:
>
> "When you take a 99-way UltraSPARC III machine and add a 100th processor,
> you get 94 percent linear scalability. You can't get 94 percent linear
> scalability on your first Intel chip. It's very, very hard to do, and they
> have not done it."

Conditionally... I would like to know the exact architecture,
and the problem set running in the system to say.

When you have noncc-NUMA, you have a Beowulf-like setup.
when you have cc-NUMA ("cc" = cache coherent), things get
truly hairy...

2002-08-17 18:36:16

by Andre Hedrick

[permalink] [raw]
Subject: Re: IDE?


Anton,

They will become PIO over DMA, and it will become interesting.
But you are correct, we are stuck wit PIO regardless.

Andre Hedrick
LAD Storage Consulting Group

On Sat, 17 Aug 2002, Anton Altaparmakov wrote:

> At 19:16 17/08/02, Jan-Benedict Glaw wrote:
> >On Fri, 2002-08-16 18:35:29 -0700, Linus Torvalds <[email protected]>
> >wrote in message <[email protected]>:
> > > On Fri, 16 Aug 2002, Alexander Viro wrote:
> >
> > > - in particular, it would only bother with PCI (or better)
> > > controllers, and with UDMA-only setups.
> >[...]
> > > And then in five years, in Linux-3.2, we might finally just drop support
> > > for the old IDE code with PIO etc. Inevitably some people will still use
> >
> >That's bad. Then, you're nailed to use old kernels without having
> >possibilities of recent kernels only because you're working with eg. old
> >Alphas, PCMCIA-IDE things or so? Bad, bad, badhorribly bad. Even it's
> >sloooow, there'll always some need for PIO-only controller support...
>
> I don't think it is possible to have DMA only drivers. On DMA
> failure/timeouts/whatever, the current DMA drivers always fall back to PIO
> mode and this is a good thing. Otherwise many transfers would simply fail.
> Dropping PIO mode fallback would mean a lot of IO errors. Any system put
> under stress will at some point fall back to PIO mode (at least judgjing
> from the limited number of systems I have) because it doesn't manage to do
> the DMA transfers in time. That was very visible during the period when
> Andre's new IDE core went into 2.5.something_early and it turned out that
> PIO was broken at that point. For example my VIA box was running just fine
> then in DMA mode but as soon as I put it under stress it blew up due to it
> falling out of DMA and the then broken PIO mode... And VIA686b is
> mainstream hardware...
>
> Best regards,
>
> Anton
>
>
> --
> "I've not lost my mind. It's backed up on tape somewhere." - Unknown
> --
> Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
> Linux NTFS Maintainer / IRC: #ntfs on irc.openprojects.net
> WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

2002-08-17 19:48:57

by Alan

[permalink] [raw]
Subject: Re: IDE?

On Sat, 2002-08-17 at 14:22, Alexander Kellett wrote:
> So, pleeease Bartlomiej/Alan/Jens, whoever. Someone step up
> to get most/some of Marcin' cleanup patches into 2.5 again.

Not interested. Its easier to go back to functionally correct code and
do the job nicely than to fix the 2.5.3x code. Right now I'm working on
Andre's current code in 2.4.20pre2-ac* starting off with only provably
identical transforms between AndreCode and C and documenting it.

Alan

2002-08-17 19:54:31

by Alan

[permalink] [raw]
Subject: Re: IDE?

On Sat, 2002-08-17 at 01:10, Linus Torvalds wrote:
> I'd like Vojtech to be a bit involved too, he seemed to do some
> much-needed cleanups for PIIX4 IDE (now gone, since we couldn't save just
> those parts..)

If he does please co-ordinate with me. I've already done a chunk of
cleanup there (and Andre has done a load too).

I want order to this. That means all the driver cleanup goes into 2.4-ac
(or "2.4-ide" or some suitable branch) first where we can verify we
aren't hitting 2.5 generic bugs and ide corruption is a meaningful
problem report. It means someone (not me) is the appointed 2.5 person
and handles stuff going to 2.5 (I'm happy to identify stuff that tests
ok in 2.4 as candidates). It also means random patches not going past
me.

If we can do it that way I'll do the job. If Linus applies random IDE
"cleanup" patches to his 2.5 tree that don't pass through Jens and me
then I'll just stop listening to 2.5 stuff.

Volunteers willing to run Cerberus test sets on 2.4 boxes with IDE
controllers would also be much appreciated. That way we can get good
coverage tests and catch badness immediately

Alan

2002-08-17 19:57:08

by Alan

[permalink] [raw]
Subject: Re: IDE?

On Sat, 2002-08-17 at 01:02, Linus Torvalds wrote:
> Even something as simple as a PIIX driver (which _should_ just register
> itself as a driver for the piix chipsets) doesn't do that. Instead, we
> have ide-pci.c, which has a list of all the chipsets it knows about, and
> then does initialization and calls the init routines that it knows about.
> That's just incestuous.

The pci scan can't go away to get the nasty BIOS ordering crap right.
Andre now has that code as it should be however. The scan loop is
basically no more than

pci_foreach_device_in_weird_ide_order()
{
call probe function(dev)
}

Which also means we are really really close to hot plug ide controllers.

The probe function->generic pci init->driver->generic->driver-> chain is
still too ugly but its getting better and lives in a struct now

2002-08-17 20:01:03

by Alan

[permalink] [raw]
Subject: Re: IDE?

On Sat, 2002-08-17 at 01:09, Larry McVoy wrote:
> > I actually don't think it's the people as much as it is the ridiculous
> > linkages inside ide.c and the hugely complicated rules. The code is messy.
>
> So of all the people you know floating around in "active hacker" state, who
> seems like the sort of person who could handle this mess? If there is no
> person, is there a description which is more specific than "wanted: person
> who wants thankless abusive non-payed job to clean up what is inherently a
> mess"?

IMHO you need
- An understanding of ATA (which is the protocol
equivalent of object oriented cobol)
- The ability to work with vendors (it needs to be someone
at a company because many vendors won't NDA with
individuals even if they are happy with GPL code off
their data sheet)
- Someone who has taste in code and understands how to
beat code into shape without breaking it
- The ability to deduce the other errata the vendor forgot
to tell you about or doesn't want to admit exists for
fear of US lawsuits (I kid you not)
- A good understanding of the block layer and its locking
especially because IDE has a heirarchy of contention
problems:
two drives one bus
two channels one DMA engine
two controllers one I/O at a time
ISA IRQ sharing locks


2002-08-17 21:56:28

by Vojtech Pavlik

[permalink] [raw]
Subject: Re: IDE?

On Sat, Aug 17, 2002 at 09:22:39AM +0100, Russell King wrote:

> On Fri, Aug 16, 2002 at 06:35:29PM -0700, Linus Torvalds wrote:
> > And then in five years, in Linux-3.2, we might finally just drop support
> > for the old IDE code with PIO etc. Inevitably some people will still use
> > it (the same way some people still use Linux-2.0 with hd.c), but it won't
> > have been in the way for making a cleaner driver in the meantime.
>
> I think you're being too ""mainstream" orientated" here. Let's look
> at something called PCMCIA. PCMCIA CF cards. They're IDE devices
> that only do PIO.
>
> The majority of ARM platforms being actively produced today provide a
> PCMCIA or CF (_not_ cardbus) socket. Neither PCI nor Cardbus makes any
> sense in these machines. Why? Because they're not your average power
> hungry desktop box that's always plugged into the mains supply.
>
> I think we're going to have PIO mode IDE around for a fair (ten at
> least?) number of years yet.

We'll need PIO for control commands anyways, but the thing is that we
won't need to speed optimize PIO and will be able to kill multi-sector
PIO completely probably.

--
Vojtech Pavlik
SuSE Labs

2002-08-17 22:07:02

by Anton Altaparmakov

[permalink] [raw]
Subject: Re: IDE?

At 20:56 17/08/02, Alan Cox wrote:
>Volunteers willing to run Cerberus test sets on 2.4 boxes with IDE
>controllers would also be much appreciated. That way we can get good
>coverage tests and catch badness immediately

If you tell me the kernel version and patches to apply which you want
tested, and what options to run cerberus with (never used it before...), I
have control over a currently idle dual Athlon MP 2000+ with an AMD-768
(rev 04) IDE controller and 3G of RAM. It has only one HD, a ST340810A
(ATA-100, 37G) attached.

btw. Is this where I get cerberus from?
http://sourceforge.net/projects/va-ctcs/

The machine won't be in use for a few more weeks (until I find the time to
configure all the software and the database server it will be connecting
to...) so I can do tests during that period.

Best regards,

Anton


--
"I've not lost my mind. It's backed up on tape somewhere." - Unknown
--
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS Maintainer / IRC: #ntfs on irc.openprojects.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

2002-08-17 22:13:20

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: IDE?

Em Sat, Aug 17, 2002 at 11:11:14PM +0100, Anton Altaparmakov escreveu:
> At 20:56 17/08/02, Alan Cox wrote:
> >Volunteers willing to run Cerberus test sets on 2.4 boxes with IDE
> >controllers would also be much appreciated. That way we can get good
> >coverage tests and catch badness immediately

> If you tell me the kernel version and patches to apply which you want
> tested, and what options to run cerberus with (never used it before...), I
> have control over a currently idle dual Athlon MP 2000+ with an AMD-768
> (rev 04) IDE controller and 3G of RAM. It has only one HD, a ST340810A
> (ATA-100, 37G) attached.

I have a dual p100 with a CMD640 so I'll test 2.4-ac or whatever you name it,
as soon as I get back from vacation (in two weeks) and I get another old disk
for this test machine. It behaves with 2.4 but loses a lot of interrupts with
2.5-MD (haven't tested after Jens got 2.4 forward port into 2.5).

- Arnaldo

2002-08-17 22:56:07

by Ruth Ivimey-Cook

[permalink] [raw]
Subject: Re: IDE? IDE-TNG driver

On Sat, 17 Aug 2002, Andre Hedrick wrote:
>
>ide_ioctl(fd, HDIO_SET_IDE_SCSI, bool)

Seems fine to me...

>Where bool does the subdriver switch.
>Just that ioctl's are being blasted and people using are frowned upon.

? so how is cdrecord (or whatever) supposed to do its stuff -- is it ioctl()
-> fcntl()? If so, I suppose that's ok, but the basic premise still exists,
surely?

>This was a feature Alan Cox poked me for to try and move away from how
>modules are basically an all or nothing grab-all.

I don't think modules are the answer to any of this:
a) some people want basically module-less kernels
b) in some environments, you need to be able to select the IO mechanism
without the ability to select the module to load.

anyway...

<slightly confused by it all>


Ruth

--
Ruth Ivimey-Cook
Software engineer and technical writer.

2002-08-17 23:01:47

by Ruth Ivimey-Cook

[permalink] [raw]
Subject: Re: Does Solaris really scale this well?

On Sat, 17 Aug 2002, Matti Aarnio wrote:

>On Sat, Aug 17, 2002 at 11:53:16AM -0600, Dax Kelson wrote:
>> From:
>> http://www.itworld.com/Man/3828/020816mcnealy/
>>
>> Scott McNealy:
>>
>> "When you take a 99-way UltraSPARC III machine and add a 100th processor,
>> you get 94 percent linear scalability. You can't get 94 percent linear
>> scalability on your first Intel chip. It's very, very hard to do, and they
>> have not done it."
>
> Conditionally... I would like to know the exact architecture,
>and the problem set running in the system to say.
>
>When you have noncc-NUMA, you have a Beowulf-like setup.
>when you have cc-NUMA ("cc" = cache coherent), things get
>truly hairy...

I've seen scientific reports of scalability that good in non-shared memory
computers (mostly in transputer arrays) where (with a scalable algorithm)
unless you got >90% you were doing something wrong. However, if you insist on
sharing main memory, I still don't believe you can get anywhere near that...
IMO 30% is doing very well once past the first few CPUs.

Regards,

Ruth

--
Ruth Ivimey-Cook
Software engineer and technical writer.

2002-08-17 23:15:23

by John Weber

[permalink] [raw]
Subject: Re: IDE?

Arnaldo Carvalho de Melo wrote:
> Em Sat, Aug 17, 2002 at 11:11:14PM +0100, Anton Altaparmakov escreveu:
>
>>At 20:56 17/08/02, Alan Cox wrote:
>>
>>>Volunteers willing to run Cerberus test sets on 2.4 boxes with IDE
>>>controllers would also be much appreciated. That way we can get good
>>>coverage tests and catch badness immediately
>>
>
>>If you tell me the kernel version and patches to apply which you want
>>tested, and what options to run cerberus with (never used it before...), I
>>have control over a currently idle dual Athlon MP 2000+ with an AMD-768
>>(rev 04) IDE controller and 3G of RAM. It has only one HD, a ST340810A
>>(ATA-100, 37G) attached.
>
>
> I have a dual p100 with a CMD640 so I'll test 2.4-ac or whatever you name it,
> as soon as I get back from vacation (in two weeks) and I get another old disk
> for this test machine. It behaves with 2.4 but loses a lot of interrupts with
> 2.5-MD (haven't tested after Jens got 2.4 forward port into 2.5).
>

I have a dual Ppro200 with a PIIX ide chip, and a PIII 866 with a VIA
686B. I'll test anything you want on these boxes.


2002-08-17 23:12:28

by Andre Hedrick

[permalink] [raw]
Subject: Re: IDE? IDE-TNG driver

On Sat, 17 Aug 2002, Ruth Ivimey-Cook wrote:

> On Sat, 17 Aug 2002, Andre Hedrick wrote:
> >
> >ide_ioctl(fd, HDIO_SET_IDE_SCSI, bool)
>
> Seems fine to me...
>
> >Where bool does the subdriver switch.
> >Just that ioctl's are being blasted and people using are frowned upon.
>
> ? so how is cdrecord (or whatever) supposed to do its stuff -- is it ioctl()
> -> fcntl()? If so, I suppose that's ok, but the basic premise still exists,
> surely?
>
> >This was a feature Alan Cox poked me for to try and move away from how
> >modules are basically an all or nothing grab-all.
>
> I don't think modules are the answer to any of this:
> a) some people want basically module-less kernels

This is designed to work regardless.

/dev/hdc == ide-cd builtin
insmod ide-scsi

ide_ioctl(fd, HDIO_SET_IDE_SCSI, bool)

converts /dev/hdc == ide-cd builtin to ide-scsi(add-in-module).

> b) in some environments, you need to be able to select the IO mechanism
> without the ability to select the module to load.

See above, I think it solves the problem.
Once ide-scsi is added to the ide_module link list it is as good as
built-in.

> anyway...
>
> <slightly confused by it all>

Me too, because I do not know the direction goal so I am doing the very
best I can. What I really need is an active development team.

Before me:

Mark Lord, Gadi Oxman, Eric Anderson worked well.

ML ide-disk and ide.c global.
GO ide-floppy, ide-tape, ide-scsi
EA ide-cd

Anyways that was long before transport layer w/ all the hardware issues
began to dominate things.

Cheers,

Andre Hedrick
LAD Storage Consulting Group

2002-08-18 00:25:56

by Scott Bronson

[permalink] [raw]
Subject: Re: IDE? IDE-TNG driver

On Sat, 2002-08-17 at 15:57, Ruth Ivimey-Cook wrote:
> a) some people want basically module-less kernels

Everyone I've heard advocating a moduleless kernel uses an argument that
boils down to "it's slightly more secure." Does anybody have a GOOD
reason for not using modules? Obsolete or embedded hardware arguments
don't count.


> b) in some environments, you need to be able to select the IO mechanism
> without the ability to select the module to load.

If that's the case, won't a kernel parameter suffice? Can you
elaborate?

- Scott



2002-08-18 00:48:25

by Russell King

[permalink] [raw]
Subject: Re: IDE?

On Sat, Aug 17, 2002 at 11:59:42PM +0200, Vojtech Pavlik wrote:
> We'll need PIO for control commands anyways, but the thing is that we
> won't need to speed optimize PIO and will be able to kill multi-sector
> PIO completely probably.

Well, I'll probably be maintaining multi-sector PIO externally to the
main kernel in that case. 95% of ARM machines have either PIO only or
in the case of those that do have PCI DMA support (netwinders) the
southbridge is soo messed up that DMA is useless on most boxes produced.

Multi-sector PIO is a fundamental requirement that I require.

--
Russell King ([email protected]) The developer of ARM Linux
http://www.arm.linux.org.uk/personal/aboutme.html

2002-08-18 00:51:19

by Larry McVoy

[permalink] [raw]
Subject: Re: Does Solaris really scale this well?

On Sun, Aug 18, 2002 at 12:03:24AM +0100, Ruth Ivimey-Cook wrote:
> >> "When you take a 99-way UltraSPARC III machine and add a 100th processor,
> >> you get 94 percent linear scalability. You can't get 94 percent linear
> >> scalability on your first Intel chip. It's very, very hard to do, and they
> >> have not done it."
>
> I've seen scientific reports of scalability that good in non-shared memory
> computers (mostly in transputer arrays) where (with a scalable algorithm)
> unless you got >90% you were doing something wrong. However, if you insist on
> sharing main memory, I still don't believe you can get anywhere near that...
> IMO 30% is doing very well once past the first few CPUs.

Please reconsider your opinion. Both Sun and SGI scale past 100 CPUs on
reasonable workloads in shared memory. Where "reasonable" != easy to do.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm

2002-08-18 01:34:15

by Linus Torvalds

[permalink] [raw]
Subject: Re: IDE?


On 17 Aug 2002, Alan Cox wrote:
>
> If we can do it that way I'll do the job. If Linus applies random IDE
> "cleanup" patches to his 2.5 tree that don't pass through Jens and me
> then I'll just stop listening to 2.5 stuff.

That may work for the low-level driver stuff, but not for things like the
partitioning fixes. Especially as some of that is different in 2.5.x
relative to 2.4.x.

In other words, I'll clearly apply anything that comes from Al, at the
very least.

Linus

2002-08-18 01:42:11

by Luigi Genoni

[permalink] [raw]
Subject: Re: Does Solaris really scale this well?

On Sat, 17 Aug 2002, Dax Kelson wrote:

> Date: Sat, 17 Aug 2002 11:53:16 -0600 (MDT)
> From: Dax Kelson <[email protected]>
> To: "[email protected]" <[email protected]>
> Subject: Does Solaris really scale this well?
>
>
> From:
>
> http://www.itworld.com/Man/3828/020816mcnealy/
>
> Scott McNealy:
>
> "When you take a 99-way UltraSPARC III machine and add a 100th processor,
> you get 94 percent linear scalability. You can't get 94 percent linear
> scalability on your first Intel chip. It's very, very hard to do, and they
> have not done it."

you can't get 94% linear scalability also on Sparc III, to say the
truth...




2002-08-18 01:54:49

by Alexander Viro

[permalink] [raw]
Subject: Re: IDE?



On Sat, 17 Aug 2002, Linus Torvalds wrote:

>
> On 17 Aug 2002, Alan Cox wrote:
> >
> > If we can do it that way I'll do the job. If Linus applies random IDE
> > "cleanup" patches to his 2.5 tree that don't pass through Jens and me
> > then I'll just stop listening to 2.5 stuff.
>
> That may work for the low-level driver stuff, but not for things like the
> partitioning fixes. Especially as some of that is different in 2.5.x
> relative to 2.4.x.
>
> In other words, I'll clearly apply anything that comes from Al, at the
> very least.

I'm going to run these patches by Alan first anyway, so...

BTW, most of partitioning patches apply to both branches - it's preliminary
cleanup part that is tricky; some of partitioning stuff is already in Jens
code and the rest will be a matter of not moving add_gendisk()/del_gendisk()
in 2.4 branch and leaving the calls of grok_partitions()/wipe_partitions()
in ide_revalidate() (2.4) while removing them in 2.5.

Cleanup that comes before that stage is common for 2.4 and 2.5, makes sense
in both and will be synced with Alan - if nothing else, to keep PITA for
Jens minimal.

Look at the patches for other drivers - gendisk splitup / removal of
BLKRRPART handling / removal of ad-hackery in open/reread partitions /
removal of grok_partitions() and wipe_partitions() is fairly small
_if_ driver has clean rules for places where it used to register disks.
That's the main trouble with IDE for my current purposes - and fixing it
makes sense for 2.4 as well as for 2.5.

2002-08-18 02:38:31

by Thomas Molina

[permalink] [raw]
Subject: Re: IDE?

On Sat, 17 Aug 2002, Anton Altaparmakov wrote:

> At 20:56 17/08/02, Alan Cox wrote:
> >Volunteers willing to run Cerberus test sets on 2.4 boxes with IDE
> >controllers would also be much appreciated. That way we can get good
> >coverage tests and catch badness immediately
>
> If you tell me the kernel version and patches to apply which you want
> tested, and what options to run cerberus with (never used it before...), I
> have control over a currently idle dual Athlon MP 2000+ with an AMD-768
> (rev 04) IDE controller and 3G of RAM. It has only one HD, a ST340810A
> (ATA-100, 37G) attached.
>
> btw. Is this where I get cerberus from?
> http://sourceforge.net/projects/va-ctcs/
>
> The machine won't be in use for a few more weeks (until I find the time to
> configure all the software and the database server it will be connecting
> to...) so I can do tests during that period.

I'm not familiar with Cerebus, but I'm willing to pitch in with any
testing you feel necessary. I just finished rebuilding my system with
removable hard drives, originally to beat against Martin's IDE code. I
now have a known good stable system I can do production with as well as a
dev drive I can restore to pristine in about 15 minutes. System is an
Athlon 1.3 GHz on Asus A7V with 384MB RAM.

Is it going to be more desireable to beat on 2.4 IDE or 2.5 IDE?

2002-08-18 02:57:30

by Erik Andersen

[permalink] [raw]
Subject: Re: IDE? IDE-TNG driver

On Sat Aug 17, 2002 at 04:06:46PM -0700, Andre Hedrick wrote:
> Mark Lord, Gadi Oxman, Eric Anderson worked well.

All these years later, I'm still known to submit
the occasional patch....

-Erik

--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

2002-08-18 03:44:46

by Scott Bronson

[permalink] [raw]
Subject: Re: IDE? IDE-TNG driver

> Everyone I've heard advocating a moduleless kernel uses an argument that
> boils down to "it's slightly more secure." Does anybody have a GOOD
> reason for not using modules? Obsolete or embedded hardware arguments
> don't count.

Someone replied off-list saying that initrds are too hard to create.

That's true. They are. One day, hopefully that will change.

Any other reasons?

- Scott




2002-08-18 04:09:25

by Thomas Molina

[permalink] [raw]
Subject: Re: IDE? IDE-TNG driver

On 17 Aug 2002, Scott Bronson wrote:

> > Everyone I've heard advocating a moduleless kernel uses an argument that
> > boils down to "it's slightly more secure." Does anybody have a GOOD
> > reason for not using modules? Obsolete or embedded hardware arguments
> > don't count.
>
> Someone replied off-list saying that initrds are too hard to create.
>
> That's true. They are. One day, hopefully that will change.
>
> Any other reasons?

Wouldn't the logic be a lot simpler if kernel developers didn't have to
worry about whether their module was built in or inserted at some point in
the future? All the bits could be assembled and symbols would be resolved
at compile time.

ICBW but it appears the largest percentage of users have modules inserted
at boot time, never to be ejected or disturbed. The modules might as well
be built in.

I've never really grokked the whole initrd anyway. What is the point of
building a kernel minus the bits it needs to actually boot the system?
That just forces this ludicrous jerry-rigged mess to provide the
capabilities that should have been built in at compile time.

Of course, we're never going to get away from modules now anyway, so the
argument is probably moot. With PCCARD, USB, Firewire, and Hotplug PCI
there are too many ways to add components not present when the system is
built on an adhoc basis. The alternative would be a bloated monstrosity
with dead code not needed by most people. I remember my early days of
Linux and seeing literally dozens of kernels compiled with different
options, with no clue how to choose the right one for my system.

2002-08-18 04:17:07

by CaT

[permalink] [raw]
Subject: Re: IDE? IDE-TNG driver

On Sat, Aug 17, 2002 at 08:47:30PM -0700, Scott Bronson wrote:
> > Everyone I've heard advocating a moduleless kernel uses an argument that
> > boils down to "it's slightly more secure." Does anybody have a GOOD
> > reason for not using modules? Obsolete or embedded hardware arguments
> > don't count.
>
> Someone replied off-list saying that initrds are too hard to create.
>
> That's true. They are. One day, hopefully that will change.
>
> Any other reasons?

Because I shouldn't have to use a feature if I don't need to use a
feature?

I dunno... I've not once heard a decent argument as to why I should
modulerise the ide subsystem, or ext3 or the video drivers and so on.

Where I -do- need to use it though, I do happily use it. One case is to
reset the eepro100 driver so that it works after my computer is brought
out of suspend mode (either RAM or disk).

But why should I -not- make a monolithic kernel when I don't have any
reason to do so? If you can provide me with some decent ones, I'll
happily start...

--
"There was a moo from under the blanket, and I knew it was not a person,
but a calf."
- http://www.iol.co.za/index.php?art_id=qw1028795041273B265

2002-08-18 08:45:54

by David Miller

[permalink] [raw]
Subject: Re: Does Solaris really scale this well?

From: Larry McVoy <[email protected]>
Date: Sat, 17 Aug 2002 17:55:17 -0700

On Sun, Aug 18, 2002 at 12:03:24AM +0100, Ruth Ivimey-Cook wrote:
> >> "When you take a 99-way UltraSPARC III machine and add a 100th processor,
> >> you get 94 percent linear scalability. You can't get 94 percent linear
> >> scalability on your first Intel chip. It's very, very hard to do, and they
> >> have not done it."
>
> I've seen scientific reports of scalability that good in non-shared memory
> computers (mostly in transputer arrays) where (with a scalable algorithm)
> unless you got >90% you were doing something wrong. However, if you insist on
> sharing main memory, I still don't believe you can get anywhere near that...
> IMO 30% is doing very well once past the first few CPUs.

Please reconsider your opinion. Both Sun and SGI scale past 100 CPUs on
reasonable workloads in shared memory. Where "reasonable" != easy to do.

Also consider that if you start having performed so badly in the
uniprocessor case like Solaris does, it doesn't take so much effort to
get good scalability percentages as you add cpus because there isn't
much to scale. :-)

To Sun's credit, they have on their side the fact that in the x86
world there still has never has been a very good large scale SMP
backplane as of yet. At least not on the order of what you'd find
on one of Sun's big boxes.

But in the same breath this is what will kill Sun in the end. Over
time, the commodity stuff inches closer and closer to what Sun's "high
end" is.

2002-08-18 10:00:47

by Vojtech Pavlik

[permalink] [raw]
Subject: Re: IDE?

On Sun, Aug 18, 2002 at 01:52:18AM +0100, Russell King wrote:

> > We'll need PIO for control commands anyways, but the thing is that we
> > won't need to speed optimize PIO and will be able to kill multi-sector
> > PIO completely probably.
>
> Well, I'll probably be maintaining multi-sector PIO externally to the
> main kernel in that case. 95% of ARM machines have either PIO only or
> in the case of those that do have PCI DMA support (netwinders) the
> southbridge is soo messed up that DMA is useless on most boxes produced.
>
> Multi-sector PIO is a fundamental requirement that I require.

Good to know that.

--
Vojtech Pavlik
SuSE Labs

2002-08-18 10:24:28

by Luigi Genoni

[permalink] [raw]
Subject: Re: Does Solaris really scale this well?

On Sat, 17 Aug 2002, Larry McVoy wrote:

> Date: Sat, 17 Aug 2002 17:55:17 -0700
> From: Larry McVoy <[email protected]>
> To: Ruth Ivimey-Cook <[email protected]>
> Cc: Matti Aarnio <[email protected]>, Dax Kelson <[email protected]>,
> "[email protected]" <[email protected]>
> Subject: Re: Does Solaris really scale this well?
>
> On Sun, Aug 18, 2002 at 12:03:24AM +0100, Ruth Ivimey-Cook wrote:
> > >> "When you take a 99-way UltraSPARC III machine and add a 100th processor,
> > >> you get 94 percent linear scalability. You can't get 94 percent linear
> > >> scalability on your first Intel chip. It's very, very hard to do, and they
> > >> have not done it."
> >
> > I've seen scientific reports of scalability that good in non-shared memory
> > computers (mostly in transputer arrays) where (with a scalable algorithm)
> > unless you got >90% you were doing something wrong. However, if you insist on
> > sharing main memory, I still don't believe you can get anywhere near that...
> > IMO 30% is doing very well once past the first few CPUs.
>
> Please reconsider your opinion. Both Sun and SGI scale past 100 CPUs on
> reasonable workloads in shared memory. Where "reasonable" != easy to do.


And where reasonable != 94%. Seriously, 94% scalability could be on a
8 CPUs 880, but, for example, I have a 64 CPUS domain on a E10k which is
far from 94% scalability (ok, an old E10k with an 83Mhz bus).
For what I saw, maybe SGI Origin 3000 is scaling
a little better with a lot of CPUS, but I also never had an E15000 around
for now...





2002-08-18 11:14:45

by Vojtech Pavlik

[permalink] [raw]
Subject: Re: IDE?

On Fri, Aug 16, 2002 at 05:10:12PM -0700, Linus Torvalds wrote:

> On Sat, 17 Aug 2002, Anton Altaparmakov wrote:
> >
> > Out of curiosity, who is going to be IDE 2.5 kernel maintainer now?
>
> Well, as I implied, Alan seems to be not completely unwilling to work on
> it, and unlike me he _can_ interact with Andre most of the time. Possibly
> Jens will do the 2.5.x side, of it (with Alan working on 2.4), but we've
> not talked it through.
>
> I'd like Vojtech to be a bit involved too, he seemed to do some
> much-needed cleanups for PIIX4 IDE (now gone, since we couldn't save just
> those parts..)

I'll make patches for 2.5 to bring the low-level driver cleanups back.
Not just piix.c - also aec62xx.c and amd74xx.c - the last one was in 2.5
for a LONG time already and I'm not particularly happy it got lost.

If desirable (What's your opinion, Alan?) I can make equivalent patches
for 2.4 as well.

--
Vojtech Pavlik
SuSE Labs

2002-08-18 12:13:38

by Alan

[permalink] [raw]
Subject: Re: IDE?

On Sun, 2002-08-18 at 12:15, Vojtech Pavlik wrote:
> I'll make patches for 2.5 to bring the low-level driver cleanups back.
> Not just piix.c - also aec62xx.c and amd74xx.c - the last one was in 2.5
> for a LONG time already and I'm not particularly happy it got lost.
>
> If desirable (What's your opinion, Alan?) I can make equivalent patches
> for 2.4 as well.

Look at 2.4.20-pre2-ac3 before you start doing that. A lot of cleanup
has been done, although there is plenty more left. A starter is to fix
the the ratemask/ratefilter stuff to not use silly while loops on the
aec/amd drivers if you are hacking on those, stick in the static
variables and document anything relevant looking.

Simple stuff first.

Alan

2002-08-18 12:31:58

by Ruth Ivimey-Cook

[permalink] [raw]
Subject: Re: Does Solaris really scale this well?

On Sat, 17 Aug 2002, Larry McVoy wrote:

>On Sun, Aug 18, 2002 at 12:03:24AM +0100, Ruth Ivimey-Cook wrote:
>> >> "When you take a 99-way UltraSPARC III machine and add a 100th processor,
>> >> you get 94 percent linear scalability. You can't get 94 percent linear
>> >> scalability on your first Intel chip. It's very, very hard to do, and they
>> >> have not done it."
>>
>> I've seen scientific reports of scalability that good in non-shared memory
>> computers (mostly in transputer arrays) where (with a scalable algorithm)
>> unless you got >90% you were doing something wrong. However, if you insist on
>> sharing main memory, I still don't believe you can get anywhere near that...
>> IMO 30% is doing very well once past the first few CPUs.
>
>Please reconsider your opinion. Both Sun and SGI scale past 100 CPUs on
>reasonable workloads in shared memory. Where "reasonable" != easy to do.

Larry,

I wasn't disputing that Sun could have say 100 cpus in a box, but that the
100th shared-memory CPU didias much work as the first. That said, I _am_ out
of date on the performance of the Sun machines; what kind of measured
performance effeciency do you get with them?

A google search turned up:
http://www.icg.tu-graz.ac.at/goller/publication/pers/node6.html (1997)
in which the author says:

Utilizing more than half of all processors is commonly agreed to be an
acceptable efficiency for parallel applications. In all three diagrams, the
efficiency never drops below this 50%-level in the area of interest

-- well, I would disagree about the implied "any parallel app", but it does
seem to be true of many SMP systems...

In the following paper, the authors benchmarked IBM, Cray and SGI
supercomputers: http://citeseer.nj.nec.com/kang99benchmarking.html (1999)

If you look at pages numbered 53 & 54, you will see graphs of time vs num
processors that are a very long way from linear, indeed in one case time
increased as cum cpus increased.

It is also instructive to note that in many cases, the peak processor power is
obtained by multiplying individual peak power by the number of processors,
with no notice taken of the costs of synchronisation, memory access or
communication. Consequently, many new owners of supercomputers are very
disappointed with their new 'baby' when they find it's not nearly as powerful
as they had been told.

Regards,

Ruth

--
Ruth Ivimey-Cook
Software engineer and technical writer.



2002-08-18 12:39:01

by Alexander Kellett

[permalink] [raw]
Subject: Re: IDE?

On Sat, Aug 17, 2002 at 08:51:09PM +0100, Alan Cox wrote:
> On Sat, 2002-08-17 at 14:22, Alexander Kellett wrote:
> > So, pleeease Bartlomiej/Alan/Jens, whoever. Someone step up
> > to get most/some of Marcin' cleanup patches into 2.5 again.
>
> Not interested. Its easier to go back to functionally correct code and
> do the job nicely than to fix the 2.5.3x code. Right now I'm working on
> Andre's current code in 2.4.20pre2-ac* starting off with only provably
> identical transforms between AndreCode and C and documenting it.

Better point, I realized my mistake right after reading Al's
"tranformation" post and grasping the sense in it. Much better approach.

Alex

2002-08-18 22:46:54

by Adam J. Richter

[permalink] [raw]
Subject: Re: IDE?

On 2002-08-17, Alan Cox wrote:
>Volunteers willing to run Cerberus test sets on 2.4 boxes with IDE
>controllers would also be much appreciated. That way we can get good
>coverage tests and catch badness immediately

From visiting the osdl.org booth a LinuxWorld, I understand
that they have a farm of 150 deliberately differently configured
computers on which you are supposed to be able to run your own
kernel tests on your own kernels.

They have a procedure for adding new tests described at
http://www.osdl.org/stp/HOWTO.Port_Tests.html.

I think it would be informative to run 2.4 ported code and
Martin's stack against IDE tests on this system. With this, you could
not only spot problems, but see problems happening in a certain pattern
which could sometimes simplify finding a bug.

Adam J. Richter __ ______________ 575 Oroville Road
[email protected] \ / Milpitas, California 95035
+1 408 309-6081 | g g d r a s i l United States of America
"Free Software For The Rest Of Us."

2002-08-18 23:45:22

by Andre Hedrick

[permalink] [raw]
Subject: Re: IDE?


OSDL does not support IDE.

See, I asked some time ago about testing raid arrays and a possible thread
driver. All you will find there is onboard hosts, and I do not believe
they stock the all the junk systems out there.

150 systems is a not a large enough sample.

500 ++ w/ various add-in cards * (the number of drive models per vendor) *
the number vendors eek stop ....

Add in ATAPI devices, CFA, etc ...

You see the combination are of a scale unchecked.

I started at one point trying to have a single drive vendor with at least
1000 or more systems in one lab to test their product over the host
hardware base, to run kernel checks. The task was beyond scale.

All it takes is a bus analyzer and a few systems known to be good to
perfect the driver. Then you deal with all the exceptions of the crap
combinations.

That is how I do it, since I have a code base that has been run over a
bus analyzer I know it works.

Cheers,


On Sun, 18 Aug 2002, Adam J. Richter wrote:

> On 2002-08-17, Alan Cox wrote:
> >Volunteers willing to run Cerberus test sets on 2.4 boxes with IDE
> >controllers would also be much appreciated. That way we can get good
> >coverage tests and catch badness immediately
>
> From visiting the osdl.org booth a LinuxWorld, I understand
> that they have a farm of 150 deliberately differently configured
> computers on which you are supposed to be able to run your own
> kernel tests on your own kernels.
>
> They have a procedure for adding new tests described at
> http://www.osdl.org/stp/HOWTO.Port_Tests.html.
>
> I think it would be informative to run 2.4 ported code and
> Martin's stack against IDE tests on this system. With this, you could
> not only spot problems, but see problems happening in a certain pattern
> which could sometimes simplify finding a bug.
>
> Adam J. Richter __ ______________ 575 Oroville Road
> [email protected] \ / Milpitas, California 95035
> +1 408 309-6081 | g g d r a s i l United States of America
> "Free Software For The Rest Of Us."
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

Andre Hedrick
LAD Storage Consulting Group

2002-08-18 23:55:17

by Adam J. Richter

[permalink] [raw]
Subject: Re: IDE?

Andre Hedrick writes:
>150 systems is a not a large enough sample.

Sorry if I wasn't clear. I did not mean "instead of" other
testing, I meant "in addition to."

Anyhow, I'm glad to hear that you had already checked out osdl.

Adam J. Richter __ ______________ 575 Oroville Road
[email protected] \ / Milpitas, California 95035
+1 408 309-6081 | g g d r a s i l United States of America
"Free Software For The Rest Of Us."

2002-08-19 00:06:25

by David Lang

[permalink] [raw]
Subject: Re: IDE? IDE-TNG driver

A couple things in favor of a monlithic kernel.

there is a slight performance advantage becouse the calls don't
have to be far calls

there is a slight memory advantage becouse you don't have the fraction of
a page of ram lost per module

with a monolithic kernel there's no chance of making a mistake and trying
to use incompatable modules with your kernel (and before you say that this
can be fixed with the kernel build remember that for many people the build
machine is not where the kernel will be run)

David Lang


On 17 Aug 2002, Scott
Bronson wrote:

> Date: 17 Aug 2002 17:28:38 -0700
> From: Scott Bronson <[email protected]>
> To: Ruth Ivimey-Cook <[email protected]>
> Cc: [email protected]
> Subject: Re: IDE? IDE-TNG driver
>
> On Sat, 2002-08-17 at 15:57, Ruth Ivimey-Cook wrote:
> > a) some people want basically module-less kernels
>
> Everyone I've heard advocating a moduleless kernel uses an argument that
> boils down to "it's slightly more secure." Does anybody have a GOOD
> reason for not using modules? Obsolete or embedded hardware arguments
> don't count.
>
>
> > b) in some environments, you need to be able to select the IO mechanism
> > without the ability to select the module to load.
>
> If that's the case, won't a kernel parameter suffice? Can you
> elaborate?
>
> - Scott
>
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

2002-08-19 09:52:07

by Vojtech Pavlik

[permalink] [raw]
Subject: Re: IDE?

On Sat, Aug 17, 2002 at 12:26:38PM -0600, Andreas Dilger wrote:
> On Aug 17, 2002 06:02 -0700, Adam J. Richter wrote:
> > I just looked at the patch to switch to "2.4 forward port"
> > version of drivers/ide. If I got my shell commands right, Martin's
> > tree is 8606 lines shorter than the 2.4 forward port.
> >
> > 2.4 forward port 49,205 lines
> > Martin's version 40,599 lines
> > ------------
> > 8,606 lines difference
> >
> > It's often amazing how much cleaning up it takes to shrink
> > code a little bit. Shrinking the IDE tree this much is a lot of
> > work to throw away.
> >
> > In comparison, I think Niklaus Wirth's Modula-2 compiler for
> > the Lilith machine was 5,000 lines.
> >
> > Is the 2.5.31 IDE tree that buggy? I would hope that stamping
> > out bugs from Martin's tree would be less work than cleaning up
> > the 2.4 version to that point again.
>
> Why don't we just start with the now-discarded 2.5 IDE code as IDE-TNG?
> If people want to develop/hack then they can use that, and if they
> want to hack on other things they use the old code. You just need to
> make the two config options mutually exclusive until the drivers learn
> to play well together (by being able to control separate drives/ctrlr).

Well, because it might be easier to just start from scratch.

--
Vojtech Pavlik
SuSE Labs

2002-08-19 13:38:19

by Dana Lacoste

[permalink] [raw]
Subject: Re: Does Solaris really scale this well? [OT]

On Sun, 2002-08-18 at 08:33, Ruth Ivimey-Cook wrote:
> It is also instructive to note that in many cases, the peak processor power is
> obtained by multiplying individual peak power by the number of processors,
> with no notice taken of the costs of synchronisation, memory access or
> communication. Consequently, many new owners of supercomputers are very
> disappointed with their new 'baby' when they find it's not nearly as powerful
> as they had been told.

It is also important to note that most benchmarks go out of their
way to analyze this situation properly in the massively parallel
supercomputer world, and that for the most part supercomputers excel
in I/O bandwidth more than CPU power, so if you bought your SV2 to
run quake you _deserve_ to only get 100fps :)

To try to bring this a little more back on topic, is there any
particular group in the Linux community who's dissatisfied with
our scalability (except for IBM, who seem to be working on this
issue already. I mean anyone who's _not_ seeing things done to
rectify the situation :)

Dana Lacoste
Ottawa

2002-08-19 13:53:09

by Dana Lacoste

[permalink] [raw]
Subject: Re: IDE?

On Sat, 2002-08-17 at 14:16, Jan-Benedict Glaw wrote:
> That's bad. Then, you're nailed to use old kernels without having
> possibilities of recent kernels only because you're working with eg. old
> Alphas, PCMCIA-IDE things or so? Bad, bad, badhorribly bad. Even it's
> sloooow, there'll always some need for PIO-only controller support...

Correct me if I'm wrong, but isn't this already the case?

Are there not several uses of 2.0.x that are not compatible with
2.2/2.4? And if 2.0 is working, then why are you worried about
being able to use 3.2? Why do we need to maintain compatibility
with OLD (not 'low-end' but OLD) hardware if there's an existing
kernel that meets that hardware's needs already?

Dana Lacoste
Ottawa

2002-08-19 14:00:58

by Tomas Szepe

[permalink] [raw]
Subject: Re: IDE?

> Are there not several uses of 2.0.x that are not compatible with
> 2.2/2.4? And if 2.0 is working, then why are you worried about
> being able to use 3.2? Why do we need to maintain compatibility
> with OLD (not 'low-end' but OLD) hardware if there's an existing
> kernel that meets that hardware's needs already?

Because new userspace won't roll with old kernels (reliably|at all).

T.

2002-08-19 14:11:09

by Alan

[permalink] [raw]
Subject: Re: IDE?

On Mon, 2002-08-19 at 14:57, Dana Lacoste wrote:
> Are there not several uses of 2.0.x that are not compatible with
> 2.2/2.4? And if 2.0 is working, then why are you worried about
> being able to use 3.2? Why do we need to maintain compatibility
> with OLD (not 'low-end' but OLD) hardware if there's an existing
> kernel that meets that hardware's needs already?

How about because in many cases only 2.4 has needed features and also
runs better. On my 20Mb 486 running Xfce 2.4 is materially faster than
2.0

2002-08-19 17:05:19

by Willy Tarreau

[permalink] [raw]
Subject: Re: IDE?

On Mon, Aug 19, 2002 at 09:57:11AM -0400, Dana Lacoste wrote:
> Why do we need to maintain compatibility
> with OLD (not 'low-end' but OLD) hardware if there's an existing
> kernel that meets that hardware's needs already?

because we always need new features, even on older hardware. If 2.4 didn't
support my 386sx, I would have had to either port iptables, cramfs, pppoe
and such things to 2.0 (or 2.2), or replace this solid-state, silent box
with a new, noisy and power hungry one. I'm really happy that this box is
still supported and hope it will still run 2.6.

Cheers,
Willy

2002-08-19 20:11:19

by Timothy D. Witham

[permalink] [raw]
Subject: Re: IDE?

Actually while we have a variety of chip sets and systems are
goal is not to have 150 different system configurations.

We tend to stick with multiple systems of the same type. For
example when we buy some 4 ways we buy 3 or 4 of the same one
at the same time. There are some very good reasons for this.
The 1st being trying to keep a bunch of different chip sets
and I/O controllers functioning while the development kernel
rolls forward underneath us is hard enough when you limit
your configurations and would be impossible with 150 different
system types. Second since we are trying to do performance
and functionality testing the ability to assign a returning
user to any one of a group of systems instead of having them
wait for the exact one they used before makes scheduling a
lot easier. :-)

While a large number of systems in you lab do use IDE
for at least there primary disk we really aren't trying
to provide variety for the "does this chip set work" type
of testing.

Tim



On Sun, 2002-08-18 at 15:49, Adam J. Richter wrote:
> On 2002-08-17, Alan Cox wrote:
> >Volunteers willing to run Cerberus test sets on 2.4 boxes with IDE
> >controllers would also be much appreciated. That way we can get good
> >coverage tests and catch badness immediately
>
> From visiting the osdl.org booth a LinuxWorld, I understand
> that they have a farm of 150 deliberately differently configured
> computers on which you are supposed to be able to run your own
> kernel tests on your own kernels.
>
> They have a procedure for adding new tests described at
> http://www.osdl.org/stp/HOWTO.Port_Tests.html.
>
> I think it would be informative to run 2.4 ported code and
> Martin's stack against IDE tests on this system. With this, you could
> not only spot problems, but see problems happening in a certain pattern
> which could sometimes simplify finding a bug.
>
> Adam J. Richter __ ______________ 575 Oroville Road
> [email protected] \ / Milpitas, California 95035
> +1 408 309-6081 | g g d r a s i l United States of America
> "Free Software For The Rest Of Us."
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
Timothy D. Witham - Lab Director - [email protected]
Open Source Development Lab Inc - A non-profit corporation
15275 SW Koll Parkway - Suite H - Beaverton OR, 97006
(503)-626-2455 x11 (office) (503)-702-2871 (cell)
(503)-626-2436 (fax)

2002-08-19 20:22:47

by Timothy D. Witham

[permalink] [raw]
Subject: Re: IDE?

On Sun, 2002-08-18 at 16:38, Andre Hedrick wrote:
>
> OSDL does not support IDE.
>
Well, some of the test systems only have IDE but no we
aren't in the "try all of the IDE chip set" support business.

> See, I asked some time ago about testing raid arrays and a possible thread
> driver. All you will find there is onboard hosts, and I do not believe
> they stock the all the junk systems out there.
>

I'm always open to proposals but I've only really seen requests of
"I would like to test vendor A's hardware". In my mind that is
a Vendor A problem not a generic OS or driver issue. Now after
saying that if there is a class of issues that require a configuration
I'm happy to look at providing that.

> 150 systems is a not a large enough sample.
>

Agreed even if they are all different chip sets and controllers.

On a side note:

I am impressed with the serial ATA and the couple of products
that I've seen so far. I think that this could remove the last barrier
for ATA drives in larger systems and wait to see what products people
come up with in the near future.



> 500 ++ w/ various add-in cards * (the number of drive models per vendor) *
> the number vendors eek stop ....
>
> Add in ATAPI devices, CFA, etc ...
>
> You see the combination are of a scale unchecked.
>
> I started at one point trying to have a single drive vendor with at least
> 1000 or more systems in one lab to test their product over the host
> hardware base, to run kernel checks. The task was beyond scale.
>
> All it takes is a bus analyzer and a few systems known to be good to
> perfect the driver. Then you deal with all the exceptions of the crap
> combinations.
>
> That is how I do it, since I have a code base that has been run over a
> bus analyzer I know it works.
>
> Cheers,
>
>
> On Sun, 18 Aug 2002, Adam J. Richter wrote:
>
> > On 2002-08-17, Alan Cox wrote:
> > >Volunteers willing to run Cerberus test sets on 2.4 boxes with IDE
> > >controllers would also be much appreciated. That way we can get good
> > >coverage tests and catch badness immediately
> >
> > From visiting the osdl.org booth a LinuxWorld, I understand
> > that they have a farm of 150 deliberately differently configured
> > computers on which you are supposed to be able to run your own
> > kernel tests on your own kernels.
> >
> > They have a procedure for adding new tests described at
> > http://www.osdl.org/stp/HOWTO.Port_Tests.html.
> >
> > I think it would be informative to run 2.4 ported code and
> > Martin's stack against IDE tests on this system. With this, you could
> > not only spot problems, but see problems happening in a certain pattern
> > which could sometimes simplify finding a bug.
> >
> > Adam J. Richter __ ______________ 575 Oroville Road
> > [email protected] \ / Milpitas, California 95035
> > +1 408 309-6081 | g g d r a s i l United States of America
> > "Free Software For The Rest Of Us."
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to [email protected]
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at http://www.tux.org/lkml/
> >
>
> Andre Hedrick
> LAD Storage Consulting Group
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
Timothy D. Witham - Lab Director - [email protected]
Open Source Development Lab Inc - A non-profit corporation
15275 SW Koll Parkway - Suite H - Beaverton OR, 97006
(503)-626-2455 x11 (office) (503)-702-2871 (cell)
(503)-626-2436 (fax)

2002-08-19 20:37:21

by Andre Hedrick

[permalink] [raw]
Subject: Re: IDE?


Hi Tim "The Wookie"!

As you know I have release SATA 1.0 which was funded by Silicon Image.
I saw at LWE last week the Intel SATA raid card using SiI SATALink.
Maybe SiI might be able to help you out in that direction.
You at least know you will have quality driver support.

Cheers,

Andre Hedrick
Linux Serial ATA Solutions
LAD Storage Consulting Group



On 19 Aug 2002, Timothy D. Witham wrote:

> On Sun, 2002-08-18 at 16:38, Andre Hedrick wrote:
> >
> > OSDL does not support IDE.
> >
> Well, some of the test systems only have IDE but no we
> aren't in the "try all of the IDE chip set" support business.
>
> > See, I asked some time ago about testing raid arrays and a possible thread
> > driver. All you will find there is onboard hosts, and I do not believe
> > they stock the all the junk systems out there.
> >
>
> I'm always open to proposals but I've only really seen requests of
> "I would like to test vendor A's hardware". In my mind that is
> a Vendor A problem not a generic OS or driver issue. Now after
> saying that if there is a class of issues that require a configuration
> I'm happy to look at providing that.
>
> > 150 systems is a not a large enough sample.
> >
>
> Agreed even if they are all different chip sets and controllers.
>
> On a side note:
>
> I am impressed with the serial ATA and the couple of products
> that I've seen so far. I think that this could remove the last barrier
> for ATA drives in larger systems and wait to see what products people
> come up with in the near future.
>
>
>
> > 500 ++ w/ various add-in cards * (the number of drive models per vendor) *
> > the number vendors eek stop ....
> >
> > Add in ATAPI devices, CFA, etc ...
> >
> > You see the combination are of a scale unchecked.
> >
> > I started at one point trying to have a single drive vendor with at least
> > 1000 or more systems in one lab to test their product over the host
> > hardware base, to run kernel checks. The task was beyond scale.
> >
> > All it takes is a bus analyzer and a few systems known to be good to
> > perfect the driver. Then you deal with all the exceptions of the crap
> > combinations.
> >
> > That is how I do it, since I have a code base that has been run over a
> > bus analyzer I know it works.
> >
> > Cheers,
> >
> >
> > On Sun, 18 Aug 2002, Adam J. Richter wrote:
> >
> > > On 2002-08-17, Alan Cox wrote:
> > > >Volunteers willing to run Cerberus test sets on 2.4 boxes with IDE
> > > >controllers would also be much appreciated. That way we can get good
> > > >coverage tests and catch badness immediately
> > >
> > > From visiting the osdl.org booth a LinuxWorld, I understand
> > > that they have a farm of 150 deliberately differently configured
> > > computers on which you are supposed to be able to run your own
> > > kernel tests on your own kernels.
> > >
> > > They have a procedure for adding new tests described at
> > > http://www.osdl.org/stp/HOWTO.Port_Tests.html.
> > >
> > > I think it would be informative to run 2.4 ported code and
> > > Martin's stack against IDE tests on this system. With this, you could
> > > not only spot problems, but see problems happening in a certain pattern
> > > which could sometimes simplify finding a bug.
> > >
> > > Adam J. Richter __ ______________ 575 Oroville Road
> > > [email protected] \ / Milpitas, California 95035
> > > +1 408 309-6081 | g g d r a s i l United States of America
> > > "Free Software For The Rest Of Us."
> > > -
> > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > > the body of a message to [email protected]
> > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > > Please read the FAQ at http://www.tux.org/lkml/
> > >
> >
> > Andre Hedrick
> > LAD Storage Consulting Group
> >
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to [email protected]
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at http://www.tux.org/lkml/
> --
> Timothy D. Witham - Lab Director - [email protected]
> Open Source Development Lab Inc - A non-profit corporation
> 15275 SW Koll Parkway - Suite H - Beaverton OR, 97006
> (503)-626-2455 x11 (office) (503)-702-2871 (cell)
> (503)-626-2436 (fax)
>


2002-08-19 23:25:50

by Thunder from the hill

[permalink] [raw]
Subject: Re: IDE?

Hi,

On Fri, 16 Aug 2002, Linus Torvalds wrote:
> - phase 2: IDE-TNG.

Couldn't we give it another name? Such as LUI - Linux Unified IDE?
Whatever? TNG is too much mainstream and sounds just too clumsy...

Thunder
--
--./../...-/. -.--/---/..-/.-./..././.-../..-. .---/..-/.../- .-
--/../-./..-/-/./--..-- ../.----./.-../.-.. --./../...-/. -.--/---/..-
.- -/---/--/---/.-./.-./---/.--/.-.-.-
--./.-/-.../.-./.././.-../.-.-.-

2002-08-20 00:12:04

by Daniel Egger

[permalink] [raw]
Subject: Re: IDE?

Am Mon, 2002-08-19 um 22.23 schrieb Timothy D. Witham:

> I am impressed with the serial ATA and the couple of products
> that I've seen so far. I think that this could remove the last barrier
> for ATA drives in larger systems and wait to see what products people
> come up with in the near future.

Not unless manufacturers like IBM stop producing cheesy ATA drives which
are not meant for real productive use.

--
Servus,
Daniel


Attachments:
signature.asc (189.00 B)
Dies ist ein digital signierter Nachrichtenteil

2002-08-20 07:43:51

by Sean Neakums

[permalink] [raw]
Subject: Re: IDE?

commence Thunder from the hill quotation:

> Hi,
>
> On Fri, 16 Aug 2002, Linus Torvalds wrote:
>> - phase 2: IDE-TNG.
>
> Couldn't we give it another name? Such as LUI - Linux Unified IDE?
> Whatever? TNG is too much mainstream and sounds just too clumsy...

Call it IDE-DS9, then.

--
/ |
[|] Sean Neakums | Questions are a burden to others;
[|] <[email protected]> | answers a prison for oneself.
\ |

2002-08-20 08:00:30

by Andre Hedrick

[permalink] [raw]
Subject: IDE-DGD update.

On Tue, 20 Aug 2002, Sean Neakums wrote:

> commence Thunder from the hill quotation:
>
> > Hi,
> >
> > On Fri, 16 Aug 2002, Linus Torvalds wrote:
> >> - phase 2: IDE-TNG.
> >
> > Couldn't we give it another name? Such as LUI - Linux Unified IDE?
> > Whatever? TNG is too much mainstream and sounds just too clumsy...
>
> Call it IDE-DS9, then.

Call it anything, I do not care.
However you should because, I will guarentee it works in closed form.
It may be stick ugly coding, but you can bet your data it will be just
where you put it when you want it back.

With the saving grace of Alan Cox and company it will be kernel correct
and maybe pleasing to read if you want, and you can still bet your data it
will be just where you put it when you want it back.

If you want a preview of clean up and broad sweeping design rework.

2.4.20-pre2-ac5 and in less than 24hours hopefully ac6.

We will be bringing to the table, hot-plugable HBA.
This will include fully modular HOST controllers.
The short side is we are restricting unloading of controllers for now.

Cheers,


Andre Hedrick
LAD Storage Consulting Group

2002-08-20 10:15:13

by Luigi Genoni

[permalink] [raw]
Subject: Re: Does Solaris really scale this well?


80% is quite possible, I have similar results with a E10K domain of
around 32 CPUs, with a 100mhz bus. Buf 80% is far from 94%...



On Tue, 20 Aug 2002, Peter Chubb wrote:

> Date: Tue, 20 Aug 2002 08:58:06 +1000
> From: Peter Chubb <[email protected]>
> To: [email protected]
> Cc: Larry McVoy <[email protected]>,
> Ruth Ivimey-Cook <[email protected]>,
> Matti Aarnio <[email protected]>, Dax Kelson <[email protected]>,
> "[email protected]" <[email protected]>
> Subject: Re: Does Solaris really scale this well?
>
> >>>>> "venom" == venom <[email protected]> writes:
>
> venom> On Sat, 17 Aug 2002, Larry McVoy wrote:
> >> Date: Sat, 17 Aug 2002 17:55:17 -0700 From: Larry McVoy
>
> venom> And where reasonable != 94%. Seriously, 94% scalability could
> venom> be on a 8 CPUs 880, but, for example, I have a 64 CPUS domain
> venom> on a E10k which is far from 94% scalability (ok, an old E10k
> venom> with an 83Mhz bus). For what I saw, maybe SGI Origin 3000 is
> venom> scaling a little better with a lot of CPUS, but I also never
> venom> had an E15000 around for now...
>
> I've played around with 8-way E10000 and a 128-way Origin.
> Both scaled reasonably from an OS perspective --- enabling more cpus on
> a mixed lots-of-small-jobs workload increased performance close to
> linearly --- from memory (and it was a couple of years ago) above
> 80%, and in some tests better than that. Unfotunately, I no longer
> have access either to the machines or to the data, as I've changed jobs...
>
> Peter C
>

2002-08-20 12:09:09

by Jakob Oestergaard

[permalink] [raw]
Subject: Re: Does Solaris really scale this well?

On Tue, Aug 20, 2002 at 12:13:44PM +0200, [email protected] wrote:
>
> 80% is quite possible, I have similar results with a E10K domain of
> around 32 CPUs, with a 100mhz bus. Buf 80% is far from 94%...
>

[shortening CC: to save electrons]

Let's end this thread shall we.

Anyone talking about scalability without talking about workload and
measurement is just spreading BS.

Anyone can get 100% (or even superlinear) scalability on a 1000
processor i486 with a 1 KHz bus, if the workload is right and
performance is measured "right".

The same person can get negative scalability on any THz (my-privates-
are-longer-and-fatter-than-yours)-bus machine out there, if the workload
is again chosen "correctly".

The thread could have gone on-topic, but it didn't :)

--
................................................................
: [email protected] : And I see the elder races, :
:.........................: putrid forms of man :
: Jakob ?stergaard : See him rise and claim the earth, :
: OZ9ABN : his downfall is at hand. :
:.........................:............{Konkhra}...............:

2002-08-20 18:53:07

by Gunther Mayer

[permalink] [raw]
Subject: Re: IDE?

Andre Hedrick wrote:

>
>That is how I do it, since I have a code base that has been run over a
>bus analyzer I know it works.
>
It would be interesting to run on a bus simulator to show the driver
behaves correct on all allowed conditions and sensible on error conditions
(i.e. printk and return an error to the application; don't hang the system).


2002-08-21 07:28:22

by Andre Hedrick

[permalink] [raw]
Subject: Re: IDE?

On Tue, 20 Aug 2002, Gunther Mayer wrote:

> Andre Hedrick wrote:
>
> >
> >That is how I do it, since I have a code base that has been run over a
> >bus analyzer I know it works.
> >
> It would be interesting to run on a bus simulator to show the driver
> behaves correct on all allowed conditions and sensible on error conditions
> (i.e. printk and return an error to the application; don't hang the system).

Greets Gunther,

Hmmm, would you expand on the direction you are point to go?

Cheers,

Andre Hedrick
LAD Storage Consulting Group

2002-08-21 10:18:31

by Vojtech Pavlik

[permalink] [raw]
Subject: Re: IDE?

On Sun, Aug 18, 2002 at 01:16:04PM +0100, Alan Cox wrote:
> On Sun, 2002-08-18 at 12:15, Vojtech Pavlik wrote:
> > I'll make patches for 2.5 to bring the low-level driver cleanups back.
> > Not just piix.c - also aec62xx.c and amd74xx.c - the last one was in 2.5
> > for a LONG time already and I'm not particularly happy it got lost.
> >
> > If desirable (What's your opinion, Alan?) I can make equivalent patches
> > for 2.4 as well.
>
> Look at 2.4.20-pre2-ac3 before you start doing that. A lot of cleanup
> has been done, although there is plenty more left. A starter is to fix
> the the ratemask/ratefilter stuff to not use silly while loops on the
> aec/amd drivers if you are hacking on those, stick in the static
> variables and document anything relevant looking.
>
> Simple stuff first.

I have completely rewritten (and very well tested) versions of the amd
and piix pci ide drivers.

I'm now looking through 2.4.20-pre2-ac5 and your version of via82cxxx.c,
and all looks quite good to me, except for some of the indentation
changes which seem to make the code fit into 78 columns at the loss of
readability. Was the file run through indent?

I'm planning to adapt the amd and piix driver to the new framework for
IDE drivers and then send you a patch.

--
Vojtech Pavlik
SuSE Labs

2002-08-21 13:16:22

by Alan

[permalink] [raw]
Subject: Re: IDE?

On Wed, 2002-08-21 at 11:17, Vojtech Pavlik wrote:
> I have completely rewritten (and very well tested) versions of the amd
> and piix pci ide drivers.

I have completely non-rewritten piix drivers that work extremely well
are now easy to read, commented and have done for a very long time. Why
do I want rewritten ones ?

> I'm now looking through 2.4.20-pre2-ac5 and your version of via82cxxx.c,
> and all looks quite good to me, except for some of the indentation
> changes which seem to make the code fit into 78 columns at the loss of
> readability. Was the file run through indent?

Andre may have indented it a bit. I've probably caused a bit of noise in
checking all the static's etc

2002-08-21 13:26:58

by Vojtech Pavlik

[permalink] [raw]
Subject: Re: IDE?

On Wed, Aug 21, 2002 at 02:20:07PM +0100, Alan Cox wrote:
>
> On Wed, 2002-08-21 at 11:17, Vojtech Pavlik wrote:
> > I have completely rewritten (and very well tested) versions of the amd
> > and piix pci ide drivers.
>
> I have completely non-rewritten piix drivers that work extremely well
> are now easy to read, commented and have done for a very long time. Why
> do I want rewritten ones ?

Even nicer? Easier to add new devices? Not having switch(deviceid)
everywhere? Don't know. I believe they're better. In any case it's a lot
of good work thrown away, yours or mine. :(

The two drivers in question attached in their form suitable for
2.4-non-ac, you can take a look. If there is a chance to get them into
2.4-ac/2.5, I can change them to fit the kernels as needed.

> > I'm now looking through 2.4.20-pre2-ac5 and your version of via82cxxx.c,
> > and all looks quite good to me, except for some of the indentation
> > changes which seem to make the code fit into 78 columns at the loss of
> > readability. Was the file run through indent?
>
> Andre may have indented it a bit. I've probably caused a bit of noise in
> checking all the static's etc

It's awful in my opinion, others may differ. Would you mind if I sent
you patch that'd put the indentation back, while keeping all the code as
you have it now?

--
Vojtech Pavlik
SuSE Labs


Attachments:
(No filename) (1.32 kB)
piix.c (15.88 kB)
amd74xx.c (12.72 kB)
Download all attachments

2002-08-21 14:02:24

by Alan

[permalink] [raw]
Subject: Re: IDE?

On Wed, 2002-08-21 at 14:27, Vojtech Pavlik wrote:
> Even nicer? Easier to add new devices? Not having switch(deviceid)
> everywhere? Don't know. I believe they're better. In any case it's a lot
> of good work thrown away, yours or mine. :(

I don't mind a few deviceid switches. I don't find them annoying and
I'll take the stability of using known good code with the daft bits like
the rate filter while loop fixed over the unknowns of changing the code.

Once its in 2.4 and 2.5 and it works then I'm all ears, because at that
point we can actively quantify and test for regressions

2002-08-21 14:13:38

by Vojtech Pavlik

[permalink] [raw]
Subject: Re: IDE?

On Wed, Aug 21, 2002 at 03:06:18PM +0100, Alan Cox wrote:

> On Wed, 2002-08-21 at 14:27, Vojtech Pavlik wrote:
> > Even nicer? Easier to add new devices? Not having switch(deviceid)
> > everywhere? Don't know. I believe they're better. In any case it's a lot
> > of good work thrown away, yours or mine. :(
>
> I don't mind a few deviceid switches. I don't find them annoying and
> I'll take the stability of using known good code with the daft bits like
> the rate filter while loop fixed over the unknowns of changing the code.
>
> Once its in 2.4 and 2.5 and it works then I'm all ears, because at that
> point we can actively quantify and test for regressions

I can wait. I've done the rewrite back in 2000, so a couple more months ...
Also, I don't necessarily push it into 2.4. However, if you want to keep
a common IDE base in both, there is no other way.

--
Vojtech Pavlik
SuSE Labs

2002-08-23 19:49:20

by Pavel Machek

[permalink] [raw]
Subject: Re: IDE?

Hi

> - phase 2: IDE-TNG. Leave the current IDE code unchanged, and plan to
> obsolete it. It's the "stable IDE", and by virtue of being stable,
> nobody will mind work on new drivers that (by definition) cannot screw
> up unless you use them.
>
> IDE-TNG would:
> - be controller-specific (ie one driver for one controller family)
> - be able to say "screw it" for old or broken setups (which are left
> fot the old IDE code)
> - in particular, it would only bother with PCI (or better)
> controllers, and with UDMA-only setups.

Would it be possible to use martin's IDE as starting point for IDE-TNG? Awfull
lot of cleanups went to that code...
Pavel
--
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.