H.P. Anvin wrote:
>Please consider deprecating or removing ide-floppy/ide-tape/ide-cdrom
>and treat all ATAPI devices as what they really are -- SCSI over IDE.
>It is a source of no ending confusion that a Linux system will not
>write CDs to an IDE CD-writer out of the box, for the simple reason
>that cdrecord needs access to the generic packet interface, which is
>only available in the nonstandard ide-scsi configuration.
Thank you for this thread!
libscg now has 5 different SCSI transport interface implementations
only for Linux.
There are still problems like e.g. USB which not really usable.
>There really seems to be no decent reason to treat ATAPI devices as
>anything else. I understand the ide-* drivers contain some
>workarounds for specific devices, but those really should be moved to
>their respective SCSI drivers anyway -- after all, manufacturers
>readily slap IDE or SCSI interfaces on the same devices anyway.
>Note that this is specific to ATAPI devices. ATA hard drives are
>another matter entirely.
A reasonable idea would be to make the ATA driver a SCSI HBAdriver and
in case that there is a need for pure ATA (e.g. Hard disks) there should
be a second interface for a ATA driver stack.
Please keep me on the CC: I am not on the list.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>Alan Cox wrote:
>> Please consider deprecating or removing ide-floppy/ide-tape/ide-cdrom
>> and treat all ATAPI devices as what they really are -- SCSI over IDE.
>They aren't.
Please don't start again to tell onprooven statements....
We had a similar discussion in January 2001 and you did not give a proove
for this statement.
>o Not all ide cdrom devices are ATAPI capable
Name one drive made after 1993, one drive made after 1995 and one drive made
after 2000 to verify your statement.
>o Many ide floppy devices can do ATAPI but get it horribly wrong
Describe the problems.
>o ide-tape is -totally- weird and unrelated to st
Describe problems and name drives that will not work via ATAPI.
>Andre did the framework ready to move to a situation where you could open
>either the ide-scsi or the ide-cdrom name without module reloading mess.
>You'd need to ask our new 2.5 IDE maintainer to finish that work off.
This is a really bad idea!
The result of such bad hacks is that nobody really tests whether the new
code works.
Let me give an example:
If you try to put ide-scsi on top of PCATA (the interface that is used
in notebooks to connect external disks and CD-writers).
Depending on the kernel version, this either causes a system panic or
just does not work at all. As all ATAPI CD-writers and CD-rom drives
have a fallback to ATA commands, nobody who does not like to use a writer
will ever notice the problem. They simply access the CD-ROM as read only
ATA disk. If ide-cd would have been banned this bug would have been fixed years
ago.
>For disk it gets much easier. Linus has already said he wants a single
>'disk' device, which once we get 32bit dev_t will be nice. With that we
>can finally turn aacraid, megaraid and other 'fake scsi' devices back to
>raw block devices without breaking compatibility assumptions, and get more
>throughput.
Sorry, this has nothing to do with dev_t
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
On Fri, 2002-07-12 at 20:37, Joerg Schilling wrote:
> >o Many ide floppy devices can do ATAPI but get it horribly wrong
>
> Describe the problems.
Go read the source code, do your own homework
> Depending on the kernel version, this either causes a system panic or
> just does not work at all. As all ATAPI CD-writers and CD-rom drives
> have a fallback to ATA commands, nobody who does not like to use a writer
> will ever notice the problem. They simply access the CD-ROM as read only
> ATA disk. If ide-cd would have been banned this bug would have been fixed years
> ago.
Which wouldnt work, because thats not what the CD interface part is
about
> >For disk it gets much easier. Linus has already said he wants a single
> >'disk' device, which once we get 32bit dev_t will be nice. With that we
> >can finally turn aacraid, megaraid and other 'fake scsi' devices back to
> >raw block devices without breaking compatibility assumptions, and get more
> >throughput.
>
>
> Sorry, this has nothing to do with dev_t
It has a huge amount to do with dev_t. It should be immediately obvious
why dev_t is a critical factor in getting that interface working in a
sane fashion.
>From [email protected] Fri Jul 12 21:43:27 2002
>On Fri, 2002-07-12 at 20:37, Joerg Schilling wrote:
>> >o Many ide floppy devices can do ATAPI but get it horribly wrong
>>
>> Describe the problems.
>Go read the source code, do your own homework
Fine! You repeat that you have no argument that stands a check.
So let us take it as prooven that there is no problem with resent
(< 10 year old) drives.
>>
>> Sorry, this has nothing to do with dev_t
>It has a huge amount to do with dev_t. It should be immediately obvious
>why dev_t is a critical factor in getting that interface working in a
>sane fashion.
If a sane driver interface depends on dev_t being 32 bits, then there
would be a lot og junk in the Linux kernel :-(
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
Hi,
On Fri, 12 Jul 2002, Joerg Schilling wrote:
> The result of such bad hacks is that nobody really tests whether the new
> code works.
I want you to prove the three things your statement says:
- that this code is only implementable via a bad hack, and your idea is
implementable via a better hack
- Andre's Code is - without a look at it - ugly
- Unification changesets stay untested
> Depending on the kernel version, this either causes a system panic or
> just does not work at all. As all ATAPI CD-writers and CD-rom drives
> have a fallback to ATA commands, nobody who does not like to use a
> writer will ever notice the problem. They simply access the CD-ROM as
> read only ATA disk. If ide-cd would have been banned this bug would have
> been fixed years ago.
Because we can't tell Linux users "your (once favorite) CD-ROM is not
implemented in Linux (any more), and will never ever be". If we explicitly
exclude hardware, where do we end?!
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
Erik Andersen wrote:
>cdrecord should use the CDROM_SEND_PACKET ioctl, then it would
>work regardless,
Wis you ever look at the cdrecord sources?
Cdrecord relies on libscg which is a generic SCSI transport library.
It has been first written in August 1986 when I wrote the first SCSI
pass through driver (for SunOS-3.0) - long before Adapted came out with
ASPI. In the 16 years of evolution, it has been ported to > 30
different platforms (not including CPU variants like sparc/x86).
If you force cdrecord to rely on CD-ROM only interfaces, you make Linux
unusable in general. Do you really like to create an unusable Linux just
to avoid creating a usable generic SCSI transport interface?
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From [email protected] Fri Jul 12 21:56:54 2002
>On Fri, 2002-07-12 at 20:49, Joerg Schilling wrote:
>> >> Describe the problems.
>>
>> >Go read the source code, do your own homework
>>
>> Fine! You repeat that you have no argument that stands a check.
>> So let us take it as prooven that there is no problem with resent
>> (< 10 year old) drives.
>Read the source code, then you'd already know you are talking crap
If you are unable to use arguments, I cannot take you for serious.
Please educate yourself about SCSI and ATAPI, this would help a lot
haveing a serious discussion.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
On Fri, 2002-07-12 at 20:49, Joerg Schilling wrote:
> >> Describe the problems.
>
> >Go read the source code, do your own homework
>
> Fine! You repeat that you have no argument that stands a check.
> So let us take it as prooven that there is no problem with resent
> (< 10 year old) drives.
Read the source code, then you'd already know you are talking crap
Hi,
On Fri, 12 Jul 2002, Joerg Schilling wrote:
> >> Describe the problems.
>
> >Go read the source code, do your own homework
>
> Fine! You repeat that you have no argument that stands a check.
No, he just told you to do the checking on your own.
> So let us take it as prooven that there is no problem with resent
> (< 10 year old) drives.
I got a drive at home which is from 1996 and doesn't do ATAPI. It's some
Mitsumi indestructible, but I can't tell you details since I won't be at
home within the next few weeks.
> >It has a huge amount to do with dev_t. It should be immediately obvious
> >why dev_t is a critical factor in getting that interface working in a
> >sane fashion.
>
> If a sane driver interface depends on dev_t being 32 bits, then there
> would be a lot og junk in the Linux kernel :-(
Say the same about any kind of vars, and watch us laugh...
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
On Fri, 2002-07-12 at 20:57, Joerg Schilling wrote:
> If you are unable to use arguments, I cannot take you for serious.
> Please educate yourself about SCSI and ATAPI, this would help a lot
> haveing a serious discussion.
Unlike you I've bothered not only to learn about ATA, ATAPI and SCSI but
also to experience real world hardware, much of it less than ten years
old and some of which (especially in the USB world) can't even get
INQUIRY right let alone actually do I/O properly.
CD burning is a side issue to stability and reliability.
In terms of supporting old hardware most of that is irrelevant to cd
recording anyway, so why do you care ? What you actually need is a
generic interface for cd packet sending.
Joerg
Don't take this personally but you are just a top level application.
The only reason you have such blinders on is because you always had a
bottom layer transport working around and doing its best to prevent
errors form showing up. You are wrong and because you are not down on the
bus at the physical layer you do see the mess.
Now your silly PCATA stupid ass Tailgate Bridge that you are boasting
about does some of the worst things anyone could ever imagine.
Oh and the bad idea you call is to permit dynamic subdriver shifting.
Now that it may never be completed you have the advantage to call it a bad
idea. I suspect you are an ASPI lover and soone SPI will die.
Next your statements about name a drive is a straw man.
More basic proof you do not look down at the hardware.
BurnProof is a result lame devices which improperly hold off the bus
because release the BUSY Bit while still performing transfers.
The very fact that a huge pile of devices went into the market place
based on SFF-8020 rev 2.5 total roasts your strawman, please try again.
Also you blanket term is incorrect to say all CDRW/ROMs fall back to ATA
transport. You can not obtain any data other than identify and even it
as a special command which will not work on ATA devices.
There is a single command for ATAPI devices called PACKET_COMMAND.
Also all ATA devices shall issue an abort should it recieve the command.
As for ide-cd being banded, if you had ever attempted to support a direct
access cdb packet driver by not restricting cdrecord to only a SG
interface we would have solved the problem a long time ago too.
So neer neer neer and that leaves us at the same point.
So instead of whining about what is there and not from your location in
end user land, try and offer something useful like a preferred API to
allow clean packet-driver interfaces. Doing that little would allow the
transport layer people and the transistion-api folks to user land to
greatly increase compatablity. Then you would not need 5 interfaces for
Linux.
Have a good day.
Cheers,
Andre Hedrick
LAD Storage Consulting Group
On Fri, 12 Jul 2002, Joerg Schilling wrote:
> >Alan Cox wrote:
>
> >> Please consider deprecating or removing ide-floppy/ide-tape/ide-cdrom
> >> and treat all ATAPI devices as what they really are -- SCSI over IDE.
>
> >They aren't.
>
> Please don't start again to tell onprooven statements....
> We had a similar discussion in January 2001 and you did not give a proove
> for this statement.
>
>
> >o Not all ide cdrom devices are ATAPI capable
>
> Name one drive made after 1993, one drive made after 1995 and one drive made
> after 2000 to verify your statement.
>
> >o Many ide floppy devices can do ATAPI but get it horribly wrong
>
> Describe the problems.
>
> >o ide-tape is -totally- weird and unrelated to st
>
> Describe problems and name drives that will not work via ATAPI.
>
> >Andre did the framework ready to move to a situation where you could open
> >either the ide-scsi or the ide-cdrom name without module reloading mess.
> >You'd need to ask our new 2.5 IDE maintainer to finish that work off.
>
> This is a really bad idea!
>
> The result of such bad hacks is that nobody really tests whether the new
> code works.
>
> Let me give an example:
>
> If you try to put ide-scsi on top of PCATA (the interface that is used
> in notebooks to connect external disks and CD-writers).
>
> Depending on the kernel version, this either causes a system panic or
> just does not work at all. As all ATAPI CD-writers and CD-rom drives
> have a fallback to ATA commands, nobody who does not like to use a writer
> will ever notice the problem. They simply access the CD-ROM as read only
> ATA disk. If ide-cd would have been banned this bug would have been fixed years
> ago.
>
> >For disk it gets much easier. Linus has already said he wants a single
> >'disk' device, which once we get 32bit dev_t will be nice. With that we
> >can finally turn aacraid, megaraid and other 'fake scsi' devices back to
> >raw block devices without breaking compatibility assumptions, and get more
> >throughput.
>
>
> Sorry, this has nothing to do with dev_t
>
> J?rg
>
> EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
> [email protected] (uni) If you don't have iso-8859-1
> [email protected] (work) chars I am J"org Schilling
> URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>Alan Cox wrote:
>> In favour of the scrap:
>>
>> 1. HPA.
>> 2. Adam J. Richter.
>> 3. Marcin Dalecki (basically due to give up on the idea
>> of gradual unification).
>In other words nobody who understands IDE is for and everyone who
>understands you can't actually get rid of ide-floppy, tape, cdrom internal
>support and knows about IDE is..
I am not sure if any of your statements before does proove that you have
knowledge on IDE, you definitely seem to miss important knowledge about SCSI.
I hope that this will not result in missing the chance of converting
to a more useful driver concept.
There are enough other OS that use a common ATAPI/SCSI driver concept and
do not have the problems you vagely name but never really describe.
I believe it's OK to drop support for 10 year old hardware in case this
is no important hardware that _really_ needs continued support (like
e.g. 9 track trapes).
ATA devices that are neither hard disks, nor do support ATAPI decently
are Y 1992 crap - so unless you like to have continued support for your
provate museum, what is the reason that you like to prevent a change
to a more usable driver interface?
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
On Fri, 12 Jul 2002, Joerg Schilling wrote:
> Erik Andersen wrote:
>
> >cdrecord should use the CDROM_SEND_PACKET ioctl, then it would
> >work regardless,
>
> Wis you ever look at the cdrecord sources?
>
> Cdrecord relies on libscg which is a generic SCSI transport library.
> It has been first written in August 1986 when I wrote the first SCSI
> pass through driver (for SunOS-3.0) - long before Adapted came out with
> ASPI. In the 16 years of evolution, it has been ported to > 30
> different platforms (not including CPU variants like sparc/x86).
>
> If you force cdrecord to rely on CD-ROM only interfaces, you make Linux
> unusable in general. Do you really like to create an unusable Linux just
> to avoid creating a usable generic SCSI transport interface?
> J?rg
>
> EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
> [email protected] (uni) If you don't have iso-8859-1
> [email protected] (work) chars I am J"org Schilling
> URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
> -
As much as I hate IDE, IDE isn't going away. All my systems use SCSI
so on machines that have CD/ROMS, I use your libraries and your tools.
Maybe somebody should make CD/ROM code that directly talks to IDE via
/dev/hdwhatever, instead of expecting you to modify your code that
has worked so well for so long.
SCSI, and your CD/ROM code isn't broken. It does not need to be fixed.
IDE drives are not SCSI drives. They shouldn't be logically connected
at any level below 'block-device'. Some block devices are like SCSI,
both the Fire-wire drives and USB drives are, for instance. IDE drives
are like floppies. You write controller-specific commands to the
controller(s). They don't belong 'connected' to SCSI in any logical
way.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
Windows-2000/Professional isn't.
On Fri, 2002-07-12 at 21:08, Joerg Schilling wrote:
> There are enough other OS that use a common ATAPI/SCSI driver concept and
> do not have the problems you vagely name but never really describe.
There are lots that fudge around and pretend scsi is the block layer
when it is not. That sort of misses the point and slows down high end
raid cards.
> I believe it's OK to drop support for 10 year old hardware in case this
> is no important hardware that _really_ needs continued support (like
> e.g. 9 track trapes).
We support and people continue to use large amounts of ten year old
hardware. Thats why children at quite a few schools have access to
computing through things like LTSP. There are people actively
maintaining much of this stuff too.
Some things such as ide tapes which have needs rather different to scsi
tape are still being made and released in newer and larger forms.
> ATA devices that are neither hard disks, nor do support ATAPI decently
> are Y 1992 crap - so unless you like to have continued support for your
> provate museum, what is the reason that you like to prevent a change
> to a more usable driver interface?
You still don't seem to understand the difference between a driver
interface and a hardware layer.
Alan
Hi,
BTW, a famous quote about this issue (accessing IDE devices over a SCSI
layer):
> Hardware is different.
> You can paint a goose yellow and call it a duck, but it is still a goose.
> The electrical/electronic interface will kill you!
-- Andre Hedrick
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
On Fri Jul 12, 2002 at 09:55:26PM +0200, Joerg Schilling wrote:
> Erik Andersen wrote:
>
> >cdrecord should use the CDROM_SEND_PACKET ioctl, then it would
> >work regardless,
>
> Wis you ever look at the cdrecord sources?
Yes, I have read the cdrecord source. As you may recall from the
bug reports I would send periodically, I maintained the Debian
cdrecord/cdrtools package from 1998 till late last year...
> Cdrecord relies on libscg which is a generic SCSI transport library.
> It has been first written in August 1986 when I wrote the first SCSI
> pass through driver (for SunOS-3.0) - long before Adapted came out with
> ASPI. In the 16 years of evolution, it has been ported to > 30
> different platforms (not including CPU variants like sparc/x86).
Yup. It's very portable, and I know you are very proud of libscg.
For Linux you have made libscg work using the "broken Linux SCSI
generic driver", and you have gone to great lengths to describe
how much you hate it.
> If you force cdrecord to rely on CD-ROM only interfaces, you make Linux
> unusable in general. Do you really like to create an unusable Linux just
> to avoid creating a usable generic SCSI transport interface?
Lets step back a moment here. The cdrecord package is not
responsible for making "Linux usable in general". It is
responsible for writing data to CD-ROMs. It is _not_ responsible
for driving scanners, hard drives, or enforcing policy on the
Linux kernel.
If you would throw away crdrecord's desire to do its own private
SCSI bus scanning, and throw away your attachment to addressing
devices only by host, channel, id, and lun a number of things
happen. For starters, Linux devices don't have to be forced to
all be sitting on the SCSI bus. You could use standard Linux
device names (i.e. /dev/hdc or /dev/scd0). And you could still
send all the SCSI/ATAPI packet commands you want to the device
that was selected using the CDROM_SEND_PACKET ioctl.
Ever look at the CDROM_SEND_PACKET ioctl?
-Erik
--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--
On Fri Jul 12, 2002 at 10:17:21PM +0100, Alan Cox wrote:
> CD burning is a side issue to stability and reliability.
>
> In terms of supporting old hardware most of that is irrelevant to cd
> recording anyway, so why do you care ? What you actually need is a
> generic interface for cd packet sending.
A generic interface for cd packet sending? Sounds useful. So
useful someone thought of it years ago, and called it the
CDROM_SEND_PACKET ioctl. Its been in the kernel since Aug 1999.
What'll those crazy Linux CD-ROM people think of next?
-Erik
--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--
> Because we can't tell Linux users "your (once favorite) CD-ROM is not
> implemented in Linux (any more), and will never ever be". If we explicitly
> exclude hardware, where do we end?!
Like other mainstream operating systems :-)
One thing that occurs to me, but that I don't necessarily think is a good idea, is that for a long time we had "old" IDE code and "new" IDE code in the kernel, and there is no reason why we couldn't do a similar thing, (I.E. have a "legacy devices will work" foo driver, and "legacy devices might break" foo driver). Personally I hate that idea, but maybe others will disagree. I'd rather support all hardware forever, because the whole point is that you can compile out what you don't want, and supporting it is not causing excessive bloat.
John.
On Fri, Jul 12, 2002 at 11:40:58PM -0600, Erik Andersen wrote:
> Yes, I have read the cdrecord source. As you may recall from the
> bug reports I would send periodically, I maintained the Debian
> cdrecord/cdrtools package from 1998 till late last year...
> Ever look at the CDROM_SEND_PACKET ioctl?
Support for that appeared in cdrecord some time ago. From what little I've
used it, it seems to work fine even though Joerg calls it pre-alpha code.
On Sat, 2002-07-13 at 07:36, [email protected] wrote:
> > Because we can't tell Linux users "your (once favorite) CD-ROM is not
> > implemented in Linux (any more), and will never ever be". If we explicitly
> > exclude hardware, where do we end?!
>
> Like other mainstream operating systems :-)
>
> One thing that occurs to me, but that I don't necessarily think is a good idea,
> is that for a long time we had "old" IDE code and "new" IDE code in the kernel,
> and there is no reason why we couldn't do a similar thing, (I.E. have
> a "legacy devices will work" foo driver, and "legacy devices might
So we'd have a legacy driver called oh say 'ide-cd' and a current one
called 'ide-scsi'.
How does that change anything ?
On 13 Jul 2002, Alan Cox wrote:
> On Sat, 2002-07-13 at 07:36, [email protected] wrote:
> > > Because we can't tell Linux users "your (once favorite) CD-ROM is not
> > > implemented in Linux (any more), and will never ever be". If we explicitly
> > > exclude hardware, where do we end?!
> >
> > Like other mainstream operating systems :-)
> >
> > One thing that occurs to me, but that I don't necessarily think is a good idea,
> > is that for a long time we had "old" IDE code and "new" IDE code in the kernel,
> > and there is no reason why we couldn't do a similar thing, (I.E. have
> > a "legacy devices will work" foo driver, and "legacy devices might
>
> So we'd have a legacy driver called oh say 'ide-cd' and a current one
> called 'ide-scsi'.
>
> How does that change anything ?
It gives us the possibility to create a "clean" design for modern hardware
while maintaining support for "legacy" hardware. You don't have to carry
around a lot of special cases and distort the design to take into account
something that almost no one uses.
One of the reasons I despise that other operating system is the way it
carries around bloat, cruft, and crap(!) to support ancient DOS stuff.
Having your legacy and current drivers gives the best of both worlds,
supporting ancient stuff while maintaining clean code for modern stuff.
> > > Because we can't tell Linux users "your (once favorite) CD-ROM is not
> > > implemented in Linux (any more), and will never ever be". If we explicitly
> > > exclude hardware, where do we end?!
> >
> > Like other mainstream operating systems :-)
> >
> > One thing that occurs to me, but that I don't necessarily think is a good idea,
> > is that for a long time we had "old" IDE code and "new" IDE code in the kernel,
> > and there is no reason why we couldn't do a similar thing, (I.E. have
> > a "legacy devices will work" foo driver, and "legacy devices might
>
> So we'd have a legacy driver called oh say 'ide-cd' and a current one
> called 'ide-scsi'.
>
> How does that change anything ?
Both of the existing drivers would become the legacy driver, and a new one would be forked from the legacy code, which abstracts the physical interface altogether. Eventually, we're going to have IDE, ATAPI (I.E. mostly non-disk IDE devices), SCSI, SASI(maybe :-)), USB, FireWire, Bluetooth, etc. The number of interfaces is just going to grow and grow.
On Sat, 2002-07-13 at 14:55, Thomas Molina wrote:
> > So we'd have a legacy driver called oh say 'ide-cd' and a current one
> > called 'ide-scsi'.
> >
> > How does that change anything ?
>
> It gives us the possibility to create a "clean" design for modern hardware
> while maintaining support for "legacy" hardware. You don't have to carry
> around a lot of special cases and distort the design to take into account
You missed the point. We -ALREADY- have ide-cd and ide-scsi. We already
do what is described, and in fact Martin and co want to remove that
seperation and stuff the gunge into the general atapi layer
On Sat, 2002-07-13 at 15:07, [email protected] wrote:
> Both of the existing drivers would become the legacy driver, and a new one would be forked from the legacy code, which abstracts the physical interface altogether. Eventually, we're going to have IDE, ATAPI (I.E. mostly non-disk IDE devices), SCSI, SASI(maybe :-)), USB, FireWire, Bluetooth, etc. The number of interfaces is just going to grow and grow.
I look forward to seeing your patches
> > Both of the existing drivers would become the legacy driver, and a new one would be forked from the legacy code, which abstracts the physical interface altogether. Eventually, we're going to have IDE, ATAPI (I.E. mostly non-disk IDE devices), SCSI, SASI(maybe :-)), USB, FireWire, Bluetooth, etc. The number of interfaces is just going to grow and grow.
>
> I look forward to seeing your patches
... if only I could get that 'Hello World' code debugged I'd be well on my way, eh? :-)
On 13 Jul 2002, Alan Cox wrote:
> On Sat, 2002-07-13 at 14:55, Thomas Molina wrote:
> > > So we'd have a legacy driver called oh say 'ide-cd' and a current one
> > > called 'ide-scsi'.
> > >
> > > How does that change anything ?
> >
> > It gives us the possibility to create a "clean" design for modern hardware
> > while maintaining support for "legacy" hardware. You don't have to carry
> > around a lot of special cases and distort the design to take into account
>
> You missed the point. We -ALREADY- have ide-cd and ide-scsi. We already
> do what is described, and in fact Martin and co want to remove that
> seperation and stuff the gunge into the general atapi layer
I thought part of the issue was a redesign of the scsi mid-layer. I was
reacting to the scenario where support for older hardware was removed.
Alan Cox writes:
|o Not all ide cdrom devices are ATAPI capable
[...]
Are there some non-ATAPI IDE CDROM's that
linux-2.5.25/drivers/ide/ide-cdrom.c supports? I was under
the impression that ide-cdrom.c operated only through ATAPI.
Adam J. Richter __ ______________ 575 Oroville Road
[email protected] \ / Milpitas, California 95035
+1 408 309-6081 | g g d r a s i l United States of America
"Free Software For The Rest Of Us."
On Sat, 13 Jul 2002, Adam J. Richter wrote:
> Alan Cox writes:
> |o Not all ide cdrom devices are ATAPI capable
> [...]
>
> Are there some non-ATAPI IDE CDROM's that
> linux-2.5.25/drivers/ide/ide-cdrom.c supports? I was under
> the impression that ide-cdrom.c operated only through ATAPI.
Wrong impression. ;)
Hint: look for STANDARD_ATAPI macro usage.
--
Bartlomiej
> Adam J. Richter __ ______________ 575 Oroville Road
> [email protected] \ / Milpitas, California 95035
> +1 408 309-6081 | g g d r a s i l United States of America
> "Free Software For The Rest Of Us."
On Fri, Jul 12, 2002 at 11:40:58PM -0600, Erik Andersen wrote:
> If you would throw away crdrecord's desire to do its own private
> SCSI bus scanning
CDROM_SEND_PACKET sounds nice, but do you have a way to:
1- Find all cdrom-ish devices in the system
2- Find if a given device is cdrom-ish
By cdrom-ish, I mean cdrom, dvdrom, cd writer, dvd writer or a
combination.
Point 1 is necessary to be a minimum user-friendly, point 2 is
necessary because you can't trust users :-)
OG.
On Sat, 13 Jul 2002, Bartlomiej Zolnierkiewicz wrote:
>On Sat, 13 Jul 2002, Adam J. Richter wrote:
[...]
>> Are there some non-ATAPI IDE CDROM's that
>> linux-2.5.25/drivers/ide/ide-cdrom.c supports? I was under
>> the impression that ide-cdrom.c operated only through ATAPI.
>Wrong impression. ;)
>Hint: look for STANDARD_ATAPI macro usage.
It looks like that macro should be renamed to something like
STANDARD_MMC. Everything that that macro controls still appears to
go through ATA Packet Interface encapsulation. Those quirks look like
they would likely be duplicated in a SCSI version of the same drives
anyhow. It should be easy to have sr_mod accomodate those drives.
Adam J. Richter __ ______________ 575 Oroville Road
[email protected] \ / Milpitas, California 95035
+1 408 309-6081 | g g d r a s i l United States of America
"Free Software For The Rest Of Us."
On Sat, 13 Jul 2002, Adam J. Richter wrote:
> On Sat, 13 Jul 2002, Bartlomiej Zolnierkiewicz wrote:
> >On Sat, 13 Jul 2002, Adam J. Richter wrote:
> [...]
> >> Are there some non-ATAPI IDE CDROM's that
> >> linux-2.5.25/drivers/ide/ide-cdrom.c supports? I was under
> >> the impression that ide-cdrom.c operated only through ATAPI.
>
> >Wrong impression. ;)
> >Hint: look for STANDARD_ATAPI macro usage.
>
> It looks like that macro should be renamed to something like
> STANDARD_MMC. Everything that that macro controls still appears to
> go through ATA Packet Interface encapsulation. Those quirks look like
Please verify against sff8020.
> they would likely be duplicated in a SCSI version of the same drives
> anyhow. It should be easy to have sr_mod accomodate those drives.
I can't find them, there are some in sr_vendor.c but they are diffirent
issues.
>
> Adam J. Richter __ ______________ 575 Oroville Road
> [email protected] \ / Milpitas, California 95035
> +1 408 309-6081 | g g d r a s i l United States of America
> "Free Software For The Rest Of Us."
On Sat Jul 13, 2002 at 01:52:17PM +0300, Anssi Saari wrote:
> On Fri, Jul 12, 2002 at 11:40:58PM -0600, Erik Andersen wrote:
> > Yes, I have read the cdrecord source. As you may recall from the
> > bug reports I would send periodically, I maintained the Debian
> > cdrecord/cdrtools package from 1998 till late last year...
>
> > Ever look at the CDROM_SEND_PACKET ioctl?
>
> Support for that appeared in cdrecord some time ago. From what little I've
> used it, it seems to work fine even though Joerg calls it pre-alpha code.
Cool. I've not read over anything newer then cdrtools-1.10.
-Erik
--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--
On Sat Jul 13, 2002 at 12:53:46PM -0400, Olivier Galibert wrote:
> On Fri, Jul 12, 2002 at 11:40:58PM -0600, Erik Andersen wrote:
> > If you would throw away crdrecord's desire to do its own private
> > SCSI bus scanning
>
> CDROM_SEND_PACKET sounds nice, but do you have a way to:
> 1- Find all cdrom-ish devices in the system
> 2- Find if a given device is cdrom-ish
>
> By cdrom-ish, I mean cdrom, dvdrom, cd writer, dvd writer or a
> combination.
>
> Point 1 is necessary to be a minimum user-friendly, point 2 is
> necessary because you can't trust users :-)
Well, one way would be to scan for them directly, i.e.:
for i in /dev/hd* ; do
ioctl(fd, HDIO_GET_IDENTITY, &ident);
if (!(ident.config & 0x4000) && ((ident.config & 0x1f00) >> 8)==5) {
/* Got an ATAPI cdrom drive */
}
for i in /dev/scd* ; do
open ($i, O_RDONLY|O_NONBLOCK)) < 0) {
/* Got a SCSI cdrom drive */
}
Of course making user space do this is pretty lame. But we
have a much better way. Each cdrom device registers with the
uniform cdrom driver, which can easily assign each registered
cdrom device a major and minor. That scanning for cdroms would
be as simple as
for i in /dev/cdrom* ; do
open ($i, O_RDONLY|O_NONBLOCK)) < 0) {
/* Found a cdrom drive */
}
-Erik
--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--
On Sat, 13 Jul 2002, Bartlomiej Zolnierkiewicz wrote:
>On Sat, 13 Jul 2002, Adam J. Richter wrote:
>> On Sat, 13 Jul 2002, Bartlomiej Zolnierkiewicz wrote:
>> >Wrong impression. ;)
>> >Hint: look for STANDARD_ATAPI macro usage.
>>
>> It looks like that macro should be renamed to something like
>> STANDARD_MMC. Everything that that macro controls still appears to
>> go through ATA Packet Interface encapsulation. Those quirks look like
>
>Please verify against sff8020.
I don't know what you mean by this. It's not a question of
whether the behavior being accomodated is conformant or nonconformant
to a standard. The question is whether the accomodations controlled
by the "STANDARD_ATAPI" macro can easily be implemented in sr_mod.
Since the accomodations are translating a couple of numbers that
are repeseneted as binary coded decimal instead of integers (0-255)
on some drives, and sending a slightly different SCSI command to
change discs on a Sanyo three CD changer, it seems that they can easily
be implemented in sr_mod.
>> they would likely be duplicated in a SCSI version of the same drives
>> anyhow. It should be easy to have sr_mod accomodate those drives.
>
>I can't find them, there are some in sr_vendor.c but they are diffirent
From you email address, I would guess that English is not your
first language. While you do write English very well, I think you made
a mistake in understanding what I siad. "It should be easy to
have sr_mod accomodate those drives" means that it would be easy for
someone to write code in the near future to do this (that is, to change
sr_mod.c). It does not mean that it has already been done.
Adam J. Richter __ ______________ 575 Oroville Road
[email protected] \ / Milpitas, California 95035
+1 408 309-6081 | g g d r a s i l United States of America
"Free Software For The Rest Of Us."
Martin Dalecki wrote:
>Workarouns in ide-floppy - ZIP drives and Clock drives.
>
>Those are the main "technical issues" which make one hessitate.
>
>
Sorry, late to this thread. Look, ide-floppy has to deal with real
world devices, which simply don't follow nice, written specifications.
If we treat these devices as standard ATAPI they simply don't work.
And a bunch of planned features which were waiting for 2.5 but are now
in limbo until the IDE subsystem is stable enough for me to work on.
Features include:
Trying again to kill the via chipset/Zip interaction (some people are
still suffering).
Kevin Flemings media detect work: needs ATA commands because the ATAPI
command set simply doesn't return the information we need. Then we can
make ide-floppy drives work *properly* with devfs.
Handling the "special" BIOS settings around LS-120/240/Zip bootable
drives.
Making sure that formatting works properly for non-standard format
capacities (i.e. 1.44MB in LS-120, 32MB in LS-240)
And yes, I have real users asking for these things.
So to me the problem is not to make everything work as SCSI, because
that simply isn't true for ide-floppy devices. They *nearly* work, so
you can get kludgy, "good enough for command line gurus" working with
ide-scsi, but there are some funnies. Does it really make sense to have
IDE/ATAPI kludges down in the SCSI tree?
I much prefer Linus's suggestion of agreeing on the top level API. I
would like to see disks, and removeable media having a single unified
namespace and set of ioctls so that the different user-space programs
don't need to worry about if they are dealing with a SCSI, PPA,
ATAPI-ish, USB, 1394 or whatever comes next drive. Is that work? yes,
but it's also about communication and keeping things in the appropriate
place. Let me hide the horrible things ide-floppy has to do from
user-space, and if that means I/we have to completely re-write the
ioctls etc so be it.
--
Paul
/* ide-floppy maintainer */
Email: [email protected]
Web: http://paulbristow.net
ICQ: 11965223
> I much prefer Linus's suggestion of agreeing on the top level API. I
> would like to see disks, and removeable media having a single unified
> namespace and set of ioctls so that the different user-space programs
> don't need to worry about if they are dealing with a SCSI, PPA,
> ATAPI-ish, USB, 1394 or whatever comes next drive. Is that work? yes,
> but it's also about communication and keeping things in the appropriate
> place. Let me hide the horrible things ide-floppy has to do from
> user-space, and if that means I/we have to completely re-write the
> ioctls etc so be it.
I totally agree, why pick an arbitrary interface, and call it the 'standard', you might as well define your own standard, which suits the needs of supporting all future interfaces, (in the near future, anyway).
John.
Paul, et al,
> So to me the problem is not to make everything work as SCSI, because
> that simply isn't true for ide-floppy devices. They *nearly* work, so
> you can get kludgy, "good enough for command line gurus" working with
> ide-scsi, but there are some funnies. Does it really make sense to have
> IDE/ATAPI kludges down in the SCSI tree?
If they arn't SCSI, don't treat them like they are.
> I much prefer Linus's suggestion of agreeing on the top level API. I
> would like to see disks, and removeable media having a single unified
> namespace and set of ioctls so that the different user-space programs
> don't need to worry about if they are dealing with a SCSI, PPA,
> ATAPI-ish, USB, 1394 or whatever comes next drive. Is that work? yes,
> but it's also about communication and keeping things in the appropriate
> place. Let me hide the horrible things ide-floppy has to do from
> user-space, and if that means I/we have to completely re-write the
> ioctls etc so be it.
Amen to this... It is a very definite need. And if I may add my $0.02 US, a
unified IDE/ATA/ATAPI API for Linux will make Linux more likely to take its
place on the desktop. As it stands, I have to fiddle with things more than I
like to to get my CD-RW to work with both CD-R and Packet Writing.
Matthew D. Pitts.
>Eric Anderson wrote:
>cdrecord should use the CDROM_SEND_PACKET ioctl, then it would
>work regardless,
This only prooves that you are uninformed :-(
In addition, the exiestence of thiid ioctl is just another proof
that there is no overall planning in Linux. Everyone adds new
interfaces without any concept. If Linux likes to convert from
a hobbyist kernel to a professional OS, there is need for overall
planning by people who know about OS concepts.
For now, I only see people who may know a lot about Linux but lack
general OS knowledge. Knowlede does not result from looking at your belly
but from looking at other people's ideas...
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>H. Peter Anvin wrote:
>OK, non-ATAPI devices need not apply, obviously, but the argument
>still applies for ATAPI devices...
It is bad to see, that you are the only person in this thread who
seems to did already think about the problem and who tries to find
a useful solution.
Nearly all other people only track personal wishes and aversions.
To find a useful solution, we need a tecnical and knowledge base
driven idea - not only trying to prevent other people's ideas to become
reality.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
H. Peter Anvin wrote:
I'm talking specifically about ATAPI devices here. As we have already
covered, not all ATA devices are ATAPI, but unless I'm completely off
the wall, ATAPI is SCSI over IDE, and should be able to be driven as
such. The lack of access to that interface using the established
interface mechanisms just bites.
You fund it!
Other people did not even think about the problem, or even lack the
needed knowledge.
Alan Cox is one of the latter ones :-( he only tries to avoid needed
changes ithout any technical reason.
If you have a IDE based CD-ROM drive that does not support ATAPI,
why not handle it the only way it makes sense?
Such a drive is no more than a read-only hard disk and may be accessed
via the HD IDE read interface. You will never be able to use it
to e.g. rip audio data off it.
If there really is a poor school that cannot afford to buy a modern
20-30 Euro CD-ROM drive, then this school can liveas long as they
are able to install the OS from the old (1992) CD-ROM they currently own.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>Alan Cox wrote:
>If you load ide-scsi they are run as ATAPI, whats the problem ? Just don't
>do that for very old ide cdroms or for some ide floppies
Sounds like you got the message to some extent.
Why don't think this to it's reasonable end: Use ATAPI for all modern
(1993..1995 or newer) drives and support what old drives can do - reading
data CD-ROM media via the IDE read interface. This can be done via
the ATA HD drive interface.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>Linus Torvalds wrote:
>I will violently oppose anything that implies that the IDE layer uses the
>SCSI layer normally. No way, Jose. I'm all for scrapping, but the thing
>that should be scrapped is ide-scsi.
Nobody who has a technical backgroupd and knows what to do wuld ever
make such a proposal.
Instead, there needs to be one or more SCSI HA driver as part of the SCSI stack.
This driver also needs to deal with plain ATA in order to be able to
coordinate access.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>H. Peter Anvin wrote:
>hen *please* make a *compatible* interface available to user space.
>This certainly can be done; the parallel port IDE interface stuff had
>exactly such an interface (/dev/pg*) -- we could have a /dev/hg*
>interface presumably. That is an acceptable solution.
I would not call the /dev/pg* nterface a cmpatible interface.
It has advantages to the interface in the ide-cdrom driver in being
able to talk to different types of drives at the end, but it is
another incompatible user interface.
>Note again that this discussion (and it's a discussion, not a voting
>session -- technical pros and cons is what applies) apply to ATAPI (SCSI
>over IDE) only. Alan has already brought up the fact of non-hard disk
>non-ATAPI devices, and IMO those devices are explicitly out of scope.
This is my idea too: CD-ROM drives should be accessed via ATAPI or
handled as ATA disk.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
Bartlomiej Zolnierkiewicz wrote:
Please stop calling ATAPI as SCSI over IDE, it is not. It is Packet
Interface over ATA (IDE). Some ATAPI/SCSI devices are functionally
equivalent because they support the same command set (i.e. MMC).
Did you ever looks at the ATAPI specs?
ATAPI _is_ SCSI over IDE with a few "bugs"/deviations:
- The inquiry format uses wrong protocol version numbers.
This mainly gives provblems with creating Mode sense / mode select
commands. If you know (e.g. from MMC-3) that the transport is
IDE, then you just bump the version to 2 and it works.
- 6-byte SCSI commands in general are not supported. This is not
a real problem for a well designed high level driver.
- Commands that take a long time are allowed to behave as if
the high level code did set the IMMED bit in the CDB.
This can be handled if the high level code handles the
resulting error codes for the following commands to
introduce a wait loop.
- There is no disconnect/reconnect.
This is nothing that high level code should be aware of.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>Thunder wrote:
>Because we can't tell Linux users "your (once favorite) CD-ROM is not
>implemented in Linux (any more), and will never ever be". If we explicitly
>exclude hardware, where do we end?!
It would help if you educate yourself before you wrtite to the thread.
This could help a lot to keep this discussion technical.
If a CD-ROM does not support ATAPI, you are not able to
- open/close/lock the door.
- Rip Audio from it.
People who really cannot affort to buy a new drive will still be able
to see the drive as read-only HD.
In addition, if the drive would support DAE via some non-standard interface
nobody would be happy with it. The DAE quality would be lousy and none of
the programs that is still supported would be able to use the drive
decently for DAE.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>Thunder wrote:
>I got a drive at home which is from 1996 and doesn't do ATAPI. It's some
>Mitsumi indestructible, but I can't tell you details since I won't be at
>home within the next few weeks.
What can you do with the drive - besides using it as CD-ROM?
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
On Sun, 2002-07-14 at 14:17, Joerg Schilling wrote:
> Did you ever looks at the ATAPI specs?
>
> ATAPI _is_ SCSI over IDE with a few "bugs"/deviations:
In other words as he said ATAPI is not SCSI
At 14:24 14/07/02, Joerg Schilling wrote:
>[...] In addition, if the drive would support DAE via some non-standard
>interface [...]. The DAE quality would be lousy [...].
This is a very presumptuous statement! You cannot assume that an
alternative interface would be of lousy quality. Maybe it is a million
times better? The current one (at least as some drives implement it on the
drive side) can be of very, very poor quality indeed.
Note: I am not saying that there is an alternative interface or anything
like that... just that your statement is fallacious.
Best regards,
Anton
--
"I've not lost my mind. It's backed up on tape somewhere." - Unknown
--
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS Maintainer / IRC: #ntfs on irc.openprojects.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
>From: Andre Hedrick <[email protected]>
First: Top posting is something that disqualifies you. Please don't
top post otherwise people will belive that you are just a troll.
>Don't take this personally but you are just a top level application.
>The only reason you have such blinders on is because you always had a
>bottom layer transport working around and doing its best to prevent
>errors form showing up. You are wrong and because you are not down on the
>bus at the physical layer you do see the mess.
I don't take this personal, but the main problem seems to be that most
of the people in this discussion have far less kernel experience than I have.
I did actively write code from many different places in different OS
implementaions. Ans more important, I did read a lot of other people's code.
If Linux people have kernel experience at all, they usually only know
a specific part of one single OS: Linux.
Writing good kernel code is more! It educates a lot to read read read
a lot in other OS sources to understand other ideas and to become
able to judge about the quality os a specific implementation.
You seem to have no experience than IDE in Linux. It would help if you
first look at other implementations in order to become able to judge yourself.
>Now your silly PCATA stupid ass Tailgate Bridge that you are boasting
>about does some of the worst things anyone could ever imagine.
???? Looks stupid (like dou did not get the message).
>Oh and the bad idea you call is to permit dynamic subdriver shifting.
>Now that it may never be completed you have the advantage to call it a bad
>idea. I suspect you are an ASPI lover and soone SPI will die.
Please read what I wrote and don't guess what I might have written.
>Next your statements about name a drive is a straw man.
>More basic proof you do not look down at the hardware.
???? Looks completely unrelated to the thread and to my mail.
>BurnProof is a result lame devices which improperly hold off the bus
>because release the BUSY Bit while still performing transfers.
>The very fact that a huge pile of devices went into the market place
>based on SFF-8020 rev 2.5 total roasts your strawman, please try again.
Again: this is completely unrelated to the problem. Why do you introduce
it?
>So instead of whining about what is there and not from your location in
>end user land, try and offer something useful like a preferred API to
>allow clean packet-driver interfaces. Doing that little would allow the
>transport layer people and the transistion-api folks to user land to
>greatly increase compatablity. Then you would not need 5 interfaces for
>Linux.
>Have a good day.
I am not whining, but you answer with unrelated stuff. Why? Are you missing
experience and arguments?
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>Eric Anderson wrote:
>Cool. I've not read over anything newer then cdrtools-1.10.
So you are living in the past...
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
Hi,
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> This is my idea too: CD-ROM drives should be accessed via ATAPI or
> handled as ATA disk.
Handling them as an ATA disk will eat you up at some extent - whilst you
can't change an ATA disk on runtime (well, not easily, at least), you must
be able to change the CD in your non-ATAPI CD-ROM. If we handle it as an
ATA disk, we'll lead into trouble. Non-ATAPI CD-ROMs aren't ATA hard
disks.
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
>From [email protected] Fri Jul 12 22:18:47 2002
>As much as I hate IDE, IDE isn't going away. All my systems use SCSI
>so on machines that have CD/ROMS, I use your libraries and your tools.
>Maybe somebody should make CD/ROM code that directly talks to IDE via
>/dev/hdwhatever, instead of expecting you to modify your code that
>has worked so well for so long.
This would be a really bad idea.
Such a change would force me to add a 6th (and unneeded) new interface.
Why? What problem would be solved if you did introduce such an interface?
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
Hi,
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> > Please stop calling ATAPI as SCSI over IDE, it is not. It is Packet
> > Interface over ATA (IDE). Some ATAPI/SCSI devices are functionally
> > equivalent because they support the same command set (i.e. MMC).
>
> ATAPI _is_ SCSI over IDE with a few "bugs"/deviations:
>
> [Blah...]
What you describe is still not SCSI. It's the Packet Interface, and it's
not run over IDE, but over ATA. You need to know the difference between
hardware style and command layers, providing that you're really a person
who has ever written a SCSI driver.
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
>From [email protected] Fri Jul 12 22:22:45 2002
>On Fri, 2002-07-12 at 21:08, Joerg Schilling wrote:
>> There are enough other OS that use a common ATAPI/SCSI driver concept and
>> do not have the problems you vagely name but never really describe.
>There are lots that fudge around and pretend scsi is the block layer
>when it is not. That sort of misses the point and slows down high end
>raid cards.
It seems that you miss to understand the needed underlying driver structures.
SCSI is not a block layer, it is a generic transport.
>> I believe it's OK to drop support for 10 year old hardware in case this
>> is no important hardware that _really_ needs continued support (like
>> e.g. 9 track trapes).
>We support and people continue to use large amounts of ten year old
>hardware. Thats why children at quite a few schools have access to
>computing through things like LTSP. There are people actively
>maintaining much of this stuff too.
I never agued to completely drop support for them. You cannot do decent
DAE with pre-1998 CD-ROM drives, so why try to support DAE for old drives?
Just use them as read-only DA drives.
>Some things such as ide tapes which have needs rather different to scsi
>tape are still being made and released in newer and larger forms.
If you have a IDE tape that acts as tape, it most likely uses SCSI
tape commands. If it acts as a big floppy, it is not relevent in this
discussion.
>> ATA devices that are neither hard disks, nor do support ATAPI decently
>> are Y 1992 crap - so unless you like to have continued support for your
>> provate museum, what is the reason that you like to prevent a change
>> to a more usable driver interface?
>You still don't seem to understand the difference between a driver
>interface and a hardware layer.
Well, from previous discussions with you, you did proove that this is exactly
what _you_ don't understand.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
Hi,
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> If a CD-ROM does not support ATAPI, you are not able to
>
> - open/close/lock the door.
This can be supported, though. Most of the non-ATAPI CD-ROM devices
support this behavior, only do they use different commands.
> In addition, if the drive would support DAE via some non-standard
> interface nobody would be happy with it. The DAE quality would be lousy
> and none of the programs that is still supported would be able to use
> the drive decently for DAE.
Not exactly. My old MITSUMI indestructible has digital audio quality above
many of the existing ATAPI devices. Old != Bad.
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
On Sun, 2002-07-14 at 15:07, Joerg Schilling wrote:
> >From [email protected] Fri Jul 12 22:22:45 2002
> >There are lots that fudge around and pretend scsi is the block layer
> >when it is not. That sort of misses the point and slows down high end
> >raid cards.
>
> It seems that you miss to understand the needed underlying driver structures.
> SCSI is not a block layer, it is a generic transport.
It is not generic - its handling of sophisticated I/O stuff is non
existant. SCSI gave rise to a convenient command set for low end devices
thats since been applied (with endless problems due to its use) to
things like fibrechannel.
Of course if you'd actually bothered to read the code (as I told you to
go do a while back) you might understand the 2.5 direction with the
block I/O layers. Using scsi command sets as a driver abstraction is a
nonsense, its incomplete, inefficient and too full of messy rules that
its not reasonable to inflict on hardware that doesn't care (eg recovery
from tagged command sequences on an error from the drive). 2.5 has a
much much saner abstraction thank you.
Alan
>From [email protected] Sat Jul 13 07:40:59 2002
>> If you force cdrecord to rely on CD-ROM only interfaces, you make Linux
>> unusable in general. Do you really like to create an unusable Linux just
>> to avoid creating a usable generic SCSI transport interface?
>Lets step back a moment here. The cdrecord package is not
>responsible for making "Linux usable in general". It is
>responsible for writing data to CD-ROMs. It is _not_ responsible
>for driving scanners, hard drives, or enforcing policy on the
>Linux kernel.
It looks like you miss important issues.
- More and more people like to use CD writers in their PC.
- There are no new "SCSI" (which rather means SCSI with
1984 transport layer) drives on the market. However,
once you have a devent SCSI transport abstraction layer
as I have in libscg, there is no difference in the high
level code.
- While it has been quite simple to add a SCSI CD writer
to a PC and it is still simple on platforms that treat
ATAPI ad SCSI over IDE, it is a big problem for novices
to make a ATAPI CD writer work on Linux.
- The people who have these sort of problems are those people
who are new to Linux and who believe that Linux is unusable
after they get those problems.
>If you would throw away crdrecord's desire to do its own private
>SCSI bus scanning, and throw away your attachment to addressing
>devices only by host, channel, id, and lun a number of things
It looks that you miss to understand what cdrecord does!
Cdrecord in special and libscg in general definitely does not
scan the bus. This is done by the kernel.
Cdrecord only tries to find all devices that already have been
found by the kernel.
>happen. For starters, Linux devices don't have to be forced to
>all be sitting on the SCSI bus. You could use standard Linux
>device names (i.e. /dev/hdc or /dev/scd0). And you could still
>send all the SCSI/ATAPI packet commands you want to the device
>that was selected using the CDROM_SEND_PACKET ioctl.
For a starter, it is easier to understand the SCSI concept of
addressing than to understand the Linux concept. In addition,
the SCSI addressing concept can be used on different platforms
in a unique way. This helps people (and GUI writers) to use
cdrecord on more than Linux only.
>Ever look at the CDROM_SEND_PACKET ioctl?
I did, but you obviously did not :-(
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From [email protected] Sat Jul 13 07:49:29 2002
>On Fri Jul 12, 2002 at 10:17:21PM +0100, Alan Cox wrote:
>> CD burning is a side issue to stability and reliability.
>>
>> In terms of supporting old hardware most of that is irrelevant to cd
>> recording anyway, so why do you care ? What you actually need is a
>> generic interface for cd packet sending.
>A generic interface for cd packet sending? Sounds useful. So
>useful someone thought of it years ago, and called it the
>CDROM_SEND_PACKET ioctl. Its been in the kernel since Aug 1999.
>What'll those crazy Linux CD-ROM people think of next?
We don't need just another unrelated interface but a generic
transort. CDROM_SEND_PACKET is not a generic interface, it is limited
to ATAPI CD-ROM's.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From [email protected] Sun Jul 14 15:35:09 2002
>On Sun, 2002-07-14 at 14:17, Joerg Schilling wrote:
>> Did you ever looks at the ATAPI specs?
>>
>> ATAPI _is_ SCSI over IDE with a few "bugs"/deviations:
>In other words as he said ATAPI is not SCSI
It would really help not to turn a technically based discussion
into personal bashing. I am still waiting to see any tecnical based
argument from you.
It seems that you are not nterested in a better technical based solution
but in preventing other people's ideas from being used.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
Hi,
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> What can you do with the drive - besides using it as CD-ROM?
Lots of stuff. It supports all the commands you mentioned, plus it is one
of the best readers I ever had. Put a milk glass CD in it, it will read
you the contents. Put any audio CD into it, it will read off at the finest
quality ever. It's bad that I can't always take it with me.
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
>From [email protected] Sun Jul 14 15:38:33 2002
>At 14:24 14/07/02, Joerg Schilling wrote:
>>[...] In addition, if the drive would support DAE via some non-standard
>>interface [...]. The DAE quality would be lousy [...].
>This is a very presumptuous statement! You cannot assume that an
>alternative interface would be of lousy quality. Maybe it is a million
>times better? The current one (at least as some drives implement it on the
>drive side) can be of very, very poor quality indeed.
Name a single drive that is DAE capable, does not support ATAPI and
doesn't do DAE in lousy quality.
>Note: I am not saying that there is an alternative interface or anything
>like that... just that your statement is fallacious.
Your statements just proove that you didn't try to get the background information
that is needed to find a useful and seminal future development.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
Hi,
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> I am not whining, but you answer with unrelated stuff. Why? Are you missing
> experience and arguments?
Andre is LAD storage consultant for a reason. He lead Linux ATA device
support to success for a long time, and he did it again recently. If he
mixed up the threads, then it's not a matter of experience on ATA.
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
On Sun, 14 Jul 2002, Joerg Schilling wrote:
>...
> If a CD-ROM does not support ATAPI, you are not able to
>
> - open/close/lock the door.
>...
Look at drivers/cdrom/mcdx.c, a driver for proprietary (the device is
connected via an ISA card to the computer) single and double speed Mitsumi
CD-ROM drives. This driver supports to open the door although the drive
definitely doesn't support ATAPI...
> J?rg
cu
Adrian
--
You only think this is a free country. Like the US the UK spends a lot of
time explaining its a free country because its a police state.
Alan Cox
> - The people who have these sort of problems are those people
> who are new to Linux and who believe that Linux is unusable
> after they get those problems.
Hmm, what a terrific example of the finest reasoning!
Joe Sixpack can't get his drive to record CDRs. -> Joe Sixpack becomes
annoyed and concludes the system is crap, which effectively makes the
OS unusable for everybody.
You have more of that?
Seriously, though, I suggest you go read the code finally and stop dissing
people on lkml.
And fix your mailer when you're at it.
T.
Hi,
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> Name a single drive that is DAE capable, does not support ATAPI and
> doesn't do DAE in lousy quality.
I know lots, but I can't tell you names. If I could tell you the name of
any of them I'd certainly have to take care of my sanity.
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
At 15:28 14/07/02, Joerg Schilling wrote:
> >From [email protected] Sun Jul 14 15:38:33 2002
> >At 14:24 14/07/02, Joerg Schilling wrote:
> >>[...] In addition, if the drive would support DAE via some non-standard
> >>interface [...]. The DAE quality would be lousy [...].
>
> >This is a very presumptuous statement! You cannot assume that an
> >alternative interface would be of lousy quality. Maybe it is a million
> >times better? The current one (at least as some drives implement it on the
> >drive side) can be of very, very poor quality indeed.
>
>Name a single drive that is DAE capable, does not support ATAPI and
>doesn't do DAE in lousy quality.
I don't need to. As I said in the following paragraph, I never claimed such
a thing existed. I am only complaining about your generalisations which to
me imply that an alternative better interface cannot and could not exist.
Which is a pile of crap. Maybe tomorrow someone will come up with a much
better interface/drive. That is all I am saying. You cannot see into the
future to be able to make such generalisations...
> >Note: I am not saying that there is an alternative interface or anything
> >like that... just that your statement is fallacious.
>
>Your statements just proove that you didn't try to get the background
>information
>that is needed to find a useful and seminal future development.
WHAT?!? Get off your high horse, will you? I never claimed anything to
prompt this response. I am just telling you off for misleading people with
your statements...
Anton, who is starting to get pissed off with Mr Joerg "I know everything"
Schilling
--
"I've not lost my mind. It's backed up on tape somewhere." - Unknown
--
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS Maintainer / IRC: #ntfs on irc.openprojects.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
Hi,
On Sun, 14 Jul 2002, Anton Altaparmakov wrote:
> Anton, who is starting to get pissed off with Mr Joerg "I know everything"
> Schilling
I am, so I want mr. Joerg "I know everything" Schilling to take the
challenge to provide us with a working patch on how to deal with it. If
he's so much more familiar with the kernel than we are, he should be able
to provide an efficient interface in no time.
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
U?ytkownik Paul Bristow napisa?:
>
>
> Martin Dalecki wrote:
>
>> Workarouns in ide-floppy - ZIP drives and Clock drives.
>>
>> Those are the main "technical issues" which make one hessitate.
>>
>>
> Sorry, late to this thread. Look, ide-floppy has to deal with real
> world devices, which simply don't follow nice, written specifications.
> If we treat these devices as standard ATAPI they simply don't work.
>
> And a bunch of planned features which were waiting for 2.5 but are now
> in limbo until the IDE subsystem is stable enough for me to work on.
> Features include:
>
> Trying again to kill the via chipset/Zip interaction (some people are
> still suffering).
>
> Kevin Flemings media detect work: needs ATA commands because the ATAPI
> command set simply doesn't return the information we need. Then we can
> make ide-floppy drives work *properly* with devfs.
>
> Handling the "special" BIOS settings around LS-120/240/Zip bootable
> drives.
> Making sure that formatting works properly for non-standard format
> capacities (i.e. 1.44MB in LS-120, 32MB in LS-240)
>
> And yes, I have real users asking for these things.
>
> So to me the problem is not to make everything work as SCSI, because
> that simply isn't true for ide-floppy devices. They *nearly* work, so
> you can get kludgy, "good enough for command line gurus" working with
> ide-scsi, but there are some funnies. Does it really make sense to have
> IDE/ATAPI kludges down in the SCSI tree?
>
> I much prefer Linus's suggestion of agreeing on the top level API. I
Just to make things clear. Personally I by no way think
that ide-scsi should remain an SCSI device. My concern is only the
fact that it would be easier in my opinnion to start off with
ide-scsi and make it *independant* from the SCSI code or device handling
by separating commonly used code for MMC_ handling out of SCSI
in a kind of library (that is it) and not a common "layer"
(this could be a second step).
However this can be accomplished the other way around as well.
It will be just considerable more work.
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> >From [email protected] Sat Jul 13 07:40:59 2002
> >happen. For starters, Linux devices don't have to be forced to
> >all be sitting on the SCSI bus. You could use standard Linux
> >device names (i.e. /dev/hdc or /dev/scd0). And you could still
> >send all the SCSI/ATAPI packet commands you want to the device
> >that was selected using the CDROM_SEND_PACKET ioctl.
>
> For a starter, it is easier to understand the SCSI concept of
> addressing than to understand the Linux concept. In addition,
> the SCSI addressing concept can be used on different platforms
The traditional SCSI concept of addressing is not compatible
with SAN style hardware, like iscsi or large fibre channel
setups.
regards,
Rik
--
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/ http://distro.conectiva.com/
>>I will violently oppose anything that implies that the IDE layer uses the
>>SCSI layer normally. No way, Jose. I'm all for scrapping, but the thing
>>that should be scrapped is ide-scsi.
>
> Nobody who has a technical backgroupd and knows what to do wuld ever
> make such a proposal.
>
> Instead, there needs to be one or more SCSI HA driver as part of the
> SCSI stack. This driver also needs to deal with plain ATA in order
> to be able to coordinate access.
>
> J?rg
Such driver would only work with ATAPI devices. Joerg, does not
seem to realize that the vast majority of IDE devices do not
support ATAPI at all. As a rule of thumb, winchesters do not,
CD-ROMs and such do, and tapes do too. To make a pseudo-HA
driver which speaks plain IDE and plugs into SCSI subsistem
would saddle the IDE with SCSI limitations, and add one more
layer for no benefit whatsoever.
-- Pete
>From [email protected] Sun Jul 14 16:18:38 2002
>On Sun, 2002-07-14 at 15:07, Joerg Schilling wrote:
>> >From [email protected] Fri Jul 12 22:22:45 2002
>> >There are lots that fudge around and pretend scsi is the block layer
>> >when it is not. That sort of misses the point and slows down high end
>> >raid cards.
>>
>> It seems that you miss to understand the needed underlying driver structures.
>> SCSI is not a block layer, it is a generic transport.
>It is not generic - its handling of sophisticated I/O stuff is non
The fact that you don't know st does not make this statement true.
>existant. SCSI gave rise to a convenient command set for low end devices
>thats since been applied (with endless problems due to its use) to
>things like fibrechannel.
The endless problems are problems caused by e.g. a bad transport implementation
in Linux. Your "fibrechannel" words leads to another problem in Linux.
usb_storage is a module that seems to suffer from a correct implementation
if concurrent driver tasks. If usb_storage steps over the 'memory stick'
interface on a Sony VAIO, it hangs itself for 10 minutes and is unable
to recognise any other drive during this time period. If you like to use
a USB CD writer on a VAIO, you need to unplug it and let the OS settle
down after the boot until you are able to reconnect the writer. If you like
to use the USB floppy the same problems ossur. This leads to the fact
that you cannot boot from a USB floppy: once the kernel is up, it cannot
mount the root disk because the driver is hung from the memory stick adaptor.
>Of course if you'd actually bothered to read the code (as I told you to
>go do a while back) you might understand the 2.5 direction with the
>block I/O layers. Using scsi command sets as a driver abstraction is a
>nonsense, its incomplete, inefficient and too full of messy rules that
>its not reasonable to inflict on hardware that doesn't care (eg recovery
>from tagged command sequences on an error from the drive). 2.5 has a
>much much saner abstraction thank you.
Well, I _did_ read the code a while ago and I did even write a patch for
the sg.c driver that helped to fix a lot of problems. But instead of using
this driver in the official kernel, _you_ preferred a sg.c hack made by a
OS novice that mainly did address some DMA specifics that should not occur
at all in sg.c in a cleanly layered kernel.
I had a concept on how to go to a more usable interface in the future.
Do you really believe that I will ever start again to put effort in a
Linux kernel module unless you did previously proove that you are willing
and able to run a tecnically based discusion?
Look at this discussion: you sit on a high horse and behave as if you had
serious kernel experiences and do not even need to react on my statements
in a senseful way. From looking at your statements it rather looks as if
you are missing the needed experience. You do not really believe that this
is the right way to treat someone with 20 year of kernel experience
on different places of the kernel and on different OS implementations, do you?
If you don't like to look like a 'I know everything but I don't have to proove'
troll, it would help a lot to have a serious discussion where you would start
to use arguments instead of just telling people no more than 'I know better'.
Give yourself a hitch and admit that you are the person who is responsible
that I believe that it is useless to try to help in Linux kernel development.
If this discussion stays as it currently looks like, it does not make sense
for me to try to find a better solution. Let me call it this way: this thread
was just another proof that it is not possible to have a technical based
solution with the Linux folks. It seems t be just a waste of time :-(
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
Folks,
These lists are run in "open for all posters, all topics ok" manner,
mainly because doing closed lists is way larger pain in the rear than
occasional through-leaked spam, or occasional bickering, but ...
.. when most of the list traffic is flame war, I am sorely tempted to
stomp in with some TABOO-filters..
/Matti Aarnio -- co-postmaster of vger.kernel.org
On Sun, Jul 14, 2002 at 08:52:28AM -0600, Thunder from the hill wrote:
> Date: Sun, 14 Jul 2002 08:52:28 -0600 (MDT)
> From: Thunder from the hill <[email protected]>
> To: Anton Altaparmakov <[email protected]>
> cc: Joerg Schilling <[email protected]>,
> <[email protected]>
> Subject: Re: IDE/ATAPI in 2.5
>
> Hi,
>
> On Sun, 14 Jul 2002, Anton Altaparmakov wrote:
> > Anton, who is starting to get pissed off with Mr Joerg "I know everything"
> > Schilling
>
> I am, so I want mr. Joerg "I know everything" Schilling to take the
> challenge to provide us with a working patch on how to deal with it. If
> he's so much more familiar with the kernel than we are, he should be able
> to provide an efficient interface in no time.
>
> Regards,
> Thunder
> --
> (Use http://www.ebb.org/ungeek if you can't decode)
> ------BEGIN GEEK CODE BLOCK------
> Version: 3.12
> GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
> N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
> e++++ h* r--- y-
> ------END GEEK CODE BLOCK------
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>From: Anton Altaparmakov <[email protected]>
>>Name a single drive that is DAE capable, does not support ATAPI and
>>doesn't do DAE in lousy quality.
>I don't need to. As I said in the following paragraph, I never claimed such
>a thing existed. I am only complaining about your generalisations which to
>me imply that an alternative better interface cannot and could not exist.
>Which is a pile of crap. Maybe tomorrow someone will come up with a much
>better interface/drive. That is all I am saying. You cannot see into the
>future to be able to make such generalisations...
Why should there be such a device when there is a standard that manufacturers
implement?
>> >Note: I am not saying that there is an alternative interface or anything
>> >like that... just that your statement is fallacious.
>>
>>Your statements just proove that you didn't try to get the background
>>information
>>that is needed to find a useful and seminal future development.
>WHAT?!? Get off your high horse, will you? I never claimed anything to
>prompt this response. I am just telling you off for misleading people with
>your statements...
>Anton, who is starting to get pissed off with Mr Joerg "I know everything"
>Schilling
Well, I get pissed of the fact that it seems to be impossible to have a
technical based discussion in the Linux kernel environment.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
On Sat, 13 Jul 2002, Adam J. Richter wrote:
> On Sat, 13 Jul 2002, Bartlomiej Zolnierkiewicz wrote:
> >On Sat, 13 Jul 2002, Adam J. Richter wrote:
> >> On Sat, 13 Jul 2002, Bartlomiej Zolnierkiewicz wrote:
> >> >Wrong impression. ;)
> >> >Hint: look for STANDARD_ATAPI macro usage.
> >>
> >> It looks like that macro should be renamed to something like
> >> STANDARD_MMC. Everything that that macro controls still appears to
> >> go through ATA Packet Interface encapsulation. Those quirks look like
> >
> >Please verify against sff8020.
>
> I don't know what you mean by this. It's not a question of
> whether the behavior being accomodated is conformant or nonconformant
> to a standard. The question is whether the accomodations controlled
> by the "STANDARD_ATAPI" macro can easily be implemented in sr_mod.
> Since the accomodations are translating a couple of numbers that
> are repeseneted as binary coded decimal instead of integers (0-255)
> on some drives, and sending a slightly different SCSI command to
> change discs on a Sanyo three CD changer, it seems that they can easily
> be implemented in sr_mod.
I agree with this, I was rather thinking of checking whole ide-cd
behaviour against spec to see if there are any more serious problems
(and I've heard about some, raising DRQ too soon or sth like that (?)).
> >> they would likely be duplicated in a SCSI version of the same drives
> >> anyhow. It should be easy to have sr_mod accomodate those drives.
> >
> >I can't find them, there are some in sr_vendor.c but they are diffirent
>
> From you email address, I would guess that English is not your
> first language. While you do write English very well, I think you made
Very funny... ;-)
btw. reading is really easy, writing is a bit harder, read below.
> a mistake in understanding what I siad. "It should be easy to
> have sr_mod accomodate those drives" means that it would be easy for
I made a mistake of not explicitly writing what I think.
> someone to write code in the near future to do this (that is, to change
> sr_mod.c). It does not mean that it has already been done.
I understood this w/o problems.
But do you understand that in might be not possible to move ide-cd
completly to sr?
--
Bartlomiej
> Adam J. Richter __ ______________ 575 Oroville Road
> [email protected] \ / Milpitas, California 95035
> +1 408 309-6081 | g g d r a s i l United States of America
> "Free Software For The Rest Of Us."
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> Well, I get pissed of the fact that it seems to be impossible to have a
> technical based discussion in the Linux kernel environment.
Then please, show us your technical arguments on why the SCSI
layer is enough for every CD writing hardware out there.
Now compare them with the results from the NAS and SCSI talks
and BoFs at the kernel summit and OLS, where everybody agreed
that the current SCSI addressing and discovery schemes just
don't cut it on things like iscsi and other network storage
solutions.
It's not just about the fact that the controller/bus/unit/lun
addressing doesn't deal well with network attached storage and
multipath, it's also about things like the impossability of
device discovery on a bus with 2^32 possible device addresses.
This, in turn, makes the current sd[a-z] and sg[a-h] more than
a little inadequate. Furthermore, you suddenly require the
ability to tell the kernel to talk to devices the kernel doesn't
yet know about (because it can't scan 2^32 device addresses at
boot time).
regards,
Rik
--
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/ http://distro.conectiva.com/
On Sun, 2002-07-14 at 16:12, Joerg Schilling wrote:
> >From [email protected] Sun Jul 14 16:18:38 2002
> >> It seems that you miss to understand the needed underlying driver structures.
> >> SCSI is not a block layer, it is a generic transport.
>
> >It is not generic - its handling of sophisticated I/O stuff is non
>
> The fact that you don't know st does not make this statement true.
Try doing the following in SCSI
- Device managed storage layout "Allocate x blocks close to handle y
and give me a new handle"
- Per I/O request readahead hints (please read on xyzK , please dont
readahead)
- Per I/O request writeback hints (write back cache is ok, write back
cache is ok only if NV, don't bother caching, streaming I/O
hint)
Nor are all the FC problems caused by bad transport implementations in
Linux. Physical layer incompatibilities occur everywhere - but aren't
the one I'm talking about. Of more concern are the inability to manage
multiple levels of caches properly.
I have controllers which can do most of the above. I don't want to talk
scsi to them and spend all my cpu time faking, decoding and recoding
command blocks, spoofing error handling and the like.
Its simply inappropriate to consider the SCSI command set as a high
level interface for block and related I/O.
As to your comments on sg. Everyone except you happened to think that
Doug Gilberts very nice sg changes were the right path. I still think it
was the right decision.
> If this discussion stays as it currently looks like, it does not make
> sense for me to try to find a better solution. Let me call it this
> way: this thread was just another proof that it is not possible to
> have a technical based solution with the Linux folks. It seems t be >
> just a waste of time :-(
The Linux development process aggressively filters bad ideas.
Alan
On Sun Jul 14, 2002 at 04:20:55PM +0200, Joerg Schilling wrote:
>
> >From [email protected] Sat Jul 13 07:49:29 2002
>
> >On Fri Jul 12, 2002 at 10:17:21PM +0100, Alan Cox wrote:
> >> CD burning is a side issue to stability and reliability.
> >>
> >> In terms of supporting old hardware most of that is irrelevant to cd
> >> recording anyway, so why do you care ? What you actually need is a
> >> generic interface for cd packet sending.
>
> >A generic interface for cd packet sending? Sounds useful. So
> >useful someone thought of it years ago, and called it the
> >CDROM_SEND_PACKET ioctl. Its been in the kernel since Aug 1999.
> >What'll those crazy Linux CD-ROM people think of next?
>
> We don't need just another unrelated interface but a generic
> transort. CDROM_SEND_PACKET is not a generic interface, it is limited
> to ATAPI CD-ROM's.
Wrong. It is a _generic CD-ROM packet interface. Thanks for not even
spending the two seconds it would take reading the kernel source code
to discover this.
$ grep -l CDC_GENERIC_PACKET drivers/scsi/sr.c drivers/ide/ide-cd.c
drivers/scsi/sr.c
drivers/ide/ide-cd.c
-Erik
--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--
On Sun Jul 14, 2002 at 02:32:15PM +0200, Joerg Schilling wrote:
> >Eric Anderson wrote:
>
> >cdrecord should use the CDROM_SEND_PACKET ioctl, then it would
> >work regardless,
>
> This only prooves that you are uninformed :-(
No. This only proves _you_ have not tried it. I've used the
CDROM_SEND_PACKET ioctl on both SCSI and ATAPI cdrom devices.
What do you need to do in cdrecord that cannot be done with it?
-Erik
--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--
On Sun, Jul 14, 2002 at 04:35:02PM +0200, Adrian Bunk wrote:
> On Sun, 14 Jul 2002, Joerg Schilling wrote:
> >...
> > If a CD-ROM does not support ATAPI, you are not able to
> >
> > - open/close/lock the door.
>
> Look at drivers/cdrom/mcdx.c, a driver for proprietary (the device is
> connected via an ISA card to the computer) single and double speed Mitsumi
> CD-ROM drives. This driver supports to open the door although the drive
> definitely doesn't support ATAPI...
...nor is it an IDE(ATA) device which this discussion is about.
BTW, I consider this discussion ridiculous and I wonder whether I?m the only
one:
J?rg, HPA and some others want to stick to the current SCSI high-level
drivers and offer more flexible transport (and to a certain extent
command) layer below. Other people are scared of some of the uglinesses of
today's SCSI code and want to offer general high-level driver (that they do
not call SCSI) with some flexible command and tranport layer below.
It's more a question of how you name it then of technical details.
And maybe in how far you allow current software (which is tailored well to
deal with our current SCSI code) to work with the future solution.
A good portion of the current SCSI midlayer code would be eliminated or
changed anyway, no matter how you call it
Given the love of Linus for the SCSI code, I would suggest not to name it
SCSI any more, but rather generic block code. I also matches reality
somewhat better.
But what we need to do then is to offer a generic interface (similar to sg)
to send down commands to the devices. Otherwise, we'd break scanners,
CD-Writers, ... which currently rely on this interface. And as we don't want
to have kernel drivers for all those, there's no alternative.
(And it should be sg like, otherwise we put a lot of load on application
developers such as J?rg, which I would call bad policy.)
Device management software will certainly also want to speak to the devices
via some exposed low-level interface, BTW. Maybe we can expose enough
information via driverfs to solve the naming stuff reasonably, i.e. allow
naming by topology, device and content, but still some things will and
should not be done in kernel space.
sg does most of this for 'real' and 'fake' (ATAPI,usb-storage,sbp2) nowadays
pretty well and we should not just scratch that because another idea looks
very sexy.
Groetjes,
--
Kurt Garloff <[email protected]> Eindhoven, NL
GPG key: See mail header, key servers Linux kernel development
SuSE Linux AG, Nuernberg, DE SCSI, Security
Hi,
> My concern is only the fact that it would be easier in my opinnion to
> start off with ide-scsi and make it *independant* from the SCSI code or
> device handling by separating commonly used code for MMC_ handling out
> of SCSI in a kind of library (that is it) and not a common "layer" (this
> could be a second step).
What about some kind of
retval_t mmc_to_hostcmd(host, cmd, ...);
response_t response_to_mmc(host, ...);
?
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
>From [email protected] Sun Jul 14 17:05:31 2002
>>>I will violently oppose anything that implies that the IDE layer uses the
>>>SCSI layer normally. No way, Jose. I'm all for scrapping, but the thing
>>>that should be scrapped is ide-scsi.
>>
>> Nobody who has a technical backgroupd and knows what to do wuld ever
>> make such a proposal.
>>
>> Instead, there needs to be one or more SCSI HA driver as part of the
>> SCSI stack. This driver also needs to deal with plain ATA in order
>> to be able to coordinate access.
>>
>> J?rg
>Such driver would only work with ATAPI devices. Joerg, does not
>seem to realize that the vast majority of IDE devices do not
>support ATAPI at all. As a rule of thumb, winchesters do not,
>CD-ROMs and such do, and tapes do too. To make a pseudo-HA
>driver which speaks plain IDE and plugs into SCSI subsistem
>would saddle the IDE with SCSI limitations, and add one more
>layer for no benefit whatsoever.
You did not seem to read my previous postings. It is simple to have
a second upper level interface for the IDE HBS driver that connects to
e.g. the ATA HD driver interface. If you like to have 100% backwards
compatibility you could even put the current non-standard part from
ide-cd on top of this 2nd type of upper level code. Make sure, that
the ATAPI/SCSI part is probed first, to allow to connect the SCSI
drivers first and lock the ATA type interface against being used by
drivers like ide-cd in case that a SCSI type of high level driver did
feel responsible to deal with a specific drive.
Of curse this will only work if there is some kind of accepted
development leadership that makes a concept, works on data structures and
finally instructs the maintainers of old code to find a way to convert
their current driver to the new model. This new model usually would
cause parts of the code that exists more than once to be removed from
all the old drivers that have been considered to represent 'the wrong way'
in the tree of the evolution of the kernel.
Stop looking at your own belly and have a look at what other OS do.
It seems to be one of the main problems in the Linux development.
People usually only look at Linux and inside Linux, they only look at
their part of the kernel. Doing it this way, they define their own
interfaces and don't look enough to the left and to the right.
To make it not look as if I am only bashing current IDE/SCSI code, let
me give an example from the filesystem part.
Have a look at the file
ftp://ftp.fokus.gmd.de/pub/unix/star/testscript/rock.tar.bz2
It is a tar archive that contains 207,440 empty files and one directory
using the names of the files in the 'rock' subdirectory from the
freedb project. This is a snapshot taken on May 30th 2002.
If you extract this archive on a machine running Solaris or FreeBSD,
it takes less than 7 minutes.
A pentium 800 running Solaris 9 results in:
# star -xp -time < rock.tar.bz2
star: WARNING: Archive is bzip2 compressed, trying to use the -bz option.
star: 10372 blocks + 1536 bytes (total of 106210816 bytes = 103721.50k).
star: Total time 405.988sec (255 kBytes/sec)
6:46.051r 12.920u 63.420s 18% 0M 0+0k 0st 0+0io 0pf+0w
You see during the 6:43, the machine is 82% idle.
A Pentium 1200 running Linux-2.5.25 (ext3) results in:
# star -xp -time < rock.tar.bz2
star: WARNING: Archive is bzip2 compressed, trying to use the -bz option.
star: 10372 blocks + 1536 bytes (total of 106210816 bytes = 103721.50k).
star: Total time 3190.483sec (32 kBytes/sec)
53:10.490r 12.299u 2970.099s 93% 0M 0+0k 0st 0+0io 4411pf+0w
You see, during the 53:20, the machine is only 7% idle!
It wasted 2900 seconds of CPU time on Linux. Let me guess: this was done
inside the function strcmp().
There are ~ 5 different filesystems on Linux, but none if the projects seem
to care about the code outside the FS low level code. I suspect, that
this is not any better if you use reiserfs.
Solaris and FreeBSD put all the effort into one filesystem trying to make
it as good as possible. In Linux, it seems that nobody prooved the overall
concept of the kernel.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From: Rik van Riel <[email protected]>
>> Well, I get pissed of the fact that it seems to be impossible to have a
>> technical based discussion in the Linux kernel environment.
>Then please, show us your technical arguments on why the SCSI
>layer is enough for every CD writing hardware out there.
Simple: there is not a single CD writer out that uses something other
than SCSI commands to write media or do DAE.
>Now compare them with the results from the NAS and SCSI talks
>and BoFs at the kernel summit and OLS, where everybody agreed
>that the current SCSI addressing and discovery schemes just
>don't cut it on things like iscsi and other network storage
>solutions.
I defined RSCSI before iscsi came out. I did not yet look at ISCSI.
There sould be just an additional IP address in the iscsi addressing
model.
>It's not just about the fact that the controller/bus/unit/lun
>addressing doesn't deal well with network attached storage and
>multipath, it's also about things like the impossability of
>device discovery on a bus with 2^32 possible device addresses.
You don't need as you might net be allowed to access many of them.
Just have a look at my RSCSI protocol. It just puts "user@host:"
before the old SCSI address.
>This, in turn, makes the current sd[a-z] and sg[a-h] more than
>a little inadequate. Furthermore, you suddenly require the
>ability to tell the kernel to talk to devices the kernel doesn't
>yet know about (because it can't scan 2^32 device addresses at
>boot time).
You are right, but this is what programs like e.g. cdrtools which use
libscg already do for two years.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
> A Pentium 1200 running Linux-2.5.25 (ext3) results in:
>
> # star -xp -time < rock.tar.bz2
> star: WARNING: Archive is bzip2 compressed, trying to use the -bz option.
> star: 10372 blocks + 1536 bytes (total of 106210816 bytes = 103721.50k).
> star: Total time 3190.483sec (32 kBytes/sec)
> 53:10.490r 12.299u 2970.099s 93% 0M 0+0k 0st 0+0io 4411pf+0w
>
> You see, during the 53:20, the machine is only 7% idle!
A Pentium 1200, eh?
More like Pentium 120 or star just doesn't cut it.
--
Athlon 1GHz:
kala@nibbler:/tmp/1$ time tar xjf rock.tar.bz2
real 3m19.703s
user 0m9.870s
sys 0m24.840s
According to top, the system was ~90% idle during the extraction.
Linux 2.4.19-rc1-ac3, reiserfs 3.6.
T.
PS. Solaris is over 60% slower than Linux 2.2/2.4 in common fs
operations on my SMP SPARCstation 10.
On Sun, Jul 14, 2002 at 08:11:33PM +0200, Joerg Schilling wrote:
> You did not seem to read my previous postings. It is simple to have
J?rg,
could you please stop starting all your mails like this one and
insulting every people who don't obviously understand what you
nearly didn't explain in 30 mails ? You're whinning about people
who don't take you seriously, but you don't try to be constructive.
Don't forget that this is a mailing list and that there are lots
of people who take the threads long after they have begun, and
please forgive them for not having read your 5000 previous lines.
Since you claim to have very clear ideas about what you'd like to
see in the kernel, please post a clear and complete proposal here
(with another subject, BTW since most people won't read it else).
This way, you'll be able to argue on technical points with some
competent and clever people, but you won't if you continue to
measure your personal experience to that of others. You say you
don't want to talk to them because, unlike you, they don't know
what a kernel is, but this is non sense. They may have a different
conception than you, that's all.
>From your claimed experience, you seem to be 45, but from your
attitude, 8. I think reality is between both, but I hope the
former.
Thanks for your understanding,
Willy
> > A Pentium 1200 running Linux-2.5.25 (ext3) results in:
> >
> > # star -xp -time < rock.tar.bz2
> > star: WARNING: Archive is bzip2 compressed, trying to use the -bz option.
> > star: 10372 blocks + 1536 bytes (total of 106210816 bytes = 103721.50k).
> > star: Total time 3190.483sec (32 kBytes/sec)
> > 53:10.490r 12.299u 2970.099s 93% 0M 0+0k 0st 0+0io 4411pf+0w
> >
> > You see, during the 53:20, the machine is only 7% idle!
>
> A Pentium 1200, eh?
> More like Pentium 120 or star just doesn't cut it.
Now I'm actually pretty sure you meant 386DX/33!
I don't know whom you're trying to fool, but even my P2/233
can get the work done in under 5 minutes:
kala@hubert:/tmp$ time tar xjf rock.tar.bz2
real 4m50.598s
user 0m36.700s
sys 1m51.860s
Linux 2.4.19-pre10-ac2, reiserfs 3.6.
T.
>From [email protected] Sun Jul 14 17:35:44 2002
>Try doing the following in SCSI
>- Device managed storage layout "Allocate x blocks close to handle y
>and give me a new handle"
You don't want do do this in SCSI because it is a task of a layer above SCSI.
>- Per I/O request readahead hints (please read on xyzK , please dont
>readahead)
Again: this is a task of a layer above SCSI.
In some cases, where you might refer to medium access level read ahead,
this is done by implementing tagged SCSI command queueing.
>- Per I/O request writeback hints (write back cache is ok, write back
>cache is ok only if NV, don't bother caching, streaming I/O
> hint)
Again: this is a task of a layer above SCSI. See Solaris and FreeBSD as
examples.
It would help if you first do some homework and read some decent kernel
sources to understand how a kernel needs to be layered to implement
e.g. Storage/FS/kernel/user interface layering.
Then use e.g. 'iostat' to see how overlapping disk I/O is done.
>I have controllers which can do most of the above. I don't want to talk
>scsi to them and spend all my cpu time faking, decoding and recoding
>command blocks, spoofing error handling and the like.
>Its simply inappropriate to consider the SCSI command set as a high
>level interface for block and related I/O.
It looks like you never did metering tests to see what amount of time
is needed to handle the SCSI protocol.
I did many tests when implementing RSCSI. Even when you include TCP/IP
times, the overhead is <= 100 ?s per SCSI command.
>As to your comments on sg. Everyone except you happened to think that
>Doug Gilberts very nice sg changes were the right path. I still think it
>was the right decision.
Not knowing what is bad may make people believe that something is good.
>> If this discussion stays as it currently looks like, it does not make
>> sense for me to try to find a better solution. Let me call it this
>> way: this thread was just another proof that it is not possible to
>> have a technical based solution with the Linux folks. It seems t be >
>> just a waste of time :-(
>The Linux development process aggressively filters bad ideas.
It definitely did not help 4 years ago, when the sg problem did came up.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
commence Tomas Szepe quotation:
> > > A Pentium 1200 running Linux-2.5.25 (ext3) results in:
> > >
> > > # star -xp -time < rock.tar.bz2
> > > star: WARNING: Archive is bzip2 compressed, trying to use the -bz option.
> > > star: 10372 blocks + 1536 bytes (total of 106210816 bytes = 103721.50k).
> > > star: Total time 3190.483sec (32 kBytes/sec)
> > > 53:10.490r 12.299u 2970.099s 93% 0M 0+0k 0st 0+0io 4411pf+0w
> > >
> > > You see, during the 53:20, the machine is only 7% idle!
> >
> > A Pentium 1200, eh?
> > More like Pentium 120 or star just doesn't cut it.
>
> Now I'm actually pretty sure you meant 386DX/33!
>
> I don't know whom you're trying to fool, but even my P2/233
> can get the work done in under 5 minutes:
>
> kala@hubert:/tmp$ time tar xjf rock.tar.bz2
>
> real 4m50.598s
> user 0m36.700s
> sys 1m51.860s
>
> Linux 2.4.19-pre10-ac2, reiserfs 3.6.
Aha, but reiserfs's directories are indexed, are they not? Whereas
ext3's directories are flat and require a linear search for lookups
and modifications. This may be what Joerg's example highlights.
--
/ |
[|] Sean Neakums | Questions are a burden to others;
[|] <[email protected]> | answers a prison for oneself.
\ |
>From: Erik Andersen <[email protected]>
>> We don't need just another unrelated interface but a generic
>> transort. CDROM_SEND_PACKET is not a generic interface, it is limited
>> to ATAPI CD-ROM's.
>Wrong. It is a _generic CD-ROM packet interface. Thanks for not even
>spending the two seconds it would take reading the kernel source code
>to discover this.
>$ grep -l CDC_GENERIC_PACKET drivers/scsi/sr.c drivers/ide/ide-cd.c
>drivers/scsi/sr.c
>drivers/ide/ide-cd.c
That does not change anything.
Having a transport of limited usability is a problem for libscg.
BTW: If it turns out that people are interested in useful discussions,
I may put more effort in reading current Linux kernel sources.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From: Erik Andersen <[email protected]>
>> >cdrecord should use the CDROM_SEND_PACKET ioctl, then it would
>> >work regardless,
>>
>> This only prooves that you are uninformed :-(
>No. This only proves _you_ have not tried it. I've used the
>CDROM_SEND_PACKET ioctl on both SCSI and ATAPI cdrom devices.
>What do you need to do in cdrecord that cannot be done with it?
The only reason, why I did add support for it was to be able
to use a CD writer in my notbook circumventing the driver bugs
that prevent to use ise-scsi on top of PCATA.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
Hi,
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> That does not change anything.
>
> Having a transport of limited usability is a problem for libscg.
What exactly are you missing w/the packet IOCTL?
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
> > > > A Pentium 1200 running Linux-2.5.25 (ext3) results in:
> > > >
> > > > # star -xp -time < rock.tar.bz2
> > > > star: WARNING: Archive is bzip2 compressed, trying to use the -bz option.
> > > > star: 10372 blocks + 1536 bytes (total of 106210816 bytes = 103721.50k).
> > > > star: Total time 3190.483sec (32 kBytes/sec)
> > > > 53:10.490r 12.299u 2970.099s 93% 0M 0+0k 0st 0+0io 4411pf+0w
> > > >
> > > > You see, during the 53:20, the machine is only 7% idle!
> > >
> > > A Pentium 1200, eh?
> > > More like Pentium 120 or star just doesn't cut it.
> >
> > Now I'm actually pretty sure you meant 386DX/33!
> >
> > I don't know whom you're trying to fool, but even my P2/233
> > can get the work done in under 5 minutes:
> >
> > kala@hubert:/tmp$ time tar xjf rock.tar.bz2
> >
> > real 4m50.598s
> > user 0m36.700s
> > sys 1m51.860s
> >
> > Linux 2.4.19-pre10-ac2, reiserfs 3.6.
>
> Aha, but reiserfs's directories are indexed, are they not? Whereas
> ext3's directories are flat and require a linear search for lookups
> and modifications. This may be what Joerg's example highlights.
Unfortunately, I don't have an ext3 partition handy to perform a quick
test, but the following Joerg's accusation has been invalidated nonetheless:
> Solaris and FreeBSD put all the effort into one filesystem trying to make
> it as good as possible. In Linux, it seems that nobody prooved the overall
> concept of the kernel.
If that's the case, how come a Linux-powered Pentium 233 can do in
5 minutes for what a Solaris-powered Pentium 800 needs 7 minutes?
I believe Joerg owes a big big apology to Al Viro here.
T.
>From: Tomas Szepe <[email protected]>
>> A Pentium 1200 running Linux-2.5.25 (ext3) results in:
>>
>> # star -xp -time < rock.tar.bz2
>> star: WARNING: Archive is bzip2 compressed, trying to use the -bz option.
>> star: 10372 blocks + 1536 bytes (total of 106210816 bytes = 103721.50k).
>> star: Total time 3190.483sec (32 kBytes/sec)
>> 53:10.490r 12.299u 2970.099s 93% 0M 0+0k 0st 0+0io 4411pf+0w
>>
>> You see, during the 53:20, the machine is only 7% idle!
>A Pentium 1200, eh?
>More like Pentium 120 or star just doesn't cut it.
star uses less CPU time than GNU tar. As GNU tar uses a proprietary
archive format, I never use it if I may avoid to use it.
>--
>Athlon 1GHz:
>kala@nibbler:/tmp/1$ time tar xjf rock.tar.bz2
>real 3m19.703s
>user 0m9.870s
>sys 0m24.840s
>According to top, the system was ~90% idle during the extraction.
>Linux 2.4.19-rc1-ac3, reiserfs 3.6.
Well, I wrote that this has been done with ext3 (I also checked ext2
which is approx. the same speed.
I don't have access to a reiserfs system that has not been compiled
with debug and I don't like to put out false statements.
>PS. Solaris is over 60% slower than Linux 2.2/2.4 in common fs
>operations on my SMP SPARCstation 10.
If you make such statements, it would help a lot of you would mention
the Solaris version you are running. I am always running a recent
Solaris beta kernel - you may have used an outdated version.
The filesystem speed on Solaris did dramatically improve to the beginning
of the year 2001. This equates Solaris 8 01/01.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
> >> A Pentium 1200 running Linux-2.5.25 (ext3) results in:
> >>
> >> # star -xp -time < rock.tar.bz2
> >> star: WARNING: Archive is bzip2 compressed, trying to use the -bz option.
> >> star: 10372 blocks + 1536 bytes (total of 106210816 bytes = 103721.50k).
> >> star: Total time 3190.483sec (32 kBytes/sec)
> >> 53:10.490r 12.299u 2970.099s 93% 0M 0+0k 0st 0+0io 4411pf+0w
> >>
> >> You see, during the 53:20, the machine is only 7% idle!
>
>
> >A Pentium 1200, eh?
> >More like Pentium 120 or star just doesn't cut it.
>
> star uses less CPU time than GNU tar. As GNU tar uses a proprietary
> archive format, I never use it if I may avoid to use it.
How does this relate to the test? I got the archive directly off your
ftp site -> the software has been dealing with exactly the same format
in both cases.
> >Athlon 1GHz:
>
> >kala@nibbler:/tmp/1$ time tar xjf rock.tar.bz2
> >real 3m19.703s
> >user 0m9.870s
> >sys 0m24.840s
>
> >According to top, the system was ~90% idle during the extraction.
> >Linux 2.4.19-rc1-ac3, reiserfs 3.6.
>
> Well, I wrote that this has been done with ext3 (I also checked ext2
> which is approx. the same speed.
> I don't have access to a reiserfs system that has not been compiled
> with debug and I don't like to put out false statements.
I honestly doubt ext3 would perform significantly worse than what I've
observed with reiserfs.
Never mind, however, the sole aim of my having tested the extraction of
rock.tar.bz2 was to show how easily you get to accuse people on lkml of
being incompetent w/o having any real support for your claims.
> >PS. Solaris is over 60% slower than Linux 2.2/2.4 in common fs
> >operations on my SMP SPARCstation 10.
>
> If you make such statements, it would help a lot of you would mention
> the Solaris version you are running. I am always running a recent
> Solaris beta kernel - you may have used an outdated version.
Umm, let's see if I can fish out the install media from somewhere...
jup, Solaris 2.6 5/98.
T.
On Sun, Jul 14, 2002 at 04:18:37PM +0200, Joerg Schilling wrote:
> For a starter, it is easier to understand the SCSI concept of
> addressing than to understand the Linux concept. In addition,
> the SCSI addressing concept can be used on different platforms
> in a unique way. This helps people (and GUI writers) to use
> cdrecord on more than Linux only.
whether it is easier is matter of taste, however in a situation
where the kernel and 99% of other applications refer to something
as '/dev/scd0' I fail to see any benefit of having another scheme.
Do you want to suggest that all other Linux apps should now use
'-dev x,y,z' instead of normal device names?
There is another problem, with your scsi transport library you
are bypassing normal Linux devices. Try
mount /dev/scd0 /mnt
cdrecord -dev 0,0,0 -blank=fast
ls -al /mnt
Nice? It certainly isn't the fault of Linux if you choose to
bypass normal device usage and it can be very annoying not
only for beginners.
Richard
>From: Tomas Szepe <[email protected]>
>> >A Pentium 1200, eh?
>> >More like Pentium 120 or star just doesn't cut it.
>>
>> star uses less CPU time than GNU tar. As GNU tar uses a proprietary
>> archive format, I never use it if I may avoid to use it.
>How does this relate to the test? I got the archive directly off your
>ftp site -> the software has been dealing with exactly the same format
>in both cases.
Well. it took me only 8 years of repeated bug reports to make the GNU tar
maintaners fix the worst problems so it is finally able to extract standard
compliant tar archives ;-).
As recent GNU tar is still unable to _list_ those tar files correctly
(try ftp://ftp.fokus.gme.de/pub/unix/star/testscripts/gtarfail.tar
and ftp://ftp.fokus.gme.de/pub/unix/star/testscripts/gtarfail2.tar),
I would never trust GNU tar. A program that behaves inconsistent in
list vs. extract mode is not what I like to use.
The real problem of GNU tar is that is does still not create POSIX.1-1988
compliantarchives while star is able to create POSIX.1-2001 archives for a year.
This causes that many archives out on ftp servgers cannot be unpacked using
compliant implementations.
To understand the problem, please fetch a recent star distribution and use the
contained program 'tartest' together with
ftp://ftp.fokus.gme.de/pub/unix/star/testscripts/ustar-all-quicktest.tar
and the instructions in:
ftp://ftp.fokus.gme.de/pub/unix/star/testscripts/README.quicktest
to find the POSIX deviations in GNU tar.
>> >Athlon 1GHz:
>>
>> >kala@nibbler:/tmp/1$ time tar xjf rock.tar.bz2
>> >real 3m19.703s
>> >user 0m9.870s
>> >sys 0m24.840s
>>
>> >According to top, the system was ~90% idle during the extraction.
>> >Linux 2.4.19-rc1-ac3, reiserfs 3.6.
>>
>> Well, I wrote that this has been done with ext3 (I also checked ext2
>> which is approx. the same speed.
>> I don't have access to a reiserfs system that has not been compiled
>> with debug and I don't like to put out false statements.
>I honestly doubt ext3 would perform significantly worse than what I've
>observed with reiserfs.
Just try it, I did try it.
>Never mind, however, the sole aim of my having tested the extraction of
>rock.tar.bz2 was to show how easily you get to accuse people on lkml of
>being incompetent w/o having any real support for your claims.
It was (as I mentioned before) to show that there need to be some sort
of high level coordination to make Linux better and address the needs of the
future.
>> >PS. Solaris is over 60% slower than Linux 2.2/2.4 in common fs
>> >operations on my SMP SPARCstation 10.
>>
>> If you make such statements, it would help a lot of you would mention
>> the Solaris version you are running. I am always running a recent
>> Solaris beta kernel - you may have used an outdated version.
>Umm, let's see if I can fish out the install media from somewhere...
>jup, Solaris 2.6 5/98.
So did you compare Solaris performance with a 4 year old Linux?
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From: Richard Zidlicky <[email protected]>
>
>> For a starter, it is easier to understand the SCSI concept of
>> addressing than to understand the Linux concept. In addition,
>> the SCSI addressing concept can be used on different platforms
>> in a unique way. This helps people (and GUI writers) to use
>> cdrecord on more than Linux only.
>whether it is easier is matter of taste, however in a situation
>where the kernel and 99% of other applications refer to something
>as '/dev/scd0' I fail to see any benefit of having another scheme.
>Do you want to suggest that all other Linux apps should now use
>'-dev x,y,z' instead of normal device names?
Did I request this? No, definitely not. Hoewver, it helps a lot
if a GUI for cdrecord may use cdrecord to find potential drives
and if there is a unique addressing scheme.
BTW: did you ever look at Solaris / HP-UX, ... and the way they
name disks?
someting like: /dev/{r}dsk/c0t0d0s0
This is SCSI bus, target, lun and slice.
>There is another problem, with your scsi transport library you
>are bypassing normal Linux devices. Try
> mount /dev/scd0 /mnt
> cdrecord -dev 0,0,0 -blank=fast
> ls -al /mnt
>Nice? It certainly isn't the fault of Linux if you choose to
>bypass normal device usage and it can be very annoying not
>only for beginners.
It is not a fault of cdrecord either.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
[OT stuff scrapped]
> >I honestly doubt ext3 would perform significantly worse than what I've
> >observed with reiserfs.
> Just try it, I did try it.
Someone else will have to carry out the test, I really can't free up
any of my partitions for a re-mkfs.
> >Never mind, however, the sole aim of my having tested the extraction of
> >rock.tar.bz2 was to show how easily you get to accuse people on lkml of
> >being incompetent w/o having any real support for your claims.
>
> It was (as I mentioned before) to show that there need to be some sort
> of high level coordination to make Linux better and address the needs of the
> future.
Works great from what I can tell.
> >> >PS. Solaris is over 60% slower than Linux 2.2/2.4 in common fs
> >> >operations on my SMP SPARCstation 10.
> >>
> >> If you make such statements, it would help a lot of you would mention
> >> the Solaris version you are running. I am always running a recent
> >> Solaris beta kernel - you may have used an outdated version.
>
> >Umm, let's see if I can fish out the install media from somewhere...
> >jup, Solaris 2.6 5/98.
>
> So did you compare Solaris performance with a 4 year old Linux?
No, but you did the comparison with the most recent version of Solaris for
me, only it was on IA-32. And the result is still the same, Linux wins hands
down.
T.
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> BTW: did you ever look at Solaris / HP-UX, ... and the way they
> name disks?
>
> someting like: /dev/{r}dsk/c0t0d0s0
> This is SCSI bus, target, lun and slice.
I wonder what they'll change it to in order to support
network attached storage.
Rik
--
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/ http://distro.conectiva.com/
>From: Tomas Szepe <[email protected]>
>> >Umm, let's see if I can fish out the install media from somewhere...
>> >jup, Solaris 2.6 5/98.
>>
>> So did you compare Solaris performance with a 4 year old Linux?
>No, but you did the comparison with the most recent version of Solaris for
>me, only it was on IA-32. And the result is still the same, Linux wins hands
>down.
The differences are not big enough to prove this. I did use a slow old IDE
disk. As you see from my timings, the Solaris test results have been mainly
caused by disk speed.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
[email protected] (Tomas Szepe) writes:
> [OT stuff scrapped]
>
> > >I honestly doubt ext3 would perform significantly worse than what I've
> > >observed with reiserfs.
> > Just try it, I did try it.
>
> Someone else will have to carry out the test, I really can't free up
> any of my partitions for a re-mkfs.
I'm running tar (the regular version not star) right now on an Athlon @
850. The fs is ext3 and the disk is a scsi drive.
So far, tar has been running for 17 min 25 sec, and that's what top says:
CPU states: 1.7% user, 98.2% system, 0.0% nice, 0.0% idle
(FYI, nothing else is taking some large amount of cpu time)
So I would say Joerg is right... :-(
--
Mathieu Chouquet-Stringer E-Mail : [email protected]
It is exactly because a man cannot do a thing that he is a
proper judge of it.
-- Oscar Wilde
[email protected] (Mathieu Chouquet-Stringer) writes:
> I'm running tar (the regular version not star) right now on an Athlon @
> 850. The fs is ext3 and the disk is a scsi drive.
> So far, tar has been running for 17 min 25 sec, and that's what top says:
> CPU states: 1.7% user, 98.2% system, 0.0% nice, 0.0% idle
>
> (FYI, nothing else is taking some large amount of cpu time)
>
> So I would say Joerg is right... :-(
BTW my kernel is 2.4.18 vanilla. I'll keep you posted when it's done.
--
Mathieu Chouquet-Stringer E-Mail : [email protected]
It is exactly because a man cannot do a thing that he is a
proper judge of it.
-- Oscar Wilde
On Sun, 2002-07-14 at 21:21, Mathieu Chouquet-Stringer wrote:
> I'm running tar (the regular version not star) right now on an Athlon @
> 850. The fs is ext3 and the disk is a scsi drive.
> So far, tar has been running for 17 min 25 sec, and that's what top says:
> CPU states: 1.7% user, 98.2% system, 0.0% nice, 0.0% idle
^^^^^^^^^^^^^^^^^^^^^^^^
Why are using PIO mode devices ?
[email protected] (Alan Cox) writes:
> On Sun, 2002-07-14 at 21:21, Mathieu Chouquet-Stringer wrote:
> > I'm running tar (the regular version not star) right now on an Athlon @
> > 850. The fs is ext3 and the disk is a scsi drive.
> > So far, tar has been running for 17 min 25 sec, and that's what top says:
> > CPU states: 1.7% user, 98.2% system, 0.0% nice, 0.0% idle
> ^^^^^^^^^^^^^^^^^^^^^^^^
>
> Why are using PIO mode devices ?
I'm using SCSI :
Jul 13 16:35:50 mcs kernel: SCSI subsystem driver Revision: 1.00
Jul 13 16:35:50 mcs kernel: PCI: Found IRQ 10 for device 00:0b.0
Jul 13 16:35:50 mcs kernel: sym.0.11.0: setting PCI_COMMAND_PARITY...
Jul 13 16:35:50 mcs kernel: sym0: <895> rev 0x1 on pci bus 0 device 11 function 0 irq 10
Jul 13 16:35:50 mcs kernel: sym0: Tekram NVRAM, ID 7, Fast-40, LVD, parity checking
Jul 13 16:35:50 mcs kernel: sym0: SCSI BUS has been reset.
Jul 13 16:35:50 mcs kernel: spurious 8259A interrupt: IRQ7.
Jul 13 16:35:50 mcs kernel: scsi0 : sym-2.1.17a
And the drive on which I'm running the test
Jul 13 16:35:50 mcs kernel: Vendor: IBM Model: DDYS-T18350N Rev: S96H
Jul 13 16:35:50 mcs kernel: Type: Direct-Access ANSI SCSI revision: 03
[...]
Jul 13 16:35:51 mcs kernel: sym0:0:0: tagged command queuing enabled, command queue depth 32.
[...]
Jul 13 16:35:51 mcs kernel: sym0:0: FAST-40 WIDE SCSI 80.0 MB/s ST (25.0 ns, offset 31)
--
Mathieu Chouquet-Stringer E-Mail : [email protected]
It is exactly because a man cannot do a thing that he is a
proper judge of it.
-- Oscar Wilde
Forgot to copy lkml... :-(
[email protected] (Alan Cox) writes:
> On Sun, 2002-07-14 at 21:21, Mathieu Chouquet-Stringer wrote:
> > I'm running tar (the regular version not star) right now on an Athlon @
> > 850. The fs is ext3 and the disk is a scsi drive.
> > So far, tar has been running for 17 min 25 sec, and that's what top says:
> > CPU states: 1.7% user, 98.2% system, 0.0% nice, 0.0% idle
> ^^^^^^^^^^^^^^^^^^^^^^^^
>
> Why are using PIO mode devices ?
That's a scsi drive so I would guess it uses dma. The interface:
Jul 13 16:35:50 mcs kernel: sym.0.11.0: setting PCI_COMMAND_PARITY...
Jul 13 16:35:50 mcs kernel: sym0: <895> rev 0x1 on pci bus 0 device 11 function 0 irq 10
Jul 13 16:35:50 mcs kernel: sym0: Tekram NVRAM, ID 7, Fast-40, LVD, parity checking
Jul 13 16:35:50 mcs kernel: sym0: SCSI BUS has been reset.
Jul 13 16:35:50 mcs kernel: spurious 8259A interrupt: IRQ7.
Jul 13 16:35:50 mcs kernel: scsi0 : sym-2.1.17a
And the drive:
Jul 13 16:35:50 mcs kernel: Vendor: IBM Model: DDYS-T18350N Rev: S96H
Jul 13 16:35:50 mcs kernel: Type: Direct-Access ANSI SCSI revision: 03
[...]
Jul 13 16:35:51 mcs kernel: sym0:0:0: tagged command queuing enabled, command queue depth 32.
[...]
Jul 13 16:35:51 mcs kernel: Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
[...]
Jul 13 16:35:51 mcs kernel: sym0:0: FAST-40 WIDE SCSI 80.0 MB/s ST (25.0 ns, offset 31)
--
Mathieu Chouquet-Stringer E-Mail : [email protected]
It is exactly because a man cannot do a thing that he is a
proper judge of it.
-- Oscar Wilde
On Sun, 2002-07-14 at 21:43, Mathieu Chouquet-Stringer wrote:
> [email protected] (Alan Cox) writes:
> > On Sun, 2002-07-14 at 21:21, Mathieu Chouquet-Stringer wrote:
> > > I'm running tar (the regular version not star) right now on an Athlon @
> > > 850. The fs is ext3 and the disk is a scsi drive.
> > > So far, tar has been running for 17 min 25 sec, and that's what top says:
> > > CPU states: 1.7% user, 98.2% system, 0.0% nice, 0.0% idle
> > ^^^^^^^^^^^^^^^^^^^^^^^^
> >
> > Why are using PIO mode devices ?
>
> I'm using SCSI :
> Jul 13 16:35:50 mcs kernel: SCSI subsystem driver Revision: 1.00
Intriguing. Something horrible is happening on your system to see 98%
system time off a bus mastering DMA controller. It should only look like
that on things like an AHA152x
commence Alan Cox quotation:
> On Sun, 2002-07-14 at 21:43, Mathieu Chouquet-Stringer wrote:
> > [email protected] (Alan Cox) writes:
> > > On Sun, 2002-07-14 at 21:21, Mathieu Chouquet-Stringer wrote:
> > > > I'm running tar (the regular version not star) right now on an Athlon @
> > > > 850. The fs is ext3 and the disk is a scsi drive.
> > > > So far, tar has been running for 17 min 25 sec, and that's what top says:
> > > > CPU states: 1.7% user, 98.2% system, 0.0% nice, 0.0% idle
> > > ^^^^^^^^^^^^^^^^^^^^^^^^
> > >
> > > Why are using PIO mode devices ?
> >
> > I'm using SCSI :
> > Jul 13 16:35:50 mcs kernel: SCSI subsystem driver Revision: 1.00
>
> Intriguing. Something horrible is happening on your system to see 98%
> system time off a bus mastering DMA controller. It should only look like
> that on things like an AHA152x
I saw something similar when I tried untarring the file, and assumed
it was due to trying to put many thousands of files into a single flat
directory.
--
/ |
[|] Sean Neakums | Questions are a burden to others;
[|] <[email protected]> | answers a prison for oneself.
\ |
> it was due to trying to put many thousands of files into a single flat
> directory.
exactly. ext2 (and many other FSs) are simply not designed for obscenely
large directories. it's unclear why Joerg brought up this rather obvious strawman.
Am Son, 2002-07-14 um 23.46 schrieb Alan Cox:
> On Sun, 2002-07-14 at 21:21, Mathieu Chouquet-Stringer wrote:
> > I'm running tar (the regular version not star) right now on an Athlon @
> > 850. The fs is ext3 and the disk is a scsi drive.
^^^^^^^^^^^^^^^^^^^^
> > So far, tar has been running for 17 min 25 sec, and that's what top says:
> > CPU states: 1.7% user, 98.2% system, 0.0% nice, 0.0% idle
> ^^^^^^^^^^^^^^^^^^^^^^^^
> Why are using PIO mode devices ?
In a SCSI system?
--
Servus,
Daniel
[email protected] (Alan Cox) writes:
> Intriguing. Something horrible is happening on your system to see 98%
> system time off a bus mastering DMA controller. It should only look like
> that on things like an AHA152x
Well, to be frank, I wouldn't blame the scsi subsystem: the disk is almost
idle and procinfo -d gives an average of 6 irqs (using the default 5 sec
delay between 2 updates)...
--
Mathieu Chouquet-Stringer E-Mail : [email protected]
It is exactly because a man cannot do a thing that he is a
proper judge of it.
-- Oscar Wilde
On Sun, Jul 14, 2002 at 05:05:46PM -0400, Mark Hahn wrote:
> > So I would say Joerg is right... :-(
>
> providing a strawman does not make and argument correct.
> we've known that ext2/3 (like ffs and many other FSs)
> simply is not designed for large directories.
>
> the big mystery is why Jeorg thought this was relevant to the atapi flamefest.
I think you're right.
--
Mathieu Chouquet-Stringer E-Mail : [email protected]
It is exactly because a man cannot do a thing that he is a
proper judge of it.
-- Oscar Wilde
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> >Now your silly PCATA stupid ass Tailgate Bridge that you are boasting
> >about does some of the worst things anyone could ever imagine.
>
> ???? Looks stupid (like dou did not get the message).
I guess I need to break it down to simple terms, and hoped that your
broadcast in expertise could cover your mouth. This makes it harder for
me because I do not communicate well over email.
Firewire 1394, USB, Parallel Port, PCMCIA/CardBus are all effective
tailgates via an alternate physical transport layer and protocol.
Therefore it should be obvious many different versions of the hardware get
it wrong. Now in other operating system which are commerial based, there
are device specific drivers to perform soft-protocol corrections to
generate the appearance of a perfect product. Much as in optics, here is
another case where two wrongs make a right. COSTAR for Hubble Space
Telescope is real world example.
> >BurnProof is a result lame devices which improperly hold off the bus
> >because release the BUSY Bit while still performing transfers.
> >The very fact that a huge pile of devices went into the market place
> >based on SFF-8020 rev 2.5 total roasts your strawman, please try again.
>
> Again: this is completely unrelated to the problem. Why do you introduce
> it?
This to counter your grand statement of there are no problems with any
ATAPI devices after 1993. Yet anyone who know about the physical bus
layer to support the ATAPI protocol, would realize problems generated by
that document. Now moving forward in time, there is a conflict in
operations between SFF-8020 rev 2.6 and Mt. Fuji and MMC and lastly Mt.
Rainer.
If you knew anything about the production industry, and maybe you do, it
would be obvious that most of the Far East and Pacific Ring hardware
people are still creating product based on SFF-8020 a retired document.
Yet you stand here and glorify straight SCSI as the transport with a
wrapper assuming that Mt. Fuji and MMC 1/2/3 out of T10 and STA will solve
all the problems of the world.
Your world thinks all hardware regardless is perfect, and that is nice.
My world knows different and attempts to let you continue to enjoy the
fantasy.
> >So instead of whining about what is there and not from your location in
> >end user land, try and offer something useful like a preferred API to
> >allow clean packet-driver interfaces. Doing that little would allow the
> >transport layer people and the transistion-api folks to user land to
> >greatly increase compatablity. Then you would not need 5 interfaces for
> >Linux.
>
> >Have a good day.
>
> I am not whining, but you answer with unrelated stuff. Why? Are you missing
> experience and arguments?
I just asked you for a formal preferred model coresponding to READ/WRITE
10/16 fixed to the OS standard CDB as the base of a Packet Interface, yet
you counter with a redirect. :-/
Put up or shut up.
Insert "Joerg Schilling" Perfect Packet Interface for review.
Thank you for the chortle,
Andre Hedrick
LAD Storage Consulting Group
[email protected] (Mathieu Chouquet-Stringer) writes:
> Well, to be frank, I wouldn't blame the scsi subsystem: the disk is almost
> idle and procinfo -d gives an average of 6 irqs (using the default 5 sec
> delay between 2 updates)...
As promised, here is the final result (even if we know it's a fs issue):
mchouque - /tmp/joerg %/usr/bin/time tar jxf rock.tar.bz2
19.69user 6796.49system 1:56:05elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (319major+951minor)pagefaults 0swaps
/usr/bin/time tar jxf rock.tar.bz2 19.69s user 6796.49s system 97% cpu 1:56:05.11 total
--
Mathieu Chouquet-Stringer E-Mail : [email protected]
It is exactly because a man cannot do a thing that he is a
proper judge of it.
-- Oscar Wilde
Hi,
On 14 Jul 2002, Mathieu Chouquet-Stringer wrote:
> mchouque - /tmp/joerg %/usr/bin/time tar jxf rock.tar.bz2
> 19.69user 6796.49system 1:56:05elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (319major+951minor)pagefaults 0swaps
> /usr/bin/time tar jxf rock.tar.bz2 19.69s user 6796.49s system 97% cpu 1:56:05.11 total
Impressive. Did the interval between file writes get longer as time
passed?
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
> > it was due to trying to put many thousands of files into a single flat
> > directory.
> exactly. ext2 (and many other FSs) are simply not designed for obscenely
> large directories. it's unclear why Joerg brought up this rather obvious
> strawman.
... and blamed it on the overall kernel architecture.
Well, at least I learned something today: with the ext[23] largedir
problem out of the way, Linux can be as much as five times faster
than Solaris in certain, very common FS operations.
Joerg Schilling wrote:
>
> ...
> ftp://ftp.fokus.gmd.de/pub/unix/star/testscript/rock.tar.bz2
>
> It is a tar archive that contains 207,440 empty files and one directory
> using the names of the files in the 'rock' subdirectory from the
> freedb project. This is a snapshot taken on May 30th 2002.
>
> If you extract this archive on a machine running Solaris or FreeBSD,
> it takes less than 7 minutes.
With ext3 and the htree directory indexing, a 500MHz PIII does
it in 57 seconds.
tar xjf ~/rock.tar.bz2 19.63s user 29.42s system 84% cpu 57.879 total
> A pentium 800 running Solaris 9 results in:
>
> # star -xp -time < rock.tar.bz2
> star: WARNING: Archive is bzip2 compressed, trying to use the -bz option.
> star: 10372 blocks + 1536 bytes (total of 106210816 bytes = 103721.50k).
> star: Total time 405.988sec (255 kBytes/sec)
> 6:46.051r 12.920u 63.420s 18% 0M 0+0k 0st 0+0io 0pf+0w
>
> You see during the 6:43, the machine is 82% idle.
>
> A Pentium 1200 running Linux-2.5.25 (ext3) results in:
>
> # star -xp -time < rock.tar.bz2
> star: WARNING: Archive is bzip2 compressed, trying to use the -bz option.
> star: 10372 blocks + 1536 bytes (total of 106210816 bytes = 103721.50k).
> star: Total time 3190.483sec (32 kBytes/sec)
> 53:10.490r 12.299u 2970.099s 93% 0M 0+0k 0st 0+0io 4411pf+0w
>
> You see, during the 53:20, the machine is only 7% idle!
>
> It wasted 2900 seconds of CPU time on Linux. Let me guess: this was done
> inside the function strcmp().
Nope. ext3 and ext2 directories use the traditional first-fit
search-from-start for directories. So adding 200k files to
a single directory is pathological.
> There are ~ 5 different filesystems on Linux, but none if the projects seem
> to care about the code outside the FS low level code. I suspect, that
> this is not any better if you use reiserfs.
>
> Solaris and FreeBSD put all the effort into one filesystem trying to make
> it as good as possible. In Linux, it seems that nobody prooved the overall
> concept of the kernel.
Apply http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.25/ext3-htree.patch
to your 2.5.25 tree, mount with `-o index' and enjoy watching ext3 eat
Solaris and FreeBSD's lunch.
htree isn't quite ready yet and development seems a little moribund.
It'd be nice to get it finished off.
-
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> >From [email protected] Fri Jul 12 22:18:47 2002
>
> >As much as I hate IDE, IDE isn't going away. All my systems use SCSI
> >so on machines that have CD/ROMS, I use your libraries and your tools.
>
> >Maybe somebody should make CD/ROM code that directly talks to IDE via
> >/dev/hdwhatever, instead of expecting you to modify your code that
> >has worked so well for so long.
>
> This would be a really bad idea.
>
> Such a change would force me to add a 6th (and unneeded) new interface.
> Why? What problem would be solved if you did introduce such an interface?
>
Well for one thing it eliminates the requirement to
include SCSI interface code on machines that don't
have SCSI. That's the practical aspect.
Now, the esoteric. Do you truly think that it is
proper to encapsulate devices in various layers?
The IDE interface, if it wasn't for the bug-workarounds,
is just a floppy disk interface that uses a different
controller chip. It is register-based, not message-
based. If you throw in a message-based control layer
(SCSI), what problems are you solving? It's a
rhetorical question. No answer is required.
Like I said before, your stuff works great. I use
it because I use SCSI. Somebody needs to write some
CD access code for IDE drives because they are not
going away. Maybe that 'somebody' is not you. You
certainly don't want to mess up good working code.
But I, for one, would feel a bit better about the
future IDE/CDROM code if you wrote it.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
Windows-2000/Professional isn't.
On Sun, Jul 14, 2002 at 04:14:35PM -0600, Thunder from the hill wrote:
> Hi,
Hi,
> On 14 Jul 2002, Mathieu Chouquet-Stringer wrote:
> > mchouque - /tmp/joerg %/usr/bin/time tar jxf rock.tar.bz2
> > 19.69user 6796.49system 1:56:05elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
> > 0inputs+0outputs (319major+951minor)pagefaults 0swaps
> > /usr/bin/time tar jxf rock.tar.bz2 19.69s user 6796.49s system 97% cpu 1:56:05.11 total
>
> Impressive. Did the interval between file writes get longer as time
> passed?
Well, I don't really know but I guess so (I ran tar without the verbose
switch, would there be another way to see this? I guess stracing should do
it but it adds such an overhead I don't think that's such a good idea).
Anyway, I'm going to erase this nice directory now...
--
Mathieu Chouquet-Stringer E-Mail : [email protected]
It is exactly because a man cannot do a thing that he is a
proper judge of it.
-- Oscar Wilde
Hi,
On Sun, 14 Jul 2002, Mathieu Chouquet-Stringer wrote:
> Anyway, I'm going to erase this nice directory now...
Please time the erase!
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
On Sun, Jul 14, 2002 at 09:38:16PM -0600, Thunder from the hill wrote:
> Hi,
Hi,
> Please time the erase!
Had the same idea: :-)
mchouque - /tmp/joerg %/usr/bin/time rm -rf rock
0.18user 6.10system 0:09.27elapsed 67%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (139major+20minor)pagefaults 0swaps
/usr/bin/time rm -rf rock 0.18s user 6.10s system 67% cpu 9.277 total
Not too bad if you think it took 1 hour something to create the
directory...
--
Mathieu Chouquet-Stringer E-Mail : [email protected]
It is exactly because a man cannot do a thing that he is a
proper judge of it.
-- Oscar Wilde
On Sun, 14 Jul 2002, Mathieu Chouquet-Stringer wrote:
> On Sun, Jul 14, 2002 at 09:38:16PM -0600, Thunder from the hill wrote:
> > Hi,
>
> Hi,
>
> > Please time the erase!
>
> Had the same idea: :-)
> mchouque - /tmp/joerg %/usr/bin/time rm -rf rock
> 0.18user 6.10system 0:09.27elapsed 67%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (139major+20minor)pagefaults 0swaps
> /usr/bin/time rm -rf rock 0.18s user 6.10s system 67% cpu 9.277 total
>
> Not too bad if you think it took 1 hour something to create the
> directory...
No wonder - with FFS/ext2/ext3 adding files into directory takes linear
scan of the entire thing (presuming that directory was originally empty)
on each file. Removing them in the order they were added is much faster -
removed entry is simply merged with the previous one in the same block.
So you skip the already emptied parts fast - each of these blocks consists
of a single (empty) entry. Then you get to the block containing the entry
you are removing and either mark it unused (if it's the first one in block)
or make the previous one (first in block) longer. You do only one name
comparison (empty entries are recognized by zero inode number). So rm -rf
is O(blocks * entries). Creating all that is O(entries^2) with much worse
constant.
FFS never had been intended to handle directories with huge amount of
entries. Neiter are most of its derivatives, including ext2 and ext3
(and realistically it's not worth the extra complexity for most of
applications).
All of that is fs-specific and has nothing to the rest of kernel (or
to the kind of kernel, actually - if you look at *BSD implementations
of the same thing you'll find the same behaviour). If you change
layout (as it had been done for several UFS variants and for htree
variant of ext3) you can get faster algorithms. Whether it's practically
interesting is a separate question, though - even if kernel side is
fast, you still have userland side to deal with and it tends to behave
quite badly when confronted with huge directories.
Steffen Persvold wrote:
>
> On Sun, 14 Jul 2002, Andrew Morton wrote:
>
> > > Solaris and FreeBSD put all the effort into one filesystem trying to make
> > > it as good as possible. In Linux, it seems that nobody prooved the overall
> > > concept of the kernel.
> >
> > Apply http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.25/ext3-htree.patch
> > to your 2.5.25 tree, mount with `-o index' and enjoy watching ext3 eat
> > Solaris and FreeBSD's lunch.
> >
> > htree isn't quite ready yet and development seems a little moribund.
> > It'd be nice to get it finished off.
> >
>
> Hi Andrew,
>
> Will the patch ever be backported to 2.4 ?
>
That's a 2.5 forward-port, actually. Christopher, Daniel, Stephen
and co are working against 2.4.
It mostly works, but seems to still need a bit of debugging of corner
cases and general hardening up.
-
>From [email protected] Mon Jul 15 11:11:59 2002
>On Sun, 14 Jul 2002, Rik van Riel wrote:
>> > BTW: did you ever look at Solaris / HP-UX, ... and the way they
>> > name disks?
>> >
>> > someting like: /dev/{r}dsk/c0t0d0s0
>> > This is SCSI bus, target, lun and slice.
>>
>> I wonder what they'll change it to in order to support
>> network attached storage.
>>
>Actually notthing:
>dbtecnocasa:{root}:/>format
>Searching for disks...done
>c2t1d0: configured with capacity of 6.56MB
>c2t1d30: configured with capacity of 34.04GB
>c2t1d31: configured with capacity of 34.04GB
>c2t1d81: configured with capacity of 34.04GB
>AVAILABLE DISK SELECTIONS:
> 0. c0t0d0 <SUN18G cyl 7506 alt 2 hd 19 sec 248>
> /pci@1f,4000/scsi@3/sd@0,0
> 1. c2t1d0 <EMC-SYMMETRIX-5567 cyl 14 alt 2 hd 15 sec 64>
> /pci@4,2000/scsi@1/sd@1,0
> 2. c2t1d30 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
> /pci@4,2000/scsi@1/sd@1,1e
> 3. c2t1d31 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
> /pci@4,2000/scsi@1/sd@1,1f
> 4. c2t1d81 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
> /pci@4,2000/scsi@1/sd@1,51
>except of c0t0d0 everything else is network attached...
How is it attached? Using FACL or ISCSI?
In any case, it seems to be a natural solution to do it this way.
In order to access a network disk, you need to obtain the right to
do so first. Once this has been done, the netork subsystem just looks
like a new SCSI bus.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
On Sun, 14 Jul 2002, Rik van Riel wrote:
> Date: Sun, 14 Jul 2002 17:18:35 -0300 (BRT)
> From: Rik van Riel <[email protected]>
> To: Joerg Schilling <[email protected]>
> Cc: [email protected], [email protected],
> [email protected]
> Subject: Re: IDE/ATAPI in 2.5
>
> On Sun, 14 Jul 2002, Joerg Schilling wrote:
>
> > BTW: did you ever look at Solaris / HP-UX, ... and the way they
> > name disks?
> >
> > someting like: /dev/{r}dsk/c0t0d0s0
> > This is SCSI bus, target, lun and slice.
>
> I wonder what they'll change it to in order to support
> network attached storage.
>
Actually notthing:
dbtecnocasa:{root}:/>format
Searching for disks...done
c2t1d0: configured with capacity of 6.56MB
c2t1d30: configured with capacity of 34.04GB
c2t1d31: configured with capacity of 34.04GB
c2t1d81: configured with capacity of 34.04GB
AVAILABLE DISK SELECTIONS:
0. c0t0d0 <SUN18G cyl 7506 alt 2 hd 19 sec 248>
/pci@1f,4000/scsi@3/sd@0,0
1. c2t1d0 <EMC-SYMMETRIX-5567 cyl 14 alt 2 hd 15 sec 64>
/pci@4,2000/scsi@1/sd@1,0
2. c2t1d30 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
/pci@4,2000/scsi@1/sd@1,1e
3. c2t1d31 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
/pci@4,2000/scsi@1/sd@1,1f
4. c2t1d81 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
/pci@4,2000/scsi@1/sd@1,51
except of c0t0d0 everything else is network attached...
On Sun, Jul 14, 2002 at 10:13:19PM +0200, Joerg Schilling wrote:
>
> BTW: did you ever look at Solaris / HP-UX, ... and the way they
> name disks?
still have fresh memories how everything was renamed and broke
some of my apps.
> someting like: /dev/{r}dsk/c0t0d0s0
>
> This is SCSI bus, target, lun and slice.
You can have that type of names with devfs, no need to work
around kernel functionality in cdrecord.
On pure IDE hardware that doesn't make sense anyway, in fact
cdrecord mapping is strongly counterintuitive here.
CD-RW = hdd = bus1,target2 but cdrecord sees it as 0,0,0.
> >There is another problem, with your scsi transport library you
> >are bypassing normal Linux devices. Try
> > mount /dev/scd0 /mnt
> > cdrecord -dev 0,0,0 -blank=fast
> > ls -al /mnt
>
> >Nice? It certainly isn't the fault of Linux if you choose to
> >bypass normal device usage and it can be very annoying not
> >only for beginners.
>
> It is not a fault of cdrecord either.
A cure would be nice and I don't see what the kernel could do
to solve this problem as long as cdrecord insists on talking
to the SCSI bus directly.
If nothing else, cdrecord manpage
- should make a big fat warning about it
- should stop claiming that it is safe to suid cdrecord
The potential for breakage is huge, people run automounters on CD's,
file managers may try to mount the CD without asking the user.
Richard
I think someone is misunderstanding some industry buzz words here ...
NAS refers to Network Attached Storage, as in via NFS, AFS, et al.
What your showing in the output of the Solaris format command are
raw SCSI LUNS attached via fibre channel (fabric or loop) or native scsi.
--Buddy
-----Original Message-----
From: [email protected]
[mailto:[email protected]]On Behalf Of Joerg Schilling
Sent: Monday, July 15, 2002 2:21 AM
To: [email protected]; [email protected]
Cc: [email protected];
[email protected]; [email protected];
[email protected]
Subject: Re: IDE/ATAPI in 2.5
>From [email protected] Mon Jul 15 11:11:59 2002
>On Sun, 14 Jul 2002, Rik van Riel wrote:
>> > BTW: did you ever look at Solaris / HP-UX, ... and the way they
>> > name disks?
>> >
>> > someting like: /dev/{r}dsk/c0t0d0s0
>> > This is SCSI bus, target, lun and slice.
>>
>> I wonder what they'll change it to in order to support
>> network attached storage.
>>
>Actually notthing:
>dbtecnocasa:{root}:/>format
>Searching for disks...done
>c2t1d0: configured with capacity of 6.56MB
>c2t1d30: configured with capacity of 34.04GB
>c2t1d31: configured with capacity of 34.04GB
>c2t1d81: configured with capacity of 34.04GB
>AVAILABLE DISK SELECTIONS:
> 0. c0t0d0 <SUN18G cyl 7506 alt 2 hd 19 sec 248>
> /pci@1f,4000/scsi@3/sd@0,0
> 1. c2t1d0 <EMC-SYMMETRIX-5567 cyl 14 alt 2 hd 15 sec 64>
> /pci@4,2000/scsi@1/sd@1,0
> 2. c2t1d30 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
> /pci@4,2000/scsi@1/sd@1,1e
> 3. c2t1d31 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
> /pci@4,2000/scsi@1/sd@1,1f
> 4. c2t1d81 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
> /pci@4,2000/scsi@1/sd@1,51
>except of c0t0d0 everything else is network attached...
How is it attached? Using FACL or ISCSI?
In any case, it seems to be a natural solution to do it this way.
In order to access a network disk, you need to obtain the right to
do so first. Once this has been done, the netork subsystem just looks
like a new SCSI bus.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353
Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling
ftp://ftp.fokus.gmd.de/pub/unix
>From [email protected] Sun Jul 14 21:02:38 2002
>J?rg,
>could you please stop starting all your mails like this one and
>insulting every people who don't obviously understand what you
>nearly didn't explain in 30 mails ? You're whinning about people
>who don't take you seriously, but you don't try to be constructive.
I have already been constructive and wrote some lines that showed up
how a driver tree could look like. Unfortunately nobody did take this
proposal as a base for a future discussion.
Being constuctive does not make sense if it is obvious that other
people don't listen.
The discussion now looks better than a day ago... If I have read my mail
from the weekend, I may try to draw an image as it may be that people in this
list do not like text.
>Since you claim to have very clear ideas about what you'd like to
>see in the kernel, please post a clear and complete proposal here
>(with another subject, BTW since most people won't read it else).
I already did (see above).
>This way, you'll be able to argue on technical points with some
>competent and clever people, but you won't if you continue to
>measure your personal experience to that of others. You say you
>don't want to talk to them because, unlike you, they don't know
>what a kernel is, but this is non sense. They may have a different
>conception than you, that's all.
I would be happy to hear about concepts. Currently it looks as if at least
some people like to keep everything as it is. This is not a conceptional OS but
a grown structure. If you like to keep code maintainable for a long time,
you need to clean up the thicket from time to time. This seems to be something
that is not done frequent enough with most free software. I know how to
clean up software and I do it from time to time with my software.
Look e.g. at star, it started in 1982 and the fully usable version is now 17
years old. It was still possible to add something like POSIX.1-2001 compliance
within ~ 2 weeks. Try to do this with GNU tar...
>>From your claimed experience, you seem to be 45, but from your
>attitude, 8. I think reality is between both, but I hope the
>former.
Well, I try to match with the crowd ;-)
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
Thunder from the hill <[email protected]> writes:
>implemented in Linux (any more), and will never ever be". If we explicitly
>exclude hardware, where do we end?!
Cleaner drivers, sane layers and an overall better working OS?
SCNR
Henning
--
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH [email protected]
Am Schwabachgrund 22 Fon.: 09131 / 50654-0 [email protected]
D-91054 Buckenhof Fax.: 09131 / 50654-20
>
>How is it attached? Using FACL or ISCSI?
>
>In any case, it seems to be a natural solution to do it this way.
>
>In order to access a network disk, you need to obtain the right to
>do so first. Once this has been done, the netork subsystem just looks
>like a new SCSI bus.
The naming is nothing we should matter with. Ask Greg KH what he
thinks about kernel enforcing a naming policy ;)
What we are going to with 2.5 is a fully userland naming policy.
So if we end up defining a uniform API to send packet commands to
those type of devices (wether they are on a SCSI bus, ATAPI, USB,
1394, ...), then the best approach is probably for the various
drivers involved to declare a class "SCSI packet" in driverfs,
then let whatever userland naming policy decide how that should
be called. The internal kernel infrastructure (wether it'sa common
driver or split drivers providing you the same interface) is of
no matter to cdrecord.
Ben.
>From: Andre Hedrick <[email protected]>
>> >Now your silly PCATA stupid ass Tailgate Bridge that you are boasting
>> >about does some of the worst things anyone could ever imagine.
>>
>> ???? Looks stupid (like dou did not get the message).
>I guess I need to break it down to simple terms, and hoped that your
>broadcast in expertise could cover your mouth. This makes it harder for
>me because I do not communicate well over email.
>Firewire 1394, USB, Parallel Port, PCMCIA/CardBus are all effective
>tailgates via an alternate physical transport layer and protocol.
>Therefore it should be obvious many different versions of the hardware get
>it wrong. Now in other operating system which are commerial based, there
>are device specific drivers to perform soft-protocol corrections to
>generate the appearance of a perfect product. Much as in optics, here is
>another case where two wrongs make a right. COSTAR for Hubble Space
>Telescope is real world example.
If _you_ had the experience you pretend, then why do you claim that the fact
that I cannot use ide-scsi with a PCATA connection to my CD writer is caused
by bad hardware?
As the drive becomes usable with CDROM_SEND_PACKET and is completely unusable
via ide-scsi it is obvious that the reason cannot be a hardware problem but must
be a driver design bug.
>If you knew anything about the production industry, and maybe you do, it
>would be obvious that most of the Far East and Pacific Ring hardware
>people are still creating product based on SFF-8020 a retired document.
If this affects the drivers, then there need to be a workaround regardless of
the driver layering model in use.
>> I am not whining, but you answer with unrelated stuff. Why? Are you missing
>> experience and arguments?
>I just asked you for a formal preferred model coresponding to READ/WRITE
>10/16 fixed to the OS standard CDB as the base of a Packet Interface, yet
>you counter with a redirect. :-/
>Put up or shut up.
>Insert "Joerg Schilling" Perfect Packet Interface for review.
Why didn't you read my short abstract I send out yesterday?
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From: "Richard B. Johnson" <[email protected]>
>> >From [email protected] Fri Jul 12 22:18:47 2002
>>
>> >As much as I hate IDE, IDE isn't going away. All my systems use SCSI
>> >so on machines that have CD/ROMS, I use your libraries and your tools.
>>
>> >Maybe somebody should make CD/ROM code that directly talks to IDE via
>> >/dev/hdwhatever, instead of expecting you to modify your code that
>> >has worked so well for so long.
>>
>> This would be a really bad idea.
>>
>> Such a change would force me to add a 6th (and unneeded) new interface.
>> Why? What problem would be solved if you did introduce such an interface?
>>
>Well for one thing it eliminates the requirement to
>include SCSI interface code on machines that don't
>have SCSI. That's the practical aspect.
>Now, the esoteric. Do you truly think that it is
>proper to encapsulate devices in various layers?
>The IDE interface, if it wasn't for the bug-workarounds,
>is just a floppy disk interface that uses a different
>controller chip. It is register-based, not message-
>based. If you throw in a message-based control layer
>(SCSI), what problems are you solving? It's a
>rhetorical question. No answer is required.
If you like to use the floppy only via the ATA registers, then
you don't like to be able to format media?
BTW: the overhead to set up DMA is higher than the overhead to set up a SCSI
CDB.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From: Richard Zidlicky <[email protected]>
>> >There is another problem, with your scsi transport library you
>> >are bypassing normal Linux devices. Try
>> > mount /dev/scd0 /mnt
>> > cdrecord -dev 0,0,0 -blank=fast
>> > ls -al /mnt
>>
>> >Nice? It certainly isn't the fault of Linux if you choose to
>> >bypass normal device usage and it can be very annoying not
>> >only for beginners.
>>
>> It is not a fault of cdrecord either.
>A cure would be nice and I don't see what the kernel could do
>to solve this problem as long as cdrecord insists on talking
>to the SCSI bus directly.
>If nothing else, cdrecord manpage
> - should make a big fat warning about it
> - should stop claiming that it is safe to suid cdrecord
>The potential for breakage is huge, people run automounters on CD's,
>file managers may try to mount the CD without asking the user.
The bad news is that it seems that the automounters are part of the GUIs and
not well enough documented. There should be:
- Something like the Solaris volume manager that is part of the base OS
and kernel folks should forbid GUI folks to add such tasks to the GUI
- The volume manager should have a documented interface that allows
programs like e.g. cdrecord to gain exclusive access to a CD drive.
Then the problem above could be solved.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
As my textual description has not been read, here comes a acsii art
of the proposal for a driver structure:
User programs
----------------------------------------------------------------
----------------------------------------------------------------
| |
| Kernel driver interface |
| |
----------------------------------------------------------------
| |
-------------------------------- ------------------------------
| | | |
| One or more SCSI | | One or more simple |
| target drivers | | Block based drivers |
| e.g. sd.c | | e.g. dsata.c |
| | | |
| | | |
-------------------------------- ------------------------------
| |
| |------------------ Locked when
| | SCSI glue
---------------------------------------------------------------- Interface is
| | | used for a
| | | specific
| SCSI glue layer | Block access glue layer | drive.
| | |
| Will check if target | |
| supports SCSI commands | |
| and lock Block access | |
| layer in this case. | |
| | |
----------------------------------------------------------------
| |
| Low level glue |
| |
----------------------------------------------------------------
| SCSI CDB/IF | SCSI CDB/IF | Block IF
-------------------------------- ------------------------------
| | | |
| | | |
| SCSI HBA driver | | ATA HBA driver |
| | | deals with simple ATA |
| Only supports | | interface and with |
| SCSI CDB interface | | ATA packet interface |
| | | |
| | | |
-------------------------------- ------------------------------
All HBA drivers include DMA setup callback functions to make the
target drivers completely independant from the HW.
Communication for all types of interfaces is HW address cookie
based and done via callback.
Above the glue layer, no knowledge about SCSI disconnect/reconnect
is needed.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
On Mon, 15 Jul 2002, Joerg Schilling wrote:
> Date: Mon, 15 Jul 2002 11:20:45 +0200 (CEST)
> From: Joerg Schilling <[email protected]>
> To: [email protected], [email protected]
> Cc: [email protected], [email protected],
> [email protected], [email protected]
> Subject: Re: IDE/ATAPI in 2.5
>
> >From [email protected] Mon Jul 15 11:11:59 2002
> >On Sun, 14 Jul 2002, Rik van Riel wrote:
>
> >> > BTW: did you ever look at Solaris / HP-UX, ... and the way they
> >> > name disks?
> >> >
> >> > someting like: /dev/{r}dsk/c0t0d0s0
> >> > This is SCSI bus, target, lun and slice.
> >>
> >> I wonder what they'll change it to in order to support
> >> network attached storage.
> >>
> >Actually notthing:
>
> >dbtecnocasa:{root}:/>format
> >Searching for disks...done
>
> >c2t1d0: configured with capacity of 6.56MB
> >c2t1d30: configured with capacity of 34.04GB
> >c2t1d31: configured with capacity of 34.04GB
> >c2t1d81: configured with capacity of 34.04GB
>
>
> >AVAILABLE DISK SELECTIONS:
> > 0. c0t0d0 <SUN18G cyl 7506 alt 2 hd 19 sec 248>
> > /pci@1f,4000/scsi@3/sd@0,0
> > 1. c2t1d0 <EMC-SYMMETRIX-5567 cyl 14 alt 2 hd 15 sec 64>
> > /pci@4,2000/scsi@1/sd@1,0
> > 2. c2t1d30 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
> > /pci@4,2000/scsi@1/sd@1,1e
> > 3. c2t1d31 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
> > /pci@4,2000/scsi@1/sd@1,1f
> > 4. c2t1d81 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
> > /pci@4,2000/scsi@1/sd@1,51
>
> >except of c0t0d0 everything else is network attached...
>
>
> How is it attached? Using FACL or ISCSI?
FACL
>
> In any case, it seems to be a natural solution to do it this way.
>
> In order to access a network disk, you need to obtain the right to
> do so first. Once this has been done, the netork subsystem just looks
> like a new SCSI bus.
Exactly.
Then I should say that this naming scheme is usefull, but so ugly...
On Mon, 15 Jul 2002, Joerg Schilling wrote:
> I would be happy to hear about concepts. Currently it looks as if at
> least some people like to keep everything as it is. This is not a
> conceptional OS but a grown structure. If you like to keep code
> maintainable for a long time, you need to clean up the thicket from time
> to time.
I couldn't agree more. Now, why do you oppose cleaning up
the "use scsi as everyone's mid layer" hack and putting a
better generic abstraction in place ?
regards,
Rik
--
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/ http://distro.conectiva.com/
>From: Rik van Riel <[email protected]>
>On Mon, 15 Jul 2002, Joerg Schilling wrote:
>> I would be happy to hear about concepts. Currently it looks as if at
>> least some people like to keep everything as it is. This is not a
>> conceptional OS but a grown structure. If you like to keep code
>> maintainable for a long time, you need to clean up the thicket from time
>> to time.
>I couldn't agree more. Now, why do you oppose cleaning up
>the "use scsi as everyone's mid layer" hack and putting a
>better generic abstraction in place ?
I never said this.
I would like to see a integrated aproach where the mid level prevents something
like ide-cd to access ATAPI drives. In this case, there is a unique address
space and generic SCSI transport systems like libscg will be able to work
in a way that is easy to understand by outsiders and does not force me to add
one workaround after the other.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
[email protected] said:
> As my textual description has not been read, here comes a acsii art of
> the proposal for a driver structure:
Actually, the diagram is very similar to what we possess internally today.
This represents where I think the SCSI stack is going:
User Level
--------------------------------------------------------------
+----------------+ +---------------------+
| Block Interface| | Character Interface |
+----------------+ +---------------------+
| |
| +--------------------+
| |
+---------------------+
| Block Layer | +----------------------+
| provides queueing | | Device Prep Functions|
| and elevator if | | st, sd, sg, etc. |
| necessary. |-----| Does transport |
| Also provides all | | translation |
| support functions | +----------------------+
| that may be shared | |
| across transports. | +----------------------+
| | | Stackable error |
| |-----| handling (not well |
| | | thought out yet) |
| | +----------------------+
| |
| | +--------------------+
| | | Transport helper |
| | | Contains all fns |
| | | not shareable by |
| | | other transports |
+---------------------+ +--------------------+
| |
| |
+-------------------------+
| Card driver (deals with |
| attached transport |
| translation |
+-------------------------+
As you can see, I plan to leverage the generic block layer to build the
transport stack. The upper layer drivers become mere request_prep_fns whose
job is to translate the request to a transport specific command.
The struct request will go all the way down to the card driver, but the card
driver may choose to deal only with the transport translation if it wants.
The driving vision for this is to move into the generic block layer as much of
the individual transport stack as makes sense (i.e. if other transports can
make use of the functions), so, for example, Tag command queueing is already
in there and shared between IDE and SCSI.
The ultimate end point will be when the correct balance between what belongs
in the generic block layer and what belongs in the transport helpers is
achieved. I speculate that, for CDROMS, this will lead to two small request
prep modules sharing quite a large helper library (The helper library would do
SCSI command translation, and probably the IDE prep module would fix up the
transport command for the specific device). However, I don't rule out that it
would lead to a single prep module for both IDE and SCSI.
The device naming issues are totally separate from the above. I intend that
driverfs will cope with them. Internally, the block layer just thinks of the
stack as a series of entry points for physical devices. Driverfs gives the
card driver freedom to provide a hierarchical ascii device name as it sees
fit. Hopefully this will finesse the so called persistent binding issues that
plague solaris.
Ultimately, this means that host and channel is subsumed into the card
identification scheme, target ID may no longer be a number and even LUN may
end up being a LUN hierarchy representation. As we do this, we'll move to
exposing persistent names, so the user shouldn't necessarily care about this.
James Bottomley
Hi,
On Mon, 15 Jul 2002, Henning P. Schmiedehausen wrote:
> >implemented in Linux (any more), and will never ever be". If we explicitly
> >exclude hardware, where do we end?!
>
> Cleaner drivers, sane layers and an overall better working OS?
That's what lots of people thought who are now sitting on their OSes,
wondering why the number of users shrinks and shrinks and shrinks...
It's IMHO not a really good idea to drop support for decent hardware,
especially if the hardware itself might work better than the replacement
which keeps up the supported standard.
Regards,
Thunder
--
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o? K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y-
------END GEEK CODE BLOCK------
>I think someone is misunderstanding some industry buzz words here ...
>NAS refers to Network Attached Storage, as in via NFS, AFS, et al.
>What your showing in the output of the Solaris format command are
>raw SCSI LUNS attached via fibre channel (fabric or loop) or native
>scsi.
Just as a matter of clarification, the above two examples NFS and AFS
are most certainly NOT Network Attached Storage aka NAS. The former is
an simple Networked File System, and the latter the first practical
distributed file system (ie: multiple client access to shares), but they
both provide access to storage resources at the FILE SYSTEM level.
The term 'NAS' (and SAN for that matter as the terms are pretty much
used interchangeably these days) refer to moving raw disk protocol
Command Descriptor Blocks aka CDBs and its associated data across the
network at the BLOCK level. But storage folks generically regard the
terms as: NAS refers to BLOCK level storage over IP networks, and SAN
to BLOCK level storage over a Fibre Channel setup.
The reason being a Storage Area Network is generally a closed, secure,
and seperate entity from ones IP network, while Network Attached
Storage is storage added directly into an existing IP network. Of
course this opens up all of related security considerations when dealing
with IP networks, but a discussion of the related issues is beyond the
scope of this reply, and dangerously off-topic.
Nicholas 'trying to keep it real' Bellinger
>>From [email protected] Mon Jul 15 11:11:59 2002
>>On Sun, 14 Jul 2002, Rik van Riel wrote:
>>> > BTW: did you ever look at Solaris / HP-UX, ... and the way they
>>> > name disks?
>>> >
>>> > someting like: /dev/{r}dsk/c0t0d0s0
>>> > This is SCSI bus, target, lun and slice.
>>>
>>> I wonder what they'll change it to in order to support
>>> network attached storage.
>>>
>>Actually notthing:
>>dbtecnocasa:{root}:/>format
>>Searching for disks...done
>>c2t1d0: configured with capacity of 6.56MB
>>c2t1d30: configured with capacity of 34.04GB
>>c2t1d31: configured with capacity of 34.04GB
>>c2t1d81: configured with capacity of 34.04GB
>>AVAILABLE DISK SELECTIONS:
>> 0. c0t0d0 <SUN18G cyl 7506 alt 2 hd 19 sec 248>
>> /pci@1f,4000/scsi@3/sd@0,0
>> 1. c2t1d0 <EMC-SYMMETRIX-5567 cyl 14 alt 2 hd 15 sec 64>
>> /pci@4,2000/scsi@1/sd@1,0
>> 2. c2t1d30 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
>> /pci@4,2000/scsi@1/sd@1,1e
>> 3. c2t1d31 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
>> /pci@4,2000/scsi@1/sd@1,1f
>> 4. c2t1d81 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
>> /pci@4,2000/scsi@1/sd@1,51
>>except of c0t0d0 everything else is network attached...
>How is it attached? Using FACL or ISCSI?
>In any case, it seems to be a natural solution to do it this way.
>In order to access a network disk, you need to obtain the right to
>do so first. Once this has been done, the netork subsystem just looks
>like a new SCSI bus.
>J?rg
I would disagree here. NAS _does_ in fact typically refer to storage that
is network attached and acessed via some form of network filesystem i.e.
NFS and friends. Typically, this is billed as just plugging more storage
into the network.
See http://www.quantum.com/Products/NAS+Servers/Default.htm for an
example. SearchStorage has a little blurb (which I think could use some
additional clarity)
http://whatis.techtarget.com/definition/0,289893,sid9_gci214410,00.html
SAN on the other hand typically requires a different HBA than that NIC you
use to access the network. Common access methods are via FiberChannel or
iSCSI. As of now SAN based storage also requires more effort, skill and
cost to provision. I would expect that to go down over time.
--jauder
On 15 Jul 2002, Nick Bellinger wrote:
>
>
> >I think someone is misunderstanding some industry buzz words here ...
> >NAS refers to Network Attached Storage, as in via NFS, AFS, et al.
> >What your showing in the output of the Solaris format command are
> >raw SCSI LUNS attached via fibre channel (fabric or loop) or native
> >scsi.
>
> Just as a matter of clarification, the above two examples NFS and AFS
> are most certainly NOT Network Attached Storage aka NAS. The former is
> an simple Networked File System, and the latter the first practical
> distributed file system (ie: multiple client access to shares), but they
> both provide access to storage resources at the FILE SYSTEM level.
>
> The term 'NAS' (and SAN for that matter as the terms are pretty much
> used interchangeably these days) refer to moving raw disk protocol
> Command Descriptor Blocks aka CDBs and its associated data across the
> network at the BLOCK level. But storage folks generically regard the
> terms as: NAS refers to BLOCK level storage over IP networks, and SAN
> to BLOCK level storage over a Fibre Channel setup.
>
> The reason being a Storage Area Network is generally a closed, secure,
> and seperate entity from ones IP network, while Network Attached
> Storage is storage added directly into an existing IP network. Of
> course this opens up all of related security considerations when dealing
> with IP networks, but a discussion of the related issues is beyond the
> scope of this reply, and dangerously off-topic.
>
>
> Nicholas 'trying to keep it real' Bellinger
>
>
> >>From [email protected] Mon Jul 15 11:11:59 2002
> >>On Sun, 14 Jul 2002, Rik van Riel wrote:
>
> >>> > BTW: did you ever look at Solaris / HP-UX, ... and the way they
> >>> > name disks?
> >>> >
> >>> > someting like: /dev/{r}dsk/c0t0d0s0
> >>> > This is SCSI bus, target, lun and slice.
> >>>
> >>> I wonder what they'll change it to in order to support
> >>> network attached storage.
> >>>
> >>Actually notthing:
>
> >>dbtecnocasa:{root}:/>format
> >>Searching for disks...done
>
> >>c2t1d0: configured with capacity of 6.56MB
> >>c2t1d30: configured with capacity of 34.04GB
> >>c2t1d31: configured with capacity of 34.04GB
> >>c2t1d81: configured with capacity of 34.04GB
>
>
> >>AVAILABLE DISK SELECTIONS:
> >> 0. c0t0d0 <SUN18G cyl 7506 alt 2 hd 19 sec 248>
> >> /pci@1f,4000/scsi@3/sd@0,0
> >> 1. c2t1d0 <EMC-SYMMETRIX-5567 cyl 14 alt 2 hd 15 sec 64>
> >> /pci@4,2000/scsi@1/sd@1,0
> >> 2. c2t1d30 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
> >> /pci@4,2000/scsi@1/sd@1,1e
> >> 3. c2t1d31 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
> >> /pci@4,2000/scsi@1/sd@1,1f
> >> 4. c2t1d81 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
> >> /pci@4,2000/scsi@1/sd@1,51
>
> >>except of c0t0d0 everything else is network attached...
>
>
> >How is it attached? Using FACL or ISCSI?
>
> >In any case, it seems to be a natural solution to do it this way.
>
> >In order to access a network disk, you need to obtain the right to
> >do so first. Once this has been done, the netork subsystem just looks
> >like a new SCSI bus.
>
> >J?rg
>
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>
On Sun, 14 Jul 2002, Andrew Morton wrote:
> > It wasted 2900 seconds of CPU time on Linux. Let me guess: this was done
> > inside the function strcmp().
>
> Nope. ext3 and ext2 directories use the traditional first-fit
> search-from-start for directories. So adding 200k files to
> a single directory is pathological.
>
> > There are ~ 5 different filesystems on Linux, but none if the projects seem
> > to care about the code outside the FS low level code. I suspect, that
> > this is not any better if you use reiserfs.
> >
> > Solaris and FreeBSD put all the effort into one filesystem trying to make
> > it as good as possible. In Linux, it seems that nobody prooved the overall
> > concept of the kernel.
>
> Apply http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.25/ext3-htree.patch
> to your 2.5.25 tree, mount with `-o index' and enjoy watching ext3 eat
> Solaris and FreeBSD's lunch.
I didn't benchmark, but how much lunch is left? UFS_DIRHASH was
introduced into FreeBSD by Ian Dowse a year ago (released in FreeBSD
4.4), and activated in FreeBSD 4.5's generic table. In case you missed
that, FreeBSD 4.6 is out since about four weeks.
options UFS_DIRHASH #Improve performance on big directories
http://www.freebsd.org/releases/4.4R/relnotes-i386.html#AEN197
http://www.freebsd.org/releases/4.5R/relnotes-i386.html#AEN250
http://www.cnri.dit.ie/Downloads/Malone_2001_bsdcon.pdf
The latter document by David Malone (November 2001) claims "MH 33 k
files create: 70s to 2.5s, pack: 240s to 2.5s, rm: 4.7s to 2s."
So might be ext3fs comes just in time for dessert, or is htree so much
faster than UFS_DIRHASH?
--
Matthias Andree
[Joerg, sorry about the two copies, I'm new to gnus and missed that I
have to do a wide reply]
Joerg Schilling <[email protected]> writes:
> As my textual description has not been read, here comes a acsii art
> of the proposal for a driver structure:
So what you are suggesting is a lot of layering between the clients
and the hardware. If you look at the history of Linux I would regard
most of the "middle layer" code as failures, what one does end up with
is a middle layer that is some sort of least common denominator that
makes noone happy. A much better choice is to place common code (what
usually ends up in a middle layer) in a library, so that a driver can
choose either to use the common code, or to implement its own better
version that can take advantage of the hardware if possible.
/Christer
--
"Just how much can I get away with and still go to heaven?"
Followup to: <[email protected]>
By author: [email protected] (Eric W. Biederman)
In newsgroup: linux.dev.kernel
> >
> > I'm talking specifically about ATAPI devices here. As we have already covered,
> > not all ATA devices are ATAPI, but unless I'm completely off the wall, ATAPI is
> > SCSI over IDE, and should be able to be driven as such. The lack of access to
> > that interface using the established interface mechanisms just bites.
>
> ATAPI is SCSI like C is C++. There are strong similarities but they
> are regulated by different groups, and have slightly different semantics.
> Last I checked cdrecord already compensates for those differences but they
> are there.
>
This is a good analogy, and in that general vein don't forget there
are considerable differences between different versions of SCSI as
well. Just like there are, in many ways, bigger differences between
K&R C and C99 than between contemporary C and C++. It should be
possible to write code that handles either one, either generically
(obviously the best) or with appropriate conditionals.
-hpa
--
<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt <[email protected]>
Followup to: <[email protected]>
By author: Erik Andersen <[email protected]>
In newsgroup: linux.dev.kernel
>
> Of course making user space do this is pretty lame. But we
> have a much better way. Each cdrom device registers with the
> uniform cdrom driver, which can easily assign each registered
> cdrom device a major and minor. That scanning for cdroms would
> be as simple as
> for i in /dev/cdrom* ; do
> open ($i, O_RDONLY|O_NONBLOCK)) < 0) {
> /* Found a cdrom drive */
> }
>
Yes, something like that would be nice.
However, the case still remains that I think Linus' proposed overall
packet infrastructure is the right thing to do. That way a uniform
API would be available for poking at any device that supports MMC (is
that the correct term these days?) commands, regardless of the type
of device and the lower-level transports.
People have -- correctly -- corrected me on the "ATAPI = SCSI over
IDE" issue. When I think of SCSI, I tend to think of what a network
engineer would call "the upper data link layer", i.e. the command
packet frame format. I did not mean to imply that the physical
interface (PHY), or the lower data link layer (MAC) where the same. I
also realize that there are differences, but *from what I've seen*
they seem to be relatively minor.
I meant to start a discussion, not "call a vote", especially not
w.r.t. the lower-level implementation details.
-hpa
--
<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt <[email protected]>
As apposed to creating a huge list of URL's to illustrate my point, since
EMC LUNS were provided below, i'll point you to EMC's "NAS" solution:
http://www.emc.com/products/networking/celerra.jsp?openfolder=storage_networ
king
They are called "EMC Celerra File Servers" and to keep you from reading the
whole site I'll sum it up here. It's a card that you slide into the
symmetrix to provide Network Attached Storage, AKA NFS front end to storage
that may or may not be also available on a SAN which implies a connection to
a fabric.
Regards,
--Buddy
-----Original Message-----
From: [email protected]
[mailto:[email protected]]On Behalf Of Nick Bellinger
Sent: Monday, July 15, 2002 11:28 AM
To: Buddy Lumpkin
Cc: [email protected]
Subject: RE: IDE/ATAPI in 2.5
>I think someone is misunderstanding some industry buzz words here ...
>NAS refers to Network Attached Storage, as in via NFS, AFS, et al.
>What your showing in the output of the Solaris format command are
>raw SCSI LUNS attached via fibre channel (fabric or loop) or native
>scsi.
Just as a matter of clarification, the above two examples NFS and AFS
are most certainly NOT Network Attached Storage aka NAS. The former is
an simple Networked File System, and the latter the first practical
distributed file system (ie: multiple client access to shares), but they
both provide access to storage resources at the FILE SYSTEM level.
The term 'NAS' (and SAN for that matter as the terms are pretty much
used interchangeably these days) refer to moving raw disk protocol
Command Descriptor Blocks aka CDBs and its associated data across the
network at the BLOCK level. But storage folks generically regard the
terms as: NAS refers to BLOCK level storage over IP networks, and SAN
to BLOCK level storage over a Fibre Channel setup.
The reason being a Storage Area Network is generally a closed, secure,
and seperate entity from ones IP network, while Network Attached
Storage is storage added directly into an existing IP network. Of
course this opens up all of related security considerations when dealing
with IP networks, but a discussion of the related issues is beyond the
scope of this reply, and dangerously off-topic.
Nicholas 'trying to keep it real' Bellinger
>>From [email protected] Mon Jul 15 11:11:59 2002
>>On Sun, 14 Jul 2002, Rik van Riel wrote:
>>> > BTW: did you ever look at Solaris / HP-UX, ... and the way they
>>> > name disks?
>>> >
>>> > someting like: /dev/{r}dsk/c0t0d0s0
>>> > This is SCSI bus, target, lun and slice.
>>>
>>> I wonder what they'll change it to in order to support
>>> network attached storage.
>>>
>>Actually notthing:
>>dbtecnocasa:{root}:/>format
>>Searching for disks...done
>>c2t1d0: configured with capacity of 6.56MB
>>c2t1d30: configured with capacity of 34.04GB
>>c2t1d31: configured with capacity of 34.04GB
>>c2t1d81: configured with capacity of 34.04GB
>>AVAILABLE DISK SELECTIONS:
>> 0. c0t0d0 <SUN18G cyl 7506 alt 2 hd 19 sec 248>
>> /pci@1f,4000/scsi@3/sd@0,0
>> 1. c2t1d0 <EMC-SYMMETRIX-5567 cyl 14 alt 2 hd 15 sec 64>
>> /pci@4,2000/scsi@1/sd@1,0
>> 2. c2t1d30 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
>> /pci@4,2000/scsi@1/sd@1,1e
>> 3. c2t1d31 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
>> /pci@4,2000/scsi@1/sd@1,1f
>> 4. c2t1d81 <EMC-SYMMETRIX-5567 cyl 37178 alt 2 hd 30 sec 64>
>> /pci@4,2000/scsi@1/sd@1,51
>>except of c0t0d0 everything else is network attached...
>How is it attached? Using FACL or ISCSI?
>In any case, it seems to be a natural solution to do it this way.
>In order to access a network disk, you need to obtain the right to
>do so first. Once this has been done, the netork subsystem just looks
>like a new SCSI bus.
>J?rg
On Mon, Jul 15, 2002 at 03:07:13PM +0200, Joerg Schilling wrote:
>
> >From: Richard Zidlicky <[email protected]>
>
> >> >There is another problem, with your scsi transport library you
> >> >are bypassing normal Linux devices. Try
> >> > mount /dev/scd0 /mnt
> >> > cdrecord -dev 0,0,0 -blank=fast
> >> > ls -al /mnt
> >>
> >> >Nice? It certainly isn't the fault of Linux if you choose to
> >> >bypass normal device usage and it can be very annoying not
> >> >only for beginners.
> >>
> >> It is not a fault of cdrecord either.
>
> >A cure would be nice and I don't see what the kernel could do
> >to solve this problem as long as cdrecord insists on talking
> >to the SCSI bus directly.
>
> >If nothing else, cdrecord manpage
> > - should make a big fat warning about it
> > - should stop claiming that it is safe to suid cdrecord
>
> >The potential for breakage is huge, people run automounters on CD's,
> >file managers may try to mount the CD without asking the user.
>
> The bad news is that it seems that the automounters are part of the GUIs and
> not well enough documented. There should be:
>
> - Something like the Solaris volume manager that is part of the base OS
> and kernel folks should forbid GUI folks to add such tasks to the GUI
Solaris vold? Thanks no, floppy access was so easy in SunOS before the
days of the volume manager.
> - The volume manager should have a documented interface that allows
> programs like e.g. cdrecord to gain exclusive access to a CD drive.
much simpler, cdrom driver needs an ioctl to claim exclusive use of
the device and cdrecord than needs to use that ioctl.
Richard
>From [email protected] Tue Jul 16 12:37:56 2002
>> >The potential for breakage is huge, people run automounters on CD's,
>> >file managers may try to mount the CD without asking the user.
>>
>> The bad news is that it seems that the automounters are part of the GUIs and
>> not well enough documented. There should be:
>>
>> - Something like the Solaris volume manager that is part of the base OS
>> and kernel folks should forbid GUI folks to add such tasks to the GUI
>Solaris vold? Thanks no, floppy access was so easy in SunOS before the
>days of the volume manager.
.... and it is even simpler since vold is present. Call volcheck to tell vold
that the media changed or use a SCSI floppy which supports to tell the kernel
that a media change did happen.
>> - The volume manager should have a documented interface that allows
>> programs like e.g. cdrecord to gain exclusive access to a CD drive.
>much simpler, cdrom driver needs an ioctl to claim exclusive use of
>the device and cdrecord than needs to use that ioctl.
This will not work. What should happen when the drive is mounted?
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From: Rik van Riel <[email protected]>
>> I would be happy to hear about concepts. Currently it looks as if at
>> least some people like to keep everything as it is. This is not a
>> conceptional OS but a grown structure. If you like to keep code
>> maintainable for a long time, you need to clean up the thicket from time
>> to time.
>I couldn't agree more. Now, why do you oppose cleaning up
>the "use scsi as everyone's mid layer" hack and putting a
>better generic abstraction in place ?
Why do you try to invert my postings?
I definitely like to see a better overall solution and I am talking about it
the same way during the last 4 years. Why should I change my mind if Linux
did not change in between?
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From [email protected] Mon Jul 15 18:09:52 2002
>Actually, the diagram is very similar to what we possess internally today.
>This represents where I think the SCSI stack is going:
> User Level
>--------------------------------------------------------------
> +----------------+ +---------------------+
> | Block Interface| | Character Interface |
> +----------------+ +---------------------+
> | |
> | +--------------------+
Why should the character interface be connected to the block layer?
This would contradict UNIX rules.
> | |
> +---------------------+
> | Block Layer | +----------------------+
> | provides queueing | | Device Prep Functions|
> | and elevator if | | st, sd, sg, etc. |
> | necessary. |-----| Does transport |
> | Also provides all | | translation |
> | support functions | +----------------------+
> | that may be shared | |
> | across transports. | +----------------------+
> | | | Stackable error |
> | |-----| handling (not well |
> | | | thought out yet) |
> | | +----------------------+
Why should the error handlers interface with the block layer?
This is not true for UNIX and it would not help. However, it would
be nice to have a way to make the blocklayer find out that e.g.
only the last sector of a multi sector read ahead instruction
did fail.
> | |
> | | +--------------------+
> | | | Transport helper |
> | | | Contains all fns |
> | | | not shareable by |
> | | | other transports |
> +---------------------+ +--------------------+
> | |
> | |
> +-------------------------+
> | Card driver (deals with |
> | attached transport |
> | translation |
> +-------------------------+
>As you can see, I plan to leverage the generic block layer to build the
>transport stack. The upper layer drivers become mere request_prep_fns whose
>job is to translate the request to a transport specific command.
>The struct request will go all the way down to the card driver, but the card
>driver may choose to deal only with the transport translation if it wants.
>The driving vision for this is to move into the generic block layer as much of
>the individual transport stack as makes sense (i.e. if other transports can
>make use of the functions), so, for example, Tag command queueing is already
>in there and shared between IDE and SCSI.
AFAIK, tagged command queuing is a SCSI specific property, why should this
be part of a generif block layer?
The block layer shuld simply send more than one request before waiting for
the requests to finish. Then a tagged command queuing aware sd.c could choose
to fill up the tagged queue that is handled by the SCSI glue layer.
>The ultimate end point will be when the correct balance between what belongs
>in the generic block layer and what belongs in the transport helpers is
>achieved. I speculate that, for CDROMS, this will lead to two small request
>prep modules sharing quite a large helper library (The helper library would do
>SCSI command translation, and probably the IDE prep module would fix up the
>transport command for the specific device). However, I don't rule out that it
>would lead to a single prep module for both IDE and SCSI.
With my proposal, anything that speaks SCSI is used via SCSI commands and a
generic SCSI driver like scg.c could access the SCSI transport aware drives
of any type. An important (communication baesed) feature of my SCSI glue layer
would be to make it possible to insert SCSI commands from scg.c without making
sd.c believe that something strange and unexpected happened to one of it's drives.
>The device naming issues are totally separate from the above. I intend that
Agreed.
>driverfs will cope with them. Internally, the block layer just thinks of the
>stack as a series of entry points for physical devices. Driverfs gives the
>card driver freedom to provide a hierarchical ascii device name as it sees
>fit. Hopefully this will finesse the so called persistent binding issues that
>plague solaris.
It would help, if somebody would correct the current SCSI addressng scheme used
in Linux. Linux currently uses something called BUS/channel/target/lun.
This does not reflect reality.
What Linux calls a SCSI bus is definitely not a SCSI bus but a SCSI HBA card.
What Linux calls a channel really is one of possibly more SCSI busses going
off one of the SCSI HBA cards. It makes sense to just count SCSI busses.
>Ultimately, this means that host and channel is subsumed into the card
>identification scheme, target ID may no longer be a number and even LUN may
>end up being a LUN hierarchy representation. As we do this, we'll move to
>exposing persistent names, so the user shouldn't necessarily care about this.
Why do you believe that you need to have something that is not a bumber?
As a result of your drawing, I introduced the block layer (which is the
traditional UNIX block cache). I also had the idea that it may be a good
idea to make the simple block based I/O driver part or slave of the SCSI
CDB based driver (although it would use a different portal to the glue layer).
This way, it would be possible to have e.g. a single name space for all
hard disks.
Let me add my modified artwork:
User programs
----------------------------------------------------------------
----------------------------------------------------------------
| |
| Kernel driver interface |
| |
----------------------------------------------------------------
| |
------------------------ |
| | |
| Block I/O layer | | Raw I/O IF
| | |
------------------------ |
| |
Block I/O IF | |
| |
----------------------------------------------------------------
| ------------------------------
| | |
| One or more SCSI | Block based I/O |
| target drivers | handling routines |
| e.g. sd.c | not used if SCSI based |
| st.c, scg.c | I/O is possible to a |
| | specific drive |
------------------------------
----------------------------------------------------------------
| |
SCSI CDB IF | Block based IF |------------------ Locked when
| | SCSI glue
---------------------------------------------------------------- Interface is
| | | used for a
| | | specific
| SCSI glue layer | Block access glue layer | drive.
| | |
| Will check if target | |
| supports SCSI commands | |
| and lock Block access | |
| layer in this case. | |
| | |
----------------------------------------------------------------
| |
| Low level glue |
| |
----------------------------------------------------------------
| SCSI CDB/IF | SCSI CDB/IF | Block IF
-------------------------------- ------------------------------
| | | |
| | | |
| SCSI HBA driver | | ATA HBA driver |
| | | deals with simple ATA |
| Only supports | | interface and with |
| SCSI CDB interface | | ATA packet interface |
| | | |
| | | |
-------------------------------- ------------------------------
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
U?ytkownik Joerg Schilling napisa?:
>>From [email protected] Mon Jul 15 18:09:52 2002
>
>
>>Actually, the diagram is very similar to what we possess internally today.
>
>
>>This represents where I think the SCSI stack is going:
>
>
>> User Level
>>--------------------------------------------------------------
>
>
>>+----------------+ +---------------------+
>>| Block Interface| | Character Interface |
>>+----------------+ +---------------------+
>> | |
>> | +--------------------+
>
>
> Why should the character interface be connected to the block layer?
> This would contradict UNIX rules.
Bullshit. UNIX sees even every single block device as both: char and block
even under the same major number. The reason above is - tape devices.
> Why should the error handlers interface with the block layer?
> This is not true for UNIX and it would not help. However, it would
> be nice to have a way to make the blocklayer find out that e.g.
> only the last sector of a multi sector read ahead instruction
> did fail.
You answer yourself.
>>The driving vision for this is to move into the generic block layer as much of
>>the individual transport stack as makes sense (i.e. if other transports can
>>make use of the functions), so, for example, Tag command queueing is already
>>in there and shared between IDE and SCSI.
>
>
> AFAIK, tagged command queuing is a SCSI specific property, why should this
> be part of a generif block layer?
Becouse it is not SCSI specific. SerialATA will *nearly* require it.
And there are ATA disks out there which do it *right now* too.
>>The ultimate end point will be when the correct balance between what belongs
>>in the generic block layer and what belongs in the transport helpers is
>>achieved. I speculate that, for CDROMS, this will lead to two small request
>>prep modules sharing quite a large helper library (The helper library would do
>>SCSI command translation, and probably the IDE prep module would fix up the
>>transport command for the specific device). However, I don't rule out that it
>>would lead to a single prep module for both IDE and SCSI.
>
>
> With my proposal, anything that speaks SCSI is used via SCSI commands and a
> generic SCSI driver like scg.c could access the SCSI transport aware drives
> of any type. An important (communication baesed) feature of my SCSI glue layer
> would be to make it possible to insert SCSI commands from scg.c without making
> sd.c believe that something strange and unexpected happened to one of it's drives.
Your proposal has one flaw - it thinks that SCSI is the mother of
all block devices and thinks that the kernel should have a SCSI driver
with some kind of direct translation for any other devices. This is
wrong becouse you go:
1. abstract access.
2. SCSI access.
3. back to abstract access.
4. ATA access.
This is at least not fine. The proposal by James takes in to
account that dealing with ATAPI ver. SCSI MMC shouldn't be done
on any kind of "layer" it fits better in the abstraction
of an utility library supposed to help unify code. But not an
additional layer between the generic one and the device transfer.
You have to have an independant transport layer for IDE attached
devices becouse the whole handling of the transfer works significantly
different from SCSI.
The proposal for pushing much of the SCSI-midlayer is simple
due to the realization that the generic BIO layer became more
intelligent in the time between. See recent SCSI TCQ handling changes.
On 2002-07-16T13:28:19,
Joerg Schilling <[email protected]> said:
> Why should the character interface be connected to the block layer?
> This would contradict UNIX rules.
How would it? At some layer, the two are merged anyway (for example, at least
on disk you'll have blocks again). Doing it up high means more unified code
below.
> AFAIK, tagged command queuing is a SCSI specific property, why should this
> be part of a generif block layer?
That is not true. Late IDE also has this, and systems like drbd - which
currently uses a quite clever heuristic to deduce barriers - could also
utilize this input.
Sincerely,
Lars Marowsky-Br?e <[email protected]>
--
Immortality is an adequate definition of high availability for me.
--- Gregory F. Pfister
On Tue, Jul 16, 2002 at 01:28:19PM +0200, Joerg Schilling wrote:
> It would help, if somebody would correct the current SCSI addressng scheme used
> in Linux. Linux currently uses something called BUS/channel/target/lun.
> This does not reflect reality.
>
> What Linux calls a SCSI bus is definitely not a SCSI bus but a SCSI HBA card.
> What Linux calls a channel really is one of possibly more SCSI busses going
> off one of the SCSI HBA cards. It makes sense to just count SCSI busses.
Well, no. It doesn't. Because the numbers will change if you add a card
(even at runtime - hotplugging USB SCSI is something real happening
today. And that'd be a very bad thing.
The way it'll be done is that you'll get the device physical path (see
driverfs) to the device, the device serial number and other identifiers
and then a hotplug/system configuration agent will choose a nice name
for it (completely configurable).
--
Vojtech Pavlik
SuSE Labs
>From [email protected] Tue Jul 16 13:59:27 2002
>> It would help, if somebody would correct the current SCSI addressng scheme used
>> in Linux. Linux currently uses something called BUS/channel/target/lun.
>> This does not reflect reality.
>>
>> What Linux calls a SCSI bus is definitely not a SCSI bus but a SCSI HBA card.
>> What Linux calls a channel really is one of possibly more SCSI busses going
>> off one of the SCSI HBA cards. It makes sense to just count SCSI busses.
>Well, no. It doesn't. Because the numbers will change if you add a card
>(even at runtime - hotplugging USB SCSI is something real happening
>today. And that'd be a very bad thing.
It hey change, then this is a Linux kernel problem. On Solaris they don't
change because Solaris manages /etc/path_to_inst
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From [email protected] Tue Jul 16 13:43:18 2002
>>> User Level
>>>--------------------------------------------------------------
>>=20
>>=20
>>>+----------------+ +---------------------+
>>>| Block Interface| | Character Interface |
>>>+----------------+ +---------------------+
>>> | |
>>> | +--------------------+
>>=20
>>=20
>> Why should the character interface be connected to the block layer?
>> This would contradict UNIX rules.
>Bullshit. UNIX sees even every single block device as both: char and bloc=
>k
>even under the same major number. The reason above is - tape devices.
Don't make conclusions from Linux, have a look at UNIX first:
- All traditional UNIXes including vanilla SVSvr4 use different
major numbers for block and char dev entries.
... I call this a design bug, but take it for given.
- UNOS, a real time UNIX clone from 1982 did have a unique driver
switch table and the same major for block/character devices.
UNOS has no connection to the UNIX genealogy tree.
- Solaris 2.1 introduced together with dynamic and virtual major device
numbers a unique interface for block and character devices.
The fact that Linux did not have character device nodes for disks until shortly
should not confuse you.
Tape block drivers only make sense with something like DECtapes.
DECtapes used as block device have been used to boot emergency recovery systems
on PDP-11 sty computers. Booting took a long long time then.
>>>The driving vision for this is to move into the generic block layer as =
>much of=20
>>>the individual transport stack as makes sense (i.e. if other transports=
> can=20
>>>make use of the functions), so, for example, Tag command queueing is al=
>ready=20
>>>in there and shared between IDE and SCSI.
>>=20
>>=20
>> AFAIK, tagged command queuing is a SCSI specific property, why should t=
>his
>> be part of a generif block layer?
>Becouse it is not SCSI specific. SerialATA will *nearly* require it.
>And there are ATA disks out there which do it *right now* too.
How do you access these properties then? Using SCSI commands?
>Your proposal has one flaw - it thinks that SCSI is the mother of
>all block devices and thinks that the kernel should have a SCSI driver
>with some kind of direct translation for any other devices. This is
>wrong becouse you go:
>1. abstract access.
Which is not a problem
>2. SCSI access.
Not a problem too for all drives that support SCSI commands.
>3. back to abstract access.
??? Looks like you missundertood even the ATAPI layering.
BTW: I don't reqiest that SCSI commands are used for ATA only drives.
>4. ATA access.
See above...
>This is at least not fine. The proposal by James takes in to
>account that dealing with ATAPI ver. SCSI MMC shouldn't be done
>on any kind of "layer" it fits better in the abstraction
>of an utility library supposed to help unify code. But not an
>additional layer between the generic one and the device transfer.
>You have to have an independant transport layer for IDE attached
>devices becouse the whole handling of the transfer works significantly
>different from SCSI.
Please show me how you would like to read a sector of audio data off
a MMC ATAPI drive without using SCSI commands. Vanilla ATA read access
is of vely limited use.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From [email protected] Tue Jul 16 13:45:59 2002
>> Why should the character interface be connected to the block layer?
>> This would contradict UNIX rules.
>How would it? At some layer, the two are merged anyway (for example, at least
>on disk you'll have blocks again). Doing it up high means more unified code
>below.
Just a hint: the block layer is for caching blocks from disk type deveices.
- Block device access is always going directly into the block cache.
So the I/O is always kernel I/O. In addition, it is async I/O - the
block layer fires it up and may wait for it later after sending out other
requests.
- Character device access is synchronous access and may be either kernel
or user space DMA access. In most cases, it is user space DMA access.
How try to ask your question again...
>> AFAIK, tagged command queuing is a SCSI specific property, why should this
>> be part of a generif block layer?
>That is not true. Late IDE also has this, and systems like drbd - which
>currently uses a quite clever heuristic to deduce barriers - could also
>utilize this input.
How is it implemented?
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
On Tue, Jul 16, 2002 at 04:06:55PM +0200, Joerg Schilling wrote:
> Just a hint: the block layer is for caching blocks from disk type deveices.
>
> - Block device access is always going directly into the block cache.
> So the I/O is always kernel I/O. In addition, it is async I/O - the
> block layer fires it up and may wait for it later after sending out other
> requests.
>
> - Character device access is synchronous access and may be either kernel
> or user space DMA access. In most cases, it is user space DMA access.
>
> How try to ask your question again...
The discussion would be much easier if you got the terminology right.
The linux block layer does no caching at all, the caching of the block device
nodes is handles by the page (>= 2.4.10) or buffer (<= 2.4.9) cache.
> >That is not true. Late IDE also has this, and systems like drbd - which
> >currently uses a quite clever heuristic to deduce barriers - could also
> >utilize this input.
>
> How is it implemented?
RTFS: drivers/ide/tcq.c (in 2.5)
And path_to_inst does not always do the RightThing(tm). [1] [2] Two
systems identically configured has the potential of having path_to_inst
look different. Especially if you have previously installed a device or
moved stuff around. And if the expectation is that a group of devices will
come up in a certain sequence (think shared tape devices for instance) and
it changes, it quickly becomes a nightmare. Not a fun proposition by any
means.
--Jauder
[1] eg http://www.myri.com/scs/documentation/mug/installation/solaris.html
[2] http://www.magma.com/support/sun.htm
On Tue, 16 Jul 2002, Joerg Schilling wrote:
> >From [email protected] Tue Jul 16 13:59:27 2002
>
> >> It would help, if somebody would correct the current SCSI addressng scheme used
> >> in Linux. Linux currently uses something called BUS/channel/target/lun.
> >> This does not reflect reality.
> >>
> >> What Linux calls a SCSI bus is definitely not a SCSI bus but a SCSI HBA card.
> >> What Linux calls a channel really is one of possibly more SCSI busses going
> >> off one of the SCSI HBA cards. It makes sense to just count SCSI busses.
>
> >Well, no. It doesn't. Because the numbers will change if you add a card
> >(even at runtime - hotplugging USB SCSI is something real happening
> >today. And that'd be a very bad thing.
>
> It hey change, then this is a Linux kernel problem. On Solaris they don't
> change because Solaris manages /etc/path_to_inst
>
> J?rg
>
> EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
> [email protected] (uni) If you don't have iso-8859-1
> [email protected] (work) chars I am J"org Schilling
> URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>
>
> >Solaris vold? Thanks no, floppy access was so easy in SunOS before the
> >days of the volume manager.
>
> .... and it is even simpler since vold is present. Call volcheck to tell vold
> that the media changed or use a SCSI floppy which supports to tell the kernel
> that a media change did happen.
when it is properly configured which doesn't seem the common case.
More often than not, things like accessing raw floppy images turn
out to be a problem.
> >> - The volume manager should have a documented interface that allows
> >> programs like e.g. cdrecord to gain exclusive access to a CD drive.
>
> >much simpler, cdrom driver needs an ioctl to claim exclusive use of
> >the device and cdrecord than needs to use that ioctl.
>
> This will not work. What should happen when the drive is mounted?
block or return -EBUSY.
Richard
[email protected] said:
> Why should the character interface be connected to the block layer?
> This would contradict UNIX rules.
Well, first and foremost Linux isn't UNIX, especially where block and
character devices are concerned. But secondly, the block layer provides
command queueing. If any device (SCSI, IDE or more exotic) can't accept a
character command (like QUEUE_FULL in SCSI), it goes back to the block layer
queue to await reissue. It's really exactly a block request without the block
position.
This is actually almost the way linux operates now: all the character tap for
a SCSI tape device does is take the read or write and convert it into an
appropriate SCSI command, which now has a block extent attached. Since we
have all the machinery for handling block command queuing in the block layer,
it makes no sense to duplicate it for the character layer.
> Why should the error handlers interface with the block layer? This is
> not true for UNIX and it would not help. However, it would be nice to
> have a way to make the blocklayer find out that e.g. only the last
> sector of a multi sector read ahead instruction did fail.
Error handling is more than local. Some errors, you are correct, can only be
handled at the SCSI layer. However, a large class of drivers (Think
multi-path or software raid) want the ability to direct how SCSI handles
errors themselves. It is unacceptable to have SCSI all on its own retry a
medium error command x times, taking minutes before the upper layers become
aware anything went wrong.
The solution is to have a stacking error handler where the error handler for
upper devices would be notified of a problem and asked for direction as soon
as it occurs.
> With my proposal, anything that speaks SCSI is used via SCSI commands
> and a generic SCSI driver like scg.c could access the SCSI transport
> aware drives of any type. An important (communication baesed) feature
> of my SCSI glue layer would be to make it possible to insert SCSI
> commands from scg.c without making sd.c believe that something
> strange and unexpected happened to one of it's drives.
But the new scheme allows that. The block queues accept translated requests
(that's really what sg does).
> It would help, if somebody would correct the current SCSI addressng
> scheme used in Linux. Linux currently uses something called BUS/
> channel/target/lun. This does not reflect reality.
No UNIX scheme reflects reality, so I suppose we're all equal
> Why do you believe that you need to have something that is not a
> bumber?
Look at a solaris fibre driver for instance. On the fabric, most of them
think of targets in terms of WWN/PORT (because that's what the fibre LIP
uses). They then have an internal database to translate what they use
(WWN/PORT, soft loop ID, etc.) into a target number for the user to see.
Next, because the fibre topology is mutable, they have to have a way of
mapping the WWN/PORT to the device across reboots, hence persistent binding.
Ultimately you get a huge chunk of code whose sole job is to preserve the
fiction that targets are numbers.
Not pretending the target is a number avoids all the above glue and complexity.
> Let me add my modified artwork:
But you're still too SCSI transport specific. The ongoing goal is to make the
physical transport protocol an adjunct to the Linux internal transport (the
struct request) so that we can treat all block/character devices on an equal
footing.
Your diagram picks one preferred transport and forces everything else to
conform. It's rather like quantum field theory: I want to treat the
underlying physics in a gauge invariant fashion (i.e. transport independent),
exposing only the essential features of the problem. You want to pick a
preferred gauge (the SCSI transport). There are always cases where the
preferred gauge makes everything much simpler, but it's simply not worth it
for the many more cases where it makes the problem much more complex.
James
On 2002-07-16T16:06:55,
Joerg Schilling <[email protected]> said:
> Just a hint: the block layer is for caching blocks from disk type deveices.
>
> - Block device access is always going directly into the block cache. So
> the I/O is always kernel I/O. In addition, it is async I/O - the block layer
> fires it up and may wait for it later after sending out other requests.
>
> - Character device access is synchronous access and may be either kernel
> or user space DMA access. In most cases, it is user space DMA access.
>
> How try to ask your question again...
Right, and you can build one on top of the other in this fashion. I don't see
the problem.
> >That is not true. Late IDE also has this, and systems like drbd - which
> >currently uses a quite clever heuristic to deduce barriers - could also
> >utilize this input.
> How is it implemented?
drbd does an analysis of the write-patterns, look at
http://www.complang.tuwien.ac.at/reisner/drbd/publications/index.html, Philipp
has written a diploma thesis on it.
Sincerely,
Lars Marowsky-Br?e <[email protected]>
--
Immortality is an adequate definition of high availability for me.
--- Gregory F. Pfister
path_to_inst is responsible for correlating the logical device path, i.e.
/sbus@2,0/fcaw@3,0/st@0,0 -> <instance number>. That instance number is used
to preserve device ordering. The next time the OS runs devfsadm,
path_to_inst is consulted to preserve history.
This is nessecary because by default, Solaris wants to index devices based
on the index of the slot, bus, and board that the card is plugged into. If
someone were to install board 4 first and install a card on sbus 2, slot 2
and later add a card on sbus 0, slot 0 path_to_inst insures that devices
stay in order.
I have managed these boxes for years and your comment below "And
path_to_inst does not always do the RightThing(tm)" needs more
clarification, I would say that the Solaris way of building devices is solid
as a rock.
How about this for instance ... When I take my laptop to work and dock it,
eth0 becomes the nic in the docking station as apposed to the pcmcia card
that was previously configured and expected to act as eth0. While im sure
there is some way to force the alias to be what it needs to be, if device
numbering was handled by a scheme like Solaris, this wouldn't be a problem.
--Buddy
-----Original Message-----
From: [email protected]
[mailto:[email protected]]On Behalf Of Jauder Ho
Sent: Tuesday, July 16, 2002 7:29 AM
To: Joerg Schilling
Cc: [email protected]; [email protected];
[email protected]
Subject: Re: IDE/ATAPI in 2.5
And path_to_inst does not always do the RightThing(tm). [1] [2] Two
systems identically configured has the potential of having path_to_inst
look different. Especially if you have previously installed a device or
moved stuff around. And if the expectation is that a group of devices will
come up in a certain sequence (think shared tape devices for instance) and
it changes, it quickly becomes a nightmare. Not a fun proposition by any
means.
--Jauder
[1] eg http://www.myri.com/scs/documentation/mug/installation/solaris.html
[2] http://www.magma.com/support/sun.htm
On Tue, 16 Jul 2002, Joerg Schilling wrote:
> >From [email protected] Tue Jul 16 13:59:27 2002
>
> >> It would help, if somebody would correct the current SCSI addressng
scheme used
> >> in Linux. Linux currently uses something called BUS/channel/target/lun.
> >> This does not reflect reality.
> >>
> >> What Linux calls a SCSI bus is definitely not a SCSI bus but a SCSI HBA
card.
> >> What Linux calls a channel really is one of possibly more SCSI busses
going
> >> off one of the SCSI HBA cards. It makes sense to just count SCSI
busses.
>
> >Well, no. It doesn't. Because the numbers will change if you add a card
> >(even at runtime - hotplugging USB SCSI is something real happening
> >today. And that'd be a very bad thing.
>
> It hey change, then this is a Linux kernel problem. On Solaris they don't
> change because Solaris manages /etc/path_to_inst
>
> J?rg
>
> EMail:[email protected] (home) J?rg Schilling D-13353
Berlin
> [email protected] (uni) If you don't have iso-8859-1
> [email protected] (work) chars I am J"org Schilling
> URL: http://www.fokus.gmd.de/usr/schilling
ftp://ftp.fokus.gmd.de/pub/unix
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>
>From: Richard Zidlicky <[email protected]>
>> >Solaris vold? Thanks no, floppy access was so easy in SunOS before the
>> >days of the volume manager.
>>
>> .... and it is even simpler since vold is present. Call volcheck to tell vold
>> that the media changed or use a SCSI floppy which supports to tell the kernel
>> that a media change did happen.
>when it is properly configured which doesn't seem the common case.
>More often than not, things like accessing raw floppy images turn
>out to be a problem.
Being properly configured _is_ the common case if you don't change things
manually. A standard Solaris system install from scratch will always result in
a usable floppy drive that is handled as expected by vold.
Try to keep your fingers away from the vold configuration files after you did a
clean install from scratch.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From: Jauder Ho <[email protected]>
>And path_to_inst does not always do the RightThing(tm). [1] [2] Two
>systems identically configured has the potential of having path_to_inst
>look different. Especially if you have previously installed a device or
>moved stuff around. And if the expectation is that a group of devices will
>come up in a certain sequence (think shared tape devices for instance) and
>it changes, it quickly becomes a nightmare. Not a fun proposition by any
>means.
The Solaris /etc/path_to_inst method works 99.9% as expected. And important,
it is very stable in special for novice users. If you really have the problems
as described, you could alwas edit /etc/path_to_inst manually and reboot.
Please note: If you install two systems with te same hardware configuration the
same way _and_ in the same _order_, the will have an identical
/etc/path_to_inst.
If I have to decide between a mostly rock solid system and a system that always
behaves in a way that is not predictable, I would take the first one.
>[1] eg http://www.myri.com/scs/documentation/mug/installation/solaris.html
The author of this driver lacks the needed experience in writing DLPI drivers.
Although this lack of knowledge seems to be not uncommon (the FORE ATM driver
is broken too), it is possible to do it right. The ATM DLPI driver I did write
in 1995 did work correctly after I did educate me about the correct interface.
You simply need to be able to deal with the fact that instance #0 os a driver
is not present while one or more instances with instance # > 0 _are_ present.
>[2] http://www.magma.com/support/sun.htm
Looks like a similar problem as above.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
>From [email protected] Tue Jul 16 16:33:32 2002
>[email protected] said:
>> Why should the character interface be connected to the block layer?
>> This would contradict UNIX rules.
>Well, first and foremost Linux isn't UNIX, especially where block and
>character devices are concerned. But secondly, the block layer provides
>command queueing. If any device (SCSI, IDE or more exotic) can't accept a
>character command (like QUEUE_FULL in SCSI), it goes back to the block layer
>queue to await reissue. It's really exactly a block request without the block
>position.
You describe here how the block layer in UNIX works....
>This is actually almost the way linux operates now: all the character tap for
>a SCSI tape device does is take the read or write and convert it into an
>appropriate SCSI command, which now has a block extent attached. Since we
>have all the machinery for handling block command queuing in the block layer,
>it makes no sense to duplicate it for the character layer.
If Linux really handles it this way, then I don't see any sense in it.
If you like to do things that are different from the UNIX default for block
or character drivers for disk devices, you should try to use mmap() and
madvise().
>Error handling is more than local. Some errors, you are correct, can only be
>handled at the SCSI layer. However, a large class of drivers (Think
>multi-path or software raid) want the ability to direct how SCSI handles
>errors themselves. It is unacceptable to have SCSI all on its own retry a
>medium error command x times, taking minutes before the upper layers become
>aware anything went wrong.
It looks like you have the wrong ideas about software raid. If you have
software raid, you will stack a SW raid driver just on top of the disk drivers
that handle the access to the real drives. The real drives first do own error
handling and if they cannot correct errors, the error is handled inside the
raid layer. As the raid layer itself will at it's top level act as if it was a
disk driver, there is no relation to the block layer.
>The solution is to have a stacking error handler where the error handler for
>upper devices would be notified of a problem and asked for direction as soon
>as it occurs.
See above, this makes no sense.
>But the new scheme allows that. The block queues accept translated requests
>(that's really what sg does).
A SCSI request is _not_ a translated request. It is the real request itself.
You usually even cannot translate a SCSI request into something else.
>> It would help, if somebody would correct the current SCSI addressng
>> Why do you believe that you need to have something that is not a
>> bumber?
>Look at a solaris fibre driver for instance. On the fabric, most of them
>think of targets in terms of WWN/PORT (because that's what the fibre LIP
>uses). They then have an internal database to translate what they use
>(WWN/PORT, soft loop ID, etc.) into a target number for the user to see.
>Next, because the fibre topology is mutable, they have to have a way of
>mapping the WWN/PORT to the device across reboots, hence persistent binding.
>Ultimately you get a huge chunk of code whose sole job is to preserve the
>fiction that targets are numbers.
You also need to authenticate yourself before you may get access to the network
media. Once you did this, you simply may assign a Bus number to the cabinet.
>> Let me add my modified artwork:
>But you're still too SCSI transport specific. The ongoing goal is to make the
>physical transport protocol an adjunct to the Linux internal transport (the
>struct request) so that we can treat all block/character devices on an equal
>footing.
You seem to forget that all main control is done via SCSI commands. You are too
anti SCSI wthout a real reason.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
On Wed, 17 Jul 2002, Joerg Schilling wrote:
> >From [email protected] Tue Jul 16 16:33:32 2002
> >Error handling is more than local. Some errors, you are correct, can only be
> >handled at the SCSI layer. However, a large class of drivers (Think
> >multi-path or software raid) want the ability to direct how SCSI handles
> >errors themselves. It is unacceptable to have SCSI all on its own retry a
> >medium error command x times, taking minutes before the upper layers become
> >aware anything went wrong.
>
> It looks like you have the wrong ideas about software raid. If you have
> software raid, you will stack a SW raid driver just on top of the disk
> drivers that handle the access to the real drives. The real drives first
> do own error handling and if they cannot correct errors, the error is
> handled inside the raid layer.
Did you even read what James wrote ?
When one of the disks in a RAID array develops a bad block
it shouldn't stall the box for minutes when the error can
be handled by simply doing the IO from other disks instead.
> >But the new scheme allows that. The block queues accept translated requests
> >(that's really what sg does).
>
> A SCSI request is _not_ a translated request. It is the real request itself.
> You usually even cannot translate a SCSI request into something else.
This suggests we shouldn't put all kinds of non-scsi devices
under the SCSI stack and should have a more generic block and
character device layer instead ...
> >But you're still too SCSI transport specific. The ongoing goal is to make the
> >physical transport protocol an adjunct to the Linux internal transport (the
> >struct request) so that we can treat all block/character devices on an equal
> >footing.
>
> You seem to forget that all main control is done via SCSI commands. You
> are too anti SCSI wthout a real reason.
... and here you contradict what you said above.
Will you do the honours or are you waiting for somebody else
to godwinate the thread ?
regards,
Rik
--
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/ http://distro.conectiva.com/
>From [email protected] Wed Jul 17 15:37:18 2002
>> >Error handling is more than local. Some errors, you are correct, can only be
>> >handled at the SCSI layer. However, a large class of drivers (Think
>> >multi-path or software raid) want the ability to direct how SCSI handles
>> >errors themselves. It is unacceptable to have SCSI all on its own retry a
>> >medium error command x times, taking minutes before the upper layers become
>> >aware anything went wrong.
>>
>> It looks like you have the wrong ideas about software raid. If you have
>> software raid, you will stack a SW raid driver just on top of the disk
>> drivers that handle the access to the real drives. The real drives first
>> do own error handling and if they cannot correct errors, the error is
>> handled inside the raid layer.
>Did you even read what James wrote ?
>When one of the disks in a RAID array develops a bad block
>it shouldn't stall the box for minutes when the error can
>be handled by simply doing the IO from other disks instead.
Is there any problem with using a ioctl() from upper layer kernel to the low
level drivers (living under the SW raid) to reduce the number of retries to a
reasonable value in this case?
The main design goal for UNIX as to keep it simple. There is no need for a
complex cross layer error control.
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
At 13:32 17/07/02, Joerg Schilling wrote:
> >Error handling is more than local. Some errors, you are correct, can
> only be
> >handled at the SCSI layer. However, a large class of drivers (Think
> >multi-path or software raid) want the ability to direct how SCSI handles
> >errors themselves. It is unacceptable to have SCSI all on its own retry a
> >medium error command x times, taking minutes before the upper layers become
> >aware anything went wrong.
>
>It looks like you have the wrong ideas about software raid. If you have
>software raid, you will stack a SW raid driver just on top of the disk
>drivers
>that handle the access to the real drives. The real drives first do own error
>handling and if they cannot correct errors, the error is handled inside the
>raid layer. As the raid layer itself will at it's top level act as if it
>was a
>disk driver, there is no relation to the block layer.
Of course it has relation to the block layer! How else are the errors going
to be passed from the disk drivers to the raid layer and then to the fs? In
an ideal world, this is what should happen so that the fs can deal with the
problem, for example on a write, the fs can mark the block as bad in the
file systems list of bad blocks, and it can change the allocation of the
block to a different location and write the data again (assuming the lower
layer cannot handle this, e.g. hardware block remapping at disk level).
Without a full error passing from the lowest to the highest level this
cannot be done and this is something that Linux needs IMO.
> >The solution is to have a stacking error handler where the error handler
> for
> >upper devices would be notified of a problem and asked for direction as
> soon
> >as it occurs.
>
>See above, this makes no sense.
It makes perfect sense actually. Also, see above.
> >But the new scheme allows that. The block queues accept translated
> requests
> >(that's really what sg does).
>
>A SCSI request is _not_ a translated request. It is the real request itself.
>You usually even cannot translate a SCSI request into something else.
That is what you are not understanding. Ideally the block layer will be
creating the scsi, request not you. And you want to do it the other way
round. You want to create the SCSI request and pass it to the kernel for
giving it to the hardware and if the hardware is not SCSI the request will
need translation. And what other people are trying to tell you is that this
is too specific for a generic block interface and that Linux wants to have
a generic block interface that will convert requests into SCSI ones or
"plug your favourite request type here" requests depending on the attached
device driver but the interface itself is abstracted above SCSI. Ok?
> >But you're still too SCSI transport specific. The ongoing goal is to
> make the
> >physical transport protocol an adjunct to the Linux internal transport (the
> >struct request) so that we can treat all block/character devices on an
> equal
> >footing.
>
>You seem to forget that all main control is done via SCSI commands.
No it is not at all. SCSI is _one_of_many_ ways to talk to hardware. And as
many people have reiterated several times SCSI is _not_sufficient_ to talk
to all hardware. So there is no way in hell SCSI is going to be the generic
block interface in Linux. We now have the struct request, which is way
better than a SCSI request and is certainly going to evolve over time to be
even more powerful than it already is.
Your argument that all other OS use SCSI as the block interface is no
argument at all. Linux doesn't do what everyone else does. It does the
technically better thing, which of course can be what other OS do, but in
this case it is not. In other words, if all people jump off a cliff, are
you going to follow just because all of the others do it? I certainly
won't... (-;
Anyway, this thread has been going on for far too long. If you believe so
strongly that SCSI is generic enough for a block interface then why don't
you code that, and then submit your code to LKML? I highly suspect you will
not be able to do something that fulfills all the different device type
requirements and maintain a clean, efficient layer at the same time. Feel
free to prove me wrong by showing me the code which does it.
Best regards,
Anton
--
"I've not lost my mind. It's backed up on tape somewhere." - Unknown
--
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS Maintainer / IRC: #ntfs on irc.openprojects.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
On Wed, 17 Jul 2002, Joerg Schilling wrote:
> Is there any problem with using a ioctl() from upper layer kernel to the low
> level drivers (living under the SW raid) to reduce the number of retries to a
> reasonable value in this case?
>
> The main design goal for UNIX as to keep it simple. There is no need for a
> complex cross layer error control.
... and ioctl(2) is a gross violation of that design goal. Ask the authors
of UNIX how they feel about that kludge, let alone propagation of said kludge
beyond the TTY layer where it had originated (or about the entire v7 TTY layer,
for that matter - v8 and later had thrown that crap away).
If you care about design philosophy of UNIX - at the very least take a hard
look at APIs you are using. Sheesh...
>From [email protected] Wed Jul 17 16:00:59 2002
>> Is there any problem with using a ioctl() from upper layer kernel to the low
>> level drivers (living under the SW raid) to reduce the number of retries to a
>> reasonable value in this case?
>>
>> The main design goal for UNIX as to keep it simple. There is no need for a
>> complex cross layer error control.
>... and ioctl(2) is a gross violation of that design goal. Ask the authors
>of UNIX how they feel about that kludge, let alone propagation of said kludge
>beyond the TTY layer where it had originated (or about the entire v7 TTY layer,
>for that matter - v8 and later had thrown that crap away).
ioctl()s do introduce a common abstract interface layer.
Tell me how else you would like to set similar things in e.g. different disk
type drivers.
In Plan 9, they did replace them with a text interface to /dev/{driver}.ctl
J?rg
EMail:[email protected] (home) J?rg Schilling D-13353 Berlin
[email protected] (uni) If you don't have iso-8859-1
[email protected] (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
On Sun, 14 Jul 2002, Joerg Schilling wrote:
> >There are lots that fudge around and pretend scsi is the block layer
> >when it is not. That sort of misses the point and slows down high end
> >raid cards.
>
> It seems that you miss to understand the needed underlying driver structures.
> SCSI is not a block layer, it is a generic transport.
Didn't he just say that? Or does " ... pretend scsi is the block layer
*WHEN IT IS NOT*" mean something else than the obvious?
--
Kelsey Hudson [email protected]
Software Engineer/UNIX Systems Administrator
Compendium Technologies, Inc (619) 725-0771
---------------------------------------------------------------------------