Hi,
The following announcements and patches introduce
Serial Attached SCSI (SAS) support for the Linux kernel.
Everything is supported.
The infrastructure is broken into
* SAS LLDD,
* SAS Layer.
The SAS LLDD does phy/OOB management, and generates SAS events
to the SAS Layer. Those events are *the only way* a SAS LLDD
communicates with the SAS Layer. If you can generate 2 types
of event, then you can use this infrastructure. The first two
are, loosely, "link was severed", "bytes were dmaed". The third
kind is "received a primitive", used for domain revalidation.
A SAS LLDD should implement the Execute Command SCSI RPC and
at least one SCSI TMF (Task Management Function), in order for
the SAS Layer to communicate with the SAS LLDD.
The SAS Layer is concerned with
* SAS Phy/Port/HA event management (LLDD generates,
SAS Layer processes),
* SAS Port management (creation/destruction),
* SAS Domain discovery and revalidation,
* SAS Domain device management,
* SCSI Host registration/deregistration,
* Device registration with SCSI Core (SAS) or libata
(SATA/PI), and
* Expander management and exporting expander control
to user space.
The SAS Layer uses the Execute Command SCSI RPC, and the TMFs
implemented by the SAS LLDD in order to manage the domain and
the domain devices.
For details please see drivers/scsi/sas-class/README.
The SAS Layer represents the SAS domain in sysfs. For each
object represented, its parent is the physical entity it attaches
to in the physical world. So in effect, kobject_get, gets
the whole chain up on which that object depends on.
In effect, the sysfs representation of the SAS domain(s)
is what you'd see in the physical world.
Hot plugging and hot unplugging of devices, domains and subdomains
is supported. Repeated hot plugging and hot unplugging is
also supported, naturally.
SAS introduces a new physical entity, an expander.
Expanders are _not_ SAS devices, and thus are _not_ SCSI devices.
Expanders are part of the Service Delivery Subsystem, in this case
SAS.
Expanders are controlled using the Serial Management Protocol (SMP).
Complete control is given to user space of all expanders found
in the domain, using an "smp_portal". More of this in the second
and third email in this series.
A user space program, "expander_conf.c" is also presented to show
how one controls expanders in the domain. It is located here:
drivers/scsi/sas-class/expanders_conf.c
The second email in this series shows an example of SAS domains
and their representation in sysfs.
The third email in this series shows an example of using the
"expander_conf.c" program to query all expanders in the domain,
showing their attributes, their phys, and their routing tables.
If you have the hardware, please give it a try. If you have
expander(s) it would be even more interesting.
Patches of the SAS Layer and of the AIC94XX SAS LLDD follow.
Luben
P.S. You can also download the patches from
http://www.geocities.com/ltuikov/
Thanks for finally posting your code.
At the core it's some really nice code dealing with host-based SAS
implementations. What's not nice is that it's not intgerating with the
SAS transport class I posted, it's duplicating things like LUN disocvery
from the SCSI core code, and adding it's own sysfs representation that's
very different from the way the SCSI core and transport classes do it.
Are you willing to work with us to intgerate it with the infrastructure
we have?
On 09/11/05 05:20, Christoph Hellwig wrote:
> Thanks for finally posting your code.
>
> At the core it's some really nice code dealing with host-based SAS
> implementations.
Thank you Christoph. Much appreciated.
> What's not nice is that it's not intgerating with the
> SAS transport class I posted,
I wish there was something I could do. HP and LSI
were aware of my efforts since the beginning of the year.
As well, you had a copy of my code July 14 this year,
long before starting your work on your SAS class for LSI and
HP (so its acceptance is guaranteed), after OLS.
We did meet at OLS and we did have the SAS BOF. I'm not sure
why you didn't want to work together?
> it's duplicating things like LUN disocvery
This is a much more involved subject than meets the eye.
> from the SCSI core code, and adding it's own sysfs representation that's
> very different from the way the SCSI core and transport classes do it.
Yes, it is time to evolve.
I've pointed out many times the shortcomings of expanding the
JB's "transport _attribute_ class" into a "transport layer" in
recent threads.
I'm sorry but over everything else, we need a common base,
(what you call "techno-gibberish") in order to see eye to eye.
> Are you willing to work with us to intgerate it with the infrastructure
> we have?
I'm sure you've already taken a closer look at the SAS code
I posted. Study it, read the spec, read the code again.
Let me know if I can help with anything.
Overall, MPT is very different in design than a disclosed
transport. Talk to HP, LSI and Dell and see what they think.
Luben
On Mon, 2005-09-12 at 17:35 -0400, Luben Tuikov wrote:
> On 09/11/05 05:20, Christoph Hellwig wrote:
> > Thanks for finally posting your code.
> >
> > At the core it's some really nice code dealing with host-based SAS
> > implementations.
>
> Thank you Christoph. Much appreciated.
>
> > What's not nice is that it's not intgerating with the
> > SAS transport class I posted,
>
> I wish there was something I could do. HP and LSI
> were aware of my efforts since the beginning of the year.
I know I am going to regret getting pulled into this ;-(.
This effort started on April. Eric Moore, Mike Miller and I started
work on a SAS transport class and then later pulled Luben it at the
suggestion of Douglas Gilbert (if I remember correctly). We later
mutually agreed that Luben would take over the transport class work as
he seemed to have much more experience with this sort of thing. The
original idea was to implement a SAS transport class that would allow
the LSI and Adaptec driver to get into kernel.org (or others at the
time) and to find a way to get SDI/CSMI API's into the kernel without
the use of IOCTL's. Luben then went off on his own and came up with his
effectively Adaptec only solution.
>
> As well, you had a copy of my code July 14 this year,
> long before starting your work on your SAS class for LSI and
> HP (so its acceptance is guaranteed), after OLS.
HP never suggested that Christoph do a SAS transport layer. We were
happy to provide some equipment when we found out that he was working on
it. Please don't speak for HP. I am sure that LSI would prefer you
don't speak for them either.
>
> We did meet at OLS and we did have the SAS BOF. I'm not sure
> why you didn't want to work together?
If my memory serves correctly, there were 10-12 people at that BOF,
representing the SCSI kernel maintainers and all of the vendors
currently providing SAS hardware. Virtually everyone disagreed with
your implementation (which you indeed emailed shortly before the
conference) that would only work with one vendor's card. The suggestion
was made that you convert your code to various library layers so that it
would work with all vendors. A suggestion which it seems that you
continue to reject.
>
> > it's duplicating things like LUN disocvery
>
> This is a much more involved subject than meets the eye.
>
> > from the SCSI core code, and adding it's own sysfs representation that's
> > very different from the way the SCSI core and transport classes do it.
>
> Yes, it is time to evolve.
>
> I've pointed out many times the shortcomings of expanding the
> JB's "transport _attribute_ class" into a "transport layer" in
> recent threads.
>
> I'm sorry but over everything else, we need a common base,
> (what you call "techno-gibberish") in order to see eye to eye.
>
> > Are you willing to work with us to intgerate it with the infrastructure
> > we have?
>
> I'm sure you've already taken a closer look at the SAS code
> I posted. Study it, read the spec, read the code again.
>
> Let me know if I can help with anything.
>
> Overall, MPT is very different in design than a disclosed
> transport. Talk to HP, LSI and Dell and see what they think.
From HP's point of view (or mine at least). We would prefer something
that works with every vendors card. Not Adaptec's (or is it Luben's)
vision of the perfect card.
Andrew
--
Andrew Patterson
Hewlett-Packard
On Mon, Sep 12, 2005 at 05:35:04PM -0400, Luben Tuikov wrote:
> > What's not nice is that it's not intgerating with the
> > SAS transport class I posted,
>
> I wish there was something I could do. HP and LSI
> were aware of my efforts since the beginning of the year.
As was I. And the reason I wrote this upper layer is that you
clearly stated multiple times (at the SAS BOF and in mail) that
you're not interested in this upper layer.
> As well, you had a copy of my code July 14 this year,
That code didn't have anything that overlaps with the code I wrote.
> long before starting your work on your SAS class for LSI and
> HP (so its acceptance is guaranteed), after OLS.
Just in case it was clear: I'm paid for this transport class by Dell.
I don't have any contractural relationship with LSI or HP, although these
companies (like most sucessfull hardware vendors) know that giving hardware
to linux people active in the area they care about helps to get thos people
actually fixing things about instead of just bitching around..
> We did meet at OLS and we did have the SAS BOF. I'm not sure
> why you didn't want to work together?
I abosultely want to. To quote from my first minimal transport class
announcement mail:
"I hope this will integrate nicely with the top-down work Luben has done
once he finally releases it publically, but for now I think we should have
something so SAS drivers can go in the tree."
> > from the SCSI core code, and adding it's own sysfs representation that's
> > very different from the way the SCSI core and transport classes do it.
>
> Yes, it is time to evolve.
>
> I've pointed out many times the shortcomings of expanding the
> JB's "transport _attribute_ class" into a "transport layer" in
> recent threads.
We need both a transport class in the original sense aswell as a library
for host-based SAS HBAs, and they need to play together nicely - whatever
term you give to them.
> Overall, MPT is very different in design than a disclosed
> transport.
I know. And we still want to cover it with a common base for what we
can have common.
On 09/13/05 06:14, Christoph Hellwig wrote:
> On Mon, Sep 12, 2005 at 05:35:04PM -0400, Luben Tuikov wrote:
>
>>>What's not nice is that it's not intgerating with the
>>>SAS transport class I posted,
>>
>>I wish there was something I could do. HP and LSI
>>were aware of my efforts since the beginning of the year.
>
>
> As was I. And the reason I wrote this upper layer is that you
> clearly stated multiple times (at the SAS BOF and in mail) that
> you're not interested in this upper layer.
Well, I never really said "Not interested". There was just
nothing I could do. By that time I know from our
original LSI/HP/SAS list that those folks have moved away
from what I was doing, plus I had my own work to do.
As to your work, or integration or whatever you decide
to do, I'm willing to help in anyway I can.
>>As well, you had a copy of my code July 14 this year,
>
> That code didn't have anything that overlaps with the code I wrote.
Well, both source code are public, people can read the code themselves
and make that decision for themselves. There is no point us in discussing
this.
> Just in case it was clear: I'm paid for this transport class by Dell.
> I don't have any contractural relationship with LSI or HP, although these
> companies (like most sucessfull hardware vendors) know that giving hardware
> to linux people active in the area they care about helps to get thos people
> actually fixing things about instead of just bitching around..
Christoph, there is smart engineers in companies too, not only in the Linux
community.
>>We did meet at OLS and we did have the SAS BOF. I'm not sure
>>why you didn't want to work together?
>
>
> I abosultely want to. To quote from my first minimal transport class
> announcement mail:
>
> "I hope this will integrate nicely with the top-down work Luben has done
> once he finally releases it publically, but for now I think we should have
> something so SAS drivers can go in the tree."
I see.
> We need both a transport class in the original sense aswell as a library
> for host-based SAS HBAs, and they need to play together nicely - whatever
> term you give to them.
I don't mind using "transport attribute class" for JB's class,
and "SAS Transport Layer" for the recent code submission.
That is, by design, JB tried to unify _attributes_ across _all_
transports. Not an easy feat by any means, but we shall leave
that at that.
While, by design (and SAM), the SAS Transport Layer, literally
sits between the interconnect and SCSI Core (SAM to be).
This separation is clear and distinct.
>>Overall, MPT is very different in design than a disclosed
>>transport.
>
> I know. And we still want to cover it with a common base for what we
> can have common.
Yes, this is very noble to have and do.
Figuring out _what_ we can have common, in such different and
distinct engineering architectures, Open Transport vs. MPT,
is not an easy feat by any means.
I don't have an indepth knowlege in MPT as you do, clearly since
I don't have specs, etc, but if you have questions or concerns
please don't hesitate to email, I'd gladly help with anything I can.
Luben