James Bottomley wrote:
> Actually, I think we can make one (which is really required ... it's a
> lot of pain to move device nodes, just look at libata). It should be
> child's play to come up with a udev rule that simply does extra symbolic
> links from /dev/cciss<n>c<n>p<n> to whatever the sd device is. That
> should hide a lot of the problem.
>
> The other issue is plugging the management ioctl in, but that can be
> done via scsi_host->ioctl.
>
> James
Another thing which would help would be to expose device type 0x0C (raid
controller) and let sg bind to it.
CCISS_REPORT_LOGICAL and CCISS_REPORT_PHYSICAL will not return this device, but
you can artificially jam it into the list of devices, and give it an 8-byte LUN
address of all zeroes. It will respond to inquiries, etc, so the regular scsi
device scan will find it if you map it to some bus/target/lun.
Then, it can be accessed via ioctls (and even SG_IO, though SG_IO seems to be
broken on some of the distros we support, I've noticed, so that doesn't help as
much as one might think.)
I think there might be an easier way (or at least it is more obvious to me) to
convert cciss to be a SCSI driver, and that is to reuse some of the existing
tape code that's in cciss_scsi.c. (Probably more obvious to me because I wrote
that code.)
It is a little hard to see what Tomo's patch does, as it's all in one step.
In the times I've converted cciss to be a SCSI driver in the past
the steps were more or less as follows (from memory):
1) copy the driver files into drivers/scsi,
2) rename everything, change the Makefile/Kconfig stuff to add in the new code.
3) take out the stuff that makes it a block driver
4) add a CCISS_REPORT_LOGICAL_LUNS call into the scsi tape device discovery
code, expose those devices, add the controller itself in to the list of devices
artificially (8-byte lun address == all zeroes).
At this point the driver will work, but performance will completely suck.
5) switch the tape code to use the block side command allocator, (cmd_alloc()
in cciss.c) and ditch the rinky-dink tape code command allocator
(scsi_cmd_alloc() in cciss_scsi.c) and jack up the number of commands
and queue depth the SCSI side of the driver will support to something
reasonable.
At this point performance will stop completely sucking.
6) rip out a bunch of the block side code.
7) (I've never actually done this step) move the necessary ioctls to the scsi
side of the driver.
The tape code has stuff for hot-plugging of things, sort of (and I sent up some
patches some time ago to make that better awhile ago, but they seemed to get
ignored for whatever reason.) This would need to change to be a bit more
careful (look at page 0x83 serial numbers to find what devices were
new/changed/gone rather than just looking at device type, which is what it's
doing now. BTW, I have patches in the works for the block driver to improve
the behavior of rebuild_lun_table, which currently sucks in the brute force way
that it does things. This patch also removes the duplicated discovery
cdde from cciss_init_one, and has cciss_init_one just call rebuild_lun_table(),
but I digress.)
In any case, I don't think what I described above was what was done here... It
looks like the tape code was ditched, and new code that does almost the same
thing was written. Maybe there was a good reason for that, I don't know.
-- steve
CC'ed linux-scsi, why did you drop it?
On Tue, 22 Jul 2008 12:57:29 -0500
[email protected] wrote:
> James Bottomley wrote:
> > Actually, I think we can make one (which is really required ... it's a
> > lot of pain to move device nodes, just look at libata). It should be
> > child's play to come up with a udev rule that simply does extra symbolic
> > links from /dev/cciss<n>c<n>p<n> to whatever the sd device is. That
> > should hide a lot of the problem.
> >
> > The other issue is plugging the management ioctl in, but that can be
> > done via scsi_host->ioctl.
> >
> > James
>
> Another thing which would help would be to expose device type 0x0C (raid
> controller) and let sg bind to it.
Yeah, it's possible (and might be useful). Actually, my first version
exposed a RAID device. I'm not sure yet what's the best way to expose
CCISS logical and physical units (including a RAID device) to SCSI
logical unit numbers.
> It is a little hard to see what Tomo's patch does, as it's all in one step.
>
> In the times I've converted cciss to be a SCSI driver in the past
> the steps were more or less as follows (from memory):
>
> 1) copy the driver files into drivers/scsi,
> 2) rename everything, change the Makefile/Kconfig stuff to add in the new code.
> 3) take out the stuff that makes it a block driver
Sounds reasonable, I did the similar.
> 4) add a CCISS_REPORT_LOGICAL_LUNS call into the scsi tape device discovery
> code, expose those devices, add the controller itself in to the list of devices
> artificially (8-byte lun address == all zeroes).
>
> At this point the driver will work, but performance will completely suck.
>
> 5) switch the tape code to use the block side command allocator, (cmd_alloc()
> in cciss.c) and ditch the rinky-dink tape code command allocator
> (scsi_cmd_alloc() in cciss_scsi.c) and jack up the number of commands
> and queue depth the SCSI side of the driver will support to something
> reasonable.
Yeah, right. I've not cleaned up this part yet. We need to rework on
command allocation and queue depth management. I think that the scsi
subsystem enables us to simplify (and improve) the queue depth
management code.
> At this point performance will stop completely sucking.
>
> 6) rip out a bunch of the block side code.
> 7) (I've never actually done this step) move the necessary ioctls to the scsi
> side of the driver.
>
> The tape code has stuff for hot-plugging of things, sort of (and I sent up some
> patches some time ago to make that better awhile ago, but they seemed to get
> ignored for whatever reason.) This would need to change to be a bit more
> careful (look at page 0x83 serial numbers to find what devices were
> new/changed/gone rather than just looking at device type, which is what it's
> doing now. BTW, I have patches in the works for the block driver to improve
> the behavior of rebuild_lun_table, which currently sucks in the brute force way
> that it does things. This patch also removes the duplicated discovery
> cdde from cciss_init_one, and has cciss_init_one just call rebuild_lun_table(),
> but I digress.)
Yeah, I think that the device scanning code is an area that we need to
rework on. We need to make it work nicely with the scsi infrastructure.
> In any case, I don't think what I described above was what was done here... It
> looks like the tape code was ditched, and new code that does almost the same
> thing was written. Maybe there was a good reason for that, I don't know.
I didn't integrate the tape code because it was just an RFC. I just
read the cciss code and converted it to a scsi driver. Of course, we
need to integrate the tape support to a scsi cciss driver.
On Thu, Jul 24, 2008 at 10:32:45AM +0900, FUJITA Tomonori wrote:
> CC'ed linux-scsi, why did you drop it?
You mean my SCSI cciss driver? For a few reasons:
At the time, it was just kind of a learning exercise, and I
didn't see it as supplanting the existing driver. I didn't know
enough about udev to know that a smooth transition process
was perhaps possible, and it seemed very disruptive. And,
seeing as how I work for HP, it isn't exactly within my
authority to just decide by myself the course the driver
development should take. Paradoxically, if a person works
for a company that has nothing to do with the driver, one
has more freedom to do whatever one wants with the code.
With help from udev it is less disruptive, but I'm still unsure
about how smoothly such a change will fit in with the installation
process of various distributions.
I will see if management here will let me send you the patches
from when I last looked at this, which was a little over a year ago.
(2.6.20 timeframe), not necessarily to use as is, but in case
maybe it helps to see how somebody else proceeded.
As for the patches I mentioned which were previously ignored
for improving how the current cciss SCSI code handled hot
plogging, if you're curious about that, check here:
http://marc.info/?l=linux-scsi&m=120213565000426&w=4
A change like that becomes more important if logical drives
are exposed through the scsi layer.
-- steve
On Thu, 24 Jul 2008 08:30:12 -0500
[email protected] wrote:
> On Thu, Jul 24, 2008 at 10:32:45AM +0900, FUJITA Tomonori wrote:
> > CC'ed linux-scsi, why did you drop it?
>
> You mean my SCSI cciss driver? For a few reasons:
Ah, I just meant that you dropped linux-scsi mailing list in the
previous mail though I sent my first RFC mail to linux-kernel and
linux-scsi.
> With help from udev it is less disruptive, but I'm still unsure
> about how smoothly such a change will fit in with the installation
> process of various distributions.
They need to make sure that udev cciss rules match with the kernel
configuration but I don't think it's difficult.
> As for the patches I mentioned which were previously ignored
> for improving how the current cciss SCSI code handled hot
> plogging, if you're curious about that, check here:
>
> http://marc.info/?l=linux-scsi&m=120213565000426&w=4
>
> A change like that becomes more important if logical drives
> are exposed through the scsi layer.
We need to think about the best way to map (expose) logical and
physical drivers to SCSI luns. As I wrote before, I'm interested in
how HP SCSI driver does.
On Thu, Jul 24, 2008 at 10:44:03PM +0900, FUJITA Tomonori wrote:
> On Thu, 24 Jul 2008 08:30:12 -0500
> [email protected] wrote:
>
> > On Thu, Jul 24, 2008 at 10:32:45AM +0900, FUJITA Tomonori wrote:
> > > CC'ed linux-scsi, why did you drop it?
> >
> > You mean my SCSI cciss driver? For a few reasons:
>
> Ah, I just meant that you dropped linux-scsi mailing list in the
> previous mail though I sent my first RFC mail to linux-kernel and
> linux-scsi.
[...]
Oh, oops. I didn't drop it on purpose. Sorry about that.
-- steve
> > As for the patches I mentioned which were previously ignored for
> > improving how the current cciss SCSI code handled hot plogging, if
> > you're curious about that, check here:
> >
> > http://marc.info/?l=linux-scsi&m=120213565000426&w=4
> >
> > A change like that becomes more important if logical drives are
> > exposed through the scsi layer.
>
> We need to think about the best way to map (expose) logical
> and physical drivers to SCSI luns. As I wrote before, I'm
> interested in how HP SCSI driver does.
We don't neccesarily want or need to expose the physical disks. The only exception is disks that are not part of a logical volume. If the disks are part of a logical volume but exposed the user may shoot themselves in the foot by destroying the firmware metadata.
-- mikem
On Thu, 24 Jul 2008 15:00:00 +0000
"Miller, Mike (OS Dev)" <[email protected]> wrote:
>
> > > As for the patches I mentioned which were previously ignored for
> > > improving how the current cciss SCSI code handled hot plogging, if
> > > you're curious about that, check here:
> > >
> > > http://marc.info/?l=linux-scsi&m=120213565000426&w=4
> > >
> > > A change like that becomes more important if logical drives are
> > > exposed through the scsi layer.
> >
> > We need to think about the best way to map (expose) logical
> > and physical drivers to SCSI luns. As I wrote before, I'm
> > interested in how HP SCSI driver does.
>
> We don't neccesarily want or need to expose the physical disks. The only exception is disks that are not part of a logical volume. If the disks are part of a logical volume but exposed the user may shoot themselves in the foot by destroying the firmware metadata.
Oops, sorry, I meant physical tape drives.
In case it is of interest, here are the patches
to the 2.6.20.4 kernel which make a SCSI Smart Array
driver that I did in April of 2007. I called the
driver "hpsa", for HP Smart Array.
https://sourceforge.net/project/showfiles.php?group_id=33072&package_id=285296
I made these patches on a system which was booted from
a SmartArray 6400 using the cciss driver, so the first
patch disables cciss support in the cciss driver for the P800,
and the resulting SCSI driver supports only the Smartarray P800,
as I was doing the SCSI driver development on a P800 I had in
the system. That's the only reason that stuff is in
there in the 1st patch.
Because of the way the first few patches move files around,
these probably aren't particularly useful directly as patches
(and of course, they're for an old kernel anyway), but might
be useful as an illustration of one way things might be done.
-- steve
On Thu, 2008-07-24 at 15:00 +0000, Miller, Mike (OS Dev) wrote:
> > > As for the patches I mentioned which were previously ignored for
> > > improving how the current cciss SCSI code handled hot plogging, if
> > > you're curious about that, check here:
> > >
> > > http://marc.info/?l=linux-scsi&m=120213565000426&w=4
> > >
> > > A change like that becomes more important if logical drives are
> > > exposed through the scsi layer.
> >
> > We need to think about the best way to map (expose) logical
> > and physical drivers to SCSI luns. As I wrote before, I'm
> > interested in how HP SCSI driver does.
>
> We don't neccesarily want or need to expose the physical disks. The
> only exception is disks that are not part of a logical volume. If the
> disks are part of a logical volume but exposed the user may shoot
> themselves in the foot by destroying the firmware metadata.
Exposing the underlying disks can be useful for things such as:
* Finding out what is in a logical drive.
* Checking and updating disk firmware.
* Running smartctrl tests on disks
> -- mikem
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
Andrew Patterson
Hewlett-Packard
> -----Original Message-----
> From: FUJITA Tomonori [mailto:[email protected]]
> Sent: Thursday, July 24, 2008 10:07 AM
> To: Miller, Mike (OS Dev)
> Cc: [email protected];
> [email protected];
> [email protected]; [email protected]
> Subject: RE: HP (Compaq) Smart Array 5xxx controller SCSI driver
>
> On Thu, 24 Jul 2008 15:00:00 +0000
> "Miller, Mike (OS Dev)" <[email protected]> wrote:
>
> >
> > > > As for the patches I mentioned which were previously
> ignored for
> > > > improving how the current cciss SCSI code handled hot
> plogging, if
> > > > you're curious about that, check here:
> > > >
> > > > http://marc.info/?l=linux-scsi&m=120213565000426&w=4
> > > >
> > > > A change like that becomes more important if logical drives are
> > > > exposed through the scsi layer.
> > >
> > > We need to think about the best way to map (expose) logical and
> > > physical drivers to SCSI luns. As I wrote before, I'm
> interested in
> > > how HP SCSI driver does.
> >
> > We don't neccesarily want or need to expose the physical
> disks. The only exception is disks that are not part of a
> logical volume. If the disks are part of a logical volume but
> exposed the user may shoot themselves in the foot by
> destroying the firmware metadata.
>
> Oops, sorry, I meant physical tape drives.
>
Tomo,
I patched your driver into the 2.6.27-rc1 kernel. After compiling the driver I get:
[root@testmonkey linux-2.6]# insmod drivers/scsi/ciss.ko
insmod: error inserting 'drivers/scsi/ciss.ko': -1 Invalid module format
[root@testmonkey linux-2.6]# gcc --version
gcc (GCC) 4.1.2 20071124 (Red Hat 4.1.2-42)
[root@testmonkey linux-2.6]# uname -r
2.6.26
I built inside the tree. Any thoughts?
-- mikem
On Thu, 7 Aug 2008 19:04:54 +0000
"Miller, Mike (OS Dev)" <[email protected]> wrote:
>
>
> > -----Original Message-----
> > From: FUJITA Tomonori [mailto:[email protected]]
> > Sent: Thursday, July 24, 2008 10:07 AM
> > To: Miller, Mike (OS Dev)
> > Cc: [email protected];
> > [email protected];
> > [email protected]; [email protected]
> > Subject: RE: HP (Compaq) Smart Array 5xxx controller SCSI driver
> >
> > On Thu, 24 Jul 2008 15:00:00 +0000
> > "Miller, Mike (OS Dev)" <[email protected]> wrote:
> >
> > >
> > > > > As for the patches I mentioned which were previously
> > ignored for
> > > > > improving how the current cciss SCSI code handled hot
> > plogging, if
> > > > > you're curious about that, check here:
> > > > >
> > > > > http://marc.info/?l=linux-scsi&m=120213565000426&w=4
> > > > >
> > > > > A change like that becomes more important if logical drives are
> > > > > exposed through the scsi layer.
> > > >
> > > > We need to think about the best way to map (expose) logical and
> > > > physical drivers to SCSI luns. As I wrote before, I'm
> > interested in
> > > > how HP SCSI driver does.
> > >
> > > We don't neccesarily want or need to expose the physical
> > disks. The only exception is disks that are not part of a
> > logical volume. If the disks are part of a logical volume but
> > exposed the user may shoot themselves in the foot by
> > destroying the firmware metadata.
> >
> > Oops, sorry, I meant physical tape drives.
> >
>
> Tomo,
> I patched your driver into the 2.6.27-rc1 kernel. After compiling the driver I get:
>
> [root@testmonkey linux-2.6]# insmod drivers/scsi/ciss.ko
> insmod: error inserting 'drivers/scsi/ciss.ko': -1 Invalid module format
You can get an error message here from `dmseg`, I think.
> [root@testmonkey linux-2.6]# gcc --version
> gcc (GCC) 4.1.2 20071124 (Red Hat 4.1.2-42)
>
> [root@testmonkey linux-2.6]# uname -r
> 2.6.26
>
> I built inside the tree. Any thoughts?
Seems that you try to do insmod a module built for 2.6.27-rc1 into
2.6.26?
> -----Original Message-----
> From: FUJITA Tomonori [mailto:[email protected]]
> Sent: Thursday, August 07, 2008 8:08 PM
> To: Miller, Mike (OS Dev)
> Cc: [email protected];
> [email protected];
> [email protected]; [email protected]
> Subject: RE: HP (Compaq) Smart Array 5xxx controller SCSI driver
>
> On Thu, 7 Aug 2008 19:04:54 +0000
> "Miller, Mike (OS Dev)" <[email protected]> wrote:
>
> >
> >
> > > -----Original Message-----
> > > From: FUJITA Tomonori [mailto:[email protected]]
> > > Sent: Thursday, July 24, 2008 10:07 AM
> > > To: Miller, Mike (OS Dev)
> > > Cc: [email protected];
> > > [email protected];
> > > [email protected]; [email protected]
> > > Subject: RE: HP (Compaq) Smart Array 5xxx controller SCSI driver
> > >
> > > On Thu, 24 Jul 2008 15:00:00 +0000
> > > "Miller, Mike (OS Dev)" <[email protected]> wrote:
> > >
> > > >
> > > > > > As for the patches I mentioned which were previously
> > > ignored for
> > > > > > improving how the current cciss SCSI code handled hot
> > > plogging, if
> > > > > > you're curious about that, check here:
> > > > > >
> > > > > > http://marc.info/?l=linux-scsi&m=120213565000426&w=4
> > > > > >
> > > > > > A change like that becomes more important if logical drives
> > > > > > are exposed through the scsi layer.
> > > > >
> > > > > We need to think about the best way to map (expose)
> logical and
> > > > > physical drivers to SCSI luns. As I wrote before, I'm
> > > interested in
> > > > > how HP SCSI driver does.
> > > >
> > > > We don't neccesarily want or need to expose the physical
> > > disks. The only exception is disks that are not part of a logical
> > > volume. If the disks are part of a logical volume but exposed the
> > > user may shoot themselves in the foot by destroying the firmware
> > > metadata.
> > >
> > > Oops, sorry, I meant physical tape drives.
> > >
> >
> > Tomo,
> > I patched your driver into the 2.6.27-rc1 kernel. After
> compiling the driver I get:
> >
> > [root@testmonkey linux-2.6]# insmod drivers/scsi/ciss.ko
> > insmod: error inserting 'drivers/scsi/ciss.ko': -1 Invalid module
> > format
>
> You can get an error message here from `dmseg`, I think.
>
>
> > [root@testmonkey linux-2.6]# gcc --version gcc (GCC) 4.1.2 20071124
> > (Red Hat 4.1.2-42)
> >
> > [root@testmonkey linux-2.6]# uname -r
> > 2.6.26
> >
> > I built inside the tree. Any thoughts?
>
> Seems that you try to do insmod a module built for 2.6.27-rc1
> into 2.6.26?
Duh. Well that was pretty stupid.