The latest driver does not load balance with STK D178 array either.
I've discovered why, but I'm not sure which direction to take with
helping out to get better closure/resolution with this.
The cause of the problem is that the QLogic driver doesn't to
transparent LUN masking it seems. The reason this is a problem, is that
when the LSI/StoragTEK controllers present their luns, AVT is enabled.
This causes LUN "ghosting" down each path from the storage to the HBAs.
This becomes a problem because when the Linux Driver is told to perform
load balancing via static bindings, the LUNs are now out of order.
(whether LUN ghosting is happening or not).
This is where transparent LUN masking would solve this problem.
Example:
HBA0 is told it's only allowed to address the even numbered volumes.
HBA1 is told it's only allowed to address the odd-numberd volumes.
When this happens, the driver can see all the volumes, and names them
0:0:0:0-16(or whatever the last number is) for HBA0, and then
1:0:0:0-16(or whatever thelast number is). But, the OS is not shown
that, it sees the real LUN numbers as presented by the storage.
(0,2,4,6,8,10,etc)
Linux is not allowed to address LUNs out of sequence, so searching for
further LUN numbers stops after 0, since 2 is the next one.
Is there a way to resolve this, either at the driver level, IMHO the
place it *should* happen. At the storage level, the place that it could
also happen, or in the Kernel?
For the time being, I'm using a temporary load balanced setup for
performance reasons since we just extended our two primary loops from 1
tray each, to 3 and 4 trays. Please advise ASAP, as in this
configuration, we cannot fail-over.
TIA
--
Austin Gonyou <[email protected]>
Systems Architect
Coremetrics, Inc.
Cel: 512-698-7250
I was hoping someone on this list might have some ideas about this.
Please respond to me at this address or the Coremetrics one. I'm not
sure exactly what approach to take on this. Any hints would be helpful.
TIA.
On Sat, 2002-10-05 at 19:09, Austin Gonyou wrote:
> The latest driver does not load balance with STK D178 array either.
>
> I've discovered why, but I'm not sure which direction to take with
> helping out to get better closure/resolution with this.
>
>
> The cause of the problem is that the QLogic driver doesn't to
> transparent LUN masking it seems. The reason this is a problem, is that
> when the LSI/StoragTEK controllers present their luns, AVT is enabled.
> This causes LUN "ghosting" down each path from the storage to the HBAs.
> This becomes a problem because when the Linux Driver is told to perform
> load balancing via static bindings, the LUNs are now out of order.
> (whether LUN ghosting is happening or not).
>
....
> Linux is not allowed to address LUNs out of sequence, so searching for
> further LUN numbers stops after 0, since 2 is the next one.
>
> Is there a way to resolve this, either at the driver level, IMHO the
> place it *should* happen. At the storage level, the place that it could
> also happen, or in the Kernel?
>
> For the time being, I'm using a temporary load balanced setup for
> performance reasons since we just extended our two primary loops from 1
> tray each, to 3 and 4 trays. Please advise ASAP, as in this
> configuration, we cannot fail-over.
>
> TIA
> --
> Austin Gonyou <[email protected]>
> Systems Architect
> Coremetrics, Inc.
> Cel: 512-698-7250
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
On 2002-10-05T19:09:26,
Austin Gonyou <[email protected]> said:
> Is there a way to resolve this, either at the driver level, IMHO the
> place it *should* happen. At the storage level, the place that it could
> also happen, or in the Kernel?
You can always use md multipathing; an extension to the 2.4 multipathing has
been implemented by Jens Axboe and yours truely and is available at
http://lars.marowsky-bree.de/dl/md-mp; we'll see how Neil takes it when he
returns from vacation ;-)
We'll also be shipping that patch as part of United Linux.
IBM also did an extension to the LVM1 code to support multipathing; I don't
have an URL handy right now, but Google will certainly help out.
For 2.5, this is still not fully hashed out, but I assume you are running 2.4
on a production system ;-)
Sincerely,
Lars Marowsky-Br?e <[email protected]>
--
Principal Squirrel
Research and Development, SuSE Linux AG
``Immortality is an adequate definition of high availability for me.''
--- Gregory F. Pfister
> > Linux is not allowed to address LUNs out of sequence, so searching for
> > further LUN numbers stops after 0, since 2 is the next one.
That's not true:
CONFIG_SCSI_REPORT_LUNS:
If you want to build with SCSI REPORT LUNS support i the kernel, say Y here.
The REPORT LUNS command is useful for devices (such as disk arrays) with large numbers of LUNs where the LUN values are not contiguous (sparse LUN).
REPORT LUNS scanning is done only for SCSI-3 devices.
> > Is there a way to resolve this, either at the driver level, IMHO the
> > place it *should* happen. At the storage level, the place that it could
> > also happen, or in the Kernel?
This is new in 2.5.x
John.
On Sun, 2002-10-06 at 05:19, [email protected] wrote:
> > > Linux is not allowed to address LUNs out of sequence, so searching for
> > > further LUN numbers stops after 0, since 2 is the next one.
>
> That's not true:
>
> CONFIG_SCSI_REPORT_LUNS:
>
> If you want to build with SCSI REPORT LUNS support i the kernel, say Y here.
> The REPORT LUNS command is useful for devices (such as disk arrays) with large numbers of LUNs where the LUN values are not contiguous (sparse LUN).
> REPORT LUNS scanning is done only for SCSI-3 devices.
I believe my kernel has that configured. I will look when I wake. It's
530 am now, and I've been setting up my volumes for a while now. Just
about time to go to sleep, then wake up and install oracle. :)
> > > Is there a way to resolve this, either at the driver level, IMHO the
> > > place it *should* happen. At the storage level, the place that it could
> > > also happen, or in the Kernel?
>
> This is new in 2.5.x
>
I see. ATM I'm using 2.4.19, but would like to get to 2.4.20, because of
the TG3 fixes.
> John.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
> > > > Linux is not allowed to address LUNs out of sequence, so searching for
> > > > further LUN numbers stops after 0, since 2 is the next one.
> >
> > That's not true:
> >
> > CONFIG_SCSI_REPORT_LUNS:
> >
> > If you want to build with SCSI REPORT LUNS support i the kernel, say Y
> > here.
> > The REPORT LUNS command is useful for devices (such as disk arrays) with
> > large numbers of LUNs where the LUN values are not contiguous (sparse LUN).
> > REPORT LUNS scanning is done only for SCSI-3 devices.
>
> I believe my kernel has that configured. I will look when I wake. It's
> 530 am now, and I've been setting up my volumes for a while now. Just
> about time to go to sleep, then wake up and install oracle. :)
All night hacking sessions, are cool :-).
> > > > Is there a way to resolve this, either at the driver level, IMHO the
> > > > place it *should* happen. At the storage level, the place that it could
> > > > also happen, or in the Kernel?
> >
> > This is new in 2.5.x
> >
>
>
> I see. ATM I'm using 2.4.19, but would like to get to 2.4.20, because of
> the TG3 fixes.
You were probably thinking of CONFIG_SCSI_MULTI_LUN above, then. That causes the kernel to probe all LUNs instead of just LUN 0, which is the default due to a lot of broken devices to respond to all LUNs instead of just LUN 0. The sparse LUN option is in addition to that in 2.5.x.
If this is for a live server, it might be easiest to hard code the LUNs you need it to probe in to 2.4.x for now, and wait until 2.6.x for proper support.
John.
On Sun, 2002-10-06 at 02:58, Lars Marowsky-Bree wrote:
> On 2002-10-05T19:09:26,
> Austin Gonyou <[email protected]> said:
>
> > Is there a way to resolve this, either at the driver level, IMHO the
> > place it *should* happen. At the storage level, the place that it could
> > also happen, or in the Kernel?
>
> You can always use md multipathing; an extension to the 2.4 multipathing has
> been implemented by Jens Axboe and yours truely and is available at
> http://lars.marowsky-bree.de/dl/md-mp; we'll see how Neil takes it when he
> returns from vacation ;-)
>
> We'll also be shipping that patch as part of United Linux.
Is this for 2.4 or 2.5. Just FMI.
> IBM also did an extension to the LVM1 code to support multipathing; I don't
> have an URL handy right now, but Google will certainly help out.
>
> For 2.5, this is still not fully hashed out, but I assume you are running 2.4
> on a production system ;-)
>
>
> Sincerely,
> Lars Marowsky-Br?e <[email protected]>
--
Austin Gonyou <[email protected]>
Coremetrics, Inc.