2005-09-09 19:44:14

by Luben Tuikov

[permalink] [raw]
Subject: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

Note on Direct Mode (normal) and Task Collector Mode.

Direct Mode
-----------
In Direct Mode (default, normal) the SAS Layer calls Execute
Command SCSI RPC of the SAS LLDD which gives the command
to the SDS _immediately_.

This is the default normal operation for all SAS LLDDs.

Task Collector Mode
-------------------

Some hardware has the ability to DMA a bunch of tasks into
the host adapter memory in one go.

Also some hardware has the ability to have pending
quite a large number of commands. For the aic94xx SAS
LLDD, this is currently 512, but can be extended to as
much memory the host has.

Task Collector Mode is advertized by the LLDD that it wants
the SAS Layer to run in that mode for that LLDD. The SAS Layer
then does a natural coalescing of requests and would send
a bunch of those, linked, in one invocation of the
Execute Command SCSI RPC.

Task collector mode accomodates this, so that the host
adapter firmware gets less interrupts.

DBMS machines may want to run in this mode.

Signed-off-by: Luben Tuikov <[email protected]>

diff -X linux-2.6.13/Documentation/dontdiff -Naur linux-2.6.13-orig/drivers/scsi/sas-class/sas_scsi_host.c linux-2.6.13/drivers/scsi/sas-class/sas_scsi_host.c
--- linux-2.6.13-orig/drivers/scsi/sas-class/sas_scsi_host.c 1969-12-31 19:00:00.000000000 -0500
+++ linux-2.6.13/drivers/scsi/sas-class/sas_scsi_host.c 2005-09-09 11:14:29.000000000 -0400
@@ -0,0 +1,991 @@
+/*
+ * Serial Attached SCSI (SAS) class SCSI Host glue.
+ *
+ * Copyright (C) 2005 Adaptec, Inc. All rights reserved.
+ * Copyright (C) 2005 Luben Tuikov <[email protected]>
+ *
+ * This file is licensed under GPLv2.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
+ * USA
+ *
+ * $Id: //depot/sas-class/sas_scsi_host.c#55 $
+ */
+
+#include "sas_internal.h"
+#include <scsi/sas/sas_discover.h>
+#include <scsi/sas/sas_task.h>
+
+#include <scsi/scsi_host.h>
+#include <scsi/scsi_device.h>
+#include <scsi/scsi_tcq.h>
+#include <scsi/scsi.h>
+
+#include <linux/err.h>
+#include <linux/blkdev.h>
+#include <linux/kobject.h>
+#include <linux/scatterlist.h>
+
+/* The SAM LUN structure should be _completely_ opaque to SCSI Core.
+ * This is why this macro here, and not using the broken
+ * scsilun_to_int(). Ideally, a SCSI LUN should be communicated in
+ * its entirety, and not as an integer. For some unknown to myself
+ * reason, SCSI Core thinks that SCSI LUNs can be interpreted as
+ * integers.
+ */
+#define SCSI_LUN(_sam_lun) ((unsigned int)be32_to_cpu(*(__be32 *)_sam_lun))
+
+/* ---------- SCSI Core device registration ---------- */
+
+int sas_register_with_scsi(struct LU *lu)
+{
+ struct scsi_device *scsi_dev;
+
+ lu->map.channel = lu->parent->port->id;
+ lu->map.id = sas_reserve_free_id(lu->parent->port);
+ if (lu->map.id == -ENOMEM)
+ return -ENOMEM;
+
+ scsi_dev = scsi_add_device(lu->parent->port->ha->core.shost,
+ lu->map.channel, lu->map.id,
+ SCSI_LUN(lu->LUN));
+ if (IS_ERR(scsi_dev))
+ return PTR_ERR(scsi_dev);
+
+ return 0;
+}
+
+void sas_unregister_with_scsi(struct LU *lu)
+{
+ if (lu->uldd_dev) {
+ struct scsi_device *scsi_dev = lu->uldd_dev;
+ scsi_remove_device(scsi_dev);
+ }
+}
+
+/* ---------- SCSI Host glue ---------- */
+
+#define TO_SAS_TASK(_scsi_cmd) ((void *)(_scsi_cmd)->host_scribble)
+#define ASSIGN_SAS_TASK(_sc, _t) do { (_sc)->host_scribble = (void *) _t; } while (0)
+
+static void sas_scsi_task_done(struct sas_task *task)
+{
+ struct task_status_struct *ts = &task->task_status;
+ struct scsi_cmnd *sc = task->uldd_task;
+ unsigned ts_flags = task->task_state_flags;
+ int hs = 0, stat = 0;
+
+ if (unlikely(!sc)) {
+ SAS_DPRINTK("task_done called with non existing SCSI cmnd!\n");
+ list_del_init(&task->list);
+ sas_free_task(task);
+ return;
+ }
+
+ if (ts->resp == SAS_TASK_UNDELIVERED) {
+ /* transport error */
+ hs = DID_NO_CONNECT;
+ } else { /* ts->resp == SAS_TASK_COMPLETE */
+ /* task delivered, what happened afterwards? */
+ switch (ts->stat) {
+ case SAS_DEV_NO_RESPONSE:
+ case SAS_INTERRUPTED:
+ case SAS_PHY_DOWN:
+ case SAS_NAK_R_ERR:
+ case SAS_OPEN_TO:
+ hs = DID_NO_CONNECT;
+ break;
+ case SAS_DATA_UNDERRUN:
+ sc->resid = ts->residual;
+ if (sc->request_bufflen - sc->resid < sc->underflow)
+ hs = DID_ERROR;
+ break;
+ case SAS_DATA_OVERRUN:
+ hs = DID_ERROR;
+ break;
+ case SAS_QUEUE_FULL:
+ hs = DID_SOFT_ERROR; /* retry */
+ break;
+ case SAS_DEVICE_UNKNOWN:
+ hs = DID_BAD_TARGET;
+ break;
+ case SAS_SG_ERR:
+ hs = DID_PARITY;
+ break;
+ case SAS_OPEN_REJECT:
+ if (ts->open_rej_reason == SAS_OREJ_RSVD_RETRY)
+ hs = DID_SOFT_ERROR; /* retry */
+ else
+ hs = DID_ERROR;
+ break;
+ case SAS_PROTO_RESPONSE:
+ SAS_DPRINTK("LLDD:%s sent SAS_PROTO_RESP for an SSP "
+ "task; please report this\n",
+ task->dev->port->ha->sas_ha_name);
+ break;
+ case SAS_ABORTED_TASK:
+ hs = DID_ABORT;
+ break;
+ case SAM_CHECK_COND:
+ memcpy(sc->sense_buffer, ts->buf,
+ max(SCSI_SENSE_BUFFERSIZE, ts->buf_valid_size));
+ stat = SAM_CHECK_COND;
+ break;
+ default:
+ stat = ts->stat;
+ break;
+ }
+ }
+ ASSIGN_SAS_TASK(sc, NULL);
+ sc->result = (hs << 16) | stat;
+ list_del_init(&task->list);
+ sas_free_task(task);
+ /* This is very ugly but this is how SCSI Core works. */
+ if (ts_flags & SAS_TASK_STATE_ABORTED)
+ scsi_finish_command(sc);
+ else
+ sc->scsi_done(sc);
+}
+
+static inline enum task_attribute sas_scsi_get_task_attr(struct scsi_cmnd *cmd)
+{
+ enum task_attribute ta = TASK_ATTR_SIMPLE;
+ if (cmd->request && blk_rq_tagged(cmd->request)) {
+ if (cmd->device->ordered_tags &&
+ (cmd->request->flags & REQ_HARDBARRIER))
+ ta = TASK_ATTR_HOQ;
+ }
+ return ta;
+}
+
+static inline struct sas_task *sas_create_task(struct scsi_cmnd *cmd,
+ struct LU *lu,
+ unsigned long gfp_flags)
+{
+ struct sas_task *task = sas_alloc_task(gfp_flags);
+
+ if (!task)
+ return NULL;
+
+ *(u32 *)cmd->sense_buffer = 0;
+ task->uldd_task = cmd;
+ ASSIGN_SAS_TASK(cmd, task);
+
+ task->dev = lu->parent;
+ task->task_proto = task->dev->tproto; /* BUG_ON(!SSP) */
+
+ task->ssp_task.retry_count = 1;
+ memcpy(task->ssp_task.LUN, lu->LUN, 8);
+ task->ssp_task.task_attr = sas_scsi_get_task_attr(cmd);
+ memcpy(task->ssp_task.cdb, cmd->cmnd, 16);
+
+ task->scatter = cmd->request_buffer;
+ task->num_scatter = cmd->use_sg;
+ task->total_xfer_len = cmd->request_bufflen;
+ task->data_dir = cmd->sc_data_direction;
+
+ task->task_done = sas_scsi_task_done;
+
+ return task;
+}
+
+static inline int sas_queue_up(struct sas_task *task)
+{
+ struct sas_ha_struct *sas_ha = task->dev->port->ha;
+ struct scsi_core *core = &sas_ha->core;
+ unsigned long flags;
+ LIST_HEAD(list);
+
+ spin_lock_irqsave(&core->task_queue_lock, flags);
+ if (sas_ha->lldd_queue_size < core->task_queue_size + 1) {
+ spin_unlock_irqrestore(&core->task_queue_lock, flags);
+ return -SAS_QUEUE_FULL;
+ }
+ list_add_tail(&task->list, &core->task_queue);
+ core->task_queue_size += 1;
+ spin_unlock_irqrestore(&core->task_queue_lock, flags);
+ up(&core->queue_thread_sema);
+
+ return 0;
+}
+
+/**
+ * sas_queuecommand -- Enqueue a command for processing
+ * @parameters: See SCSI Core documentation
+ *
+ * Note: XXX: Remove the host unlock/lock pair when SCSI Core can
+ * call us without holding an IRQ spinlock...
+ */
+static int sas_queuecommand(struct scsi_cmnd *cmd,
+ void (*scsi_done)(struct scsi_cmnd *))
+{
+ int res = 0;
+ struct LU *lu = cmd->device->hostdata;
+ struct Scsi_Host *host = cmd->device->host;
+
+ spin_unlock_irq(host->host_lock);
+ if (!lu) {
+ SAS_DPRINTK("scsi cmd 0x%p sent to non existing LU\n",
+ cmd);
+ cmd->result = DID_BAD_TARGET << 16;
+ scsi_done(cmd);
+ goto out;
+ } else {
+ struct sas_ha_struct *sas_ha = lu->parent->port->ha;
+ struct sas_task *task;
+
+ res = -ENOMEM;
+ task = sas_create_task(cmd, lu, GFP_ATOMIC);
+ if (!task)
+ goto out;
+
+ cmd->scsi_done = scsi_done;
+ /* Queue up, Direct Mode or Task Collector Mode. */
+ if (sas_ha->lldd_max_execute_num < 2)
+ res = sas_ha->lldd_execute_task(task, 1, GFP_ATOMIC);
+ else
+ res = sas_queue_up(task);
+
+ /* Examine */
+ if (res) {
+ SAS_DPRINTK("lldd_execute_task returned: %d\n", res);
+ ASSIGN_SAS_TASK(cmd, NULL);
+ sas_free_task(task);
+ if (res == -SAS_QUEUE_FULL) {
+ cmd->result = DID_SOFT_ERROR << 16; /* retry */
+ res = 0;
+ scsi_done(cmd);
+ }
+ goto out;
+ }
+ }
+out:
+ spin_lock_irq(host->host_lock);
+ return res;
+}
+
+static void sas_scsi_clear_queue_lu(struct list_head *error_q, struct LU *lu)
+{
+ struct scsi_cmnd *cmd, *n;
+
+ list_for_each_entry_safe(cmd, n, error_q, eh_entry) {
+ struct LU *x = cmd->device->hostdata;
+
+ if (x == lu)
+ list_del_init(&cmd->eh_entry);
+ }
+}
+
+static void sas_scsi_clear_queue_I_T(struct list_head *error_q,
+ struct domain_device *dev)
+{
+ struct scsi_cmnd *cmd, *n;
+
+ list_for_each_entry_safe(cmd, n, error_q, eh_entry) {
+ struct LU *y = cmd->device->hostdata;
+ struct domain_device *x = y->parent;
+
+ if (x == dev)
+ list_del_init(&cmd->eh_entry);
+ }
+}
+
+static void sas_scsi_clear_queue_port(struct list_head *error_q,
+ struct sas_port *port)
+{
+ struct scsi_cmnd *cmd, *n;
+
+ list_for_each_entry_safe(cmd, n, error_q, eh_entry) {
+ struct LU *y = cmd->device->hostdata;
+ struct sas_port *x = y->parent->port;
+
+ if (x == port)
+ list_del_init(&cmd->eh_entry);
+ }
+}
+
+enum task_disposition {
+ TASK_IS_DONE,
+ TASK_IS_ABORTED,
+ TASK_IS_AT_LU,
+ TASK_IS_NOT_AT_LU,
+};
+
+static enum task_disposition sas_scsi_find_task(struct sas_task *task)
+{
+ struct sas_ha_struct *ha = task->dev->port->ha;
+ unsigned long flags;
+ int i, res;
+
+ if (ha->lldd_max_execute_num > 1) {
+ struct scsi_core *core = &ha->core;
+ struct sas_task *t, *n;
+
+ spin_lock_irqsave(&core->task_queue_lock, flags);
+ list_for_each_entry_safe(t, n, &core->task_queue, list) {
+ if (task == t) {
+ list_del_init(&t->list);
+ spin_unlock_irqrestore(&core->task_queue_lock,
+ flags);
+ SAS_DPRINTK("%s: task 0x%p aborted from "
+ "task_queue\n",
+ __FUNCTION__, task);
+ return TASK_IS_ABORTED;
+ }
+ }
+ spin_unlock_irqrestore(&core->task_queue_lock, flags);
+ }
+
+ for (i = 0; i < 5; i++) {
+ SAS_DPRINTK("%s: aborting task 0x%p\n", __FUNCTION__, task);
+ res = task->dev->port->ha->lldd_abort_task(task);
+
+ spin_lock_irqsave(&task->task_state_lock, flags);
+ if (task->task_state_flags & SAS_TASK_STATE_DONE) {
+ spin_unlock_irqrestore(&task->task_state_lock, flags);
+ SAS_DPRINTK("%s: task 0x%p is done\n", __FUNCTION__,
+ task);
+ return TASK_IS_DONE;
+ }
+ spin_unlock_irqrestore(&task->task_state_lock, flags);
+
+ if (res == TMF_RESP_FUNC_COMPLETE) {
+ SAS_DPRINTK("%s: task 0x%p is aborted\n",
+ __FUNCTION__, task);
+ return TASK_IS_ABORTED;
+ } else if (ha->lldd_query_task) {
+ SAS_DPRINTK("%s: querying task 0x%p\n",
+ __FUNCTION__, task);
+ res = ha->lldd_query_task(task);
+ if (res == TMF_RESP_FUNC_SUCC) {
+ SAS_DPRINTK("%s: task 0x%p at LU\n",
+ __FUNCTION__, task);
+ return TASK_IS_AT_LU;
+ } else if (res == TMF_RESP_FUNC_COMPLETE) {
+ SAS_DPRINTK("%s: task 0x%p not at LU\n",
+ __FUNCTION__, task);
+ return TASK_IS_NOT_AT_LU;
+ }
+ }
+ }
+ return res;
+}
+
+static int sas_recover_lu(struct domain_device *dev, struct LU *lu)
+{
+ struct sas_ha_struct *ha = dev->port->ha;
+ int res = TMF_RESP_FUNC_FAILED;
+
+ SAS_DPRINTK("eh: device %llx LUN %llx has the task\n",
+ SAS_ADDR(dev->sas_addr),
+ SAS_ADDR(lu->LUN));
+
+ if (ha->lldd_abort_task_set)
+ res = ha->lldd_abort_task_set(dev, lu->LUN);
+
+ if (res == TMF_RESP_FUNC_FAILED) {
+ if (ha->lldd_clear_task_set)
+ res = ha->lldd_clear_task_set(dev, lu->LUN);
+ }
+
+ if (res == TMF_RESP_FUNC_FAILED) {
+ if (ha->lldd_lu_reset)
+ res = ha->lldd_lu_reset(dev, lu->LUN);
+ }
+
+ return res;
+}
+
+static int sas_recover_I_T(struct domain_device *dev)
+{
+ struct sas_ha_struct *ha = dev->port->ha;
+ int res = TMF_RESP_FUNC_FAILED;
+
+ SAS_DPRINTK("I_T nexus reset for dev %016llx\n",
+ SAS_ADDR(dev->sas_addr));
+
+ if (ha->lldd_I_T_nexus_reset)
+ res = ha->lldd_I_T_nexus_reset(dev);
+
+ return res;
+}
+
+static int sas_scsi_recover_host(struct Scsi_Host *shost)
+{
+ struct sas_ha_struct *ha = SHOST_TO_SAS_HA(shost);
+ unsigned long flags;
+ LIST_HEAD(error_q);
+ struct scsi_cmnd *cmd, *n;
+ enum task_disposition res = TASK_IS_DONE;
+ int tmf_resp;
+
+ spin_lock_irqsave(shost->host_lock, flags);
+ list_splice_init(&shost->eh_cmd_q, &error_q);
+ spin_unlock_irqrestore(shost->host_lock, flags);
+
+ SAS_DPRINTK("Enter %s\n", __FUNCTION__);
+
+ /* All tasks on this list were marked SAS_TASK_STATE_ABORTED
+ * by sas_scsi_timed_out() callback.
+ */
+Again:
+ SAS_DPRINTK("going over list...\n");
+ list_for_each_entry_safe(cmd, n, &error_q, eh_entry) {
+ struct sas_task *task = TO_SAS_TASK(cmd);
+ struct LU *lu = cmd->device->hostdata;
+
+ SAS_DPRINTK("trying to find task 0x%p\n", task);
+ list_del_init(&cmd->eh_entry);
+ res = sas_scsi_find_task(task);
+
+ cmd->eh_eflags = 0;
+ shost->host_failed--;
+
+ switch (res) {
+ case TASK_IS_DONE:
+ SAS_DPRINTK("%s: task 0x%p is done\n", __FUNCTION__,
+ task);
+ task->task_done(task);
+ continue;
+ case TASK_IS_ABORTED:
+ SAS_DPRINTK("%s: task 0x%p is aborted\n",
+ __FUNCTION__, task);
+ task->task_done(task);
+ continue;
+ case TASK_IS_AT_LU:
+ SAS_DPRINTK("task 0x%p is at LU: lu recover\n", task);
+ tmf_resp = sas_recover_lu(task->dev, lu);
+ if (tmf_resp == TMF_RESP_FUNC_COMPLETE) {
+ SAS_DPRINTK("dev %016llx LU %016llx is "
+ "recovered\n",
+ SAS_ADDR(task->dev),
+ SAS_ADDR(lu->LUN));
+ task->task_done(task);
+ sas_scsi_clear_queue_lu(&error_q, lu);
+ goto Again;
+ }
+ /* fallthrough */
+ case TASK_IS_NOT_AT_LU:
+ SAS_DPRINTK("task 0x%p is not at LU: I_T recover\n",
+ task);
+ tmf_resp = sas_recover_I_T(task->dev);
+ if (tmf_resp == TMF_RESP_FUNC_COMPLETE) {
+ SAS_DPRINTK("I_T %016llx recovered\n",
+ SAS_ADDR(task->dev->sas_addr));
+ task->task_done(task);
+ sas_scsi_clear_queue_I_T(&error_q, task->dev);
+ goto Again;
+ }
+ /* Hammer time :-) */
+ if (ha->lldd_clear_nexus_port) {
+ struct sas_port *port = task->dev->port;
+ SAS_DPRINTK("clearing nexus for port:%d\n",
+ port->id);
+ res = ha->lldd_clear_nexus_port(port);
+ if (res == TMF_RESP_FUNC_COMPLETE) {
+ SAS_DPRINTK("clear nexus port:%d "
+ "succeeded\n", port->id);
+ task->task_done(task);
+ sas_scsi_clear_queue_port(&error_q,
+ port);
+ goto Again;
+ }
+ }
+ if (ha->lldd_clear_nexus_ha) {
+ SAS_DPRINTK("clear nexus ha\n");
+ res = ha->lldd_clear_nexus_ha(ha);
+ if (res == TMF_RESP_FUNC_COMPLETE) {
+ SAS_DPRINTK("clear nexus ha "
+ "succeeded\n");
+ task->task_done(task);
+ goto out;
+ }
+ }
+ /* If we are here -- this means that no amount
+ * of effort could recover from errors. Quite
+ * possibly the HA just disappeared.
+ */
+ SAS_DPRINTK("error from device %llx, LUN %llx "
+ "couldn't be recovered in any way\n",
+ SAS_ADDR(task->dev->sas_addr),
+ SAS_ADDR(lu->LUN));
+
+ task->task_done(task);
+ goto clear_q;
+ }
+ }
+out:
+ SAS_DPRINTK("--- Exit %s\n", __FUNCTION__);
+ return 0;
+clear_q:
+ SAS_DPRINTK("--- Exit %s -- clear_q\n", __FUNCTION__);
+ list_for_each_entry_safe(cmd, n, &error_q, eh_entry) {
+ struct sas_task *task = TO_SAS_TASK(cmd);
+ list_del_init(&cmd->eh_entry);
+ task->task_done(task);
+ }
+ return 0;
+}
+
+static enum scsi_eh_timer_return sas_scsi_timed_out(struct scsi_cmnd *cmd)
+{
+ struct sas_task *task = TO_SAS_TASK(cmd);
+ unsigned long flags;
+
+ if (!task) {
+ SAS_DPRINTK("command 0x%p, task 0x%p, timed out: EH_HANDLED\n",
+ cmd, task);
+ return EH_HANDLED;
+ }
+
+ spin_lock_irqsave(&task->task_state_lock, flags);
+ if (task->task_state_flags & SAS_TASK_STATE_DONE) {
+ spin_unlock_irqrestore(&task->task_state_lock, flags);
+ SAS_DPRINTK("command 0x%p, task 0x%p, timed out: EH_HANDLED\n",
+ cmd, task);
+ return EH_HANDLED;
+ }
+ task->task_state_flags |= SAS_TASK_STATE_ABORTED;
+ spin_unlock_irqrestore(&task->task_state_lock, flags);
+
+ SAS_DPRINTK("command 0x%p, task 0x%p, timed out: EH_NOT_HANDLED\n",
+ cmd, task);
+
+ return EH_NOT_HANDLED;
+}
+
+/**
+ * sas_slave_alloc -- configure an LU which SCSI Core wants to poke at
+ * @scsi_dev: pointer to scsi device
+ *
+ * The kludge here is that the only token we have to go by in order to
+ * identify which device SCSI Core has just found about, is channel,
+ * id and lun/2. Of course this is 1) incredibly broken and 2)
+ * leftover from when SCSI Core was SPI-centric. A solution would be
+ * to pass an opaque token to scsi_add_device, which SCSI Core treats
+ * as that, an opaque token, which it sets inside scsi_dev, so we can
+ * find out which device SCSI Core is talking about. That is, how
+ * SCSI Core is _addressing_ the device is not the business of LLDD
+ * and vice versa. An event _better_ solution is if SCSI Core knew
+ * about a "SCSI device with Target ports" so we can register only the
+ * targets, and then it would do its own LU discovery... See comment
+ * in sas_do_lu_discovery().
+ */
+static int sas_slave_alloc(struct scsi_device *scsi_dev)
+{
+ struct sas_ha_struct *sas_ha = SHOST_TO_SAS_HA(scsi_dev->host);
+ struct sas_port *port = sas_ha->sas_port[scsi_dev->channel];
+ unsigned id = scsi_dev->id;
+ unsigned lun = scsi_dev->lun;
+
+ struct domain_device *dev = NULL;
+ struct LU *lu = NULL;
+
+ scsi_dev->hostdata = NULL;
+
+ list_for_each_entry(dev, &port->dev_list, dev_list_node) {
+ if (dev->dev_type == SAS_END_DEV) {
+ list_for_each_entry(lu, &dev->end_dev.LU_list, list) {
+ if (lu->map.id == id &&
+ SCSI_LUN(lu->LUN) == lun) {
+ scsi_dev->hostdata = lu;
+ lu->uldd_dev = scsi_dev;
+ goto out_loop;
+ }
+ }
+ }
+ }
+out_loop:
+ if (!scsi_dev->hostdata) {
+ SAS_DPRINTK("sas device not found! How is this possible?\n");
+ return -ENODEV;
+ }
+ kobject_get(&lu->lu_obj);
+ return 0;
+}
+
+#define SAS_DEF_QD 32
+#define SAS_MAX_QD 64
+
+static int sas_slave_configure(struct scsi_device *scsi_dev)
+{
+ struct LU *lu = scsi_dev->hostdata;
+ struct domain_device *dev;
+ struct sas_ha_struct *sas_ha;
+
+ if (!lu) {
+ SAS_DPRINTK("slave configure and no LU?!\n");
+ return -ENODEV;
+ }
+
+ dev = lu->parent;
+ sas_ha = dev->port->ha;
+
+ if (scsi_dev->inquiry_len > 7) {
+ u8 bq = (scsi_dev->inquiry[6] & 0x80) ? 1 : 0;
+ u8 cq = (scsi_dev->inquiry[7] & 0x02) ? 1 : 0;
+
+ if (bq ^ cq) {
+ lu->tm_type = (bq<<1) | cq;
+ scsi_dev->tagged_supported = 1;
+ if (cq)
+ scsi_set_tag_type(scsi_dev, MSG_ORDERED_TAG);
+ else
+ scsi_set_tag_type(scsi_dev, MSG_SIMPLE_TAG);
+ scsi_activate_tcq(scsi_dev, SAS_DEF_QD);
+ } else {
+ SAS_DPRINTK("device %llx, LUN %llx doesn't support "
+ "TCQ\n", SAS_ADDR(dev->sas_addr),
+ SAS_ADDR(lu->LUN));
+ scsi_dev->tagged_supported = 0;
+ scsi_set_tag_type(scsi_dev, 0);
+ scsi_deactivate_tcq(scsi_dev, 1);
+ }
+ }
+
+ if (dev->end_dev.itnl_timeout > 0)
+ scsi_dev->timeout = HZ +
+ msecs_to_jiffies(dev->end_dev.itnl_timeout);
+
+ return 0;
+}
+
+static void sas_slave_destroy(struct scsi_device *scsi_dev)
+{
+ struct LU *lu = scsi_dev->hostdata;
+
+ if (lu) {
+ scsi_dev->hostdata = NULL;
+ lu->uldd_dev = NULL;
+ kobject_put(&lu->lu_obj);
+ }
+ scsi_device_put(scsi_dev);
+}
+
+static int sas_change_queue_depth(struct scsi_device *scsi_dev, int new_depth)
+{
+ int res = min(new_depth, SAS_MAX_QD);
+
+ if (scsi_dev->tagged_supported)
+ scsi_adjust_queue_depth(scsi_dev, scsi_get_tag_type(scsi_dev),
+ res);
+ else {
+ struct LU *lu = scsi_dev->hostdata;
+ sas_printk("device %llx LUN %llx queue depth changed to 1\n",
+ SAS_ADDR(lu->parent->sas_addr),
+ SAS_ADDR(lu->LUN));
+ scsi_adjust_queue_depth(scsi_dev, 0, 1);
+ res = 1;
+ }
+
+ return res;
+}
+
+static int sas_change_queue_type(struct scsi_device *scsi_dev, int qt)
+{
+ struct LU *lu = scsi_dev->hostdata;
+
+ if (!scsi_dev->tagged_supported)
+ return 0;
+
+ scsi_deactivate_tcq(scsi_dev, 1);
+
+ switch (qt) {
+ case MSG_ORDERED_TAG:
+ if (lu->tm_type != TASK_MANAGEMENT_FULL)
+ qt = MSG_SIMPLE_TAG;
+ break;
+ case MSG_SIMPLE_TAG:
+ default:
+ ;
+ }
+
+ scsi_set_tag_type(scsi_dev, qt);
+ scsi_activate_tcq(scsi_dev, scsi_dev->queue_depth);
+
+ return qt;
+}
+
+static int sas_bios_param(struct scsi_device *scsi_dev,
+ struct block_device *bdev,
+ sector_t capacity, int *hsc)
+{
+ hsc[0] = 255;
+ hsc[1] = 63;
+ sector_div(capacity, 255*63);
+ hsc[2] = capacity;
+
+ return 0;
+}
+
+static const struct scsi_host_template sas_host_template = {
+ .module = THIS_MODULE,
+ /* .name is initialized */
+ .name = "",
+ .queuecommand = sas_queuecommand,
+ .eh_strategy_handler = sas_scsi_recover_host,
+ .eh_timed_out = sas_scsi_timed_out,
+ .slave_alloc = sas_slave_alloc,
+ .slave_configure = sas_slave_configure,
+ .slave_destroy = sas_slave_destroy,
+ .change_queue_depth = sas_change_queue_depth,
+ .change_queue_type = sas_change_queue_type,
+ .bios_param = sas_bios_param,
+ /* .can_queue is initialized */
+ .this_id = -1,
+ .sg_tablesize = SG_ALL,
+ .max_sectors = SCSI_DEFAULT_MAX_SECTORS,
+ /* .cmd_per_lun is initilized to .can_queue */
+ .use_clustering = ENABLE_CLUSTERING,
+};
+
+static inline void sas_init_host_template(struct sas_ha_struct *sas_ha)
+{
+ struct scsi_host_template *sht = sas_ha->core.sht;
+
+ *sht = sas_host_template;
+
+ sht->name = sas_ha->sas_ha_name;
+ sht->can_queue = sas_ha->lldd_queue_size;
+ sht->cmd_per_lun = sht->can_queue;
+}
+
+int sas_register_scsi_host(struct sas_ha_struct *sas_ha)
+{
+ int err = -ENOMEM;
+
+ sas_ha->core.sht = kmalloc(sizeof(*sas_ha->core.sht), GFP_KERNEL);
+ if (!sas_ha->core.sht)
+ return -ENOMEM;
+ memset(sas_ha->core.sht, 0, sizeof(*sas_ha->core.sht));
+
+ sas_init_host_template(sas_ha);
+
+ sas_ha->core.shost = scsi_host_alloc(sas_ha->core.sht, sizeof(void *));
+ if (!sas_ha->core.shost) {
+ printk(KERN_NOTICE "couldn't allocate scsi host\n");
+ goto out_err;
+ }
+ SHOST_TO_SAS_HA(sas_ha->core.shost) = sas_ha;
+
+ /* XXX: SCSI Core should really fix this (max vs. num of) */
+ sas_ha->core.shost->max_channel = sas_ha->num_phys - 1;
+ sas_ha->core.shost->max_id = ~0 - 1;
+ sas_ha->core.shost->max_lun = ~0 - 1;
+
+ sas_ha->core.shost->max_cmd_len = 16;
+
+ err = scsi_add_host(sas_ha->core.shost, &sas_ha->pcidev->dev);
+ if (err) {
+ scsi_host_put(sas_ha->core.shost);
+ sas_ha->core.shost = NULL;
+ goto out_err;
+ }
+ return 0;
+
+out_err:
+ kfree(sas_ha->core.sht);
+ sas_ha->core.sht = NULL;
+ return err;
+}
+
+void sas_unregister_scsi_host(struct sas_ha_struct *sas_ha)
+{
+ scsi_remove_host(sas_ha->core.shost);
+ scsi_host_put(sas_ha->core.shost);
+ sas_ha->core.shost = NULL;
+ kfree(sas_ha->core.sht);
+ sas_ha->core.sht = NULL;
+}
+
+/* ---------- Task Collector Thread implementation ---------- */
+
+static void sas_queue(struct sas_ha_struct *sas_ha)
+{
+ struct scsi_core *core = &sas_ha->core;
+ unsigned long flags;
+ LIST_HEAD(q);
+ int can_queue;
+ int res;
+
+ spin_lock_irqsave(&core->task_queue_lock, flags);
+ while (!core->queue_thread_kill &&
+ !list_empty(&core->task_queue)) {
+
+ can_queue = sas_ha->lldd_queue_size - core->task_queue_size;
+ if (can_queue >= 0) {
+ can_queue = core->task_queue_size;
+ list_splice_init(&core->task_queue, &q);
+ } else {
+ struct list_head *a, *n;
+
+ can_queue = sas_ha->lldd_queue_size;
+ list_for_each_safe(a, n, &core->task_queue) {
+ list_move_tail(a, &q);
+ if (--can_queue == 0)
+ break;
+ }
+ can_queue = sas_ha->lldd_queue_size;
+ }
+ core->task_queue_size -= can_queue;
+ spin_unlock_irqrestore(&core->task_queue_lock, flags);
+ {
+ struct sas_task *task = list_entry(q.next,
+ struct sas_task,
+ list);
+ list_del_init(&q);
+ res = sas_ha->lldd_execute_task(task, can_queue,
+ GFP_KERNEL);
+ if (unlikely(res))
+ __list_add(&q, task->list.prev, &task->list);
+ }
+ spin_lock_irqsave(&core->task_queue_lock, flags);
+ if (res) {
+ list_splice_init(&q, &core->task_queue); /*at head*/
+ core->task_queue_size += can_queue;
+ }
+ }
+ spin_unlock_irqrestore(&core->task_queue_lock, flags);
+}
+
+static DECLARE_COMPLETION(queue_th_comp);
+
+/**
+ * sas_queue_thread -- The Task Collector thread
+ * @_sas_ha: pointer to struct sas_ha
+ */
+static int sas_queue_thread(void *_sas_ha)
+{
+ struct sas_ha_struct *sas_ha = _sas_ha;
+ struct scsi_core *core = &sas_ha->core;
+
+ daemonize("sas_queue_%d", core->shost->host_no);
+ current->flags |= PF_NOFREEZE;
+
+ complete(&queue_th_comp);
+
+ while (1) {
+ down_interruptible(&core->queue_thread_sema);
+ sas_queue(sas_ha);
+ if (core->queue_thread_kill)
+ break;
+ }
+
+ complete(&queue_th_comp);
+
+ return 0;
+}
+
+/* ---------- SCSI Core struct attributes ---------- */
+
+static ssize_t show_task_queue_size(struct scsi_core *core, char *page)
+{
+ return sprintf(page, "%d\n", core->task_queue_size);
+}
+
+struct scsi_core_attribute {
+ struct attribute attr;
+ ssize_t (*show)(struct scsi_core *, char *);
+ ssize_t (*store)(struct scsi_core *, const char *, size_t len);
+};
+
+#define to_scsi_core(_obj) container_of((_obj), struct scsi_core, \
+ scsi_core_obj)
+#define to_sc_attr(_attr) container_of((_attr), struct scsi_core_attribute,\
+ attr)
+
+static ssize_t sc_show_attr(struct kobject *kobj, struct attribute *attr,
+ char *page)
+{
+ ssize_t ret = 0;
+ struct scsi_core *core = to_scsi_core(kobj);
+ struct scsi_core_attribute *sc_attr = to_sc_attr(attr);
+
+ if (sc_attr->show)
+ ret = sc_attr->show(core, page);
+ return ret;
+}
+
+static struct scsi_core_attribute sc_attrs[] = {
+ __ATTR(task_queue_size, 0444, show_task_queue_size, NULL),
+ __ATTR_NULL,
+};
+
+static struct attribute *sc_def_attrs[ARRAY_SIZE(sc_attrs)];
+
+static struct sysfs_ops sc_sysfs_ops = {
+ .show = sc_show_attr,
+};
+
+static struct kobj_type scsi_core_ktype = {
+ .sysfs_ops = &sc_sysfs_ops,
+ .default_attrs = sc_def_attrs,
+};
+
+int sas_init_queue(struct sas_ha_struct *sas_ha)
+{
+ int res;
+ struct scsi_core *core = &sas_ha->core;
+
+ spin_lock_init(&core->task_queue_lock);
+ core->task_queue_size = 0;
+ INIT_LIST_HEAD(&core->task_queue);
+ init_MUTEX_LOCKED(&core->queue_thread_sema);
+
+ res = kernel_thread(sas_queue_thread, sas_ha, 0);
+ if (res >= 0) {
+ int i;
+ wait_for_completion(&queue_th_comp);
+
+ for (i = 0; i < ARRAY_SIZE(sc_attrs)-1; i++)
+ sc_def_attrs[i] = &sc_attrs[i].attr;
+ sc_def_attrs[i] = NULL;
+
+ core->scsi_core_obj.kset = &sas_ha->ha_kset;
+ kobject_set_name(&core->scsi_core_obj, "%s", "scsi_core");
+ core->scsi_core_obj.ktype = &scsi_core_ktype;
+ }
+
+ return res < 0 ? res : 0;
+}
+
+void sas_shutdown_queue(struct sas_ha_struct *sas_ha)
+{
+ unsigned long flags;
+ struct scsi_core *core = &sas_ha->core;
+ struct sas_task *task, *n;
+
+ init_completion(&queue_th_comp);
+ core->queue_thread_kill = 1;
+ up(&core->queue_thread_sema);
+ wait_for_completion(&queue_th_comp);
+
+ if (!list_empty(&core->task_queue))
+ SAS_DPRINTK("HA: %llx: scsi core task queue is NOT empty!?\n",
+ SAS_ADDR(sas_ha->sas_addr));
+
+ spin_lock_irqsave(&core->task_queue_lock, flags);
+ list_for_each_entry_safe(task, n, &core->task_queue, list) {
+ struct scsi_cmnd *cmd = task->uldd_task;
+
+ list_del_init(&task->list);
+
+ ASSIGN_SAS_TASK(cmd, NULL);
+ sas_free_task(task);
+ cmd->result = DID_ABORT << 16;
+ cmd->scsi_done(cmd);
+ }
+ spin_unlock_irqrestore(&core->task_queue_lock, flags);
+}



2005-09-09 23:36:53

by James Bottomley

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Fri, 2005-09-09 at 15:42 -0400, Luben Tuikov wrote:
> +static const struct scsi_host_template sas_host_template = {
> + .module = THIS_MODULE,
> + /* .name is initialized */
> + .name = "",
> + .queuecommand = sas_queuecommand,
> + .eh_strategy_handler = sas_scsi_recover_host,
> + .eh_timed_out = sas_scsi_timed_out,
> + .slave_alloc = sas_slave_alloc,
> + .slave_configure = sas_slave_configure,
> + .slave_destroy = sas_slave_destroy,
> + .change_queue_depth = sas_change_queue_depth,
> + .change_queue_type = sas_change_queue_type,
> + .bios_param = sas_bios_param,
> + /* .can_queue is initialized */
> + .this_id = -1,
> + .sg_tablesize = SG_ALL,
> + .max_sectors = SCSI_DEFAULT_MAX_SECTORS,
> + /* .cmd_per_lun is initilized to .can_queue */
> + .use_clustering = ENABLE_CLUSTERING,
> +};

You can't do something like this and be generic. You intercept all of
the slave_* calls and try to provide the template.

This has produced a class that might wrapper nicely around the aic94xx
but it won't attach nicely to any other SAS driver.

You can't decide what table size and alignment your drivers are going to
have because not all will conform to them. I already know that SATA (ex
ATA) vendors are getting into the SAS market ... they have particularly
weird SG allocation and alignment requirements for some of their stuff.

To be an actual transport class, aside from actually using the transport
class infrastructure, the code actually has to provide common routines
that a class of drivers can use.

There's already an embryonic SAS class working its way through the SCSI
list. Could you look at enhancing that instead of trying to produce a
competing class?

James


2005-09-10 04:12:28

by Luben Tuikov

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

--- James Bottomley <[email protected]> wrote:

> On Fri, 2005-09-09 at 15:42 -0400, Luben Tuikov wrote:
> > +static const struct scsi_host_template sas_host_template = {
> > + .module = THIS_MODULE,
> > + /* .name is initialized */
> > + .name = "",
> > + .queuecommand = sas_queuecommand,
> > + .eh_strategy_handler = sas_scsi_recover_host,
> > + .eh_timed_out = sas_scsi_timed_out,
> > + .slave_alloc = sas_slave_alloc,
> > + .slave_configure = sas_slave_configure,
> > + .slave_destroy = sas_slave_destroy,
> > + .change_queue_depth = sas_change_queue_depth,
> > + .change_queue_type = sas_change_queue_type,
> > + .bios_param = sas_bios_param,
> > + /* .can_queue is initialized */
> > + .this_id = -1,
> > + .sg_tablesize = SG_ALL,
> > + .max_sectors = SCSI_DEFAULT_MAX_SECTORS,
> > + /* .cmd_per_lun is initilized to .can_queue */
> > + .use_clustering = ENABLE_CLUSTERING,
> > +};
>
> You can't do something like this and be generic. You intercept all of
> the slave_* calls and try to provide the template.

I don't intecept them.

A SAS LLDD doesn't implement them. It implements Execute Command SCSI RPC,
and a bunch of TMFs.

> This has produced a class that might wrapper nicely around the aic94xx
> but it won't attach nicely to any other SAS driver.

How can you say something so generic?
"it won't attach nicely to any other SAS driver"?
Do you know _all_ other SAS drivers? You don't even know _one_!
(other than aic94xx)

See how some fields are initialized? The comments on top of them even
say that. Why cannot sg_tablesize, max_sectors and use_clustering
be also initialized similarly? (when the SAS LLDD registers with the
SAS Layer)

What you're talking about can be accommodated, easily.

No self respecting SAS chip would not do 64 bit DMA, or have an sg tablesize
or any other limitation.

Naturally, aic94xx has _no limitations_. :-) But hey, our hardware just
kicks a*s!

> You can't decide what table size and alignment your drivers are going to
> have because not all will conform to them. I already know that SATA (ex
> ATA) vendors are getting into the SAS market ... they have particularly

Yes, you heard this about ATA, and you think it applies to SAS?

FUD! This is handled by the transport/hardware/sequencer.

Look at Execute Command SCSI RPC in the code -- this is _the_ whole
point!

> weird SG allocation and alignment requirements for some of their stuff.
>
> To be an actual transport class, aside from actually using the transport
> class infrastructure, the code actually has to provide common routines
> that a class of drivers can use.

OMG? You must be completely on crack!

Did you not read the Announcement emails, 0, 1, 2?

How about SAS Domain Discover? How about Expander configuration?
How about Port management? How about SAS device configuration?

(Oh, I know, the solution you're paid to push into the kernel
doesn't need those since it does all SAS in the firmware.)

Look, I repeat again: A SAS LLDD need to generate at least 2 types
of event to use this infrastructure.

You are _completely_ on crack and I am aware that the whole world
is reading this email.

Did you not look at the code and see how little a SAS LLDD needs to
implement in order to use this infrastructure?

> There's already an embryonic SAS class working its way through the SCSI
> list. Could you look at enhancing that instead of trying to produce a
> competing class?

Hmm, lets see:
I posted today, a _complete_ solution, 1000 years ahead of this
"embryonic SAS class" you speak of.

This complete solution was written of the actual SCSI specs, and it
supports _everything_, including Domain discovery, user space configuration
of expanders, _and_ a user space program was written and included, and
any kind of device there is for SAS domains.

While this "embryonic SAS class" doesn't implement _any_ of that.
It is vastly incomplete, and _most of all_ it doesn't adhere to the spec,
because, again, the firmware of the controller you're writing this
against, hides all SAS from the driver.

So why would I go to an _inferior_ "solution", which doesn't follow
any spec, SAM or SPC or SAS while there is a complete SAS solution written
which completely adheres to the specs? (which absolves this other solution,
which would've never implemented Domain discovery, again because all SAS is
hidden in the firmware)

Furthermore, why do you want to use a downgrade solution?

The answer is simple:
After "emd", Dell (Hi Matt!) learned quickly that if they want something
in SCSI Core, they have to hire the people who _make_the_decisions_ what
goes in and stays out of SCSI Core, to write that something, irrespectively
of whether those people have the know-how to do that and irrespectively of
how crappy the code is. As long as it will go into the kernel. (Naturally
everyone wants to sell their product and $1e9 is a lot of money.)

So as long as *you are on their payroll*, what are you discussing here
with me? *You have an agenda*! You are not seeking the right
solution, you're seeking the agenda that you're being paid to follow.

Furthermore, LSI's SAS controller, which Dell is using, does
*not* export _anything_ SAS to SCSI Core. This is the whole point
of MPT (their licensed technology) -- to hide the transport
in the firmware and export only pure SCSI LUs to the kernel, so that
_one_ driver can work with multiple transports which the firmware
is implementing and hiding from the driver (at least this was the inital
thought). For this reason their driver should've been included into
the kernel from the get go, but you created some work for yourself.

So this "embryonic SAS class" you speak of, is being written
to accomodate _that_particular_ product, again by the people
who make the decisions of what goes into or out of the kernel,
being on the payroll of those companies. Whether that solution
follows anything SAS, nobody cares -- you're on their payroll
and they have to sell the product.

OTOH, aic94xx is a true SAS LLDD -- it exposes SAS to a managment
layer, giving more freedom to the user to do discovery and configuration
of the domain. Thus, the possibilities are endless.

So, now let me ask you this question:
Since I posted today a _complete_ solution to SAS support in the
kernel, why don't you scrap that "embryonic SAS class" and make
that "SAS" LLDD generate those 2 events and make use of that
infrastructure which is clearly superior.

Or do you have to complete the contract you signed and got paid
to do?

SCSI Core has become one crazy circus, where very little to none SCSI
competence and expertise is present. It is vendor driven, rather
than technology driven.

I long for the days of the previous maintainer. Had it not been
for him and Andi, we may have never had scsi commands from a slab,
scsi_done queue, done_q softirq processing, scsi timer hook, etc.
Of course back then Splentec wasn't your payrol company, but there
was some common sense present in linux-scsi.

Luben

2005-09-10 13:08:51

by Arjan van de Ven

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue


> No self respecting SAS chip would not do 64 bit DMA, or have an sg tablesize
> or any other limitation.

yet... there will be :)
>
> Naturally, aic94xx has _no limitations_. :-) But hey, our hardware just
> kicks a*s!

if it has no such limits then it indeed does!


> (Oh, I know, the solution you're paid to push into the kernel
> doesn't need those since it does all SAS in the firmware.)

I think you are way out of line here. James is very prudent in
separating his job at SteelEye and his maintainership.


> Hmm, lets see:
> I posted today, a _complete_ solution, 1000 years ahead of this
> "embryonic SAS class" you speak of.

yet you post this without having had ANY discussion or earlier reviews
in the recent months. IN fact to the outside world it looks like you sat
on this code for a long time for competative reasons and just posted it
now that Christoph is getting his layer finished.


> Furthermore, why do you want to use a downgrade solution?
>
> The answer is simple:
> After "emd", Dell (Hi Matt!) learned quickly that if they want something
> in SCSI Core, they have to hire the people who _make_the_decisions_ what
> goes in and stays out of SCSI Core, to write that something, irrespectively

well EMD's failure was 100% Adaptecs fault. Adaptec was warned EARLY ON
that a dmraid like solution was going to be needed. It was just that
Adaptec decided to ignore this advice (and focus only on 2.4 and ignore
2.6 entirely) that caused Adaptec to waste all the time and money on it.


> So as long as *you are on their payroll*, what are you discussing here

James is paid by SteelEye. Not by Dell or LSI.

> with me? *You have an agenda*!

so do you.

> I long for the days of the previous maintainer.

What previous maintainer?


2005-09-10 14:30:14

by Rik van Riel

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Fri, 9 Sep 2005, Luben Tuikov wrote:

> No self respecting SAS chip would not do 64 bit DMA, or have an sg
> tablesize or any other limitation.
>
> Naturally, aic94xx has _no limitations_. :-) But hey, our hardware just
> kicks a*s!

That's very nice for you - but lets face it, a SAS layer
that'll be unable to also deal with the El-Cheapo brand
controllers isn't going to be very useful.

--
All Rights Reversed

2005-09-10 19:57:24

by Alan

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Sad, 2005-09-10 at 10:30 -0400, Rik van Riel wrote:
> That's very nice for you - but lets face it, a SAS layer
> that'll be unable to also deal with the El-Cheapo brand
> controllers isn't going to be very useful.

If future cheap SAS controllers are like cheap anything else controllers
then it is better IMHO to deal with it once the problems are visible. We
*know* from experience that hardware limits will be weirder than the
anticipated.


Alan

2005-09-11 03:56:28

by Andi Kleen

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

In gmane.linux.scsi, you wrote:
> On Fri, 9 Sep 2005, Luben Tuikov wrote:
>
>> No self respecting SAS chip would not do 64 bit DMA, or have an sg
>> tablesize or any other limitation.
>>
>> Naturally, aic94xx has _no limitations_. :-) But hey, our hardware just
>> kicks a*s!
>
> That's very nice for you - but lets face it, a SAS layer
> that'll be unable to also deal with the El-Cheapo brand
> controllers isn't going to be very useful.

Nobody knows what these bu^wlimitations will be though. So you cannot
really plan for them in advance. When someone writes drivers for such
limited hardware they can add code to handle the limitations. But it
seems reasonable to start with a clean hardware model, and only
add the hacks later when they are really needed and the requirements
are understood.

-Andi

2005-09-11 09:38:53

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Fri, Sep 09, 2005 at 09:12:18PM -0700, Luben Tuikov wrote:
> > You can't do something like this and be generic. You intercept all of
> > the slave_* calls and try to provide the template.
>
> I don't intecept them.
>
> A SAS LLDD doesn't implement them. It implements Execute Command SCSI RPC,
> and a bunch of TMFs.

It should, though. the scsi_host_template is absolutely meant to be set
in the LLDD. Take a look at libata which uses pretty much only library
routines as methods in the scsi_host_template but let's the low-level
driver supply it anyway.

> No self respecting SAS chip would not do 64 bit DMA, or have an sg tablesize
> or any other limitation.

Famous last words..

> (Oh, I know, the solution you're paid to push into the kernel
> doesn't need those since it does all SAS in the firmware.)

James isn't paid by anyone to do scsi work.

> > There's already an embryonic SAS class working its way through the SCSI
> > list. Could you look at enhancing that instead of trying to produce a
> > competing class?
>
> Hmm, lets see:
> I posted today, a _complete_ solution, 1000 years ahead of this
> "embryonic SAS class" you speak of.

They're actually serving different needs. The host-bases SAS code you
wrote should be layering below my SAS transport class.

> This complete solution was written of the actual SCSI specs,

Just because I can still talk like a normal human without resorting
to techno-babble all the time I still have read the important t10.org
specs.

> While this "embryonic SAS class" doesn't implement _any_ of that.
> It is vastly incomplete, and _most of all_ it doesn't adhere to the spec,
> because, again, the firmware of the controller you're writing this
> against, hides all SAS from the driver.

What SPEC do you think a representation of SAS domains in the linux driver
model just adhere to?

> After "emd", Dell (Hi Matt!) learned quickly that if they want something
> in SCSI Core, they have to hire the people who _make_the_decisions_ what

s/_make_the_decisions_/listen to what the folks that make decisions tell them/

> So as long as *you are on their payroll*, what are you discussing here
> with me? *You have an agenda*! You are not seeking the right
> solution, you're seeking the agenda that you're being paid to follow.

As far as I know the only payroll James is on is Steeleye Technologies,
a clustering company.

> Furthermore, LSI's SAS controller, which Dell is using, does
> *not* export _anything_ SAS to SCSI Core. This is the whole point
> of MPT (their licensed technology) -- to hide the transport
> in the firmware and export only pure SCSI LUs to the kernel, so that
> _one_ driver can work with multiple transports which the firmware
> is implementing and hiding from the driver (at least this was the inital
> thought). For this reason their driver should've been included into
> the kernel from the get go, but you created some work for yourself.

Oh, always nice to hear from someone that the driver for the technology
competing to their employers' product shouldn't be included..

There will be more SAS LLDDs that either do more things in firmware like
LSI Fusion and ones that do things in the Host like the Adaptec one. And
we need to support both. The best way to do that is to have a small top
layer that just unifies the SAS domain presentation, and a 'libsas' layer
below it for host-bases SAS implementations.

> I long for the days of the previous maintainer. Had it not been
> for him and Andi, we may have never had scsi commands from a slab,
> scsi_done queue, done_q softirq processing, scsi timer hook, etc.
> Of course back then Splentec wasn't your payrol company, but there
> was some common sense present in linux-scsi.

Could you please stop this bullshit spreading now, thanks?

2005-09-11 09:40:16

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Sat, Sep 10, 2005 at 09:20:05PM +0100, Alan Cox wrote:
> On Sad, 2005-09-10 at 10:30 -0400, Rik van Riel wrote:
> > That's very nice for you - but lets face it, a SAS layer
> > that'll be unable to also deal with the El-Cheapo brand
> > controllers isn't going to be very useful.
>
> If future cheap SAS controllers are like cheap anything else controllers
> then it is better IMHO to deal with it once the problems are visible. We
> *know* from experience that hardware limits will be weirder than the
> anticipated.

Yes, absolutely. This discussion is driving far off right now, no one
is asking Adaptec to add support for competing products here, we're just
asking to not declare the host_template in the common code, and supporting
limited controllers is one of the reasons.

2005-09-11 13:41:32

by James Bottomley

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Sun, 2005-09-11 at 05:56 +0200, [email protected] wrote:
> >> Naturally, aic94xx has _no limitations_. :-) But hey, our hardware just
> >> kicks a*s!
> >
> > That's very nice for you - but lets face it, a SAS layer
> > that'll be unable to also deal with the El-Cheapo brand
> > controllers isn't going to be very useful.
>
> Nobody knows what these bu^wlimitations will be though. So you cannot
> really plan for them in advance. When someone writes drivers for such
> limited hardware they can add code to handle the limitations. But it
> seems reasonable to start with a clean hardware model, and only
> add the hacks later when they are really needed and the requirements
> are understood.

But my point was that we already have a mechanism for coping with this:
The scsi template parameterises some of these things (max sector size,
sg table elements, clustering, etc). For less standard things it
doesn't cover the driver uses the blk_ adjustors directly from
slave_alloc/slave_configure (This is currently how USB and firewire
communicate their alignment requirements).

By wrappering both the template and the slave_alloc/slave_configure
methods in the SAS class and not providing the driver access, it can't
use existing methods to make any adjustments that may be necessary.

James


2005-09-12 13:55:57

by Luben Tuikov

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On 09/10/05 09:08, Arjan van de Ven wrote:
>>Hmm, lets see:
>>I posted today, a _complete_ solution, 1000 years ahead of this
>>"embryonic SAS class" you speak of.
>
>
> yet you post this without having had ANY discussion or earlier reviews
> in the recent months. IN fact to the outside world it looks like you sat
> on this code for a long time for competative reasons and just posted it
> now that Christoph is getting his layer finished.

Arjan,

Here's some details for you:

Earlier this year, I was working with
* LSI, and
* HP,
To create a common infrastructure for SAS Domain representation
in sysfs and common interface. (Folks who had at least
browsed the SAS spec.) This was a volunteer effort by folks who
had some knowledge of this new techonology. I got an email from
HP and this is how it started.

This is where the RFC posted to linux-scsi and linux-kernel came from.
Read this email posted on April 13, 2005:
http://marc.theaimsgroup.com/?t=111340592700006&r=1&w=2

This was the product of our 2 month email discussion. It was a good
effort.

Then when I had the infrastructure ready, I sent code to that initial
list mentioned above, *and I CC-ed James* to show him what it would
look like, Christoph also got this code (via James). This was
on July 14, 2005. Other recipients were Jeff Garzik and Doug Gilbert.

So you see, everyone _was_ aware of what we had been doing.

>>So as long as *you are on their payroll*, what are you discussing here
>
>
> James is paid by SteelEye. Not by Dell or LSI.

He needs you to defend him? How do you know who pays his salary?

Are you sure he's not getting hardware from LSI/Dell and
compensation to do this "embryonic SAS class" for their hardware?

Are you his accountant?

>>with me? *You have an agenda*!
>
>
> so do you.

No, I do not. I'm _not_ a maintainer. I'm simply a _contributor_.
I've no agenda. I _contribute code to the Linux kernel_ in order to
make Linux kernel a *storage enterprise OS of choice*.

If I were the _maintainer_ *and* _contributor_ *then* you can say that.

But I'm only a _contributor_, contributing to expand the areas where
Linux can be used. This is a good thing.

Luben

>>I long for the days of the previous maintainer.
>
>
> What previous maintainer?

2005-09-12 13:56:54

by Luben Tuikov

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On 09/10/05 10:30, Rik van Riel wrote:
> On Fri, 9 Sep 2005, Luben Tuikov wrote:
>
>
>>No self respecting SAS chip would not do 64 bit DMA, or have an sg
>>tablesize or any other limitation.
>>
>>Naturally, aic94xx has _no limitations_. :-) But hey, our hardware just
>>kicks a*s!
>
>
> That's very nice for you - but lets face it, a SAS layer
> that'll be unable to also deal with the El-Cheapo brand
> controllers isn't going to be very useful.

Yes, I agree. Let's take them as they come.

Luben

2005-09-12 14:14:03

by Jörn Engel

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Mon, 12 September 2005 09:55:48 -0400, Luben Tuikov wrote:
>
> No, I do not. I'm _not_ a maintainer. I'm simply a _contributor_.

Putting in less effort than someone else is hardly an excuse for
anything. Much less for personal attacks.

Can you please cut it off and offer technical arguments? Even if the
maintainer had a personal problem with you, technical arguments tend
to win and good code will go in. Everything else is just mailing list
noise and increased hart rates for some people.

J?rn

--
You cannot suppose that Moliere ever troubled himself to be original in the
matter of ideas. You cannot suppose that the stories he tells in his plays
have never been told before. They were culled, as you very well know.
-- Andre-Louis Moreau in Scarabouche

2005-09-12 16:08:13

by Matthew Wilcox

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Sun, Sep 11, 2005 at 10:38:47AM +0100, Christoph Hellwig wrote:
> On Fri, Sep 09, 2005 at 09:12:18PM -0700, Luben Tuikov wrote:
> > I long for the days of the previous maintainer. Had it not been
> > for him and Andi, we may have never had scsi commands from a slab,
> > scsi_done queue, done_q softirq processing, scsi timer hook, etc.
> > Of course back then Splentec wasn't your payrol company, but there
> > was some common sense present in linux-scsi.
>
> Could you please stop this bullshit spreading now, thanks?

I meant to reply to Luben's original, but I deleted it in disgust.
As the person who converted SCSI from the old bottom-half completion
processing first to a tasklet and then to a softirq, I'd like to refute
at least part of this allegation. I didn't do it in order to improve
SCSI particularly, satisfy Splentec (who are they?), or prove a point.

I did it because I wanted to remove the old bottom-half mechanism and
SCSI was one of the remaining users. That required me to learn a bit
about the SCSI stack and I got sucked in. BTW, I believe at that time,
James had alreeady taken over maintenance. I'm not actually sure who
the previous maintainer was -- was it Eric Biederman?

A colleague asked me to summarise the current dispute. I said that
Luben's point was that nobody else understood SCSI. Everybody else's
point was that Luben doesn't understand Linux kernel development. Luben,
I think you need to shut up, accept advice, stop trying to do everything
in your own driver, and stop trying to have private conversations.
Just discuss things on linux-scsi dispassionately. There's no hidden
agenda to get you or your company. But you are pissing people off,
and very soon there *will* be because of your behaviour.

2005-09-12 17:12:28

by Luben Tuikov

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On 09/11/05 09:41, James Bottomley wrote:
> On Sun, 2005-09-11 at 05:56 +0200, [email protected] wrote:
>
>>>>Naturally, aic94xx has _no limitations_. :-) But hey, our hardware just
>>>>kicks a*s!
>>>
>>>That's very nice for you - but lets face it, a SAS layer
>>>that'll be unable to also deal with the El-Cheapo brand
>>>controllers isn't going to be very useful.
>>
>>Nobody knows what these bu^wlimitations will be though. So you cannot
>>really plan for them in advance. When someone writes drivers for such
>>limited hardware they can add code to handle the limitations. But it
>>seems reasonable to start with a clean hardware model, and only
>>add the hacks later when they are really needed and the requirements
>>are understood.
>
>
> But my point was that we already have a mechanism for coping with this:
> The scsi template parameterises some of these things (max sector size,
> sg table elements, clustering, etc).

James, people have _already_ replied to your point, saying
that they want to start with a _clean_ hardware model/interface.
See Alan and Andi's emails.

I don't know why do you keep grinding this, just like the
IDR thread (you had no other code to write?) and like the
"EH turned into ACA" thread?

It is time SCSI Core started cleanly, especially now with SAS
which will completely *replace* Parallel SCSI and IDE (for SATA).

> For less standard things it
> doesn't cover the driver uses the blk_ adjustors directly from
> slave_alloc/slave_configure (This is currently how USB and firewire
> communicate their alignment requirements).
>
> By wrappering both the template and the slave_alloc/slave_configure
> methods in the SAS class and not providing the driver access, it can't
> use existing methods to make any adjustments that may be necessary.

"that may be necessary"?

You don't listen to anyone, do you?

Let's deal with those when they come. The sas_ha_struct can certainly
deal with those as I pointed out in a previous email in this thread:
http://marc.theaimsgroup.com/?l=linux-scsi&m=112632556516797&w=2

We do not want to write code for things "that may be necessary".
This has been pointed out numerous times.

Luben

2005-09-12 17:30:24

by Alan

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Llu, 2005-09-12 at 13:12 -0400, Luben Tuikov wrote:
> > But my point was that we already have a mechanism for coping with this:
> > The scsi template parameterises some of these things (max sector size,
> > sg table elements, clustering, etc).
>
> James, people have _already_ replied to your point, saying
> that they want to start with a _clean_ hardware model/interface.
> See Alan and Andi's emails.

There is a difference between worrying about stuff later and not
supporting existing things people expect to work.

> It is time SCSI Core started cleanly, especially now with SAS
> which will completely *replace* Parallel SCSI and IDE (for SATA).

You have to get from here to there, and do so without breaking anything
or making it untestable on the way. Going around the scsi limits rather
than fixing the current weaknesses may or may not be the right way but
it needs to occur in logical steps.

Alan

2005-09-12 18:46:37

by Luben Tuikov

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

Arjan,

You misspelled my email address:
It is not "[email protected]",

It is "[email protected]".

Thanks,
Luben
P.S. I've added a CC to the correct address and left
the mistaken one in.



On 09/10/05 09:08, Arjan van de Ven wrote:
>>No self respecting SAS chip would not do 64 bit DMA, or have an sg tablesize
>>or any other limitation.
>
>
> yet... there will be :)
>
>>Naturally, aic94xx has _no limitations_. :-) But hey, our hardware just
>>kicks a*s!
>
>
> if it has no such limits then it indeed does!
>
>
>
>>(Oh, I know, the solution you're paid to push into the kernel
>>doesn't need those since it does all SAS in the firmware.)
>
>
> I think you are way out of line here. James is very prudent in
> separating his job at SteelEye and his maintainership.
>
>
>
>>Hmm, lets see:
>>I posted today, a _complete_ solution, 1000 years ahead of this
>>"embryonic SAS class" you speak of.
>
>
> yet you post this without having had ANY discussion or earlier reviews
> in the recent months. IN fact to the outside world it looks like you sat
> on this code for a long time for competative reasons and just posted it
> now that Christoph is getting his layer finished.
>
>
>
>>Furthermore, why do you want to use a downgrade solution?
>>
>>The answer is simple:
>> After "emd", Dell (Hi Matt!) learned quickly that if they want something
>>in SCSI Core, they have to hire the people who _make_the_decisions_ what
>>goes in and stays out of SCSI Core, to write that something, irrespectively
>
>
> well EMD's failure was 100% Adaptecs fault. Adaptec was warned EARLY ON
> that a dmraid like solution was going to be needed. It was just that
> Adaptec decided to ignore this advice (and focus only on 2.4 and ignore
> 2.6 entirely) that caused Adaptec to waste all the time and money on it.
>
>
>
>>So as long as *you are on their payroll*, what are you discussing here
>
>
> James is paid by SteelEye. Not by Dell or LSI.
>
>
>>with me? *You have an agenda*!
>
>
> so do you.
>
>
>>I long for the days of the previous maintainer.
>
>
> What previous maintainer?
>
>
>

2005-09-12 18:47:05

by Luben Tuikov

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On 09/12/05 10:13, J?rn Engel wrote:
> On Mon, 12 September 2005 09:55:48 -0400, Luben Tuikov wrote:
>
>>No, I do not. I'm _not_ a maintainer. I'm simply a _contributor_.
>
>
> Putting in less effort than someone else is hardly an excuse for
> anything. Much less for personal attacks.
>
> Can you please cut it off and offer technical arguments? Even if the
> maintainer had a personal problem with you, technical arguments tend
> to win and good code will go in. Everything else is just mailing list
> noise and increased hart rates for some people.

For technical arguments, please read the linux-scsi mailing list since
2000.

Thanks,
Luben

2005-09-12 19:07:28

by Luben Tuikov

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On 09/12/05 12:08, Matthew Wilcox wrote:
>
> I meant to reply to Luben's original, but I deleted it in disgust.
> As the person who converted SCSI from the old bottom-half completion
> processing first to a tasklet and then to a softirq, I'd like to refute
> at least part of this allegation. I didn't do it in order to improve
> SCSI particularly, satisfy Splentec (who are they?), or prove a point.
>
> I did it because I wanted to remove the old bottom-half mechanism and
> SCSI was one of the remaining users. That required me to learn a bit
> about the SCSI stack and I got sucked in. BTW, I believe at that time,
> James had alreeady taken over maintenance. I'm not actually sure who
> the previous maintainer was -- was it Eric Biederman?
>
> A colleague asked me to summarise the current dispute. I said that
> Luben's point was that nobody else understood SCSI. Everybody else's

No, my point is that SCSI Core "development" isn't following any
spec or document or any formally accepted spec.

And as more and more transports are coming aboard, you see more
and more "kludge" from SPI making everything fit around the
SCSI Core's legacy SPI as _opposed_ to _evolving_ SCSI Core.

> point was that Luben doesn't understand Linux kernel development. Luben,
> I think you need to shut up, accept advice, stop trying to do everything
> in your own driver, and stop trying to have private conversations.

I've had private conversations _before_ I posted anything and those
were highly technical -- you know SAS stuff.

> Just discuss things on linux-scsi dispassionately. There's no hidden

I have been, many times.

Absolutely no advice I've ever posted to linux-scsi has been accepted
ever, unless someone else implemented it.

Any advice I've ever posted have been SAM/SPC related.

> agenda to get you or your company. But you are pissing people off,
> and very soon there *will* be because of your behaviour.

Personal threat on a public list?

Luben

2005-09-12 19:55:29

by Matthew Wilcox

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Mon, Sep 12, 2005 at 03:07:20PM -0400, Luben Tuikov wrote:
> > A colleague asked me to summarise the current dispute. I said that
> > Luben's point was that nobody else understood SCSI. Everybody else's
>
> No, my point is that SCSI Core "development" isn't following any
> spec or document or any formally accepted spec.

That's right. And it's actually a good thing. I'm sorry you find it so
hard to accept, but please stop telling us we're wrong and evil.

> And as more and more transports are coming aboard, you see more
> and more "kludge" from SPI making everything fit around the
> SCSI Core's legacy SPI as _opposed_ to _evolving_ SCSI Core.

Nobody wants to see that. People do want the SCSI core to evolve.
It has evolved substantially in the last few years that I've been paying
attention. Again, I'm sorry that it's not evolving in the direction that
you'd like it to, but you're not doing a very good job at persuading us
we're wrong.

> > Just discuss things on linux-scsi dispassionately. There's no hidden
>
> I have been, many times.
>
> Absolutely no advice I've ever posted to linux-scsi has been accepted
> ever, unless someone else implemented it.
>
> Any advice I've ever posted have been SAM/SPC related.

If the advice is couched in the same terms you've been using recently,
I'm not surprised.

> > agenda to get you or your company. But you are pissing people off,
> > and very soon there *will* be because of your behaviour.
>
> Personal threat on a public list?

Nope, just a prediction. Perhaps I should have used 'shall' instead
of 'will'. If you continue being abusive, you'll end up in peoples
killfiles, which would be a real shame. If you think people are being
abusive towards you, please rise above it and don't attack them in
return. We all want your cards supported by the SCSI layer in the best
possible way. We just disagree about what that best possible way is.
I think we can all agree that the worst possible way is if somebody else
ends up writing a driver for your cards because your code isn't usable.

2005-09-12 22:00:23

by Luben Tuikov

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On 09/11/05 05:38, Christoph Hellwig wrote:
>>Hmm, lets see:
>>I posted today, a _complete_ solution, 1000 years ahead of this
>>"embryonic SAS class" you speak of.
>
>
> They're actually serving different needs. The host-bases SAS code you
> wrote should be layering below my SAS transport class.

Yes, they are completely orthogonal and can co-exist.

> What SPEC do you think a representation of SAS domains in the linux driver
> model just adhere to?

See figure 10, SAS Domain class diagram, in SAS1r09e. It cross-links
you to SAM-3 (you can also use SAM-4).

> There will be more SAS LLDDs that either do more things in firmware like
> LSI Fusion and ones that do things in the Host like the Adaptec one. And
> we need to support both. The best way to do that is to have a small top
> layer that just unifies the SAS domain presentation, and a 'libsas' layer
> below it for host-bases SAS implementations.

It is a _layer_ just like it is in SAM and SPC and SAS.
It is a _layer_ by SCSI design, if you look in SAM.

SAS is not libsas, but a transport _layer_ sitting between
the interconnect (hardware) and SCSI Core (SAM/SPC).

I wish I could do something to convince you, but you have
to convince yourself of this reading SAM and trying
to draw it out (literally) yourself. Try it for
an SPI domain, FC domain and SAS domain, and then you'll
see it.

The sysfs representation is just a perk of the transport
layer, it is owned and operated by the transport layer.

"transport attribute class" is just an _attribute_ class, Christoph.
"transport layer" is a lot more involved. I sincerely hope
you can see this. E.g. domain discovery belongs in the transport
layer. In SPI, LLDDs did it; in MPT the firmware does it.

With SAS, the domain is passive, and the protocol has evolved
enough (due to SAM) to yield to a common transport layer,
where common routines are done, as the SAS code shows.

The _next_ new protocol after SAS, will also adhere to SAM.
It will not happen tomorrow, but it will come around.

The less we invent our own stuff, and the more we adhere
to a spec, the easier we'd support new techonologies, since
they will adhere to SAM.

Luben

P.S. You know when I mentioned "SCSI" above, I did _not_ mean
SCSI-2 or SCSI (Parallel SCSI). I meant SCSI-3 (SAM).




2005-09-12 23:52:24

by Stefan Richter

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

Matthew Wilcox wrote:
> On Mon, Sep 12, 2005 at 03:07:20PM -0400, Luben Tuikov wrote:
>>No, my point is that SCSI Core "development" isn't following any
>>spec or document or any formally accepted spec.
>
> That's right. And it's actually a good thing.

Are you two discussing specs for scsi subsystem development process
here, or are you discussing specs for scsi subsystem architecture?

In case of the latter: These are indispensable. The scope of the scsi
subsystem is big enough to benefit from following the T10 specs,
including their concepts, their layers. I am saying this as somebody who
is trying to bring one small part of the subsystem forward, who will
never be able to grasp what is going on throughout the whole subsystem
nor to learn all about all SCSI layers. Thereby, every bigger or smaller
layering violation, every common SCSI concept that I have trouble to
match with Linux' scsi concepts will be a bigger or smaller setback for
undertakings like mine. (Well, I'm stating the obvious, or so I hope.)
--
Stefan Richter
-=====-=-=-= =--= -==-=
http://arcgraph.de/sr/

2005-09-13 12:41:44

by Luben Tuikov

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On 09/11/05 05:40, Christoph Hellwig wrote:
>
> Yes, absolutely. This discussion is driving far off right now, no one
> is asking Adaptec to add support for competing products here, we're just
> asking to not declare the host_template in the common code, and supporting
> limited controllers is one of the reasons.

Hi Christoph,

I cannot make something to be, something that it is not.

That is, the SAS LLDD is not a "scsi host" and it will never be,
because it is just not a "scsi host".

What it is, is an access point to the transport, this is its
sole function and existance. E.g. it doesn't know about max_luns,
etc, which are purely SCSI Core-into-scsi_host concepts.
It does know about Execute Command SCSI RPC and TMFs.

The "scsi_host" template is a SCSI Core concept, which
mixes a software component (what it is) and a hardware component
(what you're talking about).

Now, see the layering infrastructure: SATA support:
The sas_sata_host.c file would also declare a scsi_host, where
it will define its own queuecommand(), eh_timed_out(),
eh_strategy_handler(), etc. and do the _protocol_ part,
whereby the SAS LLDD does the transport part.

Luben





2005-09-13 15:40:21

by Matthew Wilcox

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Mon, Sep 12, 2005 at 06:00:15PM -0400, Luben Tuikov wrote:
> "transport attribute class" is just an _attribute_ class, Christoph.
> "transport layer" is a lot more involved. I sincerely hope
> you can see this. E.g. domain discovery belongs in the transport
> layer. In SPI, LLDDs did it; in MPT the firmware does it.

LLDDs having their own domain discovery code is definitely a misfeature.
As you know, stuff is being rearranged to move more of the SPI-specific
code from both SCSI core and LLDDs into the SPI transport. I suspect
domain discovery will always be triggered by the LLDD for SPI, but at
least a driver doesn't have to have its own code to do that any more.

2005-09-14 05:56:58

by Sergey Panov

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Tue, 2005-09-13 at 09:40 -0600, Matthew Wilcox wrote:
> On Mon, Sep 12, 2005 at 06:00:15PM -0400, Luben Tuikov wrote:
> > "transport attribute class" is just an _attribute_ class, Christoph.
> > "transport layer" is a lot more involved. I sincerely hope
> > you can see this. E.g. domain discovery belongs in the transport
> > layer. In SPI, LLDDs did it; in MPT the firmware does it.
>
> LLDDs having their own domain discovery code is definitely a misfeature.

Unfortunatly it is impossible to have one discovery code shared by all
LLDD under the same transport -- in FC world, Qlogic does "remote ports"
discovery in FW, Emulex does it in LLDD. Both approaches have benefits
and drawbacks, but it would be hard, if not impossible to move Emulex
discovery code into FC transport and to force Qlogic to use it, because
Qlogic HBA are not designed for doing discovery in the driver.
But the fact that Qlogic does discovery in FW/ASIC does not turn Qlogic
HBAs into SPI. I expect the same is true in SAS -- even though MPT cards
do discovery on the card, those SAS cards can not masquerade as a SPI
cards (well, in theory, it is possible, but ...).

> As you know, stuff is being rearranged to move more of the SPI-specific
> code from both SCSI core and LLDDs into the SPI transport. I suspect
> domain discovery will always be triggered by the LLDD for SPI, but at
> least a driver doesn't have to have its own code to do that any more.

Only if it can be turned into a some sort of library LLDD may use if it
needs it. But it is only makes sense to move that code out of the LLDD
and into the transport module, if more then one LLDD can make use of it.

Sergey Panov

======================================================================
Any opinions are personal and not necessarily those of my former,
present, or future employers.



2005-09-14 10:37:23

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On Wed, Sep 14, 2005 at 01:56:27AM -0400, Sergey Panov wrote:
> > As you know, stuff is being rearranged to move more of the SPI-specific
> > code from both SCSI core and LLDDs into the SPI transport. I suspect
> > domain discovery will always be triggered by the LLDD for SPI, but at
> > least a driver doesn't have to have its own code to do that any more.
>
> Only if it can be turned into a some sort of library LLDD may use if it
> needs it. But it is only makes sense to move that code out of the LLDD
> and into the transport module, if more then one LLDD can make use of it.

Umm, that's exactly what we are doing.

2005-09-14 10:56:01

by Jeff Garzik

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

Sergey Panov wrote:
> On Tue, 2005-09-13 at 09:40 -0600, Matthew Wilcox wrote:

>>As you know, stuff is being rearranged to move more of the SPI-specific
>>code from both SCSI core and LLDDs into the SPI transport. I suspect
>>domain discovery will always be triggered by the LLDD for SPI, but at
>>least a driver doesn't have to have its own code to do that any more.

> Only if it can be turned into a some sort of library LLDD may use if it
> needs it. But it is only makes sense to move that code out of the LLDD
> and into the transport module, if more then one LLDD can make use of it.


...and in this thread, SAS, more than one LLDD -should- be able to make
use it of.

ServerWorks/Broadcom SAS+SATA hardware, for which I will soon be writing
a driver, has exactly the same needs as Adaptec SAS+SATA hardware.

Jeff


2005-09-14 13:00:32

by Luben Tuikov

[permalink] [raw]
Subject: Re: [PATCH 2.6.13 14/14] sas-class: SCSI Host glue

On 09/14/05 06:53, Jeff Garzik wrote:
> ...and in this thread, SAS, more than one LLDD -should- be able to make
> use it of.
>
> ServerWorks/Broadcom SAS+SATA hardware, for which I will soon be writing
> a driver, has exactly the same needs as Adaptec SAS+SATA hardware.

Hi Jeff,

Nice of you to finally join us.

I'm sure you've looked at the SAS code. If SW/Broadcom has the same
open transport architecture, as we do, as you say they do, then you
should have no problem generating 4 events (link up/down, BYTES_DMAED,
BROADCAST(x)) to plug right into the SAS Layer, and have it do port
management, discovery, expander configuration, etc, for you.

Please let me know how it goes. I'm very curious.
If there's any concerns with the SAS Layer, I'm willing to help
in anywhich way I can.

Luben