2008-01-21 20:42:18

by Steve Wise

[permalink] [raw]
Subject: [PATCH RESEND 0/3] RDMA/cxgb3 fixes

Hey Roland,

Please include these three iw_cxgb3 fixes for 2.6.25. The first two fix
bugs found doing Lustre testing, and the last patch correctly marks
privileged qps.

Shortlog:
RDMA/cxgb3: Flush the RQ when closing.
RDMA/cxgb3: fix page shift calculation in build_phys_page_list()
RDMA/cxgb3: Mark qp as privileged based on user capabilities.

--
Steve.


2008-01-21 20:42:32

by Steve Wise

[permalink] [raw]
Subject: [PATCH RESEND 1/3] RDMA/cxgb3: Flush the RQ when closing.


RDMA/cxgb3: Flush the RQ when closing.

- for kernel mode cqs, call event notification handler when flushing
- flush qp when moving from RTS -> CLOSING
- fixed logic to identify a kernel mode qp

Signed-off-by: Steve Wise <[email protected]>
---

drivers/infiniband/hw/cxgb3/iwch_qp.c | 7 +++++--
1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/cxgb3/iwch_qp.c b/drivers/infiniband/hw/cxgb3/iwch_qp.c
index 9bb8112..7681fdc 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_qp.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_qp.c
@@ -642,6 +642,7 @@ static void __flush_qp(struct iwch_qp *qhp, unsigned long *flag)
cxio_flush_rq(&qhp->wq, &rchp->cq, count);
spin_unlock(&qhp->lock);
spin_unlock_irqrestore(&rchp->lock, *flag);
+ (*rchp->ibcq.comp_handler)(&rchp->ibcq, rchp->ibcq.cq_context);

/* locking heirarchy: cq lock first, then qp lock. */
spin_lock_irqsave(&schp->lock, *flag);
@@ -651,6 +652,7 @@ static void __flush_qp(struct iwch_qp *qhp, unsigned long *flag)
cxio_flush_sq(&qhp->wq, &schp->cq, count);
spin_unlock(&qhp->lock);
spin_unlock_irqrestore(&schp->lock, *flag);
+ (*schp->ibcq.comp_handler)(&schp->ibcq, schp->ibcq.cq_context);

/* deref */
if (atomic_dec_and_test(&qhp->refcnt))
@@ -661,7 +663,7 @@ static void __flush_qp(struct iwch_qp *qhp, unsigned long *flag)

static void flush_qp(struct iwch_qp *qhp, unsigned long *flag)
{
- if (t3b_device(qhp->rhp))
+ if (qhp->ibqp.uobject)
cxio_set_wq_in_error(&qhp->wq);
else
__flush_qp(qhp, flag);
@@ -830,10 +832,11 @@ int iwch_modify_qp(struct iwch_dev *rhp, struct iwch_qp *qhp,
disconnect = 1;
ep = qhp->ep;
}
+ flush_qp(qhp, &flag);
break;
case IWCH_QP_STATE_TERMINATE:
qhp->attr.state = IWCH_QP_STATE_TERMINATE;
- if (t3b_device(qhp->rhp))
+ if (qhp->ibqp.uobject)
cxio_set_wq_in_error(&qhp->wq);
if (!internal)
terminate = 1;

2008-01-21 20:42:48

by Steve Wise

[permalink] [raw]
Subject: [PATCH RESEND 2/3] RDMA/cxgb3: fix page shift calculation in build_phys_page_list()


RDMA/cxgb3: fix page shift calculation in build_phys_page_list()

The existing logic incorrectly maps this buffer list:

0: addr 0x10001000, size 0x1000
1: addr 0x10002000, size 0x1000

To this bogus page list:

0: 0x10000000
1: 0x10002000

The shift calculation must also take into account the address of the first
entry masked by the page_mask as well as the last address+size rounded
up to the next page size.

Signed-off-by: Steve Wise <[email protected]>
---

drivers/infiniband/hw/cxgb3/iwch_mem.c | 7 +++++++
1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/hw/cxgb3/iwch_mem.c b/drivers/infiniband/hw/cxgb3/iwch_mem.c
index a6c2c4b..73bfd16 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_mem.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_mem.c
@@ -122,6 +122,13 @@ int build_phys_page_list(struct ib_phys_buf *buffer_list,
*total_size += buffer_list[i].size;
if (i > 0)
mask |= buffer_list[i].addr;
+ else
+ mask |= buffer_list[i].addr & PAGE_MASK;
+ if (i != num_phys_buf - 1)
+ mask |= buffer_list[i].addr + buffer_list[i].size;
+ else
+ mask |= (buffer_list[i].addr + buffer_list[i].size +
+ PAGE_SIZE - 1) & PAGE_MASK;
}

if (*total_size > 0xFFFFFFFFULL)

2008-01-21 20:43:05

by Steve Wise

[permalink] [raw]
Subject: [PATCH RESEND 3/3] RDMA/cxgb3: Mark qp as privileged based on user capabilities.


RDMA/cxgb3: Mark qp as privileged based on user capabilities.

This is needed for zero-stag support.

Signed-off-by: Steve Wise <[email protected]>
---

drivers/infiniband/hw/cxgb3/cxio_wr.h | 3 ++-
drivers/infiniband/hw/cxgb3/iwch_qp.c | 1 +
2 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/hw/cxgb3/cxio_wr.h b/drivers/infiniband/hw/cxgb3/cxio_wr.h
index c84d4ac..d72b584 100644
--- a/drivers/infiniband/hw/cxgb3/cxio_wr.h
+++ b/drivers/infiniband/hw/cxgb3/cxio_wr.h
@@ -324,7 +324,8 @@ struct t3_genbit {
};

enum rdma_init_wr_flags {
- RECVS_POSTED = 1,
+ RECVS_POSTED = (1<<0),
+ PRIV_QP = (1<<1),
};

union t3_wr {
diff --git a/drivers/infiniband/hw/cxgb3/iwch_qp.c b/drivers/infiniband/hw/cxgb3/iwch_qp.c
index 7681fdc..ea2cdd7 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_qp.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_qp.c
@@ -717,6 +717,7 @@ static int rdma_init(struct iwch_dev *rhp, struct iwch_qp *qhp,
init_attr.qp_dma_addr = qhp->wq.dma_addr;
init_attr.qp_dma_size = (1UL << qhp->wq.size_log2);
init_attr.flags = rqes_posted(qhp) ? RECVS_POSTED : 0;
+ init_attr.flags |= capable(CAP_NET_BIND_SERVICE) ? PRIV_QP : 0;
init_attr.irs = qhp->ep->rcv_seq;
PDBG("%s init_attr.rq_addr 0x%x init_attr.rq_size = %d "
"flags 0x%x qpcaps 0x%x\n", __FUNCTION__,