Since v3:
* Apply suggestions from Sean, Ira, Jason, thanks for the comments :-)
and please remind me if I missed anything :-P
* No more cap_XX_dev(), all the check are per-port now, use port 1 for
cases only care about the cm type, can't support mixing cm type device
in future.
There are plenty of lengthy code to check the transport type of IB device,
or the link layer type of it's port, but actually we are just speculating
whether a particular management/feature is supported by the device/port.
Thus instead of inferring, we should have our own mechanism for IB management
capability/protocol/feature checking, several proposals below.
This patch set will reform the method of getting transport type, we will
now using query_transport() instead of inferring from transport and link
layer respectively, also we defined the new transport type to make the
concept more reasonable.
Mapping List:
node-type link-layer old-transport new-transport
nes RNIC ETH IWARP IWARP
amso1100 RNIC ETH IWARP IWARP
cxgb3 RNIC ETH IWARP IWARP
cxgb4 RNIC ETH IWARP IWARP
usnic USNIC_UDP ETH USNIC_UDP USNIC_UDP
ocrdma IB_CA ETH IB IBOE
mlx4 IB_CA IB/ETH IB IB/IBOE
mlx5 IB_CA IB IB IB
ehca IB_CA IB IB IB
ipath IB_CA IB IB IB
mthca IB_CA IB IB IB
qib IB_CA IB IB IB
For example:
if (transport == IB) && (link-layer == ETH)
will now become:
if (query_transport() == IBOE)
Thus we will be able to get rid of the respective transport and link-layer
checking, and it will help us to add new protocol/Technology (like OPA) more
easier, also with the introduced management helpers, IB management logical
will be more clear and easier for extending.
Highlights:
The patch set covered a wide range of IB stuff, thus for those who are
familiar with the particular part, your suggestion would be invaluable ;-)
Patches haven't been tested yet, we appreciate if any one who have these
HW willing to provide his Tested-by :-)
Doug suggested the bitmask mechanism:
https://www.mail-archive.com/[email protected]/msg23765.html
which could be the plan for future reforming, we prefer that to be another
series which focus on semantic and performance.
This patch-set is somewhat 'bloated' now and it may be a good timing for
staging, I'd like to suggest we focus on improving existed helpers and push
all the further reforms into next series ;-)
Proposals:
Sean:
https://www.mail-archive.com/[email protected]/msg23339.html
Doug:
https://www.mail-archive.com/[email protected]/msg23418.html
https://www.mail-archive.com/[email protected]/msg23765.html
Jason:
https://www.mail-archive.com/[email protected]/msg23425.html
Michael Wang (27):
IB/Verbs: Implement new callback query_transport()
IB/Verbs: Implement raw management helpers
IB/Verbs: Reform IB-core mad/agent/user_mad
IB/Verbs: Reform IB-core cm
IB/Verbs: Reform IB-core sa_query
IB/Verbs: Reform IB-core multicast
IB/Verbs: Reform IB-ulp ipoib
IB/Verbs: Reform IB-ulp xprtrdma
IB/Verbs: Reform IB-core verbs/uverbs_cmd/sysfs
IB/Verbs: Reform cm related part in IB-core cma/ucm
IB/Verbs: Reform route related part in IB-core cma
IB/Verbs: Reform mcast related part in IB-core cma
IB/Verbs: Reserve legacy transport type in 'dev_addr'
IB/Verbs: Reform cma_acquire_dev()
IB/Verbs: Reform rest part in IB-core cma
IB/Verbs: Use management helper cap_ib_mad()
IB/Verbs: Use management helper cap_ib_smi()
IB/Verbs: Use management helper cap_ib_cm()
IB/Verbs: Use management helper cap_iw_cm()
IB/Verbs: Use management helper cap_ib_sa()
IB/Verbs: Use management helper cap_ib_mcast()
IB/Verbs: Use management helper cap_ipoib()
IB/Verbs: Use management helper cap_read_multi_sge()
IB/Verbs: Use management helper cap_af_ib()
IB/Verbs: Use management helper cap_eth_ah()
IB/Verbs: Clean up rdma_ib_or_iboe()
IB/Verbs: Cleanup rdma_node_get_transport()
---
drivers/infiniband/core/agent.c | 4
drivers/infiniband/core/cm.c | 28 +-
drivers/infiniband/core/cma.c | 330 ++++++++++++---------------
drivers/infiniband/core/device.c | 1
drivers/infiniband/core/mad.c | 28 +-
drivers/infiniband/core/multicast.c | 18 -
drivers/infiniband/core/sa_query.c | 41 +--
drivers/infiniband/core/sysfs.c | 8
drivers/infiniband/core/ucm.c | 5
drivers/infiniband/core/ucma.c | 27 --
drivers/infiniband/core/user_mad.c | 32 +-
drivers/infiniband/core/uverbs_cmd.c | 6
drivers/infiniband/core/verbs.c | 33 --
drivers/infiniband/hw/amso1100/c2_provider.c | 7
drivers/infiniband/hw/cxgb3/iwch_provider.c | 7
drivers/infiniband/hw/cxgb4/provider.c | 7
drivers/infiniband/hw/ehca/ehca_hca.c | 6
drivers/infiniband/hw/ehca/ehca_iverbs.h | 3
drivers/infiniband/hw/ehca/ehca_main.c | 1
drivers/infiniband/hw/ipath/ipath_verbs.c | 7
drivers/infiniband/hw/mlx4/main.c | 10
drivers/infiniband/hw/mlx5/main.c | 7
drivers/infiniband/hw/mthca/mthca_provider.c | 7
drivers/infiniband/hw/nes/nes_verbs.c | 6
drivers/infiniband/hw/ocrdma/ocrdma_main.c | 1
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 6
drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 3
drivers/infiniband/hw/qib/qib_verbs.c | 7
drivers/infiniband/hw/usnic/usnic_ib_main.c | 1
drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 6
drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 2
drivers/infiniband/ulp/ipoib/ipoib_main.c | 17 -
include/rdma/ib_verbs.h | 204 +++++++++++++++-
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 6
net/sunrpc/xprtrdma/svc_rdma_transport.c | 51 +---
35 files changed, 579 insertions(+), 354 deletions(-)
Add new callback query_transport() and implement for each HW.
Mapping List:
node-type link-layer old-transport new-transport
nes RNIC ETH IWARP IWARP
amso1100 RNIC ETH IWARP IWARP
cxgb3 RNIC ETH IWARP IWARP
cxgb4 RNIC ETH IWARP IWARP
usnic USNIC_UDP ETH USNIC_UDP USNIC_UDP
ocrdma IB_CA ETH IB IBOE
mlx4 IB_CA IB/ETH IB IB/IBOE
mlx5 IB_CA IB IB IB
ehca IB_CA IB IB IB
ipath IB_CA IB IB IB
mthca IB_CA IB IB IB
qib IB_CA IB IB IB
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/device.c | 1 +
drivers/infiniband/core/verbs.c | 4 +++-
drivers/infiniband/hw/amso1100/c2_provider.c | 7 +++++++
drivers/infiniband/hw/cxgb3/iwch_provider.c | 7 +++++++
drivers/infiniband/hw/cxgb4/provider.c | 7 +++++++
drivers/infiniband/hw/ehca/ehca_hca.c | 6 ++++++
drivers/infiniband/hw/ehca/ehca_iverbs.h | 3 +++
drivers/infiniband/hw/ehca/ehca_main.c | 1 +
drivers/infiniband/hw/ipath/ipath_verbs.c | 7 +++++++
drivers/infiniband/hw/mlx4/main.c | 10 ++++++++++
drivers/infiniband/hw/mlx5/main.c | 7 +++++++
drivers/infiniband/hw/mthca/mthca_provider.c | 7 +++++++
drivers/infiniband/hw/nes/nes_verbs.c | 6 ++++++
drivers/infiniband/hw/ocrdma/ocrdma_main.c | 1 +
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 6 ++++++
drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 3 +++
drivers/infiniband/hw/qib/qib_verbs.c | 7 +++++++
drivers/infiniband/hw/usnic/usnic_ib_main.c | 1 +
drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 6 ++++++
drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 2 ++
include/rdma/ib_verbs.h | 7 ++++++-
21 files changed, 104 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index 18c1ece..a9587c4 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -76,6 +76,7 @@ static int ib_device_check_mandatory(struct ib_device *device)
} mandatory_table[] = {
IB_MANDATORY_FUNC(query_device),
IB_MANDATORY_FUNC(query_port),
+ IB_MANDATORY_FUNC(query_transport),
IB_MANDATORY_FUNC(query_pkey),
IB_MANDATORY_FUNC(query_gid),
IB_MANDATORY_FUNC(alloc_pd),
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index f93eb8d..626c9cf 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -133,14 +133,16 @@ enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device, u8 port_
if (device->get_link_layer)
return device->get_link_layer(device, port_num);
- switch (rdma_node_get_transport(device->node_type)) {
+ switch (device->query_transport(device, port_num)) {
case RDMA_TRANSPORT_IB:
return IB_LINK_LAYER_INFINIBAND;
+ case RDMA_TRANSPORT_IBOE:
case RDMA_TRANSPORT_IWARP:
case RDMA_TRANSPORT_USNIC:
case RDMA_TRANSPORT_USNIC_UDP:
return IB_LINK_LAYER_ETHERNET;
default:
+ BUG();
return IB_LINK_LAYER_UNSPECIFIED;
}
}
diff --git a/drivers/infiniband/hw/amso1100/c2_provider.c b/drivers/infiniband/hw/amso1100/c2_provider.c
index bdf3507..d46bbb0 100644
--- a/drivers/infiniband/hw/amso1100/c2_provider.c
+++ b/drivers/infiniband/hw/amso1100/c2_provider.c
@@ -99,6 +99,12 @@ static int c2_query_port(struct ib_device *ibdev,
return 0;
}
+static enum rdma_transport_type
+c2_query_transport(struct ib_device *device, u8 port_num)
+{
+ return RDMA_TRANSPORT_IWARP;
+}
+
static int c2_query_pkey(struct ib_device *ibdev,
u8 port, u16 index, u16 * pkey)
{
@@ -801,6 +807,7 @@ int c2_register_device(struct c2_dev *dev)
dev->ibdev.dma_device = &dev->pcidev->dev;
dev->ibdev.query_device = c2_query_device;
dev->ibdev.query_port = c2_query_port;
+ dev->ibdev.query_transport = c2_query_transport;
dev->ibdev.query_pkey = c2_query_pkey;
dev->ibdev.query_gid = c2_query_gid;
dev->ibdev.alloc_ucontext = c2_alloc_ucontext;
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index 811b24a..09682e9e 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -1232,6 +1232,12 @@ static int iwch_query_port(struct ib_device *ibdev,
return 0;
}
+static enum rdma_transport_type
+iwch_query_transport(struct ib_device *device, u8 port_num)
+{
+ return RDMA_TRANSPORT_IWARP;
+}
+
static ssize_t show_rev(struct device *dev, struct device_attribute *attr,
char *buf)
{
@@ -1385,6 +1391,7 @@ int iwch_register_device(struct iwch_dev *dev)
dev->ibdev.dma_device = &(dev->rdev.rnic_info.pdev->dev);
dev->ibdev.query_device = iwch_query_device;
dev->ibdev.query_port = iwch_query_port;
+ dev->ibdev.query_transport = iwch_query_transport;
dev->ibdev.query_pkey = iwch_query_pkey;
dev->ibdev.query_gid = iwch_query_gid;
dev->ibdev.alloc_ucontext = iwch_alloc_ucontext;
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index 66bd6a2..a445e0d 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -390,6 +390,12 @@ static int c4iw_query_port(struct ib_device *ibdev, u8 port,
return 0;
}
+static enum rdma_transport_type
+c4iw_query_transport(struct ib_device *device, u8 port_num)
+{
+ return RDMA_TRANSPORT_IWARP;
+}
+
static ssize_t show_rev(struct device *dev, struct device_attribute *attr,
char *buf)
{
@@ -506,6 +512,7 @@ int c4iw_register_device(struct c4iw_dev *dev)
dev->ibdev.dma_device = &(dev->rdev.lldi.pdev->dev);
dev->ibdev.query_device = c4iw_query_device;
dev->ibdev.query_port = c4iw_query_port;
+ dev->ibdev.query_transport = c4iw_query_transport;
dev->ibdev.query_pkey = c4iw_query_pkey;
dev->ibdev.query_gid = c4iw_query_gid;
dev->ibdev.alloc_ucontext = c4iw_alloc_ucontext;
diff --git a/drivers/infiniband/hw/ehca/ehca_hca.c b/drivers/infiniband/hw/ehca/ehca_hca.c
index 9ed4d25..d5a34a6 100644
--- a/drivers/infiniband/hw/ehca/ehca_hca.c
+++ b/drivers/infiniband/hw/ehca/ehca_hca.c
@@ -242,6 +242,12 @@ query_port1:
return ret;
}
+enum rdma_transport_type
+ehca_query_transport(struct ib_device *device, u8 port_num)
+{
+ return RDMA_TRANSPORT_IB;
+}
+
int ehca_query_sma_attr(struct ehca_shca *shca,
u8 port, struct ehca_sma_attr *attr)
{
diff --git a/drivers/infiniband/hw/ehca/ehca_iverbs.h b/drivers/infiniband/hw/ehca/ehca_iverbs.h
index 22f79af..cec945f 100644
--- a/drivers/infiniband/hw/ehca/ehca_iverbs.h
+++ b/drivers/infiniband/hw/ehca/ehca_iverbs.h
@@ -49,6 +49,9 @@ int ehca_query_device(struct ib_device *ibdev, struct ib_device_attr *props);
int ehca_query_port(struct ib_device *ibdev, u8 port,
struct ib_port_attr *props);
+enum rdma_transport_type
+ehca_query_transport(struct ib_device *device, u8 port_num);
+
int ehca_query_sma_attr(struct ehca_shca *shca, u8 port,
struct ehca_sma_attr *attr);
diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c
index cd8d290..60e0a09 100644
--- a/drivers/infiniband/hw/ehca/ehca_main.c
+++ b/drivers/infiniband/hw/ehca/ehca_main.c
@@ -467,6 +467,7 @@ static int ehca_init_device(struct ehca_shca *shca)
shca->ib_device.dma_device = &shca->ofdev->dev;
shca->ib_device.query_device = ehca_query_device;
shca->ib_device.query_port = ehca_query_port;
+ shca->ib_device.query_transport = ehca_query_transport;
shca->ib_device.query_gid = ehca_query_gid;
shca->ib_device.query_pkey = ehca_query_pkey;
/* shca->in_device.modify_device = ehca_modify_device */
diff --git a/drivers/infiniband/hw/ipath/ipath_verbs.c b/drivers/infiniband/hw/ipath/ipath_verbs.c
index 44ea939..58d36e3 100644
--- a/drivers/infiniband/hw/ipath/ipath_verbs.c
+++ b/drivers/infiniband/hw/ipath/ipath_verbs.c
@@ -1638,6 +1638,12 @@ static int ipath_query_port(struct ib_device *ibdev,
return 0;
}
+static enum rdma_transport_type
+ipath_query_transport(struct ib_device *device, u8 port_num)
+{
+ return RDMA_TRANSPORT_IB;
+}
+
static int ipath_modify_device(struct ib_device *device,
int device_modify_mask,
struct ib_device_modify *device_modify)
@@ -2140,6 +2146,7 @@ int ipath_register_ib_device(struct ipath_devdata *dd)
dev->query_device = ipath_query_device;
dev->modify_device = ipath_modify_device;
dev->query_port = ipath_query_port;
+ dev->query_transport = ipath_query_transport;
dev->modify_port = ipath_modify_port;
dev->query_pkey = ipath_query_pkey;
dev->query_gid = ipath_query_gid;
diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c
index b972c0b..e1424ad 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -420,6 +420,15 @@ static int mlx4_ib_query_port(struct ib_device *ibdev, u8 port,
return __mlx4_ib_query_port(ibdev, port, props, 0);
}
+static enum rdma_transport_type
+mlx4_ib_query_transport(struct ib_device *device, u8 port_num)
+{
+ struct mlx4_dev *dev = to_mdev(device)->dev;
+
+ return dev->caps.port_mask[port_num] == MLX4_PORT_TYPE_IB ?
+ RDMA_TRANSPORT_IB : RDMA_TRANSPORT_IBOE;
+}
+
int __mlx4_ib_query_gid(struct ib_device *ibdev, u8 port, int index,
union ib_gid *gid, int netw_view)
{
@@ -2201,6 +2210,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev)
ibdev->ib_dev.query_device = mlx4_ib_query_device;
ibdev->ib_dev.query_port = mlx4_ib_query_port;
+ ibdev->ib_dev.query_transport = mlx4_ib_query_transport;
ibdev->ib_dev.get_link_layer = mlx4_ib_port_link_layer;
ibdev->ib_dev.query_gid = mlx4_ib_query_gid;
ibdev->ib_dev.query_pkey = mlx4_ib_query_pkey;
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index cc4ac1e..209c796 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -351,6 +351,12 @@ out:
return err;
}
+static enum rdma_transport_type
+mlx5_ib_query_transport(struct ib_device *device, u8 port_num)
+{
+ return RDMA_TRANSPORT_IB;
+}
+
static int mlx5_ib_query_gid(struct ib_device *ibdev, u8 port, int index,
union ib_gid *gid)
{
@@ -1336,6 +1342,7 @@ static void *mlx5_ib_add(struct mlx5_core_dev *mdev)
dev->ib_dev.query_device = mlx5_ib_query_device;
dev->ib_dev.query_port = mlx5_ib_query_port;
+ dev->ib_dev.query_transport = mlx5_ib_query_transport;
dev->ib_dev.query_gid = mlx5_ib_query_gid;
dev->ib_dev.query_pkey = mlx5_ib_query_pkey;
dev->ib_dev.modify_device = mlx5_ib_modify_device;
diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c
index 415f8e1..67ac6a4 100644
--- a/drivers/infiniband/hw/mthca/mthca_provider.c
+++ b/drivers/infiniband/hw/mthca/mthca_provider.c
@@ -179,6 +179,12 @@ static int mthca_query_port(struct ib_device *ibdev,
return err;
}
+static enum rdma_transport_type
+mthca_query_transport(struct ib_device *device, u8 port_num)
+{
+ return RDMA_TRANSPORT_IB;
+}
+
static int mthca_modify_device(struct ib_device *ibdev,
int mask,
struct ib_device_modify *props)
@@ -1281,6 +1287,7 @@ int mthca_register_device(struct mthca_dev *dev)
dev->ib_dev.dma_device = &dev->pdev->dev;
dev->ib_dev.query_device = mthca_query_device;
dev->ib_dev.query_port = mthca_query_port;
+ dev->ib_dev.query_transport = mthca_query_transport;
dev->ib_dev.modify_device = mthca_modify_device;
dev->ib_dev.modify_port = mthca_modify_port;
dev->ib_dev.query_pkey = mthca_query_pkey;
diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c
index c0d0296..8df5b61 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.c
+++ b/drivers/infiniband/hw/nes/nes_verbs.c
@@ -606,6 +606,11 @@ static int nes_query_port(struct ib_device *ibdev, u8 port, struct ib_port_attr
return 0;
}
+static enum rdma_transport_type
+nes_query_transport(struct ib_device *device, u8 port_num)
+{
+ return RDMA_TRANSPORT_IWARP;
+}
/**
* nes_query_pkey
@@ -3879,6 +3884,7 @@ struct nes_ib_device *nes_init_ofa_device(struct net_device *netdev)
nesibdev->ibdev.dev.parent = &nesdev->pcidev->dev;
nesibdev->ibdev.query_device = nes_query_device;
nesibdev->ibdev.query_port = nes_query_port;
+ nesibdev->ibdev.query_transport = nes_query_transport;
nesibdev->ibdev.query_pkey = nes_query_pkey;
nesibdev->ibdev.query_gid = nes_query_gid;
nesibdev->ibdev.alloc_ucontext = nes_alloc_ucontext;
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_main.c b/drivers/infiniband/hw/ocrdma/ocrdma_main.c
index 7a2b59a..9f4d182 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_main.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_main.c
@@ -244,6 +244,7 @@ static int ocrdma_register_device(struct ocrdma_dev *dev)
/* mandatory verbs. */
dev->ibdev.query_device = ocrdma_query_device;
dev->ibdev.query_port = ocrdma_query_port;
+ dev->ibdev.query_transport = ocrdma_query_transport;
dev->ibdev.modify_port = ocrdma_modify_port;
dev->ibdev.query_gid = ocrdma_query_gid;
dev->ibdev.get_link_layer = ocrdma_link_layer;
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
index 8771755..73bace4 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
@@ -187,6 +187,12 @@ int ocrdma_query_port(struct ib_device *ibdev,
return 0;
}
+enum rdma_transport_type
+ocrdma_query_transport(struct ib_device *device, u8 port_num)
+{
+ return RDMA_TRANSPORT_IBOE;
+}
+
int ocrdma_modify_port(struct ib_device *ibdev, u8 port, int mask,
struct ib_port_modify *props)
{
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.h b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.h
index b8f7853..4a81b63 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.h
@@ -41,6 +41,9 @@ int ocrdma_query_port(struct ib_device *, u8 port, struct ib_port_attr *props);
int ocrdma_modify_port(struct ib_device *, u8 port, int mask,
struct ib_port_modify *props);
+enum rdma_transport_type
+ocrdma_query_transport(struct ib_device *device, u8 port_num);
+
void ocrdma_get_guid(struct ocrdma_dev *, u8 *guid);
int ocrdma_query_gid(struct ib_device *, u8 port,
int index, union ib_gid *gid);
diff --git a/drivers/infiniband/hw/qib/qib_verbs.c b/drivers/infiniband/hw/qib/qib_verbs.c
index 4a35998..caad665 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.c
+++ b/drivers/infiniband/hw/qib/qib_verbs.c
@@ -1650,6 +1650,12 @@ static int qib_query_port(struct ib_device *ibdev, u8 port,
return 0;
}
+static enum rdma_transport_type
+qib_query_transport(struct ib_device *device, u8 port_num)
+{
+ return RDMA_TRANSPORT_IB;
+}
+
static int qib_modify_device(struct ib_device *device,
int device_modify_mask,
struct ib_device_modify *device_modify)
@@ -2184,6 +2190,7 @@ int qib_register_ib_device(struct qib_devdata *dd)
ibdev->query_device = qib_query_device;
ibdev->modify_device = qib_modify_device;
ibdev->query_port = qib_query_port;
+ ibdev->query_transport = qib_query_transport;
ibdev->modify_port = qib_modify_port;
ibdev->query_pkey = qib_query_pkey;
ibdev->query_gid = qib_query_gid;
diff --git a/drivers/infiniband/hw/usnic/usnic_ib_main.c b/drivers/infiniband/hw/usnic/usnic_ib_main.c
index 0d0f986..03ea9f3 100644
--- a/drivers/infiniband/hw/usnic/usnic_ib_main.c
+++ b/drivers/infiniband/hw/usnic/usnic_ib_main.c
@@ -360,6 +360,7 @@ static void *usnic_ib_device_add(struct pci_dev *dev)
us_ibdev->ib_dev.query_device = usnic_ib_query_device;
us_ibdev->ib_dev.query_port = usnic_ib_query_port;
+ us_ibdev->ib_dev.query_transport = usnic_ib_query_transport;
us_ibdev->ib_dev.query_pkey = usnic_ib_query_pkey;
us_ibdev->ib_dev.query_gid = usnic_ib_query_gid;
us_ibdev->ib_dev.get_link_layer = usnic_ib_port_link_layer;
diff --git a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
index 53bd6a2..ff9a5f7 100644
--- a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
+++ b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c
@@ -348,6 +348,12 @@ int usnic_ib_query_port(struct ib_device *ibdev, u8 port,
return 0;
}
+enum rdma_transport_type
+usnic_ib_query_transport(struct ib_device *device, u8 port_num)
+{
+ return RDMA_TRANSPORT_USNIC_UDP;
+}
+
int usnic_ib_query_qp(struct ib_qp *qp, struct ib_qp_attr *qp_attr,
int qp_attr_mask,
struct ib_qp_init_attr *qp_init_attr)
diff --git a/drivers/infiniband/hw/usnic/usnic_ib_verbs.h b/drivers/infiniband/hw/usnic/usnic_ib_verbs.h
index bb864f5..0b1633b 100644
--- a/drivers/infiniband/hw/usnic/usnic_ib_verbs.h
+++ b/drivers/infiniband/hw/usnic/usnic_ib_verbs.h
@@ -27,6 +27,8 @@ int usnic_ib_query_device(struct ib_device *ibdev,
struct ib_device_attr *props);
int usnic_ib_query_port(struct ib_device *ibdev, u8 port,
struct ib_port_attr *props);
+enum rdma_transport_type
+usnic_ib_query_transport(struct ib_device *device, u8 port_num);
int usnic_ib_query_qp(struct ib_qp *qp, struct ib_qp_attr *qp_attr,
int qp_attr_mask,
struct ib_qp_init_attr *qp_init_attr);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 65994a1..d54f91e 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -75,10 +75,13 @@ enum rdma_node_type {
};
enum rdma_transport_type {
+ /* legacy for users */
RDMA_TRANSPORT_IB,
RDMA_TRANSPORT_IWARP,
RDMA_TRANSPORT_USNIC,
- RDMA_TRANSPORT_USNIC_UDP
+ RDMA_TRANSPORT_USNIC_UDP,
+ /* new transport */
+ RDMA_TRANSPORT_IBOE,
};
__attribute_const__ enum rdma_transport_type
@@ -1501,6 +1504,8 @@ struct ib_device {
int (*query_port)(struct ib_device *device,
u8 port_num,
struct ib_port_attr *port_attr);
+ enum rdma_transport_type (*query_transport)(struct ib_device *device,
+ u8 port_num);
enum rdma_link_layer (*get_link_layer)(struct ib_device *device,
u8 port_num);
int (*query_gid)(struct ib_device *device,
--
2.1.0
Add raw helpers:
rdma_tech_ib
rdma_tech_iboe
rdma_tech_iwarp
rdma_ib_or_iboe (transition, clean up later)
To help us detect which technology the port supported.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
include/rdma/ib_verbs.h | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index d54f91e..a12e876 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1748,6 +1748,31 @@ int ib_query_port(struct ib_device *device,
enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device,
u8 port_num);
+static inline int rdma_tech_ib(struct ib_device *device, u8 port_num)
+{
+ return device->query_transport(device, port_num)
+ == RDMA_TRANSPORT_IB;
+}
+
+static inline int rdma_tech_iboe(struct ib_device *device, u8 port_num)
+{
+ return device->query_transport(device, port_num)
+ == RDMA_TRANSPORT_IBOE;
+}
+
+static inline int rdma_tech_iwarp(struct ib_device *device, u8 port_num)
+{
+ return device->query_transport(device, port_num)
+ == RDMA_TRANSPORT_IWARP;
+}
+
+static inline int rdma_ib_or_iboe(struct ib_device *device, u8 port_num)
+{
+ enum rdma_transport_type tp = device->query_transport(device, port_num);
+
+ return (tp == RDMA_TRANSPORT_IB || tp == RDMA_TRANSPORT_IBOE);
+}
+
int ib_query_gid(struct ib_device *device,
u8 port_num, int index, union ib_gid *gid);
--
2.1.0
Use raw management helpers to reform IB-core mad/agent/user_mad.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/agent.c | 2 +-
drivers/infiniband/core/mad.c | 20 ++++++++++----------
drivers/infiniband/core/user_mad.c | 26 ++++++++++++++++++++------
3 files changed, 31 insertions(+), 17 deletions(-)
diff --git a/drivers/infiniband/core/agent.c b/drivers/infiniband/core/agent.c
index f6d2961..ffdef4d 100644
--- a/drivers/infiniband/core/agent.c
+++ b/drivers/infiniband/core/agent.c
@@ -156,7 +156,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num)
goto error1;
}
- if (rdma_port_get_link_layer(device, port_num) == IB_LINK_LAYER_INFINIBAND) {
+ if (rdma_tech_ib(device, port_num)) {
/* Obtain send only MAD agent for SMI QP */
port_priv->agent[0] = ib_register_mad_agent(device, port_num,
IB_QPT_SMI, NULL, 0,
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 74c30f4..d451a47 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,
init_mad_qp(port_priv, &port_priv->qp_info[1]);
cq_size = mad_sendq_size + mad_recvq_size;
- has_smi = rdma_port_get_link_layer(device, port_num) == IB_LINK_LAYER_INFINIBAND;
+ has_smi = rdma_tech_ib(device, port_num);
if (has_smi)
cq_size *= 2;
@@ -3057,9 +3057,6 @@ static void ib_mad_init_device(struct ib_device *device)
{
int start, end, i;
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
- return;
-
if (device->node_type == RDMA_NODE_IB_SWITCH) {
start = 0;
end = 0;
@@ -3069,6 +3066,9 @@ static void ib_mad_init_device(struct ib_device *device)
}
for (i = start; i <= end; i++) {
+ if (!rdma_ib_or_iboe(device, i))
+ continue;
+
if (ib_mad_port_open(device, i)) {
dev_err(&device->dev, "Couldn't open port %d\n", i);
goto error;
@@ -3086,15 +3086,15 @@ error_agent:
dev_err(&device->dev, "Couldn't close port %d\n", i);
error:
- i--;
+ while (--i >= start) {
+ if (!rdma_ib_or_iboe(device, i))
+ continue;
- while (i >= start) {
if (ib_agent_port_close(device, i))
dev_err(&device->dev,
"Couldn't close port %d for agents\n", i);
if (ib_mad_port_close(device, i))
dev_err(&device->dev, "Couldn't close port %d\n", i);
- i--;
}
}
@@ -3102,9 +3102,6 @@ static void ib_mad_remove_device(struct ib_device *device)
{
int i, num_ports, cur_port;
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
- return;
-
if (device->node_type == RDMA_NODE_IB_SWITCH) {
num_ports = 1;
cur_port = 0;
@@ -3113,6 +3110,9 @@ static void ib_mad_remove_device(struct ib_device *device)
cur_port = 1;
}
for (i = 0; i < num_ports; i++, cur_port++) {
+ if (!rdma_ib_or_iboe(device, i))
+ continue;
+
if (ib_agent_port_close(device, cur_port))
dev_err(&device->dev,
"Couldn't close port %d for agents\n",
diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c
index 928cdd2..71fc8ba 100644
--- a/drivers/infiniband/core/user_mad.c
+++ b/drivers/infiniband/core/user_mad.c
@@ -1273,9 +1273,7 @@ static void ib_umad_add_one(struct ib_device *device)
{
struct ib_umad_device *umad_dev;
int s, e, i;
-
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
- return;
+ int count = 0;
if (device->node_type == RDMA_NODE_IB_SWITCH)
s = e = 0;
@@ -1296,11 +1294,21 @@ static void ib_umad_add_one(struct ib_device *device)
umad_dev->end_port = e;
for (i = s; i <= e; ++i) {
+ if (!rdma_ib_or_iboe(device, i))
+ continue;
+
umad_dev->port[i - s].umad_dev = umad_dev;
if (ib_umad_init_port(device, i, umad_dev,
&umad_dev->port[i - s]))
goto err;
+
+ count++;
+ }
+
+ if (!count) {
+ kobject_put(&umad_dev->kobj);
+ return;
}
ib_set_client_data(device, &umad_client, umad_dev);
@@ -1308,8 +1316,12 @@ static void ib_umad_add_one(struct ib_device *device)
return;
err:
- while (--i >= s)
+ while (--i >= s) {
+ if (!rdma_ib_or_iboe(device, i))
+ continue;
+
ib_umad_kill_port(&umad_dev->port[i - s]);
+ }
kobject_put(&umad_dev->kobj);
}
@@ -1322,8 +1334,10 @@ static void ib_umad_remove_one(struct ib_device *device)
if (!umad_dev)
return;
- for (i = 0; i <= umad_dev->end_port - umad_dev->start_port; ++i)
- ib_umad_kill_port(&umad_dev->port[i]);
+ for (i = 0; i <= umad_dev->end_port - umad_dev->start_port; ++i) {
+ if (rdma_ib_or_iboe(device, i))
+ ib_umad_kill_port(&umad_dev->port[i]);
+ }
kobject_put(&umad_dev->kobj);
}
--
2.1.0
Use raw management helpers to reform IB-core cm.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cm.c | 22 +++++++++++++++++++---
1 file changed, 19 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index e28a494..50321fe 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -3761,9 +3761,7 @@ static void cm_add_one(struct ib_device *ib_device)
unsigned long flags;
int ret;
u8 i;
-
- if (rdma_node_get_transport(ib_device->node_type) != RDMA_TRANSPORT_IB)
- return;
+ int count = 0;
cm_dev = kzalloc(sizeof(*cm_dev) + sizeof(*port) *
ib_device->phys_port_cnt, GFP_KERNEL);
@@ -3783,6 +3781,9 @@ static void cm_add_one(struct ib_device *ib_device)
set_bit(IB_MGMT_METHOD_SEND, reg_req.method_mask);
for (i = 1; i <= ib_device->phys_port_cnt; i++) {
+ if (!rdma_ib_or_iboe(ib_device, i))
+ continue;
+
port = kzalloc(sizeof *port, GFP_KERNEL);
if (!port)
goto error1;
@@ -3809,7 +3810,16 @@ static void cm_add_one(struct ib_device *ib_device)
ret = ib_modify_port(ib_device, i, 0, &port_modify);
if (ret)
goto error3;
+
+ count++;
}
+
+ if (!count) {
+ device_unregister(cm_dev->device);
+ kfree(cm_dev);
+ return;
+ }
+
ib_set_client_data(ib_device, &cm_client, cm_dev);
write_lock_irqsave(&cm.device_lock, flags);
@@ -3825,6 +3835,9 @@ error1:
port_modify.set_port_cap_mask = 0;
port_modify.clr_port_cap_mask = IB_PORT_CM_SUP;
while (--i) {
+ if (!rdma_ib_or_iboe(ib_device, i))
+ continue;
+
port = cm_dev->port[i-1];
ib_modify_port(ib_device, port->port_num, 0, &port_modify);
ib_unregister_mad_agent(port->mad_agent);
@@ -3853,6 +3866,9 @@ static void cm_remove_one(struct ib_device *ib_device)
write_unlock_irqrestore(&cm.device_lock, flags);
for (i = 1; i <= ib_device->phys_port_cnt; i++) {
+ if (!rdma_ib_or_iboe(ib_device, i))
+ continue;
+
port = cm_dev->port[i-1];
ib_modify_port(ib_device, port->port_num, 0, &port_modify);
ib_unregister_mad_agent(port->mad_agent);
--
2.1.0
Use raw management helpers to reform IB-core sa_query.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/sa_query.c | 29 ++++++++++++++++++-----------
1 file changed, 18 insertions(+), 11 deletions(-)
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index c38f030..803ccf7 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *event
struct ib_sa_port *port =
&sa_dev->port[event->element.port_num - sa_dev->start_port];
- if (rdma_port_get_link_layer(handler->device, port->port_num) != IB_LINK_LAYER_INFINIBAND)
+ if (WARN_ON(!rdma_tech_ib(handler->device, port->port_num)))
return;
spin_lock_irqsave(&port->ah_lock, flags);
@@ -540,7 +540,7 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num,
ah_attr->port_num = port_num;
ah_attr->static_rate = rec->rate;
- force_grh = rdma_port_get_link_layer(device, port_num) == IB_LINK_LAYER_ETHERNET;
+ force_grh = rdma_tech_iboe(device, port_num);
if (rec->hop_limit > 1 || force_grh) {
ah_attr->ah_flags = IB_AH_GRH;
@@ -1153,9 +1153,7 @@ static void ib_sa_add_one(struct ib_device *device)
{
struct ib_sa_device *sa_dev;
int s, e, i;
-
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
- return;
+ int count = 0;
if (device->node_type == RDMA_NODE_IB_SWITCH)
s = e = 0;
@@ -1175,7 +1173,7 @@ static void ib_sa_add_one(struct ib_device *device)
for (i = 0; i <= e - s; ++i) {
spin_lock_init(&sa_dev->port[i].ah_lock);
- if (rdma_port_get_link_layer(device, i + 1) != IB_LINK_LAYER_INFINIBAND)
+ if (!rdma_tech_ib(device, i + 1))
continue;
sa_dev->port[i].sm_ah = NULL;
@@ -1189,6 +1187,13 @@ static void ib_sa_add_one(struct ib_device *device)
goto err;
INIT_WORK(&sa_dev->port[i].update_task, update_sm_ah);
+
+ count++;
+ }
+
+ if (!count) {
+ kfree(sa_dev);
+ return;
}
ib_set_client_data(device, &sa_client, sa_dev);
@@ -1204,16 +1209,18 @@ static void ib_sa_add_one(struct ib_device *device)
if (ib_register_event_handler(&sa_dev->event_handler))
goto err;
- for (i = 0; i <= e - s; ++i)
- if (rdma_port_get_link_layer(device, i + 1) == IB_LINK_LAYER_INFINIBAND)
+ for (i = 0; i <= e - s; ++i) {
+ if (rdma_tech_ib(device, i + 1))
update_sm_ah(&sa_dev->port[i].update_task);
+ }
return;
err:
- while (--i >= 0)
- if (rdma_port_get_link_layer(device, i + 1) == IB_LINK_LAYER_INFINIBAND)
+ while (--i >= 0) {
+ if (rdma_tech_ib(device, i + 1))
ib_unregister_mad_agent(sa_dev->port[i].agent);
+ }
kfree(sa_dev);
@@ -1233,7 +1240,7 @@ static void ib_sa_remove_one(struct ib_device *device)
flush_workqueue(ib_wq);
for (i = 0; i <= sa_dev->end_port - sa_dev->start_port; ++i) {
- if (rdma_port_get_link_layer(device, i + 1) == IB_LINK_LAYER_INFINIBAND) {
+ if (rdma_tech_ib(device, i + 1)) {
ib_unregister_mad_agent(sa_dev->port[i].agent);
if (sa_dev->port[i].sm_ah)
kref_put(&sa_dev->port[i].sm_ah->ref, free_sm_ah);
--
2.1.0
Use raw management helpers to reform IB-core multicast.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/multicast.c | 12 +++---------
1 file changed, 3 insertions(+), 9 deletions(-)
diff --git a/drivers/infiniband/core/multicast.c b/drivers/infiniband/core/multicast.c
index fa17b55..24d93f5 100644
--- a/drivers/infiniband/core/multicast.c
+++ b/drivers/infiniband/core/multicast.c
@@ -780,8 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,
int index;
dev = container_of(handler, struct mcast_device, event_handler);
- if (rdma_port_get_link_layer(dev->device, event->element.port_num) !=
- IB_LINK_LAYER_INFINIBAND)
+ if (WARN_ON(!rdma_tech_ib(dev->device, event->element.port_num)))
return;
index = event->element.port_num - dev->start_port;
@@ -808,9 +807,6 @@ static void mcast_add_one(struct ib_device *device)
int i;
int count = 0;
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
- return;
-
dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,
GFP_KERNEL);
if (!dev)
@@ -824,8 +820,7 @@ static void mcast_add_one(struct ib_device *device)
}
for (i = 0; i <= dev->end_port - dev->start_port; i++) {
- if (rdma_port_get_link_layer(device, dev->start_port + i) !=
- IB_LINK_LAYER_INFINIBAND)
+ if (!rdma_tech_ib(device, dev->start_port + i))
continue;
port = &dev->port[i];
port->dev = dev;
@@ -863,8 +858,7 @@ static void mcast_remove_one(struct ib_device *device)
flush_workqueue(mcast_wq);
for (i = 0; i <= dev->end_port - dev->start_port; i++) {
- if (rdma_port_get_link_layer(device, dev->start_port + i) ==
- IB_LINK_LAYER_INFINIBAND) {
+ if (rdma_tech_ib(device, dev->start_port + i)) {
port = &dev->port[i];
deref_port(port);
wait_for_completion(&port->comp);
--
2.1.0
Use raw management helpers to reform IB-ulp ipoib.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/ulp/ipoib/ipoib_main.c | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 58b5aa3..60b379d 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -1654,9 +1654,7 @@ static void ipoib_add_one(struct ib_device *device)
struct net_device *dev;
struct ipoib_dev_priv *priv;
int s, e, p;
-
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
- return;
+ int count = 0;
dev_list = kmalloc(sizeof *dev_list, GFP_KERNEL);
if (!dev_list)
@@ -1673,15 +1671,21 @@ static void ipoib_add_one(struct ib_device *device)
}
for (p = s; p <= e; ++p) {
- if (rdma_port_get_link_layer(device, p) != IB_LINK_LAYER_INFINIBAND)
+ if (!rdma_tech_ib(device, p))
continue;
dev = ipoib_add_port("ib%d", device, p);
if (!IS_ERR(dev)) {
priv = netdev_priv(dev);
list_add_tail(&priv->list, dev_list);
+ count++;
}
}
+ if (!count) {
+ kfree(dev_list);
+ return;
+ }
+
ib_set_client_data(device, &ipoib_client, dev_list);
}
@@ -1690,9 +1694,6 @@ static void ipoib_remove_one(struct ib_device *device)
struct ipoib_dev_priv *priv, *tmp;
struct list_head *dev_list;
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
- return;
-
dev_list = ib_get_client_data(device, &ipoib_client);
if (!dev_list)
return;
--
2.1.0
Use raw management helpers to reform IB-ulp xprtrdma.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 3 +--
net/sunrpc/xprtrdma/svc_rdma_transport.c | 45 +++++++++++++-------------------
2 files changed, 19 insertions(+), 29 deletions(-)
diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
index f9f13a3..a5bed5b 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
@@ -117,8 +117,7 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,
static int rdma_read_max_sge(struct svcxprt_rdma *xprt, int sge_count)
{
- if (rdma_node_get_transport(xprt->sc_cm_id->device->node_type) ==
- RDMA_TRANSPORT_IWARP)
+ if (rdma_tech_iwarp(xprt->sc_cm_id->device, xprt->sc_cm_id->port_num))
return 1;
else
return min_t(int, sge_count, xprt->sc_max_sge);
diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c
index f609c1c..a09b7a1 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_transport.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c
@@ -851,7 +851,7 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)
struct ib_qp_init_attr qp_attr;
struct ib_device_attr devattr;
int uninitialized_var(dma_mr_acc);
- int need_dma_mr;
+ int need_dma_mr = 0;
int ret;
int i;
@@ -985,35 +985,26 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)
/*
* Determine if a DMA MR is required and if so, what privs are required
*/
- switch (rdma_node_get_transport(newxprt->sc_cm_id->device->node_type)) {
- case RDMA_TRANSPORT_IWARP:
- newxprt->sc_dev_caps |= SVCRDMA_DEVCAP_READ_W_INV;
- if (!(newxprt->sc_dev_caps & SVCRDMA_DEVCAP_FAST_REG)) {
- need_dma_mr = 1;
- dma_mr_acc =
- (IB_ACCESS_LOCAL_WRITE |
- IB_ACCESS_REMOTE_WRITE);
- } else if (!(devattr.device_cap_flags & IB_DEVICE_LOCAL_DMA_LKEY)) {
- need_dma_mr = 1;
- dma_mr_acc = IB_ACCESS_LOCAL_WRITE;
- } else
- need_dma_mr = 0;
- break;
- case RDMA_TRANSPORT_IB:
- if (!(newxprt->sc_dev_caps & SVCRDMA_DEVCAP_FAST_REG)) {
- need_dma_mr = 1;
- dma_mr_acc = IB_ACCESS_LOCAL_WRITE;
- } else if (!(devattr.device_cap_flags &
- IB_DEVICE_LOCAL_DMA_LKEY)) {
- need_dma_mr = 1;
- dma_mr_acc = IB_ACCESS_LOCAL_WRITE;
- } else
- need_dma_mr = 0;
- break;
- default:
+ if (!rdma_tech_iwarp(newxprt->sc_cm_id->device,
+ newxprt->sc_cm_id->port_num) &&
+ !rdma_ib_or_iboe(newxprt->sc_cm_id->device,
+ newxprt->sc_cm_id->port_num))
goto errout;
+
+ if (!(newxprt->sc_dev_caps & SVCRDMA_DEVCAP_FAST_REG) ||
+ !(devattr.device_cap_flags & IB_DEVICE_LOCAL_DMA_LKEY)) {
+ need_dma_mr = 1;
+ dma_mr_acc = IB_ACCESS_LOCAL_WRITE;
+ if (rdma_tech_iwarp(newxprt->sc_cm_id->device,
+ newxprt->sc_cm_id->port_num) &&
+ !(newxprt->sc_dev_caps & SVCRDMA_DEVCAP_FAST_REG))
+ dma_mr_acc |= IB_ACCESS_REMOTE_WRITE;
}
+ if (rdma_tech_iwarp(newxprt->sc_cm_id->device,
+ newxprt->sc_cm_id->port_num))
+ newxprt->sc_dev_caps |= SVCRDMA_DEVCAP_READ_W_INV;
+
/* Create the DMA MR if needed, otherwise, use the DMA LKEY */
if (need_dma_mr) {
/* Register all of physical memory */
--
2.1.0
Use raw management helpers to reform IB-core verbs/uverbs_cmd/sysfs.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/sysfs.c | 8 ++------
drivers/infiniband/core/uverbs_cmd.c | 6 ++++--
drivers/infiniband/core/verbs.c | 6 ++----
3 files changed, 8 insertions(+), 12 deletions(-)
diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c
index cbd0383..8570180 100644
--- a/drivers/infiniband/core/sysfs.c
+++ b/drivers/infiniband/core/sysfs.c
@@ -248,14 +248,10 @@ static ssize_t phys_state_show(struct ib_port *p, struct port_attribute *unused,
static ssize_t link_layer_show(struct ib_port *p, struct port_attribute *unused,
char *buf)
{
- switch (rdma_port_get_link_layer(p->ibdev, p->port_num)) {
- case IB_LINK_LAYER_INFINIBAND:
+ if (rdma_tech_ib(p->ibdev, p->port_num))
return sprintf(buf, "%s\n", "InfiniBand");
- case IB_LINK_LAYER_ETHERNET:
+ else
return sprintf(buf, "%s\n", "Ethernet");
- default:
- return sprintf(buf, "%s\n", "Unknown");
- }
}
static PORT_ATTR_RO(state);
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index a9f0489..5dc90aa 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -515,8 +515,10 @@ ssize_t ib_uverbs_query_port(struct ib_uverbs_file *file,
resp.active_width = attr.active_width;
resp.active_speed = attr.active_speed;
resp.phys_state = attr.phys_state;
- resp.link_layer = rdma_port_get_link_layer(file->device->ib_dev,
- cmd.port_num);
+ resp.link_layer = rdma_tech_ib(file->device->ib_dev,
+ cmd.port_num) ?
+ IB_LINK_LAYER_INFINIBAND :
+ IB_LINK_LAYER_ETHERNET;
if (copy_to_user((void __user *) (unsigned long) cmd.response,
&resp, sizeof resp))
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 626c9cf..7264860 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -200,11 +200,9 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 port_num, struct ib_wc *wc,
u32 flow_class;
u16 gid_index;
int ret;
- int is_eth = (rdma_port_get_link_layer(device, port_num) ==
- IB_LINK_LAYER_ETHERNET);
memset(ah_attr, 0, sizeof *ah_attr);
- if (is_eth) {
+ if (rdma_tech_iboe(device, port_num)) {
if (!(wc->wc_flags & IB_WC_GRH))
return -EPROTOTYPE;
@@ -873,7 +871,7 @@ int ib_resolve_eth_l2_attrs(struct ib_qp *qp,
union ib_gid sgid;
if ((*qp_attr_mask & IB_QP_AV) &&
- (rdma_port_get_link_layer(qp->device, qp_attr->ah_attr.port_num) == IB_LINK_LAYER_ETHERNET)) {
+ (rdma_tech_iboe(qp->device, qp_attr->ah_attr.port_num))) {
ret = ib_query_gid(qp->device, qp_attr->ah_attr.port_num,
qp_attr->ah_attr.grh.sgid_index, &sgid);
if (ret)
--
2.1.0
Use raw management helpers to reform cm related part in IB-core cma/ucm.
These checks focus on the device cm type rather than the port capability,
directly pass port 1 works currently, but can't support mixing cm type
device in future.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cma.c | 81 +++++++++++++------------------------------
drivers/infiniband/core/ucm.c | 3 +-
2 files changed, 26 insertions(+), 58 deletions(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index d570030..6b8a64d 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -735,8 +735,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
int ret = 0;
id_priv = container_of(id, struct rdma_id_private, id);
- switch (rdma_node_get_transport(id_priv->id.device->node_type)) {
- case RDMA_TRANSPORT_IB:
+ if (rdma_ib_or_iboe(id_priv->id.device, 1)) {
if (!id_priv->cm_id.ib || (id_priv->id.qp_type == IB_QPT_UD))
ret = cma_ib_init_qp_attr(id_priv, qp_attr, qp_attr_mask);
else
@@ -745,19 +744,15 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
if (qp_attr->qp_state == IB_QPS_RTR)
qp_attr->rq_psn = id_priv->seq_num;
- break;
- case RDMA_TRANSPORT_IWARP:
+ } else if (rdma_tech_iwarp(id_priv->id.device, 1)) {
if (!id_priv->cm_id.iw) {
qp_attr->qp_access_flags = 0;
*qp_attr_mask = IB_QP_STATE | IB_QP_ACCESS_FLAGS;
} else
ret = iw_cm_init_qp_attr(id_priv->cm_id.iw, qp_attr,
qp_attr_mask);
- break;
- default:
+ } else
ret = -ENOSYS;
- break;
- }
return ret;
}
@@ -1037,17 +1032,12 @@ void rdma_destroy_id(struct rdma_cm_id *id)
mutex_unlock(&id_priv->handler_mutex);
if (id_priv->cma_dev) {
- switch (rdma_node_get_transport(id_priv->id.device->node_type)) {
- case RDMA_TRANSPORT_IB:
+ if (rdma_ib_or_iboe(id_priv->id.device, 1)) {
if (id_priv->cm_id.ib)
ib_destroy_cm_id(id_priv->cm_id.ib);
- break;
- case RDMA_TRANSPORT_IWARP:
+ } else if (rdma_tech_iwarp(id_priv->id.device, 1)) {
if (id_priv->cm_id.iw)
iw_destroy_cm_id(id_priv->cm_id.iw);
- break;
- default:
- break;
}
cma_leave_mc_groups(id_priv);
cma_release_dev(id_priv);
@@ -1626,7 +1616,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,
int ret;
if (cma_family(id_priv) == AF_IB &&
- rdma_node_get_transport(cma_dev->device->node_type) != RDMA_TRANSPORT_IB)
+ !rdma_ib_or_iboe(cma_dev->device, 1))
return;
id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,
@@ -2028,7 +2018,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)
mutex_lock(&lock);
list_for_each_entry(cur_dev, &dev_list, list) {
if (cma_family(id_priv) == AF_IB &&
- rdma_node_get_transport(cur_dev->device->node_type) != RDMA_TRANSPORT_IB)
+ !rdma_ib_or_iboe(cur_dev->device, 1))
continue;
if (!cma_dev)
@@ -2060,7 +2050,7 @@ port_found:
goto out;
id_priv->id.route.addr.dev_addr.dev_type =
- (rdma_port_get_link_layer(cma_dev->device, p) == IB_LINK_LAYER_INFINIBAND) ?
+ (rdma_tech_ib(cma_dev->device, p)) ?
ARPHRD_INFINIBAND : ARPHRD_ETHER;
rdma_addr_set_sgid(&id_priv->id.route.addr.dev_addr, &gid);
@@ -2537,18 +2527,15 @@ int rdma_listen(struct rdma_cm_id *id, int backlog)
id_priv->backlog = backlog;
if (id->device) {
- switch (rdma_node_get_transport(id->device->node_type)) {
- case RDMA_TRANSPORT_IB:
+ if (rdma_ib_or_iboe(id->device, 1)) {
ret = cma_ib_listen(id_priv);
if (ret)
goto err;
- break;
- case RDMA_TRANSPORT_IWARP:
+ } else if (rdma_tech_iwarp(id->device, 1)) {
ret = cma_iw_listen(id_priv, backlog);
if (ret)
goto err;
- break;
- default:
+ } else {
ret = -ENOSYS;
goto err;
}
@@ -2884,20 +2871,15 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
id_priv->srq = conn_param->srq;
}
- switch (rdma_node_get_transport(id->device->node_type)) {
- case RDMA_TRANSPORT_IB:
+ if (rdma_ib_or_iboe(id->device, 1)) {
if (id->qp_type == IB_QPT_UD)
ret = cma_resolve_ib_udp(id_priv, conn_param);
else
ret = cma_connect_ib(id_priv, conn_param);
- break;
- case RDMA_TRANSPORT_IWARP:
+ } else if (rdma_tech_iwarp(id->device, 1))
ret = cma_connect_iw(id_priv, conn_param);
- break;
- default:
+ else
ret = -ENOSYS;
- break;
- }
if (ret)
goto err;
@@ -3000,8 +2982,7 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
id_priv->srq = conn_param->srq;
}
- switch (rdma_node_get_transport(id->device->node_type)) {
- case RDMA_TRANSPORT_IB:
+ if (rdma_ib_or_iboe(id->device, 1)) {
if (id->qp_type == IB_QPT_UD) {
if (conn_param)
ret = cma_send_sidr_rep(id_priv, IB_SIDR_SUCCESS,
@@ -3017,14 +2998,10 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
else
ret = cma_rep_recv(id_priv);
}
- break;
- case RDMA_TRANSPORT_IWARP:
+ } else if (rdma_tech_iwarp(id->device, 1))
ret = cma_accept_iw(id_priv, conn_param);
- break;
- default:
+ else
ret = -ENOSYS;
- break;
- }
if (ret)
goto reject;
@@ -3068,8 +3045,7 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,
if (!id_priv->cm_id.ib)
return -EINVAL;
- switch (rdma_node_get_transport(id->device->node_type)) {
- case RDMA_TRANSPORT_IB:
+ if (rdma_ib_or_iboe(id->device, 1)) {
if (id->qp_type == IB_QPT_UD)
ret = cma_send_sidr_rep(id_priv, IB_SIDR_REJECT, 0,
private_data, private_data_len);
@@ -3077,15 +3053,12 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,
ret = ib_send_cm_rej(id_priv->cm_id.ib,
IB_CM_REJ_CONSUMER_DEFINED, NULL,
0, private_data, private_data_len);
- break;
- case RDMA_TRANSPORT_IWARP:
+ } else if (rdma_tech_iwarp(id->device, 1)) {
ret = iw_cm_reject(id_priv->cm_id.iw,
private_data, private_data_len);
- break;
- default:
+ } else
ret = -ENOSYS;
- break;
- }
+
return ret;
}
EXPORT_SYMBOL(rdma_reject);
@@ -3099,22 +3072,18 @@ int rdma_disconnect(struct rdma_cm_id *id)
if (!id_priv->cm_id.ib)
return -EINVAL;
- switch (rdma_node_get_transport(id->device->node_type)) {
- case RDMA_TRANSPORT_IB:
+ if (rdma_ib_or_iboe(id->device, 1)) {
ret = cma_modify_qp_err(id_priv);
if (ret)
goto out;
/* Initiate or respond to a disconnect. */
if (ib_send_cm_dreq(id_priv->cm_id.ib, NULL, 0))
ib_send_cm_drep(id_priv->cm_id.ib, NULL, 0);
- break;
- case RDMA_TRANSPORT_IWARP:
+ } else if (rdma_tech_iwarp(id->device, 1)) {
ret = iw_cm_disconnect(id_priv->cm_id.iw, 0);
- break;
- default:
+ } else
ret = -EINVAL;
- break;
- }
+
out:
return ret;
}
diff --git a/drivers/infiniband/core/ucm.c b/drivers/infiniband/core/ucm.c
index f2f6393..70e0ccb 100644
--- a/drivers/infiniband/core/ucm.c
+++ b/drivers/infiniband/core/ucm.c
@@ -1253,8 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)
dev_t base;
struct ib_ucm_device *ucm_dev;
- if (!device->alloc_ucontext ||
- rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
+ if (!device->alloc_ucontext || !rdma_ib_or_iboe(device, 1))
return;
ucm_dev = kzalloc(sizeof *ucm_dev, GFP_KERNEL);
--
2.1.0
Use raw management helpers to reform route related part in IB-core cma.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cma.c | 31 ++++++++-----------------------
drivers/infiniband/core/ucma.c | 25 ++++++-------------------
2 files changed, 14 insertions(+), 42 deletions(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 6b8a64d..4adb909 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -923,13 +923,9 @@ static inline int cma_user_data_offset(struct rdma_id_private *id_priv)
static void cma_cancel_route(struct rdma_id_private *id_priv)
{
- switch (rdma_port_get_link_layer(id_priv->id.device, id_priv->id.port_num)) {
- case IB_LINK_LAYER_INFINIBAND:
+ if (rdma_tech_ib(id_priv->id.device, id_priv->id.port_num)) {
if (id_priv->query)
ib_sa_cancel_query(id_priv->query_id, id_priv->query);
- break;
- default:
- break;
}
}
@@ -1957,26 +1953,15 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)
return -EINVAL;
atomic_inc(&id_priv->refcount);
- switch (rdma_node_get_transport(id->device->node_type)) {
- case RDMA_TRANSPORT_IB:
- switch (rdma_port_get_link_layer(id->device, id->port_num)) {
- case IB_LINK_LAYER_INFINIBAND:
- ret = cma_resolve_ib_route(id_priv, timeout_ms);
- break;
- case IB_LINK_LAYER_ETHERNET:
- ret = cma_resolve_iboe_route(id_priv);
- break;
- default:
- ret = -ENOSYS;
- }
- break;
- case RDMA_TRANSPORT_IWARP:
+ if (rdma_tech_ib(id->device, id->port_num))
+ ret = cma_resolve_ib_route(id_priv, timeout_ms);
+ else if (rdma_tech_iboe(id->device, id->port_num))
+ ret = cma_resolve_iboe_route(id_priv);
+ else if (rdma_tech_iwarp(id->device, id->port_num))
ret = cma_resolve_iw_route(id_priv, timeout_ms);
- break;
- default:
+ else
ret = -ENOSYS;
- break;
- }
+
if (ret)
goto err;
diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index 45d67e9..7331c6c 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -722,26 +722,13 @@ static ssize_t ucma_query_route(struct ucma_file *file,
resp.node_guid = (__force __u64) ctx->cm_id->device->node_guid;
resp.port_num = ctx->cm_id->port_num;
- switch (rdma_node_get_transport(ctx->cm_id->device->node_type)) {
- case RDMA_TRANSPORT_IB:
- switch (rdma_port_get_link_layer(ctx->cm_id->device,
- ctx->cm_id->port_num)) {
- case IB_LINK_LAYER_INFINIBAND:
- ucma_copy_ib_route(&resp, &ctx->cm_id->route);
- break;
- case IB_LINK_LAYER_ETHERNET:
- ucma_copy_iboe_route(&resp, &ctx->cm_id->route);
- break;
- default:
- break;
- }
- break;
- case RDMA_TRANSPORT_IWARP:
+
+ if (rdma_tech_ib(ctx->cm_id->device, ctx->cm_id->port_num))
+ ucma_copy_ib_route(&resp, &ctx->cm_id->route);
+ else if (rdma_tech_iboe(ctx->cm_id->device, ctx->cm_id->port_num))
+ ucma_copy_iboe_route(&resp, &ctx->cm_id->route);
+ else if (rdma_tech_iwarp(ctx->cm_id->device, ctx->cm_id->port_num))
ucma_copy_iw_route(&resp, &ctx->cm_id->route);
- break;
- default:
- break;
- }
out:
if (copy_to_user((void __user *)(unsigned long)cmd.response,
--
2.1.0
Use raw management helpers to reform mcast related part in IB-core cma.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cma.c | 57 +++++++++++++++----------------------------
1 file changed, 19 insertions(+), 38 deletions(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 4adb909..377e2fb 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -997,17 +997,12 @@ static void cma_leave_mc_groups(struct rdma_id_private *id_priv)
mc = container_of(id_priv->mc_list.next,
struct cma_multicast, list);
list_del(&mc->list);
- switch (rdma_port_get_link_layer(id_priv->cma_dev->device, id_priv->id.port_num)) {
- case IB_LINK_LAYER_INFINIBAND:
+ if (rdma_tech_ib(id_priv->cma_dev->device,
+ id_priv->id.port_num)) {
ib_sa_free_multicast(mc->multicast.ib);
kfree(mc);
- break;
- case IB_LINK_LAYER_ETHERNET:
+ } else
kref_put(&mc->mcref, release_mc);
- break;
- default:
- break;
- }
}
}
@@ -3314,24 +3309,13 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr,
list_add(&mc->list, &id_priv->mc_list);
spin_unlock(&id_priv->lock);
- switch (rdma_node_get_transport(id->device->node_type)) {
- case RDMA_TRANSPORT_IB:
- switch (rdma_port_get_link_layer(id->device, id->port_num)) {
- case IB_LINK_LAYER_INFINIBAND:
- ret = cma_join_ib_multicast(id_priv, mc);
- break;
- case IB_LINK_LAYER_ETHERNET:
- kref_init(&mc->mcref);
- ret = cma_iboe_join_multicast(id_priv, mc);
- break;
- default:
- ret = -EINVAL;
- }
- break;
- default:
+ if (rdma_tech_iboe(id->device, id->port_num)) {
+ kref_init(&mc->mcref);
+ ret = cma_iboe_join_multicast(id_priv, mc);
+ } else if (rdma_tech_ib(id->device, id->port_num))
+ ret = cma_join_ib_multicast(id_priv, mc);
+ else
ret = -ENOSYS;
- break;
- }
if (ret) {
spin_lock_irq(&id_priv->lock);
@@ -3359,19 +3343,16 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)
ib_detach_mcast(id->qp,
&mc->multicast.ib->rec.mgid,
be16_to_cpu(mc->multicast.ib->rec.mlid));
- if (rdma_node_get_transport(id_priv->cma_dev->device->node_type) == RDMA_TRANSPORT_IB) {
- switch (rdma_port_get_link_layer(id->device, id->port_num)) {
- case IB_LINK_LAYER_INFINIBAND:
- ib_sa_free_multicast(mc->multicast.ib);
- kfree(mc);
- break;
- case IB_LINK_LAYER_ETHERNET:
- kref_put(&mc->mcref, release_mc);
- break;
- default:
- break;
- }
- }
+
+ BUG_ON(id_priv->cma_dev->device != id->device);
+
+ if (rdma_tech_ib(id->device, id->port_num)) {
+ ib_sa_free_multicast(mc->multicast.ib);
+ kfree(mc);
+ } else if (rdma_tech_iboe(id->device,
+ id->port_num))
+ kref_put(&mc->mcref, release_mc);
+
return;
}
}
--
2.1.0
Reserve the legacy transport type for the 'transport' member
of 'struct rdma_dev_addr' until we make sure this is no
longer needed.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cma.c | 25 +++++++++++++++++++++++--
1 file changed, 23 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 377e2fb..b520882 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -244,14 +244,35 @@ static inline void cma_set_ip_ver(struct cma_hdr *hdr, u8 ip_ver)
hdr->ip_version = (ip_ver << 4) | (hdr->ip_version & 0xF);
}
+static inline void cma_set_legacy_transport(struct rdma_cm_id *id)
+{
+ switch (id->device->node_type) {
+ case RDMA_NODE_IB_CA:
+ case RDMA_NODE_IB_SWITCH:
+ case RDMA_NODE_IB_ROUTER:
+ id->route.addr.dev_addr.transport = RDMA_TRANSPORT_IB;
+ break;
+ case RDMA_NODE_RNIC:
+ id->route.addr.dev_addr.transport = RDMA_TRANSPORT_IWARP;
+ break;
+ case RDMA_NODE_USNIC:
+ id->route.addr.dev_addr.transport = RDMA_TRANSPORT_USNIC;
+ break;
+ case RDMA_NODE_USNIC_UDP:
+ id->route.addr.dev_addr.transport = RDMA_TRANSPORT_USNIC_UDP;
+ break;
+ default:
+ BUG();
+ }
+}
+
static void cma_attach_to_dev(struct rdma_id_private *id_priv,
struct cma_device *cma_dev)
{
atomic_inc(&cma_dev->refcount);
id_priv->cma_dev = cma_dev;
id_priv->id.device = cma_dev->device;
- id_priv->id.route.addr.dev_addr.transport =
- rdma_node_get_transport(cma_dev->device->node_type);
+ cma_set_legacy_transport(&id_priv->id);
list_add_tail(&id_priv->list, &cma_dev->id_list);
}
--
2.1.0
Reform cma_acquire_dev() with management helpers, introduce
cma_validate_port() to make the code more clean.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cma.c | 69 +++++++++++++++++++++++++------------------
1 file changed, 41 insertions(+), 28 deletions(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index b520882..902cc1a 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -370,18 +370,36 @@ static int cma_translate_addr(struct sockaddr *addr, struct rdma_dev_addr *dev_a
return ret;
}
+static inline int cma_validate_port(struct ib_device *device, u8 port,
+ union ib_gid *gid, int dev_type)
+{
+ u8 found_port;
+ int ret = -ENODEV;
+
+ if ((dev_type == ARPHRD_INFINIBAND) && !rdma_tech_ib(device, port))
+ return ret;
+
+ if ((dev_type != ARPHRD_INFINIBAND) && rdma_tech_ib(device, port))
+ return ret;
+
+ ret = ib_find_cached_gid(device, gid, &found_port, NULL);
+
+ if (!ret && (port == found_port))
+ return 0;
+
+ return ret;
+}
+
static int cma_acquire_dev(struct rdma_id_private *id_priv,
struct rdma_id_private *listen_id_priv)
{
struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;
struct cma_device *cma_dev;
- union ib_gid gid, iboe_gid;
+ union ib_gid gid, iboe_gid, *gidp;
int ret = -ENODEV;
- u8 port, found_port;
- enum rdma_link_layer dev_ll = dev_addr->dev_type == ARPHRD_INFINIBAND ?
- IB_LINK_LAYER_INFINIBAND : IB_LINK_LAYER_ETHERNET;
+ u8 port;
- if (dev_ll != IB_LINK_LAYER_INFINIBAND &&
+ if (dev_addr->dev_type != ARPHRD_INFINIBAND &&
id_priv->id.ps == RDMA_PS_IPOIB)
return -EINVAL;
@@ -391,41 +409,36 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv,
memcpy(&gid, dev_addr->src_dev_addr +
rdma_addr_gid_offset(dev_addr), sizeof gid);
- if (listen_id_priv &&
- rdma_port_get_link_layer(listen_id_priv->id.device,
- listen_id_priv->id.port_num) == dev_ll) {
+
+ if (listen_id_priv) {
cma_dev = listen_id_priv->cma_dev;
port = listen_id_priv->id.port_num;
- if (rdma_node_get_transport(cma_dev->device->node_type) == RDMA_TRANSPORT_IB &&
- rdma_port_get_link_layer(cma_dev->device, port) == IB_LINK_LAYER_ETHERNET)
- ret = ib_find_cached_gid(cma_dev->device, &iboe_gid,
- &found_port, NULL);
- else
- ret = ib_find_cached_gid(cma_dev->device, &gid,
- &found_port, NULL);
+ gidp = rdma_tech_iboe(cma_dev->device, port) ?
+ &iboe_gid : &gid;
- if (!ret && (port == found_port)) {
- id_priv->id.port_num = found_port;
+ ret = cma_validate_port(cma_dev->device, port, gidp,
+ dev_addr->dev_type);
+ if (!ret) {
+ id_priv->id.port_num = port;
goto out;
}
}
+
list_for_each_entry(cma_dev, &dev_list, list) {
for (port = 1; port <= cma_dev->device->phys_port_cnt; ++port) {
if (listen_id_priv &&
listen_id_priv->cma_dev == cma_dev &&
listen_id_priv->id.port_num == port)
continue;
- if (rdma_port_get_link_layer(cma_dev->device, port) == dev_ll) {
- if (rdma_node_get_transport(cma_dev->device->node_type) == RDMA_TRANSPORT_IB &&
- rdma_port_get_link_layer(cma_dev->device, port) == IB_LINK_LAYER_ETHERNET)
- ret = ib_find_cached_gid(cma_dev->device, &iboe_gid, &found_port, NULL);
- else
- ret = ib_find_cached_gid(cma_dev->device, &gid, &found_port, NULL);
-
- if (!ret && (port == found_port)) {
- id_priv->id.port_num = found_port;
- goto out;
- }
+
+ gidp = rdma_tech_iboe(cma_dev->device, port) ?
+ &iboe_gid : &gid;
+
+ ret = cma_validate_port(cma_dev->device, port, gidp,
+ dev_addr->dev_type);
+ if (!ret) {
+ id_priv->id.port_num = port;
+ goto out;
}
}
}
--
2.1.0
Use raw management helpers to reform rest part in IB-core cma.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cma.c | 16 +++++++---------
1 file changed, 7 insertions(+), 9 deletions(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 902cc1a..40ca5c6 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -469,10 +469,10 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv)
pkey = ntohs(addr->sib_pkey);
list_for_each_entry(cur_dev, &dev_list, list) {
- if (rdma_node_get_transport(cur_dev->device->node_type) != RDMA_TRANSPORT_IB)
- continue;
-
for (p = 1; p <= cur_dev->device->phys_port_cnt; ++p) {
+ if (!rdma_ib_or_iboe(cur_dev->device, p))
+ continue;
+
if (ib_find_cached_pkey(cur_dev->device, p, pkey, &index))
continue;
@@ -667,10 +667,9 @@ static int cma_modify_qp_rtr(struct rdma_id_private *id_priv,
if (ret)
goto out;
- if (rdma_node_get_transport(id_priv->cma_dev->device->node_type)
- == RDMA_TRANSPORT_IB &&
- rdma_port_get_link_layer(id_priv->id.device, id_priv->id.port_num)
- == IB_LINK_LAYER_ETHERNET) {
+ BUG_ON(id_priv->cma_dev->device != id_priv->id.device);
+
+ if (rdma_tech_iboe(id_priv->id.device, id_priv->id.port_num)) {
ret = rdma_addr_find_smac_by_sgid(&sgid, qp_attr.smac, NULL);
if (ret)
@@ -734,8 +733,7 @@ static int cma_ib_init_qp_attr(struct rdma_id_private *id_priv,
int ret;
u16 pkey;
- if (rdma_port_get_link_layer(id_priv->id.device, id_priv->id.port_num) ==
- IB_LINK_LAYER_INFINIBAND)
+ if (rdma_tech_ib(id_priv->id.device, id_priv->id.port_num))
pkey = ib_addr_get_pkey(dev_addr);
else
pkey = 0xffff;
--
2.1.0
Introduce helper cap_ib_mad() to help us check if the port of an
IB device support Infiniband Management Datagrams.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/mad.c | 6 +++---
drivers/infiniband/core/user_mad.c | 6 +++---
include/rdma/ib_verbs.h | 15 +++++++++++++++
3 files changed, 21 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index d451a47..750ad3e 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -3066,7 +3066,7 @@ static void ib_mad_init_device(struct ib_device *device)
}
for (i = start; i <= end; i++) {
- if (!rdma_ib_or_iboe(device, i))
+ if (!cap_ib_mad(device, i))
continue;
if (ib_mad_port_open(device, i)) {
@@ -3087,7 +3087,7 @@ error_agent:
error:
while (--i >= start) {
- if (!rdma_ib_or_iboe(device, i))
+ if (!cap_ib_mad(device, i))
continue;
if (ib_agent_port_close(device, i))
@@ -3110,7 +3110,7 @@ static void ib_mad_remove_device(struct ib_device *device)
cur_port = 1;
}
for (i = 0; i < num_ports; i++, cur_port++) {
- if (!rdma_ib_or_iboe(device, i))
+ if (!cap_ib_mad(device, i))
continue;
if (ib_agent_port_close(device, cur_port))
diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c
index 71fc8ba..b52884b 100644
--- a/drivers/infiniband/core/user_mad.c
+++ b/drivers/infiniband/core/user_mad.c
@@ -1294,7 +1294,7 @@ static void ib_umad_add_one(struct ib_device *device)
umad_dev->end_port = e;
for (i = s; i <= e; ++i) {
- if (!rdma_ib_or_iboe(device, i))
+ if (!cap_ib_mad(device, i))
continue;
umad_dev->port[i - s].umad_dev = umad_dev;
@@ -1317,7 +1317,7 @@ static void ib_umad_add_one(struct ib_device *device)
err:
while (--i >= s) {
- if (!rdma_ib_or_iboe(device, i))
+ if (!cap_ib_mad(device, i))
continue;
ib_umad_kill_port(&umad_dev->port[i - s]);
@@ -1335,7 +1335,7 @@ static void ib_umad_remove_one(struct ib_device *device)
return;
for (i = 0; i <= umad_dev->end_port - umad_dev->start_port; ++i) {
- if (rdma_ib_or_iboe(device, i))
+ if (cap_ib_mad(device, i))
ib_umad_kill_port(&umad_dev->port[i]);
}
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index a12e876..624e963 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1773,6 +1773,21 @@ static inline int rdma_ib_or_iboe(struct ib_device *device, u8 port_num)
return (tp == RDMA_TRANSPORT_IB || tp == RDMA_TRANSPORT_IBOE);
}
+/**
+ * cap_ib_mad - Check if the port of device has the capability Infiniband
+ * Management Datagrams.
+ *
+ * @device: Device to be checked
+ * @port_num: Port number of the device
+ *
+ * Return 0 when port of the device don't support Infiniband
+ * Management Datagrams.
+ */
+static inline int cap_ib_mad(struct ib_device *device, u8 port_num)
+{
+ return rdma_ib_or_iboe(device, port_num);
+}
+
int ib_query_gid(struct ib_device *device,
u8 port_num, int index, union ib_gid *gid);
--
2.1.0
Introduce helper cap_ib_smi() to help us check if the port of an
IB device support Infiniband Subnet Management Interface.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/agent.c | 2 +-
drivers/infiniband/core/mad.c | 2 +-
include/rdma/ib_verbs.h | 15 +++++++++++++++
3 files changed, 17 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/agent.c b/drivers/infiniband/core/agent.c
index ffdef4d..61471ee 100644
--- a/drivers/infiniband/core/agent.c
+++ b/drivers/infiniband/core/agent.c
@@ -156,7 +156,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num)
goto error1;
}
- if (rdma_tech_ib(device, port_num)) {
+ if (cap_ib_smi(device, port_num)) {
/* Obtain send only MAD agent for SMI QP */
port_priv->agent[0] = ib_register_mad_agent(device, port_num,
IB_QPT_SMI, NULL, 0,
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 750ad3e..2668d4e 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,
init_mad_qp(port_priv, &port_priv->qp_info[1]);
cq_size = mad_sendq_size + mad_recvq_size;
- has_smi = rdma_tech_ib(device, port_num);
+ has_smi = cap_ib_smi(device, port_num);
if (has_smi)
cq_size *= 2;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 624e963..873b9a6 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1788,6 +1788,21 @@ static inline int cap_ib_mad(struct ib_device *device, u8 port_num)
return rdma_ib_or_iboe(device, port_num);
}
+/**
+ * cap_ib_smi - Check if the port of device has the capability Infiniband
+ * Subnet Management Interface.
+ *
+ * @device: Device to be checked
+ * @port_num: Port number of the device
+ *
+ * Return 0 when port of the device don't support Infiniband
+ * Subnet Management Interface.
+ */
+static inline int cap_ib_smi(struct ib_device *device, u8 port_num)
+{
+ return rdma_tech_ib(device, port_num);
+}
+
int ib_query_gid(struct ib_device *device,
u8 port_num, int index, union ib_gid *gid);
--
2.1.0
Introduce helper cap_ib_cm() to help us check if the port of an
IB device support Infiniband Communication Manager.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cm.c | 6 +++---
drivers/infiniband/core/cma.c | 19 +++++++++----------
drivers/infiniband/core/ucm.c | 2 +-
include/rdma/ib_verbs.h | 15 +++++++++++++++
4 files changed, 28 insertions(+), 14 deletions(-)
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index 50321fe..63418ee 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -3781,7 +3781,7 @@ static void cm_add_one(struct ib_device *ib_device)
set_bit(IB_MGMT_METHOD_SEND, reg_req.method_mask);
for (i = 1; i <= ib_device->phys_port_cnt; i++) {
- if (!rdma_ib_or_iboe(ib_device, i))
+ if (!cap_ib_cm(ib_device, i))
continue;
port = kzalloc(sizeof *port, GFP_KERNEL);
@@ -3835,7 +3835,7 @@ error1:
port_modify.set_port_cap_mask = 0;
port_modify.clr_port_cap_mask = IB_PORT_CM_SUP;
while (--i) {
- if (!rdma_ib_or_iboe(ib_device, i))
+ if (!cap_ib_cm(ib_device, i))
continue;
port = cm_dev->port[i-1];
@@ -3866,7 +3866,7 @@ static void cm_remove_one(struct ib_device *ib_device)
write_unlock_irqrestore(&cm.device_lock, flags);
for (i = 1; i <= ib_device->phys_port_cnt; i++) {
- if (!rdma_ib_or_iboe(ib_device, i))
+ if (!cap_ib_cm(ib_device, i))
continue;
port = cm_dev->port[i-1];
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 40ca5c6..7415078 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -767,7 +767,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
int ret = 0;
id_priv = container_of(id, struct rdma_id_private, id);
- if (rdma_ib_or_iboe(id_priv->id.device, 1)) {
+ if (cap_ib_cm(id_priv->id.device, 1)) {
if (!id_priv->cm_id.ib || (id_priv->id.qp_type == IB_QPT_UD))
ret = cma_ib_init_qp_attr(id_priv, qp_attr, qp_attr_mask);
else
@@ -1055,7 +1055,7 @@ void rdma_destroy_id(struct rdma_cm_id *id)
mutex_unlock(&id_priv->handler_mutex);
if (id_priv->cma_dev) {
- if (rdma_ib_or_iboe(id_priv->id.device, 1)) {
+ if (cap_ib_cm(id_priv->id.device, 1)) {
if (id_priv->cm_id.ib)
ib_destroy_cm_id(id_priv->cm_id.ib);
} else if (rdma_tech_iwarp(id_priv->id.device, 1)) {
@@ -1638,8 +1638,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,
struct rdma_cm_id *id;
int ret;
- if (cma_family(id_priv) == AF_IB &&
- !rdma_ib_or_iboe(cma_dev->device, 1))
+ if (cma_family(id_priv) == AF_IB && !cap_ib_cm(cma_dev->device, 1))
return;
id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,
@@ -2030,7 +2029,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)
mutex_lock(&lock);
list_for_each_entry(cur_dev, &dev_list, list) {
if (cma_family(id_priv) == AF_IB &&
- !rdma_ib_or_iboe(cur_dev->device, 1))
+ !cap_ib_cm(cur_dev->device, 1))
continue;
if (!cma_dev)
@@ -2539,7 +2538,7 @@ int rdma_listen(struct rdma_cm_id *id, int backlog)
id_priv->backlog = backlog;
if (id->device) {
- if (rdma_ib_or_iboe(id->device, 1)) {
+ if (cap_ib_cm(id->device, 1)) {
ret = cma_ib_listen(id_priv);
if (ret)
goto err;
@@ -2883,7 +2882,7 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
id_priv->srq = conn_param->srq;
}
- if (rdma_ib_or_iboe(id->device, 1)) {
+ if (cap_ib_cm(id->device, 1)) {
if (id->qp_type == IB_QPT_UD)
ret = cma_resolve_ib_udp(id_priv, conn_param);
else
@@ -2994,7 +2993,7 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
id_priv->srq = conn_param->srq;
}
- if (rdma_ib_or_iboe(id->device, 1)) {
+ if (cap_ib_cm(id->device, 1)) {
if (id->qp_type == IB_QPT_UD) {
if (conn_param)
ret = cma_send_sidr_rep(id_priv, IB_SIDR_SUCCESS,
@@ -3057,7 +3056,7 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,
if (!id_priv->cm_id.ib)
return -EINVAL;
- if (rdma_ib_or_iboe(id->device, 1)) {
+ if (cap_ib_cm(id->device, 1)) {
if (id->qp_type == IB_QPT_UD)
ret = cma_send_sidr_rep(id_priv, IB_SIDR_REJECT, 0,
private_data, private_data_len);
@@ -3084,7 +3083,7 @@ int rdma_disconnect(struct rdma_cm_id *id)
if (!id_priv->cm_id.ib)
return -EINVAL;
- if (rdma_ib_or_iboe(id->device, 1)) {
+ if (cap_ib_cm(id->device, 1)) {
ret = cma_modify_qp_err(id_priv);
if (ret)
goto out;
diff --git a/drivers/infiniband/core/ucm.c b/drivers/infiniband/core/ucm.c
index 70e0ccb..f7290c8 100644
--- a/drivers/infiniband/core/ucm.c
+++ b/drivers/infiniband/core/ucm.c
@@ -1253,7 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)
dev_t base;
struct ib_ucm_device *ucm_dev;
- if (!device->alloc_ucontext || !rdma_ib_or_iboe(device, 1))
+ if (!device->alloc_ucontext || !cap_ib_cm(device, 1))
return;
ucm_dev = kzalloc(sizeof *ucm_dev, GFP_KERNEL);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 873b9a6..6805e3e 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1803,6 +1803,21 @@ static inline int cap_ib_smi(struct ib_device *device, u8 port_num)
return rdma_tech_ib(device, port_num);
}
+/**
+ * cap_ib_cm - Check if the port of device has the capability Infiniband
+ * Communication Manager.
+ *
+ * @device: Device to be checked
+ * @port_num: Port number of the device
+ *
+ * Return 0 when port of the device don't support Infiniband
+ * Communication Manager.
+ */
+static inline int cap_ib_cm(struct ib_device *device, u8 port_num)
+{
+ return rdma_ib_or_iboe(device, port_num);
+}
+
int ib_query_gid(struct ib_device *device,
u8 port_num, int index, union ib_gid *gid);
--
2.1.0
Introduce helper cap_iw_cm() to help us check if the port of an
IB device support IWARP Communication Manager.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cma.c | 14 +++++++-------
include/rdma/ib_verbs.h | 15 +++++++++++++++
2 files changed, 22 insertions(+), 7 deletions(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 7415078..a6f1526 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -776,7 +776,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
if (qp_attr->qp_state == IB_QPS_RTR)
qp_attr->rq_psn = id_priv->seq_num;
- } else if (rdma_tech_iwarp(id_priv->id.device, 1)) {
+ } else if (cap_iw_cm(id_priv->id.device, 1)) {
if (!id_priv->cm_id.iw) {
qp_attr->qp_access_flags = 0;
*qp_attr_mask = IB_QP_STATE | IB_QP_ACCESS_FLAGS;
@@ -1058,7 +1058,7 @@ void rdma_destroy_id(struct rdma_cm_id *id)
if (cap_ib_cm(id_priv->id.device, 1)) {
if (id_priv->cm_id.ib)
ib_destroy_cm_id(id_priv->cm_id.ib);
- } else if (rdma_tech_iwarp(id_priv->id.device, 1)) {
+ } else if (cap_iw_cm(id_priv->id.device, 1)) {
if (id_priv->cm_id.iw)
iw_destroy_cm_id(id_priv->cm_id.iw);
}
@@ -2542,7 +2542,7 @@ int rdma_listen(struct rdma_cm_id *id, int backlog)
ret = cma_ib_listen(id_priv);
if (ret)
goto err;
- } else if (rdma_tech_iwarp(id->device, 1)) {
+ } else if (cap_iw_cm(id->device, 1)) {
ret = cma_iw_listen(id_priv, backlog);
if (ret)
goto err;
@@ -2887,7 +2887,7 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
ret = cma_resolve_ib_udp(id_priv, conn_param);
else
ret = cma_connect_ib(id_priv, conn_param);
- } else if (rdma_tech_iwarp(id->device, 1))
+ } else if (cap_iw_cm(id->device, 1))
ret = cma_connect_iw(id_priv, conn_param);
else
ret = -ENOSYS;
@@ -3009,7 +3009,7 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
else
ret = cma_rep_recv(id_priv);
}
- } else if (rdma_tech_iwarp(id->device, 1))
+ } else if (cap_iw_cm(id->device, 1))
ret = cma_accept_iw(id_priv, conn_param);
else
ret = -ENOSYS;
@@ -3064,7 +3064,7 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,
ret = ib_send_cm_rej(id_priv->cm_id.ib,
IB_CM_REJ_CONSUMER_DEFINED, NULL,
0, private_data, private_data_len);
- } else if (rdma_tech_iwarp(id->device, 1)) {
+ } else if (cap_iw_cm(id->device, 1)) {
ret = iw_cm_reject(id_priv->cm_id.iw,
private_data, private_data_len);
} else
@@ -3090,7 +3090,7 @@ int rdma_disconnect(struct rdma_cm_id *id)
/* Initiate or respond to a disconnect. */
if (ib_send_cm_dreq(id_priv->cm_id.ib, NULL, 0))
ib_send_cm_drep(id_priv->cm_id.ib, NULL, 0);
- } else if (rdma_tech_iwarp(id->device, 1)) {
+ } else if (cap_iw_cm(id->device, 1)) {
ret = iw_cm_disconnect(id_priv->cm_id.iw, 0);
} else
ret = -EINVAL;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 6805e3e..e4999f6 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1818,6 +1818,21 @@ static inline int cap_ib_cm(struct ib_device *device, u8 port_num)
return rdma_ib_or_iboe(device, port_num);
}
+/**
+ * cap_iw_cm - Check if the port of device has the capability IWARP
+ * Communication Manager.
+ *
+ * @device: Device to be checked
+ * @port_num: Port number of the device
+ *
+ * Return 0 when port of the device don't support IWARP
+ * Communication Manager.
+ */
+static inline int cap_iw_cm(struct ib_device *device, u8 port_num)
+{
+ return rdma_tech_iwarp(device, port_num);
+}
+
int ib_query_gid(struct ib_device *device,
u8 port_num, int index, union ib_gid *gid);
--
2.1.0
Introduce helper cap_ib_sa() to help us check if the port of an
IB device support Infiniband Subnet Administrator.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cma.c | 4 ++--
drivers/infiniband/core/sa_query.c | 10 +++++-----
drivers/infiniband/core/ucma.c | 2 +-
include/rdma/ib_verbs.h | 15 +++++++++++++++
4 files changed, 23 insertions(+), 8 deletions(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index a6f1526..094816f 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -955,7 +955,7 @@ static inline int cma_user_data_offset(struct rdma_id_private *id_priv)
static void cma_cancel_route(struct rdma_id_private *id_priv)
{
- if (rdma_tech_ib(id_priv->id.device, id_priv->id.port_num)) {
+ if (cap_ib_sa(id_priv->id.device, id_priv->id.port_num)) {
if (id_priv->query)
ib_sa_cancel_query(id_priv->query_id, id_priv->query);
}
@@ -1979,7 +1979,7 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)
return -EINVAL;
atomic_inc(&id_priv->refcount);
- if (rdma_tech_ib(id->device, id->port_num))
+ if (cap_ib_sa(id->device, id->port_num))
ret = cma_resolve_ib_route(id_priv, timeout_ms);
else if (rdma_tech_iboe(id->device, id->port_num))
ret = cma_resolve_iboe_route(id_priv);
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index 803ccf7..fc7e161 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *event
struct ib_sa_port *port =
&sa_dev->port[event->element.port_num - sa_dev->start_port];
- if (WARN_ON(!rdma_tech_ib(handler->device, port->port_num)))
+ if (WARN_ON(!cap_ib_sa(handler->device, port->port_num)))
return;
spin_lock_irqsave(&port->ah_lock, flags);
@@ -1173,7 +1173,7 @@ static void ib_sa_add_one(struct ib_device *device)
for (i = 0; i <= e - s; ++i) {
spin_lock_init(&sa_dev->port[i].ah_lock);
- if (!rdma_tech_ib(device, i + 1))
+ if (!cap_ib_sa(device, i + 1))
continue;
sa_dev->port[i].sm_ah = NULL;
@@ -1210,7 +1210,7 @@ static void ib_sa_add_one(struct ib_device *device)
goto err;
for (i = 0; i <= e - s; ++i) {
- if (rdma_tech_ib(device, i + 1))
+ if (cap_ib_sa(device, i + 1))
update_sm_ah(&sa_dev->port[i].update_task);
}
@@ -1218,7 +1218,7 @@ static void ib_sa_add_one(struct ib_device *device)
err:
while (--i >= 0) {
- if (rdma_tech_ib(device, i + 1))
+ if (cap_ib_sa(device, i + 1))
ib_unregister_mad_agent(sa_dev->port[i].agent);
}
@@ -1240,7 +1240,7 @@ static void ib_sa_remove_one(struct ib_device *device)
flush_workqueue(ib_wq);
for (i = 0; i <= sa_dev->end_port - sa_dev->start_port; ++i) {
- if (rdma_tech_ib(device, i + 1)) {
+ if (cap_ib_sa(device, i + 1)) {
ib_unregister_mad_agent(sa_dev->port[i].agent);
if (sa_dev->port[i].sm_ah)
kref_put(&sa_dev->port[i].sm_ah->ref, free_sm_ah);
diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index 7331c6c..bed7957 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -723,7 +723,7 @@ static ssize_t ucma_query_route(struct ucma_file *file,
resp.node_guid = (__force __u64) ctx->cm_id->device->node_guid;
resp.port_num = ctx->cm_id->port_num;
- if (rdma_tech_ib(ctx->cm_id->device, ctx->cm_id->port_num))
+ if (cap_ib_sa(ctx->cm_id->device, ctx->cm_id->port_num))
ucma_copy_ib_route(&resp, &ctx->cm_id->route);
else if (rdma_tech_iboe(ctx->cm_id->device, ctx->cm_id->port_num))
ucma_copy_iboe_route(&resp, &ctx->cm_id->route);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index e4999f6..3bfdf81 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1833,6 +1833,21 @@ static inline int cap_iw_cm(struct ib_device *device, u8 port_num)
return rdma_tech_iwarp(device, port_num);
}
+/**
+ * cap_ib_sa - Check if the port of device has the capability Infiniband
+ * Subnet Administrator.
+ *
+ * @device: Device to be checked
+ * @port_num: Port number of the device
+ *
+ * Return 0 when port of the device don't support Infiniband
+ * Subnet Administrator.
+ */
+static inline int cap_ib_sa(struct ib_device *device, u8 port_num)
+{
+ return rdma_tech_ib(device, port_num);
+}
+
int ib_query_gid(struct ib_device *device,
u8 port_num, int index, union ib_gid *gid);
--
2.1.0
Introduce helper cap_ib_mcast() to help us check if the port of an
IB device support Infiniband Multicast.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cma.c | 6 +++---
drivers/infiniband/core/multicast.c | 6 +++---
include/rdma/ib_verbs.h | 15 +++++++++++++++
3 files changed, 21 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 094816f..0ae88dc 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -1029,7 +1029,7 @@ static void cma_leave_mc_groups(struct rdma_id_private *id_priv)
mc = container_of(id_priv->mc_list.next,
struct cma_multicast, list);
list_del(&mc->list);
- if (rdma_tech_ib(id_priv->cma_dev->device,
+ if (cap_ib_mcast(id_priv->cma_dev->device,
id_priv->id.port_num)) {
ib_sa_free_multicast(mc->multicast.ib);
kfree(mc);
@@ -3343,7 +3343,7 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr,
if (rdma_tech_iboe(id->device, id->port_num)) {
kref_init(&mc->mcref);
ret = cma_iboe_join_multicast(id_priv, mc);
- } else if (rdma_tech_ib(id->device, id->port_num))
+ } else if (cap_ib_mcast(id->device, id->port_num))
ret = cma_join_ib_multicast(id_priv, mc);
else
ret = -ENOSYS;
@@ -3377,7 +3377,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)
BUG_ON(id_priv->cma_dev->device != id->device);
- if (rdma_tech_ib(id->device, id->port_num)) {
+ if (cap_ib_mcast(id->device, id->port_num)) {
ib_sa_free_multicast(mc->multicast.ib);
kfree(mc);
} else if (rdma_tech_iboe(id->device,
diff --git a/drivers/infiniband/core/multicast.c b/drivers/infiniband/core/multicast.c
index 24d93f5..bdc1880 100644
--- a/drivers/infiniband/core/multicast.c
+++ b/drivers/infiniband/core/multicast.c
@@ -780,7 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,
int index;
dev = container_of(handler, struct mcast_device, event_handler);
- if (WARN_ON(!rdma_tech_ib(dev->device, event->element.port_num)))
+ if (WARN_ON(!cap_ib_mcast(dev->device, event->element.port_num)))
return;
index = event->element.port_num - dev->start_port;
@@ -820,7 +820,7 @@ static void mcast_add_one(struct ib_device *device)
}
for (i = 0; i <= dev->end_port - dev->start_port; i++) {
- if (!rdma_tech_ib(device, dev->start_port + i))
+ if (!cap_ib_mcast(device, dev->start_port + i))
continue;
port = &dev->port[i];
port->dev = dev;
@@ -858,7 +858,7 @@ static void mcast_remove_one(struct ib_device *device)
flush_workqueue(mcast_wq);
for (i = 0; i <= dev->end_port - dev->start_port; i++) {
- if (rdma_tech_ib(device, dev->start_port + i)) {
+ if (cap_ib_mcast(device, dev->start_port + i)) {
port = &dev->port[i];
deref_port(port);
wait_for_completion(&port->comp);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 3bfdf81..b2cee8d 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1848,6 +1848,21 @@ static inline int cap_ib_sa(struct ib_device *device, u8 port_num)
return rdma_tech_ib(device, port_num);
}
+/**
+ * cap_ib_mcast - Check if the port of device has the capability Infiniband
+ * Multicast.
+ *
+ * @device: Device to be checked
+ * @port_num: Port number of the device
+ *
+ * Return 0 when port of the device don't support Infiniband
+ * Multicast.
+ */
+static inline int cap_ib_mcast(struct ib_device *device, u8 port_num)
+{
+ return cap_ib_sa(device, port_num);
+}
+
int ib_query_gid(struct ib_device *device,
u8 port_num, int index, union ib_gid *gid);
--
2.1.0
Introduce helper cap_ipoib() to help us check if the port of an
IB device support IP over Infiniband.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/ulp/ipoib/ipoib_main.c | 2 +-
include/rdma/ib_verbs.h | 15 +++++++++++++++
2 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 60b379d..a9812df 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -1671,7 +1671,7 @@ static void ipoib_add_one(struct ib_device *device)
}
for (p = s; p <= e; ++p) {
- if (!rdma_tech_ib(device, p))
+ if (!cap_ipoib(device, p))
continue;
dev = ipoib_add_port("ib%d", device, p);
if (!IS_ERR(dev)) {
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index b2cee8d..ac62b3a 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1863,6 +1863,21 @@ static inline int cap_ib_mcast(struct ib_device *device, u8 port_num)
return cap_ib_sa(device, port_num);
}
+/**
+ * cap_ipoib - Check if the port of device has the capability
+ * IP over Infiniband.
+ *
+ * @device: Device to be checked
+ * @port_num: Port number of the device
+ *
+ * Return 0 when port of the device don't support
+ * IP over Infiniband.
+ */
+static inline int cap_ipoib(struct ib_device *device, u8 port_num)
+{
+ return rdma_tech_ib(device, port_num);
+}
+
int ib_query_gid(struct ib_device *device,
u8 port_num, int index, union ib_gid *gid);
--
2.1.0
Introduce helper cap_read_multi_sge() to help us check if the port of an
IB device support RDMA Read Multiple Scatter-Gather Entries.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
include/rdma/ib_verbs.h | 15 +++++++++++++++
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 3 ++-
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index ac62b3a..60f7efb 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1878,6 +1878,21 @@ static inline int cap_ipoib(struct ib_device *device, u8 port_num)
return rdma_tech_ib(device, port_num);
}
+/**
+ * cap_read_multi_sge - Check if the port of device has the capability
+ * RDMA Read Multiple Scatter-Gather Entries.
+ *
+ * @device: Device to be checked
+ * @port_num: Port number of the device
+ *
+ * Return 0 when port of the device don't support
+ * RDMA Read Multiple Scatter-Gather Entries.
+ */
+static inline int cap_read_multi_sge(struct ib_device *device, u8 port_num)
+{
+ return !rdma_tech_iwarp(device, port_num);
+}
+
int ib_query_gid(struct ib_device *device,
u8 port_num, int index, union ib_gid *gid);
diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
index a5bed5b..7711b7a 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
@@ -117,7 +117,8 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,
static int rdma_read_max_sge(struct svcxprt_rdma *xprt, int sge_count)
{
- if (rdma_tech_iwarp(xprt->sc_cm_id->device, xprt->sc_cm_id->port_num))
+ if (!cap_read_multi_sge(xprt->sc_cm_id->device,
+ xprt->sc_cm_id->port_num))
return 1;
else
return min_t(int, sge_count, xprt->sc_max_sge);
--
2.1.0
Introduce helper cap_af_ib() to help us check if the port of an
IB device support Native Infiniband Address.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cma.c | 2 +-
include/rdma/ib_verbs.h | 15 +++++++++++++++
2 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 0ae88dc..d72a2a4 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -470,7 +470,7 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv)
list_for_each_entry(cur_dev, &dev_list, list) {
for (p = 1; p <= cur_dev->device->phys_port_cnt; ++p) {
- if (!rdma_ib_or_iboe(cur_dev->device, p))
+ if (!cap_af_ib(cur_dev->device, p))
continue;
if (ib_find_cached_pkey(cur_dev->device, p, pkey, &index))
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 60f7efb..87ab01b 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1879,6 +1879,21 @@ static inline int cap_ipoib(struct ib_device *device, u8 port_num)
}
/**
+ * cap_af_ib - Check if the port of device has the capability
+ * Native Infiniband Address.
+ *
+ * @device: Device to be checked
+ * @port_num: Port number of the device
+ *
+ * Return 0 when port of the device don't support
+ * Native Infiniband Address.
+ */
+static inline int cap_af_ib(struct ib_device *device, u8 port_num)
+{
+ return rdma_ib_or_iboe(device, port_num);
+}
+
+/**
* cap_read_multi_sge - Check if the port of device has the capability
* RDMA Read Multiple Scatter-Gather Entries.
*
--
2.1.0
Introduce helper cap_eth_ah() to help us check if the port of an
IB device support Ethernet Address Handler.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/cma.c | 6 +++---
drivers/infiniband/core/sa_query.c | 2 +-
drivers/infiniband/core/verbs.c | 2 +-
include/rdma/ib_verbs.h | 15 +++++++++++++++
4 files changed, 20 insertions(+), 5 deletions(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index d72a2a4..e91ee7a 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -733,10 +733,10 @@ static int cma_ib_init_qp_attr(struct rdma_id_private *id_priv,
int ret;
u16 pkey;
- if (rdma_tech_ib(id_priv->id.device, id_priv->id.port_num))
- pkey = ib_addr_get_pkey(dev_addr);
- else
+ if (cap_eth_ah(id_priv->id.device, id_priv->id.port_num))
pkey = 0xffff;
+ else
+ pkey = ib_addr_get_pkey(dev_addr);
ret = ib_find_cached_pkey(id_priv->id.device, id_priv->id.port_num,
pkey, &qp_attr->pkey_index);
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index fc7e161..409ec4e 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -540,7 +540,7 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num,
ah_attr->port_num = port_num;
ah_attr->static_rate = rec->rate;
- force_grh = rdma_tech_iboe(device, port_num);
+ force_grh = cap_eth_ah(device, port_num);
if (rec->hop_limit > 1 || force_grh) {
ah_attr->ah_flags = IB_AH_GRH;
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index 7264860..ee4b5cb 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -202,7 +202,7 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 port_num, struct ib_wc *wc,
int ret;
memset(ah_attr, 0, sizeof *ah_attr);
- if (rdma_tech_iboe(device, port_num)) {
+ if (cap_eth_ah(device, port_num)) {
if (!(wc->wc_flags & IB_WC_GRH))
return -EPROTOTYPE;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 87ab01b..09c83ae 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1894,6 +1894,21 @@ static inline int cap_af_ib(struct ib_device *device, u8 port_num)
}
/**
+ * cap_eth_ah - Check if the port of device has the capability
+ * Ethernet Address Handler.
+ *
+ * @device: Device to be checked
+ * @port_num: Port number of the device
+ *
+ * Return 0 when port of the device don't support
+ * Ethernet Address Handler.
+ */
+static inline int cap_eth_ah(struct ib_device *device, u8 port_num)
+{
+ return rdma_tech_iboe(device, port_num);
+}
+
+/**
* cap_read_multi_sge - Check if the port of device has the capability
* RDMA Read Multiple Scatter-Gather Entries.
*
--
2.1.0
We have finished introducing the cap_XX(), and raw helper rdma_ib_or_iboe()
is no longer necessary, thus clean it up.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
include/rdma/ib_verbs.h | 19 +++++++++----------
net/sunrpc/xprtrdma/svc_rdma_transport.c | 6 ++++--
2 files changed, 13 insertions(+), 12 deletions(-)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 09c83ae..262bf44 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1766,13 +1766,6 @@ static inline int rdma_tech_iwarp(struct ib_device *device, u8 port_num)
== RDMA_TRANSPORT_IWARP;
}
-static inline int rdma_ib_or_iboe(struct ib_device *device, u8 port_num)
-{
- enum rdma_transport_type tp = device->query_transport(device, port_num);
-
- return (tp == RDMA_TRANSPORT_IB || tp == RDMA_TRANSPORT_IBOE);
-}
-
/**
* cap_ib_mad - Check if the port of device has the capability Infiniband
* Management Datagrams.
@@ -1785,7 +1778,9 @@ static inline int rdma_ib_or_iboe(struct ib_device *device, u8 port_num)
*/
static inline int cap_ib_mad(struct ib_device *device, u8 port_num)
{
- return rdma_ib_or_iboe(device, port_num);
+ enum rdma_transport_type tp = device->query_transport(device, port_num);
+
+ return (tp == RDMA_TRANSPORT_IB || tp == RDMA_TRANSPORT_IBOE);
}
/**
@@ -1815,7 +1810,9 @@ static inline int cap_ib_smi(struct ib_device *device, u8 port_num)
*/
static inline int cap_ib_cm(struct ib_device *device, u8 port_num)
{
- return rdma_ib_or_iboe(device, port_num);
+ enum rdma_transport_type tp = device->query_transport(device, port_num);
+
+ return (tp == RDMA_TRANSPORT_IB || tp == RDMA_TRANSPORT_IBOE);
}
/**
@@ -1890,7 +1887,9 @@ static inline int cap_ipoib(struct ib_device *device, u8 port_num)
*/
static inline int cap_af_ib(struct ib_device *device, u8 port_num)
{
- return rdma_ib_or_iboe(device, port_num);
+ enum rdma_transport_type tp = device->query_transport(device, port_num);
+
+ return (tp == RDMA_TRANSPORT_IB || tp == RDMA_TRANSPORT_IBOE);
}
/**
diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c
index a09b7a1..8af6f92 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_transport.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c
@@ -987,8 +987,10 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)
*/
if (!rdma_tech_iwarp(newxprt->sc_cm_id->device,
newxprt->sc_cm_id->port_num) &&
- !rdma_ib_or_iboe(newxprt->sc_cm_id->device,
- newxprt->sc_cm_id->port_num))
+ !rdma_tech_ib(newxprt->sc_cm_id->device,
+ newxprt->sc_cm_id->port_num) &&
+ !rdma_tech_iboe(newxprt->sc_cm_id->device,
+ newxprt->sc_cm_id->port_num))
goto errout;
if (!(newxprt->sc_dev_caps & SVCRDMA_DEVCAP_FAST_REG) ||
--
2.1.0
We have get rid of all the scene using legacy rdma_node_get_transport(),
now clean it up.
Cc: Steve Wise <[email protected]>
Cc: Tom Talpey <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Doug Ledford <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Sean Hefty <[email protected]>
Signed-off-by: Michael Wang <[email protected]>
---
drivers/infiniband/core/verbs.c | 21 ---------------------
include/rdma/ib_verbs.h | 3 ---
2 files changed, 24 deletions(-)
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index ee4b5cb..bbea0c0 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -107,27 +107,6 @@ __attribute_const__ int ib_rate_to_mbps(enum ib_rate rate)
}
EXPORT_SYMBOL(ib_rate_to_mbps);
-__attribute_const__ enum rdma_transport_type
-rdma_node_get_transport(enum rdma_node_type node_type)
-{
- switch (node_type) {
- case RDMA_NODE_IB_CA:
- case RDMA_NODE_IB_SWITCH:
- case RDMA_NODE_IB_ROUTER:
- return RDMA_TRANSPORT_IB;
- case RDMA_NODE_RNIC:
- return RDMA_TRANSPORT_IWARP;
- case RDMA_NODE_USNIC:
- return RDMA_TRANSPORT_USNIC;
- case RDMA_NODE_USNIC_UDP:
- return RDMA_TRANSPORT_USNIC_UDP;
- default:
- BUG();
- return 0;
- }
-}
-EXPORT_SYMBOL(rdma_node_get_transport);
-
enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device, u8 port_num)
{
if (device->get_link_layer)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 262bf44..f9ef479 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -84,9 +84,6 @@ enum rdma_transport_type {
RDMA_TRANSPORT_IBOE,
};
-__attribute_const__ enum rdma_transport_type
-rdma_node_get_transport(enum rdma_node_type node_type);
-
enum rdma_link_layer {
IB_LINK_LAYER_UNSPECIFIED,
IB_LINK_LAYER_INFINIBAND,
--
2.1.0
On 4/16/2015 4:05 AM, Michael Wang wrote:
>
> Use raw management helpers to reform IB-core mad/agent/user_mad.
>
> Cc: Steve Wise <[email protected]>
> Cc: Tom Talpey <[email protected]>
> Cc: Jason Gunthorpe <[email protected]>
> Cc: Doug Ledford <[email protected]>
> Cc: Ira Weiny <[email protected]>
> Cc: Sean Hefty <[email protected]>
> Signed-off-by: Michael Wang <[email protected]>
> ---
> drivers/infiniband/core/agent.c | 2 +-
> drivers/infiniband/core/mad.c | 20 ++++++++++----------
> drivers/infiniband/core/user_mad.c | 26 ++++++++++++++++++++------
> 3 files changed, 31 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/infiniband/core/agent.c b/drivers/infiniband/core/agent.c
> index f6d2961..ffdef4d 100644
> --- a/drivers/infiniband/core/agent.c
> +++ b/drivers/infiniband/core/agent.c
> @@ -156,7 +156,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num)
> goto error1;
> }
>
> - if (rdma_port_get_link_layer(device, port_num) == IB_LINK_LAYER_INFINIBAND) {
> + if (rdma_tech_ib(device, port_num)) {
> /* Obtain send only MAD agent for SMI QP */
> port_priv->agent[0] = ib_register_mad_agent(device, port_num,
> IB_QPT_SMI, NULL, 0,
> diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
> index 74c30f4..d451a47 100644
> --- a/drivers/infiniband/core/mad.c
> +++ b/drivers/infiniband/core/mad.c
> @@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,
> init_mad_qp(port_priv, &port_priv->qp_info[1]);
>
> cq_size = mad_sendq_size + mad_recvq_size;
> - has_smi = rdma_port_get_link_layer(device, port_num) == IB_LINK_LAYER_INFINIBAND;
> + has_smi = rdma_tech_ib(device, port_num);
> if (has_smi)
> cq_size *= 2;
>
> @@ -3057,9 +3057,6 @@ static void ib_mad_init_device(struct ib_device *device)
> {
> int start, end, i;
>
> - if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
> - return;
> -
> if (device->node_type == RDMA_NODE_IB_SWITCH) {
> start = 0;
> end = 0;
> @@ -3069,6 +3066,9 @@ static void ib_mad_init_device(struct ib_device *device)
> }
>
> for (i = start; i <= end; i++) {
> + if (!rdma_ib_or_iboe(device, i))
> + continue;
> +
> if (ib_mad_port_open(device, i)) {
> dev_err(&device->dev, "Couldn't open port %d\n", i);
> goto error;
> @@ -3086,15 +3086,15 @@ error_agent:
> dev_err(&device->dev, "Couldn't close port %d\n", i);
>
> error:
> - i--;
> + while (--i >= start) {
> + if (!rdma_ib_or_iboe(device, i))
> + continue;
>
> - while (i >= start) {
> if (ib_agent_port_close(device, i))
> dev_err(&device->dev,
> "Couldn't close port %d for agents\n", i);
> if (ib_mad_port_close(device, i))
> dev_err(&device->dev, "Couldn't close port %d\n", i);
> - i--;
> }
> }
>
> @@ -3102,9 +3102,6 @@ static void ib_mad_remove_device(struct ib_device *device)
> {
> int i, num_ports, cur_port;
>
> - if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
> - return;
> -
> if (device->node_type == RDMA_NODE_IB_SWITCH) {
> num_ports = 1;
> cur_port = 0;
> @@ -3113,6 +3110,9 @@ static void ib_mad_remove_device(struct ib_device *device)
> cur_port = 1;
> }
> for (i = 0; i < num_ports; i++, cur_port++) {
> + if (!rdma_ib_or_iboe(device, i))
Shouldn't this be:
if (!rdma_ib_or_iboe(device, cur_port))
?
-- Hal
> + continue;
> +
> if (ib_agent_port_close(device, cur_port))
> dev_err(&device->dev,
> "Couldn't close port %d for agents\n",
> diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c
> index 928cdd2..71fc8ba 100644
> --- a/drivers/infiniband/core/user_mad.c
> +++ b/drivers/infiniband/core/user_mad.c
> @@ -1273,9 +1273,7 @@ static void ib_umad_add_one(struct ib_device *device)
> {
> struct ib_umad_device *umad_dev;
> int s, e, i;
> -
> - if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
> - return;
> + int count = 0;
>
> if (device->node_type == RDMA_NODE_IB_SWITCH)
> s = e = 0;
> @@ -1296,11 +1294,21 @@ static void ib_umad_add_one(struct ib_device *device)
> umad_dev->end_port = e;
>
> for (i = s; i <= e; ++i) {
> + if (!rdma_ib_or_iboe(device, i))
> + continue;
> +
> umad_dev->port[i - s].umad_dev = umad_dev;
>
> if (ib_umad_init_port(device, i, umad_dev,
> &umad_dev->port[i - s]))
> goto err;
> +
> + count++;
> + }
> +
> + if (!count) {
> + kobject_put(&umad_dev->kobj);
> + return;
> }
>
> ib_set_client_data(device, &umad_client, umad_dev);
> @@ -1308,8 +1316,12 @@ static void ib_umad_add_one(struct ib_device *device)
> return;
>
> err:
> - while (--i >= s)
> + while (--i >= s) {
> + if (!rdma_ib_or_iboe(device, i))
> + continue;
> +
> ib_umad_kill_port(&umad_dev->port[i - s]);
> + }
>
> kobject_put(&umad_dev->kobj);
> }
> @@ -1322,8 +1334,10 @@ static void ib_umad_remove_one(struct ib_device *device)
> if (!umad_dev)
> return;
>
> - for (i = 0; i <= umad_dev->end_port - umad_dev->start_port; ++i)
> - ib_umad_kill_port(&umad_dev->port[i]);
> + for (i = 0; i <= umad_dev->end_port - umad_dev->start_port; ++i) {
> + if (rdma_ib_or_iboe(device, i))
> + ib_umad_kill_port(&umad_dev->port[i]);
> + }
>
> kobject_put(&umad_dev->kobj);
> }
On 4/16/2015 4:05 AM, Michael Wang wrote:
>
> Use raw management helpers to reform IB-core cm.
>
> Cc: Steve Wise <[email protected]>
> Cc: Tom Talpey <[email protected]>
> Cc: Jason Gunthorpe <[email protected]>
> Cc: Doug Ledford <[email protected]>
> Cc: Ira Weiny <[email protected]>
> Cc: Sean Hefty <[email protected]>
> Signed-off-by: Michael Wang <[email protected]>
> ---
> drivers/infiniband/core/cm.c | 22 +++++++++++++++++++---
> 1 file changed, 19 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
> index e28a494..50321fe 100644
> --- a/drivers/infiniband/core/cm.c
> +++ b/drivers/infiniband/core/cm.c
> @@ -3761,9 +3761,7 @@ static void cm_add_one(struct ib_device *ib_device)
> unsigned long flags;
> int ret;
> u8 i;
> -
> - if (rdma_node_get_transport(ib_device->node_type) != RDMA_TRANSPORT_IB)
> - return;
> + int count = 0;
>
> cm_dev = kzalloc(sizeof(*cm_dev) + sizeof(*port) *
> ib_device->phys_port_cnt, GFP_KERNEL);
> @@ -3783,6 +3781,9 @@ static void cm_add_one(struct ib_device *ib_device)
>
> set_bit(IB_MGMT_METHOD_SEND, reg_req.method_mask);
> for (i = 1; i <= ib_device->phys_port_cnt; i++) {
> + if (!rdma_ib_or_iboe(ib_device, i))
> + continue;
> +
> port = kzalloc(sizeof *port, GFP_KERNEL);
> if (!port)
> goto error1;
> @@ -3809,7 +3810,16 @@ static void cm_add_one(struct ib_device *ib_device)
> ret = ib_modify_port(ib_device, i, 0, &port_modify);
> if (ret)
> goto error3;
> +
> + count++;
> }
> +
> + if (!count) {
> + device_unregister(cm_dev->device);
> + kfree(cm_dev);
> + return;
Nit: alternatively, this could be goto at end of this routine where
there is same 2 calls. I think Jason made this comment on earlier set of
these patches.
-- Hal
> + }
> +
> ib_set_client_data(ib_device, &cm_client, cm_dev);
>
> write_lock_irqsave(&cm.device_lock, flags);
> @@ -3825,6 +3835,9 @@ error1:
> port_modify.set_port_cap_mask = 0;
> port_modify.clr_port_cap_mask = IB_PORT_CM_SUP;
> while (--i) {
> + if (!rdma_ib_or_iboe(ib_device, i))
> + continue;
> +
> port = cm_dev->port[i-1];
> ib_modify_port(ib_device, port->port_num, 0, &port_modify);
> ib_unregister_mad_agent(port->mad_agent);
> @@ -3853,6 +3866,9 @@ static void cm_remove_one(struct ib_device *ib_device)
> write_unlock_irqrestore(&cm.device_lock, flags);
>
> for (i = 1; i <= ib_device->phys_port_cnt; i++) {
> + if (!rdma_ib_or_iboe(ib_device, i))
> + continue;
> +
> port = cm_dev->port[i-1];
> ib_modify_port(ib_device, port->port_num, 0, &port_modify);
> ib_unregister_mad_agent(port->mad_agent);
Hi,Hal
Thanks for the comments :-)
On 04/16/2015 02:43 PM, Hal Rosenstock wrote:
[snip]
>> @@ -3102,9 +3102,6 @@ static void ib_mad_remove_device(struct ib_device *device)
>> {
>> int i, num_ports, cur_port;
>>
>> - if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
>> - return;
>> -
>> if (device->node_type == RDMA_NODE_IB_SWITCH) {
>> num_ports = 1;
>> cur_port = 0;
>> @@ -3113,6 +3110,9 @@ static void ib_mad_remove_device(struct ib_device *device)
>> cur_port = 1;
>> }
>> for (i = 0; i < num_ports; i++, cur_port++) {
>> + if (!rdma_ib_or_iboe(device, i))
>
> Shouldn't this be:
> if (!rdma_ib_or_iboe(device, cur_port))
> ?
Agree, actually it's unnecessary to use both i and cur_port, we can just use the
same way inside ib_mad_init_device() ;-)
Regards,
Michael Wang
>
> -- Hal
>
>> + continue;
>> +
>> if (ib_agent_port_close(device, cur_port))
>> dev_err(&device->dev,
>> "Couldn't close port %d for agents\n",
>> diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c
>> index 928cdd2..71fc8ba 100644
>> --- a/drivers/infiniband/core/user_mad.c
>> +++ b/drivers/infiniband/core/user_mad.c
>> @@ -1273,9 +1273,7 @@ static void ib_umad_add_one(struct ib_device *device)
>> {
>> struct ib_umad_device *umad_dev;
>> int s, e, i;
>> -
>> - if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
>> - return;
>> + int count = 0;
>>
>> if (device->node_type == RDMA_NODE_IB_SWITCH)
>> s = e = 0;
>> @@ -1296,11 +1294,21 @@ static void ib_umad_add_one(struct ib_device *device)
>> umad_dev->end_port = e;
>>
>> for (i = s; i <= e; ++i) {
>> + if (!rdma_ib_or_iboe(device, i))
>> + continue;
>> +
>> umad_dev->port[i - s].umad_dev = umad_dev;
>>
>> if (ib_umad_init_port(device, i, umad_dev,
>> &umad_dev->port[i - s]))
>> goto err;
>> +
>> + count++;
>> + }
>> +
>> + if (!count) {
>> + kobject_put(&umad_dev->kobj);
>> + return;
>> }
>>
>> ib_set_client_data(device, &umad_client, umad_dev);
>> @@ -1308,8 +1316,12 @@ static void ib_umad_add_one(struct ib_device *device)
>> return;
>>
>> err:
>> - while (--i >= s)
>> + while (--i >= s) {
>> + if (!rdma_ib_or_iboe(device, i))
>> + continue;
>> +
>> ib_umad_kill_port(&umad_dev->port[i - s]);
>> + }
>>
>> kobject_put(&umad_dev->kobj);
>> }
>> @@ -1322,8 +1334,10 @@ static void ib_umad_remove_one(struct ib_device *device)
>> if (!umad_dev)
>> return;
>>
>> - for (i = 0; i <= umad_dev->end_port - umad_dev->start_port; ++i)
>> - ib_umad_kill_port(&umad_dev->port[i]);
>> + for (i = 0; i <= umad_dev->end_port - umad_dev->start_port; ++i) {
>> + if (rdma_ib_or_iboe(device, i))
>> + ib_umad_kill_port(&umad_dev->port[i]);
>> + }
>>
>> kobject_put(&umad_dev->kobj);
>> }
>
On 04/16/2015 02:43 PM, Hal Rosenstock wrote:
[snip]
>> +
>> + count++;
>> }
>> +
>> + if (!count) {
>> + device_unregister(cm_dev->device);
>> + kfree(cm_dev);
>> + return;
>
> Nit: alternatively, this could be goto at end of this routine where
> there is same 2 calls. I think Jason made this comment on earlier set of
> these patches.
Thanks for the notify, I guess I missed it, my bad :-P
Will be "goto free" in next version.
Regards,
Michael Wang
>
> -- Hal
>
>> + }
>> +
>> ib_set_client_data(ib_device, &cm_client, cm_dev);
>>
>> write_lock_irqsave(&cm.device_lock, flags);
>> @@ -3825,6 +3835,9 @@ error1:
>> port_modify.set_port_cap_mask = 0;
>> port_modify.clr_port_cap_mask = IB_PORT_CM_SUP;
>> while (--i) {
>> + if (!rdma_ib_or_iboe(ib_device, i))
>> + continue;
>> +
>> port = cm_dev->port[i-1];
>> ib_modify_port(ib_device, port->port_num, 0, &port_modify);
>> ib_unregister_mad_agent(port->mad_agent);
>> @@ -3853,6 +3866,9 @@ static void cm_remove_one(struct ib_device *ib_device)
>> write_unlock_irqrestore(&cm.device_lock, flags);
>>
>> for (i = 1; i <= ib_device->phys_port_cnt; i++) {
>> + if (!rdma_ib_or_iboe(ib_device, i))
>> + continue;
>> +
>> port = cm_dev->port[i-1];
>> ib_modify_port(ib_device, port->port_num, 0, &port_modify);
>> ib_unregister_mad_agent(port->mad_agent);
>
On 4/16/2015 4:08 AM, Michael Wang wrote:
>
> Use raw management helpers to reform cm related part in IB-core cma/ucm.
>
> These checks focus on the device cm type rather than the port capability,
> directly pass port 1 works currently, but can't support mixing cm type
> device in future.
This is equivalent to today where the checks are per node rather than
per port.
Should all checks here be port 1 based or only certain ones like listen
? For example, in connect/reject/disconnect, don't we already have port
? Guess this can be dealt with later as this is not a regression from
the current implementation.
-- Hal
>
> Cc: Steve Wise <[email protected]>
> Cc: Tom Talpey <[email protected]>
> Cc: Jason Gunthorpe <[email protected]>
> Cc: Doug Ledford <[email protected]>
> Cc: Ira Weiny <[email protected]>
> Cc: Sean Hefty <[email protected]>
> Signed-off-by: Michael Wang <[email protected]>
> ---
> drivers/infiniband/core/cma.c | 81 +++++++++++++------------------------------
> drivers/infiniband/core/ucm.c | 3 +-
> 2 files changed, 26 insertions(+), 58 deletions(-)
>
> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> index d570030..6b8a64d 100644
> --- a/drivers/infiniband/core/cma.c
> +++ b/drivers/infiniband/core/cma.c
> @@ -735,8 +735,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
> int ret = 0;
>
> id_priv = container_of(id, struct rdma_id_private, id);
> - switch (rdma_node_get_transport(id_priv->id.device->node_type)) {
> - case RDMA_TRANSPORT_IB:
> + if (rdma_ib_or_iboe(id_priv->id.device, 1)) {
> if (!id_priv->cm_id.ib || (id_priv->id.qp_type == IB_QPT_UD))
> ret = cma_ib_init_qp_attr(id_priv, qp_attr, qp_attr_mask);
> else
> @@ -745,19 +744,15 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
>
> if (qp_attr->qp_state == IB_QPS_RTR)
> qp_attr->rq_psn = id_priv->seq_num;
> - break;
> - case RDMA_TRANSPORT_IWARP:
> + } else if (rdma_tech_iwarp(id_priv->id.device, 1)) {
> if (!id_priv->cm_id.iw) {
> qp_attr->qp_access_flags = 0;
> *qp_attr_mask = IB_QP_STATE | IB_QP_ACCESS_FLAGS;
> } else
> ret = iw_cm_init_qp_attr(id_priv->cm_id.iw, qp_attr,
> qp_attr_mask);
> - break;
> - default:
> + } else
> ret = -ENOSYS;
> - break;
> - }
>
> return ret;
> }
> @@ -1037,17 +1032,12 @@ void rdma_destroy_id(struct rdma_cm_id *id)
> mutex_unlock(&id_priv->handler_mutex);
>
> if (id_priv->cma_dev) {
> - switch (rdma_node_get_transport(id_priv->id.device->node_type)) {
> - case RDMA_TRANSPORT_IB:
> + if (rdma_ib_or_iboe(id_priv->id.device, 1)) {
> if (id_priv->cm_id.ib)
> ib_destroy_cm_id(id_priv->cm_id.ib);
> - break;
> - case RDMA_TRANSPORT_IWARP:
> + } else if (rdma_tech_iwarp(id_priv->id.device, 1)) {
> if (id_priv->cm_id.iw)
> iw_destroy_cm_id(id_priv->cm_id.iw);
> - break;
> - default:
> - break;
> }
> cma_leave_mc_groups(id_priv);
> cma_release_dev(id_priv);
> @@ -1626,7 +1616,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,
> int ret;
>
> if (cma_family(id_priv) == AF_IB &&
> - rdma_node_get_transport(cma_dev->device->node_type) != RDMA_TRANSPORT_IB)
> + !rdma_ib_or_iboe(cma_dev->device, 1))
> return;
>
> id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,
> @@ -2028,7 +2018,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)
> mutex_lock(&lock);
> list_for_each_entry(cur_dev, &dev_list, list) {
> if (cma_family(id_priv) == AF_IB &&
> - rdma_node_get_transport(cur_dev->device->node_type) != RDMA_TRANSPORT_IB)
> + !rdma_ib_or_iboe(cur_dev->device, 1))
> continue;
>
> if (!cma_dev)
> @@ -2060,7 +2050,7 @@ port_found:
> goto out;
>
> id_priv->id.route.addr.dev_addr.dev_type =
> - (rdma_port_get_link_layer(cma_dev->device, p) == IB_LINK_LAYER_INFINIBAND) ?
> + (rdma_tech_ib(cma_dev->device, p)) ?
> ARPHRD_INFINIBAND : ARPHRD_ETHER;
>
> rdma_addr_set_sgid(&id_priv->id.route.addr.dev_addr, &gid);
> @@ -2537,18 +2527,15 @@ int rdma_listen(struct rdma_cm_id *id, int backlog)
>
> id_priv->backlog = backlog;
> if (id->device) {
> - switch (rdma_node_get_transport(id->device->node_type)) {
> - case RDMA_TRANSPORT_IB:
> + if (rdma_ib_or_iboe(id->device, 1)) {
> ret = cma_ib_listen(id_priv);
> if (ret)
> goto err;
> - break;
> - case RDMA_TRANSPORT_IWARP:
> + } else if (rdma_tech_iwarp(id->device, 1)) {
> ret = cma_iw_listen(id_priv, backlog);
> if (ret)
> goto err;
> - break;
> - default:
> + } else {
> ret = -ENOSYS;
> goto err;
> }
> @@ -2884,20 +2871,15 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
> id_priv->srq = conn_param->srq;
> }
>
> - switch (rdma_node_get_transport(id->device->node_type)) {
> - case RDMA_TRANSPORT_IB:
> + if (rdma_ib_or_iboe(id->device, 1)) {
> if (id->qp_type == IB_QPT_UD)
> ret = cma_resolve_ib_udp(id_priv, conn_param);
> else
> ret = cma_connect_ib(id_priv, conn_param);
> - break;
> - case RDMA_TRANSPORT_IWARP:
> + } else if (rdma_tech_iwarp(id->device, 1))
> ret = cma_connect_iw(id_priv, conn_param);
> - break;
> - default:
> + else
> ret = -ENOSYS;
> - break;
> - }
> if (ret)
> goto err;
>
> @@ -3000,8 +2982,7 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
> id_priv->srq = conn_param->srq;
> }
>
> - switch (rdma_node_get_transport(id->device->node_type)) {
> - case RDMA_TRANSPORT_IB:
> + if (rdma_ib_or_iboe(id->device, 1)) {
> if (id->qp_type == IB_QPT_UD) {
> if (conn_param)
> ret = cma_send_sidr_rep(id_priv, IB_SIDR_SUCCESS,
> @@ -3017,14 +2998,10 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
> else
> ret = cma_rep_recv(id_priv);
> }
> - break;
> - case RDMA_TRANSPORT_IWARP:
> + } else if (rdma_tech_iwarp(id->device, 1))
> ret = cma_accept_iw(id_priv, conn_param);
> - break;
> - default:
> + else
> ret = -ENOSYS;
> - break;
> - }
>
> if (ret)
> goto reject;
> @@ -3068,8 +3045,7 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,
> if (!id_priv->cm_id.ib)
> return -EINVAL;
>
> - switch (rdma_node_get_transport(id->device->node_type)) {
> - case RDMA_TRANSPORT_IB:
> + if (rdma_ib_or_iboe(id->device, 1)) {
> if (id->qp_type == IB_QPT_UD)
> ret = cma_send_sidr_rep(id_priv, IB_SIDR_REJECT, 0,
> private_data, private_data_len);
> @@ -3077,15 +3053,12 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,
> ret = ib_send_cm_rej(id_priv->cm_id.ib,
> IB_CM_REJ_CONSUMER_DEFINED, NULL,
> 0, private_data, private_data_len);
> - break;
> - case RDMA_TRANSPORT_IWARP:
> + } else if (rdma_tech_iwarp(id->device, 1)) {
> ret = iw_cm_reject(id_priv->cm_id.iw,
> private_data, private_data_len);
> - break;
> - default:
> + } else
> ret = -ENOSYS;
> - break;
> - }
> +
> return ret;
> }
> EXPORT_SYMBOL(rdma_reject);
> @@ -3099,22 +3072,18 @@ int rdma_disconnect(struct rdma_cm_id *id)
> if (!id_priv->cm_id.ib)
> return -EINVAL;
>
> - switch (rdma_node_get_transport(id->device->node_type)) {
> - case RDMA_TRANSPORT_IB:
> + if (rdma_ib_or_iboe(id->device, 1)) {
> ret = cma_modify_qp_err(id_priv);
> if (ret)
> goto out;
> /* Initiate or respond to a disconnect. */
> if (ib_send_cm_dreq(id_priv->cm_id.ib, NULL, 0))
> ib_send_cm_drep(id_priv->cm_id.ib, NULL, 0);
> - break;
> - case RDMA_TRANSPORT_IWARP:
> + } else if (rdma_tech_iwarp(id->device, 1)) {
> ret = iw_cm_disconnect(id_priv->cm_id.iw, 0);
> - break;
> - default:
> + } else
> ret = -EINVAL;
> - break;
> - }
> +
> out:
> return ret;
> }
> diff --git a/drivers/infiniband/core/ucm.c b/drivers/infiniband/core/ucm.c
> index f2f6393..70e0ccb 100644
> --- a/drivers/infiniband/core/ucm.c
> +++ b/drivers/infiniband/core/ucm.c
> @@ -1253,8 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)
> dev_t base;
> struct ib_ucm_device *ucm_dev;
>
> - if (!device->alloc_ucontext ||
> - rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
> + if (!device->alloc_ucontext || !rdma_ib_or_iboe(device, 1))
> return;
>
> ucm_dev = kzalloc(sizeof *ucm_dev, GFP_KERNEL);
On 4/16/2015 4:09 AM, Michael Wang wrote:
>
> Reform cma_acquire_dev() with management helpers, introduce
> cma_validate_port() to make the code more clean.
>
> Cc: Steve Wise <[email protected]>
> Cc: Tom Talpey <[email protected]>
> Cc: Jason Gunthorpe <[email protected]>
> Cc: Doug Ledford <[email protected]>
> Cc: Ira Weiny <[email protected]>
> Cc: Sean Hefty <[email protected]>
> Signed-off-by: Michael Wang <[email protected]>
> ---
> drivers/infiniband/core/cma.c | 69 +++++++++++++++++++++++++------------------
> 1 file changed, 41 insertions(+), 28 deletions(-)
>
> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> index b520882..902cc1a 100644
> --- a/drivers/infiniband/core/cma.c
> +++ b/drivers/infiniband/core/cma.c
> @@ -370,18 +370,36 @@ static int cma_translate_addr(struct sockaddr *addr, struct rdma_dev_addr *dev_a
> return ret;
> }
>
> +static inline int cma_validate_port(struct ib_device *device, u8 port,
> + union ib_gid *gid, int dev_type)
> +{
> + u8 found_port;
> + int ret = -ENODEV;
> +
> + if ((dev_type == ARPHRD_INFINIBAND) && !rdma_tech_ib(device, port))
> + return ret;
> +
> + if ((dev_type != ARPHRD_INFINIBAND) && rdma_tech_ib(device, port))
> + return ret;
> +
> + ret = ib_find_cached_gid(device, gid, &found_port, NULL);
> +
> + if (!ret && (port == found_port))
> + return 0;
> +
> + return ret;
Should the case where ret = 0 and port != found_port need to be handled
the same as currently ? It looks different to me since in this case the
port will be saved into id_priv->id.port_num whereas currently it isn't.
-- Hal
> +}
> +
> static int cma_acquire_dev(struct rdma_id_private *id_priv,
> struct rdma_id_private *listen_id_priv)
> {
> struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;
> struct cma_device *cma_dev;
> - union ib_gid gid, iboe_gid;
> + union ib_gid gid, iboe_gid, *gidp;
> int ret = -ENODEV;
> - u8 port, found_port;
> - enum rdma_link_layer dev_ll = dev_addr->dev_type == ARPHRD_INFINIBAND ?
> - IB_LINK_LAYER_INFINIBAND : IB_LINK_LAYER_ETHERNET;
> + u8 port;
>
> - if (dev_ll != IB_LINK_LAYER_INFINIBAND &&
> + if (dev_addr->dev_type != ARPHRD_INFINIBAND &&
> id_priv->id.ps == RDMA_PS_IPOIB)
> return -EINVAL;
>
> @@ -391,41 +409,36 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv,
>
> memcpy(&gid, dev_addr->src_dev_addr +
> rdma_addr_gid_offset(dev_addr), sizeof gid);
> - if (listen_id_priv &&
> - rdma_port_get_link_layer(listen_id_priv->id.device,
> - listen_id_priv->id.port_num) == dev_ll) {
> +
> + if (listen_id_priv) {
> cma_dev = listen_id_priv->cma_dev;
> port = listen_id_priv->id.port_num;
> - if (rdma_node_get_transport(cma_dev->device->node_type) == RDMA_TRANSPORT_IB &&
> - rdma_port_get_link_layer(cma_dev->device, port) == IB_LINK_LAYER_ETHERNET)
> - ret = ib_find_cached_gid(cma_dev->device, &iboe_gid,
> - &found_port, NULL);
> - else
> - ret = ib_find_cached_gid(cma_dev->device, &gid,
> - &found_port, NULL);
> + gidp = rdma_tech_iboe(cma_dev->device, port) ?
> + &iboe_gid : &gid;
>
> - if (!ret && (port == found_port)) {
> - id_priv->id.port_num = found_port;
> + ret = cma_validate_port(cma_dev->device, port, gidp,
> + dev_addr->dev_type);
> + if (!ret) {
> + id_priv->id.port_num = port;
> goto out;
> }
> }
> +
> list_for_each_entry(cma_dev, &dev_list, list) {
> for (port = 1; port <= cma_dev->device->phys_port_cnt; ++port) {
> if (listen_id_priv &&
> listen_id_priv->cma_dev == cma_dev &&
> listen_id_priv->id.port_num == port)
> continue;
> - if (rdma_port_get_link_layer(cma_dev->device, port) == dev_ll) {
> - if (rdma_node_get_transport(cma_dev->device->node_type) == RDMA_TRANSPORT_IB &&
> - rdma_port_get_link_layer(cma_dev->device, port) == IB_LINK_LAYER_ETHERNET)
> - ret = ib_find_cached_gid(cma_dev->device, &iboe_gid, &found_port, NULL);
> - else
> - ret = ib_find_cached_gid(cma_dev->device, &gid, &found_port, NULL);
> -
> - if (!ret && (port == found_port)) {
> - id_priv->id.port_num = found_port;
> - goto out;
> - }
> +
> + gidp = rdma_tech_iboe(cma_dev->device, port) ?
> + &iboe_gid : &gid;
> +
> + ret = cma_validate_port(cma_dev->device, port, gidp,
> + dev_addr->dev_type);
> + if (!ret) {
> + id_priv->id.port_num = port;
> + goto out;
> }
> }
> }
On 04/16/2015 03:10 PM, Hal Rosenstock wrote:
> On 4/16/2015 4:08 AM, Michael Wang wrote:
>>
>> Use raw management helpers to reform cm related part in IB-core cma/ucm.
>>
>> These checks focus on the device cm type rather than the port capability,
>> directly pass port 1 works currently, but can't support mixing cm type
>> device in future.
>
> This is equivalent to today where the checks are per node rather than
> per port.
>
> Should all checks here be port 1 based or only certain ones like listen
> ? For example, in connect/reject/disconnect, don't we already have port
> ? Guess this can be dealt with later as this is not a regression from
> the current implementation.
Yeah, these parts of cma may need more carve in future, like some new callback
for different CM type as Sean suggested.
Maybe directly using 1 could help to highlight the problem ;-)
Regards,
Michael Wanga
>
> -- Hal
>
>>
>> Cc: Steve Wise <[email protected]>
>> Cc: Tom Talpey <[email protected]>
>> Cc: Jason Gunthorpe <[email protected]>
>> Cc: Doug Ledford <[email protected]>
>> Cc: Ira Weiny <[email protected]>
>> Cc: Sean Hefty <[email protected]>
>> Signed-off-by: Michael Wang <[email protected]>
>> ---
>> drivers/infiniband/core/cma.c | 81 +++++++++++++------------------------------
>> drivers/infiniband/core/ucm.c | 3 +-
>> 2 files changed, 26 insertions(+), 58 deletions(-)
>>
>> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
>> index d570030..6b8a64d 100644
>> --- a/drivers/infiniband/core/cma.c
>> +++ b/drivers/infiniband/core/cma.c
>> @@ -735,8 +735,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
>> int ret = 0;
>>
>> id_priv = container_of(id, struct rdma_id_private, id);
>> - switch (rdma_node_get_transport(id_priv->id.device->node_type)) {
>> - case RDMA_TRANSPORT_IB:
>> + if (rdma_ib_or_iboe(id_priv->id.device, 1)) {
>> if (!id_priv->cm_id.ib || (id_priv->id.qp_type == IB_QPT_UD))
>> ret = cma_ib_init_qp_attr(id_priv, qp_attr, qp_attr_mask);
>> else
>> @@ -745,19 +744,15 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
>>
>> if (qp_attr->qp_state == IB_QPS_RTR)
>> qp_attr->rq_psn = id_priv->seq_num;
>> - break;
>> - case RDMA_TRANSPORT_IWARP:
>> + } else if (rdma_tech_iwarp(id_priv->id.device, 1)) {
>> if (!id_priv->cm_id.iw) {
>> qp_attr->qp_access_flags = 0;
>> *qp_attr_mask = IB_QP_STATE | IB_QP_ACCESS_FLAGS;
>> } else
>> ret = iw_cm_init_qp_attr(id_priv->cm_id.iw, qp_attr,
>> qp_attr_mask);
>> - break;
>> - default:
>> + } else
>> ret = -ENOSYS;
>> - break;
>> - }
>>
>> return ret;
>> }
>> @@ -1037,17 +1032,12 @@ void rdma_destroy_id(struct rdma_cm_id *id)
>> mutex_unlock(&id_priv->handler_mutex);
>>
>> if (id_priv->cma_dev) {
>> - switch (rdma_node_get_transport(id_priv->id.device->node_type)) {
>> - case RDMA_TRANSPORT_IB:
>> + if (rdma_ib_or_iboe(id_priv->id.device, 1)) {
>> if (id_priv->cm_id.ib)
>> ib_destroy_cm_id(id_priv->cm_id.ib);
>> - break;
>> - case RDMA_TRANSPORT_IWARP:
>> + } else if (rdma_tech_iwarp(id_priv->id.device, 1)) {
>> if (id_priv->cm_id.iw)
>> iw_destroy_cm_id(id_priv->cm_id.iw);
>> - break;
>> - default:
>> - break;
>> }
>> cma_leave_mc_groups(id_priv);
>> cma_release_dev(id_priv);
>> @@ -1626,7 +1616,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,
>> int ret;
>>
>> if (cma_family(id_priv) == AF_IB &&
>> - rdma_node_get_transport(cma_dev->device->node_type) != RDMA_TRANSPORT_IB)
>> + !rdma_ib_or_iboe(cma_dev->device, 1))
>> return;
>>
>> id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,
>> @@ -2028,7 +2018,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)
>> mutex_lock(&lock);
>> list_for_each_entry(cur_dev, &dev_list, list) {
>> if (cma_family(id_priv) == AF_IB &&
>> - rdma_node_get_transport(cur_dev->device->node_type) != RDMA_TRANSPORT_IB)
>> + !rdma_ib_or_iboe(cur_dev->device, 1))
>> continue;
>>
>> if (!cma_dev)
>> @@ -2060,7 +2050,7 @@ port_found:
>> goto out;
>>
>> id_priv->id.route.addr.dev_addr.dev_type =
>> - (rdma_port_get_link_layer(cma_dev->device, p) == IB_LINK_LAYER_INFINIBAND) ?
>> + (rdma_tech_ib(cma_dev->device, p)) ?
>> ARPHRD_INFINIBAND : ARPHRD_ETHER;
>>
>> rdma_addr_set_sgid(&id_priv->id.route.addr.dev_addr, &gid);
>> @@ -2537,18 +2527,15 @@ int rdma_listen(struct rdma_cm_id *id, int backlog)
>>
>> id_priv->backlog = backlog;
>> if (id->device) {
>> - switch (rdma_node_get_transport(id->device->node_type)) {
>> - case RDMA_TRANSPORT_IB:
>> + if (rdma_ib_or_iboe(id->device, 1)) {
>> ret = cma_ib_listen(id_priv);
>> if (ret)
>> goto err;
>> - break;
>> - case RDMA_TRANSPORT_IWARP:
>> + } else if (rdma_tech_iwarp(id->device, 1)) {
>> ret = cma_iw_listen(id_priv, backlog);
>> if (ret)
>> goto err;
>> - break;
>> - default:
>> + } else {
>> ret = -ENOSYS;
>> goto err;
>> }
>> @@ -2884,20 +2871,15 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
>> id_priv->srq = conn_param->srq;
>> }
>>
>> - switch (rdma_node_get_transport(id->device->node_type)) {
>> - case RDMA_TRANSPORT_IB:
>> + if (rdma_ib_or_iboe(id->device, 1)) {
>> if (id->qp_type == IB_QPT_UD)
>> ret = cma_resolve_ib_udp(id_priv, conn_param);
>> else
>> ret = cma_connect_ib(id_priv, conn_param);
>> - break;
>> - case RDMA_TRANSPORT_IWARP:
>> + } else if (rdma_tech_iwarp(id->device, 1))
>> ret = cma_connect_iw(id_priv, conn_param);
>> - break;
>> - default:
>> + else
>> ret = -ENOSYS;
>> - break;
>> - }
>> if (ret)
>> goto err;
>>
>> @@ -3000,8 +2982,7 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
>> id_priv->srq = conn_param->srq;
>> }
>>
>> - switch (rdma_node_get_transport(id->device->node_type)) {
>> - case RDMA_TRANSPORT_IB:
>> + if (rdma_ib_or_iboe(id->device, 1)) {
>> if (id->qp_type == IB_QPT_UD) {
>> if (conn_param)
>> ret = cma_send_sidr_rep(id_priv, IB_SIDR_SUCCESS,
>> @@ -3017,14 +2998,10 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
>> else
>> ret = cma_rep_recv(id_priv);
>> }
>> - break;
>> - case RDMA_TRANSPORT_IWARP:
>> + } else if (rdma_tech_iwarp(id->device, 1))
>> ret = cma_accept_iw(id_priv, conn_param);
>> - break;
>> - default:
>> + else
>> ret = -ENOSYS;
>> - break;
>> - }
>>
>> if (ret)
>> goto reject;
>> @@ -3068,8 +3045,7 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,
>> if (!id_priv->cm_id.ib)
>> return -EINVAL;
>>
>> - switch (rdma_node_get_transport(id->device->node_type)) {
>> - case RDMA_TRANSPORT_IB:
>> + if (rdma_ib_or_iboe(id->device, 1)) {
>> if (id->qp_type == IB_QPT_UD)
>> ret = cma_send_sidr_rep(id_priv, IB_SIDR_REJECT, 0,
>> private_data, private_data_len);
>> @@ -3077,15 +3053,12 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,
>> ret = ib_send_cm_rej(id_priv->cm_id.ib,
>> IB_CM_REJ_CONSUMER_DEFINED, NULL,
>> 0, private_data, private_data_len);
>> - break;
>> - case RDMA_TRANSPORT_IWARP:
>> + } else if (rdma_tech_iwarp(id->device, 1)) {
>> ret = iw_cm_reject(id_priv->cm_id.iw,
>> private_data, private_data_len);
>> - break;
>> - default:
>> + } else
>> ret = -ENOSYS;
>> - break;
>> - }
>> +
>> return ret;
>> }
>> EXPORT_SYMBOL(rdma_reject);
>> @@ -3099,22 +3072,18 @@ int rdma_disconnect(struct rdma_cm_id *id)
>> if (!id_priv->cm_id.ib)
>> return -EINVAL;
>>
>> - switch (rdma_node_get_transport(id->device->node_type)) {
>> - case RDMA_TRANSPORT_IB:
>> + if (rdma_ib_or_iboe(id->device, 1)) {
>> ret = cma_modify_qp_err(id_priv);
>> if (ret)
>> goto out;
>> /* Initiate or respond to a disconnect. */
>> if (ib_send_cm_dreq(id_priv->cm_id.ib, NULL, 0))
>> ib_send_cm_drep(id_priv->cm_id.ib, NULL, 0);
>> - break;
>> - case RDMA_TRANSPORT_IWARP:
>> + } else if (rdma_tech_iwarp(id->device, 1)) {
>> ret = iw_cm_disconnect(id_priv->cm_id.iw, 0);
>> - break;
>> - default:
>> + } else
>> ret = -EINVAL;
>> - break;
>> - }
>> +
>> out:
>> return ret;
>> }
>> diff --git a/drivers/infiniband/core/ucm.c b/drivers/infiniband/core/ucm.c
>> index f2f6393..70e0ccb 100644
>> --- a/drivers/infiniband/core/ucm.c
>> +++ b/drivers/infiniband/core/ucm.c
>> @@ -1253,8 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)
>> dev_t base;
>> struct ib_ucm_device *ucm_dev;
>>
>> - if (!device->alloc_ucontext ||
>> - rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
>> + if (!device->alloc_ucontext || !rdma_ib_or_iboe(device, 1))
>> return;
>>
>> ucm_dev = kzalloc(sizeof *ucm_dev, GFP_KERNEL);
>
On 04/16/2015 03:19 PM, Hal Rosenstock wrote:
[snip]
>>
>> +static inline int cma_validate_port(struct ib_device *device, u8 port,
>> + union ib_gid *gid, int dev_type)
>> +{
>> + u8 found_port;
>> + int ret = -ENODEV;
>> +
>> + if ((dev_type == ARPHRD_INFINIBAND) && !rdma_tech_ib(device, port))
>> + return ret;
>> +
>> + if ((dev_type != ARPHRD_INFINIBAND) && rdma_tech_ib(device, port))
>> + return ret;
>> +
>> + ret = ib_find_cached_gid(device, gid, &found_port, NULL);
>> +
>> + if (!ret && (port == found_port))
>> + return 0;
>> +
>> + return ret;
>
> Should the case where ret = 0 and port != found_port need to be handled
> the same as currently ? It looks different to me since in this case the
> port will be saved into id_priv->id.port_num whereas currently it isn't.
I get your point :-) what about:
ret = ib_find_cached_gid(device, gid, &found_port, NULL);
if (port != found_port)
return -ENODEV;
return ret
Regards,
Michael Wang
>
> -- Hal
>
>> +}
>> +
>> static int cma_acquire_dev(struct rdma_id_private *id_priv,
>> struct rdma_id_private *listen_id_priv)
>> {
>> struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;
>> struct cma_device *cma_dev;
>> - union ib_gid gid, iboe_gid;
>> + union ib_gid gid, iboe_gid, *gidp;
>> int ret = -ENODEV;
>> - u8 port, found_port;
>> - enum rdma_link_layer dev_ll = dev_addr->dev_type == ARPHRD_INFINIBAND ?
>> - IB_LINK_LAYER_INFINIBAND : IB_LINK_LAYER_ETHERNET;
>> + u8 port;
>>
>> - if (dev_ll != IB_LINK_LAYER_INFINIBAND &&
>> + if (dev_addr->dev_type != ARPHRD_INFINIBAND &&
>> id_priv->id.ps == RDMA_PS_IPOIB)
>> return -EINVAL;
>>
>> @@ -391,41 +409,36 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv,
>>
>> memcpy(&gid, dev_addr->src_dev_addr +
>> rdma_addr_gid_offset(dev_addr), sizeof gid);
>> - if (listen_id_priv &&
>> - rdma_port_get_link_layer(listen_id_priv->id.device,
>> - listen_id_priv->id.port_num) == dev_ll) {
>> +
>> + if (listen_id_priv) {
>> cma_dev = listen_id_priv->cma_dev;
>> port = listen_id_priv->id.port_num;
>> - if (rdma_node_get_transport(cma_dev->device->node_type) == RDMA_TRANSPORT_IB &&
>> - rdma_port_get_link_layer(cma_dev->device, port) == IB_LINK_LAYER_ETHERNET)
>> - ret = ib_find_cached_gid(cma_dev->device, &iboe_gid,
>> - &found_port, NULL);
>> - else
>> - ret = ib_find_cached_gid(cma_dev->device, &gid,
>> - &found_port, NULL);
>> + gidp = rdma_tech_iboe(cma_dev->device, port) ?
>> + &iboe_gid : &gid;
>>
>> - if (!ret && (port == found_port)) {
>> - id_priv->id.port_num = found_port;
>> + ret = cma_validate_port(cma_dev->device, port, gidp,
>> + dev_addr->dev_type);
>> + if (!ret) {
>> + id_priv->id.port_num = port;
>> goto out;
>> }
>> }
>> +
>> list_for_each_entry(cma_dev, &dev_list, list) {
>> for (port = 1; port <= cma_dev->device->phys_port_cnt; ++port) {
>> if (listen_id_priv &&
>> listen_id_priv->cma_dev == cma_dev &&
>> listen_id_priv->id.port_num == port)
>> continue;
>> - if (rdma_port_get_link_layer(cma_dev->device, port) == dev_ll) {
>> - if (rdma_node_get_transport(cma_dev->device->node_type) == RDMA_TRANSPORT_IB &&
>> - rdma_port_get_link_layer(cma_dev->device, port) == IB_LINK_LAYER_ETHERNET)
>> - ret = ib_find_cached_gid(cma_dev->device, &iboe_gid, &found_port, NULL);
>> - else
>> - ret = ib_find_cached_gid(cma_dev->device, &gid, &found_port, NULL);
>> -
>> - if (!ret && (port == found_port)) {
>> - id_priv->id.port_num = found_port;
>> - goto out;
>> - }
>> +
>> + gidp = rdma_tech_iboe(cma_dev->device, port) ?
>> + &iboe_gid : &gid;
>> +
>> + ret = cma_validate_port(cma_dev->device, port, gidp,
>> + dev_addr->dev_type);
>> + if (!ret) {
>> + id_priv->id.port_num = port;
>> + goto out;
>> }
>> }
>> }
>
On 4/16/2015 4:12 AM, Michael Wang wrote:
>
> Introduce helper cap_ib_sa() to help us check if the port of an
> IB device support Infiniband Subnet Administrator.
Nit: Administrator -> Administration
>
> Cc: Steve Wise <[email protected]>
> Cc: Tom Talpey <[email protected]>
> Cc: Jason Gunthorpe <[email protected]>
> Cc: Doug Ledford <[email protected]>
> Cc: Ira Weiny <[email protected]>
> Cc: Sean Hefty <[email protected]>
> Signed-off-by: Michael Wang <[email protected]>
> ---
> drivers/infiniband/core/cma.c | 4 ++--
> drivers/infiniband/core/sa_query.c | 10 +++++-----
> drivers/infiniband/core/ucma.c | 2 +-
> include/rdma/ib_verbs.h | 15 +++++++++++++++
> 4 files changed, 23 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> index a6f1526..094816f 100644
> --- a/drivers/infiniband/core/cma.c
> +++ b/drivers/infiniband/core/cma.c
> @@ -955,7 +955,7 @@ static inline int cma_user_data_offset(struct rdma_id_private *id_priv)
>
> static void cma_cancel_route(struct rdma_id_private *id_priv)
> {
> - if (rdma_tech_ib(id_priv->id.device, id_priv->id.port_num)) {
> + if (cap_ib_sa(id_priv->id.device, id_priv->id.port_num)) {
> if (id_priv->query)
> ib_sa_cancel_query(id_priv->query_id, id_priv->query);
> }
> @@ -1979,7 +1979,7 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)
> return -EINVAL;
>
> atomic_inc(&id_priv->refcount);
> - if (rdma_tech_ib(id->device, id->port_num))
> + if (cap_ib_sa(id->device, id->port_num))
> ret = cma_resolve_ib_route(id_priv, timeout_ms);
> else if (rdma_tech_iboe(id->device, id->port_num))
> ret = cma_resolve_iboe_route(id_priv);
> diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
> index 803ccf7..fc7e161 100644
> --- a/drivers/infiniband/core/sa_query.c
> +++ b/drivers/infiniband/core/sa_query.c
> @@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *event
> struct ib_sa_port *port =
> &sa_dev->port[event->element.port_num - sa_dev->start_port];
>
> - if (WARN_ON(!rdma_tech_ib(handler->device, port->port_num)))
> + if (WARN_ON(!cap_ib_sa(handler->device, port->port_num)))
> return;
>
> spin_lock_irqsave(&port->ah_lock, flags);
> @@ -1173,7 +1173,7 @@ static void ib_sa_add_one(struct ib_device *device)
>
> for (i = 0; i <= e - s; ++i) {
> spin_lock_init(&sa_dev->port[i].ah_lock);
> - if (!rdma_tech_ib(device, i + 1))
> + if (!cap_ib_sa(device, i + 1))
> continue;
>
> sa_dev->port[i].sm_ah = NULL;
> @@ -1210,7 +1210,7 @@ static void ib_sa_add_one(struct ib_device *device)
> goto err;
>
> for (i = 0; i <= e - s; ++i) {
> - if (rdma_tech_ib(device, i + 1))
> + if (cap_ib_sa(device, i + 1))
> update_sm_ah(&sa_dev->port[i].update_task);
> }
>
> @@ -1218,7 +1218,7 @@ static void ib_sa_add_one(struct ib_device *device)
>
> err:
> while (--i >= 0) {
> - if (rdma_tech_ib(device, i + 1))
> + if (cap_ib_sa(device, i + 1))
> ib_unregister_mad_agent(sa_dev->port[i].agent);
> }
>
> @@ -1240,7 +1240,7 @@ static void ib_sa_remove_one(struct ib_device *device)
> flush_workqueue(ib_wq);
>
> for (i = 0; i <= sa_dev->end_port - sa_dev->start_port; ++i) {
> - if (rdma_tech_ib(device, i + 1)) {
> + if (cap_ib_sa(device, i + 1)) {
> ib_unregister_mad_agent(sa_dev->port[i].agent);
> if (sa_dev->port[i].sm_ah)
> kref_put(&sa_dev->port[i].sm_ah->ref, free_sm_ah);
> diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
> index 7331c6c..bed7957 100644
> --- a/drivers/infiniband/core/ucma.c
> +++ b/drivers/infiniband/core/ucma.c
> @@ -723,7 +723,7 @@ static ssize_t ucma_query_route(struct ucma_file *file,
> resp.node_guid = (__force __u64) ctx->cm_id->device->node_guid;
> resp.port_num = ctx->cm_id->port_num;
>
> - if (rdma_tech_ib(ctx->cm_id->device, ctx->cm_id->port_num))
> + if (cap_ib_sa(ctx->cm_id->device, ctx->cm_id->port_num))
> ucma_copy_ib_route(&resp, &ctx->cm_id->route);
> else if (rdma_tech_iboe(ctx->cm_id->device, ctx->cm_id->port_num))
> ucma_copy_iboe_route(&resp, &ctx->cm_id->route);
> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
> index e4999f6..3bfdf81 100644
> --- a/include/rdma/ib_verbs.h
> +++ b/include/rdma/ib_verbs.h
> @@ -1833,6 +1833,21 @@ static inline int cap_iw_cm(struct ib_device *device, u8 port_num)
> return rdma_tech_iwarp(device, port_num);
> }
>
> +/**
> + * cap_ib_sa - Check if the port of device has the capability Infiniband
> + * Subnet Administrator.
Nit: Administrator -> Administration
> + *
> + * @device: Device to be checked
> + * @port_num: Port number of the device
> + *
> + * Return 0 when port of the device don't support Infiniband
> + * Subnet Administrator.
Nit: Administrator -> Administration
> + */
> +static inline int cap_ib_sa(struct ib_device *device, u8 port_num)
> +{
> + return rdma_tech_ib(device, port_num);
> +}
> +
> int ib_query_gid(struct ib_device *device,
> u8 port_num, int index, union ib_gid *gid);
>
On 4/16/2015 4:15 AM, Michael Wang wrote:
>
> We have get rid of all the scene using legacy rdma_node_get_transport(),
> now clean it up.
>
> Cc: Steve Wise <[email protected]>
> Cc: Tom Talpey <[email protected]>
> Cc: Jason Gunthorpe <[email protected]>
> Cc: Doug Ledford <[email protected]>
> Cc: Ira Weiny <[email protected]>
> Cc: Sean Hefty <[email protected]>
> Signed-off-by: Michael Wang <[email protected]>
> ---
> drivers/infiniband/core/verbs.c | 21 ---------------------
> include/rdma/ib_verbs.h | 3 ---
> 2 files changed, 24 deletions(-)
>
> diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
> index ee4b5cb..bbea0c0 100644
> --- a/drivers/infiniband/core/verbs.c
> +++ b/drivers/infiniband/core/verbs.c
> @@ -107,27 +107,6 @@ __attribute_const__ int ib_rate_to_mbps(enum ib_rate rate)
> }
> EXPORT_SYMBOL(ib_rate_to_mbps);
>
> -__attribute_const__ enum rdma_transport_type
> -rdma_node_get_transport(enum rdma_node_type node_type)
> -{
> - switch (node_type) {
> - case RDMA_NODE_IB_CA:
> - case RDMA_NODE_IB_SWITCH:
> - case RDMA_NODE_IB_ROUTER:
> - return RDMA_TRANSPORT_IB;
> - case RDMA_NODE_RNIC:
> - return RDMA_TRANSPORT_IWARP;
> - case RDMA_NODE_USNIC:
> - return RDMA_TRANSPORT_USNIC;
> - case RDMA_NODE_USNIC_UDP:
> - return RDMA_TRANSPORT_USNIC_UDP;
> - default:
> - BUG();
> - return 0;
> - }
> -}
> -EXPORT_SYMBOL(rdma_node_get_transport);
> -
> enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device, u8 port_num)
> {
> if (device->get_link_layer)
> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
> index 262bf44..f9ef479 100644
> --- a/include/rdma/ib_verbs.h
> +++ b/include/rdma/ib_verbs.h
> @@ -84,9 +84,6 @@ enum rdma_transport_type {
> RDMA_TRANSPORT_IBOE,
> };
>
> -__attribute_const__ enum rdma_transport_type
> -rdma_node_get_transport(enum rdma_node_type node_type);
> -
> enum rdma_link_layer {
> IB_LINK_LAYER_UNSPECIFIED,
Is IB_LINK_LAYER_UNSPECIFIED still possible ?
> IB_LINK_LAYER_INFINIBAND,
On 04/16/2015 03:36 PM, Hal Rosenstock wrote:
> On 4/16/2015 4:12 AM, Michael Wang wrote:
>>
>> Introduce helper cap_ib_sa() to help us check if the port of an
>> IB device support Infiniband Subnet Administrator.
>
> Nit: Administrator -> Administration
Will be in next version :-)
Regards,
Michael Wang
>
>>
>> Cc: Steve Wise <[email protected]>
>> Cc: Tom Talpey <[email protected]>
>> Cc: Jason Gunthorpe <[email protected]>
>> Cc: Doug Ledford <[email protected]>
>> Cc: Ira Weiny <[email protected]>
>> Cc: Sean Hefty <[email protected]>
>> Signed-off-by: Michael Wang <[email protected]>
>> ---
>> drivers/infiniband/core/cma.c | 4 ++--
>> drivers/infiniband/core/sa_query.c | 10 +++++-----
>> drivers/infiniband/core/ucma.c | 2 +-
>> include/rdma/ib_verbs.h | 15 +++++++++++++++
>> 4 files changed, 23 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
>> index a6f1526..094816f 100644
>> --- a/drivers/infiniband/core/cma.c
>> +++ b/drivers/infiniband/core/cma.c
>> @@ -955,7 +955,7 @@ static inline int cma_user_data_offset(struct rdma_id_private *id_priv)
>>
>> static void cma_cancel_route(struct rdma_id_private *id_priv)
>> {
>> - if (rdma_tech_ib(id_priv->id.device, id_priv->id.port_num)) {
>> + if (cap_ib_sa(id_priv->id.device, id_priv->id.port_num)) {
>> if (id_priv->query)
>> ib_sa_cancel_query(id_priv->query_id, id_priv->query);
>> }
>> @@ -1979,7 +1979,7 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)
>> return -EINVAL;
>>
>> atomic_inc(&id_priv->refcount);
>> - if (rdma_tech_ib(id->device, id->port_num))
>> + if (cap_ib_sa(id->device, id->port_num))
>> ret = cma_resolve_ib_route(id_priv, timeout_ms);
>> else if (rdma_tech_iboe(id->device, id->port_num))
>> ret = cma_resolve_iboe_route(id_priv);
>> diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
>> index 803ccf7..fc7e161 100644
>> --- a/drivers/infiniband/core/sa_query.c
>> +++ b/drivers/infiniband/core/sa_query.c
>> @@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *event
>> struct ib_sa_port *port =
>> &sa_dev->port[event->element.port_num - sa_dev->start_port];
>>
>> - if (WARN_ON(!rdma_tech_ib(handler->device, port->port_num)))
>> + if (WARN_ON(!cap_ib_sa(handler->device, port->port_num)))
>> return;
>>
>> spin_lock_irqsave(&port->ah_lock, flags);
>> @@ -1173,7 +1173,7 @@ static void ib_sa_add_one(struct ib_device *device)
>>
>> for (i = 0; i <= e - s; ++i) {
>> spin_lock_init(&sa_dev->port[i].ah_lock);
>> - if (!rdma_tech_ib(device, i + 1))
>> + if (!cap_ib_sa(device, i + 1))
>> continue;
>>
>> sa_dev->port[i].sm_ah = NULL;
>> @@ -1210,7 +1210,7 @@ static void ib_sa_add_one(struct ib_device *device)
>> goto err;
>>
>> for (i = 0; i <= e - s; ++i) {
>> - if (rdma_tech_ib(device, i + 1))
>> + if (cap_ib_sa(device, i + 1))
>> update_sm_ah(&sa_dev->port[i].update_task);
>> }
>>
>> @@ -1218,7 +1218,7 @@ static void ib_sa_add_one(struct ib_device *device)
>>
>> err:
>> while (--i >= 0) {
>> - if (rdma_tech_ib(device, i + 1))
>> + if (cap_ib_sa(device, i + 1))
>> ib_unregister_mad_agent(sa_dev->port[i].agent);
>> }
>>
>> @@ -1240,7 +1240,7 @@ static void ib_sa_remove_one(struct ib_device *device)
>> flush_workqueue(ib_wq);
>>
>> for (i = 0; i <= sa_dev->end_port - sa_dev->start_port; ++i) {
>> - if (rdma_tech_ib(device, i + 1)) {
>> + if (cap_ib_sa(device, i + 1)) {
>> ib_unregister_mad_agent(sa_dev->port[i].agent);
>> if (sa_dev->port[i].sm_ah)
>> kref_put(&sa_dev->port[i].sm_ah->ref, free_sm_ah);
>> diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
>> index 7331c6c..bed7957 100644
>> --- a/drivers/infiniband/core/ucma.c
>> +++ b/drivers/infiniband/core/ucma.c
>> @@ -723,7 +723,7 @@ static ssize_t ucma_query_route(struct ucma_file *file,
>> resp.node_guid = (__force __u64) ctx->cm_id->device->node_guid;
>> resp.port_num = ctx->cm_id->port_num;
>>
>> - if (rdma_tech_ib(ctx->cm_id->device, ctx->cm_id->port_num))
>> + if (cap_ib_sa(ctx->cm_id->device, ctx->cm_id->port_num))
>> ucma_copy_ib_route(&resp, &ctx->cm_id->route);
>> else if (rdma_tech_iboe(ctx->cm_id->device, ctx->cm_id->port_num))
>> ucma_copy_iboe_route(&resp, &ctx->cm_id->route);
>> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
>> index e4999f6..3bfdf81 100644
>> --- a/include/rdma/ib_verbs.h
>> +++ b/include/rdma/ib_verbs.h
>> @@ -1833,6 +1833,21 @@ static inline int cap_iw_cm(struct ib_device *device, u8 port_num)
>> return rdma_tech_iwarp(device, port_num);
>> }
>>
>> +/**
>> + * cap_ib_sa - Check if the port of device has the capability Infiniband
>> + * Subnet Administrator.
>
> Nit: Administrator -> Administration
>
>> + *
>> + * @device: Device to be checked
>> + * @port_num: Port number of the device
>> + *
>> + * Return 0 when port of the device don't support Infiniband
>> + * Subnet Administrator.
>
> Nit: Administrator -> Administration
>
>> + */
>> +static inline int cap_ib_sa(struct ib_device *device, u8 port_num)
>> +{
>> + return rdma_tech_ib(device, port_num);
>> +}
>> +
>> int ib_query_gid(struct ib_device *device,
>> u8 port_num, int index, union ib_gid *gid);
>>
>
On 04/16/2015 03:36 PM, Hal Rosenstock wrote:
[snip]
>> -EXPORT_SYMBOL(rdma_node_get_transport);
>> -
>> enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device, u8 port_num)
>> {
>> if (device->get_link_layer)
>> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
>> index 262bf44..f9ef479 100644
>> --- a/include/rdma/ib_verbs.h
>> +++ b/include/rdma/ib_verbs.h
>> @@ -84,9 +84,6 @@ enum rdma_transport_type {
>> RDMA_TRANSPORT_IBOE,
>> };
>>
>> -__attribute_const__ enum rdma_transport_type
>> -rdma_node_get_transport(enum rdma_node_type node_type);
>> -
>> enum rdma_link_layer {
>> IB_LINK_LAYER_UNSPECIFIED,
>
> Is IB_LINK_LAYER_UNSPECIFIED still possible ?
Actually it's impossible in kernel at first, all those who implemented the callback
won't return UNSPECIFIED, others all have the correct transport type (otherwise BUG())
and won't result UNSPECIFIED :-)
Regards,
Michael Wang
>
>> IB_LINK_LAYER_INFINIBAND,
On 4/16/2015 9:35 AM, Michael Wang wrote:
>
>
> On 04/16/2015 03:19 PM, Hal Rosenstock wrote:
> [snip]
>>>
>>> +static inline int cma_validate_port(struct ib_device *device, u8 port,
>>> + union ib_gid *gid, int dev_type)
>>> +{
>>> + u8 found_port;
>>> + int ret = -ENODEV;
>>> +
>>> + if ((dev_type == ARPHRD_INFINIBAND) && !rdma_tech_ib(device, port))
>>> + return ret;
>>> +
>>> + if ((dev_type != ARPHRD_INFINIBAND) && rdma_tech_ib(device, port))
>>> + return ret;
>>> +
>>> + ret = ib_find_cached_gid(device, gid, &found_port, NULL);
>>> +
>>> + if (!ret && (port == found_port))
>>> + return 0;
>>> +
>>> + return ret;
>>
>> Should the case where ret = 0 and port != found_port need to be handled
>> the same as currently ? It looks different to me since in this case the
>> port will be saved into id_priv->id.port_num whereas currently it isn't.
>
> I get your point :-) what about:
>
> ret = ib_find_cached_gid(device, gid, &found_port, NULL);
> if (port != found_port)
> return -ENODEV;
>
> return ret;
Yes, that looks to me to be consistent with the current implementation.
-- Hal
> Regards,
> Michael Wang
>
>>
>> -- Hal
>>
>>> +}
>>> +
>>> static int cma_acquire_dev(struct rdma_id_private *id_priv,
>>> struct rdma_id_private *listen_id_priv)
>>> {
>>> struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;
>>> struct cma_device *cma_dev;
>>> - union ib_gid gid, iboe_gid;
>>> + union ib_gid gid, iboe_gid, *gidp;
>>> int ret = -ENODEV;
>>> - u8 port, found_port;
>>> - enum rdma_link_layer dev_ll = dev_addr->dev_type == ARPHRD_INFINIBAND ?
>>> - IB_LINK_LAYER_INFINIBAND : IB_LINK_LAYER_ETHERNET;
>>> + u8 port;
>>>
>>> - if (dev_ll != IB_LINK_LAYER_INFINIBAND &&
>>> + if (dev_addr->dev_type != ARPHRD_INFINIBAND &&
>>> id_priv->id.ps == RDMA_PS_IPOIB)
>>> return -EINVAL;
>>>
>>> @@ -391,41 +409,36 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv,
>>>
>>> memcpy(&gid, dev_addr->src_dev_addr +
>>> rdma_addr_gid_offset(dev_addr), sizeof gid);
>>> - if (listen_id_priv &&
>>> - rdma_port_get_link_layer(listen_id_priv->id.device,
>>> - listen_id_priv->id.port_num) == dev_ll) {
>>> +
>>> + if (listen_id_priv) {
>>> cma_dev = listen_id_priv->cma_dev;
>>> port = listen_id_priv->id.port_num;
>>> - if (rdma_node_get_transport(cma_dev->device->node_type) == RDMA_TRANSPORT_IB &&
>>> - rdma_port_get_link_layer(cma_dev->device, port) == IB_LINK_LAYER_ETHERNET)
>>> - ret = ib_find_cached_gid(cma_dev->device, &iboe_gid,
>>> - &found_port, NULL);
>>> - else
>>> - ret = ib_find_cached_gid(cma_dev->device, &gid,
>>> - &found_port, NULL);
>>> + gidp = rdma_tech_iboe(cma_dev->device, port) ?
>>> + &iboe_gid : &gid;
>>>
>>> - if (!ret && (port == found_port)) {
>>> - id_priv->id.port_num = found_port;
>>> + ret = cma_validate_port(cma_dev->device, port, gidp,
>>> + dev_addr->dev_type);
>>> + if (!ret) {
>>> + id_priv->id.port_num = port;
>>> goto out;
>>> }
>>> }
>>> +
>>> list_for_each_entry(cma_dev, &dev_list, list) {
>>> for (port = 1; port <= cma_dev->device->phys_port_cnt; ++port) {
>>> if (listen_id_priv &&
>>> listen_id_priv->cma_dev == cma_dev &&
>>> listen_id_priv->id.port_num == port)
>>> continue;
>>> - if (rdma_port_get_link_layer(cma_dev->device, port) == dev_ll) {
>>> - if (rdma_node_get_transport(cma_dev->device->node_type) == RDMA_TRANSPORT_IB &&
>>> - rdma_port_get_link_layer(cma_dev->device, port) == IB_LINK_LAYER_ETHERNET)
>>> - ret = ib_find_cached_gid(cma_dev->device, &iboe_gid, &found_port, NULL);
>>> - else
>>> - ret = ib_find_cached_gid(cma_dev->device, &gid, &found_port, NULL);
>>> -
>>> - if (!ret && (port == found_port)) {
>>> - id_priv->id.port_num = found_port;
>>> - goto out;
>>> - }
>>> +
>>> + gidp = rdma_tech_iboe(cma_dev->device, port) ?
>>> + &iboe_gid : &gid;
>>> +
>>> + ret = cma_validate_port(cma_dev->device, port, gidp,
>>> + dev_addr->dev_type);
>>> + if (!ret) {
>>> + id_priv->id.port_num = port;
>>> + goto out;
>>> }
>>> }
>>> }
>>
>
On 4/16/2015 9:41 AM, Michael Wang wrote:
>
>
> On 04/16/2015 03:36 PM, Hal Rosenstock wrote:
> [snip]
>>> -EXPORT_SYMBOL(rdma_node_get_transport);
>>> -
>>> enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device, u8 port_num)
>>> {
>>> if (device->get_link_layer)
>>> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
>>> index 262bf44..f9ef479 100644
>>> --- a/include/rdma/ib_verbs.h
>>> +++ b/include/rdma/ib_verbs.h
>>> @@ -84,9 +84,6 @@ enum rdma_transport_type {
>>> RDMA_TRANSPORT_IBOE,
>>> };
>>>
>>> -__attribute_const__ enum rdma_transport_type
>>> -rdma_node_get_transport(enum rdma_node_type node_type);
>>> -
>>> enum rdma_link_layer {
>>> IB_LINK_LAYER_UNSPECIFIED,
>>
>> Is IB_LINK_LAYER_UNSPECIFIED still possible ?
>
> Actually it's impossible in kernel at first, all those who implemented the callback
> won't return UNSPECIFIED, others all have the correct transport type (otherwise BUG())
> and won't result UNSPECIFIED :-)
Should it be removed from this enum somewhere in this patch series
(perhaps early on) ?
-- Hal
> Regards,
> Michael Wang
>
>>
>>> IB_LINK_LAYER_INFINIBAND,
>
On 04/16/2015 03:42 PM, Hal Rosenstock wrote:
> On 4/16/2015 9:41 AM, Michael Wang wrote:
>>
>>
>> On 04/16/2015 03:36 PM, Hal Rosenstock wrote:
>> [snip]
>>>> -EXPORT_SYMBOL(rdma_node_get_transport);
>>>> -
>>>> enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device, u8 port_num)
>>>> {
>>>> if (device->get_link_layer)
>>>> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
>>>> index 262bf44..f9ef479 100644
>>>> --- a/include/rdma/ib_verbs.h
>>>> +++ b/include/rdma/ib_verbs.h
>>>> @@ -84,9 +84,6 @@ enum rdma_transport_type {
>>>> RDMA_TRANSPORT_IBOE,
>>>> };
>>>>
>>>> -__attribute_const__ enum rdma_transport_type
>>>> -rdma_node_get_transport(enum rdma_node_type node_type);
>>>> -
>>>> enum rdma_link_layer {
>>>> IB_LINK_LAYER_UNSPECIFIED,
>>>
>>> Is IB_LINK_LAYER_UNSPECIFIED still possible ?
>>
>> Actually it's impossible in kernel at first, all those who implemented the callback
>> won't return UNSPECIFIED, others all have the correct transport type (otherwise BUG())
>> and won't result UNSPECIFIED :-)
>
> Should it be removed from this enum somewhere in this patch series
> (perhaps early on) ?
It was still directly used by helper like ib_modify_qp_is_ok() as indicator, may be
better in another following patch to reform those part :-)
Regards,
Michael Wang
>
> -- Hal
>
>> Regards,
>> Michael Wang
>>
>>>
>>>> IB_LINK_LAYER_INFINIBAND,
>>
>
On 4/16/15 8:45 AM, Michael Wang wrote:
>
> On 04/16/2015 03:42 PM, Hal Rosenstock wrote:
>> On 4/16/2015 9:41 AM, Michael Wang wrote:
>>>
>>> On 04/16/2015 03:36 PM, Hal Rosenstock wrote:
>>> [snip]
>>>>> -EXPORT_SYMBOL(rdma_node_get_transport);
>>>>> -
>>>>> enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device, u8 port_num)
>>>>> {
>>>>> if (device->get_link_layer)
>>>>> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
>>>>> index 262bf44..f9ef479 100644
>>>>> --- a/include/rdma/ib_verbs.h
>>>>> +++ b/include/rdma/ib_verbs.h
>>>>> @@ -84,9 +84,6 @@ enum rdma_transport_type {
>>>>> RDMA_TRANSPORT_IBOE,
>>>>> };
>>>>>
>>>>> -__attribute_const__ enum rdma_transport_type
>>>>> -rdma_node_get_transport(enum rdma_node_type node_type);
>>>>> -
>>>>> enum rdma_link_layer {
>>>>> IB_LINK_LAYER_UNSPECIFIED,
>>>> Is IB_LINK_LAYER_UNSPECIFIED still possible ?
>>> Actually it's impossible in kernel at first, all those who implemented the callback
>>> won't return UNSPECIFIED, others all have the correct transport type (otherwise BUG())
>>> and won't result UNSPECIFIED :-)
>> Should it be removed from this enum somewhere in this patch series
>> (perhaps early on) ?
I don't think it's ever been 'possible.' It's purpose is to catch
initialized errors where the transport fails to initialize it's
transport type. So for example,
provider = calloc(1, sizeof *provider)
If 0 is a valid link layer type, then you wouldn't catch these kinds of
errors.
Tom
> It was still directly used by helper like ib_modify_qp_is_ok() as indicator, may be
> better in another following patch to reform those part :-)
>
> Regards,
> Michael Wang
>
>> -- Hal
>>
>>> Regards,
>>> Michael Wang
>>>
>>>>> IB_LINK_LAYER_INFINIBAND,
> > This is equivalent to today where the checks are per node rather than
> > per port.
> >
> > Should all checks here be port 1 based or only certain ones like listen
> > ? For example, in connect/reject/disconnect, don't we already have port
> > ? Guess this can be dealt with later as this is not a regression from
> > the current implementation.
>
> Yeah, these parts of cma may need more carve in future, like some new
> callback
> for different CM type as Sean suggested.
>
> Maybe directly using 1 could help to highlight the problem ;-)
Only a few checks need to be per device. I think I pointed those out previously. Testing should show anywhere that we miss fairly quickly, since port would still be 0. For the checks that can be updated to be per port, I would rather go ahead and convert them.
- Sean
????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay? ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m???? ????????I?
On 04/16/2015 04:31 PM, Hefty, Sean wrote:
>>> This is equivalent to today where the checks are per node rather than
>>> per port.
>>>
>>> Should all checks here be port 1 based or only certain ones like listen
>>> ? For example, in connect/reject/disconnect, don't we already have port
>>> ? Guess this can be dealt with later as this is not a regression from
>>> the current implementation.
>>
>> Yeah, these parts of cma may need more carve in future, like some new
>> callback
>> for different CM type as Sean suggested.
>>
>> Maybe directly using 1 could help to highlight the problem ;-)
>
> Only a few checks need to be per device. I think I pointed those out previously. Testing should show anywhere that we miss fairly quickly, since port would still be 0. For the checks that can be updated to be per port, I would rather go ahead and convert them.
Got it, will be changed in next version :-)
To be confirmed:
PORT ASSIGNED
rdma_init_qp_attr Y
rdma_destroy_id unknown
cma_listen_on_dev N
cma_bind_loopback N
rdma_listen N
rdma_connect Y
rdma_accept Y
rdma_reject Y
rdma_disconnect Y
ib_ucm_add_one N
Is this list correct?
Regards,
Michael Wang
>
> - Sean
>
On Thu, Apr 16, 2015 at 10:08:10AM +0200, Michael Wang wrote:
>
> Use raw management helpers to reform cm related part in IB-core cma/ucm.
>
> These checks focus on the device cm type rather than the port capability,
> directly pass port 1 works currently, but can't support mixing cm type
> device in future.
After the discussion settled, I ended up thinking that implementing
explicit device checks, for use by CM, and the BUG_ON at register to
require all ports have the same value was the best option.
It also looks like hardwired 1 won't work on switch ports, so it is no-go.
Jason
> After the discussion settled, I ended up thinking that implementing
> explicit device checks, for use by CM, and the BUG_ON at register to
> require all ports have the same value was the best option.
Sure, but why not update the other areas anyway? This way when listens become per port, rather than per device, we only need to update that portion of the code.
On Thu, Apr 16, 2015 at 04:55:10PM +0000, Hefty, Sean wrote:
> > After the discussion settled, I ended up thinking that implementing
> > explicit device checks, for use by CM, and the BUG_ON at register to
> > require all ports have the same value was the best option.
>
> Sure, but why not update the other areas anyway? This way when
> listens become per port, rather than per device, we only need to
> update that portion of the code.
I wasn't clear, I agree: Yes, update what is possible in the CM, but
use an explicit device test for the areas we can't fix.
Jason
On 4/16/2015 11:22 AM, Michael Wang wrote:
>
>
> On 04/16/2015 04:31 PM, Hefty, Sean wrote:
>>>> This is equivalent to today where the checks are per node rather than
>>>> per port.
>>>>
>>>> Should all checks here be port 1 based or only certain ones like listen
>>>> ? For example, in connect/reject/disconnect, don't we already have port
>>>> ? Guess this can be dealt with later as this is not a regression from
>>>> the current implementation.
>>>
>>> Yeah, these parts of cma may need more carve in future, like some new
>>> callback
>>> for different CM type as Sean suggested.
>>>
>>> Maybe directly using 1 could help to highlight the problem ;-)
>>
>> Only a few checks need to be per device. I think I pointed those out previously. Testing should show anywhere that we miss fairly quickly, since port would still be 0. For the checks that can be updated to be per port, I would rather go ahead and convert them.
>
> Got it, will be changed in next version :-)
>
> To be confirmed:
> PORT ASSIGNED
> rdma_init_qp_attr Y
> rdma_destroy_id unknown
> cma_listen_on_dev N
> cma_bind_loopback N
> rdma_listen N
Why "N"? rdma_listen() can be constrained to a single port, right?
And even if wildcarded, it needs to act on multiple ports, which is
to say, it will fail only if no ports are eligible.
Tom.
> rdma_connect Y
> rdma_accept Y
> rdma_reject Y
> rdma_disconnect Y
> ib_ucm_add_one N
>
> Is this list correct?
>
> Regards,
> Michael Wang
>
>>
>> - Sean
>>
>
>
> > To be confirmed:
> > PORT ASSIGNED
> > rdma_init_qp_attr Y
> > rdma_destroy_id unknown
> > cma_listen_on_dev N
> > cma_bind_loopback N
Bind loopback will attach to a port, but the id does not have on entry.
> > rdma_listen N
>
> Why "N"? rdma_listen() can be constrained to a single port, right?
> And even if wildcarded, it needs to act on multiple ports, which is
> to say, it will fail only if no ports are eligible.
Rdma listen should be unknown. The id may be assigned to a port. It depends on the source address.
> > rdma_connect Y
> > rdma_accept Y
> > rdma_reject Y
> > rdma_disconnect Y
> > ib_ucm_add_one N
Others look correct.
Btw, thanks for your work on this. I know this is becoming a much bigger change than you originally intended. :)
- Sean
????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay? ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m???? ????????I?
On 4/16/2015 11:58 AM, Jason Gunthorpe wrote:
> It also looks like hardwired 1 won't work on switch ports, so it is no-go.
AFAIK enhanced switch port 0 is not supported by CM/RDMA CM in the
current code. There is no need for CM/RDMA CM on base switch port 0.
>
> On 4/16/2015 11:58 AM, Jason Gunthorpe wrote:
> > It also looks like hardwired 1 won't work on switch ports, so it is no-go.
>
> AFAIK enhanced switch port 0 is not supported by CM/RDMA CM in the current
> code. There is no need for CM/RDMA CM on base switch port 0.
I concur and I thought I mentioned that before.
Ira
On Thu, Apr 16, 2015 at 01:38:07PM -0400, Hal Rosenstock wrote:
> On 4/16/2015 11:58 AM, Jason Gunthorpe wrote:
> > It also looks like hardwired 1 won't work on switch ports, so it is no-go.
>
> AFAIK enhanced switch port 0 is not supported by CM/RDMA CM in the
> current code. There is no need for CM/RDMA CM on base switch port 0.
Hurm, yet another reason to have a proper device level test to capture
all this weirdness. I am surprised to hear this, actually.
Jason
On 04/16/2015 07:21 PM, Tom Talpey wrote:
> On 4/16/2015 11:22 AM, Michael Wang wrote:
>>
>>
>> On 04/16/2015 04:31 PM, Hefty, Sean wrote:
>>>>> This is equivalent to today where the checks are per node rather than
>>>>> per port.
>>>>>
>>>>> Should all checks here be port 1 based or only certain ones like listen
>>>>> ? For example, in connect/reject/disconnect, don't we already have port
>>>>> ? Guess this can be dealt with later as this is not a regression from
>>>>> the current implementation.
>>>>
>>>> Yeah, these parts of cma may need more carve in future, like some new
>>>> callback
>>>> for different CM type as Sean suggested.
>>>>
>>>> Maybe directly using 1 could help to highlight the problem ;-)
>>>
>>> Only a few checks need to be per device. I think I pointed those out previously. Testing should show anywhere that we miss fairly quickly, since port would still be 0. For the checks that can be updated to be per port, I would rather go ahead and convert them.
>>
>> Got it, will be changed in next version :-)
>>
>> To be confirmed:
>> PORT ASSIGNED
>> rdma_init_qp_attr Y
>> rdma_destroy_id unknown
>> cma_listen_on_dev N
>> cma_bind_loopback N
>> rdma_listen N
>
> Why "N"? rdma_listen() can be constrained to a single port, right?
> And even if wildcarded, it needs to act on multiple ports, which is
> to say, it will fail only if no ports are eligible.
Yeah, it can or can't, maybe 'unknown' is better :-)
Regards,
Michael Wang
>
> Tom.
>
>
>> rdma_connect Y
>> rdma_accept Y
>> rdma_reject Y
>> rdma_disconnect Y
>> ib_ucm_add_one N
>>
>> Is this list correct?
>>
>> Regards,
>> Michael Wang
>>
>>>
>>> - Sean
>>>
>>
>>
>
On 04/16/2015 07:30 PM, Hefty, Sean wrote:
>>> To be confirmed:
>>> PORT ASSIGNED
>>> rdma_init_qp_attr Y
>>> rdma_destroy_id unknown
>>> cma_listen_on_dev N
>>> cma_bind_loopback N
>
> Bind loopback will attach to a port, but the id does not have on entry.
>
>>> rdma_listen N
>>
>> Why "N"? rdma_listen() can be constrained to a single port, right?
>> And even if wildcarded, it needs to act on multiple ports, which is
>> to say, it will fail only if no ports are eligible.
>
> Rdma listen should be unknown. The id may be assigned to a port. It depends on the source address.
Agree, so for those 'N' or 'unknow', let's use port 1 directly.
>
>>> rdma_connect Y
>>> rdma_accept Y
>>> rdma_reject Y
>>> rdma_disconnect Y
>>> ib_ucm_add_one N
>
> Others look correct.
>
> Btw, thanks for your work on this. I know this is becoming a much bigger change than you originally intended. :)
My pleasure :-) It do exceeded my expectations, fortunately I have you guys
on my side, without your comments, it's impossible to make it so far :-)
Regards,
Michael Wang
>
> - Sean
>
On 04/16/2015 05:58 PM, Jason Gunthorpe wrote:
> On Thu, Apr 16, 2015 at 10:08:10AM +0200, Michael Wang wrote:
>>
>> Use raw management helpers to reform cm related part in IB-core cma/ucm.
>>
>> These checks focus on the device cm type rather than the port capability,
>> directly pass port 1 works currently, but can't support mixing cm type
>> device in future.
>
> After the discussion settled, I ended up thinking that implementing
> explicit device checks, for use by CM, and the BUG_ON at register to
> require all ports have the same value was the best option.
>
> It also looks like hardwired 1 won't work on switch ports, so it is no-go.
AFAIK, the current HW won't trigger such Bug, actually only mlx4 using port_num
to check the link-layer (but still ib cm anyway), others are just static whatever
the port_num is.
Thus as long as the check is still count on transport type and link-layer, the
BUG may never be triggered...
Regards,
Michael Wang
>
> Jason
>