As we enter the second week of the 2.6.17 merge window, I thought it
might be helpful to give an update on what I'm thinking about the
remaining pending pieces. This is really to stimulate discussion --
if there's something that you think really should (or shouldn't) be
merged, let me know.
Of course bug fixes are always welcome and can be merged pretty much
any time -- that's the whole point of the stabilization period after
the merge window.
* PathScale ipath driver. In my git tree at
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git ipath
There is still ia64 build breakage and a lot of sparse endian
annotations to clean up before it's mergeable. Also, there are two
issues I still see:
- Working around the IB midlayer SMA (subnet management agent) /
implementation with a character device when ib_mad isn't loaded.
Maybe I'm off-base here objecting to this. Hal and Sean, as the
ib_mad guys, I'd be especially interested in your opinion of this.
- The driver depends on 64BIT && PCI_MSI, and is basically
x86_64-only for practical purposes. I think this is OK as far as
a merge goes, but it would be nice to be able to use a PCIe
device on any system with PCIe slots...
* RDMA CM. In my git tree at
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git rdma_cm
I think this is OK to merge, but I don't see much pull to get it in
right now. There are three consumers on the horizon:
- userspace RDMA CM, which exports the abstraction to userspace.
The feeling is that this interface needs more time to mature.
- NFS/RDMA. Not ready to merge right now.
- iSER. Maybe ready to merge -- I haven't heard anything recently.
My feeling is that without someone making a case for why this
should go in now, I'm going to hold off.
* IPoIB tunables. This is probably OK but I haven't seen any patches
yet, so we're running out time.
* SRP FMRs. I have a patch that I'm pretty happy with now, so I
think this will go in. This will need a lot of testing, or we may
have to turn it off by default for 2.6.17 final
So if you care about any of this, let me know what you think. And if
there's something not on this list and not in
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-2.6.17
please make sure I know about it, or it won't get merged.
Thanks,
Roland
Roland Dreier wrote:
> - Working around the IB midlayer SMA (subnet management agent) /
> implementation with a character device when ib_mad isn't loaded.
> Maybe I'm off-base here objecting to this. Hal and Sean, as the
> ib_mad guys, I'd be especially interested in your opinion of this.
I will need to spend some time reading through the PathScale code on this. I've
followed most of the discussion, though not in detail. Right now, I don't see
what's being gained by not loading ib_mad. For instance, how does the driver
handle loading ib_mad later, say when another IB device is added to the system
that requires it?
> * RDMA CM. In my git tree at
>
> git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git rdma_cm
>
> I think this is OK to merge, but I don't see much pull to get it in
> right now. There are three consumers on the horizon:
>
> - userspace RDMA CM, which exports the abstraction to userspace.
> The feeling is that this interface needs more time to mature.
>
> - NFS/RDMA. Not ready to merge right now.
>
> - iSER. Maybe ready to merge -- I haven't heard anything recently.
I agree that we need to let the userspace interface mature. And even the kernel
interface could benefit from having some real users. The code was added to the
-mm branch, correct?
The main drawback that we have not merging the RDMA CM with userspace support,
is that OpenIB release 1.0 will have very poor connection management support.
There's no easy way for userspace applications to obtain path records.
- Sean
Sean> I agree that we need to let the userspace interface mature.
Sean> And even the kernel interface could benefit from having some
Sean> real users. The code was added to the -mm branch, correct?
Yes, I've put both the rdma_cm and ipath branches into my for-mm
branch, so Andrew will pick them up automatically.
- R.
BTW, what do you think of changing rdma_wq to be a GPL export, to give
a hint that this is an internal symbol not really for general use?
In other words:
diff --git a/drivers/infiniband/core/addr.c b/drivers/infiniband/core/addr.c
index 6ebd1d1..cdf4888 100644
--- a/drivers/infiniband/core/addr.c
+++ b/drivers/infiniband/core/addr.c
@@ -56,7 +56,7 @@ static DEFINE_MUTEX(lock);
static LIST_HEAD(req_list);
static DECLARE_WORK(work, process_req, NULL);
struct workqueue_struct *rdma_wq;
-EXPORT_SYMBOL(rdma_wq);
+EXPORT_SYMBOL_GPL(rdma_wq);
static int copy_addr(struct rdma_dev_addr *dev_addr, struct net_device *dev,
unsigned char *dst_dev_addr)
Roland Dreier wrote:
> BTW, what do you think of changing rdma_wq to be a GPL export, to give
> a hint that this is an internal symbol not really for general use?
I'm actually testing a patch set now that moves rdma_wq internal to ib_addr, and
gives the RDMA CM its own WQ.
- Sean
Sean> I'm actually testing a patch set now that moves rdma_wq
Sean> internal to ib_addr, and gives the RDMA CM its own WQ.
OK, that's better still.
BTW ib_local_sa also uses rdma_wq ... it might be worth fixing that,
because it actually makes ib_local_sa depend on CONFIG_NET right now
(since ib_addr can't build without CONFIG_NET).
Not that anyone is building kernels without CONFIG_NET...
- R.
>BTW ib_local_sa also uses rdma_wq ... it might be worth fixing that,
>because it actually makes ib_local_sa depend on CONFIG_NET right now
>(since ib_addr can't build without CONFIG_NET).
Yes - I've fixed that as well.
- Sean
Roland wrote,
> - userspace RDMA CM, which exports the abstraction to userspace.
> The feeling is that this interface needs more time to mature.
The userspace RDMA-CM is currently being used by uDAPL and seems to
work fine for that application. Our testing so far shows that the code is
stable. However, uDAPL also has a socket based mechanism that can used until
the CMA
is merged. If others think it still needs more time to mature, then uDAPL
has an interim solution.
my 2 cents on this one.
woody
On 3/27/06, Roland Dreier <[email protected]> wrote:
> As we enter the second week of the 2.6.17 merge window, I thought it
> might be helpful to give an update on what I'm thinking about the
> remaining pending pieces. This is really to stimulate discussion --
> if there's something that you think really should (or shouldn't) be
> merged, let me know.
<snip>
> - iSER. Maybe ready to merge -- I haven't heard anything recently.
>
iSER needs to complete the removal of the dummy socket provider
and change the open-iscsi user-space interface.
We are coordinating this effort with open-iscsi, and rely
on some open-iscsi changes that are planned to be pushed to 2.6.18.
Therefore the plan for iSER is to push it into 2.6.18
Dan
Dan> Therefore the plan for iSER is to push it into 2.6.18
OK, thanks.
Based on all of this I'm thinking that it's better to hold back the
RDMA CM as well, since there will be no code using it ready to merge
for 2.6.17.
- R.
On Mon, 2006-03-27 at 11:56 -0800, Roland Dreier wrote:
> * PathScale ipath driver. In my git tree at
>
> git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git ipath
>
> There is still ia64 build breakage and a lot of sparse endian
> annotations to clean up before it's mergeable.
These are all fixed now.
> - Working around the IB midlayer SMA (subnet management agent) /
> implementation with a character device when ib_mad isn't loaded.
We dropped this in the round of changes I just submitted.
> - The driver depends on 64BIT && PCI_MSI, and is basically
> x86_64-only for practical purposes. I think this is OK as far as
> a merge goes, but it would be nice to be able to use a PCIe
> device on any system with PCIe slots...
We'll see what we can do about this at some point, but I don't think it
should block the merge.
<b
OK, here's a quick update on 2.6.17 merge plans:
* PathScale ipath driver. In my git tree at
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git ipath
The new version looks good to me. I'll merge it unless I hear an
objection to the latest code.
* RDMA CM. In my git tree at
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git rdma_cm
None of the users of this code look are to merge, and it looks like
there's some changes in the design happening now. Seems like this
can and should wait for 2.6.18.
* IPoIB tunable ring sizes. Still no patch yet.
* SRP FMRs. I have a patch that I like, but it's not totally stable.
I may be hitting target bugs (ie in someone else's code). On the
other hand I don't have any numbers showing a benefit, so I'm not
sure if it's worth merging this.
* (new since last time) Improved static rate handling. I will get
this in.
As before, if you care about any of this, let me know what you think.
And if there's something not on this list and not in
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-2.6.17
please make sure I know about it, or it won't get merged.
Thanks,
Roland
Oh yeah, one other thing I plan to merge unless someone objects:
* Get rid of option for building IPoIB and mthca debug output unless EMBEDDED=y
- R.
Roland Dreier wrote:
> * RDMA CM. In my git tree at
>
> git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git rdma_cm
>
> None of the users of this code look are to merge, and it looks like
> there's some changes in the design happening now. Seems like this
> can and should wait for 2.6.18.
Does it make sense to merge the first two patches of that patch series?
1. Provide common handling for marshalling data between userspace clients and
kernel mode Infiniband drivers.
2. Extend the Infiniband CM to include private data comparisons as part of its
connection request matching process.
This would make it easier to keep the openib tree in sync with the kernel.
- Sean
On Thu, Mar 30, 2006 at 03:11:17PM -0800, Roland Dreier wrote:
> Oh yeah, one other thing I plan to merge unless someone objects:
>
> * Get rid of option for building IPoIB and mthca debug output unless EMBEDDED=y
NACK. Just add a FOO_DEBUG config option, this has no?hing to do with
EMBEDDED.
Roland> * Get rid of option for building IPoIB and mthca debug
Roland> output unless EMBEDDED=y
Christoph> NACK. Just add a FOO_DEBUG config option, this has
Christoph> no?hing to do with EMBEDDED.
I think you misunderstood (probably because what I wrote was very
unclear). What I meant to say was that I wanted to make
CONFIG_INFINIBAND_IPOIB_DEBUG=y, without the option to disable it,
unless CONFIG_EMBEDDED is set. In Kconfig language, change it to:
config INFINIBAND_IPOIB_DEBUG
bool "IP-over-InfiniBand debugging" if EMBEDDED
depends on INFINIBAND_IPOIB
default y
Do you still have a problem with that? The rationale is that we want
the debugging output compiled in (it still needs to be enabled at
runtime) unless the user really really knows they don't want it,
because by the time we know we want the output, it's too late to
recompile to kernel.
- R.