Date: Tue, 14 Apr 2015 13:43:11 -0400
From: "ira.weiny" <ira.weiny@intel.com>
To: Jason Gunthorpe <jgunthorpe@obsidianresearch.com>
Cc: Michael Wang <yun.wang@profitbricks.com>,
        Roland Dreier <roland@kernel.org>, Sean Hefty <sean.hefty@intel.com>,
        Hal Rosenstock <hal.rosenstock@gmail.com>, linux-rdma@vger.kernel.org,
        linux-kernel@vger.kernel.org, Tom Tucker <tom@opengridcomputing.com>,
        Steve Wise <swise@opengridcomputing.com>,
        Hoang-Nam Nguyen <hnguyen@de.ibm.com>,
        Christoph Raisch <raisch@de.ibm.com>,
        Mike Marciniszyn <infinipath@intel.com>, Eli Cohen <eli@mellanox.com>,
        Faisal Latif <faisal.latif@intel.com>,
        Jack Morgenstein <jackm@dev.mellanox.co.il>,
        Or Gerlitz <ogerlitz@mellanox.com>, Haggai Eran <haggaie@mellanox.com>,
        Tom Talpey <tom@talpey.com>, Doug Ledford <dledford@redhat.com>
Subject: Re: [PATCH v3 07/28] IB/Verbs: Reform IB-ulp ipoib
Message-ID: <20150414174311.GA28111@phlsvsds.ph.intel.com>
References: <552BB470.4090407@profitbricks.com> <552BB5AC.6050101@profitbricks.com> <20150413192701.GA19112@obsidianresearch.com> <20150413194602.GA21467@phlsvsds.ph.intel.com> <20150413200138.GC19112@obsidianresearch.com> <20150414141806.GA7354@phlsvsds.ph.intel.com> <20150414172514.GB7682@obsidianresearch.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20150414172514.GB7682@obsidianresearch.com>
User-Agent: Mutt/1.4.2.2i
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2014
Lines: 62

On Tue, Apr 14, 2015 at 11:25:15AM -0600, Jason Gunthorpe wrote:
> On Tue, Apr 14, 2015 at 10:18:07AM -0400, ira.weiny wrote:
> 
> > After more thought and reading other opinions, I must agree we should not
> > have cap_foo_dev.
> 
> I looked at it a bit, and I think Sean has also basically said, CM
> does not support certain mixed port combinations. iWarp and IB simply
> cannot be mixed with the current CM and it doesn't look easy to fix
> that. We can fix a few point areas simply, but not all of it.
> 
> So we have to have the _dev tests, only to fill the CM's need and they
> must have the all true/all false/BUG semantics CM demands.
> 
> Verify on register.
> 
> > While the ports in ib_sa and ib_umad probably can be orthogonal the current
> > implementation does not support that and this patch series obscures that a bit.
> 
> Really? Do you see any bugs/missed things? Both were made port
> orthogonal when RoCEE was added, because RoCEE needs that.

They are not completely orthogonal:

A failure to init port 2 ends up ends up "killing" port 1 and releasing the
device associated resources.

static void ib_umad_add_one(struct ib_device *device)
{
...
                if (ib_umad_init_port(device, i, umad_dev,
                                      &umad_dev->port[i - s]))
                        goto err;
...

err:
        while (--i >= s) {
                if (!cap_ib_mad(device, i))
                        continue;

                ib_umad_kill_port(&umad_dev->port[i - s]);
        }

        kobject_put(&umad_dev->kobj);
}

> 
> CM wasn't because RoCEE and IB seem to use almost the same code,
> though I wonder if mixing really works 100%..

The support can (and should) be orthogonal but the implementation is
incomplete.

Ira

> 
> Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/