Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Fri, 12 Feb 2021 10:43:55 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Chuck Lever <chuck.lever@oracle.com>
CC:     <linux-nfs@vger.kernel.org>, <linux-rdma@vger.kernel.org>
Subject: Re: [PATCH v1] svcrdma: Hold private mutex while invoking
 rdma_accept()
Message-ID: <20210212144355.GA1696322@nvidia.com>
References: <161308170145.1097.4777972218876421176.stgit@klimt.1015granger.net>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <161308170145.1097.4777972218876421176.stgit@klimt.1015granger.net>
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from mlx.ziepe.ca (142.162.115.133) by BL0PR02CA0132.namprd02.prod.outlook.com (2603:10b6:208:35::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Fri, 12 Feb 2021 14:43:56 +0000
Received: from jgg by mlx with local (Exim 4.94)        (envelope-from <jgg@nvidia.com>)        id 1lAZfn-0077jA-4H; Fri, 12 Feb 2021 10:43:55 -0400
X-Header: ProcessedBy-CMR-outbound
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1;
        t=1613141038; bh=/vWkVSjt9tzOlMD9ioDzCqTgtE0n2rdoJh0wPEgm3E0=;
        h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:Date:
         From:To:CC:Subject:Message-ID:References:Content-Type:
         Content-Disposition:In-Reply-To:X-ClientProxiedBy:MIME-Version:
         X-MS-Exchange-MessageSentRepresentingType:X-Header;
        b=hSKnifMKlK3Ks1TeyNIQLk7XY/Ut8w+1z8extlO4pC+QrqfVZXvIEXXD2M5yK7QkD
         eDwUkoZ2XOrN10wT87m9cQHMetrX7BuHhGy2T9V8d5jPVoRQw48QC1Xmh26+1Sk9SA
         EnHb0rBhArFNmEYZrip4MsOd1jVXa2q4VPE4bdLwZMWYKxSiWqcIEgXk3UVhJUYwYJ
         4tpFx5AkuXm+K6tr7VfTqJ9S4Mupo+eyDG8tc1sVK+kdqqC8IMaXwvllUA4fAEDMtS
         0CAfkZNQ215vqrdTkFlM/uN0tzva8wAQkQ+szT4Q/JBMCTUN+uqXfegB3JEGJoQ1Lt
         tIk9ozu42bkHA==
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org

On Thu, Feb 11, 2021 at 05:15:30PM -0500, Chuck Lever wrote:
> RDMA core mutex locking was restructured by d114c6feedfe ("RDMA/cma:
> Add missing locking to rdma_accept()") [Aug 2020]. When lock
> debugging is enabled, the RPC/RDMA server trips over the new lockdep
> assertion in rdma_accept() because it doesn't call rdma_accept()
> from its CM event handler.
> 
> As a temporary fix, have svc_rdma_accept() take the mutex
> explicitly. In the meantime, let's consider how to restructure the
> RPC/RDMA transport to invoke rdma_accept() from the proper context.
> 
> Calls to svc_rdma_accept() are serialized with calls to
> svc_rdma_free() by the generic RPC server layer.
> 
> Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>

Fixes line

> ---
>  net/sunrpc/xprtrdma/svc_rdma_transport.c |    2 ++
>  1 file changed, 2 insertions(+)

It could even be right like this..

This should be inside the lock though:

        newxprt->sc_cm_id->event_handler = svc_rdma_cma_handler;

But this really funny looking, before it gets to accept the handler is
still the listen handler so any incoming events will just be
discarded.

However the rdma_accept() should fail if the state machine has been
moved from the accepting state, and I think the only meaningful event
that can be delivered here is disconnect. So the rdma_accept() failure
does trigger destroy_id, which is the right thing on disconnect anyhow.

Jason