Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Chuck Lever <chuck.lever@oracle.com>
To:     Bruce Fields <bfields@fieldses.org>
CC:     Timo Rothenpieler <timo@rothenpieler.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        Olga Kornievskaia <aglo@umich.edu>,
        Dai Ngo <dai.ngo@oracle.com>
Subject: Re: [PATCH] svcrdma: disable timeouts on rdma backchannel
Thread-Topic: [PATCH] svcrdma: disable timeouts on rdma backchannel
Thread-Index: AQHXCXOTv6ms0S5nPkS/+o1NfiK5PqpnXOQAgABgLICAAABigA==
Date:   Wed, 24 Feb 2021 20:03:54 +0000
Message-ID: <D208366F-8FB7-48C9-A380-5518AA845B6C@oracle.com>
References: <C99EA5DE-814A-418B-9685-D400F4E7B964@oracle.com>
 <20210222233619.21568-1-timo@rothenpieler.org>
 <E650E8DD-9F65-4029-8F3B-AA854AF575A1@oracle.com>
 <20210224200231.GD11591@fieldses.org>
In-Reply-To: <20210224200231.GD11591@fieldses.org>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <A56DAA65E17EEF4BAB7E49CF66D00BD1@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: d307bcbc-1064-4b78-4cfa-08d8d8ff503e
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Feb 2021 20:03:54.6171
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: oo6J/IyPSPv6zLGW7ZsG57xPQIWYPx93DVNNKi3BnXdy3058nd94bZXjic0WM1j4owT8llNFafEJE/Cco2zPGQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2439
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9905 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0
 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 bulkscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2009150000 definitions=main-2102240153
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9905 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0
 lowpriorityscore=0 spamscore=0 mlxscore=0 bulkscore=0 clxscore=1015
 priorityscore=1501 malwarescore=0 impostorscore=0 suspectscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2009150000 definitions=main-2102240154
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Feb 24, 2021, at 3:02 PM, J. Bruce Fields <bfields@fieldses.org> wrote=
:
>=20
> On Wed, Feb 24, 2021 at 02:18:18PM +0000, Chuck Lever wrote:
>>=20
>>=20
>>> On Feb 22, 2021, at 6:36 PM, Timo Rothenpieler <timo@rothenpieler.org> =
wrote:
>>>=20
>>> This brings it in line with the regular tcp backchannel, which also has
>>> all those timeouts disabled.
>>>=20
>>> Prevents the backchannel from timing out, getting some async operations
>>> like server side copying getting stuck indefinitely on the client side.
>>>=20
>>> Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
>>=20
>> Thanks for your patch! I've included it in the for-rc branch at
>>=20
>> git://git.kernel.org/pub/scm/linux/kernel/git/cel/linux.git
>=20
> So, I'm sure this patch makes sense.
>=20
> But I'm also curious why it's not recovering.

Agreed. This patch is not a substitute for proper callback channel recovery=
.


> What I think should happen:
>=20
> 	- clp->cl_cb_state should be set to NFSD4_CB_DOWN.

I think it's set to FAULT.


> 	- This should cause the next SEQUENCE reply to have
> 	  SEQ4_STATUS_CB_PATH_DOWN set.
> 	- That should poke the client to recover.  (Maybe by sending a
> 	  BIND_CONN_TO_SESSION call?)
>=20
> I'd be curious whether any of that's actually happening.
>=20
> --b.
>=20
>>=20
>>=20
>>> ---
>>> Did the same testing with this applied than before, and could not
>>> observe it getting stuck, same as with the previous patch, which I
>>> removed before testing this one.
>>>=20
>>> This obviously still does not fix the issue of it being seemingly unabl=
e
>>> to reestablish the disconnected backchannel.
>>> An event that disconnects the backchannel but leaves the main connectio=
n
>>> intact seems a pretty rare occurance though, outside of this issue.
>>>=20
>>> net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 6 +++---
>>> 1 file changed, 3 insertions(+), 3 deletions(-)
>>>=20
>>> diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xp=
rtrdma/svc_rdma_backchannel.c
>>> index 63f8be974df2..8186ab6f99f1 100644
>>> --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c
>>> +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c
>>> @@ -252,9 +252,9 @@ xprt_setup_rdma_bc(struct xprt_create *args)
>>> 	xprt->timeout =3D &xprt_rdma_bc_timeout;
>>> 	xprt_set_bound(xprt);
>>> 	xprt_set_connected(xprt);
>>> -	xprt->bind_timeout =3D RPCRDMA_BIND_TO;
>>> -	xprt->reestablish_timeout =3D RPCRDMA_INIT_REEST_TO;
>>> -	xprt->idle_timeout =3D RPCRDMA_IDLE_DISC_TO;
>>> +	xprt->bind_timeout =3D 0;
>>> +	xprt->reestablish_timeout =3D 0;
>>> +	xprt->idle_timeout =3D 0;
>>>=20
>>> 	xprt->prot =3D XPRT_TRANSPORT_BC_RDMA;
>>> 	xprt->ops =3D &xprt_rdma_bc_procs;
>>> --=20
>>> 2.25.1
>>>=20
>>=20
>> --
>> Chuck Lever

--
Chuck Lever