Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Olga Kornievskaia <olga.kornievskaia@gmail.com>
CC:     Bruce Fields <bfields@redhat.com>,
        Trond Myklebust <trondmy@hammerspace.com>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH 1/1] NFSD: fix error handling in callbacks
Thread-Topic: [PATCH 1/1] NFSD: fix error handling in callbacks
Thread-Index: AQHXFPJMATYPHlFtZkmEJfeeuT4DnKp7ykSAgABPzQCAAAVEgIAABRGAgAEqPYCAAHuCAIABHTUAgAABygCAAACLgA==
Date:   Thu, 11 Mar 2021 15:18:30 +0000
Message-ID: <FB84E90A-1A03-48B3-8BF7-D9D10AC2C9FE@oracle.com>
References: <20210309144127.57833-1-olga.kornievskaia@gmail.com>
 <YEeWK+gs4c8O7k0u@pick.fieldses.org>
 <4ca27c770577376b0a39f0cfcfb529b96d6d5aae.camel@hammerspace.com>
 <CAN-5tyFttTHRdnELORJwCER_KPGBNk4W3eLwG0Z=QkwmPQh1UQ@mail.gmail.com>
 <d205a6a77273534666b3c33065934b9f66e7b103.camel@hammerspace.com>
 <YEjb9ZadFqa9Vu9O@pick.fieldses.org>
 <CAN-5tyFZ8fS6fjOJEu2NkRYUX6HrA5XNKPWyWN+UVtQT6Gp4kQ@mail.gmail.com>
 <2BAF1E3E-6FDB-46FF-8A63-0CA7EE5B6535@oracle.com>
 <CAN-5tyEV0+DDgjXsfdiVhpeVNOprOmYQf24CdOCr+fayOmR3Bw@mail.gmail.com>
In-Reply-To: <CAN-5tyEV0+DDgjXsfdiVhpeVNOprOmYQf24CdOCr+fayOmR3Bw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <B762FDA926C1294DB392AC76EC16748E@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 85d04e35-e878-4813-6e08-08d8e4a0ed9d
X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Mar 2021 15:18:30.4506
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Mtye8xFFK5H5CcoXYa/lDGTGeC7gEVyHYInckfjnjurlNlTOXJ8gYVcjzh6/3Hlc29qhKXFRshz8jmJF7Iv4FQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4496
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9920 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 adultscore=0
 phishscore=0 malwarescore=0 mlxscore=0 bulkscore=0 mlxlogscore=999
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000
 definitions=main-2103110083
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9920 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 mlxlogscore=999
 adultscore=0 impostorscore=0 suspectscore=0 clxscore=1015 malwarescore=0
 priorityscore=1501 phishscore=0 spamscore=0 lowpriorityscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000
 definitions=main-2103110083
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Mar 11, 2021, at 10:16 AM, Olga Kornievskaia <olga.kornievskaia@gmail.=
com> wrote:
>=20
> On Thu, Mar 11, 2021 at 10:10 AM Chuck Lever III <chuck.lever@oracle.com>=
 wrote:
>>=20
>>=20
>>=20
>>> On Mar 10, 2021, at 5:09 PM, Olga Kornievskaia <olga.kornievskaia@gmail=
.com> wrote:
>>>=20
>>> On Wed, Mar 10, 2021 at 9:47 AM J. Bruce Fields <bfields@redhat.com> wr=
ote:
>>>>=20
>>>> On Tue, Mar 09, 2021 at 08:59:51PM +0000, Trond Myklebust wrote:
>>>>> On Tue, 2021-03-09 at 15:41 -0500, Olga Kornievskaia wrote:
>>>>>> On Tue, Mar 9, 2021 at 3:22 PM Trond Myklebust <
>>>>>> trondmy@hammerspace.com> wrote:
>>>>>>>=20
>>>>>>> On Tue, 2021-03-09 at 10:37 -0500, J. Bruce Fields wrote:
>>>>>>>> On Tue, Mar 09, 2021 at 09:41:27AM -0500, Olga Kornievskaia
>>>>>>>> wrote:
>>>>>>>>> From: Olga Kornievskaia <kolga@netapp.com>
>>>>>>>>>=20
>>>>>>>>> When the server tries to do a callback and a client fails it
>>>>>>>>> due to
>>>>>>>>> authentication problems, we need the server to set callback
>>>>>>>>> down
>>>>>>>>> flag in RENEW so that client can recover.
>>>>>>>>=20
>>>>>>>> I was looking at this.  It looks to me like this should really be
>>>>>>>> just:
>>>>>>>>=20
>>>>>>>>       case 1:
>>>>>>>>               if (task->tk_status)
>>>>>>>>                       nfsd4_mark_cb_down(clp, task->tk_status);
>>>>>>>>=20
>>>>>>>> If tk_status showed an error, and the ->done method doesn't
>>>>>>>> return 0
>>>>>>>> to
>>>>>>>> tell us it something worth retrying, then the callback failed
>>>>>>>> permanently, so we should mark the callback path down, regardless
>>>>>>>> of
>>>>>>>> the
>>>>>>>> exact error.
>>>>>>>=20
>>>>>>> I disagree. task->tk_status could be an unhandled NFSv4 error (see
>>>>>>> nfsd4_cb_recall_done()). The client might, for instance, be in the
>>>>>>> process of returning the delegation being recalled. Why should that
>>>>>>> result in the callback channel being marked as down?
>>>>>>>=20
>>>>>>=20
>>>>>> Are you talking about say the connection going down and server shoul=
d
>>>>>> just reconnect instead of recovering the callback channel. I assumed
>>>>>> that connection break is something that's not  recoverable by the
>>>>>> callback but perhaps I'm wrong.
>>>>>=20
>>>>> No. I'm saying that nfsd4_cb_recall_done() will return a value of '1'
>>>>> for both task->tk_status =3D=3D -EBADHANDLE and -NFS4ERR_BAD_STATEID.=
 I'm
>>>>> not seeing why either of those errors should be handled by marking th=
e
>>>>> callback channel as being down.
>>>>>=20
>>>>> Looking further, it seems that the same function will also return '1'
>>>>> without checking the value of task->tk_status if the delegation has
>>>>> been revoked or returned. So that would mean that even NFS4ERR_DELAY
>>>>> could trigger the call to nfsd4_mark_cb_down() with the above change.
>>>>=20
>>>> Yeah, OK, that's wrong, apologies.
>>>>=20
>>>> I'm just a little worried about the attempt to enumerate transport lev=
el
>>>> errors in nfsd4_cb_done().  Are we sure that EIO, ETIMEDOUT, EACCESS i=
s
>>>> the right list?
>>>=20
>>> Looking at call_transmit_status error handling, I don't think
>>> connection errors are returned. Instead the code tries to fix the
>>> connection by retrying unless the rpc_timeout is reached and then only
>>> EIO,TIMEDOUT is returned.
>>>=20
>>> Can then my original patch be considered without resubmission?
>>=20
>> Bruce has authorized v1 of this patch, but that one has the
>> uncorrected patch description. Post a v4?
>=20
> v1's description is accurate. It reflects that only authentication
> errors are handled.

Nit: The short description isn't specific to NFSv4.0, as the v3
one was.


>>>> --b.
>>>>=20
>>>>>=20
>>>>>>=20
>>>>>>>>=20
>>>>>>>> --b.
>>>>>>>>=20
>>>>>>>>>=20
>>>>>>>>> Signed-off-by: Olga Kornievskaia <kolga@netapp.com>
>>>>>>>>> ---
>>>>>>>>> fs/nfsd/nfs4callback.c | 1 +
>>>>>>>>> 1 file changed, 1 insertion(+)
>>>>>>>>>=20
>>>>>>>>> diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
>>>>>>>>> index 052be5bf9ef5..7325592b456e 100644
>>>>>>>>> --- a/fs/nfsd/nfs4callback.c
>>>>>>>>> +++ b/fs/nfsd/nfs4callback.c
>>>>>>>>> @@ -1189,6 +1189,7 @@ static void nfsd4_cb_done(struct rpc_task
>>>>>>>>> *task, void *calldata)
>>>>>>>>>               switch (task->tk_status) {
>>>>>>>>>               case -EIO:
>>>>>>>>>               case -ETIMEDOUT:
>>>>>>>>> +               case -EACCES:
>>>>>>>>>                       nfsd4_mark_cb_down(clp, task-
>>>>>>>>>> tk_status);
>>>>>>>>>               }
>>>>>>>>>               break;
>>>>>>>>> --
>>>>>>>>> 2.27.0
>>>>>>>>>=20
>>>>>>>>=20
>>>>>>>=20
>>>>>>> --
>>>>>>> Trond Myklebust
>>>>>>> Linux NFS client maintainer, Hammerspace
>>>>>>> trond.myklebust@hammerspace.com
>>>>>>>=20
>>>>>>>=20
>>>>>=20
>>>>> --
>>>>> Trond Myklebust
>>>>> Linux NFS client maintainer, Hammerspace
>>>>> trond.myklebust@hammerspace.com
>>=20
>> --
>> Chuck Lever

--
Chuck Lever