Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <64F9A492-44B9-4057-ABA5-C8202828A8DD@oracle.com>
 <1b8a24a9-5dba-3faf-8b0a-16e728a6051c@rothenpieler.org> <5DD80ADC-0A4B-4D95-8CF7-29096439DE9D@oracle.com>
 <0444ca5c-e8b6-1d80-d8a5-8469daa74970@rothenpieler.org> <cc2f55cd-57d4-d7c3-ed83-8b81ea60d821@rothenpieler.org>
 <3AF4F6CA-8B17-4AE9-82E2-21A2B9AA0774@oracle.com> <CAN-5tyHNvYWd1M7sfZNV5q3Y_GZA2-DoTd=CxYvniZ1zkB5hyw@mail.gmail.com>
 <95DB2B47-F370-4787-96D9-07CE2F551AFD@oracle.com> <CAN-5tyGXycM1MKa=Sydoo4pP85PGLuh8yjJYsoAM3U+M1NVyCw@mail.gmail.com>
 <D417C606-9E27-431E-B80E-EE927E62A316@oracle.com> <20210811201435.GA31574@fieldses.org>
In-Reply-To: <20210811201435.GA31574@fieldses.org>
From:   Olga Kornievskaia <aglo@umich.edu>
Date:   Wed, 11 Aug 2021 16:40:04 -0400
Message-ID: <CAN-5tyGq=EE4PjSbKVyKLtmhhFF_o6D9uG1QNQ-ByVMp9q8LOw@mail.gmail.com>
Subject: Re: Spurious instability with NFSoRDMA under moderate load
To:     "J. Bruce Fields" <bfields@fieldses.org>
Cc:     Chuck Lever III <chuck.lever@oracle.com>,
        Bruce Fields <bfields@redhat.com>,
        Timo Rothenpieler <timo@rothenpieler.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        Dai Ngo <dai.ngo@oracle.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, Aug 11, 2021 at 4:14 PM J. Bruce Fields <bfields@fieldses.org> wrote:
>
> On Wed, Aug 11, 2021 at 08:01:30PM +0000, Chuck Lever III wrote:
> > Probably not just CB_RECALL, but agreed, there doesn't seem to
> > be any mechanism that can re-drive callback operations when the
> > backchannel is replaced.
>
> The nfsd4_queue_cb() in nfsd4_cb_release() should queue a work item
> to run nfsd4_run_cb_work, which should set up another callback client if
> necessary.

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index 7325592b456e..ed0e76f7185c 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -1191,6 +1191,7 @@ static void nfsd4_cb_done(struct rpc_task *task,
void *calldata)
                case -ETIMEDOUT:
                case -EACCES:
                        nfsd4_mark_cb_down(clp, task->tk_status);
+                       cb->cb_need_restart = true;
                }
                break;
        default:

Something like this should requeue and retry the callback?

>
> But I doubt this is well tested.
>
> --b.