Received-SPF: pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Content-Type: text/plain;
        charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: Re: NFSv4 client locks up on larger writes with Kerberos enabled
From:   Chuck Lever <chuck.lever@oracle.com>
In-Reply-To: <20190925200723.GA11954@fieldses.org>
Date:   Thu, 26 Sep 2019 08:55:17 -0700
Cc:     Kevin Vasko <kvasko@gmail.com>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Content-Transfer-Encoding: 7bit
Message-Id: <1BC54D7A-073E-40FD-9AA3-552F1E1BD214@oracle.com>
References: <CAMd28E-pJp4=kvp62FJqGLZo-jGA2rH2OT6-hK_N=TvMiJuT2A@mail.gmail.com>
 <20190925164831.GA9366@fieldses.org>
 <57192382-86BE-4878-9AE0-B22833D56367@oracle.com>
 <CAMd28E-zcjuCfVbDCra4Av3Ewsdd-Ai=E0j3tF2GKJ8P6nG8=w@mail.gmail.com>
 <FCAD9EFD-26AD-41F0-BCCA-80E475219731@oracle.com>
 <20190925200723.GA11954@fieldses.org>
To:     Bruce Fields <bfields@fieldses.org>,
        Trond Myklebust <trond.myklebust@hammerspace.com>
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk


> On Sep 25, 2019, at 1:07 PM, Bruce Fields <bfields@fieldses.org> wrote:
> 
> On Wed, Sep 25, 2019 at 11:49:14AM -0700, Chuck Lever wrote:
>> Sounds like the NFS server is dropping the connection. With
>> GSS enabled, that's usually a sign that the GSS window has
>> overflowed.
> 
> Would that show up in the rpc statistics on the client somehow?

More likely on the server. The client just sees a disconnect
without any explanation attached.

gss_verify_header is where the checking is done on the server.
Disappointingly, I see some dprintk's in there, but no static
trace events.


> In that case--I seem to remember there's a way to configure the size of
> the client's slot table, maybe lowering that (decreasing the number of
> rpc's allowed to be outstanding at a time) would work around the
> problem.

> Should the client be doing something different to avoid or recover from
> overflows of the gss window?

The client attempts to meter the request stream so that it stays
within the bounds of the GSS sequence number window. The stream
of requests is typically unordered coming out of the transmit
queue.

There is some new code (since maybe v5.0?) that handles the
metering: gss_xmit_need_reencode().


--
Chuck Lever