Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <CAOv1SKCmdtchm5Z2NU80o49tkrHpAkPFaHKj4-vLDN5bZNCz-Q@mail.gmail.com>
 <3349D119-2F35-4A58-8061-A2659E8C6BB9@oracle.com>
In-Reply-To: <3349D119-2F35-4A58-8061-A2659E8C6BB9@oracle.com>
From:   Mike Javorski <mike.javorski@gmail.com>
Date:   Sun, 8 Aug 2021 16:23:05 -0700
Message-ID: <CAOv1SKAGuGYDr-qtdSJ0wkVkfqp5i27ZTZ8hhb9-byp4MsfVGQ@mail.gmail.com>
Subject: Re: NFS server regression in kernel 5.13 (tested w/ 5.13.9)
To:     Chuck Lever III <chuck.lever@oracle.com>
Cc:     Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

I appreciate that Chuck. It was not my intention to seek basic "user"
support, but as I don't know the intricacies of how nfs works in the
kernel, I don't know where to get the right information to share to
help with the diagnostic effort. I have read through the
troubleshooting recommendations on the linux-nfs wiki, but the issue
is beyond what is indicated there, and my setup has been running
without any NFS issues for years at this point, and I can demonstrate
the error merely by changing the booted kernel, so I feel confident
it's not a simple configuration error.

I have raised an issue on the ArchLinux bug tracker as well
(https://bugs.archlinux.org/task/71775). Given that the ArchLinux
version of the kernel has minimal delta from mainline, (I've checked
and no fs/rpc/nfs files are changed in 7 file delta vs mainline), it's
been my previous experience that they will refer me to the kernel
subsystem mailing list, but I will wait to hear back from them, and
see what they say.

- mike


- mike


On Sun, Aug 8, 2021 at 3:47 PM Chuck Lever III <chuck.lever@oracle.com> wrote:
>
>
>
> > On Aug 8, 2021, at 6:37 PM, Mike Javorski <mike.javorski@gmail.com> wrote:
> >
> > I have been experiencing nfs file access hangs with multiple release
> > versions of the 5.13.x linux kernel. In each case, all file transfers
> > freeze for 5-10 seconds and then resume. This seems worse when reading
> > through many files sequentially.
> >
> > My server:
> > - Archlinux w/ a distribution provided kernel package
> > - filesystems exported with "rw,sync,no_subtree_check,insecure" options
> >
> > Client:
> > - Archlinux w/ latest distribution provided kernel (5.13.9-arch1-1 at writing)
> > - nfs mounted via /net autofs with "soft,nodev,nosuid" options
> > (ver=4.2 is indicated in mount)
> >
> > I have tried the 5.13.x kernel several times since the first arch
> > release (most recently with 5.13.9-arch1-1), all with similar results.
> > Each time, I am forced to downgrade the linux package to a 5.12.x
> > kernel (5.12.15-arch1 as of writing) to clear up the transfer issues
> > and stabilize performance. No other changes are made between tests. I
> > have confirmed the freezing behavior using both ext4 and btrfs
> > filesystems exported from this server.
> >
> > At this point I would appreciate some guidance in what to provide in
> > order to diagnose and resolve this issue. I don't have a lot of kernel
> > debugging experience, so instruction would be helpful.
>
> Hi Mike-
>
> Thanks for the report.
>
> Since you are using a distribution kernel and don't have much
> kernel debugging experience, we typically ask you to work with
> your distributor first. linux-nfs@ is a developer's mailing
> list, we're not really prepared to provide user support.
>
> When you report the problem to Arch, you might want to have
> a description of your workload (especially if you can
> reproduce the problem often with a particular application
> or user activity), and maybe be prepared to capture a network
> trace of the failure using tcpdump.
>
>
> --
> Chuck Lever
>
>
>