Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <3E21DFEA-8DF7-484B-8122-D578BFF7F9E0@oracle.com>
 <20220904131553.bqdsfbfhmdpuujd3@zlang-mailbox> <20221109041951.wlgxac3buutvettq@shindev>
In-Reply-To: <20221109041951.wlgxac3buutvettq@shindev>
From:   Filipe Manana <fdmanana@kernel.org>
Date:   Wed, 9 Nov 2022 10:36:04 +0000
Message-ID: <CAL3q7H5eV9Sb1axmNgvcbG7UrgGTH3AovaibQuWMz44Jfo-8_w@mail.gmail.com>
Subject: Re: generic/650 makes v6.0-rc client unusable
To:     Shinichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Cc:     Zorro Lang <zlang@redhat.com>,
        "fstests@vger.kernel.org" <fstests@vger.kernel.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        Chuck Lever III <chuck.lever@oracle.com>,
        "djwong@vger.kernel.org" <djwong@vger.kernel.org>,
        "linux-xfs@vger.kernel.org" <linux-xfs@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, Nov 9, 2022 at 4:22 AM Shinichiro Kawasaki
<shinichiro.kawasaki@wdc.com> wrote:
>
> On Sep 04, 2022 / 21:15, Zorro Lang wrote:
> > On Sat, Sep 03, 2022 at 06:43:29PM +0000, Chuck Lever III wrote:
> > > While investigating some of the other issues that have been
> > > reported lately, I've found that my v6.0-rc3 NFS/TCP client
> > > goes off the rails often (but not always) during generic/650.
> > >
> > > This is the test that runs a workload while offlining and
> > > onlining CPUs. My test client has 12 physical cores.
> > >
> > > The test appears to start normally, but then after a bit
> > > the NFS server workload drops to zero and the NFS mount
> > > disappears. I can't run programs (sudo, for example) on
> > > the client. Can't log in, even on the console. The console
> > > has a constant stream of "can't rotate log: Input/Output
> > > error" type messages.
>
> I also observe this failure when I ran fstests using btrfs on my HDDs.
> The failure is recreated almost always.

I'm wondering what do you get in dmesg, any traces?

I've excluded the test from my runs for over an year now, due to some
crash that I reported
to the mm and cpu hotplug people here:

https://lore.kernel.org/linux-mm/CAL3q7H4AyrZ5erimDyO7mOVeppd5BeMw3CS=wGbzrMZrp56ktA@mail.gmail.com/

Unfortunately I had no reply from anyone who works or maintains those
subsystems.

It didn't happen very often, and I haven't tested again with recent kernels.

>
> > >
> > > I haven't looked further into this yet. Actually I'm not
> > > quite sure where to start looking.
> > >
> > > I recently switched this client from a local /home to an
> > > NFS-mounted one, and that's where the xfstests are built
> > > and run from, fwiw.
> >
> > If most of users complain generic/650, I'd like to exclude g/650 from the
> > "auto" default run group. Any more points?
>
> +1. I wish to remove it from the "auto" group. Since I can not login to the test
> machine after the failure, I suggest to put it in the "dangerous" group.
>
> --
> Shin'ichiro Kawasaki