Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Sender: Salvatore Bonaccorso <salvatore.bonaccorso@gmail.com>
Date:   Sat, 20 Feb 2021 21:04:07 +0100
From:   Salvatore Bonaccorso <carnil@debian.org>
To:     Anton Ivanov <anton.ivanov@cambridgegreys.com>
Cc:     Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        940821@bugs.debian.org, linux-nfs@vger.kernel.org,
        bfields@fieldses.org, chuck.lever@oracle.com
Subject: Re: NFS Caching broken in 4.19.37
Message-ID: <YDFrN0rZAJBbouly@eldamar.lan>
References: <5022bdc4-9f3e-9756-cbca-ada37f88ecc7@cambridgegreys.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <5022bdc4-9f3e-9756-cbca-ada37f88ecc7@cambridgegreys.com>
Precedence: bulk

Hi,

On Mon, Jul 08, 2019 at 07:19:54PM +0100, Anton Ivanov wrote:
> Hi list,
> 
> NFS caching appears broken in 4.19.37.
> 
> The more cores/threads the easier to reproduce. Tested with identical
> results on Ryzen 1600 and 1600X.
> 
> 1. Mount an openwrt build tree over NFS v4
> 2. Run make -j `cat /proc/cpuinfo | grep vendor | wc -l` ; make clean in a
> loop
> 3. Result after 3-4 iterations:
> 
> State on the client
> 
> ls -laF /var/autofs/local/src/openwrt/build_dir/target-mips_24kc_musl/linux-ar71xx_tiny/linux-4.14.125/arch/mips/include/generated/uapi/asm
> 
> total 8
> drwxr-xr-x 2 anivanov anivanov 4096 Jul  8 11:40 ./
> drwxr-xr-x 3 anivanov anivanov 4096 Jul  8 11:40 ../
> 
> State as seen on the server (mounted via nfs from localhost):
> 
> ls -laF /var/autofs/local/src/openwrt/build_dir/target-mips_24kc_musl/linux-ar71xx_tiny/linux-4.14.125/arch/mips/include/generated/uapi/asm
> total 12
> drwxr-xr-x 2 anivanov anivanov 4096 Jul  8 11:40 ./
> drwxr-xr-x 3 anivanov anivanov 4096 Jul  8 11:40 ../
> -rw-r--r-- 1 anivanov anivanov   32 Jul  8 11:40 ipcbuf.h
> 
> Actual state on the filesystem:
> 
> ls -laF /exports/work/src/openwrt/build_dir/target-mips_24kc_musl/linux-ar71xx_tiny/linux-4.14.125/arch/mips/include/generated/uapi/asm
> total 12
> drwxr-xr-x 2 anivanov anivanov 4096 Jul  8 11:40 ./
> drwxr-xr-x 3 anivanov anivanov 4096 Jul  8 11:40 ../
> -rw-r--r-- 1 anivanov anivanov   32 Jul  8 11:40 ipcbuf.h
> 
> So the client has quite clearly lost the plot. Telling it to drop caches and
> re-reading the directory shows the file present.
> 
> It is possible to reproduce this using a linux kernel tree too, just takes
> much more iterations - 10+ at least.
> 
> Both client and server run 4.19.37 from Debian buster. This is filed as
> debian bug 931500. I originally thought it to be autofs related, but IMHO it
> is actually something fundamentally broken in nfs caching resulting in cache
> corruption.

According to the reporter downstream in Debian, at
https://bugs.debian.org/940821#26 thi seem still reproducible with
more recent kernels than the initial reported. Is there anything Anton
can provide to try to track down the issue?

Anton, can you reproduce with current stable series?

Regards,
Salvatore