Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Hugh Dickins <hughd@google.com>
CC:     Linux-MM <linux-mm@kvack.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Regression in xfstests on tmpfs-backed NFS exports
Thread-Topic: Regression in xfstests on tmpfs-backed NFS exports
Thread-Index: AQHYSdpN1SFcUB4GDEa/4f/k4JlVpQ==
Date:   Wed, 6 Apr 2022 17:18:15 +0000
Message-ID: <673D708E-2DFA-4812-BB63-6A437E0C72EE@oracle.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?BgMGfwEygOpDL4N7nuBq8KmStKKhTsMV+srXWY/vUb0rAMvNoaQ9pOlPQ6ec?=
 =?us-ascii?Q?RxIlLDpopl0D1WvXIebtiVaTvVBjWq3OTcC7GQt0h8GJOu7739LxohrSnsBz?=
 =?us-ascii?Q?31xXxCkVU9cpcQwfaP8uDz5e+Md1ZQI2ofL44zGPblSMALH3XC5xgxh0OtbE?=
 =?us-ascii?Q?bxEWiG+wqc2DiqwfzV6qv/mkiiWtoPNbLhesDgoAMhl8wWoOPsnx4H6Tp8Ut?=
 =?us-ascii?Q?60QwD1A+vF4uSAEbr/Eia53Ey2iAIENRCwJYeU/VVJnm6i80Y52qryJaZg+2?=
 =?us-ascii?Q?VlJXZTn+38uHPO65AbkAWJ8AcuRwece+RXoJidN3QT9SCKq2hdssh0/9HCsf?=
 =?us-ascii?Q?vsNGcQ5RoILDdmqnu1A0bCgob97OwLVcF5Oy5reFn7jYTLoLrBKO8kkXyQuN?=
 =?us-ascii?Q?716MTP/9r6QDzzbgQS91X6JjxRqraY46LtRoqmW/3g1F8oE/BJpwX08yG0XL?=
 =?us-ascii?Q?1nU9tiV238wbPTH9SbtjAoKiLcRhcKYJUyWUuGDPMBt80sqj7TlnCzFQ5YFT?=
 =?us-ascii?Q?IwgRh7xZPQ8Dnjetx4YrSmNCsr+qT3KxUtzrgrOpZNUKZlBa8Vb1oi+pGYOV?=
 =?us-ascii?Q?C+XRC+kKO5Qbb7uWBAPfyQ4cuxeBW3YUNXseL/pBNJBESZm4Q1JgyHEvbZPq?=
 =?us-ascii?Q?N1SrIwMeXSudHYFtrtuRtsciOg+v0DWuKAWiuzEiug2Vgsaxj130JEh5wsKb?=
 =?us-ascii?Q?bFN1aNMOj4AuqUZTlCPjpDQ/4QGCzHVaoiLYyd3CDfCNzW2PNLXmHFebSb3e?=
 =?us-ascii?Q?sChA2/E7s5kq7Mi2vLGxewga+2HfzqU/F62m75EyJRHqYPeQcgaRPlvH/sCL?=
 =?us-ascii?Q?bapqVu1adObJ4OhmIUBpLKPrCbzIX8Y02dKXJsRLZg868usA5/dMdHk+dzWG?=
 =?us-ascii?Q?ceK3rtDfcyyp9ftos2fFk35mx4zQg7ERjq78TiukyGGl8Ss1CBpR3k9AlmH/?=
 =?us-ascii?Q?2j3KGDUlRoYR/y5v3XimRCazFGXs8MTDogaLD7f4++gtZzl1R3EgZK6e/gHd?=
 =?us-ascii?Q?67Cf/fXNy4wmmcHloZyUW6XRXUAFiT/G9BS1Pa70/3wOdyRJCkyQ5/CL5Qcg?=
 =?us-ascii?Q?N2BE5SDFDjrsjJPhoXvpzXJPY+oqPmEdmhKhcpcO1Do5n2GRhKGOClr4xtYT?=
 =?us-ascii?Q?igv6d5GomtmuplFvDmv/7Qu7S/brC2eoavY5qNUFaWd8RrvxL4hm0lq8urL8?=
 =?us-ascii?Q?G3XvEVsE2+vo0fdnPYtMTW5CK+fQb7PA6Dp1x5ZlPXuXAWwB+XmpROyAhhRf?=
 =?us-ascii?Q?i0ma8eMsZXVYzWmcaTvoHoZNKLkU16j5uT6iAWBUWY4a5/nqSeoE9YBRurWV?=
 =?us-ascii?Q?46SHoHFrxzff1CCf6Sf5qp3HDkV91bG+Ko5/Cnjsai1zs2v3DbzZeqPI3hVt?=
 =?us-ascii?Q?04WKiNGL1dMAw/EE95uUqMYzrztsYYLho8sAjqaZ/xtNyRuytLYKZQsjO7zI?=
 =?us-ascii?Q?2QhQ34TTd7RZ8NSetEvlxZ78XrvDaLF31tV0vRHsETwH1paODm+lF8XTaM2N?=
 =?us-ascii?Q?ZperN0MGSElX2mZDC2bJ8ufBqQOBHDrH9ymaPCjRM1IbDyMcF1KkE7GQK3Nm?=
 =?us-ascii?Q?FB0iMzvFPMAIAehi9yTk53TtbDropKQQL9NVUq9CrvYl8v6OWh7TPihELhOV?=
 =?us-ascii?Q?d4VpymETmtDRBJRe+2isgWFHUX5fJ8KkQWhBUM1pA07OHBKFuaQqJjZV0k2t?=
 =?us-ascii?Q?tyQ+/2P1O41v2a8+GXqfzRyMbx/LvS/kjmQW7kLxLQCDGFlXRJutpgUlcGDT?=
 =?us-ascii?Q?q5W6Y4eJtp660bmZxEwtxBwS9YHzp/E=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <080628C0D300A5438CC0A39C79967D5F@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bbfe1546-3a2e-4b79-8525-08da17f16fd2
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Apr 2022 17:18:15.6502
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: HAP/DQ11HuhG3u0GJ9sAaSygRTwm4OJAhQcmK/FuS8RHpqeJi1DELHmzHqzajB0vcCg0D4Mkozn46SuBCEBr/w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4644
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425,18.0.850
 definitions=2022-04-06_09:2022-04-06,2022-04-06 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxlogscore=786 spamscore=0
 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000
 definitions=main-2204060086
X-Proofpoint-ORIG-GUID: Lg4wm-Arm3P-xNRHnc-flzOxIo7i-ibv
X-Proofpoint-GUID: Lg4wm-Arm3P-xNRHnc-flzOxIo7i-ibv
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org

Good day, Hugh-

I noticed that several fsx-related tests in the xfstests suite are
failing after updating my NFS server to v5.18-rc1. I normally test
against xfs, ext4, btrfs, and tmpfs exports. tmpfs is the only export
that sees these new failures:

generic/075 2s ... [failed, exit status 1]- output mismatch (see /home/cel/=
src/xfstests/results//generic/075.out.bad)
    --- tests/generic/075.out	2014-02-13 15:40:45.000000000 -0500
    +++ /home/cel/src/xfstests/results//generic/075.out.bad	2022-04-05 16:3=
9:59.145991520 -0400
    @@ -4,15 +4,5 @@
     -----------------------------------------------
     fsx.0 : -d -N numops -S 0
     -----------------------------------------------
    -
    ------------------------------------------------
    -fsx.1 : -d -N numops -S 0 -x
    ------------------------------------------------
    ...
    (Run 'diff -u /home/cel/src/xfstests/tests/generic/075.out /home/cel/sr=
c/xfstests/results//generic/075.out.bad'  to see the entire diff)

generic/091 9s ... [failed, exit status 1]- output mismatch (see /home/cel/=
src/xfstests/results//generic/091.out.bad)
    --- tests/generic/091.out	2014-02-13 15:40:45.000000000 -0500
    +++ /home/cel/src/xfstests/results//generic/091.out.bad	2022-04-05 16:4=
1:24.329063277 -0400
    @@ -1,7 +1,75 @@
     QA output created by 091
     fsx -N 10000 -l 500000 -r PSIZE -t BSIZE -w BSIZE -Z -R -W
    -fsx -N 10000 -o 8192 -l 500000 -r PSIZE -t BSIZE -w BSIZE -Z -R -W
    -fsx -N 10000 -o 32768 -l 500000 -r PSIZE -t BSIZE -w BSIZE -Z -R -W
    -fsx -N 10000 -o 8192 -l 500000 -r PSIZE -t BSIZE -w BSIZE -Z -R -W
    -fsx -N 10000 -o 32768 -l 500000 -r PSIZE -t BSIZE -w BSIZE -Z -R -W
    -fsx -N 10000 -o 128000 -l 500000 -r PSIZE -t BSIZE -w BSIZE -Z -W
    ...
    (Run 'diff -u /home/cel/src/xfstests/tests/generic/091.out /home/cel/sr=
c/xfstests/results//generic/091.out.bad'  to see the entire diff)

generic/112 2s ... [failed, exit status 1]- output mismatch (see /home/cel/=
src/xfstests/results//generic/112.out.bad)
    --- tests/generic/112.out	2014-02-13 15:40:45.000000000 -0500
    +++ /home/cel/src/xfstests/results//generic/112.out.bad	2022-04-05 16:4=
1:38.511075170 -0400
    @@ -4,15 +4,4 @@
     -----------------------------------------------
     fsx.0 : -A -d -N numops -S 0
     -----------------------------------------------
    -
    ------------------------------------------------
    -fsx.1 : -A -d -N numops -S 0 -x
    ------------------------------------------------
    ...
    (Run 'diff -u /home/cel/src/xfstests/tests/generic/112.out /home/cel/sr=
c/xfstests/results//generic/112.out.bad'  to see the entire diff)

generic/127 49s ... - output mismatch (see /home/cel/src/xfstests/results//=
generic/127.out.bad)
    --- tests/generic/127.out	2016-08-28 12:16:20.000000000 -0400
    +++ /home/cel/src/xfstests/results//generic/127.out.bad	2022-04-05 16:4=
2:07.655099652 -0400
    @@ -4,10 +4,198 @@
     =3D=3D=3D FSX Light Mode, Memory Mapping =3D=3D=3D
     All 100000 operations completed A-OK!
     =3D=3D=3D FSX Standard Mode, No Memory Mapping =3D=3D=3D
    -All 100000 operations completed A-OK!
    +ltp/fsx -q -l 262144 -o 65536 -S 191110531 -N 100000 -R -W fsx_std_nom=
map
    +READ BAD DATA: offset =3D 0x9cb7, size =3D 0xfae3, fname =3D /tmp/mnt/=
manet.ib-2323703/fsx_std_nommap
    +OFFSET	GOOD	BAD	RANGE
    ...
    (Run 'diff -u /home/cel/src/xfstests/tests/generic/127.out /home/cel/sr=
c/xfstests/results//generic/127.out.bad'  to see the entire diff)

I bisected the problem to:

  56a8c8eb1eaf ("tmpfs: do not allocate pages on read")

generic/075 fails almost immediately without any NFS-level errors.
Likely this is data corruption rather than an overt I/O error.


--
Chuck Lever