Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
Organization: Red Hat UK Ltd. Registered Address: Red Hat UK Ltd, Amberley
        Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United
        Kingdom.
        Registered in England and Wales under Company Registration No. 3798903
From:   David Howells <dhowells@redhat.com>
To:     fstests@vger.kernel.org, samba-technical@lists.samba.org,
        linux-cifs@vger.kernel.org
cc:     dhowells@redhat.com, Steve French <sfrench@samba.org>,
        Paulo Alcantara <pc@manguebit.com>,
        Dave Chinner <david@fromorbit.com>,
        Filipe Manana <fdmanana@suse.com>,
        "Darrick J. Wong" <djwong@kernel.org>,
        linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Issues with FIEMAP, xfstests, Samba, ksmbd and CIFS
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <447323.1701860432.1@warthog.procyon.org.uk>
Date:   Wed, 06 Dec 2023 11:00:32 +0000
Message-ID: <447324.1701860432@warthog.procyon.org.uk>
Precedence: bulk

Hi,

I've been debugging apparent cifs failures with xfstests, in particular
generic/009, and I'm finding that the tests are failing because FIEMAP is not
returning exactly the expected extent map.

The problem is that the FSCTL_QUERY_ALLOCATED_RANGES smb RPC op can only
return a list of ranges that are allocated and does not return any other
information about those allocations or the gaps between them - and thus FIEMAP
cannot express this information to the extent that the test expects.

Further, as Steve also observed, the expectation that the individual subtests
should return exactly those ranges is flawed.  The filesystem is at liberty to
split extents, round up extents, bridge extents and automatically punch out
blocks of zeros.  xfstests/common/punch allows for some of this, but I wonder
if it needs to be more fuzzy.

I wonder if the best xfstests can be expected to check is that the data we
have written is within the allocated regions.

Which brings me on to FALLOC_FL_ZERO_RANGE - is this guaranteed to result in
an allocated region (if successful)?  Samba is translating FSCTL_SET_ZERO_DATA
to FALLOC_FL_PUNCH_HOLE, as is ksmbd, and then there is no allocated range to
report back (Samba and ksmbd use SEEK_HOLE/SEEK_DATA rather than FIEMAP -
would a ZERO_RANGE even show up with that?).

Finally, should the Linux cifs filesystem translate gaps in the result of
FSCTL_QUERY_ALLOCATED_RANGES into 'unwritten' extents rather than leaving them
as gaps in the list (to be reported as holes by xfs_io)?  This smacks a bit of
adjusting things for the sake of making the testsuite work when the testsuite
isn't quite compatible with the thing being tested.

So:

 - Should Samba and ksmbd be using FALLOC_FL_ZERO_RANGE rather than
   PUNCH_HOLE?

 - Should Samba and ksmbd be using FIEMAP rather than SEEK_DATA/HOLE?

 - Should xfstests be less exacting in its FIEMAP analysis - or should this be
   skipped for cifs?  I don't want to skip generic/009 as it checks some
   corner cases that need testing, but it may not be possible to make the
   exact extent matching work.

Thanks,
David