Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <CAM5tNy4d02dOyWsVk7Y-nFEyGjE=eo4nJgrap3M5DebDJz9ehw@mail.gmail.com>
 <42DE2EB5-E2E5-45D6-B0B9-8C63FD6DC67D@oracle.com>
In-Reply-To: <42DE2EB5-E2E5-45D6-B0B9-8C63FD6DC67D@oracle.com>
From:   Rick Macklem <rick.macklem@gmail.com>
Date:   Sat, 5 Aug 2023 15:05:59 -0700
Message-ID: <CAM5tNy7UL_mG20uHEgBjgK2B4d2JiT8Z6QknO1-6ByQ8GpOGNg@mail.gmail.com>
Subject: Re: RFC: new attributes
To:     Chuck Lever III <chuck.lever@oracle.com>
Cc:     Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        David Noveck <davenoveck@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Sat, Aug 5, 2023 at 7:51=E2=80=AFAM Chuck Lever III <chuck.lever@oracle.=
com> wrote:
>
> Hi Rick -
>
> > On Aug 4, 2023, at 6:18 PM, Rick Macklem <rick.macklem@gmail.com> wrote=
:
> >
> > Hi,
> >
> > I wrote an IETF draft proposing a few new attributes for NFSv4.2.
> > Since there did not seem to be interest in them, I just
> > let the draft expire.  However, David Noveck pinged
> > me w.r.t. it, so I thought I'd ask here about it.
> >
> > All the attributes are meant to be "read only, per server file system":
> > supported_ops - A bitmap of the operations supported.
> >     The motivation was that NFS4ERR_NOTSUPP is supposed to
> >      be "per server", although the rumour was that the Linux knfsd
> >      uses it "per server file system".
>
> Before crafting new protocol, we should have a look at server
> implementation behavior to see if it can be improved in this
> area.
>
> Is Linux the only problematic implementation? Send email,
> bug reports, or patches... we'll consider them.
>
This was discussed on the IETF working group mailing list some years
ago.  I was asking if NFS4ERR_NOTSUPP could be used "per server
file system" or "per server". Tom Haynes said his intent was "per server",
but that was not clear in the RFC.  The only place in any RFC where it
seemed to indicate "per server" was the definition for NFS4ERR_NOTSUPP
as follows:
15.1.1.5.  NFS4ERR_NOTSUPP (Error Code 10004)

   Operation not supported, either because the operation is an OPTIONAL
   one and is not supported by this server or because the operation MUST
   NOT be implemented in the current minor version.

Bruce Fields noted that he thought the Linux knfsd was doing NFS4ERR_NOTSUP=
P
w.r.t. optional 4.2 operations on a "per file system basis" and there was s=
ome
mumbling to the effect that it should be applicable "per server file system=
".

In FreeBSD, certain 4.2 operations (such as Allocate) can only be done
on certain file systems.
Without a way to indicate to a client that this operation is supported on
file system X but not file system Y, the server is forced to not support th=
e
operation. (It is currently controlled by a tunable that a sysadmin could
set incorrectly and result in NFS4ERR_NOTSUPP for some of the file
systems. As such, you could say that the FreeBSD server can do this.)

I do not have a Linux server with various types of file systems to confirm
if what Bruce Fields thought was the case is actually the case.

I
>
> > dir_cookie_rising - Only useful for directory delegations, which no
> >      one seems to be implementing.
>
> We've been talking privately and informally about implementing
> directory delegation in the Linux NFS server, so this one
> could be interesting. But there aren't enough details here to
> know whether this new attribute would be useful to us.
>
I wrote a bunch of code implementing directory delegations on FreeBSD,
but never completed the work to the point of testing.
I found that, for FreeBSD, it was infeasible to implement the client side
for server file systems where the directory offset cookie was not monotonic=
ally
increasing. (Basically maintaining ordering of "directory chunks" because t=
oo
difficult with being able to do so based on directory offset cookie orderin=
g.)

So, my implementation, if even completed, would only work for the case of
monotonically increasing directory offset cookies and detecting that that i=
s
not the case "on the fly" would have been messy.

Btw, I seem to recall that the Linux client optimizes directory handling
when directory offset cookies are monotonically increasing, but shuts
that down "when it sees otherwise". I do not know if the client could
be improved in this area if it knows up front?

>
> > seek_granularity - The smallest size of unallocated region reported
> >      be the Seek operation.  FreeBSD has a pathconf(2) variable called
> >      _PC_MIN_HOLE_SIZE that an application can use to decide if
> >      lseek(SEEK_DATA/SEEK_HOLE) is useful.
>
> I'm not aware of a scenario where the Linux server would provide
> a value not equal to 1, so it would be easy for us to implement.
A value of 1 is of limited use. If an application is going to use the
information (btw, I think this pathconf name was in Solaris?), the
size (such as 32K or 128K) can be more useful.
--> No point in doing Seek if the data is not sufficiently sparse.

How useful is this? I do not know (and since Linux apparently
does not provide this information, there will be few, if any,
applications that make use of it.)

>
> What would clients do with this information, aside from filling
> in a pathconf field? Might this value be of benefit for READ_PLUS?
As proposed, it does not give indications of "sparseness" for individual
files. It would, however, indicate if READ_PLUS can be useful.
--> If the server returns 0, there is no point in performing READ_PLUS.
(This was not explicitly stated in the draft, but should be.)

>
>
> > mandatory_br_locks - Byte range locks are mandatory.  No one
> >      seems to be implementing these, but a client needs to know
> >      that mandatory locking is being enforced so that it can cache
> >      data correctly.
>
> I don't have much exposure to mandatory locking, maybe Jeff
> could chime in on this one.
>
>
> > max_xattr_len - Allows the client to avoid attempting to Setxattr an
> >     attribute that is larger than the server file system supports.
>
> Can you elaborate on the problem you are trying to solve? Why
> isn't the current situation adequate?
For FreeBSD, an application can attempt to set a very large extended
attribute (I've actually done a 1Gbyte one on ZFS during testing).
As such, for NFSv4.2 it can attempt one up to the maximum allowable
size for a compound (a little over 1Mbyte).

Without this attribute, most servers will fail with NFS4ERR_XATTR2BIG,
resulting in a fair amount of "on the wire" traffic for some application
that insists on doing large ones a lot.

This attribute would allow the client to avoid putting Setxattr operations
with too large an attribute "on the wire", but it is a minor optimization.

rick

>
> Again, I don't think this would be difficult for the Linux
> server to implement, but I'd like to know why it's needed.
>
>
> --
> Chuck Lever
>
>