Date:   Thu, 9 Dec 2021 22:05:48 +0100 (CET)
From:   Richard Weinberger <richard@nod.at>
To:     linux-nfs@vger.kernel.org
Cc:     luis.turcitu@appsbroker.com, chris.chilvers@appsbroker.com,
        david.young@appsbroker.com, david <david@sigma-star.at>,
        bfields@fieldses.org,
        david oberhollenzer <david.oberhollenzer@sigma-star.at>
Message-ID: <1576494286.153679.1639083948872.JavaMail.zimbra@nod.at>
Subject: Improving NFS re-export
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Thread-Index: DIlIQMBpkOIKFedlU9SceqK8F8UnKA==
Thread-Topic: Improving NFS re-export
Precedence: bulk

Hello NFS list,

I'd like to improve the NFS re-export feature, especially wrt. crossmounts.
Currently a NFS client will face EIO when crossing a mount point on the re-exporting server.
This was discussed here[0]. While in that discussion the assumption was that check_export()
in fs/nfsd/export.c emits EIO I did further experiments and realized that EIO actually
comes from the NFS client side of the re-exporting server.

nfs_encode_fh() in fs/nfs/export.c checks for IS_AUTOMOUNT(inode), if this is the case
it refuses to create a new file handle.
So while accessing /files/disk2 directly on the re-exporting server triggers an automount,
accessing via nfsd the export function of the client side gives up.

AFAIU the suggested proxy-only-mode[1] will not address this problem, right?

One workaround is manually adding an export for each volume on the re-exporting server.
This kinda works but is tedious and error prone.

I have a crazy idea how to automate this:
Since nfs_encode_fh() in the NFS client side of the re-exporting server can detect
crossing mounts, we could install a new export on the sever side as soon the
IS_AUTOMOUNT(inode) case arises. We could even use the same fsid.
What do you think?

Another obstacle is file handle wrapping.
When re-exporting, the NFS client side adds inode and file information to each file handle,
the server side also adds information. In my test setup this enlarges a 16 bytes file handle
to 40 bytes.
The proxy-only-mode won't help us either here.

Did you consider using the opaque file handle from the server as lookup key in a
(persisted) data structure?
That way at least the client side of the re-exporting server no longer has to enlarge
the file handle with inode and file type information.
If the re-exporting server re-exports just one server (proxy-only-mode) we could also
skip adding the fsid to the handle.
What do you think?

I'm looking forward to hear your comments.

Thanks,
//richard

[0] https://marc.info/?l=linux-nfs&m=161670807413876&w=2
[1] https://linux-nfs.org/wiki/index.php/NFS_proxy-only_mode