Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Thu, 3 Dec 2020 16:13:28 -0500
From:   "bfields@fieldses.org" <bfields@fieldses.org>
To:     Trond Myklebust <trondmy@hammerspace.com>
Cc:     "daire@dneg.com" <daire@dneg.com>,
        "linux-cachefs@redhat.com" <linux-cachefs@redhat.com>,
        "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
Subject: Re: Adventures in NFS re-exporting
Message-ID: <20201203211328.GC27931@fieldses.org>
References: <279389889.68934777.1603124383614.JavaMail.zimbra@dneg.com>
 <635679406.70384074.1603272832846.JavaMail.zimbra@dneg.com>
 <20201109160256.GB11144@fieldses.org>
 <1744768451.86186596.1605186084252.JavaMail.zimbra@dneg.com>
 <1055884313.92996091.1606250106656.JavaMail.zimbra@dneg.com>
 <20201124211522.GC7173@fieldses.org>
 <932244432.93596532.1606324491501.JavaMail.zimbra@dneg.com>
 <1403656117.98163597.1606998035261.JavaMail.zimbra@dneg.com>
 <20201203185109.GB27931@fieldses.org>
 <4903965f2beb742e0eca089b5db8aa3a4cabb7f0.camel@hammerspace.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <4903965f2beb742e0eca089b5db8aa3a4cabb7f0.camel@hammerspace.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Precedence: bulk

On Thu, Dec 03, 2020 at 08:27:39PM +0000, Trond Myklebust wrote:
> On Thu, 2020-12-03 at 13:51 -0500, bfields wrote:
> > I've been scratching my head over how to handle reboot of a re-
> > exporting
> > server.  I think one way to fix it might be just to allow the re-
> > export
> > server to pass along reclaims to the original server as it receives
> > them
> > from its own clients.  It might require some protocol tweaks, I'm not
> > sure.  I'll try to get my thoughts in order and propose something.
> > 
> 
> It's more complicated than that. If the re-exporting server reboots,
> but the original server does not, then unless that re-exporting server
> persisted its lease and a full set of stateids somewhere, it will not
> be able to atomically reclaim delegation and lock state on the server
> on behalf of its clients.

By sending reclaims to the original server, I mean literally sending new
open and lock requests with the RECLAIM bit set, which would get brand
new stateids.

So, the original server would invalidate the existing client's previous
clientid and stateids--just as it normally would on reboot--but it would
optionally remember the underlying locks held by the client and allow
compatible lock reclaims.

Rough attempt:

	https://wiki.linux-nfs.org/wiki/index.php/Reboot_recovery_for_re-export_servers

Think it would fly?

--b.