Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
From:   "NeilBrown" <neilb@suse.de>
To:     "Jeff Layton" <jlayton@kernel.org>
Cc:     "Chuck Lever" <chuck.lever@oracle.com>,
        "Linux NFS Mailing List" <linux-nfs@vger.kernel.org>
Subject: Re: nfsd: another possible delegation race
In-reply-to: <fb9c520e8bd9a034eeb10285c03dcf5cd6a660c9.camel@kernel.org>
References: <166486048770.14457.133971372966856907@noble.neil.brown.name>,
 <fb9c520e8bd9a034eeb10285c03dcf5cd6a660c9.camel@kernel.org>
Date:   Wed, 05 Oct 2022 11:25:09 +1100
Message-id: <166492950933.14457.10186344229031903354@noble.neil.brown.name>
Precedence: bulk

On Tue, 04 Oct 2022, Jeff Layton wrote:
> On Tue, 2022-10-04 at 16:14 +1100, NeilBrown wrote:
> > Hi,
> >  I have a customer who experienced a crash in nfsd which appears to be
> >  related to delegation return.  I cannot completely rule-out
> >   Commit 548ec0805c39 ("nfsd: fix use-after-free due to delegation race")
...
> 
> Ok, so a DELEGRETURN is racing with a lease break?
> 

and that is exactly what the above mentioned commit fixes.  I now see
that it is the right fix for my problem as well.

Putting the delegation_hashed() check in nfsd_break_deleg_cb() doesn't
help because the problematic list-add to del_recall_lru doesn't happen
until later in a different thread, and so the nfs4_delegation could
still get unhashed before that thread adds it to the list.  The above
commit adds the protection at the right place.

Because being on ->del_recall_lru doesn't imply a reference, it is easy
for the delegation to be freed before the laundromat finds it on the
list.

As mentioned in my reply to Chuck, the refcount_inc() gives a "saturated"
error rather than a "use-after-free" error because when the delegation
was freed, slub (which SUSE uses) puts a pointer at the start of the
memory, which is exactly where the refcount is.  So it has a good chance
of becoming negative.

So my symptoms are now completely explained.  Thanks to both of you for
your help.

NeilBrown