Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <be0ef41fc2d62fb11d731754833d68e424a422ee.camel@kernel.org>
Subject: Re: [PATCH] nfsd: fix potential race in nfs4_find_file
From:   Jeff Layton <jlayton@kernel.org>
To:     NeilBrown <neilb@suse.de>
Cc:     chuck.lever@oracle.com, linux-nfs@vger.kernel.org
Date:   Fri, 06 Jan 2023 07:19:03 -0500
In-Reply-To: <167295993121.13974.8791979932693514625@noble.neil.brown.name>
References: <20230105121823.21935-1-jlayton@kernel.org>
         <167295993121.13974.8791979932693514625@noble.neil.brown.name>
Content-Type: text/plain; charset="ISO-8859-15"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.46.2 (3.46.2-1.fc37) 
MIME-Version: 1.0
Precedence: bulk

On Fri, 2023-01-06 at 10:05 +1100, NeilBrown wrote:
> On Thu, 05 Jan 2023, Jeff Layton wrote:
> > Even though there is a WARN_ON_ONCE check, it seems possible for
> > nfs4_find_file to race with the destruction of an fi_deleg_file while
> > trying to take a reference to it.
> >=20
> > put_deleg_file is done while holding the fi_lock. Take and hold it
> > when dealing with the fi_deleg_file in nfs4_find_file.
> >=20
> > Signed-off-by: Jeff Layton <jlayton@kernel.org>
> > ---
> >  fs/nfsd/nfs4state.c | 16 ++++++++++------
> >  1 file changed, 10 insertions(+), 6 deletions(-)
> >=20
> > diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> > index b68238024e49..3df3ae84bd07 100644
> > --- a/fs/nfsd/nfs4state.c
> > +++ b/fs/nfsd/nfs4state.c
> > @@ -6417,23 +6417,27 @@ nfsd4_lookup_stateid(struct nfsd4_compound_stat=
e *cstate,
> >  static struct nfsd_file *
> >  nfs4_find_file(struct nfs4_stid *s, int flags)
> >  {
> > +	struct nfsd_file *ret =3D NULL;
> > +
> >  	if (!s)
> >  		return NULL;
> > =20
> >  	switch (s->sc_type) {
> >  	case NFS4_DELEG_STID:
> > -		if (WARN_ON_ONCE(!s->sc_file->fi_deleg_file))
> > -			return NULL;
> > -		return nfsd_file_get(s->sc_file->fi_deleg_file);
> > +		spin_lock(&s->sc_file->fi_lock);
> > +		if (!WARN_ON_ONCE(!s->sc_file->fi_deleg_file))
> > +			ret =3D nfsd_file_get(s->sc_file->fi_deleg_file);
> > +		spin_unlock(&s->sc_file->fi_lock);
> > +		break;
>=20
> As an nfsd_file is freed with rcu, we don't need the spinlock.
>=20
>  rcu_read_lock()
>  ret =3D rcu_dereference(s->sc_file->fi_deleg_file);
>  if (ret)
> 	ret =3D nfsd_file_get(ret);
>  rcu_read_unlock();
>=20
> You could even put the NULL test in nfsd_file_get() and have:
>=20
>  rcu_read_lock()l;
>  ret =3D nfsd_file_get(rcu_dereference(s->sc_file->fi_deleg_file));
>  rcu_read_unlock();
>=20
> but that might not be a win.
>=20
> I agree with Chuck that the WARNing isn't helpful.
>=20
> NeilBrown
>=20

Ok, I took a look at this.

To do it right, we'd need to annotate the fi_deleg_file field with
__rcu. That means we'd need to clean up a bunch of existing
fi_deleg_file accesses to properly use rcu_dereference_protected.

This is probably worthwhile stuff to do, but it's a larger patch series
and will touch a bunch of unrelated delegation handling. At this point,
I think I'd rather just keep the spinlocking here since that should be
safe. Cleaning up delegation handling is a longer-term project that I'd
rather table for now.

I will remove the WARN_ON_ONCE though, and I think allowing
nfsd_file_get to accept a NULL pointer is probably a good thing too.
I'll resend a new series in a bit.

>=20
> >  	case NFS4_OPEN_STID:
> >  	case NFS4_LOCK_STID:
> >  		if (flags & RD_STATE)
> > -			return find_readable_file(s->sc_file);
> > +			ret =3D find_readable_file(s->sc_file);
> >  		else
> > -			return find_writeable_file(s->sc_file);
> > +			ret =3D find_writeable_file(s->sc_file);
> >  	}
> > =20
> > -	return NULL;
> > +	return ret;
> >  }
> > =20
> >  static __be32
> > --=20
> > 2.39.0
> >=20
> >=20

--=20
Jeff Layton <jlayton@kernel.org>