Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <a47cc610af621e95ba359388e93d988f1ef5b17f.camel@kernel.org>
Subject: Re: [PATCH 1/1] NFSD: fix use-after-free in __nfs42_ssc_open()
From:   Jeff Layton <jlayton@kernel.org>
To:     Greg KH <gregkh@linuxfoundation.org>
Cc:     dai.ngo@oracle.com, chuck.lever@oracle.com, kolga@netapp.com,
        hdthky0@gmail.com, linux-nfs@vger.kernel.org, security@kernel.org
Date:   Mon, 12 Dec 2022 09:31:19 -0500
In-Reply-To: <Y5czwRabTFiwah2b@kroah.com>
References: <1670786549-27041-1-git-send-email-dai.ngo@oracle.com>
         <7f68cb23445820b4a1c12b74dce0954f537ae5e2.camel@kernel.org>
         <56b0cb4f-dfe9-6892-7fef-1a2965cf1d99@oracle.com>
         <0ab8efccae708faa092a56c6935c33564814bfed.camel@kernel.org>
         <Y5czwRabTFiwah2b@kroah.com>
Content-Type: text/plain; charset="ISO-8859-15"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.46.2 (3.46.2-1.fc37) 
MIME-Version: 1.0
Precedence: bulk

On Mon, 2022-12-12 at 14:59 +0100, Greg KH wrote:
> On Mon, Dec 12, 2022 at 08:40:31AM -0500, Jeff Layton wrote:
> > On Mon, 2022-12-12 at 05:34 -0800, dai.ngo@oracle.com wrote:
> > > On 12/12/22 4:22 AM, Jeff Layton wrote:
> > > > On Sun, 2022-12-11 at 11:22 -0800, Dai Ngo wrote:
> > > > > Problem caused by source's vfsmount being unmounted but remains
> > > > > on the delayed unmount list. This happens when nfs42_ssc_open()
> > > > > return errors.
> > > > > Fixed by removing nfsd4_interssc_connect(), leave the vfsmount
> > > > > for the laundromat to unmount when idle time expires.
> > > > >=20
> > > > > Reported-by: Xingyuan Mo <hdthky0@gmail.com>
> > > > > Signed-off-by: Dai Ngo <dai.ngo@oracle.com>
> > > > > ---
> > > > >   fs/nfsd/nfs4proc.c | 23 +++++++----------------
> > > > >   1 file changed, 7 insertions(+), 16 deletions(-)
> > > > >=20
> > > > > diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> > > > > index 8beb2bc4c328..756e42cf0d01 100644
> > > > > --- a/fs/nfsd/nfs4proc.c
> > > > > +++ b/fs/nfsd/nfs4proc.c
> > > > > @@ -1463,13 +1463,6 @@ nfsd4_interssc_connect(struct nl4_server *=
nss, struct svc_rqst *rqstp,
> > > > >   	return status;
> > > > >   }
> > > > >  =20
> > > > > -static void
> > > > > -nfsd4_interssc_disconnect(struct vfsmount *ss_mnt)
> > > > > -{
> > > > > -	nfs_do_sb_deactive(ss_mnt->mnt_sb);
> > > > > -	mntput(ss_mnt);
> > > > > -}
> > > > > -
> > > > >   /*
> > > > >    * Verify COPY destination stateid.
> > > > >    *
> > > > > @@ -1572,11 +1565,6 @@ nfsd4_cleanup_inter_ssc(struct vfsmount *s=
s_mnt, struct file *filp,
> > > > >   {
> > > > >   }
> > > > >  =20
> > > > > -static void
> > > > > -nfsd4_interssc_disconnect(struct vfsmount *ss_mnt)
> > > > > -{
> > > > > -}
> > > > > -
> > > > >   static struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
> > > > >   				   struct nfs_fh *src_fh,
> > > > >   				   nfs4_stateid *stateid)
> > > > > @@ -1762,7 +1750,8 @@ static int nfsd4_do_async_copy(void *data)
> > > > >   		struct file *filp;
> > > > >  =20
> > > > >   		filp =3D nfs42_ssc_open(copy->ss_mnt, &copy->c_fh,
> > > > > -				      &copy->stateid);
> > > > > +					&copy->stateid);
> > > > > +
> > > > >   		if (IS_ERR(filp)) {
> > > > >   			switch (PTR_ERR(filp)) {
> > > > >   			case -EBADF:
> > > > > @@ -1771,7 +1760,7 @@ static int nfsd4_do_async_copy(void *data)
> > > > >   			default:
> > > > >   				nfserr =3D nfserr_offload_denied;
> > > > >   			}
> > > > > -			nfsd4_interssc_disconnect(copy->ss_mnt);
> > > > > +			/* ss_mnt will be unmounted by the laundromat */
> > > > >   			goto do_callback;
> > > > >   		}
> > > > >   		nfserr =3D nfsd4_do_copy(copy, filp, copy->nf_dst->nf_file,
> > > > > @@ -1852,8 +1841,10 @@ nfsd4_copy(struct svc_rqst *rqstp, struct =
nfsd4_compound_state *cstate,
> > > > >   	if (async_copy)
> > > > >   		cleanup_async_copy(async_copy);
> > > > >   	status =3D nfserrno(-ENOMEM);
> > > > > -	if (nfsd4_ssc_is_inter(copy))
> > > > > -		nfsd4_interssc_disconnect(copy->ss_mnt);
> > > > > +	/*
> > > > > +	 * source's vfsmount of inter-copy will be unmounted
> > > > > +	 * by the laundromat
> > > > > +	 */
> > > > >   	goto out;
> > > > >   }
> > > > >  =20
> > > > This looks reasonable at first glance, but I have some concerns wit=
h the
> > > > refcounting around ss_mnt elsewhere in this code.=A0nfsd4_ssc_setup=
_dul
> > > > looks for an existing connection and bumps the ni->nsui_refcnt if i=
t
> > > > finds one.
> > > >=20
> > > > But then later, nfsd4_cleanup_inter_ssc has a couple of cases where=
 it
> > > > just does a bare mntput:
> > > >=20
> > > >          if (!nn) {
> > > >                  mntput(ss_mnt);
> > > >                  return;
> > > >          }
> > > > ...
> > > >          if (!found) {
> > > >                  mntput(ss_mnt);
> > > >                  return;
> > > >          }
> > > >=20
> > > > The first one looks bogus. Can net_generic return NULL? If so how, =
and
> > > > why is it not a problem elsewhere in the kernel?
> > >=20
> > > it looks like net_generic can not fail, no where else check for NULL
> > > so I will remove this check.
> > >=20
> > > >=20
> > > > For the second case, if the ni is no longer on the list, where did =
the
> > > > extra ss_mnt reference come from? Maybe that should be a WARN_ON or
> > > > BUG_ON?
> > >=20
> > > if ni is not found on the list then it's a bug somewhere so I will ad=
d
> > > a BUG_ON on this.
> > >=20
> >=20
> > Probably better to just WARN_ON and let any references leak in that
> > case. A BUG_ON implies a panic in some environments, and it's best to
> > avoid that unless there really is no choice.
>=20
> WARN_ON also causes machines to boot that have panic_on_warn enabled.
> Why not just handle the error and keep going?  Why panic at all?
>=20

Who the hell sets panic_on_warn (outside of testing environments)? I'm
suggesting a WARN_ON because not finding an entry at this point
represents a bug that we'd want reported.

The caller should hold a reference to the object that holds a vfsmount
reference. It relies on that vfsmount to do a copy. If it's gone at this
point where we're releasing that reference, then we're looking at a
refcounting bug of some sort.

I would expect anyone who sets panic_on_warn to _desire_ a panic in this
situation. After all, they asked for it. Presumably they want it to do
some coredump analysis or something?

It is debatable whether the stack trace at this point would be helpful
though, so you might consider a pr_warn or something less log-spammy.
--=20
Jeff Layton <jlayton@kernel.org>