Received-SPF: pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Content-Type: text/plain;
        charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: Re: [PATCH v2 1/2] SUNRPC: Fix buffer handling of GSS MIC without
 slack
From:   Chuck Lever <chuck.lever@oracle.com>
In-Reply-To: <9f9848f4cbb03b09c7f28f8a43fb27120703ae49.1568557832.git.bcodding@redhat.com>
Date:   Sun, 15 Sep 2019 12:43:05 -0400
Cc:     trond.myklebust@hammerspace.com,
        Anna Schumaker <anna.schumaker@netapp.com>, tibbs@math.uh.edu,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        Bruce Fields <bfields@fieldses.org>, km@cm4all.com
Content-Transfer-Encoding: quoted-printable
Message-Id: <8350AC46-9CFA-410D-AC0C-EF2ACE24FD74@oracle.com>
References: <9f9848f4cbb03b09c7f28f8a43fb27120703ae49.1568557832.git.bcodding@redhat.com>
To:     Benjamin Coddington <bcodding@redhat.com>
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk

Hi Ben-


> On Sep 15, 2019, at 11:41 AM, Benjamin Coddington =
<bcodding@redhat.com> wrote:
>=20
> The GSS Message Integrity Check data for krb5i may lie partially in =
the XDR
> reply buffer's pages and tail.  If so, we try to copy the entire MIC =
into
> free space in the tail.  But as the estimations of the slack space =
required
> for authentication and verification have improved there may be less =
free
> space in the tail to complete this copy -- see commit 2c94b8eca1a2
> ("SUNRPC: Use au_rslack when computing reply buffer size").  In fact, =
there
> may only be room in the tail for a single copy of the MIC, and not =
part of
> the MIC and then another complete copy.
>=20
> The real world failure reported is that `ls` of a directory on NFS may
> sometimes return -EIO, which can be traced back to =
xdr_buf_read_netobj()
> failing to find available free space in the tail to copy the MIC.
>=20
> Fix this by checking for the case of the MIC crossing the boundaries =
of
> head, pages, and tail. If so, shift the buffer until the MIC is =
contained
> completely within the pages or tail.  This allows the remainder of the
> function to create a sub buffer that directly address the complete =
MIC.
>=20
> Signed-off-by: Benjamin Coddington <bcodding@redhat.com>
> Cc: stable@vger.kernel.org # v5.1
> ---
> net/sunrpc/xdr.c | 32 +++++++++++++++++++-------------
> 1 file changed, 19 insertions(+), 13 deletions(-)
>=20
> diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c
> index 48c93b9e525e..a29ce73c3029 100644
> --- a/net/sunrpc/xdr.c
> +++ b/net/sunrpc/xdr.c
> @@ -1237,16 +1237,29 @@ xdr_encode_word(struct xdr_buf *buf, unsigned =
int base, u32 obj)
> EXPORT_SYMBOL_GPL(xdr_encode_word);
>=20
> /* If the netobj starting offset bytes from the start of xdr_buf is =
contained
> - * entirely in the head or the tail, set object to point to it; =
otherwise
> - * try to find space for it at the end of the tail, copy it there, =
and
> - * set obj to point to it. */
> + * entirely in the head, pages, or tail, set object to point to it; =
otherwise
> + * shift the buffer until it is contained entirely within the pages =
or tail.
> + */
> int xdr_buf_read_netobj(struct xdr_buf *buf, struct xdr_netobj *obj, =
unsigned int offset)
> {
> 	struct xdr_buf subbuf;
> +	unsigned int boundary;
>=20
> 	if (xdr_decode_word(buf, offset, &obj->len))
> 		return -EFAULT;
> -	if (xdr_buf_subsegment(buf, &subbuf, offset + 4, obj->len))
> +	offset +=3D 4;
> +
> +	/* Is the obj partially in the head? */
> +	boundary =3D buf->head[0].iov_len;
> +	if (offset < boundary && (offset + obj->len) > boundary)
> +		xdr_shift_buf(buf, boundary - offset);
> +
> +	/* Is the obj partially in the pages? */
> +	boundary +=3D buf->page_len;
> +	if (offset < boundary && (offset + obj->len) > boundary)
> +		xdr_shrink_pagelen(buf, boundary - offset);
> +
> +	if (xdr_buf_subsegment(buf, &subbuf, offset, obj->len))
> 		return -EFAULT;
>=20
> 	/* Is the obj contained entirely in the head? */
> @@ -1258,17 +1271,10 @@ int xdr_buf_read_netobj(struct xdr_buf *buf, =
struct xdr_netobj *obj, unsigned in
> 	if (subbuf.tail[0].iov_len =3D=3D obj->len)
> 		return 0;
>=20
> -	/* use end of tail as storage for obj:
> -	 * (We don't copy to the beginning because then we'd have
> -	 * to worry about doing a potentially overlapping copy.
> -	 * This assumes the object is at most half the length of the
> -	 * tail.) */
> +	/* obj is in the pages: move to end of the tail */

How about "/* Find a contiguous area in @buf to hold all of @obj */" ?


> 	if (obj->len > buf->buflen - buf->len)
> 		return -ENOMEM;
> -	if (buf->tail[0].iov_len !=3D 0)
> -		obj->data =3D buf->tail[0].iov_base + =
buf->tail[0].iov_len;
> -	else
> -		obj->data =3D buf->head[0].iov_base + =
buf->head[0].iov_len;
> +	obj->data =3D buf->tail[0].iov_base + buf->tail[0].iov_len;

Your new code assumes that when krb5i is in use, the upper layer will =
always
provide a non-NULL tail->iov_len. I wouldn't swear that will always be =
true:
The reason for the "if (buf->tail[0].iov_len)" check is to see whether =
the
upper layer indeed has set up a tail. iov_len will be non-zero only when =
the
upper layer has provided a tail buffer.


> we can definitely keep the check, but
> the second half of the statement also assumes a contiguous head/tail =
range.

Well, it assumes that there is space in the head buffer after its end. =
That's
not necessarily the tail. Are we sure that in the post-35e77d21baa0 =
world
there will always be enough space after head->iov_len?

A reasonable test here would be to enable SLUB poisoning and and try =
some
complex workloads on an NFSv4 krb5i mount.


> I think it's safe to just remove the test altogether and place the =
netobj at
> the end of the tail.


I'm not convinced :-) I'd like to see more justification for this claim.

This is why in the long run we are better off using a scratch buffer =
instead
of finding a spot in @buf. The rules about how the rq_rcv_buf is set up =
are
gray; this function makes a lot of "clever" assumptions about that, and =
that
makes this logic quite brittle.

Now that we have an xdr_stream in gss_unwrap_resp_integ(), I wonder if =
you
could use the stream's scratch xdr_buf. If not, a kmalloc should do the =
trick.


> 	__read_bytes_from_xdr_buf(&subbuf, obj->data, obj->len);
> 	return 0;
> }
> --=20
> 2.20.1
>=20

--
Chuck Lever