Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Chuck Lever <chuck.lever@oracle.com>
To:     Bruce Fields <bfields@fieldses.org>
CC:     Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH v1 19/42] SUNRPC: Fix xdr_get_next_encode_buffer() page
 boundary handling
Thread-Topic: [PATCH v1 19/42] SUNRPC: Fix xdr_get_next_encode_buffer() page
 boundary handling
Thread-Index: AQHXDq5E9L4cYzCeI0iQSySXwa3yeapxRKEAgAEl6oCAABNjgIAAGDkA
Date:   Wed, 3 Mar 2021 18:19:33 +0000
Message-ID: <D8625219-C758-44C0-A74E-272019E12C2E@oracle.com>
References: <161461145466.8508.13379815439337754427.stgit@klimt.1015granger.net>
 <161461183307.8508.17196295994390119297.stgit@klimt.1015granger.net>
 <20210302221130.GG3400@fieldses.org>
 <592A34CB-C178-4272-8905-F3BA95BCE299@oracle.com>
 <20210303165251.GB1282@fieldses.org>
In-Reply-To: <20210303165251.GB1282@fieldses.org>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <0F8661984914B94089180FA0854B9235@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: cf62f906-1d1e-4bf9-525c-08d8de70e521
X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Mar 2021 18:19:33.4227
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ok4/d3EPIYHTEF1AWZypRo8zf0/fszJ7wNepsIqyrhU4nj4vHDKg7/FzoKkRxR9RpDM2rpR8aynlVgYI2JqS5g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3333
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9912 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 suspectscore=0
 mlxlogscore=999 bulkscore=0 adultscore=0 phishscore=0 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000
 definitions=main-2103030128
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9912 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 lowpriorityscore=0 clxscore=1015
 priorityscore=1501 mlxlogscore=999 suspectscore=0 malwarescore=0
 impostorscore=0 bulkscore=0 adultscore=0 mlxscore=0 phishscore=0
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2009150000 definitions=main-2103030128
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Mar 3, 2021, at 11:52 AM, Bruce Fields <bfields@fieldses.org> wrote:
>=20
> On Wed, Mar 03, 2021 at 03:43:28PM +0000, Chuck Lever wrote:
>> Hi Bruce-
>>=20
>> Thanks for your careful review of this series!
>>=20
>>=20
>>> On Mar 2, 2021, at 5:11 PM, J. Bruce Fields <bfields@fieldses.org> wrot=
e:
>>>=20
>>> On Mon, Mar 01, 2021 at 10:17:13AM -0500, Chuck Lever wrote:
>>>> The description of commit 2825a7f90753 ("nfsd4: allow encoding
>>>> across page boundaries") states:
>>>>=20
>>>>> Also we can't handle a new operation starting close to the end of
>>>>> a page.
>>>>=20
>>>> But does not detail why this is the case.
>>>=20
>>> That wasn't every helpful of me, sorry.
>>>=20
>>>> Subtracting the scratch buffer's "shift" value from the remaining
>>>> stream space seems to make reserving space close to the end of the
>>>> buf->pages array reliable.
>>>=20
>>> So, why is that?
>>>=20
>>> Thinking this through:
>>>=20
>>> When somebody asks for a buffer that would straddle a page boundary,
>>> with frag1bytes at the end of this page and frag2bytes at the start of
>>> the next page, we instead give them a buffer starting at start of the
>>> next page.  That gives them a nice contiguous buffer to write into.
>>> When they're done using it, we fix things up by copying what they wrote
>>> back to where it should be.
>>>=20
>>> That means we're temporarily wasting frag1bytes of space.  So, I don't
>>> know, maybe that's the logic behind subtracing frag1bytes from
>>> space_left.
>>>=20
>>> It means you may end up with xdr->end frag1bytes short of the next page=
.
>=20
> Wait, let me try that again:
>=20
> 	p =3D page_address(*xdr->page_ptr);
> 	xdr->p =3D (void *)p + frag2bytes;
> 	space_left =3D xdr->buf->buflen - xdr->buf->len - frag1bytes;
>        xdr->end =3D (void *)p + min_t(int, space_left, PAGE_SIZE);
>=20
> If you've still got a lot of buffer space left, then that'll put
> xdr->end frag2bytes past the end of a page, won't it?

So far I haven't heard any complaints from the kernel. I'll
instrument the code and see what comes up.


>>> I'm not sure that's right.
>>=20
>> Why would that not be OK? the next call to xdr_get_next_encode_buffer()
>> should do the right thing and bounce the new encoded data from the
>> next page into this one again.
>>=20
>> So far I have not encountered any problems. Would such a problem show
>> up with some frequency under normal use, or would it be especially
>> subtle?
>=20
> I mainly just want to make sure we've got a coherent idea what this code
> is doing....

Agreed, that's a good thing.


> For testing: large replies that aren't just read data are readdir and
> getacl.  So reading large directories with lots of variably-named files
> might be good. Also pynfs could easily send a single compound with lots
> of variable-sized reads, that might be interesting.

I've run the full git regression suite over NFSv3 many many times with
this patch applied. That includes unpacking via tar, a full build from
scratch, and then running thousands of individual tests.

So that doesn't exercise a particular corner case, but it does reflect
a broad variety of directory operations.


> Constructing a compound that will result in xdr_reserve_space calls that
> exactly hit the various corner cases may be hard.  But maybe if we just
> send a bunch of compounds that vary over some range we can still
> guarantee hitting those cases.
>=20
> --b.

--
Chuck Lever