Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Chuck Lever <chuck.lever@oracle.com>
To:     Bruce Fields <bfields@fieldses.org>
CC:     Neil Brown <neilb@suse.de>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: releasing result pages in svc_xprt_release()
Thread-Topic: releasing result pages in svc_xprt_release()
Thread-Index: AQHW9lp7CEvfS3Emm06zTXNJoADK66o/M5eAgAAGcQCAAy+VgIAACaAAgAGDpoCABhUWgIAADuyAgAFb7QA=
Date:   Sat, 6 Feb 2021 17:59:09 +0000
Message-ID: <4621E2B0-0D07-4AC0-A829-A4CD23E71EF1@oracle.com>
References: <811BE98B-F196-4EC1-899F-6B62F313640C@oracle.com>
 <87im7ffjp0.fsf@notabene.neil.brown.name>
 <597824E7-3942-4F11-958F-A6E247330A9E@oracle.com>
 <878s88fz6s.fsf@notabene.neil.brown.name>
 <700333BB-0928-4772-ADFB-77D92CCE169C@oracle.com>
 <87wnvre5cy.fsf@notabene.neil.brown.name>
 <9FC8FB98-2DD7-4B78-BD72-918F91FA11D9@oracle.com>
 <20210205211351.GC32030@fieldses.org>
In-Reply-To: <20210205211351.GC32030@fieldses.org>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <C99965F5386276468EBC5023A7AFF313@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: ddf57d57-78dc-46f4-62f7-08d8cac8e768
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Feb 2021 17:59:09.6418
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: kC1K3NI+4eaHYr3Fyw0zX19g4qCTInYGGf5GYaN9X+F+RKcx/gTl57VCVqUuUGuvyTqxKDX+2k8dSQX+5+D5qQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3272
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9887 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 suspectscore=0
 adultscore=0 phishscore=0 bulkscore=0 mlxlogscore=999 spamscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2009150000 definitions=main-2102060130
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9887 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 malwarescore=0
 priorityscore=1501 bulkscore=0 spamscore=0 impostorscore=0 mlxscore=0
 suspectscore=0 mlxlogscore=999 adultscore=0 clxscore=1015
 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2009150000 definitions=main-2102060130
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Feb 5, 2021, at 4:13 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>=20
> On Fri, Feb 05, 2021 at 08:20:28PM +0000, Chuck Lever wrote:
>> Baby steps.
>>=20
>> Because I'm perverse I started with bulk page freeing. In the course
>> of trying to invent a new API, I discovered there already is a batch
>> free_page() function called release_pages().
>>=20
>> It seems to work as advertised for pages that are truly no longer
>> in use (ie RPC/RDMA pages) but not for pages that are still in flight
>> but released (ie TCP pages).
>>=20
>> release_pages() chains the pages in the passed-in array onto a list
>> by their page->lru fields. This seems to be a problem if a page
>> is still in use.
>=20
> I thought I remembered reading an lwn article about bulk page
> allocation.  Looking around now all I can see is
>=20
> 	https://lwn.net/Articles/684616/
> 	https://lwn.net/Articles/711075/
>=20
> and I can't tell if any of that work was ever finished.

Jesper is the engineer at Red Hat I alluded to earlier, and this is
similar to what I discussed with him. I didn't see anything like
alloc_pages_bulk() when sniffing around in v5.11-rc, but my search
was not exhaustive.

I think freeing pages, for NFSD, is complicated by the need to release
pages that are still in use (either by the page cache or by the network
layer). The page freeing logic in RPC/RDMA is freeing pages that are
not in use by anyone, and I have a couple of clear approaches that
eliminate the need for that logic completely.

Therefore, IMO concentrating on making svc_alloc_arg() more efficient
should provide the biggest bang for both socket and RDMA transports.

--
Chuck Lever