Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Neil Brown <neilb@suse.de>
CC:     Mike Javorski <mike.javorski@gmail.com>,
        Mel Gorman <mgorman@suse.com>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: NFS server regression in kernel 5.13 (tested w/ 5.13.9)
Thread-Topic: NFS server regression in kernel 5.13 (tested w/ 5.13.9)
Thread-Index: AQHXjKYRxKTFwD0+C02+ZMVduTdsB6tqSi4AgAAHiwCAAZhPAIAACtMAgAS9YICAAA1rAIAAA7UAgAMLuwCAAZFegIAAyYQAgAAzUQCAAG7RAIAE0AAAgAAF0YCAAxscgIAAOP4AgAAGqYCAASxAgIAGIIuAgAAkXICAACgEAIAAWTeAgAAMTwCAABHBgIAAdRaA
Date:   Fri, 27 Aug 2021 14:13:49 +0000
Message-ID: <6CC9C852-CEE3-4657-86AD-9D5759E2BE1C@oracle.com>
References: <CAOv1SKCmdtchm5Z2NU80o49tkrHpAkPFaHKj4-vLDN5bZNCz-Q@mail.gmail.com>
 <162846730406.22632.14734595494457390936@noble.neil.brown.name>
 <CAOv1SKBZ7sGBnvG9-M+De+s=CfU=H_GBs4hJah1E4ks+NSMyPw@mail.gmail.com>
 <CAOv1SKCUM5cGuXWAc7dsXtbmPMATqd245juC+S9gVXHWiZsvmQ@mail.gmail.com>
 <162855893202.12431.3423894387218130632@noble.neil.brown.name>
 <CAOv1SKAaSbfw53LWCCrvGCHESgdtCf5h275Zkzi9_uHkqnCrdg@mail.gmail.com>
 <162882238416.1695.4958036322575947783@noble.neil.brown.name>
 <CAOv1SKB_dsam7P9pzzh_SKCtA8uE9cyFdJ=qquEfhLT42-szPA@mail.gmail.com>
 <CAOv1SKDDOj5UeUwztrMSNJnLgSoEgD8OU55hqtLHffHvaCQzzA@mail.gmail.com>
 <162907681945.1695.10796003189432247877@noble.neil.brown.name>
 <87777C39-BDDA-4E1E-83FA-5B46918A66D3@oracle.com>
 <CAOv1SKA5ByO7PYQwvd6iBcPieWxEp=BfUZuigJ=7Hm4HAmTuMA@mail.gmail.com>
 <162915491276.9892.7049267765583701172@noble.neil.brown.name>
 <162941948235.9892.6790956894845282568@noble.neil.brown.name>
 <CAOv1SKAyr0Cixc8eQf8-Fdnf=9Db_xZGsweq9K2E5AkALFqavQ@mail.gmail.com>
 <CAOv1SKDDUFpgexZ_xYCe6c2-UCBK0+vicoG+LAtG2Zhispd_jg@mail.gmail.com>
 <162960371884.9892.13803244995043191094@noble.neil.brown.name>
 <CAOv1SKBePD6N-R0uETgcSPA-LZZ4895ZJDKTY7mYvhfu184OQQ@mail.gmail.com>
 <162966962721.9892.5962616727949224286@noble.neil.brown.name>
 <CAOv1SKB6xqyduf5L5hcXOe-xMN-UJOfFeE5eXVga3TviKuH0PA@mail.gmail.com>
 <163001427749.7591.7281634750945934559@noble.neil.brown.name>
 <CAOv1SKC+3LXhM+L9MwU2D03bpeof55-g+i=r3SWEjVWcPVCi8Q@mail.gmail.com>
 <163004202961.7591.12633163545286005205@noble.neil.brown.name>
 <CAOv1SKDTcg5WDp5zf3ZGL0enJ7K693W-9TMYKcrgweyzp6Qjhg@mail.gmail.com>
 <163004848514.7591.2757618782251492498@noble.neil.brown.name>
In-Reply-To: <163004848514.7591.2757618782251492498@noble.neil.brown.name>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?FM3fjkdL8rrzHyVhlVg/Rx8Fu/hIWgm7xSfm4h6mLyGL7EVRcjcksTSoc067?=
 =?us-ascii?Q?Gx1bcPWR01cEjTa4/6SaFQ0j7szJSx/46b/uhK415MJ7jdiN3d58phBR/Yb/?=
 =?us-ascii?Q?E2WG/sDvsIQolposnIqhw2OCv8mRS/mEoJqvThQhIm52EPlApR19xZRWUaF8?=
 =?us-ascii?Q?KgMzzmS7sUHhNXSjTasTki9247350u1yYm1Ai7YqlDGPfieCSAUHJHD+40Pm?=
 =?us-ascii?Q?ObFI+nNXwWLL4xECvXo9PGcCUutdER3txTza5iDvJHlDGhb3D7dZbtuc6/qi?=
 =?us-ascii?Q?lB2GPjQUXtK/ctgqoNcZ3Cs4Ilaue158J9ahZTzAX2uagWEcsgMotRu81DuC?=
 =?us-ascii?Q?0rC//65fInkR3wDhZFMqaH99yRP+wPGO0xe9z803FpBmhxbs+JOyzdFiWPgY?=
 =?us-ascii?Q?X8AYtT3Rg3lyCpudlOU47R8HDXG+54kOS+L+VC1+uRu+13GVj/N3+Nhdlu0Q?=
 =?us-ascii?Q?BDNSP7jYWFlOf38HQ6BqNaj0iS794FwdbnkF0KyUniWhCFhIWltkRchg3yyB?=
 =?us-ascii?Q?HDGcvo+3YOeaHTe3buAP9oxfBqAxterbc4pZ8pMuc9RnXCJuXfV4bi5J3aMg?=
 =?us-ascii?Q?EkDGYCmKORbm3ExJsgxwMF+6iCidY+CYAL07y5zehWMVZDuUuP6T0YNMh3u/?=
 =?us-ascii?Q?b2RCItPrECPDdiBJMp4DLpWgdCCqlFEFUfQbgLn/aKxhv2eWCUN0Bsc8FDu0?=
 =?us-ascii?Q?4WS17oHtniQUoj3PuV4fyhv7vf045DjDFR4Jo5lwxUvDss9ylDuRh/+F2wGO?=
 =?us-ascii?Q?IG9VI52ijj1ucL/XqzJIt8BjK5gr4oYtkH8tpLQ4U4NRY3SdX62gVXsLfdG9?=
 =?us-ascii?Q?RxOMqtAbZjCBHCtPvenvjbS8NrSVNCmW1EPYmSVvH4fs89HP0R1rWJDwkL1e?=
 =?us-ascii?Q?DmntxXkZErxWPAgjCsDU+uSJfqbipJXiDK5eyhIrHjyyB3wsq58IWolhn8iB?=
 =?us-ascii?Q?7wIVFONVJqIKvnEVP4m/FjXAqSpssOfb13as2O5P6TbIJBP09d4V5C/XcOPt?=
 =?us-ascii?Q?8gbJi2B/hNhkwYQGwZc4BlgNdfUHnR9354mI+1c/AVpbtlcF9+MyOKYp2W9a?=
 =?us-ascii?Q?Li/iKLXOeWxXZpwIDAja7wBz3Q5xUXxsxrVz7+P3xywdt4ZZSuMf8YbE2cfq?=
 =?us-ascii?Q?MisuiwKXw1iEnI5rXbcUHsXumqVjfyx7vEYIlEf15++5RzcVJSsUyCj4L34u?=
 =?us-ascii?Q?2jk9rO5JMBnWteG91znUKXEqj+ptatc16rp/+hfU3iDQtNYJeDD2uQC7gWWU?=
 =?us-ascii?Q?J1ed7tBJVB3QDcUoOOmrAg6I7Pr8eKTnqg7/BP0bBNXV+NXXgyKYXIQCUtu4?=
 =?us-ascii?Q?CAJ605LnubKhNEEEtxphxXc3?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-ID: <F7553798E87FAF47A097E2515F2168F8@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4688.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2954bfe0-1f08-46a6-7ac3-08d96964e473
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Aug 2021 14:13:49.8233
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: u4EJwR6ucA2lxkSOUYnv87UnJ/gItHQcycbhw1yKMkSDpZCcycx6lVkD0FXAjVvZhMnFrW+maS56LrnnCiK5gQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4464
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10089 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 malwarescore=0
 spamscore=0 adultscore=0 mlxlogscore=999 suspectscore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000
 definitions=main-2108270089
X-Proofpoint-ORIG-GUID: HqXOiqMuvEvEMrt1NZHMesOGKrXstKZc
X-Proofpoint-GUID: HqXOiqMuvEvEMrt1NZHMesOGKrXstKZc
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Aug 27, 2021, at 3:14 AM, NeilBrown <neilb@suse.de> wrote:
>=20
> Subject: [PATCH] SUNRPC: don't pause on incomplete allocation
>=20
> alloc_pages_bulk_array() attempts to allocate at least one page based on
> the provided pages, and then opportunistically allocates more if that
> can be done without dropping the spinlock.
>=20
> So if it returns fewer than requested, that could just mean that it
> needed to drop the lock.  In that case, try again immediately.
>=20
> Only pause for a time if no progress could be made.

The case I was worried about was "no pages available on the
pcplist", in which case, alloc_pages_bulk_array() resorts
to calling __alloc_pages() and returns only one new page.

"No progess" would mean even __alloc_pages() failed.

So this patch would behave essentially like the
pre-alloc_pages_bulk_array() code: call alloc_page() for
each empty struct_page in the array without pausing. That
seems correct to me.


I would add

Fixes: f6e70aab9dfe ("SUNRPC: refresh rq_pages using a bulk page allocator"=
)


> Signed-off-by: NeilBrown <neilb@suse.de>
> ---
> net/sunrpc/svc_xprt.c | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>=20
> diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c
> index d66a8e44a1ae..99268dd95519 100644
> --- a/net/sunrpc/svc_xprt.c
> +++ b/net/sunrpc/svc_xprt.c
> @@ -662,7 +662,7 @@ static int svc_alloc_arg(struct svc_rqst *rqstp)
> {
> 	struct svc_serv *serv =3D rqstp->rq_server;
> 	struct xdr_buf *arg =3D &rqstp->rq_arg;
> -	unsigned long pages, filled;
> +	unsigned long pages, filled, prev;
>=20
> 	pages =3D (serv->sv_max_mesg + 2 * PAGE_SIZE) >> PAGE_SHIFT;
> 	if (pages > RPCSVC_MAXPAGES) {
> @@ -672,11 +672,14 @@ static int svc_alloc_arg(struct svc_rqst *rqstp)
> 		pages =3D RPCSVC_MAXPAGES;
> 	}
>=20
> -	for (;;) {
> +	for (prev =3D 0;; prev =3D filled) {
> 		filled =3D alloc_pages_bulk_array(GFP_KERNEL, pages,
> 						rqstp->rq_pages);
> 		if (filled =3D=3D pages)
> 			break;
> +		if (filled > prev)
> +			/* Made progress, don't sleep yet */
> +			continue;
>=20
> 		set_current_state(TASK_INTERRUPTIBLE);
> 		if (signalled() || kthread_should_stop()) {

--
Chuck Lever