Received-SPF: pass (google.com: domain of linux-nfs+bounces-1978-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Thu, 15 Feb 2024 17:33:36 -0500
From: Chuck Lever <chuck.lever@oracle.com>
To: "Perry, Daniel" <dtperry@amazon.com>
Cc: "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
Subject: Re: Backporting NFSD filecache improvements to longterm maintenance
 kernel release
Message-ID: <Zc6RQBTnUblpJcvo@tissot.1015granger.net>
References: <3AFDD198-5391-4EF0-BD54-A0CD0AE77FAC@amazon.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <3AFDD198-5391-4EF0-BD54-A0CD0AE77FAC@amazon.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?6ysQHqHVsO9hUpbcoUlTd6BR2WTzUihRPkggoxJJ8rNneycAolR84ljbwdd+?=
 =?us-ascii?Q?ah9JiYfAIJohE0S3fyOmwPdHqb+lhrY5pGPvwbKJpIlt8GGuXIETbpIVdRZO?=
 =?us-ascii?Q?d+bXOdUW/xAMokclZG8CMgeHh1mII31MhLeZ66sHT2ddvo2Mv9oiookKyxsR?=
 =?us-ascii?Q?TyY4bgdFZSeivhUc2hS2pYOvMV/mL7Vs3YimLK92mVQyovlmyvt/fkrW5uTc?=
 =?us-ascii?Q?lF7qdJuiHE4KyE8EacpSRgmYNv/DcBPir9ouaAbAtkwg0xleK2fPA94KROzs?=
 =?us-ascii?Q?O6HiNXxEsEa/i+kTqTbuPkM+cqNszy1Ujts6YfxGxBjWuYUubadMKMVlCXB2?=
 =?us-ascii?Q?adHtGjYE1PYLXdqI4ULJ5JDp/LzWceu8J79rKhX4UB6wCOIOrr6mslxQIFTi?=
 =?us-ascii?Q?EI3YXmrV4fY17humxusFdUcSVafFh9c7CKjlHBpSu/VxJ7j5jvIeNunYL4Kz?=
 =?us-ascii?Q?lQrbkSFZk3VAtrsWmo11FqF34bUIQ3VstcU5/KB2D5KFXkVXgR/FgiZFhmyh?=
 =?us-ascii?Q?fXhfIjjI9SucFmeBsxySu3TMxTBa8/3HRRnh4mwuOw27Dfdq4lk0N8JURw69?=
 =?us-ascii?Q?A7twbNADuQvoS1FwSK3CvSXflBQijYLuDo/1E4GbVxB4rTDtkOFHiNlVW+zK?=
 =?us-ascii?Q?rL+wIB34FYkYhAMZT1GJtSvlxfZ8lCNk2jBgc4okEMK5wRT/IOysW3oo+gWA?=
 =?us-ascii?Q?ZNz4tKmeuYZ+HLzb9xTkCjKcEpXTqyhLrckuyinL6ZZ3peuXKBLu49e49j1d?=
 =?us-ascii?Q?cK4H7cawqqqw99eETQ7qsR118Dw8LzAfCf5X5cUkg5QJ8esG7f20qEw0Dmgt?=
 =?us-ascii?Q?Z2oRmtGMJ+XlJwSQHwQ+hG/mrPLgsGe/o4XmckKTiPrsNGx4t2DE5hq8rXKV?=
 =?us-ascii?Q?tuvQfOQ5tcmAUL8P2sB4oU8sgkrTRJK83sDzljVz3escvROI1Gl77vmNDPzA?=
 =?us-ascii?Q?gvPohDhgBEyBluuAHxhBeLjpi5D08CBvx2HJLKst0A9DXF6E/dpTQs7abHqF?=
 =?us-ascii?Q?8tlr8+plSWNZsAbZxjM8s7KvKbeLicGs0aqaZJZxLhbPkVh+ibhVi2J7Avp2?=
 =?us-ascii?Q?/lFqj3CKc2uoeWGyGXaHVUvKMtmOjNKBKwi2g9JcTkBvIPS5F7u5JpKu2YkI?=
 =?us-ascii?Q?2GR8GSyXZk3srljbaTiZce7I0rh0W+GGvKtQeM0RxVh9uZdzj6ZkxKkmEKQT?=
 =?us-ascii?Q?AYJtQZskbp8hC0He7ImbfN8+EUstBT7r0fZiH5+n7DT6bn1Kgf8t8hjyjO+p?=
 =?us-ascii?Q?LXZPzyyn4fcA+XKu5RC9tKEJhM9V0+UmIWtPlRF0huwk5MdMUw9+Hr7LOnlS?=
 =?us-ascii?Q?POsXiTxwQ10HAMloSHR+d/mNn17u8XsbLK4uWRg2N5SuMf8eKeKglqvf3aNH?=
 =?us-ascii?Q?IRV4Ie1rCDatzdL5urOyiJifhO7GnqNkx14LhPBJEchEPHxYsiDb549pLaZL?=
 =?us-ascii?Q?sNSGa+GcFzEuSvvq1V/bHIW3gY6gBrbx4F+3kt9zdSgJTa4TjQr6+w1dZLLE?=
 =?us-ascii?Q?8JeCMgk4eKh7B2ycsbT8XsumXW0NUfQbtE7y1CNUMXA2g2+8AAH6B+9r80RC?=
 =?us-ascii?Q?/HJfHBzHoIrfslI/Oas8CbmGrIFD1TcxO0BULifhYThv17U3fza14ydYAmQx?=
 =?us-ascii?Q?UA=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	UrHlBKNlF6zn1+lxLYKug15WlW44Olid8pHfMRGlJ6ypSkKpU9rYbjQbLQycwGHaxDCs3QsEn/cBUE8eGdV4xotTIcFt+DpB5JZX2qAJofVfnd4HiIOi3V4STWieJtuAqD/C7nD+MRVaW/DfmDZzI31aUgG54rbxzi4aSJRvMbMdAJxoXxpt+jUanVi9ZIpdgP7jg+HHsHL2ZENTZG0OTSgPp2c9qcs8WbgxRuPvirwA14+XjG9eM47I8xrYWSi7YziJaJbEBT7irysAqabKE/L5Mqu6cgYo/npe/aPoeBKzZ8w/AAhA1fnstggAtnZkMGp0UBiEkKtMXXiGFS5D7Chw/4iiRPgvRVtu7YUfy4W7NFAhXP7euAmJ7XwtZKy1qx5Xg9j0DOLP/rZK9bZL6dnknt0QUFsZa7mkts9e2BEreDfjm/0rI9jX3CZX1O5JF6/IVxYNVnJnBSqgPymDn45gFPNRWK7L8vl/TvjYK0t5eF9KGED1NG+Gl3MJlY9tCkgOgMp+t8RXUWZicQUFtzW+LZyY8/uWVFq2UiPBacojaLVjbg7cm06V24CQIzRKl2XVjW5H9U9TQp+QYtinXjPvrUcs8Z6aTZO34O/DHzE=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 51e4f9a7-f80e-4cad-2421-08dc2e762807
X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2024 22:33:39.3985
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: WOeX1K1jMbFj53SxkZbI7rx7EARIaYipdYfGRS6EYSgBPoC96BfCFg1/NfMUVF0eRtFErGMLskvV4E1Bkq8Siw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR10MB6258
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-15_21,2024-02-14_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0
 mlxlogscore=999 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2311290000 definitions=main-2402150177
X-Proofpoint-ORIG-GUID: 93UoqOWAbaSYN3hevLanUOZSiZRUjCAo
X-Proofpoint-GUID: 93UoqOWAbaSYN3hevLanUOZSiZRUjCAo

On Thu, Feb 15, 2024 at 08:40:06PM +0000, Perry, Daniel wrote:
> Before we spend more time investigating, I first thought I'd ask if the maintainers would be open to reviewing a set of patches that backport the NFSD filecache improvements to LTS kernel 5.10. From my perspective, these patches are core to nfsd being performant and stable with nfsv4. The changes included in the original patch series are large, but from what I can tell have been relatively bug free since being introduced to the mainline.
> 
> I believe we would not be the only ones who would benefit if these changes were backported to a 5.x LTS kernel. It appears others have attempted to backport some of these changes to their own 5.x kernels (see https://marc.info/?l=linux-kernel&m=167286008910652&w=2, https://marc.info/?l=linux-nfs&m=169269659416487&w=2). Both of these submissions indicate that they encountered some issues after they backported, the latter of which mentioned a later patch resolved (https://marc.info/?l=linux-nfs&m=167293078213110&w=2). However, I'm unsure if this later patch is needed since LTS kernel 6.1 is still without this commit. The above two examples provide some hesitation on our side for backporting these changes without some assistance/guidance. 

We (Oracle) have been discussing this internally as well.

I'm not a big fan of backporting large patch series. Generally, if a
stable kernel is not working for you, the best course of action is
for you to upgrade. But I know this is not always feasible.

In this case Jeff and I never found an adequate reproducer, so we
can't nail down exactly where in the series the problem was finally
resolved. And I think the community would be better off if we had an
upstream-tested backport rather than every distribution rolling
their own.

Further, the upstream community now has more standardized CI that
works for not just the upstream kernel but also the 5.x stable
kernels as well.

And, I now have some branches in my kernel.org repo where we can
collect patches specific to each stable kernel, to organize the
testing and review process before we send pull requests to Greg
and Sasha.

(Perhaps) the bad news is I would like to see the performance and
stability issues addressed for all stable kernels between 5.4,
where the filecache was introduced, and 6.1, the kernel release
just before things stabilized again. Maybe 5.4 is not practical?
But I think fixing only 5.10.y is not good enough.

As long as the community, and especially the author of these
patches, is involved I think we can make this happen. Can we start
with v6.1.y, which should be simpler? Do you have testing or CI in
place to tell when nfsd is working satisfactorily?


-- 
Chuck Lever