Received-SPF: pass (google.com: domain of linux-nfs+bounces-1982-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Thu, 15 Feb 2024 18:04:12 -0500
From: Chuck Lever <chuck.lever@oracle.com>
To: "Perry, Daniel" <dtperry@amazon.com>
Cc: "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>
Subject: Re: Backporting NFSD filecache improvements to longterm maintenance
 kernel release
Message-ID: <Zc6YbFoLyrHEvHKn@tissot.1015granger.net>
References: <3AFDD198-5391-4EF0-BD54-A0CD0AE77FAC@amazon.com>
 <Zc6RQBTnUblpJcvo@tissot.1015granger.net>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Zc6RQBTnUblpJcvo@tissot.1015granger.net>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?hALpdzJOXRCM9fOxR7IB+7F8q3yMKVkC5LI+5yOOakIdokiS2xBeA9WA3N5O?=
 =?us-ascii?Q?wruC1sYqIQ19WpJEIrrSTa964k0IAQ/M5Zo6v9GjurmsjPSAjoDApkgM9h44?=
 =?us-ascii?Q?uu39Nx3fvH7E43tT/FUhJPpheq6LAwO4wa5ZKVeJg3C+4wlMdHoPDW/0DZ4K?=
 =?us-ascii?Q?sQlAhWqTdemqgighLlBK5FvWduExdp14m/0urZ574eY3zToIVU6HbFktQjSx?=
 =?us-ascii?Q?KQOGrsqUrzAN3fPl7tBzmG16k9EITv3hoPvLK2Zfday0OD/+YTkvadO2lGUP?=
 =?us-ascii?Q?I9DUvUuPISyZfQQQD/JI+d7cFW6MAnhozSLC+5+HhPQ4uxTbnu9Abov/d5OM?=
 =?us-ascii?Q?YjVa0StQ1H1Linal4R9VrXutOtr+IFga5Dodn1viToc7L+EwA7IimHD6wRIl?=
 =?us-ascii?Q?hQOjZjD78htSlMFqL/hTUDzaJbJJqw1Wt2m/N2044g/5l2yXbwcRCT13XhPY?=
 =?us-ascii?Q?Rg7tqp2dDvOs3VCDwqC2z6nhM4Sc3j3+ZzfZD18tBxazOeANmqoeIHsddWOj?=
 =?us-ascii?Q?I6+/nvQGM1Y0YHUrf1zC5nNFET4LalIXRafWhlkhrTf+fwmE04SgjAJdnHGX?=
 =?us-ascii?Q?pYqIkVZVgFWynY2lIkjLbcGw1TU4VbTSE6uCHi00ZNHXQ+Z0OFpsh4nu0sVM?=
 =?us-ascii?Q?Z8y5NZy6tmz/iY0apBZ4a5Yjd4FMZ9C3kv9HYDei9qNJFQ9qax28Dawm3D/V?=
 =?us-ascii?Q?pkYCpkAdkuwR8RPa0wu6fhDN6z57VGGlkBLu9SYQMlGuxlniQI1GKxn5MP/j?=
 =?us-ascii?Q?jKSlQiALAsF3cTqM1L0hMynMKt7Q0N+na9sfYfT5yvxznmUw/0PdtXyL0Gwc?=
 =?us-ascii?Q?vHeMxs8M7gx27ns3BNuWHc2F5a3ODTQfuvzcyz1e0uXEbnhJ+lF19IKXyRBg?=
 =?us-ascii?Q?j6HBW5gllIei+EZ7j2un5Iz43gqGGW2Cym7ZYiT4UhwABy63LE+YVyUml23E?=
 =?us-ascii?Q?Ehq9/JbzCyFqSoGR/PTJwtLtDd119Ecxu6qoDY1mv6w2YEY+SUl2v7Dv2v10?=
 =?us-ascii?Q?Qgg3hOtgZSLceW53FuQqGVgTGKjcnGxuRh4U9tFczHKy8hwL+Yhpiax1LHzS?=
 =?us-ascii?Q?kDIrd9Hs1auJxPd6MrrOnSVkeCrTmDgkvJE9JCNbTorOoFX5McXtefSZY2lU?=
 =?us-ascii?Q?VTnomxjp+CW9y2p5FKWbwQ6+QNrKNZy1fqhvp0IlF1iasT2DPKJhHs1x/TvF?=
 =?us-ascii?Q?uYvtODc34p+z4gUAG+pHuStl4N1Dao6AMius/sLTBko1h3hvgyx9DKy7R23i?=
 =?us-ascii?Q?F1WVDkm+AHYv3tZa8XvlDZAuFJyvpa1OLklywqfxchkGYEq/6h67SNFO/ASc?=
 =?us-ascii?Q?VMQJ3bqdguGuq35eGlrIPdzvVE1JMGfpDJw3W/le4hOfafNUoGsyQzhjiZNt?=
 =?us-ascii?Q?QArdbCByl8Hkxk9nBpPJOrjZ90p2dZMgrP6w3BU4UhmRwgl8cxcrLIWvZclC?=
 =?us-ascii?Q?F0yC347jklZcuz1++YrvuUSpk/72k4QfxThXsQ74ZpArKuywXocPm78mMBet?=
 =?us-ascii?Q?FFqCHWAGvT7CFK4ie0M1oYYs+Sh1KYC6yu3KDjTjXQO8LY1OrW8eKe+moNbP?=
 =?us-ascii?Q?TS8fwwsIGNvhc1fmyvYGFCk496bBOp915m+DhP0sDcdxmYVu20IE5Iob9sWQ?=
 =?us-ascii?Q?pA=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	4BdbELm22fOvDPZzz7j3F2qEurUMkHtA8ypdaxbZSu2ZFchSdSyBa1A9rfZzgPJZY03/QqL57Jh5xbWHr5xAgskjU6oJBQRadBYBErewqIpqdDrwhjvpkdX7M410UV8k4OXvL01Tfg1X6o6hudhnNSGlWDdQlbg88PEvmkZmEHOgYRRtYY40ulfFIqBrbrE5Xg2NwOUNVdgSidWWbjGoFdavoXBX27pMuyyXaJD7x9KMeWZDDu04yKcCUd4VlOS1KCw2ZAbF6S26+F2LAJ8nuf+2xAlOB2bnB+kN84PrrfYzr8GqWdjEzZS34ceAOHPLDnT8xee7uyQ6XwbY+7oTZTXgulNAej8dOKZcKPNLA99s2NhF0GSsICKuqJNahJjY+avaoG+4dCXdZRViFEaFgII8MIEJ/MEMrGUKPJ50mcWsnInuD864yKTodWI01synOs5KCz4+Wa6KAiU5ICiKTyHog7Ppb/1DX4Lo7PY4ideD1w1Qh/G7rQ3dEygMxtHb5DGgxeqn5w0ocgVb1D+VzPRU5ne0ykNzConNHr52bOTyyDv/bRgNbWCg63J0wK6ssDXGQAWoKp4+QCqsZWzhsUAXcbfWUCdGNFC2cknJQk4=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4183e70d-8bef-492d-3977-08dc2e7a6e67
X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2024 23:04:15.4001
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: m8sILGlCqb5cKpvznVMJms2VUlRf30K9eKv9/3wM5Q16UKg3b+K3WECElt/LvwKy6HEhwNi3o0KfRAlwx11weA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4206
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-15_22,2024-02-14_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0
 mlxlogscore=999 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2311290000 definitions=main-2402150180
X-Proofpoint-GUID: UrujIZTK1CdqPw6vE_OUFjmhlqPBikXx
X-Proofpoint-ORIG-GUID: UrujIZTK1CdqPw6vE_OUFjmhlqPBikXx

On Thu, Feb 15, 2024 at 05:33:36PM -0500, Chuck Lever wrote:
> On Thu, Feb 15, 2024 at 08:40:06PM +0000, Perry, Daniel wrote:
> > Before we spend more time investigating, I first thought I'd ask if the maintainers would be open to reviewing a set of patches that backport the NFSD filecache improvements to LTS kernel 5.10. From my perspective, these patches are core to nfsd being performant and stable with nfsv4. The changes included in the original patch series are large, but from what I can tell have been relatively bug free since being introduced to the mainline.
> > 
> > I believe we would not be the only ones who would benefit if these changes were backported to a 5.x LTS kernel. It appears others have attempted to backport some of these changes to their own 5.x kernels (see https://marc.info/?l=linux-kernel&m=167286008910652&w=2, https://marc.info/?l=linux-nfs&m=169269659416487&w=2). Both of these submissions indicate that they encountered some issues after they backported, the latter of which mentioned a later patch resolved (https://marc.info/?l=linux-nfs&m=167293078213110&w=2). However, I'm unsure if this later patch is needed since LTS kernel 6.1 is still without this commit. The above two examples provide some hesitation on our side for backporting these changes without some assistance/guidance. 
> 
> We (Oracle) have been discussing this internally as well.
> 
> I'm not a big fan of backporting large patch series. Generally, if a
> stable kernel is not working for you, the best course of action is
> for you to upgrade. But I know this is not always feasible.
> 
> In this case Jeff and I never found an adequate reproducer, so we
> can't nail down exactly where in the series the problem was finally
> resolved. And I think the community would be better off if we had an
> upstream-tested backport rather than every distribution rolling
> their own.
> 
> Further, the upstream community now has more standardized CI that
> works for not just the upstream kernel but also the 5.x stable
> kernels as well.
> 
> And, I now have some branches in my kernel.org repo where we can
> collect patches specific to each stable kernel, to organize the
> testing and review process before we send pull requests to Greg
> and Sasha.
> 
> (Perhaps) the bad news is I would like to see the performance and
> stability issues addressed for all stable kernels between 5.4,
> where the filecache was introduced, and 6.1, the kernel release
> just before things stabilized again. Maybe 5.4 is not practical?
> But I think fixing only 5.10.y is not good enough.
> 
> As long as the community, and especially the author of these
> patches, is involved I think we can make this happen. Can we start
> with v6.1.y, which should be simpler? Do you have testing or CI in
> place to tell when nfsd is working satisfactorily?

Looking at 6.1.y's filecache.c, it has already got most or all the
fixes it should have. The only thing we might want to do there is
confirm that with some testing.


-- 
Chuck Lever