Received-SPF: pass (google.com: domain of linux-nfs+bounces-2002-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Fri, 16 Feb 2024 21:17:07 -0500
From: Chuck Lever <chuck.lever@oracle.com>
To: Dan Shelton <dan.f.shelton@gmail.com>
Cc: linux-nfs@vger.kernel.org
Subject: Re: PATCH [v3 0/2] NFSD: use CB_GETATTR to handle GETATTR conflict
 with write delegation
Message-ID: <ZdAXI1oAJx0atlch@manet.1015granger.net>
References: <1708034722-15329-1-git-send-email-dai.ngo@oracle.com>
 <CAAvCNcDDb4L2tcbBCcufFg=TLeFSrui4MHFuEeNUA+1VZyXLxQ@mail.gmail.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAAvCNcDDb4L2tcbBCcufFg=TLeFSrui4MHFuEeNUA+1VZyXLxQ@mail.gmail.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?MCiv994MdiFzbSA12t21uxclf85WqLskLar71f8qPIFk8DqIQchf/21KepFj?=
 =?us-ascii?Q?xw4VyuLTEne5+iNmhvRaX7ujeB3T6qCdJ2ijPJRzBHEjeoX0pR3rSiVtXFJF?=
 =?us-ascii?Q?/8sU+2/0vao564iKBdUY0E4FJiVLJ1H9DL/npjRANy0gvcj/GQGgtu/+8OZz?=
 =?us-ascii?Q?SYNrZuy+Y0teL+cJy6gerEF8o3PmmjMVFRgzC8b6YvnrM4b91sZ6SfMHnEcZ?=
 =?us-ascii?Q?3fwaQrDl5nvkdhzxJOlyL2M8QcPqBeTK1NrOd1sGQXiwwwJTtzeeVG2LRjAu?=
 =?us-ascii?Q?HPzrW8VO2aEtfYMBzLCCWEf5U2h3S1hYq6J4XHshxAVlN7OyuZcxDTU9Q2ar?=
 =?us-ascii?Q?vfUuLaAfvJMmf8SA9gngD74t1mCPTblxqw2RS3xp01Lru8Xaky1ET1dtMxgF?=
 =?us-ascii?Q?FPgsdOhwif31flTfL1o85ngmq9QgFStUMv+r8Xlub6rgCU/e9SvIUb1mo457?=
 =?us-ascii?Q?DtIe5kSOMs4eEUNvIxwVQiFN3UwZRmU+Tqz8Y9NzBIs8pLcIY0QimqPK6cfM?=
 =?us-ascii?Q?YqeIsC//hOVLeiK9fVR7UH0VCfRX0cQz8ccOunf7oKZtgJJs4ljjcKeqYnF2?=
 =?us-ascii?Q?DN5XJs8za/H7r59KeBKd3gnXUK0FygFjbFFJlORMOMXMW0RAnNyTDgLbljax?=
 =?us-ascii?Q?W3CAgAkp8pZCEoxxvoL3dFr0fZkA8sK7t6xB1y0UxuVhJIq/7Nlp4x26aNSe?=
 =?us-ascii?Q?f2S/GHtngstG6yPcWRjGBe5Hug9ZBA0H5fY0fJ9ZshnlVGTjVYQNAGb1kQMx?=
 =?us-ascii?Q?xK0lOaU67xtPs78n7AWiwiHnk8rYZ3Tq4Mhe8J2FFfOBrlfMBiGPmje14RDz?=
 =?us-ascii?Q?QBlQcCT27HxSWkL17Scd4IxxrLEUSLrCCRlebvi2jIduL3iG98m4ZB2Ef/C6?=
 =?us-ascii?Q?iU3W5CT/wrs696d8DF72ynfLuCH/m2XB9src9pI9lbmMD3T4YQQ+DnfmmGdS?=
 =?us-ascii?Q?v+CM+7N3f/1fBOVnsWV5Z4Iw6pXBDla1JWQ9/XR/MoXx21fADqzDSOJlEnma?=
 =?us-ascii?Q?MWq75L0kSmEDWJILCHzwZLZOKRMZaNji4GLLKm20SZt9exdfmL4NKZJraTPD?=
 =?us-ascii?Q?ijqQnvBqY+Nw/2ZQif4t3bGIV1tBClpybykZf4dl4E+65aE+1gi+6QnuJxxo?=
 =?us-ascii?Q?WcwnQAedcZWLvrDjmSKLd8iTbTwEwD3UtzO458dRiCfT0Xq4weB3tStsZW2Q?=
 =?us-ascii?Q?17uIjKioFMOYL/m0EU7bYvltXz5Vl7oIhdxsibop4vmPNouhwwKHzN+w74G6?=
 =?us-ascii?Q?ys/XyMqXfI8atF9Q4Rpm615DWWV0p8GOvUoQTTu3YwLYBWAPJwckimtutOHO?=
 =?us-ascii?Q?mUZkiaDnyQPyfzNCxeAPiLL/BxcpWd5mPgV/KK0Us/gAZC0HbLmM956Qga6n?=
 =?us-ascii?Q?94FGpxSkRgwvdJrnMiyjxM5aLkWAzEUtRIUBDRhgBvLDwuEUZe7ZWHJ+nDvj?=
 =?us-ascii?Q?nXIQoplg/5eevypFMwLGbaAnk/lNkULKPmxab7ldo66fc9Z2wnwEUil+ZKpV?=
 =?us-ascii?Q?k07LLxoXSay3ItlVTQjfrQVxid19q0Dcqj7dW3uaXzHbDzmwnlPkjQPNqJjf?=
 =?us-ascii?Q?80x3tSjhEpgSSBUvJo+KE76qgLLRVD6x128+ekhTnSeFFtlJlJASE8A8xCYu?=
 =?us-ascii?Q?VA=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	b3CDjSJya0dLstw/fcSQeWNu93pSy5HsIEKuH51RNW7VZmU2dRsGB0+K99lmSffSx7TasghYoWz2Ft3oHdxqbWrz+CMQbCzAXTQ5gniAgwIZd3P6PXGjwag/tidCJg8MUkd44N/7jkJ7VImDsnf0e/x6eoUIo8hNhZTrbCsFJ/XjlbFROJbgxOjsJ5JxE8JgBmWguexuIFY9U6cIDzVpAp3ty8oigxq4UIK7ap6tw3QOzxy7DZ9nJGWXwkq8SLd7GjauB4kM3fdCN2wqHnJNnYUYvrLZ1YgwwedGeBXwSlcok+MLENc4nQanLiFYxxm/CUV6Xr7gAkvrhbXUasuVc1lXs8YfTlVi33zJdphha3jFhMnwUkzLoj2YWTjWNxkTY8Cp9jvDHPJcIGxJj5E44oH6+cOPda2Xtq8XGG6FKYOFXmubh4JCngCW2MMh+f+nTN400XsK/AJVPE5iFIyYYnurwsnZEr3nyXikUOd/fPubfedpqOnsHfFEOeawu5h2Hy+6BIAkbnF759c1XuCSxW5bj0SMs4vxjDB38dJ7uNSk7pM3/WnU2yzqg1InbP6LcX35QmEs2BUWBF2gu2YTZli+JonK9+YU0aQexyfRA34=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6fdfe94a-9e20-431e-36d6-08dc2f5e8fe0
X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Feb 2024 02:17:16.8258
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Fyuyu7JNfFah1JHdZn7SeNX0TFd0WQwttUB1Oiid2glr+hpUeK6v0ETzoR+rh5YDYjSrq/95scukDZbZCUtqXA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR10MB4428
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-02-16_25,2024-02-16_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxscore=0 adultscore=0
 mlxlogscore=999 bulkscore=0 malwarescore=0 phishscore=0 spamscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000
 definitions=main-2402170014
X-Proofpoint-ORIG-GUID: nln_cTVu4UXGc_7vH9vdhFUbvEyfazm9
X-Proofpoint-GUID: nln_cTVu4UXGc_7vH9vdhFUbvEyfazm9

On Sat, Feb 17, 2024 at 12:34:45AM +0100, Dan Shelton wrote:
> On Thu, 15 Feb 2024 at 23:11, Dai Ngo <dai.ngo@oracle.com> wrote:
> >
> > Currently GETATTR conflict with a write delegation is handled by
> > recalling the delegation before replying to the GETATTR.
> >
> > This patch series add supports for CB_GETATTR callback to get the latest
> > change_info and size information of the file from the client that holds
> > the delegation to reply to the GETATTR from the second client.
> >
> > NOTE: this patch series is mostly the same as the previous patches which
> > were backed out when un unrelated problem of NFSD server hang on reboot
> > was reported.
> >
> > The only difference is the wait_on_bit() in nfsd4_deleg_getattr_conflict was
> > replaced with wait_on_bit_timeout() with 30ms timeout to avoid a potential
> > DOS attack by exhausting NFSD kernel threads with GETATTR conflicts.
> 
> I have a concern about this static and very tiny timeout.
> What will happen if the ICMPv6 latency is well over 30ms, like 660ms
> (average 250mbit/s satellite latency)?

CB_GETATTR is an optimization for write delegation. Without
CB_GETATTR, or if the client does not respond within 30ms, the
server recalls the delegation. We expect no impact on clients
that connect on a high bandwidth-latency product link.

To lengthen that timeout would require the implementation of a
mechanism for NFSD to defer requests without tying up an NFSD
thread. So for the moment, the proposed CB_GETATTR implementation
will help fast local clients but should not negatively impact
remote clients, and we cannot in good faith provide a tunable
to extend that timeout beyond a few dozen milliseconds.


> Would that not ruin delegations?

As stated above, it should not impact write delegation, and Dai can
correct me if I'm wrong, but I believe CB_GETATTR is not used if the
server has offered a read delegation. NFSD implemented only read
delegation until very recently.

IIRC, there is instrumentation in v6.6 or v6.7's NFSD to measure
how often a CB_GETATTR might have been beneficial. I can provide
more detail when I'm in front of my desktop computer.


-- 
Chuck Lever