Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Neil Brown <neilb@suse.de>, Dai Ngo <dai.ngo@oracle.com>
CC:     Jeff Layton <jlayton@kernel.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        Olga Kornievskaia <kolga@netapp.com>,
        Tom Talpey <tom@talpey.com>
Subject: Re: [PATCH 6/6] nfsd: allow delegation state ids to be revoked and
 then freed
Thread-Topic: [PATCH 6/6] nfsd: allow delegation state ids to be revoked and
 then freed
Thread-Index: AQHaDF75bEKrShy72kWA52GSBv2SfLBkuEKAgAAOcICAACzEgIAAIeyAgACFroA=
Date:   Wed, 1 Nov 2023 15:41:49 +0000
Message-ID: <E4C94AEE-6CBA-44C4-AA45-C56E8458286E@oracle.com>
References: <20231101010049.27315-1-neilb@suse.de>
 <20231101010049.27315-7-neilb@suse.de>
 <4C3DBAFF-4C83-4DB4-A6C6-D9C4387BF1F4@oracle.com>
 <169880769331.24305.7672914147957308642@noble.neil.brown.name>
 <D2B988F4-D8F5-4A5A-BC97-F67D19A76C78@oracle.com>
 <169882459188.24305.13216722681220510683@noble.neil.brown.name>
In-Reply-To: <169882459188.24305.13216722681220510683@noble.neil.brown.name>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?8W5f1d7ypyzFqCvqh6FOENzahMHTNptuE5EPwFByPtALC/0hPcI47JukPvHv?=
 =?us-ascii?Q?5rIUdTm47B/ARraQCD8B0RcA18T/vgoo7f+aY1DLKSsopaXhtEzXvM1EzAbs?=
 =?us-ascii?Q?KMom6YAAbt9K8HBvuwpvILsBl0HUf9BX4EFf0vnD3bNnbXGHUqa5BP81TQAV?=
 =?us-ascii?Q?CgHRT+4v02u/e1+Butl5KkRDabfJpW0cY5c2fptpwXq7DuNBCL/SzBTlKksi?=
 =?us-ascii?Q?mUhS1tOPdzDZx0+VYRP0o0pDfnw2qmK1TuPN/dDjGRAAVmbkrUCbOMZzhPWW?=
 =?us-ascii?Q?Ao5km+MFcUGOASQbbNSxs9PjavpI4BCTqMxYdK1LTMkM7CAIPB6tpX89Exmf?=
 =?us-ascii?Q?n09MTidNkfYdRnr6vR89EHTQ+pWIDpK9PpwnRJxIGWsI/JudrpQhKakLLkL6?=
 =?us-ascii?Q?U4qhGYO+RhyWt+RwxOAYKSY/nuL4bNYHxYC0/idqJ2i1DhOaJ8LKPFga8kqg?=
 =?us-ascii?Q?4vh4G/1GbDfkaLT6NX4P8RrFa8zNahUnqib26JoxHCK4NQKDjeUUg5pQRsiI?=
 =?us-ascii?Q?tKLN1a/HQmumjnxmh7RrpCu4AstfMW7YfQHUqJhwpEiwU/kum/tZwlDQDwNc?=
 =?us-ascii?Q?2mD5IWoh9Jy4y2yssjAqBZTMe/bIEpI0BezAxm21DsKubj37fEe4eTmEDnUB?=
 =?us-ascii?Q?DWBNO1m4dpI4Qt20OJP9xnUgzdzRwcOtsCp1BAEvNSXz+Pm4oO/a2jrLkClF?=
 =?us-ascii?Q?A+/SX+UzdfWq0krnx9Xih2Z3d2Ollg3ZYfKkADguAE4Linvirf334yC1e/XQ?=
 =?us-ascii?Q?9GxEh4xHTneJbAAxRc8HVLbQHTQJlQzvYa3N3EEg0EOai9HHRamALtFg4kdm?=
 =?us-ascii?Q?oC2Fk522zKwEUt2g7zPtlcy5ReuPLGoTZJ1vzTw5Zypjj6mubs5a8yiFDOUj?=
 =?us-ascii?Q?VnZBeh1LcMZAVG+nuhPGCOrLKOdbOTcNxTpTc6zF8vx0Pm1NA+egU8xf+73O?=
 =?us-ascii?Q?BKMtGkICAfYGlQKyY4DC6FB3vPCowIaac/gkJ/FvrJ7zVUg697ebao9EQjyx?=
 =?us-ascii?Q?qnW71X4IqYXHN8lMMsa5NHuL5gEpU1wCF+WTgu4NS06MaNWx7PMsuzkojIkp?=
 =?us-ascii?Q?S5hJxbb+5M+RuBZOvgeFFEMW0IuGAGrgQAFl7VRy6tt4C9SK2+zeoPXXHnmv?=
 =?us-ascii?Q?D5AV3PQkpea95l+Ho5PKDC/0ERSomM17V76xZ3JWXysK6gdaLWMhjblNIucL?=
 =?us-ascii?Q?rmGVejN+o2QOVzLlvRanPXY614CCyVWe5awOlQbaM8TSgMkKJads6/EuBLEI?=
 =?us-ascii?Q?guLE5p0rs3fWn2Jh9o9XbB34Ml8nftXsEegMz0Ynds2Pi53nwBQUNPq7UZsD?=
 =?us-ascii?Q?dfTnJHmztMorvUL2fE60A/wxfOLceC5hrJI+zNTdzvklRBATb5Gc8L1Hv2gI?=
 =?us-ascii?Q?9tL5D/YBhZcVa1JW9eyAQBJnRwtNW6LEtXszlADccnzr8gtzGrklVmzz5Vsg?=
 =?us-ascii?Q?nSvZsRMk4CYAYaGNfDjIKeo8vigWxyUIeQb/0+gUZPOH1F1m4a4P577kn5dA?=
 =?us-ascii?Q?KJjCkIrrBZmP0usKok5R26RYtc5o+emdvQt2CmpYAOlNTNSlRak7/3SR6oyO?=
 =?us-ascii?Q?Z5WpC6bctbC6Rcr2Of283jR68bYe6hDEpoe3BEJQsFrKSAKxEsgsb7+0nkSz?=
 =?us-ascii?Q?Wg=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <B1DBA3101262D24797F796E21924BDDC@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: UTqxdIOtLG1ykyY7iDI5DKTBaI77BKXO2a/tgn6M5l97vkXwYMe6aH2qNQaxDNvF/yfCkEQRMxkWj92zsEPoTh4B6gCyU48t2v1vqEgt/KBGwqTmrBuyxPHqI5xNKQJyhdeFHowR+C2Nh+oigsxRX/TZh269zLqdbpQmSKp/SUY5bhZHCCInsh77WG23drPWRZIeZXR9ym8/NESd4d+rq8n7iImyXyS/6D3p26d8i32Js7+vYmUfs1jIDFvjJZ1OsmLyh8ZAQxDm5JN7kwA2z6ZNpA/Dmi+KkwUGHSiW/uo7R3gaQYgOM7gdEe9cXp7zuo5oH7iTR4znLdMYri4551Fq2aT7XHMvd50vas2NYNFbXLoMYvqeKMI0XxXu/i94+0Ih25mK/ICMYQcGY1+kfDXzH85TFr8pPJbtbJbV8cRc+EecKft/kDfAoj//fa/w5Hc2KCQv5RxWUl08aths876BV6zpuxQ46ea9+RloEp+hR2YEpxPnB/nVaXadtv0aD1AbNHee5ZtbUDod1a8udR7Hoq0zmoFYaz7VWmUawo4VgcxCr5jpZGjI5ZKz9MN2JZ0Gr4vpD9RufN/qqfRo8OQllU1GJu/73g97Q87FdXJ8EDvEoZON5ck3bIv8dGxEjy4s1EidrtaEbRF3pN1sOTy5TVuwxnh9sgdSk+AliHULmRaacWt0+zX4pYi1kvqgK7GnqFKjJfKHAcOxKYI3AD2Wo3Va5Xg36QHaEoXa7wD7fTufpgbRribRlaMVzeS4eE90WEYWEISxO3qa36S38Pjmj7AqsQ8rJHQvKWTsFb3ePu+/HAUJbemGjIHSJuGdvszOQb2026qUfyGMWUNoPuzmOzASb59/uU5BjUPmgXbW76vkDubKd28I42DVsn7Q
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e7afc380-6f84-4e4b-be9e-08dbdaf1105c
X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Nov 2023 15:41:49.9417
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Wgji7TN2iqMfmPZ+okiepAUb9/rzgDzHPGek8GqbB2kpFm5sG2zeR3MX6JnEN6f6l1oPeHw/2dlcO7S8dfK2QA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR10MB7079
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.987,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2023-11-01_13,2023-11-01_02,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0 mlxscore=0
 spamscore=0 phishscore=0 adultscore=0 mlxlogscore=999 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2310240000
 definitions=main-2311010128
X-Proofpoint-ORIG-GUID: ntMj8Kl5_fU4ubXNcabcUStCaBPV9bYZ
X-Proofpoint-GUID: ntMj8Kl5_fU4ubXNcabcUStCaBPV9bYZ
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Wed, 01 Nov 2023 08:42:11 -0700 (PDT)


> On Nov 1, 2023, at 12:43 AM, NeilBrown <neilb@suse.de> wrote:
>=20
> On Wed, 01 Nov 2023, Chuck Lever III wrote:
>>=20
>>> On Oct 31, 2023, at 8:01 PM, NeilBrown <neilb@suse.de> wrote:
>>>=20
>>> On Wed, 01 Nov 2023, Chuck Lever III wrote:
>>>> Howdy Neil-
>>>>=20
>>>>> On Oct 31, 2023, at 5:57 PM, NeilBrown <neilb@suse.de> wrote:
>>>>>=20
>>>>> Revoking state through 'unlock_filesystem' now revokes any delegation
>>>>> states found.  When the stateids are then freed by the client, the
>>>>> revoked stateids will be cleaned up correctly.
>>>>=20
>>>> Here's my derpy question of the day.
>>>>=20
>>>> "When the stateids are then freed by the client" seems to be
>>>> a repeating trope, and it concerns me a bit (probably because
>>>> I haven't yet learned how this mechanism /currently/ works)...
>>>>=20
>>>> In the case when the client has actually vanished (eg, was
>>>> destroyed by an orchestrator), it's not going to be around
>>>> to actively free revoked state. Doesn't that situation result
>>>> in pinned state on the server? I would expect that's a primary
>>>> use case for "unlock_filesystem."
>>>=20
>>> If a client is de-orchestrated then it will stop renewing its lease, an=
d
>>> regular cleanup of expired state will kick in after one lease period.
>>=20
>> Thanks for educating me.
>>=20
>> Such state actually stays around for much longer now as
>> expired but renewable state. Does unlock_filesystem need
>> to purge courtesy state too, to make the target filesystem
>> unexportable and unmountable?
>=20
> I don't think there is any special case there that we need to deal with.
> I haven't explored in detail but I think "courtesy" state is managed at
> the client level.  Some clients still have valid leases, others are
> being maintained only as a courtesy.  At the individual state level
> there is no difference.  The "unlock_filesystem" code examines all
> states for all client and selects those for the target filesystem and
> revokes those.

Dai can correct me if I've misremembered, but NFSD's courteous
server does not currently implement partial loss of state. If
any of a client's state is lost while it is disconnected, the
server discards its entire lease.

Thus if an admin unlocks a filesystem that a disconnected client
has some open files on, that client's entire lease should be
purged.


>>> So for NFSv4 we don't need to worry about disappearing clients.
>>> For NFSv3 (or more specifically for NLM) we did and locks could hang
>>> around indefinitely if the client died.
>>> For that reason we have /proc/fs/nfsd/unlock_ip which discards all NFSv=
3
>>> lock state for a given client.  Extending that to NFSv4 is not needed
>>> because of leases, and not meaningful because of trunking - a client
>>> might have several IP's.
>>>=20
>>> unlock_filesystem is for when the client is still active and we want to
>>> let it (them) continue accessing some filesystems, but not all.
>>>=20
>>> NeilBrown
>>>=20
>>>=20
>>>>=20
>>>> Maybe I've misunderstood something fundamental.
>>>>=20
>>>>=20
>>>> --
>>>> Chuck Lever
>>=20
>>=20
>> --
>> Chuck Lever


--
Chuck Lever