Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Neil Brown <neilb@suse.de>
CC:     Benjamin Coddington <bcodding@redhat.com>,
        Steve Dickson <SteveD@RedHat.com>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH] nfs.man: document requirements for NFS mounts in a
 container
Thread-Topic: [PATCH] nfs.man: document requirements for NFS mounts in a
 container
Thread-Index: AQHYLR6czeCutPWqYkWZgIa/iwKGvKyqohqA
Date:   Tue, 1 Mar 2022 15:08:13 +0000
Message-ID: <F285A122-30EC-4353-AF10-FBF6999B7F25@oracle.com>
References: <cover.1644515977.git.bcodding@redhat.com>
 <7642FA55-F3F2-4813-86E2-1B65185E6B36@oracle.com>
 <3d2992df-7ef7-50ba-4f11-f4de588620d2@redhat.com>
 <DDB59BD9-8C29-45C3-ABAF-B25EDDB63E09@oracle.com>
 <D0908E76-C163-4DBF-A93C-665492EB9DB2@redhat.com>
 <E2C56D5B-AC77-48D1-9AF6-268406648657@oracle.com>
 <4657F9AE-3B9E-4992-9334-3FF1CF18EF31@redhat.com>
 <C7533D80-25B3-4722-94A9-0440C48B8574@oracle.com>
 <945849B4-BE30-434C-88E9-8E901AAFA638@redhat.com>
 <06B01290-E375-455E-A6D7-419CA653A0D1@oracle.com>
 <948D8123-E310-4A35-BF04-C030F20EA83C@redhat.com>
 <164479707170.27779.15384523062754338136@noble.neil.brown.name>
 <863AB69A-D5D6-4F22-950C-E5F468CD4552@redhat.com>
 <42AAFEDD-F4EE-4A91-BD23-E08B1149EF1C@oracle.com>
 <3AF29DC6-2EEB-4C3E-BD6C-BE31910921AE@redhat.com>
 <9FC005FB-370E-4AFA-AD80-8599CBFCC1E0@oracle.com>
 <2965D098-7AEE-419D-BF8B-4D7AF4AB40FB@redhat.com>
 <164505339057.10228.4638327664904213534@noble.neil.brown.name>
 <164610623626.24921.6124450559951707560@noble.neil.brown.name>
In-Reply-To: <164610623626.24921.6124450559951707560@noble.neil.brown.name>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?3U5BKwxG5vkv0AW3AuPEOK3uaj0nvlRYu5h7m6ZwvyBSXn6glpCfnlFXc+Nh?=
 =?us-ascii?Q?EKsKS6FF7CmBYbyGyLyHUnsUuWjg2EDODcywx0pLjK67TZChBBtzKc0jiTuw?=
 =?us-ascii?Q?enNjd4PtDrc/TkF58I1qO/jkGQTDFGhu1g4rr470D1iAnAJwHvQcQZfvBlGv?=
 =?us-ascii?Q?qejgm4x2v2dq5V45K5FVCXfxyG4M1abeFAm8Rry3NSWppuiKlzXL4phGvMjW?=
 =?us-ascii?Q?kQTFHAfZimrFtrRm7K+ZiOieWjiLYbTNgEJw27EkDZZqt6PYPD08IWWXqYu9?=
 =?us-ascii?Q?L/dvmfV4tTD2B4TaYQo0eXaxcA0v+vC4YtnUlpUj8lOmffkmNvI98cLzsk4o?=
 =?us-ascii?Q?vkakwkjtCef0LRIImh4C8wI5b90pUcpe99zVvv8eMNJlWWkyd/PlEQPpzJdQ?=
 =?us-ascii?Q?RJKkCUTVwk5on4wBjDFvDw/XyiO3UCAGL0dGyL7u0k9UsMsYIaOEdyJqZg9/?=
 =?us-ascii?Q?j3AdYT6wEGDvPC0lNSRkszW2TXLSuE/omed8i/rONy6FuPhp4ivG4ZqmpZJI?=
 =?us-ascii?Q?JM9puAYK3Ks29inZGJ6xRgXlsILF0g+2pnliB7SF+9aEREP3MEyDeAmXRnQQ?=
 =?us-ascii?Q?K5KDFFsC/2tZUxHQ2sEWOEQmQZAucY4RpWumCB6c8M9A5F5HJKDWUCvZzDr2?=
 =?us-ascii?Q?SJ842GyuEgK8PK2eYpHMpLZyYd+Gih/BSGueQTTp/oyoRg0ESedNQkAujyBY?=
 =?us-ascii?Q?1/dSPMoZZAoNlEQvqHyC4tWtxmHyVResP69atDMng0QJ0UsL5rV3Eni7B7zS?=
 =?us-ascii?Q?X9Y+eX0WLAq38k3u9ZPWWG2gLTGz1jioCXdRS7MEtqZPTZW7Xq+tOuEPNbmC?=
 =?us-ascii?Q?cZXxvgFn37klOVWYltIlw0LXIrXHxw8SjRFseEqEqKTBmsOtNHvH8G6qx2N4?=
 =?us-ascii?Q?8bzmxNrAamp2+4H3UAxWiVRcu3+OPMquHOpZf0Uk5fcr9wGJlYbo1ZKocXQc?=
 =?us-ascii?Q?tIEWeD80XxzIPZmcR/GE7MUEtbY9XMsCZqtiOXl8nE2bzjT21tnKF92fRcJa?=
 =?us-ascii?Q?aaAKR50021dgcJdEkCo7XRMJRv6S+54TJ96aWEKG9rLKt8CpQo/HaNkgnnQf?=
 =?us-ascii?Q?Dfuyc80jTddgkiOGfu+NMo8+QEYeSjYepAgpcRy2/0ejPlcaVXkaMMPenYx/?=
 =?us-ascii?Q?wPLTkWpSKFl2lrZpAMbA0uGpQ58FvWielt6iWIoV18nsw7z5BgC/xqbwhltN?=
 =?us-ascii?Q?YD296vaB7dYtJxTjXuFWCRdSP09RzPVqo5jpXwyExqYE+3QDUDBOp8DyxdAn?=
 =?us-ascii?Q?OUuF2p6ZkrQ5xTuzxImVgcT4ChQ0zTDjGuVq+9tFMHqlcAxE5vBtJgvktL+J?=
 =?us-ascii?Q?s8mpcMFPieZcJR7eJOJgtFKIArlE5/wEGWEi6kVv3iJ14wdGDl53xXJfbMqP?=
 =?us-ascii?Q?t1cqJzPvQiRtFC2m9ulyaJm2tJSojCS+vrWMAi0gGYDbN1jBdp5u0/8iJXnd?=
 =?us-ascii?Q?5EPmoyxbAp9k0tz2rvllfU7eOhJ22NHhqREQhUYa6sqkYiRg+Iif5DcNR7UX?=
 =?us-ascii?Q?Sf43IdK6CAXhVQWQGPPBxViRE9g8kh0RdMklU08bsfD9YJSM1jdp4/PR/9NZ?=
 =?us-ascii?Q?ovk0FKUBUj3oj9NLWXl/dpavUjL7RV66hbWR0KelWB3XaOchwEkP+h4irrbF?=
 =?us-ascii?Q?QFh48QXsFTT7nX1pXmpHYlk=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <738C0E2BDA76C94991EB84575AEC1A6B@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3434fbb8-34db-484b-b17f-08d9fb954ec5
X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Mar 2022 15:08:13.8664
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: OM3EKEaA3v+7ZSJ4VFe/GPArC+EJDZC6H9YeTgcpMYWmeE0uIbQrF7wit/kZl9p//wfRsRUnbW4a/C/6KkOWZA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB5719
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10272 signatures=684655
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 bulkscore=0
 adultscore=0 phishscore=0 spamscore=0 suspectscore=0 malwarescore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2201110000 definitions=main-2203010081
X-Proofpoint-GUID: PKDnQYKDE9UU4k-pKmudfhPO2kvbO17o
X-Proofpoint-ORIG-GUID: PKDnQYKDE9UU4k-pKmudfhPO2kvbO17o
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Feb 28, 2022, at 10:43 PM, NeilBrown <neilb@suse.de> wrote:
>=20
>=20
> When mounting NFS filesystems in a network namespace using v4, some care
> must be taken to ensure a unique and stable client identity.
> Add documentation explaining the requirements for container managers.
>=20
> Signed-off-by: NeilBrown <neilb@suse.de>
> ---
>=20
> NOTE I originally suggested using uuidgen to generate a uuid from a
> container name.  I've changed it to use the name as-is because I cannot
> see a justification for using a uuid - though I think that was suggested
> somewhere in the discussion.
> If someone would like to provide that justification, I'm happy to
> include it in the document.
>=20
> Thanks,
> NeilBrown
>=20
>=20
> utils/mount/nfs.man | 63 +++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 63 insertions(+)
>=20
> diff --git a/utils/mount/nfs.man b/utils/mount/nfs.man
> index d9f34df36b42..4ab76fb2df91 100644
> --- a/utils/mount/nfs.man
> +++ b/utils/mount/nfs.man
> @@ -1844,6 +1844,69 @@ export pathname, but not both, during a remount.  =
For example,
> merges the mount option
> .B ro
> with the mount options already saved on disk for the NFS server mounted a=
t /mnt.
> +.SH "NFS IN A CONTAINER"

To be clear, this explanation is about the operation of the
Linux NFS client in a container environment. The server has
different needs that do not appear to be addressed here.
The section title should be clear that this information
pertains to the client.


> +When NFS is used to mount filesystems in a container, and specifically
> +in a separate network name-space, these mounts are treated as quite
> +separate from any mounts in a different container or not in a
> +container (i.e. in a different network name-space).

It might be helpful to provide an introductory explanation of
how mount works in general in a namespaced environment. There
might already be one somewhere. The above text needs to be
clear that we are not discussing the mount namespace.


> +.P
> +In the NFSv4 protocol, each client must have a unique identifier.

... each client must have a persistent and globally unique
identifier.


> +This is used by the server to determine when a client has restarted,
> +allowing any state from a previous instance can be discarded.

Lots of passive voice here :-)

The server associates a lease with the client's identifier
and a boot instance verifier. The server attaches all of
the client's file open and lock state to that lease, which
it preserves until the client's boot verifier changes.


> So any two
> +concurrent clients that might access the same server MUST have
> +different identifiers, and any two consecutive instances of the same
> +client SHOULD have the same identifier.

Capitalized MUST and SHOULD have specific meanings in IETF
standards that are probably not obvious to average readers
of man pages. To average readers, this looks like shouting.
Can you use something a little friendlier?


> +.P
> +Linux constructs the identifier (referred to as=20
> +.B co_ownerid
> +in the NFS specifications) from various pieces of information, three of
> +which can be controlled by the sysadmin:
> +.TP
> +Hostname
> +The hostname can be different in different containers if they
> +have different "UTS" name-spaces.  If the container system ensures
> +each container sees a unique host name,

Actually, it turns out that is a pretty big "if". We've
found that our cloud customers are not careful about
setting unique hostnames. That's exactly why the whole
uniquifier thing is so critical!


> then this is
> +sufficient for a correctly functioning NFS identifier.
> +The host name is copied when the first NFS filesystem is mounted in
> +a given network name-space.  Any subsequent change in the apparent
> +hostname will not change the NFSv4 identifier.

The purpose of using a uuid here is that, given its
definition in RFC 4122, it has very strong global
uniqueness guarantees.

Using a UUID makes hostname uniqueness irrelevant.

Again, I think our goal should be hiding all of this
detail from administrators, because once we get this
mechanism working correctly, there is absolutely no
need for administrators to bother with it.


The remaining part of this text probably should be
part of the man page for Ben's tool, or whatever is
coming next.


> +.TP
> +.B nfs.nfs4_unique_id
> +This module parameter is the same for all containers on a given host
> +so it is not useful to differentiate between containers.
> +.TP
> +.B /sys/fs/nfs/client/net/identifier
> +This virtual file (available since Linux 5.3) is local to the network
> +name-space in which it is accessed and so can provided uniqueness betwee=
n
> +containers when the hostname is uniform among containers.
> +.RS
> +.PP
> +This value is empty on name-space creation.
> +If the value is to be set, that should be done before the first
> +mount (much as the hostname is copied before the first mount).
> +If the container system has access to some sort of per-container
> +identity, then a command like
> +.RS 4
> +echo "$CONTAINER_IDENTITY" \\
> +.br
> +   > /sys/fs/nfs/client/net/identifier=20
> +.RE
> +might be suitable.  If the container system provides no stable name,
> +but does have stable storage, then something like
> +.RS 4
> +[ -s /etc/nfsv4-uuid ] || uuidgen > /etc/nfsv4-uuid &&=20
> +.br
> +cat /etc/nfsv4-uuid > /sys/fs/nfs/client/net/identifier=20
> +.RE
> +would suffice.
> +.PP
> +If a container has neither a stable name nor stable (local) storage,
> +then it is not possible to provide a stable identifier, so providing
> +a random one to ensure uniqueness would be best
> +.RS 4
> +uuidgen > /sys/fs/nfs/client/net/identifier
> +.RE
> +.RE
> .SH FILES
> .TP 1.5i
> .I /etc/fstab
> --=20
> 2.35.1
>=20

--
Chuck Lever