Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Olga Kornievskaia <olga.kornievskaia@gmail.com>
CC:     "trond.myklebust@hammerspace.com" <trond.myklebust@hammerspace.com>,
        Anna Schumaker <anna.schumaker@netapp.com>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH v1] NFSv4.1 provide mount option to toggle trunking
 discovery
Thread-Topic: [PATCH v1] NFSv4.1 provide mount option to toggle trunking
 discovery
Thread-Index: AQHYKNyvQiH+fo5gFUC0apKHyPss06yi1UiAgAAojACAAAa3AA==
Date:   Thu, 24 Feb 2022 18:20:01 +0000
Message-ID: <BA1B54FF-FAD5-40FB-80CF-65424535C5F6@oracle.com>
References: <20220223174041.77887-1-olga.kornievskaia@gmail.com>
 <77E34F86-FC2E-4CBF-AFCA-272BAA7C4040@oracle.com>
 <CAN-5tyEWWbxCtWQPaMhYdP3OW-XKbCANZKx4mk9Fz=cwbQBU6g@mail.gmail.com>
In-Reply-To: <CAN-5tyEWWbxCtWQPaMhYdP3OW-XKbCANZKx4mk9Fz=cwbQBU6g@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?XBE6CLpHsreYuE9cSydBvJLIFVluruNdvhGQwECGpVtGvBlF76Y52xq289Mx?=
 =?us-ascii?Q?KwCFWVI3MZyEV0ECSLqmGXgxYdeYyfRMhc3JCQsNvAo6i/dWVtU6oddwalWS?=
 =?us-ascii?Q?4P3FD802SbfebkkRp+gKydPHL7nzpSakDnzTB7wi7EHKwG4CiF7JW5hVgiRf?=
 =?us-ascii?Q?zjzh51KcKCLCES/lkvlwDSA+WyzdErcMgZfU0RHi8kWsmRnTIJjPOpnXiFb1?=
 =?us-ascii?Q?PpZRvfo9mOEFWipFHewCOCMH6IyncHecdUFkKmdGLcUf9ohwplXq/Pd3eNj4?=
 =?us-ascii?Q?iWUQwWRzS1D38wvrndtDOqLpASNYtFgjBVx1QIeahKoF+z+rZEKUTaezWtxV?=
 =?us-ascii?Q?0L2tRiyKMw8lcJe7QSAZplR0zmR5elp6IGW6t+wA2Jbrs1TkL2FbVepDKdiJ?=
 =?us-ascii?Q?tNeEKCmDaKc+0QZKYM5ByDX8KbduS6l4uphRhGnyM/oZZj+QF1XkqBamywZ3?=
 =?us-ascii?Q?tPthmE5iW0HLIT7vJYH0y67RU96K9lgpU6a2m2H7REp8os9OdsmrkxhyH0SD?=
 =?us-ascii?Q?tcJfTWxDS31TDeW/se6vPyucyu3KUo+IxD9zSEeo/BZBF/N5K4+wVZD5vjgw?=
 =?us-ascii?Q?I/q7tdg0tN64LxaYG3Byf3da1fcTo230w5rQMEooaSViR4F7ovvt+z1JP1ft?=
 =?us-ascii?Q?T0Hvrm+YpiaJOGgVVyA/o9VBxcBhDxhM1rrprtwUUDP9QTwQ7AUri48/Yr4r?=
 =?us-ascii?Q?gpDb9L6xYGaopAKVIb9HoSmrxgRT8+IAYJOmvw0TBj/ckKkQG7io5+ZrZCwy?=
 =?us-ascii?Q?+c9BtxHwXW5bQ0dhVztEf8VVRgxNGn2TtOxpud2L0UmCRQ7RrQtW6BohWhI1?=
 =?us-ascii?Q?GzMyrfvK1JtpDoniSozneTHx7Byk2SRramHV/bRVvSU/7tuU3KCKzcVixBNI?=
 =?us-ascii?Q?jxDNeMXiORDWuVHgTGjz+357uB6a5JbsY9sY2LCz68Tved+215iOnZNfm70P?=
 =?us-ascii?Q?N6gg62wbCTzTCzeMhBJoEXHEm6SK1D3D4Cqcs3qATMq1FtsQiNK/KfZpodVg?=
 =?us-ascii?Q?n68cRAEv057g/sotTRLauOtoZmjQolWUbIFa//DC+L98rYPiATlKmqq72F67?=
 =?us-ascii?Q?wx+WOK4ugE+SXBv+d46E6RgK0iD9f8Htu9NS/NKBdbChaZX3gI11fXbGvuIB?=
 =?us-ascii?Q?pvMgXV9BFOcPJUogLdKmbNGprt8NJelLDWfCNhhnZKzWRnfwsaVMZ9a214pQ?=
 =?us-ascii?Q?YiZtaq5GBkQcl2mVu2ibApOo/bIjtyCIbvHzbJndwaoklKmMe0JFWaow9bCt?=
 =?us-ascii?Q?+OSgEi8J0+hZ4oE63fuOJ8KfSFgPYCwLKtO/oiZd+LoIGd51NL0Ihj8uGHSh?=
 =?us-ascii?Q?z2OoYhw02qJjZ4tGgeqU92tx9skuzPpGRdvEaCpygGQPXFlVL+MyL3rvUkGM?=
 =?us-ascii?Q?u24ntnr1NIGQagQOhzLla0svOxNinvmrLeRUtMa2KDyF3TJ+ZdgqFUZ3psf6?=
 =?us-ascii?Q?AJ1xZ5ENyuwcQQVoOwR+5hnlm10QSnkbhoKVQpkz3DlbsLkirG2oiV4xnzPv?=
 =?us-ascii?Q?P8MFpBb+YrMrAcLiPU/jshM5AWp1SxHMfXgatK/LrUEXF6L1ytbm/mRL62sW?=
 =?us-ascii?Q?LanIZOW369XSBpK+DFwtH/fxzu/tSH16F7xvIZ2YrmYdCjNl8xv5DSrlIG69?=
 =?us-ascii?Q?FbVTfhgPuEOWMZ9pXDtiYwI=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <3F469AFE60B0C6459C2AC5875BFB07F6@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e16a9c3f-b766-488c-325d-08d9f7c2458c
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Feb 2022 18:20:01.0906
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: KawcrgLkJyTs2BBvGgqM08OtXmQ5wDlOrG0EHp90lKJq/0tPTC4bMjPcxRn3EvYlbScexuFBKEJpxA+oaJiq6Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB4899
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10268 signatures=684655
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 spamscore=0
 mlxscore=0 adultscore=0 mlxlogscore=999 suspectscore=0 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000
 definitions=main-2202240103
X-Proofpoint-GUID: NKiVZ_Ba6Uy9WEroGVNSfkif08lvpe8V
X-Proofpoint-ORIG-GUID: NKiVZ_Ba6Uy9WEroGVNSfkif08lvpe8V
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Feb 24, 2022, at 12:55 PM, Olga Kornievskaia <olga.kornievskaia@gmail.=
com> wrote:
>=20
> On Thu, Feb 24, 2022 at 10:30 AM Chuck Lever III <chuck.lever@oracle.com>=
 wrote:
>>=20
>>> On Feb 23, 2022, at 12:40 PM, Olga Kornievskaia <olga.kornievskaia@gmai=
l.com> wrote:
>>>=20
>>> From: Olga Kornievskaia <kolga@netapp.com>
>>>=20
>>> Introduce a new mount option -- trunkdiscovery,notrunkdiscovery -- to
>>> toggle whether or not the client will engage in actively discovery
>>> of trunking locations.
>>=20
>> An alternative solution might be to change the client's
>> probe to treat NFS4ERR_DELAY as "no trunking information
>> available" and then allow operation to proceed on the
>> known good transport.
>=20
> I'm not sure what you mean about "the known good transport".

The transport on which the client sent the
GETATTR(fs_locations).

The NFS4ERR_DELAY response means the server has no other
trunks available "at this time."


> I don't
> think the ERR_DELAY is associated with a transport. Btw, if you saw a
> previous patch which restricts fs_location query to the main transport
> makes your statement even more confusing as it would mean there is no
> good transport. Or do you mean to say we should have trunking
> discovery done asynchronous to mount by a separate kernel thread and
> therefore not impact mount steps?

Yes, something like that.

Trunking discovery that is independent of the NFS mount
process should be the goal. In fact, trunking discovery
really ought to be done in user space.

- There is now a user/kernel API for managing transports

- The trunking configuration on the server might change
  during the lifetime of the mount, so periodic checking
  is needed

- Adding an extra round trip, especially one that might
  be slowed by one or more NFS4ERR_DELAY replies, is
  going to be a problem during a mount storm

- There might be local policies that affect which network
  paths to choose for trunking

- The choice of transports might be made automatically
  by an orchestrator

- Tying this setting to a mount option is not appropriate
  because the transports are shared amount multiple NFS
  mounts


> I do object to treating a single ERR_DELAY during discovery as a
> permanent error as there are legitimate reasons to a delay in looking
> up the information that can be resolved in time by the server.
> However, I don't object to putting a time limit or number of tries on
> ERR_DELAY as safety wheels.

In the past, some have objected to /any/ delay added to
the NFS mount process.

There's no reason to hold up the mount process -- the
client can try the trunking discovery probe again in a
few moments while the mount proceeds, can't it?

If that means handing the probe to a work queue or
leaving it to user space, that seems like a more
flexible choice.


> Lastly, I think perhaps we can do both have a mount option to toggle
> discovery as well as safeguard the discovery from broken servers?

I'd really rather not add a mount option for this
purpose unless you know of another reason why trunking
discovery needs to be disabled.

The best solution is to fix the server implementations.
If that's not possible then the second best is to have
the client manage the situation without needing any
human intervention.

Adding an administrative tunable is, to my mind, an
option of the very last resort.


--
Chuck Lever