Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Nagendra Tomar <Nagendra.Tomar@microsoft.com>
CC:     Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        Trond Myklebust <trond.myklebust@hammerspace.com>,
        Anna Schumaker <anna.schumaker@netapp.com>
Subject: Re: [PATCH 0/5] nfs: Add mount option for forcing RPC requests for
 one file over one connection
Thread-Topic: [PATCH 0/5] nfs: Add mount option for forcing RPC requests for
 one file over one connection
Thread-Index: Adcfp3hcmhRvn3IHQv25MQlIHJPMUAARGOoAAAMHR+AAAek9AAAAU0WQAAIhaYA=
Date:   Tue, 23 Mar 2021 17:24:56 +0000
Message-ID: <575FF32B-463E-41D8-882F-6A746DF7FDFA@oracle.com>
References: <SG2P153MB0361E2D69F8AD8AFBF9E7CD89E649@SG2P153MB0361.APCP153.PROD.OUTLOOK.COM>
 <CD47CE3E-0C07-4019-94AF-5474D531B12D@oracle.com>
 <SG2P153MB0361BCD6D1F8F86E0648E3EB9E649@SG2P153MB0361.APCP153.PROD.OUTLOOK.COM>
 <5B030422-09B7-470D-9C7A-18C666F5817D@oracle.com>
 <SG2P153MB0361DC1EF8EDD02356D29E579E649@SG2P153MB0361.APCP153.PROD.OUTLOOK.COM>
In-Reply-To: <SG2P153MB0361DC1EF8EDD02356D29E579E649@SG2P153MB0361.APCP153.PROD.OUTLOOK.COM>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1C34BE482172144B87D2BF6F721C6D72@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: d73b1b48-7eb3-4fe3-23fd-08d8ee209413
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Mar 2021 17:24:56.2546
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: +JBIaK/9bARrflTniFYxLMIFyvSy5T6DxH7BqFKmTeRjxSJK3xoY1MICR3Xrsx3l4I5jBmrjrtFS13sinc5Dzg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR10MB4275
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9932 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 phishscore=0
 mlxlogscore=999 suspectscore=0 spamscore=0 malwarescore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000
 definitions=main-2103230128
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9932 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 phishscore=0
 lowpriorityscore=0 suspectscore=0 clxscore=1015 priorityscore=1501
 spamscore=0 adultscore=0 impostorscore=0 mlxlogscore=999 mlxscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2009150000 definitions=main-2103230128
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Mar 23, 2021, at 12:29 PM, Nagendra Tomar <Nagendra.Tomar@microsoft.co=
m> wrote:
>=20
>>> On Mar 23, 2021, at 11:57 AM, Nagendra Tomar
>> <Nagendra.Tomar@microsoft.com> wrote:
>>>=20
>>>>> On Mar 23, 2021, at 1:46 AM, Nagendra Tomar
>>>> <Nagendra.Tomar@microsoft.com> wrote:
>>>>>=20
>>>>> From: Nagendra S Tomar <natomar@microsoft.com>
>>>>=20
>>>> The flexfiles layout can handle an NFSv4.1 client and NFSv3 data
>>>> servers. In fact it was designed for exactly this kind of mix of
>>>> NFS versions.
>>>>=20
>>>> No client code change will be necessary -- there are a lot more
>>>> clients than servers. The MDS can be made to work smartly in
>>>> concert with the load balancer, over time; or it can adopt other
>>>> clever strategies.
>>>>=20
>>>> IMHO pNFS is the better long-term strategy here.
>>>=20
>>> The fundamental difference here is that the clustered NFSv3 server
>>> is available over a single virtual IP, so IIUC even if we were to use
>>> NFSv41 with flexfiles layout, all it can handover to the client is that=
 single
>>> (load-balanced) virtual IP and now when the clients do connect to the
>>> NFSv3 DS we still have the same issue. Am I understanding you right?
>>> Can you pls elaborate what you mean by "MDS can be made to work
>>> smartly in concert with the load balancer"?
>>=20
>> I had thought there were multiple NFSv3 server targets in play.
>>=20
>> If the load balancer is making them look like a single IP address,
>> then take it out of the equation: expose all the NFSv3 servers to
>> the clients and let the MDS direct operations to each data server.
>>=20
>> AIUI this is the approach (without the use of NFSv3) taken by
>> NetApp next generation clusters.
>=20
> Yeah, if could have clients access all the NFSv3 servers then I agree, pN=
FS=20
> would be a viable option. Unfortunately that's not an option in this case=
. The=20
> cluster has 100's of nodes and it's not an on-prem server, but a cloud se=
rvice,
> so the simplicity of the single LB VIP is critical.

The clients mount only the MDS. The MDS provides the DS addresses, they are
not exposed to client administrators. If the MDS adopts the load balancer's=
 IP
address, then the clients would simply mount that same server address using
NFSv4.1.

The other alternative is to make the load balancer sniff the FH from each
NFS request and direct it to a consistent NFSv3 DS. I still prefer that
over adding a very special-case mount option to the Linux client. Again,
you'd be deploying a code change in one place, under your control, instead
of on 100's of clients.


--
Chuck Lever