Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Olga Kornievskaia <olga.kornievskaia@gmail.com>
CC:     Trond Myklebust <trond.myklebust@hammerspace.com>,
        Anna Schumaker <anna.schumaker@netapp.com>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: [RFC 1/2] xprtrdma: xdr pad optimization revisted again
Thread-Topic: [RFC 1/2] xprtrdma: xdr pad optimization revisted again
Thread-Index: AQHXnb+EPQ6bfsCh/km34j6ENph6pKuMRqKA
Date:   Mon, 30 Aug 2021 17:04:18 +0000
Message-ID: <F7F9A947-B282-416F-BC65-796BF325054F@oracle.com>
References: <20210830165302.60225-1-olga.kornievskaia@gmail.com>
 <20210830165302.60225-2-olga.kornievskaia@gmail.com>
In-Reply-To: <20210830165302.60225-2-olga.kornievskaia@gmail.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?KuXrioKRZsbvckwRG7YROQnlom0gvr410E1E9TF6Il6fvu3z0Q72FYS5aile?=
 =?us-ascii?Q?WilD67z0FESsURzqNFk9ucFilqGr8fRnjj3VF8jXlJj3czq2hnQc/f0YrjTp?=
 =?us-ascii?Q?7HsVudnxcqzPTvn8qwBKMEtlqcmqeosMflGtsLAwg6VQVpcjpsZ+hQVUFCOW?=
 =?us-ascii?Q?+gxC5vVyWtagcsqdwPimSsKhYzySYJE2ErUKhvyXEBnqpExENL3rUou4qlCv?=
 =?us-ascii?Q?AsQJMyMDMIlqWrE7C90PWPhGyoVKxAzacm9yLVGi+YWyhjRuaD+9dWge0eAc?=
 =?us-ascii?Q?cEoBJlHttXJ/O34/PsIBA0/48AvUupc1iEsF0yOHSYjwsPsg/KWy8d6KWwS9?=
 =?us-ascii?Q?Hz1hfrlrAmMHKMrr/P3MW92cehBbXaluDYcM3JaH1qrj7BlCFy8uKHo0oSeh?=
 =?us-ascii?Q?UzAiEpnEdQiHfqLweFg4MrxvM2sT4Y78RitlXp9BJTmASNBHcLiBsx1TFGKX?=
 =?us-ascii?Q?WyQqzf99Ya8WPtlM5SMeYFdvFbcdjDSf+rLe2LyKnvdybpTuAxgL5V0NlmgD?=
 =?us-ascii?Q?J4/qGmNhbdZ1IEk8AKYlznkAAElUSBpAi3RiiQJa1Xxktmu/WjGZJgPNaJRR?=
 =?us-ascii?Q?3lOKHnFTWKvM1HrlJZ9pcad9+Oi9q+TJxYwfxZ4Fis1Gmh3ZVcM1YzSWOVXC?=
 =?us-ascii?Q?s9qZPk4HspoRgvChvo23U0tvFRH0xjd5hm390AiCNS1NYY7ieeo5amQxhkLN?=
 =?us-ascii?Q?MrTHp4OC+e8TX8Tcc+YeLpxryW8Bxl2bkVV+0Rl5Az68XbnmNFcsj8conqte?=
 =?us-ascii?Q?5nYrgEVRgqEajlSVnR7tBYCH7NtpgdHomOEvWfK5NCLHGtSvB/d5URJR+dn4?=
 =?us-ascii?Q?42mQtJ1LY2D3MpFMS3HnRPhsc4BwA0T7DLZwmzJs1WjtbCCwOIgffwVPaX+5?=
 =?us-ascii?Q?oDp8X2UIq6Q5hYszamkOIu+AaxYWqLINx0nIQp/xlm0Lk6PXCKBbnhXAHDb4?=
 =?us-ascii?Q?0bxnXq1090rByGBraXMbIprLEsSu8QZhPXOFioMESbDC1pm3X/0o8qE5AVCR?=
 =?us-ascii?Q?qsgGpTS4R8oNM84JJRGMquL5VKi+LU6iQyEKYk2YNGsKI+o1fl2vYPwEwLQr?=
 =?us-ascii?Q?lyV4T79FDsJClXG9y3ECOwM8MkSRlDaasF76oenGDtLVGpAk9tj1aUJki4+b?=
 =?us-ascii?Q?++2vdcLrtuY9dg54J+EYH2EP/dwNTpj1bsDZUKdEqBQnAs0T29jo8qO8UU0C?=
 =?us-ascii?Q?odvWfiUMNtRxxNzVL8HXhPB9z4gj5YS6CtbkUEVnBLoJwZAXNv9ZgKmAkMJe?=
 =?us-ascii?Q?SMNRgAGKVWuW1oeD3Oh4bq3nIoE0ahuVGt5TSXqkrItjFXctJ3CKi5Vp+uBP?=
 =?us-ascii?Q?3Jk6uriUplgKtmHNypij4rNv?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-ID: <2D0BEA26C05AE34B9D6F3968105BA062@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4688.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 290e6e86-a555-4eb7-fac8-08d96bd83463
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Aug 2021 17:04:18.4267
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: BcHGjoedG4x24yjVIs7evpC9p8EK15cersexCArAF4IyMbYXiilXIjsxBQRV5do+fWikcpjnuRmQnRnKo8ApjQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3462
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10092 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 bulkscore=0 phishscore=0
 spamscore=0 mlxlogscore=999 adultscore=0 suspectscore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000
 definitions=main-2108300116
X-Proofpoint-GUID: mgo3B-uJ82NCv3JFRRvmeMzvsCn8NeHk
X-Proofpoint-ORIG-GUID: mgo3B-uJ82NCv3JFRRvmeMzvsCn8NeHk
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org

Hi Olga-

> On Aug 30, 2021, at 12:53 PM, Olga Kornievskaia <olga.kornievskaia@gmail.=
com> wrote:
>=20
> From: Olga Kornievskaia <kolga@netapp.com>
>=20
> Given the patch "Always provide aligned buffers to the RPC read layers",
> RPC over RDMA doesn't need to look at the tail page and add that space
> to the write chunk.
>=20
> For the RFC 8166 compliant server, it must not write an XDR padding
> into the write chunk (even if space was provided). Historically
> (before RFC 8166) Solaris RDMA server has been requiring the client
> to provide space for the XDR padding and thus this client code has
> existed.

I don't understand this change.

So, the upper layer doesn't provide XDR padding any more. That doesn't
mean Solaris servers still aren't going to want to write into it. The
client still has to provide this padding from somewhere.

This suggests that "Always provide aligned buffers to the RPC read
layers" breaks our interop with Solaris servers. Does it?


> Signed-off-by: Olga Kornievskaia <kolga@netapp.com>
> ---
> net/sunrpc/xprtrdma/rpc_rdma.c | 15 ---------------
> 1 file changed, 15 deletions(-)
>=20
> diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdm=
a.c
> index c335c1361564..2c4146bcf2a8 100644
> --- a/net/sunrpc/xprtrdma/rpc_rdma.c
> +++ b/net/sunrpc/xprtrdma/rpc_rdma.c
> @@ -255,21 +255,6 @@ rpcrdma_convert_iovs(struct rpcrdma_xprt *r_xprt, st=
ruct xdr_buf *xdrbuf,
> 		page_base =3D 0;
> 	}
>=20
> -	if (type =3D=3D rpcrdma_readch)
> -		goto out;
> -
> -	/* When encoding a Write chunk, some servers need to see an
> -	 * extra segment for non-XDR-aligned Write chunks. The upper
> -	 * layer provides space in the tail iovec that may be used
> -	 * for this purpose.
> -	 */
> -	if (type =3D=3D rpcrdma_writech && r_xprt->rx_ep->re_implicit_roundup)
> -		goto out;
> -
> -	if (xdrbuf->tail[0].iov_len)

Instead of checking for a tail, we could check

	if (xdr_pad_size(xdrbuf->page_len))

and provide some tail space in that case.

> -		rpcrdma_convert_kvec(&xdrbuf->tail[0], seg, &n);
> -
> -out:
> 	if (unlikely(n > RPCRDMA_MAX_SEGS))
> 		return -EIO;
> 	return n;
> --=20
> 2.27.0
>=20

--
Chuck Lever