Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Chuck Lever <chuck.lever@oracle.com>
To:     Tom Talpey <tom@talpey.com>
CC:     Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        "linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>
Subject: Re: [PATCH v2 3/6] xprtrdma: Refactor invocations of offset_in_page()
Thread-Topic: [PATCH v2 3/6] xprtrdma: Refactor invocations of
 offset_in_page()
Thread-Index: AQHW+kkmqLH1KTPM0US61/D6LB+p56pGuv+AgAAAdgA=
Date:   Wed, 3 Feb 2021 18:11:39 +0000
Message-ID: <A8FD067A-DD97-4A5D-BCB1-83DF3FAB3842@oracle.com>
References: <161236925476.1030487.10407536259816633879.stgit@manet.1015granger.net>
 <161236944700.1030487.6859398915626711523.stgit@manet.1015granger.net>
 <d0bbab3e-851c-3388-3d1c-cbc6249a6803@talpey.com>
In-Reply-To: <d0bbab3e-851c-3388-3d1c-cbc6249a6803@talpey.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <E378D25AA203624C819894E9C3CB1ADD@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: a08b921f-b565-47dd-6d92-08d8c86f26de
X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Feb 2021 18:11:39.0766
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ikGgRRaK2ks3CazVOGRz4fjllrcw8x8NZP1izDs5Yer7cTyaCfBc9ntk65dEOpFsXyIkcEQ0TSFn0ZegqDqa+A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4656
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9884 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0 phishscore=0
 suspectscore=0 mlxlogscore=999 bulkscore=0 mlxscore=0 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000
 definitions=main-2102030107
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9884 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 impostorscore=0
 mlxscore=0 spamscore=0 bulkscore=0 priorityscore=1501 adultscore=0
 lowpriorityscore=0 malwarescore=0 phishscore=0 mlxlogscore=999
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2009150000 definitions=main-2102030107
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Feb 3, 2021, at 1:09 PM, Tom Talpey <tom@talpey.com> wrote:
>=20
> This looks good, but the earlier 1/6 patch depends on the offset_in_page
> conversion in rpcrdma_convert_kvec.

I don't think it does... sg_set_buf() handles the offset_in_page() calculat=
ion
in that case.


> Won't that complicate any bisection?
>=20
> Reviewed-By: Tom Talpey <tom@talpey.com>
>=20
> On 2/3/2021 11:24 AM, Chuck Lever wrote:
>> Clean up so that offset_in_page() is invoked less often in the
>> most common case, which is mapping xdr->pages.
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> ---
>>  net/sunrpc/xprtrdma/frwr_ops.c  |    8 +++-----
>>  net/sunrpc/xprtrdma/rpc_rdma.c  |    4 ++--
>>  net/sunrpc/xprtrdma/xprt_rdma.h |    2 +-
>>  3 files changed, 6 insertions(+), 8 deletions(-)
>> diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_o=
ps.c
>> index 13a50f77dddb..766a1048a48a 100644
>> --- a/net/sunrpc/xprtrdma/frwr_ops.c
>> +++ b/net/sunrpc/xprtrdma/frwr_ops.c
>> @@ -306,16 +306,14 @@ struct rpcrdma_mr_seg *frwr_map(struct rpcrdma_xpr=
t *r_xprt,
>>  	if (nsegs > ep->re_max_fr_depth)
>>  		nsegs =3D ep->re_max_fr_depth;
>>  	for (i =3D 0; i < nsegs;) {
>> -		sg_set_page(&mr->mr_sg[i],
>> -			    seg->mr_page,
>> -			    seg->mr_len,
>> -			    offset_in_page(seg->mr_offset));
>> +		sg_set_page(&mr->mr_sg[i], seg->mr_page,
>> +			    seg->mr_len, seg->mr_offset);
>>    		++seg;
>>  		++i;
>>  		if (ep->re_mrtype =3D=3D IB_MR_TYPE_SG_GAPS)
>>  			continue;
>> -		if ((i < nsegs && offset_in_page(seg->mr_offset)) ||
>> +		if ((i < nsegs && seg->mr_offset) ||
>>  		    offset_in_page((seg-1)->mr_offset + (seg-1)->mr_len))
>>  			break;
>>  	}
>> diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rd=
ma.c
>> index 529adb6ad4db..b3e66b8f65ab 100644
>> --- a/net/sunrpc/xprtrdma/rpc_rdma.c
>> +++ b/net/sunrpc/xprtrdma/rpc_rdma.c
>> @@ -215,7 +215,7 @@ rpcrdma_convert_kvec(struct kvec *vec, struct rpcrdm=
a_mr_seg *seg,
>>  {
>>  	if (vec->iov_len) {
>>  		seg->mr_page =3D virt_to_page(vec->iov_base);
>> -		seg->mr_offset =3D vec->iov_base;
>> +		seg->mr_offset =3D offset_in_page(vec->iov_base);
>>  		seg->mr_len =3D vec->iov_len;
>>  		++seg;
>>  		++(*n);
>> @@ -248,7 +248,7 @@ rpcrdma_convert_iovs(struct rpcrdma_xprt *r_xprt, st=
ruct xdr_buf *xdrbuf,
>>  	page_base =3D offset_in_page(xdrbuf->page_base);
>>  	while (len) {
>>  		seg->mr_page =3D *ppages;
>> -		seg->mr_offset =3D (char *)page_base;
>> +		seg->mr_offset =3D page_base;
>>  		seg->mr_len =3D min_t(u32, PAGE_SIZE - page_base, len);
>>  		len -=3D seg->mr_len;
>>  		++ppages;
>> diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_=
rdma.h
>> index 02971e183989..ed1c5444fb9d 100644
>> --- a/net/sunrpc/xprtrdma/xprt_rdma.h
>> +++ b/net/sunrpc/xprtrdma/xprt_rdma.h
>> @@ -287,7 +287,7 @@ enum {
>>  struct rpcrdma_mr_seg {
>>  	u32		mr_len;		/* length of segment */
>>  	struct page	*mr_page;	/* underlying struct page */
>> -	char		*mr_offset;	/* IN: page offset, OUT: iova */
>> +	u64		mr_offset;	/* IN: page offset, OUT: iova */
>>  };
>>    /* The Send SGE array is provisioned to send a maximum size

--
Chuck Lever