Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
MIME-Version: 1.0
References: <20231103095549.490744-1-lizhijian@fujitsu.com> <20231103095549.490744-2-lizhijian@fujitsu.com>
In-Reply-To: <20231103095549.490744-2-lizhijian@fujitsu.com>
From:   Greg Sword <gregsword0@gmail.com>
Date:   Fri, 3 Nov 2023 18:14:09 +0800
Message-ID: <CAEz=Lcu012KvkJbEC=ZeTFZk1vg51B2h66E6Hoh0JOcez58a=w@mail.gmail.com>
Subject: Re: [PATCH RFC V2 1/6] RDMA/rxe: RDMA/rxe: don't allow registering
 !PAGE_SIZE mr
To:     Li Zhijian <lizhijian@fujitsu.com>
Cc:     zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org,
        linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org,
        rpearsonhpe@gmail.com, matsuda-daisuke@fujitsu.com,
        bvanassche@acm.org, yi.zhang@redhat.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Fri, Nov 3, 2023 at 5:56=E2=80=AFPM Li Zhijian <lizhijian@fujitsu.com> w=
rote:
>
> rxe_set_page() only store one PAGE_SIZE page by the step of page_size.
> when page_size !=3D PAGE_SIZE, we cannot restore the address with wrong
> index and page_offset.
>
> Let's take a look how current the xarray is being used.
>
> 0. offset =3D iova & (page_size -1); // offset is less than page_size
>                                       but may not PAGE_SIZE
> 1. index =3D (iova - mr.iova) >> page_shift;
> 2. page =3D xa_load(&mr->page_list, index);
> 3. page_va =3D kmap_local_page(page) // map one page only, that means onl=
y
>                                       memory [page_va, page_va + PAGE_SIZ=
E)
>                                       is valid for this mapping.
> 4. memcpy(addr, page_va + offset, bytes);
>
> - when page_size > PAGE_SIZE, the offset could be beyond PAGE_SIZE,
>   then page_va + offset may be invalid.
> - when page_size < PAGE_SIZE, the offset may get lost.
>
> Note that this patch will break some ULPs that try to register 4K
> MR when PAGE_SIZE is not 4K. SRP and nvme over RXE is known to be
> impacted.
>
> Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
>
> ---
> ---
>  drivers/infiniband/sw/rxe/rxe_mr.c | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/r=
xe/rxe_mr.c
> index f54042e9aeb2..3755e530e6dc 100644
> --- a/drivers/infiniband/sw/rxe/rxe_mr.c
> +++ b/drivers/infiniband/sw/rxe/rxe_mr.c
> @@ -234,6 +234,12 @@ int rxe_map_mr_sg(struct ib_mr *ibmr, struct scatter=
list *sgl,
>         struct rxe_mr *mr =3D to_rmr(ibmr);
>         unsigned int page_size =3D mr_page_size(mr);
>
> +       if (page_size !=3D PAGE_SIZE) {
> +               rxe_err_mr(mr, "Unsupport mr page size %x, expect PAGE_SI=
ZE(%lx)\n",
> +                          page_size, PAGE_SIZE);
> +               return -EINVAL;
> +       }

Are you kidding us? What problem you are fixing? Do you make tests in your =
host?

A  rubbish patch.

> +
>         mr->nbuf =3D 0;
>         mr->page_shift =3D ilog2(page_size);
>         mr->page_mask =3D ~((u64)page_size - 1);
> --
> 2.41.0
>