Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Message-ID: <50bd7a39-b5ed-c15e-096f-2823d22cc5cc@amazon.com>
Date:   Tue, 12 Oct 2021 14:41:17 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.1.2
Subject: Re: [RFC PATCH 2/2] RDMA/efa: Add support for dmabuf memory regions
Content-Language: en-US
To:     Jason Gunthorpe <jgg@ziepe.ca>
CC:     Sumit Semwal <sumit.semwal@linaro.org>,
        =?UTF-8?Q?Christian_K=c3=b6nig?= <christian.koenig@amd.com>,
        Doug Ledford <dledford@redhat.com>,
        <linux-media@vger.kernel.org>, <dri-devel@lists.freedesktop.org>,
        <linux-kernel@vger.kernel.org>, <linux-rdma@vger.kernel.org>,
        Oded Gabbay <ogabbay@habana.ai>,
        Tomer Tayar <ttayar@habana.ai>,
        Yossi Leybovich <sleybo@amazon.com>,
        Alexander Matushevsky <matua@amazon.com>,
        Leon Romanovsky <leonro@nvidia.com>,
        Jianxin Xiong <jianxin.xiong@intel.com>,
        Firas Jahjah <firasj@amazon.com>
References: <20211007104301.76693-1-galpress@amazon.com>
 <20211007104301.76693-3-galpress@amazon.com>
 <20211007114018.GD2688930@ziepe.ca>
 <77082c57-29f8-1eba-b260-7cb658ec34d1@amazon.com>
 <20211011232839.GF2688930@ziepe.ca>
From:   Gal Pressman <galpress@amazon.com>
In-Reply-To: <20211011232839.GF2688930@ziepe.ca>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 12/10/2021 2:28, Jason Gunthorpe wrote:
> On Sun, Oct 10, 2021 at 09:55:49AM +0300, Gal Pressman wrote:
>> On 07/10/2021 14:40, Jason Gunthorpe wrote:
>>> On Thu, Oct 07, 2021 at 01:43:00PM +0300, Gal Pressman wrote:
>>>
>>>> @@ -1491,26 +1493,29 @@ static int efa_create_pbl(struct efa_dev *dev,
>>>>    return 0;
>>>>  }
>>>>
>>>> -struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,
>>>> -                   u64 virt_addr, int access_flags,
>>>> -                   struct ib_udata *udata)
>>>> +static void efa_dmabuf_invalidate_cb(struct dma_buf_attachment *attach)
>>>> +{
>>>> +  WARN_ON_ONCE(1,
>>>> +               "Invalidate callback should not be called when memory is pinned\n");
>>>> +}
>>>> +
>>>> +static struct dma_buf_attach_ops efa_dmabuf_attach_ops = {
>>>> +  .allow_peer2peer = true,
>>>> +  .move_notify = efa_dmabuf_invalidate_cb,
>>>> +};
>>>
>>> Shouldn't move_notify really just be left as NULL? I mean fixing
>>> whatever is preventing that?
>>
>> That's what I had in the previous RFC and I think Christian didn't really like it.
> 
> Well, having drivers define a dummy function that only fails looks
> a lot worse to me. If not null then it should be a general
> 'dmabuf_unsupported_move_notify' shared function

Will do.

>>>> +  err = ib_umem_dmabuf_map_pages(umem_dmabuf);
>>>> +  if (err) {
>>>> +          ibdev_dbg(&dev->ibdev, "Failed to map dmabuf pages\n");
>>>> +          goto err_unpin;
>>>> +  }
>>>> +  dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv);
>>>
>>> If it is really this simple the core code should have this logic,
>>> 'ib_umem_dmabuf_get_pinned()' or something
>>
>> Should get_pinned do just get + dma_buf_pin, or should it do
>> ib_umem_dmabuf_map_pages as well?
> 
> Yes the map_pages too, a umem is supposed to be dma mapped after
> creation.

Will do, thanks Jason.