Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <163d3271-e1e9-c325-185d-adb4391023e1@amd.com>
Date:   Wed, 20 Jul 2022 10:29:42 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: [PATCH v1 4/6] dma-buf: Acquire wait-wound context on attachment
Content-Language: en-US
To:     Dmitry Osipenko <dmitry.osipenko@collabora.com>,
        David Airlie <airlied@linux.ie>,
        Gerd Hoffmann <kraxel@redhat.com>,
        Gurchetan Singh <gurchetansingh@chromium.org>,
        Chia-I Wu <olvaffe@gmail.com>, Daniel Vetter <daniel@ffwll.ch>,
        Daniel Almeida <daniel.almeida@collabora.com>,
        Gert Wollny <gert.wollny@collabora.com>,
        Gustavo Padovan <gustavo.padovan@collabora.com>,
        Daniel Stone <daniel@fooishbar.org>,
        Tomeu Vizoso <tomeu.vizoso@collabora.com>,
        Maarten Lankhorst <maarten.lankhorst@linux.intel.com>,
        Maxime Ripard <mripard@kernel.org>,
        Thomas Zimmermann <tzimmermann@suse.de>,
        Rob Clark <robdclark@gmail.com>,
        Sumit Semwal <sumit.semwal@linaro.org>,
        "Pan, Xinhui" <Xinhui.Pan@amd.com>,
        Thierry Reding <thierry.reding@gmail.com>,
        Tomasz Figa <tfiga@chromium.org>,
        Marek Szyprowski <m.szyprowski@samsung.com>,
        Mauro Carvalho Chehab <mchehab@kernel.org>,
        Alex Deucher <alexander.deucher@amd.com>,
        Jani Nikula <jani.nikula@linux.intel.com>,
        Joonas Lahtinen <joonas.lahtinen@linux.intel.com>,
        Rodrigo Vivi <rodrigo.vivi@intel.com>,
        Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
        =?UTF-8?Q?Thomas_Hellstr=c3=b6m?= <thomas_os@shipmail.org>
Cc:     dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
        Dmitry Osipenko <digetx@gmail.com>,
        linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
        amd-gfx@lists.freedesktop.org, intel-gfx@lists.freedesktop.org,
        kernel@collabora.com, virtualization@lists.linux-foundation.org,
        spice-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org,
        linux-arm-msm@vger.kernel.org
References: <20220715005244.42198-1-dmitry.osipenko@collabora.com>
 <20220715005244.42198-5-dmitry.osipenko@collabora.com>
 <5ec9313e-8498-2838-0320-331c347ce905@amd.com>
 <1ce233a2-36c9-3698-59f0-c4ff902bec60@collabora.com>
 <43446124-b99a-32d8-f797-7ec0cdca9ee4@collabora.com>
From:   =?UTF-8?Q?Christian_K=c3=b6nig?= <christian.koenig@amd.com>
In-Reply-To: <43446124-b99a-32d8-f797-7ec0cdca9ee4@collabora.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?QkZMNmhVd01WQU5sd1hKeVBJcFhMTzFPbS85dVVwR3BBZTBFaWtNeXhyUFNw?=
 =?utf-8?B?cEZnYTE2VTJPWExJaXJHZ0YzU1lzeFdOc1F5VTZWbnluV1c4VllEblNwVXlh?=
 =?utf-8?B?OW1taEs1a3BEWU5KTXgwSU9lSjVVdkNsVmx4OXdnQldERDVZZEsySmFqaTlx?=
 =?utf-8?B?dlFRbDg3RzlCMFNmYmJ1VHRheGFTYVo1T2NTL3hieHBteStVeE15cHUxaTBS?=
 =?utf-8?B?RnBrT1R4VWc4NzVZOHgvbGxLNkZRempwQjFIYmZ2Kzk5TnVsNTZLYXdQR0VD?=
 =?utf-8?B?cExpZ01LTHl3SDlkQ0FWcjZ5RUx4LzhlU282TkQxY2l1NDl3NVlLdkVlQmtK?=
 =?utf-8?B?dWFXMEJhRFJBVS8zdnkzZmhnb1RVRlhhOVhJeGJBZXBvNTdzM0RmZ3Y2WU5H?=
 =?utf-8?B?TDA5QWVTWFo0S0JsM09GeUdJNU11UGFNNnlDVjlaRTB1NmpXbTlBYWpGaHA3?=
 =?utf-8?B?MklQYVRZSXpaZnkzeFQwcmxOOXlSTk56WlFaaFhzN0Ezd0VkNzFpN1pHeDVN?=
 =?utf-8?B?SFdGQ2hhckJtNXFqbEVBRWMzSGlaWUJXMzZjZldLZnRob2k2OWZ5Yjd2K2hQ?=
 =?utf-8?B?OHhUTTlia1lrdllMSnNaVGlJUGVuMWtMK041MVRaaExYREttcE4xQzhSNnNK?=
 =?utf-8?B?UXVpTE9iTys5OWZRK1RmeHhTbDNBQVBrVnlVczdIZmEvY0dJUVJGaHR3Q2w1?=
 =?utf-8?B?Uk5lZC9wQzZwUGJLNW93bEt4Tm9nZS9zdkVhUEx3RXU5NFdOdnBOaklQQlpv?=
 =?utf-8?B?SHIvZy84Ylc4U2ZuVzlyVjhxeFo1MCtoUm9EbU1zd2Q2cmx0S3FmWlZXTzFr?=
 =?utf-8?B?MEFoS1RkcHcwZVBpVTVSeDdlTFdkaGNYVVdCd0NRS3ZiVHY4UVZ1dWpZSWlB?=
 =?utf-8?B?REQ2M3BvblRXK2xCa0xGc0xvM2Q0YnQzL0pkdmVHOUUwZHJDSEtxZUMzSnd2?=
 =?utf-8?B?dDdnb2I4Q0hJaXdPa2hzSHZ5b24zdmpCU2UwS1UvTlE3Nk96bXlKZFZsQytz?=
 =?utf-8?B?ZEhUWlgwdk1VVVNDY2JtUUxndDYrSXhWc0VmK3E3eUVnWnlNaENFV0VRSmZG?=
 =?utf-8?B?NU9qanNleGxrbmRxTFB5Y29NWklSZHBiQ01uQk1HNmQ3NUJud293S0NuOW02?=
 =?utf-8?B?R21nY2diWUZUNTFmVFZoYTNvMDVWVjN3TWVsQ0J4YVB3VWVacUtRV3kwWjk0?=
 =?utf-8?B?RUZTSGlLeE02ZUVoVnJPTFl1Y211aUY2ZG9JSS9qNTYxOFJudTNuZFpPcEwz?=
 =?utf-8?B?MnZjWG5MY2kwQzhwSnpIaExnL0RDcXhLRm5rUXpncGJsQ0NuQWUxK3dRbzYy?=
 =?utf-8?B?c1Yyc1NKT2w3VmxIWXVRS0xzYVk4cUYwN0s4c0ZJdnlkVm0yVGZFay9kL0dT?=
 =?utf-8?B?azV5MThJdnkwVmMyQ3NXNlpiNHlYZ2cxOTh4Y2pMdFlPMVN1Z2JDK3N5eHNK?=
 =?utf-8?B?ZVlubTNhSW5aMmdXTDFnVGphQ2JXNHE3VExHUXVpcEMzVGhKL1Z3bVB0QXpW?=
 =?utf-8?B?a0h2NlAycnJZN1pXWWtEWitBTlBFdlQ1Vzc3MEk3OWRTZWZ5Snd4b2dVR0Ft?=
 =?utf-8?B?UDY1RlhUa3Rmb3FmRmo3ZjdCbTBHVlJrUXpRcGhUZStaNzFibkhldjhoaXN5?=
 =?utf-8?B?Nis4RDJpbEVUbE9maUMrY3FLRFdkb21CV1Q5Uy94eUZwWjNoRURpS1EvRkwr?=
 =?utf-8?B?MSsrOG1uNmZqaHl5VUhCTXdFM2Z6YXk4NWsyZHY3QTFoRlRTaVdKT0UxOUU1?=
 =?utf-8?B?QURDUkVvS21RZmpZS0h4dG9HUXRFT1Z6UG1VV01VNlFhZi82L2VHS3htOExq?=
 =?utf-8?B?cVhGVTNlWUNyZUtXNFJ3T3IvaGowdFBkeHlOMzlKTFYycWQ3bTE2S0w4U2lU?=
 =?utf-8?B?Q0ZRbHM1VlB3cUNEZTNDNXZvZ0xXTU45a1N2Z1QyQW9Qdk5ieHhRRXRETFln?=
 =?utf-8?B?VEVNQzdMbit1Y2RZQnFIRnJrT3RBQmI4NGo3WlFhTzFRaGV2d2dGTkRtMGZY?=
 =?utf-8?B?QzhNUWV0NDJIelJpNHNmdDZKNEJhYVg4ZmlnNTlQaVZ5WGs5N2EzSngrSEVj?=
 =?utf-8?B?QkV6cGZ5ZGc1d0pHaGxxbk5QMmR5Rm1MdzIrSGpTQjU1d051TXpVeVFIaml3?=
 =?utf-8?B?RHRRRkgwZEVhdEJVMkpNNmoxMUJPUEtQc1ZaYXRrRTVHR25sQndkT1pIeEdv?=
 =?utf-8?Q?PuNHWYQ8aDFAsTw+mMIzPtFioocfS0aCF6/g2yMs33Nn?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1e2450bb-1041-4e85-d769-08da6a2a0467
X-MS-Exchange-CrossTenant-AuthSource: BN8PR12MB3587.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jul 2022 08:29:52.3462
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: wda2XqT0St5yc2qZ9a2h3AeB9JVXAmOHLXJDcCOg7+F1X68iVqAMI68HcQ0xlQBO
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3198
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Am 19.07.22 um 22:05 schrieb Dmitry Osipenko:
> On 7/15/22 09:59, Dmitry Osipenko wrote:
>> On 7/15/22 09:50, Christian König wrote:
>>> Am 15.07.22 um 02:52 schrieb Dmitry Osipenko:
>>>> Intel i915 GPU driver uses wait-wound mutex to lock multiple GEMs on the
>>>> attachment to the i915 dma-buf. In order to let all drivers utilize
>>>> shared
>>>> wait-wound context during attachment in a general way, make dma-buf
>>>> core to
>>>> acquire the ww context internally for the attachment operation and update
>>>> i915 driver to use the importer's ww context instead of the internal one.
>>>>
>>>>   From now on all dma-buf exporters shall use the importer's ww context
>>>> for
>>>> the attachment operation.
>>>>
>>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>>> ---
>>>>    drivers/dma-buf/dma-buf.c                     |  8 +++++-
>>>>    drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  2 +-
>>>>    .../gpu/drm/i915/gem/i915_gem_execbuffer.c    |  2 +-
>>>>    drivers/gpu/drm/i915/gem/i915_gem_object.h    |  6 ++---
>>>>    drivers/gpu/drm/i915/i915_gem_evict.c         |  2 +-
>>>>    drivers/gpu/drm/i915/i915_gem_ww.c            | 26 +++++++++++++++----
>>>>    drivers/gpu/drm/i915/i915_gem_ww.h            | 15 +++++++++--
>>>>    7 files changed, 47 insertions(+), 14 deletions(-)
>>>>
>>>> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
>>>> index 0ee588276534..37545ecb845a 100644
>>>> --- a/drivers/dma-buf/dma-buf.c
>>>> +++ b/drivers/dma-buf/dma-buf.c
>>>> @@ -807,6 +807,8 @@ static struct sg_table * __map_dma_buf(struct
>>>> dma_buf_attachment *attach,
>>>>     * Optionally this calls &dma_buf_ops.attach to allow
>>>> device-specific attach
>>>>     * functionality.
>>>>     *
>>>> + * Exporters shall use ww_ctx acquired by this function.
>>>> + *
>>>>     * Returns:
>>>>     *
>>>>     * A pointer to newly created &dma_buf_attachment on success, or a
>>>> negative
>>>> @@ -822,6 +824,7 @@ dma_buf_dynamic_attach_unlocked(struct dma_buf
>>>> *dmabuf, struct device *dev,
>>>>                    void *importer_priv)
>>>>    {
>>>>        struct dma_buf_attachment *attach;
>>>> +    struct ww_acquire_ctx ww_ctx;
>>>>        int ret;
>>>>          if (WARN_ON(!dmabuf || !dev))
>>>> @@ -841,7 +844,8 @@ dma_buf_dynamic_attach_unlocked(struct dma_buf
>>>> *dmabuf, struct device *dev,
>>>>        attach->importer_ops = importer_ops;
>>>>        attach->importer_priv = importer_priv;
>>>>    -    dma_resv_lock(dmabuf->resv, NULL);
>>>> +    ww_acquire_init(&ww_ctx, &reservation_ww_class);
>>>> +    dma_resv_lock(dmabuf->resv, &ww_ctx);
>>> That won't work like this. The core property of a WW context is that you
>>> need to unwind all the locks and re-quire them with the contended one
>>> first.
>>>
>>> When you statically lock the imported one here you can't do that any more.
>> You're right. I felt that something is missing here, but couldn't
>> notice. I'll think more about this and enable
>> CONFIG_DEBUG_WW_MUTEX_SLOWPATH. Thank you!
>>
> Christian, do you think we could make an excuse for the attach()
> callback and make the exporter responsible for taking the resv lock? It
> will be inconsistent with the rest of the callbacks, where importer
> takes the lock, but it will be the simplest and least invasive solution.
> It's very messy to do a cross-driver ww locking, I don't think it's the
> right approach.

So to summarize the following calls will require that the caller hold 
the resv lock:
1. dma_buf_pin()/dma_buf_unpin()
2. dma_buf_map_attachment()/dma_buf_unmap_attachment()
3. dma_buf_vmap()/dma_buf_vunmap()
4. dma_buf_move_notify()

The following calls require that caller does not held the resv lock:
1. dma_buf_attach()/dma_buf_dynamic_attach()/dma_buf_detach()
2. dma_buf_export()/dma_buf_fd()
3. dma_buf_get()/dma_buf_put()
4. dma_buf_begin_cpu_access()/dma_buf_end_cpu_access()

If that's correct than that would work for me as well, but we should 
probably document this.

Or let me ask the other way around: What calls exactly do you need to 
change to solve your original issue? That was vmap/vunmap, wasn't it? If 
yes then let's concentrate on those for the moment.

Regards,
Christian.