Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <0e284f57-e03c-f128-f6e7-52a58edbcd54@amd.com>
Date:   Tue, 9 Aug 2022 09:55:11 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: [PATCH v2] drm/gem: Fix GEM handle release errors
Content-Language: en-US
To:     Chen Jeffy <jeffy.chen@rock-chips.com>,
        Daniel Vetter <daniel.vetter@ffwll.ch>
Cc:     Andy Yan <andy.yan@rock-chips.com>,
        Jianqun Xu <jay.xu@rock-chips.com>,
        Maxime Ripard <mripard@kernel.org>,
        Sumit Semwal <sumit.semwal@linaro.org>,
        Thomas Zimmermann <tzimmermann@suse.de>,
        linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org,
        linaro-mm-sig@lists.linaro.org, David Airlie <airlied@linux.ie>,
        Maarten Lankhorst <maarten.lankhorst@linux.intel.com>,
        linux-media@vger.kernel.org, Daniel Vetter <daniel@ffwll.ch>
References: <20220803083237.3701-1-jeffy.chen@rock-chips.com>
 <c7cb225b-7f21-8d9a-773b-efc655e6332c@amd.com>
 <7cd16264-fa84-7b50-f3ed-64f7f22dcef2@rock-chips.com>
 <64bf4e4b-4e22-0ff0-5f92-76f603c04ec0@amd.com>
 <cd806954-e94e-aec8-2b0c-4047da9a92ec@rock-chips.com>
From:   =?UTF-8?Q?Christian_K=c3=b6nig?= <christian.koenig@amd.com>
In-Reply-To: <cd806954-e94e-aec8-2b0c-4047da9a92ec@rock-chips.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TGhjM3pwUFpJb3NsbVpuTnRCNElPUkhxV3pOT29ySTZ1eU1jNTlIVjJZQXJ5?=
 =?utf-8?B?cElGeDIrTlNsQi9xQlRQdHZRY3N4Q0xCVTZzVXFTdzN4d2VkQXRoKzU4SlRT?=
 =?utf-8?B?OEVnaWR4VEJyZkEyTGNDOFhTNzZSV1BueGpoTDZTRjlwWEU0NFNJUUlITXVq?=
 =?utf-8?B?c0JralNSRmRrbGhTcWtXcUNYM3d6RG52d1A0SkNUMUlrWEJRdFFYWjdvU2ZB?=
 =?utf-8?B?dEpYRXBMRXlNTkFPNUQ1U2ovYWpneGNoSFBGZjFWRGVtU1ZqZkY2Tm1ubVB2?=
 =?utf-8?B?TEppd3dKaGFTZlQ4SGZNeUYraWhHMThBM2xtUGFaYkg3eW1aSkQzREZWVGlQ?=
 =?utf-8?B?eU1UR21pRnRSV2VSNTl3WTU4YVpzUlVRdkZUR0RSaUZYUUNxYmtFd2tsM0FT?=
 =?utf-8?B?VjdDekJMVUowSFZEWW5McHZWMWxHK0F3MTY4REdYMHlMTFFya2JNMkxkLzdB?=
 =?utf-8?B?aXRIdkwwSVdyR3dYbzJRMkEyc2ZjUTBSK25zWDFKanVPUkFBSEd3L1RxMUtr?=
 =?utf-8?B?RzRCUDhtRmdGRkdrZ1dRM1hyUEIzT3VOS2I3VUlMZk1BWmJwL2xnWlE3T21Y?=
 =?utf-8?B?REhzYnI3aVcyQTNOY21uMDg0eHpINU1WVFFhRFNqUVN0ZUFmNVFpc0RyMWhi?=
 =?utf-8?B?d0FIdW43Mmk2YkVsaHdjTE1ZT1FpQ0pNTnN0WkQ1dFFKUHJxa1pxeEQ2UFdy?=
 =?utf-8?B?dHZOeUk3YzJhV3U4UmV2MGpsWU44Y1BUV2lXOVJGTENmUWNOdVhPZ2NMQ3Bw?=
 =?utf-8?B?WXB1SmJ1UG00K1czbE1rZytmUVdQRmVQeVEzN2Vzd0JSOG9CRjJYSWVvVStY?=
 =?utf-8?B?UDZydHJiUmc4QmwrT2FJZTFSem5FdzhQdHl3NllPQkY2cjIyRE52OW0vK0hX?=
 =?utf-8?B?aE1SemtmdWxWMVBxNjdVL3NBeGIrV0huNWp4Sjl5dGZlUFExeG80VUJMRkpl?=
 =?utf-8?B?bEEyVEhPL2NtYXhSdnFwVjZtMHA5d2dXY3Ntc1RTZHJDUzZQWTR5TTBXQXZ1?=
 =?utf-8?B?R2ZyZ3EwRVEyd1Y5Z3d2SlA2OUQ4bys3Q2lNaXN1M0FrRVFkUzNFWnNRNXJC?=
 =?utf-8?B?bUVXVFU2SUFoNm1haDJRekZ5elFLZjJHYnB1QmpCWkNTeENXQnd4WlFGb2xS?=
 =?utf-8?B?TXc3ZFRidkxGUituUzJCQzUvM3h2N0ZhTjFQZDdkNHZKMnp3Z0FSWDNDRVF1?=
 =?utf-8?B?Y3BLR1ZsQStVbjhJS0dzWE5Tc00vQ2JSaGU2RjI0TXBKalNVKzl4Zld0OWZG?=
 =?utf-8?B?ZjJPdXJ2emRheGF3cmFsd3Zoend0VXhNRFBsMVlhenlBdzRKbmpramZMTlBi?=
 =?utf-8?B?TC82eDlFVWd3T2tyWkxoQXpZVnh3Z1ROOHdsaGVyVHJCM1J5eUd2cTAxeVB4?=
 =?utf-8?B?S25WR0E0Q1MvOUJlTGN4c2JXejRFNnZVWkoyTzlPMmgxdkxLM1hwR3hIZmlC?=
 =?utf-8?B?RDkwYk9ZcVIzMzBVYmZqcGlTWmtRY2FkSU5yWU0wY2dFSXRGMlBDVUVOallq?=
 =?utf-8?B?SEwvTGJaRm5kdWt2YkxqSmlSbXRTbXFpeWxXYW1mcDZMV1o1TlE3M0FDT0Zu?=
 =?utf-8?B?VWpJcnBNbjQzQ09aQW9NMWN6a043VUppWEw2bTBIK3FqSXdtWEgrNm5yRUlI?=
 =?utf-8?B?WGVQR01WamZ5R3UzVkpMS1Q5a3B5TFovTG9HbGtHbC85WlQzRU5lbFAzMnFm?=
 =?utf-8?B?a0wwdFBOTW9MR2ZEUkV2d3VWNjVEbHN2b3BmUk5lbVBpeDJGeVJjK01NVFZS?=
 =?utf-8?B?cktQODdKK3lWNVJYRXFpUGs3aEhScHJxNVhIbkpROHN1N2tWdzBYYzYvV1NL?=
 =?utf-8?B?SFBWUEQzdWFlRHJoamdkZFJSaGZHL2xkczlKT3B0NlIrdmhqb3VvWUhLcGNs?=
 =?utf-8?B?bFdJVGkrU3FqeC96RWpEWS9XVC9VSXRYNG90ZHIvUi8xbGJrejJ5QUtEdE9T?=
 =?utf-8?B?Um55QjRsb1hoZFBVKzkwMzZoVDREazF3SDIyV2xMMEVseGpvRFVsTWNRY2Ji?=
 =?utf-8?B?VlZBZ2VTSDJWN1BzRnVRYTE4RWVnRzcrSTlnUUFSK3RqOVZ1b1RYYk9hRFFo?=
 =?utf-8?B?d011cDZLdWNPV3I5b0x5T1pjQWJCTHpDamFGT3p0TEZNZXIzSlBxOUFvNDRp?=
 =?utf-8?Q?26Ghh6r8YHmrG46C9ay/M0JqS?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ff9f518-0481-44e2-88a0-08da79dc803e
X-MS-Exchange-CrossTenant-AuthSource: BN8PR12MB3587.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2022 07:55:18.0037
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EUnaJwUjvTZDq82ApoMuAsUoiVyqNDV8shMorTKIJKiTCkqRSchCKVLF+0WMZdQW
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7026
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Am 09.08.22 um 03:28 schrieb Chen Jeffy:
> Hi Christian,
>
> On 8/9 星期二 2:03, Christian König wrote:
>> Hi Jeffy,
>>
>> Am 08.08.22 um 05:51 schrieb Chen Jeffy:
>>> Hi Christian,
>>>
>>> Thanks for your reply, and sorry i didn't make it clear.
>>>
>>> On 8/8 星期一 0:52, Christian König wrote:
>>>> Am 03.08.22 um 10:32 schrieb Jeffy Chen:
>>>>> Currently we are assuming a one to one mapping between dmabuf and 
>>>>> handle
>>>>> when releasing GEM handles.
>>>>>
>>>>> But that is not always true, since we would create extra handles 
>>>>> for the
>>>>> GEM obj in cases like gem_open() and getfb{,2}().
>>>>>
>>>>> A similar issue was reported at:
>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F20211105083308.392156-1-jay.xu%40rock-chips.com%2F&amp;data=05%7C01%7Cchristian.koenig%40amd.com%7C52cd6ca16a3a415b92a708da79a67dec%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637956053232922419%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=hIuH18B10sbVAyS0D4iK6R6WYc%2BZ7mlxGcKdUae%2BW6Y%3D&amp;reserved=0 
>>>>>
>>>>>
>>>>> Another problem is that the drm_gem_remove_prime_handles() now only
>>>>> remove handle to the exported dmabuf (gem_obj->dma_buf), so the 
>>>>> imported
>>>>> ones would leak:
>>>>> WARNING: CPU: 2 PID: 236 at drivers/gpu/drm/drm_prime.c:228 
>>>>> drm_prime_destroy_file_private+0x18/0x24
>>>>>
>>>>> Let's fix these by using handle to find the exact map to remove.
>>>>
>>>> Well we are clearly something missing here. As far as I can see the 
>>>> current code is correct.
>>>>
>>>> Creating multiple GEM handles for the same DMA-buf is possible, but 
>>>> illegal. >
>>>> In other words when a GEM handle is exported as DMA-buf and 
>>>> imported again you should intentionally always get the same handle.
>>>
>>> These issue are not about having handles for importing an exported 
>>> dma-buf case, but for having multiple handles to a GEM object(which 
>>> means having multiple handles to a dma-buf).
>>>
>>> I know the drm-prime is trying to make dma-buf and handle maps one 
>>> to one, but the drm-gem is allowing to create extra handles for a 
>>> GEM object, for example:
>>> drm_gem_open_ioctl -> drm_gem_handle_create_tail
>>> drm_mode_getfb2_ioctl -> drm_gem_handle_create
>>> drm_mode_getfb -> fb->funcs->create_handle
>>
>> Yes, so far that's correct.
>>
>>>
>>>
>>> So we are allowing GEM object to have multiple handles, and GEM 
>>> object could have at most one dma-buf, doesn't that means that 
>>> dma-buf could map to multiple handles?
>>
>> No, at least not for the same GEM file private. That's the reason why 
>> the rb is indexed by the dma_buf object and not the handle.
>>
>> In other words the rb is so that you have exactly one handle for each 
>> dma_buf in each file private.
>
> I don't think so, because if user get multiple handles for the same 
> GEM obj and use drm_gem_prime_handle_to_fd() for those handles

Mhm, that works? This is illegal and should have been prevented somehow.

Let me double check the code.

Thanks for pointing that out,
Christian.


> , the current code would try to add multiple maps to rb:
> drm_prime_add_buf_handle(buf_1, hdl_1)
> drm_prime_add_buf_handle(buf_1, hdl_2)
> ...
> drm_prime_add_buf_handle(buf_1, hdl_n)
>
>>
>>>
>>> Or should we rewrite the GEM framework to limit GEM object with uniq 
>>> handle?
>>
>> No, the extra handles are expected because when you call 
>> drm_mode_getfb*() and drm_gem_open_ioctl() the caller now owns the 
>> returned GEM handle.
>>
>>>
>>> The other issue is that we are leaking dma-buf <-> handle map for 
>>> the imported dma-buf, since the drm_gem_remove_prime_handles doesn't 
>>> take care of obj->import_attach->dmabuf.
>>
>> No, that's correct as well. obj->dma_buf is set even for imported 
>> DMA-buf objects. See drm_gem_prime_fd_to_handle().
>
> Well, that obj->dma_buf would be set in 
> drm_gem_prime_fd_to_handle(create new handle), and cleared when 
> releasing the latest handle(release handle).
>
> So it doesn't cover other handle creating path.
>
> For example, a imported dma buf:
> drm_gem_prime_fd_to_handle <-- we got a handle and obj->dma_buf and 
> obj->import_attach->dmabuf
> drm_gem_handle_delete <-- we lost that handle and obj->dma_buf cleared
> drm_gem_open_ioctl/or getfb* <-- we got a new handle and 
> obj->import_attach->dmabuf
> drm_gem_handle_delete <-- we lost that handle and obj->dma_buf is 
> null, which means rb leaks.
>
>>
>> Regards,
>> Christian.
>>
>>>
>>> But of cause this can be fixed in other way:
>>> +++ b/drivers/gpu/drm/drm_gem.c
>>> @@ -180,6 +180,9 @@ drm_gem_remove_prime_handles(struct 
>>> drm_gem_object *obj, struct drm_file *filp)
>>> drm_prime_remove_buf_handle_locked(&filp->prime,
>>> obj->dma_buf);
>>>         }
>>> +       if (obj->import_attach)
>>> + drm_prime_remove_buf_handle_locked(&filp->prime,
>>> + obj->import_attach->dmabuf);
>>>         mutex_unlock(&filp->prime.lock);
>>>  }
>>>
>>>
>>>> So this is pretty much a clear NAK to this patch since it shouldn't 
>>>> be necessary or something is seriously broken somewhere else.
>>>>
>>>> Regards,
>>>> Christian.
>>>>
>>>>>
>>>>> Signed-off-by: Jeffy Chen <jeffy.chen@rock-chips.com>
>>>>> ---
>>>>>
>>>>> Changes in v2:
>>>>> Fix a typo of rbtree.
>>>>>
>>>>>   drivers/gpu/drm/drm_gem.c      | 17 +----------------
>>>>>   drivers/gpu/drm/drm_internal.h |  4 ++--
>>>>>   drivers/gpu/drm/drm_prime.c    | 20 ++++++++++++--------
>>>>>   3 files changed, 15 insertions(+), 26 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
>>>>> index eb0c2d041f13..ed39da383570 100644
>>>>> --- a/drivers/gpu/drm/drm_gem.c
>>>>> +++ b/drivers/gpu/drm/drm_gem.c
>>>>> @@ -168,21 +168,6 @@ void drm_gem_private_object_init(struct 
>>>>> drm_device *dev,
>>>>>   }
>>>>>   EXPORT_SYMBOL(drm_gem_private_object_init);
>>>>> -static void
>>>>> -drm_gem_remove_prime_handles(struct drm_gem_object *obj, struct 
>>>>> drm_file *filp)
>>>>> -{
>>>>> -    /*
>>>>> -     * Note: obj->dma_buf can't disappear as long as we still hold a
>>>>> -     * handle reference in obj->handle_count.
>>>>> -     */
>>>>> -    mutex_lock(&filp->prime.lock);
>>>>> -    if (obj->dma_buf) {
>>>>> - drm_prime_remove_buf_handle_locked(&filp->prime,
>>>>> -                           obj->dma_buf);
>>>>> -    }
>>>>> -    mutex_unlock(&filp->prime.lock);
>>>>> -}
>>>>> -
>>>>>   /**
>>>>>    * drm_gem_object_handle_free - release resources bound to 
>>>>> userspace handles
>>>>>    * @obj: GEM object to clean up.
>>>>> @@ -253,7 +238,7 @@ drm_gem_object_release_handle(int id, void 
>>>>> *ptr, void *data)
>>>>>       if (obj->funcs->close)
>>>>>           obj->funcs->close(obj, file_priv);
>>>>> -    drm_gem_remove_prime_handles(obj, file_priv);
>>>>> +    drm_prime_remove_buf_handle(&file_priv->prime, id);
>>>>>       drm_vma_node_revoke(&obj->vma_node, file_priv);
>>>>>       drm_gem_object_handle_put_unlocked(obj);
>>>>> diff --git a/drivers/gpu/drm/drm_internal.h 
>>>>> b/drivers/gpu/drm/drm_internal.h
>>>>> index 1fbbc19f1ac0..7bb98e6a446d 100644
>>>>> --- a/drivers/gpu/drm/drm_internal.h
>>>>> +++ b/drivers/gpu/drm/drm_internal.h
>>>>> @@ -74,8 +74,8 @@ int drm_prime_fd_to_handle_ioctl(struct 
>>>>> drm_device *dev, void *data,
>>>>>   void drm_prime_init_file_private(struct drm_prime_file_private 
>>>>> *prime_fpriv);
>>>>>   void drm_prime_destroy_file_private(struct 
>>>>> drm_prime_file_private *prime_fpriv);
>>>>> -void drm_prime_remove_buf_handle_locked(struct 
>>>>> drm_prime_file_private *prime_fpriv,
>>>>> -                    struct dma_buf *dma_buf);
>>>>> +void drm_prime_remove_buf_handle(struct drm_prime_file_private 
>>>>> *prime_fpriv,
>>>>> +                 uint32_t handle);
>>>>>   /* drm_drv.c */
>>>>>   struct drm_minor *drm_minor_acquire(unsigned int minor_id);
>>>>> diff --git a/drivers/gpu/drm/drm_prime.c 
>>>>> b/drivers/gpu/drm/drm_prime.c
>>>>> index e3f09f18110c..bd5366b16381 100644
>>>>> --- a/drivers/gpu/drm/drm_prime.c
>>>>> +++ b/drivers/gpu/drm/drm_prime.c
>>>>> @@ -190,29 +190,33 @@ static int 
>>>>> drm_prime_lookup_buf_handle(struct drm_prime_file_private *prime_fpri
>>>>>       return -ENOENT;
>>>>>   }
>>>>> -void drm_prime_remove_buf_handle_locked(struct 
>>>>> drm_prime_file_private *prime_fpriv,
>>>>> -                    struct dma_buf *dma_buf)
>>>>> +void drm_prime_remove_buf_handle(struct drm_prime_file_private 
>>>>> *prime_fpriv,
>>>>> +                 uint32_t handle)
>>>>>   {
>>>>>       struct rb_node *rb;
>>>>> -    rb = prime_fpriv->dmabufs.rb_node;
>>>>> +    mutex_lock(&prime_fpriv->lock);
>>>>> +
>>>>> +    rb = prime_fpriv->handles.rb_node;
>>>>>       while (rb) {
>>>>>           struct drm_prime_member *member;
>>>>> -        member = rb_entry(rb, struct drm_prime_member, dmabuf_rb);
>>>>> -        if (member->dma_buf == dma_buf) {
>>>>> +        member = rb_entry(rb, struct drm_prime_member, handle_rb);
>>>>> +        if (member->handle == handle) {
>>>>>               rb_erase(&member->handle_rb, &prime_fpriv->handles);
>>>>>               rb_erase(&member->dmabuf_rb, &prime_fpriv->dmabufs);
>>>>> -            dma_buf_put(dma_buf);
>>>>> +            dma_buf_put(member->dma_buf);
>>>>>               kfree(member);
>>>>> -            return;
>>>>> -        } else if (member->dma_buf < dma_buf) {
>>>>> +            break;
>>>>> +        } else if (member->handle < handle) {
>>>>>               rb = rb->rb_right;
>>>>>           } else {
>>>>>               rb = rb->rb_left;
>>>>>           }
>>>>>       }
>>>>> +
>>>>> +    mutex_unlock(&prime_fpriv->lock);
>>>>>   }
>>>>>   void drm_prime_init_file_private(struct drm_prime_file_private 
>>>>> *prime_fpriv)
>>>>
>>>>
>>>
>>
>>
>