Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Jiasheng Jiang <jiasheng@iscas.ac.cn>
To:     contact@emersion.fr
Cc:     harry.wentland@amd.com, sunpeng.li@amd.com,
        Rodrigo.Siqueira@amd.com, alexander.deucher@amd.com,
        christian.koenig@amd.com, Xinhui.Pan@amd.com, airlied@gmail.com,
        daniel@ffwll.ch, nicholas.kazlauskas@amd.com, roman.li@amd.com,
        aurabindo.pillai@amd.com, Jerry.Zuo@amd.com,
        dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org,
        linux-kernel@vger.kernel.org, Jiasheng Jiang <jiasheng@iscas.ac.cn>
Subject: [PATCH] drm/amd/display: rewrite the check for mods
Date:   Sat, 19 Nov 2022 21:16:50 +0800
Message-Id: <20221119131650.21525-1-jiasheng@iscas.ac.cn>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

On Thu, 17 Nov 2022 15:56:09 +0800, Simon Ser wrote:
>> @@ -638,11 +638,14 @@ static int get_plane_modifiers(struct amdgpu_device *adev, unsigned int plane_ty
>>  		return 0;
>>  
>>  	*mods = kmalloc(capacity * sizeof(uint64_t), GFP_KERNEL);
>> +	if (!*mods)
>> +		return -ENOMEM;
>> +
>>  
>>  	if (plane_type == DRM_PLANE_TYPE_CURSOR) {
>>  		add_modifier(mods, &size, &capacity, DRM_FORMAT_MOD_LINEAR);
>>  		add_modifier(mods, &size, &capacity, DRM_FORMAT_MOD_INVALID);
>> -		return *mods ? 0 : -ENOMEM;
>> +		return 0;
>>  	}
>>  
>>  	switch (adev->family) {
>> @@ -671,9 +674,6 @@ static int get_plane_modifiers(struct amdgpu_device *adev, unsigned int plane_ty
>>  	/* INVALID marks the end of the list. */
>>  	add_modifier(mods, &size, &capacity, DRM_FORMAT_MOD_INVALID);
>>  
>> -	if (!*mods)
>> -		return -ENOMEM;
>> -
>>  	return 0;
>>  }
 
> This breaks the "size" out-parameter.

No, it will not change the value of the "size".
The "size" can only be modified by add_modifier().
However, when the "*mods" is NULL, add_modifier() will return immediately,
without the execution of "*size += 1;".
Therefore, when the "*mods" is NULL, the rest of the function is useless,
which should be better to skip.

Jiang