Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Content-Type: text/plain;
        charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.21\))
Subject: Re: [PATCH 3/3] drm: omapdrm: Do no allocate non-scanout GEMs through
 DMM/TILER
From:   "H. Nikolaus Schaller" <hns@goldelico.com>
In-Reply-To: <F3F3E8E1-7907-46A4-A670-CAEF6C3DB083@goldelico.com>
Date:   Mon, 31 Oct 2022 08:57:46 +0100
Cc:     Tony Lindgren <tony@atomide.com>, tomba@kernel.org,
        airlied@linux.ie, daniel@ffwll.ch, dri-devel@lists.freedesktop.org,
        linux-kernel@vger.kernel.org, linux-omap@vger.kernel.org,
        merlijn@wizzup.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <A2089A8A-69D3-4825-B400-8EB382DC9955@goldelico.com>
References: <1642587791-13222-1-git-send-email-ivo.g.dimitrov.75@gmail.com>
 <1642587791-13222-4-git-send-email-ivo.g.dimitrov.75@gmail.com>
 <4B3F8E50-3472-4AED-9A77-3E265DF8C928@goldelico.com>
 <0000784a-ae89-1081-0ec7-fc77d3381545@gmail.com>
 <F3F3E8E1-7907-46A4-A670-CAEF6C3DB083@goldelico.com>
To:     Ivaylo Dimitrov <ivo.g.dimitrov.75@gmail.com>
Precedence: bulk


> Am 31.10.2022 um 08:44 schrieb H. Nikolaus Schaller =
<hns@goldelico.com>:
>=20
> Hi Ivaylo,
>=20
>> Am 31.10.2022 um 08:05 schrieb Ivaylo Dimitrov =
<ivo.g.dimitrov.75@gmail.com>:
>>=20
>> HI Nikolaus,
>>=20
>> On 31.10.22 =D0=B3. 0:08 =D1=87., H. Nikolaus Schaller wrote:
>>> Hi Ivaylo,
>>> it took a while until I found time to test newer kernels (mainline + =
Letux additions)
>>> on the OMAP5 Pyra but unfortunately I did not get screen display for =
v6.1. Even worse,
>>> the console was flooded by
>>=20
>> Could you elaborate on that - do you have anything on the display =
(during boot or dunno). Do you have simplefb enabled, so boot log to be =
visible on the display?
>=20
> No bootlog enabled but I have some printk in the panel driver. It is =
initially enabled, then disabled and enabled again. Then the issues =
start...
>=20
>> Is that wayland you are trying to run? Do you have PVR driver =
enabled? Did you try to boot vanilla kernel?
>=20
> I have tested with Debian Stretch with standard Xorg with "omap" =
driver. PVR is not enabled.

Have cross-checked: my setup uses the fbdev driver.

> And without your patch everything is fine on all kernels since =
4.something.
>=20
> Vanilla kernel can not be booted on that machine - there is not even a =
device tree...
>=20
>>=20
>>> [   39.419846] WARNING: CPU: 0 PID: 3673 at =
drivers/bus/omap_l3_noc.c:139 l3_interrupt_handler+0x23c/0x330
>>> [   39.429914] 44000000.l3-noc:L3 Custom Error: MASTER MPU TARGET =
GPMC (Idle): Data Access in Supervisor mode during Functional access
>>> ...
>>=20
>> I have no idea what that error is supposed to mean. @Tony?
>=20
> A coincidence is that the display is sometimes showing some artistic =
patterns. So maybe DMA accesses non-existing memory?
>=20
>>=20
>>> making the system unuseable.
>>> After doing some manual bisect by installing different kernel =
versions on the boot SD card,
>>> I was able to identify that it crept in between v5.18 and v5.19-rc1. =
A git bisect on this
>>> range (adding Letux patches on top of each bisect base) did reveal =
this patch as the first bad one.
>>> After reverting it seems as if I can use any v5.19 .. v6.1-rc2 =
kernel without issues.
>>> Now I wonder why this patch breaks my system?
>>=20
>> A wild guess - omap5 has some cache issues (as is visible from =
7cb0d6c17b96b8bf3c25de2dfde4fdeb9191f4c3), which lead to the above. =
Before the patch *all* access to the BO backing memory was done through =
TILER/DMM, mitigating the issue. After the patch, whoever tries to =
render to non-scanout buffer is doing it directly to the memory, causing =
the issue.
>>=20
>> Another possibility - someone assumes that memory is always linear, =
which is true when it is accessed through DMM, but it is not after the =
patch. Do you have my "drm: pvrsgx: dmabuf import - Do not assume =
scatterlist memory is contiguous" patch in your PVR driver? Maybe there =
is another driver that lacks similar patch.
>=20
> Yes, it is included.
>=20
> Best regards,
> Nikolaus
>=20
>>=20
>> Regards,
>> Ivo
>>=20
>>> BR and thanks,
>>> Nikolaus
>>>> Am 19.01.2022 um 11:23 schrieb Ivaylo Dimitrov =
<ivo.g.dimitrov.75@gmail.com>:
>>>>=20
>>>> On devices with DMM, all allocations are done through either DMM or =
TILER.
>>>> DMM/TILER being a limited resource means that such allocations will =
start
>>>> to fail before actual free memory is exhausted. What is even worse =
is that
>>>> with time DMM/TILER space gets fragmented to the point that even if =
we have
>>>> enough free DMM/TILER space and free memory, allocation fails =
because there
>>>> is no big enough free block in DMM/TILER space.
>>>>=20
>>>> Such failures can be easily observed with OMAP xorg DDX, for =
example -
>>>> starting few GUI applications (so buffers for their windows are =
allocated)
>>>> and then rotating landscape<->portrait while closing and opening =
new
>>>> windows soon results in allocation failures.
>>>>=20
>>>> Fix that by mapping buffers through DMM/TILER only when really =
needed,
>>>> like, for scanout buffers.
>>>>=20
>>>> Signed-off-by: Ivaylo Dimitrov <ivo.g.dimitrov.75@gmail.com>
>>>> ---
>>>> drivers/gpu/drm/omapdrm/omap_gem.c | 12 ++++++++----
>>>> 1 file changed, 8 insertions(+), 4 deletions(-)
>>>>=20
>>>> diff --git a/drivers/gpu/drm/omapdrm/omap_gem.c =
b/drivers/gpu/drm/omapdrm/omap_gem.c
>>>> index 41c1a6d..cf57179 100644
>>>> --- a/drivers/gpu/drm/omapdrm/omap_gem.c
>>>> +++ b/drivers/gpu/drm/omapdrm/omap_gem.c
>>>> @@ -821,10 +821,12 @@ int omap_gem_pin(struct drm_gem_object *obj, =
dma_addr_t *dma_addr)
>>>> 			if (ret)
>>>> 				goto fail;
>>>>=20
>>>> -			if (priv->has_dmm) {
>>>> -				ret =3D omap_gem_pin_tiler(obj);
>>>> -				if (ret)
>>>> -					goto fail;
>>>> +			if (omap_obj->flags & OMAP_BO_SCANOUT) {
>>>> +				if (priv->has_dmm) {
>>>> +					ret =3D omap_gem_pin_tiler(obj);
>>>> +					if (ret)
>>>> +						goto fail;
>>>> +				}
>>>> 			}
>>>> 		} else {
>>>> 			refcount_inc(&omap_obj->pin_cnt);
>>>> @@ -861,6 +863,8 @@ static void omap_gem_unpin_locked(struct =
drm_gem_object *obj)
>>>> 			kfree(omap_obj->sgt);
>>>> 			omap_obj->sgt =3D NULL;
>>>> 		}
>>>> +		if (!(omap_obj->flags & OMAP_BO_SCANOUT))
>>>> +			return;
>>>> 		if (priv->has_dmm) {
>>>> 			ret =3D tiler_unpin(omap_obj->block);
>>>> 			if (ret) {
>>>> --=20
>>>> 1.9.1
>>>>=20
>=20