Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <alpine.LRH.2.02.1808021242320.31834@file01.intranet.prod.int.rdu2.redhat.com>
In-Reply-To: <alpine.LRH.2.02.1808021242320.31834@file01.intranet.prod.int.rdu2.redhat.com>
From:   Matt Sealey <neko@bakuhatsu.net>
Date:   Thu, 2 Aug 2018 15:49:17 -0500
Message-ID: <CAHCPf3tFGqkYEcWNN4LaWThw_rVqT316pzLv6T7RfxwO-eZ0EA@mail.gmail.com>
Subject: Re: framebuffer corruption due to overlapping stp instructions on arm64
To:     Mikulas Patocka <mpatocka@redhat.com>
Cc:     Catalin Marinas <catalin.marinas@arm.com>,
        Russell King <linux@armlinux.org.uk>,
        Thomas Petazzoni <thomas.petazzoni@free-electrons.com>,
        Will Deacon <will.deacon@arm.com>, libc-alpha@sourceware.org,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org
Content-Type: multipart/alternative; boundary="00000000000078f650057279f2f8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

--00000000000078f650057279f2f8
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

The easiest explanation for this would be that the memory isn=E2=80=99t map=
ped
correctly. You can=E2=80=99t use PCIe memory spaces with anything other tha=
n
Device-nGnRE or stricter mappings. That=E2=80=99s just differences between =
the AMBA
and PCIe (posted/unposted) memory models.

Normal memory (cacheable or uncacheable, which Linux tends to call =E2=80=
=9Cmemory=E2=80=9D
and =E2=80=9Cwritecombine=E2=80=9D respectively) is not a good idea.

There are two options; make sure Links maps it=E2=80=99s framebuffer as Dev=
ice
memory, or the driver, or both - and make sure that only aligned accesses
happen (otherwise you=E2=80=99ll just get a synchronous exception) and ther=
e isn=E2=80=99t
a Normal memory alias.

Alternatively, tell the PCIe driver that the framebuffer is in system
memory - you can map it however you like but there=E2=80=99ll be a performa=
nce hit
if you start to use GPU acceleration, but a significant performance boost
from the PoV of the CPU. Only memory accessed from the PCIe master
interface (i.e. reads and writes generated by the card itself - telling the
GPU to pull from system memory or other DMA) can be in Normal memory and
this allows PCIe to be cache coherent with the right interconnect. The
slave port on a PCIe root complex (i.e. CPU writes) can=E2=80=99t be used w=
ith
Normal, or reorderable, and therefore your 2GB of graphics memory is going
to be slow from the point of view of the CPU.

To find the correct mapping you=E2=80=99ll need to know just how cache cohe=
rent the
PCIe RC is...

Ta,
Matt

On Thu, Aug 2, 2018 at 14:31 Mikulas Patocka <mpatocka@redhat.com> wrote:

> Hi
>
> I tried to use a PCIe graphics card on the MacchiatoBIN board and I hit a
> strange problem.
>
> When I use the links browser in graphics mode on the framebuffer, I get
> occasional pixel corruption. Links does memcpy, memset and 4-byte writes
> on the framebuffer - nothing else.
>
> I found out that the pixel corruption is caused by overlapping unaligned
> stp instructions inside memcpy. In order to avoid branching, the arm64
> memcpy implementation may write the same destination twice with different
> alignment. If I put "dmb sy" between the overlapping stp instructions, th=
e
> pixel corruption goes away.
>
> This seems like a hardware bug. Is it a known errata? Do you have any
> workarounds for it?
>
> I tried AMD card (HD 6350) and NVidia (NVS 285) and both exhibit the same
> corruption. OpenGL doesn't work (it results in artifacts on the AMD card
> and lock-up on the NVidia card), but it's quite expected if even simple
> writing to the framebuffer doesn't work.
>
> Mikulas
>

--00000000000078f650057279f2f8
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div><div dir=3D"auto">The easiest explanation for this would be that the m=
emory isn=E2=80=99t mapped correctly. You can=E2=80=99t use PCIe memory spa=
ces with anything other than Device-nGnRE or stricter mappings. That=E2=80=
=99s just differences between the AMBA and PCIe (posted/unposted) memory mo=
dels.</div></div><div dir=3D"auto"><br></div><div dir=3D"auto">Normal memor=
y (cacheable or uncacheable, which Linux tends to call =E2=80=9Cmemory=E2=
=80=9D and =E2=80=9Cwritecombine=E2=80=9D respectively) is not a good idea.=
</div><div dir=3D"auto"><br></div><div dir=3D"auto">There are two options; =
make sure Links maps it=E2=80=99s framebuffer as Device memory, or the driv=
er, or both - and make sure that only aligned accesses happen (otherwise yo=
u=E2=80=99ll just get a synchronous exception) and there isn=E2=80=99t a No=
rmal memory alias.</div><div dir=3D"auto"><br></div><div dir=3D"auto">Alter=
natively, tell the PCIe driver that the framebuffer is in system memory - y=
ou can map it however you like but there=E2=80=99ll be a performance hit if=
 you start to use GPU acceleration, but a significant performance boost fro=
m the PoV of the CPU. Only memory accessed from the PCIe master interface (=
i.e. reads and writes generated by the card itself - telling the GPU to pul=
l from system memory or other DMA) can be in Normal memory and this allows =
PCIe to be cache coherent with the right interconnect. The slave port on a =
PCIe root complex (i.e. CPU writes) can=E2=80=99t be used with Normal, or r=
eorderable, and therefore your 2GB of graphics memory is going to be slow f=
rom the point of view of the CPU.</div><div dir=3D"auto"><br></div><div dir=
=3D"auto">To find the correct mapping you=E2=80=99ll need to know just how =
cache coherent the PCIe RC is...</div><div dir=3D"auto"><br></div><div dir=
=3D"auto">Ta,</div><div dir=3D"auto">Matt</div><div dir=3D"auto"><br></div>=
<div><div class=3D"gmail_quote"><div dir=3D"ltr">On Thu, Aug 2, 2018 at 14:=
31 Mikulas Patocka &lt;<a href=3D"mailto:mpatocka@redhat.com">mpatocka@redh=
at.com</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"m=
argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi<br>
<br>
I tried to use a PCIe graphics card on the MacchiatoBIN board and I hit a <=
br>
strange problem.<br>
<br>
When I use the links browser in graphics mode on the framebuffer, I get <br=
>
occasional pixel corruption. Links does memcpy, memset and 4-byte writes <b=
r>
on the framebuffer - nothing else.<br>
<br>
I found out that the pixel corruption is caused by overlapping unaligned <b=
r>
stp instructions inside memcpy. In order to avoid branching, the arm64 <br>
memcpy implementation may write the same destination twice with different <=
br>
alignment. If I put &quot;dmb sy&quot; between the overlapping stp instruct=
ions, the <br>
pixel corruption goes away.<br>
<br>
This seems like a hardware bug. Is it a known errata? Do you have any <br>
workarounds for it?<br>
<br>
I tried AMD card (HD 6350) and NVidia (NVS 285) and both exhibit the same <=
br>
corruption. OpenGL doesn&#39;t work (it results in artifacts on the AMD car=
d <br>
and lock-up on the NVidia card), but it&#39;s quite expected if even simple=
 <br>
writing to the framebuffer doesn&#39;t work.<br>
<br>
Mikulas<br>
</blockquote></div></div>

--00000000000078f650057279f2f8--