Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Thu, 8 Apr 2021 15:26:33 +0200
From:   Thierry Reding <thierry.reding@gmail.com>
To:     Nicolin Chen <nicoleotsuka@gmail.com>
Cc:     Dmitry Osipenko <digetx@gmail.com>, Joerg Roedel <joro@8bytes.org>,
        Jonathan Hunter <jonathanh@nvidia.com>,
        Krishna Reddy <vdumpa@nvidia.com>,
        Will Deacon <will@kernel.org>,
        iommu@lists.linux-foundation.org, linux-tegra@vger.kernel.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH v1 1/2] iommu/tegra-smmu: Defer attachment of display
 clients
Message-ID: <YG8EiUXkgPTZsfIY@orome.fritz.box>
References: <20210328233256.20494-1-digetx@gmail.com>
 <20210408094241.GA31714@Asurada-Nvidia>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
        protocol="application/pgp-signature"; boundary="rPdv4N4OEIEY1wvJ"
Content-Disposition: inline
In-Reply-To: <20210408094241.GA31714@Asurada-Nvidia>
User-Agent: Mutt/2.0.6 (98f8cb83) (2021-03-06)
Precedence: bulk


--rPdv4N4OEIEY1wvJ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Apr 08, 2021 at 02:42:42AM -0700, Nicolin Chen wrote:
> On Mon, Mar 29, 2021 at 02:32:55AM +0300, Dmitry Osipenko wrote:
> > All consumer-grade Android and Chromebook devices show a splash screen
> > on boot and then display is left enabled when kernel is booted. This
> > behaviour is unacceptable in a case of implicit IOMMU domains to which
> > devices are attached during kernel boot since devices, like display
> > controller, may perform DMA at that time. We can work around this probl=
em
> > by deferring the enable of SMMU translation for a specific devices,
> > like a display controller, until the first IOMMU mapping is created,
> > which works good enough in practice because by that time h/w is already
> > stopped.
> >=20
> > Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>=20
> For both patches:
> Acked-by: Nicolin Chen <nicoleotsuka@gmail.com>
> Tested-by: Nicolin Chen <nicoleotsuka@gmail.com>
>=20
> The WAR looks good to me. Perhaps Thierry would give some input.
>=20
> Another topic:
> I think this may help work around the mc-errors, which we have
> been facing on Tegra210 also when we enable IOMMU_DOMAIN_DMA.
> (attached a test patch rebasing on these two)

Ugh... that's exactly what I was afraid of. Now everybody is going to
think that we can just work around this issue with driver-specific SMMU
hacks...

> However, GPU would also report errors using DMA domain:
>=20
>  nouveau 57000000.gpu: acr: firmware unavailable
>  nouveau 57000000.gpu: pmu: firmware unavailable
>  nouveau 57000000.gpu: gr: firmware unavailable
>  tegra-mc 70019000.memory-controller: gpusrd: read @0x00000000fffbe200: S=
ecurity violation (TrustZone violation)
>  nouveau 57000000.gpu: DRM: failed to create kernel channel, -22
>  tegra-mc 70019000.memory-controller: gpusrd: read @0x00000000fffad000: S=
ecurity violation (TrustZone violation)
>  nouveau 57000000.gpu: fifo: SCHED_ERROR 20 []
>  nouveau 57000000.gpu: fifo: SCHED_ERROR 20 []
>=20
> Looking at the address, seems that GPU allocated memory in 32-bit
> physical address space behind SMMU, so a violation happened after
> turning on DMA domain I guess...=20

The problem with GPU is... extra complicated. You're getting these
faults because you're enabling the IOMMU-backed DMA API, which then
causes the Nouveau driver allocate buffers using the DMA API instead of
explicitly allocating pages and then mapping them using the IOMMU API.
However, there are additional patches needed to teach Nouveau about how
to deal with SMMU and those haven't been merged yet. I've got prototypes
of this, but before the whole framebuffer carveout passing work makes
progress there's little sense in moving individual pieces forward.

One more not to try and cut corners. We know what the right solution is,
even if it takes a lot of work. I'm willing to ack this patch, or some
version of it, but only as a way of working around things we have no
realistic chance of fixing properly anymore. I still think it would be
best if we could derive identity mappings from command-line arguments on
these platforms because I think most of them will actually set that, and
then the solution becomes at least uniform at the SMMU level.

For Tegra210 I've already laid out a path to a solution that's going to
be generic and extend to Tegra186 and later as well.

Thierry

--rPdv4N4OEIEY1wvJ
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEiOrDCAFJzPfAjcif3SOs138+s6EFAmBvBIYACgkQ3SOs138+
s6Hezg//VHpLpwMZYFB+XfNT7BMio9KNR0anBI58Ageu8ki5TpyavNg6gIHSeYtX
27YnoTVXPfpbqB28IWKNwUpy0mQFR8qPHAoG2i558GyS66EpB0sQxbESCe5Kr9sT
qYiTWuHS64u4tSBV/snQzmC4C3QWmt7R+wkEPScIYrSfzZpcK0tlcDxRC47b29Iv
tY0dVJXGkuHpRKaSzCjA0TwUdnwbBd1QVHUVkXzjl8wBBG6IKcLfvdbUAfEZZqbT
hhcZJ+vTmJMILLg6qjwoIkM2SGDwQchXhNSnA9BST8xl5husFIewNVXjX5w4qzp+
7tUzTIUVzwP2lKou2viUhKtgv1d/4vK7m5sTqrLGbEXgDb7LxUvgO/qzYWB77Keh
qI+Zv1ePd/RVXHuuOMM6vcLvXBFN5heXtW5CvFr+uwPbdVfz6/0QTxSAYF1pew+m
3I7u5VYgQSKdZ7uekRTppUh3d/0YZspTjpjfQtCYbdE8jnyp3dDPMVJqsZ88m7Op
kWdZ6CFCcoKY7K+lhtrs1i/p3FJqlRMc+3bjfRyYsTxJu3UThMDa6mrTDwojtBq4
jlSfMKJrbwBT+ARJQQNikRm6g9aAI/Y6SlaF1GmnhCDZm9IPo3f8zr3UcgPCAHCO
eASsZZUacPziyminANJTo8m8A0k4BtaL9esNfHOG+0WyYhI9e1E=
=d+bf
-----END PGP SIGNATURE-----

--rPdv4N4OEIEY1wvJ--