Received-SPF: pass (google.com: domain of linux-kernel+bounces-132790-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <e4089855-aca9-4993-bac6-e59370cf7954@linaro.org>
Date: Fri, 5 Apr 2024 11:05:18 +0100
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
From: Tudor Ambarus <tudor.ambarus@linaro.org>
Subject: GS101 pl330 AxCACHE
To: Arnd Bergmann <arnd@kernel.org>, =?UTF-8?B?6rmA7J6s7JuQL0pBRVdPTiBLSU0=?=
 <jaewon02.kim@samsung.com>, Vinod Koul <vkoul@kernel.org>,
 dmaengine@vger.kernel.org, bumyong.lee@samsung.com, surendran.k@samsung.com,
 l.stelmach@samsung.com, m.szyprowski@samsung.com
Cc: linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
 linux-samsung-soc@vger.kernel.org,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 =?UTF-8?Q?Andr=C3=A9_Draszik?= <andre.draszik@linaro.org>,
 Peter Griffin <peter.griffin@linaro.org>,
 William McVicker <willmcvicker@google.com>, kernel-team@android.com,
 Sam Protsenko <semen.protsenko@linaro.org>,
 Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>,
 sugar.zhang@rock-chips.com, Alim Akhtar <alim.akhtar@samsung.com>,
 Sylwester Nawrocki <s.nawrocki@samsung.com>,
 Chanwoo Choi <cw00.choi@samsung.com>, jassisinghbrar@gmail.com,
 "Simek, Michal" <michal.simek@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Hi,

I'm trying to enable the pl330 DMA controller in GS101 and I'm facing
some problems related to AxCACHE. I'm writing this with the hope that
someone (from samsung?) knows a bit more about the pl330 integration and
its AxCACHE use.

For simplicity, let's consider just the DMA_MEM_TO_MEM case for the
moment. With the current upstream pl330 driver I see that the dmatest
module complains with errors like:
  [ 6572.661927][ T6358] dmatest: dma0chan0-copy0: dstbuf[0x18e0] not
copied! Expected cf, got 3f

Checked the downstream driver and found a fix:
-       desc->rqcfg.scctl = CCTRL0;
-       desc->rqcfg.dcctl = CCTRL0;
+       desc->rqcfg.scctl = CCTRL2;
+       desc->rqcfg.dcctl = CCTRL2;

According to CoreLing DMA-330 TRM [1], the {dst,src}_cache_ctrl fields
program the state of AxCACHE.

AMBA AXI and ACE Protocol Specification [2] says that these signals
specify the memory attributes of a request.
  - CTRL0 -> AxCACHE[3:0] = 0b0000 (Device non-bufferable)
  - CTRL2 -> AxCACHE[3:0] = 0b0010 (Normal Non-cacheable Non-bufferable)

AxCACHE[1] bit is the Modifiable bit. I guess that GS101 can not meet
the requirements of Non-modifiable transactions. But why? Culprit could
be TREX, the non-coherent interconnect, which collects requests from
different masters and sends the request to cache coherent interconnect
and to System-Level Cache. But I can't pin-point the exact restriction
that comes with it. Any hints here are appreciated.

Thanks for reading up to here. Let's consider now the DMA_MEM_TO_DEV and
DMA_DEV_TO_MEM cases. I guess that for simplicity the driver author
chose to always set CCTRL0 for both src and dst regardless of the memory
type, device or normal. This is possible because normal non-cacheable
memory can be accessed by any Manager using a device memory transaction
(see "A4.5 Mismatched memory attributes" section of [2]). This can be
improved however, if we use CTRL0 (thus Device non-bufferable) for DEV,
and CTRL2 (thus Normal Non-cacheable Non-bufferable) for MEM. I tried
this suggestion on GS101 for a MEM to DEV transaction and I get
timeouts. I tried the downstream driver as well. The downstream GS101
device tree defines all the USI nodes without DMA support, I guess the
to/from DEV transactions are not sorted out yet. I enabled the DMA use
for USI in downstream and ITMON complains with "Unsupported transaction
error". Any feedback is welcomed.

Thanks!
ta

[1]
https://documentation-service.arm.com/static/5e8e25befd977155116a5ad9?token=
[2]
https://documentation-service.arm.com/static/602a9df190ee6824a1e02b98?token=