Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Subject: Re: [RFC PATCH] arm64: mm: Add invalidate back in
 arch_sync_dma_for_device when FROM_DEVICE
To:     Robin Murphy <robin.murphy@arm.com>,
        Ard Biesheuvel <ardb@kernel.org>
CC:     <will@kernel.org>, <catalin.marinas@arm.com>,
        <sstabellini@kernel.org>, <jgross@suse.com>,
        <oleksandr_tyshchenko@epam.com>,
        <linux-arm-kernel@lists.infradead.org>,
        <linux-kernel@vger.kernel.org>, <wangkefeng.wang@huawei.com>,
        <linux@armlinux.org.uk>
References: <20221117072100.2720512-1-sunnanyong@huawei.com>
 <CAMj1kXENCa5TK9EnRa9Mtn+xGLrnV_X0sAkTC64GVOShDsTK1Q@mail.gmail.com>
 <f41f4fb9-9459-1fa1-494f-c20033c315da@arm.com>
From:   Nanyong Sun <sunnanyong@huawei.com>
Message-ID: <e6bd415a-201a-b3e3-48f0-beeff91f3b7a@huawei.com>
Date:   Sun, 20 Nov 2022 10:58:40 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.8.1
MIME-Version: 1.0
In-Reply-To: <f41f4fb9-9459-1fa1-494f-c20033c315da@arm.com>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 2022/11/17 18:23, Robin Murphy wrote:

> On 2022-11-17 08:24, Ard Biesheuvel wrote:
>> On Thu, 17 Nov 2022 at 07:33, Nanyong Sun <sunnanyong@huawei.com> wrote:
>>>
>>> The commit c50f11c6196f ("arm64: mm: Don't invalidate FROM_DEVICE
>>> buffers at start of DMA transfer") replaces invalidate with clean
>>> when DMA_FROM_DEVICE, this changes the behavior of functions like
>>> dma_map_single() and dma_sync_single_for_device(*, *, *, 
>>> DMA_FROM_DEVICE),
>>> then it may make some drivers works unwell because the implementation
>>> of these DMA APIs lose the original cache invalidation.
>>>
>>> Situation 1:
>> ...
>>> Situation 2:
>>> After backporting the above commit, we find a network card driver go
>>> wrong with cache inconsistency when doing DMA transfer: CPU got the
>>> stale data in cache when reading DMA data received from device.
>>
>> I suppose this means those drivers may lack dma_sync_single_for_cpu()
>> calls after the inbound transfers complete, and are instead relying on
>> the cache invalidation performed before the transfer to make the DMA'd
>> data visible to the CPU.
>
> It also commonly goes wrong the other way round when the drivers are 
> correct but DT/ACPI failed to indicate a coherent device as such. If 
> writes from the device actually snoop, they hit the still-present 
> cache line, which then gets invalidated by unmap/sync_for_cpu and the 
> new data is lost.
>
> Robin.
>
> .
Is it acceptable?  Does the coherent device MUST run with coherent 
identifier? At least they can run with no wrong before.