Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <1621400656-25678-1-git-send-email-guoren@kernel.org>
 <20210519052048.GA24853@lst.de> <CAJF2gTTjwB4U-NxCtfgMA5aR2HzoQtA8a51W5UM1LHGRbjz9pg@mail.gmail.com>
 <20210519064435.GA3076809@x1> <20210519065352.GA31590@lst.de>
In-Reply-To: <20210519065352.GA31590@lst.de>
From:   Guo Ren <guoren@kernel.org>
Date:   Thu, 20 May 2021 09:45:45 +0800
Message-ID: <CAJF2gTR4FXRbp7oky-ypdVJba6btFHpp-+dPyJStRaQX_-5rzg@mail.gmail.com>
Subject: Re: [PATCH RFC 0/3] riscv: Add DMA_COHERENT support
To:     Christoph Hellwig <hch@lst.de>
Cc:     Drew Fustini <drew@beagleboard.org>,
        Anup Patel <anup.patel@wdc.com>,
        Palmer Dabbelt <palmerdabbelt@google.com>, wefu@redhat.com,
        lazyparser@gmail.com,
        linux-riscv <linux-riscv@lists.infradead.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        linux-arch <linux-arch@vger.kernel.org>,
        linux-sunxi@lists.linux.dev, Guo Ren <guoren@linux.alibaba.com>,
        Paul Walmsley <paul.walmsley@sifive.com>,
        Nick Kossifidis <mick@ics.forth.gr>,
        Benjamin Koch <snowball@c3pb.de>,
        Matteo Croce <mcroce@linux.microsoft.com>,
        Wei Fu <tekkamanninja@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, May 19, 2021 at 2:53 PM Christoph Hellwig <hch@lst.de> wrote:
>
> On Tue, May 18, 2021 at 11:44:35PM -0700, Drew Fustini wrote:
> > This patch series looks like it might be useful for the StarFive JH7100
> > [1] [2] too as it has peripherals on a non-coherent interconnect. GMAC,
> > USB and SDIO require that the L2 cache must be manually flushed after
> > DMA operations if the data is intended to be shared with U74 cores [2].
>
> Not too much, given that the SiFive lineage CPUs have an uncached
> window, that is a totally different way to allocate uncached memory.
It's a very big MIPS smell. What's the attribute of the uncached
window? (uncached + strong-order/ uncached + weak, most vendors still
use AXI interconnect, how to deal with a bufferable attribute?) In
fact, customers' drivers use different ways to deal with DMA memory in
non-coherent SOC. Most riscv SOC vendors are from ARM, so giving them
the same way in DMA memory is a smart choice. So using PTE attributes
is more suitable.

See: https://github.com/riscv/virtual-memory/blob/main/specs/611-virtual-memory-diff.pdf
4.4.1
The draft supports custom attribute bits in PTE.

Although I do not agree with uncached windows, this patchset does not
conflict with SiFive uncached windows.

>
> > There is the RISC-V Cache Management Operation, or CMO, task group [3]
> > but I am not sure if that can help the SoC's that have already been
> > fabbed like the the D1 and the JH7100.
>
> It does, because unimplemented instructions trap into M-mode, where they
> can be emulated.
>
> Or to summarize things.  Non-coherent DMA (and not coherent as title in
> this series) requires two things:
>
>  1) allocating chunks of memory that is marked as not cachable
>  2) instructions to invalidate and/or writeback cache lines
Maybe sbi_ecall (dma_sync) is enough and let the vendor deal with it
in opensbi. From a hardware view, CMO instruction only could deal with
one cache line, then CMO-trap is not a good idea.

>
> none of which currently exists in RISV-V.  Hacking vendor specific
> cruft into the kernel doesn't scale, as shown perfectly by this
> series which requires to hard code vendor-specific non-standardized
> extensions in a kernel that makes it specific to that implementation.
>
> What we need to do is to standardize a way to do this properly, and then
> after that figure out a way to quirk in non-compliant implementations
> in a way that does not harm the general kernel.


-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/