Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220830044642.566769-1-apatel@ventanamicro.com>
 <20220830044642.566769-4-apatel@ventanamicro.com> <13199249.VsHLxoZxqI@diego>
In-Reply-To: <13199249.VsHLxoZxqI@diego>
From:   Anup Patel <apatel@ventanamicro.com>
Date:   Sat, 3 Sep 2022 21:33:57 +0530
Message-ID: <CAK9=C2XR+3bezgSAYsr08rZq=B9VLR0Aef3icBQRi9-FNyB8kQ@mail.gmail.com>
Subject: Re: [PATCH v2 3/4] RISC-V: Implement arch specific PMEM APIs
To:     =?UTF-8?Q?Heiko_St=C3=BCbner?= <heiko@sntech.de>
Cc:     Palmer Dabbelt <palmer@dabbelt.com>,
        Paul Walmsley <paul.walmsley@sifive.com>,
        Atish Patra <atishp@atishpatra.org>,
        Anup Patel <anup@brainfault.org>,
        linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org,
        Mayuresh Chitale <mchitale@ventanamicro.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Thu, Sep 1, 2022 at 9:08 PM Heiko St=C3=BCbner <heiko@sntech.de> wrote:
>
> Hi Anup,
>
> Am Dienstag, 30. August 2022, 06:46:41 CEST schrieb Anup Patel:
> > The NVDIMM PMEM driver expects arch specific APIs for cache maintenance
> > and if arch does not provide these APIs then NVDIMM PMEM driver will
> > always use MEMREMAP_WT to map persistent memory which in-turn maps as
> > UC memory type defined by the RISC-V Svpbmt specification.
> >
> > Now that the Svpbmt and Zicbom support is available in RISC-V kernel,
> > we implement PMEM APIs using ALT_CMO_OP() macros so that the NVDIMM
> > PMEM driver can use MEMREMAP_WB to map persistent memory.
>
> Zicbom is detected at runtime, though that kconfig setting changes the
> behaviour for the memremap-type at compile-time. So what happens on
> systems not using zicbom (or another cmo-variant) ?

On a system without Zicbom (or some other cmo-variant), the PMEM read
will always work but PMEM writes will not be reliable.

Currently, the generic PMEM driver has no provision to allow arch code to
disable cacheable mapping at boot-time.

Maybe we can add WARN_ONCE() for the case when arch_xyz_pmem()
is called on a system not having Zicbom ?

Regards,
Anup

>
>
> Heiko
>
> > Co-developed-by: Mayuresh Chitale <mchitale@ventanamicro.com>
> > Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com>
> > Signed-off-by: Anup Patel <apatel@ventanamicro.com>
> > ---
> >  arch/riscv/Kconfig     |  1 +
> >  arch/riscv/mm/Makefile |  1 +
> >  arch/riscv/mm/pmem.c   | 21 +++++++++++++++++++++
> >  3 files changed, 23 insertions(+)
> >  create mode 100644 arch/riscv/mm/pmem.c
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 0ebd8da388d8..37d6370d29c3 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -25,6 +25,7 @@ config RISCV
> >       select ARCH_HAS_GIGANTIC_PAGE
> >       select ARCH_HAS_KCOV
> >       select ARCH_HAS_MMIOWB
> > +     select ARCH_HAS_PMEM_API
> >       select ARCH_HAS_PTE_SPECIAL
> >       select ARCH_HAS_SET_DIRECT_MAP if MMU
> >       select ARCH_HAS_SET_MEMORY if MMU
> > diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile
> > index d76aabf4b94d..3b368e547f83 100644
> > --- a/arch/riscv/mm/Makefile
> > +++ b/arch/riscv/mm/Makefile
> > @@ -31,3 +31,4 @@ endif
> >
> >  obj-$(CONFIG_DEBUG_VIRTUAL) +=3D physaddr.o
> >  obj-$(CONFIG_RISCV_DMA_NONCOHERENT) +=3D dma-noncoherent.o
> > +obj-$(CONFIG_ARCH_HAS_PMEM_API) +=3D pmem.o
> > diff --git a/arch/riscv/mm/pmem.c b/arch/riscv/mm/pmem.c
> > new file mode 100644
> > index 000000000000..089df92ae876
> > --- /dev/null
> > +++ b/arch/riscv/mm/pmem.c
> > @@ -0,0 +1,21 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (C) 2022 Ventana Micro Systems Inc.
> > + */
> > +
> > +#include <linux/export.h>
> > +#include <linux/libnvdimm.h>
> > +
> > +#include <asm/cacheflush.h>
> > +
> > +void arch_wb_cache_pmem(void *addr, size_t size)
> > +{
> > +     ALT_CMO_OP(clean, addr, size, riscv_cbom_block_size);
> > +}
> > +EXPORT_SYMBOL_GPL(arch_wb_cache_pmem);
> > +
> > +void arch_invalidate_pmem(void *addr, size_t size)
> > +{
> > +     ALT_CMO_OP(inval, addr, size, riscv_cbom_block_size);
> > +}
> > +EXPORT_SYMBOL_GPL(arch_invalidate_pmem);
> >
>
>
>
>