Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2;
MIME-Version: 1.0
References: <20231026143122.279437-1-cleger@rivosinc.com>
In-Reply-To: <20231026143122.279437-1-cleger@rivosinc.com>
From:   Xu Lu <luxu.kernel@bytedance.com>
Date:   Thu, 7 Dec 2023 17:09:33 +0800
Message-ID: <CAPYmKFsFLQnhXRPXpcoNfO-kEyjYLXD2Hm-F-O=Yxe1WJoSL9w@mail.gmail.com>
Subject: Re: [External] [RFC PATCH 0/3] riscv: add support for SBI Supervisor
 Software Events
To:     =?UTF-8?B?Q2zDqW1lbnQgTMOpZ2Vy?= <cleger@rivosinc.com>
Cc:     Paul Walmsley <paul.walmsley@sifive.com>,
        Palmer Dabbelt <palmer@dabbelt.com>,
        Albert Ou <aou@eecs.berkeley.edu>,
        Atish Patra <atishp@atishpatra.org>,
        Anup Patel <apatel@ventanamicro.com>,
        linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org,
        Himanshu Chauhan <hchauhan@ventanamicro.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

Pardon. It seems that the code in opensbi[1] is not complete for PMU NMI no=
w.
For example, the pmu ovf irq is still delegated to supervisor mode and
thus can not really play a role as NMI. And neither the kernel nor
opensbi will inject a pmu event.

To complete the work, we think maybe 'enable_cb' and 'disable_cb'
functions can be supplied for sbi_sse_cb_ops.
When sbi_sse_enable is called to enable pmu event, the enable_cb will
be called to revoke the delegation of pmu ovf irq and enable this irq
in CSR_MIE.
When pmu ovf irq occurs, kernel traps into opensbi and opensbi injects
the event via sbi_sse_inject_event and eret back to kernel.

Please point it out if we have any misunderstanding.

By the way, how is SSE going now? We will appreciate it very much if
we can participate in some development work in kernel or opensbi and
be of any help.

Regards!

Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [1]

On Thu, Oct 26, 2023 at 10:31=E2=80=AFPM Cl=C3=A9ment L=C3=A9ger <cleger@ri=
vosinc.com> wrote:
>
> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
> to inject software events from an SBI implementation to supervisor
> software such that it preempts all other supervisor level traps and
> interrupts [1].
>
> Various events are defined and can be send asynchronously to supervisor
> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
> as platform specific events. Events can be either local (per-hart) or
> global. Events can be nested on top of each other based on priority and
> can interrupt the kernel at any time.
>
> First patch adds the SSE definitions. Second one adds support for SSE
> itself. Implementation is split between arch specific code and generic
> part (similarly to what is done for ARM SDEI). Finally, the last patch
> add support fro SSE event in the SBI PMU driver. If the SSE event is
> available from the SBI then, it will be used instead of the normal
> interrupt.
>
> Amongst the specific points that needs to be handle is the interruption
> at any point of the kernel execution and more specifically during
> exception handling. Due to the fact that the exception entry
> implementation uses the SCRATCH CSR as both the current task struct and
> as the temporary register to switch the stack and save register, it is
> difficult to reliably get the current task struct if we get interrupted
> at this specific moment. A fixup-like mechanism allows to mark the
> location of the current task struct depending on the entry level
> (user/kernel) and the location. This is then used in the SSE assembly to
> determine where is located the current task_struct.
>
> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
> handled and does not adds any overhead to existing code. Moreover, it
> provides "true" NMI-like interrupts which can interrupt the kernel at
> any time (even in exception handling). This is particularly crucial for
> RAS errors which needs to be handled as fast as possible to avoid any
> fault propagation. Additionally, SSE event handling is faster that the
> standard IRQ handling path with almost half executed instruction (700 vs
> 1590). Some complementary tests/perf measurements will be done.
>
> For testing purpose, one can use the provided SBI implementation at [3].
> This series also needs patch [4] to fix a bug in the PMU driver.
>
> Link: https://lists.riscv.org/g/tech-prs/message/515 [1]
> Link: https://lore.kernel.org/lkml/20231023082911.23242-10-luxu.kernel@by=
tedance.com/T/ [2]
> Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [3]
> Link: https://lore.kernel.org/linux-arm-kernel/20231026084010.11888-1-ale=
xghiti@rivosinc.com/ [4]
>
> ---
>
> Cl=C3=A9ment L=C3=A9ger (3):
>   riscv: add SBI SSE extension definitions
>   riscv: add support for SBI Supervisor Software Events extension
>   perf: RISC-V: add support for SSE event
>
>  arch/riscv/include/asm/asm-prototypes.h |   5 +
>  arch/riscv/include/asm/sbi.h            |  40 ++
>  arch/riscv/include/asm/sse.h            |  94 +++++
>  arch/riscv/kernel/Makefile              |   1 +
>  arch/riscv/kernel/asm-offsets.c         |  17 +
>  arch/riscv/kernel/entry.S               | 156 ++++++++
>  arch/riscv/kernel/sbi.c                 |   4 +
>  arch/riscv/kernel/sse.c                 |  97 +++++
>  arch/riscv/kernel/stacktrace.c          |  13 +
>  arch/riscv/kernel/vmlinux.lds.S         |   6 +
>  drivers/firmware/Kconfig                |  10 +
>  drivers/firmware/Makefile               |   1 +
>  drivers/firmware/riscv_sse.c            | 496 ++++++++++++++++++++++++
>  drivers/perf/riscv_pmu_sbi.c            |  51 ++-
>  include/linux/riscv_sse.h               |  56 +++
>  15 files changed, 1038 insertions(+), 9 deletions(-)
>  create mode 100644 arch/riscv/include/asm/sse.h
>  create mode 100644 arch/riscv/kernel/sse.c
>  create mode 100644 drivers/firmware/riscv_sse.c
>  create mode 100644 include/linux/riscv_sse.h
>
> --
> 2.42.0
>