Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Wed, 19 Feb 2020 13:48:51 -0800 (PST)
Subject:     Re: [PATCH v8 00/11] Add support for SBI v0.2 and CPU hotplug
In-Reply-To: <20200212014822.28684-1-atish.patra@wdc.com>
CC:     linux-kernel@vger.kernel.org, Atish Patra <Atish.Patra@wdc.com>,
        aou@eecs.berkeley.edu, allison@lohutok.net, anup@brainfault.org,
        bp@suse.de, daniel.lezcano@linaro.org, ebiederm@xmission.com,
        geert@linux-m68k.org, heiko.carstens@de.ibm.com,
        jason@lakedaemon.net, keescook@chromium.org,
        linux-riscv@lists.infradead.org, han_mao@c-sky.com,
        Mark Zyngier <maz@kernel.org>, m.szyprowski@samsung.com,
        mpe@ellerman.id.au, rppt@linux.ibm.com,
        Paul Walmsley <paul.walmsley@sifive.com>, tglx@linutronix.de,
        vincent.chen@sifive.com
From:   Palmer Dabbelt <palmer@dabbelt.com>
To:     Atish Patra <Atish.Patra@wdc.com>
Message-ID: <mhng-4031b042-7e16-4ff2-91a7-10747042e983@palmerdabbelt-glaptop1>
Mime-Version: 1.0 (MHng)
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, 11 Feb 2020 17:48:11 PST (-0800), Atish Patra wrote:
> The Supervisor Binary Interface(SBI) specification[1] now defines a
> base extension that provides extendability to add future extensions
> while maintaining backward compatibility with previous versions.
> The new version is defined as 0.2 and older version is marked as 0.1.
>
> This series adds support v0.2 and a unified calling convention
> implementation between 0.1 and 0.2. It also add other SBI v0.2
> functionality defined in [2]. The base support for SBI v0.2 is already
> available in OpenSBI v0.5. It also adds SBI HSM extension and cpu-hotplug
> support for RISC-V which requires additional patches[3] in OpenSBI.

Now that 0.2-rc1 has been tagged we should really start to get this into shape
to merge this.  My biggest worry is being able to put together a kernel that
can boot on both 0.1 and 0.2 SBIs, with the hart lottery being my major worry
there.  I just skimmed this, but I was expected to see something like 

    diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S
    index 271860fc2c3f..849ba75959ba 100644
    --- a/arch/riscv/kernel/head.S
    +++ b/arch/riscv/kernel/head.S
    @@ -187,6 +187,7 @@ relocate:
     	la a3, .Lsecondary_park
     	csrw CSR_TVEC, a3
     
    +#ifdef CONFIG_SBI_V01
     	slli a3, a0, LGREG
     	la a1, __cpu_up_stack_pointer
     	la a2, __cpu_up_task_pointer
    @@ -212,7 +213,10 @@ relocate:
     #endif
     
     	tail smp_callin
    -#endif
    +#else /* !CONFIG_SBI_V01 */
    +	pr_warn("multiple harts booted an SBI v0.2+ only kernel");
    +#endif /* CONFIG_SBI_V01 */
    +#endif /* CONFIG_SMP */
     
     END(_start)
 
but I don't.  Is there something else doing this?

> [1] https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.adoc
> [2] https://github.com/riscv/riscv-sbi-doc/pull/27
> [3] http://lists.infradead.org/pipermail/opensbi/2020-January/001050.html
>
> The patches are also available in following github repositery.
>
> OpenSBI     : https://github.com/atishp04/opensbi/tree/sbi_hsm_v1
> Linux Kernel: https://github.com/atishp04/linux/tree/sbi_v0.2_v8
>
> Changes from v7->v8:
> 1. Refactored to code to have modular cpu_ops calls.
> 2. Refactored HSM extension from sbi.c to cpu_ops_sbi.c.
> 3. Fix plic driver to handle cpu hotplug.
>
> Changes from v6->v7:
> 1. Rebased on v5.5
> 2. Fixed few compilation issues for !CONFIG_SMP and !CONFIG_RISCV_SBI
> 3. Added SBI HSM extension
> 4. Add CPU hotplug support
>
> Changes from v5->v6
> 1. Fixed few compilation issues around config.
> 2. Fixed hart mask generation issues for RFENCE & IPI extensions.
>
> Changes from v4->v5
> 1. Fixed few minor comments related to static & inline.
> 2. Make sure that every patch is boot tested individually.
>
> Changes from v3->v4.
> 1. Rebased on for-next.
> 2. Fixed issuses with checkpatch --strict.
> 3. Unfied all IPI/fence related functions.
> 4. Added Hfence related SBI calls.
>
> Changes from v2->v3.
> 1. Moved v0.1 extensions to a new config.
> 2. Added support for relacement extensions of v0.1 extensions.
>
> Changes from v1->v2
> 1. Removed the legacy calling convention.
> 2. Moved all SBI related calls to sbi.c.
> 3. Moved all SBI related macros to uapi.
>
> Atish Patra (11):
> RISC-V: Mark existing SBI as 0.1 SBI.
> RISC-V: Add basic support for SBI v0.2
> RISC-V: Add SBI v0.2 extension definitions
> RISC-V: Introduce a new config for SBI v0.1
> RISC-V: Implement new SBI v0.2 extensions
> RISC-V: Move relocate and few other functions out of __init
> RISC-V: Add cpu_ops and modify default booting method
> RISC-V: Add SBI HSM extension
> RISC-V: Add supported for ordered booting method using HSM
> irqchip/sifive-plic: Initialize the plic handler when cpu comes online
> RISC-V: Support cpu hotplug
>
> arch/riscv/Kconfig                   |  19 +-
> arch/riscv/include/asm/cpu_ops.h     |  46 +++
> arch/riscv/include/asm/sbi.h         | 194 ++++++----
> arch/riscv/include/asm/smp.h         |  24 ++
> arch/riscv/kernel/Makefile           |   6 +
> arch/riscv/kernel/cpu-hotplug.c      |  87 +++++
> arch/riscv/kernel/cpu_ops.c          |  48 +++
> arch/riscv/kernel/cpu_ops_sbi.c      | 113 ++++++
> arch/riscv/kernel/cpu_ops_spinwait.c |  42 +++
> arch/riscv/kernel/head.S             | 179 +++++----
> arch/riscv/kernel/sbi.c              | 524 ++++++++++++++++++++++++++-
> arch/riscv/kernel/setup.c            |  24 +-
> arch/riscv/kernel/smpboot.c          |  56 +--
> arch/riscv/kernel/traps.c            |   2 +-
> arch/riscv/kernel/vmlinux.lds.S      |   5 +-
> drivers/irqchip/irq-sifive-plic.c    |  34 +-
> include/linux/cpuhotplug.h           |   1 +
> 17 files changed, 1227 insertions(+), 177 deletions(-)
> create mode 100644 arch/riscv/include/asm/cpu_ops.h
> create mode 100644 arch/riscv/kernel/cpu-hotplug.c
> create mode 100644 arch/riscv/kernel/cpu_ops.c
> create mode 100644 arch/riscv/kernel/cpu_ops_sbi.c
> create mode 100644 arch/riscv/kernel/cpu_ops_spinwait.c