Received-SPF: pass (google.com: domain of linux-kernel+bounces-176055-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Fri, 10 May 2024 18:35:48 +0100
From: Catalin Marinas <catalin.marinas@arm.com>
To: Steven Price <steven.price@arm.com>
Cc: kvm@vger.kernel.org, kvmarm@lists.linux.dev,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	Marc Zyngier <maz@kernel.org>, Will Deacon <will@kernel.org>,
	James Morse <james.morse@arm.com>,
	Oliver Upton <oliver.upton@linux.dev>,
	Zenghui Yu <yuzenghui@huawei.com>,
	linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
	Joey Gouly <joey.gouly@arm.com>,
	Alexandru Elisei <alexandru.elisei@arm.com>,
	Christoffer Dall <christoffer.dall@arm.com>,
	Fuad Tabba <tabba@google.com>, linux-coco@lists.linux.dev,
	Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com>
Subject: Re: [PATCH v2 02/14] arm64: Detect if in a realm and set RIPAS RAM
Message-ID: <Zj5a9Kt6r7U9WN5E@arm.com>
References: <20240412084213.1733764-1-steven.price@arm.com>
 <20240412084213.1733764-3-steven.price@arm.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240412084213.1733764-3-steven.price@arm.com>

On Fri, Apr 12, 2024 at 09:42:01AM +0100, Steven Price wrote:
> diff --git a/arch/arm64/include/asm/rsi.h b/arch/arm64/include/asm/rsi.h
> new file mode 100644
> index 000000000000..3b56aac5dc43
> --- /dev/null
> +++ b/arch/arm64/include/asm/rsi.h
> @@ -0,0 +1,46 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2023 ARM Ltd.

You may want to update the year ;).

> + */
> +
> +#ifndef __ASM_RSI_H_
> +#define __ASM_RSI_H_
> +
> +#include <linux/jump_label.h>
> +#include <asm/rsi_cmds.h>
> +
> +extern struct static_key_false rsi_present;

Nitpick: we tend to use DECLARE_STATIC_KEY_FALSE(), it pairs with
DEFINE_STATIC_KEY_FALSE().

> +void arm64_setup_memory(void);
> +
> +void __init arm64_rsi_init(void);
> +static inline bool is_realm_world(void)
> +{
> +	return static_branch_unlikely(&rsi_present);
> +}
> +
> +static inline void set_memory_range(phys_addr_t start, phys_addr_t end,
> +				    enum ripas state)
> +{
> +	unsigned long ret;
> +	phys_addr_t top;
> +
> +	while (start != end) {
> +		ret = rsi_set_addr_range_state(start, end, state, &top);
> +		BUG_ON(ret);
> +		BUG_ON(top < start);
> +		BUG_ON(top > end);

Are these always fatal? BUG_ON() is frowned upon in general. The
alternative would be returning an error code from the function and maybe
printing a warning here (it seems that some people don't like WARN_ON
either but it's better than BUG_ON; could use a pr_err() instead). Also
if something's wrong with the RSI interface to mess up the return
values, it will be hard to debug just from those BUG_ON().

If there's no chance of continuing beyond the point, add a comment on
why we have a BUG_ON().

> diff --git a/arch/arm64/kernel/rsi.c b/arch/arm64/kernel/rsi.c
> new file mode 100644
> index 000000000000..1076649ac082
> --- /dev/null
> +++ b/arch/arm64/kernel/rsi.c
> @@ -0,0 +1,58 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2023 ARM Ltd.
> + */
> +
> +#include <linux/jump_label.h>
> +#include <linux/memblock.h>
> +#include <asm/rsi.h>
> +
> +DEFINE_STATIC_KEY_FALSE_RO(rsi_present);
> +EXPORT_SYMBOL(rsi_present);

Does this need to be made available to loadable modules?

> +
> +static bool rsi_version_matches(void)
> +{
> +	unsigned long ver;
> +	unsigned long ret = rsi_get_version(RSI_ABI_VERSION, &ver, NULL);

I wonder whether rsi_get_version() is the right name (I know it was
introduced in the previous patch but the usage is here, hence my
comment). From the RMM spec, this looks more like an
rsi_request_version() to me.

TBH, the RMM spec around versioning doesn't fully make sense to me ;). I
assume people working on it had some good reasons around the lower
revision reporting in case of an error.

> +
> +	if (ret == SMCCC_RET_NOT_SUPPORTED)
> +		return false;
> +
> +	if (ver != RSI_ABI_VERSION) {
> +		pr_err("RME: RSI version %lu.%lu not supported\n",
> +		       RSI_ABI_VERSION_GET_MAJOR(ver),
> +		       RSI_ABI_VERSION_GET_MINOR(ver));
> +		return false;
> +	}

The above check matches what the spec says but wouldn't it be clearer to
just check for ret == RSI_SUCCESS? It saves one having to read the spec
to figure out what lower revision actually means in the spec (not the
actual lowest supported but the highest while still lower than the
requested one _or_ equal to the higher revision if the lower is higher
than the requested one - if any of this makes sense to people ;), I'm
sure I missed some other combinations).

> +
> +	pr_info("RME: Using RSI version %lu.%lu\n",
> +		RSI_ABI_VERSION_GET_MAJOR(ver),
> +		RSI_ABI_VERSION_GET_MINOR(ver));
> +
> +	return true;
> +}
> +
> +void arm64_setup_memory(void)

I would give this function a better name, something to resemble the RSI
setup. Similarly for others like set_memory_range_protected/shared().
Some of the functions have 'rsi' in the name like arm64_rsi_init() but
others don't and at a first look they'd seem like some generic memory
setup on arm64, not RSI-specific.

> +{
> +	u64 i;
> +	phys_addr_t start, end;
> +
> +	if (!static_branch_unlikely(&rsi_present))
> +		return;

We have an accessor for rsi_present - is_realm_world(). Why not use
that?

> +
> +	/*
> +	 * Iterate over the available memory ranges
> +	 * and convert the state to protected memory.
> +	 */
> +	for_each_mem_range(i, &start, &end) {
> +		set_memory_range_protected(start, end);
> +	}
> +}
> +
> +void __init arm64_rsi_init(void)
> +{
> +	if (!rsi_version_matches())
> +		return;
> +
> +	static_branch_enable(&rsi_present);
> +}
> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
> index 65a052bf741f..a4bd97e74704 100644
> --- a/arch/arm64/kernel/setup.c
> +++ b/arch/arm64/kernel/setup.c
> @@ -43,6 +43,7 @@
>  #include <asm/cpu_ops.h>
>  #include <asm/kasan.h>
>  #include <asm/numa.h>
> +#include <asm/rsi.h>
>  #include <asm/scs.h>
>  #include <asm/sections.h>
>  #include <asm/setup.h>
> @@ -293,6 +294,8 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
>  	 * cpufeature code and early parameters.
>  	 */
>  	jump_label_init();
> +	/* Init RSI after jump_labels are active */
> +	arm64_rsi_init();
>  	parse_early_param();

Does it need to be this early? It's fine for now but I wonder whether we
may have some early parameter at some point that could influence what we
do in the arm64_rsi_init(). I'd move it after or maybe even as part of
the arm64_setup_memory(), though I haven't read the following patches if
they update this function.

>  
>  	dynamic_scs_init();
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index 03efd86dce0a..786fd6ce5f17 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -40,6 +40,7 @@
>  #include <asm/kvm_host.h>
>  #include <asm/memory.h>
>  #include <asm/numa.h>
> +#include <asm/rsi.h>
>  #include <asm/sections.h>
>  #include <asm/setup.h>
>  #include <linux/sizes.h>
> @@ -313,6 +314,7 @@ void __init arm64_memblock_init(void)
>  	early_init_fdt_scan_reserved_mem();
>  
>  	high_memory = __va(memblock_end_of_DRAM() - 1) + 1;
> +	arm64_setup_memory();
>  }

This feels like a random placement. This function is about memblock
initialisation. You might as well put it in paging_init(), it could make
more sense there. But I'd rather keep it in setup_arch() immediately
after arm64_memblock_init().

-- 
Catalin