Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Sun, 04 Dec 2022 14:57:53 +0000
Message-ID: <87bkojtdvy.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Akihiko Odaki <akihiko.odaki@gmail.com>
Cc:     Akihiko Odaki <akihiko.odaki@daynix.com>,
        linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev,
        kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org,
        Mathieu Poirier <mathieu.poirier@linaro.org>,
        Oliver Upton <oliver.upton@linux.dev>,
        Suzuki K Poulose <suzuki.poulose@arm.com>,
        Alexandru Elisei <alexandru.elisei@arm.com>,
        James Morse <james.morse@arm.com>,
        Will Deacon <will@kernel.org>,
        Catalin Marinas <catalin.marinas@arm.com>,
        asahi@lists.linux.dev, Alyssa Rosenzweig <alyssa@rosenzweig.io>,
        Sven Peter <sven@svenpeter.dev>,
        Hector Martin <marcan@marcan.st>
Subject: Re: [PATCH 0/3] KVM: arm64: Handle CCSIDR associativity mismatches
In-Reply-To: <d54e7e38-cdf6-ef5d-a6e6-e30ad8a59034@gmail.com>
References: <20221201104914.28944-1-akihiko.odaki@daynix.com>
        <867czbmlh1.wl-maz@kernel.org>
        <50499ee9-33fe-4f5d-9d0a-76ceef038333@daynix.com>
        <87lenqu37t.wl-maz@kernel.org>
        <525ff263-90b3-5b12-da31-171b09f9ad1b@daynix.com>
        <87h6yeta8b.wl-maz@kernel.org>
        <d54e7e38-cdf6-ef5d-a6e6-e30ad8a59034@gmail.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1
 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Fri, 02 Dec 2022 09:55:24 +0000,
Akihiko Odaki <akihiko.odaki@gmail.com> wrote:
>=20
> On 2022/12/02 18:40, Marc Zyngier wrote:
> > On Fri, 02 Dec 2022 05:17:12 +0000,
> > Akihiko Odaki <akihiko.odaki@daynix.com> wrote:
> >>=20
> >>>> On M2 MacBook Air, I have seen no other difference in standard ID
> >>>> registers and CCSIDRs are exceptions. Perhaps Apple designed this way
> >>>> so that macOS's Hypervisor can freely migrate vCPU, but I can't assu=
re
> >>>> that without more analysis. This is still enough to migrate vCPU
> >>>> running Linux at least.
> >>>=20
> >>> I guess that MacOS hides more of the underlying HW than KVM does. And
> >>> KVM definitely doesn't hide the MIDR_EL1 registers, which *are*
> >>> different between the two clusters.
> >>=20
> >> It seems KVM stores a MIDR value of a CPU and reuse it as "invariant"
> >> value for ioctls while it exposes the MIDR value each physical CPU
> >> owns to vCPU.
> >=20
> > This only affects the VMM though, and not the guest which sees the
> > MIDR of the CPU it runs on. The problem is that at short of pinning
> > the vcpus, you don't know where they will run. So any value is fair
> > game.
>=20
> Yes, my concern is that VMM can be confused if it sees something
> different from what the guest on the vCPU sees.

Well, this has been part of the ABI for about 10 years, since Rusty
introduced this notion of invariant, so userspace is already working
around it if that's an actual issue.

This would be easily addressed though, and shouldn't result in any
issue. The following should do the trick (only lightly tested on an
M1).

Thanks,

	M.

=46rom f1caacb89eb8ae40dc38669160a2f081f87f4b15 Mon Sep 17 00:00:00 2001
From: Marc Zyngier <maz@kernel.org>
Date: Sun, 4 Dec 2022 14:22:22 +0000
Subject: [PATCH] KVM: arm64: Return MIDR_EL1 to userspace as seen on the vc=
pu
 thread

When booting, KVM sample the MIDR of the CPU it initialises on,
and keep this as the value that will forever be exposed to userspace.

However, this has nothing to do with the value that the guest will
see. On an asymetric system, this can result in userspace observing
weird things, specially if it has pinned the vcpus on a *different*
set of CPUs.

Instead, return the MIDR value for the vpcu we're currently on and
that the vcpu will observe if it has been pinned onto that CPU.

For symmetric systems, this changes nothing. For asymmetric machines,
they will observe the correct MIDR value at the point of the call.

Reported-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
 arch/arm64/kvm/sys_regs.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index f4a7c5abcbca..f6bcf8ba9b2e 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1246,6 +1246,22 @@ static int set_id_reg(struct kvm_vcpu *vcpu, const s=
truct sys_reg_desc *rd,
 	return 0;
 }
=20
+static int get_midr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
+		    u64 *val)
+{
+	*val =3D read_sysreg(midr_el1);
+	return 0;
+}
+
+static int set_midr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
+		    u64 val)
+{
+	if (val !=3D read_sysreg(midr_el1))
+		return -EINVAL;
+
+	return 0;
+}
+
 static int get_raz_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r=
d,
 		       u64 *val)
 {
@@ -1432,6 +1448,7 @@ static const struct sys_reg_desc sys_reg_descs[] =3D {
=20
 	{ SYS_DESC(SYS_DBGVCR32_EL2), NULL, reset_val, DBGVCR32_EL2, 0 },
=20
+	{ SYS_DESC(SYS_MIDR_EL1), .get_user =3D get_midr, .set_user =3D set_midr =
},
 	{ SYS_DESC(SYS_MPIDR_EL1), NULL, reset_mpidr, MPIDR_EL1 },
=20
 	/*
@@ -2609,7 +2626,6 @@ id_to_sys_reg_desc(struct kvm_vcpu *vcpu, u64 id,
 		((struct sys_reg_desc *)r)->val =3D read_sysreg(reg);	\
 	}
=20
-FUNCTION_INVARIANT(midr_el1)
 FUNCTION_INVARIANT(revidr_el1)
 FUNCTION_INVARIANT(clidr_el1)
 FUNCTION_INVARIANT(aidr_el1)
@@ -2621,7 +2637,6 @@ static void get_ctr_el0(struct kvm_vcpu *v, const str=
uct sys_reg_desc *r)
=20
 /* ->val is filled in by kvm_sys_reg_table_init() */
 static struct sys_reg_desc invariant_sys_regs[] =3D {
-	{ SYS_DESC(SYS_MIDR_EL1), NULL, get_midr_el1 },
 	{ SYS_DESC(SYS_REVIDR_EL1), NULL, get_revidr_el1 },
 	{ SYS_DESC(SYS_CLIDR_EL1), NULL, get_clidr_el1 },
 	{ SYS_DESC(SYS_AIDR_EL1), NULL, get_aidr_el1 },
--=20
2.34.1


--=20
Without deviation from the norm, progress is not possible.