Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
Date:   Tue, 12 Sep 2023 14:06:03 +0100
Message-ID: <87h6nz4k3o.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     zhaoxu <zhaoxu.35@bytedance.com>
Cc:     oliver.upton@linux.dev, james.morse@arm.com,
        linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        zhouyibo@bytedance.com, zhouliang.001@bytedance.com
Subject: Re: [RFC v2] KVM: arm/arm64: optimize vSGI injection performance
In-Reply-To: <28610076-6ac7-4004-8479-e0005ff96c63@bytedance.com>
References: <20230825015811.5292-1-zhaoxu.35@bytedance.com>
        <86o7iidzwb.wl-maz@kernel.org>
        <28610076-6ac7-4004-8479-e0005ff96c63@bytedance.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2
 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk

On Tue, 12 Sep 2023 05:13:19 +0100,
zhaoxu <zhaoxu.35@bytedance.com> wrote:
> 
> 
> 
> On 2023/9/4 17:57, Marc Zyngier wrote:
> > On Fri, 25 Aug 2023 02:58:11 +0100,
> > Xu Zhao <zhaoxu.35@bytedance.com> wrote:
> [...]
> >> -	unsigned long affinity;
> >> -	int level0;
> >> +	u64 aff;
> >>   -	/*
> >> -	 * Split the current VCPU's MPIDR into affinity level 0 and the
> >> -	 * rest as this is what we have to compare against.
> >> -	 */
> >> -	affinity = kvm_vcpu_get_mpidr_aff(vcpu);
> >> -	level0 = MPIDR_AFFINITY_LEVEL(affinity, 0);
> >> -	affinity &= ~MPIDR_LEVEL_MASK;
> >> +	/* aff3 - aff1 */
> >> +	aff = (((reg) & ICC_SGI1R_AFFINITY_3_MASK) >> ICC_SGI1R_AFFINITY_3_SHIFT) << 16 |
> >> +		(((reg) & ICC_SGI1R_AFFINITY_2_MASK) >> ICC_SGI1R_AFFINITY_2_SHIFT) << 8 |
> >> +		(((reg) & ICC_SGI1R_AFFINITY_1_MASK) >> ICC_SGI1R_AFFINITY_1_SHIFT);
> > 
> > Here, you assume that you can directly map a vcpu index to an
> > affinity. It would be awesome if that was the case. However, this is
> > only valid at reset time, and userspace is perfectly allowed to change
> > this mapping by writing to the vcpu's MPIDR_EL1.
> > 
> > So this won't work at all if userspace wants to set its own specific
> > CPU numbering.
> > 
> > 	M.
> > 
> Hi Marc,
> 
> Yes, i don't think too much about userspace can change MPIDR value, I
> checked the source code of qemu, qemu create vcpu sequentially, so in
> this case, vcpu_id is equivalent to vcpu_idx which means vcpu_id
> represents the position in vcpu array.

The problem is that this is only a convention, and userspace is
totally free to use vcpu_id in a different way. Note that we have
other bugs in the KVM code that treat them interchangeably, but I'm
trying to fix that.

> These days, I'm still thinking about whether it is because of the
> content related to future vcpu hot-plug feature that vcpu_id can be
> modified, but now it seems that's not entirely the case.

There are 3 levels of identification:

- vcpu_idx, which is an internal KVM index that userspace is not
  suppose to rely on (or know)

- vcpu_id, which is provided by userspace as a CPU number, and from
  which we derive the default MPIDR_EL1 value. This is used all over
  the code to identify a CPU from userspace.

- MPIDR_EL1, which is how the architecture identifies a CPU.

For CPU hotplug, I expect usespace to set vcpu_id and MPIDR_EL1 as it
sees fit, knowing that the vpcu must have been allocated upfront (and
vcpu_idx set).

> I have read your latest patch and have been deeply inspired, and
> Thanks for agreeing with this issue.

No worries. I'd appreciate you testing it and reporting whether this
matches the results you are observing with your own patch.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.