Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <c61ce1a6393a108c76e53cb99249aba5ab318e07.camel@redhat.com>
Subject: Re: [RFC PATCH 1/2] KVM: x86: update APIC_ID also when disabling
 x2APIC in kvm_lapic_set_base
From:   Maxim Levitsky <mlevitsk@redhat.com>
To:     Emanuele Giuseppe Esposito <eesposit@redhat.com>,
        kvm@vger.kernel.org
Cc:     Sean Christopherson <seanjc@google.com>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>, Shuah Khan <shuah@kernel.org>,
        Gautam Menghani <gautammenghani201@gmail.com>,
        Zeng Guang <guang.zeng@intel.com>,
        Krish Sadhukhan <krish.sadhukhan@oracle.com>,
        Jim Mattson <jmattson@google.com>,
        linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org
Date:   Mon, 09 Jan 2023 15:37:50 +0200
In-Reply-To: <20230109130605.2013555-2-eesposit@redhat.com>
References: <20230109130605.2013555-1-eesposit@redhat.com>
         <20230109130605.2013555-2-eesposit@redhat.com>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.36.5 (3.36.5-2.fc32) 
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Mon, 2023-01-09 at 08:06 -0500, Emanuele Giuseppe Esposito wrote:
> If KVM_SET_MSR firstly enables and then disables x2APIC, make sure
> APIC_ID is actually updated correctly, since bits and offset differ from
> xAPIC and x2APIC.
> 
> Currently this is not handled correctly, as kvm_set_apic_base() will
> have msr_info->host_initiated, so switching from x2APIC to xAPIC won't
> fail, but kvm_lapic_set_base() does not handle the case.
> 
> Fixes: 8d860bbeedef ("kvm: vmx: Basic APIC virtualization controls have three settings")
> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> ---
>  arch/x86/kvm/lapic.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
> index 4efdb4a4d72c..df0a50099aa2 100644
> --- a/arch/x86/kvm/lapic.c
> +++ b/arch/x86/kvm/lapic.c
> @@ -2394,8 +2394,12 @@ void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value)
>  		}
>  	}
>  
> -	if (((old_value ^ value) & X2APIC_ENABLE) && (value & X2APIC_ENABLE))
> -		kvm_apic_set_x2apic_id(apic, vcpu->vcpu_id);
> +	if ((old_value ^ value) & X2APIC_ENABLE) {
> +		if (value & X2APIC_ENABLE)
> +			kvm_apic_set_x2apic_id(apic, vcpu->vcpu_id);
> +		else
> +			kvm_apic_set_xapic_id(apic, vcpu->vcpu_id);
> +	}
>  
>  	if ((old_value ^ value) & (MSR_IA32_APICBASE_ENABLE | X2APIC_ENABLE)) {
>  		kvm_vcpu_update_apicv(vcpu);


I don't think that this patch is 100% needed in a strict sense, but I don't object to it either.
 
The switch between x2apic and xapic mode is not allowed by X86 spec, while vise versa 
is allowed and I think that the spec says that in this case APIC ID is restored to its
default value.
 
When QEMU does x2apic->xapic switch anyway to reset the vCPU, it should both upload the IA32_APIC_BASE
and all of the apic registers via KVM_SET_LAPIC, and it looks like that is what Qemu does:
 
 
static void kvm_apic_put(CPUState *cs, run_on_cpu_data data)
{
    ...
 
 
    // this calls KVM_SET_MSRS with  MSR_IA32_APICBASE, and APIC might be with wrong apic id after this
    kvm_put_apicbase(s->cpu, s->apicbase);
 
    // this just initializes the kapic with apic state to upload
    kvm_put_apic_state(s, &kapic);
 
    // and this uploads the apic state, including the APIC ID register
 
    ret = kvm_vcpu_ioctl(CPU(s->cpu), KVM_SET_LAPIC, &kapic);
    if (ret < 0) {
        fprintf(stderr, "KVM_SET_LAPIC failed: %s\n", strerror(-ret));
        abort();
    }
}
 
SO between KVM_SET_MSRS and  KVM_SET_LAPIC, apic id is indeed != vcpu_id,
which might inhibit APICv/AVIC but after recent Sean's patch series,
the inhibition is now reversible.
 
 
In fact I think it won't hurt to make the APICV_INHIBIT_REASON_APIC_BASE_MODIFIED inhibition
reversible as well.
 
Best regards,
	Maxim Levitsky