Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Mon, 26 Apr 2021 19:18:28 +0000
From:   Sean Christopherson <seanjc@google.com>
To:     Vitaly Kuznetsov <vkuznets@redhat.com>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, Reiji Watanabe <reijiw@google.com>
Subject: Re: [PATCH v3 2/4] KVM: SVM: Clear MSR_TSC_AUX[63:32] on write
Message-ID: <YIcSBIeLpqQ0sDF7@google.com>
References: <20210423223404.3860547-1-seanjc@google.com>
 <20210423223404.3860547-3-seanjc@google.com>
 <87k0opfmo9.fsf@vitty.brq.redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87k0opfmo9.fsf@vitty.brq.redhat.com>
Precedence: bulk

On Mon, Apr 26, 2021, Vitaly Kuznetsov wrote:
> Sean Christopherson <seanjc@google.com> writes:
> 
> > Force clear bits 63:32 of MSR_TSC_AUX on write to emulate current AMD
> > CPUs, which completely ignore the upper 32 bits, including dropping them
> > on write.  Emulating AMD hardware will also allow migrating a vCPU from
> > AMD hardware to Intel hardware without requiring userspace to manually
> > clear the upper bits, which are reserved on Intel hardware.
> >
> > Presumably, MSR_TSC_AUX[63:32] are intended to be reserved on AMD, but
> > sadly the APM doesn't say _anything_ about those bits in the context of
> > MSR access.  The RDTSCP entry simply states that RCX contains bits 31:0
> > of the MSR, zero extended.  And even worse is that the RDPID description
> > implies that it can consume all 64 bits of the MSR:
> >
> >   RDPID reads the value of TSC_AUX MSR used by the RDTSCP instruction
> >   into the specified destination register. Normal operand size prefixes
> >   do not apply and the update is either 32 bit or 64 bit based on the
> >   current mode.
> >
> > Emulate current hardware behavior to give KVM the best odds of playing
> > nice with whatever the behavior of future AMD CPUs happens to be.
> >
> > Signed-off-by: Sean Christopherson <seanjc@google.com>
> > ---
> >  arch/x86/kvm/svm/svm.c | 11 ++++++++++-
> >  1 file changed, 10 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
> > index 9ed9c7bd7cfd..71d704f8d569 100644
> > --- a/arch/x86/kvm/svm/svm.c
> > +++ b/arch/x86/kvm/svm/svm.c
> > @@ -2904,8 +2904,17 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
> >  		 * direct_access_msrs.  Doing that would require a rdmsr in
> >  		 * svm_vcpu_put.
> >  		 */
> > -		svm->tsc_aux = data;
> >  		wrmsrl(MSR_TSC_AUX, svm->tsc_aux);
> 
> Hm, shouldn't this be 
> 
> wrmsrl(MSR_TSC_AUX, data) or wrmsrl(MSR_TSC_AUX, (u32)data)
> 
> then? As svm->tsc_aux wasn't updated yet.
> 
> > +
> > +		/*
> > +		 * Per Intel's SDM, bits 63:32 are reserved, but AMD's APM has
> > +		 * incomplete and conflicting architectural behavior.  Current
> > +		 * AMD CPUs completely ignore bits 63:32, i.e. they aren't
> > +		 * reserved and always read as zeros.  Emulate AMD CPU behavior
> > +		 * to avoid explosions if the vCPU is migrated from an AMD host
> > +		 * to an Intel host.
> > +		 */
> > +		svm->tsc_aux = (u32)data;
> 
> Actually, shouldn't we just move wrmsrl() here? Assuming we're not sure
> how (and if) upper 32 bits are going to be used, it would probably make
> sense to not write them to the actual MSR...

Argh.  I got too clever in trying to minimize the patch deltas and "broke" this
intermediate patch.  Once the user return framework is used, the result of
setting the MSR is checked before setting svm->tsc_aux, i.e. don't set the
virtual state if the real state could not be set.

I'm very tempted to add yet another patch to this mess to do:

		if (wrmsrl_safe(MSR_TSC_AUX, data))
			return 1;

		svm->tsc_aux = data;

And then this patch becomes:

		data = (u32)data;

		if (wrmsrl_safe(MSR_TSC_AUX, data))
			return 1;

		svm->tsc_aux = data;

The above will also make patch 3 cleaner as it will preserve the ordering of
truncating data and the wrmsr.

> >  		break;
> >  	case MSR_IA32_DEBUGCTLMSR:
> >  		if (!boot_cpu_has(X86_FEATURE_LBRV)) {
> 
> -- 
> Vitaly
>