Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Mon, 26 Apr 2021 19:38:37 +0000
From:   Sean Christopherson <seanjc@google.com>
To:     Reiji Watanabe <reijiw@google.com>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3 3/4] KVM: x86: Tie Intel and AMD behavior for
 MSR_TSC_AUX to guest CPU model
Message-ID: <YIcWvcneHWA9OPxv@google.com>
References: <20210423223404.3860547-1-seanjc@google.com>
 <20210423223404.3860547-4-seanjc@google.com>
 <CAAeT=FxhkRhwysd4mQa=iqEaje7R5nHew8ougtoyDEhL2sYxGA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAAeT=FxhkRhwysd4mQa=iqEaje7R5nHew8ougtoyDEhL2sYxGA@mail.gmail.com>
Precedence: bulk

On Sat, Apr 24, 2021, Reiji Watanabe wrote:
> > --- a/arch/x86/kvm/x86.c
> > +++ b/arch/x86/kvm/x86.c
> > @@ -1610,6 +1610,29 @@ static int __kvm_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data,
> >                  * invokes 64-bit SYSENTER.
> >                  */
> >                 data = get_canonical(data, vcpu_virt_addr_bits(vcpu));
> > +               break;
> > +       case MSR_TSC_AUX:
> > +               if (!kvm_cpu_cap_has(X86_FEATURE_RDTSCP))
> > +                       return 1;
> > +
> > +               if (!host_initiated &&
> > +                   !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP))
> > +                       return 1;
> > +
> > +               /*
> > +                * Per Intel's SDM, bits 63:32 are reserved, but AMD's APM has
> > +                * incomplete and conflicting architectural behavior.  Current
> > +                * AMD CPUs completely ignore bits 63:32, i.e. they aren't
> > +                * reserved and always read as zeros.  Enforce Intel's reserved
> > +                * bits check if and only if the guest CPU is Intel, and clear
> > +                * the bits in all other cases.  This ensures cross-vendor
> > +                * migration will provide consistent behavior for the guest.
> > +                */
> > +               if (guest_cpuid_is_intel(vcpu) && (data >> 32) != 0)
> > +                       return 1;
> > +
> > +               data = (u32)data;
> > +               break;
> >         }
> >
> >         msr.data = data;
> > @@ -1646,6 +1669,17 @@ int __kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data,
> >         if (!host_initiated && !kvm_msr_allowed(vcpu, index, KVM_MSR_FILTER_READ))
> >                 return KVM_MSR_RET_FILTERED;
> >
> > +       switch (index) {
> > +       case MSR_TSC_AUX:
> > +               if (!kvm_cpu_cap_has(X86_FEATURE_RDTSCP))
> > +                       return 1;
> > +
> > +               if (!host_initiated &&
> > +                   !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP))
> > +                       return 1;
> 
> 
> It looks Table 2-2 of the Intel SDM Vol4 (April 2021) says
> TSC_AUX is supported:
> 
>    If CPUID.80000001H:EDX[27] = 1 or CPUID.(EAX=7,ECX=0):ECX[22] = 1
> 
> Should we also check X86_FEATURE_RDPID before returning 1
> due to no RDTSCP support ?

Yep.  VMX should also clear RDPID if the ENABLE_RDTSCP control isn't supported.
That bug isn't fatal because KVM emulates RDPID on #UD, but it would be a
notieable performance hit for the guest.

There is also a kernel bug lurking; vgetcpu_cpu_init() doesn't check
X86_FEATURE_RDPID and will fail to initialize MSR_TSC_AUX if RDPID is supported
but RDTSCP is not, and __getcpu() uses RDPID.  I'll verify that's broken and
send a patch for that one too.

> There doesn't seem to be a similar description in the APM though.

AMD also documents this in Appendix E:

  CPUID Fn0000_0007_EBX_x0 Structured Extended Feature Identifiers (ECX=0)
  Bits   Field   Name
  ...
  22     RDPID   RDPID instruction and TSC_AUX MSR support.