Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Message-ID: <6df520272ae8874fbc06d60664f17b518d9b8c5e.camel@intel.com>
Subject: Re: [PATCH] KVM: x86: Add Intel CPUID.1F cpuid emulation support
From:   Xiaoyao Li <xiaoyao.li@intel.com>
To:     Like Xu <like.xu@linux.intel.com>,
        Sean Christopherson <sean.j.christopherson@intel.com>
Cc:     kvm@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Len Brown <lenb@kernel.org>, linux-kernel@vger.kernel.org
Date:   Fri, 26 Apr 2019 00:28:58 +0800
In-Reply-To: <5203b6ee-4a14-64b7-0237-dae2a977054d@linux.intel.com>
References: <1555915234-2536-1-git-send-email-like.xu@linux.intel.com>
         <20190424143238.GB18442@linux.intel.com>
         <bcc3a8ea-1fd0-124c-c472-7c096e0f8014@linux.intel.com>
         <c5d7dd8a51c606f38314ef4c7bb0702caeecbe7e.camel@intel.com>
         <30857e40-05b6-0f4c-d07c-919de08c90ac@linux.intel.com>
         <be509b34ba01a22845363aeb0b0f053f25c2d9c1.camel@intel.com>
         <7568a794-2e91-c84c-5b64-d33e394f0e2b@linux.intel.com>
         <20190425141944.GA31197@linux.intel.com>
         <5203b6ee-4a14-64b7-0237-dae2a977054d@linux.intel.com>
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, 2019-04-25 at 23:33 +0800, Like Xu wrote:
> On 2019/4/25 22:19, Sean Christopherson wrote:
> > On Thu, Apr 25, 2019 at 03:07:35PM +0800, Like Xu wrote:
> > > On 2019/4/25 14:30, Xiaoyao Li wrote:
> > > > > > Besides, the problem of simply using cpuid_exc(0x1f) in Sean's codes
> > > > > > is
> > > > > > that we cannot assmue the reserved bits 31:16 of ECX is always 0 for
> > > > > > the
> > > > > > future generation.
> > 
> > It doesn't matter if future CPUs use 31:16 for other things since this
> > code only cares about whether or not CPUID.1F exists.  The check
> > entry->eax >= 0x1f ensures that CPUID.1F will return zeros if the leaf
> > does *NOT* exist, ergo the check against CPUID.1F.ECX only needs to
> > look for a non-zero value.  CPUID.1F.ECX is the logical choice for
> > detecting support because it is guaranteed to be non-zero if the leaf
> > actually exists (because bits 15:8 will be non-zero).  Whether or not
> > bits 31:16 are non-zero is irrelevant.
> 
> Here is a case:
> 
> one future CPU may have "cpuid.0.eax > 0x1f" but not use multi-chip 
> packaging technology thus its CPUID.1F leaf **could** not exist to 
> expose multi-die info and it still use cpuid.0B leaf.
> 
> So the entry->eax >= 0x1f check is true and cpuid_ecx(0x1f) check is 
> true as well due to default return value. (one of my machines for 
> cpuid.1f.ecx would return 0x00000064 without cpuid.1f support).

You mean entry->eax >= 0x1f, and CPUID.1F leaf doesn't exist.
In this case, cpuid_ecx(0x1f) must be zero.

You should the descripton of CPUID instruction in SDM. It says:

   If a value entered for CPUID.EAX is less than or equal to the maximum input
   value and the leaf is not supported on that processor then 0 is returned in
   all the registers.

I can tell you why the cpuid.1f.exc return 0x00000064 in your machines.
That's the value of leaf 0x16, you can check the output of cpuid.0x0_eax, it
should be 0x00000016.

> When we only cares about whether or not CPUID.1F exists,
> we may need (cpuid_ecx(0x1f) & 0x0000ff00) != 0 or just follow what host 
> check_extended_topology_leaf() does.
> > 
> > > > > 
> > > > > It's true cause the statement in public spec is not "Reserved = 0" but
> > > > > "Bits 31 - 16: Reserved".
> > > > > 
> > > > > > 
> > > > > > In my opinion, Sean's codes is OK and much simple and clear.
> > > > > 
> > > > > If the host cpuid.0.eax is greater than 0x1f but actually it doesn't
> > > > > have multi-chip packaging technology and we may want to expose
> > > > > entry->eax to some value smaller than 0x1f but greater than 0x14, much
> > > > > effort needs to apply on Sean's code.
> > > > > 
> > > > > My improvement is good to overwrite cpuid.0.eax in future usage
> > > > > from the perspective of kvm feature setting not just from value check.
> > > > 
> > > > Alright, there is something wrong in your code that you haven't
> > > > realised.
> > > > 
> > > > When you do
> > > > 	entry->eax = min(entry->eax, (u32)(f_intel_pt ? 0x14 : 0xd));
> > > > 
> > > > it changes the entry->eax if entry->eax > 0x14. So you cannot directly
> > > > use
> > > > cpuid_ecx(0x1f). At least, you need to cache the value of entry->eax,
> > > > like:
> > > > 	
> > > > 	u32 max_leaf = entry->eax;
> > > > 	entry->eax = min(entry->eax, (u32)(f_intel_pt ? 0x14 : 0xd));
> > > > 	
> > > > 	//...leaf between 0x14 and 0x1f
> > > > 	
> > > > 	if (max_leaf >= 0x1f && (cpuid_ecx(0x1f) & 0x0000ff00))
> > > >   		entry->eax = 0x1f;
> > > 
> > > The cache value make no sense on this.
> > 
> > Xiaoyao is pointing out that by limiting entry->eax to 0x14 (or 0xd), then
> > it can't be used to detect support for CPUID.1F since KVM will have lost
> > the required information.
> 
> The point is that its existence does not really depend on cpuid.0.eax> = 
> 0x1f. My test machine (CLX-AP, two die in one package) has cpuid.0.eax = 
> 0x16 but does have CPUID.1F support (a bit strange on BIOS).

I think it must be something wrong with your CLX-AP.
I tested on my CLX-AP machine too, the cpuid.0x0_eax is 0x000000001f.

> > 
> > > > However, handling in increasing order in totally wrong. Since it's to
> > > > report the
> > > > max the leaf supported, we should handle in descending order, which is
> > > > what Sean
> > > > does.
> > > 
> > > There is no need to check "entry->eax >= 0x1f" before "setting entry->eax
> > > =
> > > 0x1f" if and only if cpuid_ecx(0x1f) meets requirements.
> > 
> > entry->eax absolutely needs to be checked, otherwise you have no idea what
> > CPUID leaf is actually being consumed.  For example, a CPU with a maximum
> > basic CPUID leaf of 0xB will report information that is indistinguishable
> > from the actual CPUID.1F, i.e. KVM will incorrectly think the CPU supports
> > CPUID.1F regardless of what heuristic is used to detect a "valid" CPUID.1F
> > if it only looks at the result of CPUID.1F.
> 
> Based on what I mentioned, just reconsider this proposal:
> 
> case 0:
>      entry->eax = min(entry->eax, (u32)(f_intel_pt ? 0x14 : 0xd));
>      if ((cpuid_ecx(0x1f) & 0x0000ff00)) != 0)
>          entry->eax = 0x1f;
> 
> It increases the cpuid.0.ecx value along with the minimized feature 
> requirement in a natural and readable way. Why don't we design a little 
> bit future ahead of time?

There are two things I want to state:

1. using cpuid_ecx(0x1f) to check the existence of leaf 1f is absolutely wrong.

Using the return value of E{A,B,C,D}X of cpuid leaf *N* to check the existence
of leaf *N* is totally wrong. We need first to ensure that cpuid.0_eax >= *N*,
that's the reason why we need cpuid.0_eax.

Specifically, about cpuid leaf 1f, in page 3-222 Vol.2A of latest SDM publish on
January 2019, the description of Input EAX = 1FH is:

   When CPUID executes with EAX set to 1FH, the processor returns information
   about extended topology enumeration data. Software must detect the presence
   of CPUID leaf 1FH by verifying (a) the highest leaf index supported by CPUID
   is >= 1FH, and (b) CPUID.1FH:EBX[15:0] reports a non-zero value.

2. Checking in descending manner is better that increasing manner.

In increasing manner, we have to check from the smallest one by one, there will
be some useless check for the smaller one. 
   
However, in descending manner, once it finds the supported maxmium one, there is
no need to check the smaller leaf. 

> > 
> > > An increasing manner helps to overwrite this value on demand in a flat
> > > code
> > > flow (easy to understand and maintain) not an if-else_if-else flow.
> > 
> > Maybe in the future the code will need to be refactored as more cases are
> > added, but for now an if-else is quite readable.  Worry about the future
> > when it happens. :-)
> > 
> 
>