Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935448AbdCLVSo (ORCPT ); Sun, 12 Mar 2017 17:18:44 -0400 Received: from mail-qt0-f194.google.com ([209.85.216.194]:33300 "EHLO mail-qt0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934692AbdCLVSf (ORCPT ); Sun, 12 Mar 2017 17:18:35 -0400 Date: Sun, 12 Mar 2017 17:18:31 -0400 From: "Gabriel L. Somlo" To: "Michael S. Tsirkin" Cc: Jim Mattson , LKML , Paolo Bonzini , Radim =?utf-8?B?S3LEjW3DocWZ?= , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , the arch/x86 maintainers , kvm list , linux-doc@vger.kernel.org Subject: Re: [PATCH] kvm: better MWAIT emulation for guests Message-ID: <20170312211830.GO2365@HEDWIG.INI.CMU.EDU> References: <1489098555-23856-1-git-send-email-mst@redhat.com> <20170312020015-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170312020015-mutt-send-email-mst@kernel.org> X-Clacks-Overhead: GNU Terry Pratchett User-Agent: Mutt/1.7.1 (2016-10-04) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4849 Lines: 98 On Sun, Mar 12, 2017 at 02:01:32AM +0200, Michael S. Tsirkin wrote: > On Fri, Mar 10, 2017 at 03:46:45PM -0800, Jim Mattson wrote: > > On Thu, Mar 9, 2017 at 2:29 PM, Michael S. Tsirkin wrote: > > > Some guests call mwait without checking the cpu flags. We currently > > > > "Some guests"? What guests other than Mac OS X are so ill-behaved? > > I heard about Mac OSX only but even that is hearsay for me > so I didn't want to say that explicitly. As the likely origin of said hearsay, I can confirm that Mac OS 5, 6, and 7 (Leopard through Lion) had that problem: unless explicitly provided with kernel command line argument "idlehalt=0" they'd implicitly assume MONITOR and MWAIT availability, without checking CPUID. As of MountainLion (10.8.x), that is no longer the case, and OS X gracefully falls back to an idle loop which doesn't use MONITOR/MWAIT. Regards, --Gabriel > > > emulate that as a NOP but on VMX we can do better: let guest stop the > > > CPU until timer or IPI. CPU will be busy but that isn't any worse than > > > a NOP emulation. > > > > > > Note that mwait within guests is not the same as on real hardware > > > because you must halt if you want to go deep into sleep. Thus it isn't > > > a good idea to use the regular MWAIT flag in CPUID for that. Add a flag > > > in the hypervisor leaf instead. > > > > > > Signed-off-by: Michael S. Tsirkin > > > --- > > > Documentation/virtual/kvm/cpuid.txt | 3 +++ > > > arch/x86/include/uapi/asm/kvm_para.h | 1 + > > > arch/x86/kvm/cpuid.c | 3 +++ > > > arch/x86/kvm/vmx.c | 4 ---- > > > 4 files changed, 7 insertions(+), 4 deletions(-) > > > > > > diff --git a/Documentation/virtual/kvm/cpuid.txt b/Documentation/virtual/kvm/cpuid.txt > > > index 3c65feb..5caa234 100644 > > > --- a/Documentation/virtual/kvm/cpuid.txt > > > +++ b/Documentation/virtual/kvm/cpuid.txt > > > @@ -54,6 +54,9 @@ KVM_FEATURE_PV_UNHALT || 7 || guest checks this feature bit > > > || || before enabling paravirtualized > > > || || spinlock support. > > > ------------------------------------------------------------------------------ > > > +KVM_FEATURE_MWAIT || 8 || guest can use monitor/mwait > > > + || || to halt the VCPU. > > > +------------------------------------------------------------------------------ > > > KVM_FEATURE_CLOCKSOURCE_STABLE_BIT || 24 || host will warn if no guest-side > > > || || per-cpu warps are expected in > > > || || kvmclock. > > > diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h > > > index cff0bb6..9cc77a7 100644 > > > --- a/arch/x86/include/uapi/asm/kvm_para.h > > > +++ b/arch/x86/include/uapi/asm/kvm_para.h > > > @@ -24,6 +24,7 @@ > > > #define KVM_FEATURE_STEAL_TIME 5 > > > #define KVM_FEATURE_PV_EOI 6 > > > #define KVM_FEATURE_PV_UNHALT 7 > > > +#define KVM_FEATURE_MWAIT 8 > > > > > > /* The last 8 bits are used to indicate how to interpret the flags field > > > * in pvclock structure. If no bits are set, all flags are ignored. > > > diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c > > > index efde6cc..fe3d292 100644 > > > --- a/arch/x86/kvm/cpuid.c > > > +++ b/arch/x86/kvm/cpuid.c > > > @@ -594,6 +594,9 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function, > > > if (sched_info_on()) > > > entry->eax |= (1 << KVM_FEATURE_STEAL_TIME); > > > > > > + if (this_cpu_has(X86_FEATURE_MWAIT)) > > > + entry->eax = (1 << KVM_FEATURE_MWAIT); > > > + > > > entry->ebx = 0; > > > entry->ecx = 0; > > > entry->edx = 0; > > > diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c > > > index 4bfe349..b167aba 100644 > > > --- a/arch/x86/kvm/vmx.c > > > +++ b/arch/x86/kvm/vmx.c > > > @@ -3547,13 +3547,9 @@ static __init int setup_vmcs_config(struct vmcs_config *vmcs_conf) > > > CPU_BASED_USE_IO_BITMAPS | > > > CPU_BASED_MOV_DR_EXITING | > > > CPU_BASED_USE_TSC_OFFSETING | > > > - CPU_BASED_MWAIT_EXITING | > > > - CPU_BASED_MONITOR_EXITING | > > > CPU_BASED_INVLPG_EXITING | > > > CPU_BASED_RDPMC_EXITING; > > > > > > - printk(KERN_ERR "cleared CPU_BASED_MWAIT_EXITING + CPU_BASED_MONITOR_EXITING\n"); > > > - > > > opt = CPU_BASED_TPR_SHADOW | > > > CPU_BASED_USE_MSR_BITMAPS | > > > CPU_BASED_ACTIVATE_SECONDARY_CONTROLS; > > > -- > > > MST