Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933768AbdCUWvc (ORCPT ); Tue, 21 Mar 2017 18:51:32 -0400 Received: from mail-qk0-f193.google.com ([209.85.220.193]:33520 "EHLO mail-qk0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933153AbdCUWv1 (ORCPT ); Tue, 21 Mar 2017 18:51:27 -0400 Date: Tue, 21 Mar 2017 18:51:17 -0400 From: Gabriel Somlo To: Radim =?utf-8?B?S3LEjW3DocWZ?= Cc: Nadav Amit , "Michael S. Tsirkin" , LKML , Paolo Bonzini , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , x86@kernel.org, Joerg Roedel , kvm@vger.kernel.org, linux-doc@vger.kernel.org Subject: Re: [PATCH v5 untested] kvm: better MWAIT emulation for guests Message-ID: <20170321225116.GJ2231@HEDWIG.INI.CMU.EDU> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20170321192239.GD25540@potion> X-Clacks-Overhead: GNU Terry Pratchett User-Agent: Mutt/1.8.0 (2017-02-23) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 5820 Lines: 142 On Tue, Mar 21, 2017 at 08:22:39PM +0100, Radim Krčmář wrote: > 2017-03-21 10:29-0700, Nadav Amit: > > > > > On Mar 21, 2017, at 9:58 AM, Radim Krčmář wrote: > > > > > In '-smp 2', the writing VCPU always does 10000 wakeups by writing into > > > monitored memory, but the mwaiting VCPU can be also woken up by host > > > interrupts, which might add a few exits depending on timing. > > > > > > I didn't spend much time in making the PASS/FAIL mean much, or ensuring > > > that we only get 10000 wakeups ... it is nothing to be worried about. > > > > > > Hint 240 behaves as nop even on my system, so I still don't find > > > anything insane on that machine (if OS X is exluded) ... And I get the exact same results on the MacBookAir4,2 (which exhibits no freezing or extreme sluggishness when running OS X 10.7 smp with Michael's KVM MWAIT-in-L1 patch)... > > > > From my days in Intel (10 years ago), I can say that MWAIT wakes for many > > microarchitecural events beside interrupts. > > > > Out of curiosity, aren’t you worried that on OS X the wbinvd causes an exit > > after the monitor and before the mwait? > > VM entry clears the monitoring, so it should behave just like an MWAIT > without MONITOR, which is NOP according to the spec. It does so on > modern hardware, but it definitely is a good thing to try ... > (I am worried about disabling MWAIT exits by default and it's a no-go > until we understand why OS X doesn't work.) > > Gabriel, how does testing with this change behave on the old machine? > > Thanks. > > ---8<--- > This should be the same as "wbinvd", because "wbinvd" does nothing > without non-coherent vfio. > Simply replacing "vmcall" with "wbinvd" is an option if the "vmcall" > version works as expected. > --- > diff --git a/x86/mwait.c b/x86/mwait.c > index 20f4dcbff8ae..19f988b94541 100644 > --- a/x86/mwait.c > +++ b/x86/mwait.c > @@ -54,6 +54,7 @@ int main(int argc, char **argv) > > while ((smp ? *page : resumes) < TARGET_RESUMES) { > asm volatile("monitor" :: "a" (page), "c" (0), "d" (0)); > + asm volatile("vmcall" :: "a"(-1)); > asm volatile("mwait" :: "a" (eax), "c" (ecx)); > resumes++; > } Sure thing, here's the MacPro1,1 results: [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 0' timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 0 enabling apic PASS: resumed from mwait 10000 times SUMMARY: 1 tests real 0m1.709s user 0m0.547s sys 0m0.243s [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 1' timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 1 enabling apic PASS: resumed from mwait 10000 times SUMMARY: 1 tests real 0m0.752s user 0m0.545s sys 0m0.218s [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 0' -smp 2 timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 0 -smp 2 enabling apic enabling apic FAIL: resumed from mwait 10004 times SUMMARY: 1 tests, 1 unexpected failures real 0m0.753s user 0m0.554s sys 0m0.227s [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 1' -smp 2 timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 1 -smp 2 enabling apic enabling apic PASS: resumed from mwait 10000 times SUMMARY: 1 tests real 0m0.755s user 0m0.562s sys 0m0.221s For comparison, the resuls including 'vmcall' on the MacBookAir4,2 (interesting, the results for the last test, "-append '240 1 1' -smp 2", are different): [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 0' timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 0 enabling apic PASS: resumed from mwait 10000 times SUMMARY: 1 tests real 0m0.622s user 0m0.501s sys 0m0.130s [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 1' timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 1 enabling apic PASS: resumed from mwait 10000 times SUMMARY: 1 tests real 0m0.624s user 0m0.504s sys 0m0.127s [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 0' -smp 2 timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 0 -smp 2 enabling apic enabling apic FAIL: resumed from mwait 10023 times SUMMARY: 1 tests, 1 unexpected failures real 0m0.623s user 0m0.544s sys 0m0.110s [kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '240 1 1' -smp 2 timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 240 1 1 -smp 2 enabling apic enabling apic FAIL: resumed from mwait 10006 times SUMMARY: 1 tests, 1 unexpected failures real 0m0.618s user 0m0.527s sys 0m0.121s HTH, --Gabriel