Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753324AbdCPRms (ORCPT ); Thu, 16 Mar 2017 13:42:48 -0400 Received: from mail-qk0-f193.google.com ([209.85.220.193]:36153 "EHLO mail-qk0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752464AbdCPRly (ORCPT ); Thu, 16 Mar 2017 13:41:54 -0400 Date: Thu, 16 Mar 2017 13:41:50 -0400 From: "Gabriel L. Somlo" To: "Michael S. Tsirkin" Cc: Radim =?utf-8?B?S3LEjW3DocWZ?= , linux-kernel@vger.kernel.org, Paolo Bonzini , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , x86@kernel.org, Joerg Roedel , kvm@vger.kernel.org, linux-doc@vger.kernel.org Subject: Re: [PATCH v5 untested] kvm: better MWAIT emulation for guests Message-ID: <20170316174149.GK4085@HEDWIG.INI.CMU.EDU> References: <1489612895-12799-1-git-send-email-mst@redhat.com> <20170315233534.GG2239@HEDWIG.INI.CMU.EDU> <20170316013903-mutt-send-email-mst@kernel.org> <20170316132426.GB4085@HEDWIG.INI.CMU.EDU> <20170316155550-mutt-send-email-mst@kernel.org> <20170316145819.GC4085@HEDWIG.INI.CMU.EDU> <20170316153517.GL14081@potion> <20170316160157.GN14081@potion> <20170316164749.GG4085@HEDWIG.INI.CMU.EDU> <20170316192622-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20170316192622-mutt-send-email-mst@kernel.org> X-Clacks-Overhead: GNU Terry Pratchett User-Agent: Mutt/1.7.1 (2016-10-04) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 5632 Lines: 155 On Thu, Mar 16, 2017 at 07:27:34PM +0200, Michael S. Tsirkin wrote: > On Thu, Mar 16, 2017 at 12:47:50PM -0400, Gabriel L. Somlo wrote: > > On Thu, Mar 16, 2017 at 05:01:58PM +0100, Radim Krčmář wrote: > > > 2017-03-16 16:35+0100, Radim Krčmář: > > > > 2017-03-16 10:58-0400, Gabriel L. Somlo: > > > >> The intel manual said the same thing back in 2010 as well. However, > > > >> regardless of how any flags were set, interrupt-window exiting or not, > > > >> "normal" L1 MWAIT behavior was that it woke up immediately regardless. > > > >> Remember, never going to sleep is still correct ("normal" ?) behavior > > > >> per the ISA definition of MWAIT :) > > > > > > > > I'll write a simple kvm-unit-test to better understand why it is broken > > > > for you ... > > > > > > Please get git://git.kernel.org/pub/scm/virt/kvm/kvm-unit-tests.git > > > > > > and try this, thanks! > > > > > > ---8<--- > > > x86/mwait: crappy test > > > > > > `./configure && make` to build it, then follow the comment in code to > > > try few cases. > > > > kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '0 1 1' > > timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 0 1 1 > > enabling apic > > PASS: resumed from mwait 10000 times > > SUMMARY: 1 tests > > > > real 0m10.564s > > user 0m10.339s > > sys 0m0.225s > > > > > > and > > > > kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '0 1 0' > > timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 0 1 0 > > enabling apic > > PASS: resumed from mwait 10000 times > > SUMMARY: 1 tests > > > > real 0m0.746s > > user 0m0.555s > > sys 0m0.200s > > > > Both of these with Michael's v5 patch applied, on the MacPro1,1. > > Would it make sense to try to set ECX to 0? 0 0 1 and 0 0 0. $ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '0 0 1' timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 0 0 1 enabling apic PASS: resumed from mwait 10000 times SUMMARY: 1 tests real 0m10.567s user 0m10.367s sys 0m0.210s $ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '0 0 0' timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 0 0 0 enabling apic PASS: resumed from mwait 10000 times SUMMARY: 1 tests real 0m10.549s user 0m10.352s sys 0m0.206s Both took 10 seconds. > > > Similar behavior (0 1 1 takes 10 seconds, 0 1 0 returns immediately) > > on the macbook air. > > > > If I revert to the original (nop-emulated MWAIT) kvm source, I get > > both versions to return immediately. > > > > HTH, > > --Gabriel > > > > > > > > > > > > --- > > > x86/Makefile.common | 1 + > > > x86/mwait.c | 41 +++++++++++++++++++++++++++++++++++++++++ > > > 2 files changed, 42 insertions(+) > > > create mode 100644 x86/mwait.c > > > > > > diff --git a/x86/Makefile.common b/x86/Makefile.common > > > index 1dad18ba26e1..1e708a6acd39 100644 > > > --- a/x86/Makefile.common > > > +++ b/x86/Makefile.common > > > @@ -46,6 +46,7 @@ tests-common = $(TEST_DIR)/vmexit.flat $(TEST_DIR)/tsc.flat \ > > > $(TEST_DIR)/tsc_adjust.flat $(TEST_DIR)/asyncpf.flat \ > > > $(TEST_DIR)/init.flat $(TEST_DIR)/smap.flat \ > > > $(TEST_DIR)/hyperv_synic.flat $(TEST_DIR)/hyperv_stimer.flat \ > > > + $(TEST_DIR)/mwait.flat \ > > > > > > ifdef API > > > tests-common += api/api-sample > > > diff --git a/x86/mwait.c b/x86/mwait.c > > > new file mode 100644 > > > index 000000000000..c21dab5cc97d > > > --- /dev/null > > > +++ b/x86/mwait.c > > > @@ -0,0 +1,41 @@ > > > +#include "vm.h" > > > + > > > +#define TARGET_RESUMES 10000 > > > +volatile unsigned page[4096 / 4]; > > > + > > > +/* > > > + * Execute > > > + * time TIMEOUT=20 ./x86-run x86/mwait.flat -append '0 1 1' > > > + * (first two arguments are eax and ecx for MWAIT, the third is FLAGS.IF bit) > > > + * I assume you have 1000 Hz scheduler, so the test should take about 10 > > > + * seconds to run if mwait works (host timer interrupts will kick mwait). > > > + * > > > + * If you get far less, then mwait is just nop, as in the case of > > > + * > > > + * time TIMEOUT=20 ./x86-run x86/mwait.flat -append '0 1 0' > > > + * > > > + * All other combinations of arguments should take 10 seconds. > > > + * Getting killed by the TIMEOUT most likely means that you have different HZ, > > > + * but could also be a bug ... > > > + */ > > > +int main(int argc, char **argv) > > > +{ > > > + uint32_t eax = atol(argv[1]); > > > + uint32_t ecx = atol(argv[2]); > > > + bool sti = atol(argv[3]); > > > + unsigned resumes = 0; > > > + > > > + if (sti) > > > + asm volatile ("sti"); > > > + else > > > + asm volatile ("cli"); > > > + > > > + while (resumes < TARGET_RESUMES) { > > > + asm volatile("monitor" :: "a" (page), "c" (0), "d" (0)); > > > + asm volatile("mwait" :: "a" (eax), "c" (ecx)); > > > + resumes++; > > > + } > > > + > > > + report("resumed from mwait %u times", resumes == TARGET_RESUMES, resumes); > > > + return report_summary(); > > > +} > > > -- > > > 2.11.0 > > >