DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 280F264A46
Date: Thu, 16 Mar 2017 19:27:34 +0200
From: "Michael S. Tsirkin" <mst@redhat.com>
To: "Gabriel L. Somlo" <gsomlo@gmail.com>
Cc: Radim =?utf-8?B?S3LEjW3DocWZ?= <rkrcmar@redhat.com>,
        linux-kernel@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>,
        Jonathan Corbet <corbet@lwn.net>, Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
        x86@kernel.org, Joerg Roedel <joro@8bytes.org>, kvm@vger.kernel.org,
        linux-doc@vger.kernel.org
Subject: Re: [PATCH v5 untested] kvm: better MWAIT emulation for guests
Message-ID: <20170316192622-mutt-send-email-mst@kernel.org>
References: <1489612895-12799-1-git-send-email-mst@redhat.com>
 <20170315233534.GG2239@HEDWIG.INI.CMU.EDU>
 <20170316013903-mutt-send-email-mst@kernel.org>
 <20170316132426.GB4085@HEDWIG.INI.CMU.EDU>
 <20170316155550-mutt-send-email-mst@kernel.org>
 <20170316145819.GC4085@HEDWIG.INI.CMU.EDU>
 <20170316153517.GL14081@potion>
 <20170316160157.GN14081@potion>
 <20170316164749.GG4085@HEDWIG.INI.CMU.EDU>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20170316164749.GG4085@HEDWIG.INI.CMU.EDU>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 4497
Lines: 131

On Thu, Mar 16, 2017 at 12:47:50PM -0400, Gabriel L. Somlo wrote:
> On Thu, Mar 16, 2017 at 05:01:58PM +0100, Radim Krčmář wrote:
> > 2017-03-16 16:35+0100, Radim Krčmář:
> > > 2017-03-16 10:58-0400, Gabriel L. Somlo:
> > >> The intel manual said the same thing back in 2010 as well. However,
> > >> regardless of how any flags were set, interrupt-window exiting or not,
> > >> "normal" L1 MWAIT behavior was that it woke up immediately regardless.
> > >> Remember, never going to sleep is still correct ("normal" ?) behavior
> > >> per the ISA definition of MWAIT :)
> > > 
> > > I'll write a simple kvm-unit-test to better understand why it is broken
> > > for you ...
> > 
> > Please get git://git.kernel.org/pub/scm/virt/kvm/kvm-unit-tests.git
> > 
> > and try this, thanks!
> > 
> > ---8<---
> > x86/mwait: crappy test
> > 
> > `./configure && make` to build it, then follow the comment in code to
> > try few cases.
> 
> kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '0 1 1'
> timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 0 1 1
> enabling apic
> PASS: resumed from mwait 10000 times
> SUMMARY: 1 tests
> 
> real    0m10.564s
> user    0m10.339s
> sys     0m0.225s
> 
> 
> and
> 
> kvm-unit-tests]$ time TIMEOUT=20 ./x86-run x86/mwait.flat -append '0 1 0'
> timeout -k 1s --foreground 20 qemu-kvm -nodefaults -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -kernel x86/mwait.flat -append 0 1 0
> enabling apic
> PASS: resumed from mwait 10000 times
> SUMMARY: 1 tests
> 
> real    0m0.746s
> user    0m0.555s
> sys     0m0.200s
> 
> Both of these with Michael's v5 patch applied, on the MacPro1,1.

Would it make sense to try to set ECX to 0? 0 0 1 and 0 0 0.


> Similar behavior (0 1 1 takes 10 seconds, 0 1 0 returns immediately)
> on the macbook air.
> 
> If I revert to the original (nop-emulated MWAIT) kvm source, I get
> both versions to return immediately.
> 
> HTH,
> --Gabriel
> 
> 
> 
> > 
> > ---
> >  x86/Makefile.common |  1 +
> >  x86/mwait.c         | 41 +++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 42 insertions(+)
> >  create mode 100644 x86/mwait.c
> > 
> > diff --git a/x86/Makefile.common b/x86/Makefile.common
> > index 1dad18ba26e1..1e708a6acd39 100644
> > --- a/x86/Makefile.common
> > +++ b/x86/Makefile.common
> > @@ -46,6 +46,7 @@ tests-common = $(TEST_DIR)/vmexit.flat $(TEST_DIR)/tsc.flat \
> >                 $(TEST_DIR)/tsc_adjust.flat $(TEST_DIR)/asyncpf.flat \
> >                 $(TEST_DIR)/init.flat $(TEST_DIR)/smap.flat \
> >                 $(TEST_DIR)/hyperv_synic.flat $(TEST_DIR)/hyperv_stimer.flat \
> > +               $(TEST_DIR)/mwait.flat \
> >  
> >  ifdef API
> >  tests-common += api/api-sample
> > diff --git a/x86/mwait.c b/x86/mwait.c
> > new file mode 100644
> > index 000000000000..c21dab5cc97d
> > --- /dev/null
> > +++ b/x86/mwait.c
> > @@ -0,0 +1,41 @@
> > +#include "vm.h"
> > +
> > +#define TARGET_RESUMES 10000
> > +volatile unsigned page[4096 / 4];
> > +
> > +/*
> > + * Execute
> > + *   time TIMEOUT=20 ./x86-run x86/mwait.flat -append '0 1 1'
> > + * (first two arguments are eax and ecx for MWAIT, the third is FLAGS.IF bit)
> > + * I assume you have 1000 Hz scheduler, so the test should take about 10
> > + * seconds to run if mwait works (host timer interrupts will kick mwait).
> > + *
> > + * If you get far less, then mwait is just nop, as in the case of
> > + *
> > + *   time TIMEOUT=20 ./x86-run x86/mwait.flat -append '0 1 0'
> > + *
> > + * All other combinations of arguments should take 10 seconds.
> > + * Getting killed by the TIMEOUT most likely means that you have different HZ,
> > + * but could also be a bug ...
> > + */
> > +int main(int argc, char **argv)
> > +{
> > +	uint32_t eax = atol(argv[1]);
> > +	uint32_t ecx = atol(argv[2]);
> > +	bool sti = atol(argv[3]);
> > +	unsigned resumes = 0;
> > +
> > +	if (sti)
> > +		asm volatile ("sti");
> > +	else
> > +		asm volatile ("cli");
> > +
> > +	while (resumes < TARGET_RESUMES) {
> > +		asm volatile("monitor" :: "a" (page), "c" (0), "d" (0));
> > +		asm volatile("mwait" :: "a" (eax), "c" (ecx));
> > +		resumes++;
> > +	}
> > +
> > +	report("resumed from mwait %u times", resumes == TARGET_RESUMES, resumes);
> > +	return report_summary();
> > +}
> > -- 
> > 2.11.0
> >