Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759646Ab3DDMWA (ORCPT ); Thu, 4 Apr 2013 08:22:00 -0400 Received: from mx1.redhat.com ([209.132.183.28]:11390 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759504Ab3DDMV7 (ORCPT ); Thu, 4 Apr 2013 08:21:59 -0400 Date: Thu, 4 Apr 2013 14:21:57 +0300 From: "Michael S. Tsirkin" To: Alexander Graf Cc: Marcelo Tosatti , Gleb Natapov , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , x86@kernel.org, Xiao Guangrong , Takuya Yoshikawa , Alex Williamson , Will Deacon , Christoffer Dall , Sasha Levin , Andrew Morton , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org Subject: Re: [PATCH RFC] kvm: add PV MMIO EVENTFD Message-ID: <20130404112157.GB6467@redhat.com> References: <20130404104846.GA6227@redhat.com> <6D40C5F1-849E-4F32-B391-E1A1BFA8C59D@suse.de> <20130404110442.GA6467@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3236 Lines: 91 On Thu, Apr 04, 2013 at 02:09:53PM +0200, Alexander Graf wrote: > > On 04.04.2013, at 13:04, Michael S. Tsirkin wrote: > > > On Thu, Apr 04, 2013 at 01:57:34PM +0200, Alexander Graf wrote: > >> > >> On 04.04.2013, at 12:50, Michael S. Tsirkin wrote: > >> > >>> With KVM, MMIO is much slower than PIO, due to the need to > >>> do page walk and emulation. But with EPT, it does not have to be: we > >>> know the address from the VMCS so if the address is unique, we can look > >>> up the eventfd directly, bypassing emulation. > >>> > >>> Add an interface for userspace to specify this per-address, we can > >>> use this e.g. for virtio. > >>> > >>> The implementation adds a separate bus internally. This serves two > >>> purposes: > >>> - minimize overhead for old userspace that does not use PV MMIO > >>> - minimize disruption in other code (since we don't know the length, > >>> devices on the MMIO bus only get a valid address in write, this > >>> way we don't need to touch all devices to teach them handle > >>> an dinvalid length) > >>> > >>> At the moment, this optimization is only supported for EPT on x86 and > >>> silently ignored for NPT and MMU, so everything works correctly but > >>> slowly. > >>> > >>> TODO: NPT, MMU and non x86 architectures. > >>> > >>> The idea was suggested by Peter Anvin. Lots of thanks to Gleb for > >>> pre-review and suggestions. > >>> > >>> Signed-off-by: Michael S. Tsirkin > >> > >> This still uses page fault intercepts which are orders of magnitudes > >> slower than hypercalls. > > > > Not really. Here's a test: > > compare vmcall to portio: > > > > vmcall 1519 > > ... > > outl_to_kernel 1745 > > > > compare portio to mmio: > > > > mmio-wildcard-eventfd:pci-mem 3529 > > mmio-pv-eventfd:pci-mem 1878 > > portio-wildcard-eventfd:pci-io 1846 > > > > So not orders of magnitude. > > https://dl.dropbox.com/u/8976842/KVM%20Forum%202012/MMIO%20Tuning.pdf > > Check out page 41. Higher is better (number is number of loop cycles in a second). My test system was an AMD Istanbul based box. Wow 2x difference. The difference seems to be much smaller now. Newer hardware? Newer software? > > > >> Why don't you just create a PV MMIO hypercall > >> that the guest can use to invoke MMIO accesses towards the host based > >> on physical addresses with explicit length encodings? > >> That way you simplify and speed up all code paths, exceeding the speed > >> of PIO exits even. It should also be quite easily portable, as all > >> other platforms have hypercalls available as well. > >> > >> > >> Alex > > > > I sent such a patch, but maintainers seem reluctant to add hypercalls. > > Gleb, could you comment please? > > > > A fast way to do MMIO is probably useful in any case ... > > Yes, but at least according to my numbers optimizing anything that is not hcalls is a waste of time. > > > Alex This was the implementation: 'kvm_para: add mmio word store hypercall'. Again, I don't mind. -- MST -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/