Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754406AbbFRJQf (ORCPT ); Thu, 18 Jun 2015 05:16:35 -0400 Received: from mga14.intel.com ([192.55.52.115]:48464 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754054AbbFRJQ1 (ORCPT ); Thu, 18 Jun 2015 05:16:27 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,638,1427785200"; d="scan'208";a="713158442" From: "Wu, Feng" To: Alex Williamson , Eric Auger CC: Avi Kivity , "kvm@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "pbonzini@redhat.com" , "mtosatti@redhat.com" , "Wu, Feng" Subject: RE: [v4 08/16] KVM: kvm-vfio: User API for IRQ forwarding Thread-Topic: [v4 08/16] KVM: kvm-vfio: User API for IRQ forwarding Thread-Index: AQHQp4qn2vZF+rInhU+Xj8knW4QkiJ2x7TDQ Date: Thu, 18 Jun 2015 09:16:06 +0000 Message-ID: References: <1434019912-15423-1-git-send-email-feng.wu@intel.com> <1434019912-15423-9-git-send-email-feng.wu@intel.com> <5579E884.3040500@gmail.com> <1434123695.4927.304.camel@redhat.com> <557B2994.1070900@gmail.com> <1434135815.4927.308.camel@redhat.com> <557EFA7F.9010209@linaro.org> <1434386702.4927.391.camel@redhat.com> In-Reply-To: <1434386702.4927.391.camel@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by nfs id t5I9GdC3017473 Content-Length: 5794 Lines: 115 > -----Original Message----- > From: Alex Williamson [mailto:alex.williamson@redhat.com] > Sent: Tuesday, June 16, 2015 12:45 AM > To: Eric Auger > Cc: Avi Kivity; Wu, Feng; kvm@vger.kernel.org; linux-kernel@vger.kernel.org; > pbonzini@redhat.com; mtosatti@redhat.com > Subject: Re: [v4 08/16] KVM: kvm-vfio: User API for IRQ forwarding > > On Mon, 2015-06-15 at 18:17 +0200, Eric Auger wrote: > > Hi Alex, all, > > On 06/12/2015 09:03 PM, Alex Williamson wrote: > > > On Fri, 2015-06-12 at 21:48 +0300, Avi Kivity wrote: > > >> On 06/12/2015 06:41 PM, Alex Williamson wrote: > > >>> On Fri, 2015-06-12 at 00:23 +0000, Wu, Feng wrote: > > >>>>> -----Original Message----- > > >>>>> From: Avi Kivity [mailto:avi.kivity@gmail.com] > > >>>>> Sent: Friday, June 12, 2015 3:59 AM > > >>>>> To: Wu, Feng; kvm@vger.kernel.org; linux-kernel@vger.kernel.org > > >>>>> Cc: pbonzini@redhat.com; mtosatti@redhat.com; > > >>>>> alex.williamson@redhat.com; eric.auger@linaro.org > > >>>>> Subject: Re: [v4 08/16] KVM: kvm-vfio: User API for IRQ forwarding > > >>>>> > > >>>>> On 06/11/2015 01:51 PM, Feng Wu wrote: > > >>>>>> From: Eric Auger > > >>>>>> > > >>>>>> This patch adds and documents a new KVM_DEV_VFIO_DEVICE > group > > >>>>>> and 2 device attributes: KVM_DEV_VFIO_DEVICE_FORWARD_IRQ, > > >>>>>> KVM_DEV_VFIO_DEVICE_UNFORWARD_IRQ. The purpose is to be > able > > >>>>>> to set a VFIO device IRQ as forwarded or not forwarded. > > >>>>>> the command takes as argument a handle to a new struct named > > >>>>>> kvm_vfio_dev_irq. > > >>>>> Is there no way to do this automatically? After all, vfio knows that a > > >>>>> device interrupt is forwarded to some eventfd, and kvm knows that > some > > >>>>> eventfd is forwarded to a guest interrupt. If they compare notes > > >>>>> through a central registry, they can figure out that the interrupt needs > > >>>>> to be forwarded. > > >>>> Oh, just like Eric mentioned in his reply, this description is out of context > of > > >>>> this series, I will remove them in the next version. > > >>> > > >>> I suspect Avi's question was more general. While forward/unforward is > > >>> out of context for this series, it's very similar in nature to > > >>> enabling/disabling posted interrupts. So I think the question remains > > >>> whether we really need userspace to participate in creating this > > >>> shortcut or if kvm and vfio can some how orchestrate figuring it out > > >>> automatically. > > >>> > > >>> Personally I don't know how we could do it automatically. We've always > > >>> relied on userspace to independently setup vfio and kvm such that > > >>> neither have any idea that the other is there and update each side > > >>> independently when anything changes. So it seems consistent to > continue > > >>> that here. It doesn't seem like there's much to gain performance-wise > > >>> either, updates should be a relatively rare event I'd expect. > > >>> > > >>> There's really no metadata associated with an eventfd, so "comparing > > >>> notes" automatically might imply some central registration entity. That > > >>> immediately sounds like a much more complex solution, but maybe Avi > has > > >>> some ideas to manage it. Thanks, > > >>> > > >> > > >> The idea is to have a central registry maintained by a posted interrupts > > >> manager. Both vfio and kvm pass the filp (along with extra information) > > >> to the posted interrupts manager, which, when it detects a filp match, > > >> tells each of them what to do. > > >> > > >> The advantages are: > > >> - old userspace gains the optimization without change > > >> - a userspace API is more expensive to maintain than internal kernel > > >> interfaces (CVEs, documentation, maintaining backwards compatibility) > > >> - if you can do it without a new interface, this indicates that all the > > >> information in the new interface is redundant. That means you have to > > >> check it for consistency with the existing information, so it's extra > > >> work (likely, it's exactly what the posted interrupt manager would be > > >> doing anyway). > > > > > > Yep, those all sound like good things and I believe that's similar in > > > design to the way we had originally discussed this interaction at > > > LPC/KVM Forum several years ago. I'd be in favor of that approach. > > > > I guess this discussion also is relevant wrt "[RFC v6 00/16] KVM-VFIO > > IRQ forward control" series? Or is that "central registry maintained by > > a posted interrupts manager" something more specific to x86? > > I'd think we'd want it for any sort of offload and supporting both > posted-interrupts and irq-forwarding would be a good validation. I > imagine there would be registration/de-registration callbacks separate > for interrupt producers vs interrupt consumers. Each registration > function would likely provide a struct of callbacks, probably similar to > the get_symbol callbacks proposed for the kvm-vfio device on the IRQ > producer side. The eventfd would be the token that the manager would > use to match producers and consumers. The hard part is probably > figuring out what information to retrieve from the producer and provide > to the consumer in a generic way between pci and platform, but as an > internal interface, it's not a big deal if we screw it up a few times to > start. Thanks, On posted-interrupts side, the main purpose of the new APIs is to update the IRTE when guest changes vMSI/vMSIx configuration. Alex, do you have any detailed ideas for the new solution to achieve this purpose? It should be helpful if you can share some! Thanks, Feng > > Alex ????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay?ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m???? ????????I?