Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752609AbbG1QzK (ORCPT ); Tue, 28 Jul 2015 12:55:10 -0400 Received: from foss.arm.com ([217.140.101.70]:60371 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751143AbbG1QzJ (ORCPT ); Tue, 28 Jul 2015 12:55:09 -0400 Message-ID: <55B7B3E8.9020000@arm.com> Date: Tue, 28 Jul 2015 17:55:04 +0100 From: Marc Zyngier Organization: ARM Ltd User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Icedove/31.7.0 MIME-Version: 1.0 To: Alex Williamson , Pranavkumar Sawargaonkar CC: "kvm@vger.kernel.org" , "kvmarm@lists.cs.columbia.edu" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "christoffer.dall@linaro.org" , Will Deacon , "bhelgaas@google.com" , "arnd@arndb.de" , "rob.herring@linaro.org" , "eric.auger@linaro.org" , "patches@apm.com" , Bharat Bhushan , Stuart Yoder Subject: Re: [RFC 0/2] VFIO: Add virtual MSI doorbell support. References: <1437728590-23126-1-git-send-email-pranavkumar@linaro.org> <1438100507.5211.170.camel@redhat.com> In-Reply-To: <1438100507.5211.170.camel@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3182 Lines: 68 Hi Alex, On 28/07/15 17:21, Alex Williamson wrote: > On Fri, 2015-07-24 at 14:33 +0530, Pranavkumar Sawargaonkar wrote: >> In current VFIO MSI/MSI-X implementation, linux host kernel >> allocates MSI/MSI-X vectors when userspace requests through vfio ioctls. >> Vfio creates irqfd mappings to notify MSI/MSI-X interrupts >> to the userspace when raised. >> Guest OS will see emulated MSI/MSI-X controller and receives an interrupt >> when kernel notifies the same via irqfd. >> >> Host kernel allocates MSI/MSI-X using standard linux routines >> like pci_enable_msix_range() and pci_enable_msi_range(). >> These routines along with requset_irq() in host kernel sets up >> MSI/MSI-X vectors with Physical MSI/MSI-X addresses provided by >> interrupt controller driver in host kernel. >> >> This means when a device is assigned with the guest OS, MSI/MSI-X addresses >> present in PCIe EP are the PAs programmed by the host linux kernel. >> >> In x86 MSI/MSI-X physical address range is reserved and iommu is aware >> about these addreses and transalation is bypassed for these address range. >> >> Unlike x86, ARM/ARM64 does not reserve MSI/MSI-X Physical address range and >> all the transactions including MSI go through iommu/smmu without bypass. >> This requires extending current vfio MSI layer with additional functionality >> for ARM/ARM64 by >> 1. Programing IOVA (referred as a MSI virtual doorbell address) >> in device's MSI vector as a MSI address. >> This IOVA will be provided by the userspace based on the >> MSI/MSI-X addresses reserved for the guest. >> 2. Create an IOMMU mapping between this IOVA and >> Physical address (PA) assigned to the MSI vector. >> >> This RFC is proposing a solution for MSI/MSI-X passthrough for ARM/ARM64. > > > Hi Pranavkumar, > > Freescale has the same, or very similar, need, so any solution in this > space will need to work for both ARM and powerpc. I'm not a big fan of > this approach as it seems to require the user to configure MSI/X via > ioctl and then call a separate ioctl mapping the doorbells. That's more > code for the user, more code to get wrong and potentially a gap between > configuring MSI/X and enabling mappings where we could see IOMMU faults. > > If we know that doorbell mappings are required, why can't we set aside a > bank of IOVA space and have them mapped automatically as MSI/X is being > configured? Then the user's need for special knowledge and handling of > this case is limited to setup. The IOVA space will be mapped and used > as needed, we only need the user to specify the IOVA space reserved for > this. Thanks, I guess my immediate worry is that it seems to impose a fixed mapping for all the guests, which would restrict the "shape" of the mappings we give to a guest. Or did you intend for that IOVA mapping to be defined on a "per userspace instance" basis? Thanks, M. -- Jazz is not dead. It just smells funny... -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/