From: Sethi Varun-B16395 <B16395@freescale.com>
To: Alex Williamson <alex.williamson@redhat.com>
CC: Joerg Roedel <joro@8bytes.org>, Yoder Stuart-B08248 <B08248@freescale.com>,
        Wood Scott-B07421 <B07421@freescale.com>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "galak@kernel.crashing.org" <galak@kernel.crashing.org>,
        "benh@kernel.crashing.org" <benh@kernel.crashing.org>
Subject: RE: [PATCH 5/5 v11] iommu/fsl: Freescale PAMU driver and iommu
 implementation.
Thread-Topic: [PATCH 5/5 v11] iommu/fsl: Freescale PAMU driver and iommu
 implementation.
Thread-Index: AQHOK+8t5MYhVlGTF0iQ16GsJktLA5jDI3qAgAGvMoCAATtRIIAAKn4AgAAO3OA=
Date: Thu, 4 Apr 2013 16:35:39 +0000
Message-ID: <C5ECD7A89D1DC44195F34B25E172658D459471@039-SN2MPN1-013.039d.mgd.msft.net>
References: <1364500442-20927-1-git-send-email-Varun.Sethi@freescale.com>
	 <1364500442-20927-6-git-send-email-Varun.Sethi@freescale.com>
	 <20130402161812.GI15687@8bytes.org> <1365012091.2882.252.camel@bling.home>
	 <C5ECD7A89D1DC44195F34B25E172658D45883B@039-SN2MPN1-013.039d.mgd.msft.net>
 <1365088930.2882.296.camel@bling.home>
In-Reply-To: <1365088930.2882.296.camel@bling.home>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Sender: linux-kernel-owner@vger.kernel.org
Content-Transfer-Encoding: 8bit
Content-Length: 4408
Lines: 78


> -----Original Message-----
> From: Alex Williamson [mailto:alex.williamson@redhat.com]
> Sent: Thursday, April 04, 2013 8:52 PM
> To: Sethi Varun-B16395
> Cc: Joerg Roedel; Yoder Stuart-B08248; Wood Scott-B07421;
> iommu@lists.linux-foundation.org; linuxppc-dev@lists.ozlabs.org; linux-
> kernel@vger.kernel.org; galak@kernel.crashing.org;
> benh@kernel.crashing.org
> Subject: Re: [PATCH 5/5 v11] iommu/fsl: Freescale PAMU driver and iommu
> implementation.
> 
> On Thu, 2013-04-04 at 13:00 +0000, Sethi Varun-B16395 wrote:
> >
> > > -----Original Message-----
> > > From: Alex Williamson [mailto:alex.williamson@redhat.com]
> > > Sent: Wednesday, April 03, 2013 11:32 PM
> > > To: Joerg Roedel
> > > Cc: Sethi Varun-B16395; Yoder Stuart-B08248; Wood Scott-B07421;
> > > iommu@lists.linux-foundation.org; linuxppc-dev@lists.ozlabs.org;
> > > linux- kernel@vger.kernel.org; galak@kernel.crashing.org;
> > > benh@kernel.crashing.org
> > > Subject: Re: [PATCH 5/5 v11] iommu/fsl: Freescale PAMU driver and
> > > iommu implementation.
> > >
> > > On Tue, 2013-04-02 at 18:18 +0200, Joerg Roedel wrote:
> > > > Cc'ing Alex Williamson
> > > >
> > > > Alex, can you please review the iommu-group part of this patch?
> > >
> > > Sure, it looks pretty reasonable.  AIUI, all PCI devices are below
> > > some kind of host bridge that is either new and supports
> > > partitioning or old and doesn't.  I don't know if that's a
> > > visibility or isolation requirement, perhaps PCI ACS-ish.  In the
> > > new host bridge case, each device gets a group.  This seems not to
> > > have any quirks for multifunction devices though.  On AMD and Intel
> > > IOMMUs we test multifunction device ACS support to determine whether
> > > all the functions should be in the same group.  Is there any reason
> to trust multifunction devices on PAMU?
> > >
> > [Sethi Varun-B16395] In the case where we can partition endpoints we
> > can distinguish transactions based on the bus,device,function number
> > combination. This support is available in the PCIe controller (host
> > bridge).
> 
> So can x86 IOMMUs, that's the visibility aspect of IOMMU groups.
> Visibility alone doesn't necessarily imply that a device is isolated
> though.  A multifunction PCI device that doesn't expose ACS support may
> not isolate functions from each other.  For example a peer-to-peer DMA
> between functions may not be translated by the upstream IOMMU.  IOMMU
> groups should encompass both visibility and isolation.
[Sethi Varun-B16395] We can isolate the DMA access to the host based on the to the pci bus,device,function number.
I thought that was enough to put devices in to separate iommu groups. This is a PCIe controller property which allows us to partition PCIe devices. But, what I can understand from your point is that we also need to consider isolation at PCIe device level as well. I will check for the case of multifunction devices.

> 
> > > I also find it curious what happens to the iommu group of the host
> > > bridge.  In the partitionable case the host bridge group is removed,
> > > in the non-partitionable case the host bridge group becomes the
> > > group for the children, removing the host bridge.  It's unique to
> > > PAMU so far that these host bridges are even in an iommu group (x86
> > > only adds pci devices), but I don't see it as necessarily wrong
> > > leaving it in either scenario.  Does it solve some problem to remove
> them from the groups?
> > > Thanks,
> > [Sethi Varun-B16395] The PCIe controller isn't a partitionable entity,
> > it would always be owned by the host.
> 
> Ownership of a device shouldn't play into the group context.  An IOMMU
> group should be defined by it's visibility and isolation from other
> devices.  Whether the PCIe controller is allowed to be handed to
> userspace is a question for VFIO.
[Sethi Varun-B16395] The problem is in the case, where we can't partition PCIe devices. PCIe devices share the same device group as the PCI controller. This becomes a problem while assigning the devices to the guest, as you are required to unbind all the PCIe devices including the controller from the host. PCIe controller can't be unbound from the host, so we simply delete the controller iommu_group.

-Varun
 

????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay?ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m????????????I?