From: Yoder Stuart-B08248 <B08248@freescale.com>
To: Alex Williamson <alex.williamson@redhat.com>,
        Sethi Varun-B16395 <B16395@freescale.com>
CC: Joerg Roedel <joro@8bytes.org>, Wood Scott-B07421 <B07421@freescale.com>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "galak@kernel.crashing.org" <galak@kernel.crashing.org>,
        "benh@kernel.crashing.org" <benh@kernel.crashing.org>
Subject: RE: [PATCH 5/5 v11] iommu/fsl: Freescale PAMU driver and iommu
 implementation.
Thread-Topic: [PATCH 5/5 v11] iommu/fsl: Freescale PAMU driver and iommu
 implementation.
Thread-Index: AQHOMJVFO/4GVyqmPkuty25lenzG/ZjGB5KAgAAnnQCAAAZSgA==
Date: Thu, 4 Apr 2013 16:43:03 +0000
Message-ID: <9F6FE96B71CF29479FF1CDC8046E15035A26DD@039-SN1MPN1-003.039d.mgd.msft.net>
References: <1364500442-20927-1-git-send-email-Varun.Sethi@freescale.com>
	 <1364500442-20927-6-git-send-email-Varun.Sethi@freescale.com>
	 <20130402161812.GI15687@8bytes.org> <1365012091.2882.252.camel@bling.home>
	 <C5ECD7A89D1DC44195F34B25E172658D45883B@039-SN2MPN1-013.039d.mgd.msft.net>
 <1365088930.2882.296.camel@bling.home>
In-Reply-To: <1365088930.2882.296.camel@bling.home>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Sender: linux-kernel-owner@vger.kernel.org
Content-Transfer-Encoding: 8bit
Content-Length: 4100
Lines: 78


> -----Original Message-----
> From: Alex Williamson [mailto:alex.williamson@redhat.com]
> Sent: Thursday, April 04, 2013 10:22 AM
> To: Sethi Varun-B16395
> Cc: Joerg Roedel; Yoder Stuart-B08248; Wood Scott-B07421; iommu@lists.linux-foundation.org; linuxppc-
> dev@lists.ozlabs.org; linux-kernel@vger.kernel.org; galak@kernel.crashing.org; benh@kernel.crashing.org
> Subject: Re: [PATCH 5/5 v11] iommu/fsl: Freescale PAMU driver and iommu implementation.
> 
> On Thu, 2013-04-04 at 13:00 +0000, Sethi Varun-B16395 wrote:
> >
> > > -----Original Message-----
> > > From: Alex Williamson [mailto:alex.williamson@redhat.com]
> > > Sent: Wednesday, April 03, 2013 11:32 PM
> > > To: Joerg Roedel
> > > Cc: Sethi Varun-B16395; Yoder Stuart-B08248; Wood Scott-B07421;
> > > iommu@lists.linux-foundation.org; linuxppc-dev@lists.ozlabs.org; linux-
> > > kernel@vger.kernel.org; galak@kernel.crashing.org;
> > > benh@kernel.crashing.org
> > > Subject: Re: [PATCH 5/5 v11] iommu/fsl: Freescale PAMU driver and iommu
> > > implementation.
> > >
> > > On Tue, 2013-04-02 at 18:18 +0200, Joerg Roedel wrote:
> > > > Cc'ing Alex Williamson
> > > >
> > > > Alex, can you please review the iommu-group part of this patch?
> > >
> > > Sure, it looks pretty reasonable.  AIUI, all PCI devices are below some
> > > kind of host bridge that is either new and supports partitioning or old
> > > and doesn't.  I don't know if that's a visibility or isolation
> > > requirement, perhaps PCI ACS-ish.  In the new host bridge case, each
> > > device gets a group.  This seems not to have any quirks for multifunction
> > > devices though.  On AMD and Intel IOMMUs we test multifunction device ACS
> > > support to determine whether all the functions should be in the same
> > > group.  Is there any reason to trust multifunction devices on PAMU?
> > >
> > [Sethi Varun-B16395] In the case where we can partition endpoints we
> > can distinguish transactions based on the bus,device,function number
> > combination. This support is available in the PCIe controller (host
> > bridge).
> 
> So can x86 IOMMUs, that's the visibility aspect of IOMMU groups.
> Visibility alone doesn't necessarily imply that a device is isolated
> though.  A multifunction PCI device that doesn't expose ACS support may
> not isolate functions from each other.  For example a peer-to-peer DMA
> between functions may not be translated by the upstream IOMMU.  IOMMU
> groups should encompass both visibility and isolation.
> 
> > > I also find it curious what happens to the iommu group of the host
> > > bridge.  In the partitionable case the host bridge group is removed, in
> > > the non-partitionable case the host bridge group becomes the group for
> > > the children, removing the host bridge.  It's unique to PAMU so far that
> > > these host bridges are even in an iommu group (x86 only adds pci
> > > devices), but I don't see it as necessarily wrong leaving it in either
> > > scenario.  Does it solve some problem to remove them from the groups?
> > > Thanks,
> > [Sethi Varun-B16395] The PCIe controller isn't a partitionable entity,
> > it would always be owned by the host.
> 
> Ownership of a device shouldn't play into the group context.  An IOMMU
> group should be defined by it's visibility and isolation from other
> devices.  Whether the PCIe controller is allowed to be handed to
> userspace is a question for VFIO.  Thanks,

Right now the add_device() callback gets called for all platform
devices (including PCI controller) and PCI devices.  PCI controllers
are a kind of special case in that their device tree node has a
property indicating that it is DMA capable...but in fact the
PCI controller itself does not DMA, but it's the PCI endpoints
under it.

So, as you noted the bridge/controller shouldn't be in an IOMMU
group and so since the platform 'add device' code didn't special
case PCI controllers, this patch removes their group if it's there.

Stuart
????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay?ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m????????????I?