Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Thu, 7 Nov 2019 10:13:59 +0100
From:   Jean-Philippe Brucker <jean-philippe@linaro.org>
To:     Saravana Kannan <saravanak@google.com>
Cc:     Robin Murphy <robin.murphy@arm.com>,
        John Garry <john.garry@huawei.com>,
        Bjorn Helgaas <bhelgaas@google.com>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        Will Deacon <will@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 0/7] iommu: Permit modular builds of ARM SMMU[v3] drivers
Message-ID: <20191107091359.GA3752186@lophozonia>
References: <6e457227-ca06-2998-4ffa-a58ab171ce32@arm.com>
 <20191030155444.GC19096@willie-the-truck>
 <CAGETcx9ogWQC1ZtnS_4xC3ShqBpuRSKudWEEWC22UZUEhdEU4A@mail.gmail.com>
 <20191031193758.GA2607492@lophozonia>
 <CAGETcx-MuMVvj0O-MFdfmLADEq=cQY_=x+irvhgwHhG4VeeSdg@mail.gmail.com>
 <6994ae35-2b89-2feb-2bcb-cffc5a01963c@huawei.com>
 <CAGETcx-9M8vvHA2Lykcv0hHWoC2OAw5kfBrjcNJN2CYCwR4eWQ@mail.gmail.com>
 <47418554-e7a7-f9f3-8852-60cecef3d5c7@huawei.com>
 <7e2429ed-6b25-a452-5e4d-51a5195b872f@arm.com>
 <CAGETcx8QYBfGOBNtUTR+Lq_g+bmBxMOe0q=3U5UxvVqi+640Xw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAGETcx8QYBfGOBNtUTR+Lq_g+bmBxMOe0q=3U5UxvVqi+640Xw@mail.gmail.com>
User-Agent: Mutt/1.12.2 (2019-09-21)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, Nov 06, 2019 at 10:11:55PM -0800, Saravana Kannan wrote:
> > Right, in short the fundamental problem is that of_iommu_configure() now
> > does the wrong thing. Deferring probe of the entire host bridge/root
> > complex based on "iommu-map" would indeed happen to solve the problem by
> > brute force, I think, but could lead to a dependency cycle for PCI-based
> > IOMMUs as Jean points out.
> 
> Sorry for the late reply. Got caught up on other work.
> 
> I didn't think the SMMU itself was PCI based in the example Jean gave.
> I thought it just happened to be the case where the SMMU probes after
> the pcieport but before the other children. If the SMMU itself is a
> child of the pcieport, how can it be required for the parent to
> function? How will suspend/resume even work?! I feel like I'm missing
> some context wrt to PCI that everyone else seems to know (which isn't
> surprising).

The Arm SMMU isn't PCI based, it always appears as an independent MMIO
device. John's problem is something different if I understand correctly,
where the probe order between pcieport and endpoint shouldn't affect the
IOMMU grouping, but currently does.

Two other IOMMU models are PCI based, though, AMD IOMMU and virtio-iommu
(which is a purely virtual device). In theory they can have their
programming interface anywhere in the PCI config space, but to ensure
proper software support they should be at the top of the PCI hierarchy.
AMD strongly recommends that the IOMMU is a root-complex device (4.5 -
Software and Platform Firmware Implementation Issues). Within a PCIe
system the IOMMU would have to be a Root Complex integrated Endpoint, not
be a child of a root port.

Thanks,
Jean