From: Stuart Yoder <stuart.yoder@freescale.com>
To: Will Deacon <will.deacon@arm.com>, Varun Sethi <Varun.Sethi@freescale.com>
CC: Thierry Reding <thierry.reding@gmail.com>,
        Mark Rutland <Mark.Rutland@arm.com>,
        "devicetree@vger.kernel.org" <devicetree@vger.kernel.org>,
        "linux-samsung-soc@vger.kernel.org" 
	<linux-samsung-soc@vger.kernel.org>,
        Pawel Moll <Pawel.Moll@arm.com>, "Arnd Bergmann" <arnd@arndb.de>,
        Ian Campbell <ijc+devicetree@hellion.org.uk>,
        "Grant Grundler" <grundler@chromium.org>,
        Stephen Warren <swarren@wwwdotorg.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Marc Zyngier <Marc.Zyngier@arm.com>,
        Linux IOMMU <iommu@lists.linux-foundation.org>,
        "Rob Herring" <robh+dt@kernel.org>, Kumar Gala <galak@codeaurora.org>,
        "linux-tegra@vger.kernel.org" <linux-tegra@vger.kernel.org>,
        Cho KyongHo <pullip.cho@samsung.com>,
        Dave P Martin <Dave.Martin@arm.com>,
        "linux-arm-kernel@lists.infradead.org" 
	<linux-arm-kernel@lists.infradead.org>
Subject: RE: [PATCH v2] devicetree: Add generic IOMMU device tree bindings
Thread-Topic: [PATCH v2] devicetree: Add generic IOMMU device tree bindings
Thread-Index: AQHPdsaiUOp/jHgCJ0WWjZOTtpbJjZtZJRKAgABh1wCAAAcqgIAABQyAgAQdoQCAA2YSAIAAI08AgAG4pNCAEQ6xgIAAEv9A
Date: Mon, 16 Jun 2014 16:56:32 +0000
Message-ID: <8b0492b4697943a0b1f276ef42cc8223@DM2PR03MB352.namprd03.prod.outlook.com>
References: <1400877218-4113-1-git-send-email-thierry.reding@gmail.com>
 <4545972.cM7IP1qTXQ@wuerfel>
 <CAL_Jsq+tkFNpjSEZboT+o69B_Z3a8e-ZVKaYDmLwtkyYoPy6VQ@mail.gmail.com>
 <5288123.eXagyPAxNq@wuerfel>
 <20140602104104.GD3855@e103592.cambridge.arm.com>
 <20140604143509.GF28484@ulmo> <20140604164132.GF6644@arm.com>
 <07321368d15d4ad9928ebb83af87e401@DM2PR03MB479.namprd03.prod.outlook.com>
 <20140616152739.GS16758@arm.com>
In-Reply-To: <20140616152739.GS16758@arm.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 8BIT
MIME-Version: 1.0
Sender: linux-kernel-owner@vger.kernel.org


> -----Original Message-----
> From: Will Deacon [mailto:will.deacon@arm.com]
> Sent: Monday, June 16, 2014 10:28 AM
> To: Sethi Varun-B16395
> Cc: Thierry Reding; Mark Rutland; devicetree@vger.kernel.org; linux-
> samsung-soc@vger.kernel.org; Pawel Moll; Arnd Bergmann; Ian Campbell;
> Grant Grundler; Stephen Warren; linux-kernel@vger.kernel.org; Marc
> Zyngier; Linux IOMMU; Rob Herring; Kumar Gala; linux-
> tegra@vger.kernel.org; Cho KyongHo; Dave P Martin; linux-arm-
> kernel@lists.infradead.org; Yoder Stuart-B08248
> Subject: Re: [PATCH v2] devicetree: Add generic IOMMU device tree
> bindings
> 
> Hi Varun,
> 
> On Thu, Jun 05, 2014 at 08:10:19PM +0100, Varun Sethi wrote:
> > > The set of StreamIDs that can be generated by a master is fixed in
> the
> > > hardware. The SMMU can then be programmed to map these incoming IDs
> onto
> > > a context ID (or a set of context IDs), which are the IDs used
> internally
> > > by the SMMU to find the page tables etc.
> > >
> > > The StreamID -> ContextID mapping is dynamic and controlled by
> software.
> > > The Master -> StreamIDs mapping is fixed in the hardware.
> > The Master -> StreamIDs mapping may not always be fixed in the
> hardware.
> > At, least in our case we plan to program these via software. PCI
> devices
> > is one place where this mapping would have to be dynamic (based on the
> > topology i.e. if the devices are connected to a bridge etc). Also, we
> have
> > a hot plug device architecture where the stream ID is software
> > programmable.
> >
> > Other than that, based on the isolation requirements (iommu domain)
> > software programmability offers greater flexibility.
> 
> For the sake of consistency, I'd really like to treat the master ->
> streamIDs mapping as fixed. If that means setting it once during boot in
> your case, then so be it (ideally in the firmware). That way, we just
> describe the fixed mapping to the kernel and the driver doesn't have to
> worry about changing the mapping, especially given that that's likely to
> be
> highly error-prone once the SMMU is in us.
> 
> Do you have use-cases where you really need to change these mappings
> dynamically?

Yes.  In the case of a PCI bus-- you may not know in advance how many
PCI devices there are until you probe the bus.   We have another FSL
proprietary bus we call the "fsl-mc" bus that is similar.

Another thing to consider-- starting with SMMUv2, as you know, there
is a new distributed architecture with multiple TBUs and a centralized
TCU that walks the SMMU page tables.  So instead of sprinkling multiple
SMMUs all over an SoC you now have the option a 1 central TCU and sprinkling
multiple TBUs around.   However, this means that the stream ID namespace
is now global and can be pretty limited.  In the SMMU implementation we 
have there are only 64 stream ID total for our Soc.  But we have many more
masters than that.

So we look at stream IDs as really corresponding to an 'isolation context'
and not to a bus master.  An isolation context is the domain you are
trying to isolate with the SMMU.  Devices that all belong to the same
'isolation context' can share the same stream ID, since they share
the same domain and page tables.

So, perhaps by default some/most SMMU masters may have a default stream ID
of 0x0 that is used by the host...and that could be represented
statically in the device tree.

But, we absolutely will need to dynamically set new stream IDs
into masters when a new IOMMU 'domain' is created and devices
are added to it.   All the devices in a domain will share
the same stream ID.

So whatever we do, let's please have an architecture flexible enough
to allow for this.

Thanks,
Stuart


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/