2017-06-19 09:17:35

by Magnus Damm

[permalink] [raw]
Subject: [PATCH v4 00/09] iommu/ipmmu-vmsa: r8a7795 support V4

iommu/ipmmu-vmsa: r8a7795 support V4

[PATCH v4 01/09] iommu/ipmmu-vmsa: Introduce features, break out alias
[PATCH v4 02/09] iommu/ipmmu-vmsa: Add optional root device feature
[PATCH v4 03/09] iommu/ipmmu-vmsa: Enable multi context support
[PATCH v4 04/09] iommu/ipmmu-vmsa: Make use of IOMMU_OF_DECLARE()
[PATCH v4 05/09] iommu/ipmmu-vmsa: IPMMU device is 40-bit bus master
[PATCH v4 06/09] iommu/ipmmu-vmsa: Write IMCTR twice
[PATCH v4 07/09] iommu/ipmmu-vmsa: Make IMBUSCTR setup optional
[PATCH v4 08/09] iommu/ipmmu-vmsa: Allow two bit SL0
[PATCH v4 09/09] iommu/ipmmu-vmsa: Hook up r8a7795 DT matching code

Here's an updated series for r8a7795 IPMMU support. The patches adjust
the code based on feedback from Geert and Robin together with a rebase
to include changes from the recently posted series:

[PATCH 00/04] iommu/ipmmu-vmsa: 32-bit ARM update

The DT binding for r8a7795 has been accepted for upstream merge
and this series implements support following such format:

d4e42e7 iommu/ipmmu-vmsa: Add r8a7795 DT binding

The r8a7795 IPMMU is almost register compatible with earlier devices
like r8a7790-r8a7794, however some bitfields have been shifted
slightly. On a grander scale topology has been added and interrupts
have been reworked. So now there are several "cache" IPMMU units
without interrupt that somehow communicate with IPMMU-MM that
is the only instance that supports interrupts. The code refers to
IPMMU-MM as a "root" device and the other ones as "leaf" nodes.

Changes since V3:
- Rebased on top of [PATCH 00/04] iommu/ipmmu-vmsa: 32-bit ARM update
- Patch 1/9 has been updated to use of_device_get_match_data(), thanks Robin!
- Patch 2/9 has been reworked to make it easier to follow, thanks Geert!
- Patch 3/9 now uses unsigned int for context counts - thanks Robin!
- Patch 6/9 now includes function name changes - thanks Robin!
- Patch 9/9 now uses dev_err() instead of dev_info() - thanks Geert!

Changes since V2:
- Patch 2/9 has been updated with a bug fix and to supply __ipmmu_find_root()
- Patch 4/9 now makes use of iommu_device_* functions
- Patch 5/9 sets the mask to 40 bits instead of 64 bits
- Patch 9/9 implements white list handling via ->xlate() and fixes a bug

Signed-off-by: Magnus Damm <[email protected]>
---

Developed on top of next-20170614 with the following series applied
[PATCH 00/04] iommu/ipmmu-vmsa: 32-bit ARM update

drivers/iommu/ipmmu-vmsa.c | 327 ++++++++++++++++++++++++++++++++++----------
1 file changed, 260 insertions(+), 67 deletions(-)


2017-06-19 09:17:50

by Magnus Damm

[permalink] [raw]
Subject: [PATCH v4 01/09] iommu/ipmmu-vmsa: Introduce features, break out alias

From: Magnus Damm <[email protected]>

Introduce struct ipmmu_features to track various hardware
and software implementation changes inside the driver for
different kinds of IPMMU hardware. Add use_ns_alias_offset
as a first example of a feature to control if the secure
register bank offset should be used or not.

Signed-off-by: Magnus Damm <[email protected]>
---

Changes since V3:
- Use of_device_get_match_data(), thanks Robin!

Changes since V2:
- None

Changes since V1:
- Moved patch to front of the series

drivers/iommu/ipmmu-vmsa.c | 31 ++++++++++++++++++++++++-------
1 file changed, 24 insertions(+), 7 deletions(-)

--- 0014/drivers/iommu/ipmmu-vmsa.c
+++ work/drivers/iommu/ipmmu-vmsa.c 2017-06-19 13:57:36.300607110 +0900
@@ -19,6 +19,7 @@
#include <linux/iommu.h>
#include <linux/module.h>
#include <linux/of.h>
+#include <linux/of_device.h>
#include <linux/of_platform.h>
#include <linux/platform_device.h>
#include <linux/sizes.h>
@@ -33,12 +34,16 @@

#define IPMMU_CTX_MAX 1

+struct ipmmu_features {
+ bool use_ns_alias_offset;
+};
+
struct ipmmu_vmsa_device {
struct device *dev;
void __iomem *base;
struct iommu_device iommu;
struct list_head list;
-
+ const struct ipmmu_features *features;
unsigned int num_utlbs;
spinlock_t lock; /* Protects ctx and domains[] */
DECLARE_BITMAP(ctx, IPMMU_CTX_MAX);
@@ -904,6 +909,21 @@ static void ipmmu_device_reset(struct ip
ipmmu_write(mmu, i * IM_CTX_SIZE + IMCTR, 0);
}

+static const struct ipmmu_features ipmmu_features_default = {
+ .use_ns_alias_offset = true,
+};
+
+static const struct of_device_id ipmmu_of_ids[] = {
+ {
+ .compatible = "renesas,ipmmu-vmsa",
+ .data = &ipmmu_features_default,
+ }, {
+ /* Terminator */
+ },
+};
+
+MODULE_DEVICE_TABLE(of, ipmmu_of_ids);
+
static int ipmmu_probe(struct platform_device *pdev)
{
struct ipmmu_vmsa_device *mmu;
@@ -921,6 +941,7 @@ static int ipmmu_probe(struct platform_d
mmu->num_utlbs = 32;
spin_lock_init(&mmu->lock);
bitmap_zero(mmu->ctx, IPMMU_CTX_MAX);
+ mmu->features = of_device_get_match_data(&pdev->dev);

/* Map I/O memory and request IRQ. */
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
@@ -940,7 +961,8 @@ static int ipmmu_probe(struct platform_d
* Offset the registers base unconditionally to point to the non-secure
* alias space for now.
*/
- mmu->base += IM_NS_ALIAS_OFFSET;
+ if (mmu->features->use_ns_alias_offset)
+ mmu->base += IM_NS_ALIAS_OFFSET;

irq = platform_get_irq(pdev, 0);
if (irq < 0) {
@@ -998,11 +1020,6 @@ static int ipmmu_remove(struct platform_
return 0;
}

-static const struct of_device_id ipmmu_of_ids[] = {
- { .compatible = "renesas,ipmmu-vmsa", },
- { }
-};
-
static struct platform_driver ipmmu_driver = {
.driver = {
.name = "ipmmu-vmsa",

2017-06-19 09:17:57

by Magnus Damm

[permalink] [raw]
Subject: [PATCH v4 02/09] iommu/ipmmu-vmsa: Add optional root device feature

From: Magnus Damm <[email protected]>

Add root device handling to the IPMMU driver by allowing certain
DT compat strings to enable has_cache_leaf_nodes that in turn will
support both root devices with interrupts and leaf devices that
face the actual IPMMU consumer devices.

Signed-off-by: Magnus Damm <[email protected]>
---

Changes since V3:
- Reworked root finding code to make it easier to follow, thanks Geert!

Changes since V2:
- Fixed a bug in ipmmu_find_root() when only leaf devices are present
- Broke out __ipmmu_find_root() to allow ->xlate() check for root devices

Changes since V1:
- Moved patch to earlier in the series
- Updated code to work with recent changes in:
[PATCH v3 00/06] iommu/ipmmu-vmsa: IPMMU multi-arch update V3

drivers/iommu/ipmmu-vmsa.c | 95 ++++++++++++++++++++++++++++++++++++--------
1 file changed, 78 insertions(+), 17 deletions(-)

--- 0015/drivers/iommu/ipmmu-vmsa.c
+++ work/drivers/iommu/ipmmu-vmsa.c 2017-06-19 13:59:41.050607110 +0900
@@ -36,6 +36,7 @@

struct ipmmu_features {
bool use_ns_alias_offset;
+ bool has_cache_leaf_nodes;
};

struct ipmmu_vmsa_device {
@@ -44,6 +45,7 @@ struct ipmmu_vmsa_device {
struct iommu_device iommu;
struct list_head list;
const struct ipmmu_features *features;
+ bool is_leaf;
unsigned int num_utlbs;
spinlock_t lock; /* Protects ctx and domains[] */
DECLARE_BITMAP(ctx, IPMMU_CTX_MAX);
@@ -54,6 +56,7 @@ struct ipmmu_vmsa_device {

struct ipmmu_vmsa_domain {
struct ipmmu_vmsa_device *mmu;
+ struct ipmmu_vmsa_device *root;
struct iommu_domain io_domain;

struct io_pgtable_cfg cfg;
@@ -203,6 +206,44 @@ static struct ipmmu_vmsa_iommu_priv *to_
#define IMUASID_ASID0_SHIFT 0

/* -----------------------------------------------------------------------------
+ * Root device handling
+ */
+
+static bool ipmmu_is_root(struct ipmmu_vmsa_device *mmu)
+{
+ if (mmu->features->has_cache_leaf_nodes)
+ return mmu->is_leaf ? false : true;
+ else
+ return true; /* older IPMMU hardware treated as single root */
+}
+
+static struct ipmmu_vmsa_device *__ipmmu_find_root(void)
+{
+ struct ipmmu_vmsa_device *mmu;
+ struct ipmmu_vmsa_device *root = NULL;
+
+ spin_lock(&ipmmu_devices_lock);
+
+ list_for_each_entry(mmu, &ipmmu_devices, list) {
+ if (ipmmu_is_root(mmu)) {
+ root = mmu;
+ break;
+ }
+ }
+
+ spin_unlock(&ipmmu_devices_lock);
+ return root;
+}
+
+static struct ipmmu_vmsa_device *ipmmu_find_root(struct ipmmu_vmsa_device *leaf)
+{
+ if (ipmmu_is_root(leaf))
+ return leaf;
+ else
+ return __ipmmu_find_root();
+}
+
+/* -----------------------------------------------------------------------------
* Read/Write Access
*/

@@ -219,13 +260,13 @@ static void ipmmu_write(struct ipmmu_vms

static u32 ipmmu_ctx_read(struct ipmmu_vmsa_domain *domain, unsigned int reg)
{
- return ipmmu_read(domain->mmu, domain->context_id * IM_CTX_SIZE + reg);
+ return ipmmu_read(domain->root, domain->context_id * IM_CTX_SIZE + reg);
}

static void ipmmu_ctx_write(struct ipmmu_vmsa_domain *domain, unsigned int reg,
u32 data)
{
- ipmmu_write(domain->mmu, domain->context_id * IM_CTX_SIZE + reg, data);
+ ipmmu_write(domain->root, domain->context_id * IM_CTX_SIZE + reg, data);
}

/* -----------------------------------------------------------------------------
@@ -360,7 +401,7 @@ static int ipmmu_domain_init_context(str
* TODO: Add support for coherent walk through CCI with DVM and remove
* cache handling. For now, delegate it to the io-pgtable code.
*/
- domain->cfg.iommu_dev = domain->mmu->dev;
+ domain->cfg.iommu_dev = domain->root->dev;

domain->iop = alloc_io_pgtable_ops(ARM_32_LPAE_S1, &domain->cfg,
domain);
@@ -370,7 +411,7 @@ static int ipmmu_domain_init_context(str
/*
* Find an unused context.
*/
- ret = ipmmu_domain_allocate_context(domain->mmu, domain);
+ ret = ipmmu_domain_allocate_context(domain->root, domain);
if (ret == IPMMU_CTX_MAX) {
free_io_pgtable_ops(domain->iop);
return -EBUSY;
@@ -441,7 +482,7 @@ static void ipmmu_domain_destroy_context
*/
ipmmu_ctx_write(domain, IMCTR, IMCTR_FLUSH);
ipmmu_tlb_sync(domain);
- ipmmu_domain_free_context(domain->mmu, domain->context_id);
+ ipmmu_domain_free_context(domain->root, domain->context_id);
}

/* -----------------------------------------------------------------------------
@@ -555,7 +596,7 @@ static int ipmmu_attach_device(struct io
{
struct ipmmu_vmsa_iommu_priv *priv = to_priv(dev);
struct iommu_fwspec *fwspec = dev->iommu_fwspec;
- struct ipmmu_vmsa_device *mmu = priv->mmu;
+ struct ipmmu_vmsa_device *root, *mmu = priv->mmu;
struct ipmmu_vmsa_domain *domain = to_vmsa_domain(io_domain);
unsigned long flags;
unsigned int i;
@@ -566,11 +607,18 @@ static int ipmmu_attach_device(struct io
return -ENXIO;
}

+ root = ipmmu_find_root(priv->mmu);
+ if (!root) {
+ dev_err(dev, "Unable to locate root IPMMU\n");
+ return -EAGAIN;
+ }
+
spin_lock_irqsave(&domain->lock, flags);

if (!domain->mmu) {
/* The domain hasn't been used yet, initialize it. */
domain->mmu = mmu;
+ domain->root = root;
ret = ipmmu_domain_init_context(domain);
} else if (domain->mmu != mmu) {
/*
@@ -911,6 +959,7 @@ static void ipmmu_device_reset(struct ip

static const struct ipmmu_features ipmmu_features_default = {
.use_ns_alias_offset = true,
+ .has_cache_leaf_nodes = false,
};

static const struct of_device_id ipmmu_of_ids[] = {
@@ -965,19 +1014,31 @@ static int ipmmu_probe(struct platform_d
mmu->base += IM_NS_ALIAS_OFFSET;

irq = platform_get_irq(pdev, 0);
- if (irq < 0) {
- dev_err(&pdev->dev, "no IRQ found\n");
- return irq;
- }

- ret = devm_request_irq(&pdev->dev, irq, ipmmu_irq, 0,
- dev_name(&pdev->dev), mmu);
- if (ret < 0) {
- dev_err(&pdev->dev, "failed to request IRQ %d\n", irq);
- return ret;
- }
+ /*
+ * Determine if this IPMMU instance is a leaf device by checking
+ * if the renesas,ipmmu-main property exists or not.
+ */
+ if (mmu->features->has_cache_leaf_nodes &&
+ of_find_property(pdev->dev.of_node, "renesas,ipmmu-main", NULL))
+ mmu->is_leaf = true;
+
+ /* Root devices have mandatory IRQs */
+ if (ipmmu_is_root(mmu)) {
+ if (irq < 0) {
+ dev_err(&pdev->dev, "no IRQ found\n");
+ return irq;
+ }

- ipmmu_device_reset(mmu);
+ ret = devm_request_irq(&pdev->dev, irq, ipmmu_irq, 0,
+ dev_name(&pdev->dev), mmu);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "failed to request IRQ %d\n", irq);
+ return ret;
+ }
+
+ ipmmu_device_reset(mmu);
+ }

ret = iommu_device_register(&mmu->iommu);
if (ret)

2017-06-19 09:18:12

by Magnus Damm

[permalink] [raw]
Subject: [PATCH v4 03/09] iommu/ipmmu-vmsa: Enable multi context support

From: Magnus Damm <[email protected]>

Add support for up to 8 contexts. Each context is mapped to one
domain. One domain is assigned one or more slave devices. Contexts
are allocated dynamically and slave devices are grouped together
based on which IPMMU device they are connected to. This makes slave
devices tied to the same IPMMU device share the same IOVA space.

Signed-off-by: Magnus Damm <[email protected]>
---

Changes since V3:
- Use number_of_contexts unsigned int, drop WARN_ON() - Thanks Robin!

Changes since V2:
- Updated patch description to reflect code included in:
[PATCH v7 00/07] iommu/ipmmu-vmsa: IPMMU multi-arch update V7

Changes since V1:
- Support up to 8 contexts instead of 4
- Use feature flag and runtime handling
- Default to single context

drivers/iommu/ipmmu-vmsa.c | 31 +++++++++++++++++++++++--------
1 file changed, 23 insertions(+), 8 deletions(-)

--- 0017/drivers/iommu/ipmmu-vmsa.c
+++ work/drivers/iommu/ipmmu-vmsa.c 2017-06-19 14:00:59.880607110 +0900
@@ -32,11 +32,12 @@

#include "io-pgtable.h"

-#define IPMMU_CTX_MAX 1
+#define IPMMU_CTX_MAX 8

struct ipmmu_features {
bool use_ns_alias_offset;
bool has_cache_leaf_nodes;
+ unsigned int number_of_contexts;
};

struct ipmmu_vmsa_device {
@@ -47,6 +48,7 @@ struct ipmmu_vmsa_device {
const struct ipmmu_features *features;
bool is_leaf;
unsigned int num_utlbs;
+ unsigned int num_ctx;
spinlock_t lock; /* Protects ctx and domains[] */
DECLARE_BITMAP(ctx, IPMMU_CTX_MAX);
struct ipmmu_vmsa_domain *domains[IPMMU_CTX_MAX];
@@ -363,11 +365,12 @@ static int ipmmu_domain_allocate_context

spin_lock_irqsave(&mmu->lock, flags);

- ret = find_first_zero_bit(mmu->ctx, IPMMU_CTX_MAX);
- if (ret != IPMMU_CTX_MAX) {
+ ret = find_first_zero_bit(mmu->ctx, mmu->num_ctx);
+ if (ret != mmu->num_ctx) {
mmu->domains[ret] = domain;
set_bit(ret, mmu->ctx);
- }
+ } else
+ ret = -EBUSY;

spin_unlock_irqrestore(&mmu->lock, flags);

@@ -412,9 +415,9 @@ static int ipmmu_domain_init_context(str
* Find an unused context.
*/
ret = ipmmu_domain_allocate_context(domain->root, domain);
- if (ret == IPMMU_CTX_MAX) {
+ if (ret < 0) {
free_io_pgtable_ops(domain->iop);
- return -EBUSY;
+ return ret;
}

domain->context_id = ret;
@@ -549,7 +552,7 @@ static irqreturn_t ipmmu_irq(int irq, vo
/*
* Check interrupts for all active contexts.
*/
- for (i = 0; i < IPMMU_CTX_MAX; i++) {
+ for (i = 0; i < mmu->num_ctx; i++) {
if (!mmu->domains[i])
continue;
if (ipmmu_domain_irq(mmu->domains[i]) == IRQ_HANDLED)
@@ -620,6 +623,14 @@ static int ipmmu_attach_device(struct io
domain->mmu = mmu;
domain->root = root;
ret = ipmmu_domain_init_context(domain);
+ if (ret < 0) {
+ dev_err(dev, "Unable to initialize IPMMU context\n");
+ domain->mmu = NULL;
+ domain->root = NULL;
+ } else {
+ dev_info(dev, "Using IPMMU context %u\n",
+ domain->context_id);
+ }
} else if (domain->mmu != mmu) {
/*
* Something is wrong, we can't attach two devices using
@@ -953,13 +964,14 @@ static void ipmmu_device_reset(struct ip
unsigned int i;

/* Disable all contexts. */
- for (i = 0; i < 4; ++i)
+ for (i = 0; i < mmu->num_ctx; ++i)
ipmmu_write(mmu, i * IM_CTX_SIZE + IMCTR, 0);
}

static const struct ipmmu_features ipmmu_features_default = {
.use_ns_alias_offset = true,
.has_cache_leaf_nodes = false,
+ .number_of_contexts = 1, /* software only tested with one context */
};

static const struct of_device_id ipmmu_of_ids[] = {
@@ -1013,6 +1025,9 @@ static int ipmmu_probe(struct platform_d
if (mmu->features->use_ns_alias_offset)
mmu->base += IM_NS_ALIAS_OFFSET;

+ mmu->num_ctx = min_t(unsigned int, IPMMU_CTX_MAX,
+ mmu->features->number_of_contexts);
+
irq = platform_get_irq(pdev, 0);

/*

2017-06-19 09:18:23

by Magnus Damm

[permalink] [raw]
Subject: [PATCH v4 04/09] iommu/ipmmu-vmsa: Make use of IOMMU_OF_DECLARE()

From: Magnus Damm <[email protected]>

Hook up IOMMU_OF_DECLARE() support in case CONFIG_IOMMU_DMA
is enabled. The only current supported case for 32-bit ARM
is disabled, however for 64-bit ARM usage of OF is required.

Signed-off-by: Magnus Damm <[email protected]>
---

Changes since V3:
- Reworked to fit on top of
[PATCH 00/04] iommu/ipmmu-vmsa: 32-bit ARM update

Changes since V2:
- Reworked registration code to make use of recently introduced:
iommu_device_register()
iommu_device_set_ops()
iommu_device_set_fwnode()

Changes since V1:
- Reworked slightly to fit updated patch order and
[PATCH v3 00/06] iommu/ipmmu-vmsa: IPMMU multi-arch update V3

drivers/iommu/ipmmu-vmsa.c | 43 +++++++++++++++++++++++++++++++++++++------
1 file changed, 37 insertions(+), 6 deletions(-)

--- 0019/drivers/iommu/ipmmu-vmsa.c
+++ work/drivers/iommu/ipmmu-vmsa.c 2017-06-19 14:03:53.140607110 +0900
@@ -20,6 +20,7 @@
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_device.h>
+#include <linux/of_iommu.h>
#include <linux/of_platform.h>
#include <linux/platform_device.h>
#include <linux/sizes.h>
@@ -1055,13 +1056,25 @@ static int ipmmu_probe(struct platform_d
ipmmu_device_reset(mmu);
}

- ret = iommu_device_register(&mmu->iommu);
- if (ret)
- return ret;
-
- iommu_device_set_ops(&mmu->iommu, &ipmmu_ops);
- iommu_device_set_fwnode(&mmu->iommu, &pdev->dev.of_node->fwnode);
+ /*
+ * Register the IPMMU to the IOMMU subsystem in the following cases:
+ * - R-Car Gen2 IPMMU (all devices registered)
+ * - R-Car Gen3 IPMMU (leaf devices only - skip root IPMMU-MM device)
+ */
+ if (!mmu->features->has_cache_leaf_nodes || mmu->is_leaf) {
+ ret = iommu_device_register(&mmu->iommu);
+ if (ret)
+ return ret;

+ iommu_device_set_ops(&mmu->iommu, &ipmmu_ops);
+ iommu_device_set_fwnode(&mmu->iommu,
+ &pdev->dev.of_node->fwnode);
+
+#if defined(CONFIG_IOMMU_DMA)
+ if (!iommu_present(&platform_bus_type))
+ bus_set_iommu(&platform_bus_type, &ipmmu_ops);
+#endif
+ }
/*
* We can't create the ARM mapping here as it requires the bus to have
* an IOMMU, which only happens when bus_set_iommu() is called in
@@ -1107,15 +1120,22 @@ static struct platform_driver ipmmu_driv

static int __init ipmmu_init(void)
{
+ static bool setup_done;
int ret;

+ if (setup_done)
+ return 0;
+
ret = platform_driver_register(&ipmmu_driver);
if (ret < 0)
return ret;

+#if defined(CONFIG_ARM) && !defined(CONFIG_IOMMU_DMA)
if (!iommu_present(&platform_bus_type))
bus_set_iommu(&platform_bus_type, &ipmmu_ops);
+#endif

+ setup_done = true;
return 0;
}

@@ -1127,6 +1147,17 @@ static void __exit ipmmu_exit(void)
subsys_initcall(ipmmu_init);
module_exit(ipmmu_exit);

+#ifdef CONFIG_IOMMU_DMA
+static int __init ipmmu_vmsa_iommu_of_setup(struct device_node *np)
+{
+ ipmmu_init();
+ return 0;
+}
+
+IOMMU_OF_DECLARE(ipmmu_vmsa_iommu_of, "renesas,ipmmu-vmsa",
+ ipmmu_vmsa_iommu_of_setup);
+#endif
+
MODULE_DESCRIPTION("IOMMU API for Renesas VMSA-compatible IPMMU");
MODULE_AUTHOR("Laurent Pinchart <[email protected]>");
MODULE_LICENSE("GPL v2");

2017-06-19 09:18:33

by Magnus Damm

[permalink] [raw]
Subject: [PATCH v4 05/09] iommu/ipmmu-vmsa: IPMMU device is 40-bit bus master

From: Magnus Damm <[email protected]>

The r8a7795 IPMMU supports 40-bit bus mastering. Both
the coherent DMA mask and the streaming DMA mask are
set to unlock the 40-bit address space for coherent
allocations and streaming operations.

Signed-off-by: Magnus Damm <[email protected]>
---

Changes since V3:
- None

Changes since V2:
- Updated the code and commit message to use 40 bits instead of 64 bits

Changes since V1:
- Updated the commit message

drivers/iommu/ipmmu-vmsa.c | 1 +
1 file changed, 1 insertion(+)

--- 0021/drivers/iommu/ipmmu-vmsa.c
+++ work/drivers/iommu/ipmmu-vmsa.c 2017-06-19 14:05:33.720607110 +0900
@@ -1004,6 +1004,7 @@ static int ipmmu_probe(struct platform_d
spin_lock_init(&mmu->lock);
bitmap_zero(mmu->ctx, IPMMU_CTX_MAX);
mmu->features = of_device_get_match_data(&pdev->dev);
+ dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(40));

/* Map I/O memory and request IRQ. */
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);

2017-06-19 09:18:48

by Magnus Damm

[permalink] [raw]
Subject: [PATCH v4 06/09] iommu/ipmmu-vmsa: Write IMCTR twice

From: Magnus Damm <[email protected]>

Write IMCTR both in the root device and the leaf node.

To allow access of IMCTR introduce the following function:
- ipmmu_ctx_write_all()

While at it also rename context functions:
- ipmmu_ctx_read() -> ipmmu_ctx_read_root()
- ipmmu_ctx_write() -> ipmmu_ctx_write_root()

Signed-off-by: Magnus Damm <[email protected]>
---

Changes since V3:
- Changed function names to improve code readability - Thanks Robin!

Changes since V2:
- None

Changes since V1:
- None

drivers/iommu/ipmmu-vmsa.c | 55 +++++++++++++++++++++++++++-----------------
1 file changed, 34 insertions(+), 21 deletions(-)

--- 0023/drivers/iommu/ipmmu-vmsa.c
+++ work/drivers/iommu/ipmmu-vmsa.c 2017-06-19 14:06:16.770607110 +0900
@@ -261,17 +261,28 @@ static void ipmmu_write(struct ipmmu_vms
iowrite32(data, mmu->base + offset);
}

-static u32 ipmmu_ctx_read(struct ipmmu_vmsa_domain *domain, unsigned int reg)
+static u32 ipmmu_ctx_read_root(struct ipmmu_vmsa_domain *domain,
+ unsigned int reg)
{
return ipmmu_read(domain->root, domain->context_id * IM_CTX_SIZE + reg);
}

-static void ipmmu_ctx_write(struct ipmmu_vmsa_domain *domain, unsigned int reg,
- u32 data)
+static void ipmmu_ctx_write_root(struct ipmmu_vmsa_domain *domain,
+ unsigned int reg, u32 data)
{
ipmmu_write(domain->root, domain->context_id * IM_CTX_SIZE + reg, data);
}

+static void ipmmu_ctx_write_all(struct ipmmu_vmsa_domain *domain,
+ unsigned int reg, u32 data)
+{
+ if (domain->mmu != domain->root)
+ ipmmu_write(domain->mmu,
+ domain->context_id * IM_CTX_SIZE + reg, data);
+
+ ipmmu_write(domain->root, domain->context_id * IM_CTX_SIZE + reg, data);
+}
+
/* -----------------------------------------------------------------------------
* TLB and microTLB Management
*/
@@ -281,7 +292,7 @@ static void ipmmu_tlb_sync(struct ipmmu_
{
unsigned int count = 0;

- while (ipmmu_ctx_read(domain, IMCTR) & IMCTR_FLUSH) {
+ while (ipmmu_ctx_read_root(domain, IMCTR) & IMCTR_FLUSH) {
cpu_relax();
if (++count == TLB_LOOP_TIMEOUT) {
dev_err_ratelimited(domain->mmu->dev,
@@ -296,9 +307,9 @@ static void ipmmu_tlb_invalidate(struct
{
u32 reg;

- reg = ipmmu_ctx_read(domain, IMCTR);
+ reg = ipmmu_ctx_read_root(domain, IMCTR);
reg |= IMCTR_FLUSH;
- ipmmu_ctx_write(domain, IMCTR, reg);
+ ipmmu_ctx_write_all(domain, IMCTR, reg);

ipmmu_tlb_sync(domain);
}
@@ -425,31 +436,32 @@ static int ipmmu_domain_init_context(str

/* TTBR0 */
ttbr = domain->cfg.arm_lpae_s1_cfg.ttbr[0];
- ipmmu_ctx_write(domain, IMTTLBR0, ttbr);
- ipmmu_ctx_write(domain, IMTTUBR0, ttbr >> 32);
+ ipmmu_ctx_write_root(domain, IMTTLBR0, ttbr);
+ ipmmu_ctx_write_root(domain, IMTTUBR0, ttbr >> 32);

/*
* TTBCR
* We use long descriptors with inner-shareable WBWA tables and allocate
* the whole 32-bit VA space to TTBR0.
*/
- ipmmu_ctx_write(domain, IMTTBCR, IMTTBCR_EAE |
- IMTTBCR_SH0_INNER_SHAREABLE | IMTTBCR_ORGN0_WB_WA |
- IMTTBCR_IRGN0_WB_WA | IMTTBCR_SL0_LVL_1);
+ ipmmu_ctx_write_root(domain, IMTTBCR, IMTTBCR_EAE |
+ IMTTBCR_SH0_INNER_SHAREABLE | IMTTBCR_ORGN0_WB_WA |
+ IMTTBCR_IRGN0_WB_WA | IMTTBCR_SL0_LVL_1);

/* MAIR0 */
- ipmmu_ctx_write(domain, IMMAIR0, domain->cfg.arm_lpae_s1_cfg.mair[0]);
+ ipmmu_ctx_write_root(domain, IMMAIR0,
+ domain->cfg.arm_lpae_s1_cfg.mair[0]);

/* IMBUSCR */
- ipmmu_ctx_write(domain, IMBUSCR,
- ipmmu_ctx_read(domain, IMBUSCR) &
- ~(IMBUSCR_DVM | IMBUSCR_BUSSEL_MASK));
+ ipmmu_ctx_write_root(domain, IMBUSCR,
+ ipmmu_ctx_read_root(domain, IMBUSCR) &
+ ~(IMBUSCR_DVM | IMBUSCR_BUSSEL_MASK));

/*
* IMSTR
* Clear all interrupt flags.
*/
- ipmmu_ctx_write(domain, IMSTR, ipmmu_ctx_read(domain, IMSTR));
+ ipmmu_ctx_write_root(domain, IMSTR, ipmmu_ctx_read_root(domain, IMSTR));

/*
* IMCTR
@@ -458,7 +470,8 @@ static int ipmmu_domain_init_context(str
* software management as we have no use for it. Flush the TLB as
* required when modifying the context registers.
*/
- ipmmu_ctx_write(domain, IMCTR, IMCTR_INTEN | IMCTR_FLUSH | IMCTR_MMUEN);
+ ipmmu_ctx_write_all(domain, IMCTR,
+ IMCTR_INTEN | IMCTR_FLUSH | IMCTR_MMUEN);

return 0;
}
@@ -484,7 +497,7 @@ static void ipmmu_domain_destroy_context
*
* TODO: Is TLB flush really needed ?
*/
- ipmmu_ctx_write(domain, IMCTR, IMCTR_FLUSH);
+ ipmmu_ctx_write_all(domain, IMCTR, IMCTR_FLUSH);
ipmmu_tlb_sync(domain);
ipmmu_domain_free_context(domain->root, domain->context_id);
}
@@ -500,11 +513,11 @@ static irqreturn_t ipmmu_domain_irq(stru
u32 status;
u32 iova;

- status = ipmmu_ctx_read(domain, IMSTR);
+ status = ipmmu_ctx_read_root(domain, IMSTR);
if (!(status & err_mask))
return IRQ_NONE;

- iova = ipmmu_ctx_read(domain, IMEAR);
+ iova = ipmmu_ctx_read_root(domain, IMEAR);

/*
* Clear the error status flags. Unlike traditional interrupt flag
@@ -512,7 +525,7 @@ static irqreturn_t ipmmu_domain_irq(stru
* seems to require 0. The error address register must be read before,
* otherwise its value will be 0.
*/
- ipmmu_ctx_write(domain, IMSTR, 0);
+ ipmmu_ctx_write_root(domain, IMSTR, 0);

/* Log fatal errors. */
if (status & IMSTR_MHIT)

2017-06-19 09:19:02

by Magnus Damm

[permalink] [raw]
Subject: [PATCH v4 07/09] iommu/ipmmu-vmsa: Make IMBUSCTR setup optional

From: Magnus Damm <[email protected]>

Introduce a feature to allow opt-out of setting up
IMBUSCR. The default case is unchanged.

Signed-off-by: Magnus Damm <[email protected]>
---

Changes since V3:
- None

Changes since V2:
- None

Changes since V1:
- Updated the commit message
- Reworked patch to coexist with the multi context feature

drivers/iommu/ipmmu-vmsa.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)

--- 0025/drivers/iommu/ipmmu-vmsa.c
+++ work/drivers/iommu/ipmmu-vmsa.c 2017-06-19 14:07:06.850607110 +0900
@@ -39,6 +39,7 @@ struct ipmmu_features {
bool use_ns_alias_offset;
bool has_cache_leaf_nodes;
unsigned int number_of_contexts;
+ bool setup_imbuscr;
};

struct ipmmu_vmsa_device {
@@ -453,9 +454,10 @@ static int ipmmu_domain_init_context(str
domain->cfg.arm_lpae_s1_cfg.mair[0]);

/* IMBUSCR */
- ipmmu_ctx_write_root(domain, IMBUSCR,
- ipmmu_ctx_read_root(domain, IMBUSCR) &
- ~(IMBUSCR_DVM | IMBUSCR_BUSSEL_MASK));
+ if (domain->root->features->setup_imbuscr)
+ ipmmu_ctx_write_root(domain, IMBUSCR,
+ ipmmu_ctx_read_root(domain, IMBUSCR) &
+ ~(IMBUSCR_DVM | IMBUSCR_BUSSEL_MASK));

/*
* IMSTR
@@ -986,6 +988,7 @@ static const struct ipmmu_features ipmmu
.use_ns_alias_offset = true,
.has_cache_leaf_nodes = false,
.number_of_contexts = 1, /* software only tested with one context */
+ .setup_imbuscr = true,
};

static const struct of_device_id ipmmu_of_ids[] = {

2017-06-19 09:19:12

by Magnus Damm

[permalink] [raw]
Subject: [PATCH v4 08/09] iommu/ipmmu-vmsa: Allow two bit SL0

From: Magnus Damm <[email protected]>

Introduce support for two bit SL0 bitfield in IMTTBCR
by using a separate feature flag.

Signed-off-by: Magnus Damm <[email protected]>
---

Changes since V3:
- None

Changes since V2:
- None

Changes since V1:
- None

drivers/iommu/ipmmu-vmsa.c | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)

--- 0027/drivers/iommu/ipmmu-vmsa.c
+++ work/drivers/iommu/ipmmu-vmsa.c 2017-06-19 14:08:26.100607110 +0900
@@ -40,6 +40,7 @@ struct ipmmu_features {
bool has_cache_leaf_nodes;
unsigned int number_of_contexts;
bool setup_imbuscr;
+ bool twobit_imttbcr_sl0;
};

struct ipmmu_vmsa_device {
@@ -149,6 +150,10 @@ static struct ipmmu_vmsa_iommu_priv *to_
#define IMTTBCR_TSZ0_MASK (7 << 0)
#define IMTTBCR_TSZ0_SHIFT O

+#define IMTTBCR_SL0_TWOBIT_LVL_3 (0 << 6)
+#define IMTTBCR_SL0_TWOBIT_LVL_2 (1 << 6)
+#define IMTTBCR_SL0_TWOBIT_LVL_1 (2 << 6)
+
#define IMBUSCR 0x000c
#define IMBUSCR_DVM (1 << 2)
#define IMBUSCR_BUSSEL_SYS (0 << 0)
@@ -393,6 +398,7 @@ static int ipmmu_domain_allocate_context
static int ipmmu_domain_init_context(struct ipmmu_vmsa_domain *domain)
{
u64 ttbr;
+ u32 tmp;
int ret;

/*
@@ -445,9 +451,14 @@ static int ipmmu_domain_init_context(str
* We use long descriptors with inner-shareable WBWA tables and allocate
* the whole 32-bit VA space to TTBR0.
*/
+ if (domain->root->features->twobit_imttbcr_sl0)
+ tmp = IMTTBCR_SL0_TWOBIT_LVL_1;
+ else
+ tmp = IMTTBCR_SL0_LVL_1;
+
ipmmu_ctx_write_root(domain, IMTTBCR, IMTTBCR_EAE |
IMTTBCR_SH0_INNER_SHAREABLE | IMTTBCR_ORGN0_WB_WA |
- IMTTBCR_IRGN0_WB_WA | IMTTBCR_SL0_LVL_1);
+ IMTTBCR_IRGN0_WB_WA | tmp);

/* MAIR0 */
ipmmu_ctx_write_root(domain, IMMAIR0,
@@ -989,6 +1000,7 @@ static const struct ipmmu_features ipmmu
.has_cache_leaf_nodes = false,
.number_of_contexts = 1, /* software only tested with one context */
.setup_imbuscr = true,
+ .twobit_imttbcr_sl0 = false,
};

static const struct of_device_id ipmmu_of_ids[] = {

2017-06-19 09:19:10

by Magnus Damm

[permalink] [raw]
Subject: [PATCH v4 09/09] iommu/ipmmu-vmsa: Hook up r8a7795 DT matching code

From: Magnus Damm <[email protected]>

Tie in r8a7795 features and update the IOMMU_OF_DECLARE
compat string to include the updated compat string.

Signed-off-by: Magnus Damm <[email protected]>
---

Changes since V3:
- Rebased code on top of
[PATCH 00/04] iommu/ipmmu-vmsa: 32-bit ARM update
This includes support for iommu_fwspec_add_ids()
- Use dev_err() instead of dev_info() - Thanks Geert!
- Moved single-invokation check into of_xlate() - Thanks Geert!
- Dropped TODO list

Changes since V2:
- Check for lack of root device in ->xlate()
This fixed a bug when IPMMU-MM is disabled in DT the system hangs on boot
- Added code to ipmmu_init_platform_device() to handle multiple ->xlate() calls
- Include empty white list by default
- Updated TODO list

Changes since V1:
- Enable multi context feature
- Update TODO list

drivers/iommu/ipmmu-vmsa.c | 48 ++++++++++++++++++++++++++++++++++++++++----
1 file changed, 44 insertions(+), 4 deletions(-)

--- 0029/drivers/iommu/ipmmu-vmsa.c
+++ work/drivers/iommu/ipmmu-vmsa.c 2017-06-19 14:10:58.930607110 +0900
@@ -25,6 +25,7 @@
#include <linux/platform_device.h>
#include <linux/sizes.h>
#include <linux/slab.h>
+#include <linux/sys_soc.h>

#if defined(CONFIG_ARM) && !defined(CONFIG_IOMMU_DMA)
#include <asm/dma-iommu.h>
@@ -730,10 +731,6 @@ static int ipmmu_init_platform_device(st
struct platform_device *ipmmu_pdev;
struct ipmmu_vmsa_iommu_priv *priv;

- /* Initialize once - xlate() will call multiple times */
- if (to_priv(dev))
- return 0;
-
ipmmu_pdev = of_find_device_by_node(args->np);
if (!ipmmu_pdev)
return -ENODEV;
@@ -748,11 +745,41 @@ static int ipmmu_init_platform_device(st
return 0;
}

+static bool ipmmu_slave_whitelist(struct device *dev)
+{
+ /* By default, do not allow use of IPMMU */
+ return false;
+}
+
+static const struct soc_device_attribute soc_r8a7795[] = {
+ { .soc_id = "r8a7795", },
+ { /* sentinel */ }
+};
+
static int ipmmu_of_xlate(struct device *dev,
struct of_phandle_args *spec)
{
+ /* Failing in ->attach_device() results in a hang, so make
+ * sure the root device is installed before going there
+ */
+ if (!__ipmmu_find_root()) {
+ dev_err(dev, "Unable to locate IPMMU root device\n");
+ return -ENODEV;
+ }
+
+ /* For R-Car Gen3 use a white list to opt-in slave devices */
+ if (soc_device_match(soc_r8a7795) && !ipmmu_slave_whitelist(dev))
+ return -ENODEV;
+
iommu_fwspec_add_ids(dev, spec->args, 1);

+ /*
+ * Execute ipmmu_init_platform_device() once per device.
+ * This xlate() callback will be invoked multiple times.
+ */
+ if (to_priv(dev))
+ return 0;
+
return ipmmu_init_platform_device(dev, spec);
}

@@ -1003,11 +1030,22 @@ static const struct ipmmu_features ipmmu
.twobit_imttbcr_sl0 = false,
};

+static const struct ipmmu_features ipmmu_features_r8a7795 = {
+ .use_ns_alias_offset = false,
+ .has_cache_leaf_nodes = true,
+ .number_of_contexts = 8,
+ .setup_imbuscr = false,
+ .twobit_imttbcr_sl0 = true,
+};
+
static const struct of_device_id ipmmu_of_ids[] = {
{
.compatible = "renesas,ipmmu-vmsa",
.data = &ipmmu_features_default,
}, {
+ .compatible = "renesas,ipmmu-r8a7795",
+ .data = &ipmmu_features_r8a7795,
+ }, {
/* Terminator */
},
};
@@ -1185,6 +1223,8 @@ static int __init ipmmu_vmsa_iommu_of_se

IOMMU_OF_DECLARE(ipmmu_vmsa_iommu_of, "renesas,ipmmu-vmsa",
ipmmu_vmsa_iommu_of_setup);
+IOMMU_OF_DECLARE(ipmmu_r8a7795_iommu_of, "renesas,ipmmu-r8a7795",
+ ipmmu_vmsa_iommu_of_setup);
#endif

MODULE_DESCRIPTION("IOMMU API for Renesas VMSA-compatible IPMMU");

2017-06-19 17:19:15

by Robin Murphy

[permalink] [raw]
Subject: Re: [PATCH v4 02/09] iommu/ipmmu-vmsa: Add optional root device feature

On 19/06/17 10:14, Magnus Damm wrote:
> From: Magnus Damm <[email protected]>
>
> Add root device handling to the IPMMU driver by allowing certain
> DT compat strings to enable has_cache_leaf_nodes that in turn will
> support both root devices with interrupts and leaf devices that
> face the actual IPMMU consumer devices.
>
> Signed-off-by: Magnus Damm <[email protected]>
> ---
>
> Changes since V3:
> - Reworked root finding code to make it easier to follow, thanks Geert!
>
> Changes since V2:
> - Fixed a bug in ipmmu_find_root() when only leaf devices are present
> - Broke out __ipmmu_find_root() to allow ->xlate() check for root devices
>
> Changes since V1:
> - Moved patch to earlier in the series
> - Updated code to work with recent changes in:
> [PATCH v3 00/06] iommu/ipmmu-vmsa: IPMMU multi-arch update V3
>
> drivers/iommu/ipmmu-vmsa.c | 95 ++++++++++++++++++++++++++++++++++++--------
> 1 file changed, 78 insertions(+), 17 deletions(-)
>
> --- 0015/drivers/iommu/ipmmu-vmsa.c
> +++ work/drivers/iommu/ipmmu-vmsa.c 2017-06-19 13:59:41.050607110 +0900
> @@ -36,6 +36,7 @@
>
> struct ipmmu_features {
> bool use_ns_alias_offset;
> + bool has_cache_leaf_nodes;
> };
>
> struct ipmmu_vmsa_device {
> @@ -44,6 +45,7 @@ struct ipmmu_vmsa_device {
> struct iommu_device iommu;
> struct list_head list;
> const struct ipmmu_features *features;
> + bool is_leaf;
> unsigned int num_utlbs;
> spinlock_t lock; /* Protects ctx and domains[] */
> DECLARE_BITMAP(ctx, IPMMU_CTX_MAX);
> @@ -54,6 +56,7 @@ struct ipmmu_vmsa_device {
>
> struct ipmmu_vmsa_domain {
> struct ipmmu_vmsa_device *mmu;
> + struct ipmmu_vmsa_device *root;

Would it not make more sense for this to be a property of the
ipmmu_device itself, rather than per ipmmu_domain? I may of course have
got the wrong idea of the topology here, but it seems as if mmu->is_leaf
could be expressed as mmu->root == mmu vs. mmu->root == some_other_mmu,
at which point there's one less thing to worry about in the domain.

> struct iommu_domain io_domain;
>
> struct io_pgtable_cfg cfg;
> @@ -203,6 +206,44 @@ static struct ipmmu_vmsa_iommu_priv *to_
> #define IMUASID_ASID0_SHIFT 0
>
> /* -----------------------------------------------------------------------------
> + * Root device handling
> + */
> +
> +static bool ipmmu_is_root(struct ipmmu_vmsa_device *mmu)
> +{
> + if (mmu->features->has_cache_leaf_nodes)
> + return mmu->is_leaf ? false : true> + else
> + return true; /* older IPMMU hardware treated as single root */
> +}
> +
> +static struct ipmmu_vmsa_device *__ipmmu_find_root(void)
> +{
> + struct ipmmu_vmsa_device *mmu;
> + struct ipmmu_vmsa_device *root = NULL;
> +
> + spin_lock(&ipmmu_devices_lock);
> +
> + list_for_each_entry(mmu, &ipmmu_devices, list) {
> + if (ipmmu_is_root(mmu)) {
> + root = mmu;
> + break;
> + }
> + }
> +
> + spin_unlock(&ipmmu_devices_lock);
> + return root;
> +}

I wonder if it might be tidier to use driver_for_each_device() for this,
and remove the local list at the same time as its previous user.

Either way, what happens if things end up hapening in this order:

1: probe leaf IPMMU B
2: probe device X behind IPMMU B
3: create and attach default domain for device X
4: probe root IPMMU A

We know X will defer if B isn't ready, but it doesn't seem (at a glance,
admittedly) that there's anything to enforce the expected probe ordering
between A and B. This seems like another argument for moving the
root/leaf association up to the device level, such that B can look up A
once in its own probe routine, and defer itself if necessary.

> +
> +static struct ipmmu_vmsa_device *ipmmu_find_root(struct ipmmu_vmsa_device *leaf)
> +{
> + if (ipmmu_is_root(leaf))
> + return leaf;
> + else
> + return __ipmmu_find_root();

Well actually, looking at that, I think what we have here is a very
long-winded way of implementing this:

static ipmmu_vmsa_device *root;

Yuck.

> +}
> +
> +/* -----------------------------------------------------------------------------
> * Read/Write Access
> */
>
> @@ -219,13 +260,13 @@ static void ipmmu_write(struct ipmmu_vms
>
> static u32 ipmmu_ctx_read(struct ipmmu_vmsa_domain *domain, unsigned int reg)
> {
> - return ipmmu_read(domain->mmu, domain->context_id * IM_CTX_SIZE + reg);
> + return ipmmu_read(domain->root, domain->context_id * IM_CTX_SIZE + reg);
> }
>
> static void ipmmu_ctx_write(struct ipmmu_vmsa_domain *domain, unsigned int reg,
> u32 data)
> {
> - ipmmu_write(domain->mmu, domain->context_id * IM_CTX_SIZE + reg, data);
> + ipmmu_write(domain->root, domain->context_id * IM_CTX_SIZE + reg, data);
> }
>
> /* -----------------------------------------------------------------------------
> @@ -360,7 +401,7 @@ static int ipmmu_domain_init_context(str
> * TODO: Add support for coherent walk through CCI with DVM and remove
> * cache handling. For now, delegate it to the io-pgtable code.
> */
> - domain->cfg.iommu_dev = domain->mmu->dev;
> + domain->cfg.iommu_dev = domain->root->dev;
>
> domain->iop = alloc_io_pgtable_ops(ARM_32_LPAE_S1, &domain->cfg,
> domain);
> @@ -370,7 +411,7 @@ static int ipmmu_domain_init_context(str
> /*
> * Find an unused context.
> */
> - ret = ipmmu_domain_allocate_context(domain->mmu, domain);
> + ret = ipmmu_domain_allocate_context(domain->root, domain);
> if (ret == IPMMU_CTX_MAX) {
> free_io_pgtable_ops(domain->iop);
> return -EBUSY;
> @@ -441,7 +482,7 @@ static void ipmmu_domain_destroy_context
> */
> ipmmu_ctx_write(domain, IMCTR, IMCTR_FLUSH);
> ipmmu_tlb_sync(domain);
> - ipmmu_domain_free_context(domain->mmu, domain->context_id);
> + ipmmu_domain_free_context(domain->root, domain->context_id);
> }
>
> /* -----------------------------------------------------------------------------
> @@ -555,7 +596,7 @@ static int ipmmu_attach_device(struct io
> {
> struct ipmmu_vmsa_iommu_priv *priv = to_priv(dev);
> struct iommu_fwspec *fwspec = dev->iommu_fwspec;
> - struct ipmmu_vmsa_device *mmu = priv->mmu;
> + struct ipmmu_vmsa_device *root, *mmu = priv->mmu;
> struct ipmmu_vmsa_domain *domain = to_vmsa_domain(io_domain);
> unsigned long flags;
> unsigned int i;
> @@ -566,11 +607,18 @@ static int ipmmu_attach_device(struct io
> return -ENXIO;
> }
>
> + root = ipmmu_find_root(priv->mmu);
> + if (!root) {
> + dev_err(dev, "Unable to locate root IPMMU\n");
> + return -EAGAIN;
> + }
> +
> spin_lock_irqsave(&domain->lock, flags);
>
> if (!domain->mmu) {
> /* The domain hasn't been used yet, initialize it. */
> domain->mmu = mmu;
> + domain->root = root;
> ret = ipmmu_domain_init_context(domain);
> } else if (domain->mmu != mmu) {
> /*
> @@ -911,6 +959,7 @@ static void ipmmu_device_reset(struct ip
>
> static const struct ipmmu_features ipmmu_features_default = {
> .use_ns_alias_offset = true,
> + .has_cache_leaf_nodes = false,
> };
>
> static const struct of_device_id ipmmu_of_ids[] = {
> @@ -965,19 +1014,31 @@ static int ipmmu_probe(struct platform_d
> mmu->base += IM_NS_ALIAS_OFFSET;
>
> irq = platform_get_irq(pdev, 0);
> - if (irq < 0) {
> - dev_err(&pdev->dev, "no IRQ found\n");
> - return irq;
> - }
>
> - ret = devm_request_irq(&pdev->dev, irq, ipmmu_irq, 0,
> - dev_name(&pdev->dev), mmu);
> - if (ret < 0) {
> - dev_err(&pdev->dev, "failed to request IRQ %d\n", irq);
> - return ret;
> - }
> + /*
> + * Determine if this IPMMU instance is a leaf device by checking
> + * if the renesas,ipmmu-main property exists or not.
> + */
> + if (mmu->features->has_cache_leaf_nodes &&
> + of_find_property(pdev->dev.of_node, "renesas,ipmmu-main", NULL))
> + mmu->is_leaf = true;

Given my previous comments:

if (mmu->features->has_cache_leaf_nodes) {
root_np = of_parse_phandle(pdev->dev.of_node,
"renesas,ipmmu-main", 0)
if (root_np) {
root_pdev = of_find_device_by_phandle(root_np);
mmu->root = platform_get_drvdata(root_pdev);
} else {
mmu->root = mmu;
}
}

or something along those lines?

Robin.

> +
> + /* Root devices have mandatory IRQs */
> + if (ipmmu_is_root(mmu)) {
> + if (irq < 0) {
> + dev_err(&pdev->dev, "no IRQ found\n");
> + return irq;
> + }
>
> - ipmmu_device_reset(mmu);
> + ret = devm_request_irq(&pdev->dev, irq, ipmmu_irq, 0,
> + dev_name(&pdev->dev), mmu);
> + if (ret < 0) {
> + dev_err(&pdev->dev, "failed to request IRQ %d\n", irq);
> + return ret;
> + }
> +
> + ipmmu_device_reset(mmu);
> + }
>
> ret = iommu_device_register(&mmu->iommu);
> if (ret)
>