2011-05-11 22:25:10

by Nishanth Aravamudan

[permalink] [raw]
Subject: [PATCH 0/8] pseries/iommu: bug-fixes and cleanups for dynamic dma windows

This series of patches attempts to cleanup and fix some bugs related to
dynamic dma windows and kexec/kdump. They build on the three patches I
have submitted most recently:

powerpc: fix kexec with dynamic dma windows
http://patchwork.ozlabs.org/patch/94445/
pseries/iommu: restore iommu table pointer when restoring iommu ops
http://patchwork.ozlabs.org/patch/94909/
pseries/iommu: use correct return type in dupe_ddw_if_already_created
http://patchwork.ozlabs.org/patch/95184/

The full series has been successfully tested with kdump/kexec on pseries
machine.

Milton Miller (8):
pseries/iommu: add additional checks when changing iommu mask
pseries/iommu: remove ddw property when destroying window
pseries/iommu: find windows after kexec during boot
pseries/iommu: cleanup ddw naming
powerpc: override dma_get_required_mask by platform hook and ops
dma-mapping: add get_required_mask if arch overrides default
powerpc: use the newly added get_required_mask dma_map_ops hook
powerpc: tidy up dma_map_ops after adding new hook

arch/powerpc/include/asm/device.h | 2 +
arch/powerpc/include/asm/machdep.h | 3 +-
arch/powerpc/kernel/dma-iommu.c | 28 +++++--
arch/powerpc/kernel/dma-swiotlb.c | 16 ++++
arch/powerpc/kernel/dma.c | 44 ++++++++--
arch/powerpc/kernel/ibmebus.c | 22 +++--
arch/powerpc/kernel/vio.c | 21 +++--
arch/powerpc/platforms/cell/iommu.c | 21 +++++
arch/powerpc/platforms/ps3/system-bus.c | 7 ++
arch/powerpc/platforms/pseries/iommu.c | 142 +++++++++++++++++++------------
include/linux/dma-mapping.h | 3 +
11 files changed, 224 insertions(+), 85 deletions(-)

--
1.7.4.1


2011-05-11 22:25:13

by Nishanth Aravamudan

[permalink] [raw]
Subject: [PATCH 1/8] pseries/iommu: add additional checks when changing iommu mask

From: Milton Miller <[email protected]>

Do not check dma supported until we have chosen the right dma ops.
Check that the device is pci before treating it as such.

Check the mask is supported by the selected dma ops before
committing it.

We only need to set iommu ops if it is not the current ops; this
avoids searching the tree for the iommu table unnecessarily.

Signed-off-by: Milton Miller <[email protected]>
Signed-off-by: Nishanth Aravamudan <[email protected]>
---
arch/powerpc/platforms/pseries/iommu.c | 15 +++++++++++----
1 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 44d47ac..05c101e 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -1026,9 +1026,12 @@ static int dma_set_mask_pSeriesLP(struct device *dev, u64 dma_mask)
const void *dma_window = NULL;
u64 dma_offset;

- if (!dev->dma_mask || !dma_supported(dev, dma_mask))
+ if (!dev->dma_mask)
return -EIO;

+ if (!dev_is_pci(dev))
+ goto check_mask;
+
pdev = to_pci_dev(dev);

/* only attempt to use a new window if 64-bit DMA is requested */
@@ -1059,13 +1062,17 @@ static int dma_set_mask_pSeriesLP(struct device *dev, u64 dma_mask)
}
}

- /* fall-through to iommu ops */
- if (!ddw_enabled) {
- dev_info(dev, "Using 32-bit DMA via iommu\n");
+ /* fall back on iommu ops, restore table pointer with ops */
+ if (!ddw_enabled && get_dma_ops(dev) != &dma_iommu_ops) {
+ dev_info(dev, "Restoring 32-bit DMA via iommu\n");
set_dma_ops(dev, &dma_iommu_ops);
pci_dma_dev_setup_pSeriesLP(pdev);
}

+check_mask:
+ if (!dma_supported(dev, dma_mask))
+ return -EIO;
+
*dev->dma_mask = dma_mask;
return 0;
}
--
1.7.4.1

2011-05-11 22:25:23

by Nishanth Aravamudan

[permalink] [raw]
Subject: [PATCH 2/8] pseries/iommu: remove ddw property when destroying window

From: Milton Miller <[email protected]>

If we destroy the window, we need to remove the property recording that
we setup the window. Otherwise the next kernel we kexec will be
confused.

Also we should remove the property if even if we don't find the
ibm,ddw-applicable window or if one of the property sizes is unexpected;
presumably these came from a prior kernel via kexec, and we will not be
maintaining the window with respect to memory hotplug.

Signed-off-by: Milton Miller <[email protected]>
Signed-off-by: Nishanth Aravamudan <[email protected]>
---
arch/powerpc/platforms/pseries/iommu.c | 12 ++++++++++--
1 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 05c101e..a0421ac 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -665,9 +665,12 @@ static void remove_ddw(struct device_node *np)

ddr_avail = of_get_property(np, "ibm,ddw-applicable", &len);
win64 = of_find_property(np, DIRECT64_PROPNAME, NULL);
- if (!win64 || !ddr_avail || len < 3 * sizeof(u32))
+ if (!win64)
return;

+ if (!ddr_avail || len < 3 * sizeof(u32) || win64->length < sizeof(*dwp))
+ goto delprop;
+
dwp = win64->value;
liobn = (u64)be32_to_cpu(dwp->liobn);

@@ -690,8 +693,13 @@ static void remove_ddw(struct device_node *np)
pr_debug("%s: successfully removed direct window: rtas returned "
"%d to ibm,remove-pe-dma-window(%x) %llx\n",
np->full_name, ret, ddr_avail[2], liobn);
-}

+delprop:
+ ret = of_remove_property(np, win64);
+ if (ret)
+ pr_warning("%s: failed to remove direct window property: %d\n"
+ np->full_name, ret);
+}

static u64 dupe_ddw_if_already_created(struct pci_dev *dev, struct device_node *pdn)
{
--
1.7.4.1

2011-05-11 22:25:15

by Nishanth Aravamudan

[permalink] [raw]
Subject: [PATCH 3/8] pseries/iommu: find windows after kexec during boot

From: Milton Miller <[email protected]>

Move the discovery of windows previously setup from when the pci driver
calls set_dma_mask to an arch_initcall.

When kexecing into a kernel with dynamic dma windows allocated, we need
to find the windows early so that memory hot remove will be able to
delete the tces mapping the to be removed memory and memory hotplug add
will map the new memory into the window. We should not wait for the
driver to be loaded and the device to be probed. The iommu init hooks
are before kmalloc is setup, so defer to arch_initcall.

Signed-off-by: Milton Miller <[email protected]>
Signed-off-by: Nishanth Aravamudan <[email protected]>
---
arch/powerpc/platforms/pseries/iommu.c | 52 ++++++++++++++-----------------
1 files changed, 24 insertions(+), 28 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index a0421ac..a48f126 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -695,9 +695,9 @@ static void remove_ddw(struct device_node *np)
np->full_name, ret, ddr_avail[2], liobn);

delprop:
- ret = of_remove_property(np, win64);
+ ret = prom_remove_property(np, win64);
if (ret)
- pr_warning("%s: failed to remove direct window property: %d\n"
+ pr_warning("%s: failed to remove direct window property: %d\n",
np->full_name, ret);
}

@@ -725,38 +725,38 @@ static u64 dupe_ddw_if_already_created(struct pci_dev *dev, struct device_node *
return dma_addr;
}

-static u64 dupe_ddw_if_kexec(struct pci_dev *dev, struct device_node *pdn)
+static int find_existing_ddw_windows(void)
{
- struct device_node *dn;
- struct pci_dn *pcidn;
int len;
+ struct device_node *pdn;
struct direct_window *window;
const struct dynamic_dma_window_prop *direct64;
- u64 dma_addr = 0;

- dn = pci_device_to_OF_node(dev);
- pcidn = PCI_DN(dn);
- direct64 = of_get_property(pdn, DIRECT64_PROPNAME, &len);
- if (direct64) {
- if (len < sizeof(struct dynamic_dma_window_prop)) {
+ if (!firmware_has_feature(FW_FEATURE_LPAR))
+ return 0;
+
+ for_each_node_with_property(pdn, DIRECT64_PROPNAME) {
+ direct64 = of_get_property(pdn, DIRECT64_PROPNAME, &len);
+ if (!direct64)
+ continue;
+
+ window = kzalloc(sizeof(*window), GFP_KERNEL);
+ if (!window || len < sizeof(struct dynamic_dma_window_prop)) {
+ kfree(window);
remove_ddw(pdn);
- } else {
- window = kzalloc(sizeof(*window), GFP_KERNEL);
- if (!window) {
- remove_ddw(pdn);
- } else {
- window->device = pdn;
- window->prop = direct64;
- spin_lock(&direct_window_list_lock);
- list_add(&window->list, &direct_window_list);
- spin_unlock(&direct_window_list_lock);
- dma_addr = direct64->dma_base;
- }
+ continue;
}
+
+ window->device = pdn;
+ window->prop = direct64;
+ spin_lock(&direct_window_list_lock);
+ list_add(&window->list, &direct_window_list);
+ spin_unlock(&direct_window_list_lock);
}

- return dma_addr;
+ return 0;
}
+machine_arch_initcall(pseries, find_existing_ddw_windows);

static int query_ddw(struct pci_dev *dev, const u32 *ddr_avail,
struct ddw_query_response *query)
@@ -854,10 +854,6 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)
if (dma_addr != 0)
goto out_unlock;

- dma_addr = dupe_ddw_if_kexec(dev, pdn);
- if (dma_addr != 0)
- goto out_unlock;
-
/*
* the ibm,ddw-applicable property holds the tokens for:
* ibm,query-pe-dma-window
--
1.7.4.1

2011-05-11 22:26:30

by Nishanth Aravamudan

[permalink] [raw]
Subject: [PATCH 4/8] pseries/iommu: cleanup ddw naming

From: Milton Miller <[email protected]>

When using a property refering to the availibily of dynamic dma windows
call it ddw_avail not ddr_avail.

dupe_ddw_if_already_created does not dupilcate anything, it only finds
and reuses the windows we already created, so rename it to
find_existing_ddw. Also, it does not need the pci device node, so
remove that argument.

Signed-off-by: Milton Miller <[email protected]>
Signed-off-by: Nishanth Aravamudan <[email protected]>
---
arch/powerpc/platforms/pseries/iommu.c | 42 ++++++++++++++-----------------
1 files changed, 19 insertions(+), 23 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index a48f126..01faab9 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -659,16 +659,16 @@ static void remove_ddw(struct device_node *np)
{
struct dynamic_dma_window_prop *dwp;
struct property *win64;
- const u32 *ddr_avail;
+ const u32 *ddw_avail;
u64 liobn;
int len, ret;

- ddr_avail = of_get_property(np, "ibm,ddw-applicable", &len);
+ ddw_avail = of_get_property(np, "ibm,ddw-applicable", &len);
win64 = of_find_property(np, DIRECT64_PROPNAME, NULL);
if (!win64)
return;

- if (!ddr_avail || len < 3 * sizeof(u32) || win64->length < sizeof(*dwp))
+ if (!ddw_avail || len < 3 * sizeof(u32) || win64->length < sizeof(*dwp))
goto delprop;

dwp = win64->value;
@@ -684,15 +684,15 @@ static void remove_ddw(struct device_node *np)
pr_debug("%s successfully cleared tces in window.\n",
np->full_name);

- ret = rtas_call(ddr_avail[2], 1, 1, NULL, liobn);
+ ret = rtas_call(ddw_avail[2], 1, 1, NULL, liobn);
if (ret)
pr_warning("%s: failed to remove direct window: rtas returned "
"%d to ibm,remove-pe-dma-window(%x) %llx\n",
- np->full_name, ret, ddr_avail[2], liobn);
+ np->full_name, ret, ddw_avail[2], liobn);
else
pr_debug("%s: successfully removed direct window: rtas returned "
"%d to ibm,remove-pe-dma-window(%x) %llx\n",
- np->full_name, ret, ddr_avail[2], liobn);
+ np->full_name, ret, ddw_avail[2], liobn);

delprop:
ret = prom_remove_property(np, win64);
@@ -701,16 +701,12 @@ delprop:
np->full_name, ret);
}

-static u64 dupe_ddw_if_already_created(struct pci_dev *dev, struct device_node *pdn)
+static u64 find_existing_ddw(struct device_node *pdn)
{
- struct device_node *dn;
- struct pci_dn *pcidn;
struct direct_window *window;
const struct dynamic_dma_window_prop *direct64;
u64 dma_addr = 0;

- dn = pci_device_to_OF_node(dev);
- pcidn = PCI_DN(dn);
spin_lock(&direct_window_list_lock);
/* check if we already created a window and dupe that config if so */
list_for_each_entry(window, &direct_window_list, list) {
@@ -758,7 +754,7 @@ static int find_existing_ddw_windows(void)
}
machine_arch_initcall(pseries, find_existing_ddw_windows);

-static int query_ddw(struct pci_dev *dev, const u32 *ddr_avail,
+static int query_ddw(struct pci_dev *dev, const u32 *ddw_avail,
struct ddw_query_response *query)
{
struct device_node *dn;
@@ -779,15 +775,15 @@ static int query_ddw(struct pci_dev *dev, const u32 *ddr_avail,
if (pcidn->eeh_pe_config_addr)
cfg_addr = pcidn->eeh_pe_config_addr;
buid = pcidn->phb->buid;
- ret = rtas_call(ddr_avail[0], 3, 5, (u32 *)query,
+ ret = rtas_call(ddw_avail[0], 3, 5, (u32 *)query,
cfg_addr, BUID_HI(buid), BUID_LO(buid));
dev_info(&dev->dev, "ibm,query-pe-dma-windows(%x) %x %x %x"
- " returned %d\n", ddr_avail[0], cfg_addr, BUID_HI(buid),
+ " returned %d\n", ddw_avail[0], cfg_addr, BUID_HI(buid),
BUID_LO(buid), ret);
return ret;
}

-static int create_ddw(struct pci_dev *dev, const u32 *ddr_avail,
+static int create_ddw(struct pci_dev *dev, const u32 *ddw_avail,
struct ddw_create_response *create, int page_shift,
int window_shift)
{
@@ -812,12 +808,12 @@ static int create_ddw(struct pci_dev *dev, const u32 *ddr_avail,

do {
/* extra outputs are LIOBN and dma-addr (hi, lo) */
- ret = rtas_call(ddr_avail[1], 5, 4, (u32 *)create, cfg_addr,
+ ret = rtas_call(ddw_avail[1], 5, 4, (u32 *)create, cfg_addr,
BUID_HI(buid), BUID_LO(buid), page_shift, window_shift);
} while (rtas_busy_delay(ret));
dev_info(&dev->dev,
"ibm,create-pe-dma-window(%x) %x %x %x %x %x returned %d "
- "(liobn = 0x%x starting addr = %x %x)\n", ddr_avail[1],
+ "(liobn = 0x%x starting addr = %x %x)\n", ddw_avail[1],
cfg_addr, BUID_HI(buid), BUID_LO(buid), page_shift,
window_shift, ret, create->liobn, create->addr_hi, create->addr_lo);

@@ -843,14 +839,14 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)
int page_shift;
u64 dma_addr, max_addr;
struct device_node *dn;
- const u32 *uninitialized_var(ddr_avail);
+ const u32 *uninitialized_var(ddw_avail);
struct direct_window *window;
struct property *win64;
struct dynamic_dma_window_prop *ddwprop;

mutex_lock(&direct_window_init_mutex);

- dma_addr = dupe_ddw_if_already_created(dev, pdn);
+ dma_addr = find_existing_ddw(pdn);
if (dma_addr != 0)
goto out_unlock;

@@ -862,8 +858,8 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)
* for the given node in that order.
* the property is actually in the parent, not the PE
*/
- ddr_avail = of_get_property(pdn, "ibm,ddw-applicable", &len);
- if (!ddr_avail || len < 3 * sizeof(u32))
+ ddw_avail = of_get_property(pdn, "ibm,ddw-applicable", &len);
+ if (!ddw_avail || len < 3 * sizeof(u32))
goto out_unlock;

/*
@@ -873,7 +869,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)
* of page sizes: supported and supported for migrate-dma.
*/
dn = pci_device_to_OF_node(dev);
- ret = query_ddw(dev, ddr_avail, &query);
+ ret = query_ddw(dev, ddw_avail, &query);
if (ret != 0)
goto out_unlock;

@@ -922,7 +918,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn)
goto out_free_prop;
}

- ret = create_ddw(dev, ddr_avail, &create, page_shift, len);
+ ret = create_ddw(dev, ddw_avail, &create, page_shift, len);
if (ret != 0)
goto out_free_prop;

--
1.7.4.1

2011-05-11 22:25:21

by Nishanth Aravamudan

[permalink] [raw]
Subject: [PATCH 5/8] powerpc: override dma_get_required_mask by platform hook and ops

From: Milton Miller <[email protected]>

The hook dma_get_required_mask is supposed to return the mask required
by the platform to operate efficently. The generic version of
dma_get_required_mask in driver/base/platform.c returns a mask based
only on max_pfn. However, this is likely too big for iommu systems
and could be too small for platforms that require a dma offset or have
a secondary window at a high offset.

Override the default, provide a hook in ppc_md used by pseries lpar and
cell, and provide the default answer based on memblock_end_of_DRAM(),
with hooks for get_dma_offset, and provide an implementation for iommu
that looks at the defined table size. Coverting from the end address
to the required bit mask is based on the generic implementation.

The need for this was discovered when the qla2xxx driver switched to
64 bit dma then reverted to 32 bit when dma_get_required_mask said
32 bits was sufficient.

Signed-off-by: Milton Miller <[email protected]>
Signed-off-by: Nishanth Aravamudan <[email protected]>
---
arch/powerpc/include/asm/dma-mapping.h | 3 ++
arch/powerpc/include/asm/machdep.h | 3 +-
arch/powerpc/kernel/dma-iommu.c | 13 ++++++++++
arch/powerpc/kernel/dma.c | 39 ++++++++++++++++++++++++++++++++
arch/powerpc/platforms/cell/iommu.c | 12 +++++++++
arch/powerpc/platforms/pseries/iommu.c | 27 ++++++++++++++++++++++
6 files changed, 96 insertions(+), 1 deletions(-)

diff --git a/arch/powerpc/include/asm/dma-mapping.h b/arch/powerpc/include/asm/dma-mapping.h
index dd70fac..8135e66 100644
--- a/arch/powerpc/include/asm/dma-mapping.h
+++ b/arch/powerpc/include/asm/dma-mapping.h
@@ -20,6 +20,8 @@

#define DMA_ERROR_CODE (~(dma_addr_t)0x0)

+#define ARCH_HAS_DMA_GET_REQUIRED_MASK
+
/* Some dma direct funcs must be visible for use in other dma_ops */
extern void *dma_direct_alloc_coherent(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t flag);
@@ -69,6 +71,7 @@ static inline unsigned long device_to_mask(struct device *dev)
*/
#ifdef CONFIG_PPC64
extern struct dma_map_ops dma_iommu_ops;
+extern u64 dma_iommu_get_required_mask(struct device *dev);
#endif
extern struct dma_map_ops dma_direct_ops;

diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h
index e4f0191..5837881 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -100,8 +100,9 @@ struct machdep_calls {
void (*pci_dma_dev_setup)(struct pci_dev *dev);
void (*pci_dma_bus_setup)(struct pci_bus *bus);

- /* Platform set_dma_mask override */
+ /* Platform set_dma_mask and dma_get_required_mask overrides */
int (*dma_set_mask)(struct device *dev, u64 dma_mask);
+ u64 (*dma_get_required_mask)(struct device *dev);

int (*probe)(void);
void (*setup_arch)(void); /* Optional, may be NULL */
diff --git a/arch/powerpc/kernel/dma-iommu.c b/arch/powerpc/kernel/dma-iommu.c
index e755415..1f2a711 100644
--- a/arch/powerpc/kernel/dma-iommu.c
+++ b/arch/powerpc/kernel/dma-iommu.c
@@ -90,6 +90,19 @@ static int dma_iommu_dma_supported(struct device *dev, u64 mask)
return 1;
}

+u64 dma_iommu_get_required_mask(struct device *dev)
+{
+ struct iommu_table *tbl = get_iommu_table_base(dev);
+ u64 mask;
+ if (!tbl)
+ return 0;
+
+ mask = 1ULL < (fls_long(tbl->it_offset + tbl->it_size) - 1);
+ mask += mask - 1;
+
+ return mask;
+}
+
struct dma_map_ops dma_iommu_ops = {
.alloc_coherent = dma_iommu_alloc_coherent,
.free_coherent = dma_iommu_free_coherent,
diff --git a/arch/powerpc/kernel/dma.c b/arch/powerpc/kernel/dma.c
index d238c08..97fe867 100644
--- a/arch/powerpc/kernel/dma.c
+++ b/arch/powerpc/kernel/dma.c
@@ -172,6 +172,45 @@ int dma_set_mask(struct device *dev, u64 dma_mask)
}
EXPORT_SYMBOL(dma_set_mask);

+u64 dma_get_required_mask(struct device *dev)
+{
+ struct dma_map_ops *dma_ops = get_dma_ops(dev);
+ u64 mask, end = 0;
+
+ if (ppc_md.dma_get_required_mask)
+ return ppc_md.dma_get_required_mask(dev);
+
+ if (unlikely(dma_ops == NULL))
+ return 0;
+
+#ifdef CONFIG_PPC64
+ else if (dma_ops == &dma_iommu_ops)
+ return dma_iommu_get_required_mask(dev);
+#endif
+#ifdef CONFIG_SWIOTLB
+ else if (dma_ops == &swiotlb_dma_ops) {
+ u64 max_direct_dma_addr = dev->archdata.max_direct_dma_addr;
+
+ end = memblock_end_of_DRAM();
+ if (max_direct_dma_addr && end > max_direct_dma_addr)
+ end = max_direct_dma_addr;
+ end += get_dma_offset(dev);
+ }
+#endif
+ else if (dma_ops == &dma_direct_ops)
+ end = memblock_end_of_DRAM() + get_dma_offset(dev);
+ else {
+ WARN_ONCE(1, "%s: unknown ops %p\n", __func__, dma_ops);
+ end = memblock_end_of_DRAM();
+ }
+
+ mask = 1ULL << (fls64(end) - 1);
+ mask += mask - 1;
+
+ return mask;
+}
+EXPORT_SYMBOL_GPL(dma_get_required_mask);
+
static int __init dma_init(void)
{
dma_debug_init(PREALLOC_DMA_DEBUG_ENTRIES);
diff --git a/arch/powerpc/platforms/cell/iommu.c b/arch/powerpc/platforms/cell/iommu.c
index 26a0671..5ef55f3 100644
--- a/arch/powerpc/platforms/cell/iommu.c
+++ b/arch/powerpc/platforms/cell/iommu.c
@@ -1159,6 +1159,17 @@ static int __init setup_iommu_fixed(char *str)
}
__setup("iommu_fixed=", setup_iommu_fixed);

+static u64 cell_dma_get_required_mask(struct device *dev)
+{
+ if (!dev->dma_mask)
+ return 0;
+
+ if (iommu_fixed_disabled && get_dma_ops(dev) == &dma_iommu_ops)
+ return dma_iommu_get_required_mask(dev);
+
+ return DMA_BIT_MASK(64);
+}
+
static int __init cell_iommu_init(void)
{
struct device_node *np;
@@ -1175,6 +1186,7 @@ static int __init cell_iommu_init(void)

/* Setup various ppc_md. callbacks */
ppc_md.pci_dma_dev_setup = cell_pci_dma_dev_setup;
+ ppc_md.dma_get_required_mask = cell_dma_get_required_mask;
ppc_md.tce_build = tce_build_cell;
ppc_md.tce_free = tce_free_cell;

diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 01faab9..fe5eded 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -1077,12 +1077,38 @@ check_mask:
return 0;
}

+static u64 dma_get_required_mask_pSeriesLP(struct device *dev)
+{
+ if (!dev->dma_mask)
+ return 0;
+
+ if (!disable_ddw && dev_is_pci(dev)) {
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct device_node *dn;
+
+ dn = pci_device_to_OF_node(pdev);
+
+ /* search upwards for ibm,dma-window */
+ for (; dn && PCI_DN(dn) && !PCI_DN(dn)->iommu_table;
+ dn = dn->parent)
+ if (of_get_property(dn, "ibm,dma-window", NULL))
+ break;
+ /* if there is a ibm,ddw-applicable property require 64 bits */
+ if (dn && PCI_DN(dn) &&
+ of_get_property(dn, "ibm,ddw-applicable", NULL))
+ return DMA_BIT_MASK(64);
+ }
+
+ return dma_iommu_get_required_mask(dev);
+}
+
#else /* CONFIG_PCI */
#define pci_dma_bus_setup_pSeries NULL
#define pci_dma_dev_setup_pSeries NULL
#define pci_dma_bus_setup_pSeriesLP NULL
#define pci_dma_dev_setup_pSeriesLP NULL
#define dma_set_mask_pSeriesLP NULL
+#define dma_get_required_mask_pSeriesLP NULL
#endif /* !CONFIG_PCI */

static int iommu_mem_notifier(struct notifier_block *nb, unsigned long action,
@@ -1186,6 +1212,7 @@ void iommu_init_early_pSeries(void)
ppc_md.pci_dma_bus_setup = pci_dma_bus_setup_pSeriesLP;
ppc_md.pci_dma_dev_setup = pci_dma_dev_setup_pSeriesLP;
ppc_md.dma_set_mask = dma_set_mask_pSeriesLP;
+ ppc_md.dma_get_required_mask = dma_get_required_mask_pSeriesLP;
} else {
ppc_md.tce_build = tce_build_pSeries;
ppc_md.tce_free = tce_free_pSeries;
--
1.7.4.1

2011-05-11 22:27:07

by Nishanth Aravamudan

[permalink] [raw]
Subject: [PATCH 6/8] dma-mapping: add get_required_mask if arch overrides default

From: Milton Miller <[email protected]>

If an architecture sets ARCH_HAS_DMA_GET_REQUIRED_MASK and has settable
dma_map_ops, the required mask may change by the ops implementation.
For example, a system that always has an mmu inline may only require 32
bits while a swiotlb would desire bits to cover all of memory.

Therefore add the field if the architecture does not use the generic
definition of dma_get_required_mask. The first use will by by powerpc.
Note that this does add some dependency on the order in which files are
visible here.

Signed-off-by: Milton Miller <[email protected]>
Signed-off-by: Nishanth Aravamudan <[email protected]>
---
include/linux/dma-mapping.h | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index ba8319a..d0e023b 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -49,6 +49,9 @@ struct dma_map_ops {
int (*mapping_error)(struct device *dev, dma_addr_t dma_addr);
int (*dma_supported)(struct device *dev, u64 mask);
int (*set_dma_mask)(struct device *dev, u64 mask);
+#ifdef ARCH_HAS_DMA_GET_REQUIRED_MASK
+ u64 (*get_required_mask)(struct device *dev);
+#endif
int is_phys;
};

--
1.7.4.1

2011-05-11 22:25:49

by Nishanth Aravamudan

[permalink] [raw]
Subject: [PATCH 7/8] powerpc: use the newly added get_required_mask dma_map_ops hook

From: Milton Miller <[email protected]>

Now that the generic code has dma_map_ops set, instead of having a
messy ifdef & if block in the base dma_get_required_mask hook push
the computation into the dma ops.

If the ops fails to set the get_required_mask hook default to the
width of dma_addr_t.

This also corrects ibmbus ibmebus_dma_supported to require a 64
bit mask. I doubt anything is checking or setting the dma mask on
that bus.

Signed-off-by: Milton Miller <[email protected]>
Signed-off-by: Nishanth Aravamudan <[email protected]>
---
arch/powerpc/include/asm/device.h | 2 +
arch/powerpc/include/asm/dma-mapping.h | 3 --
arch/powerpc/kernel/dma-iommu.c | 3 +-
arch/powerpc/kernel/dma-swiotlb.c | 16 ++++++++++++
arch/powerpc/kernel/dma.c | 41 ++++++++++++-------------------
arch/powerpc/kernel/ibmebus.c | 8 +++++-
arch/powerpc/kernel/vio.c | 7 ++++-
arch/powerpc/platforms/cell/iommu.c | 13 ++++++++-
arch/powerpc/platforms/ps3/system-bus.c | 7 +++++
arch/powerpc/platforms/pseries/iommu.c | 2 +-
10 files changed, 68 insertions(+), 34 deletions(-)

diff --git a/arch/powerpc/include/asm/device.h b/arch/powerpc/include/asm/device.h
index 16d25c0..d57c08a 100644
--- a/arch/powerpc/include/asm/device.h
+++ b/arch/powerpc/include/asm/device.h
@@ -37,4 +37,6 @@ struct pdev_archdata {
u64 dma_mask;
};

+#define ARCH_HAS_DMA_GET_REQUIRED_MASK
+
#endif /* _ASM_POWERPC_DEVICE_H */
diff --git a/arch/powerpc/include/asm/dma-mapping.h b/arch/powerpc/include/asm/dma-mapping.h
index 8135e66..dd70fac 100644
--- a/arch/powerpc/include/asm/dma-mapping.h
+++ b/arch/powerpc/include/asm/dma-mapping.h
@@ -20,8 +20,6 @@

#define DMA_ERROR_CODE (~(dma_addr_t)0x0)

-#define ARCH_HAS_DMA_GET_REQUIRED_MASK
-
/* Some dma direct funcs must be visible for use in other dma_ops */
extern void *dma_direct_alloc_coherent(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t flag);
@@ -71,7 +69,6 @@ static inline unsigned long device_to_mask(struct device *dev)
*/
#ifdef CONFIG_PPC64
extern struct dma_map_ops dma_iommu_ops;
-extern u64 dma_iommu_get_required_mask(struct device *dev);
#endif
extern struct dma_map_ops dma_direct_ops;

diff --git a/arch/powerpc/kernel/dma-iommu.c b/arch/powerpc/kernel/dma-iommu.c
index 1f2a711..c1ad9db 100644
--- a/arch/powerpc/kernel/dma-iommu.c
+++ b/arch/powerpc/kernel/dma-iommu.c
@@ -90,7 +90,7 @@ static int dma_iommu_dma_supported(struct device *dev, u64 mask)
return 1;
}

-u64 dma_iommu_get_required_mask(struct device *dev)
+static u64 dma_iommu_get_required_mask(struct device *dev)
{
struct iommu_table *tbl = get_iommu_table_base(dev);
u64 mask;
@@ -111,5 +111,6 @@ struct dma_map_ops dma_iommu_ops = {
.dma_supported = dma_iommu_dma_supported,
.map_page = dma_iommu_map_page,
.unmap_page = dma_iommu_unmap_page,
+ .get_required_mask = dma_iommu_get_required_mask,
};
EXPORT_SYMBOL(dma_iommu_ops);
diff --git a/arch/powerpc/kernel/dma-swiotlb.c b/arch/powerpc/kernel/dma-swiotlb.c
index 4295e0b..1ebc918 100644
--- a/arch/powerpc/kernel/dma-swiotlb.c
+++ b/arch/powerpc/kernel/dma-swiotlb.c
@@ -24,6 +24,21 @@

unsigned int ppc_swiotlb_enable;

+static u64 swiotlb_powerpc_get_required(struct device *dev)
+{
+ u64 end, mask, max_direct_dma_addr = dev->archdata.max_direct_dma_addr;
+
+ end = memblock_end_of_DRAM();
+ if (max_direct_dma_addr && end > max_direct_dma_addr)
+ end = max_direct_dma_addr;
+ end += get_dma_offset(dev);
+
+ mask = 1ULL << (fls64(end) - 1);
+ mask += mask - 1;
+
+ return mask;
+}
+
/*
* At the moment, all platforms that use this code only require
* swiotlb to be used if we're operating on HIGHMEM. Since
@@ -44,6 +59,7 @@ struct dma_map_ops swiotlb_dma_ops = {
.sync_sg_for_cpu = swiotlb_sync_sg_for_cpu,
.sync_sg_for_device = swiotlb_sync_sg_for_device,
.mapping_error = swiotlb_dma_mapping_error,
+ .get_required_mask = swiotlb_powerpc_get_required,
};

void pci_dma_dev_setup_swiotlb(struct pci_dev *pdev)
diff --git a/arch/powerpc/kernel/dma.c b/arch/powerpc/kernel/dma.c
index 97fe867..df142d1 100644
--- a/arch/powerpc/kernel/dma.c
+++ b/arch/powerpc/kernel/dma.c
@@ -96,6 +96,18 @@ static int dma_direct_dma_supported(struct device *dev, u64 mask)
#endif
}

+static u64 dma_direct_get_required_mask(struct device *dev)
+{
+ u64 end, mask;
+
+ end = memblock_end_of_DRAM() + get_dma_offset(dev);
+
+ mask = 1ULL << (fls64(end) - 1);
+ mask += mask - 1;
+
+ return mask;
+}
+
static inline dma_addr_t dma_direct_map_page(struct device *dev,
struct page *page,
unsigned long offset,
@@ -144,6 +156,7 @@ struct dma_map_ops dma_direct_ops = {
.dma_supported = dma_direct_dma_supported,
.map_page = dma_direct_map_page,
.unmap_page = dma_direct_unmap_page,
+ .get_required_mask = dma_direct_get_required_mask,
#ifdef CONFIG_NOT_COHERENT_CACHE
.sync_single_for_cpu = dma_direct_sync_single,
.sync_single_for_device = dma_direct_sync_single,
@@ -175,7 +188,6 @@ EXPORT_SYMBOL(dma_set_mask);
u64 dma_get_required_mask(struct device *dev)
{
struct dma_map_ops *dma_ops = get_dma_ops(dev);
- u64 mask, end = 0;

if (ppc_md.dma_get_required_mask)
return ppc_md.dma_get_required_mask(dev);
@@ -183,31 +195,10 @@ u64 dma_get_required_mask(struct device *dev)
if (unlikely(dma_ops == NULL))
return 0;

-#ifdef CONFIG_PPC64
- else if (dma_ops == &dma_iommu_ops)
- return dma_iommu_get_required_mask(dev);
-#endif
-#ifdef CONFIG_SWIOTLB
- else if (dma_ops == &swiotlb_dma_ops) {
- u64 max_direct_dma_addr = dev->archdata.max_direct_dma_addr;
-
- end = memblock_end_of_DRAM();
- if (max_direct_dma_addr && end > max_direct_dma_addr)
- end = max_direct_dma_addr;
- end += get_dma_offset(dev);
- }
-#endif
- else if (dma_ops == &dma_direct_ops)
- end = memblock_end_of_DRAM() + get_dma_offset(dev);
- else {
- WARN_ONCE(1, "%s: unknown ops %p\n", __func__, dma_ops);
- end = memblock_end_of_DRAM();
- }
+ if (dma_ops->get_required_mask)
+ return dma_ops->get_required_mask(dev);

- mask = 1ULL << (fls64(end) - 1);
- mask += mask - 1;
-
- return mask;
+ return DMA_BIT_MASK(8 * sizeof(dma_addr_t));
}
EXPORT_SYMBOL_GPL(dma_get_required_mask);

diff --git a/arch/powerpc/kernel/ibmebus.c b/arch/powerpc/kernel/ibmebus.c
index 28581f1..90ef2a4 100644
--- a/arch/powerpc/kernel/ibmebus.c
+++ b/arch/powerpc/kernel/ibmebus.c
@@ -125,7 +125,12 @@ static void ibmebus_unmap_sg(struct device *dev,

static int ibmebus_dma_supported(struct device *dev, u64 mask)
{
- return 1;
+ return mask == DMA_BIT_MASK(64);
+}
+
+static u64 ibmebus_dma_get_required_mask(struct device *dev)
+{
+ return DMA_BIT_MASK(64);
}

static struct dma_map_ops ibmebus_dma_ops = {
@@ -134,6 +139,7 @@ static struct dma_map_ops ibmebus_dma_ops = {
.map_sg = ibmebus_map_sg,
.unmap_sg = ibmebus_unmap_sg,
.dma_supported = ibmebus_dma_supported,
+ .get_required_mask = ibmebus_dma_get_required_mask,
.map_page = ibmebus_map_page,
.unmap_page = ibmebus_unmap_page,
};
diff --git a/arch/powerpc/kernel/vio.c b/arch/powerpc/kernel/vio.c
index 1b695fd..c049325 100644
--- a/arch/powerpc/kernel/vio.c
+++ b/arch/powerpc/kernel/vio.c
@@ -605,6 +605,11 @@ static int vio_dma_iommu_dma_supported(struct device *dev, u64 mask)
return dma_iommu_ops.dma_supported(dev, mask);
}

+static u64 vio_dma_get_required_mask(struct device *dev)
+{
+ return dma_iommu_ops.get_required_mask(dev);
+}
+
struct dma_map_ops vio_dma_mapping_ops = {
.alloc_coherent = vio_dma_iommu_alloc_coherent,
.free_coherent = vio_dma_iommu_free_coherent,
@@ -613,7 +618,7 @@ struct dma_map_ops vio_dma_mapping_ops = {
.map_page = vio_dma_iommu_map_page,
.unmap_page = vio_dma_iommu_unmap_page,
.dma_supported = vio_dma_iommu_dma_supported,
-
+ .get_required_mask = vio_dma_get_required_mask,
};

/**
diff --git a/arch/powerpc/platforms/cell/iommu.c b/arch/powerpc/platforms/cell/iommu.c
index 5ef55f3..fc46fca 100644
--- a/arch/powerpc/platforms/cell/iommu.c
+++ b/arch/powerpc/platforms/cell/iommu.c
@@ -1161,11 +1161,20 @@ __setup("iommu_fixed=", setup_iommu_fixed);

static u64 cell_dma_get_required_mask(struct device *dev)
{
+ struct dma_map_ops *dma_ops;
+
if (!dev->dma_mask)
return 0;

- if (iommu_fixed_disabled && get_dma_ops(dev) == &dma_iommu_ops)
- return dma_iommu_get_required_mask(dev);
+ if (!iommu_fixed_disabled &&
+ cell_iommu_get_fixed_address(dev) != OF_BAD_ADDR)
+ return DMA_BIT_MASK(64);
+
+ dma_ops = get_dma_ops(dev);
+ if (dma_ops->get_required_mask)
+ return dma_ops->get_required_mask(dev);
+
+ WARN_ONCE(1, "no get_required_mask in %p ops", dma_ops);

return DMA_BIT_MASK(64);
}
diff --git a/arch/powerpc/platforms/ps3/system-bus.c b/arch/powerpc/platforms/ps3/system-bus.c
index 23083c3..688141c 100644
--- a/arch/powerpc/platforms/ps3/system-bus.c
+++ b/arch/powerpc/platforms/ps3/system-bus.c
@@ -695,12 +695,18 @@ static int ps3_dma_supported(struct device *_dev, u64 mask)
return mask >= DMA_BIT_MASK(32);
}

+static u64 ps3_dma_get_required_mask(struct device *_dev)
+{
+ return DMA_BIT_MASK(32);
+}
+
static struct dma_map_ops ps3_sb_dma_ops = {
.alloc_coherent = ps3_alloc_coherent,
.free_coherent = ps3_free_coherent,
.map_sg = ps3_sb_map_sg,
.unmap_sg = ps3_sb_unmap_sg,
.dma_supported = ps3_dma_supported,
+ .get_required_mask = ps3_dma_get_required_mask,
.map_page = ps3_sb_map_page,
.unmap_page = ps3_unmap_page,
};
@@ -711,6 +717,7 @@ static struct dma_map_ops ps3_ioc0_dma_ops = {
.map_sg = ps3_ioc0_map_sg,
.unmap_sg = ps3_ioc0_unmap_sg,
.dma_supported = ps3_dma_supported,
+ .get_required_mask = ps3_dma_get_required_mask,
.map_page = ps3_ioc0_map_page,
.unmap_page = ps3_unmap_page,
};
diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index fe5eded..9f121a3 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -1099,7 +1099,7 @@ static u64 dma_get_required_mask_pSeriesLP(struct device *dev)
return DMA_BIT_MASK(64);
}

- return dma_iommu_get_required_mask(dev);
+ return dma_iommu_ops.get_required_mask(dev);
}

#else /* CONFIG_PCI */
--
1.7.4.1

2011-05-11 22:26:01

by Nishanth Aravamudan

[permalink] [raw]
Subject: [PATCH 8/8] powerpc: tidy up dma_map_ops after adding new hook

From: Milton Miller <[email protected]>

The new get_required_mask hook name is longer than many of but not all
of the prior ops. Tidy the struct initializers to align the equal signs
using the local whitespace.

Signed-off-by: Milton Miller <[email protected]>
Signed-off-by: Nishanth Aravamudan <[email protected]>
---
arch/powerpc/kernel/dma-iommu.c | 14 +++++++-------
arch/powerpc/kernel/dma.c | 16 ++++++++--------
arch/powerpc/kernel/ibmebus.c | 14 +++++++-------
arch/powerpc/kernel/vio.c | 14 +++++++-------
4 files changed, 29 insertions(+), 29 deletions(-)

diff --git a/arch/powerpc/kernel/dma-iommu.c b/arch/powerpc/kernel/dma-iommu.c
index c1ad9db..6f04b9c 100644
--- a/arch/powerpc/kernel/dma-iommu.c
+++ b/arch/powerpc/kernel/dma-iommu.c
@@ -104,13 +104,13 @@ static u64 dma_iommu_get_required_mask(struct device *dev)
}

struct dma_map_ops dma_iommu_ops = {
- .alloc_coherent = dma_iommu_alloc_coherent,
- .free_coherent = dma_iommu_free_coherent,
- .map_sg = dma_iommu_map_sg,
- .unmap_sg = dma_iommu_unmap_sg,
- .dma_supported = dma_iommu_dma_supported,
- .map_page = dma_iommu_map_page,
- .unmap_page = dma_iommu_unmap_page,
+ .alloc_coherent = dma_iommu_alloc_coherent,
+ .free_coherent = dma_iommu_free_coherent,
+ .map_sg = dma_iommu_map_sg,
+ .unmap_sg = dma_iommu_unmap_sg,
+ .dma_supported = dma_iommu_dma_supported,
+ .map_page = dma_iommu_map_page,
+ .unmap_page = dma_iommu_unmap_page,
.get_required_mask = dma_iommu_get_required_mask,
};
EXPORT_SYMBOL(dma_iommu_ops);
diff --git a/arch/powerpc/kernel/dma.c b/arch/powerpc/kernel/dma.c
index df142d1..f94df52 100644
--- a/arch/powerpc/kernel/dma.c
+++ b/arch/powerpc/kernel/dma.c
@@ -149,14 +149,14 @@ static inline void dma_direct_sync_single(struct device *dev,
#endif

struct dma_map_ops dma_direct_ops = {
- .alloc_coherent = dma_direct_alloc_coherent,
- .free_coherent = dma_direct_free_coherent,
- .map_sg = dma_direct_map_sg,
- .unmap_sg = dma_direct_unmap_sg,
- .dma_supported = dma_direct_dma_supported,
- .map_page = dma_direct_map_page,
- .unmap_page = dma_direct_unmap_page,
- .get_required_mask = dma_direct_get_required_mask,
+ .alloc_coherent = dma_direct_alloc_coherent,
+ .free_coherent = dma_direct_free_coherent,
+ .map_sg = dma_direct_map_sg,
+ .unmap_sg = dma_direct_unmap_sg,
+ .dma_supported = dma_direct_dma_supported,
+ .map_page = dma_direct_map_page,
+ .unmap_page = dma_direct_unmap_page,
+ .get_required_mask = dma_direct_get_required_mask,
#ifdef CONFIG_NOT_COHERENT_CACHE
.sync_single_for_cpu = dma_direct_sync_single,
.sync_single_for_device = dma_direct_sync_single,
diff --git a/arch/powerpc/kernel/ibmebus.c b/arch/powerpc/kernel/ibmebus.c
index 90ef2a4..73110fb 100644
--- a/arch/powerpc/kernel/ibmebus.c
+++ b/arch/powerpc/kernel/ibmebus.c
@@ -134,14 +134,14 @@ static u64 ibmebus_dma_get_required_mask(struct device *dev)
}

static struct dma_map_ops ibmebus_dma_ops = {
- .alloc_coherent = ibmebus_alloc_coherent,
- .free_coherent = ibmebus_free_coherent,
- .map_sg = ibmebus_map_sg,
- .unmap_sg = ibmebus_unmap_sg,
- .dma_supported = ibmebus_dma_supported,
+ .alloc_coherent = ibmebus_alloc_coherent,
+ .free_coherent = ibmebus_free_coherent,
+ .map_sg = ibmebus_map_sg,
+ .unmap_sg = ibmebus_unmap_sg,
+ .dma_supported = ibmebus_dma_supported,
.get_required_mask = ibmebus_dma_get_required_mask,
- .map_page = ibmebus_map_page,
- .unmap_page = ibmebus_unmap_page,
+ .map_page = ibmebus_map_page,
+ .unmap_page = ibmebus_unmap_page,
};

static int ibmebus_match_path(struct device *dev, void *data)
diff --git a/arch/powerpc/kernel/vio.c b/arch/powerpc/kernel/vio.c
index c049325..34d291d 100644
--- a/arch/powerpc/kernel/vio.c
+++ b/arch/powerpc/kernel/vio.c
@@ -611,13 +611,13 @@ static u64 vio_dma_get_required_mask(struct device *dev)
}

struct dma_map_ops vio_dma_mapping_ops = {
- .alloc_coherent = vio_dma_iommu_alloc_coherent,
- .free_coherent = vio_dma_iommu_free_coherent,
- .map_sg = vio_dma_iommu_map_sg,
- .unmap_sg = vio_dma_iommu_unmap_sg,
- .map_page = vio_dma_iommu_map_page,
- .unmap_page = vio_dma_iommu_unmap_page,
- .dma_supported = vio_dma_iommu_dma_supported,
+ .alloc_coherent = vio_dma_iommu_alloc_coherent,
+ .free_coherent = vio_dma_iommu_free_coherent,
+ .map_sg = vio_dma_iommu_map_sg,
+ .unmap_sg = vio_dma_iommu_unmap_sg,
+ .map_page = vio_dma_iommu_map_page,
+ .unmap_page = vio_dma_iommu_unmap_page,
+ .dma_supported = vio_dma_iommu_dma_supported,
.get_required_mask = vio_dma_get_required_mask,
};

--
1.7.4.1

2011-05-12 05:51:35

by Geert Uytterhoeven

[permalink] [raw]
Subject: Re: [PATCH 7/8] powerpc: use the newly added get_required_mask dma_map_ops hook

On Thu, May 12, 2011 at 00:25, Nishanth Aravamudan <[email protected]> wrote:
> diff --git a/arch/powerpc/platforms/ps3/system-bus.c b/arch/powerpc/platforms/ps3/system-bus.c
> index 23083c3..688141c 100644
> --- a/arch/powerpc/platforms/ps3/system-bus.c
> +++ b/arch/powerpc/platforms/ps3/system-bus.c
> @@ -695,12 +695,18 @@ static int ps3_dma_supported(struct device *_dev, u64 mask)
>        return mask >= DMA_BIT_MASK(32);
>  }
>
> +static u64 ps3_dma_get_required_mask(struct device *_dev)
> +{
> +       return DMA_BIT_MASK(32);

Why 32 and not 64?

> +}
> +
>  static struct dma_map_ops ps3_sb_dma_ops = {
>        .alloc_coherent = ps3_alloc_coherent,
>        .free_coherent = ps3_free_coherent,
>        .map_sg = ps3_sb_map_sg,
>        .unmap_sg = ps3_sb_unmap_sg,
>        .dma_supported = ps3_dma_supported,
> +       .get_required_mask = ps3_dma_get_required_mask,
>        .map_page = ps3_sb_map_page,
>        .unmap_page = ps3_unmap_page,
>  };
> @@ -711,6 +717,7 @@ static struct dma_map_ops ps3_ioc0_dma_ops = {
>        .map_sg = ps3_ioc0_map_sg,
>        .unmap_sg = ps3_ioc0_unmap_sg,
>        .dma_supported = ps3_dma_supported,
> +       .get_required_mask = ps3_dma_get_required_mask,
>        .map_page = ps3_ioc0_map_page,
>        .unmap_page = ps3_unmap_page,
>  };

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

2011-05-12 07:32:44

by Milton Miller

[permalink] [raw]
Subject: Re: [PATCH 7/8] powerpc: use the newly added get_required_mask dma_map_ops hook

> On Thu, May 12, 2011 at 00:25, Nishanth Aravamudan <[email protected]> wrote:
> > diff --git a/arch/powerpc/platforms/ps3/system-bus.c b/arch/powerpc/platforms/ps3/system-bus.c
> > index 23083c3..688141c 100644
> > --- a/arch/powerpc/platforms/ps3/system-bus.c
> > +++ b/arch/powerpc/platforms/ps3/system-bus.c
> > @@ -695,12 +695,18 @@ static int ps3_dma_supported(struct device *_dev, u64 mask)
> > return mask >= DMA_BIT_MASK(32);
> > }
> >
> > +static u64 ps3_dma_get_required_mask(struct device *_dev)
> > +{
> > + return DMA_BIT_MASK(32);
>
> Why 32 and not 64?

I based it on the return of ps3_dma_supported, which you can see just
above says anything at or above a 32 bit mask is ok.

I don't really know the platform, but digging a bit deeper, it looks
like this goes to ps3_map_dma in ps3/mm.c. It looks like that translates
the virt to phys to lpar (similar to absolute in iseries), and the
maps it to a bus address by a linear mapping. But no where do I see
mention of a device dma mask (neither in mm.c or system-dev.c (except
for the ps3_dma_supported local), so I assume that 32 bits is sufficient
for any device. It appears to me the code establishs a 1:1 mapping
of all possible memory with no provision for allocating blocks or
checking that a bus address belongs to another memory segment.

Feel free to point out any errors in the above analysis, otherwise
I assume the required mask matches the dma_supported op.

Does the lv1 hypervisor offer more than 4G of memory to the lpar?

milton

2011-05-19 07:44:46

by Benjamin Herrenschmidt

[permalink] [raw]
Subject: Re: [PATCH 5/8] powerpc: override dma_get_required_mask by platform hook and ops

On Wed, 2011-05-11 at 15:25 -0700, Nishanth Aravamudan wrote:
> From: Milton Miller <[email protected]>
>
> The hook dma_get_required_mask is supposed to return the mask required
> by the platform to operate efficently. The generic version of
> dma_get_required_mask in driver/base/platform.c returns a mask based
> only on max_pfn. However, this is likely too big for iommu systems
> and could be too small for platforms that require a dma offset or have
> a secondary window at a high offset.

The result of those 3 patches doesn't build on top of my current tree,
the generic dma_ops lacks the dma_get_required_mask hook. I'll have a
look again after the merge window.

Cheers,
Ben.

2011-05-19 17:46:44

by Nishanth Aravamudan

[permalink] [raw]
Subject: Re: [PATCH 5/8] powerpc: override dma_get_required_mask by platform hook and ops

On 19.05.2011 [17:43:56 +1000], Benjamin Herrenschmidt wrote:
> On Wed, 2011-05-11 at 15:25 -0700, Nishanth Aravamudan wrote:
> > From: Milton Miller <[email protected]>
> >
> > The hook dma_get_required_mask is supposed to return the mask required
> > by the platform to operate efficently. The generic version of
> > dma_get_required_mask in driver/base/platform.c returns a mask based
> > only on max_pfn. However, this is likely too big for iommu systems
> > and could be too small for platforms that require a dma offset or have
> > a secondary window at a high offset.
>
> The result of those 3 patches doesn't build on top of my current tree,
> the generic dma_ops lacks the dma_get_required_mask hook. I'll have a
> look again after the merge window.

Hrm, I think it's because for whatever reason [1] I forgot to cc you on 6/8?

https://lkml.org/lkml/2011/5/11/473

-Nish

[1] AKA over-reliance on get_maintainer.pl and lack of mental oversight!

--
Nishanth Aravamudan <[email protected]>
IBM Linux Technology Center

2011-05-25 18:48:51

by Nishanth Aravamudan

[permalink] [raw]
Subject: Re: [PATCH 5/8] powerpc: override dma_get_required_mask by platform hook and ops

Ben,

On 19.05.2011 [10:46:25 -0700], Nishanth Aravamudan wrote:
> On 19.05.2011 [17:43:56 +1000], Benjamin Herrenschmidt wrote:
> > On Wed, 2011-05-11 at 15:25 -0700, Nishanth Aravamudan wrote:
> > > From: Milton Miller <[email protected]>
> > >
> > > The hook dma_get_required_mask is supposed to return the mask required
> > > by the platform to operate efficently. The generic version of
> > > dma_get_required_mask in driver/base/platform.c returns a mask based
> > > only on max_pfn. However, this is likely too big for iommu systems
> > > and could be too small for platforms that require a dma offset or have
> > > a secondary window at a high offset.
> >
> > The result of those 3 patches doesn't build on top of my current tree,
> > the generic dma_ops lacks the dma_get_required_mask hook. I'll have a
> > look again after the merge window.
>
> Hrm, I think it's because for whatever reason [1] I forgot to cc you on 6/8?
>
> https://lkml.org/lkml/2011/5/11/473

Were you able to find the patch I refer to above?

Thanks,
Nish

--
Nishanth Aravamudan <[email protected]>
IBM Linux Technology Center