This series:
* removes the swiotlb_(arch_)_phys_to_bus and bus_to_phys __weak
hooks, replacing them with an architecture-specific phys_to_dma and
dma_to_phys interface. These are used by both PowerPC and Xen to
provide the correct mapping from physical to DMA addresses.
* removes the swiotlb_address_needs_mapping and
swiotlb_range_needs_mapping __weak functions as well as
is_buffer_dma_capable (which should never have been a generic
function). All three are replaced by a single architecture-specific
interface which meets the needs of both PowerPC and Xen.
* removes the swiotlb_virt_to_bus __weak function and replaces it with
a CONFIG_HIGHMEM compatible version when high memory is in use. This
is needed for 32 bit PowerPC swiotlb support.
* removes the swiotlb_alloc* __weak functions and replaces them with
swiotlb_init_with_buffer which allows the use of a caller allocated
buffer (and emergency pool).
I think these new interfaces are cleaner than the existing __weak
functions and isolate the swiotlb code from architecture internals.
This series does not contain any Xen or PowerPC specific changes, those
will follow in separate postings. The complete patchset has been boot
tested under Xen and native-x86 and compiled for IA64 and PowerPC
Changes since v1:
- Fixed compile error in swiotlb_dma_to_virt highmem version. Moved
#ifdef into function to avoid prototype drift.
- checkpatch fixes.
- missed a swiotlb_arch_range_needs_mapping in swiotlb.h and x86
pci-swiotlb.c and swiotlb_bus_to_phys/phys_to_bus implementations in
x86.
- additionally replaced __weak swiotlb_alloc* with
swiotlb_init_with_buffer.
Signed-off-by: Ian Campbell <[email protected]>
Cc: Becky Bruce <[email protected]>
Cc: Benjamin Herrenschmidt <[email protected]>
Cc: FUJITA Tomonori <[email protected]>
Cc: Greg KH <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: Kumar Gala <[email protected]>
Cc: Olaf Kirch <[email protected]>
Cc: Tony Luck <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
dma_map_range is intended to replace usage of both
swiotlb_arch_range_needs_mapping and
swiotlb_arch_address_needs_mapping as __weak functions as well as
replacing is_buffer_dma_capable.
phys_to_dma and dma_to_phys are intended to replace
swiotlb_phys_to_bus and swiotlb_bus_to_phys. I choose to use dma
rather than bus since a) it matches the parameters and b) avoids
confusion on x86 with the existing (but deprecated) virt_to_bus
function which relates to ISA device DMA.
Signed-off-by: Ian Campbell <[email protected]>
Cc: FUJITA Tomonori <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: Tony Luck <[email protected]>
Cc: [email protected]
---
arch/ia64/include/asm/dma-mapping.h | 23 +++++++++++++++++++++++
1 files changed, 23 insertions(+), 0 deletions(-)
diff --git a/arch/ia64/include/asm/dma-mapping.h b/arch/ia64/include/asm/dma-mapping.h
index 36c0009..47d4107 100644
--- a/arch/ia64/include/asm/dma-mapping.h
+++ b/arch/ia64/include/asm/dma-mapping.h
@@ -174,4 +174,27 @@ dma_cache_sync (struct device *dev, void *vaddr, size_t size,
#define dma_is_consistent(d, h) (1) /* all we do is coherent memory... */
+static inline dma_addr_t phys_to_dma(struct device *hwdev, phys_addr_t paddr)
+{
+ return paddr;
+}
+
+static inline phys_addr_t dma_to_phys(struct device *hwdev, dma_addr_t daddr)
+{
+ return daddr;
+}
+
+static inline bool dma_map_range(struct device *dev, u64 mask,
+ phys_addr_t addr, size_t size,
+ dma_addr_t *dma_addr_p)
+{
+ dma_addr_t dma_addr = phys_to_dma(dev, addr);
+
+ if (dma_addr + size > mask)
+ return false;
+
+ *dma_addr_p = dma_addr;
+ return true;
+}
+
#endif /* _ASM_IA64_DMA_MAPPING_H */
--
1.5.6.5
dma_map_range is intended to replace usage of both
swiotlb_arch_range_needs_mapping and
swiotlb_arch_address_needs_mapping as __weak functions as well as
replacing is_buffer_dma_capable.
phys_to_dma and dma_to_phys are intended to replace
swiotlb_phys_to_bus and swiotlb_bus_to_phys. I choose to use dma
rather than bus since a) it matches the parameters and b) avoids
confusion on x86 with the existing (but deprecated) virt_to_bus
function which relates to ISA device DMA.
Signed-off-by: Ian Campbell <[email protected]>
Cc: FUJITA Tomonori <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: Olaf Kirch <[email protected]>
Cc: Greg KH <[email protected]>
Cc: [email protected]
---
arch/x86/include/asm/dma-mapping.h | 7 +++++++
arch/x86/kernel/pci-dma.c | 26 ++++++++++++++++++++++++++
2 files changed, 33 insertions(+), 0 deletions(-)
diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
index 916cbb6..be8cb22 100644
--- a/arch/x86/include/asm/dma-mapping.h
+++ b/arch/x86/include/asm/dma-mapping.h
@@ -47,6 +47,9 @@ static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
#define dma_free_noncoherent(d, s, v, h) dma_free_coherent(d, s, v, h)
#define dma_is_consistent(d, h) (1)
+extern dma_addr_t phys_to_dma(struct device *hwdev, phys_addr_t paddr);
+extern phys_addr_t dma_to_phys(struct device *hwdev, dma_addr_t daddr);
+
extern int dma_supported(struct device *hwdev, u64 mask);
extern int dma_set_mask(struct device *dev, u64 mask);
@@ -309,4 +312,8 @@ static inline void dma_free_coherent(struct device *dev, size_t size,
ops->free_coherent(dev, size, vaddr, bus);
}
+extern bool dma_map_range(struct device *dev, u64 mask,
+ phys_addr_t addr, size_t size,
+ dma_addr_t *dma_addr_p);
+
#endif
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 745579b..f4c1b03 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -59,6 +59,32 @@ int dma_set_mask(struct device *dev, u64 mask)
}
EXPORT_SYMBOL(dma_set_mask);
+dma_addr_t phys_to_dma(struct device *hwdev, phys_addr_t paddr)
+{
+ return paddr;
+}
+EXPORT_SYMBOL_GPL(phys_to_dma);
+
+phys_addr_t dma_to_phys(struct device *hwdev, dma_addr_t daddr)
+{
+ return daddr;
+}
+EXPORT_SYMBOL_GPL(dma_to_phys);
+
+bool dma_map_range(struct device *dev, u64 mask,
+ phys_addr_t addr, size_t size,
+ dma_addr_t *dma_addr_p)
+{
+ dma_addr_t dma_addr = phys_to_dma(dev, addr);
+
+ if (dma_addr + size > mask)
+ return false;
+
+ *dma_addr_p = dma_addr;
+ return true;
+}
+EXPORT_SYMBOL_GPL(dma_map_range);
+
#ifdef CONFIG_X86_64
static __initdata void *dma32_bootmem_ptr;
static unsigned long dma32_bootmem_size __initdata = (128ULL<<20);
--
1.5.6.5
This function is intended to replaces is_buffer_dma_capable with a
more generic interface.
Signed-off-by: Ian Campbell <[email protected]>
Cc: FUJITA Tomonori <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: [email protected]
---
arch/x86/kernel/pci-dma.c | 5 +----
1 files changed, 1 insertions(+), 4 deletions(-)
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index f4c1b03..458b7b5 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -160,7 +160,6 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t size,
{
unsigned long dma_mask;
struct page *page;
- dma_addr_t addr;
dma_mask = dma_alloc_coherent_mask(dev, flag);
@@ -170,8 +169,7 @@ again:
if (!page)
return NULL;
- addr = page_to_phys(page);
- if (!is_buffer_dma_capable(dma_mask, addr, size)) {
+ if (!dma_map_range(dev, dma_mask, page_to_phys(page), size, dma_addr)) {
__free_pages(page, get_order(size));
if (dma_mask < DMA_BIT_MASK(32) && !(flag & GFP_DMA)) {
@@ -182,7 +180,6 @@ again:
return NULL;
}
- *dma_addr = addr;
return page_address(page);
}
--
1.5.6.5
Replaces use of is_buffer_dma_capable which is going away.
Signed-off-by: Ian Campbell <[email protected]>
Cc: FUJITA Tomonori <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: [email protected]
---
arch/x86/kernel/pci-nommu.c | 17 ++++++++++-------
1 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/arch/x86/kernel/pci-nommu.c b/arch/x86/kernel/pci-nommu.c
index 71d412a..b6b75a0 100644
--- a/arch/x86/kernel/pci-nommu.c
+++ b/arch/x86/kernel/pci-nommu.c
@@ -12,13 +12,15 @@
#include <asm/dma.h>
static int
-check_addr(char *name, struct device *hwdev, dma_addr_t bus, size_t size)
+check_addr(char *name, struct device *hwdev, phys_addr_t phys,
+ size_t size, dma_addr_t *bus)
{
- if (hwdev && !is_buffer_dma_capable(*hwdev->dma_mask, bus, size)) {
+ if (hwdev &&
+ !dma_map_range(hwdev, *hwdev->dma_mask, phys, size, bus)) {
if (*hwdev->dma_mask >= DMA_BIT_MASK(32))
printk(KERN_ERR
"nommu_%s: overflow %Lx+%zu of device mask %Lx\n",
- name, (long long)bus, size,
+ name, (long long)phys, size,
(long long)*hwdev->dma_mask);
return 0;
}
@@ -30,9 +32,10 @@ static dma_addr_t nommu_map_page(struct device *dev, struct page *page,
enum dma_data_direction dir,
struct dma_attrs *attrs)
{
- dma_addr_t bus = page_to_phys(page) + offset;
+ phys_addr_t phys = page_to_phys(page) + offset;
+ dma_addr_t bus;
WARN_ON(size == 0);
- if (!check_addr("map_single", dev, bus, size))
+ if (!check_addr("map_single", dev, phys, size, &bus))
return bad_dma_address;
flush_write_buffers();
return bus;
@@ -64,8 +67,8 @@ static int nommu_map_sg(struct device *hwdev, struct scatterlist *sg,
for_each_sg(sg, s, nents, i) {
BUG_ON(!sg_page(s));
- s->dma_address = sg_phys(s);
- if (!check_addr("map_sg", hwdev, s->dma_address, s->length))
+ if (!check_addr("map_sg", hwdev, sg_phys(s), s->length,
+ &s->dma_address))
return 0;
s->dma_length = s->length;
}
--
1.5.6.5
There are no current users and the problem these were intended to
solve will be solved in a different way. This effectively reverts
commit 8c5df16b.
Signed-off-by: Ian Campbell <[email protected]>
Cc: FUJITA Tomonori <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: Olaf Kirch <[email protected]>
Cc: Greg KH <[email protected]>
Cc: Ingo Molnar <[email protected]>
---
arch/x86/kernel/pci-swiotlb.c | 10 ----------
include/linux/swiotlb.h | 3 ---
lib/swiotlb.c | 15 +++------------
3 files changed, 3 insertions(+), 25 deletions(-)
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index fdcc0e2..9640e17 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -13,16 +13,6 @@
int swiotlb __read_mostly;
-void * __init swiotlb_alloc_boot(size_t size, unsigned long nslabs)
-{
- return alloc_bootmem_low_pages(size);
-}
-
-void *swiotlb_alloc(unsigned order, unsigned long nslabs)
-{
- return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order);
-}
-
static void *x86_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_addr_t *dma_handle, gfp_t flags)
{
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 1b56dbf..b5b2245 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -24,9 +24,6 @@ struct scatterlist;
extern void
swiotlb_init(void);
-extern void *swiotlb_alloc_boot(size_t bytes, unsigned long nslabs);
-extern void *swiotlb_alloc(unsigned order, unsigned long nslabs);
-
extern void
*swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_addr_t *dma_handle, gfp_t flags);
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index ca8009a..6076bb7 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -114,16 +114,6 @@ setup_io_tlb_npages(char *str)
__setup("swiotlb=", setup_io_tlb_npages);
/* make io_tlb_overflow tunable too? */
-void * __weak __init swiotlb_alloc_boot(size_t size, unsigned long nslabs)
-{
- return alloc_bootmem_low_pages(size);
-}
-
-void * __weak swiotlb_alloc(unsigned order, unsigned long nslabs)
-{
- return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order);
-}
-
static dma_addr_t swiotlb_virt_to_dma(struct device *hwdev,
volatile void *address)
{
@@ -177,7 +167,7 @@ swiotlb_init_with_default_size(size_t default_size)
/*
* Get IO TLB memory from the low pages
*/
- io_tlb_start = swiotlb_alloc_boot(bytes, io_tlb_nslabs);
+ io_tlb_start = alloc_bootmem_low_pages(bytes);
if (!io_tlb_start)
panic("Cannot allocate SWIOTLB buffer");
io_tlb_end = io_tlb_start + bytes;
@@ -233,7 +223,8 @@ swiotlb_late_init_with_default_size(size_t default_size)
bytes = io_tlb_nslabs << IO_TLB_SHIFT;
while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) {
- io_tlb_start = swiotlb_alloc(order, io_tlb_nslabs);
+ io_tlb_start = (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN,
+ order);
if (io_tlb_start)
break;
order--;
--
1.5.6.5
These new architecture-specific interfaces subsume the existing __weak
function hooks.
Signed-off-by: Ian Campbell <[email protected]>
Cc: FUJITA Tomonori <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: Olaf Kirch <[email protected]>
Cc: Greg KH <[email protected]>
Cc: Tony Luck <[email protected]>
Cc: Becky Bruce <[email protected]>
Cc: Benjamin Herrenschmidt <[email protected]>
Cc: Kumar Gala <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
---
arch/x86/kernel/pci-swiotlb.c | 10 ----------
include/linux/swiotlb.h | 5 -----
lib/swiotlb.c | 18 ++++--------------
3 files changed, 4 insertions(+), 29 deletions(-)
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index a1712f2..e89cf99 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -23,16 +23,6 @@ void *swiotlb_alloc(unsigned order, unsigned long nslabs)
return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order);
}
-dma_addr_t swiotlb_phys_to_bus(struct device *hwdev, phys_addr_t paddr)
-{
- return paddr;
-}
-
-phys_addr_t swiotlb_bus_to_phys(struct device *hwdev, dma_addr_t baddr)
-{
- return baddr;
-}
-
int __weak swiotlb_arch_range_needs_mapping(phys_addr_t paddr, size_t size)
{
return 0;
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index cb1a663..954feec 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -27,11 +27,6 @@ swiotlb_init(void);
extern void *swiotlb_alloc_boot(size_t bytes, unsigned long nslabs);
extern void *swiotlb_alloc(unsigned order, unsigned long nslabs);
-extern dma_addr_t swiotlb_phys_to_bus(struct device *hwdev,
- phys_addr_t address);
-extern phys_addr_t swiotlb_bus_to_phys(struct device *hwdev,
- dma_addr_t address);
-
extern int swiotlb_arch_range_needs_mapping(phys_addr_t paddr, size_t size);
extern void
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index bffe6d7..baa1991 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -124,25 +124,15 @@ void * __weak swiotlb_alloc(unsigned order, unsigned long nslabs)
return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order);
}
-dma_addr_t __weak swiotlb_phys_to_bus(struct device *hwdev, phys_addr_t paddr)
-{
- return paddr;
-}
-
-phys_addr_t __weak swiotlb_bus_to_phys(struct device *hwdev, dma_addr_t baddr)
-{
- return baddr;
-}
-
static dma_addr_t swiotlb_virt_to_bus(struct device *hwdev,
volatile void *address)
{
- return swiotlb_phys_to_bus(hwdev, virt_to_phys(address));
+ return phys_to_dma(hwdev, virt_to_phys(address));
}
void * __weak swiotlb_bus_to_virt(struct device *hwdev, dma_addr_t address)
{
- return phys_to_virt(swiotlb_bus_to_phys(hwdev, address));
+ return phys_to_virt(dma_to_phys(hwdev, address));
}
int __weak swiotlb_arch_address_needs_mapping(struct device *hwdev,
@@ -646,7 +636,7 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
struct dma_attrs *attrs)
{
phys_addr_t phys = page_to_phys(page) + offset;
- dma_addr_t dev_addr = swiotlb_phys_to_bus(dev, phys);
+ dma_addr_t dev_addr = phys_to_dma(dev, phys);
void *map;
BUG_ON(dir == DMA_NONE);
@@ -817,7 +807,7 @@ swiotlb_map_sg_attrs(struct device *hwdev, struct scatterlist *sgl, int nelems,
for_each_sg(sgl, sg, nelems, i) {
phys_addr_t paddr = sg_phys(sg);
- dma_addr_t dev_addr = swiotlb_phys_to_bus(hwdev, paddr);
+ dma_addr_t dev_addr = phys_to_dma(hwdev, paddr);
if (range_needs_mapping(paddr, sg->length) ||
address_needs_mapping(hwdev, dev_addr, sg->length)) {
--
1.5.6.5
Rather than supplying a __weak hook which architectures which support
highmem can overide simply provide a version of swiotlb_bus_to_virt
which works with high memory. Make it conditional since it is a more
expensive variant than the non-highmem version.
Acutal function contents taken from the PowerPC swiotlb patchset by
Becky Bruce.
Signed-off-by: Ian Campbell <[email protected]>
Cc: Becky Bruce <[email protected]>
Cc: Benjamin Herrenschmidt <[email protected]>
Cc: Kumar Gala <[email protected]>
Cc: FUJITA Tomonori <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: [email protected]
---
lib/swiotlb.c | 11 ++++++++++-
1 files changed, 10 insertions(+), 1 deletions(-)
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index d37499b..d2b296a 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -130,9 +130,18 @@ static dma_addr_t swiotlb_virt_to_bus(struct device *hwdev,
return phys_to_dma(hwdev, virt_to_phys(address));
}
-void * __weak swiotlb_bus_to_virt(struct device *hwdev, dma_addr_t address)
+static void *swiotlb_bus_to_virt(struct device *hwdev, dma_addr_t address)
{
+#ifdef CONFIG_HIGHMEM
+ unsigned long pfn = PFN_DOWN(dma_to_phys(hwdev, address));
+ void *pageaddr = page_address(pfn_to_page(pfn));
+
+ if (pageaddr != NULL)
+ return pageaddr + (address % PAGE_SIZE);
+ return NULL;
+#else
return phys_to_virt(dma_to_phys(hwdev, address));
+#endif
}
static void swiotlb_print_info(unsigned long bytes)
--
1.5.6.5
This replaces usages of address_needs_mapping, range_needs_mapping and
is_buffer_dma_capable and the __weak architecture hooks to those
functions with a more flexible single function.
Signed-off-by: Ian Campbell <[email protected]>
Cc: FUJITA Tomonori <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: Olaf Kirch <[email protected]>
Cc: Greg KH <[email protected]>
Cc: Tony Luck <[email protected]>
Cc: Becky Bruce <[email protected]>
Cc: Benjamin Herrenschmidt <[email protected]>
Cc: Kumar Gala <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
---
arch/x86/kernel/pci-swiotlb.c | 5 ---
include/linux/dma-mapping.h | 5 ---
include/linux/swiotlb.h | 2 -
lib/swiotlb.c | 59 +++++++++++++---------------------------
4 files changed, 19 insertions(+), 52 deletions(-)
diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index e89cf99..fdcc0e2 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -23,11 +23,6 @@ void *swiotlb_alloc(unsigned order, unsigned long nslabs)
return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order);
}
-int __weak swiotlb_arch_range_needs_mapping(phys_addr_t paddr, size_t size)
-{
- return 0;
-}
-
static void *x86_swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_addr_t *dma_handle, gfp_t flags)
{
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 8083b6a..85dafa1 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -96,11 +96,6 @@ static inline int is_device_dma_capable(struct device *dev)
return dev->dma_mask != NULL && *dev->dma_mask != DMA_MASK_NONE;
}
-static inline int is_buffer_dma_capable(u64 mask, dma_addr_t addr, size_t size)
-{
- return addr + size <= mask;
-}
-
#ifdef CONFIG_HAS_DMA
#include <asm/dma-mapping.h>
#else
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 954feec..1b56dbf 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -27,8 +27,6 @@ swiotlb_init(void);
extern void *swiotlb_alloc_boot(size_t bytes, unsigned long nslabs);
extern void *swiotlb_alloc(unsigned order, unsigned long nslabs);
-extern int swiotlb_arch_range_needs_mapping(phys_addr_t paddr, size_t size);
-
extern void
*swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_addr_t *dma_handle, gfp_t flags);
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index baa1991..d37499b 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -135,17 +135,6 @@ void * __weak swiotlb_bus_to_virt(struct device *hwdev, dma_addr_t address)
return phys_to_virt(dma_to_phys(hwdev, address));
}
-int __weak swiotlb_arch_address_needs_mapping(struct device *hwdev,
- dma_addr_t addr, size_t size)
-{
- return !is_buffer_dma_capable(dma_get_mask(hwdev), addr, size);
-}
-
-int __weak swiotlb_arch_range_needs_mapping(phys_addr_t paddr, size_t size)
-{
- return 0;
-}
-
static void swiotlb_print_info(unsigned long bytes)
{
phys_addr_t pstart, pend;
@@ -305,17 +294,6 @@ cleanup1:
return -ENOMEM;
}
-static inline int
-address_needs_mapping(struct device *hwdev, dma_addr_t addr, size_t size)
-{
- return swiotlb_arch_address_needs_mapping(hwdev, addr, size);
-}
-
-static inline int range_needs_mapping(phys_addr_t paddr, size_t size)
-{
- return swiotlb_force || swiotlb_arch_range_needs_mapping(paddr, size);
-}
-
static int is_swiotlb_buffer(char *addr)
{
return addr >= io_tlb_start && addr < io_tlb_end;
@@ -542,7 +520,7 @@ void *
swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_addr_t *dma_handle, gfp_t flags)
{
- dma_addr_t dev_addr;
+ phys_addr_t phys;
void *ret;
int order = get_order(size);
u64 dma_mask = DMA_BIT_MASK(32);
@@ -551,9 +529,8 @@ swiotlb_alloc_coherent(struct device *hwdev, size_t size,
dma_mask = hwdev->coherent_dma_mask;
ret = (void *)__get_free_pages(flags, order);
- if (ret &&
- !is_buffer_dma_capable(dma_mask, swiotlb_virt_to_bus(hwdev, ret),
- size)) {
+ if (ret && !dma_map_range(hwdev, dma_mask, virt_to_phys(ret),
+ size, dma_handle)) {
/*
* The allocated memory isn't reachable by the device.
*/
@@ -572,19 +549,19 @@ swiotlb_alloc_coherent(struct device *hwdev, size_t size,
}
memset(ret, 0, size);
- dev_addr = swiotlb_virt_to_bus(hwdev, ret);
+ phys = virt_to_phys(ret);
/* Confirm address can be DMA'd by device */
- if (!is_buffer_dma_capable(dma_mask, dev_addr, size)) {
- printk("hwdev DMA mask = 0x%016Lx, dev_addr = 0x%016Lx\n",
+ if (!dma_map_range(hwdev, dma_mask, phys, size, dma_handle)) {
+ printk(KERN_WARNING "hwdev DMA mask = 0x%016Lx, "
+ "physical addr = 0x%016Lx\n",
(unsigned long long)dma_mask,
- (unsigned long long)dev_addr);
+ (unsigned long long)phys);
/* DMA_TO_DEVICE to avoid memcpy in unmap_single */
do_unmap_single(hwdev, ret, size, DMA_TO_DEVICE);
return NULL;
}
- *dma_handle = dev_addr;
return ret;
}
EXPORT_SYMBOL(swiotlb_alloc_coherent);
@@ -636,7 +613,7 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
struct dma_attrs *attrs)
{
phys_addr_t phys = page_to_phys(page) + offset;
- dma_addr_t dev_addr = phys_to_dma(dev, phys);
+ dma_addr_t dev_addr;
void *map;
BUG_ON(dir == DMA_NONE);
@@ -645,8 +622,8 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
* we can safely return the device addr and not worry about bounce
* buffering it.
*/
- if (!address_needs_mapping(dev, dev_addr, size) &&
- !range_needs_mapping(phys, size))
+ if (dma_map_range(dev, dma_get_mask(dev), phys, size, &dev_addr) &&
+ !swiotlb_force)
return dev_addr;
/*
@@ -658,12 +635,12 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
map = io_tlb_overflow_buffer;
}
- dev_addr = swiotlb_virt_to_bus(dev, map);
+ phys = virt_to_phys(map);
/*
* Ensure that the address returned is DMA'ble
*/
- if (address_needs_mapping(dev, dev_addr, size))
+ if (!dma_map_range(dev, dma_get_mask(dev), phys, size, &dev_addr))
panic("map_single: bounce buffer is not DMA'ble");
return dev_addr;
@@ -807,10 +784,11 @@ swiotlb_map_sg_attrs(struct device *hwdev, struct scatterlist *sgl, int nelems,
for_each_sg(sgl, sg, nelems, i) {
phys_addr_t paddr = sg_phys(sg);
- dma_addr_t dev_addr = phys_to_dma(hwdev, paddr);
+ dma_addr_t uninitialized_var(dev_addr);
- if (range_needs_mapping(paddr, sg->length) ||
- address_needs_mapping(hwdev, dev_addr, sg->length)) {
+ if (!dma_map_range(hwdev, dma_get_mask(hwdev), dev_addr,
+ sg->length, &dev_addr) ||
+ swiotlb_force) {
void *map = map_single(hwdev, sg_phys(sg),
sg->length, dir);
if (!map) {
@@ -822,7 +800,8 @@ swiotlb_map_sg_attrs(struct device *hwdev, struct scatterlist *sgl, int nelems,
sgl[0].dma_length = 0;
return 0;
}
- sg->dma_address = swiotlb_virt_to_bus(hwdev, map);
+ paddr = virt_to_phys(map);
+ sg->dma_address = phys_to_dma(hwdev, paddr);
} else
sg->dma_address = dev_addr;
sg->dma_length = sg->length;
--
1.5.6.5
This function is intended to replaces is_buffer_dma_capable with a
more generic interface.
Signed-off-by: Ian Campbell <[email protected]>
Cc: FUJITA Tomonori <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: [email protected]
---
arch/x86/kernel/pci-gart_64.c | 6 ++++--
1 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kernel/pci-gart_64.c b/arch/x86/kernel/pci-gart_64.c
index 1e8920d..eb043bd 100644
--- a/arch/x86/kernel/pci-gart_64.c
+++ b/arch/x86/kernel/pci-gart_64.c
@@ -190,14 +190,16 @@ static void iommu_full(struct device *dev, size_t size, int dir)
static inline int
need_iommu(struct device *dev, unsigned long addr, size_t size)
{
+ dma_addr_t dma_addr;
return force_iommu ||
- !is_buffer_dma_capable(*dev->dma_mask, addr, size);
+ !dma_map_range(dev, *dev->dma_mask, addr, size, &dma_addr);
}
static inline int
nonforced_iommu(struct device *dev, unsigned long addr, size_t size)
{
- return !is_buffer_dma_capable(*dev->dma_mask, addr, size);
+ dma_addr_t dma_addr;
+ return !dma_map_range(dev, *dev->dma_mask, addr, size, &dma_addr);
}
/* Map a single continuous physical area into the IOMMU.
--
1.5.6.5
Rename swiotlb_virt_to_bus as swiotlb_virt_to_dma for consistency with
phys_to_dma.
Signed-off-by: Ian Campbell <[email protected]>
Cc: FUJITA Tomonori <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: Olaf Kirch <[email protected]>
Cc: Greg KH <[email protected]>
Cc: Ingo Molnar <[email protected]>
---
lib/swiotlb.c | 14 +++++++-------
1 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index d2b296a..ca8009a 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -124,13 +124,13 @@ void * __weak swiotlb_alloc(unsigned order, unsigned long nslabs)
return (void *)__get_free_pages(GFP_DMA | __GFP_NOWARN, order);
}
-static dma_addr_t swiotlb_virt_to_bus(struct device *hwdev,
+static dma_addr_t swiotlb_virt_to_dma(struct device *hwdev,
volatile void *address)
{
return phys_to_dma(hwdev, virt_to_phys(address));
}
-static void *swiotlb_bus_to_virt(struct device *hwdev, dma_addr_t address)
+static void *swiotlb_dma_to_virt(struct device *hwdev, dma_addr_t address)
{
#ifdef CONFIG_HIGHMEM
unsigned long pfn = PFN_DOWN(dma_to_phys(hwdev, address));
@@ -365,7 +365,7 @@ map_single(struct device *hwdev, phys_addr_t phys, size_t size, int dir)
unsigned long max_slots;
mask = dma_get_seg_boundary(hwdev);
- start_dma_addr = swiotlb_virt_to_bus(hwdev, io_tlb_start) & mask;
+ start_dma_addr = swiotlb_virt_to_dma(hwdev, io_tlb_start) & mask;
offset_slots = ALIGN(start_dma_addr, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
@@ -667,7 +667,7 @@ EXPORT_SYMBOL_GPL(swiotlb_map_page);
static void unmap_single(struct device *hwdev, dma_addr_t dev_addr,
size_t size, int dir)
{
- char *dma_addr = swiotlb_bus_to_virt(hwdev, dev_addr);
+ char *dma_addr = swiotlb_dma_to_virt(hwdev, dev_addr);
BUG_ON(dir == DMA_NONE);
@@ -704,7 +704,7 @@ static void
swiotlb_sync_single(struct device *hwdev, dma_addr_t dev_addr,
size_t size, int dir, int target)
{
- char *dma_addr = swiotlb_bus_to_virt(hwdev, dev_addr);
+ char *dma_addr = swiotlb_dma_to_virt(hwdev, dev_addr);
BUG_ON(dir == DMA_NONE);
@@ -892,7 +892,7 @@ EXPORT_SYMBOL(swiotlb_sync_sg_for_device);
int
swiotlb_dma_mapping_error(struct device *hwdev, dma_addr_t dma_addr)
{
- return (dma_addr == swiotlb_virt_to_bus(hwdev, io_tlb_overflow_buffer));
+ return (dma_addr == swiotlb_virt_to_dma(hwdev, io_tlb_overflow_buffer));
}
EXPORT_SYMBOL(swiotlb_dma_mapping_error);
@@ -905,6 +905,6 @@ EXPORT_SYMBOL(swiotlb_dma_mapping_error);
int
swiotlb_dma_supported(struct device *hwdev, u64 mask)
{
- return swiotlb_virt_to_bus(hwdev, io_tlb_end - 1) <= mask;
+ return swiotlb_virt_to_dma(hwdev, io_tlb_end - 1) <= mask;
}
EXPORT_SYMBOL(swiotlb_dma_supported);
--
1.5.6.5
This allows the generic swiotlb code to be used by users with unusual
requirements for the buffer allocation by allowing them to provide
their own swiotlb buffer.
Also:
- swiotlb_init_with_default_size is unused so collapse into
swiotlb_init.
- move declarations of swiotlb_late_init_with_default_size out of *.c
into linux/swiotlb.h where they belong
- Make io_tlb_nslabs and io_tlb_overflow non-static since callers of the
new interface will want to access them.
Signed-off-by: Ian Campbell <[email protected]>
Cc: FUJITA Tomonori <[email protected]>
Cc: Jeremy Fitzhardinge <[email protected]>
Cc: Olaf Kirch <[email protected]>
Cc: Greg KH <[email protected]>
Cc: Ingo Molnar <[email protected]>
---
arch/ia64/hp/common/hwsw_iommu.c | 3 --
arch/ia64/hp/common/sba_iommu.c | 2 -
include/linux/swiotlb.h | 14 ++++++++
lib/swiotlb.c | 64 +++++++++++++++++++++----------------
4 files changed, 50 insertions(+), 33 deletions(-)
diff --git a/arch/ia64/hp/common/hwsw_iommu.c b/arch/ia64/hp/common/hwsw_iommu.c
index e4a80d8..d557779 100644
--- a/arch/ia64/hp/common/hwsw_iommu.c
+++ b/arch/ia64/hp/common/hwsw_iommu.c
@@ -19,9 +19,6 @@
extern struct dma_map_ops sba_dma_ops, swiotlb_dma_ops;
-/* swiotlb declarations & definitions: */
-extern int swiotlb_late_init_with_default_size (size_t size);
-
/*
* Note: we need to make the determination of whether or not to use
* the sw I/O TLB based purely on the device structure. Anything else
diff --git a/arch/ia64/hp/common/sba_iommu.c b/arch/ia64/hp/common/sba_iommu.c
index 56ceb68..eeb5b49 100644
--- a/arch/ia64/hp/common/sba_iommu.c
+++ b/arch/ia64/hp/common/sba_iommu.c
@@ -46,8 +46,6 @@
#include <asm/acpi-ext.h>
-extern int swiotlb_late_init_with_default_size (size_t size);
-
#define PFX "IOC: "
/*
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index b5b2245..7c6f4c1 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -21,8 +21,22 @@ struct scatterlist;
*/
#define IO_TLB_SHIFT 11
+/*
+ * number of IO TLB slabs.
+ */
+extern unsigned long io_tlb_nslabs;
+
+/*
+ * size of emergency overflow buffer.
+ */
+extern unsigned long io_tlb_overflow;
+
extern void
swiotlb_init(void);
+extern void
+swiotlb_init_with_buffer(unsigned long bytes, void *buffer, void *emergency);
+extern int
+swiotlb_late_init_with_default_size(size_t size);
extern void
*swiotlb_alloc_coherent(struct device *hwdev, size_t size,
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 6076bb7..2449669 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -70,14 +70,14 @@ static char *io_tlb_start, *io_tlb_end;
* The number of IO TLB blocks (in groups of 64) betweeen io_tlb_start and
* io_tlb_end. This is command line adjustable via setup_io_tlb_npages.
*/
-static unsigned long io_tlb_nslabs;
+unsigned long io_tlb_nslabs;
/*
* When the IOMMU overflows we return a fallback buffer. This sets the size.
*/
-static unsigned long io_tlb_overflow = 32*1024;
+unsigned long io_tlb_overflow = 32*1024;
-void *io_tlb_overflow_buffer;
+static void *io_tlb_overflow_buffer;
/*
* This is a free list describing the number of free entries available from
@@ -148,14 +148,40 @@ static void swiotlb_print_info(unsigned long bytes)
(unsigned long long)pend);
}
+void __init
+swiotlb_init_with_buffer(unsigned long bytes, void *buffer, void *overflow)
+{
+ unsigned long i;
+
+ io_tlb_start = buffer;
+ io_tlb_end = io_tlb_start + bytes;
+
+ io_tlb_overflow_buffer = overflow;
+
+ /*
+ * Allocate and initialize the free list array. This array is used
+ * to find contiguous free memory regions of size up to IO_TLB_SEGSIZE
+ * between io_tlb_start and io_tlb_end.
+ */
+ io_tlb_list = alloc_bootmem(io_tlb_nslabs * sizeof(int));
+ for (i = 0; i < io_tlb_nslabs; i++)
+ io_tlb_list[i] = IO_TLB_SEGSIZE - OFFSET(i, IO_TLB_SEGSIZE);
+ io_tlb_index = 0;
+ io_tlb_orig_addr = alloc_bootmem(io_tlb_nslabs * sizeof(phys_addr_t));
+
+ swiotlb_print_info(bytes);
+}
+
/*
* Statically reserve bounce buffer space and initialize bounce buffer data
* structures for the software IO TLB used to implement the DMA API.
*/
void __init
-swiotlb_init_with_default_size(size_t default_size)
+swiotlb_init(void)
{
- unsigned long i, bytes;
+ unsigned long bytes;
+ void *buffer, *overflow;
+ size_t default_size = 64 * (1<<20); /* default to 64MB */
if (!io_tlb_nslabs) {
io_tlb_nslabs = (default_size >> IO_TLB_SHIFT);
@@ -167,36 +193,18 @@ swiotlb_init_with_default_size(size_t default_size)
/*
* Get IO TLB memory from the low pages
*/
- io_tlb_start = alloc_bootmem_low_pages(bytes);
- if (!io_tlb_start)
+ buffer = alloc_bootmem_low_pages(bytes);
+ if (!buffer)
panic("Cannot allocate SWIOTLB buffer");
- io_tlb_end = io_tlb_start + bytes;
-
- /*
- * Allocate and initialize the free list array. This array is used
- * to find contiguous free memory regions of size up to IO_TLB_SEGSIZE
- * between io_tlb_start and io_tlb_end.
- */
- io_tlb_list = alloc_bootmem(io_tlb_nslabs * sizeof(int));
- for (i = 0; i < io_tlb_nslabs; i++)
- io_tlb_list[i] = IO_TLB_SEGSIZE - OFFSET(i, IO_TLB_SEGSIZE);
- io_tlb_index = 0;
- io_tlb_orig_addr = alloc_bootmem(io_tlb_nslabs * sizeof(phys_addr_t));
/*
* Get the overflow emergency buffer
*/
- io_tlb_overflow_buffer = alloc_bootmem_low(io_tlb_overflow);
- if (!io_tlb_overflow_buffer)
+ overflow = alloc_bootmem_low(io_tlb_overflow);
+ if (!overflow)
panic("Cannot allocate SWIOTLB overflow buffer!\n");
- swiotlb_print_info(bytes);
-}
-
-void __init
-swiotlb_init(void)
-{
- swiotlb_init_with_default_size(64 * (1<<20)); /* default to 64MB */
+ swiotlb_init_with_buffer(bytes, buffer, overflow);
}
/*
--
1.5.6.5
On Mon, 1 Jun 2009 16:32:53 +0100
Ian Campbell <[email protected]> wrote:
> dma_map_range is intended to replace usage of both
> swiotlb_arch_range_needs_mapping and
> swiotlb_arch_address_needs_mapping as __weak functions as well as
> replacing is_buffer_dma_capable.
>
> phys_to_dma and dma_to_phys are intended to replace
> swiotlb_phys_to_bus and swiotlb_bus_to_phys. I choose to use dma
> rather than bus since a) it matches the parameters and b) avoids
> confusion on x86 with the existing (but deprecated) virt_to_bus
> function which relates to ISA device DMA.
>
> Signed-off-by: Ian Campbell <[email protected]>
> Cc: FUJITA Tomonori <[email protected]>
> Cc: Ingo Molnar <[email protected]>
> Cc: Jeremy Fitzhardinge <[email protected]>
> Cc: Tony Luck <[email protected]>
> Cc: [email protected]
> ---
> arch/ia64/include/asm/dma-mapping.h | 23 +++++++++++++++++++++++
> 1 files changed, 23 insertions(+), 0 deletions(-)
>
> diff --git a/arch/ia64/include/asm/dma-mapping.h b/arch/ia64/include/asm/dma-mapping.h
> index 36c0009..47d4107 100644
> --- a/arch/ia64/include/asm/dma-mapping.h
> +++ b/arch/ia64/include/asm/dma-mapping.h
> @@ -174,4 +174,27 @@ dma_cache_sync (struct device *dev, void *vaddr, size_t size,
>
> #define dma_is_consistent(d, h) (1) /* all we do is coherent memory... */
>
> +static inline dma_addr_t phys_to_dma(struct device *hwdev, phys_addr_t paddr)
> +{
> + return paddr;
> +}
> +
> +static inline phys_addr_t dma_to_phys(struct device *hwdev, dma_addr_t daddr)
> +{
> + return daddr;
> +}
> +
> +static inline bool dma_map_range(struct device *dev, u64 mask,
> + phys_addr_t addr, size_t size,
> + dma_addr_t *dma_addr_p)
> +{
> + dma_addr_t dma_addr = phys_to_dma(dev, addr);
> +
> + if (dma_addr + size > mask)
> + return false;
> +
> + *dma_addr_p = dma_addr;
> + return true;
> +}
> +
dma_map_range is a really confusing name. We have dma_map_single and
dma_map_sg, they are the DMA mapping API.
dma_map_range sounds like the DMA mapping API but it isn't. As I said,
Xen dom0 needs to implement something like xen_map_sg, xen_map_single,
etc, which uses some of swiotlb functions internally. Then we don't
need functions like the above.
On Mon, 1 Jun 2009 16:33:03 +0100
Ian Campbell <[email protected]> wrote:
> This allows the generic swiotlb code to be used by users with unusual
> requirements for the buffer allocation by allowing them to provide
> their own swiotlb buffer.
>
> Also:
> - swiotlb_init_with_default_size is unused so collapse into
> swiotlb_init.
> - move declarations of swiotlb_late_init_with_default_size out of *.c
> into linux/swiotlb.h where they belong
> - Make io_tlb_nslabs and io_tlb_overflow non-static since callers of the
> new interface will want to access them.
>
> Signed-off-by: Ian Campbell <[email protected]>
> Cc: FUJITA Tomonori <[email protected]>
> Cc: Jeremy Fitzhardinge <[email protected]>
> Cc: Olaf Kirch <[email protected]>
> Cc: Greg KH <[email protected]>
> Cc: Ingo Molnar <[email protected]>
> ---
> arch/ia64/hp/common/hwsw_iommu.c | 3 --
> arch/ia64/hp/common/sba_iommu.c | 2 -
> include/linux/swiotlb.h | 14 ++++++++
> lib/swiotlb.c | 64 +++++++++++++++++++++----------------
> 4 files changed, 50 insertions(+), 33 deletions(-)
Please tell me a pointer to Xen code to use this patch. I think that
we could have more clean interfaces.
On Mon, 1 Jun 2009 16:32:54 +0100
Ian Campbell <[email protected]> wrote:
> dma_map_range is intended to replace usage of both
> swiotlb_arch_range_needs_mapping and
> swiotlb_arch_address_needs_mapping as __weak functions as well as
> replacing is_buffer_dma_capable.
>
> phys_to_dma and dma_to_phys are intended to replace
> swiotlb_phys_to_bus and swiotlb_bus_to_phys. I choose to use dma
> rather than bus since a) it matches the parameters and b) avoids
> confusion on x86 with the existing (but deprecated) virt_to_bus
> function which relates to ISA device DMA.
>
> Signed-off-by: Ian Campbell <[email protected]>
> Cc: FUJITA Tomonori <[email protected]>
> Cc: Jeremy Fitzhardinge <[email protected]>
> Cc: Olaf Kirch <[email protected]>
> Cc: Greg KH <[email protected]>
> Cc: [email protected]
> ---
> arch/x86/include/asm/dma-mapping.h | 7 +++++++
> arch/x86/kernel/pci-dma.c | 26 ++++++++++++++++++++++++++
> 2 files changed, 33 insertions(+), 0 deletions(-)
>
> diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h
> index 916cbb6..be8cb22 100644
> --- a/arch/x86/include/asm/dma-mapping.h
> +++ b/arch/x86/include/asm/dma-mapping.h
> @@ -47,6 +47,9 @@ static inline int dma_mapping_error(struct device *dev, dma_addr_t dma_addr)
> #define dma_free_noncoherent(d, s, v, h) dma_free_coherent(d, s, v, h)
> #define dma_is_consistent(d, h) (1)
>
> +extern dma_addr_t phys_to_dma(struct device *hwdev, phys_addr_t paddr);
> +extern phys_addr_t dma_to_phys(struct device *hwdev, dma_addr_t daddr);
> +
> extern int dma_supported(struct device *hwdev, u64 mask);
> extern int dma_set_mask(struct device *dev, u64 mask);
>
> @@ -309,4 +312,8 @@ static inline void dma_free_coherent(struct device *dev, size_t size,
> ops->free_coherent(dev, size, vaddr, bus);
> }
>
> +extern bool dma_map_range(struct device *dev, u64 mask,
> + phys_addr_t addr, size_t size,
> + dma_addr_t *dma_addr_p);
> +
> #endif
> diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
> index 745579b..f4c1b03 100644
> --- a/arch/x86/kernel/pci-dma.c
> +++ b/arch/x86/kernel/pci-dma.c
> @@ -59,6 +59,32 @@ int dma_set_mask(struct device *dev, u64 mask)
> }
> EXPORT_SYMBOL(dma_set_mask);
>
> +dma_addr_t phys_to_dma(struct device *hwdev, phys_addr_t paddr)
> +{
> + return paddr;
> +}
> +EXPORT_SYMBOL_GPL(phys_to_dma);
> +
> +phys_addr_t dma_to_phys(struct device *hwdev, dma_addr_t daddr)
> +{
> + return daddr;
> +}
> +EXPORT_SYMBOL_GPL(dma_to_phys);
> +
> +bool dma_map_range(struct device *dev, u64 mask,
> + phys_addr_t addr, size_t size,
> + dma_addr_t *dma_addr_p)
> +{
> + dma_addr_t dma_addr = phys_to_dma(dev, addr);
> +
> + if (dma_addr + size > mask)
> + return false;
> +
> + *dma_addr_p = dma_addr;
> + return true;
> +}
> +EXPORT_SYMBOL_GPL(dma_map_range);
> +
Why can't the above functions live in arch/include/asm/dma-mapping.h,
where they should live?
On Tue, 2009-06-02 at 00:08 -0400, FUJITA Tomonori wrote:
> dma_map_range is a really confusing name. We have dma_map_single and
> dma_map_sg, they are the DMA mapping API.
> dma_map_range sounds like the DMA mapping API but it isn't.
Yes, it's not such a good name. I wonder what would be better?
Perhaps dma_range_mapped? The return value indicates whether the range
is mapped or not so this makes some sense. It also makes it clearer that
this function is not intended to actually perform the mapping if it does
not exist.
> As I said,
> Xen dom0 needs to implement something like xen_map_sg, xen_map_single,
> etc, which uses some of swiotlb functions internally. Then we don't
> need functions like the above.
xen_map_sg would be literally identical to swiotlb_map_sg in every way
apart from the additional phys<->dma address translations. Similarly for
the other swiotlb interfaces. The phys<->dma address translation is also
required for the PowerPC architecture so duplicating all that code just
for Xen doesn't really solve the problem.
Ian.
On Tue, 2009-06-02 at 00:08 -0400, FUJITA Tomonori wrote:
> Please tell me a pointer to Xen code to use this patch. I think that
> we could have more clean interfaces.
Certainly. This is the relevant bit of my current WIP patch. I'm not
happy about the interaction with the global swiotlb/swiotlb_force flags
at the moment, I was planning to spend today figuring out something
nicer (hence its WIP status).
Ian.
diff --git a/drivers/pci/xen-iommu.c b/drivers/pci/xen-iommu.c
index 4918938..ca005d8 100644
--- a/drivers/pci/xen-iommu.c
+++ b/drivers/pci/xen-iommu.c
@@ -5,8 +5,11 @@
#include <linux/module.h>
#include <linux/version.h>
#include <linux/scatterlist.h>
+#include <linux/swiotlb.h>
+#include <linux/dma-mapping.h>
#include <linux/io.h>
#include <linux/bug.h>
+#include <linux/bootmem.h>
#include <xen/interface/xen.h>
#include <xen/grant_table.h>
@@ -44,3 +47,33 @@ static phys_addr_t xen_dma_to_phys(struct device *hwdev, dma_addr_t daddr)
return machine_to_phys(XMADDR(daddr)).paddr;
}
+
+static int max_dma_bits = 32;
+
+static void xen_swiotlb_fixup(void *buf, size_t size, unsigned long nslabs)
+{
+ int i, rc;
+ int dma_bits;
+
+ printk(KERN_DEBUG "xen_swiotlb_fixup: buf=%p size=%zu\n",
+ buf, size);
+
+ dma_bits = get_order(IO_TLB_SEGSIZE << IO_TLB_SHIFT) + PAGE_SHIFT;
+
+ i = 0;
+ do {
+ int slabs = min(nslabs - i, (unsigned long)IO_TLB_SEGSIZE);
+
+ do {
+ rc = xen_create_contiguous_region(
+ (unsigned long)buf + (i << IO_TLB_SHIFT),
+ get_order(slabs << IO_TLB_SHIFT),
+ dma_bits);
+ } while (rc && dma_bits++ < max_dma_bits);
+ if (rc)
+ panic(KERN_ERR "xen_create_contiguous_region failed\n");
+
+ i += slabs;
+ } while(i < nslabs);
+}
+
@@ -300,9 +340,42 @@ void __init xen_iommu_init(void)
dma_ops = &xen_dma_ops;
- if (swiotlb) {
+ if (xen_initial_domain() || swiotlb_force) {
+ unsigned long bytes;
+ void *buffer, *overflow;
+ size_t default_size = 64 * (1<<20); /* default to 64MB */
+
printk(KERN_INFO "Xen: Enabling DMA fallback to swiotlb\n");
+
+ swiotlb = 0; /* Avoid native swiotlb initialisation path. */
+ swiotlb_force = 0;
+
dma_ops = &xen_swiotlb_dma_ops;
+
+ if (!io_tlb_nslabs) {
+ io_tlb_nslabs = (default_size >> IO_TLB_SHIFT);
+ io_tlb_nslabs = ALIGN(io_tlb_nslabs, IO_TLB_SEGSIZE);
+ }
+
+ bytes = io_tlb_nslabs << IO_TLB_SHIFT;
+
+ /*
+ * Get IO TLB memory. No need for low low pages under Xen.
+ */
+ buffer = alloc_bootmem_pages(bytes);
+ if (!buffer)
+ panic("Cannot allocate SWIOTLB buffer");
+ xen_swiotlb_fixup(buffer, bytes, io_tlb_nslabs);
+
+ /*
+ * Get the overflow emergency buffer
+ */
+ overflow = alloc_bootmem(io_tlb_overflow);
+ if (!overflow)
+ panic("Cannot allocate SWIOTLB overflow buffer!\n");
+ xen_swiotlb_fixup(overflow, io_tlb_overflow, io_tlb_overflow >> IO_TLB_SHIFT);
+
+ swiotlb_init_with_buffer(bytes, buffer, overflow);
}
}
On Mon, 1 Jun 2009, Ian Campbell wrote:
> This function is intended to replaces is_buffer_dma_capable with a
> more generic interface.
-ENOPARSE
Which function ? What means "is intended to replaces" ? Is it intended
or does it replace something ? What is the "more generic interface" ?
dma_map_range() has been introduced in patch 2/11 so it would be nice
to have a reference to that change and take the existance of
dma_map_range() as granted. There is no intention. We replace function
A with function B and the blurb about intention and more generic
interface is useless and confusing.
Thanks,
tglx
> Signed-off-by: Ian Campbell <[email protected]>
> Cc: FUJITA Tomonori <[email protected]>
> Cc: Jeremy Fitzhardinge <[email protected]>
> Cc: [email protected]
> ---
> arch/x86/kernel/pci-dma.c | 5 +----
> 1 files changed, 1 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
> index f4c1b03..458b7b5 100644
> --- a/arch/x86/kernel/pci-dma.c
> +++ b/arch/x86/kernel/pci-dma.c
> @@ -160,7 +160,6 @@ void *dma_generic_alloc_coherent(struct device *dev, size_t size,
> {
> unsigned long dma_mask;
> struct page *page;
> - dma_addr_t addr;
>
> dma_mask = dma_alloc_coherent_mask(dev, flag);
>
> @@ -170,8 +169,7 @@ again:
> if (!page)
> return NULL;
>
> - addr = page_to_phys(page);
> - if (!is_buffer_dma_capable(dma_mask, addr, size)) {
> + if (!dma_map_range(dev, dma_mask, page_to_phys(page), size, dma_addr)) {
> __free_pages(page, get_order(size));
>
> if (dma_mask < DMA_BIT_MASK(32) && !(flag & GFP_DMA)) {
> @@ -182,7 +180,6 @@ again:
> return NULL;
> }
>
> - *dma_addr = addr;
> return page_address(page);
> }
>
> --
> 1.5.6.5
>
On Mon, 1 Jun 2009, Ian Campbell wrote:
> This function is intended to replaces is_buffer_dma_capable with a
> more generic interface.
Another instance of copied and not updated changelog.
> Signed-off-by: Ian Campbell <[email protected]>
> Cc: FUJITA Tomonori <[email protected]>
> Cc: Jeremy Fitzhardinge <[email protected]>
> Cc: [email protected]
> ---
> arch/x86/kernel/pci-gart_64.c | 6 ++++--
> 1 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/kernel/pci-gart_64.c b/arch/x86/kernel/pci-gart_64.c
> index 1e8920d..eb043bd 100644
> --- a/arch/x86/kernel/pci-gart_64.c
> +++ b/arch/x86/kernel/pci-gart_64.c
> @@ -190,14 +190,16 @@ static void iommu_full(struct device *dev, size_t size, int dir)
> static inline int
> need_iommu(struct device *dev, unsigned long addr, size_t size)
> {
> + dma_addr_t dma_addr;
> return force_iommu ||
> - !is_buffer_dma_capable(*dev->dma_mask, addr, size);
> + !dma_map_range(dev, *dev->dma_mask, addr, size, &dma_addr);
> }
>
> static inline int
> nonforced_iommu(struct device *dev, unsigned long addr, size_t size)
> {
> - return !is_buffer_dma_capable(*dev->dma_mask, addr, size);
> + dma_addr_t dma_addr;
> + return !dma_map_range(dev, *dev->dma_mask, addr, size, &dma_addr);
> }
>
> /* Map a single continuous physical area into the IOMMU.
> --
> 1.5.6.5
>
On Mon, 2009-06-01 at 16:32 +0100, Ian Campbell wrote:
> This series:
> * removes the swiotlb_(arch_)_phys_to_bus and bus_to_phys __weak
> hooks, replacing them with an architecture-specific phys_to_dma and
> dma_to_phys interface. These are used by both PowerPC and Xen to
> provide the correct mapping from physical to DMA addresses.
> * removes the swiotlb_address_needs_mapping and
> swiotlb_range_needs_mapping __weak functions as well as
> is_buffer_dma_capable (which should never have been a generic
> function). All three are replaced by a single architecture-specific
> interface which meets the needs of both PowerPC and Xen.
> * removes the swiotlb_virt_to_bus __weak function and replaces it with
> a CONFIG_HIGHMEM compatible version when high memory is in use. This
> is needed for 32 bit PowerPC swiotlb support.
> * removes the swiotlb_alloc* __weak functions and replaces them with
> swiotlb_init_with_buffer which allows the use of a caller allocated
> buffer (and emergency pool).
>
> I think these new interfaces are cleaner than the existing __weak
> functions and isolate the swiotlb code from architecture internals.
>
> This series does not contain any Xen or PowerPC specific changes, those
> will follow in separate postings. The complete patchset has been boot
> tested under Xen and native-x86 and compiled for IA64 and PowerPC
>
> Changes since v1:
> - Fixed compile error in swiotlb_dma_to_virt highmem version. Moved
> #ifdef into function to avoid prototype drift.
> - checkpatch fixes.
> - missed a swiotlb_arch_range_needs_mapping in swiotlb.h and x86
> pci-swiotlb.c and swiotlb_bus_to_phys/phys_to_bus implementations in
> x86.
> - additionally replaced __weak swiotlb_alloc* with
> swiotlb_init_with_buffer.
Looks like I was only CCed on part of them... it's not very handy for me
as I end up having some of the patches in one folder and some
elsewhere :-)
I don't have a firm objection but they will have to go through Becky
and Kumar since they are the one who need swiotlb for their embedded
platforms.
Cheers,
Ben.
On Fri, 2009-07-10 at 15:55 +1000, Benjamin Herrenschmidt wrote:
> On Mon, 2009-06-01 at 16:32 +0100, Ian Campbell wrote:
> > This series:[...]
> Looks like I was only CCed on part of them... it's not very handy for me
> as I end up having some of the patches in one folder and some
> elsewhere :-)
Sorry about that -- I was concerned about spamming the world and his dog
with patches which didn't directly interest them. For any future series
of this type I'll CC everyone on everything.
Ian.