Currently, there are two users on CMA functionality, one is the DMA
subsystem and the other is the kvm on powerpc. They have their own code
to manage CMA reserved area even if they looks really similar.
>From my guess, it is caused by some needs on bitmap management. Kvm side
wants to maintain bitmap not for 1 page, but for more size. Eventually it
use bitmap where one bit represents 64 pages.
When I implement CMA related patches, I should change those two places
to apply my change and it seem to be painful to me. I want to change
this situation and reduce future code management overhead through
this patch.
This change could also help developer who want to use CMA in their
new feature development, since they can use CMA easily without
copying & pasting this reserved area management code.
v2:
Although this patchset looks very different with v1, the end result,
that is, mm/cma.c is same with v1's one. So I carry Ack to patch 6-7.
Patch 1-5 prepare some features to cover ppc kvm's requirements.
Patch 6-7 generalize CMA reserved area management code and change users
to use it.
Patch 8-10 clean-up minor things.
Joonsoo Kim (10):
DMA, CMA: clean-up log message
DMA, CMA: fix possible memory leak
DMA, CMA: separate core cma management codes from DMA APIs
DMA, CMA: support alignment constraint on cma region
DMA, CMA: support arbitrary bitmap granularity
CMA: generalize CMA reserved area management functionality
PPC, KVM, CMA: use general CMA reserved area management framework
mm, cma: clean-up cma allocation error path
mm, cma: move output param to the end of param list
mm, cma: use spinlock instead of mutex
arch/powerpc/kvm/book3s_hv_builtin.c | 17 +-
arch/powerpc/kvm/book3s_hv_cma.c | 240 ------------------------
arch/powerpc/kvm/book3s_hv_cma.h | 27 ---
drivers/base/Kconfig | 10 -
drivers/base/dma-contiguous.c | 248 ++-----------------------
include/linux/cma.h | 12 ++
include/linux/dma-contiguous.h | 3 +-
mm/Kconfig | 11 ++
mm/Makefile | 1 +
mm/cma.c | 333 ++++++++++++++++++++++++++++++++++
10 files changed, 382 insertions(+), 520 deletions(-)
delete mode 100644 arch/powerpc/kvm/book3s_hv_cma.c
delete mode 100644 arch/powerpc/kvm/book3s_hv_cma.h
create mode 100644 include/linux/cma.h
create mode 100644 mm/cma.c
--
1.7.9.5
We should free memory for bitmap when we find zone mis-match,
otherwise this memory will leak.
Additionally, I copy code comment from ppc kvm's cma code to notify
why we need to check zone mis-match.
Signed-off-by: Joonsoo Kim <[email protected]>
diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
index bd0bb81..fb0cdce 100644
--- a/drivers/base/dma-contiguous.c
+++ b/drivers/base/dma-contiguous.c
@@ -177,14 +177,24 @@ static int __init cma_activate_area(struct cma *cma)
base_pfn = pfn;
for (j = pageblock_nr_pages; j; --j, pfn++) {
WARN_ON_ONCE(!pfn_valid(pfn));
+ /*
+ * alloc_contig_range requires the pfn range
+ * specified to be in the same zone. Make this
+ * simple by forcing the entire CMA resv range
+ * to be in the same zone.
+ */
if (page_zone(pfn_to_page(pfn)) != zone)
- return -EINVAL;
+ goto err;
}
init_cma_reserved_pageblock(pfn_to_page(base_pfn));
} while (--i);
mutex_init(&cma->lock);
return 0;
+
+err:
+ kfree(cma->bitmap);
+ return -EINVAL;
}
static struct cma cma_areas[MAX_CMA_AREAS];
--
1.7.9.5
We don't need explicit 'CMA:' prefix, since we already define prefix
'cma:' in pr_fmt. So remove it.
And, some logs print function name and others doesn't. This looks
bad to me, so I unify log format to print function name consistently.
Lastly, I add one more debug log on cma_activate_area().
Signed-off-by: Joonsoo Kim <[email protected]>
diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
index 83969f8..bd0bb81 100644
--- a/drivers/base/dma-contiguous.c
+++ b/drivers/base/dma-contiguous.c
@@ -144,7 +144,7 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
}
if (selected_size && !dma_contiguous_default_area) {
- pr_debug("%s: reserving %ld MiB for global area\n", __func__,
+ pr_debug("%s(): reserving %ld MiB for global area\n", __func__,
(unsigned long)selected_size / SZ_1M);
dma_contiguous_reserve_area(selected_size, selected_base,
@@ -163,8 +163,9 @@ static int __init cma_activate_area(struct cma *cma)
unsigned i = cma->count >> pageblock_order;
struct zone *zone;
- cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
+ pr_debug("%s()\n", __func__);
+ cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
if (!cma->bitmap)
return -ENOMEM;
@@ -234,7 +235,8 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
/* Sanity checks */
if (cma_area_count == ARRAY_SIZE(cma_areas)) {
- pr_err("Not enough slots for CMA reserved regions!\n");
+ pr_err("%s(): Not enough slots for CMA reserved regions!\n",
+ __func__);
return -ENOSPC;
}
@@ -274,14 +276,15 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
*res_cma = cma;
cma_area_count++;
- pr_info("CMA: reserved %ld MiB at %08lx\n", (unsigned long)size / SZ_1M,
- (unsigned long)base);
+ pr_info("%s(): reserved %ld MiB at %08lx\n",
+ __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
/* Architecture specific contiguous memory fixup. */
dma_contiguous_early_fixup(base, size);
return 0;
err:
- pr_err("CMA: failed to reserve %ld MiB\n", (unsigned long)size / SZ_1M);
+ pr_err("%s(): failed to reserve %ld MiB\n",
+ __func__, (unsigned long)size / SZ_1M);
return ret;
}
--
1.7.9.5
To prepare future generalization work on cma area management code,
we need to separate core cma management codes from DMA APIs.
We will extend these core functions to cover requirements of
ppc kvm's cma area management functionality in following patches.
This separation helps us not to touch DMA APIs while extending
core functions.
Signed-off-by: Joonsoo Kim <[email protected]>
diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
index fb0cdce..8a44c82 100644
--- a/drivers/base/dma-contiguous.c
+++ b/drivers/base/dma-contiguous.c
@@ -231,9 +231,9 @@ core_initcall(cma_init_reserved_areas);
* If @fixed is true, reserve contiguous area at exactly @base. If false,
* reserve in range from @base to @limit.
*/
-int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
- phys_addr_t limit, struct cma **res_cma,
- bool fixed)
+static int __init __dma_contiguous_reserve_area(phys_addr_t size,
+ phys_addr_t base, phys_addr_t limit,
+ struct cma **res_cma, bool fixed)
{
struct cma *cma = &cma_areas[cma_area_count];
phys_addr_t alignment;
@@ -288,16 +288,30 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
pr_info("%s(): reserved %ld MiB at %08lx\n",
__func__, (unsigned long)size / SZ_1M, (unsigned long)base);
-
- /* Architecture specific contiguous memory fixup. */
- dma_contiguous_early_fixup(base, size);
return 0;
+
err:
pr_err("%s(): failed to reserve %ld MiB\n",
__func__, (unsigned long)size / SZ_1M);
return ret;
}
+int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
+ phys_addr_t limit, struct cma **res_cma,
+ bool fixed)
+{
+ int ret;
+
+ ret = __dma_contiguous_reserve_area(size, base, limit, res_cma, fixed);
+ if (ret)
+ return ret;
+
+ /* Architecture specific contiguous memory fixup. */
+ dma_contiguous_early_fixup(base, size);
+
+ return 0;
+}
+
static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
{
mutex_lock(&cma->lock);
@@ -316,20 +330,16 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
* global one. Requires architecture specific dev_get_cma_area() helper
* function.
*/
-struct page *dma_alloc_from_contiguous(struct device *dev, int count,
+static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
unsigned int align)
{
unsigned long mask, pfn, pageno, start = 0;
- struct cma *cma = dev_get_cma_area(dev);
struct page *page = NULL;
int ret;
if (!cma || !cma->count)
return NULL;
- if (align > CONFIG_CMA_ALIGNMENT)
- align = CONFIG_CMA_ALIGNMENT;
-
pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
count, align);
@@ -377,6 +387,17 @@ struct page *dma_alloc_from_contiguous(struct device *dev, int count,
return page;
}
+struct page *dma_alloc_from_contiguous(struct device *dev, int count,
+ unsigned int align)
+{
+ struct cma *cma = dev_get_cma_area(dev);
+
+ if (align > CONFIG_CMA_ALIGNMENT)
+ align = CONFIG_CMA_ALIGNMENT;
+
+ return __dma_alloc_from_contiguous(cma, count, align);
+}
+
/**
* dma_release_from_contiguous() - release allocated pages
* @dev: Pointer to device for which the pages were allocated.
@@ -387,10 +408,9 @@ struct page *dma_alloc_from_contiguous(struct device *dev, int count,
* It returns false when provided pages do not belong to contiguous area and
* true otherwise.
*/
-bool dma_release_from_contiguous(struct device *dev, struct page *pages,
+static bool __dma_release_from_contiguous(struct cma *cma, struct page *pages,
int count)
{
- struct cma *cma = dev_get_cma_area(dev);
unsigned long pfn;
if (!cma || !pages)
@@ -410,3 +430,11 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages,
return true;
}
+
+bool dma_release_from_contiguous(struct device *dev, struct page *pages,
+ int count)
+{
+ struct cma *cma = dev_get_cma_area(dev);
+
+ return __dma_release_from_contiguous(cma, pages, count);
+}
--
1.7.9.5
ppc kvm's cma area management needs alignment constraint on
cma region. So support it to prepare generalization of cma area
management functionality.
Additionally, add some comments which tell us why alignment
constraint is needed on cma region.
Signed-off-by: Joonsoo Kim <[email protected]>
diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
index 8a44c82..bc4c171 100644
--- a/drivers/base/dma-contiguous.c
+++ b/drivers/base/dma-contiguous.c
@@ -32,6 +32,7 @@
#include <linux/swap.h>
#include <linux/mm_types.h>
#include <linux/dma-contiguous.h>
+#include <linux/log2.h>
struct cma {
unsigned long base_pfn;
@@ -219,6 +220,7 @@ core_initcall(cma_init_reserved_areas);
* @size: Size of the reserved area (in bytes),
* @base: Base address of the reserved area optional, use 0 for any
* @limit: End address of the reserved memory (optional, 0 for any).
+ * @alignment: Alignment for the contiguous memory area, should be power of 2
* @res_cma: Pointer to store the created cma region.
* @fixed: hint about where to place the reserved area
*
@@ -233,15 +235,15 @@ core_initcall(cma_init_reserved_areas);
*/
static int __init __dma_contiguous_reserve_area(phys_addr_t size,
phys_addr_t base, phys_addr_t limit,
+ phys_addr_t alignment,
struct cma **res_cma, bool fixed)
{
struct cma *cma = &cma_areas[cma_area_count];
- phys_addr_t alignment;
int ret = 0;
- pr_debug("%s(size %lx, base %08lx, limit %08lx)\n", __func__,
- (unsigned long)size, (unsigned long)base,
- (unsigned long)limit);
+ pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
+ __func__, (unsigned long)size, (unsigned long)base,
+ (unsigned long)limit, (unsigned long)alignment);
/* Sanity checks */
if (cma_area_count == ARRAY_SIZE(cma_areas)) {
@@ -253,8 +255,17 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
if (!size)
return -EINVAL;
- /* Sanitise input arguments */
- alignment = PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order);
+ if (alignment && !is_power_of_2(alignment))
+ return -EINVAL;
+
+ /*
+ * Sanitise input arguments.
+ * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
+ * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
+ * and CMA property will be broken.
+ */
+ alignment = max(alignment,
+ (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
base = ALIGN(base, alignment);
size = ALIGN(size, alignment);
limit &= ~(alignment - 1);
@@ -302,7 +313,8 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
{
int ret;
- ret = __dma_contiguous_reserve_area(size, base, limit, res_cma, fixed);
+ ret = __dma_contiguous_reserve_area(size, base, limit, 0,
+ res_cma, fixed);
if (ret)
return ret;
--
1.7.9.5
ppc kvm's cma region management requires arbitrary bitmap granularity,
since they want to reserve very large memory and manage this region
with bitmap that one bit for several pages to reduce management overheads.
So support arbitrary bitmap granularity for following generalization.
Signed-off-by: Joonsoo Kim <[email protected]>
diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
index bc4c171..9bc9340 100644
--- a/drivers/base/dma-contiguous.c
+++ b/drivers/base/dma-contiguous.c
@@ -38,6 +38,7 @@ struct cma {
unsigned long base_pfn;
unsigned long count;
unsigned long *bitmap;
+ int order_per_bit; /* Order of pages represented by one bit */
struct mutex lock;
};
@@ -157,9 +158,38 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
static DEFINE_MUTEX(cma_mutex);
+static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
+{
+ return (1 << (align_order >> cma->order_per_bit)) - 1;
+}
+
+static unsigned long cma_bitmap_maxno(struct cma *cma)
+{
+ return cma->count >> cma->order_per_bit;
+}
+
+static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
+ unsigned long pages)
+{
+ return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
+}
+
+static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
+{
+ unsigned long bitmapno, nr_bits;
+
+ bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
+ nr_bits = cma_bitmap_pages_to_bits(cma, count);
+
+ mutex_lock(&cma->lock);
+ bitmap_clear(cma->bitmap, bitmapno, nr_bits);
+ mutex_unlock(&cma->lock);
+}
+
static int __init cma_activate_area(struct cma *cma)
{
- int bitmap_size = BITS_TO_LONGS(cma->count) * sizeof(long);
+ int bitmap_maxno = cma_bitmap_maxno(cma);
+ int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
unsigned i = cma->count >> pageblock_order;
struct zone *zone;
@@ -221,6 +251,7 @@ core_initcall(cma_init_reserved_areas);
* @base: Base address of the reserved area optional, use 0 for any
* @limit: End address of the reserved memory (optional, 0 for any).
* @alignment: Alignment for the contiguous memory area, should be power of 2
+ * @order_per_bit: Order of pages represented by one bit on bitmap.
* @res_cma: Pointer to store the created cma region.
* @fixed: hint about where to place the reserved area
*
@@ -235,7 +266,7 @@ core_initcall(cma_init_reserved_areas);
*/
static int __init __dma_contiguous_reserve_area(phys_addr_t size,
phys_addr_t base, phys_addr_t limit,
- phys_addr_t alignment,
+ phys_addr_t alignment, int order_per_bit,
struct cma **res_cma, bool fixed)
{
struct cma *cma = &cma_areas[cma_area_count];
@@ -269,6 +300,8 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
base = ALIGN(base, alignment);
size = ALIGN(size, alignment);
limit &= ~(alignment - 1);
+ /* size should be aligned with order_per_bit */
+ BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
/* Reserve memory */
if (base && fixed) {
@@ -294,6 +327,7 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
*/
cma->base_pfn = PFN_DOWN(base);
cma->count = size >> PAGE_SHIFT;
+ cma->order_per_bit = order_per_bit;
*res_cma = cma;
cma_area_count++;
@@ -313,7 +347,7 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
{
int ret;
- ret = __dma_contiguous_reserve_area(size, base, limit, 0,
+ ret = __dma_contiguous_reserve_area(size, base, limit, 0, 0,
res_cma, fixed);
if (ret)
return ret;
@@ -324,13 +358,6 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
return 0;
}
-static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
-{
- mutex_lock(&cma->lock);
- bitmap_clear(cma->bitmap, pfn - cma->base_pfn, count);
- mutex_unlock(&cma->lock);
-}
-
/**
* dma_alloc_from_contiguous() - allocate pages from contiguous area
* @dev: Pointer to device for which the allocation is performed.
@@ -345,7 +372,8 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
unsigned int align)
{
- unsigned long mask, pfn, pageno, start = 0;
+ unsigned long mask, pfn, start = 0;
+ unsigned long bitmap_maxno, bitmapno, nr_bits;
struct page *page = NULL;
int ret;
@@ -358,18 +386,19 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
if (!count)
return NULL;
- mask = (1 << align) - 1;
-
+ mask = cma_bitmap_aligned_mask(cma, align);
+ bitmap_maxno = cma_bitmap_maxno(cma);
+ nr_bits = cma_bitmap_pages_to_bits(cma, count);
for (;;) {
mutex_lock(&cma->lock);
- pageno = bitmap_find_next_zero_area(cma->bitmap, cma->count,
- start, count, mask);
- if (pageno >= cma->count) {
+ bitmapno = bitmap_find_next_zero_area(cma->bitmap,
+ bitmap_maxno, start, nr_bits, mask);
+ if (bitmapno >= bitmap_maxno) {
mutex_unlock(&cma->lock);
break;
}
- bitmap_set(cma->bitmap, pageno, count);
+ bitmap_set(cma->bitmap, bitmapno, nr_bits);
/*
* It's safe to drop the lock here. We've marked this region for
* our exclusive use. If the migration fails we will take the
@@ -377,7 +406,7 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
*/
mutex_unlock(&cma->lock);
- pfn = cma->base_pfn + pageno;
+ pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
mutex_lock(&cma_mutex);
ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
mutex_unlock(&cma_mutex);
@@ -392,7 +421,7 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
pr_debug("%s(): memory range at %p is busy, retrying\n",
__func__, pfn_to_page(pfn));
/* try again with a bit different memory target */
- start = pageno + mask + 1;
+ start = bitmapno + mask + 1;
}
pr_debug("%s(): returned %p\n", __func__, page);
--
1.7.9.5
Now, we have general CMA reserved area management framework,
so use it for future maintainabilty. There is no functional change.
Acked-by: Michal Nazarewicz <[email protected]>
Acked-by: Paolo Bonzini <[email protected]>
Signed-off-by: Joonsoo Kim <[email protected]>
diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c
index 7cde8a6..28ec226 100644
--- a/arch/powerpc/kvm/book3s_hv_builtin.c
+++ b/arch/powerpc/kvm/book3s_hv_builtin.c
@@ -16,12 +16,14 @@
#include <linux/init.h>
#include <linux/memblock.h>
#include <linux/sizes.h>
+#include <linux/cma.h>
#include <asm/cputable.h>
#include <asm/kvm_ppc.h>
#include <asm/kvm_book3s.h>
-#include "book3s_hv_cma.h"
+#define KVM_CMA_CHUNK_ORDER 18
+
/*
* Hash page table alignment on newer cpus(CPU_FTR_ARCH_206)
* should be power of 2.
@@ -43,6 +45,8 @@ static unsigned long kvm_cma_resv_ratio = 5;
unsigned long kvm_rma_pages = (1 << 27) >> PAGE_SHIFT; /* 128MB */
EXPORT_SYMBOL_GPL(kvm_rma_pages);
+static struct cma *kvm_cma;
+
/* Work out RMLS (real mode limit selector) field value for a given RMA size.
Assumes POWER7 or PPC970. */
static inline int lpcr_rmls(unsigned long rma_size)
@@ -97,7 +101,7 @@ struct kvm_rma_info *kvm_alloc_rma()
ri = kmalloc(sizeof(struct kvm_rma_info), GFP_KERNEL);
if (!ri)
return NULL;
- page = kvm_alloc_cma(kvm_rma_pages, kvm_rma_pages);
+ page = cma_alloc(kvm_cma, kvm_rma_pages, get_order(kvm_rma_pages));
if (!page)
goto err_out;
atomic_set(&ri->use_count, 1);
@@ -112,7 +116,7 @@ EXPORT_SYMBOL_GPL(kvm_alloc_rma);
void kvm_release_rma(struct kvm_rma_info *ri)
{
if (atomic_dec_and_test(&ri->use_count)) {
- kvm_release_cma(pfn_to_page(ri->base_pfn), kvm_rma_pages);
+ cma_release(kvm_cma, pfn_to_page(ri->base_pfn), kvm_rma_pages);
kfree(ri);
}
}
@@ -134,13 +138,13 @@ struct page *kvm_alloc_hpt(unsigned long nr_pages)
/* Old CPUs require HPT aligned on a multiple of its size */
if (!cpu_has_feature(CPU_FTR_ARCH_206))
align_pages = nr_pages;
- return kvm_alloc_cma(nr_pages, align_pages);
+ return cma_alloc(kvm_cma, nr_pages, get_order(align_pages));
}
EXPORT_SYMBOL_GPL(kvm_alloc_hpt);
void kvm_release_hpt(struct page *page, unsigned long nr_pages)
{
- kvm_release_cma(page, nr_pages);
+ cma_release(kvm_cma, page, nr_pages);
}
EXPORT_SYMBOL_GPL(kvm_release_hpt);
@@ -179,7 +183,8 @@ void __init kvm_cma_reserve(void)
align_size = HPT_ALIGN_PAGES << PAGE_SHIFT;
align_size = max(kvm_rma_pages << PAGE_SHIFT, align_size);
- kvm_cma_declare_contiguous(selected_size, align_size);
+ cma_declare_contiguous(selected_size, 0, 0, align_size,
+ KVM_CMA_CHUNK_ORDER - PAGE_SHIFT, &kvm_cma, false);
}
}
diff --git a/arch/powerpc/kvm/book3s_hv_cma.c b/arch/powerpc/kvm/book3s_hv_cma.c
deleted file mode 100644
index d9d3d85..0000000
--- a/arch/powerpc/kvm/book3s_hv_cma.c
+++ /dev/null
@@ -1,240 +0,0 @@
-/*
- * Contiguous Memory Allocator for ppc KVM hash pagetable based on CMA
- * for DMA mapping framework
- *
- * Copyright IBM Corporation, 2013
- * Author Aneesh Kumar K.V <[email protected]>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of the
- * License or (at your optional) any later version of the license.
- *
- */
-#define pr_fmt(fmt) "kvm_cma: " fmt
-
-#ifdef CONFIG_CMA_DEBUG
-#ifndef DEBUG
-# define DEBUG
-#endif
-#endif
-
-#include <linux/memblock.h>
-#include <linux/mutex.h>
-#include <linux/sizes.h>
-#include <linux/slab.h>
-
-#include "book3s_hv_cma.h"
-
-struct kvm_cma {
- unsigned long base_pfn;
- unsigned long count;
- unsigned long *bitmap;
-};
-
-static DEFINE_MUTEX(kvm_cma_mutex);
-static struct kvm_cma kvm_cma_area;
-
-/**
- * kvm_cma_declare_contiguous() - reserve area for contiguous memory handling
- * for kvm hash pagetable
- * @size: Size of the reserved memory.
- * @alignment: Alignment for the contiguous memory area
- *
- * This function reserves memory for kvm cma area. It should be
- * called by arch code when early allocator (memblock or bootmem)
- * is still activate.
- */
-long __init kvm_cma_declare_contiguous(phys_addr_t size, phys_addr_t alignment)
-{
- long base_pfn;
- phys_addr_t addr;
- struct kvm_cma *cma = &kvm_cma_area;
-
- pr_debug("%s(size %lx)\n", __func__, (unsigned long)size);
-
- if (!size)
- return -EINVAL;
- /*
- * Sanitise input arguments.
- * We should be pageblock aligned for CMA.
- */
- alignment = max(alignment, (phys_addr_t)(PAGE_SIZE << pageblock_order));
- size = ALIGN(size, alignment);
- /*
- * Reserve memory
- * Use __memblock_alloc_base() since
- * memblock_alloc_base() panic()s.
- */
- addr = __memblock_alloc_base(size, alignment, 0);
- if (!addr) {
- base_pfn = -ENOMEM;
- goto err;
- } else
- base_pfn = PFN_DOWN(addr);
-
- /*
- * Each reserved area must be initialised later, when more kernel
- * subsystems (like slab allocator) are available.
- */
- cma->base_pfn = base_pfn;
- cma->count = size >> PAGE_SHIFT;
- pr_info("CMA: reserved %ld MiB\n", (unsigned long)size / SZ_1M);
- return 0;
-err:
- pr_err("CMA: failed to reserve %ld MiB\n", (unsigned long)size / SZ_1M);
- return base_pfn;
-}
-
-/**
- * kvm_alloc_cma() - allocate pages from contiguous area
- * @nr_pages: Requested number of pages.
- * @align_pages: Requested alignment in number of pages
- *
- * This function allocates memory buffer for hash pagetable.
- */
-struct page *kvm_alloc_cma(unsigned long nr_pages, unsigned long align_pages)
-{
- int ret;
- struct page *page = NULL;
- struct kvm_cma *cma = &kvm_cma_area;
- unsigned long chunk_count, nr_chunk;
- unsigned long mask, pfn, pageno, start = 0;
-
-
- if (!cma || !cma->count)
- return NULL;
-
- pr_debug("%s(cma %p, count %lu, align pages %lu)\n", __func__,
- (void *)cma, nr_pages, align_pages);
-
- if (!nr_pages)
- return NULL;
- /*
- * align mask with chunk size. The bit tracks pages in chunk size
- */
- VM_BUG_ON(!is_power_of_2(align_pages));
- mask = (align_pages >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT)) - 1;
- BUILD_BUG_ON(PAGE_SHIFT > KVM_CMA_CHUNK_ORDER);
-
- chunk_count = cma->count >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT);
- nr_chunk = nr_pages >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT);
-
- mutex_lock(&kvm_cma_mutex);
- for (;;) {
- pageno = bitmap_find_next_zero_area(cma->bitmap, chunk_count,
- start, nr_chunk, mask);
- if (pageno >= chunk_count)
- break;
-
- pfn = cma->base_pfn + (pageno << (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT));
- ret = alloc_contig_range(pfn, pfn + nr_pages, MIGRATE_CMA);
- if (ret == 0) {
- bitmap_set(cma->bitmap, pageno, nr_chunk);
- page = pfn_to_page(pfn);
- memset(pfn_to_kaddr(pfn), 0, nr_pages << PAGE_SHIFT);
- break;
- } else if (ret != -EBUSY) {
- break;
- }
- pr_debug("%s(): memory range at %p is busy, retrying\n",
- __func__, pfn_to_page(pfn));
- /* try again with a bit different memory target */
- start = pageno + mask + 1;
- }
- mutex_unlock(&kvm_cma_mutex);
- pr_debug("%s(): returned %p\n", __func__, page);
- return page;
-}
-
-/**
- * kvm_release_cma() - release allocated pages for hash pagetable
- * @pages: Allocated pages.
- * @nr_pages: Number of allocated pages.
- *
- * This function releases memory allocated by kvm_alloc_cma().
- * It returns false when provided pages do not belong to contiguous area and
- * true otherwise.
- */
-bool kvm_release_cma(struct page *pages, unsigned long nr_pages)
-{
- unsigned long pfn;
- unsigned long nr_chunk;
- struct kvm_cma *cma = &kvm_cma_area;
-
- if (!cma || !pages)
- return false;
-
- pr_debug("%s(page %p count %lu)\n", __func__, (void *)pages, nr_pages);
-
- pfn = page_to_pfn(pages);
-
- if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count)
- return false;
-
- VM_BUG_ON(pfn + nr_pages > cma->base_pfn + cma->count);
- nr_chunk = nr_pages >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT);
-
- mutex_lock(&kvm_cma_mutex);
- bitmap_clear(cma->bitmap,
- (pfn - cma->base_pfn) >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT),
- nr_chunk);
- free_contig_range(pfn, nr_pages);
- mutex_unlock(&kvm_cma_mutex);
-
- return true;
-}
-
-static int __init kvm_cma_activate_area(unsigned long base_pfn,
- unsigned long count)
-{
- unsigned long pfn = base_pfn;
- unsigned i = count >> pageblock_order;
- struct zone *zone;
-
- WARN_ON_ONCE(!pfn_valid(pfn));
- zone = page_zone(pfn_to_page(pfn));
- do {
- unsigned j;
- base_pfn = pfn;
- for (j = pageblock_nr_pages; j; --j, pfn++) {
- WARN_ON_ONCE(!pfn_valid(pfn));
- /*
- * alloc_contig_range requires the pfn range
- * specified to be in the same zone. Make this
- * simple by forcing the entire CMA resv range
- * to be in the same zone.
- */
- if (page_zone(pfn_to_page(pfn)) != zone)
- return -EINVAL;
- }
- init_cma_reserved_pageblock(pfn_to_page(base_pfn));
- } while (--i);
- return 0;
-}
-
-static int __init kvm_cma_init_reserved_areas(void)
-{
- int bitmap_size, ret;
- unsigned long chunk_count;
- struct kvm_cma *cma = &kvm_cma_area;
-
- pr_debug("%s()\n", __func__);
- if (!cma->count)
- return 0;
- chunk_count = cma->count >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT);
- bitmap_size = BITS_TO_LONGS(chunk_count) * sizeof(long);
- cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
- if (!cma->bitmap)
- return -ENOMEM;
-
- ret = kvm_cma_activate_area(cma->base_pfn, cma->count);
- if (ret)
- goto error;
- return 0;
-
-error:
- kfree(cma->bitmap);
- return ret;
-}
-core_initcall(kvm_cma_init_reserved_areas);
diff --git a/arch/powerpc/kvm/book3s_hv_cma.h b/arch/powerpc/kvm/book3s_hv_cma.h
deleted file mode 100644
index 655144f..0000000
--- a/arch/powerpc/kvm/book3s_hv_cma.h
+++ /dev/null
@@ -1,27 +0,0 @@
-/*
- * Contiguous Memory Allocator for ppc KVM hash pagetable based on CMA
- * for DMA mapping framework
- *
- * Copyright IBM Corporation, 2013
- * Author Aneesh Kumar K.V <[email protected]>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of the
- * License or (at your optional) any later version of the license.
- *
- */
-
-#ifndef __POWERPC_KVM_CMA_ALLOC_H__
-#define __POWERPC_KVM_CMA_ALLOC_H__
-/*
- * Both RMA and Hash page allocation will be multiple of 256K.
- */
-#define KVM_CMA_CHUNK_ORDER 18
-
-extern struct page *kvm_alloc_cma(unsigned long nr_pages,
- unsigned long align_pages);
-extern bool kvm_release_cma(struct page *pages, unsigned long nr_pages);
-extern long kvm_cma_declare_contiguous(phys_addr_t size,
- phys_addr_t alignment) __init;
-#endif
--
1.7.9.5
Currently, there are two users on CMA functionality, one is the DMA
subsystem and the other is the kvm on powerpc. They have their own code
to manage CMA reserved area even if they looks really similar.
>From my guess, it is caused by some needs on bitmap management. Kvm side
wants to maintain bitmap not for 1 page, but for more size. Eventually it
use bitmap where one bit represents 64 pages.
When I implement CMA related patches, I should change those two places
to apply my change and it seem to be painful to me. I want to change
this situation and reduce future code management overhead through
this patch.
This change could also help developer who want to use CMA in their
new feature development, since they can use CMA easily without
copying & pasting this reserved area management code.
In previous patches, we have prepared some features to generalize
CMA reserved area management and now it's time to do it. This patch
moves core functions to mm/cma.c and change DMA APIs to use
these functions.
There is no functional change in DMA APIs.
v2: There is no big change from v1 in mm/cma.c. Mostly renaming.
Acked-by: Michal Nazarewicz <[email protected]>
Signed-off-by: Joonsoo Kim <[email protected]>
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
index 00e13ce..4eac559 100644
--- a/drivers/base/Kconfig
+++ b/drivers/base/Kconfig
@@ -283,16 +283,6 @@ config CMA_ALIGNMENT
If unsure, leave the default value "8".
-config CMA_AREAS
- int "Maximum count of the CMA device-private areas"
- default 7
- help
- CMA allows to create CMA areas for particular devices. This parameter
- sets the maximum number of such device private CMA areas in the
- system.
-
- If unsure, leave the default value "7".
-
endif
endmenu
diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
index 9bc9340..f177f73 100644
--- a/drivers/base/dma-contiguous.c
+++ b/drivers/base/dma-contiguous.c
@@ -24,25 +24,10 @@
#include <linux/memblock.h>
#include <linux/err.h>
-#include <linux/mm.h>
-#include <linux/mutex.h>
-#include <linux/page-isolation.h>
#include <linux/sizes.h>
-#include <linux/slab.h>
-#include <linux/swap.h>
-#include <linux/mm_types.h>
#include <linux/dma-contiguous.h>
#include <linux/log2.h>
-
-struct cma {
- unsigned long base_pfn;
- unsigned long count;
- unsigned long *bitmap;
- int order_per_bit; /* Order of pages represented by one bit */
- struct mutex lock;
-};
-
-struct cma *dma_contiguous_default_area;
+#include <linux/cma.h>
#ifdef CONFIG_CMA_SIZE_MBYTES
#define CMA_SIZE_MBYTES CONFIG_CMA_SIZE_MBYTES
@@ -50,6 +35,8 @@ struct cma *dma_contiguous_default_area;
#define CMA_SIZE_MBYTES 0
#endif
+struct cma *dma_contiguous_default_area;
+
/*
* Default global CMA area size can be defined in kernel's .config.
* This is useful mainly for distro maintainers to create a kernel
@@ -156,199 +143,13 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
}
}
-static DEFINE_MUTEX(cma_mutex);
-
-static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
-{
- return (1 << (align_order >> cma->order_per_bit)) - 1;
-}
-
-static unsigned long cma_bitmap_maxno(struct cma *cma)
-{
- return cma->count >> cma->order_per_bit;
-}
-
-static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
- unsigned long pages)
-{
- return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
-}
-
-static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
-{
- unsigned long bitmapno, nr_bits;
-
- bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
- nr_bits = cma_bitmap_pages_to_bits(cma, count);
-
- mutex_lock(&cma->lock);
- bitmap_clear(cma->bitmap, bitmapno, nr_bits);
- mutex_unlock(&cma->lock);
-}
-
-static int __init cma_activate_area(struct cma *cma)
-{
- int bitmap_maxno = cma_bitmap_maxno(cma);
- int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
- unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
- unsigned i = cma->count >> pageblock_order;
- struct zone *zone;
-
- pr_debug("%s()\n", __func__);
-
- cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
- if (!cma->bitmap)
- return -ENOMEM;
-
- WARN_ON_ONCE(!pfn_valid(pfn));
- zone = page_zone(pfn_to_page(pfn));
-
- do {
- unsigned j;
- base_pfn = pfn;
- for (j = pageblock_nr_pages; j; --j, pfn++) {
- WARN_ON_ONCE(!pfn_valid(pfn));
- /*
- * alloc_contig_range requires the pfn range
- * specified to be in the same zone. Make this
- * simple by forcing the entire CMA resv range
- * to be in the same zone.
- */
- if (page_zone(pfn_to_page(pfn)) != zone)
- goto err;
- }
- init_cma_reserved_pageblock(pfn_to_page(base_pfn));
- } while (--i);
-
- mutex_init(&cma->lock);
- return 0;
-
-err:
- kfree(cma->bitmap);
- return -EINVAL;
-}
-
-static struct cma cma_areas[MAX_CMA_AREAS];
-static unsigned cma_area_count;
-
-static int __init cma_init_reserved_areas(void)
-{
- int i;
-
- for (i = 0; i < cma_area_count; i++) {
- int ret = cma_activate_area(&cma_areas[i]);
- if (ret)
- return ret;
- }
-
- return 0;
-}
-core_initcall(cma_init_reserved_areas);
-
-/**
- * dma_contiguous_reserve_area() - reserve custom contiguous area
- * @size: Size of the reserved area (in bytes),
- * @base: Base address of the reserved area optional, use 0 for any
- * @limit: End address of the reserved memory (optional, 0 for any).
- * @alignment: Alignment for the contiguous memory area, should be power of 2
- * @order_per_bit: Order of pages represented by one bit on bitmap.
- * @res_cma: Pointer to store the created cma region.
- * @fixed: hint about where to place the reserved area
- *
- * This function reserves memory from early allocator. It should be
- * called by arch specific code once the early allocator (memblock or bootmem)
- * has been activated and all other subsystems have already allocated/reserved
- * memory. This function allows to create custom reserved areas for specific
- * devices.
- *
- * If @fixed is true, reserve contiguous area at exactly @base. If false,
- * reserve in range from @base to @limit.
- */
-static int __init __dma_contiguous_reserve_area(phys_addr_t size,
- phys_addr_t base, phys_addr_t limit,
- phys_addr_t alignment, int order_per_bit,
- struct cma **res_cma, bool fixed)
-{
- struct cma *cma = &cma_areas[cma_area_count];
- int ret = 0;
-
- pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
- __func__, (unsigned long)size, (unsigned long)base,
- (unsigned long)limit, (unsigned long)alignment);
-
- /* Sanity checks */
- if (cma_area_count == ARRAY_SIZE(cma_areas)) {
- pr_err("%s(): Not enough slots for CMA reserved regions!\n",
- __func__);
- return -ENOSPC;
- }
-
- if (!size)
- return -EINVAL;
-
- if (alignment && !is_power_of_2(alignment))
- return -EINVAL;
-
- /*
- * Sanitise input arguments.
- * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
- * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
- * and CMA property will be broken.
- */
- alignment = max(alignment,
- (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
- base = ALIGN(base, alignment);
- size = ALIGN(size, alignment);
- limit &= ~(alignment - 1);
- /* size should be aligned with order_per_bit */
- BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
-
- /* Reserve memory */
- if (base && fixed) {
- if (memblock_is_region_reserved(base, size) ||
- memblock_reserve(base, size) < 0) {
- ret = -EBUSY;
- goto err;
- }
- } else {
- phys_addr_t addr = memblock_alloc_range(size, alignment, base,
- limit);
- if (!addr) {
- ret = -ENOMEM;
- goto err;
- } else {
- base = addr;
- }
- }
-
- /*
- * Each reserved area must be initialised later, when more kernel
- * subsystems (like slab allocator) are available.
- */
- cma->base_pfn = PFN_DOWN(base);
- cma->count = size >> PAGE_SHIFT;
- cma->order_per_bit = order_per_bit;
- *res_cma = cma;
- cma_area_count++;
-
- pr_info("%s(): reserved %ld MiB at %08lx\n",
- __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
- return 0;
-
-err:
- pr_err("%s(): failed to reserve %ld MiB\n",
- __func__, (unsigned long)size / SZ_1M);
- return ret;
-}
-
int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
phys_addr_t limit, struct cma **res_cma,
bool fixed)
{
int ret;
- ret = __dma_contiguous_reserve_area(size, base, limit, 0, 0,
- res_cma, fixed);
+ ret = cma_declare_contiguous(size, base, limit, 0, 0, res_cma, fixed);
if (ret)
return ret;
@@ -358,124 +159,17 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
return 0;
}
-/**
- * dma_alloc_from_contiguous() - allocate pages from contiguous area
- * @dev: Pointer to device for which the allocation is performed.
- * @count: Requested number of pages.
- * @align: Requested alignment of pages (in PAGE_SIZE order).
- *
- * This function allocates memory buffer for specified device. It uses
- * device specific contiguous memory area if available or the default
- * global one. Requires architecture specific dev_get_cma_area() helper
- * function.
- */
-static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
- unsigned int align)
-{
- unsigned long mask, pfn, start = 0;
- unsigned long bitmap_maxno, bitmapno, nr_bits;
- struct page *page = NULL;
- int ret;
-
- if (!cma || !cma->count)
- return NULL;
-
- pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
- count, align);
-
- if (!count)
- return NULL;
-
- mask = cma_bitmap_aligned_mask(cma, align);
- bitmap_maxno = cma_bitmap_maxno(cma);
- nr_bits = cma_bitmap_pages_to_bits(cma, count);
-
- for (;;) {
- mutex_lock(&cma->lock);
- bitmapno = bitmap_find_next_zero_area(cma->bitmap,
- bitmap_maxno, start, nr_bits, mask);
- if (bitmapno >= bitmap_maxno) {
- mutex_unlock(&cma->lock);
- break;
- }
- bitmap_set(cma->bitmap, bitmapno, nr_bits);
- /*
- * It's safe to drop the lock here. We've marked this region for
- * our exclusive use. If the migration fails we will take the
- * lock again and unmark it.
- */
- mutex_unlock(&cma->lock);
-
- pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
- mutex_lock(&cma_mutex);
- ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
- mutex_unlock(&cma_mutex);
- if (ret == 0) {
- page = pfn_to_page(pfn);
- break;
- } else if (ret != -EBUSY) {
- clear_cma_bitmap(cma, pfn, count);
- break;
- }
- clear_cma_bitmap(cma, pfn, count);
- pr_debug("%s(): memory range at %p is busy, retrying\n",
- __func__, pfn_to_page(pfn));
- /* try again with a bit different memory target */
- start = bitmapno + mask + 1;
- }
-
- pr_debug("%s(): returned %p\n", __func__, page);
- return page;
-}
-
struct page *dma_alloc_from_contiguous(struct device *dev, int count,
unsigned int align)
{
- struct cma *cma = dev_get_cma_area(dev);
-
if (align > CONFIG_CMA_ALIGNMENT)
align = CONFIG_CMA_ALIGNMENT;
- return __dma_alloc_from_contiguous(cma, count, align);
-}
-
-/**
- * dma_release_from_contiguous() - release allocated pages
- * @dev: Pointer to device for which the pages were allocated.
- * @pages: Allocated pages.
- * @count: Number of allocated pages.
- *
- * This function releases memory allocated by dma_alloc_from_contiguous().
- * It returns false when provided pages do not belong to contiguous area and
- * true otherwise.
- */
-static bool __dma_release_from_contiguous(struct cma *cma, struct page *pages,
- int count)
-{
- unsigned long pfn;
-
- if (!cma || !pages)
- return false;
-
- pr_debug("%s(page %p)\n", __func__, (void *)pages);
-
- pfn = page_to_pfn(pages);
-
- if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count)
- return false;
-
- VM_BUG_ON(pfn + count > cma->base_pfn + cma->count);
-
- free_contig_range(pfn, count);
- clear_cma_bitmap(cma, pfn, count);
-
- return true;
+ return cma_alloc(dev_get_cma_area(dev), count, align);
}
bool dma_release_from_contiguous(struct device *dev, struct page *pages,
int count)
{
- struct cma *cma = dev_get_cma_area(dev);
-
- return __dma_release_from_contiguous(cma, pages, count);
+ return cma_release(dev_get_cma_area(dev), pages, count);
}
diff --git a/include/linux/cma.h b/include/linux/cma.h
new file mode 100644
index 0000000..e38efe9
--- /dev/null
+++ b/include/linux/cma.h
@@ -0,0 +1,12 @@
+#ifndef __CMA_H__
+#define __CMA_H__
+
+struct cma;
+
+extern int __init cma_declare_contiguous(phys_addr_t size,
+ phys_addr_t base, phys_addr_t limit,
+ phys_addr_t alignment, int order_per_bit,
+ struct cma **res_cma, bool fixed);
+extern struct page *cma_alloc(struct cma *cma, int count, unsigned int align);
+extern bool cma_release(struct cma *cma, struct page *pages, int count);
+#endif
diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h
index 772eab5..a40c1f3 100644
--- a/include/linux/dma-contiguous.h
+++ b/include/linux/dma-contiguous.h
@@ -53,9 +53,10 @@
#ifdef __KERNEL__
+#include <linux/device.h>
+
struct cma;
struct page;
-struct device;
#ifdef CONFIG_DMA_CMA
diff --git a/mm/Kconfig b/mm/Kconfig
index 3e9977a..f4899ec 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -508,6 +508,17 @@ config CMA_DEBUG
processing calls such as dma_alloc_from_contiguous().
This option does not affect warning and error messages.
+config CMA_AREAS
+ int "Maximum count of the CMA areas"
+ depends on CMA
+ default 7
+ help
+ CMA allows to create CMA areas for particular purpose, mainly,
+ used as device private area. This parameter sets the maximum
+ number of CMA area in the system.
+
+ If unsure, leave the default value "7".
+
config ZBUD
tristate
default n
diff --git a/mm/Makefile b/mm/Makefile
index 1eaa70b..bc0422b 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -62,3 +62,4 @@ obj-$(CONFIG_MEMORY_ISOLATION) += page_isolation.o
obj-$(CONFIG_ZBUD) += zbud.o
obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
+obj-$(CONFIG_CMA) += cma.o
diff --git a/mm/cma.c b/mm/cma.c
new file mode 100644
index 0000000..1e1b017
--- /dev/null
+++ b/mm/cma.c
@@ -0,0 +1,330 @@
+/*
+ * Contiguous Memory Allocator
+ *
+ * Copyright (c) 2010-2011 by Samsung Electronics.
+ * Copyright IBM Corporation, 2013
+ * Copyright LG Electronics Inc., 2014
+ * Written by:
+ * Marek Szyprowski <[email protected]>
+ * Michal Nazarewicz <[email protected]>
+ * Aneesh Kumar K.V <[email protected]>
+ * Joonsoo Kim <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License or (at your optional) any later version of the license.
+ */
+
+#define pr_fmt(fmt) "cma: " fmt
+
+#ifdef CONFIG_CMA_DEBUG
+#ifndef DEBUG
+# define DEBUG
+#endif
+#endif
+
+#include <linux/memblock.h>
+#include <linux/err.h>
+#include <linux/mm.h>
+#include <linux/mutex.h>
+#include <linux/sizes.h>
+#include <linux/slab.h>
+
+struct cma {
+ unsigned long base_pfn;
+ unsigned long count;
+ unsigned long *bitmap;
+ int order_per_bit; /* Order of pages represented by one bit */
+ struct mutex lock;
+};
+
+/*
+ * There is always at least global CMA area and a few optional
+ * areas configured in kernel .config.
+ */
+#define MAX_CMA_AREAS (1 + CONFIG_CMA_AREAS)
+
+static struct cma cma_areas[MAX_CMA_AREAS];
+static unsigned cma_area_count;
+static DEFINE_MUTEX(cma_mutex);
+
+static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
+{
+ return (1 << (align_order >> cma->order_per_bit)) - 1;
+}
+
+static unsigned long cma_bitmap_maxno(struct cma *cma)
+{
+ return cma->count >> cma->order_per_bit;
+}
+
+static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
+ unsigned long pages)
+{
+ return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
+}
+
+static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
+{
+ unsigned long bitmapno, nr_bits;
+
+ bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
+ nr_bits = cma_bitmap_pages_to_bits(cma, count);
+
+ mutex_lock(&cma->lock);
+ bitmap_clear(cma->bitmap, bitmapno, nr_bits);
+ mutex_unlock(&cma->lock);
+}
+
+static int __init cma_activate_area(struct cma *cma)
+{
+ int bitmap_maxno = cma_bitmap_maxno(cma);
+ int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
+ unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
+ unsigned i = cma->count >> pageblock_order;
+ struct zone *zone;
+
+ pr_debug("%s()\n", __func__);
+
+ cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
+ if (!cma->bitmap)
+ return -ENOMEM;
+
+ WARN_ON_ONCE(!pfn_valid(pfn));
+ zone = page_zone(pfn_to_page(pfn));
+
+ do {
+ unsigned j;
+
+ base_pfn = pfn;
+ for (j = pageblock_nr_pages; j; --j, pfn++) {
+ WARN_ON_ONCE(!pfn_valid(pfn));
+ /*
+ * alloc_contig_range requires the pfn range
+ * specified to be in the same zone. Make this
+ * simple by forcing the entire CMA resv range
+ * to be in the same zone.
+ */
+ if (page_zone(pfn_to_page(pfn)) != zone)
+ goto err;
+ }
+ init_cma_reserved_pageblock(pfn_to_page(base_pfn));
+ } while (--i);
+
+ mutex_init(&cma->lock);
+ return 0;
+
+err:
+ kfree(cma->bitmap);
+ return -EINVAL;
+}
+
+static int __init cma_init_reserved_areas(void)
+{
+ int i;
+
+ for (i = 0; i < cma_area_count; i++) {
+ int ret = cma_activate_area(&cma_areas[i]);
+
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+core_initcall(cma_init_reserved_areas);
+
+/**
+ * cma_declare_contiguous() - reserve custom contiguous area
+ * @size: Size of the reserved area (in bytes),
+ * @base: Base address of the reserved area optional, use 0 for any
+ * @limit: End address of the reserved memory (optional, 0 for any).
+ * @alignment: Alignment for the contiguous memory area, should be power of 2
+ * @order_per_bit: Order of pages represented by one bit on bitmap.
+ * @res_cma: Pointer to store the created cma region.
+ * @fixed: hint about where to place the reserved area
+ *
+ * This function reserves memory from early allocator. It should be
+ * called by arch specific code once the early allocator (memblock or bootmem)
+ * has been activated and all other subsystems have already allocated/reserved
+ * memory. This function allows to create custom reserved areas.
+ *
+ * If @fixed is true, reserve contiguous area at exactly @base. If false,
+ * reserve in range from @base to @limit.
+ */
+int __init cma_declare_contiguous(phys_addr_t size,
+ phys_addr_t base, phys_addr_t limit,
+ phys_addr_t alignment, int order_per_bit,
+ struct cma **res_cma, bool fixed)
+{
+ struct cma *cma = &cma_areas[cma_area_count];
+ int ret = 0;
+
+ pr_debug("%s(size %lx, base %08lx, limit %08lx alignment %08lx)\n",
+ __func__, (unsigned long)size, (unsigned long)base,
+ (unsigned long)limit, (unsigned long)alignment);
+
+ /* Sanity checks */
+ if (cma_area_count == ARRAY_SIZE(cma_areas)) {
+ pr_err("%s(): Not enough slots for CMA reserved regions!\n",
+ __func__);
+ return -ENOSPC;
+ }
+
+ if (!size)
+ return -EINVAL;
+
+ if (alignment && !is_power_of_2(alignment))
+ return -EINVAL;
+
+ /*
+ * Sanitise input arguments.
+ * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
+ * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
+ * and CMA property will be broken.
+ */
+ alignment = max(alignment,
+ (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
+ base = ALIGN(base, alignment);
+ size = ALIGN(size, alignment);
+ limit &= ~(alignment - 1);
+ /* size should be aligned with order_per_bit */
+ BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
+
+ /* Reserve memory */
+ if (base && fixed) {
+ if (memblock_is_region_reserved(base, size) ||
+ memblock_reserve(base, size) < 0) {
+ ret = -EBUSY;
+ goto err;
+ }
+ } else {
+ phys_addr_t addr = memblock_alloc_range(size, alignment, base,
+ limit);
+ if (!addr) {
+ ret = -ENOMEM;
+ goto err;
+ } else {
+ base = addr;
+ }
+ }
+
+ /*
+ * Each reserved area must be initialised later, when more kernel
+ * subsystems (like slab allocator) are available.
+ */
+ cma->base_pfn = PFN_DOWN(base);
+ cma->count = size >> PAGE_SHIFT;
+ cma->order_per_bit = order_per_bit;
+ *res_cma = cma;
+ cma_area_count++;
+
+ pr_info("%s(): reserved %ld MiB at %08lx\n",
+ __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
+ return 0;
+
+err:
+ pr_err("%s(): failed to reserve %ld MiB\n",
+ __func__, (unsigned long)size / SZ_1M);
+ return ret;
+}
+
+/**
+ * cma_alloc() - allocate pages from contiguous area
+ * @cma: Contiguous memory region for which the allocation is performed.
+ * @count: Requested number of pages.
+ * @align: Requested alignment of pages (in PAGE_SIZE order).
+ *
+ * This function allocates part of contiguous memory on specific
+ * contiguous memory area.
+ */
+struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
+{
+ unsigned long mask, pfn, start = 0;
+ unsigned long bitmap_maxno, bitmapno, nr_bits;
+ struct page *page = NULL;
+ int ret;
+
+ if (!cma || !cma->count)
+ return NULL;
+
+ pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
+ count, align);
+
+ if (!count)
+ return NULL;
+
+ mask = cma_bitmap_aligned_mask(cma, align);
+ bitmap_maxno = cma_bitmap_maxno(cma);
+ nr_bits = cma_bitmap_pages_to_bits(cma, count);
+
+ for (;;) {
+ mutex_lock(&cma->lock);
+ bitmapno = bitmap_find_next_zero_area(cma->bitmap,
+ bitmap_maxno, start, nr_bits, mask);
+ if (bitmapno >= bitmap_maxno) {
+ mutex_unlock(&cma->lock);
+ break;
+ }
+ bitmap_set(cma->bitmap, bitmapno, nr_bits);
+ /*
+ * It's safe to drop the lock here. We've marked this region for
+ * our exclusive use. If the migration fails we will take the
+ * lock again and unmark it.
+ */
+ mutex_unlock(&cma->lock);
+
+ pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
+ mutex_lock(&cma_mutex);
+ ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
+ mutex_unlock(&cma_mutex);
+ if (ret == 0) {
+ page = pfn_to_page(pfn);
+ break;
+ } else if (ret != -EBUSY) {
+ clear_cma_bitmap(cma, pfn, count);
+ break;
+ }
+ clear_cma_bitmap(cma, pfn, count);
+ pr_debug("%s(): memory range at %p is busy, retrying\n",
+ __func__, pfn_to_page(pfn));
+ /* try again with a bit different memory target */
+ start = bitmapno + mask + 1;
+ }
+
+ pr_debug("%s(): returned %p\n", __func__, page);
+ return page;
+}
+
+/**
+ * cma_release() - release allocated pages
+ * @cma: Contiguous memory region for which the allocation is performed.
+ * @pages: Allocated pages.
+ * @count: Number of allocated pages.
+ *
+ * This function releases memory allocated by alloc_cma().
+ * It returns false when provided pages do not belong to contiguous area and
+ * true otherwise.
+ */
+bool cma_release(struct cma *cma, struct page *pages, int count)
+{
+ unsigned long pfn;
+
+ if (!cma || !pages)
+ return false;
+
+ pr_debug("%s(page %p)\n", __func__, (void *)pages);
+
+ pfn = page_to_pfn(pages);
+
+ if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count)
+ return false;
+
+ VM_BUG_ON(pfn + count > cma->base_pfn + cma->count);
+
+ free_contig_range(pfn, count);
+ clear_cma_bitmap(cma, pfn, count);
+
+ return true;
+}
--
1.7.9.5
Conventionally, we put output param to the end of param list.
cma_declare_contiguous() doesn't look like that, so change it.
Additionally, move down cma_areas reference code to the position
where it is really needed.
Signed-off-by: Joonsoo Kim <[email protected]>
diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c
index 28ec226..97613ea 100644
--- a/arch/powerpc/kvm/book3s_hv_builtin.c
+++ b/arch/powerpc/kvm/book3s_hv_builtin.c
@@ -184,7 +184,7 @@ void __init kvm_cma_reserve(void)
align_size = max(kvm_rma_pages << PAGE_SHIFT, align_size);
cma_declare_contiguous(selected_size, 0, 0, align_size,
- KVM_CMA_CHUNK_ORDER - PAGE_SHIFT, &kvm_cma, false);
+ KVM_CMA_CHUNK_ORDER - PAGE_SHIFT, false, &kvm_cma);
}
}
diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
index f177f73..bfd4553 100644
--- a/drivers/base/dma-contiguous.c
+++ b/drivers/base/dma-contiguous.c
@@ -149,7 +149,7 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
{
int ret;
- ret = cma_declare_contiguous(size, base, limit, 0, 0, res_cma, fixed);
+ ret = cma_declare_contiguous(size, base, limit, 0, 0, fixed, res_cma);
if (ret)
return ret;
diff --git a/include/linux/cma.h b/include/linux/cma.h
index e38efe9..e53eead 100644
--- a/include/linux/cma.h
+++ b/include/linux/cma.h
@@ -6,7 +6,7 @@ struct cma;
extern int __init cma_declare_contiguous(phys_addr_t size,
phys_addr_t base, phys_addr_t limit,
phys_addr_t alignment, int order_per_bit,
- struct cma **res_cma, bool fixed);
+ bool fixed, struct cma **res_cma);
extern struct page *cma_alloc(struct cma *cma, int count, unsigned int align);
extern bool cma_release(struct cma *cma, struct page *pages, int count);
#endif
diff --git a/mm/cma.c b/mm/cma.c
index 01a0713..22a5b23 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -142,8 +142,8 @@ core_initcall(cma_init_reserved_areas);
* @limit: End address of the reserved memory (optional, 0 for any).
* @alignment: Alignment for the contiguous memory area, should be power of 2
* @order_per_bit: Order of pages represented by one bit on bitmap.
- * @res_cma: Pointer to store the created cma region.
* @fixed: hint about where to place the reserved area
+ * @res_cma: Pointer to store the created cma region.
*
* This function reserves memory from early allocator. It should be
* called by arch specific code once the early allocator (memblock or bootmem)
@@ -156,9 +156,9 @@ core_initcall(cma_init_reserved_areas);
int __init cma_declare_contiguous(phys_addr_t size,
phys_addr_t base, phys_addr_t limit,
phys_addr_t alignment, int order_per_bit,
- struct cma **res_cma, bool fixed)
+ bool fixed, struct cma **res_cma)
{
- struct cma *cma = &cma_areas[cma_area_count];
+ struct cma *cma;
int ret = 0;
pr_debug("%s(size %lx, base %08lx, limit %08lx alignment %08lx)\n",
@@ -214,6 +214,7 @@ int __init cma_declare_contiguous(phys_addr_t size,
* Each reserved area must be initialised later, when more kernel
* subsystems (like slab allocator) are available.
*/
+ cma = &cma_areas[cma_area_count];
cma->base_pfn = PFN_DOWN(base);
cma->count = size >> PAGE_SHIFT;
cma->order_per_bit = order_per_bit;
--
1.7.9.5
Currently, we should take the mutex for manipulating bitmap.
This job may be really simple and short so we don't need to sleep
if contended. So I change it to spinlock.
Signed-off-by: Joonsoo Kim <[email protected]>
diff --git a/mm/cma.c b/mm/cma.c
index 22a5b23..3085e8c 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -27,6 +27,7 @@
#include <linux/memblock.h>
#include <linux/err.h>
#include <linux/mm.h>
+#include <linux/spinlock.h>
#include <linux/mutex.h>
#include <linux/sizes.h>
#include <linux/slab.h>
@@ -36,7 +37,7 @@ struct cma {
unsigned long count;
unsigned long *bitmap;
int order_per_bit; /* Order of pages represented by one bit */
- struct mutex lock;
+ spinlock_t lock;
};
/*
@@ -72,9 +73,9 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
nr_bits = cma_bitmap_pages_to_bits(cma, count);
- mutex_lock(&cma->lock);
+ spin_lock(&cma->lock);
bitmap_clear(cma->bitmap, bitmapno, nr_bits);
- mutex_unlock(&cma->lock);
+ spin_unlock(&cma->lock);
}
static int __init cma_activate_area(struct cma *cma)
@@ -112,7 +113,7 @@ static int __init cma_activate_area(struct cma *cma)
init_cma_reserved_pageblock(pfn_to_page(base_pfn));
} while (--i);
- mutex_init(&cma->lock);
+ spin_lock_init(&cma->lock);
return 0;
err:
@@ -261,11 +262,11 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
nr_bits = cma_bitmap_pages_to_bits(cma, count);
for (;;) {
- mutex_lock(&cma->lock);
+ spin_lock(&cma->lock);
bitmapno = bitmap_find_next_zero_area(cma->bitmap,
bitmap_maxno, start, nr_bits, mask);
if (bitmapno >= bitmap_maxno) {
- mutex_unlock(&cma->lock);
+ spin_unlock(&cma->lock);
break;
}
bitmap_set(cma->bitmap, bitmapno, nr_bits);
@@ -274,7 +275,7 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
* our exclusive use. If the migration fails we will take the
* lock again and unmark it.
*/
- mutex_unlock(&cma->lock);
+ spin_unlock(&cma->lock);
pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
mutex_lock(&cma_mutex);
--
1.7.9.5
We can remove one call sites for clear_cma_bitmap() if we first
call it before checking error number.
Signed-off-by: Joonsoo Kim <[email protected]>
diff --git a/mm/cma.c b/mm/cma.c
index 1e1b017..01a0713 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -282,11 +282,12 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
if (ret == 0) {
page = pfn_to_page(pfn);
break;
- } else if (ret != -EBUSY) {
- clear_cma_bitmap(cma, pfn, count);
- break;
}
+
clear_cma_bitmap(cma, pfn, count);
+ if (ret != -EBUSY)
+ break;
+
pr_debug("%s(): memory range at %p is busy, retrying\n",
__func__, pfn_to_page(pfn));
/* try again with a bit different memory target */
--
1.7.9.5
Joonsoo Kim <[email protected]> writes:
> We don't need explicit 'CMA:' prefix, since we already define prefix
> 'cma:' in pr_fmt. So remove it.
>
> And, some logs print function name and others doesn't. This looks
> bad to me, so I unify log format to print function name consistently.
>
> Lastly, I add one more debug log on cma_activate_area().
>
> Signed-off-by: Joonsoo Kim <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 83969f8..bd0bb81 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -144,7 +144,7 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
> }
>
> if (selected_size && !dma_contiguous_default_area) {
> - pr_debug("%s: reserving %ld MiB for global area\n", __func__,
> + pr_debug("%s(): reserving %ld MiB for global area\n", __func__,
> (unsigned long)selected_size / SZ_1M);
Do we need to do function(), or just function:. I have seen the later
usage in other parts of the kernel.
>
> dma_contiguous_reserve_area(selected_size, selected_base,
> @@ -163,8 +163,9 @@ static int __init cma_activate_area(struct cma *cma)
> unsigned i = cma->count >> pageblock_order;
> struct zone *zone;
>
> - cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> + pr_debug("%s()\n", __func__);
why ?
>
> + cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> if (!cma->bitmap)
> return -ENOMEM;
>
> @@ -234,7 +235,8 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
>
> /* Sanity checks */
> if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> - pr_err("Not enough slots for CMA reserved regions!\n");
> + pr_err("%s(): Not enough slots for CMA reserved regions!\n",
> + __func__);
> return -ENOSPC;
> }
>
> @@ -274,14 +276,15 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> *res_cma = cma;
> cma_area_count++;
>
> - pr_info("CMA: reserved %ld MiB at %08lx\n", (unsigned long)size / SZ_1M,
> - (unsigned long)base);
> + pr_info("%s(): reserved %ld MiB at %08lx\n",
> + __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
>
> /* Architecture specific contiguous memory fixup. */
> dma_contiguous_early_fixup(base, size);
> return 0;
> err:
> - pr_err("CMA: failed to reserve %ld MiB\n", (unsigned long)size / SZ_1M);
> + pr_err("%s(): failed to reserve %ld MiB\n",
> + __func__, (unsigned long)size / SZ_1M);
> return ret;
> }
>
> --
> 1.7.9.5
Joonsoo Kim <[email protected]> writes:
> We should free memory for bitmap when we find zone mis-match,
> otherwise this memory will leak.
>
> Additionally, I copy code comment from ppc kvm's cma code to notify
> why we need to check zone mis-match.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Reviewed-by: Aneesh Kumar K.V <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index bd0bb81..fb0cdce 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -177,14 +177,24 @@ static int __init cma_activate_area(struct cma *cma)
> base_pfn = pfn;
> for (j = pageblock_nr_pages; j; --j, pfn++) {
> WARN_ON_ONCE(!pfn_valid(pfn));
> + /*
> + * alloc_contig_range requires the pfn range
> + * specified to be in the same zone. Make this
> + * simple by forcing the entire CMA resv range
> + * to be in the same zone.
> + */
> if (page_zone(pfn_to_page(pfn)) != zone)
> - return -EINVAL;
> + goto err;
> }
> init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> } while (--i);
>
> mutex_init(&cma->lock);
> return 0;
> +
> +err:
> + kfree(cma->bitmap);
> + return -EINVAL;
> }
>
> static struct cma cma_areas[MAX_CMA_AREAS];
> --
> 1.7.9.5
Joonsoo Kim <[email protected]> writes:
> To prepare future generalization work on cma area management code,
> we need to separate core cma management codes from DMA APIs.
> We will extend these core functions to cover requirements of
> ppc kvm's cma area management functionality in following patches.
> This separation helps us not to touch DMA APIs while extending
> core functions.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Reviewed-by: Aneesh Kumar K.V <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index fb0cdce..8a44c82 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -231,9 +231,9 @@ core_initcall(cma_init_reserved_areas);
> * If @fixed is true, reserve contiguous area at exactly @base. If false,
> * reserve in range from @base to @limit.
> */
> -int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> - phys_addr_t limit, struct cma **res_cma,
> - bool fixed)
> +static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> + phys_addr_t base, phys_addr_t limit,
> + struct cma **res_cma, bool fixed)
> {
> struct cma *cma = &cma_areas[cma_area_count];
> phys_addr_t alignment;
> @@ -288,16 +288,30 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
>
> pr_info("%s(): reserved %ld MiB at %08lx\n",
> __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
> -
> - /* Architecture specific contiguous memory fixup. */
> - dma_contiguous_early_fixup(base, size);
> return 0;
> +
> err:
> pr_err("%s(): failed to reserve %ld MiB\n",
> __func__, (unsigned long)size / SZ_1M);
> return ret;
> }
>
> +int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> + phys_addr_t limit, struct cma **res_cma,
> + bool fixed)
> +{
> + int ret;
> +
> + ret = __dma_contiguous_reserve_area(size, base, limit, res_cma, fixed);
> + if (ret)
> + return ret;
> +
> + /* Architecture specific contiguous memory fixup. */
> + dma_contiguous_early_fixup(base, size);
> +
> + return 0;
> +}
> +
> static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> {
> mutex_lock(&cma->lock);
> @@ -316,20 +330,16 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> * global one. Requires architecture specific dev_get_cma_area() helper
> * function.
> */
> -struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> +static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> unsigned int align)
> {
> unsigned long mask, pfn, pageno, start = 0;
> - struct cma *cma = dev_get_cma_area(dev);
> struct page *page = NULL;
> int ret;
>
> if (!cma || !cma->count)
> return NULL;
>
> - if (align > CONFIG_CMA_ALIGNMENT)
> - align = CONFIG_CMA_ALIGNMENT;
> -
> pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
> count, align);
>
> @@ -377,6 +387,17 @@ struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> return page;
> }
>
> +struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> + unsigned int align)
> +{
> + struct cma *cma = dev_get_cma_area(dev);
> +
> + if (align > CONFIG_CMA_ALIGNMENT)
> + align = CONFIG_CMA_ALIGNMENT;
> +
> + return __dma_alloc_from_contiguous(cma, count, align);
> +}
> +
> /**
> * dma_release_from_contiguous() - release allocated pages
> * @dev: Pointer to device for which the pages were allocated.
> @@ -387,10 +408,9 @@ struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> * It returns false when provided pages do not belong to contiguous area and
> * true otherwise.
> */
> -bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> +static bool __dma_release_from_contiguous(struct cma *cma, struct page *pages,
> int count)
> {
> - struct cma *cma = dev_get_cma_area(dev);
> unsigned long pfn;
>
> if (!cma || !pages)
> @@ -410,3 +430,11 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages,
>
> return true;
> }
> +
> +bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> + int count)
> +{
> + struct cma *cma = dev_get_cma_area(dev);
> +
> + return __dma_release_from_contiguous(cma, pages, count);
> +}
> --
> 1.7.9.5
Joonsoo Kim <[email protected]> writes:
> ppc kvm's cma area management needs alignment constraint on
> cma region. So support it to prepare generalization of cma area
> management functionality.
>
> Additionally, add some comments which tell us why alignment
> constraint is needed on cma region.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Reviewed-by: Aneesh Kumar K.V <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 8a44c82..bc4c171 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -32,6 +32,7 @@
> #include <linux/swap.h>
> #include <linux/mm_types.h>
> #include <linux/dma-contiguous.h>
> +#include <linux/log2.h>
>
> struct cma {
> unsigned long base_pfn;
> @@ -219,6 +220,7 @@ core_initcall(cma_init_reserved_areas);
> * @size: Size of the reserved area (in bytes),
> * @base: Base address of the reserved area optional, use 0 for any
> * @limit: End address of the reserved memory (optional, 0 for any).
> + * @alignment: Alignment for the contiguous memory area, should be power of 2
> * @res_cma: Pointer to store the created cma region.
> * @fixed: hint about where to place the reserved area
> *
> @@ -233,15 +235,15 @@ core_initcall(cma_init_reserved_areas);
> */
> static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> + phys_addr_t alignment,
> struct cma **res_cma, bool fixed)
> {
> struct cma *cma = &cma_areas[cma_area_count];
> - phys_addr_t alignment;
> int ret = 0;
>
> - pr_debug("%s(size %lx, base %08lx, limit %08lx)\n", __func__,
> - (unsigned long)size, (unsigned long)base,
> - (unsigned long)limit);
> + pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
> + __func__, (unsigned long)size, (unsigned long)base,
> + (unsigned long)limit, (unsigned long)alignment);
>
> /* Sanity checks */
> if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> @@ -253,8 +255,17 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> if (!size)
> return -EINVAL;
>
> - /* Sanitise input arguments */
> - alignment = PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order);
> + if (alignment && !is_power_of_2(alignment))
> + return -EINVAL;
> +
> + /*
> + * Sanitise input arguments.
> + * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
> + * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
> + * and CMA property will be broken.
> + */
> + alignment = max(alignment,
> + (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
> base = ALIGN(base, alignment);
> size = ALIGN(size, alignment);
> limit &= ~(alignment - 1);
> @@ -302,7 +313,8 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> {
> int ret;
>
> - ret = __dma_contiguous_reserve_area(size, base, limit, res_cma, fixed);
> + ret = __dma_contiguous_reserve_area(size, base, limit, 0,
> + res_cma, fixed);
> if (ret)
> return ret;
>
> --
> 1.7.9.5
Hi Joonsoo,
On Thu, Jun 12, 2014 at 12:21:38PM +0900, Joonsoo Kim wrote:
> We don't need explicit 'CMA:' prefix, since we already define prefix
> 'cma:' in pr_fmt. So remove it.
>
> And, some logs print function name and others doesn't. This looks
> bad to me, so I unify log format to print function name consistently.
>
> Lastly, I add one more debug log on cma_activate_area().
When I take a look, it just indicates cma_activate_area was called or not,
without what range for the area was reserved successfully so I couldn't see
the intention for new message. Description should explain it so that everybody
can agree on your claim.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 83969f8..bd0bb81 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -144,7 +144,7 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
> }
>
> if (selected_size && !dma_contiguous_default_area) {
> - pr_debug("%s: reserving %ld MiB for global area\n", __func__,
> + pr_debug("%s(): reserving %ld MiB for global area\n", __func__,
> (unsigned long)selected_size / SZ_1M);
>
> dma_contiguous_reserve_area(selected_size, selected_base,
> @@ -163,8 +163,9 @@ static int __init cma_activate_area(struct cma *cma)
> unsigned i = cma->count >> pageblock_order;
> struct zone *zone;
>
> - cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> + pr_debug("%s()\n", __func__);
>
> + cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> if (!cma->bitmap)
> return -ENOMEM;
>
> @@ -234,7 +235,8 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
>
> /* Sanity checks */
> if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> - pr_err("Not enough slots for CMA reserved regions!\n");
> + pr_err("%s(): Not enough slots for CMA reserved regions!\n",
> + __func__);
> return -ENOSPC;
> }
>
> @@ -274,14 +276,15 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> *res_cma = cma;
> cma_area_count++;
>
> - pr_info("CMA: reserved %ld MiB at %08lx\n", (unsigned long)size / SZ_1M,
> - (unsigned long)base);
> + pr_info("%s(): reserved %ld MiB at %08lx\n",
> + __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
>
> /* Architecture specific contiguous memory fixup. */
> dma_contiguous_early_fixup(base, size);
> return 0;
> err:
> - pr_err("CMA: failed to reserve %ld MiB\n", (unsigned long)size / SZ_1M);
> + pr_err("%s(): failed to reserve %ld MiB\n",
> + __func__, (unsigned long)size / SZ_1M);
> return ret;
> }
>
> --
> 1.7.9.5
--
Kind regards,
Minchan Kim
On Thu, Jun 12, 2014 at 12:21:39PM +0900, Joonsoo Kim wrote:
> We should free memory for bitmap when we find zone mis-match,
> otherwise this memory will leak.
Then, -stable stuff?
>
> Additionally, I copy code comment from ppc kvm's cma code to notify
> why we need to check zone mis-match.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index bd0bb81..fb0cdce 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -177,14 +177,24 @@ static int __init cma_activate_area(struct cma *cma)
> base_pfn = pfn;
> for (j = pageblock_nr_pages; j; --j, pfn++) {
> WARN_ON_ONCE(!pfn_valid(pfn));
> + /*
> + * alloc_contig_range requires the pfn range
> + * specified to be in the same zone. Make this
> + * simple by forcing the entire CMA resv range
> + * to be in the same zone.
> + */
> if (page_zone(pfn_to_page(pfn)) != zone)
> - return -EINVAL;
> + goto err;
At a first glance, I thought it would be better to handle such error
before activating.
So when I see the registration code(ie, dma_contiguous_revere_area),
I realized it is impossible because we didn't set up zone yet. :(
If so, when we detect to fail here, it would be better to report more
meaningful error message like what was successful zone and what is
new zone and failed pfn number?
> }
> init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> } while (--i);
>
> mutex_init(&cma->lock);
> return 0;
> +
> +err:
> + kfree(cma->bitmap);
> + return -EINVAL;
> }
>
> static struct cma cma_areas[MAX_CMA_AREAS];
> --
> 1.7.9.5
--
Kind regards,
Minchan Kim
On Thu, Jun 12, 2014 at 12:21:40PM +0900, Joonsoo Kim wrote:
> To prepare future generalization work on cma area management code,
> we need to separate core cma management codes from DMA APIs.
> We will extend these core functions to cover requirements of
> ppc kvm's cma area management functionality in following patches.
> This separation helps us not to touch DMA APIs while extending
> core functions.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index fb0cdce..8a44c82 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -231,9 +231,9 @@ core_initcall(cma_init_reserved_areas);
> * If @fixed is true, reserve contiguous area at exactly @base. If false,
> * reserve in range from @base to @limit.
> */
> -int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> - phys_addr_t limit, struct cma **res_cma,
> - bool fixed)
> +static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> + phys_addr_t base, phys_addr_t limit,
> + struct cma **res_cma, bool fixed)
> {
> struct cma *cma = &cma_areas[cma_area_count];
> phys_addr_t alignment;
> @@ -288,16 +288,30 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
>
> pr_info("%s(): reserved %ld MiB at %08lx\n",
> __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
> -
> - /* Architecture specific contiguous memory fixup. */
> - dma_contiguous_early_fixup(base, size);
> return 0;
> +
> err:
> pr_err("%s(): failed to reserve %ld MiB\n",
> __func__, (unsigned long)size / SZ_1M);
> return ret;
> }
>
> +int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> + phys_addr_t limit, struct cma **res_cma,
> + bool fixed)
> +{
> + int ret;
> +
> + ret = __dma_contiguous_reserve_area(size, base, limit, res_cma, fixed);
> + if (ret)
> + return ret;
> +
> + /* Architecture specific contiguous memory fixup. */
> + dma_contiguous_early_fixup(base, size);
In old, base and size are aligned with alignment and passed into arch fixup
but your patch is changing it.
I didn't look at what kinds of side effect it makes but just want to confirm.
> +
> + return 0;
> +}
> +
> static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> {
> mutex_lock(&cma->lock);
> @@ -316,20 +330,16 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> * global one. Requires architecture specific dev_get_cma_area() helper
> * function.
> */
> -struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> +static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> unsigned int align)
> {
> unsigned long mask, pfn, pageno, start = 0;
> - struct cma *cma = dev_get_cma_area(dev);
> struct page *page = NULL;
> int ret;
>
> if (!cma || !cma->count)
> return NULL;
>
> - if (align > CONFIG_CMA_ALIGNMENT)
> - align = CONFIG_CMA_ALIGNMENT;
> -
> pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
> count, align);
>
> @@ -377,6 +387,17 @@ struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> return page;
> }
>
Please move the description in __dma_alloc_from_contiguous to here exported API.
> +struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> + unsigned int align)
> +{
> + struct cma *cma = dev_get_cma_area(dev);
> +
> + if (align > CONFIG_CMA_ALIGNMENT)
> + align = CONFIG_CMA_ALIGNMENT;
> +
> + return __dma_alloc_from_contiguous(cma, count, align);
> +}
> +
> /**
> * dma_release_from_contiguous() - release allocated pages
> * @dev: Pointer to device for which the pages were allocated.
> @@ -387,10 +408,9 @@ struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> * It returns false when provided pages do not belong to contiguous area and
> * true otherwise.
> */
> -bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> +static bool __dma_release_from_contiguous(struct cma *cma, struct page *pages,
> int count)
> {
> - struct cma *cma = dev_get_cma_area(dev);
> unsigned long pfn;
>
> if (!cma || !pages)
> @@ -410,3 +430,11 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages,
>
> return true;
> }
> +
Ditto.
> +bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> + int count)
> +{
> + struct cma *cma = dev_get_cma_area(dev);
> +
> + return __dma_release_from_contiguous(cma, pages, count);
> +}
> --
> 1.7.9.5
--
Kind regards,
Minchan Kim
On Thu, Jun 12, 2014 at 10:11:19AM +0530, Aneesh Kumar K.V wrote:
> Joonsoo Kim <[email protected]> writes:
>
> > We don't need explicit 'CMA:' prefix, since we already define prefix
> > 'cma:' in pr_fmt. So remove it.
> >
> > And, some logs print function name and others doesn't. This looks
> > bad to me, so I unify log format to print function name consistently.
> >
> > Lastly, I add one more debug log on cma_activate_area().
> >
> > Signed-off-by: Joonsoo Kim <[email protected]>
> >
> > diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> > index 83969f8..bd0bb81 100644
> > --- a/drivers/base/dma-contiguous.c
> > +++ b/drivers/base/dma-contiguous.c
> > @@ -144,7 +144,7 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
> > }
> >
> > if (selected_size && !dma_contiguous_default_area) {
> > - pr_debug("%s: reserving %ld MiB for global area\n", __func__,
> > + pr_debug("%s(): reserving %ld MiB for global area\n", __func__,
> > (unsigned long)selected_size / SZ_1M);
>
> Do we need to do function(), or just function:. I have seen the later
> usage in other parts of the kernel.
Hello,
I also haven't seen this format in other kernel code, but, in cma, they use
this format as following.
function(arg1, arg2, ...): some message
If we all dislike this format, we can change it after merging this
patchset. Until then, it seems better to me to leave it as is.
>
> >
> > dma_contiguous_reserve_area(selected_size, selected_base,
> > @@ -163,8 +163,9 @@ static int __init cma_activate_area(struct cma *cma)
> > unsigned i = cma->count >> pageblock_order;
> > struct zone *zone;
> >
> > - cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> > + pr_debug("%s()\n", __func__);
>
> why ?
>
This pr_debug() comes from ppc kvm's kvm_cma_init_reserved_areas().
I want to maintain all log messages as much as possible to reduce confusion
with this generalization.
Thanks.
On Thu, Jun 12, 2014 at 02:18:53PM +0900, Minchan Kim wrote:
> Hi Joonsoo,
>
> On Thu, Jun 12, 2014 at 12:21:38PM +0900, Joonsoo Kim wrote:
> > We don't need explicit 'CMA:' prefix, since we already define prefix
> > 'cma:' in pr_fmt. So remove it.
> >
> > And, some logs print function name and others doesn't. This looks
> > bad to me, so I unify log format to print function name consistently.
> >
> > Lastly, I add one more debug log on cma_activate_area().
>
> When I take a look, it just indicates cma_activate_area was called or not,
> without what range for the area was reserved successfully so I couldn't see
> the intention for new message. Description should explain it so that everybody
> can agree on your claim.
>
Hello,
I paste the answer in other thread.
This pr_debug() comes from ppc kvm's kvm_cma_init_reserved_areas().
I want to maintain all log messages as much as possible to reduce
confusion with this generalization.
If I need to respin this patchset, I will explain more about it.
Thanks.
On Thu, Jun 12, 2014 at 12:21:41PM +0900, Joonsoo Kim wrote:
> ppc kvm's cma area management needs alignment constraint on
> cma region. So support it to prepare generalization of cma area
> management functionality.
>
> Additionally, add some comments which tell us why alignment
> constraint is needed on cma region.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 8a44c82..bc4c171 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -32,6 +32,7 @@
> #include <linux/swap.h>
> #include <linux/mm_types.h>
> #include <linux/dma-contiguous.h>
> +#include <linux/log2.h>
>
> struct cma {
> unsigned long base_pfn;
> @@ -219,6 +220,7 @@ core_initcall(cma_init_reserved_areas);
> * @size: Size of the reserved area (in bytes),
> * @base: Base address of the reserved area optional, use 0 for any
> * @limit: End address of the reserved memory (optional, 0 for any).
> + * @alignment: Alignment for the contiguous memory area, should be power of 2
> * @res_cma: Pointer to store the created cma region.
> * @fixed: hint about where to place the reserved area
> *
Pz, move the all description to new API function rather than internal one.
> @@ -233,15 +235,15 @@ core_initcall(cma_init_reserved_areas);
> */
> static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> + phys_addr_t alignment,
> struct cma **res_cma, bool fixed)
> {
> struct cma *cma = &cma_areas[cma_area_count];
> - phys_addr_t alignment;
> int ret = 0;
>
> - pr_debug("%s(size %lx, base %08lx, limit %08lx)\n", __func__,
> - (unsigned long)size, (unsigned long)base,
> - (unsigned long)limit);
> + pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
Why is it called by "align_order"?
> + __func__, (unsigned long)size, (unsigned long)base,
> + (unsigned long)limit, (unsigned long)alignment);
>
> /* Sanity checks */
> if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> @@ -253,8 +255,17 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> if (!size)
> return -EINVAL;
>
> - /* Sanitise input arguments */
> - alignment = PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order);
> + if (alignment && !is_power_of_2(alignment))
> + return -EINVAL;
> +
> + /*
> + * Sanitise input arguments.
> + * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
> + * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
I'm not a native but try for clear documenation.
Pages both ends in CMA area could be merged into adjacent unmovable
migratetype page by page allocator's buddy algorithm. In the case,
you couldn't get a contiguous memory, which is not what we want.
> + * and CMA property will be broken.
> + */
> + alignment = max(alignment,
> + (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
> base = ALIGN(base, alignment);
> size = ALIGN(size, alignment);
> limit &= ~(alignment - 1);
> @@ -302,7 +313,8 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> {
> int ret;
>
> - ret = __dma_contiguous_reserve_area(size, base, limit, res_cma, fixed);
> + ret = __dma_contiguous_reserve_area(size, base, limit, 0,
> + res_cma, fixed);
> if (ret)
> return ret;
>
> --
> 1.7.9.5
--
Kind regards,
Minchan Kim
On Thu, Jun 12, 2014 at 02:25:43PM +0900, Minchan Kim wrote:
> On Thu, Jun 12, 2014 at 12:21:39PM +0900, Joonsoo Kim wrote:
> > We should free memory for bitmap when we find zone mis-match,
> > otherwise this memory will leak.
>
> Then, -stable stuff?
I don't think so. This is just possible leak candidate, so we don't
need to push this to stable tree.
>
> >
> > Additionally, I copy code comment from ppc kvm's cma code to notify
> > why we need to check zone mis-match.
> >
> > Signed-off-by: Joonsoo Kim <[email protected]>
> >
> > diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> > index bd0bb81..fb0cdce 100644
> > --- a/drivers/base/dma-contiguous.c
> > +++ b/drivers/base/dma-contiguous.c
> > @@ -177,14 +177,24 @@ static int __init cma_activate_area(struct cma *cma)
> > base_pfn = pfn;
> > for (j = pageblock_nr_pages; j; --j, pfn++) {
> > WARN_ON_ONCE(!pfn_valid(pfn));
> > + /*
> > + * alloc_contig_range requires the pfn range
> > + * specified to be in the same zone. Make this
> > + * simple by forcing the entire CMA resv range
> > + * to be in the same zone.
> > + */
> > if (page_zone(pfn_to_page(pfn)) != zone)
> > - return -EINVAL;
> > + goto err;
>
> At a first glance, I thought it would be better to handle such error
> before activating.
> So when I see the registration code(ie, dma_contiguous_revere_area),
> I realized it is impossible because we didn't set up zone yet. :(
>
> If so, when we detect to fail here, it would be better to report more
> meaningful error message like what was successful zone and what is
> new zone and failed pfn number?
What I want to do in early phase of this patchset is to make cma code
on DMA APIs similar to ppc kvm's cma code. ppc kvm's cma code already
has this error handling logic, so I make this patch.
If we think that we need more things, we can do that on general cma code
after merging this patchset.
Thanks.
On Thu, Jun 12, 2014 at 02:52:20PM +0900, Minchan Kim wrote:
> On Thu, Jun 12, 2014 at 12:21:41PM +0900, Joonsoo Kim wrote:
> > ppc kvm's cma area management needs alignment constraint on
> > cma region. So support it to prepare generalization of cma area
> > management functionality.
> >
> > Additionally, add some comments which tell us why alignment
> > constraint is needed on cma region.
> >
> > Signed-off-by: Joonsoo Kim <[email protected]>
> >
> > diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> > index 8a44c82..bc4c171 100644
> > --- a/drivers/base/dma-contiguous.c
> > +++ b/drivers/base/dma-contiguous.c
> > @@ -32,6 +32,7 @@
> > #include <linux/swap.h>
> > #include <linux/mm_types.h>
> > #include <linux/dma-contiguous.h>
> > +#include <linux/log2.h>
> >
> > struct cma {
> > unsigned long base_pfn;
> > @@ -219,6 +220,7 @@ core_initcall(cma_init_reserved_areas);
> > * @size: Size of the reserved area (in bytes),
> > * @base: Base address of the reserved area optional, use 0 for any
> > * @limit: End address of the reserved memory (optional, 0 for any).
> > + * @alignment: Alignment for the contiguous memory area, should be power of 2
> > * @res_cma: Pointer to store the created cma region.
> > * @fixed: hint about where to place the reserved area
> > *
>
> Pz, move the all description to new API function rather than internal one.
Reason I leave all description as is is that I will remove it in
following patch. I think that moving these makes patch bigger and hard
to review.
But, if it is necessary, I will do it. :)
>
> > @@ -233,15 +235,15 @@ core_initcall(cma_init_reserved_areas);
> > */
> > static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> > phys_addr_t base, phys_addr_t limit,
> > + phys_addr_t alignment,
> > struct cma **res_cma, bool fixed)
> > {
> > struct cma *cma = &cma_areas[cma_area_count];
> > - phys_addr_t alignment;
> > int ret = 0;
> >
> > - pr_debug("%s(size %lx, base %08lx, limit %08lx)\n", __func__,
> > - (unsigned long)size, (unsigned long)base,
> > - (unsigned long)limit);
> > + pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
>
> Why is it called by "align_order"?
Oops... mistake.
I will fix it.
Thanks.
On Thu, Jun 12, 2014 at 12:21:42PM +0900, Joonsoo Kim wrote:
> ppc kvm's cma region management requires arbitrary bitmap granularity,
> since they want to reserve very large memory and manage this region
> with bitmap that one bit for several pages to reduce management overheads.
> So support arbitrary bitmap granularity for following generalization.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index bc4c171..9bc9340 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -38,6 +38,7 @@ struct cma {
> unsigned long base_pfn;
> unsigned long count;
> unsigned long *bitmap;
> + int order_per_bit; /* Order of pages represented by one bit */
Hmm, I'm not sure it's good as *general* interface even though it covers
existing usecases.
It forces a cma area should be handled by same size unit. Right?
It's really important point for this patchset's motivation so I will stop
review and wait other opinions.
> struct mutex lock;
> };
>
> @@ -157,9 +158,38 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
>
> static DEFINE_MUTEX(cma_mutex);
>
> +static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
> +{
> + return (1 << (align_order >> cma->order_per_bit)) - 1;
> +}
> +
> +static unsigned long cma_bitmap_maxno(struct cma *cma)
> +{
> + return cma->count >> cma->order_per_bit;
> +}
> +
> +static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
> + unsigned long pages)
> +{
> + return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
> +}
> +
> +static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> +{
> + unsigned long bitmapno, nr_bits;
> +
> + bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + mutex_lock(&cma->lock);
> + bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> + mutex_unlock(&cma->lock);
> +}
> +
> static int __init cma_activate_area(struct cma *cma)
> {
> - int bitmap_size = BITS_TO_LONGS(cma->count) * sizeof(long);
> + int bitmap_maxno = cma_bitmap_maxno(cma);
> + int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> unsigned i = cma->count >> pageblock_order;
> struct zone *zone;
> @@ -221,6 +251,7 @@ core_initcall(cma_init_reserved_areas);
> * @base: Base address of the reserved area optional, use 0 for any
> * @limit: End address of the reserved memory (optional, 0 for any).
> * @alignment: Alignment for the contiguous memory area, should be power of 2
> + * @order_per_bit: Order of pages represented by one bit on bitmap.
> * @res_cma: Pointer to store the created cma region.
> * @fixed: hint about where to place the reserved area
> *
> @@ -235,7 +266,7 @@ core_initcall(cma_init_reserved_areas);
> */
> static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> - phys_addr_t alignment,
> + phys_addr_t alignment, int order_per_bit,
> struct cma **res_cma, bool fixed)
> {
> struct cma *cma = &cma_areas[cma_area_count];
> @@ -269,6 +300,8 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> base = ALIGN(base, alignment);
> size = ALIGN(size, alignment);
> limit &= ~(alignment - 1);
> + /* size should be aligned with order_per_bit */
> + BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
>
> /* Reserve memory */
> if (base && fixed) {
> @@ -294,6 +327,7 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> */
> cma->base_pfn = PFN_DOWN(base);
> cma->count = size >> PAGE_SHIFT;
> + cma->order_per_bit = order_per_bit;
> *res_cma = cma;
> cma_area_count++;
>
> @@ -313,7 +347,7 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> {
> int ret;
>
> - ret = __dma_contiguous_reserve_area(size, base, limit, 0,
> + ret = __dma_contiguous_reserve_area(size, base, limit, 0, 0,
> res_cma, fixed);
> if (ret)
> return ret;
> @@ -324,13 +358,6 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> return 0;
> }
>
> -static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> -{
> - mutex_lock(&cma->lock);
> - bitmap_clear(cma->bitmap, pfn - cma->base_pfn, count);
> - mutex_unlock(&cma->lock);
> -}
> -
> /**
> * dma_alloc_from_contiguous() - allocate pages from contiguous area
> * @dev: Pointer to device for which the allocation is performed.
> @@ -345,7 +372,8 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> unsigned int align)
> {
> - unsigned long mask, pfn, pageno, start = 0;
> + unsigned long mask, pfn, start = 0;
> + unsigned long bitmap_maxno, bitmapno, nr_bits;
> struct page *page = NULL;
> int ret;
>
> @@ -358,18 +386,19 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> if (!count)
> return NULL;
>
> - mask = (1 << align) - 1;
> -
> + mask = cma_bitmap_aligned_mask(cma, align);
> + bitmap_maxno = cma_bitmap_maxno(cma);
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
>
> for (;;) {
> mutex_lock(&cma->lock);
> - pageno = bitmap_find_next_zero_area(cma->bitmap, cma->count,
> - start, count, mask);
> - if (pageno >= cma->count) {
> + bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> + bitmap_maxno, start, nr_bits, mask);
> + if (bitmapno >= bitmap_maxno) {
> mutex_unlock(&cma->lock);
> break;
> }
> - bitmap_set(cma->bitmap, pageno, count);
> + bitmap_set(cma->bitmap, bitmapno, nr_bits);
> /*
> * It's safe to drop the lock here. We've marked this region for
> * our exclusive use. If the migration fails we will take the
> @@ -377,7 +406,7 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> */
> mutex_unlock(&cma->lock);
>
> - pfn = cma->base_pfn + pageno;
> + pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> mutex_lock(&cma_mutex);
> ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> mutex_unlock(&cma_mutex);
> @@ -392,7 +421,7 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> pr_debug("%s(): memory range at %p is busy, retrying\n",
> __func__, pfn_to_page(pfn));
> /* try again with a bit different memory target */
> - start = pageno + mask + 1;
> + start = bitmapno + mask + 1;
> }
>
> pr_debug("%s(): returned %p\n", __func__, page);
> --
> 1.7.9.5
--
Kind regards,
Minchan Kim
On Thu, Jun 12, 2014 at 03:06:10PM +0900, Minchan Kim wrote:
> On Thu, Jun 12, 2014 at 12:21:42PM +0900, Joonsoo Kim wrote:
> > ppc kvm's cma region management requires arbitrary bitmap granularity,
> > since they want to reserve very large memory and manage this region
> > with bitmap that one bit for several pages to reduce management overheads.
> > So support arbitrary bitmap granularity for following generalization.
> >
> > Signed-off-by: Joonsoo Kim <[email protected]>
> >
> > diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> > index bc4c171..9bc9340 100644
> > --- a/drivers/base/dma-contiguous.c
> > +++ b/drivers/base/dma-contiguous.c
> > @@ -38,6 +38,7 @@ struct cma {
> > unsigned long base_pfn;
> > unsigned long count;
> > unsigned long *bitmap;
> > + int order_per_bit; /* Order of pages represented by one bit */
>
> Hmm, I'm not sure it's good as *general* interface even though it covers
> existing usecases.
>
> It forces a cma area should be handled by same size unit. Right?
> It's really important point for this patchset's motivation so I will stop
> review and wait other opinions.
If you pass 0 to order_per_bit, you can manage cma area in every
size(page unit) you want. If you pass certain number to order_per_bit,
you can allocate and release cma area in multiple of such page order.
I think that this is more general implementation than previous versions.
Thanks.
On Thu, Jun 12, 2014 at 03:43:55PM +0900, Joonsoo Kim wrote:
> On Thu, Jun 12, 2014 at 03:06:10PM +0900, Minchan Kim wrote:
> > On Thu, Jun 12, 2014 at 12:21:42PM +0900, Joonsoo Kim wrote:
> > > ppc kvm's cma region management requires arbitrary bitmap granularity,
> > > since they want to reserve very large memory and manage this region
> > > with bitmap that one bit for several pages to reduce management overheads.
> > > So support arbitrary bitmap granularity for following generalization.
> > >
> > > Signed-off-by: Joonsoo Kim <[email protected]>
> > >
> > > diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> > > index bc4c171..9bc9340 100644
> > > --- a/drivers/base/dma-contiguous.c
> > > +++ b/drivers/base/dma-contiguous.c
> > > @@ -38,6 +38,7 @@ struct cma {
> > > unsigned long base_pfn;
> > > unsigned long count;
> > > unsigned long *bitmap;
> > > + int order_per_bit; /* Order of pages represented by one bit */
> >
> > Hmm, I'm not sure it's good as *general* interface even though it covers
> > existing usecases.
> >
> > It forces a cma area should be handled by same size unit. Right?
> > It's really important point for this patchset's motivation so I will stop
> > review and wait other opinions.
>
> If you pass 0 to order_per_bit, you can manage cma area in every
> size(page unit) you want. If you pass certain number to order_per_bit,
> you can allocate and release cma area in multiple of such page order.
>
> I think that this is more general implementation than previous versions.
Fair enough.
--
Kind regards,
Minchan Kim
On Thu, Jun 12, 2014 at 12:21:42PM +0900, Joonsoo Kim wrote:
> ppc kvm's cma region management requires arbitrary bitmap granularity,
> since they want to reserve very large memory and manage this region
> with bitmap that one bit for several pages to reduce management overheads.
> So support arbitrary bitmap granularity for following generalization.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Acked-by: Minchan Kim <[email protected]>
Just a nit below.
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index bc4c171..9bc9340 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -38,6 +38,7 @@ struct cma {
> unsigned long base_pfn;
> unsigned long count;
> unsigned long *bitmap;
> + int order_per_bit; /* Order of pages represented by one bit */
> struct mutex lock;
> };
>
> @@ -157,9 +158,38 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
>
> static DEFINE_MUTEX(cma_mutex);
>
> +static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
> +{
> + return (1 << (align_order >> cma->order_per_bit)) - 1;
> +}
> +
> +static unsigned long cma_bitmap_maxno(struct cma *cma)
> +{
> + return cma->count >> cma->order_per_bit;
> +}
> +
> +static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
> + unsigned long pages)
> +{
> + return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
> +}
> +
> +static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> +{
> + unsigned long bitmapno, nr_bits;
> +
> + bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + mutex_lock(&cma->lock);
> + bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> + mutex_unlock(&cma->lock);
> +}
> +
> static int __init cma_activate_area(struct cma *cma)
> {
> - int bitmap_size = BITS_TO_LONGS(cma->count) * sizeof(long);
> + int bitmap_maxno = cma_bitmap_maxno(cma);
> + int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> unsigned i = cma->count >> pageblock_order;
> struct zone *zone;
> @@ -221,6 +251,7 @@ core_initcall(cma_init_reserved_areas);
> * @base: Base address of the reserved area optional, use 0 for any
> * @limit: End address of the reserved memory (optional, 0 for any).
> * @alignment: Alignment for the contiguous memory area, should be power of 2
> + * @order_per_bit: Order of pages represented by one bit on bitmap.
> * @res_cma: Pointer to store the created cma region.
> * @fixed: hint about where to place the reserved area
> *
> @@ -235,7 +266,7 @@ core_initcall(cma_init_reserved_areas);
> */
> static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> - phys_addr_t alignment,
> + phys_addr_t alignment, int order_per_bit,
> struct cma **res_cma, bool fixed)
> {
> struct cma *cma = &cma_areas[cma_area_count];
> @@ -269,6 +300,8 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> base = ALIGN(base, alignment);
> size = ALIGN(size, alignment);
> limit &= ~(alignment - 1);
> + /* size should be aligned with order_per_bit */
> + BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
>
> /* Reserve memory */
> if (base && fixed) {
> @@ -294,6 +327,7 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> */
> cma->base_pfn = PFN_DOWN(base);
> cma->count = size >> PAGE_SHIFT;
> + cma->order_per_bit = order_per_bit;
> *res_cma = cma;
> cma_area_count++;
>
> @@ -313,7 +347,7 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> {
> int ret;
>
> - ret = __dma_contiguous_reserve_area(size, base, limit, 0,
> + ret = __dma_contiguous_reserve_area(size, base, limit, 0, 0,
> res_cma, fixed);
> if (ret)
> return ret;
> @@ -324,13 +358,6 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> return 0;
> }
>
> -static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> -{
> - mutex_lock(&cma->lock);
> - bitmap_clear(cma->bitmap, pfn - cma->base_pfn, count);
> - mutex_unlock(&cma->lock);
> -}
> -
> /**
> * dma_alloc_from_contiguous() - allocate pages from contiguous area
> * @dev: Pointer to device for which the allocation is performed.
> @@ -345,7 +372,8 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> unsigned int align)
> {
> - unsigned long mask, pfn, pageno, start = 0;
> + unsigned long mask, pfn, start = 0;
> + unsigned long bitmap_maxno, bitmapno, nr_bits;
Just Nit: bitmap_maxno, bitmap_no or something consistent.
I know you love consistent when I read description in first patch
in this patchset. ;-)
> struct page *page = NULL;
> int ret;
>
> @@ -358,18 +386,19 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> if (!count)
> return NULL;
>
> - mask = (1 << align) - 1;
> -
> + mask = cma_bitmap_aligned_mask(cma, align);
> + bitmap_maxno = cma_bitmap_maxno(cma);
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
>
> for (;;) {
> mutex_lock(&cma->lock);
> - pageno = bitmap_find_next_zero_area(cma->bitmap, cma->count,
> - start, count, mask);
> - if (pageno >= cma->count) {
> + bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> + bitmap_maxno, start, nr_bits, mask);
> + if (bitmapno >= bitmap_maxno) {
> mutex_unlock(&cma->lock);
> break;
> }
> - bitmap_set(cma->bitmap, pageno, count);
> + bitmap_set(cma->bitmap, bitmapno, nr_bits);
> /*
> * It's safe to drop the lock here. We've marked this region for
> * our exclusive use. If the migration fails we will take the
> @@ -377,7 +406,7 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> */
> mutex_unlock(&cma->lock);
>
> - pfn = cma->base_pfn + pageno;
> + pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> mutex_lock(&cma_mutex);
> ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> mutex_unlock(&cma_mutex);
> @@ -392,7 +421,7 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> pr_debug("%s(): memory range at %p is busy, retrying\n",
> __func__, pfn_to_page(pfn));
> /* try again with a bit different memory target */
> - start = pageno + mask + 1;
> + start = bitmapno + mask + 1;
> }
>
> pr_debug("%s(): returned %p\n", __func__, page);
> --
> 1.7.9.5
--
Kind regards,
Minchan Kim
On Thu, Jun 12, 2014 at 12:21:43PM +0900, Joonsoo Kim wrote:
> Currently, there are two users on CMA functionality, one is the DMA
> subsystem and the other is the kvm on powerpc. They have their own code
> to manage CMA reserved area even if they looks really similar.
> >From my guess, it is caused by some needs on bitmap management. Kvm side
> wants to maintain bitmap not for 1 page, but for more size. Eventually it
> use bitmap where one bit represents 64 pages.
>
> When I implement CMA related patches, I should change those two places
> to apply my change and it seem to be painful to me. I want to change
> this situation and reduce future code management overhead through
> this patch.
>
> This change could also help developer who want to use CMA in their
> new feature development, since they can use CMA easily without
> copying & pasting this reserved area management code.
>
> In previous patches, we have prepared some features to generalize
> CMA reserved area management and now it's time to do it. This patch
> moves core functions to mm/cma.c and change DMA APIs to use
> these functions.
>
> There is no functional change in DMA APIs.
>
> v2: There is no big change from v1 in mm/cma.c. Mostly renaming.
>
> Acked-by: Michal Nazarewicz <[email protected]>
> Signed-off-by: Joonsoo Kim <[email protected]>
Acutally, I want to remove bool return of cma_release but it's not
a out of scope in this patchset.
Acked-by: Minchan Kim <[email protected]>
>
> diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
> index 00e13ce..4eac559 100644
> --- a/drivers/base/Kconfig
> +++ b/drivers/base/Kconfig
> @@ -283,16 +283,6 @@ config CMA_ALIGNMENT
>
> If unsure, leave the default value "8".
>
> -config CMA_AREAS
> - int "Maximum count of the CMA device-private areas"
> - default 7
> - help
> - CMA allows to create CMA areas for particular devices. This parameter
> - sets the maximum number of such device private CMA areas in the
> - system.
> -
> - If unsure, leave the default value "7".
> -
> endif
>
> endmenu
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 9bc9340..f177f73 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -24,25 +24,10 @@
>
> #include <linux/memblock.h>
> #include <linux/err.h>
> -#include <linux/mm.h>
> -#include <linux/mutex.h>
> -#include <linux/page-isolation.h>
> #include <linux/sizes.h>
> -#include <linux/slab.h>
> -#include <linux/swap.h>
> -#include <linux/mm_types.h>
> #include <linux/dma-contiguous.h>
> #include <linux/log2.h>
Should we remain log2.h in here?
> -
> -struct cma {
> - unsigned long base_pfn;
> - unsigned long count;
> - unsigned long *bitmap;
> - int order_per_bit; /* Order of pages represented by one bit */
> - struct mutex lock;
> -};
> -
> -struct cma *dma_contiguous_default_area;
> +#include <linux/cma.h>
>
> #ifdef CONFIG_CMA_SIZE_MBYTES
> #define CMA_SIZE_MBYTES CONFIG_CMA_SIZE_MBYTES
> @@ -50,6 +35,8 @@ struct cma *dma_contiguous_default_area;
> #define CMA_SIZE_MBYTES 0
> #endif
>
> +struct cma *dma_contiguous_default_area;
> +
> /*
> * Default global CMA area size can be defined in kernel's .config.
> * This is useful mainly for distro maintainers to create a kernel
> @@ -156,199 +143,13 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
> }
> }
>
> -static DEFINE_MUTEX(cma_mutex);
> -
> -static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
> -{
> - return (1 << (align_order >> cma->order_per_bit)) - 1;
> -}
> -
> -static unsigned long cma_bitmap_maxno(struct cma *cma)
> -{
> - return cma->count >> cma->order_per_bit;
> -}
> -
> -static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
> - unsigned long pages)
> -{
> - return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
> -}
> -
> -static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> -{
> - unsigned long bitmapno, nr_bits;
> -
> - bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> - nr_bits = cma_bitmap_pages_to_bits(cma, count);
> -
> - mutex_lock(&cma->lock);
> - bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> - mutex_unlock(&cma->lock);
> -}
> -
> -static int __init cma_activate_area(struct cma *cma)
> -{
> - int bitmap_maxno = cma_bitmap_maxno(cma);
> - int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> - unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> - unsigned i = cma->count >> pageblock_order;
> - struct zone *zone;
> -
> - pr_debug("%s()\n", __func__);
> -
> - cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> - if (!cma->bitmap)
> - return -ENOMEM;
> -
> - WARN_ON_ONCE(!pfn_valid(pfn));
> - zone = page_zone(pfn_to_page(pfn));
> -
> - do {
> - unsigned j;
> - base_pfn = pfn;
> - for (j = pageblock_nr_pages; j; --j, pfn++) {
> - WARN_ON_ONCE(!pfn_valid(pfn));
> - /*
> - * alloc_contig_range requires the pfn range
> - * specified to be in the same zone. Make this
> - * simple by forcing the entire CMA resv range
> - * to be in the same zone.
> - */
> - if (page_zone(pfn_to_page(pfn)) != zone)
> - goto err;
> - }
> - init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> - } while (--i);
> -
> - mutex_init(&cma->lock);
> - return 0;
> -
> -err:
> - kfree(cma->bitmap);
> - return -EINVAL;
> -}
> -
> -static struct cma cma_areas[MAX_CMA_AREAS];
> -static unsigned cma_area_count;
> -
> -static int __init cma_init_reserved_areas(void)
> -{
> - int i;
> -
> - for (i = 0; i < cma_area_count; i++) {
> - int ret = cma_activate_area(&cma_areas[i]);
> - if (ret)
> - return ret;
> - }
> -
> - return 0;
> -}
> -core_initcall(cma_init_reserved_areas);
> -
> -/**
> - * dma_contiguous_reserve_area() - reserve custom contiguous area
> - * @size: Size of the reserved area (in bytes),
> - * @base: Base address of the reserved area optional, use 0 for any
> - * @limit: End address of the reserved memory (optional, 0 for any).
> - * @alignment: Alignment for the contiguous memory area, should be power of 2
> - * @order_per_bit: Order of pages represented by one bit on bitmap.
> - * @res_cma: Pointer to store the created cma region.
> - * @fixed: hint about where to place the reserved area
> - *
> - * This function reserves memory from early allocator. It should be
> - * called by arch specific code once the early allocator (memblock or bootmem)
> - * has been activated and all other subsystems have already allocated/reserved
> - * memory. This function allows to create custom reserved areas for specific
> - * devices.
> - *
> - * If @fixed is true, reserve contiguous area at exactly @base. If false,
> - * reserve in range from @base to @limit.
> - */
> -static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> - phys_addr_t base, phys_addr_t limit,
> - phys_addr_t alignment, int order_per_bit,
> - struct cma **res_cma, bool fixed)
> -{
> - struct cma *cma = &cma_areas[cma_area_count];
> - int ret = 0;
> -
> - pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
> - __func__, (unsigned long)size, (unsigned long)base,
> - (unsigned long)limit, (unsigned long)alignment);
> -
> - /* Sanity checks */
> - if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> - pr_err("%s(): Not enough slots for CMA reserved regions!\n",
> - __func__);
> - return -ENOSPC;
> - }
> -
> - if (!size)
> - return -EINVAL;
> -
> - if (alignment && !is_power_of_2(alignment))
> - return -EINVAL;
> -
> - /*
> - * Sanitise input arguments.
> - * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
> - * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
> - * and CMA property will be broken.
> - */
> - alignment = max(alignment,
> - (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
> - base = ALIGN(base, alignment);
> - size = ALIGN(size, alignment);
> - limit &= ~(alignment - 1);
> - /* size should be aligned with order_per_bit */
> - BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
> -
> - /* Reserve memory */
> - if (base && fixed) {
> - if (memblock_is_region_reserved(base, size) ||
> - memblock_reserve(base, size) < 0) {
> - ret = -EBUSY;
> - goto err;
> - }
> - } else {
> - phys_addr_t addr = memblock_alloc_range(size, alignment, base,
> - limit);
> - if (!addr) {
> - ret = -ENOMEM;
> - goto err;
> - } else {
> - base = addr;
> - }
> - }
> -
> - /*
> - * Each reserved area must be initialised later, when more kernel
> - * subsystems (like slab allocator) are available.
> - */
> - cma->base_pfn = PFN_DOWN(base);
> - cma->count = size >> PAGE_SHIFT;
> - cma->order_per_bit = order_per_bit;
> - *res_cma = cma;
> - cma_area_count++;
> -
> - pr_info("%s(): reserved %ld MiB at %08lx\n",
> - __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
> - return 0;
> -
> -err:
> - pr_err("%s(): failed to reserve %ld MiB\n",
> - __func__, (unsigned long)size / SZ_1M);
> - return ret;
> -}
> -
> int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> phys_addr_t limit, struct cma **res_cma,
> bool fixed)
> {
> int ret;
>
> - ret = __dma_contiguous_reserve_area(size, base, limit, 0, 0,
> - res_cma, fixed);
> + ret = cma_declare_contiguous(size, base, limit, 0, 0, res_cma, fixed);
> if (ret)
> return ret;
>
> @@ -358,124 +159,17 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> return 0;
> }
>
> -/**
> - * dma_alloc_from_contiguous() - allocate pages from contiguous area
> - * @dev: Pointer to device for which the allocation is performed.
> - * @count: Requested number of pages.
> - * @align: Requested alignment of pages (in PAGE_SIZE order).
> - *
> - * This function allocates memory buffer for specified device. It uses
> - * device specific contiguous memory area if available or the default
> - * global one. Requires architecture specific dev_get_cma_area() helper
> - * function.
> - */
> -static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> - unsigned int align)
> -{
> - unsigned long mask, pfn, start = 0;
> - unsigned long bitmap_maxno, bitmapno, nr_bits;
> - struct page *page = NULL;
> - int ret;
> -
> - if (!cma || !cma->count)
> - return NULL;
> -
> - pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
> - count, align);
> -
> - if (!count)
> - return NULL;
> -
> - mask = cma_bitmap_aligned_mask(cma, align);
> - bitmap_maxno = cma_bitmap_maxno(cma);
> - nr_bits = cma_bitmap_pages_to_bits(cma, count);
> -
> - for (;;) {
> - mutex_lock(&cma->lock);
> - bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> - bitmap_maxno, start, nr_bits, mask);
> - if (bitmapno >= bitmap_maxno) {
> - mutex_unlock(&cma->lock);
> - break;
> - }
> - bitmap_set(cma->bitmap, bitmapno, nr_bits);
> - /*
> - * It's safe to drop the lock here. We've marked this region for
> - * our exclusive use. If the migration fails we will take the
> - * lock again and unmark it.
> - */
> - mutex_unlock(&cma->lock);
> -
> - pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> - mutex_lock(&cma_mutex);
> - ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> - mutex_unlock(&cma_mutex);
> - if (ret == 0) {
> - page = pfn_to_page(pfn);
> - break;
> - } else if (ret != -EBUSY) {
> - clear_cma_bitmap(cma, pfn, count);
> - break;
> - }
> - clear_cma_bitmap(cma, pfn, count);
> - pr_debug("%s(): memory range at %p is busy, retrying\n",
> - __func__, pfn_to_page(pfn));
> - /* try again with a bit different memory target */
> - start = bitmapno + mask + 1;
> - }
> -
> - pr_debug("%s(): returned %p\n", __func__, page);
> - return page;
> -}
> -
> struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> unsigned int align)
> {
> - struct cma *cma = dev_get_cma_area(dev);
> -
> if (align > CONFIG_CMA_ALIGNMENT)
> align = CONFIG_CMA_ALIGNMENT;
>
> - return __dma_alloc_from_contiguous(cma, count, align);
> -}
> -
> -/**
> - * dma_release_from_contiguous() - release allocated pages
> - * @dev: Pointer to device for which the pages were allocated.
> - * @pages: Allocated pages.
> - * @count: Number of allocated pages.
> - *
> - * This function releases memory allocated by dma_alloc_from_contiguous().
> - * It returns false when provided pages do not belong to contiguous area and
> - * true otherwise.
> - */
> -static bool __dma_release_from_contiguous(struct cma *cma, struct page *pages,
> - int count)
> -{
> - unsigned long pfn;
> -
> - if (!cma || !pages)
> - return false;
> -
> - pr_debug("%s(page %p)\n", __func__, (void *)pages);
> -
> - pfn = page_to_pfn(pages);
> -
> - if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count)
> - return false;
> -
> - VM_BUG_ON(pfn + count > cma->base_pfn + cma->count);
> -
> - free_contig_range(pfn, count);
> - clear_cma_bitmap(cma, pfn, count);
> -
> - return true;
> + return cma_alloc(dev_get_cma_area(dev), count, align);
> }
>
> bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> int count)
> {
> - struct cma *cma = dev_get_cma_area(dev);
> -
> - return __dma_release_from_contiguous(cma, pages, count);
> + return cma_release(dev_get_cma_area(dev), pages, count);
> }
> diff --git a/include/linux/cma.h b/include/linux/cma.h
> new file mode 100644
> index 0000000..e38efe9
> --- /dev/null
> +++ b/include/linux/cma.h
> @@ -0,0 +1,12 @@
> +#ifndef __CMA_H__
> +#define __CMA_H__
> +
> +struct cma;
> +
> +extern int __init cma_declare_contiguous(phys_addr_t size,
> + phys_addr_t base, phys_addr_t limit,
> + phys_addr_t alignment, int order_per_bit,
> + struct cma **res_cma, bool fixed);
> +extern struct page *cma_alloc(struct cma *cma, int count, unsigned int align);
> +extern bool cma_release(struct cma *cma, struct page *pages, int count);
> +#endif
> diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h
> index 772eab5..a40c1f3 100644
> --- a/include/linux/dma-contiguous.h
> +++ b/include/linux/dma-contiguous.h
> @@ -53,9 +53,10 @@
>
> #ifdef __KERNEL__
>
> +#include <linux/device.h>
> +
> struct cma;
> struct page;
> -struct device;
>
> #ifdef CONFIG_DMA_CMA
>
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 3e9977a..f4899ec 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -508,6 +508,17 @@ config CMA_DEBUG
> processing calls such as dma_alloc_from_contiguous().
> This option does not affect warning and error messages.
>
> +config CMA_AREAS
> + int "Maximum count of the CMA areas"
> + depends on CMA
> + default 7
> + help
> + CMA allows to create CMA areas for particular purpose, mainly,
> + used as device private area. This parameter sets the maximum
> + number of CMA area in the system.
> +
> + If unsure, leave the default value "7".
> +
> config ZBUD
> tristate
> default n
> diff --git a/mm/Makefile b/mm/Makefile
> index 1eaa70b..bc0422b 100644
> --- a/mm/Makefile
> +++ b/mm/Makefile
> @@ -62,3 +62,4 @@ obj-$(CONFIG_MEMORY_ISOLATION) += page_isolation.o
> obj-$(CONFIG_ZBUD) += zbud.o
> obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
> obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
> +obj-$(CONFIG_CMA) += cma.o
> diff --git a/mm/cma.c b/mm/cma.c
> new file mode 100644
> index 0000000..1e1b017
> --- /dev/null
> +++ b/mm/cma.c
> @@ -0,0 +1,330 @@
> +/*
> + * Contiguous Memory Allocator
> + *
> + * Copyright (c) 2010-2011 by Samsung Electronics.
> + * Copyright IBM Corporation, 2013
> + * Copyright LG Electronics Inc., 2014
> + * Written by:
> + * Marek Szyprowski <[email protected]>
> + * Michal Nazarewicz <[email protected]>
> + * Aneesh Kumar K.V <[email protected]>
> + * Joonsoo Kim <[email protected]>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License as
> + * published by the Free Software Foundation; either version 2 of the
> + * License or (at your optional) any later version of the license.
> + */
> +
> +#define pr_fmt(fmt) "cma: " fmt
> +
> +#ifdef CONFIG_CMA_DEBUG
> +#ifndef DEBUG
> +# define DEBUG
> +#endif
> +#endif
> +
> +#include <linux/memblock.h>
> +#include <linux/err.h>
> +#include <linux/mm.h>
> +#include <linux/mutex.h>
> +#include <linux/sizes.h>
> +#include <linux/slab.h>
> +
> +struct cma {
> + unsigned long base_pfn;
> + unsigned long count;
> + unsigned long *bitmap;
> + int order_per_bit; /* Order of pages represented by one bit */
> + struct mutex lock;
> +};
> +
> +/*
> + * There is always at least global CMA area and a few optional
> + * areas configured in kernel .config.
> + */
> +#define MAX_CMA_AREAS (1 + CONFIG_CMA_AREAS)
> +
> +static struct cma cma_areas[MAX_CMA_AREAS];
> +static unsigned cma_area_count;
> +static DEFINE_MUTEX(cma_mutex);
> +
> +static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
> +{
> + return (1 << (align_order >> cma->order_per_bit)) - 1;
> +}
> +
> +static unsigned long cma_bitmap_maxno(struct cma *cma)
> +{
> + return cma->count >> cma->order_per_bit;
> +}
> +
> +static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
> + unsigned long pages)
> +{
> + return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
> +}
> +
> +static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> +{
> + unsigned long bitmapno, nr_bits;
> +
> + bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + mutex_lock(&cma->lock);
> + bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> + mutex_unlock(&cma->lock);
> +}
> +
> +static int __init cma_activate_area(struct cma *cma)
> +{
> + int bitmap_maxno = cma_bitmap_maxno(cma);
> + int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> + unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> + unsigned i = cma->count >> pageblock_order;
> + struct zone *zone;
> +
> + pr_debug("%s()\n", __func__);
> +
> + cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> + if (!cma->bitmap)
> + return -ENOMEM;
> +
> + WARN_ON_ONCE(!pfn_valid(pfn));
> + zone = page_zone(pfn_to_page(pfn));
> +
> + do {
> + unsigned j;
> +
> + base_pfn = pfn;
> + for (j = pageblock_nr_pages; j; --j, pfn++) {
> + WARN_ON_ONCE(!pfn_valid(pfn));
> + /*
> + * alloc_contig_range requires the pfn range
> + * specified to be in the same zone. Make this
> + * simple by forcing the entire CMA resv range
> + * to be in the same zone.
> + */
> + if (page_zone(pfn_to_page(pfn)) != zone)
> + goto err;
> + }
> + init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> + } while (--i);
> +
> + mutex_init(&cma->lock);
> + return 0;
> +
> +err:
> + kfree(cma->bitmap);
> + return -EINVAL;
> +}
> +
> +static int __init cma_init_reserved_areas(void)
> +{
> + int i;
> +
> + for (i = 0; i < cma_area_count; i++) {
> + int ret = cma_activate_area(&cma_areas[i]);
> +
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> +core_initcall(cma_init_reserved_areas);
> +
> +/**
> + * cma_declare_contiguous() - reserve custom contiguous area
> + * @size: Size of the reserved area (in bytes),
> + * @base: Base address of the reserved area optional, use 0 for any
> + * @limit: End address of the reserved memory (optional, 0 for any).
> + * @alignment: Alignment for the contiguous memory area, should be power of 2
> + * @order_per_bit: Order of pages represented by one bit on bitmap.
> + * @res_cma: Pointer to store the created cma region.
> + * @fixed: hint about where to place the reserved area
> + *
> + * This function reserves memory from early allocator. It should be
> + * called by arch specific code once the early allocator (memblock or bootmem)
> + * has been activated and all other subsystems have already allocated/reserved
> + * memory. This function allows to create custom reserved areas.
> + *
> + * If @fixed is true, reserve contiguous area at exactly @base. If false,
> + * reserve in range from @base to @limit.
> + */
> +int __init cma_declare_contiguous(phys_addr_t size,
> + phys_addr_t base, phys_addr_t limit,
> + phys_addr_t alignment, int order_per_bit,
> + struct cma **res_cma, bool fixed)
> +{
> + struct cma *cma = &cma_areas[cma_area_count];
> + int ret = 0;
> +
> + pr_debug("%s(size %lx, base %08lx, limit %08lx alignment %08lx)\n",
> + __func__, (unsigned long)size, (unsigned long)base,
> + (unsigned long)limit, (unsigned long)alignment);
> +
> + /* Sanity checks */
> + if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> + pr_err("%s(): Not enough slots for CMA reserved regions!\n",
> + __func__);
> + return -ENOSPC;
> + }
> +
> + if (!size)
> + return -EINVAL;
> +
> + if (alignment && !is_power_of_2(alignment))
> + return -EINVAL;
> +
> + /*
> + * Sanitise input arguments.
> + * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
> + * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
> + * and CMA property will be broken.
> + */
> + alignment = max(alignment,
> + (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
> + base = ALIGN(base, alignment);
> + size = ALIGN(size, alignment);
> + limit &= ~(alignment - 1);
> + /* size should be aligned with order_per_bit */
> + BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
> +
> + /* Reserve memory */
> + if (base && fixed) {
> + if (memblock_is_region_reserved(base, size) ||
> + memblock_reserve(base, size) < 0) {
> + ret = -EBUSY;
> + goto err;
> + }
> + } else {
> + phys_addr_t addr = memblock_alloc_range(size, alignment, base,
> + limit);
> + if (!addr) {
> + ret = -ENOMEM;
> + goto err;
> + } else {
> + base = addr;
> + }
> + }
> +
> + /*
> + * Each reserved area must be initialised later, when more kernel
> + * subsystems (like slab allocator) are available.
> + */
> + cma->base_pfn = PFN_DOWN(base);
> + cma->count = size >> PAGE_SHIFT;
> + cma->order_per_bit = order_per_bit;
> + *res_cma = cma;
> + cma_area_count++;
> +
> + pr_info("%s(): reserved %ld MiB at %08lx\n",
> + __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
> + return 0;
> +
> +err:
> + pr_err("%s(): failed to reserve %ld MiB\n",
> + __func__, (unsigned long)size / SZ_1M);
> + return ret;
> +}
> +
> +/**
> + * cma_alloc() - allocate pages from contiguous area
> + * @cma: Contiguous memory region for which the allocation is performed.
> + * @count: Requested number of pages.
> + * @align: Requested alignment of pages (in PAGE_SIZE order).
> + *
> + * This function allocates part of contiguous memory on specific
> + * contiguous memory area.
> + */
> +struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
> +{
> + unsigned long mask, pfn, start = 0;
> + unsigned long bitmap_maxno, bitmapno, nr_bits;
> + struct page *page = NULL;
> + int ret;
> +
> + if (!cma || !cma->count)
> + return NULL;
> +
> + pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
> + count, align);
> +
> + if (!count)
> + return NULL;
> +
> + mask = cma_bitmap_aligned_mask(cma, align);
> + bitmap_maxno = cma_bitmap_maxno(cma);
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + for (;;) {
> + mutex_lock(&cma->lock);
> + bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> + bitmap_maxno, start, nr_bits, mask);
> + if (bitmapno >= bitmap_maxno) {
> + mutex_unlock(&cma->lock);
> + break;
> + }
> + bitmap_set(cma->bitmap, bitmapno, nr_bits);
> + /*
> + * It's safe to drop the lock here. We've marked this region for
> + * our exclusive use. If the migration fails we will take the
> + * lock again and unmark it.
> + */
> + mutex_unlock(&cma->lock);
> +
> + pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> + mutex_lock(&cma_mutex);
> + ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> + mutex_unlock(&cma_mutex);
> + if (ret == 0) {
> + page = pfn_to_page(pfn);
> + break;
> + } else if (ret != -EBUSY) {
> + clear_cma_bitmap(cma, pfn, count);
> + break;
> + }
> + clear_cma_bitmap(cma, pfn, count);
> + pr_debug("%s(): memory range at %p is busy, retrying\n",
> + __func__, pfn_to_page(pfn));
> + /* try again with a bit different memory target */
> + start = bitmapno + mask + 1;
> + }
> +
> + pr_debug("%s(): returned %p\n", __func__, page);
> + return page;
> +}
> +
> +/**
> + * cma_release() - release allocated pages
> + * @cma: Contiguous memory region for which the allocation is performed.
> + * @pages: Allocated pages.
> + * @count: Number of allocated pages.
> + *
> + * This function releases memory allocated by alloc_cma().
> + * It returns false when provided pages do not belong to contiguous area and
> + * true otherwise.
> + */
> +bool cma_release(struct cma *cma, struct page *pages, int count)
> +{
> + unsigned long pfn;
> +
> + if (!cma || !pages)
> + return false;
> +
> + pr_debug("%s(page %p)\n", __func__, (void *)pages);
> +
> + pfn = page_to_pfn(pages);
> +
> + if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count)
> + return false;
> +
> + VM_BUG_ON(pfn + count > cma->base_pfn + cma->count);
> +
> + free_contig_range(pfn, count);
> + clear_cma_bitmap(cma, pfn, count);
> +
> + return true;
> +}
> --
> 1.7.9.5
--
Kind regards,
Minchan Kim
On Thu, Jun 12, 2014 at 12:21:45PM +0900, Joonsoo Kim wrote:
> We can remove one call sites for clear_cma_bitmap() if we first
> call it before checking error number.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Acked-by: Minchan Kim <[email protected]>
--
Kind regards,
Minchan Kim
On Thu, Jun 12, 2014 at 12:21:46PM +0900, Joonsoo Kim wrote:
> Conventionally, we put output param to the end of param list.
> cma_declare_contiguous() doesn't look like that, so change it.
If you says "Conventionally", I'd like to suggest one more thing.
Conventionally, we put 'base' ahead 'size' but dma_contiguous_reserve_area
is opposite.
>
> Additionally, move down cma_areas reference code to the position
> where it is really needed.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
>
> diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c
> index 28ec226..97613ea 100644
> --- a/arch/powerpc/kvm/book3s_hv_builtin.c
> +++ b/arch/powerpc/kvm/book3s_hv_builtin.c
> @@ -184,7 +184,7 @@ void __init kvm_cma_reserve(void)
>
> align_size = max(kvm_rma_pages << PAGE_SHIFT, align_size);
> cma_declare_contiguous(selected_size, 0, 0, align_size,
> - KVM_CMA_CHUNK_ORDER - PAGE_SHIFT, &kvm_cma, false);
> + KVM_CMA_CHUNK_ORDER - PAGE_SHIFT, false, &kvm_cma);
> }
> }
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index f177f73..bfd4553 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -149,7 +149,7 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> {
> int ret;
>
> - ret = cma_declare_contiguous(size, base, limit, 0, 0, res_cma, fixed);
> + ret = cma_declare_contiguous(size, base, limit, 0, 0, fixed, res_cma);
> if (ret)
> return ret;
>
> diff --git a/include/linux/cma.h b/include/linux/cma.h
> index e38efe9..e53eead 100644
> --- a/include/linux/cma.h
> +++ b/include/linux/cma.h
> @@ -6,7 +6,7 @@ struct cma;
> extern int __init cma_declare_contiguous(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> phys_addr_t alignment, int order_per_bit,
> - struct cma **res_cma, bool fixed);
> + bool fixed, struct cma **res_cma);
> extern struct page *cma_alloc(struct cma *cma, int count, unsigned int align);
> extern bool cma_release(struct cma *cma, struct page *pages, int count);
> #endif
> diff --git a/mm/cma.c b/mm/cma.c
> index 01a0713..22a5b23 100644
> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -142,8 +142,8 @@ core_initcall(cma_init_reserved_areas);
> * @limit: End address of the reserved memory (optional, 0 for any).
> * @alignment: Alignment for the contiguous memory area, should be power of 2
> * @order_per_bit: Order of pages represented by one bit on bitmap.
> - * @res_cma: Pointer to store the created cma region.
> * @fixed: hint about where to place the reserved area
> + * @res_cma: Pointer to store the created cma region.
> *
> * This function reserves memory from early allocator. It should be
> * called by arch specific code once the early allocator (memblock or bootmem)
> @@ -156,9 +156,9 @@ core_initcall(cma_init_reserved_areas);
> int __init cma_declare_contiguous(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> phys_addr_t alignment, int order_per_bit,
> - struct cma **res_cma, bool fixed)
> + bool fixed, struct cma **res_cma)
> {
> - struct cma *cma = &cma_areas[cma_area_count];
> + struct cma *cma;
> int ret = 0;
>
> pr_debug("%s(size %lx, base %08lx, limit %08lx alignment %08lx)\n",
> @@ -214,6 +214,7 @@ int __init cma_declare_contiguous(phys_addr_t size,
> * Each reserved area must be initialised later, when more kernel
> * subsystems (like slab allocator) are available.
> */
> + cma = &cma_areas[cma_area_count];
> cma->base_pfn = PFN_DOWN(base);
> cma->count = size >> PAGE_SHIFT;
> cma->order_per_bit = order_per_bit;
> --
> 1.7.9.5
--
Kind regards,
Minchan Kim
On 06/12/2014 03:08 PM, Minchan Kim wrote:
> On Thu, Jun 12, 2014 at 12:21:42PM +0900, Joonsoo Kim wrote:
>> ppc kvm's cma region management requires arbitrary bitmap granularity,
>> since they want to reserve very large memory and manage this region
>> with bitmap that one bit for several pages to reduce management overheads.
>> So support arbitrary bitmap granularity for following generalization.
>>
>> Signed-off-by: Joonsoo Kim <[email protected]>
> Acked-by: Minchan Kim <[email protected]>
>
> Just a nit below.
>
>>
>> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
>> index bc4c171..9bc9340 100644
>> --- a/drivers/base/dma-contiguous.c
>> +++ b/drivers/base/dma-contiguous.c
>> @@ -38,6 +38,7 @@ struct cma {
>> unsigned long base_pfn;
>> unsigned long count;
>> unsigned long *bitmap;
>> + int order_per_bit; /* Order of pages represented by one bit */
>> struct mutex lock;
>> };
>>
>> @@ -157,9 +158,38 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
>>
>> static DEFINE_MUTEX(cma_mutex);
>>
>> +static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
>> +{
>> + return (1 << (align_order >> cma->order_per_bit)) - 1;
>> +}
>> +
>> +static unsigned long cma_bitmap_maxno(struct cma *cma)
>> +{
>> + return cma->count >> cma->order_per_bit;
>> +}
>> +
>> +static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
>> + unsigned long pages)
>> +{
>> + return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
>> +}
>> +
>> +static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
>> +{
>> + unsigned long bitmapno, nr_bits;
>> +
>> + bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
>> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
>> +
>> + mutex_lock(&cma->lock);
>> + bitmap_clear(cma->bitmap, bitmapno, nr_bits);
>> + mutex_unlock(&cma->lock);
>> +}
>> +
>> static int __init cma_activate_area(struct cma *cma)
>> {
>> - int bitmap_size = BITS_TO_LONGS(cma->count) * sizeof(long);
>> + int bitmap_maxno = cma_bitmap_maxno(cma);
>> + int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
>> unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
>> unsigned i = cma->count >> pageblock_order;
>> struct zone *zone;
>> @@ -221,6 +251,7 @@ core_initcall(cma_init_reserved_areas);
>> * @base: Base address of the reserved area optional, use 0 for any
>> * @limit: End address of the reserved memory (optional, 0 for any).
>> * @alignment: Alignment for the contiguous memory area, should be power of 2
>> + * @order_per_bit: Order of pages represented by one bit on bitmap.
>> * @res_cma: Pointer to store the created cma region.
>> * @fixed: hint about where to place the reserved area
>> *
>> @@ -235,7 +266,7 @@ core_initcall(cma_init_reserved_areas);
>> */
>> static int __init __dma_contiguous_reserve_area(phys_addr_t size,
>> phys_addr_t base, phys_addr_t limit,
>> - phys_addr_t alignment,
>> + phys_addr_t alignment, int order_per_bit,
>> struct cma **res_cma, bool fixed)
>> {
>> struct cma *cma = &cma_areas[cma_area_count];
>> @@ -269,6 +300,8 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
>> base = ALIGN(base, alignment);
>> size = ALIGN(size, alignment);
>> limit &= ~(alignment - 1);
>> + /* size should be aligned with order_per_bit */
>> + BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
>>
>> /* Reserve memory */
>> if (base && fixed) {
>> @@ -294,6 +327,7 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
>> */
>> cma->base_pfn = PFN_DOWN(base);
>> cma->count = size >> PAGE_SHIFT;
>> + cma->order_per_bit = order_per_bit;
>> *res_cma = cma;
>> cma_area_count++;
>>
>> @@ -313,7 +347,7 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
>> {
>> int ret;
>>
>> - ret = __dma_contiguous_reserve_area(size, base, limit, 0,
>> + ret = __dma_contiguous_reserve_area(size, base, limit, 0, 0,
>> res_cma, fixed);
>> if (ret)
>> return ret;
>> @@ -324,13 +358,6 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
>> return 0;
>> }
>>
>> -static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
>> -{
>> - mutex_lock(&cma->lock);
>> - bitmap_clear(cma->bitmap, pfn - cma->base_pfn, count);
>> - mutex_unlock(&cma->lock);
>> -}
>> -
>> /**
>> * dma_alloc_from_contiguous() - allocate pages from contiguous area
>> * @dev: Pointer to device for which the allocation is performed.
>> @@ -345,7 +372,8 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
>> static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
>> unsigned int align)
>> {
>> - unsigned long mask, pfn, pageno, start = 0;
>> + unsigned long mask, pfn, start = 0;
>> + unsigned long bitmap_maxno, bitmapno, nr_bits;
>
> Just Nit: bitmap_maxno, bitmap_no or something consistent.
> I know you love consistent when I read description in first patch
> in this patchset. ;-)
>
Yeah, not only in this patchset, I saw Joonsoo trying to unify all
kinds of things in the MM. This is great for newbies, IMO.
--
Thanks.
Zhang Yanfei
On Thu, Jun 12, 2014 at 04:08:11PM +0900, Minchan Kim wrote:
> On Thu, Jun 12, 2014 at 12:21:42PM +0900, Joonsoo Kim wrote:
> > ppc kvm's cma region management requires arbitrary bitmap granularity,
> > since they want to reserve very large memory and manage this region
> > with bitmap that one bit for several pages to reduce management overheads.
> > So support arbitrary bitmap granularity for following generalization.
> >
> > Signed-off-by: Joonsoo Kim <[email protected]>
> Acked-by: Minchan Kim <[email protected]>
>
Thanks.
[snip...]
> > /**
> > * dma_alloc_from_contiguous() - allocate pages from contiguous area
> > * @dev: Pointer to device for which the allocation is performed.
> > @@ -345,7 +372,8 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> > static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> > unsigned int align)
> > {
> > - unsigned long mask, pfn, pageno, start = 0;
> > + unsigned long mask, pfn, start = 0;
> > + unsigned long bitmap_maxno, bitmapno, nr_bits;
>
> Just Nit: bitmap_maxno, bitmap_no or something consistent.
> I know you love consistent when I read description in first patch
> in this patchset. ;-)
Yeah, I will fix it. :)
Thanks.
On Thu, Jun 12, 2014 at 04:13:11PM +0900, Minchan Kim wrote:
> On Thu, Jun 12, 2014 at 12:21:43PM +0900, Joonsoo Kim wrote:
> > Currently, there are two users on CMA functionality, one is the DMA
> > subsystem and the other is the kvm on powerpc. They have their own code
> > to manage CMA reserved area even if they looks really similar.
> > >From my guess, it is caused by some needs on bitmap management. Kvm side
> > wants to maintain bitmap not for 1 page, but for more size. Eventually it
> > use bitmap where one bit represents 64 pages.
> >
> > When I implement CMA related patches, I should change those two places
> > to apply my change and it seem to be painful to me. I want to change
> > this situation and reduce future code management overhead through
> > this patch.
> >
> > This change could also help developer who want to use CMA in their
> > new feature development, since they can use CMA easily without
> > copying & pasting this reserved area management code.
> >
> > In previous patches, we have prepared some features to generalize
> > CMA reserved area management and now it's time to do it. This patch
> > moves core functions to mm/cma.c and change DMA APIs to use
> > these functions.
> >
> > There is no functional change in DMA APIs.
> >
> > v2: There is no big change from v1 in mm/cma.c. Mostly renaming.
> >
> > Acked-by: Michal Nazarewicz <[email protected]>
> > Signed-off-by: Joonsoo Kim <[email protected]>
>
> Acutally, I want to remove bool return of cma_release but it's not
> a out of scope in this patchset.
>
> Acked-by: Minchan Kim <[email protected]>
>
> >
> > diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
> > index 00e13ce..4eac559 100644
> > --- a/drivers/base/Kconfig
> > +++ b/drivers/base/Kconfig
> > @@ -283,16 +283,6 @@ config CMA_ALIGNMENT
> >
> > If unsure, leave the default value "8".
> >
> > -config CMA_AREAS
> > - int "Maximum count of the CMA device-private areas"
> > - default 7
> > - help
> > - CMA allows to create CMA areas for particular devices. This parameter
> > - sets the maximum number of such device private CMA areas in the
> > - system.
> > -
> > - If unsure, leave the default value "7".
> > -
> > endif
> >
> > endmenu
> > diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> > index 9bc9340..f177f73 100644
> > --- a/drivers/base/dma-contiguous.c
> > +++ b/drivers/base/dma-contiguous.c
> > @@ -24,25 +24,10 @@
> >
> > #include <linux/memblock.h>
> > #include <linux/err.h>
> > -#include <linux/mm.h>
> > -#include <linux/mutex.h>
> > -#include <linux/page-isolation.h>
> > #include <linux/sizes.h>
> > -#include <linux/slab.h>
> > -#include <linux/swap.h>
> > -#include <linux/mm_types.h>
> > #include <linux/dma-contiguous.h>
> > #include <linux/log2.h>
>
> Should we remain log2.h in here?
>
We should remove it. I will fix it.
Thanks.
On Thu, Jun 12, 2014 at 04:19:31PM +0900, Minchan Kim wrote:
> On Thu, Jun 12, 2014 at 12:21:46PM +0900, Joonsoo Kim wrote:
> > Conventionally, we put output param to the end of param list.
> > cma_declare_contiguous() doesn't look like that, so change it.
>
> If you says "Conventionally", I'd like to suggest one more thing.
> Conventionally, we put 'base' ahead 'size' but dma_contiguous_reserve_area
> is opposite.
Okay. I will do it. :)
Thanks.
On Thu, Jun 12, 2014 at 12:21:47PM +0900, Joonsoo Kim wrote:
> Currently, we should take the mutex for manipulating bitmap.
> This job may be really simple and short so we don't need to sleep
> if contended. So I change it to spinlock.
I'm not sure it would be good always.
Maybe you remember we discussed about similar stuff about bitmap
searching in vmap friend internally, which was really painful
when it was fragmented. So, at least we need number if you really want
and I hope the number from ARM machine most popular platform for CMA
at the moment.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
>
> diff --git a/mm/cma.c b/mm/cma.c
> index 22a5b23..3085e8c 100644
> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -27,6 +27,7 @@
> #include <linux/memblock.h>
> #include <linux/err.h>
> #include <linux/mm.h>
> +#include <linux/spinlock.h>
> #include <linux/mutex.h>
> #include <linux/sizes.h>
> #include <linux/slab.h>
> @@ -36,7 +37,7 @@ struct cma {
> unsigned long count;
> unsigned long *bitmap;
> int order_per_bit; /* Order of pages represented by one bit */
> - struct mutex lock;
> + spinlock_t lock;
> };
>
> /*
> @@ -72,9 +73,9 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> nr_bits = cma_bitmap_pages_to_bits(cma, count);
>
> - mutex_lock(&cma->lock);
> + spin_lock(&cma->lock);
> bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> - mutex_unlock(&cma->lock);
> + spin_unlock(&cma->lock);
> }
>
> static int __init cma_activate_area(struct cma *cma)
> @@ -112,7 +113,7 @@ static int __init cma_activate_area(struct cma *cma)
> init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> } while (--i);
>
> - mutex_init(&cma->lock);
> + spin_lock_init(&cma->lock);
> return 0;
>
> err:
> @@ -261,11 +262,11 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
> nr_bits = cma_bitmap_pages_to_bits(cma, count);
>
> for (;;) {
> - mutex_lock(&cma->lock);
> + spin_lock(&cma->lock);
> bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> bitmap_maxno, start, nr_bits, mask);
> if (bitmapno >= bitmap_maxno) {
> - mutex_unlock(&cma->lock);
> + spin_unlock(&cma->lock);
> break;
> }
> bitmap_set(cma->bitmap, bitmapno, nr_bits);
> @@ -274,7 +275,7 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
> * our exclusive use. If the migration fails we will take the
> * lock again and unmark it.
> */
> - mutex_unlock(&cma->lock);
> + spin_unlock(&cma->lock);
>
> pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> mutex_lock(&cma_mutex);
> --
> 1.7.9.5
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to [email protected]. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
--
Kind regards,
Minchan Kim
On Thu, Jun 12, 2014 at 04:40:29PM +0900, Minchan Kim wrote:
> On Thu, Jun 12, 2014 at 12:21:47PM +0900, Joonsoo Kim wrote:
> > Currently, we should take the mutex for manipulating bitmap.
> > This job may be really simple and short so we don't need to sleep
> > if contended. So I change it to spinlock.
>
> I'm not sure it would be good always.
> Maybe you remember we discussed about similar stuff about bitmap
> searching in vmap friend internally, which was really painful
> when it was fragmented. So, at least we need number if you really want
> and I hope the number from ARM machine most popular platform for CMA
> at the moment.
Good Point!! Agreed. I will drop this one in next spin and re-submit
in separate patchset after some testing.
Thanks.
On 06/12/2014 11:21 AM, Joonsoo Kim wrote:
> We don't need explicit 'CMA:' prefix, since we already define prefix
> 'cma:' in pr_fmt. So remove it.
>
> And, some logs print function name and others doesn't. This looks
> bad to me, so I unify log format to print function name consistently.
>
> Lastly, I add one more debug log on cma_activate_area().
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Reviewed-by: Zhang Yanfei <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 83969f8..bd0bb81 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -144,7 +144,7 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
> }
>
> if (selected_size && !dma_contiguous_default_area) {
> - pr_debug("%s: reserving %ld MiB for global area\n", __func__,
> + pr_debug("%s(): reserving %ld MiB for global area\n", __func__,
> (unsigned long)selected_size / SZ_1M);
>
> dma_contiguous_reserve_area(selected_size, selected_base,
> @@ -163,8 +163,9 @@ static int __init cma_activate_area(struct cma *cma)
> unsigned i = cma->count >> pageblock_order;
> struct zone *zone;
>
> - cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> + pr_debug("%s()\n", __func__);
>
> + cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> if (!cma->bitmap)
> return -ENOMEM;
>
> @@ -234,7 +235,8 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
>
> /* Sanity checks */
> if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> - pr_err("Not enough slots for CMA reserved regions!\n");
> + pr_err("%s(): Not enough slots for CMA reserved regions!\n",
> + __func__);
> return -ENOSPC;
> }
>
> @@ -274,14 +276,15 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> *res_cma = cma;
> cma_area_count++;
>
> - pr_info("CMA: reserved %ld MiB at %08lx\n", (unsigned long)size / SZ_1M,
> - (unsigned long)base);
> + pr_info("%s(): reserved %ld MiB at %08lx\n",
> + __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
>
> /* Architecture specific contiguous memory fixup. */
> dma_contiguous_early_fixup(base, size);
> return 0;
> err:
> - pr_err("CMA: failed to reserve %ld MiB\n", (unsigned long)size / SZ_1M);
> + pr_err("%s(): failed to reserve %ld MiB\n",
> + __func__, (unsigned long)size / SZ_1M);
> return ret;
> }
>
>
--
Thanks.
Zhang Yanfei
On 06/12/2014 02:02 PM, Joonsoo Kim wrote:
> On Thu, Jun 12, 2014 at 02:25:43PM +0900, Minchan Kim wrote:
>> On Thu, Jun 12, 2014 at 12:21:39PM +0900, Joonsoo Kim wrote:
>>> We should free memory for bitmap when we find zone mis-match,
>>> otherwise this memory will leak.
>>
>> Then, -stable stuff?
>
> I don't think so. This is just possible leak candidate, so we don't
> need to push this to stable tree.
>
>>
>>>
>>> Additionally, I copy code comment from ppc kvm's cma code to notify
>>> why we need to check zone mis-match.
>>>
>>> Signed-off-by: Joonsoo Kim <[email protected]>
>>>
>>> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
>>> index bd0bb81..fb0cdce 100644
>>> --- a/drivers/base/dma-contiguous.c
>>> +++ b/drivers/base/dma-contiguous.c
>>> @@ -177,14 +177,24 @@ static int __init cma_activate_area(struct cma *cma)
>>> base_pfn = pfn;
>>> for (j = pageblock_nr_pages; j; --j, pfn++) {
>>> WARN_ON_ONCE(!pfn_valid(pfn));
>>> + /*
>>> + * alloc_contig_range requires the pfn range
>>> + * specified to be in the same zone. Make this
>>> + * simple by forcing the entire CMA resv range
>>> + * to be in the same zone.
>>> + */
>>> if (page_zone(pfn_to_page(pfn)) != zone)
>>> - return -EINVAL;
>>> + goto err;
>>
>> At a first glance, I thought it would be better to handle such error
>> before activating.
>> So when I see the registration code(ie, dma_contiguous_revere_area),
>> I realized it is impossible because we didn't set up zone yet. :(
>>
>> If so, when we detect to fail here, it would be better to report more
>> meaningful error message like what was successful zone and what is
>> new zone and failed pfn number?
>
> What I want to do in early phase of this patchset is to make cma code
> on DMA APIs similar to ppc kvm's cma code. ppc kvm's cma code already
> has this error handling logic, so I make this patch.
>
> If we think that we need more things, we can do that on general cma code
> after merging this patchset.
>
Yeah, I also like the idea. After all, this patchset aims to a general CMA
management, we could improve more after this patchset. So
Acked-by: Zhang Yanfei <[email protected]>
--
Thanks.
Zhang Yanfei
On 06/12/2014 11:21 AM, Joonsoo Kim wrote:
> ppc kvm's cma region management requires arbitrary bitmap granularity,
> since they want to reserve very large memory and manage this region
> with bitmap that one bit for several pages to reduce management overheads.
> So support arbitrary bitmap granularity for following generalization.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Acked-by: Zhang Yanfei <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index bc4c171..9bc9340 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -38,6 +38,7 @@ struct cma {
> unsigned long base_pfn;
> unsigned long count;
> unsigned long *bitmap;
> + int order_per_bit; /* Order of pages represented by one bit */
> struct mutex lock;
> };
>
> @@ -157,9 +158,38 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
>
> static DEFINE_MUTEX(cma_mutex);
>
> +static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
> +{
> + return (1 << (align_order >> cma->order_per_bit)) - 1;
> +}
> +
> +static unsigned long cma_bitmap_maxno(struct cma *cma)
> +{
> + return cma->count >> cma->order_per_bit;
> +}
> +
> +static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
> + unsigned long pages)
> +{
> + return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
> +}
> +
> +static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> +{
> + unsigned long bitmapno, nr_bits;
> +
> + bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + mutex_lock(&cma->lock);
> + bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> + mutex_unlock(&cma->lock);
> +}
> +
> static int __init cma_activate_area(struct cma *cma)
> {
> - int bitmap_size = BITS_TO_LONGS(cma->count) * sizeof(long);
> + int bitmap_maxno = cma_bitmap_maxno(cma);
> + int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> unsigned i = cma->count >> pageblock_order;
> struct zone *zone;
> @@ -221,6 +251,7 @@ core_initcall(cma_init_reserved_areas);
> * @base: Base address of the reserved area optional, use 0 for any
> * @limit: End address of the reserved memory (optional, 0 for any).
> * @alignment: Alignment for the contiguous memory area, should be power of 2
> + * @order_per_bit: Order of pages represented by one bit on bitmap.
> * @res_cma: Pointer to store the created cma region.
> * @fixed: hint about where to place the reserved area
> *
> @@ -235,7 +266,7 @@ core_initcall(cma_init_reserved_areas);
> */
> static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> - phys_addr_t alignment,
> + phys_addr_t alignment, int order_per_bit,
> struct cma **res_cma, bool fixed)
> {
> struct cma *cma = &cma_areas[cma_area_count];
> @@ -269,6 +300,8 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> base = ALIGN(base, alignment);
> size = ALIGN(size, alignment);
> limit &= ~(alignment - 1);
> + /* size should be aligned with order_per_bit */
> + BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
>
> /* Reserve memory */
> if (base && fixed) {
> @@ -294,6 +327,7 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> */
> cma->base_pfn = PFN_DOWN(base);
> cma->count = size >> PAGE_SHIFT;
> + cma->order_per_bit = order_per_bit;
> *res_cma = cma;
> cma_area_count++;
>
> @@ -313,7 +347,7 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> {
> int ret;
>
> - ret = __dma_contiguous_reserve_area(size, base, limit, 0,
> + ret = __dma_contiguous_reserve_area(size, base, limit, 0, 0,
> res_cma, fixed);
> if (ret)
> return ret;
> @@ -324,13 +358,6 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> return 0;
> }
>
> -static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> -{
> - mutex_lock(&cma->lock);
> - bitmap_clear(cma->bitmap, pfn - cma->base_pfn, count);
> - mutex_unlock(&cma->lock);
> -}
> -
> /**
> * dma_alloc_from_contiguous() - allocate pages from contiguous area
> * @dev: Pointer to device for which the allocation is performed.
> @@ -345,7 +372,8 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> unsigned int align)
> {
> - unsigned long mask, pfn, pageno, start = 0;
> + unsigned long mask, pfn, start = 0;
> + unsigned long bitmap_maxno, bitmapno, nr_bits;
> struct page *page = NULL;
> int ret;
>
> @@ -358,18 +386,19 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> if (!count)
> return NULL;
>
> - mask = (1 << align) - 1;
> -
> + mask = cma_bitmap_aligned_mask(cma, align);
> + bitmap_maxno = cma_bitmap_maxno(cma);
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
>
> for (;;) {
> mutex_lock(&cma->lock);
> - pageno = bitmap_find_next_zero_area(cma->bitmap, cma->count,
> - start, count, mask);
> - if (pageno >= cma->count) {
> + bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> + bitmap_maxno, start, nr_bits, mask);
> + if (bitmapno >= bitmap_maxno) {
> mutex_unlock(&cma->lock);
> break;
> }
> - bitmap_set(cma->bitmap, pageno, count);
> + bitmap_set(cma->bitmap, bitmapno, nr_bits);
> /*
> * It's safe to drop the lock here. We've marked this region for
> * our exclusive use. If the migration fails we will take the
> @@ -377,7 +406,7 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> */
> mutex_unlock(&cma->lock);
>
> - pfn = cma->base_pfn + pageno;
> + pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> mutex_lock(&cma_mutex);
> ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> mutex_unlock(&cma_mutex);
> @@ -392,7 +421,7 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> pr_debug("%s(): memory range at %p is busy, retrying\n",
> __func__, pfn_to_page(pfn));
> /* try again with a bit different memory target */
> - start = pageno + mask + 1;
> + start = bitmapno + mask + 1;
> }
>
> pr_debug("%s(): returned %p\n", __func__, page);
>
--
Thanks.
Zhang Yanfei
On 06/12/2014 11:21 AM, Joonsoo Kim wrote:
> Currently, there are two users on CMA functionality, one is the DMA
> subsystem and the other is the kvm on powerpc. They have their own code
> to manage CMA reserved area even if they looks really similar.
>>From my guess, it is caused by some needs on bitmap management. Kvm side
> wants to maintain bitmap not for 1 page, but for more size. Eventually it
> use bitmap where one bit represents 64 pages.
>
> When I implement CMA related patches, I should change those two places
> to apply my change and it seem to be painful to me. I want to change
> this situation and reduce future code management overhead through
> this patch.
>
> This change could also help developer who want to use CMA in their
> new feature development, since they can use CMA easily without
> copying & pasting this reserved area management code.
>
> In previous patches, we have prepared some features to generalize
> CMA reserved area management and now it's time to do it. This patch
> moves core functions to mm/cma.c and change DMA APIs to use
> these functions.
>
> There is no functional change in DMA APIs.
>
> v2: There is no big change from v1 in mm/cma.c. Mostly renaming.
>
> Acked-by: Michal Nazarewicz <[email protected]>
> Signed-off-by: Joonsoo Kim <[email protected]>
Acked-by: Zhang Yanfei <[email protected]>
>
> diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
> index 00e13ce..4eac559 100644
> --- a/drivers/base/Kconfig
> +++ b/drivers/base/Kconfig
> @@ -283,16 +283,6 @@ config CMA_ALIGNMENT
>
> If unsure, leave the default value "8".
>
> -config CMA_AREAS
> - int "Maximum count of the CMA device-private areas"
> - default 7
> - help
> - CMA allows to create CMA areas for particular devices. This parameter
> - sets the maximum number of such device private CMA areas in the
> - system.
> -
> - If unsure, leave the default value "7".
> -
> endif
>
> endmenu
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 9bc9340..f177f73 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -24,25 +24,10 @@
>
> #include <linux/memblock.h>
> #include <linux/err.h>
> -#include <linux/mm.h>
> -#include <linux/mutex.h>
> -#include <linux/page-isolation.h>
> #include <linux/sizes.h>
> -#include <linux/slab.h>
> -#include <linux/swap.h>
> -#include <linux/mm_types.h>
> #include <linux/dma-contiguous.h>
> #include <linux/log2.h>
> -
> -struct cma {
> - unsigned long base_pfn;
> - unsigned long count;
> - unsigned long *bitmap;
> - int order_per_bit; /* Order of pages represented by one bit */
> - struct mutex lock;
> -};
> -
> -struct cma *dma_contiguous_default_area;
> +#include <linux/cma.h>
>
> #ifdef CONFIG_CMA_SIZE_MBYTES
> #define CMA_SIZE_MBYTES CONFIG_CMA_SIZE_MBYTES
> @@ -50,6 +35,8 @@ struct cma *dma_contiguous_default_area;
> #define CMA_SIZE_MBYTES 0
> #endif
>
> +struct cma *dma_contiguous_default_area;
> +
> /*
> * Default global CMA area size can be defined in kernel's .config.
> * This is useful mainly for distro maintainers to create a kernel
> @@ -156,199 +143,13 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
> }
> }
>
> -static DEFINE_MUTEX(cma_mutex);
> -
> -static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
> -{
> - return (1 << (align_order >> cma->order_per_bit)) - 1;
> -}
> -
> -static unsigned long cma_bitmap_maxno(struct cma *cma)
> -{
> - return cma->count >> cma->order_per_bit;
> -}
> -
> -static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
> - unsigned long pages)
> -{
> - return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
> -}
> -
> -static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> -{
> - unsigned long bitmapno, nr_bits;
> -
> - bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> - nr_bits = cma_bitmap_pages_to_bits(cma, count);
> -
> - mutex_lock(&cma->lock);
> - bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> - mutex_unlock(&cma->lock);
> -}
> -
> -static int __init cma_activate_area(struct cma *cma)
> -{
> - int bitmap_maxno = cma_bitmap_maxno(cma);
> - int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> - unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> - unsigned i = cma->count >> pageblock_order;
> - struct zone *zone;
> -
> - pr_debug("%s()\n", __func__);
> -
> - cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> - if (!cma->bitmap)
> - return -ENOMEM;
> -
> - WARN_ON_ONCE(!pfn_valid(pfn));
> - zone = page_zone(pfn_to_page(pfn));
> -
> - do {
> - unsigned j;
> - base_pfn = pfn;
> - for (j = pageblock_nr_pages; j; --j, pfn++) {
> - WARN_ON_ONCE(!pfn_valid(pfn));
> - /*
> - * alloc_contig_range requires the pfn range
> - * specified to be in the same zone. Make this
> - * simple by forcing the entire CMA resv range
> - * to be in the same zone.
> - */
> - if (page_zone(pfn_to_page(pfn)) != zone)
> - goto err;
> - }
> - init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> - } while (--i);
> -
> - mutex_init(&cma->lock);
> - return 0;
> -
> -err:
> - kfree(cma->bitmap);
> - return -EINVAL;
> -}
> -
> -static struct cma cma_areas[MAX_CMA_AREAS];
> -static unsigned cma_area_count;
> -
> -static int __init cma_init_reserved_areas(void)
> -{
> - int i;
> -
> - for (i = 0; i < cma_area_count; i++) {
> - int ret = cma_activate_area(&cma_areas[i]);
> - if (ret)
> - return ret;
> - }
> -
> - return 0;
> -}
> -core_initcall(cma_init_reserved_areas);
> -
> -/**
> - * dma_contiguous_reserve_area() - reserve custom contiguous area
> - * @size: Size of the reserved area (in bytes),
> - * @base: Base address of the reserved area optional, use 0 for any
> - * @limit: End address of the reserved memory (optional, 0 for any).
> - * @alignment: Alignment for the contiguous memory area, should be power of 2
> - * @order_per_bit: Order of pages represented by one bit on bitmap.
> - * @res_cma: Pointer to store the created cma region.
> - * @fixed: hint about where to place the reserved area
> - *
> - * This function reserves memory from early allocator. It should be
> - * called by arch specific code once the early allocator (memblock or bootmem)
> - * has been activated and all other subsystems have already allocated/reserved
> - * memory. This function allows to create custom reserved areas for specific
> - * devices.
> - *
> - * If @fixed is true, reserve contiguous area at exactly @base. If false,
> - * reserve in range from @base to @limit.
> - */
> -static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> - phys_addr_t base, phys_addr_t limit,
> - phys_addr_t alignment, int order_per_bit,
> - struct cma **res_cma, bool fixed)
> -{
> - struct cma *cma = &cma_areas[cma_area_count];
> - int ret = 0;
> -
> - pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
> - __func__, (unsigned long)size, (unsigned long)base,
> - (unsigned long)limit, (unsigned long)alignment);
> -
> - /* Sanity checks */
> - if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> - pr_err("%s(): Not enough slots for CMA reserved regions!\n",
> - __func__);
> - return -ENOSPC;
> - }
> -
> - if (!size)
> - return -EINVAL;
> -
> - if (alignment && !is_power_of_2(alignment))
> - return -EINVAL;
> -
> - /*
> - * Sanitise input arguments.
> - * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
> - * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
> - * and CMA property will be broken.
> - */
> - alignment = max(alignment,
> - (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
> - base = ALIGN(base, alignment);
> - size = ALIGN(size, alignment);
> - limit &= ~(alignment - 1);
> - /* size should be aligned with order_per_bit */
> - BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
> -
> - /* Reserve memory */
> - if (base && fixed) {
> - if (memblock_is_region_reserved(base, size) ||
> - memblock_reserve(base, size) < 0) {
> - ret = -EBUSY;
> - goto err;
> - }
> - } else {
> - phys_addr_t addr = memblock_alloc_range(size, alignment, base,
> - limit);
> - if (!addr) {
> - ret = -ENOMEM;
> - goto err;
> - } else {
> - base = addr;
> - }
> - }
> -
> - /*
> - * Each reserved area must be initialised later, when more kernel
> - * subsystems (like slab allocator) are available.
> - */
> - cma->base_pfn = PFN_DOWN(base);
> - cma->count = size >> PAGE_SHIFT;
> - cma->order_per_bit = order_per_bit;
> - *res_cma = cma;
> - cma_area_count++;
> -
> - pr_info("%s(): reserved %ld MiB at %08lx\n",
> - __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
> - return 0;
> -
> -err:
> - pr_err("%s(): failed to reserve %ld MiB\n",
> - __func__, (unsigned long)size / SZ_1M);
> - return ret;
> -}
> -
> int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> phys_addr_t limit, struct cma **res_cma,
> bool fixed)
> {
> int ret;
>
> - ret = __dma_contiguous_reserve_area(size, base, limit, 0, 0,
> - res_cma, fixed);
> + ret = cma_declare_contiguous(size, base, limit, 0, 0, res_cma, fixed);
> if (ret)
> return ret;
>
> @@ -358,124 +159,17 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> return 0;
> }
>
> -/**
> - * dma_alloc_from_contiguous() - allocate pages from contiguous area
> - * @dev: Pointer to device for which the allocation is performed.
> - * @count: Requested number of pages.
> - * @align: Requested alignment of pages (in PAGE_SIZE order).
> - *
> - * This function allocates memory buffer for specified device. It uses
> - * device specific contiguous memory area if available or the default
> - * global one. Requires architecture specific dev_get_cma_area() helper
> - * function.
> - */
> -static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> - unsigned int align)
> -{
> - unsigned long mask, pfn, start = 0;
> - unsigned long bitmap_maxno, bitmapno, nr_bits;
> - struct page *page = NULL;
> - int ret;
> -
> - if (!cma || !cma->count)
> - return NULL;
> -
> - pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
> - count, align);
> -
> - if (!count)
> - return NULL;
> -
> - mask = cma_bitmap_aligned_mask(cma, align);
> - bitmap_maxno = cma_bitmap_maxno(cma);
> - nr_bits = cma_bitmap_pages_to_bits(cma, count);
> -
> - for (;;) {
> - mutex_lock(&cma->lock);
> - bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> - bitmap_maxno, start, nr_bits, mask);
> - if (bitmapno >= bitmap_maxno) {
> - mutex_unlock(&cma->lock);
> - break;
> - }
> - bitmap_set(cma->bitmap, bitmapno, nr_bits);
> - /*
> - * It's safe to drop the lock here. We've marked this region for
> - * our exclusive use. If the migration fails we will take the
> - * lock again and unmark it.
> - */
> - mutex_unlock(&cma->lock);
> -
> - pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> - mutex_lock(&cma_mutex);
> - ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> - mutex_unlock(&cma_mutex);
> - if (ret == 0) {
> - page = pfn_to_page(pfn);
> - break;
> - } else if (ret != -EBUSY) {
> - clear_cma_bitmap(cma, pfn, count);
> - break;
> - }
> - clear_cma_bitmap(cma, pfn, count);
> - pr_debug("%s(): memory range at %p is busy, retrying\n",
> - __func__, pfn_to_page(pfn));
> - /* try again with a bit different memory target */
> - start = bitmapno + mask + 1;
> - }
> -
> - pr_debug("%s(): returned %p\n", __func__, page);
> - return page;
> -}
> -
> struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> unsigned int align)
> {
> - struct cma *cma = dev_get_cma_area(dev);
> -
> if (align > CONFIG_CMA_ALIGNMENT)
> align = CONFIG_CMA_ALIGNMENT;
>
> - return __dma_alloc_from_contiguous(cma, count, align);
> -}
> -
> -/**
> - * dma_release_from_contiguous() - release allocated pages
> - * @dev: Pointer to device for which the pages were allocated.
> - * @pages: Allocated pages.
> - * @count: Number of allocated pages.
> - *
> - * This function releases memory allocated by dma_alloc_from_contiguous().
> - * It returns false when provided pages do not belong to contiguous area and
> - * true otherwise.
> - */
> -static bool __dma_release_from_contiguous(struct cma *cma, struct page *pages,
> - int count)
> -{
> - unsigned long pfn;
> -
> - if (!cma || !pages)
> - return false;
> -
> - pr_debug("%s(page %p)\n", __func__, (void *)pages);
> -
> - pfn = page_to_pfn(pages);
> -
> - if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count)
> - return false;
> -
> - VM_BUG_ON(pfn + count > cma->base_pfn + cma->count);
> -
> - free_contig_range(pfn, count);
> - clear_cma_bitmap(cma, pfn, count);
> -
> - return true;
> + return cma_alloc(dev_get_cma_area(dev), count, align);
> }
>
> bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> int count)
> {
> - struct cma *cma = dev_get_cma_area(dev);
> -
> - return __dma_release_from_contiguous(cma, pages, count);
> + return cma_release(dev_get_cma_area(dev), pages, count);
> }
> diff --git a/include/linux/cma.h b/include/linux/cma.h
> new file mode 100644
> index 0000000..e38efe9
> --- /dev/null
> +++ b/include/linux/cma.h
> @@ -0,0 +1,12 @@
> +#ifndef __CMA_H__
> +#define __CMA_H__
> +
> +struct cma;
> +
> +extern int __init cma_declare_contiguous(phys_addr_t size,
> + phys_addr_t base, phys_addr_t limit,
> + phys_addr_t alignment, int order_per_bit,
> + struct cma **res_cma, bool fixed);
> +extern struct page *cma_alloc(struct cma *cma, int count, unsigned int align);
> +extern bool cma_release(struct cma *cma, struct page *pages, int count);
> +#endif
> diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h
> index 772eab5..a40c1f3 100644
> --- a/include/linux/dma-contiguous.h
> +++ b/include/linux/dma-contiguous.h
> @@ -53,9 +53,10 @@
>
> #ifdef __KERNEL__
>
> +#include <linux/device.h>
> +
> struct cma;
> struct page;
> -struct device;
>
> #ifdef CONFIG_DMA_CMA
>
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 3e9977a..f4899ec 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -508,6 +508,17 @@ config CMA_DEBUG
> processing calls such as dma_alloc_from_contiguous().
> This option does not affect warning and error messages.
>
> +config CMA_AREAS
> + int "Maximum count of the CMA areas"
> + depends on CMA
> + default 7
> + help
> + CMA allows to create CMA areas for particular purpose, mainly,
> + used as device private area. This parameter sets the maximum
> + number of CMA area in the system.
> +
> + If unsure, leave the default value "7".
> +
> config ZBUD
> tristate
> default n
> diff --git a/mm/Makefile b/mm/Makefile
> index 1eaa70b..bc0422b 100644
> --- a/mm/Makefile
> +++ b/mm/Makefile
> @@ -62,3 +62,4 @@ obj-$(CONFIG_MEMORY_ISOLATION) += page_isolation.o
> obj-$(CONFIG_ZBUD) += zbud.o
> obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
> obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
> +obj-$(CONFIG_CMA) += cma.o
> diff --git a/mm/cma.c b/mm/cma.c
> new file mode 100644
> index 0000000..1e1b017
> --- /dev/null
> +++ b/mm/cma.c
> @@ -0,0 +1,330 @@
> +/*
> + * Contiguous Memory Allocator
> + *
> + * Copyright (c) 2010-2011 by Samsung Electronics.
> + * Copyright IBM Corporation, 2013
> + * Copyright LG Electronics Inc., 2014
> + * Written by:
> + * Marek Szyprowski <[email protected]>
> + * Michal Nazarewicz <[email protected]>
> + * Aneesh Kumar K.V <[email protected]>
> + * Joonsoo Kim <[email protected]>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License as
> + * published by the Free Software Foundation; either version 2 of the
> + * License or (at your optional) any later version of the license.
> + */
> +
> +#define pr_fmt(fmt) "cma: " fmt
> +
> +#ifdef CONFIG_CMA_DEBUG
> +#ifndef DEBUG
> +# define DEBUG
> +#endif
> +#endif
> +
> +#include <linux/memblock.h>
> +#include <linux/err.h>
> +#include <linux/mm.h>
> +#include <linux/mutex.h>
> +#include <linux/sizes.h>
> +#include <linux/slab.h>
> +
> +struct cma {
> + unsigned long base_pfn;
> + unsigned long count;
> + unsigned long *bitmap;
> + int order_per_bit; /* Order of pages represented by one bit */
> + struct mutex lock;
> +};
> +
> +/*
> + * There is always at least global CMA area and a few optional
> + * areas configured in kernel .config.
> + */
> +#define MAX_CMA_AREAS (1 + CONFIG_CMA_AREAS)
> +
> +static struct cma cma_areas[MAX_CMA_AREAS];
> +static unsigned cma_area_count;
> +static DEFINE_MUTEX(cma_mutex);
> +
> +static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
> +{
> + return (1 << (align_order >> cma->order_per_bit)) - 1;
> +}
> +
> +static unsigned long cma_bitmap_maxno(struct cma *cma)
> +{
> + return cma->count >> cma->order_per_bit;
> +}
> +
> +static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
> + unsigned long pages)
> +{
> + return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
> +}
> +
> +static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> +{
> + unsigned long bitmapno, nr_bits;
> +
> + bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + mutex_lock(&cma->lock);
> + bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> + mutex_unlock(&cma->lock);
> +}
> +
> +static int __init cma_activate_area(struct cma *cma)
> +{
> + int bitmap_maxno = cma_bitmap_maxno(cma);
> + int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> + unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> + unsigned i = cma->count >> pageblock_order;
> + struct zone *zone;
> +
> + pr_debug("%s()\n", __func__);
> +
> + cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> + if (!cma->bitmap)
> + return -ENOMEM;
> +
> + WARN_ON_ONCE(!pfn_valid(pfn));
> + zone = page_zone(pfn_to_page(pfn));
> +
> + do {
> + unsigned j;
> +
> + base_pfn = pfn;
> + for (j = pageblock_nr_pages; j; --j, pfn++) {
> + WARN_ON_ONCE(!pfn_valid(pfn));
> + /*
> + * alloc_contig_range requires the pfn range
> + * specified to be in the same zone. Make this
> + * simple by forcing the entire CMA resv range
> + * to be in the same zone.
> + */
> + if (page_zone(pfn_to_page(pfn)) != zone)
> + goto err;
> + }
> + init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> + } while (--i);
> +
> + mutex_init(&cma->lock);
> + return 0;
> +
> +err:
> + kfree(cma->bitmap);
> + return -EINVAL;
> +}
> +
> +static int __init cma_init_reserved_areas(void)
> +{
> + int i;
> +
> + for (i = 0; i < cma_area_count; i++) {
> + int ret = cma_activate_area(&cma_areas[i]);
> +
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> +core_initcall(cma_init_reserved_areas);
> +
> +/**
> + * cma_declare_contiguous() - reserve custom contiguous area
> + * @size: Size of the reserved area (in bytes),
> + * @base: Base address of the reserved area optional, use 0 for any
> + * @limit: End address of the reserved memory (optional, 0 for any).
> + * @alignment: Alignment for the contiguous memory area, should be power of 2
> + * @order_per_bit: Order of pages represented by one bit on bitmap.
> + * @res_cma: Pointer to store the created cma region.
> + * @fixed: hint about where to place the reserved area
> + *
> + * This function reserves memory from early allocator. It should be
> + * called by arch specific code once the early allocator (memblock or bootmem)
> + * has been activated and all other subsystems have already allocated/reserved
> + * memory. This function allows to create custom reserved areas.
> + *
> + * If @fixed is true, reserve contiguous area at exactly @base. If false,
> + * reserve in range from @base to @limit.
> + */
> +int __init cma_declare_contiguous(phys_addr_t size,
> + phys_addr_t base, phys_addr_t limit,
> + phys_addr_t alignment, int order_per_bit,
> + struct cma **res_cma, bool fixed)
> +{
> + struct cma *cma = &cma_areas[cma_area_count];
> + int ret = 0;
> +
> + pr_debug("%s(size %lx, base %08lx, limit %08lx alignment %08lx)\n",
> + __func__, (unsigned long)size, (unsigned long)base,
> + (unsigned long)limit, (unsigned long)alignment);
> +
> + /* Sanity checks */
> + if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> + pr_err("%s(): Not enough slots for CMA reserved regions!\n",
> + __func__);
> + return -ENOSPC;
> + }
> +
> + if (!size)
> + return -EINVAL;
> +
> + if (alignment && !is_power_of_2(alignment))
> + return -EINVAL;
> +
> + /*
> + * Sanitise input arguments.
> + * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
> + * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
> + * and CMA property will be broken.
> + */
> + alignment = max(alignment,
> + (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
> + base = ALIGN(base, alignment);
> + size = ALIGN(size, alignment);
> + limit &= ~(alignment - 1);
> + /* size should be aligned with order_per_bit */
> + BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
> +
> + /* Reserve memory */
> + if (base && fixed) {
> + if (memblock_is_region_reserved(base, size) ||
> + memblock_reserve(base, size) < 0) {
> + ret = -EBUSY;
> + goto err;
> + }
> + } else {
> + phys_addr_t addr = memblock_alloc_range(size, alignment, base,
> + limit);
> + if (!addr) {
> + ret = -ENOMEM;
> + goto err;
> + } else {
> + base = addr;
> + }
> + }
> +
> + /*
> + * Each reserved area must be initialised later, when more kernel
> + * subsystems (like slab allocator) are available.
> + */
> + cma->base_pfn = PFN_DOWN(base);
> + cma->count = size >> PAGE_SHIFT;
> + cma->order_per_bit = order_per_bit;
> + *res_cma = cma;
> + cma_area_count++;
> +
> + pr_info("%s(): reserved %ld MiB at %08lx\n",
> + __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
> + return 0;
> +
> +err:
> + pr_err("%s(): failed to reserve %ld MiB\n",
> + __func__, (unsigned long)size / SZ_1M);
> + return ret;
> +}
> +
> +/**
> + * cma_alloc() - allocate pages from contiguous area
> + * @cma: Contiguous memory region for which the allocation is performed.
> + * @count: Requested number of pages.
> + * @align: Requested alignment of pages (in PAGE_SIZE order).
> + *
> + * This function allocates part of contiguous memory on specific
> + * contiguous memory area.
> + */
> +struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
> +{
> + unsigned long mask, pfn, start = 0;
> + unsigned long bitmap_maxno, bitmapno, nr_bits;
> + struct page *page = NULL;
> + int ret;
> +
> + if (!cma || !cma->count)
> + return NULL;
> +
> + pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
> + count, align);
> +
> + if (!count)
> + return NULL;
> +
> + mask = cma_bitmap_aligned_mask(cma, align);
> + bitmap_maxno = cma_bitmap_maxno(cma);
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + for (;;) {
> + mutex_lock(&cma->lock);
> + bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> + bitmap_maxno, start, nr_bits, mask);
> + if (bitmapno >= bitmap_maxno) {
> + mutex_unlock(&cma->lock);
> + break;
> + }
> + bitmap_set(cma->bitmap, bitmapno, nr_bits);
> + /*
> + * It's safe to drop the lock here. We've marked this region for
> + * our exclusive use. If the migration fails we will take the
> + * lock again and unmark it.
> + */
> + mutex_unlock(&cma->lock);
> +
> + pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> + mutex_lock(&cma_mutex);
> + ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> + mutex_unlock(&cma_mutex);
> + if (ret == 0) {
> + page = pfn_to_page(pfn);
> + break;
> + } else if (ret != -EBUSY) {
> + clear_cma_bitmap(cma, pfn, count);
> + break;
> + }
> + clear_cma_bitmap(cma, pfn, count);
> + pr_debug("%s(): memory range at %p is busy, retrying\n",
> + __func__, pfn_to_page(pfn));
> + /* try again with a bit different memory target */
> + start = bitmapno + mask + 1;
> + }
> +
> + pr_debug("%s(): returned %p\n", __func__, page);
> + return page;
> +}
> +
> +/**
> + * cma_release() - release allocated pages
> + * @cma: Contiguous memory region for which the allocation is performed.
> + * @pages: Allocated pages.
> + * @count: Number of allocated pages.
> + *
> + * This function releases memory allocated by alloc_cma().
> + * It returns false when provided pages do not belong to contiguous area and
> + * true otherwise.
> + */
> +bool cma_release(struct cma *cma, struct page *pages, int count)
> +{
> + unsigned long pfn;
> +
> + if (!cma || !pages)
> + return false;
> +
> + pr_debug("%s(page %p)\n", __func__, (void *)pages);
> +
> + pfn = page_to_pfn(pages);
> +
> + if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count)
> + return false;
> +
> + VM_BUG_ON(pfn + count > cma->base_pfn + cma->count);
> +
> + free_contig_range(pfn, count);
> + clear_cma_bitmap(cma, pfn, count);
> +
> + return true;
> +}
>
--
Thanks.
Zhang Yanfei
On 06/12/2014 11:21 AM, Joonsoo Kim wrote:
> We can remove one call sites for clear_cma_bitmap() if we first
> call it before checking error number.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Reviewed-by: Zhang Yanfei <[email protected]>
>
> diff --git a/mm/cma.c b/mm/cma.c
> index 1e1b017..01a0713 100644
> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -282,11 +282,12 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
> if (ret == 0) {
> page = pfn_to_page(pfn);
> break;
> - } else if (ret != -EBUSY) {
> - clear_cma_bitmap(cma, pfn, count);
> - break;
> }
> +
> clear_cma_bitmap(cma, pfn, count);
> + if (ret != -EBUSY)
> + break;
> +
> pr_debug("%s(): memory range at %p is busy, retrying\n",
> __func__, pfn_to_page(pfn));
> /* try again with a bit different memory target */
>
--
Thanks.
Zhang Yanfei
>> Joonsoo Kim <[email protected]> writes:
>>
>> > We don't need explicit 'CMA:' prefix, since we already define prefix
>> > 'cma:' in pr_fmt. So remove it.
>> >
>> > And, some logs print function name and others doesn't. This looks
>> > bad to me, so I unify log format to print function name consistently.
>> >
>> > Lastly, I add one more debug log on cma_activate_area().
>> >
>> > Signed-off-by: Joonsoo Kim <[email protected]>
>> >
>> > diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
>> > index 83969f8..bd0bb81 100644
>> > --- a/drivers/base/dma-contiguous.c
>> > +++ b/drivers/base/dma-contiguous.c
>> > @@ -144,7 +144,7 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
>> > }
>> >
>> > if (selected_size && !dma_contiguous_default_area) {
>> > - pr_debug("%s: reserving %ld MiB for global area\n", __func__,
>> > + pr_debug("%s(): reserving %ld MiB for global area\n", __func__,
>> > (unsigned long)selected_size / SZ_1M);
> On Thu, Jun 12, 2014 at 10:11:19AM +0530, Aneesh Kumar K.V wrote:
>> Do we need to do function(), or just function:. I have seen the later
>> usage in other parts of the kernel.
On Thu, Jun 12 2014, Joonsoo Kim <[email protected]> wrote:
> I also haven't seen this format in other kernel code, but, in cma, they use
> this format as following.
>
> function(arg1, arg2, ...): some message
>
> If we all dislike this format, we can change it after merging this
> patchset. Until then, it seems better to me to leave it as is.
I used “function(arg1, arg2, …)” at the *beginning* of functions when
the arguments passed to the function were included in the message. In
all other cases I left it at just “function:” (or just no additional
prefix). IMO that's a reasonable strategy.
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz (o o)
ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo--
On Thu, Jun 12 2014, Joonsoo Kim <[email protected]> wrote:
> We don't need explicit 'CMA:' prefix, since we already define prefix
> 'cma:' in pr_fmt. So remove it.
>
> And, some logs print function name and others doesn't. This looks
> bad to me, so I unify log format to print function name consistently.
>
> Lastly, I add one more debug log on cma_activate_area().
I don't particularly care what format of logs you choose, so:
Acked-by: Michal Nazarewicz <[email protected]>
even though I'd go without empty “()”.
> Signed-off-by: Joonsoo Kim <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 83969f8..bd0bb81 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -144,7 +144,7 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
> }
>
> if (selected_size && !dma_contiguous_default_area) {
> - pr_debug("%s: reserving %ld MiB for global area\n", __func__,
> + pr_debug("%s(): reserving %ld MiB for global area\n", __func__,
> (unsigned long)selected_size / SZ_1M);
>
> dma_contiguous_reserve_area(selected_size, selected_base,
> @@ -163,8 +163,9 @@ static int __init cma_activate_area(struct cma *cma)
> unsigned i = cma->count >> pageblock_order;
> struct zone *zone;
>
> - cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> + pr_debug("%s()\n", __func__);
>
> + cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> if (!cma->bitmap)
> return -ENOMEM;
>
> @@ -234,7 +235,8 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
>
> /* Sanity checks */
> if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> - pr_err("Not enough slots for CMA reserved regions!\n");
> + pr_err("%s(): Not enough slots for CMA reserved regions!\n",
> + __func__);
> return -ENOSPC;
> }
>
> @@ -274,14 +276,15 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> *res_cma = cma;
> cma_area_count++;
>
> - pr_info("CMA: reserved %ld MiB at %08lx\n", (unsigned long)size / SZ_1M,
> - (unsigned long)base);
> + pr_info("%s(): reserved %ld MiB at %08lx\n",
> + __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
>
> /* Architecture specific contiguous memory fixup. */
> dma_contiguous_early_fixup(base, size);
> return 0;
> err:
> - pr_err("CMA: failed to reserve %ld MiB\n", (unsigned long)size / SZ_1M);
> + pr_err("%s(): failed to reserve %ld MiB\n",
> + __func__, (unsigned long)size / SZ_1M);
> return ret;
> }
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz (o o)
ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo--
On Thu, Jun 12 2014, Joonsoo Kim <[email protected]> wrote:
> We should free memory for bitmap when we find zone mis-match,
> otherwise this memory will leak.
>
> Additionally, I copy code comment from ppc kvm's cma code to notify
> why we need to check zone mis-match.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Reviewed-by: Michal Nazarewicz <[email protected]>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index bd0bb81..fb0cdce 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -177,14 +177,24 @@ static int __init cma_activate_area(struct cma *cma)
> base_pfn = pfn;
> for (j = pageblock_nr_pages; j; --j, pfn++) {
> WARN_ON_ONCE(!pfn_valid(pfn));
> + /*
> + * alloc_contig_range requires the pfn range
> + * specified to be in the same zone. Make this
> + * simple by forcing the entire CMA resv range
> + * to be in the same zone.
> + */
> if (page_zone(pfn_to_page(pfn)) != zone)
> - return -EINVAL;
> + goto err;
> }
> init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> } while (--i);
>
> mutex_init(&cma->lock);
> return 0;
> +
> +err:
> + kfree(cma->bitmap);
> + return -EINVAL;
> }
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz (o o)
ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo--
On Thu, Jun 12 2014, Michal Nazarewicz <[email protected]> wrote:
> I used “function(arg1, arg2, …)” at the *beginning* of functions when
> the arguments passed to the function were included in the message. In
> all other cases I left it at just “function:” (or just no additional
> prefix). IMO that's a reasonable strategy.
At closer inspection, I realised drivers/base/dma-contiguous.c is
Marek's code, but the above I think is still reasonable thing to do, so
I'd rather standardise on having “function(…)” only at the beginning of
a function. Just my 0.02 CHF.
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz (o o)
ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo--
On Thu, Jun 12 2014, Joonsoo Kim <[email protected]> wrote:
> To prepare future generalization work on cma area management code,
> we need to separate core cma management codes from DMA APIs.
> We will extend these core functions to cover requirements of
> ppc kvm's cma area management functionality in following patches.
> This separation helps us not to touch DMA APIs while extending
> core functions.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Acked-by: Michal Nazarewicz <[email protected]>
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz (o o)
ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo--
On Thu, Jun 12 2014, Joonsoo Kim <[email protected]> wrote:
> ppc kvm's cma area management needs alignment constraint on
I've noticed it earlier and cannot seem to get to terms with this. It
should IMO be PPC, KVM and CMA since those are acronyms. But if you
have strong feelings, it's not a big issue.
> cma region. So support it to prepare generalization of cma area
> management functionality.
>
> Additionally, add some comments which tell us why alignment
> constraint is needed on cma region.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Acked-by: Michal Nazarewicz <[email protected]>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 8a44c82..bc4c171 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -219,6 +220,7 @@ core_initcall(cma_init_reserved_areas);
> * @size: Size of the reserved area (in bytes),
> * @base: Base address of the reserved area optional, use 0 for any
> * @limit: End address of the reserved memory (optional, 0 for any).
> + * @alignment: Alignment for the contiguous memory area, should be
> power of 2
“must be power of 2 or zero”.
> * @res_cma: Pointer to store the created cma region.
> * @fixed: hint about where to place the reserved area
> *
> @@ -233,15 +235,15 @@ core_initcall(cma_init_reserved_areas);
> */
> static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> + phys_addr_t alignment,
> struct cma **res_cma, bool fixed)
> {
> struct cma *cma = &cma_areas[cma_area_count];
> - phys_addr_t alignment;
> int ret = 0;
>
> - pr_debug("%s(size %lx, base %08lx, limit %08lx)\n", __func__,
> - (unsigned long)size, (unsigned long)base,
> - (unsigned long)limit);
> + pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
> + __func__, (unsigned long)size, (unsigned long)base,
> + (unsigned long)limit, (unsigned long)alignment);
Nit: Align with the rest of the arguments, i.e.:
+ pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
+ __func__, (unsigned long)size, (unsigned long)base,
+ (unsigned long)limit, (unsigned long)alignment);
>
> /* Sanity checks */
> if (cma_area_count == ARRAY_SIZE(cma_areas)) {
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz (o o)
ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo--
On Thu, Jun 12 2014, Joonsoo Kim <[email protected]> wrote:
> ppc kvm's cma region management requires arbitrary bitmap granularity,
> since they want to reserve very large memory and manage this region
> with bitmap that one bit for several pages to reduce management overheads.
> So support arbitrary bitmap granularity for following generalization.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Acked-by: Michal Nazarewicz <[email protected]>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index bc4c171..9bc9340 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -38,6 +38,7 @@ struct cma {
> unsigned long base_pfn;
> unsigned long count;
Have you considered replacing count with maxno?
> unsigned long *bitmap;
> + int order_per_bit; /* Order of pages represented by one bit */
I'd make it unsigned.
> struct mutex lock;
> };
>
> +static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int
> count)
For consistency cma_clear_bitmap would make more sense I think. On the
other hand, you're just moving stuff around so perhaps renaming the
function at this point is not worth it any more.
> +{
> + unsigned long bitmapno, nr_bits;
> +
> + bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + mutex_lock(&cma->lock);
> + bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> + mutex_unlock(&cma->lock);
> +}
> +
> static int __init cma_activate_area(struct cma *cma)
> {
> - int bitmap_size = BITS_TO_LONGS(cma->count) * sizeof(long);
> + int bitmap_maxno = cma_bitmap_maxno(cma);
> + int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> unsigned i = cma->count >> pageblock_order;
> struct zone *zone;
bitmap_maxno is never used again, perhaps:
+ int bitmap_size = BITS_TO_LONGS(cma_bitmap_maxno(cma)) * sizeof(long);
instead? Up to you.
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz (o o)
ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo--
On Thu, Jun 12 2014, Joonsoo Kim <[email protected]> wrote:
> We can remove one call sites for clear_cma_bitmap() if we first
> call it before checking error number.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Reviewed-by: Michal Nazarewicz <[email protected]>
> diff --git a/mm/cma.c b/mm/cma.c
> index 1e1b017..01a0713 100644
> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -282,11 +282,12 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
> if (ret == 0) {
> page = pfn_to_page(pfn);
> break;
> - } else if (ret != -EBUSY) {
> - clear_cma_bitmap(cma, pfn, count);
> - break;
> }
> +
> clear_cma_bitmap(cma, pfn, count);
> + if (ret != -EBUSY)
> + break;
> +
> pr_debug("%s(): memory range at %p is busy, retrying\n",
> __func__, pfn_to_page(pfn));
> /* try again with a bit different memory target */
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz (o o)
ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo--
On Thu, Jun 12 2014, Joonsoo Kim <[email protected]> wrote:
> Conventionally, we put output param to the end of param list.
> cma_declare_contiguous() doesn't look like that, so change it.
Perhaps the function should be changed to return an error-pointer?
> Additionally, move down cma_areas reference code to the position
> where it is really needed.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Acked-by: Michal Nazarewicz <[email protected]>
>
> diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c
> index 28ec226..97613ea 100644
> --- a/arch/powerpc/kvm/book3s_hv_builtin.c
> +++ b/arch/powerpc/kvm/book3s_hv_builtin.c
> @@ -184,7 +184,7 @@ void __init kvm_cma_reserve(void)
>
> align_size = max(kvm_rma_pages << PAGE_SHIFT, align_size);
> cma_declare_contiguous(selected_size, 0, 0, align_size,
> - KVM_CMA_CHUNK_ORDER - PAGE_SHIFT, &kvm_cma, false);
> + KVM_CMA_CHUNK_ORDER - PAGE_SHIFT, false, &kvm_cma);
> }
> }
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index f177f73..bfd4553 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -149,7 +149,7 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> {
> int ret;
>
> - ret = cma_declare_contiguous(size, base, limit, 0, 0, res_cma, fixed);
> + ret = cma_declare_contiguous(size, base, limit, 0, 0, fixed, res_cma);
> if (ret)
> return ret;
>
> diff --git a/include/linux/cma.h b/include/linux/cma.h
> index e38efe9..e53eead 100644
> --- a/include/linux/cma.h
> +++ b/include/linux/cma.h
> @@ -6,7 +6,7 @@ struct cma;
> extern int __init cma_declare_contiguous(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> phys_addr_t alignment, int order_per_bit,
> - struct cma **res_cma, bool fixed);
> + bool fixed, struct cma **res_cma);
> extern struct page *cma_alloc(struct cma *cma, int count, unsigned int align);
> extern bool cma_release(struct cma *cma, struct page *pages, int count);
> #endif
> diff --git a/mm/cma.c b/mm/cma.c
> index 01a0713..22a5b23 100644
> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -142,8 +142,8 @@ core_initcall(cma_init_reserved_areas);
> * @limit: End address of the reserved memory (optional, 0 for any).
> * @alignment: Alignment for the contiguous memory area, should be power of 2
> * @order_per_bit: Order of pages represented by one bit on bitmap.
> - * @res_cma: Pointer to store the created cma region.
> * @fixed: hint about where to place the reserved area
> + * @res_cma: Pointer to store the created cma region.
> *
> * This function reserves memory from early allocator. It should be
> * called by arch specific code once the early allocator (memblock or bootmem)
> @@ -156,9 +156,9 @@ core_initcall(cma_init_reserved_areas);
> int __init cma_declare_contiguous(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> phys_addr_t alignment, int order_per_bit,
> - struct cma **res_cma, bool fixed)
> + bool fixed, struct cma **res_cma)
> {
> - struct cma *cma = &cma_areas[cma_area_count];
> + struct cma *cma;
> int ret = 0;
>
> pr_debug("%s(size %lx, base %08lx, limit %08lx alignment %08lx)\n",
> @@ -214,6 +214,7 @@ int __init cma_declare_contiguous(phys_addr_t size,
> * Each reserved area must be initialised later, when more kernel
> * subsystems (like slab allocator) are available.
> */
> + cma = &cma_areas[cma_area_count];
> cma->base_pfn = PFN_DOWN(base);
> cma->count = size >> PAGE_SHIFT;
> cma->order_per_bit = order_per_bit;
> --
> 1.7.9.5
>
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz (o o)
ooo +--<[email protected]>--<xmpp:[email protected]>--ooO--(_)--Ooo--
Joonsoo Kim <[email protected]> writes:
> We can remove one call sites for clear_cma_bitmap() if we first
> call it before checking error number.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Reviewed-by: Aneesh Kumar K.V <[email protected]>
>
> diff --git a/mm/cma.c b/mm/cma.c
> index 1e1b017..01a0713 100644
> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -282,11 +282,12 @@ struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
> if (ret == 0) {
> page = pfn_to_page(pfn);
> break;
> - } else if (ret != -EBUSY) {
> - clear_cma_bitmap(cma, pfn, count);
> - break;
> }
> +
> clear_cma_bitmap(cma, pfn, count);
> + if (ret != -EBUSY)
> + break;
> +
> pr_debug("%s(): memory range at %p is busy, retrying\n",
> __func__, pfn_to_page(pfn));
> /* try again with a bit different memory target */
> --
> 1.7.9.5
Joonsoo Kim <[email protected]> writes:
> Conventionally, we put output param to the end of param list.
> cma_declare_contiguous() doesn't look like that, so change it.
>
> Additionally, move down cma_areas reference code to the position
> where it is really needed.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Reviewed-by: Aneesh Kumar K.V <[email protected]>
>
> diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c
> index 28ec226..97613ea 100644
> --- a/arch/powerpc/kvm/book3s_hv_builtin.c
> +++ b/arch/powerpc/kvm/book3s_hv_builtin.c
> @@ -184,7 +184,7 @@ void __init kvm_cma_reserve(void)
>
> align_size = max(kvm_rma_pages << PAGE_SHIFT, align_size);
> cma_declare_contiguous(selected_size, 0, 0, align_size,
> - KVM_CMA_CHUNK_ORDER - PAGE_SHIFT, &kvm_cma, false);
> + KVM_CMA_CHUNK_ORDER - PAGE_SHIFT, false, &kvm_cma);
> }
> }
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index f177f73..bfd4553 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -149,7 +149,7 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> {
> int ret;
>
> - ret = cma_declare_contiguous(size, base, limit, 0, 0, res_cma, fixed);
> + ret = cma_declare_contiguous(size, base, limit, 0, 0, fixed, res_cma);
> if (ret)
> return ret;
>
> diff --git a/include/linux/cma.h b/include/linux/cma.h
> index e38efe9..e53eead 100644
> --- a/include/linux/cma.h
> +++ b/include/linux/cma.h
> @@ -6,7 +6,7 @@ struct cma;
> extern int __init cma_declare_contiguous(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> phys_addr_t alignment, int order_per_bit,
> - struct cma **res_cma, bool fixed);
> + bool fixed, struct cma **res_cma);
> extern struct page *cma_alloc(struct cma *cma, int count, unsigned int align);
> extern bool cma_release(struct cma *cma, struct page *pages, int count);
> #endif
> diff --git a/mm/cma.c b/mm/cma.c
> index 01a0713..22a5b23 100644
> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -142,8 +142,8 @@ core_initcall(cma_init_reserved_areas);
> * @limit: End address of the reserved memory (optional, 0 for any).
> * @alignment: Alignment for the contiguous memory area, should be power of 2
> * @order_per_bit: Order of pages represented by one bit on bitmap.
> - * @res_cma: Pointer to store the created cma region.
> * @fixed: hint about where to place the reserved area
> + * @res_cma: Pointer to store the created cma region.
> *
> * This function reserves memory from early allocator. It should be
> * called by arch specific code once the early allocator (memblock or bootmem)
> @@ -156,9 +156,9 @@ core_initcall(cma_init_reserved_areas);
> int __init cma_declare_contiguous(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> phys_addr_t alignment, int order_per_bit,
> - struct cma **res_cma, bool fixed)
> + bool fixed, struct cma **res_cma)
> {
> - struct cma *cma = &cma_areas[cma_area_count];
> + struct cma *cma;
> int ret = 0;
>
> pr_debug("%s(size %lx, base %08lx, limit %08lx alignment %08lx)\n",
> @@ -214,6 +214,7 @@ int __init cma_declare_contiguous(phys_addr_t size,
> * Each reserved area must be initialised later, when more kernel
> * subsystems (like slab allocator) are available.
> */
> + cma = &cma_areas[cma_area_count];
> cma->base_pfn = PFN_DOWN(base);
> cma->count = size >> PAGE_SHIFT;
> cma->order_per_bit = order_per_bit;
> --
> 1.7.9.5
Joonsoo Kim <[email protected]> writes:
> Currently, there are two users on CMA functionality, one is the DMA
> subsystem and the other is the kvm on powerpc. They have their own code
> to manage CMA reserved area even if they looks really similar.
> From my guess, it is caused by some needs on bitmap management. Kvm side
> wants to maintain bitmap not for 1 page, but for more size. Eventually it
> use bitmap where one bit represents 64 pages.
>
> When I implement CMA related patches, I should change those two places
> to apply my change and it seem to be painful to me. I want to change
> this situation and reduce future code management overhead through
> this patch.
>
> This change could also help developer who want to use CMA in their
> new feature development, since they can use CMA easily without
> copying & pasting this reserved area management code.
>
> v2:
> Although this patchset looks very different with v1, the end result,
> that is, mm/cma.c is same with v1's one. So I carry Ack to patch 6-7.
>
> Patch 1-5 prepare some features to cover ppc kvm's requirements.
> Patch 6-7 generalize CMA reserved area management code and change users
> to use it.
> Patch 8-10 clean-up minor things.
I wanted to test the ppc changes and found that the patch series doesn't apply
against v3.15 . Do you have a kernel tree which I can clone to test this
series ?
Joonsoo Kim <[email protected]> writes:
> Now, we have general CMA reserved area management framework,
> so use it for future maintainabilty. There is no functional change.
>
> Acked-by: Michal Nazarewicz <[email protected]>
> Acked-by: Paolo Bonzini <[email protected]>
> Signed-off-by: Joonsoo Kim <[email protected]>
Need this. We may want to keep the VM_BUG_ON by moving
KVM_CMA_CHUNK_ORDER around.
diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
index 8056107..1932e0e 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -37,8 +37,6 @@
#include <asm/ppc-opcode.h>
#include <asm/cputable.h>
-#include "book3s_hv_cma.h"
-
/* POWER7 has 10-bit LPIDs, PPC970 has 6-bit LPIDs */
#define MAX_LPID_970 63
@@ -64,7 +62,6 @@ long kvmppc_alloc_hpt(struct kvm *kvm, u32 *htab_orderp)
}
kvm->arch.hpt_cma_alloc = 0;
- VM_BUG_ON(order < KVM_CMA_CHUNK_ORDER);
page = kvm_alloc_hpt(1 << (order - PAGE_SHIFT));
if (page) {
hpt = (unsigned long)pfn_to_kaddr(page_to_pfn(page));
-aneesh
Joonsoo Kim <[email protected]> writes:
> Now, we have general CMA reserved area management framework,
> so use it for future maintainabilty. There is no functional change.
>
> Acked-by: Michal Nazarewicz <[email protected]>
> Acked-by: Paolo Bonzini <[email protected]>
> Signed-off-by: Joonsoo Kim <[email protected]>
>
> diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c
> index 7cde8a6..28ec226 100644
> --- a/arch/powerpc/kvm/book3s_hv_builtin.c
> +++ b/arch/powerpc/kvm/book3s_hv_builtin.c
> @@ -16,12 +16,14 @@
> #include <linux/init.h>
> #include <linux/memblock.h>
> #include <linux/sizes.h>
> +#include <linux/cma.h>
>
> #include <asm/cputable.h>
> #include <asm/kvm_ppc.h>
> #include <asm/kvm_book3s.h>
>
> -#include "book3s_hv_cma.h"
> +#define KVM_CMA_CHUNK_ORDER 18
> +
> /*
> * Hash page table alignment on newer cpus(CPU_FTR_ARCH_206)
> * should be power of 2.
> @@ -43,6 +45,8 @@ static unsigned long kvm_cma_resv_ratio = 5;
> unsigned long kvm_rma_pages = (1 << 27) >> PAGE_SHIFT; /* 128MB */
> EXPORT_SYMBOL_GPL(kvm_rma_pages);
>
> +static struct cma *kvm_cma;
> +
> /* Work out RMLS (real mode limit selector) field value for a given RMA size.
> Assumes POWER7 or PPC970. */
> static inline int lpcr_rmls(unsigned long rma_size)
> @@ -97,7 +101,7 @@ struct kvm_rma_info *kvm_alloc_rma()
> ri = kmalloc(sizeof(struct kvm_rma_info), GFP_KERNEL);
> if (!ri)
> return NULL;
> - page = kvm_alloc_cma(kvm_rma_pages, kvm_rma_pages);
> + page = cma_alloc(kvm_cma, kvm_rma_pages, get_order(kvm_rma_pages));
> if (!page)
> goto err_out;
> atomic_set(&ri->use_count, 1);
> @@ -112,7 +116,7 @@ EXPORT_SYMBOL_GPL(kvm_alloc_rma);
> void kvm_release_rma(struct kvm_rma_info *ri)
> {
> if (atomic_dec_and_test(&ri->use_count)) {
> - kvm_release_cma(pfn_to_page(ri->base_pfn), kvm_rma_pages);
> + cma_release(kvm_cma, pfn_to_page(ri->base_pfn), kvm_rma_pages);
> kfree(ri);
> }
> }
> @@ -134,13 +138,13 @@ struct page *kvm_alloc_hpt(unsigned long nr_pages)
> /* Old CPUs require HPT aligned on a multiple of its size */
> if (!cpu_has_feature(CPU_FTR_ARCH_206))
> align_pages = nr_pages;
> - return kvm_alloc_cma(nr_pages, align_pages);
> + return cma_alloc(kvm_cma, nr_pages, get_order(align_pages));
> }
> EXPORT_SYMBOL_GPL(kvm_alloc_hpt);
>
> void kvm_release_hpt(struct page *page, unsigned long nr_pages)
> {
> - kvm_release_cma(page, nr_pages);
> + cma_release(kvm_cma, page, nr_pages);
> }
> EXPORT_SYMBOL_GPL(kvm_release_hpt);
>
> @@ -179,7 +183,8 @@ void __init kvm_cma_reserve(void)
> align_size = HPT_ALIGN_PAGES << PAGE_SHIFT;
>
> align_size = max(kvm_rma_pages << PAGE_SHIFT, align_size);
> - kvm_cma_declare_contiguous(selected_size, align_size);
> + cma_declare_contiguous(selected_size, 0, 0, align_size,
> + KVM_CMA_CHUNK_ORDER - PAGE_SHIFT, &kvm_cma, false);
> }
> }
>
> diff --git a/arch/powerpc/kvm/book3s_hv_cma.c b/arch/powerpc/kvm/book3s_hv_cma.c
> deleted file mode 100644
> index d9d3d85..0000000
> --- a/arch/powerpc/kvm/book3s_hv_cma.c
> +++ /dev/null
> @@ -1,240 +0,0 @@
> -/*
> - * Contiguous Memory Allocator for ppc KVM hash pagetable based on CMA
> - * for DMA mapping framework
> - *
> - * Copyright IBM Corporation, 2013
> - * Author Aneesh Kumar K.V <[email protected]>
> - *
> - * This program is free software; you can redistribute it and/or
> - * modify it under the terms of the GNU General Public License as
> - * published by the Free Software Foundation; either version 2 of the
> - * License or (at your optional) any later version of the license.
> - *
> - */
> -#define pr_fmt(fmt) "kvm_cma: " fmt
> -
> -#ifdef CONFIG_CMA_DEBUG
> -#ifndef DEBUG
> -# define DEBUG
> -#endif
> -#endif
> -
> -#include <linux/memblock.h>
> -#include <linux/mutex.h>
> -#include <linux/sizes.h>
> -#include <linux/slab.h>
> -
> -#include "book3s_hv_cma.h"
> -
> -struct kvm_cma {
> - unsigned long base_pfn;
> - unsigned long count;
> - unsigned long *bitmap;
> -};
> -
> -static DEFINE_MUTEX(kvm_cma_mutex);
> -static struct kvm_cma kvm_cma_area;
> -
> -/**
> - * kvm_cma_declare_contiguous() - reserve area for contiguous memory handling
> - * for kvm hash pagetable
> - * @size: Size of the reserved memory.
> - * @alignment: Alignment for the contiguous memory area
> - *
> - * This function reserves memory for kvm cma area. It should be
> - * called by arch code when early allocator (memblock or bootmem)
> - * is still activate.
> - */
> -long __init kvm_cma_declare_contiguous(phys_addr_t size, phys_addr_t alignment)
> -{
> - long base_pfn;
> - phys_addr_t addr;
> - struct kvm_cma *cma = &kvm_cma_area;
> -
> - pr_debug("%s(size %lx)\n", __func__, (unsigned long)size);
> -
> - if (!size)
> - return -EINVAL;
> - /*
> - * Sanitise input arguments.
> - * We should be pageblock aligned for CMA.
> - */
> - alignment = max(alignment, (phys_addr_t)(PAGE_SIZE << pageblock_order));
> - size = ALIGN(size, alignment);
> - /*
> - * Reserve memory
> - * Use __memblock_alloc_base() since
> - * memblock_alloc_base() panic()s.
> - */
> - addr = __memblock_alloc_base(size, alignment, 0);
> - if (!addr) {
> - base_pfn = -ENOMEM;
> - goto err;
> - } else
> - base_pfn = PFN_DOWN(addr);
> -
> - /*
> - * Each reserved area must be initialised later, when more kernel
> - * subsystems (like slab allocator) are available.
> - */
> - cma->base_pfn = base_pfn;
> - cma->count = size >> PAGE_SHIFT;
> - pr_info("CMA: reserved %ld MiB\n", (unsigned long)size / SZ_1M);
> - return 0;
> -err:
> - pr_err("CMA: failed to reserve %ld MiB\n", (unsigned long)size / SZ_1M);
> - return base_pfn;
> -}
> -
> -/**
> - * kvm_alloc_cma() - allocate pages from contiguous area
> - * @nr_pages: Requested number of pages.
> - * @align_pages: Requested alignment in number of pages
> - *
> - * This function allocates memory buffer for hash pagetable.
> - */
> -struct page *kvm_alloc_cma(unsigned long nr_pages, unsigned long align_pages)
> -{
> - int ret;
> - struct page *page = NULL;
> - struct kvm_cma *cma = &kvm_cma_area;
> - unsigned long chunk_count, nr_chunk;
> - unsigned long mask, pfn, pageno, start = 0;
> -
> -
> - if (!cma || !cma->count)
> - return NULL;
> -
> - pr_debug("%s(cma %p, count %lu, align pages %lu)\n", __func__,
> - (void *)cma, nr_pages, align_pages);
> -
> - if (!nr_pages)
> - return NULL;
> - /*
> - * align mask with chunk size. The bit tracks pages in chunk size
> - */
> - VM_BUG_ON(!is_power_of_2(align_pages));
> - mask = (align_pages >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT)) - 1;
> - BUILD_BUG_ON(PAGE_SHIFT > KVM_CMA_CHUNK_ORDER);
> -
> - chunk_count = cma->count >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT);
> - nr_chunk = nr_pages >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT);
> -
> - mutex_lock(&kvm_cma_mutex);
> - for (;;) {
> - pageno = bitmap_find_next_zero_area(cma->bitmap, chunk_count,
> - start, nr_chunk, mask);
> - if (pageno >= chunk_count)
> - break;
> -
> - pfn = cma->base_pfn + (pageno << (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT));
> - ret = alloc_contig_range(pfn, pfn + nr_pages, MIGRATE_CMA);
> - if (ret == 0) {
> - bitmap_set(cma->bitmap, pageno, nr_chunk);
> - page = pfn_to_page(pfn);
> - memset(pfn_to_kaddr(pfn), 0, nr_pages <<
> PAGE_SHIFT);
The above memset is missing in the generic code. May be
diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
index 6f13ee6..8740b4c 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -75,6 +75,7 @@ long kvmppc_alloc_hpt(struct kvm *kvm, u32 *htab_orderp)
page = kvm_alloc_hpt(1 << (order - PAGE_SHIFT));
if (page) {
hpt = (unsigned long)pfn_to_kaddr(page_to_pfn(page));
+ memset((void *)hpt, 0, (1 << order));
kvm->arch.hpt_cma_alloc = 1;
} else
--order;
With that
Tested-by: Aneesh Kumar K.V <[email protected]>
-aneesh
Joonsoo Kim <[email protected]> writes:
> Currently, there are two users on CMA functionality, one is the DMA
> subsystem and the other is the kvm on powerpc. They have their own code
> to manage CMA reserved area even if they looks really similar.
> From my guess, it is caused by some needs on bitmap management. Kvm side
> wants to maintain bitmap not for 1 page, but for more size. Eventually it
> use bitmap where one bit represents 64 pages.
>
> When I implement CMA related patches, I should change those two places
> to apply my change and it seem to be painful to me. I want to change
> this situation and reduce future code management overhead through
> this patch.
>
> This change could also help developer who want to use CMA in their
> new feature development, since they can use CMA easily without
> copying & pasting this reserved area management code.
>
> In previous patches, we have prepared some features to generalize
> CMA reserved area management and now it's time to do it. This patch
> moves core functions to mm/cma.c and change DMA APIs to use
> these functions.
>
> There is no functional change in DMA APIs.
>
> v2: There is no big change from v1 in mm/cma.c. Mostly renaming.
>
> Acked-by: Michal Nazarewicz <[email protected]>
> Signed-off-by: Joonsoo Kim <[email protected]>
Reviewed-by: Aneesh Kumar K.V <[email protected]>
>
> diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
> index 00e13ce..4eac559 100644
> --- a/drivers/base/Kconfig
> +++ b/drivers/base/Kconfig
> @@ -283,16 +283,6 @@ config CMA_ALIGNMENT
>
> If unsure, leave the default value "8".
>
> -config CMA_AREAS
> - int "Maximum count of the CMA device-private areas"
> - default 7
> - help
> - CMA allows to create CMA areas for particular devices. This parameter
> - sets the maximum number of such device private CMA areas in the
> - system.
> -
> - If unsure, leave the default value "7".
> -
> endif
>
> endmenu
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 9bc9340..f177f73 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -24,25 +24,10 @@
>
> #include <linux/memblock.h>
> #include <linux/err.h>
> -#include <linux/mm.h>
> -#include <linux/mutex.h>
> -#include <linux/page-isolation.h>
> #include <linux/sizes.h>
> -#include <linux/slab.h>
> -#include <linux/swap.h>
> -#include <linux/mm_types.h>
> #include <linux/dma-contiguous.h>
> #include <linux/log2.h>
> -
> -struct cma {
> - unsigned long base_pfn;
> - unsigned long count;
> - unsigned long *bitmap;
> - int order_per_bit; /* Order of pages represented by one bit */
> - struct mutex lock;
> -};
> -
> -struct cma *dma_contiguous_default_area;
> +#include <linux/cma.h>
>
> #ifdef CONFIG_CMA_SIZE_MBYTES
> #define CMA_SIZE_MBYTES CONFIG_CMA_SIZE_MBYTES
> @@ -50,6 +35,8 @@ struct cma *dma_contiguous_default_area;
> #define CMA_SIZE_MBYTES 0
> #endif
>
> +struct cma *dma_contiguous_default_area;
> +
> /*
> * Default global CMA area size can be defined in kernel's .config.
> * This is useful mainly for distro maintainers to create a kernel
> @@ -156,199 +143,13 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
> }
> }
>
> -static DEFINE_MUTEX(cma_mutex);
> -
> -static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
> -{
> - return (1 << (align_order >> cma->order_per_bit)) - 1;
> -}
> -
> -static unsigned long cma_bitmap_maxno(struct cma *cma)
> -{
> - return cma->count >> cma->order_per_bit;
> -}
> -
> -static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
> - unsigned long pages)
> -{
> - return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
> -}
> -
> -static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> -{
> - unsigned long bitmapno, nr_bits;
> -
> - bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> - nr_bits = cma_bitmap_pages_to_bits(cma, count);
> -
> - mutex_lock(&cma->lock);
> - bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> - mutex_unlock(&cma->lock);
> -}
> -
> -static int __init cma_activate_area(struct cma *cma)
> -{
> - int bitmap_maxno = cma_bitmap_maxno(cma);
> - int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> - unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> - unsigned i = cma->count >> pageblock_order;
> - struct zone *zone;
> -
> - pr_debug("%s()\n", __func__);
> -
> - cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> - if (!cma->bitmap)
> - return -ENOMEM;
> -
> - WARN_ON_ONCE(!pfn_valid(pfn));
> - zone = page_zone(pfn_to_page(pfn));
> -
> - do {
> - unsigned j;
> - base_pfn = pfn;
> - for (j = pageblock_nr_pages; j; --j, pfn++) {
> - WARN_ON_ONCE(!pfn_valid(pfn));
> - /*
> - * alloc_contig_range requires the pfn range
> - * specified to be in the same zone. Make this
> - * simple by forcing the entire CMA resv range
> - * to be in the same zone.
> - */
> - if (page_zone(pfn_to_page(pfn)) != zone)
> - goto err;
> - }
> - init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> - } while (--i);
> -
> - mutex_init(&cma->lock);
> - return 0;
> -
> -err:
> - kfree(cma->bitmap);
> - return -EINVAL;
> -}
> -
> -static struct cma cma_areas[MAX_CMA_AREAS];
> -static unsigned cma_area_count;
> -
> -static int __init cma_init_reserved_areas(void)
> -{
> - int i;
> -
> - for (i = 0; i < cma_area_count; i++) {
> - int ret = cma_activate_area(&cma_areas[i]);
> - if (ret)
> - return ret;
> - }
> -
> - return 0;
> -}
> -core_initcall(cma_init_reserved_areas);
> -
> -/**
> - * dma_contiguous_reserve_area() - reserve custom contiguous area
> - * @size: Size of the reserved area (in bytes),
> - * @base: Base address of the reserved area optional, use 0 for any
> - * @limit: End address of the reserved memory (optional, 0 for any).
> - * @alignment: Alignment for the contiguous memory area, should be power of 2
> - * @order_per_bit: Order of pages represented by one bit on bitmap.
> - * @res_cma: Pointer to store the created cma region.
> - * @fixed: hint about where to place the reserved area
> - *
> - * This function reserves memory from early allocator. It should be
> - * called by arch specific code once the early allocator (memblock or bootmem)
> - * has been activated and all other subsystems have already allocated/reserved
> - * memory. This function allows to create custom reserved areas for specific
> - * devices.
> - *
> - * If @fixed is true, reserve contiguous area at exactly @base. If false,
> - * reserve in range from @base to @limit.
> - */
> -static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> - phys_addr_t base, phys_addr_t limit,
> - phys_addr_t alignment, int order_per_bit,
> - struct cma **res_cma, bool fixed)
> -{
> - struct cma *cma = &cma_areas[cma_area_count];
> - int ret = 0;
> -
> - pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
> - __func__, (unsigned long)size, (unsigned long)base,
> - (unsigned long)limit, (unsigned long)alignment);
> -
> - /* Sanity checks */
> - if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> - pr_err("%s(): Not enough slots for CMA reserved regions!\n",
> - __func__);
> - return -ENOSPC;
> - }
> -
> - if (!size)
> - return -EINVAL;
> -
> - if (alignment && !is_power_of_2(alignment))
> - return -EINVAL;
> -
> - /*
> - * Sanitise input arguments.
> - * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
> - * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
> - * and CMA property will be broken.
> - */
> - alignment = max(alignment,
> - (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
> - base = ALIGN(base, alignment);
> - size = ALIGN(size, alignment);
> - limit &= ~(alignment - 1);
> - /* size should be aligned with order_per_bit */
> - BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
> -
> - /* Reserve memory */
> - if (base && fixed) {
> - if (memblock_is_region_reserved(base, size) ||
> - memblock_reserve(base, size) < 0) {
> - ret = -EBUSY;
> - goto err;
> - }
> - } else {
> - phys_addr_t addr = memblock_alloc_range(size, alignment, base,
> - limit);
> - if (!addr) {
> - ret = -ENOMEM;
> - goto err;
> - } else {
> - base = addr;
> - }
> - }
> -
> - /*
> - * Each reserved area must be initialised later, when more kernel
> - * subsystems (like slab allocator) are available.
> - */
> - cma->base_pfn = PFN_DOWN(base);
> - cma->count = size >> PAGE_SHIFT;
> - cma->order_per_bit = order_per_bit;
> - *res_cma = cma;
> - cma_area_count++;
> -
> - pr_info("%s(): reserved %ld MiB at %08lx\n",
> - __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
> - return 0;
> -
> -err:
> - pr_err("%s(): failed to reserve %ld MiB\n",
> - __func__, (unsigned long)size / SZ_1M);
> - return ret;
> -}
> -
> int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> phys_addr_t limit, struct cma **res_cma,
> bool fixed)
> {
> int ret;
>
> - ret = __dma_contiguous_reserve_area(size, base, limit, 0, 0,
> - res_cma, fixed);
> + ret = cma_declare_contiguous(size, base, limit, 0, 0, res_cma, fixed);
> if (ret)
> return ret;
>
> @@ -358,124 +159,17 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> return 0;
> }
>
> -/**
> - * dma_alloc_from_contiguous() - allocate pages from contiguous area
> - * @dev: Pointer to device for which the allocation is performed.
> - * @count: Requested number of pages.
> - * @align: Requested alignment of pages (in PAGE_SIZE order).
> - *
> - * This function allocates memory buffer for specified device. It uses
> - * device specific contiguous memory area if available or the default
> - * global one. Requires architecture specific dev_get_cma_area() helper
> - * function.
> - */
> -static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> - unsigned int align)
> -{
> - unsigned long mask, pfn, start = 0;
> - unsigned long bitmap_maxno, bitmapno, nr_bits;
> - struct page *page = NULL;
> - int ret;
> -
> - if (!cma || !cma->count)
> - return NULL;
> -
> - pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
> - count, align);
> -
> - if (!count)
> - return NULL;
> -
> - mask = cma_bitmap_aligned_mask(cma, align);
> - bitmap_maxno = cma_bitmap_maxno(cma);
> - nr_bits = cma_bitmap_pages_to_bits(cma, count);
> -
> - for (;;) {
> - mutex_lock(&cma->lock);
> - bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> - bitmap_maxno, start, nr_bits, mask);
> - if (bitmapno >= bitmap_maxno) {
> - mutex_unlock(&cma->lock);
> - break;
> - }
> - bitmap_set(cma->bitmap, bitmapno, nr_bits);
> - /*
> - * It's safe to drop the lock here. We've marked this region for
> - * our exclusive use. If the migration fails we will take the
> - * lock again and unmark it.
> - */
> - mutex_unlock(&cma->lock);
> -
> - pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> - mutex_lock(&cma_mutex);
> - ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> - mutex_unlock(&cma_mutex);
> - if (ret == 0) {
> - page = pfn_to_page(pfn);
> - break;
> - } else if (ret != -EBUSY) {
> - clear_cma_bitmap(cma, pfn, count);
> - break;
> - }
> - clear_cma_bitmap(cma, pfn, count);
> - pr_debug("%s(): memory range at %p is busy, retrying\n",
> - __func__, pfn_to_page(pfn));
> - /* try again with a bit different memory target */
> - start = bitmapno + mask + 1;
> - }
> -
> - pr_debug("%s(): returned %p\n", __func__, page);
> - return page;
> -}
> -
> struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> unsigned int align)
> {
> - struct cma *cma = dev_get_cma_area(dev);
> -
> if (align > CONFIG_CMA_ALIGNMENT)
> align = CONFIG_CMA_ALIGNMENT;
>
> - return __dma_alloc_from_contiguous(cma, count, align);
> -}
> -
> -/**
> - * dma_release_from_contiguous() - release allocated pages
> - * @dev: Pointer to device for which the pages were allocated.
> - * @pages: Allocated pages.
> - * @count: Number of allocated pages.
> - *
> - * This function releases memory allocated by dma_alloc_from_contiguous().
> - * It returns false when provided pages do not belong to contiguous area and
> - * true otherwise.
> - */
> -static bool __dma_release_from_contiguous(struct cma *cma, struct page *pages,
> - int count)
> -{
> - unsigned long pfn;
> -
> - if (!cma || !pages)
> - return false;
> -
> - pr_debug("%s(page %p)\n", __func__, (void *)pages);
> -
> - pfn = page_to_pfn(pages);
> -
> - if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count)
> - return false;
> -
> - VM_BUG_ON(pfn + count > cma->base_pfn + cma->count);
> -
> - free_contig_range(pfn, count);
> - clear_cma_bitmap(cma, pfn, count);
> -
> - return true;
> + return cma_alloc(dev_get_cma_area(dev), count, align);
> }
>
> bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> int count)
> {
> - struct cma *cma = dev_get_cma_area(dev);
> -
> - return __dma_release_from_contiguous(cma, pages, count);
> + return cma_release(dev_get_cma_area(dev), pages, count);
> }
> diff --git a/include/linux/cma.h b/include/linux/cma.h
> new file mode 100644
> index 0000000..e38efe9
> --- /dev/null
> +++ b/include/linux/cma.h
> @@ -0,0 +1,12 @@
> +#ifndef __CMA_H__
> +#define __CMA_H__
> +
> +struct cma;
> +
> +extern int __init cma_declare_contiguous(phys_addr_t size,
> + phys_addr_t base, phys_addr_t limit,
> + phys_addr_t alignment, int order_per_bit,
> + struct cma **res_cma, bool fixed);
> +extern struct page *cma_alloc(struct cma *cma, int count, unsigned int align);
> +extern bool cma_release(struct cma *cma, struct page *pages, int count);
> +#endif
> diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h
> index 772eab5..a40c1f3 100644
> --- a/include/linux/dma-contiguous.h
> +++ b/include/linux/dma-contiguous.h
> @@ -53,9 +53,10 @@
>
> #ifdef __KERNEL__
>
> +#include <linux/device.h>
> +
> struct cma;
> struct page;
> -struct device;
>
> #ifdef CONFIG_DMA_CMA
>
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 3e9977a..f4899ec 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -508,6 +508,17 @@ config CMA_DEBUG
> processing calls such as dma_alloc_from_contiguous().
> This option does not affect warning and error messages.
>
> +config CMA_AREAS
> + int "Maximum count of the CMA areas"
> + depends on CMA
> + default 7
> + help
> + CMA allows to create CMA areas for particular purpose, mainly,
> + used as device private area. This parameter sets the maximum
> + number of CMA area in the system.
> +
> + If unsure, leave the default value "7".
> +
> config ZBUD
> tristate
> default n
> diff --git a/mm/Makefile b/mm/Makefile
> index 1eaa70b..bc0422b 100644
> --- a/mm/Makefile
> +++ b/mm/Makefile
> @@ -62,3 +62,4 @@ obj-$(CONFIG_MEMORY_ISOLATION) += page_isolation.o
> obj-$(CONFIG_ZBUD) += zbud.o
> obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
> obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
> +obj-$(CONFIG_CMA) += cma.o
> diff --git a/mm/cma.c b/mm/cma.c
> new file mode 100644
> index 0000000..1e1b017
> --- /dev/null
> +++ b/mm/cma.c
> @@ -0,0 +1,330 @@
> +/*
> + * Contiguous Memory Allocator
> + *
> + * Copyright (c) 2010-2011 by Samsung Electronics.
> + * Copyright IBM Corporation, 2013
> + * Copyright LG Electronics Inc., 2014
> + * Written by:
> + * Marek Szyprowski <[email protected]>
> + * Michal Nazarewicz <[email protected]>
> + * Aneesh Kumar K.V <[email protected]>
> + * Joonsoo Kim <[email protected]>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License as
> + * published by the Free Software Foundation; either version 2 of the
> + * License or (at your optional) any later version of the license.
> + */
> +
> +#define pr_fmt(fmt) "cma: " fmt
> +
> +#ifdef CONFIG_CMA_DEBUG
> +#ifndef DEBUG
> +# define DEBUG
> +#endif
> +#endif
> +
> +#include <linux/memblock.h>
> +#include <linux/err.h>
> +#include <linux/mm.h>
> +#include <linux/mutex.h>
> +#include <linux/sizes.h>
> +#include <linux/slab.h>
> +
> +struct cma {
> + unsigned long base_pfn;
> + unsigned long count;
> + unsigned long *bitmap;
> + int order_per_bit; /* Order of pages represented by one bit */
> + struct mutex lock;
> +};
> +
> +/*
> + * There is always at least global CMA area and a few optional
> + * areas configured in kernel .config.
> + */
> +#define MAX_CMA_AREAS (1 + CONFIG_CMA_AREAS)
> +
> +static struct cma cma_areas[MAX_CMA_AREAS];
> +static unsigned cma_area_count;
> +static DEFINE_MUTEX(cma_mutex);
> +
> +static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
> +{
> + return (1 << (align_order >> cma->order_per_bit)) - 1;
> +}
> +
> +static unsigned long cma_bitmap_maxno(struct cma *cma)
> +{
> + return cma->count >> cma->order_per_bit;
> +}
> +
> +static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
> + unsigned long pages)
> +{
> + return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
> +}
> +
> +static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> +{
> + unsigned long bitmapno, nr_bits;
> +
> + bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + mutex_lock(&cma->lock);
> + bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> + mutex_unlock(&cma->lock);
> +}
> +
> +static int __init cma_activate_area(struct cma *cma)
> +{
> + int bitmap_maxno = cma_bitmap_maxno(cma);
> + int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> + unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> + unsigned i = cma->count >> pageblock_order;
> + struct zone *zone;
> +
> + pr_debug("%s()\n", __func__);
> +
> + cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> + if (!cma->bitmap)
> + return -ENOMEM;
> +
> + WARN_ON_ONCE(!pfn_valid(pfn));
> + zone = page_zone(pfn_to_page(pfn));
> +
> + do {
> + unsigned j;
> +
> + base_pfn = pfn;
> + for (j = pageblock_nr_pages; j; --j, pfn++) {
> + WARN_ON_ONCE(!pfn_valid(pfn));
> + /*
> + * alloc_contig_range requires the pfn range
> + * specified to be in the same zone. Make this
> + * simple by forcing the entire CMA resv range
> + * to be in the same zone.
> + */
> + if (page_zone(pfn_to_page(pfn)) != zone)
> + goto err;
> + }
> + init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> + } while (--i);
> +
> + mutex_init(&cma->lock);
> + return 0;
> +
> +err:
> + kfree(cma->bitmap);
> + return -EINVAL;
> +}
> +
> +static int __init cma_init_reserved_areas(void)
> +{
> + int i;
> +
> + for (i = 0; i < cma_area_count; i++) {
> + int ret = cma_activate_area(&cma_areas[i]);
> +
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> +core_initcall(cma_init_reserved_areas);
> +
> +/**
> + * cma_declare_contiguous() - reserve custom contiguous area
> + * @size: Size of the reserved area (in bytes),
> + * @base: Base address of the reserved area optional, use 0 for any
> + * @limit: End address of the reserved memory (optional, 0 for any).
> + * @alignment: Alignment for the contiguous memory area, should be power of 2
> + * @order_per_bit: Order of pages represented by one bit on bitmap.
> + * @res_cma: Pointer to store the created cma region.
> + * @fixed: hint about where to place the reserved area
> + *
> + * This function reserves memory from early allocator. It should be
> + * called by arch specific code once the early allocator (memblock or bootmem)
> + * has been activated and all other subsystems have already allocated/reserved
> + * memory. This function allows to create custom reserved areas.
> + *
> + * If @fixed is true, reserve contiguous area at exactly @base. If false,
> + * reserve in range from @base to @limit.
> + */
> +int __init cma_declare_contiguous(phys_addr_t size,
> + phys_addr_t base, phys_addr_t limit,
> + phys_addr_t alignment, int order_per_bit,
> + struct cma **res_cma, bool fixed)
> +{
> + struct cma *cma = &cma_areas[cma_area_count];
> + int ret = 0;
> +
> + pr_debug("%s(size %lx, base %08lx, limit %08lx alignment %08lx)\n",
> + __func__, (unsigned long)size, (unsigned long)base,
> + (unsigned long)limit, (unsigned long)alignment);
> +
> + /* Sanity checks */
> + if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> + pr_err("%s(): Not enough slots for CMA reserved regions!\n",
> + __func__);
> + return -ENOSPC;
> + }
> +
> + if (!size)
> + return -EINVAL;
> +
> + if (alignment && !is_power_of_2(alignment))
> + return -EINVAL;
> +
> + /*
> + * Sanitise input arguments.
> + * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
> + * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
> + * and CMA property will be broken.
> + */
> + alignment = max(alignment,
> + (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
> + base = ALIGN(base, alignment);
> + size = ALIGN(size, alignment);
> + limit &= ~(alignment - 1);
> + /* size should be aligned with order_per_bit */
> + BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
> +
> + /* Reserve memory */
> + if (base && fixed) {
> + if (memblock_is_region_reserved(base, size) ||
> + memblock_reserve(base, size) < 0) {
> + ret = -EBUSY;
> + goto err;
> + }
> + } else {
> + phys_addr_t addr = memblock_alloc_range(size, alignment, base,
> + limit);
> + if (!addr) {
> + ret = -ENOMEM;
> + goto err;
> + } else {
> + base = addr;
> + }
> + }
> +
> + /*
> + * Each reserved area must be initialised later, when more kernel
> + * subsystems (like slab allocator) are available.
> + */
> + cma->base_pfn = PFN_DOWN(base);
> + cma->count = size >> PAGE_SHIFT;
> + cma->order_per_bit = order_per_bit;
> + *res_cma = cma;
> + cma_area_count++;
> +
> + pr_info("%s(): reserved %ld MiB at %08lx\n",
> + __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
> + return 0;
> +
> +err:
> + pr_err("%s(): failed to reserve %ld MiB\n",
> + __func__, (unsigned long)size / SZ_1M);
> + return ret;
> +}
> +
> +/**
> + * cma_alloc() - allocate pages from contiguous area
> + * @cma: Contiguous memory region for which the allocation is performed.
> + * @count: Requested number of pages.
> + * @align: Requested alignment of pages (in PAGE_SIZE order).
> + *
> + * This function allocates part of contiguous memory on specific
> + * contiguous memory area.
> + */
> +struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
> +{
> + unsigned long mask, pfn, start = 0;
> + unsigned long bitmap_maxno, bitmapno, nr_bits;
> + struct page *page = NULL;
> + int ret;
> +
> + if (!cma || !cma->count)
> + return NULL;
> +
> + pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
> + count, align);
> +
> + if (!count)
> + return NULL;
> +
> + mask = cma_bitmap_aligned_mask(cma, align);
> + bitmap_maxno = cma_bitmap_maxno(cma);
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + for (;;) {
> + mutex_lock(&cma->lock);
> + bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> + bitmap_maxno, start, nr_bits, mask);
> + if (bitmapno >= bitmap_maxno) {
> + mutex_unlock(&cma->lock);
> + break;
> + }
> + bitmap_set(cma->bitmap, bitmapno, nr_bits);
> + /*
> + * It's safe to drop the lock here. We've marked this region for
> + * our exclusive use. If the migration fails we will take the
> + * lock again and unmark it.
> + */
> + mutex_unlock(&cma->lock);
> +
> + pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> + mutex_lock(&cma_mutex);
> + ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> + mutex_unlock(&cma_mutex);
> + if (ret == 0) {
> + page = pfn_to_page(pfn);
> + break;
> + } else if (ret != -EBUSY) {
> + clear_cma_bitmap(cma, pfn, count);
> + break;
> + }
> + clear_cma_bitmap(cma, pfn, count);
> + pr_debug("%s(): memory range at %p is busy, retrying\n",
> + __func__, pfn_to_page(pfn));
> + /* try again with a bit different memory target */
> + start = bitmapno + mask + 1;
> + }
> +
> + pr_debug("%s(): returned %p\n", __func__, page);
> + return page;
> +}
> +
> +/**
> + * cma_release() - release allocated pages
> + * @cma: Contiguous memory region for which the allocation is performed.
> + * @pages: Allocated pages.
> + * @count: Number of allocated pages.
> + *
> + * This function releases memory allocated by alloc_cma().
> + * It returns false when provided pages do not belong to contiguous area and
> + * true otherwise.
> + */
> +bool cma_release(struct cma *cma, struct page *pages, int count)
> +{
> + unsigned long pfn;
> +
> + if (!cma || !pages)
> + return false;
> +
> + pr_debug("%s(page %p)\n", __func__, (void *)pages);
> +
> + pfn = page_to_pfn(pages);
> +
> + if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count)
> + return false;
> +
> + VM_BUG_ON(pfn + count > cma->base_pfn + cma->count);
> +
> + free_contig_range(pfn, count);
> + clear_cma_bitmap(cma, pfn, count);
> +
> + return true;
> +}
> --
> 1.7.9.5
Joonsoo Kim <[email protected]> writes:
> Currently, there are two users on CMA functionality, one is the DMA
> subsystem and the other is the kvm on powerpc. They have their own code
> to manage CMA reserved area even if they looks really similar.
> From my guess, it is caused by some needs on bitmap management. Kvm side
> wants to maintain bitmap not for 1 page, but for more size. Eventually it
> use bitmap where one bit represents 64 pages.
>
> When I implement CMA related patches, I should change those two places
> to apply my change and it seem to be painful to me. I want to change
> this situation and reduce future code management overhead through
> this patch.
>
> This change could also help developer who want to use CMA in their
> new feature development, since they can use CMA easily without
> copying & pasting this reserved area management code.
>
> In previous patches, we have prepared some features to generalize
> CMA reserved area management and now it's time to do it. This patch
> moves core functions to mm/cma.c and change DMA APIs to use
> these functions.
>
> There is no functional change in DMA APIs.
>
> v2: There is no big change from v1 in mm/cma.c. Mostly renaming.
>
> Acked-by: Michal Nazarewicz <[email protected]>
> Signed-off-by: Joonsoo Kim <[email protected]>
Reviewed-by: Aneesh Kumar K.V <[email protected]>
>
> diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
> index 00e13ce..4eac559 100644
> --- a/drivers/base/Kconfig
> +++ b/drivers/base/Kconfig
> @@ -283,16 +283,6 @@ config CMA_ALIGNMENT
>
> If unsure, leave the default value "8".
>
> -config CMA_AREAS
> - int "Maximum count of the CMA device-private areas"
> - default 7
> - help
> - CMA allows to create CMA areas for particular devices. This parameter
> - sets the maximum number of such device private CMA areas in the
> - system.
> -
> - If unsure, leave the default value "7".
> -
> endif
>
> endmenu
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index 9bc9340..f177f73 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -24,25 +24,10 @@
>
> #include <linux/memblock.h>
> #include <linux/err.h>
> -#include <linux/mm.h>
> -#include <linux/mutex.h>
> -#include <linux/page-isolation.h>
> #include <linux/sizes.h>
> -#include <linux/slab.h>
> -#include <linux/swap.h>
> -#include <linux/mm_types.h>
> #include <linux/dma-contiguous.h>
> #include <linux/log2.h>
> -
> -struct cma {
> - unsigned long base_pfn;
> - unsigned long count;
> - unsigned long *bitmap;
> - int order_per_bit; /* Order of pages represented by one bit */
> - struct mutex lock;
> -};
> -
> -struct cma *dma_contiguous_default_area;
> +#include <linux/cma.h>
>
> #ifdef CONFIG_CMA_SIZE_MBYTES
> #define CMA_SIZE_MBYTES CONFIG_CMA_SIZE_MBYTES
> @@ -50,6 +35,8 @@ struct cma *dma_contiguous_default_area;
> #define CMA_SIZE_MBYTES 0
> #endif
>
> +struct cma *dma_contiguous_default_area;
> +
> /*
> * Default global CMA area size can be defined in kernel's .config.
> * This is useful mainly for distro maintainers to create a kernel
> @@ -156,199 +143,13 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
> }
> }
>
> -static DEFINE_MUTEX(cma_mutex);
> -
> -static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
> -{
> - return (1 << (align_order >> cma->order_per_bit)) - 1;
> -}
> -
> -static unsigned long cma_bitmap_maxno(struct cma *cma)
> -{
> - return cma->count >> cma->order_per_bit;
> -}
> -
> -static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
> - unsigned long pages)
> -{
> - return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
> -}
> -
> -static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> -{
> - unsigned long bitmapno, nr_bits;
> -
> - bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> - nr_bits = cma_bitmap_pages_to_bits(cma, count);
> -
> - mutex_lock(&cma->lock);
> - bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> - mutex_unlock(&cma->lock);
> -}
> -
> -static int __init cma_activate_area(struct cma *cma)
> -{
> - int bitmap_maxno = cma_bitmap_maxno(cma);
> - int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> - unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> - unsigned i = cma->count >> pageblock_order;
> - struct zone *zone;
> -
> - pr_debug("%s()\n", __func__);
> -
> - cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> - if (!cma->bitmap)
> - return -ENOMEM;
> -
> - WARN_ON_ONCE(!pfn_valid(pfn));
> - zone = page_zone(pfn_to_page(pfn));
> -
> - do {
> - unsigned j;
> - base_pfn = pfn;
> - for (j = pageblock_nr_pages; j; --j, pfn++) {
> - WARN_ON_ONCE(!pfn_valid(pfn));
> - /*
> - * alloc_contig_range requires the pfn range
> - * specified to be in the same zone. Make this
> - * simple by forcing the entire CMA resv range
> - * to be in the same zone.
> - */
> - if (page_zone(pfn_to_page(pfn)) != zone)
> - goto err;
> - }
> - init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> - } while (--i);
> -
> - mutex_init(&cma->lock);
> - return 0;
> -
> -err:
> - kfree(cma->bitmap);
> - return -EINVAL;
> -}
> -
> -static struct cma cma_areas[MAX_CMA_AREAS];
> -static unsigned cma_area_count;
> -
> -static int __init cma_init_reserved_areas(void)
> -{
> - int i;
> -
> - for (i = 0; i < cma_area_count; i++) {
> - int ret = cma_activate_area(&cma_areas[i]);
> - if (ret)
> - return ret;
> - }
> -
> - return 0;
> -}
> -core_initcall(cma_init_reserved_areas);
> -
> -/**
> - * dma_contiguous_reserve_area() - reserve custom contiguous area
> - * @size: Size of the reserved area (in bytes),
> - * @base: Base address of the reserved area optional, use 0 for any
> - * @limit: End address of the reserved memory (optional, 0 for any).
> - * @alignment: Alignment for the contiguous memory area, should be power of 2
> - * @order_per_bit: Order of pages represented by one bit on bitmap.
> - * @res_cma: Pointer to store the created cma region.
> - * @fixed: hint about where to place the reserved area
> - *
> - * This function reserves memory from early allocator. It should be
> - * called by arch specific code once the early allocator (memblock or bootmem)
> - * has been activated and all other subsystems have already allocated/reserved
> - * memory. This function allows to create custom reserved areas for specific
> - * devices.
> - *
> - * If @fixed is true, reserve contiguous area at exactly @base. If false,
> - * reserve in range from @base to @limit.
> - */
> -static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> - phys_addr_t base, phys_addr_t limit,
> - phys_addr_t alignment, int order_per_bit,
> - struct cma **res_cma, bool fixed)
> -{
> - struct cma *cma = &cma_areas[cma_area_count];
> - int ret = 0;
> -
> - pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
> - __func__, (unsigned long)size, (unsigned long)base,
> - (unsigned long)limit, (unsigned long)alignment);
> -
> - /* Sanity checks */
> - if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> - pr_err("%s(): Not enough slots for CMA reserved regions!\n",
> - __func__);
> - return -ENOSPC;
> - }
> -
> - if (!size)
> - return -EINVAL;
> -
> - if (alignment && !is_power_of_2(alignment))
> - return -EINVAL;
> -
> - /*
> - * Sanitise input arguments.
> - * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
> - * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
> - * and CMA property will be broken.
> - */
> - alignment = max(alignment,
> - (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
> - base = ALIGN(base, alignment);
> - size = ALIGN(size, alignment);
> - limit &= ~(alignment - 1);
> - /* size should be aligned with order_per_bit */
> - BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
> -
> - /* Reserve memory */
> - if (base && fixed) {
> - if (memblock_is_region_reserved(base, size) ||
> - memblock_reserve(base, size) < 0) {
> - ret = -EBUSY;
> - goto err;
> - }
> - } else {
> - phys_addr_t addr = memblock_alloc_range(size, alignment, base,
> - limit);
> - if (!addr) {
> - ret = -ENOMEM;
> - goto err;
> - } else {
> - base = addr;
> - }
> - }
> -
> - /*
> - * Each reserved area must be initialised later, when more kernel
> - * subsystems (like slab allocator) are available.
> - */
> - cma->base_pfn = PFN_DOWN(base);
> - cma->count = size >> PAGE_SHIFT;
> - cma->order_per_bit = order_per_bit;
> - *res_cma = cma;
> - cma_area_count++;
> -
> - pr_info("%s(): reserved %ld MiB at %08lx\n",
> - __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
> - return 0;
> -
> -err:
> - pr_err("%s(): failed to reserve %ld MiB\n",
> - __func__, (unsigned long)size / SZ_1M);
> - return ret;
> -}
> -
> int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> phys_addr_t limit, struct cma **res_cma,
> bool fixed)
> {
> int ret;
>
> - ret = __dma_contiguous_reserve_area(size, base, limit, 0, 0,
> - res_cma, fixed);
> + ret = cma_declare_contiguous(size, base, limit, 0, 0, res_cma, fixed);
> if (ret)
> return ret;
>
> @@ -358,124 +159,17 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> return 0;
> }
>
> -/**
> - * dma_alloc_from_contiguous() - allocate pages from contiguous area
> - * @dev: Pointer to device for which the allocation is performed.
> - * @count: Requested number of pages.
> - * @align: Requested alignment of pages (in PAGE_SIZE order).
> - *
> - * This function allocates memory buffer for specified device. It uses
> - * device specific contiguous memory area if available or the default
> - * global one. Requires architecture specific dev_get_cma_area() helper
> - * function.
> - */
> -static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> - unsigned int align)
> -{
> - unsigned long mask, pfn, start = 0;
> - unsigned long bitmap_maxno, bitmapno, nr_bits;
> - struct page *page = NULL;
> - int ret;
> -
> - if (!cma || !cma->count)
> - return NULL;
> -
> - pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
> - count, align);
> -
> - if (!count)
> - return NULL;
> -
> - mask = cma_bitmap_aligned_mask(cma, align);
> - bitmap_maxno = cma_bitmap_maxno(cma);
> - nr_bits = cma_bitmap_pages_to_bits(cma, count);
> -
> - for (;;) {
> - mutex_lock(&cma->lock);
> - bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> - bitmap_maxno, start, nr_bits, mask);
> - if (bitmapno >= bitmap_maxno) {
> - mutex_unlock(&cma->lock);
> - break;
> - }
> - bitmap_set(cma->bitmap, bitmapno, nr_bits);
> - /*
> - * It's safe to drop the lock here. We've marked this region for
> - * our exclusive use. If the migration fails we will take the
> - * lock again and unmark it.
> - */
> - mutex_unlock(&cma->lock);
> -
> - pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> - mutex_lock(&cma_mutex);
> - ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> - mutex_unlock(&cma_mutex);
> - if (ret == 0) {
> - page = pfn_to_page(pfn);
> - break;
> - } else if (ret != -EBUSY) {
> - clear_cma_bitmap(cma, pfn, count);
> - break;
> - }
> - clear_cma_bitmap(cma, pfn, count);
> - pr_debug("%s(): memory range at %p is busy, retrying\n",
> - __func__, pfn_to_page(pfn));
> - /* try again with a bit different memory target */
> - start = bitmapno + mask + 1;
> - }
> -
> - pr_debug("%s(): returned %p\n", __func__, page);
> - return page;
> -}
> -
> struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> unsigned int align)
> {
> - struct cma *cma = dev_get_cma_area(dev);
> -
> if (align > CONFIG_CMA_ALIGNMENT)
> align = CONFIG_CMA_ALIGNMENT;
>
> - return __dma_alloc_from_contiguous(cma, count, align);
> -}
> -
> -/**
> - * dma_release_from_contiguous() - release allocated pages
> - * @dev: Pointer to device for which the pages were allocated.
> - * @pages: Allocated pages.
> - * @count: Number of allocated pages.
> - *
> - * This function releases memory allocated by dma_alloc_from_contiguous().
> - * It returns false when provided pages do not belong to contiguous area and
> - * true otherwise.
> - */
> -static bool __dma_release_from_contiguous(struct cma *cma, struct page *pages,
> - int count)
> -{
> - unsigned long pfn;
> -
> - if (!cma || !pages)
> - return false;
> -
> - pr_debug("%s(page %p)\n", __func__, (void *)pages);
> -
> - pfn = page_to_pfn(pages);
> -
> - if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count)
> - return false;
> -
> - VM_BUG_ON(pfn + count > cma->base_pfn + cma->count);
> -
> - free_contig_range(pfn, count);
> - clear_cma_bitmap(cma, pfn, count);
> -
> - return true;
> + return cma_alloc(dev_get_cma_area(dev), count, align);
> }
>
> bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> int count)
> {
> - struct cma *cma = dev_get_cma_area(dev);
> -
> - return __dma_release_from_contiguous(cma, pages, count);
> + return cma_release(dev_get_cma_area(dev), pages, count);
> }
> diff --git a/include/linux/cma.h b/include/linux/cma.h
> new file mode 100644
> index 0000000..e38efe9
> --- /dev/null
> +++ b/include/linux/cma.h
> @@ -0,0 +1,12 @@
> +#ifndef __CMA_H__
> +#define __CMA_H__
> +
> +struct cma;
> +
> +extern int __init cma_declare_contiguous(phys_addr_t size,
> + phys_addr_t base, phys_addr_t limit,
> + phys_addr_t alignment, int order_per_bit,
> + struct cma **res_cma, bool fixed);
> +extern struct page *cma_alloc(struct cma *cma, int count, unsigned int align);
> +extern bool cma_release(struct cma *cma, struct page *pages, int count);
> +#endif
> diff --git a/include/linux/dma-contiguous.h b/include/linux/dma-contiguous.h
> index 772eab5..a40c1f3 100644
> --- a/include/linux/dma-contiguous.h
> +++ b/include/linux/dma-contiguous.h
> @@ -53,9 +53,10 @@
>
> #ifdef __KERNEL__
>
> +#include <linux/device.h>
> +
> struct cma;
> struct page;
> -struct device;
>
> #ifdef CONFIG_DMA_CMA
>
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 3e9977a..f4899ec 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -508,6 +508,17 @@ config CMA_DEBUG
> processing calls such as dma_alloc_from_contiguous().
> This option does not affect warning and error messages.
>
> +config CMA_AREAS
> + int "Maximum count of the CMA areas"
> + depends on CMA
> + default 7
> + help
> + CMA allows to create CMA areas for particular purpose, mainly,
> + used as device private area. This parameter sets the maximum
> + number of CMA area in the system.
> +
> + If unsure, leave the default value "7".
> +
> config ZBUD
> tristate
> default n
> diff --git a/mm/Makefile b/mm/Makefile
> index 1eaa70b..bc0422b 100644
> --- a/mm/Makefile
> +++ b/mm/Makefile
> @@ -62,3 +62,4 @@ obj-$(CONFIG_MEMORY_ISOLATION) += page_isolation.o
> obj-$(CONFIG_ZBUD) += zbud.o
> obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
> obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
> +obj-$(CONFIG_CMA) += cma.o
> diff --git a/mm/cma.c b/mm/cma.c
> new file mode 100644
> index 0000000..1e1b017
> --- /dev/null
> +++ b/mm/cma.c
> @@ -0,0 +1,330 @@
> +/*
> + * Contiguous Memory Allocator
> + *
> + * Copyright (c) 2010-2011 by Samsung Electronics.
> + * Copyright IBM Corporation, 2013
> + * Copyright LG Electronics Inc., 2014
> + * Written by:
> + * Marek Szyprowski <[email protected]>
> + * Michal Nazarewicz <[email protected]>
> + * Aneesh Kumar K.V <[email protected]>
> + * Joonsoo Kim <[email protected]>
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License as
> + * published by the Free Software Foundation; either version 2 of the
> + * License or (at your optional) any later version of the license.
> + */
> +
> +#define pr_fmt(fmt) "cma: " fmt
> +
> +#ifdef CONFIG_CMA_DEBUG
> +#ifndef DEBUG
> +# define DEBUG
> +#endif
> +#endif
> +
> +#include <linux/memblock.h>
> +#include <linux/err.h>
> +#include <linux/mm.h>
> +#include <linux/mutex.h>
> +#include <linux/sizes.h>
> +#include <linux/slab.h>
> +
> +struct cma {
> + unsigned long base_pfn;
> + unsigned long count;
> + unsigned long *bitmap;
> + int order_per_bit; /* Order of pages represented by one bit */
> + struct mutex lock;
> +};
> +
> +/*
> + * There is always at least global CMA area and a few optional
> + * areas configured in kernel .config.
> + */
> +#define MAX_CMA_AREAS (1 + CONFIG_CMA_AREAS)
> +
> +static struct cma cma_areas[MAX_CMA_AREAS];
> +static unsigned cma_area_count;
> +static DEFINE_MUTEX(cma_mutex);
> +
> +static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
> +{
> + return (1 << (align_order >> cma->order_per_bit)) - 1;
> +}
> +
> +static unsigned long cma_bitmap_maxno(struct cma *cma)
> +{
> + return cma->count >> cma->order_per_bit;
> +}
> +
> +static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
> + unsigned long pages)
> +{
> + return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
> +}
> +
> +static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> +{
> + unsigned long bitmapno, nr_bits;
> +
> + bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + mutex_lock(&cma->lock);
> + bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> + mutex_unlock(&cma->lock);
> +}
> +
> +static int __init cma_activate_area(struct cma *cma)
> +{
> + int bitmap_maxno = cma_bitmap_maxno(cma);
> + int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> + unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> + unsigned i = cma->count >> pageblock_order;
> + struct zone *zone;
> +
> + pr_debug("%s()\n", __func__);
> +
> + cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL);
> + if (!cma->bitmap)
> + return -ENOMEM;
> +
> + WARN_ON_ONCE(!pfn_valid(pfn));
> + zone = page_zone(pfn_to_page(pfn));
> +
> + do {
> + unsigned j;
> +
> + base_pfn = pfn;
> + for (j = pageblock_nr_pages; j; --j, pfn++) {
> + WARN_ON_ONCE(!pfn_valid(pfn));
> + /*
> + * alloc_contig_range requires the pfn range
> + * specified to be in the same zone. Make this
> + * simple by forcing the entire CMA resv range
> + * to be in the same zone.
> + */
> + if (page_zone(pfn_to_page(pfn)) != zone)
> + goto err;
> + }
> + init_cma_reserved_pageblock(pfn_to_page(base_pfn));
> + } while (--i);
> +
> + mutex_init(&cma->lock);
> + return 0;
> +
> +err:
> + kfree(cma->bitmap);
> + return -EINVAL;
> +}
> +
> +static int __init cma_init_reserved_areas(void)
> +{
> + int i;
> +
> + for (i = 0; i < cma_area_count; i++) {
> + int ret = cma_activate_area(&cma_areas[i]);
> +
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> +core_initcall(cma_init_reserved_areas);
> +
> +/**
> + * cma_declare_contiguous() - reserve custom contiguous area
> + * @size: Size of the reserved area (in bytes),
> + * @base: Base address of the reserved area optional, use 0 for any
> + * @limit: End address of the reserved memory (optional, 0 for any).
> + * @alignment: Alignment for the contiguous memory area, should be power of 2
> + * @order_per_bit: Order of pages represented by one bit on bitmap.
> + * @res_cma: Pointer to store the created cma region.
> + * @fixed: hint about where to place the reserved area
> + *
> + * This function reserves memory from early allocator. It should be
> + * called by arch specific code once the early allocator (memblock or bootmem)
> + * has been activated and all other subsystems have already allocated/reserved
> + * memory. This function allows to create custom reserved areas.
> + *
> + * If @fixed is true, reserve contiguous area at exactly @base. If false,
> + * reserve in range from @base to @limit.
> + */
> +int __init cma_declare_contiguous(phys_addr_t size,
> + phys_addr_t base, phys_addr_t limit,
> + phys_addr_t alignment, int order_per_bit,
> + struct cma **res_cma, bool fixed)
> +{
> + struct cma *cma = &cma_areas[cma_area_count];
> + int ret = 0;
> +
> + pr_debug("%s(size %lx, base %08lx, limit %08lx alignment %08lx)\n",
> + __func__, (unsigned long)size, (unsigned long)base,
> + (unsigned long)limit, (unsigned long)alignment);
> +
> + /* Sanity checks */
> + if (cma_area_count == ARRAY_SIZE(cma_areas)) {
> + pr_err("%s(): Not enough slots for CMA reserved regions!\n",
> + __func__);
> + return -ENOSPC;
> + }
> +
> + if (!size)
> + return -EINVAL;
> +
> + if (alignment && !is_power_of_2(alignment))
> + return -EINVAL;
> +
> + /*
> + * Sanitise input arguments.
> + * CMA area should be at least MAX_ORDER - 1 aligned. Otherwise,
> + * CMA area could be merged into other MIGRATE_TYPE by buddy mechanism
> + * and CMA property will be broken.
> + */
> + alignment = max(alignment,
> + (phys_addr_t)PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order));
> + base = ALIGN(base, alignment);
> + size = ALIGN(size, alignment);
> + limit &= ~(alignment - 1);
> + /* size should be aligned with order_per_bit */
> + BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
> +
> + /* Reserve memory */
> + if (base && fixed) {
> + if (memblock_is_region_reserved(base, size) ||
> + memblock_reserve(base, size) < 0) {
> + ret = -EBUSY;
> + goto err;
> + }
> + } else {
> + phys_addr_t addr = memblock_alloc_range(size, alignment, base,
> + limit);
> + if (!addr) {
> + ret = -ENOMEM;
> + goto err;
> + } else {
> + base = addr;
> + }
> + }
> +
> + /*
> + * Each reserved area must be initialised later, when more kernel
> + * subsystems (like slab allocator) are available.
> + */
> + cma->base_pfn = PFN_DOWN(base);
> + cma->count = size >> PAGE_SHIFT;
> + cma->order_per_bit = order_per_bit;
> + *res_cma = cma;
> + cma_area_count++;
> +
> + pr_info("%s(): reserved %ld MiB at %08lx\n",
> + __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
> + return 0;
> +
> +err:
> + pr_err("%s(): failed to reserve %ld MiB\n",
> + __func__, (unsigned long)size / SZ_1M);
> + return ret;
> +}
> +
> +/**
> + * cma_alloc() - allocate pages from contiguous area
> + * @cma: Contiguous memory region for which the allocation is performed.
> + * @count: Requested number of pages.
> + * @align: Requested alignment of pages (in PAGE_SIZE order).
> + *
> + * This function allocates part of contiguous memory on specific
> + * contiguous memory area.
> + */
> +struct page *cma_alloc(struct cma *cma, int count, unsigned int align)
> +{
> + unsigned long mask, pfn, start = 0;
> + unsigned long bitmap_maxno, bitmapno, nr_bits;
> + struct page *page = NULL;
> + int ret;
> +
> + if (!cma || !cma->count)
> + return NULL;
> +
> + pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
> + count, align);
> +
> + if (!count)
> + return NULL;
> +
> + mask = cma_bitmap_aligned_mask(cma, align);
> + bitmap_maxno = cma_bitmap_maxno(cma);
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + for (;;) {
> + mutex_lock(&cma->lock);
> + bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> + bitmap_maxno, start, nr_bits, mask);
> + if (bitmapno >= bitmap_maxno) {
> + mutex_unlock(&cma->lock);
> + break;
> + }
> + bitmap_set(cma->bitmap, bitmapno, nr_bits);
> + /*
> + * It's safe to drop the lock here. We've marked this region for
> + * our exclusive use. If the migration fails we will take the
> + * lock again and unmark it.
> + */
> + mutex_unlock(&cma->lock);
> +
> + pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> + mutex_lock(&cma_mutex);
> + ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> + mutex_unlock(&cma_mutex);
> + if (ret == 0) {
> + page = pfn_to_page(pfn);
> + break;
> + } else if (ret != -EBUSY) {
> + clear_cma_bitmap(cma, pfn, count);
> + break;
> + }
> + clear_cma_bitmap(cma, pfn, count);
> + pr_debug("%s(): memory range at %p is busy, retrying\n",
> + __func__, pfn_to_page(pfn));
> + /* try again with a bit different memory target */
> + start = bitmapno + mask + 1;
> + }
> +
> + pr_debug("%s(): returned %p\n", __func__, page);
> + return page;
> +}
> +
> +/**
> + * cma_release() - release allocated pages
> + * @cma: Contiguous memory region for which the allocation is performed.
> + * @pages: Allocated pages.
> + * @count: Number of allocated pages.
> + *
> + * This function releases memory allocated by alloc_cma().
> + * It returns false when provided pages do not belong to contiguous area and
> + * true otherwise.
> + */
> +bool cma_release(struct cma *cma, struct page *pages, int count)
> +{
> + unsigned long pfn;
> +
> + if (!cma || !pages)
> + return false;
> +
> + pr_debug("%s(page %p)\n", __func__, (void *)pages);
> +
> + pfn = page_to_pfn(pages);
> +
> + if (pfn < cma->base_pfn || pfn >= cma->base_pfn + cma->count)
> + return false;
> +
> + VM_BUG_ON(pfn + count > cma->base_pfn + cma->count);
> +
> + free_contig_range(pfn, count);
> + clear_cma_bitmap(cma, pfn, count);
> +
> + return true;
> +}
> --
> 1.7.9.5
Joonsoo Kim <[email protected]> writes:
> ppc kvm's cma region management requires arbitrary bitmap granularity,
> since they want to reserve very large memory and manage this region
> with bitmap that one bit for several pages to reduce management overheads.
> So support arbitrary bitmap granularity for following generalization.
>
> Signed-off-by: Joonsoo Kim <[email protected]>
Reviewed-by: Aneesh Kumar K.V <[email protected]>
>
> diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> index bc4c171..9bc9340 100644
> --- a/drivers/base/dma-contiguous.c
> +++ b/drivers/base/dma-contiguous.c
> @@ -38,6 +38,7 @@ struct cma {
> unsigned long base_pfn;
> unsigned long count;
> unsigned long *bitmap;
> + int order_per_bit; /* Order of pages represented by one bit */
> struct mutex lock;
> };
>
> @@ -157,9 +158,38 @@ void __init dma_contiguous_reserve(phys_addr_t limit)
>
> static DEFINE_MUTEX(cma_mutex);
>
> +static unsigned long cma_bitmap_aligned_mask(struct cma *cma, int align_order)
> +{
> + return (1 << (align_order >> cma->order_per_bit)) - 1;
> +}
> +
> +static unsigned long cma_bitmap_maxno(struct cma *cma)
> +{
> + return cma->count >> cma->order_per_bit;
> +}
> +
> +static unsigned long cma_bitmap_pages_to_bits(struct cma *cma,
> + unsigned long pages)
> +{
> + return ALIGN(pages, 1 << cma->order_per_bit) >> cma->order_per_bit;
> +}
> +
> +static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> +{
> + unsigned long bitmapno, nr_bits;
> +
> + bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + mutex_lock(&cma->lock);
> + bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> + mutex_unlock(&cma->lock);
> +}
> +
> static int __init cma_activate_area(struct cma *cma)
> {
> - int bitmap_size = BITS_TO_LONGS(cma->count) * sizeof(long);
> + int bitmap_maxno = cma_bitmap_maxno(cma);
> + int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> unsigned i = cma->count >> pageblock_order;
> struct zone *zone;
> @@ -221,6 +251,7 @@ core_initcall(cma_init_reserved_areas);
> * @base: Base address of the reserved area optional, use 0 for any
> * @limit: End address of the reserved memory (optional, 0 for any).
> * @alignment: Alignment for the contiguous memory area, should be power of 2
> + * @order_per_bit: Order of pages represented by one bit on bitmap.
> * @res_cma: Pointer to store the created cma region.
> * @fixed: hint about where to place the reserved area
> *
> @@ -235,7 +266,7 @@ core_initcall(cma_init_reserved_areas);
> */
> static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> phys_addr_t base, phys_addr_t limit,
> - phys_addr_t alignment,
> + phys_addr_t alignment, int order_per_bit,
> struct cma **res_cma, bool fixed)
> {
> struct cma *cma = &cma_areas[cma_area_count];
> @@ -269,6 +300,8 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> base = ALIGN(base, alignment);
> size = ALIGN(size, alignment);
> limit &= ~(alignment - 1);
> + /* size should be aligned with order_per_bit */
> + BUG_ON(!IS_ALIGNED(size >> PAGE_SHIFT, 1 << order_per_bit));
>
> /* Reserve memory */
> if (base && fixed) {
> @@ -294,6 +327,7 @@ static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> */
> cma->base_pfn = PFN_DOWN(base);
> cma->count = size >> PAGE_SHIFT;
> + cma->order_per_bit = order_per_bit;
> *res_cma = cma;
> cma_area_count++;
>
> @@ -313,7 +347,7 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> {
> int ret;
>
> - ret = __dma_contiguous_reserve_area(size, base, limit, 0,
> + ret = __dma_contiguous_reserve_area(size, base, limit, 0, 0,
> res_cma, fixed);
> if (ret)
> return ret;
> @@ -324,13 +358,6 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> return 0;
> }
>
> -static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> -{
> - mutex_lock(&cma->lock);
> - bitmap_clear(cma->bitmap, pfn - cma->base_pfn, count);
> - mutex_unlock(&cma->lock);
> -}
> -
> /**
> * dma_alloc_from_contiguous() - allocate pages from contiguous area
> * @dev: Pointer to device for which the allocation is performed.
> @@ -345,7 +372,8 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> unsigned int align)
> {
> - unsigned long mask, pfn, pageno, start = 0;
> + unsigned long mask, pfn, start = 0;
> + unsigned long bitmap_maxno, bitmapno, nr_bits;
> struct page *page = NULL;
> int ret;
>
> @@ -358,18 +386,19 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> if (!count)
> return NULL;
>
> - mask = (1 << align) - 1;
> -
> + mask = cma_bitmap_aligned_mask(cma, align);
> + bitmap_maxno = cma_bitmap_maxno(cma);
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
>
> for (;;) {
> mutex_lock(&cma->lock);
> - pageno = bitmap_find_next_zero_area(cma->bitmap, cma->count,
> - start, count, mask);
> - if (pageno >= cma->count) {
> + bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> + bitmap_maxno, start, nr_bits, mask);
> + if (bitmapno >= bitmap_maxno) {
> mutex_unlock(&cma->lock);
> break;
> }
> - bitmap_set(cma->bitmap, pageno, count);
> + bitmap_set(cma->bitmap, bitmapno, nr_bits);
> /*
> * It's safe to drop the lock here. We've marked this region for
> * our exclusive use. If the migration fails we will take the
> @@ -377,7 +406,7 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> */
> mutex_unlock(&cma->lock);
>
> - pfn = cma->base_pfn + pageno;
> + pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> mutex_lock(&cma_mutex);
> ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> mutex_unlock(&cma_mutex);
> @@ -392,7 +421,7 @@ static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> pr_debug("%s(): memory range at %p is busy, retrying\n",
> __func__, pfn_to_page(pfn));
> /* try again with a bit different memory target */
> - start = pageno + mask + 1;
> + start = bitmapno + mask + 1;
> }
>
> pr_debug("%s(): returned %p\n", __func__, page);
> --
> 1.7.9.5
Joonsoo Kim <[email protected]> writes:
> Currently, there are two users on CMA functionality, one is the DMA
> subsystem and the other is the kvm on powerpc. They have their own code
> to manage CMA reserved area even if they looks really similar.
> From my guess, it is caused by some needs on bitmap management. Kvm side
> wants to maintain bitmap not for 1 page, but for more size. Eventually it
> use bitmap where one bit represents 64 pages.
>
> When I implement CMA related patches, I should change those two places
> to apply my change and it seem to be painful to me. I want to change
> this situation and reduce future code management overhead through
> this patch.
>
> This change could also help developer who want to use CMA in their
> new feature development, since they can use CMA easily without
> copying & pasting this reserved area management code.
>
> In previous patches, we have prepared some features to generalize
> CMA reserved area management and now it's time to do it. This patch
> moves core functions to mm/cma.c and change DMA APIs to use
> these functions.
>
> There is no functional change in DMA APIs.
>
> v2: There is no big change from v1 in mm/cma.c. Mostly renaming.
>
> Acked-by: Michal Nazarewicz <[email protected]>
> Signed-off-by: Joonsoo Kim <[email protected]>
>
.....
> +
> + mask = cma_bitmap_aligned_mask(cma, align);
> + bitmap_maxno = cma_bitmap_maxno(cma);
> + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> +
> + for (;;) {
> + mutex_lock(&cma->lock);
> + bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> + bitmap_maxno, start, nr_bits, mask);
> + if (bitmapno >= bitmap_maxno) {
> + mutex_unlock(&cma->lock);
> + break;
> + }
> + bitmap_set(cma->bitmap, bitmapno, nr_bits);
> + /*
> + * It's safe to drop the lock here. We've marked this region for
> + * our exclusive use. If the migration fails we will take the
> + * lock again and unmark it.
> + */
> + mutex_unlock(&cma->lock);
> +
> + pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> + mutex_lock(&cma_mutex);
> + ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> + mutex_unlock(&cma_mutex);
> + if (ret == 0) {
> + page = pfn_to_page(pfn);
> + break;
> + } else if (ret != -EBUSY) {
> + clear_cma_bitmap(cma, pfn, count);
> + break;
> + }
> +
For setting bit map we do
bitmap_set(cma->bitmap, bitmapno, nr_bits);
alloc_contig()..
if (error)
clear_cma_bitmap(cma, pfn, count);
Why ?
why not bitmap_clear() ?
-aneesh
On Thu, Jun 12, 2014 at 11:53:16AM +0200, Michal Nazarewicz wrote:
> On Thu, Jun 12 2014, Michal Nazarewicz <[email protected]> wrote:
> > I used “function(arg1, arg2, …)” at the *beginning* of functions when
> > the arguments passed to the function were included in the message. In
> > all other cases I left it at just “function:” (or just no additional
> > prefix). IMO that's a reasonable strategy.
>
> At closer inspection, I realised drivers/base/dma-contiguous.c is
> Marek's code, but the above I think is still reasonable thing to do, so
> I'd rather standardise on having “function(…)” only at the beginning of
> a function. Just my 0.02 CHF.
Hello,
Now, I realize that these changes aren't needed in this patchset, so I
simplify this patch just to remove redundant 'CMA' prefix. Other things
can be done after merging if we need.
Thanks.
On Thu, Jun 12, 2014 at 12:02:38PM +0200, Michal Nazarewicz wrote:
> On Thu, Jun 12 2014, Joonsoo Kim <[email protected]> wrote:
> > ppc kvm's cma area management needs alignment constraint on
>
> I've noticed it earlier and cannot seem to get to terms with this. It
> should IMO be PPC, KVM and CMA since those are acronyms. But if you
> have strong feelings, it's not a big issue.
Yes, I will fix it.
>
> > cma region. So support it to prepare generalization of cma area
> > management functionality.
> >
> > Additionally, add some comments which tell us why alignment
> > constraint is needed on cma region.
> >
> > Signed-off-by: Joonsoo Kim <[email protected]>
>
> Acked-by: Michal Nazarewicz <[email protected]>
>
> > diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> > index 8a44c82..bc4c171 100644
> > --- a/drivers/base/dma-contiguous.c
> > +++ b/drivers/base/dma-contiguous.c
> > @@ -219,6 +220,7 @@ core_initcall(cma_init_reserved_areas);
> > * @size: Size of the reserved area (in bytes),
> > * @base: Base address of the reserved area optional, use 0 for any
> > * @limit: End address of the reserved memory (optional, 0 for any).
> > + * @alignment: Alignment for the contiguous memory area, should be
> > power of 2
>
> “must be power of 2 or zero”.
Okay.
> > * @res_cma: Pointer to store the created cma region.
> > * @fixed: hint about where to place the reserved area
> > *
> > @@ -233,15 +235,15 @@ core_initcall(cma_init_reserved_areas);
> > */
> > static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> > phys_addr_t base, phys_addr_t limit,
> > + phys_addr_t alignment,
> > struct cma **res_cma, bool fixed)
> > {
> > struct cma *cma = &cma_areas[cma_area_count];
> > - phys_addr_t alignment;
> > int ret = 0;
> >
> > - pr_debug("%s(size %lx, base %08lx, limit %08lx)\n", __func__,
> > - (unsigned long)size, (unsigned long)base,
> > - (unsigned long)limit);
> > + pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
> > + __func__, (unsigned long)size, (unsigned long)base,
> > + (unsigned long)limit, (unsigned long)alignment);
>
> Nit: Align with the rest of the arguments, i.e.:
>
> + pr_debug("%s(size %lx, base %08lx, limit %08lx align_order %08lx)\n",
> + __func__, (unsigned long)size, (unsigned long)base,
> + (unsigned long)limit, (unsigned long)alignment);
What's the difference between mine and yours?
Thanks.
On Thu, Jun 12, 2014 at 12:19:54PM +0200, Michal Nazarewicz wrote:
> On Thu, Jun 12 2014, Joonsoo Kim <[email protected]> wrote:
> > ppc kvm's cma region management requires arbitrary bitmap granularity,
> > since they want to reserve very large memory and manage this region
> > with bitmap that one bit for several pages to reduce management overheads.
> > So support arbitrary bitmap granularity for following generalization.
> >
> > Signed-off-by: Joonsoo Kim <[email protected]>
>
> Acked-by: Michal Nazarewicz <[email protected]>
>
> > diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> > index bc4c171..9bc9340 100644
> > --- a/drivers/base/dma-contiguous.c
> > +++ b/drivers/base/dma-contiguous.c
> > @@ -38,6 +38,7 @@ struct cma {
> > unsigned long base_pfn;
> > unsigned long count;
>
> Have you considered replacing count with maxno?
No, I haven't.
I think that count is better than maxno, since it represent number of
pages in this region.
>
> > unsigned long *bitmap;
> > + int order_per_bit; /* Order of pages represented by one bit */
>
> I'd make it unsigned.
Will fix it.
> > struct mutex lock;
> > };
> >
> > +static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int
> > count)
>
> For consistency cma_clear_bitmap would make more sense I think. On the
> other hand, you're just moving stuff around so perhaps renaming the
> function at this point is not worth it any more.
Will fix it.
> > +{
> > + unsigned long bitmapno, nr_bits;
> > +
> > + bitmapno = (pfn - cma->base_pfn) >> cma->order_per_bit;
> > + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> > +
> > + mutex_lock(&cma->lock);
> > + bitmap_clear(cma->bitmap, bitmapno, nr_bits);
> > + mutex_unlock(&cma->lock);
> > +}
> > +
> > static int __init cma_activate_area(struct cma *cma)
> > {
> > - int bitmap_size = BITS_TO_LONGS(cma->count) * sizeof(long);
> > + int bitmap_maxno = cma_bitmap_maxno(cma);
> > + int bitmap_size = BITS_TO_LONGS(bitmap_maxno) * sizeof(long);
> > unsigned long base_pfn = cma->base_pfn, pfn = base_pfn;
> > unsigned i = cma->count >> pageblock_order;
> > struct zone *zone;
>
> bitmap_maxno is never used again, perhaps:
>
> + int bitmap_size = BITS_TO_LONGS(cma_bitmap_maxno(cma)) * sizeof(long);
>
> instead? Up to you.
Okay!!
Thanks.
On Thu, Jun 12, 2014 at 02:37:43PM +0900, Minchan Kim wrote:
> On Thu, Jun 12, 2014 at 12:21:40PM +0900, Joonsoo Kim wrote:
> > To prepare future generalization work on cma area management code,
> > we need to separate core cma management codes from DMA APIs.
> > We will extend these core functions to cover requirements of
> > ppc kvm's cma area management functionality in following patches.
> > This separation helps us not to touch DMA APIs while extending
> > core functions.
> >
> > Signed-off-by: Joonsoo Kim <[email protected]>
> >
> > diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
> > index fb0cdce..8a44c82 100644
> > --- a/drivers/base/dma-contiguous.c
> > +++ b/drivers/base/dma-contiguous.c
> > @@ -231,9 +231,9 @@ core_initcall(cma_init_reserved_areas);
> > * If @fixed is true, reserve contiguous area at exactly @base. If false,
> > * reserve in range from @base to @limit.
> > */
> > -int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> > - phys_addr_t limit, struct cma **res_cma,
> > - bool fixed)
> > +static int __init __dma_contiguous_reserve_area(phys_addr_t size,
> > + phys_addr_t base, phys_addr_t limit,
> > + struct cma **res_cma, bool fixed)
> > {
> > struct cma *cma = &cma_areas[cma_area_count];
> > phys_addr_t alignment;
> > @@ -288,16 +288,30 @@ int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> >
> > pr_info("%s(): reserved %ld MiB at %08lx\n",
> > __func__, (unsigned long)size / SZ_1M, (unsigned long)base);
> > -
> > - /* Architecture specific contiguous memory fixup. */
> > - dma_contiguous_early_fixup(base, size);
> > return 0;
> > +
> > err:
> > pr_err("%s(): failed to reserve %ld MiB\n",
> > __func__, (unsigned long)size / SZ_1M);
> > return ret;
> > }
> >
> > +int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
> > + phys_addr_t limit, struct cma **res_cma,
> > + bool fixed)
> > +{
> > + int ret;
> > +
> > + ret = __dma_contiguous_reserve_area(size, base, limit, res_cma, fixed);
> > + if (ret)
> > + return ret;
> > +
> > + /* Architecture specific contiguous memory fixup. */
> > + dma_contiguous_early_fixup(base, size);
>
> In old, base and size are aligned with alignment and passed into arch fixup
> but your patch is changing it.
> I didn't look at what kinds of side effect it makes but just want to confirm.
Good catch!!!
I will fix it.
> > +
> > + return 0;
> > +}
> > +
> > static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> > {
> > mutex_lock(&cma->lock);
> > @@ -316,20 +330,16 @@ static void clear_cma_bitmap(struct cma *cma, unsigned long pfn, int count)
> > * global one. Requires architecture specific dev_get_cma_area() helper
> > * function.
> > */
> > -struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> > +static struct page *__dma_alloc_from_contiguous(struct cma *cma, int count,
> > unsigned int align)
> > {
> > unsigned long mask, pfn, pageno, start = 0;
> > - struct cma *cma = dev_get_cma_area(dev);
> > struct page *page = NULL;
> > int ret;
> >
> > if (!cma || !cma->count)
> > return NULL;
> >
> > - if (align > CONFIG_CMA_ALIGNMENT)
> > - align = CONFIG_CMA_ALIGNMENT;
> > -
> > pr_debug("%s(cma %p, count %d, align %d)\n", __func__, (void *)cma,
> > count, align);
> >
> > @@ -377,6 +387,17 @@ struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> > return page;
> > }
> >
>
> Please move the description in __dma_alloc_from_contiguous to here exported API.
>
Okay.
> > +struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> > + unsigned int align)
> > +{
> > + struct cma *cma = dev_get_cma_area(dev);
> > +
> > + if (align > CONFIG_CMA_ALIGNMENT)
> > + align = CONFIG_CMA_ALIGNMENT;
> > +
> > + return __dma_alloc_from_contiguous(cma, count, align);
> > +}
> > +
> > /**
> > * dma_release_from_contiguous() - release allocated pages
> > * @dev: Pointer to device for which the pages were allocated.
> > @@ -387,10 +408,9 @@ struct page *dma_alloc_from_contiguous(struct device *dev, int count,
> > * It returns false when provided pages do not belong to contiguous area and
> > * true otherwise.
> > */
> > -bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> > +static bool __dma_release_from_contiguous(struct cma *cma, struct page *pages,
> > int count)
> > {
> > - struct cma *cma = dev_get_cma_area(dev);
> > unsigned long pfn;
> >
> > if (!cma || !pages)
> > @@ -410,3 +430,11 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages,
> >
> > return true;
> > }
> > +
>
> Ditto.
>
Okay.
Thanks.
On Sat, Jun 14, 2014 at 03:46:44PM +0530, Aneesh Kumar K.V wrote:
> Joonsoo Kim <[email protected]> writes:
>
> > Currently, there are two users on CMA functionality, one is the DMA
> > subsystem and the other is the kvm on powerpc. They have their own code
> > to manage CMA reserved area even if they looks really similar.
> > From my guess, it is caused by some needs on bitmap management. Kvm side
> > wants to maintain bitmap not for 1 page, but for more size. Eventually it
> > use bitmap where one bit represents 64 pages.
> >
> > When I implement CMA related patches, I should change those two places
> > to apply my change and it seem to be painful to me. I want to change
> > this situation and reduce future code management overhead through
> > this patch.
> >
> > This change could also help developer who want to use CMA in their
> > new feature development, since they can use CMA easily without
> > copying & pasting this reserved area management code.
> >
> > In previous patches, we have prepared some features to generalize
> > CMA reserved area management and now it's time to do it. This patch
> > moves core functions to mm/cma.c and change DMA APIs to use
> > these functions.
> >
> > There is no functional change in DMA APIs.
> >
> > v2: There is no big change from v1 in mm/cma.c. Mostly renaming.
> >
> > Acked-by: Michal Nazarewicz <[email protected]>
> > Signed-off-by: Joonsoo Kim <[email protected]>
> >
>
> .....
>
> > +
> > + mask = cma_bitmap_aligned_mask(cma, align);
> > + bitmap_maxno = cma_bitmap_maxno(cma);
> > + nr_bits = cma_bitmap_pages_to_bits(cma, count);
> > +
> > + for (;;) {
> > + mutex_lock(&cma->lock);
> > + bitmapno = bitmap_find_next_zero_area(cma->bitmap,
> > + bitmap_maxno, start, nr_bits, mask);
> > + if (bitmapno >= bitmap_maxno) {
> > + mutex_unlock(&cma->lock);
> > + break;
> > + }
> > + bitmap_set(cma->bitmap, bitmapno, nr_bits);
> > + /*
> > + * It's safe to drop the lock here. We've marked this region for
> > + * our exclusive use. If the migration fails we will take the
> > + * lock again and unmark it.
> > + */
> > + mutex_unlock(&cma->lock);
> > +
> > + pfn = cma->base_pfn + (bitmapno << cma->order_per_bit);
> > + mutex_lock(&cma_mutex);
> > + ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA);
> > + mutex_unlock(&cma_mutex);
> > + if (ret == 0) {
> > + page = pfn_to_page(pfn);
> > + break;
> > + } else if (ret != -EBUSY) {
> > + clear_cma_bitmap(cma, pfn, count);
> > + break;
> > + }
> > +
>
>
> For setting bit map we do
> bitmap_set(cma->bitmap, bitmapno, nr_bits);
> alloc_contig()..
> if (error)
> clear_cma_bitmap(cma, pfn, count);
>
> Why ?
>
> why not bitmap_clear() ?
>
Unlike your psuedo code, for setting bitmap, we do
- grab the mutex
- bitmap_set
- release the mutex
clear_cma_bitmap() handles these things.
Thanks.
On Sat, Jun 14, 2014 at 03:35:33PM +0530, Aneesh Kumar K.V wrote:
> Joonsoo Kim <[email protected]> writes:
>
> > Now, we have general CMA reserved area management framework,
> > so use it for future maintainabilty. There is no functional change.
> >
> > Acked-by: Michal Nazarewicz <[email protected]>
> > Acked-by: Paolo Bonzini <[email protected]>
> > Signed-off-by: Joonsoo Kim <[email protected]>
> >
> > diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c
> > index 7cde8a6..28ec226 100644
> > --- a/arch/powerpc/kvm/book3s_hv_builtin.c
> > +++ b/arch/powerpc/kvm/book3s_hv_builtin.c
> > @@ -16,12 +16,14 @@
> > #include <linux/init.h>
> > #include <linux/memblock.h>
> > #include <linux/sizes.h>
> > +#include <linux/cma.h>
> >
> > #include <asm/cputable.h>
> > #include <asm/kvm_ppc.h>
> > #include <asm/kvm_book3s.h>
> >
> > -#include "book3s_hv_cma.h"
> > +#define KVM_CMA_CHUNK_ORDER 18
> > +
> > /*
> > * Hash page table alignment on newer cpus(CPU_FTR_ARCH_206)
> > * should be power of 2.
> > @@ -43,6 +45,8 @@ static unsigned long kvm_cma_resv_ratio = 5;
> > unsigned long kvm_rma_pages = (1 << 27) >> PAGE_SHIFT; /* 128MB */
> > EXPORT_SYMBOL_GPL(kvm_rma_pages);
> >
> > +static struct cma *kvm_cma;
> > +
> > /* Work out RMLS (real mode limit selector) field value for a given RMA size.
> > Assumes POWER7 or PPC970. */
> > static inline int lpcr_rmls(unsigned long rma_size)
> > @@ -97,7 +101,7 @@ struct kvm_rma_info *kvm_alloc_rma()
> > ri = kmalloc(sizeof(struct kvm_rma_info), GFP_KERNEL);
> > if (!ri)
> > return NULL;
> > - page = kvm_alloc_cma(kvm_rma_pages, kvm_rma_pages);
> > + page = cma_alloc(kvm_cma, kvm_rma_pages, get_order(kvm_rma_pages));
> > if (!page)
> > goto err_out;
> > atomic_set(&ri->use_count, 1);
> > @@ -112,7 +116,7 @@ EXPORT_SYMBOL_GPL(kvm_alloc_rma);
> > void kvm_release_rma(struct kvm_rma_info *ri)
> > {
> > if (atomic_dec_and_test(&ri->use_count)) {
> > - kvm_release_cma(pfn_to_page(ri->base_pfn), kvm_rma_pages);
> > + cma_release(kvm_cma, pfn_to_page(ri->base_pfn), kvm_rma_pages);
> > kfree(ri);
> > }
> > }
> > @@ -134,13 +138,13 @@ struct page *kvm_alloc_hpt(unsigned long nr_pages)
> > /* Old CPUs require HPT aligned on a multiple of its size */
> > if (!cpu_has_feature(CPU_FTR_ARCH_206))
> > align_pages = nr_pages;
> > - return kvm_alloc_cma(nr_pages, align_pages);
> > + return cma_alloc(kvm_cma, nr_pages, get_order(align_pages));
> > }
> > EXPORT_SYMBOL_GPL(kvm_alloc_hpt);
> >
> > void kvm_release_hpt(struct page *page, unsigned long nr_pages)
> > {
> > - kvm_release_cma(page, nr_pages);
> > + cma_release(kvm_cma, page, nr_pages);
> > }
> > EXPORT_SYMBOL_GPL(kvm_release_hpt);
> >
> > @@ -179,7 +183,8 @@ void __init kvm_cma_reserve(void)
> > align_size = HPT_ALIGN_PAGES << PAGE_SHIFT;
> >
> > align_size = max(kvm_rma_pages << PAGE_SHIFT, align_size);
> > - kvm_cma_declare_contiguous(selected_size, align_size);
> > + cma_declare_contiguous(selected_size, 0, 0, align_size,
> > + KVM_CMA_CHUNK_ORDER - PAGE_SHIFT, &kvm_cma, false);
> > }
> > }
> >
> > diff --git a/arch/powerpc/kvm/book3s_hv_cma.c b/arch/powerpc/kvm/book3s_hv_cma.c
> > deleted file mode 100644
> > index d9d3d85..0000000
> > --- a/arch/powerpc/kvm/book3s_hv_cma.c
> > +++ /dev/null
> > @@ -1,240 +0,0 @@
> > -/*
> > - * Contiguous Memory Allocator for ppc KVM hash pagetable based on CMA
> > - * for DMA mapping framework
> > - *
> > - * Copyright IBM Corporation, 2013
> > - * Author Aneesh Kumar K.V <[email protected]>
> > - *
> > - * This program is free software; you can redistribute it and/or
> > - * modify it under the terms of the GNU General Public License as
> > - * published by the Free Software Foundation; either version 2 of the
> > - * License or (at your optional) any later version of the license.
> > - *
> > - */
> > -#define pr_fmt(fmt) "kvm_cma: " fmt
> > -
> > -#ifdef CONFIG_CMA_DEBUG
> > -#ifndef DEBUG
> > -# define DEBUG
> > -#endif
> > -#endif
> > -
> > -#include <linux/memblock.h>
> > -#include <linux/mutex.h>
> > -#include <linux/sizes.h>
> > -#include <linux/slab.h>
> > -
> > -#include "book3s_hv_cma.h"
> > -
> > -struct kvm_cma {
> > - unsigned long base_pfn;
> > - unsigned long count;
> > - unsigned long *bitmap;
> > -};
> > -
> > -static DEFINE_MUTEX(kvm_cma_mutex);
> > -static struct kvm_cma kvm_cma_area;
> > -
> > -/**
> > - * kvm_cma_declare_contiguous() - reserve area for contiguous memory handling
> > - * for kvm hash pagetable
> > - * @size: Size of the reserved memory.
> > - * @alignment: Alignment for the contiguous memory area
> > - *
> > - * This function reserves memory for kvm cma area. It should be
> > - * called by arch code when early allocator (memblock or bootmem)
> > - * is still activate.
> > - */
> > -long __init kvm_cma_declare_contiguous(phys_addr_t size, phys_addr_t alignment)
> > -{
> > - long base_pfn;
> > - phys_addr_t addr;
> > - struct kvm_cma *cma = &kvm_cma_area;
> > -
> > - pr_debug("%s(size %lx)\n", __func__, (unsigned long)size);
> > -
> > - if (!size)
> > - return -EINVAL;
> > - /*
> > - * Sanitise input arguments.
> > - * We should be pageblock aligned for CMA.
> > - */
> > - alignment = max(alignment, (phys_addr_t)(PAGE_SIZE << pageblock_order));
> > - size = ALIGN(size, alignment);
> > - /*
> > - * Reserve memory
> > - * Use __memblock_alloc_base() since
> > - * memblock_alloc_base() panic()s.
> > - */
> > - addr = __memblock_alloc_base(size, alignment, 0);
> > - if (!addr) {
> > - base_pfn = -ENOMEM;
> > - goto err;
> > - } else
> > - base_pfn = PFN_DOWN(addr);
> > -
> > - /*
> > - * Each reserved area must be initialised later, when more kernel
> > - * subsystems (like slab allocator) are available.
> > - */
> > - cma->base_pfn = base_pfn;
> > - cma->count = size >> PAGE_SHIFT;
> > - pr_info("CMA: reserved %ld MiB\n", (unsigned long)size / SZ_1M);
> > - return 0;
> > -err:
> > - pr_err("CMA: failed to reserve %ld MiB\n", (unsigned long)size / SZ_1M);
> > - return base_pfn;
> > -}
> > -
> > -/**
> > - * kvm_alloc_cma() - allocate pages from contiguous area
> > - * @nr_pages: Requested number of pages.
> > - * @align_pages: Requested alignment in number of pages
> > - *
> > - * This function allocates memory buffer for hash pagetable.
> > - */
> > -struct page *kvm_alloc_cma(unsigned long nr_pages, unsigned long align_pages)
> > -{
> > - int ret;
> > - struct page *page = NULL;
> > - struct kvm_cma *cma = &kvm_cma_area;
> > - unsigned long chunk_count, nr_chunk;
> > - unsigned long mask, pfn, pageno, start = 0;
> > -
> > -
> > - if (!cma || !cma->count)
> > - return NULL;
> > -
> > - pr_debug("%s(cma %p, count %lu, align pages %lu)\n", __func__,
> > - (void *)cma, nr_pages, align_pages);
> > -
> > - if (!nr_pages)
> > - return NULL;
> > - /*
> > - * align mask with chunk size. The bit tracks pages in chunk size
> > - */
> > - VM_BUG_ON(!is_power_of_2(align_pages));
> > - mask = (align_pages >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT)) - 1;
> > - BUILD_BUG_ON(PAGE_SHIFT > KVM_CMA_CHUNK_ORDER);
> > -
> > - chunk_count = cma->count >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT);
> > - nr_chunk = nr_pages >> (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT);
> > -
> > - mutex_lock(&kvm_cma_mutex);
> > - for (;;) {
> > - pageno = bitmap_find_next_zero_area(cma->bitmap, chunk_count,
> > - start, nr_chunk, mask);
> > - if (pageno >= chunk_count)
> > - break;
> > -
> > - pfn = cma->base_pfn + (pageno << (KVM_CMA_CHUNK_ORDER - PAGE_SHIFT));
> > - ret = alloc_contig_range(pfn, pfn + nr_pages, MIGRATE_CMA);
> > - if (ret == 0) {
> > - bitmap_set(cma->bitmap, pageno, nr_chunk);
> > - page = pfn_to_page(pfn);
> > - memset(pfn_to_kaddr(pfn), 0, nr_pages <<
> > PAGE_SHIFT);
>
>
> The above memset is missing in the generic code. May be
>
> diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> index 6f13ee6..8740b4c 100644
> --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
> +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> @@ -75,6 +75,7 @@ long kvmppc_alloc_hpt(struct kvm *kvm, u32 *htab_orderp)
> page = kvm_alloc_hpt(1 << (order - PAGE_SHIFT));
> if (page) {
> hpt = (unsigned long)pfn_to_kaddr(page_to_pfn(page));
> + memset((void *)hpt, 0, (1 << order));
> kvm->arch.hpt_cma_alloc = 1;
> } else
> --order;
>
>
>
> With that
>
> Tested-by: Aneesh Kumar K.V <[email protected]>
Okay. Will do.
Really thanks for testing!!
Thanks.
On Sat, Jun 14, 2014 at 12:55:39PM +0530, Aneesh Kumar K.V wrote:
> Joonsoo Kim <[email protected]> writes:
>
> > Currently, there are two users on CMA functionality, one is the DMA
> > subsystem and the other is the kvm on powerpc. They have their own code
> > to manage CMA reserved area even if they looks really similar.
> > From my guess, it is caused by some needs on bitmap management. Kvm side
> > wants to maintain bitmap not for 1 page, but for more size. Eventually it
> > use bitmap where one bit represents 64 pages.
> >
> > When I implement CMA related patches, I should change those two places
> > to apply my change and it seem to be painful to me. I want to change
> > this situation and reduce future code management overhead through
> > this patch.
> >
> > This change could also help developer who want to use CMA in their
> > new feature development, since they can use CMA easily without
> > copying & pasting this reserved area management code.
> >
> > v2:
> > Although this patchset looks very different with v1, the end result,
> > that is, mm/cma.c is same with v1's one. So I carry Ack to patch 6-7.
> >
> > Patch 1-5 prepare some features to cover ppc kvm's requirements.
> > Patch 6-7 generalize CMA reserved area management code and change users
> > to use it.
> > Patch 8-10 clean-up minor things.
>
>
> I wanted to test the ppc changes and found that the patch series doesn't apply
> against v3.15 . Do you have a kernel tree which I can clone to test this
> series ?
This is based on linux-next -next-20140610.
And my tree is on following link.
https://github.com/JoonsooKim/linux/tree/cma-general-v2.0-next-20140610
But, I think I'm late, because you have already added a Tested-by tag.
Thanks.
On Sat, Jun 14, 2014 at 02:23:59PM +0530, Aneesh Kumar K.V wrote:
> Joonsoo Kim <[email protected]> writes:
>
> > Now, we have general CMA reserved area management framework,
> > so use it for future maintainabilty. There is no functional change.
> >
> > Acked-by: Michal Nazarewicz <[email protected]>
> > Acked-by: Paolo Bonzini <[email protected]>
> > Signed-off-by: Joonsoo Kim <[email protected]>
>
> Need this. We may want to keep the VM_BUG_ON by moving
> KVM_CMA_CHUNK_ORDER around.
>
> diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> index 8056107..1932e0e 100644
> --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
> +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> @@ -37,8 +37,6 @@
> #include <asm/ppc-opcode.h>
> #include <asm/cputable.h>
>
> -#include "book3s_hv_cma.h"
> -
> /* POWER7 has 10-bit LPIDs, PPC970 has 6-bit LPIDs */
> #define MAX_LPID_970 63
>
> @@ -64,7 +62,6 @@ long kvmppc_alloc_hpt(struct kvm *kvm, u32 *htab_orderp)
> }
>
> kvm->arch.hpt_cma_alloc = 0;
> - VM_BUG_ON(order < KVM_CMA_CHUNK_ORDER);
> page = kvm_alloc_hpt(1 << (order - PAGE_SHIFT));
> if (page) {
> hpt = (unsigned long)pfn_to_kaddr(page_to_pfn(page));
>
>
>
> -aneesh
Okay.
So do you also want this?
@@ -131,16 +135,18 @@ struct page *kvm_alloc_hpt(unsigned long nr_pages)
{
unsigned long align_pages = HPT_ALIGN_PAGES;
+ VM_BUG_ON(get_order(nr_pages) < KVM_CMA_CHUNK_ORDER - PAGE_SHIFT);
+
/* Old CPUs require HPT aligned on a multiple of its size */
if (!cpu_has_feature(CPU_FTR_ARCH_206))
align_pages = nr_pages;
- return kvm_alloc_cma(nr_pages, align_pages);
+ return cma_alloc(kvm_cma, nr_pages, get_order(align_pages));
}
Thanks.
Joonsoo Kim <[email protected]> writes:
> On Sat, Jun 14, 2014 at 02:23:59PM +0530, Aneesh Kumar K.V wrote:
>> Joonsoo Kim <[email protected]> writes:
>>
>> > Now, we have general CMA reserved area management framework,
>> > so use it for future maintainabilty. There is no functional change.
>> >
>> > Acked-by: Michal Nazarewicz <[email protected]>
>> > Acked-by: Paolo Bonzini <[email protected]>
>> > Signed-off-by: Joonsoo Kim <[email protected]>
>>
>> Need this. We may want to keep the VM_BUG_ON by moving
>> KVM_CMA_CHUNK_ORDER around.
>>
>> diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
>> index 8056107..1932e0e 100644
>> --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
>> +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
>> @@ -37,8 +37,6 @@
>> #include <asm/ppc-opcode.h>
>> #include <asm/cputable.h>
>>
>> -#include "book3s_hv_cma.h"
>> -
>> /* POWER7 has 10-bit LPIDs, PPC970 has 6-bit LPIDs */
>> #define MAX_LPID_970 63
>>
>> @@ -64,7 +62,6 @@ long kvmppc_alloc_hpt(struct kvm *kvm, u32 *htab_orderp)
>> }
>>
>> kvm->arch.hpt_cma_alloc = 0;
>> - VM_BUG_ON(order < KVM_CMA_CHUNK_ORDER);
>> page = kvm_alloc_hpt(1 << (order - PAGE_SHIFT));
>> if (page) {
>> hpt = (unsigned long)pfn_to_kaddr(page_to_pfn(page));
>>
>>
>>
>> -aneesh
>
> Okay.
> So do you also want this?
>
> @@ -131,16 +135,18 @@ struct page *kvm_alloc_hpt(unsigned long nr_pages)
> {
> unsigned long align_pages = HPT_ALIGN_PAGES;
>
> + VM_BUG_ON(get_order(nr_pages) < KVM_CMA_CHUNK_ORDER - PAGE_SHIFT);
> +
> /* Old CPUs require HPT aligned on a multiple of its size */
> if (!cpu_has_feature(CPU_FTR_ARCH_206))
> align_pages = nr_pages;
> - return kvm_alloc_cma(nr_pages, align_pages);
> + return cma_alloc(kvm_cma, nr_pages, get_order(align_pages));
> }
That would also work.
Thanks
-aneesh
Joonsoo Kim <[email protected]> writes:
> On Sat, Jun 14, 2014 at 12:55:39PM +0530, Aneesh Kumar K.V wrote:
>> Joonsoo Kim <[email protected]> writes:
>>
>> > Currently, there are two users on CMA functionality, one is the DMA
>> > subsystem and the other is the kvm on powerpc. They have their own code
>> > to manage CMA reserved area even if they looks really similar.
>> > From my guess, it is caused by some needs on bitmap management. Kvm side
>> > wants to maintain bitmap not for 1 page, but for more size. Eventually it
>> > use bitmap where one bit represents 64 pages.
>> >
>> > When I implement CMA related patches, I should change those two places
>> > to apply my change and it seem to be painful to me. I want to change
>> > this situation and reduce future code management overhead through
>> > this patch.
>> >
>> > This change could also help developer who want to use CMA in their
>> > new feature development, since they can use CMA easily without
>> > copying & pasting this reserved area management code.
>> >
>> > v2:
>> > Although this patchset looks very different with v1, the end result,
>> > that is, mm/cma.c is same with v1's one. So I carry Ack to patch 6-7.
>> >
>> > Patch 1-5 prepare some features to cover ppc kvm's requirements.
>> > Patch 6-7 generalize CMA reserved area management code and change users
>> > to use it.
>> > Patch 8-10 clean-up minor things.
>>
>>
>> I wanted to test the ppc changes and found that the patch series doesn't apply
>> against v3.15 . Do you have a kernel tree which I can clone to test this
>> series ?
>
> This is based on linux-next -next-20140610.
> And my tree is on following link.
>
> https://github.com/JoonsooKim/linux/tree/cma-general-v2.0-next-20140610
>
> But, I think I'm late, because you have already added a Tested-by tag.
linux-next kexec is broken on ppc64, hence I hand picked few of
dependent patches for dma CMA on top of 3.15 and used that for testing.
-aneesh