A filesystem might run into a problem while calling __vmalloc(GFP_NOFS)
inside a lock.
It is expected than __vmalloc when called with GFP_NOFS should not
callback the filesystem code even incase of the increased memory
pressure. But the problem is that even if we pass this flag, __vmalloc
itself allocates memory with GFP_KERNEL.
Using GFP_KERNEL allocations may go into the memory reclaim path and try
to free memory by calling file system evict_inode function. Which might
lead into deadlock.
For further details
http://marc.info/?l=linux-mm&m=128942194520631&w=4
https://bugzilla.kernel.org/show_bug.cgi?id=30702
The patch passes the gfp allocation flag all the way down to those
allocating functions.
arch/arm/include/asm/pgalloc.h | 11 +++++-
arch/avr32/include/asm/pgalloc.h | 8 ++++-
arch/cris/include/asm/pgalloc.h | 10 ++++-
arch/frv/include/asm/pgalloc.h | 3 ++
arch/frv/include/asm/pgtable.h | 1 +
arch/frv/mm/pgalloc.c | 9 ++++-
arch/ia64/include/asm/pgalloc.h | 24 +++++++++++--
arch/m32r/include/asm/pgalloc.h | 11 ++++--
arch/m68k/include/asm/motorola_pgalloc.h | 20 +++++++++--
arch/m68k/include/asm/sun3_pgalloc.h | 14 ++++++--
arch/m68k/mm/memory.c | 9 ++++-
arch/microblaze/include/asm/pgalloc.h | 3 ++
arch/microblaze/mm/pgtable.c | 13 +++++--
arch/mips/include/asm/pgalloc.h | 22 ++++++++----
arch/mn10300/include/asm/pgalloc.h | 2 +
arch/mn10300/mm/pgtable.c | 10 ++++-
arch/parisc/include/asm/pgalloc.h | 21 ++++++++---
arch/powerpc/include/asm/pgalloc-32.h | 2 +
arch/powerpc/include/asm/pgalloc-64.h | 27 +++++++++++---
arch/powerpc/mm/pgtable_32.c | 10 ++++-
arch/s390/include/asm/pgalloc.h | 30 +++++++++++++---
arch/s390/mm/pgtable.c | 22 +++++++++---
arch/score/include/asm/pgalloc.h | 13 ++++---
arch/sh/include/asm/pgalloc.h | 8 ++++-
arch/sh/mm/pgtable.c | 8 ++++-
arch/sparc/include/asm/pgalloc_32.h | 5 +++
arch/sparc/include/asm/pgalloc_64.h | 17 ++++++++-
arch/tile/include/asm/pgalloc.h | 13 ++++++-
arch/tile/mm/pgtable.c | 10 ++++-
arch/um/include/asm/pgalloc.h | 1 +
arch/um/kernel/mem.c | 21 ++++++++---
arch/x86/include/asm/pgalloc.h | 17 ++++++++-
arch/x86/mm/pgtable.c | 8 ++++-
arch/xtensa/include/asm/pgalloc.h | 9 ++++-
arch/xtensa/mm/pgtable.c | 11 +++++-
include/asm-generic/4level-fixup.h | 8 +++-
include/asm-generic/pgtable-nopmd.h | 3 +-
include/asm-generic/pgtable-nopud.h | 1 +
include/linux/mm.h | 40 ++++++++++++++++-----
mm/memory.c | 14 ++++---
mm/vmalloc.c | 57 ++++++++++++++++++++----------
41 files changed, 427 insertions(+), 119 deletions(-)
linux/mm.h
- Added extra parameter gfp_t to __pud_alloc, __pmd_alloc, and __pte_alloc_kernel.
- Added function pud_alloc_with_mask() to allocate pud entry using the
specified allocation flag. The default modified pud_alloc() to call
pud_alloc_with_mask() with GFP_KERNEL.
- Similar modifications for pmd allocation.
- Added pte_alloc_kernel_with_mask() for pte allocations. The default
pte_alloc_kernel() modified to pte_alloc_kernel_with_mask() with GFP_KERNEL.
mm/memory.c
- Added an additional parameter gfp_t to __pte_alloc_kernel(). The function
calls __pte_alloc_one_kernel() passing the allocation flag.
- An additional argument gfp_t added to __pud_alloc(). It calls
__pud_alloc_one() and passes the allocation flag, down the call hierarchy.
- Similar modification for __pmd_alloc().
mm/vmalloc.c
- Added an extra parameter gfp_t to vmap_pte_range() and also changed the
function to call pte_alloc_kernel_with_mask() instead of pte_alloc_kernel().
- Added a gfp_t parameter to vmap_pmd_range(). Now it calls
pmd_alloc_with_mask(), also passes down the gfp_t to vmap_pte_range().
- Similar modification to vmap_pud_range().
- Added new function __vmap_page_range_noflush(), which is copy of the
function vmap_page_range_noflush(). This newly added function has an extra
allocation flag, which is passed down the call hierarchy.
- The function vmap_page_range_noflush() calls __vmap_page_range_noflush()
with allocation flag GFP_KERNEL.
- Added another version of function vmap_page_range() to pass down allocation
flag i.e. __vmap_page_range(). The function vmap_page_range() is changed to
call __vmap_page_range() with default GFP_KERNEL allocation flag.
- Added copy of function map_vm_area() i.e. __map_vm_area(). The map_vm_area()
is changed to call __map_vm_area() with default allocation flag.
These changes are done to fix the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
include/asm-generic/4level-fixup.h | 8 ++++-
include/asm-generic/pgtable-nopmd.h | 3 +-
include/asm-generic/pgtable-nopud.h | 1 +
include/linux/mm.h | 40 ++++++++++++++++++------
mm/memory.c | 14 +++++----
mm/vmalloc.c | 57 +++++++++++++++++++++++-----------
6 files changed, 85 insertions(+), 38 deletions(-)
diff --git a/include/asm-generic/4level-fixup.h b/include/asm-generic/4level-fixup.h
index 77ff547..f638309 100644
--- a/include/asm-generic/4level-fixup.h
+++ b/include/asm-generic/4level-fixup.h
@@ -10,10 +10,14 @@
#define pud_t pgd_t
-#define pmd_alloc(mm, pud, address) \
- ((unlikely(pgd_none(*(pud))) && __pmd_alloc(mm, pud, address))? \
+#define pmd_alloc_with_mask(mm, pud, address, mask) \
+ ((unlikely(pgd_none(*(pud))) && __pmd_alloc(mm, pud, address, mask))? \
NULL: pmd_offset(pud, address))
+#define pmd_alloc(mm, pud, address) \
+ pmd_alloc_with_mask(mm, pud, address, GFP_KERNEL)
+
+#define pud_alloc_with_mask(mm, pgd, address, mask) (pgd)
#define pud_alloc(mm, pgd, address) (pgd)
#define pud_offset(pgd, start) (pgd)
#define pud_none(pud) 0
diff --git a/include/asm-generic/pgtable-nopmd.h b/include/asm-generic/pgtable-nopmd.h
index 725612b..96ca8da 100644
--- a/include/asm-generic/pgtable-nopmd.h
+++ b/include/asm-generic/pgtable-nopmd.h
@@ -55,7 +55,8 @@ static inline pmd_t * pmd_offset(pud_t * pud, unsigned long address)
* allocating and freeing a pmd is trivial: the 1-entry pmd is
* inside the pud, so has no extra memory associated with it.
*/
-#define pmd_alloc_one(mm, address) NULL
+#define __pmd_alloc_one(mm, address, mask) NULL
+#define pmd_alloc_one(mm, address) NULL
static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
{
}
diff --git a/include/asm-generic/pgtable-nopud.h b/include/asm-generic/pgtable-nopud.h
index 810431d..5a21868 100644
--- a/include/asm-generic/pgtable-nopud.h
+++ b/include/asm-generic/pgtable-nopud.h
@@ -50,6 +50,7 @@ static inline pud_t * pud_offset(pgd_t * pgd, unsigned long address)
* allocating and freeing a pud is trivial: the 1-entry pud is
* inside the pgd, so has no extra memory associated with it.
*/
+#define __pud_alloc_one(mm, address, mask) NULL
#define pud_alloc_one(mm, address) NULL
#define pud_free(mm, x) do { } while (0)
#define __pud_free_tlb(tlb, x, a) do { } while (0)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 679300c..9ec4bd7 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1156,44 +1156,60 @@ static inline pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,
#ifdef __PAGETABLE_PUD_FOLDED
static inline int __pud_alloc(struct mm_struct *mm, pgd_t *pgd,
- unsigned long address)
+ unsigned long address, gfp_t gfp_mask)
{
return 0;
}
#else
-int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address);
+int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address,
+ gfp_t gfp_mask);
#endif
#ifdef __PAGETABLE_PMD_FOLDED
static inline int __pmd_alloc(struct mm_struct *mm, pud_t *pud,
- unsigned long address)
+ unsigned long address, gfp_t gfp_mask)
{
return 0;
}
#else
-int __pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address);
+int __pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address,
+ gfp_t gfp_mask);
#endif
int __pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
pmd_t *pmd, unsigned long address);
-int __pte_alloc_kernel(pmd_t *pmd, unsigned long address);
+int __pte_alloc_kernel(pmd_t *pmd, unsigned long address, gfp_t gfp_mask);
/*
* The following ifdef needed to get the 4level-fixup.h header to work.
* Remove it when 4level-fixup.h has been removed.
*/
#if defined(CONFIG_MMU) && !defined(__ARCH_HAS_4LEVEL_HACK)
-static inline pud_t *pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address)
+static inline pud_t *pud_alloc_with_mask(struct mm_struct *mm, pgd_t *pgd,
+ unsigned long address, gfp_t gfp_mask)
{
- return (unlikely(pgd_none(*pgd)) && __pud_alloc(mm, pgd, address))?
+ return (unlikely(pgd_none(*pgd)) && __pud_alloc(mm, pgd, address, gfp_mask))?
NULL: pud_offset(pgd, address);
}
-static inline pmd_t *pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address)
+static inline pud_t *pud_alloc(struct mm_struct *mm, pgd_t *pgd,
+ unsigned long address)
{
- return (unlikely(pud_none(*pud)) && __pmd_alloc(mm, pud, address))?
+ return pud_alloc_with_mask(mm, pgd, address, GFP_KERNEL);
+}
+
+static inline pmd_t *pmd_alloc_with_mask(struct mm_struct *mm, pud_t *pud,
+ unsigned long address, gfp_t gfp_mask)
+{
+ return (unlikely(pud_none(*pud)) && __pmd_alloc(mm, pud, address, gfp_mask))?
NULL: pmd_offset(pud, address);
}
+
+static inline pmd_t *pmd_alloc(struct mm_struct *mm, pud_t *pud,
+ unsigned long address)
+{
+ return pmd_alloc_with_mask(mm, pud, address, GFP_KERNEL);
+}
#endif /* CONFIG_MMU && !__ARCH_HAS_4LEVEL_HACK */
#if USE_SPLIT_PTLOCKS
@@ -1254,8 +1270,12 @@ static inline void pgtable_page_dtor(struct page *page)
pmd, address))? \
NULL: pte_offset_map_lock(mm, pmd, address, ptlp))
+#define pte_alloc_kernel_with_mask(pmd, address, mask) \
+ ((unlikely(pmd_none(*(pmd))) && __pte_alloc_kernel(pmd, address, mask))? \
+ NULL: pte_offset_kernel(pmd, address))
+
#define pte_alloc_kernel(pmd, address) \
- ((unlikely(pmd_none(*(pmd))) && __pte_alloc_kernel(pmd, address))? \
+ ((unlikely(pmd_none(*(pmd))) && __pte_alloc_kernel(pmd, address, GFP_KERNEL))? \
NULL: pte_offset_kernel(pmd, address))
extern void free_area_init(unsigned long * zones_size);
diff --git a/mm/memory.c b/mm/memory.c
index 5823698..dc4964e 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -433,9 +433,9 @@ int __pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
return 0;
}
-int __pte_alloc_kernel(pmd_t *pmd, unsigned long address)
+int __pte_alloc_kernel(pmd_t *pmd, unsigned long address, gfp_t gfp_mask)
{
- pte_t *new = pte_alloc_one_kernel(&init_mm, address);
+ pte_t *new = __pte_alloc_one_kernel(&init_mm, address, gfp_mask);
if (!new)
return -ENOMEM;
@@ -3343,9 +3343,10 @@ int handle_mm_fault(struct mm_struct *mm, struct vm_area_struct *vma,
* Allocate page upper directory.
* We've already handled the fast-path in-line.
*/
-int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address)
+int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address,
+ gfp_t gfp_mask)
{
- pud_t *new = pud_alloc_one(mm, address);
+ pud_t *new = __pud_alloc_one(mm, address, gfp_mask);
if (!new)
return -ENOMEM;
@@ -3366,9 +3367,10 @@ int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address)
* Allocate page middle directory.
* We've already handled the fast-path in-line.
*/
-int __pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address)
+int __pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address,
+ gfp_t gfp_mask)
{
- pmd_t *new = pmd_alloc_one(mm, address);
+ pmd_t *new = __pmd_alloc_one(mm, address, gfp_mask);
if (!new)
return -ENOMEM;
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index f9b1667..3df33fb 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -87,8 +87,8 @@ static void vunmap_page_range(unsigned long addr, unsigned long end)
} while (pgd++, addr = next, addr != end);
}
-static int vmap_pte_range(pmd_t *pmd, unsigned long addr,
- unsigned long end, pgprot_t prot, struct page **pages, int *nr)
+static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
+ pgprot_t prot, struct page **pages, int *nr, gfp_t gfp_mask)
{
pte_t *pte;
@@ -97,7 +97,7 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr,
* callers keep track of where we're up to.
*/
- pte = pte_alloc_kernel(pmd, addr);
+ pte = pte_alloc_kernel_with_mask(pmd, addr, gfp_mask);
if (!pte)
return -ENOMEM;
do {
@@ -114,34 +114,34 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr,
}
static int vmap_pmd_range(pud_t *pud, unsigned long addr,
- unsigned long end, pgprot_t prot, struct page **pages, int *nr)
+ unsigned long end, pgprot_t prot, struct page **pages, int *nr, gfp_t gfp_mask)
{
pmd_t *pmd;
unsigned long next;
- pmd = pmd_alloc(&init_mm, pud, addr);
+ pmd = pmd_alloc_with_mask(&init_mm, pud, addr, gfp_mask);
if (!pmd)
return -ENOMEM;
do {
next = pmd_addr_end(addr, end);
- if (vmap_pte_range(pmd, addr, next, prot, pages, nr))
+ if (vmap_pte_range(pmd, addr, next, prot, pages, nr, gfp_mask))
return -ENOMEM;
} while (pmd++, addr = next, addr != end);
return 0;
}
-static int vmap_pud_range(pgd_t *pgd, unsigned long addr,
- unsigned long end, pgprot_t prot, struct page **pages, int *nr)
+static int vmap_pud_range(pgd_t *pgd, unsigned long addr, unsigned long end,
+ pgprot_t prot, struct page **pages, int *nr, gfp_t gfp_mask)
{
pud_t *pud;
unsigned long next;
- pud = pud_alloc(&init_mm, pgd, addr);
+ pud = pud_alloc_with_mask(&init_mm, pgd, addr, gfp_mask);
if (!pud)
return -ENOMEM;
do {
next = pud_addr_end(addr, end);
- if (vmap_pmd_range(pud, addr, next, prot, pages, nr))
+ if (vmap_pmd_range(pud, addr, next, prot, pages, nr, gfp_mask))
return -ENOMEM;
} while (pud++, addr = next, addr != end);
return 0;
@@ -153,8 +153,8 @@ static int vmap_pud_range(pgd_t *pgd, unsigned long addr,
*
* Ie. pte at addr+N*PAGE_SIZE shall point to pfn corresponding to pages[N]
*/
-static int vmap_page_range_noflush(unsigned long start, unsigned long end,
- pgprot_t prot, struct page **pages)
+static int __vmap_page_range_noflush(unsigned long start, unsigned long end,
+ pgprot_t prot, struct page **pages, gfp_t gfp_mask)
{
pgd_t *pgd;
unsigned long next;
@@ -166,7 +166,7 @@ static int vmap_page_range_noflush(unsigned long start, unsigned long end,
pgd = pgd_offset_k(addr);
do {
next = pgd_addr_end(addr, end);
- err = vmap_pud_range(pgd, addr, next, prot, pages, &nr);
+ err = vmap_pud_range(pgd, addr, next, prot, pages, &nr, gfp_mask);
if (err)
return err;
} while (pgd++, addr = next, addr != end);
@@ -174,16 +174,29 @@ static int vmap_page_range_noflush(unsigned long start, unsigned long end,
return nr;
}
-static int vmap_page_range(unsigned long start, unsigned long end,
- pgprot_t prot, struct page **pages)
+
+static int vmap_page_range_noflush(unsigned long start, unsigned long end,
+ pgprot_t prot, struct page **pages)
+{
+ return __vmap_page_range_noflush(start, end, prot, pages, GFP_KERNEL);
+}
+
+static int __vmap_page_range(unsigned long start, unsigned long end,
+ pgprot_t prot, struct page **pages, gfp_t gfp_mask)
{
int ret;
- ret = vmap_page_range_noflush(start, end, prot, pages);
+ ret = __vmap_page_range_noflush(start, end, prot, pages, gfp_mask);
flush_cache_vmap(start, end);
return ret;
}
+static int vmap_page_range(unsigned long start, unsigned long end,
+ pgprot_t prot, struct page **pages)
+{
+ return __vmap_page_range(start, end, prot, pages, GFP_KERNEL);
+}
+
int is_vmalloc_or_module_addr(const void *x)
{
/*
@@ -1194,13 +1207,14 @@ void unmap_kernel_range(unsigned long addr, unsigned long size)
flush_tlb_kernel_range(addr, end);
}
-int map_vm_area(struct vm_struct *area, pgprot_t prot, struct page ***pages)
+int __map_vm_area(struct vm_struct *area, pgprot_t prot,
+ struct page ***pages, gfp_t gfp_mask)
{
unsigned long addr = (unsigned long)area->addr;
unsigned long end = addr + area->size - PAGE_SIZE;
int err;
- err = vmap_page_range(addr, end, prot, *pages);
+ err = __vmap_page_range(addr, end, prot, *pages, gfp_mask);
if (err > 0) {
*pages += err;
err = 0;
@@ -1208,6 +1222,11 @@ int map_vm_area(struct vm_struct *area, pgprot_t prot, struct page ***pages)
return err;
}
+
+int map_vm_area(struct vm_struct *area, pgprot_t prot, struct page ***pages)
+{
+ return __map_vm_area(area, prot, pages, GFP_KERNEL);
+}
EXPORT_SYMBOL_GPL(map_vm_area);
/*** Old vmalloc interfaces ***/
@@ -1522,7 +1541,7 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask,
area->pages[i] = page;
}
- if (map_vm_area(area, prot, &pages))
+ if (__map_vm_area(area, prot, &pages, gfp_mask))
goto fail;
return area->addr;
--
1.7.0.4
- Added __pte_alloc_one_kernel() function which is similar to
pte_alloc_one_kernel(), but accepts an additional argument to allow caller
to specify the allocation flag. The function pte_alloc_one_kernel() is
changed to call __pte_alloc_one_kernel().
- changes help in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/xtensa/include/asm/pgalloc.h | 9 ++++++++-
arch/xtensa/mm/pgtable.c | 11 +++++++++--
2 files changed, 17 insertions(+), 3 deletions(-)
diff --git a/arch/xtensa/include/asm/pgalloc.h b/arch/xtensa/include/asm/pgalloc.h
index 40cf9bc..d5a23ae 100644
--- a/arch/xtensa/include/asm/pgalloc.h
+++ b/arch/xtensa/include/asm/pgalloc.h
@@ -42,10 +42,17 @@ static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
extern struct kmem_cache *pgtable_cache;
+static inline pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address,
+ gfp_t gfp_mask)
+{
+ return kmem_cache_alloc(pgtable_cache, gfp_mask);
+}
+
static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
unsigned long address)
{
- return kmem_cache_alloc(pgtable_cache, GFP_KERNEL|__GFP_REPEAT);
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
diff --git a/arch/xtensa/mm/pgtable.c b/arch/xtensa/mm/pgtable.c
index 6979927..eff6c1d 100644
--- a/arch/xtensa/mm/pgtable.c
+++ b/arch/xtensa/mm/pgtable.c
@@ -12,13 +12,15 @@
#if (DCACHE_SIZE > PAGE_SIZE)
-pte_t* pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
+pte_t*
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address,
+ gfp_t gfp_mask)
{
pte_t *pte = NULL, *p;
int color = ADDR_COLOR(address);
int i;
- p = (pte_t*) __get_free_pages(GFP_KERNEL|__GFP_REPEAT, COLOR_ORDER);
+ p = (pte_t*) __get_free_pages(gfp_mask, COLOR_ORDER);
if (likely(p)) {
split_page(virt_to_page(p), COLOR_ORDER);
@@ -35,6 +37,11 @@ pte_t* pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
return pte;
}
+pte_t* pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
+{
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
+}
+
#ifdef PROFILING
int mask;
--
1.7.0.4
- Added __pud_alloc_one() function which is similar to pud_alloc_one(), but
accepts an additional argument of type gfp_t. The function pud_alloc_one()
calls __pud_alloc_one().
- similar changes for pmd allocations. The newly added function name is__pmd_alloc_one.
- changes for pte allocation. Added function __pte_alloc_one_kernel.
- changes help in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/x86/include/asm/pgalloc.h | 17 +++++++++++++++--
arch/x86/mm/pgtable.c | 8 +++++++-
2 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/pgalloc.h b/arch/x86/include/asm/pgalloc.h
index b4389a4..db83bd4 100644
--- a/arch/x86/include/asm/pgalloc.h
+++ b/arch/x86/include/asm/pgalloc.h
@@ -34,6 +34,7 @@ extern pgd_t *pgd_alloc(struct mm_struct *);
extern void pgd_free(struct mm_struct *mm, pgd_t *pgd);
extern pte_t *pte_alloc_one_kernel(struct mm_struct *, unsigned long);
+extern pte_t *__pte_alloc_one_kernel(struct mm_struct *, unsigned long, gfp_t);
extern pgtable_t pte_alloc_one(struct mm_struct *, unsigned long);
/* Should really implement gc for free page table pages. This could be
@@ -78,9 +79,15 @@ static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd,
#define pmd_pgtable(pmd) pmd_page(pmd)
#if PAGETABLE_LEVELS > 2
+static inline pmd_t *
+__pmd_alloc_one(struct mm_struct *mm, unsigned long addr, gfp_t gfp_mask)
+{
+ return (pmd_t *)get_zeroed_page(gfp_mask);
+}
+
static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return (pmd_t *)get_zeroed_page(GFP_KERNEL|__GFP_REPEAT);
+ return __pmd_alloc_one(mm, addr, GFP_KERNEL | __GFP_REPEAT);
}
static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
@@ -114,9 +121,15 @@ static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgd, pud_t *pud)
set_pgd(pgd, __pgd(_PAGE_TABLE | __pa(pud)));
}
+static inline pud_t *
+__pud_alloc_one(struct mm_struct *mm, unsigned long addr, gfp_t gfp_mask)
+{
+ return (pud_t *)get_zeroed_page(gfp_mask);
+}
+
static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return (pud_t *)get_zeroed_page(GFP_KERNEL|__GFP_REPEAT);
+ return __pud_alloc_one(mm, addr, GFP_KERNEL | __GFP_REPEAT);
}
static inline void pud_free(struct mm_struct *mm, pud_t *pud)
diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
index 0113d19..f5205b3 100644
--- a/arch/x86/mm/pgtable.c
+++ b/arch/x86/mm/pgtable.c
@@ -15,9 +15,15 @@
gfp_t __userpte_alloc_gfp = PGALLOC_GFP | PGALLOC_USER_GFP;
+pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
+{
+ return (pte_t *)__get_free_page(gfp_mask | __GFP_NOTRACK | __GFP_ZERO);
+}
+
pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
{
- return (pte_t *)__get_free_page(PGALLOC_GFP);
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long address)
--
1.7.0.4
- Added __pte_alloc_one_kernel() which is similar to pte_alloc_one_kernel()
but accepts an additional parameter to be used during page allocation.
- The function pte_alloc_one_kernel() has been modified to call
__pte_alloc_one_kernel().
- The similar changes are done for pmd allocations. The name of the newly
added function is __pmd_alloc_one().
- changes help in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/um/include/asm/pgalloc.h | 1 +
arch/um/kernel/mem.c | 21 +++++++++++++++------
2 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/arch/um/include/asm/pgalloc.h b/arch/um/include/asm/pgalloc.h
index 32c8ce4..8b6257e 100644
--- a/arch/um/include/asm/pgalloc.h
+++ b/arch/um/include/asm/pgalloc.h
@@ -27,6 +27,7 @@ extern pgd_t *pgd_alloc(struct mm_struct *);
extern void pgd_free(struct mm_struct *mm, pgd_t *pgd);
extern pte_t *pte_alloc_one_kernel(struct mm_struct *, unsigned long);
+extern pte_t *__pte_alloc_one_kernel(struct mm_struct *, unsigned long, gfp_t);
extern pgtable_t pte_alloc_one(struct mm_struct *, unsigned long);
static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
diff --git a/arch/um/kernel/mem.c b/arch/um/kernel/mem.c
index 8137ccc..e4caf17 100644
--- a/arch/um/kernel/mem.c
+++ b/arch/um/kernel/mem.c
@@ -284,12 +284,15 @@ void pgd_free(struct mm_struct *mm, pgd_t *pgd)
free_page((unsigned long) pgd);
}
-pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
+pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
{
- pte_t *pte;
+ return (pte_t *)__get_free_page(gfp_mask | __GFP_ZERO);
+}
- pte = (pte_t *)__get_free_page(GFP_KERNEL|__GFP_REPEAT|__GFP_ZERO);
- return pte;
+pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
+{
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long address)
@@ -303,15 +306,21 @@ pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long address)
}
#ifdef CONFIG_3_LEVEL_PGTABLES
-pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
+pmd_t *
+__pmd_alloc_one(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
{
- pmd_t *pmd = (pmd_t *) __get_free_page(GFP_KERNEL);
+ pmd_t *pmd = (pmd_t *) __get_free_page(gfp_mask);
if (pmd)
memset(pmd, 0, PAGE_SIZE);
return pmd;
}
+
+pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
+{
+ return __pmd_alloc_one(mm, address, GFP_KERNEL);
+}
#endif
void *uml_kmalloc(int size, int flags)
--
1.7.0.4
- Added function __pte_alloc_one() which is very similar to pte_alloc_one(),
but also accepts an additional gfp_t allocation flag.
- The pte_alloc_one() function calls __pte_alloc_one().
- Added function __pte_alloc_one_kernel() to accept and then pass the
allocation flag to __pte_alloc_one().
- The function pte_alloc_one_kernel() calls __pte_alloc_one_kernel().
- Added function __pmd_alloc_one to allocation pmd entries using the
specified allocation flag. It works by passing the allocation flag
to __pte_alloc_one().
- changes help in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/tile/include/asm/pgalloc.h | 13 ++++++++++++-
arch/tile/mm/pgtable.c | 10 ++++++++--
2 files changed, 20 insertions(+), 3 deletions(-)
diff --git a/arch/tile/include/asm/pgalloc.h b/arch/tile/include/asm/pgalloc.h
index cf52791..2dcad88 100644
--- a/arch/tile/include/asm/pgalloc.h
+++ b/arch/tile/include/asm/pgalloc.h
@@ -69,14 +69,23 @@ extern pgd_t *pgd_alloc(struct mm_struct *mm);
extern void pgd_free(struct mm_struct *mm, pgd_t *pgd);
extern pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long address);
+extern pgtable_t __pte_alloc_one(struct mm_struct *, unsigned long, gfp_t);
+
extern void pte_free(struct mm_struct *mm, struct page *pte);
#define pmd_pgtable(pmd) pmd_page(pmd)
static inline pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address,
+ gfp_t gfp_mask)
+{
+ return pfn_to_kaddr(page_to_pfn(__pte_alloc_one(mm, address, gfp_mask)));
+}
+
+static inline pte_t *
pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
{
- return pfn_to_kaddr(page_to_pfn(pte_alloc_one(mm, address)));
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
@@ -108,6 +117,8 @@ void shatter_pmd(pmd_t *pmd);
#define L1_USER_PGTABLE_ORDER L2_USER_PGTABLE_ORDER
#define pud_populate(mm, pud, pmd) \
pmd_populate_kernel((mm), (pmd_t *)(pud), (pte_t *)(pmd))
+#define __pmd_alloc_one(mm, addr, mask) \
+ ((pmd_t *)page_to_virt(__pte_alloc_one((mm), (addr), (mask))))
#define pmd_alloc_one(mm, addr) \
((pmd_t *)page_to_virt(pte_alloc_one((mm), (addr))))
#define pmd_free(mm, pmdp) \
diff --git a/arch/tile/mm/pgtable.c b/arch/tile/mm/pgtable.c
index 1f5430c..34ee920 100644
--- a/arch/tile/mm/pgtable.c
+++ b/arch/tile/mm/pgtable.c
@@ -218,9 +218,10 @@ void pgd_free(struct mm_struct *mm, pgd_t *pgd)
#define L2_USER_PGTABLE_PAGES (1 << L2_USER_PGTABLE_ORDER)
-struct page *pte_alloc_one(struct mm_struct *mm, unsigned long address)
+struct page *
+__pte_alloc_one(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
{
- gfp_t flags = GFP_KERNEL|__GFP_REPEAT|__GFP_ZERO|__GFP_COMP;
+ gfp_t flags = gfp_mask | __GFP_ZERO | __GFP_COMP;
struct page *p;
#ifdef CONFIG_HIGHPTE
@@ -235,6 +236,11 @@ struct page *pte_alloc_one(struct mm_struct *mm, unsigned long address)
return p;
}
+struct page *pte_alloc_one(struct mm_struct *mm, unsigned long address)
+{
+ return __pte_alloc_one(mm, address, GFP_KERNEL);
+}
+
/*
* Free page immediately (used in __pte_alloc if we raced with another
* process). We have to correct whatever pte_alloc_one() did before
--
1.7.0.4
32 bit architecture changes
- Added a new marcos __pmd_alloc_one() which allow caller to pass the gfp_t
allocation flag. The allocation flag is discarded, as the function
pmd_alloc_one() does not allocate memory. The code for pte allocation is
also modified the same way.
64 bit architecture changes
- Added function __pmd_alloc_one() to accept an additional allocation flag
from caller. Similar to pmd_alloc_one() the function __pmd_alloc_one()
calls quicklist_alloc() passing the allocation flag.
- pmd_alloc_one() calls __pmd_alloc_one().
- The similar changes are done for pte allocations.
- All these changes help in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/sparc/include/asm/pgalloc_32.h | 5 +++++
arch/sparc/include/asm/pgalloc_64.h | 17 +++++++++++++++--
2 files changed, 20 insertions(+), 2 deletions(-)
diff --git a/arch/sparc/include/asm/pgalloc_32.h b/arch/sparc/include/asm/pgalloc_32.h
index ca2b344..ad68597 100644
--- a/arch/sparc/include/asm/pgalloc_32.h
+++ b/arch/sparc/include/asm/pgalloc_32.h
@@ -41,6 +41,9 @@ BTFIXUPDEF_CALL(void, pgd_set, pgd_t *, pmd_t *)
BTFIXUPDEF_CALL(pmd_t *, pmd_alloc_one, struct mm_struct *, unsigned long)
#define pmd_alloc_one(mm, address) BTFIXUP_CALL(pmd_alloc_one)(mm, address)
+BTFIXUPDEF_CALL(pmd_t *, __pmd_alloc_one, struct mm_struct *, unsigned long, gfp_t)
+#define __pmd_alloc_one(mm, address, gfp_mask) BTFIXUP_CALL(pmd_alloc_one)(mm, address)
+
BTFIXUPDEF_CALL(void, free_pmd_fast, pmd_t *)
#define free_pmd_fast(pmd) BTFIXUP_CALL(free_pmd_fast)(pmd)
@@ -57,6 +60,8 @@ BTFIXUPDEF_CALL(pgtable_t , pte_alloc_one, struct mm_struct *, unsigned long)
#define pte_alloc_one(mm, address) BTFIXUP_CALL(pte_alloc_one)(mm, address)
BTFIXUPDEF_CALL(pte_t *, pte_alloc_one_kernel, struct mm_struct *, unsigned long)
#define pte_alloc_one_kernel(mm, addr) BTFIXUP_CALL(pte_alloc_one_kernel)(mm, addr)
+BTFIXUPDEF_CALL(pte_t *, __pte_alloc_one_kernel, struct mm_struct *, unsigned long, gfp_t)
+#define __pte_alloc_one_kernel(mm, addr, gfp_mask) BTFIXUP_CALL(pte_alloc_one_kernel)(mm, addr)
BTFIXUPDEF_CALL(void, free_pte_fast, pte_t *)
#define pte_free_kernel(mm, pte) BTFIXUP_CALL(free_pte_fast)(pte)
diff --git a/arch/sparc/include/asm/pgalloc_64.h b/arch/sparc/include/asm/pgalloc_64.h
index 5bdfa2c..a7952a5 100644
--- a/arch/sparc/include/asm/pgalloc_64.h
+++ b/arch/sparc/include/asm/pgalloc_64.h
@@ -26,9 +26,15 @@ static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
#define pud_populate(MM, PUD, PMD) pud_set(PUD, PMD)
+static inline pmd_t *
+__pmd_alloc_one(struct mm_struct *mm, unsigned long addr, gfp_t gfp_mask)
+{
+ return quicklist_alloc(0, gfp_mask, NULL);
+}
+
static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return quicklist_alloc(0, GFP_KERNEL, NULL);
+ return __pmd_alloc_one(mm, addr, GFP_KERNEL);
}
static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
@@ -36,10 +42,17 @@ static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
quicklist_free(0, NULL, pmd);
}
+static inline pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address,
+ gfp_t gfp_mask)
+{
+ return quicklist_alloc(0, gfp_mask, NULL);
+}
+
static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
unsigned long address)
{
- return quicklist_alloc(0, GFP_KERNEL, NULL);
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL);
}
static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
--
1.7.0.4
- Added __pte_alloc_one_kernel() function to allow caller to allocation flag.
Like pte_alloc_one_kernel() this newly added function calls
quicklist_alloc() and passes the allocation flag down the call hierarchy.
- The function pte_alloc_one_kernel() is modified to call
__pte_alloc_one_kernel() using the correct allocation flag.
- Similar changes for pmd allocations. Added function __pmd_alloc_one().
- The changes help in fixing the Bug 30702.
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/sh/include/asm/pgalloc.h | 8 +++++++-
arch/sh/mm/pgtable.c | 8 +++++++-
2 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/arch/sh/include/asm/pgalloc.h b/arch/sh/include/asm/pgalloc.h
index 8c00785..aaed989 100644
--- a/arch/sh/include/asm/pgalloc.h
+++ b/arch/sh/include/asm/pgalloc.h
@@ -31,10 +31,16 @@ static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd,
/*
* Allocate and free page tables.
*/
+static inline pte_t *__pte_alloc_one_kernel(struct mm_struct *mm,
+ unsigned long address, gfp_t gfp_mask)
+{
+ return quicklist_alloc(QUICK_PT, gfp_mask, NULL);
+}
+
static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
unsigned long address)
{
- return quicklist_alloc(QUICK_PT, GFP_KERNEL | __GFP_REPEAT, NULL);
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
diff --git a/arch/sh/mm/pgtable.c b/arch/sh/mm/pgtable.c
index 26e03a1..b938eb8 100644
--- a/arch/sh/mm/pgtable.c
+++ b/arch/sh/mm/pgtable.c
@@ -45,9 +45,15 @@ void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
set_pud(pud, __pud((unsigned long)pmd));
}
+pmd_t *
+__pmd_alloc_one(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
+{
+ return kmem_cache_alloc(pmd_cachep, gfp_mask | __GFP_ZERO);
+}
+
pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
{
- return kmem_cache_alloc(pmd_cachep, PGALLOC_GFP);
+ return __pmd_alloc_one(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
void pmd_free(struct mm_struct *mm, pmd_t *pmd)
--
1.7.0.4
- Added __pte_alloc_one_kernel() function to allocate zeroed pages using
passed allocation flag. The function pte_alloc_one_kernel() is changed to
call __pte_alloc_one_kernel() passing GFP_KERNEL | __GFP_REPEAT allocation
flags.
- Helps in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/score/include/asm/pgalloc.h | 13 +++++++------
1 files changed, 7 insertions(+), 6 deletions(-)
diff --git a/arch/score/include/asm/pgalloc.h b/arch/score/include/asm/pgalloc.h
index 059a61b..1a0a3a5 100644
--- a/arch/score/include/asm/pgalloc.h
+++ b/arch/score/include/asm/pgalloc.h
@@ -37,15 +37,16 @@ static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
free_pages((unsigned long)pgd, PGD_ORDER);
}
+static inline pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
+{
+ return (pte_t *) __get_free_pages(gfp_mask | __GFP_ZERO, PTE_ORDER);
+}
+
static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
unsigned long address)
{
- pte_t *pte;
-
- pte = (pte_t *) __get_free_pages(GFP_KERNEL|__GFP_REPEAT|__GFP_ZERO,
- PTE_ORDER);
-
- return pte;
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
static inline struct page *pte_alloc_one(struct mm_struct *mm,
--
1.7.0.4
Generic changes
- Added function __crst_table_alloc() which is similar to crst_table_alloc()
but accepts an extra argument gfp_t. The function uses given allocation
flag to allocate pages.
- The function crst_table_alloc() has been modified to call
__crst_table_alloc() and pass GFP_KERNEL allocation flag.
- Added a function __page_table_alloc() to allocate page table entries. This
function is allows caller to specify the page allocation flag. The
allocation flag is then passed to alloc_page(). The rest of the function is
copy of the original page_table_alloc().
- The function page_table_alloc() is modified to call __page_table_alloc()
and pass GFP_KERNEL | __GFP_REPEAT allocation flags.
changes for 32 (31) bit architecture (s390)
- Added macros __pud_alloc_one, __pmd_alloc_one which are similar to
pud_alloc_one and pmd_alloc_one respectively, but has an extra argument
for gfp_t.
changes for 64 bit architecture (s390x)
- Added __pud_alloc_one() allow caller to pass the memory allocation flag.
The function passes the allocation flag to __crst_table_alloc(). The rest
of the function is copy of crst_table_alloc().
- The function pud_alloc_one() is modified to call __pud_alloc_one() and
pass GFP_KERNEL allocation flag.
- The similar changes are performed for pmd allocations
- Added marco __pte_alloc_one_kernel() to accept additional argument gfp_t.
This argument is then passed to __page_table_alloc() to allocate the page
table entry.
- Modified pte_alloc_one_kernel() to call __pte_alloc_one_kernel() passing
GFP_KERNEL|__GFP_REPEAT allocation flags.
- The changes for both architectures help in fixing Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/s390/include/asm/pgalloc.h | 30 +++++++++++++++++++++++++-----
arch/s390/mm/pgtable.c | 22 +++++++++++++++++-----
2 files changed, 42 insertions(+), 10 deletions(-)
diff --git a/arch/s390/include/asm/pgalloc.h b/arch/s390/include/asm/pgalloc.h
index 082eb4e..459e89a 100644
--- a/arch/s390/include/asm/pgalloc.h
+++ b/arch/s390/include/asm/pgalloc.h
@@ -20,10 +20,12 @@
#define check_pgt_cache() do {} while (0)
unsigned long *crst_table_alloc(struct mm_struct *, int);
+unsigned long * __crst_table_alloc(struct mm_struct *, int , gfp_t);
void crst_table_free(struct mm_struct *, unsigned long *);
void crst_table_free_rcu(struct mm_struct *, unsigned long *);
unsigned long *page_table_alloc(struct mm_struct *);
+unsigned long *__page_table_alloc(struct mm_struct *, gfp_t);
void page_table_free(struct mm_struct *, unsigned long *);
void page_table_free_rcu(struct mm_struct *, unsigned long *);
void disable_noexec(struct mm_struct *, struct task_struct *);
@@ -62,9 +64,11 @@ static inline unsigned long pgd_entry_type(struct mm_struct *mm)
return _SEGMENT_ENTRY_EMPTY;
}
+#define __pud_alloc_one(mm,address,mask) ({ BUG(); ((pud_t *)2); })
#define pud_alloc_one(mm,address) ({ BUG(); ((pud_t *)2); })
#define pud_free(mm, x) do { } while (0)
+#define __pmd_alloc_one(mm,address,mask) ({ BUG(); ((pmd_t *)2); })
#define pmd_alloc_one(mm,address) ({ BUG(); ((pmd_t *)2); })
#define pmd_free(mm, x) do { } while (0)
@@ -88,22 +92,34 @@ static inline unsigned long pgd_entry_type(struct mm_struct *mm)
int crst_table_upgrade(struct mm_struct *, unsigned long limit);
void crst_table_downgrade(struct mm_struct *, unsigned long limit);
-static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long address)
+static inline pud_t *
+__pud_alloc_one(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
{
- unsigned long *table = crst_table_alloc(mm, mm->context.noexec);
+ unsigned long *table = __crst_table_alloc(mm, mm->context.noexec, gfp_mask);
if (table)
crst_table_init(table, _REGION3_ENTRY_EMPTY);
return (pud_t *) table;
}
+
+static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long address)
+{
+ return __pud_alloc_one(mm, address, GFP_KERNEL);
+}
#define pud_free(mm, pud) crst_table_free(mm, (unsigned long *) pud)
-static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long vmaddr)
+static inline pmd_t *
+__pmd_alloc_one(struct mm_struct *mm, unsigned long vmaddr, gfp_t gfp_mask)
{
- unsigned long *table = crst_table_alloc(mm, mm->context.noexec);
+ unsigned long *table = __crst_table_alloc(mm, mm->context.noexec, gfp_mask);
if (table)
crst_table_init(table, _SEGMENT_ENTRY_EMPTY);
return (pmd_t *) table;
}
+
+static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long vmaddr)
+{
+ return __pmd_alloc_one(mm, vmaddr, GFP_KERNEL);
+}
#define pmd_free(mm, pmd) crst_table_free(mm, (unsigned long *) pmd)
static inline void pgd_populate_kernel(struct mm_struct *mm,
@@ -172,7 +188,11 @@ static inline void pmd_populate(struct mm_struct *mm,
/*
* page table entry allocation/free routines.
*/
-#define pte_alloc_one_kernel(mm, vmaddr) ((pte_t *) page_table_alloc(mm))
+#define __pte_alloc_one_kernel(mm, vmaddr, mask) \
+ ((pte_t *) __page_table_alloc((mm), (mask)))
+#define pte_alloc_one_kernel(mm, vmaddr) \
+ ((pte_t *) __pte_alloc_one_kernel((mm), (vmaddr), GFP_KERNEL|__GFP_REPEAT)
+
#define pte_alloc_one(mm, vmaddr) ((pte_t *) page_table_alloc(mm))
#define pte_free_kernel(mm, pte) page_table_free(mm, (unsigned long *) pte)
diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c
index e1850c2..624854a 100644
--- a/arch/s390/mm/pgtable.c
+++ b/arch/s390/mm/pgtable.c
@@ -125,15 +125,16 @@ static int __init parse_vmalloc(char *arg)
}
early_param("vmalloc", parse_vmalloc);
-unsigned long *crst_table_alloc(struct mm_struct *mm, int noexec)
+unsigned long *
+__crst_table_alloc(struct mm_struct *mm, int noexec, gfp_t gfp_mask)
{
- struct page *page = alloc_pages(GFP_KERNEL, ALLOC_ORDER);
+ struct page *page = alloc_pages(gfp_mask, ALLOC_ORDER);
if (!page)
return NULL;
page->index = 0;
if (noexec) {
- struct page *shadow = alloc_pages(GFP_KERNEL, ALLOC_ORDER);
+ struct page *shadow = alloc_pages(gfp_mask, ALLOC_ORDER);
if (!shadow) {
__free_pages(page, ALLOC_ORDER);
return NULL;
@@ -146,6 +147,11 @@ unsigned long *crst_table_alloc(struct mm_struct *mm, int noexec)
return (unsigned long *) page_to_phys(page);
}
+unsigned long *crst_table_alloc(struct mm_struct *mm, int noexec)
+{
+ return __crst_table_alloc(mm, noexec, GFP_KERNEL);
+}
+
static void __crst_table_free(struct mm_struct *mm, unsigned long *table)
{
unsigned long *shadow = get_shadow_table(table);
@@ -267,7 +273,7 @@ void crst_table_downgrade(struct mm_struct *mm, unsigned long limit)
/*
* page table entry allocation/free routines.
*/
-unsigned long *page_table_alloc(struct mm_struct *mm)
+unsigned long *__page_table_alloc(struct mm_struct *mm, gfp_t gfp_mask)
{
struct page *page;
unsigned long *table;
@@ -284,7 +290,7 @@ unsigned long *page_table_alloc(struct mm_struct *mm)
}
if (!page) {
spin_unlock_bh(&mm->context.list_lock);
- page = alloc_page(GFP_KERNEL|__GFP_REPEAT);
+ page = alloc_page(gfp_mask);
if (!page)
return NULL;
pgtable_page_ctor(page);
@@ -309,6 +315,12 @@ unsigned long *page_table_alloc(struct mm_struct *mm)
return table;
}
+
+unsigned long *page_table_alloc(struct mm_struct *mm)
+{
+ return __page_table_alloc(mm, GFP_KERNEL | __GFP_REPEAT);
+}
+
static void __page_table_free(struct mm_struct *mm, unsigned long *table)
{
struct page *page;
--
1.7.0.4
changes for 32 bit architecture
- Added __pte_alloc_one_kernel() to allocated zeroed page using
allocation flag passed as an argument. If the slab allocator is not
initialized the allocation flag is not passed down the call hierarchy.
i.e. the call to early_get_page() is not modified.
- Changed pte_alloc_one_kernel() to call __pte_alloc_one_kernel() passing
correct gfp_t flags.
changes for 64 bit architecture
- Added __pud_alloc_one() which is similar to pud_alloc_one(). This newly
added function accepts allocation flag as a parameter and does the PUD
allocation using this GFP flag.
- The function pud_alloc_one() is changed to call __pud_alloc_one() passing
GFP_KERNEL | __GFP_REPEAT allocation flags.
- Similar changes for pmd (cache) and pte (page) allocations.
- The changes for both architectures help in fixing Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/powerpc/include/asm/pgalloc-32.h | 2 ++
arch/powerpc/include/asm/pgalloc-64.h | 27 ++++++++++++++++++++++-----
arch/powerpc/mm/pgtable_32.c | 10 ++++++++--
3 files changed, 32 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/include/asm/pgalloc-32.h b/arch/powerpc/include/asm/pgalloc-32.h
index 580cf73..21b7a94 100644
--- a/arch/powerpc/include/asm/pgalloc-32.h
+++ b/arch/powerpc/include/asm/pgalloc-32.h
@@ -35,6 +35,8 @@ extern void pgd_free(struct mm_struct *mm, pgd_t *pgd);
#endif
extern pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long addr);
+extern pte_t *__pte_alloc_one_kernel(struct mm_struct *, unsigned long, gfp_t);
+
extern pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long addr);
static inline void pgtable_free(void *table, unsigned index_size)
diff --git a/arch/powerpc/include/asm/pgalloc-64.h b/arch/powerpc/include/asm/pgalloc-64.h
index 292725c..e5ea650 100644
--- a/arch/powerpc/include/asm/pgalloc-64.h
+++ b/arch/powerpc/include/asm/pgalloc-64.h
@@ -51,10 +51,15 @@ static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
#define pgd_populate(MM, PGD, PUD) pgd_set(PGD, PUD)
+static inline pud_t *
+__pud_alloc_one(struct mm_struct *mm, unsigned long addr, gfp_t gfp_mask)
+{
+ return kmem_cache_alloc(PGT_CACHE(PUD_INDEX_SIZE), gfp_mask);
+}
+
static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return kmem_cache_alloc(PGT_CACHE(PUD_INDEX_SIZE),
- GFP_KERNEL|__GFP_REPEAT);
+ return __pud_alloc_one(mm, addr, GFP_KERNEL | __GFP_REPEAT);
}
static inline void pud_free(struct mm_struct *mm, pud_t *pud)
@@ -89,10 +94,15 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd,
#endif /* CONFIG_PPC_64K_PAGES */
+static inline pmd_t *
+__pmd_alloc_one(struct mm_struct *mm, unsigned long addr, gfp_t gfp_mask)
+{
+ return kmem_cache_alloc(PGT_CACHE(PMD_INDEX_SIZE), gfp_mask);
+}
+
static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return kmem_cache_alloc(PGT_CACHE(PMD_INDEX_SIZE),
- GFP_KERNEL|__GFP_REPEAT);
+ return __pmd_alloc_one(mm, addr, GFP_KERNEL | __GFP_REPEAT);
}
static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
@@ -100,10 +110,17 @@ static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
kmem_cache_free(PGT_CACHE(PMD_INDEX_SIZE), pmd);
}
+static inline pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address,
+ gfp_t gfp_mask)
+{
+ return (pte_t *)__get_free_page(gfp_mask | __GFP_ZERO);
+}
+
static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
unsigned long address)
{
- return (pte_t *)__get_free_page(GFP_KERNEL | __GFP_REPEAT | __GFP_ZERO);
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
diff --git a/arch/powerpc/mm/pgtable_32.c b/arch/powerpc/mm/pgtable_32.c
index 8dc41c0..736593f 100644
--- a/arch/powerpc/mm/pgtable_32.c
+++ b/arch/powerpc/mm/pgtable_32.c
@@ -95,14 +95,15 @@ void pgd_free(struct mm_struct *mm, pgd_t *pgd)
#endif
}
-__init_refok pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
+__init_refok pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
{
pte_t *pte;
extern int mem_init_done;
extern void *early_get_page(void);
if (mem_init_done) {
- pte = (pte_t *)__get_free_page(GFP_KERNEL|__GFP_REPEAT|__GFP_ZERO);
+ pte = (pte_t *)__get_free_page(gfp_mask | __GFP_ZERO);
} else {
pte = (pte_t *)early_get_page();
if (pte)
@@ -111,6 +112,11 @@ __init_refok pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long add
return pte;
}
+__init_refok pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
+{
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
+}
+
pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long address)
{
struct page *ptepage;
--
1.7.0.4
- Added __pte_alloc_one_kernel() function to allocate zeroed page using
passed allocation flag. The function pte_alloc_one_kernel() is changed to
call __pte_alloc_one_kernel() passing GFP_KERNEL | __GFP_REPEAT allocation
flags.
2-level page table support
- Added macro __pmd_alloc_one which is same as pmd_alloc_one, but has an extra
argument to pass gfp_t
3-level page table support
- Added function __pmd_alloc_one() to allocate pages using allocation flag.
The allocation flag is argument to the function.
- The function pmd_alloc_one() is changed to call __pmd_alloc_one(), passing
GFP_KERNEL | __GFP_REPEAT allocation flags.
- All these changes help in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/parisc/include/asm/pgalloc.h | 21 ++++++++++++++++-----
1 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/arch/parisc/include/asm/pgalloc.h b/arch/parisc/include/asm/pgalloc.h
index fc987a1..0284a43 100644
--- a/arch/parisc/include/asm/pgalloc.h
+++ b/arch/parisc/include/asm/pgalloc.h
@@ -61,15 +61,20 @@ static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgd, pmd_t *pmd)
(__u32)(__pa((unsigned long)pmd) >> PxD_VALUE_SHIFT));
}
-static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
+static inline pmd_t *
+__pmd_alloc_one(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
{
- pmd_t *pmd = (pmd_t *)__get_free_pages(GFP_KERNEL|__GFP_REPEAT,
- PMD_ORDER);
+ pmd_t *pmd = (pmd_t *)__get_free_pages(gfp_mask, PMD_ORDER);
if (pmd)
memset(pmd, 0, PAGE_SIZE<<PMD_ORDER);
return pmd;
}
+static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
+{
+ return __pmd_alloc_one(mm, address, GFP_KERNEL | __GFP_REPEAT);
+}
+
static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
{
#ifdef CONFIG_64BIT
@@ -90,6 +95,7 @@ static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
* inside the pgd, so has no extra memory associated with it.
*/
+#define __pmd_alloc_one(mm, addr, mask) ({ BUG(); ((pmd_t *)2); })
#define pmd_alloc_one(mm, addr) ({ BUG(); ((pmd_t *)2); })
#define pmd_free(mm, x) do { } while (0)
#define pgd_populate(mm, pmd, pte) BUG()
@@ -127,10 +133,15 @@ pte_alloc_one(struct mm_struct *mm, unsigned long address)
}
static inline pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long addr, gfp_t gfp_mask)
+{
+ return (pte_t *)__get_free_page(gfp_mask | __GFP_ZERO);
+}
+
+static inline pte_t *
pte_alloc_one_kernel(struct mm_struct *mm, unsigned long addr)
{
- pte_t *pte = (pte_t *)__get_free_page(GFP_KERNEL|__GFP_REPEAT|__GFP_ZERO);
- return pte;
+ return __pte_alloc_one_kernel(mm, addr, GFP_KERNEL | __GFP_REPEAT);
}
static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
--
1.7.0.4
- Added __pte_alloc_one_kernel to allocate page using specified GFP flag
- Changed pte_alloc_one_kernel to call __pte_alloc_one_kernel using
GFP_KERNEL | __GFP_REPEAT allocation flags
- Helps in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/mn10300/include/asm/pgalloc.h | 2 ++
arch/mn10300/mm/pgtable.c | 10 ++++++++--
2 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/arch/mn10300/include/asm/pgalloc.h b/arch/mn10300/include/asm/pgalloc.h
index 146bacf..35150ae 100644
--- a/arch/mn10300/include/asm/pgalloc.h
+++ b/arch/mn10300/include/asm/pgalloc.h
@@ -37,6 +37,8 @@ extern pgd_t *pgd_alloc(struct mm_struct *);
extern void pgd_free(struct mm_struct *, pgd_t *);
extern pte_t *pte_alloc_one_kernel(struct mm_struct *, unsigned long);
+extern pte_t *__pte_alloc_one_kernel(struct mm_struct *, unsigned long, gfp_t);
+
extern struct page *pte_alloc_one(struct mm_struct *, unsigned long);
static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
diff --git a/arch/mn10300/mm/pgtable.c b/arch/mn10300/mm/pgtable.c
index 450f7ba..05077b4 100644
--- a/arch/mn10300/mm/pgtable.c
+++ b/arch/mn10300/mm/pgtable.c
@@ -62,14 +62,20 @@ void set_pmd_pfn(unsigned long vaddr, unsigned long pfn, pgprot_t flags)
local_flush_tlb_one(vaddr);
}
-pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
+pte_t *__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address,
+ gfp_t gfp_mask)
{
- pte_t *pte = (pte_t *)__get_free_page(GFP_KERNEL|__GFP_REPEAT);
+ pte_t *pte = (pte_t *)__get_free_page(gfp_mask);
if (pte)
clear_page(pte);
return pte;
}
+pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
+{
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
+}
+
struct page *pte_alloc_one(struct mm_struct *mm, unsigned long address)
{
struct page *pte;
--
1.7.0.4
- Added __pmd_alloc_one to allocate pages using the given allocation flag.
- Changed pmd_alloc_one to call __pmd_alloc_one and pass
GFP_KERNEL | __GFP_REPEAT allocation flag.
- Added __pte_alloc_one_kernel to allocate zeroed pages using the given
allocation flag. The function pte_alloc_one_kernel() has been changed to
call __pte_alloc_one_kernel() passing GFP_KERNEL | __GFP_REPEAT
allocation flags.
- Helps in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/mips/include/asm/pgalloc.h | 22 +++++++++++++++-------
1 files changed, 15 insertions(+), 7 deletions(-)
diff --git a/arch/mips/include/asm/pgalloc.h b/arch/mips/include/asm/pgalloc.h
index 881d18b..f2c5439 100644
--- a/arch/mips/include/asm/pgalloc.h
+++ b/arch/mips/include/asm/pgalloc.h
@@ -64,14 +64,16 @@ static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
free_pages((unsigned long)pgd, PGD_ORDER);
}
+static inline pte_t *__pte_alloc_one_kernel(struct mm_struct *mm,
+ unsigned long address, gfp_t gfp_mask)
+{
+ return (pte_t *) __get_free_pages(gfp_mask | __GFP_ZERO, PTE_ORDER);
+}
+
static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
unsigned long address)
{
- pte_t *pte;
-
- pte = (pte_t *) __get_free_pages(GFP_KERNEL|__GFP_REPEAT|__GFP_ZERO, PTE_ORDER);
-
- return pte;
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
static inline struct page *pte_alloc_one(struct mm_struct *mm,
@@ -106,16 +108,22 @@ do { \
#ifndef __PAGETABLE_PMD_FOLDED
-static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
+static inline pmd_t *
+__pmd_alloc_one(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
{
pmd_t *pmd;
- pmd = (pmd_t *) __get_free_pages(GFP_KERNEL|__GFP_REPEAT, PMD_ORDER);
+ pmd = (pmd_t *) __get_free_pages(gfp_mask, PMD_ORDER);
if (pmd)
pmd_init((unsigned long)pmd, (unsigned long)invalid_pte_table);
return pmd;
}
+static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
+{
+ return __pmd_alloc_one(mm, address, GFP_KERNEL | __GFP_REPEAT);
+}
+
static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
{
free_pages((unsigned long)pmd, PMD_ORDER);
--
1.7.0.4
- Added function __pmd_alloc_one which is same as pmd_alloc_one, but accepts
an extra argument for gfp_t flag.
- Added __pte_alloc_one_kernel() to allocated zeroed page using
allocation flag passed as an argument. If the slab allocator is not
initialized the allocation flag is not passed down the call hierarchy.
i.e. early_get_page() is not modified.
- pte_alloc_one_kernel calls __pte_alloc_one_kernel using
GFP_KERNEL | __GFP_REPEAT allocation flag.
- Helps in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/microblaze/include/asm/pgalloc.h | 3 +++
arch/microblaze/mm/pgtable.c | 13 +++++++++----
2 files changed, 12 insertions(+), 4 deletions(-)
diff --git a/arch/microblaze/include/asm/pgalloc.h b/arch/microblaze/include/asm/pgalloc.h
index ebd3579..07864a1 100644
--- a/arch/microblaze/include/asm/pgalloc.h
+++ b/arch/microblaze/include/asm/pgalloc.h
@@ -106,9 +106,11 @@ extern inline void free_pgd_slow(pgd_t *pgd)
* the pgd will always be present..
*/
#define pmd_alloc_one_fast(mm, address) ({ BUG(); ((pmd_t *)1); })
+#define __pmd_alloc_one(mm, address, mask) ({ BUG(); ((pmd_t *)2); })
#define pmd_alloc_one(mm, address) ({ BUG(); ((pmd_t *)2); })
extern pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long addr);
+extern pte_t *__pte_alloc_one_kernel(struct mm_struct *, unsigned long, gfp_t);
static inline struct page *pte_alloc_one(struct mm_struct *mm,
unsigned long address)
@@ -175,6 +177,7 @@ extern inline void pte_free(struct mm_struct *mm, struct page *ptepage)
* We don't have any real pmd's, and this code never triggers because
* the pgd will always be present..
*/
+#define __pmd_alloc_one(mm, address, mask) ({ BUG(); ((pmd_t *)2); })
#define pmd_alloc_one(mm, address) ({ BUG(); ((pmd_t *)2); })
#define pmd_free(mm, x) do { } while (0)
#define __pmd_free_tlb(tlb, x, addr) pmd_free((tlb)->mm, x)
diff --git a/arch/microblaze/mm/pgtable.c b/arch/microblaze/mm/pgtable.c
index 59bf233..ae4d315 100644
--- a/arch/microblaze/mm/pgtable.c
+++ b/arch/microblaze/mm/pgtable.c
@@ -240,13 +240,12 @@ unsigned long iopa(unsigned long addr)
return pa;
}
-__init_refok pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
- unsigned long address)
+__init_refok pte_t *__pte_alloc_one_kernel(struct mm_struct *mm,
+ unsigned long address, gfp_t gfp_mask)
{
pte_t *pte;
if (mem_init_done) {
- pte = (pte_t *)__get_free_page(GFP_KERNEL |
- __GFP_REPEAT | __GFP_ZERO);
+ pte = (pte_t *)__get_free_page(gfp_mask | __GFP_ZERO);
} else {
pte = (pte_t *)early_get_page();
if (pte)
@@ -254,3 +253,9 @@ __init_refok pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
}
return pte;
}
+
+__init_refok pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
+ unsigned long address)
+{
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
+}
--
1.7.0.4
Changes for motorola (motorola_pgalloc.h)
- Added __pte_alloc_one_kernel to allocate zeroed page using given gfp_t flag
- Changed pte_alloc_one_kernel() to call __pte_alloc_one_kernel() passing
correct allocation flag.
- Added __pmd_alloc_one() to accepts gfp_t flag and pass it to
__get_pointer_table()
- Changed pmd_alloc_one() to call __pmd_alloc_one() passing GFP_KERNEL
allocation flag.
Changes for sun3 (sun3_pgalloc.h)
- Added function __pmd_alloc_one which is same as pmd_alloc_one, but accepts
an extra argument for gfp_t flag.
- Added function __pte_alloc_one_kernel to allocated page using
allocation flag passed as an argument.
- Changed pte_alloc_one_kernel to call __pte_alloc_one_kernel using
GFP_KERNEL allocation flag.
Generic Changes (mm/memory.c)
- Added __get_pointer_table() function which is same as get_pointer_table(),
but allows caller to pass the allocation flag.
- Changed get_pointer_table() to call __get_pointer_table() passing
GFP_KERNEL allocation flag.
- All these changes help in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/m68k/include/asm/motorola_pgalloc.h | 20 +++++++++++++++++---
arch/m68k/include/asm/sun3_pgalloc.h | 14 +++++++++++---
arch/m68k/mm/memory.c | 9 +++++++--
3 files changed, 35 insertions(+), 8 deletions(-)
diff --git a/arch/m68k/include/asm/motorola_pgalloc.h b/arch/m68k/include/asm/motorola_pgalloc.h
index 2f02f26..0d345e6 100644
--- a/arch/m68k/include/asm/motorola_pgalloc.h
+++ b/arch/m68k/include/asm/motorola_pgalloc.h
@@ -5,13 +5,16 @@
#include <asm/tlbflush.h>
extern pmd_t *get_pointer_table(void);
+extern pmd_t *__get_pointer_table (gfp_t);
extern int free_pointer_table(pmd_t *);
-static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
+static inline pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address,
+ gfp_t gfp_mask)
{
pte_t *pte;
- pte = (pte_t *)__get_free_page(GFP_KERNEL|__GFP_REPEAT|__GFP_ZERO);
+ pte = (pte_t *)__get_free_page(gfp_mask | __GFP_ZERO);
if (pte) {
__flush_page_to_ram(pte);
flush_tlb_kernel_page(pte);
@@ -21,6 +24,12 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long ad
return pte;
}
+static inline pte_t *
+pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
+{
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
+}
+
static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
{
cache_page(pte);
@@ -61,10 +70,15 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t page,
__free_page(page);
}
+static inline pmd_t *
+__pmd_alloc_one(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
+{
+ return __get_pointer_table(gfp_mask);
+}
static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
{
- return get_pointer_table();
+ return __pmd_alloc_one(mm, address, GFP_KERNEL);
}
static inline int pmd_free(struct mm_struct *mm, pmd_t *pmd)
diff --git a/arch/m68k/include/asm/sun3_pgalloc.h b/arch/m68k/include/asm/sun3_pgalloc.h
index 48d80d5..29cddb6 100644
--- a/arch/m68k/include/asm/sun3_pgalloc.h
+++ b/arch/m68k/include/asm/sun3_pgalloc.h
@@ -18,6 +18,7 @@
extern const char bad_pmd_string[];
+#define __pmd_alloc_one(mm,address,mask) ({ BUG(); ((pmd_t *)2); })
#define pmd_alloc_one(mm,address) ({ BUG(); ((pmd_t *)2); })
@@ -38,10 +39,11 @@ do { \
tlb_remove_page((tlb), pte); \
} while (0)
-static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
- unsigned long address)
+static inline pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address,
+ gfp_t gfp_mask)
{
- unsigned long page = __get_free_page(GFP_KERNEL|__GFP_REPEAT);
+ unsigned long page = __get_free_page(gfp_mask);
if (!page)
return NULL;
@@ -50,6 +52,12 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
return (pte_t *) (page);
}
+static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
+ unsigned long address)
+{
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
+}
+
static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
unsigned long address)
{
diff --git a/arch/m68k/mm/memory.c b/arch/m68k/mm/memory.c
index 34c77ce..1b2da3c 100644
--- a/arch/m68k/mm/memory.c
+++ b/arch/m68k/mm/memory.c
@@ -59,7 +59,7 @@ void __init init_pointer_table(unsigned long ptable)
return;
}
-pmd_t *get_pointer_table (void)
+pmd_t *__get_pointer_table (gfp_t gfp_mask)
{
ptable_desc *dp = ptable_list.next;
unsigned char mask = PD_MARKBITS (dp);
@@ -76,7 +76,7 @@ pmd_t *get_pointer_table (void)
void *page;
ptable_desc *new;
- if (!(page = (void *)get_zeroed_page(GFP_KERNEL)))
+ if (!(page = (void *)get_zeroed_page(gfp_mask)))
return NULL;
flush_tlb_kernel_page(page);
@@ -99,6 +99,11 @@ pmd_t *get_pointer_table (void)
return (pmd_t *) (page_address(PD_PAGE(dp)) + off);
}
+pmd_t *get_pointer_table (void)
+{
+ return __get_pointer_table(GFP_KERNEL);
+}
+
int free_pointer_table (pmd_t *ptable)
{
ptable_desc *dp;
--
1.7.0.4
- Added function __pmd_alloc_one which is same as pmd_alloc_one, but accepts
an extra argument for gfp_t flag.
- Added function __pte_alloc_one_kernel to allocated zeroed page using
allocation flag passed as an argument.
- Changed pte_alloc_one_kernel to call __pte_alloc_one_kernel using
GFP_KERNEL allocation flag.
- Helps in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/m32r/include/asm/pgalloc.h | 11 ++++++++---
1 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/arch/m32r/include/asm/pgalloc.h b/arch/m32r/include/asm/pgalloc.h
index 0fc7361..0c1e4ae 100644
--- a/arch/m32r/include/asm/pgalloc.h
+++ b/arch/m32r/include/asm/pgalloc.h
@@ -30,12 +30,16 @@ static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
free_page((unsigned long)pgd);
}
+static __inline__ pte_t *__pte_alloc_one_kernel(struct mm_struct *mm,
+ unsigned long address, gfp_t gfp_mask)
+{
+ return (pte_t *)__get_free_page(gfp_mask | __GFP_ZERO);
+}
+
static __inline__ pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
unsigned long address)
{
- pte_t *pte = (pte_t *)__get_free_page(GFP_KERNEL|__GFP_ZERO);
-
- return pte;
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL);
}
static __inline__ pgtable_t pte_alloc_one(struct mm_struct *mm,
@@ -66,6 +70,7 @@ static inline void pte_free(struct mm_struct *mm, pgtable_t pte)
* (In the PAE case we free the pmds as part of the pgd.)
*/
+#define __pmd_alloc_one(mm, addr,mask) ({ BUG(); ((pmd_t *)2); })
#define pmd_alloc_one(mm, addr) ({ BUG(); ((pmd_t *)2); })
#define pmd_free(mm, x) do { } while (0)
#define __pmd_free_tlb(tlb, x, addr) do { } while (0)
--
1.7.0.4
- Added __pud_alloc_one which allows caller to specify the gfp_t flag.
Like pud_alloc_one the function calls quicklist_alloc() and passes
the given allocation flag.
- Changed pud_alloc_one to call __pud_alloc_one using GFP_KERNEL.
- Added similar changes for pte and pmd allocation functions.
- Helps in fixing the Bug 30702.
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/ia64/include/asm/pgalloc.h | 24 +++++++++++++++++++++---
1 files changed, 21 insertions(+), 3 deletions(-)
diff --git a/arch/ia64/include/asm/pgalloc.h b/arch/ia64/include/asm/pgalloc.h
index 96a8d92..0e46e47 100644
--- a/arch/ia64/include/asm/pgalloc.h
+++ b/arch/ia64/include/asm/pgalloc.h
@@ -39,9 +39,15 @@ pgd_populate(struct mm_struct *mm, pgd_t * pgd_entry, pud_t * pud)
pgd_val(*pgd_entry) = __pa(pud);
}
+static inline pud_t *
+__pud_alloc_one(struct mm_struct *mm, unsigned long addr, gfp_t gfp_mask)
+{
+ return quicklist_alloc(0, gfp_mask, NULL);
+}
+
static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return quicklist_alloc(0, GFP_KERNEL, NULL);
+ return __pud_alloc_one(mm, addr, GFP_KERNEL);
}
static inline void pud_free(struct mm_struct *mm, pud_t *pud)
@@ -57,9 +63,15 @@ pud_populate(struct mm_struct *mm, pud_t * pud_entry, pmd_t * pmd)
pud_val(*pud_entry) = __pa(pmd);
}
+static inline pmd_t *
+__pmd_alloc_one(struct mm_struct *mm, unsigned long addr, gfp_t gfp_mask)
+{
+ return quicklist_alloc(0, gfp_mask, NULL);
+}
+
static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return quicklist_alloc(0, GFP_KERNEL, NULL);
+ return __pmd_alloc_one(mm, addr, GFP_KERNEL);
}
static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
@@ -95,10 +107,16 @@ static inline pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long addr)
return page;
}
+static inline pte_t *__pte_alloc_one_kernel(struct mm_struct *mm,
+ unsigned long addr, gfp_t gfp_mask)
+{
+ return quicklist_alloc(0, gfp_mask, NULL);
+}
+
static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
unsigned long addr)
{
- return quicklist_alloc(0, GFP_KERNEL, NULL);
+ return __pte_alloc_one_kernel(mm, addr, GFP_KERNEL);
}
static inline void pte_free(struct mm_struct *mm, pgtable_t pte)
--
1.7.0.4
- Added __pmd_alloc_one and __pud_alloc_one, which are same as pmd_alloc_one
and pud_alloc_one respectively, but has extra parameter to specify the
allocation flag
- Added __pte_alloc_one_kernel to allocate page using specified GFP flag
- Changed pte_alloc_one_kernel to call __pte_alloc_one_kernel passing
GFP_KERNEL allocation flag
- Helps in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/frv/include/asm/pgalloc.h | 3 +++
arch/frv/include/asm/pgtable.h | 1 +
arch/frv/mm/pgalloc.c | 9 +++++++--
3 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/arch/frv/include/asm/pgalloc.h b/arch/frv/include/asm/pgalloc.h
index 416d19a..bfc4f7c 100644
--- a/arch/frv/include/asm/pgalloc.h
+++ b/arch/frv/include/asm/pgalloc.h
@@ -35,8 +35,10 @@ extern pgd_t *pgd_alloc(struct mm_struct *);
extern void pgd_free(struct mm_struct *mm, pgd_t *);
extern pte_t *pte_alloc_one_kernel(struct mm_struct *, unsigned long);
+extern pte_t *__pte_alloc_one_kernel(struct mm_struct *, unsigned long, gfp_t);
extern pgtable_t pte_alloc_one(struct mm_struct *, unsigned long);
+extern pgtable_t __pte_alloc_one(struct mm_struct *, unsigned long, gfp_t);
static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
{
@@ -60,6 +62,7 @@ do { \
* inside the pgd, so has no extra memory associated with it.
* (In the PAE case we free the pmds as part of the pgd.)
*/
+#define __pmd_alloc_one(mm, addr,mask) ({ BUG(); ((pmd_t *) 2); })
#define pmd_alloc_one(mm, addr) ({ BUG(); ((pmd_t *) 2); })
#define pmd_free(mm, x) do { } while (0)
#define __pmd_free_tlb(tlb,x,a) do { } while (0)
diff --git a/arch/frv/include/asm/pgtable.h b/arch/frv/include/asm/pgtable.h
index 6bc241e..698e280 100644
--- a/arch/frv/include/asm/pgtable.h
+++ b/arch/frv/include/asm/pgtable.h
@@ -223,6 +223,7 @@ static inline pud_t *pud_offset(pgd_t *pgd, unsigned long address)
* allocating and freeing a pud is trivial: the 1-entry pud is
* inside the pgd, so has no extra memory associated with it.
*/
+#define __pud_alloc_one(mm, address, mask) NULL
#define pud_alloc_one(mm, address) NULL
#define pud_free(mm, x) do { } while (0)
#define __pud_free_tlb(tlb, x, address) do { } while (0)
diff --git a/arch/frv/mm/pgalloc.c b/arch/frv/mm/pgalloc.c
index c42c83d..374cd2c 100644
--- a/arch/frv/mm/pgalloc.c
+++ b/arch/frv/mm/pgalloc.c
@@ -20,14 +20,19 @@
pgd_t swapper_pg_dir[PTRS_PER_PGD] __attribute__((aligned(PAGE_SIZE)));
-pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
+pte_t *__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
{
- pte_t *pte = (pte_t *)__get_free_page(GFP_KERNEL|__GFP_REPEAT);
+ pte_t *pte = (pte_t *)__get_free_page(gfp_mask);
if (pte)
clear_page(pte);
return pte;
}
+pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
+{
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
+}
+
pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long address)
{
struct page *page;
--
1.7.0.4
- Added __pte_alloc_one_kernel to allocate page using specified GFP flag
- Changed pte_alloc_one_kernel to call __pte_alloc_one_kernel using GFP_KERNEL allocation flag
- Helps in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/cris/include/asm/pgalloc.h | 10 ++++++++--
1 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/arch/cris/include/asm/pgalloc.h b/arch/cris/include/asm/pgalloc.h
index 6da975d..20254ee 100644
--- a/arch/cris/include/asm/pgalloc.h
+++ b/arch/cris/include/asm/pgalloc.h
@@ -22,10 +22,16 @@ static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
free_page((unsigned long)pgd);
}
+static inline pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address,
+ gfp_t gfp_mask)
+{
+ return (pte_t *) __get_free_page(gfp_mask | __GFP_ZERO);
+}
+
static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
{
- pte_t *pte = (pte_t *)__get_free_page(GFP_KERNEL|__GFP_REPEAT|__GFP_ZERO);
- return pte;
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
static inline pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long address)
--
1.7.0.4
- Added __pte_alloc_one_kernel to allocate page using specified GFP flag
- Changed pte_alloc_one_kernel to call __pte_alloc_one_kernel newly added function using GFP_KERNEL allocation flag
- Helps in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/avr32/include/asm/pgalloc.h | 8 +++++++-
1 files changed, 7 insertions(+), 1 deletions(-)
diff --git a/arch/avr32/include/asm/pgalloc.h b/arch/avr32/include/asm/pgalloc.h
index bc7e8ae..cb88080 100644
--- a/arch/avr32/include/asm/pgalloc.h
+++ b/arch/avr32/include/asm/pgalloc.h
@@ -51,10 +51,16 @@ static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
quicklist_free(QUICK_PGD, NULL, pgd);
}
+static inline pte_t *__pte_alloc_one_kernel(struct mm_struct *mm,
+ unsigned long address, gfp_t gfp_mask)
+{
+ return quicklist_alloc(QUICK_PT, gfp_mask, NULL);
+}
+
static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
unsigned long address)
{
- return quicklist_alloc(QUICK_PT, GFP_KERNEL | __GFP_REPEAT, NULL);
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
--
1.7.0.4
- Added __pmd_alloc_one, which is same as pmd_alloc_one, but has extra parameter to specify the allocation flag
- Added __pte_alloc_one_kernel to allocate page using specified GFP flag
- Changed pte_alloc_one_kernel to call __pte_alloc_one_kernel newly added function using GFP_KERNEL allocation flag
- Helps in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/arm/include/asm/pgalloc.h | 11 +++++++++--
1 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/arch/arm/include/asm/pgalloc.h b/arch/arm/include/asm/pgalloc.h
index 22de005..0696068 100644
--- a/arch/arm/include/asm/pgalloc.h
+++ b/arch/arm/include/asm/pgalloc.h
@@ -28,6 +28,7 @@
/*
* Since we have only two-level page tables, these are trivial
*/
+#define __pmd_alloc_one(mm,addr,mask) ({ BUG(); ((pmd_t *)2); })
#define pmd_alloc_one(mm,addr) ({ BUG(); ((pmd_t *)2); })
#define pmd_free(mm, pmd) do { } while (0)
#define pgd_populate(mm,pmd,pte) BUG()
@@ -59,17 +60,23 @@ static inline void clean_pte_table(pte_t *pte)
* +------------+
*/
static inline pte_t *
-pte_alloc_one_kernel(struct mm_struct *mm, unsigned long addr)
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long addr, gfp_t gfp_mask)
{
pte_t *pte;
- pte = (pte_t *)__get_free_page(PGALLOC_GFP);
+ pte = (pte_t *)__get_free_page(gfp_mask | __GFP_NOTRACK | __GFP_ZERO);
if (pte)
clean_pte_table(pte);
return pte;
}
+static inline pte_t *
+pte_alloc_one_kernel(struct mm_struct *mm, unsigned long addr)
+{
+ return __pte_alloc_one_kernel(mm, addr, GFP_KERNEL | __GFP_REPEAT);
+}
+
static inline pgtable_t
pte_alloc_one(struct mm_struct *mm, unsigned long addr)
{
--
1.7.0.4
- Added __pmd_alloc_one, __pte_alloc_one_kernel to allocate pages using specified GFP flag
- changed pmd_alloc_one, pte_alloc_one_kernel to call these newly added function using GFP_KERNEL
- helps in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/alpha/include/asm/pgalloc.h | 18 ++++++++++++++----
1 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/arch/alpha/include/asm/pgalloc.h b/arch/alpha/include/asm/pgalloc.h
index bc2a0da..81281ac 100644
--- a/arch/alpha/include/asm/pgalloc.h
+++ b/arch/alpha/include/asm/pgalloc.h
@@ -38,10 +38,15 @@ pgd_free(struct mm_struct *mm, pgd_t *pgd)
}
static inline pmd_t *
+__pmd_alloc_one(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
+{
+ return (pmd_t *)__get_free_page(gfp_mask | __GFP_ZERO);
+}
+
+static inline pmd_t *
pmd_alloc_one(struct mm_struct *mm, unsigned long address)
{
- pmd_t *ret = (pmd_t *)__get_free_page(GFP_KERNEL|__GFP_REPEAT|__GFP_ZERO);
- return ret;
+ return __pmd_alloc_one(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
static inline void
@@ -51,10 +56,15 @@ pmd_free(struct mm_struct *mm, pmd_t *pmd)
}
static inline pte_t *
+__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
+{
+ return (pte_t *)__get_free_page(gfp_mask | __GFP_ZERO);
+}
+
+static inline pte_t *
pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
{
- pte_t *pte = (pte_t *)__get_free_page(GFP_KERNEL|__GFP_REPEAT|__GFP_ZERO);
- return pte;
+ return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
}
static inline void
--
1.7.0.4
On Fri, 18 Mar 2011, Prasad Joshi wrote:
> diff --git a/arch/x86/include/asm/pgalloc.h b/arch/x86/include/asm/pgalloc.h
> index b4389a4..db83bd4 100644
> --- a/arch/x86/include/asm/pgalloc.h
> +++ b/arch/x86/include/asm/pgalloc.h
> @@ -34,6 +34,7 @@ extern pgd_t *pgd_alloc(struct mm_struct *);
> extern void pgd_free(struct mm_struct *mm, pgd_t *pgd);
>
> extern pte_t *pte_alloc_one_kernel(struct mm_struct *, unsigned long);
> +extern pte_t *__pte_alloc_one_kernel(struct mm_struct *, unsigned long, gfp_t);
> extern pgtable_t pte_alloc_one(struct mm_struct *, unsigned long);
>
> /* Should really implement gc for free page table pages. This could be
> @@ -78,9 +79,15 @@ static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd,
> #define pmd_pgtable(pmd) pmd_page(pmd)
>
> #if PAGETABLE_LEVELS > 2
> +static inline pmd_t *
> +__pmd_alloc_one(struct mm_struct *mm, unsigned long addr, gfp_t gfp_mask)
> +{
> + return (pmd_t *)get_zeroed_page(gfp_mask);
> +}
There should be no break between the function type and its name.
> +
> static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
> {
> - return (pmd_t *)get_zeroed_page(GFP_KERNEL|__GFP_REPEAT);
> + return __pmd_alloc_one(mm, addr, GFP_KERNEL | __GFP_REPEAT);
> }
>
> static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
> @@ -114,9 +121,15 @@ static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgd, pud_t *pud)
> set_pgd(pgd, __pgd(_PAGE_TABLE | __pa(pud)));
> }
>
> +static inline pud_t *
> +__pud_alloc_one(struct mm_struct *mm, unsigned long addr, gfp_t gfp_mask)
> +{
> + return (pud_t *)get_zeroed_page(gfp_mask);
> +}
> +
> static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
> {
> - return (pud_t *)get_zeroed_page(GFP_KERNEL|__GFP_REPEAT);
> + return __pud_alloc_one(mm, addr, GFP_KERNEL | __GFP_REPEAT);
> }
>
> static inline void pud_free(struct mm_struct *mm, pud_t *pud)
> diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
> index 0113d19..f5205b3 100644
> --- a/arch/x86/mm/pgtable.c
> +++ b/arch/x86/mm/pgtable.c
> @@ -15,9 +15,15 @@
>
> gfp_t __userpte_alloc_gfp = PGALLOC_GFP | PGALLOC_USER_GFP;
>
> +pte_t *
> +__pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address, gfp_t gfp_mask)
> +{
> + return (pte_t *)__get_free_page(gfp_mask | __GFP_NOTRACK | __GFP_ZERO);
> +}
> +
> pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
> {
> - return (pte_t *)__get_free_page(PGALLOC_GFP);
> + return __pte_alloc_one_kernel(mm, address, GFP_KERNEL | __GFP_REPEAT);
PGALLOC_GFP represents the gfp flags intended for the allocators in this
file, there's no need to replace its use with GFP_KERNEL | __GFP_REPEAT,
we don't care that __pte_alloc_one_kernel() ORs this with __GFP_NOTRACK
and __GFP_ZERO.
> }
>
> pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long address)
On Fri, 18 Mar 2011 19:56:44 +0000
Prasad Joshi <[email protected]> wrote:
> - Added function __crst_table_alloc() which is similar to crst_table_alloc()
> but accepts an extra argument gfp_t. The function uses given allocation
> flag to allocate pages.
>
> - Added a function __page_table_alloc() to allocate page table entries. This
> function is allows caller to specify the page allocation flag. The
> allocation flag is then passed to alloc_page(). The rest of the function is
> copy of the original page_table_alloc().
The approach of this patch series seems straightforward, the only nitpick I
have is the fact that two new functions __crst_table_alloc/__page_table_alloc
are introduced. There aren't many call sites for the two original functions,
namely 4 for crst_table_alloc and 3 for page_table_alloc. Why not add the
gfp flag GFP_KERNEL to these call sites? Then the two additional functions
would not be needed.
--
blue skies,
Martin.
"Reality continues to ruin my life." - Calvin.
On Mon, Mar 21, 2011 at 09:34:12AM +0100, Martin Schwidefsky wrote:
> On Fri, 18 Mar 2011 19:56:44 +0000
> Prasad Joshi <[email protected]> wrote:
>
> > - Added function __crst_table_alloc() which is similar to crst_table_alloc()
> > but accepts an extra argument gfp_t. The function uses given allocation
> > flag to allocate pages.
> >
> > - Added a function __page_table_alloc() to allocate page table entries. This
> > function is allows caller to specify the page allocation flag. The
> > allocation flag is then passed to alloc_page(). The rest of the function is
> > copy of the original page_table_alloc().
>
> The approach of this patch series seems straightforward, the only nitpick I
> have is the fact that two new functions __crst_table_alloc/__page_table_alloc
> are introduced. There aren't many call sites for the two original functions,
> namely 4 for crst_table_alloc and 3 for page_table_alloc. Why not add the
> gfp flag GFP_KERNEL to these call sites? Then the two additional functions
> would not be needed.
Thanks a lot Martin for your reply. Here is a new patch, which the changes
you suggested.
Generic changes
- Changed crst_table_alloc() to accept a new parameter gfp_t, allocate pages
using the passed allocation flag.
- crst_table_upgrade() calls crst_table_alloc() with GFP_KERNEL.
- page_table_alloc() is also changned to accept a parameter gfp_t and allocate
page using the passed allocation flag.
- vmem_pte_alloc() calls page_table_alloc() with GFP_KERNEL | __GFP_REPEAT
changes for 32 (31) bit architecture (s390)
- Added macros __pud_alloc_one, __pmd_alloc_one which are similar to
pud_alloc_one and pmd_alloc_one respectively, but has an extra argument
for gfp_t.
changes for 64 bit architecture (s390x)
- Added __pud_alloc_one() allow caller to pass the memory allocation flag.
The function passes the allocation flag to crst_table_alloc().
- The function pud_alloc_one() is modified to call __pud_alloc_one() and
pass GFP_KERNEL allocation flag.
- The similar changes are performed for pmd allocations
- pgd_populate_kernel() calls crst_table_alloc() with GFP_KERNEL
- Added marco __pte_alloc_one_kernel() to accept additional argument gfp_t.
This argument is then passed to page_table_alloc() to allocate the page
table entry.
- Modified pte_alloc_one_kernel() to call __pte_alloc_one_kernel() passing
GFP_KERNEL|__GFP_REPEAT allocation flags.
- Modified pte_alloc_one() to call page_table_alloc() with allocation flag
GFP_KERNEL | __GFP_REPEAT
- The changes for both architectures help in fixing Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/s390/include/asm/pgalloc.h | 37 ++++++++++++++++++++++++++++---------
arch/s390/mm/pgtable.c | 13 +++++++------
arch/s390/mm/vmem.c | 2 +-
3 files changed, 36 insertions(+), 16 deletions(-)
diff --git a/arch/s390/include/asm/pgalloc.h b/arch/s390/include/asm/pgalloc.h
index 082eb4e..8068669 100644
--- a/arch/s390/include/asm/pgalloc.h
+++ b/arch/s390/include/asm/pgalloc.h
@@ -19,11 +19,11 @@
#define check_pgt_cache() do {} while (0)
-unsigned long *crst_table_alloc(struct mm_struct *, int);
+unsigned long *crst_table_alloc(struct mm_struct *, int, gfp_t);
void crst_table_free(struct mm_struct *, unsigned long *);
void crst_table_free_rcu(struct mm_struct *, unsigned long *);
-unsigned long *page_table_alloc(struct mm_struct *);
+unsigned long *page_table_alloc(struct mm_struct *, gfp_t);
void page_table_free(struct mm_struct *, unsigned long *);
void page_table_free_rcu(struct mm_struct *, unsigned long *);
void disable_noexec(struct mm_struct *, struct task_struct *);
@@ -62,9 +62,11 @@ static inline unsigned long pgd_entry_type(struct mm_struct *mm)
return _SEGMENT_ENTRY_EMPTY;
}
+#define __pud_alloc_one(mm,address,mask) ({ BUG(); ((pud_t *)2); })
#define pud_alloc_one(mm,address) ({ BUG(); ((pud_t *)2); })
#define pud_free(mm, x) do { } while (0)
+#define __pmd_alloc_one(mm,address,mask) ({ BUG(); ((pmd_t *)2); })
#define pmd_alloc_one(mm,address) ({ BUG(); ((pmd_t *)2); })
#define pmd_free(mm, x) do { } while (0)
@@ -88,22 +90,34 @@ static inline unsigned long pgd_entry_type(struct mm_struct *mm)
int crst_table_upgrade(struct mm_struct *, unsigned long limit);
void crst_table_downgrade(struct mm_struct *, unsigned long limit);
-static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long address)
+static inline pud_t *__pud_alloc_one(struct mm_struct *mm,
+ unsigned long address, gfp_t gfp_mask)
{
- unsigned long *table = crst_table_alloc(mm, mm->context.noexec);
+ unsigned long *table = crst_table_alloc(mm, mm->context.noexec, gfp_mask);
if (table)
crst_table_init(table, _REGION3_ENTRY_EMPTY);
return (pud_t *) table;
}
+
+static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long address)
+{
+ return __pud_alloc_one(mm, address, GFP_KERNEL);
+}
#define pud_free(mm, pud) crst_table_free(mm, (unsigned long *) pud)
-static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long vmaddr)
+static inline pmd_t *__pmd_alloc_one(struct mm_struct *mm,
+ unsigned long vmaddr, gfp_t gfp_mask)
{
- unsigned long *table = crst_table_alloc(mm, mm->context.noexec);
+ unsigned long *table = crst_table_alloc(mm, mm->context.noexec, gfp_mask);
if (table)
crst_table_init(table, _SEGMENT_ENTRY_EMPTY);
return (pmd_t *) table;
}
+
+static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long vmaddr)
+{
+ return __pmd_alloc_one(mm, vmaddr, GFP_KERNEL);
+}
#define pmd_free(mm, pmd) crst_table_free(mm, (unsigned long *) pmd)
static inline void pgd_populate_kernel(struct mm_struct *mm,
@@ -146,7 +160,7 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm)
INIT_LIST_HEAD(&mm->context.crst_list);
INIT_LIST_HEAD(&mm->context.pgtable_list);
return (pgd_t *)
- crst_table_alloc(mm, user_mode == SECONDARY_SPACE_MODE);
+ crst_table_alloc(mm, user_mode == SECONDARY_SPACE_MODE, GFP_KERNEL);
}
#define pgd_free(mm, pgd) crst_table_free(mm, (unsigned long *) pgd)
@@ -172,8 +186,13 @@ static inline void pmd_populate(struct mm_struct *mm,
/*
* page table entry allocation/free routines.
*/
-#define pte_alloc_one_kernel(mm, vmaddr) ((pte_t *) page_table_alloc(mm))
-#define pte_alloc_one(mm, vmaddr) ((pte_t *) page_table_alloc(mm))
+#define __pte_alloc_one_kernel(mm, vmaddr, mask) \
+ ((pte_t *) page_table_alloc((mm), (mask)))
+#define pte_alloc_one_kernel(mm, vmaddr) \
+ ((pte_t *) __pte_alloc_one_kernel((mm), (vmaddr), GFP_KERNEL|__GFP_REPEAT)
+
+#define pte_alloc_one(mm, vmaddr) \
+ ((pte_t *) page_table_alloc((mm), GFP_KERNEL | __GFP_REPEAT))
#define pte_free_kernel(mm, pte) page_table_free(mm, (unsigned long *) pte)
#define pte_free(mm, pte) page_table_free(mm, (unsigned long *) pte)
diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c
index e1850c2..9bf1632 100644
--- a/arch/s390/mm/pgtable.c
+++ b/arch/s390/mm/pgtable.c
@@ -125,15 +125,16 @@ static int __init parse_vmalloc(char *arg)
}
early_param("vmalloc", parse_vmalloc);
-unsigned long *crst_table_alloc(struct mm_struct *mm, int noexec)
+unsigned long *crst_table_alloc(struct mm_struct *mm, int noexec,
+ gfp_t gfp_mask)
{
- struct page *page = alloc_pages(GFP_KERNEL, ALLOC_ORDER);
+ struct page *page = alloc_pages(gfp_mask, ALLOC_ORDER);
if (!page)
return NULL;
page->index = 0;
if (noexec) {
- struct page *shadow = alloc_pages(GFP_KERNEL, ALLOC_ORDER);
+ struct page *shadow = alloc_pages(gfp_mask, ALLOC_ORDER);
if (!shadow) {
__free_pages(page, ALLOC_ORDER);
return NULL;
@@ -197,7 +198,7 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long limit)
BUG_ON(limit > (1UL << 53));
repeat:
- table = crst_table_alloc(mm, mm->context.noexec);
+ table = crst_table_alloc(mm, mm->context.noexec, GFP_KERNEL);
if (!table)
return -ENOMEM;
spin_lock_bh(&mm->page_table_lock);
@@ -267,7 +268,7 @@ void crst_table_downgrade(struct mm_struct *mm, unsigned long limit)
/*
* page table entry allocation/free routines.
*/
-unsigned long *page_table_alloc(struct mm_struct *mm)
+unsigned long *page_table_alloc(struct mm_struct *mm, gfp_t gfp_mask)
{
struct page *page;
unsigned long *table;
@@ -284,7 +285,7 @@ unsigned long *page_table_alloc(struct mm_struct *mm)
}
if (!page) {
spin_unlock_bh(&mm->context.list_lock);
- page = alloc_page(GFP_KERNEL|__GFP_REPEAT);
+ page = alloc_page(gfp_mask);
if (!page)
return NULL;
pgtable_page_ctor(page);
diff --git a/arch/s390/mm/vmem.c b/arch/s390/mm/vmem.c
index 34c43f2..f7e8716 100644
--- a/arch/s390/mm/vmem.c
+++ b/arch/s390/mm/vmem.c
@@ -66,7 +66,7 @@ static pte_t __ref *vmem_pte_alloc(void)
pte_t *pte;
if (slab_is_available())
- pte = (pte_t *) page_table_alloc(&init_mm);
+ pte = (pte_t *) page_table_alloc(&init_mm, GFP_KERNEL | __GFP_REPEAT);
else
pte = alloc_bootmem(PTRS_PER_PTE * sizeof(pte_t));
if (!pte)
--
1.7.0.4
>
> --
> blue skies,
> Martin.
>
> "Reality continues to ruin my life." - Calvin.
>
On Fri, Mar 18, 2011 at 03:17:42PM -0700, David Rientjes wrote:
> On Fri, 18 Mar 2011, Prasad Joshi wrote:
>
>
> There should be no break between the function type and its name.
>
>
> PGALLOC_GFP represents the gfp flags intended for the allocators in this
> file, there's no need to replace its use with GFP_KERNEL | __GFP_REPEAT,
> we don't care that __pte_alloc_one_kernel() ORs this with __GFP_NOTRACK
> and __GFP_ZERO.
>
> > }
Thanks a lot David for your reply. Sending a new patch with the changes suggested by you.
- Added __pud_alloc_one() function which is similar to pud_alloc_one(), but
accepts an additional argument of type gfp_t. The function pud_alloc_one()
calls __pud_alloc_one().
- similar changes for pmd allocations. The newly added function name is__pmd_alloc_one.
- Similar changes for pte allocation. Added function __pte_alloc_one_kernel.
- changes help in fixing the Bug 30702
Signed-off-by: Prasad Joshi <[email protected]>
Signed-off-by: Anand Mitra <[email protected]>
---
arch/x86/include/asm/pgalloc.h | 17 +++++++++++++++--
arch/x86/mm/pgtable.c | 8 +++++++-
2 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/pgalloc.h b/arch/x86/include/asm/pgalloc.h
index b4389a4..7278096 100644
--- a/arch/x86/include/asm/pgalloc.h
+++ b/arch/x86/include/asm/pgalloc.h
@@ -34,6 +34,7 @@ extern pgd_t *pgd_alloc(struct mm_struct *);
extern void pgd_free(struct mm_struct *mm, pgd_t *pgd);
extern pte_t *pte_alloc_one_kernel(struct mm_struct *, unsigned long);
+extern pte_t *__pte_alloc_one_kernel(struct mm_struct *, unsigned long, gfp_t);
extern pgtable_t pte_alloc_one(struct mm_struct *, unsigned long);
/* Should really implement gc for free page table pages. This could be
@@ -78,9 +79,15 @@ static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd,
#define pmd_pgtable(pmd) pmd_page(pmd)
#if PAGETABLE_LEVELS > 2
+static inline pmd_t *__pmd_alloc_one(struct mm_struct *mm, unsigned long addr,
+ gfp_t gfp_mask)
+{
+ return (pmd_t *)get_zeroed_page(gfp_mask);
+}
+
static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return (pmd_t *)get_zeroed_page(GFP_KERNEL|__GFP_REPEAT);
+ return __pmd_alloc_one(mm, addr, GFP_KERNEL | __GFP_REPEAT);
}
static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
@@ -114,9 +121,15 @@ static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgd, pud_t *pud)
set_pgd(pgd, __pgd(_PAGE_TABLE | __pa(pud)));
}
+static inline pud_t *__pud_alloc_one(struct mm_struct *mm, unsigned long addr,
+ gfp_t gfp_mask)
+{
+ return (pud_t *)get_zeroed_page(gfp_mask);
+}
+
static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
{
- return (pud_t *)get_zeroed_page(GFP_KERNEL|__GFP_REPEAT);
+ return __pud_alloc_one(mm, addr, GFP_KERNEL | __GFP_REPEAT);
}
static inline void pud_free(struct mm_struct *mm, pud_t *pud)
diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
index 8573b83..f1b0de7 100644
--- a/arch/x86/mm/pgtable.c
+++ b/arch/x86/mm/pgtable.c
@@ -15,9 +15,15 @@
gfp_t __userpte_alloc_gfp = PGALLOC_GFP | PGALLOC_USER_GFP;
+pte_t * __pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address,
+ gfp_t gfp_mask)
+{
+ return (pte_t *)__get_free_page(gfp_mask);
+}
+
pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address)
{
- return (pte_t *)__get_free_page(PGALLOC_GFP);
+ return __pte_alloc_one_kernel(mm, address, PGALLOC_GFP);
}
pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long address)
--
1.7.0.4
On Mon, 21 Mar 2011, Prasad Joshi wrote:
> Thanks a lot David for your reply. Sending a new patch with the changes suggested by you.
>
> - Added __pud_alloc_one() function which is similar to pud_alloc_one(), but
> accepts an additional argument of type gfp_t. The function pud_alloc_one()
> calls __pud_alloc_one().
>
> - similar changes for pmd allocations. The newly added function name is__pmd_alloc_one.
>
> - Similar changes for pte allocation. Added function __pte_alloc_one_kernel.
>
> - changes help in fixing the Bug 30702
>
> Signed-off-by: Prasad Joshi <[email protected]>
> Signed-off-by: Anand Mitra <[email protected]>
Acked-by: David Rientjes <[email protected]>
On Mon, 21 Mar 2011 19:02:39 +0000
Prasad Joshi <[email protected]> wrote:
> On Mon, Mar 21, 2011 at 09:34:12AM +0100, Martin Schwidefsky wrote:
> > On Fri, 18 Mar 2011 19:56:44 +0000
> > Prasad Joshi <[email protected]> wrote:
> >
> > > - Added function __crst_table_alloc() which is similar to crst_table_alloc()
> > > but accepts an extra argument gfp_t. The function uses given allocation
> > > flag to allocate pages.
> > >
> > > - Added a function __page_table_alloc() to allocate page table entries. This
> > > function is allows caller to specify the page allocation flag. The
> > > allocation flag is then passed to alloc_page(). The rest of the function is
> > > copy of the original page_table_alloc().
> >
> > The approach of this patch series seems straightforward, the only nitpick I
> > have is the fact that two new functions __crst_table_alloc/__page_table_alloc
> > are introduced. There aren't many call sites for the two original functions,
> > namely 4 for crst_table_alloc and 3 for page_table_alloc. Why not add the
> > gfp flag GFP_KERNEL to these call sites? Then the two additional functions
> > would not be needed.
>
> Thanks a lot Martin for your reply. Here is a new patch, which the changes
> you suggested.
Looks good now. For the s390 architecture part:
Acked-by: Martin Schwidefsky <[email protected]>
--
blue skies,
Martin.
"Reality continues to ruin my life." - Calvin.
On Fri, Mar 18, 2011 at 07:53:07PM +0000, Prasad Joshi wrote:
>
> - Added __pte_alloc_one_kernel() function to allow caller to allocation flag.
> Like pte_alloc_one_kernel() this newly added function calls
> quicklist_alloc() and passes the allocation flag down the call hierarchy.
>
> - The function pte_alloc_one_kernel() is modified to call
> __pte_alloc_one_kernel() using the correct allocation flag.
>
> - Similar changes for pmd allocations. Added function __pmd_alloc_one().
>
> - The changes help in fixing the Bug 30702.
>
> Signed-off-by: Prasad Joshi <[email protected]>
> Signed-off-by: Anand Mitra <[email protected]>
Acked-by: Paul Mundt <[email protected]>