Remove the ctor for the pgd cache. There's no point in having the
cache machinery do this via an indirect call when all pgd are freed in
the one place anyway.
Signed-off-by: Jeremy Fitzhardinge <[email protected]>
===================================================================
--- a/arch/i386/mm/init.c
+++ b/arch/i386/mm/init.c
@@ -739,11 +739,9 @@ void __init pgtable_cache_init(void)
panic("pgtable_cache_init(): cannot create pmd cache");
}
pgd_cache = kmem_cache_create("pgd",
- PTRS_PER_PGD*sizeof(pgd_t),
- PTRS_PER_PGD*sizeof(pgd_t),
- 0,
- pgd_ctor,
- PTRS_PER_PMD == 1 ? pgd_dtor : NULL);
+ PTRS_PER_PGD*sizeof(pgd_t),
+ PTRS_PER_PGD*sizeof(pgd_t),
+ 0, NULL, NULL);
if (!pgd_cache)
panic("pgtable_cache_init(): Cannot create pgd cache");
}
===================================================================
--- a/arch/i386/mm/pgtable.c
+++ b/arch/i386/mm/pgtable.c
@@ -236,7 +236,7 @@ static inline void pgd_list_del(pgd_t *p
set_page_private(next, (unsigned long)pprev);
}
-void pgd_ctor(void *pgd, struct kmem_cache *cache, unsigned long unused)
+static void pgd_ctor(pgd_t *pgd)
{
unsigned long flags;
@@ -245,7 +245,7 @@ void pgd_ctor(void *pgd, struct kmem_cac
spin_lock_irqsave(&pgd_lock, flags);
}
- clone_pgd_range((pgd_t *)pgd + USER_PTRS_PER_PGD,
+ clone_pgd_range(pgd + USER_PTRS_PER_PGD,
swapper_pg_dir + USER_PTRS_PER_PGD,
KERNEL_PGD_PTRS);
@@ -261,10 +261,12 @@ void pgd_ctor(void *pgd, struct kmem_cac
spin_unlock_irqrestore(&pgd_lock, flags);
}
-/* never called when PTRS_PER_PMD > 1 */
-void pgd_dtor(void *pgd, struct kmem_cache *cache, unsigned long unused)
+static void pgd_dtor(pgd_t *pgd)
{
unsigned long flags; /* can be called from interrupt context */
+
+ if (PTRS_PER_PMD == 1)
+ return;
paravirt_release_pd(__pa(pgd) >> PAGE_SHIFT);
spin_lock_irqsave(&pgd_lock, flags);
@@ -276,6 +278,9 @@ pgd_t *pgd_alloc(struct mm_struct *mm)
{
int i;
pgd_t *pgd = kmem_cache_alloc(pgd_cache, GFP_KERNEL);
+
+ if (pgd)
+ pgd_ctor(pgd);
if (PTRS_PER_PMD == 1 || !pgd)
return pgd;
@@ -296,6 +301,7 @@ out_oom:
paravirt_release_pd(__pa(pmd) >> PAGE_SHIFT);
kmem_cache_free(pmd_cache, pmd);
}
+ pgd_dtor(pgd);
kmem_cache_free(pgd_cache, pgd);
return NULL;
}
@@ -313,5 +319,6 @@ void pgd_free(pgd_t *pgd)
kmem_cache_free(pmd_cache, pmd);
}
/* in the non-PAE case, free_pgtables() clears user pgd entries */
+ pgd_dtor(pgd);
kmem_cache_free(pgd_cache, pgd);
}
===================================================================
--- a/include/asm-i386/pgtable.h
+++ b/include/asm-i386/pgtable.h
@@ -41,8 +41,6 @@ extern struct page *pgd_list;
extern struct page *pgd_list;
void pmd_ctor(void *, struct kmem_cache *, unsigned long);
-void pgd_ctor(void *, struct kmem_cache *, unsigned long);
-void pgd_dtor(void *, struct kmem_cache *, unsigned long);
void pgtable_cache_init(void);
void paging_init(void);
--
Jeremy Fitzhardinge wrote:
> Remove the ctor for the pgd cache. There's no point in having the
> cache machinery do this via an indirect call when all pgd are freed in
> the one place anyway.
>
> Signed-off-by: Jeremy Fitzhardinge <[email protected]>
>
Acked-by: Zachary Amsden <[email protected]>
This does introduce a bug for us, but that is trivially fixed, and I
would like to inspect final merged code to make sure bugfix is proper.
Zach
On 2/16/07, Jeremy Fitzhardinge <[email protected]> wrote:
> Remove the ctor for the pgd cache. There's no point in having the
> cache machinery do this via an indirect call when all pgd are freed in
> the one place anyway.
The reason we have slab constructors and destructors is to _avoid_
reinitializing every time we allocate an object. AFAICT your changing
the code now to do _more_ work than before, so is there some other
reason why you want to do this than avoiding an indirect call?
Pekka Enberg wrote:
> On 2/16/07, Jeremy Fitzhardinge <[email protected]> wrote:
>
>> Remove the ctor for the pgd cache. There's no point in having the
>> cache machinery do this via an indirect call when all pgd are freed in
>> the one place anyway.
>
>
> The reason we have slab constructors and destructors is to _avoid_
> reinitializing every time we allocate an object. AFAICT your changing
> the code now to do _more_ work than before, so is there some other
> reason why you want to do this than avoiding an indirect call?
Sometimes it is better for the caches to initialise an object
immediately, but in this case I think it is better to use the
slab ctor because it is very unlikely to use many cachelines
immediately anyway.
It would be nice to put the "why" in the changelogs, rather than
"what". Not everyone wants to go through the whole patchset to
decipher why Xen possibly needs something.
--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
Nick Piggin wrote:
> Pekka Enberg wrote:
>> On 2/16/07, Jeremy Fitzhardinge <[email protected]> wrote:
>>
>>> Remove the ctor for the pgd cache. There's no point in having the
>>> cache machinery do this via an indirect call when all pgd are freed in
>>> the one place anyway.
>>
>>
>> The reason we have slab constructors and destructors is to _avoid_
>> reinitializing every time we allocate an object. AFAICT your changing
>> the code now to do _more_ work than before, so is there some other
>> reason why you want to do this than avoiding an indirect call?
>
> Sometimes it is better for the caches to initialise an object
> immediately, but in this case I think it is better to use the
> slab ctor because it is very unlikely to use many cachelines
> immediately anyway.
>
> It would be nice to put the "why" in the changelogs, rather than
> "what". Not everyone wants to go through the whole patchset to
> decipher why Xen possibly needs something.
Hm, I think I was mislead by looking at kmem_cache_alloc in slob.c
rather than the one that's actually used in slab.c. There's no
particular Xen reason for this patch, so I can drop it.
J
On Thu, 15 Feb 2007, Jeremy Fitzhardinge wrote:
> Remove the ctor for the pgd cache. There's no point in having the
> cache machinery do this via an indirect call when all pgd are freed in
> the one place anyway.
Great. We finally get rid of this evil .....
On second throughts:
Are you sure that you understand what you are modifying? The ctor is not
called when you allocate an object but when the slab allocates it. Freeing
an object does not make the slab free it. It will hold it for the next
kmalloc. So you may just have reduced performance somewhat by addiung
additional ctor processing and possibly disturbed the preservation of
cache hot pgds.
Christoph Lameter wrote:
> On Thu, 15 Feb 2007, Jeremy Fitzhardinge wrote:
>
>
>> Remove the ctor for the pgd cache. There's no point in having the
>> cache machinery do this via an indirect call when all pgd are freed in
>> the one place anyway.
>>
>
> Great. We finally get rid of this evil .....
>
> On second throughts:
>
> Are you sure that you understand what you are modifying? The ctor is not
> called when you allocate an object but when the slab allocates it. Freeing
> an object does not make the slab free it. It will hold it for the next
> kmalloc. So you may just have reduced performance somewhat by addiung
> additional ctor processing and possibly disturbed the preservation of
> cache hot pgds.
It's true I wrote this patch as a result of accidentally reading slob.c
rather than slab.c, which was mostly as a result of hopeful thinking (I
knew that there has been discussion about removing all the slab cache
stuff, and I assumed that it had already happened). As a result I'm
withdrawing this patch for now, but with any luck it will be useful at
some point.
J