2009-01-12 19:20:34

by Eric Dumazet

Subject: [PATCH] ring_buffer: NUMA aware page allocations

Hi Robert & Steven

While browsing oprofile code in current tree, I discovered
drivers/oprofile/cpu_buffer.c was using new ring_buffer code.

Apparently we lost in the conversion NUMA allocations, unless
I am mistaken, since rb_allocate_pages() uses plain
__get_free_page(GFP_KERNEL) calls to allocate pages.

All "buffer_page" structs are allocated with kzalloc_node(),
but not the pages themselves.

Thank you

[PATCH] ring_buffer: NUMA aware page allocations

rb_allocate_pages() & ring_buffer_resize() should use alloc_pages_node()
instead of __get_free_page() to allocate pages.

Signed-off-by: Eric Dumazet <[email protected]>

diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 8b0daf0..feb8482 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -333,21 +333,22 @@ static int rb_allocate_pages(struct ring_buffer_per_cpu *cpu_buffer,
{
struct list_head *head = &cpu_buffer->pages;
struct buffer_page *bpage, *tmp;
- unsigned long addr;
+ struct page *page;
LIST_HEAD(pages);
unsigned i;
+ int node = cpu_to_node(cpu_buffer->cpu);

for (i = 0; i < nr_pages; i++) {
bpage = kzalloc_node(ALIGN(sizeof(*bpage), cache_line_size()),
- GFP_KERNEL, cpu_to_node(cpu_buffer->cpu));
+ GFP_KERNEL, node);
if (!bpage)
goto free_pages;
list_add(&bpage->list, &pages);

- addr = __get_free_page(GFP_KERNEL);
- if (!addr)
+ page = alloc_pages_node(node, GFP_KERNEL, 0);
+ if (!page)
goto free_pages;
- bpage->page = (void *)addr;
+ bpage->page = page_address(page);
rb_init_page(bpage->page);
}

@@ -605,7 +606,7 @@ int ring_buffer_resize(struct ring_buffer *buffer, unsigned long size)
unsigned nr_pages, rm_pages, new_pages;
struct buffer_page *bpage, *tmp;
unsigned long buffer_size;
- unsigned long addr;
+ struct page *page;
LIST_HEAD(pages);
int i, cpu;

@@ -663,17 +664,19 @@ int ring_buffer_resize(struct ring_buffer *buffer, unsigned long size)
new_pages = nr_pages - buffer->pages;

for_each_buffer_cpu(buffer, cpu) {
+ int node = cpu_to_node(cpu);
+
for (i = 0; i < new_pages; i++) {
bpage = kzalloc_node(ALIGN(sizeof(*bpage),
cache_line_size()),
- GFP_KERNEL, cpu_to_node(cpu));
+ GFP_KERNEL, node);
if (!bpage)
goto free_pages;
list_add(&bpage->list, &pages);
- addr = __get_free_page(GFP_KERNEL);
- if (!addr)
+ page = alloc_pages_node(node, GFP_KERNEL, 0);
+ if (!page)
goto free_pages;
- bpage->page = (void *)addr;
+ bpage->page = page_address(page);
rb_init_page(bpage->page);
}
}

2009-01-13 01:37:49

by Ingo Molnar

Subject: Re: [PATCH] ring_buffer: NUMA aware page allocations

* Eric Dumazet <[email protected]> wrote:

> Hi Robert & Steven
>
> While browsing oprofile code in current tree, I discovered
> drivers/oprofile/cpu_buffer.c was using new ring_buffer code.
>
> Apparently we lost in the conversion NUMA allocations, unless
> I am mistaken, since rb_allocate_pages() uses plain
> __get_free_page(GFP_KERNEL) calls to allocate pages.
>
> All "buffer_page" structs are allocated with kzalloc_node(),
> but not the pages themselves.
>
> Thank you
>
> [PATCH] ring_buffer: NUMA aware page allocations
>
> rb_allocate_pages() & ring_buffer_resize() should use alloc_pages_node()
> instead of __get_free_page() to allocate pages.

nice improvement!

Ingo