2012-05-29 17:30:13

by Luck, Tony

Subject: [PATCH] sched: Don't try allocating memory from offline nodes

From: Peter Zijlstra <[email protected]>

Allocators don't appreciate it when you try and allocate memory from
offline nodes.

Reported-by: Tony Luck <[email protected]>
Signed-off-by: Peter Zijlstra <[email protected]>
---

This patch has been sitting around since Friday.

kernel/sched/core.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

Index: linux-2.6/kernel/sched/core.c
===================================================================
--- linux-2.6.orig/kernel/sched/core.c
+++ linux-2.6/kernel/sched/core.c
@@ -6449,7 +6449,7 @@ static void sched_init_numa(void)
return;

for (j = 0; j < nr_node_ids; j++) {
- struct cpumask *mask = kzalloc_node(cpumask_size(), GFP_KERNEL, j);
+ struct cpumask *mask = kzalloc(cpumask_size(), GFP_KERNEL);
if (!mask)
return;

2012-05-30 03:21:18

by David Rientjes

Subject: Re: [PATCH] sched: Don't try allocating memory from offline nodes

On Tue, 29 May 2012, Luck, Tony wrote:

> Index: linux-2.6/kernel/sched/core.c
> ===================================================================
> --- linux-2.6.orig/kernel/sched/core.c
> +++ linux-2.6/kernel/sched/core.c
> @@ -6449,7 +6449,7 @@ static void sched_init_numa(void)
> return;
>
> for (j = 0; j < nr_node_ids; j++) {
> - struct cpumask *mask = kzalloc_node(cpumask_size(), GFP_KERNEL, j);
> + struct cpumask *mask = kzalloc(cpumask_size(), GFP_KERNEL);
> if (!mask)
> return;
>

It's definitely better if we can allocate on the node, though, so perhaps
do the same thing that I did in
http://marc.info/?l=linux-kernel&m=133778739503111 by doing
kzalloc_node(..., node_online(j) ? j : NUMA_NO_NODE)?

2012-05-30 09:44:26

by Peter Zijlstra

Subject: Re: [PATCH] sched: Don't try allocating memory from offline nodes

On Tue, 2012-05-29 at 20:21 -0700, David Rientjes wrote:
> On Tue, 29 May 2012, Luck, Tony wrote:
>
> > Index: linux-2.6/kernel/sched/core.c
> > ===================================================================
> > --- linux-2.6.orig/kernel/sched/core.c
> > +++ linux-2.6/kernel/sched/core.c
> > @@ -6449,7 +6449,7 @@ static void sched_init_numa(void)
> > return;
> >
> > for (j = 0; j < nr_node_ids; j++) {
> > - struct cpumask *mask = kzalloc_node(cpumask_size(), GFP_KERNEL, j);
> > + struct cpumask *mask = kzalloc(cpumask_size(), GFP_KERNEL);
> > if (!mask)
> > return;
> >
>
> It's definitely better if we can allocate on the node, though, so perhaps
> do the same thing that I did in
> http://marc.info/?l=linux-kernel&m=133778739503111 by doing
> kzalloc_node(..., node_online(j) ? j : NUMA_NO_NODE)?

This data isn't used overly much, only when rebuilding the sched
domains, so its not performance critical. I only used per-node
allocations because it seemed the right thing to do. If it doesn't work,
I wouldn't bother with making it more complex.

2012-05-30 21:39:31

by David Rientjes

Subject: Re: [PATCH] sched: Don't try allocating memory from offline nodes

On Wed, 30 May 2012, Peter Zijlstra wrote:

> On Tue, 2012-05-29 at 20:21 -0700, David Rientjes wrote:
> > On Tue, 29 May 2012, Luck, Tony wrote:
> >
> > > Index: linux-2.6/kernel/sched/core.c
> > > ===================================================================
> > > --- linux-2.6.orig/kernel/sched/core.c
> > > +++ linux-2.6/kernel/sched/core.c
> > > @@ -6449,7 +6449,7 @@ static void sched_init_numa(void)
> > > return;
> > >
> > > for (j = 0; j < nr_node_ids; j++) {
> > > - struct cpumask *mask = kzalloc_node(cpumask_size(), GFP_KERNEL, j);
> > > + struct cpumask *mask = kzalloc(cpumask_size(), GFP_KERNEL);
> > > if (!mask)
> > > return;
> > >
> >
> > It's definitely better if we can allocate on the node, though, so perhaps
> > do the same thing that I did in
> > http://marc.info/?l=linux-kernel&m=133778739503111 by doing
> > kzalloc_node(..., node_online(j) ? j : NUMA_NO_NODE)?
>
> This data isn't used overly much, only when rebuilding the sched
> domains, so its not performance critical. I only used per-node
> allocations because it seemed the right thing to do. If it doesn't work,
> I wouldn't bother with making it more complex.
>

Ok, if you don't think these cpumasks need locality for performance, then

Acked-by: David Rientjes <[email protected]>